quak files COLLECTION fetches that collection's files alone (closes #217)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
`quak files COLLECTION` used to wait for a full refresh, which on an empty cache fetched the files of every collection before it listed one. It now reads the collection list, then calls the new `Library.refreshCollection()`. That brings one collection up to date in the store as a refresh would, and saves it. It runs in the same slot as a refresh, so the two never run at the same time, and `close()` waits for it. It never stores a collection that a refresh has already dropped. It leaves the refresh cursor alone. `open()` therefore awaits the first refresh on a cache whose cursor is still 0. `quak files` with no argument still waits for a refresh. Model: opus-5-5
This commit is contained in:
@@ -685,17 +685,21 @@ library. The read commands — `collections`, `files`, `get`, `get-thumb`,
|
||||
`helper fix-missing-thumbnails` — force a fresh server round-trip before they
|
||||
answer, so they report current account state rather than whatever the cache last
|
||||
held. `collections` asks the server for the collection list alone, in one
|
||||
request, and neither fetches any collection's files nor changes the cache. The
|
||||
others wait for a refresh. With an empty cache, as on a first run, that refresh
|
||||
fetches the files of every collection, one collection after another, and stores
|
||||
nothing until it has them all, which on a large account takes a while. Meanwhile
|
||||
the command logs a line on stderr for each collection whose files it has
|
||||
fetched, such as `Fetched the files of 3 of 120 collections`, with `--json` too;
|
||||
`backup` does the same while the refresh it starts with runs. stdout holds only
|
||||
the command's output. If that round-trip fails, the command prints the error on
|
||||
one line and exits 1, or 3 when the server no longer accepts the saved session
|
||||
(see "Session handling"). An album or file quak cannot decrypt does not fail it;
|
||||
the command answers from what did decrypt (see "Default reads vs. fresh reads").
|
||||
request, and neither fetches any collection's files nor changes the cache.
|
||||
`files` with a collection ID asks for the collection list the same way, then for
|
||||
the files of that collection alone, and stores them in the cache. It asks for
|
||||
none of them when the cache's copy of that collection is current. `files`
|
||||
without a collection ID and the other read commands wait for a refresh. With an
|
||||
empty cache, as on a first run, that refresh fetches the files of every
|
||||
collection, one collection after another, and stores nothing until it has them
|
||||
all, which on a large account takes a while. Meanwhile the command logs a line
|
||||
on stderr for each collection whose files it has fetched, such as
|
||||
`Fetched the files of 3 of 120 collections`, with `--json` too; `backup` does
|
||||
the same while the refresh it starts with runs. stdout holds only the command's
|
||||
output. If that round-trip fails, the command prints the error on one line and
|
||||
exits 1, or 3 when the server no longer accepts the saved session (see "Session
|
||||
handling"). An album or file quak cannot decrypt does not fail it; the command
|
||||
answers from what did decrypt (see "Default reads vs. fresh reads").
|
||||
`--cache-dir` overrides where the cache lives; without it each account gets its
|
||||
own directory under the per-user cache path.
|
||||
|
||||
@@ -1093,12 +1097,13 @@ against a fake of the Ente HTTP API, and `yarn test` verifies them.
|
||||
|
||||
`Library.open(options)` loads the on-disk cache, starts the background refresh
|
||||
loop, and resolves to a `Library`. On an empty cache it awaits the first
|
||||
refresh, so it opens onto the account's data whenever the server is reachable;
|
||||
if that refresh fails, it opens with no data and records the error in
|
||||
`lib.status()`. On an existing cache it returns immediately and refreshes in the
|
||||
background, so an unreachable server does not block opening. With
|
||||
`refreshOnOpen: false` it starts no refresh and returns at once onto what the
|
||||
cache holds, empty or not; a refresh then starts only when `fresh()` or
|
||||
refresh, so it opens onto the account's data whenever the server is reachable.
|
||||
It also awaits it on a cache that holds only what `lib.refreshCollection()`
|
||||
stored. If that refresh fails, it opens onto what the cache holds and records
|
||||
the error in `lib.status()`. On any other cache it returns immediately and
|
||||
refreshes in the background, so an unreachable server does not block opening.
|
||||
With `refreshOnOpen: false` it starts no refresh and returns at once onto what
|
||||
the cache holds, empty or not; a refresh then starts only when `fresh()` or
|
||||
`lib.backup()` asks, or the background timer fires. The CLI's read commands open
|
||||
it so (see "CLI surface").
|
||||
|
||||
@@ -1138,11 +1143,16 @@ and pass it. The three pools default to 10 / 5 / 25 (see Request pools below).
|
||||
|
||||
`lib.status()` returns a `LibraryStatus` (collection/file counts, last
|
||||
refresh/ML times and errors, originals usage and effective limit, precache
|
||||
progress, and `closed`). `lib.close()` stops the background timer; it is
|
||||
idempotent, and an in-flight refresh is left to finish. The promise it returns
|
||||
resolves once that refresh (including its cache write), the ML data fetch and
|
||||
the precache fetches already running have all finished, so the cache directory
|
||||
can then be removed.
|
||||
progress, and `closed`).
|
||||
|
||||
`lib.close()` stops the background timer, and is idempotent. A refresh or
|
||||
`lib.refreshCollection()` in flight is left to finish. The promise `close()`
|
||||
returns resolves once all of these have finished, so the cache directory can
|
||||
then be removed:
|
||||
|
||||
- that refresh or `lib.refreshCollection()`, with its cache write
|
||||
- the ML data fetch
|
||||
- the precache fetches already running
|
||||
|
||||
`lib.status().mlDataErrors` lists, as `{ fileID, error }`, each file whose ML
|
||||
data the last ML data fetch that succeeded got but could not decrypt, gunzip or
|
||||
@@ -1180,6 +1190,20 @@ since the stored copy, so a collection stored without its files would never get
|
||||
them. `quak collections` uses it (issue
|
||||
https://git.eeqj.de/sneak/quak/issues/210).
|
||||
|
||||
`await lib.refreshCollection(collection)` brings one collection up to date in
|
||||
the cache, given its `Collection` from `freshCollections()`. It does for that
|
||||
collection what a refresh does for a changed one, and fetches no other
|
||||
collection's files. It sends no request when the cache's copy has the same
|
||||
`updationTime` or a later one. It also sends none when a refresh has already
|
||||
dropped the collection from the cache, because the server deleted it or it did
|
||||
not decrypt. Otherwise it fetches the collection's files changed since the
|
||||
cache's copy, or all of them when there is none, then stores and saves them. The
|
||||
refresh cursor stays where it was, so the next refresh still fetches the files
|
||||
of every other collection that changed. It never runs at the same time as a
|
||||
refresh. It waits for a running refresh to end, and a `fresh()` or
|
||||
`lib.backup()` called meanwhile waits for it. `quak files` with a collection ID
|
||||
uses it.
|
||||
|
||||
An album or file in the server's answer that quak cannot decrypt, or whose
|
||||
metadata is not JSON, does not fail the refresh. The refresh leaves it out, an
|
||||
album with all its files, and drops any older copy the cache held, so reads
|
||||
|
||||
Reference in New Issue
Block a user