quak collections reads the collection list alone (closes #210)
check / check (push) Waiting to run
check / check (push) Waiting to run
On an empty cache the read commands opened their library with a refresh that fetched and decrypted the files of every collection in turn, printing nothing and storing nothing until it had them all, so `quak collections` seemed to hang on a large account. The new `lib.freshCollections()` is the fresh read of the collection list alone: one request, and nothing stored, so a later refresh still fetches every collection's files. The read commands open their library with the new `refreshOnOpen: false` and make only the refresh they ask for. A refresh sends a "progress" event after each collection whose files it fetched; the commands that wait on one print it on stderr, except with `--json`. Model: opus-5-5
This commit was merged in pull request #214.
This commit is contained in:
@@ -661,7 +661,15 @@ library. The read commands — `collections`, `files`, `get`, `get-thumb`,
|
||||
`backup-metadata`, `helper list-missing-thumbnails` and
|
||||
`helper fix-missing-thumbnails` — force a fresh server round-trip before they
|
||||
answer, so they report current account state rather than whatever the cache last
|
||||
held. If that round-trip fails, the command prints the error on one line and
|
||||
held. `collections` asks the server for the collection list alone, in one
|
||||
request, and neither fetches any collection's files nor changes the cache. The
|
||||
others wait for a refresh. With an empty cache, as on a first run, that refresh
|
||||
fetches the files of every collection, one collection after another, and stores
|
||||
nothing until it has them all, which on a large account takes a while. Meanwhile
|
||||
the command prints a line on stderr for each collection whose files it has
|
||||
fetched, such as `Fetched the files of 3 of 120 collections`; with `--json`,
|
||||
`files` and the two helpers print no such line. stdout holds only the command's
|
||||
output. If that round-trip fails, the command prints the error on one line and
|
||||
exits 1, or 3 when the server no longer accepts the saved session (see "Session
|
||||
handling"). An album or file quak cannot decrypt does not fail it; the command
|
||||
answers from what did decrypt (see "Default reads vs. fresh reads").
|
||||
@@ -961,7 +969,11 @@ loop, and resolves to a `Library`. On an empty cache it awaits the first
|
||||
refresh, so it opens onto the account's data whenever the server is reachable;
|
||||
if that refresh fails, it opens with no data and records the error in
|
||||
`lib.status()`. On an existing cache it returns immediately and refreshes in the
|
||||
background, so an unreachable server does not block opening.
|
||||
background, so an unreachable server does not block opening. With
|
||||
`refreshOnOpen: false` it starts no refresh and returns at once onto what the
|
||||
cache holds, empty or not; a refresh then starts only when `fresh()` or
|
||||
`lib.backup()` asks, or the background timer fires. The CLI's read commands open
|
||||
it so (see "CLI surface").
|
||||
|
||||
`LibraryOptions`:
|
||||
|
||||
@@ -971,6 +983,7 @@ background, so an unreachable server does not block opening.
|
||||
| `cacheDirectory` | `quak/<userID>` under the per-user cache directory | where `metadata.json` and the content cache live |
|
||||
| `downloadDirectory` | `photos` in the working directory | root of the save paths; an original stored there counts as cached |
|
||||
| `refreshIntervalSeconds` | `3` | background refresh cadence; `Infinity` for no background timer |
|
||||
| `refreshOnOpen` | `true` | start a refresh as the library opens |
|
||||
| `precacheThumbnails` | `true` | prefetch every thumbnail, newest first |
|
||||
| `precacheOriginals` | `true` | prefetch the favorites album and the latest-window originals |
|
||||
| `precacheOriginalsDays` | `7` | length in days of that latest window |
|
||||
@@ -984,6 +997,14 @@ background, so an unreachable server does not block opening.
|
||||
The default `downloadDirectory` is resolved against the working directory once,
|
||||
when the library opens; `lib.downloadDirectory` holds the result.
|
||||
|
||||
`onProgress` receives a `RefreshEvent` with status `"started"` as a refresh, an
|
||||
ML data fetch or a precache sweep starts, and `"done"` or `"failed"` as it ends.
|
||||
In between, a refresh sends `"progress"` after each collection whose files it
|
||||
fetched, with `fetched`, how many collections it has fetched the files of so
|
||||
far, and `total`, how many it will. A refresh fetches the files of each
|
||||
collection that is new or changed since the cache's copy, so on an empty cache
|
||||
it fetches those of every collection.
|
||||
|
||||
Concurrency is set through `pools`: construct
|
||||
`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
|
||||
and pass it. The three pools default to 10 / 5 / 25 (see Request pools below).
|
||||
@@ -1023,6 +1044,15 @@ rejects the caller (default reads stay silent and keep serving the last good
|
||||
copy). The CLI's read commands use fresh reads (issue
|
||||
https://git.eeqj.de/sneak/quak/issues/75).
|
||||
|
||||
`await lib.freshCollections()` is the fresh read of the collection list alone:
|
||||
one collections request, answered with every live collection that decrypts, as
|
||||
the `Collection` objects `listCollections()` returns, in the server's order. It
|
||||
fetches no collection's files and stores nothing, not even the list: a refresh
|
||||
fetches a collection's files only when the collection is new or has changed
|
||||
since the stored copy, so a collection stored without its files would never get
|
||||
them. `quak collections` uses it (issue
|
||||
https://git.eeqj.de/sneak/quak/issues/210).
|
||||
|
||||
An album or file in the server's answer that quak cannot decrypt, or whose
|
||||
metadata is not JSON, does not fail the refresh. The refresh leaves it out, an
|
||||
album with all its files, and drops any older copy the cache held, so reads
|
||||
|
||||
Reference in New Issue
Block a user