quak files COLLECTION fetches that collection's files alone (closes #217)

`quak files COLLECTION` used to wait for a full refresh, which on an
empty cache fetched the files of every collection before it listed one.
It now reads the collection list, then calls the new
`Library.refreshCollection()`. That brings one collection up to date in
the store as a refresh would, and saves it. It runs in the same slot as
a refresh, so the two never run at the same time, and `close()` waits
for it. It never stores a collection that a refresh has already dropped.
It leaves the refresh cursor alone. `open()` therefore awaits the first
refresh on a cache whose cursor is still 0. `quak files` with no
argument still waits for a refresh.

Model: opus-5-5

README.md

@@ -2,31 +2,21 @@
 
 quak is a WTFPL-licensed TypeScript client library and CLI by
 [@sneak](https://sneak.berlin) for the [Ente](https://ente.io) end-to-end
-encrypted photo hosting service. It logs in, lists collections and their files,
+encrypted photo hosting service. It logs in, enumerates collections and files,
-and downloads and decrypts single files.
+and downloads individual images while decrypting them on the way to disk.
 
-`quak backup` downloads every file in the account into a local directory tree.
+quak also includes a resilient backup command that downloads every file in the
-It downloads and stores each file once, however many collections hold it. It
+account into a deduplicated local directory tree, skipping files that already
-skips files already on disk. It logs a failed download and moves on to the next
+exist on disk and continuing past individual download failures instead of
-file. Beside each file it writes a JSON file that holds:
+crashing. For each file it persists the basic metadata fields quak keeps (title,
-
+file type, creation and modification time, latitude, longitude, content hash),
-- the file's title and file type
+its update time, and the private and public magic metadata in full. It keeps any
-- its creation and modification times
+ML data Ente has for the file, except the CLIP embedding, and writes the file's
-- its latitude and longitude
+keywords in place of the embedding. For an image (for a live photo, its image),
-- its content hash
+it keeps its original's EXIF and XMP, with its dimensions for a JPEG only. For a
-- its update time
+video it keeps none of these three. It runs unattended from cron (see "Running
-- its private and public magic metadata, in full
+the backup from cron"). A helper subcommand can detect and regenerate missing
-- the ML data that Ente has for it, without the CLIP embedding
+thumbnails, encrypting and uploading them back to the server.
-- its keywords in place of that embedding (see "Backup layout")
-- for an image, the EXIF and XMP of its original, and its dimensions if it is a
-  JPEG
-- for a live photo, the same for its image
-
-The backup runs unattended from cron (see "Running the backup from cron").
-
-`quak helper list-missing-thumbnails` finds files whose thumbnail is missing.
-`quak helper fix-missing-thumbnails` makes a new thumbnail for such a file, then
-encrypts it and uploads it to the server.
 
 ## Getting Started
 
@@ -70,11 +60,10 @@ const client = await Client.login({
     password: "your-password",
 });
 
-// Open a cache-backed library. On an empty cache, or one that only
+// Open a cache-backed library. On an empty cache this awaits one server
-// `lib.refreshCollection()` wrote, this awaits one server refresh. Otherwise
+// refresh; on an existing cache it returns immediately and refreshes in the
-// it returns immediately and refreshes in the background. A background timer
+// background. Later refreshes start `refreshIntervalSeconds` (default 3) after
-// then refreshes again `refreshIntervalSeconds` (default 3) after its previous
+// the previous one ends.
-// refresh ends.
 const lib = await Library.open({ client });
 
 // Default reads answer synchronously from the local cache — no network.
@@ -651,17 +640,17 @@ The CLI stores the snapshot at the platform-appropriate data directory via
 `0600`. The key material is stored in cleartext in the JSON; treat this file as
 you would treat the password itself. A missing file is reported as "not logged
 in"; a file that exists but is corrupt is reported as such, naming the bad
-field. The server answers HTTP 401 when it no longer accepts the saved session's
+field. When the refresh a command starts with gets HTTP 401 from the server,
-token. A command that gets that answer prints one line,
+because it no longer accepts the saved session's token, the command prints one
-`quak: the saved session is no longer valid; run "quak login"`, with no stack
+line, `quak: the saved session is no longer valid; run "quak login"`, with no
-trace. All three exit with status 3, which means the user must run `quak login`
+stack trace. All three exit with status 3, which means the user must run
-again. `quak logout` is the exception: with no file it says there is no session
+`quak login` again. `quak logout` is the exception: with no file it says there
-and exits 0, and it handles a corrupt file or a failed server call as described
+is no session and exits 0, and it handles a corrupt file or a failed server call
-below. `quak backup` meets an expired session on the refresh that starts every
+as described below. `quak backup` meets an expired session on the refresh that
-run, before it touches any file but its lock (see "Backup layout"). A session
+starts every run, before it touches any file but its lock (see "Backup layout").
-that stops working partway through a backup instead fails each remaining file
+A session that stops working partway through a backup instead fails each
-into `failures.json`, so that run exits 1 and the next one stops at its refresh
+remaining file into `failures.json`, so that run exits 1 and the next one stops
-with status 3. No command but `quak login` ever prompts.
+at its refresh with status 3. No command but `quak login` ever prompts.
 
 `quak logout` ends the session on the server, so the token in `session.json`
 stops working even in a copy of the file, and then deletes the file. If the
@@ -691,40 +680,26 @@ quak helper fix-missing-thumbnails [--file ids] [--json]  generate + upload miss
 ```
 
 Every command except `login`, `whoami` and `logout` runs on the cache-backed
-library. Each read command makes a fresh server round-trip before it answers. It
+library. The read commands — `collections`, `files`, `get`, `get-thumb`,
-then reports the account's current state, not what the cache last held. The read
+`backup-metadata`, `helper list-missing-thumbnails` and
-commands are:
+`helper fix-missing-thumbnails` — force a fresh server round-trip before they
-
+answer, so they report current account state rather than whatever the cache last
-- `collections`
+held. `collections` asks the server for the collection list alone, in one
-- `files`
+request, and neither fetches any collection's files nor changes the cache.
-- `get`
+`files` with a collection ID asks for the collection list the same way, then for
-- `get-thumb`
+the files of that collection alone, and stores them in the cache. It asks for
-- `backup-metadata`
+none of them when the cache's copy of that collection is current. `files`
-- `helper list-missing-thumbnails`
+without a collection ID and the other read commands wait for a refresh. With an
-- `helper fix-missing-thumbnails`
+empty cache, as on a first run, that refresh fetches the files of every
-
+collection, one collection after another, and stores nothing until it has them
-`collections` asks the server for the collection list alone, in one request. It
+all, which on a large account takes a while. Meanwhile the command logs a line
-fetches no collection's files and leaves the cache as it is. `files` with a
+on stderr for each collection whose files it has fetched, such as
-collection ID asks for the collection list the same way. It then asks for the
+`Fetched the files of 3 of 120 collections`, with `--json` too; `backup` does
-files of that collection alone, and stores them in the cache. It asks for no
+the same while the refresh it starts with runs. stdout holds only the command's
-files when the cache's copy of that collection is current. The other read
+output. If that round-trip fails, the command prints the error on one line and
-commands, and `files` without a collection ID, wait for a refresh. So does
-`files` with the ID of a collection that is new since the last refresh.
-
-The first refresh fetches the files of every collection, one collection after
-another. That includes any collection that `files` with a collection ID stored
-before it. The refresh stores nothing until it has them all, and on a large
-account that takes a while. Meanwhile the command logs a line on stderr for each
-collection whose files it has fetched, such as
-`Fetched the files of 3 of 120 collections`. It logs these lines with `--json`
-too. `backup` logs them during the refresh it starts with. stdout holds only the
-command's output.
-
-If the server round-trip fails, the command prints the error on one line. It
 exits 1, or 3 when the server no longer accepts the saved session (see "Session
-handling"). An album or file that quak cannot decrypt does not fail the command.
+handling"). An album or file quak cannot decrypt does not fail it; the command
-The command answers from what did decrypt (see "Default reads vs. fresh reads").
+answers from what did decrypt (see "Default reads vs. fresh reads").
-
 `--cache-dir` overrides where the cache lives; without it each account gets its
 own directory under the per-user cache path.
 
@@ -779,10 +754,9 @@ its `Failed files:` list. Every other line is logged as it is without `--json`.
 exit status 1.
 
 `files <collectionID>` lists the files in that collection. `files` with no
-argument lists every file in the account once, collection by collection. The
+argument lists every file in the account, collection by collection, each file
-collections come in the order the server first sent them in a refresh. A file in
+once: a file in several collections is listed with the first of them, which is
-several collections is listed with the first of them, which is its
+its `collectionID` in `--json`.
-`collectionID` in `--json`.
 
 An ID given to `files`, `get`, `get-thumb` or
 `helper fix-missing-thumbnails --file` is written in decimal digits, no larger
@@ -1122,17 +1096,16 @@ against a fake of the Ente HTTP API, and `yarn test` verifies them.
 ### Opening a library
 
 `Library.open(options)` loads the on-disk cache, starts the background refresh
-loop, and resolves to a `Library`. On an empty cache, or one that only
+loop, and resolves to a `Library`. On an empty cache it awaits the first
-`lib.refreshCollection()` wrote, it awaits one refresh. It then opens onto the
+refresh, so it opens onto the account's data whenever the server is reachable.
-account's data whenever the server is reachable. If that refresh fails, it opens
+It also awaits it on a cache that holds only what `lib.refreshCollection()`
-onto what the cache holds, and records the error in `lib.status()`. On any other
+stored. If that refresh fails, it opens onto what the cache holds and records
-cache it returns immediately and refreshes in the background. An unreachable
+the error in `lib.status()`. On any other cache it returns immediately and
-server then does not block opening.
+refreshes in the background, so an unreachable server does not block opening.
-
+With `refreshOnOpen: false` it starts no refresh, and returns at once onto what
-With `refreshOnOpen: false`, `open()` starts no refresh. It returns at once onto
+the cache holds, empty or not. A refresh then starts only when `fresh()` or
-what the cache holds, even when the cache is empty. A refresh then starts only
+`lib.backup()` asks, or the background timer fires. The CLI's read commands open
-when `fresh()` or `lib.backup()` asks, or the background timer fires. The CLI's
+it so (see "CLI surface").
-read commands open the library this way (see "CLI surface").
 
 `LibraryOptions`:
 
@@ -1159,36 +1132,28 @@ when the library opens; `lib.downloadDirectory` holds the result.
 `onProgress` receives a `RefreshEvent` with status `"started"` as a refresh, an
 ML data fetch or a precache sweep starts, and `"done"` or `"failed"` as it ends.
 In between, a refresh sends `"progress"` after each collection whose files it
-fetched. That event's `fetched` is the number of collections whose files the
+fetched. That event's `fetched` is how many collections it has fetched the files
-refresh has fetched so far. Its `total` is the number of collections whose files
+of so far, and its `total` how many it will. A refresh fetches the files of each
-it will fetch in all. A refresh fetches the files of each collection that is new
+collection that is new or changed since the cache's copy, so on an empty cache
-or changed since the cache's copy. On the first refresh, that is every
+it fetches those of every collection.
-collection.
 
 Concurrency is set through `pools`: construct
 `new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
 and pass it. The three pools default to 10 / 5 / 25 (see Request pools below).
 
-`lib.status()` returns a `LibraryStatus`, which holds:
+`lib.status()` returns a `LibraryStatus` (collection/file counts, last
-
+refresh/ML times and errors, originals usage and effective limit, precache
-- the collection and file counts
+progress, and `closed`).
-- the time of the last refresh that succeeded, and the error of the latest
-  refresh if it failed
-- the same two for the ML data fetch
-- the originals cache's usage and its effective limit
-- the precache progress
-- `closed`
 
 `lib.close()` stops the background timer, and is idempotent. A refresh or
-`lib.refreshCollection()` in flight is left to finish. `close()` returns a
+`lib.refreshCollection()` in flight is left to finish. The promise `close()`
-promise that resolves once all of these have finished:
+returns resolves once all of these have finished, so the cache directory can
+then be removed:
 
 - that refresh or `lib.refreshCollection()`, with its cache write
 - the ML data fetch
 - the precache fetches already running
 
-The cache directory can then be removed.
-
 `lib.status().mlDataErrors` lists, as `{ fileID, error }`, each file whose ML
 data the last ML data fetch that succeeded got but could not decrypt, gunzip or
 parse; the `fetchMLData` event with status `"done"` carries the same list. A
@@ -1204,68 +1169,54 @@ synchronously from the copy held in RAM and never touch the network. A refresh
 changes that copy only once all its server requests have succeeded.
 `lib.refreshCollection()` changes it the same way for one collection. So a
 default read is immediate, but only as current as the last refresh or
-`lib.refreshCollection()` whose requests all succeeded. Right after `open()`
+`lib.refreshCollection()` whose requests all succeeded. Right after opening an
-returns without awaiting a refresh, that copy is the one loaded from disk.
+existing cache, it is the copy on disk.
 
 The background timer takes a turn `refreshIntervalSeconds` after its last turn
 ends. Each turn starts a refresh, unless a refresh or `lib.refreshCollection()`
 is already running.
 
-`await lib.fresh()` waits for a refresh to complete and persist. It then returns
+`await lib.fresh()` waits for a refresh to complete and persist, and returns the
-the same `{ albums, photos, timeline }` namespaces, which reflect a completed
+same `{ albums, photos, timeline }` namespaces, which then reflect a completed
-server round-trip. If a `lib.refreshCollection()` is running, `fresh()` first
+server round-trip. It first waits for a running `lib.refreshCollection()` to
-waits for it to end. When a refresh is already running, background or not,
+end. When a refresh is already running, background or not, `fresh()` waits for
-`fresh()` waits for that one. Its answer can then come from requests made before
+that one. Its answer can then come from requests made before the call. Only when
-the call. Only when none is running does it start one. A refresh that fails
+none is running does it start one. A refresh that fails rejects the caller
-rejects the caller (default reads stay silent and keep serving the last good
+(default reads stay silent and keep serving the last good copy). The CLI's read
-copy). Every CLI read command calls `fresh()`, except `quak collections`.
+commands use fresh reads (issue https://git.eeqj.de/sneak/quak/issues/75).
-`quak files` with a collection ID calls it only when `lib.refreshCollection()`
-resolves to `false`.
 
-`await lib.freshCollections()` is the fresh read of the collection list alone.
+`await lib.freshCollections()` is the fresh read of the collection list alone:
-It sends one collections request. It answers with every live collection that
+one collections request, answered with every live collection that decrypts, as
-decrypts, in the server's order, as the `Collection` objects that
+the `Collection` objects `listCollections()` returns, in the server's order. It
-`listCollections()` returns. It fetches no collection's files and stores
+fetches no collection's files and stores nothing, not even the list. A refresh
-nothing, not even the list. A refresh from a cursor past 0 fetches a
+fetches a collection's files only when the collection is new or has changed
-collection's files only when the collection is new or has changed since the
+since the stored copy. A collection stored without its files would never get
-stored copy. So does `lib.refreshCollection()`. Neither would fetch the files of
+them. `quak collections` uses it (issue
-a collection stored without them. `quak collections` uses `freshCollections()`.
+https://git.eeqj.de/sneak/quak/issues/210).
 
 `await lib.refreshCollection(collection)` brings one collection up to date in
 the cache, given its `Collection` from `freshCollections()`. It does for that
 collection what a refresh does for a changed one, and fetches no other
-collection's files. It fetches the collection's files changed since the cache's
+collection's files. It sends no request when the cache's copy has the same
-copy, or all of them when the cache has no copy. It then stores and saves them.
+`updationTime` or a later one. It also sends none when a refresh has already
-It sends no request when the cache's copy has the same `updationTime` or a later
+dropped the collection from the cache, because the server deleted it or it did
-one. It resolves to `true` when the cache then holds the collection.
+not decrypt. Otherwise it fetches the collection's files changed since the
+cache's copy, or all of them when there is none, then stores and saves them. The
+refresh cursor stays where it was, so the next refresh still fetches the files
+of every other collection that changed. It never runs at the same time as a
+refresh. It first waits for a running refresh to end. A `fresh()` or
+`lib.backup()` called after that waits for it. `quak files` with a collection ID
+uses it.
 
-`lib.refreshCollection()` adds a collection the cache does not hold only while
+An album or file in the server's answer that quak cannot decrypt, or whose
-the refresh cursor is 0. The cursor moves past 0 once a refresh receives the
+metadata is not JSON, does not fail the refresh. The refresh leaves it out, an
-account's collections. After that, for a collection the cache does not hold, it
+album with all its files, and drops any older copy the cache held, so reads
-sends no request and resolves to `false`. Such a collection is new since that
+answer from what did decrypt. The library keeps a list of what it left out, with
-refresh, or the refresh dropped it. Only a refresh may then add it, after the
+the reason, in `metadata.json`. The next refresh asks the server only for what
-collections the cache holds.
+changed, so it would not see the record again. A record stays on that list until
-
+a later refresh or `lib.refreshCollection()` stores a copy that decrypts, or
-A refresh from a cursor of 0 replaces everything the cache holds. It fetches the
+learns that the server deleted it. `lib.backup()` reports each such album, and
-files of every collection, even those `lib.refreshCollection()` stored. It
+each such file in its scope that has no copy that decrypted (see "Backup
-stores the collections in the server's order. The cache then holds what it would
-had `lib.refreshCollection()` never run.
-
-`lib.refreshCollection()` leaves the refresh cursor where it was, so the next
-refresh still fetches the files of every other collection that changed. It never
-runs at the same time as a refresh. It first waits for a running refresh or
-`lib.refreshCollection()` to end. While it runs, a `fresh()` or `lib.backup()`
-waits for it to end. `quak files` with a collection ID uses it.
-
-A refresh does not fail on an album or file that quak cannot decrypt, or whose
-metadata is not JSON. The refresh leaves out such a file, and such an album
-together with all its files. It also drops any older copy that the cache held,
-so reads answer from what did decrypt. The library keeps a list of what it left
-out, with the reason, in `metadata.json`. The next refresh asks the server only
-for what changed, so it would not see them again. An entry stays on that list
-until a later refresh or `lib.refreshCollection()` stores a copy that decrypts,
-or learns that the server deleted it. `lib.backup()` reports each such album,
-and each such file in its scope that has no copy that decrypted (see "Backup
 layout").
 
 ### Read surface
@@ -1361,16 +1312,10 @@ The GUI-facing records hold no key material and no binary, so they survive
 
 `lib.snapshot()` returns a `LibrarySnapshot` (albums newest-updated first,
 photos newest first). `lib.subscribe({ onChange })` returns `{ unsubscribe }`.
-It calls `onChange` with a `LibraryChange` whenever a refresh or
+It delivers a `LibraryChange` (`albumsChanged`, `photosChanged`,
-`lib.refreshCollection()` changes these records. A refresh or
+`fileIDsRemoved`, `albumIDsRemoved`, `refreshedAt`) whenever a refresh or
-`lib.refreshCollection()` that changes nothing does not call it. A
+`lib.refreshCollection()` alters the projection. One that changes nothing
-`LibraryChange` holds:
+delivers nothing.
-
-- `albumsChanged`
-- `photosChanged`
-- `fileIDsRemoved`
-- `albumIDsRemoved`
-- `refreshedAt`
 
 ### Thumbnails, ML search, and backup
 
@@ -1386,44 +1331,30 @@ It calls `onChange` with a `LibraryChange` whenever a refresh or
   `SimilarResult[]` (`{ fileID, score }`, cosine similarity, most similar first,
   default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes
   a query vector the caller produced elsewhere.
-- `await lib.backup(opts?)` → `BackupResult`. It takes the lock, `backup.lock`
+- `await lib.backup(opts?)` → `BackupResult`. It takes the lock in the download
-  in the download directory. It fails at once with an error whose `code` is
+  directory, and fails at once with an error whose `code` is `ELOCKED` while
-  `ELOCKED` while another backup of that directory runs. A caller can instead
+  another backup of it runs. A caller can instead take the lock itself, before
-  take the lock itself before it opens its library, as `quak backup` does.
+  it opens its library, as `quak backup` does. `await lockBackupDirectory(dir)`
-  `await lockBackupDirectory(dir)` takes it, failing the same way, and returns a
+  takes it, failing the same way, and returns a `BackupLock`. Its `release()`
-  `BackupLock`. Its `release()` removes the lock unless it was lost, and its
+  removes the lock unless it was lost, and its `lost` holds why it was lost. The
-  `lost` holds why it was lost. The caller passes it to the backup as `lock`,
+  caller passes it to the backup as `lock`, and releases it once the backup
-  and releases it once the backup ends.
+  ends. The backup waits for a refresh as `fresh()` does. It then puts every
-
+  in-scope original not already at its save path there, as `photo.download()`
-    The backup waits for a refresh as `fresh()` does. It then fetches each
+  does, through the content cache. With `includeThumbnails` it also fetches
-    in-scope original that is not yet at its save path, as `photo.download()`
+  thumbnails. It waits for an ML data fetch, and rebuilds the on-disk backup
-    does, through the content cache. With `includeThumbnails` it also fetches
+  tree with a durable failure ledger. Each file's JSON holds its ML data and
-    thumbnails. It waits for an ML data fetch, then rebuilds the backup tree. It
+  keywords. An image's JSON also holds the EXIF and XMP of its original, and its
-    records each file that failed in `failures.json`.
+  dimensions if it is a JPEG. A live photo's JSON holds these for its image. A
-
+  video's JSON holds no EXIF, XMP or dimensions. A fetched original is written
-    Each file's JSON holds its ML data and keywords. An image's JSON also holds
+  straight to its save path and not into the cache, which then counts it as
-    the EXIF and XMP of its original, and its dimensions if it is a JPEG. A live
+  present; one the cache already held is copied from there. `BackupOptions`:
-    photo's JSON holds these for its image. A video's JSON holds no EXIF, XMP or
+  `downloadDirectory` (falls back to the library's), `includeOriginals` (default
-    dimensions.
+  `true`), `includeThumbnails` (default `false`), `onlyAlbumNames`, `verify`
-
+  (default `false`), `onProgress`, and `lock`. See Backup layout above for the
-    A fetched original is written straight to its save path, not into the cache.
+  tree it writes, and for the albums `BackupResult` lists in
-    The cache then counts it as present. An original that the cache already held
+  `undecryptableAlbums` and `failedAlbums`. Whether the backup took the lock or
-    is copied from there.
+  its caller did, once the lock is lost the backup rejects with
-
+  `lost the lock on <dir>: <reason>` before it starts another file.
-    `BackupOptions` holds:
-    - `downloadDirectory`, which falls back to the library's
-    - `includeOriginals`, `true` by default
-    - `includeThumbnails`, `false` by default
-    - `onlyAlbumNames`
-    - `verify`, `false` by default
-    - `onProgress`
-    - `lock`
-
-    "Backup layout" above describes the tree that the backup writes. It also
-    describes the albums that `BackupResult` lists in `undecryptableAlbums` and
-    `failedAlbums`. Once the lock is lost, the backup rejects with
-    `lost the lock on <dir>: <reason>` before it starts another file. This holds
-    whether the backup or its caller took the lock.
 
 ### Request pools
 

TODO.md

@@ -33,13 +33,9 @@ declares one.
   one collection up to date through the new `lib.refreshCollection()`. That
   stores and saves its files as a refresh would, and leaves the refresh cursor
   where it was. When the stored copy is current, it asks for none of the files.
-  It never runs at the same time as a refresh. Once a refresh has received the
+  It never runs at the same time as a refresh. A library opened on a cache that
-  account's collections, it stores no collection the cache does not hold, and
+  only it wrote awaits its first refresh. `quak files` with no argument still
-  `quak files COLLECTION` waits for a refresh instead. A refresh from a cursor
+  waits for a refresh.
-  of 0 replaces all the cache holds, so collections keep the server's order. A
-  library opened on a cache that only `lib.refreshCollection()` wrote awaits its
-  first refresh. `quak files` with no argument still waits for a refresh. The
-  store lists what did not decrypt by collection ID and file ID.
 
 - 2026-10-22: The JSON `quak backup` and `quak backup-metadata` write for a file
   holds `keywords` in place of the CLIP embedding in its ML data

src/cli-commands.ts

@@ -131,9 +131,8 @@ const readLibraryClient = (client: Client): LibraryClient => ({
 // Open a library for a single point command. The aggressive background
 // precache (issue #48) is off, because a one-shot `collections` or `get` must
 // not start downloading the whole account. No refresh starts as it opens or on
-// a timer. Each command asks for the refresh it needs. `collections` needs none
+// a timer. Each command asks for the refresh it needs, and `collections` and
-// (issue #210). `files COLLECTION` needs one only when `refreshCollection()`
+// `files COLLECTION` need none (issues #210 and #217). While a refresh runs,
-// resolves to `false` (issue #217). While a refresh runs,
 // each collection whose files it has fetched is logged, so a large account
 // shows progress rather than nothing.
 const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>

src/cli-read.ts

@@ -12,20 +12,16 @@
 // `files COLLECTION` reads the collection list the same way, for the same
 // reason. It then brings that one collection up to date through
 // `Library.refreshCollection()` (issue #217), and fetches no other collection's
-// files. Once a refresh has received the account's collections,
+// files.
-// `refreshCollection()` stores no collection the library does not hold.
-// `files COLLECTION` then forces a refresh instead.
 //
-// `collections` lists in the server's order. `files` lists in the library's
+// `collections` lists in the server's order and `files` in the library's own
-// own enumeration order: `listCollections()`, then `listFiles()` for each.
+// enumeration order (`listFiles()`), the orders the pre-library CLI printed,
-// Those are the orders the pre-library CLI printed. The `albums`/`photos`
+// rather than the `albums`/`photos` projection's newest-first order, which
-// projection would re-sort the rows newest first. After a refresh, the
+// re-sorts the rows. The field values still come from each record's raw
-// library holds collections in the order the server first sent them. The
+// metadata via `cli-output.ts`. `files` without a collection ID lists every
-// field values still come from each record's raw metadata via
+// file in the account once (issue #211): the library keeps a record of a file
-// `cli-output.ts`. `files` without a collection ID lists every file in the
+// for each collection that holds it, and only the first collection's record is
-// account once (issue #211).
+// listed.
-// The library keeps a record of a file for each collection that holds it, and
-// only the first collection's record is listed.
 
 import type { Collection, EnteFile } from "./model/types.js";
 import type { Photo, PhotosAPI } from "./library/index.js";
@@ -37,8 +33,7 @@ import type { Photo, PhotosAPI } from "./library/index.js";
 export interface FreshReadLibrary {
     fresh(): Promise<unknown>;
     freshCollections(): Promise<Collection[]>;
-    refreshCollection(collection: Collection): Promise<boolean>;
+    refreshCollection(collection: Collection): Promise<void>;
-    getCollection(id: number): Collection | undefined;
     listCollections(): Collection[];
     listFiles(collectionID: number): EnteFile[];
     getFileByID(fileID: number): EnteFile | undefined;
@@ -51,14 +46,12 @@ export const freshCollections = (
     lib: FreshReadLibrary,
 ): Promise<Collection[]> => lib.freshCollections();
 
-// The files of one collection, in enumeration order. `undefined` (not an empty
+// The files of one collection, in enumeration order, current as of one
-// list) when the collection does not exist, so the caller can tell "no such
-// collection" from "an empty collection". They are current as of one
 // collections request and the requests that bring that collection alone up to
-// date. When `refreshCollection()` resolves to `false`, they are current as of
+// date. `undefined` (not an empty list) when the collection does not exist, so
-// a forced refresh instead. Without a collection ID, every file in the account,
+// the caller can tell "no such collection" from "an empty collection". Without
-// current as of a forced refresh. Each file is listed once, as held by the
+// a collection ID, every file in the account, current as of a forced refresh.
-// first collection that holds it.
+// Each file is listed once, as held by the first collection that holds it.
 export const freshFiles = async (
     lib: FreshReadLibrary,
     collectionID?: number,
@@ -67,10 +60,7 @@ export const freshFiles = async (
         const collections = await lib.freshCollections();
         const collection = collections.find((c) => c.id === collectionID);
         if (!collection) return undefined;
-        if (!(await lib.refreshCollection(collection))) {
+        await lib.refreshCollection(collection);
-            await lib.fresh();
-            if (!lib.getCollection(collectionID)) return undefined;
-        }
         return lib.listFiles(collectionID);
     }
     await lib.fresh();

src/library/index.ts

@@ -25,13 +25,9 @@
 // `quak collections` uses it. `refreshCollection()` brings one collection
 // from that list up to date in the store (issue #217). It fetches that
 // collection's files alone, as a refresh would. `quak files COLLECTION` uses
-// it. It never runs at the same time as a refresh. Once the stored cursor is
+// it. It never runs at the same time as a refresh. A caller that asks for each
-// past 0, it only updates a collection the store holds, and only a refresh
+// refresh it needs can also open the library with `refreshOnOpen: false`, so
-// adds one. A refresh from a stored cursor of 0 replaces all the store holds.
+// that `open()` starts none.
-// So after a refresh, the store holds collections in the order the server
-// first sent them. A caller that asks for
-// each refresh it needs can also open the library with `refreshOnOpen: false`,
-// so that `open()` starts none.
 //
 // A refresh reports through `onProgress` each collection whose files it has
 // fetched, so a large account, whose first refresh fetches the files of every
@@ -642,10 +638,10 @@ export class Library {
     // The fresh read of the collection list alone (issue #210): every live
     // collection that decrypts, in the server's order, from one collections
     // request. Unlike a refresh, it fetches no collection's files, which on a
-    // large account take long. It stores nothing. A refresh from a stored
+    // large account take long. It stores nothing. A refresh fetches a
-    // cursor past 0 fetches a collection's files only when the collection is
+    // collection's files only when the collection is new or has changed since
-    // new or has changed since the stored copy. So does `refreshCollection()`.
+    // the stored copy. A collection stored here without its files would never
-    // Neither would fetch the files of a collection stored here without them.
+    // have them fetched.
     async freshCollections(): Promise<Collection[]> {
         if (this.closed) throw new Error("the library is closed");
         const { collections } = await this.client.collectionsSince({
@@ -657,25 +653,16 @@ export class Library {
     // Bring one collection up to date in the store (issue #217), given its
     // record from `freshCollections()`. It does for that collection what a
     // refresh does for a changed one. It fetches no other collection's files.
-    // It resolves to whether the store then holds the collection.
-    //
-    // It adds a collection the store does not hold only while the stored
-    // cursor is 0, before any refresh has received a collection. After that,
-    // it stores nothing for such a collection and resolves to false. Only a
-    // refresh may then add it, after the collections the store holds.
-    //
     // It runs in the same slot as a refresh, so the two never run at the same
     // time. It first waits for a running refresh or `refreshCollection()` to
     // end. While its own work runs, a fresh read or `backup()` waits for it,
     // and the background loop skips its turn.
-    async refreshCollection(collection: Collection): Promise<boolean> {
+    async refreshCollection(collection: Collection): Promise<void> {
         while (this.cycle) await this.cycle.catch(() => {});
         if (this.closed) throw new Error("the library is closed");
-        let held = false;
+        await this.startCycle(false, () =>
-        await this.startCycle(false, async () => {
+            this.refreshCollectionOnce(collection),
-            held = await this.refreshCollectionOnce(collection);
+        );
-        });
-        return held;
     }
 
     // Back up every in-scope file to `opts.downloadDirectory`, or else the
@@ -832,13 +819,7 @@ export class Library {
     // Fetch every change since the stored cursor, then commit. All network
     // reads happen before any store mutation, so a fetch that throws leaves the
     // store untouched and the previous snapshot intact.
-    //
-    // From a stored cursor of 0, the server lists every live collection. The
-    // refresh then replaces all the store holds, and counts every collection
-    // as new. Only `refreshCollection()` can have stored anything before it.
-    // So the store ends as it would had `refreshCollection()` never run.
     private async refreshOnce(): Promise<void> {
-        const replace = this.store.collectionsSinceTime === 0;
         const page = await this.client.collectionsSince({
             sinceTime: this.store.collectionsSinceTime,
         });
@@ -846,19 +827,16 @@ export class Library {
         // Stage per-collection file diffs. A collection's files are
         // re-enumerated only when its updationTime has advanced past the cached
         // copy; an unchanged album's file list cannot have changed. New
-        // collections enumerate from the beginning of time. When the refresh
+        // collections enumerate from the beginning of time. Each collection
-        // replaces the store, every collection counts as new. Each collection
         // whose files are fetched is reported, with how many there are to
         // fetch.
-        const storedCopy = (id: number): Collection | undefined =>
-            replace ? undefined : this.store.getCollection(id);
         const changedCollections = page.collections.filter((collection) => {
-            const known = storedCopy(collection.id);
+            const known = this.store.getCollection(collection.id);
             return !known || collection.updationTime > known.updationTime;
         });
         const filePages: { collectionID: number; page: FilesPage }[] = [];
         for (const collection of changedCollections) {
-            const known = storedCopy(collection.id);
+            const known = this.store.getCollection(collection.id);
             const filePage = await this.client.filesSince({
                 collectionID: collection.id,
                 collectionKey: collection.key,
@@ -873,23 +851,26 @@ export class Library {
             });
         }
 
-        await this.commit(page, filePages, replace);
+        await this.commit(page, filePages);
     }
 
     // `refreshCollection()`'s work, run in the slot: fetch the files of one
-    // collection changed since the stored copy, then commit. It resolves to
+    // collection changed since the stored copy, then commit.
-    // whether the store then holds the collection.
+    private async refreshCollectionOnce(collection: Collection): Promise<void> {
-    private async refreshCollectionOnce(
-        collection: Collection,
-    ): Promise<boolean> {
         const known = this.store.getCollection(collection.id);
-        // A refresh has received the account's collections, and this one is
-        // not stored. It is new since then, or a refresh dropped it. Leave it
-        // to the next refresh, which adds new collections after the stored
-        // ones, in the server's order.
-        if (!known && this.store.collectionsSinceTime > 0) return false;
         // The stored copy is as new or newer, so there is nothing to fetch.
-        if (known && collection.updationTime <= known.updationTime) return true;
+        if (known && collection.updationTime <= known.updationTime) return;
+        // No copy is stored, yet the stored cursor has reached this
+        // `updationTime`. A refresh has then seen this version or a later one,
+        // and dropped the collection: it was deleted, or did not decrypt.
+        // Storing it would bring it back, and no later refresh would see it
+        // again until the server's copy changes.
+        if (
+            !known &&
+            collection.updationTime <= this.store.collectionsSinceTime
+        ) {
+            return;
+        }
         const filePage = await this.client.filesSince({
             collectionID: collection.id,
             collectionKey: collection.key,
@@ -904,22 +885,17 @@ export class Library {
                 cursor: this.store.collectionsSinceTime,
             },
             [{ collectionID: collection.id, page: filePage }],
-            false,
         );
-        return true;
     }
 
     // Commit a collections page and the file pages fetched for its collections
-    // to the store. With `replace`, the page lists every live collection, so
+    // to the store. A collection or file that does not decrypt is left out, as
-    // the commit first deletes every stored collection. A collection or file
+    // if deleted. The reason is kept in the store. No later refresh sees it
-    // that does not decrypt is left out, as if deleted. The reason is kept in
+    // again until the server's copy changes. Last, save the store when it
-    // the store. Until the server's copy changes, only a refresh from a stored
+    // holds changes not yet saved.
-    // cursor of 0 fetches it again. Last, save the store when it holds changes
-    // not yet saved.
     private async commit(
         page: CollectionsPage,
         filePages: { collectionID: number; page: FilesPage }[],
-        replace: boolean,
     ): Promise<void> {
         // The network work is done, so change the store now. `changed` says
         // whether this commit changed anything.
@@ -930,13 +906,6 @@ export class Library {
             changed = true;
         }
 
-        if (replace) {
-            for (const { id } of this.store.listCollections()) {
-                this.store.deleteCollection(id);
-                changed = true;
-            }
-        }
-
         for (const id of page.deleted) {
             if (this.store.getCollection(id)) {
                 this.store.deleteCollection(id);

src/library/precache.ts

@@ -122,7 +122,7 @@ export class Precache {
     }
 
     // Recompute the fill orders and the pinned set from the current projection.
-    // Called at open and after every change to the store.
+    // Called at open and after every refresh that changes the library.
     update(records: DerivedRecords): void {
         const photos = [...records.photos.values()].sort(
             (a, b) => b.takenAt - a.takenAt || b.fileID - a.fileID,

src/library/records.ts

@@ -71,14 +71,13 @@ export interface LibrarySnapshot {
     takenAt: number;
 }
 
-// What one refresh or `refreshCollection()` changed in the records.
 export interface LibraryChange {
-    // Full records for albums/photos added or changed.
+    // Full records for albums/photos added or changed by the refresh.
     albumsChanged: AlbumRecord[];
     photosChanged: PhotoRecord[];
     fileIDsRemoved: number[];
     albumIDsRemoved: number[];
-    // Wall-clock milliseconds when the store changed.
+    // Wall-clock milliseconds of the refresh that produced this change.
     refreshedAt: number;
 }
 

src/library/store.ts

@@ -3,7 +3,7 @@
 // The store keeps one `metadata.json` file holding the account's server
 // state: the user id, a schema version, the cursor for the incremental
 // collections listing, the decrypted collection and file records, and the
-// collections and files that did not decrypt. The whole file is read
+// collections and files the refresh could not decrypt. The whole file is read
 // into RAM on load and rewritten as a whole on save; there is no partial
 // update and no lock file. A separate refresh unit populates the store from
 // the server — this module only stores what it is given.
@@ -34,11 +34,9 @@ const FILE_MODE = 0o600;
 type StoredCollection = Omit<Collection, "key"> & { key: string };
 type StoredFile = Omit<EnteFile, "key"> & { key: string };
 
-// A collection, or with `fileID` a file in it, that did not decrypt or parse,
+// A collection, or with `fileID` a file in it, that the last refresh to fetch
-// and why. A refresh or `refreshCollection()` records it on fetching such a
+// it could not decrypt or parse, and why. It is kept until a later refresh
-// copy. A later fetch drops it on getting a copy that decrypts, or on learning
+// fetches it again, because a refresh fetches only what changed.
-// that the server deleted it. Each fetch asks only for what changed, so the
-// store keeps the record between fetches.
 export interface Undecryptable {
     collectionID: number;
     fileID?: number;
@@ -224,14 +222,8 @@ export class MetadataStore {
         this.files.delete(fileKey(collectionID, fileID));
     }
 
-    // Sorted by collection ID, then file ID, so the order does not depend on
-    // when each record was fetched.
     listUndecryptable(): Undecryptable[] {
-        return [...this.undecryptable].sort(
+        return [...this.undecryptable];
-            (a, b) =>
-                a.collectionID - b.collectionID ||
-                (a.fileID ?? 0) - (b.fileID ?? 0),
-        );
     }
 
     // Replaces any earlier record of the same collection or file.

test/cli/commands.test.ts

@@ -124,35 +124,20 @@ const FILES: Record<number, EnteFile[]> = {
 // thumbnail as empty. `withNewFile` adds new.jpg (102) to Vacation, advancing
 // the collection's updationTime as the server does. `beachInWork` adds
 // beach.jpg (100) to Work as well, so that one file is in both albums.
-// `newAlbums` adds the albums A (3) and B (4), in that order, which both hold
+// `refreshError` makes listing collections fail with that message.
-// shared.jpg (300). `refreshError` makes listing collections fail with that
-// message.
 const fakeClient = (
     opts: {
         failID?: number;
         emptyThumbID?: number;
         withNewFile?: boolean;
         beachInWork?: boolean;
-        newAlbums?: boolean;
         refreshError?: string;
     } = {},
 ) => {
-    let collections = opts.withNewFile
+    const collections = opts.withNewFile
         ? [{ ...COLLECTIONS[0], updationTime: 2 }, COLLECTIONS[1]]
         : COLLECTIONS;
     let files = FILES;
-    if (opts.newAlbums) {
-        collections = [
-            ...collections,
-            { ...collection(3, "A"), updationTime: 2 },
-            { ...collection(4, "B"), updationTime: 2 },
-        ];
-        files = {
-            ...files,
-            3: [file(300, 3, "shared.jpg")],
-            4: [file(300, 4, "shared.jpg")],
-        };
-    }
     if (opts.withNewFile) {
         files = {
             ...files,
@@ -584,57 +569,6 @@ describe("files", () => {
         ]);
         expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS);
     });
-
-    // `files 2` on an empty cache stores Work alone. The refresh that `files`
-    // then waits for must still store Vacation first, as the server lists it.
-    it("without a collection ID lists in the server's order after files 2 on an empty cache", async () => {
-        const ctx = context(fakeClient({ beachInWork: true }));
-        expect(await filesCommand(ctx, "2", {})).toBe(0);
-        stdout.text = "";
-        stderr.text = "";
-
-        expect(await filesCommand(ctx, undefined, { json: true })).toBe(0);
-        expect(
-            JSON.parse(stdout.text).map(
-                (row: { id: number; collectionID: number }) => [
-                    row.id,
-                    row.collectionID,
-                ],
-            ),
-        ).toEqual([
-            [100, 1],
-            [101, 1],
-            [200, 2],
-        ]);
-        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS);
-    });
-
-    // After a refresh, the server adds A and then B. `files 4` must not store B
-    // ahead of A, so it waits for a refresh, which stores both in that order.
-    it("without a collection ID lists new albums in the server's order after files with the second one's ID", async () => {
-        expect(await filesCommand(context(), undefined, {})).toBe(0);
-        const ctx = context(fakeClient({ newAlbums: true }));
-        stdout.text = "";
-
-        expect(await filesCommand(ctx, "4", {})).toBe(0);
-        expect(stdout.text).toBe("300\timage\tshared.jpg\n");
-        stdout.text = "";
-
-        expect(await filesCommand(ctx, undefined, { json: true })).toBe(0);
-        expect(
-            JSON.parse(stdout.text).map(
-                (row: { id: number; collectionID: number }) => [
-                    row.id,
-                    row.collectionID,
-                ],
-            ),
-        ).toEqual([
-            [100, 1],
-            [101, 1],
-            [200, 2],
-            [300, 3],
-        ]);
-    });
 });
 
 describe("get and get-thumb", () => {
@@ -1453,12 +1387,12 @@ describe("backup-metadata --exif", () => {
     });
 });
 
-// Each test first runs `files` so the cache holds the account as it was, then
+// Each test first runs `files 1` so the cache holds Vacation as it was, then
-// changes the server under it. (`collections` would not do: it reads the
+// adds a file to Vacation on the server. (`collections` would not do: it reads
-// collection list alone and stores nothing.)
+// the collection list alone and stores nothing.)
 describe("backup-metadata and the thumbnail helpers refresh first", () => {
     beforeEach(async () => {
-        expect(await filesCommand(context(), undefined, {})).toBe(0);
+        expect(await filesCommand(context(), "1", {})).toBe(0);
         stdout.text = "";
         stderr.text = "";
     });
@@ -1767,8 +1701,8 @@ describe("an account with many albums", () => {
         expect(stderr.text).toBe("");
         expect(paths).toEqual(["/collections/v2?sinceTime=0"]);
 
-        // It stored nothing that would make `refreshCollection()` skip an
+        // It stored nothing that would make a refresh skip an album's files:
-        // album's files: `files` after it still finds the album's photo.
+        // `files` after it still finds the album's photo.
         stdout.text = "";
         expect(await filesCommand(ctx, "7", {})).toBe(0);
         expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n");

test/cli/log.test.ts

@@ -234,8 +234,8 @@ describe("-q", () => {
         });
     });
 
-    // The line saying why the command failed is the fatal one, and it is
+    // The refresh's progress is left out; the line saying why the command
-    // printed.
+    // failed is the fatal one, and it is printed.
     it("prints only the line a failing command exits with", async () => {
         expect(await quak("-q", "files", "9")).toEqual({
             code: 1,

test/cli/read.test.ts

@@ -11,10 +11,8 @@
  * (`freshCollections()`, issue #210). It must not wait for a refresh, which
  * would fetch the files of every collection. `files COLLECTION` reads that list
  * too, then brings that one collection up to date (`refreshCollection()`,
- * issue #217). The stand-in serves a collection's files once `fresh()` has run,
+ * issue #217). The stand-in serves a collection's files once `fresh()` or
- * or once `refreshCollection()` has resolved to `true` for that collection.
+ * `refreshCollection()` for that collection has run.
- * When `refreshCollection()` resolves to `false`, `files COLLECTION` forces a
- * refresh instead.
  *
  * `collections` and `files` also list in the library's enumeration order
  * (`listCollections`/`listFiles`) — the order the pre-library CLI printed — not
@@ -69,18 +67,14 @@ const file = (
 
 // A library that reveals its records only after `fresh()` has been awaited, and
 // serves them in the enumeration order it was given. `freshCollections()`
-// answers the collection list in that order without a refresh. When
+// answers the collection list in that order without a refresh.
-// `refreshCollection()` resolves to true, it reveals the files of that one
+// `refreshCollection()` reveals the files of that one collection.
-// collection.
 // `photos.byID` returns a stand-in `Photo` carrying just the fileID the helper
 // passes through.
 class FakeLibrary implements FreshReadLibrary {
     freshCalls = 0;
     freshCollectionsCalls = 0;
     refreshCollectionIDs: number[] = [];
-    // What `refreshCollection()` resolves to. With false it reveals nothing,
-    // as a library that has had a refresh does for a collection it lacks.
-    refreshCollectionHolds = true;
     private refreshed = false;
 
     constructor(
@@ -99,14 +93,8 @@ class FakeLibrary implements FreshReadLibrary {
         return this.collections;
     }
 
-    async refreshCollection(collection: Collection): Promise<boolean> {
+    async refreshCollection(collection: Collection): Promise<void> {
         this.refreshCollectionIDs.push(collection.id);
-        return this.refreshCollectionHolds;
-    }
-
-    getCollection(id: number): Collection | undefined {
-        if (!this.refreshed) return undefined;
-        return this.collections.find((c) => c.id === id);
     }
 
     listCollections(): Collection[] {
@@ -115,9 +103,7 @@ class FakeLibrary implements FreshReadLibrary {
 
     listFiles(collectionID: number): EnteFile[] {
         const current =
-            this.refreshed ||
+            this.refreshed || this.refreshCollectionIDs.includes(collectionID);
-            (this.refreshCollectionHolds &&
-                this.refreshCollectionIDs.includes(collectionID));
         return current
             ? this.files.filter((f) => f.collectionID === collectionID)
             : [];
@@ -230,19 +216,6 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
         ]);
     });
 
-    it("freshFiles forces a refresh when refreshCollection resolves to false", async () => {
-        const lib = new FakeLibrary(
-            [collection(1, 100)],
-            [file(10, 1, 1_700_000_000_000_000)],
-        );
-        lib.refreshCollectionHolds = false;
-
-        const rows = await freshFiles(lib, 1);
-        expect(lib.refreshCollectionIDs).toEqual([1]);
-        expect(lib.freshCalls).toBe(1);
-        expect(rows?.map((f) => f.id)).toEqual([10]);
-    });
-
     it("freshFiles returns undefined for an unknown collection", async () => {
         const lib = new FakeLibrary([collection(1, 100)], []);
         const rows = await freshFiles(lib, 999);

test/library/fresh.test.ts

@@ -20,11 +20,9 @@
  *  5. `refreshCollection()` brings one collection up to date in the store
  *     (issue #217). It leaves the stored cursor where it was, so the next
  *     refresh still fetches the files of every other collection that changed.
- *     It never runs at the same time as a refresh. After a refresh, it stores
+ *     It never runs at the same time as a refresh, and never brings back a
- *     no collection the store does not hold: not a new one, and not one a
+ *     collection that a refresh deleted. `close()` waits for it. A library
- *     refresh deleted. `close()` waits for it. A library opened on a cache that
+ *     opened on a cache that only it wrote awaits its first refresh.
- *     only it wrote awaits its first refresh. That refresh replaces what it
- *     stored, in the server's order.
  *
  * The client is the same metadata-only mock the background-refresh tests use:
  * no crypto, no network, scripted pages, and a record of each call. A long
@@ -316,10 +314,10 @@ describe("Library.freshCollections", () => {
         rmSync(dir, { recursive: true, force: true });
     });
 
-    // A refresh from a stored cursor past 0 fetches a collection's files only
+    // A refresh fetches a collection's files only when the collection is new
-    // when the collection is new or its updationTime has advanced past the
+    // or its updationTime has advanced past the stored copy's. Had the
-    // stored copy's. So does `refreshCollection()`. Neither would fetch the
+    // collection list been stored, the next refresh would take every
-    // files of a collection stored here without them.
+    // collection for one it already holds the files of, and fetch none.
     it("asks for the collection list alone and stores nothing", async () => {
         const client = new MockClient();
         const page = {
@@ -421,7 +419,7 @@ describe("Library.refreshCollection", () => {
             });
 
             // Collection 1's files alone, since the stored copy.
-            expect(await lib.refreshCollection(collection(1, 110))).toBe(true);
+            await lib.refreshCollection(collection(1, 110));
             expect(client.filesCalls).toEqual([
                 { collectionID: 1, sinceTime: 100 },
             ]);
@@ -503,7 +501,7 @@ describe("Library.refreshCollection", () => {
             await lib.fresh();
             client.filesCalls.length = 0;
 
-            expect(await lib.refreshCollection(listed)).toBe(false);
+            await lib.refreshCollection(listed);
             expect(client.filesCalls).toEqual([]);
             expect(lib.getCollection(1)).toBeUndefined();
         } finally {
@@ -511,21 +509,6 @@ describe("Library.refreshCollection", () => {
         }
     });
 
-    // A refresh has stored collection 1. Collection 2 is new on the server
-    // since then. Only the next refresh may add it, after collection 1.
-    it("stores nothing for a collection that is new since the last refresh", async () => {
-        const { client, lib } = await openSeeded(cacheDirectory);
-        try {
-            client.filesCalls.length = 0;
-
-            expect(await lib.refreshCollection(collection(2, 110))).toBe(false);
-            expect(client.filesCalls).toEqual([]);
-            expect(lib.getCollection(2)).toBeUndefined();
-        } finally {
-            await lib.close();
-        }
-    });
-
     // The caller may remove the cache directory once `close()` resolves, so
     // the cache write must have landed by then.
     it("close() waits for a refreshCollection() in flight", async () => {
@@ -555,13 +538,12 @@ describe("Library.refreshCollection", () => {
         await refreshing;
     });
 
-    // `quak files 2` on an empty cache stores collection 2 alone. A library
+    // `quak files 1` on an empty cache stores collection 1 alone. A library
-    // opened later on that cache must not serve it as the whole account. Its
+    // opened later on that cache must not serve it as the whole account.
-    // first refresh replaces what the cache holds, in the server's order.
     it("open() awaits the first refresh on a cache that only refreshCollection() wrote", async () => {
         const client = new MockClient();
-        client.filesFor(2, {
+        client.filesFor(1, {
-            files: [file(2001, 2, 90)],
+            files: [file(1001, 1, 90)],
             deleted: [],
             cursor: 90,
         });
@@ -571,7 +553,7 @@ describe("Library.refreshCollection", () => {
             refreshOnOpen: false,
             refreshIntervalSeconds: SLOW_INTERVAL,
         });
-        expect(await first.refreshCollection(collection(2, 100))).toBe(true);
+        await first.refreshCollection(collection(1, 100));
         await first.close();
 
         client.collectionsQueue.push({
@@ -579,17 +561,11 @@ describe("Library.refreshCollection", () => {
             deleted: [],
             cursor: 100,
         });
-        client.filesFor(1, {
-            files: [file(1001, 1, 90)],
-            deleted: [],
-            cursor: 90,
-        });
         client.filesFor(2, {
             files: [file(2001, 2, 90)],
             deleted: [],
             cursor: 90,
         });
-        client.filesCalls.length = 0;
         const lib = await Library.open({
             client,
             cacheDirectory,
@@ -598,11 +574,6 @@ describe("Library.refreshCollection", () => {
         try {
             expect(lib.status().lastRefreshAt).toBeDefined();
             expect(lib.listCollections().map((c) => c.id)).toEqual([1, 2]);
-            // It fetches collection 2's files again, from the beginning.
-            expect(client.filesCalls).toEqual([
-                { collectionID: 1, sinceTime: 0 },
-                { collectionID: 2, sinceTime: 0 },
-            ]);
             expect(lib.listFiles(2).map((f) => f.id)).toEqual([2001]);
         } finally {
             await lib.close();

test/library/library.test.ts

@@ -20,9 +20,9 @@
  * 5. `close()` stops the timer and is idempotent, and its promise resolves
  *    only once an in-flight refresh has written the cache file.
  * 6. `cacheDirectory` defaults to the env-paths cache dir plus the user id.
- * 7. `open()` branches on the stored cursor. At 0, as on an empty cache, it
+ * 7. `open()` branches on the cache: an empty cache awaits the first refresh
- *    awaits the first refresh. Past 0 it serves the cache's copy at once and
+ *    (it has nothing to serve yet); an existing cache serves its copy at once
- *    refreshes in the background, so a slow or dead server never stalls
+ *    and refreshes in the background, so a slow or dead server never stalls
  *    opening.
  * 8. A save failure that leaves RAM ahead of disk keeps `status().lastError`
  *    set and keeps retrying the write; a later empty refresh does not clear it.

test/library/snapshot.test.ts

@@ -4,8 +4,7 @@
  * These are the surface the GUI consumes across Electron IPC. `snapshot()` is
  * synchronous — it reads the in-RAM store and projects it into plain records
  * (no keys) — and `subscribe({ onChange })` delivers a `LibraryChange` whenever
- * a refresh or `refreshCollection()` changes the derived records. The
+ * a background refresh actually changes the derived records. The contracts:
- * contracts:
  *
  *  1. `snapshot()` deduplicates a file across memberships into one record with
  *     every album id, orders photos newest first, and carries no key material.

test/library/store.test.ts

@@ -172,7 +172,7 @@ describe("MetadataStore", () => {
         expect(store.listCollections()).toEqual([]);
     });
 
-    it("loads an empty store when the list of what did not decrypt is damaged", async () => {
+    it("loads an empty store when the list of what the refresh could not decrypt is damaged", async () => {
         // A backup reads that list on every run, so a damaged one that loaded
         // would fail every backup until the cache was deleted. A string is
         // damaged too, though JavaScript can loop over its characters.
@@ -266,23 +266,4 @@ describe("MetadataStore", () => {
         expect(store.getFile(12345, 67890)).toBeUndefined();
         expect(store.getFile(55555, 67890)?.key).toEqual(inB.key);
     });
-
-    // A backup reports these in this order. It must not depend on whether a
-    // refresh or `Library.refreshCollection()` fetched each one first.
-    it("lists what did not decrypt by collection ID, then file ID", async () => {
-        const store = await MetadataStore.load(path);
-        store.putUndecryptable({ collectionID: 2, fileID: 7, error: "a" });
-        store.putUndecryptable({ collectionID: 3, error: "b" });
-        store.putUndecryptable({ collectionID: 1, fileID: 9, error: "c" });
-        store.putUndecryptable({ collectionID: 2, fileID: 5, error: "d" });
-
-        expect(
-            store.listUndecryptable().map((r) => [r.collectionID, r.fileID]),
-        ).toEqual([
-            [1, 9],
-            [2, 5],
-            [2, 7],
-            [3, undefined],
-        ]);
-    });
 });