quak files COLLECTION fetches that collection's files alone (closes #217)

`quak files COLLECTION` used to wait for a full refresh, which on an
empty cache fetched the files of every collection first. It now reads
the collection list, then calls the new `Library.refreshCollection()`.
That brings one collection up to date in the store as a refresh would.
Once the stored cursor is past 0, it only updates a collection the store
holds. For a new one, `quak files COLLECTION` waits for a refresh. A
refresh from a cursor of 0 replaces all the store holds, so collections
keep the server's order. `open()` awaits the first refresh while the
cursor is 0. The store lists what did not decrypt in a fixed order.

Model: opus-5-5

README.md

@@ -708,12 +708,14 @@ fetches no collection's files and leaves the cache as it is. `files` with a
 collection ID asks for the collection list the same way. It then asks for the
 files of that collection alone, and stores them in the cache. It asks for no
 files when the cache's copy of that collection is current. The other read
-commands, and `files` without a collection ID, wait for a refresh.
+commands, and `files` without a collection ID, wait for a refresh. So does
+`files` with the ID of a collection that is new since the last refresh.
 
-On an empty cache, as on a first run, that refresh fetches the files of every
-collection, one collection after another. It stores nothing until it has them
-all, and on a large account that takes a while. Meanwhile the command logs a
-line on stderr for each collection whose files it has fetched, such as
+The first refresh fetches the files of every collection, one collection after
+another. That includes any collection that `files` with a collection ID stored
+before it. The refresh stores nothing until it has them all, and on a large
+account that takes a while. Meanwhile the command logs a line on stderr for each
+collection whose files it has fetched, such as
 `Fetched the files of 3 of 120 collections`. It logs these lines with `--json`
 too. `backup` logs them during the refresh it starts with. stdout holds only the
 command's output.
@@ -777,9 +779,10 @@ its `Failed files:` list. Every other line is logged as it is without `--json`.
 exit status 1.
 
 `files <collectionID>` lists the files in that collection. `files` with no
-argument lists every file in the account, collection by collection, each file
-once: a file in several collections is listed with the first of them, which is
-its `collectionID` in `--json`.
+argument lists every file in the account once, collection by collection. The
+collections come in the order the server first sent them in a refresh. A file in
+several collections is listed with the first of them, which is its
+`collectionID` in `--json`.
 
 An ID given to `files`, `get`, `get-thumb` or
 `helper fix-missing-thumbnails --file` is written in decimal digits, no larger
@@ -1159,7 +1162,8 @@ In between, a refresh sends `"progress"` after each collection whose files it
 fetched. That event's `fetched` is the number of collections whose files the
 refresh has fetched so far. Its `total` is the number of collections whose files
 it will fetch in all. A refresh fetches the files of each collection that is new
-or changed since the cache's copy. On an empty cache, that is every collection.
+or changed since the cache's copy. On the first refresh, that is every
+collection.
 
 Concurrency is set through `pools`: construct
 `new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
@@ -1214,8 +1218,9 @@ waits for it to end. When a refresh is already running, background or not,
 `fresh()` waits for that one. Its answer can then come from requests made before
 the call. Only when none is running does it start one. A refresh that fails
 rejects the caller (default reads stay silent and keep serving the last good
-copy). Every CLI read command calls `fresh()`, except `quak collections` and
-`quak files` with a collection ID.
+copy). Every CLI read command calls `fresh()`, except `quak collections`.
+`quak files` with a collection ID calls it only when `lib.refreshCollection()`
+resolves to `false`.
 
 `await lib.freshCollections()` is the fresh read of the collection list alone.
 It sends one collections request. It answers with every live collection that
@@ -1232,8 +1237,19 @@ collection what a refresh does for a changed one, and fetches no other
 collection's files. It fetches the collection's files changed since the cache's
 copy, or all of them when the cache has no copy. It then stores and saves them.
 It sends no request when the cache's copy has the same `updationTime` or a later
-one. It also sends none when a refresh has already dropped the collection from
-the cache, because the server deleted it or it did not decrypt.
+one. It resolves to `true` when the cache then holds the collection.
+
+`lib.refreshCollection()` adds a collection the cache does not hold only while
+the refresh cursor is 0. The cursor moves past 0 once a refresh receives the
+account's collections. After that, for a collection the cache does not hold, it
+sends no request and resolves to `false`. Such a collection is new since that
+refresh, or the refresh dropped it. Only a refresh may then add it, after the
+collections the cache holds.
+
+A refresh from a cursor of 0 replaces everything the cache holds. It fetches the
+files of every collection, even those `lib.refreshCollection()` stored. It
+stores the collections in the server's order. The cache then holds what it would
+had `lib.refreshCollection()` never run.
 
 `lib.refreshCollection()` leaves the refresh cursor where it was, so the next
 refresh still fetches the files of every other collection that changed. It never

TODO.md

@@ -33,9 +33,13 @@ declares one.
   one collection up to date through the new `lib.refreshCollection()`. That
   stores and saves its files as a refresh would, and leaves the refresh cursor
   where it was. When the stored copy is current, it asks for none of the files.
-  It never runs at the same time as a refresh. A library opened on a cache that
-  only it wrote awaits its first refresh. `quak files` with no argument still
-  waits for a refresh.
+  It never runs at the same time as a refresh. Once a refresh has received the
+  account's collections, it stores no collection the cache does not hold, and
+  `quak files COLLECTION` waits for a refresh instead. A refresh from a cursor
+  of 0 replaces all the cache holds, so collections keep the server's order. A
+  library opened on a cache that only `lib.refreshCollection()` wrote awaits its
+  first refresh. `quak files` with no argument still waits for a refresh. The
+  store lists what did not decrypt by collection ID and file ID.
 
 - 2026-10-22: The JSON `quak backup` and `quak backup-metadata` write for a file
   holds `keywords` in place of the CLIP embedding in its ML data

src/cli-read.ts

@@ -12,16 +12,20 @@
 // `files COLLECTION` reads the collection list the same way, for the same
 // reason. It then brings that one collection up to date through
 // `Library.refreshCollection()` (issue #217), and fetches no other collection's
-// files.
+// files. Once a refresh has received the account's collections,
+// `refreshCollection()` stores no collection the library does not hold.
+// `files COLLECTION` then forces a refresh instead.
 //
-// `collections` lists in the server's order and `files` in the library's own
-// enumeration order (`listFiles()`), the orders the pre-library CLI printed,
-// rather than the `albums`/`photos` projection's newest-first order, which
-// re-sorts the rows. The field values still come from each record's raw
-// metadata via `cli-output.ts`. `files` without a collection ID lists every
-// file in the account once (issue #211): the library keeps a record of a file
-// for each collection that holds it, and only the first collection's record is
-// listed.
+// `collections` lists in the server's order. `files` lists in the library's
+// own enumeration order: `listCollections()`, then `listFiles()` for each.
+// Those are the orders the pre-library CLI printed. The `albums`/`photos`
+// projection would re-sort the rows newest first. After a refresh, the
+// library holds collections in the order the server first sent them. The
+// field values still come from each record's raw metadata via
+// `cli-output.ts`. `files` without a collection ID lists every file in the
+// account once (issue #211).
+// The library keeps a record of a file for each collection that holds it, and
+// only the first collection's record is listed.
 
 import type { Collection, EnteFile } from "./model/types.js";
 import type { Photo, PhotosAPI } from "./library/index.js";
@@ -33,7 +37,8 @@ import type { Photo, PhotosAPI } from "./library/index.js";
 export interface FreshReadLibrary {
     fresh(): Promise<unknown>;
     freshCollections(): Promise<Collection[]>;
-    refreshCollection(collection: Collection): Promise<void>;
+    refreshCollection(collection: Collection): Promise<boolean>;
+    getCollection(id: number): Collection | undefined;
     listCollections(): Collection[];
     listFiles(collectionID: number): EnteFile[];
     getFileByID(fileID: number): EnteFile | undefined;
@@ -46,12 +51,14 @@ export const freshCollections = (
     lib: FreshReadLibrary,
 ): Promise<Collection[]> => lib.freshCollections();
 
-// The files of one collection, in enumeration order, current as of one
+// The files of one collection, in enumeration order. `undefined` (not an empty
+// list) when the collection does not exist, so the caller can tell "no such
+// collection" from "an empty collection". They are current as of one
 // collections request and the requests that bring that collection alone up to
-// date. `undefined` (not an empty list) when the collection does not exist, so
-// the caller can tell "no such collection" from "an empty collection". Without
-// a collection ID, every file in the account, current as of a forced refresh.
-// Each file is listed once, as held by the first collection that holds it.
+// date. When `refreshCollection()` stores nothing, they are current as of a
+// forced refresh instead. Without a collection ID, every file in the account,
+// current as of a forced refresh. Each file is listed once, as held by the
+// first collection that holds it.
 export const freshFiles = async (
     lib: FreshReadLibrary,
     collectionID?: number,
@@ -60,7 +67,10 @@ export const freshFiles = async (
         const collections = await lib.freshCollections();
         const collection = collections.find((c) => c.id === collectionID);
         if (!collection) return undefined;
-        await lib.refreshCollection(collection);
+        if (!(await lib.refreshCollection(collection))) {
+            await lib.fresh();
+            if (!lib.getCollection(collectionID)) return undefined;
+        }
         return lib.listFiles(collectionID);
     }
     await lib.fresh();

src/library/index.ts

@@ -25,9 +25,12 @@
 // `quak collections` uses it. `refreshCollection()` brings one collection
 // from that list up to date in the store (issue #217). It fetches that
 // collection's files alone, as a refresh would. `quak files COLLECTION` uses
-// it. It never runs at the same time as a refresh. A caller that asks for each
-// refresh it needs can also open the library with `refreshOnOpen: false`, so
-// that `open()` starts none.
+// it. It never runs at the same time as a refresh. Once the stored cursor is
+// past 0, it only updates a collection the store holds, and only a refresh
+// adds one. A refresh from a stored cursor of 0 replaces all the store holds.
+// So the store keeps collections in the server's order. A caller that asks for
+// each refresh it needs can also open the library with `refreshOnOpen: false`,
+// so that `open()` starts none.
 //
 // A refresh reports through `onProgress` each collection whose files it has
 // fetched, so a large account, whose first refresh fetches the files of every
@@ -653,16 +656,25 @@ export class Library {
     // Bring one collection up to date in the store (issue #217), given its
     // record from `freshCollections()`. It does for that collection what a
     // refresh does for a changed one. It fetches no other collection's files.
+    // It resolves to whether the store then holds the collection.
+    //
+    // It adds a collection the store does not hold only while the stored
+    // cursor is 0, before any refresh has received a collection. After that,
+    // it stores nothing for such a collection and resolves to false. Only a
+    // refresh may then add it, after the collections the store holds.
+    //
     // It runs in the same slot as a refresh, so the two never run at the same
     // time. It first waits for a running refresh or `refreshCollection()` to
     // end. While its own work runs, a fresh read or `backup()` waits for it,
     // and the background loop skips its turn.
-    async refreshCollection(collection: Collection): Promise<void> {
+    async refreshCollection(collection: Collection): Promise<boolean> {
         while (this.cycle) await this.cycle.catch(() => {});
         if (this.closed) throw new Error("the library is closed");
-        await this.startCycle(false, () =>
-            this.refreshCollectionOnce(collection),
-        );
+        let held = false;
+        await this.startCycle(false, async () => {
+            held = await this.refreshCollectionOnce(collection);
+        });
+        return held;
     }
 
     // Back up every in-scope file to `opts.downloadDirectory`, or else the
@@ -819,7 +831,13 @@ export class Library {
     // Fetch every change since the stored cursor, then commit. All network
     // reads happen before any store mutation, so a fetch that throws leaves the
     // store untouched and the previous snapshot intact.
+    //
+    // From a stored cursor of 0, the server lists every live collection. The
+    // refresh then replaces all the store holds, and counts every collection
+    // as new. Only `refreshCollection()` can have stored anything before it.
+    // So the store ends as it would had `refreshCollection()` never run.
     private async refreshOnce(): Promise<void> {
+        const replace = this.store.collectionsSinceTime === 0;
         const page = await this.client.collectionsSince({
             sinceTime: this.store.collectionsSinceTime,
         });
@@ -827,16 +845,19 @@ export class Library {
         // Stage per-collection file diffs. A collection's files are
         // re-enumerated only when its updationTime has advanced past the cached
         // copy; an unchanged album's file list cannot have changed. New
-        // collections enumerate from the beginning of time. Each collection
+        // collections enumerate from the beginning of time. When the refresh
+        // replaces the store, every collection counts as new. Each collection
         // whose files are fetched is reported, with how many there are to
         // fetch.
+        const storedCopy = (id: number): Collection | undefined =>
+            replace ? undefined : this.store.getCollection(id);
         const changedCollections = page.collections.filter((collection) => {
-            const known = this.store.getCollection(collection.id);
+            const known = storedCopy(collection.id);
             return !known || collection.updationTime > known.updationTime;
         });
         const filePages: { collectionID: number; page: FilesPage }[] = [];
         for (const collection of changedCollections) {
-            const known = this.store.getCollection(collection.id);
+            const known = storedCopy(collection.id);
             const filePage = await this.client.filesSince({
                 collectionID: collection.id,
                 collectionKey: collection.key,
@@ -851,26 +872,23 @@ export class Library {
             });
         }
 
-        await this.commit(page, filePages);
+        await this.commit(page, filePages, replace);
     }
 
     // `refreshCollection()`'s work, run in the slot: fetch the files of one
-    // collection changed since the stored copy, then commit.
-    private async refreshCollectionOnce(collection: Collection): Promise<void> {
+    // collection changed since the stored copy, then commit. It resolves to
+    // whether the store then holds the collection.
+    private async refreshCollectionOnce(
+        collection: Collection,
+    ): Promise<boolean> {
         const known = this.store.getCollection(collection.id);
+        // A refresh has received the account's collections, and this one is
+        // not stored. It is new since then, or a refresh dropped it. Leave it
+        // to the next refresh, which adds new collections after the stored
+        // ones, in the server's order.
+        if (!known && this.store.collectionsSinceTime > 0) return false;
         // The stored copy is as new or newer, so there is nothing to fetch.
-        if (known && collection.updationTime <= known.updationTime) return;
-        // No copy is stored, yet the stored cursor has reached this
-        // `updationTime`. A refresh has then seen this version or a later one,
-        // and dropped the collection: it was deleted, or did not decrypt.
-        // Storing it would bring it back, and no later refresh would see it
-        // again until the server's copy changes.
-        if (
-            !known &&
-            collection.updationTime <= this.store.collectionsSinceTime
-        ) {
-            return;
-        }
+        if (known && collection.updationTime <= known.updationTime) return true;
         const filePage = await this.client.filesSince({
             collectionID: collection.id,
             collectionKey: collection.key,
@@ -885,17 +903,21 @@ export class Library {
                 cursor: this.store.collectionsSinceTime,
             },
             [{ collectionID: collection.id, page: filePage }],
+            false,
         );
+        return true;
     }
 
     // Commit a collections page and the file pages fetched for its collections
-    // to the store. A collection or file that does not decrypt is left out, as
-    // if deleted. The reason is kept in the store. No later refresh sees it
-    // again until the server's copy changes. Last, save the store when it
-    // holds changes not yet saved.
+    // to the store. With `replace`, the page lists every live collection, so
+    // the commit first deletes every stored collection. A collection or file
+    // that does not decrypt is left out, as if deleted. The reason is kept in
+    // the store. No later refresh sees it again until the server's copy
+    // changes. Last, save the store when it holds changes not yet saved.
     private async commit(
         page: CollectionsPage,
         filePages: { collectionID: number; page: FilesPage }[],
+        replace: boolean,
     ): Promise<void> {
         // The network work is done, so change the store now. `changed` says
         // whether this commit changed anything.
@@ -906,6 +928,13 @@ export class Library {
             changed = true;
         }
 
+        if (replace) {
+            for (const { id } of this.store.listCollections()) {
+                this.store.deleteCollection(id);
+                changed = true;
+            }
+        }
+
         for (const id of page.deleted) {
             if (this.store.getCollection(id)) {
                 this.store.deleteCollection(id);

src/library/store.ts

@@ -222,8 +222,14 @@ export class MetadataStore {
         this.files.delete(fileKey(collectionID, fileID));
     }
 
+    // Sorted by collection ID, then file ID, so the order does not depend on
+    // when each record was fetched.
     listUndecryptable(): Undecryptable[] {
-        return [...this.undecryptable];
+        return [...this.undecryptable].sort(
+            (a, b) =>
+                a.collectionID - b.collectionID ||
+                (a.fileID ?? 0) - (b.fileID ?? 0),
+        );
     }
 
     // Replaces any earlier record of the same collection or file.

test/cli/commands.test.ts

@@ -124,20 +124,35 @@ const FILES: Record<number, EnteFile[]> = {
 // thumbnail as empty. `withNewFile` adds new.jpg (102) to Vacation, advancing
 // the collection's updationTime as the server does. `beachInWork` adds
 // beach.jpg (100) to Work as well, so that one file is in both albums.
-// `refreshError` makes listing collections fail with that message.
+// `newAlbums` adds the albums A (3) and B (4), in that order, which both hold
+// shared.jpg (300). `refreshError` makes listing collections fail with that
+// message.
 const fakeClient = (
     opts: {
         failID?: number;
         emptyThumbID?: number;
         withNewFile?: boolean;
         beachInWork?: boolean;
+        newAlbums?: boolean;
         refreshError?: string;
     } = {},
 ) => {
-    const collections = opts.withNewFile
+    let collections = opts.withNewFile
         ? [{ ...COLLECTIONS[0], updationTime: 2 }, COLLECTIONS[1]]
         : COLLECTIONS;
     let files = FILES;
+    if (opts.newAlbums) {
+        collections = [
+            ...collections,
+            { ...collection(3, "A"), updationTime: 2 },
+            { ...collection(4, "B"), updationTime: 2 },
+        ];
+        files = {
+            ...files,
+            3: [file(300, 3, "shared.jpg")],
+            4: [file(300, 4, "shared.jpg")],
+        };
+    }
     if (opts.withNewFile) {
         files = {
             ...files,
@@ -569,6 +584,57 @@ describe("files", () => {
         ]);
         expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS);
     });
+
+    // `files 2` on an empty cache stores Work alone. The refresh that `files`
+    // then waits for must still store Vacation first, as the server lists it.
+    it("without a collection ID lists in the server's order after files 2 on an empty cache", async () => {
+        const ctx = context(fakeClient({ beachInWork: true }));
+        expect(await filesCommand(ctx, "2", {})).toBe(0);
+        stdout.text = "";
+        stderr.text = "";
+
+        expect(await filesCommand(ctx, undefined, { json: true })).toBe(0);
+        expect(
+            JSON.parse(stdout.text).map(
+                (row: { id: number; collectionID: number }) => [
+                    row.id,
+                    row.collectionID,
+                ],
+            ),
+        ).toEqual([
+            [100, 1],
+            [101, 1],
+            [200, 2],
+        ]);
+        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS);
+    });
+
+    // After a refresh, the server adds A and then B. `files 4` must not store B
+    // ahead of A, so it waits for a refresh, which stores both in that order.
+    it("without a collection ID lists new albums in the server's order after files with the second one's ID", async () => {
+        expect(await filesCommand(context(), undefined, {})).toBe(0);
+        const ctx = context(fakeClient({ newAlbums: true }));
+        stdout.text = "";
+
+        expect(await filesCommand(ctx, "4", {})).toBe(0);
+        expect(stdout.text).toBe("300\timage\tshared.jpg\n");
+        stdout.text = "";
+
+        expect(await filesCommand(ctx, undefined, { json: true })).toBe(0);
+        expect(
+            JSON.parse(stdout.text).map(
+                (row: { id: number; collectionID: number }) => [
+                    row.id,
+                    row.collectionID,
+                ],
+            ),
+        ).toEqual([
+            [100, 1],
+            [101, 1],
+            [200, 2],
+            [300, 3],
+        ]);
+    });
 });
 
 describe("get and get-thumb", () => {
@@ -1387,12 +1453,12 @@ describe("backup-metadata --exif", () => {
     });
 });
 
-// Each test first runs `files 1` so the cache holds Vacation as it was, then
-// adds a file to Vacation on the server. (`collections` would not do: it reads
-// the collection list alone and stores nothing.)
+// Each test first runs `files` so the cache holds the account as it was, then
+// changes the server under it. (`collections` would not do: it reads the
+// collection list alone and stores nothing.)
 describe("backup-metadata and the thumbnail helpers refresh first", () => {
     beforeEach(async () => {
-        expect(await filesCommand(context(), "1", {})).toBe(0);
+        expect(await filesCommand(context(), undefined, {})).toBe(0);
         stdout.text = "";
         stderr.text = "";
     });

test/cli/read.test.ts

@@ -12,7 +12,8 @@
  * would fetch the files of every collection. `files COLLECTION` reads that list
  * too, then brings that one collection up to date (`refreshCollection()`,
  * issue #217). The stand-in serves a collection's files once `fresh()` or
- * `refreshCollection()` for that collection has run.
+ * `refreshCollection()` for that collection has run. When `refreshCollection()`
+ * stores nothing, `files COLLECTION` forces a refresh instead.
  *
  * `collections` and `files` also list in the library's enumeration order
  * (`listCollections`/`listFiles`) — the order the pre-library CLI printed — not
@@ -75,6 +76,9 @@ class FakeLibrary implements FreshReadLibrary {
     freshCalls = 0;
     freshCollectionsCalls = 0;
     refreshCollectionIDs: number[] = [];
+    // What `refreshCollection()` resolves to. With false it reveals nothing,
+    // as a library that has had a refresh does for a collection it lacks.
+    refreshCollectionHolds = true;
     private refreshed = false;
 
     constructor(
@@ -93,8 +97,14 @@ class FakeLibrary implements FreshReadLibrary {
         return this.collections;
     }
 
-    async refreshCollection(collection: Collection): Promise<void> {
+    async refreshCollection(collection: Collection): Promise<boolean> {
         this.refreshCollectionIDs.push(collection.id);
+        return this.refreshCollectionHolds;
+    }
+
+    getCollection(id: number): Collection | undefined {
+        if (!this.refreshed) return undefined;
+        return this.collections.find((c) => c.id === id);
     }
 
     listCollections(): Collection[] {
@@ -103,7 +113,9 @@ class FakeLibrary implements FreshReadLibrary {
 
     listFiles(collectionID: number): EnteFile[] {
         const current =
-            this.refreshed || this.refreshCollectionIDs.includes(collectionID);
+            this.refreshed ||
+            (this.refreshCollectionHolds &&
+                this.refreshCollectionIDs.includes(collectionID));
         return current
             ? this.files.filter((f) => f.collectionID === collectionID)
             : [];
@@ -216,6 +228,19 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
         ]);
     });
 
+    it("freshFiles forces a refresh when refreshCollection stores nothing", async () => {
+        const lib = new FakeLibrary(
+            [collection(1, 100)],
+            [file(10, 1, 1_700_000_000_000_000)],
+        );
+        lib.refreshCollectionHolds = false;
+
+        const rows = await freshFiles(lib, 1);
+        expect(lib.refreshCollectionIDs).toEqual([1]);
+        expect(lib.freshCalls).toBe(1);
+        expect(rows?.map((f) => f.id)).toEqual([10]);
+    });
+
     it("freshFiles returns undefined for an unknown collection", async () => {
         const lib = new FakeLibrary([collection(1, 100)], []);
         const rows = await freshFiles(lib, 999);

test/library/fresh.test.ts

@@ -20,9 +20,11 @@
  *  5. `refreshCollection()` brings one collection up to date in the store
  *     (issue #217). It leaves the stored cursor where it was, so the next
  *     refresh still fetches the files of every other collection that changed.
- *     It never runs at the same time as a refresh, and never brings back a
- *     collection that a refresh deleted. `close()` waits for it. A library
- *     opened on a cache that only it wrote awaits its first refresh.
+ *     It never runs at the same time as a refresh. After a refresh, it stores
+ *     no collection the store does not hold: not a new one, and not one a
+ *     refresh deleted. `close()` waits for it. A library opened on a cache that
+ *     only it wrote awaits its first refresh. That refresh replaces what it
+ *     stored, in the server's order.
  *
  * The client is the same metadata-only mock the background-refresh tests use:
  * no crypto, no network, scripted pages, and a record of each call. A long
@@ -419,7 +421,7 @@ describe("Library.refreshCollection", () => {
             });
 
             // Collection 1's files alone, since the stored copy.
-            await lib.refreshCollection(collection(1, 110));
+            expect(await lib.refreshCollection(collection(1, 110))).toBe(true);
             expect(client.filesCalls).toEqual([
                 { collectionID: 1, sinceTime: 100 },
             ]);
@@ -501,7 +503,7 @@ describe("Library.refreshCollection", () => {
             await lib.fresh();
             client.filesCalls.length = 0;
 
-            await lib.refreshCollection(listed);
+            expect(await lib.refreshCollection(listed)).toBe(false);
             expect(client.filesCalls).toEqual([]);
             expect(lib.getCollection(1)).toBeUndefined();
         } finally {
@@ -509,6 +511,21 @@ describe("Library.refreshCollection", () => {
         }
     });
 
+    // A refresh has stored collection 1. Collection 2 is new on the server
+    // since then. Only the next refresh may add it, after collection 1.
+    it("stores nothing for a collection that is new since the last refresh", async () => {
+        const { client, lib } = await openSeeded(cacheDirectory);
+        try {
+            client.filesCalls.length = 0;
+
+            expect(await lib.refreshCollection(collection(2, 110))).toBe(false);
+            expect(client.filesCalls).toEqual([]);
+            expect(lib.getCollection(2)).toBeUndefined();
+        } finally {
+            await lib.close();
+        }
+    });
+
     // The caller may remove the cache directory once `close()` resolves, so
     // the cache write must have landed by then.
     it("close() waits for a refreshCollection() in flight", async () => {
@@ -538,12 +555,13 @@ describe("Library.refreshCollection", () => {
         await refreshing;
     });
 
-    // `quak files 1` on an empty cache stores collection 1 alone. A library
-    // opened later on that cache must not serve it as the whole account.
+    // `quak files 2` on an empty cache stores collection 2 alone. A library
+    // opened later on that cache must not serve it as the whole account. Its
+    // first refresh replaces what the cache holds, in the server's order.
     it("open() awaits the first refresh on a cache that only refreshCollection() wrote", async () => {
         const client = new MockClient();
-        client.filesFor(1, {
-            files: [file(1001, 1, 90)],
+        client.filesFor(2, {
+            files: [file(2001, 2, 90)],
             deleted: [],
             cursor: 90,
         });
@@ -553,7 +571,7 @@ describe("Library.refreshCollection", () => {
             refreshOnOpen: false,
             refreshIntervalSeconds: SLOW_INTERVAL,
         });
-        await first.refreshCollection(collection(1, 100));
+        expect(await first.refreshCollection(collection(2, 100))).toBe(true);
         await first.close();
 
         client.collectionsQueue.push({
@@ -561,11 +579,17 @@ describe("Library.refreshCollection", () => {
             deleted: [],
             cursor: 100,
         });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
         client.filesFor(2, {
             files: [file(2001, 2, 90)],
             deleted: [],
             cursor: 90,
         });
+        client.filesCalls.length = 0;
         const lib = await Library.open({
             client,
             cacheDirectory,
@@ -574,6 +598,11 @@ describe("Library.refreshCollection", () => {
         try {
             expect(lib.status().lastRefreshAt).toBeDefined();
             expect(lib.listCollections().map((c) => c.id)).toEqual([1, 2]);
+            // It fetches collection 2's files again, from the beginning.
+            expect(client.filesCalls).toEqual([
+                { collectionID: 1, sinceTime: 0 },
+                { collectionID: 2, sinceTime: 0 },
+            ]);
             expect(lib.listFiles(2).map((f) => f.id)).toEqual([2001]);
         } finally {
             await lib.close();

test/library/store.test.ts

@@ -266,4 +266,23 @@ describe("MetadataStore", () => {
         expect(store.getFile(12345, 67890)).toBeUndefined();
         expect(store.getFile(55555, 67890)?.key).toEqual(inB.key);
     });
+
+    // A backup reports these in this order. It must not depend on whether a
+    // refresh or `Library.refreshCollection()` fetched each one first.
+    it("lists what did not decrypt by collection ID, then file ID", async () => {
+        const store = await MetadataStore.load(path);
+        store.putUndecryptable({ collectionID: 2, fileID: 7, error: "a" });
+        store.putUndecryptable({ collectionID: 3, error: "b" });
+        store.putUndecryptable({ collectionID: 1, fileID: 9, error: "c" });
+        store.putUndecryptable({ collectionID: 2, fileID: 5, error: "d" });
+
+        expect(
+            store.listUndecryptable().map((r) => [r.collectionID, r.fileID]),
+        ).toEqual([
+            [1, 9],
+            [2, 5],
+            [2, 7],
+            [3, undefined],
+        ]);
+    });
 });