quak get and get-thumb stop fetching collections once they find the file (closes #222)

`quak get` and `quak get-thumb` used to wait for a full refresh, which on
an empty cache fetched the files of every collection. `freshFile` now
reads the collection list, then one collection's files at a time, and
stops at the first that holds the file. It tries first the collections
the cache says hold the file, and stores nothing. The commands fetch the
content through the new `Library.originalOf()` and
`Library.thumbnailOf()`, which take the file record itself, since the
store may not hold it.

They no longer print a refresh's progress lines. The `-q` test that
relied on `get` refreshing now runs `helper fix-missing-thumbnails`.

Model: opus-5-5

README.md

@@ -687,22 +687,21 @@ answer, so they report current account state rather than whatever the cache last
 held. `collections` asks the server for the collection list alone, in one
 request, and neither fetches any collection's files nor changes the cache.
 `files` with a collection ID asks for the same list, then for that one
-collection's files, and does not change the cache either. On an empty cache,
+collection's files, and does not change the cache either. `get` and `get-thumb`
-`get` and `get-thumb` ask for the same list, then for one collection's files at
+ask for the same list, then for one collection's files at a time, until they
-a time, until they find the file. They store none of the collections or file
+find the file. They ask first for the collections the cache says hold the file.
-records they read. The original or thumbnail they fetch is kept in the cache.
+They store nothing they read. The original or thumbnail they fetch is kept in
-Once the cache holds the account's collections, they wait for a refresh instead.
+the cache. The others wait for a refresh, and so does `files` without a
-The others wait for a refresh, and so does `files` without a collection ID. With
+collection ID. With an empty cache, as on a first run, that refresh fetches the
-an empty cache, as on a first run, that refresh fetches the files of every
+files of every collection, one collection after another, and stores nothing
-collection, one collection after another, and stores nothing until it has them
+until it has them all, which on a large account takes a while. Meanwhile the
-all, which on a large account takes a while. Meanwhile the command logs a line
+command logs a line on stderr for each collection whose files it has fetched,
-on stderr for each collection whose files it has fetched, such as
+such as `Fetched the files of 3 of 120 collections`, with `--json` too; `backup`
-`Fetched the files of 3 of 120 collections`, with `--json` too; `backup` does
+does the same while the refresh it starts with runs. stdout holds only the
-the same while the refresh it starts with runs. stdout holds only the command's
+command's output. If that round-trip fails, the command prints the error on one
-output. If that round-trip fails, the command prints the error on one line and
+line and exits 1, or 3 when the server no longer accepts the saved session (see
-exits 1, or 3 when the server no longer accepts the saved session (see "Session
+"Session handling"). An album or file quak cannot decrypt does not fail it; the
-handling"). An album or file quak cannot decrypt does not fail it; the command
+command answers from what did decrypt (see "Default reads vs. fresh reads").
-answers from what did decrypt (see "Default reads vs. fresh reads").
 `--cache-dir` overrides where the cache lives; without it each account gets its
 own directory under the per-user cache path.
 
@@ -1191,8 +1190,7 @@ https://git.eeqj.de/sneak/quak/issues/210).
 It answers with every live file in that collection that decrypts, in the
 server's order. `collection` is a record that `freshCollections()` returned. The
 method fetches no other collection's files and stores nothing. `quak files` with
-a collection ID uses it, and so do `quak get` and `quak get-thumb` on an empty
+a collection ID uses it, and so do `quak get` and `quak get-thumb`.
-cache.
 
 An album or file in the server's answer that quak cannot decrypt, or whose
 metadata is not JSON, does not fail the refresh. The refresh leaves it out, an
@@ -1277,9 +1275,8 @@ in the cache, which does not make `isLocal` true; only `download()` and
 
 `await lib.originalOf(file, opts?)` and `await lib.thumbnailOf(file, opts?)`
 fetch and cache as `photo.original()` and `photo.thumbnail()` do. `file` is an
-`EnteFile` the caller holds, and the library need not hold it. On an empty
+`EnteFile` the caller holds, and the library need not hold it. `quak get` and
-cache, `quak get` and `quak get-thumb` pass one that `lib.freshFiles()`
+`quak get-thumb` pass one that `lib.freshFiles()` returned.
-returned.
 
 Lower-level accessors that return decrypted model objects (which hold key
 material) are also available: `listCollections()`, `getCollection(id)`,

TODO.md

@@ -26,14 +26,13 @@ declares one.
 
 # Completed Steps
 
-- 2026-10-26: On an empty cache, `quak get` and `quak get-thumb` stop fetching
+- 2026-10-26: `quak get` and `quak get-thumb` stop fetching collections' files
-  collections' files once they find the file
+  once they find the file (https://git.eeqj.de/sneak/quak/issues/222). They used
-  (https://git.eeqj.de/sneak/quak/issues/222). They used to wait for a refresh,
+  to wait for a refresh, which on an empty cache fetched the files of every
-  which on an empty cache fetched the files of every collection. Now they read
+  collection. They now read the collection list, then one collection's files at
-  the collection list, then one collection's files at a time. They store none of
+  a time, first those the cache says hold the file. They store nothing they
-  the collections or file records they read, and fetch the original or thumbnail
+  read, and fetch the file through the new `lib.originalOf()` and
-  through the new `lib.originalOf()` and `lib.thumbnailOf()`. Once the cache
+  `lib.thumbnailOf()`.
-  holds the account's collections, they still wait for a refresh.
 
 - 2026-10-26: `quak files` with a collection ID fetches that collection's files
   alone (https://git.eeqj.de/sneak/quak/issues/217). It used to wait for a

src/cli-commands.ts

@@ -132,10 +132,10 @@ const readLibraryClient = (client: Client): LibraryClient => ({
 // precache (issue #48) is off, because a one-shot `collections` or `get` must
 // not start downloading the whole account. No refresh starts as the library
 // opens or on a timer. Each command asks for the refresh it needs.
-// `collections` and `files` with a collection ID need none (issues #210 and
+// `collections` needs none (issue #210). Nor do `get`, `get-thumb` and `files`
-// #217). Nor do `get` and `get-thumb` on an empty cache (issue #222). While a
+// with a collection ID (issues #217 and #222). While a refresh runs, each
-// refresh runs, each collection whose files it has fetched is logged, so a
+// collection whose files it has fetched is logged, so a large account shows
-// large account shows progress rather than nothing.
+// progress rather than nothing.
 const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>
     Library.open({
         client: readLibraryClient(client),
@@ -323,8 +323,8 @@ export const getCommand = async (
 
     const lib = await openReadLibrary(ctx, client);
     try {
-        // Force a server round-trip so the file resolves against current state
+        // Ask the server for the file as it is now (issue #36 amendment,
-        // (issue #36 amendment, issue #52).
+        // issues #52 and #222).
         const file = await freshFile(lib, fileID);
         if (!file) {
             ctx.log("fatal", `File ${fileID} not found`);
@@ -380,8 +380,8 @@ export const getThumbCommand = async (
 
     const lib = await openReadLibrary(ctx, client);
     try {
-        // Force a server round-trip so the file resolves against current state
+        // Ask the server for the file as it is now (issue #36 amendment,
-        // (issue #36 amendment, issue #52).
+        // issues #52 and #222).
         const file = await freshFile(lib, fileID);
         if (!file) {
             ctx.log("fatal", `File ${fileID} not found`);

src/cli-read.ts

@@ -2,16 +2,16 @@
 //
 // `collections`, `files`, `get`, and `get-thumb` must answer for
 // the account's state at the moment the command runs, not for whatever the
-// local cache last happened to hold (owner amendment, issue #36). `get`,
+// local cache last happened to hold (owner amendment, issue #36). `files`
-// `get-thumb` and `files` without a collection ID therefore force a server
+// without a collection ID therefore forces a server round-trip through
-// round-trip through `Library.fresh()` and only then read, so a change made
+// `Library.fresh()` and only then reads, so a change made elsewhere is visible.
-// elsewhere is visible. `collections` asks the library for the collection list
+// `collections` asks the library for the collection list alone through
-// alone through `Library.freshCollections()` (issue #210): a refresh would also
+// `Library.freshCollections()` (issue #210): a refresh would also fetch the
-// fetch the files of every collection, which on a large account takes long and
+// files of every collection, which on a large account takes long and which
-// which `collections` does not print. `files` with a collection ID reads that
+// `collections` does not print. `files` with a collection ID reads that list
-// list too, then that one collection's files through `Library.freshFiles()`
+// too, then that one collection's files through `Library.freshFiles()` (issue
-// (issue #217). On an empty cache, `get` and `get-thumb` read that list, then
+// #217). `get` and `get-thumb` read that list, then one collection's files at a
-// one collection's files at a time, until they find the file (issue #222).
+// time, until they find the file (issue #222).
 //
 // `collections` and `files` with a collection ID list in the server's order.
 // `files` without one lists in the library's own enumeration order
@@ -33,7 +33,7 @@ export interface FreshReadLibrary {
     freshFiles(collection: Collection): Promise<EnteFile[]>;
     listCollections(): Collection[];
     listFiles(collectionID: number): EnteFile[];
-    getFileByID(fileID: number): EnteFile | undefined;
+    getFile(collectionID: number, fileID: number): EnteFile | undefined;
 }
 
 // Every live collection, current as of one collections request, in the
@@ -70,20 +70,23 @@ export const freshFiles = async (
 };
 
 // One file as the server holds it now, or `undefined` when no collection holds
-// it. When the cache holds the account's collections, it refreshes and looks
+// it. It reads the collection list, then one collection's files at a time, and
-// the file up in the cache. On an empty cache, it reads the collection list,
+// stops at the first collection that holds the file. It tries first the
-// then one collection's files at a time, and stops at the first collection
+// collections the cache says hold the file, then the rest in the server's
-// that holds the file. That search stores nothing.
+// order. It stores nothing.
 export const freshFile = async (
     lib: FreshReadLibrary,
     fileID: number,
 ): Promise<EnteFile | undefined> => {
-    // Only a refresh stores collections, so a cache holding any holds them all.
+    const collections = await lib.freshCollections();
-    if (lib.listCollections().length > 0) {
+    // Whether the cache holds the file in collection `c`.
-        await lib.fresh();
+    const inCache = (c: Collection): boolean =>
-        return lib.getFileByID(fileID);
+        lib.getFile(c.id, fileID) !== undefined;
-    }
+    const ordered = [
-    for (const collection of await lib.freshCollections()) {
+        ...collections.filter(inCache),
+        ...collections.filter((c) => !inCache(c)),
+    ];
+    for (const collection of ordered) {
         const files = await lib.freshFiles(collection);
         const file = files.find((f) => f.id === fileID);
         if (file) return file;

src/library/index.ts

@@ -647,8 +647,8 @@ export class Library {
     }
 
     // The original of `file`, fetched and cached as `Photo.original()` does.
-    // `file` is used as given, so the store need not hold it. On an empty
+    // `file` is used as given, so the store need not hold it. `quak get`
-    // cache, `quak get` passes a file that `freshFiles()` returned.
+    // passes a file that `freshFiles()` returned.
     async originalOf(
         file: EnteFile,
         opts?: ContentOptions,
@@ -662,8 +662,8 @@ export class Library {
     }
 
     // The thumbnail of `file`, fetched and cached as `Photo.thumbnail()` does.
-    // `file` is used as given, so the store need not hold it. On an empty
+    // `file` is used as given, so the store need not hold it. `quak get-thumb`
-    // cache, `quak get-thumb` passes a file that `freshFiles()` returned.
+    // passes a file that `freshFiles()` returned.
     async thumbnailOf(
         file: EnteFile,
         opts?: ContentOptions,

test/cli/commands.test.ts

@@ -1816,7 +1816,7 @@ describe("an account with many albums", () => {
 
     // Photo 200 is in album 2, so `get` asks for no file of album 3 or later.
     // The last request downloads the photo's original.
-    it("get on an empty cache asks for the albums' files only until one holds the photo", async () => {
+    it("get asks for the albums' files only until one holds the photo", async () => {
         const out = join(root, "photo.jpg");
         expect(await getCommand(accountContext(), "200", { out })).toBe(0);
 
@@ -1831,9 +1831,9 @@ describe("an account with many albums", () => {
         expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
     });
 
-    // `files` first fills the cache, so `get` waits for a refresh. No album has
+    // `files` first fills the cache. The cache then says album 40, the last,
-    // changed since, so the refresh asks for no album's files.
+    // holds photo 4000, so `get` asks for that album's files alone.
-    it("get on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => {
+    it("get asks first for the files of the album the cache says holds the photo", async () => {
         const ctx = accountContext();
         expect(await filesCommand(ctx, undefined, {})).toBe(0);
         requests.length = 0;
@@ -1841,18 +1841,14 @@ describe("an account with many albums", () => {
 
         const out = join(root, "photo.jpg");
         expect(await getCommand(ctx, "4000", { out })).toBe(0);
+
         expect(readFileSync(out, "utf-8")).toBe("photo-40.jpg");
         expect(stderr.text).toBe(`12 bytes -> ${out}\n`);
         expect(requests).toEqual([
-            "/collections/v2?sinceTime=1",
+            "/collections/v2?sinceTime=0",
+            "/collections/v2/diff?collectionID=40&sinceTime=0",
             "/?fileID=4000",
         ]);
-
-        requests.length = 0;
-        stderr.text = "";
-        expect(await getCommand(ctx, "999", { out })).toBe(1);
-        expect(stderr.text).toBe("File 999 not found\n");
-        expect(requests).toEqual(["/collections/v2?sinceTime=1"]);
     });
 
     it("files counts on stderr each album whose files its refresh fetched", async () => {

test/cli/read.test.ts

@@ -3,16 +3,16 @@
  * issue #36, issue #52).
  *
  * The `collections`, `files`, `get`, and `get-thumb` commands must answer for
- * current server state, not the local cache. `get`, `get-thumb` and `files`
+ * current server state, not the local cache. `files` without a collection ID
- * without a collection ID force a `Library.fresh()` round-trip before they
+ * forces a `Library.fresh()` round-trip before it reads. The stand-in library
- * read. The stand-in library below serves no cached record until `fresh()` has
+ * below serves no cached record until `fresh()` has been awaited, so a helper
- * been awaited, so a helper that read without refreshing would come back empty
+ * that read without refreshing would come back empty and fail here.
- * and fail here. `collections` asks the library for the collection list alone
+ * `collections` asks the library for the collection list alone
  * (`freshCollections()`, issue #210). `files` with a collection ID asks for
  * that list, then for that one collection's files (`freshFiles()`, issue
- * #217). Neither may wait for a refresh, which would fetch the files of every
+ * #217). `get` and `get-thumb` ask for that list, then for one collection's
- * collection. On an empty cache, `get` and `get-thumb` ask for that list, then
+ * files at a time, until they find the file (issue #222). None of these may
- * for one collection's files at a time, until they find the file (issue #222).
+ * wait for a refresh, which would fetch the files of every collection.
  *
  * `collections` and `files` also list in the order the library gives them, not
  * the albums/photos projection's newest-first order. The fixtures are seeded in
@@ -105,9 +105,8 @@ class FakeLibrary implements FreshReadLibrary {
             : [];
     }
 
-    getFileByID(fileID: number): EnteFile | undefined {
+    getFile(collectionID: number, fileID: number): EnteFile | undefined {
-        if (!this.refreshed) return undefined;
+        return this.listFiles(collectionID).find((f) => f.id === fileID);
-        return this.files.find((f) => f.id === fileID);
     }
 }
 
@@ -222,7 +221,7 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
             ],
         );
 
-    it("freshFile on an empty cache reads each collection's files in the server's order until one holds the file", async () => {
+    it("freshFile reads each collection's files in the server's order until one holds the file", async () => {
         const lib = threeCollections();
 
         const found = await freshFile(lib, 20);
@@ -234,18 +233,16 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
     });
 
     // The stand-in's cache holds what the server holds once `fresh()` has run.
-    it("freshFile on a cache that holds the collections refreshes, then looks the file up in the cache", async () => {
+    it("freshFile reads first the files of the collection the cache says holds the file", async () => {
         const lib = threeCollections();
         await lib.fresh();
 
         const found = await freshFile(lib, 30);
-        expect(lib.freshCalls).toBe(2);
+        expect(lib.freshFilesCalls).toEqual([3]);
-        expect(lib.freshCollectionsCalls).toBe(0);
-        expect(lib.freshFilesCalls).toEqual([]);
         expect(found?.collectionID).toBe(3);
     });
 
-    it("freshFile on an empty cache returns undefined for an unknown file, after reading every collection's files", async () => {
+    it("freshFile returns undefined for an unknown file, after reading every collection's files", async () => {
         const lib = threeCollections();
 
         const found = await freshFile(lib, 404);