quak files COLLECTION fetches that collection's files alone (closes #217)

`quak files COLLECTION` used to wait for a full refresh, which on an
empty cache fetched the files of every collection before it listed one.
It now reads the collection list, then calls the new
`Library.refreshCollection()`. That brings one collection up to date in
the store as a refresh would, and saves it. It runs in the same slot as
a refresh, so the two never run at the same time, and `close()` waits
for it. It never stores a collection that a refresh has already dropped.
It leaves the refresh cursor alone. `open()` therefore awaits the first
refresh on a cache whose cursor is still 0. `quak files` with no
argument still waits for a refresh.

Model: opus-5-5

TODO.md

@@ -26,30 +26,16 @@ declares one.
 
 # Completed Steps
 
-- 2026-10-26: On an empty cache, `quak helper fix-missing-thumbnails --file`
+- 2026-10-25: `quak files COLLECTION` fetches the files of that collection alone
-  stops fetching collections' files once it finds every file it names
+  (https://git.eeqj.de/sneak/quak/issues/217). It used to wait for a refresh,
-  (https://git.eeqj.de/sneak/quak/issues/223). It used to wait for a refresh,
+  which on an empty cache fetched the files of every collection first. It now
-  which on an empty cache fetched the files of every collection. Now it reads
+  reads the collection list through `lib.freshCollections()`, then brings that
-  the collection list, then one collection's files at a time, the most recently
+  one collection up to date through the new `lib.refreshCollection()`. That
-  updated first. It stores none of the collections or file records it reads. It
+  stores and saves its files as a refresh would, and leaves the refresh cursor
-  fetches each original through `lib.originalOf()`, which keeps it in the cache.
+  where it was. When the stored copy is current, it asks for none of the files.
-  Once the cache holds the account's collections, it still waits for a refresh.
+  It never runs at the same time as a refresh. A library opened on a cache that
-
+  only it wrote awaits its first refresh. `quak files` with no argument still
-- 2026-10-26: On an empty cache, `quak get` and `quak get-thumb` stop fetching
+  waits for a refresh.
-  collections' files once they find the file
-  (https://git.eeqj.de/sneak/quak/issues/222). They used to wait for a refresh,
-  which on an empty cache fetched the files of every collection. Now they read
-  the collection list, then one collection's files at a time. They store none of
-  the collections or file records they read, and fetch the original or thumbnail
-  through the new `lib.originalOf()` and `lib.thumbnailOf()`. Once the cache
-  holds the account's collections, they still wait for a refresh.
-
-- 2026-10-26: `quak files` with a collection ID fetches that collection's files
-  alone (https://git.eeqj.de/sneak/quak/issues/217). It used to wait for a
-  refresh, which on an empty cache fetched the files of every collection. It now
-  reads the collection list through `lib.freshCollections()`, then the one
-  collection's files through the new `lib.freshFiles()`. It stores nothing, and
-  the files print in the server's order.
 
 - 2026-10-22: The JSON `quak backup` and `quak backup-metadata` write for a file
   holds `keywords` in place of the CLIP embedding in its ML data

src/cli-commands.ts

@@ -37,16 +37,9 @@ import {
     thumbnailName,
 } from "./cli-output.js";
 import { type Log, logRefresh } from "./cli-log.js";
-import {
+import { freshCollections, freshFiles, freshFile } from "./cli-read.js";
-    cachedFilesByID,
-    freshCollections,
-    freshFiles,
-    freshFile,
-    freshFilesByID,
-} from "./cli-read.js";
 import { withExtension } from "./filename.js";
 import { runMetadataBackup } from "./metadata-backup.js";
-import type { Collection, EnteFile } from "./model/types.js";
 import { listMissingThumbnails, fixMissingThumbnails } from "./thumbnails.js";
 
 export interface CliContext {
@@ -137,13 +130,11 @@ const readLibraryClient = (client: Client): LibraryClient => ({
 
 // Open a library for a single point command. The aggressive background
 // precache (issue #48) is off, because a one-shot `collections` or `get` must
-// not start downloading the whole account. No refresh starts as the library
+// not start downloading the whole account. No refresh starts as it opens or on
-// opens or on a timer. Each command asks for the refresh it needs.
+// a timer. Each command asks for the refresh it needs, and `collections` and
-// `collections` and `files` with a collection ID need none (issues #210 and
+// `files COLLECTION` need none (issues #210 and #217). While a refresh runs,
-// #217). Nor do `get` and `get-thumb` on an empty cache (issue #222). Nor does
+// each collection whose files it has fetched is logged, so a large account
-// `helper fix-missing-thumbnails --file` on an empty cache (issue #223). While
+// shows progress rather than nothing.
-// a refresh runs, each collection whose files it has fetched is logged, so a
-// large account shows progress rather than nothing.
 const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>
     Library.open({
         client: readLibraryClient(client),
@@ -292,11 +283,11 @@ export const filesCommand = async (
 
     const lib = await openReadLibrary(ctx, client);
     try {
-        // Ask the server for current state (issue #36 amendment, issue #52).
+        // Force a server round-trip and list in enumeration order (issue #36
-        // Each file prints from its own decrypted metadata (raw title,
+        // amendment, issue #52). Each file prints from its own decrypted
-        // microsecond creationTime) via cli-output. The files print in the
+        // metadata (raw title, microsecond creationTime) via cli-output, and in
-        // order `freshFiles` returns them, not the projection's newest-first
+        // the pre-library CLI's enumeration order, not the projection's
-        // order.
+        // newest-first order.
         const files = await freshFiles(lib, collectionID);
         if (!files) {
             ctx.log("fatal", `Collection ${collectionID} not found`);
@@ -333,13 +324,14 @@ export const getCommand = async (
     try {
         // Force a server round-trip so the file resolves against current state
         // (issue #36 amendment, issue #52).
-        const file = await freshFile(lib, fileID);
+        const resolved = await freshFile(lib, fileID);
-        if (!file) {
+        if (!resolved) {
             ctx.log("fatal", `File ${fileID} not found`);
             return 1;
         }
+        const { photo, file } = resolved;
 
-        const result = await lib.originalOf(file);
+        const result = await photo.original();
         // Default name is the file's own title, as the pre-library CLI used
         // (not the editedName-preferring projection title) (issue #52).
         if (result.videoPath === undefined) {
@@ -390,13 +382,14 @@ export const getThumbCommand = async (
     try {
         // Force a server round-trip so the file resolves against current state
         // (issue #36 amendment, issue #52).
-        const file = await freshFile(lib, fileID);
+        const resolved = await freshFile(lib, fileID);
-        if (!file) {
+        if (!resolved) {
             ctx.log("fatal", `File ${fileID} not found`);
             return 1;
         }
+        const { photo, file } = resolved;
 
-        const result = await lib.thumbnailOf(file);
+        const result = await photo.thumbnail();
         // Default name is thumb_<file's own title>, as the pre-library CLI
         // used (not the projection title) (issue #52).
         const outPath = opts.out ?? thumbnailName(file);
@@ -573,13 +566,10 @@ export const fixMissingThumbnailsCommand = async (
     }
     const lib = await openReadLibrary(ctx, client);
     try {
-        let found: Map<number, { file: EnteFile; collection: Collection }>;
+        // Refresh first so files added since the cache was written are found;
-        if (fileIDs.length > 0) {
+        // a failed refresh throws.
-            found = await freshFilesByID(lib, fileIDs);
+        await lib.fresh();
-        } else {
+        if (fileIDs.length === 0) {
-            // Refresh first so files added since the cache was written are
-            // checked; a failed refresh throws.
-            await lib.fresh();
             ctx.log("info", "Scanning for missing thumbnails...");
             const missing = await listMissingThumbnails(
                 lib,
@@ -592,14 +582,12 @@ export const fixMissingThumbnailsCommand = async (
                 return 0;
             }
             ctx.log("info", `Found ${fileIDs.length} file(s) to fix.`);
-            found = cachedFilesByID(lib, fileIDs);
         }
 
         const results = await fixMissingThumbnails(
             lib,
             client,
             fileIDs,
-            found,
             (msg, level) => ctx.log(level ?? "info", msg),
         );
 

src/cli-log.ts

@@ -1,18 +1,15 @@
 // The CLI's log level, and the one function the commands write stderr lines
 // through.
 //
-// Every line quak logs goes to stderr, at one of six levels, from the
+// Every line quak prints on stderr is a log line at one of six levels, from
-// quietest: fatal, error, warning, info, debug, trace. A run prints the lines
+// the quietest: fatal, error, warning, info, debug, trace. A run prints the
-// at its own level and at every quieter one. Its level is info unless `-v` or
+// lines at its own level and at every quieter one. Its level is info unless
-// `-q` asks for another (see "What quak prints" in the README). A command that
+// `-v` or `-q` asks for another (see "CLI surface" in the README). Every
-// runs and exits non-zero logs a fatal line saying why, so even `-q`, which
+// command that exits non-zero logs a fatal line saying why, so even `-q`, which
-// prints only fatal lines, never leaves such an exit unexplained. An error
+// prints only fatal lines, never leaves a non-zero exit unexplained. An error
 // line reports something that failed, such as a file or an album, and a
 // warning something that did not fail but needs the user's attention. stdout
-// carries a command's output and the prompts of `login`, and is not a log.
+// carries only a command's output and is not a log.
-// Two things on stderr are not log lines: the `error:` message for a
-// command line that quak cannot parse, and the usage printed when no command
-// is given.
 //
 // The library itself writes no lines: its layers report through callbacks, and
 // the CLI logs what they report. `logRequests` gives the API client options

src/cli-read.ts

@@ -2,40 +2,42 @@
 //
 // `collections`, `files`, `get`, and `get-thumb` must answer for
 // the account's state at the moment the command runs, not for whatever the
-// local cache last happened to hold (owner amendment, issue #36). `get`,
+// local cache last happened to hold (owner amendment, issue #36). `files`
-// `get-thumb` and `files` without a collection ID therefore force a server
+// without a collection ID, `get` and `get-thumb` therefore force a server
-// round-trip through `Library.fresh()` and only then read, so a change made
+// round-trip through `Library.fresh()`, and only then read. A collection, file,
-// elsewhere is visible. `collections` asks the library for the collection list
+// or metadata change made elsewhere is then visible. `collections` asks the
-// alone through `Library.freshCollections()` (issue #210): a refresh would also
+// library for the collection list alone through `Library.freshCollections()`
-// fetch the files of every collection, which on a large account takes long and
+// (issue #210). A refresh would also fetch the files of every collection, which
-// which `collections` does not print. `files` with a collection ID reads that
+// `collections` does not print, and on a large account that takes long.
-// list too, then that one collection's files through `Library.freshFiles()`
+// `files COLLECTION` reads the collection list the same way, for the same
-// (issue #217). On an empty cache, `get` and `get-thumb` read that list, then
+// reason. It then brings that one collection up to date through
-// one collection's files at a time, until they find the file (issue #222).
+// `Library.refreshCollection()` (issue #217), and fetches no other collection's
-// `helper fix-missing-thumbnails --file` does the same until it finds every
+// files.
-// file it names (issue #223).
 //
-// `collections` and `files` with a collection ID list in the server's order.
+// `collections` lists in the server's order and `files` in the library's own
-// `files` without one lists in the library's own enumeration order
+// enumeration order (`listFiles()`), the orders the pre-library CLI printed,
-// (`listFiles()`). Neither command uses the `albums`/`photos` projection's
+// rather than the `albums`/`photos` projection's newest-first order, which
-// newest-first order, which re-sorts the rows. The field values still come
+// re-sorts the rows. The field values still come from each record's raw
-// from each record's raw metadata via `cli-output.ts`. `files` without a
+// metadata via `cli-output.ts`. `files` without a collection ID lists every
-// collection ID lists every file in the account once (issue #211): the library
+// file in the account once (issue #211): the library keeps a record of a file
-// keeps a record of a file for each collection that holds it, and only the
+// for each collection that holds it, and only the first collection's record is
-// first collection's record is listed.
+// listed.
 
 import type { Collection, EnteFile } from "./model/types.js";
+import type { Photo, PhotosAPI } from "./library/index.js";
 
 // The slice of `Library` these helpers read. `Library` satisfies it
-// structurally; a test can drive them with a stand-in that records each fresh
+// structurally. A test can drive them with a stand-in that records the
-// read and serves records in a known enumeration order.
+// `fresh()`, `freshCollections()` and `refreshCollection()` calls. The
+// stand-in serves records in a known enumeration order.
 export interface FreshReadLibrary {
     fresh(): Promise<unknown>;
     freshCollections(): Promise<Collection[]>;
-    freshFiles(collection: Collection): Promise<EnteFile[]>;
+    refreshCollection(collection: Collection): Promise<void>;
     listCollections(): Collection[];
     listFiles(collectionID: number): EnteFile[];
     getFileByID(fileID: number): EnteFile | undefined;
+    photos: Pick<PhotosAPI, "byID">;
 }
 
 // Every live collection, current as of one collections request, in the
@@ -44,11 +46,12 @@ export const freshCollections = (
     lib: FreshReadLibrary,
 ): Promise<Collection[]> => lib.freshCollections();
 
-// The files of one collection, as the server holds them now, in the server's
+// The files of one collection, in enumeration order, current as of one
-// order. `undefined` (not an empty list) when the collection does not exist, so
+// collections request and the requests that bring that collection alone up to
+// date. `undefined` (not an empty list) when the collection does not exist, so
 // the caller can tell "no such collection" from "an empty collection". Without
-// a collection ID, every file in the account as of a forced refresh, each once,
+// a collection ID, every file in the account, current as of a forced refresh.
-// as held by the first collection that holds it.
+// Each file is listed once, as held by the first collection that holds it.
 export const freshFiles = async (
     lib: FreshReadLibrary,
     collectionID?: number,
@@ -56,7 +59,9 @@ export const freshFiles = async (
     if (collectionID !== undefined) {
         const collections = await lib.freshCollections();
         const collection = collections.find((c) => c.id === collectionID);
-        return collection ? lib.freshFiles(collection) : undefined;
+        if (!collection) return undefined;
+        await lib.refreshCollection(collection);
+        return lib.listFiles(collectionID);
     }
     await lib.fresh();
     const listed = new Set<number>();
@@ -71,80 +76,16 @@ export const freshFiles = async (
     return files;
 };
 
-// One file as the server holds it now, or `undefined` when no collection holds
+// One file, current as of a forced refresh, resolved to both its content
-// it. When the cache holds the account's collections, it refreshes and looks
+// handle (`Photo`, for fetching bytes) and its raw record (`EnteFile`, for the
-// the file up in the cache. On an empty cache, it reads the collection list,
+// default output name and field values). `undefined` when the file is unknown.
-// then one collection's files at a time, and stops at the first collection
-// that holds the file. That search stores nothing.
 export const freshFile = async (
     lib: FreshReadLibrary,
     fileID: number,
-): Promise<EnteFile | undefined> => {
+): Promise<{ photo: Photo; file: EnteFile } | undefined> => {
-    // Only a refresh stores collections, so a cache holding any holds them all.
+    await lib.fresh();
-    if (lib.listCollections().length > 0) {
+    const photo = lib.photos.byID({ fileID });
-        await lib.fresh();
+    const file = lib.getFileByID(fileID);
-        return lib.getFileByID(fileID);
+    if (!photo || !file) return undefined;
-    }
+    return { photo, file };
-    for (const collection of await lib.freshCollections()) {
-        const files = await lib.freshFiles(collection);
-        const file = files.find((f) => f.id === fileID);
-        if (file) return file;
-    }
-    return undefined;
-};
-
-// The most recently updated collection first, and the higher ID first on a
-// tie. `Library.albums.list()` lists albums in this order.
-const byNewestCollection = (a: Collection, b: Collection): number =>
-    b.updationTime - a.updationTime || b.id - a.id;
-
-// Each file named by `fileIDs` that the cache holds, mapped to its record and
-// the collection it was found in. Collections are searched in
-// `byNewestCollection` order, and a file is found in the first that holds it.
-// So `helper fix-missing-thumbnails` names a file after the same album as
-// `helper list-missing-thumbnails`. It reads the cache alone, so the caller
-// refreshes first.
-export const cachedFilesByID = (
-    lib: FreshReadLibrary,
-    fileIDs: number[],
-): Map<number, { file: EnteFile; collection: Collection }> => {
-    const wanted = new Set(fileIDs);
-    const found = new Map<number, { file: EnteFile; collection: Collection }>();
-    const collections = [...lib.listCollections()].sort(byNewestCollection);
-    for (const collection of collections) {
-        for (const file of lib.listFiles(collection.id)) {
-            if (wanted.has(file.id) && !found.has(file.id)) {
-                found.set(file.id, { file, collection });
-            }
-        }
-    }
-    return found;
-};
-
-// Each file named by `fileIDs` as the server holds it now, mapped to its record
-// and collection as `cachedFilesByID` maps it. When the cache holds the
-// account's collections, it refreshes and looks the files up in the cache. On
-// an empty cache, it reads the collection list, then one collection's files at
-// a time, and stops once it has found every file. That search stores nothing.
-export const freshFilesByID = async (
-    lib: FreshReadLibrary,
-    fileIDs: number[],
-): Promise<Map<number, { file: EnteFile; collection: Collection }>> => {
-    // Only a refresh stores collections, so a cache holding any holds them all.
-    if (lib.listCollections().length > 0) {
-        await lib.fresh();
-        return cachedFilesByID(lib, fileIDs);
-    }
-    const wanted = new Set(fileIDs);
-    const found = new Map<number, { file: EnteFile; collection: Collection }>();
-    const collections = await lib.freshCollections();
-    for (const collection of [...collections].sort(byNewestCollection)) {
-        if (found.size === wanted.size) break;
-        for (const file of await lib.freshFiles(collection)) {
-            if (wanted.has(file.id) && !found.has(file.id)) {
-                found.set(file.id, { file, collection });
-            }
-        }
-    }
-    return found;
 };

src/library/content.ts

@@ -581,34 +581,14 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         fileID: number,
         opts?: ContentOptions,
     ): Promise<ContentResult> {
-        const file = this.fileOrThrow(fileID);
+        return this.get(fileID, "original", "on-demand", opts?.onProgress);
-        return this.get(file, "original", "on-demand", opts?.onProgress);
     }
 
     async thumbnail(
         fileID: number,
         opts?: ContentOptions,
     ): Promise<ContentResult> {
-        const file = this.fileOrThrow(fileID);
+        return this.get(fileID, "thumbnail", "on-demand", opts?.onProgress);
-        return this.get(file, "thumbnail", "on-demand", opts?.onProgress);
-    }
-
-    // The original of `file`, fetched and cached as `original` does. `file` is
-    // used as given, so `getFile` need not know it.
-    async originalOf(
-        file: EnteFile,
-        opts?: ContentOptions,
-    ): Promise<ContentResult> {
-        return this.get(file, "original", "on-demand", opts?.onProgress);
-    }
-
-    // The thumbnail of `file`, fetched and cached as `thumbnail` does. `file`
-    // is used as given, so `getFile` need not know it.
-    async thumbnailOf(
-        file: EnteFile,
-        opts?: ContentOptions,
-    ): Promise<ContentResult> {
-        return this.get(file, "thumbnail", "on-demand", opts?.onProgress);
     }
 
     // Put the original at the save path of `file` under the download directory
@@ -635,7 +615,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         destination: string,
     ): Promise<ContentResult> {
         const result = await this.acquire(
-            this.fileOrThrow(fileID),
+            fileID,
             "original",
             "on-demand",
             undefined,
@@ -658,7 +638,8 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         fileID: number,
         destination: string,
     ): Promise<{ path: string; videoPath?: string }> {
-        const file = this.fileOrThrow(fileID);
+        const file = this.getFile(fileID);
+        if (!file) throw new Error(`content cache: unknown file ${fileID}`);
         return this.pools.content.run(
             async () => {
                 const fetched = await this.fetchInto(
@@ -740,8 +721,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         onProgress: ((event: EnsureEvent) => void) | undefined,
     ): Promise<EnsureResult> {
         try {
-            const file = this.fileOrThrow(fileID);
+            const result = await this.acquire(fileID, kind, priority, signal);
-            const result = await this.acquire(file, kind, priority, signal);
             const status = result.cached ? "skipped" : "done";
             onProgress?.({ fileID, status, path: result.path });
             return { fileID, path: result.path };
@@ -756,16 +736,8 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         }
     }
 
-    // The record `getFile` gives for `fileID`. It throws for a file `getFile`
-    // does not know.
-    private fileOrThrow(fileID: number): EnteFile {
-        const file = this.getFile(fileID);
-        if (!file) throw new Error(`content cache: unknown file ${fileID}`);
-        return file;
-    }
-
     private async get(
-        file: EnteFile,
+        fileID: number,
         kind: Kind,
         priority: Priority,
         onProgress: ((event: ContentEvent) => void) | undefined,
@@ -773,7 +745,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         const onByte: ProgressCallback | undefined = onProgress
             ? (bytesDone) => onProgress({ status: "downloading", bytesDone })
             : undefined;
-        const result = await this.acquire(file, kind, priority, undefined, {
+        const result = await this.acquire(fileID, kind, priority, undefined, {
             onByte,
         });
         onProgress?.(
@@ -795,7 +767,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
     // photo's original is present only with its video, and is returned with
     // it.
     private async acquire(
-        file: EnteFile,
+        fileID: number,
         kind: Kind,
         priority: Priority,
         signal: AbortSignal | undefined,
@@ -806,7 +778,8 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         videoPath?: string;
         cached: boolean;
     }> {
-        const fileID = file.id;
+        const file = this.getFile(fileID);
+        if (!file) throw new Error(`content cache: unknown file ${fileID}`);
         const isLivePhoto =
             kind === "original" && file.metadata.fileType === "livePhoto";
 

src/library/index.ts

@@ -1,15 +1,16 @@
 // The library surface over the local cache.
 //
 // `Library.open()` loads the on-disk metadata store (issue #41), then starts
-// the refresh loop. When the cache loaded empty it awaits the first refresh,
+// the refresh loop. When no refresh has stored a collection yet (the stored
-// so the library never opens onto an empty store it could have filled; when an
+// cursor is 0), `open()` awaits the first refresh. The cache is then empty, or
-// existing copy loaded, that first refresh runs in the background and `open()`
+// holds only what `refreshCollection()` stored, and the library must not serve
-// returns as soon as the cached data is ready to serve — a slow or unreachable
+// it as the whole account. Otherwise that first refresh runs in the
-// server no longer stalls opening. A background timer then refreshes every
+// background. `open()` returns as soon as the cached data is ready to serve,
-// `refreshIntervalSeconds`. Every default read is answered from RAM — no
+// so a slow or unreachable server does not stall opening. A background timer
-// default read touches the network. There is deliberately no `sync()`, no
+// then refreshes every `refreshIntervalSeconds`. Every default read is
-// `refresh()`, no `serverReachable` flag, and no "before each read" mode
+// answered from RAM — no default read touches the network. There is
-// (design #36).
+// deliberately no `sync()`, no `refresh()`, no `serverReachable` flag, and no
+// "before each read" mode (design #36).
 //
 // `fresh()` is the one exception (issue #75, an owner amendment to #36): it
 // forces a refresh, awaits it, and only then hands back the read namespaces, so
@@ -20,9 +21,11 @@
 //
 // `freshCollections()` is the fresh read of the collection list alone (issue
 // #210): one collections request, no collection's files, and nothing stored,
-// for a caller such as `quak collections` that needs no more. `freshFiles()`
+// for a caller such as `quak collections` that needs no more.
-// is the fresh read of one collection's files, and stores nothing either
+// `refreshCollection()` brings one collection from that list up to date in the
-// (issue #217). A caller that asks for each refresh it needs can also open the
+// store (issue #217). It fetches that collection's files alone, as a refresh
+// would. `quak files COLLECTION` uses it. It never runs at the same time as a
+// refresh. A caller that asks for each refresh it needs can also open the
 // library with `refreshOnOpen: false`, so that `open()` starts none.
 //
 // A refresh reports through `onProgress` each collection whose files it has
@@ -32,7 +35,7 @@
 // A refresh stages all of its network work first and only mutates the store
 // once every fetch has succeeded. A refresh that fails partway therefore never
 // becomes visible to reads: the last good snapshot stays in place, and the
-// failure surfaces through `onProgress` and `status()` instead. A commit that
+// failure surfaces through `onProgress` and `status()` instead. A refresh that
 // mutates RAM but then fails to persist keeps `status().lastError` set and the
 // store marked unsaved until a later save actually lands, so a stuck disk is
 // never masked by a subsequent empty refresh.
@@ -66,8 +69,6 @@ import {
     ContentCache,
     savePath,
     storedAtSavePath,
-    type ContentOptions,
-    type ContentResult,
     type ContentSource,
     type ThumbnailsAPI,
     type EnsureOptions,
@@ -309,12 +310,16 @@ export class Library {
     private readonly precache?: Precache;
 
     private timer?: ReturnType<typeof setTimeout>;
-    // The in-flight refresh cycle, or undefined when none runs. One slot serves
+    // The in-flight refresh cycle or `refreshCollection()`, or undefined when
-    // both paths: the background loop skips when it is set, and a fresh read
+    // neither runs. One slot holds both, so they never run at the same time.
-    // (issue #75) coalesces onto it or starts one. The promise carries the
+    // The background loop skips while it is set. A fresh read (issue #75)
-    // cycle's real outcome (it rejects on failure); the background loop ignores
+    // coalesces onto a running refresh. It waits for a running
-    // that, a fresh read propagates it.
+    // `refreshCollection()` to end, and then joins or starts a refresh. The
+    // promise carries the real outcome (it rejects on failure); the background
+    // loop ignores that, a fresh read propagates it.
     private cycle?: Promise<void>;
+    // Whether `cycle` is a refresh cycle rather than a `refreshCollection()`.
+    private cycleIsRefresh = false;
     // Guards the ML fetch pass so a slow backfill never runs twice at once; a
     // refresh whose pass is still running kicks nothing new. Holds the running
     // pass, so `close()` and `backup()` can wait for it. It rejects when the
@@ -396,14 +401,16 @@ export class Library {
         this.mldata = makeMLDataAPI(() => this.mlStore);
     }
 
-    // Load the cache and start the refresh loop. With an empty cache the first
+    // Load the cache and start the refresh loop. When the stored cursor is 0,
-    // refresh is awaited, so `open()` resolves onto populated data whenever the
+    // the first refresh is awaited. Such a cache is empty, or holds only what
-    // server is reachable; that awaited refresh may still fail, and the library
+    // `refreshCollection()` stored. `open()` then resolves onto the whole
-    // then opens empty with the failure recorded in `status()`. With an
+    // account whenever the server is reachable. If that refresh fails, the
-    // existing cache the first refresh runs in the background and `open()`
+    // library opens onto what the cache holds, with the failure recorded in
-    // returns as soon as the cached data is ready — an unreachable server does
+    // `status()`. With any other cache the first refresh runs in the
-    // not block opening. With `refreshOnOpen: false` there is no first
+    // background, and `open()` returns as soon as the cached data is ready.
-    // refresh: `open()` returns onto the cache as it is.
+    // An unreachable server then does not block opening. With
+    // `refreshOnOpen: false` there is no first refresh: `open()` returns onto
+    // the cache as it is.
     static async open(opts: LibraryOptions): Promise<Library> {
         const { userID } = opts.client.whoami();
         const cacheDirectory =
@@ -509,13 +516,17 @@ export class Library {
         if (opts.refreshOnOpen === false) {
             // The caller asks for each refresh it needs.
             lib.scheduleNext();
-        } else if (store.loadedFromDisk) {
+        } else if (store.collectionsSinceTime > 0) {
-            // An existing copy already answers reads; refresh in the background
+            // The stored cursor has moved, so a refresh has stored the
-            // and start the interval once that first cycle settles.
+            // account's collections and the cache's copy answers reads.
+            // Refresh in the background, and start the interval once that
+            // first cycle settles.
             void lib.runRefresh().then(() => lib.scheduleNext());
         } else {
-            // Nothing was cached: wait for the first refresh to fill the store
+            // The stored cursor is still 0, so no refresh has stored any
-            // (or fail) rather than resolve onto an empty library.
+            // collection: the cache is empty, or holds only what
+            // `refreshCollection()` stored. Wait for the first refresh to fill
+            // the store (or fail) rather than resolve onto part of the account.
             await lib.runRefresh();
             lib.scheduleNext();
         }
@@ -628,52 +639,19 @@ export class Library {
         return collections;
     }
 
-    // The fresh read of one collection's files (issue #217): every live file
+    // Bring one collection up to date in the store (issue #217), given its
-    // in it that decrypts, in the server's order. `collection` is a record
+    // record from `freshCollections()`. It does for that collection what a
-    // `freshCollections()` returned, whose key decrypts the files. It fetches
+    // refresh does for a changed one. It fetches no other collection's files.
-    // no other collection's files. Like `freshCollections()`, it stores
+    // It runs in the same slot as a refresh, so the two never run at the same
-    // nothing. It returns each file once, as the latest copy the server sent,
+    // time. It first waits for a running refresh or `refreshCollection()` to
-    // where the first copy was.
+    // end. A fresh read or `backup()` that starts meanwhile waits for it, and
-    async freshFiles(collection: Collection): Promise<EnteFile[]> {
+    // the background loop skips its turn.
+    async refreshCollection(collection: Collection): Promise<void> {
+        while (this.cycle) await this.cycle.catch(() => {});
         if (this.closed) throw new Error("the library is closed");
-        const { files } = await this.client.filesSince({
+        await this.startCycle(false, () =>
-            collectionID: collection.id,
+            this.refreshCollectionOnce(collection),
-            collectionKey: collection.key,
+        );
-            sinceTime: 0,
-        });
-        const byID = new Map<number, EnteFile>();
-        for (const file of files) byID.set(file.id, file);
-        return [...byID.values()];
-    }
-
-    // The original of `file`, fetched and cached as `Photo.original()` does.
-    // `file` is used as given, so the store need not hold it. On an empty
-    // cache, `quak get` passes a file that `freshFiles()` returned.
-    async originalOf(
-        file: EnteFile,
-        opts?: ContentOptions,
-    ): Promise<ContentResult> {
-        if (!this.cache) {
-            throw new Error(
-                "originalOf requires a library opened with a content cache",
-            );
-        }
-        return this.cache.originalOf(file, opts);
-    }
-
-    // The thumbnail of `file`, fetched and cached as `Photo.thumbnail()` does.
-    // `file` is used as given, so the store need not hold it. On an empty
-    // cache, `quak get-thumb` passes a file that `freshFiles()` returned.
-    async thumbnailOf(
-        file: EnteFile,
-        opts?: ContentOptions,
-    ): Promise<ContentResult> {
-        if (!this.cache) {
-            throw new Error(
-                "thumbnailOf requires a library opened with a content cache",
-            );
-        }
-        return this.cache.thumbnailOf(file, opts);
     }
 
     // Back up every in-scope file to `opts.downloadDirectory`, or else the
@@ -725,11 +703,13 @@ export class Library {
         );
     }
 
-    // Stop the background timer. Idempotent. An in-flight refresh is left to
+    // Stop the background timer. Idempotent. A refresh or `refreshCollection()`
-    // finish; it will not schedule another cycle once closed. The returned
+    // in flight is left to finish, and nothing starts after it. The returned
-    // promise resolves once that refresh (including its cache write), the ML
+    // promise resolves once all of these have finished:
-    // fetch pass and the precache fetches already running have all finished,
+    //  - that refresh or `refreshCollection()`, with its cache write;
-    // so a caller can then remove the cache directory. A refresh failure is
+    //  - the ML fetch pass;
+    //  - the precache fetches already running.
+    // A caller can then remove the cache directory. A refresh failure is
     // reported through `status()`, not thrown here.
     async close(): Promise<void> {
         this.closed = true;
@@ -753,33 +733,40 @@ export class Library {
         this.timer.unref?.();
     }
 
-    // The background loop's refresh: run a cycle unless one is already in flight
+    // The background loop's refresh: run a cycle unless a refresh or
-    // (or the library is closed), and never let a failure escape — the
+    // `refreshCollection()` is in flight (or the library is closed), and never
-    // background path reports errors through `status()`/`onProgress`, it does
+    // let a failure escape. The background path reports errors through
-    // not throw. Resolves once the cycle it started (or skipped past) settles.
+    // `status()`/`onProgress`; it does not throw. Resolves once the cycle it
+    // started (or skipped past) settles.
     private runRefresh(): Promise<void> {
         if (this.closed || this.cycle) return Promise.resolve();
-        return this.startCycle().catch(() => {});
+        return this.startCycle(true, () => this.refreshCycle()).catch(() => {});
     }
 
     // A fresh read's refresh (issue #75): force a cycle and await it, rejecting
     // if it fails. Concurrent fresh reads coalesce onto the one in-flight cycle
     // — the background loop's included — so they never fan out into redundant
-    // server round-trips.
+    // server round-trips. A running `refreshCollection()` is waited for first.
-    private refreshNow(): Promise<void> {
+    private async refreshNow(): Promise<void> {
-        if (this.closed) {
+        while (this.cycle && !this.cycleIsRefresh) {
-            return Promise.reject(new Error("the library is closed"));
+            await this.cycle.catch(() => {});
         }
-        return this.cycle ?? this.startCycle();
+        if (this.closed) throw new Error("the library is closed");
+        return this.cycle ?? this.startCycle(true, () => this.refreshCycle());
     }
 
-    // Start one refresh cycle and record it as the in-flight cycle so every
+    // Start `work` and record it in the in-flight slot, so every other caller
-    // caller coalesces onto it. The returned promise carries the cycle's real
+    // sees it. `isRefresh` says whether it is a refresh cycle, which a fresh
-    // outcome; each caller attaches the handling its own path needs, and the
+    // read can coalesce onto. The returned promise carries the real outcome.
-    // slot is cleared once the cycle settles.
+    // Each caller attaches the handling its own path needs, and the slot is
-    private startCycle(): Promise<void> {
+    // cleared once the work settles.
-        const cycle = this.refreshCycle();
+    private startCycle(
+        isRefresh: boolean,
+        work: () => Promise<void>,
+    ): Promise<void> {
+        const cycle = work();
         this.cycle = cycle;
+        this.cycleIsRefresh = isRefresh;
         void cycle.then(
             () => {
                 if (this.cycle === cycle) this.cycle = undefined;
@@ -817,10 +804,7 @@ export class Library {
 
     // Fetch every change since the stored cursor, then commit. All network
     // reads happen before any store mutation, so a fetch that throws leaves the
-    // store untouched and the previous snapshot intact. A collection or file
+    // store untouched and the previous snapshot intact.
-    // that does not decrypt is left out instead, as if deleted, with the reason
-    // kept in the store: the cursor moves past it, so no later refresh sees it
-    // again until the server's copy changes.
     private async refreshOnce(): Promise<void> {
         const page = await this.client.collectionsSince({
             sinceTime: this.store.collectionsSinceTime,
@@ -853,6 +837,51 @@ export class Library {
             });
         }
 
+        await this.commit(page, filePages);
+    }
+
+    // `refreshCollection()`'s work, run in the slot: fetch the files of one
+    // collection changed since the stored copy, then commit.
+    private async refreshCollectionOnce(collection: Collection): Promise<void> {
+        const known = this.store.getCollection(collection.id);
+        // The stored copy is as new or newer, so there is nothing to fetch.
+        if (known && collection.updationTime <= known.updationTime) return;
+        // No copy is stored, yet the stored cursor has reached this
+        // `updationTime`. A refresh has then seen this version or a later one,
+        // and dropped the collection: it was deleted, or did not decrypt.
+        // Storing it would bring it back for good, since no later refresh sees
+        // it again.
+        if (
+            !known &&
+            collection.updationTime <= this.store.collectionsSinceTime
+        ) {
+            return;
+        }
+        const filePage = await this.client.filesSince({
+            collectionID: collection.id,
+            collectionKey: collection.key,
+            sinceTime: known ? known.updationTime : 0,
+        });
+        // The stored cursor stays where it is, so the next refresh still
+        // fetches the files of every other collection that changed.
+        await this.commit(
+            {
+                collections: [collection],
+                deleted: [],
+                cursor: this.store.collectionsSinceTime,
+            },
+            [{ collectionID: collection.id, page: filePage }],
+        );
+    }
+
+    // Commit a collections page and the file pages fetched for its collections
+    // to the store, then save. A collection or file that does not decrypt is
+    // left out, as if deleted. The reason is kept in the store. No later
+    // refresh sees it again until the server's copy changes.
+    private async commit(
+        page: CollectionsPage,
+        filePages: { collectionID: number; page: FilesPage }[],
+    ): Promise<void> {
         // Network work done; commit to the store and persist only if something
         // actually changed.
         let changed = false;
@@ -915,10 +944,10 @@ export class Library {
         if (changed) this.unsaved = true;
 
         // Reproject and notify subscribers of the delta. This tracks RAM (what
-        // reads see), so it fires whether or not the save below succeeds; a
+        // reads see), so it fires whether or not the save below succeeds. A
-        // save failure surfaces separately through `status().lastError`.
+        // failed save rejects the caller, and a refresh also records it in
-        // `lastRecords` advances every changed refresh so the next diff is
+        // `status().lastError`. `lastRecords` advances every changed refresh so
-        // against current state.
+        // the next diff is against current state.
         if (changed) {
             const next = this.deriveNow();
             if (this.subscribers.size > 0) {
@@ -935,10 +964,9 @@ export class Library {
 
         // Persist whenever RAM holds changes disk has not accepted — including
         // changes an earlier cycle staged whose save failed. `unsaved` clears
-        // only once a save lands, so a save failure both stays visible through
+        // only once a save lands, so a later commit retries a failed save even
-        // `status().lastError` (the throw below records it) and keeps being
+        // when it changes nothing. A failed save rejects the caller, and a
-        // retried, instead of a later empty refresh silently clearing it while
+        // refresh also records it in `status().lastError`.
-        // the on-disk cache is still behind RAM.
         if (this.unsaved) {
             await this.store.save();
             this.unsaved = false;

src/library/store.ts

@@ -70,12 +70,6 @@ export class MetadataStore {
     userID = 0;
     collectionsSinceTime: Microseconds = 0;
 
-    // True when `load` populated this store from a valid existing file; false
-    // on a first run or a missing/corrupt/wrong-version file that loaded empty.
-    // `Library.open` reads it to decide whether the first refresh may run in
-    // the background (an existing copy already serves reads) or must be awaited.
-    loadedFromDisk = false;
-
     private readonly collections = new Map<number, Collection>();
     private readonly files = new Map<string, EnteFile>();
     private undecryptable: Undecryptable[] = [];
@@ -102,7 +96,6 @@ export class MetadataStore {
             if (parsed.schemaVersion !== METADATA_SCHEMA_VERSION) {
                 return store;
             }
-            store.loadedFromDisk = true;
             store.userID = parsed.userID ?? 0;
             store.collectionsSinceTime = parsed.collectionsSinceTime ?? 0;
             for (const stored of parsed.collections ?? []) {

src/thumbnails.ts

@@ -5,7 +5,7 @@ import type { Client } from "./client.js";
 import type { Library } from "./library/index.js";
 import { ApiError } from "./api/client.js";
 import { encryptBlob, toBase64 } from "./crypto/index.js";
-import type { Collection, EnteFile } from "./model/types.js";
+import type { EnteFile } from "./model/types.js";
 
 // The server refuses a thumbnail larger than the one it already records for the
 // file (`thumbnail.size`, the encrypted size), so these encodings are tried
@@ -226,18 +226,16 @@ const encryptThumbnailWithin = (
     return undefined;
 };
 
-// Regenerate and upload a thumbnail for each requested file. `found` holds the
+// Regenerate and upload a thumbnail for each requested file. Originals are read
-// record of each requested file that a collection holds, and the collection it
+// through the library's content cache (`photo.original()`); the generated
-// was found in. Each original is read through the library's content cache, by
+// thumbnail is JPEG-encoded, encrypted under the file's own key, and registered
-// `lib.originalOf(file)`. The new thumbnail is a JPEG, encrypted under the
+// with the server — the encrypt-and-upload path is unchanged. Each file is
-// file's own key. It is uploaded, then registered with the server. Each file is
 // resolved to one outcome (fixed / skipped / failed) and a failure on one file
 // never stops the others.
 export const fixMissingThumbnails = async (
     lib: Library,
     client: Client,
     fileIDs: number[],
-    found: Map<number, { file: EnteFile; collection: Collection }>,
     onProgress?: ProgressCallback,
 ): Promise<ThumbnailFixResult[]> => {
     const log = onProgress ?? (() => {});
@@ -245,8 +243,31 @@ export const fixMissingThumbnails = async (
     const api = client.getApiClient();
     const { userID } = client.whoami();
 
+    // Resolve each requested fileID to its file record and owning album by
+    // enumerating the library, each file taken from the first album that holds
+    // it. The raw `EnteFile` carries the per-file key the thumbnail is
+    // encrypted under, which the projected records deliberately do not.
+    const wanted = new Set(fileIDs);
+    const fileMap = new Map<
+        number,
+        { file: EnteFile; collectionName: string }
+    >();
+    for (const album of lib.albums.list()) {
+        for (const photo of album.photos.list()) {
+            if (!wanted.has(photo.fileID) || fileMap.has(photo.fileID))
+                continue;
+            const file = lib.getFile(album.collectionID, photo.fileID);
+            if (file) {
+                fileMap.set(photo.fileID, {
+                    file,
+                    collectionName: album.name,
+                });
+            }
+        }
+    }
+
     for (const fileID of fileIDs) {
-        const entry = found.get(fileID);
+        const entry = fileMap.get(fileID);
         if (!entry) {
             results.push({
                 fileID,
@@ -258,8 +279,7 @@ export const fixMissingThumbnails = async (
             continue;
         }
 
-        const { file } = entry;
+        const { file, collectionName } = entry;
-        const collectionName = entry.collection.name;
         const title = file.metadata.title;
 
         const skipReason = reasonToSkip(file, userID);
@@ -277,10 +297,15 @@ export const fixMissingThumbnails = async (
         const maxSize = file.thumbnail.size!;
 
         try {
+            const photo = lib.photos.byID({ fileID });
+            if (!photo) {
+                throw new Error("file not present in the library cache");
+            }
+
             log(
                 `[${collectionName}] Downloading ${title} for thumbnail generation...`,
             );
-            const { path } = await lib.originalOf(file);
+            const { path } = await photo.original();
             const fileBytes = new Uint8Array(readFileSync(path));
 
             if (!isJpeg(fileBytes)) {

test/cli/backup-end-to-end.test.ts

@@ -418,7 +418,7 @@ describe("the first backup of the account", () => {
         expect(downloads.sort()).toEqual([100, 200, 300]);
     });
 
-    // The tree under "The backup directory" in the README. Each original is at
+    // The tree "Backup layout" in the README describes. Each original is at
     // its save path, `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>`, with
     // the extension of its title; the live photo is its image and its video,
     // each with the extension it had inside the ZIP, and a `.livephoto.json`

test/cli/commands.test.ts

@@ -194,6 +194,9 @@ const FETCHED_BOTH_ALBUMS =
     "Fetched the files of 1 of 2 collections\n" +
     "Fetched the files of 2 of 2 collections\n";
 
+// The same for an account with one album.
+const FETCHED_ONE_ALBUM = "Fetched the files of 1 of 1 collections\n";
+
 // Collects everything written to it.
 class Output {
     text = "";
@@ -573,27 +576,27 @@ describe("get and get-thumb", () => {
         const out = join(root, "diagram.png");
         expect(await getCommand(context(), "200", { out })).toBe(0);
         expect(readFileSync(out)).toEqual(Buffer.alloc(7, 200));
-        expect(stderr.text).toBe(`7 bytes -> ${out}\n`);
+        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + `7 bytes -> ${out}\n`);
     });
 
     it("get-thumb finds a file in any album without --collection", async () => {
         const out = join(root, "thumb.jpg");
         expect(await getThumbCommand(context(), "200", { out })).toBe(0);
         expect(readFileSync(out)).toEqual(Buffer.alloc(3, 200));
-        expect(stderr.text).toBe(`3 bytes -> ${out}\n`);
+        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + `3 bytes -> ${out}\n`);
     });
 
     it("get exits 1 when no album has the file", async () => {
         const out = join(root, "x");
         expect(await getCommand(context(), "999", { out })).toBe(1);
-        expect(stderr.text).toBe("File 999 not found\n");
+        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + "File 999 not found\n");
         expect(existsSync(out)).toBe(false);
     });
 
     it("get-thumb exits 1 when no album has the file", async () => {
         const out = join(root, "x");
         expect(await getThumbCommand(context(), "999", { out })).toBe(1);
-        expect(stderr.text).toBe("File 999 not found\n");
+        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + "File 999 not found\n");
         expect(existsSync(out)).toBe(false);
     });
 });
@@ -641,7 +644,7 @@ describe("names from an album name or a title over 255 bytes", () => {
         expect(Buffer.byteLength(name)).toBe(255);
         expect(code).toBe(0);
         expect(readdirSync(dir)).toEqual([name]);
-        expect(stderr.text).toBe(`7 bytes -> ${name}\n`);
+        expect(stderr.text).toBe(FETCHED_ONE_ALBUM + `7 bytes -> ${name}\n`);
     });
 
     it("get-thumb cuts the title short so that thumb_ and it fit in 255 bytes, and exits 0", async () => {
@@ -653,7 +656,7 @@ describe("names from an album name or a title over 255 bytes", () => {
         expect(Buffer.byteLength(name)).toBe(255);
         expect(code).toBe(0);
         expect(readdirSync(dir)).toEqual([name]);
-        expect(stderr.text).toBe(`3 bytes -> ${name}\n`);
+        expect(stderr.text).toBe(FETCHED_ONE_ALBUM + `3 bytes -> ${name}\n`);
     });
 
     it("backup-metadata cuts the album name short so that its folder's name fits in 255 bytes, and exits 0", async () => {
@@ -742,7 +745,8 @@ describe("a live photo", () => {
             Buffer.from(VIDEO),
         );
         expect(stderr.text).toBe(
-            `${IMAGE.length} bytes -> IMG_0300.heic\n` +
+            FETCHED_ONE_ALBUM +
+                `${IMAGE.length} bytes -> IMG_0300.heic\n` +
                 `${VIDEO.length} bytes -> IMG_0300.mov\n`,
         );
     });
@@ -796,7 +800,8 @@ describe("a live photo", () => {
         expect(readFileSync(out)).toEqual(Buffer.from(IMAGE));
         expect(readFileSync(video)).toEqual(Buffer.from(VIDEO));
         expect(stderr.text).toBe(
-            `${IMAGE.length} bytes -> ${out}\n${VIDEO.length} bytes -> ${video}\n`,
+            FETCHED_ONE_ALBUM +
+                `${IMAGE.length} bytes -> ${out}\n${VIDEO.length} bytes -> ${video}\n`,
         );
     });
 
@@ -810,7 +815,8 @@ describe("a live photo", () => {
         expect(existsSync(out)).toBe(false);
         expect(existsSync(join(root, "photo.mov"))).toBe(false);
         expect(stderr.text).toBe(
-            `File 300 is a live photo, and its video would also be written to ${out}\n`,
+            FETCHED_ONE_ALBUM +
+                `File 300 is a live photo, and its video would also be written to ${out}\n`,
         );
     });
 
@@ -1353,23 +1359,6 @@ describe("helper fix-missing-thumbnails", () => {
         ).toBe(1);
         expect(stderr.text).toBe(last);
     });
-
-    // beach.jpg (100) is in Vacation (1) and Work (2), both updated at the same
-    // time. The fixer names a file after the first album holding it in the
-    // order of `Library.albums.list()`. On a tie, the album with the higher ID
-    // comes first, so beach.jpg is named after Work.
-    it("names a photo in two albums after the same album on an empty cache as on a cache that holds the account", async () => {
-        const ctx = context(fakeClient({ beachInWork: true }));
-        const opts = { file: ["100"], json: true };
-        const inWork = [{ fileID: 100, collection: "Work", status: "skipped" }];
-        expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0);
-        expect(JSON.parse(stdout.text)).toMatchObject(inWork);
-
-        expect(await filesCommand(ctx, undefined, {})).toBe(0);
-        stdout.text = "";
-        expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0);
-        expect(JSON.parse(stdout.text)).toMatchObject(inWork);
-    });
 });
 
 describe("backup-metadata --exif", () => {
@@ -1398,12 +1387,12 @@ describe("backup-metadata --exif", () => {
     });
 });
 
-// Each test first runs `files` so the cache holds the account as it was, then
+// Each test first runs `files 1` so the cache holds Vacation as it was, then
-// changes the server under it. (`collections` and `files` with a collection ID
+// adds a file to Vacation on the server. (`collections` would not do: it reads
-// would not do: they store nothing.)
+// the collection list alone and stores nothing.)
 describe("backup-metadata and the thumbnail helpers refresh first", () => {
     beforeEach(async () => {
-        expect(await filesCommand(context(), undefined, {})).toBe(0);
+        expect(await filesCommand(context(), "1", {})).toBe(0);
         stdout.text = "";
         stderr.text = "";
     });
@@ -1586,83 +1575,22 @@ describe("an ID given on the command line", () => {
     );
 });
 
-// An account with many albums (issues #210, #217 and #222). On an empty cache
+// An account with many albums (issue #210). On an empty cache `collections`
-// `collections`, `files` with a collection ID and `get` used to wait for a
+// used to wait for a refresh that fetched and decrypted the files of every
-// refresh that fetched and decrypted the files of every album, one album after
+// album, one album after another. It printed nothing until then, so on such an
-// another. So did `helper fix-missing-thumbnails --file` (issue #223). On such
+// account it seemed to hang. `files` with an album's ID did the same before it
-// an account they seemed to hang. The account here has 40 albums of one photo
+// listed that album (issue #217). The account here has 40 albums of one photo
-// each, encrypted as Ente's clients encrypt them, on the fake Ente server of
+// each, encrypted as Ente's clients encrypt them. The fake Ente server of
-// `test/ente-server.ts`. The tests record the path and query of each request
+// `test/ente-server.ts` serves them. A wrapper around it records the path and
-// quak sends. The session is the one logging in to that server saves.
+// query of each request quak sends. The session is the one logging in to that
+// server saves.
 describe("an account with many albums", () => {
     const ALBUMS = 40;
     const PASSWORD = "many-albums";
-    const requests: string[] = [];
+    const paths: string[] = [];
     let server: typeof globalThis.fetch;
     let snapshot: ClientSnapshot;
 
-    // What `files` without a collection ID prints on an empty cache: every
-    // album's photo on stdout, and on stderr each album whose files its
-    // refresh fetched.
-    let everyPhoto = "";
-    let progress = "";
-    for (let n = 1; n <= ALBUMS; n++) {
-        everyPhoto += `${n * 100}\timage\tphoto-${n}.jpg\n`;
-        progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`;
-    }
-
-    // The key of each album, by album ID.
-    const albumKeys = new Map<number, Uint8Array>();
-
-    // A photo in album `collectionID` as the server sends it, and its original
-    // encrypted as the server stores it. The original's bytes are the title.
-    // These tests fetch no thumbnail. The server records a thumbnail size, so
-    // `helper fix-missing-thumbnails` downloads the original before it skips
-    // the photo as not a JPEG.
-    const photoRow = (photo: {
-        id: number;
-        collectionID: number;
-        title: string;
-        updationTime: number;
-    }): { row: Record<string, unknown>; original: Uint8Array } => {
-        const photoKey = sodium.crypto_secretstream_xchacha20poly1305_keygen();
-        const photoKeyBox = encryptSecretbox(
-            photoKey,
-            albumKeys.get(photo.collectionID)!,
-        );
-        const metadata = encryptStreamBlob(
-            new TextEncoder().encode(
-                JSON.stringify({
-                    title: photo.title,
-                    fileType: 0,
-                    creationTime: 1000,
-                    modificationTime: 1000,
-                }),
-            ),
-            photoKey,
-        );
-        const original = encryptStreamBlob(
-            new TextEncoder().encode(photo.title),
-            photoKey,
-        );
-        const row = {
-            id: photo.id,
-            collectionID: photo.collectionID,
-            ownerID: USER_ID,
-            encryptedKey: toBase64(photoKeyBox.ciphertext),
-            keyDecryptionNonce: toBase64(photoKeyBox.nonce),
-            metadata: {
-                encryptedData: toBase64(metadata.ciphertext),
-                decryptionHeader: toBase64(metadata.header),
-            },
-            file: { decryptionHeader: toBase64(original.header) },
-            thumbnail: { decryptionHeader: "" },
-            info: { thumbSize: 1000 },
-            updationTime: photo.updationTime,
-        };
-        return { row, original: original.ciphertext };
-    };
-
     // Album n is named `Album n` and holds one photo, n * 100, titled
     // `photo-n.jpg`.
     beforeAll(async () => {
@@ -1670,10 +1598,8 @@ describe("an account with many albums", () => {
         const account = await buildAccount(PASSWORD);
         const collections: Record<string, unknown>[] = [];
         const filesByCollection: Record<number, Record<string, unknown>[]> = {};
-        const fileCiphertexts: Record<number, Uint8Array> = {};
         for (let id = 1; id <= ALBUMS; id++) {
             const albumKey = sodium.crypto_secretbox_keygen();
-            albumKeys.set(id, albumKey);
             const albumKeyBox = encryptSecretbox(albumKey, account.masterKey);
             const name = encryptSecretbox(
                 new TextEncoder().encode(`Album ${id}`),
@@ -1689,14 +1615,37 @@ describe("an account with many albums", () => {
                 type: "album",
                 updationTime: 1,
             });
-            const { row, original } = photoRow({
+            const photoKey =
-                id: id * 100,
+                sodium.crypto_secretstream_xchacha20poly1305_keygen();
-                collectionID: id,
+            const photoKeyBox = encryptSecretbox(photoKey, albumKey);
-                title: `photo-${id}.jpg`,
+            const metadata = encryptStreamBlob(
-                updationTime: 1,
+                new TextEncoder().encode(
-            });
+                    JSON.stringify({
-            filesByCollection[id] = [row];
+                        title: `photo-${id}.jpg`,
-            fileCiphertexts[id * 100] = original;
+                        fileType: 0,
+                        creationTime: 1000,
+                        modificationTime: 1000,
+                    }),
+                ),
+                photoKey,
+            );
+            filesByCollection[id] = [
+                {
+                    id: id * 100,
+                    collectionID: id,
+                    ownerID: USER_ID,
+                    encryptedKey: toBase64(photoKeyBox.ciphertext),
+                    keyDecryptionNonce: toBase64(photoKeyBox.nonce),
+                    metadata: {
+                        encryptedData: toBase64(metadata.ciphertext),
+                        decryptionHeader: toBase64(metadata.header),
+                    },
+                    // These tests fetch no original and no thumbnail.
+                    file: { decryptionHeader: "" },
+                    thumbnail: { decryptionHeader: "" },
+                    updationTime: 1,
+                },
+            ];
         }
         const ente = buildMetaFetch({
             verifier: account.verifier,
@@ -1706,7 +1655,7 @@ describe("an account with many albums", () => {
             collections,
             filesByCollection,
             encryptedMLData: {},
-            fileCiphertexts,
+            fileCiphertexts: {},
         });
         server = (async (input: RequestInfo | URL, init?: RequestInit) => {
             const url =
@@ -1716,7 +1665,7 @@ describe("an account with many albums", () => {
                       ? input.href
                       : input.url;
             const { pathname, search } = new URL(url);
-            requests.push(pathname + search);
+            paths.push(pathname + search);
             return ente(input, init);
         }) as typeof globalThis.fetch;
         const client = await Client.login({
@@ -1728,17 +1677,16 @@ describe("an account with many albums", () => {
     });
 
     beforeEach(() => {
-        requests.length = 0;
+        paths.length = 0;
     });
 
-    // A context whose saved session sends its requests to `fetch`, by default
+    // A context whose saved session sends its requests to the fake server.
-    // the fake server.
+    const accountContext = (): CliContext => {
-    const accountContext = (fetch = server): CliContext => {
         const ctx = context();
         saveSession(ctx.sessionDir, snapshot);
         return {
             ...ctx,
-            loadSession: (path) => loadSession(path, { fetch }),
+            loadSession: (path) => loadSession(path, { fetch: server }),
         };
     };
 
@@ -1751,219 +1699,72 @@ describe("an account with many albums", () => {
             albums += `${id}\talbum\tAlbum ${id}\n`;
         expect(stdout.text).toBe(albums);
         expect(stderr.text).toBe("");
-        expect(requests).toEqual(["/collections/v2?sinceTime=0"]);
+        expect(paths).toEqual(["/collections/v2?sinceTime=0"]);
 
         // It stored nothing that would make a refresh skip an album's files:
-        // `files` after it still lists every album's photo.
+        // `files` after it still finds the album's photo.
         stdout.text = "";
-        expect(await filesCommand(ctx, undefined, {})).toBe(0);
+        expect(await filesCommand(ctx, "7", {})).toBe(0);
-        expect(stdout.text).toBe(everyPhoto);
+        expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n");
     });
 
-    it("files with a collection ID asks for the album list and that album's files alone", async () => {
+    it("files fetches the files of the album it lists and of no other", async () => {
         expect(await filesCommand(accountContext(), "7", {})).toBe(0);
 
         expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n");
         expect(stderr.text).toBe("");
-        expect(requests).toEqual([
+        expect(paths).toEqual([
             "/collections/v2?sinceTime=0",
             "/collections/v2/diff?collectionID=7&sinceTime=0",
         ]);
     });
 
-    // Here the server sends album 7's files in two pages. Photo 700 changed
+    // The first listing stored the album, so the second finds its stored copy
-    // between them, so both pages hold a copy of it.
+    // current and asks for no album's files.
-    it("files with a collection ID lists only the later copy of a photo sent in two pages", async () => {
+    it("files run again for the same album sends no request for its files", async () => {
-        const firstPage = [
-            photoRow({
-                id: 700,
-                collectionID: 7,
-                title: "photo-7.jpg",
-                updationTime: 1,
-            }).row,
-            photoRow({
-                id: 701,
-                collectionID: 7,
-                title: "photo-701.jpg",
-                updationTime: 1,
-            }).row,
-        ];
-        const secondPage = [
-            photoRow({
-                id: 700,
-                collectionID: 7,
-                title: "photo-7-edited.jpg",
-                updationTime: 2,
-            }).row,
-        ];
-        const paged = (async (input: RequestInfo | URL, init?: RequestInit) => {
-            const url = new URL(input instanceof Request ? input.url : input);
-            if (
-                url.pathname !== "/collections/v2/diff" ||
-                url.searchParams.get("collectionID") !== "7"
-            ) {
-                return server(input, init);
-            }
-            const first = url.searchParams.get("sinceTime") === "0";
-            return new Response(
-                JSON.stringify({
-                    diff: first ? firstPage : secondPage,
-                    hasMore: first,
-                }),
-            );
-        }) as typeof globalThis.fetch;
-
-        expect(await filesCommand(accountContext(paged), "7", {})).toBe(0);
-
-        // Photo 700 is listed once, with the later title, before photo 701.
-        expect(stdout.text).toBe(
-            "700\timage\tphoto-7-edited.jpg\n701\timage\tphoto-701.jpg\n",
-        );
-    });
-
-    // The cache stays empty, so a later `files` without a collection ID still
-    // fetches the files of all 40 albums.
-    it("files with a collection ID stores nothing in the cache", async () => {
         const ctx = accountContext();
         expect(await filesCommand(ctx, "7", {})).toBe(0);
-        expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
-
         stdout.text = "";
-        expect(await filesCommand(ctx, undefined, {})).toBe(0);
+        paths.length = 0;
-        expect(stdout.text).toBe(everyPhoto);
+
-        expect(stderr.text).toBe(progress);
+        expect(await filesCommand(ctx, "7", {})).toBe(0);
+        expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n");
+        expect(paths).toEqual(["/collections/v2?sinceTime=0"]);
     });
 
-    // Photo 200 is in album 2, so `get` asks for no file of album 3 or later.
+    it("files with no album's ID counts on stderr each album whose files its refresh fetched", async () => {
-    // The last request downloads the photo's original.
-    it("get on an empty cache asks for the albums' files only until one holds the photo", async () => {
-        const out = join(root, "photo.jpg");
-        expect(await getCommand(accountContext(), "200", { out })).toBe(0);
-
-        expect(readFileSync(out, "utf-8")).toBe("photo-2.jpg");
-        expect(stderr.text).toBe(`11 bytes -> ${out}\n`);
-        expect(requests).toEqual([
-            "/collections/v2?sinceTime=0",
-            "/collections/v2/diff?collectionID=1&sinceTime=0",
-            "/collections/v2/diff?collectionID=2&sinceTime=0",
-            "/?fileID=200",
-        ]);
-        expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
-    });
-
-    // `files` first fills the cache, so `get` waits for a refresh. No album has
-    // changed since, so the refresh asks for no album's files.
-    it("get on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => {
-        const ctx = accountContext();
-        expect(await filesCommand(ctx, undefined, {})).toBe(0);
-        requests.length = 0;
-        stderr.text = "";
-
-        const out = join(root, "photo.jpg");
-        expect(await getCommand(ctx, "4000", { out })).toBe(0);
-        expect(readFileSync(out, "utf-8")).toBe("photo-40.jpg");
-        expect(stderr.text).toBe(`12 bytes -> ${out}\n`);
-        expect(requests).toEqual([
-            "/collections/v2?sinceTime=1",
-            "/?fileID=4000",
-        ]);
-
-        requests.length = 0;
-        stderr.text = "";
-        expect(await getCommand(ctx, "999", { out })).toBe(1);
-        expect(stderr.text).toBe("File 999 not found\n");
-        expect(requests).toEqual(["/collections/v2?sinceTime=1"]);
-    });
-
-    // The helper searches the most recently updated album first. All 40 were
-    // updated at the same time, and on a tie the higher ID comes first. Photo
-    // 3900 is in album 39, so the helper asks for no file of album 38 or
-    // earlier. The last request downloads the photo's original.
-    it("fix-missing-thumbnails --file on an empty cache asks for the albums' files only until one holds the photo", async () => {
-        const opts = { file: ["3900"], json: true };
-        expect(await fixMissingThumbnailsCommand(accountContext(), opts)).toBe(
-            0,
-        );
-
-        const reason =
-            "unsupported image format (only baseline JPEG can be regenerated)";
-        expect(JSON.parse(stdout.text)).toEqual([
-            {
-                fileID: 3900,
-                title: "photo-39.jpg",
-                collection: "Album 39",
-                status: "skipped",
-                reason,
-            },
-        ]);
-        expect(stderr.text).toBe(
-            "[Album 39] Downloading photo-39.jpg for thumbnail generation...\n" +
-                `[Album 39] Skipping photo-39.jpg: ${reason}\n`,
-        );
-        expect(requests).toEqual([
-            "/collections/v2?sinceTime=0",
-            "/collections/v2/diff?collectionID=40&sinceTime=0",
-            "/collections/v2/diff?collectionID=39&sinceTime=0",
-            "/?fileID=3900",
-        ]);
-        expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
-    });
-
-    // `files` first fills the cache, so the helper waits for a refresh. No
-    // album has changed since, so the refresh asks for no album's files.
-    it("fix-missing-thumbnails --file on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => {
-        const ctx = accountContext();
-        expect(await filesCommand(ctx, undefined, {})).toBe(0);
-        requests.length = 0;
-        stdout.text = "";
-        stderr.text = "";
-
-        const opts = { file: ["3900"], json: true };
-        expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0);
-        const reason =
-            "unsupported image format (only baseline JPEG can be regenerated)";
-        expect(JSON.parse(stdout.text)).toEqual([
-            {
-                fileID: 3900,
-                title: "photo-39.jpg",
-                collection: "Album 39",
-                status: "skipped",
-                reason,
-            },
-        ]);
-        expect(stderr.text).toBe(
-            "[Album 39] Downloading photo-39.jpg for thumbnail generation...\n" +
-                `[Album 39] Skipping photo-39.jpg: ${reason}\n`,
-        );
-        expect(requests).toEqual([
-            "/collections/v2?sinceTime=1",
-            "/?fileID=3900",
-        ]);
-
-        requests.length = 0;
-        stdout.text = "";
-        const notFound = { file: ["999"], json: true };
-        expect(await fixMissingThumbnailsCommand(ctx, notFound)).toBe(1);
-        expect(JSON.parse(stdout.text)).toMatchObject([
-            { fileID: 999, reason: "file not found in any collection" },
-        ]);
-        expect(requests).toEqual(["/collections/v2?sinceTime=1"]);
-    });
-
-    it("files counts on stderr each album whose files its refresh fetched", async () => {
         expect(await filesCommand(accountContext(), undefined, {})).toBe(0);
 
-        expect(stdout.text).toBe(everyPhoto);
+        let photos = "";
+        let progress = "";
+        for (let n = 1; n <= ALBUMS; n++) {
+            photos += `${n * 100}\timage\tphoto-${n}.jpg\n`;
+            progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`;
+        }
+        expect(stdout.text).toBe(photos);
         expect(stderr.text).toBe(progress);
     });
 
     // `--json` shapes stdout only, so the refresh's progress is still counted
     // on stderr.
-    it("files --json counts on stderr each album whose files its refresh fetched", async () => {
+    it("files --json with no album's ID counts on stderr each album whose files its refresh fetched", async () => {
-        const opts = { json: true };
+        expect(
-        expect(await filesCommand(accountContext(), undefined, opts)).toBe(0);
+            await filesCommand(accountContext(), undefined, { json: true }),
+        ).toBe(0);
 
-        expect(JSON.parse(stdout.text)).toHaveLength(ALBUMS);
+        const rows = JSON.parse(stdout.text);
+        expect(rows).toHaveLength(ALBUMS);
+        expect(rows[6]).toEqual({
+            id: 700,
+            title: "photo-7.jpg",
+            fileType: "image",
+            creationTime: 1000,
+            collectionID: 7,
+        });
+        let progress = "";
+        for (let n = 1; n <= ALBUMS; n++) {
+            progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`;
+        }
         expect(stderr.text).toBe(progress);
     });
 });

test/cli/log.test.ts

@@ -234,15 +234,13 @@ describe("-q", () => {
         });
     });
 
-    // No album holds file 999, so fixing it fails. The summary from
+    // The refresh's progress is left out; the line saying why the command
-    // `--- Done ---` to the end of the `Failed files:` list is left out. The
+    // failed is the fatal one, and it is printed.
-    // line saying why the command failed is the fatal one, and it is printed.
     it("prints only the line a failing command exits with", async () => {
-        const fix = ["helper", "fix-missing-thumbnails", "--file", "999"];
+        expect(await quak("-q", "files", "9")).toEqual({
-        expect(await quak("-q", ...fix)).toEqual({
             code: 1,
             stdout: "",
-            stderr: `quak: 1 file(s) failed; see the "Failed files:" list, or the JSON with --json\n`,
+            stderr: "Collection 9 not found\n",
         });
     });
 
@@ -328,7 +326,7 @@ describe("what the library reports", () => {
         ]);
     });
 
-    // Each operation is named in plain words.
+    // Each operation is named in plain words, as the README names it.
     it("logs a refresh's progress at info, and its start and end at debug", () => {
         const { log, lines } = collect();
         const onProgress = logRefresh(log);

test/cli/read.test.ts

@@ -3,25 +3,23 @@
  * issue #36, issue #52).
  *
  * The `collections`, `files`, `get`, and `get-thumb` commands must answer for
- * current server state, not the local cache. `get`, `get-thumb` and `files`
+ * current server state, not the local cache. `files` without a collection ID,
- * without a collection ID force a `Library.fresh()` round-trip before they
+ * `get` and `get-thumb` force a `Library.fresh()` round-trip before they read.
- * read. The stand-in library below serves no cached record until `fresh()` has
+ * The stand-in library below serves nothing until `fresh()` has been awaited.
- * been awaited, so a helper that read without refreshing would come back empty
+ * A helper that read without refreshing would come back empty and fail here.
- * and fail here. `collections` asks the library for the collection list alone
+ * `collections` asks the library for the collection list alone
- * (`freshCollections()`, issue #210). `files` with a collection ID asks for
+ * (`freshCollections()`, issue #210). It must not wait for a refresh, which
- * that list, then for that one collection's files (`freshFiles()`, issue
+ * would fetch the files of every collection. `files COLLECTION` reads that list
- * #217). Neither may wait for a refresh, which would fetch the files of every
+ * too, then brings that one collection up to date (`refreshCollection()`,
- * collection. On an empty cache, `get` and `get-thumb` ask for that list, then
+ * issue #217). The stand-in serves a collection's files once `fresh()` or
- * for one collection's files at a time, until they find the file (issue #222).
+ * `refreshCollection()` for that collection has run.
- * `helper fix-missing-thumbnails --file` does the same until it finds every
- * file it names, reading the most recently updated collection first (issue
- * #223).
  *
- * `collections` and `files` also list in the order the library gives them, not
+ * `collections` and `files` also list in the library's enumeration order
+ * (`listCollections`/`listFiles`) — the order the pre-library CLI printed — not
  * the albums/photos projection's newest-first order. The fixtures are seeded in
- * an order that a newest-first sort would rearrange, so a regression to the
+ * an enumeration order that a newest-first sort would rearrange, so a
- * projection order would fail here too. Field values still come from the raw
+ * regression to the projection order would fail here too. Field values still
- * metadata via `cli-output.ts`.
+ * come from the raw metadata via `cli-output.ts`.
  */
 
 import { describe, it, expect } from "vitest";
@@ -30,10 +28,10 @@ import {
     freshCollections,
     freshFiles,
     freshFile,
-    freshFilesByID,
     type FreshReadLibrary,
 } from "../../src/cli-read.js";
 import { fileListRow } from "../../src/cli-output.js";
+import type { Photo } from "../../src/library/index.js";
 import type { Collection, EnteFile } from "../../src/model/types.js";
 
 const collection = (id: number, updationTime: number): Collection => ({
@@ -67,15 +65,16 @@ const file = (
     updationTime: creationTime,
 });
 
-// A library that reveals its cached records only after `fresh()` has been
+// A library that reveals its records only after `fresh()` has been awaited, and
-// awaited, and serves them in the enumeration order it was given.
+// serves them in the enumeration order it was given. `freshCollections()`
-// `freshCollections()` and `freshFiles()` answer in that order without a
+// answers the collection list in that order without a refresh.
-// refresh.
+// `refreshCollection()` reveals the files of that one collection.
+// `photos.byID` returns a stand-in `Photo` carrying just the fileID the helper
+// passes through.
 class FakeLibrary implements FreshReadLibrary {
     freshCalls = 0;
     freshCollectionsCalls = 0;
-    // The ID of each collection `freshFiles()` was asked for.
+    refreshCollectionIDs: number[] = [];
-    freshFilesCalls: number[] = [];
     private refreshed = false;
 
     constructor(
@@ -94,9 +93,8 @@ class FakeLibrary implements FreshReadLibrary {
         return this.collections;
     }
 
-    async freshFiles(collection: Collection): Promise<EnteFile[]> {
+    async refreshCollection(collection: Collection): Promise<void> {
-        this.freshFilesCalls.push(collection.id);
+        this.refreshCollectionIDs.push(collection.id);
-        return this.files.filter((f) => f.collectionID === collection.id);
     }
 
     listCollections(): Collection[] {
@@ -104,7 +102,9 @@ class FakeLibrary implements FreshReadLibrary {
     }
 
     listFiles(collectionID: number): EnteFile[] {
-        return this.refreshed
+        const current =
+            this.refreshed || this.refreshCollectionIDs.includes(collectionID);
+        return current
             ? this.files.filter((f) => f.collectionID === collectionID)
             : [];
     }
@@ -113,6 +113,14 @@ class FakeLibrary implements FreshReadLibrary {
         if (!this.refreshed) return undefined;
         return this.files.find((f) => f.id === fileID);
     }
+
+    photos = {
+        byID: ({ fileID }: { fileID: number }): Photo | undefined => {
+            if (!this.refreshed) return undefined;
+            if (!this.files.some((f) => f.id === fileID)) return undefined;
+            return { fileID } as unknown as Photo;
+        },
+    };
 }
 
 describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
@@ -136,9 +144,10 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
         expect(rows.map((c) => c.id)).not.toEqual(newestFirst);
     });
 
-    it("freshFiles with a collection ID reads that collection's files alone and keeps their order and raw fields", async () => {
+    it("freshFiles brings the collection alone up to date, lists in enumeration order, keeps raw fields", async () => {
-        // Given in the order 10, 11, 12; creationTimes ascending, so a
+        // Enumeration order by id 10, 11, 12; creationTimes ascending, so a
-        // newest-first sort would reverse them.
+        // newest-first sort would reverse them. Collection 2's file is not
+        // listed.
         const files = [
             file(10, 1, 1_700_000_000_000_000),
             file(11, 1, 1_700_000_000_000_001),
@@ -146,14 +155,14 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
             file(20, 2, 1_700_000_000_000_003),
         ];
         const lib = new FakeLibrary(
-            [collection(1, 100), collection(2, 200)],
+            [collection(1, 100), collection(2, 100)],
             files,
         );
 
         const rows = await freshFiles(lib, 1);
-        expect(lib.freshCalls).toBe(0);
         expect(lib.freshCollectionsCalls).toBe(1);
-        expect(lib.freshFilesCalls).toEqual([1]);
+        expect(lib.refreshCollectionIDs).toEqual([1]);
+        expect(lib.freshCalls).toBe(0);
         expect(rows?.map((f) => f.id)).toEqual([10, 11, 12]);
 
         // Field values come from raw metadata: microsecond creationTime and the
@@ -210,117 +219,29 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
     it("freshFiles returns undefined for an unknown collection", async () => {
         const lib = new FakeLibrary([collection(1, 100)], []);
         const rows = await freshFiles(lib, 999);
+        expect(lib.freshCollectionsCalls).toBe(1);
+        expect(lib.refreshCollectionIDs).toEqual([]);
         expect(lib.freshCalls).toBe(0);
-        expect(lib.freshFilesCalls).toEqual([]);
         expect(rows).toBeUndefined();
     });
 
-    // Three collections, each holding one file.
+    it("freshFile refreshes first, then resolves the photo and its raw record", async () => {
-    const threeCollections = (): FakeLibrary =>
+        const f = file(10, 1, 1_700_000_000_000_000);
-        new FakeLibrary(
+        const lib = new FakeLibrary([collection(1, 100)], [f]);
-            [collection(1, 100), collection(2, 200), collection(3, 300)],
+
-            [
+        const resolved = await freshFile(lib, 10);
-                file(10, 1, 1_700_000_000_000_000),
+        expect(lib.freshCalls).toBe(1);
-                file(20, 2, 1_700_000_000_000_001),
+        expect(resolved?.photo.fileID).toBe(10);
-                file(30, 3, 1_700_000_000_000_002),
+        expect(resolved?.file.metadata.title).toBe("file-10.jpg");
-            ],
+        expect(resolved?.file.metadata.creationTime).toBe(
+            1_700_000_000_000_000,
         );
-
-    it("freshFile on an empty cache reads each collection's files in the server's order until one holds the file", async () => {
-        const lib = threeCollections();
-
-        const found = await freshFile(lib, 20);
-        expect(lib.freshCalls).toBe(0);
-        expect(lib.freshCollectionsCalls).toBe(1);
-        expect(lib.freshFilesCalls).toEqual([1, 2]);
-        expect(found?.metadata.title).toBe("file-20.jpg");
-        expect(found?.metadata.creationTime).toBe(1_700_000_000_000_001);
     });
 
-    // The stand-in's cache holds what the server holds once `fresh()` has run.
+    it("freshFile returns undefined for an unknown file", async () => {
-    it("freshFile on a cache that holds the collections refreshes, then looks the file up in the cache", async () => {
+        const lib = new FakeLibrary([collection(1, 100)], []);
-        const lib = threeCollections();
+        const resolved = await freshFile(lib, 404);
-        await lib.fresh();
+        expect(lib.freshCalls).toBe(1);
-
+        expect(resolved).toBeUndefined();
-        const found = await freshFile(lib, 30);
-        expect(lib.freshCalls).toBe(2);
-        expect(lib.freshCollectionsCalls).toBe(0);
-        expect(lib.freshFilesCalls).toEqual([]);
-        expect(found?.collectionID).toBe(3);
-    });
-
-    it("freshFile on an empty cache returns undefined for an unknown file, after reading every collection's files", async () => {
-        const lib = threeCollections();
-
-        const found = await freshFile(lib, 404);
-        expect(lib.freshCalls).toBe(0);
-        expect(lib.freshFilesCalls).toEqual([1, 2, 3]);
-        expect(found).toBeUndefined();
-    });
-
-    // The server lists collections 1, 2 and 3, and collection 3 was updated
-    // most recently. File 11 is in collections 1 and 2.
-    const fileInTwoCollections = (): FakeLibrary =>
-        new FakeLibrary(
-            [collection(1, 100), collection(2, 200), collection(3, 300)],
-            [
-                file(10, 1, 1_700_000_000_000_000),
-                file(11, 1, 1_700_000_000_000_001),
-                file(11, 2, 1_700_000_000_000_001),
-                file(20, 2, 1_700_000_000_000_002),
-                file(30, 3, 1_700_000_000_000_003),
-            ],
-        );
-
-    it("freshFilesByID on an empty cache reads the most recently updated collection's files first, until it has found every file", async () => {
-        const lib = fileInTwoCollections();
-
-        const found = await freshFilesByID(lib, [11, 20]);
-        expect(lib.freshCalls).toBe(0);
-        expect(lib.freshCollectionsCalls).toBe(1);
-        expect(lib.freshFilesCalls).toEqual([3, 2]);
-        expect(found.get(11)?.collection.id).toBe(2);
-        expect(found.get(11)?.file.collectionID).toBe(2);
-        expect(found.get(20)?.collection.id).toBe(2);
-    });
-
-    // Collection 1 was updated last, though collections 2 and 3 have higher
-    // IDs. File 11 is in collections 1 and 2.
-    it("freshFilesByID on an empty cache reads the most recently updated collection first, though its ID is the lowest", async () => {
-        const lib = new FakeLibrary(
-            [collection(1, 400), collection(2, 200), collection(3, 300)],
-            [
-                file(11, 1, 1_700_000_000_000_001),
-                file(11, 2, 1_700_000_000_000_001),
-            ],
-        );
-
-        const found = await freshFilesByID(lib, [11]);
-        expect(lib.freshFilesCalls).toEqual([1]);
-        expect(found.get(11)?.collection.id).toBe(1);
-        expect(found.get(11)?.file.collectionID).toBe(1);
-    });
-
-    // The stand-in's cache holds what the server holds once `fresh()` has run.
-    it("freshFilesByID on a cache that holds the collections refreshes, then finds each file in the most recently updated collection that holds it", async () => {
-        const lib = fileInTwoCollections();
-        await lib.fresh();
-
-        const found = await freshFilesByID(lib, [11, 404]);
-        expect(lib.freshCalls).toBe(2);
-        expect(lib.freshCollectionsCalls).toBe(0);
-        expect(lib.freshFilesCalls).toEqual([]);
-        expect(found.get(11)?.collection.id).toBe(2);
-        expect(found.get(11)?.file.collectionID).toBe(2);
-        expect(found.has(404)).toBe(false);
-    });
-
-    it("freshFilesByID on an empty cache reads every collection's files when a file is in none of them", async () => {
-        const lib = fileInTwoCollections();
-
-        const found = await freshFilesByID(lib, [10, 404]);
-        expect(lib.freshFilesCalls).toEqual([3, 2, 1]);
-        expect(found.get(10)?.collection.id).toBe(1);
-        expect(found.has(404)).toBe(false);
     });
 });

test/library/content-library.test.ts

@@ -253,12 +253,6 @@ describe("Library content wiring", () => {
         await expect(
             lib.thumbnails.ensure({ fileIDs: [1], priority: "visible" }),
         ).rejects.toThrow(/content cache/i);
-        await expect(lib.originalOf(file(1, 1))).rejects.toThrow(
-            /content cache/i,
-        );
-        await expect(lib.thumbnailOf(file(1, 1))).rejects.toThrow(
-            /content cache/i,
-        );
         await lib.close();
     });
 });

test/library/content.test.ts

@@ -375,25 +375,6 @@ describe("ContentCache.original / thumbnail", () => {
         await cache.open();
         await expect(cache.original(999)).rejects.toThrow(/unknown file/i);
     });
-
-    // `getFile` knows no file here, so the cache can fetch only the file it is
-    // given.
-    it("originalOf and thumbnailOf fetch and cache the file they are given", async () => {
-        const { cache, source } = buildCache({ files: [] });
-        await cache.open();
-
-        const original = await cache.originalOf(file(5));
-        const thumbnail = await cache.thumbnailOf(file(5));
-
-        expect(source.originalCalls).toEqual([5]);
-        expect(source.thumbnailCalls).toEqual([5]);
-        expect(original.path).toBe(join(cacheDir, "originals", "5.jpg"));
-        expect(thumbnail.path).toBe(join(cacheDir, "thumbnails", "5.jpg"));
-        expect(cache.pathsFor(5)).toEqual({
-            originalPath: original.path,
-            thumbnailPath: thumbnail.path,
-        });
-    });
 });
 
 describe("ContentCache.ensureThumbnails", () => {

test/library/fresh.test.ts

@@ -17,6 +17,12 @@
  *  4. `freshCollections()` reads the collection list alone, with one request
  *     and no collection's files, and stores nothing, so the next refresh still
  *     fetches the files of every collection (issue #210).
+ *  5. `refreshCollection()` brings one collection up to date in the store
+ *     (issue #217). It leaves the stored cursor where it was, so the next
+ *     refresh still fetches the files of every other collection that changed.
+ *     It never runs at the same time as a refresh, and never brings back a
+ *     collection that a refresh deleted. `close()` waits for it. A library
+ *     opened on a cache that only it wrote awaits its first refresh.
  *
  * The client is the same metadata-only mock the background-refresh tests use:
  * no crypto, no network, scripted pages, and a record of each call. A long
@@ -30,6 +36,7 @@ import { tmpdir } from "node:os";
 import { join } from "node:path";
 
 import { Library } from "../../src/library/index.js";
+import { MetadataStore } from "../../src/library/store.js";
 import type { CollectionsPage, FilesPage } from "../../src/client.js";
 import type { Collection, EnteFile } from "../../src/model/types.js";
 
@@ -144,6 +151,26 @@ const openSeeded = async (
     return { client, lib };
 };
 
+// Hold the client's next files request until `release()` is called. `started`
+// resolves once that request has been sent.
+const holdNextFilesRequest = (
+    client: MockClient,
+): { started: Promise<void>; release: () => void } => {
+    let release!: () => void;
+    const held = new Promise<void>((r) => (release = r));
+    let sent!: () => void;
+    const started = new Promise<void>((r) => (sent = r));
+    const realFilesSince = client.filesSince.bind(client);
+    client.filesSince = async (args) => {
+        client.filesSince = realFilesSince;
+        const page = await realFilesSince(args);
+        sent();
+        await held;
+        return page;
+    };
+    return { started, release };
+};
+
 describe("Library.fresh", () => {
     let dir: string;
     let cacheDirectory: string;
@@ -340,3 +367,216 @@ describe("Library.freshCollections", () => {
         }
     });
 });
+
+describe("Library.refreshCollection", () => {
+    let dir: string;
+    let cacheDirectory: string;
+
+    beforeEach(() => {
+        dir = mkdtempSync(join(tmpdir(), "quak-refresh-collection-"));
+        cacheDirectory = join(dir, "cache");
+    });
+
+    afterEach(() => {
+        rmSync(dir, { recursive: true, force: true });
+    });
+
+    // Collections 1 and 2 are stored at updationTime 100, with a stored cursor
+    // of 100, and then both change on the server.
+    it("fetches the files of one changed collection and leaves the others to the next refresh", async () => {
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100), collection(2, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+        client.filesFor(2, {
+            files: [file(2001, 2, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: SLOW_INTERVAL,
+        });
+        try {
+            client.filesCalls.length = 0;
+            client.filesFor(1, {
+                files: [file(1002, 1, 110)],
+                deleted: [],
+                cursor: 110,
+            });
+            client.filesFor(2, {
+                files: [file(2002, 2, 120)],
+                deleted: [],
+                cursor: 120,
+            });
+
+            // Collection 1's files alone, since the stored copy.
+            await lib.refreshCollection(collection(1, 110));
+            expect(client.filesCalls).toEqual([
+                { collectionID: 1, sinceTime: 100 },
+            ]);
+            expect(lib.listFiles(1).map((f) => f.id)).toEqual([1001, 1002]);
+            expect(lib.listFiles(2).map((f) => f.id)).toEqual([2001]);
+
+            // The stored copy is now current, so no request.
+            await lib.refreshCollection(collection(1, 110));
+            expect(client.filesCalls).toHaveLength(1);
+
+            // The next refresh asks from the stored cursor, 100, so it still
+            // sees collection 2 changed and fetches its files.
+            client.collectionsQueue.push({
+                collections: [collection(1, 110), collection(2, 120)],
+                deleted: [],
+                cursor: 120,
+            });
+            await lib.fresh();
+            expect(client.collectionsSinceTimes).toEqual([0, 100]);
+            expect(client.filesCalls).toEqual([
+                { collectionID: 1, sinceTime: 100 },
+                { collectionID: 2, sinceTime: 100 },
+            ]);
+            expect(lib.listFiles(2).map((f) => f.id)).toEqual([2001, 2002]);
+        } finally {
+            await lib.close();
+        }
+    });
+
+    // Collection 1 is stored at 100. `refreshCollection()` asks for it at 110,
+    // which adds file 1002. While that files request waits, a fresh read
+    // starts a refresh. The server tells that refresh that collection 1 was
+    // deleted at 130.
+    it("holds off a refresh that starts during the files request, which then deletes the collection", async () => {
+        const { client, lib } = await openSeeded(cacheDirectory);
+        const files = holdNextFilesRequest(client);
+        try {
+            client.filesFor(1, {
+                files: [file(1002, 1, 110)],
+                deleted: [],
+                cursor: 110,
+            });
+            client.collectionsQueue.push({
+                collections: [],
+                deleted: [1],
+                cursor: 130,
+            });
+
+            const refreshing = lib.refreshCollection(collection(1, 110));
+            await files.started;
+            const reading = lib.fresh();
+            // The refresh has not sent its collections request yet.
+            await new Promise((r) => setImmediate(r));
+            expect(client.collectionsSinceTimes).toEqual([0]);
+
+            files.release();
+            await refreshing;
+            await reading;
+            expect(client.collectionsSinceTimes).toEqual([0, 100]);
+            expect(lib.getCollection(1)).toBeUndefined();
+            expect(lib.listFiles(1)).toEqual([]);
+        } finally {
+            files.release();
+            await lib.close();
+        }
+    });
+
+    // `freshCollections()` lists collection 1 at 110. Before
+    // `refreshCollection()` is called, a refresh learns that it was deleted at
+    // 130.
+    it("stores nothing for a collection that a refresh deleted before the call", async () => {
+        const { client, lib } = await openSeeded(cacheDirectory);
+        try {
+            client.collectionsQueue.push(
+                { collections: [collection(1, 110)], deleted: [], cursor: 110 },
+                { collections: [], deleted: [1], cursor: 130 },
+            );
+            const [listed] = await lib.freshCollections();
+            await lib.fresh();
+            client.filesCalls.length = 0;
+
+            await lib.refreshCollection(listed);
+            expect(client.filesCalls).toEqual([]);
+            expect(lib.getCollection(1)).toBeUndefined();
+        } finally {
+            await lib.close();
+        }
+    });
+
+    // The caller may remove the cache directory once `close()` resolves, so
+    // the cache write must have landed by then.
+    it("close() waits for a refreshCollection() in flight", async () => {
+        const { client, lib } = await openSeeded(cacheDirectory);
+        const files = holdNextFilesRequest(client);
+        client.filesFor(1, {
+            files: [file(1002, 1, 110)],
+            deleted: [],
+            cursor: 110,
+        });
+
+        const refreshing = lib.refreshCollection(collection(1, 110));
+        await files.started;
+        let closed = false;
+        const closing = lib.close().then(() => {
+            closed = true;
+        });
+        await new Promise((r) => setImmediate(r));
+        expect(closed).toBe(false);
+
+        files.release();
+        await closing;
+        const saved = await MetadataStore.load(
+            join(cacheDirectory, "metadata.json"),
+        );
+        expect(saved.listFiles(1).map((f) => f.id)).toEqual([1001, 1002]);
+        await refreshing;
+    });
+
+    // `quak files 1` on an empty cache stores collection 1 alone. A library
+    // opened later on that cache must not serve it as the whole account.
+    it("open() awaits the first refresh on a cache that only refreshCollection() wrote", async () => {
+        const client = new MockClient();
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+        const first = await Library.open({
+            client,
+            cacheDirectory,
+            refreshOnOpen: false,
+            refreshIntervalSeconds: SLOW_INTERVAL,
+        });
+        await first.refreshCollection(collection(1, 100));
+        await first.close();
+
+        client.collectionsQueue.push({
+            collections: [collection(1, 100), collection(2, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(2, {
+            files: [file(2001, 2, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: SLOW_INTERVAL,
+        });
+        try {
+            expect(lib.status().lastRefreshAt).toBeDefined();
+            expect(lib.listCollections().map((c) => c.id)).toEqual([1, 2]);
+            expect(lib.listFiles(2).map((f) => f.id)).toEqual([2001]);
+        } finally {
+            await lib.close();
+        }
+    });
+});

test/thumbnails/thumbnails.test.ts

@@ -6,12 +6,10 @@
  * working thumbnails, others return 404 or empty bodies. The tests
  * verify that the detection and repair logic handles each case correctly.
  *
- * `listMissingThumbnails` takes an open `Library` to enumerate the files. Both
+ * As of issue #52 both helpers take an open `Library` for enumeration and the
- * helpers take the `Client` for the API operations: the thumbnail existence
+ * `Client` for the API operations that stay unchanged (the thumbnail existence
- * check, and the encrypt-and-upload path. `fixMissingThumbnails` takes the
+ * check, and the encrypt-and-upload path). `fixMissingThumbnails` reads each
- * record of each file, which these tests look up with `cachedFilesByID`. It
+ * original through the library's content cache (`photo.original()`).
- * reads each original through the library's content cache, by
- * `lib.originalOf(file)`.
  *
  * `fixMissingThumbnails` is the most complex function in quak: it
  * downloads the original file, generates a JPEG thumbnail with jpeg-js,
@@ -43,7 +41,6 @@ import {
 import { SRP, SrpServer } from "fast-srp-hap";
 import { Client } from "../../src/client.js";
 import { Library } from "../../src/library/index.js";
-import { cachedFilesByID } from "../../src/cli-read.js";
 import {
     listMissingThumbnails,
     fixMissingThumbnails,
@@ -659,12 +656,7 @@ describe("fixMissingThumbnails", () => {
         const client = await login(buildThumbFetch(fixMock));
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(
+        const results = await fixMissingThumbnails(lib, client, [101]);
-            lib,
-            client,
-            [101],
-            cachedFilesByID(lib, [101]),
-        );
         lib.close();
 
         expect(results.length).toBe(1);
@@ -701,12 +693,7 @@ describe("fixMissingThumbnails", () => {
         const client = await login(buildThumbFetch(fixMock));
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(
+        const results = await fixMissingThumbnails(lib, client, [999]);
-            lib,
-            client,
-            [999],
-            cachedFilesByID(lib, [999]),
-        );
         lib.close();
 
         expect(results.length).toBe(1);
@@ -728,7 +715,6 @@ describe("fixMissingThumbnails", () => {
             lib,
             client,
             [101, 102],
-            cachedFilesByID(lib, [101, 102]),
             (msg, level) => {
                 if (level === "error") errors.push(msg);
             },
@@ -759,7 +745,6 @@ describe("fixMissingThumbnails", () => {
             lib,
             client,
             [101, 103, 104],
-            cachedFilesByID(lib, [101, 103, 104]),
         );
         lib.close();
 
@@ -794,12 +779,7 @@ describe("fixMissingThumbnails", () => {
         const client = await login(counted.fetch);
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(
+        const results = await fixMissingThumbnails(lib, client, [101]);
-            lib,
-            client,
-            [101],
-            cachedFilesByID(lib, [101]),
-        );
         lib.close();
 
         expect(results[0]!.status).toBe("skipped");
@@ -820,12 +800,7 @@ describe("fixMissingThumbnails", () => {
         const client = await login(counted.fetch);
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(
+        const results = await fixMissingThumbnails(lib, client, [101]);
-            lib,
-            client,
-            [101],
-            cachedFilesByID(lib, [101]),
-        );
         lib.close();
 
         expect(results[0]!.status).toBe("skipped");
@@ -866,12 +841,7 @@ describe("fixMissingThumbnails", () => {
         const client = await login(buildThumbFetch(fixMock));
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(
+        const results = await fixMissingThumbnails(lib, client, [101]);
-            lib,
-            client,
-            [101],
-            cachedFilesByID(lib, [101]),
-        );
         lib.close();
 
         expect(results[0]!.status).toBe("fixed");