Precache all thumbnails and pinned originals from Library.open (closes #48)

Two background fills start inside open() with no caller input, through the
shared pools (#45) at background priority, so on-demand work always preempts
them. Thumbnails: every file newest first until all are on disk, sharing the
thumbnail pool with thumbnails.ensure. Originals: the pinned set — the
favorites album, then the latest-week window ending at the newest file —
through the content pool. The pinned set is the eviction predicate (#47), so a
pinned original is never evicted and a file that leaves the set (favorite
removed or window moved on a later refresh) becomes an ordinary, evictable
original. A cached file costs one map lookup and no fetch; each refresh re-kicks
the fills to pick up new files and retry failures. Progress via open()
onProgress and status() (thumbnailsCached/Total, originalsCached/Pinned).

Model: opus-4-8

README.md

@@ -38,47 +38,34 @@ yarn quak get 67890 --out ./photo.jpg
 yarn quak backup ./my-backup
 ```
 
-For library use, the primary surface is the cache-backed `Library`:
+For library use:
 
 ```ts
-import { Client, Library } from "quak";
+import { Client } from "quak";
 
-// Log in once; the client satisfies the library's client interface.
 const client = await Client.login({
     email: "you@example.com",
     password: "your-password",
 });
 
-// Open a cache-backed library. On an empty cache this awaits one server
+for (const c of await client.listCollections()) {
-// refresh; on an existing cache it returns immediately and refreshes in the
+    console.log(c.id, c.name);
-// background every `refreshIntervalSeconds` (default 3).
+    const files = await client.listFiles(c.id, c.key);
-const lib = await Library.open({ client });
+    for (const f of files) {
-
+        console.log(`  ${f.metadata.title} [${f.metadata.fileType}]`);
-// Default reads answer synchronously from the local cache — no network.
-for (const album of lib.albums.list()) {
-    console.log(album.collectionID, album.name);
-    for (const photo of album.photos.list()) {
-        console.log(`  ${photo.title} [${photo.fileType}]`);
     }
 }
 
-// Fresh reads await a server round-trip and answer with current state.
+// Download a file
-const { albums } = await lib.fresh();
+const files = await client.listFiles(collectionID, collectionKey);
-console.log(`${albums.list().length} albums as of now`);
+await client.downloadFile(files[0], "./photo.jpg");
 
-// Fetch (and cache) one photo's full-resolution bytes.
+// Serialize session for later (consumer handles persistence)
-const photo = lib.photos.byID({ fileID: 12345 });
+const snapshot = client.toJSON();
-if (photo) {
+// ... later:
-    const { path } = await photo.original();
+const restored = Client.fromJSON(snapshot);
-    console.log(`original at ${path}`);
-}
-
-lib.close();
 ```
 
-The lower-level `Client` (login, session serialization, and the raw
-enumeration/download calls) is exported too and documented under Design below.
-
 ## Entrypoints
 
 This repository adheres to the
@@ -433,7 +420,6 @@ you would treat the password itself.
 ### CLI surface
 
 ```
-quak [--cache-dir <path>] <command>             global: local metadata/content cache location
 quak login                                      interactive or QUAK_EMAIL/QUAK_PASSWORD
 quak whoami                                     print logged-in account as JSON
 quak logout                                     delete saved session
@@ -442,28 +428,13 @@ quak files --collection <id> [--json]           list files in a collection
 quak get <fileID> [--out path] [--collection]   download and decrypt a file
 quak get-thumb <fileID> [--out] [--collection]  download and decrypt a thumbnail
 quak backup <dir> [--json]                      full incremental backup
-quak backup-metadata <dir> [--exif]             dump all decrypted metadata as JSON
 quak helper list-missing-thumbnails [--json]    find files with missing thumbnails
 quak helper fix-missing-thumbnails [--file ids] generate + upload missing thumbnails
 ```
 
-Every command runs on the same cache-backed library. The read commands —
+`get` and `get-thumb` search all collections for the file ID when `--collection`
-`collections`, `files`, `get`, and `get-thumb` — force a fresh server round-trip
+is not specified. All listing and backup commands support `--json` for
-before they answer, so they report current account state rather than whatever
+machine-readable output.
-the cache last held. `--cache-dir` overrides where the cache lives; without it
-each account gets its own directory under the per-user cache path.
-
-`get` and `get-thumb` resolve the file by ID directly, so `--collection` is
-accepted for backward compatibility but ignored. `backup-metadata --exif` (alias
-`--all`) additionally downloads each file to extract full EXIF/IPTC/XMP
-metadata. The listing and backup commands support `--json` for machine-readable
-output.
-
-`helper fix-missing-thumbnails` regenerates thumbnails for baseline JPEG images
-only, because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG
-image (PNG, HEIC) or a video is reported as `skipped` (unsupported format), kept
-distinct from a `failed` repair, and does not affect the exit code; a genuine
-failure still exits non-zero.
 
 ### Backup layout
 
@@ -489,7 +460,7 @@ code is non-zero if any files failed.
 
 - [x] Retry policy: no retry on 4xx, exponential backoff on 5xx and network
       errors
-- [x] Update the API reference section below to match the current implementation
+- [ ] Update the API reference section below to match the current implementation
 - [x] `make docker` green
 - [ ] Tag `v1.0.0`
 
@@ -503,179 +474,25 @@ Future (desktop client, separate repo):
 
 ## API reference
 
-The library's primary surface is the cache-backed `Library`; the lower-level
+The API reference section below is from an earlier draft and does not fully
-`Client` sits underneath it and is covered by the Design sections above. The
+reflect the current implementation. The authoritative API documentation is in
-test suite is the canonical, executable documentation — `test/library/` and
+the test files, particularly `test/client/usage.test.ts` which is a literate
-`test/client/usage.test.ts` walk every operation, and `yarn test` verifies them.
+tutorial walking through every operation. Run `yarn test` to verify the examples
+are correct.
 
-### Opening a library
+The key types and their actual signatures can be found in:
 
-`Library.open(options)` loads the on-disk cache, starts the background refresh
-loop, and resolves to a `Library`. On an empty cache it awaits the first refresh
-so it never opens onto empty data; on an existing cache it returns immediately
-and refreshes in the background, so an unreachable server does not block
-opening.
-
-`LibraryOptions`:
-
-| Option                   | Default                     | Meaning                                                               |
-| ------------------------ | --------------------------- | --------------------------------------------------------------------- |
-| `client`                 | required                    | the account client (a `Client`, or any `LibraryClient`)               |
-| `cacheDirectory`         | `<XDG cache>/quak/<userID>` | where `metadata.json` and the content cache live                      |
-| `downloadDirectory`      | none                        | backup destination; an original already stored there counts as cached |
-| `refreshIntervalSeconds` | `3`                         | background refresh cadence                                            |
-| `precacheThumbnails`     | `true`                      | prefetch every thumbnail, newest first                                |
-| `precacheOriginals`      | `true`                      | prefetch the favorites album and the latest-window originals          |
-| `precacheOriginalsDays`  | `7`                         | length in days of that latest window                                  |
-| `cacheOriginalsMaxBytes` | 100 GiB                     | hard ceiling on the originals cache                                   |
-| `freeBelowBytes`         | 50 GiB                      | free space to protect on the volume; the effective limit adapts down  |
-| `isOriginalPinned`       | none                        | extra predicate for originals that must never be evicted              |
-| `pools`                  | fresh `RequestPools`        | the bounded request pools (sets concurrency)                          |
-| `onProgress`             | none                        | refresh/ML/precache progress callback (`RefreshEvent`)                |
-| `contentSource`          | the client's own            | override the byte source (mainly for tests)                           |
-
-Concurrency is set through `pools`: construct
-`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
-and pass it. The three pools default to 10 / 5 / 25 (see Request pools below).
-
-`lib.status()` returns a `LibraryStatus` (collection/file counts, last
-refresh/ML times and errors, originals usage and effective limit, precache
-progress, and `closed`). `lib.close()` stops the background timer; it is
-idempotent, and an in-flight refresh is left to finish.
-
-### Default reads vs. fresh reads
-
-Default reads — `lib.albums`, `lib.photos`, `lib.timeline` — answer
-synchronously from the last refreshed copy held in RAM and never touch the
-network. The background timer refreshes that copy every
-`refreshIntervalSeconds`, so a default read is immediate but may be up to one
-interval stale.
-
-`await lib.fresh()` forces a refresh, waits for it to complete and persist, and
-returns the same `{ albums, photos, timeline }` namespaces — now guaranteed to
-reflect a completed server round-trip. Concurrent `fresh()` calls coalesce onto
-one refresh, and a refresh that fails rejects the caller (default reads stay
-silent and keep serving the last good copy). The CLI's read commands use fresh
-reads (issue https://git.eeqj.de/sneak/quak/issues/75).
-
-### Read surface
-
-- `lib.albums.list()` → `Album[]`, newest-updated first.
-  `lib.albums.byID({ collectionID })` and `byName({ albumName })` →
-  `Album | undefined`.
-- `lib.photos.byID({ fileID })` → `Photo | undefined`.
-  `lib.photos.records({ fileIDs })` → `PhotoRecord[]` in the requested order,
-  each id once, unknown ids dropped.
-- `lib.timeline.groups({ groupBy, filter? })` → `TimelineGroup[]`, grouped by
-  `"day" | "week" | "month"` (keys `YYYY-MM-DD`, ISO `YYYY-Www`, `YYYY-MM`),
-  newest group first. A `PhotoFilter` combines `albumID`, `text`
-  (title/caption/album-name substring), `fileTypes`, `hasLocation`, and
-  `includeArchived`; hidden photos are always excluded.
-
-An `Album` exposes its record fields and `album.photos.list()` → `Photo[]`
-(newest first). A `Photo` exposes its record fields, `photo.record()` →
-`PhotoRecord`, and two content methods:
-
-- `await photo.original(opts?)` → `{ path, bytes }` — the full-resolution file.
-- `await photo.thumbnail(opts?)` → `{ path, bytes }`.
-
-Both serve from the on-disk content cache when the bytes are present and
-otherwise fetch through the pools; `opts.onProgress` reports per-file progress.
-They throw when the library was opened without a content source.
-
-Lower-level accessors that return decrypted model objects (which hold key
-material) are also available: `listCollections()`, `getCollection(id)`,
-`listFiles(collectionID)`, `getFile(collectionID, fileID)`, and
-`getFileByID(fileID)`.
-
-### Records and change notifications
-
-The GUI-facing records hold no key material and no binary, so they survive
-`structuredClone`/JSON across the Electron IPC boundary:
-
-- `PhotoRecord`: `fileID`, `albumIDs`, `title`, `takenAt` (milliseconds),
-  `fileType`, optional `caption` / `width` / `height` / `latitude` /
-  `longitude`, `isArchived`, `isHidden`, and `thumbnailPath` / `originalPath`
-  once the bytes are cached.
-- `AlbumRecord`: `collectionID`, `name`, `type`, `isShared`, `updationTime`, and
-  `fileIDs` (newest first).
-- `LibrarySnapshot`: `{ albums, photos, takenAt }`.
-
-`lib.snapshot()` returns a `LibrarySnapshot` (albums newest-updated first,
-photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
-(`albumsChanged`, `photosChanged`, `fileIDsRemoved`, `albumIDsRemoved`,
-`refreshedAt`) whenever a refresh alters the projection, and returns
-`{ unsubscribe }`; a refresh that changes nothing delivers nothing.
-
-### Thumbnails, ML search, and backup
-
-- `lib.thumbnails.ensure({ fileIDs, priority, signal?, onProgress? })`
-  prefetches thumbnails through the thumbnail pool, deduped by fileID, returning
-  one `EnsureResult` (`{ fileID, path?, error? }`) per file. `priority` is
-  `"visible" | "ahead" | "background"`; only `"visible"` preempts background
-  work.
-- `lib.mldata` searches the CLIP index built from Ente's per-file ML data:
-  `forFile({ fileID })` → `Promise<MLData | undefined>` (the whole stored
-  payload — face boxes, landmarks, embedding — read from disk on demand);
-  `similar({ fileID, limit? })` and `searchByEmbedding({ embedding, limit? })` →
-  `SimilarResult[]` (`{ fileID, score }`, cosine similarity, most similar first,
-  default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes
-  a query vector the caller produced elsewhere.
-- `await lib.backup(opts?)` → `BackupResult`. It refreshes, fetches every
-  in-scope original (and, with `includeThumbnails`, thumbnails) through the
-  content cache, and rebuilds the on-disk backup tree with a durable failure
-  ledger. `BackupOptions`: `downloadDirectory` (falls back to the one `open()`
-  was given), `includeOriginals` (default `true`), `includeThumbnails` (default
-  `false`), `onlyAlbumNames`, and `onProgress`. See Backup layout above for the
-  tree it writes.
-
-### Request pools
-
-`RequestPools` holds three independent bounded pools — metadata (10), content
-(5), thumbnails (25) — because Ente meters these traffic classes differently.
-Each pool orders on-demand work ahead of background/precache work and dedups
-in-flight fetches by key, and an idle pool never lends its slots to a busy one.
-
-### On-disk cache layout
-
-Under `cacheDirectory`:
-
-```
-<cacheDirectory>/
-    metadata.json                 decrypted account state + refresh cursor
-    originals/<fileID>.<ext>      cached full-resolution files
-    thumbnails/<fileID>.jpg       cached thumbnails
-    mldata/
-        <fileID>.json             one decrypted ML payload per file
-        clip.f32, clip.json       the packed CLIP index and its id list
-        fetched.json              per-file fetch bookkeeping
-```
-
-A stored file appears only via an atomic temp-then-rename, so its presence means
-it is complete. The design also calls for a content-hash comparison against
-`FileMetadata.hash` on each fetched original; that check is deferred (issue
-https://git.eeqj.de/sneak/quak/issues/68) because the exact hash construction
-cannot yet be confirmed against the repo's fixtures.
-
-### Key types by source file
-
-- `src/library/index.ts`: `Library`, `LibraryOptions`, `LibraryStatus`,
-  `LibraryClient`, `RefreshEvent`
-- `src/library/read.ts`: `Album`, `Photo`, `AlbumsAPI`, `PhotosAPI`,
-  `TimelineAPI`, `PhotoFilter`, `TimelineGroup`, `GroupBy`
-- `src/library/content.ts`: `ContentResult`, `ContentOptions`, `ThumbnailsAPI`,
-  `EnsureOptions`, `EnsureResult`, `ContentSource`
-- `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`,
-  `LibraryChange`
-- `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult`
-- `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool`
-- `src/backup.ts`: `BackupOptions`, `BackupResult`, `BackupError`
 - `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot`
-- `src/api/client.ts`: `ApiClient`, `ApiClientOptions`, `StreamOptions`
+- `src/api/client.ts`: `ApiClient`, `ApiClientOptions`, `ApiError`,
+  `StreamOptions`
 - `src/errors.ts`: `ApiError`, `TruncatedStreamError`
 - `src/retry.ts`: `withRetry`, `isRetryable`, `isSafeToReplay`, `RetryOptions`
-- `src/model/types.ts`: `Collection`, `EnteFile`, `FileMetadata`, `FileType`,
+- `src/auth/types.ts`: `KeyAttributes`, `SRPAttributes`,
-  `CollectionType`, `RawCollection`, `RawEnteFile`
+  `AuthorizationResponse`, `LoginChallenge`
+- `src/model/types.ts`: `Collection`, `EnteFile`, `FileMetadata`, `FileBlob`,
+  `RawCollection`, `RawEnteFile`, `RawMagicMetadata`
+- `src/download/index.ts`: `DownloadResult`
+- `src/backup.ts`: `BackupResult`, `BackupError`
 - `src/thumbnails.ts`: `MissingThumbnailInfo`, `ThumbnailFixResult`
 
 ## Source attribution

TODO.md

@@ -14,19 +14,10 @@ pre-1.0
 
 # Next Step
 
-Tag v1.0.0.
+Update the README API reference section to match the current implementation.
 
 # Completed Steps
 
-- 2026-09-22: Rewrote the README API reference (and the Getting Started / usage
-  snippets) to match the shipped cache/API library on `next` (issue 53, issue
-  13). Documented `Library.open` and its options, the default-read vs `fresh()`
-  distinction (and that the CLI's read commands are fresh), the record types and
-  `snapshot()`/`subscribe()`, the `albums`/`photos`/`timeline` read surface,
-  `Photo` content methods, `thumbnails.ensure`, the `mldata` search surface,
-  `backup()`, the three request pools (10/5/25), and the on-disk cache layout;
-  noted the deferred content-hash integrity check (issue 68). Docs-only; no code
-  changed.
 - 2026-09-22: Added resumable, deletion-aware enumeration to `Client` (issue 38,
   closes issue 7). `collectionsSince`/`filesSince` take a starting cursor,
   decrypt live records, surface tombstoned ids in a separate `deleted` list (a
@@ -131,6 +122,7 @@ Tag v1.0.0.
 
 # Future Steps
 
+- Tag v1.0.0.
 - Future desktop client, separate repo:
     - Electron app skeleton consuming this library.
     - Local SQLite cache keyed on (collectionID, fileID, updationTime).

bin/quak.ts

@@ -2,26 +2,13 @@
 
 import { input, password as passwordPrompt } from "@inquirer/prompts";
 import { stdout, stderr } from "node:process";
-import {
+import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
-    copyFileSync,
-    existsSync,
-    mkdirSync,
-    readFileSync,
-    writeFileSync,
-} from "node:fs";
 import { join } from "node:path";
 import { Command } from "commander";
 import envPaths from "env-paths";
 import { Client, type ClientSnapshot } from "../src/client.js";
 import { init } from "../src/crypto/index.js";
-import { Library, type LibraryClient } from "../src/library/index.js";
+import { Library } from "../src/library/index.js";
-import {
-    fileListRow,
-    fileListLine,
-    originalName,
-    thumbnailName,
-} from "../src/cli-output.js";
-import { freshCollections, freshFiles, freshFile } from "../src/cli-read.js";
 import { runMetadataBackup } from "../src/metadata-backup.js";
 import {
     listMissingThumbnails,
@@ -68,61 +55,7 @@ const program = new Command();
 program
     .name("quak")
     .description("CLI for the Ente end-to-end encrypted photo service")
-    .version("0.0.0")
+    .version("0.0.0");
-    .option(
-        "--cache-dir <path>",
-        "Directory for the local metadata/content cache " +
-            "(default: the per-user cache directory)",
-    );
-
-// The `--cache-dir` global, or undefined to let the library pick its per-user
-// default keyed by the account id.
-const cacheDirOption = (): string | undefined =>
-    program.opts<{ cacheDir?: string }>().cacheDir;
-
-// A library client that omits `fetchMLData`, so the point commands below do not
-// kick the library's background ML backfill: they read metadata, or fetch one
-// file's content, and exit. `backup` and `backup-metadata` handle ML on their
-// own terms. The content source is kept so `get`/`get-thumb`/`--exif` can fetch
-// originals through the on-disk cache.
-const readLibraryClient = (client: Client): LibraryClient => ({
-    whoami: () => client.whoami(),
-    collectionsSince: (args) => client.collectionsSince(args),
-    filesSince: (args) => client.filesSince(args),
-    contentSource: () => client.contentSource(),
-});
-
-// Open a library for a single point command: the aggressive background precache
-// (issue #48) is off — a one-shot `collections` or `get` must not start
-// downloading the whole account — and the refresh interval is long so no second
-// refresh fires mid-command.
-const openReadLibrary = (client: Client): Promise<Library> =>
-    Library.open({
-        client: readLibraryClient(client),
-        cacheDirectory: cacheDirOption(),
-        refreshIntervalSeconds: 3600,
-        precacheThumbnails: false,
-        precacheOriginals: false,
-    });
-
-// Close the library and exit once stdout/stderr have drained. `process.exit`
-// alone can truncate buffered piped output, and the library keeps the event
-// loop alive with a background refresh, so a plain return could hang; this does
-// neither.
-const finish = (lib: Library | undefined, code: number): void => {
-    lib?.close();
-    const pending = [stdout, stderr].filter((s) => s.writableLength > 0);
-    if (pending.length === 0) {
-        process.exit(code);
-        return;
-    }
-    let remaining = pending.length;
-    for (const s of pending) {
-        s.once("drain", () => {
-            if (--remaining === 0) process.exit(code);
-        });
-    }
-};
 
 program
     .command("login")
@@ -183,11 +116,7 @@ program
     .action(async (opts: { json?: boolean }) => {
         await init();
         const client = requireSession();
-        const lib = await openReadLibrary(client);
+        const collections = await client.listCollections();
-        // Force a server round-trip and list in enumeration order (issue #36
-        // amendment, issue #52): the pre-library CLI printed current state in
-        // this order, not the albums projection's newest-first order.
-        const collections = await freshCollections(lib);
 
         if (opts.json) {
             stdout.write(
@@ -211,7 +140,6 @@ program
                 );
             }
         }
-        finish(lib, 0);
     });
 
 program
@@ -231,29 +159,36 @@ program
             process.exit(1);
         }
 
-        const lib = await openReadLibrary(client);
+        const collections = await client.listCollections();
-        // Force a server round-trip and list in enumeration order (issue #36
+        const col = collections.find((c) => c.id === collectionID);
-        // amendment, issue #52). Each file prints from its own decrypted
+        if (!col) {
-        // metadata (raw title, microsecond creationTime) via cli-output, and in
-        // the pre-library CLI's enumeration order, not the projection's
-        // newest-first order.
-        const files = await freshFiles(lib, collectionID);
-        if (!files) {
             stderr.write(`Collection ${collectionID} not found\n`);
-            finish(lib, 1);
+            process.exit(1);
-            return;
         }
 
+        const files = await client.listFiles(col.id, col.key);
+
         if (opts.json) {
             stdout.write(
-                JSON.stringify(files.map(fileListRow), null, 2) + "\n",
+                JSON.stringify(
+                    files.map((f) => ({
+                        id: f.id,
+                        title: f.metadata.title,
+                        fileType: f.metadata.fileType,
+                        creationTime: f.metadata.creationTime,
+                        collectionID: f.collectionID,
+                    })),
+                    null,
+                    2,
+                ) + "\n",
             );
         } else {
-            for (const file of files) {
+            for (const f of files) {
-                stdout.write(fileListLine(file) + "\n");
+                stdout.write(
+                    `${f.id}\t${f.metadata.fileType}\t${f.metadata.title}\n`,
+                );
             }
         }
-        finish(lib, 0);
     });
 
 program
@@ -261,70 +196,98 @@ program
     .description("Download and decrypt a single file")
     .argument("<fileID>", "File ID (from `quak files`)")
     .option("--out <path>", "Output file path")
-    .option("--collection <id>", "Accepted for compatibility; ignored")
+    .option(
-    .action(async (fileIDStr: string, opts: { out?: string }) => {
+        "--collection <id>",
-        await init();
+        "Collection ID (required to look up the file key)",
-        const client = requireSession();
+    )
-        const fileID = Number(fileIDStr);
+    .action(
-        if (!Number.isFinite(fileID)) {
+        async (
-            stderr.write("Invalid file ID\n");
+            fileIDStr: string,
-            process.exit(1);
+            opts: { out?: string; collection?: string },
-        }
+        ) => {
+            await init();
+            const client = requireSession();
+            const fileID = Number(fileIDStr);
+            if (!Number.isFinite(fileID)) {
+                stderr.write("Invalid file ID\n");
+                process.exit(1);
+            }
 
-        const lib = await openReadLibrary(client);
+            const collections = await client.listCollections();
-        // Force a server round-trip so the file resolves against current state
+            let targetCol;
-        // (issue #36 amendment, issue #52).
+            if (opts.collection) {
-        const resolved = await freshFile(lib, fileID);
+                targetCol = collections.find(
-        if (!resolved) {
+                    (c) => c.id === Number(opts.collection),
+                );
+            }
+
+            // Search all collections (or the specified one) for the file
+            const searchCols = targetCol ? [targetCol] : collections;
+            for (const col of searchCols) {
+                const files = await client.listFiles(col.id, col.key);
+                const file = files.find((f) => f.id === fileID);
+                if (file) {
+                    const result = await client.downloadFile(file, opts.out);
+                    stderr.write(
+                        `${result.bytesWritten} bytes -> ${result.path}\n`,
+                    );
+                    return;
+                }
+            }
             stderr.write(`File ${fileID} not found\n`);
-            finish(lib, 1);
+            process.exit(1);
-            return;
+        },
-        }
+    );
-        const { photo, file } = resolved;
-
-        const result = await photo.original();
-        // Default name is the file's own title, as the pre-library CLI used
-        // (not the editedName-preferring projection title) (issue #52).
-        const outPath = opts.out ?? originalName(file);
-        copyFileSync(result.path, outPath);
-        stderr.write(`${result.bytes} bytes -> ${outPath}\n`);
-        finish(lib, 0);
-    });
 
 program
     .command("get-thumb")
     .description("Download and decrypt a thumbnail")
     .argument("<fileID>", "File ID (from `quak files`)")
     .option("--out <path>", "Output file path")
-    .option("--collection <id>", "Accepted for compatibility; ignored")
+    .option(
-    .action(async (fileIDStr: string, opts: { out?: string }) => {
+        "--collection <id>",
-        await init();
+        "Collection ID (required to look up the file key)",
-        const client = requireSession();
+    )
-        const fileID = Number(fileIDStr);
+    .action(
-        if (!Number.isFinite(fileID)) {
+        async (
-            stderr.write("Invalid file ID\n");
+            fileIDStr: string,
-            process.exit(1);
+            opts: { out?: string; collection?: string },
-        }
+        ) => {
+            await init();
+            const client = requireSession();
+            const fileID = Number(fileIDStr);
+            if (!Number.isFinite(fileID)) {
+                stderr.write("Invalid file ID\n");
+                process.exit(1);
+            }
 
-        const lib = await openReadLibrary(client);
+            const collections = await client.listCollections();
-        // Force a server round-trip so the file resolves against current state
+            let targetCol;
-        // (issue #36 amendment, issue #52).
+            if (opts.collection) {
-        const resolved = await freshFile(lib, fileID);
+                targetCol = collections.find(
-        if (!resolved) {
+                    (c) => c.id === Number(opts.collection),
+                );
+            }
+
+            const searchCols = targetCol ? [targetCol] : collections;
+            for (const col of searchCols) {
+                const files = await client.listFiles(col.id, col.key);
+                const file = files.find((f) => f.id === fileID);
+                if (file) {
+                    const result = await client.downloadThumbnail(
+                        file,
+                        opts.out,
+                    );
+                    stderr.write(
+                        `${result.bytesWritten} bytes -> ${result.path}\n`,
+                    );
+                    return;
+                }
+            }
             stderr.write(`File ${fileID} not found\n`);
-            finish(lib, 1);
+            process.exit(1);
-            return;
+        },
-        }
+    );
-        const { photo, file } = resolved;
-
-        const result = await photo.thumbnail();
-        // Default name is thumb_<file's own title>, as the pre-library CLI
-        // used (not the projection title) (issue #52).
-        const outPath = opts.out ?? thumbnailName(file);
-        copyFileSync(result.path, outPath);
-        stderr.write(`${result.bytes} bytes -> ${outPath}\n`);
-        finish(lib, 0);
-    });
 
 program
     .command("backup-metadata")
@@ -340,12 +303,10 @@ program
     .action(async (dir: string, opts: { exif?: boolean; all?: boolean }) => {
         await init();
         const client = requireSession();
-        const lib = await openReadLibrary(client);
+        await runMetadataBackup(client, dir, {
-        await runMetadataBackup(lib, client, dir, {
             exif: opts.exif || opts.all,
             onProgress: (msg) => stderr.write(msg + "\n"),
         });
-        finish(lib, 0);
     });
 
 program
@@ -360,17 +321,14 @@ program
         const client = requireSession();
 
         stderr.write("Starting backup...\n");
-        const lib = await Library.open({
+        const lib = await Library.open({ client, downloadDirectory: dir });
-            client,
-            downloadDirectory: dir,
-            cacheDirectory: cacheDirOption(),
-        });
         const result = await lib.backup({
             downloadDirectory: dir,
             onProgress: (msg) => {
                 if (!opts.json) stderr.write(msg + "\n");
             },
         });
+        lib.close();
 
         if (opts.json) {
             stdout.write(JSON.stringify(result, null, 2) + "\n");
@@ -390,7 +348,7 @@ program
             }
         }
 
-        finish(lib, result.failed > 0 ? 1 : 0);
+        process.exit(result.failed > 0 ? 1 : 0);
     });
 
 const helper = program
@@ -404,8 +362,7 @@ helper
     .action(async (opts: { json?: boolean }) => {
         await init();
         const client = requireSession();
-        const lib = await openReadLibrary(client);
+        const missing = await listMissingThumbnails(client, (msg) => {
-        const missing = await listMissingThumbnails(lib, client, (msg) => {
             if (!opts.json) stderr.write(msg + "\n");
         });
 
@@ -425,7 +382,6 @@ helper
                 }
             }
         }
-        finish(lib, 0);
     });
 
 helper
@@ -441,61 +397,44 @@ helper
     .action(async (opts: { file?: string[]; json?: boolean }) => {
         await init();
         const client = requireSession();
-        const lib = await openReadLibrary(client);
 
         let fileIDs: number[];
         if (opts.file && opts.file.length > 0) {
             fileIDs = opts.file.map(Number).filter(Number.isFinite);
         } else {
             stderr.write("Scanning for missing thumbnails...\n");
-            const missing = await listMissingThumbnails(lib, client, (msg) => {
+            const missing = await listMissingThumbnails(client, (msg) => {
                 if (!opts.json) stderr.write(msg + "\n");
             });
             fileIDs = missing.map((m) => m.fileID);
             if (fileIDs.length === 0) {
                 stderr.write("No missing thumbnails found.\n");
-                finish(lib, 0);
                 return;
             }
             stderr.write(`Found ${fileIDs.length} file(s) to fix.\n`);
         }
 
-        const results = await fixMissingThumbnails(
+        const results = await fixMissingThumbnails(client, fileIDs, (msg) => {
-            lib,
+            if (!opts.json) stderr.write(msg + "\n");
-            client,
+        });
-            fileIDs,
-            (msg) => {
-                if (!opts.json) stderr.write(msg + "\n");
-            },
-        );
 
         if (opts.json) {
             stdout.write(JSON.stringify(results, null, 2) + "\n");
         } else {
-            const fixed = results.filter((r) => r.status === "fixed").length;
+            const ok = results.filter((r) => r.success).length;
-            const skipped = results.filter(
+            const fail = results.filter((r) => !r.success).length;
-                (r) => r.status === "skipped",
-            ).length;
-            const failed = results.filter((r) => r.status === "failed").length;
             stderr.write(`\n--- Done ---\n`);
-            stderr.write(`  Fixed:   ${fixed}\n`);
+            stderr.write(`  Fixed:  ${ok}\n`);
-            stderr.write(`  Skipped: ${skipped}\n`);
+            stderr.write(`  Failed: ${fail}\n`);
-            stderr.write(`  Failed:  ${failed}\n`);
+            if (fail > 0) {
-            if (skipped > 0) {
-                stderr.write("\nSkipped (unsupported format):\n");
-                for (const r of results.filter((r) => r.status === "skipped")) {
-                    stderr.write(`  ${r.fileID}\t${r.title}\t${r.reason}\n`);
-                }
-            }
-            if (failed > 0) {
                 stderr.write("\nFailed files:\n");
-                for (const r of results.filter((r) => r.status === "failed")) {
+                for (const r of results.filter((r) => !r.success)) {
-                    stderr.write(`  ${r.fileID}\t${r.title}\t${r.reason}\n`);
+                    stderr.write(`  ${r.fileID}\t${r.title}\t${r.error}\n`);
                 }
             }
         }
 
-        finish(lib, results.some((r) => r.status === "failed") ? 1 : 0);
+        process.exit(results.some((r) => !r.success) ? 1 : 0);
     });
 
 await init();

src/cli-output.ts

@@ -1,40 +0,0 @@
-// How the CLI presents a file's identity in `files`, `get`, and `get-thumb`.
-//
-// These read the file's own decrypted metadata — the raw title and the
-// creationTime in microseconds — rather than the `PhotoRecord` projection the
-// rest of the library exposes. The projection prefers `editedName`/`editedTime`
-// and reports time in milliseconds, which is right for a photo browser but
-// would change the CLI's externally-visible output. The pre-library CLI printed
-// `metadata.title` and `metadata.creationTime` and named downloads after
-// `metadata.title`, and issue #52 requires that output stay byte-identical, so
-// the commands shape their output from the raw `EnteFile` through here.
-
-import type { EnteFile, FileType, Microseconds } from "./model/types.js";
-
-// One row of `quak files --json`.
-export interface FileListRow {
-    id: number;
-    title: string;
-    fileType: FileType;
-    creationTime: Microseconds;
-    collectionID: number;
-}
-
-export const fileListRow = (file: EnteFile): FileListRow => ({
-    id: file.id,
-    title: file.metadata.title,
-    fileType: file.metadata.fileType,
-    creationTime: file.metadata.creationTime,
-    collectionID: file.collectionID,
-});
-
-// One line of `quak files` in its human, tab-separated form.
-export const fileListLine = (file: EnteFile): string =>
-    `${file.id}\t${file.metadata.fileType}\t${file.metadata.title}`;
-
-// Default output path for `quak get` when `--out` is not given.
-export const originalName = (file: EnteFile): string => file.metadata.title;
-
-// Default output path for `quak get-thumb` when `--out` is not given.
-export const thumbnailName = (file: EnteFile): string =>
-    `thumb_${file.metadata.title}`;

src/cli-read.ts

@@ -1,62 +0,0 @@
-// How the CLI's read commands obtain current data.
-//
-// `collections`, `files --collection`, `get`, and `get-thumb` must answer for
-// the account's state at the moment the command runs, not for whatever the
-// local cache last happened to hold (owner amendment, issue #36). Each helper
-// therefore forces a server round-trip through `Library.fresh()` and only then
-// reads — so a collection, file, or metadata change made elsewhere is visible.
-//
-// `collections` and `files` also list in the library's own enumeration order —
-// `listCollections()`/`listFiles()`, the order the pre-library CLI printed —
-// rather than the `albums`/`photos` projection's newest-first order, which
-// re-sorts the rows. The field values still come from each record's raw
-// metadata via `cli-output.ts`.
-
-import type { Collection, EnteFile } from "./model/types.js";
-import type { Photo, PhotosAPI } from "./library/index.js";
-
-// The slice of `Library` these helpers read. `Library` satisfies it
-// structurally; a test can drive them with a stand-in that records the
-// `fresh()` call and serves records in a known enumeration order.
-export interface FreshReadLibrary {
-    fresh(): Promise<unknown>;
-    listCollections(): Collection[];
-    getCollection(id: number): Collection | undefined;
-    listFiles(collectionID: number): EnteFile[];
-    getFileByID(fileID: number): EnteFile | undefined;
-    photos: Pick<PhotosAPI, "byID">;
-}
-
-// Every live collection, current as of a forced refresh, in enumeration order.
-export const freshCollections = async (
-    lib: FreshReadLibrary,
-): Promise<Collection[]> => {
-    await lib.fresh();
-    return lib.listCollections();
-};
-
-// The files of one collection, current as of a forced refresh, in enumeration
-// order. `undefined` (not an empty list) when the collection does not exist, so
-// the caller can tell "no such collection" from "an empty collection".
-export const freshFiles = async (
-    lib: FreshReadLibrary,
-    collectionID: number,
-): Promise<EnteFile[] | undefined> => {
-    await lib.fresh();
-    if (!lib.getCollection(collectionID)) return undefined;
-    return lib.listFiles(collectionID);
-};
-
-// One file, current as of a forced refresh, resolved to both its content
-// handle (`Photo`, for fetching bytes) and its raw record (`EnteFile`, for the
-// default output name and field values). `undefined` when the file is unknown.
-export const freshFile = async (
-    lib: FreshReadLibrary,
-    fileID: number,
-): Promise<{ photo: Photo; file: EnteFile } | undefined> => {
-    await lib.fresh();
-    const photo = lib.photos.byID({ fileID });
-    const file = lib.getFileByID(fileID);
-    if (!photo || !file) return undefined;
-    return { photo, file };
-};

src/library/index.ts

@@ -6,17 +6,10 @@
 // existing copy loaded, that first refresh runs in the background and `open()`
 // returns as soon as the cached data is ready to serve — a slow or unreachable
 // server no longer stalls opening. A background timer then refreshes every
-// `refreshIntervalSeconds`. Every default read is answered from RAM — no
+// `refreshIntervalSeconds`. Every read is answered from RAM — no read touches
-// default read touches the network. There is deliberately no `sync()`, no
+// the network. There is deliberately no `sync()`, no `refresh()`, no
-// `refresh()`, no `serverReachable` flag, and no "before each read" mode
+// `serverReachable` flag, and no "before each read" mode (design #36): the
-// (design #36).
+// only ways state changes are the refreshes above.
-//
-// `fresh()` is the one exception (issue #75, an owner amendment to #36): it
-// forces a refresh, awaits it, and only then hands back the read namespaces, so
-// a caller that needs server-current data can ask for it. Concurrent `fresh()`
-// calls coalesce onto one in-flight refresh, and a refresh that fails rejects
-// the caller (the default reads stay silent and serve the last good copy). The
-// default methods and the background loop are unchanged.
 //
 // A refresh stages all of its network work first and only mutates the store
 // once every fetch has succeeded. A refresh that fails partway therefore never
@@ -47,7 +40,6 @@ import {
     type AlbumsAPI,
     type PhotosAPI,
     type TimelineAPI,
-    type FreshReads,
 } from "./read.js";
 import {
     ContentCache,
@@ -56,7 +48,6 @@ import {
     type EnsureOptions,
     type EnsureResult,
 } from "./content.js";
-import { makeMLDataAPI, type MLDataAPI } from "./mlsearch.js";
 import { Precache } from "./precache.js";
 
 export {
@@ -65,7 +56,6 @@ export {
     type AlbumsAPI,
     type PhotosAPI,
     type TimelineAPI,
-    type FreshReads,
     type PhotoFilter,
     type TimelineGroup,
     type GroupBy,
@@ -82,7 +72,6 @@ export {
     type EnsureResult,
     type EnsureEvent,
 } from "./content.js";
-export { type MLDataAPI, type SimilarResult } from "./mlsearch.js";
 import type { CollectionsPage, FilesPage } from "../client.js";
 import { MLDATA_BATCH_SIZE, type MLData } from "../mldata-fetch.js";
 import type { Collection, EnteFile } from "../model/types.js";
@@ -239,10 +228,6 @@ export class Library {
     // The thumbnail-prefetch surface (issue #46): drives the thumbnail pool
     // with priority, dedup, and abort.
     readonly thumbnails: ThumbnailsAPI;
-    // The content-similarity search surface over the CLIP index (issue #50).
-    // Present whether or not ML fetching is enabled; with no ML store it
-    // returns empty results.
-    readonly mldata: MLDataAPI;
 
     private readonly client: LibraryClient;
     private readonly store: MetadataStore;
@@ -254,17 +239,12 @@ export class Library {
     private readonly onProgress?: RefreshProgressCallback;
     private readonly pools: RequestPools;
     // The ML-data cache, present only when the client can fetch ML data.
-    private readonly mlStore?: MLDataStore;
+    private readonly mldata?: MLDataStore;
     // The local precache (#48), present only when the content cache is.
     private readonly precache?: Precache;
 
     private timer?: ReturnType<typeof setTimeout>;
-    // The in-flight refresh cycle, or undefined when none runs. One slot serves
+    private refreshing = false;
-    // both paths: the background loop skips when it is set, and a fresh read
-    // (issue #75) coalesces onto it or starts one. The promise carries the
-    // cycle's real outcome (it rejects on failure); the background loop ignores
-    // that, a fresh read propagates it.
-    private cycle?: Promise<void>;
     // Guards the ML fetch pass so a slow backfill never runs twice at once; a
     // refresh whose pass is still running kicks nothing new.
     private mlFetching = false;
@@ -305,7 +285,7 @@ export class Library {
         this.intervalMs = args.intervalMs;
         this.onProgress = args.onProgress;
         this.pools = args.pools;
-        this.mlStore = args.mldata;
+        this.mldata = args.mldata;
         this.cache = args.cache;
         this.precache = args.precache;
         this.lastRecords = this.deriveNow();
@@ -328,8 +308,6 @@ export class Library {
                 return this.cache.ensureThumbnails(opts);
             },
         };
-        // Reads the ML store live so results grow as ML data is fetched.
-        this.mldata = makeMLDataAPI(() => this.mlStore);
     }
 
     // Load the cache and start the refresh loop. With an empty cache the first
@@ -447,13 +425,6 @@ export class Library {
         return this.store.getFile(collectionID, fileID);
     }
 
-    // Any membership of a file, addressed by file id alone. A file's own
-    // metadata (title, creationTime) is identical across the collections it
-    // belongs to, so this serves the point commands that hold only a fileID.
-    getFileByID(fileID: number): EnteFile | undefined {
-        return this.store.getFileByID(fileID);
-    }
-
     // A synchronous, RAM-only projection of the whole library into plain
     // records (no keys), the surface the GUI reads across IPC. Photos are
     // deduplicated to one record per file and ordered newest first.
@@ -482,7 +453,7 @@ export class Library {
         for (const c of collections) {
             files += this.store.listFiles(c.id).length;
         }
-        const ml = this.mlStore?.stats();
+        const ml = this.mldata?.stats();
         const originals = this.cache?.originalsStatus();
         const pre = this.precache?.status();
         return {
@@ -505,21 +476,6 @@ export class Library {
         };
     }
 
-    // Fresh reads (issue #75, owner amendment to design #36). Force a refresh,
-    // wait for it to complete and persist, then hand back the same
-    // `albums`/`photos`/`timeline` namespaces — now guaranteed to reflect a
-    // completed server round-trip. Concurrent calls coalesce onto one refresh;
-    // a refresh that fails rejects here, where the default namespaces would
-    // instead stay silent and serve the last good copy.
-    async fresh(): Promise<FreshReads> {
-        await this.refreshNow();
-        return {
-            albums: this.albums,
-            photos: this.photos,
-            timeline: this.timeline,
-        };
-    }
-
     // Back up every in-scope file to `downloadDirectory` in the historical
     // on-disk layout, with a durable failure ledger (issue #51). Refreshes
     // first, fetches pending originals (and optional thumbnails) through the
@@ -579,48 +535,11 @@ export class Library {
         this.timer.unref?.();
     }
 
-    // The background loop's refresh: run a cycle unless one is already in flight
+    // One refresh cycle, guarded so a failure never escapes and overlapping
-    // (or the library is closed), and never let a failure escape — the
+    // cycles never run. Errors are reported, not thrown.
-    // background path reports errors through `status()`/`onProgress`, it does
+    private async runRefresh(): Promise<void> {
-    // not throw. Resolves once the cycle it started (or skipped past) settles.
+        if (this.closed || this.refreshing) return;
-    private runRefresh(): Promise<void> {
+        this.refreshing = true;
-        if (this.closed || this.cycle) return Promise.resolve();
-        return this.startCycle().catch(() => {});
-    }
-
-    // A fresh read's refresh (issue #75): force a cycle and await it, rejecting
-    // if it fails. Concurrent fresh reads coalesce onto the one in-flight cycle
-    // — the background loop's included — so they never fan out into redundant
-    // server round-trips.
-    private refreshNow(): Promise<void> {
-        if (this.closed) {
-            return Promise.reject(new Error("the library is closed"));
-        }
-        return this.cycle ?? this.startCycle();
-    }
-
-    // Start one refresh cycle and record it as the in-flight cycle so every
-    // caller coalesces onto it. The returned promise carries the cycle's real
-    // outcome; each caller attaches the handling its own path needs, and the
-    // slot is cleared once the cycle settles.
-    private startCycle(): Promise<void> {
-        const cycle = this.refreshCycle();
-        this.cycle = cycle;
-        void cycle.then(
-            () => {
-                if (this.cycle === cycle) this.cycle = undefined;
-            },
-            () => {
-                if (this.cycle === cycle) this.cycle = undefined;
-            },
-        );
-        return cycle;
-    }
-
-    // One refresh cycle: the network fetch and commit, wrapped in the progress
-    // events and status bookkeeping. Throws when the refresh fails so a fresh
-    // read can reject; `runRefresh` swallows that throw for the background loop.
-    private async refreshCycle(): Promise<void> {
         this.emit({ operation: "refresh", status: "started" });
         try {
             await this.refreshOnce();
@@ -636,7 +555,8 @@ export class Library {
             const error = err instanceof Error ? err.message : String(err);
             this.lastError = error;
             this.emit({ operation: "refresh", status: "failed", error });
-            throw err;
+        } finally {
+            this.refreshing = false;
         }
     }
 
@@ -741,7 +661,7 @@ export class Library {
     // advanced), through the metadata pool, and update the CLIP index. Guarded
     // so passes never overlap; a failure is reported, not thrown.
     private async runMLFetch(): Promise<void> {
-        const mldata = this.mlStore;
+        const mldata = this.mldata;
         // Bind so the call keeps the client as its receiver when invoked
         // through the pool below.
         const fetchMLData = this.client.fetchMLData?.bind(this.client);

src/library/mlsearch.ts

@@ -1,129 +0,0 @@
-// The content-similarity search surface over the CLIP index (issue #50).
-//
-// This is `lib.mldata`. It answers three questions against the ML-data cache
-// (#49) without touching the network:
-//
-//   - `forFile` returns the whole stored payload (face boxes, landmarks,
-//     embedding) for a file, read from disk on demand — the only method here
-//     that touches the disk, and the only one that is async.
-//   - `similar` and `searchByEmbedding` rank fileIDs by cosine similarity over
-//     the packed `Float32Array` index alone. That index (~50k×512) already
-//     lives in RAM, so each query is a plain loop over it and nothing else.
-//
-// quak bundles no text encoder (owner-deferred), so `searchByEmbedding` takes
-// the query vector the caller has produced elsewhere; `similar` uses the
-// query file's own indexed embedding.
-
-import type { MLData } from "../mldata-fetch.js";
-import type { MLDataStore, MLIndex } from "./mldata.js";
-
-// How many nearest files a query returns when the caller names no limit.
-const DEFAULT_LIMIT = 20;
-
-// One ranked result: a fileID and its cosine similarity to the query, in
-// [-1, 1]. Callers wanting only the ids read `.fileID`.
-export interface SimilarResult {
-    fileID: number;
-    score: number;
-}
-
-export interface MLDataAPI {
-    // The whole stored ML payload for a file, or undefined when it is not
-    // cached. Reads the payload from disk, so it is async.
-    forFile(args: { fileID: number }): Promise<MLData | undefined>;
-    // The files nearest the given file by cosine over their CLIP embeddings,
-    // most similar first, excluding the file itself. Empty when the file has
-    // no indexed embedding.
-    similar(args: { fileID: number; limit?: number }): SimilarResult[];
-    // The files nearest a caller-supplied query embedding by cosine, most
-    // similar first. Empty when the query is the wrong length for the index,
-    // has zero magnitude, or the index is empty.
-    searchByEmbedding(args: {
-        embedding: ArrayLike<number>;
-        limit?: number;
-    }): SimilarResult[];
-}
-
-// Rank the packed index by cosine similarity to `query`, most similar first,
-// and return the top `limit`. `skip` (a query file's own id) is left out. Both
-// each row's magnitude and the query's are computed here rather than cached:
-// the index mutates as ML data is fetched, and one plain pass over ~50k×512
-// floats is fast enough that a norm cache would only add a staleness bug. A
-// zero-magnitude vector has no direction, so it is dropped rather than divided
-// by zero.
-const topByCosine = (
-    index: MLIndex,
-    query: ArrayLike<number>,
-    limit: number,
-    skip?: number,
-): SimilarResult[] => {
-    const { fileIDs, embeddingLength, embeddings } = index;
-    if (embeddingLength === 0 || query.length !== embeddingLength) return [];
-
-    // Every indexed read below is in range: the inner loops run to
-    // `embeddingLength`, the query is exactly that long (checked above), and
-    // the packed buffer holds `fileIDs.length * embeddingLength` floats.
-    // `noUncheckedIndexedAccess` still widens each read to `number | undefined`,
-    // so they are asserted non-null rather than paying a per-element guard in
-    // this hot ~50k×512 loop.
-    let queryNorm = 0;
-    for (let k = 0; k < embeddingLength; k++) {
-        const q = query[k]!;
-        queryNorm += q * q;
-    }
-    queryNorm = Math.sqrt(queryNorm);
-    if (queryNorm === 0) return [];
-
-    const results: SimilarResult[] = [];
-    for (let i = 0; i < fileIDs.length; i++) {
-        const id = fileIDs[i]!;
-        if (id === skip) continue;
-        const base = i * embeddingLength;
-        let dot = 0;
-        let norm = 0;
-        for (let k = 0; k < embeddingLength; k++) {
-            const v = embeddings[base + k]!;
-            dot += query[k]! * v;
-            norm += v * v;
-        }
-        if (norm === 0) continue;
-        results.push({
-            fileID: id,
-            score: dot / (queryNorm * Math.sqrt(norm)),
-        });
-    }
-
-    // Descending score, ties broken by ascending fileID for a stable order.
-    results.sort((a, b) => b.score - a.score || a.fileID - b.fileID);
-    return results.slice(0, Math.max(0, Math.trunc(limit)));
-};
-
-// Build the search surface over a store the library supplies lazily (the store
-// is absent when the client cannot fetch ML data). Reading it per call keeps
-// the surface current as the index grows.
-export const makeMLDataAPI = (
-    store: () => MLDataStore | undefined,
-): MLDataAPI => ({
-    forFile: ({ fileID }): Promise<MLData | undefined> => {
-        const s = store();
-        return s ? s.readPayload(fileID) : Promise.resolve(undefined);
-    },
-    similar: ({ fileID, limit }): SimilarResult[] => {
-        const s = store();
-        if (!s) return [];
-        const index = s.getIndex();
-        const pos = index.fileIDs.indexOf(fileID);
-        if (pos < 0) return [];
-        const base = pos * index.embeddingLength;
-        const query = index.embeddings.subarray(
-            base,
-            base + index.embeddingLength,
-        );
-        return topByCosine(index, query, limit ?? DEFAULT_LIMIT, fileID);
-    },
-    searchByEmbedding: ({ embedding, limit }): SimilarResult[] => {
-        const s = store();
-        if (!s) return [];
-        return topByCosine(s.getIndex(), embedding, limit ?? DEFAULT_LIMIT);
-    },
-});

src/library/read.ts

@@ -193,15 +193,6 @@ export interface TimelineAPI {
     groups(args: { groupBy: GroupBy; filter?: PhotoFilter }): TimelineGroup[];
 }
 
-// The surface `Library.fresh()` resolves to (issue #75). It is the same three
-// read namespaces as the default `albums`/`photos`/`timeline`, handed back only
-// after a forced refresh has brought the local copy current.
-export interface FreshReads {
-    albums: AlbumsAPI;
-    photos: PhotosAPI;
-    timeline: TimelineAPI;
-}
-
 export const makeAlbumsAPI = (
     derive: () => DerivedRecords,
     content?: PhotoContent,

src/metadata-backup.ts

@@ -1,9 +1,15 @@
-import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import {
+    mkdirSync,
+    mkdtempSync,
+    readFileSync,
+    rmSync,
+    writeFileSync,
+} from "node:fs";
 import { join } from "node:path";
+import { tmpdir } from "node:os";
 import * as jpeg from "jpeg-js";
 import exifReader from "exif-reader";
 import type { Client } from "./client.js";
-import type { Library, Photo } from "./library/index.js";
 import { fetchMLData } from "./mldata-fetch.js";
 import type { EnteFile } from "./model/types.js";
 
@@ -98,29 +104,24 @@ const extractImageMetadata = (
     }
 };
 
-// Read a file's original bytes through the library's content cache and extract
-// its embedded image metadata. The bytes come from `photo.original()` — the
-// same on-disk cache the rest of the library fills — rather than a fresh
-// per-call download to a throwaway temp file.
 const extractExif = async (
-    photo: Photo,
+    client: Client,
+    file: EnteFile,
 ): Promise<Record<string, unknown> | undefined> => {
+    const tmpDir = mkdtempSync(join(tmpdir(), "quak-exif-"));
     try {
-        const { path } = await photo.original();
+        const origPath = join(tmpDir, "original");
-        const fileBytes = new Uint8Array(readFileSync(path));
+        await client.downloadFile(file, origPath);
+        const fileBytes = new Uint8Array(readFileSync(origPath));
         return extractImageMetadata(fileBytes);
     } catch {
         return undefined;
+    } finally {
+        rmSync(tmpDir, { recursive: true, force: true });
     }
 };
 
-// Dump every decrypted metadata layer the account holds into a directory tree
-// of plain JSON: account, per-collection, and per-file records including the
-// private and public magic metadata and (by default) the ML data. Collections
-// and files are enumerated from the library's cache rather than a fresh server
-// scan; the ML fetch and EXIF extraction are unchanged.
 export const runMetadataBackup = async (
-    lib: Library,
     client: Client,
     outDir: string,
     opts?: MetadataBackupOptions,
@@ -138,19 +139,13 @@ export const runMetadataBackup = async (
     );
 
     log("Fetching collections...");
+    const collections = await client.listCollections();
 
-    // Enumerate through the library's read surface. Each album carries its
+    const allFiles: { file: EnteFile; colDirName: string }[] = [];
-    // photos, but the full decrypted `Collection`/`EnteFile` records (with the
-    // magic-metadata layers this dump exists to preserve) come from the
-    // library's by-id accessors.
-    const allFiles: { file: EnteFile; photo: Photo; colDirName: string }[] = [];
     const fileKeys = new Map<number, Uint8Array>();
     const seenFileIDs = new Set<number>();
 
-    for (const album of lib.albums.list()) {
+    for (const col of collections) {
-        const col = lib.getCollection(album.collectionID);
-        if (!col) continue;
-
         const dirName = `${col.id}-${sanitizePath(col.name || "unnamed")}`;
         const colDir = join(outDir, "collections", dirName);
         mkdirSync(colDir, { recursive: true });
@@ -175,13 +170,11 @@ export const runMetadataBackup = async (
         );
 
         log(`[${col.name}] Fetching files...`);
-        const photos = album.photos.list();
+        const files = await client.listFiles(col.id, col.key);
-        log(`[${col.name}] ${photos.length} file(s)`);
+        log(`[${col.name}] ${files.length} file(s)`);
 
-        for (const photo of photos) {
+        for (const file of files) {
-            const file = lib.getFile(col.id, photo.fileID);
+            allFiles.push({ file, colDirName: dirName });
-            if (!file) continue;
-            allFiles.push({ file, photo, colDirName: dirName });
             if (!seenFileIDs.has(file.id)) {
                 fileKeys.set(file.id, file.key);
                 seenFileIDs.add(file.id);
@@ -198,7 +191,7 @@ export const runMetadataBackup = async (
     log(`Got ML data for ${mlDataMap.size} file(s)`);
 
     const writtenFileIDs = new Set<number>();
-    for (const { file, photo, colDirName } of allFiles) {
+    for (const { file, colDirName } of allFiles) {
         const colDir = join(outDir, "collections", colDirName);
 
         const fileMeta: Record<string, unknown> = {
@@ -217,7 +210,7 @@ export const runMetadataBackup = async (
 
         if (wantExif && !writtenFileIDs.has(file.id)) {
             log(`[${file.metadata.title}] Extracting EXIF...`);
-            const exifData = await extractExif(photo);
+            const exifData = await extractExif(client, file);
             if (exifData) fileMeta.imageMetadata = exifData;
         }
         writtenFileIDs.add(file.id);

src/thumbnails.ts

@@ -2,10 +2,13 @@ import { createHash } from "node:crypto";
 import { readFileSync } from "node:fs";
 import * as jpeg from "jpeg-js";
 import type { Client } from "./client.js";
-import type { Library } from "./library/index.js";
 import { ApiError } from "./api/client.js";
 import { encryptBlob, toBase64 } from "./crypto/index.js";
+import { downloadFile } from "./download/index.js";
 import type { EnteFile } from "./model/types.js";
+import { mkdtempSync, rmSync } from "node:fs";
+import { join } from "node:path";
+import { tmpdir } from "node:os";
 
 const THUMB_MAX_DIMENSION = 720;
 const THUMB_JPEG_QUALITY = 50;
@@ -17,51 +20,34 @@ export interface MissingThumbnailInfo {
     reason: string;
 }
 
-// Three outcomes, not two. "fixed": a thumbnail was generated and uploaded.
-// "failed": something went wrong (download, encode, upload) and the file still
-// has no thumbnail. "skipped": the file is a format this helper cannot
-// regenerate — a video, or an image that is not a baseline JPEG. Skipped is a
-// deliberate, expected outcome, not an error (issue #17): the repair path is
-// JPEG-only because `jpeg-js` is, and a PNG or HEIC is left for a format-aware
-// tool rather than reported as a failure.
-export type ThumbnailFixStatus = "fixed" | "skipped" | "failed";
-
 export interface ThumbnailFixResult {
     fileID: number;
     title: string;
     collection: string;
-    status: ThumbnailFixStatus;
+    success: boolean;
-    // Why the file was skipped or failed; unset when it was fixed.
+    error?: string;
-    reason?: string;
 }
 
 export type ProgressCallback = (message: string) => void;
 
-// Enumerate every file the library knows about, newest album first, each file
-// once, and report those whose server-side thumbnail is missing. "Missing" is
-// only two answers: an empty body, or a 404. Any other error reaching this
-// point has already exhausted its retries — a failing server, a dropped
-// connection, a deadline — and says nothing about whether the thumbnail
-// exists, so it is logged and the file is left unreported. That distinction is
-// what stops `fix-missing-thumbnails` from regenerating and uploading over
-// thumbnails that were fine all along while the CDN was briefly returning 500s.
 export const listMissingThumbnails = async (
-    lib: Library,
     client: Client,
     onProgress?: ProgressCallback,
 ): Promise<MissingThumbnailInfo[]> => {
     const log = onProgress ?? (() => {});
-    const api = client.getApiClient();
     const missing: MissingThumbnailInfo[] = [];
     const seen = new Set<number>();
 
-    for (const album of lib.albums.list()) {
+    const collections = await client.listCollections();
-        log(`[${album.name}] Checking thumbnails...`);
+    for (const col of collections) {
-        for (const photo of album.photos.list()) {
+        log(`[${col.name}] Checking thumbnails...`);
-            if (seen.has(photo.fileID)) continue;
+        const files = await client.listFiles(col.id, col.key);
-            seen.add(photo.fileID);
+        for (const file of files) {
+            if (seen.has(file.id)) continue;
+            seen.add(file.id);
             try {
-                const stream = await api.getThumbnailStream(photo.fileID);
+                const api = client.getApiClient();
+                const stream = await api.getThumbnailStream(file.id);
                 const reader = stream.getReader();
                 let totalBytes = 0;
                 for (;;) {
@@ -71,23 +57,35 @@ export const listMissingThumbnails = async (
                 }
                 if (totalBytes === 0) {
                     missing.push({
-                        fileID: photo.fileID,
+                        fileID: file.id,
-                        title: photo.title,
+                        title: file.metadata.title,
-                        collection: album.name,
+                        collection: col.name,
                         reason: "empty thumbnail (0 bytes)",
                     });
                 }
             } catch (err) {
+                // A 404 is the server stating the thumbnail is not there:
+                // that, and an empty body, are the only two answers that mean
+                // "missing". Anything else reaching this point is a failure
+                // that already exhausted its retries — a failing server, a
+                // dropped connection, a deadline — and says nothing about
+                // whether the thumbnail exists.
+                //
+                // The distinction is what stops `helper
+                // fix-missing-thumbnails` from downloading originals,
+                // regenerating thumbnails and uploading them over thumbnails
+                // that were fine all along, because the CDN was briefly
+                // returning 500s while this ran.
                 if (err instanceof ApiError && err.status === 404) {
                     missing.push({
-                        fileID: photo.fileID,
+                        fileID: file.id,
-                        title: photo.title,
+                        title: file.metadata.title,
-                        collection: album.name,
+                        collection: col.name,
                         reason: "thumbnail not found (HTTP 404)",
                     });
                 } else {
                     log(
-                        `[${album.name}] Could not check ${photo.title}: ${err instanceof Error ? err.message : String(err)} (not reported as missing)`,
+                        `[${col.name}] Could not check ${file.metadata.title}: ${err instanceof Error ? err.message : String(err)} (not reported as missing)`,
                     );
                 }
             }
@@ -96,7 +94,7 @@ export const listMissingThumbnails = async (
     return missing;
 };
 
-// Bilinear resize of an RGBA pixel buffer.
+// Bilinear resize of RGBA pixel buffer
 const resizeRGBA = (
     src: Uint8Array,
     srcW: number,
@@ -163,33 +161,7 @@ const generateThumbnail = (fileBytes: Uint8Array): Uint8Array => {
     return new Uint8Array(encoded.data);
 };
 
-// A baseline/JFIF JPEG starts with the SOI marker 0xFFD8. `jpeg-js` decodes
-// only JPEG, so this signature check is what separates a file the helper can
-// regenerate from one it must skip: a PNG, HEIC, or the odd non-image byte
-// stream all fail this and are reported as skipped rather than crashing the
-// decoder into an opaque failure (issue #17).
-const isJpeg = (bytes: Uint8Array): boolean =>
-    bytes.length >= 2 && bytes[0] === 0xff && bytes[1] === 0xd8;
-
-// The reason a file cannot have a JPEG thumbnail regenerated for it from its
-// metadata alone, before any bytes are fetched, or undefined when it might. A
-// non-image (video, live photo) is unsupported outright; a still image still
-// has to be checked against its actual bytes once downloaded.
-const unsupportedByType = (file: EnteFile): string | undefined => {
-    if (file.metadata.fileType !== "image") {
-        return `unsupported file type: ${file.metadata.fileType} (only JPEG images can be regenerated)`;
-    }
-    return undefined;
-};
-
-// Regenerate and upload a thumbnail for each requested file. Originals are read
-// through the library's content cache (`photo.original()`); the generated
-// thumbnail is JPEG-encoded, encrypted under the file's own key, and registered
-// with the server — the encrypt-and-upload path is unchanged. Each file is
-// resolved to one outcome (fixed / skipped / failed) and a failure on one file
-// never stops the others.
 export const fixMissingThumbnails = async (
-    lib: Library,
     client: Client,
     fileIDs: number[],
     onProgress?: ProgressCallback,
@@ -198,24 +170,19 @@ export const fixMissingThumbnails = async (
     const results: ThumbnailFixResult[] = [];
     const api = client.getApiClient();
 
-    // Resolve each requested fileID to its file record and owning album by
+    const collections = await client.listCollections();
-    // enumerating the library, each file taken from the first album that holds
-    // it. The raw `EnteFile` carries the per-file key the thumbnail is
-    // encrypted under, which the projected records deliberately do not.
-    const wanted = new Set(fileIDs);
     const fileMap = new Map<
         number,
         { file: EnteFile; collectionName: string }
     >();
-    for (const album of lib.albums.list()) {
+
-        for (const photo of album.photos.list()) {
+    for (const col of collections) {
-            if (!wanted.has(photo.fileID) || fileMap.has(photo.fileID))
+        const files = await client.listFiles(col.id, col.key);
-                continue;
+        for (const file of files) {
-            const file = lib.getFile(album.collectionID, photo.fileID);
+            if (fileIDs.includes(file.id) && !fileMap.has(file.id)) {
-            if (file) {
+                fileMap.set(file.id, {
-                fileMap.set(photo.fileID, {
                     file,
-                    collectionName: album.name,
+                    collectionName: col.name,
                 });
             }
         }
@@ -228,61 +195,33 @@ export const fixMissingThumbnails = async (
                 fileID,
                 title: "unknown",
                 collection: "unknown",
-                status: "failed",
+                success: false,
-                reason: "file not found in any collection",
+                error: "file not found in any collection",
             });
             continue;
         }
 
         const { file, collectionName } = entry;
-        const title = file.metadata.title;
+        const tmpDir = mkdtempSync(join(tmpdir(), "quak-thumb-"));
-
-        const typeReason = unsupportedByType(file);
-        if (typeReason) {
-            log(`[${collectionName}] Skipping ${title}: ${typeReason}`);
-            results.push({
-                fileID,
-                title,
-                collection: collectionName,
-                status: "skipped",
-                reason: typeReason,
-            });
-            continue;
-        }
 
         try {
-            const photo = lib.photos.byID({ fileID });
+            log(
-            if (!photo) {
+                `[${collectionName}] Downloading ${file.metadata.title} for thumbnail generation...`,
-                throw new Error("file not present in the library cache");
+            );
-            }
+            const origPath = join(tmpDir, "original");
+            await downloadFile(api, file, origPath);
 
             log(
-                `[${collectionName}] Downloading ${title} for thumbnail generation...`,
+                `[${collectionName}] Generating thumbnail for ${file.metadata.title}...`,
             );
-            const { path } = await photo.original();
+            const fileBytes = readFileSync(origPath);
-            const fileBytes = new Uint8Array(readFileSync(path));
+            const thumbJpeg = generateThumbnail(new Uint8Array(fileBytes));
-
-            if (!isJpeg(fileBytes)) {
-                const reason =
-                    "unsupported image format (only baseline JPEG can be regenerated)";
-                log(`[${collectionName}] Skipping ${title}: ${reason}`);
-                results.push({
-                    fileID,
-                    title,
-                    collection: collectionName,
-                    status: "skipped",
-                    reason,
-                });
-                continue;
-            }
-
-            log(`[${collectionName}] Generating thumbnail for ${title}...`);
-            const thumbJpeg = generateThumbnail(fileBytes);
 
             log(
                 `[${collectionName}] Encrypting and uploading thumbnail (${thumbJpeg.length} bytes)...`,
             );
             const { header, ciphertext } = encryptBlob(thumbJpeg, file.key);
+
             const md5 = createHash("md5").update(ciphertext).digest("base64");
             const { objectKey, url } = await api.getUploadURL(
                 ciphertext.length,
@@ -291,24 +230,28 @@ export const fixMissingThumbnails = async (
             await api.putFile(url, ciphertext);
             await api.updateThumbnail(file.id, objectKey, toBase64(header));
 
-            log(`[${collectionName}] Thumbnail uploaded for ${title}`);
-            results.push({
-                fileID,
-                title,
-                collection: collectionName,
-                status: "fixed",
-            });
-        } catch (err) {
             log(
-                `[${collectionName}] FAILED ${title}: ${err instanceof Error ? err.message : err}`,
+                `[${collectionName}] Thumbnail uploaded for ${file.metadata.title}`,
             );
             results.push({
                 fileID,
-                title,
+                title: file.metadata.title,
                 collection: collectionName,
-                status: "failed",
+                success: true,
-                reason: err instanceof Error ? err.message : String(err),
             });
+        } catch (err) {
+            log(
+                `[${collectionName}] FAILED ${file.metadata.title}: ${err instanceof Error ? err.message : err}`,
+            );
+            results.push({
+                fileID,
+                title: file.metadata.title,
+                collection: collectionName,
+                success: false,
+                error: err instanceof Error ? err.message : String(err),
+            });
+        } finally {
+            rmSync(tmpDir, { recursive: true, force: true });
         }
     }
 

test/cli/metadata-backup.test.ts

@@ -2,17 +2,13 @@
  * Tests for `quak backup-metadata <dir>`.
  *
  * This command dumps all decrypted account metadata into a directory
- * tree of plain JSON files, without downloading any file content (unless
+ * tree of plain JSON files, without downloading any file content. It
- * `--exif` is given). It is fast and produces a complete plaintext record of
+ * is fast (no multi-megabyte downloads) and produces a complete
- * every collection name, file title, creation date, GPS coordinate, camera
+ * plaintext record of every collection name, file title, creation
- * model, caption, face label, and any other metadata the Ente clients have
+ * date, GPS coordinate, camera model, caption, face label, and any
- * attached.
+ * other metadata the Ente clients have attached.
  *
- * As of issue #52 it runs on the library API: `runMetadataBackup(lib, client,
+ * Layout:
- * dir)` enumerates collections and files from the library's cache rather than
- * scanning the client directly, and `--exif` reads each original through the
- * library's content cache (`photo.original()`). The ML fetch is unchanged. The
- * output tree is identical:
  *
  *     <dir>/
  *         account.json                { email, userID }
@@ -48,11 +44,7 @@ import {
 } from "../../src/crypto/index.js";
 import * as jpegJs from "jpeg-js";
 import { Client } from "../../src/client.js";
-import { Library } from "../../src/library/index.js";
+import { runMetadataBackup } from "../../src/metadata-backup.js";
-import {
-    runMetadataBackup,
-    type MetadataBackupOptions,
-} from "../../src/metadata-backup.js";
 import type { KeyAttributes } from "../../src/auth/types.js";
 
 const TEST_EMAIL = "metabackup@example.com";
@@ -439,50 +431,16 @@ afterAll(() => {
         rmSync(testDir, { recursive: true, force: true });
 });
 
-// Log in against the mock and open a library over its cache. The point commands
-// open the library with the background precache off and a long refresh interval;
-// the same here keeps the test deterministic (no thumbnail/original prefetch it
-// did not ask for, no second refresh mid-test).
-const openLib = async (client: Client): Promise<Library> =>
-    Library.open({
-        // The library client omits `fetchMLData`, matching how the CLI opens
-        // point commands: `runMetadataBackup` fetches ML data itself through
-        // the client, so the library's background backfill would only be a
-        // redundant second pass over the same endpoint.
-        client: {
-            whoami: () => client.whoami(),
-            collectionsSince: (args) => client.collectionsSince(args),
-            filesSince: (args) => client.filesSince(args),
-            contentSource: () => client.contentSource(),
-        },
-        cacheDirectory: mkdtempSync(join(testDir, "cache-")),
-        refreshIntervalSeconds: 3600,
-        precacheThumbnails: false,
-        precacheOriginals: false,
-    });
-
-// Run one metadata backup end to end: fresh client, fresh library, then close.
-const runBackup = async (
-    outDir: string,
-    opts?: MetadataBackupOptions,
-): Promise<void> => {
-    const client = await Client.login({
-        email: TEST_EMAIL,
-        password: TEST_PASSWORD,
-        apiOptions: { fetch: buildMetaFetch(mock) },
-    });
-    const lib = await openLib(client);
-    try {
-        await runMetadataBackup(lib, client, outDir, opts);
-    } finally {
-        lib.close();
-    }
-};
-
 describe("quak backup-metadata", () => {
     it("writes account.json with email and userID", async () => {
         const outDir = join(testDir, "full");
-        await runBackup(outDir);
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildMetaFetch(mock) },
+        });
+
+        await runMetadataBackup(client, outDir);
 
         const account = JSON.parse(
             readFileSync(join(outDir, "account.json"), "utf-8"),
@@ -493,7 +451,13 @@ describe("quak backup-metadata", () => {
 
     it("creates per-collection directories with _collection.json", async () => {
         const outDir = join(testDir, "collections");
-        await runBackup(outDir);
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildMetaFetch(mock) },
+        });
+
+        await runMetadataBackup(client, outDir);
 
         const collDirs = readdirSync(join(outDir, "collections"));
         expect(collDirs.length).toBe(2);
@@ -514,7 +478,13 @@ describe("quak backup-metadata", () => {
 
     it("decrypts collection-level pubMagicMetadata", async () => {
         const outDir = join(testDir, "coll-magic");
-        await runBackup(outDir);
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildMetaFetch(mock) },
+        });
+
+        await runMetadataBackup(client, outDir);
 
         const collDirs = readdirSync(join(outDir, "collections"));
         const vacDir = collDirs.find((d) => d.includes("Vacation"))!;
@@ -531,7 +501,13 @@ describe("quak backup-metadata", () => {
 
     it("writes per-file JSON with all three metadata layers", async () => {
         const outDir = join(testDir, "file-meta");
-        await runBackup(outDir);
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildMetaFetch(mock) },
+        });
+
+        await runMetadataBackup(client, outDir);
 
         const collDirs = readdirSync(join(outDir, "collections"));
         const vacDir = collDirs.find((d) => d.includes("Vacation"))!;
@@ -550,7 +526,13 @@ describe("quak backup-metadata", () => {
 
     it("handles files with no magic metadata gracefully", async () => {
         const outDir = join(testDir, "no-magic");
-        await runBackup(outDir);
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildMetaFetch(mock) },
+        });
+
+        await runMetadataBackup(client, outDir);
 
         const collDirs = readdirSync(join(outDir, "collections"));
         const workDir = collDirs.find((d) => d.includes("Work"))!;
@@ -568,8 +550,14 @@ describe("quak backup-metadata", () => {
 
     it("is incremental: second run does not fail", async () => {
         const outDir = join(testDir, "incremental");
-        await runBackup(outDir);
+        const client = await Client.login({
-        await runBackup(outDir);
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildMetaFetch(mock) },
+        });
+
+        await runMetadataBackup(client, outDir);
+        await runMetadataBackup(client, outDir);
 
         const account = JSON.parse(
             readFileSync(join(outDir, "account.json"), "utf-8"),
@@ -579,7 +567,13 @@ describe("quak backup-metadata", () => {
 
     it("fetches and decrypts ML data by default", async () => {
         const outDir = join(testDir, "ml-data");
-        await runBackup(outDir);
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildMetaFetch(mock) },
+        });
+
+        await runMetadataBackup(client, outDir);
 
         const collDirs = readdirSync(join(outDir, "collections"));
         const vacDir = collDirs.find((d) => d.includes("Vacation"))!;
@@ -603,7 +597,13 @@ describe("quak backup-metadata", () => {
 
     it("extracts EXIF from downloaded files when --exif is set", async () => {
         const outDir = join(testDir, "exif-data");
-        await runBackup(outDir, { exif: true });
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildMetaFetch(mock) },
+        });
+
+        await runMetadataBackup(client, outDir, { exif: true });
 
         const collDirs = readdirSync(join(outDir, "collections"));
         const vacDir = collDirs.find((d) => d.includes("Vacation"))!;

test/cli/output.test.ts

@@ -1,76 +0,0 @@
-// The CLI presents a file by its own decrypted metadata, not the PhotoRecord
-// projection (issue #52). For a renamed file the two disagree: the projection
-// prefers `editedName` and reports `editedTime` in milliseconds, while the CLI
-// must print the raw `metadata.title` and `metadata.creationTime` (microseconds)
-// and name downloads after the raw title, byte-identical to the pre-library CLI.
-//
-// This locks in that contrast: the shared output helpers emit the raw values,
-// and the projection of the same file emits the edited ones — so a regression
-// that re-sourced the CLI from the projection would fail here.
-
-import { describe, it, expect } from "vitest";
-
-import {
-    fileListRow,
-    fileListLine,
-    originalName,
-    thumbnailName,
-} from "../../src/cli-output.js";
-import { deriveRecords } from "../../src/library/records.js";
-import type { EnteFile } from "../../src/model/types.js";
-
-// Microseconds, as Ente stores times.
-const RAW_CREATION = 1700000000000000;
-const EDITED_TIME = 1710000000000000;
-const RAW_TITLE = "IMG_0001.HEIC";
-const EDITED_NAME = "Sunset.heic";
-
-// A file the user has renamed and re-dated: basic metadata holds the original
-// title and capture time; public magic metadata holds the edits.
-const renamedFile: EnteFile = {
-    id: 100,
-    collectionID: 10,
-    ownerID: 42,
-    key: new Uint8Array(),
-    metadata: {
-        title: RAW_TITLE,
-        fileType: "image",
-        creationTime: RAW_CREATION,
-        modificationTime: RAW_CREATION,
-    },
-    pubMagicMetadata: { editedName: EDITED_NAME, editedTime: EDITED_TIME },
-    file: { decryptionHeader: "" },
-    thumbnail: { decryptionHeader: "" },
-    updationTime: RAW_CREATION,
-};
-
-describe("CLI file output (issue #52)", () => {
-    it("emits the raw title and microsecond creationTime for --json", () => {
-        expect(fileListRow(renamedFile)).toEqual({
-            id: 100,
-            title: RAW_TITLE,
-            fileType: "image",
-            creationTime: RAW_CREATION,
-            collectionID: 10,
-        });
-    });
-
-    it("emits the raw title in the human column", () => {
-        expect(fileListLine(renamedFile)).toBe(`100\timage\t${RAW_TITLE}`);
-    });
-
-    it("names downloads after the raw title", () => {
-        expect(originalName(renamedFile)).toBe(RAW_TITLE);
-        expect(thumbnailName(renamedFile)).toBe(`thumb_${RAW_TITLE}`);
-    });
-
-    it("does not use the editedName/editedTime projection", () => {
-        const record = deriveRecords([], [renamedFile]).photos.get(100);
-        // The projection prefers the edits and reports milliseconds; the CLI
-        // helpers above deliberately do not.
-        expect(record?.title).toBe(EDITED_NAME);
-        expect(record?.takenAt).toBe(Math.floor(EDITED_TIME / 1000));
-        expect(fileListRow(renamedFile).title).not.toBe(record?.title);
-        expect(fileListRow(renamedFile).creationTime).not.toBe(record?.takenAt);
-    });
-});

test/cli/read.test.ts

@@ -1,195 +0,0 @@
-/**
- * Tests for the CLI read helpers (`src/cli-read.ts`, owner amendment to
- * issue #36, issue #52).
- *
- * The `collections`, `files`, `get`, and `get-thumb` commands must answer for
- * current server state, not the local cache, so each helper forces a
- * `Library.fresh()` round-trip before it reads. The stand-in library below
- * serves nothing until `fresh()` has been awaited, so a helper that read
- * without refreshing would come back empty and fail here.
- *
- * `collections` and `files` also list in the library's enumeration order
- * (`listCollections`/`listFiles`) — the order the pre-library CLI printed — not
- * the albums/photos projection's newest-first order. The fixtures are seeded in
- * an enumeration order that a newest-first sort would rearrange, so a
- * regression to the projection order would fail here too. Field values still
- * come from the raw metadata via `cli-output.ts`.
- */
-
-import { describe, it, expect } from "vitest";
-
-import {
-    freshCollections,
-    freshFiles,
-    freshFile,
-    type FreshReadLibrary,
-} from "../../src/cli-read.js";
-import { fileListRow } from "../../src/cli-output.js";
-import type { Photo } from "../../src/library/index.js";
-import type { Collection, EnteFile } from "../../src/model/types.js";
-
-const collection = (id: number, updationTime: number): Collection => ({
-    id,
-    ownerID: 42,
-    key: new Uint8Array(),
-    name: `album-${id}`,
-    type: "album",
-    updationTime,
-    isShared: false,
-});
-
-// Microseconds, as Ente stores times.
-const file = (
-    id: number,
-    collectionID: number,
-    creationTime: number,
-): EnteFile => ({
-    id,
-    collectionID,
-    ownerID: 42,
-    key: new Uint8Array(),
-    metadata: {
-        title: `file-${id}.jpg`,
-        fileType: "image",
-        creationTime,
-        modificationTime: creationTime,
-    },
-    file: { decryptionHeader: "" },
-    thumbnail: { decryptionHeader: "" },
-    updationTime: creationTime,
-});
-
-// A library that reveals its records only after `fresh()` has been awaited, and
-// serves them in the enumeration order it was given. `photos.byID` returns a
-// stand-in `Photo` carrying just the fileID the helper passes through.
-class FakeLibrary implements FreshReadLibrary {
-    freshCalls = 0;
-    private refreshed = false;
-
-    constructor(
-        private readonly collections: Collection[],
-        private readonly files: EnteFile[],
-    ) {}
-
-    async fresh(): Promise<unknown> {
-        this.freshCalls++;
-        this.refreshed = true;
-        return {};
-    }
-
-    listCollections(): Collection[] {
-        return this.refreshed ? this.collections : [];
-    }
-
-    getCollection(id: number): Collection | undefined {
-        return this.listCollections().find((c) => c.id === id);
-    }
-
-    listFiles(collectionID: number): EnteFile[] {
-        return this.refreshed
-            ? this.files.filter((f) => f.collectionID === collectionID)
-            : [];
-    }
-
-    getFileByID(fileID: number): EnteFile | undefined {
-        if (!this.refreshed) return undefined;
-        return this.files.find((f) => f.id === fileID);
-    }
-
-    photos = {
-        byID: ({ fileID }: { fileID: number }): Photo | undefined => {
-            if (!this.refreshed) return undefined;
-            if (!this.files.some((f) => f.id === fileID)) return undefined;
-            return { fileID } as unknown as Photo;
-        },
-    };
-}
-
-describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
-    it("freshCollections refreshes first, then lists in enumeration order", async () => {
-        // Enumeration order 2, 1, 3; a newest-first sort would be 3, 2, 1.
-        const lib = new FakeLibrary(
-            [collection(2, 200), collection(1, 300), collection(3, 100)],
-            [],
-        );
-        const rows = await freshCollections(lib);
-
-        expect(lib.freshCalls).toBe(1);
-        expect(rows.map((c) => c.id)).toEqual([2, 1, 3]);
-        // The projection's newest-first order is a different sequence, so this
-        // is not accidentally that order.
-        const newestFirst = [...rows]
-            .sort((a, b) => b.updationTime - a.updationTime)
-            .map((c) => c.id);
-        expect(newestFirst).toEqual([1, 2, 3]);
-        expect(rows.map((c) => c.id)).not.toEqual(newestFirst);
-    });
-
-    it("freshFiles refreshes first, lists in enumeration order, keeps raw fields", async () => {
-        // Enumeration order by id 10, 11, 12; creationTimes ascending, so a
-        // newest-first sort would reverse them.
-        const files = [
-            file(10, 1, 1_700_000_000_000_000),
-            file(11, 1, 1_700_000_000_000_001),
-            file(12, 1, 1_700_000_000_000_002),
-        ];
-        const lib = new FakeLibrary([collection(1, 100)], files);
-
-        const rows = await freshFiles(lib, 1);
-        expect(lib.freshCalls).toBe(1);
-        expect(rows?.map((f) => f.id)).toEqual([10, 11, 12]);
-
-        // Field values come from raw metadata: microsecond creationTime and the
-        // raw title, unchanged.
-        expect(rows?.map(fileListRow)).toEqual([
-            {
-                id: 10,
-                title: "file-10.jpg",
-                fileType: "image",
-                creationTime: 1_700_000_000_000_000,
-                collectionID: 1,
-            },
-            {
-                id: 11,
-                title: "file-11.jpg",
-                fileType: "image",
-                creationTime: 1_700_000_000_000_001,
-                collectionID: 1,
-            },
-            {
-                id: 12,
-                title: "file-12.jpg",
-                fileType: "image",
-                creationTime: 1_700_000_000_000_002,
-                collectionID: 1,
-            },
-        ]);
-    });
-
-    it("freshFiles returns undefined for an unknown collection", async () => {
-        const lib = new FakeLibrary([collection(1, 100)], []);
-        const rows = await freshFiles(lib, 999);
-        expect(lib.freshCalls).toBe(1);
-        expect(rows).toBeUndefined();
-    });
-
-    it("freshFile refreshes first, then resolves the photo and its raw record", async () => {
-        const f = file(10, 1, 1_700_000_000_000_000);
-        const lib = new FakeLibrary([collection(1, 100)], [f]);
-
-        const resolved = await freshFile(lib, 10);
-        expect(lib.freshCalls).toBe(1);
-        expect(resolved?.photo.fileID).toBe(10);
-        expect(resolved?.file.metadata.title).toBe("file-10.jpg");
-        expect(resolved?.file.metadata.creationTime).toBe(
-            1_700_000_000_000_000,
-        );
-    });
-
-    it("freshFile returns undefined for an unknown file", async () => {
-        const lib = new FakeLibrary([collection(1, 100)], []);
-        const resolved = await freshFile(lib, 404);
-        expect(lib.freshCalls).toBe(1);
-        expect(resolved).toBeUndefined();
-    });
-});

test/library/fresh.test.ts

@@ -1,272 +0,0 @@
-/**
- * Tests for fresh reads (issue #75, an owner amendment to design #36).
- *
- * The default read namespaces answer from RAM and never touch the network; a
- * background loop keeps the local copy current. `fresh()` adds an awaited path:
- * it forces a refresh, waits for it to complete and persist, and only then
- * hands back the `albums`/`photos`/`timeline` namespaces, guaranteeing the
- * local copy reflects a completed server round-trip. The contracts here:
- *
- *  1. A fresh read observes a server change that a same-instant default read
- *     would miss (the default path has not refreshed yet).
- *  2. Concurrent fresh reads coalesce onto one in-flight refresh — three
- *     concurrent `fresh()` calls make exactly one collections round-trip, not
- *     three.
- *  3. A refresh that fails rejects the fresh read (currency was unavailable),
- *     while the default reads stay silent and keep serving the last good copy.
- *
- * The client is the same metadata-only mock the background-refresh tests use:
- * no crypto, no network, scripted pages, and a record of each call. A long
- * refresh interval keeps the background timer out of the way so each test's
- * refreshes are exactly the ones it triggers.
- */
-
-import { describe, it, expect, beforeEach, afterEach } from "vitest";
-import { mkdtempSync, rmSync } from "node:fs";
-import { tmpdir } from "node:os";
-import { join } from "node:path";
-
-import { Library } from "../../src/library/index.js";
-import type { CollectionsPage, FilesPage } from "../../src/client.js";
-import type { Collection, EnteFile } from "../../src/model/types.js";
-
-const USER_ID = 42;
-
-// Long enough that no background tick fires during a test; each test's
-// refreshes are only the ones its own `fresh()` calls force.
-const SLOW_INTERVAL = 3600;
-
-const collection = (id: number, updationTime: number): Collection => ({
-    id,
-    ownerID: USER_ID,
-    key: new Uint8Array([id & 0xff]),
-    name: `album-${id}`,
-    type: "album",
-    updationTime,
-    isShared: false,
-});
-
-const file = (
-    id: number,
-    collectionID: number,
-    updationTime: number,
-): EnteFile => ({
-    id,
-    collectionID,
-    ownerID: USER_ID,
-    key: new Uint8Array([id & 0xff]),
-    metadata: {
-        title: `file-${id}.jpg`,
-        fileType: "image",
-        creationTime: updationTime,
-        modificationTime: updationTime,
-    },
-    file: { decryptionHeader: "aGVhZGVy" },
-    thumbnail: { decryptionHeader: "dGh1bWI=" },
-    updationTime,
-});
-
-class MockClient {
-    userID = USER_ID;
-    failCollections = false;
-    collectionsQueue: CollectionsPage[] = [];
-    filesByCollection = new Map<number, FilesPage[]>();
-
-    collectionsSinceTimes: number[] = [];
-    filesCalls: { collectionID: number; sinceTime: number }[] = [];
-
-    whoami(): { email: string; userID: number } {
-        return { email: "user@example.com", userID: this.userID };
-    }
-
-    async collectionsSince(args: {
-        sinceTime: number;
-    }): Promise<CollectionsPage> {
-        this.collectionsSinceTimes.push(args.sinceTime);
-        if (this.failCollections) throw new Error("network down");
-        return (
-            this.collectionsQueue.shift() ?? {
-                collections: [],
-                deleted: [],
-                cursor: args.sinceTime,
-            }
-        );
-    }
-
-    async filesSince(args: {
-        collectionID: number;
-        collectionKey: Uint8Array;
-        sinceTime: number;
-    }): Promise<FilesPage> {
-        this.filesCalls.push({
-            collectionID: args.collectionID,
-            sinceTime: args.sinceTime,
-        });
-        const queue = this.filesByCollection.get(args.collectionID);
-        return (
-            queue?.shift() ?? {
-                files: [],
-                deleted: [],
-                cursor: args.sinceTime,
-            }
-        );
-    }
-
-    filesFor(collectionID: number, ...pages: FilesPage[]): void {
-        this.filesByCollection.set(collectionID, pages);
-    }
-}
-
-// A client seeded with one collection and one file, opened with an empty cache
-// so the initial refresh is awaited and post-`open()` state is deterministic.
-const openSeeded = async (
-    cacheDirectory: string,
-): Promise<{ client: MockClient; lib: Library }> => {
-    const client = new MockClient();
-    client.collectionsQueue.push({
-        collections: [collection(1, 100)],
-        deleted: [],
-        cursor: 100,
-    });
-    client.filesFor(1, {
-        files: [file(1001, 1, 90)],
-        deleted: [],
-        cursor: 90,
-    });
-    const lib = await Library.open({
-        client,
-        cacheDirectory,
-        refreshIntervalSeconds: SLOW_INTERVAL,
-    });
-    return { client, lib };
-};
-
-describe("Library.fresh", () => {
-    let dir: string;
-    let cacheDirectory: string;
-
-    beforeEach(() => {
-        dir = mkdtempSync(join(tmpdir(), "quak-fresh-"));
-        cacheDirectory = join(dir, "cache");
-    });
-
-    afterEach(() => {
-        rmSync(dir, { recursive: true, force: true });
-    });
-
-    it("observes a server change a same-instant default read would miss", async () => {
-        const { client, lib } = await openSeeded(cacheDirectory);
-        try {
-            // A new file appears on the server after open, advancing its
-            // collection so the next refresh re-enumerates it.
-            client.collectionsQueue.push({
-                collections: [collection(1, 200)],
-                deleted: [],
-                cursor: 200,
-            });
-            client.filesFor(1, {
-                files: [file(1002, 1, 190)],
-                deleted: [],
-                cursor: 190,
-            });
-
-            // A default read at this instant has not refreshed: it misses 1002.
-            expect(lib.photos.byID({ fileID: 1002 })).toBeUndefined();
-
-            // A fresh read forces the round-trip and sees it.
-            const reads = await lib.fresh();
-            expect(reads.photos.byID({ fileID: 1002 })?.fileID).toBe(1002);
-            // And the change is now live for the default namespaces too.
-            expect(lib.photos.byID({ fileID: 1002 })?.fileID).toBe(1002);
-        } finally {
-            lib.close();
-        }
-    });
-
-    it("coalesces concurrent fresh reads onto one in-flight refresh", async () => {
-        const { client, lib } = await openSeeded(cacheDirectory);
-        try {
-            client.collectionsQueue.push({
-                collections: [collection(1, 200)],
-                deleted: [],
-                cursor: 200,
-            });
-            client.filesFor(1, {
-                files: [file(1002, 1, 190)],
-                deleted: [],
-                cursor: 190,
-            });
-
-            const collectionsBefore = client.collectionsSinceTimes.length;
-            const filesBefore = client.filesCalls.length;
-
-            // Gate the next collections fetch so all three fresh reads are in
-            // flight together before any of them completes.
-            let release: () => void = () => {};
-            const gate = new Promise<void>((r) => {
-                release = r;
-            });
-            const inner = client.collectionsSince.bind(client);
-            client.collectionsSince = async (args: { sinceTime: number }) => {
-                await gate;
-                return inner(args);
-            };
-
-            const all = Promise.all([lib.fresh(), lib.fresh(), lib.fresh()]);
-            release();
-            const [a, b, c] = await all;
-
-            // Exactly one collections round-trip and one file round-trip served
-            // all three fresh reads.
-            expect(client.collectionsSinceTimes.length).toBe(
-                collectionsBefore + 1,
-            );
-            expect(client.filesCalls.length).toBe(filesBefore + 1);
-
-            // All three observed the change.
-            for (const reads of [a, b, c]) {
-                expect(reads.photos.byID({ fileID: 1002 })?.fileID).toBe(1002);
-            }
-        } finally {
-            lib.close();
-        }
-    });
-
-    it("rejects the fresh read when the refresh fails, leaving defaults intact", async () => {
-        const { client, lib } = await openSeeded(cacheDirectory);
-        try {
-            client.failCollections = true;
-
-            // Two concurrent fresh reads both reject, and share one failed
-            // round-trip rather than each making its own.
-            const collectionsBefore = client.collectionsSinceTimes.length;
-            const first = lib.fresh();
-            const second = lib.fresh();
-            await expect(first).rejects.toThrow(/network down/);
-            await expect(second).rejects.toThrow(/network down/);
-            expect(client.collectionsSinceTimes.length).toBe(
-                collectionsBefore + 1,
-            );
-
-            // The default reads never rejected: they still serve the last good
-            // copy, and the failure surfaced through status().
-            expect(lib.photos.byID({ fileID: 1001 })?.fileID).toBe(1001);
-            expect(lib.status().lastError).toMatch(/network down/);
-
-            // Recovery: once the server answers, a fresh read resolves current.
-            client.failCollections = false;
-            client.collectionsQueue.push({
-                collections: [collection(2, 300)],
-                deleted: [],
-                cursor: 300,
-            });
-            const reads = await lib.fresh();
-            expect(reads.albums.byID({ collectionID: 2 })?.collectionID).toBe(
-                2,
-            );
-            expect(lib.status().lastError).toBeUndefined();
-        } finally {
-            lib.close();
-        }
-    });
-});

test/library/mlsearch.test.ts

@@ -1,108 +0,0 @@
-/**
- * Tests for the content-similarity search surface over the CLIP index
- * (issue #50).
- *
- * The surface is `lib.mldata`: `forFile` reads the full stored payload from
- * disk, while `similar` and `searchByEmbedding` rank fileIDs by cosine
- * similarity over the in-RAM `Float32Array` index alone (no disk, no network).
- * The fixture uses axis-aligned vectors so the correct cosine ranking is
- * obvious by inspection; cosine ignores magnitude, so `[2, 0, 0]` ranks above
- * `[0.8, 0.6, 0]` for a `[1, 0, 0]` query.
- */
-
-import { describe, it, expect, beforeEach, afterEach } from "vitest";
-import { mkdtempSync, rmSync } from "node:fs";
-import { tmpdir } from "node:os";
-import { join } from "node:path";
-
-import { MLDataStore } from "../../src/library/mldata.js";
-import { makeMLDataAPI, type MLDataAPI } from "../../src/library/mlsearch.js";
-import type { MLData } from "../../src/mldata-fetch.js";
-
-// A payload shaped like Ente's: a CLIP embedding plus face data that only the
-// on-disk payload carries (never the RAM index).
-const payload = (embedding: number[]): MLData => ({
-    face: { faces: [{ faceID: "f", detection: { box: { x: 0.5 } } }] },
-    clip: { embedding },
-});
-
-// A small fixture index. Directions are chosen so every cosine ranking below
-// is unambiguous.
-const fixture = (): Map<number, MLData> =>
-    new Map([
-        [10, payload([1, 0, 0])],
-        [20, payload([0.8, 0.6, 0])],
-        [30, payload([0, 1, 0])],
-        [40, payload([-1, 0, 0])],
-        [50, payload([2, 0, 0])],
-    ]);
-
-describe("lib.mldata content-similarity search", () => {
-    let dir: string;
-    let store: MLDataStore;
-    let api: MLDataAPI;
-
-    beforeEach(async () => {
-        dir = mkdtempSync(join(tmpdir(), "quak-mlsearch-"));
-        store = await MLDataStore.open(dir);
-        const updation = new Map([...fixture().keys()].map((id) => [id, 1]));
-        await store.storeFetched(fixture(), updation);
-        api = makeMLDataAPI(() => store);
-    });
-
-    afterEach(() => {
-        rmSync(dir, { recursive: true, force: true });
-    });
-
-    it("forFile returns the whole stored payload, or undefined when uncached", async () => {
-        const full = await api.forFile({ fileID: 20 });
-        expect(full).toBeDefined();
-        // Face data lives only in the payload, never in the RAM index.
-        expect(full?.face).toBeDefined();
-        expect(full?.clip).toEqual({ embedding: [0.8, 0.6, 0] });
-        expect(await api.forFile({ fileID: 999 })).toBeUndefined();
-    });
-
-    it("similar ranks other files by cosine and excludes the query itself", () => {
-        // Query is file 10 = [1, 0, 0]. By cosine: 50 (1.0) > 20 (0.8) >
-        // 30 (0) > 40 (-1); 10 itself is left out.
-        const ranked = api.similar({ fileID: 10 });
-        expect(ranked.map((r) => r.fileID)).toEqual([50, 20, 30, 40]);
-        // Cosine ignores magnitude: [2,0,0] is a perfect match for [1,0,0].
-        expect(ranked[0]).toMatchObject({ fileID: 50 });
-        expect(ranked[0].score).toBeCloseTo(1, 5);
-    });
-
-    it("similar honours limit and returns [] for an unindexed file", () => {
-        expect(
-            api.similar({ fileID: 10, limit: 2 }).map((r) => r.fileID),
-        ).toEqual([50, 20]);
-        expect(api.similar({ fileID: 999 })).toEqual([]);
-    });
-
-    it("searchByEmbedding ranks the index by cosine to the query vector", () => {
-        // Query [0, 1, 0]: 30 (1.0) > 20 (0.6) > {10, 40, 50} all 0, broken by
-        // ascending fileID.
-        const ranked = api.searchByEmbedding({ embedding: [0, 1, 0] });
-        expect(ranked.map((r) => r.fileID)).toEqual([30, 20, 10, 40, 50]);
-        expect(ranked[0].score).toBeCloseTo(1, 5);
-
-        expect(
-            api
-                .searchByEmbedding({ embedding: [0, 1, 0], limit: 2 })
-                .map((r) => r.fileID),
-        ).toEqual([30, 20]);
-    });
-
-    it("searchByEmbedding returns [] for a wrong-length or zero query", () => {
-        expect(api.searchByEmbedding({ embedding: [1, 0] })).toEqual([]);
-        expect(api.searchByEmbedding({ embedding: [0, 0, 0] })).toEqual([]);
-    });
-
-    it("degrades to empty results when no ML store is present", async () => {
-        const none = makeMLDataAPI(() => undefined);
-        expect(await none.forFile({ fileID: 10 })).toBeUndefined();
-        expect(none.similar({ fileID: 10 })).toEqual([]);
-        expect(none.searchByEmbedding({ embedding: [1, 0, 0] })).toEqual([]);
-    });
-});

test/thumbnails/thumbnails.test.ts

@@ -6,30 +6,17 @@
  * working thumbnails, others return 404 or empty bodies. The tests
  * verify that the detection and repair logic handles each case correctly.
  *
- * As of issue #52 both helpers take an open `Library` for enumeration and the
- * `Client` for the API operations that stay unchanged (the thumbnail existence
- * check, and the encrypt-and-upload path). `fixMissingThumbnails` reads each
- * original through the library's content cache (`photo.original()`).
- *
  * `fixMissingThumbnails` is the most complex function in quak: it
  * downloads the original file, generates a JPEG thumbnail with jpeg-js,
  * encrypts it with secretstream push, gets a presigned upload URL,
  * uploads to S3, and registers the new thumbnail with the API. The
  * test verifies each step actually happened and the uploaded data is
  * a valid encrypted blob that decrypts to a JPEG.
- *
- * It regenerates thumbnails for baseline JPEGs only, because `jpeg-js` decodes
- * only JPEG. A non-JPEG image (PNG, HEIC) or a video is reported as "skipped
- * (unsupported)" rather than crashing the decoder into an opaque failure
- * (issue #17); the mixed test below locks that distinction down.
  */
 
-import { existsSync, mkdtempSync, rmSync } from "node:fs";
-import { join } from "node:path";
-import { tmpdir } from "node:os";
 import sodium from "libsodium-wrappers-sumo";
 import * as jpegJs from "jpeg-js";
-import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { beforeAll, describe, expect, it } from "vitest";
 import {
     init,
     toBase64,
@@ -40,7 +27,6 @@ import {
 } from "../../src/crypto/index.js";
 import { SRP, SrpServer } from "fast-srp-hap";
 import { Client } from "../../src/client.js";
-import { Library } from "../../src/library/index.js";
 import {
     listMissingThumbnails,
     fixMissingThumbnails,
@@ -56,7 +42,6 @@ const TEST_EMAIL = "thumb@example.com";
 const TEST_PASSWORD = "thumbpass";
 const TEST_OPS = 2;
 const TEST_MEM = 64 * 1024 * 1024;
-const TEST_TIME = 1700000000000000;
 
 interface ThumbMockState {
     verifier: Buffer;
@@ -78,17 +63,8 @@ interface ThumbMockState {
 }
 
 let mock: ThumbMockState;
-let tmpRoot: string;
 
-// PNG signature bytes — enough for `fixMissingThumbnails` to recognise a
+const buildThumbMock = async (): Promise<ThumbMockState> => {
-// non-JPEG image and skip it. It need not be a decodable PNG.
-const PNG_BYTES = new Uint8Array([
-    0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d,
-]);
-
-const buildThumbMock = async (opts?: {
-    extraFormats?: boolean;
-}): Promise<ThumbMockState> => {
     const kekSalt = sodium.randombytes_buf(sodium.crypto_pwhash_SALTBYTES);
     const kek = await deriveKEK(TEST_PASSWORD, kekSalt, TEST_OPS, TEST_MEM);
     const loginSubKeyBytes = deriveLoginSubkey(kek);
@@ -126,6 +102,7 @@ const buildThumbMock = async (opts?: {
         opsLimit: TEST_OPS,
     };
 
+    // One collection with 3 files: ok thumbnail, empty thumbnail, 404 thumbnail
     const collKey = sodium.crypto_secretbox_keygen();
     const ckN = sodium.randombytes_buf(sodium.crypto_secretbox_NONCEBYTES);
     const encCK = sodium.crypto_secretbox_easy(collKey, ckN, masterKey);
@@ -141,11 +118,10 @@ const buildThumbMock = async (opts?: {
         encryptedName: toBase64(encCN),
         nameDecryptionNonce: toBase64(cnN),
         type: "album",
-        updationTime: TEST_TIME,
+        updationTime: 1700000000000000,
     };
 
-    // Generate a real tiny JPEG via jpeg-js, used as the encrypted body of the
+    // Generate a real tiny JPEG via jpeg-js
-    // JPEG files so a repair actually decodes and re-encodes real pixels.
     const w = 100;
     const h = 80;
     const pixels = new Uint8Array(w * h * 4);
@@ -155,32 +131,26 @@ const buildThumbMock = async (opts?: {
         pixels[i + 2] = 0; // B
         pixels[i + 3] = 255; // A
     }
-    const tinyJpeg = new Uint8Array(
+    const tinyJpeg = jpegJs.encode(
-        jpegJs.encode({ data: pixels, width: w, height: h }, 80).data,
+        { data: pixels, width: w, height: h },
-    );
+        80,
+    ).data;
 
     const fileKeys: Record<number, Uint8Array> = {};
     const fileCiphertexts: Record<number, Uint8Array> = {};
+    const rawFiles: Record<string, unknown>[] = [];
 
-    // Build one raw file record: encrypt its metadata and its body under a
+    for (const fileID of [100, 101, 102]) {
-    // fresh per-file key, and record the key and ciphertext for the mock to
-    // serve and for the test to verify against.
-    const makeRawFile = (
-        fileID: number,
-        fileType: number,
-        title: string,
-        body: Uint8Array,
-    ): Record<string, unknown> => {
         const fk = sodium.crypto_secretstream_xchacha20poly1305_keygen();
         fileKeys[fileID] = fk;
         const fkN = sodium.randombytes_buf(sodium.crypto_secretbox_NONCEBYTES);
         const encFK = sodium.crypto_secretbox_easy(fk, fkN, collKey);
 
         const meta = JSON.stringify({
-            title,
+            title: `file-${fileID}.jpg`,
-            fileType,
+            fileType: 0,
-            creationTime: TEST_TIME,
+            creationTime: 1700000000000000,
-            modificationTime: TEST_TIME,
+            modificationTime: 1700000000000000,
         });
         const metaPush =
             sodium.crypto_secretstream_xchacha20poly1305_init_push(fk);
@@ -191,17 +161,18 @@ const buildThumbMock = async (opts?: {
             sodium.crypto_secretstream_xchacha20poly1305_TAG_FINAL,
         );
 
+        // Encrypt the tiny JPEG as the file body
         const filePush =
             sodium.crypto_secretstream_xchacha20poly1305_init_push(fk);
         const encFile = sodium.crypto_secretstream_xchacha20poly1305_push(
             filePush.state,
-            body,
+            new Uint8Array(tinyJpeg),
             null,
             sodium.crypto_secretstream_xchacha20poly1305_TAG_FINAL,
         );
         fileCiphertexts[fileID] = encFile;
 
-        return {
+        rawFiles.push({
             id: fileID,
             collectionID: 1,
             ownerID: 42,
@@ -215,30 +186,8 @@ const buildThumbMock = async (opts?: {
             thumbnail: {
                 decryptionHeader: toBase64(sodium.randombytes_buf(24)),
             },
-            updationTime: TEST_TIME,
+            updationTime: 1700000000000000,
-        };
+        });
-    };
-
-    // Three JPEG files: ok thumbnail, empty thumbnail, 404 thumbnail.
-    const rawFiles: Record<string, unknown>[] = [];
-    for (const fileID of [100, 101, 102]) {
-        rawFiles.push(makeRawFile(fileID, 0, `file-${fileID}.jpg`, tinyJpeg));
-    }
-    const thumbnailBehavior: Record<number, "ok" | "empty" | "404" | "500"> = {
-        100: "ok",
-        101: "empty",
-        102: "404",
-    };
-
-    // For the issue #17 mixed test: a non-JPEG image and a video, both with a
-    // missing (404) thumbnail so they surface in the missing list too.
-    if (opts?.extraFormats) {
-        rawFiles.push(makeRawFile(103, 0, "file-103.png", PNG_BYTES));
-        rawFiles.push(
-            makeRawFile(104, 1, "file-104.mp4", new Uint8Array([0, 0, 0, 1])),
-        );
-        thumbnailBehavior[103] = "404";
-        thumbnailBehavior[104] = "404";
     }
 
     return {
@@ -257,7 +206,11 @@ const buildThumbMock = async (opts?: {
         filesByCollection: { 1: rawFiles },
         fileCiphertexts,
         fileKeys,
-        thumbnailBehavior,
+        thumbnailBehavior: {
+            100: "ok",
+            101: "empty",
+            102: "404",
+        },
         uploadedThumbnails: [],
     };
 };
@@ -428,31 +381,6 @@ const countingFetch = (
     return { fetch: fake as typeof globalThis.fetch, matched: () => matched };
 };
 
-// Open a library over a mock-backed client. As the CLI does for point commands,
-// the background precache is off and the refresh interval is long, and the
-// library client omits `fetchMLData` so no background ML fetch runs. The real
-// `Client` is still used for the API operations the helpers perform directly.
-const openLib = (client: Client): Promise<Library> =>
-    Library.open({
-        client: {
-            whoami: () => client.whoami(),
-            collectionsSince: (args) => client.collectionsSince(args),
-            filesSince: (args) => client.filesSince(args),
-            contentSource: () => client.contentSource(),
-        },
-        cacheDirectory: mkdtempSync(join(tmpRoot, "cache-")),
-        refreshIntervalSeconds: 3600,
-        precacheThumbnails: false,
-        precacheOriginals: false,
-    });
-
-const login = (fetch: typeof globalThis.fetch, retry?: RetryOptions) =>
-    Client.login({
-        email: TEST_EMAIL,
-        password: TEST_PASSWORD,
-        apiOptions: retry ? { fetch, retry } : { fetch },
-    });
-
 // ---------------------------------------------------------------------------
 // Tests
 // ---------------------------------------------------------------------------
@@ -461,21 +389,17 @@ beforeAll(async () => {
     await init();
     await sodium.ready;
     mock = await buildThumbMock();
-    tmpRoot = mkdtempSync(join(tmpdir(), "quak-thumb-test-"));
-});
-
-afterAll(() => {
-    if (tmpRoot && existsSync(tmpRoot))
-        rmSync(tmpRoot, { recursive: true, force: true });
 });
 
 describe("listMissingThumbnails", () => {
     it("identifies files with empty and 404 thumbnails, ignores working ones", async () => {
-        const client = await login(buildThumbFetch(mock));
+        const client = await Client.login({
-        const lib = await openLib(client);
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildThumbFetch(mock) },
+        });
 
-        const missing = await listMissingThumbnails(lib, client);
+        const missing = await listMissingThumbnails(client);
-        lib.close();
 
         // File 100 has a working thumbnail → not reported
         // File 101 has an empty thumbnail → reported
@@ -512,11 +436,13 @@ describe("listMissingThumbnails", () => {
             buildThumbFetch(failingMock),
             (url) => url.includes("thumbnails.ente.io") && url.includes("102"),
         );
-        const client = await login(counted.fetch, { ...noWait });
+        const client = await Client.login({
-        const lib = await openLib(client);
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: counted.fetch, retry: { ...noWait } },
+        });
 
-        const missing = await listMissingThumbnails(lib, client);
+        const missing = await listMissingThumbnails(client);
-        lib.close();
 
         // Only the genuinely empty thumbnail is reported.
         expect(missing.map((m) => m.fileID)).toEqual([101]);
@@ -549,11 +475,13 @@ describe("listMissingThumbnails", () => {
             return inner(input, init);
         }) as typeof globalThis.fetch;
 
-        const client = await login(fetch, { ...noWait });
+        const client = await Client.login({
-        const lib = await openLib(client);
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch, retry: { ...noWait } },
+        });
 
-        const missing = await listMissingThumbnails(lib, client);
+        const missing = await listMissingThumbnails(client);
-        lib.close();
 
         expect(missing.map((m) => m.fileID)).toEqual([101]);
         expect(thumbRequests).toBe(4);
@@ -572,11 +500,13 @@ describe("listMissingThumbnails", () => {
         mockWithDupes.filesByCollection[2] =
             mockWithDupes.filesByCollection[1]!;
 
-        const client = await login(buildThumbFetch(mockWithDupes));
+        const client = await Client.login({
-        const lib = await openLib(client);
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildThumbFetch(mockWithDupes) },
+        });
 
-        const missing = await listMissingThumbnails(lib, client);
+        const missing = await listMissingThumbnails(client);
-        lib.close();
 
         // Should still be 2, not 4 (each file checked only once)
         expect(missing.length).toBe(2);
@@ -586,14 +516,16 @@ describe("listMissingThumbnails", () => {
 describe("fixMissingThumbnails", () => {
     it("downloads original, generates thumbnail, encrypts, uploads, and registers", async () => {
         const fixMock = await buildThumbMock();
-        const client = await login(buildThumbFetch(fixMock));
+        const client = await Client.login({
-        const lib = await openLib(client);
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildThumbFetch(fixMock) },
+        });
 
-        const results = await fixMissingThumbnails(lib, client, [101]);
+        const results = await fixMissingThumbnails(client, [101]);
-        lib.close();
 
         expect(results.length).toBe(1);
-        expect(results[0]!.status).toBe("fixed");
+        expect(results[0]!.success).toBe(true);
         expect(results[0]!.fileID).toBe(101);
         expect(results[0]!.title).toBe("file-101.jpg");
         expect(results[0]!.collection).toBe("Photos");
@@ -623,76 +555,48 @@ describe("fixMissingThumbnails", () => {
 
     it("reports failure for nonexistent file IDs without crashing", async () => {
         const fixMock = await buildThumbMock();
-        const client = await login(buildThumbFetch(fixMock));
+        const client = await Client.login({
-        const lib = await openLib(client);
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildThumbFetch(fixMock) },
+        });
 
-        const results = await fixMissingThumbnails(lib, client, [999]);
+        const results = await fixMissingThumbnails(client, [999]);
-        lib.close();
 
         expect(results.length).toBe(1);
-        expect(results[0]!.status).toBe("failed");
+        expect(results[0]!.success).toBe(false);
         expect(results[0]!.fileID).toBe(999);
-        expect(results[0]!.reason).toContain("not found");
+        expect(results[0]!.error).toContain("not found");
     });
 
     it("continues after one file fails and reports mixed results", async () => {
         const fixMock = await buildThumbMock();
-        // Make file 102 fail by removing its ciphertext so the download 404s.
+        // Make file 102 fail by removing its ciphertext so download fails
         delete fixMock.fileCiphertexts[102];
 
-        const client = await login(buildThumbFetch(fixMock));
+        const client = await Client.login({
-        const lib = await openLib(client);
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildThumbFetch(fixMock) },
+        });
 
-        const results = await fixMissingThumbnails(lib, client, [101, 102]);
+        const results = await fixMissingThumbnails(client, [101, 102]);
-        lib.close();
 
         expect(results.length).toBe(2);
         const success = results.find((r) => r.fileID === 101)!;
         const failure = results.find((r) => r.fileID === 102)!;
-        expect(success.status).toBe("fixed");
+        expect(success.success).toBe(true);
-        expect(failure.status).toBe("failed");
+        expect(failure.success).toBe(false);
-    });
-
-    it("skips a non-JPEG image and a video as unsupported, not failed (issue #17)", async () => {
-        // A PNG and a video both throw inside the JPEG decoder. The helper must
-        // recognise them up front and report "skipped", distinct from a genuine
-        // "failed", and must not upload anything for them. The JPEG in the same
-        // batch is still repaired.
-        const fixMock = await buildThumbMock({ extraFormats: true });
-        const client = await login(buildThumbFetch(fixMock));
-        const lib = await openLib(client);
-
-        const results = await fixMissingThumbnails(
-            lib,
-            client,
-            [101, 103, 104],
-        );
-        lib.close();
-
-        const jpeg = results.find((r) => r.fileID === 101)!;
-        const png = results.find((r) => r.fileID === 103)!;
-        const video = results.find((r) => r.fileID === 104)!;
-
-        expect(jpeg.status).toBe("fixed");
-
-        // The PNG is a still image but not a JPEG: skipped only after its bytes
-        // are inspected.
-        expect(png.status).toBe("skipped");
-        expect(png.reason).toContain("JPEG");
-
-        // The video is skipped from its type alone, before any download.
-        expect(video.status).toBe("skipped");
-        expect(video.reason).toContain("video");
-
-        // Only the JPEG was uploaded; the two skipped files touched no upload.
-        expect(fixMock.uploadedThumbnails.length).toBe(1);
-        expect(fixMock.uploadedThumbnails[0]!.fileID).toBe(101);
     });
 });
 
 describe("Client.getApiClient", () => {
     it("returns the ApiClient when logged in", async () => {
-        const client = await login(buildThumbFetch(mock));
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildThumbFetch(mock) },
+        });
 
         const api = client.getApiClient();
         expect(api).toBeDefined();
@@ -700,7 +604,11 @@ describe("Client.getApiClient", () => {
     });
 
     it("throws after logout", async () => {
-        const client = await login(buildThumbFetch(mock));
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: { fetch: buildThumbFetch(mock) },
+        });
         client.logout();
 
         expect(() => client.getApiClient()).toThrow(/logged out/);