Rewrite the README API reference for the new library surface (closes #53)

Rewrites the README API reference to match the shipped library surface: Library.open options (XDG cache dir + userID, downloadDirectory, refreshIntervalSeconds, precache and cache-size options), default vs fresh reads (Library.fresh()), snapshot/subscribe, albums/photos/timeline, Photo.original/thumbnail, thumbnails.ensure, mldata search, backup, the request pools and on-disk layout, and the CLI (--cache-dir, fresh reads). Every documented signature verified against the code. Notes the deferred content-hash integrity check (#68). Also issue #13.

Model: opus-4-8

README.md

@@ -38,34 +38,47 @@ yarn quak get 67890 --out ./photo.jpg
 yarn quak backup ./my-backup
 ```
 
-For library use:
+For library use, the primary surface is the cache-backed `Library`:
 
 ```ts
-import { Client } from "quak";
+import { Client, Library } from "quak";
 
+// Log in once; the client satisfies the library's client interface.
 const client = await Client.login({
     email: "you@example.com",
     password: "your-password",
 });
 
-for (const c of await client.listCollections()) {
+// Open a cache-backed library. On an empty cache this awaits one server
-    console.log(c.id, c.name);
+// refresh; on an existing cache it returns immediately and refreshes in the
-    const files = await client.listFiles(c.id, c.key);
+// background every `refreshIntervalSeconds` (default 3).
-    for (const f of files) {
+const lib = await Library.open({ client });
-        console.log(`  ${f.metadata.title} [${f.metadata.fileType}]`);
+
+// Default reads answer synchronously from the local cache — no network.
+for (const album of lib.albums.list()) {
+    console.log(album.collectionID, album.name);
+    for (const photo of album.photos.list()) {
+        console.log(`  ${photo.title} [${photo.fileType}]`);
     }
 }
 
-// Download a file
+// Fresh reads await a server round-trip and answer with current state.
-const files = await client.listFiles(collectionID, collectionKey);
+const { albums } = await lib.fresh();
-await client.downloadFile(files[0], "./photo.jpg");
+console.log(`${albums.list().length} albums as of now`);
 
-// Serialize session for later (consumer handles persistence)
+// Fetch (and cache) one photo's full-resolution bytes.
-const snapshot = client.toJSON();
+const photo = lib.photos.byID({ fileID: 12345 });
-// ... later:
+if (photo) {
-const restored = Client.fromJSON(snapshot);
+    const { path } = await photo.original();
+    console.log(`original at ${path}`);
+}
+
+lib.close();
 ```
 
+The lower-level `Client` (login, session serialization, and the raw
+enumeration/download calls) is exported too and documented under Design below.
+
 ## Entrypoints
 
 This repository adheres to the
@@ -429,18 +442,22 @@ quak files --collection <id> [--json]           list files in a collection
 quak get <fileID> [--out path] [--collection]   download and decrypt a file
 quak get-thumb <fileID> [--out] [--collection]  download and decrypt a thumbnail
 quak backup <dir> [--json]                      full incremental backup
+quak backup-metadata <dir> [--exif]             dump all decrypted metadata as JSON
 quak helper list-missing-thumbnails [--json]    find files with missing thumbnails
 quak helper fix-missing-thumbnails [--file ids] generate + upload missing thumbnails
 ```
 
-Every command reads through the local library cache: the first run fetches the
+Every command runs on the same cache-backed library. The read commands —
-account's metadata from the server, and later runs serve from the cache and
+`collections`, `files`, `get`, and `get-thumb` — force a fresh server round-trip
-refresh in the background. `--cache-dir` overrides where that cache lives;
+before they answer, so they report current account state rather than whatever
-without it each account gets its own directory under the per-user cache path.
+the cache last held. `--cache-dir` overrides where the cache lives; without it
+each account gets its own directory under the per-user cache path.
 
 `get` and `get-thumb` resolve the file by ID directly, so `--collection` is
-accepted for backward compatibility but ignored. All listing and backup commands
+accepted for backward compatibility but ignored. `backup-metadata --exif` (alias
-support `--json` for machine-readable output.
+`--all`) additionally downloads each file to extract full EXIF/IPTC/XMP
+metadata. The listing and backup commands support `--json` for machine-readable
+output.
 
 `helper fix-missing-thumbnails` regenerates thumbnails for baseline JPEG images
 only, because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG
@@ -472,7 +489,7 @@ code is non-zero if any files failed.
 
 - [x] Retry policy: no retry on 4xx, exponential backoff on 5xx and network
       errors
-- [ ] Update the API reference section below to match the current implementation
+- [x] Update the API reference section below to match the current implementation
 - [x] `make docker` green
 - [ ] Tag `v1.0.0`
 
@@ -486,25 +503,179 @@ Future (desktop client, separate repo):
 
 ## API reference
 
-The API reference section below is from an earlier draft and does not fully
+The library's primary surface is the cache-backed `Library`; the lower-level
-reflect the current implementation. The authoritative API documentation is in
+`Client` sits underneath it and is covered by the Design sections above. The
-the test files, particularly `test/client/usage.test.ts` which is a literate
+test suite is the canonical, executable documentation — `test/library/` and
-tutorial walking through every operation. Run `yarn test` to verify the examples
+`test/client/usage.test.ts` walk every operation, and `yarn test` verifies them.
-are correct.
 
-The key types and their actual signatures can be found in:
+### Opening a library
 
+`Library.open(options)` loads the on-disk cache, starts the background refresh
+loop, and resolves to a `Library`. On an empty cache it awaits the first refresh
+so it never opens onto empty data; on an existing cache it returns immediately
+and refreshes in the background, so an unreachable server does not block
+opening.
+
+`LibraryOptions`:
+
+| Option                   | Default                     | Meaning                                                               |
+| ------------------------ | --------------------------- | --------------------------------------------------------------------- |
+| `client`                 | required                    | the account client (a `Client`, or any `LibraryClient`)               |
+| `cacheDirectory`         | `<XDG cache>/quak/<userID>` | where `metadata.json` and the content cache live                      |
+| `downloadDirectory`      | none                        | backup destination; an original already stored there counts as cached |
+| `refreshIntervalSeconds` | `3`                         | background refresh cadence                                            |
+| `precacheThumbnails`     | `true`                      | prefetch every thumbnail, newest first                                |
+| `precacheOriginals`      | `true`                      | prefetch the favorites album and the latest-window originals          |
+| `precacheOriginalsDays`  | `7`                         | length in days of that latest window                                  |
+| `cacheOriginalsMaxBytes` | 100 GiB                     | hard ceiling on the originals cache                                   |
+| `freeBelowBytes`         | 50 GiB                      | free space to protect on the volume; the effective limit adapts down  |
+| `isOriginalPinned`       | none                        | extra predicate for originals that must never be evicted              |
+| `pools`                  | fresh `RequestPools`        | the bounded request pools (sets concurrency)                          |
+| `onProgress`             | none                        | refresh/ML/precache progress callback (`RefreshEvent`)                |
+| `contentSource`          | the client's own            | override the byte source (mainly for tests)                           |
+
+Concurrency is set through `pools`: construct
+`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
+and pass it. The three pools default to 10 / 5 / 25 (see Request pools below).
+
+`lib.status()` returns a `LibraryStatus` (collection/file counts, last
+refresh/ML times and errors, originals usage and effective limit, precache
+progress, and `closed`). `lib.close()` stops the background timer; it is
+idempotent, and an in-flight refresh is left to finish.
+
+### Default reads vs. fresh reads
+
+Default reads — `lib.albums`, `lib.photos`, `lib.timeline` — answer
+synchronously from the last refreshed copy held in RAM and never touch the
+network. The background timer refreshes that copy every
+`refreshIntervalSeconds`, so a default read is immediate but may be up to one
+interval stale.
+
+`await lib.fresh()` forces a refresh, waits for it to complete and persist, and
+returns the same `{ albums, photos, timeline }` namespaces — now guaranteed to
+reflect a completed server round-trip. Concurrent `fresh()` calls coalesce onto
+one refresh, and a refresh that fails rejects the caller (default reads stay
+silent and keep serving the last good copy). The CLI's read commands use fresh
+reads (issue https://git.eeqj.de/sneak/quak/issues/75).
+
+### Read surface
+
+- `lib.albums.list()` → `Album[]`, newest-updated first.
+  `lib.albums.byID({ collectionID })` and `byName({ albumName })` →
+  `Album | undefined`.
+- `lib.photos.byID({ fileID })` → `Photo | undefined`.
+  `lib.photos.records({ fileIDs })` → `PhotoRecord[]` in the requested order,
+  each id once, unknown ids dropped.
+- `lib.timeline.groups({ groupBy, filter? })` → `TimelineGroup[]`, grouped by
+  `"day" | "week" | "month"` (keys `YYYY-MM-DD`, ISO `YYYY-Www`, `YYYY-MM`),
+  newest group first. A `PhotoFilter` combines `albumID`, `text`
+  (title/caption/album-name substring), `fileTypes`, `hasLocation`, and
+  `includeArchived`; hidden photos are always excluded.
+
+An `Album` exposes its record fields and `album.photos.list()` → `Photo[]`
+(newest first). A `Photo` exposes its record fields, `photo.record()` →
+`PhotoRecord`, and two content methods:
+
+- `await photo.original(opts?)` → `{ path, bytes }` — the full-resolution file.
+- `await photo.thumbnail(opts?)` → `{ path, bytes }`.
+
+Both serve from the on-disk content cache when the bytes are present and
+otherwise fetch through the pools; `opts.onProgress` reports per-file progress.
+They throw when the library was opened without a content source.
+
+Lower-level accessors that return decrypted model objects (which hold key
+material) are also available: `listCollections()`, `getCollection(id)`,
+`listFiles(collectionID)`, `getFile(collectionID, fileID)`, and
+`getFileByID(fileID)`.
+
+### Records and change notifications
+
+The GUI-facing records hold no key material and no binary, so they survive
+`structuredClone`/JSON across the Electron IPC boundary:
+
+- `PhotoRecord`: `fileID`, `albumIDs`, `title`, `takenAt` (milliseconds),
+  `fileType`, optional `caption` / `width` / `height` / `latitude` /
+  `longitude`, `isArchived`, `isHidden`, and `thumbnailPath` / `originalPath`
+  once the bytes are cached.
+- `AlbumRecord`: `collectionID`, `name`, `type`, `isShared`, `updationTime`, and
+  `fileIDs` (newest first).
+- `LibrarySnapshot`: `{ albums, photos, takenAt }`.
+
+`lib.snapshot()` returns a `LibrarySnapshot` (albums newest-updated first,
+photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
+(`albumsChanged`, `photosChanged`, `fileIDsRemoved`, `albumIDsRemoved`,
+`refreshedAt`) whenever a refresh alters the projection, and returns
+`{ unsubscribe }`; a refresh that changes nothing delivers nothing.
+
+### Thumbnails, ML search, and backup
+
+- `lib.thumbnails.ensure({ fileIDs, priority, signal?, onProgress? })`
+  prefetches thumbnails through the thumbnail pool, deduped by fileID, returning
+  one `EnsureResult` (`{ fileID, path?, error? }`) per file. `priority` is
+  `"visible" | "ahead" | "background"`; only `"visible"` preempts background
+  work.
+- `lib.mldata` searches the CLIP index built from Ente's per-file ML data:
+  `forFile({ fileID })` → `Promise<MLData | undefined>` (the whole stored
+  payload — face boxes, landmarks, embedding — read from disk on demand);
+  `similar({ fileID, limit? })` and `searchByEmbedding({ embedding, limit? })` →
+  `SimilarResult[]` (`{ fileID, score }`, cosine similarity, most similar first,
+  default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes
+  a query vector the caller produced elsewhere.
+- `await lib.backup(opts?)` → `BackupResult`. It refreshes, fetches every
+  in-scope original (and, with `includeThumbnails`, thumbnails) through the
+  content cache, and rebuilds the on-disk backup tree with a durable failure
+  ledger. `BackupOptions`: `downloadDirectory` (falls back to the one `open()`
+  was given), `includeOriginals` (default `true`), `includeThumbnails` (default
+  `false`), `onlyAlbumNames`, and `onProgress`. See Backup layout above for the
+  tree it writes.
+
+### Request pools
+
+`RequestPools` holds three independent bounded pools — metadata (10), content
+(5), thumbnails (25) — because Ente meters these traffic classes differently.
+Each pool orders on-demand work ahead of background/precache work and dedups
+in-flight fetches by key, and an idle pool never lends its slots to a busy one.
+
+### On-disk cache layout
+
+Under `cacheDirectory`:
+
+```
+<cacheDirectory>/
+    metadata.json                 decrypted account state + refresh cursor
+    originals/<fileID>.<ext>      cached full-resolution files
+    thumbnails/<fileID>.jpg       cached thumbnails
+    mldata/
+        <fileID>.json             one decrypted ML payload per file
+        clip.f32, clip.json       the packed CLIP index and its id list
+        fetched.json              per-file fetch bookkeeping
+```
+
+A stored file appears only via an atomic temp-then-rename, so its presence means
+it is complete. The design also calls for a content-hash comparison against
+`FileMetadata.hash` on each fetched original; that check is deferred (issue
+https://git.eeqj.de/sneak/quak/issues/68) because the exact hash construction
+cannot yet be confirmed against the repo's fixtures.
+
+### Key types by source file
+
+- `src/library/index.ts`: `Library`, `LibraryOptions`, `LibraryStatus`,
+  `LibraryClient`, `RefreshEvent`
+- `src/library/read.ts`: `Album`, `Photo`, `AlbumsAPI`, `PhotosAPI`,
+  `TimelineAPI`, `PhotoFilter`, `TimelineGroup`, `GroupBy`
+- `src/library/content.ts`: `ContentResult`, `ContentOptions`, `ThumbnailsAPI`,
+  `EnsureOptions`, `EnsureResult`, `ContentSource`
+- `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`,
+  `LibraryChange`
+- `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult`
+- `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool`
+- `src/backup.ts`: `BackupOptions`, `BackupResult`, `BackupError`
 - `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot`
-- `src/api/client.ts`: `ApiClient`, `ApiClientOptions`, `ApiError`,
+- `src/api/client.ts`: `ApiClient`, `ApiClientOptions`, `StreamOptions`
-  `StreamOptions`
 - `src/errors.ts`: `ApiError`, `TruncatedStreamError`
 - `src/retry.ts`: `withRetry`, `isRetryable`, `isSafeToReplay`, `RetryOptions`
-- `src/auth/types.ts`: `KeyAttributes`, `SRPAttributes`,
+- `src/model/types.ts`: `Collection`, `EnteFile`, `FileMetadata`, `FileType`,
-  `AuthorizationResponse`, `LoginChallenge`
+  `CollectionType`, `RawCollection`, `RawEnteFile`
-- `src/model/types.ts`: `Collection`, `EnteFile`, `FileMetadata`, `FileBlob`,
-  `RawCollection`, `RawEnteFile`, `RawMagicMetadata`
-- `src/download/index.ts`: `DownloadResult`
-- `src/backup.ts`: `BackupResult`, `BackupError`
 - `src/thumbnails.ts`: `MissingThumbnailInfo`, `ThumbnailFixResult`
 
 ## Source attribution

TODO.md

@@ -14,10 +14,19 @@ pre-1.0
 
 # Next Step
 
-Update the README API reference section to match the current implementation.
+Tag v1.0.0.
 
 # Completed Steps
 
+- 2026-09-22: Rewrote the README API reference (and the Getting Started / usage
+  snippets) to match the shipped cache/API library on `next` (issue 53, issue
+  13). Documented `Library.open` and its options, the default-read vs `fresh()`
+  distinction (and that the CLI's read commands are fresh), the record types and
+  `snapshot()`/`subscribe()`, the `albums`/`photos`/`timeline` read surface,
+  `Photo` content methods, `thumbnails.ensure`, the `mldata` search surface,
+  `backup()`, the three request pools (10/5/25), and the on-disk cache layout;
+  noted the deferred content-hash integrity check (issue 68). Docs-only; no code
+  changed.
 - 2026-09-22: Added resumable, deletion-aware enumeration to `Client` (issue 38,
   closes issue 7). `collectionsSince`/`filesSince` take a starting cursor,
   decrypt live records, surface tombstoned ids in a separate `deleted` list (a
@@ -122,7 +131,6 @@ Update the README API reference section to match the current implementation.
 
 # Future Steps
 
-- Tag v1.0.0.
 - Future desktop client, separate repo:
     - Electron app skeleton consuming this library.
     - Local SQLite cache keyed on (collectionID, fileID, updationTime).

bin/quak.ts

@@ -21,6 +21,7 @@ import {
     originalName,
     thumbnailName,
 } from "../src/cli-output.js";
+import { freshCollections, freshFiles, freshFile } from "../src/cli-read.js";
 import { runMetadataBackup } from "../src/metadata-backup.js";
 import {
     listMissingThumbnails,
@@ -183,27 +184,30 @@ program
         await init();
         const client = requireSession();
         const lib = await openReadLibrary(client);
-        const albums = lib.albums.list();
+        // Force a server round-trip and list in enumeration order (issue #36
+        // amendment, issue #52): the pre-library CLI printed current state in
+        // this order, not the albums projection's newest-first order.
+        const collections = await freshCollections(lib);
 
         if (opts.json) {
             stdout.write(
                 JSON.stringify(
-                    albums.map((a) => ({
+                    collections.map((c) => ({
-                        id: a.collectionID,
+                        id: c.id,
-                        name: a.name,
+                        name: c.name,
-                        type: a.type,
+                        type: c.type,
-                        ownerID: lib.getCollection(a.collectionID)?.ownerID,
+                        ownerID: c.ownerID,
-                        isShared: a.isShared,
+                        isShared: c.isShared,
-                        updationTime: a.updationTime,
+                        updationTime: c.updationTime,
                     })),
                     null,
                     2,
                 ) + "\n",
             );
         } else {
-            for (const a of albums) {
+            for (const c of collections) {
                 stdout.write(
-                    `${a.collectionID}\t${a.type}\t${a.name}${a.isShared ? " (shared)" : ""}\n`,
+                    `${c.id}\t${c.type}\t${c.name}${c.isShared ? " (shared)" : ""}\n`,
                 );
             }
         }
@@ -228,21 +232,18 @@ program
         }
 
         const lib = await openReadLibrary(client);
-        const album = lib.albums.byID({ collectionID });
+        // Force a server round-trip and list in enumeration order (issue #36
-        if (!album) {
+        // amendment, issue #52). Each file prints from its own decrypted
+        // metadata (raw title, microsecond creationTime) via cli-output, and in
+        // the pre-library CLI's enumeration order, not the projection's
+        // newest-first order.
+        const files = await freshFiles(lib, collectionID);
+        if (!files) {
             stderr.write(`Collection ${collectionID} not found\n`);
             finish(lib, 1);
             return;
         }
 
-        // Present each file from its own decrypted metadata, not the
-        // PhotoRecord projection, so the raw title and the microsecond
-        // creationTime print as the pre-library CLI did (issue #52). The
-        // album's photo order is kept; only the field source changes.
-        const files = album.photos.list().flatMap((p) => {
-            const file = lib.getFile(collectionID, p.fileID);
-            return file ? [file] : [];
-        });
         if (opts.json) {
             stdout.write(
                 JSON.stringify(files.map(fileListRow), null, 2) + "\n",
@@ -271,13 +272,15 @@ program
         }
 
         const lib = await openReadLibrary(client);
-        const photo = lib.photos.byID({ fileID });
+        // Force a server round-trip so the file resolves against current state
-        const file = lib.getFileByID(fileID);
+        // (issue #36 amendment, issue #52).
-        if (!photo || !file) {
+        const resolved = await freshFile(lib, fileID);
+        if (!resolved) {
             stderr.write(`File ${fileID} not found\n`);
             finish(lib, 1);
             return;
         }
+        const { photo, file } = resolved;
 
         const result = await photo.original();
         // Default name is the file's own title, as the pre-library CLI used
@@ -304,13 +307,15 @@ program
         }
 
         const lib = await openReadLibrary(client);
-        const photo = lib.photos.byID({ fileID });
+        // Force a server round-trip so the file resolves against current state
-        const file = lib.getFileByID(fileID);
+        // (issue #36 amendment, issue #52).
-        if (!photo || !file) {
+        const resolved = await freshFile(lib, fileID);
+        if (!resolved) {
             stderr.write(`File ${fileID} not found\n`);
             finish(lib, 1);
             return;
         }
+        const { photo, file } = resolved;
 
         const result = await photo.thumbnail();
         // Default name is thumb_<file's own title>, as the pre-library CLI

src/cli-read.ts

@@ -0,0 +1,62 @@
+// How the CLI's read commands obtain current data.
+//
+// `collections`, `files --collection`, `get`, and `get-thumb` must answer for
+// the account's state at the moment the command runs, not for whatever the
+// local cache last happened to hold (owner amendment, issue #36). Each helper
+// therefore forces a server round-trip through `Library.fresh()` and only then
+// reads — so a collection, file, or metadata change made elsewhere is visible.
+//
+// `collections` and `files` also list in the library's own enumeration order —
+// `listCollections()`/`listFiles()`, the order the pre-library CLI printed —
+// rather than the `albums`/`photos` projection's newest-first order, which
+// re-sorts the rows. The field values still come from each record's raw
+// metadata via `cli-output.ts`.
+
+import type { Collection, EnteFile } from "./model/types.js";
+import type { Photo, PhotosAPI } from "./library/index.js";
+
+// The slice of `Library` these helpers read. `Library` satisfies it
+// structurally; a test can drive them with a stand-in that records the
+// `fresh()` call and serves records in a known enumeration order.
+export interface FreshReadLibrary {
+    fresh(): Promise<unknown>;
+    listCollections(): Collection[];
+    getCollection(id: number): Collection | undefined;
+    listFiles(collectionID: number): EnteFile[];
+    getFileByID(fileID: number): EnteFile | undefined;
+    photos: Pick<PhotosAPI, "byID">;
+}
+
+// Every live collection, current as of a forced refresh, in enumeration order.
+export const freshCollections = async (
+    lib: FreshReadLibrary,
+): Promise<Collection[]> => {
+    await lib.fresh();
+    return lib.listCollections();
+};
+
+// The files of one collection, current as of a forced refresh, in enumeration
+// order. `undefined` (not an empty list) when the collection does not exist, so
+// the caller can tell "no such collection" from "an empty collection".
+export const freshFiles = async (
+    lib: FreshReadLibrary,
+    collectionID: number,
+): Promise<EnteFile[] | undefined> => {
+    await lib.fresh();
+    if (!lib.getCollection(collectionID)) return undefined;
+    return lib.listFiles(collectionID);
+};
+
+// One file, current as of a forced refresh, resolved to both its content
+// handle (`Photo`, for fetching bytes) and its raw record (`EnteFile`, for the
+// default output name and field values). `undefined` when the file is unknown.
+export const freshFile = async (
+    lib: FreshReadLibrary,
+    fileID: number,
+): Promise<{ photo: Photo; file: EnteFile } | undefined> => {
+    await lib.fresh();
+    const photo = lib.photos.byID({ fileID });
+    const file = lib.getFileByID(fileID);
+    if (!photo || !file) return undefined;
+    return { photo, file };
+};

src/library/index.ts

@@ -6,10 +6,17 @@
 // existing copy loaded, that first refresh runs in the background and `open()`
 // returns as soon as the cached data is ready to serve — a slow or unreachable
 // server no longer stalls opening. A background timer then refreshes every
-// `refreshIntervalSeconds`. Every read is answered from RAM — no read touches
+// `refreshIntervalSeconds`. Every default read is answered from RAM — no
-// the network. There is deliberately no `sync()`, no `refresh()`, no
+// default read touches the network. There is deliberately no `sync()`, no
-// `serverReachable` flag, and no "before each read" mode (design #36): the
+// `refresh()`, no `serverReachable` flag, and no "before each read" mode
-// only ways state changes are the refreshes above.
+// (design #36).
+//
+// `fresh()` is the one exception (issue #75, an owner amendment to #36): it
+// forces a refresh, awaits it, and only then hands back the read namespaces, so
+// a caller that needs server-current data can ask for it. Concurrent `fresh()`
+// calls coalesce onto one in-flight refresh, and a refresh that fails rejects
+// the caller (the default reads stay silent and serve the last good copy). The
+// default methods and the background loop are unchanged.
 //
 // A refresh stages all of its network work first and only mutates the store
 // once every fetch has succeeded. A refresh that fails partway therefore never
@@ -40,6 +47,7 @@ import {
     type AlbumsAPI,
     type PhotosAPI,
     type TimelineAPI,
+    type FreshReads,
 } from "./read.js";
 import {
     ContentCache,
@@ -57,6 +65,7 @@ export {
     type AlbumsAPI,
     type PhotosAPI,
     type TimelineAPI,
+    type FreshReads,
     type PhotoFilter,
     type TimelineGroup,
     type GroupBy,
@@ -250,7 +259,12 @@ export class Library {
     private readonly precache?: Precache;
 
     private timer?: ReturnType<typeof setTimeout>;
-    private refreshing = false;
+    // The in-flight refresh cycle, or undefined when none runs. One slot serves
+    // both paths: the background loop skips when it is set, and a fresh read
+    // (issue #75) coalesces onto it or starts one. The promise carries the
+    // cycle's real outcome (it rejects on failure); the background loop ignores
+    // that, a fresh read propagates it.
+    private cycle?: Promise<void>;
     // Guards the ML fetch pass so a slow backfill never runs twice at once; a
     // refresh whose pass is still running kicks nothing new.
     private mlFetching = false;
@@ -491,6 +505,21 @@ export class Library {
         };
     }
 
+    // Fresh reads (issue #75, owner amendment to design #36). Force a refresh,
+    // wait for it to complete and persist, then hand back the same
+    // `albums`/`photos`/`timeline` namespaces — now guaranteed to reflect a
+    // completed server round-trip. Concurrent calls coalesce onto one refresh;
+    // a refresh that fails rejects here, where the default namespaces would
+    // instead stay silent and serve the last good copy.
+    async fresh(): Promise<FreshReads> {
+        await this.refreshNow();
+        return {
+            albums: this.albums,
+            photos: this.photos,
+            timeline: this.timeline,
+        };
+    }
+
     // Back up every in-scope file to `downloadDirectory` in the historical
     // on-disk layout, with a durable failure ledger (issue #51). Refreshes
     // first, fetches pending originals (and optional thumbnails) through the
@@ -550,11 +579,48 @@ export class Library {
         this.timer.unref?.();
     }
 
-    // One refresh cycle, guarded so a failure never escapes and overlapping
+    // The background loop's refresh: run a cycle unless one is already in flight
-    // cycles never run. Errors are reported, not thrown.
+    // (or the library is closed), and never let a failure escape — the
-    private async runRefresh(): Promise<void> {
+    // background path reports errors through `status()`/`onProgress`, it does
-        if (this.closed || this.refreshing) return;
+    // not throw. Resolves once the cycle it started (or skipped past) settles.
-        this.refreshing = true;
+    private runRefresh(): Promise<void> {
+        if (this.closed || this.cycle) return Promise.resolve();
+        return this.startCycle().catch(() => {});
+    }
+
+    // A fresh read's refresh (issue #75): force a cycle and await it, rejecting
+    // if it fails. Concurrent fresh reads coalesce onto the one in-flight cycle
+    // — the background loop's included — so they never fan out into redundant
+    // server round-trips.
+    private refreshNow(): Promise<void> {
+        if (this.closed) {
+            return Promise.reject(new Error("the library is closed"));
+        }
+        return this.cycle ?? this.startCycle();
+    }
+
+    // Start one refresh cycle and record it as the in-flight cycle so every
+    // caller coalesces onto it. The returned promise carries the cycle's real
+    // outcome; each caller attaches the handling its own path needs, and the
+    // slot is cleared once the cycle settles.
+    private startCycle(): Promise<void> {
+        const cycle = this.refreshCycle();
+        this.cycle = cycle;
+        void cycle.then(
+            () => {
+                if (this.cycle === cycle) this.cycle = undefined;
+            },
+            () => {
+                if (this.cycle === cycle) this.cycle = undefined;
+            },
+        );
+        return cycle;
+    }
+
+    // One refresh cycle: the network fetch and commit, wrapped in the progress
+    // events and status bookkeeping. Throws when the refresh fails so a fresh
+    // read can reject; `runRefresh` swallows that throw for the background loop.
+    private async refreshCycle(): Promise<void> {
         this.emit({ operation: "refresh", status: "started" });
         try {
             await this.refreshOnce();
@@ -570,8 +636,7 @@ export class Library {
             const error = err instanceof Error ? err.message : String(err);
             this.lastError = error;
             this.emit({ operation: "refresh", status: "failed", error });
-        } finally {
+            throw err;
-            this.refreshing = false;
         }
     }
 

src/library/read.ts

@@ -193,6 +193,15 @@ export interface TimelineAPI {
     groups(args: { groupBy: GroupBy; filter?: PhotoFilter }): TimelineGroup[];
 }
 
+// The surface `Library.fresh()` resolves to (issue #75). It is the same three
+// read namespaces as the default `albums`/`photos`/`timeline`, handed back only
+// after a forced refresh has brought the local copy current.
+export interface FreshReads {
+    albums: AlbumsAPI;
+    photos: PhotosAPI;
+    timeline: TimelineAPI;
+}
+
 export const makeAlbumsAPI = (
     derive: () => DerivedRecords,
     content?: PhotoContent,

test/cli/read.test.ts

@@ -0,0 +1,195 @@
+/**
+ * Tests for the CLI read helpers (`src/cli-read.ts`, owner amendment to
+ * issue #36, issue #52).
+ *
+ * The `collections`, `files`, `get`, and `get-thumb` commands must answer for
+ * current server state, not the local cache, so each helper forces a
+ * `Library.fresh()` round-trip before it reads. The stand-in library below
+ * serves nothing until `fresh()` has been awaited, so a helper that read
+ * without refreshing would come back empty and fail here.
+ *
+ * `collections` and `files` also list in the library's enumeration order
+ * (`listCollections`/`listFiles`) — the order the pre-library CLI printed — not
+ * the albums/photos projection's newest-first order. The fixtures are seeded in
+ * an enumeration order that a newest-first sort would rearrange, so a
+ * regression to the projection order would fail here too. Field values still
+ * come from the raw metadata via `cli-output.ts`.
+ */
+
+import { describe, it, expect } from "vitest";
+
+import {
+    freshCollections,
+    freshFiles,
+    freshFile,
+    type FreshReadLibrary,
+} from "../../src/cli-read.js";
+import { fileListRow } from "../../src/cli-output.js";
+import type { Photo } from "../../src/library/index.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+const collection = (id: number, updationTime: number): Collection => ({
+    id,
+    ownerID: 42,
+    key: new Uint8Array(),
+    name: `album-${id}`,
+    type: "album",
+    updationTime,
+    isShared: false,
+});
+
+// Microseconds, as Ente stores times.
+const file = (
+    id: number,
+    collectionID: number,
+    creationTime: number,
+): EnteFile => ({
+    id,
+    collectionID,
+    ownerID: 42,
+    key: new Uint8Array(),
+    metadata: {
+        title: `file-${id}.jpg`,
+        fileType: "image",
+        creationTime,
+        modificationTime: creationTime,
+    },
+    file: { decryptionHeader: "" },
+    thumbnail: { decryptionHeader: "" },
+    updationTime: creationTime,
+});
+
+// A library that reveals its records only after `fresh()` has been awaited, and
+// serves them in the enumeration order it was given. `photos.byID` returns a
+// stand-in `Photo` carrying just the fileID the helper passes through.
+class FakeLibrary implements FreshReadLibrary {
+    freshCalls = 0;
+    private refreshed = false;
+
+    constructor(
+        private readonly collections: Collection[],
+        private readonly files: EnteFile[],
+    ) {}
+
+    async fresh(): Promise<unknown> {
+        this.freshCalls++;
+        this.refreshed = true;
+        return {};
+    }
+
+    listCollections(): Collection[] {
+        return this.refreshed ? this.collections : [];
+    }
+
+    getCollection(id: number): Collection | undefined {
+        return this.listCollections().find((c) => c.id === id);
+    }
+
+    listFiles(collectionID: number): EnteFile[] {
+        return this.refreshed
+            ? this.files.filter((f) => f.collectionID === collectionID)
+            : [];
+    }
+
+    getFileByID(fileID: number): EnteFile | undefined {
+        if (!this.refreshed) return undefined;
+        return this.files.find((f) => f.id === fileID);
+    }
+
+    photos = {
+        byID: ({ fileID }: { fileID: number }): Photo | undefined => {
+            if (!this.refreshed) return undefined;
+            if (!this.files.some((f) => f.id === fileID)) return undefined;
+            return { fileID } as unknown as Photo;
+        },
+    };
+}
+
+describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
+    it("freshCollections refreshes first, then lists in enumeration order", async () => {
+        // Enumeration order 2, 1, 3; a newest-first sort would be 3, 2, 1.
+        const lib = new FakeLibrary(
+            [collection(2, 200), collection(1, 300), collection(3, 100)],
+            [],
+        );
+        const rows = await freshCollections(lib);
+
+        expect(lib.freshCalls).toBe(1);
+        expect(rows.map((c) => c.id)).toEqual([2, 1, 3]);
+        // The projection's newest-first order is a different sequence, so this
+        // is not accidentally that order.
+        const newestFirst = [...rows]
+            .sort((a, b) => b.updationTime - a.updationTime)
+            .map((c) => c.id);
+        expect(newestFirst).toEqual([1, 2, 3]);
+        expect(rows.map((c) => c.id)).not.toEqual(newestFirst);
+    });
+
+    it("freshFiles refreshes first, lists in enumeration order, keeps raw fields", async () => {
+        // Enumeration order by id 10, 11, 12; creationTimes ascending, so a
+        // newest-first sort would reverse them.
+        const files = [
+            file(10, 1, 1_700_000_000_000_000),
+            file(11, 1, 1_700_000_000_000_001),
+            file(12, 1, 1_700_000_000_000_002),
+        ];
+        const lib = new FakeLibrary([collection(1, 100)], files);
+
+        const rows = await freshFiles(lib, 1);
+        expect(lib.freshCalls).toBe(1);
+        expect(rows?.map((f) => f.id)).toEqual([10, 11, 12]);
+
+        // Field values come from raw metadata: microsecond creationTime and the
+        // raw title, unchanged.
+        expect(rows?.map(fileListRow)).toEqual([
+            {
+                id: 10,
+                title: "file-10.jpg",
+                fileType: "image",
+                creationTime: 1_700_000_000_000_000,
+                collectionID: 1,
+            },
+            {
+                id: 11,
+                title: "file-11.jpg",
+                fileType: "image",
+                creationTime: 1_700_000_000_000_001,
+                collectionID: 1,
+            },
+            {
+                id: 12,
+                title: "file-12.jpg",
+                fileType: "image",
+                creationTime: 1_700_000_000_000_002,
+                collectionID: 1,
+            },
+        ]);
+    });
+
+    it("freshFiles returns undefined for an unknown collection", async () => {
+        const lib = new FakeLibrary([collection(1, 100)], []);
+        const rows = await freshFiles(lib, 999);
+        expect(lib.freshCalls).toBe(1);
+        expect(rows).toBeUndefined();
+    });
+
+    it("freshFile refreshes first, then resolves the photo and its raw record", async () => {
+        const f = file(10, 1, 1_700_000_000_000_000);
+        const lib = new FakeLibrary([collection(1, 100)], [f]);
+
+        const resolved = await freshFile(lib, 10);
+        expect(lib.freshCalls).toBe(1);
+        expect(resolved?.photo.fileID).toBe(10);
+        expect(resolved?.file.metadata.title).toBe("file-10.jpg");
+        expect(resolved?.file.metadata.creationTime).toBe(
+            1_700_000_000_000_000,
+        );
+    });
+
+    it("freshFile returns undefined for an unknown file", async () => {
+        const lib = new FakeLibrary([collection(1, 100)], []);
+        const resolved = await freshFile(lib, 404);
+        expect(lib.freshCalls).toBe(1);
+        expect(resolved).toBeUndefined();
+    });
+});

test/library/fresh.test.ts

@@ -0,0 +1,272 @@
+/**
+ * Tests for fresh reads (issue #75, an owner amendment to design #36).
+ *
+ * The default read namespaces answer from RAM and never touch the network; a
+ * background loop keeps the local copy current. `fresh()` adds an awaited path:
+ * it forces a refresh, waits for it to complete and persist, and only then
+ * hands back the `albums`/`photos`/`timeline` namespaces, guaranteeing the
+ * local copy reflects a completed server round-trip. The contracts here:
+ *
+ *  1. A fresh read observes a server change that a same-instant default read
+ *     would miss (the default path has not refreshed yet).
+ *  2. Concurrent fresh reads coalesce onto one in-flight refresh — three
+ *     concurrent `fresh()` calls make exactly one collections round-trip, not
+ *     three.
+ *  3. A refresh that fails rejects the fresh read (currency was unavailable),
+ *     while the default reads stay silent and keep serving the last good copy.
+ *
+ * The client is the same metadata-only mock the background-refresh tests use:
+ * no crypto, no network, scripted pages, and a record of each call. A long
+ * refresh interval keeps the background timer out of the way so each test's
+ * refreshes are exactly the ones it triggers.
+ */
+
+import { describe, it, expect, beforeEach, afterEach } from "vitest";
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { Library } from "../../src/library/index.js";
+import type { CollectionsPage, FilesPage } from "../../src/client.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+const USER_ID = 42;
+
+// Long enough that no background tick fires during a test; each test's
+// refreshes are only the ones its own `fresh()` calls force.
+const SLOW_INTERVAL = 3600;
+
+const collection = (id: number, updationTime: number): Collection => ({
+    id,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    name: `album-${id}`,
+    type: "album",
+    updationTime,
+    isShared: false,
+});
+
+const file = (
+    id: number,
+    collectionID: number,
+    updationTime: number,
+): EnteFile => ({
+    id,
+    collectionID,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    metadata: {
+        title: `file-${id}.jpg`,
+        fileType: "image",
+        creationTime: updationTime,
+        modificationTime: updationTime,
+    },
+    file: { decryptionHeader: "aGVhZGVy" },
+    thumbnail: { decryptionHeader: "dGh1bWI=" },
+    updationTime,
+});
+
+class MockClient {
+    userID = USER_ID;
+    failCollections = false;
+    collectionsQueue: CollectionsPage[] = [];
+    filesByCollection = new Map<number, FilesPage[]>();
+
+    collectionsSinceTimes: number[] = [];
+    filesCalls: { collectionID: number; sinceTime: number }[] = [];
+
+    whoami(): { email: string; userID: number } {
+        return { email: "user@example.com", userID: this.userID };
+    }
+
+    async collectionsSince(args: {
+        sinceTime: number;
+    }): Promise<CollectionsPage> {
+        this.collectionsSinceTimes.push(args.sinceTime);
+        if (this.failCollections) throw new Error("network down");
+        return (
+            this.collectionsQueue.shift() ?? {
+                collections: [],
+                deleted: [],
+                cursor: args.sinceTime,
+            }
+        );
+    }
+
+    async filesSince(args: {
+        collectionID: number;
+        collectionKey: Uint8Array;
+        sinceTime: number;
+    }): Promise<FilesPage> {
+        this.filesCalls.push({
+            collectionID: args.collectionID,
+            sinceTime: args.sinceTime,
+        });
+        const queue = this.filesByCollection.get(args.collectionID);
+        return (
+            queue?.shift() ?? {
+                files: [],
+                deleted: [],
+                cursor: args.sinceTime,
+            }
+        );
+    }
+
+    filesFor(collectionID: number, ...pages: FilesPage[]): void {
+        this.filesByCollection.set(collectionID, pages);
+    }
+}
+
+// A client seeded with one collection and one file, opened with an empty cache
+// so the initial refresh is awaited and post-`open()` state is deterministic.
+const openSeeded = async (
+    cacheDirectory: string,
+): Promise<{ client: MockClient; lib: Library }> => {
+    const client = new MockClient();
+    client.collectionsQueue.push({
+        collections: [collection(1, 100)],
+        deleted: [],
+        cursor: 100,
+    });
+    client.filesFor(1, {
+        files: [file(1001, 1, 90)],
+        deleted: [],
+        cursor: 90,
+    });
+    const lib = await Library.open({
+        client,
+        cacheDirectory,
+        refreshIntervalSeconds: SLOW_INTERVAL,
+    });
+    return { client, lib };
+};
+
+describe("Library.fresh", () => {
+    let dir: string;
+    let cacheDirectory: string;
+
+    beforeEach(() => {
+        dir = mkdtempSync(join(tmpdir(), "quak-fresh-"));
+        cacheDirectory = join(dir, "cache");
+    });
+
+    afterEach(() => {
+        rmSync(dir, { recursive: true, force: true });
+    });
+
+    it("observes a server change a same-instant default read would miss", async () => {
+        const { client, lib } = await openSeeded(cacheDirectory);
+        try {
+            // A new file appears on the server after open, advancing its
+            // collection so the next refresh re-enumerates it.
+            client.collectionsQueue.push({
+                collections: [collection(1, 200)],
+                deleted: [],
+                cursor: 200,
+            });
+            client.filesFor(1, {
+                files: [file(1002, 1, 190)],
+                deleted: [],
+                cursor: 190,
+            });
+
+            // A default read at this instant has not refreshed: it misses 1002.
+            expect(lib.photos.byID({ fileID: 1002 })).toBeUndefined();
+
+            // A fresh read forces the round-trip and sees it.
+            const reads = await lib.fresh();
+            expect(reads.photos.byID({ fileID: 1002 })?.fileID).toBe(1002);
+            // And the change is now live for the default namespaces too.
+            expect(lib.photos.byID({ fileID: 1002 })?.fileID).toBe(1002);
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("coalesces concurrent fresh reads onto one in-flight refresh", async () => {
+        const { client, lib } = await openSeeded(cacheDirectory);
+        try {
+            client.collectionsQueue.push({
+                collections: [collection(1, 200)],
+                deleted: [],
+                cursor: 200,
+            });
+            client.filesFor(1, {
+                files: [file(1002, 1, 190)],
+                deleted: [],
+                cursor: 190,
+            });
+
+            const collectionsBefore = client.collectionsSinceTimes.length;
+            const filesBefore = client.filesCalls.length;
+
+            // Gate the next collections fetch so all three fresh reads are in
+            // flight together before any of them completes.
+            let release: () => void = () => {};
+            const gate = new Promise<void>((r) => {
+                release = r;
+            });
+            const inner = client.collectionsSince.bind(client);
+            client.collectionsSince = async (args: { sinceTime: number }) => {
+                await gate;
+                return inner(args);
+            };
+
+            const all = Promise.all([lib.fresh(), lib.fresh(), lib.fresh()]);
+            release();
+            const [a, b, c] = await all;
+
+            // Exactly one collections round-trip and one file round-trip served
+            // all three fresh reads.
+            expect(client.collectionsSinceTimes.length).toBe(
+                collectionsBefore + 1,
+            );
+            expect(client.filesCalls.length).toBe(filesBefore + 1);
+
+            // All three observed the change.
+            for (const reads of [a, b, c]) {
+                expect(reads.photos.byID({ fileID: 1002 })?.fileID).toBe(1002);
+            }
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("rejects the fresh read when the refresh fails, leaving defaults intact", async () => {
+        const { client, lib } = await openSeeded(cacheDirectory);
+        try {
+            client.failCollections = true;
+
+            // Two concurrent fresh reads both reject, and share one failed
+            // round-trip rather than each making its own.
+            const collectionsBefore = client.collectionsSinceTimes.length;
+            const first = lib.fresh();
+            const second = lib.fresh();
+            await expect(first).rejects.toThrow(/network down/);
+            await expect(second).rejects.toThrow(/network down/);
+            expect(client.collectionsSinceTimes.length).toBe(
+                collectionsBefore + 1,
+            );
+
+            // The default reads never rejected: they still serve the last good
+            // copy, and the failure surfaced through status().
+            expect(lib.photos.byID({ fileID: 1001 })?.fileID).toBe(1001);
+            expect(lib.status().lastError).toMatch(/network down/);
+
+            // Recovery: once the server answers, a fresh read resolves current.
+            client.failCollections = false;
+            client.collectionsQueue.push({
+                collections: [collection(2, 300)],
+                deleted: [],
+                cursor: 300,
+            });
+            const reads = await lib.fresh();
+            expect(reads.albums.byID({ collectionID: 2 })?.collectionID).toBe(
+                2,
+            );
+            expect(lib.status().lastError).toBeUndefined();
+        } finally {
+            lib.close();
+        }
+    });
+});