Rewrite the README API reference for the new library surface (closes #53)

Rewrites the README API reference to match the shipped library surface: Library.open options (XDG cache dir + userID, downloadDirectory, refreshIntervalSeconds, precache and cache-size options), default vs fresh reads (Library.fresh()), snapshot/subscribe, albums/photos/timeline, Photo.original/thumbnail, thumbnails.ensure, mldata search, backup, the request pools and on-disk layout, and the CLI (--cache-dir, fresh reads). Every documented signature verified against the code. Notes the deferred content-hash integrity check (#68). Also issue #13.

Model: opus-4-8

README.md

@@ -38,34 +38,47 @@ yarn quak get 67890 --out ./photo.jpg
 yarn quak backup ./my-backup
 ```
 
-For library use:
+For library use, the primary surface is the cache-backed `Library`:
 
 ```ts
-import { Client } from "quak";
+import { Client, Library } from "quak";
 
+// Log in once; the client satisfies the library's client interface.
 const client = await Client.login({
     email: "you@example.com",
     password: "your-password",
 });
 
-for (const c of await client.listCollections()) {
+// Open a cache-backed library. On an empty cache this awaits one server
-    console.log(c.id, c.name);
+// refresh; on an existing cache it returns immediately and refreshes in the
-    const files = await client.listFiles(c.id, c.key);
+// background every `refreshIntervalSeconds` (default 3).
-    for (const f of files) {
+const lib = await Library.open({ client });
-        console.log(`  ${f.metadata.title} [${f.metadata.fileType}]`);
+
+// Default reads answer synchronously from the local cache — no network.
+for (const album of lib.albums.list()) {
+    console.log(album.collectionID, album.name);
+    for (const photo of album.photos.list()) {
+        console.log(`  ${photo.title} [${photo.fileType}]`);
     }
 }
 
-// Download a file
+// Fresh reads await a server round-trip and answer with current state.
-const files = await client.listFiles(collectionID, collectionKey);
+const { albums } = await lib.fresh();
-await client.downloadFile(files[0], "./photo.jpg");
+console.log(`${albums.list().length} albums as of now`);
 
-// Serialize session for later (consumer handles persistence)
+// Fetch (and cache) one photo's full-resolution bytes.
-const snapshot = client.toJSON();
+const photo = lib.photos.byID({ fileID: 12345 });
-// ... later:
+if (photo) {
-const restored = Client.fromJSON(snapshot);
+    const { path } = await photo.original();
+    console.log(`original at ${path}`);
+}
+
+lib.close();
 ```
 
+The lower-level `Client` (login, session serialization, and the raw
+enumeration/download calls) is exported too and documented under Design below.
+
 ## Entrypoints
 
 This repository adheres to the
@@ -429,18 +442,22 @@ quak files --collection <id> [--json]           list files in a collection
 quak get <fileID> [--out path] [--collection]   download and decrypt a file
 quak get-thumb <fileID> [--out] [--collection]  download and decrypt a thumbnail
 quak backup <dir> [--json]                      full incremental backup
+quak backup-metadata <dir> [--exif]             dump all decrypted metadata as JSON
 quak helper list-missing-thumbnails [--json]    find files with missing thumbnails
 quak helper fix-missing-thumbnails [--file ids] generate + upload missing thumbnails
 ```
 
-Every command reads through the local library cache: the first run fetches the
+Every command runs on the same cache-backed library. The read commands —
-account's metadata from the server, and later runs serve from the cache and
+`collections`, `files`, `get`, and `get-thumb` — force a fresh server round-trip
-refresh in the background. `--cache-dir` overrides where that cache lives;
+before they answer, so they report current account state rather than whatever
-without it each account gets its own directory under the per-user cache path.
+the cache last held. `--cache-dir` overrides where the cache lives; without it
+each account gets its own directory under the per-user cache path.
 
 `get` and `get-thumb` resolve the file by ID directly, so `--collection` is
-accepted for backward compatibility but ignored. All listing and backup commands
+accepted for backward compatibility but ignored. `backup-metadata --exif` (alias
-support `--json` for machine-readable output.
+`--all`) additionally downloads each file to extract full EXIF/IPTC/XMP
+metadata. The listing and backup commands support `--json` for machine-readable
+output.
 
 `helper fix-missing-thumbnails` regenerates thumbnails for baseline JPEG images
 only, because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG
@@ -472,7 +489,7 @@ code is non-zero if any files failed.
 
 - [x] Retry policy: no retry on 4xx, exponential backoff on 5xx and network
       errors
-- [ ] Update the API reference section below to match the current implementation
+- [x] Update the API reference section below to match the current implementation
 - [x] `make docker` green
 - [ ] Tag `v1.0.0`
 
@@ -486,25 +503,179 @@ Future (desktop client, separate repo):
 
 ## API reference
 
-The API reference section below is from an earlier draft and does not fully
+The library's primary surface is the cache-backed `Library`; the lower-level
-reflect the current implementation. The authoritative API documentation is in
+`Client` sits underneath it and is covered by the Design sections above. The
-the test files, particularly `test/client/usage.test.ts` which is a literate
+test suite is the canonical, executable documentation — `test/library/` and
-tutorial walking through every operation. Run `yarn test` to verify the examples
+`test/client/usage.test.ts` walk every operation, and `yarn test` verifies them.
-are correct.
 
-The key types and their actual signatures can be found in:
+### Opening a library
 
+`Library.open(options)` loads the on-disk cache, starts the background refresh
+loop, and resolves to a `Library`. On an empty cache it awaits the first refresh
+so it never opens onto empty data; on an existing cache it returns immediately
+and refreshes in the background, so an unreachable server does not block
+opening.
+
+`LibraryOptions`:
+
+| Option                   | Default                     | Meaning                                                               |
+| ------------------------ | --------------------------- | --------------------------------------------------------------------- |
+| `client`                 | required                    | the account client (a `Client`, or any `LibraryClient`)               |
+| `cacheDirectory`         | `<XDG cache>/quak/<userID>` | where `metadata.json` and the content cache live                      |
+| `downloadDirectory`      | none                        | backup destination; an original already stored there counts as cached |
+| `refreshIntervalSeconds` | `3`                         | background refresh cadence                                            |
+| `precacheThumbnails`     | `true`                      | prefetch every thumbnail, newest first                                |
+| `precacheOriginals`      | `true`                      | prefetch the favorites album and the latest-window originals          |
+| `precacheOriginalsDays`  | `7`                         | length in days of that latest window                                  |
+| `cacheOriginalsMaxBytes` | 100 GiB                     | hard ceiling on the originals cache                                   |
+| `freeBelowBytes`         | 50 GiB                      | free space to protect on the volume; the effective limit adapts down  |
+| `isOriginalPinned`       | none                        | extra predicate for originals that must never be evicted              |
+| `pools`                  | fresh `RequestPools`        | the bounded request pools (sets concurrency)                          |
+| `onProgress`             | none                        | refresh/ML/precache progress callback (`RefreshEvent`)                |
+| `contentSource`          | the client's own            | override the byte source (mainly for tests)                           |
+
+Concurrency is set through `pools`: construct
+`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
+and pass it. The three pools default to 10 / 5 / 25 (see Request pools below).
+
+`lib.status()` returns a `LibraryStatus` (collection/file counts, last
+refresh/ML times and errors, originals usage and effective limit, precache
+progress, and `closed`). `lib.close()` stops the background timer; it is
+idempotent, and an in-flight refresh is left to finish.
+
+### Default reads vs. fresh reads
+
+Default reads — `lib.albums`, `lib.photos`, `lib.timeline` — answer
+synchronously from the last refreshed copy held in RAM and never touch the
+network. The background timer refreshes that copy every
+`refreshIntervalSeconds`, so a default read is immediate but may be up to one
+interval stale.
+
+`await lib.fresh()` forces a refresh, waits for it to complete and persist, and
+returns the same `{ albums, photos, timeline }` namespaces — now guaranteed to
+reflect a completed server round-trip. Concurrent `fresh()` calls coalesce onto
+one refresh, and a refresh that fails rejects the caller (default reads stay
+silent and keep serving the last good copy). The CLI's read commands use fresh
+reads (issue https://git.eeqj.de/sneak/quak/issues/75).
+
+### Read surface
+
+- `lib.albums.list()` → `Album[]`, newest-updated first.
+  `lib.albums.byID({ collectionID })` and `byName({ albumName })` →
+  `Album | undefined`.
+- `lib.photos.byID({ fileID })` → `Photo | undefined`.
+  `lib.photos.records({ fileIDs })` → `PhotoRecord[]` in the requested order,
+  each id once, unknown ids dropped.
+- `lib.timeline.groups({ groupBy, filter? })` → `TimelineGroup[]`, grouped by
+  `"day" | "week" | "month"` (keys `YYYY-MM-DD`, ISO `YYYY-Www`, `YYYY-MM`),
+  newest group first. A `PhotoFilter` combines `albumID`, `text`
+  (title/caption/album-name substring), `fileTypes`, `hasLocation`, and
+  `includeArchived`; hidden photos are always excluded.
+
+An `Album` exposes its record fields and `album.photos.list()` → `Photo[]`
+(newest first). A `Photo` exposes its record fields, `photo.record()` →
+`PhotoRecord`, and two content methods:
+
+- `await photo.original(opts?)` → `{ path, bytes }` — the full-resolution file.
+- `await photo.thumbnail(opts?)` → `{ path, bytes }`.
+
+Both serve from the on-disk content cache when the bytes are present and
+otherwise fetch through the pools; `opts.onProgress` reports per-file progress.
+They throw when the library was opened without a content source.
+
+Lower-level accessors that return decrypted model objects (which hold key
+material) are also available: `listCollections()`, `getCollection(id)`,
+`listFiles(collectionID)`, `getFile(collectionID, fileID)`, and
+`getFileByID(fileID)`.
+
+### Records and change notifications
+
+The GUI-facing records hold no key material and no binary, so they survive
+`structuredClone`/JSON across the Electron IPC boundary:
+
+- `PhotoRecord`: `fileID`, `albumIDs`, `title`, `takenAt` (milliseconds),
+  `fileType`, optional `caption` / `width` / `height` / `latitude` /
+  `longitude`, `isArchived`, `isHidden`, and `thumbnailPath` / `originalPath`
+  once the bytes are cached.
+- `AlbumRecord`: `collectionID`, `name`, `type`, `isShared`, `updationTime`, and
+  `fileIDs` (newest first).
+- `LibrarySnapshot`: `{ albums, photos, takenAt }`.
+
+`lib.snapshot()` returns a `LibrarySnapshot` (albums newest-updated first,
+photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
+(`albumsChanged`, `photosChanged`, `fileIDsRemoved`, `albumIDsRemoved`,
+`refreshedAt`) whenever a refresh alters the projection, and returns
+`{ unsubscribe }`; a refresh that changes nothing delivers nothing.
+
+### Thumbnails, ML search, and backup
+
+- `lib.thumbnails.ensure({ fileIDs, priority, signal?, onProgress? })`
+  prefetches thumbnails through the thumbnail pool, deduped by fileID, returning
+  one `EnsureResult` (`{ fileID, path?, error? }`) per file. `priority` is
+  `"visible" | "ahead" | "background"`; only `"visible"` preempts background
+  work.
+- `lib.mldata` searches the CLIP index built from Ente's per-file ML data:
+  `forFile({ fileID })` → `Promise<MLData | undefined>` (the whole stored
+  payload — face boxes, landmarks, embedding — read from disk on demand);
+  `similar({ fileID, limit? })` and `searchByEmbedding({ embedding, limit? })` →
+  `SimilarResult[]` (`{ fileID, score }`, cosine similarity, most similar first,
+  default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes
+  a query vector the caller produced elsewhere.
+- `await lib.backup(opts?)` → `BackupResult`. It refreshes, fetches every
+  in-scope original (and, with `includeThumbnails`, thumbnails) through the
+  content cache, and rebuilds the on-disk backup tree with a durable failure
+  ledger. `BackupOptions`: `downloadDirectory` (falls back to the one `open()`
+  was given), `includeOriginals` (default `true`), `includeThumbnails` (default
+  `false`), `onlyAlbumNames`, and `onProgress`. See Backup layout above for the
+  tree it writes.
+
+### Request pools
+
+`RequestPools` holds three independent bounded pools — metadata (10), content
+(5), thumbnails (25) — because Ente meters these traffic classes differently.
+Each pool orders on-demand work ahead of background/precache work and dedups
+in-flight fetches by key, and an idle pool never lends its slots to a busy one.
+
+### On-disk cache layout
+
+Under `cacheDirectory`:
+
+```
+<cacheDirectory>/
+    metadata.json                 decrypted account state + refresh cursor
+    originals/<fileID>.<ext>      cached full-resolution files
+    thumbnails/<fileID>.jpg       cached thumbnails
+    mldata/
+        <fileID>.json             one decrypted ML payload per file
+        clip.f32, clip.json       the packed CLIP index and its id list
+        fetched.json              per-file fetch bookkeeping
+```
+
+A stored file appears only via an atomic temp-then-rename, so its presence means
+it is complete. The design also calls for a content-hash comparison against
+`FileMetadata.hash` on each fetched original; that check is deferred (issue
+https://git.eeqj.de/sneak/quak/issues/68) because the exact hash construction
+cannot yet be confirmed against the repo's fixtures.
+
+### Key types by source file
+
+- `src/library/index.ts`: `Library`, `LibraryOptions`, `LibraryStatus`,
+  `LibraryClient`, `RefreshEvent`
+- `src/library/read.ts`: `Album`, `Photo`, `AlbumsAPI`, `PhotosAPI`,
+  `TimelineAPI`, `PhotoFilter`, `TimelineGroup`, `GroupBy`
+- `src/library/content.ts`: `ContentResult`, `ContentOptions`, `ThumbnailsAPI`,
+  `EnsureOptions`, `EnsureResult`, `ContentSource`
+- `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`,
+  `LibraryChange`
+- `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult`
+- `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool`
+- `src/backup.ts`: `BackupOptions`, `BackupResult`, `BackupError`
 - `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot`
-- `src/api/client.ts`: `ApiClient`, `ApiClientOptions`, `ApiError`,
+- `src/api/client.ts`: `ApiClient`, `ApiClientOptions`, `StreamOptions`
-  `StreamOptions`
 - `src/errors.ts`: `ApiError`, `TruncatedStreamError`
 - `src/retry.ts`: `withRetry`, `isRetryable`, `isSafeToReplay`, `RetryOptions`
-- `src/auth/types.ts`: `KeyAttributes`, `SRPAttributes`,
+- `src/model/types.ts`: `Collection`, `EnteFile`, `FileMetadata`, `FileType`,
-  `AuthorizationResponse`, `LoginChallenge`
+  `CollectionType`, `RawCollection`, `RawEnteFile`
-- `src/model/types.ts`: `Collection`, `EnteFile`, `FileMetadata`, `FileBlob`,
-  `RawCollection`, `RawEnteFile`, `RawMagicMetadata`
-- `src/download/index.ts`: `DownloadResult`
-- `src/backup.ts`: `BackupResult`, `BackupError`
 - `src/thumbnails.ts`: `MissingThumbnailInfo`, `ThumbnailFixResult`
 
 ## Source attribution

TODO.md

@@ -14,10 +14,19 @@ pre-1.0
 
 # Next Step
 
-Update the README API reference section to match the current implementation.
+Tag v1.0.0.
 
 # Completed Steps
 
+- 2026-09-22: Rewrote the README API reference (and the Getting Started / usage
+  snippets) to match the shipped cache/API library on `next` (issue 53, issue
+  13). Documented `Library.open` and its options, the default-read vs `fresh()`
+  distinction (and that the CLI's read commands are fresh), the record types and
+  `snapshot()`/`subscribe()`, the `albums`/`photos`/`timeline` read surface,
+  `Photo` content methods, `thumbnails.ensure`, the `mldata` search surface,
+  `backup()`, the three request pools (10/5/25), and the on-disk cache layout;
+  noted the deferred content-hash integrity check (issue 68). Docs-only; no code
+  changed.
 - 2026-09-22: Added resumable, deletion-aware enumeration to `Client` (issue 38,
   closes issue 7). `collectionsSince`/`filesSince` take a starting cursor,
   decrypt live records, surface tombstoned ids in a separate `deleted` list (a
@@ -122,7 +131,6 @@ Update the README API reference section to match the current implementation.
 
 # Future Steps
 
-- Tag v1.0.0.
 - Future desktop client, separate repo:
     - Electron app skeleton consuming this library.
     - Local SQLite cache keyed on (collectionID, fileID, updationTime).