On-disk content and thumbnail cache with per-photo fetch and prefetch (closes #46)

Add src/library/content.ts: a ContentCache keyed by fileID under
cacheDirectory (flat originals/ and thumbnails/, 0700/0600), fetching through
the request pools (#45) and the streaming decrypt / atomic writer (#40) so
present-means-complete. One shared pool set serves both this cache and the
ML-data fetch. Photo.original and thumbnail return {path,bytes}, skipped when
present; lib.thumbnails.ensure drives the thumbnail pool with priority, dedup,
and abort.

Integrity rests on the streaming decrypt (every chunk authenticated, renamed
in only on TAG_FINAL) plus a non-empty check. The stored-hash / size compare
is deferred (tracked in #68): the only in-repo hash fixture is a placeholder,
and the stored size is the encrypted object size, not the decrypted length.

Judgement call: three thumbnail priorities map onto two tiers.

Model: opus-4-8

Dockerfile

@@ -1,36 +1,27 @@
-# Lint stage — fast feedback on formatting and lint issues
-# node 22.22.0 on Alpine 3.23.3 (node:22-alpine), 2026-08-09
-FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS lint
-WORKDIR /app
-COPY script/ script/
-COPY package.json yarn.lock ./
-RUN script/bootstrap
-COPY . .
-RUN make fmt-check
-RUN make lint
-
-# Check stage — the full suite and the build
+# Test and build image: the suite, then the compile.
+#
+# Linting deliberately does not happen here. `script/lint` is a build of
+# Dockerfile.lint, and `script/check` calls `script/lint`, so running
+# `make check` in this image would mean running `docker build` inside a
+# container. Lint runs exactly once, in Dockerfile.lint; script/cibuild
+# builds that first and this second.
 # node 22.22.0 on Alpine 3.23.3 (node:22-alpine), 2026-08-09
 FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS check
 WORKDIR /app
 
-# Force BuildKit to run the lint stage before proceeding. Without this the
-# two stages run in parallel and a lint failure can lose the race.
-COPY --from=lint /app/yarn.lock /dev/null
-
 COPY script/ script/
 COPY package.json yarn.lock ./
 RUN script/bootstrap
 COPY . .
 
-# CHECK_EPOCH is a cache buster: without it Docker serves `make check` from
+# CHECK_EPOCH is a cache buster: without it Docker serves the test layer from
 # cache on an unchanged tree, the suite never executes, and the build still
 # exits 0. The guard makes an absent argument a hard failure — an unset ARG
 # is the empty string, which is a perfectly stable cache key, so a plain
 # `docker build .` would otherwise still get the false green. Fail closed.
 ARG CHECK_EPOCH
 RUN [ -n "$CHECK_EPOCH" ] || exit 1
-RUN make check
+RUN make test
 
 ARG CHECK_EPOCH
 RUN [ -n "$CHECK_EPOCH" ] || exit 1

Dockerfile.lint

@@ -0,0 +1,35 @@
+# Lint image: every lint run happens here, and nowhere else. The repo is
+# COPYed into a digest-pinned image and the linters run as build steps, so a
+# successful build IS a clean lint. `script/lint` does nothing but build this
+# file, which also works where the docker daemon is remote and bind mounts are
+# impossible. Nothing that runs inside a container may call `script/lint`:
+# that is why Dockerfile no longer runs `make check`.
+# node 22.22.0 on Alpine 3.23.3 (node:22-alpine), 2026-08-09
+FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS lint
+WORKDIR /app
+
+# Manifests before sources, so the dependency install layer stays cached
+# until package.json or yarn.lock changes. script/bootstrap ends in
+# `yarn install --frozen-lockfile`; the lint steps below are deliberately
+# not cached.
+COPY script/ script/
+COPY package.json yarn.lock ./
+RUN script/bootstrap
+
+COPY . .
+
+# LINT_EPOCH is a cache buster, with the same fail-closed contract as
+# CHECK_EPOCH in Dockerfile. No lint cache is wanted: on an unchanged tree
+# Docker serves the linter layers in well under a second, having linted
+# nothing, and the build still exits 0. The guard makes an absent argument a
+# hard failure — an unset ARG is the empty string, which is a perfectly
+# stable cache key, so a plain `docker build -f Dockerfile.lint .` would
+# otherwise get exactly that false green. Every layer below this one is a
+# child of the guard, so a fresh epoch forces all of them to execute.
+ARG LINT_EPOCH
+RUN [ -n "$LINT_EPOCH" ] || exit 1
+
+# The linters are invoked directly rather than through `make lint`, because
+# `make lint` is the build of this file.
+RUN yarn run eslint .
+RUN yarn run prettier --check .

README.md

@@ -86,31 +86,70 @@ alpine. We provide:
   files the compiler wrote, and make the CLI executable (our own extension)
 - `script/test` — run the test suite (vitest, hard-capped at 30s where `timeout`
   is available, verbose rerun on failure)
-- `script/lint` — run eslint and a prettier check
+- `script/lint` — run eslint and a prettier check, by building
+  `Dockerfile.lint`; requires docker (see Linting below)
 - `script/fmt` — format all files with prettier (writes)
-- `script/fmt-check` — check formatting (read-only)
-- `script/check` — run all checks: `test`, `lint`, `fmt-check` (our own
-  extension)
-- `script/docker` — build the Docker image, tagged via `script/projectname`
-- `script/cibuild` — cd to the repo root and run the image build (what CI runs;
-  the build runs `make fmt-check` and `make lint` in a first stage, then
-  `make check` and `make build` in a second)
+- `script/fmt-check` — check formatting on the host (read-only); standalone, and
+  not called by `script/check` or `script/precommit`, because `script/lint`
+  already checks formatting in the container (see Linting below)
+- `script/check` — run all checks: `test`, `lint` (our own extension)
+- `script/docker` — build the test and build image, tagged via
+  `script/projectname`
+- `script/cibuild` — cd to the repo root and build both images (what CI runs):
+  `script/lint` first, then the `Dockerfile` image, which runs `make test` and
+  `make build`
 - `script/precommit` — run by the git pre-commit hook (our own extension); runs
-  `script/lint` and `script/fmt-check` but deliberately not the tests, so the
-  TDD red-phase commit can land
+  `script/lint`, which checks both lint and formatting, but deliberately not the
+  tests, so the TDD red-phase commit can land
 - `script/install-precommit` — installs the git pre-commit hook (our own
   extension); `make hooks` shims to it
 
 `make hooks` installs the pre-commit hook that runs `script/precommit`.
 
-Both `script/docker` and `script/cibuild` pass
-`--build-arg CHECK_EPOCH="$(date +%s)"`. The Dockerfile refuses to build without
-it. This is deliberate: on an unchanged tree Docker would otherwise serve the
-`make check` layer from cache, so the suite would never run and the build would
-still exit 0. A changing epoch invalidates the check and build layers on every
-invocation while leaving the dependency layers below them cached, and the
-missing-argument guard means a bare `docker build .` fails loudly instead of
-quietly reporting a green it did not earn.
+### Linting
+
+Linting runs in a container, one way, everywhere. `script/lint` builds
+`Dockerfile.lint`, which copies the repo into a digest-pinned node image and
+runs eslint and prettier as build steps, so a successful build is a clean lint.
+There is no host lint path: docker is required to lint, and that also works
+where the docker daemon is remote and bind mounts are impossible.
+
+The formatting check is part of that, not a step beside it. `script/check` and
+`script/precommit` therefore call `script/lint` and stop; neither calls
+`script/fmt-check` as well, which would run prettier a second time over the same
+tree for the same verdict — and the weaker of the two, since the host's prettier
+is whatever the working tree has installed. So `make check` and the pre-commit
+hook both still fail on a badly formatted tree, and prettier runs exactly once
+in each. `test/packaging/lint-once.test.ts` asserts that count by walking the
+invocation graph, so a second pass cannot creep back in unnoticed.
+
+`script/fmt-check` remains as a standalone entrypoint for asking the formatting
+question on its own, without docker and without the rest of lint. Its verdict
+cannot drift from the container's: prettier is pinned to an exact version,
+installed from `yarn.lock` under `--frozen-lockfile` in both places, and reads
+`.gitignore` as its default ignore file — which is why `.dockerignore`
+deliberately keeps `.gitignore` in the build context.
+
+Lint happens in exactly one place, which constrains the rest of the build.
+`script/check` calls `script/lint`, so `make check` cannot run inside a
+container without asking for docker inside docker. The image built from
+`Dockerfile` therefore runs `make test` and `make build` and does not lint;
+`script/cibuild` builds `Dockerfile.lint` first and that image second, so CI
+gets both verdicts.
+
+### Build epochs
+
+`script/lint` passes `--build-arg LINT_EPOCH="$(date +%s)"`, and `script/docker`
+and `script/cibuild` pass `--build-arg CHECK_EPOCH="$(date +%s)"`. Both
+Dockerfiles refuse to build without their argument. This is deliberate: on an
+unchanged tree Docker would otherwise serve the linter and test layers from
+cache, so nothing would run and the build would still exit 0 — a lint build over
+an untouched tree returns success in well under a second, having linted nothing.
+A changing epoch invalidates every layer below the guard on every invocation
+while leaving the dependency layers above them cached, and the missing-argument
+guard means a bare `docker build .` fails loudly instead of quietly reporting a
+green it did not earn: an unset build argument is the empty string, which is a
+perfectly stable cache key.
 
 ## Rationale
 
@@ -145,8 +184,10 @@ All work on quak is test-driven. No exceptions.
 3. Subsequent commits add the implementation and any refactors needed to make
    the tests pass.
 4. A feature branch can only be merged into `main` when `make check` is green.
-   `main` is always green. The Dockerfile runs `make check` and `make build`, so
-   neither a red branch nor one that does not compile can pass CI.
+   `main` is always green. CI runs `script/cibuild`, which lints via
+   `Dockerfile.lint` and then runs `make test` and `make build` in the
+   `Dockerfile` image, so neither a red branch nor one that does not compile can
+   pass CI.
 5. Tests are the canonical API documentation for this library. Every test file
    is commented thoroughly enough that a reader who has never seen quak can
    learn how to use it from the tests alone. Comments explain why a behavior
@@ -160,11 +201,11 @@ All work on quak is test-driven. No exceptions.
    history must still show tests landing before (or with) the matching
    implementation.
 8. The pre-commit hook installed by `make hooks` runs `script/precommit`, which
-   runs the lint and format checks but not the full `make check`. This is
-   deliberate so the TDD red-phase commit (failing tests, no implementation yet)
-   can land. The full `make check` runs as part of the image build, which is
-   what CI executes via `script/cibuild`, so a red branch still cannot reach
-   `main`.
+   runs `script/lint` — eslint and the prettier check, in the container — but
+   not the tests, and so not the full `make check`. This is deliberate so the
+   TDD red-phase commit (failing tests, no implementation yet) can land. The
+   suite runs as part of the image build, which is what CI executes via
+   `script/cibuild`, so a red branch still cannot reach `main`.
 
 ## Design
 
@@ -191,7 +232,8 @@ quak/
         quak.ts         CLI entrypoint (commander.js)
     test/               unit + integration tests (vitest)
     Makefile
-    Dockerfile
+    Dockerfile          test suite and compile
+    Dockerfile.lint     eslint and prettier, as build steps
     package.json
     tsconfig.json
 ```
@@ -478,9 +520,14 @@ documents:
   implementation. Tests are the canonical API documentation and must be
   commented thoroughly. `main` is always green.
 
-- **Required checks before every commit:** `make lint` (eslint + prettier check)
-  and `make fmt-check` must pass. The pre-commit hook enforces this.
-  `make check` (which also runs tests) must pass before merging to `main`.
+- **Required checks before every commit:** `make lint` must pass — that is
+  eslint plus the prettier check, and it builds `Dockerfile.lint`, so it needs
+  docker. The pre-commit hook enforces exactly that. `make check` (which also
+  runs the tests) must pass before merging to `main`. `make fmt-check` is
+  available for a host-side formatting check on its own, but it is not a
+  separate requirement: `make lint` already covers it, and running both would
+  check formatting twice. Never invoke eslint or prettier directly; linting runs
+  in the container only.
 
 - **Formatting:** prettier with 4-space indents and `proseWrap: always` for
   markdown. Use `make fmt` to format. Use `yarn` not `npm`.

TODO.md

@@ -18,6 +18,62 @@ Update the README API reference section to match the current implementation.
 
 # Completed Steps
 
+- 2026-09-22: Added resumable, deletion-aware enumeration to `Client` (issue 38,
+  closes issue 7). `collectionsSince`/`filesSince` take a starting cursor,
+  decrypt live records, surface tombstoned ids in a separate `deleted` list (a
+  tombstone has nothing to decrypt, so it is a bare id, not a hollow record),
+  and return the max `updationTime` seen as the cursor to resume from.
+  `filesSince` refuses to loop when the diff reports `hasMore` without advancing
+  the cursor (issue 7). `listCollections`/`listFiles` are now thin wrappers that
+  enumerate from `sinceTime: 0` and drop deletions, so existing callers are
+  unaffected.
+- 2026-09-22: Carried file size, thumbnail size, and the deletion flag through
+  `decryptFile` (issue 37, foundation for the cache/API design). Live files now
+  populate `file.size`/`thumbnail.size` from the server's `info` (left
+  `undefined` when the server omits it), and `isDeleted` is carried from the
+  diff row onto `EnteFile`. No caller change: `listFiles` still filters deleted
+  rows before decrypting. Surfacing a tombstone through decryption belongs to
+  the enumeration unit (issue 38).
+- 2026-08-10: Made `lint-once.test.ts` enforce what its header claims. It walked
+  `make check` only, so it never read `Dockerfile` — the image CI builds through
+  `script/cibuild` — and a second `prettier --check .` could be added there with
+  the suite staying green. The walk now also starts at
+  `.gitea/workflows/check.yml` and follows its `run:` steps, so the graph under
+  test is the one CI executes rather than the one someone assumed it executes.
+  The lockfile assertion was a substring check against the whole of
+  `script/bootstrap`, which has two install sites and so reported the branch the
+  containers never take; the two branches are now resolved separately and every
+  `yarn install` in each is required to be `--frozen-lockfile`. Prettier is
+  counted per occurrence instead of per line, so two invocations chained with
+  `&&` no longer read as one, and edges are followed on counted lines instead of
+  being skipped. Every way for the walk to reach nothing — an unknown target, an
+  unknown script, a missing file, a node with no commands, an unknown node kind
+  — is a thrown error rather than a quiet zero. Every assertion in the file was
+  mutation-tested individually.
+- 2026-08-10: Stopped `make check` running `prettier --check .` twice. Since
+  linting moved into Docker, the duplicate was one container pass and one host
+  pass of the same check: `script/lint` builds `Dockerfile.lint`, which runs
+  prettier as a build step, and `script/check` then called `script/fmt-check` as
+  well. The host call is gone from `script/check` and from `script/precommit`;
+  the container keeps checking formatting, because a successful
+  `Dockerfile.lint` build is what CI treats as proof of a clean tree, and it is
+  also what still fails the pre-commit hook on a badly formatted tree.
+  `script/fmt-check` survives as a standalone entrypoint, whose verdict cannot
+  drift from the container's. A test walks the invocation graph from each
+  entrypoint — through the Makefile shims, the `script/` calls and the
+  `docker build` — and asserts the prettier count, so the duplication cannot
+  come back unnoticed.
+- 2026-08-10: Moved all linting into Docker. `script/lint` builds a new root
+  `Dockerfile.lint`, which copies the repo into the digest-pinned node image and
+  runs eslint and prettier as build steps, so a successful build is a clean
+  lint; no host lint path remains and `yarn lint` is gone from `package.json`. A
+  fail-closed `LINT_EPOCH` guard stops Docker serving the linter layers from
+  cache, which is how a lint build returns success in under a second having
+  linted nothing. The lint stage inside `Dockerfile` and its `COPY --from=lint`
+  ordering hack are gone: that image now runs `make test` and `make build` only,
+  because `script/check` calls `script/lint` and running it in a container would
+  mean docker inside docker. `script/cibuild` builds the lint image first, then
+  the test and build image.
 - 2026-08-09: Made `make docker` green and policy-conformant. Multi-stage
   Dockerfile: a lint stage runs `make fmt-check` and `make lint`, and the check
   stage takes a `COPY --from=lint` dependency on it before running `make check`

package.json

@@ -24,7 +24,6 @@
         "build": "script/build",
         "quak": "node ./dist/bin/quak.js",
         "test": "vitest run",
-        "lint": "eslint .",
         "fmt": "prettier --write .",
         "fmt-check": "prettier --check ."
     },

script/check

@@ -1,6 +1,19 @@
 #!/bin/sh
-# script/check: run all checks (test, lint, fmt-check). Our own
-# extension to scripts-to-rule-them-all. Must not modify any files.
+# script/check: run all checks (test, lint). Our own extension to
+# scripts-to-rule-them-all. Must not modify any files.
+#
+# The formatting check is part of lint, not a step of its own:
+# script/lint builds Dockerfile.lint, which runs eslint AND
+# `prettier --check .` as build steps. Calling script/fmt-check here as
+# well would run prettier a second time over the same tree for the same
+# verdict — the weaker of the two, since the host toolchain is whatever
+# the working tree happens to have installed while the container's is
+# digest-pinned. script/fmt-check remains a standalone entrypoint for
+# asking the formatting question by itself.
+#
+# script/lint builds Dockerfile.lint, so this script requires docker and
+# must never be run from inside a container: that is why the Dockerfile
+# image runs script/test and script/build rather than this.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -8,7 +21,6 @@ SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
 main() {
     "$SCRIPT_DIR/test"
     "$SCRIPT_DIR/lint"
-    "$SCRIPT_DIR/fmt-check"
 }
 
 main "$@"

script/cibuild

@@ -1,16 +1,23 @@
 #!/bin/sh
-# script/cibuild: run the CI build. The Dockerfile runs script/check and
-# script/build, and CHECK_EPOCH differs on every invocation, so those two
-# layers cannot be served from Docker's cache: a green build here means
-# the checks ran now, not that a previous run was remembered. The layers
-# below the epoch (bootstrap, yarn install) are unaffected and stay
-# cached. A build that omits the argument fails by design.
+# script/cibuild: run the CI build, which is both images in a defined order.
+#
+# First script/lint, which builds Dockerfile.lint and is the one and only
+# place linting happens — it goes first so a lint failure is reported before
+# the slower suite runs. Then the Dockerfile image, which runs script/test
+# and script/build. CHECK_EPOCH and LINT_EPOCH differ on every invocation, so
+# neither the linters nor the suite can be served from Docker's cache: a
+# green build here means the checks ran now, not that a previous run was
+# remembered. The layers below the epochs (bootstrap, yarn install) are
+# unaffected and stay cached. A build that omits the arguments fails by
+# design.
 set -eu
 
-ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
 
 main() {
     cd "$ROOT"
+    "$SCRIPT_DIR/lint"
     docker build --build-arg CHECK_EPOCH="$(date +%s)" .
 }
 

script/docker

@@ -3,7 +3,8 @@
 # Identical in all repos; the tag comes from script/projectname.
 # CHECK_EPOCH is passed for the same reason script/cibuild passes it: the
 # Dockerfile refuses to build without it, so that no path to an image can
-# quietly serve the check and build layers from cache.
+# quietly serve the test and build layers from cache. This builds the test
+# and build image only; linting is a separate image, built by script/lint.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"

script/lint

@@ -1,13 +1,24 @@
 #!/bin/sh
-# script/lint: run the linter (eslint plus a prettier check).
+# script/lint: run the linters. eslint and prettier are never run against
+# the working tree from here: linting runs via docker only, one way,
+# everywhere — script/lint builds Dockerfile.lint, which COPYs the repo into
+# the pinned node image and runs the linters as build steps. That works even
+# when the docker daemon is remote and bind mounts are impossible.
+#
+# LINT_EPOCH is passed on every invocation because no lint cache is wanted:
+# on an unchanged tree Docker would otherwise serve the linter layers, having
+# linted nothing, and still exit 0. Dockerfile.lint refuses to build without
+# the argument, so no path to a lint result can quietly come from cache.
+#
+# Nothing that runs inside a container may call this script; see the header
+# of Dockerfile.
 set -eu
 
 ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
 
 main() {
     cd "$ROOT"
-    yarn run eslint .
-    yarn run prettier --check .
+    docker build --build-arg LINT_EPOCH="$(date +%s)" -f Dockerfile.lint .
 }
 
 main "$@"

script/precommit

@@ -2,17 +2,25 @@
 # script/precommit: run by the git pre-commit hook; fails the commit if
 # checks fail. Our own extension to scripts-to-rule-them-all.
 #
-# Runs lint and fmt-check but deliberately NOT the tests, so the TDD
-# red-phase commit (failing tests, no implementation yet) can land. CI
-# runs make check via docker build, which catches any branch that
-# ships red.
+# Runs lint but deliberately NOT the tests, so the TDD red-phase commit
+# (failing tests, no implementation yet) can land. CI runs
+# script/cibuild, which builds both images and so catches any branch
+# that ships red.
+#
+# The formatting check is still enforced here, because script/lint is a
+# build of Dockerfile.lint and that runs `prettier --check .` as a build
+# step: a badly formatted tree fails this hook, and therefore the
+# commit. Calling script/fmt-check as well would only run prettier a
+# second time over the same tree for the same verdict.
+#
+# script/lint is a docker build (Dockerfile.lint); docker is required to
+# commit, which is the point of linting one way, everywhere.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
 
 main() {
     "$SCRIPT_DIR/lint"
-    "$SCRIPT_DIR/fmt-check"
 }
 
 main "$@"

src/client.ts

@@ -7,11 +7,16 @@ import {
 } from "./auth/login.js";
 import { unwrapAuth } from "./auth/unwrap.js";
 import { init, fromBase64, toBase64 } from "./crypto/index.js";
+import { fetchMLDataBatch, type MLData } from "./mldata-fetch.js";
 import { decryptCollection, decryptFile } from "./model/index.js";
 import {
     downloadFile as dlFile,
     downloadThumbnail as dlThumb,
 } from "./download/index.js";
+import {
+    makeDownloadContentSource,
+    type ContentSource,
+} from "./library/content.js";
 import type {
     Collection,
     EnteFile,
@@ -37,6 +42,22 @@ export interface ClientSnapshot {
     publicKey: string;
 }
 
+// The result of a resumable enumeration. Live decrypted records and deleted
+// ids are kept apart on purpose: a tombstone carries no key or metadata to
+// decrypt, so it is a bare id rather than a hollowed-out record. `cursor` is
+// the max `updationTime` seen, to pass back into the next call.
+export interface CollectionsPage {
+    collections: Collection[];
+    deleted: number[];
+    cursor: number;
+}
+
+export interface FilesPage {
+    files: EnteFile[];
+    deleted: number[];
+    cursor: number;
+}
+
 export class Client {
     private readonly api: ApiClient;
     private readonly email: string;
@@ -124,6 +145,14 @@ export class Client {
         return this.api;
     }
 
+    // The content-cache byte source over this client's API: each fetch is the
+    // download layer's request + streaming decrypt + atomic write. `Library`
+    // calls this to enable the on-disk content cache.
+    contentSource(): ContentSource {
+        this.assertLoggedIn();
+        return makeDownloadContentSource(this.api);
+    }
+
     private assertLoggedIn(): void {
         if (this.loggedOut) throw new Error("Client has been logged out");
     }
@@ -150,52 +179,123 @@ export class Client {
         this.api.clearAuthToken();
     }
 
-    async listCollections(): Promise<Collection[]> {
+    // Enumerate collections changed since `sinceTime`. Live collections are
+    // decrypted; tombstoned ones (isDeleted) are surfaced as bare ids. The
+    // returned cursor is the max `updationTime` seen — including tombstones, so
+    // the next sync resumes past them — falling back to `sinceTime` when the
+    // response is empty. `/collections/v2` returns the whole changed set in one
+    // response, so there is no pagination here.
+    async collectionsSince(args: {
+        sinceTime: number;
+    }): Promise<CollectionsPage> {
         this.assertLoggedIn();
-        const { collections } = await this.api.getJSON<{
+        const { collections: raws } = await this.api.getJSON<{
             collections: RawCollection[];
-        }>("/collections/v2", { sinceTime: 0 });
-        // The sync API keeps returning deleted collections as tombstones
-        // (isDeleted: true); their diff endpoint 404s, so drop them.
-        return collections
-            .filter((raw) => !raw.isDeleted)
-            .map((raw) =>
-                decryptCollection(
-                    raw,
-                    {
-                        masterKey: this.masterKey,
-                        publicKey: this.publicKey,
-                        secretKey: this.secretKey,
-                    },
-                    this.userID,
-                ),
-            );
+        }>("/collections/v2", { sinceTime: args.sinceTime });
+
+        const collections: Collection[] = [];
+        const deleted: number[] = [];
+        let cursor = args.sinceTime;
+        for (const raw of raws) {
+            if (raw.isDeleted) {
+                deleted.push(raw.id);
+            } else {
+                collections.push(
+                    decryptCollection(
+                        raw,
+                        {
+                            masterKey: this.masterKey,
+                            publicKey: this.publicKey,
+                            secretKey: this.secretKey,
+                        },
+                        this.userID,
+                    ),
+                );
+            }
+            if (raw.updationTime > cursor) cursor = raw.updationTime;
+        }
+        return { collections, deleted, cursor };
     }
 
-    async listFiles(
-        collectionID: number,
-        collectionKey: Uint8Array,
-    ): Promise<EnteFile[]> {
+    // Enumerate a collection's files changed since `sinceTime`, paginating the
+    // diff from that cursor. Live rows are decrypted; tombstoned ones are
+    // surfaced as bare ids. Returns the final cursor to resume from.
+    async filesSince(args: {
+        collectionID: number;
+        collectionKey: Uint8Array;
+        sinceTime: number;
+    }): Promise<FilesPage> {
         this.assertLoggedIn();
-        const allFiles: EnteFile[] = [];
-        let sinceTime = 0;
+        const { collectionID, collectionKey } = args;
+        const files: EnteFile[] = [];
+        const deleted: number[] = [];
+        let cursor = args.sinceTime;
         for (;;) {
             const { diff, hasMore } = await this.api.getJSON<{
                 diff: RawEnteFile[];
                 hasMore: boolean;
-            }>("/collections/v2/diff", { collectionID, sinceTime });
+            }>("/collections/v2/diff", { collectionID, sinceTime: cursor });
 
+            let pageMax = cursor;
             for (const raw of diff) {
-                if (!raw.isDeleted) {
-                    allFiles.push(decryptFile(raw, collectionKey));
-                }
-                if (raw.updationTime > sinceTime) {
-                    sinceTime = raw.updationTime;
+                if (raw.isDeleted) {
+                    deleted.push(raw.id);
+                } else {
+                    files.push(decryptFile(raw, collectionKey));
                 }
+                if (raw.updationTime > pageMax) pageMax = raw.updationTime;
             }
-            if (!hasMore) break;
+
+            if (!hasMore) {
+                cursor = pageMax;
+                break;
+            }
+            // The server says there is more, but this page did not advance the
+            // cursor: following hasMore would refetch the same page forever
+            // (#7). Stop with a clear error instead of looping.
+            if (pageMax <= cursor) {
+                throw new Error(
+                    `/collections/v2/diff for collection ${collectionID} ` +
+                        `returned hasMore with a cursor that did not advance ` +
+                        `(stuck at ${cursor}); refusing to loop`,
+                );
+            }
+            cursor = pageMax;
         }
-        return allFiles;
+        return { files, deleted, cursor };
+    }
+
+    // Whole-account listing: every live collection, deletions hidden. A thin
+    // wrapper over `collectionsSince` from the beginning of time.
+    async listCollections(): Promise<Collection[]> {
+        const { collections } = await this.collectionsSince({ sinceTime: 0 });
+        return collections;
+    }
+
+    // Every live file in a collection, deletions hidden. A thin wrapper over
+    // `filesSince` from the beginning of time.
+    async listFiles(
+        collectionID: number,
+        collectionKey: Uint8Array,
+    ): Promise<EnteFile[]> {
+        const { files } = await this.filesSince({
+            collectionID,
+            collectionKey,
+            sinceTime: 0,
+        });
+        return files;
+    }
+
+    // Fetch machine-learning data (face detections + CLIP embeddings) for up
+    // to a batch of files, each decrypted with its own key. One request; the
+    // library batches at `MLDATA_BATCH_SIZE` and schedules each batch through
+    // its metadata request pool.
+    async fetchMLData(args: {
+        fileIDs: number[];
+        fileKeys: Map<number, Uint8Array>;
+    }): Promise<Map<number, MLData>> {
+        this.assertLoggedIn();
+        return fetchMLDataBatch(this.api, args.fileIDs, args.fileKeys);
     }
 
     async downloadFile(

src/download/index.ts

@@ -1,5 +1,6 @@
 import { randomUUID } from "node:crypto";
-import { rename, rm, writeFile } from "node:fs/promises";
+import { open, rename, rm } from "node:fs/promises";
+import type { FileHandle } from "node:fs/promises";
 import { dirname, join } from "node:path";
 import {
     fromBase64,
@@ -19,42 +20,101 @@ export interface DownloadResult {
     bytesWritten: number;
 }
 
+// Fired as decrypted plaintext accumulates, with the running total of
+// plaintext bytes recovered so far. Within one download it is non-decreasing
+// and its last value equals the final `bytesWritten`. A retry restarts the
+// file from byte zero (see `fetchAndDecrypt`), so a fresh attempt begins its
+// own count from zero.
+export type ProgressCallback = (bytesDone: number) => void;
+
 const ENC_CHUNK_SIZE = STREAM_CHUNK_SIZE + STREAM_CHUNK_OVERHEAD;
 
+// Decrypt a secretstream body, handing each plaintext chunk to `sink` as it is
+// produced rather than accumulating the whole file. Peak memory is one
+// ciphertext chunk of network buffer plus one plaintext chunk — bounded by
+// `STREAM_CHUNK_SIZE` regardless of the file's size — so a multi-gigabyte video
+// no longer needs its size again in RAM. Returns the total plaintext length.
+//
+// The truncation contract is exactly the buffered version's, only the sink is
+// new: a body cut short still decrypts and authenticates up to its last whole
+// chunk, so the absence of TAG_FINAL is the sole evidence it was cut short, and
+// this throws rather than let a caller keep a short file. The sink has already
+// seen those chunks by then; the caller (`decryptToTemp`) stages them in a temp
+// file that is renamed into place only on a clean return, so a throw leaves
+// nothing on disk.
 const streamDecrypt = async (
     stream: ReadableStream<Uint8Array>,
     header: Uint8Array,
     key: Uint8Array,
-): Promise<Uint8Array> => {
+    sink: (plaintext: Uint8Array) => Promise<void>,
+    onProgress?: ProgressCallback,
+): Promise<number> => {
     const state = initStreamPull(header, key);
     const reader = stream.getReader();
-    let buffer = new Uint8Array(0);
-    const plainChunks: Uint8Array[] = [];
+    // Incoming reads are held as-is and only stitched into a contiguous chunk
+    // at each `ENC_CHUNK_SIZE` boundary, so every received byte is copied once.
+    // Concatenating on each read instead — reallocating the whole accumulator
+    // per read — is O(n^2) in the bytes buffered, and for a 4 MiB chunk that
+    // memory churn dwarfs the libsodium decryption itself.
+    const pending: Uint8Array[] = [];
+    let pendingBytes = 0;
     let totalPlain = 0;
     let chunksPulled = 0;
     let lastTag = -1;
 
+    // Remove the first `size` bytes from `pending` as one contiguous buffer.
+    // A read that straddles the boundary is split with `subarray` (a view, no
+    // copy); its tail stays queued for the next chunk. `size` never exceeds
+    // `pendingBytes`, so the queue always holds enough.
+    const takeContiguous = (size: number): Uint8Array => {
+        const out = new Uint8Array(size);
+        let offset = 0;
+        while (offset < size) {
+            const piece = pending[0]!;
+            const need = size - offset;
+            if (piece.length <= need) {
+                out.set(piece, offset);
+                offset += piece.length;
+                pending.shift();
+            } else {
+                out.set(piece.subarray(0, need), offset);
+                pending[0] = piece.subarray(need);
+                offset += need;
+            }
+        }
+        pendingBytes -= size;
+        return out;
+    };
+
+    const consume = async (
+        plaintext: Uint8Array,
+        tag: number,
+    ): Promise<void> => {
+        await sink(plaintext);
+        totalPlain += plaintext.length;
+        chunksPulled++;
+        lastTag = tag;
+        onProgress?.(totalPlain);
+    };
+
     for (;;) {
         const { done, value } = await reader.read();
-        if (value) {
-            const merged = new Uint8Array(buffer.length + value.length);
-            merged.set(buffer);
-            merged.set(value, buffer.length);
-            buffer = merged;
+        if (value && value.length > 0) {
+            pending.push(value);
+            pendingBytes += value.length;
         }
 
-        while (buffer.length >= ENC_CHUNK_SIZE) {
-            const encChunk = buffer.slice(0, ENC_CHUNK_SIZE);
-            buffer = buffer.slice(ENC_CHUNK_SIZE);
+        while (pendingBytes >= ENC_CHUNK_SIZE) {
+            const encChunk = takeContiguous(ENC_CHUNK_SIZE);
+            // A whole chunk that fails to authenticate while the stream carries
+            // on is corruption, not truncation; that error propagates unchanged.
             const { plaintext, tag } = pullStreamChunk(state, encChunk);
-            plainChunks.push(plaintext);
-            totalPlain += plaintext.length;
-            chunksPulled++;
-            lastTag = tag;
+            await consume(plaintext, tag);
         }
 
         if (done) {
-            if (buffer.length > 0) {
+            if (pendingBytes > 0) {
+                const buffer = takeContiguous(pendingBytes);
                 // Whatever is left over once every whole chunk has been
                 // consumed must be the stream's final chunk, and a final
                 // chunk that actually arrived in full authenticates. If it
@@ -62,7 +122,9 @@ const streamDecrypt = async (
                 // ordinary shape of a dropped connection. Poly1305 cannot
                 // tell a partial chunk from a corrupt one, so this is
                 // reported as the truncation it almost always is, with the
-                // authentication failure kept as the error's cause.
+                // authentication failure kept as the error's cause. Only the
+                // pull is guarded: a sink failure on a chunk that did
+                // authenticate is a disk error, not a truncation.
                 let pulled;
                 try {
                     pulled = pullStreamChunk(state, buffer);
@@ -72,10 +134,7 @@ const streamDecrypt = async (
                         { cause: err },
                     );
                 }
-                plainChunks.push(pulled.plaintext);
-                totalPlain += pulled.plaintext.length;
-                chunksPulled++;
-                lastTag = pulled.tag;
+                await consume(pulled.plaintext, pulled.tag);
             }
             break;
         }
@@ -84,8 +143,6 @@ const streamDecrypt = async (
     // Only the last chunk of a secretstream carries TAG_FINAL. Everything a
     // dropped connection did deliver still decrypts and authenticates, so the
     // absence of TAG_FINAL is the only evidence that the body was cut short.
-    // Returning a short plaintext here would put a corrupt file on disk that
-    // later backup runs would treat as complete.
     if (chunksPulled === 0) {
         throw new TruncatedStreamError(
             "download: stream truncated: response body contained no secretstream chunks",
@@ -97,31 +154,54 @@ const streamDecrypt = async (
             `download: stream truncated: last chunk tag ${lastTag}, expected TAG_FINAL (${tagFinal})`,
         );
     }
-
-    const result = new Uint8Array(totalPlain);
-    let offset = 0;
-    for (const chunk of plainChunks) {
-        result.set(chunk, offset);
-        offset += chunk.length;
-    }
-    return result;
+    return totalPlain;
 };
 
-// Write `plaintext` to `destination` atomically: stage it in a temporary
-// sibling file (same directory, so the rename cannot cross a filesystem
-// boundary) and rename it into place. Callers therefore never observe a
-// partially written destination, and a pre-existing file at that path is
+// Stage a write to `destination` atomically and durably, then rename it into
+// place. `fill` writes the contents into the open temp file handle — either the
+// whole buffer at once (`writeAtomic`) or chunk by chunk as they decrypt
+// (`decryptToTemp`). The temp file is a sibling of the destination (same
+// directory, so the rename cannot cross a filesystem boundary), so callers
+// never observe a partially written destination, and a pre-existing file is
 // replaced only once the new contents are complete on disk.
-const writeAtomic = async (
+//
+// Durability against a power cut needs two fsyncs. Without them the write can
+// return while the data or the rename is still only in the kernel's page
+// cache, and a crash then resurrects an empty renamed file — exactly the
+// corruption a later backup run treats as a complete download. So the temp
+// file's contents are fsynced before the rename, and the containing directory
+// is fsynced after it, so both the bytes and the new directory entry are on
+// stable storage before this returns.
+//
+// On any failure — including a `fill` that throws because the stream was
+// truncated — the temp file is removed, so the destination is untouched and no
+// scratch file is left to fill the disk on repeated failures.
+const stageAtomic = async (
     destination: string,
-    plaintext: Uint8Array,
+    fill: (handle: FileHandle) => Promise<void>,
 ): Promise<void> => {
+    const dir = dirname(destination);
     // The random suffix keeps concurrent downloads of the same destination
     // from stepping on each other's temporary file.
-    const tmpPath = join(dirname(destination), `.quak-${randomUUID()}.tmp`);
+    const tmpPath = join(dir, `.quak-${randomUUID()}.tmp`);
     try {
-        await writeFile(tmpPath, plaintext);
+        const handle = await open(tmpPath, "w");
+        try {
+            await fill(handle);
+            await handle.sync();
+        } finally {
+            await handle.close();
+        }
         await rename(tmpPath, destination);
+        // Fsync the directory so the rename itself survives a crash: renaming
+        // over a synced temp file still leaves the new directory entry in the
+        // page cache until the directory is synced.
+        const dirHandle = await open(dir, "r");
+        try {
+            await dirHandle.sync();
+        } finally {
+            await dirHandle.close();
+        }
     } catch (err) {
         // Best-effort cleanup. A failure to remove the temporary file must
         // never replace the error that actually explains what went wrong.
@@ -130,7 +210,44 @@ const writeAtomic = async (
     }
 };
 
-// Fetch a stream and decrypt it, retrying the whole sequence.
+// Write `plaintext` to `destination` atomically and durably. Exported so the
+// metadata store can reuse the same durable write for small whole-buffer
+// payloads; originals go through `decryptToTemp` instead so they never buffer.
+export const writeAtomic = async (
+    destination: string,
+    plaintext: Uint8Array,
+): Promise<void> =>
+    stageAtomic(destination, (handle) => handle.writeFile(plaintext));
+
+// Decrypt `stream` straight to `destination`, one plaintext chunk at a time,
+// under the atomic writer's temp-then-rename discipline. Memory stays bounded
+// by the chunk size: each decrypted chunk is written to the temp file and
+// dropped. The rename happens only after the stream authenticates as terminated
+// on TAG_FINAL; a truncated stream throws and leaves the destination untouched.
+// Returns the plaintext length written.
+const decryptToTemp = async (
+    destination: string,
+    stream: ReadableStream<Uint8Array>,
+    header: Uint8Array,
+    key: Uint8Array,
+    onProgress?: ProgressCallback,
+): Promise<number> => {
+    let bytesWritten = 0;
+    await stageAtomic(destination, async (handle) => {
+        bytesWritten = await streamDecrypt(
+            stream,
+            header,
+            key,
+            async (plaintext) => {
+                await handle.write(plaintext);
+            },
+            onProgress,
+        );
+    });
+    return bytesWritten;
+};
+
+// Fetch a stream and decrypt it to `destination`, retrying the whole sequence.
 //
 // The request is only the first third of a download. `getXStream` returns as
 // soon as headers arrive, and the bytes are pulled here, so a socket reset
@@ -143,53 +260,60 @@ const writeAtomic = async (
 // four attempts would mean sixteen requests for one file. The policy comes
 // from the client so a caller that configured one gets it here too.
 //
-// A retry starts the file over from byte zero: the secretstream pull state is
-// not resumable and there is no Range support on these endpoints.
+// Because the plaintext is streamed to disk rather than buffered, the atomic
+// write is part of the retried unit. A retry starts the file over from byte
+// zero — the secretstream pull state is not resumable and there is no Range
+// support — staging into a fresh temp file each time: a failed attempt writes
+// and then removes its own temp file, and only the attempt that reaches
+// TAG_FINAL renames one into place, so a download that needed three tries still
+// performs exactly one rename over the destination.
 const fetchAndDecrypt = async (
     api: ApiClient,
     openStream: () => Promise<ReadableStream<Uint8Array>>,
     header: Uint8Array,
     key: Uint8Array,
-): Promise<Uint8Array> =>
+    destination: string,
+    onProgress?: ProgressCallback,
+): Promise<number> =>
     withRetry(async () => {
         const stream = await openStream();
-        return streamDecrypt(stream, header, key);
+        return decryptToTemp(destination, stream, header, key, onProgress);
     }, api.getRetryOptions());
 
 export const downloadFile = async (
     api: ApiClient,
     file: EnteFile,
     outPath?: string,
+    onProgress?: ProgressCallback,
 ): Promise<DownloadResult> => {
     const resolvedPath = outPath ?? file.metadata.title;
     const header = fromBase64(file.file.decryptionHeader);
-    const plaintext = await fetchAndDecrypt(
+    const bytesWritten = await fetchAndDecrypt(
         api,
         () => api.getFileStream(file.id, { retry: false }),
         header,
         file.key,
+        resolvedPath,
+        onProgress,
     );
-    // Outside the retry, deliberately: only the attempt that produced a
-    // complete, authenticated plaintext gets to stage a temporary file, so a
-    // download that needed three tries still performs exactly one write and
-    // one rename.
-    await writeAtomic(resolvedPath, plaintext);
-    return { path: resolvedPath, bytesWritten: plaintext.length };
+    return { path: resolvedPath, bytesWritten };
 };
 
 export const downloadThumbnail = async (
     api: ApiClient,
     file: EnteFile,
     outPath?: string,
+    onProgress?: ProgressCallback,
 ): Promise<DownloadResult> => {
     const resolvedPath = outPath ?? `thumb_${file.metadata.title}`;
     const header = fromBase64(file.thumbnail.decryptionHeader);
-    const plaintext = await fetchAndDecrypt(
+    const bytesWritten = await fetchAndDecrypt(
         api,
         () => api.getThumbnailStream(file.id, { retry: false }),
         header,
         file.key,
+        resolvedPath,
+        onProgress,
     );
-    await writeAtomic(resolvedPath, plaintext);
-    return { path: resolvedPath, bytesWritten: plaintext.length };
+    return { path: resolvedPath, bytesWritten };
 };

src/index.ts

@@ -1,6 +1,12 @@
 export const VERSION = "0.0.0";
 
-export { Client, type LoginOptions, type ClientSnapshot } from "./client.js";
+export {
+    Client,
+    type LoginOptions,
+    type ClientSnapshot,
+    type CollectionsPage,
+    type FilesPage,
+} from "./client.js";
 export {
     ApiClient,
     ApiError,
@@ -27,6 +33,49 @@ export {
     requestEmailOTP,
     submitEmailOTP,
 } from "./auth/login.js";
+export {
+    Library,
+    DEFAULT_REFRESH_INTERVAL_SECONDS,
+    Album,
+    Photo,
+    type LibraryClient,
+    type LibraryOptions,
+    type LibraryStatus,
+    type RefreshEvent,
+    type RefreshProgressCallback,
+    type AlbumsAPI,
+    type PhotosAPI,
+    type TimelineAPI,
+    type PhotoFilter,
+    type TimelineGroup,
+    type GroupBy,
+    type ContentSource,
+    type ContentResult,
+    type ContentEvent,
+    type ContentOptions,
+    type PhotoContent,
+    type ThumbnailsAPI,
+    type ThumbnailPriority,
+    type EnsureOptions,
+    type EnsureResult,
+    type EnsureEvent,
+} from "./library/index.js";
+export {
+    RequestPools,
+    BoundedPool,
+    DEFAULT_METADATA_CONCURRENCY,
+    DEFAULT_CONTENT_CONCURRENCY,
+    DEFAULT_THUMBNAIL_CONCURRENCY,
+    type RequestPoolsOptions,
+    type Priority,
+    type RunOptions,
+} from "./library/pools.js";
+export type {
+    AlbumRecord,
+    PhotoRecord,
+    LibrarySnapshot,
+    LibraryChange,
+} from "./library/records.js";
 export { decryptCollection, decryptFile } from "./model/index.js";
 export { downloadFile, downloadThumbnail } from "./download/index.js";
 export type {

src/library/content.ts

@@ -0,0 +1,416 @@
+// The on-disk content and thumbnail cache keyed by fileID (issue #46).
+//
+// Layout under `cacheDirectory`: `originals/<fileID>.<ext>` and
+// `thumbnails/<fileID>.<ext>`, flat directories at 0700 with files at 0600.
+// Content appears only by the streaming atomic writer's rename (the download
+// layer, #40), so a file that exists is whole — "present means complete". The
+// directory listing taken at `open()` is the record of what is cached, and the
+// orphan temp files a crashed write may have left are reaped there.
+//
+// A fetch goes through the shared request pools (#45): the content pool for
+// originals, the thumbnail pool for thumbnails. The pool limits concurrency,
+// orders on-demand work ahead of background, and dedups by key so a fileID
+// requested twice while the first is still in flight downloads once.
+//
+// Integrity. The reused streaming decrypt is the enforced guarantee: every
+// chunk is authenticated and the writer renames the file into place only once
+// the stream ends on TAG_FINAL, so a truncated or corrupt fetch throws and
+// nothing is stored. On top of that this module refuses to record a stored file
+// that came out empty. The design also asks for a content-hash comparison
+// against `FileMetadata.hash` (with a `fileSize` fallback); that is deferred —
+// see the PR — because the exact hash construction cannot be confirmed against
+// the repo's fixtures and `FileBlob.size` is the encrypted object size, not the
+// decrypted length this layer has.
+
+import { existsSync, statSync } from "node:fs";
+import { chmod, mkdir, readdir, rm, stat } from "node:fs/promises";
+import { extname, join } from "node:path";
+
+import type { ApiClient } from "../api/client.js";
+import {
+    downloadFile,
+    downloadThumbnail,
+    type ProgressCallback,
+} from "../download/index.js";
+import type { EnteFile } from "../model/types.js";
+import type { Priority, RequestPools } from "./pools.js";
+
+const DIR_MODE = 0o700;
+const FILE_MODE = 0o600;
+const TEMP_PREFIX = ".quak-";
+const TEMP_SUFFIX = ".tmp";
+// Ente thumbnails are always JPEG, so the cache stores them with a fixed
+// extension rather than deriving one from the (image or video) title.
+const THUMBNAIL_EXT = ".jpg";
+
+type Kind = "original" | "thumbnail";
+
+// The priority a caller attaches to a thumbnail prefetch. The pool has two
+// tiers, so this three-value surface collapses onto them: only a currently
+// visible thumbnail preempts (on-demand); "ahead" prefetch and speculative
+// "background" work both yield to it.
+export type ThumbnailPriority = "visible" | "ahead" | "background";
+
+const poolPriorityOf = (priority: ThumbnailPriority): Priority =>
+    priority === "visible" ? "on-demand" : "background";
+
+export interface ContentResult {
+    path: string;
+    bytes: number;
+}
+
+// Progress for a single `original`/`thumbnail` call. A present file emits one
+// `skipped` event and nothing else; a fetched file emits `downloading` as
+// plaintext lands and a final `done`.
+export type ContentEvent =
+    | { status: "skipped"; bytes: number }
+    | { status: "downloading"; bytesDone: number }
+    | { status: "done"; bytes: number };
+
+export interface ContentOptions {
+    onProgress?: (event: ContentEvent) => void;
+}
+
+// The Photo-facing content surface (the read wrappers call these). The cache
+// implements it; a library opened without a content source leaves it absent.
+export interface PhotoContent {
+    original(fileID: number, opts?: ContentOptions): Promise<ContentResult>;
+    thumbnail(fileID: number, opts?: ContentOptions): Promise<ContentResult>;
+}
+
+export interface EnsureResult {
+    fileID: number;
+    path?: string;
+    error?: string;
+}
+
+export interface EnsureEvent {
+    fileID: number;
+    status: "skipped" | "done" | "failed" | "aborted";
+    path?: string;
+    error?: string;
+}
+
+export interface EnsureOptions {
+    fileIDs: number[];
+    priority: ThumbnailPriority;
+    signal?: AbortSignal;
+    onProgress?: (event: EnsureEvent) => void;
+}
+
+export interface ThumbnailsAPI {
+    ensure(args: EnsureOptions): Promise<EnsureResult[]>;
+}
+
+// The byte source the cache fetches through. The real implementation streams
+// and decrypts to the destination via the download layer; tests inject a
+// stand-in so the cache logic runs with no crypto and no network. Pool routing,
+// dedup, present-checks and integrity live in the cache, not here.
+export interface ContentSource {
+    original(args: {
+        file: EnteFile;
+        destination: string;
+        onProgress?: ProgressCallback;
+    }): Promise<{ bytesWritten: number }>;
+    thumbnail(args: {
+        file: EnteFile;
+        destination: string;
+        onProgress?: ProgressCallback;
+    }): Promise<{ bytesWritten: number }>;
+}
+
+// The production source: each fetch is the download layer's request +
+// streaming decrypt + atomic write + retry as one unit.
+export const makeDownloadContentSource = (api: ApiClient): ContentSource => ({
+    original: ({ file, destination, onProgress }) =>
+        downloadFile(api, file, destination, onProgress),
+    thumbnail: ({ file, destination, onProgress }) =>
+        downloadThumbnail(api, file, destination, onProgress),
+});
+
+export interface CachedPaths {
+    originalPath?: string;
+    thumbnailPath?: string;
+}
+
+export interface ContentCacheOptions {
+    pools: RequestPools;
+    source: ContentSource;
+    cacheDirectory: string;
+    // The backup destination (issue-level `downloadDirectory`). An original
+    // already stored there by a backup counts as present, so the cache serves
+    // it rather than fetching a second copy.
+    downloadDirectory?: string;
+    // Resolve any membership of a file; every membership shares the underlying
+    // content key, so any one decrypts the same bytes.
+    getFile: (fileID: number) => EnteFile | undefined;
+}
+
+// Thrown inside a pooled task to drop a queued fetch that was aborted before it
+// started running. Never escapes `ensureThumbnails`.
+class AbortDrop extends Error {
+    constructor() {
+        super("aborted");
+        this.name = "AbortDrop";
+    }
+}
+
+const originalName = (file: EnteFile): string => {
+    const ext = extname(file.metadata.title || "") || ".bin";
+    return `${file.id}${ext}`;
+};
+
+// The fileID a cache filename encodes, or undefined when the name is not one
+// the cache writes (`<digits><ext>`).
+const fileIDFromName = (name: string): number | undefined => {
+    const base = name.slice(0, name.length - extname(name).length);
+    if (!/^\d+$/.test(base)) return undefined;
+    const id = Number(base);
+    return Number.isSafeInteger(id) ? id : undefined;
+};
+
+// Size of a regular file, or undefined if it is absent (or not a regular file).
+const fileSize = (path: string): number | undefined => {
+    try {
+        const s = statSync(path);
+        return s.isFile() ? s.size : undefined;
+    } catch {
+        return undefined;
+    }
+};
+
+export class ContentCache implements PhotoContent, ThumbnailsAPI {
+    private readonly pools: RequestPools;
+    private readonly source: ContentSource;
+    private readonly downloadDirectory?: string;
+    private readonly getFile: (fileID: number) => EnteFile | undefined;
+    private readonly originalsDir: string;
+    private readonly thumbnailsDir: string;
+    // fileID -> absolute path of the cached bytes, seeded from the directory
+    // listing at open() and extended as fetches store new files.
+    private readonly originals = new Map<number, string>();
+    private readonly thumbnails = new Map<number, string>();
+
+    constructor(opts: ContentCacheOptions) {
+        this.pools = opts.pools;
+        this.source = opts.source;
+        this.downloadDirectory = opts.downloadDirectory;
+        this.getFile = opts.getFile;
+        this.originalsDir = join(opts.cacheDirectory, "originals");
+        this.thumbnailsDir = join(opts.cacheDirectory, "thumbnails");
+    }
+
+    // Prepare the cache directories, reap orphan temp files, and take the
+    // record of what is already cached. Called once before the cache serves.
+    async open(): Promise<void> {
+        await this.ensureDir(this.originalsDir);
+        await this.ensureDir(this.thumbnailsDir);
+        await this.scan(this.originalsDir, this.originals);
+        await this.scan(this.thumbnailsDir, this.thumbnails);
+    }
+
+    // The cache paths known for a file, for the record projection to expose as
+    // `originalPath`/`thumbnailPath`.
+    pathsFor(fileID: number): CachedPaths {
+        const out: CachedPaths = {};
+        const original = this.originals.get(fileID);
+        if (original !== undefined) out.originalPath = original;
+        const thumbnail = this.thumbnails.get(fileID);
+        if (thumbnail !== undefined) out.thumbnailPath = thumbnail;
+        return out;
+    }
+
+    async original(
+        fileID: number,
+        opts?: ContentOptions,
+    ): Promise<ContentResult> {
+        return this.get(fileID, "original", "on-demand", opts?.onProgress);
+    }
+
+    async thumbnail(
+        fileID: number,
+        opts?: ContentOptions,
+    ): Promise<ContentResult> {
+        return this.get(fileID, "thumbnail", "on-demand", opts?.onProgress);
+    }
+
+    async ensure(args: EnsureOptions): Promise<EnsureResult[]> {
+        return this.ensureThumbnails(args);
+    }
+
+    async ensureThumbnails(args: EnsureOptions): Promise<EnsureResult[]> {
+        const priority = poolPriorityOf(args.priority);
+        // Dedup the request list so a repeated fileID is fetched once and
+        // reported once, in first-requested order.
+        const seen = new Set<number>();
+        const unique: number[] = [];
+        for (const id of args.fileIDs) {
+            if (!seen.has(id)) {
+                seen.add(id);
+                unique.push(id);
+            }
+        }
+        return Promise.all(
+            unique.map((fileID) =>
+                this.ensureOne(fileID, priority, args.signal, args.onProgress),
+            ),
+        );
+    }
+
+    private async ensureOne(
+        fileID: number,
+        priority: Priority,
+        signal: AbortSignal | undefined,
+        onProgress: ((event: EnsureEvent) => void) | undefined,
+    ): Promise<EnsureResult> {
+        try {
+            const result = await this.acquire(
+                fileID,
+                "thumbnail",
+                priority,
+                signal,
+            );
+            const status = result.cached ? "skipped" : "done";
+            onProgress?.({ fileID, status, path: result.path });
+            return { fileID, path: result.path };
+        } catch (err) {
+            if (err instanceof AbortDrop) {
+                onProgress?.({ fileID, status: "aborted" });
+                return { fileID, error: "aborted" };
+            }
+            const error = err instanceof Error ? err.message : String(err);
+            onProgress?.({ fileID, status: "failed", error });
+            return { fileID, error };
+        }
+    }
+
+    private async get(
+        fileID: number,
+        kind: Kind,
+        priority: Priority,
+        onProgress: ((event: ContentEvent) => void) | undefined,
+    ): Promise<ContentResult> {
+        const onByte: ProgressCallback | undefined = onProgress
+            ? (bytesDone) => onProgress({ status: "downloading", bytesDone })
+            : undefined;
+        const result = await this.acquire(fileID, kind, priority, undefined, {
+            onByte,
+        });
+        onProgress?.(
+            result.cached
+                ? { status: "skipped", bytes: result.bytes }
+                : { status: "done", bytes: result.bytes },
+        );
+        return { path: result.path, bytes: result.bytes };
+    }
+
+    // The core: return the cached path if present, else fetch through the pool,
+    // store, and return it. `cached` distinguishes a present hit (no network,
+    // no download event) from a fresh fetch.
+    private async acquire(
+        fileID: number,
+        kind: Kind,
+        priority: Priority,
+        signal: AbortSignal | undefined,
+        opts?: { onByte?: ProgressCallback },
+    ): Promise<{ path: string; bytes: number; cached: boolean }> {
+        const file = this.getFile(fileID);
+        if (!file) throw new Error(`content cache: unknown file ${fileID}`);
+
+        const known = kind === "original" ? this.originals : this.thumbnails;
+        const cached = known.get(fileID);
+        if (cached !== undefined) {
+            const size = fileSize(cached);
+            if (size !== undefined && size > 0)
+                return { path: cached, bytes: size, cached: true };
+            // A recorded file that has since gone re-fetches below.
+            known.delete(fileID);
+        }
+
+        // An original a backup already stored counts as present.
+        if (kind === "original" && this.downloadDirectory !== undefined) {
+            const backupPath = join(
+                this.downloadDirectory,
+                "originals",
+                originalName(file),
+            );
+            const size = fileSize(backupPath);
+            if (size !== undefined && size > 0) {
+                this.originals.set(fileID, backupPath);
+                return { path: backupPath, bytes: size, cached: true };
+            }
+        }
+
+        const dir =
+            kind === "original" ? this.originalsDir : this.thumbnailsDir;
+        const dest =
+            kind === "original"
+                ? join(dir, originalName(file))
+                : join(dir, `${fileID}${THUMBNAIL_EXT}`);
+        const pool =
+            kind === "original" ? this.pools.content : this.pools.thumbnails;
+
+        return pool.run(
+            async () => {
+                // Dropping queued work on abort: a task still waiting for a slot
+                // when the signal fired sees it here and never touches the
+                // network. A task already past this point is in flight and runs
+                // to completion.
+                if (signal?.aborted) throw new AbortDrop();
+
+                await this.download(file, dest, kind, opts?.onByte);
+                await chmod(dest, FILE_MODE);
+                const size = (await stat(dest)).size;
+                if (size === 0) {
+                    throw new Error(
+                        `content cache: ${kind} ${fileID} stored empty`,
+                    );
+                }
+                known.set(fileID, dest);
+                return { path: dest, bytes: size, cached: false };
+            },
+            { priority, key: fileID },
+        );
+    }
+
+    private async download(
+        file: EnteFile,
+        destination: string,
+        kind: Kind,
+        onProgress: ProgressCallback | undefined,
+    ): Promise<number> {
+        const args = { file, destination, onProgress };
+        const result =
+            kind === "original"
+                ? await this.source.original(args)
+                : await this.source.thumbnail(args);
+        return result.bytesWritten;
+    }
+
+    private async ensureDir(dir: string): Promise<void> {
+        // chmod after mkdir so the mode is tightened even when the directory
+        // already existed with a looser one; mkdir alone would not.
+        await mkdir(dir, { recursive: true, mode: DIR_MODE });
+        await chmod(dir, DIR_MODE);
+    }
+
+    private async scan(dir: string, into: Map<number, string>): Promise<void> {
+        let entries: string[];
+        try {
+            entries = await readdir(dir);
+        } catch {
+            return;
+        }
+        for (const name of entries) {
+            if (name.startsWith(TEMP_PREFIX) && name.endsWith(TEMP_SUFFIX)) {
+                await rm(join(dir, name), { force: true }).catch(
+                    () => undefined,
+                );
+                continue;
+            }
+            const id = fileIDFromName(name);
+            const path = join(dir, name);
+            if (id !== undefined && existsSync(path)) into.set(id, path);
+        }
+    }
+}

src/library/index.ts

@@ -0,0 +1,629 @@
+// The library surface over the local cache.
+//
+// `Library.open()` loads the on-disk metadata store (issue #41), then starts
+// the refresh loop. When the cache loaded empty it awaits the first refresh,
+// so the library never opens onto an empty store it could have filled; when an
+// existing copy loaded, that first refresh runs in the background and `open()`
+// returns as soon as the cached data is ready to serve — a slow or unreachable
+// server no longer stalls opening. A background timer then refreshes every
+// `refreshIntervalSeconds`. Every read is answered from RAM — no read touches
+// the network. There is deliberately no `sync()`, no `refresh()`, no
+// `serverReachable` flag, and no "before each read" mode (design #36): the
+// only ways state changes are the refreshes above.
+//
+// A refresh stages all of its network work first and only mutates the store
+// once every fetch has succeeded. A refresh that fails partway therefore never
+// becomes visible to reads: the last good snapshot stays in place, and the
+// failure surfaces through `onProgress` and `status()` instead. A commit that
+// mutates RAM but then fails to persist keeps `status().lastError` set and the
+// store marked unsaved until a later save actually lands, so a stuck disk is
+// never masked by a subsequent empty refresh.
+
+import { join } from "node:path";
+import envPaths from "env-paths";
+
+import { MetadataStore } from "./store.js";
+import { MLDataStore } from "./mldata.js";
+import { RequestPools } from "./pools.js";
+import {
+    deriveRecords,
+    snapshotFrom,
+    diffRecords,
+    type DerivedRecords,
+    type LibrarySnapshot,
+    type LibraryChange,
+} from "./records.js";
+import {
+    makeAlbumsAPI,
+    makePhotosAPI,
+    makeTimelineAPI,
+    type AlbumsAPI,
+    type PhotosAPI,
+    type TimelineAPI,
+} from "./read.js";
+import {
+    ContentCache,
+    type ContentSource,
+    type ThumbnailsAPI,
+    type EnsureOptions,
+    type EnsureResult,
+} from "./content.js";
+
+export {
+    Album,
+    Photo,
+    type AlbumsAPI,
+    type PhotosAPI,
+    type TimelineAPI,
+    type PhotoFilter,
+    type TimelineGroup,
+    type GroupBy,
+} from "./read.js";
+export {
+    type ContentSource,
+    type ContentResult,
+    type ContentEvent,
+    type ContentOptions,
+    type PhotoContent,
+    type ThumbnailsAPI,
+    type ThumbnailPriority,
+    type EnsureOptions,
+    type EnsureResult,
+    type EnsureEvent,
+} from "./content.js";
+import type { CollectionsPage, FilesPage } from "../client.js";
+import { MLDATA_BATCH_SIZE, type MLData } from "../mldata-fetch.js";
+import type { Collection, EnteFile } from "../model/types.js";
+
+export const DEFAULT_REFRESH_INTERVAL_SECONDS = 3;
+
+// The slice of `Client` the library depends on. Narrowing to an interface lets
+// tests drive a mock with no crypto or network; the real `Client` satisfies it
+// structurally.
+export interface LibraryClient {
+    whoami(): { email: string; userID: number };
+    collectionsSince(args: { sinceTime: number }): Promise<CollectionsPage>;
+    filesSince(args: {
+        collectionID: number;
+        collectionKey: Uint8Array;
+        sinceTime: number;
+    }): Promise<FilesPage>;
+    // Fetch ML data (face detections + CLIP embeddings) for up to a batch of
+    // files. Optional: a client without it simply disables ML fetching, leaving
+    // the metadata refresh untouched.
+    fetchMLData?(args: {
+        fileIDs: number[];
+        fileKeys: Map<number, Uint8Array>;
+    }): Promise<Map<number, MLData>>;
+    // The byte source for the on-disk content cache. Optional so a mock client
+    // that only serves metadata still satisfies the interface; when absent (and
+    // no explicit `contentSource` is passed to `open`) the content cache is
+    // disabled and `Photo.original`/`thumbnail` and `thumbnails.ensure` throw.
+    contentSource?(): ContentSource;
+}
+
+// A progress event for one unit of background work. A metadata "refresh" or an
+// ML "fetchMLData" pass each fire "started" before their network work and then
+// exactly one of "done" or "failed"; "failed" carries the error message and
+// an ML "done" reports how many payloads it stored.
+export interface RefreshEvent {
+    operation: "refresh" | "fetchMLData";
+    status: "started" | "done" | "failed";
+    error?: string;
+    fetched?: number;
+}
+
+export type RefreshProgressCallback = (event: RefreshEvent) => void;
+
+export interface LibraryOptions {
+    client: LibraryClient;
+    // Where `metadata.json` lives. Defaults to the env-paths cache directory
+    // plus the user id, so each account has its own cache.
+    cacheDirectory?: string;
+    // Persistent backup destination. The refresh loop does not use it; the
+    // content cache treats an original already stored there as present.
+    downloadDirectory?: string;
+    refreshIntervalSeconds?: number;
+    onProgress?: RefreshProgressCallback;
+    // The bounded request pools (issue #45), shared by the ML-data fetch (the
+    // metadata pool) and the content cache. Defaults to a fresh set at the
+    // design's caps.
+    pools?: RequestPools;
+    // Overrides the client's own `contentSource()`; mainly for tests that drive
+    // the cache with a stand-in source.
+    contentSource?: ContentSource;
+}
+
+export interface LibraryStatus {
+    userID: number;
+    collections: number;
+    files: number;
+    // Wall-clock ms of the last refresh that succeeded, or undefined if none
+    // has yet.
+    lastRefreshAt?: number;
+    // The message from the most recent refresh, set only while that refresh
+    // failed; cleared by the next success.
+    lastError?: string;
+    // Wall-clock ms of the last ML fetch pass that succeeded, or undefined if
+    // none has yet (or ML fetching is disabled).
+    lastMLFetchAt?: number;
+    // The most recent ML fetch pass's error, set only while it failed.
+    lastMLError?: string;
+    // ML payloads stored on disk and CLIP embeddings in the index; undefined
+    // when ML fetching is disabled.
+    mlStored?: number;
+    mlIndexed?: number;
+    closed: boolean;
+}
+
+export class Library {
+    readonly cacheDirectory: string;
+    readonly downloadDirectory?: string;
+
+    // The in-process read surface (issue #44). Each namespace answers
+    // synchronously from the live record projection; no read touches the
+    // network.
+    readonly albums: AlbumsAPI;
+    readonly photos: PhotosAPI;
+    readonly timeline: TimelineAPI;
+    // The thumbnail-prefetch surface (issue #46): drives the thumbnail pool
+    // with priority, dedup, and abort.
+    readonly thumbnails: ThumbnailsAPI;
+
+    private readonly client: LibraryClient;
+    private readonly store: MetadataStore;
+    // The on-disk content cache, or undefined when no content source is
+    // available (a metadata-only client with no explicit source).
+    private readonly cache?: ContentCache;
+    private readonly userID: number;
+    private readonly intervalMs: number;
+    private readonly onProgress?: RefreshProgressCallback;
+    private readonly pools: RequestPools;
+    // The ML-data cache, present only when the client can fetch ML data.
+    private readonly mldata?: MLDataStore;
+
+    private timer?: ReturnType<typeof setTimeout>;
+    private refreshing = false;
+    // Guards the ML fetch pass so a slow backfill never runs twice at once; a
+    // refresh whose pass is still running kicks nothing new.
+    private mlFetching = false;
+    private closed = false;
+    private lastRefreshAt?: number;
+    private lastError?: string;
+    private lastMLFetchAt?: number;
+    private lastMLError?: string;
+    // The plain-record projection as of the last refresh, and the GUI change
+    // subscribers. A refresh that alters the projection notifies each with the
+    // delta; `lastRecords` is kept current every refresh so a subscriber that
+    // joins later diffs against the state its own `snapshot()` already returned.
+    private readonly subscribers = new Set<(change: LibraryChange) => void>();
+    private lastRecords: DerivedRecords;
+    // RAM holds changes disk has not yet accepted (an earlier save failed).
+    // Cleared only when a save actually succeeds; keeps the store trying to
+    // persist and the failure visible in `status()` until then.
+    private unsaved = false;
+
+    private constructor(args: {
+        client: LibraryClient;
+        store: MetadataStore;
+        userID: number;
+        cacheDirectory: string;
+        downloadDirectory?: string;
+        intervalMs: number;
+        onProgress?: RefreshProgressCallback;
+        pools: RequestPools;
+        mldata?: MLDataStore;
+        cache?: ContentCache;
+    }) {
+        this.client = args.client;
+        this.store = args.store;
+        this.userID = args.userID;
+        this.cacheDirectory = args.cacheDirectory;
+        this.downloadDirectory = args.downloadDirectory;
+        this.intervalMs = args.intervalMs;
+        this.onProgress = args.onProgress;
+        this.pools = args.pools;
+        this.mldata = args.mldata;
+        this.cache = args.cache;
+        this.lastRecords = this.deriveNow();
+
+        // The read namespaces derive fresh from the store on each call, so they
+        // always reflect the latest refresh.
+        const derive = (): DerivedRecords => this.deriveNow();
+        this.albums = makeAlbumsAPI(derive, this.cache);
+        this.photos = makePhotosAPI(derive, this.cache);
+        this.timeline = makeTimelineAPI(derive);
+        this.thumbnails = {
+            ensure: (opts: EnsureOptions): Promise<EnsureResult[]> => {
+                if (!this.cache) {
+                    return Promise.reject(
+                        new Error(
+                            "thumbnails.ensure requires a library opened with a content cache",
+                        ),
+                    );
+                }
+                return this.cache.ensureThumbnails(opts);
+            },
+        };
+    }
+
+    // Load the cache and start the refresh loop. With an empty cache the first
+    // refresh is awaited, so `open()` resolves onto populated data whenever the
+    // server is reachable; that awaited refresh may still fail, and the library
+    // then opens empty with the failure recorded in `status()`. With an
+    // existing cache the first refresh runs in the background and `open()`
+    // returns as soon as the cached data is ready — an unreachable server does
+    // not block opening.
+    static async open(opts: LibraryOptions): Promise<Library> {
+        const { userID } = opts.client.whoami();
+        const cacheDirectory =
+            opts.cacheDirectory ??
+            join(envPaths("quak", { suffix: "" }).cache, String(userID));
+        const store = await MetadataStore.load(
+            join(cacheDirectory, "metadata.json"),
+        );
+        const intervalMs =
+            (opts.refreshIntervalSeconds ?? DEFAULT_REFRESH_INTERVAL_SECONDS) *
+            1000;
+
+        // One request-pool set serves both the ML-data fetch and the content
+        // cache, so both honour the same concurrency caps.
+        const pools = opts.pools ?? new RequestPools();
+
+        // The ML cache only earns its keep when the client can fetch ML data;
+        // a client without that capability opens no `mldata/` directory.
+        const mldata = opts.client.fetchMLData
+            ? await MLDataStore.open(join(cacheDirectory, "mldata"))
+            : undefined;
+
+        // Build the content cache from an explicit source or the client's own,
+        // and take its record of what is already cached (and reap orphan temp
+        // files) before the first projection, so cached paths are present from
+        // the start and the first refresh raises no spurious path-change diff.
+        const source = opts.contentSource ?? opts.client.contentSource?.();
+        let cache: ContentCache | undefined;
+        if (source) {
+            cache = new ContentCache({
+                pools,
+                source,
+                cacheDirectory,
+                downloadDirectory: opts.downloadDirectory,
+                getFile: (fileID) => store.getFileByID(fileID),
+            });
+            await cache.open();
+        }
+
+        const lib = new Library({
+            client: opts.client,
+            store,
+            userID,
+            cacheDirectory,
+            downloadDirectory: opts.downloadDirectory,
+            intervalMs,
+            onProgress: opts.onProgress,
+            pools,
+            mldata,
+            cache,
+        });
+
+        if (store.loadedFromDisk) {
+            // An existing copy already answers reads; refresh in the background
+            // and start the interval once that first cycle settles.
+            void lib.runRefresh().then(() => lib.scheduleNext());
+        } else {
+            // Nothing was cached: wait for the first refresh to fill the store
+            // (or fail) rather than resolve onto an empty library.
+            await lib.runRefresh();
+            lib.scheduleNext();
+        }
+        return lib;
+    }
+
+    listCollections(): Collection[] {
+        return this.store.listCollections();
+    }
+
+    getCollection(id: number): Collection | undefined {
+        return this.store.getCollection(id);
+    }
+
+    listFiles(collectionID: number): EnteFile[] {
+        return this.store.listFiles(collectionID);
+    }
+
+    getFile(collectionID: number, fileID: number): EnteFile | undefined {
+        return this.store.getFile(collectionID, fileID);
+    }
+
+    // A synchronous, RAM-only projection of the whole library into plain
+    // records (no keys), the surface the GUI reads across IPC. Photos are
+    // deduplicated to one record per file and ordered newest first.
+    snapshot(): LibrarySnapshot {
+        return snapshotFrom(this.deriveNow(), Date.now());
+    }
+
+    // Deliver a `LibraryChange` whenever a refresh alters the projection. A
+    // refresh that changes nothing delivers nothing. The returned handle's
+    // `unsubscribe` stops delivery.
+    subscribe(args: { onChange: (change: LibraryChange) => void }): {
+        unsubscribe: () => void;
+    } {
+        const { onChange } = args;
+        this.subscribers.add(onChange);
+        return {
+            unsubscribe: () => {
+                this.subscribers.delete(onChange);
+            },
+        };
+    }
+
+    status(): LibraryStatus {
+        let files = 0;
+        const collections = this.store.listCollections();
+        for (const c of collections) {
+            files += this.store.listFiles(c.id).length;
+        }
+        const ml = this.mldata?.stats();
+        return {
+            userID: this.store.userID,
+            collections: collections.length,
+            files,
+            lastRefreshAt: this.lastRefreshAt,
+            lastError: this.lastError,
+            lastMLFetchAt: this.lastMLFetchAt,
+            lastMLError: this.lastMLError,
+            mlStored: ml?.stored,
+            mlIndexed: ml?.indexed,
+            closed: this.closed,
+        };
+    }
+
+    // Stop the background timer. Idempotent. An in-flight refresh is left to
+    // finish; it will not schedule another cycle once closed.
+    close(): void {
+        this.closed = true;
+        if (this.timer !== undefined) {
+            clearTimeout(this.timer);
+            this.timer = undefined;
+        }
+    }
+
+    private scheduleNext(): void {
+        if (this.closed) return;
+        this.timer = setTimeout(() => {
+            void this.runRefresh().then(() => this.scheduleNext());
+        }, this.intervalMs);
+        // Do not keep the process alive for the sake of the timer.
+        this.timer.unref?.();
+    }
+
+    // One refresh cycle, guarded so a failure never escapes and overlapping
+    // cycles never run. Errors are reported, not thrown.
+    private async runRefresh(): Promise<void> {
+        if (this.closed || this.refreshing) return;
+        this.refreshing = true;
+        this.emit({ operation: "refresh", status: "started" });
+        try {
+            await this.refreshOnce();
+            this.lastRefreshAt = Date.now();
+            this.lastError = undefined;
+            this.emit({ operation: "refresh", status: "done" });
+            // Backfill ML data for the files this refresh knows about. It runs
+            // outside the refresh's success/failure so a fetch or disk problem
+            // there never marks the metadata refresh failed, and it is not
+            // awaited so it never stalls the refresh interval.
+            void this.runMLFetch();
+        } catch (err) {
+            const error = err instanceof Error ? err.message : String(err);
+            this.lastError = error;
+            this.emit({ operation: "refresh", status: "failed", error });
+        } finally {
+            this.refreshing = false;
+        }
+    }
+
+    // Fetch every change since the stored cursor, then commit. All network
+    // reads happen before any store mutation, so a fetch that throws leaves the
+    // store untouched and the previous snapshot intact.
+    private async refreshOnce(): Promise<void> {
+        const page = await this.client.collectionsSince({
+            sinceTime: this.store.collectionsSinceTime,
+        });
+
+        // Stage per-collection file diffs. A collection's files are
+        // re-enumerated only when its updationTime has advanced past the cached
+        // copy; an unchanged album's file list cannot have changed. New
+        // collections enumerate from the beginning of time.
+        const filePages: { collectionID: number; page: FilesPage }[] = [];
+        for (const collection of page.collections) {
+            const known = this.store.getCollection(collection.id);
+            if (known && collection.updationTime <= known.updationTime)
+                continue;
+            const filePage = await this.client.filesSince({
+                collectionID: collection.id,
+                collectionKey: collection.key,
+                sinceTime: known ? known.updationTime : 0,
+            });
+            filePages.push({ collectionID: collection.id, page: filePage });
+        }
+
+        // Network work done; commit to the store and persist only if something
+        // actually changed.
+        let changed = false;
+
+        if (this.store.userID !== this.userID) {
+            this.store.userID = this.userID;
+            changed = true;
+        }
+
+        for (const id of page.deleted) {
+            if (this.store.getCollection(id)) {
+                this.store.deleteCollection(id);
+                changed = true;
+            }
+        }
+        for (const collection of page.collections) {
+            this.store.putCollection(collection);
+            changed = true;
+        }
+
+        for (const { collectionID, page: filePage } of filePages) {
+            for (const id of filePage.deleted) {
+                if (this.store.getFile(collectionID, id)) {
+                    this.store.deleteFile(collectionID, id);
+                    changed = true;
+                }
+            }
+            for (const f of filePage.files) {
+                this.store.putFile(f);
+                changed = true;
+            }
+        }
+
+        if (page.cursor !== this.store.collectionsSinceTime) {
+            this.store.collectionsSinceTime = page.cursor;
+            changed = true;
+        }
+
+        if (changed) this.unsaved = true;
+
+        // Reproject and notify subscribers of the delta. This tracks RAM (what
+        // reads see), so it fires whether or not the save below succeeds; a
+        // save failure surfaces separately through `status().lastError`.
+        // `lastRecords` advances every changed refresh so the next diff is
+        // against current state.
+        if (changed) {
+            const next = this.deriveNow();
+            if (this.subscribers.size > 0) {
+                const change = diffRecords(this.lastRecords, next, Date.now());
+                if (change) this.notify(change);
+            }
+            this.lastRecords = next;
+        }
+
+        // Persist whenever RAM holds changes disk has not accepted — including
+        // changes an earlier cycle staged whose save failed. `unsaved` clears
+        // only once a save lands, so a save failure both stays visible through
+        // `status().lastError` (the throw below records it) and keeps being
+        // retried, instead of a later empty refresh silently clearing it while
+        // the on-disk cache is still behind RAM.
+        if (this.unsaved) {
+            await this.store.save();
+            this.unsaved = false;
+        }
+    }
+
+    // One ML fetch pass: fetch, decrypt and store the ML data for every file
+    // the store knows about that is not cached (or whose `updationTime` has
+    // advanced), through the metadata pool, and update the CLIP index. Guarded
+    // so passes never overlap; a failure is reported, not thrown.
+    private async runMLFetch(): Promise<void> {
+        const mldata = this.mldata;
+        // Bind so the call keeps the client as its receiver when invoked
+        // through the pool below.
+        const fetchMLData = this.client.fetchMLData?.bind(this.client);
+        if (!mldata || !fetchMLData || this.closed || this.mlFetching) return;
+
+        const files = this.uniqueFiles();
+        const needed = mldata.neededFor(files);
+        if (needed.length === 0) return;
+
+        this.mlFetching = true;
+        this.emit({ operation: "fetchMLData", status: "started" });
+        try {
+            const fileKeys = new Map<number, Uint8Array>();
+            const updation = new Map<number, number>();
+            for (const f of files) {
+                fileKeys.set(f.id, f.key);
+                updation.set(f.id, f.updationTime);
+            }
+
+            let stored = 0;
+            for (let i = 0; i < needed.length; i += MLDATA_BATCH_SIZE) {
+                if (this.closed) break;
+                const batch = needed.slice(i, i + MLDATA_BATCH_SIZE);
+                const payloads = await this.pools.metadata.run(
+                    () => fetchMLData({ fileIDs: batch, fileKeys }),
+                    { priority: "background" },
+                );
+                stored += (await mldata.storeFetched(payloads, updation))
+                    .stored;
+            }
+
+            this.lastMLFetchAt = Date.now();
+            this.lastMLError = undefined;
+            this.emit({
+                operation: "fetchMLData",
+                status: "done",
+                fetched: stored,
+            });
+        } catch (err) {
+            const error = err instanceof Error ? err.message : String(err);
+            this.lastMLError = error;
+            this.emit({ operation: "fetchMLData", status: "failed", error });
+        } finally {
+            this.mlFetching = false;
+        }
+    }
+
+    // The distinct files the store holds, one entry per fileID (a file in
+    // several collections shares its ML data), each carrying the key and the
+    // newest `updationTime` seen across its memberships.
+    private uniqueFiles(): {
+        id: number;
+        key: Uint8Array;
+        updationTime: number;
+    }[] {
+        const byID = new Map<
+            number,
+            { id: number; key: Uint8Array; updationTime: number }
+        >();
+        for (const collection of this.store.listCollections()) {
+            for (const f of this.store.listFiles(collection.id)) {
+                const seen = byID.get(f.id);
+                if (seen === undefined || f.updationTime > seen.updationTime)
+                    byID.set(f.id, {
+                        id: f.id,
+                        key: f.key,
+                        updationTime: f.updationTime,
+                    });
+            }
+        }
+        return [...byID.values()];
+    }
+
+    // Gather every file membership and project the store into by-id records,
+    // filling each record's cache paths from the content cache when present.
+    private deriveNow(): DerivedRecords {
+        const collections = this.store.listCollections();
+        const files: EnteFile[] = [];
+        for (const c of collections) files.push(...this.store.listFiles(c.id));
+        const cache = this.cache;
+        return deriveRecords(
+            collections,
+            files,
+            cache ? (fileID) => cache.pathsFor(fileID) : undefined,
+        );
+    }
+
+    private notify(change: LibraryChange): void {
+        for (const onChange of this.subscribers) {
+            // A misbehaving subscriber must not break the loop or its peers.
+            try {
+                onChange(change);
+            } catch {
+                // ignore
+            }
+        }
+    }
+
+    private emit(event: RefreshEvent): void {
+        if (!this.onProgress) return;
+        // A misbehaving callback must not break the refresh loop.
+        try {
+            this.onProgress(event);
+        } catch {
+            // ignore
+        }
+    }
+}

src/library/mldata.ts

@@ -0,0 +1,378 @@
+// The on-disk cache of Ente's per-file machine-learning data and the CLIP
+// index derived from it (issue #49).
+//
+// Under `<cacheDirectory>/mldata/` this keeps:
+//
+//   - `<fileID>.json` — one decrypted, gunzipped payload per file, written by
+//     rename. Its presence means it is complete: a torn write never leaves a
+//     half-file, so the set of these files is the source of truth for what is
+//     cached. The full payload (face boxes, landmarks, embeddings) is read back
+//     from here on demand and never held in RAM.
+//
+//   - `clip.f32` + `clip.json` — the derived index the content search runs on.
+//     `clip.json` lists the indexed fileIDs in order plus the embedding length;
+//     `clip.f32` is those CLIP embeddings packed as one `Float32Array`, so the
+//     index loads in a single read with no per-vector parse. The index is
+//     rebuilt from the payloads whenever it is missing or structurally
+//     disagrees with the files present, and appended to as new payloads arrive.
+//
+//   - `fetched.json` — a small map of fileID to the `updationTime` it was
+//     fetched at. This is best-effort bookkeeping for refetch decisions (a file
+//     whose `updationTime` later advances is refetched); the payloads, not this
+//     file, remain the record of what is cached, so losing it only forgoes
+//     update-driven refetch until the next fetch rewrites it.
+//
+// In RAM this holds only the id list and the packed `Float32Array`.
+
+import { mkdir, readFile, readdir } from "node:fs/promises";
+import { join } from "node:path";
+
+import { writeAtomic } from "../download/index.js";
+import type { MLData } from "../mldata-fetch.js";
+
+const CLIP_VECTORS = "clip.f32";
+const CLIP_INDEX = "clip.json";
+const FETCHED = "fetched.json";
+// A payload file is named for its fileID alone; the derived files above are
+// not, so this pattern picks out payloads and nothing else.
+const PAYLOAD_RE = /^(\d+)\.json$/;
+const BYTES_PER_FLOAT = 4;
+
+// The on-disk form of `clip.json`.
+interface ClipIndexFile {
+    fileIDs: number[];
+    embeddingLength: number;
+}
+
+// A file the model knows about, for deciding what to fetch.
+export interface MLDataFile {
+    id: number;
+    updationTime: number;
+}
+
+// The RAM index the search reads: `fileIDs[i]` owns the `embeddingLength`
+// floats of `embeddings` starting at `i * embeddingLength`.
+export interface MLIndex {
+    fileIDs: number[];
+    embeddingLength: number;
+    embeddings: Float32Array;
+}
+
+// Pull the CLIP embedding out of a payload, or undefined when it is absent or
+// misshapen. Kept strict so a bad payload is skipped rather than corrupting the
+// packed index.
+const clipEmbedding = (payload: MLData): number[] | undefined => {
+    const clip = payload.clip;
+    if (typeof clip !== "object" || clip === null) return undefined;
+    const embedding = (clip as { embedding?: unknown }).embedding;
+    if (!Array.isArray(embedding)) return undefined;
+    if (embedding.some((v) => typeof v !== "number" || !Number.isFinite(v)))
+        return undefined;
+    return embedding as number[];
+};
+
+export class MLDataStore {
+    readonly dir: string;
+
+    // fileIDs whose payload JSON is present on disk (present means complete).
+    private readonly present = new Set<number>();
+    // fileID -> updationTime it was fetched at.
+    private readonly fetched = new Map<number, number>();
+
+    // The packed index and where each id sits in it.
+    private ids: number[] = [];
+    private embeddingLength = 0;
+    private embeddings = new Float32Array(0);
+    private readonly pos = new Map<number, number>();
+
+    private constructor(dir: string) {
+        this.dir = dir;
+    }
+
+    // Open (creating the directory) and load the id list and packed index into
+    // RAM, rebuilding the index from the payloads when it is missing or does
+    // not match the files present.
+    static async open(dir: string): Promise<MLDataStore> {
+        const store = new MLDataStore(dir);
+        await mkdir(dir, { recursive: true });
+        await store.loadPresent();
+        await store.loadFetched();
+        if (!(await store.tryLoadIndex())) await store.rebuildIndex();
+        return store;
+    }
+
+    // The fileIDs among `files` that must be fetched: every file with no
+    // payload yet (first run, then new files), plus any whose `updationTime`
+    // has advanced past the one its cached payload was fetched at. Returned
+    // sorted and unique.
+    neededFor(files: MLDataFile[]): number[] {
+        const latest = new Map<number, number>();
+        for (const f of files) {
+            const seen = latest.get(f.id);
+            if (seen === undefined || f.updationTime > seen)
+                latest.set(f.id, f.updationTime);
+        }
+        const needed: number[] = [];
+        for (const [id, updationTime] of latest) {
+            if (!this.present.has(id)) {
+                needed.push(id);
+                continue;
+            }
+            const at = this.fetched.get(id);
+            if (at !== undefined && updationTime > at) needed.push(id);
+        }
+        return needed.sort((a, b) => a - b);
+    }
+
+    // Store a batch of fetched payloads: write one file per id, fold their CLIP
+    // embeddings into the packed index (in place for a refetch, appended for a
+    // new file), and persist the derived files. Returns how many payloads were
+    // stored and how many ids the index now holds.
+    async storeFetched(
+        payloads: Map<number, MLData>,
+        updation: Map<number, number>,
+    ): Promise<{ stored: number; indexed: number }> {
+        if (payloads.size === 0) return { stored: 0, indexed: this.ids.length };
+
+        for (const [id, payload] of payloads) {
+            await this.writePayload(id, payload);
+            this.present.add(id);
+            const at = updation.get(id);
+            if (at !== undefined) this.fetched.set(id, at);
+        }
+
+        const updates: { at: number; vector: number[] }[] = [];
+        const appends: { id: number; vector: number[] }[] = [];
+        for (const [id, payload] of payloads) {
+            const vector = clipEmbedding(payload);
+            if (!vector) continue;
+            if (this.embeddingLength === 0 && this.ids.length === 0)
+                this.embeddingLength = vector.length;
+            // The index is fixed-width; a vector of another length (never seen
+            // from Ente's CLIP model) is stored but left out of the index.
+            if (vector.length !== this.embeddingLength) continue;
+            const at = this.pos.get(id);
+            if (at !== undefined) updates.push({ at, vector });
+            else appends.push({ id, vector });
+        }
+
+        for (const { at, vector } of updates)
+            this.embeddings.set(vector, at * this.embeddingLength);
+
+        if (appends.length > 0) {
+            const length = this.embeddingLength;
+            const grown = new Float32Array(
+                this.embeddings.length + appends.length * length,
+            );
+            grown.set(this.embeddings);
+            let offset = this.embeddings.length;
+            for (const { id, vector } of appends) {
+                grown.set(vector, offset);
+                this.pos.set(id, this.ids.length);
+                this.ids.push(id);
+                offset += length;
+            }
+            this.embeddings = grown;
+        }
+
+        await this.persistIndex();
+        await this.persistFetched();
+        return { stored: payloads.size, indexed: this.ids.length };
+    }
+
+    // The packed index the search runs on. The id list is copied so callers
+    // cannot disturb the store's own order; the embeddings are the live buffer.
+    getIndex(): MLIndex {
+        return {
+            fileIDs: [...this.ids],
+            embeddingLength: this.embeddingLength,
+            embeddings: this.embeddings,
+        };
+    }
+
+    // The full payload for a file, read from disk, or undefined when it is not
+    // cached or does not parse.
+    async readPayload(fileID: number): Promise<MLData | undefined> {
+        if (!this.present.has(fileID)) return undefined;
+        let raw: string;
+        try {
+            raw = await readFile(this.payloadPath(fileID), "utf-8");
+        } catch {
+            return undefined;
+        }
+        try {
+            return JSON.parse(raw) as MLData;
+        } catch {
+            return undefined;
+        }
+    }
+
+    stats(): { stored: number; indexed: number } {
+        return { stored: this.present.size, indexed: this.ids.length };
+    }
+
+    private payloadPath(id: number): string {
+        return join(this.dir, `${id}.json`);
+    }
+
+    private async writePayload(id: number, payload: MLData): Promise<void> {
+        await writeAtomic(
+            this.payloadPath(id),
+            new TextEncoder().encode(JSON.stringify(payload)),
+        );
+    }
+
+    private async loadPresent(): Promise<void> {
+        let names: string[];
+        try {
+            names = await readdir(this.dir);
+        } catch {
+            return;
+        }
+        for (const name of names) {
+            const match = PAYLOAD_RE.exec(name);
+            if (match) this.present.add(Number(match[1]));
+        }
+    }
+
+    private async loadFetched(): Promise<void> {
+        let raw: string;
+        try {
+            raw = await readFile(join(this.dir, FETCHED), "utf-8");
+        } catch {
+            return;
+        }
+        try {
+            const parsed = JSON.parse(raw) as Record<string, unknown>;
+            for (const [key, value] of Object.entries(parsed)) {
+                const id = Number(key);
+                if (
+                    Number.isInteger(id) &&
+                    typeof value === "number" &&
+                    this.present.has(id)
+                )
+                    this.fetched.set(id, value);
+            }
+        } catch {
+            // Corrupt bookkeeping degrades refetch decisions, never fails open.
+        }
+    }
+
+    // Load the packed index if it is present and agrees with the payloads in
+    // both directions: every id it names must still be present, its vector file
+    // must be exactly the size the id count and embedding length imply, and no
+    // embedding-bearing payload on disk may be missing from it. Returns whether
+    // it loaded.
+    private async tryLoadIndex(): Promise<boolean> {
+        let metaRaw: string;
+        try {
+            metaRaw = await readFile(join(this.dir, CLIP_INDEX), "utf-8");
+        } catch {
+            return false;
+        }
+        let meta: ClipIndexFile;
+        try {
+            meta = JSON.parse(metaRaw) as ClipIndexFile;
+        } catch {
+            return false;
+        }
+        if (
+            !Array.isArray(meta.fileIDs) ||
+            typeof meta.embeddingLength !== "number"
+        )
+            return false;
+        if (meta.fileIDs.some((id) => !this.present.has(id))) return false;
+
+        // The reverse must hold too. A payload carrying an embedding but absent
+        // from the index means the index is stale — realistically the process
+        // died after storeFetched renamed the payloads into place but before it
+        // rewrote clip.json/clip.f32. Loading such an index as "consistent"
+        // would drop those embeddings for good (neededFor sees the payloads
+        // present and never refetches), so treat it as a disagreement and
+        // rebuild. Only present ids the index omits are read; a payload
+        // legitimately without an embedding stays out and forces no rebuild.
+        const indexed = new Set(meta.fileIDs);
+        for (const id of this.present) {
+            if (indexed.has(id)) continue;
+            const payload = await this.readPayload(id);
+            if (payload && clipEmbedding(payload)) return false;
+        }
+
+        let bytes: Buffer;
+        try {
+            bytes = await readFile(join(this.dir, CLIP_VECTORS));
+        } catch {
+            return false;
+        }
+        const expected =
+            meta.fileIDs.length * meta.embeddingLength * BYTES_PER_FLOAT;
+        if (bytes.byteLength !== expected) return false;
+
+        // One read, no parse: copy into an aligned buffer and view it as
+        // floats. The copy is needed because a Buffer from the pool can start
+        // at an offset a Float32Array cannot be laid over.
+        const aligned = new Uint8Array(bytes.byteLength);
+        aligned.set(bytes);
+        this.embeddings = new Float32Array(aligned.buffer);
+        this.embeddingLength = meta.embeddingLength;
+        this.ids = [...meta.fileIDs];
+        this.pos.clear();
+        this.ids.forEach((id, i) => this.pos.set(id, i));
+        return true;
+    }
+
+    // Rebuild the packed index by reading every payload present, then persist
+    // it. Payloads without a CLIP embedding (or of an unexpected length) are
+    // simply not indexed.
+    private async rebuildIndex(): Promise<void> {
+        this.ids = [];
+        this.pos.clear();
+        this.embeddingLength = 0;
+        const vectors: number[][] = [];
+        for (const id of [...this.present].sort((a, b) => a - b)) {
+            const payload = await this.readPayload(id);
+            if (!payload) continue;
+            const vector = clipEmbedding(payload);
+            if (!vector) continue;
+            if (this.embeddingLength === 0)
+                this.embeddingLength = vector.length;
+            if (vector.length !== this.embeddingLength) continue;
+            this.pos.set(id, this.ids.length);
+            this.ids.push(id);
+            vectors.push(vector);
+        }
+        const length = this.embeddingLength;
+        const packed = new Float32Array(this.ids.length * length);
+        vectors.forEach((vector, i) => packed.set(vector, i * length));
+        this.embeddings = packed;
+        await this.persistIndex();
+    }
+
+    private async persistIndex(): Promise<void> {
+        const meta: ClipIndexFile = {
+            fileIDs: this.ids,
+            embeddingLength: this.embeddingLength,
+        };
+        await writeAtomic(
+            join(this.dir, CLIP_INDEX),
+            new TextEncoder().encode(JSON.stringify(meta)),
+        );
+        await writeAtomic(
+            join(this.dir, CLIP_VECTORS),
+            new Uint8Array(
+                this.embeddings.buffer,
+                this.embeddings.byteOffset,
+                this.embeddings.byteLength,
+            ),
+        );
+    }
+
+    private async persistFetched(): Promise<void> {
+        const record: Record<string, number> = {};
+        for (const [id, at] of this.fetched) record[id] = at;
+        await writeAtomic(
+            join(this.dir, FETCHED),
+            new TextEncoder().encode(JSON.stringify(record)),
+        );
+    }
+}

src/library/pools.ts

@@ -0,0 +1,181 @@
+// Three bounded request pools for metadata, content, and thumbnails (issue
+// #45).
+//
+// Ente meters differently by traffic class, so quak keeps three independent
+// pools instead of one global limit: metadata is cheap and chatty, original
+// content is heavy, thumbnails are small but numerous. Each pool is a
+// `BoundedPool` — a plain concurrency limiter — and the three run at the
+// design's caps (10 / 5 / 25) unless the caller overrides them.
+//
+// Two behaviours beyond a bare limiter, both per pool:
+//
+//   - Priority: work waiting for a slot is ordered on-demand before
+//     background/precache, so a slot that frees up serves the request a user is
+//     waiting on ahead of speculative prefetch. Within one priority the order
+//     is first-come-first-served.
+//
+//   - In-flight dedup: a task submitted under a `key` that a still-pending task
+//     already carries is not run a second time; both callers await the one
+//     result. The key is released the moment that task settles — success or
+//     failure — so a later request for the same key runs afresh. Callers key by
+//     the id whose fetch must not be duplicated (a fileID, say).
+//
+// The pool holds a task's slot for that task's entire lifetime. A task that
+// retries internally is doing so inside its slot: the slot is not freed between
+// attempts, which is what keeps a retrying request counted against the cap. The
+// pool knows nothing of the retry policy; it only holds the slot until the
+// task's promise settles.
+//
+// This module is self-contained infrastructure. Routing a given `Client` call
+// to the right pool belongs to the unit that wires the pools into the cache;
+// here there is only the machinery.
+
+export const DEFAULT_METADATA_CONCURRENCY = 10;
+export const DEFAULT_CONTENT_CONCURRENCY = 5;
+export const DEFAULT_THUMBNAIL_CONCURRENCY = 25;
+
+// On-demand work is served before background/precache work waiting in the same
+// pool.
+export type Priority = "on-demand" | "background";
+
+export interface RunOptions {
+    // Defaults to "background": an unmarked request yields to on-demand work.
+    priority?: Priority;
+    // When set, a task already pending under this key is shared instead of run
+    // again. Omit for work that must always execute.
+    key?: string | number;
+}
+
+// One queued submission awaiting a slot. `start` runs the task and holds the
+// slot until it settles.
+interface Waiter {
+    priority: Priority;
+    // Submission order, used to break ties within a priority (FIFO).
+    seq: number;
+    start: () => void;
+}
+
+export class BoundedPool {
+    readonly concurrency: number;
+
+    private active = 0;
+    private nextSeq = 0;
+    private readonly waiting: Waiter[] = [];
+    // Keyed by a caller-supplied dedup key; holds the shared promise for as
+    // long as that task is pending, cleared when it settles.
+    private readonly pending = new Map<string | number, Promise<unknown>>();
+
+    constructor(concurrency: number) {
+        if (!Number.isInteger(concurrency) || concurrency < 1) {
+            throw new RangeError(
+                `concurrency must be a positive integer, got ${concurrency}`,
+            );
+        }
+        this.concurrency = concurrency;
+    }
+
+    // Submit `task` to the pool. It runs once a slot is free, subject to
+    // priority; the returned promise settles with the task's result. With a
+    // `key`, a still-pending submission under the same key is returned instead
+    // of running `task` again.
+    run<T>(task: () => Promise<T>, opts: RunOptions = {}): Promise<T> {
+        const { key } = opts;
+        if (key !== undefined) {
+            const shared = this.pending.get(key);
+            if (shared !== undefined) return shared as Promise<T>;
+        }
+
+        const promise = this.enqueue(task, opts.priority ?? "background");
+
+        if (key !== undefined) {
+            this.pending.set(key, promise);
+            const release = (): void => {
+                // Only clear our own entry: a fresh submission under the same
+                // key after this one settled must not be evicted here.
+                if (this.pending.get(key) === promise) this.pending.delete(key);
+            };
+            promise.then(release, release);
+        }
+        return promise;
+    }
+
+    private enqueue<T>(task: () => Promise<T>, priority: Priority): Promise<T> {
+        return new Promise<T>((resolve, reject) => {
+            const start = (): void => {
+                this.active++;
+                // Hold the slot until the task fully settles — every internal
+                // retry included — then admit the next waiter.
+                void (async () => {
+                    try {
+                        resolve(await task());
+                    } catch (err) {
+                        reject(err);
+                    } finally {
+                        this.active--;
+                        this.pump();
+                    }
+                })();
+            };
+            this.waiting.push({ priority, seq: this.nextSeq++, start });
+            this.pump();
+        });
+    }
+
+    // Admit waiters until the pool is full or the queue is empty.
+    private pump(): void {
+        while (this.active < this.concurrency) {
+            const next = this.takeNext();
+            if (next === undefined) return;
+            next.start();
+        }
+    }
+
+    // Remove and return the highest-priority waiter: on-demand before
+    // background, earliest submission first within a priority.
+    private takeNext(): Waiter | undefined {
+        let bestIndex = -1;
+        let best: Waiter | undefined;
+        for (let i = 0; i < this.waiting.length; i++) {
+            const w = this.waiting[i];
+            if (w === undefined) continue;
+            if (best === undefined || this.precedes(w, best)) {
+                best = w;
+                bestIndex = i;
+            }
+        }
+        if (best === undefined) return undefined;
+        this.waiting.splice(bestIndex, 1);
+        return best;
+    }
+
+    private precedes(a: Waiter, b: Waiter): boolean {
+        if (a.priority !== b.priority) return a.priority === "on-demand";
+        return a.seq < b.seq;
+    }
+}
+
+export interface RequestPoolsOptions {
+    metadataConcurrency?: number;
+    contentConcurrency?: number;
+    thumbnailConcurrency?: number;
+}
+
+// The three pools the design calls for, each independent: an idle pool never
+// lends its slots to a busy one.
+export class RequestPools {
+    readonly metadata: BoundedPool;
+    readonly content: BoundedPool;
+    readonly thumbnails: BoundedPool;
+
+    constructor(opts: RequestPoolsOptions = {}) {
+        this.metadata = new BoundedPool(
+            opts.metadataConcurrency ?? DEFAULT_METADATA_CONCURRENCY,
+        );
+        this.content = new BoundedPool(
+            opts.contentConcurrency ?? DEFAULT_CONTENT_CONCURRENCY,
+        );
+        this.thumbnails = new BoundedPool(
+            opts.thumbnailConcurrency ?? DEFAULT_THUMBNAIL_CONCURRENCY,
+        );
+    }
+}

src/library/read.ts

@@ -0,0 +1,379 @@
+// The in-process read surface over the local cache (issue #44).
+//
+// A CLI or an in-process script reads albums, photos, and a grouped timeline
+// through `lib.albums`, `lib.photos`, and `lib.timeline`. Every call is
+// answered synchronously from the same plain-record projection the GUI reads
+// (`deriveRecords`, issue #43); nothing here touches the network. Every method
+// takes a single named-argument object.
+//
+// The `Album` and `Photo` classes are thin, in-process-only wrappers over
+// those records: a caller that holds an object reference gets typed field
+// access and, for an album, its photos. They are not sent across IPC — the
+// plain records are the serializable surface, and `record()` returns one.
+//
+// A `Photo` also fetches its own bytes: `original()` and `thumbnail()` go
+// through the on-disk content cache (issue #46), the one place in this module
+// that is not synchronous and RAM-only. A library opened without a content
+// source leaves that cache absent, and those two methods then throw.
+
+import type { CollectionType, FileType } from "../model/types.js";
+import type { ContentOptions, ContentResult, PhotoContent } from "./content.js";
+import type { AlbumRecord, PhotoRecord, DerivedRecords } from "./records.js";
+
+// Newest first, with fileID as a stable tiebreak so equal-timed files order
+// deterministically — the same order the record projection uses.
+const byNewest = (a: PhotoRecord, b: PhotoRecord): number =>
+    b.takenAt - a.takenAt || b.fileID - a.fileID;
+
+// Albums newest updated first, collection id breaking ties. This is the order
+// `albums.list` returns and the order `byName` resolves a name collision in.
+const byNewestAlbum = (a: AlbumRecord, b: AlbumRecord): number =>
+    b.updationTime - a.updationTime || b.collectionID - a.collectionID;
+
+// A single photo. Field access mirrors `PhotoRecord`; `record()` returns the
+// underlying plain record for callers that need the IPC-safe value.
+export class Photo {
+    constructor(
+        private readonly rec: PhotoRecord,
+        private readonly content?: PhotoContent,
+    ) {}
+
+    get fileID(): number {
+        return this.rec.fileID;
+    }
+    get albumIDs(): number[] {
+        return this.rec.albumIDs;
+    }
+    get title(): string {
+        return this.rec.title;
+    }
+    get takenAt(): number {
+        return this.rec.takenAt;
+    }
+    get fileType(): FileType {
+        return this.rec.fileType;
+    }
+    get caption(): string | undefined {
+        return this.rec.caption;
+    }
+    get width(): number | undefined {
+        return this.rec.width;
+    }
+    get height(): number | undefined {
+        return this.rec.height;
+    }
+    get latitude(): number | undefined {
+        return this.rec.latitude;
+    }
+    get longitude(): number | undefined {
+        return this.rec.longitude;
+    }
+    get isArchived(): boolean {
+        return this.rec.isArchived;
+    }
+    get isHidden(): boolean {
+        return this.rec.isHidden;
+    }
+
+    record(): PhotoRecord {
+        return this.rec;
+    }
+
+    // Fetch and cache the full-resolution original, returning its on-disk path
+    // and byte length. Served from the cache (or the backup download directory)
+    // when already present, otherwise fetched through the content pool.
+    async original(opts?: ContentOptions): Promise<ContentResult> {
+        return this.contentOrThrow().original(this.rec.fileID, opts);
+    }
+
+    // As `original`, for the thumbnail, through the thumbnail pool.
+    async thumbnail(opts?: ContentOptions): Promise<ContentResult> {
+        return this.contentOrThrow().thumbnail(this.rec.fileID, opts);
+    }
+
+    private contentOrThrow(): PhotoContent {
+        if (!this.content) {
+            throw new Error(
+                "Photo content requires a library opened with a content cache",
+            );
+        }
+        return this.content;
+    }
+}
+
+// A single album. `photos.list()` returns the album's photos as wrappers,
+// newest first (the record already stores `fileIDs` in that order).
+export class Album {
+    constructor(
+        private readonly rec: AlbumRecord,
+        private readonly records: DerivedRecords,
+        private readonly content?: PhotoContent,
+    ) {}
+
+    get collectionID(): number {
+        return this.rec.collectionID;
+    }
+    get name(): string {
+        return this.rec.name;
+    }
+    get type(): CollectionType {
+        return this.rec.type;
+    }
+    get isShared(): boolean {
+        return this.rec.isShared;
+    }
+    get updationTime(): number {
+        return this.rec.updationTime;
+    }
+    get fileIDs(): number[] {
+        return this.rec.fileIDs;
+    }
+
+    get photos(): { list: () => Photo[] } {
+        return { list: (): Photo[] => this.listPhotos() };
+    }
+
+    record(): AlbumRecord {
+        return this.rec;
+    }
+
+    private listPhotos(): Photo[] {
+        const out: Photo[] = [];
+        for (const id of this.rec.fileIDs) {
+            const p = this.records.photos.get(id);
+            if (p) out.push(new Photo(p, this.content));
+        }
+        return out;
+    }
+}
+
+export interface AlbumsAPI {
+    list(): Album[];
+    byName(args: { albumName: string }): Album | undefined;
+    byID(args: { collectionID: number }): Album | undefined;
+}
+
+export interface PhotosAPI {
+    byID(args: { fileID: number }): Photo | undefined;
+    // Plain records for the requested ids, in the order requested, each id at
+    // most once, unknown ids dropped.
+    records(args: { fileIDs: number[] }): PhotoRecord[];
+}
+
+export type GroupBy = "day" | "week" | "month";
+
+// A filter over the timeline. All fields are optional and combine with AND.
+// Hidden photos are never included, regardless of this filter.
+export interface PhotoFilter {
+    // Keep only photos that belong to this album.
+    albumID?: number;
+    // Case-insensitive substring of the title, caption, or any album name the
+    // photo belongs to.
+    text?: string;
+    // Keep only photos of one of these types.
+    fileTypes?: FileType[];
+    // `true` keeps only geotagged photos; `false` keeps only those without a
+    // location; omitted places no constraint.
+    hasLocation?: boolean;
+    // Archived photos are excluded unless this is `true`. Defaults to `false`.
+    includeArchived?: boolean;
+}
+
+export interface TimelineGroup {
+    // The period's identity: `YYYY-MM-DD` for day, `YYYY-Www` (ISO 8601 week,
+    // e.g. `2025-W32`) for week, and `YYYY-MM` for month.
+    key: string;
+    // Local-time milliseconds at the start of the period.
+    startsAt: number;
+    // The period's files, newest first, each file once.
+    fileIDs: number[];
+}
+
+export interface TimelineAPI {
+    groups(args: { groupBy: GroupBy; filter?: PhotoFilter }): TimelineGroup[];
+}
+
+export const makeAlbumsAPI = (
+    derive: () => DerivedRecords,
+    content?: PhotoContent,
+): AlbumsAPI => ({
+    list: (): Album[] => {
+        const records = derive();
+        return [...records.albums.values()]
+            .sort(byNewestAlbum)
+            .map((rec) => new Album(rec, records, content));
+    },
+    byID: ({ collectionID }): Album | undefined => {
+        const records = derive();
+        const rec = records.albums.get(collectionID);
+        return rec ? new Album(rec, records, content) : undefined;
+    },
+    byName: ({ albumName }): Album | undefined => {
+        const records = derive();
+        // Names are not unique in Ente; resolve a collision deterministically
+        // to the newest-updated album, matching `list` order.
+        const match = [...records.albums.values()]
+            .sort(byNewestAlbum)
+            .find((rec) => rec.name === albumName);
+        return match ? new Album(match, records, content) : undefined;
+    },
+});
+
+export const makePhotosAPI = (
+    derive: () => DerivedRecords,
+    content?: PhotoContent,
+): PhotosAPI => ({
+    byID: ({ fileID }): Photo | undefined => {
+        const rec = derive().photos.get(fileID);
+        return rec ? new Photo(rec, content) : undefined;
+    },
+    records: ({ fileIDs }): PhotoRecord[] => {
+        const { photos } = derive();
+        const seen = new Set<number>();
+        const out: PhotoRecord[] = [];
+        for (const id of fileIDs) {
+            if (seen.has(id)) continue;
+            const rec = photos.get(id);
+            if (rec) {
+                out.push(rec);
+                seen.add(id);
+            }
+        }
+        return out;
+    },
+});
+
+export const makeTimelineAPI = (derive: () => DerivedRecords): TimelineAPI => ({
+    groups: ({ groupBy, filter }): TimelineGroup[] => {
+        const records = derive();
+        return groupPhotos(filterPhotos(records, filter), groupBy);
+    },
+});
+
+// Apply a `PhotoFilter` to the projection. Hidden photos are always dropped;
+// archived photos are dropped unless `includeArchived` asks for them.
+const filterPhotos = (
+    records: DerivedRecords,
+    filter?: PhotoFilter,
+): PhotoRecord[] => {
+    const f = filter ?? {};
+    const includeArchived = f.includeArchived ?? false;
+    const needle = f.text?.toLowerCase();
+    const out: PhotoRecord[] = [];
+    for (const rec of records.photos.values()) {
+        if (rec.isHidden) continue;
+        if (rec.isArchived && !includeArchived) continue;
+        if (f.albumID !== undefined && !rec.albumIDs.includes(f.albumID))
+            continue;
+        if (f.fileTypes !== undefined && !f.fileTypes.includes(rec.fileType))
+            continue;
+        if (f.hasLocation !== undefined) {
+            const has =
+                rec.latitude !== undefined && rec.longitude !== undefined;
+            if (has !== f.hasLocation) continue;
+        }
+        if (needle !== undefined && !matchesText(rec, needle, records))
+            continue;
+        out.push(rec);
+    }
+    return out;
+};
+
+const matchesText = (
+    rec: PhotoRecord,
+    needle: string,
+    records: DerivedRecords,
+): boolean => {
+    if (rec.title.toLowerCase().includes(needle)) return true;
+    if (rec.caption !== undefined && rec.caption.toLowerCase().includes(needle))
+        return true;
+    for (const id of rec.albumIDs) {
+        const album = records.albums.get(id);
+        if (album && album.name.toLowerCase().includes(needle)) return true;
+    }
+    return false;
+};
+
+// Bucket photos into periods, groups newest first, members newest first.
+const groupPhotos = (
+    photos: PhotoRecord[],
+    groupBy: GroupBy,
+): TimelineGroup[] => {
+    const buckets = new Map<
+        string,
+        { startsAt: number; recs: PhotoRecord[] }
+    >();
+    for (const rec of photos) {
+        const { key, startsAt } = periodOf(rec.takenAt, groupBy);
+        const bucket = buckets.get(key);
+        if (bucket) bucket.recs.push(rec);
+        else buckets.set(key, { startsAt, recs: [rec] });
+    }
+    const groups: TimelineGroup[] = [];
+    for (const [key, bucket] of buckets) {
+        bucket.recs.sort(byNewest);
+        groups.push({
+            key,
+            startsAt: bucket.startsAt,
+            fileIDs: bucket.recs.map((r) => r.fileID),
+        });
+    }
+    groups.sort((a, b) => b.startsAt - a.startsAt);
+    return groups;
+};
+
+const pad = (n: number): string => String(n).padStart(2, "0");
+
+const dateKey = (d: Date): string =>
+    `${d.getFullYear()}-${pad(d.getMonth() + 1)}-${pad(d.getDate())}`;
+
+const WEEK_MS = 7 * 24 * 60 * 60 * 1000;
+
+// The ISO 8601 week key `YYYY-Www` for the week starting at the given Monday.
+// The week-year is the year of that week's Thursday, so it can differ from the
+// calendar year at the January/December boundary (e.g. 2024-12-30 is 2025-W01).
+const isoWeekKey = (monday: Date): string => {
+    const thursday = new Date(
+        monday.getFullYear(),
+        monday.getMonth(),
+        monday.getDate() + 3,
+    );
+    const isoYear = thursday.getFullYear();
+    // Thursday of ISO week 1 is the Thursday of the week containing January 4.
+    const jan4 = new Date(isoYear, 0, 4);
+    const week1Thursday = new Date(
+        isoYear,
+        0,
+        4 + 3 - ((jan4.getDay() + 6) % 7),
+    );
+    const week =
+        1 +
+        Math.round((thursday.getTime() - week1Thursday.getTime()) / WEEK_MS);
+    return `${isoYear}-W${pad(week)}`;
+};
+
+// The period a millisecond instant falls in, in local time. Weeks start on
+// Monday. `Date` normalizes out-of-range day arguments, so the week's Monday
+// is correct across month and year boundaries.
+const periodOf = (
+    takenAt: number,
+    groupBy: GroupBy,
+): { key: string; startsAt: number } => {
+    const d = new Date(takenAt);
+    const year = d.getFullYear();
+    const month = d.getMonth();
+    const day = d.getDate();
+
+    if (groupBy === "month") {
+        const start = new Date(year, month, 1);
+        return { key: `${year}-${pad(month + 1)}`, startsAt: start.getTime() };
+    }
+    if (groupBy === "week") {
+        // getDay(): 0=Sunday..6=Saturday; shift so Monday is the week start.
+        const fromMonday = (d.getDay() + 6) % 7;
+        const start = new Date(year, month, day - fromMonday);
+        return { key: isoWeekKey(start), startsAt: start.getTime() };
+    }
+    const start = new Date(year, month, day);
+    return { key: dateKey(start), startsAt: start.getTime() };
+};

src/library/records.ts

@@ -0,0 +1,270 @@
+// Plain records projected from the decrypted store, and the diff between two
+// projections. These are the library's GUI-facing surface: they hold no key
+// material and no binary, so they survive `structuredClone`/JSON across the
+// Electron IPC boundary where methods and file keys cannot go (design #36,
+// owner ruling 5). The decrypted `Collection`/`EnteFile` objects stay in RAM in
+// the main process; the window only ever sees these records.
+//
+// Ente holds edited/basic times in microseconds; records expose `takenAt` in
+// milliseconds. The magic-metadata field names below are the ones the Ente
+// clients write, confirmed against the repo's own fixtures: `w`/`h` in
+// test/cli/metadata-backup.test.ts, `visibility` in test/library/store.test.ts.
+
+import type {
+    Collection,
+    CollectionType,
+    EnteFile,
+    FileType,
+} from "../model/types.js";
+
+// Ente private-magic-metadata visibility values.
+const VISIBILITY_ARCHIVED = 1;
+const VISIBILITY_HIDDEN = 2;
+
+// A single photo, deduplicated across the collections it belongs to. No key,
+// no binary: safe to send to a window.
+export interface PhotoRecord {
+    fileID: number;
+    // Every collection this file is a member of, ascending.
+    albumIDs: number[];
+    // `pubMagicMetadata.editedName` when the user renamed the file, else the
+    // basic-metadata title.
+    title: string;
+    // Milliseconds. `pubMagicMetadata.editedTime` when the user edited the
+    // date, else basic-metadata `creationTime`.
+    takenAt: number;
+    fileType: FileType;
+    caption?: string;
+    width?: number;
+    height?: number;
+    latitude?: number;
+    longitude?: number;
+    isArchived: boolean;
+    isHidden: boolean;
+    // Local cache paths, set once a later phase caches the bytes; unset here.
+    thumbnailPath?: string;
+    originalPath?: string;
+}
+
+export interface AlbumRecord {
+    collectionID: number;
+    name: string;
+    // `favorites` identifies the account's favorites album.
+    type: CollectionType;
+    isShared: boolean;
+    updationTime: number;
+    // The album's files, newest first.
+    fileIDs: number[];
+}
+
+export interface LibrarySnapshot {
+    albums: AlbumRecord[];
+    photos: PhotoRecord[];
+    // Wall-clock milliseconds when the snapshot was taken.
+    takenAt: number;
+}
+
+export interface LibraryChange {
+    // Full records for albums/photos added or changed by the refresh.
+    albumsChanged: AlbumRecord[];
+    photosChanged: PhotoRecord[];
+    fileIDsRemoved: number[];
+    albumIDsRemoved: number[];
+    // Wall-clock milliseconds of the refresh that produced this change.
+    refreshedAt: number;
+}
+
+// The by-id projection of the store at one moment; the source for both
+// `snapshotFrom` (sorted arrays for the GUI) and `diffRecords` (change sets).
+export interface DerivedRecords {
+    albums: Map<number, AlbumRecord>;
+    photos: Map<number, PhotoRecord>;
+}
+
+const asString = (v: unknown): string | undefined =>
+    typeof v === "string" && v.length > 0 ? v : undefined;
+
+const asNumber = (v: unknown): number | undefined =>
+    typeof v === "number" && Number.isFinite(v) ? v : undefined;
+
+const microsToMillis = (micros: number): number => Math.floor(micros / 1000);
+
+// Newest first, with fileID as a stable tiebreak so equal-timed files order
+// deterministically.
+const byNewestPhoto = (a: PhotoRecord, b: PhotoRecord): number =>
+    b.takenAt - a.takenAt || b.fileID - a.fileID;
+
+// Build one PhotoRecord from every membership of a file. The memberships share
+// the same underlying file, so metadata is read from a single representative
+// (the most recently synced, lowest collection id to break ties); `albumIDs`
+// gathers them all.
+const toPhotoRecord = (
+    fileID: number,
+    memberships: EnteFile[],
+): PhotoRecord => {
+    const albumIDs = memberships
+        .map((m) => m.collectionID)
+        .sort((a, b) => a - b);
+    const rep = memberships.reduce((best, m) =>
+        m.updationTime > best.updationTime ||
+        (m.updationTime === best.updationTime &&
+            m.collectionID < best.collectionID)
+            ? m
+            : best,
+    );
+
+    const pub = rep.pubMagicMetadata ?? {};
+    const priv = rep.magicMetadata ?? {};
+
+    const takenAtMicros = asNumber(pub.editedTime) ?? rep.metadata.creationTime;
+    const visibility = asNumber(priv.visibility);
+
+    const record: PhotoRecord = {
+        fileID,
+        albumIDs,
+        title: asString(pub.editedName) ?? rep.metadata.title,
+        takenAt: microsToMillis(takenAtMicros),
+        fileType: rep.metadata.fileType,
+        isArchived: visibility === VISIBILITY_ARCHIVED,
+        isHidden: visibility === VISIBILITY_HIDDEN,
+    };
+
+    const caption = asString(pub.caption);
+    if (caption !== undefined) record.caption = caption;
+    const width = asNumber(pub.w);
+    if (width !== undefined) record.width = width;
+    const height = asNumber(pub.h);
+    if (height !== undefined) record.height = height;
+    if (rep.metadata.latitude !== undefined)
+        record.latitude = rep.metadata.latitude;
+    if (rep.metadata.longitude !== undefined)
+        record.longitude = rep.metadata.longitude;
+
+    return record;
+};
+
+const toAlbumRecord = (
+    collection: Collection,
+    files: EnteFile[],
+    takenAtByFile: Map<number, number>,
+): AlbumRecord => {
+    const fileIDs = files
+        .filter((f) => f.collectionID === collection.id)
+        .map((f) => f.id)
+        .sort(
+            (a, b) =>
+                (takenAtByFile.get(b) ?? 0) - (takenAtByFile.get(a) ?? 0) ||
+                b - a,
+        );
+    return {
+        collectionID: collection.id,
+        name: collection.name,
+        type: collection.type,
+        isShared: collection.isShared,
+        updationTime: collection.updationTime,
+        fileIDs,
+    };
+};
+
+// The cache paths known for a file, so the projection can expose them on the
+// record without the read layer reaching into the content cache itself.
+export type CachedPathLookup = (fileID: number) => {
+    originalPath?: string;
+    thumbnailPath?: string;
+};
+
+// Project the decrypted collections and file memberships into by-id records.
+// `files` is every membership (a file appears once per collection it is in).
+// `cachedPaths`, when given, fills each record's cache paths.
+export const deriveRecords = (
+    collections: Collection[],
+    files: EnteFile[],
+    cachedPaths?: CachedPathLookup,
+): DerivedRecords => {
+    const byFileID = new Map<number, EnteFile[]>();
+    for (const f of files) {
+        const arr = byFileID.get(f.id);
+        if (arr) arr.push(f);
+        else byFileID.set(f.id, [f]);
+    }
+
+    const photos = new Map<number, PhotoRecord>();
+    const takenAtByFile = new Map<number, number>();
+    for (const [fileID, memberships] of byFileID) {
+        const record = toPhotoRecord(fileID, memberships);
+        if (cachedPaths) {
+            const paths = cachedPaths(fileID);
+            if (paths.originalPath !== undefined)
+                record.originalPath = paths.originalPath;
+            if (paths.thumbnailPath !== undefined)
+                record.thumbnailPath = paths.thumbnailPath;
+        }
+        photos.set(fileID, record);
+        takenAtByFile.set(fileID, record.takenAt);
+    }
+
+    const albums = new Map<number, AlbumRecord>();
+    for (const c of collections) {
+        albums.set(c.id, toAlbumRecord(c, files, takenAtByFile));
+    }
+
+    return { albums, photos };
+};
+
+// Sorted, GUI-ready arrays: albums newest updated first, photos newest first.
+export const snapshotFrom = (
+    records: DerivedRecords,
+    takenAt: number,
+): LibrarySnapshot => ({
+    albums: [...records.albums.values()].sort(
+        (a, b) =>
+            b.updationTime - a.updationTime || b.collectionID - a.collectionID,
+    ),
+    photos: [...records.photos.values()].sort(byNewestPhoto),
+    takenAt,
+});
+
+// Records compare by value; they are plain and built with a fixed key order, so
+// a serialized form is a sound equality key.
+const same = (a: unknown, b: unknown): boolean =>
+    JSON.stringify(a) === JSON.stringify(b);
+
+const diffMap = <T>(
+    prev: Map<number, T>,
+    next: Map<number, T>,
+): { changed: T[]; removed: number[] } => {
+    const changed: T[] = [];
+    for (const [id, record] of next) {
+        const before = prev.get(id);
+        if (before === undefined || !same(before, record)) changed.push(record);
+    }
+    const removed: number[] = [];
+    for (const id of prev.keys()) if (!next.has(id)) removed.push(id);
+    removed.sort((a, b) => a - b);
+    return { changed, removed };
+};
+
+// The change between two projections, or undefined when nothing changed.
+export const diffRecords = (
+    prev: DerivedRecords,
+    next: DerivedRecords,
+    refreshedAt: number,
+): LibraryChange | undefined => {
+    const albums = diffMap(prev.albums, next.albums);
+    const photos = diffMap(prev.photos, next.photos);
+    if (
+        albums.changed.length === 0 &&
+        albums.removed.length === 0 &&
+        photos.changed.length === 0 &&
+        photos.removed.length === 0
+    ) {
+        return undefined;
+    }
+    return {
+        albumsChanged: albums.changed,
+        photosChanged: photos.changed,
+        fileIDsRemoved: photos.removed,
+        albumIDsRemoved: albums.removed,
+        refreshedAt,
+    };
+};

src/library/store.ts

@@ -0,0 +1,205 @@
+// On-disk JSON metadata store for the local cache.
+//
+// The store keeps one `metadata.json` file holding the account's server
+// state: the user id, a schema version, the cursor for the incremental
+// collections listing, and the decrypted collection and file records. The
+// whole file is read into RAM on load and rewritten as a whole on save; there
+// is no partial update and no lock file. A separate refresh unit populates the
+// store from the server — this module only stores what it is given.
+//
+// The file is a cache, so it is never trusted to exist or to be intact: a
+// missing or unreadable file loads as an empty store rather than an error, and
+// the refresh unit then repopulates it.
+
+import { mkdir, chmod, readFile } from "node:fs/promises";
+import { dirname } from "node:path";
+
+import { writeAtomic } from "../download/index.js";
+import type { Collection, EnteFile, Microseconds } from "../model/types.js";
+
+// Bumped only when the on-disk shape changes incompatibly. A file written
+// under a different version is discarded on load (see `load`): re-fetching
+// from the server is always safe and cheaper than migrating a cache.
+export const METADATA_SCHEMA_VERSION = 1;
+
+// Directory and file modes match `session.json`: the records hold decrypted
+// key material, so on a shared machine only the owner may read them.
+const DIR_MODE = 0o700;
+const FILE_MODE = 0o600;
+
+// On-disk shapes. They mirror the in-memory model exactly except for the
+// binary `key`, which JSON cannot hold and which is stored as base64.
+type StoredCollection = Omit<Collection, "key"> & { key: string };
+type StoredFile = Omit<EnteFile, "key"> & { key: string };
+
+interface StoredMetadata {
+    schemaVersion: number;
+    userID: number;
+    collectionsSinceTime: Microseconds;
+    collections: StoredCollection[];
+    files: StoredFile[];
+}
+
+const encodeKey = (key: Uint8Array): string =>
+    Buffer.from(key).toString("base64");
+
+const decodeKey = (encoded: string): Uint8Array =>
+    new Uint8Array(Buffer.from(encoded, "base64"));
+
+// A file membership is identified by the pair (collectionID, fileID): the same
+// underlying file can belong to several collections, each a distinct record
+// with its own key.
+const fileKey = (collectionID: number, fileID: number): string =>
+    `${collectionID}:${fileID}`;
+
+export class MetadataStore {
+    readonly path: string;
+    readonly schemaVersion = METADATA_SCHEMA_VERSION;
+    userID = 0;
+    collectionsSinceTime: Microseconds = 0;
+
+    // True when `load` populated this store from a valid existing file; false
+    // on a first run or a missing/corrupt/wrong-version file that loaded empty.
+    // `Library.open` reads it to decide whether the first refresh may run in
+    // the background (an existing copy already serves reads) or must be awaited.
+    loadedFromDisk = false;
+
+    private readonly collections = new Map<number, Collection>();
+    private readonly files = new Map<string, EnteFile>();
+
+    private constructor(path: string) {
+        this.path = path;
+    }
+
+    // Load the store at `path`. A missing file, an unreadable one, unparseable
+    // contents, or a mismatched schema version all yield an empty store bound
+    // to that path — never a thrown error, because the file is only a cache.
+    static async load(path: string): Promise<MetadataStore> {
+        const store = new MetadataStore(path);
+
+        let raw: string;
+        try {
+            raw = await readFile(path, "utf8");
+        } catch {
+            return store;
+        }
+
+        try {
+            const parsed = JSON.parse(raw) as StoredMetadata;
+            if (parsed.schemaVersion !== METADATA_SCHEMA_VERSION) {
+                return store;
+            }
+            store.loadedFromDisk = true;
+            store.userID = parsed.userID ?? 0;
+            store.collectionsSinceTime = parsed.collectionsSinceTime ?? 0;
+            for (const stored of parsed.collections ?? []) {
+                const collection: Collection = {
+                    ...stored,
+                    key: decodeKey(stored.key),
+                };
+                store.collections.set(collection.id, collection);
+            }
+            for (const stored of parsed.files ?? []) {
+                const file: EnteFile = {
+                    ...stored,
+                    key: decodeKey(stored.key),
+                };
+                store.files.set(fileKey(file.collectionID, file.id), file);
+            }
+        } catch {
+            // Any corruption discards the partial result: a half-read cache is
+            // worse than an empty one, since the refresh unit will rebuild it.
+            return new MetadataStore(path);
+        }
+
+        return store;
+    }
+
+    // Rewrite the whole file. The directory is created 0700 and the file left
+    // 0600; the write itself is the download layer's durable atomic writer
+    // (temp file, fsync, rename, dir fsync), so a reader never sees a partial
+    // file and a crash cannot leave a truncated one. There is no lock file and
+    // no `sync()` beyond the writer's own fsyncs.
+    async save(): Promise<void> {
+        const model: StoredMetadata = {
+            schemaVersion: METADATA_SCHEMA_VERSION,
+            userID: this.userID,
+            collectionsSinceTime: this.collectionsSinceTime,
+            collections: [...this.collections.values()].map((c) => ({
+                ...c,
+                key: encodeKey(c.key),
+            })),
+            files: [...this.files.values()].map((f) => ({
+                ...f,
+                key: encodeKey(f.key),
+            })),
+        };
+
+        const dir = dirname(this.path);
+        // chmod after mkdir so the mode is 0700 even when the directory
+        // already existed with a looser mode; mkdir alone would not tighten
+        // an existing directory.
+        await mkdir(dir, { recursive: true, mode: DIR_MODE });
+        await chmod(dir, DIR_MODE);
+
+        const payload = new TextEncoder().encode(
+            JSON.stringify(model, null, 2),
+        );
+        await writeAtomic(this.path, payload);
+        // The atomic writer's temp file inherits the default mode; tighten the
+        // renamed file to 0600. The 0700 directory already keeps other users
+        // out during the brief window before this runs.
+        await chmod(this.path, FILE_MODE);
+    }
+
+    getCollection(id: number): Collection | undefined {
+        return this.collections.get(id);
+    }
+
+    listCollections(): Collection[] {
+        return [...this.collections.values()];
+    }
+
+    putCollection(collection: Collection): void {
+        this.collections.set(collection.id, collection);
+    }
+
+    // Removing a collection also drops its file memberships: a file record is
+    // only meaningful as part of a collection the cache still knows about.
+    deleteCollection(id: number): void {
+        this.collections.delete(id);
+        for (const [key, file] of this.files) {
+            if (file.collectionID === id) {
+                this.files.delete(key);
+            }
+        }
+    }
+
+    getFile(collectionID: number, fileID: number): EnteFile | undefined {
+        return this.files.get(fileKey(collectionID, fileID));
+    }
+
+    // Any membership of a file, or undefined. Every membership re-wraps the
+    // same underlying content key, so any one is enough to fetch the bytes;
+    // the content cache resolves a fileID to a file this way.
+    getFileByID(fileID: number): EnteFile | undefined {
+        for (const file of this.files.values()) {
+            if (file.id === fileID) return file;
+        }
+        return undefined;
+    }
+
+    listFiles(collectionID: number): EnteFile[] {
+        return [...this.files.values()].filter(
+            (f) => f.collectionID === collectionID,
+        );
+    }
+
+    putFile(file: EnteFile): void {
+        this.files.set(fileKey(file.collectionID, file.id), file);
+    }
+
+    deleteFile(collectionID: number, fileID: number): void {
+        this.files.delete(fileKey(collectionID, fileID));
+    }
+}

src/metadata-backup.ts

@@ -1,4 +1,3 @@
-import { gunzipSync } from "node:zlib";
 import {
     mkdirSync,
     mkdtempSync,
@@ -11,7 +10,7 @@ import { tmpdir } from "node:os";
 import * as jpeg from "jpeg-js";
 import exifReader from "exif-reader";
 import type { Client } from "./client.js";
-import { decryptBlob, fromBase64 } from "./crypto/index.js";
+import { fetchMLData } from "./mldata-fetch.js";
 import type { EnteFile } from "./model/types.js";
 
 export type ProgressCallback = (message: string) => void;
@@ -24,50 +23,6 @@ export interface MetadataBackupOptions {
 const sanitizePath = (name: string): string =>
     name.replace(/[/\\:*?"<>|]/g, "_").replace(/^\.+/, "_");
 
-interface RawRemoteFileData {
-    fileID: number;
-    encryptedData: string;
-    decryptionHeader: string;
-    updatedAt?: number;
-}
-
-const fetchMLDataForFiles = async (
-    client: Client,
-    fileIDs: number[],
-    fileKeys: Map<number, Uint8Array>,
-): Promise<Map<number, Record<string, unknown>>> => {
-    const api = client.getApiClient();
-    const result = new Map<number, Record<string, unknown>>();
-    const batchSize = 200;
-
-    for (let i = 0; i < fileIDs.length; i += batchSize) {
-        const batch = fileIDs.slice(i, i + batchSize);
-        const { data } = await api.postJSON<{ data: RawRemoteFileData[] }>(
-            "/files/data/fetch",
-            { type: "mldata", fileIDs: batch },
-        );
-
-        for (const entry of data ?? []) {
-            const key = fileKeys.get(entry.fileID);
-            if (!key) continue;
-            try {
-                const decrypted = decryptBlob(
-                    fromBase64(entry.encryptedData),
-                    fromBase64(entry.decryptionHeader),
-                    key,
-                );
-                const jsonStr = gunzipSync(Buffer.from(decrypted)).toString(
-                    "utf-8",
-                );
-                result.set(entry.fileID, JSON.parse(jsonStr));
-            } catch {
-                // Corrupted ML data for this file; skip it
-            }
-        }
-    }
-    return result;
-};
-
 // Extract the raw EXIF APP1 segment from JPEG bytes. Returns the EXIF
 // data buffer (starting after the APP1 length field, at the "Exif\0\0"
 // header) or undefined if no APP1 marker is found.
@@ -228,8 +183,8 @@ export const runMetadataBackup = async (
     }
 
     log("Fetching ML data (face detections, CLIP embeddings)...");
-    const mlDataMap = await fetchMLDataForFiles(
-        client,
+    const mlDataMap = await fetchMLData(
+        client.getApiClient(),
         [...fileKeys.keys()],
         fileKeys,
     );

src/mldata-fetch.ts

@@ -0,0 +1,93 @@
+// Fetch and decrypt Ente's per-file machine-learning data ("magic" search
+// data: face detections + CLIP embeddings).
+//
+// The data lives behind `/files/data/fetch` with `type: "mldata"`. Each entry
+// comes back encrypted under the file's own key and gzipped; decrypting and
+// gunzipping yields the JSON payload
+// `{ face: { faces: [...] }, clip: { embedding } }`. Ente caps a request at 200
+// ids, so `fetchMLData` batches for callers that want many at once while
+// `fetchMLDataBatch` is the single-request unit the library submits to its
+// request pool.
+
+import { gunzipSync } from "node:zlib";
+
+import type { ApiClient } from "./api/client.js";
+import { decryptBlob, fromBase64 } from "./crypto/index.js";
+
+// The most ids one `/files/data/fetch` request may carry.
+export const MLDATA_BATCH_SIZE = 200;
+
+// The decrypted, gunzipped per-file payload. Its concrete shape is Ente's; the
+// store keeps the whole object verbatim and each consumer reads the fields it
+// needs, so it stays an open record rather than a fixed interface.
+export type MLData = Record<string, unknown>;
+
+interface RawRemoteFileData {
+    fileID: number;
+    encryptedData: string;
+    decryptionHeader: string;
+    updatedAt?: number;
+}
+
+// Decrypt one entry with its file key and gunzip the JSON payload. Returns
+// undefined when the key is unknown or the entry does not decrypt/parse, so one
+// corrupt file never fails a whole batch.
+const decodeEntry = (
+    entry: RawRemoteFileData,
+    key: Uint8Array | undefined,
+): MLData | undefined => {
+    if (!key) return undefined;
+    try {
+        const decrypted = decryptBlob(
+            fromBase64(entry.encryptedData),
+            fromBase64(entry.decryptionHeader),
+            key,
+        );
+        const json = gunzipSync(Buffer.from(decrypted)).toString("utf-8");
+        return JSON.parse(json) as MLData;
+    } catch {
+        return undefined;
+    }
+};
+
+// Fetch ML data for up to `MLDATA_BATCH_SIZE` ids in a single request. This is
+// the unit the request pools schedule; callers with more ids split them into
+// batches and submit each batch to the pool.
+export const fetchMLDataBatch = async (
+    api: ApiClient,
+    fileIDs: number[],
+    fileKeys: Map<number, Uint8Array>,
+): Promise<Map<number, MLData>> => {
+    const { data } = await api.postJSON<{ data: RawRemoteFileData[] }>(
+        "/files/data/fetch",
+        { type: "mldata", fileIDs },
+    );
+    const result = new Map<number, MLData>();
+    for (const entry of data ?? []) {
+        const payload = decodeEntry(entry, fileKeys.get(entry.fileID));
+        if (payload) result.set(entry.fileID, payload);
+    }
+    return result;
+};
+
+// Fetch ML data for arbitrarily many ids, batching at `MLDATA_BATCH_SIZE`. Used
+// by the one-shot metadata backup; the library fetches through its request pool
+// with `fetchMLDataBatch` instead.
+export const fetchMLData = async (
+    api: ApiClient,
+    fileIDs: number[],
+    fileKeys: Map<number, Uint8Array>,
+): Promise<Map<number, MLData>> => {
+    const result = new Map<number, MLData>();
+    for (let i = 0; i < fileIDs.length; i += MLDATA_BATCH_SIZE) {
+        const batch = fileIDs.slice(i, i + MLDATA_BATCH_SIZE);
+        for (const [id, payload] of await fetchMLDataBatch(
+            api,
+            batch,
+            fileKeys,
+        )) {
+            result.set(id, payload);
+        }
+    }
+    return result;
+};

src/model/decrypt.ts

@@ -120,9 +120,16 @@ export const decryptFile = (
         metadata,
         magicMetadata,
         pubMagicMetadata,
-        file: { decryptionHeader: raw.file.decryptionHeader },
-        thumbnail: { decryptionHeader: raw.thumbnail.decryptionHeader },
+        file: {
+            decryptionHeader: raw.file.decryptionHeader,
+            size: raw.info?.fileSize,
+        },
+        thumbnail: {
+            decryptionHeader: raw.thumbnail.decryptionHeader,
+            size: raw.info?.thumbSize,
+        },
         updationTime: raw.updationTime,
+        isDeleted: raw.isDeleted,
     };
 };
 

src/model/types.ts

@@ -48,6 +48,9 @@ export interface EnteFile {
     file: FileBlob;
     thumbnail: FileBlob;
     updationTime: Microseconds;
+    // Set from the diff row's flag. Live files decode with it absent/false;
+    // deleted rows are filtered out before decryptFile, so it is not set here.
+    isDeleted?: boolean;
 }
 
 // The key material a logged-in client holds, everything needed to decrypt

test/client/enumeration.test.ts

@@ -0,0 +1,367 @@
+/**
+ * Tests for the resumable, deletion-aware enumeration variants on `Client`:
+ * `collectionsSince` and `filesSince`.
+ *
+ * The whole-account methods `listCollections` / `listFiles` always start at
+ * `sinceTime: 0` and hide deletions. The cache refresh needs the opposite:
+ * start from a saved cursor, learn what was deleted, and get back a cursor to
+ * resume from next time. These two methods provide that.
+ *
+ * The return shape keeps live records and tombstones apart — `collections` /
+ * `files` are decrypted live records, `deleted` is a plain list of the ids the
+ * server tombstoned. A tombstone carries no decryptable key or metadata, so it
+ * is a bare id rather than a hollowed-out `Collection` / `EnteFile`.
+ *
+ * All tests inject a fake `fetch` and drive a real `Client` (built with
+ * `Client.fromJSON`) so the decryption path runs for real. Live rows are built
+ * with libsodium exactly as the server would encrypt them; tombstone rows carry
+ * only the fields the code reads (`id`, `updationTime`, `isDeleted`), because
+ * they are never decrypted.
+ */
+
+import sodium from "libsodium-wrappers-sumo";
+import { beforeAll, describe, expect, it } from "vitest";
+import { init, toBase64 } from "../../src/crypto/index.js";
+import { Client, type ClientSnapshot } from "../../src/client.js";
+
+// ---------------------------------------------------------------------------
+// Fixtures
+// ---------------------------------------------------------------------------
+
+const USER_ID = 42;
+
+interface Keys {
+    masterKey: Uint8Array;
+    publicKey: Uint8Array;
+    secretKey: Uint8Array;
+}
+
+const buildKeys = (): Keys => {
+    const kp = sodium.crypto_box_keypair();
+    return {
+        masterKey: sodium.crypto_secretbox_keygen(),
+        publicKey: kp.publicKey,
+        secretKey: kp.privateKey,
+    };
+};
+
+const snapshotFor = (keys: Keys): ClientSnapshot => ({
+    email: "user@example.com",
+    userID: USER_ID,
+    token: "test-token",
+    masterKey: toBase64(keys.masterKey),
+    secretKey: toBase64(keys.secretKey),
+    publicKey: toBase64(keys.publicKey),
+});
+
+const secretboxEncrypt = (
+    plaintext: Uint8Array,
+    key: Uint8Array,
+): { ciphertext: Uint8Array; nonce: Uint8Array } => {
+    const nonce = sodium.randombytes_buf(sodium.crypto_secretbox_NONCEBYTES);
+    return {
+        ciphertext: sodium.crypto_secretbox_easy(plaintext, nonce, key),
+        nonce,
+    };
+};
+
+/** An owned collection row as the server sends it, keyed under the master key. */
+const ownedCollectionRow = (
+    masterKey: Uint8Array,
+    opts: { id: number; name: string; updationTime: number },
+): Record<string, unknown> => {
+    const collectionKey = sodium.crypto_secretbox_keygen();
+    const { ciphertext: encKey, nonce: keyNonce } = secretboxEncrypt(
+        collectionKey,
+        masterKey,
+    );
+    const { ciphertext: encName, nonce: nameNonce } = secretboxEncrypt(
+        new TextEncoder().encode(opts.name),
+        collectionKey,
+    );
+    return {
+        id: opts.id,
+        owner: { id: USER_ID },
+        encryptedKey: toBase64(encKey),
+        keyDecryptionNonce: toBase64(keyNonce),
+        encryptedName: toBase64(encName),
+        nameDecryptionNonce: toBase64(nameNonce),
+        type: "album",
+        updationTime: opts.updationTime,
+    };
+};
+
+/** A live file row inside a collection, keyed under that collection's key. */
+const fileRow = (
+    collectionKey: Uint8Array,
+    opts: { id: number; title: string; updationTime: number },
+): Record<string, unknown> => {
+    const fileKey = sodium.crypto_secretbox_keygen();
+    const { ciphertext: encFileKey, nonce: fileKeyNonce } = secretboxEncrypt(
+        fileKey,
+        collectionKey,
+    );
+    const metadata = {
+        title: opts.title,
+        fileType: 0,
+        creationTime: opts.updationTime,
+        modificationTime: opts.updationTime,
+    };
+    const push =
+        sodium.crypto_secretstream_xchacha20poly1305_init_push(fileKey);
+    const encMeta = sodium.crypto_secretstream_xchacha20poly1305_push(
+        push.state,
+        new TextEncoder().encode(JSON.stringify(metadata)),
+        null,
+        sodium.crypto_secretstream_xchacha20poly1305_TAG_FINAL,
+    );
+    return {
+        id: opts.id,
+        collectionID: 1,
+        ownerID: USER_ID,
+        encryptedKey: toBase64(encFileKey),
+        keyDecryptionNonce: toBase64(fileKeyNonce),
+        metadata: {
+            encryptedData: toBase64(encMeta),
+            decryptionHeader: toBase64(push.header),
+        },
+        file: { decryptionHeader: toBase64(sodium.randombytes_buf(24)) },
+        thumbnail: { decryptionHeader: toBase64(sodium.randombytes_buf(24)) },
+        updationTime: opts.updationTime,
+    };
+};
+
+/** A tombstone row. Never decrypted, so only these fields are ever read. */
+const tombstoneRow = (
+    id: number,
+    updationTime: number,
+): Record<string, unknown> => ({
+    id,
+    updationTime,
+    isDeleted: true,
+});
+
+const jsonResponse = (body: unknown): Response =>
+    new Response(JSON.stringify(body), {
+        status: 200,
+        headers: { "content-type": "application/json" },
+    });
+
+/**
+ * A fetch that serves canned responses in order and records the `sinceTime`
+ * query parameter each request carried, so tests can prove the cursor is
+ * threaded from one page (and one call) to the next.
+ */
+const recordingFetch = (
+    ...responses: Response[]
+): { fetch: typeof globalThis.fetch; sinceTimes: (string | null)[] } => {
+    const sinceTimes: (string | null)[] = [];
+    let i = 0;
+    const fake = async (input: RequestInfo | URL): Promise<Response> => {
+        const url =
+            typeof input === "string"
+                ? input
+                : input instanceof URL
+                  ? input.href
+                  : input.url;
+        sinceTimes.push(new URL(url).searchParams.get("sinceTime"));
+        if (i >= responses.length) {
+            throw new Error(`recordingFetch: no response for call #${i}`);
+        }
+        return responses[i++]!;
+    };
+    return { fetch: fake as typeof globalThis.fetch, sinceTimes };
+};
+
+// ---------------------------------------------------------------------------
+// Tests
+// ---------------------------------------------------------------------------
+
+describe("Client.filesSince", () => {
+    beforeAll(async () => {
+        await init();
+        await sodium.ready;
+    });
+
+    it("pages from the given cursor, decrypts live rows, and collects tombstones", async () => {
+        const keys = buildKeys();
+        const collectionKey = sodium.crypto_secretbox_keygen();
+
+        // Page 1 mixes a live file and a tombstone; the tombstone has the
+        // higher updationTime, so it — not the live row — sets the cursor the
+        // second page must be fetched from.
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({
+                diff: [
+                    fileRow(collectionKey, {
+                        id: 1001,
+                        title: "first.jpg",
+                        updationTime: 100,
+                    }),
+                    tombstoneRow(1002, 150),
+                ],
+                hasMore: true,
+            }),
+            jsonResponse({
+                diff: [
+                    fileRow(collectionKey, {
+                        id: 1003,
+                        title: "second.jpg",
+                        updationTime: 200,
+                    }),
+                ],
+                hasMore: false,
+            }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const { files, deleted, cursor } = await client.filesSince({
+            collectionID: 1,
+            collectionKey,
+            sinceTime: 0,
+        });
+
+        expect(files.map((f) => f.id)).toEqual([1001, 1003]);
+        expect(files.map((f) => f.metadata.title)).toEqual([
+            "first.jpg",
+            "second.jpg",
+        ]);
+        expect(deleted).toEqual([1002]);
+        expect(cursor).toBe(200);
+        // First request started at the caller's cursor; the second resumed
+        // from the max updationTime seen on the first page (the tombstone's).
+        expect(sinceTimes).toEqual(["0", "150"]);
+    });
+
+    it("fetches only newer rows when the returned cursor is passed back in", async () => {
+        const keys = buildKeys();
+        const collectionKey = sodium.crypto_secretbox_keygen();
+
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({ diff: [], hasMore: false }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const result = await client.filesSince({
+            collectionID: 1,
+            collectionKey,
+            sinceTime: 200,
+        });
+
+        expect(result.files).toEqual([]);
+        expect(result.deleted).toEqual([]);
+        // An empty diff advances nothing: the cursor falls back to the input.
+        expect(result.cursor).toBe(200);
+        expect(sinceTimes).toEqual(["200"]);
+    });
+
+    it("stops and throws when the server claims more but does not advance (#7)", async () => {
+        const keys = buildKeys();
+        const collectionKey = sodium.crypto_secretbox_keygen();
+
+        // hasMore is true, but the page's max updationTime (50) does not exceed
+        // the cursor the request was made with (50). Following hasMore here
+        // would refetch this same page forever.
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({ diff: [tombstoneRow(1, 50)], hasMore: true }),
+            jsonResponse({ diff: [tombstoneRow(1, 50)], hasMore: true }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        await expect(
+            client.filesSince({
+                collectionID: 1,
+                collectionKey,
+                sinceTime: 50,
+            }),
+        ).rejects.toThrow(/not advance|non-advancing/i);
+        // It gave up after the first page rather than looping.
+        expect(sinceTimes).toEqual(["50"]);
+    });
+});
+
+describe("Client.collectionsSince", () => {
+    beforeAll(async () => {
+        await init();
+        await sodium.ready;
+    });
+
+    it("decrypts live collections, collects tombstones, and returns a cursor", async () => {
+        const keys = buildKeys();
+
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({
+                collections: [
+                    ownedCollectionRow(keys.masterKey, {
+                        id: 1,
+                        name: "Vacation",
+                        updationTime: 100,
+                    }),
+                    tombstoneRow(3, 150),
+                ],
+            }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const { collections, deleted, cursor } = await client.collectionsSince({
+            sinceTime: 0,
+        });
+
+        expect(collections.map((c) => c.id)).toEqual([1]);
+        expect(collections[0]!.name).toBe("Vacation");
+        expect(deleted).toEqual([3]);
+        // The tombstone's updationTime advances the cursor too, so the next
+        // sync starts after it rather than seeing it again.
+        expect(cursor).toBe(150);
+        expect(sinceTimes).toEqual(["0"]);
+    });
+
+    it("falls back to the input cursor on an empty response", async () => {
+        const keys = buildKeys();
+
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({ collections: [] }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const result = await client.collectionsSince({ sinceTime: 150 });
+
+        expect(result.collections).toEqual([]);
+        expect(result.deleted).toEqual([]);
+        expect(result.cursor).toBe(150);
+        expect(sinceTimes).toEqual(["150"]);
+    });
+});
+
+describe("Client list wrappers still hide deletions", () => {
+    beforeAll(async () => {
+        await init();
+        await sodium.ready;
+    });
+
+    it("listFiles drops tombstones and returns only live files", async () => {
+        const keys = buildKeys();
+        const collectionKey = sodium.crypto_secretbox_keygen();
+
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({
+                diff: [
+                    fileRow(collectionKey, {
+                        id: 7,
+                        title: "keep.jpg",
+                        updationTime: 100,
+                    }),
+                    tombstoneRow(8, 150),
+                ],
+                hasMore: false,
+            }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const files = await client.listFiles(1, collectionKey);
+
+        expect(files.map((f) => f.id)).toEqual([7]);
+        // The wrapper starts a full enumeration from zero.
+        expect(sinceTimes).toEqual(["0"]);
+    });
+});

test/download/download.test.ts

@@ -72,7 +72,11 @@ import { init, toBase64, STREAM_CHUNK_SIZE } from "../../src/crypto/index.js";
 import { ApiClient } from "../../src/api/client.js";
 import { ApiError, TruncatedStreamError } from "../../src/errors.js";
 import type { RetryOptions } from "../../src/retry.js";
-import { downloadFile, downloadThumbnail } from "../../src/download/index.js";
+import {
+    downloadFile,
+    downloadThumbnail,
+    writeAtomic,
+} from "../../src/download/index.js";
 import type { EnteFile, FileMetadata } from "../../src/model/types.js";
 
 // ---------------------------------------------------------------------------
@@ -101,17 +105,79 @@ const renameHook = vi.hoisted(() => ({
     failWith: null as Error | null,
 }));
 
+/**
+ * `open` is wrapped so the tests can observe the durability fsyncs the atomic
+ * writer performs — which are otherwise invisible: an fsync leaves no trace in
+ * the file's contents. Each `FileHandle.sync()` is recorded, and rename and
+ * sync events are appended to a single ordered `events` log so a test can pin
+ * the sequence "fsync the temp file, rename, fsync the directory" that makes a
+ * write survive a power cut. The flag the handle was opened with distinguishes
+ * the temp file (`w`) from its containing directory (`r`).
+ */
+const durabilityHook = vi.hoisted(() => ({
+    events: [] as string[],
+}));
+
+/**
+ * `FileHandle.write` is wrapped so the tests can watch the streaming decrypt
+ * path put plaintext on disk one chunk at a time. This is the direct evidence
+ * that memory is bounded by the chunk size and not the file size: a buffered
+ * downloader would hand the whole file to a single write, whereas the streaming
+ * one issues one write per secretstream chunk, none larger than
+ * `STREAM_CHUNK_SIZE`. Each write records the temp path it targeted and its
+ * length. `writeFile` (which the whole-buffer `writeAtomic` uses) is a distinct
+ * native call and does not go through this method, so only the streaming path
+ * is observed here.
+ */
+const writeHook = vi.hoisted(() => ({
+    writes: [] as { path: string; length: number }[],
+}));
+
 vi.mock("node:fs/promises", async (importOriginal) => {
     const actual = await importOriginal<typeof import("node:fs/promises")>();
     const { existsSync: sourceExists } = await import("node:fs");
     return {
         ...actual,
+        open: async (
+            path: Parameters<typeof actual.open>[0],
+            flags?: Parameters<typeof actual.open>[1],
+            ...rest: unknown[]
+        ): Promise<Awaited<ReturnType<typeof actual.open>>> => {
+            const handle = await actual.open(
+                path,
+                flags as Parameters<typeof actual.open>[1],
+                ...(rest as []),
+            );
+            const realSync = handle.sync.bind(handle);
+            handle.sync = async (): Promise<void> => {
+                durabilityHook.events.push(`sync:${String(flags)}:${path}`);
+                await realSync();
+            };
+            const realWrite = handle.write.bind(handle);
+            handle.write = (async (
+                data: unknown,
+                ...rest2: unknown[]
+            ): Promise<unknown> => {
+                if (data instanceof Uint8Array) {
+                    writeHook.writes.push({
+                        path: String(path),
+                        length: data.length,
+                    });
+                }
+                return (realWrite as (...a: unknown[]) => Promise<unknown>)(
+                    data,
+                    ...rest2,
+                );
+            }) as typeof handle.write;
+            return handle;
+        },
         rename: async (from: string, to: string): Promise<void> => {
             renameHook.calls.push({
                 from,
                 to,
                 sourceExisted: sourceExists(from),
             });
+            durabilityHook.events.push(`rename:${to}`);
             if (renameHook.failWith !== null) {
                 throw renameHook.failWith;
             }
@@ -123,6 +189,8 @@ vi.mock("node:fs/promises", async (importOriginal) => {
 beforeEach(() => {
     renameHook.calls.length = 0;
     renameHook.failWith = null;
+    durabilityHook.events.length = 0;
+    writeHook.writes.length = 0;
 });
 
 let testDir: string;
@@ -911,10 +979,13 @@ describe.each(entryPoints)("$name retries", ({ name, download }) => {
     });
 
     it("stages one temp file for the attempt that succeeded, not one per attempt", async () => {
-        // The atomic write stays outside the retry loop. A retried download
-        // must not leave a trail of half-written scratch files, and the
-        // destination must be touched exactly once — by the attempt that
-        // produced a complete, authenticated plaintext.
+        // Each streaming attempt stages into its own temp file, but a retried
+        // download must not leave a trail of half-written scratch files: a
+        // failed attempt removes its temp file, and the destination is renamed
+        // into place exactly once — by the attempt that produced a complete,
+        // authenticated plaintext. (Here the two failed attempts reset before a
+        // whole chunk is pulled, so they write nothing; the point stands either
+        // way — see the retry-restart test below, where they do write.)
         const { key, header, ciphertext } = smallFixture(42);
         const { fetch } = scriptedCdnFetch(
             { kind: "reset", bytes: ciphertext.slice(0, 16) },
@@ -1028,6 +1099,99 @@ describe.each(entryPoints)("$name retries", ({ name, download }) => {
     });
 });
 
+// ---------------------------------------------------------------------------
+// Streaming decrypt to disk
+//
+// The plaintext is never held whole in memory: each secretstream chunk is
+// written to the temp file as it is decrypted, so peak memory is bounded by the
+// chunk size rather than the file size. These tests watch the writes directly
+// (see `writeHook`) rather than infer memory behaviour from the final file.
+// ---------------------------------------------------------------------------
+
+describe.each(entryPoints)("$name streams to disk", ({ name, download }) => {
+    const freshDir = (): string =>
+        mkdtempSync(join(testDir, `${name}-stream-`));
+
+    /** Writes recorded against staged temp files (not the `writeFile` path). */
+    const tempWrites = (): { path: string; length: number }[] =>
+        writeHook.writes.filter((w) => w.path.endsWith(".tmp"));
+
+    it("writes one chunk at a time, none larger than STREAM_CHUNK_SIZE", async () => {
+        // The multi-chunk fixture decrypts to one full 4 MiB chunk plus a small
+        // final chunk. A streaming writer therefore issues exactly two writes,
+        // of STREAM_CHUNK_SIZE and then the final chunk's length — never a
+        // single write carrying the whole 4 MiB + 1 KiB file. That per-chunk
+        // shape is what "memory bounded by chunk size" means in practice: the
+        // plaintext is handed to the filesystem and dropped, chunk by chunk.
+        const { api, file } = fixtureFor(
+            multiChunkKey,
+            multiChunk.header,
+            multiChunk.body,
+        );
+        const outPath = join(freshDir(), "streamed.bin");
+
+        const result = await download(api, file, outPath);
+
+        const writes = tempWrites();
+        expect(writes.map((w) => w.length)).toEqual([
+            STREAM_CHUNK_SIZE,
+            multiChunk.plaintext.length - STREAM_CHUNK_SIZE,
+        ]);
+        // No single write ever carried the whole file, and every write fits in
+        // one chunk's worth of memory.
+        for (const w of writes) {
+            expect(w.length).toBeLessThanOrEqual(STREAM_CHUNK_SIZE);
+        }
+        expect(result.bytesWritten).toBe(multiChunk.plaintext.length);
+        expectSameBytes(readFileSync(outPath), multiChunk.plaintext);
+    });
+
+    it("restarts from byte zero on a retry, replacing the temp file cleanly", async () => {
+        // The secretstream pull state is not resumable, so a retry cannot
+        // continue a half-written file — it must start over. The first attempt
+        // here delivers a complete leading chunk and then stops before the
+        // TAG_FINAL chunk: 4 MiB of plaintext lands in a temp file, then the
+        // download is rejected as truncated and that temp file is discarded.
+        // The retry streams the whole body into a *fresh* temp file, so the
+        // destination ends up with exactly the plaintext once — never the
+        // leading chunk twice, and never a stale temp file left behind.
+        const truncatedBody = multiChunk.body.slice(
+            0,
+            multiChunk.finalChunkOffset,
+        );
+        const { fetch, requests } = scriptedCdnFetch(
+            { kind: "body", bytes: truncatedBody },
+            { kind: "body", bytes: multiChunk.body },
+        );
+        const api = new ApiClient({
+            fetch,
+            retry: { ...noWait, attempts: 4 },
+        });
+        const file = buildMockEnteFile(
+            multiChunkKey,
+            multiChunk.header,
+            multiChunk.header,
+        );
+        const dir = freshDir();
+        const outPath = join(dir, "retry-restart.bin");
+
+        const result = await download(api, file, outPath);
+
+        expect(requests()).toBe(2);
+        // Both attempts streamed to disk, each into its own temp file: the
+        // truncated first attempt wrote before it failed, proving the retry did
+        // not resume a partial file but replaced it.
+        const distinctTemps = new Set(tempWrites().map((w) => w.path));
+        expect(distinctTemps.size).toBe(2);
+        // The destination holds the complete plaintext exactly once, and no
+        // temp file survives.
+        expect(result.bytesWritten).toBe(multiChunk.plaintext.length);
+        expectSameBytes(readFileSync(outPath), multiChunk.plaintext);
+        expect(renameHook.calls).toHaveLength(1);
+        expect(readdirSync(dir)).toEqual(["retry-restart.bin"]);
+    });
+});
+
 describe("download retries: corruption is not retried", () => {
     it("gives up immediately on a chunk that failed to authenticate", async () => {
         // A whole chunk that failed to authenticate while the stream
@@ -1061,3 +1225,182 @@ describe("download retries: corruption is not retried", () => {
         expect(requests()).toBe(1);
     });
 });
+
+// ---------------------------------------------------------------------------
+// Fragmented network reads
+//
+// A CDN does not hand the body over one secretstream chunk at a time; it
+// arrives in whatever pieces the socket produces, many of them far smaller than
+// a chunk and most straddling a chunk boundary. `streamDecrypt` reassembles
+// those pieces before decrypting, copying each received byte once rather than
+// recopying the whole accumulator on every read. This is the path the other
+// fixtures never take — their mock fetch delivers each body as a single
+// `Response` value, i.e. one read — so it is exercised explicitly here.
+// ---------------------------------------------------------------------------
+
+/**
+ * A fetch that serves `body` through a `ReadableStream` sliced into many
+ * fixed-size pieces, imitating a socket that trickles bytes in. `pieceSize` is
+ * chosen not to divide the chunk framing evenly, so pieces straddle the
+ * `ENC_CHUNK_SIZE` boundary the downloader splits on — the case a single-value
+ * body can never produce. `emitted` reports how many pieces were yielded, so a
+ * test can assert the body really was fragmented and not delivered whole.
+ */
+const mockFetchForFragmentedBody = (
+    body: Uint8Array,
+    pieceSize: number,
+): { fetch: typeof globalThis.fetch; emitted: () => number } => {
+    let pieces = 0;
+    const fake = async (): Promise<Response> =>
+        new Response(
+            new ReadableStream<Uint8Array>({
+                start(controller) {
+                    for (let off = 0; off < body.length; off += pieceSize) {
+                        controller.enqueue(body.subarray(off, off + pieceSize));
+                        pieces++;
+                    }
+                    controller.close();
+                },
+            }),
+            { status: 200 },
+        );
+    return { fetch: fake as typeof globalThis.fetch, emitted: () => pieces };
+};
+
+describe("streamDecrypt fragmented reads", () => {
+    it("decrypts a multi-chunk body delivered in many small pieces", async () => {
+        // The multi-chunk fixture (one full 4 MiB chunk plus a small final
+        // chunk) delivered in 1000-byte pieces: several thousand reads, with
+        // the piece that spans the 4 MiB + 17 byte chunk boundary split across
+        // two chunks by the reassembler. The plaintext must come out
+        // byte-identical to the single-read case, and the chunk framing must be
+        // untouched: exactly two writes, `STREAM_CHUNK_SIZE` then the final
+        // chunk, the same as when the body arrives whole. If the boundary
+        // handling were off by a byte under fragmentation, either the pull
+        // would fail to authenticate or the write sizes would shift.
+        const { fetch, emitted } = mockFetchForFragmentedBody(
+            multiChunk.body,
+            1000,
+        );
+        const api = new ApiClient({ fetch });
+        const file = buildMockEnteFile(
+            multiChunkKey,
+            multiChunk.header,
+            multiChunk.header,
+        );
+        const dir = mkdtempSync(join(testDir, "fragmented-"));
+        const outPath = join(dir, "fragmented.bin");
+
+        const result = await downloadFile(api, file, outPath);
+
+        // The body really was trickled in, not handed over whole.
+        expect(emitted()).toBeGreaterThan(1000);
+
+        const writes = writeHook.writes.filter((w) => w.path.endsWith(".tmp"));
+        expect(writes.map((w) => w.length)).toEqual([
+            STREAM_CHUNK_SIZE,
+            multiChunk.plaintext.length - STREAM_CHUNK_SIZE,
+        ]);
+        expect(result.bytesWritten).toBe(multiChunk.plaintext.length);
+        expectSameBytes(readFileSync(outPath), multiChunk.plaintext);
+    });
+});
+
+// ---------------------------------------------------------------------------
+// Durable atomic writes
+//
+// `writeAtomic` is exported so the metadata store can reuse the same
+// power-cut-safe write. Its durability is the point: the bytes and the new
+// directory entry must both be on stable storage before it returns, so a crash
+// immediately afterwards cannot resurrect an empty renamed file (#22 area 1).
+// ---------------------------------------------------------------------------
+
+describe("writeAtomic", () => {
+    it("fsyncs the temp file before the rename and the directory after", async () => {
+        const dir = mkdtempSync(join(testDir, "atomic-"));
+        const dest = join(dir, "durable.bin");
+        const bytes = patternBytes(2048, 71);
+
+        await writeAtomic(dest, bytes);
+
+        expect(readFileSync(dest)).toEqual(Buffer.from(bytes));
+        // The order is the durability contract: fsync the staged temp file so
+        // its contents are on disk, rename it into place, then fsync the
+        // directory so that new entry is on disk too. Do the directory fsync
+        // before the rename, or skip it, and a crash can lose the rename.
+        expect(durabilityHook.events).toHaveLength(3);
+        expect(durabilityHook.events[0]).toMatch(/^sync:w:.*\.tmp$/);
+        expect(durabilityHook.events[1]).toBe(`rename:${dest}`);
+        expect(durabilityHook.events[2]).toBe(`sync:r:${dir}`);
+    });
+
+    it("leaves no temp file behind when the write cannot be renamed", async () => {
+        const dir = mkdtempSync(join(testDir, "atomic-fail-"));
+        const dest = join(dir, "unrenamable.bin");
+        renameHook.failWith = new Error("simulated rename failure");
+
+        await expect(writeAtomic(dest, patternBytes(64, 72))).rejects.toThrow(
+            "simulated rename failure",
+        );
+
+        // The staged temp file was fsynced, then the rename failed; the cleanup
+        // path must remove it so a repeatedly failing write cannot fill the disk.
+        expect(existsSync(dest)).toBe(false);
+        expect(readdirSync(dir)).toEqual([]);
+    });
+});
+
+// ---------------------------------------------------------------------------
+// Per-chunk progress
+//
+// Callers streaming a large file want bytes-written as it lands, not only the
+// final total. The hook fires as decrypted plaintext accumulates; its values
+// are non-decreasing and its last value is exactly `bytesWritten`.
+// ---------------------------------------------------------------------------
+
+describe.each(entryPoints)("$name progress", ({ name, download }) => {
+    it("reports monotonic progress ending at bytesWritten", async () => {
+        // The multi-chunk fixture pulls one full 4 MiB chunk and then a small
+        // final chunk, so the callback fires more than once and monotonicity is
+        // actually observable rather than trivially true for a single fire.
+        const { api, file } = fixtureFor(
+            multiChunkKey,
+            multiChunk.header,
+            multiChunk.body,
+        );
+        const outPath = join(
+            mkdtempSync(join(testDir, `${name}-progress-`)),
+            "p.bin",
+        );
+        const seen: number[] = [];
+
+        const result = await download(api, file, outPath, (bytesDone) => {
+            seen.push(bytesDone);
+        });
+
+        expect(seen.length).toBeGreaterThan(1);
+        for (let i = 1; i < seen.length; i++) {
+            expect(seen[i]!).toBeGreaterThan(seen[i - 1]!);
+        }
+        expect(seen[seen.length - 1]).toBe(result.bytesWritten);
+        expect(result.bytesWritten).toBe(multiChunk.plaintext.length);
+    });
+
+    it("downloads normally when no progress callback is given", async () => {
+        // The callback is optional and its absence must be side-effect-free:
+        // the download succeeds exactly as it does elsewhere in this file.
+        const plaintext = patternBytes(300, 73);
+        const key = sodium.crypto_secretstream_xchacha20poly1305_keygen();
+        const { header, ciphertext } = encryptFileBody(plaintext, key);
+        const { api, file } = fixtureFor(key, header, ciphertext);
+        const outPath = join(
+            mkdtempSync(join(testDir, `${name}-noprog-`)),
+            "n.bin",
+        );
+
+        const result = await download(api, file, outPath);
+
+        expect(result.bytesWritten).toBe(plaintext.length);
+        expectSameBytes(readFileSync(outPath), plaintext);
+    });
+});

test/library/content-library.test.ts

@@ -0,0 +1,152 @@
+/**
+ * Integration between `Library` and the content cache (issue #46).
+ *
+ * The cache itself is covered in `content.test.ts`; this file locks the wiring:
+ * `Library.open` builds the cache from a content source, `lib.photos` hands out
+ * `Photo` objects that fetch through it, `lib.thumbnails.ensure` drives it, and
+ * a cached path shows up on the projected record. A library opened without a
+ * content source leaves those methods throwing rather than silently doing
+ * nothing.
+ */
+
+import { describe, it, expect, beforeEach, afterEach } from "vitest";
+import { mkdtempSync, rmSync, existsSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { Library } from "../../src/library/index.js";
+import type { ContentSource } from "../../src/library/content.js";
+import type { CollectionsPage, FilesPage } from "../../src/client.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+const USER_ID = 7;
+
+const collection = (id: number): Collection => ({
+    id,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    name: `album-${id}`,
+    type: "album",
+    updationTime: 1,
+    isShared: false,
+});
+
+const file = (id: number, collectionID: number): EnteFile => ({
+    id,
+    collectionID,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    metadata: {
+        title: `file-${id}.jpg`,
+        fileType: "image",
+        creationTime: 1,
+        modificationTime: 1,
+    },
+    file: { decryptionHeader: "aGVhZGVy" },
+    thumbnail: { decryptionHeader: "dGh1bWI=" },
+    updationTime: 1,
+});
+
+// A metadata-only client serving one album with one file, once.
+class MockClient {
+    served = false;
+    whoami(): { email: string; userID: number } {
+        return { email: "u@example.com", userID: USER_ID };
+    }
+    async collectionsSince(): Promise<CollectionsPage> {
+        if (this.served) return { collections: [], deleted: [], cursor: 1 };
+        this.served = true;
+        return { collections: [collection(1)], deleted: [], cursor: 1 };
+    }
+    async filesSince(): Promise<FilesPage> {
+        return { files: [file(1, 1)], deleted: [], cursor: 1 };
+    }
+}
+
+// A content source that writes a marker file and counts thumbnail fetches.
+const stubSource = (): ContentSource & { thumbCalls: () => number } => {
+    let thumbCalls = 0;
+    return {
+        thumbCalls: () => thumbCalls,
+        original: async ({ destination }) => {
+            writeFileSync(destination, "orig-bytes");
+            return { bytesWritten: 10 };
+        },
+        thumbnail: async ({ destination }) => {
+            thumbCalls++;
+            writeFileSync(destination, "thumb");
+            return { bytesWritten: 5 };
+        },
+    };
+};
+
+let root: string;
+
+beforeEach(() => {
+    root = mkdtempSync(join(tmpdir(), "quak-content-lib-"));
+});
+
+afterEach(() => {
+    if (root && existsSync(root))
+        rmSync(root, { recursive: true, force: true });
+});
+
+describe("Library content wiring", () => {
+    it("fetches a thumbnail through a Photo and records its cache path", async () => {
+        const source = stubSource();
+        const lib = await Library.open({
+            client: new MockClient(),
+            cacheDirectory: join(root, "cache"),
+            contentSource: source,
+            refreshIntervalSeconds: 3600,
+        });
+
+        const photo = lib.photos.byID({ fileID: 1 });
+        expect(photo).toBeDefined();
+        const result = await photo!.thumbnail();
+        expect(source.thumbCalls()).toBe(1);
+        expect(result.path).toBe(join(root, "cache", "thumbnails", "1.jpg"));
+        expect(existsSync(result.path)).toBe(true);
+
+        // The cached path is now on the projected record.
+        expect(lib.photos.byID({ fileID: 1 })!.record().thumbnailPath).toBe(
+            result.path,
+        );
+        lib.close();
+    });
+
+    it("drives thumbnails.ensure through the cache", async () => {
+        const source = stubSource();
+        const lib = await Library.open({
+            client: new MockClient(),
+            cacheDirectory: join(root, "cache"),
+            contentSource: source,
+            refreshIntervalSeconds: 3600,
+        });
+
+        const results = await lib.thumbnails.ensure({
+            fileIDs: [1],
+            priority: "visible",
+        });
+        expect(results).toEqual([
+            { fileID: 1, path: join(root, "cache", "thumbnails", "1.jpg") },
+        ]);
+        lib.close();
+    });
+
+    it("throws from content methods when opened without a content source", async () => {
+        const lib = await Library.open({
+            client: new MockClient(),
+            cacheDirectory: join(root, "cache"),
+            refreshIntervalSeconds: 3600,
+        });
+
+        await expect(
+            lib.photos.byID({ fileID: 1 })!.thumbnail(),
+        ).rejects.toThrow(/content cache/i);
+        await expect(
+            lib.thumbnails.ensure({ fileIDs: [1], priority: "visible" }),
+        ).rejects.toThrow(/content cache/i);
+        lib.close();
+    });
+});

test/library/content.test.ts

@@ -0,0 +1,423 @@
+/**
+ * Tests for the on-disk content and thumbnail cache (issue #46).
+ *
+ * The cache keys stored bytes by `fileID` under `cacheDirectory`:
+ * `originals/<fileID>.<ext>` and `thumbnails/<fileID>.<ext>`. Its contract:
+ *
+ * 1. **Fetch once, then serve from disk.** The first `original`/`thumbnail`
+ *    fetches through the request pool and stores the bytes; the next finds the
+ *    file present and returns its path with a single `skipped` event and no
+ *    network. A file already sitting in the backup `downloadDirectory` counts
+ *    as present too.
+ * 2. **Present-means-complete.** Content appears only by the streaming atomic
+ *    writer's rename, so a file that exists is whole. The directory listing
+ *    taken at `open()` is the record of what is cached, and the orphan temp
+ *    files a crashed write may have left are reaped there.
+ * 3. **`thumbnails.ensure` drives the thumbnail pool with priority, dedup, and
+ *    abort.** A `fileID` asked for twice downloads once; a visible request is
+ *    served ahead of a background one; and an `AbortSignal` drops work still
+ *    queued while letting an in-flight fetch finish.
+ *
+ * The `ContentSource` is a stand-in: it writes deterministic bytes to the
+ * destination and returns the count, so the cache logic is exercised with no
+ * crypto and no network. Ordering tests gate the stand-in on explicit deferreds
+ * and assert the persisted result, never a bare call or a timer.
+ */
+
+import { describe, it, expect, beforeEach, afterEach } from "vitest";
+import {
+    mkdtempSync,
+    rmSync,
+    existsSync,
+    writeFileSync,
+    mkdirSync,
+    statSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import {
+    ContentCache,
+    type ContentSource,
+    type EnsureEvent,
+} from "../../src/library/content.js";
+import { RequestPools } from "../../src/library/pools.js";
+import type { EnteFile } from "../../src/model/types.js";
+
+const file = (id: number, title = `file-${id}.jpg`): EnteFile => ({
+    id,
+    collectionID: 1,
+    ownerID: 1,
+    key: new Uint8Array([id & 0xff]),
+    metadata: {
+        title,
+        fileType: "image",
+        creationTime: 0,
+        modificationTime: 0,
+    },
+    file: { decryptionHeader: "aGVhZGVy" },
+    thumbnail: { decryptionHeader: "dGh1bWI=" },
+    updationTime: 0,
+});
+
+// A deferred with externally callable resolve, used to gate the stand-in source
+// so ordering is controlled by the test rather than by timing.
+const deferred = (): { promise: Promise<void>; resolve: () => void } => {
+    let resolve!: () => void;
+    const promise = new Promise<void>((r) => {
+        resolve = r;
+    });
+    return { promise, resolve };
+};
+
+// A ContentSource that writes `${kind}:${fileID}` bytes to the destination and
+// records every call. `gate` optionally blocks a call until released, and
+// `completed` records the order in which fetches finished — the observable used
+// by the priority and abort tests instead of a timer.
+class StubSource implements ContentSource {
+    originalCalls: number[] = [];
+    thumbnailCalls: number[] = [];
+    completed: number[] = [];
+    emptyFor = new Set<number>();
+    gates = new Map<number, Promise<void>>();
+
+    private async run(
+        kind: "original" | "thumbnail",
+        file: EnteFile,
+        destination: string,
+    ): Promise<{ bytesWritten: number }> {
+        const gate = this.gates.get(file.id);
+        if (gate) await gate;
+        const bytes = this.emptyFor.has(file.id)
+            ? new Uint8Array(0)
+            : new TextEncoder().encode(`${kind}:${file.id}`);
+        writeFileSync(destination, bytes);
+        this.completed.push(file.id);
+        return { bytesWritten: bytes.length };
+    }
+
+    async original(args: {
+        file: EnteFile;
+        destination: string;
+    }): Promise<{ bytesWritten: number }> {
+        this.originalCalls.push(args.file.id);
+        return this.run("original", args.file, args.destination);
+    }
+
+    async thumbnail(args: {
+        file: EnteFile;
+        destination: string;
+    }): Promise<{ bytesWritten: number }> {
+        this.thumbnailCalls.push(args.file.id);
+        return this.run("thumbnail", args.file, args.destination);
+    }
+}
+
+let root: string;
+let cacheDir: string;
+
+beforeEach(() => {
+    root = mkdtempSync(join(tmpdir(), "quak-content-"));
+    cacheDir = join(root, "cache");
+});
+
+afterEach(() => {
+    if (root && existsSync(root))
+        rmSync(root, { recursive: true, force: true });
+});
+
+const buildCache = (
+    args: {
+        source?: ContentSource;
+        files?: EnteFile[];
+        pools?: RequestPools;
+        downloadDirectory?: string;
+    } = {},
+): { cache: ContentCache; source: StubSource } => {
+    const source = (args.source as StubSource) ?? new StubSource();
+    const byID = new Map<number, EnteFile>();
+    for (const f of args.files ?? [file(1), file(2), file(3)])
+        byID.set(f.id, f);
+    const cache = new ContentCache({
+        pools: args.pools ?? new RequestPools(),
+        source,
+        cacheDirectory: cacheDir,
+        downloadDirectory: args.downloadDirectory,
+        getFile: (id) => byID.get(id),
+    });
+    return { cache, source };
+};
+
+describe("ContentCache.open", () => {
+    it("creates the cache directories with 0700 permissions", async () => {
+        const { cache } = buildCache();
+        await cache.open();
+
+        const originals = join(cacheDir, "originals");
+        const thumbnails = join(cacheDir, "thumbnails");
+        expect(existsSync(originals)).toBe(true);
+        expect(existsSync(thumbnails)).toBe(true);
+        expect(statSync(originals).mode & 0o777).toBe(0o700);
+        expect(statSync(thumbnails).mode & 0o777).toBe(0o700);
+    });
+
+    it("reaps orphan temp files but keeps complete content", async () => {
+        const originals = join(cacheDir, "originals");
+        const thumbnails = join(cacheDir, "thumbnails");
+        mkdirSync(originals, { recursive: true });
+        mkdirSync(thumbnails, { recursive: true });
+        const orphan = join(originals, ".quak-abc123.tmp");
+        const complete = join(originals, "1.jpg");
+        const thumb = join(thumbnails, "2.jpg");
+        writeFileSync(orphan, "half-written");
+        writeFileSync(complete, "whole");
+        writeFileSync(thumb, "whole-thumb");
+
+        const { cache } = buildCache();
+        await cache.open();
+
+        expect(existsSync(orphan)).toBe(false);
+        expect(existsSync(complete)).toBe(true);
+        expect(existsSync(thumb)).toBe(true);
+    });
+
+    it("records already-cached files so their paths appear in pathsFor", async () => {
+        const originals = join(cacheDir, "originals");
+        const thumbnails = join(cacheDir, "thumbnails");
+        mkdirSync(originals, { recursive: true });
+        mkdirSync(thumbnails, { recursive: true });
+        writeFileSync(join(originals, "1.jpg"), "orig");
+        writeFileSync(join(thumbnails, "1.jpg"), "thumb");
+
+        const { cache } = buildCache();
+        await cache.open();
+
+        expect(cache.pathsFor(1)).toEqual({
+            originalPath: join(originals, "1.jpg"),
+            thumbnailPath: join(thumbnails, "1.jpg"),
+        });
+        expect(cache.pathsFor(2)).toEqual({});
+    });
+});
+
+describe("ContentCache.original / thumbnail", () => {
+    it("fetches once, then serves the cached file with a single skipped event", async () => {
+        const { cache, source } = buildCache();
+        await cache.open();
+
+        const events: string[] = [];
+        const first = await cache.original(1, {
+            onProgress: (e) => events.push(e.status),
+        });
+        expect(source.originalCalls).toEqual([1]);
+        expect(first.path).toBe(join(cacheDir, "originals", "1.jpg"));
+        expect(first.bytes).toBe("original:1".length);
+        expect(existsSync(first.path)).toBe(true);
+        expect(statSync(first.path).mode & 0o777).toBe(0o600);
+        expect(cache.pathsFor(1).originalPath).toBe(first.path);
+
+        const skips: string[] = [];
+        const second = await cache.original(1, {
+            onProgress: (e) => skips.push(e.status),
+        });
+        // No second download, and exactly one skipped event.
+        expect(source.originalCalls).toEqual([1]);
+        expect(second.path).toBe(first.path);
+        expect(skips).toEqual(["skipped"]);
+    });
+
+    it("serves a file already present in the download directory without fetching", async () => {
+        const downloadDirectory = join(root, "backup");
+        mkdirSync(join(downloadDirectory, "originals"), { recursive: true });
+        const backupPath = join(downloadDirectory, "originals", "1.jpg");
+        writeFileSync(backupPath, "from-backup");
+
+        const { cache, source } = buildCache({ downloadDirectory });
+        await cache.open();
+
+        const events: EnsureEvent["status"][] = [];
+        const result = await cache.original(1, {
+            onProgress: (e) => events.push(e.status),
+        });
+
+        expect(source.originalCalls).toEqual([]);
+        expect(result.path).toBe(backupPath);
+        expect(result.bytes).toBe("from-backup".length);
+        expect(events).toEqual(["skipped"]);
+    });
+
+    it("fetches and caches a thumbnail", async () => {
+        const { cache, source } = buildCache();
+        await cache.open();
+
+        const result = await cache.thumbnail(2);
+        expect(source.thumbnailCalls).toEqual([2]);
+        expect(result.path).toBe(join(cacheDir, "thumbnails", "2.jpg"));
+        expect(existsSync(result.path)).toBe(true);
+        expect(cache.pathsFor(2).thumbnailPath).toBe(result.path);
+    });
+
+    it("shares one download between concurrent callers for the same file", async () => {
+        const { cache, source } = buildCache();
+        await cache.open();
+        const gate = deferred();
+        source.gates.set(1, gate.promise);
+
+        const a = cache.original(1);
+        const b = cache.original(1);
+        gate.resolve();
+        const [ra, rb] = await Promise.all([a, b]);
+
+        expect(source.originalCalls).toEqual([1]);
+        expect(ra.path).toBe(rb.path);
+    });
+
+    it("does not record a path when the fetched file is empty", async () => {
+        const { cache, source } = buildCache();
+        source.emptyFor.add(1);
+        await cache.open();
+
+        await expect(cache.original(1)).rejects.toThrow(/empty/i);
+        expect(cache.pathsFor(1).originalPath).toBeUndefined();
+    });
+
+    it("rejects an unknown file", async () => {
+        const { cache } = buildCache({ files: [] });
+        await cache.open();
+        await expect(cache.original(999)).rejects.toThrow(/unknown file/i);
+    });
+});
+
+describe("ContentCache.ensureThumbnails", () => {
+    it("downloads once for a file listed twice and reports every id", async () => {
+        const { cache, source } = buildCache();
+        await cache.open();
+
+        const results = await cache.ensureThumbnails({
+            fileIDs: [1, 1, 2],
+            priority: "visible",
+        });
+
+        expect(source.thumbnailCalls.sort()).toEqual([1, 2]);
+        expect(results).toEqual([
+            { fileID: 1, path: join(cacheDir, "thumbnails", "1.jpg") },
+            { fileID: 2, path: join(cacheDir, "thumbnails", "2.jpg") },
+        ]);
+    });
+
+    it("skips present files and reports a skipped event", async () => {
+        const thumbnails = join(cacheDir, "thumbnails");
+        mkdirSync(thumbnails, { recursive: true });
+        writeFileSync(join(thumbnails, "1.jpg"), "present");
+
+        const { cache, source } = buildCache();
+        await cache.open();
+
+        const events: EnsureEvent[] = [];
+        const results = await cache.ensureThumbnails({
+            fileIDs: [1, 2],
+            priority: "ahead",
+            onProgress: (e) => events.push(e),
+        });
+
+        expect(source.thumbnailCalls).toEqual([2]);
+        expect(results).toEqual([
+            { fileID: 1, path: join(thumbnails, "1.jpg") },
+            { fileID: 2, path: join(thumbnails, "2.jpg") },
+        ]);
+        expect(events).toContainEqual({
+            fileID: 1,
+            status: "skipped",
+            path: join(thumbnails, "1.jpg"),
+        });
+    });
+
+    it("serves a visible request ahead of an already-queued background one", async () => {
+        // One thumbnail slot, so exactly one fetch runs at a time and the rest
+        // wait in the pool. A background fetch takes the slot; a background and
+        // a visible fetch queue behind it. When the slot frees, the pool must
+        // pick the visible (on-demand) request ahead of the background one that
+        // was submitted first. The completion order is the observable.
+        const pools = new RequestPools({ thumbnailConcurrency: 1 });
+        const { cache, source } = buildCache({ pools });
+        await cache.open();
+
+        const gateA = deferred();
+        const gateB = deferred();
+        const gateC = deferred();
+        source.gates.set(1, gateA.promise);
+        source.gates.set(2, gateB.promise);
+        source.gates.set(3, gateC.promise);
+
+        const bgFirst = cache.ensureThumbnails({
+            fileIDs: [1],
+            priority: "background",
+        });
+        // Let fetch 1 take the only slot before the others queue.
+        await Promise.resolve();
+        const bgSecond = cache.ensureThumbnails({
+            fileIDs: [2],
+            priority: "background",
+        });
+        const visible = cache.ensureThumbnails({
+            fileIDs: [3],
+            priority: "visible",
+        });
+
+        gateA.resolve();
+        gateC.resolve();
+        gateB.resolve();
+        await Promise.all([bgFirst, bgSecond, visible]);
+
+        // 1 ran first (it held the slot). Of the two that were queued, the
+        // visible id 3 was served before the background id 2.
+        expect(source.completed).toEqual([1, 3, 2]);
+    });
+
+    it("drops queued work on abort but keeps an in-flight fetch", async () => {
+        const pools = new RequestPools({ thumbnailConcurrency: 1 });
+        const { cache, source } = buildCache({ pools });
+        await cache.open();
+
+        const gate = deferred();
+        source.gates.set(1, gate.promise);
+        const controller = new AbortController();
+
+        const pending = cache.ensureThumbnails({
+            fileIDs: [1, 2],
+            priority: "ahead",
+            signal: controller.signal,
+        });
+        // Fetch 1 is in flight (holds the slot); 2 is queued.
+        await Promise.resolve();
+        controller.abort();
+        gate.resolve();
+
+        const results = await pending;
+
+        // The in-flight fetch finished and is kept; the queued one was dropped
+        // before it ran.
+        expect(source.thumbnailCalls).toEqual([1]);
+        expect(results).toEqual([
+            { fileID: 1, path: join(cacheDir, "thumbnails", "1.jpg") },
+            { fileID: 2, error: "aborted" },
+        ]);
+    });
+
+    it("captures a per-file failure without failing the batch", async () => {
+        const { cache } = buildCache({ files: [file(1)] });
+        await cache.open();
+
+        const results = await cache.ensureThumbnails({
+            fileIDs: [1, 2],
+            priority: "background",
+        });
+
+        expect(results[0]).toEqual({
+            fileID: 1,
+            path: join(cacheDir, "thumbnails", "1.jpg"),
+        });
+        expect(results[1]?.fileID).toBe(2);
+        expect(results[1]?.error).toMatch(/unknown file/i);
+    });
+});

test/library/library.test.ts

@@ -0,0 +1,669 @@
+/**
+ * Tests for `Library.open()` and its transparent background refresh loop.
+ *
+ * The library keeps the account's server state in a `MetadataStore` (issue
+ * #41) and pulls changes with the resumable, tombstone-aware enumerators on
+ * `Client` (issue #38: `collectionsSince` / `filesSince`). `open()` loads the
+ * cache, does one refresh, then refreshes again every `refreshIntervalSeconds`
+ * on a background timer. The design (#36) forbids an exposed `sync()`, a
+ * `serverReachable` flag, a `lib.refresh()` method, and a "before each read"
+ * mode. The contracts exercised here:
+ *
+ * 1. Reads are answered from RAM. A read never calls the client.
+ * 2. `open()` does an initial refresh, then the interval keeps refreshing;
+ *    each refresh resumes from the stored cursor and applies diffs + tombstones.
+ * 3. The cache is rewritten only when a refresh actually changes something.
+ * 4. A failed refresh is invisible to reads: the last good data stays, the
+ *    failure surfaces via `onProgress` ("failed") and `status()`, and a later
+ *    success clears the error. `open()` itself resolves even when the first
+ *    refresh fails (offline start from cache).
+ * 5. `close()` stops the timer and is idempotent.
+ * 6. `cacheDirectory` defaults to the env-paths cache dir plus the user id.
+ * 7. `open()` branches on the cache: an empty cache awaits the first refresh
+ *    (it has nothing to serve yet); an existing cache serves its copy at once
+ *    and refreshes in the background, so a slow or dead server never stalls
+ *    opening.
+ * 8. A save failure that leaves RAM ahead of disk keeps `status().lastError`
+ *    set and keeps retrying the write; a later empty refresh does not clear it.
+ *
+ * The client is a mock: no crypto, no network. It serves scripted pages and
+ * records the `sinceTime` each call carried so cursor threading is provable.
+ *
+ * On an empty cache `open()` awaits the initial refresh (including its cache
+ * write), so state right after `open()` is deterministic; the tests that
+ * inspect post-`open()` state seed no cache and rely on that. Tests for an
+ * existing-cache open seed a store first and prove `open()` returns without
+ * waiting for the network. The interval tests then use real timers with a
+ * short interval and `vi.waitFor`: a fake clock cannot settle the real
+ * fsync-and-rename cache write, and empty diffs never write, so the eventual
+ * state is stable to poll for.
+ */
+
+import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import envPaths from "env-paths";
+
+import { Library, type RefreshEvent } from "../../src/library/index.js";
+import { MetadataStore } from "../../src/library/store.js";
+import type { CollectionsPage, FilesPage } from "../../src/client.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+const USER_ID = 42;
+
+// Short enough that a couple of ticks pass within a test, long enough not to
+// spin; interval tests poll for the eventual state rather than counting ticks.
+const FAST_INTERVAL = 0.02;
+
+const collection = (
+    id: number,
+    updationTime: number,
+    name = `album-${id}`,
+): Collection => ({
+    id,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    name,
+    type: "album",
+    updationTime,
+    isShared: false,
+});
+
+const file = (
+    id: number,
+    collectionID: number,
+    updationTime: number,
+): EnteFile => ({
+    id,
+    collectionID,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    metadata: {
+        title: `file-${id}.jpg`,
+        fileType: "image",
+        creationTime: updationTime,
+        modificationTime: updationTime,
+    },
+    file: { decryptionHeader: "aGVhZGVy" },
+    thumbnail: { decryptionHeader: "dGh1bWI=" },
+    updationTime,
+});
+
+/**
+ * A mock `Client`. `collectionsSince` shifts one page off `collectionsQueue`
+ * per call (an empty diff that advances nothing when the queue runs dry);
+ * `filesSince` shifts from a per-collection queue. `failCollections` makes the
+ * next and all further collection fetches throw, to simulate an offline server.
+ */
+class MockClient {
+    userID = USER_ID;
+    failCollections = false;
+    collectionsQueue: CollectionsPage[] = [];
+    filesByCollection = new Map<number, FilesPage[]>();
+
+    collectionsSinceTimes: number[] = [];
+    filesCalls: { collectionID: number; sinceTime: number }[] = [];
+
+    whoami(): { email: string; userID: number } {
+        return { email: "user@example.com", userID: this.userID };
+    }
+
+    async collectionsSince(args: {
+        sinceTime: number;
+    }): Promise<CollectionsPage> {
+        this.collectionsSinceTimes.push(args.sinceTime);
+        if (this.failCollections) throw new Error("network down");
+        return (
+            this.collectionsQueue.shift() ?? {
+                collections: [],
+                deleted: [],
+                cursor: args.sinceTime,
+            }
+        );
+    }
+
+    async filesSince(args: {
+        collectionID: number;
+        collectionKey: Uint8Array;
+        sinceTime: number;
+    }): Promise<FilesPage> {
+        this.filesCalls.push({
+            collectionID: args.collectionID,
+            sinceTime: args.sinceTime,
+        });
+        const queue = this.filesByCollection.get(args.collectionID);
+        return (
+            queue?.shift() ?? {
+                files: [],
+                deleted: [],
+                cursor: args.sinceTime,
+            }
+        );
+    }
+
+    filesFor(collectionID: number, ...pages: FilesPage[]): void {
+        this.filesByCollection.set(collectionID, pages);
+    }
+}
+
+describe("Library.open and background refresh", () => {
+    let dir: string;
+    let cacheDirectory: string;
+
+    beforeEach(() => {
+        dir = mkdtempSync(join(tmpdir(), "quak-library-"));
+        cacheDirectory = join(dir, "cache");
+    });
+
+    afterEach(() => {
+        rmSync(dir, { recursive: true, force: true });
+    });
+
+    it("does an initial refresh and answers reads from the cache", async () => {
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90), file(1002, 1, 95)],
+            deleted: [],
+            cursor: 95,
+        });
+
+        const lib = await Library.open({ client, cacheDirectory });
+        try {
+            expect(lib.listCollections().map((c) => c.id)).toEqual([1]);
+            expect(lib.listFiles(1).map((f) => f.id)).toEqual([1001, 1002]);
+            expect(lib.getFile(1, 1001)?.metadata.title).toBe("file-1001.jpg");
+
+            const status = lib.status();
+            expect(status.userID).toBe(USER_ID);
+            expect(status.collections).toBe(1);
+            expect(status.files).toBe(2);
+            expect(status.lastRefreshAt).toBeGreaterThan(0);
+            expect(status.lastError).toBeUndefined();
+
+            // The initial refresh persisted the cache to disk.
+            const reloaded = await MetadataStore.load(
+                join(cacheDirectory, "metadata.json"),
+            );
+            expect(reloaded.getFile(1, 1001)?.id).toBe(1001);
+            expect(reloaded.collectionsSinceTime).toBe(100);
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("reads never call the client", async () => {
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+
+        const lib = await Library.open({ client, cacheDirectory });
+        try {
+            const collectionCalls = client.collectionsSinceTimes.length;
+            const fileCalls = client.filesCalls.length;
+
+            lib.listCollections();
+            lib.getCollection(1);
+            lib.listFiles(1);
+            lib.getFile(1, 1001);
+            lib.status();
+
+            expect(client.collectionsSinceTimes.length).toBe(collectionCalls);
+            expect(client.filesCalls.length).toBe(fileCalls);
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("resumes each refresh from the stored cursor", async () => {
+        // Seed a cache with a cursor and a collection, as a prior run left it.
+        const path = join(cacheDirectory, "metadata.json");
+        const seed = await MetadataStore.load(path);
+        seed.userID = USER_ID;
+        seed.collectionsSinceTime = 500;
+        seed.putCollection(collection(1, 400));
+        seed.putFile(file(1001, 1, 400));
+        await seed.save();
+
+        const client = new MockClient();
+        // The collection's updationTime advances (400 -> 600), so its files are
+        // re-enumerated from the collection's stored updationTime (400).
+        client.collectionsQueue.push({
+            collections: [collection(1, 600)],
+            deleted: [],
+            cursor: 600,
+        });
+        client.filesFor(1, {
+            files: [file(1002, 1, 550)],
+            deleted: [],
+            cursor: 550,
+        });
+
+        // Opening from an existing cache serves the seeded copy at once and
+        // refreshes in the background, so the refresh's effects are polled for.
+        const lib = await Library.open({ client, cacheDirectory });
+        try {
+            await vi.waitFor(
+                () => {
+                    // Collections resumed from the stored cursor, and files were
+                    // re-enumerated from the stored collection updationTime.
+                    expect(client.collectionsSinceTimes[0]).toBe(500);
+                    expect(client.filesCalls).toEqual([
+                        { collectionID: 1, sinceTime: 400 },
+                    ]);
+                    expect(lib.listFiles(1).map((f) => f.id)).toEqual([
+                        1001, 1002,
+                    ]);
+                },
+                { timeout: 2000, interval: 5 },
+            );
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("does not re-enumerate a collection whose updationTime did not advance", async () => {
+        const path = join(cacheDirectory, "metadata.json");
+        const seed = await MetadataStore.load(path);
+        seed.userID = USER_ID;
+        seed.collectionsSinceTime = 100;
+        seed.putCollection(collection(1, 400));
+        await seed.save();
+
+        const client = new MockClient();
+        // The collection comes back in the diff (its metadata changed) but at
+        // the same updationTime, so its files must not be re-fetched.
+        client.collectionsQueue.push({
+            collections: [collection(1, 400, "renamed")],
+            deleted: [],
+            cursor: 400,
+        });
+
+        // Existing cache: the rename lands via the background refresh.
+        const lib = await Library.open({ client, cacheDirectory });
+        try {
+            await vi.waitFor(
+                () => expect(lib.getCollection(1)?.name).toBe("renamed"),
+                { timeout: 2000, interval: 5 },
+            );
+            // The collection's updationTime did not advance, so its files were
+            // never re-fetched.
+            expect(client.filesCalls).toEqual([]);
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("applies diffs and tombstones on the interval", async () => {
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100), collection(2, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+        client.filesFor(2, {
+            files: [file(2001, 2, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        try {
+            expect(lib.listCollections().map((c) => c.id)).toEqual([1, 2]);
+            expect(lib.listFiles(2).map((f) => f.id)).toEqual([2001]);
+
+            // Next refresh: collection 2 is tombstoned; collection 1 gains a
+            // file and loses its old one.
+            client.filesFor(1, {
+                files: [file(1002, 1, 190)],
+                deleted: [1001],
+                cursor: 190,
+            });
+            client.collectionsQueue.push({
+                collections: [collection(1, 200)],
+                deleted: [2],
+                cursor: 200,
+            });
+
+            await vi.waitFor(
+                () => {
+                    expect(lib.listCollections().map((c) => c.id)).toEqual([1]);
+                    expect(lib.listFiles(1).map((f) => f.id)).toEqual([1002]);
+                    // Collection 2's files went with it.
+                    expect(lib.listFiles(2)).toEqual([]);
+                },
+                { timeout: 2000, interval: 5 },
+            );
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("rewrites the cache only when a refresh changes something", async () => {
+        const saveSpy = vi.spyOn(MetadataStore.prototype, "save");
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        try {
+            // The initial refresh changed everything, so it saved once.
+            expect(saveSpy).toHaveBeenCalledTimes(1);
+
+            // Several empty-diff ticks pass; none of them may rewrite the file.
+            await new Promise((r) => setTimeout(r, FAST_INTERVAL * 1000 * 4));
+            expect(saveSpy).toHaveBeenCalledTimes(1);
+
+            // A real change triggers exactly one more rewrite; later empty ticks
+            // still do not, so the count settles at two.
+            client.collectionsQueue.push({
+                collections: [collection(2, 200)],
+                deleted: [],
+                cursor: 200,
+            });
+            await vi.waitFor(() => expect(saveSpy).toHaveBeenCalledTimes(2), {
+                timeout: 2000,
+                interval: 5,
+            });
+            await new Promise((r) => setTimeout(r, FAST_INTERVAL * 1000 * 4));
+            expect(saveSpy).toHaveBeenCalledTimes(2);
+        } finally {
+            lib.close();
+            saveSpy.mockRestore();
+        }
+    });
+
+    it("keeps a failed refresh invisible to reads and recovers later", async () => {
+        const events: RefreshEvent[] = [];
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+            onProgress: (e) => events.push(e),
+        });
+        try {
+            expect(lib.listFiles(1).map((f) => f.id)).toEqual([1001]);
+
+            // The server goes away; refreshes now fail.
+            client.failCollections = true;
+            await vi.waitFor(
+                () => expect(lib.status().lastError).toMatch(/network down/),
+                { timeout: 2000, interval: 5 },
+            );
+
+            // Reads still see the last good data; the failure was reported.
+            expect(lib.listFiles(1).map((f) => f.id)).toEqual([1001]);
+            expect(
+                events.some(
+                    (e) => e.operation === "refresh" && e.status === "failed",
+                ),
+            ).toBe(true);
+
+            // Recovery: a later refresh succeeds and clears the error.
+            client.failCollections = false;
+            client.collectionsQueue.push({
+                collections: [collection(2, 300)],
+                deleted: [],
+                cursor: 300,
+            });
+            await vi.waitFor(
+                () => {
+                    expect(lib.status().lastError).toBeUndefined();
+                    expect(lib.listCollections().map((c) => c.id)).toEqual([
+                        1, 2,
+                    ]);
+                },
+                { timeout: 2000, interval: 5 },
+            );
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("resolves open() even when the first refresh fails", async () => {
+        const client = new MockClient();
+        client.failCollections = true;
+        const events: RefreshEvent[] = [];
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            onProgress: (e) => events.push(e),
+        });
+        try {
+            // Nothing was cached and the server is unreachable: reads are empty,
+            // but the library opened and the failure is on record.
+            expect(lib.listCollections()).toEqual([]);
+            expect(lib.status().lastError).toMatch(/network down/);
+            expect(lib.status().lastRefreshAt).toBeUndefined();
+            expect(
+                events.some(
+                    (e) => e.operation === "refresh" && e.status === "failed",
+                ),
+            ).toBe(true);
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("opens from an existing cache without waiting for the first refresh", async () => {
+        // Seed a cache as a prior run left it.
+        const path = join(cacheDirectory, "metadata.json");
+        const seed = await MetadataStore.load(path);
+        seed.userID = USER_ID;
+        seed.collectionsSinceTime = 500;
+        seed.putCollection(collection(1, 400));
+        seed.putFile(file(1001, 1, 400));
+        await seed.save();
+
+        // The server never answers this run's first refresh.
+        const client = new MockClient();
+        client.collectionsSince = () => new Promise<CollectionsPage>(() => {});
+
+        // open() must resolve from the cache without blocking on the network,
+        // and reads must serve the seeded copy.
+        const lib = await Library.open({ client, cacheDirectory });
+        try {
+            expect(lib.listCollections().map((c) => c.id)).toEqual([1]);
+            expect(lib.listFiles(1).map((f) => f.id)).toEqual([1001]);
+            // The first refresh is still outstanding: nothing has completed or
+            // failed yet.
+            expect(lib.status().lastRefreshAt).toBeUndefined();
+            expect(lib.status().lastError).toBeUndefined();
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("awaits the first refresh on a first run with an empty cache", async () => {
+        // No cache on disk: open() must not resolve until the first fetch does,
+        // so it never hands back an empty library it could have filled.
+        let releaseFirstFetch: (page: CollectionsPage) => void = () => {};
+        const gate = new Promise<CollectionsPage>((resolve) => {
+            releaseFirstFetch = resolve;
+        });
+        const client = new MockClient();
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+        client.collectionsSince = async (args: { sinceTime: number }) => {
+            client.collectionsSinceTimes.push(args.sinceTime);
+            return gate;
+        };
+
+        let opened = false;
+        const openPromise = Library.open({ client, cacheDirectory }).then(
+            (l) => {
+                opened = true;
+                return l;
+            },
+        );
+
+        // While the first fetch is outstanding, open() has not resolved.
+        await new Promise((r) => setTimeout(r, 20));
+        expect(opened).toBe(false);
+
+        // Completing the fetch lets open() resolve with the data in place.
+        releaseFirstFetch({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        const lib = await openPromise;
+        try {
+            expect(opened).toBe(true);
+            expect(lib.listCollections().map((c) => c.id)).toEqual([1]);
+            expect(lib.listFiles(1).map((f) => f.id)).toEqual([1001]);
+            expect(lib.status().lastRefreshAt).toBeGreaterThan(0);
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("keeps a save failure visible until a save actually succeeds", async () => {
+        const saveSpy = vi
+            .spyOn(MetadataStore.prototype, "save")
+            .mockRejectedValue(new Error("disk full"));
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        try {
+            // The initial refresh mutated RAM but its save failed, so the error
+            // is on record and no refresh has counted as successful.
+            expect(lib.status().lastError).toMatch(/disk full/);
+            expect(lib.status().lastRefreshAt).toBeUndefined();
+
+            // Empty-diff ticks pass. Each still retries the unsaved write and
+            // still fails, so the error never silently clears and the refresh
+            // clock never advances — RAM must not run ahead of disk unnoticed.
+            const savesBefore = saveSpy.mock.calls.length;
+            await new Promise((r) => setTimeout(r, FAST_INTERVAL * 1000 * 4));
+            expect(saveSpy.mock.calls.length).toBeGreaterThan(savesBefore);
+            expect(lib.status().lastError).toMatch(/disk full/);
+            expect(lib.status().lastRefreshAt).toBeUndefined();
+
+            // Once the disk recovers, the next tick persists the pending change
+            // and only then clears the error and advances the clock.
+            saveSpy.mockRestore();
+            await vi.waitFor(
+                () => {
+                    expect(lib.status().lastError).toBeUndefined();
+                    expect(lib.status().lastRefreshAt).toBeGreaterThan(0);
+                },
+                { timeout: 2000, interval: 5 },
+            );
+            const reloaded = await MetadataStore.load(
+                join(cacheDirectory, "metadata.json"),
+            );
+            expect(reloaded.getFile(1, 1001)?.id).toBe(1001);
+        } finally {
+            lib.close();
+            saveSpy.mockRestore();
+        }
+    });
+
+    it("close() stops the timer and is idempotent", async () => {
+        const client = new MockClient();
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        const callsAfterOpen = client.collectionsSinceTimes.length;
+
+        lib.close();
+        lib.close(); // second close must not throw
+        expect(lib.status().closed).toBe(true);
+
+        // No further refreshes fire once closed.
+        await new Promise((r) => setTimeout(r, FAST_INTERVAL * 1000 * 5));
+        expect(client.collectionsSinceTimes.length).toBe(callsAfterOpen);
+    });
+
+    it("defaults cacheDirectory to the env-paths cache dir plus user id", async () => {
+        const xdg = join(dir, "xdg-cache");
+        const prev = process.env.XDG_CACHE_HOME;
+        process.env.XDG_CACHE_HOME = xdg;
+        try {
+            const client = new MockClient();
+            const lib = await Library.open({ client });
+            try {
+                const expected = join(
+                    envPaths("quak", { suffix: "" }).cache,
+                    String(USER_ID),
+                );
+                expect(lib.cacheDirectory).toBe(expected);
+                expect(lib.cacheDirectory.startsWith(xdg)).toBe(true);
+                expect(lib.cacheDirectory.endsWith(String(USER_ID))).toBe(true);
+            } finally {
+                lib.close();
+            }
+        } finally {
+            if (prev === undefined) delete process.env.XDG_CACHE_HOME;
+            else process.env.XDG_CACHE_HOME = prev;
+        }
+    });
+});

test/library/mldata.test.ts

@@ -0,0 +1,449 @@
+/**
+ * Tests for the ML-data cache and its derived CLIP index (issue #49).
+ *
+ * Two layers are exercised:
+ *
+ *   1. `MLDataStore` on its own: storing one payload file per fileID (present
+ *      means complete), building a `clip.f32` + `clip.json` index that reloads
+ *      in a single read, rebuilding that index from the payloads when it is
+ *      missing or disagrees with the files present, appending as new payloads
+ *      arrive, overwriting a refetched file in place, and deciding what to
+ *      (re)fetch as `updationTime` advances.
+ *
+ *   2. `Library` wiring: after each refresh the library fetches ML data through
+ *      the metadata pool for every known file not yet cached, is incremental on
+ *      later refreshes, and refetches a file whose `updationTime` advanced.
+ *
+ * Embedding values are chosen to be exactly representable as float32 so the
+ * round-trip through `clip.f32` compares equal.
+ */
+
+import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
+import { existsSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { MLDataStore } from "../../src/library/mldata.js";
+import { Library } from "../../src/library/index.js";
+import type { CollectionsPage, FilesPage } from "../../src/client.js";
+import type { MLData } from "../../src/mldata-fetch.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+// A payload shaped like Ente's: a CLIP embedding plus face data that only the
+// on-disk payload carries (never the RAM index).
+const payload = (embedding: number[]): MLData => ({
+    face: {
+        faces: [{ faceID: "f", detection: { box: { x: 0.5 } } }],
+    },
+    clip: { embedding },
+});
+
+describe("MLDataStore", () => {
+    let dir: string;
+
+    beforeEach(() => {
+        dir = mkdtempSync(join(tmpdir(), "quak-mldata-"));
+    });
+
+    afterEach(() => {
+        rmSync(dir, { recursive: true, force: true });
+    });
+
+    it("stores one payload file per fileID and builds a one-read index", async () => {
+        const store = await MLDataStore.open(dir);
+        const res = await store.storeFetched(
+            new Map([
+                [100, payload([0.5, 0.25, 0.75])],
+                [200, payload([1, -2, 0.5])],
+            ]),
+            new Map([
+                [100, 10],
+                [200, 20],
+            ]),
+        );
+        expect(res).toEqual({ stored: 2, indexed: 2 });
+
+        // One payload file per fileID, and the derived index files.
+        expect(existsSync(join(dir, "100.json"))).toBe(true);
+        expect(existsSync(join(dir, "200.json"))).toBe(true);
+        expect(existsSync(join(dir, "clip.f32"))).toBe(true);
+        expect(existsSync(join(dir, "clip.json"))).toBe(true);
+
+        // Reopening loads the index from disk in one read.
+        const reopened = await MLDataStore.open(dir);
+        const index = reopened.getIndex();
+        expect(index.fileIDs).toEqual([100, 200]);
+        expect(index.embeddingLength).toBe(3);
+        expect([...index.embeddings]).toEqual([0.5, 0.25, 0.75, 1, -2, 0.5]);
+
+        // The full payload (face boxes) is read back from disk on demand.
+        const full = await reopened.readPayload(100);
+        expect(full?.face).toBeDefined();
+        expect(await reopened.readPayload(999)).toBeUndefined();
+    });
+
+    it("rebuilds the index from payloads when it is missing", async () => {
+        const store = await MLDataStore.open(dir);
+        await store.storeFetched(
+            new Map([[100, payload([0.5, 0.25, 0.75])]]),
+            new Map([[100, 10]]),
+        );
+
+        // The derived index is lost but the payloads survive.
+        rmSync(join(dir, "clip.f32"));
+        rmSync(join(dir, "clip.json"));
+
+        const reopened = await MLDataStore.open(dir);
+        const index = reopened.getIndex();
+        expect(index.fileIDs).toEqual([100]);
+        expect([...index.embeddings]).toEqual([0.5, 0.25, 0.75]);
+        expect(existsSync(join(dir, "clip.f32"))).toBe(true);
+    });
+
+    it("rebuilds the index when it disagrees with the files present", async () => {
+        const store = await MLDataStore.open(dir);
+        await store.storeFetched(
+            new Map([
+                [100, payload([0.5, 0.25, 0.75])],
+                [200, payload([1, -2, 0.5])],
+            ]),
+            new Map([
+                [100, 10],
+                [200, 20],
+            ]),
+        );
+
+        // A payload disappears out from under the index (leaving it referencing
+        // a file no longer present); the index must be rebuilt from what is
+        // actually on disk.
+        rmSync(join(dir, "200.json"));
+
+        const reopened = await MLDataStore.open(dir);
+        expect(reopened.getIndex().fileIDs).toEqual([100]);
+    });
+
+    it("rebuilds the index when a payload on disk is missing from it", async () => {
+        const store = await MLDataStore.open(dir);
+        await store.storeFetched(
+            new Map([[100, payload([0.5, 0.25, 0.75])]]),
+            new Map([[100, 10]]),
+        );
+
+        // A crash between storeFetched renaming a payload into place and
+        // rewriting the index leaves the payload complete on disk but absent
+        // from clip.json. Write a second payload directly to reproduce that
+        // torn state without touching the index.
+        writeFileSync(
+            join(dir, "200.json"),
+            JSON.stringify(payload([1, -2, 0.5])),
+        );
+
+        // Reopening self-heals with no manual delete: the index is rebuilt from
+        // the payloads to include the orphaned embedding.
+        const reopened = await MLDataStore.open(dir);
+        const index = reopened.getIndex();
+        expect(index.fileIDs).toEqual([100, 200]);
+        expect([...index.embeddings]).toEqual([0.5, 0.25, 0.75, 1, -2, 0.5]);
+    });
+
+    it("appends new payloads and overwrites a refetched file in place", async () => {
+        const store = await MLDataStore.open(dir);
+        await store.storeFetched(
+            new Map([[100, payload([0.5, 0.25, 0.75])]]),
+            new Map([[100, 10]]),
+        );
+        // A later batch adds a new file: appended after the first.
+        await store.storeFetched(
+            new Map([[200, payload([1, -2, 0.5])]]),
+            new Map([[200, 20]]),
+        );
+        // Refetching 100 (its embedding changed) updates it in place, not a
+        // duplicate row.
+        await store.storeFetched(
+            new Map([[100, payload([9, 9, 9])]]),
+            new Map([[100, 30]]),
+        );
+
+        const index = store.getIndex();
+        expect(index.fileIDs).toEqual([100, 200]);
+        expect([...index.embeddings]).toEqual([9, 9, 9, 1, -2, 0.5]);
+    });
+
+    it("keeps a payload without a CLIP embedding out of the index", async () => {
+        const store = await MLDataStore.open(dir);
+        const res = await store.storeFetched(
+            new Map<number, MLData>([[100, { face: { faces: [] } }]]),
+            new Map([[100, 10]]),
+        );
+        expect(res.stored).toBe(1);
+        expect(res.indexed).toBe(0);
+        // The payload is still cached (present means complete).
+        expect(existsSync(join(dir, "100.json"))).toBe(true);
+        expect(store.getIndex().fileIDs).toEqual([]);
+    });
+
+    it("fetches only what is missing or has a newer updationTime", async () => {
+        const store = await MLDataStore.open(dir);
+        await store.storeFetched(
+            new Map([[100, payload([0.5, 0.25, 0.75])]]),
+            new Map([[100, 10]]),
+        );
+
+        // 100 is cached and current; 200 has never been fetched.
+        expect(
+            store.neededFor([
+                { id: 100, updationTime: 10 },
+                { id: 200, updationTime: 5 },
+            ]),
+        ).toEqual([200]);
+
+        // 100's updationTime advanced past what it was fetched at: refetch.
+        expect(store.neededFor([{ id: 100, updationTime: 15 }])).toEqual([100]);
+
+        // Nothing advanced: nothing to fetch.
+        expect(store.neededFor([{ id: 100, updationTime: 10 }])).toEqual([]);
+    });
+
+    it("survives a corrupt index without losing the payloads", async () => {
+        const store = await MLDataStore.open(dir);
+        await store.storeFetched(
+            new Map([[100, payload([0.5, 0.25, 0.75])]]),
+            new Map([[100, 10]]),
+        );
+        writeFileSync(join(dir, "clip.json"), "not json");
+
+        const reopened = await MLDataStore.open(dir);
+        expect(reopened.getIndex().fileIDs).toEqual([100]);
+    });
+});
+
+// --- Library wiring ---------------------------------------------------------
+
+const USER_ID = 42;
+const FAST_INTERVAL = 0.02;
+
+const collection = (id: number, updationTime: number): Collection => ({
+    id,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    name: `album-${id}`,
+    type: "album",
+    updationTime,
+    isShared: false,
+});
+
+const file = (
+    id: number,
+    collectionID: number,
+    updationTime: number,
+): EnteFile => ({
+    id,
+    collectionID,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    metadata: {
+        title: `file-${id}.jpg`,
+        fileType: "image",
+        creationTime: updationTime,
+        modificationTime: updationTime,
+    },
+    file: { decryptionHeader: "aGVhZGVy" },
+    thumbnail: { decryptionHeader: "dGh1bWI=" },
+    updationTime,
+});
+
+// A mock client that serves scripted collection/file pages and per-file ML
+// payloads, recording every ML fetch request so incremental behaviour is
+// provable.
+class MLMockClient {
+    userID = USER_ID;
+    collectionsQueue: CollectionsPage[] = [];
+    filesByCollection = new Map<number, FilesPage[]>();
+    mlByFile = new Map<number, MLData>();
+    mlFetchCalls: number[][] = [];
+
+    whoami(): { email: string; userID: number } {
+        return { email: "user@example.com", userID: this.userID };
+    }
+
+    async collectionsSince(args: {
+        sinceTime: number;
+    }): Promise<CollectionsPage> {
+        return (
+            this.collectionsQueue.shift() ?? {
+                collections: [],
+                deleted: [],
+                cursor: args.sinceTime,
+            }
+        );
+    }
+
+    async filesSince(args: {
+        collectionID: number;
+        collectionKey: Uint8Array;
+        sinceTime: number;
+    }): Promise<FilesPage> {
+        const queue = this.filesByCollection.get(args.collectionID);
+        return (
+            queue?.shift() ?? {
+                files: [],
+                deleted: [],
+                cursor: args.sinceTime,
+            }
+        );
+    }
+
+    async fetchMLData(args: {
+        fileIDs: number[];
+        fileKeys: Map<number, Uint8Array>;
+    }): Promise<Map<number, MLData>> {
+        this.mlFetchCalls.push([...args.fileIDs]);
+        const result = new Map<number, MLData>();
+        for (const id of args.fileIDs) {
+            const p = this.mlByFile.get(id);
+            if (p) result.set(id, p);
+        }
+        return result;
+    }
+
+    filesFor(collectionID: number, ...pages: FilesPage[]): void {
+        this.filesByCollection.set(collectionID, pages);
+    }
+}
+
+describe("Library ML-data fetch on refresh", () => {
+    let dir: string;
+    let cacheDirectory: string;
+
+    beforeEach(() => {
+        dir = mkdtempSync(join(tmpdir(), "quak-lib-mldata-"));
+        cacheDirectory = join(dir, "cache");
+    });
+
+    afterEach(() => {
+        rmSync(dir, { recursive: true, force: true });
+    });
+
+    it("fetches, stores and indexes ML data for known files, then is incremental", async () => {
+        const client = new MLMockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90), file(1002, 1, 95)],
+            deleted: [],
+            cursor: 95,
+        });
+        client.mlByFile.set(1001, payload([0.5, 0.25, 0.75]));
+        client.mlByFile.set(1002, payload([1, -2, 0.5]));
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        try {
+            // Wait on `lastMLFetchAt`, set only once the pass has persisted the
+            // index and payloads — not on the in-RAM counts, which advance
+            // before `storeFetched` writes to disk, so the reopen below reads
+            // the committed index rather than racing the write.
+            await vi.waitFor(
+                () => {
+                    expect(lib.status().lastMLFetchAt).toBeGreaterThan(0);
+                    expect(lib.status().mlIndexed).toBe(2);
+                    expect(lib.status().mlStored).toBe(2);
+                },
+                { timeout: 2000, interval: 5 },
+            );
+
+            // Both files were fetched, in one batch.
+            expect(client.mlFetchCalls.flat().sort((a, b) => a - b)).toEqual([
+                1001, 1002,
+            ]);
+            const callsAfterFirst = client.mlFetchCalls.length;
+
+            // The index is on disk and reloads to the same shape.
+            const reopened = await MLDataStore.open(
+                join(cacheDirectory, "mldata"),
+            );
+            expect(reopened.getIndex().fileIDs).toEqual([1001, 1002]);
+
+            // Later refreshes with nothing new must not refetch.
+            await new Promise((r) => setTimeout(r, FAST_INTERVAL * 1000 * 5));
+            expect(client.mlFetchCalls.length).toBe(callsAfterFirst);
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("refetches a file whose updationTime advanced", async () => {
+        const client = new MLMockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+        client.mlByFile.set(1001, payload([0.5, 0.25, 0.75]));
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        try {
+            // Wait on `lastMLFetchAt`, set only after the first pass has
+            // persisted, not on `mlIndexed`, which is bumped in RAM before the
+            // write lands.
+            await vi.waitFor(
+                () => expect(lib.status().lastMLFetchAt).toBeGreaterThan(0),
+                { timeout: 2000, interval: 5 },
+            );
+            const callsBefore = client.mlFetchCalls.length;
+
+            // The file changes on the server (updationTime advances) with a new
+            // embedding; the next refresh must refetch it.
+            client.mlByFile.set(1001, payload([9, 9, 9]));
+            client.collectionsQueue.push({
+                collections: [collection(1, 200)],
+                deleted: [],
+                cursor: 200,
+            });
+            client.filesFor(1, {
+                files: [file(1001, 1, 190)],
+                deleted: [],
+                cursor: 190,
+            });
+
+            // Poll the persisted index itself, not the fetch-call log: a call
+            // is recorded the instant the mock is entered, but `storeFetched`
+            // rewrites `clip.f32` only after it resolves, so an earlier reopen
+            // would read the pre-refetch vector. Reopening reads only committed
+            // (atomically renamed) files, so this sees the new embedding once —
+            // and only once — the store has written it.
+            await vi.waitFor(
+                async () => {
+                    const reopened = await MLDataStore.open(
+                        join(cacheDirectory, "mldata"),
+                    );
+                    expect([...reopened.getIndex().embeddings]).toEqual([
+                        9, 9, 9,
+                    ]);
+                },
+                { timeout: 2000, interval: 20 },
+            );
+
+            // The refetch really went back to the server for 1001.
+            expect(client.mlFetchCalls.length).toBeGreaterThan(callsBefore);
+            expect(client.mlFetchCalls.flat()).toContain(1001);
+        } finally {
+            lib.close();
+        }
+    });
+});

test/library/pools.test.ts

@@ -0,0 +1,345 @@
+/**
+ * Tests for `src/library/pools.ts` — the three bounded request pools (issue
+ * #45).
+ *
+ * A `BoundedPool` runs submitted tasks with a fixed concurrency cap. Within a
+ * pool, on-demand work runs before background work, and a task submitted with a
+ * key that a still-pending task already carries is not run twice — both callers
+ * share the one result. `RequestPools` bundles the three the design calls for
+ * (metadata 10, content 5, thumbnails 25); the pools are independent, so an
+ * idle pool never lends its slots to a busy one.
+ *
+ * ## How the tasks are controlled
+ *
+ * Every task here is a gate: it reports when it *starts* and then blocks until
+ * the test *releases* it, so the test decides exactly how many run at once and
+ * in what order they finish. A shared tracker counts how many tasks are running
+ * at any instant and records the peak, which is what the concurrency assertions
+ * read. No assertion is about wall-clock time.
+ *
+ * `drain()` returns a promise that settles on a macrotask, which flushes the
+ * microtask queue the pool schedules its starts on; the tests await it to let
+ * the pool react to a submission or a release before they inspect it.
+ */
+
+import { describe, it, expect } from "vitest";
+
+import {
+    BoundedPool,
+    RequestPools,
+    DEFAULT_METADATA_CONCURRENCY,
+    DEFAULT_CONTENT_CONCURRENCY,
+    DEFAULT_THUMBNAIL_CONCURRENCY,
+} from "../../src/library/pools.js";
+
+// Settle on a macrotask so every microtask the pool queued has run.
+const drain = (): Promise<void> =>
+    new Promise((resolve) => setTimeout(resolve, 0));
+
+// A controllable task. `task` blocks until `release()` (resolve) or `fail()`
+// (reject) is called; `startOrder` records the sequence in which tasks began.
+interface Gate<T> {
+    task: () => Promise<T>;
+    release: (value: T) => void;
+    fail: (err: unknown) => void;
+    started: () => boolean;
+    runs: () => number;
+}
+
+// Tracks how many gated tasks are running concurrently across a whole test.
+class Tracker {
+    active = 0;
+    peak = 0;
+    readonly starts: string[] = [];
+
+    gate<T>(label = ""): Gate<T> {
+        let settleResolve!: (value: T) => void;
+        let settleReject!: (err: unknown) => void;
+        const settled = new Promise<T>((resolve, reject) => {
+            settleResolve = resolve;
+            settleReject = reject;
+        });
+        let started = false;
+        let runs = 0;
+        const task = async (): Promise<T> => {
+            started = true;
+            runs++;
+            this.active++;
+            this.peak = Math.max(this.peak, this.active);
+            this.starts.push(label);
+            try {
+                return await settled;
+            } finally {
+                this.active--;
+            }
+        };
+        return {
+            task,
+            release: (value: T) => settleResolve(value),
+            fail: (err: unknown) => settleReject(err),
+            started: () => started,
+            runs: () => runs,
+        };
+    }
+}
+
+describe("BoundedPool concurrency cap", () => {
+    it("never runs more than `concurrency` tasks at once", async () => {
+        const pool = new BoundedPool(3);
+        const t = new Tracker();
+        const gates = Array.from({ length: 5 }, () => t.gate<void>());
+
+        const done = gates.map((g) => pool.run(g.task));
+        await drain();
+
+        // Three started, two queued behind the cap.
+        expect(t.active).toBe(3);
+        expect(gates.slice(0, 3).every((g) => g.started())).toBe(true);
+        expect(gates.slice(3).some((g) => g.started())).toBe(false);
+
+        // Finishing one admits exactly one more; the cap holds.
+        gates[0]!.release();
+        await drain();
+        expect(t.active).toBe(3);
+        expect(gates[3]!.started()).toBe(true);
+        expect(gates[4]!.started()).toBe(false);
+
+        for (const g of gates.slice(1)) g.release();
+        await Promise.all(done);
+        expect(t.peak).toBe(3);
+    });
+
+    it("rejects a non-positive or non-integer concurrency", () => {
+        expect(() => new BoundedPool(0)).toThrow(RangeError);
+        expect(() => new BoundedPool(-1)).toThrow(RangeError);
+        expect(() => new BoundedPool(2.5)).toThrow(RangeError);
+    });
+});
+
+describe("BoundedPool priority ordering", () => {
+    it("runs on-demand work before background, FIFO within a priority", async () => {
+        const pool = new BoundedPool(1);
+        const t = new Tracker();
+        const a = t.gate<void>("a");
+        const b = t.gate<void>("b");
+        const c = t.gate<void>("c");
+        const d = t.gate<void>("d");
+
+        // `a` takes the only slot; the rest queue.
+        void pool.run(a.task, { priority: "background" });
+        await drain();
+        void pool.run(b.task, { priority: "background" });
+        void pool.run(c.task, { priority: "on-demand" });
+        void pool.run(d.task, { priority: "background" });
+        await drain();
+        expect(t.starts).toEqual(["a"]);
+
+        // The on-demand `c` jumps ahead of the earlier-queued background `b`.
+        a.release();
+        await drain();
+        expect(t.starts).toEqual(["a", "c"]);
+
+        // Then background work drains in submission order: `b` before `d`.
+        c.release();
+        await drain();
+        expect(t.starts).toEqual(["a", "c", "b"]);
+        b.release();
+        await drain();
+        expect(t.starts).toEqual(["a", "c", "b", "d"]);
+        d.release();
+    });
+
+    it("defaults to background priority", async () => {
+        const pool = new BoundedPool(1);
+        const t = new Tracker();
+        const a = t.gate<void>("a");
+        const plain = t.gate<void>("plain");
+        const urgent = t.gate<void>("urgent");
+
+        void pool.run(a.task);
+        await drain();
+        void pool.run(plain.task); // no options -> background
+        void pool.run(urgent.task, { priority: "on-demand" });
+        await drain();
+
+        a.release();
+        await drain();
+        expect(t.starts).toEqual(["a", "urgent"]);
+        urgent.release();
+        plain.release();
+    });
+});
+
+describe("BoundedPool in-flight dedup", () => {
+    it("fetches a key once and hands both callers the same result", async () => {
+        const pool = new BoundedPool(5);
+        const t = new Tracker();
+        const g = t.gate<number>();
+
+        const first = pool.run(g.task, { key: 7 });
+        const second = pool.run(g.task, { key: 7 });
+        await drain();
+
+        expect(g.runs()).toBe(1);
+        expect(first).toBe(second);
+
+        g.release(99);
+        expect(await first).toBe(99);
+        expect(await second).toBe(99);
+    });
+
+    it("dedups only while in flight; a settled key runs again", async () => {
+        const pool = new BoundedPool(5);
+        const t = new Tracker();
+        const g1 = t.gate<number>();
+
+        const first = pool.run(g1.task, { key: 7 });
+        await drain();
+        g1.release(1);
+        expect(await first).toBe(1);
+
+        // The key is free again once its task settled.
+        const g2 = t.gate<number>();
+        const third = pool.run(g2.task, { key: 7 });
+        await drain();
+        expect(g2.started()).toBe(true);
+        g2.release(2);
+        expect(await third).toBe(2);
+    });
+
+    it("propagates a rejection to every deduped caller", async () => {
+        const pool = new BoundedPool(5);
+        const t = new Tracker();
+        const g = t.gate<number>();
+
+        const first = pool.run(g.task, { key: 7 });
+        const second = pool.run(g.task, { key: 7 });
+        await drain();
+
+        const boom = new Error("boom");
+        g.fail(boom);
+        await expect(first).rejects.toBe(boom);
+        await expect(second).rejects.toBe(boom);
+
+        // A failed key is also freed, so it may be retried by a fresh submit.
+        const g2 = t.gate<number>();
+        const retry = pool.run(g2.task, { key: 7 });
+        await drain();
+        expect(g2.started()).toBe(true);
+        g2.release(5);
+        expect(await retry).toBe(5);
+    });
+});
+
+describe("BoundedPool slot lifetime", () => {
+    it("holds one slot for a task's whole lifetime, retries included", async () => {
+        const pool = new BoundedPool(1);
+        const t = new Tracker();
+
+        // A task that internally makes two attempts before succeeding — the
+        // shape of a retrying request. It must occupy exactly one slot for the
+        // whole of that, so no other task may start until it finally settles.
+        const attempt1 = t.gate<void>("attempt1");
+        const attempt2 = t.gate<void>("attempt2");
+        const retrying = async (): Promise<void> => {
+            try {
+                await attempt1.task();
+            } catch {
+                await attempt2.task();
+            }
+        };
+        const other = t.gate<void>("other");
+
+        const running = pool.run(retrying);
+        await drain();
+        void pool.run(other.task);
+        await drain();
+
+        // First attempt is in flight and holds the only slot.
+        expect(t.starts).toEqual(["attempt1"]);
+        expect(other.started()).toBe(false);
+
+        // The retry is still the same task in the same slot; `other` waits.
+        attempt1.fail(new Error("transient"));
+        await drain();
+        expect(t.starts).toEqual(["attempt1", "attempt2"]);
+        expect(other.started()).toBe(false);
+
+        // Only when the whole task settles does the slot free.
+        attempt2.release();
+        await running;
+        await drain();
+        expect(other.started()).toBe(true);
+        other.release();
+    });
+});
+
+describe("RequestPools", () => {
+    it("exposes three pools at the design's default caps", () => {
+        expect(DEFAULT_METADATA_CONCURRENCY).toBe(10);
+        expect(DEFAULT_CONTENT_CONCURRENCY).toBe(5);
+        expect(DEFAULT_THUMBNAIL_CONCURRENCY).toBe(25);
+
+        const pools = new RequestPools();
+        expect(pools.metadata.concurrency).toBe(10);
+        expect(pools.content.concurrency).toBe(5);
+        expect(pools.thumbnails.concurrency).toBe(25);
+    });
+
+    it("takes overridden caps", () => {
+        const pools = new RequestPools({
+            metadataConcurrency: 1,
+            contentConcurrency: 2,
+            thumbnailConcurrency: 3,
+        });
+        expect(pools.metadata.concurrency).toBe(1);
+        expect(pools.content.concurrency).toBe(2);
+        expect(pools.thumbnails.concurrency).toBe(3);
+    });
+
+    it("keeps pools independent: an idle pool lends no slots", async () => {
+        const pools = new RequestPools({ contentConcurrency: 1 });
+        const t = new Tracker();
+        const c1 = t.gate<void>();
+        const c2 = t.gate<void>();
+        const c3 = t.gate<void>();
+
+        // The content pool is capped at 1. The thumbnail pool sits idle with 25
+        // free slots — none of which may be borrowed to run a second content
+        // task.
+        void pools.content.run(c1.task);
+        void pools.content.run(c2.task);
+        void pools.content.run(c3.task);
+        await drain();
+        expect(t.active).toBe(1);
+
+        c1.release();
+        await drain();
+        expect(t.active).toBe(1);
+        c2.release();
+        await drain();
+        expect(t.active).toBe(1);
+        c3.release();
+        await drain();
+        expect(t.peak).toBe(1);
+    });
+
+    it("runs different pools concurrently", async () => {
+        const pools = new RequestPools({
+            metadataConcurrency: 1,
+            contentConcurrency: 1,
+        });
+        const t = new Tracker();
+        const m = t.gate<void>();
+        const c = t.gate<void>();
+
+        void pools.metadata.run(m.task);
+        void pools.content.run(c.task);
+        await drain();
+
+        // One slot each, in two independent pools: both run at once.
+        expect(t.active).toBe(2);
+        m.release();
+        c.release();
+    });
+});

test/library/read.test.ts

@@ -0,0 +1,537 @@
+/**
+ * Tests for the in-process read surface (issue #44).
+ *
+ * Phase 1 (#43) projected the decrypted store into plain `AlbumRecord` /
+ * `PhotoRecord` values. This phase adds the read API a CLI or in-process script
+ * uses, all served from RAM with no network:
+ *
+ *  - `lib.albums`  — `list` / `byName` / `byID`, returning thin `Album` wrappers.
+ *  - `lib.photos`  — `byID` (a `Photo` wrapper) and `records` (plain records).
+ *  - `lib.timeline.groups` — photos bucketed by local day / week / month.
+ *
+ * Every call takes a single named-argument object; there are no positional
+ * arguments. The wrapper classes are for in-process callers only (they hold
+ * object identity, not JSON); the plain records remain the IPC-safe surface.
+ * Content-fetch methods (`Photo.original` / `thumbnail`) are a later unit and
+ * deliberately absent here — this surface is read-only.
+ *
+ * The detailed cases drive the API factories directly over a hand-built
+ * projection (`deriveRecords`), which keeps them free of disk and timers. A
+ * final section opens a real `Library` to prove the namespaces are wired to the
+ * live store and that a read never touches the client.
+ */
+
+import { describe, it, expect, beforeAll, afterAll } from "vitest";
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import {
+    deriveRecords,
+    type DerivedRecords,
+} from "../../src/library/records.js";
+import {
+    Album,
+    Photo,
+    makeAlbumsAPI,
+    makePhotosAPI,
+    makeTimelineAPI,
+    type TimelineGroup,
+} from "../../src/library/read.js";
+import { Library } from "../../src/library/index.js";
+import { MetadataStore } from "../../src/library/store.js";
+import type { CollectionsPage, FilesPage } from "../../src/client.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+const OWNER = 42;
+
+// Ente stores times in microseconds; records expose milliseconds. These
+// helpers keep the fixtures readable: `ms(...)` picks an epoch-millisecond
+// instant, `micros(...)` is what the fixture stores so the derived record's
+// `takenAt` comes back as the same millisecond value.
+const ms = (epochMillis: number): number => epochMillis;
+const micros = (epochMillis: number): number => epochMillis * 1000;
+
+const collection = (
+    id: number,
+    opts: Partial<Collection> = {},
+): Collection => ({
+    id,
+    ownerID: OWNER,
+    key: new Uint8Array([id & 0xff, 1, 2, 3]),
+    name: `album-${id}`,
+    type: "album",
+    updationTime: micros(1_700_000_000_000),
+    isShared: false,
+    ...opts,
+});
+
+const file = (
+    id: number,
+    collectionID: number,
+    opts: Partial<EnteFile> & {
+        creationTime?: number;
+        title?: string;
+        fileType?: EnteFile["metadata"]["fileType"];
+        latitude?: number;
+        longitude?: number;
+    } = {},
+): EnteFile => {
+    const { creationTime, title, fileType, latitude, longitude, ...rest } =
+        opts;
+    const metadata: EnteFile["metadata"] = {
+        title: title ?? `file-${id}.jpg`,
+        fileType: fileType ?? "image",
+        creationTime: creationTime ?? micros(1_700_000_000_000),
+        modificationTime: micros(1_700_000_000_000),
+    };
+    if (latitude !== undefined) metadata.latitude = latitude;
+    if (longitude !== undefined) metadata.longitude = longitude;
+    return {
+        id,
+        collectionID,
+        ownerID: OWNER,
+        key: new Uint8Array([id & 0xff, 9, 8, 7]),
+        metadata,
+        file: { decryptionHeader: "aGVhZGVy" },
+        thumbnail: { decryptionHeader: "dGh1bWI=" },
+        updationTime: micros(1_700_000_000_000),
+        ...rest,
+    };
+};
+
+// Build the three API objects over one fixed projection, the way `Library`
+// wires them over its live store.
+const apis = (records: DerivedRecords) => {
+    const derive = () => records;
+    return {
+        albums: makeAlbumsAPI(derive),
+        photos: makePhotosAPI(derive),
+        timeline: makeTimelineAPI(derive),
+    };
+};
+
+// Every file id present across all timeline groups, in group-then-member order.
+const allFileIDs = (groups: TimelineGroup[]): number[] =>
+    groups.flatMap((g) => g.fileIDs);
+
+describe("lib.albums", () => {
+    it("lists albums as Album wrappers, newest updated first", () => {
+        const records = deriveRecords(
+            [
+                collection(1, { updationTime: micros(1_700_000_000_000) }),
+                collection(2, { updationTime: micros(1_705_000_000_000) }),
+            ],
+            [file(10, 1), file(20, 2)],
+        );
+        const albums = apis(records).albums.list();
+        expect(albums.every((a) => a instanceof Album)).toBe(true);
+        // Collection 2 updated later, so it sorts ahead of collection 1.
+        expect(albums.map((a) => a.collectionID)).toEqual([2, 1]);
+    });
+
+    it("exposes album fields and its photos newest first", () => {
+        const records = deriveRecords(
+            [
+                collection(7, {
+                    name: "Trip",
+                    type: "favorites",
+                    isShared: true,
+                }),
+            ],
+            [
+                file(1, 7, { creationTime: micros(1_600_000_000_000) }),
+                file(2, 7, { creationTime: micros(1_800_000_000_000) }),
+                file(3, 7, { creationTime: micros(1_700_000_000_000) }),
+            ],
+        );
+        const album = apis(records).albums.byID({ collectionID: 7 })!;
+        expect(album.name).toBe("Trip");
+        expect(album.type).toBe("favorites");
+        expect(album.isShared).toBe(true);
+        expect(album.fileIDs).toEqual([2, 3, 1]);
+
+        const photos = album.photos.list();
+        expect(photos.every((p) => p instanceof Photo)).toBe(true);
+        expect(photos.map((p) => p.fileID)).toEqual([2, 3, 1]);
+
+        // record() hands back the plain, JSON-safe projection.
+        expect("key" in album.record()).toBe(false);
+    });
+
+    it("finds an album by exact name and returns undefined when absent", () => {
+        const records = deriveRecords(
+            [
+                collection(1, { name: "Berlin" }),
+                collection(2, { name: "Paris" }),
+            ],
+            [file(10, 1), file(20, 2)],
+        );
+        const { albums } = apis(records);
+        expect(albums.byName({ albumName: "Paris" })?.collectionID).toBe(2);
+        expect(albums.byName({ albumName: "paris" })).toBeUndefined();
+        expect(albums.byName({ albumName: "Nowhere" })).toBeUndefined();
+    });
+
+    it("returns undefined for an unknown collection id", () => {
+        const records = deriveRecords([collection(1)], [file(10, 1)]);
+        expect(
+            apis(records).albums.byID({ collectionID: 999 }),
+        ).toBeUndefined();
+    });
+});
+
+describe("lib.photos", () => {
+    it("byID returns a Photo wrapper carrying the mapped fields", () => {
+        const records = deriveRecords(
+            [collection(1)],
+            [
+                file(1001, 1, {
+                    title: "IMG.jpg",
+                    fileType: "video",
+                    creationTime: micros(1_699_000_000_000),
+                    latitude: 52.52,
+                    longitude: 13.405,
+                    pubMagicMetadata: { caption: "at the lake" },
+                }),
+            ],
+        );
+        const photo = apis(records).photos.byID({ fileID: 1001 })!;
+        expect(photo).toBeInstanceOf(Photo);
+        expect(photo.title).toBe("IMG.jpg");
+        expect(photo.fileType).toBe("video");
+        expect(photo.takenAt).toBe(ms(1_699_000_000_000));
+        expect(photo.caption).toBe("at the lake");
+        expect(photo.latitude).toBeCloseTo(52.52);
+        expect(photo.isArchived).toBe(false);
+        expect(photo.isHidden).toBe(false);
+        // The wrapper hands back the plain record, IPC-safe.
+        expect("key" in photo.record()).toBe(false);
+    });
+
+    it("byID returns undefined for an unknown file id", () => {
+        const records = deriveRecords([collection(1)], [file(1, 1)]);
+        expect(apis(records).photos.byID({ fileID: 999 })).toBeUndefined();
+    });
+
+    it("records() returns plain records in requested order, deduped, skipping unknowns", () => {
+        const records = deriveRecords(
+            [collection(1)],
+            [file(1, 1), file(2, 1), file(3, 1)],
+        );
+        const out = apis(records).photos.records({
+            fileIDs: [3, 1, 3, 999, 2],
+        });
+        // Requested order preserved; the repeated 3 appears once; 999 is dropped.
+        expect(out.map((r) => r.fileID)).toEqual([3, 1, 2]);
+        // Plain records, not wrappers, and JSON round-trips whole.
+        expect(out[0]).not.toBeInstanceOf(Photo);
+        expect(JSON.parse(JSON.stringify(out[0]))).toEqual(out[0]);
+    });
+
+    it("emits one record for a file even when it belongs to several albums", () => {
+        // File 1001 is a member of collections 1 and 2.
+        const records = deriveRecords(
+            [collection(1), collection(2)],
+            [file(1001, 1), file(1001, 2)],
+        );
+        const out = apis(records).photos.records({ fileIDs: [1001, 1001] });
+        expect(out).toHaveLength(1);
+        expect(out[0]!.albumIDs).toEqual([1, 2]);
+    });
+});
+
+describe("lib.timeline grouping", () => {
+    // Group keys and `startsAt` are computed in local time. Pinning the zone to
+    // UTC makes the expected values exact and lets the fixtures use `Date.UTC`.
+    const savedTZ = process.env.TZ;
+    beforeAll(() => {
+        process.env.TZ = "UTC";
+    });
+    afterAll(() => {
+        if (savedTZ === undefined) delete process.env.TZ;
+        else process.env.TZ = savedTZ;
+    });
+
+    it("buckets by local day, newest group and newest member first", () => {
+        const records = deriveRecords(
+            [collection(1)],
+            [
+                file(1, 1, { creationTime: micros(Date.UTC(2024, 0, 15, 9)) }),
+                file(2, 1, { creationTime: micros(Date.UTC(2024, 0, 15, 18)) }),
+                file(3, 1, { creationTime: micros(Date.UTC(2024, 0, 16, 12)) }),
+                file(4, 1, { creationTime: micros(Date.UTC(2024, 1, 1, 12)) }),
+            ],
+        );
+        const groups = apis(records).timeline.groups({ groupBy: "day" });
+        expect(groups.map((g) => g.key)).toEqual([
+            "2024-02-01",
+            "2024-01-16",
+            "2024-01-15",
+        ]);
+        // Group start is local midnight of the day.
+        expect(groups[2]!.startsAt).toBe(Date.UTC(2024, 0, 15));
+        // Within the 2024-01-15 group, the later photo (id 2) is first.
+        expect(groups[2]!.fileIDs).toEqual([2, 1]);
+    });
+
+    it("buckets by week with weeks starting on Monday", () => {
+        // 2024-01-15 is a Monday; the week runs through Sunday 2024-01-21.
+        const records = deriveRecords(
+            [collection(1)],
+            [
+                file(1, 1, { creationTime: micros(Date.UTC(2024, 0, 15, 12)) }), // Mon
+                file(2, 1, { creationTime: micros(Date.UTC(2024, 0, 17, 12)) }), // Wed
+                file(3, 1, { creationTime: micros(Date.UTC(2024, 0, 21, 12)) }), // Sun
+                file(4, 1, { creationTime: micros(Date.UTC(2024, 0, 22, 12)) }), // next Mon
+            ],
+        );
+        const groups = apis(records).timeline.groups({ groupBy: "week" });
+        // ISO week keys: 2024-01-15 is in 2024-W03, the next Monday in 2024-W04.
+        expect(groups.map((g) => g.key)).toEqual(["2024-W04", "2024-W03"]);
+        const first = groups.find((g) => g.key === "2024-W03")!;
+        expect(first.startsAt).toBe(Date.UTC(2024, 0, 15));
+        // The Sunday belongs to the Monday-started week, not the next one.
+        expect(first.fileIDs.sort((a, b) => a - b)).toEqual([1, 2, 3]);
+    });
+
+    it("assigns a Sunday to the preceding Monday's week across a month boundary", () => {
+        // 2024-01-14 is a Sunday; its week started Monday 2024-01-08.
+        const records = deriveRecords(
+            [collection(1)],
+            [file(1, 1, { creationTime: micros(Date.UTC(2024, 0, 14, 12)) })],
+        );
+        const groups = apis(records).timeline.groups({ groupBy: "week" });
+        expect(groups.map((g) => g.key)).toEqual(["2024-W02"]);
+        expect(groups[0]!.startsAt).toBe(Date.UTC(2024, 0, 8));
+    });
+
+    it("buckets by month", () => {
+        const records = deriveRecords(
+            [collection(1)],
+            [
+                file(1, 1, { creationTime: micros(Date.UTC(2024, 0, 3, 12)) }),
+                file(2, 1, { creationTime: micros(Date.UTC(2024, 0, 28, 12)) }),
+                file(3, 1, { creationTime: micros(Date.UTC(2024, 1, 9, 12)) }),
+            ],
+        );
+        const groups = apis(records).timeline.groups({ groupBy: "month" });
+        expect(groups.map((g) => g.key)).toEqual(["2024-02", "2024-01"]);
+        expect(groups[1]!.startsAt).toBe(Date.UTC(2024, 0, 1));
+        expect(groups[1]!.fileIDs).toEqual([2, 1]);
+    });
+
+    it("lists each file once even when it belongs to several albums", () => {
+        // File 1001 is in collections 1 and 2 but must appear once in a group.
+        const records = deriveRecords(
+            [collection(1), collection(2)],
+            [
+                file(1001, 1, {
+                    creationTime: micros(Date.UTC(2024, 0, 15, 12)),
+                }),
+                file(1001, 2, {
+                    creationTime: micros(Date.UTC(2024, 0, 15, 12)),
+                }),
+            ],
+        );
+        const groups = apis(records).timeline.groups({ groupBy: "day" });
+        expect(allFileIDs(groups)).toEqual([1001]);
+    });
+});
+
+describe("lib.timeline uses local time, not UTC", () => {
+    const savedTZ = process.env.TZ;
+    afterAll(() => {
+        if (savedTZ === undefined) delete process.env.TZ;
+        else process.env.TZ = savedTZ;
+    });
+
+    it("buckets by the viewer's local day", () => {
+        // Kolkata is UTC+5:30 with no DST. An instant at 2024-01-14T20:00Z is
+        // 2024-01-15 01:30 local, so it belongs to the local day 2024-01-15.
+        process.env.TZ = "Asia/Kolkata";
+        const records = deriveRecords(
+            [collection(1)],
+            [file(1, 1, { creationTime: micros(Date.UTC(2024, 0, 14, 20)) })],
+        );
+        const groups = apis(records).timeline.groups({ groupBy: "day" });
+        expect(groups[0]!.key).toBe("2024-01-15");
+        // Local midnight of 2024-01-15, which is 2024-01-14T18:30Z.
+        expect(groups[0]!.startsAt).toBe(new Date(2024, 0, 15).getTime());
+        expect(groups[0]!.startsAt).toBe(Date.UTC(2024, 0, 14, 18, 30));
+    });
+});
+
+describe("PhotoFilter", () => {
+    // A fixture spanning albums, file types, geotags, captions, and the two
+    // visibility states, all on the same local day so grouping is incidental.
+    const day = (h: number): number => micros(Date.UTC(2024, 2, 4, h));
+    const records = (): DerivedRecords =>
+        deriveRecords(
+            [
+                collection(1, { name: "Holidays" }),
+                collection(2, { name: "Work" }),
+            ],
+            [
+                file(1, 1, {
+                    title: "Beach sunset",
+                    creationTime: day(1),
+                    latitude: 1,
+                    longitude: 2,
+                }),
+                file(2, 1, {
+                    title: "clip.mov",
+                    fileType: "video",
+                    creationTime: day(2),
+                }),
+                file(3, 2, {
+                    title: "invoice scan",
+                    creationTime: day(3),
+                    pubMagicMetadata: { caption: "SUNSET colours" },
+                }),
+                file(4, 2, {
+                    title: "archived note",
+                    creationTime: day(4),
+                    magicMetadata: { visibility: 1 },
+                }),
+                file(5, 2, {
+                    title: "secret",
+                    creationTime: day(5),
+                    magicMetadata: { visibility: 2 },
+                }),
+            ],
+        );
+
+    const idsWith = (
+        filter: Parameters<
+            ReturnType<typeof apis>["timeline"]["groups"]
+        >[0]["filter"],
+    ): number[] =>
+        allFileIDs(
+            apis(records()).timeline.groups({ groupBy: "day", filter }),
+        ).sort((a, b) => a - b);
+
+    it("never includes hidden photos and excludes archived by default", () => {
+        // No filter: hidden (5) always gone, archived (4) gone unless asked for.
+        expect(idsWith(undefined)).toEqual([1, 2, 3]);
+    });
+
+    it("includes archived photos when includeArchived is set, hidden still never", () => {
+        expect(idsWith({ includeArchived: true })).toEqual([1, 2, 3, 4]);
+    });
+
+    it("filters by album membership", () => {
+        expect(idsWith({ albumID: 1 })).toEqual([1, 2]);
+        expect(idsWith({ albumID: 2 })).toEqual([3]);
+    });
+
+    it("filters by file type", () => {
+        expect(idsWith({ fileTypes: ["video"] })).toEqual([2]);
+        expect(idsWith({ fileTypes: ["image", "video"] })).toEqual([1, 2, 3]);
+    });
+
+    it("filters by presence or absence of location", () => {
+        expect(idsWith({ hasLocation: true })).toEqual([1]);
+        expect(idsWith({ hasLocation: false })).toEqual([2, 3]);
+    });
+
+    it("matches text case-insensitively against title, caption, and album name", () => {
+        // Title match (case-insensitive): "Beach sunset".
+        expect(idsWith({ text: "SUNSET" })).toEqual([1, 3]);
+        // Caption-only match: file 3's caption is "SUNSET colours".
+        expect(idsWith({ text: "colours" })).toEqual([3]);
+        // Album-name match: everything in "Holidays".
+        expect(idsWith({ text: "holiday" })).toEqual([1, 2]);
+    });
+
+    it("combines filters", () => {
+        // Images in album 1 with a location: only file 1.
+        expect(
+            idsWith({ albumID: 1, fileTypes: ["image"], hasLocation: true }),
+        ).toEqual([1]);
+    });
+});
+
+/**
+ * A minimal mock `Client`, enough for `Library.open` to run its refresh loop.
+ * The queues are empty, so the background refresh over a seeded cache changes
+ * nothing; the counters prove that a read never calls the client.
+ */
+class MockClient {
+    userID = OWNER;
+    collectionsCalls = 0;
+    filesCalls = 0;
+
+    whoami(): { email: string; userID: number } {
+        return { email: "user@example.com", userID: this.userID };
+    }
+    async collectionsSince(args: {
+        sinceTime: number;
+    }): Promise<CollectionsPage> {
+        this.collectionsCalls++;
+        return { collections: [], deleted: [], cursor: args.sinceTime };
+    }
+    async filesSince(args: {
+        collectionID: number;
+        collectionKey: Uint8Array;
+        sinceTime: number;
+    }): Promise<FilesPage> {
+        this.filesCalls++;
+        return { files: [], deleted: [], cursor: args.sinceTime };
+    }
+}
+
+describe("Library exposes the read surface over its live store", () => {
+    let dir: string;
+
+    beforeAll(() => {
+        dir = mkdtempSync(join(tmpdir(), "quak-read-"));
+    });
+    afterAll(() => {
+        rmSync(dir, { recursive: true, force: true });
+    });
+
+    it("serves albums, photos, and timeline from RAM without calling the client", async () => {
+        const cacheDirectory = join(dir, "cache");
+        const path = join(cacheDirectory, "metadata.json");
+
+        // Seed a cache as a prior run left it, so open() serves it at once.
+        const seed = await MetadataStore.load(path);
+        seed.userID = OWNER;
+        seed.collectionsSinceTime = 100;
+        seed.putCollection(collection(1, { name: "Seeded" }));
+        seed.putFile(
+            file(1001, 1, { creationTime: micros(Date.UTC(2024, 5, 1, 12)) }),
+        );
+        await seed.save();
+
+        const client = new MockClient();
+        // A long interval keeps the background timer from firing during the test.
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: 3600,
+        });
+        try {
+            const collectionsBefore = client.collectionsCalls;
+            const filesBefore = client.filesCalls;
+
+            expect(lib.albums.list().map((a) => a.name)).toEqual(["Seeded"]);
+            expect(
+                lib.albums.byName({ albumName: "Seeded" })?.collectionID,
+            ).toBe(1);
+            expect(lib.photos.byID({ fileID: 1001 })?.fileID).toBe(1001);
+            expect(
+                lib.photos.records({ fileIDs: [1001] }).map((r) => r.fileID),
+            ).toEqual([1001]);
+            const groups = lib.timeline.groups({ groupBy: "month" });
+            expect(allFileIDs(groups)).toEqual([1001]);
+
+            // Reads are answered from RAM: no read called the client.
+            expect(client.collectionsCalls).toBe(collectionsBefore);
+            expect(client.filesCalls).toBe(filesBefore);
+        } finally {
+            lib.close();
+        }
+    });
+});

test/library/records.test.ts

@@ -0,0 +1,279 @@
+/**
+ * Tests for the plain-record mapping in `src/library/records.ts` (issue #43).
+ *
+ * The library keeps decrypted `Collection`/`EnteFile` objects in RAM, but those
+ * carry binary keys and cannot cross the Electron IPC boundary. `deriveRecords`
+ * projects them into plain `AlbumRecord`/`PhotoRecord` values — no keys, no
+ * `Uint8Array`, JSON-safe — that the GUI process consumes. This file pins:
+ *
+ *  1. Field mapping from `metadata` and the two magic-metadata layers, using the
+ *     real Ente field names confirmed against the fixtures in
+ *     `test/cli/metadata-backup.test.ts` (`w`/`h`) and `test/library/store.test.ts`
+ *     (`visibility`): title/takenAt precedence, caption, width/height, geo,
+ *     visibility → isArchived/isHidden, fileType.
+ *  2. `takenAt` is milliseconds; Ente stores creationTime/editedTime in
+ *     microseconds, so the record divides by 1000.
+ *  3. Deduplication: one `PhotoRecord` per fileID even when the file belongs to
+ *     several collections, with every membership's collection id in `albumIDs`.
+ *  4. Ordering: photos and album `fileIDs` are newest first.
+ *  5. No key material survives the projection.
+ *  6. `diffRecords` reports exactly what changed between two derivations, and
+ *     returns undefined when nothing changed.
+ */
+
+import { describe, it, expect } from "vitest";
+import {
+    deriveRecords,
+    snapshotFrom,
+    diffRecords,
+} from "../../src/library/records.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+const OWNER = 42;
+
+// Microsecond epoch values, as Ente stores times. 1e15 ≈ 2001 in microseconds.
+const T = (micros: number): number => micros;
+
+const collection = (
+    id: number,
+    opts: Partial<Collection> = {},
+): Collection => ({
+    id,
+    ownerID: OWNER,
+    key: new Uint8Array([id & 0xff, 1, 2, 3]),
+    name: `album-${id}`,
+    type: "album",
+    updationTime: T(1_700_000_000_000_000),
+    isShared: false,
+    ...opts,
+});
+
+const file = (
+    id: number,
+    collectionID: number,
+    opts: Partial<EnteFile> & {
+        creationTime?: number;
+        title?: string;
+    } = {},
+): EnteFile => {
+    const { creationTime, title, ...rest } = opts;
+    return {
+        id,
+        collectionID,
+        ownerID: OWNER,
+        key: new Uint8Array([id & 0xff, 9, 8, 7]),
+        metadata: {
+            title: title ?? `file-${id}.jpg`,
+            fileType: "image",
+            creationTime: creationTime ?? T(1_700_000_000_000_000),
+            modificationTime: T(1_700_000_000_000_000),
+        },
+        file: { decryptionHeader: "aGVhZGVy" },
+        thumbnail: { decryptionHeader: "dGh1bWI=" },
+        updationTime: T(1_700_000_000_000_000),
+        ...rest,
+    };
+};
+
+describe("deriveRecords: photo mapping", () => {
+    it("projects a file into a plain PhotoRecord with no key material", () => {
+        const f = file(1001, 1, {
+            metadata: {
+                title: "IMG_1.jpg",
+                fileType: "image",
+                creationTime: T(1_699_000_000_000_000),
+                modificationTime: T(1_699_000_000_000_000),
+                latitude: 52.52,
+                longitude: 13.405,
+            },
+        });
+        const { photos } = deriveRecords([collection(1)], [f]);
+        const rec = photos.get(1001)!;
+
+        expect(rec.fileID).toBe(1001);
+        expect(rec.albumIDs).toEqual([1]);
+        expect(rec.title).toBe("IMG_1.jpg");
+        expect(rec.fileType).toBe("image");
+        expect(rec.latitude).toBeCloseTo(52.52);
+        expect(rec.longitude).toBeCloseTo(13.405);
+        expect(rec.isArchived).toBe(false);
+        expect(rec.isHidden).toBe(false);
+
+        // Safe to send over IPC: no key, no Uint8Array, JSON round-trips whole.
+        expect("key" in rec).toBe(false);
+        expect(JSON.parse(JSON.stringify(rec))).toEqual(rec);
+    });
+
+    it("takenAt is creationTime converted from microseconds to milliseconds", () => {
+        const f = file(1001, 1, { creationTime: T(1_699_000_000_000_000) });
+        const { photos } = deriveRecords([collection(1)], [f]);
+        expect(photos.get(1001)!.takenAt).toBe(1_699_000_000_000);
+    });
+
+    it("prefers pubMagicMetadata.editedName and editedTime over metadata", () => {
+        const f = file(1001, 1, {
+            title: "original.jpg",
+            creationTime: T(1_699_000_000_000_000),
+            pubMagicMetadata: {
+                editedName: "Sunset over the bay",
+                editedTime: T(1_650_000_000_000_000),
+            },
+        });
+        const { photos } = deriveRecords([collection(1)], [f]);
+        const rec = photos.get(1001)!;
+        expect(rec.title).toBe("Sunset over the bay");
+        expect(rec.takenAt).toBe(1_650_000_000_000);
+    });
+
+    it("falls back to metadata when the edited fields are empty or absent", () => {
+        const f = file(1001, 1, {
+            title: "original.jpg",
+            creationTime: T(1_699_000_000_000_000),
+            pubMagicMetadata: { editedName: "" },
+        });
+        const { photos } = deriveRecords([collection(1)], [f]);
+        const rec = photos.get(1001)!;
+        expect(rec.title).toBe("original.jpg");
+        expect(rec.takenAt).toBe(1_699_000_000_000);
+    });
+
+    it("maps caption and width/height from the public magic metadata", () => {
+        const f = file(1001, 1, {
+            pubMagicMetadata: {
+                caption: "at the beach",
+                w: 3000,
+                h: 2000,
+            },
+        });
+        const rec = deriveRecords([collection(1)], [f]).photos.get(1001)!;
+        expect(rec.caption).toBe("at the beach");
+        expect(rec.width).toBe(3000);
+        expect(rec.height).toBe(2000);
+    });
+
+    it("omits optional fields that are absent from the metadata", () => {
+        const rec = deriveRecords([collection(1)], [file(1001, 1)]).photos.get(
+            1001,
+        )!;
+        expect("caption" in rec).toBe(false);
+        expect("width" in rec).toBe(false);
+        expect("height" in rec).toBe(false);
+        expect("latitude" in rec).toBe(false);
+    });
+
+    it("reads archived and hidden from private magicMetadata.visibility", () => {
+        const archived = file(1, 1, { magicMetadata: { visibility: 1 } });
+        const hidden = file(2, 1, { magicMetadata: { visibility: 2 } });
+        const visible = file(3, 1, { magicMetadata: { visibility: 0 } });
+        const { photos } = deriveRecords(
+            [collection(1)],
+            [archived, hidden, visible],
+        );
+        expect(photos.get(1)).toMatchObject({
+            isArchived: true,
+            isHidden: false,
+        });
+        expect(photos.get(2)).toMatchObject({
+            isArchived: false,
+            isHidden: true,
+        });
+        expect(photos.get(3)).toMatchObject({
+            isArchived: false,
+            isHidden: false,
+        });
+    });
+});
+
+describe("deriveRecords: dedup and ordering", () => {
+    it("emits one PhotoRecord per fileID across memberships, all albums listed", () => {
+        // File 1001 belongs to collections 1 and 2; 2002 only to 2.
+        const files = [
+            file(1001, 1, { creationTime: T(1_700_000_000_000_000) }),
+            file(1001, 2, { creationTime: T(1_700_000_000_000_000) }),
+            file(2002, 2, { creationTime: T(1_710_000_000_000_000) }),
+        ];
+        const { photos } = deriveRecords([collection(1), collection(2)], files);
+        expect([...photos.keys()].sort((a, b) => a - b)).toEqual([1001, 2002]);
+        expect(photos.get(1001)!.albumIDs).toEqual([1, 2]);
+        expect(photos.get(2002)!.albumIDs).toEqual([2]);
+    });
+
+    it("orders snapshot photos newest first by takenAt", () => {
+        const files = [
+            file(1, 1, { creationTime: T(1_600_000_000_000_000) }),
+            file(2, 1, { creationTime: T(1_800_000_000_000_000) }),
+            file(3, 1, { creationTime: T(1_700_000_000_000_000) }),
+        ];
+        const snap = snapshotFrom(deriveRecords([collection(1)], files), 123);
+        expect(snap.photos.map((p) => p.fileID)).toEqual([2, 3, 1]);
+        expect(snap.takenAt).toBe(123);
+    });
+
+    it("orders album fileIDs newest first", () => {
+        const files = [
+            file(1, 7, { creationTime: T(1_600_000_000_000_000) }),
+            file(2, 7, { creationTime: T(1_800_000_000_000_000) }),
+            file(3, 7, { creationTime: T(1_700_000_000_000_000) }),
+        ];
+        const { albums } = deriveRecords([collection(7)], files);
+        expect(albums.get(7)!.fileIDs).toEqual([2, 3, 1]);
+    });
+});
+
+describe("deriveRecords: album mapping", () => {
+    it("carries collection identity, sharing, and the favorites type", () => {
+        const fav = collection(9, {
+            name: "Favorites",
+            type: "favorites",
+            isShared: true,
+            updationTime: T(1_705_000_000_000_000),
+        });
+        const rec = deriveRecords([fav], [file(1, 9)]).albums.get(9)!;
+        expect(rec).toMatchObject({
+            collectionID: 9,
+            name: "Favorites",
+            type: "favorites",
+            isShared: true,
+            updationTime: T(1_705_000_000_000_000),
+        });
+        expect("key" in rec).toBe(false);
+        expect(JSON.parse(JSON.stringify(rec))).toEqual(rec);
+    });
+});
+
+describe("diffRecords", () => {
+    const at = 999;
+
+    it("returns undefined when nothing changed", () => {
+        const a = deriveRecords([collection(1)], [file(1, 1)]);
+        const b = deriveRecords([collection(1)], [file(1, 1)]);
+        expect(diffRecords(a, b, at)).toBeUndefined();
+    });
+
+    it("reports added and changed albums and photos and removals", () => {
+        const before = deriveRecords(
+            [collection(1), collection(2)],
+            [file(1, 1), file(2, 2)],
+        );
+        // Collection 2 is gone (album + its only file removed). Collection 1 is
+        // renamed (changed album), gains file 3, and file 1 is retitled.
+        const after = deriveRecords(
+            [collection(1, { name: "renamed" })],
+            [
+                file(1, 1, {
+                    pubMagicMetadata: { editedName: "new title" },
+                }),
+                file(3, 1),
+            ],
+        );
+        const change = diffRecords(before, after, at)!;
+        expect(change.refreshedAt).toBe(at);
+        expect(change.albumIDsRemoved).toEqual([2]);
+        expect(change.fileIDsRemoved).toEqual([2]);
+        expect(change.albumsChanged.map((a) => a.collectionID)).toEqual([1]);
+        expect(change.albumsChanged[0]!.name).toBe("renamed");
+        expect(
+            change.photosChanged.map((p) => p.fileID).sort((x, y) => x - y),
+        ).toEqual([1, 3]);
+    });
+});

test/library/snapshot.test.ts

@@ -0,0 +1,303 @@
+/**
+ * Tests for `Library.snapshot()` and `Library.subscribe()` (issue #43).
+ *
+ * These are the surface the GUI consumes across Electron IPC. `snapshot()` is
+ * synchronous — it reads the in-RAM store and projects it into plain records
+ * (no keys) — and `subscribe({ onChange })` delivers a `LibraryChange` whenever
+ * a background refresh actually changes the derived records. The contracts:
+ *
+ *  1. `snapshot()` deduplicates a file across memberships into one record with
+ *     every album id, orders photos newest first, and carries no key material.
+ *  2. `subscribe` fires on a refresh that changes something, with the exact
+ *     changed and removed sets for both albums and photos.
+ *  3. A refresh that changes nothing (an empty diff) fires no change.
+ *  4. `unsubscribe()` stops further delivery.
+ *
+ * The client is the same scripted mock used by the refresh-loop tests: no
+ * crypto, no network. Interval tests use a short real interval and `vi.waitFor`.
+ */
+
+import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { Library } from "../../src/library/index.js";
+import type { LibraryChange } from "../../src/library/records.js";
+import type { CollectionsPage, FilesPage } from "../../src/client.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+const USER_ID = 42;
+const FAST_INTERVAL = 0.02;
+
+const collection = (
+    id: number,
+    updationTime: number,
+    name = `album-${id}`,
+): Collection => ({
+    id,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    name,
+    type: "album",
+    updationTime,
+    isShared: false,
+});
+
+const file = (
+    id: number,
+    collectionID: number,
+    creationTime: number,
+): EnteFile => ({
+    id,
+    collectionID,
+    ownerID: USER_ID,
+    key: new Uint8Array([id & 0xff]),
+    metadata: {
+        title: `file-${id}.jpg`,
+        fileType: "image",
+        creationTime,
+        modificationTime: creationTime,
+    },
+    file: { decryptionHeader: "aGVhZGVy" },
+    thumbnail: { decryptionHeader: "dGh1bWI=" },
+    updationTime: creationTime,
+});
+
+class MockClient {
+    userID = USER_ID;
+    collectionsQueue: CollectionsPage[] = [];
+    filesByCollection = new Map<number, FilesPage[]>();
+
+    whoami(): { email: string; userID: number } {
+        return { email: "user@example.com", userID: this.userID };
+    }
+
+    async collectionsSince(args: {
+        sinceTime: number;
+    }): Promise<CollectionsPage> {
+        return (
+            this.collectionsQueue.shift() ?? {
+                collections: [],
+                deleted: [],
+                cursor: args.sinceTime,
+            }
+        );
+    }
+
+    async filesSince(args: {
+        collectionID: number;
+        collectionKey: Uint8Array;
+        sinceTime: number;
+    }): Promise<FilesPage> {
+        const queue = this.filesByCollection.get(args.collectionID);
+        return (
+            queue?.shift() ?? {
+                files: [],
+                deleted: [],
+                cursor: args.sinceTime,
+            }
+        );
+    }
+
+    filesFor(collectionID: number, ...pages: FilesPage[]): void {
+        this.filesByCollection.set(collectionID, pages);
+    }
+}
+
+describe("Library.snapshot and Library.subscribe", () => {
+    let dir: string;
+    let cacheDirectory: string;
+
+    beforeEach(() => {
+        dir = mkdtempSync(join(tmpdir(), "quak-snapshot-"));
+        cacheDirectory = join(dir, "cache");
+    });
+
+    afterEach(() => {
+        rmSync(dir, { recursive: true, force: true });
+    });
+
+    it("snapshot() dedupes across memberships, orders newest first, holds no keys", async () => {
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100), collection(2, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        // File 1001 is in both collections; 2002 only in collection 2 and newer.
+        client.filesFor(1, {
+            files: [file(1001, 1, 1_600_000_000_000_000)],
+            deleted: [],
+            cursor: 1_600_000_000_000_000,
+        });
+        client.filesFor(2, {
+            files: [
+                file(1001, 2, 1_600_000_000_000_000),
+                file(2002, 2, 1_800_000_000_000_000),
+            ],
+            deleted: [],
+            cursor: 1_800_000_000_000_000,
+        });
+
+        const lib = await Library.open({ client, cacheDirectory });
+        try {
+            const snap = lib.snapshot();
+
+            // One record per fileID, newest first, both albums on the shared file.
+            expect(snap.photos.map((p) => p.fileID)).toEqual([2002, 1001]);
+            const shared = snap.photos.find((p) => p.fileID === 1001)!;
+            expect(shared.albumIDs).toEqual([1, 2]);
+            expect(shared.takenAt).toBe(1_600_000_000_000);
+
+            expect(snap.albums.map((a) => a.collectionID).sort()).toEqual([
+                1, 2,
+            ]);
+
+            // Nothing carries key material; the whole snapshot is JSON-safe.
+            expect(JSON.parse(JSON.stringify(snap))).toEqual(snap);
+            for (const p of snap.photos) expect("key" in p).toBe(false);
+            for (const a of snap.albums) expect("key" in a).toBe(false);
+        } finally {
+            lib.close();
+        }
+    });
+
+    it("subscribe fires on a refresh change with the correct changed/removed sets", async () => {
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100), collection(2, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 1_600_000_000_000_000)],
+            deleted: [],
+            cursor: 1_600_000_000_000_000,
+        });
+        client.filesFor(2, {
+            files: [file(2002, 2, 1_600_000_000_000_000)],
+            deleted: [],
+            cursor: 1_600_000_000_000_000,
+        });
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        const changes: LibraryChange[] = [];
+        const { unsubscribe } = lib.subscribe({
+            onChange: (c) => changes.push(c),
+        });
+        try {
+            // Next refresh: collection 2 (and its file) tombstoned; collection 1
+            // gains file 1003.
+            client.filesFor(1, {
+                files: [file(1003, 1, 1_700_000_000_000_000)],
+                deleted: [],
+                cursor: 1_700_000_000_000_000,
+            });
+            client.collectionsQueue.push({
+                collections: [collection(1, 200)],
+                deleted: [2],
+                cursor: 200,
+            });
+
+            await vi.waitFor(() => expect(changes.length).toBeGreaterThan(0), {
+                timeout: 2000,
+                interval: 5,
+            });
+
+            const change = changes[0]!;
+            expect(change.albumIDsRemoved).toEqual([2]);
+            expect(change.fileIDsRemoved).toEqual([2002]);
+            expect(change.photosChanged.map((p) => p.fileID)).toEqual([1003]);
+            expect(change.albumsChanged.map((a) => a.collectionID)).toEqual([
+                1,
+            ]);
+            expect(change.refreshedAt).toBeGreaterThan(0);
+        } finally {
+            unsubscribe();
+            lib.close();
+        }
+    });
+
+    it("a refresh that changes nothing fires no change", async () => {
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 1_600_000_000_000_000)],
+            deleted: [],
+            cursor: 1_600_000_000_000_000,
+        });
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        const changes: LibraryChange[] = [];
+        const { unsubscribe } = lib.subscribe({
+            onChange: (c) => changes.push(c),
+        });
+        try {
+            // Let several empty-diff ticks pass; none may deliver a change.
+            await new Promise((r) => setTimeout(r, FAST_INTERVAL * 1000 * 6));
+            expect(changes).toEqual([]);
+        } finally {
+            unsubscribe();
+            lib.close();
+        }
+    });
+
+    it("unsubscribe stops further delivery", async () => {
+        const client = new MockClient();
+        client.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        client.filesFor(1, {
+            files: [file(1001, 1, 1_600_000_000_000_000)],
+            deleted: [],
+            cursor: 1_600_000_000_000_000,
+        });
+
+        const lib = await Library.open({
+            client,
+            cacheDirectory,
+            refreshIntervalSeconds: FAST_INTERVAL,
+        });
+        const changes: LibraryChange[] = [];
+        const { unsubscribe } = lib.subscribe({
+            onChange: (c) => changes.push(c),
+        });
+        unsubscribe();
+        try {
+            client.collectionsQueue.push({
+                collections: [collection(3, 300)],
+                deleted: [],
+                cursor: 300,
+            });
+            client.filesFor(3, {
+                files: [file(3003, 3, 1_700_000_000_000_000)],
+                deleted: [],
+                cursor: 1_700_000_000_000_000,
+            });
+            // The change lands in the store, but the cancelled subscriber sees
+            // nothing.
+            await vi.waitFor(
+                () => expect(lib.snapshot().albums.length).toBe(2),
+                { timeout: 2000, interval: 5 },
+            );
+            expect(changes).toEqual([]);
+        } finally {
+            lib.close();
+        }
+    });
+});

test/library/store.test.ts

@@ -0,0 +1,246 @@
+/**
+ * Tests for the on-disk JSON metadata store (`MetadataStore`).
+ *
+ * The store is the local cache the library keeps of the account's server
+ * state: one `metadata.json` file holding the user id, a schema version, the
+ * cursor for the incremental collections listing, and the decrypted
+ * collection and file records. The whole file is read into RAM on load and
+ * rewritten as a whole on save. A separate refresh unit (issue #42) is what
+ * populates it; this unit only stores.
+ *
+ * Four contracts are load-bearing and each is exercised below:
+ *
+ * 1. **Round-trip fidelity.** Everything put into the store — including the
+ *    binary decryption keys, which JSON cannot hold directly and which the
+ *    store base64-encodes — comes back byte-for-byte after a save and a fresh
+ *    load. A cache that quietly dropped or mangled a field would hand the
+ *    caller wrong keys or stale metadata.
+ *
+ * 2. **A missing or corrupt file loads as an empty store, never an error.**
+ *    The file is only a cache: if it is absent (first run) or unreadable
+ *    (interrupted write on an older build, disk corruption, hand-editing),
+ *    the right answer is to start empty and let the refresh unit repopulate,
+ *    not to crash the whole library.
+ *
+ * 3. **Writes are atomic and durable.** The store reuses the same
+ *    fsync-before-rename atomic writer the download layer uses, so a reader
+ *    never sees a half-written file and a crash cannot leave a truncated one.
+ *    The observable consequence tested here is that a save leaves exactly the
+ *    destination file behind — no temporary sibling — and that overwriting an
+ *    existing store preserves a complete, re-loadable file.
+ *
+ * 4. **Permissions match `session.json`.** The directory is `0700` and the
+ *    file is `0600`, because the records contain decrypted key material and
+ *    must not be readable by other users on a shared machine.
+ */
+
+import { describe, it, expect, beforeEach, afterEach } from "vitest";
+import {
+    mkdtempSync,
+    rmSync,
+    readdirSync,
+    statSync,
+    writeFileSync,
+    mkdirSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import {
+    MetadataStore,
+    METADATA_SCHEMA_VERSION,
+} from "../../src/library/store.js";
+import type { Collection, EnteFile } from "../../src/model/types.js";
+
+// A representative decrypted collection, including a binary key and all three
+// magic-metadata layers, so the round-trip test proves every field survives.
+const sampleCollection = (): Collection => ({
+    id: 12345,
+    ownerID: 42,
+    key: new Uint8Array([1, 2, 3, 4, 5, 6, 7, 8]),
+    name: "Holiday 2026",
+    type: "album",
+    updationTime: 1_700_000_000_000_000,
+    isShared: true,
+    magicMetadata: { visibility: 0 },
+    pubMagicMetadata: { subType: 0, coverID: 999 },
+    sharedMagicMetadata: { note: "shared with a friend" },
+});
+
+// A representative decrypted file membership: metadata, both blob headers, a
+// binary key, a content hash, and file/thumbnail sizes.
+const sampleFile = (): EnteFile => ({
+    id: 67890,
+    collectionID: 12345,
+    ownerID: 42,
+    key: new Uint8Array([9, 8, 7, 6, 5, 4, 3, 2, 1]),
+    metadata: {
+        title: "IMG_0001.jpg",
+        fileType: "image",
+        creationTime: 1_699_000_000_000_000,
+        modificationTime: 1_699_000_500_000_000,
+        latitude: 52.52,
+        longitude: 13.405,
+        hash: "sha256:deadbeef",
+    },
+    magicMetadata: { editedName: "sunset" },
+    pubMagicMetadata: { editedTime: 1_699_000_600_000_000 },
+    file: { decryptionHeader: "ZmlsZUhlYWRlcg==", size: 4_194_304 },
+    thumbnail: { decryptionHeader: "dGh1bWJIZWFkZXI=", size: 8192 },
+    updationTime: 1_700_000_100_000_000,
+});
+
+describe("MetadataStore", () => {
+    let dir: string;
+    let path: string;
+
+    beforeEach(() => {
+        dir = mkdtempSync(join(tmpdir(), "quak-store-"));
+        // Deliberately nest the store one level below the temp dir so save()
+        // has to create its own directory and set its mode.
+        path = join(dir, "cache", "metadata.json");
+    });
+
+    afterEach(() => {
+        rmSync(dir, { recursive: true, force: true });
+    });
+
+    it("round-trips the whole model, keys and all", async () => {
+        const store = await MetadataStore.load(path);
+        store.userID = 42;
+        store.collectionsSinceTime = 1_700_000_000_000_000;
+        store.putCollection(sampleCollection());
+        store.putFile(sampleFile());
+        await store.save();
+
+        const reloaded = await MetadataStore.load(path);
+        expect(reloaded.userID).toBe(42);
+        expect(reloaded.collectionsSinceTime).toBe(1_700_000_000_000_000);
+
+        // The binary key must come back as the exact bytes, not a base64
+        // string or a plain object of numbered keys.
+        const collection = reloaded.getCollection(12345);
+        expect(collection).toEqual(sampleCollection());
+        expect(collection?.key).toBeInstanceOf(Uint8Array);
+
+        const file = reloaded.getFile(12345, 67890);
+        expect(file).toEqual(sampleFile());
+        expect(file?.key).toBeInstanceOf(Uint8Array);
+
+        expect(reloaded.listCollections()).toHaveLength(1);
+        expect(reloaded.listFiles(12345)).toHaveLength(1);
+    });
+
+    it("writes the declared schema version", async () => {
+        const store = await MetadataStore.load(path);
+        await store.save();
+        const reloaded = await MetadataStore.load(path);
+        expect(reloaded.schemaVersion).toBe(METADATA_SCHEMA_VERSION);
+    });
+
+    it("loads an empty store when the file is missing", async () => {
+        const store = await MetadataStore.load(path);
+        expect(store.userID).toBe(0);
+        expect(store.listCollections()).toEqual([]);
+        expect(store.getCollection(1)).toBeUndefined();
+    });
+
+    it("loads an empty store when the file is corrupt", async () => {
+        mkdirSync(join(dir, "cache"), { recursive: true });
+        writeFileSync(path, "{ this is not valid json ][");
+        const store = await MetadataStore.load(path);
+        expect(store.listCollections()).toEqual([]);
+        expect(store.listFiles(12345)).toEqual([]);
+    });
+
+    it("loads an empty store when the schema version does not match", async () => {
+        // A cache written by a future build with an incompatible schema is
+        // discarded rather than misread; the refresh unit repopulates it.
+        mkdirSync(join(dir, "cache"), { recursive: true });
+        writeFileSync(
+            path,
+            JSON.stringify({
+                schemaVersion: METADATA_SCHEMA_VERSION + 1,
+                userID: 42,
+                collectionsSinceTime: 0,
+                collections: [],
+                files: [],
+            }),
+        );
+        const store = await MetadataStore.load(path);
+        expect(store.userID).toBe(0);
+        expect(store.listCollections()).toEqual([]);
+    });
+
+    it("creates the directory 0700 and the file 0600", async () => {
+        const store = await MetadataStore.load(path);
+        store.putCollection(sampleCollection());
+        await store.save();
+
+        // Directory 0700, file 0600: on a shared machine the decrypted keys
+        // in this file must be readable only by their owner. Mask to the
+        // permission bits; the file-type bits are not part of the assertion.
+        expect(statSync(join(dir, "cache")).mode & 0o777).toBe(0o700);
+        expect(statSync(path).mode & 0o777).toBe(0o600);
+    });
+
+    it("leaves exactly the destination behind, with no temp sibling", async () => {
+        const store = await MetadataStore.load(path);
+        store.putCollection(sampleCollection());
+        await store.save();
+        // The atomic writer stages a temporary file and renames it into
+        // place; on success nothing temporary is left in the directory.
+        expect(readdirSync(join(dir, "cache"))).toEqual(["metadata.json"]);
+    });
+
+    it("overwrites an existing store atomically and stays re-loadable", async () => {
+        const first = await MetadataStore.load(path);
+        first.userID = 1;
+        first.putCollection(sampleCollection());
+        await first.save();
+
+        const second = await MetadataStore.load(path);
+        second.userID = 2;
+        second.deleteCollection(12345);
+        await second.save();
+
+        const reloaded = await MetadataStore.load(path);
+        expect(reloaded.userID).toBe(2);
+        expect(reloaded.getCollection(12345)).toBeUndefined();
+        expect(readdirSync(join(dir, "cache"))).toEqual(["metadata.json"]);
+    });
+
+    it("deletes a collection together with its file memberships", async () => {
+        const store = await MetadataStore.load(path);
+        store.putCollection(sampleCollection());
+        store.putFile(sampleFile());
+        store.deleteCollection(12345);
+        expect(store.getCollection(12345)).toBeUndefined();
+        expect(store.getFile(12345, 67890)).toBeUndefined();
+        expect(store.listFiles(12345)).toEqual([]);
+    });
+
+    it("scopes file records to their collection membership", async () => {
+        // The same underlying file can be a member of two collections, each a
+        // separate record with its own key. Storing one must not touch the
+        // other, and lookups are per membership.
+        const store = await MetadataStore.load(path);
+        const inA = sampleFile();
+        const inB: EnteFile = {
+            ...sampleFile(),
+            collectionID: 55555,
+            key: new Uint8Array([100, 101, 102]),
+        };
+        store.putFile(inA);
+        store.putFile(inB);
+
+        expect(store.getFile(12345, 67890)?.key).toEqual(inA.key);
+        expect(store.getFile(55555, 67890)?.key).toEqual(inB.key);
+        expect(store.listFiles(12345)).toHaveLength(1);
+        expect(store.listFiles(55555)).toHaveLength(1);
+
+        store.deleteFile(12345, 67890);
+        expect(store.getFile(12345, 67890)).toBeUndefined();
+        expect(store.getFile(55555, 67890)?.key).toEqual(inB.key);
+    });
+});

test/model/decrypt.test.ts

@@ -145,7 +145,12 @@ const buildSharedRawCollection = (
 
 const buildRawFile = (
     collectionKey: Uint8Array,
-    opts?: { title?: string; fileType?: number; creationTime?: number },
+    opts?: {
+        title?: string;
+        fileType?: number;
+        creationTime?: number;
+        info?: { fileSize?: number; thumbSize?: number };
+    },
 ): RawEnteFile => {
     const fileKey = sodium.crypto_secretbox_keygen();
     const { ciphertext: encFileKey, nonce: fileKeyNonce } = secretboxEncrypt(
@@ -186,6 +191,7 @@ const buildRawFile = (
         },
         file: { decryptionHeader: toBase64(sodium.randombytes_buf(24)) },
         thumbnail: { decryptionHeader: toBase64(sodium.randombytes_buf(24)) },
+        info: opts?.info,
         updationTime: 1700000000000000,
     };
 };
@@ -357,4 +363,47 @@ describe("model.decryptFile", () => {
 
         expect(() => decryptFile(raw, wrongKey)).toThrow();
     });
+
+    it("carries the file and thumbnail byte sizes from info", () => {
+        // The server reports the encrypted-blob sizes in `info`; the cache
+        // needs them without a HEAD request, so decryptFile must copy them
+        // onto the file and thumbnail blobs.
+        const masterKey = sodium.crypto_secretbox_keygen();
+        const { collectionKey } = buildRawCollection(masterKey);
+        const raw = buildRawFile(collectionKey, {
+            info: { fileSize: 4096, thumbSize: 512 },
+        });
+
+        const file = decryptFile(raw, collectionKey);
+
+        expect(file.file.size).toBe(4096);
+        expect(file.thumbnail.size).toBe(512);
+    });
+
+    it("leaves the sizes undefined when the server omits info", () => {
+        // Older files predate the info field; the sizes must stay undefined
+        // rather than become 0, so callers can tell "unknown" from "empty".
+        const masterKey = sodium.crypto_secretbox_keygen();
+        const { collectionKey } = buildRawCollection(masterKey);
+        const raw = buildRawFile(collectionKey);
+        expect(raw.info).toBeUndefined();
+
+        const file = decryptFile(raw, collectionKey);
+
+        expect(file.file.size).toBeUndefined();
+        expect(file.thumbnail.size).toBeUndefined();
+    });
+
+    it("carries the deletion flag from the diff row", () => {
+        // The diff marks a deleted row with isDeleted; decryptFile copies it
+        // onto the file so a caller can tell a deleted row from a live one.
+        const masterKey = sodium.crypto_secretbox_keygen();
+        const { collectionKey } = buildRawCollection(masterKey);
+        const raw = buildRawFile(collectionKey);
+        raw.isDeleted = true;
+
+        const file = decryptFile(raw, collectionKey);
+
+        expect(file.isDeleted).toBe(true);
+    });
 });

test/packaging/build-context.test.ts

@@ -12,7 +12,7 @@
 //
 // Neither shows up as a build failure, so they are asserted here.
 import { describe, expect, it } from "vitest";
-import { readFileSync } from "node:fs";
+import { existsSync, readFileSync } from "node:fs";
 import { fileURLToPath } from "node:url";
 import { join } from "node:path";
 
@@ -46,4 +46,19 @@ describe(".dockerignore", () => {
     it("leaves .gitignore in the build context for prettier", () => {
         expect(dockerignore).not.toContain(".gitignore");
     });
+
+    // Both images are built from this same context, and the lint image runs
+    // eslint and prettier across it. BuildKit lets a `<dockerfile>.dockerignore`
+    // shadow the root one for a single build; such a file would silently give
+    // the lint build a different, unreviewed context — and eslint's flat config
+    // does not ignore dot-directories, so a stray `.claude/` worktree would be
+    // linted.
+    it.each(["Dockerfile", "Dockerfile.lint"])(
+        "is not shadowed by a per-Dockerfile ignore file for %s",
+        (name) => {
+            expect(existsSync(join(repoRoot, `${name}.dockerignore`))).toBe(
+                false,
+            );
+        },
+    );
 });

test/packaging/entrypoints.test.ts

@@ -1,7 +1,9 @@
 // The package manifest promises three files that only exist after a build:
 // `main`, `types`, and the `quak` binary. Nothing in the test suite used to
-// look at them, and `make check` runs test, lint and fmt-check but never the
-// build, so `tsconfig.json` and `package.json` were free to drift apart. They
+// look at them, and `make check` runs the suite and the lint container but
+// never the build, so `tsconfig.json` and `package.json` were free to drift
+// apart. (The formatting check is part of the lint container, not a step of
+// its own; `test/packaging/lint-once.test.ts` is what holds that shape.) They
 // did: `rootDir` was `./src` while `include` also pulled in `bin/**/*`, which
 // is TS6059, and no build had succeeded for as long as that was true.
 //

test/packaging/lint-docker.test.ts

@@ -0,0 +1,184 @@
+// Linting runs in Docker, one way, everywhere: `script/lint` builds
+// `Dockerfile.lint`, which COPYs the repo into a digest-pinned image and runs
+// eslint and prettier as build steps, so a successful build IS a clean lint.
+//
+// Three things can quietly undo that, and none of them shows up as a build
+// failure, which is why they are asserted here:
+//
+// 1. Recursion. `script/check` calls `script/lint`, and `script/lint` is now a
+//    `docker build`. Anything that runs `make check` inside a container is
+//    therefore asking for Docker inside Docker, and CI breaks. The image built
+//    from `Dockerfile` runs the suite and the compile only; lint happens once,
+//    in `Dockerfile.lint`.
+// 2. Cache. A lint build over an unchanged tree returns success in well under a
+//    second having linted nothing. The `LINT_EPOCH` guard is what forces the
+//    linter layers to execute, and it has to fail closed: an unset build
+//    argument is the empty string, which is a perfectly stable cache key, so an
+//    invocation that omits it must be rejected rather than served a cached
+//    green.
+// 3. A host lint path surviving alongside the container one, which would let a
+//    lint result come from an unpinned local toolchain.
+import { describe, expect, it } from "vitest";
+import { readFileSync } from "node:fs";
+import { fileURLToPath } from "node:url";
+import { join } from "node:path";
+
+const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
+
+const read = (name: string): string =>
+    readFileSync(join(repoRoot, name), "utf-8");
+
+// The executable lines of a shell script or Dockerfile: comments carry the
+// reasoning and frequently name the very commands these tests forbid, so they
+// would otherwise trigger every assertion below.
+const instructions = (name: string): string[] =>
+    read(name)
+        .split("\n")
+        .map((line) => line.trim())
+        .filter((line) => line !== "" && !line.startsWith("#"));
+
+const lintScript = instructions("script/lint");
+const dockerfileLint = instructions("Dockerfile.lint");
+const dockerfile = instructions("Dockerfile");
+const cibuild = instructions("script/cibuild");
+
+const has = (lines: string[], pattern: RegExp): boolean =>
+    lines.some((line) => pattern.test(line));
+
+describe("script/lint", () => {
+    it("lints by building Dockerfile.lint", () => {
+        expect(has(lintScript, /docker build .*-f Dockerfile\.lint/)).toBe(
+            true,
+        );
+    });
+
+    // The whole point of the ruling: no invocation of a linter against the
+    // working tree survives, so a lint verdict can only come from the pinned
+    // image.
+    it("runs no linter on the host", () => {
+        expect(has(lintScript, /eslint|prettier/)).toBe(false);
+    });
+
+    // Without a fresh epoch the build is served from cache in under a second,
+    // having linted nothing, and still exits 0.
+    it("passes a fresh LINT_EPOCH on every run", () => {
+        expect(
+            has(lintScript, /--build-arg LINT_EPOCH="\$\(date \+%s\)"/),
+        ).toBe(true);
+    });
+});
+
+describe("Dockerfile.lint", () => {
+    // Tag references are server-mutable, so they are remote code execution.
+    it("pins its base image by digest", () => {
+        expect(has(dockerfileLint, /^FROM \S+@sha256:[0-9a-f]{64}/)).toBe(true);
+    });
+
+    it("runs eslint as a build step", () => {
+        expect(has(dockerfileLint, /^RUN .*eslint \./)).toBe(true);
+    });
+
+    it("runs prettier as a build step", () => {
+        expect(has(dockerfileLint, /^RUN .*prettier --check \./)).toBe(true);
+    });
+
+    // An unset ARG is the empty string, and an empty string is a perfectly
+    // stable cache key. Rejecting it is what stops a bare
+    // `docker build -f Dockerfile.lint .` from reporting a green it did not
+    // earn.
+    it("refuses to build without LINT_EPOCH", () => {
+        expect(has(dockerfileLint, /^ARG LINT_EPOCH$/)).toBe(true);
+        expect(
+            has(dockerfileLint, /^RUN \[ -n "\$LINT_EPOCH" \] \|\| exit 1$/),
+        ).toBe(true);
+    });
+
+    // The guard only forces execution of the layers below it, so both linters
+    // have to sit after it. Layer order is the mechanism, not a style choice.
+    it("puts both linters below the epoch guard", () => {
+        const guard = dockerfileLint.findIndex((line) =>
+            /^RUN \[ -n "\$LINT_EPOCH" \]/.test(line),
+        );
+        const linters = dockerfileLint
+            .map((line, index) => ({ line, index }))
+            .filter(({ line }) => /^RUN .*(eslint|prettier)/.test(line));
+
+        expect(linters.length).toBeGreaterThan(0);
+        for (const { line, index } of linters) {
+            expect(
+                index,
+                `${line} must run below the LINT_EPOCH guard`,
+            ).toBeGreaterThan(guard);
+        }
+    });
+
+    // Dependency installation is the slow layer and has nothing to do with the
+    // sources, so it caches separately: manifests first, sources afterwards.
+    it("copies the manifests before the sources", () => {
+        const manifests = dockerfileLint.findIndex((line) =>
+            /^COPY package\.json yarn\.lock/.test(line),
+        );
+        const sources = dockerfileLint.findIndex((line) =>
+            /^COPY \. \.$/.test(line),
+        );
+
+        expect(manifests).toBeGreaterThanOrEqual(0);
+        expect(sources).toBeGreaterThan(manifests);
+    });
+
+    // script/lint is a docker build; a lint step that shelled out to it would
+    // recurse.
+    it("does not call script/lint or make lint", () => {
+        expect(has(dockerfileLint, /make lint|script\/lint/)).toBe(false);
+    });
+});
+
+describe("Dockerfile", () => {
+    // `make check` runs script/lint, which is a docker build, so an image that
+    // ran it would need a Docker daemon inside the container.
+    it("does not run make check, make lint or script/lint", () => {
+        expect(
+            has(dockerfile, /make check|make lint|script\/(check|lint)/),
+        ).toBe(false);
+    });
+
+    // The replaced lint stage took a `COPY --from=lint` dependency to order
+    // itself before the check stage. Dockerfile.lint is that stage now, and
+    // two definitions of how to lint is one too many.
+    it("has no lint stage", () => {
+        expect(has(dockerfile, /AS lint\b|--from=lint\b/)).toBe(false);
+    });
+
+    it("still runs the suite and the build under the epoch guard", () => {
+        expect(has(dockerfile, /^RUN make test$/)).toBe(true);
+        expect(has(dockerfile, /^RUN make build$/)).toBe(true);
+        expect(
+            has(dockerfile, /^RUN \[ -n "\$CHECK_EPOCH" \] \|\| exit 1$/),
+        ).toBe(true);
+    });
+});
+
+describe("script/cibuild", () => {
+    // CI has to get both verdicts. Lint goes first so the fast failure is
+    // reported before the suite runs.
+    it("builds the lint image before the test and build image", () => {
+        const lint = cibuild.findIndex((line) => /\/lint"/.test(line));
+        const check = cibuild.findIndex((line) =>
+            /docker build .*CHECK_EPOCH/.test(line),
+        );
+
+        expect(lint).toBeGreaterThanOrEqual(0);
+        expect(check).toBeGreaterThan(lint);
+    });
+});
+
+describe("package.json", () => {
+    // `yarn lint` was a second, unpinned way to get a lint verdict, from
+    // whatever eslint the working tree happened to have installed.
+    it("exposes no host lint script", () => {
+        const pkg = JSON.parse(read("package.json")) as {
+            scripts: Record<string, string>;
+        };
+        expect(pkg.scripts.lint).toBeUndefined();
+    });
+});

test/packaging/lint-once.test.ts

@@ -0,0 +1,503 @@
+// `make check` used to run `prettier --check .` twice: once inside the lint
+// container (`script/lint` builds `Dockerfile.lint`, which runs eslint and
+// prettier as build steps) and once again on the host, because `script/check`
+// also called `script/fmt-check`. Two passes, one verdict, and the host one is
+// the weaker of the two — its prettier is whatever the working tree happens to
+// have installed, while the container's is digest-pinned and installed under
+// `--frozen-lockfile`.
+//
+// The fix was to delete the host call from `script/check` and `script/precommit`.
+// Nothing about that fix is self-enforcing: anyone can wire `script/fmt-check`
+// back in, or add a prettier step to a Dockerfile, and every build stays green
+// while quietly doing the work twice again. So the count is asserted here
+// rather than promised in a comment.
+//
+// The assertion is a static walk of the invocation graph, not a string match
+// against one file. Starting from an entrypoint, it follows every edge the repo
+// actually uses to reach another command — `run:` steps in the CI workflow,
+// `"$SCRIPT_DIR/<name>"` and `script/<name>` into other scripts, `make <target>`
+// through the Makefile shims, `yarn run <name>` through the `package.json`
+// scripts, and `docker build -f <file>` into that Dockerfile's `RUN` steps — and
+// counts the prettier invocations it finds. A prettier call added anywhere in
+// that graph is therefore caught, wherever it is added.
+//
+// Two entrypoints are walked, because they cover different graphs: `make check`
+// is what a developer runs, and `.gitea/workflows/check.yml` is what CI runs.
+// The CI walk starts at the workflow file rather than at a hand-picked script,
+// so "the path CI executes" is read out of the repo instead of assumed; it
+// reaches `script/cibuild`, and through it the `Dockerfile` image that `make
+// check` never touches. Walking only `make check` is how a duplicate prettier
+// pass in `Dockerfile` stayed invisible.
+//
+// Undercounting is the failure mode that would make this test worthless. Three
+// things guard against it: the walk is asserted to have reached the nodes that
+// matter, an unresolvable or empty node is a thrown error rather than a quiet
+// zero, and prettier is counted per occurrence rather than per line, so two
+// invocations chained with `&&` cannot read as one.
+import { describe, expect, it } from "vitest";
+import { readFileSync } from "node:fs";
+import { fileURLToPath } from "node:url";
+import { join } from "node:path";
+
+const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
+
+const read = (name: string): string =>
+    readFileSync(join(repoRoot, name), "utf-8");
+
+// A backslash at end of line continues the command; the resolver has to see the
+// whole invocation, since the interesting flags (`-f Dockerfile.lint`) can sit
+// on the continuation.
+const joinContinuations = (text: string): string[] => {
+    const joined: string[] = [];
+    for (const raw of text.split("\n")) {
+        const line = raw.trim();
+        const previous = joined[joined.length - 1];
+        if (previous !== undefined && previous.endsWith("\\")) {
+            joined[joined.length - 1] =
+                `${previous.slice(0, -1).trim()} ${line}`;
+        } else {
+            joined.push(line);
+        }
+    }
+    return joined;
+};
+
+// Comments are stripped everywhere. The headers of these scripts explain the
+// duplication this test exists to prevent, and therefore name `prettier` and
+// `script/fmt-check` repeatedly; counting them would make the test assert the
+// prose instead of the behaviour.
+const executable = (text: string): string[] =>
+    joinContinuations(text).filter(
+        (line) => line !== "" && !line.startsWith("#"),
+    );
+
+// Every occurrence, not "does this line mention prettier": a line that reads
+// `yarn run prettier --check . && yarn run prettier --check src` is two passes
+// over the same tree, which is exactly the bug this file exists to catch, and
+// counting it as one would hide it. `.prettierrc` and `.prettierignore` are not
+// invocations and do not match, because `\b` requires a non-word character
+// after the name.
+const countPrettier = (line: string): number =>
+    (line.match(/\bprettier\b/g) ?? []).length;
+
+// Makefile targets are thin shims (`check:` / tab / `@script/check`), so a
+// `make <target>` edge has to resolve through them to keep "per `make check`"
+// meaning what it says. Recipe lines are the tab-indented ones.
+const makeRecipes = (): Map<string, string[]> => {
+    const recipes = new Map<string, string[]>();
+    let current: string | null = null;
+    for (const raw of read("Makefile").split("\n")) {
+        if (raw.startsWith("\t")) {
+            if (current !== null) {
+                recipes.get(current)?.push(raw.trim().replace(/^[@-]+/, ""));
+            }
+            continue;
+        }
+        const target = /^([a-z][a-z-]*)\s*:(?!=)/.exec(raw);
+        current = target === null ? null : target[1];
+        if (current !== null && !recipes.has(current)) {
+            recipes.set(current, []);
+        }
+    }
+    return recipes;
+};
+
+const recipes = makeRecipes();
+
+const packageScripts = (): Record<string, string> => {
+    const pkg = JSON.parse(read("package.json")) as {
+        scripts?: Record<string, string>;
+    };
+    return pkg.scripts ?? {};
+};
+
+const scripts = packageScripts();
+
+// Node keys: `script/<name>`, `docker:<Dockerfile>`, `make:<target>`,
+// `yarn:<package.json script>`, `workflow:<CI workflow file>`.
+const resolve = (node: string): string[] => {
+    if (node.startsWith("script/")) return executable(read(node));
+    if (node.startsWith("docker:")) {
+        return executable(read(node.slice("docker:".length)))
+            .filter((line) => line.startsWith("RUN "))
+            .map((line) => line.slice("RUN ".length));
+    }
+    // The `run:` steps of a workflow, in file order. `uses:` steps are actions,
+    // not commands, and have no edges into this repo's graph. A `run: |` block
+    // would resolve to the bare `|`, which reaches nothing and therefore fails
+    // the count rather than passing quietly.
+    if (node.startsWith("workflow:")) {
+        return executable(read(node.slice("workflow:".length)))
+            .filter((line) => /^-?\s*run:\s*\S/.test(line))
+            .map((line) => line.replace(/^-?\s*run:\s*/, ""));
+    }
+    if (node.startsWith("make:")) {
+        const target = node.slice("make:".length);
+        const recipe = recipes.get(target);
+        // A renamed or deleted target must be a loud failure: silently walking
+        // an empty recipe would report zero prettier invocations, which reads
+        // like the tidiest possible result.
+        if (recipe === undefined) {
+            throw new Error(`no such Makefile target: ${target}`);
+        }
+        return recipe;
+    }
+    if (node.startsWith("yarn:")) {
+        const name = node.slice("yarn:".length);
+        const script = scripts[name];
+        if (script === undefined) {
+            throw new Error(`no such package.json script: ${name}`);
+        }
+        return [script];
+    }
+    throw new Error(`unresolvable node: ${node}`);
+};
+
+// Same reasoning as the missing-target error, applied to every node kind: a
+// node that resolves to no commands contributes zero prettier invocations and
+// zero edges, which is indistinguishable from a clean result. Fail instead.
+const commandsOf = (node: string): string[] => {
+    const commands = resolve(node);
+    if (commands.length === 0) {
+        throw new Error(`node resolved to no commands: ${node}`);
+    }
+    return commands;
+};
+
+const edgesOf = (line: string): string[] => {
+    const edges: string[] = [];
+
+    // `"$SCRIPT_DIR/lint"`, `"$ROOT/script/lint"` and a bare `script/lint` are
+    // all the same edge.
+    for (const match of line.matchAll(
+        /(?:\$SCRIPT_DIR|\$\{SCRIPT_DIR\}|script)\/([a-z][a-z-]*)/g,
+    )) {
+        edges.push(`script/${match[1]}`);
+    }
+
+    // Only real targets: `pkg_install gnumake make make make` in
+    // script/bootstrap is a package name, not an invocation of this Makefile.
+    for (const match of line.matchAll(/\bmake\s+([a-z][a-z-]*)/g)) {
+        if (recipes.has(match[1] ?? "")) edges.push(`make:${match[1]}`);
+    }
+
+    // Same rule for yarn: `yarn run prettier` is the linter itself (counted,
+    // not followed), `yarn run fmt-check` would be a package.json script that
+    // runs it indirectly.
+    for (const match of line.matchAll(/\byarn(?:\s+run)?\s+([a-z][a-z-]*)/g)) {
+        if ((match[1] ?? "") in scripts) edges.push(`yarn:${match[1]}`);
+    }
+
+    // The container lint pass lives behind a `docker build`; without following
+    // it the count would miss the one invocation that is supposed to survive.
+    if (/\bdocker\s+build\b/.test(line)) {
+        const file = /\s-f\s+(\S+)/.exec(line);
+        edges.push(`docker:${file === null ? "Dockerfile" : file[1]}`);
+    }
+
+    return edges;
+};
+
+interface Walk {
+    prettier: number;
+    reached: Set<string>;
+}
+
+// Repeated invocations must count repeatedly — running the same script twice is
+// exactly the bug — so nodes are not deduplicated. The path stack is only there
+// to turn a cycle into a loud failure instead of a hang.
+//
+// Counting and edge-following both happen for every line: a line that invokes
+// prettier can also invoke something else, and skipping the edges of counted
+// lines silently truncated the graph.
+const walk = (node: string, path: string[] = [], into?: Walk): Walk => {
+    const result = into ?? { prettier: 0, reached: new Set<string>() };
+    if (path.includes(node)) {
+        throw new Error(`invocation cycle: ${[...path, node].join(" -> ")}`);
+    }
+    result.reached.add(node);
+
+    for (const line of commandsOf(node)) {
+        result.prettier += countPrettier(line);
+        for (const edge of edgesOf(line)) {
+            walk(edge, [...path, node], result);
+        }
+    }
+    return result;
+};
+
+describe("prettier runs exactly once per make check", () => {
+    const check = walk("make:check");
+
+    // The headline assertion, and the one the issue is about.
+    it("invokes prettier once for the whole of make check", () => {
+        expect(check.prettier).toBe(1);
+    });
+
+    // Guards against the count being 1 (or 0) because the walk never got
+    // anywhere. `make check` has to reach the suite, the lint script, and the
+    // Dockerfile whose build IS the lint verdict.
+    it.each(["script/check", "script/test", "script/lint", "Dockerfile.lint"])(
+        "reaches %s while counting",
+        (node) => {
+            const key = node.startsWith("script/") ? node : `docker:${node}`;
+            expect([...check.reached]).toContain(key);
+        },
+    );
+
+    // The one that survives is the container's, not the host's: that is the
+    // authoritative verdict, since a successful Dockerfile.lint build is what
+    // CI treats as proof of a clean tree.
+    it("keeps the surviving invocation inside the lint container", () => {
+        expect(walk("docker:Dockerfile.lint").prettier).toBe(1);
+    });
+
+    it("does not reach the host formatting check from make check", () => {
+        expect([...check.reached]).not.toContain("script/fmt-check");
+    });
+});
+
+describe("prettier runs exactly once per CI build", () => {
+    // Rooted at the workflow file, so this is the graph CI executes rather than
+    // the graph someone believed CI executes. `make check` cannot stand in for
+    // it: CI runs script/cibuild, which builds Dockerfile as well as
+    // Dockerfile.lint, and nothing under `make check` ever reads Dockerfile.
+    const ci = walk("workflow:.gitea/workflows/check.yml");
+
+    it("invokes prettier once for the whole CI build", () => {
+        expect(ci.prettier).toBe(1);
+    });
+
+    // script/cibuild is here because the workflow is asserted to run it;
+    // Dockerfile is here because it is the half of the CI graph that the
+    // `make check` walk cannot see.
+    it.each([
+        "script/cibuild",
+        "script/lint",
+        "docker:Dockerfile.lint",
+        "docker:Dockerfile",
+    ])("reaches %s while counting", (node) => {
+        expect([...ci.reached]).toContain(node);
+    });
+
+    // The test and build image must not lint: linting is Dockerfile.lint's job,
+    // and a prettier step added here would be a second pass over the same tree
+    // for the same verdict — on the one path where it matters most.
+    it("keeps prettier out of the test and build image", () => {
+        expect(walk("docker:Dockerfile").prettier).toBe(0);
+    });
+});
+
+describe("the standalone entrypoints still do what their names say", () => {
+    // REPO_POLICIES.md requires both `make lint` and `make fmt-check` to exist
+    // and mean something. Dropping fmt-check from script/check must not turn it
+    // into a target nobody can use, and must not leave `make check` passing
+    // because both halves became no-ops.
+    it("still checks formatting under make fmt-check", () => {
+        expect(walk("make:fmt-check").prettier).toBe(1);
+    });
+
+    it("still checks formatting under make lint", () => {
+        expect(walk("make:lint").prettier).toBe(1);
+    });
+});
+
+describe("script/precommit", () => {
+    // Same duplication as script/check, same fix. The hook still catches a
+    // badly formatted tree before the commit lands, because script/lint is the
+    // container prettier run — that is the whole reason the host call could go.
+    it("checks formatting exactly once", () => {
+        expect(walk("script/precommit").prettier).toBe(1);
+    });
+
+    it("gets that check from the lint container", () => {
+        expect([...walk("script/precommit").reached]).toContain(
+            "docker:Dockerfile.lint",
+        );
+    });
+});
+
+// script/bootstrap installs the dependencies, and it has two install sites: one
+// for the case where yarn has to be reached through nvm, and one for the case
+// where yarn is already on PATH. A substring check against the whole file
+// cannot tell them apart, so it reports the first and says nothing about the
+// second — which is the one the containers take, because the pinned node image
+// ships yarn. Both are resolved separately here.
+const installBranches = (): { withoutYarn: string[]; withYarn: string[] } => {
+    const lines = executable(read("script/bootstrap"));
+    const open = lines.findIndex((line) =>
+        /^install_js_deps\s*\(\)/.test(line),
+    );
+    if (open === -1) {
+        throw new Error("script/bootstrap: no install_js_deps function");
+    }
+    const close = lines.indexOf("}", open);
+    const body = lines.slice(open + 1, close === -1 ? undefined : close);
+    const guard = body.findIndex((line) =>
+        /^if\b.*\bmissing yarn\b/.test(line),
+    );
+    const otherwise = body.indexOf("else", guard);
+    const end = body.indexOf("fi", otherwise);
+    if (guard === -1 || otherwise === -1 || end === -1) {
+        throw new Error(
+            "script/bootstrap: install_js_deps is not the expected " +
+                "if missing yarn / else / fi shape",
+        );
+    }
+    return {
+        withoutYarn: body.slice(guard + 1, otherwise),
+        withYarn: body.slice(otherwise + 1, end),
+    };
+};
+
+// Every `yarn install` in the given lines, with its flags, so an unpinned
+// install cannot hide next to a pinned one.
+const yarnInstalls = (lines: string[]): string[] =>
+    lines.flatMap((line) =>
+        [...line.matchAll(/\byarn install\b[^"'&|;]*/g)].map((match) =>
+            match[0].trim(),
+        ),
+    );
+
+describe("host and container prettier cannot disagree", () => {
+    // With the host pass gone from `make check`, `make fmt-check` is the only
+    // host-side formatting check left, and the container is the gate. The two
+    // must keep producing the same verdict on the same tree, or a developer
+    // running `make fmt-check` gets a green that CI then rejects.
+    //
+    // Three things make them agree, and all three are load-bearing:
+    it("pins the same prettier for both", () => {
+        const pkg = JSON.parse(read("package.json")) as {
+            devDependencies: Record<string, string>;
+        };
+        // An exact version, not a range: `^3.8.1` would let the container and
+        // the host resolve different builds with different formatting.
+        expect(pkg.devDependencies.prettier).toMatch(/^\d+\.\d+\.\d+$/);
+    });
+
+    it("installs from the lockfile on the branch the container takes", () => {
+        // Both images are FROM a node image, which ships yarn, so `missing
+        // yarn` is false and this is the branch that runs in the container.
+        const installs = yarnInstalls(installBranches().withYarn);
+        expect(installs).not.toHaveLength(0);
+        for (const install of installs) {
+            expect(install).toContain("--frozen-lockfile");
+        }
+    });
+
+    it("installs from the lockfile on the nvm branch too", () => {
+        // Not the container's branch, but it is the one a developer without
+        // yarn on PATH gets, and their prettier has to match the container's.
+        const installs = yarnInstalls(installBranches().withoutYarn);
+        expect(installs).not.toHaveLength(0);
+        for (const install of installs) {
+            expect(install).toContain("--frozen-lockfile");
+        }
+    });
+
+    it("runs script/bootstrap inside the lint container", () => {
+        // Without this the lockfile assertions above would be about a script
+        // the container never executes.
+        expect([...walk("docker:Dockerfile.lint").reached]).toContain(
+            "script/bootstrap",
+        );
+    });
+
+    it("keeps .gitignore in the build context", () => {
+        // Prettier 3 reads .gitignore as a default ignore file, so excluding it
+        // from the context would change which files the container checks.
+        const dockerignore = read(".dockerignore")
+            .split("\n")
+            .map((line) => line.trim());
+        expect(dockerignore).not.toContain(".gitignore");
+    });
+});
+
+describe("the walk cannot pass vacuously", () => {
+    // An earlier draft of this file computed a Makefile target as
+    // `node.slice("make:")` — a string where a number belongs, which coerces to
+    // NaN and made every target resolve to nothing. The count went to zero and
+    // an assertion of "not twice" would have been satisfied by a walk that had
+    // read nothing at all. Every way of reaching nothing is therefore an
+    // error here, and the ways are tested rather than assumed.
+    it("reports zero for a subgraph that does not run prettier", () => {
+        expect(walk("make:clean").prettier).toBe(0);
+    });
+
+    it("refuses a Makefile target that does not exist", () => {
+        expect(() => walk("make:no-such-target")).toThrow(
+            /no such Makefile target/,
+        );
+    });
+
+    it("refuses a package.json script that does not exist", () => {
+        expect(() => walk("yarn:no-such-script")).toThrow(
+            /no such package.json script/,
+        );
+    });
+
+    it("refuses a script that does not exist", () => {
+        expect(() => walk("script/no-such-script")).toThrow(/ENOENT/);
+    });
+
+    it("refuses a node that resolves to no commands", () => {
+        // .dockerignore has no RUN steps, standing in for a Dockerfile whose
+        // steps a restructure moved somewhere the resolver cannot see.
+        expect(() => walk("docker:.dockerignore")).toThrow(
+            /resolved to no commands/,
+        );
+    });
+
+    it("refuses a node kind it does not understand", () => {
+        expect(() => walk("nonsense")).toThrow(/unresolvable node/);
+    });
+
+    it("refuses to walk in circles", () => {
+        expect(() => walk("make:check", ["script/check"])).toThrow(
+            /invocation cycle/,
+        );
+    });
+});
+
+describe("the resolver reads what the shell would run", () => {
+    // Counting per line is how `yarn run prettier --check . && yarn run
+    // prettier --check src` read as a single invocation.
+    it("counts every prettier invocation on a line", () => {
+        expect(
+            countPrettier(
+                "yarn run prettier --check . && yarn run prettier --check src",
+            ),
+        ).toBe(2);
+    });
+
+    it("does not count the config files as invocations", () => {
+        expect(countPrettier("COPY .prettierrc .prettierignore ./")).toBe(0);
+    });
+
+    // The counting `continue` also dropped every edge that shared a line with a
+    // prettier call, so a whole subtree could be hidden behind one `&&`.
+    it("still follows the edges of a line that invokes prettier", () => {
+        expect(
+            edgesOf('yarn run prettier --check . && "$SCRIPT_DIR/lint"'),
+        ).toContain("script/lint");
+    });
+
+    it("resolves every spelling of a script call to one node", () => {
+        expect(
+            edgesOf('"$SCRIPT_DIR/lint" "${SCRIPT_DIR}/test" script/fmt'),
+        ).toEqual(["script/lint", "script/test", "script/fmt"]);
+    });
+
+    it("follows a bare docker build to Dockerfile and -f to its file", () => {
+        expect(edgesOf("docker build .")).toContain("docker:Dockerfile");
+        expect(edgesOf("docker build -f Dockerfile.lint .")).toContain(
+            "docker:Dockerfile.lint",
+        );
+    });
+
+    it("reads the run steps of the CI workflow and not its uses steps", () => {
+        expect(commandsOf("workflow:.gitea/workflows/check.yml")).toEqual([
+            "script/cibuild",
+        ]);
+    });
+});