Stream decrypted downloads to disk with bounded memory (closes #40)

Originals no longer buffer the whole decrypted file in RAM. `streamDecrypt`
writes each secretstream chunk to the staged temp file as it is pulled and
returns the byte count, so peak memory is one chunk, not the file size. The
temp-then-rename fsync discipline of the exported `writeAtomic` is factored
into a shared helper that both the whole-buffer path and the streaming path
use.

The rename still happens only after the stream authenticates on `TAG_FINAL`;
a truncated or corrupt stream throws and removes the temp file, leaving the
destination untouched as before. Because the plaintext is no longer buffered,
the atomic write moved inside the retry: each attempt streams from byte zero
into its own temp file and only a complete attempt renames.

Closes #21.

Model: opus-4-8

Dockerfile

@@ -1,36 +1,27 @@
-# Lint stage — fast feedback on formatting and lint issues
+# Test and build image: the suite, then the compile.
-# node 22.22.0 on Alpine 3.23.3 (node:22-alpine), 2026-08-09
+#
-FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS lint
+# Linting deliberately does not happen here. `script/lint` is a build of
-WORKDIR /app
+# Dockerfile.lint, and `script/check` calls `script/lint`, so running
-COPY script/ script/
+# `make check` in this image would mean running `docker build` inside a
-COPY package.json yarn.lock ./
+# container. Lint runs exactly once, in Dockerfile.lint; script/cibuild
-RUN script/bootstrap
+# builds that first and this second.
-COPY . .
-RUN make fmt-check
-RUN make lint
-
-# Check stage — the full suite and the build
 # node 22.22.0 on Alpine 3.23.3 (node:22-alpine), 2026-08-09
 FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS check
 WORKDIR /app
 
-# Force BuildKit to run the lint stage before proceeding. Without this the
-# two stages run in parallel and a lint failure can lose the race.
-COPY --from=lint /app/yarn.lock /dev/null
-
 COPY script/ script/
 COPY package.json yarn.lock ./
 RUN script/bootstrap
 COPY . .
 
-# CHECK_EPOCH is a cache buster: without it Docker serves `make check` from
+# CHECK_EPOCH is a cache buster: without it Docker serves the test layer from
 # cache on an unchanged tree, the suite never executes, and the build still
 # exits 0. The guard makes an absent argument a hard failure — an unset ARG
 # is the empty string, which is a perfectly stable cache key, so a plain
 # `docker build .` would otherwise still get the false green. Fail closed.
 ARG CHECK_EPOCH
 RUN [ -n "$CHECK_EPOCH" ] || exit 1
-RUN make check
+RUN make test
 
 ARG CHECK_EPOCH
 RUN [ -n "$CHECK_EPOCH" ] || exit 1

Dockerfile.lint

@@ -0,0 +1,35 @@
+# Lint image: every lint run happens here, and nowhere else. The repo is
+# COPYed into a digest-pinned image and the linters run as build steps, so a
+# successful build IS a clean lint. `script/lint` does nothing but build this
+# file, which also works where the docker daemon is remote and bind mounts are
+# impossible. Nothing that runs inside a container may call `script/lint`:
+# that is why Dockerfile no longer runs `make check`.
+# node 22.22.0 on Alpine 3.23.3 (node:22-alpine), 2026-08-09
+FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS lint
+WORKDIR /app
+
+# Manifests before sources, so the dependency install layer stays cached
+# until package.json or yarn.lock changes. script/bootstrap ends in
+# `yarn install --frozen-lockfile`; the lint steps below are deliberately
+# not cached.
+COPY script/ script/
+COPY package.json yarn.lock ./
+RUN script/bootstrap
+
+COPY . .
+
+# LINT_EPOCH is a cache buster, with the same fail-closed contract as
+# CHECK_EPOCH in Dockerfile. No lint cache is wanted: on an unchanged tree
+# Docker serves the linter layers in well under a second, having linted
+# nothing, and the build still exits 0. The guard makes an absent argument a
+# hard failure — an unset ARG is the empty string, which is a perfectly
+# stable cache key, so a plain `docker build -f Dockerfile.lint .` would
+# otherwise get exactly that false green. Every layer below this one is a
+# child of the guard, so a fresh epoch forces all of them to execute.
+ARG LINT_EPOCH
+RUN [ -n "$LINT_EPOCH" ] || exit 1
+
+# The linters are invoked directly rather than through `make lint`, because
+# `make lint` is the build of this file.
+RUN yarn run eslint .
+RUN yarn run prettier --check .

README.md

@@ -86,31 +86,70 @@ alpine. We provide:
   files the compiler wrote, and make the CLI executable (our own extension)
 - `script/test` — run the test suite (vitest, hard-capped at 30s where `timeout`
   is available, verbose rerun on failure)
-- `script/lint` — run eslint and a prettier check
+- `script/lint` — run eslint and a prettier check, by building
+  `Dockerfile.lint`; requires docker (see Linting below)
 - `script/fmt` — format all files with prettier (writes)
-- `script/fmt-check` — check formatting (read-only)
+- `script/fmt-check` — check formatting on the host (read-only); standalone, and
-- `script/check` — run all checks: `test`, `lint`, `fmt-check` (our own
+  not called by `script/check` or `script/precommit`, because `script/lint`
-  extension)
+  already checks formatting in the container (see Linting below)
-- `script/docker` — build the Docker image, tagged via `script/projectname`
+- `script/check` — run all checks: `test`, `lint` (our own extension)
-- `script/cibuild` — cd to the repo root and run the image build (what CI runs;
+- `script/docker` — build the test and build image, tagged via
-  the build runs `make fmt-check` and `make lint` in a first stage, then
+  `script/projectname`
-  `make check` and `make build` in a second)
+- `script/cibuild` — cd to the repo root and build both images (what CI runs):
+  `script/lint` first, then the `Dockerfile` image, which runs `make test` and
+  `make build`
 - `script/precommit` — run by the git pre-commit hook (our own extension); runs
-  `script/lint` and `script/fmt-check` but deliberately not the tests, so the
+  `script/lint`, which checks both lint and formatting, but deliberately not the
-  TDD red-phase commit can land
+  tests, so the TDD red-phase commit can land
 - `script/install-precommit` — installs the git pre-commit hook (our own
   extension); `make hooks` shims to it
 
 `make hooks` installs the pre-commit hook that runs `script/precommit`.
 
-Both `script/docker` and `script/cibuild` pass
+### Linting
-`--build-arg CHECK_EPOCH="$(date +%s)"`. The Dockerfile refuses to build without
+
-it. This is deliberate: on an unchanged tree Docker would otherwise serve the
+Linting runs in a container, one way, everywhere. `script/lint` builds
-`make check` layer from cache, so the suite would never run and the build would
+`Dockerfile.lint`, which copies the repo into a digest-pinned node image and
-still exit 0. A changing epoch invalidates the check and build layers on every
+runs eslint and prettier as build steps, so a successful build is a clean lint.
-invocation while leaving the dependency layers below them cached, and the
+There is no host lint path: docker is required to lint, and that also works
-missing-argument guard means a bare `docker build .` fails loudly instead of
+where the docker daemon is remote and bind mounts are impossible.
-quietly reporting a green it did not earn.
+
+The formatting check is part of that, not a step beside it. `script/check` and
+`script/precommit` therefore call `script/lint` and stop; neither calls
+`script/fmt-check` as well, which would run prettier a second time over the same
+tree for the same verdict — and the weaker of the two, since the host's prettier
+is whatever the working tree has installed. So `make check` and the pre-commit
+hook both still fail on a badly formatted tree, and prettier runs exactly once
+in each. `test/packaging/lint-once.test.ts` asserts that count by walking the
+invocation graph, so a second pass cannot creep back in unnoticed.
+
+`script/fmt-check` remains as a standalone entrypoint for asking the formatting
+question on its own, without docker and without the rest of lint. Its verdict
+cannot drift from the container's: prettier is pinned to an exact version,
+installed from `yarn.lock` under `--frozen-lockfile` in both places, and reads
+`.gitignore` as its default ignore file — which is why `.dockerignore`
+deliberately keeps `.gitignore` in the build context.
+
+Lint happens in exactly one place, which constrains the rest of the build.
+`script/check` calls `script/lint`, so `make check` cannot run inside a
+container without asking for docker inside docker. The image built from
+`Dockerfile` therefore runs `make test` and `make build` and does not lint;
+`script/cibuild` builds `Dockerfile.lint` first and that image second, so CI
+gets both verdicts.
+
+### Build epochs
+
+`script/lint` passes `--build-arg LINT_EPOCH="$(date +%s)"`, and `script/docker`
+and `script/cibuild` pass `--build-arg CHECK_EPOCH="$(date +%s)"`. Both
+Dockerfiles refuse to build without their argument. This is deliberate: on an
+unchanged tree Docker would otherwise serve the linter and test layers from
+cache, so nothing would run and the build would still exit 0 — a lint build over
+an untouched tree returns success in well under a second, having linted nothing.
+A changing epoch invalidates every layer below the guard on every invocation
+while leaving the dependency layers above them cached, and the missing-argument
+guard means a bare `docker build .` fails loudly instead of quietly reporting a
+green it did not earn: an unset build argument is the empty string, which is a
+perfectly stable cache key.
 
 ## Rationale
 
@@ -145,8 +184,10 @@ All work on quak is test-driven. No exceptions.
 3. Subsequent commits add the implementation and any refactors needed to make
    the tests pass.
 4. A feature branch can only be merged into `main` when `make check` is green.
-   `main` is always green. The Dockerfile runs `make check` and `make build`, so
+   `main` is always green. CI runs `script/cibuild`, which lints via
-   neither a red branch nor one that does not compile can pass CI.
+   `Dockerfile.lint` and then runs `make test` and `make build` in the
+   `Dockerfile` image, so neither a red branch nor one that does not compile can
+   pass CI.
 5. Tests are the canonical API documentation for this library. Every test file
    is commented thoroughly enough that a reader who has never seen quak can
    learn how to use it from the tests alone. Comments explain why a behavior
@@ -160,11 +201,11 @@ All work on quak is test-driven. No exceptions.
    history must still show tests landing before (or with) the matching
    implementation.
 8. The pre-commit hook installed by `make hooks` runs `script/precommit`, which
-   runs the lint and format checks but not the full `make check`. This is
+   runs `script/lint` — eslint and the prettier check, in the container — but
-   deliberate so the TDD red-phase commit (failing tests, no implementation yet)
+   not the tests, and so not the full `make check`. This is deliberate so the
-   can land. The full `make check` runs as part of the image build, which is
+   TDD red-phase commit (failing tests, no implementation yet) can land. The
-   what CI executes via `script/cibuild`, so a red branch still cannot reach
+   suite runs as part of the image build, which is what CI executes via
-   `main`.
+   `script/cibuild`, so a red branch still cannot reach `main`.
 
 ## Design
 
@@ -191,7 +232,8 @@ quak/
         quak.ts         CLI entrypoint (commander.js)
     test/               unit + integration tests (vitest)
     Makefile
-    Dockerfile
+    Dockerfile          test suite and compile
+    Dockerfile.lint     eslint and prettier, as build steps
     package.json
     tsconfig.json
 ```
@@ -478,9 +520,14 @@ documents:
   implementation. Tests are the canonical API documentation and must be
   commented thoroughly. `main` is always green.
 
-- **Required checks before every commit:** `make lint` (eslint + prettier check)
+- **Required checks before every commit:** `make lint` must pass — that is
-  and `make fmt-check` must pass. The pre-commit hook enforces this.
+  eslint plus the prettier check, and it builds `Dockerfile.lint`, so it needs
-  `make check` (which also runs tests) must pass before merging to `main`.
+  docker. The pre-commit hook enforces exactly that. `make check` (which also
+  runs the tests) must pass before merging to `main`. `make fmt-check` is
+  available for a host-side formatting check on its own, but it is not a
+  separate requirement: `make lint` already covers it, and running both would
+  check formatting twice. Never invoke eslint or prettier directly; linting runs
+  in the container only.
 
 - **Formatting:** prettier with 4-space indents and `proseWrap: always` for
   markdown. Use `make fmt` to format. Use `yarn` not `npm`.

TODO.md

@@ -18,6 +18,73 @@ Update the README API reference section to match the current implementation.
 
 # Completed Steps
 
+- 2026-09-22: Streamed decrypted downloads straight to disk instead of buffering
+  a whole file in memory (issue 40, subsumes issue 21).
+  `downloadFile`/`downloadThumbnail` write each secretstream chunk to the temp
+  file as it is decrypted and rename into place only after the stream
+  authenticates on `TAG_FINAL`, so peak memory is bounded by the 4 MiB chunk
+  size rather than the file size. Because the plaintext is no longer buffered,
+  the atomic write moved inside the retry: each attempt stages its own temp file
+  from byte zero and only a complete attempt renames, so a truncated stream
+  still leaves no destination file and a retry replaces the temp cleanly.
+  `writeAtomic` stays exported for small whole-buffer payloads (thumbnails,
+  metadata) via a shared temp-then-rename helper.
+- 2026-09-22: Added resumable, deletion-aware enumeration to `Client` (issue 38,
+  closes issue 7). `collectionsSince`/`filesSince` take a starting cursor,
+  decrypt live records, surface tombstoned ids in a separate `deleted` list (a
+  tombstone has nothing to decrypt, so it is a bare id, not a hollow record),
+  and return the max `updationTime` seen as the cursor to resume from.
+  `filesSince` refuses to loop when the diff reports `hasMore` without advancing
+  the cursor (issue 7). `listCollections`/`listFiles` are now thin wrappers that
+  enumerate from `sinceTime: 0` and drop deletions, so existing callers are
+  unaffected.
+- 2026-09-22: Carried file size, thumbnail size, and the deletion flag through
+  `decryptFile` (issue 37, foundation for the cache/API design). Live files now
+  populate `file.size`/`thumbnail.size` from the server's `info` (left
+  `undefined` when the server omits it), and `isDeleted` is carried from the
+  diff row onto `EnteFile`. No caller change: `listFiles` still filters deleted
+  rows before decrypting. Surfacing a tombstone through decryption belongs to
+  the enumeration unit (issue 38).
+- 2026-08-10: Made `lint-once.test.ts` enforce what its header claims. It walked
+  `make check` only, so it never read `Dockerfile` — the image CI builds through
+  `script/cibuild` — and a second `prettier --check .` could be added there with
+  the suite staying green. The walk now also starts at
+  `.gitea/workflows/check.yml` and follows its `run:` steps, so the graph under
+  test is the one CI executes rather than the one someone assumed it executes.
+  The lockfile assertion was a substring check against the whole of
+  `script/bootstrap`, which has two install sites and so reported the branch the
+  containers never take; the two branches are now resolved separately and every
+  `yarn install` in each is required to be `--frozen-lockfile`. Prettier is
+  counted per occurrence instead of per line, so two invocations chained with
+  `&&` no longer read as one, and edges are followed on counted lines instead of
+  being skipped. Every way for the walk to reach nothing — an unknown target, an
+  unknown script, a missing file, a node with no commands, an unknown node kind
+  — is a thrown error rather than a quiet zero. Every assertion in the file was
+  mutation-tested individually.
+- 2026-08-10: Stopped `make check` running `prettier --check .` twice. Since
+  linting moved into Docker, the duplicate was one container pass and one host
+  pass of the same check: `script/lint` builds `Dockerfile.lint`, which runs
+  prettier as a build step, and `script/check` then called `script/fmt-check` as
+  well. The host call is gone from `script/check` and from `script/precommit`;
+  the container keeps checking formatting, because a successful
+  `Dockerfile.lint` build is what CI treats as proof of a clean tree, and it is
+  also what still fails the pre-commit hook on a badly formatted tree.
+  `script/fmt-check` survives as a standalone entrypoint, whose verdict cannot
+  drift from the container's. A test walks the invocation graph from each
+  entrypoint — through the Makefile shims, the `script/` calls and the
+  `docker build` — and asserts the prettier count, so the duplication cannot
+  come back unnoticed.
+- 2026-08-10: Moved all linting into Docker. `script/lint` builds a new root
+  `Dockerfile.lint`, which copies the repo into the digest-pinned node image and
+  runs eslint and prettier as build steps, so a successful build is a clean
+  lint; no host lint path remains and `yarn lint` is gone from `package.json`. A
+  fail-closed `LINT_EPOCH` guard stops Docker serving the linter layers from
+  cache, which is how a lint build returns success in under a second having
+  linted nothing. The lint stage inside `Dockerfile` and its `COPY --from=lint`
+  ordering hack are gone: that image now runs `make test` and `make build` only,
+  because `script/check` calls `script/lint` and running it in a container would
+  mean docker inside docker. `script/cibuild` builds the lint image first, then
+  the test and build image.
 - 2026-08-09: Made `make docker` green and policy-conformant. Multi-stage
   Dockerfile: a lint stage runs `make fmt-check` and `make lint`, and the check
   stage takes a `COPY --from=lint` dependency on it before running `make check`

package.json

@@ -24,7 +24,6 @@
         "build": "script/build",
         "quak": "node ./dist/bin/quak.js",
         "test": "vitest run",
-        "lint": "eslint .",
         "fmt": "prettier --write .",
         "fmt-check": "prettier --check ."
     },

script/check

@@ -1,6 +1,19 @@
 #!/bin/sh
-# script/check: run all checks (test, lint, fmt-check). Our own
+# script/check: run all checks (test, lint). Our own extension to
-# extension to scripts-to-rule-them-all. Must not modify any files.
+# scripts-to-rule-them-all. Must not modify any files.
+#
+# The formatting check is part of lint, not a step of its own:
+# script/lint builds Dockerfile.lint, which runs eslint AND
+# `prettier --check .` as build steps. Calling script/fmt-check here as
+# well would run prettier a second time over the same tree for the same
+# verdict — the weaker of the two, since the host toolchain is whatever
+# the working tree happens to have installed while the container's is
+# digest-pinned. script/fmt-check remains a standalone entrypoint for
+# asking the formatting question by itself.
+#
+# script/lint builds Dockerfile.lint, so this script requires docker and
+# must never be run from inside a container: that is why the Dockerfile
+# image runs script/test and script/build rather than this.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -8,7 +21,6 @@ SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
 main() {
     "$SCRIPT_DIR/test"
     "$SCRIPT_DIR/lint"
-    "$SCRIPT_DIR/fmt-check"
 }
 
 main "$@"

script/cibuild

@@ -1,16 +1,23 @@
 #!/bin/sh
-# script/cibuild: run the CI build. The Dockerfile runs script/check and
+# script/cibuild: run the CI build, which is both images in a defined order.
-# script/build, and CHECK_EPOCH differs on every invocation, so those two
+#
-# layers cannot be served from Docker's cache: a green build here means
+# First script/lint, which builds Dockerfile.lint and is the one and only
-# the checks ran now, not that a previous run was remembered. The layers
+# place linting happens — it goes first so a lint failure is reported before
-# below the epoch (bootstrap, yarn install) are unaffected and stay
+# the slower suite runs. Then the Dockerfile image, which runs script/test
-# cached. A build that omits the argument fails by design.
+# and script/build. CHECK_EPOCH and LINT_EPOCH differ on every invocation, so
+# neither the linters nor the suite can be served from Docker's cache: a
+# green build here means the checks ran now, not that a previous run was
+# remembered. The layers below the epochs (bootstrap, yarn install) are
+# unaffected and stay cached. A build that omits the arguments fails by
+# design.
 set -eu
 
-ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
 
 main() {
     cd "$ROOT"
+    "$SCRIPT_DIR/lint"
     docker build --build-arg CHECK_EPOCH="$(date +%s)" .
 }
 

script/docker

@@ -3,7 +3,8 @@
 # Identical in all repos; the tag comes from script/projectname.
 # CHECK_EPOCH is passed for the same reason script/cibuild passes it: the
 # Dockerfile refuses to build without it, so that no path to an image can
-# quietly serve the check and build layers from cache.
+# quietly serve the test and build layers from cache. This builds the test
+# and build image only; linting is a separate image, built by script/lint.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"

script/lint

@@ -1,13 +1,24 @@
 #!/bin/sh
-# script/lint: run the linter (eslint plus a prettier check).
+# script/lint: run the linters. eslint and prettier are never run against
+# the working tree from here: linting runs via docker only, one way,
+# everywhere — script/lint builds Dockerfile.lint, which COPYs the repo into
+# the pinned node image and runs the linters as build steps. That works even
+# when the docker daemon is remote and bind mounts are impossible.
+#
+# LINT_EPOCH is passed on every invocation because no lint cache is wanted:
+# on an unchanged tree Docker would otherwise serve the linter layers, having
+# linted nothing, and still exit 0. Dockerfile.lint refuses to build without
+# the argument, so no path to a lint result can quietly come from cache.
+#
+# Nothing that runs inside a container may call this script; see the header
+# of Dockerfile.
 set -eu
 
 ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
 
 main() {
     cd "$ROOT"
-    yarn run eslint .
+    docker build --build-arg LINT_EPOCH="$(date +%s)" -f Dockerfile.lint .
-    yarn run prettier --check .
 }
 
 main "$@"

script/precommit

@@ -2,17 +2,25 @@
 # script/precommit: run by the git pre-commit hook; fails the commit if
 # checks fail. Our own extension to scripts-to-rule-them-all.
 #
-# Runs lint and fmt-check but deliberately NOT the tests, so the TDD
+# Runs lint but deliberately NOT the tests, so the TDD red-phase commit
-# red-phase commit (failing tests, no implementation yet) can land. CI
+# (failing tests, no implementation yet) can land. CI runs
-# runs make check via docker build, which catches any branch that
+# script/cibuild, which builds both images and so catches any branch
-# ships red.
+# that ships red.
+#
+# The formatting check is still enforced here, because script/lint is a
+# build of Dockerfile.lint and that runs `prettier --check .` as a build
+# step: a badly formatted tree fails this hook, and therefore the
+# commit. Calling script/fmt-check as well would only run prettier a
+# second time over the same tree for the same verdict.
+#
+# script/lint is a docker build (Dockerfile.lint); docker is required to
+# commit, which is the point of linting one way, everywhere.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
 
 main() {
     "$SCRIPT_DIR/lint"
-    "$SCRIPT_DIR/fmt-check"
 }
 
 main "$@"

src/client.ts

@@ -37,6 +37,22 @@ export interface ClientSnapshot {
     publicKey: string;
 }
 
+// The result of a resumable enumeration. Live decrypted records and deleted
+// ids are kept apart on purpose: a tombstone carries no key or metadata to
+// decrypt, so it is a bare id rather than a hollowed-out record. `cursor` is
+// the max `updationTime` seen, to pass back into the next call.
+export interface CollectionsPage {
+    collections: Collection[];
+    deleted: number[];
+    cursor: number;
+}
+
+export interface FilesPage {
+    files: EnteFile[];
+    deleted: number[];
+    cursor: number;
+}
+
 export class Client {
     private readonly api: ApiClient;
     private readonly email: string;
@@ -150,16 +166,28 @@ export class Client {
         this.api.clearAuthToken();
     }
 
-    async listCollections(): Promise<Collection[]> {
+    // Enumerate collections changed since `sinceTime`. Live collections are
+    // decrypted; tombstoned ones (isDeleted) are surfaced as bare ids. The
+    // returned cursor is the max `updationTime` seen — including tombstones, so
+    // the next sync resumes past them — falling back to `sinceTime` when the
+    // response is empty. `/collections/v2` returns the whole changed set in one
+    // response, so there is no pagination here.
+    async collectionsSince(args: {
+        sinceTime: number;
+    }): Promise<CollectionsPage> {
         this.assertLoggedIn();
-        const { collections } = await this.api.getJSON<{
+        const { collections: raws } = await this.api.getJSON<{
             collections: RawCollection[];
-        }>("/collections/v2", { sinceTime: 0 });
+        }>("/collections/v2", { sinceTime: args.sinceTime });
-        // The sync API keeps returning deleted collections as tombstones
+
-        // (isDeleted: true); their diff endpoint 404s, so drop them.
+        const collections: Collection[] = [];
-        return collections
+        const deleted: number[] = [];
-            .filter((raw) => !raw.isDeleted)
+        let cursor = args.sinceTime;
-            .map((raw) =>
+        for (const raw of raws) {
+            if (raw.isDeleted) {
+                deleted.push(raw.id);
+            } else {
+                collections.push(
                     decryptCollection(
                         raw,
                         {
@@ -171,31 +199,78 @@ export class Client {
                     ),
                 );
             }
+            if (raw.updationTime > cursor) cursor = raw.updationTime;
+        }
+        return { collections, deleted, cursor };
+    }
 
-    async listFiles(
+    // Enumerate a collection's files changed since `sinceTime`, paginating the
-        collectionID: number,
+    // diff from that cursor. Live rows are decrypted; tombstoned ones are
-        collectionKey: Uint8Array,
+    // surfaced as bare ids. Returns the final cursor to resume from.
-    ): Promise<EnteFile[]> {
+    async filesSince(args: {
+        collectionID: number;
+        collectionKey: Uint8Array;
+        sinceTime: number;
+    }): Promise<FilesPage> {
         this.assertLoggedIn();
-        const allFiles: EnteFile[] = [];
+        const { collectionID, collectionKey } = args;
-        let sinceTime = 0;
+        const files: EnteFile[] = [];
+        const deleted: number[] = [];
+        let cursor = args.sinceTime;
         for (;;) {
             const { diff, hasMore } = await this.api.getJSON<{
                 diff: RawEnteFile[];
                 hasMore: boolean;
-            }>("/collections/v2/diff", { collectionID, sinceTime });
+            }>("/collections/v2/diff", { collectionID, sinceTime: cursor });
 
+            let pageMax = cursor;
             for (const raw of diff) {
-                if (!raw.isDeleted) {
+                if (raw.isDeleted) {
-                    allFiles.push(decryptFile(raw, collectionKey));
+                    deleted.push(raw.id);
+                } else {
+                    files.push(decryptFile(raw, collectionKey));
                 }
-                if (raw.updationTime > sinceTime) {
+                if (raw.updationTime > pageMax) pageMax = raw.updationTime;
-                    sinceTime = raw.updationTime;
             }
+
+            if (!hasMore) {
+                cursor = pageMax;
+                break;
             }
-            if (!hasMore) break;
+            // The server says there is more, but this page did not advance the
+            // cursor: following hasMore would refetch the same page forever
+            // (#7). Stop with a clear error instead of looping.
+            if (pageMax <= cursor) {
+                throw new Error(
+                    `/collections/v2/diff for collection ${collectionID} ` +
+                        `returned hasMore with a cursor that did not advance ` +
+                        `(stuck at ${cursor}); refusing to loop`,
+                );
             }
-        return allFiles;
+            cursor = pageMax;
+        }
+        return { files, deleted, cursor };
+    }
+
+    // Whole-account listing: every live collection, deletions hidden. A thin
+    // wrapper over `collectionsSince` from the beginning of time.
+    async listCollections(): Promise<Collection[]> {
+        const { collections } = await this.collectionsSince({ sinceTime: 0 });
+        return collections;
+    }
+
+    // Every live file in a collection, deletions hidden. A thin wrapper over
+    // `filesSince` from the beginning of time.
+    async listFiles(
+        collectionID: number,
+        collectionKey: Uint8Array,
+    ): Promise<EnteFile[]> {
+        const { files } = await this.filesSince({
+            collectionID,
+            collectionKey,
+            sinceTime: 0,
+        });
+        return files;
     }
 
     async downloadFile(

src/download/index.ts

@@ -1,5 +1,6 @@
 import { randomUUID } from "node:crypto";
-import { rename, rm, writeFile } from "node:fs/promises";
+import { open, rename, rm } from "node:fs/promises";
+import type { FileHandle } from "node:fs/promises";
 import { dirname, join } from "node:path";
 import {
     fromBase64,
@@ -19,21 +20,53 @@ export interface DownloadResult {
     bytesWritten: number;
 }
 
+// Fired as decrypted plaintext accumulates, with the running total of
+// plaintext bytes recovered so far. Within one download it is non-decreasing
+// and its last value equals the final `bytesWritten`. A retry restarts the
+// file from byte zero (see `fetchAndDecrypt`), so a fresh attempt begins its
+// own count from zero.
+export type ProgressCallback = (bytesDone: number) => void;
+
 const ENC_CHUNK_SIZE = STREAM_CHUNK_SIZE + STREAM_CHUNK_OVERHEAD;
 
+// Decrypt a secretstream body, handing each plaintext chunk to `sink` as it is
+// produced rather than accumulating the whole file. Peak memory is one
+// ciphertext chunk of network buffer plus one plaintext chunk — bounded by
+// `STREAM_CHUNK_SIZE` regardless of the file's size — so a multi-gigabyte video
+// no longer needs its size again in RAM. Returns the total plaintext length.
+//
+// The truncation contract is exactly the buffered version's, only the sink is
+// new: a body cut short still decrypts and authenticates up to its last whole
+// chunk, so the absence of TAG_FINAL is the sole evidence it was cut short, and
+// this throws rather than let a caller keep a short file. The sink has already
+// seen those chunks by then; the caller (`decryptToTemp`) stages them in a temp
+// file that is renamed into place only on a clean return, so a throw leaves
+// nothing on disk.
 const streamDecrypt = async (
     stream: ReadableStream<Uint8Array>,
     header: Uint8Array,
     key: Uint8Array,
-): Promise<Uint8Array> => {
+    sink: (plaintext: Uint8Array) => Promise<void>,
+    onProgress?: ProgressCallback,
+): Promise<number> => {
     const state = initStreamPull(header, key);
     const reader = stream.getReader();
     let buffer = new Uint8Array(0);
-    const plainChunks: Uint8Array[] = [];
     let totalPlain = 0;
     let chunksPulled = 0;
     let lastTag = -1;
 
+    const consume = async (
+        plaintext: Uint8Array,
+        tag: number,
+    ): Promise<void> => {
+        await sink(plaintext);
+        totalPlain += plaintext.length;
+        chunksPulled++;
+        lastTag = tag;
+        onProgress?.(totalPlain);
+    };
+
     for (;;) {
         const { done, value } = await reader.read();
         if (value) {
@@ -46,11 +79,10 @@ const streamDecrypt = async (
         while (buffer.length >= ENC_CHUNK_SIZE) {
             const encChunk = buffer.slice(0, ENC_CHUNK_SIZE);
             buffer = buffer.slice(ENC_CHUNK_SIZE);
+            // A whole chunk that fails to authenticate while the stream carries
+            // on is corruption, not truncation; that error propagates unchanged.
             const { plaintext, tag } = pullStreamChunk(state, encChunk);
-            plainChunks.push(plaintext);
+            await consume(plaintext, tag);
-            totalPlain += plaintext.length;
-            chunksPulled++;
-            lastTag = tag;
         }
 
         if (done) {
@@ -62,7 +94,9 @@ const streamDecrypt = async (
                 // ordinary shape of a dropped connection. Poly1305 cannot
                 // tell a partial chunk from a corrupt one, so this is
                 // reported as the truncation it almost always is, with the
-                // authentication failure kept as the error's cause.
+                // authentication failure kept as the error's cause. Only the
+                // pull is guarded: a sink failure on a chunk that did
+                // authenticate is a disk error, not a truncation.
                 let pulled;
                 try {
                     pulled = pullStreamChunk(state, buffer);
@@ -72,10 +106,7 @@ const streamDecrypt = async (
                         { cause: err },
                     );
                 }
-                plainChunks.push(pulled.plaintext);
+                await consume(pulled.plaintext, pulled.tag);
-                totalPlain += pulled.plaintext.length;
-                chunksPulled++;
-                lastTag = pulled.tag;
             }
             break;
         }
@@ -84,8 +115,6 @@ const streamDecrypt = async (
     // Only the last chunk of a secretstream carries TAG_FINAL. Everything a
     // dropped connection did deliver still decrypts and authenticates, so the
     // absence of TAG_FINAL is the only evidence that the body was cut short.
-    // Returning a short plaintext here would put a corrupt file on disk that
-    // later backup runs would treat as complete.
     if (chunksPulled === 0) {
         throw new TruncatedStreamError(
             "download: stream truncated: response body contained no secretstream chunks",
@@ -97,31 +126,54 @@ const streamDecrypt = async (
             `download: stream truncated: last chunk tag ${lastTag}, expected TAG_FINAL (${tagFinal})`,
         );
     }
-
+    return totalPlain;
-    const result = new Uint8Array(totalPlain);
-    let offset = 0;
-    for (const chunk of plainChunks) {
-        result.set(chunk, offset);
-        offset += chunk.length;
-    }
-    return result;
 };
 
-// Write `plaintext` to `destination` atomically: stage it in a temporary
+// Stage a write to `destination` atomically and durably, then rename it into
-// sibling file (same directory, so the rename cannot cross a filesystem
+// place. `fill` writes the contents into the open temp file handle — either the
-// boundary) and rename it into place. Callers therefore never observe a
+// whole buffer at once (`writeAtomic`) or chunk by chunk as they decrypt
-// partially written destination, and a pre-existing file at that path is
+// (`decryptToTemp`). The temp file is a sibling of the destination (same
+// directory, so the rename cannot cross a filesystem boundary), so callers
+// never observe a partially written destination, and a pre-existing file is
 // replaced only once the new contents are complete on disk.
-const writeAtomic = async (
+//
+// Durability against a power cut needs two fsyncs. Without them the write can
+// return while the data or the rename is still only in the kernel's page
+// cache, and a crash then resurrects an empty renamed file — exactly the
+// corruption a later backup run treats as a complete download. So the temp
+// file's contents are fsynced before the rename, and the containing directory
+// is fsynced after it, so both the bytes and the new directory entry are on
+// stable storage before this returns.
+//
+// On any failure — including a `fill` that throws because the stream was
+// truncated — the temp file is removed, so the destination is untouched and no
+// scratch file is left to fill the disk on repeated failures.
+const stageAtomic = async (
     destination: string,
-    plaintext: Uint8Array,
+    fill: (handle: FileHandle) => Promise<void>,
 ): Promise<void> => {
+    const dir = dirname(destination);
     // The random suffix keeps concurrent downloads of the same destination
     // from stepping on each other's temporary file.
-    const tmpPath = join(dirname(destination), `.quak-${randomUUID()}.tmp`);
+    const tmpPath = join(dir, `.quak-${randomUUID()}.tmp`);
     try {
-        await writeFile(tmpPath, plaintext);
+        const handle = await open(tmpPath, "w");
+        try {
+            await fill(handle);
+            await handle.sync();
+        } finally {
+            await handle.close();
+        }
         await rename(tmpPath, destination);
+        // Fsync the directory so the rename itself survives a crash: renaming
+        // over a synced temp file still leaves the new directory entry in the
+        // page cache until the directory is synced.
+        const dirHandle = await open(dir, "r");
+        try {
+            await dirHandle.sync();
+        } finally {
+            await dirHandle.close();
+        }
     } catch (err) {
         // Best-effort cleanup. A failure to remove the temporary file must
         // never replace the error that actually explains what went wrong.
@@ -130,7 +182,44 @@ const writeAtomic = async (
     }
 };
 
-// Fetch a stream and decrypt it, retrying the whole sequence.
+// Write `plaintext` to `destination` atomically and durably. Exported so the
+// metadata store can reuse the same durable write for small whole-buffer
+// payloads; originals go through `decryptToTemp` instead so they never buffer.
+export const writeAtomic = async (
+    destination: string,
+    plaintext: Uint8Array,
+): Promise<void> =>
+    stageAtomic(destination, (handle) => handle.writeFile(plaintext));
+
+// Decrypt `stream` straight to `destination`, one plaintext chunk at a time,
+// under the atomic writer's temp-then-rename discipline. Memory stays bounded
+// by the chunk size: each decrypted chunk is written to the temp file and
+// dropped. The rename happens only after the stream authenticates as terminated
+// on TAG_FINAL; a truncated stream throws and leaves the destination untouched.
+// Returns the plaintext length written.
+const decryptToTemp = async (
+    destination: string,
+    stream: ReadableStream<Uint8Array>,
+    header: Uint8Array,
+    key: Uint8Array,
+    onProgress?: ProgressCallback,
+): Promise<number> => {
+    let bytesWritten = 0;
+    await stageAtomic(destination, async (handle) => {
+        bytesWritten = await streamDecrypt(
+            stream,
+            header,
+            key,
+            async (plaintext) => {
+                await handle.write(plaintext);
+            },
+            onProgress,
+        );
+    });
+    return bytesWritten;
+};
+
+// Fetch a stream and decrypt it to `destination`, retrying the whole sequence.
 //
 // The request is only the first third of a download. `getXStream` returns as
 // soon as headers arrive, and the bytes are pulled here, so a socket reset
@@ -143,53 +232,60 @@ const writeAtomic = async (
 // four attempts would mean sixteen requests for one file. The policy comes
 // from the client so a caller that configured one gets it here too.
 //
-// A retry starts the file over from byte zero: the secretstream pull state is
+// Because the plaintext is streamed to disk rather than buffered, the atomic
-// not resumable and there is no Range support on these endpoints.
+// write is part of the retried unit. A retry starts the file over from byte
+// zero — the secretstream pull state is not resumable and there is no Range
+// support — staging into a fresh temp file each time: a failed attempt writes
+// and then removes its own temp file, and only the attempt that reaches
+// TAG_FINAL renames one into place, so a download that needed three tries still
+// performs exactly one rename over the destination.
 const fetchAndDecrypt = async (
     api: ApiClient,
     openStream: () => Promise<ReadableStream<Uint8Array>>,
     header: Uint8Array,
     key: Uint8Array,
-): Promise<Uint8Array> =>
+    destination: string,
+    onProgress?: ProgressCallback,
+): Promise<number> =>
     withRetry(async () => {
         const stream = await openStream();
-        return streamDecrypt(stream, header, key);
+        return decryptToTemp(destination, stream, header, key, onProgress);
     }, api.getRetryOptions());
 
 export const downloadFile = async (
     api: ApiClient,
     file: EnteFile,
     outPath?: string,
+    onProgress?: ProgressCallback,
 ): Promise<DownloadResult> => {
     const resolvedPath = outPath ?? file.metadata.title;
     const header = fromBase64(file.file.decryptionHeader);
-    const plaintext = await fetchAndDecrypt(
+    const bytesWritten = await fetchAndDecrypt(
         api,
         () => api.getFileStream(file.id, { retry: false }),
         header,
         file.key,
+        resolvedPath,
+        onProgress,
     );
-    // Outside the retry, deliberately: only the attempt that produced a
+    return { path: resolvedPath, bytesWritten };
-    // complete, authenticated plaintext gets to stage a temporary file, so a
-    // download that needed three tries still performs exactly one write and
-    // one rename.
-    await writeAtomic(resolvedPath, plaintext);
-    return { path: resolvedPath, bytesWritten: plaintext.length };
 };
 
 export const downloadThumbnail = async (
     api: ApiClient,
     file: EnteFile,
     outPath?: string,
+    onProgress?: ProgressCallback,
 ): Promise<DownloadResult> => {
     const resolvedPath = outPath ?? `thumb_${file.metadata.title}`;
     const header = fromBase64(file.thumbnail.decryptionHeader);
-    const plaintext = await fetchAndDecrypt(
+    const bytesWritten = await fetchAndDecrypt(
         api,
         () => api.getThumbnailStream(file.id, { retry: false }),
         header,
         file.key,
+        resolvedPath,
+        onProgress,
     );
-    await writeAtomic(resolvedPath, plaintext);
+    return { path: resolvedPath, bytesWritten };
-    return { path: resolvedPath, bytesWritten: plaintext.length };
 };

src/index.ts

@@ -1,6 +1,12 @@
 export const VERSION = "0.0.0";
 
-export { Client, type LoginOptions, type ClientSnapshot } from "./client.js";
+export {
+    Client,
+    type LoginOptions,
+    type ClientSnapshot,
+    type CollectionsPage,
+    type FilesPage,
+} from "./client.js";
 export {
     ApiClient,
     ApiError,

src/model/decrypt.ts

@@ -120,9 +120,16 @@ export const decryptFile = (
         metadata,
         magicMetadata,
         pubMagicMetadata,
-        file: { decryptionHeader: raw.file.decryptionHeader },
+        file: {
-        thumbnail: { decryptionHeader: raw.thumbnail.decryptionHeader },
+            decryptionHeader: raw.file.decryptionHeader,
+            size: raw.info?.fileSize,
+        },
+        thumbnail: {
+            decryptionHeader: raw.thumbnail.decryptionHeader,
+            size: raw.info?.thumbSize,
+        },
         updationTime: raw.updationTime,
+        isDeleted: raw.isDeleted,
     };
 };
 

src/model/types.ts

@@ -48,6 +48,9 @@ export interface EnteFile {
     file: FileBlob;
     thumbnail: FileBlob;
     updationTime: Microseconds;
+    // Set from the diff row's flag. Live files decode with it absent/false;
+    // deleted rows are filtered out before decryptFile, so it is not set here.
+    isDeleted?: boolean;
 }
 
 // The key material a logged-in client holds, everything needed to decrypt

test/client/enumeration.test.ts

@@ -0,0 +1,367 @@
+/**
+ * Tests for the resumable, deletion-aware enumeration variants on `Client`:
+ * `collectionsSince` and `filesSince`.
+ *
+ * The whole-account methods `listCollections` / `listFiles` always start at
+ * `sinceTime: 0` and hide deletions. The cache refresh needs the opposite:
+ * start from a saved cursor, learn what was deleted, and get back a cursor to
+ * resume from next time. These two methods provide that.
+ *
+ * The return shape keeps live records and tombstones apart — `collections` /
+ * `files` are decrypted live records, `deleted` is a plain list of the ids the
+ * server tombstoned. A tombstone carries no decryptable key or metadata, so it
+ * is a bare id rather than a hollowed-out `Collection` / `EnteFile`.
+ *
+ * All tests inject a fake `fetch` and drive a real `Client` (built with
+ * `Client.fromJSON`) so the decryption path runs for real. Live rows are built
+ * with libsodium exactly as the server would encrypt them; tombstone rows carry
+ * only the fields the code reads (`id`, `updationTime`, `isDeleted`), because
+ * they are never decrypted.
+ */
+
+import sodium from "libsodium-wrappers-sumo";
+import { beforeAll, describe, expect, it } from "vitest";
+import { init, toBase64 } from "../../src/crypto/index.js";
+import { Client, type ClientSnapshot } from "../../src/client.js";
+
+// ---------------------------------------------------------------------------
+// Fixtures
+// ---------------------------------------------------------------------------
+
+const USER_ID = 42;
+
+interface Keys {
+    masterKey: Uint8Array;
+    publicKey: Uint8Array;
+    secretKey: Uint8Array;
+}
+
+const buildKeys = (): Keys => {
+    const kp = sodium.crypto_box_keypair();
+    return {
+        masterKey: sodium.crypto_secretbox_keygen(),
+        publicKey: kp.publicKey,
+        secretKey: kp.privateKey,
+    };
+};
+
+const snapshotFor = (keys: Keys): ClientSnapshot => ({
+    email: "user@example.com",
+    userID: USER_ID,
+    token: "test-token",
+    masterKey: toBase64(keys.masterKey),
+    secretKey: toBase64(keys.secretKey),
+    publicKey: toBase64(keys.publicKey),
+});
+
+const secretboxEncrypt = (
+    plaintext: Uint8Array,
+    key: Uint8Array,
+): { ciphertext: Uint8Array; nonce: Uint8Array } => {
+    const nonce = sodium.randombytes_buf(sodium.crypto_secretbox_NONCEBYTES);
+    return {
+        ciphertext: sodium.crypto_secretbox_easy(plaintext, nonce, key),
+        nonce,
+    };
+};
+
+/** An owned collection row as the server sends it, keyed under the master key. */
+const ownedCollectionRow = (
+    masterKey: Uint8Array,
+    opts: { id: number; name: string; updationTime: number },
+): Record<string, unknown> => {
+    const collectionKey = sodium.crypto_secretbox_keygen();
+    const { ciphertext: encKey, nonce: keyNonce } = secretboxEncrypt(
+        collectionKey,
+        masterKey,
+    );
+    const { ciphertext: encName, nonce: nameNonce } = secretboxEncrypt(
+        new TextEncoder().encode(opts.name),
+        collectionKey,
+    );
+    return {
+        id: opts.id,
+        owner: { id: USER_ID },
+        encryptedKey: toBase64(encKey),
+        keyDecryptionNonce: toBase64(keyNonce),
+        encryptedName: toBase64(encName),
+        nameDecryptionNonce: toBase64(nameNonce),
+        type: "album",
+        updationTime: opts.updationTime,
+    };
+};
+
+/** A live file row inside a collection, keyed under that collection's key. */
+const fileRow = (
+    collectionKey: Uint8Array,
+    opts: { id: number; title: string; updationTime: number },
+): Record<string, unknown> => {
+    const fileKey = sodium.crypto_secretbox_keygen();
+    const { ciphertext: encFileKey, nonce: fileKeyNonce } = secretboxEncrypt(
+        fileKey,
+        collectionKey,
+    );
+    const metadata = {
+        title: opts.title,
+        fileType: 0,
+        creationTime: opts.updationTime,
+        modificationTime: opts.updationTime,
+    };
+    const push =
+        sodium.crypto_secretstream_xchacha20poly1305_init_push(fileKey);
+    const encMeta = sodium.crypto_secretstream_xchacha20poly1305_push(
+        push.state,
+        new TextEncoder().encode(JSON.stringify(metadata)),
+        null,
+        sodium.crypto_secretstream_xchacha20poly1305_TAG_FINAL,
+    );
+    return {
+        id: opts.id,
+        collectionID: 1,
+        ownerID: USER_ID,
+        encryptedKey: toBase64(encFileKey),
+        keyDecryptionNonce: toBase64(fileKeyNonce),
+        metadata: {
+            encryptedData: toBase64(encMeta),
+            decryptionHeader: toBase64(push.header),
+        },
+        file: { decryptionHeader: toBase64(sodium.randombytes_buf(24)) },
+        thumbnail: { decryptionHeader: toBase64(sodium.randombytes_buf(24)) },
+        updationTime: opts.updationTime,
+    };
+};
+
+/** A tombstone row. Never decrypted, so only these fields are ever read. */
+const tombstoneRow = (
+    id: number,
+    updationTime: number,
+): Record<string, unknown> => ({
+    id,
+    updationTime,
+    isDeleted: true,
+});
+
+const jsonResponse = (body: unknown): Response =>
+    new Response(JSON.stringify(body), {
+        status: 200,
+        headers: { "content-type": "application/json" },
+    });
+
+/**
+ * A fetch that serves canned responses in order and records the `sinceTime`
+ * query parameter each request carried, so tests can prove the cursor is
+ * threaded from one page (and one call) to the next.
+ */
+const recordingFetch = (
+    ...responses: Response[]
+): { fetch: typeof globalThis.fetch; sinceTimes: (string | null)[] } => {
+    const sinceTimes: (string | null)[] = [];
+    let i = 0;
+    const fake = async (input: RequestInfo | URL): Promise<Response> => {
+        const url =
+            typeof input === "string"
+                ? input
+                : input instanceof URL
+                  ? input.href
+                  : input.url;
+        sinceTimes.push(new URL(url).searchParams.get("sinceTime"));
+        if (i >= responses.length) {
+            throw new Error(`recordingFetch: no response for call #${i}`);
+        }
+        return responses[i++]!;
+    };
+    return { fetch: fake as typeof globalThis.fetch, sinceTimes };
+};
+
+// ---------------------------------------------------------------------------
+// Tests
+// ---------------------------------------------------------------------------
+
+describe("Client.filesSince", () => {
+    beforeAll(async () => {
+        await init();
+        await sodium.ready;
+    });
+
+    it("pages from the given cursor, decrypts live rows, and collects tombstones", async () => {
+        const keys = buildKeys();
+        const collectionKey = sodium.crypto_secretbox_keygen();
+
+        // Page 1 mixes a live file and a tombstone; the tombstone has the
+        // higher updationTime, so it — not the live row — sets the cursor the
+        // second page must be fetched from.
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({
+                diff: [
+                    fileRow(collectionKey, {
+                        id: 1001,
+                        title: "first.jpg",
+                        updationTime: 100,
+                    }),
+                    tombstoneRow(1002, 150),
+                ],
+                hasMore: true,
+            }),
+            jsonResponse({
+                diff: [
+                    fileRow(collectionKey, {
+                        id: 1003,
+                        title: "second.jpg",
+                        updationTime: 200,
+                    }),
+                ],
+                hasMore: false,
+            }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const { files, deleted, cursor } = await client.filesSince({
+            collectionID: 1,
+            collectionKey,
+            sinceTime: 0,
+        });
+
+        expect(files.map((f) => f.id)).toEqual([1001, 1003]);
+        expect(files.map((f) => f.metadata.title)).toEqual([
+            "first.jpg",
+            "second.jpg",
+        ]);
+        expect(deleted).toEqual([1002]);
+        expect(cursor).toBe(200);
+        // First request started at the caller's cursor; the second resumed
+        // from the max updationTime seen on the first page (the tombstone's).
+        expect(sinceTimes).toEqual(["0", "150"]);
+    });
+
+    it("fetches only newer rows when the returned cursor is passed back in", async () => {
+        const keys = buildKeys();
+        const collectionKey = sodium.crypto_secretbox_keygen();
+
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({ diff: [], hasMore: false }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const result = await client.filesSince({
+            collectionID: 1,
+            collectionKey,
+            sinceTime: 200,
+        });
+
+        expect(result.files).toEqual([]);
+        expect(result.deleted).toEqual([]);
+        // An empty diff advances nothing: the cursor falls back to the input.
+        expect(result.cursor).toBe(200);
+        expect(sinceTimes).toEqual(["200"]);
+    });
+
+    it("stops and throws when the server claims more but does not advance (#7)", async () => {
+        const keys = buildKeys();
+        const collectionKey = sodium.crypto_secretbox_keygen();
+
+        // hasMore is true, but the page's max updationTime (50) does not exceed
+        // the cursor the request was made with (50). Following hasMore here
+        // would refetch this same page forever.
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({ diff: [tombstoneRow(1, 50)], hasMore: true }),
+            jsonResponse({ diff: [tombstoneRow(1, 50)], hasMore: true }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        await expect(
+            client.filesSince({
+                collectionID: 1,
+                collectionKey,
+                sinceTime: 50,
+            }),
+        ).rejects.toThrow(/not advance|non-advancing/i);
+        // It gave up after the first page rather than looping.
+        expect(sinceTimes).toEqual(["50"]);
+    });
+});
+
+describe("Client.collectionsSince", () => {
+    beforeAll(async () => {
+        await init();
+        await sodium.ready;
+    });
+
+    it("decrypts live collections, collects tombstones, and returns a cursor", async () => {
+        const keys = buildKeys();
+
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({
+                collections: [
+                    ownedCollectionRow(keys.masterKey, {
+                        id: 1,
+                        name: "Vacation",
+                        updationTime: 100,
+                    }),
+                    tombstoneRow(3, 150),
+                ],
+            }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const { collections, deleted, cursor } = await client.collectionsSince({
+            sinceTime: 0,
+        });
+
+        expect(collections.map((c) => c.id)).toEqual([1]);
+        expect(collections[0]!.name).toBe("Vacation");
+        expect(deleted).toEqual([3]);
+        // The tombstone's updationTime advances the cursor too, so the next
+        // sync starts after it rather than seeing it again.
+        expect(cursor).toBe(150);
+        expect(sinceTimes).toEqual(["0"]);
+    });
+
+    it("falls back to the input cursor on an empty response", async () => {
+        const keys = buildKeys();
+
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({ collections: [] }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const result = await client.collectionsSince({ sinceTime: 150 });
+
+        expect(result.collections).toEqual([]);
+        expect(result.deleted).toEqual([]);
+        expect(result.cursor).toBe(150);
+        expect(sinceTimes).toEqual(["150"]);
+    });
+});
+
+describe("Client list wrappers still hide deletions", () => {
+    beforeAll(async () => {
+        await init();
+        await sodium.ready;
+    });
+
+    it("listFiles drops tombstones and returns only live files", async () => {
+        const keys = buildKeys();
+        const collectionKey = sodium.crypto_secretbox_keygen();
+
+        const { fetch, sinceTimes } = recordingFetch(
+            jsonResponse({
+                diff: [
+                    fileRow(collectionKey, {
+                        id: 7,
+                        title: "keep.jpg",
+                        updationTime: 100,
+                    }),
+                    tombstoneRow(8, 150),
+                ],
+                hasMore: false,
+            }),
+        );
+        const client = Client.fromJSON(snapshotFor(keys), { fetch });
+
+        const files = await client.listFiles(1, collectionKey);
+
+        expect(files.map((f) => f.id)).toEqual([7]);
+        // The wrapper starts a full enumeration from zero.
+        expect(sinceTimes).toEqual(["0"]);
+    });
+});

test/download/download.test.ts

@@ -72,7 +72,11 @@ import { init, toBase64, STREAM_CHUNK_SIZE } from "../../src/crypto/index.js";
 import { ApiClient } from "../../src/api/client.js";
 import { ApiError, TruncatedStreamError } from "../../src/errors.js";
 import type { RetryOptions } from "../../src/retry.js";
-import { downloadFile, downloadThumbnail } from "../../src/download/index.js";
+import {
+    downloadFile,
+    downloadThumbnail,
+    writeAtomic,
+} from "../../src/download/index.js";
 import type { EnteFile, FileMetadata } from "../../src/model/types.js";
 
 // ---------------------------------------------------------------------------
@@ -101,17 +105,79 @@ const renameHook = vi.hoisted(() => ({
     failWith: null as Error | null,
 }));
 
+/**
+ * `open` is wrapped so the tests can observe the durability fsyncs the atomic
+ * writer performs — which are otherwise invisible: an fsync leaves no trace in
+ * the file's contents. Each `FileHandle.sync()` is recorded, and rename and
+ * sync events are appended to a single ordered `events` log so a test can pin
+ * the sequence "fsync the temp file, rename, fsync the directory" that makes a
+ * write survive a power cut. The flag the handle was opened with distinguishes
+ * the temp file (`w`) from its containing directory (`r`).
+ */
+const durabilityHook = vi.hoisted(() => ({
+    events: [] as string[],
+}));
+
+/**
+ * `FileHandle.write` is wrapped so the tests can watch the streaming decrypt
+ * path put plaintext on disk one chunk at a time. This is the direct evidence
+ * that memory is bounded by the chunk size and not the file size: a buffered
+ * downloader would hand the whole file to a single write, whereas the streaming
+ * one issues one write per secretstream chunk, none larger than
+ * `STREAM_CHUNK_SIZE`. Each write records the temp path it targeted and its
+ * length. `writeFile` (which the whole-buffer `writeAtomic` uses) is a distinct
+ * native call and does not go through this method, so only the streaming path
+ * is observed here.
+ */
+const writeHook = vi.hoisted(() => ({
+    writes: [] as { path: string; length: number }[],
+}));
+
 vi.mock("node:fs/promises", async (importOriginal) => {
     const actual = await importOriginal<typeof import("node:fs/promises")>();
     const { existsSync: sourceExists } = await import("node:fs");
     return {
         ...actual,
+        open: async (
+            path: Parameters<typeof actual.open>[0],
+            flags?: Parameters<typeof actual.open>[1],
+            ...rest: unknown[]
+        ): Promise<Awaited<ReturnType<typeof actual.open>>> => {
+            const handle = await actual.open(
+                path,
+                flags as Parameters<typeof actual.open>[1],
+                ...(rest as []),
+            );
+            const realSync = handle.sync.bind(handle);
+            handle.sync = async (): Promise<void> => {
+                durabilityHook.events.push(`sync:${String(flags)}:${path}`);
+                await realSync();
+            };
+            const realWrite = handle.write.bind(handle);
+            handle.write = (async (
+                data: unknown,
+                ...rest2: unknown[]
+            ): Promise<unknown> => {
+                if (data instanceof Uint8Array) {
+                    writeHook.writes.push({
+                        path: String(path),
+                        length: data.length,
+                    });
+                }
+                return (realWrite as (...a: unknown[]) => Promise<unknown>)(
+                    data,
+                    ...rest2,
+                );
+            }) as typeof handle.write;
+            return handle;
+        },
         rename: async (from: string, to: string): Promise<void> => {
             renameHook.calls.push({
                 from,
                 to,
                 sourceExisted: sourceExists(from),
             });
+            durabilityHook.events.push(`rename:${to}`);
             if (renameHook.failWith !== null) {
                 throw renameHook.failWith;
             }
@@ -123,6 +189,8 @@ vi.mock("node:fs/promises", async (importOriginal) => {
 beforeEach(() => {
     renameHook.calls.length = 0;
     renameHook.failWith = null;
+    durabilityHook.events.length = 0;
+    writeHook.writes.length = 0;
 });
 
 let testDir: string;
@@ -911,10 +979,13 @@ describe.each(entryPoints)("$name retries", ({ name, download }) => {
     });
 
     it("stages one temp file for the attempt that succeeded, not one per attempt", async () => {
-        // The atomic write stays outside the retry loop. A retried download
+        // Each streaming attempt stages into its own temp file, but a retried
-        // must not leave a trail of half-written scratch files, and the
+        // download must not leave a trail of half-written scratch files: a
-        // destination must be touched exactly once — by the attempt that
+        // failed attempt removes its temp file, and the destination is renamed
-        // produced a complete, authenticated plaintext.
+        // into place exactly once — by the attempt that produced a complete,
+        // authenticated plaintext. (Here the two failed attempts reset before a
+        // whole chunk is pulled, so they write nothing; the point stands either
+        // way — see the retry-restart test below, where they do write.)
         const { key, header, ciphertext } = smallFixture(42);
         const { fetch } = scriptedCdnFetch(
             { kind: "reset", bytes: ciphertext.slice(0, 16) },
@@ -1028,6 +1099,99 @@ describe.each(entryPoints)("$name retries", ({ name, download }) => {
     });
 });
 
+// ---------------------------------------------------------------------------
+// Streaming decrypt to disk
+//
+// The plaintext is never held whole in memory: each secretstream chunk is
+// written to the temp file as it is decrypted, so peak memory is bounded by the
+// chunk size rather than the file size. These tests watch the writes directly
+// (see `writeHook`) rather than infer memory behaviour from the final file.
+// ---------------------------------------------------------------------------
+
+describe.each(entryPoints)("$name streams to disk", ({ name, download }) => {
+    const freshDir = (): string =>
+        mkdtempSync(join(testDir, `${name}-stream-`));
+
+    /** Writes recorded against staged temp files (not the `writeFile` path). */
+    const tempWrites = (): { path: string; length: number }[] =>
+        writeHook.writes.filter((w) => w.path.endsWith(".tmp"));
+
+    it("writes one chunk at a time, none larger than STREAM_CHUNK_SIZE", async () => {
+        // The multi-chunk fixture decrypts to one full 4 MiB chunk plus a small
+        // final chunk. A streaming writer therefore issues exactly two writes,
+        // of STREAM_CHUNK_SIZE and then the final chunk's length — never a
+        // single write carrying the whole 4 MiB + 1 KiB file. That per-chunk
+        // shape is what "memory bounded by chunk size" means in practice: the
+        // plaintext is handed to the filesystem and dropped, chunk by chunk.
+        const { api, file } = fixtureFor(
+            multiChunkKey,
+            multiChunk.header,
+            multiChunk.body,
+        );
+        const outPath = join(freshDir(), "streamed.bin");
+
+        const result = await download(api, file, outPath);
+
+        const writes = tempWrites();
+        expect(writes.map((w) => w.length)).toEqual([
+            STREAM_CHUNK_SIZE,
+            multiChunk.plaintext.length - STREAM_CHUNK_SIZE,
+        ]);
+        // No single write ever carried the whole file, and every write fits in
+        // one chunk's worth of memory.
+        for (const w of writes) {
+            expect(w.length).toBeLessThanOrEqual(STREAM_CHUNK_SIZE);
+        }
+        expect(result.bytesWritten).toBe(multiChunk.plaintext.length);
+        expectSameBytes(readFileSync(outPath), multiChunk.plaintext);
+    });
+
+    it("restarts from byte zero on a retry, replacing the temp file cleanly", async () => {
+        // The secretstream pull state is not resumable, so a retry cannot
+        // continue a half-written file — it must start over. The first attempt
+        // here delivers a complete leading chunk and then stops before the
+        // TAG_FINAL chunk: 4 MiB of plaintext lands in a temp file, then the
+        // download is rejected as truncated and that temp file is discarded.
+        // The retry streams the whole body into a *fresh* temp file, so the
+        // destination ends up with exactly the plaintext once — never the
+        // leading chunk twice, and never a stale temp file left behind.
+        const truncatedBody = multiChunk.body.slice(
+            0,
+            multiChunk.finalChunkOffset,
+        );
+        const { fetch, requests } = scriptedCdnFetch(
+            { kind: "body", bytes: truncatedBody },
+            { kind: "body", bytes: multiChunk.body },
+        );
+        const api = new ApiClient({
+            fetch,
+            retry: { ...noWait, attempts: 4 },
+        });
+        const file = buildMockEnteFile(
+            multiChunkKey,
+            multiChunk.header,
+            multiChunk.header,
+        );
+        const dir = freshDir();
+        const outPath = join(dir, "retry-restart.bin");
+
+        const result = await download(api, file, outPath);
+
+        expect(requests()).toBe(2);
+        // Both attempts streamed to disk, each into its own temp file: the
+        // truncated first attempt wrote before it failed, proving the retry did
+        // not resume a partial file but replaced it.
+        const distinctTemps = new Set(tempWrites().map((w) => w.path));
+        expect(distinctTemps.size).toBe(2);
+        // The destination holds the complete plaintext exactly once, and no
+        // temp file survives.
+        expect(result.bytesWritten).toBe(multiChunk.plaintext.length);
+        expectSameBytes(readFileSync(outPath), multiChunk.plaintext);
+        expect(renameHook.calls).toHaveLength(1);
+        expect(readdirSync(dir)).toEqual(["retry-restart.bin"]);
+    });
+});
+
 describe("download retries: corruption is not retried", () => {
     it("gives up immediately on a chunk that failed to authenticate", async () => {
         // A whole chunk that failed to authenticate while the stream
@@ -1061,3 +1225,102 @@ describe("download retries: corruption is not retried", () => {
         expect(requests()).toBe(1);
     });
 });
+
+// ---------------------------------------------------------------------------
+// Durable atomic writes
+//
+// `writeAtomic` is exported so the metadata store can reuse the same
+// power-cut-safe write. Its durability is the point: the bytes and the new
+// directory entry must both be on stable storage before it returns, so a crash
+// immediately afterwards cannot resurrect an empty renamed file (#22 area 1).
+// ---------------------------------------------------------------------------
+
+describe("writeAtomic", () => {
+    it("fsyncs the temp file before the rename and the directory after", async () => {
+        const dir = mkdtempSync(join(testDir, "atomic-"));
+        const dest = join(dir, "durable.bin");
+        const bytes = patternBytes(2048, 71);
+
+        await writeAtomic(dest, bytes);
+
+        expect(readFileSync(dest)).toEqual(Buffer.from(bytes));
+        // The order is the durability contract: fsync the staged temp file so
+        // its contents are on disk, rename it into place, then fsync the
+        // directory so that new entry is on disk too. Do the directory fsync
+        // before the rename, or skip it, and a crash can lose the rename.
+        expect(durabilityHook.events).toHaveLength(3);
+        expect(durabilityHook.events[0]).toMatch(/^sync:w:.*\.tmp$/);
+        expect(durabilityHook.events[1]).toBe(`rename:${dest}`);
+        expect(durabilityHook.events[2]).toBe(`sync:r:${dir}`);
+    });
+
+    it("leaves no temp file behind when the write cannot be renamed", async () => {
+        const dir = mkdtempSync(join(testDir, "atomic-fail-"));
+        const dest = join(dir, "unrenamable.bin");
+        renameHook.failWith = new Error("simulated rename failure");
+
+        await expect(writeAtomic(dest, patternBytes(64, 72))).rejects.toThrow(
+            "simulated rename failure",
+        );
+
+        // The staged temp file was fsynced, then the rename failed; the cleanup
+        // path must remove it so a repeatedly failing write cannot fill the disk.
+        expect(existsSync(dest)).toBe(false);
+        expect(readdirSync(dir)).toEqual([]);
+    });
+});
+
+// ---------------------------------------------------------------------------
+// Per-chunk progress
+//
+// Callers streaming a large file want bytes-written as it lands, not only the
+// final total. The hook fires as decrypted plaintext accumulates; its values
+// are non-decreasing and its last value is exactly `bytesWritten`.
+// ---------------------------------------------------------------------------
+
+describe.each(entryPoints)("$name progress", ({ name, download }) => {
+    it("reports monotonic progress ending at bytesWritten", async () => {
+        // The multi-chunk fixture pulls one full 4 MiB chunk and then a small
+        // final chunk, so the callback fires more than once and monotonicity is
+        // actually observable rather than trivially true for a single fire.
+        const { api, file } = fixtureFor(
+            multiChunkKey,
+            multiChunk.header,
+            multiChunk.body,
+        );
+        const outPath = join(
+            mkdtempSync(join(testDir, `${name}-progress-`)),
+            "p.bin",
+        );
+        const seen: number[] = [];
+
+        const result = await download(api, file, outPath, (bytesDone) => {
+            seen.push(bytesDone);
+        });
+
+        expect(seen.length).toBeGreaterThan(1);
+        for (let i = 1; i < seen.length; i++) {
+            expect(seen[i]!).toBeGreaterThan(seen[i - 1]!);
+        }
+        expect(seen[seen.length - 1]).toBe(result.bytesWritten);
+        expect(result.bytesWritten).toBe(multiChunk.plaintext.length);
+    });
+
+    it("downloads normally when no progress callback is given", async () => {
+        // The callback is optional and its absence must be side-effect-free:
+        // the download succeeds exactly as it does elsewhere in this file.
+        const plaintext = patternBytes(300, 73);
+        const key = sodium.crypto_secretstream_xchacha20poly1305_keygen();
+        const { header, ciphertext } = encryptFileBody(plaintext, key);
+        const { api, file } = fixtureFor(key, header, ciphertext);
+        const outPath = join(
+            mkdtempSync(join(testDir, `${name}-noprog-`)),
+            "n.bin",
+        );
+
+        const result = await download(api, file, outPath);
+
+        expect(result.bytesWritten).toBe(plaintext.length);
+        expectSameBytes(readFileSync(outPath), plaintext);
+    });
+});

test/model/decrypt.test.ts

@@ -145,7 +145,12 @@ const buildSharedRawCollection = (
 
 const buildRawFile = (
     collectionKey: Uint8Array,
-    opts?: { title?: string; fileType?: number; creationTime?: number },
+    opts?: {
+        title?: string;
+        fileType?: number;
+        creationTime?: number;
+        info?: { fileSize?: number; thumbSize?: number };
+    },
 ): RawEnteFile => {
     const fileKey = sodium.crypto_secretbox_keygen();
     const { ciphertext: encFileKey, nonce: fileKeyNonce } = secretboxEncrypt(
@@ -186,6 +191,7 @@ const buildRawFile = (
         },
         file: { decryptionHeader: toBase64(sodium.randombytes_buf(24)) },
         thumbnail: { decryptionHeader: toBase64(sodium.randombytes_buf(24)) },
+        info: opts?.info,
         updationTime: 1700000000000000,
     };
 };
@@ -357,4 +363,47 @@ describe("model.decryptFile", () => {
 
         expect(() => decryptFile(raw, wrongKey)).toThrow();
     });
+
+    it("carries the file and thumbnail byte sizes from info", () => {
+        // The server reports the encrypted-blob sizes in `info`; the cache
+        // needs them without a HEAD request, so decryptFile must copy them
+        // onto the file and thumbnail blobs.
+        const masterKey = sodium.crypto_secretbox_keygen();
+        const { collectionKey } = buildRawCollection(masterKey);
+        const raw = buildRawFile(collectionKey, {
+            info: { fileSize: 4096, thumbSize: 512 },
+        });
+
+        const file = decryptFile(raw, collectionKey);
+
+        expect(file.file.size).toBe(4096);
+        expect(file.thumbnail.size).toBe(512);
+    });
+
+    it("leaves the sizes undefined when the server omits info", () => {
+        // Older files predate the info field; the sizes must stay undefined
+        // rather than become 0, so callers can tell "unknown" from "empty".
+        const masterKey = sodium.crypto_secretbox_keygen();
+        const { collectionKey } = buildRawCollection(masterKey);
+        const raw = buildRawFile(collectionKey);
+        expect(raw.info).toBeUndefined();
+
+        const file = decryptFile(raw, collectionKey);
+
+        expect(file.file.size).toBeUndefined();
+        expect(file.thumbnail.size).toBeUndefined();
+    });
+
+    it("carries the deletion flag from the diff row", () => {
+        // The diff marks a deleted row with isDeleted; decryptFile copies it
+        // onto the file so a caller can tell a deleted row from a live one.
+        const masterKey = sodium.crypto_secretbox_keygen();
+        const { collectionKey } = buildRawCollection(masterKey);
+        const raw = buildRawFile(collectionKey);
+        raw.isDeleted = true;
+
+        const file = decryptFile(raw, collectionKey);
+
+        expect(file.isDeleted).toBe(true);
+    });
 });

test/packaging/build-context.test.ts

@@ -12,7 +12,7 @@
 //
 // Neither shows up as a build failure, so they are asserted here.
 import { describe, expect, it } from "vitest";
-import { readFileSync } from "node:fs";
+import { existsSync, readFileSync } from "node:fs";
 import { fileURLToPath } from "node:url";
 import { join } from "node:path";
 
@@ -46,4 +46,19 @@ describe(".dockerignore", () => {
     it("leaves .gitignore in the build context for prettier", () => {
         expect(dockerignore).not.toContain(".gitignore");
     });
+
+    // Both images are built from this same context, and the lint image runs
+    // eslint and prettier across it. BuildKit lets a `<dockerfile>.dockerignore`
+    // shadow the root one for a single build; such a file would silently give
+    // the lint build a different, unreviewed context — and eslint's flat config
+    // does not ignore dot-directories, so a stray `.claude/` worktree would be
+    // linted.
+    it.each(["Dockerfile", "Dockerfile.lint"])(
+        "is not shadowed by a per-Dockerfile ignore file for %s",
+        (name) => {
+            expect(existsSync(join(repoRoot, `${name}.dockerignore`))).toBe(
+                false,
+            );
+        },
+    );
 });

test/packaging/entrypoints.test.ts

@@ -1,7 +1,9 @@
 // The package manifest promises three files that only exist after a build:
 // `main`, `types`, and the `quak` binary. Nothing in the test suite used to
-// look at them, and `make check` runs test, lint and fmt-check but never the
+// look at them, and `make check` runs the suite and the lint container but
-// build, so `tsconfig.json` and `package.json` were free to drift apart. They
+// never the build, so `tsconfig.json` and `package.json` were free to drift
+// apart. (The formatting check is part of the lint container, not a step of
+// its own; `test/packaging/lint-once.test.ts` is what holds that shape.) They
 // did: `rootDir` was `./src` while `include` also pulled in `bin/**/*`, which
 // is TS6059, and no build had succeeded for as long as that was true.
 //

test/packaging/lint-docker.test.ts

@@ -0,0 +1,184 @@
+// Linting runs in Docker, one way, everywhere: `script/lint` builds
+// `Dockerfile.lint`, which COPYs the repo into a digest-pinned image and runs
+// eslint and prettier as build steps, so a successful build IS a clean lint.
+//
+// Three things can quietly undo that, and none of them shows up as a build
+// failure, which is why they are asserted here:
+//
+// 1. Recursion. `script/check` calls `script/lint`, and `script/lint` is now a
+//    `docker build`. Anything that runs `make check` inside a container is
+//    therefore asking for Docker inside Docker, and CI breaks. The image built
+//    from `Dockerfile` runs the suite and the compile only; lint happens once,
+//    in `Dockerfile.lint`.
+// 2. Cache. A lint build over an unchanged tree returns success in well under a
+//    second having linted nothing. The `LINT_EPOCH` guard is what forces the
+//    linter layers to execute, and it has to fail closed: an unset build
+//    argument is the empty string, which is a perfectly stable cache key, so an
+//    invocation that omits it must be rejected rather than served a cached
+//    green.
+// 3. A host lint path surviving alongside the container one, which would let a
+//    lint result come from an unpinned local toolchain.
+import { describe, expect, it } from "vitest";
+import { readFileSync } from "node:fs";
+import { fileURLToPath } from "node:url";
+import { join } from "node:path";
+
+const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
+
+const read = (name: string): string =>
+    readFileSync(join(repoRoot, name), "utf-8");
+
+// The executable lines of a shell script or Dockerfile: comments carry the
+// reasoning and frequently name the very commands these tests forbid, so they
+// would otherwise trigger every assertion below.
+const instructions = (name: string): string[] =>
+    read(name)
+        .split("\n")
+        .map((line) => line.trim())
+        .filter((line) => line !== "" && !line.startsWith("#"));
+
+const lintScript = instructions("script/lint");
+const dockerfileLint = instructions("Dockerfile.lint");
+const dockerfile = instructions("Dockerfile");
+const cibuild = instructions("script/cibuild");
+
+const has = (lines: string[], pattern: RegExp): boolean =>
+    lines.some((line) => pattern.test(line));
+
+describe("script/lint", () => {
+    it("lints by building Dockerfile.lint", () => {
+        expect(has(lintScript, /docker build .*-f Dockerfile\.lint/)).toBe(
+            true,
+        );
+    });
+
+    // The whole point of the ruling: no invocation of a linter against the
+    // working tree survives, so a lint verdict can only come from the pinned
+    // image.
+    it("runs no linter on the host", () => {
+        expect(has(lintScript, /eslint|prettier/)).toBe(false);
+    });
+
+    // Without a fresh epoch the build is served from cache in under a second,
+    // having linted nothing, and still exits 0.
+    it("passes a fresh LINT_EPOCH on every run", () => {
+        expect(
+            has(lintScript, /--build-arg LINT_EPOCH="\$\(date \+%s\)"/),
+        ).toBe(true);
+    });
+});
+
+describe("Dockerfile.lint", () => {
+    // Tag references are server-mutable, so they are remote code execution.
+    it("pins its base image by digest", () => {
+        expect(has(dockerfileLint, /^FROM \S+@sha256:[0-9a-f]{64}/)).toBe(true);
+    });
+
+    it("runs eslint as a build step", () => {
+        expect(has(dockerfileLint, /^RUN .*eslint \./)).toBe(true);
+    });
+
+    it("runs prettier as a build step", () => {
+        expect(has(dockerfileLint, /^RUN .*prettier --check \./)).toBe(true);
+    });
+
+    // An unset ARG is the empty string, and an empty string is a perfectly
+    // stable cache key. Rejecting it is what stops a bare
+    // `docker build -f Dockerfile.lint .` from reporting a green it did not
+    // earn.
+    it("refuses to build without LINT_EPOCH", () => {
+        expect(has(dockerfileLint, /^ARG LINT_EPOCH$/)).toBe(true);
+        expect(
+            has(dockerfileLint, /^RUN \[ -n "\$LINT_EPOCH" \] \|\| exit 1$/),
+        ).toBe(true);
+    });
+
+    // The guard only forces execution of the layers below it, so both linters
+    // have to sit after it. Layer order is the mechanism, not a style choice.
+    it("puts both linters below the epoch guard", () => {
+        const guard = dockerfileLint.findIndex((line) =>
+            /^RUN \[ -n "\$LINT_EPOCH" \]/.test(line),
+        );
+        const linters = dockerfileLint
+            .map((line, index) => ({ line, index }))
+            .filter(({ line }) => /^RUN .*(eslint|prettier)/.test(line));
+
+        expect(linters.length).toBeGreaterThan(0);
+        for (const { line, index } of linters) {
+            expect(
+                index,
+                `${line} must run below the LINT_EPOCH guard`,
+            ).toBeGreaterThan(guard);
+        }
+    });
+
+    // Dependency installation is the slow layer and has nothing to do with the
+    // sources, so it caches separately: manifests first, sources afterwards.
+    it("copies the manifests before the sources", () => {
+        const manifests = dockerfileLint.findIndex((line) =>
+            /^COPY package\.json yarn\.lock/.test(line),
+        );
+        const sources = dockerfileLint.findIndex((line) =>
+            /^COPY \. \.$/.test(line),
+        );
+
+        expect(manifests).toBeGreaterThanOrEqual(0);
+        expect(sources).toBeGreaterThan(manifests);
+    });
+
+    // script/lint is a docker build; a lint step that shelled out to it would
+    // recurse.
+    it("does not call script/lint or make lint", () => {
+        expect(has(dockerfileLint, /make lint|script\/lint/)).toBe(false);
+    });
+});
+
+describe("Dockerfile", () => {
+    // `make check` runs script/lint, which is a docker build, so an image that
+    // ran it would need a Docker daemon inside the container.
+    it("does not run make check, make lint or script/lint", () => {
+        expect(
+            has(dockerfile, /make check|make lint|script\/(check|lint)/),
+        ).toBe(false);
+    });
+
+    // The replaced lint stage took a `COPY --from=lint` dependency to order
+    // itself before the check stage. Dockerfile.lint is that stage now, and
+    // two definitions of how to lint is one too many.
+    it("has no lint stage", () => {
+        expect(has(dockerfile, /AS lint\b|--from=lint\b/)).toBe(false);
+    });
+
+    it("still runs the suite and the build under the epoch guard", () => {
+        expect(has(dockerfile, /^RUN make test$/)).toBe(true);
+        expect(has(dockerfile, /^RUN make build$/)).toBe(true);
+        expect(
+            has(dockerfile, /^RUN \[ -n "\$CHECK_EPOCH" \] \|\| exit 1$/),
+        ).toBe(true);
+    });
+});
+
+describe("script/cibuild", () => {
+    // CI has to get both verdicts. Lint goes first so the fast failure is
+    // reported before the suite runs.
+    it("builds the lint image before the test and build image", () => {
+        const lint = cibuild.findIndex((line) => /\/lint"/.test(line));
+        const check = cibuild.findIndex((line) =>
+            /docker build .*CHECK_EPOCH/.test(line),
+        );
+
+        expect(lint).toBeGreaterThanOrEqual(0);
+        expect(check).toBeGreaterThan(lint);
+    });
+});
+
+describe("package.json", () => {
+    // `yarn lint` was a second, unpinned way to get a lint verdict, from
+    // whatever eslint the working tree happened to have installed.
+    it("exposes no host lint script", () => {
+        const pkg = JSON.parse(read("package.json")) as {
+            scripts: Record<string, string>;
+        };
+        expect(pkg.scripts.lint).toBeUndefined();
+    });
+});

test/packaging/lint-once.test.ts

@@ -0,0 +1,503 @@
+// `make check` used to run `prettier --check .` twice: once inside the lint
+// container (`script/lint` builds `Dockerfile.lint`, which runs eslint and
+// prettier as build steps) and once again on the host, because `script/check`
+// also called `script/fmt-check`. Two passes, one verdict, and the host one is
+// the weaker of the two — its prettier is whatever the working tree happens to
+// have installed, while the container's is digest-pinned and installed under
+// `--frozen-lockfile`.
+//
+// The fix was to delete the host call from `script/check` and `script/precommit`.
+// Nothing about that fix is self-enforcing: anyone can wire `script/fmt-check`
+// back in, or add a prettier step to a Dockerfile, and every build stays green
+// while quietly doing the work twice again. So the count is asserted here
+// rather than promised in a comment.
+//
+// The assertion is a static walk of the invocation graph, not a string match
+// against one file. Starting from an entrypoint, it follows every edge the repo
+// actually uses to reach another command — `run:` steps in the CI workflow,
+// `"$SCRIPT_DIR/<name>"` and `script/<name>` into other scripts, `make <target>`
+// through the Makefile shims, `yarn run <name>` through the `package.json`
+// scripts, and `docker build -f <file>` into that Dockerfile's `RUN` steps — and
+// counts the prettier invocations it finds. A prettier call added anywhere in
+// that graph is therefore caught, wherever it is added.
+//
+// Two entrypoints are walked, because they cover different graphs: `make check`
+// is what a developer runs, and `.gitea/workflows/check.yml` is what CI runs.
+// The CI walk starts at the workflow file rather than at a hand-picked script,
+// so "the path CI executes" is read out of the repo instead of assumed; it
+// reaches `script/cibuild`, and through it the `Dockerfile` image that `make
+// check` never touches. Walking only `make check` is how a duplicate prettier
+// pass in `Dockerfile` stayed invisible.
+//
+// Undercounting is the failure mode that would make this test worthless. Three
+// things guard against it: the walk is asserted to have reached the nodes that
+// matter, an unresolvable or empty node is a thrown error rather than a quiet
+// zero, and prettier is counted per occurrence rather than per line, so two
+// invocations chained with `&&` cannot read as one.
+import { describe, expect, it } from "vitest";
+import { readFileSync } from "node:fs";
+import { fileURLToPath } from "node:url";
+import { join } from "node:path";
+
+const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
+
+const read = (name: string): string =>
+    readFileSync(join(repoRoot, name), "utf-8");
+
+// A backslash at end of line continues the command; the resolver has to see the
+// whole invocation, since the interesting flags (`-f Dockerfile.lint`) can sit
+// on the continuation.
+const joinContinuations = (text: string): string[] => {
+    const joined: string[] = [];
+    for (const raw of text.split("\n")) {
+        const line = raw.trim();
+        const previous = joined[joined.length - 1];
+        if (previous !== undefined && previous.endsWith("\\")) {
+            joined[joined.length - 1] =
+                `${previous.slice(0, -1).trim()} ${line}`;
+        } else {
+            joined.push(line);
+        }
+    }
+    return joined;
+};
+
+// Comments are stripped everywhere. The headers of these scripts explain the
+// duplication this test exists to prevent, and therefore name `prettier` and
+// `script/fmt-check` repeatedly; counting them would make the test assert the
+// prose instead of the behaviour.
+const executable = (text: string): string[] =>
+    joinContinuations(text).filter(
+        (line) => line !== "" && !line.startsWith("#"),
+    );
+
+// Every occurrence, not "does this line mention prettier": a line that reads
+// `yarn run prettier --check . && yarn run prettier --check src` is two passes
+// over the same tree, which is exactly the bug this file exists to catch, and
+// counting it as one would hide it. `.prettierrc` and `.prettierignore` are not
+// invocations and do not match, because `\b` requires a non-word character
+// after the name.
+const countPrettier = (line: string): number =>
+    (line.match(/\bprettier\b/g) ?? []).length;
+
+// Makefile targets are thin shims (`check:` / tab / `@script/check`), so a
+// `make <target>` edge has to resolve through them to keep "per `make check`"
+// meaning what it says. Recipe lines are the tab-indented ones.
+const makeRecipes = (): Map<string, string[]> => {
+    const recipes = new Map<string, string[]>();
+    let current: string | null = null;
+    for (const raw of read("Makefile").split("\n")) {
+        if (raw.startsWith("\t")) {
+            if (current !== null) {
+                recipes.get(current)?.push(raw.trim().replace(/^[@-]+/, ""));
+            }
+            continue;
+        }
+        const target = /^([a-z][a-z-]*)\s*:(?!=)/.exec(raw);
+        current = target === null ? null : target[1];
+        if (current !== null && !recipes.has(current)) {
+            recipes.set(current, []);
+        }
+    }
+    return recipes;
+};
+
+const recipes = makeRecipes();
+
+const packageScripts = (): Record<string, string> => {
+    const pkg = JSON.parse(read("package.json")) as {
+        scripts?: Record<string, string>;
+    };
+    return pkg.scripts ?? {};
+};
+
+const scripts = packageScripts();
+
+// Node keys: `script/<name>`, `docker:<Dockerfile>`, `make:<target>`,
+// `yarn:<package.json script>`, `workflow:<CI workflow file>`.
+const resolve = (node: string): string[] => {
+    if (node.startsWith("script/")) return executable(read(node));
+    if (node.startsWith("docker:")) {
+        return executable(read(node.slice("docker:".length)))
+            .filter((line) => line.startsWith("RUN "))
+            .map((line) => line.slice("RUN ".length));
+    }
+    // The `run:` steps of a workflow, in file order. `uses:` steps are actions,
+    // not commands, and have no edges into this repo's graph. A `run: |` block
+    // would resolve to the bare `|`, which reaches nothing and therefore fails
+    // the count rather than passing quietly.
+    if (node.startsWith("workflow:")) {
+        return executable(read(node.slice("workflow:".length)))
+            .filter((line) => /^-?\s*run:\s*\S/.test(line))
+            .map((line) => line.replace(/^-?\s*run:\s*/, ""));
+    }
+    if (node.startsWith("make:")) {
+        const target = node.slice("make:".length);
+        const recipe = recipes.get(target);
+        // A renamed or deleted target must be a loud failure: silently walking
+        // an empty recipe would report zero prettier invocations, which reads
+        // like the tidiest possible result.
+        if (recipe === undefined) {
+            throw new Error(`no such Makefile target: ${target}`);
+        }
+        return recipe;
+    }
+    if (node.startsWith("yarn:")) {
+        const name = node.slice("yarn:".length);
+        const script = scripts[name];
+        if (script === undefined) {
+            throw new Error(`no such package.json script: ${name}`);
+        }
+        return [script];
+    }
+    throw new Error(`unresolvable node: ${node}`);
+};
+
+// Same reasoning as the missing-target error, applied to every node kind: a
+// node that resolves to no commands contributes zero prettier invocations and
+// zero edges, which is indistinguishable from a clean result. Fail instead.
+const commandsOf = (node: string): string[] => {
+    const commands = resolve(node);
+    if (commands.length === 0) {
+        throw new Error(`node resolved to no commands: ${node}`);
+    }
+    return commands;
+};
+
+const edgesOf = (line: string): string[] => {
+    const edges: string[] = [];
+
+    // `"$SCRIPT_DIR/lint"`, `"$ROOT/script/lint"` and a bare `script/lint` are
+    // all the same edge.
+    for (const match of line.matchAll(
+        /(?:\$SCRIPT_DIR|\$\{SCRIPT_DIR\}|script)\/([a-z][a-z-]*)/g,
+    )) {
+        edges.push(`script/${match[1]}`);
+    }
+
+    // Only real targets: `pkg_install gnumake make make make` in
+    // script/bootstrap is a package name, not an invocation of this Makefile.
+    for (const match of line.matchAll(/\bmake\s+([a-z][a-z-]*)/g)) {
+        if (recipes.has(match[1] ?? "")) edges.push(`make:${match[1]}`);
+    }
+
+    // Same rule for yarn: `yarn run prettier` is the linter itself (counted,
+    // not followed), `yarn run fmt-check` would be a package.json script that
+    // runs it indirectly.
+    for (const match of line.matchAll(/\byarn(?:\s+run)?\s+([a-z][a-z-]*)/g)) {
+        if ((match[1] ?? "") in scripts) edges.push(`yarn:${match[1]}`);
+    }
+
+    // The container lint pass lives behind a `docker build`; without following
+    // it the count would miss the one invocation that is supposed to survive.
+    if (/\bdocker\s+build\b/.test(line)) {
+        const file = /\s-f\s+(\S+)/.exec(line);
+        edges.push(`docker:${file === null ? "Dockerfile" : file[1]}`);
+    }
+
+    return edges;
+};
+
+interface Walk {
+    prettier: number;
+    reached: Set<string>;
+}
+
+// Repeated invocations must count repeatedly — running the same script twice is
+// exactly the bug — so nodes are not deduplicated. The path stack is only there
+// to turn a cycle into a loud failure instead of a hang.
+//
+// Counting and edge-following both happen for every line: a line that invokes
+// prettier can also invoke something else, and skipping the edges of counted
+// lines silently truncated the graph.
+const walk = (node: string, path: string[] = [], into?: Walk): Walk => {
+    const result = into ?? { prettier: 0, reached: new Set<string>() };
+    if (path.includes(node)) {
+        throw new Error(`invocation cycle: ${[...path, node].join(" -> ")}`);
+    }
+    result.reached.add(node);
+
+    for (const line of commandsOf(node)) {
+        result.prettier += countPrettier(line);
+        for (const edge of edgesOf(line)) {
+            walk(edge, [...path, node], result);
+        }
+    }
+    return result;
+};
+
+describe("prettier runs exactly once per make check", () => {
+    const check = walk("make:check");
+
+    // The headline assertion, and the one the issue is about.
+    it("invokes prettier once for the whole of make check", () => {
+        expect(check.prettier).toBe(1);
+    });
+
+    // Guards against the count being 1 (or 0) because the walk never got
+    // anywhere. `make check` has to reach the suite, the lint script, and the
+    // Dockerfile whose build IS the lint verdict.
+    it.each(["script/check", "script/test", "script/lint", "Dockerfile.lint"])(
+        "reaches %s while counting",
+        (node) => {
+            const key = node.startsWith("script/") ? node : `docker:${node}`;
+            expect([...check.reached]).toContain(key);
+        },
+    );
+
+    // The one that survives is the container's, not the host's: that is the
+    // authoritative verdict, since a successful Dockerfile.lint build is what
+    // CI treats as proof of a clean tree.
+    it("keeps the surviving invocation inside the lint container", () => {
+        expect(walk("docker:Dockerfile.lint").prettier).toBe(1);
+    });
+
+    it("does not reach the host formatting check from make check", () => {
+        expect([...check.reached]).not.toContain("script/fmt-check");
+    });
+});
+
+describe("prettier runs exactly once per CI build", () => {
+    // Rooted at the workflow file, so this is the graph CI executes rather than
+    // the graph someone believed CI executes. `make check` cannot stand in for
+    // it: CI runs script/cibuild, which builds Dockerfile as well as
+    // Dockerfile.lint, and nothing under `make check` ever reads Dockerfile.
+    const ci = walk("workflow:.gitea/workflows/check.yml");
+
+    it("invokes prettier once for the whole CI build", () => {
+        expect(ci.prettier).toBe(1);
+    });
+
+    // script/cibuild is here because the workflow is asserted to run it;
+    // Dockerfile is here because it is the half of the CI graph that the
+    // `make check` walk cannot see.
+    it.each([
+        "script/cibuild",
+        "script/lint",
+        "docker:Dockerfile.lint",
+        "docker:Dockerfile",
+    ])("reaches %s while counting", (node) => {
+        expect([...ci.reached]).toContain(node);
+    });
+
+    // The test and build image must not lint: linting is Dockerfile.lint's job,
+    // and a prettier step added here would be a second pass over the same tree
+    // for the same verdict — on the one path where it matters most.
+    it("keeps prettier out of the test and build image", () => {
+        expect(walk("docker:Dockerfile").prettier).toBe(0);
+    });
+});
+
+describe("the standalone entrypoints still do what their names say", () => {
+    // REPO_POLICIES.md requires both `make lint` and `make fmt-check` to exist
+    // and mean something. Dropping fmt-check from script/check must not turn it
+    // into a target nobody can use, and must not leave `make check` passing
+    // because both halves became no-ops.
+    it("still checks formatting under make fmt-check", () => {
+        expect(walk("make:fmt-check").prettier).toBe(1);
+    });
+
+    it("still checks formatting under make lint", () => {
+        expect(walk("make:lint").prettier).toBe(1);
+    });
+});
+
+describe("script/precommit", () => {
+    // Same duplication as script/check, same fix. The hook still catches a
+    // badly formatted tree before the commit lands, because script/lint is the
+    // container prettier run — that is the whole reason the host call could go.
+    it("checks formatting exactly once", () => {
+        expect(walk("script/precommit").prettier).toBe(1);
+    });
+
+    it("gets that check from the lint container", () => {
+        expect([...walk("script/precommit").reached]).toContain(
+            "docker:Dockerfile.lint",
+        );
+    });
+});
+
+// script/bootstrap installs the dependencies, and it has two install sites: one
+// for the case where yarn has to be reached through nvm, and one for the case
+// where yarn is already on PATH. A substring check against the whole file
+// cannot tell them apart, so it reports the first and says nothing about the
+// second — which is the one the containers take, because the pinned node image
+// ships yarn. Both are resolved separately here.
+const installBranches = (): { withoutYarn: string[]; withYarn: string[] } => {
+    const lines = executable(read("script/bootstrap"));
+    const open = lines.findIndex((line) =>
+        /^install_js_deps\s*\(\)/.test(line),
+    );
+    if (open === -1) {
+        throw new Error("script/bootstrap: no install_js_deps function");
+    }
+    const close = lines.indexOf("}", open);
+    const body = lines.slice(open + 1, close === -1 ? undefined : close);
+    const guard = body.findIndex((line) =>
+        /^if\b.*\bmissing yarn\b/.test(line),
+    );
+    const otherwise = body.indexOf("else", guard);
+    const end = body.indexOf("fi", otherwise);
+    if (guard === -1 || otherwise === -1 || end === -1) {
+        throw new Error(
+            "script/bootstrap: install_js_deps is not the expected " +
+                "if missing yarn / else / fi shape",
+        );
+    }
+    return {
+        withoutYarn: body.slice(guard + 1, otherwise),
+        withYarn: body.slice(otherwise + 1, end),
+    };
+};
+
+// Every `yarn install` in the given lines, with its flags, so an unpinned
+// install cannot hide next to a pinned one.
+const yarnInstalls = (lines: string[]): string[] =>
+    lines.flatMap((line) =>
+        [...line.matchAll(/\byarn install\b[^"'&|;]*/g)].map((match) =>
+            match[0].trim(),
+        ),
+    );
+
+describe("host and container prettier cannot disagree", () => {
+    // With the host pass gone from `make check`, `make fmt-check` is the only
+    // host-side formatting check left, and the container is the gate. The two
+    // must keep producing the same verdict on the same tree, or a developer
+    // running `make fmt-check` gets a green that CI then rejects.
+    //
+    // Three things make them agree, and all three are load-bearing:
+    it("pins the same prettier for both", () => {
+        const pkg = JSON.parse(read("package.json")) as {
+            devDependencies: Record<string, string>;
+        };
+        // An exact version, not a range: `^3.8.1` would let the container and
+        // the host resolve different builds with different formatting.
+        expect(pkg.devDependencies.prettier).toMatch(/^\d+\.\d+\.\d+$/);
+    });
+
+    it("installs from the lockfile on the branch the container takes", () => {
+        // Both images are FROM a node image, which ships yarn, so `missing
+        // yarn` is false and this is the branch that runs in the container.
+        const installs = yarnInstalls(installBranches().withYarn);
+        expect(installs).not.toHaveLength(0);
+        for (const install of installs) {
+            expect(install).toContain("--frozen-lockfile");
+        }
+    });
+
+    it("installs from the lockfile on the nvm branch too", () => {
+        // Not the container's branch, but it is the one a developer without
+        // yarn on PATH gets, and their prettier has to match the container's.
+        const installs = yarnInstalls(installBranches().withoutYarn);
+        expect(installs).not.toHaveLength(0);
+        for (const install of installs) {
+            expect(install).toContain("--frozen-lockfile");
+        }
+    });
+
+    it("runs script/bootstrap inside the lint container", () => {
+        // Without this the lockfile assertions above would be about a script
+        // the container never executes.
+        expect([...walk("docker:Dockerfile.lint").reached]).toContain(
+            "script/bootstrap",
+        );
+    });
+
+    it("keeps .gitignore in the build context", () => {
+        // Prettier 3 reads .gitignore as a default ignore file, so excluding it
+        // from the context would change which files the container checks.
+        const dockerignore = read(".dockerignore")
+            .split("\n")
+            .map((line) => line.trim());
+        expect(dockerignore).not.toContain(".gitignore");
+    });
+});
+
+describe("the walk cannot pass vacuously", () => {
+    // An earlier draft of this file computed a Makefile target as
+    // `node.slice("make:")` — a string where a number belongs, which coerces to
+    // NaN and made every target resolve to nothing. The count went to zero and
+    // an assertion of "not twice" would have been satisfied by a walk that had
+    // read nothing at all. Every way of reaching nothing is therefore an
+    // error here, and the ways are tested rather than assumed.
+    it("reports zero for a subgraph that does not run prettier", () => {
+        expect(walk("make:clean").prettier).toBe(0);
+    });
+
+    it("refuses a Makefile target that does not exist", () => {
+        expect(() => walk("make:no-such-target")).toThrow(
+            /no such Makefile target/,
+        );
+    });
+
+    it("refuses a package.json script that does not exist", () => {
+        expect(() => walk("yarn:no-such-script")).toThrow(
+            /no such package.json script/,
+        );
+    });
+
+    it("refuses a script that does not exist", () => {
+        expect(() => walk("script/no-such-script")).toThrow(/ENOENT/);
+    });
+
+    it("refuses a node that resolves to no commands", () => {
+        // .dockerignore has no RUN steps, standing in for a Dockerfile whose
+        // steps a restructure moved somewhere the resolver cannot see.
+        expect(() => walk("docker:.dockerignore")).toThrow(
+            /resolved to no commands/,
+        );
+    });
+
+    it("refuses a node kind it does not understand", () => {
+        expect(() => walk("nonsense")).toThrow(/unresolvable node/);
+    });
+
+    it("refuses to walk in circles", () => {
+        expect(() => walk("make:check", ["script/check"])).toThrow(
+            /invocation cycle/,
+        );
+    });
+});
+
+describe("the resolver reads what the shell would run", () => {
+    // Counting per line is how `yarn run prettier --check . && yarn run
+    // prettier --check src` read as a single invocation.
+    it("counts every prettier invocation on a line", () => {
+        expect(
+            countPrettier(
+                "yarn run prettier --check . && yarn run prettier --check src",
+            ),
+        ).toBe(2);
+    });
+
+    it("does not count the config files as invocations", () => {
+        expect(countPrettier("COPY .prettierrc .prettierignore ./")).toBe(0);
+    });
+
+    // The counting `continue` also dropped every edge that shared a line with a
+    // prettier call, so a whole subtree could be hidden behind one `&&`.
+    it("still follows the edges of a line that invokes prettier", () => {
+        expect(
+            edgesOf('yarn run prettier --check . && "$SCRIPT_DIR/lint"'),
+        ).toContain("script/lint");
+    });
+
+    it("resolves every spelling of a script call to one node", () => {
+        expect(
+            edgesOf('"$SCRIPT_DIR/lint" "${SCRIPT_DIR}/test" script/fmt'),
+        ).toEqual(["script/lint", "script/test", "script/fmt"]);
+    });
+
+    it("follows a bare docker build to Dockerfile and -f to its file", () => {
+        expect(edgesOf("docker build .")).toContain("docker:Dockerfile");
+        expect(edgesOf("docker build -f Dockerfile.lint .")).toContain(
+            "docker:Dockerfile.lint",
+        );
+    });
+
+    it("reads the run steps of the CI workflow and not its uses steps", () => {
+        expect(commandsOf("workflow:.gitea/workflows/check.yml")).toEqual([
+            "script/cibuild",
+        ]);
+    });
+});