quak/TODO.md

# Workflow

- branch (from `main`)
- do the work in Next Step
- move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work)
- merge to `main` if the branch is not protected, otherwise open a PR
- push

# Status

pre-1.0

# Next Step

Tag v1.0.0.

# Completed Steps

- 2026-09-23: Pinned three guards reviewers found untested (issue 89). The
  download idle deadline's timer is unref'd, so it can never keep the process
  alive, and a test checks no timer is left after a download completes or fails.
  A test covers the rejection of `#` in a request path. The EXIF scan compares
  the `Exif` header only in an APP1 segment of length 8 or more, so it never
  reads the next segment's bytes, with a test for a short one.
- 2026-09-23: Made the download deadline an idle deadline (issue 24).
  `downloadTimeoutMs` now aborts a file or thumbnail download only after no
  bytes have arrived for that long, default 60 seconds, instead of bounding the
  whole transfer at 10 minutes, so a slow download that keeps making progress
  completes. A download that fails before reading the whole body cancels it, so
  a failed file no longer holds its connection.
- 2026-09-23: Every `ApiClient` request URL is now built by one function next to
  the class (issue 18), so a self-hosted `apiOrigin` with a base path keeps it
  on every request, a path works with or without a leading slash, and query
  parameters are percent-encoded. A path containing `?` or `#` is rejected with
  an error instead of being silently cut.
- 2026-09-23: Made the CLI testable and tested it (issue 12). The command bodies
  moved from `bin/quak.ts` into `src/cli-commands.ts` as functions that take
  their options and a context (output streams, session directory, cache
  directory, session loader) and return an exit code; `bin/quak.ts` only wires
  them to commander and exits with the code once stdout and stderr have drained,
  so nothing below it calls `process.exit`. `test/cli/commands.test.ts` drives
  them with a fake client: session file modes, logout, the missing and corrupt
  session paths, and the output and exit code of `whoami`, `collections`,
  `files`, `get`, `get-thumb`, `backup` and `helper list-missing-thumbnails`.
- 2026-09-23: Hardened the backup tree's atomic copy (issue 22). `copyAtomic`
  fsyncs its temp file before the rename and the directory after it, through the
  download writer's `fsyncPath`; each backup run deletes `.quak-backup-*.tmp`
  files whose process is no longer running. The README backup layout names the
  temp files and states that the rename replaces a symlink and takes the temp
  file's permissions. Added tests for a missing and an unwritable destination
  directory for `downloadFile` and `downloadThumbnail`.
- 2026-09-23: Hardened the JPEG EXIF scan behind `backup-metadata --exif` (issue
  11). Every segment length is checked against the remaining bytes and lengths
  under 2 stop the scan, so a truncated or corrupt original can neither throw
  nor loop. A malformed or unparseable EXIF segment is recorded as
  `imageMetadata.exifError`, and a failure to read the original as
  `imageMetadataError` in the per-file JSON, instead of the field being left
  out.
- 2026-09-22: Hardened the retry classifier (issue 80). A `POST` or `PUT` is
  replayed only when every errno in the cause chain is a connect errno, and it
  no longer follows redirects. `getRetryOptions()` returns a copy. Tests pin
  every errno the classifier names, the cause-chain depth limit, cycle
  termination, and a fresh deadline per attempt for every retrying entry point.
  The README's endpoint list is the one place that names the requests the replay
  rule covers.
- 2026-09-22: Stopped `make test` collecting tests from checkouts nested under
  `.claude/` (issue 25). vitest ignores `.gitignore` when finding tests, so a
  nested checkout ran the whole suite again; `vitest.config.ts` now adds
  `.claude/**` to vitest's default excludes, and
  `test/packaging/nested-checkout.test.ts` plants a nested checkout in a temp
  directory and fails if vitest would collect it.
- 2026-09-22: Dropped the deprecated `@types/libsodium-wrappers-sumo` stub from
  `devDependencies` (issue 27). It shipped no declarations; the types come from
  `libsodium-wrappers-sumo` itself. `yarn.lock` regenerated by `yarn remove`.
- 2026-09-22: Hardened the client session lifecycle (issue 10).
  `Client.fromJSON` checks every snapshot field and each key's decoded length
  and names the bad field; `toJSON` reads the token through
  `ApiClient.getAuthToken` and throws when there is none; `logout` zeroes the
  key buffers, and `collectionsSince` re-checks for logout after its request so
  it never decrypts with zeroed keys. The CLI reports a corrupt session file
  separately from a missing one (`src/cli-session.ts`).
- 2026-09-22: Sanitized file names taken from server metadata (issue 9). A new
  `src/filename.ts` holds the one sanitizer, used by `quak get`/`get-thumb`
  without `--out`, `downloadFile`/`downloadThumbnail` without `outPath`, and the
  backup and metadata backup trees; it removes separators, control characters,
  leading dots and Windows device names, and falls back to a name built from the
  ID for an empty title. Originals-cache extensions are letters and digits only,
  else `.bin`. A user-supplied path is used as is. `decryptFile` reads a missing
  or non-string title as "" and rejects metadata that is not a JSON object.
- 2026-09-22: Rewrote the README API reference (and the Getting Started / usage
  snippets) to match the shipped cache/API library on `next` (issue 53, issue
  13). Documented `Library.open` and its options, the default-read vs `fresh()`
  distinction (and that the CLI's read commands are fresh), the record types and
  `snapshot()`/`subscribe()`, the `albums`/`photos`/`timeline` read surface,
  `Photo` content methods, `thumbnails.ensure`, the `mldata` search surface,
  `backup()`, the three request pools (10/5/25), and the on-disk cache layout;
  noted the deferred content-hash integrity check (issue 68). Docs-only; no code
  changed.
- 2026-09-22: Added resumable, deletion-aware enumeration to `Client` (issue 38,
  closes issue 7). `collectionsSince`/`filesSince` take a starting cursor,
  decrypt live records, surface tombstoned ids in a separate `deleted` list (a
  tombstone has nothing to decrypt, so it is a bare id, not a hollow record),
  and return the max `updationTime` seen as the cursor to resume from.
  `filesSince` refuses to loop when the diff reports `hasMore` without advancing
  the cursor (issue 7). `listCollections`/`listFiles` are now thin wrappers that
  enumerate from `sinceTime: 0` and drop deletions, so existing callers are
  unaffected.
- 2026-09-22: Carried file size, thumbnail size, and the deletion flag through
  `decryptFile` (issue 37, foundation for the cache/API design). Live files now
  populate `file.size`/`thumbnail.size` from the server's `info` (left
  `undefined` when the server omits it), and `isDeleted` is carried from the
  diff row onto `EnteFile`. No caller change: `listFiles` still filters deleted
  rows before decrypting. Surfacing a tombstone through decryption belongs to
  the enumeration unit (issue 38).
- 2026-08-10: Made `lint-once.test.ts` enforce what its header claims. It walked
  `make check` only, so it never read `Dockerfile` — the image CI builds through
  `script/cibuild` — and a second `prettier --check .` could be added there with
  the suite staying green. The walk now also starts at
  `.gitea/workflows/check.yml` and follows its `run:` steps, so the graph under
  test is the one CI executes rather than the one someone assumed it executes.
  The lockfile assertion was a substring check against the whole of
  `script/bootstrap`, which has two install sites and so reported the branch the
  containers never take; the two branches are now resolved separately and every
  `yarn install` in each is required to be `--frozen-lockfile`. Prettier is
  counted per occurrence instead of per line, so two invocations chained with
  `&&` no longer read as one, and edges are followed on counted lines instead of
  being skipped. Every way for the walk to reach nothing — an unknown target, an
  unknown script, a missing file, a node with no commands, an unknown node kind
  — is a thrown error rather than a quiet zero. Every assertion in the file was
  mutation-tested individually.
- 2026-08-10: Stopped `make check` running `prettier --check .` twice. Since
  linting moved into Docker, the duplicate was one container pass and one host
  pass of the same check: `script/lint` builds `Dockerfile.lint`, which runs
  prettier as a build step, and `script/check` then called `script/fmt-check` as
  well. The host call is gone from `script/check` and from `script/precommit`;
  the container keeps checking formatting, because a successful
  `Dockerfile.lint` build is what CI treats as proof of a clean tree, and it is
  also what still fails the pre-commit hook on a badly formatted tree.
  `script/fmt-check` survives as a standalone entrypoint, whose verdict cannot
  drift from the container's. A test walks the invocation graph from each
  entrypoint — through the Makefile shims, the `script/` calls and the
  `docker build` — and asserts the prettier count, so the duplication cannot
  come back unnoticed.
- 2026-08-10: Moved all linting into Docker. `script/lint` builds a new root
  `Dockerfile.lint`, which copies the repo into the digest-pinned node image and
  runs eslint and prettier as build steps, so a successful build is a clean
  lint; no host lint path remains and `yarn lint` is gone from `package.json`. A
  fail-closed `LINT_EPOCH` guard stops Docker serving the linter layers from
  cache, which is how a lint build returns success in under a second having
  linted nothing. The lint stage inside `Dockerfile` and its `COPY --from=lint`
  ordering hack are gone: that image now runs `make test` and `make build` only,
  because `script/check` calls `script/lint` and running it in a container would
  mean docker inside docker. `script/cibuild` builds the lint image first, then
  the test and build image.
- 2026-08-09: Made `make docker` green and policy-conformant. Multi-stage
  Dockerfile: a lint stage runs `make fmt-check` and `make lint`, and the check
  stage takes a `COPY --from=lint` dependency on it before running `make check`
  and `make build`. `CHECK_EPOCH` and a fail-closed guard stop Docker serving
  those two layers from cache, which is what let a build report success without
  running the suite. `script/projectname` says `quak`, so the image is tagged
  `quak`; `script/bootstrap` updates apt lists before installing, so a Debian
  base works; `.dockerignore` no longer ships the compiled binary, the caches or
  agent worktrees into the build context, and keeps `.gitignore` in it for
  prettier.
- 2026-08-09: Fixed the TypeScript build. `rootDir` is the repo root, so `bin/`
  compiles alongside `src/` instead of failing with TS6059; output is
  `dist/src/` and `dist/bin/`, which is where `main`, `types` and `bin.quak` now
  point. `script/build` verifies the declared entrypoints exist after the
  compiler runs and makes the CLI executable, the Dockerfile runs `make build`
  as well as `make check`, and a `quak` script makes the README's
  `yarn quak <command>` examples work.
- 2026-08-09: Retry policy: no retry on 4xx (except `408` and `429`),
  exponential backoff with full jitter on 5xx, transport failures and truncated
  transfers, under per-attempt deadlines that cover the response body as well as
  the request. Downloads retry request, stream consumption and decryption as one
  unit; `postJSON` and `putJSON` are replayed only when the connection was never
  established.
- 2026-08-09: Downloads verify the secretstream terminated on `TAG_FINAL` and
  write output atomically: a truncated body is rejected instead of landing on
  disk as a short file, and plaintext is staged in a sibling temp file and
  renamed into place, so a failed download leaves the destination untouched.
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
  shims, README Entrypoints section
- 2026-06-10: Decrypted collections shared by other users (sealed-box keys);
  listCollections drops deleted-collection tombstones.
- 2026-06-10: Login hardening: dual-2FA empty-string fields handled, TOTP
  preferred when a passkey is also enrolled, interactive input via
  @inquirer/prompts.
- 2026-06-10: Replaced sharp with pure JS (jpeg-js + exif-reader); added
  single-binary bun build and make install.
- 2026-06-09: Added backup-metadata command (ML data always included, --exif
  opt-in); rewrote README to match the implementation; added thumbnail helper
  tests.
- 2026-05-13: Full CLI surface: login, backup with dedup symlink layout,
  collections, files, get, get-thumb, thumbnail repair helpers.
- 2026-05-13: Client OO API with literate usage tests; file download and
  decryption; all three metadata layers decrypted and persisted; renamed quack
  to quak.
- 2026-05-11: SRP login flow (email OTP + TOTP) and ApiClient.

# Future Steps

- Future desktop client, separate repo:
    - Electron app skeleton consuming this library.
    - Local SQLite cache keyed on (collectionID, fileID, updationTime).
    - Background sync worker streaming new files into the cache.
    - Gallery UI: thumbnails, full-image view, basic search.
    - Upload, delete, and share operations in the library.