quak login --server: log in to a self-hosted Ente server (closes #243)

`quak login` takes the server's API address from `--server` or
`QUAK_SERVER`, refusing anything that is not an http or https URL. The
client's snapshot now keeps a non-default `apiOrigin`, so the session
file names the server and every later command uses it. A snapshot's
`apiOrigin` wins over the one in `fromJSON`'s `apiOptions`, since the
token belongs to that server.

Model: opus-5-5

Makefile

@@ -26,10 +26,8 @@ check:
 build:
 	@script/build
 
-# Bundles the built dist/, so the binary reports the version script/build
-# stamped.
-build-bin: build
-	nix-shell -p bun --run "bun build dist/bin/quak.js --compile --outfile bin/quak"
+build-bin:
+	@script/build-bin
 
 install: build-bin
 	mkdir -p ~/bin

TODO.md

@@ -2,7 +2,7 @@
 
 - branch from `next`
 - do the work in Next Step
-- move Next Step to the top of Completed Steps
+- delete Next Step from `TODO.md`
 - move the top item of Future Steps into Next Step
 - commit (`TODO.md` changes in the same commit as the work)
 - push
@@ -24,696 +24,6 @@ only sneak has, so it waits on him.
 Tagging and releases are decided by sneak alone, and happen only when he
 declares one.
 
-# Completed Steps
-
-- 2026-10-18: `-v`/`--verbose` raises the CLI's log level once for each repeat,
-  and `-q`/`--quiet` prints only fatal errors
-  (https://git.eeqj.de/sneak/quak/issues/212). The levels are fatal, error,
-  warning, info, debug and trace; the default is info, everything quak printed
-  on stderr before, `-v` gives debug and `-vv` trace. `src/cli-log.ts` holds the
-  level and the log the commands write stderr lines through. Debug adds each
-  retry, which `withRetry` reports through the new `onRetry`, and the start and
-  end of each refresh and ML data fetch; trace adds each request, which
-  `ApiClient` reports through the new `onRequest`. `--json` no longer keeps the
-  progress lines off stderr, so `backup` and `--json` runs log a refresh's
-  progress too. Every line that reports a failed file or album is at error,
-  which the progress callbacks of `backup`, `backup-metadata` and the thumbnail
-  helpers now say, and a command that exits 1 because files or albums failed
-  ends with one fatal line saying how many and where to look.
-
-- 2026-10-18: `quak files` takes the collection ID as an optional argument
-  instead of the `--collection` option, which is gone
-  (https://git.eeqj.de/sneak/quak/issues/211). With no argument it lists every
-  file in the account once, collection by collection; a file in several
-  collections is listed with the first of them, which is its `collectionID` in
-  `--json`. The fresh read in `src/cli-read.ts` does this.
-
-- 2026-10-18: Names quak builds from an album name or a title fit in a file name
-  (https://git.eeqj.de/sneak/quak/issues/205). `shortenKeepingExtension` in
-  `src/filename.ts` cuts a name short to a number of bytes, keeping its
-  extension when it fits and always its first character, and can put text right
-  before the extension; the backup's `withID` uses it for an ID.
-  `defaultFileName` there gives the name a file is saved under when the caller
-  gives no path: a prefix such as `thumb_` and the title with unsafe characters
-  replaced, cut short with it to 255 bytes, and, for a live photo's image or
-  video, with that part's own extension before the cut. `quak get`,
-  `quak get-thumb`, `downloadFile` and `downloadThumbnail` use it without
-  `--out` or `outPath`. `backup-metadata` cuts an album's name short so that its
-  folder, `<id>-<name>`, fits in 255 bytes. `safeExtension` gives `.bin` for an
-  extension of more than 16 letters and digits, so a save path's name always
-  fits.
-
-- 2026-10-18: `quak collections` no longer waits for the files of every
-  collection (issue 210). On an empty cache it opened its library with a refresh
-  that fetched and decrypted the files of each collection in turn, printing
-  nothing and storing nothing until all were done, so on a large account it
-  seemed to hang. It now reads the collection list alone through the new
-  `lib.freshCollections()`, one request that stores nothing. The read commands
-  open their library with the new `refreshOnOpen: false`, so each makes only the
-  refresh it asks for. A refresh sends a `"progress"` event after each
-  collection whose files it fetched, and the commands that wait on one print it
-  on stderr, except with `--json`.
-
-- 2026-10-16: ML data the server sends that does not decrypt, gunzip or parse is
-  no longer taken for no ML data (issue 198). `fetchMLDataBatch` returns why for
-  each such file beside the payloads; ML data that parses to something other
-  than a JSON object, such as `null`, is one of them. The library keeps the
-  reasons from its last ML data fetch that succeeded in memory, not in the
-  cache, and gives them in `status().mlDataErrors` and in the `fetchMLData`
-  event with status `"done"`; a fetch still running or one that fails does not
-  change them. `quak backup` writes the reason as `mlDataError` in the file's
-  JSON when no ML data is cached for it, and records the file in `failures.json`
-  with an error starting `ML data: `; `backup-metadata` writes `mlDataError`
-  too. Both exit 1. Tests serve ML data encrypted under a key that is not the
-  file's.
-
-- 2026-10-16: With `--verify`, an original whose hash does not match is no
-  longer removed before its replacement is downloaded (issue 199).
-  `replaceOriginal` in `src/library/content.ts` downloads the replacement to a
-  temporary file beside it, a live photo's image and video each to its own,
-  hashes it and renames it over the original only when it matches. When the
-  download fails or does not match either, the original keeps its bytes and
-  links, and the file goes into `failures.json`. The replacement is always
-  downloaded, through the new `fetchOriginal` of the content cache, never copied
-  from the cache.
-
-- 2026-10-16: The `~/bin/quak` that `make install` builds retries a refused
-  connection and a name that does not resolve, and the backup records either as
-  transient (issue 206). Bun's `fetch` reports them with codes of its own,
-  `ConnectionRefused` and `FailedToOpenSocket`, which `src/retry.ts` now counts
-  as transport failures. A `POST` or `PUT` is not replayed on them, because bun
-  also gives them when it resends a request on a new connection after a
-  connection it kept open closed, and the server may already have the request.
-  Bun already gave `ECONNRESET` for a reset and a `TimeoutError` for a deadline.
-  `classify` in `src/backup.ts` now records any `TimeoutError` as transient, so
-  a stalled download, whose error says only that it stalled, is no longer
-  `unknown`. The shapes were found by running the binary against local
-  listeners.
-
-- 2026-10-15: An album whose folder or JSON `quak backup` cannot write no longer
-  ends the run before the other albums and `failures.json` are written (issue
-  197). The run prints `FAILED album <name>: <reason>`, lists the album's ID in
-  `failedAlbums` in the result and in `--json`, goes on, and exits 1. An album
-  whose folder would have the name of another album's JSON, as `x.json` beside
-  `x` would, gets its ID added. A folder or link name over 255 bytes is cut
-  short between two characters, by `shortenToBytes` in `src/filename.ts`, and
-  gets the ID.
-
-- 2026-10-15: A network failure is printed and recorded with its reason, not as
-  just `fetch failed` (issue 196). `describeError` in `src/errors.ts` gives an
-  error's message followed by the message of each error on its `cause` chain,
-  such as `fetch failed: connect ECONNREFUSED 1.2.3.4:443`, and each address's
-  reason when a host has several and none could be reached. `run` prints that,
-  and the backup logs it and records it in `failures.json`, where `classify` now
-  calls the failure transient when the chain holds an errno that
-  `hasTransportCode` in `src/retry.ts` recognises, for a failed ML data fetch
-  too. The walk along the chain moved from `src/retry.ts` to `src/errors.ts`,
-  which both use. Tests refuse every connection of a backup, then only one
-  file's download, and fail the ML data fetch.
-
-- 2026-10-15: The README says that `make install` needs nix (issue 200), in
-  Getting Started and in "Running the backup from cron": `make build-bin` runs
-  bun through `nix-shell -p bun`. The `Makefile` is unchanged.
-
-- 2026-10-14: `lib.backup()`, and so `quak backup`, works on up to the content
-  pool's limit of originals at once, 5 by default, instead of one at a time
-  (issue 182). That many workers take files from one shared list, each doing all
-  the work for one original, its `--verify` hashing included, so no more
-  originals than that are open at once. `BackupLibrary` says the limit as
-  `contentConcurrency`. A test holds each download at the fake Ente server to
-  count those in flight. Once a worker has thrown, as one does when `onProgress`
-  throws, or the lock is lost, no worker takes another file; the backup fails
-  once the originals under way are in place, and a lost lock stops it before it
-  fetches ML data.
-
-- 2026-10-14: Next Step said the cache design, issue 36, waits on sneak's
-  review, though that issue closed on 2026-10-03 (issue 194). It now names the
-  issues open on the tracker.
-
-- 2026-10-14: A backup whose lock is lost no longer dies with a stack trace
-  (issue 184). `lockBackupDirectory` gives proper-lockfile an `onCompromised`
-  that records why the lock was lost instead of throwing from its timer, and
-  returns the lock, which `quak backup` passes to `lib.backup()` as `lock`. The
-  backup then stops before it starts another file with
-  `lost the lock on DIR: REASON`, which `quak backup` prints as one line before
-  it exits 1. Releasing a lost lock leaves `backup.lock` alone. Tests in
-  `test/cli/backup-end-to-end.test.ts` remove the lock, or put another backup's
-  in its place, while a download is held open.
-
-- 2026-10-14: An album or file quak cannot decrypt, or whose metadata is not
-  JSON, no longer fails the refresh, so one bad record no longer stops
-  `quak backup` from storing anything (issue 181). The refresh leaves it out, an
-  album with its files, and the library keeps it with the reason in
-  `metadata.json` until the server sends a copy that decrypts or deletes it.
-  `quak backup` stores every other file, records each such file in
-  `failures.json` with an error starting `could not be decrypted:`, names each
-  such album's ID in a printed line, or with `--json` in `undecryptableAlbums`,
-  and exits 1 while any remain. A file whose copy in another album decrypts is
-  not recorded: its original is stored from that copy, and the album where it
-  does not decrypt neither links nor lists it. The read commands answer from
-  what decrypted.
-
-- 2026-10-14: `quak backup` no longer asks the server for changes every 3
-  seconds while it runs (issue 183). It opens its library with
-  `refreshIntervalSeconds: Infinity`, which the library now takes as no
-  background timer, so the backup refreshes only as it starts and no timed
-  refresh starts another ML data pass during the run. A test holds a backup's
-  download open while two hours pass and counts its `/collections/v2` requests.
-
-- 2026-10-14: `test/cli/backup-end-to-end.test.ts` runs `quak backup` as cron
-  does (issue 186): `backupCommand` and `run`, the real `loadSession` reading a
-  session saved after logging in, and every request answered by a fake of the
-  Ente HTTP API serving data encrypted with libsodium. It covers the backup tree
-  of an account with two albums of its own and one shared with it, a second run
-  that downloads nothing, a download that succeeds after two 503s, a 404 that
-  lands in `failures.json` with exit 1, a refresh answered 401 with exit 3, and
-  `--verify` putting back an original changed on disk. The fake server and the
-  account it logs in moved from the backup-metadata test into
-  `test/ente-server.ts`, unchanged.
-
-- 2026-10-14: `files --collection`, `get`, `get-thumb` and
-  `helper fix-missing-thumbnails --file` take an ID only when it is written in
-  decimal digits and is a safe integer (issue 185). Before, `Number()` read
-  `0x10` as 16, `1e3` as 1000 and an empty string as 0, and `--file` dropped an
-  ID it could not read without a word. Anything else now prints
-  `Invalid file ID: <arg>` or `Invalid collection ID: <arg>` and exits 1 before
-  any request; one bad `--file` ID fails the whole command. `parseID` in
-  `src/cli-commands.ts` does the check for all four.
-
-- 2026-10-06: The README says how to run `quak backup` from cron (issue 170):
-  log in once as the job's user, a crontab with a backup every night and
-  `--verify` on Sundays appending to a log file, and the `HOME` and
-  `XDG_DATA_HOME` the job needs to find the saved session. A table gives each
-  exit code: 0, the backup is complete; 1, files are in `failures.json` or
-  another error stopped the run; 2, another backup of the directory is running;
-  3, there is no usable session. The introduction lists everything the backup
-  keeps for each file.
-
-- 2026-10-06: `quak backup --verify` and `lib.backup({ verify: true })` hash
-  each original already at its save path as the download check does, streamed, a
-  live photo as `<imageHash>:<videoHash>` (issue 168). One that does not match
-  the content hash its metadata records is logged, removed (both files of a live
-  photo) and put back in the same run, downloaded or copied from the cache, and
-  what is put back is hashed too. One that still does not match, or whose
-  download fails, goes into `failures.json`. One with no recorded hash is left
-  alone. The result, `--json` and the summary gain `verified`, `mismatched` and
-  `unchecked`. Without `--verify` nothing is hashed.
-
-- 2026-10-06: Two backups of the same directory never run at once (issue 169).
-  `lib.backup()` takes a lock, `backup.lock` in its download directory, made
-  with `proper-lockfile`, before its refresh, and removes it when it ends,
-  whether it succeeds or fails. A second backup of the directory, from another
-  process or the same one, fails at once with an error naming the directory.
-  `quak backup` takes the lock before it opens its library, so a refused run
-  sends no request; it prints the error as one line and exits 2. A lock that has
-  gone 10 seconds untouched, left by a run that could not remove it, is taken
-  over by the next run.
-
-- 2026-10-06: `quak backup` writes each original's EXIF, XMP and dimensions into
-  the file's JSON as `imageMetadata`, what `backup-metadata --exif` records
-  (issue 167): for a live photo from its image, for a video nothing, and `{}`
-  for an original with none of them. A failed read puts the reason in
-  `imageMetadataError` and fails neither the file nor the run. An original is
-  read when the run stores it or when its JSON has neither field; otherwise the
-  field is taken from that JSON. The hand-built JPEGs moved to
-  `test/exif-jpeg.ts`, beside the HEIC.
-
-- 2026-10-06: When the server answers HTTP 401 and that ends a command that
-  loads the saved session, because the server no longer accepts its token, the
-  command prints one line,
-  `quak: the saved session is no longer valid; run "quak login"`, and exits 3
-  (issue 164). `quak backup` meets that 401 on the refresh it starts with,
-  before it touches any file. For the same commands, a missing or corrupt
-  session file keeps its message and also exits 3, so a cron job can tell that
-  the user must log in again. `quak logout` is unchanged. `run` in
-  `src/cli-run.ts` recognises the 401, which reaches it unchanged.
-
-- 2026-10-06: `quak backup` retries a failed request for longer than the other
-  commands do (issue 165). `src/retry.ts` exports `UNATTENDED_RETRY_OPTIONS`
-  beside the unchanged default: 10 attempts, a 1 s base delay and a 60 s cap, so
-  a request that keeps failing waits at most 243 s before it gives up.
-  `bin/quak.ts` loads the backup's session with them, so its refresh, ML data
-  and downloads all use them. What is retried and the backoff formula are
-  unchanged.
-
-- 2026-10-06: The files this repository copies from `sneak/prompts` are copied
-  again from its commit `dd4027b` (issue 171). `.gitignore` and `.dockerignore`
-  keep out more secret files, and this repository's build artifacts follow the
-  copied content. `script/check` runs `script/fmt-check` again, `script/cibuild`
-  runs `script/bootstrap` and `script/check` before the image build,
-  `script/fmt` and `script/fmt-check` find the pinned yarn under nvm, and the
-  image's last stage marks `/app` safe for git.
-
-- 2026-10-06: `quak backup` writes the account and album records
-  `backup-metadata` writes (issue 166): `account.json` with the account's
-  `email` and `userID`, and in each album's JSON its `ownerID`, `isShared`,
-  `updationTime` and, when present, its three layers of magic metadata. Each
-  file's JSON gains its `updationTime`.
-
-- 2026-10-05: `quak backup` writes each file's ML data (its faces and CLIP
-  embedding) into the file's JSON as `mlData`, the payload
-  `lib.mldata.forFile()` returns (issue 163). `lib.backup()` waits for an ML
-  data fetch before it writes the JSON files. When that fetch fails, each file
-  whose ML data is not cached gets the reason in `mlDataError` and counts as
-  failed, and the next run fetches it again.
-
-- 2026-10-03: The download-albums example test no longer fails on a disk with
-  under 50 GiB free (issue 160). It opens its libraries with
-  `freeBelowBytes: 0`, so the free space of the disk it runs on cannot shrink
-  the cache, evict the original it cached and add a fetch to the count. No other
-  test's result depends on that free space. The 50 GiB default is unchanged.
-
-- 2026-10-02: `docker build .` stamps the commit's tag or short commit, not
-  `dev` (issue 154). `script/build` writes the version `script/version` prints
-  into `dist/package.json`: the `VERSION` environment variable or build arg when
-  one is given, otherwise `git describe --tags --always`, otherwise the version
-  `package.json` declares. `.dockerignore` sends `.git`, and a checkout with
-  `.git` whose version comes out empty, `dev` or `unknown` fails the build.
-  `make build-bin` bundles the built `dist/`, so the single binary reports the
-  same version.
-
-- 2026-10-02: `photo.exif()` returns every EXIF tag in the file as `ExifTags`,
-  keyed by tag name, each as exifreader decodes it, not only the thirteen common
-  fields (issue 156). The embedded thumbnail's tags are under `Thumbnail`,
-  without the thumbnail image. The thirteen typed methods stay, each picking its
-  field from the tags `exif()` returns, typed as in `PhotoExif`. The example
-  script's JSON files now carry every tag.
-
-- 2026-10-01: The content cache no longer looks for a live photo that an earlier
-  version cached as one ZIP (issue 151). When the cache opens, a live photo's
-  file that no JSON file names is now always left alone.
-
-- 2026-10-01: `examples/download-albums.ts` logs in, opens the library, and for
-  every album downloads each photo to its save path, writes the photo's record
-  and EXIF fields to a JSON file beside it, and writes the album's photos to
-  `albums/<collectionID>.json` (issue 144). The build compiles it to
-  `dist/examples/`; the README's "Examples" section says how to run it.
-
-- 2026-10-01: A `Photo` has one async method for each field of `exif()`, named
-  and typed as in `PhotoExif`: `make()`, `model()`, `lensModel()`,
-  `dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`,
-  `iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()`
-  and `gpsAltitude()` (issue 148). Each calls `exif()` and returns its one
-  field, or undefined when the file lacks it. `Photo` implements a type with one
-  method per `PhotoExif` field, so the build's type check fails when a field has
-  no method. A test checks, on the JPEG and the HEIC, that each method gives the
-  same value as `exif()`.
-
-- 2026-10-01: Each original's save path is
-  `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>` under the library's
-  download directory, which defaults to `photos` in the working directory (issue
-  143). The date is the photo's `takenAt` in the machine's time zone.
-  `photo.savePath` is always a string, with or without a content cache, and
-  `isLocal` is true only when the original is at its save path.
-  `photo.download()` puts the original there, copied from the cache when the
-  cache holds it and fetched otherwise. `lib.backup()` does the same for each
-  file, writes each file's JSON beside its original, and links `collections/` to
-  the save paths.
-
-- 2026-10-01: `photo.exif()` and `backup-metadata --exif` read EXIF from
-  HEIC/HEIF originals, a live photo's HEIC image included, as well as JPEG and
-  the other image formats `exifreader` reads (issue 145). `exifreader` replaces
-  `exif-reader` and the JPEG segment scan; `PhotoExif` is unchanged. The `exif`
-  field of `backup-metadata --exif` is now exifreader's tag output, and
-  `exifRaw` holds the whole EXIF block it could not read. The tests use a real
-  HEIC, `test/exif.heic`.
-
-- 2026-10-01: A `Photo` has `savePath`, `isLocal`, `content()`, `exif()`,
-  `modifiedAt`, `hash` and `year` (issue 141). `savePath` is where
-  `lib.backup()` writes the original under the library's download directory; for
-  a live photo not yet stored, it carries the title's extension, and the backup
-  may store the image under a different one. `isLocal` says whether the whole
-  original is there. Both look only at the disk. `content()` returns the
-  original's bytes and `exif()` the common EXIF fields of a JPEG; both may
-  download the original, and `exif()` downloads no video. `PhotoRecord` gains
-  `modifiedAt` and `hash`, and the JPEG EXIF scan moved to `src/exif.ts`.
-
-- 2026-09-29: The "Workflow" list at the top of this file now says to branch
-  from `next` and open a pull request that targets `next`, that the repository
-  manager squash-merges a reviewed pull request into `next`, and that only sneak
-  merges `next` into `main`, as the README does (issue 137).
-
-- 2026-09-29: The README's "Development workflow" and "For LLMs" sections now
-  say work branches from `next`, every pull request targets `next`, the
-  repository manager squash-merges reviewed pull requests into `next`, and only
-  sneak merges `next` into `main` (issue 135).
-
-- 2026-09-29: Brought the README and this file in line with `next` after the
-  milestone merge (issue 132). The Next Step says no implementation work is open
-  and the cache design (issue 36) waits on sneak's review. README corrections:
-  the Getting Started comments say when the library refreshes; most, not all,
-  Makefile targets call a script; `script/lint` and `script/test` have no host
-  path, though `yarn test` does; the SRP handshake uses `fast-srp-hap`, outside
-  `crypto/`, and a thumbnail upload's MD5 uses `node:crypto`; the SRP password
-  is the first 16 bytes of a 32-byte subkey; the key attributes and token come
-  after SRP, after the TOTP code SRP may ask for, or after the email OTP that
-  replaces SRP when the account has email MFA on, and quak cannot answer a
-  passkey; the auth token is sent as URL-safe base64 with padding; every
-  `TypeError` is retried; each download attempt writes its own temporary files,
-  two for a live photo, and only the attempt that completes renames them into
-  place; `runMetadataBackup` records a failed download in the file's JSON; a
-  second `client.logout()` does not throw; `quak logout` with no session exits
-  0, and names the cache directory only when it knows it; `login`, `whoami` and
-  `logout` open no library; `--exif` records XMP and, for a JPEG, EXIF, and no
-  IPTC; which commands take `--json`; a failed ML data request may not have been
-  retried; the thumbnail fixer is not limited to baseline JPEG; `quak backup`
-  still fetches ML data; a backup's JSON holds the basic metadata fields quak
-  keeps and the private and public magic metadata, not every decrypted field,
-  and the README no longer lists what the magic metadata holds; the default
-  cache directory is the per-user one, not an XDG path on macOS; pinned
-  originals can exceed `cacheOriginalsMaxBytes`; which tests cover which
-  operations; a default read is only as current as the last refresh whose
-  requests all succeeded; `fresh()` and `lib.backup()` join a refresh already
-  running; a `Photo` has no `thumbnailPath` or `originalPath`; and `408` and
-  `429` are retried.
-
-- 2026-09-28: Tested the live-photo writer's fsyncs (issue 130). A test checks
-  that the image's and the video's temp files are fsynced before either is
-  renamed into place, and the directory after both renames, as the `writeAtomic`
-  test does for one file.
-
-- 2026-09-23: Live photos are stored as their image and their video (issue 107).
-  A live photo, which Ente stores as one ZIP, is unpacked as it downloads into
-  `<fileID>.<ext>` for the image and for the video, each with its extension from
-  the ZIP, beside `<fileID>.livephoto.json`, which names the two. Both are
-  checked against the recorded hash and renamed into place only when both are
-  complete. A ZIP holding a second image or video, or whose two files come to
-  more than 20 times its size plus 16 MiB, is refused. The backup and the
-  content cache count the live photo as stored only with both files, the album
-  folder links both, `quak get` writes both, and the content result gives the
-  video as `videoPath`. A ZIP an earlier version stored is replaced on the next
-  backup run, and removed from the cache when the library opens it.
-
-- 2026-09-23: Settled the package metadata (issue 6). quak is not published, so
-  `package.json` is marked `"private": true` and the `files` field is gone.
-  `engines.node` is `>=22`, the major version `script/bootstrap` and the
-  `Dockerfile` use. An `exports` map makes `.` and `./package.json` the only
-  importable paths; `runMetadataBackup`, the thumbnail helpers and their types
-  stay internal to the CLI.
-
-- 2026-09-23: Brought the README and this file in line with the tree (issue
-  111). The layout lists `src/library/` and the other source files, the backup
-  layout names `failures.json` and the optional `thumbnails/`, "Opening a
-  library" says what happens when the first refresh fails, the Testing section
-  gives the 60-second hard cap and 20-second target for `make test` and names
-  the 90-second `timeout` in the `Dockerfile` as the backstop for a hung test,
-  and "Tag v1.0.0" is no longer listed as the next step.
-
-- 2026-09-23: Tested `quak login` and `backup-metadata --exif` (issue 110).
-  `loginCommand` takes its login function and its prompts from `CliContext`, and
-  `bin/quak.ts` passes `Client.login` and the terminal prompts. Tests cover a
-  login from `QUAK_EMAIL` and `QUAK_PASSWORD` with no prompt, the TOTP prompt, a
-  failed login, and the saved session's modes, and show that `--exif` and
-  `--all` each turn on EXIF extraction and that it is off without them.
-
-- 2026-09-23: `quak backup` writes each original once and no longer fills the
-  cache (issue 106). An original fetched for a backup is written by the download
-  writer straight into the backup's `originals/`, and the content cache records
-  it there instead of keeping its own copy; one the cache already held is still
-  copied. `quak backup` opens its library with the thumbnail and originals
-  precache off.
-
-- 2026-09-23: `backup-metadata`, `helper list-missing-thumbnails` and
-  `helper fix-missing-thumbnails` refresh before they answer (issue 100). Each
-  awaits `lib.fresh()` before reading, so a file added since the cache was
-  written is included, and a failed refresh prints one line and exits 1 instead
-  of answering from a stale or empty cache. The README lists them among the
-  commands that refresh first.
-
-- 2026-09-23: `quak backup` waits for the server refresh and fails when it fails
-  (issue 99). `lib.backup()` joins a refresh already running or starts one, as
-  `fresh()` does, and rejects before touching any file when it fails, leaving
-  `failures.json` as it was, so `quak backup` prints the error as one line and
-  exits 1 instead of backing up the previous run's file list, or nothing, and
-  exiting 0.
-
-- 2026-09-23: CLI errors print a message instead of a stack trace (issue 102).
-  An error a command throws is printed as one `quak: MESSAGE` line on stderr and
-  the CLI exits 1 once output has drained. The wrapper that does this moved from
-  `bin/quak.ts` to `src/cli-run.ts`, and `bin/quak.ts` now awaits
-  `program.parseAsync()`.
-
-- 2026-09-23: Opening a library no longer deletes another process's download in
-  progress (issue 105). The download writer's temp files are named
-  `.quak-<pid>-<random>.tmp`, and `removeLeftoverTempFiles`, moved from the
-  backup into the download module, deletes a `.quak-*.tmp` file only when the
-  process ID in its name is no longer running. The content cache calls it at
-  `open()` for `originals/` and `thumbnails/`, the backup as before.
-
-- 2026-09-23: Re-vendored the lint and test setup from the template (issue 96).
-  Linting and testing are the `lint` and `test` phases of the `Dockerfile`;
-  `script/lint` and `script/test` each build one with `--no-cache`, and the last
-  stage compiles and depends on both, so `script/cibuild` is one build.
-  `Dockerfile.lint`, `CHECK_EPOCH`, `LINT_EPOCH` and the tests that checked them
-  are gone; `REPO_POLICIES.md` is re-copied.
-
-- 2026-09-23: Stopped `helper fix-missing-thumbnails` retrying files the server
-  always refuses (issue 109). Both thumbnail helpers skip a file another account
-  owns without fetching it. The fixer skips a file whose recorded thumbnail size
-  is 0 or unknown before downloading it, and otherwise tries smaller encodings
-  (720 px quality 50 down to 160 px quality 20) until the encrypted thumbnail is
-  no larger than that size, skipping the file if none fits.
-
-- 2026-09-23: Tested the live-photo hash check's error paths (issue 117). Tests
-  download a live photo whose ZIP names an unknown compression method, one whose
-  ZIP has no image entry and one with no video entry, and check that nothing is
-  stored and the error names the file ID; the unreadable one is not retried.
-
-- 2026-09-23: `quak logout` ends the session on the server (issue 108). It calls
-  `POST /users/logout` through the new `Client.logoutOnServer()`, then deletes
-  `session.json` even when that call fails, says so and exits 1. It prints the
-  account's cache directory and says it still holds decrypted data. The default
-  cache path is now `defaultCacheDirectory()` in the library, shared with
-  `Library.open`.
-
-- 2026-09-23: Fixed the backup's per-collection folders (issue 103). Two files
-  in one collection with the same title, and two collections with the same name,
-  each get their ID added to the name (`IMG_0001 (12345).JPG`, `Trip (10)/`), so
-  none replaces another's symlink or JSON. Each run removes symlinks into
-  `originals/` for files no longer in the collection, and the folders of deleted
-  or renamed collections, leaving anything else in `collections/` alone. The
-  README backup layout states the naming rule.
-
-- 2026-09-23: Checked downloaded originals against their recorded content hash
-  (issue 68). `downloadFile`, which `quak get`, the content cache and backup all
-  use, hashes the decrypted bytes (unkeyed BLAKE2b-512, standard base64) and
-  stores nothing on a mismatch, failing with an error naming the file ID. A live
-  photo ZIP is unpacked as it streams with `fflate` and its image and video
-  hashed separately as `<imageHash>:<videoHash>`. `decryptFile` reads older
-  clients' `imageHash` and `videoHash` fields for live photos. A file with no
-  recorded hash is stored unchecked.
-
-- 2026-09-23: Kept one account's cache from mixing with another's (issue 104).
-  When `metadata.json` in the cache directory was written for a different,
-  non-zero user ID than the client's, `Library.open` deletes it and `mldata/`
-  and starts empty, so the first refresh enumerates from 0. This only happens
-  with `--cache-dir` or an explicit `cacheDirectory`; the default path already
-  includes the user ID. A test opens one account's cache as another account.
-
-- 2026-09-23: `backup-metadata` no longer stops on one failed ML data request
-  (issue 101). Each request of up to 200 files is tried on its own; a failed one
-  is logged, its files are written with the reason in `mlDataError`, and the
-  command exits 1 once the dump is complete. `fetchMLData`, which only this
-  command used, is gone; the command calls `fetchMLDataBatch` per batch.
-- 2026-09-23: Single-sourced the version string (issue 5). `package.json` is the
-  only place it is written: `src/index.ts` imports it for `VERSION` and
-  `bin/quak.ts` passes `VERSION` to commander. tsc copies `package.json` to
-  `dist/package.json`, so the import resolves from the built output too, and
-  `script/build` runs the built CLI with `--version` to prove it. A test checks
-  that `VERSION` and `quak --version` both equal the `package.json` version.
-- 2026-09-23: Tested that `Library.close()` waits for the originals precache
-  (issue 93). The test that holds a precache fetch open while `close()` runs now
-  runs once with only the thumbnail fill and once with only the originals fill,
-  so dropping either wait from `Precache.close()` fails a test.
-- 2026-09-23: Fixed two intermittently failing library tests (issue 90).
-  `Library.close()` now returns a promise that resolves once an in-flight
-  refresh (including its cache write), the ML data fetch and running precache
-  sweeps have finished; the library tests await it, so `afterEach` no longer
-  removes the cache directory while something is still writing into it. The
-  precache test waits for both fills to report "done" instead of for its stub
-  source to be called, which happened before the cache recorded the file.
-- 2026-09-23: Pinned three guards reviewers found untested (issue 89). The
-  download idle deadline's timer is unref'd, so it can never keep the process
-  alive, and a test checks no timer is left after a download completes or fails.
-  A test covers the rejection of `#` in a request path. The EXIF scan compares
-  the `Exif` header only in an APP1 segment of length 8 or more, so it never
-  reads the next segment's bytes, with a test for a short one.
-- 2026-09-23: Made the download deadline an idle deadline (issue 24).
-  `downloadTimeoutMs` now aborts a file or thumbnail download only after no
-  bytes have arrived for that long, default 60 seconds, instead of bounding the
-  whole transfer at 10 minutes, so a slow download that keeps making progress
-  completes. A download that fails before reading the whole body cancels it, so
-  a failed file no longer holds its connection.
-- 2026-09-23: Every `ApiClient` request URL is now built by one function next to
-  the class (issue 18), so a self-hosted `apiOrigin` with a base path keeps it
-  on every request, a path works with or without a leading slash, and query
-  parameters are percent-encoded. A path containing `?` or `#` is rejected with
-  an error instead of being silently cut.
-- 2026-09-23: Made the CLI testable and tested it (issue 12). The command bodies
-  moved from `bin/quak.ts` into `src/cli-commands.ts` as functions that take
-  their options and a context (output streams, session directory, cache
-  directory, session loader) and return an exit code; `bin/quak.ts` only wires
-  them to commander and exits with the code once stdout and stderr have drained,
-  so nothing below it calls `process.exit`. `test/cli/commands.test.ts` drives
-  them with a fake client: session file modes, logout, the missing and corrupt
-  session paths, and the output and exit code of `whoami`, `collections`,
-  `files`, `get`, `get-thumb`, `backup` and `helper list-missing-thumbnails`.
-- 2026-09-23: Hardened the backup tree's atomic copy (issue 22). `copyAtomic`
-  fsyncs its temp file before the rename and the directory after it, through the
-  download writer's `fsyncPath`; each backup run deletes `.quak-backup-*.tmp`
-  files whose process is no longer running. The README backup layout names the
-  temp files and states that the rename replaces a symlink and takes the temp
-  file's permissions. Added tests for a missing and an unwritable destination
-  directory for `downloadFile` and `downloadThumbnail`.
-- 2026-09-23: Hardened the JPEG EXIF scan behind `backup-metadata --exif` (issue
-  11). Every segment length is checked against the remaining bytes and lengths
-  under 2 stop the scan, so a truncated or corrupt original can neither throw
-  nor loop. A malformed or unparseable EXIF segment is recorded as
-  `imageMetadata.exifError`, and a failure to read the original as
-  `imageMetadataError` in the per-file JSON, instead of the field being left
-  out.
-- 2026-09-22: Hardened the retry classifier (issue 80). A `POST` or `PUT` is
-  replayed only when every errno in the cause chain is a connect errno, and it
-  no longer follows redirects. `getRetryOptions()` returns a copy. Tests pin
-  every errno the classifier names, the cause-chain depth limit, cycle
-  termination, and a fresh deadline per attempt for every retrying entry point.
-  The README's endpoint list is the one place that names the requests the replay
-  rule covers.
-- 2026-09-22: Stopped `make test` collecting tests from checkouts nested under
-  `.claude/` (issue 25). vitest ignores `.gitignore` when finding tests, so a
-  nested checkout ran the whole suite again; `vitest.config.ts` now adds
-  `.claude/**` to vitest's default excludes, and
-  `test/packaging/nested-checkout.test.ts` plants a nested checkout in a temp
-  directory and fails if vitest would collect it.
-- 2026-09-22: Dropped the deprecated `@types/libsodium-wrappers-sumo` stub from
-  `devDependencies` (issue 27). It shipped no declarations; the types come from
-  `libsodium-wrappers-sumo` itself. `yarn.lock` regenerated by `yarn remove`.
-- 2026-09-22: Hardened the client session lifecycle (issue 10).
-  `Client.fromJSON` checks every snapshot field and each key's decoded length
-  and names the bad field; `toJSON` reads the token through
-  `ApiClient.getAuthToken` and throws when there is none; `logout` zeroes the
-  key buffers, and `collectionsSince` re-checks for logout after its request so
-  it never decrypts with zeroed keys. The CLI reports a corrupt session file
-  separately from a missing one (`src/cli-session.ts`).
-- 2026-09-22: Sanitized file names taken from server metadata (issue 9). A new
-  `src/filename.ts` holds the one sanitizer, used by `quak get`/`get-thumb`
-  without `--out`, `downloadFile`/`downloadThumbnail` without `outPath`, and the
-  backup and metadata backup trees; it removes separators, control characters,
-  leading dots and Windows device names, and falls back to a name built from the
-  ID for an empty title. Originals-cache extensions are letters and digits only,
-  else `.bin`. A user-supplied path is used as is. `decryptFile` reads a missing
-  or non-string title as "" and rejects metadata that is not a JSON object.
-- 2026-09-22: Rewrote the README API reference (and the Getting Started / usage
-  snippets) to match the shipped cache/API library on `next` (issue 53, issue
-  13). Documented `Library.open` and its options, the default-read vs `fresh()`
-  distinction (and that the CLI's read commands are fresh), the record types and
-  `snapshot()`/`subscribe()`, the `albums`/`photos`/`timeline` read surface,
-  `Photo` content methods, `thumbnails.ensure`, the `mldata` search surface,
-  `backup()`, the three request pools (10/5/25), and the on-disk cache layout;
-  noted the deferred content-hash integrity check (issue 68). Docs-only; no code
-  changed.
-- 2026-09-22: Added resumable, deletion-aware enumeration to `Client` (issue 38,
-  closes issue 7). `collectionsSince`/`filesSince` take a starting cursor,
-  decrypt live records, surface tombstoned ids in a separate `deleted` list (a
-  tombstone has nothing to decrypt, so it is a bare id, not a hollow record),
-  and return the max `updationTime` seen as the cursor to resume from.
-  `filesSince` refuses to loop when the diff reports `hasMore` without advancing
-  the cursor (issue 7). `listCollections`/`listFiles` are now thin wrappers that
-  enumerate from `sinceTime: 0` and drop deletions, so existing callers are
-  unaffected.
-- 2026-09-22: Carried file size, thumbnail size, and the deletion flag through
-  `decryptFile` (issue 37, foundation for the cache/API design). Live files now
-  populate `file.size`/`thumbnail.size` from the server's `info` (left
-  `undefined` when the server omits it), and `isDeleted` is carried from the
-  diff row onto `EnteFile`. No caller change: `listFiles` still filters deleted
-  rows before decrypting. Surfacing a tombstone through decryption belongs to
-  the enumeration unit (issue 38).
-- 2026-08-10: Made `lint-once.test.ts` enforce what its header claims. It walked
-  `make check` only, so it never read `Dockerfile` — the image CI builds through
-  `script/cibuild` — and a second `prettier --check .` could be added there with
-  the suite staying green. The walk now also starts at
-  `.gitea/workflows/check.yml` and follows its `run:` steps, so the graph under
-  test is the one CI executes rather than the one someone assumed it executes.
-  The lockfile assertion was a substring check against the whole of
-  `script/bootstrap`, which has two install sites and so reported the branch the
-  containers never take; the two branches are now resolved separately and every
-  `yarn install` in each is required to be `--frozen-lockfile`. Prettier is
-  counted per occurrence instead of per line, so two invocations chained with
-  `&&` no longer read as one, and edges are followed on counted lines instead of
-  being skipped. Every way for the walk to reach nothing — an unknown target, an
-  unknown script, a missing file, a node with no commands, an unknown node kind
-  — is a thrown error rather than a quiet zero. Every assertion in the file was
-  mutation-tested individually.
-- 2026-08-10: Stopped `make check` running `prettier --check .` twice. Since
-  linting moved into Docker, the duplicate was one container pass and one host
-  pass of the same check: `script/lint` builds `Dockerfile.lint`, which runs
-  prettier as a build step, and `script/check` then called `script/fmt-check` as
-  well. The host call is gone from `script/check` and from `script/precommit`;
-  the container keeps checking formatting, because a successful
-  `Dockerfile.lint` build is what CI treats as proof of a clean tree, and it is
-  also what still fails the pre-commit hook on a badly formatted tree.
-  `script/fmt-check` survives as a standalone entrypoint, whose verdict cannot
-  drift from the container's. A test walks the invocation graph from each
-  entrypoint — through the Makefile shims, the `script/` calls and the
-  `docker build` — and asserts the prettier count, so the duplication cannot
-  come back unnoticed.
-- 2026-08-10: Moved all linting into Docker. `script/lint` builds a new root
-  `Dockerfile.lint`, which copies the repo into the digest-pinned node image and
-  runs eslint and prettier as build steps, so a successful build is a clean
-  lint; no host lint path remains and `yarn lint` is gone from `package.json`. A
-  fail-closed `LINT_EPOCH` guard stops Docker serving the linter layers from
-  cache, which is how a lint build returns success in under a second having
-  linted nothing. The lint stage inside `Dockerfile` and its `COPY --from=lint`
-  ordering hack are gone: that image now runs `make test` and `make build` only,
-  because `script/check` calls `script/lint` and running it in a container would
-  mean docker inside docker. `script/cibuild` builds the lint image first, then
-  the test and build image.
-- 2026-08-09: Made `make docker` green and policy-conformant. Multi-stage
-  Dockerfile: a lint stage runs `make fmt-check` and `make lint`, and the check
-  stage takes a `COPY --from=lint` dependency on it before running `make check`
-  and `make build`. `CHECK_EPOCH` and a fail-closed guard stop Docker serving
-  those two layers from cache, which is what let a build report success without
-  running the suite. `script/projectname` says `quak`, so the image is tagged
-  `quak`; `script/bootstrap` updates apt lists before installing, so a Debian
-  base works; `.dockerignore` no longer ships the compiled binary, the caches or
-  agent worktrees into the build context, and keeps `.gitignore` in it for
-  prettier.
-- 2026-08-09: Fixed the TypeScript build. `rootDir` is the repo root, so `bin/`
-  compiles alongside `src/` instead of failing with TS6059; output is
-  `dist/src/` and `dist/bin/`, which is where `main`, `types` and `bin.quak` now
-  point. `script/build` verifies the declared entrypoints exist after the
-  compiler runs and makes the CLI executable, the Dockerfile runs `make build`
-  as well as `make check`, and a `quak` script makes the README's
-  `yarn quak <command>` examples work.
-- 2026-08-09: Retry policy: no retry on 4xx (except `408` and `429`),
-  exponential backoff with full jitter on 5xx, transport failures and truncated
-  transfers, under per-attempt deadlines that cover the response body as well as
-  the request. Downloads retry request, stream consumption and decryption as one
-  unit; `postJSON` and `putJSON` are replayed only when the connection was never
-  established.
-- 2026-08-09: Downloads verify the secretstream terminated on `TAG_FINAL` and
-  write output atomically: a truncated body is rejected instead of landing on
-  disk as a short file, and plaintext is staged in a sibling temp file and
-  renamed into place, so a failed download leaves the destination untouched.
-- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
-  shims, README Entrypoints section
-- 2026-06-10: Decrypted collections shared by other users (sealed-box keys);
-  listCollections drops deleted-collection tombstones.
-- 2026-06-10: Login hardening: dual-2FA empty-string fields handled, TOTP
-  preferred when a passkey is also enrolled, interactive input via
-  @inquirer/prompts.
-- 2026-06-10: Replaced sharp with pure JS (jpeg-js + exif-reader); added
-  single-binary bun build and make install.
-- 2026-06-09: Added backup-metadata command (ML data always included, --exif
-  opt-in); rewrote README to match the implementation; added thumbnail helper
-  tests.
-- 2026-05-13: Full CLI surface: login, backup with dedup symlink layout,
-  collections, files, get, get-thumb, thumbnail repair helpers.
-- 2026-05-13: Client OO API with literate usage tests; file download and
-  decryption; all three metadata layers decrypted and persisted; renamed quack
-  to quak.
-- 2026-05-11: SRP login flow (email OTP + TOTP) and ApiClient.
-
 # Future Steps
 
 - Future desktop client, separate repo:

bin/clip-keyword-vectors

@@ -0,0 +1,31 @@
+#!/bin/sh
+# Writes src/clip-keyword-vectors.ts and test/clip-keywords-sunset.json by
+# running bin/clip-keyword-vectors.py in a Python image pinned by digest, with
+# its packages pinned by hash. Run it by hand from a checkout, then `make fmt`;
+# no gate runs it. Besides the Python image, it downloads about 265 MB, most of
+# it Ente's two CLIP models.
+set -eu
+
+ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+
+# python:3.13-slim (Python 3.13.17 on Debian 13.7), 2026-10-21. The wheels
+# below are the ones for this image on linux/amd64.
+IMAGE=python@sha256:57b29a7c050c4e679e4bea572e14b3965cf4b5839de2da49d55e49ffc4024f39
+
+main() {
+    cd "$ROOT"
+    docker run --rm -i --platform linux/amd64 \
+        --user "$(id -u):$(id -g)" -e HOME=/tmp \
+        -v "$ROOT:/repo" -w /repo "$IMAGE" sh -c '
+            python -m venv /tmp/venv &&
+            /tmp/venv/bin/pip install --quiet --disable-pip-version-check \
+                --no-cache-dir --no-deps --require-hashes -r /dev/stdin &&
+            /tmp/venv/bin/python bin/clip-keyword-vectors.py' <<'EOF'
+numpy==2.5.3 --hash=sha256:a5fa86b80fd24bcd1aff83ad23be44ea323de3f787be8f8b15d4a65621e25321
+onnxruntime==1.30.0 --hash=sha256:86f940afc801ea9681a4da8af84fbe95e1d9ea7d80903952cc1bfad54faad38f
+pillow==12.3.0 --hash=sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3
+tokenizers==0.23.2 --hash=sha256:41c2f84d172449b4dadb9cdc508e3e364076613c35b16e76ecfe47a60d1e3305
+EOF
+}
+
+main "$@"

bin/clip-keyword-vectors.py

@@ -0,0 +1,158 @@
+"""Write src/clip-keyword-vectors.ts and test/clip-keywords-sunset.json.
+
+bin/clip-keyword-vectors runs this in a pinned Python image. It downloads
+Ente's Discover list, Ente's CLIP text and image models, the CLIP tokenizer
+and a public-domain photo of a sunset, checks each against its sha256, and
+writes:
+
+- src/clip-keyword-vectors.ts: each Discover category of Ente's photos app,
+  with the text embedding Ente's CLIP text model gives its query;
+- test/clip-keywords-sunset.json: the CLIP embedding Ente's image model gives
+  the photo, prepared as Ente's apps prepare a photo.
+
+Both embeddings are scaled to length 1, as Ente's apps scale them.
+"""
+
+import hashlib
+import io
+import json
+import urllib.request
+
+import numpy as np
+import onnxruntime
+from PIL import Image, ImageOps
+from tokenizers import Tokenizer
+
+ENTE_COMMIT = "f5ad70a860ed25d0c8057b0efb4bd6eaefe4b0c5"
+DISCOVER_PATH = "mobile/apps/photos/assets/discover.json"
+DISCOVER_URL = (
+    f"https://raw.githubusercontent.com/ente-io/ente/{ENTE_COMMIT}/{DISCOVER_PATH}"
+)
+DISCOVER_SHA256 = "cf70ac7116b592f7921ca4cf67cdbb421a8e6b287fbe44b2be678b8d198e5c24"
+
+TEXT_MODEL = "mobileclip_s2_text_opset18_quant.onnx"
+TEXT_MODEL_SHA256 = "d92f33dfcff83077fc2e0d3414250710efbb51795dfd89767bdbefb5fdc47322"
+IMAGE_MODEL = "mobileclip_s2_image_gelu_opset20.onnx"
+IMAGE_MODEL_SHA256 = "205a430af825e501c5138e5bb9abea942482a7a4fd4a680e98e47cf0830dce7e"
+
+# The CLIP tokenizer Hugging Face publishes for OpenAI's CLIP. Its vocabulary
+# and merges are those of bpe_simple_vocab_16e6.txt, the vocabulary Ente's
+# apps tokenize with.
+TOKENIZER_URL = (
+    "https://huggingface.co/openai/clip-vit-base-patch32/resolve/"
+    "3d74acf9a28c67741b2f4f2ea7635f0aaf6f0268/tokenizer.json"
+)
+TOKENIZER_SHA256 = "b556ac8c99757ffb677208af34bc8c6721572114111a6e0aaf5fa69ff0b8d842"
+
+# "Sunset off the Massachusetts coast", a NOAA photo in the public domain:
+# https://commons.wikimedia.org/wiki/File:Sunset_off_the_Massachusetts_coast_-_NOAA.jpg
+PHOTO_URL = (
+    "https://upload.wikimedia.org/wikipedia/commons/d/db/"
+    "Sunset_off_the_Massachusetts_coast_-_NOAA.jpg"
+)
+PHOTO_SHA256 = "be62f3289c9bf7b75af68cd6978021d54e8a531d5d4cc493f6e9fbf5e1ab74d3"
+
+# Ente's text model takes 77 token ids, padded with 0; its image model takes
+# a 256x256 RGB image.
+TOKEN_COUNT = 77
+IMAGE_SIZE = 256
+
+
+def download(url, sha256):
+    # Wikimedia refuses requests without a User-Agent.
+    request = urllib.request.Request(url, headers={"User-Agent": "quak"})
+    with urllib.request.urlopen(request) as response:
+        data = response.read()
+    digest = hashlib.sha256(data).hexdigest()
+    if digest != sha256:
+        raise SystemExit(f"{url} has sha256 {digest}, not {sha256}")
+    return data
+
+
+def ente_model(name, sha256):
+    return onnxruntime.InferenceSession(
+        download(f"https://models.ente.com/{name}", sha256),
+        providers=["CPUExecutionProvider"],
+    )
+
+
+def run(session, tensor):
+    (output,) = session.run(None, {session.get_inputs()[0].name: tensor})
+    embedding = output[0]
+    return embedding / np.linalg.norm(embedding)
+
+
+def text_embedding(session, tokenizer, query):
+    ids = tokenizer.encode(query).ids
+    ids += [0] * (TOKEN_COUNT - len(ids))
+    return run(session, np.array([ids], dtype=np.int32))
+
+
+def image_embedding(session, photo):
+    image = ImageOps.exif_transpose(Image.open(io.BytesIO(photo))).convert("RGB")
+    # Scale to cover IMAGE_SIZE x IMAGE_SIZE and crop the centre.
+    square = ImageOps.fit(image, (IMAGE_SIZE, IMAGE_SIZE), Image.Resampling.BILINEAR)
+    pixels = np.asarray(square, dtype=np.float32) / 255
+    return run(session, pixels.transpose(2, 0, 1)[np.newaxis])
+
+
+def numbers(embedding):
+    return ", ".join(str(value) for value in embedding)
+
+
+def write_categories(prompts, embeddings):
+    lines = [
+        "// Written by bin/clip-keyword-vectors; do not edit by hand.",
+        "//",
+        "// The Discover categories of Ente's photos app, in their order, from",
+        f"// {DISCOVER_PATH} in https://github.com/ente-io/ente",
+        f"// at commit {ENTE_COMMIT}. Each has the text",
+        f"// embedding Ente's CLIP text model ({TEXT_MODEL})",
+        "// gives its query, scaled to length 1. `showVideo` is false for a category",
+        "// that leaves out videos.",
+        "",
+        "export const DISCOVER_CATEGORIES = [",
+    ]
+    for prompt, embedding in zip(prompts, embeddings):
+        lines += [
+            "    {",
+            f"        title: {json.dumps(prompt['title'])},",
+            f"        query: {json.dumps(prompt['query'])},",
+            f"        minScore: {json.dumps(prompt['minScore'])},",
+            f"        showVideo: {json.dumps(prompt.get('showVideo', True))},",
+            f"        textEmbedding: [{numbers(embedding)}],",
+            "    },",
+        ]
+    lines.append("];")
+    with open("src/clip-keyword-vectors.ts", "w", encoding="utf-8") as file:
+        file.write("\n".join(lines) + "\n")
+
+
+def write_fixture(embedding):
+    with open("test/clip-keywords-sunset.json", "w", encoding="utf-8") as file:
+        file.write(
+            "{\n"
+            f'    "photo": {json.dumps(PHOTO_URL)},\n'
+            f'    "sha256": {json.dumps(PHOTO_SHA256)},\n'
+            f'    "embedding": [{numbers(embedding)}]\n'
+            "}\n"
+        )
+
+
+def main():
+    prompts = json.loads(download(DISCOVER_URL, DISCOVER_SHA256))["prompts"]
+    tokenizer = Tokenizer.from_str(
+        download(TOKENIZER_URL, TOKENIZER_SHA256).decode("utf-8")
+    )
+    text_model = ente_model(TEXT_MODEL, TEXT_MODEL_SHA256)
+    embeddings = [
+        text_embedding(text_model, tokenizer, prompt["query"]) for prompt in prompts
+    ]
+    write_categories(prompts, embeddings)
+
+    image_model = ente_model(IMAGE_MODEL, IMAGE_MODEL_SHA256)
+    write_fixture(image_embedding(image_model, download(PHOTO_URL, PHOTO_SHA256)))
+
+
+if __name__ == "__main__":
+    main()

bin/quak.ts

@@ -74,7 +74,11 @@ const context = (): CliContext => ({
     sessionDir: paths.data,
     cacheDir: program.opts<ProgramOptions>().cacheDir,
     loadSession: (path) => loadSession(path, logRequests(log())),
-    login: (opts) => Client.login({ ...opts, apiOptions: logRequests(log()) }),
+    login: (opts) =>
+        Client.login({
+            ...opts,
+            apiOptions: { ...opts.apiOptions, ...logRequests(log()) },
+        }),
     prompt: (message) => input({ message }),
     promptSecret: (message) => password({ message, mask: true }),
 });
@@ -85,7 +89,12 @@ const run = (command: Promise<number>): Promise<void> =>
 program
     .command("login")
     .description("Log in to an Ente account and save the session")
-    .action(() => run(loginCommand(context())));
+    .option(
+        "--server <url>",
+        "Address of a self-hosted Ente server's API, saved with the session " +
+            "(default: QUAK_SERVER, then Ente's hosted service)",
+    )
+    .action((opts: { server?: string }) => run(loginCommand(context(), opts)));
 
 program
     .command("whoami")
@@ -123,7 +132,6 @@ program
     .description("Download and decrypt a single file")
     .argument("<fileID>", "File ID (from `quak files`)")
     .option("--out <path>", "Output file path")
-    .option("--collection <id>", "Accepted for compatibility; ignored")
     .action((fileID: string, opts: { out?: string }) =>
         run(getCommand(context(), fileID, opts)),
     );
@@ -133,7 +141,6 @@ program
     .description("Download and decrypt a thumbnail")
     .argument("<fileID>", "File ID (from `quak files`)")
     .option("--out <path>", "Output file path")
-    .option("--collection <id>", "Accepted for compatibility; ignored")
     .action((fileID: string, opts: { out?: string }) =>
         run(getThumbCommand(context(), fileID, opts)),
     );

package.json

@@ -12,7 +12,7 @@
     "private": true,
     "type": "module",
     "engines": {
-        "node": ">=22"
+        "node": "^22.13.0 || >=23.5.0"
     },
     "main": "./dist/src/index.js",
     "types": "./dist/src/index.d.ts",

script/build-bin

@@ -0,0 +1,85 @@
+#!/bin/sh
+# script/build-bin: run script/build, then bundle the built dist/ into one
+# executable, bin/quak, which reports the version script/build stamped. Our own
+# extension to scripts-to-rule-them-all.
+#
+# bun compiles the binary from a copy of its own executable, so the binary
+# loads the same libraries as the bun that built it. nix's bun loads libraries
+# from the nix store: on Linux its C library, on macOS its ICU. A binary
+# compiled from it loads them too. nix-collect-garbage can delete them, and the
+# binary then no longer starts. This script therefore compiles with bun's own
+# release build, which loads only the system's libraries. It downloads that
+# build and checks it against the hash below before running it.
+set -eu
+
+ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+
+# bun 1.3.13, 2026-11-11, and the SHA-256 of each release archive below. The
+# x86-64 archives are the baseline builds, which also run on CPUs without AVX2.
+# The musl archives are for Linux systems that use the musl C library, such as
+# Alpine.
+BUN_VERSION="1.3.13"
+BUN_LINUX_X64_BASELINE_SHA256="9d8a24292a7068090205daac0a5a223f5f69736f5287e37bf88d3b4031edc750"
+BUN_LINUX_X64_MUSL_BASELINE_SHA256="88ca7c7ad235b498f549eea2f770f434e9f0f5e9ba95168a2d3a1f235184c394"
+BUN_LINUX_AARCH64_SHA256="70bae41b3908b0a120e1e58c5c8af30e74afae3b8d11b0d3fdd8e787ddfb4b22"
+BUN_LINUX_AARCH64_MUSL_SHA256="5385e978107ce4934298d8d6afe9bfbb898683f6cc23e6753a0da60bc60c5b81"
+BUN_DARWIN_X64_BASELINE_SHA256="a98ba6a480f22fda9b343626b906a4e26aa53618bf85d2bc5928ecf2ba45f0ed"
+BUN_DARWIN_AARCH64_SHA256="5467e3f65dba526b9fea98f0cce04efafc0c63e169733ec27b876a3ad32da190"
+
+# compile_with_downloaded_bun <archive-name> <sha256>
+compile_with_downloaded_bun() {
+    tmp="$(mktemp -d)"
+    trap 'rm -rf "$tmp"' EXIT
+    curl -fsSL -o "$tmp/$1.zip" \
+        "https://github.com/oven-sh/bun/releases/download/bun-v$BUN_VERSION/$1.zip"
+    # macOS always has shasum, but not every macOS has sha256sum.
+    if command -v sha256sum >/dev/null 2>&1; then
+        echo "$2  $tmp/$1.zip" | sha256sum -c -
+    else
+        echo "$2  $tmp/$1.zip" | shasum -a 256 -c -
+    fi
+    unzip -q "$tmp/$1.zip" -d "$tmp"
+    "$tmp/$1/bun" build dist/bin/quak.js --compile --outfile bin/quak
+}
+
+main() {
+    cd "$ROOT"
+    "$ROOT/script/build"
+    platform="$(uname -s)-$(uname -m)"
+    # musl's ldd begins its version output with "musl libc"; glibc's does not.
+    if ldd --version 2>&1 | grep -q '^musl libc'; then
+        platform="$platform-musl"
+    fi
+    case "$platform" in
+        Linux-x86_64)
+            compile_with_downloaded_bun bun-linux-x64-baseline \
+                "$BUN_LINUX_X64_BASELINE_SHA256"
+            ;;
+        Linux-x86_64-musl)
+            compile_with_downloaded_bun bun-linux-x64-musl-baseline \
+                "$BUN_LINUX_X64_MUSL_BASELINE_SHA256"
+            ;;
+        Linux-aarch64)
+            compile_with_downloaded_bun bun-linux-aarch64 \
+                "$BUN_LINUX_AARCH64_SHA256"
+            ;;
+        Linux-aarch64-musl)
+            compile_with_downloaded_bun bun-linux-aarch64-musl \
+                "$BUN_LINUX_AARCH64_MUSL_SHA256"
+            ;;
+        Darwin-x86_64)
+            compile_with_downloaded_bun bun-darwin-x64-baseline \
+                "$BUN_DARWIN_X64_BASELINE_SHA256"
+            ;;
+        Darwin-arm64)
+            compile_with_downloaded_bun bun-darwin-aarch64 \
+                "$BUN_DARWIN_AARCH64_SHA256"
+            ;;
+        *)
+            echo "build-bin: bun has no release build for $(uname -s) on $(uname -m)" >&2
+            exit 1
+            ;;
+    esac
+}
+
+main "$@"

src/api/client.ts

@@ -354,7 +354,13 @@ export class ApiClient {
         });
     }
 
-    async putFile(presignedURL: string, data: Uint8Array): Promise<void> {
+    // `contentMD5` is the value given to `getUploadURL`. The server signs it
+    // into the URL, and storage refuses a PUT without that `Content-MD5`.
+    async putFile(
+        presignedURL: string,
+        data: Uint8Array,
+        contentMD5: string,
+    ): Promise<void> {
         // Idempotent despite being a write: a presigned PUT stores one whole
         // object at one key in one request, so replaying it either overwrites
         // the same bytes or lands them for the first time. There is no partial
@@ -370,6 +376,7 @@ export class ApiClient {
                     headers: {
                         "Content-Type": "application/octet-stream",
                         "Content-Length": String(data.length),
+                        "Content-MD5": contentMD5,
                     },
                     body: data,
                     signal: AbortSignal.timeout(this.requestTimeoutMs),

src/backup.ts

@@ -14,9 +14,9 @@
 //         YYYY/YYYY-MM/YYYY-MM-DD/
 //             YYYY-MM-DD.<fileID>.<ext>    the decrypted bytes (the save path)
 //             YYYY-MM-DD.<fileID>.json     per-file metadata sidecar, with
-//                                          the file's ML data and its
-//                                          original's EXIF, XMP and
-//                                          dimensions
+//                                          the file's ML data, its
+//                                          keywords, and its original's
+//                                          EXIF, XMP and dimensions
 //         collections/<name>/<title>       symlink to the original
 //         collections/<name>.json          per-collection metadata
 //         account.json                     the account's email and user ID
@@ -77,6 +77,7 @@ import { readFile } from "node:fs/promises";
 import { dirname, extname, join, relative, resolve } from "node:path";
 import lockfile from "proper-lockfile";
 
+import { clipKeywords } from "./clip-keywords.js";
 import { removeLeftoverTempFiles } from "./download/index.js";
 import { describeError } from "./errors.js";
 import {
@@ -497,8 +498,9 @@ const imageMetadataFor = async (
     }
 };
 
-// The file's JSON: its basic fields, its magic metadata, its ML data or the
-// reason the ML data is missing, and its image metadata.
+// The file's JSON: its basic fields, its magic metadata, its ML data without
+// the CLIP embedding and the keywords that embedding gives, or the reason the
+// ML data is missing, and its image metadata.
 const writeSidecar = (
     path: string,
     file: EnteFile,
@@ -514,7 +516,13 @@ const writeSidecar = (
     };
     if (file.magicMetadata) meta.magicMetadata = file.magicMetadata;
     if (file.pubMagicMetadata) meta.pubMagicMetadata = file.pubMagicMetadata;
-    if (ml.mlData) meta.mlData = ml.mlData;
+    if (ml.mlData) {
+        const { clip: _clip, ...mlDataWithoutClip } = ml.mlData;
+        meta.mlData = mlDataWithoutClip;
+        const isVideo = file.metadata.fileType === "video";
+        const keywords = clipKeywords(ml.mlData, isVideo);
+        if (keywords) meta.keywords = keywords;
+    }
     if (ml.mlDataError) meta.mlDataError = ml.mlDataError;
     if (image.imageMetadata) meta.imageMetadata = image.imageMetadata;
     if (image.imageMetadataError) {

src/cli-commands.ts

@@ -37,9 +37,16 @@ import {
     thumbnailName,
 } from "./cli-output.js";
 import { type Log, logRefresh } from "./cli-log.js";
-import { freshCollections, freshFiles, freshFile } from "./cli-read.js";
+import {
+    cachedFilesByID,
+    freshCollections,
+    freshFiles,
+    freshFile,
+    freshFilesByID,
+} from "./cli-read.js";
 import { withExtension } from "./filename.js";
 import { runMetadataBackup } from "./metadata-backup.js";
+import type { Collection, EnteFile } from "./model/types.js";
 import { listMissingThumbnails, fixMissingThumbnails } from "./thumbnails.js";
 
 export interface CliContext {
@@ -128,12 +135,15 @@ const readLibraryClient = (client: Client): LibraryClient => ({
     contentSource: () => client.contentSource(),
 });
 
-// Open a library for a single point command: the aggressive background precache
-// (issue #48) is off — a one-shot `collections` or `get` must not start
-// downloading the whole account — and no refresh starts as it opens or on a
-// timer: each command asks for the one it needs, and `collections` needs none
-// (issue #210). While a refresh runs, each collection whose files it has
-// fetched is logged, so a large account shows progress rather than nothing.
+// Open a library for a single point command. The aggressive background
+// precache (issue #48) is off, because a one-shot `collections` or `get` must
+// not start downloading the whole account. No refresh starts as the library
+// opens or on a timer. Each command asks for the refresh it needs.
+// `collections` and `files` with a collection ID need none (issues #210 and
+// #217). Nor do `get` and `get-thumb` on an empty cache (issue #222). Nor does
+// `helper fix-missing-thumbnails --file` on an empty cache (issue #223). While
+// a refresh runs, each collection whose files it has fetched is logged, so a
+// large account shows progress rather than nothing.
 const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>
     Library.open({
         client: readLibraryClient(client),
@@ -145,8 +155,31 @@ const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>
         onProgress: logRefresh(ctx.log),
     });
 
-export const loginCommand = async (ctx: CliContext): Promise<number> => {
+// Logs in to the server `--server` or `QUAK_SERVER` names, the flag first, or
+// to Ente's hosted service when neither is set. The saved session keeps the
+// server, so every later command uses it.
+// Whether `text` can be an Ente server's address: an http or https URL, which
+// may have a path, and no query or fragment, which request URLs could not
+// carry.
+const isServerAddress = (text: string): boolean =>
+    URL.canParse(text) &&
+    ["http:", "https:"].includes(new URL(text).protocol) &&
+    !/[?#]/.test(text);
+
+export const loginCommand = async (
+    ctx: CliContext,
+    opts: { server?: string },
+): Promise<number> => {
     await init();
+    const server = opts.server ?? process.env.QUAK_SERVER;
+    if (server !== undefined && !isServerAddress(server)) {
+        ctx.log(
+            "fatal",
+            `Invalid server address "${server}": it must be an http or ` +
+                `https URL with no query or fragment`,
+        );
+        return 1;
+    }
     const email = process.env.QUAK_EMAIL ?? (await ctx.prompt("Email"));
     const password =
         process.env.QUAK_PASSWORD ?? (await ctx.promptSecret("Password"));
@@ -158,6 +191,8 @@ export const loginCommand = async (ctx: CliContext): Promise<number> => {
             password,
             totp: async () => ctx.prompt("TOTP code: "),
             emailOTP: async () => ctx.prompt("Email verification code: "),
+            apiOptions:
+                server === undefined ? undefined : { apiOrigin: server },
         });
 
         saveSession(ctx.sessionDir, client.toJSON());
@@ -282,11 +317,11 @@ export const filesCommand = async (
 
     const lib = await openReadLibrary(ctx, client);
     try {
-        // Force a server round-trip and list in enumeration order (issue #36
-        // amendment, issue #52). Each file prints from its own decrypted
-        // metadata (raw title, microsecond creationTime) via cli-output, and in
-        // the pre-library CLI's enumeration order, not the projection's
-        // newest-first order.
+        // Ask the server for current state (issue #36 amendment, issue #52).
+        // Each file prints from its own decrypted metadata (raw title,
+        // microsecond creationTime) via cli-output. The files print in the
+        // order `freshFiles` returns them, not the projection's newest-first
+        // order.
         const files = await freshFiles(lib, collectionID);
         if (!files) {
             ctx.log("fatal", `Collection ${collectionID} not found`);
@@ -323,14 +358,13 @@ export const getCommand = async (
     try {
         // Force a server round-trip so the file resolves against current state
         // (issue #36 amendment, issue #52).
-        const resolved = await freshFile(lib, fileID);
-        if (!resolved) {
+        const file = await freshFile(lib, fileID);
+        if (!file) {
             ctx.log("fatal", `File ${fileID} not found`);
             return 1;
         }
-        const { photo, file } = resolved;
 
-        const result = await photo.original();
+        const result = await lib.originalOf(file);
         // Default name is the file's own title, as the pre-library CLI used
         // (not the editedName-preferring projection title) (issue #52).
         if (result.videoPath === undefined) {
@@ -381,14 +415,13 @@ export const getThumbCommand = async (
     try {
         // Force a server round-trip so the file resolves against current state
         // (issue #36 amendment, issue #52).
-        const resolved = await freshFile(lib, fileID);
-        if (!resolved) {
+        const file = await freshFile(lib, fileID);
+        if (!file) {
             ctx.log("fatal", `File ${fileID} not found`);
             return 1;
         }
-        const { photo, file } = resolved;
 
-        const result = await photo.thumbnail();
+        const result = await lib.thumbnailOf(file);
         // Default name is thumb_<file's own title>, as the pre-library CLI
         // used (not the projection title) (issue #52).
         const outPath = opts.out ?? thumbnailName(file);
@@ -565,10 +598,13 @@ export const fixMissingThumbnailsCommand = async (
     }
     const lib = await openReadLibrary(ctx, client);
     try {
-        // Refresh first so files added since the cache was written are found;
-        // a failed refresh throws.
+        let found: Map<number, { file: EnteFile; collection: Collection }>;
+        if (fileIDs.length > 0) {
+            found = await freshFilesByID(lib, fileIDs);
+        } else {
+            // Refresh first so files added since the cache was written are
+            // checked; a failed refresh throws.
             await lib.fresh();
-        if (fileIDs.length === 0) {
             ctx.log("info", "Scanning for missing thumbnails...");
             const missing = await listMissingThumbnails(
                 lib,
@@ -581,12 +617,14 @@ export const fixMissingThumbnailsCommand = async (
                 return 0;
             }
             ctx.log("info", `Found ${fileIDs.length} file(s) to fix.`);
+            found = cachedFilesByID(lib, fileIDs);
         }
 
         const results = await fixMissingThumbnails(
             lib,
             client,
             fileIDs,
+            found,
             (msg, level) => ctx.log(level ?? "info", msg),
         );
 

src/cli-log.ts

@@ -1,15 +1,18 @@
 // The CLI's log level, and the one function the commands write stderr lines
 // through.
 //
-// Every line quak prints on stderr is a log line at one of six levels, from
-// the quietest: fatal, error, warning, info, debug, trace. A run prints the
-// lines at its own level and at every quieter one. Its level is info unless
-// `-v` or `-q` asks for another (see "CLI surface" in the README). Every
-// command that exits non-zero logs a fatal line saying why, so even `-q`, which
-// prints only fatal lines, never leaves a non-zero exit unexplained. An error
+// Every line quak logs goes to stderr, at one of six levels, from the
+// quietest: fatal, error, warning, info, debug, trace. A run prints the lines
+// at its own level and at every quieter one. Its level is info unless `-v` or
+// `-q` asks for another (see "What quak prints" in the README). A command that
+// runs and exits non-zero logs a fatal line saying why, so even `-q`, which
+// prints only fatal lines, never leaves such an exit unexplained. An error
 // line reports something that failed, such as a file or an album, and a
 // warning something that did not fail but needs the user's attention. stdout
-// carries only a command's output and is not a log.
+// carries a command's output and the prompts of `login`, and is not a log.
+// Two things on stderr are not log lines: the `error:` message for a
+// command line that quak cannot parse, and the usage printed when no command
+// is given.
 //
 // The library itself writes no lines: its layers report through callbacks, and
 // the CLI logs what they report. `logRequests` gives the API client options

src/cli-read.ts

@@ -2,38 +2,40 @@
 //
 // `collections`, `files`, `get`, and `get-thumb` must answer for
 // the account's state at the moment the command runs, not for whatever the
-// local cache last happened to hold (owner amendment, issue #36). `files`,
-// `get` and `get-thumb` therefore force a server round-trip through
-// `Library.fresh()` and only then read — so a collection, file, or metadata
-// change made elsewhere is visible. `collections` asks the library for the
-// collection list alone through `Library.freshCollections()` (issue #210): a
-// refresh would also fetch the files of every collection, which on a large
-// account takes long and which `collections` does not print.
+// local cache last happened to hold (owner amendment, issue #36). `get`,
+// `get-thumb` and `files` without a collection ID therefore force a server
+// round-trip through `Library.fresh()` and only then read, so a change made
+// elsewhere is visible. `collections` asks the library for the collection list
+// alone through `Library.freshCollections()` (issue #210): a refresh would also
+// fetch the files of every collection, which on a large account takes long and
+// which `collections` does not print. `files` with a collection ID reads that
+// list too, then that one collection's files through `Library.freshFiles()`
+// (issue #217). On an empty cache, `get` and `get-thumb` read that list, then
+// one collection's files at a time, until they find the file (issue #222).
+// `helper fix-missing-thumbnails --file` does the same until it finds every
+// file it names (issue #223).
 //
-// `collections` lists in the server's order and `files` in the library's own
-// enumeration order (`listFiles()`), the orders the pre-library CLI printed,
-// rather than the `albums`/`photos` projection's newest-first order, which
-// re-sorts the rows. The field values still come from each record's raw
-// metadata via `cli-output.ts`. `files` without a collection ID lists every
-// file in the account once (issue #211): the library keeps a record of a file
-// for each collection that holds it, and only the first collection's record is
-// listed.
+// `collections` and `files` with a collection ID list in the server's order.
+// `files` without one lists in the library's own enumeration order
+// (`listFiles()`). Neither command uses the `albums`/`photos` projection's
+// newest-first order, which re-sorts the rows. The field values still come
+// from each record's raw metadata via `cli-output.ts`. `files` without a
+// collection ID lists every file in the account once (issue #211): the library
+// keeps a record of a file for each collection that holds it, and only the
+// first collection's record is listed.
 
 import type { Collection, EnteFile } from "./model/types.js";
-import type { Photo, PhotosAPI } from "./library/index.js";
 
 // The slice of `Library` these helpers read. `Library` satisfies it
-// structurally; a test can drive them with a stand-in that records the
-// `fresh()` and `freshCollections()` calls and serves records in a known
-// enumeration order.
+// structurally; a test can drive them with a stand-in that records each fresh
+// read and serves records in a known enumeration order.
 export interface FreshReadLibrary {
     fresh(): Promise<unknown>;
     freshCollections(): Promise<Collection[]>;
-    getCollection(id: number): Collection | undefined;
+    freshFiles(collection: Collection): Promise<EnteFile[]>;
     listCollections(): Collection[];
     listFiles(collectionID: number): EnteFile[];
     getFileByID(fileID: number): EnteFile | undefined;
-    photos: Pick<PhotosAPI, "byID">;
 }
 
 // Every live collection, current as of one collections request, in the
@@ -42,21 +44,21 @@ export const freshCollections = (
     lib: FreshReadLibrary,
 ): Promise<Collection[]> => lib.freshCollections();
 
-// The files of one collection, current as of a forced refresh, in enumeration
+// The files of one collection, as the server holds them now, in the server's
 // order. `undefined` (not an empty list) when the collection does not exist, so
 // the caller can tell "no such collection" from "an empty collection". Without
-// a collection ID, every file in the account, each once, as held by the first
-// collection that holds it.
+// a collection ID, every file in the account as of a forced refresh, each once,
+// as held by the first collection that holds it.
 export const freshFiles = async (
     lib: FreshReadLibrary,
     collectionID?: number,
 ): Promise<EnteFile[] | undefined> => {
-    await lib.fresh();
     if (collectionID !== undefined) {
-        return lib.getCollection(collectionID)
-            ? lib.listFiles(collectionID)
-            : undefined;
+        const collections = await lib.freshCollections();
+        const collection = collections.find((c) => c.id === collectionID);
+        return collection ? lib.freshFiles(collection) : undefined;
     }
+    await lib.fresh();
     const listed = new Set<number>();
     const files: EnteFile[] = [];
     for (const collection of lib.listCollections()) {
@@ -69,16 +71,80 @@ export const freshFiles = async (
     return files;
 };
 
-// One file, current as of a forced refresh, resolved to both its content
-// handle (`Photo`, for fetching bytes) and its raw record (`EnteFile`, for the
-// default output name and field values). `undefined` when the file is unknown.
+// One file as the server holds it now, or `undefined` when no collection holds
+// it. When the cache holds the account's collections, it refreshes and looks
+// the file up in the cache. On an empty cache, it reads the collection list,
+// then one collection's files at a time, and stops at the first collection
+// that holds the file. That search stores nothing.
 export const freshFile = async (
     lib: FreshReadLibrary,
     fileID: number,
-): Promise<{ photo: Photo; file: EnteFile } | undefined> => {
+): Promise<EnteFile | undefined> => {
+    // Only a refresh stores collections, so a cache holding any holds them all.
+    if (lib.listCollections().length > 0) {
         await lib.fresh();
-    const photo = lib.photos.byID({ fileID });
-    const file = lib.getFileByID(fileID);
-    if (!photo || !file) return undefined;
-    return { photo, file };
+        return lib.getFileByID(fileID);
+    }
+    for (const collection of await lib.freshCollections()) {
+        const files = await lib.freshFiles(collection);
+        const file = files.find((f) => f.id === fileID);
+        if (file) return file;
+    }
+    return undefined;
+};
+
+// The most recently updated collection first, and the higher ID first on a
+// tie. `Library.albums.list()` lists albums in this order.
+const byNewestCollection = (a: Collection, b: Collection): number =>
+    b.updationTime - a.updationTime || b.id - a.id;
+
+// Each file named by `fileIDs` that the cache holds, mapped to its record and
+// the collection it was found in. Collections are searched in
+// `byNewestCollection` order, and a file is found in the first that holds it.
+// So `helper fix-missing-thumbnails` names a file after the same album as
+// `helper list-missing-thumbnails`. It reads the cache alone, so the caller
+// refreshes first.
+export const cachedFilesByID = (
+    lib: FreshReadLibrary,
+    fileIDs: number[],
+): Map<number, { file: EnteFile; collection: Collection }> => {
+    const wanted = new Set(fileIDs);
+    const found = new Map<number, { file: EnteFile; collection: Collection }>();
+    const collections = [...lib.listCollections()].sort(byNewestCollection);
+    for (const collection of collections) {
+        for (const file of lib.listFiles(collection.id)) {
+            if (wanted.has(file.id) && !found.has(file.id)) {
+                found.set(file.id, { file, collection });
+            }
+        }
+    }
+    return found;
+};
+
+// Each file named by `fileIDs` as the server holds it now, mapped to its record
+// and collection as `cachedFilesByID` maps it. When the cache holds the
+// account's collections, it refreshes and looks the files up in the cache. On
+// an empty cache, it reads the collection list, then one collection's files at
+// a time, and stops once it has found every file. That search stores nothing.
+export const freshFilesByID = async (
+    lib: FreshReadLibrary,
+    fileIDs: number[],
+): Promise<Map<number, { file: EnteFile; collection: Collection }>> => {
+    // Only a refresh stores collections, so a cache holding any holds them all.
+    if (lib.listCollections().length > 0) {
+        await lib.fresh();
+        return cachedFilesByID(lib, fileIDs);
+    }
+    const wanted = new Set(fileIDs);
+    const found = new Map<number, { file: EnteFile; collection: Collection }>();
+    const collections = await lib.freshCollections();
+    for (const collection of [...collections].sort(byNewestCollection)) {
+        if (found.size === wanted.size) break;
+        for (const file of await lib.freshFiles(collection)) {
+            if (wanted.has(file.id) && !found.has(file.id)) {
+                found.set(file.id, { file, collection });
+            }
+        }
+    }
+    return found;
 };

src/client.ts

@@ -40,6 +40,9 @@ export interface ClientSnapshot {
     masterKey: string;
     secretKey: string;
     publicKey: string;
+    // The `apiOrigin` the client was given, so a restored client talks to
+    // the server that issued its token. Absent means the default server.
+    apiOrigin?: string;
 }
 
 // The result of a resumable enumeration. Live decrypted records and deleted
@@ -73,6 +76,7 @@ export class Client {
     private readonly masterKey: Uint8Array;
     private readonly secretKey: Uint8Array;
     private readonly publicKey: Uint8Array;
+    private readonly apiOrigin: string | undefined;
     private loggedOut = false;
 
     private constructor(
@@ -82,6 +86,7 @@ export class Client {
         masterKey: Uint8Array,
         secretKey: Uint8Array,
         publicKey: Uint8Array,
+        apiOrigin: string | undefined,
     ) {
         this.api = api;
         this.email = email;
@@ -89,6 +94,7 @@ export class Client {
         this.masterKey = masterKey;
         this.secretKey = secretKey;
         this.publicKey = publicKey;
+        this.apiOrigin = apiOrigin;
     }
 
     static async login(opts: LoginOptions): Promise<Client> {
@@ -130,12 +136,15 @@ export class Client {
             unwrapped.masterKey,
             unwrapped.secretKey,
             unwrapped.publicKey,
+            opts.apiOptions?.apiOrigin,
         );
     }
 
     // Restore a client from a `toJSON()` snapshot. The snapshot usually comes
     // straight from `JSON.parse` of a file on disk, so every field is checked
-    // before use; a bad one throws an error naming it. Needs `init()` first.
+    // before use; a bad one throws an error naming it. The snapshot's
+    // `apiOrigin`, when it has one, is used instead of `apiOptions.apiOrigin`.
+    // Needs `init()` first.
     static fromJSON(snapshot: unknown, apiOptions?: ApiClientOptions): Client {
         const invalid = (field: string, problem: string): Error =>
             new Error(`Invalid session data: ${field} ${problem}`);
@@ -152,6 +161,14 @@ export class Client {
         if (!Number.isInteger(s.userID)) {
             throw invalid("userID", "must be an integer");
         }
+        if (
+            s.apiOrigin !== undefined &&
+            (typeof s.apiOrigin !== "string" || s.apiOrigin === "")
+        ) {
+            throw invalid("apiOrigin", "must be a non-empty string");
+        }
+        const apiOrigin =
+            (s.apiOrigin as string | undefined) ?? apiOptions?.apiOrigin;
         const key = (field: string): Uint8Array => {
             const value = s[field];
             if (typeof value !== "string") {
@@ -175,6 +192,7 @@ export class Client {
 
         const api = new ApiClient({
             ...apiOptions,
+            apiOrigin,
             authToken: s.token as string,
         });
         return new Client(
@@ -184,6 +202,7 @@ export class Client {
             key("masterKey"),
             key("secretKey"),
             key("publicKey"),
+            apiOrigin,
         );
     }
 
@@ -222,6 +241,9 @@ export class Client {
             masterKey: toBase64(this.masterKey),
             secretKey: toBase64(this.secretKey),
             publicKey: toBase64(this.publicKey),
+            ...(this.apiOrigin === undefined
+                ? {}
+                : { apiOrigin: this.apiOrigin }),
         };
     }
 

src/clip-keywords.ts

@@ -0,0 +1,57 @@
+// Keywords for a file from its CLIP embedding.
+//
+// Ente stores no keywords. A file's ML data holds a CLIP embedding, the 512
+// numbers Ente's image model gave the photo, and Ente's photos app sorts files
+// into the categories of its Discover section with it: a file falls in a
+// category when the cosine similarity of its CLIP embedding and the text
+// embedding of the category's query is at least the category's `minScore`,
+// unless the file is a video and the category leaves out videos. A file's
+// keywords are the titles of the categories it falls in. The categories and
+// their text embeddings are in `clip-keyword-vectors.ts`.
+
+import { DISCOVER_CATEGORIES } from "./clip-keyword-vectors.js";
+import { clipEmbedding } from "./library/mldata.js";
+import type { MLData } from "./mldata-fetch.js";
+
+// How many numbers a CLIP embedding from Ente's image model has.
+const EMBEDDING_LENGTH = 512;
+
+const norm = (vector: number[]): number => Math.hypot(...vector);
+
+const cosineSimilarity = (a: number[], b: number[]): number => {
+    let dot = 0;
+    a.forEach((value, i) => {
+        dot += value * (b[i] ?? 0);
+    });
+    return dot / (norm(a) * norm(b));
+};
+
+// The titles of the Discover categories a file falls in, highest score first,
+// or undefined when its ML data has no CLIP embedding of 512 finite numbers
+// that are not all zero. Tests pass their own `categories`.
+export const clipKeywords = (
+    mlData: MLData,
+    isVideo: boolean,
+    categories = DISCOVER_CATEGORIES,
+): string[] | undefined => {
+    const embedding = clipEmbedding(mlData);
+    if (
+        !embedding ||
+        embedding.length !== EMBEDDING_LENGTH ||
+        norm(embedding) === 0
+    ) {
+        return undefined;
+    }
+
+    const matches: { title: string; score: number }[] = [];
+    for (const category of categories) {
+        if (isVideo && !category.showVideo) continue;
+        const score = cosineSimilarity(embedding, category.textEmbedding);
+        if (score >= category.minScore) {
+            matches.push({ title: category.title, score });
+        }
+    }
+    return matches
+        .sort((a, b) => b.score - a.score)
+        .map((match) => match.title);
+};

src/library/content.ts

@@ -581,14 +581,34 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         fileID: number,
         opts?: ContentOptions,
     ): Promise<ContentResult> {
-        return this.get(fileID, "original", "on-demand", opts?.onProgress);
+        const file = this.fileOrThrow(fileID);
+        return this.get(file, "original", "on-demand", opts?.onProgress);
     }
 
     async thumbnail(
         fileID: number,
         opts?: ContentOptions,
     ): Promise<ContentResult> {
-        return this.get(fileID, "thumbnail", "on-demand", opts?.onProgress);
+        const file = this.fileOrThrow(fileID);
+        return this.get(file, "thumbnail", "on-demand", opts?.onProgress);
+    }
+
+    // The original of `file`, fetched and cached as `original` does. `file` is
+    // used as given, so `getFile` need not know it.
+    async originalOf(
+        file: EnteFile,
+        opts?: ContentOptions,
+    ): Promise<ContentResult> {
+        return this.get(file, "original", "on-demand", opts?.onProgress);
+    }
+
+    // The thumbnail of `file`, fetched and cached as `thumbnail` does. `file`
+    // is used as given, so `getFile` need not know it.
+    async thumbnailOf(
+        file: EnteFile,
+        opts?: ContentOptions,
+    ): Promise<ContentResult> {
+        return this.get(file, "thumbnail", "on-demand", opts?.onProgress);
     }
 
     // Put the original at the save path of `file` under the download directory
@@ -615,7 +635,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         destination: string,
     ): Promise<ContentResult> {
         const result = await this.acquire(
-            fileID,
+            this.fileOrThrow(fileID),
             "original",
             "on-demand",
             undefined,
@@ -638,8 +658,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         fileID: number,
         destination: string,
     ): Promise<{ path: string; videoPath?: string }> {
-        const file = this.getFile(fileID);
-        if (!file) throw new Error(`content cache: unknown file ${fileID}`);
+        const file = this.fileOrThrow(fileID);
         return this.pools.content.run(
             async () => {
                 const fetched = await this.fetchInto(
@@ -721,7 +740,8 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         onProgress: ((event: EnsureEvent) => void) | undefined,
     ): Promise<EnsureResult> {
         try {
-            const result = await this.acquire(fileID, kind, priority, signal);
+            const file = this.fileOrThrow(fileID);
+            const result = await this.acquire(file, kind, priority, signal);
             const status = result.cached ? "skipped" : "done";
             onProgress?.({ fileID, status, path: result.path });
             return { fileID, path: result.path };
@@ -736,8 +756,16 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         }
     }
 
+    // The record `getFile` gives for `fileID`. It throws for a file `getFile`
+    // does not know.
+    private fileOrThrow(fileID: number): EnteFile {
+        const file = this.getFile(fileID);
+        if (!file) throw new Error(`content cache: unknown file ${fileID}`);
+        return file;
+    }
+
     private async get(
-        fileID: number,
+        file: EnteFile,
         kind: Kind,
         priority: Priority,
         onProgress: ((event: ContentEvent) => void) | undefined,
@@ -745,7 +773,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         const onByte: ProgressCallback | undefined = onProgress
             ? (bytesDone) => onProgress({ status: "downloading", bytesDone })
             : undefined;
-        const result = await this.acquire(fileID, kind, priority, undefined, {
+        const result = await this.acquire(file, kind, priority, undefined, {
             onByte,
         });
         onProgress?.(
@@ -767,7 +795,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
     // photo's original is present only with its video, and is returned with
     // it.
     private async acquire(
-        fileID: number,
+        file: EnteFile,
         kind: Kind,
         priority: Priority,
         signal: AbortSignal | undefined,
@@ -778,8 +806,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
         videoPath?: string;
         cached: boolean;
     }> {
-        const file = this.getFile(fileID);
-        if (!file) throw new Error(`content cache: unknown file ${fileID}`);
+        const fileID = file.id;
         const isLivePhoto =
             kind === "original" && file.metadata.fileType === "livePhoto";
 

src/library/index.ts

@@ -20,9 +20,10 @@
 //
 // `freshCollections()` is the fresh read of the collection list alone (issue
 // #210): one collections request, no collection's files, and nothing stored,
-// for a caller such as `quak collections` that needs no more. A caller that
-// asks for each refresh it needs can also open the library with
-// `refreshOnOpen: false`, so that `open()` starts none.
+// for a caller such as `quak collections` that needs no more. `freshFiles()`
+// is the fresh read of one collection's files, and stores nothing either
+// (issue #217). A caller that asks for each refresh it needs can also open the
+// library with `refreshOnOpen: false`, so that `open()` starts none.
 //
 // A refresh reports through `onProgress` each collection whose files it has
 // fetched, so a large account, whose first refresh fetches the files of every
@@ -65,6 +66,8 @@ import {
     ContentCache,
     savePath,
     storedAtSavePath,
+    type ContentOptions,
+    type ContentResult,
     type ContentSource,
     type ThumbnailsAPI,
     type EnsureOptions,
@@ -625,6 +628,54 @@ export class Library {
         return collections;
     }
 
+    // The fresh read of one collection's files (issue #217): every live file
+    // in it that decrypts, in the server's order. `collection` is a record
+    // `freshCollections()` returned, whose key decrypts the files. It fetches
+    // no other collection's files. Like `freshCollections()`, it stores
+    // nothing. It returns each file once, as the latest copy the server sent,
+    // where the first copy was.
+    async freshFiles(collection: Collection): Promise<EnteFile[]> {
+        if (this.closed) throw new Error("the library is closed");
+        const { files } = await this.client.filesSince({
+            collectionID: collection.id,
+            collectionKey: collection.key,
+            sinceTime: 0,
+        });
+        const byID = new Map<number, EnteFile>();
+        for (const file of files) byID.set(file.id, file);
+        return [...byID.values()];
+    }
+
+    // The original of `file`, fetched and cached as `Photo.original()` does.
+    // `file` is used as given, so the store need not hold it. On an empty
+    // cache, `quak get` passes a file that `freshFiles()` returned.
+    async originalOf(
+        file: EnteFile,
+        opts?: ContentOptions,
+    ): Promise<ContentResult> {
+        if (!this.cache) {
+            throw new Error(
+                "originalOf requires a library opened with a content cache",
+            );
+        }
+        return this.cache.originalOf(file, opts);
+    }
+
+    // The thumbnail of `file`, fetched and cached as `Photo.thumbnail()` does.
+    // `file` is used as given, so the store need not hold it. On an empty
+    // cache, `quak get-thumb` passes a file that `freshFiles()` returned.
+    async thumbnailOf(
+        file: EnteFile,
+        opts?: ContentOptions,
+    ): Promise<ContentResult> {
+        if (!this.cache) {
+            throw new Error(
+                "thumbnailOf requires a library opened with a content cache",
+            );
+        }
+        return this.cache.thumbnailOf(file, opts);
+    }
+
     // Back up every in-scope file to `opts.downloadDirectory`, or else the
     // library's, each original at its save path, with a durable failure
     // ledger (issue #51). Takes the lock in that directory first, unless the

src/library/mldata.ts

@@ -61,7 +61,7 @@ export interface MLIndex {
 // Pull the CLIP embedding out of a payload, or undefined when it is absent or
 // misshapen. Kept strict so a bad payload is skipped rather than corrupting the
 // packed index.
-const clipEmbedding = (payload: MLData): number[] | undefined => {
+export const clipEmbedding = (payload: MLData): number[] | undefined => {
     const clip = payload.clip;
     if (typeof clip !== "object" || clip === null) return undefined;
     const embedding = (clip as { embedding?: unknown }).embedding;

src/metadata-backup.ts

@@ -1,6 +1,7 @@
 import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
 import { join } from "node:path";
 import * as jpeg from "jpeg-js";
+import { clipKeywords } from "./clip-keywords.js";
 import type { Client } from "./client.js";
 import { describeError } from "./errors.js";
 import { readExifTags } from "./exif.js";
@@ -97,7 +98,8 @@ const extractExif = async (
 
 // Dump every decrypted metadata layer the account holds into a directory tree
 // of plain JSON: account, per-collection, and per-file records including the
-// private and public magic metadata and (by default) the ML data. Collections
+// private and public magic metadata and the ML data, with the keywords of its
+// CLIP embedding in place of the embedding (see clip-keywords.ts). Collections
 // and files are enumerated from the library's cache, which the caller refreshes
 // first. Returns how many files have no ML data because their ML data request
 // failed or their ML data did not decrypt, gunzip or parse; they are still
@@ -230,8 +232,16 @@ export const runMetadataBackup = async (
         if (file.pubMagicMetadata)
             fileMeta.pubMagicMetadata = file.pubMagicMetadata;
 
+        // The ML data without the CLIP embedding, and the keywords that
+        // embedding gives.
         const ml = mlDataMap.get(file.id);
-        if (ml) fileMeta.mlData = ml;
+        if (ml) {
+            const { clip: _clip, ...mlDataWithoutClip } = ml;
+            fileMeta.mlData = mlDataWithoutClip;
+            const isVideo = file.metadata.fileType === "video";
+            const keywords = clipKeywords(ml, isVideo);
+            if (keywords) fileMeta.keywords = keywords;
+        }
         const mlError = mlDataErrors.get(file.id);
         if (mlError) fileMeta.mlDataError = mlError;
 

src/thumbnails.ts

@@ -5,7 +5,7 @@ import type { Client } from "./client.js";
 import type { Library } from "./library/index.js";
 import { ApiError } from "./api/client.js";
 import { encryptBlob, toBase64 } from "./crypto/index.js";
-import type { EnteFile } from "./model/types.js";
+import type { Collection, EnteFile } from "./model/types.js";
 
 // The server refuses a thumbnail larger than the one it already records for the
 // file (`thumbnail.size`, the encrypted size), so these encodings are tried
@@ -226,16 +226,18 @@ const encryptThumbnailWithin = (
     return undefined;
 };
 
-// Regenerate and upload a thumbnail for each requested file. Originals are read
-// through the library's content cache (`photo.original()`); the generated
-// thumbnail is JPEG-encoded, encrypted under the file's own key, and registered
-// with the server — the encrypt-and-upload path is unchanged. Each file is
+// Regenerate and upload a thumbnail for each requested file. `found` holds the
+// record of each requested file that a collection holds, and the collection it
+// was found in. Each original is read through the library's content cache, by
+// `lib.originalOf(file)`. The new thumbnail is a JPEG, encrypted under the
+// file's own key. It is uploaded, then registered with the server. Each file is
 // resolved to one outcome (fixed / skipped / failed) and a failure on one file
 // never stops the others.
 export const fixMissingThumbnails = async (
     lib: Library,
     client: Client,
     fileIDs: number[],
+    found: Map<number, { file: EnteFile; collection: Collection }>,
     onProgress?: ProgressCallback,
 ): Promise<ThumbnailFixResult[]> => {
     const log = onProgress ?? (() => {});
@@ -243,31 +245,8 @@ export const fixMissingThumbnails = async (
     const api = client.getApiClient();
     const { userID } = client.whoami();
 
-    // Resolve each requested fileID to its file record and owning album by
-    // enumerating the library, each file taken from the first album that holds
-    // it. The raw `EnteFile` carries the per-file key the thumbnail is
-    // encrypted under, which the projected records deliberately do not.
-    const wanted = new Set(fileIDs);
-    const fileMap = new Map<
-        number,
-        { file: EnteFile; collectionName: string }
-    >();
-    for (const album of lib.albums.list()) {
-        for (const photo of album.photos.list()) {
-            if (!wanted.has(photo.fileID) || fileMap.has(photo.fileID))
-                continue;
-            const file = lib.getFile(album.collectionID, photo.fileID);
-            if (file) {
-                fileMap.set(photo.fileID, {
-                    file,
-                    collectionName: album.name,
-                });
-            }
-        }
-    }
-
     for (const fileID of fileIDs) {
-        const entry = fileMap.get(fileID);
+        const entry = found.get(fileID);
         if (!entry) {
             results.push({
                 fileID,
@@ -279,7 +258,8 @@ export const fixMissingThumbnails = async (
             continue;
         }
 
-        const { file, collectionName } = entry;
+        const { file } = entry;
+        const collectionName = entry.collection.name;
         const title = file.metadata.title;
 
         const skipReason = reasonToSkip(file, userID);
@@ -297,15 +277,10 @@ export const fixMissingThumbnails = async (
         const maxSize = file.thumbnail.size!;
 
         try {
-            const photo = lib.photos.byID({ fileID });
-            if (!photo) {
-                throw new Error("file not present in the library cache");
-            }
-
             log(
                 `[${collectionName}] Downloading ${title} for thumbnail generation...`,
             );
-            const { path } = await photo.original();
+            const { path } = await lib.originalOf(file);
             const fileBytes = new Uint8Array(readFileSync(path));
 
             if (!isJpeg(fileBytes)) {
@@ -350,7 +325,7 @@ export const fixMissingThumbnails = async (
                 ciphertext.length,
                 md5,
             );
-            await api.putFile(url, ciphertext);
+            await api.putFile(url, ciphertext, md5);
             await api.updateThumbnail(file.id, objectKey, toBase64(header));
 
             log(`[${collectionName}] Thumbnail uploaded for ${title}`);

test/api/client.test.ts

@@ -780,7 +780,7 @@ describe("ApiClient timeouts", () => {
 
         await client.getJSON("/a");
         await client.postJSON("/b", {});
-        await client.putFile("https://s3.example/x", new Uint8Array([1]));
+        await client.putFile("https://s3.example/x", new Uint8Array([1]), "m");
         await client.putJSON("/c", {});
 
         for (const call of calls) {
@@ -825,7 +825,8 @@ describe("ApiClient timeouts", () => {
             [
                 "putFile",
                 () => new Response(null, { status: 200 }),
-                (c) => c.putFile("https://s3.example/x", new Uint8Array([1])),
+                (c) =>
+                    c.putFile("https://s3.example/x", new Uint8Array([1]), "m"),
             ],
             [
                 "getFileStream",
@@ -1049,7 +1050,7 @@ describe("ApiClient error typing", () => {
         const client = new ApiClient({ fetch, retry: noWait });
 
         const err: unknown = await client
-            .putFile("https://s3.example/obj", new Uint8Array([1, 2]))
+            .putFile("https://s3.example/obj", new Uint8Array([1, 2]), "m")
             .catch((e: unknown) => e);
 
         expect(err).toBeInstanceOf(ApiError);
@@ -1067,7 +1068,11 @@ describe("ApiClient error typing", () => {
         );
         const client = new ApiClient({ fetch, retry: noWait });
 
-        await client.putFile("https://s3.example/obj", new Uint8Array([1, 2]));
+        await client.putFile(
+            "https://s3.example/obj",
+            new Uint8Array([1, 2]),
+            "m",
+        );
         expect(calls).toHaveLength(2);
     });
 
@@ -1277,6 +1282,7 @@ describe("ApiClient onRequest", () => {
         await client.putFile(
             "https://s3.example/obj?X-Amz-Signature=abc",
             new Uint8Array([1, 2]),
+            "m",
         );
 
         expect(requests).toEqual([

test/api/upload.test.ts

@@ -80,13 +80,20 @@ describe("ApiClient.putFile", () => {
         const client = new ApiClient({ fetch, authToken: "secret-tok" });
         const data = new Uint8Array([1, 2, 3, 4, 5]);
 
-        await client.putFile("https://s3.example.com/presigned?sig=abc", data);
+        await client.putFile(
+            "https://s3.example.com/presigned?sig=abc",
+            data,
+            "md5base64==",
+        );
 
         expect(calls[0]!.url).toBe("https://s3.example.com/presigned?sig=abc");
         expect(calls[0]!.init?.method).toBe("PUT");
         const headers = new Headers(calls[0]!.init?.headers as HeadersInit);
         expect(headers.get("Content-Type")).toBe("application/octet-stream");
         expect(headers.get("Content-Length")).toBe("5");
+        // The server signs the MD5 into the URL; storage refuses the PUT
+        // without it.
+        expect(headers.get("Content-MD5")).toBe("md5base64==");
         // Must NOT leak auth headers to S3
         expect(headers.has("X-Auth-Token")).toBe(false);
         expect(headers.has("X-Client-Package")).toBe(false);
@@ -99,7 +106,11 @@ describe("ApiClient.putFile", () => {
         const client = new ApiClient({ fetch });
 
         await expect(
-            client.putFile("https://s3.example.com/bad", new Uint8Array(10)),
+            client.putFile(
+                "https://s3.example.com/bad",
+                new Uint8Array(10),
+                "md5base64==",
+            ),
         ).rejects.toThrow(/403/);
     });
 });

test/cli/backup-end-to-end.test.ts

@@ -86,10 +86,17 @@ const DAY = "2024/2024-03/2024-03-01";
 const TOAST = new TextEncoder().encode("a few seconds of the wedding toast");
 
 // Ente's ML data for beach.jpg: a face it found, and the photo's CLIP
-// embedding.
+// embedding, here that of a photo of a sunset (see clip-keywords.test.ts).
 const ML_DATA = {
     face: { faces: [{ faceID: "100_1", score: 0.98 }] },
-    clip: { embedding: [0.5, 0.6, 0.7] },
+    clip: {
+        embedding: JSON.parse(
+            readFileSync(
+                new URL("../clip-keywords-sunset.json", import.meta.url),
+                "utf-8",
+            ),
+        ).embedding,
+    },
 };
 
 // Each file's ID, its owner, its original as uploaded, and the metadata its
@@ -411,7 +418,7 @@ describe("the first backup of the account", () => {
         expect(downloads.sort()).toEqual([100, 200, 300]);
     });
 
-    // The tree "Backup layout" in the README describes. Each original is at
+    // The tree under "The backup directory" in the README. Each original is at
     // its save path, `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>`, with
     // the extension of its title; the live photo is its image and its video,
     // each with the extension it had inside the ZIP, and a `.livephoto.json`
@@ -486,7 +493,10 @@ describe("the first backup of the account", () => {
             },
             updationTime: TAKEN,
         });
-        expect(beach.mlData).toEqual(ML_DATA);
+        // The JSON holds the ML data without its CLIP embedding, and the
+        // keywords the embedding gives in its place.
+        expect(beach.mlData).toEqual({ face: ML_DATA.face });
+        expect(beach.keywords).toEqual(["Sunset"]);
         expect(beach.imageMetadata.exif.Make.description).toBe("Canon");
         expect(beach.imageMetadata.exif.Model.description).toBe("EOS R5");
 
@@ -497,6 +507,7 @@ describe("the first backup of the account", () => {
             metadata: { title: "toast.mp4", fileType: "video" },
         });
         expect(toast).not.toHaveProperty("mlData");
+        expect(toast).not.toHaveProperty("keywords");
         expect(toast).not.toHaveProperty("imageMetadata");
         expect(toast).not.toHaveProperty("imageMetadataError");
 

test/cli/backup.test.ts

@@ -52,6 +52,7 @@ import { tmpdir } from "node:os";
 import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
 
 import { runBackup, type BackupLibrary } from "../../src/backup.js";
+import { DISCOVER_CATEGORIES } from "../../src/clip-keyword-vectors.js";
 import { Library } from "../../src/library/index.js";
 import type { ContentSource } from "../../src/library/content.js";
 import type { CollectionsPage, FilesPage } from "../../src/client.js";
@@ -946,9 +947,19 @@ describe("the backup lock", () => {
     });
 });
 
-// Also serves ML data: file 100 has `ML_PAYLOAD`, the other two have none.
-// While `mlError` is set, every ML data request fails with it.
-const ML_PAYLOAD = { face: { faces: [] }, clip: { embedding: [0.5, 0.25] } };
+// Also serves ML data: file 100 has `ML_PAYLOAD`, whose CLIP embedding is that
+// of a photo of a sunset (see clip-keywords.test.ts), and the other two have
+// none. While `mlError` is set, every ML data request fails with it.
+const SUNSET_EMBEDDING: number[] = JSON.parse(
+    readFileSync(
+        new URL("../clip-keywords-sunset.json", import.meta.url),
+        "utf-8",
+    ),
+).embedding;
+const ML_PAYLOAD = {
+    face: { faces: [] },
+    clip: { embedding: SUNSET_EMBEDDING },
+};
 
 class MLClient extends MockClient {
     mlError?: string;
@@ -968,16 +979,99 @@ const fileJSON = (outDir: string, fileID: number): Record<string, unknown> =>
     JSON.parse(readFileSync(saved(outDir, `${fileID}.json`), "utf-8"));
 
 describe("ML data in each file's JSON", () => {
-    it("writes a file's ML data, and no ML field for a file that has none", async () => {
+    // The JSON holds the ML data without its CLIP embedding, and in its place
+    // the keywords the embedding gives.
+    it("writes a file's ML data and keywords, and no ML field for a file that has none", async () => {
         const lib = await openLibrary(stubSource(), new MLClient());
         const outDir = join(root, "backup");
 
         const result = await lib.backup({ downloadDirectory: outDir });
 
         expect(result.failed).toBe(0);
-        expect(fileJSON(outDir, 100).mlData).toEqual(ML_PAYLOAD);
+        expect(fileJSON(outDir, 100).mlData).toEqual({ face: { faces: [] } });
+        expect(fileJSON(outDir, 100).keywords).toEqual(["Sunset"]);
         expect(fileJSON(outDir, 101)).not.toHaveProperty("mlData");
         expect(fileJSON(outDir, 101)).not.toHaveProperty("mlDataError");
+        expect(fileJSON(outDir, 101)).not.toHaveProperty("keywords");
+        await lib.close();
+    });
+
+    // A file's ML data can hold faces and no CLIP embedding. Its JSON then has
+    // the ML data and no keywords.
+    it("writes no keywords for ML data without a CLIP embedding", async () => {
+        const faces = { face: { faces: [{ faceID: "101_1", score: 0.9 }] } };
+
+        // Gives file 101 ML data holding only faces.
+        class FacesClient extends MockClient {
+            async fetchMLData(args: { fileIDs: number[] }): Promise<{
+                payloads: Map<number, MLData>;
+                errors: Map<number, unknown>;
+            }> {
+                const payloads = new Map<number, MLData>();
+                if (args.fileIDs.includes(101)) payloads.set(101, faces);
+                return { payloads, errors: new Map() };
+            }
+        }
+        const lib = await openLibrary(stubSource(), new FacesClient());
+        const outDir = join(root, "backup");
+
+        const result = await lib.backup({ downloadDirectory: outDir });
+
+        expect(result.failed).toBe(0);
+        expect(fileJSON(outDir, 101).mlData).toEqual(faces);
+        expect(fileJSON(outDir, 101)).not.toHaveProperty("keywords");
+        await lib.close();
+    });
+
+    // Ente's photos app keeps videos out of some Discover categories, Identity
+    // among them. Here a photo and a video share one CLIP embedding, Identity's
+    // own text embedding, so the photo gets the keyword Identity and the video
+    // does not.
+    it("keeps a video out of a category that leaves out videos", async () => {
+        const identity = DISCOVER_CATEGORIES.find(
+            (c) => c.title === "Identity",
+        )!;
+        expect(identity.showVideo).toBe(false);
+        const toast = file(102, 1, "toast.mp4");
+        const video = {
+            ...toast,
+            metadata: { ...toast.metadata, fileType: "video" as const },
+        };
+
+        // Serves beach.jpg and the video in the Vacation album, and gives
+        // each ML data holding only that embedding.
+        class VideoClient extends MockClient {
+            override async filesSince(args: {
+                collectionID: number;
+            }): Promise<FilesPage> {
+                const files =
+                    args.collectionID === 1
+                        ? [file(100, 1, "beach.jpg"), video]
+                        : [];
+                return { files, deleted: [], cursor: 1 };
+            }
+            async fetchMLData(args: { fileIDs: number[] }): Promise<{
+                payloads: Map<number, MLData>;
+                errors: Map<number, unknown>;
+            }> {
+                const payloads = new Map<number, MLData>();
+                for (const id of args.fileIDs) {
+                    payloads.set(id, {
+                        clip: { embedding: identity.textEmbedding },
+                    });
+                }
+                return { payloads, errors: new Map() };
+            }
+        }
+        const lib = await openLibrary(stubSource(), new VideoClient());
+        const outDir = join(root, "backup");
+
+        const result = await lib.backup({ downloadDirectory: outDir });
+
+        expect(result.failed).toBe(0);
+        expect(fileJSON(outDir, 100).keywords).toContain("Identity");
+        expect(fileJSON(outDir, 102)).toHaveProperty("keywords");
+        expect(fileJSON(outDir, 102).keywords).not.toContain("Identity");
         await lib.close();
     });
 
@@ -993,7 +1087,7 @@ describe("ML data in each file's JSON", () => {
         const result = await lib.backup({ downloadDirectory: outDir });
 
         // File 100's ML data was cached by the first run and is kept.
-        expect(fileJSON(outDir, 100).mlData).toEqual(ML_PAYLOAD);
+        expect(fileJSON(outDir, 100).mlData).toEqual({ face: { faces: [] } });
         expect(fileJSON(outDir, 100)).not.toHaveProperty("mlDataError");
         for (const fileID of [101, 200]) {
             expect(fileJSON(outDir, fileID).mlDataError).toBe(
@@ -1026,7 +1120,7 @@ describe("ML data in each file's JSON", () => {
 
         expect(result.failed).toBe(0);
         expect(result.skipped).toBe(3);
-        expect(fileJSON(outDir, 100).mlData).toEqual(ML_PAYLOAD);
+        expect(fileJSON(outDir, 100).mlData).toEqual({ face: { faces: [] } });
         for (const fileID of [100, 101, 200]) {
             expect(fileJSON(outDir, fileID)).not.toHaveProperty("mlDataError");
         }

test/cli/bin.test.ts

@@ -1,12 +1,19 @@
 /**
  * Tests for the retry options `bin/quak.ts` loads the saved session with
  * (`quak backup` gets the unattended ones, every other command the default),
- * each with the callbacks that log its requests and retries, and for
- * `quak files` taking its collection ID as an optional argument.
+ * each with the callbacks that log its requests and retries, for `quak files`
+ * taking its collection ID as an optional argument, for `quak get` and
+ * `quak get-thumb` refusing `--collection`, and for `quak login` taking
+ * `--server`.
  *
  * Each test runs `bin/quak.ts`, as the smoke test does, with its session loader
  * replaced by one that records the options it is given and reports no session,
  * so the command stops before it makes any request.
+ *
+ * The first load of the sources `bin/quak.ts` imports compiles them, and the
+ * first `init()` starts libsodium. On a loaded host that takes longer than a
+ * test's timeout, so both happen here, before the tests run. Each test's
+ * `vi.resetModules()` then only runs the compiled sources again.
  */
 
 import { mkdtempSync, rmSync } from "node:fs";
@@ -14,8 +21,13 @@ import { tmpdir } from "node:os";
 import { join } from "node:path";
 import { afterAll, afterEach, describe, expect, it, vi } from "vitest";
 import type { ApiClientOptions } from "../../src/api/client.js";
+import "../../src/cli-commands.js";
+import { init } from "../../src/crypto/index.js";
+import "../../src/index.js";
 import { UNATTENDED_RETRY_OPTIONS } from "../../src/retry.js";
 
+await init();
+
 const loaded = vi.hoisted(() => [] as (ApiClientOptions | undefined)[]);
 
 vi.mock("../../src/cli-session.js", () => ({
@@ -95,3 +107,24 @@ describe("bin/quak.ts files", () => {
         expect(await quak("files", "7")).toMatch(/^Not logged in/);
     });
 });
+
+describe("bin/quak.ts get and get-thumb", () => {
+    it.each(["get", "get-thumb"])(
+        "%s refuses --collection as an unknown option",
+        async (command) => {
+            expect(await quak(command, "1", "--collection", "2")).toMatch(
+                /^error: unknown option '--collection'/,
+            );
+        },
+    );
+});
+
+// The command refuses an address that is not a URL before it asks for anything
+// or sends any request.
+describe("bin/quak.ts login", () => {
+    it("hands --server to the command", async () => {
+        expect(await quak("login", "--server", "ente.example.com")).toMatch(
+            /^Invalid server address "ente\.example\.com"/,
+        );
+    });
+});

test/cli/commands.test.ts

@@ -194,9 +194,6 @@ const FETCHED_BOTH_ALBUMS =
     "Fetched the files of 1 of 2 collections\n" +
     "Fetched the files of 2 of 2 collections\n";
 
-// The same for an account with one album.
-const FETCHED_ONE_ALBUM = "Fetched the files of 1 of 1 collections\n";
-
 // Collects everything written to it.
 class Output {
     text = "";
@@ -331,6 +328,7 @@ describe("login", () => {
         prompts = [];
         vi.stubEnv("QUAK_EMAIL", "cli@example.com");
         vi.stubEnv("QUAK_PASSWORD", "hunter2");
+        vi.stubEnv("QUAK_SERVER", undefined);
     });
 
     afterEach(() => {
@@ -343,12 +341,14 @@ describe("login", () => {
             calls.push(opts);
             return loggedIn;
         });
-        expect(await loginCommand(ctx)).toBe(0);
+        expect(await loginCommand(ctx, {})).toBe(0);
 
         expect(prompts).toEqual([]);
         expect(calls).toHaveLength(1);
         expect(calls[0]!.email).toBe("cli@example.com");
         expect(calls[0]!.password).toBe("hunter2");
+        // No server named: the client uses Ente's hosted service.
+        expect(calls[0]!.apiOptions).toBeUndefined();
         const path = join(ctx.sessionDir, "session.json");
         expect(stderr.text).toBe(
             "Authenticating...\n" +
@@ -359,7 +359,7 @@ describe("login", () => {
 
     it("saves the session with mode 0600 in a directory with mode 0700", async () => {
         const ctx = loginContext(async () => loggedIn);
-        expect(await loginCommand(ctx)).toBe(0);
+        expect(await loginCommand(ctx, {})).toBe(0);
 
         expect(statSync(ctx.sessionDir).mode & 0o777).toBe(0o700);
         const path = join(ctx.sessionDir, "session.json");
@@ -373,7 +373,7 @@ describe("login", () => {
             code = await opts.totp!();
             return loggedIn;
         });
-        expect(await loginCommand(ctx)).toBe(0);
+        expect(await loginCommand(ctx, {})).toBe(0);
 
         expect(prompts).toEqual(["TOTP code: "]);
         expect(code).toBe("123456");
@@ -383,13 +383,68 @@ describe("login", () => {
         const ctx = loginContext(async () => {
             throw new Error("HTTP 401 from server");
         });
-        expect(await loginCommand(ctx)).toBe(1);
+        expect(await loginCommand(ctx, {})).toBe(1);
 
         expect(stderr.text).toBe(
             "Authenticating...\nLogin failed: HTTP 401 from server\n",
         );
         expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
     });
+
+    it("logs in to the server --server names, ahead of QUAK_SERVER", async () => {
+        vi.stubEnv("QUAK_SERVER", "https://other.example.com");
+        const calls: LoginOptions[] = [];
+        const ctx = loginContext(async (opts) => {
+            calls.push(opts);
+            return loggedIn;
+        });
+        expect(
+            await loginCommand(ctx, { server: "https://ente.example.com" }),
+        ).toBe(0);
+
+        expect(calls[0]!.apiOptions).toEqual({
+            apiOrigin: "https://ente.example.com",
+        });
+    });
+
+    it("logs in to the server QUAK_SERVER names without --server", async () => {
+        vi.stubEnv("QUAK_SERVER", "http://192.0.2.1:8080/ente");
+        const calls: LoginOptions[] = [];
+        const ctx = loginContext(async (opts) => {
+            calls.push(opts);
+            return loggedIn;
+        });
+        expect(await loginCommand(ctx, {})).toBe(0);
+
+        expect(calls[0]!.apiOptions).toEqual({
+            apiOrigin: "http://192.0.2.1:8080/ente",
+        });
+    });
+
+    it.each([
+        "",
+        "ente.example.com",
+        "ftp://ente.example.com",
+        "https://ente.example.com/?region=eu",
+        "https://ente.example.com/#top",
+    ])(
+        "refuses the server address %j before asking for anything",
+        async (server) => {
+            const ctx = loginContext(async () => {
+                throw new Error("login must not be called");
+            });
+            expect(await loginCommand(ctx, { server })).toBe(1);
+
+            expect(prompts).toEqual([]);
+            expect(stderr.text).toBe(
+                `Invalid server address "${server}": it must be an http or ` +
+                    `https URL with no query or fragment\n`,
+            );
+            expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(
+                false,
+            );
+        },
+    );
 });
 
 // These use a real client read from the session file, over a fake API that
@@ -535,9 +590,7 @@ describe("files", () => {
 
     it("exits 1 for an unknown collection", async () => {
         expect(await filesCommand(context(), "9", {})).toBe(1);
-        expect(stderr.text).toBe(
-            FETCHED_BOTH_ALBUMS + "Collection 9 not found\n",
-        );
+        expect(stderr.text).toBe("Collection 9 not found\n");
     });
 
     // Without a collection ID it lists every file in the account once (issue
@@ -574,31 +627,31 @@ describe("files", () => {
 });
 
 describe("get and get-thumb", () => {
-    it("get finds a file in any album without --collection", async () => {
+    it("get finds a file in any album", async () => {
         const out = join(root, "diagram.png");
         expect(await getCommand(context(), "200", { out })).toBe(0);
         expect(readFileSync(out)).toEqual(Buffer.alloc(7, 200));
-        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + `7 bytes -> ${out}\n`);
+        expect(stderr.text).toBe(`7 bytes -> ${out}\n`);
     });
 
-    it("get-thumb finds a file in any album without --collection", async () => {
+    it("get-thumb finds a file in any album", async () => {
         const out = join(root, "thumb.jpg");
         expect(await getThumbCommand(context(), "200", { out })).toBe(0);
         expect(readFileSync(out)).toEqual(Buffer.alloc(3, 200));
-        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + `3 bytes -> ${out}\n`);
+        expect(stderr.text).toBe(`3 bytes -> ${out}\n`);
     });
 
     it("get exits 1 when no album has the file", async () => {
         const out = join(root, "x");
         expect(await getCommand(context(), "999", { out })).toBe(1);
-        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + "File 999 not found\n");
+        expect(stderr.text).toBe("File 999 not found\n");
         expect(existsSync(out)).toBe(false);
     });
 
     it("get-thumb exits 1 when no album has the file", async () => {
         const out = join(root, "x");
         expect(await getThumbCommand(context(), "999", { out })).toBe(1);
-        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + "File 999 not found\n");
+        expect(stderr.text).toBe("File 999 not found\n");
         expect(existsSync(out)).toBe(false);
     });
 });
@@ -646,7 +699,7 @@ describe("names from an album name or a title over 255 bytes", () => {
         expect(Buffer.byteLength(name)).toBe(255);
         expect(code).toBe(0);
         expect(readdirSync(dir)).toEqual([name]);
-        expect(stderr.text).toBe(FETCHED_ONE_ALBUM + `7 bytes -> ${name}\n`);
+        expect(stderr.text).toBe(`7 bytes -> ${name}\n`);
     });
 
     it("get-thumb cuts the title short so that thumb_ and it fit in 255 bytes, and exits 0", async () => {
@@ -658,7 +711,7 @@ describe("names from an album name or a title over 255 bytes", () => {
         expect(Buffer.byteLength(name)).toBe(255);
         expect(code).toBe(0);
         expect(readdirSync(dir)).toEqual([name]);
-        expect(stderr.text).toBe(FETCHED_ONE_ALBUM + `3 bytes -> ${name}\n`);
+        expect(stderr.text).toBe(`3 bytes -> ${name}\n`);
     });
 
     it("backup-metadata cuts the album name short so that its folder's name fits in 255 bytes, and exits 0", async () => {
@@ -747,7 +800,6 @@ describe("a live photo", () => {
             Buffer.from(VIDEO),
         );
         expect(stderr.text).toBe(
-            FETCHED_ONE_ALBUM +
             `${IMAGE.length} bytes -> IMG_0300.heic\n` +
                 `${VIDEO.length} bytes -> IMG_0300.mov\n`,
         );
@@ -802,7 +854,6 @@ describe("a live photo", () => {
         expect(readFileSync(out)).toEqual(Buffer.from(IMAGE));
         expect(readFileSync(video)).toEqual(Buffer.from(VIDEO));
         expect(stderr.text).toBe(
-            FETCHED_ONE_ALBUM +
             `${IMAGE.length} bytes -> ${out}\n${VIDEO.length} bytes -> ${video}\n`,
         );
     });
@@ -817,7 +868,6 @@ describe("a live photo", () => {
         expect(existsSync(out)).toBe(false);
         expect(existsSync(join(root, "photo.mov"))).toBe(false);
         expect(stderr.text).toBe(
-            FETCHED_ONE_ALBUM +
             `File 300 is a live photo, and its video would also be written to ${out}\n`,
         );
     });
@@ -1361,6 +1411,23 @@ describe("helper fix-missing-thumbnails", () => {
         ).toBe(1);
         expect(stderr.text).toBe(last);
     });
+
+    // beach.jpg (100) is in Vacation (1) and Work (2), both updated at the same
+    // time. The fixer names a file after the first album holding it in the
+    // order of `Library.albums.list()`. On a tie, the album with the higher ID
+    // comes first, so beach.jpg is named after Work.
+    it("names a photo in two albums after the same album on an empty cache as on a cache that holds the account", async () => {
+        const ctx = context(fakeClient({ beachInWork: true }));
+        const opts = { file: ["100"], json: true };
+        const inWork = [{ fileID: 100, collection: "Work", status: "skipped" }];
+        expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0);
+        expect(JSON.parse(stdout.text)).toMatchObject(inWork);
+
+        expect(await filesCommand(ctx, undefined, {})).toBe(0);
+        stdout.text = "";
+        expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0);
+        expect(JSON.parse(stdout.text)).toMatchObject(inWork);
+    });
 });
 
 describe("backup-metadata --exif", () => {
@@ -1390,11 +1457,11 @@ describe("backup-metadata --exif", () => {
 });
 
 // Each test first runs `files` so the cache holds the account as it was, then
-// changes the server under it. (`collections` would not do: it reads the
-// collection list alone and stores nothing.)
+// changes the server under it. (`collections` and `files` with a collection ID
+// would not do: they store nothing.)
 describe("backup-metadata and the thumbnail helpers refresh first", () => {
     beforeEach(async () => {
-        expect(await filesCommand(context(), "1", {})).toBe(0);
+        expect(await filesCommand(context(), undefined, {})).toBe(0);
         stdout.text = "";
         stderr.text = "";
     });
@@ -1577,20 +1644,83 @@ describe("an ID given on the command line", () => {
     );
 });
 
-// An account with many albums (issue #210). On an empty cache `collections`
-// used to wait for a refresh that fetched and decrypted the files of every
-// album, one album after another, before it printed anything, so on such an
-// account it seemed to hang. The account here has 40 albums of one photo each,
-// encrypted as Ente's clients encrypt them, on the fake Ente server of
-// `test/ente-server.ts`, which also records the path of each request quak
-// sends. The session is the one logging in to that server saves.
+// An account with many albums (issues #210, #217 and #222). On an empty cache
+// `collections`, `files` with a collection ID and `get` used to wait for a
+// refresh that fetched and decrypted the files of every album, one album after
+// another. So did `helper fix-missing-thumbnails --file` (issue #223). On such
+// an account they seemed to hang. The account here has 40 albums of one photo
+// each, encrypted as Ente's clients encrypt them, on the fake Ente server of
+// `test/ente-server.ts`. The tests record the path and query of each request
+// quak sends. The session is the one logging in to that server saves.
 describe("an account with many albums", () => {
     const ALBUMS = 40;
     const PASSWORD = "many-albums";
-    const paths: string[] = [];
+    const requests: string[] = [];
     let server: typeof globalThis.fetch;
     let snapshot: ClientSnapshot;
 
+    // What `files` without a collection ID prints on an empty cache: every
+    // album's photo on stdout, and on stderr each album whose files its
+    // refresh fetched.
+    let everyPhoto = "";
+    let progress = "";
+    for (let n = 1; n <= ALBUMS; n++) {
+        everyPhoto += `${n * 100}\timage\tphoto-${n}.jpg\n`;
+        progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`;
+    }
+
+    // The key of each album, by album ID.
+    const albumKeys = new Map<number, Uint8Array>();
+
+    // A photo in album `collectionID` as the server sends it, and its original
+    // encrypted as the server stores it. The original's bytes are the title.
+    // These tests fetch no thumbnail. The server records a thumbnail size, so
+    // `helper fix-missing-thumbnails` downloads the original before it skips
+    // the photo as not a JPEG.
+    const photoRow = (photo: {
+        id: number;
+        collectionID: number;
+        title: string;
+        updationTime: number;
+    }): { row: Record<string, unknown>; original: Uint8Array } => {
+        const photoKey = sodium.crypto_secretstream_xchacha20poly1305_keygen();
+        const photoKeyBox = encryptSecretbox(
+            photoKey,
+            albumKeys.get(photo.collectionID)!,
+        );
+        const metadata = encryptStreamBlob(
+            new TextEncoder().encode(
+                JSON.stringify({
+                    title: photo.title,
+                    fileType: 0,
+                    creationTime: 1000,
+                    modificationTime: 1000,
+                }),
+            ),
+            photoKey,
+        );
+        const original = encryptStreamBlob(
+            new TextEncoder().encode(photo.title),
+            photoKey,
+        );
+        const row = {
+            id: photo.id,
+            collectionID: photo.collectionID,
+            ownerID: USER_ID,
+            encryptedKey: toBase64(photoKeyBox.ciphertext),
+            keyDecryptionNonce: toBase64(photoKeyBox.nonce),
+            metadata: {
+                encryptedData: toBase64(metadata.ciphertext),
+                decryptionHeader: toBase64(metadata.header),
+            },
+            file: { decryptionHeader: toBase64(original.header) },
+            thumbnail: { decryptionHeader: "" },
+            info: { thumbSize: 1000 },
+            updationTime: photo.updationTime,
+        };
+        return { row, original: original.ciphertext };
+    };
+
     // Album n is named `Album n` and holds one photo, n * 100, titled
     // `photo-n.jpg`.
     beforeAll(async () => {
@@ -1598,8 +1728,10 @@ describe("an account with many albums", () => {
         const account = await buildAccount(PASSWORD);
         const collections: Record<string, unknown>[] = [];
         const filesByCollection: Record<number, Record<string, unknown>[]> = {};
+        const fileCiphertexts: Record<number, Uint8Array> = {};
         for (let id = 1; id <= ALBUMS; id++) {
             const albumKey = sodium.crypto_secretbox_keygen();
+            albumKeys.set(id, albumKey);
             const albumKeyBox = encryptSecretbox(albumKey, account.masterKey);
             const name = encryptSecretbox(
                 new TextEncoder().encode(`Album ${id}`),
@@ -1615,37 +1747,14 @@ describe("an account with many albums", () => {
                 type: "album",
                 updationTime: 1,
             });
-            const photoKey =
-                sodium.crypto_secretstream_xchacha20poly1305_keygen();
-            const photoKeyBox = encryptSecretbox(photoKey, albumKey);
-            const metadata = encryptStreamBlob(
-                new TextEncoder().encode(
-                    JSON.stringify({
-                        title: `photo-${id}.jpg`,
-                        fileType: 0,
-                        creationTime: 1000,
-                        modificationTime: 1000,
-                    }),
-                ),
-                photoKey,
-            );
-            filesByCollection[id] = [
-                {
+            const { row, original } = photoRow({
                 id: id * 100,
                 collectionID: id,
-                    ownerID: USER_ID,
-                    encryptedKey: toBase64(photoKeyBox.ciphertext),
-                    keyDecryptionNonce: toBase64(photoKeyBox.nonce),
-                    metadata: {
-                        encryptedData: toBase64(metadata.ciphertext),
-                        decryptionHeader: toBase64(metadata.header),
-                    },
-                    // These tests fetch no original and no thumbnail.
-                    file: { decryptionHeader: "" },
-                    thumbnail: { decryptionHeader: "" },
+                title: `photo-${id}.jpg`,
                 updationTime: 1,
-                },
-            ];
+            });
+            filesByCollection[id] = [row];
+            fileCiphertexts[id * 100] = original;
         }
         const ente = buildMetaFetch({
             verifier: account.verifier,
@@ -1655,7 +1764,7 @@ describe("an account with many albums", () => {
             collections,
             filesByCollection,
             encryptedMLData: {},
-            fileCiphertexts: {},
+            fileCiphertexts,
         });
         server = (async (input: RequestInfo | URL, init?: RequestInit) => {
             const url =
@@ -1664,7 +1773,8 @@ describe("an account with many albums", () => {
                     : input instanceof URL
                       ? input.href
                       : input.url;
-            paths.push(new URL(url).pathname);
+            const { pathname, search } = new URL(url);
+            requests.push(pathname + search);
             return ente(input, init);
         }) as typeof globalThis.fetch;
         const client = await Client.login({
@@ -1676,16 +1786,17 @@ describe("an account with many albums", () => {
     });
 
     beforeEach(() => {
-        paths.length = 0;
+        requests.length = 0;
     });
 
-    // A context whose saved session sends its requests to the fake server.
-    const accountContext = (): CliContext => {
+    // A context whose saved session sends its requests to `fetch`, by default
+    // the fake server.
+    const accountContext = (fetch = server): CliContext => {
         const ctx = context();
         saveSession(ctx.sessionDir, snapshot);
         return {
             ...ctx,
-            loadSession: (path) => loadSession(path, { fetch: server }),
+            loadSession: (path) => loadSession(path, { fetch }),
         };
     };
 
@@ -1698,46 +1809,260 @@ describe("an account with many albums", () => {
             albums += `${id}\talbum\tAlbum ${id}\n`;
         expect(stdout.text).toBe(albums);
         expect(stderr.text).toBe("");
-        expect(paths).toEqual(["/collections/v2"]);
+        expect(requests).toEqual(["/collections/v2?sinceTime=0"]);
 
         // It stored nothing that would make a refresh skip an album's files:
-        // `files` after it still finds the album's photo.
+        // `files` after it still lists every album's photo.
         stdout.text = "";
-        expect(await filesCommand(ctx, "7", {})).toBe(0);
-        expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n");
+        expect(await filesCommand(ctx, undefined, {})).toBe(0);
+        expect(stdout.text).toBe(everyPhoto);
     });
 
-    it("files counts on stderr each album whose files its refresh fetched", async () => {
+    it("files with a collection ID asks for the album list and that album's files alone", async () => {
         expect(await filesCommand(accountContext(), "7", {})).toBe(0);
 
         expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n");
-        let progress = "";
-        for (let n = 1; n <= ALBUMS; n++) {
-            progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`;
+        expect(stderr.text).toBe("");
+        expect(requests).toEqual([
+            "/collections/v2?sinceTime=0",
+            "/collections/v2/diff?collectionID=7&sinceTime=0",
+        ]);
+    });
+
+    // Here the server sends album 7's files in two pages. Photo 700 changed
+    // between them, so both pages hold a copy of it.
+    it("files with a collection ID lists only the later copy of a photo sent in two pages", async () => {
+        const firstPage = [
+            photoRow({
+                id: 700,
+                collectionID: 7,
+                title: "photo-7.jpg",
+                updationTime: 1,
+            }).row,
+            photoRow({
+                id: 701,
+                collectionID: 7,
+                title: "photo-701.jpg",
+                updationTime: 1,
+            }).row,
+        ];
+        const secondPage = [
+            photoRow({
+                id: 700,
+                collectionID: 7,
+                title: "photo-7-edited.jpg",
+                updationTime: 2,
+            }).row,
+        ];
+        const paged = (async (input: RequestInfo | URL, init?: RequestInit) => {
+            const url = new URL(input instanceof Request ? input.url : input);
+            if (
+                url.pathname !== "/collections/v2/diff" ||
+                url.searchParams.get("collectionID") !== "7"
+            ) {
+                return server(input, init);
             }
+            const first = url.searchParams.get("sinceTime") === "0";
+            return new Response(
+                JSON.stringify({
+                    diff: first ? firstPage : secondPage,
+                    hasMore: first,
+                }),
+            );
+        }) as typeof globalThis.fetch;
+
+        expect(await filesCommand(accountContext(paged), "7", {})).toBe(0);
+
+        // Photo 700 is listed once, with the later title, before photo 701.
+        expect(stdout.text).toBe(
+            "700\timage\tphoto-7-edited.jpg\n701\timage\tphoto-701.jpg\n",
+        );
+    });
+
+    // The cache stays empty, so a later `files` without a collection ID still
+    // fetches the files of all 40 albums.
+    it("files with a collection ID stores nothing in the cache", async () => {
+        const ctx = accountContext();
+        expect(await filesCommand(ctx, "7", {})).toBe(0);
+        expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
+
+        stdout.text = "";
+        expect(await filesCommand(ctx, undefined, {})).toBe(0);
+        expect(stdout.text).toBe(everyPhoto);
+        expect(stderr.text).toBe(progress);
+    });
+
+    // Photo 200 is in album 2, so `get` asks for no file of album 3 or later.
+    // The last request downloads the photo's original.
+    it("get on an empty cache asks for the albums' files only until one holds the photo", async () => {
+        const out = join(root, "photo.jpg");
+        expect(await getCommand(accountContext(), "200", { out })).toBe(0);
+
+        expect(readFileSync(out, "utf-8")).toBe("photo-2.jpg");
+        expect(stderr.text).toBe(`11 bytes -> ${out}\n`);
+        expect(requests).toEqual([
+            "/collections/v2?sinceTime=0",
+            "/collections/v2/diff?collectionID=1&sinceTime=0",
+            "/collections/v2/diff?collectionID=2&sinceTime=0",
+            "/?fileID=200",
+        ]);
+        expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
+    });
+
+    // A self-hosted server serves downloads under its own address, at
+    // `/files/download/<id>`, so no request goes to Ente's hosted service.
+    it("login --server saves the server, and a later get sends every request to it", async () => {
+        vi.stubEnv("QUAK_EMAIL", "many@example.com");
+        vi.stubEnv("QUAK_PASSWORD", PASSWORD);
+        vi.stubEnv("QUAK_SERVER", undefined);
+        const hosts = new Set<string>();
+        const fetch = (async (input: RequestInfo | URL, init?: RequestInit) => {
+            hosts.add(
+                new URL(input instanceof Request ? input.url : input).host,
+            );
+            return server(input, init);
+        }) as typeof globalThis.fetch;
+        const ctx: CliContext = {
+            ...context(),
+            login: (opts) =>
+                Client.login({
+                    ...opts,
+                    apiOptions: { ...opts.apiOptions, fetch },
+                }),
+            loadSession: (path) => loadSession(path, { fetch }),
+        };
+        const out = join(root, "photo.jpg");
+        try {
+            const opts = { server: "https://ente.example.com" };
+            expect(await loginCommand(ctx, opts)).toBe(0);
+            requests.length = 0;
+            expect(await getCommand(ctx, "200", { out })).toBe(0);
+        } finally {
+            vi.unstubAllEnvs();
+        }
+
+        const path = join(ctx.sessionDir, "session.json");
+        expect(JSON.parse(readFileSync(path, "utf-8")).apiOrigin).toBe(
+            "https://ente.example.com",
+        );
+        expect(readFileSync(out, "utf-8")).toBe("photo-2.jpg");
+        expect(requests.at(-1)).toBe("/files/download/200");
+        expect([...hosts]).toEqual(["ente.example.com"]);
+    });
+
+    // `files` first fills the cache, so `get` waits for a refresh. No album has
+    // changed since, so the refresh asks for no album's files.
+    it("get on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => {
+        const ctx = accountContext();
+        expect(await filesCommand(ctx, undefined, {})).toBe(0);
+        requests.length = 0;
+        stderr.text = "";
+
+        const out = join(root, "photo.jpg");
+        expect(await getCommand(ctx, "4000", { out })).toBe(0);
+        expect(readFileSync(out, "utf-8")).toBe("photo-40.jpg");
+        expect(stderr.text).toBe(`12 bytes -> ${out}\n`);
+        expect(requests).toEqual([
+            "/collections/v2?sinceTime=1",
+            "/?fileID=4000",
+        ]);
+
+        requests.length = 0;
+        stderr.text = "";
+        expect(await getCommand(ctx, "999", { out })).toBe(1);
+        expect(stderr.text).toBe("File 999 not found\n");
+        expect(requests).toEqual(["/collections/v2?sinceTime=1"]);
+    });
+
+    // The helper searches the most recently updated album first. All 40 were
+    // updated at the same time, and on a tie the higher ID comes first. Photo
+    // 3900 is in album 39, so the helper asks for no file of album 38 or
+    // earlier. The last request downloads the photo's original.
+    it("fix-missing-thumbnails --file on an empty cache asks for the albums' files only until one holds the photo", async () => {
+        const opts = { file: ["3900"], json: true };
+        expect(await fixMissingThumbnailsCommand(accountContext(), opts)).toBe(
+            0,
+        );
+
+        const reason =
+            "unsupported image format (only baseline JPEG can be regenerated)";
+        expect(JSON.parse(stdout.text)).toEqual([
+            {
+                fileID: 3900,
+                title: "photo-39.jpg",
+                collection: "Album 39",
+                status: "skipped",
+                reason,
+            },
+        ]);
+        expect(stderr.text).toBe(
+            "[Album 39] Downloading photo-39.jpg for thumbnail generation...\n" +
+                `[Album 39] Skipping photo-39.jpg: ${reason}\n`,
+        );
+        expect(requests).toEqual([
+            "/collections/v2?sinceTime=0",
+            "/collections/v2/diff?collectionID=40&sinceTime=0",
+            "/collections/v2/diff?collectionID=39&sinceTime=0",
+            "/?fileID=3900",
+        ]);
+        expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
+    });
+
+    // `files` first fills the cache, so the helper waits for a refresh. No
+    // album has changed since, so the refresh asks for no album's files.
+    it("fix-missing-thumbnails --file on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => {
+        const ctx = accountContext();
+        expect(await filesCommand(ctx, undefined, {})).toBe(0);
+        requests.length = 0;
+        stdout.text = "";
+        stderr.text = "";
+
+        const opts = { file: ["3900"], json: true };
+        expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0);
+        const reason =
+            "unsupported image format (only baseline JPEG can be regenerated)";
+        expect(JSON.parse(stdout.text)).toEqual([
+            {
+                fileID: 3900,
+                title: "photo-39.jpg",
+                collection: "Album 39",
+                status: "skipped",
+                reason,
+            },
+        ]);
+        expect(stderr.text).toBe(
+            "[Album 39] Downloading photo-39.jpg for thumbnail generation...\n" +
+                `[Album 39] Skipping photo-39.jpg: ${reason}\n`,
+        );
+        expect(requests).toEqual([
+            "/collections/v2?sinceTime=1",
+            "/?fileID=3900",
+        ]);
+
+        requests.length = 0;
+        stdout.text = "";
+        const notFound = { file: ["999"], json: true };
+        expect(await fixMissingThumbnailsCommand(ctx, notFound)).toBe(1);
+        expect(JSON.parse(stdout.text)).toMatchObject([
+            { fileID: 999, reason: "file not found in any collection" },
+        ]);
+        expect(requests).toEqual(["/collections/v2?sinceTime=1"]);
+    });
+
+    it("files counts on stderr each album whose files its refresh fetched", async () => {
+        expect(await filesCommand(accountContext(), undefined, {})).toBe(0);
+
+        expect(stdout.text).toBe(everyPhoto);
         expect(stderr.text).toBe(progress);
     });
 
     // `--json` shapes stdout only, so the refresh's progress is still counted
     // on stderr.
     it("files --json counts on stderr each album whose files its refresh fetched", async () => {
-        expect(await filesCommand(accountContext(), "7", { json: true })).toBe(
-            0,
-        );
+        const opts = { json: true };
+        expect(await filesCommand(accountContext(), undefined, opts)).toBe(0);
 
-        expect(JSON.parse(stdout.text)).toEqual([
-            {
-                id: 700,
-                title: "photo-7.jpg",
-                fileType: "image",
-                creationTime: 1000,
-                collectionID: 7,
-            },
-        ]);
-        let progress = "";
-        for (let n = 1; n <= ALBUMS; n++) {
-            progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`;
-        }
+        expect(JSON.parse(stdout.text)).toHaveLength(ALBUMS);
         expect(stderr.text).toBe(progress);
     });
 });

test/cli/log.test.ts

@@ -234,13 +234,15 @@ describe("-q", () => {
         });
     });
 
-    // The refresh's progress is left out; the line saying why the command
-    // failed is the fatal one, and it is printed.
+    // No album holds file 999, so fixing it fails. The summary from
+    // `--- Done ---` to the end of the `Failed files:` list is left out. The
+    // line saying why the command failed is the fatal one, and it is printed.
     it("prints only the line a failing command exits with", async () => {
-        expect(await quak("-q", "files", "9")).toEqual({
+        const fix = ["helper", "fix-missing-thumbnails", "--file", "999"];
+        expect(await quak("-q", ...fix)).toEqual({
             code: 1,
             stdout: "",
-            stderr: "Collection 9 not found\n",
+            stderr: `quak: 1 file(s) failed; see the "Failed files:" list, or the JSON with --json\n`,
         });
     });
 
@@ -326,7 +328,7 @@ describe("what the library reports", () => {
         ]);
     });
 
-    // Each operation is named in plain words, as the README names it.
+    // Each operation is named in plain words.
     it("logs a refresh's progress at info, and its start and end at debug", () => {
         const { log, lines } = collect();
         const onProgress = logRefresh(log);

test/cli/metadata-backup.test.ts

@@ -41,6 +41,7 @@ import { beforeAll, afterAll, describe, expect, it, vi } from "vitest";
 import { init, toBase64, encryptBlob } from "../../src/crypto/index.js";
 import * as jpegJs from "jpeg-js";
 import { Client } from "../../src/client.js";
+import { DISCOVER_CATEGORIES } from "../../src/clip-keyword-vectors.js";
 import { Library } from "../../src/library/index.js";
 import {
     runMetadataBackup,
@@ -56,8 +57,8 @@ import {
     type MetaMockState,
 } from "../ente-server.js";
 
-// One file per ML data request, so the two files of the mock account are
-// fetched in two requests and one of them can fail on its own.
+// One file per ML data request, so each file of the mock account is fetched in
+// a request of its own and one of them can fail on its own.
 vi.mock("../../src/mldata-fetch.js", async (importOriginal) => ({
     ...(await importOriginal<typeof import("../../src/mldata-fetch.js")>()),
     MLDATA_BATCH_SIZE: 1,
@@ -66,6 +67,21 @@ vi.mock("../../src/mldata-fetch.js", async (importOriginal) => ({
 const TEST_EMAIL = "metabackup@example.com";
 const TEST_PASSWORD = "metapass";
 
+// The CLIP embedding of a photo of a sunset (see clip-keywords.test.ts), which
+// the ML data of beach.jpg holds.
+const SUNSET_EMBEDDING: number[] = JSON.parse(
+    readFileSync(
+        new URL("../clip-keywords-sunset.json", import.meta.url),
+        "utf-8",
+    ),
+).embedding;
+
+// Identity, a Discover category of Ente's photos app that leaves out videos.
+const IDENTITY = DISCOVER_CATEGORIES.find((c) => c.title === "Identity")!;
+
+// ML data holding faces and no CLIP embedding, which file 302 has.
+const FACES_ONLY = { face: { faces: [{ faceID: "face-def", score: 0.9 }] } };
+
 let mock: MetaMockState;
 let testDir: string;
 
@@ -204,6 +220,65 @@ const buildMetaMock = async (): Promise<MetaMockState> => {
         updationTime: 1710000000000000,
     };
 
+    // A file in coll 10 with the ML data given. `fileType` is Ente's number: 0
+    // an image, 1 a video.
+    const vacationFile = (
+        id: number,
+        title: string,
+        fileType: number,
+        mlData: object,
+    ) => {
+        const fileKey = sodium.crypto_secretstream_xchacha20poly1305_keygen();
+        const { ciphertext: encKey, nonce } = encryptSecretbox(fileKey, ck1);
+        const meta = encryptStreamBlob(
+            new TextEncoder().encode(
+                JSON.stringify({
+                    title,
+                    fileType,
+                    creationTime: 1700000000000000,
+                    modificationTime: 1700000000000000,
+                }),
+            ),
+            fileKey,
+        );
+        const ml = encryptBlob(
+            new Uint8Array(gzipSync(Buffer.from(JSON.stringify(mlData)))),
+            fileKey,
+        );
+        return {
+            raw: {
+                id,
+                collectionID: 10,
+                ownerID: 42,
+                encryptedKey: toBase64(encKey),
+                keyDecryptionNonce: toBase64(nonce),
+                metadata: {
+                    encryptedData: toBase64(meta.ciphertext),
+                    decryptionHeader: toBase64(meta.header),
+                },
+                file: {
+                    decryptionHeader: toBase64(sodium.randombytes_buf(24)),
+                },
+                thumbnail: {
+                    decryptionHeader: toBase64(sodium.randombytes_buf(24)),
+                },
+                updationTime: 1700000000000000,
+            },
+            mlData: {
+                encryptedData: toBase64(ml.ciphertext),
+                decryptionHeader: toBase64(ml.header),
+            },
+        };
+    };
+    // Files 300 and 301 in coll 10: a photo and a video whose ML data holds
+    // only a CLIP embedding, Identity's own text embedding.
+    const identityML = { clip: { embedding: IDENTITY.textEmbedding } };
+    const identityPhoto = vacationFile(300, "passport.jpg", 0, identityML);
+    const identityVideo = vacationFile(301, "passport.mp4", 1, identityML);
+    // File 302 in coll 10: a photo whose ML data holds faces and no CLIP
+    // embedding.
+    const facesPhoto = vacationFile(302, "party.jpg", 0, FACES_ONLY);
+
     // Encrypt ML data for file 100 (gzipped JSON, encrypted with file key)
     const mlPayload = JSON.stringify({
         face: {
@@ -230,7 +305,7 @@ const buildMetaMock = async (): Promise<MetaMockState> => {
         clip: {
             version: 1,
             client: "test",
-            embedding: [0.5, 0.6, 0.7],
+            embedding: SUNSET_EMBEDDING,
         },
     });
     const gzipped = gzipSync(Buffer.from(mlPayload));
@@ -270,12 +345,23 @@ const buildMetaMock = async (): Promise<MetaMockState> => {
         keyAttributes: account.keyAttributes,
         encryptedToken: account.encryptedToken,
         collections: [rawColl1, rawColl2],
-        filesByCollection: { 10: [rawFile1], 20: [rawFile2] },
+        filesByCollection: {
+            10: [
+                rawFile1,
+                identityPhoto.raw,
+                identityVideo.raw,
+                facesPhoto.raw,
+            ],
+            20: [rawFile2],
+        },
         encryptedMLData: {
             100: {
                 encryptedData: toBase64(mlCiphertext),
                 decryptionHeader: toBase64(mlHeader),
             },
+            300: identityPhoto.mlData,
+            301: identityVideo.mlData,
+            302: facesPhoto.mlData,
         },
         fileCiphertexts: { 100: encFileBody1 },
     };
@@ -432,7 +518,9 @@ describe("quak backup-metadata", () => {
         expect(account.email).toBe(TEST_EMAIL);
     });
 
-    it("fetches and decrypts ML data by default", async () => {
+    // The CLIP embedding is left out of the ML data, and the keywords it gives
+    // are written in its place.
+    it("fetches and decrypts ML data by default, with keywords for the CLIP embedding", async () => {
         const outDir = join(testDir, "ml-data");
         await runBackup(outDir);
 
@@ -452,8 +540,41 @@ describe("quak backup-metadata", () => {
         expect(fileMeta.mlData.face.faces[0].faceID).toBe("face-abc");
         expect(fileMeta.mlData.face.faces[0].score).toBeCloseTo(0.98);
         expect(fileMeta.mlData.face.faces[0].detection.box.x).toBeCloseTo(0.1);
-        expect(fileMeta.mlData.clip).toBeDefined();
-        expect(fileMeta.mlData.clip.embedding).toEqual([0.5, 0.6, 0.7]);
+        expect(fileMeta.mlData).not.toHaveProperty("clip");
+        expect(fileMeta.keywords).toEqual(["Sunset"]);
+    });
+
+    // Ente's photos app keeps videos out of some Discover categories, Identity
+    // among them. Files 300 and 301, a photo and a video, share one CLIP
+    // embedding, Identity's own text embedding, so the photo gets the keyword
+    // Identity and the video does not.
+    it("keeps a video out of a category that leaves out videos", async () => {
+        expect(IDENTITY.showVideo).toBe(false);
+        const outDir = join(testDir, "video-keywords");
+        await runBackup(outDir);
+
+        const vacation = join(outDir, "collections", "10-Vacation");
+        const fileJSON = (fileID: number) =>
+            JSON.parse(readFileSync(join(vacation, `${fileID}.json`), "utf-8"));
+        expect(fileJSON(300).keywords).toContain("Identity");
+        expect(fileJSON(301)).toHaveProperty("keywords");
+        expect(fileJSON(301).keywords).not.toContain("Identity");
+    });
+
+    // A file's ML data can hold faces and no CLIP embedding. File 302's JSON
+    // then has the ML data and no keywords.
+    it("writes no keywords for ML data without a CLIP embedding", async () => {
+        const outDir = join(testDir, "no-clip");
+        await runBackup(outDir);
+
+        const fileMeta = JSON.parse(
+            readFileSync(
+                join(outDir, "collections", "10-Vacation", "302.json"),
+                "utf-8",
+            ),
+        );
+        expect(fileMeta.mlData).toEqual(FACES_ONLY);
+        expect(fileMeta).not.toHaveProperty("keywords");
     });
 
     it("extracts EXIF from downloaded files when --exif is set", async () => {
@@ -541,7 +662,7 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
                 "utf-8",
             ),
         );
-        expect(ok.mlData.clip.embedding).toEqual([0.5, 0.6, 0.7]);
+        expect(ok.keywords).toEqual(["Sunset"]);
         expect(ok.mlDataError).toBeUndefined();
 
         const failed = JSON.parse(
@@ -599,7 +720,7 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
                 "utf-8",
             ),
         );
-        expect(ok.mlData.clip.embedding).toEqual([0.5, 0.6, 0.7]);
+        expect(ok.keywords).toEqual(["Sunset"]);
     });
 
     it("exits 0 when every ML data request succeeds", async () => {

test/cli/read.test.ts

@@ -3,20 +3,25 @@
  * issue #36, issue #52).
  *
  * The `collections`, `files`, `get`, and `get-thumb` commands must answer for
- * current server state, not the local cache. `files`, `get` and `get-thumb`
- * force a `Library.fresh()` round-trip before they read; the stand-in library
- * below serves nothing until `fresh()` has been awaited, so a helper that read
- * without refreshing would come back empty and fail here. `collections` asks
- * the library for the collection list alone (`freshCollections()`, issue
- * #210), and must not wait for a refresh, which would fetch the files of every
- * collection.
+ * current server state, not the local cache. `get`, `get-thumb` and `files`
+ * without a collection ID force a `Library.fresh()` round-trip before they
+ * read. The stand-in library below serves no cached record until `fresh()` has
+ * been awaited, so a helper that read without refreshing would come back empty
+ * and fail here. `collections` asks the library for the collection list alone
+ * (`freshCollections()`, issue #210). `files` with a collection ID asks for
+ * that list, then for that one collection's files (`freshFiles()`, issue
+ * #217). Neither may wait for a refresh, which would fetch the files of every
+ * collection. On an empty cache, `get` and `get-thumb` ask for that list, then
+ * for one collection's files at a time, until they find the file (issue #222).
+ * `helper fix-missing-thumbnails --file` does the same until it finds every
+ * file it names, reading the most recently updated collection first (issue
+ * #223).
  *
- * `collections` and `files` also list in the library's enumeration order
- * (`listCollections`/`listFiles`) — the order the pre-library CLI printed — not
+ * `collections` and `files` also list in the order the library gives them, not
  * the albums/photos projection's newest-first order. The fixtures are seeded in
- * an enumeration order that a newest-first sort would rearrange, so a
- * regression to the projection order would fail here too. Field values still
- * come from the raw metadata via `cli-output.ts`.
+ * an order that a newest-first sort would rearrange, so a regression to the
+ * projection order would fail here too. Field values still come from the raw
+ * metadata via `cli-output.ts`.
  */
 
 import { describe, it, expect } from "vitest";
@@ -25,10 +30,10 @@ import {
     freshCollections,
     freshFiles,
     freshFile,
+    freshFilesByID,
     type FreshReadLibrary,
 } from "../../src/cli-read.js";
 import { fileListRow } from "../../src/cli-output.js";
-import type { Photo } from "../../src/library/index.js";
 import type { Collection, EnteFile } from "../../src/model/types.js";
 
 const collection = (id: number, updationTime: number): Collection => ({
@@ -62,14 +67,15 @@ const file = (
     updationTime: creationTime,
 });
 
-// A library that reveals its records only after `fresh()` has been awaited, and
-// serves them in the enumeration order it was given. `freshCollections()`
-// answers the collection list in that order without a refresh. `photos.byID`
-// returns a stand-in `Photo` carrying just the fileID the helper passes
-// through.
+// A library that reveals its cached records only after `fresh()` has been
+// awaited, and serves them in the enumeration order it was given.
+// `freshCollections()` and `freshFiles()` answer in that order without a
+// refresh.
 class FakeLibrary implements FreshReadLibrary {
     freshCalls = 0;
     freshCollectionsCalls = 0;
+    // The ID of each collection `freshFiles()` was asked for.
+    freshFilesCalls: number[] = [];
     private refreshed = false;
 
     constructor(
@@ -88,9 +94,9 @@ class FakeLibrary implements FreshReadLibrary {
         return this.collections;
     }
 
-    getCollection(id: number): Collection | undefined {
-        if (!this.refreshed) return undefined;
-        return this.collections.find((c) => c.id === id);
+    async freshFiles(collection: Collection): Promise<EnteFile[]> {
+        this.freshFilesCalls.push(collection.id);
+        return this.files.filter((f) => f.collectionID === collection.id);
     }
 
     listCollections(): Collection[] {
@@ -107,14 +113,6 @@ class FakeLibrary implements FreshReadLibrary {
         if (!this.refreshed) return undefined;
         return this.files.find((f) => f.id === fileID);
     }
-
-    photos = {
-        byID: ({ fileID }: { fileID: number }): Photo | undefined => {
-            if (!this.refreshed) return undefined;
-            if (!this.files.some((f) => f.id === fileID)) return undefined;
-            return { fileID } as unknown as Photo;
-        },
-    };
 }
 
 describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
@@ -138,18 +136,24 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
         expect(rows.map((c) => c.id)).not.toEqual(newestFirst);
     });
 
-    it("freshFiles refreshes first, lists in enumeration order, keeps raw fields", async () => {
-        // Enumeration order by id 10, 11, 12; creationTimes ascending, so a
+    it("freshFiles with a collection ID reads that collection's files alone and keeps their order and raw fields", async () => {
+        // Given in the order 10, 11, 12; creationTimes ascending, so a
         // newest-first sort would reverse them.
         const files = [
             file(10, 1, 1_700_000_000_000_000),
             file(11, 1, 1_700_000_000_000_001),
             file(12, 1, 1_700_000_000_000_002),
+            file(20, 2, 1_700_000_000_000_003),
         ];
-        const lib = new FakeLibrary([collection(1, 100)], files);
+        const lib = new FakeLibrary(
+            [collection(1, 100), collection(2, 200)],
+            files,
+        );
 
         const rows = await freshFiles(lib, 1);
-        expect(lib.freshCalls).toBe(1);
+        expect(lib.freshCalls).toBe(0);
+        expect(lib.freshCollectionsCalls).toBe(1);
+        expect(lib.freshFilesCalls).toEqual([1]);
         expect(rows?.map((f) => f.id)).toEqual([10, 11, 12]);
 
         // Field values come from raw metadata: microsecond creationTime and the
@@ -206,27 +210,117 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
     it("freshFiles returns undefined for an unknown collection", async () => {
         const lib = new FakeLibrary([collection(1, 100)], []);
         const rows = await freshFiles(lib, 999);
-        expect(lib.freshCalls).toBe(1);
+        expect(lib.freshCalls).toBe(0);
+        expect(lib.freshFilesCalls).toEqual([]);
         expect(rows).toBeUndefined();
     });
 
-    it("freshFile refreshes first, then resolves the photo and its raw record", async () => {
-        const f = file(10, 1, 1_700_000_000_000_000);
-        const lib = new FakeLibrary([collection(1, 100)], [f]);
-
-        const resolved = await freshFile(lib, 10);
-        expect(lib.freshCalls).toBe(1);
-        expect(resolved?.photo.fileID).toBe(10);
-        expect(resolved?.file.metadata.title).toBe("file-10.jpg");
-        expect(resolved?.file.metadata.creationTime).toBe(
-            1_700_000_000_000_000,
+    // Three collections, each holding one file.
+    const threeCollections = (): FakeLibrary =>
+        new FakeLibrary(
+            [collection(1, 100), collection(2, 200), collection(3, 300)],
+            [
+                file(10, 1, 1_700_000_000_000_000),
+                file(20, 2, 1_700_000_000_000_001),
+                file(30, 3, 1_700_000_000_000_002),
+            ],
         );
+
+    it("freshFile on an empty cache reads each collection's files in the server's order until one holds the file", async () => {
+        const lib = threeCollections();
+
+        const found = await freshFile(lib, 20);
+        expect(lib.freshCalls).toBe(0);
+        expect(lib.freshCollectionsCalls).toBe(1);
+        expect(lib.freshFilesCalls).toEqual([1, 2]);
+        expect(found?.metadata.title).toBe("file-20.jpg");
+        expect(found?.metadata.creationTime).toBe(1_700_000_000_000_001);
     });
 
-    it("freshFile returns undefined for an unknown file", async () => {
-        const lib = new FakeLibrary([collection(1, 100)], []);
-        const resolved = await freshFile(lib, 404);
-        expect(lib.freshCalls).toBe(1);
-        expect(resolved).toBeUndefined();
+    // The stand-in's cache holds what the server holds once `fresh()` has run.
+    it("freshFile on a cache that holds the collections refreshes, then looks the file up in the cache", async () => {
+        const lib = threeCollections();
+        await lib.fresh();
+
+        const found = await freshFile(lib, 30);
+        expect(lib.freshCalls).toBe(2);
+        expect(lib.freshCollectionsCalls).toBe(0);
+        expect(lib.freshFilesCalls).toEqual([]);
+        expect(found?.collectionID).toBe(3);
+    });
+
+    it("freshFile on an empty cache returns undefined for an unknown file, after reading every collection's files", async () => {
+        const lib = threeCollections();
+
+        const found = await freshFile(lib, 404);
+        expect(lib.freshCalls).toBe(0);
+        expect(lib.freshFilesCalls).toEqual([1, 2, 3]);
+        expect(found).toBeUndefined();
+    });
+
+    // The server lists collections 1, 2 and 3, and collection 3 was updated
+    // most recently. File 11 is in collections 1 and 2.
+    const fileInTwoCollections = (): FakeLibrary =>
+        new FakeLibrary(
+            [collection(1, 100), collection(2, 200), collection(3, 300)],
+            [
+                file(10, 1, 1_700_000_000_000_000),
+                file(11, 1, 1_700_000_000_000_001),
+                file(11, 2, 1_700_000_000_000_001),
+                file(20, 2, 1_700_000_000_000_002),
+                file(30, 3, 1_700_000_000_000_003),
+            ],
+        );
+
+    it("freshFilesByID on an empty cache reads the most recently updated collection's files first, until it has found every file", async () => {
+        const lib = fileInTwoCollections();
+
+        const found = await freshFilesByID(lib, [11, 20]);
+        expect(lib.freshCalls).toBe(0);
+        expect(lib.freshCollectionsCalls).toBe(1);
+        expect(lib.freshFilesCalls).toEqual([3, 2]);
+        expect(found.get(11)?.collection.id).toBe(2);
+        expect(found.get(11)?.file.collectionID).toBe(2);
+        expect(found.get(20)?.collection.id).toBe(2);
+    });
+
+    // Collection 1 was updated last, though collections 2 and 3 have higher
+    // IDs. File 11 is in collections 1 and 2.
+    it("freshFilesByID on an empty cache reads the most recently updated collection first, though its ID is the lowest", async () => {
+        const lib = new FakeLibrary(
+            [collection(1, 400), collection(2, 200), collection(3, 300)],
+            [
+                file(11, 1, 1_700_000_000_000_001),
+                file(11, 2, 1_700_000_000_000_001),
+            ],
+        );
+
+        const found = await freshFilesByID(lib, [11]);
+        expect(lib.freshFilesCalls).toEqual([1]);
+        expect(found.get(11)?.collection.id).toBe(1);
+        expect(found.get(11)?.file.collectionID).toBe(1);
+    });
+
+    // The stand-in's cache holds what the server holds once `fresh()` has run.
+    it("freshFilesByID on a cache that holds the collections refreshes, then finds each file in the most recently updated collection that holds it", async () => {
+        const lib = fileInTwoCollections();
+        await lib.fresh();
+
+        const found = await freshFilesByID(lib, [11, 404]);
+        expect(lib.freshCalls).toBe(2);
+        expect(lib.freshCollectionsCalls).toBe(0);
+        expect(lib.freshFilesCalls).toEqual([]);
+        expect(found.get(11)?.collection.id).toBe(2);
+        expect(found.get(11)?.file.collectionID).toBe(2);
+        expect(found.has(404)).toBe(false);
+    });
+
+    it("freshFilesByID on an empty cache reads every collection's files when a file is in none of them", async () => {
+        const lib = fileInTwoCollections();
+
+        const found = await freshFilesByID(lib, [10, 404]);
+        expect(lib.freshFilesCalls).toEqual([3, 2, 1]);
+        expect(found.get(10)?.collection.id).toBe(1);
+        expect(found.has(404)).toBe(false);
     });
 });

test/client/session.test.ts

@@ -47,6 +47,20 @@ describe("Client.toJSON", () => {
         expect(Client.fromJSON(snapshot).toJSON()).toEqual(snapshot);
     });
 
+    it("round-trips a snapshot that names a server", () => {
+        const snapshot = {
+            ...validSnapshot(),
+            apiOrigin: "https://ente.example.com",
+        };
+        expect(Client.fromJSON(snapshot).toJSON()).toEqual(snapshot);
+    });
+
+    it("names the server apiOptions gave fromJSON", () => {
+        const apiOrigin = "https://ente.example.com";
+        const client = Client.fromJSON(validSnapshot(), { apiOrigin });
+        expect(client.toJSON().apiOrigin).toBe(apiOrigin);
+    });
+
     it("throws instead of emitting a snapshot without a token", () => {
         const client = Client.fromJSON(validSnapshot());
         client.getApiClient().clearAuthToken();
@@ -77,6 +91,8 @@ describe("Client.fromJSON", () => {
         ["publicKey", undefined],
         ["publicKey", "not base64!"],
         ["publicKey", shortKey],
+        ["apiOrigin", 7],
+        ["apiOrigin", ""],
     ])("rejects %s = %j, naming the field", (field, value) => {
         const snapshot: Record<string, unknown> = { ...validSnapshot() };
         snapshot[field] = value;
@@ -85,6 +101,30 @@ describe("Client.fromJSON", () => {
         );
     });
 
+    // The token belongs to the snapshot's server, so that server wins over
+    // the one in `apiOptions`.
+    it("sends requests to the snapshot's server", async () => {
+        const urls: string[] = [];
+        const client = Client.fromJSON(
+            { ...validSnapshot(), apiOrigin: "https://ente.example.com/" },
+            {
+                apiOrigin: "https://other.example.com",
+                fetch: async (input) => {
+                    urls.push(String(input));
+                    return new Response(JSON.stringify({ collections: [] }), {
+                        status: 200,
+                        headers: { "content-type": "application/json" },
+                    });
+                },
+            },
+        );
+
+        await client.listCollections();
+        expect(urls).toEqual([
+            "https://ente.example.com/collections/v2?sinceTime=0",
+        ]);
+    });
+
     it.each([null, "a string", 42])("rejects a non-object %j", (value) => {
         expect(() => Client.fromJSON(value)).toThrow(
             /^Invalid session data: not a JSON object/,

test/clip-keywords-sunset.json

@@ -0,0 +1,109 @@
+{
+    "photo": "https://upload.wikimedia.org/wikipedia/commons/d/db/Sunset_off_the_Massachusetts_coast_-_NOAA.jpg",
+    "sha256": "be62f3289c9bf7b75af68cd6978021d54e8a531d5d4cc493f6e9fbf5e1ab74d3",
+    "embedding": [
+        -0.0021123015, -0.04437615, -0.062354308, 0.006249992, 0.022141185,
+        0.011159019, 0.01227392, -0.00030532264, 0.03186437, 0.09196922,
+        0.039111234, -0.037084445, -0.032962628, -0.019820206, -0.013141825,
+        0.0047183544, 0.017137334, 0.046581306, 0.0047622235, 0.04982521,
+        0.023128154, 0.032080866, 0.037709303, 0.019241406, 0.0032100384,
+        -0.014275843, 0.049337693, 0.025953151, 0.014263995, -0.0062328726,
+        -0.040504098, -0.015083303, -0.008123875, -0.006233639, -0.05352683,
+        -0.0048083956, 0.0763486, -0.013155067, 0.025114346, 0.022016674,
+        0.0001588023, -0.010341572, -0.0037518279, -0.019181008, 0.04549337,
+        -0.0010725886, 0.004636866, 0.017177915, -0.0117317885, 0.0026097358,
+        0.030738998, 0.013158572, -0.08856403, -0.0121391015, 0.011477229,
+        -0.0033104012, 0.035524756, -0.018607879, -0.0071739843, -0.031906743,
+        0.0056713386, -0.0044195866, 0.00292471, 0.00061686034, -0.043075223,
+        0.09040427, 0.001522537, -0.011445652, 0.009609886, -0.0053201183,
+        -0.0057452377, 0.00972853, -0.0024397348, -0.021255083, 0.015893811,
+        -0.006020548, -0.09428023, 0.014277583, -0.017975299, -0.011878315,
+        0.011825521, 0.00014203975, -0.0020437292, -0.05305343, -0.024840152,
+        -0.011336872, -0.002796541, -0.00036171873, 0.004530499, 0.044486605,
+        0.021602718, -0.01292857, 0.024530916, 0.019559948, -0.039928272,
+        -0.0681174, 0.049101, 0.02461532, 0.0365531, -0.04070358, -0.011011939,
+        -0.0027188526, -0.020370876, 0.012904856, 0.043157287, -0.051658224,
+        -0.3726012, 0.019549074, -0.0499252, -0.04800559, -0.011493859,
+        0.02893256, 0.060341954, 0.0013929688, -0.051283866, -0.024286803,
+        0.044124287, -0.005995806, -0.062817656, 0.007369614, -0.021470634,
+        -0.028147545, 0.00411659, 0.04234773, 0.03787647, 0.008579682,
+        -0.038568925, -0.027498025, -0.028416378, 0.0022012168, 0.010906191,
+        -0.013745787, 0.018267838, 0.013531097, -0.061559845, 0.004682517,
+        -0.005993303, 0.0028113858, -0.0028286583, 0.0026029332, -0.06693403,
+        -0.0072381413, -0.02419584, 0.0037009937, -0.0011296495, -0.030310456,
+        0.0005999609, 0.009945354, 0.024915686, 0.038672045, 0.023009432,
+        -0.05093051, -0.09101409, -0.06686167, -0.06211481, -0.022600474,
+        0.0024902367, -0.0067171026, -0.016769778, -0.0032803628, 0.008623083,
+        -0.0016110836, 0.014925073, 0.033811048, 0.012829449, -0.052575592,
+        0.03728281, -0.0051201633, 0.021238064, -0.031086605, -0.0043402985,
+        0.25054494, -0.028396713, -0.010725523, -0.023445267, 0.021751683,
+        0.003130971, 0.012270548, -0.014836475, 0.014466885, 0.0021478906,
+        -0.088153616, 0.015356315, 0.039639276, -0.040283255, 0.014923924,
+        -0.029303031, -0.004475201, 0.047299225, -0.06726505, -0.012491676,
+        -0.002080582, 0.03506341, 0.04146718, 0.0074114986, -0.009582633,
+        0.006623749, -0.05454898, -0.0034387247, -0.014780821, 0.03698328,
+        -0.034441486, 0.22032379, -0.021792699, 0.008001907, 0.0341019,
+        0.0025784625, -0.0071689896, -0.014151225, -0.050547987, -0.030372301,
+        -0.03288388, -0.042987447, -0.0033656165, 0.021502906, -0.0030374376,
+        -0.024413, -0.03506603, 0.030891743, 0.0069994032, 0.004887366,
+        0.030707076, 0.0065348223, 0.0008240555, 0.016809713, -0.024151647,
+        -0.0040458064, 0.044567548, 0.03908212, -0.061782435, 0.0026002831,
+        0.02003242, -0.0115451, -0.045135964, -0.0663542, 0.005240975,
+        0.0010777109, -0.009276002, -0.011919081, 0.027654119, -0.013634551,
+        0.0129154315, -0.013195512, 0.040571272, -0.099290505, -0.04656693,
+        -0.20979854, 0.04181004, -0.06769166, -0.04121946, 0.018260444,
+        0.005617506, -0.15694222, 0.032472793, 0.014189598, 0.008345103,
+        -0.041118417, 0.0074527003, -0.011494104, 0.004906474, -0.0027100458,
+        0.0009994594, -0.041061398, -0.045274835, 0.015187793, 0.034275036,
+        -0.02813289, 0.02098019, -0.015043198, 0.03716667, 0.013542642,
+        -0.008104835, 0.014965486, -0.027978558, 0.021163663, -0.006942737,
+        -0.034969814, -0.038961593, 0.016802894, -0.012318175, -0.047821593,
+        -0.01933216, 0.0064432374, 0.0322358, -0.020437784, 0.02659447,
+        -0.03022022, -0.019443465, -0.094178624, -0.017009107, 0.03860671,
+        -0.00023695534, -0.033089854, -0.014677462, 0.012356399, 0.018040499,
+        -0.19360113, 0.011710817, -0.019871691, 0.020124774, 0.024948105,
+        -0.024573864, -0.028349072, -0.041731454, 0.032646254, 0.015098165,
+        0.0018447012, -0.01129152, -0.02481687, 0.036048, 0.20797811,
+        0.04317757, -0.017239755, 0.033431605, -0.011384422, 0.0099174455,
+        -0.19739223, -0.008901062, -0.056426458, 0.014591864, 0.0044474266,
+        0.008209326, -0.012241819, 0.042553645, -0.021552453, 0.008611734,
+        0.048944127, -0.013872049, 0.019179989, 0.008183264, -0.053223096,
+        -0.027327595, -0.0033542586, -0.012353858, -0.010941209, 0.010212546,
+        0.029714134, -0.011385593, -0.0013529479, 0.014505385, 0.01148663,
+        0.012613636, -0.0074430504, -0.014008101, -0.028146038, -0.042778917,
+        0.02143165, 0.016812252, -0.06286215, 0.02085532, -0.035128277,
+        0.0241282, 0.01397908, -0.034992814, 0.013153766, 0.0068236147,
+        0.013594184, -0.06859393, 0.051849406, 0.029675119, -0.0037699796,
+        -0.00013216575, 0.02916647, 0.031859048, -0.015006051, 0.0001439966,
+        0.0026584554, -0.0046948595, 0.039754897, 0.034873154, 0.0012870439,
+        -0.011516082, -0.074089095, 0.019241294, 0.00076195505, 0.0027334464,
+        -0.036206458, -0.0011109262, -0.048838835, -0.00204685, 0.01179991,
+        0.0068136845, 0.009852495, 0.017308988, 0.03159545, 0.058803353,
+        0.038116634, 0.008836143, -0.010460637, -0.010994099, -0.0012926942,
+        -0.048370842, 0.037696566, -0.08093239, -0.0041242186, -0.005866373,
+        0.02048607, -0.12549639, 0.011705739, -0.0021425558, 0.018078592,
+        -0.038228087, 0.004245971, -0.11560507, 0.01381203, -0.02734893,
+        0.029595908, 0.101082794, -0.042842254, -0.08397301, -0.1546237,
+        -0.037624564, 0.0069606556, -0.008020479, 0.0043268185, 0.016504597,
+        0.044554908, -0.05240209, 0.020519335, -0.050994243, -0.019861093,
+        0.022066997, -0.0188911, 0.012919696, -0.022872692, -0.012961374,
+        -0.020071112, 0.0074553066, 0.06402014, 0.04065211, 0.014139608,
+        0.06221546, -0.02574805, 0.010114545, 0.020619567, 0.032264773,
+        -0.009148828, 0.022849955, -0.016593235, -0.032456633, -0.006737303,
+        -0.0037493357, -0.065912865, -0.0024181998, -0.0037048983, -0.019304488,
+        0.009085037, 0.057529982, -0.0378338, 0.01792042, 0.01885012,
+        -0.0043600523, 0.017382368, -0.027792772, 0.006749118, -0.0053297766,
+        -0.011628587, 0.015736576, 0.024801968, 0.011543913, -0.01159873,
+        -0.000759976, 0.042183932, 0.05764386, -0.06422416, 0.010577374,
+        -0.02261249, 0.02282793, -0.05141075, -0.11287078, -0.020670144,
+        -0.020953743, 0.030885734, -0.012597732, -0.121472664, 0.028447151,
+        -0.0415664, -0.010061708, 0.06441338, 0.0271207, -0.0016693748,
+        -0.0045694243, 0.020342957, 0.030528478, 0.007129517, -0.0029600926,
+        -0.007061397, 0.06036486, -0.004736018, -0.0090786135, -0.07601116,
+        -0.039556272, -0.051269546, -0.019473685, 0.029858151, 0.028464556,
+        0.012424476, 0.013742254, 0.008742927, 0.006065751, -0.009892663,
+        0.07699999, -0.003443197, 0.0027863425, 0.029197995, 0.0211013,
+        0.01098379, 0.016684106, -0.028917195, 0.008122037, 0.0265241,
+        -0.04461223
+    ]
+}

test/clip-keywords.test.ts

@@ -0,0 +1,129 @@
+// `clipKeywords`: the keywords quak writes for a file in place of its CLIP
+// embedding.
+//
+// Ente's ML data for a file holds a CLIP embedding, 512 numbers its image
+// model gave the photo. Ente's photos app puts a file in a category of its
+// Discover section when the cosine similarity of that embedding and the text
+// embedding of the category's query is at least the category's minimum score,
+// and keeps videos out of some categories. `clipKeywords` applies that rule
+// and returns the titles of the categories a file falls in, highest score
+// first; `quak backup` and `quak backup-metadata` write them as `keywords`.
+
+import { readFileSync } from "node:fs";
+import { describe, expect, it } from "vitest";
+
+import { DISCOVER_CATEGORIES } from "../src/clip-keyword-vectors.js";
+import { clipKeywords } from "../src/clip-keywords.js";
+
+// 512 numbers: the ones given, then zeros.
+const vector = (...start: number[]): number[] => [
+    ...start,
+    ...new Array<number>(512 - start.length).fill(0),
+];
+
+const withEmbedding = (embedding: unknown) => ({
+    face: { faces: [] },
+    clip: { embedding },
+});
+
+// Two made-up categories, each with a text embedding along one axis. The
+// embedding [3, 4] has a cosine similarity of 3/5 with "Across" and 4/5 with
+// "Up". Videos are kept out of "Across".
+const ACROSS = {
+    title: "Across",
+    query: "across",
+    minScore: 0.6,
+    showVideo: false,
+    textEmbedding: vector(1),
+};
+const UP = {
+    title: "Up",
+    query: "up",
+    minScore: 0.5,
+    showVideo: true,
+    textEmbedding: vector(0, 1),
+};
+
+describe("clipKeywords", () => {
+    it("gives each category whose minimum score the similarity reaches, highest score first", () => {
+        expect(
+            clipKeywords(withEmbedding(vector(3, 4)), false, [ACROSS, UP]),
+        ).toEqual(["Up", "Across"]);
+    });
+
+    it("leaves out a category when the similarity is just below its minimum score", () => {
+        const stricter = { ...ACROSS, minScore: 0.6 + 1e-9 };
+
+        expect(
+            clipKeywords(withEmbedding(vector(3, 4)), false, [stricter, UP]),
+        ).toEqual(["Up"]);
+    });
+
+    it("keeps a video out of a category that leaves out videos", () => {
+        expect(
+            clipKeywords(withEmbedding(vector(3, 4)), true, [ACROSS, UP]),
+        ).toEqual(["Up"]);
+    });
+
+    it("gives an empty list for an embedding that falls in no category", () => {
+        expect(
+            clipKeywords(withEmbedding(vector(0, 0, 1)), false, [ACROSS, UP]),
+        ).toEqual([]);
+    });
+
+    it.each([
+        ["no CLIP embedding", { face: { faces: [] } }],
+        ["a CLIP part that is not an object", { clip: null }],
+        ["a CLIP part without an embedding", { clip: {} }],
+        ["an embedding of 511 numbers", withEmbedding(vector(3, 4).slice(1))],
+        ["an embedding with NaN in it", withEmbedding(vector(3, NaN))],
+        ["an embedding with Infinity in it", withEmbedding(vector(Infinity))],
+        ["an embedding of strings", withEmbedding(vector(3, 4).map(String))],
+        ["an embedding of zeros", withEmbedding(vector())],
+    ])("gives no keywords for ML data with %s", (_, mlData) => {
+        expect(clipKeywords(mlData, false, [ACROSS, UP])).toBeUndefined();
+    });
+});
+
+// The CLIP embedding Ente's image model gives a public-domain NOAA photo of a
+// sunset, made by bin/clip-keyword-vectors from the photo at `photo`.
+const SUNSET: { photo: string; sha256: string; embedding: number[] } =
+    JSON.parse(
+        readFileSync(
+            new URL("clip-keywords-sunset.json", import.meta.url),
+            "utf-8",
+        ),
+    );
+
+describe("Ente's Discover categories", () => {
+    it("are the 16 of Ente's photos app, each with a text embedding of norm 1", () => {
+        expect(DISCOVER_CATEGORIES.map((c) => c.title)).toEqual([
+            "Identity",
+            "Screenshots",
+            "QR Codes",
+            "Receipts",
+            "Notes",
+            "Memes",
+            "Visiting Cards",
+            "Babies",
+            "Pets",
+            "Selfies",
+            "Wallpapers",
+            "Food",
+            "Celebrations",
+            "Sunset",
+            "Hills",
+            "Greenery",
+        ]);
+        for (const category of DISCOVER_CATEGORIES) {
+            expect(category.textEmbedding).toHaveLength(512);
+            expect(Math.hypot(...category.textEmbedding)).toBeCloseTo(1, 5);
+        }
+    });
+
+    it("give a photo of a sunset the keyword Sunset and no other", () => {
+        expect(
+            clipKeywords({ clip: { embedding: SUNSET.embedding } }, false),
+        ).toEqual(["Sunset"]);
+    });
+});

test/ente-server.ts

@@ -19,8 +19,11 @@ import { SRP, SrpServer } from "fast-srp-hap";
 import { toBase64, deriveKEK, deriveLoginSubkey } from "../src/crypto/index.js";
 import type { KeyAttributes } from "../src/auth/types.js";
 
-const TEST_OPS = 2;
-const TEST_MEM = 64 * 1024 * 1024;
+// The account's password is stretched at libsodium's lowest Argon2id cost.
+// Every login stretches it again, and at a realistic cost that takes seconds on
+// a loaded host.
+const TEST_OPS = 1;
+const TEST_MEM = 8192;
 
 export interface MetaMockState {
     verifier: Buffer;

test/library/content-library.test.ts

@@ -253,6 +253,12 @@ describe("Library content wiring", () => {
         await expect(
             lib.thumbnails.ensure({ fileIDs: [1], priority: "visible" }),
         ).rejects.toThrow(/content cache/i);
+        await expect(lib.originalOf(file(1, 1))).rejects.toThrow(
+            /content cache/i,
+        );
+        await expect(lib.thumbnailOf(file(1, 1))).rejects.toThrow(
+            /content cache/i,
+        );
         await lib.close();
     });
 });

test/library/content.test.ts

@@ -375,6 +375,25 @@ describe("ContentCache.original / thumbnail", () => {
         await cache.open();
         await expect(cache.original(999)).rejects.toThrow(/unknown file/i);
     });
+
+    // `getFile` knows no file here, so the cache can fetch only the file it is
+    // given.
+    it("originalOf and thumbnailOf fetch and cache the file they are given", async () => {
+        const { cache, source } = buildCache({ files: [] });
+        await cache.open();
+
+        const original = await cache.originalOf(file(5));
+        const thumbnail = await cache.thumbnailOf(file(5));
+
+        expect(source.originalCalls).toEqual([5]);
+        expect(source.thumbnailCalls).toEqual([5]);
+        expect(original.path).toBe(join(cacheDir, "originals", "5.jpg"));
+        expect(thumbnail.path).toBe(join(cacheDir, "thumbnails", "5.jpg"));
+        expect(cache.pathsFor(5)).toEqual({
+            originalPath: original.path,
+            thumbnailPath: thumbnail.path,
+        });
+    });
 });
 
 describe("ContentCache.ensureThumbnails", () => {

test/packaging/build-bin.test.ts

@@ -0,0 +1,241 @@
+// `script/build-bin` compiles `bin/quak`, the single binary that the README's
+// cron setup runs. bun compiles it from a copy of its own executable, so the
+// binary loads the same libraries as the bun that built it. nix's bun loads
+// libraries from the nix store, which garbage collection can delete: on Linux
+// its C library, on macOS its ICU. The script therefore downloads bun's own
+// release build, which loads only the system's libraries.
+//
+// What the script promises, and why each test matters:
+//
+// - Each system, machine and C library that bun publishes a build for gets its
+//   own archive: Linux on x86-64 or arm64, with glibc or with musl, as on
+//   Alpine, and macOS on Intel or Apple silicon. A bun built for another
+//   system, machine or C library does not start, and `make install` fails.
+// - The archive's SHA-256 must match the one in the script before anything in
+//   the archive runs. Without that check, a tampered download would build the
+//   binary that cron runs.
+// - Any other system or machine stops with an error, and never falls back to
+//   nix's bun. Building with nix's bun would bring back the dependency on the
+//   nix store.
+//
+// Each test copies the script into a fresh directory, which the script then
+// treats as the checkout, and runs it there. Stubs first on PATH stand in for
+// `uname`, `ldd`, `curl`, `unzip`, `nix-shell` and `shasum`, so no test
+// downloads anything or runs a real bun.
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import { execFileSync, spawnSync } from "node:child_process";
+import {
+    chmodSync,
+    copyFileSync,
+    existsSync,
+    mkdirSync,
+    mkdtempSync,
+    readFileSync,
+    rmSync,
+    symlinkSync,
+    writeFileSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { fileURLToPath } from "node:url";
+
+const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
+
+// The stubs, keyed by the command each replaces. `uname` and `ldd` describe the
+// system that runBuildBin passes in the environment. `curl`, `unzip`,
+// `nix-shell` and `shasum` write the arguments they were given, one per line,
+// to a file named after the command in $STUB_LOG_DIR, which also shows that
+// they ran.
+const stubs: Record<string, string> = {
+    uname: `
+        case "$1" in
+            -s) echo "$STUB_UNAME_S" ;;
+            -m) echo "$STUB_UNAME_M" ;;
+        esac`,
+    // The script looks only at the start of the line: "musl libc" or not.
+    ldd: `
+        if [ "$STUB_LIBC" = musl ]; then
+            echo "musl libc (x86_64)"
+        else
+            echo "ldd (GNU libc) 2.36"
+        fi`,
+    // Writes a file that is not the archive to the path after -o.
+    curl: `
+        printf '%s\\n' "$@" > "$STUB_LOG_DIR/curl"
+        while [ $# -gt 1 ]; do
+            if [ "$1" = -o ]; then echo "not the archive" > "$2"; fi
+            shift
+        done`,
+    // Unpacks a stub bun where the real archive puts bun: in a directory named
+    // after the archive, inside the directory after -d. That bun writes
+    // bin/quak, as a real compile would.
+    unzip: `
+        printf '%s\\n' "$@" > "$STUB_LOG_DIR/unzip"
+        for arg in "$@"; do
+            case "$arg" in *.zip) archive="$arg" ;; esac
+            if [ "$previous" = -d ]; then directory="$arg"; fi
+            previous="$arg"
+        done
+        bun="$directory/$(basename "$archive" .zip)/bun"
+        mkdir -p "$(dirname "$bun")"
+        echo '#!/bin/sh' > "$bun"
+        echo 'mkdir -p bin && echo binary > bin/quak' >> "$bun"
+        chmod +x "$bun"`,
+    "nix-shell": `
+        printf '%s\\n' "$@" > "$STUB_LOG_DIR/nix-shell"`,
+    // The script checks the hash with `shasum` where there is no `sha256sum`.
+    // The stub fails, as `shasum` does when the hash does not match.
+    shasum: `
+        printf '%s\\n' "$@" > "$STUB_LOG_DIR/shasum"
+        exit 1`,
+};
+
+let checkout = "";
+
+const writeScript = (path: string, body: string): void => {
+    writeFileSync(path, `#!/bin/sh\n${body}\n`);
+    chmodSync(path, 0o755);
+};
+
+// A checkout holding the script, a stub script/build that does nothing, the
+// stubs, and the directory they log to.
+beforeEach(() => {
+    checkout = mkdtempSync(join(tmpdir(), "quak-build-bin-"));
+    mkdirSync(join(checkout, "script"));
+    copyFileSync(
+        join(repoRoot, "script/build-bin"),
+        join(checkout, "script/build-bin"),
+    );
+    chmodSync(join(checkout, "script/build-bin"), 0o755);
+    writeScript(join(checkout, "script/build"), "exit 0");
+    mkdirSync(join(checkout, "stubs"));
+    for (const [command, body] of Object.entries(stubs)) {
+        writeScript(join(checkout, "stubs", command), body);
+    }
+    mkdirSync(join(checkout, "log"));
+});
+
+afterEach(() => {
+    rmSync(checkout, { recursive: true, force: true });
+});
+
+// Runs the script with nothing in its environment but the stubs' settings and a
+// PATH. Unless the caller passes one, the PATH finds the stubs first and then
+// the host's commands.
+const runBuildBin = (
+    system: string,
+    machine: string,
+    {
+        libc = "glibc",
+        path = `${join(checkout, "stubs")}:${process.env.PATH}`,
+    } = {},
+) =>
+    spawnSync(join(checkout, "script/build-bin"), {
+        cwd: checkout,
+        encoding: "utf-8",
+        env: {
+            PATH: path,
+            STUB_UNAME_S: system,
+            STUB_UNAME_M: machine,
+            STUB_LIBC: libc,
+            STUB_LOG_DIR: join(checkout, "log"),
+        },
+    });
+
+// The real commands that the script and the stubs run.
+const hostCommands = [
+    "basename",
+    "chmod",
+    "dirname",
+    "grep",
+    "mkdir",
+    "mktemp",
+    "rm",
+];
+
+// A Mac has no `ldd`, and not every Mac has `sha256sum`. This makes a PATH that
+// finds neither: the stubs other than `ldd`, then links to hostCommands.
+const makeMacPath = (): string => {
+    rmSync(join(checkout, "stubs", "ldd"));
+    mkdirSync(join(checkout, "host-commands"));
+    for (const command of hostCommands) {
+        const target = execFileSync("sh", ["-c", `command -v ${command}`], {
+            encoding: "utf-8",
+        }).trim();
+        symlinkSync(target, join(checkout, "host-commands", command));
+    }
+    return `${join(checkout, "stubs")}:${join(checkout, "host-commands")}`;
+};
+
+const ran = (command: string): boolean =>
+    existsSync(join(checkout, "log", command));
+
+const argumentsOf = (command: string): string[] =>
+    readFileSync(join(checkout, "log", command), "utf-8")
+        .trimEnd()
+        .split("\n");
+
+describe("script/build-bin", () => {
+    // The stub curl writes a file that is not the archive, so the hash check
+    // must stop the script before unzip runs. A script without the check would
+    // run unzip, and the stub bun it unpacks would write bin/quak. On Linux the
+    // check must use `sha256sum`, since a stock Alpine has no `shasum`.
+    it.each([
+        ["Linux", "x86_64", "glibc", "bun-linux-x64-baseline.zip"],
+        ["Linux", "x86_64", "musl", "bun-linux-x64-musl-baseline.zip"],
+        ["Linux", "aarch64", "glibc", "bun-linux-aarch64.zip"],
+        ["Linux", "aarch64", "musl", "bun-linux-aarch64-musl.zip"],
+    ])(
+        "on %s %s with %s, downloads %s and stops when its hash does not match",
+        (system, machine, libc, archive) => {
+            const result = runBuildBin(system, machine, { libc });
+            expect(argumentsOf("curl")).toContain(
+                "https://github.com/oven-sh/bun/releases/download/bun-v1.3.13/" +
+                    archive,
+            );
+            expect(ran("shasum")).toBe(false);
+            expect(result.status).not.toBe(0);
+            expect(ran("unzip")).toBe(false);
+            expect(ran("nix-shell")).toBe(false);
+            expect(existsSync(join(checkout, "bin/quak"))).toBe(false);
+        },
+    );
+
+    // On the PATH from makeMacPath the script finds no `sha256sum` and checks
+    // the hash with the stub `shasum`. That stub fails, so the script must stop
+    // before unzip runs.
+    // On Apple silicon `uname -m` says arm64, where bun's archive says aarch64.
+    it.each([
+        ["Darwin", "x86_64", "bun-darwin-x64-baseline.zip"],
+        ["Darwin", "arm64", "bun-darwin-aarch64.zip"],
+    ])(
+        "on %s %s, downloads %s and stops when shasum rejects its hash",
+        (system, machine, archive) => {
+            const result = runBuildBin(system, machine, {
+                path: makeMacPath(),
+            });
+            expect(argumentsOf("curl")).toContain(
+                "https://github.com/oven-sh/bun/releases/download/bun-v1.3.13/" +
+                    archive,
+            );
+            expect(argumentsOf("shasum")).toEqual(["-a", "256", "-c", "-"]);
+            expect(result.status).not.toBe(0);
+            expect(ran("unzip")).toBe(false);
+            expect(ran("nix-shell")).toBe(false);
+            expect(existsSync(join(checkout, "bin/quak"))).toBe(false);
+        },
+    );
+
+    it.each([
+        ["Linux", "riscv64"],
+        ["FreeBSD", "amd64"],
+    ])(
+        "stops on %s %s, which bun publishes no build for",
+        (system, machine) => {
+            const result = runBuildBin(system, machine);
+            expect(result.status).not.toBe(0);
+            expect(ran("curl")).toBe(false);
+            expect(ran("nix-shell")).toBe(false);
+        },
+    );
+});

test/thumbnails/thumbnails.test.ts

@@ -6,10 +6,12 @@
  * working thumbnails, others return 404 or empty bodies. The tests
  * verify that the detection and repair logic handles each case correctly.
  *
- * As of issue #52 both helpers take an open `Library` for enumeration and the
- * `Client` for the API operations that stay unchanged (the thumbnail existence
- * check, and the encrypt-and-upload path). `fixMissingThumbnails` reads each
- * original through the library's content cache (`photo.original()`).
+ * `listMissingThumbnails` takes an open `Library` to enumerate the files. Both
+ * helpers take the `Client` for the API operations: the thumbnail existence
+ * check, and the encrypt-and-upload path. `fixMissingThumbnails` takes the
+ * record of each file, which these tests look up with `cachedFilesByID`. It
+ * reads each original through the library's content cache, by
+ * `lib.originalOf(file)`.
  *
  * `fixMissingThumbnails` is the most complex function in quak: it
  * downloads the original file, generates a JPEG thumbnail with jpeg-js,
@@ -41,6 +43,7 @@ import {
 import { SRP, SrpServer } from "fast-srp-hap";
 import { Client } from "../../src/client.js";
 import { Library } from "../../src/library/index.js";
+import { cachedFilesByID } from "../../src/cli-read.js";
 import {
     listMissingThumbnails,
     fixMissingThumbnails,
@@ -54,8 +57,11 @@ import type { RetryOptions } from "../../src/retry.js";
 
 const TEST_EMAIL = "thumb@example.com";
 const TEST_PASSWORD = "thumbpass";
-const TEST_OPS = 2;
-const TEST_MEM = 64 * 1024 * 1024;
+// The password is stretched at libsodium's lowest Argon2id cost. Each test
+// builds an account and logs in, and at a realistic cost that takes seconds on
+// a loaded host.
+const TEST_OPS = 1;
+const TEST_MEM = 8192;
 const TEST_TIME = 1700000000000000;
 
 interface ThumbMockState {
@@ -267,6 +273,7 @@ const buildThumbMock = async (opts?: {
 
 const buildThumbFetch = (m: ThumbMockState) => {
     let srpServer: SrpServer;
+    let signedMD5: string | undefined;
 
     return (async (
         input: RequestInfo | URL,
@@ -359,8 +366,9 @@ const buildThumbFetch = (m: ThumbMockState) => {
             return new Response("not found", { status: 404 });
         }
 
-        // Upload URL minting
+        // Upload URL minting. The real server signs the MD5 into the URL.
         if (path === "/files/upload-url") {
+            signedMD5 = JSON.parse(init?.body as string).contentMD5;
             return json({
                 objectKey: `42/thumb-${Date.now()}`,
                 url: "https://s3.mock.test/presigned-put",
@@ -369,6 +377,11 @@ const buildThumbFetch = (m: ThumbMockState) => {
 
         // Presigned PUT (S3 upload)
         if (url.startsWith("https://s3.mock.test/")) {
+            // Storage refuses a PUT that lacks the signed `Content-MD5`.
+            const headers = new Headers(init?.headers);
+            if (headers.get("Content-MD5") !== signedMD5) {
+                return new Response("AccessDenied", { status: 400 });
+            }
             const body = init?.body;
             // Store the uploaded bytes for later inspection
             if (body instanceof Uint8Array) {
@@ -656,7 +669,12 @@ describe("fixMissingThumbnails", () => {
         const client = await login(buildThumbFetch(fixMock));
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(lib, client, [101]);
+        const results = await fixMissingThumbnails(
+            lib,
+            client,
+            [101],
+            cachedFilesByID(lib, [101]),
+        );
         lib.close();
 
         expect(results.length).toBe(1);
@@ -693,7 +711,12 @@ describe("fixMissingThumbnails", () => {
         const client = await login(buildThumbFetch(fixMock));
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(lib, client, [999]);
+        const results = await fixMissingThumbnails(
+            lib,
+            client,
+            [999],
+            cachedFilesByID(lib, [999]),
+        );
         lib.close();
 
         expect(results.length).toBe(1);
@@ -715,6 +738,7 @@ describe("fixMissingThumbnails", () => {
             lib,
             client,
             [101, 102],
+            cachedFilesByID(lib, [101, 102]),
             (msg, level) => {
                 if (level === "error") errors.push(msg);
             },
@@ -745,6 +769,7 @@ describe("fixMissingThumbnails", () => {
             lib,
             client,
             [101, 103, 104],
+            cachedFilesByID(lib, [101, 103, 104]),
         );
         lib.close();
 
@@ -779,7 +804,12 @@ describe("fixMissingThumbnails", () => {
         const client = await login(counted.fetch);
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(lib, client, [101]);
+        const results = await fixMissingThumbnails(
+            lib,
+            client,
+            [101],
+            cachedFilesByID(lib, [101]),
+        );
         lib.close();
 
         expect(results[0]!.status).toBe("skipped");
@@ -800,7 +830,12 @@ describe("fixMissingThumbnails", () => {
         const client = await login(counted.fetch);
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(lib, client, [101]);
+        const results = await fixMissingThumbnails(
+            lib,
+            client,
+            [101],
+            cachedFilesByID(lib, [101]),
+        );
         lib.close();
 
         expect(results[0]!.status).toBe("skipped");
@@ -841,7 +876,12 @@ describe("fixMissingThumbnails", () => {
         const client = await login(buildThumbFetch(fixMock));
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(lib, client, [101]);
+        const results = await fixMissingThumbnails(
+            lib,
+            client,
+            [101],
+            cachedFilesByID(lib, [101]),
+        );
         lib.close();
 
         expect(results[0]!.status).toBe("fixed");

vitest.config.ts

@@ -3,8 +3,15 @@ import { configDefaults, defineConfig } from "vitest/config";
 // vitest does not read .gitignore when looking for tests. A checkout nested
 // under .claude/ has its own test/ tree, and without this exclude the suite
 // runs once per nested checkout and still reports success.
+//
+// vitest starts one worker per CPU core by default, and a shared host has
+// dozens. Four keep the suite well inside the Dockerfile's time limit. vitest
+// 2 rejects a maxWorkers below its default minWorkers, so minWorkers is set
+// too.
 export default defineConfig({
     test: {
         exclude: [...configDefaults.exclude, ".claude/**"],
+        minWorkers: 1,
+        maxWorkers: 4,
     },
 });