README.md rewritten sentence by sentence (closes #219)

The old text served as the list of what a reader needs covered. The new
text leads with what quak is, what it is for and how to use it. Every
command, option, exit code and file layout follows in reference sections,
then the library reference, then design, entrypoints and development.
Every statement was checked against the code on next; the places where
the old text disagreed with it are listed in the pull request. No sentence
runs past about 30 words, every table is now a list, and no issue links or
project history remain in text meant for users. Two code comments that
named README sections now name the new ones.

Model: fable-5-1

README.md

@@ -73,8 +73,7 @@ and `dist/` is not committed. Clone and build it as above, then add the clone to
 your own program's `package.json` by its path, for example
 `yarn add file:../quak`. The package's `main`, `types` and `exports` fields
 point into `dist/`, so the clone must be built first. The package is an ES
-module only: `package.json` sets `"type": "module"`, and its `exports` field
-offers only `import`, so a CommonJS program cannot `require` it.
+module only, so a CommonJS program cannot `require` it.
 
 ```ts
 import { Client, Library } from "quak";
@@ -90,8 +89,9 @@ const client = await Client.login({
 const lib = await Library.open({
     client,
     // Without these two, open() also starts downloading every thumbnail in
-    // the account and the latest week's originals into the cache. It fetches
-    // machine-learning data after each refresh either way.
+    // the account, and the originals of the favorites album and the latest
+    // week, into the cache. It fetches machine-learning data after each
+    // refresh either way.
     precacheThumbnails: false,
     precacheOriginals: false,
 });
@@ -122,10 +122,12 @@ await lib.close();
 for. It refreshes every 3 seconds until `close()`. It also downloads, in the
 background, every thumbnail in the account and the originals of the favorites
 album and of the latest week (see "Opening a library"). The example turns those
-two downloads off. After each refresh the library also fetches each file's
-machine-learning data into the cache. That is the faces and the other data
-Ente's apps compute for a photo (see "The JSON beside each original"). No option
-turns this off.
+two downloads off. After each refresh that succeeds, the library also fetches
+each file's machine-learning data into the cache. That is the faces and the
+other data Ente's apps compute for a photo (see "The JSON beside each
+original"). No option in `LibraryOptions` turns this off. A `client` without a
+`fetchMLData` method fetches none, which is how quak's own read commands open
+their library.
 
 The `Client` under the library handles login, saved sessions and the raw
 requests. The "Library reference" below covers both.
@@ -134,9 +136,9 @@ requests. The "Library reference" below covers both.
 
 `quak backup` never prompts, so cron can run it. `make install` builds quak as
 one binary and copies it to `~/bin/quak`. Besides what Getting Started needs,
-`make install` needs [nix](https://nixos.org/download). It uses bun, a
-JavaScript runtime, to bundle the built `dist/` into one executable. It runs bun
-through `nix-shell -p bun`.
+`make install` needs `make` and [nix](https://nixos.org/download). It uses bun,
+a JavaScript runtime, to bundle the built `dist/` into one executable. It runs
+bun through `nix-shell -p bun`.
 
 Log in once with `~/bin/quak`, as the user the cron job will run as:
 
@@ -156,10 +158,11 @@ output to `~/quak-backup.log`. Sunday's run adds `--verify`:
 
 A run with `--verify` does all a plain run does. It also hashes each original
 already in the backup and compares the result with the content hash. The content
-hash is the hash of the file's bytes that Ente recorded at upload. It replaces
-any original that does not match. Sunday's run is the `--verify` run, rather
-than a second job that night, because two backups of one directory cannot run at
-once. The second one exits 2 without doing anything.
+hash is the hash of the file's bytes that Ente recorded at upload. It fetches a
+new copy of each original that does not match, and replaces the original once
+the copy matches. Sunday's run is the `--verify` run, rather than a second job
+that night, because two backups of one directory cannot run at once. The second
+one exits 2 without doing anything.
 
 Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab
 names quak by its full path. quak finds the saved session through `HOME`, which
@@ -227,9 +230,12 @@ set, it asks on the terminal. The password shows as `*` characters while it is
 typed.
 
 When the account has a second factor, quak asks for the code. It asks
-`TOTP code: ` for a time-based code from an authenticator app, and
-`Email verification code: ` for a code sent by email. An account whose only
-second factor is a passkey cannot log in with quak.
+`TOTP code: ` for a time-based code from an authenticator app. An account whose
+only second factor is a passkey cannot log in with quak. When the account has
+email verification on, quak asks `Email verification code: ` for the code sent
+by email. The password is then used only to unlock the keys. After the email
+code quak expects the login to complete. When the server asks for a second
+factor at that point, login fails.
 
 On success it logs `Logged in as <email> (user <id>)` and
 `Session saved to <path>`. On failure it logs `Login failed: <reason>`, and the
@@ -239,7 +245,9 @@ to stdout.
 #### `quak whoami`
 
 Prints the saved session's account on stdout as one line of JSON:
-`{"email":"you@example.com","userID":12345}`.
+`{"email":"you@example.com","userID":12345}`. It reads the session file only and
+sends no request, so it succeeds even when the server no longer accepts the
+session.
 
 #### `quak logout`
 
@@ -374,7 +382,8 @@ thumbnail in the account, one collection at a time, newest collection first, so
 on a large account it takes a while. A thumbnail counts as missing when the
 server answers with an empty body or with HTTP 404. Any other failure is logged
 as `[<collection>] Could not check <title>: <reason> (not reported as missing)`,
-and the file is left out of the list.
+and the file is left out of the list. The command exits 0 even when some files
+could not be checked.
 
 The server lets only a file's owner replace its thumbnail. A file another
 account owns, in a collection shared with you, is skipped and not checked.
@@ -417,12 +426,13 @@ Each file ends in one of three states:
   `--file` is held by no collection.
 
 Without `--json`, the command ends with counts of fixed, skipped and failed
-files, a `Skipped:` list and a `Failed files:` list. With `--json`, one JSON
-array on stdout replaces them; each object holds `fileID`, `title`,
-`collection`, `status` and, for a skipped or failed file, `reason`. When any
-file failed, the exit status is 1. A skipped file does not change it. When
-nothing needs fixing, the command logs `No missing thumbnails found.` and
-exits 0. With `--json` it then writes nothing on stdout.
+files. A `Skipped:` list and a `Failed files:` list follow when they have
+entries. With `--json`, one JSON array on stdout replaces them; each object
+holds `fileID`, `title`, `collection`, `status` and, for a skipped or failed
+file, `reason`. When any file failed, the exit status is 1. A skipped file does
+not change it. When nothing needs fixing, the command logs
+`No missing thumbnails found.` and exits 0. With `--json` it then writes nothing
+on stdout.
 
 quak refuses the whole command, before any request, when one `--file` ID is
 invalid (see "IDs").
@@ -453,9 +463,9 @@ collection fetched, such as `Fetched the files of 3 of 120 collections`.
 
 When a refresh or a search fails, the command logs the error on one line and
 exits 1, or 3 when the server no longer accepts the saved session. A collection
-or file that quak cannot decrypt does not stop a refresh: the command answers
-from what did decrypt (see "Default reads and fresh reads"). `backup` counts it
-as failed (see "What quak cannot decrypt").
+or file that quak cannot decrypt does not stop a refresh or a search: the
+command leaves it out and answers from what did decrypt (see "Default reads and
+fresh reads"). `backup` counts it as failed (see "What quak cannot decrypt").
 
 ### IDs
 
@@ -507,11 +517,12 @@ one that exits non-zero prints only its fatal lines. The prompts of `login` are
 not log lines, and `-q` leaves them alone.
 
 With `--json`, a command's JSON on stdout takes the place of its summary on
-stderr. For `backup`, that is the lines from `--- Backup complete ---` to the
-end of the `Failed files:` list. For `list-missing-thumbnails`, it is
+stderr. For `backup`, that is the `--- Backup complete ---` line, the counts and
+the `Failed files:` list. For `list-missing-thumbnails`, it is
 `No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`. For
-`fix-missing-thumbnails`, it is the lines from `--- Done ---` to the end of the
-`Failed files:` list. Every other line is logged as without `--json`.
+`fix-missing-thumbnails`, it is the `--- Done ---` line, the counts, the
+`Skipped:` list and the `Failed files:` list. Every other line is logged as
+without `--json`.
 
 ### Exit codes
 
@@ -769,11 +780,13 @@ too. It is then cut short before the ID, just far enough to fit. IDs never
 change, so a name stays the same from run to run until such a clash appears or
 goes away.
 
-Each run removes the symlinks that no longer belong in a collection's directory.
-It removes the directory and JSON of a collection that was deleted or renamed,
-once the directory holds nothing else. Nothing else under `collections/` is
-touched: a file or symlink you put there stays, and a directory that still holds
-one stays too, with its JSON.
+Each run removes every symlink in a collection's directory that points into the
+date folders and is not one of its current links, yours included. A file or
+symlink of yours at the name of one of its links is replaced by the link. A
+directory at that name stays, and the file goes into `failures.json`. Anything
+else you put under `collections/` stays. A run removes the directory and JSON of
+a collection that was deleted or renamed, once the directory holds nothing else.
+One that still holds something of yours stays, with its JSON.
 
 A collection whose directory or JSON cannot be written does not stop the run.
 For example, a file you put there may already have the directory's name. The run
@@ -837,8 +850,8 @@ file outside them. `failures.json` is deleted when the list is empty.
 `backup.lock` keeps two backups of the same directory from running at once, such
 as a cron run that starts while the previous one is still going. A backup
 creates `<dir>` if it is missing and takes the lock before its refresh, so a
-refused run sends no request. It removes the lock when it ends, whether it
-succeeded or failed.
+refused run sends no request. A run that still holds the lock removes it when it
+ends, whether it succeeded or failed.
 
 The lock is a directory that
 [proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates.
@@ -886,13 +899,13 @@ The summary and the `--json` output add the counts `verified`, `mismatched` and
 `unchecked`. Each counts only originals that were already stored. A mismatch
 that was replaced with matching bytes does not make the exit code non-zero.
 
-Two kinds of failure stay in `failures.json` only until the next run:
+A run without `--verify` drops two kinds of failure from `failures.json`:
 
 - a mismatched original whose new copy failed;
 - a stored original that cannot be read.
 
-A run without `--verify` finds the original in place, counts it as skipped, and
-drops the entry, so that run can exit 0. Only a `--verify` run hashes it again.
+That run finds the original in place, counts it as skipped and drops the entry,
+so it can exit 0. Only a `--verify` run hashes the original again.
 
 Without `--verify`, no stored original is hashed, the summary is unchanged, and
 the three counts are 0 in the JSON.
@@ -918,7 +931,11 @@ scheme. Opening a library deletes such leftovers in the cache's `originals/` and
 The rename replaces whatever was at the destination rather than writing through
 it. A symlink there is replaced, not followed, and the new file has the
 temporary file's permissions, not those of the file it replaced. Each original
-is written with mode `0600`, readable by its owner only.
+is written with mode `0600`, readable by its owner only. Everything else the
+backup writes, its folders and JSON files included, gets the mode your umask
+allows, which usually lets every user on the machine read it. The JSON beside an
+original holds the file's location and faces. The `backup-metadata` tree is
+written the same way.
 
 ### The metadata dump directory
 
@@ -1026,10 +1043,11 @@ onto what the cache holds, empty or not. A refresh then starts only when
 - `contentSource`: a replacement for the client's own byte source. Mainly for
   tests.
 
-In each pool, your own requests take the next free slot ahead of the two
-background downloads. A file already in the cache is not fetched again. When the
-originals cache grows past its limit, the least recently used originals are
-evicted, but never a pinned one, and never one whose fetch is still in flight.
+In the content and thumbnail pools, your own requests take the next free slot
+ahead of the two background downloads. A file already in the cache is not
+fetched again. When the originals cache grows past its limit, the least recently
+used originals are evicted, but never a pinned one, and never one whose fetch is
+still in flight.
 
 `onProgress` receives a `RefreshEvent`. Its `operation` names the work:
 `"refresh"`, `"fetchMLData"` for a machine-learning fetch, and
@@ -1257,7 +1275,9 @@ symlinks. Once the lock is lost, it rejects with
 `BackupOptions`:
 
 - `downloadDirectory`: where the tree lives. Default: the library's.
-- `includeOriginals`: fetch and store originals. Default `true`.
+- `includeOriginals`: fetch and store originals. Default `true`. With `false`,
+  the backup also fetches no machine-learning data, and writes no JSON beside
+  the originals and no symlinks.
 - `includeThumbnails`: also store thumbnails as `thumbnails/<fileID>.jpg` beside
   `collections/`. Default `false`. `quak backup` does not set it.
 - `onlyAlbumNames`: back up only the albums with these names.
@@ -1278,11 +1298,13 @@ releases it once the backup ends.
 many requests run at once, because Ente meters these three kinds of traffic
 differently:
 
-- metadata, with 10 slots;
-- content, with 5;
-- thumbnails, with 25.
+- metadata, with 10 slots: only the machine-learning data requests;
+- content, with 5: originals;
+- thumbnails, with 25: thumbnails.
 
-Construct one with
+Only the requests named above go through the pools. A refresh,
+`freshCollections()` and `freshFiles()` send theirs outside them. Construct one
+with
 `new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
 and pass it as `pools`. Each pool serves on-demand work ahead of background
 work. A task submitted under a key that is still pending is not run a second
@@ -1411,8 +1433,9 @@ return, quak never keeps a truncated file.
 `Client.login({ email, password, totp?, emailOTP?, apiOptions? })` logs in and
 resolves to a `Client`. `totp` and `emailOTP` are callbacks that return the code
 when the account asks for one. Without the callback the account needs, login
-throws. A passkey challenge always throws. `apiOptions` configures the HTTP
-client (see below).
+throws. A passkey challenge always throws. After an email code, login expects
+the key attributes and the token. When the server asks for a second factor
+instead, login throws. `apiOptions` configures the HTTP client (see below).
 
 The `Client` holds the session token, the master key, the secret key and the
 public key in memory. There is no on-disk session store in the library; the
@@ -1422,6 +1445,13 @@ write wherever it likes. `Client.fromJSON(snapshot, apiOptions?)` restores a
 working client from that snapshot without logging in again. It checks every
 field and each key's length first, and throws an error naming the bad field.
 
+`Client.fromJSON` works only once libsodium, the cryptography library quak uses,
+is loaded. Only `Client.login` loads it. A program that restores a session in a
+new process first runs `await sodium.ready`, with `sodium` imported from
+`libsodium-wrappers-sumo`, which quak depends on. The exported `beginLogin`,
+`unwrapAuth`, `decryptCollection`, `decryptFile`, `downloadFile` and
+`downloadThumbnail` need the same.
+
 `client.logout()` clears the token and zeroes the key buffers in place. After
 it, every other method on that client throws. It does not contact the server, so
 the token stays valid there and in any saved snapshot. Call
@@ -1444,8 +1474,10 @@ return only the decrypted records that are not deleted, from `sinceTime` `0`.
 `client.fetchMLData({ fileIDs, fileKeys })` fetches and decrypts the
 machine-learning data of up to 200 files. `client.downloadFile(file, outPath?)`
 and `client.downloadThumbnail(file, outPath?)` download and decrypt one file to
-`outPath`. Without `outPath`, they write to the working directory under the
-names `quak get` and `quak get-thumb` would use.
+`outPath`. For a live photo, `downloadFile` writes the image and the video, each
+named `outPath` with its own extension. `quak get --out` instead puts the image
+at the path itself. Without `outPath`, they write to the working directory under
+the names `quak get` and `quak get-thumb` would use.
 
 `ApiClientOptions`:
 
@@ -1640,8 +1672,8 @@ Login derives the keys in this order:
    the code (`POST /users/two-factor/verify`); the server then returns the key
    attributes and the token. quak cannot answer a passkey request. When the
    account has email verification on (`isEmailMFAEnabled`), an email code
-   replaces SRP, and the server returns the key attributes and the token after
-   it.
+   replaces SRP. quak then expects the key attributes and the token in the
+   answer, and fails the login when the server asks for a second factor instead.
 5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK
    yields the 32-byte master key.
 6. `crypto_secretbox_open_easy` over the encrypted secret key with the master
@@ -1720,36 +1752,35 @@ This repository adheres to the
 standard. Normalized scripts in `script/` are the entrypoints for the
 development workflow, and most Makefile targets are thin shims that call them.
 The scripts are POSIX `sh`, not bash, so they run in minimal containers such as
-Alpine. The scripts marked "ours" are this repository's own additions to the
-standard.
+Alpine. Of these, only `bootstrap`, `setup`, `test` and `cibuild` come from the
+standard. The others are additions to it.
 
 - `script/bootstrap`: install all dependencies. It installs `make`, `git`, Node
   and `yarn` when they are missing, then runs `yarn install --frozen-lockfile`.
 - `script/setup`: set up the repo for development after a fresh clone. It runs
   `script/bootstrap`, then `script/install-precommit`.
-- `script/projectname` (ours): print the project name. `script/docker` uses it
-  for the image tag.
-- `script/build` (ours): compile the TypeScript sources into `dist/` and stamp
-  the version into `dist/package.json`. Then check that the entrypoints
+- `script/projectname`: print the project name. `script/docker` uses it for the
+  image tag.
+- `script/build`: compile the TypeScript sources into `dist/` and stamp the
+  version into `dist/package.json`. Then check that the entrypoints
   `package.json` declares (`main`, `types`, `bin`) were written, and make the
   command line executable.
-- `script/version` (ours): print the version `script/build` stamps (see
-  "Version").
+- `script/version`: print the version `script/build` stamps (see "Version").
 - `script/test`: run the test suite, by building the `test` phase of the
   `Dockerfile`. Requires Docker.
 - `script/lint`: run eslint and a prettier check, by building the `lint` phase
   of the `Dockerfile`. Requires Docker.
 - `script/fmt`: format all files with prettier. It rewrites files in place.
 - `script/fmt-check`: check formatting on the host. It is read-only.
-- `script/check` (ours): run all checks: `test`, `lint`, `fmt-check`.
+- `script/check`: run all checks: `test`, `lint`, `fmt-check`.
 - `script/docker`: build the image, tagged with `script/projectname`.
 - `script/cibuild`: what CI runs: `script/bootstrap`, then `script/check`, then
   the image build.
-- `script/precommit` (ours): run by the git pre-commit hook. It runs
-  `script/lint` and `script/fmt-check`, and not the tests, so that a commit of
-  failing tests can land (see "Development workflow").
-- `script/install-precommit` (ours): install the git pre-commit hook.
-  `make hooks` calls it.
+- `script/precommit`: run by the git pre-commit hook. It runs `script/lint` and
+  `script/fmt-check`, and not the tests, so that a commit of failing tests can
+  land (see "Development workflow").
+- `script/install-precommit`: install the git pre-commit hook. `make hooks`
+  calls it.
 
 The Makefile targets are:
 
@@ -1816,8 +1847,8 @@ empty, `dev` or `unknown`, because such a build could not be traced to its
 commit.
 
 `.dockerignore` therefore does not leave out `.git`, so any `docker build .` of
-a clone stamps the commit it was built from. A shallow clone stamps a tag only
-when the cloned commit itself carries one, and otherwise the short commit.
+a clone stamps the commit it was built from. A clone of depth 1 stamps a tag
+only when its commit carries one, and otherwise the short commit.
 `.dockerignore` does leave out `.git/config`, which holds the clone's remote URL
 and any credential in it; `git describe` does not need that file.
 `script/docker` and `script/cibuild` work out the version on the host, with
@@ -1873,7 +1904,10 @@ These apply to people and to coding agents alike.
     - the nvm archive `script/bootstrap` downloads, and the Python packages
       `bin/clip-keyword-vectors` installs, by SHA-256 hash;
     - Node and yarn, when `script/bootstrap` installs them, by version.
-- bun is not pinned: `make build-bin` uses whatever `nix-shell -p bun` provides.
+- Not pinned:
+    - bun, which `make build-bin` takes from `nix-shell -p bun`;
+    - `make` and `git`, which `script/bootstrap` installs from the system
+      package manager when they are missing, as every `Dockerfile` stage does.
 - Follow the workflow above: branches off `next`, tests first and committed
   failing, and `next` merged into `main` by sneak alone.
 - Before every commit, `make lint` and `make fmt-check` must pass. `make lint`