-v raises the CLI's log level per repeat; -q prints only fatal errors (closes #212)

Every line quak prints on stderr now goes through one log, in src/cli-log.ts, at one of six levels. The default, info, prints what quak printed before. -v adds each retry and the start and end of each refresh and ML data fetch; -vv adds each request with its result. -q prints only fatal lines, and is refused with -v; both are program options. Every non-zero exit logs one fatal line, so a run that exits 1 because files failed says how many and where to look. Every line reporting a failure is at error. --json changes stdout only, so backup and --json runs log their progress on stderr. withRetry and ApiClient report retries and requests through new callbacks.

Model: opus-5-5

README.md

@@ -35,8 +35,11 @@ yarn quak login
 # List the user's collections (albums).
 yarn quak collections
 
-# List files in a collection.
+# List every file in the account.
-yarn quak files --collection 12345
+yarn quak files
+
+# List the files in a collection.
+yarn quak files 12345
 
 # Download and decrypt a single file.
 yarn quak get 67890 --out ./photo.jpg
@@ -134,12 +137,12 @@ matters.
 
 ### Exit codes
 
-| Code | Meaning                                                                                                                                                                                                        |
+| Code | Meaning                                                                                                                                                                                                                                         |
-| ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `0`  | The backup is complete: every original is at its save path, and no file failed.                                                                                                                                |
+| `0`  | The backup is complete: every original is at its save path, and no file failed.                                                                                                                                                                 |
-| `1`  | The run finished with files in `failures.json` or albums it could not decrypt or write (see "Backup layout"), or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. |
+| `1`  | The run finished with files in `failures.json` or albums it could not decrypt or write (see "Backup layout"), and its last line says how many, or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. |
-| `2`  | Another backup of the same directory is running. This run sent no request and changed nothing.                                                                                                                 |
+| `2`  | Another backup of the same directory is running. This run sent no request and changed nothing.                                                                                                                                                  |
-| `3`  | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as.                                                    |
+| `3`  | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as.                                                                                     |
 
 `<message>` is the error's message, followed by the message of each error that
 caused it. Under Node, as `yarn quak` runs quak, `fetch` reports every failure
@@ -375,6 +378,8 @@ quak/
         thumbnails.ts   detect + regenerate missing thumbnails
         client.ts       high-level Client class assembled from the above
         cli-commands.ts the CLI's commands as functions returning exit codes
+        cli-log.ts      the CLI's log level, and the one function the
+                        commands write stderr lines through
         cli-output.ts   how the CLI prints a file's title and time
         cli-read.ts     fresh reads for the CLI's read commands
         cli-run.ts      run a command, print its error, exit with its code
@@ -453,6 +458,12 @@ A custom API endpoint is configurable for self-hosted servers via the
 constructor option `apiOrigin`. When set, file downloads route through
 `<apiOrigin>/files/download/<id>` instead of the dedicated CDN host.
 
+The constructor option `onRequest` is called once for each request sent, a
+retried request once for each attempt, with its method, its URL and the status
+of the answer or the error the request failed with. The URL of a presigned
+upload is given without its query, which holds the upload's signature. The CLI
+logs each request with `-vv` (see "CLI surface").
+
 Required request headers on every authenticated call:
 
 - `X-Auth-Token`: the decrypted auth token from login.
@@ -529,7 +540,9 @@ each wait is drawn at random below its ceiling. Every other command uses the
 defaults. A library user gets the same budget by passing
 `UNATTENDED_RETRY_OPTIONS` as `ApiClientOptions.retry`. `sleep` and `random` are
 injectable through the same option, which is how the test suite exercises the
-whole policy without waiting.
+whole policy without waiting. `onRetry`, given there too, is called before each
+wait with the error the attempt failed with, the attempt's number and the wait;
+the CLI logs it with `-v` (see "CLI surface").
 
 Two deadlines, renewed for each attempt:
 
@@ -643,11 +656,12 @@ thumbnails), for the user to delete if they want it gone.
 
 ```
 quak [--cache-dir <path>] <command>                       global: local metadata/content cache location
+quak [-v | -vv | -q] <command>                            global: log more on stderr, or only fatal errors
 quak login                                                interactive or QUAK_EMAIL/QUAK_PASSWORD
 quak whoami                                               print logged-in account as JSON
 quak logout                                               end the session, delete it
 quak collections [--json]                                 list all collections
-quak files --collection <id> [--json]                     list files in a collection
+quak files [collectionID] [--json]                        list files in a collection, or every file
 quak get <fileID> [--out path] [--collection]             download and decrypt a file
 quak get-thumb <fileID> [--out] [--collection]            download and decrypt a thumbnail
 quak backup <dir> [--json] [--verify]                     full incremental backup
@@ -666,17 +680,72 @@ request, and neither fetches any collection's files nor changes the cache. The
 others wait for a refresh. With an empty cache, as on a first run, that refresh
 fetches the files of every collection, one collection after another, and stores
 nothing until it has them all, which on a large account takes a while. Meanwhile
-the command prints a line on stderr for each collection whose files it has
+the command logs a line on stderr for each collection whose files it has
-fetched, such as `Fetched the files of 3 of 120 collections`; with `--json`,
+fetched, such as `Fetched the files of 3 of 120 collections`, with `--json` too;
-`files` and the two helpers print no such line. stdout holds only the command's
+`backup` does the same while the refresh it starts with runs. stdout holds only
-output. If that round-trip fails, the command prints the error on one line and
+the command's output. If that round-trip fails, the command prints the error on
-exits 1, or 3 when the server no longer accepts the saved session (see "Session
+one line and exits 1, or 3 when the server no longer accepts the saved session
-handling"). An album or file quak cannot decrypt does not fail it; the command
+(see "Session handling"). An album or file quak cannot decrypt does not fail it;
-answers from what did decrypt (see "Default reads vs. fresh reads").
+the command answers from what did decrypt (see "Default reads vs. fresh reads").
 `--cache-dir` overrides where the cache lives; without it each account gets its
 own directory under the per-user cache path.
 
-An ID given to `files --collection`, `get`, `get-thumb` or
+`-v` (`--verbose`) and `-q` (`--quiet`) set how much quak logs on stderr. Like
+`--cache-dir`, they go before or after any command's name. Every line quak
+prints on stderr is a log line at one of six levels, from the quietest: fatal,
+error, warning, info, debug and trace. A run prints the lines at its own level
+and at every quieter one. stdout holds only the command's output, at every
+level.
+
+- Every command that exits non-zero logs a fatal line saying why, such as
+  `quak: <message>`, `Not logged in`, an invalid ID or a collection that does
+  not exist. A command that exits 1 because files or albums failed logs it last,
+  saying how many failed and where to look: `backup` points to `failures.json`
+  and its `FAILED` lines, `backup-metadata` to the `mlDataError` in the JSON of
+  each such file, and `helper fix-missing-thumbnails` to its `Failed files:`
+  list, or to its JSON with `--json`.
+- An error line reports something that failed: the `FAILED` lines that `backup`
+  and `helper fix-missing-thumbnails` log as they work,
+  `ML data of file <id> could not be read` and
+  `ML data request for <n> file(s) failed` from `backup-metadata`,
+  `Could not check <title>` from both thumbnail helpers, and the `Failed files:`
+  list at the end of `backup` and of `helper fix-missing-thumbnails`.
+- A warning reports something that did not fail but needs your attention: after
+  `logout`, that the cache directory still holds decrypted data.
+- Every other line quak logs by default is info.
+
+With `--json`, a command's result on stdout takes the place of its summary on
+stderr: for `backup`, the lines from `--- Backup complete ---` to the end of its
+`Failed files:` list; for `helper list-missing-thumbnails`,
+`No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`; and
+for `helper fix-missing-thumbnails`, the lines from `--- Done ---` to the end of
+its `Failed files:` list. Every other line is logged as it is without `--json`.
+
+- With neither option the level is info: the lines quak prints as it works, such
+  as a refresh's progress, the files a backup fetches, its summary and the files
+  that failed.
+- `-v` raises the level to debug, which adds each retry of a request, with why
+  the attempt failed and how long quak waits before the next, and the start and
+  end of each refresh and each ML data fetch, with the error of one that failed.
+- `-vv` raises it to trace, which adds each request quak sends: its method, its
+  URL and the status of the answer, or the error the request failed with. The
+  URL of an upload of a thumbnail is logged without its query, which holds the
+  upload's signature. More repeats stay at trace, and `--verbose --verbose` is
+  `-vv`.
+- `-q` lowers it to fatal: a run that exits 0 prints nothing on stderr, and one
+  that exits non-zero prints only its fatal lines. The prompts of `login` are
+  not log lines, and `-q` leaves them as they are.
+
+`-q` with `-v` is refused with one line,
+`error: option '-q, --quiet' cannot be used with option '-v, --verbose'`, and
+exit status 1.
+
+`files <collectionID>` lists the files in that collection. `files` with no
+argument lists every file in the account, collection by collection, each file
+once: a file in several collections is listed with the first of them, which is
+its `collectionID` in `--json`.
+
+An ID given to `files`, `get`, `get-thumb` or
 `helper fix-missing-thumbnails --file` is written in decimal digits, no larger
 than 9007199254740991. Anything else, such as `0x10`, `1e3` or `-1`, is refused
 before any request with one line, `Invalid file ID: <arg>` or
@@ -723,7 +792,7 @@ fails, the error is logged, each of its files is written with the reason in an
 `mlDataError` field instead of `mlData`, and the dump goes on. A file whose ML
 data does not decrypt, gunzip or parse is logged and written the same way, with
 why in `mlDataError`. The exit code is non-zero if any file was written with
-`mlDataError`.
+`mlDataError`, and the last line then says how many were.
 
 `helper fix-missing-thumbnails` regenerates thumbnails for JPEG images only,
 because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG image
@@ -820,8 +889,8 @@ reads vs. fresh reads") goes into `failures.json` with an error that starts
 `could not be decrypted:` and gives the reason. If a copy of it in another album
 decrypted, it does not: its original is stored from that copy, and the album
 where it does not decrypt neither links nor lists it. An album the library could
-not decrypt has no name quak can read and no files it can list. A run without
+not decrypt has no name quak can read and no files it can list. `quak backup`
-`--json` prints one line for it,
+prints one line for it on stderr,
 `FAILED album <id>: could not be decrypted: <reason>`, and `--json` and the
 `BackupResult` list its ID in `undecryptableAlbums`. Such a file in
 `failures.json`, or such an album, keeps the exit code at 1, run after run,
@@ -875,7 +944,7 @@ appears or goes away.
 
 A collection whose directory or JSON cannot be written, such as one whose
 directory name is taken by a file you put in `collections/`, does not stop the
-run. A run without `--json` prints `FAILED album <name>: <reason>` for it,
+run. `quak backup` prints `FAILED album <name>: <reason>` for it on stderr,
 `--json` and the `BackupResult` list its ID in `failedAlbums`, and the exit code
 is 1. The other collections and `failures.json` are still written.
 

TODO.md

@@ -17,17 +17,37 @@ pre-1.0
 
 # Next Step
 
-No bug is open. Two changes sneak asked for are open:
+No bug is open, and no change sneak asked for is open. The check of quak against
-https://git.eeqj.de/sneak/quak/issues/211 and
+a live Ente account, https://git.eeqj.de/sneak/quak/issues/187, needs an account
-https://git.eeqj.de/sneak/quak/issues/212. The check of quak against a live Ente
+only sneak has, so it waits on him.
-account, https://git.eeqj.de/sneak/quak/issues/187, needs an account only sneak
-has, so it waits on him.
 
 Tagging and releases are decided by sneak alone, and happen only when he
 declares one.
 
 # Completed Steps
 
+- 2026-10-18: `-v`/`--verbose` raises the CLI's log level once for each repeat,
+  and `-q`/`--quiet` prints only fatal errors
+  (https://git.eeqj.de/sneak/quak/issues/212). The levels are fatal, error,
+  warning, info, debug and trace; the default is info, everything quak printed
+  on stderr before, `-v` gives debug and `-vv` trace. `src/cli-log.ts` holds the
+  level and the log the commands write stderr lines through. Debug adds each
+  retry, which `withRetry` reports through the new `onRetry`, and the start and
+  end of each refresh and ML data fetch; trace adds each request, which
+  `ApiClient` reports through the new `onRequest`. `--json` no longer keeps the
+  progress lines off stderr, so `backup` and `--json` runs log a refresh's
+  progress too. Every line that reports a failed file or album is at error,
+  which the progress callbacks of `backup`, `backup-metadata` and the thumbnail
+  helpers now say, and a command that exits 1 because files or albums failed
+  ends with one fatal line saying how many and where to look.
+
+- 2026-10-18: `quak files` takes the collection ID as an optional argument
+  instead of the `--collection` option, which is gone
+  (https://git.eeqj.de/sneak/quak/issues/211). With no argument it lists every
+  file in the account once, collection by collection; a file in several
+  collections is listed with the first of them, which is its `collectionID` in
+  `--json`. The fresh read in `src/cli-read.ts` does this.
+
 - 2026-10-18: Names quak builds from an album name or a title fit in a file name
   (https://git.eeqj.de/sneak/quak/issues/205). `shortenKeepingExtension` in
   `src/filename.ts` cuts a name short to a number of bytes, keeping its

bin/quak.ts

@@ -2,8 +2,9 @@
 
 import { stdout, stderr } from "node:process";
 import { input, password } from "@inquirer/prompts";
-import { Command } from "commander";
+import { Command, Option } from "commander";
 import envPaths from "env-paths";
+import { logLevel, logRequests, stderrLog, type Log } from "../src/cli-log.js";
 import { init } from "../src/crypto/index.js";
 import {
     type CliContext,
@@ -29,6 +30,9 @@ const paths = envPaths("quak", { suffix: "" });
 
 const program = new Command();
 
+// Program options, so they are taken before or after any command's name.
+// `-v` counts its repeats, the way commander's documentation counts a repeated
+// option.
 program
     .name("quak")
     .description("CLI for the Ente end-to-end encrypted photo service")
@@ -37,21 +41,46 @@ program
         "--cache-dir <path>",
         "Directory for the local metadata/content cache " +
             "(default: the per-user cache directory)",
-    );
+    )
+    .option(
+        "-v, --verbose",
+        "Log more on stderr: debug lines with -v, trace lines too with -vv",
+        (_value: string, previous: number) => previous + 1,
+        0,
+    )
+    .addOption(
+        new Option(
+            "-q, --quiet",
+            "Print nothing on stderr but the error a failing command exits with",
+        ).conflicts("verbose"),
+    )
+    .configureHelp({ showGlobalOptions: true });
+
+interface ProgramOptions {
+    cacheDir?: string;
+    verbose: number;
+    quiet?: boolean;
+}
+
+// The log for this run, at the level `-v` and `-q` ask for.
+const log = (): Log => {
+    const { verbose, quiet } = program.opts<ProgramOptions>();
+    return stderrLog(stderr, logLevel(verbose, quiet ?? false));
+};
 
 const context = (): CliContext => ({
     stdout,
-    stderr,
+    log: log(),
     sessionDir: paths.data,
-    cacheDir: program.opts<{ cacheDir?: string }>().cacheDir,
+    cacheDir: program.opts<ProgramOptions>().cacheDir,
-    loadSession,
+    loadSession: (path) => loadSession(path, logRequests(log())),
-    login: (opts) => Client.login(opts),
+    login: (opts) => Client.login({ ...opts, apiOptions: logRequests(log()) }),
     prompt: (message) => input({ message }),
     promptSecret: (message) => password({ message, mask: true }),
 });
 
 const run = (command: Promise<number>): Promise<void> =>
-    runCommand(command, stdout, stderr, (code) => process.exit(code));
+    runCommand(command, stdout, stderr, log(), (code) => process.exit(code));
 
 program
     .command("login")
@@ -78,14 +107,15 @@ program
 
 program
     .command("files")
-    .description("List files in a collection")
+    .description("List the files in a collection, or every file in the account")
-    .requiredOption(
+    .argument(
-        "--collection <id>",
+        "[collectionID]",
-        "Collection ID (from `quak collections`)",
+        "Collection ID (from `quak collections`); without it, every file " +
+            "is listed once",
     )
     .option("--json", "Output as JSON array")
-    .action((opts: { collection: string; json?: boolean }) =>
+    .action((collectionID: string | undefined, opts: { json?: boolean }) =>
-        run(filesCommand(context(), opts)),
+        run(filesCommand(context(), collectionID, opts)),
     );
 
 program
@@ -142,7 +172,10 @@ program
                 {
                     ...context(),
                     loadSession: (path) =>
-                        loadSession(path, { retry: UNATTENDED_RETRY_OPTIONS }),
+                        loadSession(
+                            path,
+                            logRequests(log(), UNATTENDED_RETRY_OPTIONS),
+                        ),
                 },
                 dir,
                 opts,

src/api/client.ts

@@ -36,6 +36,15 @@ export interface ApiClientOptions {
     retry?: RetryOptions;
     requestTimeoutMs?: number;
     downloadTimeoutMs?: number;
+    // Called once for each request sent, with the status of its response or
+    // the error it failed with, for a caller that logs requests. A request
+    // that is retried is reported once for each attempt.
+    onRequest?: (request: {
+        method: string;
+        url: string;
+        status?: number;
+        error?: unknown;
+    }) => void;
 }
 
 export interface StreamOptions {
@@ -159,6 +168,7 @@ export class ApiClient {
     private readonly retry: ResolvedRetryOptions;
     private readonly requestTimeoutMs: number;
     private readonly downloadTimeoutMs: number;
+    private readonly onRequest: NonNullable<ApiClientOptions["onRequest"]>;
     private token: string | undefined;
 
     constructor(opts?: ApiClientOptions) {
@@ -180,6 +190,7 @@ export class ApiClient {
             opts?.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS;
         this.downloadTimeoutMs =
             opts?.downloadTimeoutMs ?? DEFAULT_DOWNLOAD_TIMEOUT_MS;
+        this.onRequest = opts?.onRequest ?? (() => {});
         this.token = opts?.authToken;
     }
 
@@ -214,6 +225,33 @@ export class ApiClient {
         return h;
     }
 
+    // Send one request and report it to `onRequest`, with the status of its
+    // response or the error it failed with. `reportedURL` is the URL given to
+    // `onRequest`, when that must not be the one sent.
+    private async send(
+        url: string,
+        init: RequestInit & { method: string },
+        reportedURL = url,
+    ): Promise<Response> {
+        let resp: Response;
+        try {
+            resp = await this._fetch(url, init);
+        } catch (err) {
+            this.onRequest({
+                method: init.method,
+                url: reportedURL,
+                error: err,
+            });
+            throw err;
+        }
+        this.onRequest({
+            method: init.method,
+            url: reportedURL,
+            status: resp.status,
+        });
+        return resp;
+    }
+
     private async throwIfError(resp: Response): Promise<void> {
         if (resp.ok) return;
         const requestID = resp.headers.get("x-request-id") ?? undefined;
@@ -257,7 +295,7 @@ export class ApiClient {
         const url = buildURL(this.apiOrigin, path, query);
         // A GET changes nothing, so it is retried under the full policy.
         return withRetry(async () => {
-            const resp = await this._fetch(url, {
+            const resp = await this.send(url, {
                 method: "GET",
                 headers: this.headers(),
                 signal: AbortSignal.timeout(this.requestTimeoutMs),
@@ -280,7 +318,7 @@ export class ApiClient {
         // 3xx status, which is not retried.
         return withRetry(
             async () => {
-                const resp = await this._fetch(url, {
+                const resp = await this.send(url, {
                     method: "POST",
                     headers: this.headers({
                         "Content-Type": "application/json",
@@ -321,16 +359,23 @@ export class ApiClient {
         // object at one key in one request, so replaying it either overwrites
         // the same bytes or lands them for the first time. There is no partial
         // state to protect, hence the full policy rather than the POST rule.
+        // It is reported without its query, which holds the upload's
+        // signature: a log keeps what it is given.
+        const { origin, pathname } = new URL(presignedURL);
         await withRetry(async () => {
-            const resp = await this._fetch(presignedURL, {
+            const resp = await this.send(
-                method: "PUT",
+                presignedURL,
-                headers: {
+                {
-                    "Content-Type": "application/octet-stream",
+                    method: "PUT",
-                    "Content-Length": String(data.length),
+                    headers: {
+                        "Content-Type": "application/octet-stream",
+                        "Content-Length": String(data.length),
+                    },
+                    body: data,
+                    signal: AbortSignal.timeout(this.requestTimeoutMs),
                 },
-                body: data,
+                origin + pathname,
-                signal: AbortSignal.timeout(this.requestTimeoutMs),
+            );
-            });
             if (!resp.ok) {
                 // An ApiError, not a bare Error: without the status on the
                 // error the upload path cannot be classified at all, and a
@@ -348,7 +393,7 @@ export class ApiClient {
         // Same replay and redirect rules as `postJSON`, for the same reasons.
         return withRetry(
             async () => {
-                const resp = await this._fetch(url, {
+                const resp = await this.send(url, {
                     method: "PUT",
                     headers: this.headers({
                         "Content-Type": "application/json",
@@ -394,7 +439,7 @@ export class ApiClient {
             // headers, when no bytes have arrived either.
             const deadline = idleDeadline(this.downloadTimeoutMs);
             try {
-                const resp = await this._fetch(url, {
+                const resp = await this.send(url, {
                     method: "GET",
                     headers: this.headers(),
                     signal: deadline.signal,

src/backup.ts

@@ -101,7 +101,10 @@ import type { MLData } from "./mldata-fetch.js";
 import type { Collection, EnteFile, FileMetadata } from "./model/types.js";
 import { hasTransportCode } from "./retry.js";
 
-export type ProgressCallback = (message: string) => void;
+// Reports one line of the backup's progress. `level` is "error" when the line
+// reports something that failed, such as a file or an album, and is left out
+// for every other line.
+export type ProgressCallback = (message: string, level?: "error") => void;
 
 // The lock in a backup directory, as `lockBackupDirectory` returns it.
 export interface BackupLock {
@@ -654,7 +657,7 @@ const runLockedBackup = async (
     for (const record of lib.undecryptable()) {
         const error = `could not be decrypted: ${record.error}`;
         if (record.fileID === undefined) {
-            log(`FAILED album ${record.collectionID}: ${error}`);
+            log(`FAILED album ${record.collectionID}: ${error}`, "error");
             undecryptableAlbums.push(record.collectionID);
         } else if (scope.has(record.collectionID)) {
             const copies = memberships.get(record.fileID);
@@ -699,6 +702,7 @@ const runLockedBackup = async (
             } catch (err) {
                 log(
                     `FAILED verifying original ${file.metadata.title}: ${describeError(err)}`,
+                    "error",
                 );
                 recordFailure(
                     file.id,
@@ -732,6 +736,7 @@ const runLockedBackup = async (
         } catch (err) {
             log(
                 `FAILED original ${file.metadata.title}: ${describeError(err)}`,
+                "error",
             );
             recordFailure(
                 file.id,
@@ -807,7 +812,7 @@ const runLockedBackup = async (
             await lib.fetchMLData();
         } catch (err) {
             mlDataFailure = err;
-            log(`FAILED ML data: ${describeError(err)}`);
+            log(`FAILED ML data: ${describeError(err)}`, "error");
         }
         for (const file of distinct.values()) {
             if (lock.lost) throw lock.lost;
@@ -868,7 +873,10 @@ const runLockedBackup = async (
             downloadDirectory,
         );
     } catch (err) {
-        log(`FAILED removing old album directories: ${describeError(err)}`);
+        log(
+            `FAILED removing old album directories: ${describeError(err)}`,
+            "error",
+        );
     }
 
     // An album whose folder or JSON cannot be written, such as one whose
@@ -897,6 +905,7 @@ const runLockedBackup = async (
             } catch (err) {
                 log(
                     `FAILED removing old links in ${c.name}: ${describeError(err)}`,
+                    "error",
                 );
             }
 
@@ -915,6 +924,7 @@ const runLockedBackup = async (
                 } catch (err) {
                     log(
                         `FAILED symlink ${c.name}/${linkName}: ${describeError(err)}`,
+                        "error",
                     );
                     recordFailure(
                         link.file.id,
@@ -931,7 +941,7 @@ const runLockedBackup = async (
                 metaFiles,
             );
         } catch (err) {
-            log(`FAILED album ${c.name}: ${describeError(err)}`);
+            log(`FAILED album ${c.name}: ${describeError(err)}`, "error");
             failedAlbums.push(c.id);
         }
     }

src/cli-commands.ts

@@ -4,7 +4,10 @@
 // code; a thrown error is left to the caller. Nothing here calls
 // `process.exit`: `bin/quak.ts` wires these to the command line, and `run` in
 // `cli-run.ts` prints a thrown error as one line and exits once output has
-// drained. Output must stay byte-identical (see `cli-output.ts`).
+// drained. Output must stay byte-identical (see `cli-output.ts`). Each line on
+// stderr goes through `ctx.log` at its level (see `cli-log.ts`). Every command
+// that exits non-zero logs a fatal line saying why; one that exits 1 because
+// files or albums failed logs it last, saying how many and where to look.
 
 import {
     copyFileSync,
@@ -33,6 +36,7 @@ import {
     originalName,
     thumbnailName,
 } from "./cli-output.js";
+import { type Log, logRefresh } from "./cli-log.js";
 import { freshCollections, freshFiles, freshFile } from "./cli-read.js";
 import { withExtension } from "./filename.js";
 import { runMetadataBackup } from "./metadata-backup.js";
@@ -40,7 +44,8 @@ import { listMissingThumbnails, fixMissingThumbnails } from "./thumbnails.js";
 
 export interface CliContext {
     stdout: { write(text: string): unknown };
-    stderr: { write(text: string): unknown };
+    // Writes a line on stderr at a log level, if the run's level prints it.
+    log: Log;
     // Directory holding `session.json`.
     sessionDir: string;
     // The `--cache-dir` global, or undefined to let the library pick its
@@ -82,16 +87,16 @@ const requireSession = (ctx: CliContext): Client | undefined => {
     try {
         client = ctx.loadSession(sessionPath(ctx));
     } catch (err) {
-        ctx.stderr.write(
+        ctx.log("fatal", err instanceof Error ? err.message : String(err));
-            `${err instanceof Error ? err.message : err}\n` +
+        ctx.log(
-                `Run "quak logout" and then "quak login" to replace it.\n`,
+            "fatal",
+            `Run "quak logout" and then "quak login" to replace it.`,
         );
         return undefined;
     }
     if (!client) {
-        ctx.stderr.write(
+        ctx.log("fatal", `Not logged in. Run "quak login" first.`);
-            `Not logged in. Run "quak login" first.\nSession file: ${sessionPath(ctx)}\n`,
+        ctx.log("fatal", `Session file: ${sessionPath(ctx)}`);
-        );
         return undefined;
     }
     return client;
@@ -107,7 +112,7 @@ const parseID = (
 ): number | undefined => {
     const id = Number(text);
     if (/^[0-9]+$/.test(text) && Number.isSafeInteger(id)) return id;
-    ctx.stderr.write(`Invalid ${kind} ID: ${text}\n`);
+    ctx.log("fatal", `Invalid ${kind} ID: ${text}`);
     return undefined;
 };
 
@@ -128,13 +133,8 @@ const readLibraryClient = (client: Client): LibraryClient => ({
 // downloading the whole account — and no refresh starts as it opens or on a
 // timer: each command asks for the one it needs, and `collections` needs none
 // (issue #210). While a refresh runs, each collection whose files it has
-// fetched is counted on stderr, unless `json` is set, so a large account shows
+// fetched is logged, so a large account shows progress rather than nothing.
-// progress rather than nothing.
+const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>
-const openReadLibrary = (
-    ctx: CliContext,
-    client: Client,
-    opts: { json?: boolean } = {},
-): Promise<Library> =>
     Library.open({
         client: readLibraryClient(client),
         cacheDirectory: ctx.cacheDir,
@@ -142,12 +142,7 @@ const openReadLibrary = (
         refreshIntervalSeconds: Infinity,
         precacheThumbnails: false,
         precacheOriginals: false,
-        onProgress: (event) => {
+        onProgress: logRefresh(ctx.log),
-            if (opts.json || event.status !== "progress") return;
-            ctx.stderr.write(
-                `Fetched the files of ${event.fetched} of ${event.total} collections\n`,
-            );
-        },
     });
 
 export const loginCommand = async (ctx: CliContext): Promise<number> => {
@@ -156,7 +151,7 @@ export const loginCommand = async (ctx: CliContext): Promise<number> => {
     const password =
         process.env.QUAK_PASSWORD ?? (await ctx.promptSecret("Password"));
 
-    ctx.stderr.write("Authenticating...\n");
+    ctx.log("info", "Authenticating...");
     try {
         const client = await ctx.login({
             email,
@@ -167,11 +162,12 @@ export const loginCommand = async (ctx: CliContext): Promise<number> => {
 
         saveSession(ctx.sessionDir, client.toJSON());
         const info = client.whoami();
-        ctx.stderr.write(`Logged in as ${info.email} (user ${info.userID})\n`);
+        ctx.log("info", `Logged in as ${info.email} (user ${info.userID})`);
-        ctx.stderr.write(`Session saved to ${sessionPath(ctx)}\n`);
+        ctx.log("info", `Session saved to ${sessionPath(ctx)}`);
     } catch (err) {
-        ctx.stderr.write(
+        ctx.log(
-            `Login failed: ${err instanceof Error ? err.message : err}\n`,
+            "fatal",
+            `Login failed: ${err instanceof Error ? err.message : err}`,
         );
         return 1;
     }
@@ -193,7 +189,7 @@ export const whoamiCommand = async (ctx: CliContext): Promise<number> => {
 export const logoutCommand = async (ctx: CliContext): Promise<number> => {
     const path = sessionPath(ctx);
     if (!existsSync(path)) {
-        ctx.stderr.write("No session found.\n");
+        ctx.log("info", "No session found.");
         return 0;
     }
     await init();
@@ -211,16 +207,15 @@ export const logoutCommand = async (ctx: CliContext): Promise<number> => {
     }
     unlinkSync(path);
     if (failure === undefined) {
-        ctx.stderr.write("Session ended on the server.\n");
+        ctx.log("info", "Session ended on the server.");
     } else {
-        ctx.stderr.write(
+        ctx.log("fatal", `Could not end the session on the server: ${failure}`);
-            `Could not end the session on the server: ${failure}\n`,
-        );
     }
-    ctx.stderr.write("Session deleted.\n");
+    ctx.log("info", "Session deleted.");
     if (cacheDir !== undefined) {
-        ctx.stderr.write(
+        ctx.log(
-            `Cache directory ${cacheDir} still holds decrypted data; delete it to remove that data.\n`,
+            "warning",
+            `Cache directory ${cacheDir} still holds decrypted data; delete it to remove that data.`,
         );
     }
     return failure === undefined ? 0 : 1;
@@ -269,17 +264,23 @@ export const collectionsCommand = async (
     }
 };
 
+// Lists the files of the collection `collectionIDStr` names, or without it every
+// file in the account, each once (issue #211).
 export const filesCommand = async (
     ctx: CliContext,
-    opts: { collection: string; json?: boolean },
+    collectionIDStr: string | undefined,
+    opts: { json?: boolean },
 ): Promise<number> => {
     await init();
     const client = requireSession(ctx);
     if (!client) return 3;
-    const collectionID = parseID(ctx, "collection", opts.collection);
+    const collectionID =
-    if (collectionID === undefined) return 1;
+        collectionIDStr === undefined
+            ? undefined
+            : parseID(ctx, "collection", collectionIDStr);
+    if (collectionIDStr !== undefined && collectionID === undefined) return 1;
 
-    const lib = await openReadLibrary(ctx, client, opts);
+    const lib = await openReadLibrary(ctx, client);
     try {
         // Force a server round-trip and list in enumeration order (issue #36
         // amendment, issue #52). Each file prints from its own decrypted
@@ -288,7 +289,7 @@ export const filesCommand = async (
         // newest-first order.
         const files = await freshFiles(lib, collectionID);
         if (!files) {
-            ctx.stderr.write(`Collection ${collectionID} not found\n`);
+            ctx.log("fatal", `Collection ${collectionID} not found`);
             return 1;
         }
 
@@ -324,7 +325,7 @@ export const getCommand = async (
         // (issue #36 amendment, issue #52).
         const resolved = await freshFile(lib, fileID);
         if (!resolved) {
-            ctx.stderr.write(`File ${fileID} not found\n`);
+            ctx.log("fatal", `File ${fileID} not found`);
             return 1;
         }
         const { photo, file } = resolved;
@@ -335,7 +336,7 @@ export const getCommand = async (
         if (result.videoPath === undefined) {
             const outPath = opts.out ?? originalName(file);
             copyFileSync(result.path, outPath);
-            ctx.stderr.write(`${result.bytes} bytes -> ${outPath}\n`);
+            ctx.log("info", `${result.bytes} bytes -> ${outPath}`);
             return 0;
         }
         // A live photo is written as its image and its video, each named after
@@ -349,17 +350,16 @@ export const getCommand = async (
                 ? originalName(file, videoExt)
                 : withExtension(opts.out, videoExt);
         if (imageOut.toLowerCase() === videoOut.toLowerCase()) {
-            ctx.stderr.write(
+            ctx.log(
-                `File ${fileID} is a live photo, and its video would also be written to ${imageOut}\n`,
+                "fatal",
+                `File ${fileID} is a live photo, and its video would also be written to ${imageOut}`,
             );
             return 1;
         }
         copyFileSync(result.path, imageOut);
         copyFileSync(result.videoPath, videoOut);
-        ctx.stderr.write(
+        ctx.log("info", `${result.bytes} bytes -> ${imageOut}`);
-            `${result.bytes} bytes -> ${imageOut}\n` +
+        ctx.log("info", `${statSync(videoOut).size} bytes -> ${videoOut}`);
-                `${statSync(videoOut).size} bytes -> ${videoOut}\n`,
-        );
         return 0;
     } finally {
         await lib.close();
@@ -383,7 +383,7 @@ export const getThumbCommand = async (
         // (issue #36 amendment, issue #52).
         const resolved = await freshFile(lib, fileID);
         if (!resolved) {
-            ctx.stderr.write(`File ${fileID} not found\n`);
+            ctx.log("fatal", `File ${fileID} not found`);
             return 1;
         }
         const { photo, file } = resolved;
@@ -393,7 +393,7 @@ export const getThumbCommand = async (
         // used (not the projection title) (issue #52).
         const outPath = opts.out ?? thumbnailName(file);
         copyFileSync(result.path, outPath);
-        ctx.stderr.write(`${result.bytes} bytes -> ${outPath}\n`);
+        ctx.log("info", `${result.bytes} bytes -> ${outPath}`);
         return 0;
     } finally {
         await lib.close();
@@ -415,9 +415,14 @@ export const backupMetadataCommand = async (
         await lib.fresh();
         const result = await runMetadataBackup(lib, client, dir, {
             exif: opts.exif || opts.all,
-            onProgress: (msg) => ctx.stderr.write(msg + "\n"),
+            onProgress: (msg, level) => ctx.log(level ?? "info", msg),
         });
-        return result.filesWithMLDataError > 0 ? 1 : 0;
+        if (result.filesWithMLDataError === 0) return 0;
+        ctx.log(
+            "fatal",
+            `quak: the ML data of ${result.filesWithMLDataError} file(s) could not be read; see mlDataError in their JSON in ${dir}`,
+        );
+        return 1;
     } finally {
         await lib.close();
     }
@@ -432,7 +437,7 @@ export const backupCommand = async (
     const client = requireSession(ctx);
     if (!client) return 3;
 
-    ctx.stderr.write("Starting backup...\n");
+    ctx.log("info", "Starting backup...");
     // The lock is taken before the library opens and starts its refresh, so a
     // run refused while another backup of `dir` runs sends no request.
     let lock: BackupLock;
@@ -440,7 +445,7 @@ export const backupCommand = async (
         lock = await lockBackupDirectory(dir);
     } catch (err) {
         if ((err as NodeJS.ErrnoException).code !== "ELOCKED") throw err;
-        ctx.stderr.write(`quak: ${(err as Error).message}\n`);
+        ctx.log("fatal", `quak: ${(err as Error).message}`);
         return 2;
     }
     try {
@@ -455,45 +460,48 @@ export const backupCommand = async (
             refreshIntervalSeconds: Infinity,
             precacheThumbnails: false,
             precacheOriginals: false,
+            onProgress: logRefresh(ctx.log),
         });
         try {
             const result = await lib.backup({
                 downloadDirectory: dir,
                 lock,
                 verify: opts.verify,
-                onProgress: (msg) => {
+                onProgress: (msg, level) => ctx.log(level ?? "info", msg),
-                    if (!opts.json) ctx.stderr.write(msg + "\n");
-                },
             });
 
             if (opts.json) {
                 ctx.stdout.write(JSON.stringify(result, null, 2) + "\n");
             } else {
-                ctx.stderr.write("\n--- Backup complete ---\n");
+                ctx.log("info", "\n--- Backup complete ---");
-                ctx.stderr.write(`  Total files: ${result.totalFiles}\n`);
+                ctx.log("info", `  Total files: ${result.totalFiles}`);
-                ctx.stderr.write(`  Downloaded:  ${result.downloaded}\n`);
+                ctx.log("info", `  Downloaded:  ${result.downloaded}`);
-                ctx.stderr.write(`  Skipped:     ${result.skipped}\n`);
+                ctx.log("info", `  Skipped:     ${result.skipped}`);
                 if (opts.verify) {
-                    ctx.stderr.write(`  Verified:    ${result.verified}\n`);
+                    ctx.log("info", `  Verified:    ${result.verified}`);
-                    ctx.stderr.write(`  Mismatched:  ${result.mismatched}\n`);
+                    ctx.log("info", `  Mismatched:  ${result.mismatched}`);
-                    ctx.stderr.write(`  Unchecked:   ${result.unchecked}\n`);
+                    ctx.log("info", `  Unchecked:   ${result.unchecked}`);
                 }
-                ctx.stderr.write(`  Failed:      ${result.failed}\n`);
+                ctx.log("info", `  Failed:      ${result.failed}`);
                 if (result.errors.length > 0) {
-                    ctx.stderr.write("\nFailed files:\n");
+                    ctx.log("error", "\nFailed files:");
                     for (const e of result.errors) {
-                        ctx.stderr.write(
+                        ctx.log(
-                            `  [${e.collection}] ${e.title} (id ${e.fileID}): ${e.error}\n`,
+                            "error",
+                            `  [${e.collection}] ${e.title} (id ${e.fileID}): ${e.error}`,
                         );
                     }
                 }
             }
 
-            return result.failed > 0 ||
+            const albums =
-                result.undecryptableAlbums.length > 0 ||
+                result.undecryptableAlbums.length + result.failedAlbums.length;
-                result.failedAlbums.length > 0
+            if (result.failed === 0 && albums === 0) return 0;
-                ? 1
+            ctx.log(
-                : 0;
+                "fatal",
+                `quak: ${result.failed} file(s) and ${albums} album(s) failed; see ${join(dir, "failures.json")} and the FAILED lines`,
+            );
+            return 1;
         } finally {
             await lib.close();
         }
@@ -509,23 +517,24 @@ export const listMissingThumbnailsCommand = async (
     await init();
     const client = requireSession(ctx);
     if (!client) return 3;
-    const lib = await openReadLibrary(ctx, client, opts);
+    const lib = await openReadLibrary(ctx, client);
     try {
         // Refresh first so files added since the cache was written are
         // checked; a failed refresh throws.
         await lib.fresh();
-        const missing = await listMissingThumbnails(lib, client, (msg) => {
+        const missing = await listMissingThumbnails(lib, client, (msg, level) =>
-            if (!opts.json) ctx.stderr.write(msg + "\n");
+            ctx.log(level ?? "info", msg),
-        });
+        );
 
         if (opts.json) {
             ctx.stdout.write(JSON.stringify(missing, null, 2) + "\n");
         } else {
             if (missing.length === 0) {
-                ctx.stderr.write("No missing thumbnails found.\n");
+                ctx.log("info", "No missing thumbnails found.");
             } else {
-                ctx.stderr.write(
+                ctx.log(
-                    `\n${missing.length} file(s) with missing thumbnails:\n`,
+                    "info",
+                    `\n${missing.length} file(s) with missing thumbnails:`,
                 );
                 for (const m of missing) {
                     ctx.stdout.write(
@@ -554,33 +563,34 @@ export const fixMissingThumbnailsCommand = async (
         if (fileID === undefined) return 1;
         fileIDs.push(fileID);
     }
-    const lib = await openReadLibrary(ctx, client, opts);
+    const lib = await openReadLibrary(ctx, client);
     try {
         // Refresh first so files added since the cache was written are found;
         // a failed refresh throws.
         await lib.fresh();
         if (fileIDs.length === 0) {
-            ctx.stderr.write("Scanning for missing thumbnails...\n");
+            ctx.log("info", "Scanning for missing thumbnails...");
-            const missing = await listMissingThumbnails(lib, client, (msg) => {
+            const missing = await listMissingThumbnails(
-                if (!opts.json) ctx.stderr.write(msg + "\n");
+                lib,
-            });
+                client,
+                (msg, level) => ctx.log(level ?? "info", msg),
+            );
             fileIDs = missing.map((m) => m.fileID);
             if (fileIDs.length === 0) {
-                ctx.stderr.write("No missing thumbnails found.\n");
+                ctx.log("info", "No missing thumbnails found.");
                 return 0;
             }
-            ctx.stderr.write(`Found ${fileIDs.length} file(s) to fix.\n`);
+            ctx.log("info", `Found ${fileIDs.length} file(s) to fix.`);
         }
 
         const results = await fixMissingThumbnails(
             lib,
             client,
             fileIDs,
-            (msg) => {
+            (msg, level) => ctx.log(level ?? "info", msg),
-                if (!opts.json) ctx.stderr.write(msg + "\n");
-            },
         );
 
+        const failed = results.filter((r) => r.status === "failed").length;
         if (opts.json) {
             ctx.stdout.write(JSON.stringify(results, null, 2) + "\n");
         } else {
@@ -588,30 +598,30 @@ export const fixMissingThumbnailsCommand = async (
             const skipped = results.filter(
                 (r) => r.status === "skipped",
             ).length;
-            const failed = results.filter((r) => r.status === "failed").length;
+            ctx.log("info", `\n--- Done ---`);
-            ctx.stderr.write(`\n--- Done ---\n`);
+            ctx.log("info", `  Fixed:   ${fixed}`);
-            ctx.stderr.write(`  Fixed:   ${fixed}\n`);
+            ctx.log("info", `  Skipped: ${skipped}`);
-            ctx.stderr.write(`  Skipped: ${skipped}\n`);
+            ctx.log("info", `  Failed:  ${failed}`);
-            ctx.stderr.write(`  Failed:  ${failed}\n`);
             if (skipped > 0) {
-                ctx.stderr.write("\nSkipped:\n");
+                ctx.log("info", "\nSkipped:");
                 for (const r of results.filter((r) => r.status === "skipped")) {
-                    ctx.stderr.write(
+                    ctx.log("info", `  ${r.fileID}\t${r.title}\t${r.reason}`);
-                        `  ${r.fileID}\t${r.title}\t${r.reason}\n`,
-                    );
                 }
             }
             if (failed > 0) {
-                ctx.stderr.write("\nFailed files:\n");
+                ctx.log("error", "\nFailed files:");
                 for (const r of results.filter((r) => r.status === "failed")) {
-                    ctx.stderr.write(
+                    ctx.log("error", `  ${r.fileID}\t${r.title}\t${r.reason}`);
-                        `  ${r.fileID}\t${r.title}\t${r.reason}\n`,
-                    );
                 }
             }
         }
 
-        return results.some((r) => r.status === "failed") ? 1 : 0;
+        if (failed === 0) return 0;
+        ctx.log(
+            "fatal",
+            `quak: ${failed} file(s) failed; see the "Failed files:" list, or the JSON with --json`,
+        );
+        return 1;
     } finally {
         await lib.close();
     }

src/cli-log.ts

@@ -0,0 +1,99 @@
+// The CLI's log level, and the one function the commands write stderr lines
+// through.
+//
+// Every line quak prints on stderr is a log line at one of six levels, from
+// the quietest: fatal, error, warning, info, debug, trace. A run prints the
+// lines at its own level and at every quieter one. Its level is info unless
+// `-v` or `-q` asks for another (see "CLI surface" in the README). Every
+// command that exits non-zero logs a fatal line saying why, so even `-q`, which
+// prints only fatal lines, never leaves a non-zero exit unexplained. An error
+// line reports something that failed, such as a file or an album, and a
+// warning something that did not fail but needs the user's attention. stdout
+// carries only a command's output and is not a log.
+//
+// The library itself writes no lines: its layers report through callbacks, and
+// the CLI logs what they report. `logRequests` gives the API client options
+// that log each request and each retry, and `logRefresh` the library's
+// `onProgress` that logs what a refresh and an ML data fetch report.
+
+import type { ApiClientOptions } from "./api/client.js";
+import { describeError } from "./errors.js";
+import type { RefreshEvent } from "./library/index.js";
+import type { RetryOptions } from "./retry.js";
+
+const LEVELS = ["fatal", "error", "warning", "info", "debug", "trace"] as const;
+
+export type LogLevel = (typeof LEVELS)[number];
+
+// Writes `line` on stderr when the run's level prints lines at `level`.
+export type Log = (level: LogLevel, line: string) => void;
+
+// The level asked for by `-v` given `verbose` times, or by `-q`: info without
+// either, debug for `-v`, trace for `-vv` and more, fatal for `-q`.
+// `bin/quak.ts` refuses `-q` and `-v` together.
+export const logLevel = (verbose: number, quiet: boolean): LogLevel => {
+    if (quiet) return "fatal";
+    if (verbose === 0) return "info";
+    return verbose === 1 ? "debug" : "trace";
+};
+
+// The log that writes each line at `level` or quieter to `stderr`, ending it
+// with a newline.
+export const stderrLog =
+    (stderr: { write(text: string): unknown }, level: LogLevel): Log =>
+    (lineLevel, line) => {
+        if (LEVELS.indexOf(lineLevel) <= LEVELS.indexOf(level)) {
+            stderr.write(line + "\n");
+        }
+    };
+
+// API client options that log each request at trace, with the status of its
+// response or the error it failed with, and each retry at debug, with why and
+// how long it waits. `retry` is the retry policy, the default when not given.
+export const logRequests = (
+    log: Log,
+    retry?: RetryOptions,
+): ApiClientOptions => ({
+    retry: {
+        ...retry,
+        onRetry: (err, attempt, delayMs) =>
+            log(
+                "debug",
+                `Attempt ${attempt} failed, retrying in ${Math.round(delayMs)} ms: ${describeError(err)}`,
+            ),
+    },
+    onRequest: ({ method, url, status, error }) =>
+        log(
+            "trace",
+            `${method} ${url}: ${error === undefined ? `HTTP ${status}` : describeError(error)}`,
+        ),
+});
+
+// How a log line names each operation the library reports the start and end
+// of.
+const OPERATION_NAMES: Record<RefreshEvent["operation"], string> = {
+    refresh: "Refresh",
+    fetchMLData: "ML data fetch",
+    precacheThumbnails: "Thumbnail precache",
+    precacheOriginals: "Original precache",
+};
+
+// A library `onProgress` that logs, at info, each collection whose files a
+// refresh has fetched, and at debug the start and end of each refresh and ML
+// data fetch, with the error of one that failed.
+export const logRefresh =
+    (log: Log) =>
+    (event: RefreshEvent): void => {
+        if (event.status === "progress") {
+            log(
+                "info",
+                `Fetched the files of ${event.fetched} of ${event.total} collections`,
+            );
+            return;
+        }
+        const error = event.error === undefined ? "" : `: ${event.error}`;
+        log(
+            "debug",
+            `${OPERATION_NAMES[event.operation]} ${event.status}${error}`,
+        );
+    };

src/cli-read.ts

@@ -1,6 +1,6 @@
 // How the CLI's read commands obtain current data.
 //
-// `collections`, `files --collection`, `get`, and `get-thumb` must answer for
+// `collections`, `files`, `get`, and `get-thumb` must answer for
 // the account's state at the moment the command runs, not for whatever the
 // local cache last happened to hold (owner amendment, issue #36). `files`,
 // `get` and `get-thumb` therefore force a server round-trip through
@@ -14,7 +14,10 @@
 // enumeration order (`listFiles()`), the orders the pre-library CLI printed,
 // rather than the `albums`/`photos` projection's newest-first order, which
 // re-sorts the rows. The field values still come from each record's raw
-// metadata via `cli-output.ts`.
+// metadata via `cli-output.ts`. `files` without a collection ID lists every
+// file in the account once (issue #211): the library keeps a record of a file
+// for each collection that holds it, and only the first collection's record is
+// listed.
 
 import type { Collection, EnteFile } from "./model/types.js";
 import type { Photo, PhotosAPI } from "./library/index.js";
@@ -27,6 +30,7 @@ export interface FreshReadLibrary {
     fresh(): Promise<unknown>;
     freshCollections(): Promise<Collection[]>;
     getCollection(id: number): Collection | undefined;
+    listCollections(): Collection[];
     listFiles(collectionID: number): EnteFile[];
     getFileByID(fileID: number): EnteFile | undefined;
     photos: Pick<PhotosAPI, "byID">;
@@ -40,14 +44,29 @@ export const freshCollections = (
 
 // The files of one collection, current as of a forced refresh, in enumeration
 // order. `undefined` (not an empty list) when the collection does not exist, so
-// the caller can tell "no such collection" from "an empty collection".
+// the caller can tell "no such collection" from "an empty collection". Without
+// a collection ID, every file in the account, each once, as held by the first
+// collection that holds it.
 export const freshFiles = async (
     lib: FreshReadLibrary,
-    collectionID: number,
+    collectionID?: number,
 ): Promise<EnteFile[] | undefined> => {
     await lib.fresh();
-    if (!lib.getCollection(collectionID)) return undefined;
+    if (collectionID !== undefined) {
-    return lib.listFiles(collectionID);
+        return lib.getCollection(collectionID)
+            ? lib.listFiles(collectionID)
+            : undefined;
+    }
+    const listed = new Set<number>();
+    const files: EnteFile[] = [];
+    for (const collection of lib.listCollections()) {
+        for (const file of lib.listFiles(collection.id)) {
+            if (listed.has(file.id)) continue;
+            listed.add(file.id);
+            files.push(file);
+        }
+    }
+    return files;
 };
 
 // One file, current as of a forced refresh, resolved to both its content

src/cli-run.ts

@@ -2,21 +2,23 @@
 
 import type { Writable } from "node:stream";
 import { ApiError } from "./api/client.js";
+import type { Log } from "./cli-log.js";
 import { describeError } from "./errors.js";
 
 // Run a command and exit with its code once stdout/stderr have drained.
 // Exiting before the drain can truncate piped output, and the library can keep
 // the event loop alive after a command returns, so a plain return could hang.
-// An error the command throws is printed as one `quak: MESSAGE` line, without
+// An error the command throws is logged as one fatal `quak: MESSAGE` line,
-// the stack trace, and exits 1. MESSAGE is the error as `describeError` gives
+// without the stack trace, and exits 1. MESSAGE is the error as
-// it, so it carries the reason Node's `fetch` puts on `cause`. A 401 from the
+// `describeError` gives it, so it carries the reason Node's `fetch` puts on
-// server means it no longer accepts the saved session: that prints one line
+// `cause`. A 401 from the server means it no longer accepts the saved session:
-// saying to log in again and exits 3, as a missing or corrupt session file
+// that logs one fatal line saying to log in again and exits 3, as a missing or
-// does.
+// corrupt session file does.
 export const run = async (
     command: Promise<number>,
     stdout: Writable,
     stderr: Writable,
+    log: Log,
     exit: (code: number) => void,
 ): Promise<void> => {
     let code: number;
@@ -24,12 +26,13 @@ export const run = async (
         code = await command;
     } catch (err) {
         if (err instanceof ApiError && err.status === 401) {
-            stderr.write(
+            log(
-                `quak: the saved session is no longer valid; run "quak login"\n`,
+                "fatal",
+                `quak: the saved session is no longer valid; run "quak login"`,
             );
             code = 3;
         } else {
-            stderr.write(`quak: ${describeError(err)}\n`);
+            log("fatal", `quak: ${describeError(err)}`);
             code = 1;
         }
     }

src/filename.ts

@@ -92,7 +92,10 @@ export const shortenKeepingExtension = (
 // short so that the whole fits in a file name, keeping the title's extension
 // whenever it fits. `ext` (".mov"), when given, takes the place of the
 // title's extension before the name is cut short; a live photo's image and
-// video are each named with their own.
+// video are each named with their own. The cut can leave a device name
+// (`CONSOLE.<long extension>` becomes `CON.<long extension>`); that name gets
+// a leading `_`, as `sanitizeFileName` gives one, and is cut a byte shorter to
+// leave room for it.
 export const defaultFileName = (
     title: string,
     fileID: number,
@@ -101,11 +104,8 @@ export const defaultFileName = (
     const { prefix = "", ext } = opts;
     const safe = sanitizeFileName(title, `file-${fileID}`);
     const name = ext === undefined ? safe : withExtension(safe, ext);
-    return (
+    const room = MAX_NAME_BYTES - Buffer.byteLength(prefix);
-        prefix +
+    const cut = prefix + shortenKeepingExtension(name, room);
-        shortenKeepingExtension(
+    if (!RESERVED_DEVICE_NAME.test(cut)) return cut;
-            name,
+    return `_${prefix}${shortenKeepingExtension(name, room - 1)}`;
-            MAX_NAME_BYTES - Buffer.byteLength(prefix),
-        )
-    );
 };

src/metadata-backup.ts

@@ -17,7 +17,10 @@ import {
 } from "./mldata-fetch.js";
 import type { EnteFile } from "./model/types.js";
 
-export type ProgressCallback = (message: string) => void;
+// Reports one line of the dump's progress. `level` is "error" when the line
+// reports ML data that could not be read, and is left out for every other
+// line.
+export type ProgressCallback = (message: string, level?: "error") => void;
 
 export interface MetadataBackupOptions {
     exif?: boolean;
@@ -195,13 +198,17 @@ export const runMetadataBackup = async (
             for (const [id, payload] of payloads) mlDataMap.set(id, payload);
             for (const [id, err] of errors) {
                 const reason = describeError(err);
-                log(`ML data of file ${id} could not be read: ${reason}`);
+                log(
+                    `ML data of file ${id} could not be read: ${reason}`,
+                    "error",
+                );
                 mlDataErrors.set(id, reason);
             }
         } catch (err) {
             const reason = err instanceof Error ? err.message : String(err);
             log(
                 `ML data request for ${batch.length} file(s) failed: ${reason}`,
+                "error",
             );
             for (const id of batch) mlDataErrors.set(id, reason);
         }

src/retry.ts

@@ -24,6 +24,9 @@ export interface RetryOptions {
     sleep?: (ms: number) => Promise<void>;
     // Injected so the jitter is reproducible under test.
     random?: () => number;
+    // Called before each wait with the error that failed attempt number
+    // `attempt` and how long the wait is, for a caller that logs retries.
+    onRetry?: (err: unknown, attempt: number, delayMs: number) => void;
 }
 
 export type ResolvedRetryOptions = Required<RetryOptions>;
@@ -35,6 +38,7 @@ export const DEFAULT_RETRY_OPTIONS: ResolvedRetryOptions = {
     sleep: (ms: number): Promise<void> =>
         new Promise((resolve) => setTimeout(resolve, ms)),
     random: Math.random,
+    onRetry: () => {},
 };
 
 // For a run nobody is watching, such as `quak backup` from cron. A request that
@@ -55,6 +59,7 @@ export const resolveRetryOptions = (
     maxDelayMs: opts?.maxDelayMs ?? DEFAULT_RETRY_OPTIONS.maxDelayMs,
     sleep: opts?.sleep ?? DEFAULT_RETRY_OPTIONS.sleep,
     random: opts?.random ?? DEFAULT_RETRY_OPTIONS.random,
+    onRetry: opts?.onRetry ?? DEFAULT_RETRY_OPTIONS.onRetry,
 });
 
 // Transport failures: the request did not complete, for reasons below HTTP.
@@ -234,7 +239,9 @@ export const withRetry = async <T>(
                 // and its status intact.
                 throw err;
             }
-            await policy.sleep(backoffMs(attempt, policy));
+            const delayMs = backoffMs(attempt, policy);
+            policy.onRetry(err, attempt, delayMs);
+            await policy.sleep(delayMs);
         }
     }
 };

src/thumbnails.ts

@@ -49,7 +49,10 @@ export interface ThumbnailFixResult {
     reason?: string;
 }
 
-export type ProgressCallback = (message: string) => void;
+// Reports one line of progress. `level` is "error" when the line reports a
+// file that could not be checked or fixed, and is left out for every other
+// line.
+export type ProgressCallback = (message: string, level?: "error") => void;
 
 // Enumerate every file the library knows about, newest album first, each file
 // once, and report those whose server-side thumbnail is missing. "Missing" is
@@ -111,6 +114,7 @@ export const listMissingThumbnails = async (
                 } else {
                     log(
                         `[${album.name}] Could not check ${photo.title}: ${err instanceof Error ? err.message : String(err)} (not reported as missing)`,
+                        "error",
                     );
                 }
             }
@@ -359,6 +363,7 @@ export const fixMissingThumbnails = async (
         } catch (err) {
             log(
                 `[${collectionName}] FAILED ${title}: ${err instanceof Error ? err.message : err}`,
+                "error",
             );
             results.push({
                 fileID,

test/api/client.test.ts

@@ -1222,3 +1222,65 @@ describe("ApiClient non-idempotent requests", () => {
         }
     });
 });
+
+// `onRequest` hears of each request the client sends, as its response arrives
+// or as it fails. The CLI logs these with `-vv`. A retried request is reported
+// once for each attempt.
+describe("ApiClient onRequest", () => {
+    it("reports each request with its status, or with the error it failed with", async () => {
+        const refused = new TypeError("fetch failed");
+        const { fetch } = scriptedFetch(
+            refused,
+            textResponse("unavailable", 503),
+            jsonResponse({ collections: [] }),
+            jsonResponse({}),
+            streamResponse(new Uint8Array([1, 2, 3])),
+        );
+        const requests: unknown[] = [];
+        const client = new ApiClient({
+            fetch,
+            retry: noWait,
+            onRequest: (request) => requests.push(request),
+        });
+
+        await client.getJSON("/collections/v2", { sinceTime: 0 });
+        await client.postJSON("/users/logout", {});
+        await readAll(await client.getFileStream(7));
+
+        const collections = "https://api.ente.io/collections/v2?sinceTime=0";
+        expect(requests).toEqual([
+            { method: "GET", url: collections, error: refused },
+            { method: "GET", url: collections, status: 503 },
+            { method: "GET", url: collections, status: 200 },
+            {
+                method: "POST",
+                url: "https://api.ente.io/users/logout",
+                status: 200,
+            },
+            {
+                method: "GET",
+                url: "https://files.ente.io/?fileID=7",
+                status: 200,
+            },
+        ]);
+    });
+
+    it("reports a presigned upload without its query, which holds the upload's signature", async () => {
+        const { fetch } = scriptedFetch(new Response(null, { status: 200 }));
+        const requests: unknown[] = [];
+        const client = new ApiClient({
+            fetch,
+            retry: noWait,
+            onRequest: (request) => requests.push(request),
+        });
+
+        await client.putFile(
+            "https://s3.example/obj?X-Amz-Signature=abc",
+            new Uint8Array([1, 2]),
+        );
+
+        expect(requests).toEqual([
+            { method: "PUT", url: "https://s3.example/obj", status: 200 },
+        ]);
+    });
+});

test/cli/backup-end-to-end.test.ts

@@ -54,6 +54,7 @@ import {
     saveSession,
     type CliContext,
 } from "../../src/cli-commands.js";
+import { stderrLog } from "../../src/cli-log.js";
 import { run } from "../../src/cli-run.js";
 import { loadSession } from "../../src/cli-session.js";
 import { Client } from "../../src/client.js";
@@ -303,9 +304,10 @@ const backup = async (
     const prompted = async (): Promise<never> => {
         throw new Error("quak backup prompted");
     };
+    const log = stderrLog(stderr, "info");
     const ctx: CliContext = {
         stdout,
-        stderr,
+        log,
         sessionDir,
         cacheDir: `${dir}-cache`,
         loadSession: (path) =>
@@ -319,7 +321,7 @@ const backup = async (
         promptSecret: prompted,
     };
     const code = await new Promise<number>((resolve) => {
-        void run(backupCommand(ctx, dir, opts), stdout, stderr, resolve);
+        void run(backupCommand(ctx, dir, opts), stdout, stderr, log, resolve);
     });
     return { code, stderr: printed, stdout: output };
 };
@@ -1104,6 +1106,9 @@ describe("a backup whose lock is lost", () => {
         expect(code).toBe(1);
         expect(stderr).toBe(
             "Starting backup...\n" +
+                "Fetched the files of 1 of 3 collections\n" +
+                "Fetched the files of 2 of 3 collections\n" +
+                "Fetched the files of 3 of 3 collections\n" +
                 "Refreshing library...\n" +
                 "Fetching original beach.jpg (100)...\n" +
                 "Fetching original IMG_0002.HEIC (300)...\n" +

test/cli/backup-undecryptable.test.ts

@@ -42,6 +42,7 @@ import {
     saveSession,
     type CliContext,
 } from "../../src/cli-commands.js";
+import { stderrLog } from "../../src/cli-log.js";
 import { loadSession } from "../../src/cli-session.js";
 import { Client, type ClientSnapshot } from "../../src/client.js";
 import { encryptBlob, init, toBase64 } from "../../src/crypto/index.js";
@@ -304,7 +305,7 @@ const context = (ente: FakeEnte): CliContext => {
     saveSession(join(root, "session"), ente.snapshot());
     return {
         stdout,
-        stderr,
+        log: stderrLog(stderr, "info"),
         sessionDir: join(root, "session"),
         cacheDir: join(root, "cache"),
         loadSession: (path) =>
@@ -370,6 +371,24 @@ describe("quak backup with records it cannot decrypt", () => {
         expect(stderr.text).toContain("  Failed:      2\n");
     });
 
+    // The album is an error line. The last line is fatal and counts the files
+    // and the album, so `-q`, which prints fatal lines only, still says why
+    // the backup exits 1.
+    it("logs the album at error, and ends with one fatal line counting the files and the album", async () => {
+        const ctx = {
+            ...context(brokenAccount()),
+            log: stderrLog(stderr, "error"),
+        };
+        expect(await backupCommand(ctx, backupDir(), {})).toBe(1);
+
+        expect(stderr.text).toMatch(
+            /^FAILED album 2: could not be decrypted: /,
+        );
+        expect(stderr.text.split("\n").at(-2)).toBe(
+            `quak: 2 file(s) and 1 album(s) failed; see ${join(backupDir(), "failures.json")} and the FAILED lines`,
+        );
+    });
+
     it("names each album it cannot decrypt in the --json result", async () => {
         expect(await backup(brokenAccount(), { json: true })).toBe(1);
 
@@ -548,7 +567,7 @@ describe("the read commands with records quak cannot decrypt", () => {
         expect(await collectionsCommand(context(ente), {})).toBe(0);
         expect(stdout.text).toBe("1\talbum\tTrip\n");
 
-        expect(await filesCommand(context(ente), { collection: "1" })).toBe(0);
+        expect(await filesCommand(context(ente), "1", {})).toBe(0);
         expect(stdout.text).toBe("10\timage\tphoto-10.jpg\n");
     });
 });

test/cli/bin.test.ts

@@ -1,6 +1,8 @@
 /**
- * Tests for the retry options `bin/quak.ts` loads the saved session with:
+ * Tests for the retry options `bin/quak.ts` loads the saved session with
- * `quak backup` gets the unattended ones, every other command the default.
+ * (`quak backup` gets the unattended ones, every other command the default),
+ * each with the callbacks that log its requests and retries, and for
+ * `quak files` taking its collection ID as an optional argument.
  *
  * Each test runs `bin/quak.ts`, as the smoke test does, with its session loader
  * replaced by one that records the options it is given and reports no session,
@@ -26,14 +28,19 @@ vi.mock("../../src/cli-session.js", () => ({
 const argv = process.argv;
 const dir = mkdtempSync(join(tmpdir(), "quak-bin-test-"));
 
-// Run `quak <args>` to completion. The "Not logged in" message and the exit
+// Run `quak <args>` to completion and return what it wrote on stderr, which is
-// are swallowed.
+// kept off the terminal. The exit is swallowed.
-const quak = async (...args: string[]): Promise<void> => {
+const quak = async (...args: string[]): Promise<string> => {
-    vi.spyOn(process.stderr, "write").mockImplementation(() => true);
+    let written = "";
+    vi.spyOn(process.stderr, "write").mockImplementation((text) => {
+        written += String(text);
+        return true;
+    });
     vi.spyOn(process, "exit").mockImplementation(() => undefined as never);
     process.argv = ["node", "quak", ...args];
     vi.resetModules();
     await import("../../bin/quak.js");
+    return written;
 };
 
 afterEach(() => {
@@ -59,13 +66,32 @@ describe("bin/quak.ts session loading", () => {
                     attempts,
                     baseDelayMs,
                     maxDelayMs,
+                    onRetry: expect.any(Function),
                 }),
+                onRequest: expect.any(Function),
             },
         ]);
     });
 
-    it("loads the session for another command with the default options", async () => {
+    it("loads the session for another command with the default retry options", async () => {
         await quak("collections");
-        expect(loaded).toEqual([undefined]);
+        expect(loaded).toEqual([
+            {
+                retry: { onRetry: expect.any(Function) },
+                onRequest: expect.any(Function),
+            },
+        ]);
+    });
+});
+
+// The command line takes `files` with or without a collection ID, and the
+// command runs as far as its session check either way (issue #211).
+describe("bin/quak.ts files", () => {
+    it("runs without a collection ID", async () => {
+        expect(await quak("files")).toMatch(/^Not logged in/);
+    });
+
+    it("runs with a collection ID", async () => {
+        expect(await quak("files", "7")).toMatch(/^Not logged in/);
     });
 });

test/cli/commands.test.ts

@@ -51,6 +51,7 @@ import {
     listMissingThumbnailsCommand,
     fixMissingThumbnailsCommand,
 } from "../../src/cli-commands.js";
+import { stderrLog } from "../../src/cli-log.js";
 import { run } from "../../src/cli-run.js";
 import { loadSession } from "../../src/cli-session.js";
 import {
@@ -121,25 +122,31 @@ const FILES: Record<number, EnteFile[]> = {
 // An original is 7 bytes and a thumbnail 3. `failID` makes that file's
 // original fail; `emptyThumbID` makes the server report that file's
 // thumbnail as empty. `withNewFile` adds new.jpg (102) to Vacation, advancing
-// the collection's updationTime as the server does, and `refreshError` makes
+// the collection's updationTime as the server does. `beachInWork` adds
-// listing collections fail with that message.
+// beach.jpg (100) to Work as well, so that one file is in both albums.
+// `refreshError` makes listing collections fail with that message.
 const fakeClient = (
     opts: {
         failID?: number;
         emptyThumbID?: number;
         withNewFile?: boolean;
+        beachInWork?: boolean;
         refreshError?: string;
     } = {},
 ) => {
     const collections = opts.withNewFile
         ? [{ ...COLLECTIONS[0], updationTime: 2 }, COLLECTIONS[1]]
         : COLLECTIONS;
-    const files = opts.withNewFile
+    let files = FILES;
-        ? {
+    if (opts.withNewFile) {
-              ...FILES,
+        files = {
-              1: [...FILES[1], { ...file(102, 1, "new.jpg"), updationTime: 2 }],
+            ...files,
-          }
+            1: [...files[1], { ...file(102, 1, "new.jpg"), updationTime: 2 }],
-        : FILES;
+        };
+    }
+    if (opts.beachInWork) {
+        files = { ...files, 2: [...files[2], file(100, 2, "beach.jpg")] };
+    }
     const source: ContentSource = {
         original: async ({ file: f, destination }) => {
             if (f.id === opts.failID) throw new Error("HTTP 500 from server");
@@ -202,9 +209,11 @@ let root: string;
 let stdout: Output;
 let stderr: Output;
 
+// The commands log at the default level, info, as `quak` without `-v` or `-q`
+// does; the levels have their own tests, in `log.test.ts`.
 const context = (client: Client | null = fakeClient()): CliContext => ({
     stdout,
-    stderr,
+    log: stderrLog(stderr, "info"),
     sessionDir: join(root, "session"),
     cacheDir: join(root, "cache"),
     loadSession: () => client,
@@ -257,7 +266,7 @@ describe("session file", () => {
         const dir = join(root, "backup");
         expect(await whoamiCommand(ctx)).toBe(3);
         expect(await collectionsCommand(ctx, {})).toBe(3);
-        expect(await filesCommand(ctx, { collection: "1" })).toBe(3);
+        expect(await filesCommand(ctx, "1", {})).toBe(3);
         expect(await getCommand(ctx, "100", {})).toBe(3);
         expect(await getThumbCommand(ctx, "100", {})).toBe(3);
         expect(await backupMetadataCommand(ctx, dir, {})).toBe(3);
@@ -504,17 +513,14 @@ describe("collections", () => {
 
 describe("files", () => {
     it("prints one tab-separated line per file", async () => {
-        expect(await filesCommand(context(), { collection: "1" })).toBe(0);
+        expect(await filesCommand(context(), "1", {})).toBe(0);
         expect(stdout.text).toBe(
             "100\timage\tbeach.jpg\n" + "101\timage\tsunset.jpg\n",
         );
     });
 
     it("prints a JSON array with --json", async () => {
-        const code = await filesCommand(context(), {
+        const code = await filesCommand(context(), "2", { json: true });
-            collection: "2",
-            json: true,
-        });
         expect(code).toBe(0);
         expect(JSON.parse(stdout.text)).toEqual([
             {
@@ -528,11 +534,43 @@ describe("files", () => {
     });
 
     it("exits 1 for an unknown collection", async () => {
-        expect(await filesCommand(context(), { collection: "9" })).toBe(1);
+        expect(await filesCommand(context(), "9", {})).toBe(1);
         expect(stderr.text).toBe(
             FETCHED_BOTH_ALBUMS + "Collection 9 not found\n",
         );
     });
+
+    // Without a collection ID it lists every file in the account once (issue
+    // #211). beach.jpg is in Vacation and Work, and Vacation comes first, so it
+    // is listed once, as Vacation's.
+    it("without a collection ID prints every file once", async () => {
+        const ctx = context(fakeClient({ beachInWork: true }));
+        expect(await filesCommand(ctx, undefined, {})).toBe(0);
+        expect(stdout.text).toBe(
+            "100\timage\tbeach.jpg\n" +
+                "101\timage\tsunset.jpg\n" +
+                "200\timage\tdiagram.png\n",
+        );
+        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS);
+    });
+
+    it("without a collection ID gives each file the first album holding it with --json", async () => {
+        const ctx = context(fakeClient({ beachInWork: true }));
+        expect(await filesCommand(ctx, undefined, { json: true })).toBe(0);
+        expect(
+            JSON.parse(stdout.text).map(
+                (row: { id: number; collectionID: number }) => [
+                    row.id,
+                    row.collectionID,
+                ],
+            ),
+        ).toEqual([
+            [100, 1],
+            [101, 1],
+            [200, 2],
+        ]);
+        expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS);
+    });
 });
 
 describe("get and get-thumb", () => {
@@ -875,6 +913,40 @@ describe("backup", () => {
         );
     });
 
+    // A failed file is an error line both where the backup fails it and in
+    // the list at the end. The last line is fatal, so `-q`, which prints
+    // fatal lines only, still says why the backup exits 1.
+    it("logs a failed file at error, and ends with one fatal line counting the failures", async () => {
+        const dir = join(root, "backup");
+        const last = `quak: 1 file(s) and 0 album(s) failed; see ${join(dir, "failures.json")} and the FAILED lines\n`;
+        const ctx = context(fakeClient({ failID: 101 }));
+        expect(
+            await backupCommand(
+                { ...ctx, log: stderrLog(stderr, "error") },
+                dir,
+                {},
+            ),
+        ).toBe(1);
+        expect(stderr.text).toBe(
+            "FAILED original sunset.jpg: HTTP 500 from server\n" +
+                "\nFailed files:\n" +
+                "  [Vacation] sunset.jpg (id 101): HTTP 500 from server\n" +
+                last,
+        );
+
+        stderr.text = "";
+        expect(
+            await backupCommand(
+                { ...ctx, log: stderrLog(stderr, "fatal") },
+                dir,
+                { json: true },
+            ),
+        ).toBe(1);
+        expect(stderr.text).toBe(last);
+    });
+
+    // `--json` shapes stdout only: the log lines stay on stderr, and the JSON
+    // takes the place of the summary.
     it("prints the result as JSON with --json, still exiting 1 on a failure", async () => {
         const ctx = context(fakeClient({ failID: 101 }));
         const code = await backupCommand(ctx, join(root, "backup"), {
@@ -889,7 +961,31 @@ describe("backup", () => {
             failed: 1,
         });
         expect(result.errors[0].fileID).toBe(101);
-        expect(stderr.text).toBe("Starting backup...\n");
+        expect(stderr.text).toContain(
+            "Fetching original sunset.jpg (101)...\n",
+        );
+        expect(stderr.text).not.toContain("Backup complete");
+    });
+
+    // On an empty cache the library's first refresh fetches the files of
+    // every album before the backup can start, which on a large account takes
+    // a while; each album it has fetched the files of is counted on stderr,
+    // with `--json` too.
+    it("counts on stderr each album whose files its first refresh fetched, with --json too", async () => {
+        const start =
+            "Starting backup...\n" +
+            FETCHED_BOTH_ALBUMS +
+            "Refreshing library...\n";
+        for (const json of [false, true]) {
+            stderr.text = "";
+            const dir = mkdtempSync(join(root, "backup-"));
+            const ctx = {
+                ...context(),
+                cacheDir: mkdtempSync(join(root, "cache-")),
+            };
+            expect(await backupCommand(ctx, dir, { json })).toBe(0);
+            expect(stderr.text.slice(0, start.length)).toBe(start);
+        }
     });
 
     it("--verify downloads again an original that does not match its hash, prints the counts, and exits 0", async () => {
@@ -979,6 +1075,7 @@ describe("backup", () => {
                 backupCommand(ctx, dir, {}),
                 new PassThrough(),
                 runStderr,
+                stderrLog(runStderr, "info"),
                 resolve,
             );
         });
@@ -1219,7 +1316,9 @@ describe("helper list-missing-thumbnails", () => {
         expect(stderr.text).toContain("No missing thumbnails found.\n");
     });
 
-    it("prints a JSON array with --json and no progress", async () => {
+    // `--json` shapes stdout only: the progress stays on stderr, and the JSON
+    // takes the place of the count and the list.
+    it("prints a JSON array with --json, and its progress on stderr", async () => {
         const ctx = context(fakeClient({ emptyThumbID: 200 }));
         expect(await listMissingThumbnailsCommand(ctx, { json: true })).toBe(0);
         expect(JSON.parse(stdout.text)).toEqual([
@@ -1230,7 +1329,37 @@ describe("helper list-missing-thumbnails", () => {
                 reason: "empty thumbnail (0 bytes)",
             },
         ]);
-        expect(stderr.text).toBe("");
+        expect(stderr.text).toContain(FETCHED_BOTH_ALBUMS);
+        expect(stderr.text).toContain("[Work] Checking thumbnails...\n");
+        expect(stderr.text).not.toContain("with missing thumbnails");
+    });
+});
+
+describe("helper fix-missing-thumbnails", () => {
+    // File 999 is in no album, so fixing it fails. The list of failed files
+    // is at error, and the last line is fatal, so `-q`, which prints fatal
+    // lines only, still says why the command exits 1.
+    it("lists a failed file at error, and ends with one fatal line counting the failures", async () => {
+        const last = `quak: 1 file(s) failed; see the "Failed files:" list, or the JSON with --json\n`;
+        const ctx = { ...context(), log: stderrLog(stderr, "error") };
+        expect(await fixMissingThumbnailsCommand(ctx, { file: ["999"] })).toBe(
+            1,
+        );
+        expect(stderr.text).toBe(
+            "\nFailed files:\n" +
+                "  999\tunknown\tfile not found in any collection\n" +
+                last,
+        );
+
+        stderr.text = "";
+        const quiet = { ...context(), log: stderrLog(stderr, "fatal") };
+        expect(
+            await fixMissingThumbnailsCommand(quiet, {
+                file: ["999"],
+                json: true,
+            }),
+        ).toBe(1);
+        expect(stderr.text).toBe(last);
     });
 });
 
@@ -1265,7 +1394,7 @@ describe("backup-metadata --exif", () => {
 // collection list alone and stores nothing.)
 describe("backup-metadata and the thumbnail helpers refresh first", () => {
     beforeEach(async () => {
-        expect(await filesCommand(context(), { collection: "1" })).toBe(0);
+        expect(await filesCommand(context(), "1", {})).toBe(0);
         stdout.text = "";
         stderr.text = "";
     });
@@ -1324,10 +1453,10 @@ describe("backup-metadata and the thumbnail helpers refresh first", () => {
     });
 });
 
-// `files --collection`, `get`, `get-thumb` and `helper fix-missing-thumbnails
+// `files`, `get`, `get-thumb` and `helper fix-missing-thumbnails --file` take
-// --file` take an ID only when it is written in decimal digits and is a safe
+// an ID only when it is written in decimal digits and is a safe integer.
-// integer. `Number()` alone reads `0x10` as 16, `1e3` as 1000 and an empty
+// `Number()` alone reads `0x10` as 16, `1e3` as 1000 and an empty string as 0,
-// string as 0, so the command would act on a file the user never named.
+// so the command would act on a file the user never named.
 // Anything else is refused with one line naming it, and exit 1, before any
 // request.
 //
@@ -1381,21 +1510,18 @@ describe("an ID given on the command line", () => {
 
     it("files looks up a decimal collection ID", async () => {
         const ctx = emptyAccount();
-        expect(await filesCommand(ctx, { collection: "1" })).toBe(1);
+        expect(await filesCommand(ctx, "1", {})).toBe(1);
         expect(stderr.text).toBe("Collection 1 not found\n");
         expect(requests).not.toHaveLength(0);
     });
 
-    it.each(NOT_IDS)(
+    it.each(NOT_IDS)("files refuses %j before any request", async (arg) => {
-        "files refuses --collection %j before any request",
+        const ctx = emptyAccount();
-        async (arg) => {
+        expect(await filesCommand(ctx, arg, {})).toBe(1);
-            const ctx = emptyAccount();
+        expect(stderr.text).toBe(`Invalid collection ID: ${arg}\n`);
-            expect(await filesCommand(ctx, { collection: arg })).toBe(1);
+        expect(stdout.text).toBe("");
-            expect(stderr.text).toBe(`Invalid collection ID: ${arg}\n`);
+        expect(requests).toHaveLength(0);
-            expect(stdout.text).toBe("");
+    });
-            expect(requests).toHaveLength(0);
-        },
-    );
 
     it("get looks up a decimal file ID", async () => {
         expect(await getCommand(emptyAccount(), "100", {})).toBe(1);
@@ -1577,14 +1703,12 @@ describe("an account with many albums", () => {
         // It stored nothing that would make a refresh skip an album's files:
         // `files` after it still finds the album's photo.
         stdout.text = "";
-        expect(await filesCommand(ctx, { collection: "7" })).toBe(0);
+        expect(await filesCommand(ctx, "7", {})).toBe(0);
         expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n");
     });
 
     it("files counts on stderr each album whose files its refresh fetched", async () => {
-        expect(await filesCommand(accountContext(), { collection: "7" })).toBe(
+        expect(await filesCommand(accountContext(), "7", {})).toBe(0);
-            0,
-        );
 
         expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n");
         let progress = "";
@@ -1594,15 +1718,12 @@ describe("an account with many albums", () => {
         expect(stderr.text).toBe(progress);
     });
 
-    // With `--json` the refresh's progress is not printed, as `backup` and
+    // `--json` shapes stdout only, so the refresh's progress is still counted
-    // the thumbnail helpers print none of theirs.
+    // on stderr.
-    it("files --json prints no progress", async () => {
+    it("files --json counts on stderr each album whose files its refresh fetched", async () => {
-        expect(
+        expect(await filesCommand(accountContext(), "7", { json: true })).toBe(
-            await filesCommand(accountContext(), {
+            0,
-                collection: "7",
+        );
-                json: true,
-            }),
-        ).toBe(0);
 
         expect(JSON.parse(stdout.text)).toEqual([
             {
@@ -1613,6 +1734,10 @@ describe("an account with many albums", () => {
                 collectionID: 7,
             },
         ]);
-        expect(stderr.text).toBe("");
+        let progress = "";
+        for (let n = 1; n <= ALBUMS; n++) {
+            progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`;
+        }
+        expect(stderr.text).toBe(progress);
     });
 });

test/cli/log.test.ts

@@ -0,0 +1,357 @@
+/**
+ * Tests for the CLI's log levels, set with `-v`/`--verbose` and `-q`/`--quiet`
+ * (`src/cli-log.ts`, issue #212).
+ *
+ * Every line quak prints on stderr is a log line at one of six levels, from
+ * the quietest: fatal, error, warning, info, debug, trace. A run prints the
+ * lines at its own level and at every quieter one. The level is info unless
+ * asked otherwise: `-v` raises it to debug, `-vv` to trace, and more repeats
+ * stay at trace. `-q` lowers it to fatal, so a run prints only the line a
+ * command prints as it fails with a non-zero exit code. stdout holds the
+ * command's output at every level.
+ *
+ * Most tests run `bin/quak.ts`, as the smoke test does, against an account of
+ * one album holding one photo: the saved session is in a temporary directory,
+ * and `fetch` is a fake of the Ente server that serves the album and the photo
+ * encrypted as Ente's clients encrypt them. Each run has a cache directory of
+ * its own, so its refresh fetches the album's files, as on a first run.
+ */
+
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import sodium from "libsodium-wrappers-sumo";
+import {
+    afterAll,
+    afterEach,
+    beforeAll,
+    describe,
+    expect,
+    it,
+    vi,
+} from "vitest";
+
+import { ApiClient } from "../../src/api/client.js";
+import { saveSession } from "../../src/cli-commands.js";
+import { logRefresh, logRequests, type Log } from "../../src/cli-log.js";
+import { init, toBase64 } from "../../src/crypto/index.js";
+import { encryptSecretbox, encryptStreamBlob } from "../ente-server.js";
+
+// `bin/quak.ts` reads the saved session from the data directory env-paths
+// gives it, which here is a temporary directory made before the first run.
+const paths = vi.hoisted(() => ({ data: "" }));
+vi.mock("env-paths", () => ({ default: () => paths }));
+
+const USER_ID = 42;
+const root = mkdtempSync(join(tmpdir(), "quak-log-test-"));
+const argv = process.argv;
+
+class ExitCalled extends Error {}
+
+// Run `quak <args>` to its exit, with a cache of its own, and return its exit
+// code and what it wrote on stdout and on stderr, which are kept off the
+// terminal. `process.exit` is stubbed to throw, so nothing runs after it, as
+// nothing does after a real exit.
+const quak = async (
+    ...args: string[]
+): Promise<{ code: number; stdout: string; stderr: string }> => {
+    let stdout = "";
+    let stderr = "";
+    let code = NaN;
+    vi.spyOn(process.stdout, "write").mockImplementation((text) => {
+        stdout += String(text);
+        return true;
+    });
+    vi.spyOn(process.stderr, "write").mockImplementation((text) => {
+        stderr += String(text);
+        return true;
+    });
+    vi.spyOn(process, "exit").mockImplementation((exitCode) => {
+        code = Number(exitCode ?? 0);
+        throw new ExitCalled();
+    });
+    const cache = mkdtempSync(join(root, "cache-"));
+    process.argv = ["node", "quak", "--cache-dir", cache, ...args];
+    vi.resetModules();
+    await expect(import("../../bin/quak.js")).rejects.toBeInstanceOf(
+        ExitCalled,
+    );
+    return { code, stdout, stderr };
+};
+
+// Album 1, "Trips", of user 42, holding photo 100, "beach.jpg". The album's
+// key is encrypted with the account's master key, the photo's key with the
+// album's, and the photo's metadata with its own key. Only the session and
+// these two records are needed: the tests never log in.
+beforeAll(async () => {
+    await init();
+    await sodium.ready;
+    const masterKey = sodium.crypto_secretbox_keygen();
+    const keyPair = sodium.crypto_box_keypair();
+    paths.data = join(root, "session");
+    saveSession(paths.data, {
+        email: "log@example.com",
+        userID: USER_ID,
+        token: "token",
+        masterKey: toBase64(masterKey),
+        secretKey: toBase64(keyPair.privateKey),
+        publicKey: toBase64(keyPair.publicKey),
+    });
+
+    const albumKey = sodium.crypto_secretbox_keygen();
+    const albumKeyBox = encryptSecretbox(albumKey, masterKey);
+    const name = encryptSecretbox(new TextEncoder().encode("Trips"), albumKey);
+    const album = {
+        id: 1,
+        owner: { id: USER_ID },
+        encryptedKey: toBase64(albumKeyBox.ciphertext),
+        keyDecryptionNonce: toBase64(albumKeyBox.nonce),
+        encryptedName: toBase64(name.ciphertext),
+        nameDecryptionNonce: toBase64(name.nonce),
+        type: "album",
+        updationTime: 1,
+    };
+    const photoKey = sodium.crypto_secretstream_xchacha20poly1305_keygen();
+    const photoKeyBox = encryptSecretbox(photoKey, albumKey);
+    const metadata = encryptStreamBlob(
+        new TextEncoder().encode(
+            JSON.stringify({
+                title: "beach.jpg",
+                fileType: 0,
+                creationTime: 1000,
+                modificationTime: 1000,
+            }),
+        ),
+        photoKey,
+    );
+    const photo = {
+        id: 100,
+        collectionID: 1,
+        ownerID: USER_ID,
+        encryptedKey: toBase64(photoKeyBox.ciphertext),
+        keyDecryptionNonce: toBase64(photoKeyBox.nonce),
+        metadata: {
+            encryptedData: toBase64(metadata.ciphertext),
+            decryptionHeader: toBase64(metadata.header),
+        },
+        // These tests fetch no original and no thumbnail.
+        file: { decryptionHeader: "" },
+        thumbnail: { decryptionHeader: "" },
+        updationTime: 1,
+    };
+
+    const json = (body: unknown): Response =>
+        new Response(JSON.stringify(body), {
+            headers: { "content-type": "application/json" },
+        });
+    vi.stubGlobal("fetch", async (input: string) => {
+        const { pathname } = new URL(input);
+        if (pathname === "/collections/v2") {
+            return json({ collections: [album] });
+        }
+        if (pathname === "/collections/v2/diff") {
+            return json({ diff: [photo], hasMore: false });
+        }
+        return new Response("not found", { status: 404 });
+    });
+});
+
+afterEach(() => {
+    process.argv = argv;
+    vi.restoreAllMocks();
+});
+
+afterAll(() => {
+    vi.unstubAllGlobals();
+    rmSync(root, { recursive: true, force: true });
+});
+
+// What `quak files` writes on stdout at every level.
+const FILES = "100\timage\tbeach.jpg\n";
+
+// What it writes on stderr at each level. Info is the refresh's progress, one
+// line for the one album whose files it fetched. Debug adds the start and end
+// of the refresh, and trace each request with the status of its answer.
+const INFO = "Fetched the files of 1 of 1 collections\n";
+const DEBUG = "Refresh started\n" + INFO + "Refresh done\n";
+const TRACE =
+    "Refresh started\n" +
+    "GET https://api.ente.io/collections/v2?sinceTime=0: HTTP 200\n" +
+    "GET https://api.ente.io/collections/v2/diff?collectionID=1&sinceTime=0: HTTP 200\n" +
+    INFO +
+    "Refresh done\n";
+
+describe("the log level", () => {
+    it("is info by default", async () => {
+        expect(await quak("files")).toEqual({
+            code: 0,
+            stdout: FILES,
+            stderr: INFO,
+        });
+    });
+
+    it("is debug with -v", async () => {
+        expect(await quak("-v", "files")).toEqual({
+            code: 0,
+            stdout: FILES,
+            stderr: DEBUG,
+        });
+    });
+
+    it("is trace with -vv", async () => {
+        expect(await quak("-vv", "files")).toEqual({
+            code: 0,
+            stdout: FILES,
+            stderr: TRACE,
+        });
+    });
+
+    it("stays at trace with -vvv", async () => {
+        expect((await quak("-vvv", "files")).stderr).toBe(TRACE);
+    });
+
+    it("counts --verbose --verbose as -vv", async () => {
+        expect((await quak("--verbose", "--verbose", "files")).stderr).toBe(
+            TRACE,
+        );
+    });
+
+    // Both are options of the program, taken before or after the command's
+    // name, as `--cache-dir` is.
+    it("is set by an option after the command's name too", async () => {
+        expect((await quak("files", "-v")).stderr).toBe(DEBUG);
+        expect((await quak("files", "--json", "-vv")).stderr).toBe(TRACE);
+        expect((await quak("files", "-q")).stderr).toBe("");
+    });
+});
+
+describe("-q", () => {
+    it("prints nothing on stderr when the command works", async () => {
+        expect(await quak("-q", "files")).toEqual({
+            code: 0,
+            stdout: FILES,
+            stderr: "",
+        });
+    });
+
+    // The refresh's progress is left out; the line saying why the command
+    // failed is the fatal one, and it is printed.
+    it("prints only the line a failing command exits with", async () => {
+        expect(await quak("-q", "files", "9")).toEqual({
+            code: 1,
+            stdout: "",
+            stderr: "Collection 9 not found\n",
+        });
+    });
+
+    it("is refused with -v, in one line, before the command runs", async () => {
+        for (const args of [
+            ["-q", "-v", "files"],
+            ["files", "--verbose", "--quiet"],
+        ]) {
+            const { code, stdout, stderr } = await quak(...args);
+            expect(code).toBe(1);
+            expect(stdout).toBe("");
+            expect(stderr).toBe(
+                "error: option '-q, --quiet' cannot be used with option '-v, --verbose'\n",
+            );
+        }
+    });
+});
+
+// `--help` lists both options, and each command's help lists them too, since
+// each command takes them.
+describe("--help", () => {
+    it("lists -v and -q for the program and for each command", async () => {
+        for (const args of [["--help"], ["files", "--help"]]) {
+            const { stdout } = await quak(...args);
+            expect(stdout).toContain("-v, --verbose");
+            expect(stdout).toContain("-q, --quiet");
+        }
+    });
+});
+
+// The library writes no lines itself. Its layers report through callbacks,
+// and the CLI logs what they report.
+describe("what the library reports", () => {
+    const collect = (): { log: Log; lines: string[] } => {
+        const lines: string[] = [];
+        return { log: (level, line) => lines.push(`${level}: ${line}`), lines };
+    };
+
+    // A request answered 503 and retried: each attempt is a request, logged
+    // at trace with its status, and the retry is logged at debug with how
+    // long it waits and why.
+    it("logs each request at trace and each retry at debug", async () => {
+        const { log, lines } = collect();
+        const answers = [
+            new Response("unavailable", { status: 503 }),
+            new Response("{}", {
+                headers: { "content-type": "application/json" },
+            }),
+        ];
+        const client = new ApiClient({
+            ...logRequests(log, {
+                sleep: async () => {},
+                random: () => 0.5,
+            }),
+            fetch: async () => answers.shift()!,
+        });
+
+        await client.getJSON("/collections/v2", { sinceTime: 0 });
+
+        const url = "https://api.ente.io/collections/v2?sinceTime=0";
+        expect(lines).toEqual([
+            `trace: GET ${url}: HTTP 503`,
+            "debug: Attempt 1 failed, retrying in 250 ms: HTTP 503",
+            `trace: GET ${url}: HTTP 200`,
+        ]);
+    });
+
+    it("logs a request that failed with its error", async () => {
+        const { log, lines } = collect();
+        const client = new ApiClient({
+            ...logRequests(log, { attempts: 1 }),
+            fetch: async () => {
+                throw new TypeError("fetch failed", {
+                    cause: new Error("connect ECONNREFUSED 1.2.3.4:443"),
+                });
+            },
+        });
+
+        await expect(client.getJSON("/collections/v2")).rejects.toThrow();
+
+        expect(lines).toEqual([
+            "trace: GET https://api.ente.io/collections/v2: fetch failed: connect ECONNREFUSED 1.2.3.4:443",
+        ]);
+    });
+
+    // Each operation is named in plain words, as the README names it.
+    it("logs a refresh's progress at info, and its start and end at debug", () => {
+        const { log, lines } = collect();
+        const onProgress = logRefresh(log);
+
+        onProgress({ operation: "refresh", status: "started" });
+        onProgress({
+            operation: "refresh",
+            status: "progress",
+            fetched: 1,
+            total: 2,
+        });
+        onProgress({
+            operation: "refresh",
+            status: "failed",
+            error: "HTTP 503",
+        });
+        onProgress({ operation: "fetchMLData", status: "started" });
+        onProgress({ operation: "fetchMLData", status: "done", fetched: 3 });
+
+        expect(lines).toEqual([
+            "debug: Refresh started",
+            "info: Fetched the files of 1 of 2 collections",
+            "debug: Refresh failed: HTTP 503",
+            "debug: ML data fetch started",
+            "debug: ML data fetch done",
+        ]);
+    });
+});

test/cli/metadata-backup.test.ts

@@ -47,6 +47,7 @@ import {
     type MetadataBackupOptions,
 } from "../../src/metadata-backup.js";
 import { backupMetadataCommand } from "../../src/cli-commands.js";
+import { stderrLog, type LogLevel } from "../../src/cli-log.js";
 import {
     buildAccount,
     buildMetaFetch,
@@ -490,9 +491,12 @@ describe("quak backup-metadata", () => {
 });
 
 describe("quak backup-metadata when a file's ML data cannot be read", () => {
-    // Run the CLI command against `fetch`, the mock unless given, and return
+    // Run the CLI command against `fetch`, the mock unless given, logging at
-    // its exit code, stderr and output directory.
+    // `level`, and return its exit code, stderr and output directory.
-    const runCommand = async (fetch = buildMetaFetch(mock)) => {
+    const runCommand = async (
+        fetch = buildMetaFetch(mock),
+        level: LogLevel = "info",
+    ) => {
         const client = await Client.login({
             email: TEST_EMAIL,
             password: TEST_PASSWORD,
@@ -506,7 +510,10 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
         const code = await backupMetadataCommand(
             {
                 stdout: { write: () => true },
-                stderr: { write: (text: string) => (stderr += text) },
+                log: stderrLog(
+                    { write: (text: string) => (stderr += text) },
+                    level,
+                ),
                 sessionDir: testDir,
                 cacheDir: mkdtempSync(join(testDir, "cache-")),
                 loadSession: () => client,
@@ -517,13 +524,16 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
         return { code, stderr, outDir };
     };
 
+    // The failed request is logged at error, so it is the first line printed
+    // at that level.
     it("writes every file, marks the failed batch's files, and exits 1", async () => {
         const { code, stderr, outDir } = await runCommand(
             buildMetaFetch(mock, 200),
+            "error",
         );
 
         expect(code).toBe(1);
-        expect(stderr).toContain("ML data request for 1 file(s) failed");
+        expect(stderr).toMatch(/^ML data request for 1 file\(s\) failed: /);
 
         const ok = JSON.parse(
             readFileSync(
@@ -546,7 +556,9 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
     });
 
     // The server sends ML data for diagram.png encrypted under a key that is
-    // not the file's, so quak cannot decrypt it.
+    // not the file's, so quak cannot decrypt it. That is logged at error, and
+    // the last line is fatal, so `-q`, which prints fatal lines only, still
+    // says why the command exits 1.
     it("writes the reason for ML data that does not decrypt, and exits 1", async () => {
         const otherKey = sodium.crypto_secretstream_xchacha20poly1305_keygen();
         const { header, ciphertext } = encryptBlob(
@@ -564,12 +576,14 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
                     },
                 },
             }),
+            "error",
         );
 
         expect(code).toBe(1);
         const reason = "secretstream chunk authentication failed";
-        expect(stderr).toContain(
+        expect(stderr).toBe(
-            `ML data of file 200 could not be read: ${reason}\n`,
+            `ML data of file 200 could not be read: ${reason}\n` +
+                `quak: the ML data of 1 file(s) could not be read; see mlDataError in their JSON in ${outDir}\n`,
         );
         const failed = JSON.parse(
             readFileSync(

test/cli/read.test.ts

@@ -93,6 +93,10 @@ class FakeLibrary implements FreshReadLibrary {
         return this.collections.find((c) => c.id === id);
     }
 
+    listCollections(): Collection[] {
+        return this.refreshed ? this.collections : [];
+    }
+
     listFiles(collectionID: number): EnteFile[] {
         return this.refreshed
             ? this.files.filter((f) => f.collectionID === collectionID)
@@ -175,6 +179,30 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => {
         ]);
     });
 
+    // Without a collection ID, `files` lists every file in the account once
+    // (issue #211). File 11 is in collections 1 and 2; collection 2 is
+    // enumerated first, so 11 is listed once, with collection 2's record.
+    it("freshFiles without a collection ID lists every file once, from the first collection holding it", async () => {
+        const files = [
+            file(10, 1, 1_700_000_000_000_000),
+            file(11, 1, 1_700_000_000_000_001),
+            file(20, 2, 1_700_000_000_000_002),
+            file(11, 2, 1_700_000_000_000_001),
+        ];
+        const lib = new FakeLibrary(
+            [collection(2, 100), collection(1, 200)],
+            files,
+        );
+
+        const rows = await freshFiles(lib);
+        expect(lib.freshCalls).toBe(1);
+        expect(rows?.map((f) => [f.id, f.collectionID])).toEqual([
+            [20, 2],
+            [11, 2],
+            [10, 1],
+        ]);
+    });
+
     it("freshFiles returns undefined for an unknown collection", async () => {
         const lib = new FakeLibrary([collection(1, 100)], []);
         const rows = await freshFiles(lib, 999);

test/cli/run.test.ts

@@ -6,6 +6,7 @@ import { PassThrough } from "node:stream";
 import { describe, it, expect } from "vitest";
 
 import { ApiError } from "../../src/api/client.js";
+import { stderrLog } from "../../src/cli-log.js";
 import { run } from "../../src/cli-run.js";
 
 // A stream whose written text is kept in `text`; writes finish at once, so
@@ -23,7 +24,13 @@ const runToExit = async (
     const stdout = collector();
     const stderr = collector();
     const code = await new Promise<number>((resolve) => {
-        void run(command, stdout.stream, stderr.stream, resolve);
+        void run(
+            command,
+            stdout.stream,
+            stderr.stream,
+            stderrLog(stderr.stream, "info"),
+            resolve,
+        );
     });
     return { code, stdout: stdout.text(), stderr: stderr.text() };
 };

test/filename/filename.test.ts

@@ -216,4 +216,13 @@ describe("defaultFileName", () => {
             `${"t".repeat(250)}.heic`,
         );
     });
+
+    // Cutting `CONSOLE` or `NULL` short before a long extension leaves `CON`
+    // or `NUL`, so the name gets the `_` `sanitizeFileName` gives a device
+    // name, and the part before the extension gives up a byte for it.
+    it("never returns a Windows device name, even once cut short", () => {
+        const ext = `.${"e".repeat(251)}`;
+        expect(defaultFileName(`CONSOLE${ext}`, 42)).toBe(`_CO${ext}`);
+        expect(defaultFileName(`NULL${ext}`, 42)).toBe(`_NU${ext}`);
+    });
 });

test/retry/retry.test.ts

@@ -555,6 +555,36 @@ describe("withRetry", () => {
         expect(delays).toHaveLength(2);
     });
 
+    it("tells `onRetry` of each attempt it retries, before the wait", async () => {
+        // A caller that logs retries, as the CLI does with `-v`, learns why
+        // each attempt failed and how long the next one is put off. The last
+        // attempt is not retried, so it is not reported.
+        const { sleep, delays } = recordingSleep();
+        const retries: [string, number, number][] = [];
+        let calls = 0;
+        const failure = withRetry(
+            () => {
+                calls++;
+                return Promise.reject(new ApiError(`attempt ${calls}`, 503));
+            },
+            {
+                attempts: 3,
+                sleep,
+                random: () => 0.5,
+                onRetry: (err, attempt, delayMs) => {
+                    retries.push([(err as Error).message, attempt, delayMs]);
+                },
+            },
+        );
+
+        await expect(failure).rejects.toThrow("attempt 3");
+        expect(retries).toEqual([
+            ["attempt 1", 1, 250],
+            ["attempt 2", 2, 500],
+        ]);
+        expect(delays).toEqual([250, 500]);
+    });
+
     it("does not retry at all when attempts is 1", async () => {
         const { sleep, delays } = recordingSleep();
         let calls = 0;
@@ -708,6 +738,7 @@ describe("retry defaults", () => {
         expect(resolved.maxDelayMs).toBe(DEFAULT_RETRY_OPTIONS.maxDelayMs);
         expect(typeof resolved.sleep).toBe("function");
         expect(typeof resolved.random).toBe("function");
+        expect(typeof resolved.onRetry).toBe("function");
     });
 
     it("resolves to the defaults when given nothing", () => {

test/thumbnails/thumbnails.test.ts

@@ -544,11 +544,22 @@ describe("listMissingThumbnails", () => {
         const client = await login(counted.fetch, { ...noWait });
         const lib = await openLib(client);
 
-        const missing = await listMissingThumbnails(lib, client);
+        const errors: string[] = [];
+        const missing = await listMissingThumbnails(
+            lib,
+            client,
+            (msg, level) => {
+                if (level === "error") errors.push(msg);
+            },
+        );
         lib.close();
 
-        // Only the genuinely empty thumbnail is reported.
+        // Only the genuinely empty thumbnail is reported. The file that
+        // could not be checked is reported as an error.
         expect(missing.map((m) => m.fileID)).toEqual([101]);
+        expect(errors).toEqual([
+            expect.stringMatching(/^\[.*\] Could not check file-102\.jpg: /),
+        ]);
         // And the 500 was retried rather than accepted as an answer: four
         // attempts is the library default.
         expect(counted.matched()).toBe(4);
@@ -699,7 +710,15 @@ describe("fixMissingThumbnails", () => {
         const client = await login(buildThumbFetch(fixMock));
         const lib = await openLib(client);
 
-        const results = await fixMissingThumbnails(lib, client, [101, 102]);
+        const errors: string[] = [];
+        const results = await fixMissingThumbnails(
+            lib,
+            client,
+            [101, 102],
+            (msg, level) => {
+                if (level === "error") errors.push(msg);
+            },
+        );
         lib.close();
 
         expect(results.length).toBe(2);
@@ -707,6 +726,10 @@ describe("fixMissingThumbnails", () => {
         const failure = results.find((r) => r.fileID === 102)!;
         expect(success.status).toBe("fixed");
         expect(failure.status).toBe("failed");
+        // The failed file, and only it, is reported as an error.
+        expect(errors).toEqual([
+            expect.stringMatching(/^\[.*\] FAILED file-102\.jpg: /),
+        ]);
     });
 
     it("skips a non-JPEG image and a video as unsupported, not failed (issue #17)", async () => {