A network failure is printed and recorded with its reason (closes #196)

Node's `fetch` rejects every failure below HTTP as `fetch failed` and puts
the reason on `cause`. `describeError` in `src/errors.ts` gives the message
followed by the messages on the cause chain, and for an `AggregateError`,
which Node gives when every address of a host fails, the message of each
address. `run` prints it, and the backup logs and records it. `classify`
reads that text and calls a failure transient when `hasTransportCode`, now
exported by `src/retry.ts`, finds a transport errno on the chain. A failed
ML data fetch keeps its error as the cause. The bounded walk along the
chain moved from `src/retry.ts` to `src/errors.ts`.

A truncated download's message leaves the authentication failure to its
cause.

Model: opus-5-5

README.md

@@ -19,6 +19,10 @@ encrypting and uploading them back to the server.
 
 ## Getting Started
 
+The commands below need git, node 22 or later, and yarn. `make install`, which
+builds the single binary that "Running the backup from cron" uses, also needs
+[nix](https://nixos.org/download).
+
 ```bash
 git clone https://git.eeqj.de/sneak/quak.git
 cd quak
@@ -86,9 +90,12 @@ enumeration/download calls) is exported too and documented under Design below.
 ## Running the backup from cron
 
 `quak backup` never prompts, so cron can run it. `make install` builds quak as a
-single binary and copies it to `~/bin/quak`. Log in once with it, as the user
-the cron job will run as; the session is saved in that user's data directory
-(see "Session handling"):
+single binary and copies it to `~/bin/quak`. Besides what Getting Started needs,
+it needs [nix](https://nixos.org/download), even where bun is already installed:
+`make install` runs `make build-bin`, which bundles the built `dist/` with bun
+run through `nix-shell -p bun`. Log in once with `~/bin/quak`, as the user the
+cron job will run as; the session is saved in that user's data directory (see
+"Session handling"):
 
 ```bash
 ~/bin/quak login
@@ -138,8 +145,12 @@ matters.
 caused it. Node's `fetch` reports every failure below HTTP as just
 `fetch failed` and gives the reason as its cause, so a run that could not reach
 the server prints, for example,
-`quak: fetch failed: connect ECONNREFUSED 1.2.3.4:443`. A file whose download
-failed that way is in `failures.json` with that error, as a transient failure.
+`quak: fetch failed: connect ECONNREFUSED 1.2.3.4:443`. When the server has
+several addresses and none of them could be reached, the line gives the reason
+for each, separated by `, `. A file whose download failed that way is in
+`failures.json` with that error, as a transient failure. So is each file with no
+ML data cached when the ML data fetch failed that way, its error starting
+`ML data: `.
 
 After a `1`, the next run fetches each missing original again, fetches the ML
 data that is not cached, and rebuilds the album links. An album quak could not

TODO.md

@@ -17,12 +17,11 @@ pre-1.0
 
 # Next Step
 
-The backup issues https://git.eeqj.de/sneak/quak/issues/197,
+Three bugs are open: https://git.eeqj.de/sneak/quak/issues/197,
 https://git.eeqj.de/sneak/quak/issues/198 and
-https://git.eeqj.de/sneak/quak/issues/199, and the README issue
-https://git.eeqj.de/sneak/quak/issues/200, are open. The check of quak against a
-live Ente account, https://git.eeqj.de/sneak/quak/issues/187, needs an account
-only sneak has, so it waits on him.
+https://git.eeqj.de/sneak/quak/issues/199. The check of quak against a live Ente
+account, https://git.eeqj.de/sneak/quak/issues/187, needs an account only sneak
+has, so it waits on him.
 
 Tagging and releases are decided by sneak alone, and happen only when he
 declares one.
@@ -32,12 +31,18 @@ declares one.
 - 2026-10-15: A network failure is printed and recorded with its reason, not as
   just `fetch failed` (issue 196). `describeError` in `src/errors.ts` gives an
   error's message followed by the message of each error on its `cause` chain,
-  such as `fetch failed: connect ECONNREFUSED 1.2.3.4:443`. `run` prints that,
+  such as `fetch failed: connect ECONNREFUSED 1.2.3.4:443`, and each address's
+  reason when a host has several and none could be reached. `run` prints that,
   and the backup logs it and records it in `failures.json`, where `classify` now
   calls the failure transient when the chain holds an errno that
-  `hasTransportCode` in `src/retry.ts` recognises. The walk along the chain
-  moved from `src/retry.ts` to `src/errors.ts`, which both use. Tests refuse
-  every connection of a backup, and then only one file's download.
+  `hasTransportCode` in `src/retry.ts` recognises, for a failed ML data fetch
+  too. The walk along the chain moved from `src/retry.ts` to `src/errors.ts`,
+  which both use. Tests refuse every connection of a backup, then only one
+  file's download, and fail the ML data fetch.
+
+- 2026-10-15: The README says that `make install` needs nix (issue 200), in
+  Getting Started and in "Running the backup from cron": `make build-bin` runs
+  bun through `nix-shell -p bun`. The `Makefile` is unchanged.
 
 - 2026-10-14: `lib.backup()`, and so `quak backup`, works on up to the content
   pool's limit of originals at once, 5 by default, instead of one at a time

src/backup.ts

@@ -221,13 +221,13 @@ const isPresent = (path: string): boolean => {
 
 // Best-effort classification for the ledger. Retryable server/network problems
 // are transient; refusals and local filesystem/decrypt errors are permanent;
-// anything else is unknown. The error code and message are inspected, and so
-// is each errno on the error's cause chain, where Node's `fetch` puts it.
+// anything else is unknown. The error code and the error as `describeError`
+// gives it are inspected, and so is each errno on the error's cause chain,
+// where Node's `fetch` puts it.
 const classify = (err: unknown): FailureClass => {
     if (hasTransportCode(err)) return "transient";
     const e = err as NodeJS.ErrnoException;
-    const text =
-        `${e?.code ?? ""} ${err instanceof Error ? err.message : String(err)}`.toLowerCase();
+    const text = `${e?.code ?? ""} ${describeError(err)}`.toLowerCase();
     if (
         /timeout|timed out|econnreset|econnrefused|econnaborted|network|socket|eai_again|throttl|temporarily|429|500|502|503|504/.test(
             text,
@@ -788,11 +788,13 @@ const runLockedBackup = async (
     // instead and is recorded as failed. The next run fetches its ML data
     // again because none is cached.
     if (includeOriginals) {
+        let mlDataFailure: unknown;
         let mlDataError: string | undefined;
         try {
             log("Fetching ML data...");
             await lib.fetchMLData();
         } catch (err) {
+            mlDataFailure = err;
             mlDataError = errorMessage(err);
             log(`FAILED ML data: ${mlDataError}`);
         }
@@ -817,7 +819,7 @@ const runLockedBackup = async (
                     file.id,
                     file.metadata.title,
                     collectionName.get(file.collectionID) ?? "",
-                    new Error(`ML data: ${mlDataError}`),
+                    new Error("ML data", { cause: mlDataFailure }),
                 );
             } else {
                 writeSidecar(path, file, { mlData }, image);

src/download/index.ts

@@ -135,15 +135,17 @@ const streamDecrypt = async (
                     // cannot tell a partial chunk from a corrupt one, so this
                     // is reported as the truncation it almost always is, with
                     // the authentication failure kept as the error's cause.
-                    // Only the pull is guarded: a sink failure on a chunk
-                    // that did authenticate is a disk error, not a
+                    // The message leaves that failure to the cause, which
+                    // `describeError` appends when the error is printed or
+                    // recorded. Only the pull is guarded: a sink failure on a
+                    // chunk that did authenticate is a disk error, not a
                     // truncation.
                     let pulled;
                     try {
                         pulled = pullStreamChunk(state, buffer);
                     } catch (err) {
                         throw new TruncatedStreamError(
-                            `download: stream truncated: response body ended with ${buffer.length} trailing bytes that did not authenticate as a final chunk (transfer stopped mid-chunk, or the data is corrupt)`,
+                            `download: stream truncated: response body ended with ${buffer.length} trailing bytes (transfer stopped mid-chunk, or the data is corrupt)`,
                             { cause: err },
                         );
                     }

src/errors.ts

@@ -75,13 +75,23 @@ export const causeChain = (
 // An error as quak prints it and records it in `failures.json`: its message,
 // then the message of each error on its `cause` chain, joined with ": ". Node's
 // `fetch` rejects with just "fetch failed" and puts the reason on `cause`, so
-// this gives "fetch failed: connect ECONNREFUSED 1.2.3.4:443". A cause whose
-// message is already in the text before it is left out.
+// this gives "fetch failed: connect ECONNREFUSED 1.2.3.4:443". When the host
+// has several addresses and every one fails, the cause is an `AggregateError`
+// with an empty message and one error per address in `errors`, so an
+// `AggregateError` is given as the messages of its errors, joined with ", ".
+// A cause whose message is already in the text before it is left out.
 export const describeError = (err: unknown): string => {
     if (!(err instanceof Error)) return String(err);
     let text = err.message;
     for (const cause of causeChain(err).chain.slice(1)) {
-        const { message } = cause as { message?: unknown };
+        const message =
+            cause instanceof AggregateError
+                ? cause.errors
+                      .map((e: unknown) =>
+                          e instanceof Error ? e.message : String(e),
+                      )
+                      .join(", ")
+                : (cause as { message?: unknown }).message;
         if (typeof message === "string" && !text.includes(message)) {
             text += `: ${message}`;
         }

test/cli/backup-end-to-end.test.ts

@@ -329,16 +329,41 @@ const downloadOf = (input: RequestInfo | URL): number | undefined => {
         : undefined;
 };
 
-// What Node's `fetch` rejects with when the connection is refused, as for
-// every failure below HTTP: a TypeError that says only "fetch failed", with
-// the reason, and its errno, as its cause.
-const refused = (): TypeError =>
-    new TypeError("fetch failed", {
-        cause: Object.assign(new Error("connect ECONNREFUSED 1.2.3.4:443"), {
-            code: "ECONNREFUSED",
-        }),
+// The error Node gives for a connection to `address` that was refused.
+const refusedAt = (address: string): Error =>
+    Object.assign(new Error(`connect ECONNREFUSED ${address}`), {
+        code: "ECONNREFUSED",
     });
 
+// What Node's `fetch` rejects with when the server refuses the connection, as
+// for every failure below HTTP: a TypeError that says only "fetch failed",
+// with the reason, and its errno, as its cause. When the server has several
+// addresses and each one refuses, the cause is an AggregateError with an empty
+// message, one error per address and the errno of the first. `reason` is the
+// error as quak prints it and records it in `failures.json`.
+const REFUSED = [
+    {
+        addresses: "its one address",
+        error: (): TypeError =>
+            new TypeError("fetch failed", { cause: refusedAt("1.2.3.4:443") }),
+        reason: "fetch failed: connect ECONNREFUSED 1.2.3.4:443",
+    },
+    {
+        addresses: "each of its two addresses",
+        error: (): TypeError =>
+            new TypeError("fetch failed", {
+                cause: Object.assign(
+                    new AggregateError([
+                        refusedAt("2001:db8::1:443"),
+                        refusedAt("1.2.3.4:443"),
+                    ]),
+                    { code: "ECONNREFUSED" },
+                ),
+            }),
+        reason: "fetch failed: connect ECONNREFUSED 2001:db8::1:443, connect ECONNREFUSED 1.2.3.4:443",
+    },
+];
+
 // The fake server, pushing onto `downloads` the ID of each file whose original
 // is downloaded.
 const recordingDownloads =
@@ -629,38 +654,90 @@ describe("quak backup", () => {
         expect(readdirSync(dir)).toEqual([]);
     });
 
-    // The server refuses every connection. quak tries each request again,
-    // here without waiting, and then the run stops at the refresh it starts
-    // with. The line it prints gives the reason Node's `fetch` put on the
-    // error's cause, so a cron log tells a refused connection from a name
-    // that did not resolve, and the exit code is 1.
-    it("prints why the refresh could not reach the server, and exits 1", async () => {
-        const dir = mkdtempSync(join(root, "backup-"));
-        const fetch: typeof globalThis.fetch = async () => {
-            throw refused();
-        };
+    // The server refuses every connection, at its one address or at each of
+    // several. quak tries each request again, here without waiting, and then
+    // the run stops at the refresh it starts with. The line it prints gives
+    // the reason Node's `fetch` put on the error's cause, so a cron log tells
+    // a refused connection from a name that did not resolve, and the exit
+    // code is 1.
+    it.each(REFUSED)(
+        "prints why the refresh could not reach the server at $addresses, and exits 1",
+        async ({ error, reason }) => {
+            const dir = mkdtempSync(join(root, "backup-"));
+            const fetch: typeof globalThis.fetch = async () => {
+                throw error();
+            };
 
-        const { code, stderr } = await backup(dir, {}, fetch);
+            const { code, stderr } = await backup(dir, {}, fetch);
 
-        expect(code).toBe(1);
-        expect(stderr).toBe(
-            "Starting backup...\n" +
-                "Refreshing library...\n" +
-                "quak: fetch failed: connect ECONNREFUSED 1.2.3.4:443\n",
-        );
-    });
+            expect(code).toBe(1);
+            expect(stderr).toBe(
+                "Starting backup...\n" +
+                    "Refreshing library...\n" +
+                    `quak: ${reason}\n`,
+            );
+        },
+    );
 
-    // The files CDN refuses every connection for toast.mp4. Once its retries
-    // are spent, the file goes into `failures.json` with the reason, as a
-    // transient failure, since a later run may well reach the CDN, and the
-    // lines the backup prints for it give the same reason. The rest of the
-    // account is backed up, and the exit code is 1.
-    it("records why a download could not reach the server in failures.json, as transient, and exits 1", async () => {
-        let attempts = 0;
+    // The files CDN refuses every connection for toast.mp4, at its one
+    // address or at each of several. Once its retries are spent, the file
+    // goes into `failures.json` with the reason, as a transient failure,
+    // since a later run may well reach the CDN, and the lines the backup
+    // prints for it give the same reason. The rest of the account is backed
+    // up, and the exit code is 1.
+    it.each(REFUSED)(
+        "records why a download could not reach the server at $addresses in failures.json, as transient, and exits 1",
+        async ({ error, reason }) => {
+            let attempts = 0;
+            const fetch: typeof globalThis.fetch = async (input, init) => {
+                if (downloadOf(input) === 200) {
+                    attempts++;
+                    throw error();
+                }
+                return server(input, init);
+            };
+            const dir = mkdtempSync(join(root, "backup-"));
+
+            const { code, stderr } = await backup(dir, {}, fetch);
+
+            expect(code).toBe(1);
+            expect(attempts).toBe(UNATTENDED_RETRY_OPTIONS.attempts);
+            const failures = readJSON(join(dir, "failures.json"));
+            expect(Object.keys(failures.files)).toEqual(["200"]);
+            expect(failures.files["200"]).toMatchObject({
+                fileID: 200,
+                title: "toast.mp4",
+                classification: "transient",
+                attempts: 1,
+                error: reason,
+            });
+            expect(stderr).toContain(`FAILED original toast.mp4: ${reason}\n`);
+            expect(stderr).toContain(
+                `[Friend's Wedding] toast.mp4 (id 200): ${reason}\n`,
+            );
+            expect(readFileSync(join(dir, DAY, "2024-03-01.100.jpg"))).toEqual(
+                Buffer.from(JPEG_WITH_EXIF),
+            );
+            expect(existsSync(join(dir, DAY, "2024-03-01.200.mp4"))).toBe(
+                false,
+            );
+        },
+    );
+
+    // The name of Ente's API server does not resolve when quak fetches the ML
+    // data, after every original has been backed up. No file has ML data
+    // cached yet, so each goes into `failures.json` with the reason, as a
+    // transient failure, as a download that could not reach the server does,
+    // and the exit code is 1.
+    it("records why the ML data fetch could not reach the server in failures.json, as transient, and exits 1", async () => {
         const fetch: typeof globalThis.fetch = async (input, init) => {
-            if (downloadOf(input) === 200) {
-                attempts++;
-                throw refused();
+            if (new URL(String(input)).pathname === "/files/data/fetch") {
+                throw new TypeError("fetch failed", {
+                    cause: Object.assign(
+                        new Error("getaddrinfo ENOTFOUND api.ente.io"),
+                        { code: "ENOTFOUND" },
+                    ),
+                });
             }
             return server(input, init);
         };
@@ -669,25 +746,21 @@ describe("quak backup", () => {
         const { code, stderr } = await backup(dir, {}, fetch);
 
         expect(code).toBe(1);
-        expect(attempts).toBe(UNATTENDED_RETRY_OPTIONS.attempts);
-        const error = "fetch failed: connect ECONNREFUSED 1.2.3.4:443";
+        const error =
+            "ML data: fetch failed: getaddrinfo ENOTFOUND api.ente.io";
+        expect(stderr).toContain(`FAILED ${error}\n`);
         const failures = readJSON(join(dir, "failures.json"));
-        expect(Object.keys(failures.files)).toEqual(["200"]);
-        expect(failures.files["200"]).toMatchObject({
-            fileID: 200,
-            title: "toast.mp4",
-            classification: "transient",
-            attempts: 1,
-            error,
-        });
-        expect(stderr).toContain(`FAILED original toast.mp4: ${error}\n`);
-        expect(stderr).toContain(
-            `[Friend's Wedding] toast.mp4 (id 200): ${error}\n`,
-        );
-        expect(readFileSync(join(dir, DAY, "2024-03-01.100.jpg"))).toEqual(
-            Buffer.from(JPEG_WITH_EXIF),
-        );
-        expect(existsSync(join(dir, DAY, "2024-03-01.200.mp4"))).toBe(false);
+        expect(
+            Object.values(failures.files).map((f) => [
+                f.fileID,
+                f.classification,
+                f.error,
+            ]),
+        ).toEqual([
+            [100, "transient", error],
+            [200, "transient", error],
+            [300, "transient", error],
+        ]);
     });
 
     // Something changed beach.jpg on disk after it was backed up. A run with

test/cli/run.test.ts

@@ -67,6 +67,31 @@ describe("run", () => {
         });
     });
 
+    // When the host has several addresses and every one refuses, Node's
+    // `fetch` gives as the cause an AggregateError with an empty message and
+    // one error per address. The printed line gives each address's reason.
+    it("prints the reason for each address a fetch could not reach", async () => {
+        const refusedAt = (address: string): Error =>
+            Object.assign(new Error(`connect ECONNREFUSED ${address}`), {
+                code: "ECONNREFUSED",
+            });
+        const refused = new TypeError("fetch failed", {
+            cause: Object.assign(
+                new AggregateError([
+                    refusedAt("2001:db8::1:443"),
+                    refusedAt("1.2.3.4:443"),
+                ]),
+                { code: "ECONNREFUSED" },
+            ),
+        });
+        const result = await runToExit(Promise.reject(refused));
+        expect(result).toEqual({
+            code: 1,
+            stdout: "",
+            stderr: "quak: fetch failed: connect ECONNREFUSED 2001:db8::1:443, connect ECONNREFUSED 1.2.3.4:443\n",
+        });
+    });
+
     // An error that already quotes its cause's message is printed as it is.
     it("does not print a cause's message twice", async () => {
         const reset = new Error("read ECONNRESET");

test/download/download.test.ts

@@ -74,7 +74,11 @@ import {
 } from "vitest";
 import { init, toBase64, STREAM_CHUNK_SIZE } from "../../src/crypto/index.js";
 import { ApiClient } from "../../src/api/client.js";
-import { ApiError, TruncatedStreamError } from "../../src/errors.js";
+import {
+    ApiError,
+    describeError,
+    TruncatedStreamError,
+} from "../../src/errors.js";
 import type { RetryOptions } from "../../src/retry.js";
 import {
     downloadFile,
@@ -834,6 +838,11 @@ describe.each(entryPoints)(
             expect(((err as Error).cause as Error).message).toMatch(
                 /authentication failed/i,
             );
+            // As quak prints it and records it in `failures.json`, the error
+            // states the authentication failure once, from its cause.
+            expect(describeError(err)).toMatch(
+                /^download: stream truncated: response body ended with \d+ trailing bytes \(transfer stopped mid-chunk, or the data is corrupt\): secretstream chunk authentication failed$/,
+            );
 
             // And, as with every other failure, the destination is untouched
             // and no staged temp file survives.