Narrow the replay errno set to failures that prove no connection existed

`CONNECT_CODES` drives `isSafeToReplay`, which is the only thing standing
between a transport failure and a replayed `POST /users/two-factor/verify`.
It included `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` on the stated
grounds that those errnos can only be reported before any request byte was
written. That is not true on Linux: an ICMP destination-unreachable
delivered on an already-established connection sets the socket error and
the next read or write returns `EHOSTUNREACH` or `ENETUNREACH`, and a local
interface going down after the request was fully written surfaces as
`ENETDOWN` the same way. In each case the server may already have received
and acted on the request -- exactly the ambiguity the rule exists to
exclude, on the paths that consume a second-factor attempt or register a
thumbnail.

The three are dropped from `CONNECT_CODES` and stay in `TRANSPORT_CODES`,
so they remain retryable for the idempotent calls; only replay eligibility
narrows. What is left -- `ENOTFOUND`, `EAI_AGAIN`, `ECONNREFUSED` -- means
no TCP connection to the server ever existed, so no request byte can have
been transmitted.

The justification is corrected everywhere it was stated: the comment on
`CONNECT_CODES`, the one on `isSafeToReplay`, the `postJSON` call site, the
README's idempotency section and the `client.test.ts` docblock. All of them
now describe what the narrowed set actually establishes rather than
claiming a proof it did not support.

The narrowing is enforced by the suite rather than asserted in a comment:
the three errnos join `ECONNRESET`/`EPIPE`/`ETIMEDOUT` in the
`isSafeToReplay`-returns-false test, with companion `isRetryable` assertions
so a future edit cannot make them non-retryable by accident. Putting the
three back into `CONNECT_CODES` turns that test red (1 failure, verified).

README.md

@@ -305,12 +305,18 @@ only guarded the initial request would leave the same hang one layer down.
 **Non-idempotent requests are not blindly replayed.** `postJSON` and `putJSON`
 reach `/users/srp/create-session`, `/users/two-factor/verify` — which consumes
 one of a small number of second-factor attempts — and `/files/thumbnail`. They
-are retried only on failures that prove no request byte reached the server,
+are retried only on the three failures that establish no TCP connection to the
-which means the connection was never established (`ECONNREFUSED`, `ENOTFOUND`,
+server ever existed, so no request byte can have been transmitted: `ENOTFOUND`
-and the like). A 5xx, a mid-flight reset and a deadline are all left to the
+and `EAI_AGAIN` (name resolution produced no address) and `ECONNREFUSED` (the
-caller, because each of them can happen after the server has already acted.
+peer refused the connection). A 5xx, a mid-flight reset and a deadline are all
-`putFile` is exempt: a presigned PUT stores one whole object at one key in one
+left to the caller, because each of them can happen after the server has already
-request, so replaying it has no partial state to damage.
+acted. The routing errnos `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` are
+excluded for the same reason, despite looking like connect-time failures: on
+Linux an ICMP unreachable arriving mid-flight, or a local interface going down
+after the request was written, delivers them on an already-established socket.
+They stay retryable for the idempotent calls. `putFile` is exempt: a presigned
+PUT stores one whole object at one key in one request, so replaying it has no
+partial state to damage.
 
 A download is retried as a whole — request, stream consumption, and decryption —
 because a socket reset after the response headers have arrived surfaces in the

src/api/client.ts

@@ -227,11 +227,12 @@ export class ApiClient {
         // Idempotency: this reaches `/users/srp/create-session`,
         // `/users/two-factor/verify` and `/users/ott`, all of which change
         // server state — verifying a second factor consumes one of a small
-        // number of attempts. So a POST is replayed only when the failure
+        // number of attempts. So a POST is replayed only on a failure that
-        // proves the request never reached the server, which in practice means
+        // establishes no TCP connection to the server ever existed: DNS
-        // the connection was never established. A 5xx, a mid-flight reset and
+        // produced no address, or the peer refused the connection. A 5xx, a
-        // a timeout are all left to the caller, because each of them can occur
+        // mid-flight reset, a routing errno (which Linux also delivers on an
-        // after the server has already acted.
+        // established socket) and a timeout are all left to the caller,
+        // because each of them can occur after the server has already acted.
         return withRetry(
             async () => {
                 const resp = await this._fetch(url, {

src/retry.ts

@@ -62,17 +62,22 @@ const TRANSPORT_CODES = new Set([
     "ENETDOWN",
 ]);
 
-// The subset of the above that can only happen before any request byte was
+// The subset of the above that can only be reported before a TCP connection
-// written: name resolution failed, or the connection was refused or never
+// exists, and therefore before any request byte could have been written: name
-// routed. See `isSafeToReplay`.
+// resolution produced no address (`ENOTFOUND`, `EAI_AGAIN`) or the peer
-const CONNECT_CODES = new Set([
+// refused the connection with an RST to the SYN (`ECONNREFUSED`).
-    "ENOTFOUND",
+//
-    "EAI_AGAIN",
+// The routing errnos — `EHOSTUNREACH`, `ENETUNREACH`, `ENETDOWN` — are
-    "ECONNREFUSED",
+// deliberately absent even though they look like connect-time failures. On
-    "EHOSTUNREACH",
+// Linux they are also delivered on an already-established socket: an ICMP
-    "ENETUNREACH",
+// destination-unreachable arriving mid-flight sets the socket error and the
-    "ENETDOWN",
+// next read or write returns it, and a local interface going down after the
-]);
+// request was fully written surfaces the same way. In those cases the server
+// may already have received and acted on the request, which is exactly the
+// ambiguity this set exists to exclude. They stay in `TRANSPORT_CODES`, so
+// they remain retryable for idempotent calls; only replay eligibility is
+// narrowed. See `isSafeToReplay`.
+const CONNECT_CODES = new Set(["ENOTFOUND", "EAI_AGAIN", "ECONNREFUSED"]);
 
 // `cause` is an arbitrary user-settable property and nothing prevents it from
 // forming a cycle, so the walk is bounded. Hanging the process would be a
@@ -148,13 +153,16 @@ export const isRetryable = (err: unknown): boolean => {
 // `isRetryable` is the wrong question for a request that changes state.
 // quak's non-idempotent calls are `/users/srp/create-session`,
 // `/users/two-factor/verify` — which consumes one of a small number of 2FA
-// attempts — and `/files/thumbnail`. They are replayed only when the failure
+// attempts — and `/files/thumbnail`. They are replayed only on the failures in
-// proves no request byte reached the server, which means the connection was
+// `CONNECT_CODES`, which establish that no TCP connection to the server ever
-// never established.
+// existed: there was no address to connect to, or the peer refused the
+// connection outright. A request byte cannot have been transmitted, so the
+// server cannot have acted.
 //
 // Everything else is ambiguous. A 5xx proves the server did process the
 // request. A reset or a broken pipe can arrive after it was fully sent and
-// acted on. A deadline says nothing at all about the server's state.
+// acted on. A routing errno can be delivered on an established socket. A
+// deadline says nothing at all about the server's state.
 export const isSafeToReplay = (err: unknown): boolean =>
     isRetryable(err) && causeCodes(err).some((code) => CONNECT_CODES.has(code));
 

test/api/client.test.ts

@@ -824,10 +824,11 @@ describe("ApiClient non-idempotent requests", () => {
      * state: `/users/srp/create-session`, `/users/two-factor/verify` — which
      * consumes one of a small number of 2FA attempts — and `/files/thumbnail`.
      *
-     * They are retried only when the failure proves the request never reached
+     * They are retried only on a failure that establishes no TCP connection to
-     * the server, which in practice means the connection was never
+     * the server ever existed — DNS produced no address, or the peer refused
-     * established. Everything else is ambiguous: a 5xx proves the server did
+     * the connection — so no request byte can have been transmitted.
-     * process the request, and a reset or a timeout can arrive after it did.
+     * Everything else is ambiguous: a 5xx proves the server did process the
+     * request, and a reset or a timeout can arrive after it did.
      * Replaying under that ambiguity can burn a 2FA attempt or register a
      * thumbnail twice, and neither is worth the round trip it saves.
      */

test/retry/retry.test.ts

@@ -282,18 +282,13 @@ describe("isSafeToReplay", () => {
      * quak's non-idempotent calls are `/users/srp/create-session`,
      * `/users/two-factor/verify` (which consumes one of a limited number of
      * 2FA attempts) and `/files/thumbnail`. A blind replay of any of them can
-     * do real damage, so they retry only on failures that prove no request
+     * do real damage, so they retry only on the failures that establish no TCP
-     * byte ever reached the server — which means the connection was never
+     * connection to the server ever existed — DNS produced no address, or the
-     * established.
+     * peer refused the connection — and therefore that no request byte can
+     * have been transmitted.
      */
     it("replays only failures where the connection was never established", () => {
-        for (const code of [
+        for (const code of ["ENOTFOUND", "EAI_AGAIN", "ECONNREFUSED"]) {
-            "ENOTFOUND",
-            "EAI_AGAIN",
-            "ECONNREFUSED",
-            "EHOSTUNREACH",
-            "ENETUNREACH",
-        ]) {
             expect(isSafeToReplay(errnoError(code))).toBe(true);
         }
         // Also when undici has buried it, which is how it actually arrives.
@@ -317,6 +312,22 @@ describe("isSafeToReplay", () => {
         expect(isSafeToReplay(errnoError("ECONNRESET"))).toBe(false);
         expect(isSafeToReplay(errnoError("EPIPE"))).toBe(false);
         expect(isSafeToReplay(errnoError("ETIMEDOUT"))).toBe(false);
+        // The routing errnos look like connect-time failures but are not. On
+        // Linux an ICMP destination-unreachable delivered on an established
+        // connection sets the socket error, and the next read or write returns
+        // `EHOSTUNREACH` or `ENETUNREACH`; a local interface going down after
+        // the request was fully written surfaces as `ENETDOWN` the same way.
+        // In each case the server may already have consumed the request — a
+        // replayed `/users/two-factor/verify` would burn a second attempt.
+        // They remain retryable for the idempotent calls; this asserts only
+        // that they are not replayable.
+        expect(isSafeToReplay(errnoError("EHOSTUNREACH"))).toBe(false);
+        expect(isSafeToReplay(errnoError("ENETUNREACH"))).toBe(false);
+        expect(isSafeToReplay(errnoError("ENETDOWN"))).toBe(false);
+        // ...and that the narrowing did not make them non-retryable.
+        expect(isRetryable(errnoError("EHOSTUNREACH"))).toBe(true);
+        expect(isRetryable(errnoError("ENETUNREACH"))).toBe(true);
+        expect(isRetryable(errnoError("ENETDOWN"))).toBe(true);
         expect(
             isSafeToReplay(new DOMException("timed out", "TimeoutError")),
         ).toBe(false);