The binary make install builds retries a refused or unresolvable connection (closes #206)
check / check (push) Waiting to run
check / check (push) Waiting to run
Bun's `fetch`, which `~/bin/quak` runs under, rejects with a plain `Error` that has no `cause` and a code of bun's own. Running the built binary against local listeners gave `ConnectionRefused` for a refused connection and for a failed TLS handshake, and `ConnectionRefused` or `FailedToOpenSocket` for a name that does not resolve. Both codes join `TRANSPORT_CODES` and `CONNECT_CODES` in `src/retry.ts`, so they are retried, replayed, and called transient by `classify`. A reset already came as `ECONNRESET` and a deadline as a `TimeoutError`; the tests pin every shape found. Judgement call: a failed TLS handshake is replayed, as no request byte was sent. Not changed: a certificate bun rejects is not retried. Model: opus-5-5
This commit is contained in:
@@ -151,7 +151,9 @@ for each, separated by `, `. A file whose download failed that way is in
|
|||||||
`failures.json` with that error, as a transient failure. So is each file with no
|
`failures.json` with that error, as a transient failure. So is each file with no
|
||||||
ML data cached when the ML data fetch failed that way, its error starting
|
ML data cached when the ML data fetch failed that way, its error starting
|
||||||
`ML data: `. The `~/bin/quak` that `make install` builds runs under bun, and
|
`ML data: `. The `~/bin/quak` that `make install` builds runs under bun, and
|
||||||
prints bun's own wording for such a failure.
|
prints bun's own wording for such a failure, with no reason after it, such as
|
||||||
|
`quak: Unable to connect. Is the computer able to access the url?` for a refused
|
||||||
|
connection. It records such a failure in `failures.json` as transient too.
|
||||||
|
|
||||||
After a `1`, the next run fetches each missing original again, fetches the ML
|
After a `1`, the next run fetches each missing original again, fetches the ML
|
||||||
data that is not cached, and rebuilds the album links. An album quak could not
|
data that is not cached, and rebuilds the album links. An album quak could not
|
||||||
@@ -494,6 +496,17 @@ request is repeated only when repeating it could produce a different answer:
|
|||||||
declining to retry a transient one costs a single file that the next run picks
|
declining to retry a transient one costs a single file that the next run picks
|
||||||
up.
|
up.
|
||||||
|
|
||||||
|
The `~/bin/quak` that `make install` builds runs under bun, whose `fetch`
|
||||||
|
rejects with a plain `Error` that has no `cause` and carries a code of bun's
|
||||||
|
own. quak retries a refused connection, a name that does not resolve, a reset
|
||||||
|
connection and a deadline under bun as it does under Node. Bun reports a refused
|
||||||
|
connection, and a TLS handshake that failed, as `ConnectionRefused`. For a name
|
||||||
|
that does not resolve it gives `ConnectionRefused` on some requests and
|
||||||
|
`FailedToOpenSocket` on others. A connection reset or closed before the response
|
||||||
|
ended is `ECONNRESET`, as under Node, and a deadline is the same `TimeoutError`.
|
||||||
|
A certificate bun rejects, reported with a code such as
|
||||||
|
`DEPTH_ZERO_SELF_SIGNED_CERT`, is not retried.
|
||||||
|
|
||||||
Backoff is exponential with full jitter: the delay before retry _n_ is
|
Backoff is exponential with full jitter: the delay before retry _n_ is
|
||||||
`random() * min(maxDelayMs, baseDelayMs * 2 ** (n - 1))`. The exponential term
|
`random() * min(maxDelayMs, baseDelayMs * 2 ** (n - 1))`. The exponential term
|
||||||
is the ceiling and the wait is drawn below it, so a client that lost many
|
is the ceiling and the wait is drawn below it, so a client that lost many
|
||||||
@@ -541,17 +554,20 @@ second-factor attempts. They are retried only when every errno in the error's
|
|||||||
`cause` chain is one of the three that establish no TCP connection to the server
|
`cause` chain is one of the three that establish no TCP connection to the server
|
||||||
ever existed, so no request byte can have been transmitted: `ENOTFOUND` and
|
ever existed, so no request byte can have been transmitted: `ENOTFOUND` and
|
||||||
`EAI_AGAIN` (name resolution produced no address) and `ECONNREFUSED` (the peer
|
`EAI_AGAIN` (name resolution produced no address) and `ECONNREFUSED` (the peer
|
||||||
refused the connection). A 5xx, a mid-flight reset and a deadline are all left
|
refused the connection). Under bun these failures are `ConnectionRefused` and
|
||||||
to the caller, because each of them can happen after the server has already
|
`FailedToOpenSocket`. Bun also gives `ConnectionRefused` for a TLS handshake
|
||||||
acted. These two do not follow redirects either: a redirect means the server
|
that failed; the TCP connection existed then, but the request is only sent once
|
||||||
already received the request, so it is reported as an error and not retried. The
|
the handshake is done. A 5xx, a mid-flight reset and a deadline are all left to
|
||||||
routing errnos `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` are excluded for the
|
the caller, because each of them can happen after the server has already acted.
|
||||||
same reason, despite looking like connect-time failures: on Linux an ICMP
|
These two do not follow redirects either: a redirect means the server already
|
||||||
unreachable arriving mid-flight, or a local interface going down after the
|
received the request, so it is reported as an error and not retried. The routing
|
||||||
request was written, delivers them on an already-established socket. They stay
|
errnos `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` are excluded for the same
|
||||||
retryable for the idempotent calls. `putFile` is exempt: a presigned PUT stores
|
reason, despite looking like connect-time failures: on Linux an ICMP unreachable
|
||||||
one whole object at one key in one request, so replaying it has no partial state
|
arriving mid-flight, or a local interface going down after the request was
|
||||||
to damage.
|
written, delivers them on an already-established socket. They stay retryable for
|
||||||
|
the idempotent calls. `putFile` is exempt: a presigned PUT stores one whole
|
||||||
|
object at one key in one request, so replaying it has no partial state to
|
||||||
|
damage.
|
||||||
|
|
||||||
A download is retried as a whole — request, stream consumption, and decryption —
|
A download is retried as a whole — request, stream consumption, and decryption —
|
||||||
because a socket reset after the response headers have arrived surfaces in the
|
because a socket reset after the response headers have arrived surfaces in the
|
||||||
|
|||||||
@@ -17,11 +17,10 @@ pre-1.0
|
|||||||
|
|
||||||
# Next Step
|
# Next Step
|
||||||
|
|
||||||
Five bugs are open: https://git.eeqj.de/sneak/quak/issues/197,
|
Four bugs are open: https://git.eeqj.de/sneak/quak/issues/197,
|
||||||
https://git.eeqj.de/sneak/quak/issues/198,
|
https://git.eeqj.de/sneak/quak/issues/198,
|
||||||
https://git.eeqj.de/sneak/quak/issues/199,
|
https://git.eeqj.de/sneak/quak/issues/199 and
|
||||||
https://git.eeqj.de/sneak/quak/issues/205 and
|
https://git.eeqj.de/sneak/quak/issues/205. The check of quak against a live Ente
|
||||||
https://git.eeqj.de/sneak/quak/issues/206. The check of quak against a live Ente
|
|
||||||
account, https://git.eeqj.de/sneak/quak/issues/187, needs an account only sneak
|
account, https://git.eeqj.de/sneak/quak/issues/187, needs an account only sneak
|
||||||
has, so it waits on him.
|
has, so it waits on him.
|
||||||
|
|
||||||
@@ -30,6 +29,14 @@ declares one.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-16: The `~/bin/quak` that `make install` builds retries a refused
|
||||||
|
connection and a name that does not resolve, and the backup records either as
|
||||||
|
transient (issue 206). Bun's `fetch` reports them with codes of its own,
|
||||||
|
`ConnectionRefused` and `FailedToOpenSocket`, which `src/retry.ts` now counts
|
||||||
|
as transport failures that are safe to replay. Bun already gave `ECONNRESET`
|
||||||
|
for a reset and a `TimeoutError` for a deadline. The shapes were found by
|
||||||
|
running the binary against local listeners.
|
||||||
|
|
||||||
- 2026-10-15: A network failure is printed and recorded with its reason, not as
|
- 2026-10-15: A network failure is printed and recorded with its reason, not as
|
||||||
just `fetch failed` (issue 196). `describeError` in `src/errors.ts` gives an
|
just `fetch failed` (issue 196). `describeError` in `src/errors.ts` gives an
|
||||||
error's message followed by the message of each error on its `cause` chain,
|
error's message followed by the message of each error on its `cause` chain,
|
||||||
|
|||||||
+36
-15
@@ -58,6 +58,13 @@ export const resolveRetryOptions = (
|
|||||||
});
|
});
|
||||||
|
|
||||||
// Transport failures: the request did not complete, for reasons below HTTP.
|
// Transport failures: the request did not complete, for reasons below HTTP.
|
||||||
|
//
|
||||||
|
// The errnos are Node's. The binary `make install` builds runs under bun,
|
||||||
|
// whose `fetch` rejects with a plain `Error` that has no `cause` and a `code`
|
||||||
|
// of bun's own; `CONNECT_CODES` says what bun's two codes mean. For a
|
||||||
|
// connection reset or closed before the response ended, bun gives Node's
|
||||||
|
// `ECONNRESET`. These were found by running that binary, built with bun
|
||||||
|
// 1.3.13, against listeners on 127.0.0.1.
|
||||||
const TRANSPORT_CODES = new Set([
|
const TRANSPORT_CODES = new Set([
|
||||||
"ECONNRESET",
|
"ECONNRESET",
|
||||||
"ECONNABORTED",
|
"ECONNABORTED",
|
||||||
@@ -70,12 +77,19 @@ const TRANSPORT_CODES = new Set([
|
|||||||
"ENETUNREACH",
|
"ENETUNREACH",
|
||||||
"ENETRESET",
|
"ENETRESET",
|
||||||
"ENETDOWN",
|
"ENETDOWN",
|
||||||
|
// Bun's.
|
||||||
|
"ConnectionRefused",
|
||||||
|
"FailedToOpenSocket",
|
||||||
]);
|
]);
|
||||||
|
|
||||||
// The subset of the above that can only be reported before a TCP connection
|
// The subset of the above that can only be reported before any request byte
|
||||||
// exists, and therefore before any request byte could have been written: name
|
// could have been written. Node reports these before a TCP connection exists:
|
||||||
// resolution produced no address (`ENOTFOUND`, `EAI_AGAIN`) or the peer
|
// name resolution produced no address (`ENOTFOUND`, `EAI_AGAIN`) or the peer
|
||||||
// refused the connection with an RST to the SYN (`ECONNREFUSED`).
|
// refused the connection with an RST to the SYN (`ECONNREFUSED`). Bun gives
|
||||||
|
// `ConnectionRefused` for a refused connection, and `ConnectionRefused` or
|
||||||
|
// `FailedToOpenSocket` for a name that does not resolve. It also gives
|
||||||
|
// `ConnectionRefused` for a TLS handshake that failed, after the TCP
|
||||||
|
// connection exists, but the request is only sent once the handshake is done.
|
||||||
//
|
//
|
||||||
// The routing errnos — `EHOSTUNREACH`, `ENETUNREACH`, `ENETDOWN` — are
|
// The routing errnos — `EHOSTUNREACH`, `ENETUNREACH`, `ENETDOWN` — are
|
||||||
// deliberately absent even though they look like connect-time failures. On
|
// deliberately absent even though they look like connect-time failures. On
|
||||||
@@ -87,7 +101,13 @@ const TRANSPORT_CODES = new Set([
|
|||||||
// ambiguity this set exists to exclude. They stay in `TRANSPORT_CODES`, so
|
// ambiguity this set exists to exclude. They stay in `TRANSPORT_CODES`, so
|
||||||
// they remain retryable for idempotent calls; only replay eligibility is
|
// they remain retryable for idempotent calls; only replay eligibility is
|
||||||
// narrowed. See `isSafeToReplay`.
|
// narrowed. See `isSafeToReplay`.
|
||||||
const CONNECT_CODES = new Set(["ENOTFOUND", "EAI_AGAIN", "ECONNREFUSED"]);
|
const CONNECT_CODES = new Set([
|
||||||
|
"ENOTFOUND",
|
||||||
|
"EAI_AGAIN",
|
||||||
|
"ECONNREFUSED",
|
||||||
|
"ConnectionRefused",
|
||||||
|
"FailedToOpenSocket",
|
||||||
|
]);
|
||||||
|
|
||||||
// Collect every `code` in an error's cause chain. undici does not put the
|
// Collect every `code` in an error's cause chain. undici does not put the
|
||||||
// errno on the error it throws — it hangs the underlying socket error off
|
// errno on the error it throws — it hangs the underlying socket error off
|
||||||
@@ -105,7 +125,7 @@ const causeCodes = (err: unknown): { codes: string[]; complete: boolean } => {
|
|||||||
return { codes, complete };
|
return { codes, complete };
|
||||||
};
|
};
|
||||||
|
|
||||||
// Does the error's cause chain hold the errno of a transport failure, one of
|
// Does the error's cause chain hold the code of a transport failure, one of
|
||||||
// `TRANSPORT_CODES`? `isRetryable` retries such an error, and `classify` in
|
// `TRANSPORT_CODES`? `isRetryable` retries such an error, and `classify` in
|
||||||
// `src/backup.ts` calls it transient.
|
// `src/backup.ts` calls it transient.
|
||||||
export const hasTransportCode = (err: unknown): boolean =>
|
export const hasTransportCode = (err: unknown): boolean =>
|
||||||
@@ -152,7 +172,8 @@ export const isRetryable = (err: unknown): boolean => {
|
|||||||
// deliberately literal. Demanding a recognised `cause` instead would
|
// deliberately literal. Demanding a recognised `cause` instead would
|
||||||
// classify real network failures as permanent and fail backups that should
|
// classify real network failures as permanent and fail backups that should
|
||||||
// have succeeded; the cost of the imprecision is bounded by the attempt
|
// have succeeded; the cost of the imprecision is bounded by the attempt
|
||||||
// count.
|
// count. Bun's fetch rejects with a plain `Error` instead, which is
|
||||||
|
// recognised by its code (see `TRANSPORT_CODES`).
|
||||||
if (err instanceof TypeError) return true;
|
if (err instanceof TypeError) return true;
|
||||||
return hasTransportCode(err);
|
return hasTransportCode(err);
|
||||||
};
|
};
|
||||||
@@ -163,18 +184,18 @@ export const isRetryable = (err: unknown): boolean => {
|
|||||||
// `postJSON` and `putJSON` use this for every `POST` and `PUT` listed in the
|
// `postJSON` and `putJSON` use this for every `POST` and `PUT` listed in the
|
||||||
// README under "Endpoints used"; verifying a second factor, for one, consumes
|
// README under "Endpoints used"; verifying a second factor, for one, consumes
|
||||||
// one of a small number of attempts. They are replayed only on the failures in
|
// one of a small number of attempts. They are replayed only on the failures in
|
||||||
// `CONNECT_CODES`, which establish that no TCP connection to the server ever
|
// `CONNECT_CODES`, which establish that no request byte was ever transmitted:
|
||||||
// existed: there was no address to connect to, or the peer refused the
|
// there was no address to connect to, the peer refused the connection
|
||||||
// connection outright. A request byte cannot have been transmitted, so the
|
// outright, or, under bun, the TLS handshake failed. The server cannot have
|
||||||
// server cannot have acted.
|
// acted.
|
||||||
//
|
//
|
||||||
// Everything else is ambiguous. A 5xx proves the server did process the
|
// Everything else is ambiguous. A 5xx proves the server did process the
|
||||||
// request. A reset or a broken pipe can arrive after it was fully sent and
|
// request. A reset or a broken pipe can arrive after it was fully sent and
|
||||||
// acted on. A routing errno can be delivered on an established socket. A
|
// acted on. A routing errno can be delivered on an established socket. A
|
||||||
// deadline says nothing at all about the server's state. So every errno in the
|
// deadline says nothing at all about the server's state. So every code in the
|
||||||
// cause chain must be a connect errno: one other errno anywhere in the chain
|
// cause chain must be one of `CONNECT_CODES`: one other code anywhere in the
|
||||||
// is doubt, and doubt is not replayed. A chain longer than the walk is doubt
|
// chain is doubt, and doubt is not replayed. A chain longer than the walk is
|
||||||
// too: the links below the limit were never read.
|
// doubt too: the links below the limit were never read.
|
||||||
export const isSafeToReplay = (err: unknown): boolean => {
|
export const isSafeToReplay = (err: unknown): boolean => {
|
||||||
const { codes, complete } = causeCodes(err);
|
const { codes, complete } = causeCodes(err);
|
||||||
return (
|
return (
|
||||||
|
|||||||
@@ -724,6 +724,50 @@ describe("quak backup", () => {
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// The download of toast.mp4 fails as bun's `fetch` reports a failure, in
|
||||||
|
// the binary `make install` builds: a plain `Error` with bun's own code
|
||||||
|
// and message, and no cause. These are what bun gave for a refused
|
||||||
|
// connection, a name that did not resolve and a reset connection. The
|
||||||
|
// file is tried as often as under Node, and goes into `failures.json`
|
||||||
|
// with that message, as a transient failure.
|
||||||
|
it.each([
|
||||||
|
[
|
||||||
|
"ConnectionRefused",
|
||||||
|
"Unable to connect. Is the computer able to access the url?",
|
||||||
|
],
|
||||||
|
["FailedToOpenSocket", "Was there a typo in the url or port?"],
|
||||||
|
[
|
||||||
|
"ECONNRESET",
|
||||||
|
"The socket connection was closed unexpectedly. For more information, pass `verbose: true` in the second argument to fetch()",
|
||||||
|
],
|
||||||
|
])(
|
||||||
|
"records a download that failed with bun's %s in failures.json, as transient",
|
||||||
|
async (bunCode, message) => {
|
||||||
|
let attempts = 0;
|
||||||
|
const fetch: typeof globalThis.fetch = async (input, init) => {
|
||||||
|
if (downloadOf(input) === 200) {
|
||||||
|
attempts++;
|
||||||
|
throw Object.assign(new Error(message), {
|
||||||
|
code: bunCode,
|
||||||
|
errno: 0,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return server(input, init);
|
||||||
|
};
|
||||||
|
const dir = mkdtempSync(join(root, "backup-"));
|
||||||
|
|
||||||
|
const { code } = await backup(dir, {}, fetch);
|
||||||
|
|
||||||
|
expect(code).toBe(1);
|
||||||
|
expect(attempts).toBe(UNATTENDED_RETRY_OPTIONS.attempts);
|
||||||
|
const failures = readJSON(join(dir, "failures.json"));
|
||||||
|
expect(failures.files["200"]).toMatchObject({
|
||||||
|
classification: "transient",
|
||||||
|
error: message,
|
||||||
|
});
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
// The name of Ente's API server does not resolve when quak fetches the ML
|
// The name of Ente's API server does not resolve when quak fetches the ML
|
||||||
// data, after every original has been backed up. No file has ML data
|
// data, after every original has been backed up. No file has ML data
|
||||||
// cached yet, so each goes into `failures.json` with the reason, as a
|
// cached yet, so each goes into `failures.json` with the reason, as a
|
||||||
|
|||||||
@@ -77,6 +77,14 @@ const recordingSleep = (): {
|
|||||||
const errnoError = (code: string, message = code): Error =>
|
const errnoError = (code: string, message = code): Error =>
|
||||||
Object.assign(new Error(message), { code });
|
Object.assign(new Error(message), { code });
|
||||||
|
|
||||||
|
/**
|
||||||
|
* An error shaped like what bun's `fetch` rejects with for a failure below
|
||||||
|
* HTTP: a plain `Error` with bun's own message, a `code`, an `errno` of 0, and
|
||||||
|
* no `cause`.
|
||||||
|
*/
|
||||||
|
const bunError = (code: string, message: string): Error =>
|
||||||
|
Object.assign(new Error(message), { code, errno: 0 });
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Classification
|
// Classification
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -244,6 +252,80 @@ describe("isRetryable: transport failures", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("isRetryable and isSafeToReplay: bun's fetch errors", () => {
|
||||||
|
// The binary `make install` builds runs quak under bun, whose `fetch`
|
||||||
|
// does not reject as Node's does. Each shape here was found by running
|
||||||
|
// that binary, built with bun 1.3.13, against listeners on 127.0.0.1 and
|
||||||
|
// a name under `.invalid`.
|
||||||
|
|
||||||
|
it("retries and replays a refused connection", () => {
|
||||||
|
// Bun gives the same error when the TLS handshake fails. The TCP
|
||||||
|
// connection existed then, but the request is only sent once the
|
||||||
|
// handshake is done, so replaying it is as safe.
|
||||||
|
const refused = bunError(
|
||||||
|
"ConnectionRefused",
|
||||||
|
"Unable to connect. Is the computer able to access the url?",
|
||||||
|
);
|
||||||
|
expect(isRetryable(refused)).toBe(true);
|
||||||
|
expect(isSafeToReplay(refused)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("retries and replays a name that does not resolve, with either code bun gives", () => {
|
||||||
|
// For the same name bun gave `ConnectionRefused` on some requests and
|
||||||
|
// `FailedToOpenSocket` on others.
|
||||||
|
for (const unresolved of [
|
||||||
|
bunError(
|
||||||
|
"ConnectionRefused",
|
||||||
|
"Unable to connect. Is the computer able to access the url?",
|
||||||
|
),
|
||||||
|
bunError(
|
||||||
|
"FailedToOpenSocket",
|
||||||
|
"Was there a typo in the url or port?",
|
||||||
|
),
|
||||||
|
]) {
|
||||||
|
expect(isRetryable(unresolved)).toBe(true);
|
||||||
|
expect(isSafeToReplay(unresolved)).toBe(true);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("retries a connection closed before the response ended, and does not replay it", () => {
|
||||||
|
// Bun gives Node's errno, `ECONNRESET`, for a reset and for a close,
|
||||||
|
// before the response or part-way through the body. The request may
|
||||||
|
// already have reached the server.
|
||||||
|
const reset = bunError(
|
||||||
|
"ECONNRESET",
|
||||||
|
"The socket connection was closed unexpectedly. For more information, pass `verbose: true` in the second argument to fetch()",
|
||||||
|
);
|
||||||
|
expect(isRetryable(reset)).toBe(true);
|
||||||
|
expect(isSafeToReplay(reset)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("retries a request that ran out of time, and does not replay it", () => {
|
||||||
|
// What bun's fetch rejects with when `AbortSignal.timeout()`, which
|
||||||
|
// bounds each JSON request, fires.
|
||||||
|
const timedOut = new DOMException(
|
||||||
|
"The operation timed out.",
|
||||||
|
"TimeoutError",
|
||||||
|
);
|
||||||
|
expect(isRetryable(timedOut)).toBe(true);
|
||||||
|
expect(isSafeToReplay(timedOut)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not retry a certificate bun rejects", () => {
|
||||||
|
// A certificate that fails verification fails the same way however
|
||||||
|
// often it is asked. Under Node it is retried only because every
|
||||||
|
// `TypeError` is.
|
||||||
|
expect(
|
||||||
|
isRetryable(
|
||||||
|
bunError(
|
||||||
|
"DEPTH_ZERO_SELF_SIGNED_CERT",
|
||||||
|
"self signed certificate",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe("isRetryable: stream truncation versus corruption", () => {
|
describe("isRetryable: stream truncation versus corruption", () => {
|
||||||
it("retries a truncated stream", () => {
|
it("retries a truncated stream", () => {
|
||||||
// Truncation is a transfer that stopped early. The bytes that did
|
// Truncation is a transfer that stopped early. The bytes that did
|
||||||
|
|||||||
Reference in New Issue
Block a user