sneak
f3cf4af833
All checks were successful
check / check (push) Successful in 20s
No retry on 4xx, backoff on 5xx and transport failures, and a deadline on every request. Before this, one transient 503 or TCP reset failed a file for good, and a CDN connection that went quiet after accepting the request blocked `quak backup` forever, because there was no timeout anywhere. src/retry.ts holds the policy: a classifier that decides whether another attempt could produce a different answer, and a loop that acts on it with exponential backoff and full jitter. Retried: 5xx, 408, 429, transport failures (the errno is read out of the cause chain, which is where Node's fetch puts it), deadline aborts, and truncated transfers. Not retried: every other 4xx, and anything unrecognised — a wrongly retried permanent failure delays every remaining file, while a wrongly abandoned transient one costs a single file the next run picks up. Attempt count, delays, sleep and jitter source are all configurable through ApiClientOptions; sleep being injectable is what lets the suite exercise the policy without waiting. Truncation needed a type before it could be classified. streamDecrypt threw plain Errors whose messages began "download: stream truncated", and classifying on message text would mean the next reword silently turned every truncated download into a permanent failure. It now throws TruncatedStreamError, which lives in src/errors.ts alongside ApiError so the classifier can recognise both without importing the modules that import it; api/client.ts re-exports ApiError, so it stays one class and every existing import path still resolves. Downloads retry the request, the stream consumption and the decryption together. Only the first of those happens inside ApiClient: a socket reset after the headers arrived throws in streamDecrypt, and retrying the request alone would never see it. The client's own retry is switched off for those two calls so the budgets do not multiply into sixteen requests per file, and the atomic write stays outside the loop so a download that took three attempts still performs one write and one rename. Non-idempotent requests are not blindly replayed. postJSON and putJSON reach create-session, two-factor/verify — which burns one of a few second-factor attempts — and files/thumbnail, so they retry only when the connection was never established and the server provably never saw the request. putFile is exempt and retries fully: a presigned PUT stores one whole object at one key, with no partial state to damage. It now throws ApiError with the status, as do the two null-body paths, which previously threw bare Errors that nothing could classify. Timeouts come from AbortSignal.timeout(), renewed per attempt: 30s for JSON and upload calls, 10 minutes for file bodies, since a value short enough to keep a hung API call from stalling a backup would cancel a legitimate multi-gigabyte download. The download deadline is enforced over the body rather than only the headers, by racing each read against the signal, so the guarantee does not depend on the fetch implementation tearing down a stream it already handed over. listMissingThumbnails now separates a genuine 404 from an exhausted retry. Its bare catch reported both as missing, which after this change would have let a few minutes of 500s talk fix-missing-thumbnails into regenerating and re-uploading thumbnails that were fine. runBackup and runMetadataBackup are untouched: the retry sits below them and their per-file resilience is unchanged.
2.5 KiB
2.5 KiB
Workflow
- branch (from
main) - do the work in Next Step
- move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step
- commit (
TODO.mdchanges in the same commit as the work) - merge to
mainif the branch is not protected, otherwise open a PR - push
Status
pre-1.0
Next Step
Update the README API reference section to match the current implementation.
Completed Steps
- 2026-08-09: Retry policy: no retry on 4xx (except
408and429), exponential backoff with full jitter on 5xx, transport failures and truncated transfers, under per-attempt deadlines that cover the response body as well as the request. Downloads retry request, stream consumption and decryption as one unit;postJSONandputJSONare replayed only when the connection was never established. - 2026-08-09: Downloads verify the secretstream terminated on
TAG_FINALand write output atomically: a truncated body is rejected instead of landing on disk as a short file, and plaintext is staged in a sibling temp file and renamed into place, so a failed download leaves the destination untouched. - 2026-07-07 Adopted scripts-to-rule-them-all:
script/entrypoints, Makefile shims, README Entrypoints section - 2026-06-10: Decrypted collections shared by other users (sealed-box keys); listCollections drops deleted-collection tombstones.
- 2026-06-10: Login hardening: dual-2FA empty-string fields handled, TOTP preferred when a passkey is also enrolled, interactive input via @inquirer/prompts.
- 2026-06-10: Replaced sharp with pure JS (jpeg-js + exif-reader); added single-binary bun build and make install.
- 2026-06-09: Added backup-metadata command (ML data always included, --exif opt-in); rewrote README to match the implementation; added thumbnail helper tests.
- 2026-05-13: Full CLI surface: login, backup with dedup symlink layout, collections, files, get, get-thumb, thumbnail repair helpers.
- 2026-05-13: Client OO API with literate usage tests; file download and decryption; all three metadata layers decrypted and persisted; renamed quack to quak.
- 2026-05-11: SRP login flow (email OTP + TOTP) and ApiClient.
Future Steps
- Make
make dockergreen. - Tag v1.0.0.
- Future desktop client, separate repo:
- Electron app skeleton consuming this library.
- Local SQLite cache keyed on (collectionID, fileID, updationTime).
- Background sync worker streaming new files into the cache.
- Gallery UI: thumbnails, full-image view, basic search.
- Upload, delete, and share operations in the library.