The README layout lists src/library/ and the other source files, the
backup layout names failures.json and the optional thumbnails/, "Opening
a library" says an empty cache opens with no data when the first refresh
fails, and the test time limit is the 90 seconds the Dockerfile sets.
The next step in both files is storing live photos (issue 107) instead
of a v1.0.0 tag; tagging and releases are sneak's call alone.
Model: opus-5-5
An original fetched for a backup is now written by the download writer
straight into the backup's originals/, and the content cache records it
there instead of keeping its own copy. One the cache already held is
still copied. `quak backup` opens its library with the thumbnail and
originals precache off, as the one-shot commands do.
Model: opus-5-5
backup-metadata, helper list-missing-thumbnails and helper
fix-missing-thumbnails now await lib.fresh() before reading the library,
as collections, files, get and get-thumb already do. A file added since
the cache was written is included, and a failed refresh is thrown, so
the CLI prints one line and exits 1 instead of answering from a stale
or empty cache. The README lists them among the commands that refresh
first.
Model: opus-5-5
The download writer's temp files are now named .quak-<pid>-<random>.tmp.
removeLeftoverTempFiles moves from the backup into the download module and
deletes a .quak-*.tmp file only when the process ID in its name is no
longer running; the content cache calls it at open() instead of deleting
every temp file, so a download in progress in another process sharing the
cache survives. The README backup layout and TODO.md are updated.
Model: opus-5-5
Follows the template: Dockerfile.lint is gone; the Dockerfile has a lint
phase (eslint, prettier --check .) and a test phase (vitest, run as the
node user, which the not-writable-directory tests need), and its
last stage compiles and depends on both. script/lint and script/test
build one phase each with --no-cache; script/docker and script/cibuild
pass --no-cache, so CHECK_EPOCH and LINT_EPOCH are removed.
script/cibuild is the single image build, so CI runs lint and the tests
once each. The tests that checked the old layout are deleted,
REPO_POLICIES.md is re-copied and the README describes the new layout.
Model: opus-5-5
The server accepts a new thumbnail only from the file's owner and only when
it is no larger than the thumbnail size it records. Both thumbnail helpers
now skip files another account owns without fetching them. The fixer skips
a file whose recorded thumbnail size is 0 or unknown before downloading it,
and otherwise tries smaller encodings until the encrypted thumbnail fits,
skipping the file if none does.
Model: opus-5-5
logout now calls POST /users/logout with the saved token through the new
Client.logoutOnServer(), then deletes session.json even when that call
fails; in that case it says the server session could not be ended and
exits 1. It prints the account's cache directory and says it still holds
decrypted data. The default cache path moves into
defaultCacheDirectory(), shared with Library.open, so both name the same
directory.
Model: opus-5-5
Within a collection's folder, files whose sanitized titles match (ignoring
case) each get their file ID added before the extension, and collections
whose sanitized names match get their ID added. This repeats until no name,
including one with an ID added, matches another, so no symlink or JSON
replaces another. Names are chosen across all collections, so a scoped run
names folders the same as a full one.
Each run first removes symlinks into originals/ that no longer belong to a
collection, and the folders quak wrote (a sibling JSON with an album ID) for
collections that are gone or renamed. Anything else is left alone; a folder
still holding user files keeps its JSON.
Model: opus-5-5
downloadFile, shared by quak get, the content cache and backup, hashes
the decrypted bytes (unkeyed BLAKE2b-512, standard base64) and stores
nothing on a mismatch, failing with an error naming the file ID. A live
photo ZIP is unpacked as it streams with fflate's Unzip, in small
slices so memory stays bounded however far an entry expands, and its
image and video hashed separately as <imageHash>:<videoHash>.
decryptFile reads the older imageHash/videoHash fields for live
photos. A file with no recorded hash is stored unchecked.
Model: opus-5-5
When metadata.json in the cache directory was written for a different,
non-zero user ID than the client's, Library.open deletes it and mldata/
and loads an empty store, so the first refresh enumerates from 0 and none
of the other account's collections, files, keys or ML results are served.
Only reachable with --cache-dir or an explicit cacheDirectory.
Model: opus-5-5
Each ML data request of up to 200 files is now tried on its own. A request
that still fails after its retries is logged, its files are written with the
reason in `mlDataError`, and the command exits 1 once the dump is complete.
`fetchMLData`, used only here, is removed in favour of a per-batch loop over
`fetchMLDataBatch`.
Model: opus-5-5
Library.close() now returns a promise that resolves once the work it
started has finished: an in-flight refresh with its cache write, the ML
data fetch, and running precache sweeps. The interval test could see a
refresh's new state, close, and remove the directory while the write was
still running. Every library test and CLI command now awaits close(), and
tests hold each of the three writes open to prove close() waits for it.
The precache test waited for its stub source to be called, but the cache
records a file only after checking it on disk, so status() could lag. It
now waits for both fills to report "done".
Model: opus-5-5
downloadTimeoutMs now aborts a file or thumbnail download only when no
bytes have arrived for that long (default 60 s, was a 600 s cap on the
whole transfer), so a slow download that keeps making progress completes.
The abort reason is still a TimeoutError, so retry classification is
unchanged. A download that fails before reading the whole response body
cancels it, including when the temp file cannot be opened or the header
is malformed, so a failed file no longer holds its connection.
Model: opus-5-5
The backup copy now fsyncs its temp file before the rename and the
directory after it, using the download writer's new fsyncPath helper.
Each backup run deletes .quak-backup-*.tmp files whose process is no
longer running, leaving those of a concurrent backup alone. The rename
sites and the README backup layout state that a symlink at the
destination is replaced and the new file takes the temp file's
permissions, and the README names the temp files. Adds tests for a
missing and an unwritable destination directory for downloadFile and
downloadThumbnail.
Model: opus-5-5
A POST or PUT is replayed only when every errno in the cause chain is a
connect errno and the walk reached the end of the chain, and
postJSON/putJSON no longer follow redirects, so a redirect is an
ApiError that is not retried. getRetryOptions() returns a copy. New
tests pin every errno the classifier names, the cause-chain depth
limit, a chain deeper than the limit, a two-error cycle, and a fresh
deadline per attempt for every retrying entry point. The README
endpoint list is now the one place naming the requests the replay rule
covers; code comments point to it.
Model: opus-5-5
Client.fromJSON checks every snapshot field and each key's decoded length
and throws an error naming the bad field. toJSON reads the token through a
new ApiClient.getAuthToken and throws when there is none. logout zeroes the
key buffers in place; collectionsSince re-checks for logout after its
request so it never decrypts with zeroed keys. The CLI now reports a
corrupt session file separately from a missing one.
Model: opus-5-5
Rewrites the README API reference to match the shipped library surface: Library.open options (XDG cache dir + userID, downloadDirectory, refreshIntervalSeconds, precache and cache-size options), default vs fresh reads (Library.fresh()), snapshot/subscribe, albums/photos/timeline, Photo.original/thumbnail, thumbnails.ensure, mldata search, backup, the request pools and on-disk layout, and the CLI (--cache-dir, fresh reads). Every documented signature verified against the code. Notes the deferred content-hash integrity check (#68). Also issue #13.
Model: opus-4-8
Ports the CLI to the library API. collections/files/get/get-thumb use the fresh read variants (Library.fresh(), current server state); backup runs lib.backup; backup-metadata and the missing-thumbnail helpers enumerate via the library. files/get output is byte-identical to the pre-port CLI — raw metadata.title, microsecond creationTime, pre-port row order — exit codes unchanged. Adds --cache-dir; fixes the helper JPEG-only assumption (closes#17).
Model: opus-4-8
sneak, 2026-09-05: "inference instance stopped. undo its rogue work." The
reverted commits stay in history; nothing else on next is touched.
Model: fable-5-1
The false green: script/lint chaining a second docker build after the
lint image ran prettier twice and counted once. The misresolution: a
bare docker build followed by cp -f resolved to the cp's file. Pin
both, and each separator that bounds an invocation.
edgesOf used a single test/exec for docker build, so a line with two
builds produced one edge and the file flag was searched across the whole
line. Follow every occurrence with matchAll and slice each one to the
next shell separator before looking for its flag.
Also correct the README's claim that script/fmt-check is the one
formatting path left on the host.
The Linting section this PR adds claimed "There is no host lint path" and
then, two paragraphs later, documented script/fmt-check as a host-side
formatting check. Both cannot be true, and the absolute one is the false one.
What is true is narrower: no lint path reachable from script/check or
script/precommit runs on the host, so every lint verdict those two produce
comes from the container. script/fmt-check stays as a standalone entrypoint,
now stated as the one host formatting path with nothing reaching it. The
"exactly one place" and "in the container only" phrasings elsewhere are
qualified the same way.
script/check ran script/test, script/lint and script/fmt-check. Since
linting moved into Docker, script/lint is a build of Dockerfile.lint,
which runs `prettier --check .` as a build step — so make check checked
formatting twice over the same tree: once in the container and once on
the host. script/precommit had the same pair.
Drop the script/fmt-check call from both. The container keeps the check,
because a successful Dockerfile.lint build is what CI treats as proof of
a clean tree, and it is the stronger of the two verdicts: its prettier is
digest-pinned and installed under --frozen-lockfile, while the host's is
whatever the working tree happens to have. The pre-commit hook is
unchanged in what it catches — script/lint still fails a badly formatted
tree, and therefore the commit.
script/fmt-check survives as a standalone entrypoint, as REPO_POLICIES.md
requires, for asking the formatting question by itself without docker.
Its verdict cannot drift from the container's: prettier is pinned to an
exact version, installed from yarn.lock in both places, and reads
.gitignore as its default ignore file, which is why .dockerignore keeps
.gitignore in the build context.
The count is asserted rather than promised. test/packaging/lint-once.test.ts
walks the invocation graph from each entrypoint — through the Makefile
shims, the script/ calls, the package.json scripts and the docker build
into Dockerfile.lint's RUN steps — and counts prettier invocations: one
per make check, one per script/precommit, and one each for make lint and
make fmt-check alone, so neither can become a no-op that satisfies the
count trivially. The walk also asserts which nodes it reached, so a
restructure that defeats the resolver fails the test instead of quietly
counting zero.
Observed: 2 prettier invocations per make check before, 1 after.
Linting now happens in one place only: a new root Dockerfile.lint copies
the repo into the digest-pinned node image already used by Dockerfile and
runs eslint and prettier as build steps, so a successful build is a clean
lint. script/lint is reduced to building it, which also works where the
docker daemon is remote and bind mounts are impossible. No host lint path
survives: the "lint" script is gone from package.json, so there is no
second, unpinned way to get a lint verdict.
Caching is waived for lint, because a lint build over an unchanged tree
returns success in well under a second having linted nothing. LINT_EPOCH
is the cache buster and it fails closed exactly as CHECK_EPOCH does: an
unset ARG is the empty string, which is a perfectly stable cache key, so
the guard rejects it and a bare `docker build -f Dockerfile.lint .` errors
out instead of serving a green it did not earn. Both linters sit below the
guard, so a fresh epoch forces them to execute while the bootstrap and
dependency layers above stay cached.
That makes script/lint a docker build, which nothing inside a container
may call. script/check calls script/lint, so the Dockerfile image can no
longer run make check: the lint stage and its COPY --from=lint ordering
hack are deleted, and the remaining stage runs make test and make build
under the existing CHECK_EPOCH guard. script/cibuild is now the composite
gate and builds the lint image first, so a lint failure is reported before
the slower suite runs.
The .dockerignore exclusions are unchanged and still apply to the lint
build, including the .claude/ exclusion (eslint's flat config does not
ignore dot-directories, so a nested worktree in the context would be
linted) and the deliberate exception that keeps .gitignore in the context
for prettier. A new test asserts no per-Dockerfile ignore file shadows the
root one for either image, and test/packaging/lint-docker.test.ts asserts
the whole shape: the docker-only lint path, the digest pin, manifests
copied before sources, the fail-closed guard with both linters below it,
the absence of a lint stage or make check in Dockerfile, and the build
order in script/cibuild.
The image build reported a green it had not earned. `script/cibuild` is a
bare `docker build .`, and with `COPY . .` followed by `RUN make check`,
an unchanged tree served that layer from cache: the suite never ran and
the build still exited 0, while the script's header comment asserted the
opposite.
CHECK_EPOCH, passed by `script/cibuild` and `script/docker`, changes the
cache key of the check and build layers on every invocation. It is
guarded, because an unset ARG is the empty string and therefore a stable
key: without the guard a plain `docker build .` — the command the policy
names, and the one anyone debugging types — would still get the false
green. A missing argument is now a hard failure rather than a silent
degradation to the behaviour the epoch was added to prevent.
The Dockerfile is now two stages: `fmt-check` and `lint` run first, and
the check stage takes a `COPY --from=lint` dependency on them, so a
formatting mistake fails the build in seconds instead of racing the
suite to the finish. Both stages stay pinned to the same digest.
The remaining fixes are one-liners that had made the target unusable:
`script/projectname` still printed the pre-rename name, so `make docker`
tagged its image after a name this project dropped in May;
`script/bootstrap` installed without fetching apt's package lists, which
cannot work on a Debian base; and `.dockerignore` had drifted far enough
from `.gitignore` to ship a ~100 MB compiled binary and any agent
worktree under `.claude/` into the build context. The second of those is
a correctness problem, not a size one — vitest globs a copied worktree's
tests alongside the real ones and runs the suite twice over. `.gitignore`
itself stays in the context, because prettier reads it as a default
ignore file and dropping it would change what `make fmt-check` sees.
rootDir was ./src while include also matched bin/**/*, which is TS6059: tsc
refuses to emit at all when a compiled file sits outside rootDir. rootDir is
now the repository root, which is the smallest change that makes the two
agree and leaves the source layout the README documents alone. Output keeps
the shape of the source tree, so main and types move to dist/src/index.js and
dist/src/index.d.ts while bin.quak stays at dist/bin/quak.js. The alternative,
moving the CLI body into src/ behind a shim in bin/, would hold main at
dist/index.js at the cost of churning the CLI and contradicting the layout
diagram in the README.
Clearing TS6059 exposed two type errors that had never been reached, because
the config error aborts before checking: StateAddress was read as a namespace
member off the default import, and the secretstream pull was called without
the additional-data argument, which libsodium does not make optional. The type
is now taken from the module's named export and the pull passes null for ad,
matching the null already passed on the push side in encryptBlob. Neither
changes what runs.
noEmitOnError stops a failed build from leaving output behind. It emitted
despite the error before, which is how a stale bin/quak.js came to sit next to
bin/quak.ts in a working tree, where eslint then read it and failed make check
on a generated file.
script/build compiles and then checks that the files package.json advertises
are among the ones the compiler wrote, since tsc knows nothing about the
manifest and a green build could still ship a package whose main resolves to
nothing. It also sets the executable bit on the bin entries, which tsc does
not carry over from the source even though it does copy the shebang. The
Makefile target is now a shim over it, as the other targets are, and
package.json's build script points at it so yarn build gets the same checks.
The Dockerfile runs make build after make check, so a branch that does not
compile cannot reach main. What make check itself runs is unchanged.
package.json gains a quak script, so the yarn quak commands the README's
Getting Started block has always listed resolve to the built CLI.
`CONNECT_CODES` drives `isSafeToReplay`, which is the only thing standing
between a transport failure and a replayed `POST /users/two-factor/verify`.
It included `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` on the stated
grounds that those errnos can only be reported before any request byte was
written. That is not true on Linux: an ICMP destination-unreachable
delivered on an already-established connection sets the socket error and
the next read or write returns `EHOSTUNREACH` or `ENETUNREACH`, and a local
interface going down after the request was fully written surfaces as
`ENETDOWN` the same way. In each case the server may already have received
and acted on the request -- exactly the ambiguity the rule exists to
exclude, on the paths that consume a second-factor attempt or register a
thumbnail.
The three are dropped from `CONNECT_CODES` and stay in `TRANSPORT_CODES`,
so they remain retryable for the idempotent calls; only replay eligibility
narrows. What is left -- `ENOTFOUND`, `EAI_AGAIN`, `ECONNREFUSED` -- means
no TCP connection to the server ever existed, so no request byte can have
been transmitted.
The justification is corrected everywhere it was stated: the comment on
`CONNECT_CODES`, the one on `isSafeToReplay`, the `postJSON` call site, the
README's idempotency section and the `client.test.ts` docblock. All of them
now describe what the narrowed set actually establishes rather than
claiming a proof it did not support.
The narrowing is enforced by the suite rather than asserted in a comment:
the three errnos join `ECONNRESET`/`EPIPE`/`ETIMEDOUT` in the
`isSafeToReplay`-returns-false test, with companion `isRetryable` assertions
so a future edit cannot make them non-retryable by accident. Putting the
three back into `CONNECT_CODES` turns that test red (1 failure, verified).
No retry on 4xx, backoff on 5xx and transport failures, and a deadline on
every request. Before this, one transient 503 or TCP reset failed a file
for good, and a CDN connection that went quiet after accepting the request
blocked `quak backup` forever, because there was no timeout anywhere.
src/retry.ts holds the policy: a classifier that decides whether another
attempt could produce a different answer, and a loop that acts on it with
exponential backoff and full jitter. Retried: 5xx, 408, 429, transport
failures (the errno is read out of the cause chain, which is where Node's
fetch puts it), deadline aborts, and truncated transfers. Not retried:
every other 4xx, and anything unrecognised — a wrongly retried permanent
failure delays every remaining file, while a wrongly abandoned transient
one costs a single file the next run picks up. Attempt count, delays,
sleep and jitter source are all configurable through ApiClientOptions;
sleep being injectable is what lets the suite exercise the policy without
waiting.
Truncation needed a type before it could be classified. streamDecrypt
threw plain Errors whose messages began "download: stream truncated", and
classifying on message text would mean the next reword silently turned
every truncated download into a permanent failure. It now throws
TruncatedStreamError, which lives in src/errors.ts alongside ApiError so
the classifier can recognise both without importing the modules that
import it; api/client.ts re-exports ApiError, so it stays one class and
every existing import path still resolves.
Downloads retry the request, the stream consumption and the decryption
together. Only the first of those happens inside ApiClient: a socket
reset after the headers arrived throws in streamDecrypt, and retrying the
request alone would never see it. The client's own retry is switched off
for those two calls so the budgets do not multiply into sixteen requests
per file, and the atomic write stays outside the loop so a download that
took three attempts still performs one write and one rename.
Non-idempotent requests are not blindly replayed. postJSON and putJSON
reach create-session, two-factor/verify — which burns one of a few
second-factor attempts — and files/thumbnail, so they retry only when the
connection was never established and the server provably never saw the
request. putFile is exempt and retries fully: a presigned PUT stores one
whole object at one key, with no partial state to damage. It now throws
ApiError with the status, as do the two null-body paths, which previously
threw bare Errors that nothing could classify.
Timeouts come from AbortSignal.timeout(), renewed per attempt: 30s for
JSON and upload calls, 10 minutes for file bodies, since a value short
enough to keep a hung API call from stalling a backup would cancel a
legitimate multi-gigabyte download. The download deadline is enforced
over the body rather than only the headers, by racing each read against
the signal, so the guarantee does not depend on the fetch implementation
tearing down a stream it already handed over.
listMissingThumbnails now separates a genuine 404 from an exhausted
retry. Its bare catch reported both as missing, which after this change
would have let a few minutes of 500s talk fix-missing-thumbnails into
regenerating and re-uploading thumbnails that were fine. runBackup and
runMetadataBackup are untouched: the retry sits below them and their
per-file resilience is unchanged.
Fixes accumulated drift from the original spec-first README:
- Intro: added paragraph describing backup, metadata decryption, and
thumbnail repair capabilities
- Rationale: removed the 'deliberately scoped to read operations'
claim (no longer true since thumbnail upload exists); called out
the Go CLI's crash-on-failure bug as explicit motivation
- Getting Started: fixed library example to use actual Client.login()
API, removed nonexistent fromSavedSession/getFile methods, added
backup CLI example
- Layout: fixed to match actual directory structure (download/,
backup.ts, thumbnails.ts; removed nonexistent session/)
- Session handling: replaced the fictional encrypted-keychain
SessionStore description with the actual implementation (plain JSON
via env-paths, consumer-managed persistence via toJSON/fromJSON)
- CLI surface: added backup, helper list-missing-thumbnails, and
helper fix-missing-thumbnails commands
- Backup layout: new section documenting the originals/ + collections/
symlink structure
- API reference: replaced the stale type declarations with pointers
to the actual source files and a note that test/client/usage.test.ts
is the authoritative API tutorial
- TODO: collapsed completed phases, kept only open items
- For LLMs: new section summarizing repo policies, TDD workflow,
required checks, formatting rules, and pointers to REPO_POLICIES.md
and LLM_PROSE_TELLS.md
Complete CLI surface:
quak login interactive or QUAK_EMAIL/QUAK_PASSWORD
quak whoami print logged-in account
quak logout delete session
quak collections list all albums (--json)
quak files list files in a collection (--json)
quak get <id> download+decrypt a file (--out, --collection)
quak get-thumb <id> download+decrypt a thumbnail
quak backup <dir> full incremental backup
get/get-thumb search all collections for the file ID when --collection
is not specified. All listing commands support --json.
Live-tested: collections list, file list, single file download (472 KB
JPEG from the dev account, verified as valid JPEG with EXIF intact).
German for 'quack', matching the Ente (German for 'duck') naming. All
references updated: package name, CLI binary, X-Client-Package header,
test descriptions, temp dir prefixes, README, Makefile docker tag.
Exports Client, all lower-level modules, and all types from
src/index.ts. Replaces Phase 7 (on-disk session persistence) with
the Client class phase: session lives in the object, consumer
handles persistence via toJSON/fromJSON.
The implementation is exactly the decryption chain documented in the
test file: deriveKEK -> decryptBox(masterKey) -> decryptBox(secretKey)
-> decryptSealed(token) -> toBase64URL. Errors from the underlying
crypto primitives propagate; the only added validation is the up-front
check that the response actually contains both keyAttributes and
encryptedToken (caller bug if not).
Also re-exports the auth/unwrap and auth/types public surface from
src/index.ts.
All 38 tests pass; make check and make docker are green.
Tests are deliberately excluded from the pre-commit hook so that the
TDD red-phase commit (failing tests landing before implementation) can
land on a feature branch. CI still runs full make check via docker build
so a red branch cannot reach main.
All changes go on a feature branch; the first commit on the branch is the
failing test suite for the change; the branch can only merge to main when
make check is green. Tests are the canonical API documentation and must be
commented thoroughly so a reader can learn the library from them.
Reframes the project as the protocol foundation for a future Electron-based
Ente desktop client (the existing official clients are unsatisfactory).
Adds an API reference section with TypeScript declarations for every
exported name across crypto, auth, model, api, session, and Client modules.
Adds Phase 10 desktop-client TODO items so a future agent can pick up the
plan.
sneak
clawbot
clawbot