Commit Graph
71 Commits
Author SHA1 Message Date
sneak 9c921e95dc Check the lock before every file, and pass the lock itself to the backup
check / check (push) Waiting to run
The backup now checks its lock after the refresh and before each file of
the thumbnail, JSON and album passes, as well as before each original and
before failures.json. lockBackupDirectory returns the lock, with release()
and lost, in place of a release function, and lib.backup() takes it as
lock in place of lockHeld: true; quak backup passes it. This drops the
process-wide map from directory path to lock, which missed a lock taken
through another path to the same directory.

Model: opus-5-5
2026-10-14 05:09:55 +00:00
sneak 1b1bcc1f3e A backup whose lock is lost stops with one line, not a stack trace (closes #184)
lockBackupDirectory now passes proper-lockfile an onCompromised that records
why the lock was lost, where the default threw from a timer and ended the
process. The backup checks its lock before each original and before it
writes failures.json, and fails with "lost the lock on DIR: REASON", which
quak backup prints through run as one line before exiting 1. Releasing a
lost lock leaves backup.lock alone, as it may be another backup's. The lock
of each directory is kept by path so that a backup run with lockHeld, as
quak backup runs it, finds the lock its caller took.

Model: opus-5-5
2026-10-14 05:09:54 +00:00
clawbot bb3b5cd2da One album or file quak cannot decrypt no longer stops a refresh or a backup (closes #181)
check / check (push) Waiting to run
A file or album whose key, metadata or name does not decrypt used to
throw out of the refresh, so quak backup stopped before storing
anything, every run. Client.collectionsSince and filesSince now return
such a row in undecryptable with the reason, the refresh leaves it out,
and the store keeps the list in metadata.json until a later refresh
decrypts or deletes the record. lib.backup() records each such file in
failures.json unless a copy in another album decrypted, in which case it
stores the original from that copy. Each such album is listed in
BackupResult.undecryptableAlbums and printed as one FAILED album line.
quak backup exits 1 while either remains. An album or file that no
longer decrypts loses its album folder, JSON and links; its original
stays.

Model: opus-5-5
2026-10-14 07:08:57 +02:00
clawbot 288825b11a quak backup no longer refreshes its library on a timer (closes #183)
check / check (push) Waiting to run
The backup opened its library with the default 3-second refresh interval,
so for its whole run it asked the server for changes every few seconds,
and each refresh started another ML data pass. It now opens it with
refreshIntervalSeconds: Infinity. The library takes Infinity as no
background timer, since setTimeout would turn it into 1 ms. A refresh
interval of 3600, as the read commands use, would still refresh once an
hour in a backup that runs for hours.

Model: opus-5-5
2026-10-14 04:10:14 +02:00
clawbot 54133cfa5f Test quak backup end to end against a fake Ente server (closes #186)
check / check (push) Waiting to run
test/cli/backup-end-to-end.test.ts runs backupCommand through run, as
bin/quak.ts does, with the real loadSession reading a session saved after
logging in, and every request answered by a fake of the Ente HTTP API serving
data encrypted with libsodium. It covers the backup tree of an account with
two albums of its own and one shared with it, a second run that downloads
nothing, a download that succeeds after two 503s, a 404 recorded in
failures.json with exit 1, a refresh answered 401 with exit 3, and --verify
putting back an original changed on disk.

The fake server and the account it logs in moved from the backup-metadata
test into test/ente-server.ts, unchanged, so both tests use them.

Model: opus-5-5
2026-10-14 03:38:38 +02:00
clawbot ab28d86aaa An ID on the command line is decimal digits only (closes #185)
check / check (push) Waiting to run
files --collection, get, get-thumb and helper fix-missing-thumbnails
--file read an ID with Number(), so 0x10 meant 16, 1e3 meant 1000 and
an empty string meant 0, and --file dropped an ID it could not read.
parseID in src/cli-commands.ts now takes an ID only when it is decimal
digits and a safe integer; anything else prints "Invalid file ID: ARG"
or "Invalid collection ID: ARG" and the command exits 1. The fixer
checks every --file ID before it opens the library, so one bad ID fails
the whole command before any request.

Model: opus-5-5
2026-10-14 03:06:35 +02:00
clawbot 2402ec97f7 README: running quak backup from cron, and its exit codes (closes #170)
check / check (push) Successful in 3m59s
A new README section shows how to run quak backup unattended: log in once
as the job's user, a crontab with a backup every night and --verify on
Sundays, output appended to a log file, and the HOME and XDG_DATA_HOME the
job needs to find the saved session. A table gives each exit code as the
code returns it, and says which failures the next run retries. The
introduction now lists everything the backup keeps for each file. It, the
backup layout tree and the lib.backup() entry say EXIF and XMP are kept for
an image only, and dimensions for a JPEG only.

Judgement call: the --verify run takes Sunday's slot rather than a second
job that night, since an overlapping run would exit 2.

Model: opus-5-5
2026-10-07 00:47:37 +02:00
clawbot ddf58af3cc quak backup --verify re-hashes stored originals and downloads again any that do not match (closes #168)
check / check (push) Failing after 1m22s
`--verify`, or `lib.backup({ verify: true })`, hashes each original already
at its save path as the download check does, streamed, a live photo as
`<imageHash>:<videoHash>`. A mismatch is logged, removed and put back in the
same run, and what is put back is hashed too, since a copy from the content
cache is not checked; one that still does not match, or a failed fetch, goes
into `failures.json`. A file with no recorded hash counts as unchecked. The
result, the summary and `--json` gain `verified`, `mismatched` and
`unchecked`.

Judgement call: a stored original that cannot be read for hashing is recorded as failed and left in place.
Judgement call: a copy put back that still does not match stays at its save path and is not counted as downloaded.
Judgement call: the summary prints the three counts only with `--verify`.

Model: opus-5-5
2026-10-06 22:47:26 +02:00
clawbot bd77422965 quak backup refuses to run while another backup of the same directory runs, exit 2 (closes #169)
check / check (push) Successful in 3m10s
lib.backup() takes a lock, backup.lock in its download directory, made
with proper-lockfile, before its refresh, and removes it when it ends. A
second backup of the directory fails at once with an error naming it.
quak backup takes the lock itself before it opens its library and passes
lockHeld to the backup, so a refused run sends no request; it prints the
error as one line and exits 2. A lock untouched for 10 seconds, left by a
run that could not remove it, is taken over.

Deviation: yarn.lock was regenerated by yarn add in the pinned node image.
Judgement call: a run failing at its refresh leaves the directory, empty.
Judgement call: a lock removed mid-run stops that run with an uncaught
error, the library's default.

Model: opus-5-5
2026-10-06 20:30:51 +02:00
clawbot 31b50a211d quak backup writes each original's EXIF, XMP and dimensions into its JSON (closes #167)
check / check (push) Successful in 3m18s
Each file's JSON gains imageMetadata, what extractImageMetadata finds in
the stored original (for a live photo, its image), or the reason the
read failed in imageMetadataError; a failed read fails neither the file
nor the run. A video is not read. An original is read when the run
stores it or when its JSON has neither field; otherwise the field is
carried over from that JSON, so a run does not read every original
again. The hand-built JPEG fixtures move to test/exif-jpeg.ts so the
backup tests can use them.

Judgement call: an original with nothing to record gets imageMetadata
{} instead of no field, so it is not read again on every run.

Model: opus-5-5
2026-10-06 16:47:31 +02:00
clawbot f6317109bc An expired session exits 3 with one line saying to run quak login (closes #164)
check / check (push) Successful in 3m8s
A 401 from the server that ends a command reaches run in src/cli-run.ts as
an ApiError; run prints one line saying to run "quak login" and exits 3.
quak backup meets it on its first refresh, before it touches any file. For
the commands that load the saved session, a missing or corrupt session file
keeps its message and also exits 3.

Judgement call: quak logout is unchanged; it handles its own errors and
deletes the session file whatever the server answers.
Judgement call: quak backup still prints its two progress lines before the
error line.

Model: opus-5-5
2026-10-06 14:01:48 +02:00
clawbot ad0e11407f quak backup retries failed requests for longer (closes #165)
check / check (push) Successful in 4m16s
src/retry.ts exports UNATTENDED_RETRY_OPTIONS beside the unchanged
default: 10 attempts, a 1 s base delay and a 60 s cap, so a request that
keeps failing waits at most 243 s before it gives up. bin/quak.ts loads
the backup's session with them, so its refresh, ML data and downloads
all use them; every other command keeps the default. What is retried
and the backoff formula are unchanged.

Model: opus-5-5
2026-10-06 12:47:40 +02:00
clawbot fb4fff59fe Re-vendor the canonical files from sneak/prompts at dd4027b (closes #171)
check / check (push) Successful in 4m45s
.dockerignore, .gitignore, .prettierignore and REPO_POLICIES.md are
copied from sneak/prompts at dd4027b; .editorconfig, check.yml and
.prettierrc already matched. This repository's build artifacts follow
the copied content in .gitignore and, anchored, in .dockerignore. The
scripts follow the new policy: check runs fmt-check, cibuild bootstraps
and runs check before the image build, fmt and fmt-check find the
pinned yarn under nvm, and the image's last stage marks /app safe for
git. The build-context test expects the new pattern forms.

Judgement call: script/precommit runs lint and fmt-check without the tests, so the README's red-phase test commit can still land; the policy allows that form only where local testing is impossible.
Not applicable: the issue's .golangci.yml entries; this TypeScript repository has none.

Model: opus-5-5
2026-10-06 10:30:29 +02:00
clawbot 2483c85321 quak backup writes the account and album records backup-metadata writes (closes #166)
check / check (push) Successful in 1m23s
quak backup now writes account.json with the account's email and user ID,
adds each album's ownerID, isShared, updationTime and, when present, its
magicMetadata, pubMagicMetadata and sharedMagicMetadata to the album's
JSON, and adds updationTime to each file's JSON. The fields and their order
match backup-metadata's account.json, _collection.json and per-file JSON.
The interface runBackup drives gains whoami, which the library passes
through from its client.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-06 07:13:24 +02:00
clawbot ab4d5d2cc2 quak backup writes each file's ML data into its JSON (closes #163)
check / check (push) Successful in 1m37s
lib.backup() now waits for an ML data fetch, joining the one its refresh
started or starting one, before it writes the per-file JSON, and each
file's JSON carries the cached payload as mlData. When the fetch fails,
each file with no cached ML data gets mlDataError and an entry in
failures.json, so the result counts it as failed and quak backup exits 1;
the next run fetches again.

Judgement call: the wait comes after the originals are downloaded, so the fetch runs alongside the downloads.
Judgement call: a failed ML fetch is recorded per file in failures.json, which is how the exit code goes non-zero without changing src/cli-commands.ts.

Model: opus-5-5
2026-10-06 05:13:27 +02:00
clawbot 9e94542a57 docker build . stamps the git tag or short commit, not dev (closes #154)
check / check (push) Failing after 46s
A plain `docker build .` now stamps the version from git rather than `dev`/`0.0.0`. After `tsc`, `script/build` writes into `dist/package.json` the version `script/version` decides: `VERSION` when given, otherwise `git describe --tags --always` (the tag; or tag, commits since and short commit; or the short commit), otherwise `package.json`'s. A checkout with `.git` that yields an empty, `dev` or `unknown` version fails the build. `.dockerignore` sends `.git` but not `.git/config`, so no remote URL or credential reaches the image. `ARG VERSION` has no default, and the host scripts' version still wins.

Not changed: `REPO_POLICIES.md` still says `ARG VERSION=dev` until the shared policy changes.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-02 06:12:56 +02:00
clawbot e50d2a78c8 exif() returns every EXIF tag in the file (closes #156)
check / check (push) Failing after 59s
`photo.exif()` now returns every EXIF tag in the file, as the owner ruled, typed `ExifTags`: each tag keyed by name with `exifreader`'s `id`, `value`, `description` and `computed`. A tag with no name is keyed `undefined-` plus its number, and the embedded thumbnail's tags sit under `Thumbnail`. The thirteen typed methods stay, with the same names and types; each now picks its field from `exif()`'s tags. GPS latitude and longitude are worked out from the GPS tags and their reference tags, and a position with no reference tags gives neither. `backup-metadata --exif` output is unchanged.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-02 05:30:05 +02:00
clawbot 0c995a8c4f Remove the content cache's handling of an earlier version's live-photo ZIP (closes #151)
check / check (push) Failing after 44s
quak is pre-1.0 and keeps no handling of old data. The content cache no longer recognises or removes a live photo that an earlier quak version cached as one ZIP. A live-photo download no longer removes what was at its destination before renaming the image and video into place; the rename already replaces it. The README sentences and the tests about that old ZIP are gone. The two removed tests that also covered current behaviour are replaced by tests with no ZIP: the cache re-fetching a live photo recorded with no video, and the library telling the cache which files are live photos when it opens.

Model: opus-5-5
2026-10-02 04:14:30 +02:00
clawbot 10afa7a7f4 Example script: download every album's photos and metadata (closes #144)
check / check (push) Successful in 1m20s
`examples/download-albums.ts` logs in from `QUAK_EMAIL` and `QUAK_PASSWORD`, then walks every album from `await lib.fresh()`. For each photo it runs `photo.download()` to the photo's save path and writes `{savePath}.json`, which holds the photo's record and EXIF. For each album it writes `{dir}/albums/{collectionID}.json` with the album's name and its photos' save paths. A file is written only when its content changes, so a second run downloads and rewrites nothing.

The script opens the library with prefetching off, as `quak backup` does. The build type-checks `examples/`. The README says how to run it.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-02 00:33:00 +02:00
clawbot d40f339c0b Photo: one async method per EXIF field (closes #148)
check / check (push) Successful in 1m37s
`Photo` gains thirteen async methods, one per `PhotoExif` field and named after it: `make()`, `model()`, `lensModel()`, `dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`, `iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()` and `gpsAltitude()`. Each calls `exif()` and returns its one field, or `undefined` when the file lacks it. `exif()` is unchanged.

`Photo` implements a type built from `PhotoExif`'s keys, so the type check fails when a field has no method. Each call reads the original again; a caller that wants several fields calls `exif()` once.

Model: opus-5-5
2026-10-02 00:09:24 +02:00
clawbot 10e1a9ef39 Save path ./photos/YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.fileID.ext; download() from the cache first (closes #143)
check / check (push) Successful in 1m25s
Originals are saved at `{downloadDirectory}/YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.{fileID}{ext}`. The date is the photo's `takenAt` in local time, and `downloadDirectory` defaults to `./photos`, resolved when the library opens. The old `originals/` layout is gone.

`photo.download()` writes the original to `savePath`. It copies from the cache when the cache holds the original, and fetches otherwise. `lib.backup()` uses the same path and rule, and every album in `collections/` links to it. `isLocal` is true only when the original is at `savePath`.

For a file in several albums, one rule picks the copy everything uses: the most recently synced, with the lowest album ID breaking a tie.

Model: opus-5-5
2026-10-01 23:20:58 +02:00
clawbot 67d554fb46 exif(): read HEIF/HEIC originals with exifreader (closes #145)
check / check (push) Successful in 2m10s
`photo.exif()` and `quak backup-metadata --exif` now read EXIF through `exifreader`. HEIC/HEIF originals get EXIF, including a live photo's image, as do the other formats `exifreader` reads. It replaces `exif-reader` and the hand-written JPEG scan. `PhotoExif` is unchanged.

The `backup-metadata` dump now holds `exifreader`'s tag output, with unnamed tags keyed `undefined-` plus their number. GPS altitude without a reference counts as above sea level. A latitude or longitude without its hemisphere tag, an unreadable text tag, and a date the parser rejects each give no field.

Licence: `exifreader` is MPL-2.0, used unmodified.

Model: opus-5-5
2026-10-01 22:34:09 +02:00
clawbot 2b598d3622 Photo: save path, is-local, content bytes, metadata and EXIF getters (closes #141)
check / check (push) Successful in 1m24s
`Photo` gains:

- `savePath` and `isLocal`: synchronous, disk only. Where `lib.backup()` writes the original under the library's `downloadDirectory`, and whether all of it is there; a copy only in the cache does not count.
- `content()` and `exif()`: async, may download. `exif()` reads the common EXIF fields of a JPEG and returns `{}` for anything else.
- The getters `modifiedAt` and `hash`, also on `PhotoRecord`, and `year`.

For a live photo the backup has not stored yet, `savePath` carries the title's extension, and the image may be stored under a different one. The JPEG EXIF scan moved to `src/exif.ts`. The exported `PhotoContent` interface gains `savePath` and `isLocal`.

Judgement call: `iso` is read only when the file stores it as a single number.

Model: opus-5-5
2026-10-01 17:58:23 +02:00
clawbot c27e2cb629 README development workflow: branch from and merge into next (closes #135)
check / check (push) Successful in 1m0s
"Development workflow" and "For LLMs" said work branches off main and merges into main. They now say work branches from next, every pull request targets next, the repository manager squash-merges reviewed pull requests into next once make check is green, and only sneak merges next into main. The rest of both sections is unchanged.

Docs only; merged under the docs-only rule without an adversarial review.

Model: opus-5-5
2026-09-29 05:38:36 +02:00
clawbot e6a9e929c2 Bring README and TODO.md in line with next after the milestone merge (closes #132)
check / check (push) Successful in 1m10s
README.md and TODO.md were read end to end against the code after the milestone merge, and every sentence the code contradicted was corrected. The corrections cover the login and key-derivation steps (the TOTP step, email OTP, the SRP library), retries and download staging, which CLI commands take which options, which of each file's metadata the backup keeps, how default and fresh reads behave, the cache options, and what the tests cover. TODO.md's next step now says no implementation work is open and the cache design waits on sneak.

Docs only.
Left as written: the development workflow's `main` base, a process question.
Merged under the docs-only rule after four reviews; the fix for the fourth review's one finding was not re-reviewed.

Model: opus-5-5
2026-09-29 05:22:57 +02:00
clawbot 9e6deb21eb Store live photos as their image and their video (closes #107)
check / check (push) Successful in 41s
A live photo, which Ente stores as one ZIP, is unpacked as it downloads
into its image and its video, each `<fileID>.<ext>` with its extension
from the ZIP, beside `<fileID>.livephoto.json`, which names the two.
Both are checked against the recorded hash and renamed into place only
when both are complete. A ZIP with a second image or video, or whose
parts come to more than 20 times its size plus 16 MiB, is refused. The
backup and the content cache count a live photo as stored only with both
files, album folders link both, `quak get` writes both, and the content
result gives the video as `videoPath`. A ZIP an earlier version stored
is replaced.

Model: opus-5-5
2026-09-28 18:06:01 +02:00
clawbot 6a7a10f489 Bring README and TODO.md in line with the tree on next2 (closes #111)
check / check (push) Successful in 57s
The README layout lists src/library/ and the other source files, the
backup layout names failures.json and the optional thumbnails/, "Opening
a library" says an empty cache opens with no data when the first refresh
fails, and the Testing section gives the 60-second hard cap and 20-second
target for make test, with the 90-second timeout in the Dockerfile's test
phase as the backstop that catches a hung test. The next step in both
files is storing live photos
(#107) instead of a v1.0.0 tag;
tagging and releases are sneak's call alone.

Model: opus-5-5
2026-09-23 10:08:03 +02:00
clawbot ae76eb3f74 Write each backed-up original once and skip the precache (closes #106)
check / check (push) Successful in 49s
An original fetched for a backup is now written by the download writer
straight into the backup's originals/, and the content cache records it
there instead of keeping its own copy. One the cache already held is
still copied. `quak backup` opens its library with the thumbnail and
originals precache off, as the one-shot commands do.

Model: opus-5-5
2026-09-23 08:48:02 +02:00
clawbot f52c77f155 Refresh before backup-metadata and the thumbnail helpers answer (closes #100)
check / check (push) Successful in 1m0s
backup-metadata, helper list-missing-thumbnails and helper
fix-missing-thumbnails now await lib.fresh() before reading the library,
as collections, files, get and get-thumb already do. A file added since
the cache was written is included, and a failed refresh is thrown, so
the CLI prints one line and exits 1 instead of answering from a stale
or empty cache. The README lists them among the commands that refresh
first.

Model: opus-5-5
2026-09-23 08:11:37 +02:00
clawbot fc396d1ecc Keep another process's downloads when opening a library (closes #105)
check / check (push) Successful in 1m18s
The download writer's temp files are now named .quak-<pid>-<random>.tmp.
removeLeftoverTempFiles moves from the backup into the download module and
deletes a .quak-*.tmp file only when the process ID in its name is no
longer running; the content cache calls it at open() instead of deleting
every temp file, so a download in progress in another process sharing the
cache survives. The README backup layout and TODO.md are updated.

Model: opus-5-5
2026-09-23 07:25:28 +02:00
clawbot cb61582ae6 Make lint and test phases of the Dockerfile (closes #96)
check / check (push) Successful in 1m45s
Follows the template: Dockerfile.lint is gone; the Dockerfile has a lint
phase (eslint, prettier --check .) and a test phase (vitest, run as the
node user, which the not-writable-directory tests need), and its
last stage compiles and depends on both. script/lint and script/test
build one phase each with --no-cache; script/docker and script/cibuild
pass --no-cache, so CHECK_EPOCH and LINT_EPOCH are removed.
script/cibuild is the single image build, so CI runs lint and the tests
once each. The tests that checked the old layout are deleted,
REPO_POLICIES.md is re-copied and the README describes the new layout.

Model: opus-5-5
2026-09-23 07:18:46 +02:00
clawbot cda57eebda Skip thumbnail repairs the server always refuses (closes #109)
check / check (push) Successful in 49s
The server accepts a new thumbnail only from the file's owner and only when
it is no larger than the thumbnail size it records. Both thumbnail helpers
now skip files another account owns without fetching them. The fixer skips
a file whose recorded thumbnail size is 0 or unknown before downloading it,
and otherwise tries smaller encodings until the encrypted thumbnail fits,
skipping the file if none does.

Model: opus-5-5
2026-09-23 07:03:33 +02:00
clawbot 4bb75ca323 quak logout ends the session on the server (closes #108)
check / check (push) Successful in 54s
logout now calls POST /users/logout with the saved token through the new
Client.logoutOnServer(), then deletes session.json even when that call
fails; in that case it says the server session could not be ended and
exits 1. It prints the account's cache directory and says it still holds
decrypted data. The default cache path moves into
defaultCacheDirectory(), shared with Library.open, so both name the same
directory.

Model: opus-5-5
2026-09-23 06:51:12 +02:00
clawbot cd05a458dc Give backup album folders distinct names and remove stale entries (closes #103)
check / check (push) Successful in 32s
Within a collection's folder, files whose sanitized titles match (ignoring
case) each get their file ID added before the extension, and collections
whose sanitized names match get their ID added. This repeats until no name,
including one with an ID added, matches another, so no symlink or JSON
replaces another. Names are chosen across all collections, so a scoped run
names folders the same as a full one.

Each run first removes symlinks into originals/ that no longer belong to a
collection, and the folders quak wrote (a sibling JSON with an album ID) for
collections that are gone or renamed. Anything else is left alone; a folder
still holding user files keeps its JSON.

Model: opus-5-5
2026-09-23 06:36:12 +02:00
clawbot c19943a520 Check downloaded originals against their recorded content hash (closes #68)
check / check (push) Successful in 58s
downloadFile, shared by quak get, the content cache and backup, hashes
the decrypted bytes (unkeyed BLAKE2b-512, standard base64) and stores
nothing on a mismatch, failing with an error naming the file ID. A live
photo ZIP is unpacked as it streams with fflate's Unzip, in small
slices so memory stays bounded however far an entry expands, and its
image and video hashed separately as <imageHash>:<videoHash>.
decryptFile reads the older imageHash/videoHash fields for live
photos. A file with no recorded hash is stored unchecked.

Model: opus-5-5
2026-09-23 06:16:57 +02:00
clawbot 390401af2c Start empty when the cache directory holds another account's cache (closes #104)
check / check (push) Successful in 35s
When metadata.json in the cache directory was written for a different,
non-zero user ID than the client's, Library.open deletes it and mldata/
and loads an empty store, so the first refresh enumerates from 0 and none
of the other account's collections, files, keys or ML results are served.
Only reachable with --cache-dir or an explicit cacheDirectory.

Model: opus-5-5
2026-09-23 05:56:14 +02:00
clawbot bf3b20df2f backup-metadata: keep going when an ML data request fails (closes #101)
check / check (push) Successful in 27s
Each ML data request of up to 200 files is now tried on its own. A request
that still fails after its retries is logged, its files are written with the
reason in `mlDataError`, and the command exits 1 once the dump is complete.
`fetchMLData`, used only here, is removed in favour of a per-batch loop over
`fetchMLDataBatch`.

Model: opus-5-5
2026-09-23 05:49:38 +02:00
clawbot f1836ced57 Fix two intermittently failing library tests (closes #90)
check / check (push) Successful in 27s
Library.close() now returns a promise that resolves once the work it
started has finished: an in-flight refresh with its cache write, the ML
data fetch, and running precache sweeps. The interval test could see a
refresh's new state, close, and remove the directory while the write was
still running. Every library test and CLI command now awaits close(), and
tests hold each of the three writes open to prove close() waits for it.

The precache test waited for its stub source to be called, but the cache
records a file only after checking it on disk, so status() could lag. It
now waits for both fills to report "done".

Model: opus-5-5
2026-09-23 04:38:00 +02:00
clawbot 28a2beeab8 Make the download deadline an idle deadline and cancel failed bodies (closes #24)
check / check (push) Successful in 17s
downloadTimeoutMs now aborts a file or thumbnail download only when no
bytes have arrived for that long (default 60 s, was a 600 s cap on the
whole transfer), so a slow download that keeps making progress completes.
The abort reason is still a TimeoutError, so retry classification is
unchanged. A download that fails before reading the whole response body
cancels it, including when the temp file cannot be opened or the header
is malformed, so a failed file no longer holds its connection.

Model: opus-5-5
2026-09-23 03:14:42 +02:00
clawbot ed535be1da Harden the backup tree's atomic copy (closes #22)
check / check (push) Successful in 27s
The backup copy now fsyncs its temp file before the rename and the
directory after it, using the download writer's new fsyncPath helper.
Each backup run deletes .quak-backup-*.tmp files whose process is no
longer running, leaving those of a concurrent backup alone. The rename
sites and the README backup layout state that a symlink at the
destination is replaced and the new file takes the temp file's
permissions, and the README names the temp files. Adds tests for a
missing and an unwritable destination directory for downloadFile and
downloadThumbnail.

Model: opus-5-5
2026-09-23 02:44:46 +02:00
clawbot d545dcd8b1 Harden the retry classifier and pin per-attempt deadlines (closes #80)
check / check (push) Successful in 42s
A POST or PUT is replayed only when every errno in the cause chain is a
connect errno and the walk reached the end of the chain, and
postJSON/putJSON no longer follow redirects, so a redirect is an
ApiError that is not retried. getRetryOptions() returns a copy. New
tests pin every errno the classifier names, the cause-chain depth
limit, a chain deeper than the limit, a two-error cycle, and a fresh
deadline per attempt for every retrying entry point. The README
endpoint list is now the one place naming the requests the replay rule
covers; code comments point to it.

Model: opus-5-5
2026-09-23 02:38:01 +02:00
clawbot b7d6ab99f4 Validate session snapshots and wipe keys on logout (closes #10)
check / check (push) Successful in 41s
Client.fromJSON checks every snapshot field and each key's decoded length
and throws an error naming the bad field. toJSON reads the token through a
new ApiClient.getAuthToken and throws when there is none. logout zeroes the
key buffers in place; collectionsSince re-checks for logout after its
request so it never decrypts with zeroed keys. The CLI now reports a
corrupt session file separately from a missing one.

Model: opus-5-5
2026-09-23 02:08:02 +02:00
clawbot fe952d3e62 Rewrite the README API reference for the new library surface (closes #53)
check / check (push) Successful in 28s
Rewrites the README API reference to match the shipped library surface: Library.open options (XDG cache dir + userID, downloadDirectory, refreshIntervalSeconds, precache and cache-size options), default vs fresh reads (Library.fresh()), snapshot/subscribe, albums/photos/timeline, Photo.original/thumbnail, thumbnails.ensure, mldata search, backup, the request pools and on-disk layout, and the CLI (--cache-dir, fresh reads). Every documented signature verified against the code. Notes the deferred content-hash integrity check (#68). Also issue #13.

Model: opus-4-8
2026-09-23 00:27:30 +02:00
clawbot d23d3f8f47 Port the quak CLI to the library API (closes #52)
check / check (push) Successful in 17s
Ports the CLI to the library API. collections/files/get/get-thumb use the fresh read variants (Library.fresh(), current server state); backup runs lib.backup; backup-metadata and the missing-thumbnail helpers enumerate via the library. files/get output is byte-identical to the pre-port CLI — raw metadata.title, microsecond creationTime, pre-port row order — exit codes unchanged. Adds --cache-dir; fixes the helper JPEG-only assumption (closes #17).

Model: opus-4-8
2026-09-23 00:00:55 +02:00
clawbot d1d6cdd4f0 Revert the 16 commits pushed to next on 2026-09-04 by an agent outside the managed fleet
check / check (push) Successful in 1m40s
sneak, 2026-09-05: "inference instance stopped. undo its rogue work." The
reverted commits stay in history; nothing else on next is touched.

Model: fable-5-1
2026-09-05 09:33:23 +00:00
user ffc817522e test(lint-once): pin the two-build and later-flag shapes
check / check (push) Successful in 15s
The false green: script/lint chaining a second docker build after the
lint image ran prettier twice and counted once. The misresolution: a
bare docker build followed by cp -f resolved to the cp's file. Pin
both, and each separator that bounds an invocation.
2026-09-04 11:51:17 +00:00
user 88a5fcaa87 lint-once: bound the docker file flag to its own invocation
check / check (push) Failing after 42s
edgesOf used a single test/exec for docker build, so a line with two
builds produced one edge and the file flag was searched across the whole
line. Follow every occurrence with matchAll and slice each one to the
next shell separator before looking for its flag.

Also correct the README's claim that script/fmt-check is the one
formatting path left on the host.
2026-09-04 11:49:38 +00:00
user bd88eced84 README: state the container-lint claim as what is actually true
check / check (push) Successful in 1m14s
The Linting section this PR adds claimed "There is no host lint path" and
then, two paragraphs later, documented script/fmt-check as a host-side
formatting check. Both cannot be true, and the absolute one is the false one.

What is true is narrower: no lint path reachable from script/check or
script/precommit runs on the host, so every lint verdict those two produce
comes from the container. script/fmt-check stays as a standalone entrypoint,
now stated as the one host formatting path with nothing reaching it. The
"exactly one place" and "in the container only" phrasings elsewhere are
qualified the same way.
2026-09-04 11:27:49 +00:00
sneak a73f0abbe8 Check formatting once per make check, in the container (closes #29)
check / check (push) Successful in 59s
script/check ran script/test, script/lint and script/fmt-check. Since
linting moved into Docker, script/lint is a build of Dockerfile.lint,
which runs `prettier --check .` as a build step — so make check checked
formatting twice over the same tree: once in the container and once on
the host. script/precommit had the same pair.

Drop the script/fmt-check call from both. The container keeps the check,
because a successful Dockerfile.lint build is what CI treats as proof of
a clean tree, and it is the stronger of the two verdicts: its prettier is
digest-pinned and installed under --frozen-lockfile, while the host's is
whatever the working tree happens to have. The pre-commit hook is
unchanged in what it catches — script/lint still fails a badly formatted
tree, and therefore the commit.

script/fmt-check survives as a standalone entrypoint, as REPO_POLICIES.md
requires, for asking the formatting question by itself without docker.
Its verdict cannot drift from the container's: prettier is pinned to an
exact version, installed from yarn.lock in both places, and reads
.gitignore as its default ignore file, which is why .dockerignore keeps
.gitignore in the build context.

The count is asserted rather than promised. test/packaging/lint-once.test.ts
walks the invocation graph from each entrypoint — through the Makefile
shims, the script/ calls, the package.json scripts and the docker build
into Dockerfile.lint's RUN steps — and counts prettier invocations: one
per make check, one per script/precommit, and one each for make lint and
make fmt-check alone, so neither can become a no-op that satisfies the
count trivially. The walk also asserts which nodes it reached, so a
restructure that defeats the resolver fails the test instead of quietly
counting zero.

Observed: 2 prettier invocations per make check before, 1 after.
2026-08-10 13:05:03 +00:00
sneak fed39d19cf Run all linting in Docker via Dockerfile.lint (closes #30)
check / check (push) Successful in 1m2s
Linting now happens in one place only: a new root Dockerfile.lint copies
the repo into the digest-pinned node image already used by Dockerfile and
runs eslint and prettier as build steps, so a successful build is a clean
lint. script/lint is reduced to building it, which also works where the
docker daemon is remote and bind mounts are impossible. No host lint path
survives: the "lint" script is gone from package.json, so there is no
second, unpinned way to get a lint verdict.

Caching is waived for lint, because a lint build over an unchanged tree
returns success in well under a second having linted nothing. LINT_EPOCH
is the cache buster and it fails closed exactly as CHECK_EPOCH does: an
unset ARG is the empty string, which is a perfectly stable cache key, so
the guard rejects it and a bare `docker build -f Dockerfile.lint .` errors
out instead of serving a green it did not earn. Both linters sit below the
guard, so a fresh epoch forces them to execute while the bootstrap and
dependency layers above stay cached.

That makes script/lint a docker build, which nothing inside a container
may call. script/check calls script/lint, so the Dockerfile image can no
longer run make check: the lint stage and its COPY --from=lint ordering
hack are deleted, and the remaining stage runs make test and make build
under the existing CHECK_EPOCH guard. script/cibuild is now the composite
gate and builds the lint image first, so a lint failure is reported before
the slower suite runs.

The .dockerignore exclusions are unchanged and still apply to the lint
build, including the .claude/ exclusion (eslint's flat config does not
ignore dot-directories, so a nested worktree in the context would be
linted) and the deliberate exception that keeps .gitignore in the context
for prettier. A new test asserts no per-Dockerfile ignore file shadows the
root one for either image, and test/packaging/lint-docker.test.ts asserts
the whole shape: the docker-only lint path, the digest pin, manifests
copied before sources, the fail-closed guard with both linters below it,
the absence of a lint stage or make check in Dockerfile, and the build
order in script/cibuild.
2026-08-10 12:39:23 +00:00