Rewrite the backup command on the library API with a durable failure ledger #51

New Issue

2026-09-22T09:11:12+02:00

clawbot commented

2026-09-22 09:11:12 +02:00

Cache/API implementation (#36), phase 5.
Depends on the content/thumbnail-store and read-surface units.

Goal

Rewrite the backup command on the library API with a durable failure ledger,
keeping the external backup <dir> layout and exit-code contract unchanged.

Scope

lib.backup(args?: { downloadDirectory?; includeOriginals? (default true); includeThumbnails? (default false); onlyAlbumNames?; onProgress? })
→ BackupResult. Refresh, then for each pending file fetch the original
(and optionally thumbnail) through the pools, then materialise the
collections/ symlink views + per-collection JSON from the model.
downloadDirectory layout unchanged: originals/<fileID>.<ext> +
originals/<fileID>.json sidecars, collections/<name>/ symlinks,
collections/<name>.json. An original already in the cache is copied, not
re-downloaded. backup() with no downloadDirectory (neither in open() nor
the call) throws before any network traffic — it does not fall back to the
cache.
Crash-safety: present-means-complete for bytes; derived views (symlinks + JSON)
are rebuilt from the model each run, so they need no crash-safety and their
rebuild repairs stale sidecars and symlink failures. A symlink failure must not
abort the run (#8).
Durable failures.json: a file that fails after retries is recorded with
classification, attempt count, and last-tried time; the run continues; the
next run retries transient/unknown and can de-prioritise permanent. Exit
non-zero while unresolved failures remain. Replaces today's lost in-memory
errors[].
Port test/cli/backup.test.ts to the new implementation.

Definition of done

Backup produces today's on-disk layout; idempotent no-op on a
present+verified file; resumable after interruption via the model + directory
listing; symlink failure does not abort; failures.json durable; exit codes
preserved.
make check green. Subsumes #8; overlaps
#22 (orphan reaping) and
#9 (originals keyed by fileID remove the
filename hazard; the symlink view still sanitises titles).

Grounding

src/backup.ts (rewritten on the library), test/cli/backup.test.ts.

Dispatch notes: TDD; no scripted edits; no interactive questions; plain language.
Squash subject ends (closes #<this issue>). End every message with
Model: opus-4-8.

Model: opus-4-8

Cache/API implementation (https://git.eeqj.de/sneak/quak/issues/36), phase 5. Depends on the content/thumbnail-store and read-surface units. ## Goal Rewrite the backup command on the library API with a durable failure ledger, keeping the external `backup <dir>` layout and exit-code contract unchanged. ## Scope - `lib.backup(args?: { downloadDirectory?; includeOriginals? (default true); includeThumbnails? (default false); onlyAlbumNames?; onProgress? })` → `BackupResult`. Refresh, then for each pending file fetch the original (and optionally thumbnail) through the pools, then materialise the `collections/` symlink views + per-collection JSON from the model. - `downloadDirectory` layout unchanged: `originals/<fileID>.<ext>` + `originals/<fileID>.json` sidecars, `collections/<name>/` symlinks, `collections/<name>.json`. An original already in the cache is copied, not re-downloaded. `backup()` with no `downloadDirectory` (neither in `open()` nor the call) throws before any network traffic — it does not fall back to the cache. - Crash-safety: present-means-complete for bytes; derived views (symlinks + JSON) are rebuilt from the model each run, so they need no crash-safety and their rebuild repairs stale sidecars and symlink failures. A symlink failure must not abort the run (https://git.eeqj.de/sneak/quak/issues/8). - Durable `failures.json`: a file that fails after retries is recorded with classification, attempt count, and last-tried time; the run continues; the next run retries transient/unknown and can de-prioritise permanent. Exit non-zero while unresolved failures remain. Replaces today's lost in-memory `errors[]`. - Port `test/cli/backup.test.ts` to the new implementation. ## Definition of done - Backup produces today's on-disk layout; idempotent no-op on a present+verified file; resumable after interruption via the model + directory listing; symlink failure does not abort; `failures.json` durable; exit codes preserved. - `make check` green. Subsumes https://git.eeqj.de/sneak/quak/issues/8; overlaps https://git.eeqj.de/sneak/quak/issues/22 (orphan reaping) and https://git.eeqj.de/sneak/quak/issues/9 (originals keyed by fileID remove the filename hazard; the symlink view still sanitises titles). ## Grounding `src/backup.ts` (rewritten on the library), `test/cli/backup.test.ts`. Dispatch notes: TDD; no scripted edits; no interactive questions; plain language. Squash subject ends ` (closes #<this issue>)`. End every message with `Model: opus-4-8`. Model: opus-4-8

clawbot referenced this issue

2026-09-22 09:11:56 +02:00

Design for review: a cache layer (SQLite metadata + on-disk content/thumbnail store) and a crash-safe backup on it #36

clawbot referenced this issue from a commit

2026-09-22 18:55:48 +02:00

Rebuild backup on the library API with a durable failure ledger (closes #51)

clawbot referenced a pull request that will close this issue

2026-09-22 18:56:55 +02:00

Rebuild backup on the library API with a durable failure ledger #70

clawbot commented

2026-09-22 18:57:11 +02:00

Implemented in #70 (branch issue-51-backup-rewrite, base next).

lib.backup() refreshes, fetches pending originals (and, with includeThumbnails, thumbnails) through the content cache and pools, then rebuilds the per-file sidecars, per-collection symlink trees, and per-collection JSON from the model. The downloadDirectory layout and the exit-code contract are unchanged; a call with no download directory rejects before any network.

Present originals are treated as complete (idempotent, resumable from the directory listing); per-file download and symlink failures are recorded in a durable failures.json (classification, attempt count, last-tried) without aborting the run, subsuming #8. test/cli/backup.test.ts is ported onto the new API. make check and make build are green.

Model: opus-4-8

Implemented in https://git.eeqj.de/sneak/quak/pulls/70 (branch `issue-51-backup-rewrite`, base `next`). `lib.backup()` refreshes, fetches pending originals (and, with `includeThumbnails`, thumbnails) through the content cache and pools, then rebuilds the per-file sidecars, per-collection symlink trees, and per-collection JSON from the model. The `downloadDirectory` layout and the exit-code contract are unchanged; a call with no download directory rejects before any network. Present originals are treated as complete (idempotent, resumable from the directory listing); per-file download and symlink failures are recorded in a durable `failures.json` (classification, attempt count, last-tried) without aborting the run, subsuming https://git.eeqj.de/sneak/quak/issues/8. `test/cli/backup.test.ts` is ported onto the new API. `make check` and `make build` are green. Model: opus-4-8

clawbot referenced this issue from a commit

2026-09-22 19:21:25 +02:00

Rebuild backup on the library API with a durable failure ledger (closes #51)

clawbot referenced this issue from a commit

2026-09-22 20:05:49 +02:00

Rebuild backup on the library API with a durable failure ledger (closes #51)

clawbot closed this issue

2026-09-22 20:21:50 +02:00

clawbot referenced this issue from a commit

2026-09-22 20:21:50 +02:00

Rewrite the backup command on the library API with a durable failure ledger (closes #51)

Sign in to join this conversation.