quak backup downloads one original at a time #182

New Issue

2026-10-14T01:31:46+02:00

clawbot commented

2026-10-14 01:31:46 +02:00

Problem

The first loop of runLockedBackup in src/backup.ts awaits each original before it starts the next, so quak backup and lib.backup() download one file at a time. The content request pool allows 5 at once (RequestPools, configurable through LibraryOptions.pools), and the README says the pools set concurrency, but the backup never uses more than one slot. A first backup of a large account takes several times longer than it needs to, and a library user who raises contentConcurrency gets no faster backup.

Definition of done

The backup works on up to the content pool's limit of originals at once (5 by default; a library opened with other pools uses that limit). The limit covers everything done per original, including the --verify hashing, so the number of open files stays bounded however large the account is.
Everything else stays as it is: the counts in the result, failures.json, the exit code, the --verify rules (a mismatch is put back and hashed again), and one progress line per fetched or failed file.
Tests: with a content source that holds each fetch until released, the backup has the pool's limit of fetches in flight and never more; a failure in one does not stop the others; the result matches a run of the same files one at a time.
README: the "Backup layout" or "Request pools" text says the backup fetches up to the content pool's limit at once.

Notes for the implementer

Use the library's existing pool rather than a new helper or dependency; BackupLibrary may need to say what the limit is. Keep the loop readable for a newcomer.
Thumbnails (includeThumbnails) may stay one at a time.

Model: opus-5-5

## Problem The first loop of `runLockedBackup` in `src/backup.ts` awaits each original before it starts the next, so `quak backup` and `lib.backup()` download one file at a time. The content request pool allows 5 at once (`RequestPools`, configurable through `LibraryOptions.pools`), and the README says the pools set concurrency, but the backup never uses more than one slot. A first backup of a large account takes several times longer than it needs to, and a library user who raises `contentConcurrency` gets no faster backup. ## Definition of done - The backup works on up to the content pool's limit of originals at once (5 by default; a library opened with other `pools` uses that limit). The limit covers everything done per original, including the `--verify` hashing, so the number of open files stays bounded however large the account is. - Everything else stays as it is: the counts in the result, `failures.json`, the exit code, the `--verify` rules (a mismatch is put back and hashed again), and one progress line per fetched or failed file. - Tests: with a content source that holds each fetch until released, the backup has the pool's limit of fetches in flight and never more; a failure in one does not stop the others; the result matches a run of the same files one at a time. - README: the "Backup layout" or "Request pools" text says the backup fetches up to the content pool's limit at once. ## Notes for the implementer - Use the library's existing pool rather than a new helper or dependency; `BackupLibrary` may need to say what the limit is. Keep the loop readable for a newcomer. - Thumbnails (`includeThumbnails`) may stay one at a time. Model: opus-5-5

clawbot added this to the 1.0.0 milestone 2026-10-14 01:33:12 +02:00

clawbot referenced this issue from a commit

2026-10-14 05:16:19 +02:00

quak backup works on up to the content pool's limit of originals at once (closes #182)

clawbot referenced a pull request that will close this issue

2026-10-14 05:16:38 +02:00

quak backup works on up to the content pool's limit of originals at once (closes #182) #192

clawbot commented

2026-10-14 05:16:49 +02:00

Built in #192: lib.backup() and quak backup work on up to the content pool's limit of originals at once, their --verify hashing included; thumbnails stay one at a time.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/quak/pulls/192: `lib.backup()` and `quak backup` work on up to the content pool's limit of originals at once, their `--verify` hashing included; thumbnails stay one at a time. Model: opus-5-5

clawbot referenced this issue

2026-10-14 05:21:46 +02:00

A backup whose lock is lost stops with one line, not a stack trace (closes #184) #193

clawbot referenced this issue from a commit

2026-10-14 07:25:17 +02:00

quak backup works on up to the content pool's limit of originals at once (closes #182)

clawbot referenced this issue

2026-10-14 07:42:50 +02:00

TODO.md's Next Step names the open issues, not a closed one (closes #194) #195

clawbot referenced this issue

2026-10-14 07:42:55 +02:00

`TODO.md` says the cache design waits on sneak's review; that issue is closed #194