quak backup works on up to the content pool's limit of originals at once (closes #182)
check / check (push) Canceled after 0s

The backup awaited each original before starting the next, so it used
one of the content pool's five slots. lib.backup(), and so quak backup,
now runs as many workers as the content pool's limit (5 by default, or
the contentConcurrency of the pools the library was opened with), each
taking the next file from one shared list, so the limit covers the
--verify hashing too. Once a worker has thrown or the lock is lost, no
worker takes another file; the backup waits for the files under way and
checks its lock before the ML data fetch. runBackup rejects a
contentConcurrency that is not a positive integer before it takes the
lock. Failed files are listed in the order they failed.

Model: opus-5-5
This commit was merged in pull request #192.
This commit is contained in:
2026-10-14 09:06:59 +02:00
parent 22ee2db479
commit 3412e42af5
7 changed files with 763 additions and 83 deletions
+8 -3
View File
@@ -779,9 +779,9 @@ A running backup loses its lock when someone removes `backup.lock`, or when
another backup takes it over because the running one left it untouched for 10
seconds, as a stopped or stalled process does. The backup finds this out the
next time it goes to touch the lock, which it does every 5 seconds, and then
stops before it starts another file. The original it is downloading at that
moment is still stored. It leaves `failures.json` and `backup.lock` as they are,
since another backup may be using them. `quak backup` prints
stops before it starts another file. The originals it is downloading at that
moment are still stored. It leaves `failures.json` and `backup.lock` as they
are, since another backup may be using them. `quak backup` prints
`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and
`lib.backup()` rejects with `lost the lock on <dir>: <reason>`.
@@ -1096,6 +1096,11 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
Each pool orders on-demand work ahead of background/precache work and dedups
in-flight fetches by key, and an idle pool never lends its slots to a busy one.
`lib.backup()`, and so `quak backup`, works on up to the content pool's limit of
originals at once, 5 unless the library was opened with other `pools`. The limit
covers everything it does for one original, hashing it for `verify` included, so
however large the account is, no more originals than that are open at once.
### On-disk cache layout
Under `cacheDirectory`: