lib.backup() starts no further original once its lock is lost

Issue 184 checked the backup's lock before each original, in the loop
this branch replaces with several workers. Each worker now checks the
lock before it takes the next file, so once the lock is lost no further
original starts; the originals under way are finished, and the backup
rejects once every worker has stopped. The README says the originals
under way, not the one, are still stored.

Model: opus-5-5
This commit is contained in:
2026-10-14 06:34:09 +00:00
parent 7f9bf5eb90
commit d95c49e548
3 changed files with 13 additions and 8 deletions
+3 -3
View File
@@ -779,9 +779,9 @@ A running backup loses its lock when someone removes `backup.lock`, or when
another backup takes it over because the running one left it untouched for 10 another backup takes it over because the running one left it untouched for 10
seconds, as a stopped or stalled process does. The backup finds this out the seconds, as a stopped or stalled process does. The backup finds this out the
next time it goes to touch the lock, which it does every 5 seconds, and then next time it goes to touch the lock, which it does every 5 seconds, and then
stops before it starts another file. The original it is downloading at that stops before it starts another file. The originals it is downloading at that
moment is still stored. It leaves `failures.json` and `backup.lock` as they are, moment are still stored. It leaves `failures.json` and `backup.lock` as they
since another backup may be using them. `quak backup` prints are, since another backup may be using them. `quak backup` prints
`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and `quak: lost the lock on <dir>: <reason>` as one line and exits 1, and
`lib.backup()` rejects with `lost the lock on <dir>: <reason>`. `lib.backup()` rejects with `lost the lock on <dir>: <reason>`.
+2 -1
View File
@@ -32,7 +32,8 @@ declares one.
the work for one original, its `--verify` hashing included, so no more the work for one original, its `--verify` hashing included, so no more
originals than that are open at once. `BackupLibrary` says the limit as originals than that are open at once. `BackupLibrary` says the limit as
`contentConcurrency`. A test holds each download at the fake Ente server to `contentConcurrency`. A test holds each download at the fake Ente server to
count those in flight. count those in flight. Each worker checks the backup's lock before it takes
the next file, so once the lock is lost no further original starts.
- 2026-10-14: Next Step said the cache design, issue 36, waits on sneak's - 2026-10-14: Next Step said the cache design, issue 36, waits on sneak's
review, though that issue closed on 2026-10-03 (issue 194). It now names the review, though that issue closed on 2026-10-03 (issue 194). It now names the
+8 -4
View File
@@ -730,13 +730,17 @@ const runLockedBackup = async (
// Up to the content pool's limit of originals are worked on at once, // Up to the content pool's limit of originals are worked on at once,
// their `verify` hashing included, so however large the account is, no // their `verify` hashing included, so however large the account is, no
// more than that many are open. Each worker takes the next file from the // more than that many are open. Each worker takes the next file from the
// iterator they all share until none is left. A worker that throws, as // iterator they all share until none is left, and throws instead once the
// one does when `onProgress` throws, fails the backup only once every // lock is lost. A worker that throws, as one also does when `onProgress`
// worker has stopped, so none writes into the backup after it has ended. // throws, fails the backup only once every worker has stopped, so none
// writes into the backup after it has ended.
if (includeOriginals) { if (includeOriginals) {
const pending = distinct.values(); const pending = distinct.values();
const worker = async (): Promise<void> => { const worker = async (): Promise<void> => {
for (const file of pending) await storeOriginal(file); for (const file of pending) {
if (lock.lost) throw lock.lost;
await storeOriginal(file);
}
}; };
const workers = await Promise.allSettled( const workers = await Promise.allSettled(
Array.from({ length: lib.contentConcurrency }, worker), Array.from({ length: lib.contentConcurrency }, worker),