lib.backup() starts no further original once its lock is lost
check / check (push) Waiting to run

Issue 184 checked the backup's lock before each original, in the loop
this branch replaces with several workers. Each worker now checks the
lock before it takes the next file, so once the lock is lost no further
original starts; the originals under way are finished, and the backup
rejects once every worker has stopped. The README says the originals
under way, not the one, are still stored.

Model: opus-5-5
This commit is contained in:
2026-10-14 05:46:39 +00:00
parent 75463c834c
commit 0f7bcb3553
3 changed files with 13 additions and 8 deletions
+3 -3
View File
@@ -779,9 +779,9 @@ A running backup loses its lock when someone removes `backup.lock`, or when
another backup takes it over because the running one left it untouched for 10
seconds, as a stopped or stalled process does. The backup finds this out the
next time it goes to touch the lock, which it does every 5 seconds, and then
stops before it starts another file. The original it is downloading at that
moment is still stored. It leaves `failures.json` and `backup.lock` as they are,
since another backup may be using them. `quak backup` prints
stops before it starts another file. The originals it is downloading at that
moment are still stored. It leaves `failures.json` and `backup.lock` as they
are, since another backup may be using them. `quak backup` prints
`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and
`lib.backup()` rejects with `lost the lock on <dir>: <reason>`.
+2 -1
View File
@@ -31,7 +31,8 @@ declares one.
the work for one original, its `--verify` hashing included, so no more
originals than that are open at once. `BackupLibrary` says the limit as
`contentConcurrency`. A test holds each download at the fake Ente server to
count those in flight.
count those in flight. Each worker checks the backup's lock before it takes
the next file, so once the lock is lost no further original starts.
- 2026-10-14: A backup whose lock is lost no longer dies with a stack trace
(issue 184). `lockBackupDirectory` gives proper-lockfile an `onCompromised`
+8 -4
View File
@@ -730,13 +730,17 @@ const runLockedBackup = async (
// Up to the content pool's limit of originals are worked on at once,
// their `verify` hashing included, so however large the account is, no
// more than that many are open. Each worker takes the next file from the
// iterator they all share until none is left. A worker that throws, as
// one does when `onProgress` throws, fails the backup only once every
// worker has stopped, so none writes into the backup after it has ended.
// iterator they all share until none is left, and throws instead once the
// lock is lost. A worker that throws, as one also does when `onProgress`
// throws, fails the backup only once every worker has stopped, so none
// writes into the backup after it has ended.
if (includeOriginals) {
const pending = distinct.values();
const worker = async (): Promise<void> => {
for (const file of pending) await storeOriginal(file);
for (const file of pending) {
if (lock.lost) throw lock.lost;
await storeOriginal(file);
}
};
const workers = await Promise.allSettled(
Array.from({ length: lib.contentConcurrency }, worker),