lib.backup() starts no further original once its lock is lost
Issue 184 checked the backup's lock before each original, in the loop this branch replaces with several workers. Each worker now checks the lock before it takes the next file, so once the lock is lost no further original starts; the originals under way are finished, and the backup rejects once every worker has stopped. The README says the originals under way, not the one, are still stored. Model: opus-5-5
This commit is contained in:
@@ -779,9 +779,9 @@ A running backup loses its lock when someone removes `backup.lock`, or when
|
|||||||
another backup takes it over because the running one left it untouched for 10
|
another backup takes it over because the running one left it untouched for 10
|
||||||
seconds, as a stopped or stalled process does. The backup finds this out the
|
seconds, as a stopped or stalled process does. The backup finds this out the
|
||||||
next time it goes to touch the lock, which it does every 5 seconds, and then
|
next time it goes to touch the lock, which it does every 5 seconds, and then
|
||||||
stops before it starts another file. The original it is downloading at that
|
stops before it starts another file. The originals it is downloading at that
|
||||||
moment is still stored. It leaves `failures.json` and `backup.lock` as they are,
|
moment are still stored. It leaves `failures.json` and `backup.lock` as they
|
||||||
since another backup may be using them. `quak backup` prints
|
are, since another backup may be using them. `quak backup` prints
|
||||||
`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and
|
`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and
|
||||||
`lib.backup()` rejects with `lost the lock on <dir>: <reason>`.
|
`lib.backup()` rejects with `lost the lock on <dir>: <reason>`.
|
||||||
|
|
||||||
|
|||||||
@@ -32,7 +32,8 @@ declares one.
|
|||||||
the work for one original, its `--verify` hashing included, so no more
|
the work for one original, its `--verify` hashing included, so no more
|
||||||
originals than that are open at once. `BackupLibrary` says the limit as
|
originals than that are open at once. `BackupLibrary` says the limit as
|
||||||
`contentConcurrency`. A test holds each download at the fake Ente server to
|
`contentConcurrency`. A test holds each download at the fake Ente server to
|
||||||
count those in flight.
|
count those in flight. Each worker checks the backup's lock before it takes
|
||||||
|
the next file, so once the lock is lost no further original starts.
|
||||||
|
|
||||||
- 2026-10-14: Next Step said the cache design, issue 36, waits on sneak's
|
- 2026-10-14: Next Step said the cache design, issue 36, waits on sneak's
|
||||||
review, though that issue closed on 2026-10-03 (issue 194). It now names the
|
review, though that issue closed on 2026-10-03 (issue 194). It now names the
|
||||||
|
|||||||
+8
-4
@@ -730,13 +730,17 @@ const runLockedBackup = async (
|
|||||||
// Up to the content pool's limit of originals are worked on at once,
|
// Up to the content pool's limit of originals are worked on at once,
|
||||||
// their `verify` hashing included, so however large the account is, no
|
// their `verify` hashing included, so however large the account is, no
|
||||||
// more than that many are open. Each worker takes the next file from the
|
// more than that many are open. Each worker takes the next file from the
|
||||||
// iterator they all share until none is left. A worker that throws, as
|
// iterator they all share until none is left, and throws instead once the
|
||||||
// one does when `onProgress` throws, fails the backup only once every
|
// lock is lost. A worker that throws, as one also does when `onProgress`
|
||||||
// worker has stopped, so none writes into the backup after it has ended.
|
// throws, fails the backup only once every worker has stopped, so none
|
||||||
|
// writes into the backup after it has ended.
|
||||||
if (includeOriginals) {
|
if (includeOriginals) {
|
||||||
const pending = distinct.values();
|
const pending = distinct.values();
|
||||||
const worker = async (): Promise<void> => {
|
const worker = async (): Promise<void> => {
|
||||||
for (const file of pending) await storeOriginal(file);
|
for (const file of pending) {
|
||||||
|
if (lock.lost) throw lock.lost;
|
||||||
|
await storeOriginal(file);
|
||||||
|
}
|
||||||
};
|
};
|
||||||
const workers = await Promise.allSettled(
|
const workers = await Promise.allSettled(
|
||||||
Array.from({ length: lib.contentConcurrency }, worker),
|
Array.from({ length: lib.contentConcurrency }, worker),
|
||||||
|
|||||||
Reference in New Issue
Block a user