quak backup works on up to the content pool's limit of originals at once (closes #182) #192
@@ -779,9 +779,9 @@ A running backup loses its lock when someone removes `backup.lock`, or when
|
||||
another backup takes it over because the running one left it untouched for 10
|
||||
seconds, as a stopped or stalled process does. The backup finds this out the
|
||||
next time it goes to touch the lock, which it does every 5 seconds, and then
|
||||
stops before it starts another file. The original it is downloading at that
|
||||
moment is still stored. It leaves `failures.json` and `backup.lock` as they are,
|
||||
since another backup may be using them. `quak backup` prints
|
||||
stops before it starts another file. The originals it is downloading at that
|
||||
moment are still stored. It leaves `failures.json` and `backup.lock` as they
|
||||
are, since another backup may be using them. `quak backup` prints
|
||||
`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and
|
||||
`lib.backup()` rejects with `lost the lock on <dir>: <reason>`.
|
||||
|
||||
|
||||
@@ -32,7 +32,8 @@ declares one.
|
||||
the work for one original, its `--verify` hashing included, so no more
|
||||
originals than that are open at once. `BackupLibrary` says the limit as
|
||||
`contentConcurrency`. A test holds each download at the fake Ente server to
|
||||
count those in flight.
|
||||
count those in flight. Each worker checks the backup's lock before it takes
|
||||
the next file, so once the lock is lost no further original starts.
|
||||
|
||||
- 2026-10-14: Next Step said the cache design, issue 36, waits on sneak's
|
||||
review, though that issue closed on 2026-10-03 (issue 194). It now names the
|
||||
|
||||
+8
-4
@@ -730,13 +730,17 @@ const runLockedBackup = async (
|
||||
// Up to the content pool's limit of originals are worked on at once,
|
||||
// their `verify` hashing included, so however large the account is, no
|
||||
// more than that many are open. Each worker takes the next file from the
|
||||
// iterator they all share until none is left. A worker that throws, as
|
||||
// one does when `onProgress` throws, fails the backup only once every
|
||||
// worker has stopped, so none writes into the backup after it has ended.
|
||||
// iterator they all share until none is left, and throws instead once the
|
||||
// lock is lost. A worker that throws, as one also does when `onProgress`
|
||||
// throws, fails the backup only once every worker has stopped, so none
|
||||
// writes into the backup after it has ended.
|
||||
if (includeOriginals) {
|
||||
const pending = distinct.values();
|
||||
const worker = async (): Promise<void> => {
|
||||
for (const file of pending) await storeOriginal(file);
|
||||
for (const file of pending) {
|
||||
if (lock.lost) throw lock.lost;
|
||||
await storeOriginal(file);
|
||||
}
|
||||
};
|
||||
const workers = await Promise.allSettled(
|
||||
Array.from({ length: lib.contentConcurrency }, worker),
|
||||
|
||||
Reference in New Issue
Block a user