quak backup works on up to the content pool's limit of originals at once (closes #182) #192
No files matched your search
@@ -779,9 +779,9 @@ A running backup loses its lock when someone removes `backup.lock`, or when
|
|||||||
another backup takes it over because the running one left it untouched for 10
|
another backup takes it over because the running one left it untouched for 10
|
||||||
seconds, as a stopped or stalled process does. The backup finds this out the
|
seconds, as a stopped or stalled process does. The backup finds this out the
|
||||||
next time it goes to touch the lock, which it does every 5 seconds, and then
|
next time it goes to touch the lock, which it does every 5 seconds, and then
|
||||||
stops before it starts another file. The original it is downloading at that
|
stops before it starts another file. The originals it is downloading at that
|
||||||
moment is still stored. It leaves `failures.json` and `backup.lock` as they are,
|
moment are still stored. It leaves `failures.json` and `backup.lock` as they
|
||||||
since another backup may be using them. `quak backup` prints
|
are, since another backup may be using them. `quak backup` prints
|
||||||
`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and
|
`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and
|
||||||
`lib.backup()` rejects with `lost the lock on <dir>: <reason>`.
|
`lib.backup()` rejects with `lost the lock on <dir>: <reason>`.
|
||||||
|
|
||||||
|
|||||||
@@ -32,7 +32,8 @@ declares one.
|
|||||||
the work for one original, its `--verify` hashing included, so no more
|
the work for one original, its `--verify` hashing included, so no more
|
||||||
originals than that are open at once. `BackupLibrary` says the limit as
|
originals than that are open at once. `BackupLibrary` says the limit as
|
||||||
`contentConcurrency`. A test holds each download at the fake Ente server to
|
`contentConcurrency`. A test holds each download at the fake Ente server to
|
||||||
count those in flight.
|
count those in flight. Each worker checks the backup's lock before it takes
|
||||||
|
the next file, so once the lock is lost no further original starts.
|
||||||
|
|
||||||
- 2026-10-14: Next Step said the cache design, issue 36, waits on sneak's
|
- 2026-10-14: Next Step said the cache design, issue 36, waits on sneak's
|
||||||
review, though that issue closed on 2026-10-03 (issue 194). It now names the
|
review, though that issue closed on 2026-10-03 (issue 194). It now names the
|
||||||
|
|||||||
+8
-4
@@ -730,13 +730,17 @@ const runLockedBackup = async (
|
|||||||
// Up to the content pool's limit of originals are worked on at once,
|
// Up to the content pool's limit of originals are worked on at once,
|
||||||
// their `verify` hashing included, so however large the account is, no
|
// their `verify` hashing included, so however large the account is, no
|
||||||
// more than that many are open. Each worker takes the next file from the
|
// more than that many are open. Each worker takes the next file from the
|
||||||
// iterator they all share until none is left. A worker that throws, as
|
// iterator they all share until none is left, and throws instead once the
|
||||||
// one does when `onProgress` throws, fails the backup only once every
|
// lock is lost. A worker that throws, as one also does when `onProgress`
|
||||||
// worker has stopped, so none writes into the backup after it has ended.
|
// throws, fails the backup only once every worker has stopped, so none
|
||||||
|
// writes into the backup after it has ended.
|
||||||
if (includeOriginals) {
|
if (includeOriginals) {
|
||||||
const pending = distinct.values();
|
const pending = distinct.values();
|
||||||
const worker = async (): Promise<void> => {
|
const worker = async (): Promise<void> => {
|
||||||
for (const file of pending) await storeOriginal(file);
|
for (const file of pending) {
|
||||||
|
if (lock.lost) throw lock.lost;
|
||||||
|
await storeOriginal(file);
|
||||||
|
}
|
||||||
};
|
};
|
||||||
const workers = await Promise.allSettled(
|
const workers = await Promise.allSettled(
|
||||||
Array.from({ length: lib.contentConcurrency }, worker),
|
Array.from({ length: lib.contentConcurrency }, worker),
|
||||||
|
|||||||
Reference in new issue
Block a user