quak backup works on up to the content pool's limit of originals at once (closes #182)
check / check (push) Canceled after 0s

The backup awaited each original before starting the next, so it used
one of the content pool's five slots. lib.backup(), and so quak backup,
now runs as many workers as the content pool's limit (5 by default, or
the contentConcurrency of the pools the library was opened with), each
taking the next file from one shared list, so the limit covers the
--verify hashing too. Once a worker has thrown or the lock is lost, no
worker takes another file; the backup waits for the files under way and
checks its lock before the ML data fetch. runBackup rejects a
contentConcurrency that is not a positive integer before it takes the
lock. Failed files are listed in the order they failed.

Model: opus-5-5
This commit was merged in pull request #192.
This commit is contained in:
2026-10-14 09:06:59 +02:00
parent 22ee2db479
commit 3412e42af5
7 changed files with 763 additions and 83 deletions
+22
View File
@@ -909,6 +909,27 @@ describe("the backup lock", () => {
expect(existsSync(lockPath(outDir))).toBe(false);
await lib.close();
});
// `runBackup` takes a library object from its caller. With a
// `contentConcurrency` below 1, or one that is not a number, it would
// back up no original and report no error. It rejects any that is not a
// positive integer, as `BoundedPool` does, before it takes the lock,
// which would create the download directory, and before it asks the
// library for anything: the object here has nothing else.
it("rejects a library whose contentConcurrency is not a positive integer, before taking the lock", async () => {
const outDir = join(root, "backup");
for (const contentConcurrency of [undefined, 0, -1, 1.5, NaN]) {
const lib = { contentConcurrency } as unknown as BackupLibrary;
await expect(
runBackup(lib, { downloadDirectory: outDir }),
).rejects.toThrow(
`contentConcurrency must be a positive integer, got ${contentConcurrency}`,
);
expect(existsSync(outDir)).toBe(false);
}
});
});
// Also serves ML data: file 100 has `ML_PAYLOAD`, the other two have none.
@@ -1491,6 +1512,7 @@ describe("backup album folders", () => {
thumbnail: async () => {
throw new Error("no thumbnails in this stand-in");
},
contentConcurrency: 5,
fetchMLData: async () => {},
mlData: async () => undefined,
whoami: () => ({ email: "backup@example.com", userID: USER_ID }),