quak backup refuses to run while another backup of the same directory runs, exit 2 (closes #169)
check / check (push) Successful in 3m36s
check / check (push) Successful in 3m36s
lib.backup() takes a lock, backup.lock in its download directory, made with proper-lockfile, before its refresh, and removes it when it ends. A second backup of the directory fails at once with an error naming it; quak backup prints it as one line and exits 2. A lock untouched for 10 seconds, left by a killed run, is taken over. Deviation: yarn.lock was regenerated by yarn add in the pinned node image. Judgement call: a run failing at its refresh leaves the directory, empty. Judgement call: a lock removed mid-run stops that run with an uncaught error, the library's default. Model: opus-5-5
This commit is contained in:
+48
-14
@@ -1,11 +1,12 @@
|
||||
// The backup command, rebuilt on the library API (issue #51).
|
||||
//
|
||||
// `lib.backup()` waits for a completed refresh of the library (a failed one
|
||||
// fails the backup before any file is touched), then, for every file in scope,
|
||||
// puts its original at its save path under `downloadDirectory`, as
|
||||
// `Photo.download()` does, waits for an ML data fetch, and rebuilds the derived
|
||||
// views (per-file sidecars, per-collection symlink trees, per-collection JSON)
|
||||
// from the model. The on-disk layout:
|
||||
// `lib.backup()` takes the lock in `downloadDirectory`, and fails at once when
|
||||
// another backup of it holds the lock. It waits for a completed refresh of the
|
||||
// library (a failed one fails the backup before any file is touched), then, for
|
||||
// every file in scope, puts its original at its save path under
|
||||
// `downloadDirectory`, as `Photo.download()` does, waits for an ML data fetch,
|
||||
// and rebuilds the derived views (per-file sidecars, per-collection symlink
|
||||
// trees, per-collection JSON) from the model. The on-disk layout:
|
||||
//
|
||||
// <downloadDirectory>/
|
||||
// YYYY/YYYY-MM/YYYY-MM-DD/
|
||||
@@ -17,6 +18,7 @@
|
||||
// collections/<name>/<title> symlink to the original
|
||||
// collections/<name>.json per-collection metadata
|
||||
// account.json the account's email and user ID
|
||||
// backup.lock the lock, while a backup runs
|
||||
// failures.json durable ledger of unresolved failures
|
||||
//
|
||||
// A live photo's original is its image and its video, each with its own
|
||||
@@ -60,6 +62,7 @@ import {
|
||||
} from "node:fs";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { dirname, extname, join, relative, resolve } from "node:path";
|
||||
import lockfile from "proper-lockfile";
|
||||
|
||||
import { removeLeftoverTempFiles } from "./download/index.js";
|
||||
import { sanitizeFileName, withExtension } from "./filename.js";
|
||||
@@ -450,17 +453,12 @@ const writeAlbumJSON = (
|
||||
writeFileSync(path, JSON.stringify(album, null, 2));
|
||||
};
|
||||
|
||||
export const runBackup = async (
|
||||
// The backup itself, which `runBackup` below runs while it holds the lock.
|
||||
const runLockedBackup = async (
|
||||
lib: BackupLibrary,
|
||||
opts: BackupOptions,
|
||||
downloadDirectory: string,
|
||||
): Promise<BackupResult> => {
|
||||
const downloadDirectory = opts.downloadDirectory;
|
||||
if (!downloadDirectory) {
|
||||
throw new Error(
|
||||
"backup requires a downloadDirectory (pass one to backup() or " +
|
||||
"open the library with one)",
|
||||
);
|
||||
}
|
||||
const includeOriginals = opts.includeOriginals ?? true;
|
||||
const includeThumbnails = opts.includeThumbnails ?? false;
|
||||
const log = opts.onProgress ?? (() => {});
|
||||
@@ -736,3 +734,39 @@ export const runBackup = async (
|
||||
errors,
|
||||
};
|
||||
};
|
||||
|
||||
// Only one backup of a directory runs at a time, in this process or another:
|
||||
// a second one fails at once with an error whose `code` is `ELOCKED`. The lock
|
||||
// is the directory `backup.lock`, whose modification time proper-lockfile
|
||||
// keeps current while the backup runs. One it has not touched for 10 seconds
|
||||
// was left by a run that was killed, and is taken over.
|
||||
export const runBackup = async (
|
||||
lib: BackupLibrary,
|
||||
opts: BackupOptions,
|
||||
): Promise<BackupResult> => {
|
||||
const downloadDirectory = opts.downloadDirectory;
|
||||
if (!downloadDirectory) {
|
||||
throw new Error(
|
||||
"backup requires a downloadDirectory (pass one to backup() or " +
|
||||
"open the library with one)",
|
||||
);
|
||||
}
|
||||
mkdirSync(downloadDirectory, { recursive: true });
|
||||
let release: () => Promise<void>;
|
||||
try {
|
||||
release = await lockfile.lock(downloadDirectory, {
|
||||
lockfilePath: join(downloadDirectory, "backup.lock"),
|
||||
});
|
||||
} catch (err) {
|
||||
if ((err as NodeJS.ErrnoException).code !== "ELOCKED") throw err;
|
||||
throw Object.assign(
|
||||
new Error(`another backup of ${downloadDirectory} is running`),
|
||||
{ code: "ELOCKED" },
|
||||
);
|
||||
}
|
||||
try {
|
||||
return await runLockedBackup(lib, opts, downloadDirectory);
|
||||
} finally {
|
||||
await release();
|
||||
}
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user