quak backup refuses to run while another backup of the same directory runs, exit 2 (closes #169)

lib.backup() takes a lock, backup.lock in its download directory, made
with proper-lockfile, before its refresh, and removes it when it ends. A
second backup of the directory fails at once with an error naming it.
quak backup takes the lock itself before it opens its library and passes
lockHeld to the backup, so a refused run sends no request; it prints the
error as one line and exits 2. A lock untouched for 10 seconds, left by a
run that could not remove it, is taken over.

Deviation: yarn.lock was regenerated by yarn add in the pinned node image.
Judgement call: a run failing at its refresh leaves the directory, empty.
Judgement call: a lock removed mid-run stops that run with an uncaught
error, the library's default.

Model: opus-5-5

README.md

@@ -669,17 +669,18 @@ library's `lib.backup({ includeThumbnails: true })` also writes
 `backup.lock` keeps two backups of the same directory from running at once, such
 as a cron run that starts while the previous one is still going. A backup
 creates `<dir>` if it is missing and takes the lock before its refresh, and
-removes the lock when it ends, whether it succeeds or fails. The lock is a
+removes the lock when it ends, whether it succeeds or fails; `quak backup` takes
+it before it opens its library, so a refused run sends no request. The lock is a
 directory that
 [proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates
 and keeps touching while the backup runs. A second backup of the directory, from
 another process or the same one, fails at once: `quak backup` prints
-`quak: another backup of <dir> is running` and exits with status 2. It opens its
+`quak: another backup of <dir> is running` and exits with status 2. A run
-library before the backup takes the lock, so a refused run still waits for the
+stopped with Ctrl-C or `kill` removes the lock as it exits. Only a run that
-refresh that opening the library starts before it exits. A run that is killed
+cannot, such as one killed with SIGKILL or cut off by a crash or power loss,
-leaves the lock behind; once it has gone 10 seconds untouched, the next run
+leaves it behind; once it has gone 10 seconds untouched, the next run takes it
-takes it over, so nobody has to remove it. The lock is outside the date folders
+over, so nobody has to remove it. The lock is outside the date folders and
-and `collections/`, so it is never taken for an original, and the removal of old
+`collections/`, so it is never taken for an original, and the removal of old
 album directories never touches it.
 
 A collection's directory and JSON are named after the collection, and a symlink
@@ -931,17 +932,21 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
   a query vector the caller produced elsewhere.
 - `await lib.backup(opts?)` → `BackupResult`. It takes the lock in the download
   directory, and fails at once with an error whose `code` is `ELOCKED` while
-  another backup of it runs. It waits for a refresh as `fresh()` does, puts
+  another backup of it runs. A caller can instead take the lock itself, before
-  every in-scope original not already at its save path there as
+  it opens its library, as `quak backup` does: `await lockBackupDirectory(dir)`
-  `photo.download()` does (and, with `includeThumbnails`, fetches thumbnails)
+  takes it, failing the same way, and returns the function that releases it, and
-  through the content cache, waits for an ML data fetch, and rebuilds the
+  the caller passes `lockHeld: true` to the backup. It waits for a refresh as
-  on-disk backup tree, each file's JSON with its ML data and its original's
+  `fresh()` does, puts every in-scope original not already at its save path
-  EXIF, XMP and dimensions, with a durable failure ledger. A fetched original is
+  there as `photo.download()` does (and, with `includeThumbnails`, fetches
-  written straight to its save path and not into the cache, which then counts it
+  thumbnails) through the content cache, waits for an ML data fetch, and
-  as present; one the cache already held is copied from there. `BackupOptions`:
+  rebuilds the on-disk backup tree, each file's JSON with its ML data and its
-  `downloadDirectory` (falls back to the library's), `includeOriginals` (default
+  original's EXIF, XMP and dimensions, with a durable failure ledger. A fetched
-  `true`), `includeThumbnails` (default `false`), `onlyAlbumNames`, and
+  original is written straight to its save path and not into the cache, which
-  `onProgress`. See Backup layout above for the tree it writes.
+  then counts it as present; one the cache already held is copied from there.
+  `BackupOptions`: `downloadDirectory` (falls back to the library's),
+  `includeOriginals` (default `true`), `includeThumbnails` (default `false`),
+  `onlyAlbumNames`, `onProgress`, and `lockHeld` (default `false`). See Backup
+  layout above for the tree it writes.
 
 ### Request pools
 

TODO.md

@@ -29,10 +29,11 @@ declares one.
   `lib.backup()` takes a lock, `backup.lock` in its download directory, made
   with `proper-lockfile`, before its refresh, and removes it when it ends,
   whether it succeeds or fails. A second backup of the directory, from another
-  process or the same one, fails at once with an error naming the directory;
+  process or the same one, fails at once with an error naming the directory.
-  `quak backup` prints it as one line and exits 2. A lock that has gone 10
+  `quak backup` takes the lock before it opens its library, so a refused run
-  seconds untouched, left by a run that was killed, is taken over by the next
+  sends no request; it prints the error as one line and exits 2. A lock that has
-  run.
+  gone 10 seconds untouched, left by a run that could not remove it, is taken
+  over by the next run.
 
 - 2026-10-06: `quak backup` writes each original's EXIF, XMP and dimensions into
   the file's JSON as `imageMetadata`, what `backup-metadata --exif` records

src/backup.ts

@@ -1,12 +1,13 @@
 // The backup command, rebuilt on the library API (issue #51).
 //
-// `lib.backup()` takes the lock in `downloadDirectory`, and fails at once when
+// `lib.backup()` takes the lock in `downloadDirectory` (unless its caller
-// another backup of it holds the lock. It waits for a completed refresh of the
+// already holds it), and fails at once when another backup of it holds the
-// library (a failed one fails the backup before any file is touched), then, for
+// lock. It waits for a completed refresh of the library (a failed one fails the
-// every file in scope, puts its original at its save path under
+// backup before any file is touched), then, for every file in scope, puts its
-// `downloadDirectory`, as `Photo.download()` does, waits for an ML data fetch,
+// original at its save path under `downloadDirectory`, as `Photo.download()`
-// and rebuilds the derived views (per-file sidecars, per-collection symlink
+// does, waits for an ML data fetch, and rebuilds the derived views (per-file
-// trees, per-collection JSON) from the model. The on-disk layout:
+// sidecars, per-collection symlink trees, per-collection JSON) from the model.
+// The on-disk layout:
 //
 //     <downloadDirectory>/
 //         YYYY/YYYY-MM/YYYY-MM-DD/
@@ -92,6 +93,10 @@ export interface BackupOptions {
     // Restrict the backup to albums with these names; others are left untouched.
     onlyAlbumNames?: string[];
     onProgress?: ProgressCallback;
+    // The caller already holds the lock in `downloadDirectory`, taken with
+    // `lockBackupDirectory`, and releases it itself, so the backup does not
+    // take it. `quak backup` takes it before it opens its library.
+    lockHeld?: boolean;
 }
 
 export interface BackupError {
@@ -453,7 +458,7 @@ const writeAlbumJSON = (
     writeFileSync(path, JSON.stringify(album, null, 2));
 };
 
-// The backup itself, which `runBackup` below runs while it holds the lock.
+// The backup itself, which `runBackup` below runs while the lock is held.
 const runLockedBackup = async (
     lib: BackupLibrary,
     opts: BackupOptions,
@@ -735,11 +740,32 @@ const runLockedBackup = async (
     };
 };
 
-// Only one backup of a directory runs at a time, in this process or another:
+// Only one backup of a directory runs at a time, in this process or another.
-// a second one fails at once with an error whose `code` is `ELOCKED`. The lock
+// Creates `downloadDirectory` if it is missing, takes the lock in it and
-// is the directory `backup.lock`, whose modification time proper-lockfile
+// returns the function that releases it; while another backup holds the lock,
-// keeps current while the backup runs. One it has not touched for 10 seconds
+// fails at once with an error whose `code` is `ELOCKED`. The lock is the
-// was left by a run that was killed, and is taken over.
+// directory `backup.lock`, whose modification time proper-lockfile keeps
+// current while it is held. One it has not touched for 10 seconds was left by
+// a run that could not remove it, and is taken over.
+export const lockBackupDirectory = async (
+    downloadDirectory: string,
+): Promise<() => Promise<void>> => {
+    mkdirSync(downloadDirectory, { recursive: true });
+    try {
+        return await lockfile.lock(downloadDirectory, {
+            lockfilePath: join(downloadDirectory, "backup.lock"),
+        });
+    } catch (err) {
+        if ((err as NodeJS.ErrnoException).code !== "ELOCKED") throw err;
+        throw Object.assign(
+            new Error(`another backup of ${downloadDirectory} is running`),
+            { code: "ELOCKED" },
+        );
+    }
+};
+
+// Runs the backup holding the lock, which it takes and releases itself unless
+// the caller already holds it (`opts.lockHeld`).
 export const runBackup = async (
     lib: BackupLibrary,
     opts: BackupOptions,
@@ -751,19 +777,10 @@ export const runBackup = async (
                 "open the library with one)",
         );
     }
-    mkdirSync(downloadDirectory, { recursive: true });
+    if (opts.lockHeld) {
-    let release: () => Promise<void>;
+        return runLockedBackup(lib, opts, downloadDirectory);
-    try {
-        release = await lockfile.lock(downloadDirectory, {
-            lockfilePath: join(downloadDirectory, "backup.lock"),
-        });
-    } catch (err) {
-        if ((err as NodeJS.ErrnoException).code !== "ELOCKED") throw err;
-        throw Object.assign(
-            new Error(`another backup of ${downloadDirectory} is running`),
-            { code: "ELOCKED" },
-        );
     }
+    const release = await lockBackupDirectory(downloadDirectory);
     try {
         return await runLockedBackup(lib, opts, downloadDirectory);
     } finally {

src/cli-commands.ts

@@ -20,9 +20,9 @@ import {
     type ClientSnapshot,
     type LoginOptions,
 } from "./client.js";
+import { lockBackupDirectory } from "./backup.js";
 import { init } from "./crypto/index.js";
 import {
-    type BackupResult,
     defaultCacheDirectory,
     Library,
     type LibraryClient,
@@ -409,6 +409,17 @@ export const backupCommand = async (
     if (!client) return 3;
 
     ctx.stderr.write("Starting backup...\n");
+    // The lock is taken before the library opens and starts its refresh, so a
+    // run refused while another backup of `dir` runs sends no request.
+    let release: () => Promise<void>;
+    try {
+        release = await lockBackupDirectory(dir);
+    } catch (err) {
+        if ((err as NodeJS.ErrnoException).code !== "ELOCKED") throw err;
+        ctx.stderr.write(`quak: ${(err as Error).message}\n`);
+        return 2;
+    }
+    try {
         // The precache is off: the backup fetches what it needs, and must not
         // also fill the cache with every thumbnail and the recent originals.
         const lib = await Library.open({
@@ -419,20 +430,13 @@ export const backupCommand = async (
             precacheOriginals: false,
         });
         try {
-        let result: BackupResult;
+            const result = await lib.backup({
-        try {
-            result = await lib.backup({
                 downloadDirectory: dir,
+                lockHeld: true,
                 onProgress: (msg) => {
                     if (!opts.json) ctx.stderr.write(msg + "\n");
                 },
             });
-        } catch (err) {
-            // Another backup of `dir` is running and holds its lock.
-            if ((err as NodeJS.ErrnoException).code !== "ELOCKED") throw err;
-            ctx.stderr.write(`quak: ${(err as Error).message}\n`);
-            return 2;
-        }
 
             if (opts.json) {
                 ctx.stdout.write(JSON.stringify(result, null, 2) + "\n");
@@ -456,6 +460,9 @@ export const backupCommand = async (
         } finally {
             await lib.close();
         }
+    } finally {
+        await release();
+    }
 };
 
 export const listMissingThumbnailsCommand = async (

src/index.ts

@@ -67,6 +67,7 @@ export {
     type EnsureOptions,
     type EnsureResult,
     type EnsureEvent,
+    lockBackupDirectory,
     runBackup,
     type BackupOptions,
     type BackupResult,

src/library/index.ts

@@ -94,6 +94,7 @@ import type { Collection, EnteFile } from "../model/types.js";
 import { runBackup, type BackupOptions, type BackupResult } from "../backup.js";
 
 export {
+    lockBackupDirectory,
     runBackup,
     type BackupOptions,
     type BackupResult,
@@ -566,8 +567,9 @@ export class Library {
 
     // Back up every in-scope file to `opts.downloadDirectory`, or else the
     // library's, each original at its save path, with a durable failure
-    // ledger (issue #51). Takes the lock in that directory first, failing at
+    // ledger (issue #51). Takes the lock in that directory first, unless
-    // once while another backup of it runs (see `runBackup`). Waits for a
+    // `opts.lockHeld` says the caller holds it, failing at once while another
+    // backup of it runs (see `runBackup`). Waits for a
     // completed refresh, as `fresh()` does, joining one already running, and
     // rejects before touching any file but the lock when it fails. Then puts
     // pending originals at their save paths as `Photo.download()` does (and

test/cli/commands.test.ts

@@ -704,6 +704,7 @@ describe("backup", () => {
                 "  Failed:      0\n",
         );
         expect(stdout.text).toBe("");
+        expect(existsSync(join(dir, "backup.lock"))).toBe(false);
     });
 
     // The backup opens its library with the precache off: it fetches the
@@ -827,7 +828,7 @@ describe("backup", () => {
         expect(readdirSync(dir)).toEqual([]);
     });
 
-    it("exits 2 with one line naming the directory while another backup of it runs", async () => {
+    it("exits 2 with one line naming the directory, sending no request, while another backup of it runs", async () => {
         const dir = join(root, "backup");
         // The lock another backup holds. Its modification time is set an hour
         // ahead, so it stays current however long this test takes.
@@ -835,12 +836,27 @@ describe("backup", () => {
         mkdirSync(lock, { recursive: true });
         const hourAhead = new Date(Date.now() + 3_600_000);
         utimesSync(lock, hourAhead, hourAhead);
+        // A client whose refresh never finishes, so a run that started one
+        // before it exits would never return.
+        let requests = 0;
+        const never = (): Promise<never> => {
+            requests++;
+            return new Promise(() => {});
+        };
+        const client = {
+            ...fakeClient(),
+            collectionsSince: never,
+            filesSince: never,
+        } as unknown as Client;
 
-        expect(await backupCommand(context(), dir, {})).toBe(2);
+        expect(await backupCommand(context(client), dir, {})).toBe(2);
+        expect(requests).toBe(0);
         expect(stderr.text).toBe(
             `Starting backup...\nquak: another backup of ${dir} is running\n`,
         );
         expect(readdirSync(dir)).toEqual(["backup.lock"]);
+        // Opening the library would have made its cache directory.
+        expect(existsSync(join(root, "cache"))).toBe(false);
     });
 });