quak backup --verify re-hashes stored originals and downloads again any that do not match (closes #168)

`--verify`, or `lib.backup({ verify: true })`, hashes each original already
at its save path as the download check does, streamed, a live photo as
`<imageHash>:<videoHash>`. A mismatch is logged, removed and put back in the
same run, and what is put back is hashed too, since a copy from the content
cache is not checked; one that still does not match, or a failed fetch, goes
into `failures.json`. A file with no recorded hash counts as unchecked. The
result, the summary and `--json` gain `verified`, `mismatched` and
`unchecked`.

Judgement call: a stored original that cannot be read for hashing is recorded as failed and left in place.
Judgement call: a copy put back that still does not match stays at its save path and is not counted as downloaded.
Judgement call: the summary prints the three counts only with `--verify`.

Model: opus-5-5

README.md

@@ -582,8 +582,9 @@ reads no tag from is recorded, base64, as `exifRaw`, with the reason in
 `exifError`. `collections`, `files`, `backup`, `helper list-missing-thumbnails`
 and `helper fix-missing-thumbnails` take `--json` for machine-readable output.
 
-`backup --verify` also hashes the originals already in the backup and downloads
-again any that do not match the content hash Ente records (see "Backup layout").
+`backup --verify` also hashes the originals already in the backup and replaces
+any that do not match the content hash Ente records; one it cannot replace goes
+into `failures.json` (see "Backup layout").
 
 `backup-metadata` fetches ML data in requests of up to 200 files. When a request
 fails, the error is logged, each of its files is written with the reason in an
@@ -705,15 +706,19 @@ original already at its save path the way a download is checked (see "On-disk
 cache layout" below): its bytes, read in chunks, or a live photo's image and
 video, joined as `<imageHash>:<videoHash>`. An original that matches the content
 hash its metadata records is left as it is. One that does not is logged on one
-line naming the file, deleted (a live photo's image and video both), and
-downloaded again in the same run like a missing one; if that download fails, the
-file goes into `failures.json`. A file whose metadata records no hash is left as
-it is and counted as unchecked. A stored original that cannot be read is left as
-it is and counts as failed. The summary and `--json` add the counts `verified`,
+line naming the file, deleted (a live photo's image and video both), and put
+back in the same run like a missing one: downloaded, or copied from the cache if
+the cache holds it. What is put back is hashed too, because a copy from the
+cache is not checked as a download is. If it still does not match, it stays at
+its save path and the file goes into `failures.json`, as it does when the
+download fails. A file whose metadata records no hash is left as it is and
+counted as unchecked. A stored original that cannot be read is left as it is and
+counts as failed. The summary and `--json` add the counts `verified`,
 `mismatched` and `unchecked`, all of originals that were already stored; one
-first downloaded in this run is in none of them. A mismatch that was downloaded
-again does not make the exit code non-zero. Without `--verify` nothing is
-hashed, the summary is unchanged, and the three counts are 0 in `--json`.
+first downloaded in this run is in none of them. A mismatch that was put back
+with matching bytes does not make the exit code non-zero. Without `--verify`
+nothing is hashed, the summary is unchanged, and the three counts are 0 in
+`--json`.
 
 Each original is written to a temporary file in the same directory, synced to
 disk, and renamed into place, so an original is either complete or absent, even
@@ -940,8 +945,8 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
   then counts it as present; one the cache already held is copied from there.
   `BackupOptions`: `downloadDirectory` (falls back to the library's),
   `includeOriginals` (default `true`), `includeThumbnails` (default `false`),
-  `onlyAlbumNames`, and `onProgress`. See Backup layout above for the tree it
-  writes.
+  `onlyAlbumNames`, `verify` (default `false`), and `onProgress`. See Backup
+  layout above for the tree it writes.
 
 ### Request pools
 

TODO.md

@@ -29,10 +29,11 @@ declares one.
   each original already at its save path as the download check does, streamed, a
   live photo as `<imageHash>:<videoHash>` (issue 168). One that does not match
   the content hash its metadata records is logged, removed (both files of a live
-  photo) and downloaded again in the same run; a failed download goes into
-  `failures.json`. One with no recorded hash is left alone. The result, `--json`
-  and the summary gain `verified`, `mismatched` and `unchecked`. Without
-  `--verify` nothing is hashed.
+  photo) and put back in the same run, downloaded or copied from the cache, and
+  what is put back is hashed too. One that still does not match, or whose
+  download fails, goes into `failures.json`. One with no recorded hash is left
+  alone. The result, `--json` and the summary gain `verified`, `mismatched` and
+  `unchecked`. Without `--verify` nothing is hashed.
 
 - 2026-10-06: `quak backup` writes each original's EXIF, XMP and dimensions into
   the file's JSON as `imageMetadata`, what `backup-metadata --exif` records

src/backup.ts

@@ -37,7 +37,8 @@
 //
 // With `verify`, each original already at its save path is hashed as the
 // download check hashes it, and one that does not match the content hash its
-// metadata records is removed and fetched again in the same run.
+// metadata records is removed and fetched again in the same run. What is put
+// back is hashed too, and recorded as failed if it still does not match.
 //
 // Resilience (issue #8): no per-file condition aborts the run. A failed
 // download, a failed symlink, or ML data missing because the ML data fetch
@@ -606,6 +607,7 @@ export const runBackup = async (
     if (includeOriginals) {
         for (const [fileID, file] of distinct) {
             let stored = storedAtSavePath(downloadDirectory, file);
+            let mismatch = false;
             if (stored !== undefined && verify) {
                 try {
                     if (file.metadata.hash === undefined) {
@@ -619,6 +621,7 @@ export const runBackup = async (
                             `MISMATCH original ${file.metadata.title} (${fileID}): its bytes do not match its content hash`,
                         );
                         mismatched++;
+                        mismatch = true;
                         rmSync(stored.path);
                         if (stored.videoPath !== undefined) {
                             rmSync(stored.videoPath);
@@ -646,10 +649,24 @@ export const runBackup = async (
                 // A fetched original is written straight to its save path (a
                 // live photo beside it); only one that was already cached
                 // elsewhere is copied.
-                await placeOriginal(downloadDirectory, file, (dest) =>
-                    lib.original(fileID, dest),
+                const placed = await placeOriginal(
+                    downloadDirectory,
+                    file,
+                    (dest) => lib.original(fileID, dest),
                 );
                 storedThisRun.add(fileID);
+                // A copy from the cache is not checked as a download is and can
+                // hold the same bad bytes, so what is put back after a
+                // mismatch is hashed too. A bad copy stays where it is and the
+                // file fails.
+                if (
+                    mismatch &&
+                    (await storedHash(placed)) !== file.metadata.hash
+                ) {
+                    throw new Error(
+                        "the original put back does not match its content hash either",
+                    );
+                }
                 downloaded++;
             } catch (err) {
                 log(

test/cli/backup.test.ts

@@ -1308,6 +1308,33 @@ describe("backup with verify", () => {
         await lib.close();
     });
 
+    it("records in failures.json an original the cache puts back with the same bad bytes", async () => {
+        const lib = await openLibrary(stubSource(), new HashedClient());
+        const outDir = join(root, "backup");
+        // The cache holds a bad copy, and a backup copies an original the
+        // cache holds to its save path.
+        const cached = await lib.photos.byID({ fileID: 100 })!.original();
+        writeFileSync(cached.path, corrupt);
+        await lib.backup({ downloadDirectory: outDir });
+
+        const result = await lib.backup({
+            downloadDirectory: outDir,
+            verify: true,
+        });
+
+        expect(result).toMatchObject({
+            downloaded: 0,
+            mismatched: 1,
+            failed: 1,
+        });
+        expect(result.errors.map((e) => e.error)).toEqual([
+            "the original put back does not match its content hash either",
+        ]);
+        expect(Object.keys(readLedger(outDir).files)).toEqual(["100"]);
+        expect(readFileSync(saved(outDir, "100.jpg"))).toEqual(corrupt);
+        await lib.close();
+    });
+
     it("hashes nothing without verify", async () => {
         const { lib, outDir } = await backedUp();
         writeFileSync(saved(outDir, "100.jpg"), corrupt);