quak backup --verify re-hashes stored originals and downloads again any that do not match (closes #168)
check / check (push) Successful in 3m15s

`--verify`, or `lib.backup({ verify: true })`, hashes each original already
at its save path as the download check does, streamed, a live photo as
`<imageHash>:<videoHash>`. A mismatch is logged, removed and fetched again in
the same run; a failed fetch goes into `failures.json`. A file with no
recorded hash counts as unchecked. The result, the summary and `--json` gain
`verified`, `mismatched` and `unchecked`.

Judgement call: a stored original that cannot be read for hashing is recorded as failed and left in place.
Judgement call: the summary prints the three counts only with `--verify`.
Judgement call: the README's `BackupOptions` list does not name `verify`, to stay clear of #178's edit of that paragraph; Backup layout documents it.

Model: opus-5-5
This commit is contained in:
2026-10-06 14:57:36 +00:00
parent 31b50a211d
commit 73846b32fc
8 changed files with 391 additions and 7 deletions
+19 -1
View File
@@ -549,7 +549,7 @@ quak collections [--json] list all collections
quak files --collection <id> [--json] list files in a collection
quak get <fileID> [--out path] [--collection] download and decrypt a file
quak get-thumb <fileID> [--out] [--collection] download and decrypt a thumbnail
quak backup <dir> [--json] full incremental backup
quak backup <dir> [--json] [--verify] full incremental backup
quak backup-metadata <dir> [--exif] dump the metadata quak keeps as JSON
quak helper list-missing-thumbnails [--json] find files with missing thumbnails
quak helper fix-missing-thumbnails [--file ids] [--json] generate + upload missing thumbnails
@@ -582,6 +582,9 @@ reads no tag from is recorded, base64, as `exifRaw`, with the reason in
`exifError`. `collections`, `files`, `backup`, `helper list-missing-thumbnails`
and `helper fix-missing-thumbnails` take `--json` for machine-readable output.
`backup --verify` also hashes the originals already in the backup and downloads
again any that do not match the content hash Ente records (see "Backup layout").
`backup-metadata` fetches ML data in requests of up to 200 files. When a request
fails, the error is logged, each of its files is written with the reason in an
`mlDataError` field instead of `mlData`, and the dump goes on. The exit code is
@@ -697,6 +700,21 @@ if any files failed. `quak backup` opens its library with the thumbnail and
originals precache off, so the only file content it fetches is the originals the
backup stores.
With `--verify`, or `lib.backup({ verify: true })`, a run also hashes each
original already at its save path the way a download is checked (see "On-disk
cache layout" below): its bytes, read in chunks, or a live photo's image and
video, joined as `<imageHash>:<videoHash>`. An original that matches the content
hash its metadata records is left as it is. One that does not is logged on one
line naming the file, deleted (a live photo's image and video both), and
downloaded again in the same run like a missing one; if that download fails, the
file goes into `failures.json`. A file whose metadata records no hash is left as
it is and counted as unchecked. A stored original that cannot be read is left as
it is and counts as failed. The summary and `--json` add the counts `verified`,
`mismatched` and `unchecked`, all of originals that were already stored; one
first downloaded in this run is in none of them. A mismatch that was downloaded
again does not make the exit code non-zero. Without `--verify` nothing is
hashed, the summary is unchanged, and the three counts are 0 in `--json`.
Each original is written to a temporary file in the same directory, synced to
disk, and renamed into place, so an original is either complete or absent, even
after a power cut. A downloaded original's temporary file is named