quak backup --verify re-hashes stored originals and downloads again any that do not match (closes #168)
check / check (push) Successful in 3m15s
check / check (push) Successful in 3m15s
`--verify`, or `lib.backup({ verify: true })`, hashes each original already
at its save path as the download check does, streamed, a live photo as
`<imageHash>:<videoHash>`. A mismatch is logged, removed and fetched again in
the same run; a failed fetch goes into `failures.json`. A file with no
recorded hash counts as unchecked. The result, the summary and `--json` gain
`verified`, `mismatched` and `unchecked`.
Judgement call: a stored original that cannot be read for hashing is recorded as failed and left in place.
Judgement call: the summary prints the three counts only with `--verify`.
Judgement call: the README's `BackupOptions` list does not name `verify`, to stay clear of #178's edit of that paragraph; Backup layout documents it.
Model: opus-5-5
This commit is contained in:
@@ -549,7 +549,7 @@ quak collections [--json] list all collections
|
||||
quak files --collection <id> [--json] list files in a collection
|
||||
quak get <fileID> [--out path] [--collection] download and decrypt a file
|
||||
quak get-thumb <fileID> [--out] [--collection] download and decrypt a thumbnail
|
||||
quak backup <dir> [--json] full incremental backup
|
||||
quak backup <dir> [--json] [--verify] full incremental backup
|
||||
quak backup-metadata <dir> [--exif] dump the metadata quak keeps as JSON
|
||||
quak helper list-missing-thumbnails [--json] find files with missing thumbnails
|
||||
quak helper fix-missing-thumbnails [--file ids] [--json] generate + upload missing thumbnails
|
||||
@@ -582,6 +582,9 @@ reads no tag from is recorded, base64, as `exifRaw`, with the reason in
|
||||
`exifError`. `collections`, `files`, `backup`, `helper list-missing-thumbnails`
|
||||
and `helper fix-missing-thumbnails` take `--json` for machine-readable output.
|
||||
|
||||
`backup --verify` also hashes the originals already in the backup and downloads
|
||||
again any that do not match the content hash Ente records (see "Backup layout").
|
||||
|
||||
`backup-metadata` fetches ML data in requests of up to 200 files. When a request
|
||||
fails, the error is logged, each of its files is written with the reason in an
|
||||
`mlDataError` field instead of `mlData`, and the dump goes on. The exit code is
|
||||
@@ -697,6 +700,21 @@ if any files failed. `quak backup` opens its library with the thumbnail and
|
||||
originals precache off, so the only file content it fetches is the originals the
|
||||
backup stores.
|
||||
|
||||
With `--verify`, or `lib.backup({ verify: true })`, a run also hashes each
|
||||
original already at its save path the way a download is checked (see "On-disk
|
||||
cache layout" below): its bytes, read in chunks, or a live photo's image and
|
||||
video, joined as `<imageHash>:<videoHash>`. An original that matches the content
|
||||
hash its metadata records is left as it is. One that does not is logged on one
|
||||
line naming the file, deleted (a live photo's image and video both), and
|
||||
downloaded again in the same run like a missing one; if that download fails, the
|
||||
file goes into `failures.json`. A file whose metadata records no hash is left as
|
||||
it is and counted as unchecked. A stored original that cannot be read is left as
|
||||
it is and counts as failed. The summary and `--json` add the counts `verified`,
|
||||
`mismatched` and `unchecked`, all of originals that were already stored; one
|
||||
first downloaded in this run is in none of them. A mismatch that was downloaded
|
||||
again does not make the exit code non-zero. Without `--verify` nothing is
|
||||
hashed, the summary is unchanged, and the three counts are 0 in `--json`.
|
||||
|
||||
Each original is written to a temporary file in the same directory, synced to
|
||||
disk, and renamed into place, so an original is either complete or absent, even
|
||||
after a power cut. A downloaded original's temporary file is named
|
||||
|
||||
Reference in New Issue
Block a user