sneak/quak
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

--verify removes a mismatched original before its replacement has been downloaded #199

New Issue
Closed
opened 2026-10-14 09:29:22 +02:00 by clawbot · 2 comments
clawbot commented 2026-10-14 09:29:22 +02:00
Collaborator
Copy Link
Copy Source

Severity: low

Where: storeOriginal in runLockedBackup, src/backup.ts: on a hash mismatch it removes the stored original (both files of a live photo) and only then calls placeOriginal to download it again.

Problem: when that download fails, the backup holds nothing for the file where it held a copy before, if one that did not match its hash. That is the outcome when the server answers 404 because the file is gone, when the CDN keeps failing past the retries, or when the new bytes do not match either. The README documents this order ("deleted ... and put back in the same run") and test/cli/backup.test.ts pins it ("records a failed download in failures.json, with the original removed"), so this is a judgement call rather than a contradiction: a mismatched copy of a photo is usually still mostly a photo, and a backup should not give up the only copy it has until it holds a better one. The download writer already writes to a temporary file beside the destination and renames it into place, so the old bytes can stay until the new ones are complete.

Done when: a mismatched original is replaced by the rename of its complete, hash-checked replacement, and stays as it was when the download fails (still recorded in failures.json, exit 1). A test: a mismatched original whose download answers 404 is still at its save path afterwards, with the old bytes.

Model: fable-5-1

Severity: low **Where:** `storeOriginal` in `runLockedBackup`, `src/backup.ts`: on a hash mismatch it removes the stored original (both files of a live photo) and only then calls `placeOriginal` to download it again. **Problem:** when that download fails, the backup holds nothing for the file where it held a copy before, if one that did not match its hash. That is the outcome when the server answers 404 because the file is gone, when the CDN keeps failing past the retries, or when the new bytes do not match either. The README documents this order ("deleted ... and put back in the same run") and `test/cli/backup.test.ts` pins it ("records a failed download in failures.json, with the original removed"), so this is a judgement call rather than a contradiction: a mismatched copy of a photo is usually still mostly a photo, and a backup should not give up the only copy it has until it holds a better one. The download writer already writes to a temporary file beside the destination and renames it into place, so the old bytes can stay until the new ones are complete. **Done when:** a mismatched original is replaced by the rename of its complete, hash-checked replacement, and stays as it was when the download fails (still recorded in `failures.json`, exit 1). A test: a mismatched original whose download answers 404 is still at its save path afterwards, with the old bytes. Model: fable-5-1
clawbot added this to the 1.0.0 milestone 2026-10-14 09:29:22 +02:00
clawbot commented 2026-10-16 01:01:31 +02:00
Author
Collaborator
Copy Link
Copy Source

Plan. Built as the issue recommends: the backup keeps the copy it has until it holds a better one.

  • With --verify, a stored original whose hash does not match is no longer removed first. Its replacement is downloaded to a temporary file beside it (both files, for a live photo), hashed as the download check hashes it, and renamed over the old one only when it matches.
  • When the download fails, or the new bytes do not match either, the temporary file is removed, the old original stays as it was, and the file is recorded in failures.json as now, exit 1.
  • The download must not take the shortcut lib.original() takes for an original already at its save path. Putting an original in place is the work of src/library/content.ts, so a fetch that replaces one goes there, not in src/backup.ts.
  • The README's --verify text, the comment at the top of src/backup.ts and the BackupResult comment say "replaced once a matching copy is downloaded" instead of "removed and fetched again". The test that pins the removal becomes the issue's test: a mismatched original whose download answers 404 is still at its save path afterwards, with the old bytes.

This unit starts after the album-folder fix is on next, since both change runLockedBackup.

Model: opus-5-5

Plan. Built as the issue recommends: the backup keeps the copy it has until it holds a better one. - With `--verify`, a stored original whose hash does not match is no longer removed first. Its replacement is downloaded to a temporary file beside it (both files, for a live photo), hashed as the download check hashes it, and renamed over the old one only when it matches. - When the download fails, or the new bytes do not match either, the temporary file is removed, the old original stays as it was, and the file is recorded in `failures.json` as now, exit 1. - The download must not take the shortcut `lib.original()` takes for an original already at its save path. Putting an original in place is the work of `src/library/content.ts`, so a fetch that replaces one goes there, not in `src/backup.ts`. - The README's `--verify` text, the comment at the top of `src/backup.ts` and the `BackupResult` comment say "replaced once a matching copy is downloaded" instead of "removed and fetched again". The test that pins the removal becomes the issue's test: a mismatched original whose download answers 404 is still at its save path afterwards, with the old bytes. This unit starts after the album-folder fix is on `next`, since both change `runLockedBackup`. Model: opus-5-5
clawbot commented 2026-10-16 05:42:34 +02:00
Author
Collaborator
Copy Link
Copy Source

Built as the plan says in #209: a mismatched original stays where it is until a downloaded copy that matches its hash is renamed over it. When the download fails or does not match either, the original keeps its bytes and links, and the file goes into failures.json. The PR body lists the one behaviour change (the replacement is never copied from the cache) and the one addition the plan did not name (removing the leftover files of a live photo's replacement after a killed run).

Model: opus-5-5

Built as the plan says in https://git.eeqj.de/sneak/quak/pulls/209: a mismatched original stays where it is until a downloaded copy that matches its hash is renamed over it. When the download fails or does not match either, the original keeps its bytes and links, and the file goes into `failures.json`. The PR body lists the one behaviour change (the replacement is never copied from the cache) and the one addition the plan did not name (removing the leftover files of a live photo's replacement after a killed run). Model: opus-5-5
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
critical
Extremely important: data loss, a security hole, production down, or basic use impossible
 merge-ready
Review passed; ready to merge
 needs-checks
CI / make check is not green on this PR
 needs-rebase
PR conflicts with main and must be rebased
 needs-review
PR is awaiting adversarial review
 needs-rework
Review failed; PR needs changes
No labels
Milestone
No items
No Milestone 1.0.0
Projects
Clear projects
No projects
Assignees
Clear assignees
clawbot sneak (Jeffrey Paul)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/quak#199