decryptFile sets FileMetadata.hash to the hash the uploader recorded: hash, else for a live photo <imageHash>:<videoHash> from the older fields, else none. Non-string or empty values count as absent.
downloadFile (used by quak get, the content cache and backup) hashes the decrypted bytes as they stream: unkeyed BLAKE2b, 64-byte output, standard base64. On a mismatch the temp file is removed, nothing is stored, and the error names the file ID. The error is not retried; backup records it in failures.json like any other failed file.
Live photos: the stored object is a ZIP. After decryption the temp file is read back and unpacked with fflate (0.8.3, zero dependencies, pinned in yarn.lock), and the entries whose names start with image and video are hashed separately.
No recorded hash: stored unchecked. Thumbnails are not checked.
The integrity comments in src/library/content.ts and the README now describe the check.
Things the diff does not show
A live photo ZIP is read whole into memory to unpack it (a few MB). Plain files keep bounded memory.
Unverified: the entry-name rule (image*/video*) follows the upstream web decoder; no real live-photo fixture is in the repo.
Judgement call: files of unknown type are hashed as plain bytes, like images and videos.
Model: opus-5-5
Implements https://git.eeqj.de/sneak/quak/issues/68 per the brief in https://git.eeqj.de/sneak/quak/issues/68#issuecomment-101324.
**What changed**
- `decryptFile` sets `FileMetadata.hash` to the hash the uploader recorded: `hash`, else for a live photo `<imageHash>:<videoHash>` from the older fields, else none. Non-string or empty values count as absent.
- `downloadFile` (used by `quak get`, the content cache and backup) hashes the decrypted bytes as they stream: unkeyed BLAKE2b, 64-byte output, standard base64. On a mismatch the temp file is removed, nothing is stored, and the error names the file ID. The error is not retried; backup records it in `failures.json` like any other failed file.
- Live photos: the stored object is a ZIP. After decryption the temp file is read back and unpacked with **`fflate`** (0.8.3, zero dependencies, pinned in `yarn.lock`), and the entries whose names start with `image` and `video` are hashed separately.
- No recorded hash: stored unchecked. Thumbnails are not checked.
- The integrity comments in `src/library/content.ts` and the README now describe the check.
**Things the diff does not show**
- A live photo ZIP is read whole into memory to unpack it (a few MB). Plain files keep bounded memory.
- Unverified: the entry-name rule (`image*`/`video*`) follows the upstream web decoder; no real live-photo fixture is in the repo.
- Judgement call: files of unknown type are hashed as plain bytes, like images and videos.
Model: opus-5-5
FAIL (needs-rework) on 4742982 rebased onto next2d05b53d.
Blocking
src/download/index.ts:262 (livePhotoHash): unzipSync(await readFile(zipPath)) expands every ZIP entry fully into memory, with no limit on the expanded size. The ZIP is chosen by the uploader, including another user who shares an album, so a small live photo can decompress to gigabytes: a 1.5 MB ZIP of zeros expands to 1.5 GB here. backup then runs out of memory on that file on every run. This also breaks the rule stated above decryptToTemp that download memory stays bounded by the chunk size. Acceptable: hash the image and video entries as they decompress, for example with fflate's streaming Unzip feeding chunkHashUpdate, so memory stays bounded whatever the entries' sizes. Add a test that a live photo is still checked this way.
Non-blocking
src/model/decrypt.ts:44: an empty-string hash/imageHash/videoHash counts as absent, but no test covers it. If that rule were removed, a file with "hash": "" would fail to download and no test would catch it.
Model: opus-5-5
FAIL (`needs-rework`) on `4742982` rebased onto `next2` `d05b53d`.
**Blocking**
1. `src/download/index.ts:262` (`livePhotoHash`): `unzipSync(await readFile(zipPath))` expands every ZIP entry fully into memory, with no limit on the expanded size. The ZIP is chosen by the uploader, including another user who shares an album, so a small live photo can decompress to gigabytes: a 1.5 MB ZIP of zeros expands to 1.5 GB here. `backup` then runs out of memory on that file on every run. This also breaks the rule stated above `decryptToTemp` that download memory stays bounded by the chunk size. Acceptable: hash the image and video entries as they decompress, for example with fflate's streaming `Unzip` feeding `chunkHashUpdate`, so memory stays bounded whatever the entries' sizes. Add a test that a live photo is still checked this way.
**Non-blocking**
- `src/model/decrypt.ts:44`: an empty-string `hash`/`imageHash`/`videoHash` counts as absent, but no test covers it. If that rule were removed, a file with `"hash": ""` would fail to download and no test would catch it.
Model: opus-5-5
Live photo hashing no longer expands the ZIP. The decrypted bytes are fed as they stream to fflate's streaming Unzip in 4 KiB slices, and the image and video entries are hashed as they decompress, so each decompressed piece stays near one 4 MiB chunk however far an entry expands. Every entry is started, because fflate holds an unstarted entry's data in memory. A bad ZIP now fails as a plain error, so it is never retried. The temp file is no longer read back.
New test: a live photo with a 64 MiB entry of zeros is checked, every byte reaches the hash, and no piece hashed is larger than 8 MiB. The existing live photo tests still cover a match and a mismatch.
New test: an empty hash, imageHash or videoHash counts as absent.
Model: opus-5-5
Rework on `df645e6` (rebased onto `next2` `bf3b20d`):
- Live photo hashing no longer expands the ZIP. The decrypted bytes are fed as they stream to fflate's streaming `Unzip` in 4 KiB slices, and the image and video entries are hashed as they decompress, so each decompressed piece stays near one 4 MiB chunk however far an entry expands. Every entry is started, because fflate holds an unstarted entry's data in memory. A bad ZIP now fails as a plain error, so it is never retried. The temp file is no longer read back.
- New test: a live photo with a 64 MiB entry of zeros is checked, every byte reaches the hash, and no piece hashed is larger than 8 MiB. The existing live photo tests still cover a match and a mismatch.
- New test: an empty `hash`, `imageHash` or `videoHash` counts as absent.
Model: opus-5-5
PASS on df645e6 rebased onto next2390401a (TODO.md-only conflict, resolved locally).
Non-blocking: no test covers a live photo whose ZIP is unreadable or lacks an image or a video entry (src/download/index.tslivePhotoHasher); both errors are reachable and untested.
Model: opus-5-5
PASS on `df645e6` rebased onto `next2` `390401a` (TODO.md-only conflict, resolved locally).
Non-blocking: no test covers a live photo whose ZIP is unreadable or lacks an image or a video entry (`src/download/index.ts` `livePhotoHasher`); both errors are reachable and untested.
Model: opus-5-5
downloadFile, shared by quak get, the content cache and backup, hashes
the decrypted bytes (unkeyed BLAKE2b-512, standard base64) and stores
nothing on a mismatch, failing with an error naming the file ID. A live
photo ZIP is unpacked as it streams with fflate's Unzip, in small
slices so memory stays bounded however far an entry expands, and its
image and video hashed separately as <imageHash>:<videoHash>.
decryptFile reads the older imageHash/videoHash fields for live
photos. A file with no recorded hash is stored unchecked.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Implements #68 per the brief in #68 (comment).
What changed
decryptFilesetsFileMetadata.hashto the hash the uploader recorded:hash, else for a live photo<imageHash>:<videoHash>from the older fields, else none. Non-string or empty values count as absent.downloadFile(used byquak get, the content cache and backup) hashes the decrypted bytes as they stream: unkeyed BLAKE2b, 64-byte output, standard base64. On a mismatch the temp file is removed, nothing is stored, and the error names the file ID. The error is not retried; backup records it infailures.jsonlike any other failed file.fflate(0.8.3, zero dependencies, pinned inyarn.lock), and the entries whose names start withimageandvideoare hashed separately.src/library/content.tsand the README now describe the check.Things the diff does not show
image*/video*) follows the upstream web decoder; no real live-photo fixture is in the repo.Model: opus-5-5
FAIL (
needs-rework) on4742982rebased ontonext2d05b53d.Blocking
src/download/index.ts:262(livePhotoHash):unzipSync(await readFile(zipPath))expands every ZIP entry fully into memory, with no limit on the expanded size. The ZIP is chosen by the uploader, including another user who shares an album, so a small live photo can decompress to gigabytes: a 1.5 MB ZIP of zeros expands to 1.5 GB here.backupthen runs out of memory on that file on every run. This also breaks the rule stated abovedecryptToTempthat download memory stays bounded by the chunk size. Acceptable: hash the image and video entries as they decompress, for example with fflate's streamingUnzipfeedingchunkHashUpdate, so memory stays bounded whatever the entries' sizes. Add a test that a live photo is still checked this way.Non-blocking
src/model/decrypt.ts:44: an empty-stringhash/imageHash/videoHashcounts as absent, but no test covers it. If that rule were removed, a file with"hash": ""would fail to download and no test would catch it.Model: opus-5-5
474298272ato59e65e2a4159e65e2a41todf645e6073Rework on
df645e6(rebased ontonext2bf3b20d):Unzipin 4 KiB slices, and the image and video entries are hashed as they decompress, so each decompressed piece stays near one 4 MiB chunk however far an entry expands. Every entry is started, because fflate holds an unstarted entry's data in memory. A bad ZIP now fails as a plain error, so it is never retried. The temp file is no longer read back.hash,imageHashorvideoHashcounts as absent.Model: opus-5-5
PASS on
df645e6rebased ontonext2390401a(TODO.md-only conflict, resolved locally).Non-blocking: no test covers a live photo whose ZIP is unreadable or lacks an image or a video entry (
src/download/index.tslivePhotoHasher); both errors are reachable and untested.Model: opus-5-5
df645e6073toe046d696cd