Return an error, not a panic, on a malformed snapshot database #249

Merged
clawbot merged 1 commits from issue-231-malformed-snapshot-db-errors into next 2026-10-06 21:16:29 +02:00
Collaborator

Closes #231.

Restore and snapshot restore --verify read the downloaded snapshot database, which is not trusted, and a few places still crashed on a malformed row:

  • Error messages cut chunk hashes with [:16], so a shorter hash panicked (newRestorePlan and two messages in writeFileChunks). They now use the existing shortHash.
  • verifyFile used the chunks row without checking that it exists, and allocated the chunk size from that row in one piece. A missing row is now an error, a negative size is rejected, and the chunk is hashed with io.CopyN from the restored file, as deep verify already does for blob_chunks lengths.

restore_malformed_db_test.go writes a snapshot database by hand: a short chunk hash with no blob_chunks row, a short hash whose blob_chunks row reads past the end of its blob, and, under verify, a missing chunks row, a short hash, and a size that is negative or larger than the file. Each now gets an error. The missing-row case turns foreign keys off to write its row.

What the diff does not show:

  • A restored file shorter than its chunks now fails verify as short read instead of unexpected EOF; errShortChunkRead could not be reached before.
  • Judgement call: verifyFile's expectedHash[:16] was not in the issue's line list, but a short hash under --verify reaches it with the same panic, so it is fixed here too.
  • Judgement call: the remaining [:16] slices of blob hashes (downloadBlobToCache, the sweeper) are left alone, because buildBlobIndexes checks those hashes with isBlobHash first.

Model: opus-5-5

Closes https://git.eeqj.de/sneak/vaultik/issues/231. Restore and `snapshot restore --verify` read the downloaded snapshot database, which is not trusted, and a few places still crashed on a malformed row: - Error messages cut chunk hashes with `[:16]`, so a shorter hash panicked (`newRestorePlan` and two messages in `writeFileChunks`). They now use the existing `shortHash`. - `verifyFile` used the `chunks` row without checking that it exists, and allocated the chunk size from that row in one piece. A missing row is now an error, a negative size is rejected, and the chunk is hashed with `io.CopyN` from the restored file, as deep verify already does for `blob_chunks` lengths. `restore_malformed_db_test.go` writes a snapshot database by hand: a short chunk hash with no `blob_chunks` row, a short hash whose `blob_chunks` row reads past the end of its blob, and, under verify, a missing `chunks` row, a short hash, and a size that is negative or larger than the file. Each now gets an error. The missing-row case turns foreign keys off to write its row. What the diff does not show: - A restored file shorter than its chunks now fails verify as `short read` instead of `unexpected EOF`; `errShortChunkRead` could not be reached before. - Judgement call: `verifyFile`'s `expectedHash[:16]` was not in the issue's line list, but a short hash under `--verify` reaches it with the same panic, so it is fixed here too. - Judgement call: the remaining `[:16]` slices of blob hashes (`downloadBlobToCache`, the sweeper) are left alone, because `buildBlobIndexes` checks those hashes with `isBlobHash` first. Model: opus-5-5
clawbot added the needs-review label 2026-10-06 19:17:47 +02:00
clawbot self-assigned this 2026-10-06 19:17:47 +02:00
Author
Collaborator
  1. The branch no longer merges into current next (d276d89): TODO.md:25 conflicts with the #222 entry that landed at the top of Completed Steps. Rebase onto current next and keep both entries, this one first.

Model: opus-5-5

1. The branch no longer merges into current `next` (`d276d89`): `TODO.md:25` conflicts with the https://git.eeqj.de/sneak/vaultik/issues/222 entry that landed at the top of Completed Steps. Rebase onto current `next` and keep both entries, this one first. Model: opus-5-5
clawbot added needs-rebase and removed needs-review labels 2026-10-06 20:14:21 +02:00
clawbot added 1 commit 2026-10-06 20:38:33 +02:00
Restore cut chunk hashes from the snapshot database to 16 characters
for its error messages, so a shorter hash panicked. Those messages now
use shortHash. Under --verify, a file_chunks row with no chunks row was
dereferenced, and the chunk size from the database was allocated in
one piece, so a negative or huge size panicked. A missing row is now an
error, a negative size is rejected, and each chunk is hashed by
streaming it from the restored file.

A restored file shorter than its chunks now fails verify as a short
read instead of an unexpected EOF.

Model: opus-5-5
clawbot force-pushed issue-231-malformed-snapshot-db-errors from d035e89631 to 87b1c16205 2026-10-06 20:38:33 +02:00 Compare
Author
Collaborator
  1. Rebased onto current next (d276d89); TODO.md keeps both Completed Steps entries, the #231 entry first, then #222. Nothing else changed.

Model: opus-5-5

1. Rebased onto current `next` (`d276d89`); `TODO.md` keeps both Completed Steps entries, the https://git.eeqj.de/sneak/vaultik/issues/231 entry first, then https://git.eeqj.de/sneak/vaultik/issues/222. Nothing else changed. Model: opus-5-5
clawbot added needs-review and removed needs-rebase labels 2026-10-06 20:49:31 +02:00
Author
Collaborator

Review passed.

Model: opus-5-5

Review passed. Model: opus-5-5
clawbot merged commit f59086c0e5 into next 2026-10-06 21:16:29 +02:00
clawbot deleted branch issue-231-malformed-snapshot-db-errors 2026-10-06 21:16:29 +02:00
Sign in to join this conversation.