A reused blob is recorded with the size of a local copy that was never stored, so snapshot verify fails on a good snapshot #300

Open
opened 2026-10-13 00:47:09 +02:00 by clawbot · 1 comment
Collaborator

Found by the defect hunt in #288 (part 2).

What is wrong: when a finished blob already exists at the destination, the upload is skipped (uploadBlobIfNeeded in internal/snapshot/scanner.go), but the blob row and the manifest still get the size of the local copy (from the packer's UpdateFinished, through recordBlobMetadata; generateBlobManifest in internal/snapshot/snapshot.go writes it). If compression_level or the recipients changed since the stored object was written, the sizes differ: shallow and deep verify then fail, and the listed sizes are wrong. Restore works.

How it was shown: a 1.9 MB text file was backed up at level 3 (stored blob 602206 bytes), then database delete --force, compression_level: 19, and snapshot create: "Blob 23036003a9d6... (544 kB) already exists ... Skipping upload". After that snapshot verify reported "Wrong size: 1 blobs" (exit 1) and --deep "blob size mismatch: ... S3 has 602206 bytes, database has 544258 bytes" (exit 1).

Definition of done

  • When the upload is skipped because the blob is already stored, the blob row and the manifest record the size of the stored object.
  • Shallow and deep verify pass on such a snapshot, and a test covers it.

Model: opus-5-5

Found by the defect hunt in https://git.eeqj.de/sneak/vaultik/issues/288 (part 2). **What is wrong:** when a finished blob already exists at the destination, the upload is skipped (`uploadBlobIfNeeded` in `internal/snapshot/scanner.go`), but the blob row and the manifest still get the size of the local copy (from the packer's `UpdateFinished`, through `recordBlobMetadata`; `generateBlobManifest` in `internal/snapshot/snapshot.go` writes it). If `compression_level` or the recipients changed since the stored object was written, the sizes differ: shallow and deep verify then fail, and the listed sizes are wrong. Restore works. **How it was shown:** a 1.9 MB text file was backed up at level 3 (stored blob 602206 bytes), then `database delete --force`, `compression_level: 19`, and `snapshot create`: "Blob 23036003a9d6... (544 kB) already exists ... Skipping upload". After that `snapshot verify` reported "Wrong size: 1 blobs" (exit 1) and `--deep` "blob size mismatch: ... S3 has 602206 bytes, database has 544258 bytes" (exit 1). **Definition of done** - When the upload is skipped because the blob is already stored, the blob row and the manifest record the size of the stored object. - Shallow and deep verify pass on such a snapshot, and a test covers it. Model: opus-5-5
clawbot self-assigned this 2026-10-13 00:47:09 +02:00
Author
Collaborator

Fixed in #341: when a backup finds a blob already stored and skips the upload, the blob row now gets the size of the object in storage, so the manifest and the snapshot's database carry that size and shallow and deep snapshot verify pass on such a snapshot.

Model: opus-5-5

Fixed in https://git.eeqj.de/sneak/vaultik/pulls/341: when a backup finds a blob already stored and skips the upload, the blob row now gets the size of the object in storage, so the manifest and the snapshot's database carry that size and shallow and deep `snapshot verify` pass on such a snapshot. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/vaultik#300