remote nuke leaves a snapshot database without a manifest, and non-blob files under blobs/, and says the store is empty #291

Open
opened 2026-10-13 00:29:19 +02:00 by clawbot · 0 comments
Collaborator

Found by the defect hunt in #288 (part 4).

What is wrong: NukeRemote (internal/vaultik/snapshot.go) deletes only the metadata keys that listAllRemoteSnapshotKeys returns, which requires a manifest.json.zst, and only the blobs that listAllRemoteBlobs returns, which skips names that are not blob hashes. Two kinds of object survive while nuke reports the store empty and exits 0:

  • metadata/KEY/db.zst.age with no manifest. uploadSnapshotArtifacts uploads the database before the manifest, so a failed manifest upload leaves this behind, and once the next backup drops the incomplete local record nothing removes it.
  • any name under blobs/ that is not a blob hash, such as a Finder .DS_Store.

The README says nuke leaves "the bucket prefix empty".

How it was shown: a backup whose manifest upload was blocked left metadata/52a6.../db.zst.age; after one good backup, remote nuke --force printed "Backup destination store is now empty.", exited 0, and that file was still there. With blobs/0b/01/.DS_Store present, nuke logged "Skipping non-conforming object under blobs/", printed "now empty", exited 0, and the file remained.

Definition of done

  • remote nuke --force deletes every object under metadata/ and blobs/, whatever its name, including a snapshot directory with no manifest, and including leftover .partial files behind a symlinked metadata/ or blobs/ directory (DeletePartialUploads does not follow such a link today; noticed while fixing #296).
  • It prints that the store is empty only when a final listing finds nothing left; otherwise it names what is left and exits non-zero.
  • Tests cover both kinds of leftover.

Model: opus-5-5

Found by the defect hunt in https://git.eeqj.de/sneak/vaultik/issues/288 (part 4). **What is wrong:** `NukeRemote` (`internal/vaultik/snapshot.go`) deletes only the metadata keys that `listAllRemoteSnapshotKeys` returns, which requires a `manifest.json.zst`, and only the blobs that `listAllRemoteBlobs` returns, which skips names that are not blob hashes. Two kinds of object survive while nuke reports the store empty and exits 0: - `metadata/KEY/db.zst.age` with no manifest. `uploadSnapshotArtifacts` uploads the database before the manifest, so a failed manifest upload leaves this behind, and once the next backup drops the incomplete local record nothing removes it. - any name under `blobs/` that is not a blob hash, such as a Finder `.DS_Store`. The README says nuke leaves "the bucket prefix empty". **How it was shown:** a backup whose manifest upload was blocked left `metadata/52a6.../db.zst.age`; after one good backup, `remote nuke --force` printed "Backup destination store is now empty.", exited 0, and that file was still there. With `blobs/0b/01/.DS_Store` present, nuke logged "Skipping non-conforming object under blobs/", printed "now empty", exited 0, and the file remained. **Definition of done** - `remote nuke --force` deletes every object under `metadata/` and `blobs/`, whatever its name, including a snapshot directory with no manifest, and including leftover `.partial` files behind a symlinked `metadata/` or `blobs/` directory (`DeletePartialUploads` does not follow such a link today; noticed while fixing https://git.eeqj.de/sneak/vaultik/issues/296). - It prints that the store is empty only when a final listing finds nothing left; otherwise it names what is left and exits non-zero. - Tests cover both kinds of leftover. Model: opus-5-5
clawbot self-assigned this 2026-10-13 00:29:19 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/vaultik#291