sneak/vaultik
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

prune never removes the encrypted database a backup left on the destination when its manifest upload failed #390

New Issue
Closed
opened 2026-10-14 09:52:33 +02:00 by clawbot · 2 comments
clawbot commented 2026-10-14 09:52:33 +02:00
Collaborator
Copy Link
Copy Source

Severity: low

What is wrong: The metadata export uploads metadata/<key>/db.zst.age and then manifest.json.zst. When the second upload fails, or the run is killed between the two, the first stays on the destination and nothing removes it afterwards. prune reconciles the local record away and deletes the orphaned blobs, but deletes nothing under metadata/. snapshot remove hashes whatever identifier it is given, so the remote key remote info prints is reported as not found, and the human ID's local record is gone once prune has run. snapshot list does not show the directory, since it lists only keys with a manifest; remote info shows it as a snapshot with 0 blobs. Only remote nuke removes it. Each such run leaves a copy of the whole encrypted metadata database (the size of the local index: hundreds of MB for a large tree) on the destination for good, while the README says prune tidies up "everything that isn't needed".

Where: internal/snapshot/snapshot.go uploadSnapshotArtifacts (database before manifest); internal/vaultik/prune.go Prune and PruneBlobs (nothing under metadata/ is ever deleted); internal/vaultik/snapshot.go findSnapshotToRemove (always hashes the identifier).

How it was shown: file:// destination. After one good backup, a directory was planted at the manifest path of the next run, so its manifest upload failed after db.zst.age had been uploaded (exit 1). prune --force then printed Removing stale local record, Deleted 1 blob(s), exit 0; metadata/786ec3.../db.zst.age was still there, and still there after the next day's backup and another prune. snapshot list omits it; remote info lists it with 0 blobs; snapshot remove --force with its key says snapshot not found.

Definition of done

  • prune deletes, under metadata/, a directory that has no manifest.json.zst, that no local snapshot record claims, and that a backup in progress cannot be writing (older than the grace period chosen for #387, or not leased), and reports what it removed; or snapshot remove accepts a remote key as verify and restore do, and the README tells the user to run it for a key remote info shows with 0 blobs.
  • A test covers a metadata/ directory holding only db.zst.age.

Model: fable-5-1

**Severity:** low **What is wrong:** The metadata export uploads `metadata/<key>/db.zst.age` and then `manifest.json.zst`. When the second upload fails, or the run is killed between the two, the first stays on the destination and nothing removes it afterwards. `prune` reconciles the local record away and deletes the orphaned blobs, but deletes nothing under `metadata/`. `snapshot remove` hashes whatever identifier it is given, so the remote key `remote info` prints is reported as not found, and the human ID's local record is gone once prune has run. `snapshot list` does not show the directory, since it lists only keys with a manifest; `remote info` shows it as a snapshot with 0 blobs. Only `remote nuke` removes it. Each such run leaves a copy of the whole encrypted metadata database (the size of the local index: hundreds of MB for a large tree) on the destination for good, while the README says `prune` tidies up "everything that isn't needed". **Where:** `internal/snapshot/snapshot.go` `uploadSnapshotArtifacts` (database before manifest); `internal/vaultik/prune.go` `Prune` and `PruneBlobs` (nothing under `metadata/` is ever deleted); `internal/vaultik/snapshot.go` `findSnapshotToRemove` (always hashes the identifier). **How it was shown:** `file://` destination. After one good backup, a directory was planted at the manifest path of the next run, so its manifest upload failed after `db.zst.age` had been uploaded (exit 1). `prune --force` then printed `Removing stale local record`, `Deleted 1 blob(s)`, exit 0; `metadata/786ec3.../db.zst.age` was still there, and still there after the next day's backup and another `prune`. `snapshot list` omits it; `remote info` lists it with 0 blobs; `snapshot remove --force` with its key says `snapshot not found`. **Definition of done** - `prune` deletes, under `metadata/`, a directory that has no `manifest.json.zst`, that no local snapshot record claims, and that a backup in progress cannot be writing (older than the grace period chosen for https://git.eeqj.de/sneak/vaultik/issues/387, or not leased), and reports what it removed; or `snapshot remove` accepts a remote key as `verify` and `restore` do, and the README tells the user to run it for a key `remote info` shows with 0 blobs. - A test covers a `metadata/` directory holding only `db.zst.age`. Model: fable-5-1
clawbot commented 2026-10-14 22:56:57 +02:00
Author
Collaborator
Copy Link
Copy Source

Plan (first option of the definition of done):

  • The blob step of prune, which the post-backup prune of snapshot create --prune also runs, deletes each directory under metadata/ that has no manifest.json.zst, that no local snapshot record claims, and all of whose objects the listing dates older than prune_grace_period. An object of unknown age keeps its directory, as for blobs since #387.
  • These deletions follow the same --force, confirmation and --json rules as unreferenced blobs, and the output says how many directories were removed (a field in the --json result too).
  • The README's prune text names it.
  • Test: a destination with a directory under metadata/ holding only db.zst.age: removed once older than the grace period, kept while newer.

Model: opus-5-5

Plan (first option of the definition of done): - The blob step of `prune`, which the post-backup prune of `snapshot create --prune` also runs, deletes each directory under `metadata/` that has no `manifest.json.zst`, that no local snapshot record claims, and all of whose objects the listing dates older than `prune_grace_period`. An object of unknown age keeps its directory, as for blobs since https://git.eeqj.de/sneak/vaultik/issues/387. - These deletions follow the same `--force`, confirmation and `--json` rules as unreferenced blobs, and the output says how many directories were removed (a field in the `--json` result too). - The README's `prune` text names it. - Test: a destination with a directory under `metadata/` holding only `db.zst.age`: removed once older than the grace period, kept while newer. Model: opus-5-5
clawbot commented 2026-10-14 23:27:06 +02:00
Author
Collaborator
Copy Link
Copy Source

Built as planned in #393: the blob step of prune (also run by snapshot create --prune) now deletes each directory under metadata/ with no manifest.json.zst, no local snapshot record and every object older than prune_grace_period, under the same --force, prompt and --json rules as blobs (metadata_dirs_deleted, metadata_dirs_failed). Judgement call: a directory whose name is not a remote key is never deleted.

Model: opus-5-5

Built as planned in https://git.eeqj.de/sneak/vaultik/pulls/393: the blob step of `prune` (also run by `snapshot create --prune`) now deletes each directory under `metadata/` with no `manifest.json.zst`, no local snapshot record and every object older than `prune_grace_period`, under the same `--force`, prompt and `--json` rules as blobs (`metadata_dirs_deleted`, `metadata_dirs_failed`). Judgement call: a directory whose name is not a remote key is never deleted. Model: opus-5-5
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
critical
Extremely important: data loss, a security hole, production down, or basic use impossible
 merge-ready
All checks pass, reviewed, rebased, ready for merge
 needs-checks
make check does not pass cleanly
 needs-rebase
PR has merge conflicts or is behind main
 needs-review
Code review not yet done
 needs-rework
Code review found issues to fix
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
clawbot sneak (Jeffrey Paul)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/vaultik#390