One defect hunt on next before the release candidate: use vaultik the way a user would, and read the code paths for correctness. Real defects only: no new features, no refactors for taste, nothing already decided or waiting on sneak (#239 and #204 are out of scope).
Parts, one worker each
Use the built binary from a fresh config: config init, backups to a file:// destination (first, unchanged, changed, deleted and added files, more than one path), snapshot list, snapshot verify and --deep, snapshot restore (whole and by path, --verify), snapshot purge, snapshot remove, prune, remote info, remote nuke. Failure cases: a destination that is full (a small size-limited mount inside a container), not writable, or removed part-way; a missing or wrong private key; a damaged or missing blob; an interrupted backup and restore; a second writer while one runs.
Read the backup path: scan, chunking, packing, upload, local index writes, metadata export (internal/snapshot, internal/blob, internal/chunker, internal/blobgen writer, the backup side of internal/vaultik and internal/database).
Read restore and verify: internal/vaultik restore, restore plan, blob fetch and cache, verify; internal/blobgen reader; manifest and snapshot ID handling.
Read retention and destination management: prune, snapshot purge, snapshot remove, snapshot list, info, remote info, remote nuke, the process lock, and the snapshot tables they change.
Read the storage backends, config and command line: internal/storage, internal/s3, internal/config, internal/cli.
Definition of done
Each part's worker posts one comment here listing every defect it confirmed: what is wrong, where, how it was shown (a failing run or test), and what correct behaviour is. A candidate it could not confirm is left out.
Each confirmed defect not already covered by a closed or open issue is filed as its own issue with a definition of done, and fixed as one reviewed PR on next.
This issue closes when all five reports are in and filed.
Model: opus-5-5
One defect hunt on `next` before the release candidate: use vaultik the way a user would, and read the code paths for correctness. Real defects only: no new features, no refactors for taste, nothing already decided or waiting on sneak (https://git.eeqj.de/sneak/vaultik/issues/239 and https://git.eeqj.de/sneak/vaultik/issues/204 are out of scope).
**Parts, one worker each**
1. Use the built binary from a fresh config: `config init`, backups to a `file://` destination (first, unchanged, changed, deleted and added files, more than one path), `snapshot list`, `snapshot verify` and `--deep`, `snapshot restore` (whole and by path, `--verify`), `snapshot purge`, `snapshot remove`, `prune`, `remote info`, `remote nuke`. Failure cases: a destination that is full (a small size-limited mount inside a container), not writable, or removed part-way; a missing or wrong private key; a damaged or missing blob; an interrupted backup and restore; a second writer while one runs.
2. Read the backup path: scan, chunking, packing, upload, local index writes, metadata export (`internal/snapshot`, `internal/blob`, `internal/chunker`, `internal/blobgen` writer, the backup side of `internal/vaultik` and `internal/database`).
3. Read restore and verify: `internal/vaultik` restore, restore plan, blob fetch and cache, verify; `internal/blobgen` reader; manifest and snapshot ID handling.
4. Read retention and destination management: `prune`, `snapshot purge`, `snapshot remove`, `snapshot list`, `info`, `remote info`, `remote nuke`, the process lock, and the snapshot tables they change.
5. Read the storage backends, config and command line: `internal/storage`, `internal/s3`, `internal/config`, `internal/cli`.
**Definition of done**
- Each part's worker posts one comment here listing every defect it confirmed: what is wrong, where, how it was shown (a failing run or test), and what correct behaviour is. A candidate it could not confirm is left out.
- Each confirmed defect not already covered by a closed or open issue is filed as its own issue with a definition of done, and fixed as one reviewed PR on `next`.
- This issue closes when all five reports are in and filed.
Model: opus-5-5
clawbot
self-assigned this 2026-10-13 00:12:58 +02:00
Seven defects confirmed with real runs of ./vaultik (make build on next at fa56165, file:// destination, own config, index and lock directory). No test was written.
1. A retention duration of 293 years or more deletes every snapshot
What is wrong:parseDuration (internal/vaultik/helpers.go) computes time.Duration(n) * year (and the same for w, mo, d) with no overflow check. From 293y up the product wraps to a negative value, so selectSnapshotsToPurge (internal/vaultik/snapshot.go) puts the cutoff in the future and selects every snapshot. A user who writes --prune --keep-newer-than 1000y to keep everything loses every snapshot of that name, including the one just made, and then PruneBlobs deletes their blobs. The command exits 0.
How shown: After two backups, echo n | vaultik snapshot purge --older-than 1000y listed all four snapshots, including one taken a second earlier. 292y lists none, while 293y, 15300w and 3600mo each list all of them. vaultik snapshot create --cron --prune --keep-newer-than 1000y home exited 0 and left no home snapshot and none of its blobs on the store.
Correct: a duration too large to represent is rejected as invalid. It never selects a snapshot newer than the duration.
2. prune and remote nuke exit 0 when a delete fails, and nuke still says the store is empty
What is wrong:deleteUnreferencedBlobs (internal/vaultik/prune.go) logs each failed blob delete and counts it in BlobsFailed, but PruneBlobs still returns nil. removeAllLocalSnapshots and removeAllOrphanRemotes (internal/vaultik/snapshot.go) count failed metadata deletes and only warn. After either kind of failure, NukeRemote prints "Backup destination store is now empty." The nuke warning also says "Run 'vaultik prune' once the remote is healthy", but prune never removes metadata (issue #251 fixed the same advice in snapshot remove).
How shown: With one blob's directory made read-only (chmod 555), prune --json printed "blobs_failed": 1 and exited 0. remote nuke --force printed "Failed to delete 1 blob(s)", then "Backup destination store is now empty.", exited 0, and left the blob in place. With one metadata directory read-only instead, nuke printed the warning and "now empty", exited 0, and left four files.
Correct: the exit status is non-zero when any delete fails, and nuke does not claim the store is empty.
3. remote nuke leaves objects it does not recognise and says the store is empty
What is wrong:NukeRemote deletes only the metadata keys that listAllRemoteSnapshotKeys returns, which requires a manifest.json.zst, and only the blobs that listAllRemoteBlobs returns, which skips names that are not blob hashes. Two kinds of object therefore survive:
metadata/KEY/db.zst.age with no manifest. uploadSnapshotArtifacts uploads the database before the manifest, so a failed manifest upload leaves this behind. Once the next backup drops the incomplete local record, nothing removes it.
any name under blobs/ that is not a blob hash, such as a Finder .DS_Store.
The README says nuke leaves "the bucket prefix empty".
How shown: I blocked a real backup's manifest upload by putting a directory at the manifest path. The backup failed and left metadata/52a6.../db.zst.age with no manifest. After one good backup, remote nuke --force printed "Backup destination store is now empty." and exited 0, and that db.zst.age was still there. With blobs/0b/01/.DS_Store present, nuke logged "Skipping non-conforming object under blobs/", printed "now empty", exited 0, and the file remained.
Correct: nuke deletes everything under metadata/ and blobs/, or it names what it left and exits non-zero.
4. remote nuke against an unreachable destination wipes the local index, then fails
What is wrong:RemoveAllSnapshots (internal/vaultik/snapshot.go) turns a failed listing into a warning. Its comment says this is so a local-only remove is not blocked, but nuke is not local-only. executeRemoveAll then deletes every local snapshot record even though each remote delete fails. CleanupOrphanedData drops every file, chunk and blob row. Only after that does PruneBlobs fail.
How shown: With two snapshots, I renamed the destination directory to simulate an unplugged volume and ran remote nuke --force. It printed "Removed 2 snapshot(s)" and "Run 'vaultik prune' once the remote is healthy ...", then "Remote nuke failed", and exited 1. The local index then held 0 snapshots, 0 files and 0 blobs. With the directory back, the store still held all six objects. The snapshots are now remote-only rows with their names lost, and prune never removes them.
Correct: nuke stops before it touches the local index when it cannot list the destination, and at most drops the local records of snapshots whose remote metadata it actually deleted.
5. snapshot remove reports success for a snapshot that does not exist
What is wrong:RemoveSnapshot (internal/vaultik/snapshot.go) never checks that the ID exists. deleteSnapshotFromLocalDB deletes zero rows without error, and deleteRemoteSnapshotByKey finds nothing to delete.
How shown:snapshot remove --force hunt_home_1999-01-01T00:00:00Z printed "Removed snapshot ... from local database" and "Removed snapshot metadata from remote storage", and exited 0. --json gave "remote_removed": true, and --dry-run printed "Would remove snapshot". Passing the remote key that snapshot list prints for a snapshot (restore and verify accept it) gave the same two success lines and left that metadata/KEY/ untouched.
Correct: an ID found in neither the local index nor the store is an error with a non-zero exit. The alternative is to accept a remote key, as restore does.
6. --cron and --quiet do not silence prune output, so the README's cron line produces output on every run
What is wrong:PruneBlobs (internal/vaultik/prune.go) and CleanupLocalSnapshots (internal/vaultik/snapshot.go) write their status lines with stdoutf and printlnStdout, not through internal/ui. The README says --cron is silent on total success and that --quiet silences status lines.
How shown:snapshot create --cron --prune etc and snapshot create --cron --prune --keep-newer-than 4w etc both exited 0 and printed "No unreferenced blobs to remove." on stdout, or "Found N unreferenced blob(s) ... Deleted N blob(s) ..." when there were blobs to delete. vaultik -q prune --force printed "No stale local snapshots found." and "No unreferenced blobs to remove."
Correct: a successful run prints nothing under --cron or --quiet.
7. remote info counts a non-blob file under blobs/ as an orphaned blob
What is wrong:scanRemoteBlobStorage (internal/vaultik/info.go) counts every four-part key under blobs/ as a blob. Prune's listAllRemoteBlobs skips names that are not blob hashes.
How shown: With blobs/56/f6/.DS_Store present, remote info reported "Total blobs on remote: 3", "Orphaned (unreferenced): 1 (4 B)" and "Run 'vaultik prune' to remove orphaned blobs." prune --force logged "Skipping non-conforming object under blobs/" and printed "No unreferenced blobs to remove.", and remote info kept giving the same advice.
Correct:remote info applies the same blob-name check as prune.
Outside this part (for part 5 to take or drop)
chunk_size: 1MB is rejected with "chunk_size must be at least 1MB" (internal/config/config.go): the minimum is 1 MiB, while 1MB is read as 1,000,000 bytes. 1MiB works.
Model: opus-5-5
## Part 4: Read retention and destination management
Seven defects confirmed with real runs of `./vaultik` (`make build` on `next` at `fa56165`, `file://` destination, own config, index and lock directory). No test was written.
### 1. A retention duration of 293 years or more deletes every snapshot
- **What is wrong:** `parseDuration` (`internal/vaultik/helpers.go`) computes `time.Duration(n) * year` (and the same for `w`, `mo`, `d`) with no overflow check. From `293y` up the product wraps to a negative value, so `selectSnapshotsToPurge` (`internal/vaultik/snapshot.go`) puts the cutoff in the future and selects every snapshot. A user who writes `--prune --keep-newer-than 1000y` to keep everything loses every snapshot of that name, including the one just made, and then `PruneBlobs` deletes their blobs. The command exits 0.
- **How shown:** After two backups, `echo n | vaultik snapshot purge --older-than 1000y` listed all four snapshots, including one taken a second earlier. `292y` lists none, while `293y`, `15300w` and `3600mo` each list all of them. `vaultik snapshot create --cron --prune --keep-newer-than 1000y home` exited 0 and left no `home` snapshot and none of its blobs on the store.
- **Correct:** a duration too large to represent is rejected as invalid. It never selects a snapshot newer than the duration.
### 2. `prune` and `remote nuke` exit 0 when a delete fails, and nuke still says the store is empty
- **What is wrong:** `deleteUnreferencedBlobs` (`internal/vaultik/prune.go`) logs each failed blob delete and counts it in `BlobsFailed`, but `PruneBlobs` still returns nil. `removeAllLocalSnapshots` and `removeAllOrphanRemotes` (`internal/vaultik/snapshot.go`) count failed metadata deletes and only warn. After either kind of failure, `NukeRemote` prints "Backup destination store is now empty." The nuke warning also says "Run 'vaultik prune' once the remote is healthy", but prune never removes metadata (issue #251 fixed the same advice in `snapshot remove`).
- **How shown:** With one blob's directory made read-only (`chmod 555`), `prune --json` printed `"blobs_failed": 1` and exited 0. `remote nuke --force` printed "Failed to delete 1 blob(s)", then "Backup destination store is now empty.", exited 0, and left the blob in place. With one metadata directory read-only instead, nuke printed the warning and "now empty", exited 0, and left four files.
- **Correct:** the exit status is non-zero when any delete fails, and nuke does not claim the store is empty.
### 3. `remote nuke` leaves objects it does not recognise and says the store is empty
- **What is wrong:** `NukeRemote` deletes only the metadata keys that `listAllRemoteSnapshotKeys` returns, which requires a `manifest.json.zst`, and only the blobs that `listAllRemoteBlobs` returns, which skips names that are not blob hashes. Two kinds of object therefore survive:
- `metadata/KEY/db.zst.age` with no manifest. `uploadSnapshotArtifacts` uploads the database before the manifest, so a failed manifest upload leaves this behind. Once the next backup drops the incomplete local record, nothing removes it.
- any name under `blobs/` that is not a blob hash, such as a Finder `.DS_Store`.
The README says nuke leaves "the bucket prefix empty".
- **How shown:** I blocked a real backup's manifest upload by putting a directory at the manifest path. The backup failed and left `metadata/52a6.../db.zst.age` with no manifest. After one good backup, `remote nuke --force` printed "Backup destination store is now empty." and exited 0, and that `db.zst.age` was still there. With `blobs/0b/01/.DS_Store` present, nuke logged "Skipping non-conforming object under blobs/", printed "now empty", exited 0, and the file remained.
- **Correct:** nuke deletes everything under `metadata/` and `blobs/`, or it names what it left and exits non-zero.
### 4. `remote nuke` against an unreachable destination wipes the local index, then fails
- **What is wrong:** `RemoveAllSnapshots` (`internal/vaultik/snapshot.go`) turns a failed listing into a warning. Its comment says this is so a local-only remove is not blocked, but nuke is not local-only. `executeRemoveAll` then deletes every local snapshot record even though each remote delete fails. `CleanupOrphanedData` drops every file, chunk and blob row. Only after that does `PruneBlobs` fail.
- **How shown:** With two snapshots, I renamed the destination directory to simulate an unplugged volume and ran `remote nuke --force`. It printed "Removed 2 snapshot(s)" and "Run 'vaultik prune' once the remote is healthy ...", then "Remote nuke failed", and exited 1. The local index then held 0 snapshots, 0 files and 0 blobs. With the directory back, the store still held all six objects. The snapshots are now remote-only rows with their names lost, and `prune` never removes them.
- **Correct:** nuke stops before it touches the local index when it cannot list the destination, and at most drops the local records of snapshots whose remote metadata it actually deleted.
### 5. `snapshot remove` reports success for a snapshot that does not exist
- **What is wrong:** `RemoveSnapshot` (`internal/vaultik/snapshot.go`) never checks that the ID exists. `deleteSnapshotFromLocalDB` deletes zero rows without error, and `deleteRemoteSnapshotByKey` finds nothing to delete.
- **How shown:** `snapshot remove --force hunt_home_1999-01-01T00:00:00Z` printed "Removed snapshot ... from local database" and "Removed snapshot metadata from remote storage", and exited 0. `--json` gave `"remote_removed": true`, and `--dry-run` printed "Would remove snapshot". Passing the remote key that `snapshot list` prints for a snapshot (restore and verify accept it) gave the same two success lines and left that `metadata/KEY/` untouched.
- **Correct:** an ID found in neither the local index nor the store is an error with a non-zero exit. The alternative is to accept a remote key, as restore does.
### 6. `--cron` and `--quiet` do not silence prune output, so the README's cron line produces output on every run
- **What is wrong:** `PruneBlobs` (`internal/vaultik/prune.go`) and `CleanupLocalSnapshots` (`internal/vaultik/snapshot.go`) write their status lines with `stdoutf` and `printlnStdout`, not through `internal/ui`. The README says `--cron` is silent on total success and that `--quiet` silences status lines.
- **How shown:** `snapshot create --cron --prune etc` and `snapshot create --cron --prune --keep-newer-than 4w etc` both exited 0 and printed "No unreferenced blobs to remove." on stdout, or "Found N unreferenced blob(s) ... Deleted N blob(s) ..." when there were blobs to delete. `vaultik -q prune --force` printed "No stale local snapshots found." and "No unreferenced blobs to remove."
- **Correct:** a successful run prints nothing under `--cron` or `--quiet`.
### 7. `remote info` counts a non-blob file under `blobs/` as an orphaned blob
- **What is wrong:** `scanRemoteBlobStorage` (`internal/vaultik/info.go`) counts every four-part key under `blobs/` as a blob. Prune's `listAllRemoteBlobs` skips names that are not blob hashes.
- **How shown:** With `blobs/56/f6/.DS_Store` present, `remote info` reported "Total blobs on remote: 3", "Orphaned (unreferenced): 1 (4 B)" and "Run 'vaultik prune' to remove orphaned blobs." `prune --force` logged "Skipping non-conforming object under blobs/" and printed "No unreferenced blobs to remove.", and `remote info` kept giving the same advice.
- **Correct:** `remote info` applies the same blob-name check as prune.
### Outside this part (for part 5 to take or drop)
`chunk_size: 1MB` is rejected with "chunk_size must be at least 1MB" (`internal/config/config.go`): the minimum is 1 MiB, while `1MB` is read as 1,000,000 bytes. `1MiB` works.
Model: opus-5-5
Eight confirmed defects on next at fa56165, plus one config defect found while setting up (outside part 2).
1. With the legacy s3.* configuration, a changed bucket or prefix is not detected: the next backup reports success and cannot be restored
What: the local index is bound to its destination only through storage_url. With the s3.* form (README: still supported), the check is skipped. After s3.bucket or s3.prefix changes, the next backup treats every chunk as already stored, uploads nothing, and completes a snapshot whose blobs are only at the old destination.
Shown: throwaway test TestHuntLegacyS3ConfigNewBucketIsRefused (internal/vaultik, kept local). It uses a legacy config (empty StorageURL), backs up to bucket A, sets s3.bucket to B and backs up to B. The second backup succeeds: backup to the new bucket reported success with 0 blobs uploaded there; restoring test-host_second_... from it: ... downloading blob 328c1947f63b2fd2: object not found.
Correct: bind legacy configs too (for example on endpoint, bucket and prefix) and refuse on a mismatch, as for storage_url.
2. A reused blob keeps the size of a local re-encryption that was never stored, so snapshot verify fails on a good snapshot
What: when a finished blob already exists at the destination, the upload is skipped. The blob row and the manifest still get the size of the local copy. If compression_level or the recipients changed since the stored object was written, the sizes differ. Shallow and deep verify then fail, and the listed sizes are wrong. Restore works.
Where: internal/snapshot/scanner.gouploadBlobIfNeeded (Stat succeeds, upload skipped) and recordBlobMetadata. The size comes from the packer's UpdateFinished, and internal/snapshot/snapshot.gogenerateBlobManifest writes it.
Shown: back up a 1.9 MB text file at level 3 (stored blob 602206 bytes), run database delete --force, set compression_level: 19, then snapshot create: "Blob 23036003a9d6... (544 kB) already exists ... Skipping upload". After that, snapshot verify reports "Wrong size: 1 blobs" (exit 1), and --deep reports "blob size mismatch: ... S3 has 602206 bytes, database has 544258 bytes" (exit 1). restore --verify succeeds and the file is identical.
Correct: record the size of the object actually stored (Stat returns it), or upload over it.
3. Ctrl-C during a backup transaction exits 1 with "FATAL: failed to rollback transaction"
What: transactions are bound to the run's context. On cancel, database/sql rolls the transaction back itself, so the explicit Rollback returns sql.ErrTxDone. WithTx treats that as fatal and calls os.Exit(1). The user gets a message that suggests index damage, and exit 1 instead of 130 "interrupted before the command finished". Deferred cleanup is skipped.
Where: internal/database/repositories.goWithTx (Fatalf on any rollback error).
Shown: 8000 empty files, one backup, then 6 runs, each sent SIGINT as soon as "Associating ... unchanged files" printed. All 6 exited 1 with FATAL: failed to rollback transaction: sql: transaction has already been committed or rolled back.
Correct: treat ErrTxDone after a failed body as rolled back and return the body's error, so the run ends as interrupted (130).
4. A failed or interrupted backup leaves its half-written blob in $TMPDIR
What: when snapshot create stops on a file error or Ctrl-C between blobs, the blob being packed is never closed or removed. Its vaultik-blob-*.tmp (encrypted data, up to blob_size_limit, 10 GB by default) stays behind, and every such run adds one. Nothing removes them later.
Where: internal/snapshot/scanner.goprocessPhase returns without discarding the packer's current blob. internal/blob/packer.go creates the file in startNewBlob and removes it only in finalizeCurrentBlob.
Shown: a source with a readable 200 kB a1 and a chmod 000 file b2. snapshot create exits 1 and leaves $TMPDIR/vaultik-blob-619287593.tmp (131 kB); a second run left a second file. A SIGINT run (exit 130) left vaultik-blob-846065746.tmp.
Correct: on any error or cancel, close and remove the in-progress blob's temp file.
5. **/name exclude patterns miss a match directly under the source root
What: the config init template says patterns without a leading / match anywhere in the tree and that ** is supported. **/node_modules and **/*.pyc exclude sub/node_modules and sub/y.pyc, but back up node_modules/ and x.pyc at the top of the source path. config.example.yml uses exactly these two patterns.
Where: internal/snapshot/scanner.gomatchesUnanchored. None of the strings it tries for a top-level entry contains the / that the pattern requires.
Shown: a snapshot with exclude: ["**/node_modules", "**/*.pyc"]; after the backup the index holds src/node_modules/top.js and src/x.pyc.
Correct: **/x also matches x at the source root, as in gitignore.
6. "deleted" counts files that another snapshot backs up, on every run
What: the deleted figure counts every indexed file under the source path that this run's walk did not record. That includes files this snapshot excludes but another snapshot backs up. In config.example.yml, home excludes /Library/Caches, which system (/) backs up, and dev excludes **/node_modules, which home backs up.
Where: internal/snapshot/scanner.goloadKnownFiles and detectDeletedFilesFromMap.
Shown: home = src excluding /photos, and photos = src/photos with 3 files. Every run of home after the first prints "detected 4 deleted files" and "Files: 1 examined, 0 backed up, 1 unchanged, 4 deleted", "6.0 kB deleted", with nothing deleted.
Correct: count a file as deleted only when it no longer exists, not when it is excluded from this snapshot.
7. Directories and symlinks are counted as files to back up on every run
What: every run queues every directory and symlink for phase 2. The counts mix them in inconsistently. A first backup of 7 files and 1 symlink in 4 directories prints "Enumerated 8 snapshot source files (20 kB total), 12 to back up". A second run with nothing changed prints "5 to back up (23 B)", "Backing up 5 snapshot source files", "Files: 8 examined, 1 backed up, 7 unchanged" and "Data: 20 kB total (4 B backed up)". The symlink is always "backed up" with its target length counted as data, and the directories' own stat sizes count as data to back up. With nested snapshots the same run printed "(11 B total)" in the enumeration line and "Data: 4 B total" in the summary.
Where: internal/snapshot/scanner.gorecordSpecialEntry (queues every directory and symlink; symlinks counted as changed through updateScanEntryStats) and summarizeScanPhase (counts and sizes filesToProcess, while the enumerated count leaves directories out).
Correct: both sides of the enumeration line count the same entries. An unchanged tree reports nothing to back up and nothing backed up, and directory sizes are not counted as data.
8. The pre-backup index prune reports total rows as "remain"
What: every snapshot create prints lines such as "Incomplete snapshots: 0 removed (1 remain)" and "Orphaned files: 0 removed (12 remain)". The second figure is all snapshots, files, chunks or blobs left in the index, not incomplete or orphaned ones. After removing the only incomplete snapshot it printed "1 removed (1 remain)".
Where: internal/vaultik/snapshot.goPruneDatabase (the four Detailf lines).
Correct: say what the figure counts, or print the number of incomplete or orphaned rows left.
Outside part 2 (config), found while setting up
chunk_size: 1MB is rejected with "chunk_size must be at least 1MB": the minimum is 1048576 and 1MB parses as 1000000. The config init template's chunk_size comment (internal/cli/config.go) says "Accepts: 1MB, 10M, 64KB", but 1MB and 64KB are both rejected. Where: internal/config/config.gominChunkSize and errChunkSizeTooSmall. Correct: the message names the real minimum, and the template lists only accepted values.
Model: opus-5-5
## Part 2: Read the backup path
Eight confirmed defects on `next` at `fa56165`, plus one config defect found while setting up (outside part 2).
**1. With the legacy `s3.*` configuration, a changed bucket or prefix is not detected: the next backup reports success and cannot be restored**
- What: the local index is bound to its destination only through `storage_url`. With the `s3.*` form (README: still supported), the check is skipped. After `s3.bucket` or `s3.prefix` changes, the next backup treats every chunk as already stored, uploads nothing, and completes a snapshot whose blobs are only at the old destination.
- Where: `internal/vaultik/storage_bind.go` `EnsureStorageBinding` (`if configured == ""` returns nil).
- Shown: throwaway test `TestHuntLegacyS3ConfigNewBucketIsRefused` (`internal/vaultik`, kept local). It uses a legacy config (empty `StorageURL`), backs up to bucket A, sets `s3.bucket` to B and backs up to B. The second backup succeeds: `backup to the new bucket reported success with 0 blobs uploaded there; restoring test-host_second_... from it: ... downloading blob 328c1947f63b2fd2: object not found`.
- Correct: bind legacy configs too (for example on endpoint, bucket and prefix) and refuse on a mismatch, as for `storage_url`.
**2. A reused blob keeps the size of a local re-encryption that was never stored, so `snapshot verify` fails on a good snapshot**
- What: when a finished blob already exists at the destination, the upload is skipped. The blob row and the manifest still get the size of the local copy. If `compression_level` or the recipients changed since the stored object was written, the sizes differ. Shallow and deep verify then fail, and the listed sizes are wrong. Restore works.
- Where: `internal/snapshot/scanner.go` `uploadBlobIfNeeded` (Stat succeeds, upload skipped) and `recordBlobMetadata`. The size comes from the packer's `UpdateFinished`, and `internal/snapshot/snapshot.go` `generateBlobManifest` writes it.
- Shown: back up a 1.9 MB text file at level 3 (stored blob 602206 bytes), run `database delete --force`, set `compression_level: 19`, then `snapshot create`: "Blob 23036003a9d6... (544 kB) already exists ... Skipping upload". After that, `snapshot verify` reports "Wrong size: 1 blobs" (exit 1), and `--deep` reports "blob size mismatch: ... S3 has 602206 bytes, database has 544258 bytes" (exit 1). `restore --verify` succeeds and the file is identical.
- Correct: record the size of the object actually stored (Stat returns it), or upload over it.
**3. Ctrl-C during a backup transaction exits 1 with "FATAL: failed to rollback transaction"**
- What: transactions are bound to the run's context. On cancel, `database/sql` rolls the transaction back itself, so the explicit `Rollback` returns `sql.ErrTxDone`. `WithTx` treats that as fatal and calls `os.Exit(1)`. The user gets a message that suggests index damage, and exit 1 instead of 130 "interrupted before the command finished". Deferred cleanup is skipped.
- Where: `internal/database/repositories.go` `WithTx` (`Fatalf` on any rollback error).
- Shown: 8000 empty files, one backup, then 6 runs, each sent SIGINT as soon as "Associating ... unchanged files" printed. All 6 exited 1 with `FATAL: failed to rollback transaction: sql: transaction has already been committed or rolled back`.
- Correct: treat `ErrTxDone` after a failed body as rolled back and return the body's error, so the run ends as interrupted (130).
**4. A failed or interrupted backup leaves its half-written blob in `$TMPDIR`**
- What: when `snapshot create` stops on a file error or Ctrl-C between blobs, the blob being packed is never closed or removed. Its `vaultik-blob-*.tmp` (encrypted data, up to `blob_size_limit`, 10 GB by default) stays behind, and every such run adds one. Nothing removes them later.
- Where: `internal/snapshot/scanner.go` `processPhase` returns without discarding the packer's current blob. `internal/blob/packer.go` creates the file in `startNewBlob` and removes it only in `finalizeCurrentBlob`.
- Shown: a source with a readable 200 kB `a1` and a `chmod 000` file `b2`. `snapshot create` exits 1 and leaves `$TMPDIR/vaultik-blob-619287593.tmp` (131 kB); a second run left a second file. A SIGINT run (exit 130) left `vaultik-blob-846065746.tmp`.
- Correct: on any error or cancel, close and remove the in-progress blob's temp file.
**5. `**/name` exclude patterns miss a match directly under the source root**
- What: the `config init` template says patterns without a leading `/` match anywhere in the tree and that `**` is supported. `**/node_modules` and `**/*.pyc` exclude `sub/node_modules` and `sub/y.pyc`, but back up `node_modules/` and `x.pyc` at the top of the source path. `config.example.yml` uses exactly these two patterns.
- Where: `internal/snapshot/scanner.go` `matchesUnanchored`. None of the strings it tries for a top-level entry contains the `/` that the pattern requires.
- Shown: a snapshot with `exclude: ["**/node_modules", "**/*.pyc"]`; after the backup the index holds `src/node_modules/top.js` and `src/x.pyc`.
- Correct: `**/x` also matches `x` at the source root, as in gitignore.
**6. "deleted" counts files that another snapshot backs up, on every run**
- What: the deleted figure counts every indexed file under the source path that this run's walk did not record. That includes files this snapshot excludes but another snapshot backs up. In `config.example.yml`, `home` excludes `/Library/Caches`, which `system` (`/`) backs up, and `dev` excludes `**/node_modules`, which `home` backs up.
- Where: `internal/snapshot/scanner.go` `loadKnownFiles` and `detectDeletedFilesFromMap`.
- Shown: `home` = `src` excluding `/photos`, and `photos` = `src/photos` with 3 files. Every run of `home` after the first prints "detected 4 deleted files" and "Files: 1 examined, 0 backed up, 1 unchanged, 4 deleted", "6.0 kB deleted", with nothing deleted.
- Correct: count a file as deleted only when it no longer exists, not when it is excluded from this snapshot.
**7. Directories and symlinks are counted as files to back up on every run**
- What: every run queues every directory and symlink for phase 2. The counts mix them in inconsistently. A first backup of 7 files and 1 symlink in 4 directories prints "Enumerated 8 snapshot source files (20 kB total), 12 to back up". A second run with nothing changed prints "5 to back up (23 B)", "Backing up 5 snapshot source files", "Files: 8 examined, 1 backed up, 7 unchanged" and "Data: 20 kB total (4 B backed up)". The symlink is always "backed up" with its target length counted as data, and the directories' own stat sizes count as data to back up. With nested snapshots the same run printed "(11 B total)" in the enumeration line and "Data: 4 B total" in the summary.
- Where: `internal/snapshot/scanner.go` `recordSpecialEntry` (queues every directory and symlink; symlinks counted as changed through `updateScanEntryStats`) and `summarizeScanPhase` (counts and sizes `filesToProcess`, while the enumerated count leaves directories out).
- Correct: both sides of the enumeration line count the same entries. An unchanged tree reports nothing to back up and nothing backed up, and directory sizes are not counted as data.
**8. The pre-backup index prune reports total rows as "remain"**
- What: every `snapshot create` prints lines such as "Incomplete snapshots: 0 removed (1 remain)" and "Orphaned files: 0 removed (12 remain)". The second figure is all snapshots, files, chunks or blobs left in the index, not incomplete or orphaned ones. After removing the only incomplete snapshot it printed "1 removed (1 remain)".
- Where: `internal/vaultik/snapshot.go` `PruneDatabase` (the four `Detailf` lines).
- Correct: say what the figure counts, or print the number of incomplete or orphaned rows left.
**Outside part 2 (config), found while setting up**
- `chunk_size: 1MB` is rejected with "chunk_size must be at least 1MB": the minimum is 1048576 and `1MB` parses as 1000000. The `config init` template's `chunk_size` comment (`internal/cli/config.go`) says "Accepts: 1MB, 10M, 64KB", but `1MB` and `64KB` are both rejected. Where: `internal/config/config.go` `minChunkSize` and `errChunkSizeTooSmall`. Correct: the message names the real minimum, and the template lists only accepted values.
Model: opus-5-5
make build on next at fa56165, file:// destinations, own config, index, lock and $TMPDIR. Every item was shown with real runs of ./vaultik; no test was written.
1. A destination removed part-way through a backup is recreated, and the backup reports success with a blob missing
What: if the destination directory disappears after some blobs are uploaded, the next upload recreates it. The run prints "Finished successfully." and exits 0, but a blob it uploaded earlier is gone, so the snapshot cannot be restored. The local index still records that blob as stored, so every later backup also exits 0 ("0 backed up, 53 unchanged") and is just as unrestorable. Only snapshot verify notices.
Where: internal/storage/file.gowriteAtomic (MkdirAll for the blob's directory also recreates the destination root).
Shown: 9 MB source packed into 2 blobs. The backup was paused (SIGSTOP) once the first blob landed, the destination removed, the backup resumed: exit 0, destination holds 1 of 2 blobs. snapshot verify says "1 blobs are missing" (exit 1). The next snapshot create exits 0; restoring its snapshot fails with "downloading blob d750...: object not found".
Correct: a write never recreates a missing file:// destination root, so the run fails and the next run discards the incomplete snapshot's blobs and uploads them again. This is the mid-run case, not #239's start-of-run case, but the owner's answer there (a missing file:// destination is an error) covers it if enforced on every write.
2. An invalid --keep-newer-than is rejected only after the backup, which then reports "Snapshot creation failed" although the snapshot was made
What: the duration is first parsed by the post-backup prune. With a value the parser rejects (1.5d, 0), the backup completes and uploads its snapshot, then the command prints "Snapshot creation failed: post-backup prune: purging old snapshots: invalid duration" and exits 1. Retention never runs, so such a cron line adds a snapshot every night, reports each as failed, and never deletes any.
Where: internal/vaultik/snapshot.gorunPostBackupPrune (gets the raw string); internal/cli/snapshot.go does not check --keep-newer-than.
Shown: snapshot create ex --prune --keep-newer-than 1.5d exits 1 with that message; snapshot list --json then shows the new snapshot testhost_ex_2026-10-12T22:33:14Z with remote_present: true.
Correct: reject the duration before the backup starts (non-zero exit, nothing created). A failure in the post-backup prune is reported as a prune failure, not as a failed snapshot.
3. config set with a list index on a key that does not exist writes a map, and then no command can load the config
What: vaultik config set age_recipients.0 age1... (the README quickstart command) on a config without an age_recipients key exits 0 and writes age_recipients: / 0: age1.... Every later command fails with "failed to parse config: yaml: unmarshal errors: cannot unmarshal !!map into []string". Same with config set snapshots.NAME.paths.0 /path. An index into an existing list works.
Where: internal/cli/config.goyamlSetInMapping (creates a mapping for any missing key, even when the next path element is a number).
Correct: create a sequence when the next element is numeric, or refuse with an error. config set never writes a file the loader rejects.
4. snapshot restore ignores requested paths that match nothing, and exits 0
What: snapshot restore ID TARGET /src1/a /src1/typo restores /src1/a, says nothing about /src1/typo, and exits 0. When no path matches (a typo, or a relative path such as src1/a), it prints "Warning: No files found to restore." and exits 0 with nothing restored.
Where: internal/vaultik/restore.goRestore (returns nil on an empty file list; nothing checks each requested path).
Correct: a requested path that matches nothing in the snapshot is an error naming that path, with a non-zero exit.
5. snapshot create --skip-errors exits 0 and prints "Finished successfully." after skipping files
What: with one unreadable file and one unreadable directory, the run prints the skip errors, "Files: 140 examined, 4 backed up, 135 unchanged, 1 failed", then "Finished successfully.", exit 0 (also under --cron). The skipped directory is not counted as failed. A script or cron wrapper cannot tell a complete backup from one missing files, while snapshot restore under the same flag exits non-zero when files failed (#218).
Where: internal/vaultik/snapshot.go (totalFilesFailed feeds the summary only; the run returns success).
Correct: exit non-zero when any file or directory was skipped, as restore does, and do not print "Finished successfully.".
6. snapshot list shows NEW CHUNK SIZE 0 B for a snapshot whose change is a modified file
What: the local index keeps one file row per path, shared by every snapshot and rewritten in place when the file changes, so earlier snapshots appear to reference the new chunks. New data from changed files is never counted, and older snapshots' figures change after later backups.
Shown: 50 bytes changed in a 5 MB file, then a backup: the run uploaded a 1.3 MB blob, and snapshot list shows that snapshot's NEW CHUNK SIZE as 0 B.
Correct: count the chunks this snapshot references that no earlier snapshot referenced.
7. snapshot list and the snapshot purge listing print 1024-based sizes labelled "MB"
What: the same 6,329,198 bytes show as "6.0 MB" in snapshot list, and as "6.3 MB" in the snapshot create summary, snapshot verify and remote info.
Where: internal/vaultik/helpers.goformatBytes (divides by 1024, prints "KB"/"MB"), used by snapshot_list.go and the purge listing in snapshot.go. Everything else uses humanize.Bytes (powers of 1000).
Correct: one unit system across commands.
8. snapshot verify --deep --json reports "missing": 0 and "verified": 0 when it fails
What: deep verify sets verified only on success and never sets missing. With one of two blobs deleted, --deep --json gives "verified": 0, "missing": 0 (shallow --json gives "verified": 1, "missing": 1). With bytes flipped in one of two blobs it gives "verified": 0, although the text run reports "Verified 1/2 blobs" before failing.
Where: internal/vaultik/verify.goRunDeepVerify.
Correct: the counts say what was checked and what was missing, as in shallow mode.
9. config edit fails when $EDITOR carries arguments
What: EDITOR="cat -n" vaultik config edit fails with exec: "cat -n": executable file not found in $PATH, exit 1. Common settings such as code --wait or emacsclient -t fail the same way.
Where: internal/cli/config.gonewConfigEditCommand (the whole of $EDITOR is the program name).
Correct: split $EDITOR into words (or run it through sh -c), as git and crontab do.
10. Every backup reports files deleted long ago as "deleted" again
What: a file deleted from disk keeps its index row while any older snapshot lists it, and the deleted figure counts every indexed file this walk did not see. After two files were removed, the next backup correctly reported "2 deleted (202 kB)"; every later unchanged backup of that snapshot reports "2 deleted (202 kB)" again, and so does the first backup of another snapshot name over the same path.
Where: internal/snapshot/scanner.godetectDeletedFilesFromMap. Same function as part 2's item 6 but a different trigger, which a fix for excluded files does not cover.
Correct: count a file as deleted only if the previous snapshot of this name had it.
11. Every restore by a non-root user warns that ownership was not preserved, even when it was
What: "Restore did not preserve file ownership: chown(2) requires root. Re-run as root ..." is printed on every non-root restore, although every recorded uid/gid (1000:1000 here) equals the restoring user's, so ownership is exactly as backed up.
Where: internal/vaultik/restore.gofinishRestore (checks only os.Geteuid() != 0).
Correct: warn only when some restored entry's recorded owner differs from the restoring user.
12. remote nuke tells the user to run prune for the blobs it is about to delete
What: a successful remote nuke --force prints "Note: Removed snapshots' blobs remain on the remote. Run 'vaultik prune' to delete any blobs ...", then deletes every blob itself and says the store is empty.
Where: internal/vaultik/snapshot.goexecuteRemoveAll. Next to #290, which covers the advice on the failure path.
Correct: no prune advice in nuke's output.
13. After storage_url changes, snapshot list advises prune, which then refuses
What: with the index bound to the old destination, snapshot list reports its snapshots as "local snapshot record(s) not found in backup destination store" and says "Run 'vaultik prune' to remove stale local records." prune, snapshot purge and snapshot remove then refuse because the index is bound to another destination.
Where: internal/vaultik/snapshot_list.goreportListDrift (does not check the binding).
Correct: when the index is bound to another destination, say so, with the same choices the other commands give (revert storage_url, or database delete).
Also reproduced, already reported
Part 2's items 4 (blob temp file left in $TMPDIR, also after an ENOSPC on $TMPDIR), 7 and 8; the chunk_size: 1MB rejection from parts 2 and 4 (config.example.yml also lists 500KB under "Supports:"); and #293.
Model: opus-5-5
## Part 1: Use the binary as a user would
`make build` on `next` at `fa56165`, `file://` destinations, own config, index, lock and `$TMPDIR`. Every item was shown with real runs of `./vaultik`; no test was written.
### 1. A destination removed part-way through a backup is recreated, and the backup reports success with a blob missing
- What: if the destination directory disappears after some blobs are uploaded, the next upload recreates it. The run prints "Finished successfully." and exits 0, but a blob it uploaded earlier is gone, so the snapshot cannot be restored. The local index still records that blob as stored, so every later backup also exits 0 ("0 backed up, 53 unchanged") and is just as unrestorable. Only `snapshot verify` notices.
- Where: `internal/storage/file.go` `writeAtomic` (`MkdirAll` for the blob's directory also recreates the destination root).
- Shown: 9 MB source packed into 2 blobs. The backup was paused (SIGSTOP) once the first blob landed, the destination removed, the backup resumed: exit 0, destination holds 1 of 2 blobs. `snapshot verify` says "1 blobs are missing" (exit 1). The next `snapshot create` exits 0; restoring its snapshot fails with "downloading blob d750...: object not found".
- Correct: a write never recreates a missing `file://` destination root, so the run fails and the next run discards the incomplete snapshot's blobs and uploads them again. This is the mid-run case, not #239's start-of-run case, but the owner's answer there (a missing `file://` destination is an error) covers it if enforced on every write.
### 2. An invalid `--keep-newer-than` is rejected only after the backup, which then reports "Snapshot creation failed" although the snapshot was made
- What: the duration is first parsed by the post-backup prune. With a value the parser rejects (`1.5d`, `0`), the backup completes and uploads its snapshot, then the command prints "Snapshot creation failed: post-backup prune: purging old snapshots: invalid duration" and exits 1. Retention never runs, so such a cron line adds a snapshot every night, reports each as failed, and never deletes any.
- Where: `internal/vaultik/snapshot.go` `runPostBackupPrune` (gets the raw string); `internal/cli/snapshot.go` does not check `--keep-newer-than`.
- Shown: `snapshot create ex --prune --keep-newer-than 1.5d` exits 1 with that message; `snapshot list --json` then shows the new snapshot `testhost_ex_2026-10-12T22:33:14Z` with `remote_present: true`.
- Correct: reject the duration before the backup starts (non-zero exit, nothing created). A failure in the post-backup prune is reported as a prune failure, not as a failed snapshot.
### 3. `config set` with a list index on a key that does not exist writes a map, and then no command can load the config
- What: `vaultik config set age_recipients.0 age1...` (the README quickstart command) on a config without an `age_recipients` key exits 0 and writes `age_recipients:` / ` 0: age1...`. Every later command fails with "failed to parse config: yaml: unmarshal errors: cannot unmarshal !!map into []string". Same with `config set snapshots.NAME.paths.0 /path`. An index into an existing list works.
- Where: `internal/cli/config.go` `yamlSetInMapping` (creates a mapping for any missing key, even when the next path element is a number).
- Correct: create a sequence when the next element is numeric, or refuse with an error. `config set` never writes a file the loader rejects.
### 4. `snapshot restore` ignores requested paths that match nothing, and exits 0
- What: `snapshot restore ID TARGET /src1/a /src1/typo` restores `/src1/a`, says nothing about `/src1/typo`, and exits 0. When no path matches (a typo, or a relative path such as `src1/a`), it prints "Warning: No files found to restore." and exits 0 with nothing restored.
- Where: `internal/vaultik/restore.go` `Restore` (returns nil on an empty file list; nothing checks each requested path).
- Correct: a requested path that matches nothing in the snapshot is an error naming that path, with a non-zero exit.
### 5. `snapshot create --skip-errors` exits 0 and prints "Finished successfully." after skipping files
- What: with one unreadable file and one unreadable directory, the run prints the skip errors, "Files: 140 examined, 4 backed up, 135 unchanged, 1 failed", then "Finished successfully.", exit 0 (also under `--cron`). The skipped directory is not counted as failed. A script or cron wrapper cannot tell a complete backup from one missing files, while `snapshot restore` under the same flag exits non-zero when files failed (#218).
- Where: `internal/vaultik/snapshot.go` (`totalFilesFailed` feeds the summary only; the run returns success).
- Correct: exit non-zero when any file or directory was skipped, as restore does, and do not print "Finished successfully.".
### 6. `snapshot list` shows NEW CHUNK SIZE 0 B for a snapshot whose change is a modified file
- What: the local index keeps one file row per path, shared by every snapshot and rewritten in place when the file changes, so earlier snapshots appear to reference the new chunks. New data from changed files is never counted, and older snapshots' figures change after later backups.
- Where: `internal/database/snapshots.go` `GetSnapshotNewChunkSize`.
- Shown: 50 bytes changed in a 5 MB file, then a backup: the run uploaded a 1.3 MB blob, and `snapshot list` shows that snapshot's NEW CHUNK SIZE as `0 B`.
- Correct: count the chunks this snapshot references that no earlier snapshot referenced.
### 7. `snapshot list` and the `snapshot purge` listing print 1024-based sizes labelled "MB"
- What: the same 6,329,198 bytes show as "6.0 MB" in `snapshot list`, and as "6.3 MB" in the `snapshot create` summary, `snapshot verify` and `remote info`.
- Where: `internal/vaultik/helpers.go` `formatBytes` (divides by 1024, prints "KB"/"MB"), used by `snapshot_list.go` and the purge listing in `snapshot.go`. Everything else uses `humanize.Bytes` (powers of 1000).
- Correct: one unit system across commands.
### 8. `snapshot verify --deep --json` reports `"missing": 0` and `"verified": 0` when it fails
- What: deep verify sets `verified` only on success and never sets `missing`. With one of two blobs deleted, `--deep --json` gives `"verified": 0, "missing": 0` (shallow `--json` gives `"verified": 1, "missing": 1`). With bytes flipped in one of two blobs it gives `"verified": 0`, although the text run reports "Verified 1/2 blobs" before failing.
- Where: `internal/vaultik/verify.go` `RunDeepVerify`.
- Correct: the counts say what was checked and what was missing, as in shallow mode.
### 9. `config edit` fails when `$EDITOR` carries arguments
- What: `EDITOR="cat -n" vaultik config edit` fails with `exec: "cat -n": executable file not found in $PATH`, exit 1. Common settings such as `code --wait` or `emacsclient -t` fail the same way.
- Where: `internal/cli/config.go` `newConfigEditCommand` (the whole of `$EDITOR` is the program name).
- Correct: split `$EDITOR` into words (or run it through `sh -c`), as git and crontab do.
### 10. Every backup reports files deleted long ago as "deleted" again
- What: a file deleted from disk keeps its index row while any older snapshot lists it, and the deleted figure counts every indexed file this walk did not see. After two files were removed, the next backup correctly reported "2 deleted (202 kB)"; every later unchanged backup of that snapshot reports "2 deleted (202 kB)" again, and so does the first backup of another snapshot name over the same path.
- Where: `internal/snapshot/scanner.go` `detectDeletedFilesFromMap`. Same function as part 2's item 6 but a different trigger, which a fix for excluded files does not cover.
- Correct: count a file as deleted only if the previous snapshot of this name had it.
### 11. Every restore by a non-root user warns that ownership was not preserved, even when it was
- What: "Restore did not preserve file ownership: chown(2) requires root. Re-run as root ..." is printed on every non-root restore, although every recorded uid/gid (1000:1000 here) equals the restoring user's, so ownership is exactly as backed up.
- Where: `internal/vaultik/restore.go` `finishRestore` (checks only `os.Geteuid() != 0`).
- Correct: warn only when some restored entry's recorded owner differs from the restoring user.
### 12. `remote nuke` tells the user to run `prune` for the blobs it is about to delete
- What: a successful `remote nuke --force` prints "Note: Removed snapshots' blobs remain on the remote. Run 'vaultik prune' to delete any blobs ...", then deletes every blob itself and says the store is empty.
- Where: `internal/vaultik/snapshot.go` `executeRemoveAll`. Next to #290, which covers the advice on the failure path.
- Correct: no prune advice in nuke's output.
### 13. After `storage_url` changes, `snapshot list` advises `prune`, which then refuses
- What: with the index bound to the old destination, `snapshot list` reports its snapshots as "local snapshot record(s) not found in backup destination store" and says "Run 'vaultik prune' to remove stale local records." `prune`, `snapshot purge` and `snapshot remove` then refuse because the index is bound to another destination.
- Where: `internal/vaultik/snapshot_list.go` `reportListDrift` (does not check the binding).
- Correct: when the index is bound to another destination, say so, with the same choices the other commands give (revert `storage_url`, or `database delete`).
### Also reproduced, already reported
Part 2's items 4 (blob temp file left in `$TMPDIR`, also after an ENOSPC on `$TMPDIR`), 7 and 8; the `chunk_size: 1MB` rejection from parts 2 and 4 (`config.example.yml` also lists `500KB` under "Supports:"); and #293.
Model: opus-5-5
Part 5: Read the storage backends, config and command line
Each item below was confirmed on next at fa56165, with make build's ./vaultik and a throwaway config, index and destination in a temp dir, unless a test is named.
1. A symlinked metadata or blobs directory in a file:// destination lists as empty; prune then deletes every blob
What is wrong:FileStorer.List and ListStream (internal/storage/file.go) walk with afero.Walk, which does not follow a symlink, so a symlinked metadata/ or blobs/ is listed as one non-directory key. Stat, Get and Put do follow it, so backups and restores keep working while every listing is empty.
Shown: after a backup, mv dest/metadata elsewhere/ && ln -s elsewhere/metadata dest/metadata. snapshot list then warns "1 local snapshot record(s) not found in backup destination store". prune --force prints "Removing stale local record …" and "Deleted 1 blob(s)", leaving 0 blobs while both metadata files are still in place behind the link. With blobs/ symlinked instead, remote info --json gives total_blob_count: 0 beside referenced_blob_count: 1, and remote nuke --force prints "Backup destination store is now empty." with the blob still there.
Correct: a listing reads through a symlinked directory the way Stat and Get do, or fails. It never reports a store it could not read as empty.
2. A relative file:// path is accepted, so the destination changes with the working directory; a backup exits 0 and cannot be restored
What is wrong:ParseStorageURL (internal/storage/url.go) takes everything after file:// verbatim. storage_url: "file://dest" is accepted and resolved against $PWD. The local index's destination binding (internal/vaultik/storage_bind.go) compares the raw string, so it never notices the destination moved. The same verbatim handling skips percent-decoding: file:///…/my%20stick/backup creates a new directory literally named my%20stick beside my stick. parseRcloneURL strips the leading /, so rclone://loc/srv/x on a local-type remote writes under $PWD/srv/x.
Shown: with storage_url: "file://dest", snapshot create run from A/, a file changed, then snapshot create run from B/: both exit 0. The unchanged file's blob exists only under A/dest, so from B/, snapshot verify fails with "1 blobs are missing" and snapshot restore fails with "object not found".
Correct: the destination does not depend on the working directory. A file:// URL whose path is not absolute is rejected, percent-escapes are decoded, and rclone://remote/abs/path keeps its leading / (or a relative path on a local remote is refused).
3. An S3 listing that the server marks truncated without a continuation token is taken as complete
What is wrong:Client.ListObjects and ListObjectsStream (internal/s3/client.go) use the SDK paginator, which ends silently when a page has IsTruncated=true but no NextContinuationToken. A server that answers V2 listings V1-style does that. prune then counts only the manifests on the first page, and treats the blobs of the remaining snapshots as unreferenced (collectReferencedBlobs, internal/vaultik/prune.go).
Shown: throwaway test TestHuntS3ListingCutShortIsNotAnError in internal/storage puts gofakes3 behind a handler that pages by 2 and removes NextContinuationToken, then stores 5 keys. Under make test it fails: "List: server marked the listing truncated, got 2 of 5 keys and no error", and the same for ListStream.
Correct: a truncated page without a token is an error, so prune deletes nothing.
4. snapshot remove of an ID that does not exist reports success and exits 0
What is wrong:RemoveSnapshot and removeSnapshotRemote (internal/vaultik/snapshot.go) hash whatever ID is given and never check that it matches anything. A mistyped ID, or the remote key that snapshot list prints for a remote-only snapshot (the only name a new host has, and one verify and restore accept), removes nothing.
Shown: on a fresh index, snapshot remove --force KEY, with KEY the remote_key from snapshot list --json, prints "Removed snapshot '…' from local database" and "Removed snapshot metadata from remote storage", then exits 0; snapshot list still shows the snapshot. snapshot remove --force nosuchid gives the same.
Correct: an ID matching neither a local record nor a remote key fails with "not found" and exit 1. A remote key (or an unambiguous leading part) is accepted as it is by verify and restore.
5. An invalid --keep-newer-than is checked only after the backup, which is then reported as failed; without --prune it is ignored
What is wrong:snapshot create (internal/cli/snapshot.go) does not check --keep-newer-than. It is parsed in the post-backup prune.
Shown:snapshot create --cron --prune --keep-newer-than 4x uploads a new snapshot (the count goes 5 to 6), then prints "Snapshot creation failed: post-backup prune: purging old snapshots: invalid duration: unknown time unit "x"" and exits 1. Retention never runs. snapshot create --keep-newer-than garbage without --prune backs up and exits 0.
Correct: the value (and its use without --prune) is rejected before the backup starts. A prune failure after a good backup does not say the snapshot creation failed.
6. Shallow snapshot verify reports any error reading a blob's size as "Missing", and drops the error
What is wrong:verifyManifestBlobs and the db.zst.age check (internal/vaultik/snapshot.go) count every Stat error as missing: permission denied, an S3 403, a network error.
Shown:chmod 000 on a blob's shard directory (the blob is present), then snapshot verify: "Missing: ecb65bb9… (219 B)", "FAILED - 1 blobs are missing", exit 1. The real error is printed nowhere.
Correct: only ErrNotFound counts as missing. Any other error is reported as "could not check", with its cause.
7. --cron is not silent after a successful --prune, and prune, snapshot purge, snapshot remove and remote nuke ignore --quiet
What is wrong:PruneBlobs (internal/vaultik/prune.go), and CleanupLocalSnapshots, RemoveSnapshot, RemoveAllSnapshots and confirmAndExecutePurge (internal/vaultik/snapshot.go), write through v.stdoutf/printlnStdout with no quiet check. The README says --cron is "Silent on total success" and --quiet "Suppress[es] non-error output".
Shown: the README's own cron line, snapshot create --cron --prune --keep-newer-than 4w, exits 0 and prints "No unreferenced blobs to remove.", or "Found 3 unreferenced blob(s) totaling 681 B … Deleted 3 blob(s)", on every run. -q prune --force prints "No stale local snapshots found." and "No unreferenced blobs to remove."; -q snapshot purge --keep-latest --force, -q snapshot remove --force ID (and --dry-run) and -q remote nuke --force print their full reports.
Correct: nothing on stdout on success under --cron or --quiet. Warnings and errors are still printed.
8. A mistyped subcommand under snapshot, config, remote, database or completion exits 0 and does nothing
What is wrong: the group commands in internal/cli have no Args/RunE, so cobra prints the group's help on stdout and returns success. vaultik bogus at the top level exits 1.
Shown:vaultik snapshot craete src prints the snapshot help and exits 0, as do vaultik config inti and vaultik completion nosuchshell. In crontab, such a line makes no backup and reports no error.
Correct: an unknown subcommand is an error, exit 1, as at the top level.
9. Byte sizes: the defaults and the chunk-size minimum are binary units, while the docs, the parser and the display are decimal
What is wrong:defaultChunkSize, defaultBlobSizeLimit and minChunkSize (internal/config/config.go) are 10 MiB, 10 GiB and 1 MiB. Size parses 10MB as 10,000,000 bytes (humanize.ParseBytes), and info prints decimal units.
Shown:chunk_size: 1MB is rejected with "chunk_size must be at least 1MB". The config init template lists 1MB and 64KB as accepted values for chunk_size; both are rejected. blob_size_limit: 40MB, four times the documented 10MB default chunk size (the README's stated minimum), is rejected. With no size settings, info shows "Blob Size Limit: 11 GB", although the documented default is 10GB. Writing that documented default out (blob_size_limit: 10GB) changes it: info then shows "10 GB".
Correct: the documented defaults, minimums and template examples are what the code accepts and uses: one unit convention across the defaults, the parser, the messages and the docs.
10. A config validation error that contains "): " loses its start, including which setting is wrong
What is wrong:cleanStartupError (internal/cli/app.go) cuts each startup error at its last "): ", meaning to strip fx's annotations. The blob_size_limit message itself contains "(… size spread): ".
Shown: with blob_size_limit: 40MB, info, snapshot create and snapshot list print only "ERROR: need at least 41943040 bytes" (exit 1). database delete, which loads the config without fx, prints the whole message: "failed to load config: invalid config: blob_size_limit must be at least … : need at least 41943040 bytes".
Correct: only fx's annotations are removed; the underlying message is shown whole.
11. RCLONE_CONFIG is ignored
What is wrong:NewRcloneStorer (internal/storage/rclone.go) calls configfile.Install() but never applies RCLONE_CONFIG. In rclone, only its CLI flag layer reads that variable.
Shown: with RCLONE_CONFIG pointing at a file that defines [loc] type = local, snapshot create fails with "rclone remote not found in config: loc". It succeeds once the same file is copied to $XDG_CONFIG_HOME/rclone/rclone.conf.
Correct: vaultik reads the rclone config rclone itself would read, RCLONE_CONFIG included (README: "Requires rclone to be configured separately").
12. Credentials can still be carried in storage_url, and are then printed
What is wrong: the #166 check in parseS3URL/parseRcloneURL (internal/storage/url.go) rejects only the outer URL's userinfo and unknown query parameters. Userinfo inside the endpoint= value, and rclone connection-string parameters in the remote name (remote,param=value), pass.
Shown:storage_url: "s3://bkt/p?endpoint=https://AKIAUSERINFO:SECRETUSERINFO@127.0.0.1:1" is accepted. info prints it in "Location:" and "Storage URL:", and remote info in "Location:". rclone://loc,description=SECRETPARAMVALUE//path is accepted and passed to rclone. The parameter appears in info and in snapshot create's upload lines.
Correct: both forms are rejected without echoing the value, as the outer userinfo is.
13. remote nuke tells the user the blobs remain and to run prune, then deletes them
What is wrong:NukeRemote (internal/vaultik/prune.go) reuses RemoveAllSnapshots (internal/vaultik/snapshot.go), which prints the remove follow-up note.
Shown:remote nuke --force prints "Note: Removed snapshots' blobs remain on the remote. Run 'vaultik prune' to delete any blobs …", then "Deleted 1 blob(s)".
Correct:remote nuke does not print advice that its own next step makes false.
Model: opus-5-5
## Part 5: Read the storage backends, config and command line
Each item below was confirmed on `next` at `fa56165`, with `make build`'s `./vaultik` and a throwaway config, index and destination in a temp dir, unless a test is named.
### 1. A symlinked `metadata` or `blobs` directory in a `file://` destination lists as empty; `prune` then deletes every blob
- **What is wrong:** `FileStorer.List` and `ListStream` (`internal/storage/file.go`) walk with `afero.Walk`, which does not follow a symlink, so a symlinked `metadata/` or `blobs/` is listed as one non-directory key. `Stat`, `Get` and `Put` do follow it, so backups and restores keep working while every listing is empty.
- **Shown:** after a backup, `mv dest/metadata elsewhere/ && ln -s elsewhere/metadata dest/metadata`. `snapshot list` then warns "1 local snapshot record(s) not found in backup destination store". `prune --force` prints "Removing stale local record …" and "Deleted 1 blob(s)", leaving 0 blobs while both metadata files are still in place behind the link. With `blobs/` symlinked instead, `remote info --json` gives `total_blob_count: 0` beside `referenced_blob_count: 1`, and `remote nuke --force` prints "Backup destination store is now empty." with the blob still there.
- **Correct:** a listing reads through a symlinked directory the way `Stat` and `Get` do, or fails. It never reports a store it could not read as empty.
### 2. A relative `file://` path is accepted, so the destination changes with the working directory; a backup exits 0 and cannot be restored
- **What is wrong:** `ParseStorageURL` (`internal/storage/url.go`) takes everything after `file://` verbatim. `storage_url: "file://dest"` is accepted and resolved against `$PWD`. The local index's destination binding (`internal/vaultik/storage_bind.go`) compares the raw string, so it never notices the destination moved. The same verbatim handling skips percent-decoding: `file:///…/my%20stick/backup` creates a new directory literally named `my%20stick` beside `my stick`. `parseRcloneURL` strips the leading `/`, so `rclone://loc/srv/x` on a local-type remote writes under `$PWD/srv/x`.
- **Shown:** with `storage_url: "file://dest"`, `snapshot create` run from `A/`, a file changed, then `snapshot create` run from `B/`: both exit 0. The unchanged file's blob exists only under `A/dest`, so from `B/`, `snapshot verify` fails with "1 blobs are missing" and `snapshot restore` fails with "object not found".
- **Correct:** the destination does not depend on the working directory. A `file://` URL whose path is not absolute is rejected, percent-escapes are decoded, and `rclone://remote/abs/path` keeps its leading `/` (or a relative path on a local remote is refused).
### 3. An S3 listing that the server marks truncated without a continuation token is taken as complete
- **What is wrong:** `Client.ListObjects` and `ListObjectsStream` (`internal/s3/client.go`) use the SDK paginator, which ends silently when a page has `IsTruncated=true` but no `NextContinuationToken`. A server that answers V2 listings V1-style does that. `prune` then counts only the manifests on the first page, and treats the blobs of the remaining snapshots as unreferenced (`collectReferencedBlobs`, `internal/vaultik/prune.go`).
- **Shown:** throwaway test `TestHuntS3ListingCutShortIsNotAnError` in `internal/storage` puts gofakes3 behind a handler that pages by 2 and removes `NextContinuationToken`, then stores 5 keys. Under `make test` it fails: "List: server marked the listing truncated, got 2 of 5 keys and no error", and the same for `ListStream`.
- **Correct:** a truncated page without a token is an error, so `prune` deletes nothing.
### 4. `snapshot remove` of an ID that does not exist reports success and exits 0
- **What is wrong:** `RemoveSnapshot` and `removeSnapshotRemote` (`internal/vaultik/snapshot.go`) hash whatever ID is given and never check that it matches anything. A mistyped ID, or the remote key that `snapshot list` prints for a remote-only snapshot (the only name a new host has, and one `verify` and `restore` accept), removes nothing.
- **Shown:** on a fresh index, `snapshot remove --force KEY`, with `KEY` the `remote_key` from `snapshot list --json`, prints "Removed snapshot '…' from local database" and "Removed snapshot metadata from remote storage", then exits 0; `snapshot list` still shows the snapshot. `snapshot remove --force nosuchid` gives the same.
- **Correct:** an ID matching neither a local record nor a remote key fails with "not found" and exit 1. A remote key (or an unambiguous leading part) is accepted as it is by `verify` and `restore`.
### 5. An invalid `--keep-newer-than` is checked only after the backup, which is then reported as failed; without `--prune` it is ignored
- **What is wrong:** `snapshot create` (`internal/cli/snapshot.go`) does not check `--keep-newer-than`. It is parsed in the post-backup prune.
- **Shown:** `snapshot create --cron --prune --keep-newer-than 4x` uploads a new snapshot (the count goes 5 to 6), then prints "Snapshot creation failed: post-backup prune: purging old snapshots: invalid duration: unknown time unit "x"" and exits 1. Retention never runs. `snapshot create --keep-newer-than garbage` without `--prune` backs up and exits 0.
- **Correct:** the value (and its use without `--prune`) is rejected before the backup starts. A prune failure after a good backup does not say the snapshot creation failed.
### 6. Shallow `snapshot verify` reports any error reading a blob's size as "Missing", and drops the error
- **What is wrong:** `verifyManifestBlobs` and the `db.zst.age` check (`internal/vaultik/snapshot.go`) count every `Stat` error as missing: permission denied, an S3 403, a network error.
- **Shown:** `chmod 000` on a blob's shard directory (the blob is present), then `snapshot verify`: "Missing: ecb65bb9… (219 B)", "FAILED - 1 blobs are missing", exit 1. The real error is printed nowhere.
- **Correct:** only `ErrNotFound` counts as missing. Any other error is reported as "could not check", with its cause.
### 7. `--cron` is not silent after a successful `--prune`, and `prune`, `snapshot purge`, `snapshot remove` and `remote nuke` ignore `--quiet`
- **What is wrong:** `PruneBlobs` (`internal/vaultik/prune.go`), and `CleanupLocalSnapshots`, `RemoveSnapshot`, `RemoveAllSnapshots` and `confirmAndExecutePurge` (`internal/vaultik/snapshot.go`), write through `v.stdoutf`/`printlnStdout` with no quiet check. The README says `--cron` is "Silent on total success" and `--quiet` "Suppress[es] non-error output".
- **Shown:** the README's own cron line, `snapshot create --cron --prune --keep-newer-than 4w`, exits 0 and prints "No unreferenced blobs to remove.", or "Found 3 unreferenced blob(s) totaling 681 B … Deleted 3 blob(s)", on every run. `-q prune --force` prints "No stale local snapshots found." and "No unreferenced blobs to remove."; `-q snapshot purge --keep-latest --force`, `-q snapshot remove --force ID` (and `--dry-run`) and `-q remote nuke --force` print their full reports.
- **Correct:** nothing on stdout on success under `--cron` or `--quiet`. Warnings and errors are still printed.
### 8. A mistyped subcommand under `snapshot`, `config`, `remote`, `database` or `completion` exits 0 and does nothing
- **What is wrong:** the group commands in `internal/cli` have no `Args`/`RunE`, so cobra prints the group's help on stdout and returns success. `vaultik bogus` at the top level exits 1.
- **Shown:** `vaultik snapshot craete src` prints the `snapshot` help and exits 0, as do `vaultik config inti` and `vaultik completion nosuchshell`. In crontab, such a line makes no backup and reports no error.
- **Correct:** an unknown subcommand is an error, exit 1, as at the top level.
### 9. Byte sizes: the defaults and the chunk-size minimum are binary units, while the docs, the parser and the display are decimal
- **What is wrong:** `defaultChunkSize`, `defaultBlobSizeLimit` and `minChunkSize` (`internal/config/config.go`) are 10 MiB, 10 GiB and 1 MiB. `Size` parses `10MB` as 10,000,000 bytes (`humanize.ParseBytes`), and `info` prints decimal units.
- **Shown:** `chunk_size: 1MB` is rejected with "chunk_size must be at least 1MB". The `config init` template lists `1MB` and `64KB` as accepted values for `chunk_size`; both are rejected. `blob_size_limit: 40MB`, four times the documented `10MB` default chunk size (the README's stated minimum), is rejected. With no size settings, `info` shows "Blob Size Limit: 11 GB", although the documented default is `10GB`. Writing that documented default out (`blob_size_limit: 10GB`) changes it: `info` then shows "10 GB".
- **Correct:** the documented defaults, minimums and template examples are what the code accepts and uses: one unit convention across the defaults, the parser, the messages and the docs.
### 10. A config validation error that contains "): " loses its start, including which setting is wrong
- **What is wrong:** `cleanStartupError` (`internal/cli/app.go`) cuts each startup error at its last "): ", meaning to strip fx's annotations. The `blob_size_limit` message itself contains "(… size spread): ".
- **Shown:** with `blob_size_limit: 40MB`, `info`, `snapshot create` and `snapshot list` print only "ERROR: need at least 41943040 bytes" (exit 1). `database delete`, which loads the config without fx, prints the whole message: "failed to load config: invalid config: blob_size_limit must be at least … : need at least 41943040 bytes".
- **Correct:** only fx's annotations are removed; the underlying message is shown whole.
### 11. `RCLONE_CONFIG` is ignored
- **What is wrong:** `NewRcloneStorer` (`internal/storage/rclone.go`) calls `configfile.Install()` but never applies `RCLONE_CONFIG`. In rclone, only its CLI flag layer reads that variable.
- **Shown:** with `RCLONE_CONFIG` pointing at a file that defines `[loc] type = local`, `snapshot create` fails with "rclone remote not found in config: loc". It succeeds once the same file is copied to `$XDG_CONFIG_HOME/rclone/rclone.conf`.
- **Correct:** vaultik reads the rclone config `rclone` itself would read, `RCLONE_CONFIG` included (README: "Requires rclone to be configured separately").
### 12. Credentials can still be carried in `storage_url`, and are then printed
- **What is wrong:** the #166 check in `parseS3URL`/`parseRcloneURL` (`internal/storage/url.go`) rejects only the outer URL's userinfo and unknown query parameters. Userinfo inside the `endpoint=` value, and rclone connection-string parameters in the remote name (`remote,param=value`), pass.
- **Shown:** `storage_url: "s3://bkt/p?endpoint=https://AKIAUSERINFO:SECRETUSERINFO@127.0.0.1:1"` is accepted. `info` prints it in "Location:" and "Storage URL:", and `remote info` in "Location:". `rclone://loc,description=SECRETPARAMVALUE//path` is accepted and passed to rclone. The parameter appears in `info` and in `snapshot create`'s upload lines.
- **Correct:** both forms are rejected without echoing the value, as the outer userinfo is.
### 13. `remote nuke` tells the user the blobs remain and to run `prune`, then deletes them
- **What is wrong:** `NukeRemote` (`internal/vaultik/prune.go`) reuses `RemoveAllSnapshots` (`internal/vaultik/snapshot.go`), which prints the remove follow-up note.
- **Shown:** `remote nuke --force` prints "Note: Removed snapshots' blobs remain on the remote. Run 'vaultik prune' to delete any blobs …", then "Deleted 1 blob(s)".
- **Correct:** `remote nuke` does not print advice that its own next step makes false.
Model: opus-5-5
Six defects confirmed on next at fa56165, with real runs of ./vaultik (make build, file:// destination, own config and index in a temp dir) and two throwaway tests that fail under make test (not committed; essential lines below).
1. --skip-errors --verify counts a file that failed to restore again as a checksum failure
verifyRestoredFiles (internal/vaultik/restore.go) verifies every selected file, including those handleRestoreFileError already recorded, and adds them to the same FilesFailed/FailedFiles; handleRestoreVerification then calls them all checksum mismatches.
Shown: 12 files in 4 blobs, the blob holding p8.bin removed from the store, vaultik --skip-errors snapshot restore --verify $SID r2:
Warning: 1 file(s) failed to restore:
.../src/pics/p8.bin
Verifying 11 files (12 MB).
ERROR: Verification failed: 2 files did not match expected checksums.
.../src/pics/p8.bin
.../src/pics/p8.bin
ERROR: Restore failed: 2 files failed verification
No restored file mismatched. Correct: verify only the files that were written; report 1 failed to restore and 0 failed verification, each path once.
2. A restore path argument that selects nothing is not reported, and the command exits 0
getFilesToRestore drops a path that matches nothing; when none match, Restore warns No files found to restore. and returns nil (internal/vaultik/restore.go). Each of these exits 0:
snapshot restore $SID r3 $W/src/docs/typo.txt: No files found to restore.
a relative path (srv/code/.../src/docs): No files found to restore.
a configured path that is a symlink (paths: [$W/srclink], srclink pointing to src): entries are stored under the resolved $W/src, so snapshot restore $SID r9 $W/srclink/docs gives No files found to restore.
Correct: a path argument that matches nothing in the snapshot is an error naming it, with a non-zero exit.
3. A restore with path arguments keeps blobs in the temporary cache until it ends
restoreSweeper.blobStillNeeded (internal/vaultik/restore_sweeper.go) keeps a cached blob while any file in the snapshot database that uses one of its chunks is not in the restored set. Files outside the selection (and files that failed under --skip-errors) never enter that set, so such a blob is never freed. Each entry is a whole decrypted blob in $TMPDIR (up to blob_size_limit, 10 GB by default), so the temporary space grows with the number of blobs touched instead of staying near one blob.
Shown: throwaway test on the TestRestoreLocalityAndReadAt fixture (3 blobs), restoring by path every src-*.bin except src-005.bin and src-015.bin:
require.NoError(t,v.Restore(&RestoreOptions{SnapshotID:snapshotID,TargetDir:restoreDir,Paths:paths}))assert.LessOrEqual(t,cacheRef.PeakLen(),1)// "3" is not less than or equal to "1"
The same fixture restored without paths peaks at 1. Correct: free a blob once no file still pending in this restore needs it.
4. Interrupting a restore during a stalled blob read leaves the decrypted snapshot database and decrypted file data in $TMPDIR
downloadBlobToCache copies the blob with io.Copy in blobDiskCache.PutFromReader (internal/vaultik/restore.go, blobcache.go). The context is checked only between blobs, and the file:// backend's Get returns an os.File that ignores it. When one blob read outlasts shutdownTimeout (30 s), the process exits without the restore's cleanup. #159 required that "the blob download into the cache stops on cancel"; its fix checks only between blobs. With the default 10 GB blob_size_limit on a USB stick or a slow NAS, one blob takes minutes.
Shown: one blob on the file:// destination replaced by a FIFO that delivers the first 1 MB and then stalls (standing in for a slow device); snapshot restore; SIGINT after 4 s. vaultik exits 130 after 30 s, logging Error during shutdown: context deadline exceeded, and leaves $TMPDIR/vaultik-restore-*/snapshot.db (plaintext SQLite) and $TMPDIR/vaultik-blobcache-*/HASH (917504 bytes of decrypted file content). The warning that decrypted files may remain is not printed.
Correct: cancelling stops the blob stream, so the cleanup runs before exit.
5. Under --skip-errors, one file whose chunk is in no blob aborts the whole restore before anything is restored
newRestorePlan (internal/vaultik/restore_plan.go) returns errPlanChunkMissing (or errPlanBlobIDMissing) for the first such file and restoreAllFiles returns it; it never reaches handleRestoreFileError. Earlier backup defects such as #214 left exactly such snapshots, and --skip-errors is how a user gets the rest back.
Shown: throwaway test: 20 single-chunk files in two blobs, one blob's snapshot_blobs and uploads rows deleted from the local index, metadata exported again, restore with SkipErrors: true:
restore error: building restore plan: planning .../src/file-00.bin: chunk missing from blob map: b70c709106b3e1b6
"0" is not positive (no file restored)
Correct: under --skip-errors the files that need such a chunk are reported as failed and every other file is restored (non-zero exit); without the flag, abort as now.
6. Restore and verify error lines repeat a prefix or name the wrong store
Missing blob during restore: downloading blob b9a4c28a839c5c5c: downloading blob b9a4c28a839c5c5c: object not found (downloadNextBlobSet wraps the already-prefixed error from FetchBlob).
Failed shallow verify: Verification failed: verification failed: 1 blobs have the wrong size (formatVerifyResult wraps errSnapshotVerifyFailed under the cli's own prefix).
verify --deep on a file:// store with a truncated blob: S3 has 1000 bytes, database has 3420651 bytes (verifyBlobExistenceFromDB).
Correct: each prefix once; name the destination store, not S3.
Outside this part (config), seen during setup: chunk_size: 1MB is rejected with invalid config: chunk_size must be at least 1MB, because 1MB reads as 1,000,000 bytes and the minimum is 1 MiB (minChunkSize, errChunkSizeTooSmall in internal/config/config.go).
Model: opus-5-5
## Part 3: Read restore and verify
Six defects confirmed on `next` at `fa56165`, with real runs of `./vaultik` (`make build`, `file://` destination, own config and index in a temp dir) and two throwaway tests that fail under `make test` (not committed; essential lines below).
### 1. `--skip-errors --verify` counts a file that failed to restore again as a checksum failure
`verifyRestoredFiles` (`internal/vaultik/restore.go`) verifies every selected file, including those `handleRestoreFileError` already recorded, and adds them to the same `FilesFailed`/`FailedFiles`; `handleRestoreVerification` then calls them all checksum mismatches.
Shown: 12 files in 4 blobs, the blob holding `p8.bin` removed from the store, `vaultik --skip-errors snapshot restore --verify $SID r2`:
```
Warning: 1 file(s) failed to restore:
.../src/pics/p8.bin
Verifying 11 files (12 MB).
ERROR: Verification failed: 2 files did not match expected checksums.
.../src/pics/p8.bin
.../src/pics/p8.bin
ERROR: Restore failed: 2 files failed verification
```
No restored file mismatched. Correct: verify only the files that were written; report 1 failed to restore and 0 failed verification, each path once.
### 2. A restore path argument that selects nothing is not reported, and the command exits 0
`getFilesToRestore` drops a path that matches nothing; when none match, `Restore` warns `No files found to restore.` and returns nil (`internal/vaultik/restore.go`). Each of these exits 0:
- `snapshot restore $SID r3 $W/src/docs/typo.txt`: `No files found to restore.`
- `snapshot restore $SID r4 $W/src/docs $W/src/doc`: restores `docs`, says nothing about `$W/src/doc`.
- a relative path (`srv/code/.../src/docs`): `No files found to restore.`
- a configured path that is a symlink (`paths: [$W/srclink]`, `srclink` pointing to `src`): entries are stored under the resolved `$W/src`, so `snapshot restore $SID r9 $W/srclink/docs` gives `No files found to restore.`
Correct: a path argument that matches nothing in the snapshot is an error naming it, with a non-zero exit.
### 3. A restore with path arguments keeps blobs in the temporary cache until it ends
`restoreSweeper.blobStillNeeded` (`internal/vaultik/restore_sweeper.go`) keeps a cached blob while any file in the snapshot database that uses one of its chunks is not in the restored set. Files outside the selection (and files that failed under `--skip-errors`) never enter that set, so such a blob is never freed. Each entry is a whole decrypted blob in `$TMPDIR` (up to `blob_size_limit`, 10 GB by default), so the temporary space grows with the number of blobs touched instead of staying near one blob.
Shown: throwaway test on the `TestRestoreLocalityAndReadAt` fixture (3 blobs), restoring by path every `src-*.bin` except `src-005.bin` and `src-015.bin`:
```go
require.NoError(t, v.Restore(&RestoreOptions{SnapshotID: snapshotID, TargetDir: restoreDir, Paths: paths}))
assert.LessOrEqual(t, cacheRef.PeakLen(), 1)
// "3" is not less than or equal to "1"
```
The same fixture restored without paths peaks at 1. Correct: free a blob once no file still pending in this restore needs it.
### 4. Interrupting a restore during a stalled blob read leaves the decrypted snapshot database and decrypted file data in `$TMPDIR`
`downloadBlobToCache` copies the blob with `io.Copy` in `blobDiskCache.PutFromReader` (`internal/vaultik/restore.go`, `blobcache.go`). The context is checked only between blobs, and the `file://` backend's `Get` returns an `os.File` that ignores it. When one blob read outlasts `shutdownTimeout` (30 s), the process exits without the restore's cleanup. #159 required that "the blob download into the cache stops on cancel"; its fix checks only between blobs. With the default 10 GB `blob_size_limit` on a USB stick or a slow NAS, one blob takes minutes.
Shown: one blob on the `file://` destination replaced by a FIFO that delivers the first 1 MB and then stalls (standing in for a slow device); `snapshot restore`; SIGINT after 4 s. vaultik exits 130 after 30 s, logging `Error during shutdown: context deadline exceeded`, and leaves `$TMPDIR/vaultik-restore-*/snapshot.db` (plaintext SQLite) and `$TMPDIR/vaultik-blobcache-*/HASH` (917504 bytes of decrypted file content). The warning that decrypted files may remain is not printed.
Correct: cancelling stops the blob stream, so the cleanup runs before exit.
### 5. Under `--skip-errors`, one file whose chunk is in no blob aborts the whole restore before anything is restored
`newRestorePlan` (`internal/vaultik/restore_plan.go`) returns `errPlanChunkMissing` (or `errPlanBlobIDMissing`) for the first such file and `restoreAllFiles` returns it; it never reaches `handleRestoreFileError`. Earlier backup defects such as #214 left exactly such snapshots, and `--skip-errors` is how a user gets the rest back.
Shown: throwaway test: 20 single-chunk files in two blobs, one blob's `snapshot_blobs` and `uploads` rows deleted from the local index, metadata exported again, restore with `SkipErrors: true`:
```
restore error: building restore plan: planning .../src/file-00.bin: chunk missing from blob map: b70c709106b3e1b6
"0" is not positive (no file restored)
```
Correct: under `--skip-errors` the files that need such a chunk are reported as failed and every other file is restored (non-zero exit); without the flag, abort as now.
### 6. Restore and verify error lines repeat a prefix or name the wrong store
- Missing blob during restore: `downloading blob b9a4c28a839c5c5c: downloading blob b9a4c28a839c5c5c: object not found` (`downloadNextBlobSet` wraps the already-prefixed error from `FetchBlob`).
- Failed shallow verify: `Verification failed: verification failed: 1 blobs have the wrong size` (`formatVerifyResult` wraps `errSnapshotVerifyFailed` under the cli's own prefix).
- `verify --deep` on a `file://` store with a truncated blob: `S3 has 1000 bytes, database has 3420651 bytes` (`verifyBlobExistenceFromDB`).
Correct: each prefix once; name the destination store, not S3.
Outside this part (config), seen during setup: `chunk_size: 1MB` is rejected with `invalid config: chunk_size must be at least 1MB`, because `1MB` reads as 1,000,000 bytes and the minimum is 1 MiB (`minChunkSize`, `errChunkSizeTooSmall` in `internal/config/config.go`).
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
One defect hunt on
nextbefore the release candidate: use vaultik the way a user would, and read the code paths for correctness. Real defects only: no new features, no refactors for taste, nothing already decided or waiting on sneak (#239 and #204 are out of scope).Parts, one worker each
config init, backups to afile://destination (first, unchanged, changed, deleted and added files, more than one path),snapshot list,snapshot verifyand--deep,snapshot restore(whole and by path,--verify),snapshot purge,snapshot remove,prune,remote info,remote nuke. Failure cases: a destination that is full (a small size-limited mount inside a container), not writable, or removed part-way; a missing or wrong private key; a damaged or missing blob; an interrupted backup and restore; a second writer while one runs.internal/snapshot,internal/blob,internal/chunker,internal/blobgenwriter, the backup side ofinternal/vaultikandinternal/database).internal/vaultikrestore, restore plan, blob fetch and cache, verify;internal/blobgenreader; manifest and snapshot ID handling.prune,snapshot purge,snapshot remove,snapshot list,info,remote info,remote nuke, the process lock, and the snapshot tables they change.internal/storage,internal/s3,internal/config,internal/cli.Definition of done
next.Model: opus-5-5
Part 4: Read retention and destination management
Seven defects confirmed with real runs of
./vaultik(make buildonnextatfa56165,file://destination, own config, index and lock directory). No test was written.1. A retention duration of 293 years or more deletes every snapshot
parseDuration(internal/vaultik/helpers.go) computestime.Duration(n) * year(and the same forw,mo,d) with no overflow check. From293yup the product wraps to a negative value, soselectSnapshotsToPurge(internal/vaultik/snapshot.go) puts the cutoff in the future and selects every snapshot. A user who writes--prune --keep-newer-than 1000yto keep everything loses every snapshot of that name, including the one just made, and thenPruneBlobsdeletes their blobs. The command exits 0.echo n | vaultik snapshot purge --older-than 1000ylisted all four snapshots, including one taken a second earlier.292ylists none, while293y,15300wand3600moeach list all of them.vaultik snapshot create --cron --prune --keep-newer-than 1000y homeexited 0 and left nohomesnapshot and none of its blobs on the store.2.
pruneandremote nukeexit 0 when a delete fails, and nuke still says the store is emptydeleteUnreferencedBlobs(internal/vaultik/prune.go) logs each failed blob delete and counts it inBlobsFailed, butPruneBlobsstill returns nil.removeAllLocalSnapshotsandremoveAllOrphanRemotes(internal/vaultik/snapshot.go) count failed metadata deletes and only warn. After either kind of failure,NukeRemoteprints "Backup destination store is now empty." The nuke warning also says "Run 'vaultik prune' once the remote is healthy", but prune never removes metadata (issue #251 fixed the same advice insnapshot remove).chmod 555),prune --jsonprinted"blobs_failed": 1and exited 0.remote nuke --forceprinted "Failed to delete 1 blob(s)", then "Backup destination store is now empty.", exited 0, and left the blob in place. With one metadata directory read-only instead, nuke printed the warning and "now empty", exited 0, and left four files.3.
remote nukeleaves objects it does not recognise and says the store is emptyWhat is wrong:
NukeRemotedeletes only the metadata keys thatlistAllRemoteSnapshotKeysreturns, which requires amanifest.json.zst, and only the blobs thatlistAllRemoteBlobsreturns, which skips names that are not blob hashes. Two kinds of object therefore survive:metadata/KEY/db.zst.agewith no manifest.uploadSnapshotArtifactsuploads the database before the manifest, so a failed manifest upload leaves this behind. Once the next backup drops the incomplete local record, nothing removes it.blobs/that is not a blob hash, such as a Finder.DS_Store.The README says nuke leaves "the bucket prefix empty".
How shown: I blocked a real backup's manifest upload by putting a directory at the manifest path. The backup failed and left
metadata/52a6.../db.zst.agewith no manifest. After one good backup,remote nuke --forceprinted "Backup destination store is now empty." and exited 0, and thatdb.zst.agewas still there. Withblobs/0b/01/.DS_Storepresent, nuke logged "Skipping non-conforming object under blobs/", printed "now empty", exited 0, and the file remained.Correct: nuke deletes everything under
metadata/andblobs/, or it names what it left and exits non-zero.4.
remote nukeagainst an unreachable destination wipes the local index, then failsRemoveAllSnapshots(internal/vaultik/snapshot.go) turns a failed listing into a warning. Its comment says this is so a local-only remove is not blocked, but nuke is not local-only.executeRemoveAllthen deletes every local snapshot record even though each remote delete fails.CleanupOrphanedDatadrops every file, chunk and blob row. Only after that doesPruneBlobsfail.remote nuke --force. It printed "Removed 2 snapshot(s)" and "Run 'vaultik prune' once the remote is healthy ...", then "Remote nuke failed", and exited 1. The local index then held 0 snapshots, 0 files and 0 blobs. With the directory back, the store still held all six objects. The snapshots are now remote-only rows with their names lost, andprunenever removes them.5.
snapshot removereports success for a snapshot that does not existRemoveSnapshot(internal/vaultik/snapshot.go) never checks that the ID exists.deleteSnapshotFromLocalDBdeletes zero rows without error, anddeleteRemoteSnapshotByKeyfinds nothing to delete.snapshot remove --force hunt_home_1999-01-01T00:00:00Zprinted "Removed snapshot ... from local database" and "Removed snapshot metadata from remote storage", and exited 0.--jsongave"remote_removed": true, and--dry-runprinted "Would remove snapshot". Passing the remote key thatsnapshot listprints for a snapshot (restore and verify accept it) gave the same two success lines and left thatmetadata/KEY/untouched.6.
--cronand--quietdo not silence prune output, so the README's cron line produces output on every runPruneBlobs(internal/vaultik/prune.go) andCleanupLocalSnapshots(internal/vaultik/snapshot.go) write their status lines withstdoutfandprintlnStdout, not throughinternal/ui. The README says--cronis silent on total success and that--quietsilences status lines.snapshot create --cron --prune etcandsnapshot create --cron --prune --keep-newer-than 4w etcboth exited 0 and printed "No unreferenced blobs to remove." on stdout, or "Found N unreferenced blob(s) ... Deleted N blob(s) ..." when there were blobs to delete.vaultik -q prune --forceprinted "No stale local snapshots found." and "No unreferenced blobs to remove."--cronor--quiet.7.
remote infocounts a non-blob file underblobs/as an orphaned blobscanRemoteBlobStorage(internal/vaultik/info.go) counts every four-part key underblobs/as a blob. Prune'slistAllRemoteBlobsskips names that are not blob hashes.blobs/56/f6/.DS_Storepresent,remote inforeported "Total blobs on remote: 3", "Orphaned (unreferenced): 1 (4 B)" and "Run 'vaultik prune' to remove orphaned blobs."prune --forcelogged "Skipping non-conforming object under blobs/" and printed "No unreferenced blobs to remove.", andremote infokept giving the same advice.remote infoapplies the same blob-name check as prune.Outside this part (for part 5 to take or drop)
chunk_size: 1MBis rejected with "chunk_size must be at least 1MB" (internal/config/config.go): the minimum is 1 MiB, while1MBis read as 1,000,000 bytes.1MiBworks.Model: opus-5-5
Part 2: Read the backup path
Eight confirmed defects on
nextatfa56165, plus one config defect found while setting up (outside part 2).1. With the legacy
s3.*configuration, a changed bucket or prefix is not detected: the next backup reports success and cannot be restoredstorage_url. With thes3.*form (README: still supported), the check is skipped. Afters3.bucketors3.prefixchanges, the next backup treats every chunk as already stored, uploads nothing, and completes a snapshot whose blobs are only at the old destination.internal/vaultik/storage_bind.goEnsureStorageBinding(if configured == ""returns nil).TestHuntLegacyS3ConfigNewBucketIsRefused(internal/vaultik, kept local). It uses a legacy config (emptyStorageURL), backs up to bucket A, setss3.bucketto B and backs up to B. The second backup succeeds:backup to the new bucket reported success with 0 blobs uploaded there; restoring test-host_second_... from it: ... downloading blob 328c1947f63b2fd2: object not found.storage_url.2. A reused blob keeps the size of a local re-encryption that was never stored, so
snapshot verifyfails on a good snapshotcompression_levelor the recipients changed since the stored object was written, the sizes differ. Shallow and deep verify then fail, and the listed sizes are wrong. Restore works.internal/snapshot/scanner.gouploadBlobIfNeeded(Stat succeeds, upload skipped) andrecordBlobMetadata. The size comes from the packer'sUpdateFinished, andinternal/snapshot/snapshot.gogenerateBlobManifestwrites it.database delete --force, setcompression_level: 19, thensnapshot create: "Blob 23036003a9d6... (544 kB) already exists ... Skipping upload". After that,snapshot verifyreports "Wrong size: 1 blobs" (exit 1), and--deepreports "blob size mismatch: ... S3 has 602206 bytes, database has 544258 bytes" (exit 1).restore --verifysucceeds and the file is identical.3. Ctrl-C during a backup transaction exits 1 with "FATAL: failed to rollback transaction"
database/sqlrolls the transaction back itself, so the explicitRollbackreturnssql.ErrTxDone.WithTxtreats that as fatal and callsos.Exit(1). The user gets a message that suggests index damage, and exit 1 instead of 130 "interrupted before the command finished". Deferred cleanup is skipped.internal/database/repositories.goWithTx(Fatalfon any rollback error).FATAL: failed to rollback transaction: sql: transaction has already been committed or rolled back.ErrTxDoneafter a failed body as rolled back and return the body's error, so the run ends as interrupted (130).4. A failed or interrupted backup leaves its half-written blob in
$TMPDIRsnapshot createstops on a file error or Ctrl-C between blobs, the blob being packed is never closed or removed. Itsvaultik-blob-*.tmp(encrypted data, up toblob_size_limit, 10 GB by default) stays behind, and every such run adds one. Nothing removes them later.internal/snapshot/scanner.goprocessPhasereturns without discarding the packer's current blob.internal/blob/packer.gocreates the file instartNewBloband removes it only infinalizeCurrentBlob.a1and achmod 000fileb2.snapshot createexits 1 and leaves$TMPDIR/vaultik-blob-619287593.tmp(131 kB); a second run left a second file. A SIGINT run (exit 130) leftvaultik-blob-846065746.tmp.5.
**/nameexclude patterns miss a match directly under the source rootconfig inittemplate says patterns without a leading/match anywhere in the tree and that**is supported.**/node_modulesand**/*.pycexcludesub/node_modulesandsub/y.pyc, but back upnode_modules/andx.pycat the top of the source path.config.example.ymluses exactly these two patterns.internal/snapshot/scanner.gomatchesUnanchored. None of the strings it tries for a top-level entry contains the/that the pattern requires.exclude: ["**/node_modules", "**/*.pyc"]; after the backup the index holdssrc/node_modules/top.jsandsrc/x.pyc.**/xalso matchesxat the source root, as in gitignore.6. "deleted" counts files that another snapshot backs up, on every run
config.example.yml,homeexcludes/Library/Caches, whichsystem(/) backs up, anddevexcludes**/node_modules, whichhomebacks up.internal/snapshot/scanner.goloadKnownFilesanddetectDeletedFilesFromMap.home=srcexcluding/photos, andphotos=src/photoswith 3 files. Every run ofhomeafter the first prints "detected 4 deleted files" and "Files: 1 examined, 0 backed up, 1 unchanged, 4 deleted", "6.0 kB deleted", with nothing deleted.7. Directories and symlinks are counted as files to back up on every run
internal/snapshot/scanner.gorecordSpecialEntry(queues every directory and symlink; symlinks counted as changed throughupdateScanEntryStats) andsummarizeScanPhase(counts and sizesfilesToProcess, while the enumerated count leaves directories out).8. The pre-backup index prune reports total rows as "remain"
snapshot createprints lines such as "Incomplete snapshots: 0 removed (1 remain)" and "Orphaned files: 0 removed (12 remain)". The second figure is all snapshots, files, chunks or blobs left in the index, not incomplete or orphaned ones. After removing the only incomplete snapshot it printed "1 removed (1 remain)".internal/vaultik/snapshot.goPruneDatabase(the fourDetailflines).Outside part 2 (config), found while setting up
chunk_size: 1MBis rejected with "chunk_size must be at least 1MB": the minimum is 1048576 and1MBparses as 1000000. Theconfig inittemplate'schunk_sizecomment (internal/cli/config.go) says "Accepts: 1MB, 10M, 64KB", but1MBand64KBare both rejected. Where:internal/config/config.gominChunkSizeanderrChunkSizeTooSmall. Correct: the message names the real minimum, and the template lists only accepted values.Model: opus-5-5
Part 1: Use the binary as a user would
make buildonnextatfa56165,file://destinations, own config, index, lock and$TMPDIR. Every item was shown with real runs of./vaultik; no test was written.1. A destination removed part-way through a backup is recreated, and the backup reports success with a blob missing
snapshot verifynotices.internal/storage/file.gowriteAtomic(MkdirAllfor the blob's directory also recreates the destination root).snapshot verifysays "1 blobs are missing" (exit 1). The nextsnapshot createexits 0; restoring its snapshot fails with "downloading blob d750...: object not found".file://destination root, so the run fails and the next run discards the incomplete snapshot's blobs and uploads them again. This is the mid-run case, not #239's start-of-run case, but the owner's answer there (a missingfile://destination is an error) covers it if enforced on every write.2. An invalid
--keep-newer-thanis rejected only after the backup, which then reports "Snapshot creation failed" although the snapshot was made1.5d,0), the backup completes and uploads its snapshot, then the command prints "Snapshot creation failed: post-backup prune: purging old snapshots: invalid duration" and exits 1. Retention never runs, so such a cron line adds a snapshot every night, reports each as failed, and never deletes any.internal/vaultik/snapshot.gorunPostBackupPrune(gets the raw string);internal/cli/snapshot.godoes not check--keep-newer-than.snapshot create ex --prune --keep-newer-than 1.5dexits 1 with that message;snapshot list --jsonthen shows the new snapshottesthost_ex_2026-10-12T22:33:14Zwithremote_present: true.3.
config setwith a list index on a key that does not exist writes a map, and then no command can load the configvaultik config set age_recipients.0 age1...(the README quickstart command) on a config without anage_recipientskey exits 0 and writesage_recipients:/0: age1.... Every later command fails with "failed to parse config: yaml: unmarshal errors: cannot unmarshal !!map into []string". Same withconfig set snapshots.NAME.paths.0 /path. An index into an existing list works.internal/cli/config.goyamlSetInMapping(creates a mapping for any missing key, even when the next path element is a number).config setnever writes a file the loader rejects.4.
snapshot restoreignores requested paths that match nothing, and exits 0snapshot restore ID TARGET /src1/a /src1/typorestores/src1/a, says nothing about/src1/typo, and exits 0. When no path matches (a typo, or a relative path such assrc1/a), it prints "Warning: No files found to restore." and exits 0 with nothing restored.internal/vaultik/restore.goRestore(returns nil on an empty file list; nothing checks each requested path).5.
snapshot create --skip-errorsexits 0 and prints "Finished successfully." after skipping files--cron). The skipped directory is not counted as failed. A script or cron wrapper cannot tell a complete backup from one missing files, whilesnapshot restoreunder the same flag exits non-zero when files failed (#218).internal/vaultik/snapshot.go(totalFilesFailedfeeds the summary only; the run returns success).6.
snapshot listshows NEW CHUNK SIZE 0 B for a snapshot whose change is a modified fileinternal/database/snapshots.goGetSnapshotNewChunkSize.snapshot listshows that snapshot's NEW CHUNK SIZE as0 B.7.
snapshot listand thesnapshot purgelisting print 1024-based sizes labelled "MB"snapshot list, and as "6.3 MB" in thesnapshot createsummary,snapshot verifyandremote info.internal/vaultik/helpers.goformatBytes(divides by 1024, prints "KB"/"MB"), used bysnapshot_list.goand the purge listing insnapshot.go. Everything else useshumanize.Bytes(powers of 1000).8.
snapshot verify --deep --jsonreports"missing": 0and"verified": 0when it failsverifiedonly on success and never setsmissing. With one of two blobs deleted,--deep --jsongives"verified": 0, "missing": 0(shallow--jsongives"verified": 1, "missing": 1). With bytes flipped in one of two blobs it gives"verified": 0, although the text run reports "Verified 1/2 blobs" before failing.internal/vaultik/verify.goRunDeepVerify.9.
config editfails when$EDITORcarries argumentsEDITOR="cat -n" vaultik config editfails withexec: "cat -n": executable file not found in $PATH, exit 1. Common settings such ascode --waitoremacsclient -tfail the same way.internal/cli/config.gonewConfigEditCommand(the whole of$EDITORis the program name).$EDITORinto words (or run it throughsh -c), as git and crontab do.10. Every backup reports files deleted long ago as "deleted" again
internal/snapshot/scanner.godetectDeletedFilesFromMap. Same function as part 2's item 6 but a different trigger, which a fix for excluded files does not cover.11. Every restore by a non-root user warns that ownership was not preserved, even when it was
internal/vaultik/restore.gofinishRestore(checks onlyos.Geteuid() != 0).12.
remote nuketells the user to runprunefor the blobs it is about to deleteremote nuke --forceprints "Note: Removed snapshots' blobs remain on the remote. Run 'vaultik prune' to delete any blobs ...", then deletes every blob itself and says the store is empty.internal/vaultik/snapshot.goexecuteRemoveAll. Next to #290, which covers the advice on the failure path.13. After
storage_urlchanges,snapshot listadvisesprune, which then refusessnapshot listreports its snapshots as "local snapshot record(s) not found in backup destination store" and says "Run 'vaultik prune' to remove stale local records."prune,snapshot purgeandsnapshot removethen refuse because the index is bound to another destination.internal/vaultik/snapshot_list.goreportListDrift(does not check the binding).storage_url, ordatabase delete).Also reproduced, already reported
Part 2's items 4 (blob temp file left in
$TMPDIR, also after an ENOSPC on$TMPDIR), 7 and 8; thechunk_size: 1MBrejection from parts 2 and 4 (config.example.ymlalso lists500KBunder "Supports:"); and #293.Model: opus-5-5
Part 5: Read the storage backends, config and command line
Each item below was confirmed on
nextatfa56165, withmake build's./vaultikand a throwaway config, index and destination in a temp dir, unless a test is named.1. A symlinked
metadataorblobsdirectory in afile://destination lists as empty;prunethen deletes every blobFileStorer.ListandListStream(internal/storage/file.go) walk withafero.Walk, which does not follow a symlink, so a symlinkedmetadata/orblobs/is listed as one non-directory key.Stat,GetandPutdo follow it, so backups and restores keep working while every listing is empty.mv dest/metadata elsewhere/ && ln -s elsewhere/metadata dest/metadata.snapshot listthen warns "1 local snapshot record(s) not found in backup destination store".prune --forceprints "Removing stale local record …" and "Deleted 1 blob(s)", leaving 0 blobs while both metadata files are still in place behind the link. Withblobs/symlinked instead,remote info --jsongivestotal_blob_count: 0besidereferenced_blob_count: 1, andremote nuke --forceprints "Backup destination store is now empty." with the blob still there.StatandGetdo, or fails. It never reports a store it could not read as empty.2. A relative
file://path is accepted, so the destination changes with the working directory; a backup exits 0 and cannot be restoredParseStorageURL(internal/storage/url.go) takes everything afterfile://verbatim.storage_url: "file://dest"is accepted and resolved against$PWD. The local index's destination binding (internal/vaultik/storage_bind.go) compares the raw string, so it never notices the destination moved. The same verbatim handling skips percent-decoding:file:///…/my%20stick/backupcreates a new directory literally namedmy%20stickbesidemy stick.parseRcloneURLstrips the leading/, sorclone://loc/srv/xon a local-type remote writes under$PWD/srv/x.storage_url: "file://dest",snapshot createrun fromA/, a file changed, thensnapshot createrun fromB/: both exit 0. The unchanged file's blob exists only underA/dest, so fromB/,snapshot verifyfails with "1 blobs are missing" andsnapshot restorefails with "object not found".file://URL whose path is not absolute is rejected, percent-escapes are decoded, andrclone://remote/abs/pathkeeps its leading/(or a relative path on a local remote is refused).3. An S3 listing that the server marks truncated without a continuation token is taken as complete
Client.ListObjectsandListObjectsStream(internal/s3/client.go) use the SDK paginator, which ends silently when a page hasIsTruncated=truebut noNextContinuationToken. A server that answers V2 listings V1-style does that.prunethen counts only the manifests on the first page, and treats the blobs of the remaining snapshots as unreferenced (collectReferencedBlobs,internal/vaultik/prune.go).TestHuntS3ListingCutShortIsNotAnErrorininternal/storageputs gofakes3 behind a handler that pages by 2 and removesNextContinuationToken, then stores 5 keys. Undermake testit fails: "List: server marked the listing truncated, got 2 of 5 keys and no error", and the same forListStream.prunedeletes nothing.4.
snapshot removeof an ID that does not exist reports success and exits 0RemoveSnapshotandremoveSnapshotRemote(internal/vaultik/snapshot.go) hash whatever ID is given and never check that it matches anything. A mistyped ID, or the remote key thatsnapshot listprints for a remote-only snapshot (the only name a new host has, and oneverifyandrestoreaccept), removes nothing.snapshot remove --force KEY, withKEYtheremote_keyfromsnapshot list --json, prints "Removed snapshot '…' from local database" and "Removed snapshot metadata from remote storage", then exits 0;snapshot liststill shows the snapshot.snapshot remove --force nosuchidgives the same.verifyandrestore.5. An invalid
--keep-newer-thanis checked only after the backup, which is then reported as failed; without--pruneit is ignoredsnapshot create(internal/cli/snapshot.go) does not check--keep-newer-than. It is parsed in the post-backup prune.snapshot create --cron --prune --keep-newer-than 4xuploads a new snapshot (the count goes 5 to 6), then prints "Snapshot creation failed: post-backup prune: purging old snapshots: invalid duration: unknown time unit "x"" and exits 1. Retention never runs.snapshot create --keep-newer-than garbagewithout--prunebacks up and exits 0.--prune) is rejected before the backup starts. A prune failure after a good backup does not say the snapshot creation failed.6. Shallow
snapshot verifyreports any error reading a blob's size as "Missing", and drops the errorverifyManifestBlobsand thedb.zst.agecheck (internal/vaultik/snapshot.go) count everyStaterror as missing: permission denied, an S3 403, a network error.chmod 000on a blob's shard directory (the blob is present), thensnapshot verify: "Missing: ecb65bb9… (219 B)", "FAILED - 1 blobs are missing", exit 1. The real error is printed nowhere.ErrNotFoundcounts as missing. Any other error is reported as "could not check", with its cause.7.
--cronis not silent after a successful--prune, andprune,snapshot purge,snapshot removeandremote nukeignore--quietPruneBlobs(internal/vaultik/prune.go), andCleanupLocalSnapshots,RemoveSnapshot,RemoveAllSnapshotsandconfirmAndExecutePurge(internal/vaultik/snapshot.go), write throughv.stdoutf/printlnStdoutwith no quiet check. The README says--cronis "Silent on total success" and--quiet"Suppress[es] non-error output".snapshot create --cron --prune --keep-newer-than 4w, exits 0 and prints "No unreferenced blobs to remove.", or "Found 3 unreferenced blob(s) totaling 681 B … Deleted 3 blob(s)", on every run.-q prune --forceprints "No stale local snapshots found." and "No unreferenced blobs to remove.";-q snapshot purge --keep-latest --force,-q snapshot remove --force ID(and--dry-run) and-q remote nuke --forceprint their full reports.--cronor--quiet. Warnings and errors are still printed.8. A mistyped subcommand under
snapshot,config,remote,databaseorcompletionexits 0 and does nothinginternal/clihave noArgs/RunE, so cobra prints the group's help on stdout and returns success.vaultik bogusat the top level exits 1.vaultik snapshot craete srcprints thesnapshothelp and exits 0, as dovaultik config intiandvaultik completion nosuchshell. In crontab, such a line makes no backup and reports no error.9. Byte sizes: the defaults and the chunk-size minimum are binary units, while the docs, the parser and the display are decimal
defaultChunkSize,defaultBlobSizeLimitandminChunkSize(internal/config/config.go) are 10 MiB, 10 GiB and 1 MiB.Sizeparses10MBas 10,000,000 bytes (humanize.ParseBytes), andinfoprints decimal units.chunk_size: 1MBis rejected with "chunk_size must be at least 1MB". Theconfig inittemplate lists1MBand64KBas accepted values forchunk_size; both are rejected.blob_size_limit: 40MB, four times the documented10MBdefault chunk size (the README's stated minimum), is rejected. With no size settings,infoshows "Blob Size Limit: 11 GB", although the documented default is10GB. Writing that documented default out (blob_size_limit: 10GB) changes it:infothen shows "10 GB".10. A config validation error that contains "): " loses its start, including which setting is wrong
cleanStartupError(internal/cli/app.go) cuts each startup error at its last "): ", meaning to strip fx's annotations. Theblob_size_limitmessage itself contains "(… size spread): ".blob_size_limit: 40MB,info,snapshot createandsnapshot listprint only "ERROR: need at least 41943040 bytes" (exit 1).database delete, which loads the config without fx, prints the whole message: "failed to load config: invalid config: blob_size_limit must be at least … : need at least 41943040 bytes".11.
RCLONE_CONFIGis ignoredNewRcloneStorer(internal/storage/rclone.go) callsconfigfile.Install()but never appliesRCLONE_CONFIG. In rclone, only its CLI flag layer reads that variable.RCLONE_CONFIGpointing at a file that defines[loc] type = local,snapshot createfails with "rclone remote not found in config: loc". It succeeds once the same file is copied to$XDG_CONFIG_HOME/rclone/rclone.conf.rcloneitself would read,RCLONE_CONFIGincluded (README: "Requires rclone to be configured separately").12. Credentials can still be carried in
storage_url, and are then printedparseS3URL/parseRcloneURL(internal/storage/url.go) rejects only the outer URL's userinfo and unknown query parameters. Userinfo inside theendpoint=value, and rclone connection-string parameters in the remote name (remote,param=value), pass.storage_url: "s3://bkt/p?endpoint=https://AKIAUSERINFO:SECRETUSERINFO@127.0.0.1:1"is accepted.infoprints it in "Location:" and "Storage URL:", andremote infoin "Location:".rclone://loc,description=SECRETPARAMVALUE//pathis accepted and passed to rclone. The parameter appears ininfoand insnapshot create's upload lines.13.
remote nuketells the user the blobs remain and to runprune, then deletes themNukeRemote(internal/vaultik/prune.go) reusesRemoveAllSnapshots(internal/vaultik/snapshot.go), which prints the remove follow-up note.remote nuke --forceprints "Note: Removed snapshots' blobs remain on the remote. Run 'vaultik prune' to delete any blobs …", then "Deleted 1 blob(s)".remote nukedoes not print advice that its own next step makes false.Model: opus-5-5
Part 3: Read restore and verify
Six defects confirmed on
nextatfa56165, with real runs of./vaultik(make build,file://destination, own config and index in a temp dir) and two throwaway tests that fail undermake test(not committed; essential lines below).1.
--skip-errors --verifycounts a file that failed to restore again as a checksum failureverifyRestoredFiles(internal/vaultik/restore.go) verifies every selected file, including thosehandleRestoreFileErroralready recorded, and adds them to the sameFilesFailed/FailedFiles;handleRestoreVerificationthen calls them all checksum mismatches.Shown: 12 files in 4 blobs, the blob holding
p8.binremoved from the store,vaultik --skip-errors snapshot restore --verify $SID r2:No restored file mismatched. Correct: verify only the files that were written; report 1 failed to restore and 0 failed verification, each path once.
2. A restore path argument that selects nothing is not reported, and the command exits 0
getFilesToRestoredrops a path that matches nothing; when none match,RestorewarnsNo files found to restore.and returns nil (internal/vaultik/restore.go). Each of these exits 0:snapshot restore $SID r3 $W/src/docs/typo.txt:No files found to restore.snapshot restore $SID r4 $W/src/docs $W/src/doc: restoresdocs, says nothing about$W/src/doc.srv/code/.../src/docs):No files found to restore.paths: [$W/srclink],srclinkpointing tosrc): entries are stored under the resolved$W/src, sosnapshot restore $SID r9 $W/srclink/docsgivesNo files found to restore.Correct: a path argument that matches nothing in the snapshot is an error naming it, with a non-zero exit.
3. A restore with path arguments keeps blobs in the temporary cache until it ends
restoreSweeper.blobStillNeeded(internal/vaultik/restore_sweeper.go) keeps a cached blob while any file in the snapshot database that uses one of its chunks is not in the restored set. Files outside the selection (and files that failed under--skip-errors) never enter that set, so such a blob is never freed. Each entry is a whole decrypted blob in$TMPDIR(up toblob_size_limit, 10 GB by default), so the temporary space grows with the number of blobs touched instead of staying near one blob.Shown: throwaway test on the
TestRestoreLocalityAndReadAtfixture (3 blobs), restoring by path everysrc-*.binexceptsrc-005.binandsrc-015.bin:The same fixture restored without paths peaks at 1. Correct: free a blob once no file still pending in this restore needs it.
4. Interrupting a restore during a stalled blob read leaves the decrypted snapshot database and decrypted file data in
$TMPDIRdownloadBlobToCachecopies the blob withio.CopyinblobDiskCache.PutFromReader(internal/vaultik/restore.go,blobcache.go). The context is checked only between blobs, and thefile://backend'sGetreturns anos.Filethat ignores it. When one blob read outlastsshutdownTimeout(30 s), the process exits without the restore's cleanup. #159 required that "the blob download into the cache stops on cancel"; its fix checks only between blobs. With the default 10 GBblob_size_limiton a USB stick or a slow NAS, one blob takes minutes.Shown: one blob on the
file://destination replaced by a FIFO that delivers the first 1 MB and then stalls (standing in for a slow device);snapshot restore; SIGINT after 4 s. vaultik exits 130 after 30 s, loggingError during shutdown: context deadline exceeded, and leaves$TMPDIR/vaultik-restore-*/snapshot.db(plaintext SQLite) and$TMPDIR/vaultik-blobcache-*/HASH(917504 bytes of decrypted file content). The warning that decrypted files may remain is not printed.Correct: cancelling stops the blob stream, so the cleanup runs before exit.
5. Under
--skip-errors, one file whose chunk is in no blob aborts the whole restore before anything is restorednewRestorePlan(internal/vaultik/restore_plan.go) returnserrPlanChunkMissing(orerrPlanBlobIDMissing) for the first such file andrestoreAllFilesreturns it; it never reacheshandleRestoreFileError. Earlier backup defects such as #214 left exactly such snapshots, and--skip-errorsis how a user gets the rest back.Shown: throwaway test: 20 single-chunk files in two blobs, one blob's
snapshot_blobsanduploadsrows deleted from the local index, metadata exported again, restore withSkipErrors: true:Correct: under
--skip-errorsthe files that need such a chunk are reported as failed and every other file is restored (non-zero exit); without the flag, abort as now.6. Restore and verify error lines repeat a prefix or name the wrong store
downloading blob b9a4c28a839c5c5c: downloading blob b9a4c28a839c5c5c: object not found(downloadNextBlobSetwraps the already-prefixed error fromFetchBlob).Verification failed: verification failed: 1 blobs have the wrong size(formatVerifyResultwrapserrSnapshotVerifyFailedunder the cli's own prefix).verify --deepon afile://store with a truncated blob:S3 has 1000 bytes, database has 3420651 bytes(verifyBlobExistenceFromDB).Correct: each prefix once; name the destination store, not S3.
Outside this part (config), seen during setup:
chunk_size: 1MBis rejected withinvalid config: chunk_size must be at least 1MB, because1MBreads as 1,000,000 bytes and the minimum is 1 MiB (minChunkSize,errChunkSizeTooSmallininternal/config/config.go).Model: opus-5-5