The verb surface accumulated overlapping cleanup commands. Consolidate
so each cleanup verb has one meaning:
- Rename 'database purge' -> 'database delete'. The command removes the
SQLite file entirely; "purge" wrongly suggested purging contents.
- Fold 'snapshot cleanup' into 'prune'. Prune now runs three passes:
reconcile local snapshots against the remote (previously the
standalone cleanup command), drop orphaned local rows, then delete
unreferenced remote blobs. One command, one mental model.
- Delete 'store info'. Its output was a strict subset of 'remote info',
which already prints storage type + location. Any user reaching for
either should reach for 'remote info'.
- Drop 'snapshot remove --all'. It duplicated 'remote nuke --force'.
'remote nuke' is the single supported entry point for wiping the
destination store.
Also update the storage-binding error message to reference the new
'vaultik database delete' name.
The local index tracks which chunks and blobs already exist at the
backup destination. Nothing was recording *which* destination, so
changing storage_url and running a backup left the scanner treating
every already-seen chunk as still-present at the new (empty) location.
Uploads were skipped silently and the resulting snapshots pointed at
blobs that don't exist at the new destination.
Fix: record storage_url in a new local_meta key-value table on first
mutating command, and refuse to proceed when the configured URL later
differs from the stored one. The error explains the two recovery
paths (revert the config, or run 'vaultik database purge' to discard
the index and rebuild from a fresh full backup).
Wired into snapshot create / prune / snapshot remove / snapshot purge
/ snapshot cleanup. Read-only inspection commands (snapshot list,
remote info, store info) are exempt.
snapshot rm <id> now does the full cleanup: removes the local index
entry, strips the snapshot's metadata from the destination store, and
prunes any blobs that were only referenced by the just-removed manifest.
The --remote flag is retired; --local-only opts out for the rare case
where the user wants to forget a snapshot locally without touching the
remote.
If the destination store is unreachable, the local-DB removal still
completes and a warning is emitted; the user can rerun 'vaultik prune'
to retry the remote half later.
RemoveAllSnapshots gets the same treatment: after deleting every
snapshot's metadata (local + remote + orphan keys), an automatic blob
prune sweep removes the now-unreferenced blob set.
Two related changes, both addressing leakage and brittleness around
the public bytes the destination store sees.
First, every remote storage path that previously embedded a human
snapshot ID (e.g. metadata/heraklion_berlin.sneak.fs.photos.2026.
catalog_2026-06-24T07:00:15Z/...) now uses the hashed remote key:
RemoteSnapshotKey(id) = hex(SHA256(SHA256("vaultik|" + id)))
Applied at:
* uploadSnapshotArtifacts (snapshot create write path)
* the manifest.json.zst snapshot_id field — manifest is
unencrypted, so the human ID would otherwise be readable to
anyone with bucket-list permission
* cleanupIncompleteSnapshots metadata-existence probe
* snapshot restore / verify (downloadSnapshotDB,
loadVerificationData)
* downloadManifestByKey, deleteRemoteSnapshotByKey
* CleanupLocalSnapshots reconciliation
* the locally-driven removal paths (RemoveSnapshot,
RemoveAllSnapshots, confirmAndExecutePurge)
The local index database keeps human IDs everywhere — the hash is a
boundary translation, not a rename. A directory listing of the
backup destination now looks like
"metadata/<64-hex>/{db.zst.age,manifest.json.zst}" with no host,
snapshot-name, or timestamp information visible.
Second, snapshot list no longer fails just because remote storage is
unreachable, and only consults the remote when the local machine can
plausibly decrypt:
* Listing is always driven by the local index database — that's
what holds the human IDs, timestamps, and per-snapshot stats
that the table actually shows.
* If no age secret key is configured, we skip remote listing
entirely (the box is treated as a write-only backup machine —
there's no value showing it remote-only keys it could never
restore).
* If a key IS configured, we try the remote listing; failures
(volume unmounted, permission denied, network error) downgrade
to a warning instead of aborting the command.
* When the remote listing succeeds, we cross-reference by hashing
each local human ID and diffing against the returned key set.
Local-only snapshots get the existing "stale local record"
cleanup hint; remote-only keys are surfaced as a single
"NOTE: N remote snapshot(s) found in backup destination store
but not in local database" line.
FileStorer construction also no longer does an eager mkdir — the
basePath is recorded and the directory is created lazily on first
write. A missing or unmounted destination during `snapshot list`
should NOT block the command, and now it doesn't.
RemoveAllSnapshots is rewritten to drive deletion from the local
index instead of from a remote listing, hashing each local ID to
find the corresponding remote key. Orphan remote keys (no matching
local snapshot) are handled separately and only deleted when
--remote is set. Existing tests are updated to hash storage paths
through the new RemoteSnapshotKey helper.
The hash format is a hard pre-1.0 break: existing remote snapshots
written under the human-ID path scheme are no longer readable; they
need to be either re-uploaded under the new scheme or manually
renamed. There is no fallback path; matching the project policy of
"no migrations pre-1.0."
The CLI had two commands named "prune" doing different jobs (local
DB orphan cleanup vs. remote blob garbage collection), which was
confusing and forced a manual two-step workflow after deleting any
snapshot.
Single user-facing prune surface is now `vaultik prune`, which calls
PruneDatabase (local orphan cleanup) then PruneBlobs (remote unref
blob GC). Snapshot deletion paths (snapshot remove, snapshot remove
--all, snapshot purge) auto-run CleanupOrphanedData inline so the
local index database doesn't accumulate ghost rows after every
removal — the user observed ~39k orphaned files and 2 orphaned blobs
after a remove --all because that cleanup was previously a separate
opt-in command. `snapshot prune` is removed.
Also addresses the doc/help-string drift the user audit caught:
* cli/prune.go help text used to reference a non-existent
`vaultik purge` command.
* cli/config.go get/set short/long examples were S3-specific
(s3.bucket) when the primary storage configuration is
storage_url.
* vaultik/info.go printed S3 Bucket/Endpoint/Region labels
unconditionally; for file:// or rclone:// users those rows
were empty. The Storage Configuration block now prints the
storer's Type+Location first, the storage_url string when set,
and only emits S3 rows that are actually populated.
* vaultik/info.go's "Run 'vaultik prune --remote'" hint
referenced a flag that doesn't exist.
* vaultik/blobcache.go's doc comment claimed LRU eviction, which
is no longer the restore-time policy (the sweeper drives
eviction; LRU is the safety-net fallback when maxBytes is
finite).
* README.md listed `vaultik restore`, `vaultik snapshot prune`,
and `s3.bucket` example, all out of date.
README's roadmap section is rewritten with concrete pre-1.0 items
(security audit, error-condition tests, parallel blob downloads,
restart of interrupted restore, …) so the next-steps surface
matches what the project actually still needs.
The cleanup calls are guarded against a nil SnapshotManager so
tests that construct a bare Vaultik struct continue to work.
remote nuke: new subcommand that deletes every snapshot's metadata and
every blob from remote storage, leaving the bucket prefix empty.
Requires --force.
User-facing 'Processing' is now 'Backing up' everywhere it referred to
the chunking/upload phase. Files summary line says 'backed up' instead
of 'processed'.
ui.Speed now formats bytes/sec input as bits/sec output (bit/s, Kbit/s,
Mbit/s, Gbit/s). Network transfer rates are conventionally expressed
in bits — the per-blob heartbeat now matches the per-snapshot summary
line which has always been bits/sec.
Module path changed from git.eeqj.de/sneak/vaultik to
sneak.berlin/go/vaultik (vanity redirect). All imports, ldflags,
Dockerfile, goreleaser config, and docs updated. App data/config
directories now use plain "vaultik" instead of the reverse-DNS name.
README:
- New copy-pasteable quickstart at top: go install, config init,
age keypair, config set for key + file:// destination, home backup
- All command names in command details are code-quoted
- config set/get gained sequence index support (age_recipients.0)
so lists are settable from the CLI
- Dockerfile build is CGO_ENABLED=0 to match the pure-Go build
Address all four review concerns on PR #31:
1. Fix missed bare fmt.Println() in VerifySnapshotWithOptions (line 620)
2. Replace all direct fmt.Fprintf(v.Stdout,...) / fmt.Fprintln(v.Stdout,...) /
fmt.Fscanln(v.Stdin,...) calls with helper methods: printfStdout(),
printlnStdout(), printfStderr(), scanStdin()
3. Route progress bar and stderr output through v.Stderr instead of os.Stderr
in restore.go (concern #4: v.Stderr now actually used)
4. Rename exported Outputf to unexported printfStdout (YAGNI: only helpers
actually used are created)
Multiple methods wrote directly to os.Stdout instead of using the injectable
v.Stdout writer, breaking the TestVaultik testing infrastructure and making
output impossible to capture or redirect.
Fixed in: ListSnapshots, PurgeSnapshots, VerifySnapshotWithOptions,
PruneBlobs, outputPruneBlobsJSON, outputRemoveJSON, ShowInfo, RemoteInfo.
- Add global --quiet/-q flag to suppress non-error output
- Add --json flag to verify, snapshot rm, and prune commands
- Add config file permission check (warns if world/group readable)
- Update TODO.md to remove completed items
Storage backend:
- Add internal/storage package with Storer interface
- Implement FileStorer for local filesystem storage (file:// URLs)
- Implement S3Storer wrapping existing s3.Client
- Support storage_url config field (s3:// or file://)
- Migrate all consumers to use storage.Storer interface
PID locking:
- Add internal/pidlock package to prevent concurrent instances
- Acquire lock before app start, release on exit
- Detect stale locks from crashed processes
Scan progress improvements:
- Add fast file enumeration pass before stat() phase
- Use enumerated set for deletion detection (no extra filesystem access)
- Show progress with percentage, files/sec, elapsed time, and ETA
- Change "changed" to "changed/new" for clarity
Config improvements:
- Add tilde expansion for paths (~/)
- Use xdg library for platform-specific default index path
- Create comprehensive integration tests with mock S3 client
- Add in-memory filesystem and SQLite database support for testing
- Test full backup workflow including chunking, packing, and uploading
- Add test to verify encrypted blob content
- Fix scanner to use afero filesystem for temp file cleanup
- Demonstrate successful backup and verification with mock dependencies
