The CLI had two commands named "prune" doing different jobs (local
DB orphan cleanup vs. remote blob garbage collection), which was
confusing and forced a manual two-step workflow after deleting any
snapshot.
Single user-facing prune surface is now `vaultik prune`, which calls
PruneDatabase (local orphan cleanup) then PruneBlobs (remote unref
blob GC). Snapshot deletion paths (snapshot remove, snapshot remove
--all, snapshot purge) auto-run CleanupOrphanedData inline so the
local index database doesn't accumulate ghost rows after every
removal — the user observed ~39k orphaned files and 2 orphaned blobs
after a remove --all because that cleanup was previously a separate
opt-in command. `snapshot prune` is removed.
Also addresses the doc/help-string drift the user audit caught:
* cli/prune.go help text used to reference a non-existent
`vaultik purge` command.
* cli/config.go get/set short/long examples were S3-specific
(s3.bucket) when the primary storage configuration is
storage_url.
* vaultik/info.go printed S3 Bucket/Endpoint/Region labels
unconditionally; for file:// or rclone:// users those rows
were empty. The Storage Configuration block now prints the
storer's Type+Location first, the storage_url string when set,
and only emits S3 rows that are actually populated.
* vaultik/info.go's "Run 'vaultik prune --remote'" hint
referenced a flag that doesn't exist.
* vaultik/blobcache.go's doc comment claimed LRU eviction, which
is no longer the restore-time policy (the sweeper drives
eviction; LRU is the safety-net fallback when maxBytes is
finite).
* README.md listed `vaultik restore`, `vaultik snapshot prune`,
and `s3.bucket` example, all out of date.
README's roadmap section is rewritten with concrete pre-1.0 items
(security audit, error-condition tests, parallel blob downloads,
restart of interrupted restore, …) so the next-steps surface
matches what the project actually still needs.
The cleanup calls are guarded against a nil SnapshotManager so
tests that construct a bare Vaultik struct continue to work.
Renames the top-level `restore` command to `vaultik snapshot restore`
for consistency with `vaultik snapshot create`. The factory follows the
sibling pattern (newSnapshotRestoreCommand) and its file is renamed to
snapshot_restore.go to match.
remote nuke: new subcommand that deletes every snapshot's metadata and
every blob from remote storage, leaving the bucket prefix empty.
Requires --force.
User-facing 'Processing' is now 'Backing up' everywhere it referred to
the chunking/upload phase. Files summary line says 'backed up' instead
of 'processed'.
ui.Speed now formats bytes/sec input as bits/sec output (bit/s, Kbit/s,
Mbit/s, Gbit/s). Network transfer rates are conventionally expressed
in bits — the per-blob heartbeat now matches the per-snapshot summary
line which has always been bits/sec.
restore aborts on the first per-file failure by default, surfacing
the file path and the underlying error and suggesting --skip-errors
to continue past failures.
--skip-errors moved from a 'snapshot create' subcommand flag to a
top-level persistent flag on the root command. It applies to both
snapshot create and restore. Old 'vaultik snapshot create --skip-
errors' still works because persistent flags are inherited.
- New ui.Detail method for indented continuation lines under a
preceding Complete (visually same as Progress: " 》" in white).
- Snapshot summary lines (Files/Data/Storage/Upload/Duration) are
now Detail lines indented under "Created snapshot X.".
- Local index database prune complete result lines (incomplete
snapshots, orphaned files/chunks/blobs) are also Detail lines
under a clean Complete header.
- "Files: ... to process" → "Files: ... processed" (they have been
processed by the time we emit the summary).
- "Data: ... (... to process)" → "Data: ... (... processed)".
- ui.Writer now tracks warning and error counts emitted; Vaultik
prints "Finished successfully." or "Finished (with N warnings)."
as the final line of CreateSnapshot.
Progress lines now use the form:
..., <subject> elapsed: <dur>, <subject> ETA: <time> (est remain <dur>).
ui.Time formats same-day times as HH:MM:SS and other-day times as
YYYY-MM-DD HH:MM:SS, with no timezone suffix (local time is implied).
The local-index-database prune complete line now shows remaining
counts for each category:
... 1 incomplete snapshots removed (3 remain), 3783 orphaned files
removed (42 remain), ...
❌ is a thin black-and-white cross that gets lost against terminal
backgrounds and the ANSI red text. 🛑 is a solid red octagon that
reads unmistakably as 'stop/error' at a glance, even when the user
isn't reading the line carefully.
All user-facing output now goes through a single ui.Writer with a
uniform style:
》 (white) for begin / info / notice
》 (green) for complete / success
Warning: for warnings (orange)
ERROR: for errors (red)
》 (indented) for progress heartbeats
Color is enabled when stdout is a TTY and NO_COLOR is unset.
Standards:
- Complete-sentence messages with fully qualified terms ("backup
destination store", "local index database", "snapshot source
files enumeration").
- Every Complete has a matching Begin.
- Natural verb tense conveys state ("Uploading" -> "Uploaded"). The
words "begin"/"complete" never appear in message bodies; the marker
color carries that information.
- ETA means clock time, not duration. Progress lines say "estimated
remaining time (<dur>), finish at <time>" with both labeled.
Adds globals.CommitDate (populated by Makefile/Dockerfile/goreleaser
via ldflags from `git show -s --format=%cI HEAD`) and a startup banner
printed once per invocation.
Strips fx call-chain noise from startup errors so users see the actual
underlying error (e.g. "creating base path: mkdir /Volumes/BACKUPS:
permission denied" instead of three layers of "could not build
arguments for function ...").
README documents the output style and the ui package conventions.
When the scanner hits a permission-denied error (TCC-protected
directories on macOS without Full Disk Access, or any other EPERM),
the error now names the offending path and includes platform-specific
remediation instructions. On macOS it points the user at System
Settings -> Privacy & Security -> Full Disk Access. On other
platforms it suggests --skip-errors.
The error wraps os.ErrPermission so errors.Is still works for callers
that care about the underlying error.
README quickstart and snapshot create docs now mention the macOS FDA
requirement.
Module path changed from git.eeqj.de/sneak/vaultik to
sneak.berlin/go/vaultik (vanity redirect). All imports, ldflags,
Dockerfile, goreleaser config, and docs updated. App data/config
directories now use plain "vaultik" instead of the reverse-DNS name.
README:
- New copy-pasteable quickstart at top: go install, config init,
age keypair, config set for key + file:// destination, home backup
- All command names in command details are code-quoted
- config set/get gained sequence index support (age_recipients.0)
so lists are settable from the CLI
- Dockerfile build is CGO_ENABLED=0 to match the pure-Go build
The config command group manages the config file:
config init - write default config (moved from top-level init)
config edit - open the config in $EDITOR (falls back to vi)
config get - print a value by dotted YAML path (s3.bucket)
config set - set a scalar value by dotted YAML path
get/set operate on the yaml.Node tree so comments and formatting in
the config file are preserved across edits. set creates intermediate
maps as needed.
New init command writes a default config file with commented
explanations for every setting. Uses XDG config directory via
github.com/adrg/xdg for platform-appropriate paths:
macOS: ~/Library/Application Support/vaultik/config.yml
Linux: ~/.config/vaultik/config.yml
root: /etc/vaultik/config.yml
Config resolution now searches the XDG path before /etc/vaultik/.
Refuses to overwrite an existing file. Created with 0600 permissions.
README quickstart rewritten as a single copy-pasteable shell block
walking through install, keygen, init, edit, first backup, verify,
and cron setup.
--cron now sets Vaultik.Stdout to io.Discard so all user-facing output
is suppressed, not just the scanner progress. Errors still go to stderr
via the structured logger.
snapshot list now warns when local snapshot records have no matching
remote metadata, and suggests 'vaultik snapshot cleanup' instead of
silently deleting them.
snapshot cleanup is a new subcommand that explicitly removes stale
local snapshot records. syncWithRemote (used by purge) still does
this automatically since purge is already destructive.
.gitignore changed from 'vaultik' to '/vaultik' so it only matches
the binary at the repo root, not the internal/vaultik/ directory.
snapshot create --prune now accepts --keep-newer-than <duration> (e.g.
4w, 30d, 6mo) to keep a rolling window of snapshots instead of only
the latest. Supports d/w/mo/y units and combinations (2w3d).
Without --keep-newer-than, --prune still defaults to keep-latest-only.
README now covers: storage backends (s3/file/rclone), all CLI commands
with full flag docs, configuration reference table, architecture overview,
roadmap (post-1.0 only), and development workflow.
TODO.md removed — completed items dropped, remaining roadmap items
merged into README.
ARCHITECTURE.md updated: correct snapshot ID format, storage.Storer
instead of s3.Client, binary SQLite export instead of SQL dump.
The --daemon flag, BackupInterval, FullScanInterval, MinTimeBetweenRun
config fields, and DirtyPath model were placeholders for a never-shipped
daemon mode and have been removed. Daemon mode is out of scope for 1.0.
- Add internal/types package with type-safe wrappers for IDs, hashes,
paths, and credentials (FileID, BlobID, ChunkHash, etc.)
- Implement driver.Valuer and sql.Scanner for UUID-based types
- Add `vaultik version` command showing version, commit, go version
- Add `--verify` flag to restore command that checksums all restored
files against expected chunk hashes with progress bar
- Remove fetch.go (dead code, functionality in restore)
- Clean up TODO.md, remove completed items
- Update all database and snapshot code to use new custom types
- Implement exclude patterns with anchored pattern support:
- Patterns starting with / only match from root of source dir
- Unanchored patterns match anywhere in path
- Support for glob patterns (*.log, .*, **/*.pack)
- Directory patterns skip entire subtrees
- Add gobwas/glob dependency for pattern matching
- Add 16 comprehensive tests for exclude functionality
- Add snapshot prune command to clean orphaned data:
- Removes incomplete snapshots from database
- Cleans orphaned files, chunks, and blobs
- Runs automatically at backup start for consistency
- Add snapshot remove command for deleting snapshots
- Add VAULTIK_AGE_SECRET_KEY environment variable support
- Fix duplicate fx module provider in restore command
- Change snapshot ID format to hostname_YYYY-MM-DDTHH:MM:SSZ
- Changed blob table to use ID (UUID) as primary key instead of hash
- Blob records are now created at packing start, enabling immediate chunk associations
- Implemented streaming chunking to process large files without memory exhaustion
- Fixed blob manifest generation to include all referenced blobs
- Updated all foreign key references from blob_hash to blob_id
- Added progress reporting and improved error handling
- Enforced encryption requirement for all blob packing
- Updated tests to use test encryption keys
- Added Cyrillic transliteration to README
- Change all commands to use flags (--bucket, --prefix, etc.)
- Add --config flag to backup command
- Support VAULTIK_CONFIG environment variable for config path
- Use /etc/vaultik/config.yml as default config location
- Add test/config.yaml for testing
- Update tests to use environment variable for config path
- Add .gitignore for build artifacts and local configs
- Update documentation to reflect new CLI syntax
- Set up cobra CLI with all commands (backup, restore, prune, verify, fetch)
- Integrate uber/fx for dependency injection and lifecycle management
- Add globals package with build-time variables (Version, Commit)
- Implement config loading from YAML with validation
- Create core data models (FileInfo, ChunkInfo, BlobInfo, Snapshot)
- Add Makefile with build, test, lint, and clean targets
- Include minimal test suite for compilation verification
- Update documentation with --quick flag for verify command
- Fix markdown numbering in implementation TODO
- Expand README with full CLI documentation, architecture details, and features
- Add comprehensive 87-step implementation plan to DESIGN.md
- Document all commands, configuration options, and security considerations
- Define complete API signatures and data structures
