Commit Graph
96 Commits
Author SHA1 Message Date
sneak c6021ea494 Keep unreferenced blobs newer than prune_grace_period (closes #387)
check / check (push) Waiting to run
A backup uploads its blobs before the manifest that lists them, and the
process lock is per user per machine, so a prune from another host or
user sharing the destination deleted a running backup's blobs; that
backup still reported success and could not be restored.

Prune now deletes an unreferenced blob only once it is older than the new
config key prune_grace_period (default 24h), and reports how many it kept
(blobs_too_new under --json). The storage listing now carries each
object's modification time. remote nuke is unchanged. The README says what
the lock covers and that a shared destination relies on the grace period
or needs one prefix per host.

Model: opus-5-5
2026-10-14 08:04:05 +00:00
clawbot 5b8d84e38a Do not prune after a backup that skipped unreadable files (closes #382)
check / check (push) Waiting to run
snapshot create --skip-errors --prune pruned even when files were
skipped, so by default it deleted the previous snapshot of that name,
the only one still holding a file that had become unreadable, along
with blobs only that snapshot referenced.

The prune is now skipped exactly when the run exits non-zero for
entries --skip-errors could not read or list: older snapshots are
kept, a warning says why, and the run still exits non-zero. A file
removed during the run does not count, so such a run still prunes.
The README's --prune text and the flag's help say so.

Model: opus-5-5
2026-10-14 07:44:50 +02:00
clawbot 7a950ea2aa Exit non-zero when snapshot create --skip-errors skips anything it cannot read (closes #310)
check / check (push) Waiting to run
A backup under --skip-errors printed its skip errors, then "Finished
successfully.", and exited 0, so a cron wrapper could not tell it from
a complete backup.

Each skipped entry, including a directory the walk could not list, is
now counted as examined and failed. If --skip-errors skipped anything
it could not read or list, the run still keeps its snapshots and
prunes as before, then fails with "Backup completed, but N file(s)
could not be read". A file removed during the run, even before the
walk reaches it, counts once as failed and leaves the exit status
alone; without the flag it no longer stops the backup. Excluded
entries are never counted.

Model: opus-5-5
2026-10-14 07:13:48 +02:00
clawbot 16b03dfda5 Keep the local index when remote nuke cannot list or find the destination (closes #292)
check / check (push) Waiting to run
`remote nuke` took a destination store it could not list, or one
without `vaultik-destination.txt` while the local index held
snapshots, as a warning: it then cleared the local snapshot records,
and for an unplugged volume's empty mount point reported the store
empty and exited zero. Nuke now fails on either before it removes
anything, without its advice to run it again, which would fail the
same way. The bulk remove it runs fails on either too; only a
local-only remove still goes on with a warning. Nuke also deletes each
snapshot's remote metadata before its local record, so a failed delete
keeps the snapshot in the local index rather than leaving it on the
store as a remote-only snapshot.

Model: opus-5-5
2026-10-14 06:47:38 +02:00
clawbot a0de8bfdb7 Refuse a backup destination an unplugged volume left empty (closes #239)
check / check (push) Waiting to run
Every backup writes vaultik-destination.txt at the destination root.
While the local index holds snapshots stored there, a destination
without it is not present: snapshot create refuses before writing,
prune and snapshot purge fail, and snapshot list and snapshot remove
act as for a destination that cannot be listed. snapshot create and
snapshot remove check it first. remote nuke deletes the marker last.
The file:// backend never creates its destination directory, which
protects a directory on a removable volume, not a mount point.

Tests back up into directories that exist; fixtures for a written
destination carry the marker.

Judgement call: the marker is rewritten on every backup.
Deviation: for the complexity limit, two blocks moved, unchanged, into
snapshotNamesToCreate and confirmRemoveSnapshot.

Model: opus-5-5
2026-10-14 05:46:42 +02:00
clawbot 65036bba1b Make snapshot list report a changed destination instead of advising prune (closes #317)
check / check (push) Waiting to run
When the local index is bound to a different destination than the
configured one, snapshot list now gives the warning prune, snapshot
purge and snapshot remove refuse with: both destinations and the two
ways out (revert the configuration, or run vaultik database delete).
Below the table it replaces the local-only note and its prune advice.
With --json it goes to stderr through the logger, so stdout stays the
document, whose local rows show remote_present false.

Reading the binding moves out of EnsureStorageBinding into
storageBinding, which records nothing, so the read-only list can call
it without binding a fresh index.

Model: opus-5-5
2026-10-14 04:29:36 +02:00
clawbot 9fd88a34c7 Fail snapshot remove when a destination-only snapshot's metadata cannot be deleted (closes #340)
check / check (push) Waiting to run
When a snapshot was only on the destination store and deleting its
metadata there failed, snapshot remove warned and exited 0 having
removed nothing. That failure is now an error. A snapshot in the local
index keeps its behaviour: it is removed from the local index and the
failed delete on the destination store is a warning.

The README and the help text said the `vaultik prune` follow-up is
printed on every success; they now say it is printed only when metadata
was removed from the destination store, and never under --json. The
comments on RemoveSnapshot and removeSnapshotRemote describe both
cases; removeSnapshotRemote's also says it returns true only when the
delete finished without an error and removed at least one object.

Model: opus-5-5
2026-10-14 02:20:01 +02:00
clawbot 30285c7a8c Silence prune and snapshot removal status under --cron and --quiet (closes #294)
check / check (push) Waiting to run
PruneBlobs, CleanupLocalSnapshots, RemoveSnapshot, RemoveAllSnapshots
and snapshot purge wrote their status lines straight to stdout, so
`snapshot create --cron --prune` printed on every successful run and
--quiet did not silence `prune`, `snapshot purge`, `snapshot remove` or
`remote nuke`. Those lines now go through internal/ui, which drops them
under --cron and --quiet; warnings, errors and --json documents still
print.

The confirmation prompts still write straight to stdout, as the
`database delete` prompt does, together with what they ask to delete:
the snapshot list before the purge prompt, and the blob count and size
before the prune prompt. With --force those lines are status lines.

The two purge test fixtures gained the UI writer production always sets.

Model: opus-5-5
2026-10-14 01:10:43 +02:00
clawbot 2c087be577 Fail restore for a requested path that matches nothing (closes #309)
check / check (push) Waiting to run
snapshot restore said nothing about a requested path that matched no
entry in the snapshot (a typo, a relative path, or a path through a
symlinked snapshot path, whose entries are stored under the resolved
directory) and exited 0, even when nothing matched. Each such path is
now named in a warning as soon as the file list is built, so it shows
even when the restore then stops on an error, and restore returns an
error after restoring the paths that do match. The restore summary
moved into finishRestore to keep Restore within the length limit.

Making a path through a symlink match is not part of this change.
Judgement call: when the restore itself fails, that error is returned.

Model: opus-5-5
2026-10-13 23:14:22 +02:00
clawbot 37d525c322 Make remote nuke delete every object under metadata/ and blobs/ (closes #291)
check / check (push) Waiting to run
remote nuke deleted only the snapshot metadata and blobs the snapshot
and blob listings recognise, so a snapshot's database whose manifest
was never uploaded, and any name under blobs/ that is not a blob hash,
survived while nuke reported the store empty.

After removing every snapshot, nuke now deletes every object listed
under metadata/ and blobs/, whatever its name, in place of the blob
prune step. A delete that fails is logged and the others still run. A
listing afterwards decides the result: nuke reports the store empty
only when it finds nothing, and otherwise names each object left and
fails.

Judgement call: nuke no longer prints the blob prune summary.

Model: opus-5-5
2026-10-13 22:50:09 +02:00
clawbot f89fac34f7 Make config set create a list for an index under a missing key (closes #308)
check / check (push) Waiting to run
`config set age_recipients.0 age1...` on a file without `age_recipients`
wrote a map keyed `0`, after which no command could load the config. A
missing key is now created as a list when the next path element is a
number.

Before writing, `config set` checks the result with the config loader,
refusing only a failure the change brings in: a config that loaded must
still load; one that does not (still being filled in, or with a `${...}`
reference that cannot be filled in here) is checked on the changed
setting alone. The new `config.Parse`, split out of `config.Load`,
decodes config data, filling in references, without validating.

Rule suppressed: gosec G304 where `config.Load` and the new tests read a
file.

Model: opus-5-5
2026-10-13 22:43:41 +02:00
clawbot 3361e9ebd9 Record each snapshot's uncompressed size when its backup runs (closes #360)
check / check (push) Canceled after 0s
snapshot list summed the chunks a snapshot reaches through its file rows
in the local index. A file's row is shared by every snapshot that lists
it and rewritten when the file changes, so backing up a changed file
changed every older snapshot's UNCOMPRESSED SIZE.

The backup now takes that total when it records the snapshot's stats,
into a new snapshots column, uncompressed_chunk_size, and snapshot list
reads the column, as it already does for new_chunk_size.

Model: opus-5-5
2026-10-13 20:43:19 +02:00
clawbot fb8abcd049 Stop restore --verify re-reporting files that failed to restore (closes #323)
check / check (push) Canceled after 0s
With --skip-errors, a file that failed to restore was verified anyway,
failed again because it was never written, and was added to the same
failure list as the restore failure. restore --verify then listed it
twice and called every entry a checksum mismatch.

Verification now skips the files that failed to restore and keeps its
own failures in a separate list, so failed restores and failed
verifications are reported apart, each path once. The restore still
exits non-zero. The tests cover a restore with skipped files where
nothing fails verification, and one where another restored file also
fails it.

Judgement call: the second test has its own trailing-byte file-system
wrapper; the one in restore_perms_test.go is in the internal test package.

Model: opus-5-5
2026-10-13 20:12:48 +02:00
clawbot 740f474c59 Make prune and remote nuke fail when a delete fails (closes #290)
check / check (push) Canceled after 0s
prune, and the prune step of snapshot create --prune, counted a blob
it could not delete and still exited 0. It now prints its summary or
--json document and then returns an error, which replaces the
"Failed to delete N blob(s)" line.

remote nuke only warned when a snapshot's metadata could not be
deleted, advised vaultik prune, which never removes metadata, and then
reported the store empty. It now stops with an error, does not report
the store empty, and says to run vaultik remote nuke --force again. It
checks that the local index belongs to the configured destination
before deleting anything, and that error carries no such advice. A
successful nuke no longer advises prune.

Model: opus-5-5
2026-10-13 19:29:52 +02:00
clawbot bffd4578fc Record each snapshot's new chunk size when its backup runs (closes #311)
check / check (push) Successful in 12m7s
snapshot list worked out NEW CHUNK SIZE from the snapshot's file rows. A
file's row is shared by every snapshot that lists it and rewritten when
the file changes, so a snapshot whose change was a modified file showed
0 B, and older snapshots' figures moved after later backups.

The scanner now totals the size of the chunks it stores that were not
stored before (the chunks chunk_count counts), the backup writes that
total to a new snapshots column, new_chunk_size, and snapshot list reads
it.

Judgement call: the figure is fixed when the backup runs, so removing an
earlier snapshot no longer raises a later snapshot's figure.

Model: opus-5-5
2026-10-13 16:44:14 +02:00
clawbot 5f6455ba17 Refuse a relative file:// storage_url and decode its escapes (closes #299)
check / check (push) Canceled after 0s
The path after file:// was used as written. file://dest resolved
against the working directory, so a backup run from another directory
wrote elsewhere, exited 0 and left a snapshot that could not be
restored; my%20stick created a directory with that literal name.

Config load now refuses a file:// storage_url whose path is not
absolute. ParseStorageURL refuses it too and decodes percent-escapes;
a literal space still works. The rule is written in both packages
because config cannot import storage, which imports config.

A % not followed by two hex digits now fails when the storage is
opened, not at config load.

Model: opus-5-5
2026-10-13 11:27:59 +02:00
clawbot 35eed79b1a Check --keep-newer-than before the backup starts (closes #307)
check / check (push) Successful in 13m18s
snapshot create read --keep-newer-than only in the post-backup prune,
so a value the duration parser rejects failed the run after the
snapshot was uploaded: a nightly cron line added a snapshot each night,
reported it as failed, and never deleted any. Without --prune the flag
was ignored. The command now checks the flag before it starts and fails
on an invalid duration or a missing --prune.

A failed post-backup prune is now wrapped in ErrPostBackupPruneFailed
and reported as "Backup completed, but post-backup prune failed: ..."
rather than as a failed snapshot creation. The exit status stays 1.

Model: opus-5-5
2026-10-13 09:44:44 +02:00
clawbot 94412d9ef3 Stop remote info counting a non-blob file under blobs/ (closes #295)
check / check (push) Canceled after 0s
remote info counted every key under blobs/ with four or more path parts
as a blob, while prune skips a name that is not a blob hash. A stray
file such as .DS_Store was reported as an orphaned blob, with advice to
run prune, which then deleted nothing. remote info now takes its blob
list from prune's listAllRemoteBlobs, so its blob and orphan figures,
in the table and in --json, cover exactly what prune would consider.

blobKeyParts moves to prune.go, its only remaining user.

Model: opus-5-5
2026-10-13 08:33:27 +02:00
clawbot 167c0a0746 Make snapshot remove fail for an unknown snapshot ID (closes #293)
check / check (push) Successful in 12m4s
snapshot remove never checked that the ID existed: it deleted nothing,
reported the snapshot removed from the local database and from the
destination store, and exited 0, under --json, --dry-run and
--local-only too. It now looks the ID up in the local index and, only
when that has no row and --local-only is not set, on the destination
store; finding neither is an error. The command reports a removal from
the local database only when the local index had the row, and
remote_removed is true only when metadata was actually deleted.

Judgement call: under --local-only the destination store is not
consulted, so the snapshot must be in the local index.

Model: opus-5-5
2026-10-13 05:55:22 +02:00
clawbot 07c957f84b Bind the local index to the s3.* destination too (closes #298)
check / check (push) Successful in 15m35s
The local index was bound to its destination only through storage_url.
With the s3.* form nothing was checked, so after s3.bucket or s3.prefix
changed the next backup treated every chunk as already stored, uploaded
nothing, and completed a snapshot that could not be restored.

Without storage_url, the binding now records the destination the s3.*
fields name, written as an s3:// URL from s3.endpoint, s3.bucket and
s3.prefix, and a change is refused with the same error and advice. The
advice line now names the s3.* fields as well as storage_url.

Judgement call: the fields are compared as written, so an equivalent
spelling (a trailing slash on the prefix, a scheme on the endpoint) is
also refused, as it already is for storage_url.

Model: opus-5-5
2026-10-13 03:48:36 +02:00
clawbot 0d0368df81 Report unknown blob figures for a snapshot whose manifest cannot be read (closes #272)
check / check (push) Canceled after 0s
When remote info could not read a snapshot's manifest, the orphan
figures were unknown but the snapshot's row still gave 0 blobs and 0 B,
in the table and in --json. The row's blob count and blob size are now
unknown, and null in --json.

A directory with no manifest, as an interrupted backup leaves, still
shows 0: the orphan figures count its blobs as orphaned, so it
references none.

The constant holding the "unknown" text is renamed from countUnknown to
unknownText, since it now also stands for a size.

Model: opus-5-5
2026-10-08 08:46:11 +02:00
clawbot 70f008a21a Document that --json skips the confirmation prompt (closes #268)
check / check (push) Canceled after 0s
snapshot remove and prune delete without asking under --json, because a
prompt on stdout would break the JSON document. Their --json help and
README entries named only --force as skipping the prompt. Both now say
--json skips it too. One test checks the --json help of both commands;
another runs prune with --json, no --force and empty stdin, and checks
that the unreferenced blob is deleted and stdout holds only the JSON
document.

Model: opus-5-5
2026-10-08 04:59:30 +02:00
clawbot 32c4a46b49 Write rclone uploads under a temporary name and move them into place (closes #266)
check / check (push) Canceled after 0s
The rclone backend wrote each object straight to its key, so killing an
upload to a local or sftp remote left a truncated object there that the
next backup trusted.

On every remote with a server-side move, an object is now written under
a name ending in `.partial` and moved onto its key with rclone's
operations.Move, which removes an object already at the key first;
drive, dropbox and others will not move onto one. Listings skip
`.partial` names. Rclone's own copy also requires the PartialUploads
flag; this does not, because hdfs shows a file while it is written
without setting it. Remotes without a move are written in place.

Model: opus-5-5
2026-10-07 22:59:26 +02:00
clawbot d87202fb70 Correct doc and help sentences that are false about the code (closes #233)
check / check (push) Canceled after 0s
A blob is written whole to a temporary file and uploaded once finished,
not streamed to storage. The README, ARCHITECTURE.md and
config.example.yml now say a backup needs free temporary space for each
blob (twice that for an rclone destination that cannot stream uploads)
and for the metadata export's copies of the local index, in $TMPDIR, or
partly in /var/tmp when TMPDIR is unset. Also corrected: the snapshot ID
format, what restore reads and how incomplete snapshots are removed in
docs/DATAMODEL.md, what source_path holds, the index_path default, the
config search order in the snapshot create help, what snapshot remove
cleans up in the prune help, how the release installs Go, and the
script/release and script/fmt-check comments.

Model: opus-5-5
2026-10-07 17:59:27 +02:00
clawbot 7696f83258 Leave remote info orphan figures unknown when a manifest is unreadable (closes #228)
check / check (push) Canceled after 0s
When a manifest could not be read, remote info skipped it, counted that
snapshot's blobs as orphaned and advised running prune. The orphan
figures are now unknown in that case, with no prune advice; --json gives
them as null and lists the unreadable remote keys in
unreadable_manifests. Only a listed manifest.json.zst is read, so a
directory without one, as an interrupted backup leaves, keeps the
figures known.

Names under metadata/ were used unchecked and printed raw. A name that
is not a remote key is now skipped with a warning. A manifest under it
is not read either, so it also leaves the figures unknown; --json counts
such manifests in skipped_manifest_count.

Model: opus-5-5
2026-10-07 10:59:26 +02:00
clawbot cdc60c4dfa Write only the document to stdout from snapshot remove --json (closes #251)
check / check (push) Canceled after 0s
When the destination store cannot be reached, `snapshot remove` still
removes the snapshot from the local index and warns. The warning went
through the UI, which writes to stdout, so under `--json` it landed
ahead of the document and `| jq` failed on a command that exited 0.
Under `--json` the UI warning is now skipped; the logger's warning on
stderr carries the follow-up, with the snapshot ID as a field.

The follow-up, in the warning, the README and the command's help, said
`vaultik prune` would finish the cleanup, but `prune` never removes
snapshot metadata. They now say to run `vaultik snapshot remove` for the
snapshot again once the destination store is reachable.

Model: opus-5-5
2026-10-07 04:46:13 +02:00
clawbot 85d4ef118d Keep command output to the README's stdout and stderr rules (closes #224)
check / check (push) Successful in 16m49s
The startup banner moves from stdout to stderr, so a `completion`
script, a `config get` value and the hidden `__complete` command print
only their own output. `--quiet`, `--cron` and `--json` still suppress
it.

A failing `remote info`, `prune` or `snapshot remove` under `--json`
now reports its error on stderr. Their reporters returned early under
`--json`, so the failure reached neither stream.

`snapshot verify --quiet` writes no report. A failure is still
returned and printed on stderr, with the same exit status.

Judgement call: the banner's stream, posted on the issue for the owner.

Model: opus-5-5
2026-10-07 01:12:15 +02:00
clawbot 14fc4c9893 Load a config that has no age recipient (closes #221)
check / check (push) Successful in 13m47s
The README's steps for restoring on another machine failed at the first
command: `config init` wrote a placeholder recipient, and `config.Load`
rejects any recipient that does not parse. `config init` now writes an
empty `age_recipients` list, `config.Load` accepts an empty list, and
`snapshot create` refuses to start without a recipient. A malformed
recipient is still rejected at load.

The recovery-host test now builds its config with `config init` and
`config set` and reads it through `config.Load`, so it imports
`internal/cli`. On a fresh file, `config set age_recipients.0` writes the
list in flow style (`[age1...]`).

Model: opus-5-5
2026-10-06 15:46:13 +02:00
clawbot 66c80a70e3 Skip files of an unreadable blob under restore --skip-errors (closes #218)
check / check (push) Successful in 12m34s
A blob that failed to download ended `snapshot restore` even with
--skip-errors, after restoring whichever files came first. The download
error now goes through the same per-file handling as any other restore
error, once for every pending file that references the blob. With
--skip-errors those files are reported as failed, the rest are
restored, and the command still exits non-zero. Without the flag the
restore still aborts; the error now also names one affected file and
suggests --skip-errors. A cancelled restore still ends at once. The
--skip-errors help text and README now limit the packing and storage
caveat to snapshot creation.

Model: opus-5-5
2026-10-06 14:46:05 +02:00
clawbot 81f83b29f6 Re-vendor the canonical files from sneak/prompts at dd4027b (closes #213)
check / check (push) Successful in 12m47s
Linting and testing become the lint and test phases of the Dockerfile,
and the build stage depends on both. Dockerfile.lint, CHECK_EPOCH and
the tests that checked them are removed. Every docker build in script/
passes --no-cache, and script/cibuild runs script/bootstrap first. A
host without Go gets the go.mod version from script/install-go in
.tool/go, which bootstrap, the Makefile, fmt, fmt-check, precommit and
release add to PATH; fmt-check skips .tool. The image takes its version
from the VERSION build arg or git describe, dev without .git. This
repo's own entries follow the canonical content in .gitignore and
.editorconfig. The golangci-lint v2.14.0 findings are fixed. The rules
in CLAUDE.md move into AGENTS.md. IsDevVersion counts "unknown".

Model: opus-5-5
2026-10-06 12:46:13 +02:00
clawbot 35cf985c18 Re-chunk a known file whose chunks no uploaded blob holds (closes #214)
check / check (push) Successful in 6m53s
check / check (pull_request) Successful in 6m17s
File rows are shared by every snapshot and updated in place, while a
blob row is deleted once no snapshot references it. Removing the newest
snapshot, or the prune after an interrupted run, could drop the only
blob holding a changed file's current chunks while an older snapshot
kept the file row. The next backup compared metadata only, skipped the
file, and completed a snapshot that could not restore it.

The scanner now loads the IDs of known files that list a chunk no
uploaded blob holds and re-chunks them even when their metadata is
unchanged.

The tests append to a file, so the file keeps its first chunk in a blob
the first snapshot still references. Each backup run gets its own
snapshot name, so the second-precision snapshot IDs differ without
sleeping.

Model: opus-5-5
2026-10-06 04:46:16 +02:00
clawbot 070090124a Stamp the tag or short commit in a plain docker build (closes #211)
check / check (push) Successful in 3m35s
check / check (pull_request) Successful in 3m16s
A plain `docker build .` stamped `dev`: `.dockerignore` left out `.git`
and the Dockerfile defaulted VERSION to `dev`. `.dockerignore` now
sends `.git` without `.git/config`. Given no build arguments, the
builder stamps `git describe --tags --always` and the commit and date
from git, and fails if `.git` is present but yields no version. The
empty CHECK_EPOCH refusal is gone so the plain build succeeds.
`script/version` now prints `git describe --tags --always --dirty`, so
make, the scripts and a plain build agree. `vaultik version` treats
the short commit, tag-N-gHASH forms and any version ending in `-dirty`
as development builds, so they keep the development-build notice.

Model: opus-5-5
2026-10-02 10:04:14 +02:00
clawbot 584444b619 List only after-1.0 work in the README roadmap (closes #208)
check / check (push) Successful in 3m35s
check / check (pull_request) Successful in 3m36s
The README roadmap and the TODO.md Next Step still described finished
1.0 work as remaining. The roadmap now lists only work planned after
1.0. Its security item says the code was reviewed before 1.0, every bug
found was fixed, and the accepted risks are listed; an outside audit
stays as after-1.0 work. The error-condition item is gone because every
failure case it listed has a fault-injection test. Daemon mode is added.
TODO.md says the 1.0 work is complete on next and that merging and
tagging are the owner's.

Judgement call: dropped the human-readable size flags item; no command
flag takes a raw-integer size.

Model: opus-5-5
2026-10-01 21:41:41 +02:00
clawbot 6e1f499048 Document that migrations are supported and none are added before 1.0 (closes #68)
check / check (push) Successful in 3m6s
check / check (pull_request) Successful in 3m8s
The docs now say vaultik supports migrations. The numbered files in `internal/database/schema/` are migrations: `schema_migrations` records which have run, and opening a database applies any that have not. None are added before 1.0 because nothing is installed anywhere yet, so a schema change edits `001.sql` directly. After 1.0 each change is a new numbered file, and an existing local database is migrated when vaultik is updated.

`docs/DATAMODEL.md` owns the explanation. The README caveat and roadmap entry and `AGENTS.md` policy 13 link to it. This replaces the wording from #146, which said there was no upgrade path.

Disclosure: `CLAUDE.md` line 33, the owner's file, changes from "do not need to support migrations" to "do not add migrations before 1.0".

Model: opus-5-5
2026-09-28 20:21:38 +02:00
clawbot eed117fe25 Route direct-stdout command output through internal/ui (closes #149)
check / check (pull_request) Successful in 1m27s
check / check (push) Successful in 3m18s
version, info, remote info, config, and database delete wrote plain text straight to stdout, so they were unstyled and ignored --quiet. Output is now governed by internal/ui in two buckets. Status lines and confirmations (config init, config set, the database delete prompt) go through the ui message methods and are silenced by --quiet. The data a command exists to produce is written plain -- the version/info/remote-info reports, the snapshot list table, config get values, and the --json documents -- and is never suppressed, since a script depends on it and a marker would corrupt a table or document. The database delete confirmation prompt is always shown. Pure-cli commands reach ui through a small commandUI helper.

Model: opus-4-8
2026-09-22 20:28:50 +02:00
clawbot 1548c0f933 Correct the security claims in docs and comments, and record the accepted risks (closes #171)
check / check (pull_request) Successful in 1m58s
check / check (push) Successful in 2m49s
Docs and comments only; no behaviour change. Corrects ten overclaims the security review found: snapshot names are hashed but the hash uses no secret, so a guessed hostname and name can be confirmed; a blob is named by hex(SHA256(SHA256(uncompressed contents))), stated once in docs/REPOSTRUCTURE.md and referenced elsewhere; double hashing does not hide known content (blob packing does); age uses ChaCha20-Poly1305, not XChaCha20; encryption is required, not optional; a snapshot is marked complete before its metadata is uploaded; the export comment now matches its only caller; deep verify detects corruption, not authorship; adding a recipient does not reach existing data; restore examples target a user-owned directory.

Adds an Accepted Risks subsection under Security Considerations with the seven documented risks, cross-referenced from the README.

Model: opus-4-8
2026-09-22 19:28:31 +02:00
clawbot d88ed64489 Parse the age identity key once and accept every identity in it (closes #165)
check / check (pull_request) Successful in 2m41s
check / check (push) Successful in 2m56s
Restore and verify --deep now parse the configured age secret key a single time through a new helper that uses age.ParseIdentities and hands every identity to age.Decrypt. A key file with several identities (a whole age-keygen file) is fully accepted, so a blob encrypted to any of its recipients decrypts, not just the first.

The helper is the first step of both commands, so a missing or unparseable key fails before anything is downloaded. Its error names the config source and never echoes the key value. config.extractAgeSecretKey and its silent fallback are removed; the key is stored raw and parsed only where decryption happens. README, the restore help, and the missing-key error now read the key from a file with \$(cat ...) rather than typed literally, keeping it out of shell history.

Model: opus-4-8
2026-09-22 15:45:27 +02:00
clawbot 7e611b95db Check blob sizes and the database in shallow verify (closes #169)
check / check (push) Successful in 1m21s
check / check (pull_request) Successful in 2m37s
Shallow snapshot verify only checked that each blob object existed and then reported "All blobs verified", overstating what it did.

It now compares each blob stored size against the manifest compressed_size, using the same comparison as the deep path, and checks that the snapshot encrypted database (db.zst.age) is present. A blob of the wrong size no longer counts as verified. The final line reports only what was checked: presence and size, not contents.

The README verify description and the CLI short/long text are corrected to match. Removed the now-unused resolveAndDownloadManifest helper and errBlobsMissing sentinel.

Model: opus-4-8
2026-09-22 14:28:44 +02:00
clawbot b4654f8e52 Abort the run when packing fails, even under --skip-errors (closes #161)
check / check (push) Successful in 1m22s
check / check (pull_request) Successful in 3m2s
A chunk is registered as pending (known, scanner-pending, packer pending-row) before it is packed. Under --skip-errors the scanner skipped a file on any processing error, including a failure inside addChunkToPacker (packing, database, encryption, upload). The pending chunk then stayed queued and a later blob finalize inserted it into the chunks table with no blob_chunks row, so a snapshot could complete holding a file whose chunk is in no blob and cannot be restored.

Errors from addChunkToPacker are now marked and abort the run regardless of --skip-errors; only open and read errors are skipped. The bookkeeping order is unchanged. Flag help and comments now say only unreadable files are skipped.

Model: opus-4-8
2026-09-22 12:28:44 +02:00
clawbot 4c83e82543 Reject a blob_size_limit below the largest possible chunk (closes #167)
check / check (push) Successful in 1m20s
check / check (pull_request) Successful in 1m16s
Validate only rejected blob_size_limit below chunk_size, but the chunker can emit chunks up to chunk_size times the FastCDC size spread (four times), and the packer puts a single chunk of any size into an otherwise empty blob. A limit between one and four times chunk_size therefore let a blob reach four times the configured maximum, with most blobs holding a single chunk and so exposing individual chunk lengths to anyone who can list the destination.

Validate now rejects blob_size_limit below chunk_size times the spread, reusing the chunker's one constant (now exported as ChunkSizeSpread) instead of a second literal. The rule is stated in the error text, the Validate comment, the README config table, config.example.yml, and the generated config template.

Model: opus-4-8
2026-09-22 11:45:41 +02:00
clawbot 3abe9cbd9e Scope the PID lock to mutating commands (closes #150)
check / check (push) Successful in 1m19s
check / check (pull_request) Successful in 1m16s
RunWithApp took the process-wide PID lock for every fx-backed command, so read-only commands (info, snapshot list, snapshot verify, remote info) failed with "already running" while a backup held it.

AppOptions now carries a lockMode declared at each call site: only mutating commands (snapshot create, snapshot purge, snapshot remove, prune, remote nuke) acquire the lock; read-only ones run without it. snapshot restore is classified read-only -- it writes only to its target directory, not the local index or remote store. The decision moves to a small acquireLockIfMutating helper, with a test that a read-only command runs while the lock is held and two mutators still exclude. The README locking section is rewritten to match.

Model: opus-4-8
2026-09-22 11:01:29 +02:00
clawbot 42f4e648d7 Reconcile docs with code across the accuracy-sweep items (closes #74)
check / check (pull_request) Successful in 2m38s
check / check (push) Successful in 4m22s
Docs-only sweep of the accuracy items. Corrected ARCHITECTURE.md chunk sizes and the fx config type; documented the ls/rm aliases, the CPU/MEM profile env vars, the age_secret_key threat-model caveat, the four zstd presets, and a new locking section for the process-wide PID lock. Narrowed the internal/ui output claim to what holds today (refactor deferred to #149); lock-scoping deferred to #150. Added the missing ARCHITECTURE.md and config.example.yml README links. Every claim re-verified against the tree.

Disclosure: a pre-existing gomodguard linter deprecation surfaced during the gate; unrelated.

Model: opus-4-8
2026-09-22 09:05:36 +02:00
clawbot 343129f891 Accept a remote key for restore and verify, and document it (closes #124)
check / check (push) Failing after 1s
check / check (pull_request) Failing after 1s
A machine restoring after the original is gone has no local index and cannot know a snapshot's human ID; snapshot list shows such snapshots only by their remote key, but restore and verify accepted only the human ID, so recovery could not be done as documented.

Restore and verify now also accept a remote key, or an unambiguous leading part of it as snapshot list prints it, resolved against the store's metadata listing. Human IDs are never pure hex, which tells the two forms apart. Deep verify reads the single snapshot in the downloaded per-snapshot database. A new README section walks the recovery end to end; a test backs up, then lists, restores and deep-verifies with an empty index, another hostname and no age_recipients.

model: claude-opus-4-8 (implementation, review); claude-fable-5-1 (merge)
2026-09-22 01:01:25 +02:00
clawbot aab6a87f8c Reconcile the schema/migration docs with the code (closes #68)
check / check (pull_request) Failing after 1s
check / check (push) Successful in 2m46s
Four documents told different stories about the database schema. docs/DATAMODEL.md now owns the explanation and separates two things: the policy, which is unchanged (no supported upgrade path between versions; delete the local index with vaultik database delete and back up again), and the schema bootstrap that does exist (numbered files in internal/database/schema applied to a fresh database and recorded in schema_migrations).

README.md and AGENTS.md are reworded to match and link there. AGENTS.md names the real file to edit, internal/database/schema/001.sql, and notes that the pre-1.0 disposability clause expires on tagging. No code changed.

Judgement call: REPO_POLICIES.md still names a different schema file; it is cross-project policy and was left alone.

model: claude-opus-4-8 (implementation, review); claude-fable-5-1 (merge)
2026-09-21 21:58:37 +02:00
clawbot 89ebfc78e2 Use one duration parser and fix the --older-than months example (closes #123)
check / check (push) Failing after 1s
check / check (pull_request) Failing after 1s
Two parseDuration functions existed with different grammars; only the one in internal/vaultik/helpers.go was reachable from a flag. The unused copy in internal/cli/duration.go is deleted, so no flag accepts anything it did not before.

The README gave 6m as the six-months example for snapshot purge --older-than, but m is minutes: that command removed every snapshot older than six minutes. The example is now 6mo, and the help for --older-than and --keep-newer-than states that m is minutes and mo is months.

The parser now rejects negative durations, which it used to accept or silently make positive.

model: claude-opus-4-8 (implementation, review); claude-fable-5-1 (merge)

Co-authored-by: clawbot <clawbot@noreply.example.org>
2026-09-21 20:58:30 +02:00
clawbot 75a10d3a22 Hash-verify the Go toolchain in the release workflow (closes #105)
check / check (push) Failing after 0s
check / check (pull_request) Successful in 4m13s
The release workflow installed Go with actions/setup-go, which pins the action but not the Go archive it downloads, so the compiler that builds the published binaries was verified against nothing in this repo.

New script/install-go, modelled on script/install-goreleaser, downloads the go.dev archive for the version in go.mod and refuses it unless its sha256 matches the value committed in the script. It fails if its version disagrees with go.mod, and on any OS or architecture other than the Linux release runners. GOTOOLCHAIN=local on the release step keeps the verified toolchain from switching itself.

Judgement call: release path only; script/bootstrap still uses the host Go.

model: claude-opus-4-8 (implementation, review); claude-fable-5-1 (merge)
2026-09-21 19:48:52 +02:00
clawbot 3d56dd7eb0 VACUUM snapshot metadata through the sqlite driver, not a CLI (closes #120)
check / check (push) Failing after 1s
check / check (pull_request) Successful in 2m57s
snapshot create compacted the metadata database by running a sqlite3 command-line binary, after every blob had already been uploaded. On a host without that binary, which includes anyone who installed with go install, the backup failed at the last step, and two tests failed the same way.

VACUUM now runs through the Go sqlite driver the program already uses, and its error is returned to the caller. The runtime Docker image no longer installs the sqlite package, since nothing in the binary calls it.

model: claude-opus-4-8 (implementation, review); claude-fable-5-1 (merge)
2026-09-21 19:41:28 +02:00
clawbot 753bc3ef60 Correct remote layout and privacy docs for hashed snapshot keys (closes #67)
check / check (pull_request) Failing after 1s
check / check (push) Successful in 3m11s
Three documents showed the remote layout with a plaintext snapshot ID as the metadata directory name, and docs/REPOSTRUCTURE.md blamed those IDs for the observable backup time. The store actually names each metadata directory with a one-way hash of the ID, so hostname and snapshot name are not visible; the backup time is, through the plaintext timestamp in the manifest, which is accepted behaviour.

README, ARCHITECTURE.md, docs/DATAMODEL.md and docs/REPOSTRUCTURE.md now show the hashed layout, the derivation is documented once, and the privacy section lists what the unencrypted manifest exposes. Two code comments that claimed the timestamp was hidden are corrected. No behaviour change.

Judgement call: docs/DATAMODEL.md was not named in the issue but had the same error.

model: claude-opus-4-8 (implementation, review); claude-fable-5-1 (merge)
2026-09-21 19:24:44 +02:00
clawbot d2a0510cb4 Trigger CI on next, not only main (closes #122)
check / check (pull_request) Failing after 0s
check / check (push) Successful in 3m33s
`check.yml` ran only on push to `main` and on pull requests against `main`. Every unit is a PR based on `next`, and `next` is pushed on each squash-merge, so no unit PR and no push to `next` ever ran CI; a broken `next` would first surface on the milestone PR. `next` is added to both branch lists; nothing else in the workflow changes. The README Entrypoints section now says where CI runs.

Disclosure: the CI run on the PR itself fired (the proof the trigger works) but was red because the runner had no disk space left before any check step ran; the local gate was green.

Model: opus-4-8 (implementation and review)
model: claude-fable-5
2026-09-21 14:55:59 +02:00
sneak d257f8f658 Lint in a container as a build step, via Dockerfile.lint (closes #113)
check / check (pull_request) Successful in 2m58s
Every lint run now happens inside its own container, invoked through
script/lint, and linting is a build step rather than a container
command: a successful build of the new root Dockerfile.lint IS a clean
lint. That shape also works where the docker daemon is remote and bind
mounts are impossible.

Its FROM line -- golangci/golangci-lint:v2.12.2, pinned by digest -- is
now the only pin of the linter version in this repo.

A container per run has its own lint cache and its own golangci-lint
lock, both discarded with it, so neither cross-worktree contamination
nor lock contention exists any more. The machinery that defended
against them is therefore gone: the per-worktree cache directories, the
lock-retry loop, and script/lint-audit, which existed to catch findings
replayed from a cache that no longer exists. So is the host lint path
in its entirety -- the native escape hatch, its version detection, and
VAULTIK_LINT_IN_CONTAINER in both script/lint and the Dockerfile.
Nothing lints on the host, at any version.

A cached build lints nothing, so the CHECK_EPOCH mechanism the product
Dockerfile already used is what makes a green mean something:
ARG CHECK_EPOCH with no default, placed below the module layers so
dependency caching survives, a `RUN [ -n "$CHECK_EPOCH" ] || exit 1`
guard so a build that withholds the arg fails instead of replaying, and
the value expanded into the lint command itself. script/lint computes
`epoch="$(date +%s%N)$$"` as a bare assignment on its own line, because
inline in the argument a failing substitution does not abort under
`set -eu` and yields a constant empty epoch -- which is exactly the
false green being prevented.

The product Dockerfile loses its lint stage rather than gaining a
second linter pin. That stage ran `make lint`, which is now
`docker build`: docker-in-docker inside a BuildKit step with no daemon.
Calling golangci-lint directly there instead would have meant two
independently bumpable digests for one tool. `make fmt-check` moves
beside `make test` in the builder stage, and script/cibuild now builds
Dockerfile.lint and then Dockerfile, each with its own fresh epoch,
failing on either. Consequence, stated in comments rather than left to
be discovered: script/docker builds the product image only and no
longer lints; script/check and script/cibuild are the gates.

`golangci-lint config verify` runs as its own epoch-keyed layer, above
the lint. It is not belt-and-braces: `golangci-lint run` rejects a
config it cannot PARSE but silently IGNORES an unknown top-level KEY.
Renaming .golangci.yml's `linters:` to `linterz:` -- one character --
discards `default: all`, the disable list and every threshold, leaves
only the small default linter set running, and exits 0 reporting
`0 issues.` on a tree the real config fails with an lll finding, in a
run whose lint layer demonstrably executed. That is a set-but-
ineffective config falling back to defaults instead of failing loudly,
sitting in the gate's own configuration. `config verify` catches it and
does so with the network genuinely off at this pin: under
`docker run --network none` against the pinned digest it exits 0 on
this repo's config and exits 3 on the `linterz:` variant. It is keyed
on CHECK_EPOCH like the lint itself, because a cached validation
validates nothing.

script/lint-fix is kept, reimplemented as a bind-mounted
docker run against the image parsed out of Dockerfile.lint -- a build
step cannot write fixes back to the worktree -- and its header states
outright that it is a developer convenience, never a gate, and needs a
local daemon.

cmd/vaultik/lintdocker_test.go parses both Dockerfiles and both scripts
and fails if any part of the mechanism is dropped: the digest pin, the
defaultless ARG below `go mod download`, the emptiness guard, the
expansion of the epoch into each check command, the bare per-invocation
epoch assignment in both scripts, cibuild building both files, the
config verification running before the lint, and -- structurally, not
by searching for one retired variable name -- that no script invokes
golangci-lint except through docker. Every one of those losses is
silent: the build still exits 0 and nothing is checked, which is why
they are asserted rather than trusted. The scanner behind the last of
those has its own test, because a structural check that goes blind
passes on every tree, including a broken one.

script/lint takes no arguments now, and says so instead of dropping
them: a build step has no command line to pass linter flags to.
2026-08-10 13:38:45 +00:00