sneak/vaultik
1
0 Fork 0
Code Issues 13 Pull Requests 5 Actions Packages Projects Releases Wiki Activity
Files
c7aee9456618750b26761b2a210f23b7867fc441
vaultik/docs/TODO.md
T
sneak c7aee94566
check / check (push) Failing after 18m51s
Details
Move documents to docs/ and the example config to configs/ (closes #411) 
ARCHITECTURE.md and TODO.md move to docs/, config.example.yml to
configs/, as REPO_POLICIES.md asks of the root. Links, the
no-snapshots error message and the example-config test follow them.

Model: opus-5-5
2026-11-10 20:02:10 +00:00

1215 lines
76 KiB
Markdown
Raw Blame History

# Workflow
- branch (from `main`)
- do the work in Next Step
- move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work)
- merge to `main` if the branch is not protected, otherwise open a PR
- push
# Status
pre-1.0
# Next Step
The 1.0 work is complete on `next`: the scope settled on
[issue #125](https://git.eeqj.de/sneak/vaultik/issues/125) has landed, and so
has every fix from the defect hunt before the release candidate
([issue #288](https://git.eeqj.de/sneak/vaultik/issues/288)). What is left is
merging `next` to `main` and tagging the release, both the owner's; a `v*` tag
runs the release build in `.gitea/workflows/release.yml`.
# Completed Steps
- 2026-11-10: `script/fmt` and `script/fmt-check` now format and check all
Markdown with `prettier`
([issue #411](https://git.eeqj.de/sneak/vaultik/issues/411)). The architecture
document and this file moved to `docs/`, and the example config to `configs/`.
- 2026-10-15: Stopped `snapshot create --prune` pruning after a backup in which
a volume mounted below a configured path in an earlier snapshot of the same
name was not mounted
([issue #395](https://git.eeqj.de/sneak/vaultik/issues/395)). Such a prune
could delete every snapshot still holding the volume's files. Each snapshot
now records in the local index the directories it found on a different
filesystem from their parent and found an entry in that the backup does not
exclude.
- 2026-10-14: Stopped `snapshot create --prune` pruning after a backup that
found nothing in a configured path
([issue #388](https://git.eeqj.de/sneak/vaultik/issues/388)), as in the mount
point of a volume that is not mounted. Such a prune could delete every
snapshot still holding that path's files. A configured path that is a symlink
whose target is gone, as one into such a volume, now fails the backup, with or
without `--prune`.
- 2026-10-14: Stopped `snapshot create --prune` pruning after a backup that
skipped files it could not read
([issue #382](https://git.eeqj.de/sneak/vaultik/issues/382)). Such a prune
could delete the only snapshot still holding those files.
- 2026-10-14: Made `snapshot create --skip-errors` exit non-zero, instead of 0,
when it skips anything it cannot read
([issue #310](https://git.eeqj.de/sneak/vaultik/issues/310)).
- 2026-10-14: Kept the local index when `remote nuke` cannot list or find the
destination store
([issue #292](https://git.eeqj.de/sneak/vaultik/issues/292)). Nuke had cleared
the local snapshot records anyway; it now fails before removing anything.
- 2026-10-14: Stopped counting directories and unchanged symlinks as files to
back up ([issue #305](https://git.eeqj.de/sneak/vaultik/issues/305)).
- 2026-10-14: Refused a backup destination that an unplugged volume left empty,
as the owner ruled
([issue #239](https://git.eeqj.de/sneak/vaultik/issues/239)). Every backup now
writes `vaultik-destination.txt` at the destination root, and while the local
index holds snapshots stored there, a destination without it is treated as not
present.
- 2026-10-14: Warned about restored ownership only when an entry's owner or
group differs from the recorded one
([issue #316](https://git.eeqj.de/sneak/vaultik/issues/316)); a restore by a
non-root user had warned even when none did.
- 2026-10-14: Made `snapshot list` report a changed destination instead of
advising `prune`, which refuses in that case
([issue #317](https://git.eeqj.de/sneak/vaultik/issues/317)).
- 2026-10-14: Counted a file as deleted by a backup only if the previous
snapshot of the same name listed it
([issue #304](https://git.eeqj.de/sneak/vaultik/issues/304)); files that
snapshot did not list had been counted too.
- 2026-10-14: Named each prefix once in restore, verify and purge errors, some
of which repeated it
([issue #327](https://git.eeqj.de/sneak/vaultik/issues/327)). A deep verify
log line that named S3 whatever the destination now names storage.
- 2026-10-14: Refused credentials in a storage URL's endpoint or rclone remote
name, which were accepted and could then be printed
([issue #322](https://git.eeqj.de/sneak/vaultik/issues/322)).
- 2026-10-14: Ran `$EDITOR` through the shell in `config edit`, so a value with
arguments, such as `code --wait`, no longer fails
([issue #315](https://git.eeqj.de/sneak/vaultik/issues/315)).
- 2026-10-14: Made `snapshot remove` fail, instead of exiting 0, when it cannot
delete the metadata of a snapshot that is only on the destination store
([issue #340](https://git.eeqj.de/sneak/vaultik/issues/340)). The README and
the help now say when it prints the `vaultik prune` follow-up.
- 2026-10-14: Labelled the totals in the local index cleanup before each backup
as totals ([issue #306](https://git.eeqj.de/sneak/vaultik/issues/306)). Each
was shown as `(N remain)`, which read as rows left to clean up.
- 2026-10-13: Freed a cached blob once no file still to restore needs it
([issue #324](https://git.eeqj.de/sneak/vaultik/issues/324)); one also used by
a file outside the selected paths had stayed in the temporary cache until the
restore ended.
- 2026-10-13: Kept the whole startup error when it contains `): `
([issue #320](https://git.eeqj.de/sneak/vaultik/issues/320)). Such an error
was cut at its last `): `, so the `blob_size_limit` one lost the setting's
name.
- 2026-10-13: Silenced the status lines of `prune` and snapshot removal under
`--cron` and `--quiet`
([issue #294](https://git.eeqj.de/sneak/vaultik/issues/294)).
- 2026-10-13: Read the rclone config file that `RCLONE_CONFIG` names
([issue #321](https://git.eeqj.de/sneak/vaultik/issues/321)); a remote defined
only there had been reported as not found.
- 2026-10-13: Made `/**/` in an exclude pattern match zero directories, as in
gitignore ([issue #338](https://git.eeqj.de/sneak/vaultik/issues/338));
`a/**/*.log` had missed `a/x.log`.
- 2026-10-13: Made the `chunk_size` and `blob_size_limit` defaults and the
`chunk_size` minimum 1000-based, like the size parser and the docs
([issue #313](https://git.eeqj.de/sneak/vaultik/issues/313)).
- 2026-10-13: Kept a snapshot's remote metadata when `snapshot purge` cannot
remove it from the local index
([issue #337](https://git.eeqj.de/sneak/vaultik/issues/337)); the purge had
deleted the metadata anyway and exited 0.
- 2026-10-13: Made `snapshot restore` fail for a requested path that matches
nothing, which it had ignored
([issue #309](https://git.eeqj.de/sneak/vaultik/issues/309)).
- 2026-10-13: Made `remote nuke` delete every object under `metadata/` and
`blobs/` ([issue #291](https://git.eeqj.de/sneak/vaultik/issues/291)); it had
left some, such as a stray `.DS_Store`, while reporting the store empty.
- 2026-10-13: Made `config set` create a list for an index under a missing key
([issue #308](https://git.eeqj.de/sneak/vaultik/issues/308)); it had written a
map, after which no command could load the config.
- 2026-10-13: Listed a `file://` destination through a symlinked `metadata/` or
`blobs/`, which had listed as empty
([issue #296](https://git.eeqj.de/sneak/vaultik/issues/296)). A link to a
missing target now fails the listing
([issue #332](https://git.eeqj.de/sneak/vaultik/issues/332)).
- 2026-10-13: Made `snapshot verify --deep --json` report its blob counts
([issue #314](https://git.eeqj.de/sneak/vaultik/issues/314)); a failed run had
reported 0 missing and 0 verified.
- 2026-10-13: Limited and retried the module download in the Docker build
([issue #351](https://git.eeqj.de/sneak/vaultik/issues/351)), which had hung
for as long as a stalled connection to the module proxy lasted.
- 2026-10-13: Recorded each snapshot's uncompressed size when its backup runs
([issue #360](https://git.eeqj.de/sneak/vaultik/issues/360)). A later backup
could change an older snapshot's UNCOMPRESSED SIZE in `snapshot list`.
- 2026-10-13: Stopped `snapshot restore --skip-errors --verify` reporting a file
that failed to restore again as a checksum failure
([issue #323](https://git.eeqj.de/sneak/vaultik/issues/323)).
- 2026-10-13: Recorded the stored size of a blob a backup finds already stored,
not the size of its local copy, which can differ, so `snapshot verify` could
fail on a good snapshot
([issue #300](https://git.eeqj.de/sneak/vaultik/issues/300)).
- 2026-10-13: Stopped the local index and the blob upload step ending the
process on a failed close of query results or a failed temp file cleanup
([issue #345](https://git.eeqj.de/sneak/vaultik/issues/345)); both are now
returned as errors.
- 2026-10-13: Closed the snapshot database's storage reader too when restore or
deep verify is interrupted
([issue #356](https://git.eeqj.de/sneak/vaultik/issues/356)).
- 2026-10-13: Made `prune` and `remote nuke` fail when a delete on the backup
destination store fails
([issue #290](https://git.eeqj.de/sneak/vaultik/issues/290)).
- 2026-10-13: Made an S3 listing that the server cuts short an error
([issue #297](https://git.eeqj.de/sneak/vaultik/issues/297)); `prune` had
taken it as complete and could delete blobs still in use.
- 2026-10-13: Made shallow `snapshot verify` report a blob it could not check
apart from a missing one
([issue #318](https://git.eeqj.de/sneak/vaultik/issues/318)); any error
checking a blob had counted as missing, its cause dropped.
- 2026-10-13: Restored the other files when a file's chunk is in no blob under
`--skip-errors` ([issue #326](https://git.eeqj.de/sneak/vaultik/issues/326));
that file had stopped the restore before anything was restored.
- 2026-10-13: Recorded each snapshot's new chunk size when its backup runs
([issue #311](https://git.eeqj.de/sneak/vaultik/issues/311)). `snapshot list`
could show a NEW CHUNK SIZE of 0 B for a snapshot whose change was a modified
file.
- 2026-10-13: Made a backup interrupted during a local index write exit as
interrupted ([issue #301](https://git.eeqj.de/sneak/vaultik/issues/301)); it
could end with `FATAL: failed to rollback transaction` and exit 1 instead
of 130.
- 2026-10-13: Printed `snapshot list` and `snapshot purge` sizes like every
other command ([issue #312](https://git.eeqj.de/sneak/vaultik/issues/312)).
They, and the `bytes` value in terminal log lines, were 1024-based but
labelled KB or MB, and now use 1000-based units.
- 2026-10-13: Closed each blob's storage reader when restore or deep verify is
interrupted, which ends a stalled read unless it is stuck in the kernel
([issue #325](https://git.eeqj.de/sneak/vaultik/issues/325)).
- 2026-10-13: Made a mistyped subcommand under `snapshot`, `config`, `remote`,
`database` or `completion` an error
([issue #319](https://git.eeqj.de/sneak/vaultik/issues/319)); each had printed
its help and exited 0.
- 2026-10-13: Made an exclude pattern starting with `**/` match directly under
the snapshot path too, as in gitignore
([issue #303](https://git.eeqj.de/sneak/vaultik/issues/303)).
- 2026-10-13: Refused a relative `file://` storage URL and decoded its
percent-escapes ([issue #299](https://git.eeqj.de/sneak/vaultik/issues/299));
a relative path had moved with the working directory.
- 2026-10-13: Removed the blob temp file when a backup stops part-way
([issue #302](https://git.eeqj.de/sneak/vaultik/issues/302)); the half-written
blob had stayed in `$TMPDIR`.
- 2026-10-13: Checked `--keep-newer-than` before a backup starts
([issue #307](https://git.eeqj.de/sneak/vaultik/issues/307)); an invalid value
had been read only by the prune after the backup, so retention never ran.
- 2026-10-13: Stopped `remote info` counting a file under `blobs/` that is not a
blob as an orphaned blob
([issue #295](https://git.eeqj.de/sneak/vaultik/issues/295)). A stray file
such as `.DS_Store` was reported as an orphaned blob and the report advised
running `prune`, which skips such a file and deleted nothing. `remote info`
now takes its blob list from the same listing `prune` uses, in the table and
in `--json`.
- 2026-10-13: Made `snapshot remove` fail for a snapshot that does not exist
([issue #293](https://git.eeqj.de/sneak/vaultik/issues/293)). The command
never checked the ID: it reported the snapshot removed from the local database
and from the destination store and exited 0, under `--json`, `--dry-run` and
`--local-only` too. An ID that names no snapshot in the local index or on the
destination store is now an error; with `--local-only`, the snapshot must be
in the local index. The command now reports a removal from either side only
when one happened, and `remote_removed` is true only when metadata was
deleted.
- 2026-10-12: Bound the local index to the destination an `s3.*` config names
([issue #298](https://git.eeqj.de/sneak/vaultik/issues/298)). Only
`storage_url` was checked, so after `s3.bucket` or `s3.prefix` changed, the
next backup treated every chunk as already stored, uploaded nothing, and
completed a snapshot that could not be restored. Without `storage_url`, the
local index is now bound to `s3.endpoint`, `s3.bucket` and `s3.prefix`, and a
change is refused as a changed `storage_url` is.
- 2026-10-12: Rejected a retention duration longer than about 292 years
([issue #289](https://git.eeqj.de/sneak/vaultik/issues/289)). The number given
to `--older-than` or `--keep-newer-than` was multiplied by the length of its
unit with no limit, so from `293y`, `15251w`, `3559mo` or `106752d`, or a sum
of parts such as `200y200y`, the result wrapped to a negative duration. The
cutoff then lay in the future, every snapshot of the name was deleted, the one
just made included, and the command exited 0. Such a duration is now rejected
as invalid and nothing is deleted.
- 2026-10-08: Made a backup cancelled under `--skip-errors` stop at once
([issue #286](https://git.eeqj.de/sneak/vaultik/issues/286)). After Ctrl-C or
SIGTERM, phase 2 treated the cancellation error from each remaining file like
an unreadable file: it opened the file, printed an error line for it, counted
it as failed and went on to the next one. The run now stops at the first file
after the cancellation, and no file is counted as failed because of it.
- 2026-10-08: Made `remote nuke` delete the `.partial` files that uploads cut
off part-way leave on the destination store
([issue #281](https://git.eeqj.de/sneak/vaultik/issues/281)). The file and
rclone listings skip such a file, so the command left it in place and still
reported the store empty. It now removes them under `metadata/` and `blobs/`
as its last step.
- 2026-10-08: Made a local index error while recording a directory or symlink
stop a backup under `--skip-errors`
([issue #284](https://git.eeqj.de/sneak/vaultik/issues/284)). Phase 2 only
records such an entry in the local index, since it has no data to open or
read, but an error doing so was skipped like an unreadable file. The snapshot
then completed without the entry, and a restore did not recreate it. The error
now stops the backup, as it does without the flag.
- 2026-10-08: Counted a file that a backup could not store as failed
([issue #280](https://git.eeqj.de/sneak/vaultik/issues/280)). A file that
phase 1 counted and phase 2 could not open, because it was unreadable under
`--skip-errors` or removed in between, was reported in the summary as
unchanged with its bytes as backed up, and the `snapshots` row's `file_count`
and `total_size` included it. The summary now counts it as failed, and its
data total and the row leave it out.
- 2026-10-08: Made `remote info` report a snapshot's blob count and blob size as
unknown when its manifest cannot be read
([issue #272](https://git.eeqj.de/sneak/vaultik/issues/272)). The orphan
figures were already unknown in that case, but the snapshot's row still gave 0
blobs and 0 B, in the table and in `--json`. The row now reads `unknown` and
`--json` gives `null`. A directory with no manifest still shows 0, since the
orphan figures count its blobs as orphaned.
- 2026-10-08: Kept the progress line of a snapshot with more than one path
within 100% ([issue #271](https://git.eeqj.de/sneak/vaultik/issues/271)). The
bytes and files processed counted every path, but the totals they were divided
by held only the current path's, so a second path smaller than the first
showed more than 100% and an ETA of `unknown`. The totals now add up over the
paths scanned so far. The rate behind the ETA restarts when each path's scan
phase ends, so that phase does not lower the rate while the path is processed;
until then the rate is the previous path's.
- 2026-10-08: Made a second `snapshot create` of one name succeed when it starts
in the same second as the first
([issue #270](https://git.eeqj.de/sneak/vaultik/issues/270)). The timestamp in
a snapshot ID is in whole seconds, so the second run got the first run's ID
and failed with `UNIQUE constraint failed: snapshots.id`. When the local index
already has a snapshot with the ID, the create now waits a second and takes a
new timestamp.
- 2026-10-07: Documented that `--json` skips the confirmation prompt of
`snapshot remove` and `prune`
([issue #268](https://git.eeqj.de/sneak/vaultik/issues/268)). Both commands
delete without asking under `--json`, since a prompt on stdout would break the
JSON document, but the help and the README described only `--force` as
skipping it. The `--json` help of both commands and their README entries now
say so.
- 2026-10-07: Made a symlink whose target cannot be read stop the backup
([issue #269](https://git.eeqj.de/sneak/vaultik/issues/269)). It was left out
of the snapshot with only a debug log line, even without `--skip-errors`. It
now aborts the run, or with `--skip-errors` is skipped with the
`Failed to access` error line that any other entry the scan cannot read gets.
- 2026-10-07: Stopped a killed rclone upload from leaving a truncated object
under its key ([issue #266](https://git.eeqj.de/sneak/vaultik/issues/266)).
The rclone backend wrote each object straight to its key, so killing an upload
to a local or sftp remote left a truncated object there; the next backup found
the key with `Stat`, skipped the upload and recorded a snapshot that could not
be restored. On a remote with a server-side move, an object is now written
under a temporary name ending in `.partial` and moved into place, and listings
skip such names. Remotes without one are still written in place.
- 2026-10-07: Made an interrupted command exit 130 and say so
([issue #267](https://git.eeqj.de/sneak/vaultik/issues/267)). Ctrl-C or
SIGTERM during `snapshot create`, `snapshot restore` or `snapshot verify`
exited 0 with no error line (`snapshot verify --json` exited 1, also without
one), so a `--cron` run that never finished looked like a success. A command
stopped by either signal now exits 130 and prints
`interrupted before the command finished` on stderr, under `--cron` and
`--json` too.
- 2026-10-07: Corrected documentation, help text and comments that were false
about the code ([issue #233](https://git.eeqj.de/sneak/vaultik/issues/233)). A
blob is not streamed to storage. The README, `ARCHITECTURE.md` and
`config.example.yml` now say a backup needs free temporary space, because each
blob is written whole to a temporary file before it is uploaded (up to about
`blob_size_limit`; an rclone destination that cannot stream uploads needs
about twice that) and the metadata export writes copies of the local index.
Temporary files go to `$TMPDIR` (default `/tmp`); with `TMPDIR` unset, SQLite
writes one of those copies to `/var/tmp`. Also corrected: the snapshot ID
format, what restore reads and how incomplete snapshots are removed in
`docs/DATAMODEL.md`, what `source_path` holds, the `index_path` and config
file defaults, what `snapshot remove` cleans up, how the release gets its Go
toolchain, and the `script/release` and `script/fmt-check` comments.
- 2026-10-07: Cut the time the `internal/vaultik` and `internal/database` tests
take ([issue #235](https://git.eeqj.de/sneak/vaultik/issues/235)). Most of the
`internal/vaultik` time went to 24 tests that ran one at a time only because
they call `log.Initialize`; they now call it before `t.Parallel()`, as the
package's other tests do. `TestLargeDatasets` committed each of its 1,500
inserts on its own and now makes them in one transaction.
`TestDedupOnlySnapshotRestores` gives its second backup its own snapshot name
instead of sleeping past the one-second timestamp in the snapshot ID.
- 2026-10-07: Made two messages say only what is true
([issue #240](https://git.eeqj.de/sneak/vaultik/issues/240)). A config file
that others can read was warned about as containing S3 credentials even when
it set none, as a `file://` config does. The warning now says the file may
contain S3 credentials only when `s3.access_key_id` or `s3.secret_access_key`
is set, since either may come from a `${...}` reference rather than the file,
and otherwise says the file is readable by others. `snapshot purge` against a
destination store it could not list gave an error with
`listing remote snapshots:` in it twice; the prefix now appears once.
- 2026-10-07: Made `s3.part_size` set the multipart upload part size
([issue #232](https://git.eeqj.de/sneak/vaultik/issues/232)). It was loaded
and defaulted but never passed to the S3 client, whose uploader used a fixed
10MiB part. It now reaches the uploader for `storage_url` and for the `s3.*`
fields, and a part size S3 refuses, below 5MiB or above 5GiB, `0` included,
fails at config load. A blob too large for S3's limit of 10,000 parts at the
configured size is uploaded in larger parts. The docs gave the default as
`5MB`, which the config file reads as 5,000,000 bytes, below the minimum; they
now say `5MiB`.
- 2026-10-07: Made per-name retention work when the hostname contains `_`
([issue #230](https://git.eeqj.de/sneak/vaultik/issues/230)). A snapshot ID is
`hostname_name_timestamp`, and the name was read as everything between the
first and the last `_`, so with `hostname: my_host` the name `home` came out
as `host_home`. `snapshot purge --keep-latest --snapshot home` then printed
"No snapshots to delete", and `snapshot create --prune` purged nothing without
a message. The name is now read using the hostname the `snapshots` table
stores with each snapshot, cut at its first `.` as it is in the ID.
- 2026-10-07: Made `remote info` stop reporting a snapshot's blobs as orphaned
when its manifest cannot be read, and stop printing raw names from under
`metadata/` ([issue #228](https://git.eeqj.de/sneak/vaultik/issues/228)). A
manifest it failed to read was skipped, so that snapshot's blobs were counted
as orphaned and the report advised running `vaultik prune`. The orphan figures
are now unknown in that case, with no prune advice, and `--json` gives them as
`null` with the unreadable remote keys in `unreadable_manifests`. A name under
`metadata/` that is not 64 lowercase hex characters is now skipped with a
warning instead of being printed, control characters included. A manifest
under a skipped name is then not read either, so it also leaves the orphan
figures unknown, and `--json` counts such manifests in
`skipped_manifest_count`. A directory with no manifest in it, as left by an
interrupted backup, leaves the figures known.
- 2026-10-07: Made `config set` keep a string that looks like a number
([issue #229](https://git.eeqj.de/sneak/vaultik/issues/229)). It wrote every
value unquoted, and `config.Load` reads the file through untyped YAML, so an
access key `00112233` loaded as `38043` and a hostname `007` as `7`. A value
for a string setting in `config.Config` is now tagged as a YAML string, which
the file quotes wherever YAML would read a number or a boolean; other settings
are still written unquoted.
- 2026-10-07: Made a backup notice a file rewritten with its size unchanged and
a new mtime in the same second as the one in the index
([issue #226](https://git.eeqj.de/sneak/vaultik/issues/226)). The `files`
table held mtime in whole seconds and the scanner compared whole seconds, so
every later snapshot kept the old content. A new `mtime_nsec` column holds the
nanoseconds within the second that `mtime` holds, and the scanner compares the
full mtime. A local index created before the change lacks the column and is
rebuilt with `vaultik database delete` and a full backup.
- 2026-10-07: Made taking the process-wide lock atomic
([issue #227](https://git.eeqj.de/sneak/vaultik/issues/227)). The lock read
`vaultik.pid`, checked whether that PID was alive and then wrote its own, so
two writers started together could both pass the check and both run. It is now
an `flock` on `vaultik.pid`, held until the run ends; the kernel drops it when
the process exits, so a crash leaves no lock behind. A clean exit now empties
the file instead of deleting it, because deleting it would let two later runs
each lock a different file.
- 2026-10-06: Made `snapshot remove --json` write only its document to stdout
when the destination store cannot be reached
([issue #251](https://git.eeqj.de/sneak/vaultik/issues/251)). Its warning that
the snapshot's metadata was left on the destination store went to stdout ahead
of the document, breaking `| jq` on a command that exited 0. Under `--json`
the warning now reaches stderr only, through the logger. The warning, the
README and the command's help said `vaultik prune` would finish the cleanup,
but `prune` never removes snapshot metadata; they now say to run
`vaultik snapshot remove` for the snapshot again once the destination store is
reachable.
- 2026-10-06: Made the backup summary and the `snapshots` row count each file,
byte and upload once
([issue #225](https://git.eeqj.de/sneak/vaultik/issues/225)). The scanner
added a file's bytes again for each new chunk and counted a file as unchanged
for each chunk already stored, so a first backup reported twice its size and
"backed up" could go negative. Upload figures came from the progress reporter,
which `--cron` turns off, and `blob_count` counted earlier paths' blobs again
for each later path. The scanner now counts uploads itself; `blob_size`,
`blob_uncompressed_size` and `compression_ratio` describe the blobs the
snapshot references, and `docs/DATAMODEL.md` now says `chunk_count` and
`blob_count` count what the run added.
- 2026-10-06: Made command output follow the README's stdout and stderr rules
([issue #224](https://git.eeqj.de/sneak/vaultik/issues/224)). The startup
banner went to stdout, so a `completion` script or a `config get` value
started with it; the banner now goes to stderr. A failing `remote info`,
`prune` or `snapshot remove` under `--json` printed nothing on either stream,
and now reports its error on stderr. `snapshot verify --quiet` printed its
whole report; it now prints none, and a failure still reaches stderr with the
same exit status.
- 2026-10-06: Made a backup without `--cron` of a snapshot with two or more
`paths` complete instead of panicking with `close of closed channel`
([issue #253](https://git.eeqj.de/sneak/vaultik/issues/253)). `Scan` runs once
per path and started and stopped the progress reporter each time, and a second
stop panics. The reporter is now started and stopped once per snapshot, around
the scans of all its paths.
- 2026-10-06: Made a restore path argument select only that path and what is
beneath it ([issue #223](https://git.eeqj.de/sneak/vaultik/issues/223)). The
lookup matched with SQL `LIKE`, so `/home/u/doc` also restored `doc2`, `DOC`
and `doc.txt.bak`, and a `_` or `%` in the path acted as a wildcard. A backup
used the same lookup to load the known files of each configured path, so files
of a longer sibling path were counted as deleted.
`FileRepository.ListUnderPath`, which replaces `ListByPrefix`, returns the
file at the path and every file whose path starts with the path plus `/`,
compared case-sensitively.
- 2026-10-06: Made restore return an error instead of panicking on a malformed
snapshot database
([issue #231](https://git.eeqj.de/sneak/vaultik/issues/231)). A chunk hash
shorter than 16 characters crashed the error message naming it, and `--verify`
dereferenced a missing `chunks` row and allocated whatever chunk size the
database gave. Those messages now go through `shortHash`, a missing row is an
error, and `--verify` rejects a negative size and hashes each chunk as a
stream.
- 2026-10-06: Made `s3://bucket/prefix` and `s3://bucket/prefix/` the same
destination ([issue #222](https://git.eeqj.de/sneak/vaultik/issues/222)). The
S3 client put the prefix directly in front of each key, so a prefix without a
trailing slash stored `prefixblobs/...`. A non-empty prefix is now joined to
every key with one `/`, giving the README's `<bucket>/<prefix>/blobs/...`
layout. The `s3.prefix` config setting goes through the same client and gets
the same join.
- 2026-10-06: Made restore apply owners, modes and times in an order that keeps
them ([issue #219](https://git.eeqj.de/sneak/vaultik/issues/219)). A directory
got its stored mode and mtime before its contents were written, so a read-only
directory came back without its files and a non-empty one carried the time of
the restore. Directories are now created owner-only and get their stored
owner, mode and mtime once the restore loop is done, each before its parent. A
file's mode is applied after its chown, which on Linux clears setuid and
setgid, and a symlink gets its stored owner (as root) and mtime on the link
itself.
- 2026-10-06: Made `go.mod` what `go mod tidy` writes, so the pre-commit hook no
longer stops every commit
([issue #246](https://git.eeqj.de/sneak/vaultik/issues/246)). A test in
`internal/cli` imports `github.com/spf13/pflag` directly, but `go.mod` still
marked it `// indirect`, and `script/precommit` fails whenever the tidy
changes `go.mod`. It is now in the direct `require` block.
- 2026-10-06: Made the README's steps for restoring on another machine work
([issue #221](https://git.eeqj.de/sneak/vaultik/issues/221)). `config init`
wrote a placeholder recipient that `config.Load` rejects, so every command on
the new machine failed before reaching the store. The file now has an empty
`age_recipients` list, `config.Load` accepts an empty list, and
`snapshot create` refuses to run without a recipient.
- 2026-10-06: Made `snapshot restore --skip-errors` skip the files that need a
blob it cannot download
([issue #218](https://git.eeqj.de/sneak/vaultik/issues/218)). A missing or
damaged blob ended the restore even with the flag, after restoring whichever
files happened to come first. Every file that needs such a blob is now
reported as failed, the rest are restored, and the command still exits
non-zero. Without the flag the blob error still aborts.
- 2026-10-06: Re-vendored the canonical files from `sneak/prompts` at `dd4027b`
([issue #213](https://git.eeqj.de/sneak/vaultik/issues/213)). Linting and
testing are now the `lint` and `test` phases of the `Dockerfile`, and the
build stage depends on both. `Dockerfile.lint`, `CHECK_EPOCH` and the tests
that checked them are gone; every `docker build` in `script/` passes
`--no-cache` instead. golangci-lint is v2.14.0, with its new findings fixed in
the code, and the rules in `CLAUDE.md` now live in `AGENTS.md`.
- 2026-10-06: Made the local index actually run in WAL mode with a busy timeout
([issue #217](https://git.eeqj.de/sneak/vaultik/issues/217)). The connection
settings were written in a form the SQLite driver ignores, so the index ran
without either and `snapshot list` or `info` during a backup could make the
backup's next write fail with `database is locked`. They are now `_pragma=`
parameters, and the metadata export copies the open index with `VACUUM INTO`,
because a copy of the file alone misses rows still in the `-wal` file.
- 2026-10-06: Made a backup record the real uid and gid of files, directories
and symlinks ([issue #216](https://git.eeqj.de/sneak/vaultik/issues/216)). The
scanner asked the stat result for `Uid()` and `Gid()` methods, which
`*syscall.Stat_t` does not have, so every entry was stored as `0:0` and a
restore as root gave every file to root. It now reads the `Uid` and `Gid`
fields of `*syscall.Stat_t`.
- 2026-10-06: Made a `file://` destination whose directory is missing count as
one that cannot be listed
([issue #220](https://git.eeqj.de/sneak/vaultik/issues/220)). The file backend
listed a missing directory as an empty store, so with the quickstart's USB
stick unplugged `snapshot list` reported every local snapshot as missing from
the store, `snapshot remove` claimed to have removed metadata it never
reached, and `prune` dropped every local snapshot record. Listing a missing
destination directory is now an error; a first backup still creates the
directory.
- 2026-10-06: Made `--older-than` and `--keep-newer-than` reject a duration with
characters outside its number-and-unit parts
([issue #215](https://git.eeqj.de/sneak/vaultik/issues/215)). The parser
picked out the parts it recognised and skipped the rest, so `1.0y` became zero
and `--prune --keep-newer-than 1.0y` deleted every snapshot of the backed-up
names, the new one included. `1.0y`, `1,5y`, `30 days`, `x7d` and a bare `0`
are now errors; decimals still work in Go units such as `1.5h`.
- 2026-10-06: Made a backup re-chunk a known file that lists a chunk no uploaded
blob holds ([issue #214](https://git.eeqj.de/sneak/vaultik/issues/214)). File
rows are shared by every snapshot and updated in place, so removing or pruning
the only snapshot that referenced a changed file's current blob left an older
snapshot keeping a row that matched the disk while no blob held its chunks.
The next backup skipped the file and completed a snapshot that could not
restore it.
- 2026-09-22: Routed the last direct-to-stdout command output through
`internal/ui` ([issue #149](https://git.eeqj.de/sneak/vaultik/issues/149)).
The `version`, `info`, `remote info`, `config`, and `database delete` commands
wrote plain text straight to stdout, so they were unstyled and ignored
`--quiet`. Output now falls in two buckets, both governed by `internal/ui`:
status lines and confirmations go through its message methods (styled, and
`--quiet` silences them), while the data a command exists to produce — the
reports, the `snapshot list` table, `config get` values, and the `--json`
documents — is written plain. `--quiet` silences the human reports and tables
but never the `config get` value or the `--json` documents, which a script
depends on, and the `database delete` confirmation prompt is always shown. The
README output-style section now states this rule.
- 2026-09-22: Validated blob hashes, offsets and lengths read back from the
destination before using them
([issue #155](https://git.eeqj.de/sneak/vaultik/issues/155)). A blob hash
taken from the downloaded database or the store listing was trusted unchecked,
so a hostile remote could drive a decrypted blob to be written outside the
cache directory (a hash like `aa/../../etc`) or crash a command with a short
or negative value. `blobDiskCache.path` now refuses any key containing a path
separator, and `ReadAt` rejects a negative offset or length and bounds with
`length > size-offset` so a sum cannot overflow past the check. A new
`isBlobHash` helper (a plain function, not a method — the packer stores
`temp-placeholder-{uuid}` as a hash) gates `FetchBlob`, shallow and deep
verify; the `blobs/` and `metadata/` listings skip a non-conforming name with
a warning; and short-hash prefixes in log and error text go through a
`shortHash` helper that cannot panic. `verify`'s chunk reader also rejects a
negative `blob_chunks` length and streams the chunk instead of allocating a
database-supplied size. `restore.go` and `internal/database` were left
untouched to avoid colliding with the in-flight
[issue #156](https://git.eeqj.de/sneak/vaultik/issues/156) work; the
cache-path and `FetchBlob` guards already stop the unsafe write and fetch, so
restore's own `buildBlobIndexes` early check is deferred as fail-fast defense
in depth.
- 2026-09-21: Stopped an interrupted blob upload from making a later backup
deduplicate against data that was never stored
([issue #148](https://git.eeqj.de/sneak/vaultik/issues/148)). The packer
commits a blob's `chunks`, `blob_chunks`, and `blobs` rows before the upload
is attempted, so a failed upload left chunk rows behind and the next run
skipped re-uploading them, producing a snapshot that reported success but
could not be restored. A run now deduplicates only against chunks held by a
blob whose `uploaded_ts` is set, and at startup drops any un-uploaded blob
rows (and the chunks they orphan) so the affected data is re-chunked and
re-uploaded. Blobs recorded with no remote backend are marked uploaded so this
invariant holds uniformly.
- 2026-09-22: Made restore refuse any snapshot path that would write outside the
target directory ([issue #154](https://git.eeqj.de/sneak/vaultik/issues/154)).
`restoreFile` and `verifyRestoredFiles` joined the stored path onto the target
with no containment check, so a `..` segment or an absolute path escaped the
target and a restored symlink could redirect a later child write anywhere on
disk. Every stored path is now rejected unless `filepath.IsLocal` accepts it
with the leading separator removed, and each existing ancestor directory below
the target is `Lstat`ed to refuse descending through a symlink; honest
symlinks pointing outside the tree are still written verbatim. age decryption
proves a snapshot is readable, not honest, and restore usually runs as root.
- 2026-09-21: Stopped `--json` from silencing stderr diagnostics
([issue #112](https://git.eeqj.de/sneak/vaultik/issues/112)). `--json` used to
be folded into `Quiet`, which pinned the log level to `WARN`, so
`prune --json` gave a machine consumer no record of the local index rows it
deleted even under `--verbose`. `--json` now quiets only the stdout UI (the
JSON document must stay clean, per
[issue #108](https://git.eeqj.de/sneak/vaultik/issues/108)); the stderr log
level follows `--verbose`/`--debug` again. The coupling was removed the same
way for `snapshot verify`, `snapshot remove`, and `remote info`, which carried
it for the same outdated reason.
- 2026-09-21: Stopped `prune` from reporting a failed row count as 0
([issue #96](https://git.eeqj.de/sneak/vaultik/issues/96)). The seven
`getTableCount` reads in `PruneDatabase` discarded their error, so a query
that could not run became a plausible `0` and the before/after delta computed
from it looked like real work. Each read now logs at warn on failure and
renders as `unknown`, never `0`, so an empty table is distinguishable from one
that could not be queried. The counts have no `--json` representation — under
`--json` the summary is suppressed entirely — so nothing there can show a
false `0`.
- 2026-09-21: Made the s3 storage backend report a missing object as
`storage.ErrNotFound`, like the `file` and `rclone` backends and as the
`Storer` interface documents. `S3Storer.Get` and `Stat` returned the raw AWS
SDK error, so `errors.Is(err, storage.ErrNotFound)` was false on s3 and
callers branched differently per backend. Added a small `s3.IsNotFound` helper
(reused by `HeadObject`) and a test that a missing key maps to `ErrNotFound`
([issue #129](https://git.eeqj.de/sneak/vaultik/issues/129)).
- 2026-09-21: Fixed `verify --deep` reporting healthy snapshots as corrupt. Its
final blob-integrity check hashed the encrypted downloaded bytes with a single
SHA256 and compared that to the blob ID, which is the double SHA256 of the
plaintext, so the two could never match. It now hashes the decompressed
plaintext and compares the double SHA256. Added a test that backs up a real
snapshot, deep-verifies it, then flips a byte in one stored blob and confirms
deep verification then fails
([issue #131](https://git.eeqj.de/sneak/vaultik/issues/131)).
- 2026-09-21: Made `snapshot create` VACUUM the per-snapshot metadata database
through the `modernc.org/sqlite` driver instead of shelling out to the
external `sqlite` command-line binary (issue #120). A backup no longer needs
that binary on `PATH`, so `make check` passes on a stock `go install` host;
`script/bootstrap` and the `Dockerfile` (both the test-build and the shipped
runtime stage) no longer install it, and a new test asserts the uploaded
database keeps no pages from deleted rows. Dropped the now-false note on the
2026-08-07 entry below that said bootstrap installs it.
- 2026-09-21: Made `.gitea/workflows/check.yml` run on pushes to `main` and
`next` and on pull requests against either, so unit PRs (whose base is `next`)
and `next` itself get a CI run instead of relying on a local `make check`
([issue #122](https://git.eeqj.de/sneak/vaultik/issues/122)).
- 2026-09-21: Hash-verified the Go toolchain in the release workflow
([issue #105](https://git.eeqj.de/sneak/vaultik/issues/105)). New
`script/install-go` downloads the exact `go.dev` archive for `go.mod`'s `go`
directive and refuses it unless its sha256 matches a value committed in the
script; `.gitea/workflows/release.yml` calls it instead of `actions/setup-go`,
which verified the downloaded toolchain against nothing in the repo.
`GOTOOLCHAIN: local` on the release step keeps that exact compiler from
auto-switching. Bumping Go now touches `go.mod`, the checksum, and the
`Dockerfile` `golang` digest together.
- 2026-09-21: Collapsed the two duration parsers into one and fixed the
`--older-than` months example
([issue #123](https://git.eeqj.de/sneak/vaultik/issues/123)). Two functions
named `parseDuration` existed with different grammars;
`snapshot purge --older-than` and `--keep-newer-than` both already went
through the one in `internal/vaultik`, while the richer copy in
`internal/cli/duration.go` was reachable only from its own test. Kept the
live-path parser and deleted the unused one, so no flag's accepted grammar
changes. The trap the issue was filed over: `README.md` documented `6m` as the
months example for `--older-than`, but `m` is minutes, so the documented
command deleted every snapshot older than six minutes on a destructive flag.
Corrected the doc to `6mo` and put both flags' help text on one example list
that states `m` is minutes and `mo` is months. The surviving parser now
rejects negatives, which it previously accepted (`-5h`) or silently made
positive (`-5d`). Table-driven tests cover every unit, `6m` as six minutes,
`6mo` as 180 days, and rejection of a bare number, an unknown unit, and a
negative.
- 2026-08-10: Moved every lint run into its own container, as a build step
([issue #113](https://git.eeqj.de/sneak/vaultik/issues/113)). New root
`Dockerfile.lint`, built by `script/lint`, runs
`golangci-lint run --config .golangci.yml ./...` as a `RUN` instruction in the
digest-pinned `golangci/golangci-lint` image: a successful build of that file
_is_ a clean lint, and it works even where the daemon is remote and bind
mounts are impossible. That `FROM` line is now the only pin of the linter
version in the repo.
This supersedes the per-worktree cache isolation landed for
[issue #99](https://git.eeqj.de/sneak/vaultik/issues/99). Isolation fixed
cross-worktree contamination but not lock contention — two concurrent runs
with entirely separate cache directories still collided. A container per run
has its own cache and its own lock, so the whole class is gone, and with it
the per-worktree cache machinery, the lock-retry loop, and
`script/lint-audit`, which existed to catch replayed findings from a cache
that no longer exists. The host lint path went too: no escape hatch, no
`VAULTIK_LINT_IN_CONTAINER`, no version detection. Nothing lints on the host
at any version.
A cached build lints nothing, so the same `CHECK_EPOCH` mechanism the
product `Dockerfile` already used is what makes the green mean something:
`ARG CHECK_EPOCH` with no default below the module layers, a
`RUN [ -n "$CHECK_EPOCH" ] || exit 1` guard, the value expanded into each
check command, and a fresh `$(date +%s%N)$$` per invocation computed as a
bare assignment. `cmd/vaultik/lintdocker_test.go` parses both Dockerfiles
and both scripts and fails if any part of that is dropped, because every way
of losing it is silent. No test asserts that no script runs the host linter:
`script/lint` is the one lint entry point and runs `golangci-lint` only
inside the container, and keeping it that way is a review matter, not
something a test proves.
The product `Dockerfile` lost its lint stage rather than gaining a second
linter pin: `make lint` is now `docker build`, so the stage would have been
docker-in-docker with no daemon, and calling `golangci-lint` directly there
would have restored the two-pins drift of
[issue #78](https://git.eeqj.de/sneak/vaultik/issues/78). `make fmt-check`
moved beside `make test` in the builder stage, and `script/cibuild` now
builds `Dockerfile.lint` and then `Dockerfile`, each with its own fresh
epoch. Consequence, stated rather than left to be found: `script/docker`
builds the product image only and no longer lints; the gates are
`script/check` and `script/cibuild`.
`golangci-lint config verify` runs as its own epoch-keyed layer, above the
lint. `golangci-lint run` rejects a config it cannot parse but silently
ignores an unknown top-level _key_: renaming `linters:` to `linterz:`
discarded `default: all` and every threshold and still exited 0 on a tree
the real config fails. `config verify` catches that, and it does so with the
network off at this pin — checked under `docker run --network none`, not
assumed. An earlier revision omitted it on the claim that it fetches its
schema over live HTTPS; that claim was false at v2.12.2.
`script/lint-fix` is kept, reimplemented as a bind-mounted `docker run`
against the image parsed out of `Dockerfile.lint` — it cannot be a build
step, because fixes have to land in the worktree — and marked in its header
as a developer convenience that no gate reads.
- 2026-08-09: Finished the `--json` stdout contract and gave `make build` a rule
([issue #108](https://git.eeqj.de/sneak/vaultik/issues/108),
[issue #110](https://git.eeqj.de/sneak/vaultik/issues/110)). Two unrelated
defects of the same shape — a command reporting something it did not do —
landed together because both are small.
`CleanupLocalSnapshots` wrote three prose lines to stdout with no `--json`
awareness, covering every branch of the function, so no input avoided them
and `vaultik prune --json | jq` failed even after
[issue #106](https://git.eeqj.de/sneak/vaultik/issues/106) removed the
banner. `-q` never helped either: `printlnStdout` and `stdoutf` write
straight to `Vaultik.Stdout` and never consult `Vaultik.UI`, which is what
`SetQuiet` affects. The issue offered three fixes and asked for a decision.
Taken: thread `*PruneOptions` into the function and gate each write on
`!opts.JSON`, matching `PruneBlobs` (its sibling phase, which already takes
the same struct), `RemoveSnapshot` and `remote info`, so the package has one
pattern rather than two. Rejected: moving the lines to `log.Info`, because
the logger's default level is `slog.LevelWarn`, so that would not relocate
them to stderr — it would delete them from a plain `vaultik prune`, and the
removal of rows from the local index is not something to narrate only under
`--verbose`. Also rejected: putting the stale-record count into
`PruneBlobsResult`, whose every field is blob-scoped and which is produced
by the later phase; a prune document covering both phases is a reasonable
thing to want, but it is a schema design question and not a stream-hygiene
fix. The narration is duplicated as `log.Info` records, which `PruneBlobs`
already does alongside its own prints, so the events survive on stderr for
anyone running `--verbose`.
`make build` printed "Nothing to be done for 'build'" and exited 0 without
producing a binary: `build` was listed in `.PHONY` with no `build:` rule
anywhere, and declaring a name phony is exactly what converts make's "No
rule to make target" error into a silent success. Fixed with
`build: vaultik`, keeping `vaultik:` as the file rule. The audit the issue
asked for covers all 19 `.PHONY` names; `build` was the only one without a
rule, and `vaultik` is correctly absent from `.PHONY`, being a real file
target.
Tests, each verified to fail with the fix reverted rather than assumed to:
`CleanupLocalSnapshots` leaves stdout untouched under `--json` in all three
branches (stale records, none, empty index) and still emits every line
without it, so the guard cannot be satisfied by deleting the output;
`prune --json` run end to end through `Entry`, cobra and fx over the
process's real stdout descriptor against a `file://` store, asserting
exactly one JSON document, in both the stale and non-stale branches; and a
parse of the `Makefile` asserting every `.PHONY` name has a rule and that
`build` reaches the rule that produces the binary, which keeps the audit
true for names added later. That last one is a parse rather than an
invocation of `make`, since `make test` is what runs it and shelling back
into `make build` would nest a build inside the test run. The property a
parse cannot establish — that the recipe still fails when the build fails —
was verified by hand against a deliberately broken tree: `make build` exits
2 and produces nothing. `cmd/vaultik` gains its first test file, so
`make test` now reports 16 packages `ok` where it reported 15.
- 2026-08-09: Stopped the startup banner from contaminating `--json` documents
([issue #106](https://git.eeqj.de/sneak/vaultik/issues/106)). `Entry` writes
the banner to stdout before cobra parses anything, and the flag scan that
suppresses it knew `--quiet`, `-q` and `--cron` but not `--json`, so every
`--json` document arrived behind two lines of prose and a blank line, and
`vaultik snapshot list --json | jq` failed. With the logger already on stderr
from [issue #82](https://git.eeqj.de/sneak/vaultik/issues/82), this was the
last writer that could put something on stdout that the caller did not ask
for. The design question the issue raised — extend the raw-argv scan, or move
the banner after parsing — is answered in favour of the scan: the banner is
printed first deliberately, so that it still appears when cobra rejects the
arguments and on `--help`, and after parsing there is no single place that
covers those paths. The stated cost of the scan, that `--json` is a subcommand
flag matched anywhere in the vector, is a cost `--cron` already carries — it
exists only on `snapshot create` — so this adds an instance of an accepted
imprecision rather than a new kind, and the two error directions are not
symmetric: a false positive loses a decorative banner, a false negative
corrupts a document. Regression tests at the CLI layer, where
`internal/vaultik`'s existing guard cannot reach: one runs `Entry` itself over
the process's real stdout descriptor, through cobra and fx to the document,
made hermetic by `file://` storage; a second covers the argument vectors of
all five `--json` commands; a third asserts the banner is still printed
without a suppressing flag, so the first cannot be satisfied by deleting the
banner. Also corrected `AGENTS.md` policy 9, which still keyed the
structured-log format on stdout's TTY-ness after #82 moved that decision to
stderr — a rules file that misdescribes the code misleads exactly the readers
who trust it most. Two smaller findings from the same review: `bytesAttrKey`'s
human-readable byte formatting silently stopped applying under an open group,
because the key reaching the comparison is group-qualified (`transfer.bytes`),
now matched on its final segment and tested both ways; and `listEnv.stderr` in
`snapshot_list_test.go`, assigned but never read since those tests began
capturing the process's stderr, is removed. `Vaultik.Stderr` is kept — nothing
writes to it today, which its comment now says outright.
- 2026-08-09: Moved the logger to stderr and fixed `TTYHandler`'s discarded
attributes ([issue #82](https://git.eeqj.de/sneak/vaultik/issues/82),
[issue #97](https://git.eeqj.de/sneak/vaultik/issues/97)). Two defects in
`internal/log`, fixed together because both live in the handler construction
path. The first: both handlers were built over `os.Stdout`, and `WARN`/`ERROR`
are never suppressed, so a config file with group- or world-readable
permissions was enough to put a log record inside a `--json` document and
break `jq`. Diagnostics now go to stderr, and the TTY/JSON format choice
follows stderr rather than stdout — testing the wrong stream would colorize
records on a redirected stderr whenever stdout happened to be a terminal. This
is user-visible: `--verbose` and `--debug` output moves to stderr too, which
is documented in `README.md` under "stdout and stderr". It also let the local
workaround in `internal/vaultik/snapshot_list.go` go: `warnWhileListing` had
been hand-rolling structured-log formatting to reach a non-stdout writer, and
the `jsonOutput` parameter threaded through the remote-listing helpers existed
only to choose between the two writers. The collect-then-emit machinery around
`listingWarning` stays, but on its remaining merit — warnings emitted in key
order after `group.Wait()` are deterministic run to run, where emitting from
the fetch workers would order them by network timing. The second defect:
`TTYHandler.WithAttrs` and `WithGroup` discarded their arguments and returned
the receiver while their doc comments claimed otherwise, so `log.With`
attributes vanished on a terminal and appeared correctly in CI — failing
precisely when someone is debugging interactively. Both now return a new
handler (the receiver is never written to, since `slog` permits concurrent
derivation), attributes persist across records, and grouping is implemented as
dotted key prefixes, which is the only honest rendering for a format with
nowhere to nest. New tests cover both, including one that feeds the same
derivation chain to the TTY and JSON handlers and compares the attribute sets,
so the two paths cannot drift apart again. Found and filed while verifying:
the startup banner is written to stdout and `--json` does not suppress it
([issue #106](https://git.eeqj.de/sneak/vaultik/issues/106)), which is a
separate writer on a separate path and the remaining source of stdout
contamination.
- 2026-08-09: Made the tagged-release path actually work on Gitea
([issue #65](https://git.eeqj.de/sneak/vaultik/issues/65)). Three independent
blockers, one of which was the whole release: `.goreleaser.yaml` had no
`gitea_urls:` block, so goreleaser defaulted to the GitHub API and a
`goreleaser release` from this repo would have failed or published where
nobody is looking. It now points at `https://git.eeqj.de/api/v1`. The version
is the second: it was a hardcoded `VERSION := 1.0.0-rc.1` in the `Makefile`,
so every local build claimed to be a release candidate that had never been
tagged and did not exist, while `git tag -l` was empty and `internal/globals`
defaulted to `dev`. Version now comes from git via the new `script/version` —
the exact tag with a leading `v` stripped (so a `make` build and a goreleaser
build of one commit report the same string, and it matches the archive names),
otherwise `dev-<12-char sha>`, with `-dirty` appended in either case when
tracked files are modified. Untracked files are deliberately not counted,
matching `git describe --dirty`. The same honesty was owed by the snapshot
path: `snapshot.version_template` was `{{ incpatch .Version }}-next`, which
manufactures a release number from the last tag and, with no tags at all, from
goreleaser's fabricated `v0.0.0`; it now emits the same `dev-<sha>`. The one
non-obvious consequence is that `internal/cli/version.go` gated its "this is a
development build" notice on the version being exactly `dev`, so the moment
untagged builds began carrying a commit sha that notice would have gone silent
and an unreleased binary would have read as a release — the gate is now
`globals.IsDevVersion`, which is a predicate over a string rather than a
comparison against a global precisely so it can be tested, and it is tested at
the boundary (`1.0.0-dev` is a release, `dev-<sha>` is not). Release
automation is the third blocker: a tag-triggered
`.gitea/workflows/release.yml` runs the build in CI rather than from a laptop,
with `fetch-depth: 0` because a shallow checkout has no tags and would
silently mislabel the release, and with the `RELEASE_TOKEN` repository secret
passed as `GITEA_TOKEN` (documented in `README.md`; the runner's automatic
token is not used because it is not guaranteed to carry release write scope).
`script/release` unsets any `GITHUB_TOKEN`/`GITLAB_TOKEN` it finds, since
goreleaser chooses its forge from whichever token variable is set and refuses
to run when it sees more than one — a runner-provided token must not get to
decide where these artifacts are published. `make release` and
`make release-snapshot`, the last two Makefile targets that were not shims,
now call `script/release` and `script/release-snapshot`, which resolve
goreleaser exactly the way `script/lint` resolves the linter: a `PATH` binary
is used only at the pinned version, never as a silent fallback.
`script/bootstrap` installs it, from a sha256-verified GitHub release archive
per `REPO_POLICIES.md`, via a separate `script/install-goreleaser` — separate
because `script/bootstrap` hard-fails without a usable Docker daemon by
design, and the release runner needs goreleaser without needing Docker.
Verified by running the thing rather than reading it: `make release-snapshot`
produced four archives and `checksums.txt`, and the linux/amd64 binary from
`dist/` reports `dev-<sha>` with the development-build notice. Tag handling
was exercised in a throwaway repository rather than by tagging this one; no
tag was created here, since that is the owner's call. Signing, SBOM,
reproducible builds, completions and a man page are out of scope by the issue.
- 2026-08-09: Isolated the lint cache per worktree and context-gated the native
lint path (issues #99, #80). One defect seen twice: `script/lint` decided
whether it could skip the pinned image by asking what version was on `PATH`
rather than where it was running, and cache isolation is part of that same
question. The cache was one directory per repo, shared by every worktree on
the host, so two checkouts with identical Go file contents collided and
golangci-lint replayed the stored analysis — paths and all. The loud direction
of that failure (a clean tree failed by a dirty sibling) is the harmless one;
the silent direction, a dirty tree **passed** by a clean sibling, is a sixth
way for a gate here to report a green it did not earn. The cache is now keyed
on a digest of the worktree path, and every run is audited by the new
`script/lint-audit`, which rejects output citing any file that is not in the
tree being linted — a backstop that runs on clean output too, because that is
the case nobody investigates. Caches record the worktree they belong to and
are collected when it disappears, so throwaway worktrees do not accumulate
them; the whole tree lives under `XDG_CACHE_HOME` and is disposable. The
`parallel golangci-lint is running` refusal is now a bounded retry rather than
a verdict: it is not a lint result, and exiting non-zero on it is
indistinguishable to a caller from real findings (#88 showed a private cache
does not remove that contention). The native path now requires
`VAULTIK_LINT_IN_CONTAINER=1`, set only by the `Dockerfile` lint stage, in
addition to matching the pin, so a developer's locally installed 2.12.2 no
longer bypasses the digest pin; `/.dockerenv` was rejected as the signal
because `dockerd` creates it for `docker run` and it is not reliably present
during a BuildKit `docker build`, which is the case the exception exists for.
Version detection uses `golangci-lint version --short` with the old banner
scrape kept only as a fallback. `script/bootstrap` no longer prints
`bootstrap complete` on a machine that cannot run the gate: a missing docker,
or one whose daemon is unreachable, is a hard failure naming exactly what
breaks. Verification was by reproduction rather than inspection — two
concurrent lints from two worktrees of differing cleanliness, a real run made
to report an outside path, a matching linter shimmed onto `PATH`, and a `PATH`
with docker removed — and is recorded on the pull request.
- 2026-08-09: Closed the fifth false-green mechanism (issues #93, #69).
`script/test` omitted `-count=1`, so Go's test result cache could satisfy the
gate outright: a second back-to-back `make test` printed the full set of 14
`ok` lines, every one marked `(cached)`, having executed no test at all. Since
`ok <pkg> (cached)` is an `ok` line, the "14 `ok` lines means the suite ran"
signal this repo leans on was forgeable, one level below the Docker layer
cache that #85 addressed. Fixed with `-count=1` unconditionally rather than
only in the container, because the pre-commit hook runs the same script and a
gate honest only in CI is dishonest where people rely on it most;
`test-coverage` got the same flag, and `script/check` inherits it by calling
`script/test`. In the same area, `make test-integration` was deleted rather
than made real: no file in the repo carried a build tag, so
`-tags=integration` selected nothing and the target was an exact duplicate of
`make test`. Tagging a subset was rejected because the entire suite runs in
well under a minute, and a scheme whose failure mode is "some tests silently
stopped running" is a poor trade for those seconds in a repo with this
particular history. The `-timeout` was raised from 30s after measuring rather
than after assuming: the standing claim that cold-cache compilation is charged
against `-timeout` is **false**, disproved by a containerised run that spent
46s compiling and still reported per-package durations within noise of a warm
host run. `-timeout` reaches the test binary as `-test.timeout` and its clock
starts inside `testing.M.Run`, after the build. The real exposure was margin,
not compilation. The 120s landed on is a **deliberate, documented divergence**
from `REPO_POLICIES.md:192`, which mandates 30s, and from that file's
canonical recipe at `:212-214`; the divergence is recorded in `script/test`'s
comment because `REPO_POLICIES.md` is org-canonical and not editable here, and
issue #101 proposes amending the policy text upstream. Numbers and the full
verification are recorded once, on the pull request, and are deliberately not
restated here.
- 2026-08-09: Triaged all fifteen stale remote branches (issue #71) and deleted
fourteen of them; the full per-branch disposition with evidence is recorded on
that issue. Method mattered more than the outcome here: a three-dot
`git diff main...branch` diffs from the merge base, so it replays everything
that landed on `main` after the branch diverged and makes any old branch look
like it holds unlanded work. That artifact is what made `golangci-v2.12.2`
appear to carry 126 files of unpushed changes when its tree was byte-identical
to `main`'s. Every containment claim here therefore rests on two-dot tip
diffs, tree-hash equality, `git cherry`, and `git branch -r --merged`. Nine
branches were plain ancestors of `main` with zero `git cherry` `+` commits.
`golangci-v2.12.2` had landed squashed as `cc58583`, whose tree hash equals
the branch tip's exactly; note the hash recorded in the issue had gone stale
because `main` advanced, so the check had to be redone rather than repeated.
`fix/sync-snapshot-cleanup` was redundant, its one line already on `main` in
`syncWithRemote`. `feature/restore-progress-bar` was superseded by
`printRestoreProgress` and the disk-backed blob cache, and had become actively
regressive — it would have deleted `internal/blobgen/compress_test.go`, the
#28 regression test that landed separately. The two branches this issue was
filed for both turned out to be closed questions that `main` had already moved
past by a recorded decision, so neither was landed and no regression test was
owed: `ctime` no longer exists anywhere in the codebase after `1c72a37`
removed the column, the `File.CTime` field and every use (#54/#55), and change
detection compares size, mtime, mode, uid and gid only, exactly as
`ARCHITECTURE.md` documents — so the silently-skipped-file data-loss risk that
made this a 1.0 item does not exist. The SQL allow-list branch would have
reverted `bfd7334`, which replaced that very allow-list with regex
sanitisation on review feedback, and would have broken
`getTableCount("snapshots")` because its allow-list omits that table.
`feature/daemon-mode` is untouched and deferred to #94 pending an owner
decision, so it is the one branch besides `main` still on the remote. The
stale `TODO.md` entry named in the issue needed no fix: `e496aa3` had already
removed it. No product code changed.
- 2026-08-09: Adopted the remaining upstream `CHECK_EPOCH` hardening (issue
#91), closing the gap #85 knowingly left open. Four changes, all four decided
as adopt upstream in `sneak/prompts` #26. (1) Each check stage now asserts
`[ -n "$CHECK_EPOCH" ] || exit 1` before running anything, so a build that
supplies no `--build-arg` fails instead of lying. This is the item that
mattered: an unset `ARG` is an empty string and an empty string is a stable
cache key, so the second and every later bare `docker build .` on an unchanged
tree replayed all three check layers and still exited 0 — and `docker build .`
is the command `REPO_POLICIES.md` names verbatim as a thing that must be
green, so the documented command was precisely the one that lied. Failed steps
are never cached, which is what makes the guard fire on every invocation
rather than once. (2) The epoch is now expanded into each check command rather
than left as a bare declaration, so the cache miss no longer depends on
BuildKit's unreferenced-`ARG` handling staying as it is, and the value appears
in the build log. (3) `script/cibuild` uses `epoch="$(date +%s%N)$$"`, unique
per invocation rather than per second; `%N` alone is insufficient because
busybox drops it silently and exits 0, and `$$` is what makes the guarantee
hold regardless. The bare-assignment form is kept deliberately — inlined in an
argument, a failing substitution does not abort under `set -eu` and would
yield an empty constant epoch, restoring the exact false green being fixed.
(4) `script/docker` passes the same fresh arg, so the two entrypoints cannot
disagree about whether the tree is green; local builds are almost always warm,
which made it the likelier fooling in practice. The `ARG` placement from #85
is unchanged, below `apk add`, `COPY go.mod go.sum` and `go mod download`, so
dependency layers still cache and the build is not cold. Verified by negative
control rather than inspection — a bare `docker build .` run twice back to
back, plus back-to-back pairs of both scripts and a host-side `make check`;
the measurements are recorded once, in the PR verification comment, rather
than restated here. `.golangci.yml`, the lint-stage `FROM` line and its
digest, `script/lint`, `REPO_POLICIES.md` and `.gitea/workflows/check.yml` are
all untouched.
- 2026-08-09: Stopped `script/cibuild` from reporting a green it did not earn
(issue #85). A bare `docker build .` let Docker serve the check layers from
the layer cache whenever the tree had not changed: the checks never executed
and the build still exited 0. The fix is an `ARG CHECK_EPOCH` declared
immediately above the check `RUN`s in both the lint stage and the builder
stage (`ARG` scope is per-stage, so each declares its own), with
`script/cibuild` assigning `epoch="$(date +%s)"` and passing
`--build-arg CHECK_EPOCH="$epoch"`. The assignment is separate on purpose:
under `set -eu` a command substitution that fails inside an argument does not
abort the script, which would leave an empty constant `CHECK_EPOCH` and
restore the very false green being fixed. Placement is the rest of the point —
the `ARG` sits below the `apk add`, `COPY go.mod go.sum`, and
`go mod download` layers, so only the checks are invalidated and the
dependency layers still cache. The guarantee is conditional on a fresh value
rather than absolute: a bare `docker build .` gets an empty `CHECK_EPOCH` and
can still serve the check layers from cache, which `README.md` and the
`Dockerfile` now say plainly, with issue #91 tracking the upstream hardening
(expanded `ARG` form, unset guard, per-invocation epoch, `script/docker`) that
would close it. Verified by re-running the reproduction plus the
withheld-`--build-arg` counterfactual; the measurements are recorded once, in
the PR #89 verification comment, rather than restated here. `.golangci.yml`,
the lint-stage `FROM` line and its digest, `script/lint`, and
`.gitea/workflows/check.yml` are all untouched.
- 2026-08-09: Corrected the `Vaultik.UI` doc comment (issue #84). It claimed the
cli layer replaces the writer with a discarding one in `--cron` mode; the
actual mechanism is `UI.SetQuiet(true)` in `setupGlobals`, which drops
Begin/Complete/Info/Notice/Detail/ Progress/Banner but still emits Warning and
Error. The `--cron` line in `README.md` said "Silent unless error", which
understated what survives, and now names warnings too. The other `--cron`
comments (`internal/log/log.go`, `internal/cli/snapshot.go`,
`internal/vaultik/snapshot.go`) were audited and already accurate. Comments
and docs only, no behavior change.
- 2026-08-09: Made `snapshot list` list the destination store without the
private key (issue #64). The listing is now the union of the local index and a
single streamed listing of the `metadata/` prefix, with no `age_secret_key`
gate — the manifest is unencrypted, so a host holding only the public key can
enumerate its own backups and a host that lost its local index can still see
them. A remote-only snapshot's hostname and name are deliberately not
recovered (they are not recoverable without the private key, and making them
so would undo the privacy property tracked in issue #81); such rows are
labelled by an abbreviation of their remote key and carry the real timestamp
and compressed size from the manifest, with `<remote only>` in the two columns
that require the local index. Local-only snapshots are reported as drift, and
the hint now names `vaultik prune`, which exists, instead of
`vaultik snapshot cleanup`, which does not. `reportRemoteDrift` collapsed into
the merged view. Every remote manifest read in the codebase now goes through
`downloadManifestByKey`, so issue #81 has one call site to change. Review
rework: snapshot timestamps now normalize to UTC in `scanSnapshotRows`, the
one place they enter the domain, so the merged TIMESTAMP column cannot show
local time for a locally tracked row and UTC for a remote-only row on a
non-UTC host; `GetIncompleteByHostname` was folded onto that same scanner.
`--json` now reports the unreadable-manifest count and the 1000-row truncation
on stderr instead of returning a silently short document (the document's shape
is unchanged). The two per-snapshot `log.Warn` calls on the listing path now
route through the same JSON-aware writer as the existing workaround, so one
corrupt manifest can no longer put a log line on stdout ahead of the document
and break `| jq` — still a local workaround pending issue #82. Verified with
`script/cibuild` and with an uncached `make check` (`0 issues.`, no cached
test packages), plus end to end against a `file://` destination with no secret
key present.
- 2026-08-09: Closed the gap between `make lint` and CI (issue #78).
`script/lint` now runs the digest-pinned `golangci-lint` image taken from the
`Dockerfile` lint stage, which is the single source of truth for the linter
version; the duplicate pin in the `Makefile` `deps` target and the unpinned
`golangci-lint` install in `script/bootstrap` are gone. A `golangci-lint` on
`PATH` is used only when its version is exactly the pinned one (which is how
the lint stage runs it inside the container); anything else goes through
Docker, and a missing or unreachable Docker daemon is a hard error rather than
a silent fallback. Only the **lint** leg of `make check` became equivalent to
`script/cibuild`; its tests and `gofmt` still run on the host against the host
toolchain, as `README.md` states. An earlier version of this entry claimed
`make check` was "as trustworthy as `script/cibuild`" outright, which
overstated it; corrected under issue #80.
- 2026-08-09: Finished the lint remediation under the canonical `.golangci.yml`
(issue #61, which also unblocks issue #59). The remaining findings were fixed
behavior-preservingly: `wsl_v5` whitespace, `sqlclosecheck`, and `prealloc`.
The `sqlclosecheck` sites now close `sql.Rows` in a deferred closure instead
of via the `CloseRows` helper, which the linter could not see through. Only
the `revive` package-name findings remain suppressed, with per-site `//nolint`
directives; the package-rename question behind them is tracked in issue #76.
Verified with `script/cibuild`, which exits 0 — that is the only trustworthy
gate, because `script/lint` runs whatever `golangci-lint` happens to be on
`PATH` rather than the pinned v2.12.2 that CI and the `Dockerfile` use, so
`make check` can report green on findings CI still fails. That tooling gap is
tracked in issue #78.
- 2026-08-09: The earlier next step "reconcile the uncommitted `ARCHITECTURE.md`
edits on `main`" needed no work: the working tree is clean and
`ARCHITECTURE.md` is committed on `main`.
- 2026-08-07: Updated golangci-lint to v2.12.2 everywhere it is pinned
(`Dockerfile` lint stage, `Makefile` deps target), replaced `.golangci.yml`
with the canonical config (v2 schema, `default: all`), and remediated the bulk
of the lint findings it surfaced (issue #61): behavior-preserving fixes across
every package, 2,990 findings down to 80. `make test` and `make fmt-check`
were green at that point but `make lint` was still red; the commit message
claiming `make check` was green was wrong.
- 2026-08-07: Added the standard `.golangci.yml` and `.editorconfig` (issue
#59); lint findings under the new config are tracked in issue #61.
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
shims, README Entrypoints section
- 2026-07-02: Consolidated CLI verbs, retired overlapping commands; bound the
local index to its backup destination URL.
- 2026-06-28: snapshot rm now removes metadata only and prints the prune
command; restore skips chown when running as non-root.
- 2026-06-26: Snapshot IDs hashed at the storage boundary; snapshot list made
resilient to bad remote entries.
- 2026-06-24: Collapsed snapshot prune into vaultik prune; restore streams blobs
to disk and restores files in blob-locality order; cron output fixes.
- 2026-06-17: Restore overhaul: ReadAt chunk reads from cached blobs,
reference-counted blob sweeper, integration tests; new internal/ui output
layer, banner, and progress lines.
- 2025-12-18: Added ARCHITECTURE.md and godoc coverage for exported API.
- 2025-07-26: End-to-end integration tests; manifest format refactor; renamed
backup to snapshot; afero filesystem abstraction.
- 2025-07-20: Initial design and implementation: cobra + fx CLI skeleton, SQLite
index database, UUID blob storage with streaming chunking.
# Future Steps
Work planned after 1.0 is listed in the README [roadmap](../README.md#roadmap).
Reference in new issue View Git Blame Copy Permalink