The README section opened with the commands that need nothing and the
config commands, and reached the requirement that covers most commands
third. It now states that rule first, then lists the commands that
need less. The facts are unchanged; only the order is.
Model: fable-5-1
Given a remote key, `snapshot verify` and `snapshot restore` resolve it
through `listAllRemoteSnapshotKeys`, which checks the destination marker,
so they fail when the marker is missing. Given a snapshot ID they do not
check it. Add them to the "These commands fail" list in `README.md` and
`docs/REPOSTRUCTURE.md`.
Model: fable-5-1
A new or changed symlink goes through recordNonRegularFile in the
scanner and is never opened. README.md, ARCHITECTURE.md and
docs/DATAMODEL.md now say so where they describe what a backup reads,
and README.md no longer lists a changed link target as a reason to
read a file.
Model: fable-5-1
The documents use "remote key" for something else, so "remote" as
rclone's noun is now defined on first use: a remote is a storage service
set up in rclone. Changed in `README.md` (the destination bullet),
`docs/REPOSTRUCTURE.md` (the opening) and `config.example.yml` (the
`storage_url` comment).
Model: fable-5-1
- `README.md`, `docs/REPOSTRUCTURE.md`, `internal/cli/snapshot.go`: a
backup needs only write permission on S3; an rclone remote that can
move objects also needs read and delete, and an upload there stops
when its look-up is refused.
- `docs/DATAMODEL.md`, `docs/REPOSTRUCTURE.md`: a restore reads no
file, chunk or blob rows from the local index, but it does open it.
- `README.md`: confirmation prompts go to stdout.
- `internal/cli/remote.go`: `remote nuke` leaves the backup destination
store empty, not a bucket prefix.
- `README.md`: a TODO section, as `REPO_POLICIES.md` requires.
Model: fable-5-1
Fixes the four findings of the sixth review: `snapshot remove` with
`--local-only`, the two comma-chained permission lists, six undefined terms,
and the sentence about an unfinished snapshot in the local index.
Then reads every sentence of `README.md`, `ARCHITECTURE.md`,
`docs/DATAMODEL.md`, `docs/REPOSTRUCTURE.md` and the comments in
`config.example.yml` against the code for four faults: a command's needs or
behaviour stated without regard to a flag, a false absolute, three or more
things chained with commas, and a technical word not defined on first use.
Each sentence changed is listed on the pull request.
The help string of `snapshot remove --dry-run` now says it needs no
permission to delete.
Model: fable-5-1
Under `--json`, `snapshot list` prints no warning for a snapshot with
no manifest at the destination; the README now says so and points to
`remote_present`. The stdout list names `snapshot create` as well as
`snapshot restore` for the per-file error lines. `snapshot create`
stops "before it writes anything at the destination", since the
destination binding is saved first. `prune_grace_period` is a Go
duration, so the README now says only which units it rejects. The
commands that run while a backup is in progress are a bulleted list.
Model: fable-5-1
Warnings and the per-file errors of `snapshot restore` are written to stdout
through `internal/ui`, so the stdout list names them and the redirect example
says a file captures them. Only `make test` and `make lint` run inside Docker;
`make test-coverage` runs on the host. Some `Makefile` targets (`build`,
`clean`, `deps`, `test-coverage`, `local`, `install`) call no `script/` entry,
so "Most" replaces "only".
Fixes the two findings of the fourth review of
#399.
Model: fable-5-1
The last sentence of "The private key" and the matching comment in
config.example.yml named only the restore machine. snapshot verify --deep
needs the key too. Both now name both commands. The "Everyday use" block
gains a comment saying its next three commands run on a machine that holds
the private key.
Model: fable-5-1
README.md: the opening no longer says the private key is needed only to
restore; a backup never needs it, a deep verify does.
ARCHITECTURE.md: the concurrency note says each open database uses one
connection, since the metadata export opens a second one.
docs/REPOSTRUCTURE.md: the size leak is about a secret near the
outsider's data in the same blob, not the same chunk, because all new
chunks of a blob share one zstd stream.
Every path used as link text in the four documents is in backticks.
Model: fable-5-1
A search of the four documents, the help text and config.example.yml
for the review's faults found these.
- prune help, and snapshot purge and snapshot list in README.md: a
snapshot is dropped or reported when its manifest is gone, not its
metadata or directory.
- README.md, version: the date printed is the commit date.
- README.md, Configuration: names the commands that start without a
config that loads.
- README.md, snapshot purge: "each such snapshot" named nothing.
- README.md and docs/DATAMODEL.md: a backup can upload unchanged
content again, so "uploads only what changed" was false.
- docs/REPOSTRUCTURE.md: db.zst.age holds more than one snapshot's
rows.
- ARCHITECTURE.md: "database records" read as a noun.
Model: fable-5-1
The review of the rewrite named these faults in other files. The same
sentences were still in two documents.
- docs/REPOSTRUCTURE.md, "The marker file": "index records" read as a
noun. It now says the next backup would not upload data that the
local index says is stored.
- docs/REPOSTRUCTURE.md, "Chunk boundaries use no secret": a chunk is
not always between 2.5 MB and 40 MB. Only a file's last chunk can be
smaller than a quarter of chunk_size.
- ARCHITECTURE.md, "Snapshot metadata": the export does not delete
everything outside the snapshot. It now says what is deleted, and
that local_meta and schema_migrations are kept.
Model: fable-5-1
snapshot list says why a remote-only snapshot is listed by its remote
key. snapshot verify says what access to the destination it needs. The
note on the age-keygen file is two sentences in snapshot verify and
snapshot restore. remote nuke says "backup destination store".
config.example.yml names the commands that run without a snapshot.
Only help strings and comments change.
Model: fable-5-1
The rewrite removed the README's roadmap, which TODO.md linked to. The
roadmap items are now under Future Steps in TODO.md, in place of the
link.
A comment in internal/storage/s3_test.go cited the README for the
destination layout. That layout is in docs/REPOSTRUCTURE.md.
Model: fable-5-1
ARCHITECTURE.md: the chunk size limits, the commit date, the cases in
which a backup skips its prune, and two sentences that read badly.
docs/DATAMODEL.md: which repository type serves which table, what the
metadata export deletes and keeps, the blobs table in a restore, and
when a chunk row is deleted.
docs/REPOSTRUCTURE.md: how an object's size relates to its contents,
what blob sizes can leak and when, and a sentence moved out of the
prune rules.
All three now say that prune drops a local snapshot whose manifest is
gone, not whose directory is gone.
Model: fable-5-1
The first sentence now names the licence, the kind of program, the
language and the author. The README gains a Design section, and its
sections carry the names REPO_POLICIES.md asks for: Rationale and
Getting started, which now holds the install command. Entrypoints opens
with the Scripts to Rule Them All statement.
Corrected: when a file is read again, the chunk size limits, and the
first prune step, which looks for a snapshot's manifest. Two sentences
that misparsed are reworded.
A test comment that named the README quickstart follows the rename.
Model: fable-5-1
Split sentences that carried two or three ideas, turned the cases in
which a backup does not find a mount point into a list, and named the
figures that remote info reports as unknown.
Model: fable-5-1
snapshot verify --help now says --deep needs the age private key, from
VAULTIK_AGE_SECRET_KEY or age_secret_key, as snapshot restore does.
Both now name the config key as well as the environment variable.
The help of snapshot create, list, purge, remove, prune, remote info
and remote nuke says which access to the destination it needs.
config.example.yml gains age_secret_key, and no longer tells the reader
to pipe age-keygen into grep, which discards the private key.
Only help strings and comments change.
Model: fable-5-1
Rewritten from scratch against the code on next: the layout of the
destination, what it protects, the accepted risks, the order of writes
and the rules that keep prune from deleting data in use.
Corrected: a restore needs only db.zst.age and the blobs it reads, not
the manifest. The snapshot ID form without a name is gone, since no
command creates one.
Model: fable-5-1
Rewritten from scratch against internal/database on next. Every table
and column in schema/001.sql is listed, with the indexes.
Added what the old text lacked: files.source_path, the local_meta and
schema_migrations tables, and five indexes. Corrected what it had
wrong: blobs.blob_hash holds a placeholder, not NULL, until the blob is
closed; blob_chunks rows are written when the blob is closed; queries
are parameterised, not prepared.
Model: fable-5-1
Rewritten from scratch against the code on next: the packages, the fx
wiring, and the steps of a backup, a restore, a verify and a prune.
Names that no longer exist in the code are gone: the S3 client as the
upload step, blob.BlobWithReader, PurgeSnapshots and the repository
interfaces. The storage layer is described through storage.Storer.
Model: fable-5-1
Rewritten from scratch against the code on next: what the program is,
how to use it, a command reference that says what each command needs,
the configuration reference as nested bullets, then development and
releasing.
snapshot verify --deep now says it needs the private key, from
VAULTIK_AGE_SECRET_KEY or age_secret_key, as snapshot restore does.
The roadmap section is removed: it described features that are not
built.
Model: fable-5-1
A failed command printed its error twice on stderr: as an ERROR log record and as the final ERROR: line the README documents. The command handlers, a restore failure that ends the command and a failed blob upload now log that error at info level or not at all, so without --verbose or --debug only the final line appears.
Model: opus-5-5
config set storage_url to an s3:// URL was refused, without saying why, when s3.access_key_id and s3.secret_access_key were not yet set. The refusal now names both keys and says to set them first. config set still refuses any change that leaves the config unable to load, and clearing a key gets the plain refusal.
Model: opus-5-5
A non-numeric chunk_size or blob_size_limit failed with an error that named neither the setting nor the value. Every size setting now fails at load with its name and the quoted value, for example chunk_size: "banana" is not a size; use a number of bytes or a unit such as 10MB. A value too large for a size says so, with the setting name.
Model: opus-5-5
A relative index_path, or VAULTIK_INDEX_PATH, was accepted and resolved against whatever directory vaultik ran in. Loading the config now refuses a relative or empty value, quoting it, with the same wording as the file:// check; ~ is still expanded first.
Model: opus-5-5
The default config backs up the home directory, which holds vaultik's local index and PID file, so a backup stored a torn copy of the index it was writing. The scan now skips the index, its SQLite side files and the PID file whatever the config says, including when index_path or its directory is a symlink. database purge removes the same list of index files.
Model: opus-5-5
A binary from go install, the README's first command, reported itself as version dev, a development build, although Go had recorded the tag it was built from. When the linker flags leave the version, commit or commit date unset, they are now filled in from the Go build info, so such a binary reports its tag and counts as a release. Builds through make, script/docker and goreleaser are unchanged.
Model: opus-5-5
Pressing Ctrl-C or sending SIGTERM during a backup logged the interrupted blob upload as an ERROR before the interrupted message. The upload step now skips that record when the backup's context has been cancelled, whatever error the storage returned. An upload that fails for any other reason is still logged.
Model: opus-5-5
With a private key the backup was not encrypted to, restore and deep verify failed with age's "no identity matched any of the recipients" behind a doubled prefix. They now say in one sentence that the key from VAULTIK_AGE_SECRET_KEY or age_secret_key is not one of the keys the snapshot, or a blob, was encrypted to. A blob reused after a recipient change gets the same message.
Model: opus-5-5
A misspelled or unknown config key was accepted silently, so a typo in exclude backed up everything it meant to leave out. Loading now refuses a key that no setting has, at any depth, naming its dotted path and, when it is in the file, its line; keys brought in by ${...} references are checked too. config set refuses an unknown key. A top-level env: section stays accepted, since the config library uses it. Anchors that contain themselves fail with an error.
Model: opus-5-5
A blob that is not a valid age stream made restore print age's error, which quotes the blob's raw bytes, once for every file that needed it. The error now says the blob is not a valid age stream and names its hash, without quoting any bytes. Under --skip-errors a blob that failed is reported once, with the number of files it affected, and is not downloaded again for each file.
Model: opus-5-5
A first backup that failed before writing anything, on a missing bucket, a refused credential or missing recipients, still bound the local index to the configured destination, so correcting a mistyped storage_url was refused afterwards. A backup now records the binding only after it has written the marker file at the destination. prune, snapshot purge, snapshot remove and remote nuke only refuse a different binding and leave an unbound index unbound.
Model: opus-5-5
snapshot restore and snapshot verify given an ID that names no snapshot in the local index or at the destination failed with "object not found" on a hashed metadata path. They now look the ID up the way snapshot remove does and say that no such snapshot exists, before printing anything else.
Model: opus-5-5
prune, snapshot purge, snapshot remove and database delete exited 0 when their confirmation prompt was declined or left unanswered, so a script could not tell that nothing was deleted. They now end with "deletion not confirmed" and exit 1. The [y/N] prompts accept y or Y; database delete still asks for yes.
Model: opus-5-5
The internal/vaultik tests took 36 to 56 seconds under go test -race and ran past the 90-second timeout when the host was busy. Tests that built the same backup now copy one shared backup, and the interruption tests use smaller sources. No assertion is weakened and no test is removed. Measured back to back on next under the same load, the package takes 55% of its former time.
Model: opus-5-5
sneak ruled that TODO is not a history document. docs/TODO.md drops the Completed Steps section and every other record of past work, and keeps the open steps. The Workflow step that moved a finished Next Step into Completed Steps now deletes it.
Model: opus-5-5
REPO_POLICIES.md names prettier as the Markdown formatter and keeps documents out of the root. script/fmt and script/fmt-check now run prettier over all Markdown (4-space indents, proseWrap always) besides gofmt, with node and yarn installed by script/bootstrap as in sneak/prompts. All Markdown is reformatted with no text change. ARCHITECTURE.md and TODO.md move to docs/, config.example.yml to configs/, and every reference follows.
Model: opus-5-5
vaultik version printed globals.CommitDate, the date of the commit the binary was built from, labelled "build date", and the startup banner called it "built on". Both now say "commit date". The test sets Commit and CommitDate to distinct values and checks the label, padding and value of every report line.
Model: opus-5-5
The status and progress lines of remote info and snapshot verify (shallow and deep) were written with stdoutf, so they had no marker and no color, unlike every other command's. They now go through Beginf, Completef and Progressf, as the README's output style section says. The reports stay plain, and --quiet and --json still drop these lines.
Model: opus-5-5
The default config backed up /Applications on every system. That directory exists only on macOS, so on Linux the first snapshot create stopped on the missing path. config init now writes the apps snapshot only when runtime.GOOS is darwin; the macOS file is unchanged.
Model: opus-5-5 (implementation, review); opus-5-5 (merge)
Two doc comments described behaviour that no longer exists. The comment on `ExportSnapshotMetadata` described an SQL dump uploaded under `snapshots/`; it now lists the steps the function performs and the two keys it writes. The comment on `EnsureStorageBinding` named `vaultik database purge`; the command is `vaultik database delete`. Comments only.
Model: opus-5-5
snapshot create --prune purged the older snapshots holding a volume's
files when the volume, mounted below a configured path, was not
mounted: its empty mount point read as deleted files.
The scan now records in the local index each directory at or below a
configured path that the walk found an entry in and that, once listed,
is on a different device from its parent; listing mounts an
automounted volume. Under --prune, when an earlier snapshot of that
name recorded such a directory and this run does not, the run skips
the prune, keeps every older snapshot and exits non-zero, naming the
directory. For a volume retired on purpose, snapshot purge removes the
older snapshots.
Model: opus-5-5
A backup that stopped between uploading db.zst.age and its manifest
left that directory under metadata/ on the destination for good:
prune deleted nothing there.
The blob step of prune, which snapshot create --prune also runs, now
deletes each directory under metadata/ that has no manifest.json.zst,
that no local snapshot record claims, and all of whose objects are
older than prune_grace_period; an object listed with no time keeps its
directory, and only names that are remote keys are deleted. The prompt
and the --force output name each such directory, the --json result
counts them (metadata_dirs_deleted, metadata_dirs_failed), and a failed
delete fails the run. Every description of the grace period now
covers these directories.
Model: opus-5-5
snapshot create --prune of a snapshot whose path was an empty
directory, as the mount point of a volume that is not mounted is,
recorded an empty snapshot and then purged the older snapshots that
held the data, with their blobs.
The scan now counts the entries it finds in each configured path.
Under --prune, a run in which a path of a snapshot it made held
nothing skips the prune, keeps every older snapshot, names each such
path with its snapshot, and exits non-zero, as
#382 does for unreadable
files. Only the current run is looked at. Without --prune this check
does nothing. A configured path that is a symlink whose target is
gone now fails every run, with or without --prune.
Model: opus-5-5
A source host whose credentials allow only PUT, which the documents
say is enough to back up, failed every backup after its first: the
marker check looks the marker up with an S3 HEAD, which those
credentials are refused, and any error other than not found was
fatal.
Such an error now means the check could not be made: it is logged at
info level, so --cron stays silent, and the command goes on. Not found
still stops it. The documents now say so, and that such a host does
not notice an emptied destination, so vaultik database delete must be
run on it after the destination is emptied. The fault-injecting test
store gains a Stat hook for the test.
Model: opus-5-5
A backup uploads its blobs before the manifest that lists them, and the
process lock covers one user on one machine, so a prune from another host
or user sharing the destination deleted a running backup's blobs.
Prune now deletes an unreferenced blob only once its listed time is older
than the new config key prune_grace_period (default 24h; a negative value
is refused). A newer blob, or one listed with no time, is kept and counted
(blobs_too_new). The rclone backend takes each object's time from the
listing (use_server_modtime).
Decision: a blob a backup reuses from an earlier run that stopped before
its manifest keeps its old time and is not protected; the docs and config
comments say so.
Model: opus-5-5
Adds a Completed Steps entry, newest first, for every issue from the
defect hunt (#288) that a merged
fix closed and that had none: the hunt's own issues, those filed while
fixing them, and the owner's ruling on
#239. Each entry says in one or
two sentences what was wrong and what changed; the detail is in each
fix's commit. Next Step now says the hunt's fixes have landed.
Judgement call: #332, closed as
covered by the fix for #296, is
named in that entry rather than given its own.
Judgement call: #351, a Docker
build fix filed while gating these fixes, gets an entry.
Model: opus-5-5
sneak
clawbot
clawbot