Fix nine review findings in the documents and help text
check / check (push) Failing after 19m7s

- snapshot verify and snapshot restore also list the destination for a
  snapshot ID that is not in the local index: README, REPOSTRUCTURE,
  ARCHITECTURE, DATAMODEL and the two commands' help.
- prune on a separate machine deletes only unused data and does not
  remove old snapshots.
- The --quiet usage string names everything the flag leaves out.
- A warning written as a log record stays on stderr.
- config set on a config that does not load refuses three things.
- No test is held back by a build tag.
- A plain docker build . adds no -dirty.
- The backup's order of work records directories.
- TODO.md says what parallel downloads would speed up and names
  restorePlan and restoreSweeper.

Model: fable-5-1
This commit is contained in:
clawbot committed 2026-11-11 07:11:05 +00:00
1 parent 1fcdfcc1c6
commit 2bf51d65e3
8 files changed
+52 -31

No files matched your search

+24 -12
View File
@@ -308,8 +308,11 @@ is `vaultik config set compression_level 99`. The file is then left unchanged.
A config that does not load yet is checked less strictly. An example is the file
`config init` writes, which has no `storage_url`. There the command refuses only
two things: an unknown key, and a value of the wrong type, such as a word where
a number belongs.
three things:
- an unknown key
- a value of the wrong type, such as a word where a number belongs
- a `${ENV:NAME}` value whose variable is unset or empty
### `snapshot create`
@@ -477,7 +480,8 @@ Without `--deep`, it checks two things:
- The snapshot's encrypted database is present.
This reads no blob contents and needs no private key. It needs permission to
read at the destination, and to list it when you give a remote key.
read at the destination, and to list it when you give a remote key, or a
snapshot ID that is not in the local index.
Flags:
@@ -583,7 +587,7 @@ It needs:
- The age private key, as for `snapshot verify --deep`. See
[The private key](#the-private-key).
- Permission to read at the destination, and to list it when you give a remote
key.
key, or a snapshot ID that is not in the local index.
- Write access to `<target-dir>`. The command creates the directory when it does
not exist.
- Free temporary space for the decrypted blobs. See
@@ -1066,12 +1070,14 @@ An rclone remote that can move objects also needs permission to read and delete.
An upload there looks up the object it replaces and deletes it. It stops when
the look-up is refused.
`snapshot verify` and `snapshot restore` list only when you give a remote key.
`snapshot remove --local-only` does not reach the destination.
`snapshot remove --dry-run` deletes nothing.
`snapshot verify` and `snapshot restore` list only when you give a remote key,
or a snapshot ID that is not in the local index. `snapshot remove --local-only`
does not reach the destination. `snapshot remove --dry-run` deletes nothing.
A machine that runs `snapshot create --prune` can delete its own backups. To
keep it to write-only credentials, run `prune` from a separate, trusted machine.
`prune` there deletes only data that no snapshot uses. It does not remove old
snapshots.
A backup tries to look up the marker file and each blob before it writes. On S3
it goes on when the destination refuses those look-ups.
@@ -1095,7 +1101,8 @@ not exist, commands treat the destination as not present:
- `prune`
- `snapshot purge`
- `remote nuke`
- `snapshot verify` and `snapshot restore`, when given a remote key
- `snapshot verify` and `snapshot restore`, when given a remote key, or a
snapshot ID that is not in the local index
- `snapshot list` shows the local index alone.
- `snapshot remove` deletes nothing at the destination.
@@ -1210,8 +1217,10 @@ stderr carries the rest:
- log records, including those from `--verbose` and `--debug`
- the error that a failed command ends with
So `vaultik snapshot list > out.txt` captures the list and any warnings. The
banner, log records and a final error stay on your terminal.
So `vaultik snapshot list > out.txt` captures the list and the warnings printed
below it. The banner, log records and a final error stay on your terminal. A
warning written as a log record, such as the one about a config file others can
read, stays there too.
Log records are colored lines when stderr is a terminal. They are JSON, one
object per line, when stderr is redirected.
@@ -1409,8 +1418,8 @@ call them.
Every `docker build` in these scripts uses `--no-cache`. A cached layer would
skip the tests or the linter and still report success.
The test suite has no separate integration target. Every test runs every time.
Each test binary is stopped after 90 seconds.
The test suite has no separate integration target. No test is held back by a
build tag. Each test binary is stopped after 90 seconds.
To change the Go version, change these together:
@@ -1474,6 +1483,9 @@ The version a binary reports comes from git. It is the output of
`script/version` gives this value to the `Makefile`. `script/docker` and
`script/cibuild` compute it themselves.
A plain `docker build .` adds no `-dirty`. The `Dockerfile` runs `git describe`
without `--dirty`.
`make` outside a git checkout, and `docker build .` without `.git`, stamp `dev`.
An image built by `script/docker` or `script/cibuild` outside a git checkout
reports `unknown`.
+5 -4
View File
@@ -291,10 +291,11 @@ therefore marks a backup that did not finish.
- With `--verify`, `verifyRestoredFiles` reads each restored regular file and
checks its chunk hashes.
Given a snapshot ID, the restore does not read the local index. Given a remote
key, it reads the local index only to check the destination marker. It does not
read the manifest. It finds each file's chunks and blobs through the downloaded
database.
Given a snapshot ID, the restore looks it up in the local index. When it is not
there, the restore checks the destination marker and lists the snapshot's
directory. Given a remote key, it reads the local index only to check the
destination marker. It does not read the manifest. It finds each file's chunks
and blobs through the downloaded database.
### Checks on the downloaded database
+4 -3
View File
@@ -299,6 +299,7 @@ A backup works on the tables in this order:
- It walks each path. It compares each regular file and symlink with its row in
`files`.
- It links each unchanged file to the snapshot in `snapshot_files`.
- It writes a `files` row and a `snapshot_files` row for each directory.
- It records each mount point in `snapshot_mount_points`.
- It reads each changed regular file and stores it. See below. A changed symlink
is recorded without being read.
@@ -362,9 +363,9 @@ still be in the `-wal` file beside it.
### Restore
A restore reads no file, chunk or blob rows from the local index. Given a remote
key, it reads `snapshots` and `local_meta` there to check the destination
marker.
A restore reads no file, chunk or blob rows from the local index. It reads
`snapshots` and `local_meta` there, to look up a snapshot ID and to check the
destination marker.
- It downloads and decrypts `db.zst.age` into a temporary directory.
- It opens that file read-only. It applies no migration to it.
+6 -3
View File
@@ -59,7 +59,8 @@ not exist, commands treat the destination as not present:
- `prune`
- `snapshot purge`
- `remote nuke`
- `snapshot verify` and `snapshot restore`, when given a remote key
- `snapshot verify` and `snapshot restore`, when given a remote key, or a
snapshot ID that is not in the local index
- `snapshot list` shows the local index alone.
- `snapshot remove` deletes nothing at the destination.
@@ -334,12 +335,14 @@ An upload there looks up the object it replaces and deletes it. It stops when
the look-up is refused.
`snapshot verify` and `snapshot restore` list only when they are given a remote
key. `snapshot remove --local-only` does not reach the destination.
key, or a snapshot ID that is not in the local index.
`snapshot remove --local-only` does not reach the destination.
`snapshot remove --dry-run` deletes nothing.
`snapshot create --prune` runs a prune on the backed-up machine. That machine
then holds credentials that can delete its own backups. To keep it to write-only
credentials, run `prune` from a separate, trusted machine.
credentials, run `prune` from a separate, trusted machine. `prune` there deletes
only data that no snapshot uses. It does not remove old snapshots.
### New recipients cannot read old data
+5 -4
View File
@@ -41,10 +41,11 @@ Work planned after 1.0, loosely ordered by priority.
- **Parallel blob downloads during restore.** A restore downloads one blob at a
time now. With a fast S3 endpoint and a multi-core machine, restore speed is
bound by the fetch, decryption and decompression of each blob. Running several
of those in parallel against the disk cache would close most of the remaining
gap. This needs to interact correctly with the locality planner and the
sweeper.
bound by the fetch, decryption and decompression of each blob. Fetching,
decrypting and decompressing several blobs at once would make such a restore
faster. The change must work with `restorePlan`, which orders the files, and
with `restoreSweeper`, which deletes a cached blob once no remaining file
needs it.
- **Bandwidth limiting (`--bwlimit`).** For both upload and download. It is
useful for backing up over a shared link. It is tricky to make work correctly
with parallel downloads.
+3 -2
View File
@@ -58,8 +58,9 @@ an rclone remote. The backed-up machine needs no private key: only
cmd.PersistentFlags().BoolVar(&rootFlags.Debug, "debug", false,
"Enable debug output")
cmd.PersistentFlags().BoolVarP(&rootFlags.Quiet, "quiet", "q", false,
"Suppress status output (warnings, errors, JSON documents, "+
"'config get' values and confirmation prompts are still printed)")
"Leave out the banner, status lines, tables and reports (warnings, "+
"errors, JSON documents, 'config get' values and confirmation "+
"prompts are still printed)")
cmd.PersistentFlags().BoolVar(&rootFlags.SkipErrors, "skip-errors", false,
"Skip files that cannot be read when creating a snapshot, or "+
"that cannot be restored when restoring, instead of aborting "+
+2 -1
View File
@@ -252,7 +252,8 @@ snapshot's encrypted database is present. It does not read blob
contents and does not need the age private key.
Needs permission to read the backup destination store, and to list it
when a remote key is given.
when given a remote key, or a snapshot ID that is not in the local
index.
Use --deep to download, decrypt, and re-hash every blob to detect
corruption -- integrity, not who wrote it.
+3 -2
View File
@@ -49,8 +49,9 @@ typing it, so that it does not land in your shell history:
export VAULTIK_AGE_SECRET_KEY="$(cat vaultik_backup_private_key.txt)"
Also needs permission to read the backup destination store (and to list
it when a remote key is given), and free space in the temporary directory
for the decrypted blobs.
it when given a remote key, or a snapshot ID that is not in the local
index), and free space in the temporary directory for the decrypted
blobs.
Examples:
# Restore entire snapshot