sneak/vaultik
1
0 Fork 0
Code Issues 6 Pull Requests 5 Actions Packages Projects Releases Wiki Activity

Read all five documents against the code and fix what the sixth review found
check / check (push) Successful in 12m5s
Details

Browse Source 
Fixes the four findings of the sixth review: `snapshot remove` with
`--local-only`, the two comma-chained permission lists, six undefined terms,
and the sentence about an unfinished snapshot in the local index.

Then reads every sentence of `README.md`, `ARCHITECTURE.md`,
`docs/DATAMODEL.md`, `docs/REPOSTRUCTURE.md` and the comments in
`config.example.yml` against the code for four faults: a command's needs or
behaviour stated without regard to a flag, a false absolute, three or more
things chained with commas, and a technical word not defined on first use.
Each sentence changed is listed on the pull request.

The help string of `snapshot remove --dry-run` now says it needs no
permission to delete.

Model: fable-5-1
This commit is contained in:
clawbot committed 2026-11-09 01:57:09 +00:00
1 parent a133b8838e
commit 46b2a9fc78
6 files changed
+341 -193
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+183 -92
View File
@@ -3,7 +3,7 @@
`vaultik` is an MIT-licensed command-line tool for incremental, encrypted
backups, written in Go by [@sneak](https://sneak.berlin). It encrypts your files
with a public key and uploads them to an S3-compatible object store, a local
directory or an rclone remote.
directory or any storage service that [rclone](https://rclone.org/) supports.
The machine being backed up holds only the public key. It cannot decrypt its own
backups. You keep the private key somewhere else. A backup never needs it.
@@ -25,13 +25,14 @@ It was written to meet these requirements together:
## Features
- Encryption with [age](https://age-encryption.org/) (X25519 and
ChaCha20-Poly1305), to one or more public keys.
- Encryption with [age](https://age-encryption.org/), to one or more public
keys. age uses X25519 keys and the ChaCha20-Poly1305 cipher.
- Incremental backups. A file is read again only when it has changed, or when
part of its content has to be stored again.
- Deduplication. Identical content is stored once, across files and across
snapshots.
- zstd compression at a level you choose.
- Compression with [zstd](https://github.com/facebook/zstd), at a level you
choose.
- Small files are packed together into large objects. This keeps the number of
S3 requests low.
- File contents, names, paths and metadata are encrypted at the destination.
@@ -91,12 +92,13 @@ vaultik snapshot list
The default config is written for macOS. It backs up your home directory and
`/Applications`, with a list of excludes. A configured path that does not exist
stops the backup. On another system, run `vaultik config edit` first and change
the `snapshots` section.
stops the backup, unless you give `--skip-errors`. On another system, run
`vaultik config edit` first and change the `snapshots` section.
On macOS, grant your terminal app Full Disk Access before the first backup. The
setting is under System Settings, Privacy & Security, Full Disk Access. Without
it the backup stops with a permission error on a protected directory.
it the backup stops with a permission error on a protected directory, unless you
give `--skip-errors`.
## Everyday use
@@ -155,7 +157,7 @@ Use that remote key in place of the snapshot ID:
```sh
export VAULTIK_AGE_SECRET_KEY="$(cat vaultik_backup_private_key.txt)"
# Restore everything, then check the chunk hashes of every restored file.
# Restore everything, then check the chunk hashes of the restored files.
vaultik snapshot restore --verify <remote-key> ~/vaultik-restore
# Optionally, also check every blob of the snapshot at the destination.
@@ -194,10 +196,14 @@ vaultik version
### What every command needs
`config init`, `completion` and `version` need nothing.
`completion` and `version` need nothing. `config init` needs write access to the
config directory.
`config edit`, `config get` and `config set` need a config file that exists. It
does not have to be complete.
These commands need a config file that exists, even an incomplete one:
- `config edit`
- `config get`
- `config set`
Every other command needs a config file that loads. See
[Configuration](#configuration) for the checks.
@@ -219,24 +225,34 @@ sets file owners only when it runs as root.
[Where the config file is](#where-the-config-file-is).
- `--verbose`, `-v`: write informational log records to stderr.
- `--debug`: write debug log records to stderr.
- `--quiet`, `-q`: print only warnings, errors and the output a script reads.
See [Output](#output).
- `--quiet`, `-q`: leave out the banner, the status lines, the tables and the
reports. See [Output](#output).
- `--skip-errors`: skip a file that cannot be read during a backup, or cannot be
restored during a restore. Without it the command stops at the first such
file.
`--quiet` and `--cron` override `--verbose` and `--debug`.
`--skip-errors` applies to `snapshot create` and `snapshot restore` only. A
command that skipped anything still finishes, and then exits non-zero.
### Durations
`--keep-newer-than`, `--older-than` and `prune_grace_period` take a duration.
These take a duration:
- `--keep-newer-than`
- `--older-than`
- `prune_grace_period`
These units are accepted:
- `d`: days.
- `w`: weeks.
- `mo`: months of 30 days.
- `y`: years of 365 days.
- `h`, `m` and `s`: hours, minutes and seconds, as in `36h` or `90m`.
- `h`: hours, as in `36h`.
- `m`: minutes, as in `90m`.
- `s`: seconds.
`m` means minutes and `mo` means months. Days, weeks, months and years take
whole numbers and can be combined, as in `2w3d`.
@@ -249,9 +265,9 @@ Writes a default config file, with a comment that explains each setting it
lists.
- It writes to the path from `--config`, or else from `VAULTIK_CONFIG`.
- Otherwise it writes to `config.yml` in the platform config directory. That is
`~/.config/vaultik/` on Linux and `~/Library/Application Support/vaultik/` on
macOS.
- Otherwise it writes to `config.yml` in the platform config directory. By
default that is `~/.config/vaultik/` on Linux and
`~/Library/Application Support/vaultik/` on macOS.
- As root it writes to `/etc/vaultik/config.yml` instead.
- It creates the file with mode `0600`, because the file can hold credentials.
- It refuses to overwrite a file that exists.
@@ -275,7 +291,8 @@ Sets one value in the config file. The key is a dotted path, as in
- A key that does not exist yet is created. A missing parent becomes a list when
the next part of the path is a number, and a map otherwise.
- In a list, the index one past the end appends an entry.
- The command prints the key it set, and never the value.
- The command prints the key it set, and not the value. A refusal for a value
of the wrong type quotes the beginning of that value.
- It sets the file's mode to `0600` when group or others could read it.
The command refuses a change that would stop the config from loading. An example
@@ -301,7 +318,8 @@ It needs:
It does not need the private key.
A file is read again when any of these holds:
A file is read when the local index does not know it, or when any of these
holds:
- Its size has changed since the last backup.
- Its modification time has changed.
@@ -383,9 +401,9 @@ config.
### `snapshot list`
Lists the snapshots in the local index and the snapshots at the destination, as
one table. Before the table, the command prints the snapshot names and paths in
the config.
Lists the finished snapshots in the local index and the snapshots at the
destination, as one table. Before the table, the command prints the snapshot
names and paths in the config.
It needs permission to list and read at the destination. It needs no private
key.
@@ -402,7 +420,8 @@ The table has these columns:
A snapshot at the destination that is not in the local index is shown as
`<remote only:...>`. The dots stand for the first 12 characters of its remote
key. Its last two columns read `<remote only>`, because those sizes come from
the local index. Its timestamp is the time its backup finished.
the local index. Its timestamp is the time its manifest was written, at the end
of its backup.
The command reports these below the table:
@@ -424,7 +443,8 @@ marker file although the local index holds snapshots.
- `remote_key`: the full 64-character remote key.
- `timestamp`
- `compressed_size`
- `uncompressed_size` and `new_chunk_size`: left out for a remote-only snapshot.
- `uncompressed_size` and `new_chunk_size`: left out when zero, and for a
remote-only snapshot.
- `locally_tracked`: whether the snapshot is in the local index.
- `remote_present`: whether the snapshot is at the destination. It is `null`
when the destination could not be listed.
@@ -469,7 +489,8 @@ variable wins when both are set. See [The private key](#the-private-key).
- It downloads and decrypts every blob. It checks the hash of each chunk, and
the hash that names the blob.
It stops at the first blob that fails.
The presence check counts every missing or wrong-size blob. The download check
stops at the first blob that fails.
`--deep` proves that the stored data is intact and decrypts with your key. It
does not prove who wrote the data.
@@ -510,20 +531,22 @@ cannot be listed or has no marker file.
Removes one snapshot. The argument must be a snapshot ID. A remote key is not
accepted.
It needs the [lock](#locking), and permission to list and delete at the
destination. It needs no private key.
It needs the [lock](#locking). It needs permission to list and delete at the
destination, unless you give `--local-only`. It needs no private key.
Flags:
- `--force`, `-f`: do not ask for confirmation.
- `--dry-run`: say what would be removed, and remove nothing.
- `--dry-run`: say what would be removed, and remove nothing. It needs no
permission to delete.
- `--local-only`: remove the snapshot from the local index only. The destination
is not changed.
- `--json`: print the result as JSON. This also skips the confirmation.
The command removes the snapshot from the local index. It also deletes the
snapshot's directory under `metadata/` at the destination. Its blobs stay. Run
`vaultik prune` afterwards to delete the blobs that no snapshot uses any more.
The command removes the snapshot from the local index. Without `--local-only`,
it also deletes the snapshot's directory under `metadata/` at the destination.
Its blobs stay. Run `vaultik prune` afterwards to delete the blobs that no
snapshot uses any more.
An ID that is in neither the local index nor the destination is an error. With
`--local-only`, the snapshot must be in the local index.
@@ -586,8 +609,8 @@ Restore into a new directory that only you can write, not a shared one such as
Flags:
- `--verify`: after the restore, read every restored file and check its chunk
hashes.
- `--verify`: after the restore, read every restored regular file and check its
chunk hashes.
With `--skip-errors`, a file that cannot be restored is reported and skipped,
and the command exits non-zero. `--verify` does not check a skipped file.
@@ -666,12 +689,13 @@ Reports what the destination holds:
It needs permission to list and read at the destination. It needs no private
key.
When a manifest cannot be read, the figures for blobs that no manifest lists are
reported as unknown. That snapshot's own blob count and size are unknown too.
The command skips a name under `metadata/` that is not a remote key, with a
warning. It does not count a file under `blobs/` whose name is not a blob hash.
When a manifest cannot be read, that snapshot's blob count and size are unknown.
The figures for blobs that no manifest lists are unknown when a manifest cannot
be read or sits under a skipped name.
`--json` prints the report as JSON. Unknown figures are `null`.
`unreadable_manifests` lists the remote key of each manifest that could not be
read. `skipped_manifest_count` counts the manifests under skipped names.
@@ -715,7 +739,12 @@ Use this command after you point the config at a different destination. See
### `completion <shell>`
Prints a completion script for `bash`, `zsh`, `fish` or `powershell`.
Prints a completion script for one of these shells:
- `bash`
- `zsh`
- `fish`
- `powershell`
```sh
# zsh, for the current session
@@ -736,31 +765,51 @@ vaultik completion fish > ~/.config/fish/completions/vaultik.fish
### `version`
Prints the version, the commit, the commit date and the platform. It labels the
commit date `build date`. It also says when the binary is a development build,
not a release.
Prints these:
- the version
- the commit
- the commit date, labelled `build date`
- the Go version
- the platform
- the author
- the homepage
- the license
It also says when the binary is a development build, not a release.
## Configuration
The config file is YAML. `vaultik config init` writes one to start from, and
[`config.example.yml`](config.example.yml) is a longer example.
A command refuses to start when the config does not load. The `config` commands,
`completion` and `version` are the exceptions. The config loads when all of
these hold:
A command refuses to start when the config does not load. These commands are
the exceptions:
- the `config` commands
- `completion`
- `version`
The config loads when it is valid YAML and all of these hold:
- Every entry in `age_recipients` is an age X25519 public key.
- `snapshots` has at least one name, and every name has at least one path.
- A destination is set, with credentials when it is S3.
- A destination is set. Its `storage_url` has one of the three schemes under
[Destinations](#destinations). An S3 destination has credentials.
- Every `${ENV:NAME}` names an environment variable with a non-empty value.
- Every other setting below is within its limits.
`vaultik` warns when group or others can read the config file.
Every command that needs the local index warns when group or others can read the
config file.
A value written as `${ENV:NAME}` is replaced by the environment variable `NAME`.
This keeps a credential out of the file.
A leading `~` is expanded to your home directory in snapshot paths, in
`index_path` and in a `file://~/...` destination.
A leading `~` is expanded to your home directory in these places:
- snapshot paths
- `index_path`
- a `file://~/...` destination
### Where the config file is
@@ -768,11 +817,13 @@ A leading `~` is expanded to your home directory in snapshot paths, in
- the path given with `--config`
- the path in the `VAULTIK_CONFIG` environment variable
- `config.yml` in the platform config directory, which is `~/.config/vaultik/`
on Linux and `~/Library/Application Support/vaultik/` on macOS
- `config.yml` in the platform config directory, which by default is
`~/.config/vaultik/` on Linux and `~/Library/Application Support/vaultik/` on
macOS
- `/etc/vaultik/config.yml`
A path from `--config` or `VAULTIK_CONFIG` must exist.
A command that reads the config fails when the path from `--config` or
`VAULTIK_CONFIG` does not exist. `config init` creates the file there.
### Settings
@@ -806,7 +857,8 @@ A path from `--config` or `VAULTIK_CONFIG` must exist.
- `s3.region`
- The S3 region. Default: `us-east-1`.
- `s3.part_size`
- The size of each part of a multipart upload. Default: `5MiB`.
- A large blob is uploaded to S3 in parts. This is the size of each part.
Default: `5MiB`.
- It must be between `5MiB` and `5GiB`.
- A blob too large for 10,000 parts of this size is uploaded in larger
parts.
@@ -826,8 +878,11 @@ A path from `--config` or `VAULTIK_CONFIG` must exist.
- It must be at least four times `chunk_size`.
- `compression_level`
- The zstd compression level, from `1` to `19`. Default: `3`.
- Levels `1` to `2`, `3` to `5`, `6` to `9` and `10` to `19` form four
groups. Levels in one group compress the same way.
- Levels in one of these groups compress the same way:
- `1` to `2`
- `3` to `5`
- `6` to `9`
- `10` to `19`
- `prune_grace_period`
- How old unused data must be before `prune` deletes it. Default: `24h`.
- It applies to a blob that no manifest lists. It also applies to every
@@ -839,8 +894,8 @@ A path from `--config` or `VAULTIK_CONFIG` must exist.
- Only the part before the first `.` is used.
- `index_path`
- The path of the local index.
- Default: `index.sqlite` in the platform data directory. That is
`~/.local/share/vaultik/` on Linux and
- Default: `index.sqlite` in the platform data directory. By default that
is `~/.local/share/vaultik/` on Linux and
`~/Library/Application Support/vaultik/` on macOS.
Sizes take a number of bytes or a unit. `MB` and `GB` are powers of 1000. `MiB`
@@ -856,7 +911,9 @@ and `GiB` are powers of 1024.
destination.
- `EDITOR`: the editor that `config edit` starts.
- `TMPDIR`: the directory for temporary files.
- `NO_COLOR`: when set, status lines have no color.
- `NO_COLOR`: a non-empty value turns off color in status lines.
- `XDG_CONFIG_HOME` and `XDG_DATA_HOME`: move the platform config and data
directories.
- `VAULTIK_CPUPROFILE`: write a CPU profile to this path. For debugging.
- `VAULTIK_MEMPROFILE`: write a heap profile to this path on exit. For
debugging.
@@ -966,15 +1023,26 @@ A leftover `.partial` file is ignored. You can delete it.
### Destination permissions
Each command needs different permissions at the destination.
Commands need these permissions at the destination:
- `snapshot create` needs only permission to write objects. A machine you back
up can therefore hold credentials that can neither read nor delete.
- `snapshot list`, `snapshot verify`, `snapshot restore` and `remote info` list
and read.
- `snapshot purge`, `snapshot remove` and `remote nuke` list and delete.
- `snapshot create` without `--prune` needs only permission to write objects. A
machine you back up can therefore hold credentials that can neither read nor
delete.
- These commands list and read:
- `snapshot list`
- `snapshot verify`
- `snapshot restore`
- `remote info`
- These commands list and delete:
- `snapshot purge`
- `snapshot remove`
- `remote nuke`
- `prune` and `snapshot create --prune` list, read and delete.
`snapshot verify` and `snapshot restore` list only when you give a remote key.
`snapshot remove --local-only` does not reach the destination.
`snapshot remove --dry-run` deletes nothing.
A machine that runs `snapshot create --prune` can delete its own backups. To
keep it to write-only credentials, run `prune` from a separate, trusted machine.
@@ -991,12 +1059,15 @@ unplugged volume leaves an empty directory at its mount point. Without the
marker, that directory would look like a destination whose snapshots are all
gone.
The check applies while the local index holds snapshots. When the destination
then answers that the marker does not exist, commands treat the destination as
not present:
The check applies while the local index holds snapshots and belongs to the
configured destination. When the destination then answers that the marker does
not exist, commands treat the destination as not present:
- `snapshot create` refuses to write to it.
- `prune`, `snapshot purge` and `remote nuke` fail.
- These commands fail:
- `prune`
- `snapshot purge`
- `remote nuke`
- `snapshot list` shows the local index alone.
- `snapshot remove` deletes nothing at the destination.
@@ -1025,7 +1096,11 @@ different destination:
- `remote nuke`
The destination that is compared is `storage_url`. When `storage_url` is not
set, it is `s3.endpoint`, `s3.bucket` and `s3.prefix`.
set, these three settings are compared instead:
- `s3.endpoint`
- `s3.bucket`
- `s3.prefix`
There are two ways out:
@@ -1057,8 +1132,7 @@ The new snapshot cannot be restored afterwards.
## Locking
Commands that change data take a lock, so that only one of them runs at a time.
These commands take it:
Five commands take a lock, so that only one of them runs at a time:
- `snapshot create`
- `snapshot purge`
@@ -1068,9 +1142,9 @@ These commands take it:
A second one does not wait. It exits at once with an "already running" error.
The lock is the file `vaultik.pid` in the platform data directory. That is
`~/.local/share/vaultik/` on Linux and `~/Library/Application Support/vaultik/`
on macOS.
The lock is the file `vaultik.pid` in the platform data directory. By default
that is `~/.local/share/vaultik/` on Linux and
`~/Library/Application Support/vaultik/` on macOS.
The lock covers one user on one machine. It does not stop another user or
another machine from writing to the same destination. See
@@ -1111,7 +1185,8 @@ banner, log records and a final error stay on your terminal.
Log records are colored lines when stderr is a terminal. They are JSON, one
object per line, when stderr is redirected.
Status lines are colored only on a terminal, and never when `NO_COLOR` is set.
Status lines are colored only on a terminal, and never when `NO_COLOR` has a
non-empty value.
### `--json`
@@ -1183,8 +1258,8 @@ destination, what it protects and the risks it accepts.
## Limitations
- **No extended attributes.** ACLs, macOS Finder metadata, quarantine flags and
SELinux labels are not backed up.
- **No extended attributes.** Access control lists, macOS Finder metadata,
quarantine flags and SELinux labels are not backed up.
- **No hard links.** Two hard links to one file are backed up as two files. The
content is stored once, but a restore creates two separate files.
- **No sparse files.** A sparse file is backed up at its full size, zeros
@@ -1260,25 +1335,33 @@ call them.
- `script/test`: runs the whole test suite, by building the `test` stage of the
`Dockerfile`.
- `script/lint`: runs the linter, by building the `lint` stage of the
`Dockerfile`. The linter never runs on your machine.
`Dockerfile`. The linter runs only inside Docker.
- `script/lint-fix`: applies the linter's automatic fixes to your files. It
needs a local Docker daemon. Run `script/lint` afterwards.
- `script/fmt`: formats the Go code.
- `script/fmt-check`: checks the formatting and changes nothing.
- `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`.
- `script/precommit`: runs `go mod tidy`, `go fmt` and `script/check`. The git
pre-commit hook calls it.
- `script/check`: runs these three scripts:
- `script/test`
- `script/lint`
- `script/fmt-check`
- `script/precommit`: the git pre-commit hook calls it. It runs these three:
- `go mod tidy`
- `go fmt`
- `script/check`
- `script/install-precommit`: installs that hook.
- `script/docker`: builds the Docker image. The build runs the linter and the
tests too.
- `script/cibuild`: what CI runs on every push. It runs `script/bootstrap`,
`script/check` and the image build.
- `script/cibuild`: what CI runs on every push. It runs these three:
- `script/bootstrap`
- `script/check`
- the image build
- `script/version`: prints the version to build into the binary.
- `script/projectname`: prints the project name, which tags the Docker image.
- `script/install-go`: installs the Go version that `go.mod` names into
`.tool/go`. It checks the download against a checksum in the script.
- `script/install-goreleaser`: installs the pinned `goreleaser` into
`.tool/bin`, checked the same way.
- `script/install-goreleaser`: installs the pinned
[`goreleaser`](https://goreleaser.com/) into `.tool/bin`, checked the same
way.
- `script/release`: builds and publishes the release files.
- `script/release-snapshot`: builds the release files into `./dist` and
publishes nothing.
@@ -1289,15 +1372,17 @@ skip the tests or the linter and still report success.
The test suite has no separate integration target. Every test runs every time.
Each test binary is stopped after 90 seconds.
To change the Go version, change these together: `go.mod`, the version and
checksums in `script/install-go`, and the two `golang` image digests in the
`Dockerfile`.
To change the Go version, change these together:
- `go.mod`
- the version and checksums in `script/install-go`
- the two `golang` image digests in the `Dockerfile`
### Messages to the user
Status lines, warnings and errors go through `internal/ui`. The data a command
exists to print is written plain, with no marker and no color. That covers
tables, reports, JSON documents and `config get` values.
Write status lines, warnings and errors through `internal/ui`. The data a
command exists to print is written plain, with no marker and no color. That
covers tables, reports, JSON documents and `config get` values.
`internal/ui` has these kinds of message:
@@ -1383,8 +1468,14 @@ version.
`make release-snapshot` builds the same files into `./dist` and publishes
nothing.
Each archive holds the binary, `LICENSE` and `README.md`. The release files are
not signed, have no SBOM and are not built reproducibly.
Each archive holds three files:
- the binary
- `LICENSE`
- `README.md`
The release files are not signed, have no software bill of materials and are
not built reproducibly.
## license