sneak/vaultik
1
0 Fork 0
Code Issues 13 Pull Requests 5 Actions Packages Projects Releases Wiki Activity
Files
3a748961148a25ca6ebd3cd08660e5c683f0b946
vaultik/README.md
T
sneak 3a74896114
check / check (push) Failing after 19m39s
Details
Refuse config keys that no setting has (closes #419) 
config.Parse walks the file as written and fails on the first key that
matches no setting, naming its dotted path and line, so a typo in
exclude or prune_grace_period no longer falls back silently. Keys that a
${...} reference brings in are checked once smartconfig has filled it
in, and named without a line. YAML merge keys and a top-level env
section, which smartconfig reads, still load. config set refuses such a
key before writing. The key-to-field lookup moves from internal/cli to
config.SettingType so both use it.

Model: opus-5-5
2026-11-11 01:23:51 +00:00

1116 lines
66 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# vaultik (ваултик)
`vaultik` is an incremental backup tool written in Go. It encrypts data using an
`age` public key and uploads each encrypted blob directly to a remote
S3-compatible object store. It requires no private keys, secrets, or credentials
(other than those required to PUT to encrypted object storage, such as S3 API
keys) stored on the backed-up system.
## quickstart
```sh
# install
go install sneak.berlin/go/vaultik/cmd/vaultik@latest
# create a default config file (prints the path it wrote to)
vaultik config init
# generate an age keypair; keep the private key file somewhere safe and
# offline — you need it to restore, and the backed-up machine does not need it
age-keygen -o vaultik_backup_private_key.txt
grep 'public key' vaultik_backup_private_key.txt
# configure the encryption key and backup destination
vaultik config set age_recipients.0 age1YOUR_PUBLIC_KEY_HERE
vaultik config set storage_url "file:///Volumes/usbstick/mybackup"
# create the destination directory; vaultik never creates it, so a backup
# to a volume that is not plugged in fails instead of filling the disk
mkdir /Volumes/usbstick/mybackup
# macOS only: grant your terminal app Full Disk Access first
# (System Settings → Privacy & Security → Full Disk Access), otherwise
# the backup will abort with a permission error on protected directories
# run your first backup (the default config backs up ~, and /Applications
# on macOS only, with sensible excludes)
vaultik snapshot create
# see what you have
vaultik snapshot list
```
Features:
- modern encryption ([age](https://age-encryption.org/), X25519 +
ChaCha20-Poly1305)
- content-defined chunking with deduplication (FastCDC)
- incremental backups (a file is re-chunked only when it changed or a chunk it
lists is held by no uploaded blob)
- multithreaded zstd compression at configurable levels
- content-addressed immutable storage
- local state tracking in SQLite (enables write-only incremental backups)
- no mutable remote metadata
- no plaintext file paths or metadata in remote storage
- packs small files into large blobs (keeps S3 operation counts down)
- backs up regular files, symlinks, empty directories, and file permissions
- pluggable storage backends: S3, local filesystem, rclone (70+ providers)
- pure Go (no CGO), cross-compiles to linux/darwin × amd64/arm64
## why
Other backup tools like `restic`, `borg`, and `duplicity` are designed for
environments where the source host can store secrets and has access to
decryption keys. `vaultik` is for environments where you don't want to store
backup decryption keys on your hosts — only public keys for encryption.
Requirements that no existing tool meets:
- open source
- no passphrases or private keys on the source host
- incremental
- compressed
- encrypted
- s3 compatible without an intermediate step or tool
## daily use
```sh
# verify a snapshot (shallow: checks all blobs are present with the listed size)
vaultik snapshot verify <snapshot-id>
# put the private key file in the environment (reading it from the file
# keeps the key out of your shell history); the whole age-keygen file,
# with one or more identities, is accepted
export VAULTIK_AGE_SECRET_KEY="$(cat vaultik_backup_private_key.txt)"
# deep verify (downloads every blob, decrypts it, and re-hashes it to
# detect corruption — this checks integrity, not who wrote the blob)
vaultik snapshot verify --deep <snapshot-id>
# restore (requires the private key). Restore into a new directory you own,
# writable only by you — not a shared location like /tmp
vaultik snapshot restore <snapshot-id> ~/vaultik-restore
# daily cron job: back up, keep a 4-week rolling window of snapshots
# 0 3 * * * vaultik snapshot create --cron --prune --keep-newer-than 4w
```
On a destination shared between hosts or users, the prune in that cron line
relies on `prune_grace_period` (default `24h`) to leave the blobs of a backup
another one is still running, and its directory under `metadata/` that has no
manifest yet, so it must be longer than the longest backup run. It does not
protect a blob that backup reuses from an earlier run that stopped before its
manifest: the prune can delete such a blob, and the new snapshot then cannot be
restored. To not depend on the grace period, give each host its own prefix or
directory. See [locking](#locking).
## restoring on another machine
Restoring on a host that never ran the backup — a replacement machine after the
original is gone — is the case vaultik is built for. That host needs only three
things: the `vaultik` binary, the age **private** key, and the storage
credentials for the destination. It does **not** need the local index, the
original config file, or the original hostname.
```sh
# install
go install sneak.berlin/go/vaultik/cmd/vaultik@latest
# create a config and point it at the ORIGINAL backup destination
vaultik config init
vaultik config set storage_url "s3://bucket/prefix?endpoint=https://s3.example.com"
vaultik config set s3.access_key_id "..."
vaultik config set s3.secret_access_key "..."
# see what is on the destination store
vaultik snapshot list
```
`snapshot list` reads the destination store without the private key. A snapshot
that is not in this host's (empty) local index is shown as remote-only: its row
is identified by `<remote only:...>` rather than by a `hostname_name_timestamp`
name, because the name lives only in the local index and the encrypted database
and cannot be recovered from the store. Its timestamp and compressed size are
real. (See the `snapshot list` description under
[command details](#command-details) for the full explanation.)
Use that remote key — the hex printed inside `<remote only:...>`, or the full
`remote_key` from `snapshot list --json` — to restore and verify:
```sh
# put the private key file in the environment (reading it from the file
# keeps the key out of your shell history)
export VAULTIK_AGE_SECRET_KEY="$(cat vaultik_backup_private_key.txt)"
# restore everything to a new directory you own (writable only by you, not a
# shared location like /tmp), then check every restored file's chunk hashes
vaultik snapshot restore --verify <remote-key> ~/vaultik-restore
# optionally, deep-verify the snapshot against the store (downloads every
# blob, decrypts it, and re-hashes it to detect corruption — this checks
# integrity, not who wrote the blob)
vaultik snapshot verify --deep <remote-key>
```
`age_recipients` (the public key) is not needed to restore — only the private
key in `VAULTIK_AGE_SECRET_KEY`. Both the abbreviated key printed in the table
and the full 64-character key from `--json` are accepted; a leading part of the
key is enough as long as it is unambiguous.
---
## cli
### commands
```sh
vaultik [--config <path>] config init
vaultik [--config <path>] config edit
vaultik [--config <path>] config get <key>
vaultik [--config <path>] config set <key> <value>
vaultik [--config <path>] snapshot create [snapshot-names...] [--cron] [--prune] [--keep-newer-than <duration>]
vaultik [--config <path>] snapshot list [--json] # alias: ls
vaultik [--config <path>] snapshot verify <snapshot-id> [--deep] [--json]
vaultik [--config <path>] snapshot purge [--keep-latest | --older-than <duration>] [--snapshot <name>...] [--force]
vaultik [--config <path>] snapshot remove <snapshot-id> [--dry-run] [--force] [--local-only] [--json] # alias: rm
vaultik [--config <path>] snapshot restore <snapshot-id> <target-dir> [paths...] [--verify]
vaultik [--config <path>] prune [--force] [--json]
vaultik [--config <path>] info
vaultik [--config <path>] remote info [--json]
vaultik [--config <path>] remote nuke --force
vaultik [--config <path>] database delete [--force]
vaultik completion <bash|zsh|fish|powershell>
vaultik version
```
### global flags
- `--config <path>`: Path to config file (default: `$VAULTIK_CONFIG`, then
platform config dir, then `/etc/vaultik/config.yml`)
- `--verbose`, `-v`: Enable verbose output (on stderr — see below)
- `--debug`: Enable debug output (on stderr — see below)
- `--quiet`, `-q`: Suppress non-error output (also suppresses startup banner)
- `--skip-errors`: Skip files that cannot be read when creating a snapshot, or
that cannot be restored when restoring, instead of aborting. Packing and
storage errors while creating a snapshot (which would leave a chunk recorded
but not stored) still abort the run. A run that skipped anything still
finishes but exits non-zero: `snapshot create` keeps the snapshot and counts
each file or directory it skipped as failed. A file removed during the backup
is also counted as failed but does not change the exit status.
### locking
Commands that write persistent state — `snapshot create`, `snapshot purge`,
`snapshot remove`, `prune`, and `remote nuke` — take a process-wide lock at
`$XDG_DATA_HOME/vaultik/vaultik.pid` (`~/.local/share/vaultik/vaultik.pid` on
Linux) for the whole run. Only one of them runs at a time: a second one exits
immediately with an "already running" error rather than waiting, so two writers
can never corrupt the local index.
The lock covers one user on one machine. It does not stop another user, or
another host, from writing to the same destination store at the same time. A
backup uploads its blobs, and then its `db.zst.age`, before its manifest, so a
prune from elsewhere at that moment finds blobs no snapshot lists and a
directory under `metadata/` with no manifest. It deletes such a blob only once
it is older than `prune_grace_period` (default `24h`), and such a directory only
once every object in it is, so the grace period must be longer than the longest
backup run of any host or user sharing the destination. The grace period does
not protect a blob a backup reuses from an earlier run that stopped before its
manifest: a backup skips the upload of a blob already on the destination, so
that blob keeps the time of its first upload, and a prune from elsewhere during
the backup can delete it; the new snapshot then cannot be restored.
`remote nuke` deletes everything regardless. To not depend on the grace period,
give each host or user its own prefix (or directory) on the destination.
Read-only commands — `info`, `snapshot list`, `snapshot verify`, and
`remote info` — do not take the lock and are never blocked, so they run even
while a backup is in progress. `snapshot restore` does not take the lock either:
it writes only to the target directory you name, not the local index or the
destination store. `config`, `database delete`, `completion`, and `version` do
not take the lock.
### stdout and stderr
Log output — everything from `--verbose` and `--debug`, and every warning and
error the logger emits — goes to **stderr**, and so does the startup banner.
stdout carries the output you asked for: tables, the documents produced by
`--json`, `config get` values, and completion scripts.
This means `vaultik snapshot list --verbose > out.txt` captures the listing and
leaves the diagnostics on your terminal. To capture both, redirect stderr as
well (`> out.txt 2> log.txt`, or `> out.txt 2>&1` to interleave them).
The split is what makes `--json` usable from a script. Warnings and errors are
never suppressed — not by `--quiet`, not by `--cron` — so a logger on stdout
would eventually land a log line inside a JSON document and break the parse. A
config file with group- or world-readable permissions is enough to trigger it.
Format follows the stream: when stderr is a terminal the records are colorized
one-liners, and when it is redirected or piped they are JSON, one object per
line.
Under `--json`, stdout holds the document and nothing else. The startup banner
is suppressed, as `--quiet` and `--cron` suppress it, and the progress narration
a command would otherwise print — such as the stale local records `prune`
reconciles away — is suppressed too, so it cannot land ahead of the document.
Every `--json` command therefore pipes on its own, with no additional flag:
`vaultik snapshot list --json | jq .` and `vaultik prune --json | jq .` both
work as written.
### environment variables
- `VAULTIK_AGE_SECRET_KEY`: Age private key for decryption (required for
`snapshot restore` and `snapshot verify --deep`). May hold the whole
`age-keygen` file — comments and every identity in it are accepted. Set it
from the file, e.g.
`export VAULTIK_AGE_SECRET_KEY="$(cat vaultik_backup_private_key.txt)"`, so
the key is not typed into your shell history.
- `VAULTIK_CONFIG`: Path to config file (overridden by `--config`)
- `VAULTIK_INDEX_PATH`: Override local SQLite index path
- `VAULTIK_CPUPROFILE`: Write a CPU profile to this path for the duration of the
run (development/debugging)
- `VAULTIK_MEMPROFILE`: Write a heap profile to this path when the run exits
(development/debugging)
### shell completion
```sh
# zsh: load for the current session
source <(vaultik completion zsh)
# zsh: install permanently
vaultik completion zsh > "${fpath[1]}/_vaultik"
# bash: load for the current session
source <(vaultik completion bash)
# bash: install permanently (Linux)
vaultik completion bash > /etc/bash_completion.d/vaultik
# fish
vaultik completion fish > ~/.config/fish/completions/vaultik.fish
```
### command details
**`config init`**: Write a default config file with commented explanations for
every setting. Writes to the path from `--config`, `$VAULTIK_CONFIG`, or the
platform config directory (`~/Library/Application Support/vaultik/` on macOS,
`~/.config/vaultik/` on Linux, `/etc/vaultik/` as root). Refuses to overwrite an
existing file. Created with mode `0600` since it will contain credentials.
**`config edit`**: Open the config file in `$EDITOR` (falls back to `vi`).
**`config get`**: Print a config value addressed by dotted YAML path (e.g.
`vaultik config get storage_url`). Non-scalar values print as YAML.
**`config set`**: Set a scalar config value by dotted YAML path (e.g.
`vaultik config set compression_level 9`,
`vaultik config set storage_url "file:///mnt/backups"`). Comments and formatting
in the file are preserved. Missing keys along the path are created: as a list
when the next path element is a number (so
`vaultik config set age_recipients.0 age1...` works on a file without
`age_recipients`), otherwise as a map. If the config loads, a change after which
it would not, such as `vaultik config set compression_level 99`, is refused and
the file is left unchanged. A config that does not load is checked one setting
at a time. Examples are the file `config init` writes before `storage_url` is
set, and one with a `${ENV:...}` reference to a variable not set in this shell.
In such a config, an unknown key or a value of the wrong type is refused.
**`snapshot create`**: Perform incremental backup of configured snapshots.
- Optional snapshot names argument to create specific snapshots (default: all)
- On macOS, the terminal application running vaultik needs Full Disk Access
(System Settings → Privacy & Security → Full Disk Access) to read
TCC-protected directories; without it the backup aborts with a permission
error that explains how to fix it
- `--cron`: Silent on total success; warnings and errors are still printed (for
crontab)
- `--prune`: After backup, drop older snapshots of each backed-up name, remove
orphaned blobs older than `prune_grace_period` from remote storage, and delete
each directory under `metadata/` that has no `manifest.json.zst` once every
object in it is older than `prune_grace_period` and its snapshot is not in the
local index, as `prune` does. By default keeps only the latest snapshot per
name; use `--keep-newer-than` for a rolling window. When `--skip-errors`
skipped a file or directory it could not read or list, the prune does not run
and every older snapshot is kept, since one of them may be the only one that
holds what was skipped; the command says so and exits non-zero. When the
backup found nothing in a configured path, as in the mount point of a volume
that is not mounted, the prune does not run either: every older snapshot is
kept, and the command names the path and exits non-zero, on every run until
the path holds something again. Take a path that is empty on purpose out of
the config, or remove the older snapshots by hand with `snapshot purge`. When
an earlier snapshot of the same name in the local index found a volume mounted
at or below a configured path (a directory on a different filesystem from its
parent, with an entry in it that the backup does not exclude) and this backup
does not, the prune does not run either: every older snapshot is kept, and the
command names the directory and exits non-zero, on every run until the volume
is mounted again. A backup also does not find the volume when it finds no such
entry in its directory, as when an automounted volume cannot be mounted, or
when that directory is no longer backed up, because it is excluded or its
configured path was removed. If any of this is on purpose, as for a retired
volume, remove the older snapshots by hand with `snapshot purge`. A file
removed during the backup does not stop the prune.
- `--keep-newer-than <duration>`: With `--prune`, keep snapshots newer than this
duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`). Without
`--prune`, or with a value that is not a valid duration, the command fails
before the backup starts.
- Fails before it writes anything when a `file://` destination directory does
not exist, or when the local index holds snapshots and the destination answers
that it has no `vaultik-destination.txt`. When that look-up fails for any
reason other than not found, for example when S3 refuses it to credentials
that allow only PUT, the check is skipped (see
[remote storage layout](#remote-storage-layout))
- Fails, before anything is pruned, when a configured path is a symlink whose
target does not exist, as one into a volume that is not mounted
**`snapshot list`**: Show every snapshot known to this host — the union of the
local index and the backup destination store — with timestamps and three sizes
per snapshot (compressed remote size; total uncompressed chunk size; size of the
chunks that snapshot stored that were not stored before). Both chunk sizes are
recorded when the snapshot's backup runs.
Listing the destination store does **not** require the age secret key, so it
works in vaultik's intended configuration, where the backed-up host holds only
the public key. A host that has lost its local index can still see what it has
backed up.
What that host cannot see is a remote-only snapshot's name. The snapshot ID is
hashed at the storage boundary and the manifest records only the hash, so
hostname and snapshot name exist solely in the local index and in the encrypted
per-snapshot database. Snapshots found only on the destination store are
therefore listed as `<remote only:<abbreviated remote key>>` and show
`<remote only>` in the uncompressed and "new chunk" columns, which come only
from the local index. Their timestamp and compressed size are real, read from
the manifest.
Snapshots in the local index with no counterpart on the destination store are
reported below the table as drift, with the `vaultik prune` invocation that
reconciles them. If the local index is bound to a different destination than the
configured one (after a change to `storage_url`, or `s3.endpoint`, `s3.bucket`
or `s3.prefix`), the command says so instead and gives the same two ways out
that `prune` gives: revert the configuration, or run `vaultik database delete`.
If the destination store cannot be listed (unmounted volume, permission denied,
network down), the command warns, falls back to the local index alone, and still
exits zero. It does the same when the destination answers that it has no
`vaultik-destination.txt` while the local index holds snapshots stored there, as
when a volume mounted at the destination is unplugged and leaves its empty mount
point behind.
- `--json`: Output in JSON format. Each entry carries `locally_tracked` (whether
the snapshot is in the local index), `remote_key` (the full 64-character
storage key), and `remote_present` (whether it was seen on the destination
store, or `null` if the destination could not be listed). Warnings about an
unlistable destination, a local index bound to a different destination (with
both destinations and the same two ways out), unreadable manifests, and a
truncated listing all go to stderr through the logger, so stdout stays a
single parseable document.
**`snapshot verify`**: Verify snapshot integrity.
- Default (shallow): checks that every blob the manifest lists is present in
storage with the size the manifest records, and that the encrypted database is
present. It does not read blob contents.
- `--deep`: Downloads and decrypts each blob, verifies chunk hashes against the
encrypted metadata database
- Accepts the same identifiers as `snapshot restore`: a snapshot ID, or a
remote-only snapshot's remote key (or an unambiguous leading part of it)
- `--json`: Output results as JSON
**`snapshot purge`**: Remove old snapshots based on criteria. Retention is
per-snapshot-name (`--keep-latest` keeps the latest of each name, not the latest
globally).
- `--keep-latest`: Keep only the most recent snapshot of each name
- `--older-than <duration>`: Remove snapshots older than duration (e.g. `30d`,
`4w`, `6mo`, `1y`; `m` is minutes, `mo` is months)
- `--snapshot <name>`: Restrict to specific snapshot names (repeat for multiple)
- `--force`: Skip confirmation prompt
**`snapshot remove`**: Remove one snapshot. By default this removes the snapshot
from the local index and strips the snapshot's metadata from the backup
destination store. Blobs are NOT touched — deleting blobs requires reading every
remaining remote manifest (the destination store may hold snapshots this host
doesn't know about), which is what `vaultik prune` does. The command prints the
`vaultik prune` invocation to run as a follow-up only when it removed the
snapshot's metadata from the destination store, and never under `--json`. An ID
that names no snapshot in the local index or on the destination store is an
error; with `--local-only`, the snapshot must be in the local index. Local row
cleanup (files, chunks, blobs the snapshot was the last referrer for) runs
automatically. If removing the snapshot's metadata from the destination store
fails (for example because the store is unreachable, or the local index holds
snapshots and the destination answers that it has no `vaultik-destination.txt`),
the local-DB removal still completes and a warning is emitted; run
`vaultik snapshot remove <snapshot-id>` again once the store is reachable to
remove the snapshot's metadata from it (`vaultik prune` does not). For a
snapshot that is only on the destination store, that failure is an error. To
wipe everything on the destination in one go, use `vaultik remote nuke --force`.
- `--local-only`: Skip remote cleanup; only touch the local index
- `--dry-run`: Show what would be deleted without deleting
- `--force`: Skip confirmation prompt
- `--json`: Output result as JSON. Also skips the confirmation prompt, as
`--force` does.
**`snapshot restore`**: Restore files from a backup snapshot.
- Requires `VAULTIK_AGE_SECRET_KEY` environment variable
- Accepts a snapshot ID, or — for a snapshot only on the destination store — its
remote key (or an unambiguous leading part of it) as shown by `snapshot list`.
See [restoring on another machine](#restoring-on-another-machine).
- Optional path arguments to restore specific files/directories (default: all).
A path that matches nothing in the snapshot is named in a warning, and the
command exits non-zero after restoring the paths that match.
- Preserves file permissions, timestamps, ownership (ownership requires root),
symlinks, and empty directories
- `--verify`: After restoring, verify every restored file's chunk hashes match.
A file skipped under `--skip-errors` is reported as failed to restore and is
not verified.
**`prune`**: Tidy up everything that isn't needed. Runs three passes: (1)
reconcile the local index against the destination store — any local snapshot
whose remote metadata is missing is dropped from the local index; (2) delete
orphaned local rows (files, chunks, blobs no longer referenced by any completed
snapshot); (3) list every remote manifest on the destination store to compute
the still-referenced blob set and delete any blob not in that set that is older
than `prune_grace_period` (default `24h`). A newer one is kept and counted: it
may belong to a backup still running, on this host or another sharing the
destination store, that has not yet uploaded its manifest. The grace period does
not protect a blob such a backup reuses from an earlier run that stopped before
its manifest: that blob keeps the time of its first upload, so it can be
deleted. Step (3) also deletes each directory under `metadata/` that has no
`manifest.json.zst`, as a backup leaves when it stops after uploading
`db.zst.age` but before its manifest, once every object in it is older than
`prune_grace_period` and its snapshot is not in the local index. An object the
listing gives no time for keeps its directory, and a directory whose name is not
a remote key is left alone. Step (3) reads all remote manifests — network cost
scales with the number of snapshots. `snapshot create --prune` runs the same
cleanup automatically; this is the manual entry point for the same work. If the
destination store cannot be listed, or the local index holds snapshots and the
destination answers that it has no `vaultik-destination.txt` (an unplugged
volume's empty mount point), prune fails and deletes nothing. `snapshot purge`
does the same. If a blob or such a directory cannot be deleted, the command (and
`snapshot create --prune`) exits non-zero.
- `--force`: Skip confirmation prompt
- `--json`: Output stats as JSON. Also skips the confirmation prompt, as
`--force` does. A blob that cannot be deleted is counted in `blobs_failed`,
and the document is printed before the non-zero exit. A blob kept for being
newer than `prune_grace_period`, or because the listing of the destination
gives no time for it, is counted in `blobs_too_new` and not in `blobs_found`.
A directory under `metadata/` with no manifest is counted in
`metadata_dirs_deleted` once deleted, or in `metadata_dirs_failed` if it
cannot be.
**`info`**: Display system configuration, storage settings, encryption
recipients, and local database statistics.
**`remote info`**: Show storage backend type and location plus detailed remote
storage inventory: per-snapshot metadata sizes, blob counts, and orphaned blob
detection. A name under `metadata/` that is not a remote key is skipped with a
warning and is not printed. If a listed `manifest.json.zst` cannot be read, or
sits under a skipped name, the orphaned blob figures are reported as unknown;
`--json` gives them as `null`, lists the remote key of each unreadable manifest
in `unreadable_manifests` and counts the manifests under skipped names in
`skipped_manifest_count`. An unreadable manifest also leaves its snapshot's blob
count and blob size unknown, `null` in `--json`. A file under `blobs/` whose
name is not a blob hash is not counted as a blob, since `prune` never deletes
it.
- `--json`: Output as JSON
**`remote nuke`**: Delete every object under `metadata/` and `blobs/` on the
backup destination store, whatever its name, and then `vaultik-destination.txt`,
leaving the bucket prefix empty. Destructive and irreversible. This is the
single supported way to wipe the entire destination store. A snapshot is removed
from the local index only once its metadata is deleted from the destination
store. If the destination store cannot be listed, or the local index holds
snapshots and the destination answers that it has no `vaultik-destination.txt`
(an unplugged volume's empty mount point), the command fails before it removes
anything. If a listing afterwards still finds anything under `metadata/` or
`blobs/`, the command names each object left and exits non-zero; running it
again deletes what is left.
- `--force`: Required to confirm destruction.
**`database delete`**: Delete the local SQLite state database file entirely.
Remote storage is unaffected; the next backup will do a full scan and
re-deduplicate against existing remote blobs, and the local index will re-bind
to the currently configured storage destination. Use this after changing
`storage_url`, or `s3.endpoint`, `s3.bucket` or `s3.prefix`, to a different
destination.
- `--force`: Skip confirmation prompt
---
## storage backends
vaultik supports three storage backends, selected via the `storage_url` config
field:
**S3** (`s3://bucket/prefix?endpoint=host&region=us-east-1`): Any S3-compatible
object store. Credentials are read from `s3.access_key_id` and
`s3.secret_access_key` in the config file.
**Local filesystem** (`file:///path/to/backup`): Stores blobs and metadata on a
local or mounted filesystem. Useful for testing or backing up to a NAS. The path
must be absolute; a relative one is refused when the config loads.
Percent-escapes in the path are decoded, so `my%20stick` and `my stick` name the
same directory. The directory must exist before the first backup: vaultik never
creates it. A directory on a removable volume is gone while the volume is not
mounted, so a backup then fails instead of writing to the disk under the mount
point. The mount point itself stays behind when the volume is unplugged: there
only `vaultik-destination.txt` stops such a backup, and only while the local
index holds snapshots.
**Rclone** (`rclone://remote/path`): Uses rclone's 70+ supported cloud
providers. Requires rclone to be configured separately (`rclone config`).
An upload cut off part-way leaves nothing under the object's name on S3, which
shows an object only once its upload has completed, and on the local filesystem
backend, which writes a temporary file and renames it into place. Rclone remotes
with a server-side move (local and sftp among them) are written under a
temporary name ending in `.partial` and moved into place. Rclone remotes without
one are written in place: where such a remote shows a file while it is still
being written, a killed upload can leave a truncated object under its name,
which a later backup takes for complete. A leftover `.partial` file is ignored
and can be deleted.
A destination shared between hosts or users, each with its own local index,
relies on `prune_grace_period` to keep one's prune from deleting the blobs of
another's backup still running, or its directory under `metadata/` that has no
manifest yet, so it must be longer than the longest backup run. It does not
protect a blob that backup reuses from an earlier run that stopped before its
manifest, which the prune can delete. To not depend on the grace period, give
each host its own prefix (or directory). See [locking](#locking).
Legacy S3 configuration via `s3.*` fields (endpoint, bucket, prefix, etc.) is
still supported for backward compatibility. `storage_url` takes precedence if
both are set.
---
## architecture
For an implementation-level view of the internals — the data model, the `fx`
dependency-injection wiring, and the scanner — see
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).
### remote storage layout
```
<bucket>/<prefix>/
├── vaultik-destination.txt # Marker written by every backup
├── blobs/
│ └── <aa>/<bb>/<full_blob_hash>
└── metadata/
└── <remote-key>/
├── db.zst.age # Encrypted binary SQLite database
└── manifest.json.zst # Unencrypted blob list (for pruning)
```
- `vaultik-destination.txt` holds one fixed line of text, with no host name,
path or time. While the local index holds snapshots stored there, a
destination that answers a look-up of it with not found is treated as not
present: `snapshot create` refuses to write to it, `prune`, `snapshot purge`
and `remote nuke` fail, and `snapshot list` and `snapshot remove` behave as
for a destination that cannot be listed. This keeps an unplugged volume's
empty mount point from being taken for a destination whose snapshots are all
gone. Mount the volume if it is removable; if the destination was emptied on
purpose, run `vaultik database delete` and back up again. When the look-up
fails for any reason other than not found, the check is skipped and the
command goes on. S3 refuses it on every run to credentials that allow only
PUT, so a host with such credentials does not notice a destination emptied
since its last backup, and its next backup would leave out the data its local
index records as already stored. After the destination is emptied, run
`vaultik database delete` on such a host before it backs up again.
- Blobs are two-level directory sharded using the first 4 hex chars of the blob
hash
- `db.zst.age` is a binary SQLite database (zstd compressed, age encrypted)
containing all file metadata, chunk mappings, and relationships for the
snapshot
- `manifest.json.zst` is an unencrypted compressed JSON blob list, enabling
pruning without the private key
Snapshot IDs follow the human-readable format
`<hostname>_<snapshot-name>_<RFC3339-timestamp>` (e.g.
`server1_home_2025-06-01T12:00:00Z`), but this ID is never written to the
destination store in plaintext. Each snapshot's metadata directory is named with
its `<remote-key>`, a one-way double SHA-256 hash of the ID, so a plain listing
of the store shows no hostname or snapshot name. The hash uses no secret,
though, so an observer who guesses a candidate hostname and snapshot name can
hash it and confirm the snapshot is present; the remote key keeps names out of a
listing but does not hide them from a guess. The backup time is not hidden
either: manifest.json.zst carries a plaintext timestamp, and object modification
times are visible at the storage layer regardless. For example,
`server1_home_2025-06-01T12:00:00Z` is stored under
`metadata/17f97bcde958748af076b926af59823943db59e80ce7170b40f124dfa28f64aa/`.
See [docs/REPOSTRUCTURE.md](docs/REPOSTRUCTURE.md#remote-key-derivation) for the
derivation, and
[Security Considerations](docs/REPOSTRUCTURE.md#security-considerations)
(including [Accepted Risks](docs/REPOSTRUCTURE.md#accepted-risks)) for what the
format does and does not protect.
### data flow
**backup:**
1. Open local SQLite index, load known files and chunks into memory
2. Walk source directories, compare mtime/size/mode against index
3. For changed/new files: chunk using content-defined chunking (FastCDC)
4. For symlinks and directories: record metadata (no chunking)
5. For each chunk: hash, check dedup, add to blob packer
6. When blob reaches size threshold: compress (zstd), encrypt (age), upload
7. Build snapshot metadata database, compress, encrypt, upload
8. Create unencrypted blob manifest for pruning support
**restore:**
1. Download and decrypt `metadata/<remote-key>/db.zst.age`
2. Open the binary SQLite database
3. Query files (optionally filtered by paths)
4. Download and decrypt required blobs
5. Extract chunks, reconstruct files
6. Restore permissions, timestamps, ownership, symlinks
**prune:**
1. List all snapshot manifests
2. Build set of all referenced blob hashes
3. List all blobs in storage
4. Delete any blob not in the referenced set that is older than
`prune_grace_period`
5. Delete each directory under `metadata/` with no manifest whose objects are
all older than `prune_grace_period`, unless its snapshot is in the local
index
### chunking and deduplication
- Content-defined chunking using the FastCDC algorithm
- Average chunk size: configurable (default 10MB)
- Deduplication at file level (unchanged files skipped, unless a chunk the file
lists is held by no uploaded blob) and chunk level (identical chunks across
files stored once)
- Multiple chunks packed into blobs to reduce object count
### encryption
- Asymmetric encryption using age (X25519 + ChaCha20-Poly1305)
- Only the public key is needed on the source host
- Each blob and each metadata database is encrypted independently
- Multiple recipients supported (encrypt to multiple keys)
### compression
- zstd compression at configurable level (1-19, default 3). The level is
accepted as 1-19 but maps onto zstd's four internal speed presets: 1-2
fastest, 3-5 default, 6-9 better, 10-19 best. Levels within the same band
compress identically.
- Applied before encryption at the blob level
---
## configuration reference
Run `vaultik config init` to generate a fully commented config file; a complete
annotated example also lives in
[`configs/config.example.yml`](configs/config.example.yml). Key fields:
| Field | Default | Description |
| -------------------- | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `age_recipients` | (required by `snapshot create`) | Age public keys for encryption. Other commands run without one, so a machine that only restores can leave it empty |
| `age_secret_key` | (unset) | Age private key for decryption (`snapshot restore`, `snapshot verify --deep`). Setting it in the config file places the private key on the backed-up host, defeating the public-key-only design (see "why" above). Prefer the `VAULTIK_AGE_SECRET_KEY` environment variable, supplied only on the machine you restore from. |
| `snapshots` | (required) | Named snapshot definitions with paths and excludes |
| `storage_url` | | Storage backend URL (`s3://`, `file://`, `rclone://`) |
| `s3.*` | | Legacy S3 configuration (endpoint, bucket, credentials) |
| `exclude` | | Global exclude patterns (applied to all snapshots) |
| `chunk_size` | `10MB` | Average chunk size for content-defined chunking |
| `blob_size_limit` | `10GB` | Maximum blob size before splitting. Must be at least four times `chunk_size` (the largest chunk the chunker can emit), otherwise a single-chunk blob could exceed the limit. A backup needs free temporary space, because each blob is written whole to a temporary file before it is uploaded (up to about `blob_size_limit`; an rclone destination that cannot stream uploads needs about twice that) and the metadata export writes copies of the local index. Temporary files go to `$TMPDIR` (default `/tmp`); with `TMPDIR` unset, SQLite writes one of those copies to `/var/tmp` |
| `compression_level` | `3` | zstd compression level (1-19) |
| `prune_grace_period` | `24h` | How old a blob that no snapshot lists must be before `prune` deletes it, and how old every object in a directory under `metadata/` with no manifest must be before `prune` deletes that directory. A backup uploads its blobs and its `db.zst.age` before its manifest, so on a destination shared between hosts or users this must be longer than the longest backup run (see [locking](#locking)). It does not protect a blob a backup reuses from an earlier run that stopped before its manifest: that blob keeps the time of its first upload, so a prune can delete it. A duration such as `36h` or `90m`; `0s` deletes such blobs and directories at once |
| `hostname` | system hostname | Hostname used in snapshot IDs |
| `index_path` | platform data dir | Local SQLite index path |
---
## limitations
- **No extended attributes (xattrs).** ACLs, macOS Finder metadata, quarantine
flags, SELinux labels, and other extended attributes are not backed up or
restored.
- **No hard link detection.** Two hard links to the same inode are backed up as
independent files. Content deduplication means the data is stored once, but
the hard link relationship is lost on restore.
- **No sparse file support.** Sparse files are fully materialized during backup.
A 100 GB sparse VM disk that is mostly zeros will consume the full
(compressed) size in storage.
- **No bandwidth limiting.** Uploads and downloads use whatever bandwidth is
available. There is no `--bwlimit` flag yet.
- **No parallel blob downloads during restore.** Blobs are fetched sequentially.
Restore speed is bound by single-stream throughput.
- **Device nodes, named pipes, and sockets are silently skipped.** Only regular
files, directories, and symlinks are backed up.
- **Before 1.0, an update can make the local index unusable.** Vaultik supports
schema migrations, but none are added before 1.0 because there is no installed
base yet. If an update leaves your local index unusable, run
`vaultik database delete` and then a full backup; remote storage is
unaffected. After 1.0, the local index is migrated when vaultik is updated.
See [`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations).
- **Files that change during backup may be inconsistent.** There is no
filesystem snapshot or freeze. If a file is modified between the scan and
chunk phases, the backed-up copy may reflect a partial write.
- **Ownership restoration requires root.** File uid/gid are recorded and
restored, but `chown` requires elevated privileges. Without root, files are
restored with the current user's ownership.
---
## roadmap
Work planned after 1.0. Loosely ordered by priority.
### correctness and operability
- **Outside security audit.** Before 1.0 the encryption and blob-generation code
was reviewed: every bug the review found was fixed, and the risks it accepted
are listed in [Accepted Risks](docs/REPOSTRUCTURE.md#accepted-risks). No
outside audit has been done. age + zstd + content-defined chunking is mostly
off-the-shelf pieces, but the seams (key handling, recipient parsing, manifest
trust boundary, restore-time identity validation) need an outside read.
- **Verify restored content end-to-end in CI.** The current integration test
does this for a small synthetic snapshot but not at scale. A nightly job
against a multi-GB representative snapshot would catch silent regressions in
the chunker, packer, or restore planner.
### performance
- **Parallel blob downloads during restore.** Single-stream right now. With a
fast S3 endpoint and a multi-core machine restore is bound by per-blob fetch +
decrypt + decompress; running N of those in parallel against the disk cache
would close most of the remaining gap. Needs to interact correctly with the
locality planner and sweeper.
- **Bandwidth limiting (`--bwlimit`).** Both upload and download. Useful for
backing up over a shared link. Tricky to make work correctly with the
parallel-download story.
- **Restart of interrupted restore.** Today restore is restartable in the sense
that re-running it overwrites partial output; it doesn't resume from where it
stopped or skip already-present files. A `--resume` mode that checks targets
before fetching blobs would matter for very large restores.
- **Daemon mode.** A long-running mode that watches for file changes so frequent
backups, such as hourly, skip the full scan. It adds little for the usual runs
from cron every 12 to 36 hours. See
[issue #204](https://git.eeqj.de/sneak/vaultik/issues/204).
### usability
- **Man pages and richer `--help` examples.** Cobra generates basic help; man
pages would be a separate target.
- **`vaultik snapshot diff <a> <b>`** — show which files changed between two
snapshots without restoring either.
- **Status reporting hook for `--cron`.** When a backup fails silently in cron,
the user has no idea. A configurable webhook / email / `notify-send` hook on
completion (success and failure) would close the loop.
### infrastructure
- **Schema migrations after 1.0.** Migrations are supported, but none are added
before 1.0 because there is no installed base yet. After 1.0, each schema
change is a new migration, so an existing local index is migrated when vaultik
is updated (see [`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations)).
- **Storage backend coverage tests.** S3, file://, and rclone:// all share the
Storer interface but the rclone path is the least exercised in CI.
---
## output style
Every command's user-facing output is governed by `internal/ui`, in one of two
ways. Color is enabled when the stream written to is a TTY and the `NO_COLOR`
environment variable is unset (https://no-color.org/).
- **Status, progress, warnings, and errors** go through the `internal/ui`
message methods below: marker-prefixed, colored on a TTY, and — except
warnings and errors — silenced by `--quiet`. This is the operational narration
of the long-running commands (`snapshot create`, `prune`, `snapshot restore`,
and the like) and the confirmations of `config init`, `config set`, and
`database delete`.
- **The data a command exists to produce** is written plain, with no marker and
no color, because a marker would corrupt a table or a parsed document. This
covers the `version`, `info`, `remote info` and `snapshot verify` reports, the
`snapshot list` table, `config get` values, and every `--json` document.
`--quiet` silences the human reports and tables (`version`, `info`,
`remote info`, `snapshot verify`, `snapshot list`) but never the
machine-consumed `config get` value or the `--json` documents, which a script
depends on. The confirmation prompts of `database delete`, `prune`,
`snapshot purge` and `snapshot remove`, each with what it asks to delete, are
also written this way and always shown: they are an interactive exchange the
operator must see.
`internal/ui` writes to stdout; it is the output the user asked for. The
exceptions are the startup banner and the error a failed command ends with,
which go to stderr. Structured log records are a different thing and go through
`internal/log`, which writes to stderr (see "stdout and stderr" above).
Message classes:
| Class | Marker | Alignment | Use for |
| -------- | ------------------------------ | --------- | ------------------------------------------------------------------------------ |
| Banner | none | column 0 | The startup line printed once per invocation, on stderr |
| Begin | `》` (white) | column 0 | An operation is about to start (present-continuous verb) |
| Complete | `》` (green) | column 0 | An operation just finished (past-tense verb) |
| Info | `》` (white) | column 0 | Neutral status update |
| Notice | `》` (cyan) | column 0 | Important note that is not a warning |
| Warning | `⚠️ Warning:` (orange/yellow) | column 0 | Recoverable problem |
| Error | `🛑 ERROR:` (red) | column 0 | Operation aborted |
| Progress | ` 》` (white) | column 2 | Heartbeat or per-item status during a long-running operation |
| Detail | ` 》` (white) | column 2 | Continuation/sub-line of a preceding Complete (visually identical to Progress) |
Conventions:
- Messages are complete English sentences ending with a period.
- Fully qualify terms — say "backup destination store" instead of "storage",
"snapshot source files enumeration" instead of "scan", "local index database"
instead of "database".
- Every operation that emits a Complete also emits a corresponding Begin.
Operations that print only a Begin (because completion is obvious from a later
Begin) should be rare and intentional.
- Use natural verb tense to signal state: "Uploading" for Begin, "Uploaded" for
Complete. Never write the words "begin" or "complete" in the body — the marker
color already conveys that.
- All elapsed and remaining-time fields are explicitly scoped to their subject:
write "blob upload elapsed: 30s, blob upload ETA: 03:15:00 (est remain 14s)",
never just "elapsed 30s, ETA 14s".
- "ETA" means an absolute clock time (when the operation will finish), not a
remaining-duration. Use `ui.Time()` for the former and `ui.Duration()` for the
latter, and label both.
- `ui.Time` formats same-day times as `HH:MM:SS` and other-day times as
`YYYY-MM-DD HH:MM:SS`. No timezone — local time is implied.
Value colorizers in `internal/ui` colorize specific value types consistently.
Compose messages from these helpers rather than embedding ANSI escapes inline:
| Helper | Color | Use for |
| ---------- | --------- | --------------------------------------------------------- |
| `Hex` | cyan | Blob hashes, chunk hashes (truncated to 12 chars + `...`) |
| `Snapshot` | bold cyan | Snapshot IDs (untruncated) |
| `Path` | blue | Filesystem paths |
| `Size` | magenta | Byte counts (human-readable) |
| `Speed` | magenta | Bytes-per-second rates |
| `Duration` | yellow | Elapsed or remaining time |
| `Time` | yellow | Absolute clock times |
| `Count` | magenta | Integer counts with thousands separators |
| `Percent` | magenta | Percentages |
When `NO_COLOR` is set or output is not a TTY, all helpers return plain text and
the marker prefixes (`》`, `Warning:`, `ERROR:`) emit without ANSI escapes. The
emoji prefixes on Warning and Error are always emitted regardless of color
setting (emoji are not color).
## requirements
- Go 1.26 or later
- Docker, with a reachable daemon, to test, lint, check, or commit:
`script/test` and `script/lint` build the `test` and `lint` phases of the
`Dockerfile`, and `make check` and the pre-commit hook both run them. A
`golangci-lint` installed on `PATH` is not a substitute and is never used on a
host, whatever its version.
- S3-compatible object storage (or local filesystem, or rclone remote)
## development workflow
All changes follow this workflow. No exceptions.
1. Create a feature branch off `main`.
2. Write tests.
3. Write the implementation.
4. Fix implementation errors until it compiles and tests pass.
5. Fix linting errors (`make lint`).
6. Update documentation and README as required by the change.
7. Format code (`make fmt`).
8. Run `make check` (lint + fmt-check + test). Fix any issues. Repeat until
clean.
9. Commit on the branch.
10. Merge to `main`.
11. Push.
Do not commit directly to `main`. Do not skip steps.
Repository policies for AI agents are in [`AGENTS.md`](AGENTS.md).
## Entrypoints
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call them. We
provide:
- `script/bootstrap` — install all development dependencies (Go, Go module
download, and node, yarn and `prettier` for Markdown). Node comes from the
host, or else at a pinned version through `nvm`. A host without Go gets the
`go.mod` version through `script/install-go`, in `.tool/go`, which
`script/bootstrap` itself, the `Makefile`, `script/fmt`, `script/fmt-check`,
`script/precommit` and `script/release` add to their `PATH`. It deliberately
does not install `golangci-lint`; see `script/lint` below.
- `script/setup` — make a fresh clone ready for development: runs
`script/bootstrap`, then `script/install-precommit`
- `script/projectname` — print the project name (used for the Docker image tag)
- `script/version` — print the version string to bake into the binary. The
`Makefile`'s `LDFLAGS` call this. See [releasing](#releasing) for the rules.
- `script/install-goreleaser` — install the pinned `goreleaser` into `.tool/bin`
from a sha256-verified release archive. Idempotent, and called by
`script/bootstrap`; the release workflow calls it directly because it needs
`goreleaser` but not the Docker daemon `script/bootstrap` insists on.
- `script/install-go` — install the Go toolchain named by `go.mod`'s `go`
directive into `.tool/go` from a sha256-verified `go.dev` archive, for Linux
or macOS on amd64 or arm64. Idempotent. On a CI runner it also puts
`.tool/go/bin` on `PATH` for the steps that follow. Called by the release
workflow, which needs a host Go for `goreleaser` to shell out to, and by
`script/bootstrap` on a host without Go. `actions/setup-go` is not used
because it verifies the downloaded toolchain against no value in this repo.
Bumping Go edits `go.mod`, the checksums in this script, and the two `golang`
digests in the `Dockerfile` together.
- `script/release` — cross-compile and publish the release artifacts with the
pinned `goreleaser`. Refuses a `goreleaser` on `PATH` whose version is not the
pinned one, because a different version would build a different release from
the same tag.
- `script/release-snapshot` — the same build with no publishing and no tagging,
into `./dist`
- `script/test` — run the test suite by building the `test` phase of the
`Dockerfile` (verbose rerun on failure). This runs _everything_: there is no
separate integration target and no build-tagged subset held back, so the full
round-trip tests in `internal/vaultik/integration_test.go` run on every
invocation. The 90s `-timeout` is a hang backstop rather than a performance
budget; it applies per test binary, to test execution only.
- `script/lint` — lint by building the `lint` phase of the `Dockerfile`, which
runs `golangci-lint config verify` and then
`golangci-lint run --config .golangci.yml ./...` as build steps in the
digest-pinned `golangci-lint` image. Nothing lints on the host. That `FROM`
line is the only pin of the linter version, and it changes in the same commit
as a re-vendored `.golangci.yml`.
- `script/lint-fix` — apply the linter's autofixes (rewrites files), using the
same pinned image, parsed out of the `lint` phase's `FROM` line. It cannot be
a build step, because fixes have to land in the worktree, so it bind-mounts
the tree into a `docker run` and therefore needs a _local_ daemon. It is a
developer convenience and never a gate: no gate reads its exit status. Run
`make lint` afterwards to find out whether the tree is clean.
- `script/fmt` — format all code (writes): Go with `go fmt`, Markdown with
`prettier`
- `script/fmt-check` — check formatting (read-only). It runs `gofmt` on the
host, over every Go file outside `.tool`, and `prettier --check` over every
Markdown file.
- `script/check` — run `script/test`, `script/lint`, and `script/fmt-check`.
- `script/docker` — build the Docker image tagged via `script/projectname`,
stamped with the version `git describe --tags --always --dirty` gives on the
host, or `unknown` outside a git checkout. The image's build stage depends on
the `lint` and `test` phases, so this lints and tests too.
- `script/cibuild` — CI entrypoint: runs `script/bootstrap`, `script/check`, and
then the same image build as `script/docker`. `.gitea/workflows/check.yml`
runs it on every push.
Every `docker build` in these scripts passes `--no-cache`, because on an
unchanged tree a cached check layer is replayed without running and the
build still exits 0. A plain `docker build .` is therefore no evidence that
the checks ran. The cost is that `script/cibuild` runs the `lint` and `test`
phases twice: once in `script/check` and again in the image build.
- `script/precommit` — pre-commit gate: `go mod tidy` + `go fmt` (must not
change files), then `script/check`
- `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit`
## releasing
### version numbers
The version a binary reports comes from git, not from a constant in a file. It
is `git describe --tags --always --dirty`, which `script/version` runs for the
`Makefile`, and `script/docker` and `script/cibuild` run themselves:
- `HEAD` is exactly on a tag → that tag, such as `v1.0.0`.
- a commit after a tag → `<tag>-<N>-g<short sha>`.
- no tag reachable → the short commit sha.
- any of these, with uncommitted changes to tracked files → a `-dirty` suffix,
because a modified checkout of a tag is not that tag.
A `docker build .` of a clone, with no build arguments, runs the same
`git describe` (without `--dirty`) on the `.git` in its build context, so it
stamps the same value for a clean commit; the build fails if the context carries
`.git` and no version, commit or commit date comes out. A binary built without
git metadata reports `dev`, or `unknown` when `script/docker` or
`script/cibuild` built it outside a git checkout.
`goreleaser` stamps a release binary with the tag minus its leading `v`, so the
tag `v1.0.0` produces `vaultik 1.0.0`, matching the archive name
`vaultik_1.0.0_linux_amd64.tar.gz`. `goreleaser --snapshot` stamps
`dev-<12 chars of the commit sha>` rather than inventing the next patch number.
`vaultik version` calls a build a development build when its version is `dev`,
`unknown`, `dev-<sha>`, the short commit sha or `<tag>-<N>-g<short sha>`, with
or without `-dirty`; only a plain tag, such as `v1.0.0` or `1.0.0`, is a
release. If `script/version` cannot be run at all, `make` stops with an error
instead of building an unversioned binary, and a binary that somehow carries an
empty version string still reports itself as a development build.
### cutting a release
Releases are cut by CI from a tag, not from a workstation:
```
git tag -a v1.2.3 -m 'v1.2.3'
git push origin v1.2.3
```
`.gitea/workflows/release.yml` triggers on `v*` tags, installs a Go toolchain
and the pinned `goreleaser`, and runs `script/release`, which builds
`linux,darwin × amd64,arm64` archives plus `checksums.txt` and publishes them to
this repository's Gitea releases as a draft. `.goreleaser.yaml` has a
`gitea_urls:` block pointing at `https://git.eeqj.de/api/v1`; without it
`goreleaser` would talk to the GitHub API.
The workflow needs one repository Actions secret:
| Secret | What it is |
| --------------- | ------------------------------------------------------------------------------------------------------- |
| `RELEASE_TOKEN` | A Gitea access token with `write:repository` scope, owned by an account that can publish releases here. |
It is passed to `goreleaser` as `GITEA_TOKEN`. The runner's automatic token is
deliberately not used: it is not guaranteed to carry release write access.
The Go toolchain that compiles the released binaries is installed by
`script/install-go`, which downloads the version named by `go.mod` (currently
`1.26.1`, the same version the `Dockerfile` builder stage pins by digest) and
refuses the archive unless its sha256 matches the value committed in the script.
`goreleaser` shells out to `go` for every cross-compile, so without that step
the release would either fail outright or ship binaries built by whatever
unpinned toolchain the runner happened to carry.
To rehearse the whole build without publishing or tagging anything:
```
make release-snapshot
```
Artifacts land in `./dist`, which is gitignored.
Release artifacts are not signed, carry no SBOM, and are not built reproducibly;
the archives contain the binary, `LICENSE`, and `README.md` only (no shell
completions or man page).
## license
[MIT](https://opensource.org/license/mit/)
## author
Made with love and lots of expensive SOTA AI by [sneak](https://sneak.berlin) in
Berlin in the summer of 2025.
Released as a free software gift to the world, no strings attached.
Contact: [sneak@sneak.berlin](mailto:sneak@sneak.berlin)
[https://keys.openpgp.org/vks/v1/by-fingerprint/5539AD00DE4C42F3AFE11575052443F4DF2A55C2](https://keys.openpgp.org/vks/v1/by-fingerprint/5539AD00DE4C42F3AFE11575052443F4DF2A55C2)
Reference in new issue View Git Blame Copy Permalink