Fix the ten findings of the review on the destination, the marker file and chunk sizes
check / check (push) Waiting to run

README.md: the "Getting started" destination advice, the Full Disk Access
paragraph moved above the block with the reopen step, the mount-point
case, the remote nuke failure sentence, the hostname setting, the upload
order under "Sharing a destination", the confirmation list and the dev
stamp sentence. configs/config.example.yml: the storage_url and
blob_size_limit comments. docs/ARCHITECTURE.md and docs/DATAMODEL.md:
the Snapshot ID and Marker file terms, and "marker file" in place of
"destination marker".

Model: fable-5-1
This commit is contained in:
clawbot committed 2026-11-11 09:11:51 +00:00
1 parent a8b9c57e93
commit 6e4e8062a1
4 files changed
+46 -30

No files matched your search

+27 -21
View File
@@ -67,6 +67,11 @@ Installing needs Go 1.26.1 or later. `go install` builds the newest tagged
version. `vaultik version` then reports that tag. Generating a key pair needs
`age-keygen`, which comes with [age](https://age-encryption.org/).
On macOS, grant your terminal app Full Disk Access before the first backup. Then
quit and reopen the terminal. The setting is under System Settings, Privacy &
Security, Full Disk Access. Without it the backup stops with a permission error
on a protected directory, unless you give `--skip-errors`.
```sh
# Install vaultik.
go install sneak.berlin/go/vaultik/cmd/vaultik@latest
@@ -92,15 +97,11 @@ vaultik snapshot create
vaultik snapshot list
```
The block uses macOS paths. On another system, put the destination in a
directory of your own. The default config backs up your home directory, with a
list of excludes. On macOS it also backs up `/Applications`. A configured path
that does not exist stops the backup, unless you give `--skip-errors`.
On macOS, grant your terminal app Full Disk Access before the first backup. The
setting is under System Settings, Privacy & Security, Full Disk Access. Without
it the backup stops with a permission error on a protected directory, unless you
give `--skip-errors`.
The block uses macOS paths. On another system, put the destination outside every
path the config backs up, such as on another disk. The default config backs up
your home directory, with a list of excludes. On macOS it also backs up
`/Applications`. A configured path that does not exist stops the backup, unless
you give `--skip-errors`.
## Everyday use
@@ -407,7 +408,7 @@ entry in it that is not excluded.
A backup does not find a mount point in these cases:
- The volume is not mounted.
- The directory holds no entry that the backup reads.
- The directory holds no entry, or only excluded ones.
- The directory is now excluded.
- The directory is no longer under a configured path.
@@ -738,8 +739,8 @@ directory at the destination is deleted.
The command fails before it deletes anything when the destination cannot be
listed, or has no marker file although the local index holds snapshots.
If objects remain afterwards, the command names each one and exits non-zero.
Running it again deletes what is left.
If a delete fails, the command exits non-zero. Running it again deletes what is
left.
### `database delete`
@@ -911,8 +912,8 @@ A command that reads the config fails when the path from `--config` or
- `0s` deletes such data at once. A negative value is refused.
- See [Sharing a destination](#sharing-a-destination).
- `hostname`
- The hostname used in snapshot IDs. Default: the system hostname.
- Only the part before the first `.` is used.
- The hostname used in snapshot IDs, up to its first `.`. Default: the
system hostname.
- `index_path`
- The path of the local index.
- It must be an absolute path. A leading `~` is expanded first. The same
@@ -1151,8 +1152,8 @@ Give each machine and each user its own destination, such as its own prefix or
directory. The rest of this section applies only when two of them share one.
A prune from one machine can meet a backup from another that is still running. A
backup uploads its blobs first, then its database, and its manifest last. Until
the manifest exists, no manifest lists the new blobs.
backup uploads a snapshot's blobs first, then its database, and its manifest
last. Until the manifest exists, no manifest lists the new blobs.
`prune_grace_period` protects those blobs. A prune keeps a blob that no manifest
lists until the blob is older than the grace period. It keeps a directory under
@@ -1257,8 +1258,13 @@ Neither flag hides these:
- `130`: the command was interrupted by Ctrl-C or `SIGTERM`.
A deletion prompt that is declined or left unanswered ends the command with the
error `deletion not confirmed` and exit status `1`. `prune`, `snapshot purge`,
`snapshot remove` and `database delete` ask one.
error `deletion not confirmed` and exit status `1`. These commands ask for
confirmation:
- `prune`
- `snapshot purge`
- `snapshot remove`
- `database delete`
An interrupted command removes its temporary files before it exits. It gives up
after 30 seconds.
@@ -1487,9 +1493,9 @@ The version a binary reports comes from git. It is the output of
A plain `docker build .` adds no `-dirty`. The `Dockerfile` runs `git describe`
without `--dirty`.
`make` outside a git checkout, and `docker build .` without `.git`, stamp `dev`.
An image built by `script/docker` or `script/cibuild` outside a git checkout
reports `unknown`.
Outside a git checkout, `make` stamps `dev`. So does `docker build .` without
`.git`. An image built by `script/docker` or `script/cibuild` outside a git
checkout reports `unknown`.
A binary built by plain `go install` or `go build` is not stamped. It reports
the version Go recorded in it:
+6 -4
View File
@@ -273,7 +273,8 @@ snapshots:
exclude:
- "*.tmp"
# Storage URL - use either this OR the s3 section below
# Storage URL. When it is not set, the endpoint, bucket and prefix in the s3
# section below are used instead.
# Supports: s3://bucket/prefix, file:///path, rclone://remote/path
# An s3:// URL also needs s3.access_key_id and s3.secret_access_key below
# A file:// path must be absolute, and the directory must already exist:
@@ -339,9 +340,10 @@ storage_url: "rclone://myremote/path/to/backups"
# Must be at least four times chunk_size (the largest chunk the chunker can
# emit); a smaller limit would let a single-chunk blob exceed it.
# Chunking uses no secret (the parameters of the FastCDC chunking algorithm
# are fixed and public). At a large limit a blob holds hundreds of chunks, so
# individual chunk lengths are not visible in its size; lowering the limit
# toward chunk_size exposes them.
# are fixed and public). A full blob at a large limit holds hundreds of
# chunks, so its size does not show any one chunk's length. The last blob of
# each configured path can hold a single chunk. Lowering the limit toward
# chunk_size exposes chunk lengths.
# A backup needs free temporary space, because each blob is written whole to
# a temporary file before it is uploaded (up to about blob_size_limit; an
# rclone destination that cannot stream uploads needs about twice that) and
+8 -4
View File
@@ -16,12 +16,16 @@ Two other documents cover the stored data:
## Terms
- **Snapshot**: one backup of the paths listed under one name in the config.
- **Snapshot ID**: the name of a snapshot, in the form
`<hostname>_<snapshot-name>_<timestamp>`.
- **Chunk**: a piece of a file's content. A chunk is named by the SHA-256 hash
of its content.
- **Blob**: one object at the destination. It holds many chunks, compressed and
then encrypted.
- **Local index**: the SQLite database on the backed-up machine.
- **Destination**: the place the backups are stored.
- **Marker file**: `vaultik-destination.txt` at the destination. See
[The marker file](REPOSTRUCTURE.md#the-marker-file).
- **Manifest**: the unencrypted list of the blobs one snapshot uses.
- **Remote key**: a hash of the snapshot ID. It names the snapshot's directory
at the destination.
@@ -292,10 +296,10 @@ therefore marks a backup that did not finish.
checks its chunk hashes.
Given a snapshot ID, the restore looks it up in the local index. When it is not
there, the restore checks the destination marker and lists the snapshot's
directory. Given a remote key, it reads the local index only to check the
destination marker. It does not read the manifest. It finds each file's chunks
and blobs through the downloaded database.
there, the restore checks the marker file and lists the snapshot's directory.
Given a remote key, it reads the local index only to check the marker file. It
does not read the manifest. It finds each file's chunks and blobs through the
downloaded database.
### Checks on the downloaded database
+5 -1
View File
@@ -17,11 +17,15 @@ copy that was uploaded with the snapshot.
## Terms
- **Snapshot**: one backup of the paths listed under one name in the config.
- **Snapshot ID**: the name of a snapshot, in the form
`<hostname>_<snapshot-name>_<timestamp>`.
- **Chunk**: a piece of a file's content, named by the SHA-256 hash of that
content.
- **Blob**: one object at the destination. It holds many chunks, compressed and
then encrypted.
- **Destination**: the place the backups are stored.
- **Marker file**: `vaultik-destination.txt` at the destination. See
[The marker file](REPOSTRUCTURE.md#the-marker-file).
- **Manifest**: the unencrypted list of the blobs one snapshot uses.
- **Remote key**: a hash of the snapshot ID. It names the snapshot's directory
at the destination.
@@ -367,7 +371,7 @@ still be in the `-wal` file beside it.
A restore reads no file, chunk or blob rows from the local index. It reads
`snapshots` and `local_meta` there, to look up a snapshot ID and to check the
destination marker.
marker file.
- It downloads and decrypts `db.zst.age` into a temporary directory.
- It opens that file read-only. It applies no migration to it.