Update the rewritten documents for the fixes merged into next
The documents, the example config and the help text are rebased onto the fourteen fixes on next. The sentences those fixes made false now match the code: an unknown config key, a relative index_path and a value that is not a size are refused when the config loads; a backup saves the destination in the local index only after it writes the marker file, and no other command saves it; a backup skips the local index, its side files and the lock file; a binary from go install reports its tag and counts as a release; a declined deletion prompt exits 1 with "deletion not confirmed"; the S3 keys are set before an s3:// storage_url. No document quoted the old messages for an unknown snapshot ID, a wrong private key or an unreadable blob. Model: fable-5-1
This commit is contained in:
1 parent
da300fc91c
commit
a0efb2a5bc
4 files changed
+67
-26
No files matched your search
@@ -63,8 +63,10 @@ It was written to meet these requirements together:
|
|||||||
|
|
||||||
## Getting started
|
## Getting started
|
||||||
|
|
||||||
Installing needs Go 1.26.1 or later. Generating a key pair needs `age-keygen`,
|
Installing needs Go 1.26.1 or later. `go install` builds the newest tagged
|
||||||
which comes with [age](https://age-encryption.org/).
|
version. `vaultik version` then reports that tag and calls the build a release.
|
||||||
|
Generating a key pair needs `age-keygen`, which comes with
|
||||||
|
[age](https://age-encryption.org/).
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Install vaultik.
|
# Install vaultik.
|
||||||
@@ -140,14 +142,18 @@ It does not need the original local index, config file or hostname.
|
|||||||
```sh
|
```sh
|
||||||
# Create a config and point it at the original destination.
|
# Create a config and point it at the original destination.
|
||||||
vaultik config init
|
vaultik config init
|
||||||
vaultik config set storage_url "s3://bucket/prefix?endpoint=https://s3.example.com"
|
|
||||||
vaultik config set s3.access_key_id "..."
|
vaultik config set s3.access_key_id "..."
|
||||||
vaultik config set s3.secret_access_key "..."
|
vaultik config set s3.secret_access_key "..."
|
||||||
|
vaultik config set storage_url "s3://bucket/prefix?endpoint=https://s3.example.com"
|
||||||
|
|
||||||
# See what the destination holds. This needs no private key.
|
# See what the destination holds. This needs no private key.
|
||||||
vaultik snapshot list
|
vaultik snapshot list
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Set the two keys before `storage_url`. On a config that already loads,
|
||||||
|
`config set` refuses an `s3://` URL while they are missing, and says to set them
|
||||||
|
first.
|
||||||
|
|
||||||
On this machine the local index is empty, so the list cannot show snapshot IDs.
|
On this machine the local index is empty, so the list cannot show snapshot IDs.
|
||||||
Each snapshot appears as `<remote only:...>`, with the first 12 characters of
|
Each snapshot appears as `<remote only:...>`, with the first 12 characters of
|
||||||
its remote key. The snapshot ID is stored only in the local index and in the
|
its remote key. The snapshot ID is stored only in the local index and in the
|
||||||
@@ -289,8 +295,9 @@ Sets one value in the config file. The key is a dotted path, as in
|
|||||||
`vaultik config set compression_level 9`.
|
`vaultik config set compression_level 9`.
|
||||||
|
|
||||||
- Comments and formatting in the file are kept.
|
- Comments and formatting in the file are kept.
|
||||||
- A key that does not exist yet is created. A missing parent becomes a list when
|
- A key that no setting has is refused. A key that is not in the file yet is
|
||||||
the next part of the path is a number, and a map otherwise.
|
created. A missing parent becomes a list when the next part of the path is a
|
||||||
|
number, and a map otherwise.
|
||||||
- In a list, the index one past the end appends an entry.
|
- In a list, the index one past the end appends an entry.
|
||||||
- The command prints the key it set, and not the value. A refusal for a value of
|
- The command prints the key it set, and not the value. A refusal for a value of
|
||||||
the wrong type quotes the beginning of that value.
|
the wrong type quotes the beginning of that value.
|
||||||
@@ -301,7 +308,8 @@ is `vaultik config set compression_level 99`. The file is then left unchanged.
|
|||||||
|
|
||||||
A config that does not load yet is checked less strictly. An example is the file
|
A config that does not load yet is checked less strictly. An example is the file
|
||||||
`config init` writes, which has no `storage_url`. There the command refuses only
|
`config init` writes, which has no `storage_url`. There the command refuses only
|
||||||
a value of the wrong type, such as a word where a number belongs.
|
two things: an unknown key, and a value of the wrong type, such as a word where
|
||||||
|
a number belongs.
|
||||||
|
|
||||||
### `snapshot create`
|
### `snapshot create`
|
||||||
|
|
||||||
@@ -332,6 +340,10 @@ holds:
|
|||||||
Every other file is recorded in the new snapshot without being read. A symlink
|
Every other file is recorded in the new snapshot without being read. A symlink
|
||||||
has no content to read, so it is only recorded.
|
has no content to read, so it is only recorded.
|
||||||
|
|
||||||
|
The backup never stores the local index, the `-wal` and `-shm` files beside it,
|
||||||
|
or the lock file `vaultik.pid`. It skips them even when a configured path holds
|
||||||
|
them.
|
||||||
|
|
||||||
Flags:
|
Flags:
|
||||||
|
|
||||||
- `--cron`: print nothing when the backup succeeds. Warnings and errors are
|
- `--cron`: print nothing when the backup succeeds. Warnings and errors are
|
||||||
@@ -798,6 +810,8 @@ The config loads when it is valid YAML and all of these hold:
|
|||||||
- A destination is set. Its `storage_url` has one of the three schemes under
|
- A destination is set. Its `storage_url` has one of the three schemes under
|
||||||
[Destinations](#destinations). An S3 destination has credentials.
|
[Destinations](#destinations). An S3 destination has credentials.
|
||||||
- Every `${ENV:NAME}` names an environment variable with a non-empty value.
|
- Every `${ENV:NAME}` names an environment variable with a non-empty value.
|
||||||
|
- Every key names a setting listed below. An unknown key is refused, and the
|
||||||
|
error names it.
|
||||||
- Every other setting below is within its limits.
|
- Every other setting below is within its limits.
|
||||||
|
|
||||||
Every command that needs the local index warns when group or others can read the
|
Every command that needs the local index warns when group or others can read the
|
||||||
@@ -895,12 +909,15 @@ A command that reads the config fails when the path from `--config` or
|
|||||||
- Only the part before the first `.` is used.
|
- Only the part before the first `.` is used.
|
||||||
- `index_path`
|
- `index_path`
|
||||||
- The path of the local index.
|
- The path of the local index.
|
||||||
|
- It must be an absolute path. A leading `~` is expanded first. The same
|
||||||
|
holds for `VAULTIK_INDEX_PATH`.
|
||||||
- Default: `index.sqlite` in the platform data directory. By default that is
|
- Default: `index.sqlite` in the platform data directory. By default that is
|
||||||
`~/.local/share/vaultik/` on Linux and
|
`~/.local/share/vaultik/` on Linux and
|
||||||
`~/Library/Application Support/vaultik/` on macOS.
|
`~/Library/Application Support/vaultik/` on macOS.
|
||||||
|
|
||||||
Sizes take a number of bytes or a unit. `MB` and `GB` are powers of 1000. `MiB`
|
Sizes take a number of bytes or a unit. `MB` and `GB` are powers of 1000. `MiB`
|
||||||
and `GiB` are powers of 1024.
|
and `GiB` are powers of 1024. A value that is not a size is refused when the
|
||||||
|
config loads. The error names the setting and quotes the value.
|
||||||
|
|
||||||
### Environment variables
|
### Environment variables
|
||||||
|
|
||||||
@@ -1091,9 +1108,11 @@ run `vaultik database delete` on such a machine before it backs up again.
|
|||||||
The local index belongs to one destination. It records which blobs are stored
|
The local index belongs to one destination. It records which blobs are stored
|
||||||
there.
|
there.
|
||||||
|
|
||||||
The first command that changes data saves the configured destination in the
|
A backup saves the configured destination in the local index once it has written
|
||||||
local index. Later, these commands refuse to run when the config names a
|
the marker file there. A backup that fails earlier saves nothing, so a mistyped
|
||||||
different destination:
|
`storage_url` can be corrected. `prune`, `snapshot purge`, `snapshot remove` and
|
||||||
|
`remote nuke` never save it. These commands refuse to run when the config names
|
||||||
|
a different destination from the saved one:
|
||||||
|
|
||||||
- `snapshot create`
|
- `snapshot create`
|
||||||
- `snapshot purge`
|
- `snapshot purge`
|
||||||
@@ -1223,6 +1242,10 @@ Neither flag hides these:
|
|||||||
- `1`: the command failed.
|
- `1`: the command failed.
|
||||||
- `130`: the command was interrupted by Ctrl-C or `SIGTERM`.
|
- `130`: the command was interrupted by Ctrl-C or `SIGTERM`.
|
||||||
|
|
||||||
|
A deletion prompt that is declined or left unanswered ends the command with the
|
||||||
|
error `deletion not confirmed` and exit status `1`. `prune`, `snapshot purge`,
|
||||||
|
`snapshot remove` and `database delete` ask one.
|
||||||
|
|
||||||
An interrupted command removes its temporary files before it exits. It gives up
|
An interrupted command removes its temporary files before it exits. It gives up
|
||||||
after 30 seconds.
|
after 30 seconds.
|
||||||
|
|
||||||
@@ -1446,8 +1469,18 @@ The version a binary reports comes from git. It is the output of
|
|||||||
`script/version` gives this value to the `Makefile`. `script/docker` and
|
`script/version` gives this value to the `Makefile`. `script/docker` and
|
||||||
`script/cibuild` compute it themselves.
|
`script/cibuild` compute it themselves.
|
||||||
|
|
||||||
A binary built without git reports `dev`. An image built by `script/docker` or
|
`make` outside a git checkout, and `docker build .` without `.git`, stamp `dev`.
|
||||||
`script/cibuild` outside a git checkout reports `unknown`.
|
An image built by `script/docker` or `script/cibuild` outside a git checkout
|
||||||
|
reports `unknown`.
|
||||||
|
|
||||||
|
A binary built by plain `go install` or `go build` is not stamped. It reports
|
||||||
|
the version Go recorded in it:
|
||||||
|
|
||||||
|
- A `go install` of a tag reports that tag, such as `v1.0.0`. It counts as a
|
||||||
|
release.
|
||||||
|
- A `go build` of an untagged commit reports a Go pseudo-version, such as
|
||||||
|
`v1.0.1-0.20261110231226-0123456789ab`.
|
||||||
|
- A build where Go recorded no version reports `dev`.
|
||||||
|
|
||||||
`goreleaser` builds the release binaries. It uses the tag without its leading
|
`goreleaser` builds the release binaries. It uses the tag without its leading
|
||||||
`v`, so the tag `v1.0.0` gives `vaultik 1.0.0`. `script/release-snapshot` stamps
|
`v`, so the tag `v1.0.0` gives `vaultik 1.0.0`. `script/release-snapshot` stamps
|
||||||
|
|||||||
@@ -320,6 +320,7 @@ storage_url: "rclone://myremote/path/to/backups"
|
|||||||
# Path to local SQLite index database
|
# Path to local SQLite index database
|
||||||
# This database tracks file state for incremental backups
|
# This database tracks file state for incremental backups
|
||||||
# The VAULTIK_INDEX_PATH environment variable overrides this setting
|
# The VAULTIK_INDEX_PATH environment variable overrides this setting
|
||||||
|
# Both must be absolute paths; a leading ~ is expanded
|
||||||
# Default: the platform data directory, e.g.
|
# Default: the platform data directory, e.g.
|
||||||
# macOS: ~/Library/Application Support/vaultik/index.sqlite
|
# macOS: ~/Library/Application Support/vaultik/index.sqlite
|
||||||
# Linux: ~/.local/share/vaultik/index.sqlite
|
# Linux: ~/.local/share/vaultik/index.sqlite
|
||||||
|
|||||||
+13
-8
@@ -124,9 +124,9 @@ These commands do not build the application:
|
|||||||
|
|
||||||
`Vaultik.CreateSnapshot` runs a backup. It does these steps in order:
|
`Vaultik.CreateSnapshot` runs a backup. It does these steps in order:
|
||||||
|
|
||||||
- `EnsureStorageBinding` checks that the local index belongs to the configured
|
- `markDestination` checks that the local index is not bound to another
|
||||||
destination.
|
destination. It then checks the marker file, writes it, and binds the local
|
||||||
- `markDestination` checks the marker file and then writes it.
|
index to the configured destination.
|
||||||
- `PruneDatabase` removes unfinished snapshots and unused records from the local
|
- `PruneDatabase` removes unfinished snapshots and unused records from the local
|
||||||
index.
|
index.
|
||||||
- `createNamedSnapshot` runs once for each snapshot name.
|
- `createNamedSnapshot` runs once for each snapshot name.
|
||||||
@@ -181,6 +181,8 @@ The walk handles each kind of entry differently:
|
|||||||
- A device node, a named pipe or a socket is skipped.
|
- A device node, a named pipe or a socket is skipped.
|
||||||
- An entry that matches an exclude pattern is skipped. For a directory, so is
|
- An entry that matches an exclude pattern is skipped. For a directory, so is
|
||||||
everything in it.
|
everything in it.
|
||||||
|
- The local index, its SQLite side files and the lock file `vaultik.pid` are
|
||||||
|
skipped, whatever the exclude patterns say.
|
||||||
|
|
||||||
### Deduplication
|
### Deduplication
|
||||||
|
|
||||||
@@ -344,7 +346,7 @@ the private key:
|
|||||||
|
|
||||||
## Prune
|
## Prune
|
||||||
|
|
||||||
`Vaultik.Prune` calls `EnsureStorageBinding` and then does three steps:
|
`Vaultik.Prune` calls `checkStorageBinding` and then does three steps:
|
||||||
|
|
||||||
- `CleanupLocalSnapshots` drops from the local index each snapshot whose
|
- `CleanupLocalSnapshots` drops from the local index each snapshot whose
|
||||||
manifest is gone from the destination.
|
manifest is gone from the destination.
|
||||||
@@ -386,15 +388,18 @@ With no `storage_url`, it builds the S3 implementation from the `s3` settings.
|
|||||||
|
|
||||||
## The marker file and the destination binding
|
## The marker file and the destination binding
|
||||||
|
|
||||||
Two checks keep the local index and the destination in step.
|
These checks keep the local index and the destination in step.
|
||||||
|
|
||||||
- `checkDestinationMarker` looks for `vaultik-destination.txt`. It fails when
|
- `checkDestinationMarker` looks for `vaultik-destination.txt`. It fails when
|
||||||
the file is reported missing while the local index holds snapshots. It skips
|
the file is reported missing while the local index holds snapshots. It skips
|
||||||
the check when the local index belongs to another destination.
|
the check when the local index belongs to another destination.
|
||||||
- `EnsureStorageBinding` records the configured destination in the local index
|
- `checkStorageBinding` fails when the local index is bound to a different
|
||||||
on first use. Later it fails when the config names a different destination.
|
destination from the configured one. It records nothing.
|
||||||
|
- `EnsureStorageBinding` binds an unbound local index to the configured
|
||||||
|
destination. Only `markDestination` calls it, after the marker file is
|
||||||
|
written, so a backup that writes nothing leaves the local index unbound.
|
||||||
|
|
||||||
Every command that takes the lock calls `EnsureStorageBinding` first.
|
Every command that takes the lock calls `checkStorageBinding` first.
|
||||||
|
|
||||||
## Concurrency
|
## Concurrency
|
||||||
|
|
||||||
|
|||||||
+8
-6
@@ -268,9 +268,9 @@ Settings that belong to this copy of the index.
|
|||||||
- `key`: the name of the setting. Primary key.
|
- `key`: the name of the setting. Primary key.
|
||||||
- `value`: its value.
|
- `value`: its value.
|
||||||
|
|
||||||
The only key is `storage_url`. It holds the destination that the index was first
|
The only key is `storage_url`. A backup sets it once it has written the marker
|
||||||
used with. These commands refuse to run when the config names a different
|
file at the destination. No other command sets it. These commands refuse to run
|
||||||
destination:
|
when the config names a different destination:
|
||||||
|
|
||||||
- `snapshot create`
|
- `snapshot create`
|
||||||
- `snapshot purge`
|
- `snapshot purge`
|
||||||
@@ -291,7 +291,8 @@ The migrations that have run.
|
|||||||
|
|
||||||
A backup works on the tables in this order:
|
A backup works on the tables in this order:
|
||||||
|
|
||||||
- It saves the destination in `local_meta` when the index has none yet.
|
- It saves the destination in `local_meta` when the index has none yet. It does
|
||||||
|
so once it has written the marker file at the destination.
|
||||||
- It deletes unfinished snapshots, and the rows that no snapshot uses.
|
- It deletes unfinished snapshots, and the rows that no snapshot uses.
|
||||||
- It creates a row in `snapshots`, with `completed_at` set to `NULL`.
|
- It creates a row in `snapshots`, with `completed_at` set to `NULL`.
|
||||||
- It deletes the `blobs` rows that were never uploaded.
|
- It deletes the `blobs` rows that were never uploaded.
|
||||||
@@ -379,8 +380,9 @@ It does not read the manifest.
|
|||||||
|
|
||||||
### Prune
|
### Prune
|
||||||
|
|
||||||
`prune` first saves the destination in `local_meta` when the index has none yet.
|
`prune` saves nothing in `local_meta`. It refuses to run when `local_meta` holds
|
||||||
It then changes the local index in two ways:
|
a different destination from the configured one. It changes the local index in
|
||||||
|
two ways:
|
||||||
|
|
||||||
- It deletes each snapshot whose manifest is gone from the destination.
|
- It deletes each snapshot whose manifest is gone from the destination.
|
||||||
- It deletes unfinished snapshots, and the rows that no snapshot uses.
|
- It deletes unfinished snapshots, and the rows that no snapshot uses.
|
||||||
|
|||||||
Reference in new issue
Block a user