From a0efb2a5bcf4c5d5508e99c60c8dc60b71687815 Mon Sep 17 00:00:00 2001 From: sneak Date: Wed, 11 Nov 2026 06:02:33 +0000 Subject: [PATCH] Update the rewritten documents for the fixes merged into next The documents, the example config and the help text are rebased onto the fourteen fixes on next. The sentences those fixes made false now match the code: an unknown config key, a relative index_path and a value that is not a size are refused when the config loads; a backup saves the destination in the local index only after it writes the marker file, and no other command saves it; a backup skips the local index, its side files and the lock file; a binary from go install reports its tag and counts as a release; a declined deletion prompt exits 1 with "deletion not confirmed"; the S3 keys are set before an s3:// storage_url. No document quoted the old messages for an unknown snapshot ID, a wrong private key or an unreadable blob. Model: fable-5-1 --- README.md | 57 ++++++++++++++++++++++++++++++-------- configs/config.example.yml | 1 + docs/ARCHITECTURE.md | 21 ++++++++------ docs/DATAMODEL.md | 14 ++++++---- 4 files changed, 67 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 138ea8d..1e49c0a 100644 --- a/README.md +++ b/README.md @@ -63,8 +63,10 @@ It was written to meet these requirements together: ## Getting started -Installing needs Go 1.26.1 or later. Generating a key pair needs `age-keygen`, -which comes with [age](https://age-encryption.org/). +Installing needs Go 1.26.1 or later. `go install` builds the newest tagged +version. `vaultik version` then reports that tag and calls the build a release. +Generating a key pair needs `age-keygen`, which comes with +[age](https://age-encryption.org/). ```sh # Install vaultik. @@ -140,14 +142,18 @@ It does not need the original local index, config file or hostname. ```sh # Create a config and point it at the original destination. vaultik config init -vaultik config set storage_url "s3://bucket/prefix?endpoint=https://s3.example.com" vaultik config set s3.access_key_id "..." vaultik config set s3.secret_access_key "..." +vaultik config set storage_url "s3://bucket/prefix?endpoint=https://s3.example.com" # See what the destination holds. This needs no private key. vaultik snapshot list ``` +Set the two keys before `storage_url`. On a config that already loads, +`config set` refuses an `s3://` URL while they are missing, and says to set them +first. + On this machine the local index is empty, so the list cannot show snapshot IDs. Each snapshot appears as ``, with the first 12 characters of its remote key. The snapshot ID is stored only in the local index and in the @@ -289,8 +295,9 @@ Sets one value in the config file. The key is a dotted path, as in `vaultik config set compression_level 9`. - Comments and formatting in the file are kept. -- A key that does not exist yet is created. A missing parent becomes a list when - the next part of the path is a number, and a map otherwise. +- A key that no setting has is refused. A key that is not in the file yet is + created. A missing parent becomes a list when the next part of the path is a + number, and a map otherwise. - In a list, the index one past the end appends an entry. - The command prints the key it set, and not the value. A refusal for a value of the wrong type quotes the beginning of that value. @@ -301,7 +308,8 @@ is `vaultik config set compression_level 99`. The file is then left unchanged. A config that does not load yet is checked less strictly. An example is the file `config init` writes, which has no `storage_url`. There the command refuses only -a value of the wrong type, such as a word where a number belongs. +two things: an unknown key, and a value of the wrong type, such as a word where +a number belongs. ### `snapshot create` @@ -332,6 +340,10 @@ holds: Every other file is recorded in the new snapshot without being read. A symlink has no content to read, so it is only recorded. +The backup never stores the local index, the `-wal` and `-shm` files beside it, +or the lock file `vaultik.pid`. It skips them even when a configured path holds +them. + Flags: - `--cron`: print nothing when the backup succeeds. Warnings and errors are @@ -798,6 +810,8 @@ The config loads when it is valid YAML and all of these hold: - A destination is set. Its `storage_url` has one of the three schemes under [Destinations](#destinations). An S3 destination has credentials. - Every `${ENV:NAME}` names an environment variable with a non-empty value. +- Every key names a setting listed below. An unknown key is refused, and the + error names it. - Every other setting below is within its limits. Every command that needs the local index warns when group or others can read the @@ -895,12 +909,15 @@ A command that reads the config fails when the path from `--config` or - Only the part before the first `.` is used. - `index_path` - The path of the local index. + - It must be an absolute path. A leading `~` is expanded first. The same + holds for `VAULTIK_INDEX_PATH`. - Default: `index.sqlite` in the platform data directory. By default that is `~/.local/share/vaultik/` on Linux and `~/Library/Application Support/vaultik/` on macOS. Sizes take a number of bytes or a unit. `MB` and `GB` are powers of 1000. `MiB` -and `GiB` are powers of 1024. +and `GiB` are powers of 1024. A value that is not a size is refused when the +config loads. The error names the setting and quotes the value. ### Environment variables @@ -1091,9 +1108,11 @@ run `vaultik database delete` on such a machine before it backs up again. The local index belongs to one destination. It records which blobs are stored there. -The first command that changes data saves the configured destination in the -local index. Later, these commands refuse to run when the config names a -different destination: +A backup saves the configured destination in the local index once it has written +the marker file there. A backup that fails earlier saves nothing, so a mistyped +`storage_url` can be corrected. `prune`, `snapshot purge`, `snapshot remove` and +`remote nuke` never save it. These commands refuse to run when the config names +a different destination from the saved one: - `snapshot create` - `snapshot purge` @@ -1223,6 +1242,10 @@ Neither flag hides these: - `1`: the command failed. - `130`: the command was interrupted by Ctrl-C or `SIGTERM`. +A deletion prompt that is declined or left unanswered ends the command with the +error `deletion not confirmed` and exit status `1`. `prune`, `snapshot purge`, +`snapshot remove` and `database delete` ask one. + An interrupted command removes its temporary files before it exits. It gives up after 30 seconds. @@ -1446,8 +1469,18 @@ The version a binary reports comes from git. It is the output of `script/version` gives this value to the `Makefile`. `script/docker` and `script/cibuild` compute it themselves. -A binary built without git reports `dev`. An image built by `script/docker` or -`script/cibuild` outside a git checkout reports `unknown`. +`make` outside a git checkout, and `docker build .` without `.git`, stamp `dev`. +An image built by `script/docker` or `script/cibuild` outside a git checkout +reports `unknown`. + +A binary built by plain `go install` or `go build` is not stamped. It reports +the version Go recorded in it: + +- A `go install` of a tag reports that tag, such as `v1.0.0`. It counts as a + release. +- A `go build` of an untagged commit reports a Go pseudo-version, such as + `v1.0.1-0.20261110231226-0123456789ab`. +- A build where Go recorded no version reports `dev`. `goreleaser` builds the release binaries. It uses the tag without its leading `v`, so the tag `v1.0.0` gives `vaultik 1.0.0`. `script/release-snapshot` stamps diff --git a/configs/config.example.yml b/configs/config.example.yml index dbcbcb3..4186342 100644 --- a/configs/config.example.yml +++ b/configs/config.example.yml @@ -320,6 +320,7 @@ storage_url: "rclone://myremote/path/to/backups" # Path to local SQLite index database # This database tracks file state for incremental backups # The VAULTIK_INDEX_PATH environment variable overrides this setting +# Both must be absolute paths; a leading ~ is expanded # Default: the platform data directory, e.g. # macOS: ~/Library/Application Support/vaultik/index.sqlite # Linux: ~/.local/share/vaultik/index.sqlite diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index a02a7e2..c2230b6 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -124,9 +124,9 @@ These commands do not build the application: `Vaultik.CreateSnapshot` runs a backup. It does these steps in order: -- `EnsureStorageBinding` checks that the local index belongs to the configured - destination. -- `markDestination` checks the marker file and then writes it. +- `markDestination` checks that the local index is not bound to another + destination. It then checks the marker file, writes it, and binds the local + index to the configured destination. - `PruneDatabase` removes unfinished snapshots and unused records from the local index. - `createNamedSnapshot` runs once for each snapshot name. @@ -181,6 +181,8 @@ The walk handles each kind of entry differently: - A device node, a named pipe or a socket is skipped. - An entry that matches an exclude pattern is skipped. For a directory, so is everything in it. +- The local index, its SQLite side files and the lock file `vaultik.pid` are + skipped, whatever the exclude patterns say. ### Deduplication @@ -344,7 +346,7 @@ the private key: ## Prune -`Vaultik.Prune` calls `EnsureStorageBinding` and then does three steps: +`Vaultik.Prune` calls `checkStorageBinding` and then does three steps: - `CleanupLocalSnapshots` drops from the local index each snapshot whose manifest is gone from the destination. @@ -386,15 +388,18 @@ With no `storage_url`, it builds the S3 implementation from the `s3` settings. ## The marker file and the destination binding -Two checks keep the local index and the destination in step. +These checks keep the local index and the destination in step. - `checkDestinationMarker` looks for `vaultik-destination.txt`. It fails when the file is reported missing while the local index holds snapshots. It skips the check when the local index belongs to another destination. -- `EnsureStorageBinding` records the configured destination in the local index - on first use. Later it fails when the config names a different destination. +- `checkStorageBinding` fails when the local index is bound to a different + destination from the configured one. It records nothing. +- `EnsureStorageBinding` binds an unbound local index to the configured + destination. Only `markDestination` calls it, after the marker file is + written, so a backup that writes nothing leaves the local index unbound. -Every command that takes the lock calls `EnsureStorageBinding` first. +Every command that takes the lock calls `checkStorageBinding` first. ## Concurrency diff --git a/docs/DATAMODEL.md b/docs/DATAMODEL.md index 25e5ad6..30b3874 100644 --- a/docs/DATAMODEL.md +++ b/docs/DATAMODEL.md @@ -268,9 +268,9 @@ Settings that belong to this copy of the index. - `key`: the name of the setting. Primary key. - `value`: its value. -The only key is `storage_url`. It holds the destination that the index was first -used with. These commands refuse to run when the config names a different -destination: +The only key is `storage_url`. A backup sets it once it has written the marker +file at the destination. No other command sets it. These commands refuse to run +when the config names a different destination: - `snapshot create` - `snapshot purge` @@ -291,7 +291,8 @@ The migrations that have run. A backup works on the tables in this order: -- It saves the destination in `local_meta` when the index has none yet. +- It saves the destination in `local_meta` when the index has none yet. It does + so once it has written the marker file at the destination. - It deletes unfinished snapshots, and the rows that no snapshot uses. - It creates a row in `snapshots`, with `completed_at` set to `NULL`. - It deletes the `blobs` rows that were never uploaded. @@ -379,8 +380,9 @@ It does not read the manifest. ### Prune -`prune` first saves the destination in `local_meta` when the index has none yet. -It then changes the local index in two ways: +`prune` saves nothing in `local_meta`. It refuses to run when `local_meta` holds +a different destination from the configured one. It changes the local index in +two ways: - It deletes each snapshot whose manifest is gone from the destination. - It deletes unfinished snapshots, and the rows that no snapshot uses.