diff --git a/README.md b/README.md index 138ea8d..1e49c0a 100644 --- a/README.md +++ b/README.md @@ -63,8 +63,10 @@ It was written to meet these requirements together: ## Getting started -Installing needs Go 1.26.1 or later. Generating a key pair needs `age-keygen`, -which comes with [age](https://age-encryption.org/). +Installing needs Go 1.26.1 or later. `go install` builds the newest tagged +version. `vaultik version` then reports that tag and calls the build a release. +Generating a key pair needs `age-keygen`, which comes with +[age](https://age-encryption.org/). ```sh # Install vaultik. @@ -140,14 +142,18 @@ It does not need the original local index, config file or hostname. ```sh # Create a config and point it at the original destination. vaultik config init -vaultik config set storage_url "s3://bucket/prefix?endpoint=https://s3.example.com" vaultik config set s3.access_key_id "..." vaultik config set s3.secret_access_key "..." +vaultik config set storage_url "s3://bucket/prefix?endpoint=https://s3.example.com" # See what the destination holds. This needs no private key. vaultik snapshot list ``` +Set the two keys before `storage_url`. On a config that already loads, +`config set` refuses an `s3://` URL while they are missing, and says to set them +first. + On this machine the local index is empty, so the list cannot show snapshot IDs. Each snapshot appears as ``, with the first 12 characters of its remote key. The snapshot ID is stored only in the local index and in the @@ -289,8 +295,9 @@ Sets one value in the config file. The key is a dotted path, as in `vaultik config set compression_level 9`. - Comments and formatting in the file are kept. -- A key that does not exist yet is created. A missing parent becomes a list when - the next part of the path is a number, and a map otherwise. +- A key that no setting has is refused. A key that is not in the file yet is + created. A missing parent becomes a list when the next part of the path is a + number, and a map otherwise. - In a list, the index one past the end appends an entry. - The command prints the key it set, and not the value. A refusal for a value of the wrong type quotes the beginning of that value. @@ -301,7 +308,8 @@ is `vaultik config set compression_level 99`. The file is then left unchanged. A config that does not load yet is checked less strictly. An example is the file `config init` writes, which has no `storage_url`. There the command refuses only -a value of the wrong type, such as a word where a number belongs. +two things: an unknown key, and a value of the wrong type, such as a word where +a number belongs. ### `snapshot create` @@ -332,6 +340,10 @@ holds: Every other file is recorded in the new snapshot without being read. A symlink has no content to read, so it is only recorded. +The backup never stores the local index, the `-wal` and `-shm` files beside it, +or the lock file `vaultik.pid`. It skips them even when a configured path holds +them. + Flags: - `--cron`: print nothing when the backup succeeds. Warnings and errors are @@ -798,6 +810,8 @@ The config loads when it is valid YAML and all of these hold: - A destination is set. Its `storage_url` has one of the three schemes under [Destinations](#destinations). An S3 destination has credentials. - Every `${ENV:NAME}` names an environment variable with a non-empty value. +- Every key names a setting listed below. An unknown key is refused, and the + error names it. - Every other setting below is within its limits. Every command that needs the local index warns when group or others can read the @@ -895,12 +909,15 @@ A command that reads the config fails when the path from `--config` or - Only the part before the first `.` is used. - `index_path` - The path of the local index. + - It must be an absolute path. A leading `~` is expanded first. The same + holds for `VAULTIK_INDEX_PATH`. - Default: `index.sqlite` in the platform data directory. By default that is `~/.local/share/vaultik/` on Linux and `~/Library/Application Support/vaultik/` on macOS. Sizes take a number of bytes or a unit. `MB` and `GB` are powers of 1000. `MiB` -and `GiB` are powers of 1024. +and `GiB` are powers of 1024. A value that is not a size is refused when the +config loads. The error names the setting and quotes the value. ### Environment variables @@ -1091,9 +1108,11 @@ run `vaultik database delete` on such a machine before it backs up again. The local index belongs to one destination. It records which blobs are stored there. -The first command that changes data saves the configured destination in the -local index. Later, these commands refuse to run when the config names a -different destination: +A backup saves the configured destination in the local index once it has written +the marker file there. A backup that fails earlier saves nothing, so a mistyped +`storage_url` can be corrected. `prune`, `snapshot purge`, `snapshot remove` and +`remote nuke` never save it. These commands refuse to run when the config names +a different destination from the saved one: - `snapshot create` - `snapshot purge` @@ -1223,6 +1242,10 @@ Neither flag hides these: - `1`: the command failed. - `130`: the command was interrupted by Ctrl-C or `SIGTERM`. +A deletion prompt that is declined or left unanswered ends the command with the +error `deletion not confirmed` and exit status `1`. `prune`, `snapshot purge`, +`snapshot remove` and `database delete` ask one. + An interrupted command removes its temporary files before it exits. It gives up after 30 seconds. @@ -1446,8 +1469,18 @@ The version a binary reports comes from git. It is the output of `script/version` gives this value to the `Makefile`. `script/docker` and `script/cibuild` compute it themselves. -A binary built without git reports `dev`. An image built by `script/docker` or -`script/cibuild` outside a git checkout reports `unknown`. +`make` outside a git checkout, and `docker build .` without `.git`, stamp `dev`. +An image built by `script/docker` or `script/cibuild` outside a git checkout +reports `unknown`. + +A binary built by plain `go install` or `go build` is not stamped. It reports +the version Go recorded in it: + +- A `go install` of a tag reports that tag, such as `v1.0.0`. It counts as a + release. +- A `go build` of an untagged commit reports a Go pseudo-version, such as + `v1.0.1-0.20261110231226-0123456789ab`. +- A build where Go recorded no version reports `dev`. `goreleaser` builds the release binaries. It uses the tag without its leading `v`, so the tag `v1.0.0` gives `vaultik 1.0.0`. `script/release-snapshot` stamps diff --git a/configs/config.example.yml b/configs/config.example.yml index dbcbcb3..4186342 100644 --- a/configs/config.example.yml +++ b/configs/config.example.yml @@ -320,6 +320,7 @@ storage_url: "rclone://myremote/path/to/backups" # Path to local SQLite index database # This database tracks file state for incremental backups # The VAULTIK_INDEX_PATH environment variable overrides this setting +# Both must be absolute paths; a leading ~ is expanded # Default: the platform data directory, e.g. # macOS: ~/Library/Application Support/vaultik/index.sqlite # Linux: ~/.local/share/vaultik/index.sqlite diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index a02a7e2..c2230b6 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -124,9 +124,9 @@ These commands do not build the application: `Vaultik.CreateSnapshot` runs a backup. It does these steps in order: -- `EnsureStorageBinding` checks that the local index belongs to the configured - destination. -- `markDestination` checks the marker file and then writes it. +- `markDestination` checks that the local index is not bound to another + destination. It then checks the marker file, writes it, and binds the local + index to the configured destination. - `PruneDatabase` removes unfinished snapshots and unused records from the local index. - `createNamedSnapshot` runs once for each snapshot name. @@ -181,6 +181,8 @@ The walk handles each kind of entry differently: - A device node, a named pipe or a socket is skipped. - An entry that matches an exclude pattern is skipped. For a directory, so is everything in it. +- The local index, its SQLite side files and the lock file `vaultik.pid` are + skipped, whatever the exclude patterns say. ### Deduplication @@ -344,7 +346,7 @@ the private key: ## Prune -`Vaultik.Prune` calls `EnsureStorageBinding` and then does three steps: +`Vaultik.Prune` calls `checkStorageBinding` and then does three steps: - `CleanupLocalSnapshots` drops from the local index each snapshot whose manifest is gone from the destination. @@ -386,15 +388,18 @@ With no `storage_url`, it builds the S3 implementation from the `s3` settings. ## The marker file and the destination binding -Two checks keep the local index and the destination in step. +These checks keep the local index and the destination in step. - `checkDestinationMarker` looks for `vaultik-destination.txt`. It fails when the file is reported missing while the local index holds snapshots. It skips the check when the local index belongs to another destination. -- `EnsureStorageBinding` records the configured destination in the local index - on first use. Later it fails when the config names a different destination. +- `checkStorageBinding` fails when the local index is bound to a different + destination from the configured one. It records nothing. +- `EnsureStorageBinding` binds an unbound local index to the configured + destination. Only `markDestination` calls it, after the marker file is + written, so a backup that writes nothing leaves the local index unbound. -Every command that takes the lock calls `EnsureStorageBinding` first. +Every command that takes the lock calls `checkStorageBinding` first. ## Concurrency diff --git a/docs/DATAMODEL.md b/docs/DATAMODEL.md index 25e5ad6..30b3874 100644 --- a/docs/DATAMODEL.md +++ b/docs/DATAMODEL.md @@ -268,9 +268,9 @@ Settings that belong to this copy of the index. - `key`: the name of the setting. Primary key. - `value`: its value. -The only key is `storage_url`. It holds the destination that the index was first -used with. These commands refuse to run when the config names a different -destination: +The only key is `storage_url`. A backup sets it once it has written the marker +file at the destination. No other command sets it. These commands refuse to run +when the config names a different destination: - `snapshot create` - `snapshot purge` @@ -291,7 +291,8 @@ The migrations that have run. A backup works on the tables in this order: -- It saves the destination in `local_meta` when the index has none yet. +- It saves the destination in `local_meta` when the index has none yet. It does + so once it has written the marker file at the destination. - It deletes unfinished snapshots, and the rows that no snapshot uses. - It creates a row in `snapshots`, with `completed_at` set to `NULL`. - It deletes the `blobs` rows that were never uploaded. @@ -379,8 +380,9 @@ It does not read the manifest. ### Prune -`prune` first saves the destination in `local_meta` when the index has none yet. -It then changes the local index in two ways: +`prune` saves nothing in `local_meta`. It refuses to run when `local_meta` holds +a different destination from the configured one. It changes the local index in +two ways: - It deletes each snapshot whose manifest is gone from the destination. - It deletes unfinished snapshots, and the rows that no snapshot uses.