Go on with a backup when the destination marker cannot be checked (closes #389)
check / check (push) Waiting to run

A source host whose credentials allow only PUT, which the documents
say is enough to back up, failed every backup after its first: the
marker check looks the marker up with an S3 HEAD, which those
credentials are refused, and any error other than not found was
fatal.

Such an error now means the check could not be made: it is logged at
info level, so --cron stays silent, and the command goes on. Not found
still stops it. The documents now say so, and that such a host does
not notice an emptied destination, so vaultik database delete must be
run on it after the destination is emptied. The fault-injecting test
store gains a Stat hook for the test.

Model: opus-5-5
This commit was merged in pull request #392.
This commit is contained in:
2026-10-15 02:02:35 +02:00
parent 6a57750a7b
commit 850e2de25a
9 changed files with 132 additions and 54 deletions
+27 -18
View File
@@ -332,7 +332,9 @@ value where a list belongs or a word where a number belongs), is refused.
command fails before the backup starts.
* Fails before it writes anything when a `file://` destination directory
does not exist, or when the local index holds snapshots and the
destination has no `vaultik-destination.txt` (see
destination answers that it has no `vaultik-destination.txt`. When that
look-up fails for any reason other than not found, for example when S3
refuses it to credentials that allow only PUT, the check is skipped (see
[remote storage layout](#remote-storage-layout))
**`snapshot list`**: Show every snapshot known to this host — the union
@@ -368,9 +370,9 @@ gives: revert the configuration, or run `vaultik database delete`.
If the destination store cannot be listed (unmounted volume,
permission denied, network down), the command warns, falls back to the
local index alone, and still exits zero. It does the same when the
destination has no `vaultik-destination.txt` while the local index
holds snapshots stored there, as when a volume mounted at the
destination is unplugged and leaves its empty mount point behind.
destination answers that it has no `vaultik-destination.txt` while the
local index holds snapshots stored there, as when a volume mounted at
the destination is unplugged and leaves its empty mount point behind.
* `--json`: Output in JSON format. Each entry carries `locally_tracked`
(whether the snapshot is in the local index), `remote_key` (the full
64-character storage key), and `remote_present` (whether it was seen
@@ -413,8 +415,8 @@ with `--local-only`, the snapshot must be in the local index. Local row
cleanup (files, chunks, blobs the snapshot was the last referrer for)
runs automatically. If removing the snapshot's metadata from the
destination store fails (for example because the store is unreachable,
or the local index holds snapshots and the destination has no
`vaultik-destination.txt`), the local-DB removal still completes and a
or the local index holds snapshots and the destination answers that it
has no `vaultik-destination.txt`), the local-DB removal still completes and a
warning is emitted; run `vaultik snapshot remove <snapshot-id>` again
once the store is reachable to remove the snapshot's metadata from it
(`vaultik prune` does not). For a snapshot that is only on the
@@ -457,9 +459,9 @@ be deleted. Step (3) reads all remote manifests — network cost scales with
the number of snapshots.
`snapshot create --prune` runs the same cleanup automatically; this is the
manual entry point for the same work. If the destination store cannot
be listed, or the local index holds snapshots and the destination has
no `vaultik-destination.txt` (an unplugged volume's empty mount point),
prune fails and deletes nothing. `snapshot purge` does the same. If a
be listed, or the local index holds snapshots and the destination answers
that it has no `vaultik-destination.txt` (an unplugged volume's empty mount
point), prune fails and deletes nothing. `snapshot purge` does the same. If a
blob cannot be deleted, the command (and `snapshot create --prune`)
exits non-zero.
* `--force`: Skip confirmation prompt
@@ -494,8 +496,8 @@ and irreversible. This is the single supported way to wipe the entire
destination store. A snapshot is removed from the local index only once
its metadata is deleted from the destination store. If the destination
store cannot be listed, or the local index holds snapshots and the
destination has no `vaultik-destination.txt` (an unplugged volume's
empty mount point), the command fails before it removes anything. If a
destination answers that it has no `vaultik-destination.txt` (an unplugged
volume's empty mount point), the command fails before it removes anything. If a
listing afterwards still finds anything under `metadata/` or `blobs/`,
the command names each object left and exits non-zero; running it again
deletes what is left.
@@ -577,15 +579,22 @@ For an implementation-level view of the internals — the data model, the
```
* `vaultik-destination.txt` holds one fixed line of text, with no host
name, path or time. A destination without it is treated as not
present while the local index holds snapshots stored there:
`snapshot create` refuses to write to it, `prune`, `snapshot purge`
and `remote nuke` fail, and `snapshot list` and `snapshot remove`
behave as for a destination that cannot be listed. This keeps an
unplugged volume's empty mount point from being taken for a
destination whose snapshots are all gone. Mount
name, path or time. While the local index holds snapshots stored
there, a destination that answers a look-up of it with not found is
treated as not present: `snapshot create` refuses to write to it,
`prune`, `snapshot purge` and `remote nuke` fail, and `snapshot list`
and `snapshot remove` behave as for a destination that cannot be
listed. This keeps an unplugged volume's empty mount point from being
taken for a destination whose snapshots are all gone. Mount
the volume if it is removable; if the destination was emptied on
purpose, run `vaultik database delete` and back up again.
When the look-up fails for any reason other than not found, the check
is skipped and the command goes on. S3 refuses it on every run to
credentials that allow only PUT, so a host with such credentials does
not notice a destination emptied since its last backup, and its next
backup would leave out the data its local index records as already
stored. After the destination is emptied, run
`vaultik database delete` on such a host before it backs up again.
* Blobs are two-level directory sharded using the first 4 hex chars of the blob hash
* `db.zst.age` is a binary SQLite database (zstd compressed, age encrypted)
containing all file metadata, chunk mappings, and relationships for the snapshot