From c8ad289e4691b20bab6e69722cc2bf047ca8d72d Mon Sep 17 00:00:00 2001 From: sneak Date: Fri, 6 Nov 2026 18:27:15 +0000 Subject: [PATCH] Rewrite docs/REPOSTRUCTURE.md sentence by sentence Rewritten from scratch against the code on next: the layout of the destination, what it protects, the accepted risks, the order of writes and the rules that keep prune from deleting data in use. Corrected: a restore needs only db.zst.age and the blobs it reads, not the manifest. The snapshot ID form without a name is gone, since no command creates one. Model: fable-5-1 --- docs/REPOSTRUCTURE.md | 572 +++++++++++++++++++++++------------------- 1 file changed, 311 insertions(+), 261 deletions(-) diff --git a/docs/REPOSTRUCTURE.md b/docs/REPOSTRUCTURE.md index 81f34c9..99acd1d 100644 --- a/docs/REPOSTRUCTURE.md +++ b/docs/REPOSTRUCTURE.md @@ -1,158 +1,167 @@ -# Vaultik S3 Repository Structure +# vaultik destination layout -This document describes the structure and organization of data stored in the S3 -bucket by Vaultik. +This document describes what `vaultik` stores at the destination. It covers the +layout, what the layout protects, and the risks it accepts. -## Overview +The destination is the place the backups are stored. It is an S3 bucket and +prefix, a local directory or an rclone remote. The layout is the same on all +three. -Vaultik stores all backup data in an S3-compatible object store. The repository -consists of two main components, plus a marker file: +## Terms -1. **Blobs** - The actual backup data (content-addressed, encrypted) -2. **Metadata** - Snapshot information and manifests (partially encrypted) +- **Snapshot**: one backup of the paths listed under one name in the config. +- **Snapshot ID**: the name of a snapshot, in the form + `__`. +- **Chunk**: a piece of a file's content, named by the SHA-256 hash of that + content. +- **Blob**: one object under `blobs/`. It holds many chunks. +- **Local index**: the SQLite database on the backed-up machine. +- **Recipient**: an age public key that backups are encrypted to. -## Directory Structure +## Layout ``` -// +/ ├── vaultik-destination.txt ├── blobs/ │ └── / │ └── / -│ └── +│ └── └── metadata/ └── / ├── db.zst.age └── manifest.json.zst ``` -The metadata subdirectory is named with the **remote key**, a one-way hash of -the snapshot ID, not with the human-readable snapshot ID itself. See -[Remote Key Derivation](#remote-key-derivation). +The destination holds three things: -## Marker File (`vaultik-destination.txt`) +- `vaultik-destination.txt`, the marker file +- `blobs/`, which holds the file contents +- `metadata/`, which holds one directory for each snapshot + +## The marker file + +Every backup writes `vaultik-destination.txt` before it uploads anything else. +The file holds one fixed line of text. It names no host, path or time. + +The marker tells an emptied or missing destination apart from a new one. An +unplugged volume leaves an empty directory at its mount point. Without the +marker, that directory would look like a destination whose snapshots are all +gone. + +The check applies while the local index holds snapshots. When the destination +then answers that the marker does not exist, commands treat the destination as +not present: + +- `snapshot create` refuses to write to it. +- `prune`, `snapshot purge` and `remote nuke` fail. +- `snapshot list` shows the local index alone. +- `snapshot remove` deletes nothing at the destination. + +The check is skipped when the look-up fails for any other reason. S3 refuses +the look-up to credentials that can only write. A machine with such credentials +therefore does not notice an emptied destination. Its next backup would leave +out the data its local index records as stored. After a destination is emptied, +run `vaultik database delete` on such a machine before it backs up again. -Every backup writes this file at the root before it uploads anything. It holds -one fixed line of plain text and names no host, path or time. While the local -index holds snapshots stored there, a destination that answers a look-up of the -marker with not found is treated as not present: `snapshot create` refuses to -write to it, and `prune`, `snapshot purge`, `snapshot remove`, `snapshot list` -and `remote nuke` behave as for a destination that cannot be listed. An -unplugged volume leaves an empty directory at its mount point, and without the -marker that directory would read as a destination whose snapshots are all gone. `remote nuke` deletes the marker last. -When the look-up of the marker fails for any reason other than not found, the -check is skipped and the command goes on. S3 refuses it on every run to -credentials that allow only PUT, so a host with such credentials does not notice -a destination emptied since its last backup, and its next backup would leave out -the data its local index records as already stored. After the destination is -emptied, run `vaultik database delete` on such a host before it backs up again. +## Blobs -## Blobs Directory (`blobs/`) +A blob holds the content of many chunks, one after another. -### Structure - -- **Path format**: `blobs///` +- **Path**: `blobs///`. - **Example**: - `blobs/ca/fe/cafebabe1234567890abcdef1234567890abcdef1234567890abcdef12345678` -- **Sharding**: The two-level directory structure (using the first 4 characters - of the hash) prevents any single directory from containing too many objects + `blobs/ca/fe/cafebabe1234567890abcdef1234567890abcdef1234567890abcdef12345678`. +- **Format**: the chunk data, compressed with zstd and then encrypted with age. +- **Recipients**: the public keys in `age_recipients` when the blob was written. -### Content +The two directory levels keep any one directory from holding too many objects. -- **What it contains**: Packed collections of content-defined chunks from files -- **Format**: Zstandard compressed, then Age encrypted -- **Encryption**: Always encrypted with Age using the configured recipients -- **Naming**: Content-addressed. The blob's name is - `hex(SHA256(SHA256(uncompressed blob contents)))` — the double SHA-256 of the - concatenated chunk data, computed before compression and encryption, not a - hash of the stored (compressed, encrypted) bytes. One consequence: only a - holder of the age private key can check a stored blob's integrity, because - matching a blob to its name means decrypting and decompressing it first — - which is what `restore` and `verify --deep` do. Implemented in - `internal/blobgen` (`DoubleSHA256`). This is the canonical description of blob - naming; other documents and comments point here. +### Blob names -### Why Encrypted +A blob's name is `hex(SHA256(SHA256(uncompressed contents)))`. The contents are +the chunk data, joined in order, before compression and encryption. The name is +not a hash of the stored bytes. -Blobs contain the actual file data from backups and must be encrypted for -security. The content-addressing ensures deduplication while the encryption -ensures privacy. +This has two consequences: -## Metadata Directory (`metadata/`) +- Checking a blob against its name needs the private key. The blob must be + decrypted and decompressed first. `snapshot restore` and + `snapshot verify --deep` do this. +- Without the key, only a blob's presence and size can be checked. + `snapshot verify` does this. -Each snapshot has its own subdirectory. The directory is **not** named with the -human-readable snapshot ID; it is named with the remote key — a one-way hash of -that ID. The human ID is never written to the destination store as a directory -name (see [Remote Key Derivation](#remote-key-derivation)). +`blobgen.DoubleSHA256` in `internal/blobgen` computes the name. -### Snapshot ID Format +### Uploads -The human-readable snapshot ID is used in CLI arguments, log lines, and the -local database. It is not written to the destination store. +A backup looks a blob up by name before it uploads it. It does not upload a +blob that is already there. -- **Format**: `__` (or `_` - if no name was specified) -- **Example**: `laptop_home_2024-01-15T14:30:52Z` -- **Components**: - - Short hostname (everything before the first dot is stripped from the FQDN) - - Snapshot name from the configured `snapshots:` map (optional) - - RFC3339 UTC timestamp +When the destination refuses the look-up, the backup uploads the blob. An +object with that name is then replaced by one that holds the same chunk data. -This ID reveals the hostname, the configured snapshot name, and the backup time, -so it is never used as the on-disk directory name — the remote key is used -instead. +## Metadata -### Remote Key Derivation +Each snapshot has one directory under `metadata/`. The directory is named with +the snapshot's remote key, not with its snapshot ID. -The remote key is `hex(SHA256(SHA256("vaultik|" + snapshot-id)))`: a double -SHA-256 over the snapshot ID, with a `vaultik|` domain-separation prefix. The -result is a 64-character hex string. The hash is not reversible, but it uses no -secret: an observer who guesses a candidate hostname and snapshot name can hash -it the same way and confirm whether that snapshot is present. The remote key -keeps names out of a plain listing; it does not hide them from a guess. -Implemented in `internal/snapshot/remotekey.go`. +### Snapshot IDs -Worked example: +A snapshot ID has the form `__`. An example +is `laptop_home_2024-01-15T14:30:52Z`. It has three parts: + +- the hostname, up to its first `.` +- the snapshot name from the config +- the time the backup started, in UTC + +The snapshot ID is used on the command line, in log records and in the local +index. It reveals the hostname, the snapshot name and the backup time. It is +therefore never written to the destination without encryption. + +### Remote keys + +The remote key is `hex(SHA256(SHA256("vaultik|" + snapshot ID)))`. It is 64 hex +characters long. `internal/snapshot/remotekey.go` computes it. + +For example: - Snapshot ID: `server1_home_2025-06-01T12:00:00Z` - Remote key: `17f97bcde958748af076b926af59823943db59e80ce7170b40f124dfa28f64aa` - Directory: `metadata/17f97bcde958748af076b926af59823943db59e80ce7170b40f124dfa28f64aa/` -A plain listing of the destination store therefore shows only these hashes, not -the hostname or snapshot name of any backup — but because the hash uses no -secret, a guessed hostname and snapshot name can be hashed and confirmed against -the listing. The same remote key is stored in the manifest's `snapshot_id` -field. +A listing of the destination therefore shows no hostname and no snapshot name. -### Files in Each Snapshot Directory +The hash cannot be reversed, but it uses no secret. Someone who guesses a +snapshot ID can hash it the same way and see whether that snapshot exists. The +remote key keeps names out of a listing. It does not hide them from a guess. -#### `db.zst.age` - Encrypted Database +### `db.zst.age` -- **What it contains**: Pruned binary SQLite database for this snapshot -- **Format**: Binary SQLite → Zstandard compressed → Age encrypted -- **Encryption**: Encrypted with Age -- **Purpose**: Contains full file metadata, chunk mappings, and all - relationships -- **Why encrypted**: Contains sensitive metadata like file paths, permissions, - and ownership +The snapshot's database. A restore needs it. -#### `manifest.json.zst` - Unencrypted Blob Manifest +- **Contents**: a SQLite database that holds this snapshot only. It lists every + file with its metadata, its chunks, and the blob that holds each chunk. +- **Format**: the database file, compressed with zstd and then encrypted with + age. -- **What it contains**: JSON list of all blob hashes referenced by this snapshot -- **Format**: JSON → Zstandard compressed (NOT encrypted) -- **Encryption**: NOT encrypted -- **Purpose**: Enables pruning operations without requiring decryption keys -- **Structure**: +It is encrypted because it holds file paths, permissions and owners. +[DATAMODEL.md](DATAMODEL.md) describes its tables. + +### `manifest.json.zst` + +The list of the blobs the snapshot uses. + +- **Format**: JSON, compressed with zstd. It is not encrypted. ```json { "snapshot_id": "17f97bcde958748af076b926af59823943db59e80ce7170b40f124dfa28f64aa", - "timestamp": "2025-06-01T12:00:00Z", - "blob_count": 42, - "total_compressed_size": 1048576, + "timestamp": "2025-06-01T12:04:31Z", + "blob_count": 2, + "total_compressed_size": 57344, "blobs": [ { "hash": "cafebabe1234567890abcdef1234567890abcdef1234567890abcdef12345678", @@ -166,189 +175,230 @@ field. } ``` -`snapshot_id` is the remote key (a hash), not the human ID; `timestamp` is -written in the clear. +- `snapshot_id`: the remote key, not the snapshot ID. +- `timestamp`: when the manifest was written, at the end of the backup. +- `blob_count`: the number of blobs. +- `total_compressed_size`: the total size of those blobs, in bytes. +- `blobs`: the hash and the size of each blob. -### Why Manifest is Unencrypted +The manifest holds no file name and no path. -The manifest must be readable without the private key to enable: +The manifest is not encrypted so that these commands work without the private +key: -1. **Pruning operations** - Identifying unreferenced blobs for deletion -2. **Storage analysis** - Understanding space usage without decryption -3. **Verification** - Checking blob existence without decryption -4. **Cross-snapshot deduplication analysis** - Finding shared blobs between - snapshots +- `prune`, which must know which blobs are still in use +- `snapshot verify`, which checks that each blob is present +- `snapshot list` and `remote info`, which report what the destination holds -The manifest contains the remote key, the backup timestamp, the blob count and -total compressed size, and each blob's hash and compressed size. It contains no -file names, paths, or other decrypted metadata. +## What is protected -## Security Considerations +These are encrypted: -### What's Encrypted +- file contents, in the blobs +- file names, paths, permissions, owners and times, in `db.zst.age` +- which chunks make up which file, in `db.zst.age` -- **All file content** (in blobs) -- **All file metadata** (paths, permissions, timestamps, ownership in - db.zst.age) -- **File-to-chunk mappings** (in db.zst.age) +These are not encrypted: -### What's Not Encrypted +- the remote key of each snapshot +- the time each backup finished, in the manifest +- the hash and the compressed size of each blob, in the manifest +- the number of blobs and their total size for each snapshot, in the manifest -- **The remote key** — directory names and the manifest `snapshot_id`, a one-way - hash of the snapshot ID (see [Remote Key Derivation](#remote-key-derivation)) -- **The backup timestamp** (in manifest.json.zst) -- **Blob hashes and their compressed sizes** (in manifest.json.zst) -- **Blob count and total compressed size per snapshot** (in manifest.json.zst) +Someone who can read the destination therefore learns these: -### Privacy Implications +- when each backup finished +- how many blobs each snapshot uses, and how large they are +- which blobs two snapshots share +- whether a snapshot ID they guess exists -From the unencrypted data, an observer of the destination store can determine: +They do not learn these: -- **When each backup was taken** — not from the directory name, which is a - one-way hash, but from the plaintext `timestamp` field in manifest.json.zst, - which is published in the clear -- How many blobs each snapshot references, and the total compressed size -- The compressed size of each blob, and which blobs are shared between snapshots - (deduplication patterns) -- **Whether a guessed hostname and snapshot name are present** — the remote key - is an unkeyed hash, so an observer holding candidate names can hash each one - and match it against the directory listing. The human ID is never published, - so it cannot be read off directly, but it can be confirmed by guessing. +- any hostname or snapshot name, unless they guess it +- file names or paths +- file contents +- permissions or owners +- the directory structure +- which chunks belong to which file -Together these give an observer a timing-and-size profile of every snapshot. -This is an accepted, documented property of the format, not a defect: the -manifest is unencrypted so that pruning can run without the private key, and the -timing channel could not be closed by encrypting it anyway — object creation -times and per-object sizes stay visible at the storage layer on both `s3://` and -`file://` destinations regardless. +The times and sizes could not be hidden by encrypting the manifest. The +destination itself shows when each object was written and how large it is. -An observer cannot determine: +## Accepted risks -- The hostname or snapshot name of any backup by reading it off the store — the - directory name and the manifest `snapshot_id` are unkeyed hashes of the human - ID, so the text is never published (though a guessed name can be confirmed, as - above) -- File names or paths -- File contents -- File permissions or ownership -- Directory structure -- Which chunks belong to which files +These are known properties of the format. They are written down so that you can +weigh them. -### Accepted Risks +### No proof of who wrote the data -These are known, deliberate properties of the format and the tooling, recorded -so an operator can weigh them rather than discover them. +A restore and `snapshot verify --deep` prove two things. The data decrypts with +your private key, and it matches its hashes. They do not prove who wrote it. -1. **No proof of authorship.** Restore and `verify --deep` prove that data - decrypts with the age private key and matches its unkeyed content hashes. - They do not prove who wrote it: anyone who knows a recipient public key and - can replace objects on the destination can substitute a snapshot they built. - The recipient string is not stored at the destination, but a compromised - backed-up host has it. Defences live on the destination side — bucket - versioning or object lock, credentials for the source host that cannot delete - or overwrite existing versions, and pruning from a trusted host. Note that S3 - `PutObject` overwrites an existing key, so PUT permission alone is not - append-only. -2. **Compression reveals sizes.** Blobs and `db.zst.age` are zstd-compressed - then age-encrypted; the manifest is compressed only. age does not pad, so an - object's size is the exact compressed length of its contents. All new chunks - packed into one blob share a single zstd stream (8 MiB window, 4 MiB at - compression levels 1-2), and a blob is closed at `blob_size_limit` and at the - end of each configured path. Because a stored chunk is never packed again, - someone who can write into a backed-up file and watch blob sizes learns - something only when their controlled data and a secret land in the same chunk - of a file that keeps changing. Advice: back up any outsider-writable - directory as its own snapshot. -3. **Chunking uses no secret.** The FastCDC parameters are fixed and public. The - default 10 MB average yields chunks between 2.5 MB and 40 MB, and any file of - 2.5 MB or less is a single chunk. At the default 10 GB `blob_size_limit` a - blob holds hundreds of chunks, so individual chunk lengths are not visible in - the blob's size; lowering the limit toward the chunk size begins to expose - them. -4. **Decrypted data on local disk.** Several commands stage plaintext under - `$TMPDIR`: `snapshot restore` writes decrypted blobs under - `vaultik-blobcache-*/` (no size cap) and the decrypted metadata database at - `vaultik-restore-*/snapshot.db`; `verify --deep` writes that database at - `vaultik-verify-*/snapshot.db`; `snapshot create` keeps a plaintext copy of - the index at `vaultik-snapshot-*/snapshot.db`. These files are created `0600` - and removed on success, but a `kill -9` or a power loss leaves them behind — - delete any leftover `vaultik-*` directory under `$TMPDIR` by hand. `$TMPDIR` - should be trusted to the same degree as the restore target. -5. **Store permissions differ per command.** The backed-up host needs only PUT - to run `snapshot create`: it writes blobs and metadata and neither reads nor - deletes them. Other commands need more — `snapshot verify`, - `snapshot restore`, and `prune` list and read; `prune`, `snapshot purge`, - `snapshot remove`, and `remote nuke` also delete. The recommended cron line - uses `--prune`, which runs `prune` on the backed-up host, so granting that - host `--prune` gives it credentials that can delete its own backups. To keep - the source host to PUT only, prune from a separate trusted host instead. -6. **Changing recipients does not re-encrypt existing data.** Deduplicated - chunks and same-named blobs already on the destination stay encrypted to the - recipients in force when they were written. A new snapshot that reuses them - cannot be restored with a newly added recipient's key alone, because those - reused objects were never encrypted to it. To make everything readable by a - new key, run `vaultik database delete` and take a full backup to a fresh - destination or prefix. -7. **X25519 recipients only.** vaultik rejects age ssh and plugin recipients. - Long-lived ciphertext held by a third party (the destination operator) has no - fallback if X25519 is ever broken: there is no second recipient type and no - post-quantum option. +The hashes use no secret. Someone who knows a recipient's public key and can +replace objects at the destination can put a snapshot of their own there. The +public key is not stored at the destination, but a backed-up machine that is +broken into has it. -## Consistency Guarantees +The defences are on the destination's side: -1. **Blobs are immutable** - Once written, a blob is never modified -2. **Blobs are written before metadata** - A snapshot's metadata is only written - after all its blobs are successfully uploaded -3. **Metadata is written atomically** - Both db.zst.age and manifest.json.zst - are written as complete files -4. **A snapshot is marked complete in the local DB only after its metadata is - uploaded** - `finalizeSnapshotMetadata` runs `ExportSnapshotMetadata` first - and records completion (`MarkSnapshotComplete`) only once the export succeeds - (see the backup data flow in [ARCHITECTURE.md](ARCHITECTURE.md)). A crash - during the export therefore leaves the snapshot incomplete, so the next - backup's `PruneDatabase` drops it and re-backs-up its data, rather than - leaving a completed-looking row in the local index with no matching metadata - on the destination store. (A crash in the brief moment after the export - succeeds but before completion is recorded leaves a fully-restorable snapshot - on the destination that the local index drops as incomplete on the next run; - `snapshot list` then reports it honestly as remote-only, which is the safe - direction: the destination copy stays restorable.) +- object versioning or object lock on the bucket +- credentials for the backed-up machine that cannot delete or overwrite +- running `prune` from a trusted machine -## Pruning Safety +An S3 `PutObject` replaces an existing object. Permission to write alone +therefore does not stop overwrites. -The prune operation is safe because: +### Sizes are visible -1. It keeps every blob listed in any snapshot's manifest and deletes only blobs - that no manifest references. Under `metadata/` it deletes only a directory - that has no `manifest.json.zst`, whose name is a remote key, and that no - local snapshot record claims -2. Manifests are unencrypted and can be read without keys -3. If any manifest cannot be downloaded or decoded, prune deletes nothing and - exits with an error, rather than treating that snapshot's blobs as - unreferenced -4. It deletes an unreferenced blob only once the blob is older than - `prune_grace_period` (default `24h`), and a directory under `metadata/` with - no manifest only once every object in it is. A backup uploads its blobs and - its `db.zst.age` before its manifest, so a prune running meanwhile, on this - host or another sharing the destination, leaves them as long as no backup run - takes longer than the grace period. The grace period does not protect a blob - a backup reuses from an earlier run that stopped before its manifest: the - backup skips the upload of a blob already on the destination, so that blob - keeps the time of its first upload, and a prune before the backup's manifest - is written can delete it; the new snapshot then cannot be restored. - `remote nuke` has no such check +age adds no padding. The size of an object is the exact size of its compressed +contents. -## Restoration Requirements +All the new chunks in one blob share one zstd stream. The stream looks back 8 +MiB, or 4 MiB at compression levels 1 and 2. A blob is closed at +`blob_size_limit` and at the end of each configured path. A chunk that is +already stored is never packed again. -To restore from a backup, you need: +Someone who can write into a backed-up file and watch blob sizes can learn +something in one case. Their data and a secret must land in the same chunk, in +a file that keeps changing. -1. **The Age private key** - To decrypt blobs and database -2. **The snapshot metadata** - Both files from the snapshot's metadata directory -3. **All referenced blobs** - As listed in the manifest +Back up a directory that outsiders can write to as a snapshot of its own. -The restoration process: +### Chunk boundaries use no secret -1. Download and decrypt the database dump to understand file structure -2. Download and decrypt the required blobs -3. Reconstruct files from their chunks -4. Restore file metadata (permissions, timestamps, etc.) +The FastCDC parameters are fixed and public. With the default `chunk_size` of +10 MB, a chunk is between 2.5 MB and 40 MB. A file of 2.5 MB or less is one +chunk. + +At the default `blob_size_limit` of 10 GB, a blob holds hundreds of chunks. The +size of the blob then does not show the size of any one chunk. A limit close to +`chunk_size` begins to show chunk sizes. + +### Decrypted data on the local disk + +These commands write unencrypted data to the temporary directory: + +- `snapshot restore` writes decrypted blobs under `vaultik-blobcache-*/`. There + is no limit on their total size. +- `snapshot restore` writes the decrypted database to + `vaultik-restore-*/snapshot.db`. +- `snapshot verify --deep` writes the decrypted database to + `vaultik-verify-*/snapshot.db`. +- `snapshot create` writes a copy of the local index to + `vaultik-snapshot-*/snapshot.db`. + +These files are readable only by their owner, and are removed when the command +ends. A `kill -9` or a power loss leaves them behind. Delete any leftover +`vaultik-*` entry in the temporary directory yourself. + +Trust the temporary directory as much as the directory you restore into. + +### Commands need different permissions + +`snapshot create` needs only permission to write objects. Other commands need +more: + +- `snapshot list`, `snapshot verify`, `snapshot restore` and `remote info` list + and read. +- `snapshot purge`, `snapshot remove` and `remote nuke` list and delete. +- `prune` lists, reads and deletes. + +`snapshot create --prune` runs a prune on the backed-up machine. That machine +then holds credentials that can delete its own backups. To keep it to +write-only credentials, run `prune` from a separate, trusted machine. + +### New recipients cannot read old data + +Changing `age_recipients` does not re-encrypt what is already stored. A chunk +that is already stored is not written again, so it stays encrypted to the +earlier recipients. + +A new snapshot that uses such chunks cannot be restored with a new recipient's +key alone. To make everything readable by a new key, run +`vaultik database delete` and take a full backup to a new destination. + +### X25519 recipients only + +`vaultik` accepts age X25519 public keys only. It rejects age SSH and plugin +recipients. + +The encrypted data is held by a third party for a long time. There is no second +kind of recipient to fall back on if X25519 is broken, and no post-quantum +option. + +## Order of writes + +A backup writes in a fixed order: + +- the marker file +- the blobs +- `db.zst.age` +- `manifest.json.zst` + +A snapshot's metadata is written only after all its blobs are uploaded. The +manifest is written last. A directory under `metadata/` that has no manifest +therefore belongs to a backup that did not finish. + +The local index marks a snapshot complete only after both metadata files are +uploaded. A backup that stops during that upload leaves the snapshot +unfinished. The next backup removes it from the local index and stores its data +again. + +One short gap remains. A backup can stop after the upload and before the +snapshot is marked complete. The destination then holds a snapshot that can be +restored, and the local index does not list it. `snapshot list` shows it as +remote only. + +An upload that is cut off leaves no partial object under the final name on S3 +or on a local directory. The same holds for an rclone remote that can move +objects. An rclone remote that cannot move objects can keep a partial object. + +## Pruning + +`prune` deletes two kinds of object at the destination: + +- a blob that no manifest lists +- a directory under `metadata/` that has no manifest + +These rules keep it from deleting data that is in use: + +- It keeps every blob that any manifest lists. +- It reads the manifests, which are not encrypted, so it needs no private key. +- It deletes nothing when any manifest cannot be downloaded or decoded. +- It deletes a blob only once the blob is older than `prune_grace_period`. +- It deletes a directory only once every object in it is that old. +- It keeps a directory whose snapshot is in the local index. +- It leaves alone a directory whose name is not a remote key. + +`prune_grace_period` defaults to `24h`. It protects a backup that is still +running. Such a backup has uploaded blobs and perhaps `db.zst.age`, but not yet +its manifest. A prune from this machine or another keeps those objects, as long +as no backup takes longer than the grace period. + +The grace period does not protect one case. A backup does not upload a blob +that is already at the destination, so that blob keeps the time of its first +upload. The blob can be left over from an earlier backup that stopped before +its manifest. A prune that runs before the new manifest is written can then +delete that blob. The new snapshot cannot be restored afterwards. + +To avoid this case, give each machine and each user its own destination. + +`remote nuke` applies none of these rules. + +## What a restore needs + +- the age private key +- the snapshot's `db.zst.age` +- the blobs that hold the files to restore + +A restore does not read the manifest and does not need the local index. + +It works in these steps: + +- It downloads and decrypts `db.zst.age`. +- It checks that the database is the one for the snapshot that was asked for. +- It downloads and decrypts the blobs it needs. It checks each blob against its + name. +- It writes each file from its chunks. +- It sets each file's permissions and times, and its owner when it runs as + root.