sneak/vaultik
1
0 Fork 0
Code Issues 13 Pull Requests 5 Actions 1 Packages Projects Releases Wiki Activity

Rewrite docs/REPOSTRUCTURE.md sentence by sentence

Browse Source 
Rewritten from scratch against the code on next: the layout of the
destination, what it protects, the accepted risks, the order of writes
and the rules that keep prune from deleting data in use.

Corrected: a restore needs only db.zst.age and the blobs it reads, not
the manifest. The snapshot ID form without a name is gone, since no
command creates one.

Model: fable-5-1
This commit is contained in:
sneak authored and clawbot committed 2026-11-11 07:01:54 +00:00
1 parent 1d743e2255
commit c8ad289e46
1 file changed
+311 -261
Download Patch File Download Diff File Expand all files Collapse all files
docs/REPOSTRUCTURE.md
+311 -261
View File
@@ -1,158 +1,167 @@
# Vaultik S3 Repository Structure
# vaultik destination layout
This document describes the structure and organization of data stored in the S3
bucket by Vaultik.
This document describes what `vaultik` stores at the destination. It covers the
layout, what the layout protects, and the risks it accepts.
## Overview
The destination is the place the backups are stored. It is an S3 bucket and
prefix, a local directory or an rclone remote. The layout is the same on all
three.
Vaultik stores all backup data in an S3-compatible object store. The repository
consists of two main components, plus a marker file:
## Terms
1. **Blobs** - The actual backup data (content-addressed, encrypted)
2. **Metadata** - Snapshot information and manifests (partially encrypted)
- **Snapshot**: one backup of the paths listed under one name in the config.
- **Snapshot ID**: the name of a snapshot, in the form
`<hostname>_<snapshot-name>_<timestamp>`.
- **Chunk**: a piece of a file's content, named by the SHA-256 hash of that
content.
- **Blob**: one object under `blobs/`. It holds many chunks.
- **Local index**: the SQLite database on the backed-up machine.
- **Recipient**: an age public key that backups are encrypted to.
## Directory Structure
## Layout
```
<bucket>/<prefix>/
<destination>/
├── vaultik-destination.txt
├── blobs/
│ └── <hash[0:2]>/
│ └── <hash[2:4]>/
│ └── <full-hash>
│ └── <hash>
└── metadata/
└── <remote-key>/
├── db.zst.age
└── manifest.json.zst
```
The metadata subdirectory is named with the **remote key**, a one-way hash of
the snapshot ID, not with the human-readable snapshot ID itself. See
[Remote Key Derivation](#remote-key-derivation).
The destination holds three things:
## Marker File (`vaultik-destination.txt`)
- `vaultik-destination.txt`, the marker file
- `blobs/`, which holds the file contents
- `metadata/`, which holds one directory for each snapshot
## The marker file
Every backup writes `vaultik-destination.txt` before it uploads anything else.
The file holds one fixed line of text. It names no host, path or time.
The marker tells an emptied or missing destination apart from a new one. An
unplugged volume leaves an empty directory at its mount point. Without the
marker, that directory would look like a destination whose snapshots are all
gone.
The check applies while the local index holds snapshots. When the destination
then answers that the marker does not exist, commands treat the destination as
not present:
- `snapshot create` refuses to write to it.
- `prune`, `snapshot purge` and `remote nuke` fail.
- `snapshot list` shows the local index alone.
- `snapshot remove` deletes nothing at the destination.
The check is skipped when the look-up fails for any other reason. S3 refuses
the look-up to credentials that can only write. A machine with such credentials
therefore does not notice an emptied destination. Its next backup would leave
out the data its local index records as stored. After a destination is emptied,
run `vaultik database delete` on such a machine before it backs up again.
Every backup writes this file at the root before it uploads anything. It holds
one fixed line of plain text and names no host, path or time. While the local
index holds snapshots stored there, a destination that answers a look-up of the
marker with not found is treated as not present: `snapshot create` refuses to
write to it, and `prune`, `snapshot purge`, `snapshot remove`, `snapshot list`
and `remote nuke` behave as for a destination that cannot be listed. An
unplugged volume leaves an empty directory at its mount point, and without the
marker that directory would read as a destination whose snapshots are all gone.
`remote nuke` deletes the marker last.
When the look-up of the marker fails for any reason other than not found, the
check is skipped and the command goes on. S3 refuses it on every run to
credentials that allow only PUT, so a host with such credentials does not notice
a destination emptied since its last backup, and its next backup would leave out
the data its local index records as already stored. After the destination is
emptied, run `vaultik database delete` on such a host before it backs up again.
## Blobs
## Blobs Directory (`blobs/`)
A blob holds the content of many chunks, one after another.
### Structure
- **Path format**: `blobs/<first-2-chars>/<next-2-chars>/<full-hash>`
- **Path**: `blobs/<first 2 characters>/<next 2 characters>/<full hash>`.
- **Example**:
`blobs/ca/fe/cafebabe1234567890abcdef1234567890abcdef1234567890abcdef12345678`
- **Sharding**: The two-level directory structure (using the first 4 characters
of the hash) prevents any single directory from containing too many objects
`blobs/ca/fe/cafebabe1234567890abcdef1234567890abcdef1234567890abcdef12345678`.
- **Format**: the chunk data, compressed with zstd and then encrypted with age.
- **Recipients**: the public keys in `age_recipients` when the blob was written.
### Content
The two directory levels keep any one directory from holding too many objects.
- **What it contains**: Packed collections of content-defined chunks from files
- **Format**: Zstandard compressed, then Age encrypted
- **Encryption**: Always encrypted with Age using the configured recipients
- **Naming**: Content-addressed. The blob's name is
`hex(SHA256(SHA256(uncompressed blob contents)))` — the double SHA-256 of the
concatenated chunk data, computed before compression and encryption, not a
hash of the stored (compressed, encrypted) bytes. One consequence: only a
holder of the age private key can check a stored blob's integrity, because
matching a blob to its name means decrypting and decompressing it first —
which is what `restore` and `verify --deep` do. Implemented in
`internal/blobgen` (`DoubleSHA256`). This is the canonical description of blob
naming; other documents and comments point here.
### Blob names
### Why Encrypted
A blob's name is `hex(SHA256(SHA256(uncompressed contents)))`. The contents are
the chunk data, joined in order, before compression and encryption. The name is
not a hash of the stored bytes.
Blobs contain the actual file data from backups and must be encrypted for
security. The content-addressing ensures deduplication while the encryption
ensures privacy.
This has two consequences:
## Metadata Directory (`metadata/`)
- Checking a blob against its name needs the private key. The blob must be
decrypted and decompressed first. `snapshot restore` and
`snapshot verify --deep` do this.
- Without the key, only a blob's presence and size can be checked.
`snapshot verify` does this.
Each snapshot has its own subdirectory. The directory is **not** named with the
human-readable snapshot ID; it is named with the remote key — a one-way hash of
that ID. The human ID is never written to the destination store as a directory
name (see [Remote Key Derivation](#remote-key-derivation)).
`blobgen.DoubleSHA256` in `internal/blobgen` computes the name.
### Snapshot ID Format
### Uploads
The human-readable snapshot ID is used in CLI arguments, log lines, and the
local database. It is not written to the destination store.
A backup looks a blob up by name before it uploads it. It does not upload a
blob that is already there.
- **Format**: `<hostname>_<snapshot-name>_<RFC3339>` (or `<hostname>_<RFC3339>`
if no name was specified)
- **Example**: `laptop_home_2024-01-15T14:30:52Z`
- **Components**:
- Short hostname (everything before the first dot is stripped from the FQDN)
- Snapshot name from the configured `snapshots:` map (optional)
- RFC3339 UTC timestamp
When the destination refuses the look-up, the backup uploads the blob. An
object with that name is then replaced by one that holds the same chunk data.
This ID reveals the hostname, the configured snapshot name, and the backup time,
so it is never used as the on-disk directory name — the remote key is used
instead.
## Metadata
### Remote Key Derivation
Each snapshot has one directory under `metadata/`. The directory is named with
the snapshot's remote key, not with its snapshot ID.
The remote key is `hex(SHA256(SHA256("vaultik|" + snapshot-id)))`: a double
SHA-256 over the snapshot ID, with a `vaultik|` domain-separation prefix. The
result is a 64-character hex string. The hash is not reversible, but it uses no
secret: an observer who guesses a candidate hostname and snapshot name can hash
it the same way and confirm whether that snapshot is present. The remote key
keeps names out of a plain listing; it does not hide them from a guess.
Implemented in `internal/snapshot/remotekey.go`.
### Snapshot IDs
Worked example:
A snapshot ID has the form `<hostname>_<snapshot-name>_<timestamp>`. An example
is `laptop_home_2024-01-15T14:30:52Z`. It has three parts:
- the hostname, up to its first `.`
- the snapshot name from the config
- the time the backup started, in UTC
The snapshot ID is used on the command line, in log records and in the local
index. It reveals the hostname, the snapshot name and the backup time. It is
therefore never written to the destination without encryption.
### Remote keys
The remote key is `hex(SHA256(SHA256("vaultik|" + snapshot ID)))`. It is 64 hex
characters long. `internal/snapshot/remotekey.go` computes it.
For example:
- Snapshot ID: `server1_home_2025-06-01T12:00:00Z`
- Remote key: `17f97bcde958748af076b926af59823943db59e80ce7170b40f124dfa28f64aa`
- Directory:
`metadata/17f97bcde958748af076b926af59823943db59e80ce7170b40f124dfa28f64aa/`
A plain listing of the destination store therefore shows only these hashes, not
the hostname or snapshot name of any backup — but because the hash uses no
secret, a guessed hostname and snapshot name can be hashed and confirmed against
the listing. The same remote key is stored in the manifest's `snapshot_id`
field.
A listing of the destination therefore shows no hostname and no snapshot name.
### Files in Each Snapshot Directory
The hash cannot be reversed, but it uses no secret. Someone who guesses a
snapshot ID can hash it the same way and see whether that snapshot exists. The
remote key keeps names out of a listing. It does not hide them from a guess.
#### `db.zst.age` - Encrypted Database
### `db.zst.age`
- **What it contains**: Pruned binary SQLite database for this snapshot
- **Format**: Binary SQLite → Zstandard compressed → Age encrypted
- **Encryption**: Encrypted with Age
- **Purpose**: Contains full file metadata, chunk mappings, and all
relationships
- **Why encrypted**: Contains sensitive metadata like file paths, permissions,
and ownership
The snapshot's database. A restore needs it.
#### `manifest.json.zst` - Unencrypted Blob Manifest
- **Contents**: a SQLite database that holds this snapshot only. It lists every
file with its metadata, its chunks, and the blob that holds each chunk.
- **Format**: the database file, compressed with zstd and then encrypted with
age.
- **What it contains**: JSON list of all blob hashes referenced by this snapshot
- **Format**: JSON → Zstandard compressed (NOT encrypted)
- **Encryption**: NOT encrypted
- **Purpose**: Enables pruning operations without requiring decryption keys
- **Structure**:
It is encrypted because it holds file paths, permissions and owners.
[DATAMODEL.md](DATAMODEL.md) describes its tables.
### `manifest.json.zst`
The list of the blobs the snapshot uses.
- **Format**: JSON, compressed with zstd. It is not encrypted.
```json
{
"snapshot_id": "17f97bcde958748af076b926af59823943db59e80ce7170b40f124dfa28f64aa",
"timestamp": "2025-06-01T12:00:00Z",
"blob_count": 42,
"total_compressed_size": 1048576,
"timestamp": "2025-06-01T12:04:31Z",
"blob_count": 2,
"total_compressed_size": 57344,
"blobs": [
{
"hash": "cafebabe1234567890abcdef1234567890abcdef1234567890abcdef12345678",
@@ -166,189 +175,230 @@ field.
}
```
`snapshot_id` is the remote key (a hash), not the human ID; `timestamp` is
written in the clear.
- `snapshot_id`: the remote key, not the snapshot ID.
- `timestamp`: when the manifest was written, at the end of the backup.
- `blob_count`: the number of blobs.
- `total_compressed_size`: the total size of those blobs, in bytes.
- `blobs`: the hash and the size of each blob.
### Why Manifest is Unencrypted
The manifest holds no file name and no path.
The manifest must be readable without the private key to enable:
The manifest is not encrypted so that these commands work without the private
key:
1. **Pruning operations** - Identifying unreferenced blobs for deletion
2. **Storage analysis** - Understanding space usage without decryption
3. **Verification** - Checking blob existence without decryption
4. **Cross-snapshot deduplication analysis** - Finding shared blobs between
snapshots
- `prune`, which must know which blobs are still in use
- `snapshot verify`, which checks that each blob is present
- `snapshot list` and `remote info`, which report what the destination holds
The manifest contains the remote key, the backup timestamp, the blob count and
total compressed size, and each blob's hash and compressed size. It contains no
file names, paths, or other decrypted metadata.
## What is protected
## Security Considerations
These are encrypted:
### What's Encrypted
- file contents, in the blobs
- file names, paths, permissions, owners and times, in `db.zst.age`
- which chunks make up which file, in `db.zst.age`
- **All file content** (in blobs)
- **All file metadata** (paths, permissions, timestamps, ownership in
db.zst.age)
- **File-to-chunk mappings** (in db.zst.age)
These are not encrypted:
### What's Not Encrypted
- the remote key of each snapshot
- the time each backup finished, in the manifest
- the hash and the compressed size of each blob, in the manifest
- the number of blobs and their total size for each snapshot, in the manifest
- **The remote key** — directory names and the manifest `snapshot_id`, a one-way
hash of the snapshot ID (see [Remote Key Derivation](#remote-key-derivation))
- **The backup timestamp** (in manifest.json.zst)
- **Blob hashes and their compressed sizes** (in manifest.json.zst)
- **Blob count and total compressed size per snapshot** (in manifest.json.zst)
Someone who can read the destination therefore learns these:
### Privacy Implications
- when each backup finished
- how many blobs each snapshot uses, and how large they are
- which blobs two snapshots share
- whether a snapshot ID they guess exists
From the unencrypted data, an observer of the destination store can determine:
They do not learn these:
- **When each backup was taken** — not from the directory name, which is a
one-way hash, but from the plaintext `timestamp` field in manifest.json.zst,
which is published in the clear
- How many blobs each snapshot references, and the total compressed size
- The compressed size of each blob, and which blobs are shared between snapshots
(deduplication patterns)
- **Whether a guessed hostname and snapshot name are present** — the remote key
is an unkeyed hash, so an observer holding candidate names can hash each one
and match it against the directory listing. The human ID is never published,
so it cannot be read off directly, but it can be confirmed by guessing.
- any hostname or snapshot name, unless they guess it
- file names or paths
- file contents
- permissions or owners
- the directory structure
- which chunks belong to which file
Together these give an observer a timing-and-size profile of every snapshot.
This is an accepted, documented property of the format, not a defect: the
manifest is unencrypted so that pruning can run without the private key, and the
timing channel could not be closed by encrypting it anyway — object creation
times and per-object sizes stay visible at the storage layer on both `s3://` and
`file://` destinations regardless.
The times and sizes could not be hidden by encrypting the manifest. The
destination itself shows when each object was written and how large it is.
An observer cannot determine:
## Accepted risks
- The hostname or snapshot name of any backup by reading it off the store — the
directory name and the manifest `snapshot_id` are unkeyed hashes of the human
ID, so the text is never published (though a guessed name can be confirmed, as
above)
- File names or paths
- File contents
- File permissions or ownership
- Directory structure
- Which chunks belong to which files
These are known properties of the format. They are written down so that you can
weigh them.
### Accepted Risks
### No proof of who wrote the data
These are known, deliberate properties of the format and the tooling, recorded
so an operator can weigh them rather than discover them.
A restore and `snapshot verify --deep` prove two things. The data decrypts with
your private key, and it matches its hashes. They do not prove who wrote it.
1. **No proof of authorship.** Restore and `verify --deep` prove that data
decrypts with the age private key and matches its unkeyed content hashes.
They do not prove who wrote it: anyone who knows a recipient public key and
can replace objects on the destination can substitute a snapshot they built.
The recipient string is not stored at the destination, but a compromised
backed-up host has it. Defences live on the destination side — bucket
versioning or object lock, credentials for the source host that cannot delete
or overwrite existing versions, and pruning from a trusted host. Note that S3
`PutObject` overwrites an existing key, so PUT permission alone is not
append-only.
2. **Compression reveals sizes.** Blobs and `db.zst.age` are zstd-compressed
then age-encrypted; the manifest is compressed only. age does not pad, so an
object's size is the exact compressed length of its contents. All new chunks
packed into one blob share a single zstd stream (8 MiB window, 4 MiB at
compression levels 1-2), and a blob is closed at `blob_size_limit` and at the
end of each configured path. Because a stored chunk is never packed again,
someone who can write into a backed-up file and watch blob sizes learns
something only when their controlled data and a secret land in the same chunk
of a file that keeps changing. Advice: back up any outsider-writable
directory as its own snapshot.
3. **Chunking uses no secret.** The FastCDC parameters are fixed and public. The
default 10 MB average yields chunks between 2.5 MB and 40 MB, and any file of
2.5 MB or less is a single chunk. At the default 10 GB `blob_size_limit` a
blob holds hundreds of chunks, so individual chunk lengths are not visible in
the blob's size; lowering the limit toward the chunk size begins to expose
them.
4. **Decrypted data on local disk.** Several commands stage plaintext under
`$TMPDIR`: `snapshot restore` writes decrypted blobs under
`vaultik-blobcache-*/` (no size cap) and the decrypted metadata database at
`vaultik-restore-*/snapshot.db`; `verify --deep` writes that database at
`vaultik-verify-*/snapshot.db`; `snapshot create` keeps a plaintext copy of
the index at `vaultik-snapshot-*/snapshot.db`. These files are created `0600`
and removed on success, but a `kill -9` or a power loss leaves them behind —
delete any leftover `vaultik-*` directory under `$TMPDIR` by hand. `$TMPDIR`
should be trusted to the same degree as the restore target.
5. **Store permissions differ per command.** The backed-up host needs only PUT
to run `snapshot create`: it writes blobs and metadata and neither reads nor
deletes them. Other commands need more — `snapshot verify`,
`snapshot restore`, and `prune` list and read; `prune`, `snapshot purge`,
`snapshot remove`, and `remote nuke` also delete. The recommended cron line
uses `--prune`, which runs `prune` on the backed-up host, so granting that
host `--prune` gives it credentials that can delete its own backups. To keep
the source host to PUT only, prune from a separate trusted host instead.
6. **Changing recipients does not re-encrypt existing data.** Deduplicated
chunks and same-named blobs already on the destination stay encrypted to the
recipients in force when they were written. A new snapshot that reuses them
cannot be restored with a newly added recipient's key alone, because those
reused objects were never encrypted to it. To make everything readable by a
new key, run `vaultik database delete` and take a full backup to a fresh
destination or prefix.
7. **X25519 recipients only.** vaultik rejects age ssh and plugin recipients.
Long-lived ciphertext held by a third party (the destination operator) has no
fallback if X25519 is ever broken: there is no second recipient type and no
post-quantum option.
The hashes use no secret. Someone who knows a recipient's public key and can
replace objects at the destination can put a snapshot of their own there. The
public key is not stored at the destination, but a backed-up machine that is
broken into has it.
## Consistency Guarantees
The defences are on the destination's side:
1. **Blobs are immutable** - Once written, a blob is never modified
2. **Blobs are written before metadata** - A snapshot's metadata is only written
after all its blobs are successfully uploaded
3. **Metadata is written atomically** - Both db.zst.age and manifest.json.zst
are written as complete files
4. **A snapshot is marked complete in the local DB only after its metadata is
uploaded** - `finalizeSnapshotMetadata` runs `ExportSnapshotMetadata` first
and records completion (`MarkSnapshotComplete`) only once the export succeeds
(see the backup data flow in [ARCHITECTURE.md](ARCHITECTURE.md)). A crash
during the export therefore leaves the snapshot incomplete, so the next
backup's `PruneDatabase` drops it and re-backs-up its data, rather than
leaving a completed-looking row in the local index with no matching metadata
on the destination store. (A crash in the brief moment after the export
succeeds but before completion is recorded leaves a fully-restorable snapshot
on the destination that the local index drops as incomplete on the next run;
`snapshot list` then reports it honestly as remote-only, which is the safe
direction: the destination copy stays restorable.)
- object versioning or object lock on the bucket
- credentials for the backed-up machine that cannot delete or overwrite
- running `prune` from a trusted machine
## Pruning Safety
An S3 `PutObject` replaces an existing object. Permission to write alone
therefore does not stop overwrites.
The prune operation is safe because:
### Sizes are visible
1. It keeps every blob listed in any snapshot's manifest and deletes only blobs
that no manifest references. Under `metadata/` it deletes only a directory
that has no `manifest.json.zst`, whose name is a remote key, and that no
local snapshot record claims
2. Manifests are unencrypted and can be read without keys
3. If any manifest cannot be downloaded or decoded, prune deletes nothing and
exits with an error, rather than treating that snapshot's blobs as
unreferenced
4. It deletes an unreferenced blob only once the blob is older than
`prune_grace_period` (default `24h`), and a directory under `metadata/` with
no manifest only once every object in it is. A backup uploads its blobs and
its `db.zst.age` before its manifest, so a prune running meanwhile, on this
host or another sharing the destination, leaves them as long as no backup run
takes longer than the grace period. The grace period does not protect a blob
a backup reuses from an earlier run that stopped before its manifest: the
backup skips the upload of a blob already on the destination, so that blob
keeps the time of its first upload, and a prune before the backup's manifest
is written can delete it; the new snapshot then cannot be restored.
`remote nuke` has no such check
age adds no padding. The size of an object is the exact size of its compressed
contents.
## Restoration Requirements
All the new chunks in one blob share one zstd stream. The stream looks back 8
MiB, or 4 MiB at compression levels 1 and 2. A blob is closed at
`blob_size_limit` and at the end of each configured path. A chunk that is
already stored is never packed again.
To restore from a backup, you need:
Someone who can write into a backed-up file and watch blob sizes can learn
something in one case. Their data and a secret must land in the same chunk, in
a file that keeps changing.
1. **The Age private key** - To decrypt blobs and database
2. **The snapshot metadata** - Both files from the snapshot's metadata directory
3. **All referenced blobs** - As listed in the manifest
Back up a directory that outsiders can write to as a snapshot of its own.
The restoration process:
### Chunk boundaries use no secret
1. Download and decrypt the database dump to understand file structure
2. Download and decrypt the required blobs
3. Reconstruct files from their chunks
4. Restore file metadata (permissions, timestamps, etc.)
The FastCDC parameters are fixed and public. With the default `chunk_size` of
10 MB, a chunk is between 2.5 MB and 40 MB. A file of 2.5 MB or less is one
chunk.
At the default `blob_size_limit` of 10 GB, a blob holds hundreds of chunks. The
size of the blob then does not show the size of any one chunk. A limit close to
`chunk_size` begins to show chunk sizes.
### Decrypted data on the local disk
These commands write unencrypted data to the temporary directory:
- `snapshot restore` writes decrypted blobs under `vaultik-blobcache-*/`. There
is no limit on their total size.
- `snapshot restore` writes the decrypted database to
`vaultik-restore-*/snapshot.db`.
- `snapshot verify --deep` writes the decrypted database to
`vaultik-verify-*/snapshot.db`.
- `snapshot create` writes a copy of the local index to
`vaultik-snapshot-*/snapshot.db`.
These files are readable only by their owner, and are removed when the command
ends. A `kill -9` or a power loss leaves them behind. Delete any leftover
`vaultik-*` entry in the temporary directory yourself.
Trust the temporary directory as much as the directory you restore into.
### Commands need different permissions
`snapshot create` needs only permission to write objects. Other commands need
more:
- `snapshot list`, `snapshot verify`, `snapshot restore` and `remote info` list
and read.
- `snapshot purge`, `snapshot remove` and `remote nuke` list and delete.
- `prune` lists, reads and deletes.
`snapshot create --prune` runs a prune on the backed-up machine. That machine
then holds credentials that can delete its own backups. To keep it to
write-only credentials, run `prune` from a separate, trusted machine.
### New recipients cannot read old data
Changing `age_recipients` does not re-encrypt what is already stored. A chunk
that is already stored is not written again, so it stays encrypted to the
earlier recipients.
A new snapshot that uses such chunks cannot be restored with a new recipient's
key alone. To make everything readable by a new key, run
`vaultik database delete` and take a full backup to a new destination.
### X25519 recipients only
`vaultik` accepts age X25519 public keys only. It rejects age SSH and plugin
recipients.
The encrypted data is held by a third party for a long time. There is no second
kind of recipient to fall back on if X25519 is broken, and no post-quantum
option.
## Order of writes
A backup writes in a fixed order:
- the marker file
- the blobs
- `db.zst.age`
- `manifest.json.zst`
A snapshot's metadata is written only after all its blobs are uploaded. The
manifest is written last. A directory under `metadata/` that has no manifest
therefore belongs to a backup that did not finish.
The local index marks a snapshot complete only after both metadata files are
uploaded. A backup that stops during that upload leaves the snapshot
unfinished. The next backup removes it from the local index and stores its data
again.
One short gap remains. A backup can stop after the upload and before the
snapshot is marked complete. The destination then holds a snapshot that can be
restored, and the local index does not list it. `snapshot list` shows it as
remote only.
An upload that is cut off leaves no partial object under the final name on S3
or on a local directory. The same holds for an rclone remote that can move
objects. An rclone remote that cannot move objects can keep a partial object.
## Pruning
`prune` deletes two kinds of object at the destination:
- a blob that no manifest lists
- a directory under `metadata/` that has no manifest
These rules keep it from deleting data that is in use:
- It keeps every blob that any manifest lists.
- It reads the manifests, which are not encrypted, so it needs no private key.
- It deletes nothing when any manifest cannot be downloaded or decoded.
- It deletes a blob only once the blob is older than `prune_grace_period`.
- It deletes a directory only once every object in it is that old.
- It keeps a directory whose snapshot is in the local index.
- It leaves alone a directory whose name is not a remote key.
`prune_grace_period` defaults to `24h`. It protects a backup that is still
running. Such a backup has uploaded blobs and perhaps `db.zst.age`, but not yet
its manifest. A prune from this machine or another keeps those objects, as long
as no backup takes longer than the grace period.
The grace period does not protect one case. A backup does not upload a blob
that is already at the destination, so that blob keeps the time of its first
upload. The blob can be left over from an earlier backup that stopped before
its manifest. A prune that runs before the new manifest is written can then
delete that blob. The new snapshot cannot be restored afterwards.
To avoid this case, give each machine and each user its own destination.
`remote nuke` applies none of these rules.
## What a restore needs
- the age private key
- the snapshot's `db.zst.age`
- the blobs that hold the files to restore
A restore does not read the manifest and does not need the local index.
It works in these steps:
- It downloads and decrypts `db.zst.age`.
- It checks that the database is the one for the snapshot that was asked for.
- It downloads and decrypts the blobs it needs. It checks each blob against its
name.
- It writes each file from its chunks.
- It sets each file's permissions and times, and its owner when it runs as
root.