README.md: the opening no longer says the private key is needed only to restore; a backup never needs it, a deep verify does. ARCHITECTURE.md: the concurrency note says each open database uses one connection, since the metadata export opens a second one. docs/REPOSTRUCTURE.md: the size leak is about a secret near the outsider's data in the same blob, not the same chunk, because all new chunks of a blob share one zstd stream. Every path used as link text in the four documents is in backticks. Model: fable-5-1
This commit is contained in:
1 parent
1ecfe12e78
commit
34d7d13d3e
4 files changed
+17
-17
No files matched your search
+6
-7
@@ -9,8 +9,8 @@ things:
|
|||||||
|
|
||||||
Two other documents cover the stored data:
|
Two other documents cover the stored data:
|
||||||
|
|
||||||
- [docs/DATAMODEL.md](docs/DATAMODEL.md) describes the local index.
|
- [`docs/DATAMODEL.md`](docs/DATAMODEL.md) describes the local index.
|
||||||
- [docs/REPOSTRUCTURE.md](docs/REPOSTRUCTURE.md) describes the layout of the
|
- [`docs/REPOSTRUCTURE.md`](docs/REPOSTRUCTURE.md) describes the layout of the
|
||||||
destination.
|
destination.
|
||||||
|
|
||||||
## Terms
|
## Terms
|
||||||
@@ -60,7 +60,7 @@ source files
|
|||||||
and the reader that reverses it.
|
and the reader that reverses it.
|
||||||
- `internal/database`: the local index. It holds the schema and the repository
|
- `internal/database`: the local index. It holds the schema and the repository
|
||||||
types that read and write the tables.
|
types that read and write the tables.
|
||||||
[docs/DATAMODEL.md](docs/DATAMODEL.md#code) says which type serves which
|
[`docs/DATAMODEL.md`](docs/DATAMODEL.md#code) says which type serves which
|
||||||
table.
|
table.
|
||||||
- `internal/storage`: the `Storer` interface and its three implementations, for
|
- `internal/storage`: the `Storer` interface and its three implementations, for
|
||||||
S3, a local directory and rclone.
|
S3, a local directory and rclone.
|
||||||
@@ -225,7 +225,7 @@ file is never held in memory.
|
|||||||
|
|
||||||
The blob's name is `hex(SHA256(SHA256(uncompressed contents)))`. It is computed
|
The blob's name is `hex(SHA256(SHA256(uncompressed contents)))`. It is computed
|
||||||
from the chunk data before compression and encryption.
|
from the chunk data before compression and encryption.
|
||||||
[docs/REPOSTRUCTURE.md](docs/REPOSTRUCTURE.md#blobs) says what follows from
|
[`docs/REPOSTRUCTURE.md`](docs/REPOSTRUCTURE.md#blobs) says what follows from
|
||||||
that.
|
that.
|
||||||
|
|
||||||
When a blob is closed, `Scanner.handleBlobReady` stores it:
|
When a blob is closed, `Scanner.handleBlobReady` stores it:
|
||||||
@@ -353,7 +353,7 @@ of object, once they are older than `prune_grace_period`:
|
|||||||
in the local index
|
in the local index
|
||||||
|
|
||||||
It deletes nothing when any manifest cannot be read.
|
It deletes nothing when any manifest cannot be read.
|
||||||
[docs/REPOSTRUCTURE.md](docs/REPOSTRUCTURE.md#pruning) explains why this is
|
[`docs/REPOSTRUCTURE.md`](docs/REPOSTRUCTURE.md#pruning) explains why this is
|
||||||
safe, and where it is not.
|
safe, and where it is not.
|
||||||
|
|
||||||
## The destination
|
## The destination
|
||||||
@@ -390,8 +390,7 @@ Every command that changes data calls `EnsureStorageBinding` first.
|
|||||||
|
|
||||||
## Concurrency
|
## Concurrency
|
||||||
|
|
||||||
- The local index is opened with one connection, so its queries run one at a
|
- Each open database uses one connection, so its queries run one at a time.
|
||||||
time.
|
|
||||||
- The index runs in SQLite's WAL mode. A second process can therefore read it
|
- The index runs in SQLite's WAL mode. A second process can therefore read it
|
||||||
while a backup writes to it.
|
while a backup writes to it.
|
||||||
- `Packer` guards its state with a mutex.
|
- `Packer` guards its state with a mutex.
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ with a public key and uploads them to an S3-compatible object store, a local
|
|||||||
directory or an rclone remote.
|
directory or an rclone remote.
|
||||||
|
|
||||||
The machine being backed up holds only the public key. It cannot decrypt its own
|
The machine being backed up holds only the public key. It cannot decrypt its own
|
||||||
backups. You keep the private key somewhere else and need it only to restore.
|
backups. You keep the private key somewhere else. A backup never needs it.
|
||||||
|
|
||||||
## Rationale
|
## Rationale
|
||||||
|
|
||||||
@@ -1167,7 +1167,7 @@ destination still learns these:
|
|||||||
The last point holds because a remote key is a plain hash of the snapshot ID,
|
The last point holds because a remote key is a plain hash of the snapshot ID,
|
||||||
made without a secret.
|
made without a secret.
|
||||||
|
|
||||||
[docs/REPOSTRUCTURE.md](docs/REPOSTRUCTURE.md) describes the layout of the
|
[`docs/REPOSTRUCTURE.md`](docs/REPOSTRUCTURE.md) describes the layout of the
|
||||||
destination, what it protects and the risks it accepts.
|
destination, what it protects and the risks it accepts.
|
||||||
|
|
||||||
## Limitations
|
## Limitations
|
||||||
@@ -1207,9 +1207,9 @@ is a trimmed, encrypted copy of the local index.
|
|||||||
|
|
||||||
These documents give the detail:
|
These documents give the detail:
|
||||||
|
|
||||||
- [ARCHITECTURE.md](ARCHITECTURE.md) describes how the code is organised.
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md) describes how the code is organised.
|
||||||
- [docs/DATAMODEL.md](docs/DATAMODEL.md) describes the local index.
|
- [`docs/DATAMODEL.md`](docs/DATAMODEL.md) describes the local index.
|
||||||
- [docs/REPOSTRUCTURE.md](docs/REPOSTRUCTURE.md) describes the layout of the
|
- [`docs/REPOSTRUCTURE.md`](docs/REPOSTRUCTURE.md) describes the layout of the
|
||||||
destination.
|
destination.
|
||||||
|
|
||||||
## Development
|
## Development
|
||||||
|
|||||||
+1
-1
@@ -135,7 +135,7 @@ One row for each blob.
|
|||||||
- `uploaded_ts`: when the upload finished. It is `NULL` before that.
|
- `uploaded_ts`: when the upload finished. It is `NULL` before that.
|
||||||
|
|
||||||
`blob_hash` is `hex(SHA256(SHA256(uncompressed contents)))`. See
|
`blob_hash` is `hex(SHA256(SHA256(uncompressed contents)))`. See
|
||||||
[REPOSTRUCTURE.md](REPOSTRUCTURE.md#blobs).
|
[`REPOSTRUCTURE.md`](REPOSTRUCTURE.md#blobs).
|
||||||
|
|
||||||
Once a blob is uploaded, `compressed_size` is the size of the object at the
|
Once a blob is uploaded, `compressed_size` is the size of the object at the
|
||||||
destination. A blob that was already there can have a different size from the
|
destination. A blob that was already there can have a different size from the
|
||||||
|
|||||||
@@ -149,7 +149,7 @@ The snapshot's database. A restore needs it.
|
|||||||
age.
|
age.
|
||||||
|
|
||||||
It is encrypted because it holds file paths, permissions and owners.
|
It is encrypted because it holds file paths, permissions and owners.
|
||||||
[DATAMODEL.md](DATAMODEL.md) describes its tables.
|
[`DATAMODEL.md`](DATAMODEL.md) describes its tables.
|
||||||
|
|
||||||
### `manifest.json.zst`
|
### `manifest.json.zst`
|
||||||
|
|
||||||
@@ -261,15 +261,16 @@ MiB, or 4 MiB at compression levels 1 and 2. A blob is closed at
|
|||||||
`blob_size_limit` and at the end of each configured path. A chunk that is
|
`blob_size_limit` and at the end of each configured path. A chunk that is
|
||||||
already stored is never packed again.
|
already stored is never packed again.
|
||||||
|
|
||||||
Data that an outsider writes compresses smaller when it matches a secret in the
|
Data that an outsider writes compresses smaller when it matches a secret near it
|
||||||
same chunk, so the blob gets smaller. The blob's size therefore shows the
|
in the same blob, so the blob gets smaller. The blob's size therefore shows the
|
||||||
outsider whether their data matched the secret.
|
outsider whether their data matched the secret.
|
||||||
|
|
||||||
This works only when all of these hold:
|
This works only when all of these hold:
|
||||||
|
|
||||||
- The outsider can write into a backed-up file.
|
- The outsider can write into a backed-up file.
|
||||||
- The outsider can see the blob sizes at the destination.
|
- The outsider can see the blob sizes at the destination.
|
||||||
- The outsider's data and the secret land in the same chunk.
|
- The outsider's data and the secret are in the same blob, no further apart than
|
||||||
|
the stream looks back.
|
||||||
- The file keeps changing.
|
- The file keeps changing.
|
||||||
|
|
||||||
Back up a directory that outsiders can write to as a snapshot of its own.
|
Back up a directory that outsiders can write to as a snapshot of its own.
|
||||||
|
|||||||
Reference in new issue
Block a user