Fix the review findings in the three design documents
check / check (push) Successful in 13m23s

ARCHITECTURE.md: the chunk size limits, the commit date, the cases in
which a backup skips its prune, and two sentences that read badly.

docs/DATAMODEL.md: which repository type serves which table, what the
metadata export deletes and keeps, the blobs table in a restore, and
when a chunk row is deleted.

docs/REPOSTRUCTURE.md: how an object's size relates to its contents,
what blob sizes can leak and when, and a sentence moved out of the
prune rules.

All three now say that prune drops a local snapshot whose manifest is
gone, not whose directory is gone.

Model: fable-5-1
This commit is contained in:
2026-11-06 19:34:28 +00:00
parent a52a8a781c
commit 252db3259a
3 changed files with 63 additions and 33 deletions
+28 -15
View File
@@ -327,7 +327,12 @@ holds the file's new chunks is stored.
The metadata of a snapshot is made from a copy of the index:
- The index is copied into a temporary directory with `VACUUM INTO`.
- Every row that does not belong to this snapshot is deleted from the copy.
- The rows of every other snapshot are deleted from the copy. So are the files
and blobs that this snapshot does not use. A chunk is then deleted when no
remaining file and no remaining blob uses it.
- The `local_meta` and `schema_migrations` tables are kept as they are. The
uploaded database therefore names the destination, and lists the migrations
that have run.
- `VACUUM` rewrites the copy, so that no deleted row is left in the file.
- The copy is compressed with zstd and encrypted with age.
- The manifest is built from the copy's `snapshot_blobs` table.
@@ -347,6 +352,7 @@ A restore does not use the local index.
- It reads the files to restore from `files`.
- It finds each file's chunks in `file_chunks`.
- It finds the blob and offset of each chunk in `blob_chunks`.
- It finds the name of each blob in `blobs`.
It does not read the manifest.
@@ -354,7 +360,7 @@ It does not read the manifest.
`prune` changes the local index in two ways:
- It deletes each snapshot whose directory is gone from the destination.
- It deletes each snapshot whose manifest is gone from the destination.
- It deletes unfinished snapshots, and the rows that no snapshot uses.
The second step removes unused rows in this order:
@@ -362,7 +368,7 @@ The second step removes unused rows in this order:
- files that are in no snapshot
- blobs that are in no snapshot
- `blob_chunks` rows whose chunk is gone
- chunks that no file uses
- chunks that no file and no blob uses
`prune` then deletes unused data at the destination. That step reads the
manifests, not the local index. It reads the local index only to keep a
@@ -370,19 +376,26 @@ directory under `metadata/` whose snapshot is still recorded there.
## Code
Each table has a repository type in `internal/database`:
Each repository type in `internal/database` serves one or more tables:
- `FileRepository`
- `ChunkRepository`
- `FileChunkRepository`
- `ChunkFileRepository`
- `BlobRepository`
- `BlobChunkRepository`
- `SnapshotRepository`
- `UploadRepository`
- `LocalMetaRepository`
- `FileRepository`: `files`
- `ChunkRepository`: `chunks`
- `FileChunkRepository`: `file_chunks`
- `ChunkFileRepository`: `chunk_files`
- `BlobRepository`: `blobs`
- `BlobChunkRepository`: `blob_chunks`
- `SnapshotRepository`:
- `snapshots`
- `snapshot_files`
- `snapshot_blobs`
- `snapshot_mount_points`
- `UploadRepository`: `uploads`
- `LocalMetaRepository`: `local_meta`
`Repositories` holds one of each.
No repository type serves `schema_migrations`. The code that applies the
migrations reads and writes it.
`Repositories` holds one of each type.
`Repositories.WithTx` runs a function in one transaction:
@@ -401,5 +414,5 @@ rolled back.
`snapshot list`, can read the index while a backup writes to it.
- A connection waits up to 10 seconds for a lock before it fails.
- Foreign keys are enforced.
- A process opens one connection, so all its access to the index is in turn.
- Each open database uses one connection, so its queries run one at a time.
- Queries pass their values as parameters.