Run make fmt over the Markdown
Only prettier's output; no text change. Model: opus-5-5
This commit is contained in:
sneak committed
1 parent
7a3cd581b2
commit
514e4f9f8b
6 files changed
+2236
-2047
No files matched your search
+145
-64
@@ -2,7 +2,9 @@
|
||||
|
||||
## Overview
|
||||
|
||||
Vaultik uses a local SQLite database to track file metadata, chunk mappings, and blob associations during the backup process. This database serves as an index for incremental backups and enables efficient deduplication.
|
||||
Vaultik uses a local SQLite database to track file metadata, chunk mappings, and
|
||||
blob associations during the backup process. This database serves as an index
|
||||
for incremental backups and enables efficient deduplication.
|
||||
|
||||
**Important Notes:**
|
||||
|
||||
@@ -25,19 +27,23 @@ recorded `001.sql` as run, so it keeps the old schema and can become unusable;
|
||||
`vaultik database delete` followed by a full backup rebuilds it.
|
||||
|
||||
**After 1.0** each schema change is a new numbered file, so an existing local
|
||||
database is migrated the first time an updated vaultik opens it. A file that
|
||||
has shipped in a release is never edited.
|
||||
database is migrated the first time an updated vaultik opens it. A file that has
|
||||
shipped in a release is never edited.
|
||||
|
||||
## Database Tables
|
||||
|
||||
### 1. `files`
|
||||
|
||||
Stores metadata about files in the filesystem being backed up.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `id` (TEXT PRIMARY KEY) - UUID for the file record
|
||||
- `path` (TEXT NOT NULL UNIQUE) - Absolute file path
|
||||
- `mtime` (INTEGER NOT NULL) - Modification time, whole seconds since the Unix epoch
|
||||
- `mtime_nsec` (INTEGER NOT NULL) - Nanoseconds within that second, 0 to 999999999
|
||||
- `mtime` (INTEGER NOT NULL) - Modification time, whole seconds since the Unix
|
||||
epoch
|
||||
- `mtime_nsec` (INTEGER NOT NULL) - Nanoseconds within that second, 0 to
|
||||
999999999
|
||||
- `size` (INTEGER NOT NULL) - File size in bytes
|
||||
- `mode` (INTEGER NOT NULL) - Unix file permissions and type
|
||||
- `uid` (INTEGER NOT NULL) - User ID of file owner
|
||||
@@ -45,23 +51,30 @@ Stores metadata about files in the filesystem being backed up.
|
||||
- `link_target` (TEXT) - Symlink target path (NULL for regular files)
|
||||
|
||||
**Indexes:**
|
||||
|
||||
- `idx_files_path` on `path` for efficient lookups
|
||||
|
||||
**Purpose:** Tracks file metadata to detect changes between backup runs. Used for incremental backup decisions. The UUID primary key provides stable references that don't change if files are moved.
|
||||
**Purpose:** Tracks file metadata to detect changes between backup runs. Used
|
||||
for incremental backup decisions. The UUID primary key provides stable
|
||||
references that don't change if files are moved.
|
||||
|
||||
### 2. `chunks`
|
||||
|
||||
Stores information about content-defined chunks created from files.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `chunk_hash` (TEXT PRIMARY KEY) - SHA256 hash of chunk content
|
||||
- `size` (INTEGER NOT NULL) - Chunk size in bytes
|
||||
|
||||
**Purpose:** Enables deduplication by tracking unique chunks across all files.
|
||||
|
||||
### 3. `file_chunks`
|
||||
|
||||
Maps files to their constituent chunks in order.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `file_id` (TEXT) - File ID (FK to files.id)
|
||||
- `idx` (INTEGER) - Chunk index within file (0-based)
|
||||
- `chunk_hash` (TEXT) - Chunk hash (FK to chunks.chunk_hash)
|
||||
@@ -70,35 +83,49 @@ Maps files to their constituent chunks in order.
|
||||
**Purpose:** Allows reconstruction of files from chunks during restore.
|
||||
|
||||
### 4. `chunk_files`
|
||||
|
||||
Reverse mapping showing which files contain each chunk.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `chunk_hash` (TEXT) - Chunk hash (FK to chunks.chunk_hash)
|
||||
- `file_id` (TEXT) - File ID (FK to files.id)
|
||||
- `file_offset` (INTEGER) - Byte offset of chunk within file
|
||||
- `length` (INTEGER) - Length of chunk in bytes
|
||||
- PRIMARY KEY (`chunk_hash`, `file_id`)
|
||||
|
||||
**Purpose:** Supports efficient queries for chunk usage and deduplication statistics.
|
||||
**Purpose:** Supports efficient queries for chunk usage and deduplication
|
||||
statistics.
|
||||
|
||||
### 5. `blobs`
|
||||
|
||||
Stores information about packed, compressed, and encrypted blob files.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `id` (TEXT PRIMARY KEY) - UUID assigned when blob creation starts
|
||||
- `blob_hash` (TEXT UNIQUE) - `hex(SHA256(SHA256(uncompressed blob contents)))`, computed before compression and encryption (NULL until finalized); see [REPOSTRUCTURE.md](REPOSTRUCTURE.md#blobs-directory-blobs)
|
||||
- `blob_hash` (TEXT UNIQUE) - `hex(SHA256(SHA256(uncompressed blob contents)))`,
|
||||
computed before compression and encryption (NULL until finalized); see
|
||||
[REPOSTRUCTURE.md](REPOSTRUCTURE.md#blobs-directory-blobs)
|
||||
- `created_ts` (INTEGER NOT NULL) - Creation timestamp
|
||||
- `finished_ts` (INTEGER) - Finalization timestamp (NULL if in progress)
|
||||
- `uncompressed_size` (INTEGER NOT NULL DEFAULT 0) - Total size of chunks before compression
|
||||
- `compressed_size` (INTEGER NOT NULL DEFAULT 0) - Size after compression and encryption. Once uploaded, the size of the object in storage: when the blob was already stored, under another compression level or other recipients, this can differ from the size of the local copy
|
||||
- `uncompressed_size` (INTEGER NOT NULL DEFAULT 0) - Total size of chunks before
|
||||
compression
|
||||
- `compressed_size` (INTEGER NOT NULL DEFAULT 0) - Size after compression and
|
||||
encryption. Once uploaded, the size of the object in storage: when the blob
|
||||
was already stored, under another compression level or other recipients, this
|
||||
can differ from the size of the local copy
|
||||
- `uploaded_ts` (INTEGER) - Upload completion timestamp (NULL if not uploaded)
|
||||
|
||||
**Purpose:** Tracks blob lifecycle from creation through upload. The UUID primary key allows immediate association of chunks with blobs.
|
||||
**Purpose:** Tracks blob lifecycle from creation through upload. The UUID
|
||||
primary key allows immediate association of chunks with blobs.
|
||||
|
||||
### 6. `blob_chunks`
|
||||
|
||||
Maps chunks to the blobs that contain them.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `blob_id` (TEXT) - Blob ID (FK to blobs.id)
|
||||
- `chunk_hash` (TEXT) - Chunk hash (FK to chunks.chunk_hash)
|
||||
- `offset` (INTEGER) - Byte offset of chunk within blob (before compression)
|
||||
@@ -108,34 +135,46 @@ Maps chunks to the blobs that contain them.
|
||||
**Purpose:** Enables chunk retrieval from blobs during restore operations.
|
||||
|
||||
### 7. `snapshots`
|
||||
|
||||
Tracks backup snapshots.
|
||||
|
||||
**Columns:**
|
||||
- `id` (TEXT PRIMARY KEY) - Snapshot ID (format: `hostname_name_timestamp`, e.g. `server1_home_2025-06-01T12:00:00Z`: the hostname up to its first `.`, the snapshot name, and an RFC 3339 UTC timestamp)
|
||||
|
||||
- `id` (TEXT PRIMARY KEY) - Snapshot ID (format: `hostname_name_timestamp`, e.g.
|
||||
`server1_home_2025-06-01T12:00:00Z`: the hostname up to its first `.`, the
|
||||
snapshot name, and an RFC 3339 UTC timestamp)
|
||||
- `hostname` (TEXT) - Hostname where backup was created
|
||||
- `vaultik_version` (TEXT) - Version of Vaultik used
|
||||
- `vaultik_git_revision` (TEXT) - Git revision of Vaultik used
|
||||
- `started_at` (INTEGER) - Start timestamp
|
||||
- `completed_at` (INTEGER) - Completion timestamp (NULL if in progress)
|
||||
- `file_count` (INTEGER) - Number of files in snapshot
|
||||
- `chunk_count` (INTEGER) - Number of chunks this snapshot stored that were not stored before
|
||||
- `new_chunk_size` (INTEGER) - Total size of those chunks before compression; `snapshot list` shows it as NEW CHUNK SIZE
|
||||
- `chunk_count` (INTEGER) - Number of chunks this snapshot stored that were not
|
||||
stored before
|
||||
- `new_chunk_size` (INTEGER) - Total size of those chunks before compression;
|
||||
`snapshot list` shows it as NEW CHUNK SIZE
|
||||
- `blob_count` (INTEGER) - Number of blobs this snapshot created
|
||||
- `total_size` (INTEGER) - Total size of all files
|
||||
- `uncompressed_chunk_size` (INTEGER) - Total size before compression of the distinct chunks the snapshot's files reference, recorded when its backup ran; `snapshot list` shows it as UNCOMPRESSED SIZE
|
||||
- `uncompressed_chunk_size` (INTEGER) - Total size before compression of the
|
||||
distinct chunks the snapshot's files reference, recorded when its backup ran;
|
||||
`snapshot list` shows it as UNCOMPRESSED SIZE
|
||||
- `blob_size` (INTEGER) - Total compressed size of all referenced blobs
|
||||
- `blob_uncompressed_size` (INTEGER) - Total uncompressed size of all referenced blobs
|
||||
- `blob_uncompressed_size` (INTEGER) - Total uncompressed size of all referenced
|
||||
blobs
|
||||
- `compression_ratio` (REAL) - Compression ratio achieved
|
||||
- `compression_level` (INTEGER) - Compression level used for this snapshot
|
||||
- `upload_bytes` (INTEGER) - Total bytes uploaded during this snapshot
|
||||
- `upload_duration_ms` (INTEGER) - Total milliseconds spent uploading to S3
|
||||
|
||||
**Purpose:** Provides snapshot metadata and statistics including version tracking for compatibility.
|
||||
**Purpose:** Provides snapshot metadata and statistics including version
|
||||
tracking for compatibility.
|
||||
|
||||
### 8. `snapshot_files`
|
||||
|
||||
Maps snapshots to the files they contain.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `snapshot_id` (TEXT) - Snapshot ID (FK to snapshots.id)
|
||||
- `file_id` (TEXT) - File ID (FK to files.id)
|
||||
- PRIMARY KEY (`snapshot_id`, `file_id`)
|
||||
@@ -143,114 +182,146 @@ Maps snapshots to the files they contain.
|
||||
**Purpose:** Records which files are included in each snapshot.
|
||||
|
||||
### 9. `snapshot_blobs`
|
||||
|
||||
Maps snapshots to the blobs they reference.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `snapshot_id` (TEXT) - Snapshot ID (FK to snapshots.id)
|
||||
- `blob_id` (TEXT) - Blob ID (FK to blobs.id)
|
||||
- `blob_hash` (TEXT) - Denormalized blob hash for manifest generation
|
||||
- PRIMARY KEY (`snapshot_id`, `blob_id`)
|
||||
|
||||
**Purpose:** Tracks blob dependencies for snapshots and enables manifest generation.
|
||||
**Purpose:** Tracks blob dependencies for snapshots and enables manifest
|
||||
generation.
|
||||
|
||||
### 10. `snapshot_mount_points`
|
||||
The directories at or below a configured path, the configured path itself included, that a snapshot's backup found an entry in and, once it had listed them, found on a different filesystem from their parent directory (a different device number), as a mount point is. An automount point whose volume cannot be mounted is not recorded, because the backup finds no entry in it.
|
||||
|
||||
The directories at or below a configured path, the configured path itself
|
||||
included, that a snapshot's backup found an entry in and, once it had listed
|
||||
them, found on a different filesystem from their parent directory (a different
|
||||
device number), as a mount point is. An automount point whose volume cannot be
|
||||
mounted is not recorded, because the backup finds no entry in it.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `snapshot_id` (TEXT) - Snapshot ID (FK to snapshots.id)
|
||||
- `path` (TEXT) - Absolute path of the directory
|
||||
- PRIMARY KEY (`snapshot_id`, `path`)
|
||||
|
||||
**Purpose:** Lets `snapshot create --prune` skip the prune when a backup does not find a mount point that an earlier snapshot of the same name found, as when the volume is not mounted.
|
||||
**Purpose:** Lets `snapshot create --prune` skip the prune when a backup does
|
||||
not find a mount point that an earlier snapshot of the same name found, as when
|
||||
the volume is not mounted.
|
||||
|
||||
### 11. `uploads`
|
||||
|
||||
Tracks blob upload metrics.
|
||||
|
||||
**Columns:**
|
||||
|
||||
- `blob_hash` (TEXT PRIMARY KEY) - Hash of uploaded blob
|
||||
- `snapshot_id` (TEXT NOT NULL) - The snapshot that triggered this upload (FK to snapshots.id)
|
||||
- `snapshot_id` (TEXT NOT NULL) - The snapshot that triggered this upload (FK to
|
||||
snapshots.id)
|
||||
- `uploaded_at` (INTEGER) - Upload timestamp
|
||||
- `size` (INTEGER) - Size of uploaded blob
|
||||
- `duration_ms` (INTEGER) - Upload duration in milliseconds
|
||||
|
||||
**Purpose:** Performance monitoring and tracking which blobs were newly created (uploaded) during each snapshot.
|
||||
**Purpose:** Performance monitoring and tracking which blobs were newly created
|
||||
(uploaded) during each snapshot.
|
||||
|
||||
## Data Flow and Operations
|
||||
|
||||
### 1. Backup Process
|
||||
|
||||
1. **File Scanning**
|
||||
- `INSERT OR REPLACE INTO files` - Update file metadata
|
||||
- `SELECT * FROM files WHERE path = ?` - Check if file has changed
|
||||
- `INSERT INTO snapshot_files` - Add file to current snapshot
|
||||
- `INSERT OR IGNORE INTO snapshot_mount_points` - Record each directory on a different filesystem from its parent that the walk found an entry in
|
||||
- `INSERT OR REPLACE INTO files` - Update file metadata
|
||||
- `SELECT * FROM files WHERE path = ?` - Check if file has changed
|
||||
- `INSERT INTO snapshot_files` - Add file to current snapshot
|
||||
- `INSERT OR IGNORE INTO snapshot_mount_points` - Record each directory on a
|
||||
different filesystem from its parent that the walk found an entry in
|
||||
|
||||
2. **Chunking** (for changed files)
|
||||
- `INSERT OR IGNORE INTO chunks` - Store new chunks
|
||||
- `INSERT INTO file_chunks` - Map chunks to file
|
||||
- `INSERT INTO chunk_files` - Create reverse mapping
|
||||
- `INSERT OR IGNORE INTO chunks` - Store new chunks
|
||||
- `INSERT INTO file_chunks` - Map chunks to file
|
||||
- `INSERT INTO chunk_files` - Create reverse mapping
|
||||
|
||||
3. **Blob Packing**
|
||||
- `INSERT INTO blobs` - Create blob record with UUID (blob_hash NULL)
|
||||
- `INSERT INTO blob_chunks` - Associate chunks with blob immediately
|
||||
- `UPDATE blobs SET blob_hash = ?, finished_ts = ?` - Finalize blob after packing
|
||||
- `INSERT INTO blobs` - Create blob record with UUID (blob_hash NULL)
|
||||
- `INSERT INTO blob_chunks` - Associate chunks with blob immediately
|
||||
- `UPDATE blobs SET blob_hash = ?, finished_ts = ?` - Finalize blob after
|
||||
packing
|
||||
|
||||
4. **Upload**
|
||||
- `UPDATE blobs SET uploaded_ts = ?, compressed_size = ?` - Mark blob as uploaded, with the size of the object in storage (the upload is skipped when the blob is already stored)
|
||||
- `INSERT INTO uploads` - Record upload metrics with snapshot_id
|
||||
- `INSERT INTO snapshot_blobs` - Associate blob with snapshot
|
||||
- `UPDATE blobs SET uploaded_ts = ?, compressed_size = ?` - Mark blob as
|
||||
uploaded, with the size of the object in storage (the upload is skipped
|
||||
when the blob is already stored)
|
||||
- `INSERT INTO uploads` - Record upload metrics with snapshot_id
|
||||
- `INSERT INTO snapshot_blobs` - Associate blob with snapshot
|
||||
|
||||
5. **Snapshot Completion**
|
||||
- `UPDATE snapshots SET completed_at = ?, stats...` - Finalize snapshot
|
||||
- Generate and upload blob manifest from `snapshot_blobs`
|
||||
- `UPDATE snapshots SET completed_at = ?, stats...` - Finalize snapshot
|
||||
- Generate and upload blob manifest from `snapshot_blobs`
|
||||
|
||||
### 2. Incremental Backup
|
||||
|
||||
1. **Change Detection**
|
||||
- `SELECT * FROM files WHERE path = ?` - Get previous file metadata
|
||||
- Compare mtime, size, mode to detect changes
|
||||
- Re-chunk a file that lists a chunk no uploaded blob holds, even when its metadata is unchanged
|
||||
- Skip unchanged files but still add to `snapshot_files`
|
||||
- `SELECT * FROM files WHERE path = ?` - Get previous file metadata
|
||||
- Compare mtime, size, mode to detect changes
|
||||
- Re-chunk a file that lists a chunk no uploaded blob holds, even when its
|
||||
metadata is unchanged
|
||||
- Skip unchanged files but still add to `snapshot_files`
|
||||
|
||||
2. **Chunk Reuse**
|
||||
- `SELECT * FROM blob_chunks WHERE chunk_hash = ?` - Find existing chunks
|
||||
- `INSERT INTO snapshot_blobs` - Reference existing blobs for unchanged files
|
||||
- `SELECT * FROM blob_chunks WHERE chunk_hash = ?` - Find existing chunks
|
||||
- `INSERT INTO snapshot_blobs` - Reference existing blobs for unchanged
|
||||
files
|
||||
|
||||
### 3. Snapshot Metadata Export
|
||||
|
||||
After a snapshot is completed:
|
||||
|
||||
1. Copy database to temporary file
|
||||
2. Clean temporary database to contain only current snapshot data
|
||||
3. VACUUM the trimmed database so deleted rows leave no pages behind
|
||||
4. Compress with zstd and encrypt with age
|
||||
5. Upload to S3 as `metadata/{remote-key}/db.zst.age`
|
||||
6. Generate blob manifest and upload as `metadata/{remote-key}/manifest.json.zst`
|
||||
6. Generate blob manifest and upload as
|
||||
`metadata/{remote-key}/manifest.json.zst`
|
||||
|
||||
The `{remote-key}` directory name is a one-way hash of the human snapshot ID, so the ID is never written to the store in plaintext. The hash uses no secret, so a guessed hostname and snapshot name can still be confirmed against a listing; see [REPOSTRUCTURE.md](REPOSTRUCTURE.md#remote-key-derivation) and its [Accepted Risks](REPOSTRUCTURE.md#accepted-risks).
|
||||
The `{remote-key}` directory name is a one-way hash of the human snapshot ID, so
|
||||
the ID is never written to the store in plaintext. The hash uses no secret, so a
|
||||
guessed hostname and snapshot name can still be confirmed against a listing; see
|
||||
[REPOSTRUCTURE.md](REPOSTRUCTURE.md#remote-key-derivation) and its
|
||||
[Accepted Risks](REPOSTRUCTURE.md#accepted-risks).
|
||||
|
||||
### 4. Restore Process
|
||||
|
||||
The restore process doesn't use the local database. Instead:
|
||||
1. Downloads and decrypts the snapshot's metadata database (`db.zst.age`) from S3
|
||||
2. Downloads the blobs holding the chunks of the files being restored, found through that database's `blob_chunks` table; the manifest is not read
|
||||
|
||||
1. Downloads and decrypts the snapshot's metadata database (`db.zst.age`) from
|
||||
S3
|
||||
2. Downloads the blobs holding the chunks of the files being restored, found
|
||||
through that database's `blob_chunks` table; the manifest is not read
|
||||
3. Reconstructs files from decrypted and decompressed chunks
|
||||
|
||||
### 5. Pruning
|
||||
|
||||
1. **Identify Unreferenced Blobs**
|
||||
- Query blobs not referenced by any remaining snapshot
|
||||
- Delete from S3, once older than `prune_grace_period`, and from the local
|
||||
database
|
||||
- Query blobs not referenced by any remaining snapshot
|
||||
- Delete from S3, once older than `prune_grace_period`, and from the local
|
||||
database
|
||||
2. **Delete Metadata Directories With No Manifest**
|
||||
- Delete each directory under `metadata/` that has no `manifest.json.zst`
|
||||
and no local snapshot record, once every object in it is older than
|
||||
`prune_grace_period`
|
||||
- Delete each directory under `metadata/` that has no `manifest.json.zst`
|
||||
and no local snapshot record, once every object in it is older than
|
||||
`prune_grace_period`
|
||||
|
||||
### 6. Incomplete Snapshot Cleanup
|
||||
|
||||
Before each backup:
|
||||
|
||||
1. Query incomplete snapshots (where `completed_at IS NULL`)
|
||||
2. Delete each one and all its associations, without checking S3 for its metadata
|
||||
2. Delete each one and all its associations, without checking S3 for its
|
||||
metadata
|
||||
3. Clean up orphaned files, chunks, and blobs
|
||||
|
||||
## Repository Pattern
|
||||
@@ -260,18 +331,20 @@ Vaultik uses a repository pattern for database access:
|
||||
- `FileRepository` - CRUD operations for files and file metadata
|
||||
- `ChunkRepository` - CRUD operations for content chunks
|
||||
- `FileChunkRepository` - Manage file-to-chunk mappings
|
||||
- `ChunkFileRepository` - Manage chunk-to-file reverse mappings
|
||||
- `ChunkFileRepository` - Manage chunk-to-file reverse mappings
|
||||
- `BlobRepository` - Manage blob lifecycle (creation, finalization, upload)
|
||||
- `BlobChunkRepository` - Manage blob-to-chunk associations
|
||||
- `SnapshotRepository` - Manage snapshots and their relationships
|
||||
- `UploadRepository` - Track blob upload metrics
|
||||
|
||||
Each repository provides methods like:
|
||||
|
||||
- `Create()` - Insert new record
|
||||
- `GetByID()` / `GetByPath()` / `GetByHash()` - Retrieve records
|
||||
- `Update()` - Update existing records
|
||||
- `Delete()` - Remove records
|
||||
- Specialized queries for each entity type (e.g., `DeleteOrphaned()`, `GetIncompleteByHostname()`)
|
||||
- Specialized queries for each entity type (e.g., `DeleteOrphaned()`,
|
||||
`GetIncompleteByHostname()`)
|
||||
|
||||
## Transaction Management
|
||||
|
||||
@@ -284,25 +357,33 @@ err := repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
|
||||
```
|
||||
|
||||
This ensures consistency, especially important for operations like:
|
||||
|
||||
- Creating file-chunk mappings
|
||||
- Associating chunks with blobs
|
||||
- Updating snapshot statistics
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
1. **Indexes**:
|
||||
- Primary keys are automatically indexed
|
||||
- `idx_files_path` on `files(path)` for efficient file lookups
|
||||
1. **Indexes**:
|
||||
- Primary keys are automatically indexed
|
||||
- `idx_files_path` on `files(path)` for efficient file lookups
|
||||
|
||||
2. **Prepared Statements**: All queries use prepared statements for performance and security
|
||||
2. **Prepared Statements**: All queries use prepared statements for performance
|
||||
and security
|
||||
|
||||
3. **Batch Operations**: Where possible, operations are batched within transactions
|
||||
3. **Batch Operations**: Where possible, operations are batched within
|
||||
transactions
|
||||
|
||||
4. **Write-Ahead Logging**: The local index runs in SQLite WAL mode with a 10-second busy timeout, so a read-only command such as `snapshot list` can read it while a backup writes to it. Committed rows can sit in the `-wal` file beside the index until a checkpoint, so the metadata export copies the index through SQLite (`VACUUM INTO`), not as a file
|
||||
4. **Write-Ahead Logging**: The local index runs in SQLite WAL mode with a
|
||||
10-second busy timeout, so a read-only command such as `snapshot list` can
|
||||
read it while a backup writes to it. Committed rows can sit in the `-wal`
|
||||
file beside the index until a checkpoint, so the metadata export copies the
|
||||
index through SQLite (`VACUUM INTO`), not as a file
|
||||
|
||||
## Data Integrity
|
||||
|
||||
1. **Foreign Keys**: Enforced through CASCADE DELETE and application-level repository methods
|
||||
1. **Foreign Keys**: Enforced through CASCADE DELETE and application-level
|
||||
repository methods
|
||||
2. **Unique Constraints**: Chunk hashes, file paths, and blob hashes are unique
|
||||
3. **Null Handling**: Nullable fields clearly indicate in-progress operations
|
||||
4. **Timestamp Tracking**: All major operations record timestamps for auditing
|
||||
4. **Timestamp Tracking**: All major operations record timestamps for auditing
|
||||
Reference in new issue
Block a user