Re-chunk a known file whose chunks no uploaded blob holds (closes #214)
check / check (pull_request) Successful in 7m37s

File rows are shared by every snapshot and updated in place, while a
blob row is deleted once no snapshot references it. Removing the newest
snapshot, or the prune after an interrupted run, could drop the only
blob holding a changed file's current chunks while an older snapshot
kept the file row. The next backup compared metadata only, skipped the
file, and completed a snapshot that could not restore it.

The scanner now loads the IDs of known files that list a chunk no
uploaded blob holds and re-chunks them even when their metadata is
unchanged.

The tests append to a file, so the file keeps its first chunk in a blob
the first snapshot still references. Each backup run gets its own
snapshot name, so the second-precision snapshot IDs differ without
sleeping.

Model: opus-5-5
This commit is contained in:
2026-10-06 01:57:01 +00:00
parent 070090124a
commit d47282a1f1
7 changed files with 343 additions and 4 deletions
+5 -3
View File
@@ -40,7 +40,8 @@ Features:
* modern encryption ([age](https://age-encryption.org/), X25519 + ChaCha20-Poly1305)
* content-defined chunking with deduplication (FastCDC)
* incremental backups (only changed files are re-chunked)
* incremental backups (a file is re-chunked only when it changed or a
chunk it lists is held by no uploaded blob)
* multithreaded zstd compression at configurable levels
* content-addressed immutable storage
* local state tracking in SQLite (enables write-only incremental backups)
@@ -499,8 +500,9 @@ format does and does not protect.
* Content-defined chunking using the FastCDC algorithm
* Average chunk size: configurable (default 10MB)
* Deduplication at file level (unchanged files skipped) and chunk level
(identical chunks across files stored once)
* Deduplication at file level (unchanged files skipped, unless a chunk
the file lists is held by no uploaded blob) and chunk level (identical
chunks across files stored once)
* Multiple chunks packed into blobs to reduce object count
### encryption