Trust only uploaded blobs for deduplication (closes #148)
check / check (pull_request) Successful in 1m23s
check / check (push) Successful in 3m10s

An interrupted blob upload left the blob's chunks, blob_chunks, and blobs rows committed before the upload was attempted, so a later run deduplicated against data that never reached storage and produced a snapshot that reported success but could not be restored.

Fix (issue option b): a chunk counts as known only when a blob holding it has uploaded_ts set, and each run drops un-uploaded blob rows and the chunks they orphan at startup, so the affected data is re-chunked and re-uploaded. A blob recorded with no remote backend is marked uploaded so the invariant holds uniformly.

The reproduction is the interrupted-upload test from #72: its t.Skip is removed and it passes against this fix, and this branch's earlier duplicate copy is dropped. The interrupted metadata-export case is split to #177.

Model: opus-4-8
This commit was merged in pull request #175.
This commit is contained in:
2026-09-22 10:29:21 +02:00
parent 6b7517a4dc
commit 38ebfd843a
5 changed files with 124 additions and 15 deletions
+8 -8
View File
@@ -349,16 +349,16 @@ func TestInterruptedBlobUploadRecordsNoUploadedBlob(t *testing.T) {
}
// Scenario 1b: after an interrupted upload, a retry on the same local
// index must produce a restorable snapshot. It does not: the interrupted
// run's chunk rows persist, the retry deduplicates against them, and the
// backup silently emits a snapshot referencing data never stored. Skipped
// pending the fix. See https://git.eeqj.de/sneak/vaultik/issues/148.
// index must produce a restorable snapshot. The interrupted run leaves
// the blob's chunk rows in the index; the fix for
// https://git.eeqj.de/sneak/vaultik/issues/148 discards those un-uploaded
// blob rows at the start of the next scan and deduplicates only against
// chunks in a blob that was actually uploaded, so the retry re-chunks and
// re-uploads the affected data instead of silently referencing data that
// never reached storage.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestBackupRetryAfterInterruptedUploadIsRestorable(t *testing.T) {
t.Skip("blocked on https://git.eeqj.de/sneak/vaultik/issues/148: " +
"retry after an interrupted upload silently produces an " +
"unrestorable snapshot")
log.Initialize(log.Config{})
fs := afero.NewOsFs()
@@ -418,7 +418,7 @@ func TestBackupRetryAfterInterruptedUploadIsRestorable(t *testing.T) {
// with blobs and a database but no manifest. verify and snapshot list
// must report the damage honestly rather than crashing or passing.
// Automatic detection and repair of this partial state on the next run
// is tracked in https://git.eeqj.de/sneak/vaultik/issues/148 and is not
// is tracked in https://git.eeqj.de/sneak/vaultik/issues/177 and is not
// asserted here.
//
//nolint:paralleltest // installs the global logger via log.Initialize