Trust only uploaded blobs for deduplication (closes #148)
An interrupted blob upload left the blob's chunks, blob_chunks, and blobs rows committed before the upload was attempted, so a later run deduplicated against data that never reached storage and produced a snapshot that reported success but could not be restored. Fix (issue option b): a chunk counts as known only when a blob holding it has uploaded_ts set, and each run drops un-uploaded blob rows and the chunks they orphan at startup, so the affected data is re-chunked and re-uploaded. A blob recorded with no remote backend is marked uploaded so the invariant holds uniformly. The reproduction is the interrupted-upload test from #72: its t.Skip is removed and it passes against this fix, and this branch's earlier duplicate copy is dropped. The interrupted metadata-export case is split to #177. Model: opus-4-8
This commit was merged in pull request #175.
This commit is contained in:
@@ -349,16 +349,16 @@ func TestInterruptedBlobUploadRecordsNoUploadedBlob(t *testing.T) {
|
||||
}
|
||||
|
||||
// Scenario 1b: after an interrupted upload, a retry on the same local
|
||||
// index must produce a restorable snapshot. It does not: the interrupted
|
||||
// run's chunk rows persist, the retry deduplicates against them, and the
|
||||
// backup silently emits a snapshot referencing data never stored. Skipped
|
||||
// pending the fix. See https://git.eeqj.de/sneak/vaultik/issues/148.
|
||||
// index must produce a restorable snapshot. The interrupted run leaves
|
||||
// the blob's chunk rows in the index; the fix for
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/148 discards those un-uploaded
|
||||
// blob rows at the start of the next scan and deduplicates only against
|
||||
// chunks in a blob that was actually uploaded, so the retry re-chunks and
|
||||
// re-uploads the affected data instead of silently referencing data that
|
||||
// never reached storage.
|
||||
//
|
||||
//nolint:paralleltest // installs the global logger via log.Initialize
|
||||
func TestBackupRetryAfterInterruptedUploadIsRestorable(t *testing.T) {
|
||||
t.Skip("blocked on https://git.eeqj.de/sneak/vaultik/issues/148: " +
|
||||
"retry after an interrupted upload silently produces an " +
|
||||
"unrestorable snapshot")
|
||||
log.Initialize(log.Config{})
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
@@ -418,7 +418,7 @@ func TestBackupRetryAfterInterruptedUploadIsRestorable(t *testing.T) {
|
||||
// with blobs and a database but no manifest. verify and snapshot list
|
||||
// must report the damage honestly rather than crashing or passing.
|
||||
// Automatic detection and repair of this partial state on the next run
|
||||
// is tracked in https://git.eeqj.de/sneak/vaultik/issues/148 and is not
|
||||
// is tracked in https://git.eeqj.de/sneak/vaultik/issues/177 and is not
|
||||
// asserted here.
|
||||
//
|
||||
//nolint:paralleltest // installs the global logger via log.Initialize
|
||||
|
||||
Reference in New Issue
Block a user