Interrupted blob upload leaves a partial object the next run trusts as complete (file:// backend) #130

New Issue

2026-09-21T09:43:56+02:00

clawbot commented

2026-09-21 09:43:56 +02:00

Found while adding fault-injection tests for #72.

The file backend writes a blob by creating the destination file and streaming into it (internal/storage/file.go, PutWithProgress) with no temp-file-and-rename. If the upload is cut off mid-stream, a truncated object is left at the blob's final key.

On the next backup, uploadBlobIfNeeded (internal/snapshot/scanner.go) treats any object that Stat finds as a complete blob and skips re-upload, so the truncated object is recorded in the local index as a complete blob. The snapshot then reports success but is unrestorable, and only an explicit read-back reveals it.

Reproduction: internal/vaultik/fault_injection_test.go, TestFaultInjection_MidBlobFailureRetryStaysConsistent (skipped, points to this issue).

Scope: S3 PutObject is atomic, so this is specific to the file:// backend (and any other backend that writes non-atomically).

Fix direction (owner's call): write blobs atomically (temp + rename), and/or have uploadBlobIfNeeded verify the existing object's size before trusting it.

Model: opus-4-8

Found while adding fault-injection tests for https://git.eeqj.de/sneak/vaultik/issues/72. The file backend writes a blob by creating the destination file and streaming into it (`internal/storage/file.go`, `PutWithProgress`) with no temp-file-and-rename. If the upload is cut off mid-stream, a truncated object is left at the blob's final key. On the next backup, `uploadBlobIfNeeded` (`internal/snapshot/scanner.go`) treats any object that `Stat` finds as a complete blob and skips re-upload, so the truncated object is recorded in the local index as a complete blob. The snapshot then reports success but is unrestorable, and only an explicit read-back reveals it. Reproduction: `internal/vaultik/fault_injection_test.go`, `TestFaultInjection_MidBlobFailureRetryStaysConsistent` (skipped, points to this issue). Scope: S3 `PutObject` is atomic, so this is specific to the `file://` backend (and any other backend that writes non-atomically). Fix direction (owner's call): write blobs atomically (temp + rename), and/or have `uploadBlobIfNeeded` verify the existing object's size before trusting it. Model: opus-4-8

clawbot added this to the 1.0.0 milestone 2026-09-21 09:47:34 +02:00

clawbot referenced this issue from a commit

2026-09-21 21:01:06 +02:00

Write file:// blobs atomically via temp file and rename (closes #130)

clawbot referenced a pull request that will close this issue

2026-09-21 21:01:37 +02:00

Write file:// blobs atomically via temp file and rename (closes #130) #143

clawbot commented

2026-09-21 21:01:47 +02:00

Fixed in #143.

The file:// backend now writes each object to a temp file, fsyncs it, and renames it onto the key, so an interrupted upload leaves a stray .partial temp file (skipped by List/ListStream) rather than a truncated object the next run trusts as a complete blob. S3 is already atomic and unaffected. Includes tests that fail without the change.

model: claude-opus-4-8

Fixed in https://git.eeqj.de/sneak/vaultik/pulls/143. The `file://` backend now writes each object to a temp file, fsyncs it, and renames it onto the key, so an interrupted upload leaves a stray `.partial` temp file (skipped by `List`/`ListStream`) rather than a truncated object the next run trusts as a complete blob. S3 is already atomic and unaffected. Includes tests that fail without the change. model: claude-opus-4-8

clawbot closed this issue

2026-09-21 21:24:43 +02:00

clawbot referenced this issue from a commit

2026-09-21 21:24:44 +02:00

Write file:// blobs atomically via temp file and rename (closes #130)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sneak/vaultik#130