Mark a snapshot complete only after its metadata export succeeds (closes #177)
check / check (pull_request) Successful in 3m11s

finalizeSnapshotMetadata marked the snapshot complete and then exported its
metadata. A crash after completion but before/during the export left the
local index showing the snapshot as complete while the destination had no
manifest or database, and PruneDatabase (which drops only NULL completed_at
rows) kept it: a silently unrestorable snapshot.

Reorder so completion is recorded last. CompleteSnapshot is split into
PopulateSnapshotBlobs (before the export, which reads snapshot_blobs) and
MarkSnapshotComplete (after it). An interrupted export now leaves the
snapshot incomplete, so the next run's PruneDatabase drops it and re-backs-up
the data. The reverse tiny window leaves a fully restorable snapshot at the
destination that the index reports honestly as remote-only.

Update REPOSTRUCTURE.md guarantee 4 and the ARCHITECTURE.md flow to the new
order. Add a fault-injection test driving the full create path.

Model: opus-4-8
This commit is contained in:
2026-09-22 17:44:03 +00:00
parent 1548c0f933
commit 3a1a0d7927
5 changed files with 263 additions and 22 deletions
+43 -5
View File
@@ -201,11 +201,14 @@ func (sm *SnapshotManager) UpdateSnapshotStatsExtended(
})
}
// CompleteSnapshot marks a snapshot as completed and ensures snapshot_blobs
// is populated with every blob holding any chunk referenced by the
// snapshot's files (including deduplicated blobs uploaded by prior
// snapshots). Without this, fully-deduplicated snapshots are unrestorable.
func (sm *SnapshotManager) CompleteSnapshot(
// PopulateSnapshotBlobs ensures snapshot_blobs holds an entry for every
// blob that stores a chunk referenced by the snapshot's files, including
// blobs deduplicated from earlier snapshots. Without it, a fully
// deduplicated snapshot would record no blobs and be unrestorable.
//
// This must run before ExportSnapshotMetadata: the blob manifest and the
// trimmed metadata database are both built from snapshot_blobs.
func (sm *SnapshotManager) PopulateSnapshotBlobs(
ctx context.Context, snapshotID string,
) error {
err := sm.repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
@@ -219,6 +222,25 @@ func (sm *SnapshotManager) CompleteSnapshot(
"snapshot_id", snapshotID, "added", added)
}
return nil
})
if err != nil {
return fmt.Errorf("populating snapshot blobs: %w", err)
}
return nil
}
// MarkSnapshotComplete records the snapshot's completion timestamp. On the
// backup path this runs only after ExportSnapshotMetadata has succeeded, so
// the local index never marks a snapshot complete while the destination
// holds no manifest or database for it. A crash before this point leaves the
// snapshot incomplete, and the next run's PruneDatabase drops it. See
// https://git.eeqj.de/sneak/vaultik/issues/177.
func (sm *SnapshotManager) MarkSnapshotComplete(
ctx context.Context, snapshotID string,
) error {
err := sm.repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
return sm.repos.Snapshots.MarkComplete(ctx, tx, snapshotID)
})
if err != nil {
@@ -230,6 +252,22 @@ func (sm *SnapshotManager) CompleteSnapshot(
return nil
}
// CompleteSnapshot populates snapshot_blobs and then marks the snapshot
// complete. The backup path (finalizeSnapshotMetadata) instead calls the two
// halves separately, with the metadata export between them, so completion is
// recorded only after a successful export. This convenience is for callers
// that do not interleave an export.
func (sm *SnapshotManager) CompleteSnapshot(
ctx context.Context, snapshotID string,
) error {
err := sm.PopulateSnapshotBlobs(ctx, snapshotID)
if err != nil {
return err
}
return sm.MarkSnapshotComplete(ctx, snapshotID)
}
// ExportSnapshotMetadata exports snapshot metadata to S3
//
// This method executes the complete snapshot metadata export process: