Mark a snapshot complete only after its metadata export succeeds (closes #177)
check / check (pull_request) Successful in 3m11s
check / check (pull_request) Successful in 3m11s
finalizeSnapshotMetadata marked the snapshot complete and then exported its metadata. A crash after completion but before/during the export left the local index showing the snapshot as complete while the destination had no manifest or database, and PruneDatabase (which drops only NULL completed_at rows) kept it: a silently unrestorable snapshot. Reorder so completion is recorded last. CompleteSnapshot is split into PopulateSnapshotBlobs (before the export, which reads snapshot_blobs) and MarkSnapshotComplete (after it). An interrupted export now leaves the snapshot incomplete, so the next run's PruneDatabase drops it and re-backs-up the data. The reverse tiny window leaves a fully restorable snapshot at the destination that the index reports honestly as remote-only. Update REPOSTRUCTURE.md guarantee 4 and the ARCHITECTURE.md flow to the new order. Add a fault-injection test driving the full create path. Model: opus-4-8
This commit is contained in:
@@ -201,11 +201,14 @@ func (sm *SnapshotManager) UpdateSnapshotStatsExtended(
|
||||
})
|
||||
}
|
||||
|
||||
// CompleteSnapshot marks a snapshot as completed and ensures snapshot_blobs
|
||||
// is populated with every blob holding any chunk referenced by the
|
||||
// snapshot's files (including deduplicated blobs uploaded by prior
|
||||
// snapshots). Without this, fully-deduplicated snapshots are unrestorable.
|
||||
func (sm *SnapshotManager) CompleteSnapshot(
|
||||
// PopulateSnapshotBlobs ensures snapshot_blobs holds an entry for every
|
||||
// blob that stores a chunk referenced by the snapshot's files, including
|
||||
// blobs deduplicated from earlier snapshots. Without it, a fully
|
||||
// deduplicated snapshot would record no blobs and be unrestorable.
|
||||
//
|
||||
// This must run before ExportSnapshotMetadata: the blob manifest and the
|
||||
// trimmed metadata database are both built from snapshot_blobs.
|
||||
func (sm *SnapshotManager) PopulateSnapshotBlobs(
|
||||
ctx context.Context, snapshotID string,
|
||||
) error {
|
||||
err := sm.repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
|
||||
@@ -219,6 +222,25 @@ func (sm *SnapshotManager) CompleteSnapshot(
|
||||
"snapshot_id", snapshotID, "added", added)
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("populating snapshot blobs: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// MarkSnapshotComplete records the snapshot's completion timestamp. On the
|
||||
// backup path this runs only after ExportSnapshotMetadata has succeeded, so
|
||||
// the local index never marks a snapshot complete while the destination
|
||||
// holds no manifest or database for it. A crash before this point leaves the
|
||||
// snapshot incomplete, and the next run's PruneDatabase drops it. See
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/177.
|
||||
func (sm *SnapshotManager) MarkSnapshotComplete(
|
||||
ctx context.Context, snapshotID string,
|
||||
) error {
|
||||
err := sm.repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
|
||||
return sm.repos.Snapshots.MarkComplete(ctx, tx, snapshotID)
|
||||
})
|
||||
if err != nil {
|
||||
@@ -230,6 +252,22 @@ func (sm *SnapshotManager) CompleteSnapshot(
|
||||
return nil
|
||||
}
|
||||
|
||||
// CompleteSnapshot populates snapshot_blobs and then marks the snapshot
|
||||
// complete. The backup path (finalizeSnapshotMetadata) instead calls the two
|
||||
// halves separately, with the metadata export between them, so completion is
|
||||
// recorded only after a successful export. This convenience is for callers
|
||||
// that do not interleave an export.
|
||||
func (sm *SnapshotManager) CompleteSnapshot(
|
||||
ctx context.Context, snapshotID string,
|
||||
) error {
|
||||
err := sm.PopulateSnapshotBlobs(ctx, snapshotID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return sm.MarkSnapshotComplete(ctx, snapshotID)
|
||||
}
|
||||
|
||||
// ExportSnapshotMetadata exports snapshot metadata to S3
|
||||
//
|
||||
// This method executes the complete snapshot metadata export process:
|
||||
|
||||
Reference in New Issue
Block a user