Keep unreferenced blobs newer than prune_grace_period (closes #387)
check / check (push) Successful in 11m32s
check / check (push) Successful in 11m32s
A backup uploads its blobs before the manifest that lists them, and the process lock is per user per machine, so a prune from another host or user sharing the destination deleted a running backup's blobs; that backup still reported success and could not be restored. Prune now deletes an unreferenced blob only once it is older than the new config key prune_grace_period (default 24h), and reports how many it kept (blobs_too_new under --json). The storage listing now carries each object's modification time. remote nuke is unchanged. The README says what the lock covers and that a shared destination relies on the grace period or needs one prefix per host. Model: opus-5-5
This commit is contained in:
@@ -11,6 +11,7 @@ import (
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"filippo.io/age"
|
||||
"git.eeqj.de/sneak/smartconfig"
|
||||
@@ -38,6 +39,7 @@ const (
|
||||
defaultChunkSize = Size(10 * 1000 * 1000) // 10MB
|
||||
defaultS3PartSize = Size(5 * 1024 * 1024) // 5MiB
|
||||
defaultCompressionLevel = 3
|
||||
defaultPruneGracePeriod = 24 * time.Hour
|
||||
minChunkSize = 1000 * 1000 // 1MB
|
||||
minCompressionLevel = 1
|
||||
maxCompressionLevel = 19
|
||||
@@ -187,6 +189,13 @@ type Config struct {
|
||||
Snapshots map[string]SnapshotConfig `yaml:"snapshots"`
|
||||
CompressionLevel int `yaml:"compression_level"`
|
||||
|
||||
// PruneGracePeriod is how old a blob no manifest lists must be before
|
||||
// a prune deletes it. A backup lists its blobs in a manifest only
|
||||
// once it has uploaded them all, so a younger one may belong to a
|
||||
// backup still running, on this host or another sharing the
|
||||
// destination store.
|
||||
PruneGracePeriod time.Duration `yaml:"prune_grace_period"`
|
||||
|
||||
// StorageURL specifies the storage backend using a URL format.
|
||||
// Takes precedence over S3Config if set.
|
||||
// Supported formats:
|
||||
@@ -288,6 +297,7 @@ func Parse(data []byte) (*Config, error) {
|
||||
ChunkSize: defaultChunkSize,
|
||||
IndexPath: filepath.Join(xdg.DataHome, appName, "index.sqlite"),
|
||||
CompressionLevel: defaultCompressionLevel,
|
||||
PruneGracePeriod: defaultPruneGracePeriod,
|
||||
S3: S3Config{PartSize: defaultS3PartSize},
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user