Keep unreferenced blobs newer than prune_grace_period (closes #387)
check / check (push) Successful in 11m59s
check / check (push) Successful in 11m59s
A backup uploads its blobs before the manifest that lists them, and the process lock covers one user on one machine, so a prune from another host or user sharing the destination deleted a running backup's blobs. Prune now deletes an unreferenced blob only once its listed time is older than the new config key prune_grace_period (default 24h; a negative value is refused). A newer blob, or one listed with no time, is kept and counted (blobs_too_new). A backup uploads a blob already on the destination again instead of skipping it, so it gets a fresh time. The rclone backend takes each object's time from the listing (use_server_modtime). Cost: after database delete, the next backup uploads all its data again. Side effect: a rebuilt blob replaces the stored copy; if compression_level or the recipients changed since, older snapshots listing it fail verify. Model: opus-5-5
This commit is contained in:
@@ -14,6 +14,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ErrNotFound is returned when an object does not exist.
|
||||
@@ -27,7 +28,10 @@ type ProgressCallback func(bytesTransferred int64) error
|
||||
type ObjectInfo struct {
|
||||
Key string // Object key/path
|
||||
Size int64 // Size in bytes
|
||||
Err error // Error for streaming results (nil on success)
|
||||
// ModTime is when the object was written, as the store reports it.
|
||||
// Only ListStream sets it; Stat leaves it zero.
|
||||
ModTime time.Time
|
||||
Err error // Error for streaming results (nil on success)
|
||||
}
|
||||
|
||||
// Info provides human-readable storage configuration.
|
||||
@@ -67,7 +71,8 @@ type Storer interface {
|
||||
// For large result sets, prefer ListStream.
|
||||
List(ctx context.Context, prefix string) ([]string, error)
|
||||
|
||||
// ListStream returns a channel of ObjectInfo for large result sets.
|
||||
// ListStream returns a channel of ObjectInfo, with each object's key,
|
||||
// size and modification time, for large result sets.
|
||||
// The channel is closed when listing completes.
|
||||
// If an error occurs during listing, the final item will have Err set.
|
||||
ListStream(ctx context.Context, prefix string) <-chan ObjectInfo
|
||||
|
||||
Reference in New Issue
Block a user