Author SHA1 Message Date
clawbot 35eed79b1a Check --keep-newer-than before the backup starts (closes #307)
check / check (push) Waiting to run
snapshot create read --keep-newer-than only in the post-backup prune,
so a value the duration parser rejects failed the run after the
snapshot was uploaded: a nightly cron line added a snapshot each night,
reported it as failed, and never deleted any. Without --prune the flag
was ignored. The command now checks the flag before it starts and fails
on an invalid duration or a missing --prune.

A failed post-backup prune is now wrapped in ErrPostBackupPruneFailed
and reported as "Backup completed, but post-backup prune failed: ..."
rather than as a failed snapshot creation. The exit status stays 1.

Model: opus-5-5
2026-10-13 09:44:44 +02:00
clawbot 94412d9ef3 Stop remote info counting a non-blob file under blobs/ (closes #295)
check / check (push) Waiting to run
remote info counted every key under blobs/ with four or more path parts
as a blob, while prune skips a name that is not a blob hash. A stray
file such as .DS_Store was reported as an orphaned blob, with advice to
run prune, which then deleted nothing. remote info now takes its blob
list from prune's listAllRemoteBlobs, so its blob and orphan figures,
in the table and in --json, cover exactly what prune would consider.

blobKeyParts moves to prune.go, its only remaining user.

Model: opus-5-5
2026-10-13 08:33:27 +02:00
clawbot 167c0a0746 Make snapshot remove fail for an unknown snapshot ID (closes #293)
check / check (push) Waiting to run
snapshot remove never checked that the ID existed: it deleted nothing,
reported the snapshot removed from the local database and from the
destination store, and exited 0, under --json, --dry-run and
--local-only too. It now looks the ID up in the local index and, only
when that has no row and --local-only is not set, on the destination
store; finding neither is an error. The command reports a removal from
the local database only when the local index had the row, and
remote_removed is true only when metadata was actually deleted.

Judgement call: under --local-only the destination store is not
consulted, so the snapshot must be in the local index.

Model: opus-5-5
2026-10-13 05:55:22 +02:00
9 changed files with 432 additions and 58 deletions
+11 -5
View File
@@ -286,7 +286,9 @@ needed.
remove orphaned blobs from remote storage. By default keeps only the latest remove orphaned blobs from remote storage. By default keeps only the latest
snapshot per name; use `--keep-newer-than` for a rolling window. snapshot per name; use `--keep-newer-than` for a rolling window.
* `--keep-newer-than <duration>`: With `--prune`, keep snapshots newer than * `--keep-newer-than <duration>`: With `--prune`, keep snapshots newer than
this duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`) this duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`).
Without `--prune`, or with a value that is not a valid duration, the
command fails before the backup starts.
**`snapshot list`**: Show every snapshot known to this host — the union **`snapshot list`**: Show every snapshot known to this host — the union
of the local index and the backup destination store — with timestamps of the local index and the backup destination store — with timestamps
@@ -349,9 +351,11 @@ the backup destination store. Blobs are NOT touched — deleting blobs
requires reading every remaining remote manifest (the destination store requires reading every remaining remote manifest (the destination store
may hold snapshots this host doesn't know about), which is what may hold snapshots this host doesn't know about), which is what
`vaultik prune` does. On success the command prints the exact `vaultik `vaultik prune` does. On success the command prints the exact `vaultik
prune` invocation to run as a follow-up. Local row cleanup (files, prune` invocation to run as a follow-up. An ID that names no snapshot in
chunks, blobs the snapshot was the last referrer for) runs the local index or on the destination store is an error; with
automatically. If the destination store is unreachable, the local-DB `--local-only`, the snapshot must be in the local index. Local row
cleanup (files, chunks, blobs the snapshot was the last referrer for)
runs automatically. If the destination store is unreachable, the local-DB
removal still completes and a warning is emitted; run `vaultik snapshot removal still completes and a warning is emitted; run `vaultik snapshot
remove <snapshot-id>` again once the store is reachable to remove the remove <snapshot-id>` again once the store is reachable to remove the
snapshot's metadata from it (`vaultik prune` does not). To wipe everything snapshot's metadata from it (`vaultik prune` does not). To wipe everything
@@ -399,7 +403,9 @@ orphaned blob figures are reported as unknown; `--json` gives them as
`null`, lists the remote key of each unreadable manifest in `null`, lists the remote key of each unreadable manifest in
`unreadable_manifests` and counts the manifests under skipped names in `unreadable_manifests` and counts the manifests under skipped names in
`skipped_manifest_count`. An unreadable manifest also leaves its `skipped_manifest_count`. An unreadable manifest also leaves its
snapshot's blob count and blob size unknown, `null` in `--json`. snapshot's blob count and blob size unknown, `null` in `--json`. A file
under `blobs/` whose name is not a blob hash is not counted as a blob,
since `prune` never deletes it.
* `--json`: Output as JSON * `--json`: Output as JSON
**`remote nuke`**: Delete every snapshot's metadata and every blob from the **`remote nuke`**: Delete every snapshot's metadata and every blob from the
+18
View File
@@ -22,6 +22,24 @@ the tag exists and is exercised; what is left is merging `next` to
# Completed Steps # Completed Steps
- 2026-10-13: Stopped `remote info` counting a file under `blobs/` that
is not a blob as an orphaned blob
([issue #295](https://git.eeqj.de/sneak/vaultik/issues/295)). A stray
file such as `.DS_Store` was reported as an orphaned blob and the
report advised running `prune`, which skips such a file and deleted
nothing. `remote info` now takes its blob list from the same listing
`prune` uses, in the table and in `--json`.
- 2026-10-13: Made `snapshot remove` fail for a snapshot that does not
exist ([issue #293](https://git.eeqj.de/sneak/vaultik/issues/293)).
The command never checked the ID: it reported the snapshot removed
from the local database and from the destination store and exited 0,
under `--json`, `--dry-run` and `--local-only` too. An ID that names
no snapshot in the local index or on the destination store is now an
error; with `--local-only`, the snapshot must be in the local index.
The command now reports a removal from either side only when one
happened, and `remote_removed` is true only when metadata was deleted.
- 2026-10-12: Bound the local index to the destination an `s3.*` config - 2026-10-12: Bound the local index to the destination an `s3.*` config
names ([issue #298](https://git.eeqj.de/sneak/vaultik/issues/298)). names ([issue #298](https://git.eeqj.de/sneak/vaultik/issues/298)).
Only `storage_url` was checked, so after `s3.bucket` or `s3.prefix` Only `storage_url` was checked, so after `s3.bucket` or `s3.prefix`
@@ -0,0 +1,108 @@
package cli //nolint:testpackage // shares runEntry and the hermetic config helpers
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Flags of `snapshot create` used to build argument vectors below.
const (
flagPrune = "--prune"
flagKeepNewerThan = "--keep-newer-than"
)
// TestEntrySnapshotCreateRejectsKeepNewerThanBeforeBackup checks that a
// --keep-newer-than the post-backup prune could not use stops `snapshot
// create` before the backup: the command fails and the destination
// store stays empty. It used to be read only by the prune, after the
// snapshot was uploaded, and without --prune it was ignored.
//
//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
func TestEntrySnapshotCreateRejectsKeepNewerThanBeforeBackup(t *testing.T) {
for _, testCase := range []struct {
name string
args []string
}{
{
name: "decimal in a day count",
args: []string{flagPrune, flagKeepNewerThan, "1.5d"},
},
{
name: "unknown unit",
args: []string{flagPrune, flagKeepNewerThan, "4x"},
},
{
name: "without --prune",
args: []string{flagKeepNewerThan, "4w"},
},
} {
t.Run(testCase.name, func(t *testing.T) {
configPath, storeDir := writeBackupConfig(t)
code, _, _ := runEntry(t, append(
[]string{flagConfig, configPath, cmdSnapshot, cmdCreate},
testCase.args...)...)
assert.Equal(t, 1, code)
stored, err := os.ReadDir(storeDir)
require.NoError(t, err)
assert.Empty(t, stored, "nothing may be uploaded")
})
}
}
// TestEntrySnapshotCreateReportsPostBackupPruneFailure makes the
// post-backup prune fail after a good backup, by planting a manifest
// that cannot be decoded, and checks that the failure is reported as the
// prune's and not as a failed snapshot creation.
//
//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
func TestEntrySnapshotCreateReportsPostBackupPruneFailure(t *testing.T) {
configPath, storeDir := writeBackupConfig(t)
// A remote key is 64 lowercase hex characters. The prune reads every
// manifest on the store and fails on one it cannot decode.
plantedDir := filepath.Join(storeDir, "metadata", strings.Repeat("a", 64))
require.NoError(t, os.MkdirAll(plantedDir, 0o750))
require.NoError(t, os.WriteFile(
filepath.Join(plantedDir, "manifest.json.zst"),
[]byte("not a valid manifest"), 0o600))
code, _, stderr := runEntry(t,
flagConfig, configPath, cmdSnapshot, cmdCreate, flagPrune)
assert.Equal(t, 1, code)
assert.Contains(t, stderr, "Backup completed, but post-backup prune failed")
assert.NotContains(t, stderr, "Snapshot creation failed")
snapshots, err := os.ReadDir(filepath.Join(storeDir, "metadata"))
require.NoError(t, err)
assert.Len(t, snapshots, 2,
"the new snapshot is on the store beside the planted one")
}
// writeBackupConfig builds a hermetic config with one small file in its
// source directory and an empty destination store directory. Returns the
// config path and the store directory.
func writeBackupConfig(t *testing.T) (string, string) {
t.Helper()
configPath := writeHermeticPruneConfig(t, false)
// writeHermeticPruneConfig puts the source and store directories
// beside the config file.
dir := filepath.Dir(configPath)
sourceDir := filepath.Join(dir, "source")
require.NoError(t, os.Mkdir(sourceDir, 0o750))
require.NoError(t, os.WriteFile(
filepath.Join(sourceDir, "file.txt"), []byte("hello"), 0o600))
return configPath, filepath.Join(dir, "store")
}
+17 -1
View File
@@ -71,6 +71,11 @@ otherwise from the platform config directory (~/.config/vaultik/config.yml
on Linux), then /etc/vaultik/config.yml.`, on Linux), then /etc/vaultik/config.yml.`,
Args: cobra.ArbitraryArgs, Args: cobra.ArbitraryArgs,
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
err := opts.Validate()
if err != nil {
return err
}
// Pass snapshot names from args // Pass snapshot names from args
opts.Snapshots = args opts.Snapshots = args
// --skip-errors is a global flag on the root command. // --skip-errors is a global flag on the root command.
@@ -97,6 +102,13 @@ on Linux), then /etc/vaultik/config.yml.`,
}, func(v *vaultik.Vaultik) error { }, func(v *vaultik.Vaultik) error {
return v.CreateSnapshot(opts) return v.CreateSnapshot(opts)
}, func(err error) { }, func(err error) {
if errors.Is(err, vaultik.ErrPostBackupPruneFailed) {
log.Error("Post-backup prune failed", "error", err)
ReportErrorf("Backup completed, but %v", err)
return
}
log.Error("Snapshot creation failed", "error", err) log.Error("Snapshot creation failed", "error", err)
ReportErrorf("Snapshot creation failed: %v", err) ReportErrorf("Snapshot creation failed: %v", err)
}) })
@@ -255,8 +267,12 @@ manifest (the destination store may hold snapshots this host doesn't
know about), which is what 'vaultik prune' does. On success the command know about), which is what 'vaultik prune' does. On success the command
prints the exact 'vaultik prune' invocation to run as a follow-up. prints the exact 'vaultik prune' invocation to run as a follow-up.
An ID that names no snapshot in the local index or on the destination
store is an error.
Use --local-only to skip the remote half (e.g. when you want to forget a Use --local-only to skip the remote half (e.g. when you want to forget a
snapshot locally without touching the destination store). snapshot locally without touching the destination store); the snapshot
must then be in the local index.
If the remote is unreachable, the local-database removal still completes If the remote is unreachable, the local-database removal still completes
and a warning is emitted; run 'vaultik snapshot remove <snapshot-id>' again and a warning is emitted; run 'vaultik snapshot remove <snapshot-id>' again
+8 -19
View File
@@ -167,10 +167,6 @@ const (
// metadataKeyParts is the minimum "/"-separated segment count of a // metadataKeyParts is the minimum "/"-separated segment count of a
// metadata object key (metadata/<snapshot-id>/<filename>). // metadata object key (metadata/<snapshot-id>/<filename>).
metadataKeyParts = 3 metadataKeyParts = 3
// blobKeyParts is the minimum "/"-separated segment count of a blob
// object key (blobs/<aa>/<bb>/<hash>).
blobKeyParts = 4
) )
// SnapshotMetadataInfo contains information about a single snapshot's metadata // SnapshotMetadataInfo contains information about a single snapshot's metadata
@@ -454,23 +450,16 @@ func (v *Vaultik) scanRemoteBlobStorage(
v.stdoutf("Scanning blobs...\n") v.stdoutf("Scanning blobs...\n")
} }
blobCh := v.Storage.ListStream(v.ctx, "blobs/") // Use prune's listing, so a file under blobs/ that prune would never
allBlobs := make(map[string]int64) // delete is not counted as a blob, and so not as an orphaned one.
allBlobs, err := v.listAllRemoteBlobs()
if err != nil {
return err
}
for obj := range blobCh { for _, size := range allBlobs {
if obj.Err != nil {
return fmt.Errorf("listing blobs: %w", obj.Err)
}
parts := strings.Split(obj.Key, "/")
if len(parts) < blobKeyParts {
continue
}
hash := parts[3]
allBlobs[hash] = obj.Size
result.TotalBlobCount++ result.TotalBlobCount++
result.TotalBlobSize += obj.Size result.TotalBlobSize += size
} }
// A blob named only by a manifest that could not be read, or by one // A blob named only by a manifest that could not be read, or by one
+4
View File
@@ -23,6 +23,10 @@ var errNukeRequiresForce = errors.New(
// metadata. // metadata.
const metadataDirName = "metadata" const metadataDirName = "metadata"
// blobKeyParts is the "/"-separated segment count of a blob object key
// (blobs/<aa>/<bb>/<hash>).
const blobKeyParts = 4
// NukeRemote deletes every snapshot's metadata and every blob from remote // NukeRemote deletes every snapshot's metadata and every blob from remote
// storage, along with any object an upload cut off part-way left under a // storage, along with any object an upload cut off part-way left under a
// temporary `.partial` name. After this returns successfully the bucket // temporary `.partial` name. After this returns successfully the bucket
+33
View File
@@ -185,6 +185,39 @@ func TestRemoteInfo_SkipsNonConformingMetadataName(t *testing.T) {
assert.NotContains(t, doc, "unreadable_manifests") assert.NotContains(t, doc, "unreadable_manifests")
} }
// TestRemoteInfo_DoesNotCountNonBlobFileUnderBlobs checks that a file
// under blobs/ whose name is not a blob hash is not counted as a blob.
// prune never deletes such a file, so counting it as an orphaned blob
// would advise a prune that removes nothing.
func TestRemoteInfo_DoesNotCountNonBlobFileUnderBlobs(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newListEnv(t)
env.addRemote(t, listRemoteID, time.Date(2026, 3, 2, 0, 0, 0, 0, time.UTC))
addBlob(t, env.store.testStorer, testBlobHashA)
require.NoError(t, env.store.Put(context.Background(),
"blobs/"+testBlobHashA[:2]+"/"+testBlobHashA[2:4]+"/.DS_Store",
bytes.NewReader([]byte("junk"))))
require.NoError(t, env.v.RemoteInfo(false))
text := env.stdout.String()
assert.Contains(t, text, "Total blobs on remote: 1 (")
assert.Contains(t, text, "Orphaned (unreferenced): 0 (")
assert.NotContains(t, text, "vaultik prune")
env.stdout.Reset()
require.NoError(t, env.v.RemoteInfo(true))
var doc map[string]any
require.NoError(t, json.Unmarshal(env.stdout.Bytes(), &doc))
assert.InDelta(t, 1, doc["total_blob_count"], 0)
assert.InDelta(t, 0, doc["orphaned_blob_count"], 0)
assert.InDelta(t, 0, doc["orphaned_blob_size"], 0)
}
// TestRemoteInfo_DirectoryWithoutManifestLeavesOrphansKnown checks that // TestRemoteInfo_DirectoryWithoutManifestLeavesOrphansKnown checks that
// a directory under metadata/ holding no manifest.json.zst, such as one // a directory under metadata/ holding no manifest.json.zst, such as one
// left by a backup interrupted before its manifest upload, leaves the // left by a backup interrupted before its manifest upload, leaves the
+115
View File
@@ -3,17 +3,21 @@ package vaultik_test
import ( import (
"bytes" "bytes"
"context" "context"
"database/sql"
"io" "io"
"strings" "strings"
"sync" "sync"
"testing" "testing"
"time"
"github.com/klauspost/compress/zstd" "github.com/klauspost/compress/zstd"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log" "sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/snapshot" "sneak.berlin/go/vaultik/internal/snapshot"
"sneak.berlin/go/vaultik/internal/storage" "sneak.berlin/go/vaultik/internal/storage"
"sneak.berlin/go/vaultik/internal/types"
"sneak.berlin/go/vaultik/internal/vaultik" "sneak.berlin/go/vaultik/internal/vaultik"
) )
@@ -214,6 +218,37 @@ func addBlob(t *testing.T, store *testStorer, hash string) {
require.NoError(t, err) require.NoError(t, err)
} }
// newLocalIndex returns an in-memory local index holding a completed
// snapshot row for each of ids.
func newLocalIndex(t *testing.T, ids ...string) *database.Repositories {
t.Helper()
ctx := context.Background()
db, err := database.New(ctx, ":memory:")
require.NoError(t, err)
t.Cleanup(func() { _ = db.Close() })
repos := database.NewRepositories(db)
for _, id := range ids {
completedAt := time.Now().UTC()
snap := &database.Snapshot{
ID: types.SnapshotID(id),
Hostname: snapHostname,
StartedAt: completedAt,
CompletedAt: &completedAt,
}
err = repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
return repos.Snapshots.Create(ctx, tx, snap)
})
require.NoError(t, err)
}
return repos
}
// ============================================================================ // ============================================================================
// Unit Tests for RemoveSnapshot // Unit Tests for RemoveSnapshot
// ============================================================================ // ============================================================================
@@ -233,6 +268,7 @@ func TestRemoveSnapshot_LocalOnly_PreservesRemote(t *testing.T) {
addBlob(t, store, blobA) addBlob(t, store, blobA)
tv := vaultik.NewForTesting(store) tv := vaultik.NewForTesting(store)
tv.Repositories = newLocalIndex(t, "snapshot-001")
opts := &vaultik.RemoveOptions{Force: true, LocalOnly: true} opts := &vaultik.RemoveOptions{Force: true, LocalOnly: true}
result, err := tv.RemoveSnapshot("snapshot-001", opts) result, err := tv.RemoveSnapshot("snapshot-001", opts)
@@ -268,6 +304,7 @@ func TestRemoveSnapshot_DefaultRemovesMetadataNotBlobs(t *testing.T) {
addBlob(t, store, blobShared) addBlob(t, store, blobShared)
tv := vaultik.NewForTesting(store) tv := vaultik.NewForTesting(store)
tv.Repositories = newLocalIndex(t, "snapshot-001", "snapshot-002")
opts := &vaultik.RemoveOptions{Force: true} opts := &vaultik.RemoveOptions{Force: true}
result, err := tv.RemoveSnapshot("snapshot-001", opts) result, err := tv.RemoveSnapshot("snapshot-001", opts)
@@ -316,6 +353,84 @@ func TestRemoveSnapshot_DryRun(t *testing.T) {
assert.Contains(t, tv.Stdout.String(), "[Dry run - no changes made]") assert.Contains(t, tv.Stdout.String(), "[Dry run - no changes made]")
} }
// TestRemoveSnapshot_UnknownIDIsError checks that an ID naming no
// snapshot in the local index or on the destination store is an error
// under every option, and that nothing is reported as removed.
func TestRemoveSnapshot_UnknownIDIsError(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
for name, opts := range map[string]vaultik.RemoveOptions{
"force": {Force: true},
"json": {JSON: true},
"dry-run": {Force: true, DryRun: true},
"local-only": {Force: true, LocalOnly: true},
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
store := newTestStorer()
addManifest(t, store, "snapshot-001", nil)
tv := vaultik.NewForTesting(store)
tv.Repositories = newLocalIndex(t, "snapshot-001")
result, err := tv.RemoveSnapshot("snapshot-999", &opts)
require.ErrorContains(t, err, "snapshot not found")
assert.False(t, result.RemoteRemoved)
assert.Empty(t, tv.Stdout.String())
assert.True(t, store.hasKey(remoteKeyPath("snapshot-001")))
})
}
}
// TestRemoveSnapshot_NoMetadataOnDestination checks that removing a
// snapshot whose metadata is not on the destination store does not
// report metadata removed from it.
func TestRemoveSnapshot_NoMetadataOnDestination(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
tv := vaultik.NewForTesting(newTestStorer())
tv.Repositories = newLocalIndex(t, "snapshot-001")
result, err := tv.RemoveSnapshot("snapshot-001",
&vaultik.RemoveOptions{Force: true})
require.NoError(t, err)
assert.False(t, result.RemoteRemoved)
out := tv.Stdout.String()
assert.Contains(t, out, "Removed snapshot 'snapshot-001' from local database")
assert.NotContains(t, out, "Removed snapshot metadata from remote storage")
}
// TestRemoveSnapshot_NotInLocalIndex checks that removing a snapshot
// only the destination store holds removes its metadata there without
// reporting a removal from the local database.
func TestRemoveSnapshot_NotInLocalIndex(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
addManifest(t, store, "snapshot-001", nil)
tv := vaultik.NewForTesting(store)
tv.Repositories = newLocalIndex(t)
result, err := tv.RemoveSnapshot("snapshot-001",
&vaultik.RemoveOptions{Force: true})
require.NoError(t, err)
assert.True(t, result.RemoteRemoved)
assert.False(t, store.hasKey(remoteKeyPath("snapshot-001")))
out := tv.Stdout.String()
assert.NotContains(t, out, "from local database")
assert.Contains(t, out, "Removed snapshot metadata from remote storage")
}
func TestRemoveAllSnapshots_RequiresForce(t *testing.T) { func TestRemoveAllSnapshots_RequiresForce(t *testing.T) {
log.Initialize(log.Config{}) log.Initialize(log.Config{})
t.Parallel() t.Parallel()
+118 -33
View File
@@ -23,12 +23,19 @@ var (
errNoSnapshotsInConfig = errors.New("no snapshots configured") errNoSnapshotsInConfig = errors.New("no snapshots configured")
errSnapshotVerifyFailed = errors.New("verification failed") errSnapshotVerifyFailed = errors.New("verification failed")
errRemoveAllNeedsForce = errors.New("--all requires --force") errRemoveAllNeedsForce = errors.New("--all requires --force")
errSnapshotNotFound = errors.New("snapshot not found")
errInvalidTableName = errors.New("invalid table name") errInvalidTableName = errors.New("invalid table name")
errNoAgeRecipients = errors.New( errNoAgeRecipients = errors.New(
"creating a snapshot needs at least one public key in " + "creating a snapshot needs at least one public key in " +
"age_recipients (generate a keypair with: age-keygen)") "age_recipients (generate a keypair with: age-keygen)")
errKeepNewerThanNeedsPrune = errors.New("--keep-newer-than requires --prune")
) )
// ErrPostBackupPruneFailed wraps a failure of the post-backup prune,
// which runs only after every snapshot was created and uploaded, so the
// caller can report it apart from a failed backup.
var ErrPostBackupPruneFailed = errors.New("post-backup prune failed")
// listRecentLimit caps how many snapshot rows are fetched from the // listRecentLimit caps how many snapshot rows are fetched from the
// local index for listing and purge operations. // local index for listing and purge operations.
const listRecentLimit = 10000 const listRecentLimit = 10000
@@ -44,6 +51,26 @@ type SnapshotCreateOptions struct {
Snapshots []string // Optional list of snapshot names to process (empty = all) Snapshots []string // Optional list of snapshot names to process (empty = all)
} }
// Validate fails when KeepNewerThan is set without Prune or is not a
// valid duration. Only the post-backup prune reads KeepNewerThan, after
// the backup, so the caller checks it before starting one.
func (opts *SnapshotCreateOptions) Validate() error {
if opts.KeepNewerThan == "" {
return nil
}
if !opts.Prune {
return errKeepNewerThanNeedsPrune
}
_, err := parseDuration(opts.KeepNewerThan)
if err != nil {
return fmt.Errorf("--keep-newer-than: %w", err)
}
return nil
}
// CreateSnapshot executes the snapshot creation operation // CreateSnapshot executes the snapshot creation operation
func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error { func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
// config.Load accepts an empty list, since listing, verifying and // config.Load accepts an empty list, since listing, verifying and
@@ -122,7 +149,7 @@ func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
if opts.Prune { if opts.Prune {
err = v.runPostBackupPrune(snapshotNames, opts.KeepNewerThan) err = v.runPostBackupPrune(snapshotNames, opts.KeepNewerThan)
if err != nil { if err != nil {
return fmt.Errorf("post-backup prune: %w", err) return fmt.Errorf("%w: %w", ErrPostBackupPruneFailed, err)
} }
} }
@@ -643,7 +670,7 @@ func (v *Vaultik) confirmAndExecutePurge(
"snapshot_id", snapshotID, "error", err) "snapshot_id", snapshotID, "error", err)
} }
err = v.deleteRemoteSnapshotByKey(snapshot.RemoteSnapshotKey(snapshotID)) _, err = v.deleteRemoteSnapshotByKey(snapshot.RemoteSnapshotKey(snapshotID))
if err != nil { if err != nil {
return fmt.Errorf("deleting snapshot %s from remote: %w", snapshotID, err) return fmt.Errorf("deleting snapshot %s from remote: %w", snapshotID, err)
} }
@@ -1153,6 +1180,11 @@ func (v *Vaultik) RemoveSnapshot(
return result, err return result, err
} }
inLocalIndex, err := v.findSnapshotToRemove(snapshotID, opts)
if err != nil {
return result, err
}
if opts.DryRun { if opts.DryRun {
return v.removeSnapshotDryRun(result, snapshotID, opts) return v.removeSnapshotDryRun(result, snapshotID, opts)
} }
@@ -1161,11 +1193,13 @@ func (v *Vaultik) RemoveSnapshot(
return result, nil return result, nil
} }
log.Info("Removing snapshot from local database", "snapshot_id", snapshotID) if inLocalIndex {
log.Info("Removing snapshot from local database", "snapshot_id", snapshotID)
err = v.deleteSnapshotFromLocalDB(snapshotID) err = v.deleteSnapshotFromLocalDB(snapshotID)
if err != nil { if err != nil {
return result, fmt.Errorf("removing from local database: %w", err) return result, fmt.Errorf("removing from local database: %w", err)
}
} }
if !opts.LocalOnly { if !opts.LocalOnly {
@@ -1184,9 +1218,11 @@ func (v *Vaultik) RemoveSnapshot(
return result, v.outputRemoveJSON(result) return result, v.outputRemoveJSON(result)
} }
v.stdoutf("Removed snapshot '%s' from local database\n", snapshotID) if inLocalIndex {
v.stdoutf("Removed snapshot '%s' from local database\n", snapshotID)
}
if !opts.LocalOnly && result.RemoteRemoved { if result.RemoteRemoved {
v.printlnStdout("Removed snapshot metadata from remote storage") v.printlnStdout("Removed snapshot metadata from remote storage")
v.stdoutf("\nNote: The removed snapshot's blobs remain on the remote. "+ v.stdoutf("\nNote: The removed snapshot's blobs remain on the remote. "+
"Run '%s' to delete any blobs no longer referenced by any "+ "Run '%s' to delete any blobs no longer referenced by any "+
@@ -1196,6 +1232,45 @@ func (v *Vaultik) RemoveSnapshot(
return result, nil return result, nil
} }
// findSnapshotToRemove reports whether the local index has a row for
// snapshotID, and returns errSnapshotNotFound when there is nothing to
// remove: no row and, unless LocalOnly is set, no metadata on the
// destination store either. The destination store is consulted only
// when the local index has no row, so a snapshot in the local index can
// still be removed from it while the destination store is unreachable.
func (v *Vaultik) findSnapshotToRemove(
snapshotID string, opts *RemoveOptions,
) (bool, error) {
if v.Repositories != nil {
snap, err := v.Repositories.Snapshots.GetByID(v.ctx, snapshotID)
if err != nil {
return false, fmt.Errorf("looking up snapshot in local index: %w", err)
}
if snap != nil {
return true, nil
}
}
if opts.LocalOnly {
return false, fmt.Errorf("%w in the local index: %s",
errSnapshotNotFound, snapshotID)
}
objects, err := v.remoteSnapshotObjects(snapshot.RemoteSnapshotKey(snapshotID))
if err != nil {
return false, fmt.Errorf("looking up snapshot on destination store: %w", err)
}
if len(objects) == 0 {
return false, fmt.Errorf(
"%w in the local index or on the destination store: %s",
errSnapshotNotFound, snapshotID)
}
return false, nil
}
// removeSnapshotDryRun reports what RemoveSnapshot would do without // removeSnapshotDryRun reports what RemoveSnapshot would do without
// making changes. // making changes.
func (v *Vaultik) removeSnapshotDryRun( func (v *Vaultik) removeSnapshotDryRun(
@@ -1255,14 +1330,14 @@ func (v *Vaultik) confirmRemoveSnapshot(snapshotID string, opts *RemoveOptions)
// didn't finish and to run `vaultik snapshot remove` for the snapshot // didn't finish and to run `vaultik snapshot remove` for the snapshot
// again once the destination store is reachable (`vaultik prune` never // again once the destination store is reachable (`vaultik prune` never
// removes snapshot metadata). Returns true when the remote removal // removes snapshot metadata). Returns true when the remote removal
// succeeded. // succeeded and deleted something.
func (v *Vaultik) removeSnapshotRemote(snapshotID string, opts *RemoveOptions) bool { func (v *Vaultik) removeSnapshotRemote(snapshotID string, opts *RemoveOptions) bool {
log.Info("Removing snapshot metadata from remote storage", log.Info("Removing snapshot metadata from remote storage",
"snapshot_id", snapshotID) "snapshot_id", snapshotID)
remoteKey := snapshot.RemoteSnapshotKey(snapshotID) remoteKey := snapshot.RemoteSnapshotKey(snapshotID)
err := v.deleteRemoteSnapshotByKey(remoteKey) deleted, err := v.deleteRemoteSnapshotByKey(remoteKey)
if err != nil { if err != nil {
log.Warn("Could not remove snapshot metadata from remote storage; "+ log.Warn("Could not remove snapshot metadata from remote storage; "+
"run '"+snapshotRemoveCommandHint+"' with the snapshot's ID "+ "run '"+snapshotRemoveCommandHint+"' with the snapshot's ID "+
@@ -1280,7 +1355,7 @@ func (v *Vaultik) removeSnapshotRemote(snapshotID string, opts *RemoveOptions) b
return false return false
} }
return true return deleted > 0
} }
// RemoveAllSnapshots removes every snapshot known to the local // RemoveAllSnapshots removes every snapshot known to the local
@@ -1572,7 +1647,7 @@ func (v *Vaultik) removeAllLocalSnapshots(
} }
if !opts.LocalOnly { if !opts.LocalOnly {
err = v.deleteRemoteSnapshotByKey( _, err = v.deleteRemoteSnapshotByKey(
snapshot.RemoteSnapshotKey(snapshotID)) snapshot.RemoteSnapshotKey(snapshotID))
if err != nil { if err != nil {
log.Warn("Failed to remove snapshot metadata from remote", log.Warn("Failed to remove snapshot metadata from remote",
@@ -1598,7 +1673,7 @@ func (v *Vaultik) removeAllOrphanRemotes(
for _, key := range orphanRemoteKeys { for _, key := range orphanRemoteKeys {
log.Info("Removing orphan remote snapshot", "remote_key", key) log.Info("Removing orphan remote snapshot", "remote_key", key)
err := v.deleteRemoteSnapshotByKey(key) _, err := v.deleteRemoteSnapshotByKey(key)
if err != nil { if err != nil {
log.Warn("Failed to remove orphan from remote", log.Warn("Failed to remove orphan from remote",
"remote_key", key, "error", err) "remote_key", key, "error", err)
@@ -1645,34 +1720,44 @@ func (v *Vaultik) deleteSnapshotFromLocalDB(snapshotID string) error {
} }
// deleteRemoteSnapshotByKey removes everything under // deleteRemoteSnapshotByKey removes everything under
// metadata/<remoteKey>/ on the destination store. The argument is a // metadata/<remoteKey>/ on the destination store and returns how many
// remote key (double-SHA256 derivation), not a human snapshot ID; // objects it deleted. The argument is a remote key (double-SHA256
// callers that have a human ID must hash via snapshot.RemoteSnapshotKey // derivation), not a human snapshot ID; callers that have a human ID
// first. // must hash via snapshot.RemoteSnapshotKey first.
func (v *Vaultik) deleteRemoteSnapshotByKey(remoteKey string) error { func (v *Vaultik) deleteRemoteSnapshotByKey(remoteKey string) (int, error) {
prefix := fmt.Sprintf("metadata/%s/", remoteKey) objectsToDelete, err := v.remoteSnapshotObjects(remoteKey)
objectCh := v.Storage.ListStream(v.ctx, prefix) if err != nil {
return 0, err
var objectsToDelete []string
for object := range objectCh {
if object.Err != nil {
return fmt.Errorf("listing objects: %w", object.Err)
}
objectsToDelete = append(objectsToDelete, object.Key)
} }
for _, key := range objectsToDelete { for i, key := range objectsToDelete {
err := v.Storage.Delete(v.ctx, key) err = v.Storage.Delete(v.ctx, key)
if err != nil { if err != nil {
return fmt.Errorf("removing %s: %w", key, err) return i, fmt.Errorf("removing %s: %w", key, err)
} }
log.Debug("Deleted remote object", "key", key) log.Debug("Deleted remote object", "key", key)
} }
return nil return len(objectsToDelete), nil
}
// remoteSnapshotObjects lists the keys of every object under
// metadata/<remoteKey>/ on the destination store.
func (v *Vaultik) remoteSnapshotObjects(remoteKey string) ([]string, error) {
prefix := fmt.Sprintf("metadata/%s/", remoteKey)
var keys []string
for object := range v.Storage.ListStream(v.ctx, prefix) {
if object.Err != nil {
return nil, fmt.Errorf("listing objects: %w", object.Err)
}
keys = append(keys, object.Key)
}
return keys, nil
} }
// outputRemoveJSON outputs the removal result as JSON // outputRemoveJSON outputs the removal result as JSON