Compare commits

..
Author SHA1 Message Date
sneak c0c79cd0a2 Make prune and remote nuke fail when a delete fails (closes #290)
check / check (push) Waiting to run
prune, and the prune step of snapshot create --prune, counted a blob
it could not delete and still exited 0. It now prints its summary or
--json document and then returns an error, which replaces the
"Failed to delete N blob(s)" line.

remote nuke only warned when a snapshot's metadata could not be
deleted, advised vaultik prune, which never removes metadata, and then
reported the store empty. It now stops with an error, does not report
the store empty, and says to run vaultik remote nuke --force again. It
checks that the local index belongs to the configured destination
before deleting anything, and that error carries no such advice. A
successful nuke no longer advises prune.

Model: opus-5-5
2026-10-13 05:44:12 +00:00
12 changed files with 294 additions and 138 deletions
+7 -7
View File
@@ -384,10 +384,12 @@ manifest on the destination store to compute the still-referenced blob
set and delete any blob not in that set. Step (3) reads all remote set and delete any blob not in that set. Step (3) reads all remote
manifests — network cost scales with the number of snapshots. `snapshot manifests — network cost scales with the number of snapshots. `snapshot
create --prune` runs the same cleanup automatically; this is the create --prune` runs the same cleanup automatically; this is the
manual entry point for the same work. manual entry point for the same work. If a blob cannot be deleted, the
command (and `snapshot create --prune`) exits non-zero.
* `--force`: Skip confirmation prompt * `--force`: Skip confirmation prompt
* `--json`: Output stats as JSON. Also skips the confirmation prompt, as * `--json`: Output stats as JSON. Also skips the confirmation prompt, as
`--force` does. `--force` does. A blob that cannot be deleted is counted in
`blobs_failed`, and the document is printed before the non-zero exit.
**`info`**: Display system configuration, storage settings, encryption **`info`**: Display system configuration, storage settings, encryption
recipients, and local database statistics. recipients, and local database statistics.
@@ -407,7 +409,8 @@ snapshot's blob count and blob size unknown, `null` in `--json`.
**`remote nuke`**: Delete every snapshot's metadata and every blob from the **`remote nuke`**: Delete every snapshot's metadata and every blob from the
backup destination store, leaving the bucket prefix empty. Destructive and backup destination store, leaving the bucket prefix empty. Destructive and
irreversible. This is the single supported way to wipe the entire irreversible. This is the single supported way to wipe the entire
destination store. destination store. If anything cannot be deleted, the command exits
non-zero; running it again deletes what is left.
* `--force`: Required to confirm destruction. * `--force`: Required to confirm destruction.
**`database delete`**: Delete the local SQLite state database file **`database delete`**: Delete the local SQLite state database file
@@ -429,10 +432,7 @@ object store. Credentials are read from `s3.access_key_id` and
`s3.secret_access_key` in the config file. `s3.secret_access_key` in the config file.
**Local filesystem** (`file:///path/to/backup`): Stores blobs and metadata on **Local filesystem** (`file:///path/to/backup`): Stores blobs and metadata on
a local or mounted filesystem. Useful for testing or backing up to a NAS. The a local or mounted filesystem. Useful for testing or backing up to a NAS.
path must be absolute; a relative one is refused when the config loads.
Percent-escapes in the path are decoded, so `my%20stick` and `my stick` name
the same directory.
**Rclone** (`rclone://remote/path`): Uses rclone's 70+ supported cloud **Rclone** (`rclone://remote/path`): Uses rclone's 70+ supported cloud
providers. Requires rclone to be configured separately (`rclone config`). providers. Requires rclone to be configured separately (`rclone config`).
+10 -10
View File
@@ -22,16 +22,16 @@ the tag exists and is exercised; what is left is merging `next` to
# Completed Steps # Completed Steps
- 2026-10-13: Refused a `file://` `storage_url` whose path is relative - 2026-10-13: Made `prune` and `remote nuke` exit non-zero when a delete
([issue #299](https://git.eeqj.de/sneak/vaultik/issues/299)). The text fails ([issue #290](https://git.eeqj.de/sneak/vaultik/issues/290)).
after `file://` was used as written, so `file://dest` resolved against `prune`, and the prune step of `snapshot create --prune`, counted a
the working directory: a backup run from another directory wrote to a blob it could not delete and still exited 0; it now prints its summary
different destination, still exited 0, and left a snapshot that could or `--json` document and then fails. `remote nuke` warned about a
not be restored. Such a URL now fails when the config loads, with an snapshot's metadata it could not delete, said to run `vaultik prune`,
error asking for an absolute path. Percent-escapes in the path are now which never removes metadata, and then reported the store empty. It
decoded, so `my%20stick` names the directory `my stick` instead of now fails without that report and says to run `vaultik remote nuke
creating one named `my%20stick`; a path written with a literal space --force` again, and a successful nuke no longer advises `prune` for
works as before. blobs it deletes itself.
- 2026-10-13: Made `snapshot remove` fail for a snapshot that does not - 2026-10-13: Made `snapshot remove` fail for a snapshot that does not
exist ([issue #293](https://git.eeqj.de/sneak/vaultik/issues/293)). exist ([issue #293](https://git.eeqj.de/sneak/vaultik/issues/293)).
+2 -12
View File
@@ -61,9 +61,6 @@ var (
errBadS3PartSize = errors.New("s3.part_size must be between 5MiB and 5GiB") errBadS3PartSize = errors.New("s3.part_size must be between 5MiB and 5GiB")
errBadStorageScheme = errors.New( errBadStorageScheme = errors.New(
"storage_url must start with s3://, file://, or rclone://") "storage_url must start with s3://, file://, or rclone://")
errFileURLNotAbsolute = errors.New(
"a file:// storage_url needs an absolute path, " +
"as in file:///path/to/backup")
errStorageNotConfigured = errors.New( errStorageNotConfigured = errors.New(
"storage not configured; set storage_url or provide s3.endpoint + " + "storage not configured; set storage_url or provide s3.endpoint + " +
"s3.bucket + credentials") "s3.bucket + credentials")
@@ -464,18 +461,11 @@ func (c *Config) validateStorage() error {
// validateStorageURL validates URL-based storage configuration. File and // validateStorageURL validates URL-based storage configuration. File and
// rclone URLs need no credentials; S3 URLs require the legacy s3.* // rclone URLs need no credentials; S3 URLs require the legacy s3.*
// credential fields. A file URL must name an absolute path: a relative one // credential fields.
// would resolve against the working directory, so the destination would
// move with it.
func (c *Config) validateStorageURL() error { func (c *Config) validateStorageURL() error {
switch { switch {
case strings.HasPrefix(c.StorageURL, "file://"): case strings.HasPrefix(c.StorageURL, "file://"):
// storage.ParseStorageURL applies the same rule, but config // File storage doesn't need S3 credentials
// cannot import storage, which imports config.
if !strings.HasPrefix(c.StorageURL, "file:///") {
return errFileURLNotAbsolute
}
return nil return nil
case strings.HasPrefix(c.StorageURL, "rclone://"): case strings.HasPrefix(c.StorageURL, "rclone://"):
// Rclone storage uses rclone's own config // Rclone storage uses rclone's own config
-29
View File
@@ -337,35 +337,6 @@ func TestLoadS3PartSize(t *testing.T) {
}) })
} }
// TestLoadRejectsRelativeFileURL checks that a file:// storage_url with a
// relative path fails at load. Accepted, it would resolve against the
// working directory, so a backup run from another directory would write
// to a different destination and still succeed.
func TestLoadRejectsRelativeFileURL(t *testing.T) {
t.Parallel()
const relativeConfig = "snapshots:\n" +
" test:\n" +
" paths: [/tmp/vaultik-test-source]\n" +
"storage_url: file://dest\n"
path := filepath.Join(t.TempDir(), "config.yml")
err := os.WriteFile(path, []byte(relativeConfig), 0o600)
if err != nil {
t.Fatalf("write config: %v", err)
}
_, err = Load(path)
if !errors.Is(err, errFileURLNotAbsolute) {
t.Fatalf("Load() error = %v, want errFileURLNotAbsolute", err)
}
if !strings.Contains(err.Error(), "absolute path") {
t.Errorf("error %q does not say to use an absolute path", err.Error())
}
}
// TestValidateAgeRecipients checks that recipients are parsed at config load // TestValidateAgeRecipients checks that recipients are parsed at config load
// (a bad entry fails immediately, not mid-backup) and that no invalid entry — // (a bad entry fails immediately, not mid-backup) and that no invalid entry —
// least of all a pasted secret key — is echoed in the error. An empty list // least of all a pasted secret key — is echoed in the error. An empty list
+23 -9
View File
@@ -1,15 +1,16 @@
// Package faultstore provides a storage.Storer wrapper that injects // Package faultstore provides a storage.Storer wrapper that injects
// faults on demand, so tests can reproduce the failure modes a real // faults on demand, so tests can reproduce the failure modes a real
// backend exhibits: an upload that fails partway, a backend that reports // backend exhibits: an upload that fails partway, a backend that reports
// success while storing nothing, and reads that return corrupt or // success while storing nothing, reads that return corrupt or truncated
// truncated bytes. It is the seam called for by the fault-injection // bytes, and a delete the backend refuses. It is the seam called for by
// tests (sneak/vaultik issue 72) and is meant to be reused by future // the fault-injection tests (sneak/vaultik issue 72) and is meant to be
// tests rather than re-implemented per case. // reused by future tests rather than re-implemented per case.
// //
// The wrapper delegates every method to the inner Storer. Two hooks // The wrapper delegates every method to the inner Storer. Three hooks
// change that: OnPut decides the fate of each write, and OnGet decides // change that: OnPut decides the fate of each write, OnGet decides how
// how each read's bytes are returned. Both are keyed by the object key, // each read's bytes are returned, and OnDelete decides whether each
// so a test can fault only blobs, only metadata, or a single object. // delete fails. All are keyed by the object key, so a test can fault
// only blobs, only metadata, or a single object.
package faultstore package faultstore
import ( import (
@@ -25,6 +26,10 @@ import (
// ErrInjectedUpload is returned by a Put the OnPut hook chose to fail. // ErrInjectedUpload is returned by a Put the OnPut hook chose to fail.
var ErrInjectedUpload = errors.New("faultstore: injected upload failure") var ErrInjectedUpload = errors.New("faultstore: injected upload failure")
// ErrInjectedDelete is returned by a Delete the OnDelete hook chose to
// fail.
var ErrInjectedDelete = errors.New("faultstore: injected delete failure")
// PutAction is the disposition OnPut assigns to a write. // PutAction is the disposition OnPut assigns to a write.
type PutAction int type PutAction int
@@ -66,6 +71,11 @@ type Storer struct {
// OnGet, when set, is consulted for every Get with the object key // OnGet, when set, is consulted for every Get with the object key
// and damages the returned bytes accordingly. // and damages the returned bytes accordingly.
OnGet func(key string) GetFault OnGet func(key string) GetFault
// OnDelete, when set, is consulted before every Delete with the
// object key. Returning true fails the delete and leaves the object
// in place.
OnDelete func(key string) bool
} }
// New wraps inner. inner must be non-nil. // New wraps inner. inner must be non-nil.
@@ -154,8 +164,12 @@ func (f *Storer) Stat(ctx context.Context, key string) (*storage.ObjectInfo, err
return f.inner.Stat(ctx, key) return f.inner.Stat(ctx, key)
} }
// Delete delegates unchanged. // Delete removes the object unless OnDelete fails the delete.
func (f *Storer) Delete(ctx context.Context, key string) error { func (f *Storer) Delete(ctx context.Context, key string) error {
if f.OnDelete != nil && f.OnDelete(key) {
return fmt.Errorf("%w for %q", ErrInjectedDelete, key)
}
return f.inner.Delete(ctx, key) return f.inner.Delete(ctx, key)
} }
+12 -26
View File
@@ -17,10 +17,8 @@ const (
// Sentinel errors for storage URL parsing. // Sentinel errors for storage URL parsing.
var ( var (
ErrEmptyStorageURL = errors.New("storage URL is empty") ErrEmptyStorageURL = errors.New("storage URL is empty")
ErrEmptyFilePath = errors.New("file URL path is empty") ErrEmptyFilePath = errors.New("file URL path is empty")
ErrRelativeFilePath = errors.New(
"file URL needs an absolute path, as in file:///path/to/backup")
ErrMissingBucket = errors.New("s3 URL missing bucket name") ErrMissingBucket = errors.New("s3 URL missing bucket name")
ErrMissingRemote = errors.New("rclone URL missing remote name") ErrMissingRemote = errors.New("rclone URL missing remote name")
ErrUnsupportedScheme = errors.New( ErrUnsupportedScheme = errors.New(
@@ -53,8 +51,17 @@ func ParseStorageURL(rawURL string) (*URL, error) {
return nil, ErrEmptyStorageURL return nil, ErrEmptyStorageURL
} }
// Handle file:// URLs
if after, ok := strings.CutPrefix(rawURL, "file://"); ok { if after, ok := strings.CutPrefix(rawURL, "file://"); ok {
return parseFileURL(after) path := after
if path == "" {
return nil, ErrEmptyFilePath
}
return &URL{
Scheme: schemeFile,
Prefix: path,
}, nil
} }
if strings.HasPrefix(rawURL, "s3://") { if strings.HasPrefix(rawURL, "s3://") {
@@ -68,27 +75,6 @@ func ParseStorageURL(rawURL string) (*URL, error) {
return nil, ErrUnsupportedScheme return nil, ErrUnsupportedScheme
} }
// parseFileURL parses the path that follows file://. The path must be
// absolute: a relative one would resolve against the working directory,
// so the destination would move with it. Percent-escapes are decoded, so
// my%20stick names the directory "my stick"; a literal space is kept.
func parseFileURL(rawPath string) (*URL, error) {
if rawPath == "" {
return nil, ErrEmptyFilePath
}
if !strings.HasPrefix(rawPath, "/") {
return nil, ErrRelativeFilePath
}
path, err := url.PathUnescape(rawPath)
if err != nil {
return nil, fmt.Errorf("invalid file URL path: %w", err)
}
return &URL{Scheme: schemeFile, Prefix: path}, nil
}
// parseS3URL parses an s3://bucket/prefix URL. It rejects credentials in // parseS3URL parses an s3://bucket/prefix URL. It rejects credentials in
// the userinfo and any query parameter other than endpoint, region and // the userinfo and any query parameter other than endpoint, region and
// ssl, so a credential-bearing URL is never stored or echoed. // ssl, so a credential-bearing URL is never stored or echoed.
+2 -16
View File
@@ -14,10 +14,7 @@ import (
func TestParseStorageURLValid(t *testing.T) { func TestParseStorageURLValid(t *testing.T) {
t.Parallel() t.Parallel()
const ( const bucket = "mybucket"
bucket = "mybucket"
fileScheme = "file"
)
cases := []struct { cases := []struct {
name string name string
@@ -27,17 +24,7 @@ func TestParseStorageURLValid(t *testing.T) {
{ {
name: "file absolute path", name: "file absolute path",
raw: "file:///var/backups/vaultik", raw: "file:///var/backups/vaultik",
want: &storage.URL{Scheme: fileScheme, Prefix: "/var/backups/vaultik"}, want: &storage.URL{Scheme: "file", Prefix: "/var/backups/vaultik"},
},
{
name: "file percent-escaped path is decoded",
raw: "file:///Volumes/my%20stick/backup",
want: &storage.URL{Scheme: fileScheme, Prefix: "/Volumes/my stick/backup"},
},
{
name: "file path with a literal space",
raw: "file:///Volumes/my stick/backup",
want: &storage.URL{Scheme: fileScheme, Prefix: "/Volumes/my stick/backup"},
}, },
{ {
name: "s3 bucket and prefix, ssl defaults on", name: "s3 bucket and prefix, ssl defaults on",
@@ -103,7 +90,6 @@ func TestParseStorageURLErrors(t *testing.T) {
}{ }{
{"empty url", "", storage.ErrEmptyStorageURL}, {"empty url", "", storage.ErrEmptyStorageURL},
{"file empty path", "file://", storage.ErrEmptyFilePath}, {"file empty path", "file://", storage.ErrEmptyFilePath},
{"file relative path", "file://dest", storage.ErrRelativeFilePath},
{"s3 missing bucket", "s3://", storage.ErrMissingBucket}, {"s3 missing bucket", "s3://", storage.ErrMissingBucket},
{"s3 missing bucket with path", "s3:///justprefix", storage.ErrMissingBucket}, {"s3 missing bucket with path", "s3:///justprefix", storage.ErrMissingBucket},
{"rclone missing remote", "rclone://", storage.ErrMissingRemote}, {"rclone missing remote", "rclone://", storage.ErrMissingRemote},
+81 -1
View File
@@ -5,12 +5,14 @@ import (
"io/fs" "io/fs"
"os" "os"
"path/filepath" "path/filepath"
"strings"
"testing" "testing"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/log" "sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/snapshot" "sneak.berlin/go/vaultik/internal/snapshot"
"sneak.berlin/go/vaultik/internal/storage/faultstore"
) )
// TestNukeRemoteLeavesNoFiles checks that remote nuke leaves no file // TestNukeRemoteLeavesNoFiles checks that remote nuke leaves no file
@@ -23,7 +25,7 @@ func TestNukeRemoteLeavesNoFiles(t *testing.T) {
ctx := context.Background() ctx := context.Background()
storeDir := filepath.Join(t.TempDir(), "store") storeDir := filepath.Join(t.TempDir(), "store")
v, repos, _ := backUpToFileDestination(ctx, t, storeDir) v, repos, out := backUpToFileDestination(ctx, t, storeDir)
snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit) snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
require.NoError(t, err) require.NoError(t, err)
@@ -61,4 +63,82 @@ func TestNukeRemoteLeavesNoFiles(t *testing.T) {
}) })
require.NoError(t, err) require.NoError(t, err)
assert.Empty(t, files) assert.Empty(t, files)
// Nuke deletes the blobs itself, so advice to run prune is wrong.
assert.Contains(t, out.String(), "Backup destination store is now empty.")
assert.NotContains(t, out.String(), "vaultik prune")
}
// TestNukeRemoteFailsWhenADeleteFails checks that remote nuke returns an
// error when a snapshot's metadata or a blob cannot be deleted, does not
// report the store empty, and advises running nuke again: prune deletes
// no metadata.
func TestNukeRemoteFailsWhenADeleteFails(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
for _, refusedPrefix := range []string{"metadata/", "blobs/"} {
t.Run(refusedPrefix, func(t *testing.T) {
t.Parallel()
ctx := context.Background()
storeDir := filepath.Join(t.TempDir(), "store")
v, _, out := backUpToFileDestination(ctx, t, storeDir)
store := faultstore.New(v.Storage)
store.OnDelete = func(key string) bool {
return strings.HasPrefix(key, refusedPrefix)
}
v.Storage = store
require.Error(t, v.NukeRemote(true))
assert.NotContains(t, out.String(), "now empty")
assert.Contains(t, out.String(), "vaultik remote nuke --force")
assert.NotContains(t, out.String(), "vaultik prune")
})
}
}
// TestNukeRemoteRefusesAnotherDestinationWithoutRetryAdvice checks that
// remote nuke, when the local index belongs to a different destination
// than the configured one, fails without advising to run nuke again:
// that run would fail the same way.
func TestNukeRemoteRefusesAnotherDestinationWithoutRetryAdvice(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
storeDir := filepath.Join(t.TempDir(), "store")
v, _, out := backUpToFileDestination(ctx, t, storeDir)
v.Config.StorageURL = "file://" + filepath.Join(t.TempDir(), "elsewhere")
err := v.NukeRemote(true)
require.ErrorContains(t, err, "bound to a different backup destination")
assert.NotContains(t, out.String(), "vaultik remote nuke --force")
}
// TestNukeRemoteFailsWhenALocalSnapshotCannotBeRemoved checks that a
// snapshot remote nuke cannot remove from the local index fails the
// run: nuke then leaves that snapshot's metadata on the destination
// store.
func TestNukeRemoteFailsWhenALocalSnapshotCannotBeRemoved(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
storeDir := filepath.Join(t.TempDir(), "store")
v, _, out := backUpToFileDestination(ctx, t, storeDir)
_, err := v.DB.Conn().ExecContext(ctx, `
CREATE TRIGGER refuse_snapshot_delete BEFORE DELETE ON snapshots
BEGIN SELECT RAISE(ABORT, 'simulated index error'); END`)
require.NoError(t, err)
require.Error(t, v.NukeRemote(true))
assert.NotContains(t, out.String(), "now empty")
assert.Contains(t, out.String(), "vaultik remote nuke --force")
} }
+39 -7
View File
@@ -19,6 +19,14 @@ type PruneOptions struct {
var errNukeRequiresForce = errors.New( var errNukeRequiresForce = errors.New(
"nuke requires --force (this deletes ALL remote snapshots and blobs)") "nuke requires --force (this deletes ALL remote snapshots and blobs)")
// errBlobDeleteFailed reports unreferenced blobs prune could not delete.
var errBlobDeleteFailed = errors.New("blob(s) could not be deleted")
// nukeCommandHint is the command suggested when a remote nuke could not
// delete everything: running it again deletes what is left, which
// `vaultik prune` does not do for snapshot metadata.
const nukeCommandHint = "vaultik remote nuke --force"
// metadataDirName is the top-level remote directory holding snapshot // metadataDirName is the top-level remote directory holding snapshot
// metadata. // metadata.
const metadataDirName = "metadata" const metadataDirName = "metadata"
@@ -35,6 +43,29 @@ func (v *Vaultik) NukeRemote(force bool) error {
return errNukeRequiresForce return errNukeRequiresForce
} }
// Checked before anything is deleted: running nuke again would fail
// the same way, so the advice below must not follow this error.
err := v.EnsureStorageBinding()
if err != nil {
return err
}
err = v.emptyDestinationStore()
if err != nil {
v.UI.Warningf("Run '%s' again to delete what is left on the "+
"backup destination store.", nukeCommandHint)
return err
}
v.UI.Completef("Backup destination store is now empty.")
return nil
}
// emptyDestinationStore does the deleting for NukeRemote and stops at
// the first step that fails.
func (v *Vaultik) emptyDestinationStore() error {
v.UI.Beginf("Removing all snapshot metadata from backup destination store.") v.UI.Beginf("Removing all snapshot metadata from backup destination store.")
_, err := v.RemoveAllSnapshots(&RemoveOptions{Force: true}) _, err := v.RemoveAllSnapshots(&RemoveOptions{Force: true})
@@ -58,8 +89,6 @@ func (v *Vaultik) NukeRemote(force bool) error {
} }
} }
v.UI.Completef("Backup destination store is now empty.")
return nil return nil
} }
@@ -162,14 +191,17 @@ func (v *Vaultik) PruneBlobs(opts *PruneOptions) error {
v.deleteUnreferencedBlobs(unreferencedBlobs, allBlobs, result) v.deleteUnreferencedBlobs(unreferencedBlobs, allBlobs, result)
if opts.JSON { if opts.JSON {
return v.outputPruneBlobsJSON(result) err = v.outputPruneBlobsJSON(result)
if err != nil {
return err
}
} else {
v.stdoutf("\nDeleted %d blob(s) totaling %s\n",
result.BlobsDeleted, ubytes(result.BytesFreed))
} }
v.stdoutf("\nDeleted %d blob(s) totaling %s\n",
result.BlobsDeleted, ubytes(result.BytesFreed))
if result.BlobsFailed > 0 { if result.BlobsFailed > 0 {
v.stdoutf("Failed to delete %d blob(s)\n", result.BlobsFailed) return fmt.Errorf("%d %w", result.BlobsFailed, errBlobDeleteFailed)
} }
return nil return nil
@@ -0,0 +1,98 @@
package vaultik_test
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/storage/faultstore"
"sneak.berlin/go/vaultik/internal/vaultik"
)
// refuseBlobDeletes is an OnDelete hook that fails every blob delete.
func refuseBlobDeletes(key string) bool {
return strings.HasPrefix(key, "blobs/")
}
// TestPruneBlobs_FailedDeleteReturnsError checks that prune returns an
// error, so the command exits non-zero, when an unreferenced blob cannot
// be deleted.
func TestPruneBlobs_FailedDeleteReturnsError(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newListEnv(t)
// The store holds no manifest, so nothing references this blob.
addBlob(t, env.store.testStorer, testBlobHashA)
store := faultstore.New(env.store)
store.OnDelete = refuseBlobDeletes
env.v.Storage = store
err := env.v.PruneBlobs(&vaultik.PruneOptions{Force: true})
require.ErrorContains(t, err, "1 blob(s) could not be deleted")
}
// TestPruneBlobs_JSONFailedDeleteReturnsError checks that prune --json
// prints its document and then returns an error when an unreferenced
// blob cannot be deleted.
func TestPruneBlobs_JSONFailedDeleteReturnsError(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newListEnv(t)
addBlob(t, env.store.testStorer, testBlobHashA)
store := faultstore.New(env.store)
store.OnDelete = refuseBlobDeletes
env.v.Storage = store
err := env.v.PruneBlobs(&vaultik.PruneOptions{JSON: true})
require.Error(t, err, "prune must fail when a blob cannot be deleted")
var result vaultik.PruneBlobsResult
require.NoError(t, json.Unmarshal(env.stdout.Bytes(), &result),
"stdout must hold the JSON document, got:\n%s", env.stdout.String())
assert.Equal(t, 1, result.BlobsFailed)
assert.Equal(t, 0, result.BlobsDeleted)
}
// TestCreateSnapshotPrune_FailedBlobDeleteReturnsError checks that the
// prune step of snapshot create --prune fails the run when an
// unreferenced blob cannot be deleted.
func TestCreateSnapshotPrune_FailedBlobDeleteReturnsError(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
storeDir := filepath.Join(t.TempDir(), "store")
v, _, _ := backUpToFileDestination(ctx, t, storeDir)
// No snapshot references this blob, so the prune step deletes it.
orphan := filepath.Join(storeDir, "blobs", testBlobHashA[:2],
testBlobHashA[2:4], testBlobHashA)
require.NoError(t, os.MkdirAll(filepath.Dir(orphan), 0o750))
require.NoError(t, os.WriteFile(orphan, []byte("orphan"), 0o600))
store := faultstore.New(v.Storage)
store.OnDelete = refuseBlobDeletes
v.Storage = store
err := v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
Cron: true,
Prune: true,
})
require.ErrorContains(t, err, "1 blob(s) could not be deleted")
assert.FileExists(t, orphan)
}
+2 -1
View File
@@ -476,7 +476,8 @@ func TestRemoveAllSnapshots_WithForce(t *testing.T) {
out := tv.Stdout.String() out := tv.Stdout.String()
assert.Contains(t, out, "Removed 2 snapshot(s)") assert.Contains(t, out, "Removed 2 snapshot(s)")
assert.Contains(t, out, "Removed snapshot metadata from remote storage") assert.Contains(t, out, "Removed snapshot metadata from remote storage")
assert.Contains(t, out, "vaultik prune") // Remote nuke, the only caller, deletes the blobs next itself.
assert.NotContains(t, out, "vaultik prune")
} }
func TestRemoveAllSnapshots_DryRun(t *testing.T) { func TestRemoveAllSnapshots_DryRun(t *testing.T) {
+18 -20
View File
@@ -24,6 +24,7 @@ var (
errSnapshotVerifyFailed = errors.New("verification failed") errSnapshotVerifyFailed = errors.New("verification failed")
errRemoveAllNeedsForce = errors.New("--all requires --force") errRemoveAllNeedsForce = errors.New("--all requires --force")
errSnapshotNotFound = errors.New("snapshot not found") errSnapshotNotFound = errors.New("snapshot not found")
errSnapshotRemoveFailed = errors.New("snapshot(s) could not be removed")
errInvalidTableName = errors.New("invalid table name") errInvalidTableName = errors.New("invalid table name")
errNoAgeRecipients = errors.New( errNoAgeRecipients = errors.New(
"creating a snapshot needs at least one public key in " + "creating a snapshot needs at least one public key in " +
@@ -1513,8 +1514,8 @@ func (v *Vaultik) handleRemoveAllDryRun(
// executeRemoveAll deletes every local snapshot and, unless LocalOnly // executeRemoveAll deletes every local snapshot and, unless LocalOnly
// is set, every corresponding remote metadata directory plus any // is set, every corresponding remote metadata directory plus any
// orphan remote keys. Blobs are not touched; the printed prune-command // orphan remote keys. Blobs are not touched. It returns an error when
// hint is the next step. // any snapshot could not be removed.
func (v *Vaultik) executeRemoveAll( func (v *Vaultik) executeRemoveAll(
localSnaps, orphanRemoteKeys []string, opts *RemoveOptions, localSnaps, orphanRemoteKeys []string, opts *RemoveOptions,
) (*RemoveResult, error) { ) (*RemoveResult, error) {
@@ -1528,18 +1529,11 @@ func (v *Vaultik) executeRemoveAll(
result := &RemoveResult{} result := &RemoveResult{}
remoteErrors := v.removeAllLocalSnapshots(localSnaps, opts, result) failures := v.removeAllLocalSnapshots(localSnaps, opts, result)
if !opts.LocalOnly { if !opts.LocalOnly {
remoteErrors += v.removeAllOrphanRemotes(orphanRemoteKeys, result) failures += v.removeAllOrphanRemotes(orphanRemoteKeys, result)
result.RemoteRemoved = failures == 0
if remoteErrors == 0 {
result.RemoteRemoved = true
} else if v.UI != nil {
v.UI.Warningf("Some remote metadata deletions failed. Run '%s' "+
"once the remote is healthy to clean up unreferenced blobs.",
pruneCommandHint)
}
} }
if v.SnapshotManager != nil { if v.SnapshotManager != nil {
@@ -1549,6 +1543,10 @@ func (v *Vaultik) executeRemoveAll(
} }
} }
if failures > 0 {
return result, fmt.Errorf("%d %w", failures, errSnapshotRemoveFailed)
}
if opts.JSON { if opts.JSON {
return result, v.outputRemoveJSON(result) return result, v.outputRemoveJSON(result)
} }
@@ -1557,9 +1555,6 @@ func (v *Vaultik) executeRemoveAll(
if !opts.LocalOnly && result.RemoteRemoved { if !opts.LocalOnly && result.RemoteRemoved {
v.printlnStdout("Removed snapshot metadata from remote storage") v.printlnStdout("Removed snapshot metadata from remote storage")
v.stdoutf("\nNote: Removed snapshots' blobs remain on the remote. "+
"Run '%s' to delete any blobs no longer referenced by any "+
"remaining remote snapshot.\n", pruneCommandHint)
} }
return result, nil return result, nil
@@ -1602,12 +1597,13 @@ func (v *Vaultik) deleteIncompleteSnapshot(snapshotIDStr string, id any) bool {
} }
// removeAllLocalSnapshots removes each known local snapshot (and, unless // removeAllLocalSnapshots removes each known local snapshot (and, unless
// --local-only, its remote metadata), returning the number of remote // --local-only, its remote metadata), returning the number of snapshots
// deletion failures. // it could not remove. A snapshot left in the local index keeps its
// remote metadata too, since it is not an orphan remote key.
func (v *Vaultik) removeAllLocalSnapshots( func (v *Vaultik) removeAllLocalSnapshots(
localSnaps []string, opts *RemoveOptions, result *RemoveResult, localSnaps []string, opts *RemoveOptions, result *RemoveResult,
) int { ) int {
remoteErrors := 0 failures := 0
for _, snapshotID := range localSnaps { for _, snapshotID := range localSnaps {
log.Info("Removing snapshot", "snapshot_id", snapshotID) log.Info("Removing snapshot", "snapshot_id", snapshotID)
@@ -1617,6 +1613,8 @@ func (v *Vaultik) removeAllLocalSnapshots(
log.Error("Failed to remove from local database", log.Error("Failed to remove from local database",
"snapshot_id", snapshotID, "error", err) "snapshot_id", snapshotID, "error", err)
failures++
continue continue
} }
@@ -1627,14 +1625,14 @@ func (v *Vaultik) removeAllLocalSnapshots(
log.Warn("Failed to remove snapshot metadata from remote", log.Warn("Failed to remove snapshot metadata from remote",
"snapshot_id", snapshotID, "error", err) "snapshot_id", snapshotID, "error", err)
remoteErrors++ failures++
} }
} }
result.SnapshotsRemoved = append(result.SnapshotsRemoved, snapshotID) result.SnapshotsRemoved = append(result.SnapshotsRemoved, snapshotID)
} }
return remoteErrors return failures
} }
// removeAllOrphanRemotes deletes remote snapshot metadata that has no // removeAllOrphanRemotes deletes remote snapshot metadata that has no