Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f172b42d85 |
@@ -349,11 +349,9 @@ the backup destination store. Blobs are NOT touched — deleting blobs
|
||||
requires reading every remaining remote manifest (the destination store
|
||||
may hold snapshots this host doesn't know about), which is what
|
||||
`vaultik prune` does. On success the command prints the exact `vaultik
|
||||
prune` invocation to run as a follow-up. An ID that names no snapshot in
|
||||
the local index or on the destination store is an error; with
|
||||
`--local-only`, the snapshot must be in the local index. Local row
|
||||
cleanup (files, chunks, blobs the snapshot was the last referrer for)
|
||||
runs automatically. If the destination store is unreachable, the local-DB
|
||||
prune` invocation to run as a follow-up. Local row cleanup (files,
|
||||
chunks, blobs the snapshot was the last referrer for) runs
|
||||
automatically. If the destination store is unreachable, the local-DB
|
||||
removal still completes and a warning is emitted; run `vaultik snapshot
|
||||
remove <snapshot-id>` again once the store is reachable to remove the
|
||||
snapshot's metadata from it (`vaultik prune` does not). To wipe everything
|
||||
@@ -384,12 +382,10 @@ manifest on the destination store to compute the still-referenced blob
|
||||
set and delete any blob not in that set. Step (3) reads all remote
|
||||
manifests — network cost scales with the number of snapshots. `snapshot
|
||||
create --prune` runs the same cleanup automatically; this is the
|
||||
manual entry point for the same work. If a blob cannot be deleted, the
|
||||
command (and `snapshot create --prune`) exits non-zero.
|
||||
manual entry point for the same work.
|
||||
* `--force`: Skip confirmation prompt
|
||||
* `--json`: Output stats as JSON. Also skips the confirmation prompt, as
|
||||
`--force` does. A blob that cannot be deleted is counted in
|
||||
`blobs_failed`, and the document is printed before the non-zero exit.
|
||||
`--force` does.
|
||||
|
||||
**`info`**: Display system configuration, storage settings, encryption
|
||||
recipients, and local database statistics.
|
||||
@@ -409,16 +405,14 @@ snapshot's blob count and blob size unknown, `null` in `--json`.
|
||||
**`remote nuke`**: Delete every snapshot's metadata and every blob from the
|
||||
backup destination store, leaving the bucket prefix empty. Destructive and
|
||||
irreversible. This is the single supported way to wipe the entire
|
||||
destination store. If anything cannot be deleted, the command exits
|
||||
non-zero; running it again deletes what is left.
|
||||
destination store.
|
||||
* `--force`: Required to confirm destruction.
|
||||
|
||||
**`database delete`**: Delete the local SQLite state database file
|
||||
entirely. Remote storage is unaffected; the next backup will do a full
|
||||
scan and re-deduplicate against existing remote blobs, and the local
|
||||
index will re-bind to the currently configured storage destination.
|
||||
Use this after changing `storage_url`, or `s3.endpoint`, `s3.bucket` or
|
||||
`s3.prefix`, to a different destination.
|
||||
Use this after changing `storage_url` to a different destination.
|
||||
* `--force`: Skip confirmation prompt
|
||||
|
||||
---
|
||||
|
||||
@@ -22,35 +22,15 @@ the tag exists and is exercised; what is left is merging `next` to
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-13: Made `prune` and `remote nuke` exit non-zero when a delete
|
||||
fails ([issue #290](https://git.eeqj.de/sneak/vaultik/issues/290)).
|
||||
`prune`, and the prune step of `snapshot create --prune`, counted a
|
||||
blob it could not delete and still exited 0; it now prints its summary
|
||||
or `--json` document and then fails. `remote nuke` warned about a
|
||||
snapshot's metadata it could not delete, said to run `vaultik prune`,
|
||||
which never removes metadata, and then reported the store empty. It
|
||||
now fails without that report and says to run `vaultik remote nuke
|
||||
--force` again, and a successful nuke no longer advises `prune` for
|
||||
blobs it deletes itself.
|
||||
|
||||
- 2026-10-13: Made `snapshot remove` fail for a snapshot that does not
|
||||
exist ([issue #293](https://git.eeqj.de/sneak/vaultik/issues/293)).
|
||||
The command never checked the ID: it reported the snapshot removed
|
||||
from the local database and from the destination store and exited 0,
|
||||
under `--json`, `--dry-run` and `--local-only` too. An ID that names
|
||||
no snapshot in the local index or on the destination store is now an
|
||||
error; with `--local-only`, the snapshot must be in the local index.
|
||||
The command now reports a removal from either side only when one
|
||||
happened, and `remote_removed` is true only when metadata was deleted.
|
||||
|
||||
- 2026-10-12: Bound the local index to the destination an `s3.*` config
|
||||
names ([issue #298](https://git.eeqj.de/sneak/vaultik/issues/298)).
|
||||
Only `storage_url` was checked, so after `s3.bucket` or `s3.prefix`
|
||||
changed, the next backup treated every chunk as already stored,
|
||||
uploaded nothing, and completed a snapshot that could not be restored.
|
||||
Without `storage_url`, the local index is now bound to `s3.endpoint`,
|
||||
`s3.bucket` and `s3.prefix`, and a change is refused as a changed
|
||||
`storage_url` is.
|
||||
- 2026-10-12: Made a `file://` destination list through a symlinked
|
||||
directory ([issue #296](https://git.eeqj.de/sneak/vaultik/issues/296)).
|
||||
A `metadata/` or `blobs/` directory that was a symlink to a directory
|
||||
elsewhere was listed as one file, so the destination store listed as
|
||||
empty while backups and restores still worked through the link.
|
||||
`prune` then removed the local snapshot records and deleted every blob,
|
||||
and `remote info` and `remote nuke` found no blobs. The listings now
|
||||
read through the link as `Get` and `Stat` do; a link pointing back up
|
||||
the tree makes them fail.
|
||||
|
||||
- 2026-10-12: Rejected a retention duration longer than about 292 years
|
||||
([issue #289](https://git.eeqj.de/sneak/vaultik/issues/289)). The
|
||||
|
||||
@@ -255,12 +255,8 @@ manifest (the destination store may hold snapshots this host doesn't
|
||||
know about), which is what 'vaultik prune' does. On success the command
|
||||
prints the exact 'vaultik prune' invocation to run as a follow-up.
|
||||
|
||||
An ID that names no snapshot in the local index or on the destination
|
||||
store is an error.
|
||||
|
||||
Use --local-only to skip the remote half (e.g. when you want to forget a
|
||||
snapshot locally without touching the destination store); the snapshot
|
||||
must then be in the local index.
|
||||
snapshot locally without touching the destination store).
|
||||
|
||||
If the remote is unreachable, the local-database removal still completes
|
||||
and a warning is emitted; run 'vaultik snapshot remove <snapshot-id>' again
|
||||
|
||||
@@ -1,16 +1,15 @@
|
||||
// Package faultstore provides a storage.Storer wrapper that injects
|
||||
// faults on demand, so tests can reproduce the failure modes a real
|
||||
// backend exhibits: an upload that fails partway, a backend that reports
|
||||
// success while storing nothing, reads that return corrupt or truncated
|
||||
// bytes, and a delete the backend refuses. It is the seam called for by
|
||||
// the fault-injection tests (sneak/vaultik issue 72) and is meant to be
|
||||
// reused by future tests rather than re-implemented per case.
|
||||
// success while storing nothing, and reads that return corrupt or
|
||||
// truncated bytes. It is the seam called for by the fault-injection
|
||||
// tests (sneak/vaultik issue 72) and is meant to be reused by future
|
||||
// tests rather than re-implemented per case.
|
||||
//
|
||||
// The wrapper delegates every method to the inner Storer. Three hooks
|
||||
// change that: OnPut decides the fate of each write, OnGet decides how
|
||||
// each read's bytes are returned, and OnDelete decides whether each
|
||||
// delete fails. All are keyed by the object key, so a test can fault
|
||||
// only blobs, only metadata, or a single object.
|
||||
// The wrapper delegates every method to the inner Storer. Two hooks
|
||||
// change that: OnPut decides the fate of each write, and OnGet decides
|
||||
// how each read's bytes are returned. Both are keyed by the object key,
|
||||
// so a test can fault only blobs, only metadata, or a single object.
|
||||
package faultstore
|
||||
|
||||
import (
|
||||
@@ -26,10 +25,6 @@ import (
|
||||
// ErrInjectedUpload is returned by a Put the OnPut hook chose to fail.
|
||||
var ErrInjectedUpload = errors.New("faultstore: injected upload failure")
|
||||
|
||||
// ErrInjectedDelete is returned by a Delete the OnDelete hook chose to
|
||||
// fail.
|
||||
var ErrInjectedDelete = errors.New("faultstore: injected delete failure")
|
||||
|
||||
// PutAction is the disposition OnPut assigns to a write.
|
||||
type PutAction int
|
||||
|
||||
@@ -71,11 +66,6 @@ type Storer struct {
|
||||
// OnGet, when set, is consulted for every Get with the object key
|
||||
// and damages the returned bytes accordingly.
|
||||
OnGet func(key string) GetFault
|
||||
|
||||
// OnDelete, when set, is consulted before every Delete with the
|
||||
// object key. Returning true fails the delete and leaves the object
|
||||
// in place.
|
||||
OnDelete func(key string) bool
|
||||
}
|
||||
|
||||
// New wraps inner. inner must be non-nil.
|
||||
@@ -164,12 +154,8 @@ func (f *Storer) Stat(ctx context.Context, key string) (*storage.ObjectInfo, err
|
||||
return f.inner.Stat(ctx, key)
|
||||
}
|
||||
|
||||
// Delete removes the object unless OnDelete fails the delete.
|
||||
// Delete delegates unchanged.
|
||||
func (f *Storer) Delete(ctx context.Context, key string) error {
|
||||
if f.OnDelete != nil && f.OnDelete(key) {
|
||||
return fmt.Errorf("%w for %q", ErrInjectedDelete, key)
|
||||
}
|
||||
|
||||
return f.inner.Delete(ctx, key)
|
||||
}
|
||||
|
||||
|
||||
@@ -120,9 +120,9 @@ func (f *FileStorer) Delete(_ context.Context, key string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// List returns all keys with the given prefix. It fails when the
|
||||
// destination directory is missing; a missing prefix under it is an
|
||||
// empty listing.
|
||||
// List returns all keys with the given prefix, reading through symlinked
|
||||
// directories. It fails when the destination directory is missing; a
|
||||
// missing prefix under it is an empty listing.
|
||||
func (f *FileStorer) List(ctx context.Context, prefix string) ([]string, error) {
|
||||
var keys []string
|
||||
|
||||
@@ -143,7 +143,7 @@ func (f *FileStorer) List(ctx context.Context, prefix string) ([]string, error)
|
||||
return keys, nil // Empty list for non-existent prefix
|
||||
}
|
||||
|
||||
err = afero.Walk(f.fs, basePath, func(path string, info os.FileInfo, err error) error {
|
||||
err = f.walk(basePath, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -176,7 +176,8 @@ func (f *FileStorer) List(ctx context.Context, prefix string) ([]string, error)
|
||||
}
|
||||
|
||||
// ListStream returns a channel of ObjectInfo for large result sets. Like
|
||||
// List, it sends an error when the destination directory is missing.
|
||||
// List, it reads through symlinked directories and sends an error when
|
||||
// the destination directory is missing.
|
||||
func (f *FileStorer) ListStream(ctx context.Context, prefix string) <-chan ObjectInfo {
|
||||
ch := make(chan ObjectInfo)
|
||||
|
||||
@@ -204,7 +205,7 @@ func (f *FileStorer) ListStream(ctx context.Context, prefix string) <-chan Objec
|
||||
return // Empty channel for non-existent prefix
|
||||
}
|
||||
|
||||
_ = afero.Walk(f.fs, basePath, func(path string, info os.FileInfo, err error) error {
|
||||
_ = f.walk(basePath, func(path string, info os.FileInfo, err error) error {
|
||||
// Check context cancellation
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
@@ -352,6 +353,17 @@ func (f *FileStorer) writeAtomic(
|
||||
return nil
|
||||
}
|
||||
|
||||
// walk is afero.Walk, except that it reads through a symlinked directory
|
||||
// the way Get and Stat do, where afero.Walk reports the link as one file.
|
||||
// afero.Walk uses Lstat, which does not follow a link, whenever the
|
||||
// filesystem has one; the struct below carries only the afero.Fs methods,
|
||||
// so the walk uses Stat instead. A link back up the tree is followed until
|
||||
// the operating system refuses the path for too many levels of links, and
|
||||
// the walk then fails.
|
||||
func (f *FileStorer) walk(root string, fn filepath.WalkFunc) error {
|
||||
return afero.Walk(struct{ afero.Fs }{f.fs}, root, fn)
|
||||
}
|
||||
|
||||
// fullPath returns the full filesystem path for a key.
|
||||
func (f *FileStorer) fullPath(key string) string {
|
||||
return filepath.Join(f.basePath, key)
|
||||
|
||||
@@ -4,7 +4,10 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/vaultik/internal/storage"
|
||||
@@ -60,3 +63,100 @@ func TestFileStorerListMissingDestination(t *testing.T) {
|
||||
t.Errorf("ListStream error = %v, want a not-exist error", streamErr)
|
||||
}
|
||||
}
|
||||
|
||||
// TestFileStorerListsThroughSymlinkedDirectory checks that List and
|
||||
// ListStream read through a symlinked metadata/ or blobs/ directory, as Get
|
||||
// and Stat do, instead of listing the link as one key. Listing the whole
|
||||
// destination meets the link part-way through the walk; listing its prefix
|
||||
// starts the walk at the link.
|
||||
func TestFileStorerListsThroughSymlinkedDirectory(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, dir := range []string{"metadata", "blobs"} {
|
||||
t.Run(dir, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
base := t.TempDir()
|
||||
key := dir + "/aa/bb/object"
|
||||
|
||||
s, err := storage.NewFileStorer(base)
|
||||
if err != nil {
|
||||
t.Fatalf("NewFileStorer: %v", err)
|
||||
}
|
||||
|
||||
err = s.Put(context.Background(), key, strings.NewReader("data"))
|
||||
if err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
|
||||
linkPath := filepath.Join(base, dir)
|
||||
movedPath := filepath.Join(t.TempDir(), dir)
|
||||
|
||||
err = os.Rename(linkPath, movedPath)
|
||||
if err != nil {
|
||||
t.Fatalf("moving %s away: %v", dir, err)
|
||||
}
|
||||
|
||||
err = os.Symlink(movedPath, linkPath)
|
||||
if err != nil {
|
||||
t.Fatalf("linking %s: %v", dir, err)
|
||||
}
|
||||
|
||||
want := []string{key}
|
||||
|
||||
for _, prefix := range []string{"", dir + "/"} {
|
||||
got := listSorted(t, s, prefix)
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("List(%q) = %v, want %v", prefix, got, want)
|
||||
}
|
||||
|
||||
got = listStreamKeys(t, s, prefix)
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("ListStream(%q) = %v, want %v", prefix, got, want)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestFileStorerListFailsOnSymlinkLoop checks that a symlink pointing back
|
||||
// at the destination makes List and ListStream fail instead of walking it
|
||||
// forever.
|
||||
func TestFileStorerListFailsOnSymlinkLoop(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ctx := context.Background()
|
||||
base := t.TempDir()
|
||||
|
||||
s, err := storage.NewFileStorer(base)
|
||||
if err != nil {
|
||||
t.Fatalf("NewFileStorer: %v", err)
|
||||
}
|
||||
|
||||
err = s.Put(ctx, testBlobKey, strings.NewReader("data"))
|
||||
if err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
|
||||
err = os.Symlink(base, filepath.Join(base, "blobs", "loop"))
|
||||
if err != nil {
|
||||
t.Fatalf("Symlink: %v", err)
|
||||
}
|
||||
|
||||
keys, err := s.List(ctx, "blobs/")
|
||||
if err == nil {
|
||||
t.Errorf("List = %d keys and no error, want an error", len(keys))
|
||||
}
|
||||
|
||||
var streamErr error
|
||||
|
||||
for object := range s.ListStream(ctx, "blobs/") {
|
||||
if object.Err != nil {
|
||||
streamErr = object.Err
|
||||
}
|
||||
}
|
||||
|
||||
if streamErr == nil {
|
||||
t.Error("ListStream sent no error, want one")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,14 +5,12 @@ import (
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/vaultik/internal/log"
|
||||
"sneak.berlin/go/vaultik/internal/snapshot"
|
||||
"sneak.berlin/go/vaultik/internal/storage/faultstore"
|
||||
)
|
||||
|
||||
// TestNukeRemoteLeavesNoFiles checks that remote nuke leaves no file
|
||||
@@ -25,7 +23,7 @@ func TestNukeRemoteLeavesNoFiles(t *testing.T) {
|
||||
|
||||
ctx := context.Background()
|
||||
storeDir := filepath.Join(t.TempDir(), "store")
|
||||
v, repos, out := backUpToFileDestination(ctx, t, storeDir)
|
||||
v, repos, _ := backUpToFileDestination(ctx, t, storeDir)
|
||||
|
||||
snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
|
||||
require.NoError(t, err)
|
||||
@@ -63,82 +61,4 @@ func TestNukeRemoteLeavesNoFiles(t *testing.T) {
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.Empty(t, files)
|
||||
|
||||
// Nuke deletes the blobs itself, so advice to run prune is wrong.
|
||||
assert.Contains(t, out.String(), "Backup destination store is now empty.")
|
||||
assert.NotContains(t, out.String(), "vaultik prune")
|
||||
}
|
||||
|
||||
// TestNukeRemoteFailsWhenADeleteFails checks that remote nuke returns an
|
||||
// error when a snapshot's metadata or a blob cannot be deleted, does not
|
||||
// report the store empty, and advises running nuke again: prune deletes
|
||||
// no metadata.
|
||||
func TestNukeRemoteFailsWhenADeleteFails(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
for _, refusedPrefix := range []string{"metadata/", "blobs/"} {
|
||||
t.Run(refusedPrefix, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ctx := context.Background()
|
||||
storeDir := filepath.Join(t.TempDir(), "store")
|
||||
v, _, out := backUpToFileDestination(ctx, t, storeDir)
|
||||
|
||||
store := faultstore.New(v.Storage)
|
||||
store.OnDelete = func(key string) bool {
|
||||
return strings.HasPrefix(key, refusedPrefix)
|
||||
}
|
||||
v.Storage = store
|
||||
|
||||
require.Error(t, v.NukeRemote(true))
|
||||
|
||||
assert.NotContains(t, out.String(), "now empty")
|
||||
assert.Contains(t, out.String(), "vaultik remote nuke --force")
|
||||
assert.NotContains(t, out.String(), "vaultik prune")
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestNukeRemoteRefusesAnotherDestinationWithoutRetryAdvice checks that
|
||||
// remote nuke, when the local index belongs to a different destination
|
||||
// than the configured one, fails without advising to run nuke again:
|
||||
// that run would fail the same way.
|
||||
func TestNukeRemoteRefusesAnotherDestinationWithoutRetryAdvice(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
ctx := context.Background()
|
||||
storeDir := filepath.Join(t.TempDir(), "store")
|
||||
v, _, out := backUpToFileDestination(ctx, t, storeDir)
|
||||
|
||||
v.Config.StorageURL = "file://" + filepath.Join(t.TempDir(), "elsewhere")
|
||||
|
||||
err := v.NukeRemote(true)
|
||||
|
||||
require.ErrorContains(t, err, "bound to a different backup destination")
|
||||
assert.NotContains(t, out.String(), "vaultik remote nuke --force")
|
||||
}
|
||||
|
||||
// TestNukeRemoteFailsWhenALocalSnapshotCannotBeRemoved checks that a
|
||||
// snapshot remote nuke cannot remove from the local index fails the
|
||||
// run: nuke then leaves that snapshot's metadata on the destination
|
||||
// store.
|
||||
func TestNukeRemoteFailsWhenALocalSnapshotCannotBeRemoved(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
ctx := context.Background()
|
||||
storeDir := filepath.Join(t.TempDir(), "store")
|
||||
v, _, out := backUpToFileDestination(ctx, t, storeDir)
|
||||
|
||||
_, err := v.DB.Conn().ExecContext(ctx, `
|
||||
CREATE TRIGGER refuse_snapshot_delete BEFORE DELETE ON snapshots
|
||||
BEGIN SELECT RAISE(ABORT, 'simulated index error'); END`)
|
||||
require.NoError(t, err)
|
||||
|
||||
require.Error(t, v.NukeRemote(true))
|
||||
|
||||
assert.NotContains(t, out.String(), "now empty")
|
||||
assert.Contains(t, out.String(), "vaultik remote nuke --force")
|
||||
}
|
||||
|
||||
@@ -19,14 +19,6 @@ type PruneOptions struct {
|
||||
var errNukeRequiresForce = errors.New(
|
||||
"nuke requires --force (this deletes ALL remote snapshots and blobs)")
|
||||
|
||||
// errBlobDeleteFailed reports unreferenced blobs prune could not delete.
|
||||
var errBlobDeleteFailed = errors.New("blob(s) could not be deleted")
|
||||
|
||||
// nukeCommandHint is the command suggested when a remote nuke could not
|
||||
// delete everything: running it again deletes what is left, which
|
||||
// `vaultik prune` does not do for snapshot metadata.
|
||||
const nukeCommandHint = "vaultik remote nuke --force"
|
||||
|
||||
// metadataDirName is the top-level remote directory holding snapshot
|
||||
// metadata.
|
||||
const metadataDirName = "metadata"
|
||||
@@ -43,29 +35,6 @@ func (v *Vaultik) NukeRemote(force bool) error {
|
||||
return errNukeRequiresForce
|
||||
}
|
||||
|
||||
// Checked before anything is deleted: running nuke again would fail
|
||||
// the same way, so the advice below must not follow this error.
|
||||
err := v.EnsureStorageBinding()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
err = v.emptyDestinationStore()
|
||||
if err != nil {
|
||||
v.UI.Warningf("Run '%s' again to delete what is left on the "+
|
||||
"backup destination store.", nukeCommandHint)
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
v.UI.Completef("Backup destination store is now empty.")
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// emptyDestinationStore does the deleting for NukeRemote and stops at
|
||||
// the first step that fails.
|
||||
func (v *Vaultik) emptyDestinationStore() error {
|
||||
v.UI.Beginf("Removing all snapshot metadata from backup destination store.")
|
||||
|
||||
_, err := v.RemoveAllSnapshots(&RemoveOptions{Force: true})
|
||||
@@ -89,6 +58,8 @@ func (v *Vaultik) emptyDestinationStore() error {
|
||||
}
|
||||
}
|
||||
|
||||
v.UI.Completef("Backup destination store is now empty.")
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -191,17 +162,14 @@ func (v *Vaultik) PruneBlobs(opts *PruneOptions) error {
|
||||
v.deleteUnreferencedBlobs(unreferencedBlobs, allBlobs, result)
|
||||
|
||||
if opts.JSON {
|
||||
err = v.outputPruneBlobsJSON(result)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
v.stdoutf("\nDeleted %d blob(s) totaling %s\n",
|
||||
result.BlobsDeleted, ubytes(result.BytesFreed))
|
||||
return v.outputPruneBlobsJSON(result)
|
||||
}
|
||||
|
||||
v.stdoutf("\nDeleted %d blob(s) totaling %s\n",
|
||||
result.BlobsDeleted, ubytes(result.BytesFreed))
|
||||
|
||||
if result.BlobsFailed > 0 {
|
||||
return fmt.Errorf("%d %w", result.BlobsFailed, errBlobDeleteFailed)
|
||||
v.stdoutf("Failed to delete %d blob(s)\n", result.BlobsFailed)
|
||||
}
|
||||
|
||||
return nil
|
||||
|
||||
@@ -1,98 +0,0 @@
|
||||
package vaultik_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/vaultik/internal/log"
|
||||
"sneak.berlin/go/vaultik/internal/storage/faultstore"
|
||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||
)
|
||||
|
||||
// refuseBlobDeletes is an OnDelete hook that fails every blob delete.
|
||||
func refuseBlobDeletes(key string) bool {
|
||||
return strings.HasPrefix(key, "blobs/")
|
||||
}
|
||||
|
||||
// TestPruneBlobs_FailedDeleteReturnsError checks that prune returns an
|
||||
// error, so the command exits non-zero, when an unreferenced blob cannot
|
||||
// be deleted.
|
||||
func TestPruneBlobs_FailedDeleteReturnsError(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
env := newListEnv(t)
|
||||
|
||||
// The store holds no manifest, so nothing references this blob.
|
||||
addBlob(t, env.store.testStorer, testBlobHashA)
|
||||
|
||||
store := faultstore.New(env.store)
|
||||
store.OnDelete = refuseBlobDeletes
|
||||
env.v.Storage = store
|
||||
|
||||
err := env.v.PruneBlobs(&vaultik.PruneOptions{Force: true})
|
||||
|
||||
require.ErrorContains(t, err, "1 blob(s) could not be deleted")
|
||||
}
|
||||
|
||||
// TestPruneBlobs_JSONFailedDeleteReturnsError checks that prune --json
|
||||
// prints its document and then returns an error when an unreferenced
|
||||
// blob cannot be deleted.
|
||||
func TestPruneBlobs_JSONFailedDeleteReturnsError(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
env := newListEnv(t)
|
||||
addBlob(t, env.store.testStorer, testBlobHashA)
|
||||
|
||||
store := faultstore.New(env.store)
|
||||
store.OnDelete = refuseBlobDeletes
|
||||
env.v.Storage = store
|
||||
|
||||
err := env.v.PruneBlobs(&vaultik.PruneOptions{JSON: true})
|
||||
|
||||
require.Error(t, err, "prune must fail when a blob cannot be deleted")
|
||||
|
||||
var result vaultik.PruneBlobsResult
|
||||
|
||||
require.NoError(t, json.Unmarshal(env.stdout.Bytes(), &result),
|
||||
"stdout must hold the JSON document, got:\n%s", env.stdout.String())
|
||||
assert.Equal(t, 1, result.BlobsFailed)
|
||||
assert.Equal(t, 0, result.BlobsDeleted)
|
||||
}
|
||||
|
||||
// TestCreateSnapshotPrune_FailedBlobDeleteReturnsError checks that the
|
||||
// prune step of snapshot create --prune fails the run when an
|
||||
// unreferenced blob cannot be deleted.
|
||||
func TestCreateSnapshotPrune_FailedBlobDeleteReturnsError(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
ctx := context.Background()
|
||||
storeDir := filepath.Join(t.TempDir(), "store")
|
||||
v, _, _ := backUpToFileDestination(ctx, t, storeDir)
|
||||
|
||||
// No snapshot references this blob, so the prune step deletes it.
|
||||
orphan := filepath.Join(storeDir, "blobs", testBlobHashA[:2],
|
||||
testBlobHashA[2:4], testBlobHashA)
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(orphan), 0o750))
|
||||
require.NoError(t, os.WriteFile(orphan, []byte("orphan"), 0o600))
|
||||
|
||||
store := faultstore.New(v.Storage)
|
||||
store.OnDelete = refuseBlobDeletes
|
||||
v.Storage = store
|
||||
|
||||
err := v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
|
||||
Cron: true,
|
||||
Prune: true,
|
||||
})
|
||||
|
||||
require.ErrorContains(t, err, "1 blob(s) could not be deleted")
|
||||
assert.FileExists(t, orphan)
|
||||
}
|
||||
@@ -3,21 +3,17 @@ package vaultik_test
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"io"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/klauspost/compress/zstd"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/vaultik/internal/database"
|
||||
"sneak.berlin/go/vaultik/internal/log"
|
||||
"sneak.berlin/go/vaultik/internal/snapshot"
|
||||
"sneak.berlin/go/vaultik/internal/storage"
|
||||
"sneak.berlin/go/vaultik/internal/types"
|
||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||
)
|
||||
|
||||
@@ -218,37 +214,6 @@ func addBlob(t *testing.T, store *testStorer, hash string) {
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
// newLocalIndex returns an in-memory local index holding a completed
|
||||
// snapshot row for each of ids.
|
||||
func newLocalIndex(t *testing.T, ids ...string) *database.Repositories {
|
||||
t.Helper()
|
||||
|
||||
ctx := context.Background()
|
||||
|
||||
db, err := database.New(ctx, ":memory:")
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = db.Close() })
|
||||
|
||||
repos := database.NewRepositories(db)
|
||||
|
||||
for _, id := range ids {
|
||||
completedAt := time.Now().UTC()
|
||||
snap := &database.Snapshot{
|
||||
ID: types.SnapshotID(id),
|
||||
Hostname: snapHostname,
|
||||
StartedAt: completedAt,
|
||||
CompletedAt: &completedAt,
|
||||
}
|
||||
|
||||
err = repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
|
||||
return repos.Snapshots.Create(ctx, tx, snap)
|
||||
})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
return repos
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Unit Tests for RemoveSnapshot
|
||||
// ============================================================================
|
||||
@@ -268,7 +233,6 @@ func TestRemoveSnapshot_LocalOnly_PreservesRemote(t *testing.T) {
|
||||
addBlob(t, store, blobA)
|
||||
|
||||
tv := vaultik.NewForTesting(store)
|
||||
tv.Repositories = newLocalIndex(t, "snapshot-001")
|
||||
|
||||
opts := &vaultik.RemoveOptions{Force: true, LocalOnly: true}
|
||||
result, err := tv.RemoveSnapshot("snapshot-001", opts)
|
||||
@@ -304,7 +268,6 @@ func TestRemoveSnapshot_DefaultRemovesMetadataNotBlobs(t *testing.T) {
|
||||
addBlob(t, store, blobShared)
|
||||
|
||||
tv := vaultik.NewForTesting(store)
|
||||
tv.Repositories = newLocalIndex(t, "snapshot-001", "snapshot-002")
|
||||
|
||||
opts := &vaultik.RemoveOptions{Force: true}
|
||||
result, err := tv.RemoveSnapshot("snapshot-001", opts)
|
||||
@@ -353,84 +316,6 @@ func TestRemoveSnapshot_DryRun(t *testing.T) {
|
||||
assert.Contains(t, tv.Stdout.String(), "[Dry run - no changes made]")
|
||||
}
|
||||
|
||||
// TestRemoveSnapshot_UnknownIDIsError checks that an ID naming no
|
||||
// snapshot in the local index or on the destination store is an error
|
||||
// under every option, and that nothing is reported as removed.
|
||||
func TestRemoveSnapshot_UnknownIDIsError(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
for name, opts := range map[string]vaultik.RemoveOptions{
|
||||
"force": {Force: true},
|
||||
"json": {JSON: true},
|
||||
"dry-run": {Force: true, DryRun: true},
|
||||
"local-only": {Force: true, LocalOnly: true},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store := newTestStorer()
|
||||
addManifest(t, store, "snapshot-001", nil)
|
||||
|
||||
tv := vaultik.NewForTesting(store)
|
||||
tv.Repositories = newLocalIndex(t, "snapshot-001")
|
||||
|
||||
result, err := tv.RemoveSnapshot("snapshot-999", &opts)
|
||||
|
||||
require.ErrorContains(t, err, "snapshot not found")
|
||||
assert.False(t, result.RemoteRemoved)
|
||||
assert.Empty(t, tv.Stdout.String())
|
||||
assert.True(t, store.hasKey(remoteKeyPath("snapshot-001")))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestRemoveSnapshot_NoMetadataOnDestination checks that removing a
|
||||
// snapshot whose metadata is not on the destination store does not
|
||||
// report metadata removed from it.
|
||||
func TestRemoveSnapshot_NoMetadataOnDestination(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
tv := vaultik.NewForTesting(newTestStorer())
|
||||
tv.Repositories = newLocalIndex(t, "snapshot-001")
|
||||
|
||||
result, err := tv.RemoveSnapshot("snapshot-001",
|
||||
&vaultik.RemoveOptions{Force: true})
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.False(t, result.RemoteRemoved)
|
||||
|
||||
out := tv.Stdout.String()
|
||||
assert.Contains(t, out, "Removed snapshot 'snapshot-001' from local database")
|
||||
assert.NotContains(t, out, "Removed snapshot metadata from remote storage")
|
||||
}
|
||||
|
||||
// TestRemoveSnapshot_NotInLocalIndex checks that removing a snapshot
|
||||
// only the destination store holds removes its metadata there without
|
||||
// reporting a removal from the local database.
|
||||
func TestRemoveSnapshot_NotInLocalIndex(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
store := newTestStorer()
|
||||
addManifest(t, store, "snapshot-001", nil)
|
||||
|
||||
tv := vaultik.NewForTesting(store)
|
||||
tv.Repositories = newLocalIndex(t)
|
||||
|
||||
result, err := tv.RemoveSnapshot("snapshot-001",
|
||||
&vaultik.RemoveOptions{Force: true})
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.True(t, result.RemoteRemoved)
|
||||
assert.False(t, store.hasKey(remoteKeyPath("snapshot-001")))
|
||||
|
||||
out := tv.Stdout.String()
|
||||
assert.NotContains(t, out, "from local database")
|
||||
assert.Contains(t, out, "Removed snapshot metadata from remote storage")
|
||||
}
|
||||
|
||||
func TestRemoveAllSnapshots_RequiresForce(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
@@ -476,8 +361,7 @@ func TestRemoveAllSnapshots_WithForce(t *testing.T) {
|
||||
out := tv.Stdout.String()
|
||||
assert.Contains(t, out, "Removed 2 snapshot(s)")
|
||||
assert.Contains(t, out, "Removed snapshot metadata from remote storage")
|
||||
// Remote nuke, the only caller, deletes the blobs next itself.
|
||||
assert.NotContains(t, out, "vaultik prune")
|
||||
assert.Contains(t, out, "vaultik prune")
|
||||
}
|
||||
|
||||
func TestRemoveAllSnapshots_DryRun(t *testing.T) {
|
||||
|
||||
+52
-109
@@ -23,8 +23,6 @@ var (
|
||||
errNoSnapshotsInConfig = errors.New("no snapshots configured")
|
||||
errSnapshotVerifyFailed = errors.New("verification failed")
|
||||
errRemoveAllNeedsForce = errors.New("--all requires --force")
|
||||
errSnapshotNotFound = errors.New("snapshot not found")
|
||||
errSnapshotRemoveFailed = errors.New("snapshot(s) could not be removed")
|
||||
errInvalidTableName = errors.New("invalid table name")
|
||||
errNoAgeRecipients = errors.New(
|
||||
"creating a snapshot needs at least one public key in " +
|
||||
@@ -645,7 +643,7 @@ func (v *Vaultik) confirmAndExecutePurge(
|
||||
"snapshot_id", snapshotID, "error", err)
|
||||
}
|
||||
|
||||
_, err = v.deleteRemoteSnapshotByKey(snapshot.RemoteSnapshotKey(snapshotID))
|
||||
err = v.deleteRemoteSnapshotByKey(snapshot.RemoteSnapshotKey(snapshotID))
|
||||
if err != nil {
|
||||
return fmt.Errorf("deleting snapshot %s from remote: %w", snapshotID, err)
|
||||
}
|
||||
@@ -1155,11 +1153,6 @@ func (v *Vaultik) RemoveSnapshot(
|
||||
return result, err
|
||||
}
|
||||
|
||||
inLocalIndex, err := v.findSnapshotToRemove(snapshotID, opts)
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
|
||||
if opts.DryRun {
|
||||
return v.removeSnapshotDryRun(result, snapshotID, opts)
|
||||
}
|
||||
@@ -1168,13 +1161,11 @@ func (v *Vaultik) RemoveSnapshot(
|
||||
return result, nil
|
||||
}
|
||||
|
||||
if inLocalIndex {
|
||||
log.Info("Removing snapshot from local database", "snapshot_id", snapshotID)
|
||||
log.Info("Removing snapshot from local database", "snapshot_id", snapshotID)
|
||||
|
||||
err = v.deleteSnapshotFromLocalDB(snapshotID)
|
||||
if err != nil {
|
||||
return result, fmt.Errorf("removing from local database: %w", err)
|
||||
}
|
||||
err = v.deleteSnapshotFromLocalDB(snapshotID)
|
||||
if err != nil {
|
||||
return result, fmt.Errorf("removing from local database: %w", err)
|
||||
}
|
||||
|
||||
if !opts.LocalOnly {
|
||||
@@ -1193,11 +1184,9 @@ func (v *Vaultik) RemoveSnapshot(
|
||||
return result, v.outputRemoveJSON(result)
|
||||
}
|
||||
|
||||
if inLocalIndex {
|
||||
v.stdoutf("Removed snapshot '%s' from local database\n", snapshotID)
|
||||
}
|
||||
v.stdoutf("Removed snapshot '%s' from local database\n", snapshotID)
|
||||
|
||||
if result.RemoteRemoved {
|
||||
if !opts.LocalOnly && result.RemoteRemoved {
|
||||
v.printlnStdout("Removed snapshot metadata from remote storage")
|
||||
v.stdoutf("\nNote: The removed snapshot's blobs remain on the remote. "+
|
||||
"Run '%s' to delete any blobs no longer referenced by any "+
|
||||
@@ -1207,45 +1196,6 @@ func (v *Vaultik) RemoveSnapshot(
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// findSnapshotToRemove reports whether the local index has a row for
|
||||
// snapshotID, and returns errSnapshotNotFound when there is nothing to
|
||||
// remove: no row and, unless LocalOnly is set, no metadata on the
|
||||
// destination store either. The destination store is consulted only
|
||||
// when the local index has no row, so a snapshot in the local index can
|
||||
// still be removed from it while the destination store is unreachable.
|
||||
func (v *Vaultik) findSnapshotToRemove(
|
||||
snapshotID string, opts *RemoveOptions,
|
||||
) (bool, error) {
|
||||
if v.Repositories != nil {
|
||||
snap, err := v.Repositories.Snapshots.GetByID(v.ctx, snapshotID)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("looking up snapshot in local index: %w", err)
|
||||
}
|
||||
|
||||
if snap != nil {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
|
||||
if opts.LocalOnly {
|
||||
return false, fmt.Errorf("%w in the local index: %s",
|
||||
errSnapshotNotFound, snapshotID)
|
||||
}
|
||||
|
||||
objects, err := v.remoteSnapshotObjects(snapshot.RemoteSnapshotKey(snapshotID))
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("looking up snapshot on destination store: %w", err)
|
||||
}
|
||||
|
||||
if len(objects) == 0 {
|
||||
return false, fmt.Errorf(
|
||||
"%w in the local index or on the destination store: %s",
|
||||
errSnapshotNotFound, snapshotID)
|
||||
}
|
||||
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// removeSnapshotDryRun reports what RemoveSnapshot would do without
|
||||
// making changes.
|
||||
func (v *Vaultik) removeSnapshotDryRun(
|
||||
@@ -1305,14 +1255,14 @@ func (v *Vaultik) confirmRemoveSnapshot(snapshotID string, opts *RemoveOptions)
|
||||
// didn't finish and to run `vaultik snapshot remove` for the snapshot
|
||||
// again once the destination store is reachable (`vaultik prune` never
|
||||
// removes snapshot metadata). Returns true when the remote removal
|
||||
// succeeded and deleted something.
|
||||
// succeeded.
|
||||
func (v *Vaultik) removeSnapshotRemote(snapshotID string, opts *RemoveOptions) bool {
|
||||
log.Info("Removing snapshot metadata from remote storage",
|
||||
"snapshot_id", snapshotID)
|
||||
|
||||
remoteKey := snapshot.RemoteSnapshotKey(snapshotID)
|
||||
|
||||
deleted, err := v.deleteRemoteSnapshotByKey(remoteKey)
|
||||
err := v.deleteRemoteSnapshotByKey(remoteKey)
|
||||
if err != nil {
|
||||
log.Warn("Could not remove snapshot metadata from remote storage; "+
|
||||
"run '"+snapshotRemoveCommandHint+"' with the snapshot's ID "+
|
||||
@@ -1330,7 +1280,7 @@ func (v *Vaultik) removeSnapshotRemote(snapshotID string, opts *RemoveOptions) b
|
||||
return false
|
||||
}
|
||||
|
||||
return deleted > 0
|
||||
return true
|
||||
}
|
||||
|
||||
// RemoveAllSnapshots removes every snapshot known to the local
|
||||
@@ -1514,8 +1464,8 @@ func (v *Vaultik) handleRemoveAllDryRun(
|
||||
|
||||
// executeRemoveAll deletes every local snapshot and, unless LocalOnly
|
||||
// is set, every corresponding remote metadata directory plus any
|
||||
// orphan remote keys. Blobs are not touched. It returns an error when
|
||||
// any snapshot could not be removed.
|
||||
// orphan remote keys. Blobs are not touched; the printed prune-command
|
||||
// hint is the next step.
|
||||
func (v *Vaultik) executeRemoveAll(
|
||||
localSnaps, orphanRemoteKeys []string, opts *RemoveOptions,
|
||||
) (*RemoveResult, error) {
|
||||
@@ -1529,11 +1479,18 @@ func (v *Vaultik) executeRemoveAll(
|
||||
|
||||
result := &RemoveResult{}
|
||||
|
||||
failures := v.removeAllLocalSnapshots(localSnaps, opts, result)
|
||||
remoteErrors := v.removeAllLocalSnapshots(localSnaps, opts, result)
|
||||
|
||||
if !opts.LocalOnly {
|
||||
failures += v.removeAllOrphanRemotes(orphanRemoteKeys, result)
|
||||
result.RemoteRemoved = failures == 0
|
||||
remoteErrors += v.removeAllOrphanRemotes(orphanRemoteKeys, result)
|
||||
|
||||
if remoteErrors == 0 {
|
||||
result.RemoteRemoved = true
|
||||
} else if v.UI != nil {
|
||||
v.UI.Warningf("Some remote metadata deletions failed. Run '%s' "+
|
||||
"once the remote is healthy to clean up unreferenced blobs.",
|
||||
pruneCommandHint)
|
||||
}
|
||||
}
|
||||
|
||||
if v.SnapshotManager != nil {
|
||||
@@ -1543,10 +1500,6 @@ func (v *Vaultik) executeRemoveAll(
|
||||
}
|
||||
}
|
||||
|
||||
if failures > 0 {
|
||||
return result, fmt.Errorf("%d %w", failures, errSnapshotRemoveFailed)
|
||||
}
|
||||
|
||||
if opts.JSON {
|
||||
return result, v.outputRemoveJSON(result)
|
||||
}
|
||||
@@ -1555,6 +1508,9 @@ func (v *Vaultik) executeRemoveAll(
|
||||
|
||||
if !opts.LocalOnly && result.RemoteRemoved {
|
||||
v.printlnStdout("Removed snapshot metadata from remote storage")
|
||||
v.stdoutf("\nNote: Removed snapshots' blobs remain on the remote. "+
|
||||
"Run '%s' to delete any blobs no longer referenced by any "+
|
||||
"remaining remote snapshot.\n", pruneCommandHint)
|
||||
}
|
||||
|
||||
return result, nil
|
||||
@@ -1597,13 +1553,12 @@ func (v *Vaultik) deleteIncompleteSnapshot(snapshotIDStr string, id any) bool {
|
||||
}
|
||||
|
||||
// removeAllLocalSnapshots removes each known local snapshot (and, unless
|
||||
// --local-only, its remote metadata), returning the number of snapshots
|
||||
// it could not remove. A snapshot left in the local index keeps its
|
||||
// remote metadata too, since it is not an orphan remote key.
|
||||
// --local-only, its remote metadata), returning the number of remote
|
||||
// deletion failures.
|
||||
func (v *Vaultik) removeAllLocalSnapshots(
|
||||
localSnaps []string, opts *RemoveOptions, result *RemoveResult,
|
||||
) int {
|
||||
failures := 0
|
||||
remoteErrors := 0
|
||||
|
||||
for _, snapshotID := range localSnaps {
|
||||
log.Info("Removing snapshot", "snapshot_id", snapshotID)
|
||||
@@ -1613,26 +1568,24 @@ func (v *Vaultik) removeAllLocalSnapshots(
|
||||
log.Error("Failed to remove from local database",
|
||||
"snapshot_id", snapshotID, "error", err)
|
||||
|
||||
failures++
|
||||
|
||||
continue
|
||||
}
|
||||
|
||||
if !opts.LocalOnly {
|
||||
_, err = v.deleteRemoteSnapshotByKey(
|
||||
err = v.deleteRemoteSnapshotByKey(
|
||||
snapshot.RemoteSnapshotKey(snapshotID))
|
||||
if err != nil {
|
||||
log.Warn("Failed to remove snapshot metadata from remote",
|
||||
"snapshot_id", snapshotID, "error", err)
|
||||
|
||||
failures++
|
||||
remoteErrors++
|
||||
}
|
||||
}
|
||||
|
||||
result.SnapshotsRemoved = append(result.SnapshotsRemoved, snapshotID)
|
||||
}
|
||||
|
||||
return failures
|
||||
return remoteErrors
|
||||
}
|
||||
|
||||
// removeAllOrphanRemotes deletes remote snapshot metadata that has no
|
||||
@@ -1645,7 +1598,7 @@ func (v *Vaultik) removeAllOrphanRemotes(
|
||||
for _, key := range orphanRemoteKeys {
|
||||
log.Info("Removing orphan remote snapshot", "remote_key", key)
|
||||
|
||||
_, err := v.deleteRemoteSnapshotByKey(key)
|
||||
err := v.deleteRemoteSnapshotByKey(key)
|
||||
if err != nil {
|
||||
log.Warn("Failed to remove orphan from remote",
|
||||
"remote_key", key, "error", err)
|
||||
@@ -1692,44 +1645,34 @@ func (v *Vaultik) deleteSnapshotFromLocalDB(snapshotID string) error {
|
||||
}
|
||||
|
||||
// deleteRemoteSnapshotByKey removes everything under
|
||||
// metadata/<remoteKey>/ on the destination store and returns how many
|
||||
// objects it deleted. The argument is a remote key (double-SHA256
|
||||
// derivation), not a human snapshot ID; callers that have a human ID
|
||||
// must hash via snapshot.RemoteSnapshotKey first.
|
||||
func (v *Vaultik) deleteRemoteSnapshotByKey(remoteKey string) (int, error) {
|
||||
objectsToDelete, err := v.remoteSnapshotObjects(remoteKey)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
// metadata/<remoteKey>/ on the destination store. The argument is a
|
||||
// remote key (double-SHA256 derivation), not a human snapshot ID;
|
||||
// callers that have a human ID must hash via snapshot.RemoteSnapshotKey
|
||||
// first.
|
||||
func (v *Vaultik) deleteRemoteSnapshotByKey(remoteKey string) error {
|
||||
prefix := fmt.Sprintf("metadata/%s/", remoteKey)
|
||||
objectCh := v.Storage.ListStream(v.ctx, prefix)
|
||||
|
||||
var objectsToDelete []string
|
||||
|
||||
for object := range objectCh {
|
||||
if object.Err != nil {
|
||||
return fmt.Errorf("listing objects: %w", object.Err)
|
||||
}
|
||||
|
||||
objectsToDelete = append(objectsToDelete, object.Key)
|
||||
}
|
||||
|
||||
for i, key := range objectsToDelete {
|
||||
err = v.Storage.Delete(v.ctx, key)
|
||||
for _, key := range objectsToDelete {
|
||||
err := v.Storage.Delete(v.ctx, key)
|
||||
if err != nil {
|
||||
return i, fmt.Errorf("removing %s: %w", key, err)
|
||||
return fmt.Errorf("removing %s: %w", key, err)
|
||||
}
|
||||
|
||||
log.Debug("Deleted remote object", "key", key)
|
||||
}
|
||||
|
||||
return len(objectsToDelete), nil
|
||||
}
|
||||
|
||||
// remoteSnapshotObjects lists the keys of every object under
|
||||
// metadata/<remoteKey>/ on the destination store.
|
||||
func (v *Vaultik) remoteSnapshotObjects(remoteKey string) ([]string, error) {
|
||||
prefix := fmt.Sprintf("metadata/%s/", remoteKey)
|
||||
|
||||
var keys []string
|
||||
|
||||
for object := range v.Storage.ListStream(v.ctx, prefix) {
|
||||
if object.Err != nil {
|
||||
return nil, fmt.Errorf("listing objects: %w", object.Err)
|
||||
}
|
||||
|
||||
keys = append(keys, object.Key)
|
||||
}
|
||||
|
||||
return keys, nil
|
||||
return nil
|
||||
}
|
||||
|
||||
// outputRemoveJSON outputs the removal result as JSON
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
)
|
||||
|
||||
// errStorageBindingMismatch is returned when the local index database is
|
||||
// bound to a different destination than the configured one.
|
||||
// bound to a different destination than the configured storage_url.
|
||||
var errStorageBindingMismatch = errors.New(
|
||||
"local index is bound to a different backup destination")
|
||||
|
||||
@@ -24,9 +24,8 @@ var errStorageBindingMismatch = errors.New(
|
||||
//
|
||||
// Behaviour:
|
||||
// - On first use (empty stored value), record the configured
|
||||
// destination (storage_url, or the s3.* endpoint, bucket and
|
||||
// prefix) and log the binding.
|
||||
// - When the stored value matches the configured destination, do
|
||||
// storage_url and log the binding.
|
||||
// - When the stored value matches the configured storage_url, do
|
||||
// nothing and return nil.
|
||||
// - When the two differ, refuse with an error that tells the user
|
||||
// how to recover (revert the config, or run `vaultik database
|
||||
@@ -47,10 +46,12 @@ func (v *Vaultik) EnsureStorageBinding() error {
|
||||
|
||||
configured := v.Config.StorageURL
|
||||
if configured == "" {
|
||||
// The s3.* form: bind on its endpoint, bucket and prefix,
|
||||
// written the way storage_url would name the same destination.
|
||||
s3 := v.Config.S3
|
||||
configured = "s3://" + s3.Bucket + "/" + s3.Prefix + "?endpoint=" + s3.Endpoint
|
||||
// Some legacy configs still use the split s3.* keys instead of
|
||||
// storage_url. Falling back to a synthetic URL for those would
|
||||
// hide the fact that the binding is loose. Instead, treat
|
||||
// unset as "nothing to bind against" — the check is
|
||||
// necessarily best-effort for those configs.
|
||||
return nil
|
||||
}
|
||||
|
||||
stored, err := v.Repositories.LocalMeta.Get(v.ctx, database.LocalMetaKeyStorageURL)
|
||||
@@ -95,8 +96,7 @@ func buildBindingMismatchMessage(stored, configured string) string {
|
||||
"future snapshots referencing blobs that don't exist at the new destination.\n" +
|
||||
"\n" +
|
||||
"To proceed, either:\n" +
|
||||
" - revert storage_url (or the s3.* endpoint, bucket and prefix) in your\n" +
|
||||
" config to the bound destination, or\n" +
|
||||
" - revert storage_url in your config to the bound destination, or\n" +
|
||||
" - run 'vaultik database delete' to discard the local index and rebuild it\n" +
|
||||
" from a fresh full backup against the new destination"
|
||||
}
|
||||
|
||||
@@ -76,51 +76,3 @@ func TestEnsureStorageBinding_MismatchRefuses(t *testing.T) {
|
||||
assert.Contains(t, err.Error(), "file:///mnt/backups/new")
|
||||
assert.Contains(t, err.Error(), "vaultik database delete")
|
||||
}
|
||||
|
||||
// A config with no storage_url names its destination with s3.endpoint,
|
||||
// s3.bucket and s3.prefix; a change to any of them must be refused just
|
||||
// as a changed storage_url is.
|
||||
func TestEnsureStorageBinding_S3FieldsChangeRefuses(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
bound := config.S3Config{
|
||||
Endpoint: "https://s3.example.com",
|
||||
Bucket: "bucket-a",
|
||||
Prefix: "backups",
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
changed config.S3Config
|
||||
}{
|
||||
{"bucket", config.S3Config{
|
||||
Endpoint: bound.Endpoint, Bucket: "bucket-b", Prefix: bound.Prefix,
|
||||
}},
|
||||
{"prefix", config.S3Config{
|
||||
Endpoint: bound.Endpoint, Bucket: bound.Bucket, Prefix: "other",
|
||||
}},
|
||||
{"endpoint", config.S3Config{
|
||||
Endpoint: "https://s3.other.com", Bucket: bound.Bucket, Prefix: bound.Prefix,
|
||||
}},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tv, _ := buildBindTestVaultik(t, "")
|
||||
|
||||
tv.Config.S3 = bound
|
||||
require.NoError(t, tv.EnsureStorageBinding(), "first use binds")
|
||||
require.NoError(t, tv.EnsureStorageBinding(), "same destination passes")
|
||||
|
||||
tv.Config.S3 = tt.changed
|
||||
err := tv.EnsureStorageBinding()
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(),
|
||||
"local index is bound to a different backup destination")
|
||||
assert.Contains(t, err.Error(), "vaultik database delete")
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
package vaultik_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/vaultik/internal/log"
|
||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||
)
|
||||
|
||||
// TestPruneDeletesNothingThroughSymlinkedDirectory checks that prune keeps
|
||||
// every blob and the local snapshot record when the metadata/ or blobs/
|
||||
// directory of a file:// destination is a symlink to a directory elsewhere
|
||||
// (https://git.eeqj.de/sneak/vaultik/issues/296). A backup writes through
|
||||
// such a link, so prune has to list through it too.
|
||||
func TestPruneDeletesNothingThroughSymlinkedDirectory(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
for _, dir := range []string{"metadata", "blobs"} {
|
||||
t.Run(dir, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ctx := context.Background()
|
||||
storeDir := filepath.Join(t.TempDir(), "store")
|
||||
v, repos, _ := backUpToFileDestination(ctx, t, storeDir)
|
||||
|
||||
var blobFiles []string
|
||||
|
||||
err := filepath.WalkDir(filepath.Join(storeDir, "blobs"),
|
||||
func(path string, entry fs.DirEntry, err error) error {
|
||||
if err == nil && !entry.IsDir() {
|
||||
blobFiles = append(blobFiles, path)
|
||||
}
|
||||
|
||||
return err
|
||||
})
|
||||
require.NoError(t, err)
|
||||
require.NotEmpty(t, blobFiles)
|
||||
|
||||
linkPath := filepath.Join(storeDir, dir)
|
||||
movedPath := filepath.Join(t.TempDir(), dir)
|
||||
require.NoError(t, os.Rename(linkPath, movedPath))
|
||||
require.NoError(t, os.Symlink(movedPath, linkPath))
|
||||
|
||||
require.NoError(t, v.Prune(&vaultik.PruneOptions{Force: true}))
|
||||
|
||||
// When blobs/ is the link, these paths run through it.
|
||||
for _, blobFile := range blobFiles {
|
||||
assert.FileExists(t, blobFile, "prune must delete no blob")
|
||||
}
|
||||
|
||||
snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
|
||||
require.NoError(t, err)
|
||||
assert.Len(t, snapshots, 1, "prune must delete no local snapshot record")
|
||||
})
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user