Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
28a272903f | ||
|
|
7f3fb97012 |
No files matched your search
@@ -196,9 +196,10 @@ vaultik version
|
||||
that cannot be restored when restoring, instead of aborting. Packing and
|
||||
storage errors while creating a snapshot (which would leave a chunk recorded
|
||||
but not stored) still abort the run. A run that skipped anything still
|
||||
finishes but exits non-zero: `snapshot create` keeps the snapshot and counts
|
||||
each file or directory it skipped as failed. A file removed during the backup
|
||||
is also counted as failed but does not change the exit status.
|
||||
finishes but exits `3` (see [exit status](#exit-status)): `snapshot create`
|
||||
keeps the snapshot and counts each file or directory it skipped as failed. A
|
||||
file removed during the backup is also counted as failed but does not change
|
||||
the exit status.
|
||||
|
||||
### locking
|
||||
|
||||
@@ -271,6 +272,19 @@ Every `--json` command therefore pipes on its own, with no additional flag:
|
||||
`vaultik snapshot list --json | jq .` and `vaultik prune --json | jq .` both
|
||||
work as written.
|
||||
|
||||
### exit status
|
||||
|
||||
- `0`: the command succeeded.
|
||||
- `1`: the command failed.
|
||||
- `3`: `snapshot create` or `snapshot restore` finished, but `--skip-errors`
|
||||
skipped files it could not read or restore. `restic backup` uses the same
|
||||
status for this case.
|
||||
- `130`: Ctrl-C, `SIGINT` or `SIGTERM` stopped the command.
|
||||
|
||||
A cron job can page on `1` and only log `3`. A run that skipped files and also
|
||||
failed in another way exits `1`. For example, a restore that skipped files and
|
||||
was asked for a path the snapshot does not hold exits `1`.
|
||||
|
||||
### environment variables
|
||||
|
||||
- `VAULTIK_AGE_SECRET_KEY`: Age private key for decryption (required for
|
||||
@@ -347,24 +361,25 @@ In such a config, an unknown key or a value of the wrong type is refused.
|
||||
local index, as `prune` does. By default keeps only the latest snapshot per
|
||||
name; use `--keep-newer-than` for a rolling window. When `--skip-errors`
|
||||
skipped a file or directory it could not read or list, the prune does not run
|
||||
and every older snapshot is kept, since one of them may be the only one that
|
||||
holds what was skipped; the command says so and exits non-zero. When the
|
||||
backup found nothing in a configured path, as in the mount point of a volume
|
||||
that is not mounted, the prune does not run either: every older snapshot is
|
||||
kept, and the command names the path and exits non-zero, on every run until
|
||||
the path holds something again. Take a path that is empty on purpose out of
|
||||
the config, or remove the older snapshots by hand with `snapshot purge`. When
|
||||
an earlier snapshot of the same name in the local index found a volume mounted
|
||||
at or below a configured path (a directory on a different filesystem from its
|
||||
parent, with an entry in it that the backup does not exclude) and this backup
|
||||
does not, the prune does not run either: every older snapshot is kept, and the
|
||||
command names the directory and exits non-zero, on every run until the volume
|
||||
is mounted again. A backup also does not find the volume when it finds no such
|
||||
entry in its directory, as when an automounted volume cannot be mounted, or
|
||||
when that directory is no longer backed up, because it is excluded or its
|
||||
configured path was removed. If any of this is on purpose, as for a retired
|
||||
volume, remove the older snapshots by hand with `snapshot purge`. A file
|
||||
removed during the backup does not stop the prune.
|
||||
and every older snapshot is kept. One of them may be the only one that holds
|
||||
what was skipped. The command says so and exits `3`. The prune does not run
|
||||
either when the backup found nothing in a configured path, as in the mount
|
||||
point of a volume that is not mounted. Every older snapshot is kept, and the
|
||||
command names the path and exits `1`, on every run until the path holds
|
||||
something again. Take a path that is empty on purpose out of the config, or
|
||||
remove the older snapshots by hand with `snapshot purge`. The prune does not
|
||||
run either when this backup does not find a volume that an earlier snapshot of
|
||||
the same name in the local index found. The volume's mount point is a
|
||||
directory at or below a configured path. That directory is on a different
|
||||
filesystem from its parent and holds an entry the backup does not exclude.
|
||||
Every older snapshot is kept, and the command names the directory and exits
|
||||
`1`, on every run until the volume is mounted again. A backup also does not
|
||||
find the volume when it finds no such entry in its directory, as when an
|
||||
automounted volume cannot be mounted. Nor does it find the volume when that
|
||||
directory is no longer backed up, because it is excluded or its configured
|
||||
path was removed. If any of this is on purpose, as for a retired volume,
|
||||
remove the older snapshots by hand with `snapshot purge`. A file removed
|
||||
during the backup does not stop the prune.
|
||||
- `--keep-newer-than <duration>`: With `--prune`, keep snapshots newer than this
|
||||
duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`). Without
|
||||
`--prune`, or with a value that is not a valid duration, the command fails
|
||||
@@ -554,12 +569,6 @@ count and blob size unknown, `null` in `--json`. A file under `blobs/` whose
|
||||
name is not a blob hash is not counted as a blob, since `prune` never deletes
|
||||
it.
|
||||
|
||||
A blob that a manifest lists but the destination does not hold is reported as
|
||||
missing, with a count and size. The report names each snapshot that lists a
|
||||
missing blob, since that snapshot cannot be restored. The command then exits
|
||||
non-zero. `--json` gives these in `missing_blob_count`, `missing_blob_size` and
|
||||
`snapshots_missing_blobs`.
|
||||
|
||||
- `--json`: Output as JSON
|
||||
|
||||
**`remote nuke`**: Delete every object under `metadata/` and `blobs/` on the
|
||||
|
||||
+7
-6
@@ -198,7 +198,8 @@ func RunApp(ctx context.Context, app *fx.App) error {
|
||||
// (or, under `snapshot verify --json`, put in its document). Entry
|
||||
// turns it into a non-zero exit status without printing anything
|
||||
// further, so the error line is not doubled. It flows up from
|
||||
// RunOperation through cobra to Entry.
|
||||
// RunOperation through cobra to Entry, wrapping the operation's error
|
||||
// so that Entry can choose the exit status from it.
|
||||
var errReported = errors.New("operation failed")
|
||||
|
||||
// errInterrupted marks an operation that SIGINT or SIGTERM stopped
|
||||
@@ -232,8 +233,8 @@ func RunOperation(
|
||||
) error {
|
||||
var (
|
||||
mu sync.Mutex
|
||||
finished bool // op returned before any interrupt cancelled it
|
||||
failed bool // op finished with an error
|
||||
finished bool // op returned before any interrupt cancelled it
|
||||
opErr error // what op returned, when it finished
|
||||
)
|
||||
|
||||
opts.Invokes = append(opts.Invokes,
|
||||
@@ -258,7 +259,7 @@ func RunOperation(
|
||||
|
||||
mu.Lock()
|
||||
finished = true
|
||||
failed = err != nil
|
||||
opErr = err
|
||||
mu.Unlock()
|
||||
}
|
||||
|
||||
@@ -303,8 +304,8 @@ func RunOperation(
|
||||
switch {
|
||||
case !finished:
|
||||
return errInterrupted
|
||||
case failed:
|
||||
return errReported
|
||||
case opErr != nil:
|
||||
return fmt.Errorf("%w: %w", errReported, opErr)
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
|
||||
+16
-2
@@ -9,6 +9,7 @@ import (
|
||||
|
||||
"sneak.berlin/go/vaultik/internal/globals"
|
||||
"sneak.berlin/go/vaultik/internal/ui"
|
||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||
)
|
||||
|
||||
// shortCommitLen is the number of git commit hash characters shown in
|
||||
@@ -20,6 +21,13 @@ const shortCommitLen = 12
|
||||
// shell reports for a command stopped by Ctrl-C.
|
||||
const exitCodeInterrupted = 130
|
||||
|
||||
// exitCodeSkippedFiles is the exit status of a `snapshot create` or
|
||||
// `snapshot restore` that finished but skipped files under
|
||||
// --skip-errors. It is the status `restic backup` uses for a snapshot
|
||||
// that could not read some files, so a cron job can tell such a run
|
||||
// from one that made no backup.
|
||||
const exitCodeSkippedFiles = 3
|
||||
|
||||
// Entry is the main entry point for the CLI application.
|
||||
// It first calls globals.FillUnsetFromBuildInfo, which fills in the
|
||||
// version, commit and commit date that the linker flags did not set.
|
||||
@@ -30,8 +38,9 @@ const exitCodeInterrupted = 130
|
||||
// The banner goes to stderr because stdout carries only the output the
|
||||
// user asked for, such as a completion script or a `config get` value.
|
||||
//
|
||||
// It returns the process exit code (0 on success, 130 when interrupted,
|
||||
// 1 on any other error) rather than calling os.Exit, so that main's
|
||||
// It returns the process exit code (0 on success, 3 when files were
|
||||
// skipped, 130 when interrupted, 1 on any other error) rather than
|
||||
// calling os.Exit, so that main's
|
||||
// deferred profile writers run before the process ends. See run in
|
||||
// cmd/vaultik/main.go.
|
||||
func Entry() int {
|
||||
@@ -58,6 +67,11 @@ func Entry() int {
|
||||
return exitCodeInterrupted
|
||||
}
|
||||
|
||||
if errors.Is(err, vaultik.ErrFilesUnreadable) ||
|
||||
errors.Is(err, vaultik.ErrFilesFailedRestore) {
|
||||
return exitCodeSkippedFiles
|
||||
}
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
package cli //nolint:testpackage // shares runEntry and the hermetic config helpers
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const flagSkipErrors = "--skip-errors"
|
||||
|
||||
// TestEntrySnapshotCreateSkipErrorsExitsNonZeroWhenItSkips backs up,
|
||||
// under --skip-errors, a source directory that does not exist, which the
|
||||
// walk skips. The command exits with exitCodeSkippedFiles and reports
|
||||
// the backup as completed, not as a failed snapshot creation. See
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/310 and
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/439.
|
||||
//
|
||||
//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
|
||||
func TestEntrySnapshotCreateSkipErrorsExitsNonZeroWhenItSkips(t *testing.T) {
|
||||
// writeBackupConfig would create the source directory.
|
||||
configPath := writeHermeticPruneConfig(t, false)
|
||||
|
||||
code, stdout, stderr := runEntry(t,
|
||||
flagConfig, configPath, flagSkipErrors, cmdSnapshot, cmdCreate)
|
||||
|
||||
assert.Equal(t, exitCodeSkippedFiles, code)
|
||||
assert.Contains(t, stdout,
|
||||
"Files: 1 examined, 0 backed up, 0 unchanged, 1 failed.")
|
||||
assert.NotContains(t, stdout, "Finished successfully.")
|
||||
assert.Contains(t, stderr,
|
||||
"Backup completed, but 1 file(s) could not be read")
|
||||
assert.NotContains(t, stderr, "Snapshot creation failed")
|
||||
}
|
||||
|
||||
// TestEntrySnapshotRestoreSkipErrorsExitStatus backs up, deletes the
|
||||
// blobs from the destination, and restores under --skip-errors, so every
|
||||
// file is skipped. A restore that only skipped files exits with
|
||||
// exitCodeSkippedFiles. One that was also asked for a path the snapshot
|
||||
// does not hold exits 1. See
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/439.
|
||||
//
|
||||
//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
|
||||
func TestEntrySnapshotRestoreSkipErrorsExitStatus(t *testing.T) {
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
missingPaths []string
|
||||
want int
|
||||
}{
|
||||
{name: "skipped files only", want: exitCodeSkippedFiles},
|
||||
{
|
||||
name: "and a path that matches nothing",
|
||||
missingPaths: []string{"/no/such/path"},
|
||||
want: 1,
|
||||
},
|
||||
} {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
configPath, storeDir := writeBackupConfigWithPrivateKey(t)
|
||||
|
||||
code, _, _ := runEntry(t,
|
||||
flagConfig, configPath, cmdSnapshot, cmdCreate)
|
||||
require.Equal(t, 0, code)
|
||||
|
||||
require.NoError(t, os.RemoveAll(filepath.Join(storeDir, blobsDir)))
|
||||
|
||||
// writeBackupConfig puts the source directory beside the
|
||||
// config file.
|
||||
sourceDir := filepath.Join(filepath.Dir(configPath), "source")
|
||||
args := append([]string{
|
||||
flagConfig, configPath, flagSkipErrors, cmdSnapshot,
|
||||
cmdRestore, onlySnapshotID(t, configPath), t.TempDir(),
|
||||
sourceDir,
|
||||
}, testCase.missingPaths...)
|
||||
|
||||
code, _, stderr := runEntry(t, args...)
|
||||
|
||||
assert.Equal(t, testCase.want, code)
|
||||
assert.Contains(t, stderr, "1 file(s) failed to restore")
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
package cli //nolint:testpackage // shares runEntry and the hermetic config helpers
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
// TestEntrySnapshotCreateSkipErrorsExitsNonZeroWhenItSkips backs up,
|
||||
// under --skip-errors, a source directory that does not exist, which the
|
||||
// walk skips. The command exits non-zero and reports the backup as
|
||||
// completed, not as a failed snapshot creation. See
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/310.
|
||||
//
|
||||
//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
|
||||
func TestEntrySnapshotCreateSkipErrorsExitsNonZeroWhenItSkips(t *testing.T) {
|
||||
// writeBackupConfig would create the source directory.
|
||||
configPath := writeHermeticPruneConfig(t, false)
|
||||
|
||||
code, stdout, stderr := runEntry(t,
|
||||
flagConfig, configPath, "--skip-errors", cmdSnapshot, cmdCreate)
|
||||
|
||||
assert.Equal(t, 1, code)
|
||||
assert.Contains(t, stdout,
|
||||
"Files: 1 examined, 0 backed up, 0 unchanged, 1 failed.")
|
||||
assert.NotContains(t, stdout, "Finished successfully.")
|
||||
assert.Contains(t, stderr,
|
||||
"Backup completed, but 1 file(s) could not be read")
|
||||
assert.NotContains(t, stderr, "Snapshot creation failed")
|
||||
}
|
||||
@@ -75,9 +75,7 @@ func newRemoteInfoCommand() *cobra.Command {
|
||||
- Size of all snapshot metadata (per snapshot and total)
|
||||
- Count and total size of all blobs
|
||||
- Count and size of referenced blobs (from all manifests)
|
||||
- Count and size of orphaned blobs (not referenced by any manifest)
|
||||
- Count and size of referenced blobs missing from remote storage, and the
|
||||
snapshots that list them; the command exits non-zero when any are missing`,
|
||||
- Count and size of orphaned blobs (not referenced by any manifest)`,
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, _ []string) error {
|
||||
// Use unified config resolution
|
||||
|
||||
@@ -9,7 +9,6 @@ import (
|
||||
|
||||
"github.com/dustin/go-humanize"
|
||||
"sneak.berlin/go/vaultik/internal/log"
|
||||
"sneak.berlin/go/vaultik/internal/storage"
|
||||
)
|
||||
|
||||
// ShowInfo displays system and configuration information
|
||||
@@ -188,9 +187,6 @@ type SnapshotMetadataInfo struct {
|
||||
// backup interrupted before its manifest upload leaves a directory
|
||||
// without one, which prune does not treat as a snapshot.
|
||||
hasManifest bool
|
||||
|
||||
// Hashes of the blobs the snapshot's manifest lists.
|
||||
blobHashes []string
|
||||
}
|
||||
|
||||
// RemoteInfoResult contains all remote storage information
|
||||
@@ -220,13 +216,6 @@ type RemoteInfoResult struct {
|
||||
OrphanedBlobCount *int `json:"orphaned_blob_count"`
|
||||
OrphanedBlobSize *int64 `json:"orphaned_blob_size"`
|
||||
|
||||
// Blobs that a manifest lists but the destination does not hold,
|
||||
// and the remote key of each snapshot whose manifest lists one.
|
||||
// Those snapshots cannot be restored.
|
||||
MissingBlobCount int `json:"missing_blob_count"`
|
||||
MissingBlobSize int64 `json:"missing_blob_size"`
|
||||
SnapshotsMissingBlobs []string `json:"snapshots_missing_blobs,omitempty"`
|
||||
|
||||
// Remote key of each snapshot whose manifest could not be read
|
||||
UnreadableManifests []string `json:"unreadable_manifests,omitempty"`
|
||||
|
||||
@@ -294,30 +283,19 @@ func (v *Vaultik) RemoteInfo(jsonOutput bool) error {
|
||||
"snapshots", result.TotalMetadataCount,
|
||||
"total_blobs", result.TotalBlobCount,
|
||||
"referenced_blobs", result.ReferencedBlobCount,
|
||||
"missing_blobs", result.MissingBlobCount,
|
||||
"unreadable_manifests", len(result.UnreadableManifests))
|
||||
|
||||
if jsonOutput {
|
||||
enc := json.NewEncoder(v.Stdout)
|
||||
enc.SetIndent("", " ")
|
||||
|
||||
err = enc.Encode(result)
|
||||
if err != nil {
|
||||
return fmt.Errorf("encoding JSON: %w", err)
|
||||
}
|
||||
return enc.Encode(result)
|
||||
}
|
||||
|
||||
if showText {
|
||||
v.printRemoteInfoTable(result)
|
||||
}
|
||||
|
||||
// The report is written first, so the exit status adds to it.
|
||||
if result.MissingBlobCount > 0 {
|
||||
return fmt.Errorf("%w: %d blob(s) that snapshots list are missing; "+
|
||||
"%d snapshot(s) cannot be restored", errProblemsInStorage,
|
||||
result.MissingBlobCount, len(result.SnapshotsMissingBlobs))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -427,7 +405,6 @@ func (v *Vaultik) collectReferencedBlobsFromManifests(
|
||||
for _, blob := range manifest.Blobs {
|
||||
referencedBlobs[blob.Hash] = blob.CompressedSize
|
||||
blobsSize += blob.CompressedSize
|
||||
info.blobHashes = append(info.blobHashes, blob.Hash)
|
||||
}
|
||||
|
||||
info.BlobCount = &blobCount
|
||||
@@ -485,8 +462,6 @@ func (v *Vaultik) scanRemoteBlobStorage(
|
||||
result.TotalBlobSize += blob.Size
|
||||
}
|
||||
|
||||
countMissingBlobs(result, referencedBlobs, allBlobs)
|
||||
|
||||
// A blob named only by a manifest that could not be read, or by one
|
||||
// under a skipped name, would be counted as orphaned, so the orphan
|
||||
// figures stay unknown.
|
||||
@@ -512,33 +487,6 @@ func (v *Vaultik) scanRemoteBlobStorage(
|
||||
return nil
|
||||
}
|
||||
|
||||
// countMissingBlobs records in result the referenced blobs that
|
||||
// allBlobs, the listing of the destination, does not hold, and the
|
||||
// snapshots whose manifests list one of them. It reads the snapshots
|
||||
// from result.Snapshots, so populateRemoteInfoResult must run first.
|
||||
func countMissingBlobs(
|
||||
result *RemoteInfoResult, referencedBlobs map[string]int64,
|
||||
allBlobs map[string]storage.ObjectInfo,
|
||||
) {
|
||||
for hash, size := range referencedBlobs {
|
||||
if _, present := allBlobs[hash]; !present {
|
||||
result.MissingBlobCount++
|
||||
result.MissingBlobSize += size
|
||||
}
|
||||
}
|
||||
|
||||
for _, info := range result.Snapshots {
|
||||
for _, hash := range info.blobHashes {
|
||||
if _, present := allBlobs[hash]; !present {
|
||||
result.SnapshotsMissingBlobs = append(
|
||||
result.SnapshotsMissingBlobs, info.SnapshotID)
|
||||
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// printRemoteInfoTable renders the human-readable remote info output
|
||||
func (v *Vaultik) printRemoteInfoTable(result *RemoteInfoResult) {
|
||||
const (
|
||||
@@ -591,12 +539,6 @@ func (v *Vaultik) printRemoteInfoTable(result *RemoteInfoResult) {
|
||||
"", "", ubytes(result.TotalMetadataSize))
|
||||
}
|
||||
|
||||
v.printBlobStorageSummary(result)
|
||||
}
|
||||
|
||||
// printBlobStorageSummary renders the blob storage section that ends
|
||||
// the human-readable remote info output.
|
||||
func (v *Vaultik) printBlobStorageSummary(result *RemoteInfoResult) {
|
||||
v.stdoutf("\n=== Blob Storage ===\n")
|
||||
v.stdoutf("Total blobs on remote: %s (%s)\n",
|
||||
humanize.Comma(int64(result.TotalBlobCount)),
|
||||
@@ -604,31 +546,22 @@ func (v *Vaultik) printBlobStorageSummary(result *RemoteInfoResult) {
|
||||
v.stdoutf("Referenced by snapshots: %s (%s)\n",
|
||||
humanize.Comma(int64(result.ReferencedBlobCount)),
|
||||
ubytes(result.ReferencedBlobSize))
|
||||
v.stdoutf("Missing from remote: %s (%s)\n",
|
||||
humanize.Comma(int64(result.MissingBlobCount)),
|
||||
ubytes(result.MissingBlobSize))
|
||||
|
||||
if result.OrphanedBlobCount == nil {
|
||||
v.stdoutf("Orphaned (unreferenced): unknown "+
|
||||
"(%d manifest(s) could not be read, "+
|
||||
"%d manifest(s) under a non-conforming name skipped)\n",
|
||||
len(result.UnreadableManifests), result.SkippedManifestCount)
|
||||
} else {
|
||||
v.stdoutf("Orphaned (unreferenced): %s (%s)\n",
|
||||
humanize.Comma(int64(*result.OrphanedBlobCount)),
|
||||
ubytes(*result.OrphanedBlobSize))
|
||||
|
||||
if *result.OrphanedBlobCount > 0 {
|
||||
v.stdoutf("\nRun 'vaultik prune' to remove orphaned blobs.\n")
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if len(result.SnapshotsMissingBlobs) > 0 {
|
||||
v.stdoutf("\nSnapshots that cannot be restored (blobs missing):\n")
|
||||
v.stdoutf("Orphaned (unreferenced): %s (%s)\n",
|
||||
humanize.Comma(int64(*result.OrphanedBlobCount)),
|
||||
ubytes(*result.OrphanedBlobSize))
|
||||
|
||||
for _, id := range result.SnapshotsMissingBlobs {
|
||||
v.stdoutf(" %s\n", id)
|
||||
}
|
||||
if *result.OrphanedBlobCount > 0 {
|
||||
v.stdoutf("\nRun 'vaultik prune' to remove orphaned blobs.\n")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -83,7 +83,6 @@ func TestRemoteInfo_UnreadableManifestLeavesSnapshotBlobsUnknown(t *testing.T) {
|
||||
env := newListEnv(t)
|
||||
readableKey := env.addRemote(t, listRemoteID,
|
||||
time.Date(2026, 3, 2, 0, 0, 0, 0, time.UTC))
|
||||
addBlob(t, env.store.testStorer, testBlobHashA)
|
||||
|
||||
unreadableKey := snapshot.RemoteSnapshotKey(listLocalID)
|
||||
require.NoError(t, env.store.Put(context.Background(),
|
||||
@@ -271,7 +270,6 @@ func TestRemoteInfo_StatusLineUsesUIMarker(t *testing.T) {
|
||||
|
||||
env := newListEnv(t)
|
||||
env.addRemote(t, listRemoteID, time.Date(2026, 3, 2, 0, 0, 0, 0, time.UTC))
|
||||
addBlob(t, env.store.testStorer, testBlobHashA)
|
||||
|
||||
const statusLine = ui.Marker + " Scanning snapshot metadata...\n"
|
||||
|
||||
@@ -283,61 +281,3 @@ func TestRemoteInfo_StatusLineUsesUIMarker(t *testing.T) {
|
||||
require.NoError(t, env.v.RemoteInfo(false))
|
||||
assert.NotContains(t, env.stdout.String(), "Scanning snapshot metadata")
|
||||
}
|
||||
|
||||
// TestRemoteInfo_ReportsBlobsMissingFromDestination checks that a blob a
|
||||
// manifest lists but the destination does not hold is counted as
|
||||
// missing, that the snapshot listing it is named, and that remote info
|
||||
// then fails after writing its report.
|
||||
func TestRemoteInfo_ReportsBlobsMissingFromDestination(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
env := newListEnv(t)
|
||||
|
||||
// The first snapshot lists blob A, which is not stored. The second
|
||||
// lists blob B, which is.
|
||||
brokenKey := env.addRemote(t, listRemoteID,
|
||||
time.Date(2026, 3, 2, 0, 0, 0, 0, time.UTC))
|
||||
addManifest(t, env.store.testStorer, listLocalID, []string{testBlobHashB})
|
||||
addBlob(t, env.store.testStorer, testBlobHashB)
|
||||
|
||||
const wantError = "1 blob(s) that snapshots list are missing; " +
|
||||
"1 snapshot(s) cannot be restored"
|
||||
|
||||
err := env.v.RemoteInfo(false)
|
||||
require.ErrorContains(t, err, wantError)
|
||||
|
||||
text := env.stdout.String()
|
||||
assert.Contains(t, text, "Missing from remote: 1 (5.0 MB)")
|
||||
assert.Contains(t, text, "Snapshots that cannot be restored (blobs missing):\n"+
|
||||
" "+brokenKey+"\n")
|
||||
assert.NotContains(t, text, snapshot.RemoteSnapshotKey(listLocalID)+"\n")
|
||||
|
||||
env.stdout.Reset()
|
||||
require.ErrorContains(t, env.v.RemoteInfo(true), wantError)
|
||||
|
||||
var doc map[string]any
|
||||
|
||||
require.NoError(t, json.Unmarshal(env.stdout.Bytes(), &doc))
|
||||
assert.InDelta(t, 1, doc["missing_blob_count"], 0)
|
||||
assert.InDelta(t, fiveMegabytes, doc["missing_blob_size"], 0)
|
||||
assert.Equal(t, []any{brokenKey}, doc["snapshots_missing_blobs"])
|
||||
|
||||
// --quiet drops the report, not the failure.
|
||||
env.stdout.Reset()
|
||||
env.v.UI.SetQuiet(true)
|
||||
require.ErrorContains(t, env.v.RemoteInfo(false), wantError)
|
||||
assert.Empty(t, env.stdout.String())
|
||||
env.v.UI.SetQuiet(false)
|
||||
|
||||
// With blob A stored again nothing is missing.
|
||||
addBlob(t, env.store.testStorer, testBlobHashA)
|
||||
env.stdout.Reset()
|
||||
require.NoError(t, env.v.RemoteInfo(true))
|
||||
|
||||
doc = nil
|
||||
require.NoError(t, json.Unmarshal(env.stdout.Bytes(), &doc))
|
||||
assert.InDelta(t, 0, doc["missing_blob_count"], 0)
|
||||
assert.InDelta(t, 0, doc["missing_blob_size"], 0)
|
||||
assert.NotContains(t, doc, "snapshots_missing_blobs")
|
||||
}
|
||||
@@ -25,9 +25,13 @@ import (
|
||||
"sneak.berlin/go/vaultik/internal/types"
|
||||
)
|
||||
|
||||
// ErrFilesFailedRestore is returned when --skip-errors skipped files the
|
||||
// restore could not write. Every other file was restored, so the caller
|
||||
// can report it apart from a failed restore.
|
||||
var ErrFilesFailedRestore = errors.New("file(s) failed to restore")
|
||||
|
||||
// Sentinel errors for restore failures.
|
||||
var (
|
||||
errFilesFailedRestore = errors.New("file(s) failed to restore")
|
||||
errPathsNotInSnapshot = errors.New("requested path(s) not in the snapshot")
|
||||
errFilesFailedVerify = errors.New("files failed verification")
|
||||
errDecryptionKeyRequired = errors.New(
|
||||
@@ -201,9 +205,9 @@ func (v *Vaultik) Restore(opts *RestoreOptions) error {
|
||||
}
|
||||
|
||||
// finishRestore prints the restore summary and the post-restore
|
||||
// warnings, runs optional verification, and converts any failed-file
|
||||
// count into an error. Otherwise it returns unmatchedErr, the error for
|
||||
// the requested paths that matched nothing (nil when there are none).
|
||||
// warnings, runs optional verification, and returns the error the
|
||||
// restore ends with: one for the files --skip-errors skipped, one for
|
||||
// unmatchedErr, the requested paths that matched nothing, or nil.
|
||||
func (v *Vaultik) finishRestore(
|
||||
repos *database.Repositories,
|
||||
files []*database.File,
|
||||
@@ -250,11 +254,18 @@ func (v *Vaultik) finishRestore(
|
||||
}
|
||||
}
|
||||
|
||||
if result.FilesFailed > 0 {
|
||||
return fmt.Errorf("%d %w", result.FilesFailed, errFilesFailedRestore)
|
||||
switch {
|
||||
case result.FilesFailed > 0 && unmatchedErr != nil:
|
||||
// A requested path that matched nothing outweighs skipped files:
|
||||
// the error wraps only unmatchedErr, so the command does not exit
|
||||
// as a restore that only skipped files.
|
||||
return fmt.Errorf("%w; %d %s",
|
||||
unmatchedErr, result.FilesFailed, ErrFilesFailedRestore.Error())
|
||||
case result.FilesFailed > 0:
|
||||
return fmt.Errorf("%d %w", result.FilesFailed, ErrFilesFailedRestore)
|
||||
default:
|
||||
return unmatchedErr
|
||||
}
|
||||
|
||||
return unmatchedErr
|
||||
}
|
||||
|
||||
// warnIfOwnershipNotPreserved warns once if some entry restored under
|
||||
|
||||
@@ -80,7 +80,7 @@ func TestRestoreFreesBlobsNoPendingFileNeeds(t *testing.T) {
|
||||
TargetDir: restoreDir,
|
||||
SkipErrors: true,
|
||||
})
|
||||
require.ErrorIs(t, err, errFilesFailedRestore)
|
||||
require.ErrorIs(t, err, ErrFilesFailedRestore)
|
||||
|
||||
requireRestoredFiles(t, fs, restoreDir, originals)
|
||||
assert.LessOrEqualf(t, peak, 1,
|
||||
|
||||
@@ -49,8 +49,7 @@ var ErrFilesUnreadable = errors.New("file(s) could not be read")
|
||||
// prune because the scan found nothing in a configured path of a
|
||||
// snapshot, as in the mount point of a volume that is not mounted. The
|
||||
// snapshots are complete and kept, so the caller can report it apart
|
||||
// from a failed backup. A run that also skipped unreadable files
|
||||
// returns ErrFilesUnreadable instead.
|
||||
// from a failed backup.
|
||||
var ErrSourcePathEmpty = errors.New("source path(s) were empty")
|
||||
|
||||
// ErrVolumeNotMounted is returned when --prune skipped the post-backup
|
||||
@@ -59,8 +58,8 @@ var ErrSourcePathEmpty = errors.New("source path(s) were empty")
|
||||
// mounted there is not mounted, the run found no entry in it, or the
|
||||
// directory is no longer backed up. The snapshots are complete and
|
||||
// kept, so the caller can report it apart from a failed backup. A run
|
||||
// that also skipped unreadable files or found a configured path empty
|
||||
// returns ErrFilesUnreadable or ErrSourcePathEmpty instead.
|
||||
// that also found a configured path empty returns ErrSourcePathEmpty
|
||||
// instead.
|
||||
var ErrVolumeNotMounted = errors.New(
|
||||
"mount point(s) an earlier snapshot recorded were not found")
|
||||
|
||||
@@ -243,15 +242,29 @@ func (v *Vaultik) skipPostBackupPrune(
|
||||
if filesUnreadable > 0 {
|
||||
v.UI.Warningf("Skipped the post-backup prune because %d file(s) "+
|
||||
"could not be read; older snapshots are kept.", filesUnreadable)
|
||||
}
|
||||
|
||||
var err error
|
||||
|
||||
switch {
|
||||
case len(emptyPaths) > 0:
|
||||
err = fmt.Errorf("%d %w", len(emptyPaths), ErrSourcePathEmpty)
|
||||
case len(unmountedPaths) > 0:
|
||||
err = fmt.Errorf("%d %w", len(unmountedPaths), ErrVolumeNotMounted)
|
||||
default:
|
||||
return fmt.Errorf("%d %w", filesUnreadable, ErrFilesUnreadable)
|
||||
}
|
||||
|
||||
if len(emptyPaths) > 0 {
|
||||
return fmt.Errorf("%d %w", len(emptyPaths), ErrSourcePathEmpty)
|
||||
// An empty path or a missing volume can mean a whole volume is not
|
||||
// in the backup, which outweighs skipped files: the error wraps only
|
||||
// the former, so the command does not exit as a backup that only
|
||||
// skipped files.
|
||||
if filesUnreadable > 0 {
|
||||
return fmt.Errorf("%w; %d %s",
|
||||
err, filesUnreadable, ErrFilesUnreadable.Error())
|
||||
}
|
||||
|
||||
return fmt.Errorf("%d %w", len(unmountedPaths), ErrVolumeNotMounted)
|
||||
return err
|
||||
}
|
||||
|
||||
// finishCreateSnapshot ends a run whose snapshots are all complete and
|
||||
|
||||
@@ -349,7 +349,6 @@ func TestSnapshotListAndPurgePrintSizesLikeRemoteInfo(t *testing.T) {
|
||||
env.addRemote(t, listLocalID, start)
|
||||
require.NoError(t, env.v.Repositories.Snapshots.UpdateCounts(
|
||||
context.Background(), nil, listLocalID, 0, 0, 0, 0, fiveMegabytes))
|
||||
addBlob(t, env.store.testStorer, testBlobHashA)
|
||||
|
||||
require.NoError(t, env.v.RemoteInfo(false))
|
||||
assert.Contains(t, findTableRow(t, env.stdout.String(), remoteKey[:16]),
|
||||
|
||||
@@ -176,8 +176,11 @@ func TestBackupThroughSymlinkWhoseTargetIsGoneDoesNotPrune(t *testing.T) {
|
||||
|
||||
// A backup under --skip-errors --prune that both skipped a file it could
|
||||
// not read and found nothing in a configured path names the empty path
|
||||
// too when it skips the prune. See
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/388.
|
||||
// too when it skips the prune. It ends with the empty path's error, which
|
||||
// names the unreadable file too but does not wrap ErrFilesUnreadable, so
|
||||
// the command does not exit as a backup that only skipped files. See
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/388 and
|
||||
// https://git.eeqj.de/sneak/vaultik/issues/439.
|
||||
func TestBackupWithEmptySourcePathAndUnreadableFileNamesBoth(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
@@ -207,7 +210,10 @@ func TestBackupWithEmptySourcePathAndUnreadableFileNamesBoth(t *testing.T) {
|
||||
SkipErrors: true,
|
||||
Snapshots: []string{snapshotName},
|
||||
})
|
||||
require.ErrorIs(t, err, vaultik.ErrFilesUnreadable)
|
||||
require.ErrorIs(t, err, vaultik.ErrSourcePathEmpty)
|
||||
require.NotErrorIs(t, err, vaultik.ErrFilesUnreadable)
|
||||
require.EqualError(t, err,
|
||||
"1 source path(s) were empty; 1 file(s) could not be read")
|
||||
assert.Contains(t, env.out.String(),
|
||||
"Skipped the post-backup prune because 1 file(s) could not be read")
|
||||
assert.Contains(t, env.out.String(),
|
||||
|
||||
Reference in new issue
Block a user