Compare commits

...
Author SHA1 Message Date
sneak 6ed84bf6d8 Fail restore for a requested path that matches nothing (closes #309)
check / check (push) Waiting to run
snapshot restore said nothing about a requested path that matched no
entry in the snapshot (a typo, a relative path, or a path through a
configured snapshot path that is a symlink, whose entries are stored
under the resolved directory) and exited 0, also when nothing matched
at all. Each such path is now named in a warning, and restore returns
an error after restoring the paths that do match.

Making a path through a symlink match is not part of this change.
Judgement call: when files also fail to restore, that error is the one
returned; the unmatched paths are still named.

Model: opus-5-5
2026-10-13 08:34:09 +00:00
clawbot 35eed79b1a Check --keep-newer-than before the backup starts (closes #307)
check / check (push) Waiting to run
snapshot create read --keep-newer-than only in the post-backup prune,
so a value the duration parser rejects failed the run after the
snapshot was uploaded: a nightly cron line added a snapshot each night,
reported it as failed, and never deleted any. Without --prune the flag
was ignored. The command now checks the flag before it starts and fails
on an invalid duration or a missing --prune.

A failed post-backup prune is now wrapped in ErrPostBackupPruneFailed
and reported as "Backup completed, but post-backup prune failed: ..."
rather than as a failed snapshot creation. The exit status stays 1.

Model: opus-5-5
2026-10-13 09:44:44 +02:00
clawbot 94412d9ef3 Stop remote info counting a non-blob file under blobs/ (closes #295)
check / check (push) Waiting to run
remote info counted every key under blobs/ with four or more path parts
as a blob, while prune skips a name that is not a blob hash. A stray
file such as .DS_Store was reported as an orphaned blob, with advice to
run prune, which then deleted nothing. remote info now takes its blob
list from prune's listAllRemoteBlobs, so its blob and orphan figures,
in the table and in --json, cover exactly what prune would consider.

blobKeyParts moves to prune.go, its only remaining user.

Model: opus-5-5
2026-10-13 08:33:27 +02:00
clawbot 167c0a0746 Make snapshot remove fail for an unknown snapshot ID (closes #293)
check / check (push) Waiting to run
snapshot remove never checked that the ID existed: it deleted nothing,
reported the snapshot removed from the local database and from the
destination store, and exited 0, under --json, --dry-run and
--local-only too. It now looks the ID up in the local index and, only
when that has no row and --local-only is not set, on the destination
store; finding neither is an error. The command reports a removal from
the local database only when the local index had the row, and
remote_removed is true only when metadata was actually deleted.

Judgement call: under --local-only the destination store is not
consulted, so the snapshot must be in the local index.

Model: opus-5-5
2026-10-13 05:55:22 +02:00
clawbot 07c957f84b Bind the local index to the s3.* destination too (closes #298)
check / check (push) Waiting to run
The local index was bound to its destination only through storage_url.
With the s3.* form nothing was checked, so after s3.bucket or s3.prefix
changed the next backup treated every chunk as already stored, uploaded
nothing, and completed a snapshot that could not be restored.

Without storage_url, the binding now records the destination the s3.*
fields name, written as an s3:// URL from s3.endpoint, s3.bucket and
s3.prefix, and a change is refused with the same error and advice. The
advice line now names the s3.* fields as well as storage_url.

Judgement call: the fields are compared as written, so an equivalent
spelling (a trailing slash on the prefix, a scheme on the endpoint) is
also refused, as it already is for storage_url.

Model: opus-5-5
2026-10-13 03:48:36 +02:00
clawbot c9a519c233 Reject a retention duration longer than about 292 years (closes #289)
check / check (push) Failing after 3m36s
parseDuration multiplied each number by the length of its unit with no
limit, so from 293y (or 15251w, 3559mo, 106752d, or a sum such as
200y200y) the result wrapped to a negative duration. The purge cutoff
then lay in the future and every snapshot of the name was selected, for
--older-than and --keep-newer-than alike. Each part is now checked
before it is added, and a total past the largest time.Duration is
rejected as an invalid duration, so nothing is deleted. Go units were
already safe: time.ParseDuration rejects an overflow.

With --prune the backup still runs before the duration is read; only
the prune step fails.

Model: opus-5-5
2026-10-13 01:51:24 +02:00
clawbot fa56165fe4 Stop a cancelled backup at once under --skip-errors (closes #286)
check / check (push) Canceled after 0s
After Ctrl-C or SIGTERM, phase 2 of a --skip-errors backup treated the
cancellation error from each remaining file like an unreadable file: it
opened the file, printed an error line, counted it as failed and moved
on to the next. The processing loop now checks for cancellation before
each file, and an error from a file once the run is cancelled stops the
run instead of being skipped. The loop check is what stops a run
cancelled while an empty file is open, since an empty file has no
chunks and nothing reads the context while it is backed up.

The --skip-errors test helper now takes the scan's context.

Model: opus-5-5
2026-10-08 15:12:07 +02:00
clawbot 34f61fb408 Make remote nuke delete leftover .partial uploads (closes #281)
check / check (push) Canceled after 0s
The file and rclone listings skip an object whose name ends in
`.partial`, the temporary name a `file://` or rclone upload writes
before moving the object into place. `remote nuke` deletes only what
the listings return, so it left the `.partial` objects killed uploads
leave behind and still reported the destination store empty.

Storer gains DeletePartialUploads. The file and rclone backends remove
every `.partial` object under the prefix; S3 has none to remove, since
it shows an object only once its upload completes. `remote nuke` calls
it for `metadata/` and `blobs/` as its last step.

Empty directories under a `file://` destination are still left behind.

Model: opus-5-5
2026-10-08 14:17:20 +02:00
clawbot 05bf73c48a Never skip a local index error for a directory or symlink (closes #284)
check / check (push) Canceled after 0s
Under --skip-errors, an error recording a directory or symlink in the
local index was skipped like an unreadable file. The snapshot completed
without the entry, and a restore did not recreate it.

processFileWithErrorHandling now records a directory or symlink itself
and returns any error, which stops the backup with or without the flag.
The skip sees only a regular file's errors, so countFailedFile no longer
checks for a directory.

The test injects the error with a SQLite trigger that refuses the
directory's files row.

Model: opus-5-5
2026-10-08 12:59:26 +02:00
clawbot 79a73fa122 Count a file a backup could not store as failed (closes #280)
check / check (push) Canceled after 0s
A file that phase 1 of a backup counted and phase 2 could not open,
because it was unreadable under --skip-errors or removed in between,
was added to the unchanged count while its size stayed in
BytesScanned. The summary showed it as unchanged with its bytes backed
up, and the snapshots row's file_count and total_size included it.

The scanner now counts such a file in FilesFailed and takes its size
out of BytesScanned. The summary's files line adds "N failed", and
file_count leaves the file out.

A directory phase 2 cannot record is not counted as failed, since
phase 1 counts no directories; that case has no test.

Model: opus-5-5
2026-10-08 10:12:10 +02:00
clawbot 0d0368df81 Report unknown blob figures for a snapshot whose manifest cannot be read (closes #272)
check / check (push) Canceled after 0s
When remote info could not read a snapshot's manifest, the orphan
figures were unknown but the snapshot's row still gave 0 blobs and 0 B,
in the table and in --json. The row's blob count and blob size are now
unknown, and null in --json.

A directory with no manifest, as an interrupted backup leaves, still
shows 0: the orphan figures count its blobs as orphaned, so it
references none.

The constant holding the "unknown" text is renamed from countUnknown to
unknownText, since it now also stands for a size.

Model: opus-5-5
2026-10-08 08:46:11 +02:00
clawbot c06c4d201b Keep the progress line of a multi-path snapshot within 100% (closes #271)
check / check (push) Canceled after 0s
One progress reporter spans every path of a snapshot. Its counts of
files and bytes processed run across all the paths, while the totals
they were divided by were reset to each path's own, so a second path
smaller than the first showed more than 100% and an ETA of `unknown`.
The totals now add up over the paths scanned so far. The rate behind
the ETA restarts when each path's scan phase ends, so that phase does
not lower the rate while the path is processed; until then the rate is
the previous path's. SetTotalSize is renamed AddTotalSize because it
now adds.

The issue says the ETA went negative; it was computed negative and
printed as `unknown`.

Model: opus-5-5
2026-10-08 07:29:11 +02:00
clawbot 70f008a21a Document that --json skips the confirmation prompt (closes #268)
check / check (push) Canceled after 0s
snapshot remove and prune delete without asking under --json, because a
prompt on stdout would break the JSON document. Their --json help and
README entries named only --force as skipping the prompt. Both now say
--json skips it too. One test checks the --json help of both commands;
another runs prune with --json, no --force and empty stdin, and checks
that the unreferenced blob is deleted and stdout holds only the JSON
document.

Model: opus-5-5
2026-10-08 04:59:30 +02:00
39 changed files with 1808 additions and 192 deletions
+21 -9
View File
@@ -286,7 +286,9 @@ needed.
remove orphaned blobs from remote storage. By default keeps only the latest remove orphaned blobs from remote storage. By default keeps only the latest
snapshot per name; use `--keep-newer-than` for a rolling window. snapshot per name; use `--keep-newer-than` for a rolling window.
* `--keep-newer-than <duration>`: With `--prune`, keep snapshots newer than * `--keep-newer-than <duration>`: With `--prune`, keep snapshots newer than
this duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`) this duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`).
Without `--prune`, or with a value that is not a valid duration, the
command fails before the backup starts.
**`snapshot list`**: Show every snapshot known to this host — the union **`snapshot list`**: Show every snapshot known to this host — the union
of the local index and the backup destination store — with timestamps of the local index and the backup destination store — with timestamps
@@ -349,9 +351,11 @@ the backup destination store. Blobs are NOT touched — deleting blobs
requires reading every remaining remote manifest (the destination store requires reading every remaining remote manifest (the destination store
may hold snapshots this host doesn't know about), which is what may hold snapshots this host doesn't know about), which is what
`vaultik prune` does. On success the command prints the exact `vaultik `vaultik prune` does. On success the command prints the exact `vaultik
prune` invocation to run as a follow-up. Local row cleanup (files, prune` invocation to run as a follow-up. An ID that names no snapshot in
chunks, blobs the snapshot was the last referrer for) runs the local index or on the destination store is an error; with
automatically. If the destination store is unreachable, the local-DB `--local-only`, the snapshot must be in the local index. Local row
cleanup (files, chunks, blobs the snapshot was the last referrer for)
runs automatically. If the destination store is unreachable, the local-DB
removal still completes and a warning is emitted; run `vaultik snapshot removal still completes and a warning is emitted; run `vaultik snapshot
remove <snapshot-id>` again once the store is reachable to remove the remove <snapshot-id>` again once the store is reachable to remove the
snapshot's metadata from it (`vaultik prune` does not). To wipe everything snapshot's metadata from it (`vaultik prune` does not). To wipe everything
@@ -359,7 +363,8 @@ on the destination in one go, use `vaultik remote nuke --force`.
* `--local-only`: Skip remote cleanup; only touch the local index * `--local-only`: Skip remote cleanup; only touch the local index
* `--dry-run`: Show what would be deleted without deleting * `--dry-run`: Show what would be deleted without deleting
* `--force`: Skip confirmation prompt * `--force`: Skip confirmation prompt
* `--json`: Output result as JSON * `--json`: Output result as JSON. Also skips the confirmation prompt, as
`--force` does.
**`snapshot restore`**: Restore files from a backup snapshot. **`snapshot restore`**: Restore files from a backup snapshot.
* Requires `VAULTIK_AGE_SECRET_KEY` environment variable * Requires `VAULTIK_AGE_SECRET_KEY` environment variable
@@ -367,7 +372,9 @@ on the destination in one go, use `vaultik remote nuke --force`.
store — its remote key (or an unambiguous leading part of it) as shown store — its remote key (or an unambiguous leading part of it) as shown
by `snapshot list`. See by `snapshot list`. See
[restoring on another machine](#restoring-on-another-machine). [restoring on another machine](#restoring-on-another-machine).
* Optional path arguments to restore specific files/directories (default: all) * Optional path arguments to restore specific files/directories (default: all).
A path that matches nothing in the snapshot is named in a warning, and the
command exits non-zero after restoring the paths that match
* Preserves file permissions, timestamps, ownership (ownership requires root), * Preserves file permissions, timestamps, ownership (ownership requires root),
symlinks, and empty directories symlinks, and empty directories
* `--verify`: After restoring, verify every file's chunk hashes match * `--verify`: After restoring, verify every file's chunk hashes match
@@ -383,7 +390,8 @@ manifests — network cost scales with the number of snapshots. `snapshot
create --prune` runs the same cleanup automatically; this is the create --prune` runs the same cleanup automatically; this is the
manual entry point for the same work. manual entry point for the same work.
* `--force`: Skip confirmation prompt * `--force`: Skip confirmation prompt
* `--json`: Output stats as JSON * `--json`: Output stats as JSON. Also skips the confirmation prompt, as
`--force` does.
**`info`**: Display system configuration, storage settings, encryption **`info`**: Display system configuration, storage settings, encryption
recipients, and local database statistics. recipients, and local database statistics.
@@ -396,7 +404,10 @@ key is skipped with a warning and is not printed. If a listed
orphaned blob figures are reported as unknown; `--json` gives them as orphaned blob figures are reported as unknown; `--json` gives them as
`null`, lists the remote key of each unreadable manifest in `null`, lists the remote key of each unreadable manifest in
`unreadable_manifests` and counts the manifests under skipped names in `unreadable_manifests` and counts the manifests under skipped names in
`skipped_manifest_count`. `skipped_manifest_count`. An unreadable manifest also leaves its
snapshot's blob count and blob size unknown, `null` in `--json`. A file
under `blobs/` whose name is not a blob hash is not counted as a blob,
since `prune` never deletes it.
* `--json`: Output as JSON * `--json`: Output as JSON
**`remote nuke`**: Delete every snapshot's metadata and every blob from the **`remote nuke`**: Delete every snapshot's metadata and every blob from the
@@ -409,7 +420,8 @@ destination store.
entirely. Remote storage is unaffected; the next backup will do a full entirely. Remote storage is unaffected; the next backup will do a full
scan and re-deduplicate against existing remote blobs, and the local scan and re-deduplicate against existing remote blobs, and the local
index will re-bind to the currently configured storage destination. index will re-bind to the currently configured storage destination.
Use this after changing `storage_url` to a different destination. Use this after changing `storage_url`, or `s3.endpoint`, `s3.bucket` or
`s3.prefix`, to a different destination.
* `--force`: Skip confirmation prompt * `--force`: Skip confirmation prompt
--- ---
+98
View File
@@ -22,6 +22,96 @@ the tag exists and is exercised; what is left is merging `next` to
# Completed Steps # Completed Steps
- 2026-10-13: Stopped `remote info` counting a file under `blobs/` that
is not a blob as an orphaned blob
([issue #295](https://git.eeqj.de/sneak/vaultik/issues/295)). A stray
file such as `.DS_Store` was reported as an orphaned blob and the
report advised running `prune`, which skips such a file and deleted
nothing. `remote info` now takes its blob list from the same listing
`prune` uses, in the table and in `--json`.
- 2026-10-13: Made `snapshot remove` fail for a snapshot that does not
exist ([issue #293](https://git.eeqj.de/sneak/vaultik/issues/293)).
The command never checked the ID: it reported the snapshot removed
from the local database and from the destination store and exited 0,
under `--json`, `--dry-run` and `--local-only` too. An ID that names
no snapshot in the local index or on the destination store is now an
error; with `--local-only`, the snapshot must be in the local index.
The command now reports a removal from either side only when one
happened, and `remote_removed` is true only when metadata was deleted.
- 2026-10-12: Bound the local index to the destination an `s3.*` config
names ([issue #298](https://git.eeqj.de/sneak/vaultik/issues/298)).
Only `storage_url` was checked, so after `s3.bucket` or `s3.prefix`
changed, the next backup treated every chunk as already stored,
uploaded nothing, and completed a snapshot that could not be restored.
Without `storage_url`, the local index is now bound to `s3.endpoint`,
`s3.bucket` and `s3.prefix`, and a change is refused as a changed
`storage_url` is.
- 2026-10-12: Rejected a retention duration longer than about 292 years
([issue #289](https://git.eeqj.de/sneak/vaultik/issues/289)). The
number given to `--older-than` or `--keep-newer-than` was multiplied
by the length of its unit with no limit, so from `293y`, `15251w`,
`3559mo` or `106752d`, or a sum of parts such as `200y200y`, the
result wrapped to a negative duration. The cutoff then lay in the
future, every snapshot of the name was deleted, the one just made
included, and the command exited 0. Such a duration is now rejected
as invalid and nothing is deleted.
- 2026-10-08: Made a backup cancelled under `--skip-errors` stop at once
([issue #286](https://git.eeqj.de/sneak/vaultik/issues/286)). After
Ctrl-C or SIGTERM, phase 2 treated the cancellation error from each
remaining file like an unreadable file: it opened the file, printed an
error line for it, counted it as failed and went on to the next one.
The run now stops at the first file after the cancellation, and no
file is counted as failed because of it.
- 2026-10-08: Made `remote nuke` delete the `.partial` files that
uploads cut off part-way leave on the destination store
([issue #281](https://git.eeqj.de/sneak/vaultik/issues/281)). The
file and rclone listings skip such a file, so the command left it in
place and still reported the store empty. It now removes them under
`metadata/` and `blobs/` as its last step.
- 2026-10-08: Made a local index error while recording a directory or
symlink stop a backup under `--skip-errors`
([issue #284](https://git.eeqj.de/sneak/vaultik/issues/284)). Phase 2
only records such an entry in the local index, since it has no data
to open or read, but an error doing so was skipped like an unreadable
file. The snapshot then completed without the entry, and a restore
did not recreate it. The error now stops the backup, as it does
without the flag.
- 2026-10-08: Counted a file that a backup could not store as failed
([issue #280](https://git.eeqj.de/sneak/vaultik/issues/280)). A file
that phase 1 counted and phase 2 could not open, because it was
unreadable under `--skip-errors` or removed in between, was reported
in the summary as unchanged with its bytes as backed up, and the
`snapshots` row's `file_count` and `total_size` included it. The
summary now counts it as failed, and its data total and the row leave
it out.
- 2026-10-08: Made `remote info` report a snapshot's blob count and
blob size as unknown when its manifest cannot be read
([issue #272](https://git.eeqj.de/sneak/vaultik/issues/272)). The
orphan figures were already unknown in that case, but the snapshot's
row still gave 0 blobs and 0 B, in the table and in `--json`. The row
now reads `unknown` and `--json` gives `null`. A directory with no
manifest still shows 0, since the orphan figures count its blobs as
orphaned.
- 2026-10-08: Kept the progress line of a snapshot with more than one
path within 100%
([issue #271](https://git.eeqj.de/sneak/vaultik/issues/271)). The
bytes and files processed counted every path, but the totals they
were divided by held only the current path's, so a second path
smaller than the first showed more than 100% and an ETA of `unknown`.
The totals now add up over the paths scanned so far. The rate behind
the ETA restarts when each path's scan phase ends, so that phase does
not lower the rate while the path is processed; until then the rate
is the previous path's.
- 2026-10-08: Made a second `snapshot create` of one name succeed when - 2026-10-08: Made a second `snapshot create` of one name succeed when
it starts in the same second as the first it starts in the same second as the first
([issue #270](https://git.eeqj.de/sneak/vaultik/issues/270)). The ([issue #270](https://git.eeqj.de/sneak/vaultik/issues/270)). The
@@ -30,6 +120,14 @@ the tag exists and is exercised; what is left is merging `next` to
snapshots.id`. When the local index already has a snapshot with the snapshots.id`. When the local index already has a snapshot with the
ID, the create now waits a second and takes a new timestamp. ID, the create now waits a second and takes a new timestamp.
- 2026-10-07: Documented that `--json` skips the confirmation prompt of
`snapshot remove` and `prune`
([issue #268](https://git.eeqj.de/sneak/vaultik/issues/268)). Both
commands delete without asking under `--json`, since a prompt on stdout
would break the JSON document, but the help and the README described
only `--force` as skipping it. The `--json` help of both commands and
their README entries now say so.
- 2026-10-07: Made a symlink whose target cannot be read stop the backup - 2026-10-07: Made a symlink whose target cannot be read stop the backup
([issue #269](https://git.eeqj.de/sneak/vaultik/issues/269)). It was ([issue #269](https://git.eeqj.de/sneak/vaultik/issues/269)). It was
left out of the snapshot with only a debug log line, even without left out of the snapshot with only a debug log line, even without
@@ -0,0 +1,108 @@
package cli //nolint:testpackage // shares runEntry and the hermetic config helpers
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Flags of `snapshot create` used to build argument vectors below.
const (
flagPrune = "--prune"
flagKeepNewerThan = "--keep-newer-than"
)
// TestEntrySnapshotCreateRejectsKeepNewerThanBeforeBackup checks that a
// --keep-newer-than the post-backup prune could not use stops `snapshot
// create` before the backup: the command fails and the destination
// store stays empty. It used to be read only by the prune, after the
// snapshot was uploaded, and without --prune it was ignored.
//
//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
func TestEntrySnapshotCreateRejectsKeepNewerThanBeforeBackup(t *testing.T) {
for _, testCase := range []struct {
name string
args []string
}{
{
name: "decimal in a day count",
args: []string{flagPrune, flagKeepNewerThan, "1.5d"},
},
{
name: "unknown unit",
args: []string{flagPrune, flagKeepNewerThan, "4x"},
},
{
name: "without --prune",
args: []string{flagKeepNewerThan, "4w"},
},
} {
t.Run(testCase.name, func(t *testing.T) {
configPath, storeDir := writeBackupConfig(t)
code, _, _ := runEntry(t, append(
[]string{flagConfig, configPath, cmdSnapshot, cmdCreate},
testCase.args...)...)
assert.Equal(t, 1, code)
stored, err := os.ReadDir(storeDir)
require.NoError(t, err)
assert.Empty(t, stored, "nothing may be uploaded")
})
}
}
// TestEntrySnapshotCreateReportsPostBackupPruneFailure makes the
// post-backup prune fail after a good backup, by planting a manifest
// that cannot be decoded, and checks that the failure is reported as the
// prune's and not as a failed snapshot creation.
//
//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
func TestEntrySnapshotCreateReportsPostBackupPruneFailure(t *testing.T) {
configPath, storeDir := writeBackupConfig(t)
// A remote key is 64 lowercase hex characters. The prune reads every
// manifest on the store and fails on one it cannot decode.
plantedDir := filepath.Join(storeDir, "metadata", strings.Repeat("a", 64))
require.NoError(t, os.MkdirAll(plantedDir, 0o750))
require.NoError(t, os.WriteFile(
filepath.Join(plantedDir, "manifest.json.zst"),
[]byte("not a valid manifest"), 0o600))
code, _, stderr := runEntry(t,
flagConfig, configPath, cmdSnapshot, cmdCreate, flagPrune)
assert.Equal(t, 1, code)
assert.Contains(t, stderr, "Backup completed, but post-backup prune failed")
assert.NotContains(t, stderr, "Snapshot creation failed")
snapshots, err := os.ReadDir(filepath.Join(storeDir, "metadata"))
require.NoError(t, err)
assert.Len(t, snapshots, 2,
"the new snapshot is on the store beside the planted one")
}
// writeBackupConfig builds a hermetic config with one small file in its
// source directory and an empty destination store directory. Returns the
// config path and the store directory.
func writeBackupConfig(t *testing.T) (string, string) {
t.Helper()
configPath := writeHermeticPruneConfig(t, false)
// writeHermeticPruneConfig puts the source and store directories
// beside the config file.
dir := filepath.Dir(configPath)
sourceDir := filepath.Join(dir, "source")
require.NoError(t, os.Mkdir(sourceDir, 0o750))
require.NoError(t, os.WriteFile(
filepath.Join(sourceDir, "file.txt"), []byte("hello"), 0o600))
return configPath, filepath.Join(dir, "store")
}
+24
View File
@@ -0,0 +1,24 @@
package cli //nolint:testpackage // exercises the unexported command constructor
import (
"testing"
"github.com/spf13/cobra"
"github.com/stretchr/testify/assert"
)
// TestJSONHelpSaysConfirmationPromptIsSkipped checks that the --json help
// of `snapshot remove` and `prune` says the flag skips the confirmation
// prompt. Both delete without asking under --json, since a prompt on
// stdout would break the JSON document.
func TestJSONHelpSaysConfirmationPromptIsSkipped(t *testing.T) {
t.Parallel()
for _, cmd := range []*cobra.Command{
newSnapshotRemoveCommand(),
NewPruneCommand(),
} {
assert.Contains(t, cmd.Flags().Lookup("json").Usage,
"skips the confirmation prompt", cmd.Name())
}
}
+2 -1
View File
@@ -57,7 +57,8 @@ referenced.`,
} }
cmd.Flags().BoolVar(&opts.Force, "force", false, "Skip confirmation prompt") cmd.Flags().BoolVar(&opts.Force, "force", false, "Skip confirmation prompt")
cmd.Flags().BoolVar(&opts.JSON, "json", false, "Output pruning stats as JSON") cmd.Flags().BoolVar(&opts.JSON, "json", false,
"Output pruning stats as JSON; skips the confirmation prompt, as --force does")
return cmd return cmd
} }
+19 -2
View File
@@ -71,6 +71,11 @@ otherwise from the platform config directory (~/.config/vaultik/config.yml
on Linux), then /etc/vaultik/config.yml.`, on Linux), then /etc/vaultik/config.yml.`,
Args: cobra.ArbitraryArgs, Args: cobra.ArbitraryArgs,
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
err := opts.Validate()
if err != nil {
return err
}
// Pass snapshot names from args // Pass snapshot names from args
opts.Snapshots = args opts.Snapshots = args
// --skip-errors is a global flag on the root command. // --skip-errors is a global flag on the root command.
@@ -97,6 +102,13 @@ on Linux), then /etc/vaultik/config.yml.`,
}, func(v *vaultik.Vaultik) error { }, func(v *vaultik.Vaultik) error {
return v.CreateSnapshot(opts) return v.CreateSnapshot(opts)
}, func(err error) { }, func(err error) {
if errors.Is(err, vaultik.ErrPostBackupPruneFailed) {
log.Error("Post-backup prune failed", "error", err)
ReportErrorf("Backup completed, but %v", err)
return
}
log.Error("Snapshot creation failed", "error", err) log.Error("Snapshot creation failed", "error", err)
ReportErrorf("Snapshot creation failed: %v", err) ReportErrorf("Snapshot creation failed: %v", err)
}) })
@@ -255,8 +267,12 @@ manifest (the destination store may hold snapshots this host doesn't
know about), which is what 'vaultik prune' does. On success the command know about), which is what 'vaultik prune' does. On success the command
prints the exact 'vaultik prune' invocation to run as a follow-up. prints the exact 'vaultik prune' invocation to run as a follow-up.
An ID that names no snapshot in the local index or on the destination
store is an error.
Use --local-only to skip the remote half (e.g. when you want to forget a Use --local-only to skip the remote half (e.g. when you want to forget a
snapshot locally without touching the destination store). snapshot locally without touching the destination store); the snapshot
must then be in the local index.
If the remote is unreachable, the local-database removal still completes If the remote is unreachable, the local-database removal still completes
and a warning is emitted; run 'vaultik snapshot remove <snapshot-id>' again and a warning is emitted; run 'vaultik snapshot remove <snapshot-id>' again
@@ -280,7 +296,8 @@ nuke --force' — it is the single supported entry point for that.`,
cmd.Flags().BoolVarP(&opts.Force, "force", "f", false, "Skip confirmation prompt") cmd.Flags().BoolVarP(&opts.Force, "force", "f", false, "Skip confirmation prompt")
cmd.Flags().BoolVar(&opts.DryRun, "dry-run", false, cmd.Flags().BoolVar(&opts.DryRun, "dry-run", false,
"Show what would be removed without removing") "Show what would be removed without removing")
cmd.Flags().BoolVar(&opts.JSON, "json", false, "Output result as JSON") cmd.Flags().BoolVar(&opts.JSON, "json", false,
"Output result as JSON; skips the confirmation prompt, as --force does")
cmd.Flags().BoolVar(&opts.LocalOnly, "local-only", false, cmd.Flags().BoolVar(&opts.LocalOnly, "local-only", false,
"Skip remote cleanup; only touch the local index") "Skip remote cleanup; only touch the local index")
+2
View File
@@ -30,6 +30,8 @@ This command will restore files from the specified snapshot to the
target directory. target directory.
If no paths are specified, all files are restored. If no paths are specified, all files are restored.
If paths are specified, only matching files/directories are restored. If paths are specified, only matching files/directories are restored.
A path that matches nothing in the snapshot is named in a warning, and
the command exits non-zero after restoring the paths that match.
The snapshot may be named by its ID or, when restoring on a host with no The snapshot may be named by its ID or, when restoring on a host with no
local index, by the remote key that 'snapshot list' prints for a local index, by the remote key that 'snapshot list' prints for a
+69 -53
View File
@@ -56,23 +56,27 @@ const (
// ProgressStats holds atomic counters for progress tracking // ProgressStats holds atomic counters for progress tracking
type ProgressStats struct { type ProgressStats struct {
FilesScanned atomic.Int64 // Total files seen during scan (includes skipped) FilesScanned atomic.Int64 // Total files seen during scan (includes skipped)
FilesProcessed atomic.Int64 // Files actually processed in phase 2 FilesProcessed atomic.Int64 // Files actually processed in phase 2
FilesSkipped atomic.Int64 // Files skipped due to no changes FilesSkipped atomic.Int64 // Files skipped due to no changes
BytesScanned atomic.Int64 // Bytes from new/changed files only BytesScanned atomic.Int64 // Bytes from new/changed files only
BytesSkipped atomic.Int64 // Bytes from unchanged files BytesSkipped atomic.Int64 // Bytes from unchanged files
BytesProcessed atomic.Int64 // Actual bytes processed (for ETA calculation) BytesProcessed atomic.Int64 // Actual bytes processed (for ETA calculation)
ChunksCreated atomic.Int64 ChunksCreated atomic.Int64
BlobsCreated atomic.Int64 BlobsCreated atomic.Int64
BlobsUploaded atomic.Int64 BlobsUploaded atomic.Int64
BytesUploaded atomic.Int64 BytesUploaded atomic.Int64
CurrentFile atomic.Value // stores string CurrentFile atomic.Value // stores string
TotalSize atomic.Int64 // Total size to process (set after scan phase) TotalSize atomic.Int64 // Size to process in the paths scanned so far
TotalFiles atomic.Int64 // Total files to process in phase 2 TotalFiles atomic.Int64 // Files to process in the paths scanned so far
ProcessStartTime atomic.Value // stores time.Time when processing starts StartTime time.Time
StartTime time.Time mu sync.RWMutex
mu sync.RWMutex lastDetailTime time.Time
lastDetailTime time.Time
// Guarded by mu: when the last scan phase ended, and
// BytesProcessed at that moment.
processStartTime time.Time
processStartBytes int64
// Upload tracking // Upload tracking
CurrentUpload atomic.Value // stores *UploadInfo CurrentUpload atomic.Value // stores *UploadInfo
@@ -148,10 +152,36 @@ func (pr *ProgressReporter) GetStats() *ProgressStats {
return pr.stats return pr.stats
} }
// SetTotalSize sets the total size to process (after scan phase) // AddTotalSize adds the size one path of the snapshot has to process to
func (pr *ProgressReporter) SetTotalSize(size int64) { // the total, once that path's scan phase is done, and starts measuring
pr.stats.TotalSize.Store(size) // the processing rate again. The processed counts run across every path,
pr.stats.ProcessStartTime.Store(time.Now().UTC()) // so the total does too. Restarting the rate keeps a path's scan phase,
// which processes nothing, out of the rate while that path is processed.
// During a later path's scan phase the rate is still the previous
// path's, and falls as that scan goes on.
func (pr *ProgressReporter) AddTotalSize(size int64) {
pr.stats.TotalSize.Add(size)
pr.stats.mu.Lock()
defer pr.stats.mu.Unlock()
pr.stats.processStartTime = time.Now().UTC()
pr.stats.processStartBytes = pr.stats.BytesProcessed.Load()
}
// processRate returns the bytes processed per second since the last scan
// phase ended, or 0 before the first path's scan phase is done.
func (s *ProgressStats) processRate() float64 {
s.mu.RLock()
defer s.mu.RUnlock()
if s.processStartTime.IsZero() {
return 0
}
processed := s.BytesProcessed.Load() - s.processStartBytes
return float64(processed) / time.Since(s.processStartTime).Seconds()
} }
// Helper functions // Helper functions
@@ -357,19 +387,12 @@ func (pr *ProgressReporter) printSummaryStatus() {
// Calculate ETA if we have total size and are processing // Calculate ETA if we have total size and are processing
etaStr := "" etaStr := ""
if totalSize > 0 && bytesProcessed > 0 { processRate := pr.stats.processRate()
processStart, ok := pr.stats.ProcessStartTime.Load().(time.Time) if totalSize > 0 && processRate > 0 {
if ok && !processStart.IsZero() { remainingBytes := totalSize - bytesProcessed
processElapsed := time.Since(processStart) remainingSeconds := float64(remainingBytes) / processRate
eta := time.Duration(remainingSeconds * float64(time.Second))
rate := float64(bytesProcessed) / processElapsed.Seconds() etaStr = " | ETA: " + formatDuration(eta)
if rate > 0 {
remainingBytes := totalSize - bytesProcessed
remainingSeconds := float64(remainingBytes) / rate
eta := time.Duration(remainingSeconds * float64(time.Second))
etaStr = " | ETA: " + formatDuration(eta)
}
}
} }
rate := float64(bytesScanned+bytesSkipped) / elapsed.Seconds() rate := float64(bytesScanned+bytesSkipped) / elapsed.Seconds()
@@ -421,25 +444,18 @@ func (pr *ProgressReporter) printDetailedStatus() {
log.Info("Elapsed time", "duration", formatDuration(elapsed)) log.Info("Elapsed time", "duration", formatDuration(elapsed))
// Calculate and show ETA if we have data // Calculate and show ETA if we have data
if totalSize > 0 && bytesProcessed > 0 { processRate := pr.stats.processRate()
processStart, ok := pr.stats.ProcessStartTime.Load().(time.Time) if totalSize > 0 && processRate > 0 {
if ok && !processStart.IsZero() { remainingBytes := totalSize - bytesProcessed
processElapsed := time.Since(processStart) remainingSeconds := float64(remainingBytes) / processRate
eta := time.Duration(remainingSeconds * float64(time.Second))
processRate := float64(bytesProcessed) / processElapsed.Seconds() percentComplete := float64(bytesProcessed) / float64(totalSize) * percentScale
if processRate > 0 { log.Info("Overall progress",
remainingBytes := totalSize - bytesProcessed "percent", fmt.Sprintf("%.1f%%", percentComplete),
remainingSeconds := float64(remainingBytes) / processRate "processed", humanize.Bytes(safeUint64(bytesProcessed)),
eta := time.Duration(remainingSeconds * float64(time.Second)) "total", humanize.Bytes(safeUint64(totalSize)),
percentComplete := float64(bytesProcessed) / float64(totalSize) * percentScale "rate", humanize.Bytes(uint64(processRate))+"/s",
log.Info("Overall progress", "eta", formatDuration(eta))
"percent", fmt.Sprintf("%.1f%%", percentComplete),
"processed", humanize.Bytes(safeUint64(bytesProcessed)),
"total", humanize.Bytes(safeUint64(totalSize)),
"rate", humanize.Bytes(uint64(processRate))+"/s",
"eta", formatDuration(eta))
}
}
} }
log.Info("Files processed", log.Info("Files processed",
+54
View File
@@ -0,0 +1,54 @@
//nolint:testpackage // exercises the unexported processRate helper
package snapshot
import (
"testing"
"time"
)
// TestProcessRateNotLoweredByLaterScanPhase feeds the reporter two paths
// that each take 10 seconds to process, the second after a 30-second scan
// phase. That scan phase processes nothing, so the rate while the second
// path is processed must be that path's own.
func TestProcessRateNotLoweredByLaterScanPhase(t *testing.T) {
t.Parallel()
const (
pathSize = 1000
processingTime = 10 * time.Second
scanTime = 30 * time.Second
)
// Never started, but Stop releases its tickers and signal handler.
progress := NewProgressReporter()
defer progress.Stop()
stats := progress.GetStats()
// Moves the processing start time back instead of sleeping.
elapse := func(d time.Duration) {
stats.mu.Lock()
defer stats.mu.Unlock()
stats.processStartTime = stats.processStartTime.Add(-d)
}
progress.AddTotalSize(pathSize)
stats.BytesProcessed.Add(pathSize)
elapse(processingTime)
elapse(scanTime)
progress.AddTotalSize(pathSize)
stats.BytesProcessed.Add(pathSize)
elapse(processingTime)
want := pathSize / processingTime.Seconds()
got := stats.processRate()
// The test's own run time adds to the elapsed time, so got is a hair
// under want.
if got > want || got < want*0.99 {
t.Errorf("rate while the second path is processed is %.1f bytes/s, "+
"want %.1f", got, want)
}
}
+94
View File
@@ -0,0 +1,94 @@
package snapshot_test
import (
"context"
"path/filepath"
"strings"
"testing"
"github.com/spf13/afero"
"sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/snapshot"
)
// TestProgressPercentWithSecondPathSmaller backs up two paths with one
// scanner, as a snapshot with two paths does, the second path smaller
// than the first. The progress line divides the bytes processed by the
// total size, so both must count both paths to stay within 100%.
func TestProgressPercentWithSecondPathSmaller(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
files := map[string]string{
"/large/one.txt": strings.Repeat("1", 4000),
"/large/two.txt": strings.Repeat("2", 4000),
"/small/three.txt": strings.Repeat("3", 1000),
}
for path, content := range files {
err := fs.MkdirAll(filepath.Dir(path), 0755)
if err != nil {
t.Fatalf("mkdir: %v", err)
}
err = afero.WriteFile(fs, path, []byte(content), 0644)
if err != nil {
t.Fatalf("write %s: %v", path, err)
}
}
db, err := database.NewTestDB()
if err != nil {
t.Fatalf("create test db: %v", err)
}
t.Cleanup(func() {
cerr := db.Close()
if cerr != nil {
t.Errorf("close db: %v", cerr)
}
})
repos := database.NewRepositories(db)
scanner := snapshot.NewScanner(snapshot.ScannerConfig{
FS: fs,
ChunkSize: int64(1024 * 16),
Repositories: repos,
MaxBlobSize: int64(1024 * 1024),
CompressionLevel: 3,
AgeRecipients: []string{testAgePublicKey},
EnableProgress: true,
})
// Never started, but Stop releases its tickers and signal handler.
progress := scanner.GetProgress()
defer progress.Stop()
ctx := context.Background()
snapshotID := "test-snapshot-progress"
createTestSnapshotRecord(ctx, t, repos, snapshotID)
for _, path := range []string{"/large", "/small"} {
_, err := scanner.Scan(ctx, path, snapshotID)
if err != nil {
t.Fatalf("scanning %s: %v", path, err)
}
}
stats := progress.GetStats()
// Directories count toward the total size but produce no chunks, so
// the percentage ends just under 100%.
percent := float64(stats.BytesProcessed.Load()) /
float64(stats.TotalSize.Load()) * 100
if percent > 100 {
t.Errorf("progress after both paths is %.1f%%, want at most 100%%",
percent)
}
if stats.FilesProcessed.Load() != stats.TotalFiles.Load() {
t.Errorf("progress after both paths is %d of %d files, want all",
stats.FilesProcessed.Load(), stats.TotalFiles.Load())
}
}
+45 -14
View File
@@ -133,10 +133,13 @@ type ScannerConfig struct {
// ScanResult contains the results of a scan operation. Files and bytes // ScanResult contains the results of a scan operation. Files and bytes
// are counted per file: BytesScanned is the size of the new and changed // are counted per file: BytesScanned is the size of the new and changed
// files, BytesSkipped that of the unchanged ones. // files, BytesSkipped that of the unchanged ones. FilesFailed counts the
// new and changed files that phase 2 could not store; FilesScanned
// includes them and BytesScanned does not.
type ScanResult struct { type ScanResult struct {
FilesScanned int FilesScanned int
FilesSkipped int FilesSkipped int
FilesFailed int
FilesDeleted int FilesDeleted int
BytesScanned int64 BytesScanned int64
BytesSkipped int64 BytesSkipped int64
@@ -407,8 +410,8 @@ func (s *Scanner) summarizeScanPhase(
} }
if s.progress != nil { if s.progress != nil {
s.progress.SetTotalSize(totalSizeToProcess) s.progress.AddTotalSize(totalSizeToProcess)
s.progress.GetStats().TotalFiles.Store(int64(len(filesToProcess))) s.progress.GetStats().TotalFiles.Add(int64(len(filesToProcess)))
} }
log.Info("Phase 1 complete", log.Info("Phase 1 complete",
@@ -1309,6 +1312,13 @@ func (s *Scanner) processPhase(
// Process each file // Process each file
for _, fileToProcess := range filesToProcess { for _, fileToProcess := range filesToProcess {
// Check context cancellation
select {
case <-ctx.Done():
return ctx.Err()
default:
}
// Update progress // Update progress
if s.progress != nil { if s.progress != nil {
s.progress.GetStats().CurrentFile.Store(fileToProcess.Path) s.progress.GetStats().CurrentFile.Store(fileToProcess.Path)
@@ -1345,13 +1355,32 @@ func (s *Scanner) processPhase(
return s.finalizeProcessPhase(ctx, result) return s.finalizeProcessPhase(ctx, result)
} }
// processFileWithErrorHandling wraps processFileStreaming with error recovery for // processFileWithErrorHandling records a directory or symlink, or wraps
// deleted files and skip-errors mode. Returns (skipped, error). // processFileStreaming for a regular file with error recovery for deleted
// files and skip-errors mode. Returns (skipped, error).
func (s *Scanner) processFileWithErrorHandling( func (s *Scanner) processFileWithErrorHandling(
ctx context.Context, fileToProcess *FileToProcess, result *ScanResult, ctx context.Context, fileToProcess *FileToProcess, result *ScanResult,
) (bool, error) { ) (bool, error) {
// A directory or symlink has no data to open or read; it is only
// recorded in the local index. An error recording it stops the run
// even under --skip-errors, or the snapshot would complete without it.
mode := os.FileMode(fileToProcess.File.Mode)
if mode&os.ModeSymlink != 0 || mode.IsDir() {
err := s.recordNonRegularFile(ctx, fileToProcess)
if err != nil {
return false, fmt.Errorf("processing file %s: %w", fileToProcess.Path, err)
}
return false, nil
}
err := s.processFileStreaming(ctx, fileToProcess, result) err := s.processFileStreaming(ctx, fileToProcess, result)
if err != nil { if err != nil {
// A cancelled run stops here even under --skip-errors, rather than
// counting this file as failed and going on to the next one.
if ctx.Err() != nil {
return false, fmt.Errorf("processing file %s: %w", fileToProcess.Path, err)
}
// A packer/database/encryption/upload failure means the chunk's data // A packer/database/encryption/upload failure means the chunk's data
// may not have been stored. Skipping the file would let the snapshot // may not have been stored. Skipping the file would let the snapshot
// record a file whose chunk is in no blob and cannot be restored, so // record a file whose chunk is in no blob and cannot be restored, so
@@ -1365,7 +1394,7 @@ func (s *Scanner) processFileWithErrorHandling(
log.Warn("File was deleted during backup, skipping", log.Warn("File was deleted during backup, skipping",
"path", fileToProcess.Path) "path", fileToProcess.Path)
result.FilesSkipped++ countFailedFile(fileToProcess, result)
return true, nil return true, nil
} }
@@ -1376,7 +1405,7 @@ func (s *Scanner) processFileWithErrorHandling(
s.ui.Errorf("Failed to process %s: %v. Skipping (--skip-errors).", s.ui.Errorf("Failed to process %s: %v. Skipping (--skip-errors).",
s.ui.Path(fileToProcess.Path), err) s.ui.Path(fileToProcess.Path), err)
result.FilesSkipped++ countFailedFile(fileToProcess, result)
return true, nil return true, nil
} }
@@ -1387,6 +1416,13 @@ func (s *Scanner) processFileWithErrorHandling(
return false, nil return false, nil
} }
// countFailedFile counts a file that phase 2 could not store as failed
// and takes its size back out of BytesScanned, where phase 1 put it.
func countFailedFile(fileToProcess *FileToProcess, result *ScanResult) {
result.FilesFailed++
result.BytesScanned -= fileToProcess.FileInfo.Size()
}
// printProcessingProgress prints a periodic progress line during the process phase, // printProcessingProgress prints a periodic progress line during the process phase,
// showing files processed, bytes transferred, throughput, and ETA // showing files processed, bytes transferred, throughput, and ETA
func (s *Scanner) printProcessingProgress( func (s *Scanner) printProcessingProgress(
@@ -1755,16 +1791,11 @@ func (e *packerError) Error() string { return e.err.Error() }
func (e *packerError) Unwrap() error { return e.err } func (e *packerError) Unwrap() error { return e.err }
// processFileStreaming processes a file by streaming chunks directly to the packer // processFileStreaming processes a regular file by streaming chunks directly
// to the packer
func (s *Scanner) processFileStreaming( func (s *Scanner) processFileStreaming(
ctx context.Context, fileToProcess *FileToProcess, result *ScanResult, ctx context.Context, fileToProcess *FileToProcess, result *ScanResult,
) error { ) error {
// Symlinks and directories have no data to chunk — just record them in the DB.
mode := os.FileMode(fileToProcess.File.Mode)
if mode&os.ModeSymlink != 0 || mode.IsDir() {
return s.recordNonRegularFile(ctx, fileToProcess)
}
file, err := s.fs.Open(fileToProcess.Path) file, err := s.fs.Open(fileToProcess.Path)
if err != nil { if err != nil {
return fmt.Errorf("opening file: %w", wrapPermissionError(fileToProcess.Path, err)) return fmt.Errorf("opening file: %w", wrapPermissionError(fileToProcess.Path, err))
+90 -11
View File
@@ -82,6 +82,32 @@ func (f *readFailFs) Open(name string) (afero.File, error) {
return file, nil return file, nil
} }
// cancelOnOpenFs cancels the run when the scanner opens the target file to
// back it up, as Ctrl-C partway through processing would, and records each
// file opened after that.
type cancelOnOpenFs struct {
afero.Fs
target string
cancel context.CancelFunc
cancelled bool
openedAfterCancel []string
}
//nolint:ireturn // afero.Fs.Open is defined to return the interface.
func (f *cancelOnOpenFs) Open(name string) (afero.File, error) {
if f.cancelled {
f.openedAfterCancel = append(f.openedAfterCancel, name)
}
if name == f.target {
f.cancel()
f.cancelled = true
}
return f.Fs.Open(name)
}
// linkRemovedAfterLstatFs is the real filesystem, except that the symlink at // linkRemovedAfterLstatFs is the real filesystem, except that the symlink at
// target is removed right after the walk lstats it, as happens when a link is // target is removed right after the walk lstats it, as happens when a link is
// deleted during a backup. The scanner's readlink of it then fails. // deleted during a backup. The scanner's readlink of it then fails.
@@ -128,15 +154,16 @@ func writeSkipErrorTestFile(t *testing.T, fs afero.Fs, path, content string) {
} }
} }
// runSkipErrorScan scans source on fs with the given skip-errors setting, // runSkipErrorScan scans source on fs under ctx with the given skip-errors
// printing user-facing messages to uiw (nil discards them), and returns the // setting, printing user-facing messages to uiw (nil discards them), and
// repositories (for inspection) and the scan error. // returns the repositories (for inspection) and the scan error.
func runSkipErrorScan( func runSkipErrorScan(
t *testing.T, fs afero.Fs, source string, skipErrors bool, uiw *ui.Writer, ctx context.Context, t *testing.T, fs afero.Fs, source string,
skipErrors bool, uiw *ui.Writer,
) (*database.Repositories, error) { ) (*database.Repositories, error) {
t.Helper() t.Helper()
db, err := database.NewTestDB() db, err := database.New(ctx, ":memory:")
if err != nil { if err != nil {
t.Fatalf("create test db: %v", err) t.Fatalf("create test db: %v", err)
} }
@@ -161,7 +188,6 @@ func runSkipErrorScan(
SkipErrors: skipErrors, SkipErrors: skipErrors,
}) })
ctx := context.Background()
snapshotID := "test-snapshot-skip-errors" snapshotID := "test-snapshot-skip-errors"
createTestSnapshotRecord(ctx, t, repos, snapshotID) createTestSnapshotRecord(ctx, t, repos, snapshotID)
@@ -185,7 +211,7 @@ func TestScannerPackingFailureAbortsUnderSkipErrors(t *testing.T) {
writeSkipErrorTestFile(t, fs, "/source/file1.txt", "first file content") writeSkipErrorTestFile(t, fs, "/source/file1.txt", "first file content")
writeSkipErrorTestFile(t, fs, "/source/file2.txt", "second file content") writeSkipErrorTestFile(t, fs, "/source/file2.txt", "second file content")
repos, err := runSkipErrorScan(t, fs, "/source", true, nil) repos, err := runSkipErrorScan(context.Background(), t, fs, "/source", true, nil)
if err == nil { if err == nil {
t.Fatal("expected scan to abort on the packer error, got nil") t.Fatal("expected scan to abort on the packer error, got nil")
} }
@@ -212,7 +238,7 @@ func TestScannerReadErrorAbortsWithoutSkipErrors(t *testing.T) {
fs := &readFailFs{Fs: afero.NewMemMapFs(), target: target} fs := &readFailFs{Fs: afero.NewMemMapFs(), target: target}
writeSkipErrorTestFile(t, fs, target, "content that cannot be read") writeSkipErrorTestFile(t, fs, target, "content that cannot be read")
_, err := runSkipErrorScan(t, fs, "/source", false, nil) _, err := runSkipErrorScan(context.Background(), t, fs, "/source", false, nil)
if err == nil { if err == nil {
t.Fatal("expected scan to fail on the read error, got nil") t.Fatal("expected scan to fail on the read error, got nil")
} }
@@ -228,7 +254,7 @@ func TestScannerReadErrorSkippedWithSkipErrors(t *testing.T) {
fs := &readFailFs{Fs: afero.NewMemMapFs(), target: target} fs := &readFailFs{Fs: afero.NewMemMapFs(), target: target}
writeSkipErrorTestFile(t, fs, target, "content that cannot be read") writeSkipErrorTestFile(t, fs, target, "content that cannot be read")
repos, err := runSkipErrorScan(t, fs, "/source", true, nil) repos, err := runSkipErrorScan(context.Background(), t, fs, "/source", true, nil)
if err != nil { if err != nil {
t.Fatalf("expected scan to complete with --skip-errors, got %v", err) t.Fatalf("expected scan to complete with --skip-errors, got %v", err)
} }
@@ -267,7 +293,7 @@ func TestScannerUnreadableSymlinkAbortsWithoutSkipErrors(t *testing.T) {
sourceDir, linkPath := writeSymlinkSource(t) sourceDir, linkPath := writeSymlinkSource(t)
fs := &linkRemovedAfterLstatFs{t: t, target: linkPath} fs := &linkRemovedAfterLstatFs{t: t, target: linkPath}
_, err := runSkipErrorScan(t, fs, sourceDir, false, nil) _, err := runSkipErrorScan(context.Background(), t, fs, sourceDir, false, nil)
if !errors.Is(err, os.ErrNotExist) { if !errors.Is(err, os.ErrNotExist) {
t.Fatalf("expected scan to fail on the removed symlink, got %v", err) t.Fatalf("expected scan to fail on the removed symlink, got %v", err)
} }
@@ -283,7 +309,7 @@ func TestScannerUnreadableSymlinkSkippedWithSkipErrors(t *testing.T) {
fs := &linkRemovedAfterLstatFs{t: t, target: linkPath} fs := &linkRemovedAfterLstatFs{t: t, target: linkPath}
uiw := ui.NewWithColor(io.Discard, false) uiw := ui.NewWithColor(io.Discard, false)
repos, err := runSkipErrorScan(t, fs, sourceDir, true, uiw) repos, err := runSkipErrorScan(context.Background(), t, fs, sourceDir, true, uiw)
if err != nil { if err != nil {
t.Fatalf("expected scan to complete with --skip-errors, got %v", err) t.Fatalf("expected scan to complete with --skip-errors, got %v", err)
} }
@@ -302,3 +328,56 @@ func TestScannerUnreadableSymlinkSkippedWithSkipErrors(t *testing.T) {
t.Fatalf("expected %s not to be recorded", linkPath) t.Fatalf("expected %s not to be recorded", linkPath)
} }
} }
// TestScannerCancelStopsSkipErrorsRun checks that a --skip-errors backup
// cancelled partway through processing returns the cancellation error,
// opens no further file, and reports no file as failed.
func TestScannerCancelStopsSkipErrorsRun(t *testing.T) {
t.Parallel()
tests := []struct {
name string
targetContent string
}{
// The cancellation lands while the target is being read.
{name: "while reading a file", targetContent: "first file content"},
// An empty target has no chunks, so the cancellation goes unnoticed
// until the run moves on to the next file.
{name: "between files", targetContent: ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
const target = "/source/a.txt"
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
fs := &cancelOnOpenFs{
Fs: afero.NewMemMapFs(), target: target, cancel: cancel,
}
writeSkipErrorTestFile(t, fs, target, tt.targetContent)
writeSkipErrorTestFile(t, fs, "/source/b.txt", "second file content")
writeSkipErrorTestFile(t, fs, "/source/c.txt", "third file content")
uiw := ui.NewWithColor(io.Discard, false)
_, err := runSkipErrorScan(ctx, t, fs, "/source", true, uiw)
if !errors.Is(err, context.Canceled) {
t.Fatalf("expected the cancellation error, got %v", err)
}
if len(fs.openedAfterCancel) != 0 {
t.Fatalf("expected no file opened after the cancellation, got %v",
fs.openedAfterCancel)
}
if uiw.ErrorCount() != 0 {
t.Fatalf("expected no file reported as failed, got %d error lines",
uiw.ErrorCount())
}
})
}
}
+1 -1
View File
@@ -892,7 +892,7 @@ func (sm *SnapshotManager) getFileSize(path string) int64 {
// BackupStats contains statistics from a backup operation // BackupStats contains statistics from a backup operation
type BackupStats struct { type BackupStats struct {
FilesScanned int FilesScanned int
TotalSize int64 // Total size of all files examined TotalSize int64 // Total size of the files in the snapshot
ChunksCreated int ChunksCreated int
BlobsCreated int BlobsCreated int
BytesUploaded int64 BytesUploaded int64
@@ -171,6 +171,11 @@ func (f *Storer) ListStream(
return f.inner.ListStream(ctx, prefix) return f.inner.ListStream(ctx, prefix)
} }
// DeletePartialUploads delegates unchanged.
func (f *Storer) DeletePartialUploads(ctx context.Context, prefix string) error {
return f.inner.DeletePartialUploads(ctx, prefix)
}
// Info delegates unchanged. // Info delegates unchanged.
func (f *Storer) Info() storage.Info { func (f *Storer) Info() storage.Info {
return f.inner.Info() return f.inner.Info()
+36
View File
@@ -242,6 +242,42 @@ func (f *FileStorer) ListStream(ctx context.Context, prefix string) <-chan Objec
return ch return ch
} }
// DeletePartialUploads removes every file under prefix whose name ends in
// tempSuffix. A missing prefix has none to remove.
func (f *FileStorer) DeletePartialUploads(ctx context.Context, prefix string) error {
basePath := f.fullPath(prefix)
exists, err := afero.Exists(f.fs, basePath)
if err != nil {
return fmt.Errorf("checking path: %w", err)
}
if !exists {
return nil
}
err = afero.Walk(f.fs, basePath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if ctx.Err() != nil {
return ctx.Err()
}
if info.IsDir() || !strings.HasSuffix(info.Name(), tempSuffix) {
return nil
}
return f.fs.Remove(path)
})
if err != nil {
return fmt.Errorf("walking directory: %w", err)
}
return nil
}
// Info returns human-readable storage location information. // Info returns human-readable storage location information.
func (f *FileStorer) Info() Info { func (f *FileStorer) Info() Info {
return Info{ return Info{
+42
View File
@@ -120,3 +120,45 @@ func TestFileStorer_ListSkipsPartialFiles(t *testing.T) {
t.Fatalf("ListStream should return only the real key, got %v", streamed) t.Fatalf("ListStream should return only the real key, got %v", streamed)
} }
} }
// TestFileStorer_DeletePartialUploads checks that a leftover temp file is
// removed and the object at the real key is kept.
func TestFileStorer_DeletePartialUploads(t *testing.T) {
t.Parallel()
base := t.TempDir()
f, err := storage.NewFileStorer(base)
if err != nil {
t.Fatalf("NewFileStorer: %v", err)
}
ctx := context.Background()
err = f.Put(ctx, testBlobKey, strings.NewReader("blob-bytes"))
if err != nil {
t.Fatalf("Put: %v", err)
}
leftover := filepath.Join(base, testBlobKey+"-123456.partial")
err = os.WriteFile(leftover, []byte("half"), 0o600)
if err != nil {
t.Fatalf("writing leftover temp file: %v", err)
}
err = f.DeletePartialUploads(ctx, "blobs/")
if err != nil {
t.Fatalf("DeletePartialUploads: %v", err)
}
_, err = os.Stat(leftover)
if !os.IsNotExist(err) {
t.Errorf("leftover temp file was not removed: %v", err)
}
_, err = f.Stat(ctx, testBlobKey)
if err != nil {
t.Errorf("Stat of the real key: %v", err)
}
}
+25
View File
@@ -213,6 +213,31 @@ func (r *RcloneStorer) ListStream(
return ch return ch
} }
// DeletePartialUploads removes every object under prefix whose name ends
// in tempSuffix.
func (r *RcloneStorer) DeletePartialUploads(ctx context.Context, prefix string) error {
var partial []fs.Object
err := operations.ListFn(ctx, r.fsys, func(obj fs.Object) {
key := obj.Remote()
if strings.HasPrefix(key, prefix) && strings.HasSuffix(key, tempSuffix) {
partial = append(partial, obj)
}
})
if err != nil {
return fmt.Errorf("listing objects: %w", err)
}
for _, obj := range partial {
err = obj.Remove(ctx)
if err != nil {
return fmt.Errorf("removing object: %w", err)
}
}
return nil
}
// Info returns human-readable storage location information. // Info returns human-readable storage location information.
func (r *RcloneStorer) Info() Info { func (r *RcloneStorer) Info() Info {
location := r.remote location := r.remote
+41
View File
@@ -198,6 +198,47 @@ func TestRcloneStorerListSkipsPartialFiles(t *testing.T) {
} }
} }
// TestRcloneStorerDeletePartialUploads checks that a temporary file left
// by a killed upload is removed and the object at the real key is kept.
//
//nolint:paralleltest // NewRcloneStorer installs the process-global rclone config
func TestRcloneStorerDeletePartialUploads(t *testing.T) {
dir := t.TempDir()
ctx := context.Background()
s, err := storage.NewRcloneStorer(ctx, ":local", dir)
if err != nil {
t.Fatalf("NewRcloneStorer: %v", err)
}
err = s.Put(ctx, testBlobKey, strings.NewReader("blob-bytes"))
if err != nil {
t.Fatalf("Put: %v", err)
}
leftover := filepath.Join(dir, testBlobKey+"-123456.partial")
err = os.WriteFile(leftover, []byte("half"), 0o600)
if err != nil {
t.Fatalf("writing leftover temp file: %v", err)
}
err = s.DeletePartialUploads(ctx, "blobs/")
if err != nil {
t.Fatalf("DeletePartialUploads: %v", err)
}
_, err = os.Stat(leftover)
if !os.IsNotExist(err) {
t.Errorf("leftover temp file was not removed: %v", err)
}
_, err = s.Stat(ctx, testBlobKey)
if err != nil {
t.Errorf("Stat of the real key: %v", err)
}
}
// newRcloneStorerOnWrappedLocal registers name as rclone's local backend // newRcloneStorerOnWrappedLocal registers name as rclone's local backend
// wrapped by wrap, and builds an rclone backend on it rooted at a fresh // wrapped by wrap, and builds an rclone backend on it rooted at a fresh
// temp directory. wrap changes the features the local backend reports, so // temp directory. wrap changes the features the local backend reports, so
+6
View File
@@ -99,6 +99,12 @@ func (s *S3Storer) ListStream(ctx context.Context, prefix string) <-chan ObjectI
return ch return ch
} }
// DeletePartialUploads has nothing to remove: S3 shows an object only once
// its upload has completed, so an upload cut off part-way leaves none.
func (s *S3Storer) DeletePartialUploads(_ context.Context, _ string) error {
return nil
}
// Info returns human-readable storage location information. // Info returns human-readable storage location information.
func (s *S3Storer) Info() Info { func (s *S3Storer) Info() Info {
return Info{ return Info{
+6
View File
@@ -71,6 +71,12 @@ type Storer interface {
// If an error occurs during listing, the final item will have Err set. // If an error occurs during listing, the final item will have Err set.
ListStream(ctx context.Context, prefix string) <-chan ObjectInfo ListStream(ctx context.Context, prefix string) <-chan ObjectInfo
// DeletePartialUploads removes every object under prefix that an
// upload cut off part-way left under a temporary name ending in
// `.partial`. The file and rclone backends' List and ListStream skip
// such an object.
DeletePartialUploads(ctx context.Context, prefix string) error
// Info returns human-readable storage location information. // Info returns human-readable storage location information.
Info() Info Info() Info
} }
@@ -257,6 +257,10 @@ func (s *stubLister) List(_ context.Context, _ string) ([]string, error) {
return nil, errStubUnused return nil, errStubUnused
} }
func (s *stubLister) DeletePartialUploads(_ context.Context, _ string) error {
return errStubUnused
}
func (s *stubLister) Info() storage.Info { func (s *stubLister) Info() storage.Info {
return storage.Info{} return storage.Info{}
} }
+24 -8
View File
@@ -3,6 +3,7 @@ package vaultik
import ( import (
"errors" "errors"
"fmt" "fmt"
"math"
"regexp" "regexp"
"strconv" "strconv"
"strings" "strings"
@@ -47,6 +48,10 @@ const (
year = 365 * day year = 365 * day
) )
// maxDuration is the largest value a time.Duration can hold, about 292
// years.
const maxDuration = time.Duration(math.MaxInt64)
// A snapshot ID split on "_" has at least a hostname and a trailing // A snapshot ID split on "_" has at least a hostname and a trailing
// timestamp. // timestamp.
const minSnapshotIDParts = 2 const minSnapshotIDParts = 2
@@ -145,10 +150,10 @@ func parseSnapshotName(snapshotID, hostname string) string {
// parseDuration parses a duration string with support for human-friendly units: // parseDuration parses a duration string with support for human-friendly units:
// d/day/days, w/week/weeks, mo/month/months, y/year/years, plus standard Go // d/day/days, w/week/weeks, mo/month/months, y/year/years, plus standard Go
// duration units. Following Go, m is minutes and mo is months. A bare number, // duration units. Following Go, m is minutes and mo is months. A bare number,
// an unknown unit, and a negative value are all rejected. Outside the Go // an unknown unit, a negative value, and a total longer than maxDuration are
// units the input must be whole numbers each followed directly by a unit // all rejected. Outside the Go units the input must be whole numbers each
// (2w3d), so 1.0y and 30 days are errors; decimals work only in Go units // followed directly by a unit (2w3d), so 1.0y and 30 days are errors;
// (1.5h). // decimals work only in Go units (1.5h).
func parseDuration(s string) (time.Duration, error) { func parseDuration(s string) (time.Duration, error) {
if strings.HasPrefix(strings.TrimSpace(s), "-") { if strings.HasPrefix(strings.TrimSpace(s), "-") {
return 0, errNegativeDuration return 0, errNegativeDuration
@@ -181,19 +186,30 @@ func parseDuration(s string) (time.Duration, error) {
return 0, fmt.Errorf("invalid number %q: %w", match[1], err) return 0, fmt.Errorf("invalid number %q: %w", match[1], err)
} }
var unitLength time.Duration
unit := strings.ToLower(match[2]) unit := strings.ToLower(match[2])
switch unit { switch unit {
case "d", "day", "days": case "d", "day", "days":
total += time.Duration(n) * day unitLength = day
case "w", "week", "weeks": case "w", "week", "weeks":
total += time.Duration(n) * week unitLength = week
case "mo", "month", "months": case "mo", "month", "months":
total += time.Duration(n) * month unitLength = month
case "y", "year", "years": case "y", "year", "years":
total += time.Duration(n) * year unitLength = year
default: default:
return 0, fmt.Errorf("%w %q", errUnknownTimeUnit, unit) return 0, fmt.Errorf("%w %q", errUnknownTimeUnit, unit)
} }
// Past maxDuration the sum wraps to a negative value, which would
// put the purge cutoff in the future.
if time.Duration(n) > (maxDuration-total)/unitLength {
return 0, fmt.Errorf("%w: %q is longer than about 292 years",
errInvalidDuration, s)
}
total += time.Duration(n) * unitLength
} }
return total, nil return total, nil
+11
View File
@@ -111,6 +111,17 @@ func TestParseDuration(t *testing.T) {
{"30 days", 0, true}, {"30 days", 0, true},
{"30 days ago", 0, true}, {"30 days ago", 0, true},
{"x7d", 0, true}, {"x7d", 0, true},
// The largest time.Duration is about 292 years. A larger value
// must be rejected, not wrapped to a negative one, which would
// put the cutoff in the future and select every snapshot.
{"292y", 292 * 365 * 24 * time.Hour, false},
{"293y", 0, true},
{"3559mo", 0, true},
{"15251w", 0, true},
{"106752d", 0, true},
{"2562048h", 0, true},
{"200y200y", 0, true},
{"292y6mo", 0, true},
} }
for _, tt := range tests { for _, tt := range tests {
+32 -25
View File
@@ -167,10 +167,6 @@ const (
// metadataKeyParts is the minimum "/"-separated segment count of a // metadataKeyParts is the minimum "/"-separated segment count of a
// metadata object key (metadata/<snapshot-id>/<filename>). // metadata object key (metadata/<snapshot-id>/<filename>).
metadataKeyParts = 3 metadataKeyParts = 3
// blobKeyParts is the minimum "/"-separated segment count of a blob
// object key (blobs/<aa>/<bb>/<hash>).
blobKeyParts = 4
) )
// SnapshotMetadataInfo contains information about a single snapshot's metadata // SnapshotMetadataInfo contains information about a single snapshot's metadata
@@ -181,8 +177,11 @@ type SnapshotMetadataInfo struct {
ManifestSize int64 `json:"manifest_size"` ManifestSize int64 `json:"manifest_size"`
DatabaseSize int64 `json:"database_size"` DatabaseSize int64 `json:"database_size"`
TotalSize int64 `json:"total_size"` TotalSize int64 `json:"total_size"`
BlobCount int `json:"blob_count"`
BlobsSize int64 `json:"blobs_size"` // Both stay nil (null in the JSON) when the snapshot's manifest was
// listed but could not be read.
BlobCount *int `json:"blob_count"`
BlobsSize *int64 `json:"blobs_size"`
// Set when the listing holds this snapshot's manifest.json.zst. A // Set when the listing holds this snapshot's manifest.json.zst. A
// backup interrupted before its manifest upload leaves a directory // backup interrupted before its manifest upload leaves a directory
@@ -380,6 +379,10 @@ func (v *Vaultik) collectReferencedBlobsFromManifests(
for _, snapshotID := range snapshotIDs { for _, snapshotID := range snapshotIDs {
info := snapshotMetadata[snapshotID] info := snapshotMetadata[snapshotID]
if !info.hasManifest { if !info.hasManifest {
// The orphan figures count this directory's blobs as
// orphaned, so it references none.
info.BlobCount, info.BlobsSize = new(int), new(int64)
continue continue
} }
@@ -395,7 +398,7 @@ func (v *Vaultik) collectReferencedBlobsFromManifests(
continue continue
} }
info.BlobCount = manifest.BlobCount blobCount := manifest.BlobCount
var blobsSize int64 var blobsSize int64
@@ -404,7 +407,8 @@ func (v *Vaultik) collectReferencedBlobsFromManifests(
blobsSize += blob.CompressedSize blobsSize += blob.CompressedSize
} }
info.BlobsSize = blobsSize info.BlobCount = &blobCount
info.BlobsSize = &blobsSize
} }
return referencedBlobs, unreadable return referencedBlobs, unreadable
@@ -446,23 +450,16 @@ func (v *Vaultik) scanRemoteBlobStorage(
v.stdoutf("Scanning blobs...\n") v.stdoutf("Scanning blobs...\n")
} }
blobCh := v.Storage.ListStream(v.ctx, "blobs/") // Use prune's listing, so a file under blobs/ that prune would never
allBlobs := make(map[string]int64) // delete is not counted as a blob, and so not as an orphaned one.
allBlobs, err := v.listAllRemoteBlobs()
if err != nil {
return err
}
for obj := range blobCh { for _, size := range allBlobs {
if obj.Err != nil {
return fmt.Errorf("listing blobs: %w", obj.Err)
}
parts := strings.Split(obj.Key, "/")
if len(parts) < blobKeyParts {
continue
}
hash := parts[3]
allBlobs[hash] = obj.Size
result.TotalBlobCount++ result.TotalBlobCount++
result.TotalBlobSize += obj.Size result.TotalBlobSize += size
} }
// A blob named only by a manifest that could not be read, or by one // A blob named only by a manifest that could not be read, or by one
@@ -516,13 +513,23 @@ func (v *Vaultik) printRemoteInfoTable(result *RemoteInfoResult) {
v.stdoutf("%s", separator) v.stdoutf("%s", separator)
for _, info := range result.Snapshots { for _, info := range result.Snapshots {
blobCount := unknownText
if info.BlobCount != nil {
blobCount = humanize.Comma(int64(*info.BlobCount))
}
blobsSize := unknownText
if info.BlobsSize != nil {
blobsSize = ubytes(*info.BlobsSize)
}
v.stdoutf(rowFormat, v.stdoutf(rowFormat,
truncateString(info.SnapshotID, snapshotIDColWidth), truncateString(info.SnapshotID, snapshotIDColWidth),
ubytes(info.ManifestSize), ubytes(info.ManifestSize),
ubytes(info.DatabaseSize), ubytes(info.DatabaseSize),
ubytes(info.TotalSize), ubytes(info.TotalSize),
humanize.Comma(int64(info.BlobCount)), blobCount,
ubytes(info.BlobsSize), blobsSize,
) )
} }
+6
View File
@@ -155,6 +155,12 @@ func (m *MockStorer) ListStream(
return ch return ch
} }
// DeletePartialUploads has nothing to remove: Put stores each object
// under its key at once.
func (m *MockStorer) DeletePartialUploads(_ context.Context, _ string) error {
return nil
}
func (m *MockStorer) Info() storage.Info { func (m *MockStorer) Info() storage.Info {
return storage.Info{ return storage.Info{
Type: "mock", Type: "mock",
+64
View File
@@ -0,0 +1,64 @@
package vaultik_test
import (
"context"
"io/fs"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/snapshot"
)
// TestNukeRemoteLeavesNoFiles checks that remote nuke leaves no file
// under a file:// destination, including the `.partial` files uploads
// killed part-way leave next to a snapshot's metadata and next to where a
// blob would have been.
func TestNukeRemoteLeavesNoFiles(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
storeDir := filepath.Join(t.TempDir(), "store")
v, repos, _ := backUpToFileDestination(ctx, t, storeDir)
snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
require.NoError(t, err)
require.Len(t, snapshots, 1)
snapshotKey := snapshot.RemoteSnapshotKey(snapshots[0].ID.String())
hash := testBlobHashA
leftovers := []string{
filepath.Join(storeDir, "metadata", snapshotKey,
"db.zst.age-123456.partial"),
filepath.Join(storeDir, "blobs", hash[:2], hash[2:4],
hash+"-123456.partial"),
}
for _, leftover := range leftovers {
require.NoError(t, os.MkdirAll(filepath.Dir(leftover), 0o750))
require.NoError(t, os.WriteFile(leftover, []byte("half an upload"), 0o600))
}
require.NoError(t, v.NukeRemote(true))
var files []string
err = filepath.WalkDir(storeDir,
func(path string, entry fs.DirEntry, err error) error {
if err != nil {
return err
}
if !entry.IsDir() {
files = append(files, path)
}
return nil
})
require.NoError(t, err)
assert.Empty(t, files)
}
+16 -2
View File
@@ -23,9 +23,14 @@ var errNukeRequiresForce = errors.New(
// metadata. // metadata.
const metadataDirName = "metadata" const metadataDirName = "metadata"
// blobKeyParts is the "/"-separated segment count of a blob object key
// (blobs/<aa>/<bb>/<hash>).
const blobKeyParts = 4
// NukeRemote deletes every snapshot's metadata and every blob from remote // NukeRemote deletes every snapshot's metadata and every blob from remote
// storage. After this returns successfully the bucket prefix is empty and // storage, along with any object an upload cut off part-way left under a
// the next backup starts from scratch. // temporary `.partial` name. After this returns successfully the bucket
// prefix is empty and the next backup starts from scratch.
// //
// Refuses to run unless force is true. The caller is responsible for // Refuses to run unless force is true. The caller is responsible for
// confirming with the user. // confirming with the user.
@@ -48,6 +53,15 @@ func (v *Vaultik) NukeRemote(force bool) error {
return fmt.Errorf("pruning blobs: %w", err) return fmt.Errorf("pruning blobs: %w", err)
} }
// The file and rclone listings skip `.partial` objects, so the two
// steps above never delete them.
for _, prefix := range []string{"metadata/", "blobs/"} {
err = v.Storage.DeletePartialUploads(v.ctx, prefix)
if err != nil {
return fmt.Errorf("deleting partial uploads: %w", err)
}
}
v.UI.Completef("Backup destination store is now empty.") v.UI.Completef("Backup destination store is now empty.")
return nil return nil
+2 -2
View File
@@ -42,7 +42,7 @@ func TestTableCountForReportSurfacesReadFailure(t *testing.T) {
assert.Nil(t, missing, "a failed read is unknown, not a count") assert.Nil(t, missing, "a failed read is unknown, not a count")
// The rendered count for a failed read must say unknown, never 0. // The rendered count for a failed read must say unknown, never 0.
assert.Equal(t, countUnknown, countText(missing)) assert.Equal(t, unknownText, countText(missing))
assert.NotEqual(t, "0", countText(missing)) assert.NotEqual(t, "0", countText(missing))
} }
@@ -57,7 +57,7 @@ func TestCountTextDistinguishesEmptyFromUnknown(t *testing.T) {
assert.Equal(t, "0", countText(&zero)) assert.Equal(t, "0", countText(&zero))
assert.Equal(t, "7", countText(&seven)) assert.Equal(t, "7", countText(&seven))
assert.Equal(t, countUnknown, countText(nil)) assert.Equal(t, unknownText, countText(nil))
} }
// TestCountDiffUnknownWhenEitherSideUnknown checks that a delta computed // TestCountDiffUnknownWhenEitherSideUnknown checks that a delta computed
+40
View File
@@ -0,0 +1,40 @@
package vaultik_test
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/vaultik"
)
// TestPruneBlobs_JSONDeletesWithoutAsking checks that prune with --json
// and without --force deletes an unreferenced blob without the
// confirmation prompt. Stdin is empty, so a prompt would read no answer
// and cancel, and its text would come before the JSON document.
func TestPruneBlobs_JSONDeletesWithoutAsking(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newListEnv(t)
// The store holds no manifest, so nothing references this blob.
addBlob(t, env.store.testStorer, testBlobHashA)
err := env.v.PruneBlobs(&vaultik.PruneOptions{JSON: true})
require.NoError(t, err)
blobKey := "blobs/" + testBlobHashA[:2] + "/" + testBlobHashA[2:4] +
"/" + testBlobHashA
assert.False(t, env.store.hasKey(blobKey),
"the unreferenced blob must be deleted")
var result vaultik.PruneBlobsResult
require.NoError(t, json.Unmarshal(env.stdout.Bytes(), &result),
"stdout must hold only the JSON document, got:\n%s",
env.stdout.String())
assert.Equal(t, 1, result.BlobsDeleted)
}
+23
View File
@@ -267,6 +267,29 @@ func TestPurgeOlderThan_WithNameFilter(t *testing.T) {
assert.Contains(t, remaining, snapHomeT0) assert.Contains(t, remaining, snapHomeT0)
} }
func TestPurgeOlderThan_TooLargeDeletesNothing(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
snapshotIDs := []string{snapHomeT0, snapHomeT1}
v := setupPurgeTest(t, snapHostname, snapshotIDs)
err := v.PurgeSnapshotsWithOptions(&vaultik.SnapshotPurgeOptions{
OlderThan: "1000y",
Force: true,
})
require.Error(t, err)
err = v.PurgeSnapshotsWithOptions(&vaultik.SnapshotPurgeOptions{
OlderThan: "292y",
Force: true,
})
require.NoError(t, err)
assert.ElementsMatch(t, snapshotIDs, listRemainingSnapshots(t, v))
}
func TestPurgeKeepLatest_ThreeNames(t *testing.T) { func TestPurgeKeepLatest_ThreeNames(t *testing.T) {
log.Initialize(log.Config{}) log.Initialize(log.Config{})
t.Parallel() t.Parallel()
+102
View File
@@ -4,6 +4,7 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/json" "encoding/json"
"strings"
"testing" "testing"
"time" "time"
@@ -69,6 +70,74 @@ func TestRemoteInfo_UnreadableManifestLeavesOrphansUnknown(t *testing.T) {
assert.Equal(t, []any{unreadableKey}, doc["unreadable_manifests"]) assert.Equal(t, []any{unreadableKey}, doc["unreadable_manifests"])
} }
// TestRemoteInfo_UnreadableManifestLeavesSnapshotBlobsUnknown checks
// that the row of a snapshot whose manifest cannot be read gives its
// blob count and blob size as unknown in the table and as null in
// --json, not as 0. A directory without a manifest still shows 0: the
// orphan figures count its blobs as orphaned, so it references none.
func TestRemoteInfo_UnreadableManifestLeavesSnapshotBlobsUnknown(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newListEnv(t)
readableKey := env.addRemote(t, listRemoteID,
time.Date(2026, 3, 2, 0, 0, 0, 0, time.UTC))
unreadableKey := snapshot.RemoteSnapshotKey(listLocalID)
require.NoError(t, env.store.Put(context.Background(),
"metadata/"+unreadableKey+"/manifest.json.zst",
bytes.NewReader([]byte("not a valid manifest"))))
noManifestKey := snapshot.RemoteSnapshotKey("testhost_home_2026-03-03T10:00:00Z")
require.NoError(t, env.store.Put(context.Background(),
"metadata/"+noManifestKey+"/db.zst.age",
bytes.NewReader([]byte("not a valid database"))))
require.NoError(t, env.v.RemoteInfo(false))
// The table truncates the remote key, so a row is found by a prefix.
wantUnknown := map[string]int{readableKey: 0, unreadableKey: 2, noManifestKey: 0}
for key, want := range wantUnknown {
var row string
for line := range strings.SplitSeq(env.stdout.String(), "\n") {
if strings.HasPrefix(line, key[:16]) {
row = line
}
}
require.NotEmpty(t, row, "no table row for %s", key)
assert.Equal(t, want, strings.Count(row, "unknown"), "row: %q", row)
}
env.stdout.Reset()
require.NoError(t, env.v.RemoteInfo(true))
var doc struct {
Snapshots []map[string]any `json:"snapshots"`
}
require.NoError(t, json.Unmarshal(env.stdout.Bytes(), &doc))
require.Len(t, doc.Snapshots, len(wantUnknown))
for _, entry := range doc.Snapshots {
switch entry["snapshot_id"] {
case readableKey:
assert.InDelta(t, 1, entry["blob_count"], 0)
assert.InDelta(t, fiveMegabytes, entry["blobs_size"], 0)
case noManifestKey:
assert.InDelta(t, 0, entry["blob_count"], 0)
assert.InDelta(t, 0, entry["blobs_size"], 0)
default:
assert.Equal(t, unreadableKey, entry["snapshot_id"])
assert.Contains(t, entry, "blob_count")
assert.Nil(t, entry["blob_count"])
assert.Contains(t, entry, "blobs_size")
assert.Nil(t, entry["blobs_size"])
}
}
}
// TestRemoteInfo_SkipsNonConformingMetadataName checks that a directory // TestRemoteInfo_SkipsNonConformingMetadataName checks that a directory
// under metadata/ whose name is not a remote key is left out of the // under metadata/ whose name is not a remote key is left out of the
// report, and that the orphan figures are unknown when it holds a // report, and that the orphan figures are unknown when it holds a
@@ -116,6 +185,39 @@ func TestRemoteInfo_SkipsNonConformingMetadataName(t *testing.T) {
assert.NotContains(t, doc, "unreadable_manifests") assert.NotContains(t, doc, "unreadable_manifests")
} }
// TestRemoteInfo_DoesNotCountNonBlobFileUnderBlobs checks that a file
// under blobs/ whose name is not a blob hash is not counted as a blob.
// prune never deletes such a file, so counting it as an orphaned blob
// would advise a prune that removes nothing.
func TestRemoteInfo_DoesNotCountNonBlobFileUnderBlobs(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newListEnv(t)
env.addRemote(t, listRemoteID, time.Date(2026, 3, 2, 0, 0, 0, 0, time.UTC))
addBlob(t, env.store.testStorer, testBlobHashA)
require.NoError(t, env.store.Put(context.Background(),
"blobs/"+testBlobHashA[:2]+"/"+testBlobHashA[2:4]+"/.DS_Store",
bytes.NewReader([]byte("junk"))))
require.NoError(t, env.v.RemoteInfo(false))
text := env.stdout.String()
assert.Contains(t, text, "Total blobs on remote: 1 (")
assert.Contains(t, text, "Orphaned (unreferenced): 0 (")
assert.NotContains(t, text, "vaultik prune")
env.stdout.Reset()
require.NoError(t, env.v.RemoteInfo(true))
var doc map[string]any
require.NoError(t, json.Unmarshal(env.stdout.Bytes(), &doc))
assert.InDelta(t, 1, doc["total_blob_count"], 0)
assert.InDelta(t, 0, doc["orphaned_blob_count"], 0)
assert.InDelta(t, 0, doc["orphaned_blob_size"], 0)
}
// TestRemoteInfo_DirectoryWithoutManifestLeavesOrphansKnown checks that // TestRemoteInfo_DirectoryWithoutManifestLeavesOrphansKnown checks that
// a directory under metadata/ holding no manifest.json.zst, such as one // a directory under metadata/ holding no manifest.json.zst, such as one
// left by a backup interrupted before its manifest upload, leaves the // left by a backup interrupted before its manifest upload, leaves the
+121
View File
@@ -3,17 +3,21 @@ package vaultik_test
import ( import (
"bytes" "bytes"
"context" "context"
"database/sql"
"io" "io"
"strings" "strings"
"sync" "sync"
"testing" "testing"
"time"
"github.com/klauspost/compress/zstd" "github.com/klauspost/compress/zstd"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log" "sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/snapshot" "sneak.berlin/go/vaultik/internal/snapshot"
"sneak.berlin/go/vaultik/internal/storage" "sneak.berlin/go/vaultik/internal/storage"
"sneak.berlin/go/vaultik/internal/types"
"sneak.berlin/go/vaultik/internal/vaultik" "sneak.berlin/go/vaultik/internal/vaultik"
) )
@@ -125,6 +129,12 @@ func (s *testStorer) ListStream(
return ch return ch
} }
// DeletePartialUploads has nothing to remove: Put stores each object
// under its key at once.
func (s *testStorer) DeletePartialUploads(_ context.Context, _ string) error {
return nil
}
func (s *testStorer) Info() storage.Info { func (s *testStorer) Info() storage.Info {
return storage.Info{ return storage.Info{
Type: testLabel, Type: testLabel,
@@ -208,6 +218,37 @@ func addBlob(t *testing.T, store *testStorer, hash string) {
require.NoError(t, err) require.NoError(t, err)
} }
// newLocalIndex returns an in-memory local index holding a completed
// snapshot row for each of ids.
func newLocalIndex(t *testing.T, ids ...string) *database.Repositories {
t.Helper()
ctx := context.Background()
db, err := database.New(ctx, ":memory:")
require.NoError(t, err)
t.Cleanup(func() { _ = db.Close() })
repos := database.NewRepositories(db)
for _, id := range ids {
completedAt := time.Now().UTC()
snap := &database.Snapshot{
ID: types.SnapshotID(id),
Hostname: snapHostname,
StartedAt: completedAt,
CompletedAt: &completedAt,
}
err = repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
return repos.Snapshots.Create(ctx, tx, snap)
})
require.NoError(t, err)
}
return repos
}
// ============================================================================ // ============================================================================
// Unit Tests for RemoveSnapshot // Unit Tests for RemoveSnapshot
// ============================================================================ // ============================================================================
@@ -227,6 +268,7 @@ func TestRemoveSnapshot_LocalOnly_PreservesRemote(t *testing.T) {
addBlob(t, store, blobA) addBlob(t, store, blobA)
tv := vaultik.NewForTesting(store) tv := vaultik.NewForTesting(store)
tv.Repositories = newLocalIndex(t, "snapshot-001")
opts := &vaultik.RemoveOptions{Force: true, LocalOnly: true} opts := &vaultik.RemoveOptions{Force: true, LocalOnly: true}
result, err := tv.RemoveSnapshot("snapshot-001", opts) result, err := tv.RemoveSnapshot("snapshot-001", opts)
@@ -262,6 +304,7 @@ func TestRemoveSnapshot_DefaultRemovesMetadataNotBlobs(t *testing.T) {
addBlob(t, store, blobShared) addBlob(t, store, blobShared)
tv := vaultik.NewForTesting(store) tv := vaultik.NewForTesting(store)
tv.Repositories = newLocalIndex(t, "snapshot-001", "snapshot-002")
opts := &vaultik.RemoveOptions{Force: true} opts := &vaultik.RemoveOptions{Force: true}
result, err := tv.RemoveSnapshot("snapshot-001", opts) result, err := tv.RemoveSnapshot("snapshot-001", opts)
@@ -310,6 +353,84 @@ func TestRemoveSnapshot_DryRun(t *testing.T) {
assert.Contains(t, tv.Stdout.String(), "[Dry run - no changes made]") assert.Contains(t, tv.Stdout.String(), "[Dry run - no changes made]")
} }
// TestRemoveSnapshot_UnknownIDIsError checks that an ID naming no
// snapshot in the local index or on the destination store is an error
// under every option, and that nothing is reported as removed.
func TestRemoveSnapshot_UnknownIDIsError(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
for name, opts := range map[string]vaultik.RemoveOptions{
"force": {Force: true},
"json": {JSON: true},
"dry-run": {Force: true, DryRun: true},
"local-only": {Force: true, LocalOnly: true},
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
store := newTestStorer()
addManifest(t, store, "snapshot-001", nil)
tv := vaultik.NewForTesting(store)
tv.Repositories = newLocalIndex(t, "snapshot-001")
result, err := tv.RemoveSnapshot("snapshot-999", &opts)
require.ErrorContains(t, err, "snapshot not found")
assert.False(t, result.RemoteRemoved)
assert.Empty(t, tv.Stdout.String())
assert.True(t, store.hasKey(remoteKeyPath("snapshot-001")))
})
}
}
// TestRemoveSnapshot_NoMetadataOnDestination checks that removing a
// snapshot whose metadata is not on the destination store does not
// report metadata removed from it.
func TestRemoveSnapshot_NoMetadataOnDestination(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
tv := vaultik.NewForTesting(newTestStorer())
tv.Repositories = newLocalIndex(t, "snapshot-001")
result, err := tv.RemoveSnapshot("snapshot-001",
&vaultik.RemoveOptions{Force: true})
require.NoError(t, err)
assert.False(t, result.RemoteRemoved)
out := tv.Stdout.String()
assert.Contains(t, out, "Removed snapshot 'snapshot-001' from local database")
assert.NotContains(t, out, "Removed snapshot metadata from remote storage")
}
// TestRemoveSnapshot_NotInLocalIndex checks that removing a snapshot
// only the destination store holds removes its metadata there without
// reporting a removal from the local database.
func TestRemoveSnapshot_NotInLocalIndex(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
addManifest(t, store, "snapshot-001", nil)
tv := vaultik.NewForTesting(store)
tv.Repositories = newLocalIndex(t)
result, err := tv.RemoveSnapshot("snapshot-001",
&vaultik.RemoveOptions{Force: true})
require.NoError(t, err)
assert.True(t, result.RemoteRemoved)
assert.False(t, store.hasKey(remoteKeyPath("snapshot-001")))
out := tv.Stdout.String()
assert.NotContains(t, out, "from local database")
assert.Contains(t, out, "Removed snapshot metadata from remote storage")
}
func TestRemoveAllSnapshots_RequiresForce(t *testing.T) { func TestRemoveAllSnapshots_RequiresForce(t *testing.T) {
log.Initialize(log.Config{}) log.Initialize(log.Config{})
t.Parallel() t.Parallel()
+43 -15
View File
@@ -27,6 +27,7 @@ import (
// Sentinel errors for restore failures. // Sentinel errors for restore failures.
var ( var (
errFilesFailedRestore = errors.New("file(s) failed to restore") errFilesFailedRestore = errors.New("file(s) failed to restore")
errPathsNotInSnapshot = errors.New("requested path(s) not in the snapshot")
errFilesFailedVerify = errors.New("files failed verification") errFilesFailedVerify = errors.New("files failed verification")
errDecryptionKeyRequired = errors.New( errDecryptionKeyRequired = errors.New(
"decryption key required for restore\n\n" + "decryption key required for restore\n\n" +
@@ -149,7 +150,7 @@ func (v *Vaultik) Restore(opts *RestoreOptions) error {
repos := database.NewRepositories(tempDB) repos := database.NewRepositories(tempDB)
// Step 2: Get list of files to restore // Step 2: Get list of files to restore
files, err := v.getFilesToRestore(v.ctx, repos, opts.Paths) files, unmatched, err := v.getFilesToRestore(v.ctx, repos, opts.Paths)
if err != nil { if err != nil {
return fmt.Errorf("getting files to restore: %w", err) return fmt.Errorf("getting files to restore: %w", err)
} }
@@ -158,7 +159,7 @@ func (v *Vaultik) Restore(opts *RestoreOptions) error {
log.Warn("No files found to restore") log.Warn("No files found to restore")
v.UI.Warningf("No files found to restore.") v.UI.Warningf("No files found to restore.")
return nil return v.reportUnmatchedPaths(unmatched)
} }
log.Info("Found files to restore", "count", len(files)) log.Info("Found files to restore", "count", len(files))
@@ -198,14 +199,16 @@ func (v *Vaultik) Restore(opts *RestoreOptions) error {
v.UI.Duration(result.Duration), v.UI.Duration(result.Duration),
) )
return v.finishRestore(repos, files, opts, result) return v.finishRestore(repos, files, unmatched, opts, result)
} }
// finishRestore emits the post-restore warnings, runs optional // finishRestore emits the post-restore warnings, runs optional
// verification, and converts any failed-file count into an error. // verification, and converts any failed-file count or requested path
// that matched nothing into an error.
func (v *Vaultik) finishRestore( func (v *Vaultik) finishRestore(
repos *database.Repositories, repos *database.Repositories,
files []*database.File, files []*database.File,
unmatched []string,
opts *RestoreOptions, opts *RestoreOptions,
result *RestoreResult, result *RestoreResult,
) error { ) error {
@@ -223,6 +226,8 @@ func (v *Vaultik) finishRestore(
} }
} }
unmatchedErr := v.reportUnmatchedPaths(unmatched)
// Run verification if requested // Run verification if requested
if opts.Verify { if opts.Verify {
err := v.handleRestoreVerification(repos, files, opts, result) err := v.handleRestoreVerification(repos, files, opts, result)
@@ -235,7 +240,23 @@ func (v *Vaultik) finishRestore(
return fmt.Errorf("%d %w", result.FilesFailed, errFilesFailedRestore) return fmt.Errorf("%d %w", result.FilesFailed, errFilesFailedRestore)
} }
return nil return unmatchedErr
}
// reportUnmatchedPaths warns, by name, about each requested path that
// matched nothing in the snapshot, and returns an error counting them,
// or nil when there are none.
func (v *Vaultik) reportUnmatchedPaths(unmatched []string) error {
if len(unmatched) == 0 {
return nil
}
for _, path := range unmatched {
log.Warn("Requested path matches nothing in the snapshot", "path", path)
v.UI.Warningf("%s matches nothing in the snapshot.", v.UI.Path(path))
}
return fmt.Errorf("%d %w", len(unmatched), errPathsNotInSnapshot)
} }
// restoreIdentities parses the configured age secret key once into every // restoreIdentities parses the configured age secret key once into every
@@ -812,27 +833,34 @@ func (v *Vaultik) materializeSnapshotDB(
return db, tempDir, nil return db, tempDir, nil
} }
// getFilesToRestore returns the list of files to restore based on path filters // getFilesToRestore returns the list of files to restore based on path
// filters, and the filters, as given, that matched no file.
func (v *Vaultik) getFilesToRestore( func (v *Vaultik) getFilesToRestore(
ctx context.Context, repos *database.Repositories, pathFilters []string, ctx context.Context, repos *database.Repositories, pathFilters []string,
) ([]*database.File, error) { ) ([]*database.File, []string, error) {
// If no filters, get all files // If no filters, get all files
if len(pathFilters) == 0 { if len(pathFilters) == 0 {
return repos.Files.ListAll(ctx) files, err := repos.Files.ListAll(ctx)
return files, nil, err
} }
// Get files matching the path filters // Get files matching the path filters
var result []*database.File var (
result []*database.File
unmatched []string
)
seen := make(map[string]bool) seen := make(map[string]bool)
for _, filter := range pathFilters { for _, filter := range pathFilters {
// Normalize the filter path files, err := repos.Files.ListUnderPath(ctx, filepath.Clean(filter))
filter = filepath.Clean(filter)
files, err := repos.Files.ListUnderPath(ctx, filter)
if err != nil { if err != nil {
return nil, fmt.Errorf("listing files under %s: %w", filter, err) return nil, nil, fmt.Errorf("listing files under %s: %w", filter, err)
}
if len(files) == 0 {
unmatched = append(unmatched, filter)
} }
for _, file := range files { for _, file := range files {
@@ -843,7 +871,7 @@ func (v *Vaultik) getFilesToRestore(
} }
} }
return result, nil return result, unmatched, nil
} }
// buildChunkToBlobMap creates a mapping from chunk hash to blob information // buildChunkToBlobMap creates a mapping from chunk hash to blob information
@@ -0,0 +1,142 @@
package vaultik_test
import (
"bytes"
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/config"
"sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/storage"
"sneak.berlin/go/vaultik/internal/ui"
"sneak.berlin/go/vaultik/internal/vaultik"
)
// These tests cover https://git.eeqj.de/sneak/vaultik/issues/309: a path
// given to restore that matches nothing in the snapshot is named in a
// warning and makes the restore fail, after the paths that do match are
// restored.
func TestRestoreNamesUnmatchedPathAndRestoresTheMatchingOne(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
dataDir := filepath.Join(tempDir, "src")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
ctx := context.Background()
cfg := faultTestConfig()
files := writeFaultSourceTree(t, fs, dataDir)
store, err := storage.NewFileStorer(storeDir)
require.NoError(t, err)
db, err := database.New(ctx, dbPath)
require.NoError(t, err)
id := fullFaultBackup(ctx, t, fs, store, cfg,
database.NewRepositories(db), dataDir, dbPath, "unmatched")
require.NoError(t, db.Close())
matched := filepath.Join(dataDir, "small.txt")
unmatched := filepath.Join(dataDir, "typo.txt")
var out bytes.Buffer
v := newReaderVaultik(ctx, cfg, store, nil, fs)
v.UI = ui.NewWithColor(&out, false)
err = v.Restore(&vaultik.RestoreOptions{
SnapshotID: id,
TargetDir: restoreDir,
Paths: []string{matched, unmatched},
})
require.Error(t, err, "a path that matches nothing must fail the restore")
assert.Contains(t, out.String(), unmatched,
"the path that matches nothing must be named")
assert.NotContains(t, out.String(), matched,
"the path that matches must not be reported")
got, err := afero.ReadFile(fs, filepath.Join(restoreDir, matched))
require.NoError(t, err, "the path that matches must still be restored")
assert.Equal(t, files[matched], got)
}
// When no requested path matches, nothing is restored and each path is
// named. One path is relative. The other goes through the symlink that
// is the snapshot's configured path, while the entries are stored under
// the directory the symlink points to.
func TestRestoreNamesEveryPathWhenNothingMatches(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
const snapshotName = "linked"
fs := afero.NewOsFs()
tempDir := t.TempDir()
dataDir := filepath.Join(tempDir, "src")
linkDir := filepath.Join(tempDir, "srclink")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
ctx := context.Background()
writeFaultSourceTree(t, fs, dataDir)
require.NoError(t, os.Symlink(dataDir, linkDir))
cfg := faultTestConfig()
cfg.IndexPath = dbPath
cfg.ChunkSize = config.Size(faultChunkSize)
cfg.Snapshots = map[string]config.SnapshotConfig{
snapshotName: {Paths: []string{linkDir}},
}
store, err := storage.NewFileStorer(storeDir)
require.NoError(t, err)
db, err := database.New(ctx, dbPath)
require.NoError(t, err)
repos := database.NewRepositories(db)
v := newBackupVaultik(ctx, cfg, store, repos, db, fs)
require.NoError(t, backUp(v, snapshotName))
id := localSnapshotID(ctx, t, repos, snapshotName)
require.NoError(t, db.Close())
throughLink := filepath.Join(linkDir, "sub")
relative := strings.TrimPrefix(filepath.Join(dataDir, "small.txt"), "/")
var out bytes.Buffer
reader := newReaderVaultik(ctx, cfg, store, nil, fs)
reader.UI = ui.NewWithColor(&out, false)
err = reader.Restore(&vaultik.RestoreOptions{
SnapshotID: id,
TargetDir: restoreDir,
Paths: []string{throughLink, relative},
})
require.Error(t, err, "a restore that matches nothing must fail")
assert.Contains(t, out.String(), throughLink,
"the path through the symlink must be named")
assert.Contains(t, out.String(), relative,
"the relative path must be named")
_, err = fs.Stat(restoreDir)
assert.True(t, os.IsNotExist(err), "nothing must be restored")
}
+134 -39
View File
@@ -23,12 +23,19 @@ var (
errNoSnapshotsInConfig = errors.New("no snapshots configured") errNoSnapshotsInConfig = errors.New("no snapshots configured")
errSnapshotVerifyFailed = errors.New("verification failed") errSnapshotVerifyFailed = errors.New("verification failed")
errRemoveAllNeedsForce = errors.New("--all requires --force") errRemoveAllNeedsForce = errors.New("--all requires --force")
errSnapshotNotFound = errors.New("snapshot not found")
errInvalidTableName = errors.New("invalid table name") errInvalidTableName = errors.New("invalid table name")
errNoAgeRecipients = errors.New( errNoAgeRecipients = errors.New(
"creating a snapshot needs at least one public key in " + "creating a snapshot needs at least one public key in " +
"age_recipients (generate a keypair with: age-keygen)") "age_recipients (generate a keypair with: age-keygen)")
errKeepNewerThanNeedsPrune = errors.New("--keep-newer-than requires --prune")
) )
// ErrPostBackupPruneFailed wraps a failure of the post-backup prune,
// which runs only after every snapshot was created and uploaded, so the
// caller can report it apart from a failed backup.
var ErrPostBackupPruneFailed = errors.New("post-backup prune failed")
// listRecentLimit caps how many snapshot rows are fetched from the // listRecentLimit caps how many snapshot rows are fetched from the
// local index for listing and purge operations. // local index for listing and purge operations.
const listRecentLimit = 10000 const listRecentLimit = 10000
@@ -44,6 +51,26 @@ type SnapshotCreateOptions struct {
Snapshots []string // Optional list of snapshot names to process (empty = all) Snapshots []string // Optional list of snapshot names to process (empty = all)
} }
// Validate fails when KeepNewerThan is set without Prune or is not a
// valid duration. Only the post-backup prune reads KeepNewerThan, after
// the backup, so the caller checks it before starting one.
func (opts *SnapshotCreateOptions) Validate() error {
if opts.KeepNewerThan == "" {
return nil
}
if !opts.Prune {
return errKeepNewerThanNeedsPrune
}
_, err := parseDuration(opts.KeepNewerThan)
if err != nil {
return fmt.Errorf("--keep-newer-than: %w", err)
}
return nil
}
// CreateSnapshot executes the snapshot creation operation // CreateSnapshot executes the snapshot creation operation
func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error { func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
// config.Load accepts an empty list, since listing, verifying and // config.Load accepts an empty list, since listing, verifying and
@@ -122,7 +149,7 @@ func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
if opts.Prune { if opts.Prune {
err = v.runPostBackupPrune(snapshotNames, opts.KeepNewerThan) err = v.runPostBackupPrune(snapshotNames, opts.KeepNewerThan)
if err != nil { if err != nil {
return fmt.Errorf("post-backup prune: %w", err) return fmt.Errorf("%w: %w", ErrPostBackupPruneFailed, err)
} }
} }
@@ -185,6 +212,7 @@ type snapshotStats struct {
totalBlobs int totalBlobs int
totalBytesSkipped int64 totalBytesSkipped int64
totalFilesSkipped int totalFilesSkipped int
totalFilesFailed int
totalFilesDeleted int totalFilesDeleted int
totalBytesDeleted int64 totalBytesDeleted int64
totalBytesUploaded int64 totalBytesUploaded int64
@@ -315,6 +343,7 @@ func (v *Vaultik) scanAllDirectories(
stats.totalChunks += result.ChunksCreated stats.totalChunks += result.ChunksCreated
stats.totalBlobs += result.BlobsCreated stats.totalBlobs += result.BlobsCreated
stats.totalFilesSkipped += result.FilesSkipped stats.totalFilesSkipped += result.FilesSkipped
stats.totalFilesFailed += result.FilesFailed
stats.totalBytesSkipped += result.BytesSkipped stats.totalBytesSkipped += result.BytesSkipped
stats.totalFilesDeleted += result.FilesDeleted stats.totalFilesDeleted += result.FilesDeleted
stats.totalBytesDeleted += result.BytesDeleted stats.totalBytesDeleted += result.BytesDeleted
@@ -326,6 +355,7 @@ func (v *Vaultik) scanAllDirectories(
"path", dir, "path", dir,
"files", result.FilesScanned, "files", result.FilesScanned,
"files_skipped", result.FilesSkipped, "files_skipped", result.FilesSkipped,
"files_failed", result.FilesFailed,
"bytes", result.BytesScanned, "bytes", result.BytesScanned,
"bytes_skipped", result.BytesSkipped, "bytes_skipped", result.BytesSkipped,
"chunks", result.ChunksCreated, "chunks", result.ChunksCreated,
@@ -359,9 +389,11 @@ func (v *Vaultik) finalizeSnapshotMetadata(
return fmt.Errorf("getting snapshot blob sizes: %w", err) return fmt.Errorf("getting snapshot blob sizes: %w", err)
} }
// file_count and total_size leave out the files that could not be
// stored; stats.totalBytes already does.
extStats := snapshot.ExtendedBackupStats{ extStats := snapshot.ExtendedBackupStats{
BackupStats: snapshot.BackupStats{ BackupStats: snapshot.BackupStats{
FilesScanned: stats.totalFiles, FilesScanned: stats.totalFiles - stats.totalFilesFailed,
TotalSize: stats.totalBytes + stats.totalBytesSkipped, TotalSize: stats.totalBytes + stats.totalBytesSkipped,
ChunksCreated: stats.totalChunks, ChunksCreated: stats.totalChunks,
BlobsCreated: stats.totalBlobs, BlobsCreated: stats.totalBlobs,
@@ -409,7 +441,8 @@ func (v *Vaultik) printSnapshotSummary(
snapshotID string, startTime time.Time, stats *snapshotStats, snapshotID string, startTime time.Time, stats *snapshotStats,
) { ) {
snapshotDuration := time.Since(startTime) snapshotDuration := time.Since(startTime)
totalFilesChanged := stats.totalFiles - stats.totalFilesSkipped totalFilesChanged := stats.totalFiles - stats.totalFilesSkipped -
stats.totalFilesFailed
totalBytesAll := stats.totalBytes + stats.totalBytesSkipped totalBytesAll := stats.totalBytes + stats.totalBytesSkipped
var compressionRatio float64 var compressionRatio float64
@@ -426,6 +459,10 @@ func (v *Vaultik) printSnapshotSummary(
v.UI.Count(stats.totalFiles), v.UI.Count(stats.totalFiles),
v.UI.Count(totalFilesChanged), v.UI.Count(totalFilesChanged),
v.UI.Count(stats.totalFilesSkipped)) v.UI.Count(stats.totalFilesSkipped))
if stats.totalFilesFailed > 0 {
filesMsg += fmt.Sprintf(", %s failed", v.UI.Count(stats.totalFilesFailed))
}
if stats.totalFilesDeleted > 0 { if stats.totalFilesDeleted > 0 {
filesMsg += fmt.Sprintf(", %s deleted", v.UI.Count(stats.totalFilesDeleted)) filesMsg += fmt.Sprintf(", %s deleted", v.UI.Count(stats.totalFilesDeleted))
} }
@@ -633,7 +670,7 @@ func (v *Vaultik) confirmAndExecutePurge(
"snapshot_id", snapshotID, "error", err) "snapshot_id", snapshotID, "error", err)
} }
err = v.deleteRemoteSnapshotByKey(snapshot.RemoteSnapshotKey(snapshotID)) _, err = v.deleteRemoteSnapshotByKey(snapshot.RemoteSnapshotKey(snapshotID))
if err != nil { if err != nil {
return fmt.Errorf("deleting snapshot %s from remote: %w", snapshotID, err) return fmt.Errorf("deleting snapshot %s from remote: %w", snapshotID, err)
} }
@@ -1143,6 +1180,11 @@ func (v *Vaultik) RemoveSnapshot(
return result, err return result, err
} }
inLocalIndex, err := v.findSnapshotToRemove(snapshotID, opts)
if err != nil {
return result, err
}
if opts.DryRun { if opts.DryRun {
return v.removeSnapshotDryRun(result, snapshotID, opts) return v.removeSnapshotDryRun(result, snapshotID, opts)
} }
@@ -1151,11 +1193,13 @@ func (v *Vaultik) RemoveSnapshot(
return result, nil return result, nil
} }
log.Info("Removing snapshot from local database", "snapshot_id", snapshotID) if inLocalIndex {
log.Info("Removing snapshot from local database", "snapshot_id", snapshotID)
err = v.deleteSnapshotFromLocalDB(snapshotID) err = v.deleteSnapshotFromLocalDB(snapshotID)
if err != nil { if err != nil {
return result, fmt.Errorf("removing from local database: %w", err) return result, fmt.Errorf("removing from local database: %w", err)
}
} }
if !opts.LocalOnly { if !opts.LocalOnly {
@@ -1174,9 +1218,11 @@ func (v *Vaultik) RemoveSnapshot(
return result, v.outputRemoveJSON(result) return result, v.outputRemoveJSON(result)
} }
v.stdoutf("Removed snapshot '%s' from local database\n", snapshotID) if inLocalIndex {
v.stdoutf("Removed snapshot '%s' from local database\n", snapshotID)
}
if !opts.LocalOnly && result.RemoteRemoved { if result.RemoteRemoved {
v.printlnStdout("Removed snapshot metadata from remote storage") v.printlnStdout("Removed snapshot metadata from remote storage")
v.stdoutf("\nNote: The removed snapshot's blobs remain on the remote. "+ v.stdoutf("\nNote: The removed snapshot's blobs remain on the remote. "+
"Run '%s' to delete any blobs no longer referenced by any "+ "Run '%s' to delete any blobs no longer referenced by any "+
@@ -1186,6 +1232,45 @@ func (v *Vaultik) RemoveSnapshot(
return result, nil return result, nil
} }
// findSnapshotToRemove reports whether the local index has a row for
// snapshotID, and returns errSnapshotNotFound when there is nothing to
// remove: no row and, unless LocalOnly is set, no metadata on the
// destination store either. The destination store is consulted only
// when the local index has no row, so a snapshot in the local index can
// still be removed from it while the destination store is unreachable.
func (v *Vaultik) findSnapshotToRemove(
snapshotID string, opts *RemoveOptions,
) (bool, error) {
if v.Repositories != nil {
snap, err := v.Repositories.Snapshots.GetByID(v.ctx, snapshotID)
if err != nil {
return false, fmt.Errorf("looking up snapshot in local index: %w", err)
}
if snap != nil {
return true, nil
}
}
if opts.LocalOnly {
return false, fmt.Errorf("%w in the local index: %s",
errSnapshotNotFound, snapshotID)
}
objects, err := v.remoteSnapshotObjects(snapshot.RemoteSnapshotKey(snapshotID))
if err != nil {
return false, fmt.Errorf("looking up snapshot on destination store: %w", err)
}
if len(objects) == 0 {
return false, fmt.Errorf(
"%w in the local index or on the destination store: %s",
errSnapshotNotFound, snapshotID)
}
return false, nil
}
// removeSnapshotDryRun reports what RemoveSnapshot would do without // removeSnapshotDryRun reports what RemoveSnapshot would do without
// making changes. // making changes.
func (v *Vaultik) removeSnapshotDryRun( func (v *Vaultik) removeSnapshotDryRun(
@@ -1245,14 +1330,14 @@ func (v *Vaultik) confirmRemoveSnapshot(snapshotID string, opts *RemoveOptions)
// didn't finish and to run `vaultik snapshot remove` for the snapshot // didn't finish and to run `vaultik snapshot remove` for the snapshot
// again once the destination store is reachable (`vaultik prune` never // again once the destination store is reachable (`vaultik prune` never
// removes snapshot metadata). Returns true when the remote removal // removes snapshot metadata). Returns true when the remote removal
// succeeded. // succeeded and deleted something.
func (v *Vaultik) removeSnapshotRemote(snapshotID string, opts *RemoveOptions) bool { func (v *Vaultik) removeSnapshotRemote(snapshotID string, opts *RemoveOptions) bool {
log.Info("Removing snapshot metadata from remote storage", log.Info("Removing snapshot metadata from remote storage",
"snapshot_id", snapshotID) "snapshot_id", snapshotID)
remoteKey := snapshot.RemoteSnapshotKey(snapshotID) remoteKey := snapshot.RemoteSnapshotKey(snapshotID)
err := v.deleteRemoteSnapshotByKey(remoteKey) deleted, err := v.deleteRemoteSnapshotByKey(remoteKey)
if err != nil { if err != nil {
log.Warn("Could not remove snapshot metadata from remote storage; "+ log.Warn("Could not remove snapshot metadata from remote storage; "+
"run '"+snapshotRemoveCommandHint+"' with the snapshot's ID "+ "run '"+snapshotRemoveCommandHint+"' with the snapshot's ID "+
@@ -1270,7 +1355,7 @@ func (v *Vaultik) removeSnapshotRemote(snapshotID string, opts *RemoveOptions) b
return false return false
} }
return true return deleted > 0
} }
// RemoveAllSnapshots removes every snapshot known to the local // RemoveAllSnapshots removes every snapshot known to the local
@@ -1562,7 +1647,7 @@ func (v *Vaultik) removeAllLocalSnapshots(
} }
if !opts.LocalOnly { if !opts.LocalOnly {
err = v.deleteRemoteSnapshotByKey( _, err = v.deleteRemoteSnapshotByKey(
snapshot.RemoteSnapshotKey(snapshotID)) snapshot.RemoteSnapshotKey(snapshotID))
if err != nil { if err != nil {
log.Warn("Failed to remove snapshot metadata from remote", log.Warn("Failed to remove snapshot metadata from remote",
@@ -1588,7 +1673,7 @@ func (v *Vaultik) removeAllOrphanRemotes(
for _, key := range orphanRemoteKeys { for _, key := range orphanRemoteKeys {
log.Info("Removing orphan remote snapshot", "remote_key", key) log.Info("Removing orphan remote snapshot", "remote_key", key)
err := v.deleteRemoteSnapshotByKey(key) _, err := v.deleteRemoteSnapshotByKey(key)
if err != nil { if err != nil {
log.Warn("Failed to remove orphan from remote", log.Warn("Failed to remove orphan from remote",
"remote_key", key, "error", err) "remote_key", key, "error", err)
@@ -1635,34 +1720,44 @@ func (v *Vaultik) deleteSnapshotFromLocalDB(snapshotID string) error {
} }
// deleteRemoteSnapshotByKey removes everything under // deleteRemoteSnapshotByKey removes everything under
// metadata/<remoteKey>/ on the destination store. The argument is a // metadata/<remoteKey>/ on the destination store and returns how many
// remote key (double-SHA256 derivation), not a human snapshot ID; // objects it deleted. The argument is a remote key (double-SHA256
// callers that have a human ID must hash via snapshot.RemoteSnapshotKey // derivation), not a human snapshot ID; callers that have a human ID
// first. // must hash via snapshot.RemoteSnapshotKey first.
func (v *Vaultik) deleteRemoteSnapshotByKey(remoteKey string) error { func (v *Vaultik) deleteRemoteSnapshotByKey(remoteKey string) (int, error) {
prefix := fmt.Sprintf("metadata/%s/", remoteKey) objectsToDelete, err := v.remoteSnapshotObjects(remoteKey)
objectCh := v.Storage.ListStream(v.ctx, prefix) if err != nil {
return 0, err
var objectsToDelete []string
for object := range objectCh {
if object.Err != nil {
return fmt.Errorf("listing objects: %w", object.Err)
}
objectsToDelete = append(objectsToDelete, object.Key)
} }
for _, key := range objectsToDelete { for i, key := range objectsToDelete {
err := v.Storage.Delete(v.ctx, key) err = v.Storage.Delete(v.ctx, key)
if err != nil { if err != nil {
return fmt.Errorf("removing %s: %w", key, err) return i, fmt.Errorf("removing %s: %w", key, err)
} }
log.Debug("Deleted remote object", "key", key) log.Debug("Deleted remote object", "key", key)
} }
return nil return len(objectsToDelete), nil
}
// remoteSnapshotObjects lists the keys of every object under
// metadata/<remoteKey>/ on the destination store.
func (v *Vaultik) remoteSnapshotObjects(remoteKey string) ([]string, error) {
prefix := fmt.Sprintf("metadata/%s/", remoteKey)
var keys []string
for object := range v.Storage.ListStream(v.ctx, prefix) {
if object.Err != nil {
return nil, fmt.Errorf("listing objects: %w", object.Err)
}
keys = append(keys, object.Key)
}
return keys, nil
} }
// outputRemoveJSON outputs the removal result as JSON // outputRemoveJSON outputs the removal result as JSON
@@ -1756,9 +1851,9 @@ func (v *Vaultik) PruneDatabase() (*PruneResult, error) {
return result, nil return result, nil
} }
// countUnknown is what a count reads as when its query could not be run, // unknownText is what a count or size reads as when it could not be
// distinct from "0", which means the table really was empty. // determined, distinct from "0", which is a real zero.
const countUnknown = "unknown" const unknownText = "unknown"
// tableCountForReport returns the row count of a table for the prune // tableCountForReport returns the row count of a table for the prune
// summary, or nil if the count could not be read. A read failure is // summary, or nil if the count could not be read. A read failure is
@@ -1795,7 +1890,7 @@ func countDiff(before, after *int64) *int64 {
// one that could not be queried. // one that could not be queried.
func countText(count *int64) string { func countText(count *int64) string {
if count == nil { if count == nil {
return countUnknown return unknownText
} }
return strconv.FormatInt(*count, 10) return strconv.FormatInt(*count, 10)
@@ -0,0 +1,123 @@
package vaultik_test
import (
"bytes"
"context"
"fmt"
"os"
"path/filepath"
"testing"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/config"
"sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/storage"
"sneak.berlin/go/vaultik/internal/ui"
"sneak.berlin/go/vaultik/internal/vaultik"
)
// openFailFs is the real filesystem, except that opening path fails
// with err. Phase 1 of a backup only lstats a file, so it still counts
// path; phase 2 is the first to open it.
type openFailFs struct {
afero.OsFs
path string
err error
}
//nolint:ireturn // afero.Fs.Open is defined to return the interface.
func (f *openFailFs) Open(name string) (afero.File, error) {
if name == f.path {
return nil, &os.PathError{Op: "open", Path: name, Err: f.err}
}
return f.OsFs.Open(name)
}
// A file that phase 2 cannot open is reported as failed, not as
// unchanged, and neither the summary's data total nor the snapshots row
// counts it. See https://git.eeqj.de/sneak/vaultik/issues/280.
func TestSnapshotSummaryCountsFileNotStoredAsFailed(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
tests := []struct {
name string
openErr error
skipErrors bool
}{
// What a normal user gets opening a file with mode 000.
{name: "unopenable under skip-errors",
openErr: os.ErrPermission, skipErrors: true},
// What opening a file removed after phase 1 gives.
{name: "removed between the phases",
openErr: os.ErrNotExist, skipErrors: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
ctx := context.Background()
// The scan walks the source path with symlinks resolved, so
// failedPath must be spelled the same way to match.
tempDir, err := filepath.EvalSymlinks(t.TempDir())
require.NoError(t, err)
srcDir := filepath.Join(tempDir, "src")
failedPath := filepath.Join(srcDir, "failed.txt")
storedContent := []byte("this file is backed up")
storedSize := int64(len(storedContent))
fs := &openFailFs{path: failedPath, err: tt.openErr}
require.NoError(t, fs.MkdirAll(srcDir, 0o755))
require.NoError(t, afero.WriteFile(fs,
filepath.Join(srcDir, "stored.txt"), storedContent, 0o644))
require.NoError(t, afero.WriteFile(fs,
failedPath, []byte("this file cannot be opened"), 0o644))
cfg := faultTestConfig()
cfg.IndexPath = filepath.Join(tempDir, "index.sqlite")
cfg.Snapshots = map[string]config.SnapshotConfig{
"src": {Paths: []string{srcDir}},
}
store, err := storage.NewFileStorer(filepath.Join(tempDir, "remote"))
require.NoError(t, err)
db, err := database.New(ctx, cfg.IndexPath)
require.NoError(t, err)
t.Cleanup(func() { _ = db.Close() })
repos := database.NewRepositories(db)
out := &bytes.Buffer{}
v := newBackupVaultik(ctx, cfg, store, repos, db, fs)
v.UI = ui.NewWithColor(out, false)
require.NoError(t, v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
SkipErrors: tt.skipErrors,
Snapshots: []string{"src"},
}))
summary := out.String()
assert.Contains(t, summary,
"Files: 2 examined, 1 backed up, 0 unchanged, 1 failed.")
assert.Contains(t, summary,
fmt.Sprintf("Data: %s total (%s backed up).",
v.UI.Size(storedSize), v.UI.Size(storedSize)))
snap, err := repos.Snapshots.GetByID(ctx,
localSnapshotID(ctx, t, repos, "src"))
require.NoError(t, err)
require.NotNil(t, snap)
assert.Equal(t, int64(1), snap.FileCount)
assert.Equal(t, storedSize, snap.TotalSize)
})
}
}
@@ -0,0 +1,75 @@
package vaultik_test
import (
"context"
"fmt"
"path/filepath"
"testing"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/config"
"sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/storage"
"sneak.berlin/go/vaultik/internal/vaultik"
)
// --skip-errors skips only a file that cannot be opened or read. A local
// index error while a directory is recorded stops the backup, and the
// snapshot is not recorded as complete. See
// https://git.eeqj.de/sneak/vaultik/issues/284.
func TestSkipErrorsBackupStopsOnIndexErrorRecordingDirectory(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
// The scan walks the source path with symlinks resolved, so dirPath
// must be spelled the same way to match the row the scan inserts.
tempDir, err := filepath.EvalSymlinks(t.TempDir())
require.NoError(t, err)
srcDir := filepath.Join(tempDir, "src")
dirPath := filepath.Join(srcDir, "dir")
fs := afero.NewOsFs()
require.NoError(t, fs.MkdirAll(dirPath, 0o755))
require.NoError(t, afero.WriteFile(fs,
filepath.Join(dirPath, "file.txt"), []byte("file content"), 0o644))
cfg := faultTestConfig()
cfg.IndexPath = filepath.Join(tempDir, "index.sqlite")
cfg.Snapshots = map[string]config.SnapshotConfig{
"tree": {Paths: []string{srcDir}},
}
store, err := storage.NewFileStorer(filepath.Join(tempDir, "remote"))
require.NoError(t, err)
db, err := database.New(ctx, cfg.IndexPath)
require.NoError(t, err)
t.Cleanup(func() { _ = db.Close() })
// The local index refuses the directory's files row.
_, err = db.Conn().ExecContext(ctx, fmt.Sprintf(`
CREATE TRIGGER refuse_directory BEFORE INSERT ON files
WHEN NEW.path = '%s'
BEGIN SELECT RAISE(ABORT, 'simulated index error'); END`, dirPath))
require.NoError(t, err)
repos := database.NewRepositories(db)
v := newBackupVaultik(ctx, cfg, store, repos, db, fs)
err = v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
SkipErrors: true,
Snapshots: []string{"tree"},
})
require.ErrorContains(t, err, "simulated index error")
snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
require.NoError(t, err)
require.Len(t, snapshots, 1)
assert.Nil(t, snapshots[0].CompletedAt)
}
+10 -10
View File
@@ -9,7 +9,7 @@ import (
) )
// errStorageBindingMismatch is returned when the local index database is // errStorageBindingMismatch is returned when the local index database is
// bound to a different destination than the configured storage_url. // bound to a different destination than the configured one.
var errStorageBindingMismatch = errors.New( var errStorageBindingMismatch = errors.New(
"local index is bound to a different backup destination") "local index is bound to a different backup destination")
@@ -24,8 +24,9 @@ var errStorageBindingMismatch = errors.New(
// //
// Behaviour: // Behaviour:
// - On first use (empty stored value), record the configured // - On first use (empty stored value), record the configured
// storage_url and log the binding. // destination (storage_url, or the s3.* endpoint, bucket and
// - When the stored value matches the configured storage_url, do // prefix) and log the binding.
// - When the stored value matches the configured destination, do
// nothing and return nil. // nothing and return nil.
// - When the two differ, refuse with an error that tells the user // - When the two differ, refuse with an error that tells the user
// how to recover (revert the config, or run `vaultik database // how to recover (revert the config, or run `vaultik database
@@ -46,12 +47,10 @@ func (v *Vaultik) EnsureStorageBinding() error {
configured := v.Config.StorageURL configured := v.Config.StorageURL
if configured == "" { if configured == "" {
// Some legacy configs still use the split s3.* keys instead of // The s3.* form: bind on its endpoint, bucket and prefix,
// storage_url. Falling back to a synthetic URL for those would // written the way storage_url would name the same destination.
// hide the fact that the binding is loose. Instead, treat s3 := v.Config.S3
// unset as "nothing to bind against" — the check is configured = "s3://" + s3.Bucket + "/" + s3.Prefix + "?endpoint=" + s3.Endpoint
// necessarily best-effort for those configs.
return nil
} }
stored, err := v.Repositories.LocalMeta.Get(v.ctx, database.LocalMetaKeyStorageURL) stored, err := v.Repositories.LocalMeta.Get(v.ctx, database.LocalMetaKeyStorageURL)
@@ -96,7 +95,8 @@ func buildBindingMismatchMessage(stored, configured string) string {
"future snapshots referencing blobs that don't exist at the new destination.\n" + "future snapshots referencing blobs that don't exist at the new destination.\n" +
"\n" + "\n" +
"To proceed, either:\n" + "To proceed, either:\n" +
" - revert storage_url in your config to the bound destination, or\n" + " - revert storage_url (or the s3.* endpoint, bucket and prefix) in your\n" +
" config to the bound destination, or\n" +
" - run 'vaultik database delete' to discard the local index and rebuild it\n" + " - run 'vaultik database delete' to discard the local index and rebuild it\n" +
" from a fresh full backup against the new destination" " from a fresh full backup against the new destination"
} }
+48
View File
@@ -76,3 +76,51 @@ func TestEnsureStorageBinding_MismatchRefuses(t *testing.T) {
assert.Contains(t, err.Error(), "file:///mnt/backups/new") assert.Contains(t, err.Error(), "file:///mnt/backups/new")
assert.Contains(t, err.Error(), "vaultik database delete") assert.Contains(t, err.Error(), "vaultik database delete")
} }
// A config with no storage_url names its destination with s3.endpoint,
// s3.bucket and s3.prefix; a change to any of them must be refused just
// as a changed storage_url is.
func TestEnsureStorageBinding_S3FieldsChangeRefuses(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
bound := config.S3Config{
Endpoint: "https://s3.example.com",
Bucket: "bucket-a",
Prefix: "backups",
}
tests := []struct {
name string
changed config.S3Config
}{
{"bucket", config.S3Config{
Endpoint: bound.Endpoint, Bucket: "bucket-b", Prefix: bound.Prefix,
}},
{"prefix", config.S3Config{
Endpoint: bound.Endpoint, Bucket: bound.Bucket, Prefix: "other",
}},
{"endpoint", config.S3Config{
Endpoint: "https://s3.other.com", Bucket: bound.Bucket, Prefix: bound.Prefix,
}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
tv, _ := buildBindTestVaultik(t, "")
tv.Config.S3 = bound
require.NoError(t, tv.EnsureStorageBinding(), "first use binds")
require.NoError(t, tv.EnsureStorageBinding(), "same destination passes")
tv.Config.S3 = tt.changed
err := tv.EnsureStorageBinding()
require.Error(t, err)
assert.Contains(t, err.Error(),
"local index is bound to a different backup destination")
assert.Contains(t, err.Error(), "vaultik database delete")
})
}
}