Compare commits

3 Commits
Author SHA1 Message Date
sneak cc884419de Apply restored owners, modes and times in an order that keeps them (closes #219)
check / check (push) Successful in 10m17s
A directory got its stored mode and mtime before its contents were
written, so a read-only directory came back without its files and a
non-empty one carried the time of the restore. Directories are now
created owner-only (0700) and get their stored owner, mode and mtime
after the restore loop, each before its parent, skipping any whose
place a symlink has since taken. A file's mode is now applied after its
chown, which on Linux clears setuid and setgid. A symlink gets its
stored owner (as root) and mtime on the link itself, through
golang.org/x/sys/unix, now a direct dependency.

An interrupted restore leaves its directories at 0700.

Model: opus-5-5
2026-10-06 15:11:12 +00:00
clawbot 315b6483b8 Mark github.com/spf13/pflag as a direct dependency in go.mod (closes #246)
check / check (push) Successful in 12m34s
internal/cli/snapshot_restore_test.go imports github.com/spf13/pflag
directly, but go.mod still marked it // indirect. script/precommit runs
go mod tidy and fails when that changes go.mod, so the pre-commit hook
stopped every commit. This is the go mod tidy output: pflag moves to the
direct require block, and go.sum does not change. script/cibuild does
not run the tidy, which is why the gate stayed green.

Model: opus-5-5
2026-10-06 16:59:26 +02:00
clawbot 14fc4c9893 Load a config that has no age recipient (closes #221)
check / check (push) Successful in 13m47s
The README's steps for restoring on another machine failed at the first
command: `config init` wrote a placeholder recipient, and `config.Load`
rejects any recipient that does not parse. `config init` now writes an
empty `age_recipients` list, `config.Load` accepts an empty list, and
`snapshot create` refuses to start without a recipient. A malformed
recipient is still rejected at load.

The recovery-host test now builds its config with `config init` and
`config set` and reads it through `config.Load`, so it imports
`internal/cli`. On a fresh file, `config set age_recipients.0` writes the
list in flow style (`[age1...]`).

Model: opus-5-5
2026-10-06 15:46:13 +02:00
12 changed files with 582 additions and 50 deletions
+1 -1
View File
@@ -530,7 +530,7 @@ complete annotated example also lives in
| Field | Default | Description | | Field | Default | Description |
|-------|---------|-------------| |-------|---------|-------------|
| `age_recipients` | (required) | Age public keys for encryption | | `age_recipients` | (required by `snapshot create`) | Age public keys for encryption. Other commands run without one, so a machine that only restores can leave it empty |
| `age_secret_key` | (unset) | Age private key for decryption (`snapshot restore`, `snapshot verify --deep`). Setting it in the config file places the private key on the backed-up host, defeating the public-key-only design (see "why" above). Prefer the `VAULTIK_AGE_SECRET_KEY` environment variable, supplied only on the machine you restore from. | | `age_secret_key` | (unset) | Age private key for decryption (`snapshot restore`, `snapshot verify --deep`). Setting it in the config file places the private key on the backed-up host, defeating the public-key-only design (see "why" above). Prefer the `VAULTIK_AGE_SECRET_KEY` environment variable, supplied only on the machine you restore from. |
| `snapshots` | (required) | Named snapshot definitions with paths and excludes | | `snapshots` | (required) | Named snapshot definitions with paths and excludes |
| `storage_url` | | Storage backend URL (`s3://`, `file://`, `rclone://`) | | `storage_url` | | Storage backend URL (`s3://`, `file://`, `rclone://`) |
+27
View File
@@ -22,6 +22,33 @@ the tag exists and is exercised; what is left is merging `next` to
# Completed Steps # Completed Steps
- 2026-10-06: Made restore apply owners, modes and times in an order
that keeps them
([issue #219](https://git.eeqj.de/sneak/vaultik/issues/219)). A
directory got its stored mode and mtime before its contents were
written, so a read-only directory came back without its files and a
non-empty one carried the time of the restore. Directories are now
created owner-only and get their stored owner, mode and mtime once the
restore loop is done, each before its parent. A file's mode is applied
after its chown, which on Linux clears setuid and setgid, and a
symlink gets its stored owner (as root) and mtime on the link itself.
- 2026-10-06: Made `go.mod` what `go mod tidy` writes, so the
pre-commit hook no longer stops every commit
([issue #246](https://git.eeqj.de/sneak/vaultik/issues/246)). A test
in `internal/cli` imports `github.com/spf13/pflag` directly, but
`go.mod` still marked it `// indirect`, and `script/precommit` fails
whenever the tidy changes `go.mod`. It is now in the direct `require`
block.
- 2026-10-06: Made the README's steps for restoring on another machine
work ([issue #221](https://git.eeqj.de/sneak/vaultik/issues/221)).
`config init` wrote a placeholder recipient that `config.Load`
rejects, so every command on the new machine failed before reaching
the store. The file now has an empty `age_recipients` list,
`config.Load` accepts an empty list, and `snapshot create` refuses to
run without a recipient.
- 2026-10-06: Made `snapshot restore --skip-errors` skip the files that - 2026-10-06: Made `snapshot restore --skip-errors` skip the files that
need a blob it cannot download need a blob it cannot download
([issue #218](https://git.eeqj.de/sneak/vaultik/issues/218)). A missing ([issue #218](https://git.eeqj.de/sneak/vaultik/issues/218)). A missing
+2 -1
View File
@@ -3,7 +3,8 @@
# Copy this file and uncomment/modify the values you need # Copy this file and uncomment/modify the values you need
# Age recipient public keys for encryption # Age recipient public keys for encryption
# This is REQUIRED - backups are encrypted to these public keys # Backups are encrypted to these public keys. snapshot create needs at least
# one; listing, verifying and restoring do not
# Generate with: age-keygen | grep "public key" # Generate with: age-keygen | grep "public key"
age_recipients: age_recipients:
- age1cj2k2addawy294f6k2gr2mf9gps9r3syplryxca3nvxj3daqm96qfp84tz - age1cj2k2addawy294f6k2gr2mf9gps9r3syplryxca3nvxj3daqm96qfp84tz
+2 -2
View File
@@ -20,9 +20,11 @@ require (
github.com/rclone/rclone v1.72.1 github.com/rclone/rclone v1.72.1
github.com/spf13/afero v1.15.0 github.com/spf13/afero v1.15.0
github.com/spf13/cobra v1.10.1 github.com/spf13/cobra v1.10.1
github.com/spf13/pflag v1.0.10
github.com/stretchr/testify v1.11.1 github.com/stretchr/testify v1.11.1
go.uber.org/fx v1.24.0 go.uber.org/fx v1.24.0
golang.org/x/sync v0.18.0 golang.org/x/sync v0.18.0
golang.org/x/sys v0.38.0
golang.org/x/term v0.37.0 golang.org/x/term v0.37.0
gopkg.in/yaml.v3 v3.0.1 gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.38.0 modernc.org/sqlite v1.38.0
@@ -225,7 +227,6 @@ require (
github.com/smarty/assertions v1.16.0 // indirect github.com/smarty/assertions v1.16.0 // indirect
github.com/sony/gobreaker v1.0.0 // indirect github.com/sony/gobreaker v1.0.0 // indirect
github.com/spacemonkeygo/monkit/v3 v3.0.25-0.20251022131615-eb24eb109368 // indirect github.com/spacemonkeygo/monkit/v3 v3.0.25-0.20251022131615-eb24eb109368 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/t3rm1n4l/go-mega v0.0.0-20251031123324-a804aaa87491 // indirect github.com/t3rm1n4l/go-mega v0.0.0-20251031123324-a804aaa87491 // indirect
github.com/tidwall/gjson v1.18.0 // indirect github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/match v1.1.1 // indirect
@@ -263,7 +264,6 @@ require (
golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect
golang.org/x/net v0.47.0 // indirect golang.org/x/net v0.47.0 // indirect
golang.org/x/oauth2 v0.33.0 // indirect golang.org/x/oauth2 v0.33.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/text v0.31.0 // indirect golang.org/x/text v0.31.0 // indirect
golang.org/x/time v0.14.0 // indirect golang.org/x/time v0.14.0 // indirect
golang.org/x/tools v0.38.0 // indirect golang.org/x/tools v0.38.0 // indirect
+7 -4
View File
@@ -45,16 +45,19 @@ const defaultConfigTemplate = `# vaultik configuration
# ─── REQUIRED ──────────────────────────────────────────────────────────────── # ─── REQUIRED ────────────────────────────────────────────────────────────────
# Age recipient public keys for encryption. # Age recipient public keys for encryption. snapshot create needs at least
# one; listing, verifying and restoring do not, so a machine that only
# restores can leave this empty.
# Backups are encrypted to ALL listed recipients; any one of the corresponding # Backups are encrypted to ALL listed recipients; any one of the corresponding
# private keys can decrypt. Adding a recipient later does not re-encrypt data # private keys can decrypt. Adding a recipient later does not re-encrypt data
# already stored: deduplicated chunks and existing blobs stay encrypted to the # already stored: deduplicated chunks and existing blobs stay encrypted to the
# earlier recipients, so a newly added key cannot restore them on its own (see # earlier recipients, so a newly added key cannot restore them on its own (see
# docs/REPOSTRUCTURE.md, Accepted Risks). Generate a keypair with: # docs/REPOSTRUCTURE.md, Accepted Risks). Generate a keypair and add its
# public key with:
# age-keygen -o vaultik_backup_private_key.txt # age-keygen -o vaultik_backup_private_key.txt
# grep 'public key' vaultik_backup_private_key.txt # grep 'public key' vaultik_backup_private_key.txt
age_recipients: # vaultik config set age_recipients.0 age1...
- age1REPLACE_WITH_YOUR_PUBLIC_KEY age_recipients: []
# Named snapshots. Each snapshot backs up one or more paths and can have its # Named snapshots. Each snapshot backs up one or more paths and can have its
# own exclude patterns in addition to the global excludes below. # own exclude patterns in addition to the global excludes below.
+41 -2
View File
@@ -24,8 +24,10 @@ func TestDefaultConfigTemplateParses(t *testing.T) {
t.Fatalf("default config template is not valid YAML: %v", err) t.Fatalf("default config template is not valid YAML: %v", err)
} }
if len(cfg.AgeRecipients) != 1 { // A placeholder recipient would fail config.Load, so the template
t.Errorf("expected 1 placeholder age recipient, got %d", len(cfg.AgeRecipients)) // leaves the list empty.
if len(cfg.AgeRecipients) != 0 {
t.Errorf("expected no age recipients, got %d", len(cfg.AgeRecipients))
} }
home, ok := cfg.Snapshots["home"] home, ok := cfg.Snapshots["home"]
@@ -55,6 +57,43 @@ func TestDefaultConfigTemplateParses(t *testing.T) {
} }
} }
// TestConfigSetRecipientOnFreshConfig follows the README quickstart: on the
// file `config init` writes, `config set age_recipients.0` and
// `config set storage_url` give a config that loads with that recipient.
func TestConfigSetRecipientOnFreshConfig(t *testing.T) {
t.Parallel()
const recipient = "age1278m9q7dp3chsh2dcy82qk27v047zywyvtxwnj4cvt0z65jw6a7q5dqhfj"
path := filepath.Join(t.TempDir(), "config.yml")
err := os.WriteFile(path, []byte(defaultConfigTemplate), configFileMode)
if err != nil {
t.Fatalf("write config: %v", err)
}
out := ui.NewWithColor(&bytes.Buffer{}, false)
err = writeConfigSet(out, path, "age_recipients.0", recipient)
if err != nil {
t.Fatalf("config set age_recipients.0: %v", err)
}
err = writeConfigSet(out, path, "storage_url", "file:///mnt/backups")
if err != nil {
t.Fatalf("config set storage_url: %v", err)
}
cfg, err := config.Load(path)
if err != nil {
t.Fatalf("config.Load: %v", err)
}
if len(cfg.AgeRecipients) != 1 || cfg.AgeRecipients[0] != recipient {
t.Errorf("age_recipients = %v, want [%s]", cfg.AgeRecipients, recipient)
}
}
const testYAML = `# top comment const testYAML = `# top comment
compression_level: 3 compression_level: 3
age_recipients: age_recipients:
+5 -10
View File
@@ -42,9 +42,7 @@ const (
// Sentinel validation errors. // Sentinel validation errors.
var ( var (
errNoConfigPath = errors.New("config path not provided") errNoConfigPath = errors.New("config path not provided")
errNoAgeRecipients = errors.New(
"at least one age_recipient is required (generate with: age-keygen)")
errRecipientIsSecretKey = errors.New( errRecipientIsSecretKey = errors.New(
"an age secret key was given where a public key (age1...) belongs") "an age secret key was given where a public key (age1...) belongs")
errRecipientNotX25519 = errors.New( errRecipientNotX25519 = errors.New(
@@ -323,9 +321,10 @@ func Load(path string) (*Config, error) {
// Validate checks if the configuration is valid and complete. // Validate checks if the configuration is valid and complete.
// It ensures all required fields are present and have valid values: // It ensures all required fields are present and have valid values:
// - At least one age recipient must be specified, and every recipient must // - Every age recipient must parse as an X25519 age1... public key (so a
// parse as an X25519 age1... public key (so a bad entry fails at load, not // bad entry fails at load, not mid-backup); errors name the position,
// mid-backup); errors name the position, never the value // never the value. An empty list is accepted, because only snapshot
// create needs a recipient and it checks for one itself
// - At least one snapshot must be configured with at least one path // - At least one snapshot must be configured with at least one path
// - Storage must be configured (either storage_url or s3.* fields) // - Storage must be configured (either storage_url or s3.* fields)
// - Chunk size must be at least 1MB // - Chunk size must be at least 1MB
@@ -336,10 +335,6 @@ func Load(path string) (*Config, error) {
// //
// Returns an error describing the first validation failure encountered. // Returns an error describing the first validation failure encountered.
func (c *Config) Validate() error { func (c *Config) Validate() error {
if len(c.AgeRecipients) == 0 {
return errNoAgeRecipients
}
for i, recipient := range c.AgeRecipients { for i, recipient := range c.AgeRecipients {
err := validateAgeRecipient(recipient) err := validateAgeRecipient(recipient)
if err != nil { if err != nil {
+8 -2
View File
@@ -223,7 +223,8 @@ func TestValidateBlobSizeLimit(t *testing.T) {
// TestValidateAgeRecipients checks that recipients are parsed at config load // TestValidateAgeRecipients checks that recipients are parsed at config load
// (a bad entry fails immediately, not mid-backup) and that no invalid entry — // (a bad entry fails immediately, not mid-backup) and that no invalid entry —
// least of all a pasted secret key — is echoed in the error. // least of all a pasted secret key — is echoed in the error. An empty list
// loads, because only snapshot create needs a recipient.
func TestValidateAgeRecipients(t *testing.T) { func TestValidateAgeRecipients(t *testing.T) {
t.Parallel() t.Parallel()
@@ -244,7 +245,12 @@ func TestValidateAgeRecipients(t *testing.T) {
wantErr bool wantErr bool
}{ }{
{ {
name: "config init placeholder is rejected", name: "no recipients is accepted",
recipients: nil,
wantErr: false,
},
{
name: "placeholder recipient is rejected",
recipients: []string{"age1REPLACE_WITH_YOUR_PUBLIC_KEY"}, recipients: []string{"age1REPLACE_WITH_YOUR_PUBLIC_KEY"},
wantErr: true, wantErr: true,
}, },
+86 -16
View File
@@ -10,11 +10,13 @@ import (
"math" "math"
"os" "os"
"path/filepath" "path/filepath"
"slices"
"strings" "strings"
"time" "time"
"filippo.io/age" "filippo.io/age"
"github.com/spf13/afero" "github.com/spf13/afero"
"golang.org/x/sys/unix"
"sneak.berlin/go/vaultik/internal/blobgen" "sneak.berlin/go/vaultik/internal/blobgen"
"sneak.berlin/go/vaultik/internal/database" "sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log" "sneak.berlin/go/vaultik/internal/log"
@@ -63,6 +65,12 @@ const snapshotDBFilename = "snapshot.db"
// directories themselves get their stored mode). // directories themselves get their stored mode).
const restoreDirMode = 0o755 const restoreDirMode = 0o755
// restoreDirCreateMode is the owner-only mode a directory from the
// snapshot is created with during restore, so its contents can be written
// whatever its stored mode. The stored mode is applied after the restore
// loop, by applyDirectoryMetadata.
const restoreDirCreateMode = 0o700
// restoreFileMode is the restrictive mode a regular file is created with // restoreFileMode is the restrictive mode a regular file is created with
// during restore. Content is written while the file holds this mode; the // during restore. Content is written while the file holds this mode; the
// stored mode is applied only after the file is fully written and closed, // stored mode is applied only after the file is fully written and closed,
@@ -332,6 +340,8 @@ func (v *Vaultik) restoreAllFiles(
return nil, err return nil, err
} }
session.applyDirectoryMetadata()
return result, nil return result, nil
} }
@@ -901,6 +911,9 @@ type restoreSession struct {
// the call entirely as non-root and emit one warning at the end // the call entirely as non-root and emit one warning at the end
// of the restore explaining that ownership was not preserved. // of the restore explaining that ownership was not preserved.
runningAsRoot bool runningAsRoot bool
// directories holds every restored directory, for
// applyDirectoryMetadata to finish after the restore loop.
directories []*database.File
} }
// containedRestorePath resolves rel — a path read from the snapshot // containedRestorePath resolves rel — a path read from the snapshot
@@ -1007,6 +1020,8 @@ func (s *restoreSession) restoreSymlink(file *database.File, targetPath string)
if err != nil { if err != nil {
return fmt.Errorf("creating symlink: %w", err) return fmt.Errorf("creating symlink: %w", err)
} }
s.applySymlinkMetadata(file, targetPath)
} else { } else {
log.Debug("Symlink creation not supported on this filesystem", log.Debug("Symlink creation not supported on this filesystem",
"path", file.Path, "target", file.LinkTarget) "path", file.Path, "target", file.LinkTarget)
@@ -1019,35 +1034,90 @@ func (s *restoreSession) restoreSymlink(file *database.File, targetPath string)
return nil return nil
} }
// restoreDirectory restores a directory with its permissions, mtime, // restoreDirectory creates a directory with restoreDirCreateMode. Its
// and (on real filesystems, with sufficient privileges) ownership. // stored mode, owner and mtime are applied after the restore loop by
// applyDirectoryMetadata: a read-only stored mode would block writing
// its contents, and writing them changes its mtime.
func (s *restoreSession) restoreDirectory( func (s *restoreSession) restoreDirectory(
file *database.File, targetPath string, file *database.File, targetPath string,
) error { ) error {
err := s.v.Fs.MkdirAll(targetPath, os.FileMode(file.Mode)) err := s.v.Fs.MkdirAll(targetPath, restoreDirCreateMode)
if err != nil { if err != nil {
return fmt.Errorf("creating directory: %w", err) return fmt.Errorf("creating directory: %w", err)
} }
// MkdirAll applies the process umask, so chmod to the exact stored s.directories = append(s.directories, file)
// mode. A failure here is non-fatal.
err = s.v.Fs.Chmod(targetPath, os.FileMode(file.Mode))
if err != nil {
log.Debug("Failed to set permissions", "path", targetPath, "error", err)
}
s.applyFileMetadata(file, targetPath)
s.result.FilesRestored++ s.result.FilesRestored++
return nil return nil
} }
// applyDirectoryMetadata applies the stored owner, mtime and mode to every
// restored directory, each before its parent, so a parent whose stored
// mode denies search does not block its children. Failures are logged at
// debug level and do not abort the restore.
func (s *restoreSession) applyDirectoryMetadata() {
// A path sorts after its parent's, so reverse order puts every
// directory before its parent.
slices.SortFunc(s.directories, func(a, b *database.File) int {
return strings.Compare(b.Path.String(), a.Path.String())
})
for _, dir := range s.directories {
targetPath, err := containedRestorePath(
s.v.Fs, s.opts.TargetDir, dir.Path.String())
if err != nil {
log.Debug("Failed to set directory metadata",
"path", dir.Path, "error", err)
continue
}
// A later entry can have put a symlink in the directory's place,
// for example one stored as "/d/" next to the directory "/d". The
// calls below follow symlinks, so they would change its target.
info, err := lstatIfPossible(s.v.Fs, targetPath)
if err != nil || !info.IsDir() {
log.Debug("Not setting directory metadata: no longer a directory",
"path", targetPath, "error", err)
continue
}
s.applyFileMetadata(dir, targetPath)
err = s.v.Fs.Chmod(targetPath, os.FileMode(dir.Mode))
if err != nil {
log.Debug("Failed to set permissions", "path", targetPath, "error", err)
}
}
}
// applySymlinkMetadata applies ownership (when running as root) and mtime
// to a restored symlink itself; os.Chown and Chtimes would follow it.
// Failures are logged at debug level and do not abort the restore.
func (s *restoreSession) applySymlinkMetadata(file *database.File, targetPath string) {
if s.runningAsRoot {
err := os.Lchown(targetPath, int(file.UID), int(file.GID))
if err != nil {
log.Debug("Failed to set ownership", "path", targetPath, "error", err)
}
}
mtime := unix.NsecToTimeval(file.MTime.UnixNano())
err := unix.Lutimes(targetPath, []unix.Timeval{mtime, mtime})
if err != nil {
log.Debug("Failed to set mtime", "path", targetPath, "error", err)
}
}
// applyFileMetadata applies ownership (when running as root on a real // applyFileMetadata applies ownership (when running as root on a real
// filesystem) and mtime to a restored path. Permission mode is applied // filesystem) and mtime to a restored path. The caller applies the mode
// separately by each caller, with different failure handling, so it is // afterwards: on Linux a chown clears the setuid and setgid bits of a
// not touched here. Failures are logged at debug level and do not abort // regular file. Failures are logged at debug level and do not abort the
// the restore. // restore.
func (s *restoreSession) applyFileMetadata(file *database.File, targetPath string) { func (s *restoreSession) applyFileMetadata(file *database.File, targetPath string) {
if s.runningAsRoot { if s.runningAsRoot {
if _, ok := s.v.Fs.(*afero.OsFs); ok { if _, ok := s.v.Fs.(*afero.OsFs); ok {
@@ -1138,8 +1208,8 @@ func (s *restoreSession) restoreRegularFile(
return fmt.Errorf("closing output file: %w", err) return fmt.Errorf("closing output file: %w", err)
} }
s.applyRestoredFileMode(file, targetPath)
s.applyFileMetadata(file, targetPath) s.applyFileMetadata(file, targetPath)
s.applyRestoredFileMode(file, targetPath)
s.result.FilesRestored++ s.result.FilesRestored++
s.result.BytesRestored += bytesWritten s.result.BytesRestored += bytesWritten
@@ -10,6 +10,7 @@ import (
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/cli"
"sneak.berlin/go/vaultik/internal/config" "sneak.berlin/go/vaultik/internal/config"
"sneak.berlin/go/vaultik/internal/database" "sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log" "sneak.berlin/go/vaultik/internal/log"
@@ -27,10 +28,12 @@ import (
// //
// The backup half writes a snapshot with one index and hostname. The // The backup half writes a snapshot with one index and hostname. The
// restore half throws that index away entirely: a fresh, empty index and // restore half throws that index away entirely: a fresh, empty index and
// a config that shares nothing with the original but the storage location // a config written by `config init` and `config set storage_url`, as in
// and the secret key. If restore or verify needed the original local // the README's steps for restoring on another machine, which shares
// index — or the human snapshot ID that only that index holds — this test // nothing with the original but the storage location and the secret key.
// could not run, because the recovery host can know neither. // If restore or verify needed the original local index — or
// the human snapshot ID that only that index holds — this test could not
// run, because the recovery host can know neither.
func TestRestoreOnAnotherMachine(t *testing.T) { func TestRestoreOnAnotherMachine(t *testing.T) {
log.Initialize(log.Config{}) log.Initialize(log.Config{})
t.Parallel() t.Parallel()
@@ -58,7 +61,12 @@ func TestRestoreOnAnotherMachine(t *testing.T) {
// Recovery host: a fresh empty index, a different hostname, and no // Recovery host: a fresh empty index, a different hostname, and no
// age_recipients — only the secret key and the same storage location. // age_recipients — only the secret key and the same storage location.
recovery, stdout := newRecoveryHost(ctx, t, fs, storer) configPath := filepath.Join(tempDir, "config.yml")
runVaultikCommand(t, "--config", configPath, "config", "init")
runVaultikCommand(t, "--config", configPath,
"config", "set", "storage_url", "file://"+storeDir)
recovery, stdout := newRecoveryHost(ctx, t, fs, storer, configPath)
// The recovery index really is empty. This is the assertion that makes // The recovery index really is empty. This is the assertion that makes
// the test a guard against restore quietly depending on the original // the test a guard against restore quietly depending on the original
@@ -93,6 +101,10 @@ func TestRestoreOnAnotherMachine(t *testing.T) {
remote.RemoteKey, &vaultik.VerifyOptions{Deep: true})) remote.RemoteKey, &vaultik.VerifyOptions{Deep: true}))
assertRestoredTreeMatches(t, fs, restoreDir, sourceFiles) assertRestoredTreeMatches(t, fs, restoreDir, sourceFiles)
// With no public key configured, a backup must refuse to start.
err = recovery.CreateSnapshot(&vaultik.SnapshotCreateOptions{Cron: true})
require.ErrorContains(t, err, "age_recipients")
} }
// writeRecoverySourceTree writes a small source tree spanning several // writeRecoverySourceTree writes a small source tree spanning several
@@ -118,14 +130,24 @@ func writeRecoverySourceTree(
} }
// newRecoveryHost builds the Vaultik a replacement machine would run: an // newRecoveryHost builds the Vaultik a replacement machine would run: an
// empty in-memory index, a hostname different from the backup host, no // empty in-memory index, a hostname different from the backup host, and
// age_recipients, and only the secret key plus the shared storer. It // only the secret key plus the shared storer. Its config is read from
// returns the instance and the buffer its stdout is wired to. // configPath by config.Load, as every command reads it. It returns the
// instance and the buffer its stdout is wired to.
func newRecoveryHost( func newRecoveryHost(
ctx context.Context, t *testing.T, fs afero.Fs, storer storage.Storer, ctx context.Context, t *testing.T, fs afero.Fs, storer storage.Storer,
configPath string,
) (*vaultik.Vaultik, *bytes.Buffer) { ) (*vaultik.Vaultik, *bytes.Buffer) {
t.Helper() t.Helper()
cfg, err := config.Load(configPath)
require.NoError(t, err)
// Set directly rather than through VAULTIK_AGE_SECRET_KEY, which a
// parallel test cannot change.
cfg.AgeSecretKey = testAgeSecretKey
cfg.Hostname = "recovery-host"
recoveryDB, err := database.New(ctx, ":memory:") recoveryDB, err := database.New(ctx, ":memory:")
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { _ = recoveryDB.Close() }) t.Cleanup(func() { _ = recoveryDB.Close() })
@@ -133,10 +155,7 @@ func newRecoveryHost(
stdout := &bytes.Buffer{} stdout := &bytes.Buffer{}
recovery := &vaultik.Vaultik{ recovery := &vaultik.Vaultik{
Config: &config.Config{ Config: cfg,
AgeSecretKey: testAgeSecretKey,
Hostname: "recovery-host",
},
Storage: storer, Storage: storer,
Fs: fs, Fs: fs,
Repositories: database.NewRepositories(recoveryDB), Repositories: database.NewRepositories(recoveryDB),
@@ -150,6 +169,20 @@ func newRecoveryHost(
return recovery, stdout return recovery, stdout
} }
// runVaultikCommand runs one vaultik command line in-process and fails the
// test if it returns an error. The command writes the cli package's global
// flag variables, so it must not be called from two tests that run at the
// same time.
func runVaultikCommand(t *testing.T, args ...string) {
t.Helper()
cmd := cli.NewRootCommand()
cmd.SetArgs(args)
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
require.NoError(t, cmd.Execute())
}
// assertRestoredTreeMatches byte-compares every restored file against its // assertRestoredTreeMatches byte-compares every restored file against its
// source content. // source content.
func assertRestoredTreeMatches( func assertRestoredTreeMatches(
+349
View File
@@ -0,0 +1,349 @@
package vaultik //nolint:testpackage // drives unexported restore internals
import (
"context"
"os"
"path/filepath"
"syscall"
"testing"
"time"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/types"
)
// These tests check that restore applies each entry's owner, mode and
// mtime in an order that keeps them.
const (
readOnlyDirMode = uint32(os.ModeDir | 0o555)
unsearchableMode = uint32(os.ModeDir | 0o600)
plainDirMode = uint32(os.ModeDir | 0o755)
plainFileMode = uint32(0o644)
setuidFileMode = uint32(os.ModeSetuid | 0o755)
otherOwnerID = uint32(4321)
writableTestMode = 0o755
symlinkTargetPath = "/nonexistent/target"
memTargetDir = "/restore"
// Owner bits a normal user needs on a directory to create an entry
// in it, and to change an entry in it.
ownerWriteAndSearch = os.FileMode(0o300)
ownerSearch = os.FileMode(0o100)
)
// normalUserFs refuses what the kernel refuses a normal user. make test
// runs as root, which a read-only or unsearchable directory does not
// stop, so without it the tests below could not fail there. Creating an
// entry needs owner write and search on the directory holding it;
// changing an entry's mode or times needs owner search. Only that one
// directory is checked, not every ancestor.
type normalUserFs struct {
afero.Fs
}
//nolint:ireturn // afero.Fs.OpenFile is defined to return the interface
func (fs normalUserFs) OpenFile(
name string, flag int, perm os.FileMode,
) (afero.File, error) {
if flag&os.O_CREATE != 0 {
err := fs.checkParent(name, ownerWriteAndSearch)
if err != nil {
return nil, err
}
}
return fs.Fs.OpenFile(name, flag, perm)
}
func (fs normalUserFs) MkdirAll(path string, perm os.FileMode) error {
_, err := fs.Stat(path)
if err != nil {
err = fs.checkParent(path, ownerWriteAndSearch)
if err != nil {
return err
}
}
return fs.Fs.MkdirAll(path, perm)
}
func (fs normalUserFs) Chmod(name string, mode os.FileMode) error {
err := fs.checkParent(name, ownerSearch)
if err != nil {
return err
}
return fs.Fs.Chmod(name, mode)
}
func (fs normalUserFs) Chtimes(name string, atime, mtime time.Time) error {
err := fs.checkParent(name, ownerSearch)
if err != nil {
return err
}
return fs.Fs.Chtimes(name, atime, mtime)
}
// checkParent returns a permission error when the directory holding name
// exists and its owner bits lack any of need.
func (fs normalUserFs) checkParent(name string, need os.FileMode) error {
info, err := fs.Stat(filepath.Dir(name))
if err == nil && info.Mode().Perm()&need != need {
return &os.PathError{Op: "access", Path: name, Err: os.ErrPermission}
}
return nil
}
// TestRestoreFillsReadOnlyDirectory checks that a read-only directory
// still receives the entries inside it, and ends with its stored mode.
func TestRestoreFillsReadOnlyDirectory(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
rows, repos := makeFiles(ctx, t, []*database.File{
{Path: "/ro", Mode: readOnlyDirMode},
{Path: "/ro/file", Mode: plainFileMode},
{Path: "/ro/sub", Mode: readOnlyDirMode},
{Path: "/ro/sub/file", Mode: plainFileMode},
})
fs := afero.NewMemMapFs()
v := newContainmentVaultik(ctx, normalUserFs{Fs: fs})
_, err := v.restoreAllFiles(rows, repos,
&RestoreOptions{TargetDir: memTargetDir}, nil, nil)
require.NoError(t, err)
for _, path := range []string{"/ro/file", "/ro/sub/file"} {
_, err := fs.Stat(filepath.Join(memTargetDir, path))
require.NoErrorf(t, err, "file inside a read-only directory: %s", path)
}
for _, dir := range []string{"/ro", "/ro/sub"} {
info, err := fs.Stat(filepath.Join(memTargetDir, dir))
require.NoError(t, err)
assert.Equalf(t, os.FileMode(0o555), info.Mode().Perm(), "mode of %s", dir)
}
}
// TestRestoreKeepsNonEmptyDirectoryMTime checks that a directory keeps
// its stored mtime although entries were written into it. It runs on the
// real filesystem, where writing an entry changes its directory's mtime.
func TestRestoreKeepsNonEmptyDirectoryMTime(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
targetDir := t.TempDir()
mtime := time.Date(2001, time.February, 3, 4, 5, 6, 0, time.UTC)
rows, repos := makeFiles(ctx, t, []*database.File{
{Path: "/dir", Mode: plainDirMode, MTime: mtime},
{Path: "/dir/file", Mode: plainFileMode, MTime: mtime},
})
v := newContainmentVaultik(ctx, afero.NewOsFs())
_, err := v.restoreAllFiles(rows, repos,
&RestoreOptions{TargetDir: targetDir}, nil, nil)
require.NoError(t, err)
info, err := os.Stat(filepath.Join(targetDir, "dir"))
require.NoError(t, err)
assert.Truef(t, info.ModTime().Equal(mtime),
"directory mtime is %s, stored %s", info.ModTime(), mtime)
}
// TestRestoreFinishesChildBeforeUnsearchableParent checks that a
// directory inside one whose stored mode denies search still gets its
// own stored mode and mtime.
func TestRestoreFinishesChildBeforeUnsearchableParent(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
mtime := time.Date(2001, time.February, 3, 4, 5, 6, 0, time.UTC)
rows, repos := makeFiles(ctx, t, []*database.File{
{Path: "/locked", Mode: unsearchableMode, MTime: mtime},
{Path: "/locked/sub", Mode: plainDirMode, MTime: mtime},
})
fs := afero.NewMemMapFs()
v := newContainmentVaultik(ctx, normalUserFs{Fs: fs})
_, err := v.restoreAllFiles(rows, repos,
&RestoreOptions{TargetDir: memTargetDir}, nil, nil)
require.NoError(t, err)
info, err := fs.Stat(filepath.Join(memTargetDir, "locked"))
require.NoError(t, err)
assert.Equal(t, os.FileMode(0o600), info.Mode().Perm())
info, err = fs.Stat(filepath.Join(memTargetDir, "locked", "sub"))
require.NoError(t, err)
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm())
assert.Truef(t, info.ModTime().Equal(mtime),
"mtime of sub is %s, stored %s", info.ModTime(), mtime)
}
// TestRestoreLeavesSymlinkedDirectoryTargetAlone checks that a directory
// whose place a later entry takes with a symlink does not hand its stored
// owner, mode and mtime to whatever the symlink points at. "/d" and "/d/"
// are different stored paths for the same place on disk.
func TestRestoreLeavesSymlinkedDirectoryTargetAlone(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
tempDir := t.TempDir()
targetDir := filepath.Join(tempDir, "target")
outsideDir := filepath.Join(tempDir, "outside")
require.NoError(t, os.Mkdir(outsideDir, writableTestMode))
before, err := os.Stat(outsideDir)
require.NoError(t, err)
mtime := time.Date(2001, time.February, 3, 4, 5, 6, 0, time.UTC)
rows, repos := makeFiles(ctx, t, []*database.File{
{
Path: "/d",
Mode: readOnlyDirMode,
UID: otherOwnerID,
GID: otherOwnerID,
MTime: mtime,
},
{Path: "/d/", LinkTarget: types.FilePath(outsideDir), MTime: mtime},
})
v := newContainmentVaultik(ctx, afero.NewOsFs())
_, err = v.restoreAllFiles(rows, repos,
&RestoreOptions{TargetDir: targetDir}, nil, nil)
require.NoError(t, err)
after, err := os.Stat(outsideDir)
require.NoError(t, err)
assert.Equal(t, before.Mode(), after.Mode())
assert.Truef(t, after.ModTime().Equal(before.ModTime()),
"mtime changed from %s to %s", before.ModTime(), after.ModTime())
beforeOwner, ok := before.Sys().(*syscall.Stat_t)
require.True(t, ok)
afterOwner, ok := after.Sys().(*syscall.Stat_t)
require.True(t, ok)
assert.Equal(t, beforeOwner.Uid, afterOwner.Uid)
assert.Equal(t, beforeOwner.Gid, afterOwner.Gid)
}
// TestRestoreKeepsSetuidThroughChown checks that a setuid file keeps the
// bit when restore changes its owner. Linux clears setuid on any chown of
// a regular file, even one to its current owner, so the file is recorded
// with the current user as owner and the session is told it runs as
// root: the chown then needs no privilege.
func TestRestoreKeepsSetuidThroughChown(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
targetDir := t.TempDir()
info, err := os.Stat(targetDir)
require.NoError(t, err)
owner, ok := info.Sys().(*syscall.Stat_t)
require.True(t, ok)
rows, repos := makeFiles(ctx, t, []*database.File{{
Path: "/suid",
Mode: setuidFileMode,
UID: owner.Uid,
GID: owner.Gid,
MTime: time.Date(2001, time.February, 3, 4, 5, 6, 0, time.UTC),
}})
session := &restoreSession{
v: newContainmentVaultik(ctx, afero.NewOsFs()),
ctx: ctx,
repos: repos,
opts: &RestoreOptions{TargetDir: targetDir},
result: &RestoreResult{},
runningAsRoot: true,
}
require.NoError(t, session.restoreFile(rows[0]))
info, err = os.Stat(filepath.Join(targetDir, "suid"))
require.NoError(t, err)
assert.Equal(t, os.FileMode(setuidFileMode),
info.Mode()&(os.ModeSetuid|os.ModePerm))
}
// TestRestoreSetsSymlinkMTime checks that a restored symlink gets its
// stored mtime on the link itself. The link dangles, so a call that
// follows it would fail.
func TestRestoreSetsSymlinkMTime(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
targetDir := t.TempDir()
mtime := time.Date(2001, time.February, 3, 4, 5, 6, 0, time.UTC)
rows, repos := makeFiles(ctx, t, []*database.File{{
Path: "/link",
LinkTarget: types.FilePath(symlinkTargetPath),
MTime: mtime,
}})
v := newContainmentVaultik(ctx, afero.NewOsFs())
_, err := v.restoreAllFiles(rows, repos,
&RestoreOptions{TargetDir: targetDir}, nil, nil)
require.NoError(t, err)
info, err := os.Lstat(filepath.Join(targetDir, "link"))
require.NoError(t, err)
assert.Truef(t, info.ModTime().Equal(mtime),
"symlink mtime is %s, stored %s", info.ModTime(), mtime)
}
// TestRestoreSetsSymlinkOwnerAsRoot checks that a symlink restored as
// root gets its stored owner on the link itself.
func TestRestoreSetsSymlinkOwnerAsRoot(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
if os.Geteuid() != 0 {
t.Skip("giving a file to another user needs root")
}
ctx := context.Background()
targetDir := t.TempDir()
rows, repos := makeFiles(ctx, t, []*database.File{{
Path: "/link",
LinkTarget: types.FilePath(symlinkTargetPath),
UID: otherOwnerID,
GID: otherOwnerID,
MTime: time.Date(2001, time.February, 3, 4, 5, 6, 0, time.UTC),
}})
v := newContainmentVaultik(ctx, afero.NewOsFs())
_, err := v.restoreAllFiles(rows, repos,
&RestoreOptions{TargetDir: targetDir}, nil, nil)
require.NoError(t, err)
info, err := os.Lstat(filepath.Join(targetDir, "link"))
require.NoError(t, err)
owner, ok := info.Sys().(*syscall.Stat_t)
require.True(t, ok)
assert.Equal(t, otherOwnerID, owner.Uid)
assert.Equal(t, otherOwnerID, owner.Gid)
}
+9
View File
@@ -23,6 +23,9 @@ var (
errSnapshotVerifyFailed = errors.New("verification failed") errSnapshotVerifyFailed = errors.New("verification failed")
errRemoveAllNeedsForce = errors.New("--all requires --force") errRemoveAllNeedsForce = errors.New("--all requires --force")
errInvalidTableName = errors.New("invalid table name") errInvalidTableName = errors.New("invalid table name")
errNoAgeRecipients = errors.New(
"creating a snapshot needs at least one public key in " +
"age_recipients (generate a keypair with: age-keygen)")
) )
// listRecentLimit caps how many snapshot rows are fetched from the // listRecentLimit caps how many snapshot rows are fetched from the
@@ -42,6 +45,12 @@ type SnapshotCreateOptions struct {
// CreateSnapshot executes the snapshot creation operation // CreateSnapshot executes the snapshot creation operation
func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error { func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
// config.Load accepts an empty list, since listing, verifying and
// restoring need no public key.
if len(v.Config.AgeRecipients) == 0 {
return errNoAgeRecipients
}
overallStartTime := time.Now() overallStartTime := time.Now()
log.Info("Starting snapshot creation", log.Info("Starting snapshot creation",