sneak/vaultik
1
0
Fork 0
Code Issues 19 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Update golangci-lint to v2.12.2 with canonical config (#62)
All checks were successful
check / check (push) Successful in 5s
Details

Browse Source 
Updates golangci-lint to v2.12.2 everywhere it is pinned and installs the canonical `.golangci.yml`, then remediates every finding the new linter/config surfaces so `make check` is green.

## Version bump

- `Dockerfile` lint stage: `golangci/golangci-lint:v2.11.3-alpine` -> `v2.12.2-alpine` (digest-pinned, date comment updated)
- `Makefile` `deps` target: `go install` moved from the old v1 module path at `@latest` to the pinned `github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2`
- `.golangci.yml` replaced with the canonical config (v2 schema; settings under `linters.settings` so the thresholds actually apply; `default: all` with the standard six disables)
- `script/bootstrap` installs golangci-lint via the system package manager and carries no version pin, so it is unchanged
- CI (`.gitea/workflows/check.yml`) only runs `script/cibuild`, so it needed no change

## Lint remediation

The canonical config surfaced ~3,300 findings across 56k lines. All are fixed, behavior-preserving; incorporates and supersedes the per-package mechanical passes already merged to `main` (refs #61). Highlights:

- `err113`: dynamic errors replaced with package sentinels + `%w` wrapping; comparisons via `errors.Is`
- `goprintffuncname`: printf-style helpers renamed with an `f` suffix (`ui.Writer` message methods, `cli.ReportErrorf`, `database.Fatalf`) and all call sites updated
- `revive` stutter renames: `blob.Handler`, `blob.WithReader`, `blob.ChunkPosition`, `storage.URL`, `storage.Info`; missing doc comments added
- `contextcheck`/`noctx`: `context.Context` threaded through `blob.Packer` and the scanner call sites; context-aware `exec`/`sql` variants
- `funlen`/`cyclop`/`gocognit`/`dupl`: oversized and duplicated functions split into focused helpers (production and test code)
- tests: `t.Parallel()` added where safe (global logger init kept in the serial phase for `-race`), `t.TempDir()`/`t.Helper()` adopted, several suites converted to external test packages
- `gosec`: bounded integer conversions, `ReadHeaderTimeout` on the test HTTP server; remaining warnings suppressed per-site with justifications
- remaining `nolint` directives are rare, targeted, and each carries a reason (e.g. `nilnil` not-found contract in the repository layer, fx module globals, on-disk snake_case struct tags)
- removed the deprecated `log.LogOptions` alias (callers migrated to `log.Options`)

`make check` (tests with `-race`, lint, fmt-check) passes.

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #62
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
This commit was merged in pull request #62.
This commit is contained in:
clawbot
2026-08-07 23:22:48 +02:00
committed by Jeffrey Paul
parent b87b72d4b9
commit cc58583130
126 changed files with 8184 additions and 5470 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
internal/vaultik/blob_fetch.go
View File

@@ -14,6 +14,10 @@ import (
"sneak.berlin/go/vaultik/internal/log"
)
// errBlobHashMismatch is returned when a fetched blob's content hash does
// not match the expected double-SHA-256 hash.
var errBlobHashMismatch = errors.New("blob hash mismatch")
// hashVerifyReader wraps a blobgen.Reader and verifies the double-SHA-256 hash
// of decrypted plaintext when Close is called. It reuses the hash that
// blobgen.Reader already computes internally via its TeeReader, avoiding
@@ -46,7 +50,8 @@ func (h *hashVerifyReader) Close() error {
actualHashHex := hex.EncodeToString(secondHasher.Sum(nil))
if actualHashHex != h.blobHash {
return fmt.Errorf("blob hash mismatch: expected %s, got %s", h.blobHash[:16], actualHashHex[:16])
return fmt.Errorf("%w: expected %s, got %s",
errBlobHashMismatch, h.blobHash[:16], actualHashHex[:16])
}
}
@@ -61,7 +66,9 @@ func (h *hashVerifyReader) Close() error {
// returns a streaming reader that computes the double-SHA-256 hash on the fly.
// The hash is verified when the returned reader is closed (after fully reading).
// This avoids buffering the entire blob in memory.
func (v *Vaultik) FetchAndDecryptBlob(ctx context.Context, blobHash string, expectedSize int64, identity age.Identity) (io.ReadCloser, error) {
func (v *Vaultik) FetchAndDecryptBlob(
ctx context.Context, blobHash string, expectedSize int64, identity age.Identity,
) (io.ReadCloser, error) {
rc, _, err := v.FetchBlob(ctx, blobHash, expectedSize)
if err != nil {
return nil, err
@@ -85,7 +92,9 @@ func (v *Vaultik) FetchAndDecryptBlob(ctx context.Context, blobHash string, expe
// Times the Storage.Get and Storage.Stat round-trips separately at
// debug level so we can see whether the size-only Stat (which is an
// extra request on every fetch) is hurting throughput.
func (v *Vaultik) FetchBlob(ctx context.Context, blobHash string, expectedSize int64) (io.ReadCloser, int64, error) {
func (v *Vaultik) FetchBlob(
ctx context.Context, blobHash string, expectedSize int64,
) (io.ReadCloser, int64, error) {
blobPath := fmt.Sprintf("blobs/%s/%s/%s", blobHash[:2], blobHash[2:4], blobHash)
t0 := time.Now()

56
internal/vaultik/blob_fetch_hash_test.go
View File

@@ -14,20 +14,17 @@ import (
"sneak.berlin/go/vaultik/internal/vaultik"
)
// TestFetchAndDecryptBlobVerifiesHash verifies that FetchAndDecryptBlob checks
// the double-SHA-256 hash of the decrypted plaintext against the expected blob hash.
func TestFetchAndDecryptBlobVerifiesHash(t *testing.T) {
identity, err := age.GenerateX25519Identity()
if err != nil {
t.Fatalf("generating identity: %v", err)
}
// Create test data and encrypt it using blobgen.Writer
plaintext := []byte("hello world test data for blob hash verification")
// buildHashTestBlob encrypts plaintext with blobgen.Writer and returns
// the encrypted bytes plus the expected double-SHA-256 hash.
func buildHashTestBlob(
t *testing.T, identity *age.X25519Identity, plaintext []byte,
) ([]byte, string) {
t.Helper()
var encBuf bytes.Buffer
writer, err := blobgen.NewWriter(&encBuf, 1, []string{identity.Recipient().String()})
writer, err := blobgen.NewWriter(&encBuf, 1,
[]string{identity.Recipient().String()})
if err != nil {
t.Fatalf("creating blobgen writer: %v", err)
}
@@ -42,9 +39,8 @@ func TestFetchAndDecryptBlobVerifiesHash(t *testing.T) {
t.Fatalf("closing writer: %v", err)
}
encryptedData := encBuf.Bytes()
// Compute correct double-SHA-256 hash of the plaintext (matches blobgen.Writer.Sum256)
// Compute the double-SHA-256 hash of the plaintext (matches
// blobgen.Writer.Sum256).
firstHash := sha256.Sum256(plaintext)
secondHash := sha256.Sum256(firstHash[:])
correctHash := hex.EncodeToString(secondHash[:])
@@ -52,12 +48,30 @@ func TestFetchAndDecryptBlobVerifiesHash(t *testing.T) {
// Verify our hash matches what blobgen.Writer produces
writerHash := hex.EncodeToString(writer.Sum256())
if correctHash != writerHash {
t.Fatalf("hash computation mismatch: manual=%s, writer=%s", correctHash, writerHash)
t.Fatalf("hash computation mismatch: manual=%s, writer=%s",
correctHash, writerHash)
}
return encBuf.Bytes(), correctHash
}
// TestFetchAndDecryptBlobVerifiesHash verifies that FetchAndDecryptBlob checks
// the double-SHA-256 hash of the decrypted plaintext against the expected blob hash.
func TestFetchAndDecryptBlobVerifiesHash(t *testing.T) {
t.Parallel()
identity, err := age.GenerateX25519Identity()
if err != nil {
t.Fatalf("generating identity: %v", err)
}
plaintext := []byte("hello world test data for blob hash verification")
encryptedData, correctHash := buildHashTestBlob(t, identity, plaintext)
// Set up mock storage with the blob at the correct path
mockStorage := NewMockStorer()
blobPath := "blobs/" + correctHash[:2] + "/" + correctHash[2:4] + "/" + correctHash
blobPath := "blobs/" + correctHash[:2] + "/" +
correctHash[2:4] + "/" + correctHash
mockStorage.mu.Lock()
mockStorage.data[blobPath] = encryptedData
@@ -67,7 +81,10 @@ func TestFetchAndDecryptBlobVerifiesHash(t *testing.T) {
ctx := context.Background()
t.Run("correct hash succeeds", func(t *testing.T) {
rc, err := tv.FetchAndDecryptBlob(ctx, correctHash, int64(len(encryptedData)), identity)
t.Parallel()
rc, err := tv.FetchAndDecryptBlob(
ctx, correctHash, int64(len(encryptedData)), identity)
if err != nil {
t.Fatalf("expected success, got error: %v", err)
}
@@ -88,6 +105,8 @@ func TestFetchAndDecryptBlobVerifiesHash(t *testing.T) {
})
t.Run("wrong hash fails", func(t *testing.T) {
t.Parallel()
// Use a fake hash that doesn't match the actual plaintext
fakeHash := strings.Repeat("ab", 32) // 64 hex chars
fakePath := "blobs/" + fakeHash[:2] + "/" + fakeHash[2:4] + "/" + fakeHash
@@ -96,7 +115,8 @@ func TestFetchAndDecryptBlobVerifiesHash(t *testing.T) {
mockStorage.data[fakePath] = encryptedData
mockStorage.mu.Unlock()
rc, err := tv.FetchAndDecryptBlob(ctx, fakeHash, int64(len(encryptedData)), identity)
rc, err := tv.FetchAndDecryptBlob(
ctx, fakeHash, int64(len(encryptedData)), identity)
if err != nil {
t.Fatalf("unexpected error opening stream: %v", err)
}

117
internal/vaultik/blobcache.go
View File

@@ -1,6 +1,7 @@
package vaultik
import (
"errors"
"fmt"
"io"
"os"
@@ -8,6 +9,15 @@ import (
"sync"
)
// Sentinel errors for blob cache lookups.
var (
errCacheKeyMissing = errors.New("key not in cache")
errCacheReadBeyondBlob = errors.New("read beyond blob size")
)
// blobCacheFileMode is the permission mode for cached blob files.
const blobCacheFileMode = 0o600
// blobDiskCacheEntry tracks a cached blob on disk.
type blobDiskCacheEntry struct {
key string
@@ -61,54 +71,8 @@ func newBlobDiskCache(maxBytes int64) (*blobDiskCache, error) {
}, nil
}
func (c *blobDiskCache) path(key string) string {
return filepath.Join(c.dir, key)
}
func (c *blobDiskCache) unlink(e *blobDiskCacheEntry) {
if e.prev != nil {
e.prev.next = e.next
} else {
c.head = e.next
}
if e.next != nil {
e.next.prev = e.prev
} else {
c.tail = e.prev
}
e.prev = nil
e.next = nil
}
func (c *blobDiskCache) pushFront(e *blobDiskCacheEntry) {
e.prev = nil
e.next = c.head
if c.head != nil {
c.head.prev = e
}
c.head = e
if c.tail == nil {
c.tail = e
}
}
func (c *blobDiskCache) evictLRU() {
if c.tail == nil {
return
}
victim := c.tail
c.unlink(victim)
delete(c.items, victim.key)
c.curBytes -= victim.size
_ = os.Remove(c.path(victim.key))
}
// Put writes blob data to disk cache. Entries larger than maxBytes are silently skipped.
// Put writes blob data to disk cache. Entries larger than maxBytes are
// silently skipped.
func (c *blobDiskCache) Put(key string, data []byte) error {
entrySize := int64(len(data))
@@ -127,7 +91,7 @@ func (c *blobDiskCache) Put(key string, data []byte) error {
delete(c.items, key)
}
err := os.WriteFile(c.path(key), data, 0600)
err := os.WriteFile(c.path(key), data, blobCacheFileMode)
if err != nil {
return fmt.Errorf("writing blob to cache: %w", err)
}
@@ -166,7 +130,8 @@ func (c *blobDiskCache) PutFromReader(key string, r io.Reader) (int64, error) {
}
c.mu.Unlock()
f, err := os.OpenFile(c.path(key), os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600)
f, err := os.OpenFile(
c.path(key), os.O_CREATE|os.O_TRUNC|os.O_WRONLY, blobCacheFileMode)
if err != nil {
return 0, fmt.Errorf("creating cache file: %w", err)
}
@@ -255,13 +220,14 @@ func (c *blobDiskCache) ReadAt(key string, offset, length int64) ([]byte, error)
if !ok {
c.mu.Unlock()
return nil, fmt.Errorf("key %q not in cache", key)
return nil, fmt.Errorf("%w: %q", errCacheKeyMissing, key)
}
if offset+length > e.size {
c.mu.Unlock()
return nil, fmt.Errorf("read beyond blob size: offset=%d length=%d size=%d", offset, length, e.size)
return nil, fmt.Errorf("%w: offset=%d length=%d size=%d",
errCacheReadBeyondBlob, offset, length, e.size)
}
c.unlink(e)
@@ -379,3 +345,50 @@ func (c *blobDiskCache) Close() error {
return os.RemoveAll(c.dir)
}
func (c *blobDiskCache) path(key string) string {
return filepath.Join(c.dir, key)
}
func (c *blobDiskCache) unlink(e *blobDiskCacheEntry) {
if e.prev != nil {
e.prev.next = e.next
} else {
c.head = e.next
}
if e.next != nil {
e.next.prev = e.prev
} else {
c.tail = e.prev
}
e.prev = nil
e.next = nil
}
func (c *blobDiskCache) pushFront(e *blobDiskCacheEntry) {
e.prev = nil
e.next = c.head
if c.head != nil {
c.head.prev = e
}
c.head = e
if c.tail == nil {
c.tail = e
}
}
func (c *blobDiskCache) evictLRU() {
if c.tail == nil {
return
}
victim := c.tail
c.unlink(victim)
delete(c.items, victim.key)
c.curBytes -= victim.size
_ = os.Remove(c.path(victim.key))
}

18
internal/vaultik/blobcache_test.go
View File

@@ -1,4 +1,4 @@
package vaultik
package vaultik //nolint:testpackage // exercises unexported blobDiskCache
import (
"bytes"
@@ -8,6 +8,8 @@ import (
)
func TestBlobDiskCache_BasicGetPut(t *testing.T) {
t.Parallel()
cache, err := newBlobDiskCache(1 << 20)
if err != nil {
t.Fatal(err)
@@ -37,6 +39,8 @@ func TestBlobDiskCache_BasicGetPut(t *testing.T) {
}
func TestBlobDiskCache_EvictionUnderPressure(t *testing.T) {
t.Parallel()
maxBytes := int64(1000)
cache, err := newBlobDiskCache(maxBytes)
@@ -49,7 +53,7 @@ func TestBlobDiskCache_EvictionUnderPressure(t *testing.T) {
for i := range 5 {
data := make([]byte, 300)
err := cache.Put(fmt.Sprintf("key%d", i), data)
err = cache.Put(fmt.Sprintf("key%d", i), data)
if err != nil {
t.Fatal(err)
}
@@ -69,6 +73,8 @@ func TestBlobDiskCache_EvictionUnderPressure(t *testing.T) {
}
func TestBlobDiskCache_OversizedEntryRejected(t *testing.T) {
t.Parallel()
cache, err := newBlobDiskCache(100)
if err != nil {
t.Fatal(err)
@@ -88,6 +94,8 @@ func TestBlobDiskCache_OversizedEntryRejected(t *testing.T) {
}
func TestBlobDiskCache_UpdateInPlace(t *testing.T) {
t.Parallel()
cache, err := newBlobDiskCache(1 << 20)
if err != nil {
t.Fatal(err)
@@ -123,6 +131,8 @@ func TestBlobDiskCache_UpdateInPlace(t *testing.T) {
}
func TestBlobDiskCache_ReadAt(t *testing.T) {
t.Parallel()
cache, err := newBlobDiskCache(1 << 20)
if err != nil {
t.Fatal(err)
@@ -162,6 +172,8 @@ func TestBlobDiskCache_ReadAt(t *testing.T) {
}
func TestBlobDiskCache_Close(t *testing.T) {
t.Parallel()
cache, err := newBlobDiskCache(1 << 20)
if err != nil {
t.Fatal(err)
@@ -179,6 +191,8 @@ func TestBlobDiskCache_Close(t *testing.T) {
}
func TestBlobDiskCache_LRUOrder(t *testing.T) {
t.Parallel()
cache, err := newBlobDiskCache(200)
if err != nil {
t.Fatal(err)

61
internal/vaultik/helpers.go
View File

@@ -8,13 +8,56 @@ import (
"strings"
"time"
"github.com/dustin/go-humanize"
"sneak.berlin/go/vaultik/internal/types"
)
// percentScale converts a 0..1 ratio into a percentage.
const percentScale = 100
// progressLogEvery is how many processed items pass between progress
// log lines in long-running loops.
const progressLogEvery = 100
// ubytes renders a byte count with humanize.Bytes, clamping negative
// values to zero so the int64→uint64 conversion cannot overflow.
func ubytes(n int64) string {
if n < 0 {
n = 0
}
return humanize.Bytes(uint64(n))
}
// Sentinel errors for snapshot ID and duration parsing.
var (
errMalformedSnapshotID = errors.New(
"invalid snapshot ID format: expected hostname_snapshotname_timestamp")
errInvalidDuration = errors.New("invalid duration")
errUnknownTimeUnit = errors.New("unknown time unit")
)
// Time-unit lengths used by parseDuration.
const (
day = 24 * time.Hour
week = 7 * day
month = 30 * day
year = 365 * day
)
// Snapshot IDs split on "_" into hostname, optional name parts, and a
// trailing timestamp.
const (
minSnapshotIDParts = 2
minSnapshotIDNameParts = 3
)
// SnapshotInfo contains information about a snapshot.
// UncompressedSize and NewChunkSize are populated only when the snapshot
// is present in the local database; LocallyTracked indicates whether
// those values are meaningful.
//
//nolint:tagliatelle // snake_case is the established output format
type SnapshotInfo struct {
ID types.SnapshotID `json:"id"`
Timestamp time.Time `json:"timestamp"`
@@ -44,8 +87,8 @@ func formatBytes(bytes int64) string {
// Format: hostname_snapshotname_2026-01-12T14:41:15Z
func parseSnapshotTimestamp(snapshotID string) (time.Time, error) {
parts := strings.Split(snapshotID, "_")
if len(parts) < 2 {
return time.Time{}, errors.New("invalid snapshot ID format: expected hostname_snapshotname_timestamp")
if len(parts) < minSnapshotIDParts {
return time.Time{}, errMalformedSnapshotID
}
// Last part is the RFC3339 timestamp
@@ -65,7 +108,7 @@ func parseSnapshotTimestamp(snapshotID string) (time.Time, error) {
// Returns the snapshot name, or empty string if the ID is malformed.
func parseSnapshotName(snapshotID string) string {
parts := strings.Split(snapshotID, "_")
if len(parts) < 3 {
if len(parts) < minSnapshotIDNameParts {
// Format: hostname_timestamp — no snapshot name
return ""
}
@@ -88,7 +131,7 @@ func parseDuration(s string) (time.Duration, error) {
matches := re.FindAllStringSubmatch(s, -1)
if len(matches) == 0 {
return 0, fmt.Errorf("invalid duration: %q", s)
return 0, fmt.Errorf("%w: %q", errInvalidDuration, s)
}
var total time.Duration
@@ -102,15 +145,15 @@ func parseDuration(s string) (time.Duration, error) {
unit := strings.ToLower(match[2])
switch unit {
case "d", "day", "days":
total += time.Duration(n) * 24 * time.Hour
total += time.Duration(n) * day
case "w", "week", "weeks":
total += time.Duration(n) * 7 * 24 * time.Hour
total += time.Duration(n) * week
case "mo", "month", "months":
total += time.Duration(n) * 30 * 24 * time.Hour
total += time.Duration(n) * month
case "y", "year", "years":
total += time.Duration(n) * 365 * 24 * time.Hour
total += time.Duration(n) * year
default:
return 0, fmt.Errorf("unknown time unit %q", unit)
return 0, fmt.Errorf("%w %q", errUnknownTimeUnit, unit)
}
}

20
internal/vaultik/helpers_test.go
View File

@@ -1,4 +1,4 @@
package vaultik
package vaultik //nolint:testpackage // exercises unexported parse helpers
import (
"testing"
@@ -6,6 +6,8 @@ import (
)
func TestParseSnapshotName(t *testing.T) {
t.Parallel()
tests := []struct {
name string
snapshotID string
@@ -30,15 +32,20 @@ func TestParseSnapshotName(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := parseSnapshotName(tt.snapshotID)
if got != tt.want {
t.Errorf("parseSnapshotName(%q) = %q, want %q", tt.snapshotID, got, tt.want)
t.Errorf("parseSnapshotName(%q) = %q, want %q",
tt.snapshotID, got, tt.want)
}
})
}
}
func TestParseDuration(t *testing.T) {
t.Parallel()
tests := []struct {
input string
want time.Duration
@@ -56,6 +63,8 @@ func TestParseDuration(t *testing.T) {
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
t.Parallel()
got, err := parseDuration(tt.input)
if tt.err {
if err == nil {
@@ -77,6 +86,8 @@ func TestParseDuration(t *testing.T) {
}
func TestParseSnapshotTimestamp(t *testing.T) {
t.Parallel()
tests := []struct {
name string
snapshotID string
@@ -106,9 +117,12 @@ func TestParseSnapshotTimestamp(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
_, err := parseSnapshotTimestamp(tt.snapshotID)
if (err != nil) != tt.wantErr {
t.Errorf("parseSnapshotTimestamp(%q) error = %v, wantErr %v", tt.snapshotID, err, tt.wantErr)
t.Errorf("parseSnapshotTimestamp(%q) error = %v, wantErr %v",
tt.snapshotID, err, tt.wantErr)
}
})
}

292
internal/vaultik/info.go
View File

@@ -15,129 +15,160 @@ import (
// ShowInfo displays system and configuration information
func (v *Vaultik) ShowInfo() error {
// System Information
v.printfStdout("=== System Information ===\n")
v.printfStdout("OS/Architecture: %s/%s\n", runtime.GOOS, runtime.GOARCH)
v.printfStdout("Version: %s\n", v.Globals.Version)
v.printfStdout("Commit: %s\n", v.Globals.Commit)
v.printfStdout("Go Version: %s\n", runtime.Version())
v.stdoutf("=== System Information ===\n")
v.stdoutf("OS/Architecture: %s/%s\n", runtime.GOOS, runtime.GOARCH)
v.stdoutf("Version: %s\n", v.Globals.Version)
v.stdoutf("Commit: %s\n", v.Globals.Commit)
v.stdoutf("Go Version: %s\n", runtime.Version())
v.printlnStdout()
// Storage Configuration. The backend is selected by storage_url
// (s3://, file://, rclone://); the legacy s3.* fields are only
// printed when they're actually populated, since the URL scheme
// is the primary configuration.
v.printfStdout("=== Storage Configuration ===\n")
v.showStorageConfig()
v.showBackupSettings()
// Encryption Configuration
v.stdoutf("=== Encryption Configuration ===\n")
v.stdoutf("Recipients:\n")
for _, recipient := range v.Config.AgeRecipients {
v.stdoutf(" - %s\n", recipient)
}
v.printlnStdout()
v.showLocalDatabase()
return nil
}
// showStorageConfig prints the storage configuration section. The
// backend is selected by storage_url (s3://, file://, rclone://); the
// legacy s3.* fields are only printed when they're actually populated,
// since the URL scheme is the primary configuration.
func (v *Vaultik) showStorageConfig() {
v.stdoutf("=== Storage Configuration ===\n")
storageInfo := v.Storage.Info()
v.printfStdout("Type: %s\n", storageInfo.Type)
v.printfStdout("Location: %s\n", storageInfo.Location)
v.stdoutf("Type: %s\n", storageInfo.Type)
v.stdoutf("Location: %s\n", storageInfo.Location)
if v.Config.StorageURL != "" {
v.printfStdout("Storage URL: %s\n", v.Config.StorageURL)
v.stdoutf("Storage URL: %s\n", v.Config.StorageURL)
}
if v.Config.S3.Bucket != "" {
v.printfStdout("S3 Bucket: %s\n", v.Config.S3.Bucket)
v.stdoutf("S3 Bucket: %s\n", v.Config.S3.Bucket)
}
if v.Config.S3.Prefix != "" {
v.printfStdout("S3 Prefix: %s\n", v.Config.S3.Prefix)
v.stdoutf("S3 Prefix: %s\n", v.Config.S3.Prefix)
}
if v.Config.S3.Endpoint != "" {
v.printfStdout("S3 Endpoint: %s\n", v.Config.S3.Endpoint)
v.stdoutf("S3 Endpoint: %s\n", v.Config.S3.Endpoint)
}
if v.Config.S3.Region != "" {
v.printfStdout("S3 Region: %s\n", v.Config.S3.Region)
v.stdoutf("S3 Region: %s\n", v.Config.S3.Region)
}
v.printlnStdout()
}
// Backup Settings
v.printfStdout("=== Backup Settings ===\n")
// showBackupSettings prints the configured snapshots, exclude patterns,
// and chunking/compression settings.
func (v *Vaultik) showBackupSettings() {
v.stdoutf("=== Backup Settings ===\n")
// Show configured snapshots
v.printfStdout("Snapshots:\n")
v.stdoutf("Snapshots:\n")
for _, name := range v.Config.SnapshotNames() {
snap := v.Config.Snapshots[name]
v.printfStdout(" %s:\n", name)
v.stdoutf(" %s:\n", name)
for _, path := range snap.Paths {
v.printfStdout(" - %s\n", path)
v.stdoutf(" - %s\n", path)
}
if len(snap.Exclude) > 0 {
v.printfStdout(" exclude: %s\n", strings.Join(snap.Exclude, ", "))
v.stdoutf(" exclude: %s\n", strings.Join(snap.Exclude, ", "))
}
}
// Global exclude patterns
if len(v.Config.Exclude) > 0 {
v.printfStdout("Global Exclude: %s\n", strings.Join(v.Config.Exclude, ", "))
v.stdoutf("Global Exclude: %s\n", strings.Join(v.Config.Exclude, ", "))
}
v.printfStdout("Compression: zstd level %d\n", v.Config.CompressionLevel)
v.printfStdout("Chunk Size: %s\n", humanize.Bytes(uint64(v.Config.ChunkSize)))
v.printfStdout("Blob Size Limit: %s\n", humanize.Bytes(uint64(v.Config.BlobSizeLimit)))
v.stdoutf("Compression: zstd level %d\n", v.Config.CompressionLevel)
v.stdoutf("Chunk Size: %s\n", ubytes(int64(v.Config.ChunkSize)))
v.stdoutf("Blob Size Limit: %s\n", ubytes(int64(v.Config.BlobSizeLimit)))
v.printlnStdout()
}
// Encryption Configuration
v.printfStdout("=== Encryption Configuration ===\n")
v.printfStdout("Recipients:\n")
for _, recipient := range v.Config.AgeRecipients {
v.printfStdout(" - %s\n", recipient)
}
v.printlnStdout()
// Local Database
v.printfStdout("=== Local Database ===\n")
v.printfStdout("Index Path: %s\n", v.Config.IndexPath)
// showLocalDatabase prints the local index database section, including
// record counts when the index exists.
func (v *Vaultik) showLocalDatabase() {
v.stdoutf("=== Local Database ===\n")
v.stdoutf("Index Path: %s\n", v.Config.IndexPath)
// Check if index file exists and get its size
info, err := v.Fs.Stat(v.Config.IndexPath)
if err == nil {
v.printfStdout("Index Size: %s\n", humanize.Bytes(uint64(info.Size())))
if err != nil {
v.stdoutf("Index Size: (not created)\n")
// Get snapshot count from database
query := `SELECT COUNT(*) FROM snapshots WHERE completed_at IS NOT NULL`
var snapshotCount int
err := v.DB.Conn().QueryRowContext(v.ctx, query).Scan(&snapshotCount)
if err == nil {
v.printfStdout("Snapshots: %d\n", snapshotCount)
}
// Get blob count from database
query = `SELECT COUNT(*) FROM blobs`
var blobCount int
err = v.DB.Conn().QueryRowContext(v.ctx, query).Scan(&blobCount)
if err == nil {
v.printfStdout("Blobs: %d\n", blobCount)
}
// Get file count from database
query = `SELECT COUNT(*) FROM files`
var fileCount int
err = v.DB.Conn().QueryRowContext(v.ctx, query).Scan(&fileCount)
if err == nil {
v.printfStdout("Files: %d\n", fileCount)
}
} else {
v.printfStdout("Index Size: (not created)\n")
return
}
return nil
v.stdoutf("Index Size: %s\n", ubytes(info.Size()))
// Get snapshot count from database
query := `SELECT COUNT(*) FROM snapshots WHERE completed_at IS NOT NULL`
var snapshotCount int
err = v.DB.Conn().QueryRowContext(v.ctx, query).Scan(&snapshotCount)
if err == nil {
v.stdoutf("Snapshots: %d\n", snapshotCount)
}
// Get blob count from database
query = `SELECT COUNT(*) FROM blobs`
var blobCount int
err = v.DB.Conn().QueryRowContext(v.ctx, query).Scan(&blobCount)
if err == nil {
v.stdoutf("Blobs: %d\n", blobCount)
}
// Get file count from database
query = `SELECT COUNT(*) FROM files`
var fileCount int
err = v.DB.Conn().QueryRowContext(v.ctx, query).Scan(&fileCount)
if err == nil {
v.stdoutf("Files: %d\n", fileCount)
}
}
// Table layout constants for the human-readable remote info output.
const (
// snapshotIDColWidth is the SNAPSHOT column width in the remote
// info table.
snapshotIDColWidth = 45
// metadataKeyParts is the minimum "/"-separated segment count of a
// metadata object key (metadata/<snapshot-id>/<filename>).
metadataKeyParts = 3
// blobKeyParts is the minimum "/"-separated segment count of a blob
// object key (blobs/<aa>/<bb>/<hash>).
blobKeyParts = 4
)
// SnapshotMetadataInfo contains information about a single snapshot's metadata
//
//nolint:tagliatelle // snake_case is the established JSON output format
type SnapshotMetadataInfo struct {
SnapshotID string `json:"snapshot_id"`
ManifestSize int64 `json:"manifest_size"`
@@ -148,6 +179,8 @@ type SnapshotMetadataInfo struct {
}
// RemoteInfoResult contains all remote storage information
//
//nolint:tagliatelle // snake_case is the established JSON output format
type RemoteInfoResult struct {
// Storage info
StorageType string `json:"storage_type"`
@@ -182,11 +215,11 @@ func (v *Vaultik) RemoteInfo(jsonOutput bool) error {
result.StorageLocation = storageInfo.Location
if !jsonOutput {
v.printfStdout("=== Remote Storage ===\n")
v.printfStdout("Type: %s\n", storageInfo.Type)
v.printfStdout("Location: %s\n", storageInfo.Location)
v.stdoutf("=== Remote Storage ===\n")
v.stdoutf("Type: %s\n", storageInfo.Type)
v.stdoutf("Location: %s\n", storageInfo.Location)
v.printlnStdout()
v.printfStdout("Scanning snapshot metadata...\n")
v.stdoutf("Scanning snapshot metadata...\n")
}
snapshotMetadata, snapshotIDs, err := v.collectSnapshotMetadata()
@@ -195,7 +228,7 @@ func (v *Vaultik) RemoteInfo(jsonOutput bool) error {
}
if !jsonOutput {
v.printfStdout("Downloading %d manifest(s)...\n", len(snapshotIDs))
v.stdoutf("Downloading %d manifest(s)...\n", len(snapshotIDs))
}
referencedBlobs := v.collectReferencedBlobsFromManifests(snapshotIDs, snapshotMetadata)
@@ -225,8 +258,11 @@ func (v *Vaultik) RemoteInfo(jsonOutput bool) error {
return nil
}
// collectSnapshotMetadata scans remote metadata and returns per-snapshot info and sorted IDs
func (v *Vaultik) collectSnapshotMetadata() (map[string]*SnapshotMetadataInfo, []string, error) {
// collectSnapshotMetadata scans remote metadata and returns
// per-snapshot info and sorted IDs.
func (v *Vaultik) collectSnapshotMetadata() (
map[string]*SnapshotMetadataInfo, []string, error,
) {
snapshotMetadata := make(map[string]*SnapshotMetadataInfo)
metadataCh := v.Storage.ListStream(v.ctx, "metadata/")
@@ -236,7 +272,7 @@ func (v *Vaultik) collectSnapshotMetadata() (map[string]*SnapshotMetadataInfo, [
}
parts := strings.Split(obj.Key, "/")
if len(parts) < 3 {
if len(parts) < metadataKeyParts {
continue
}
@@ -268,8 +304,11 @@ func (v *Vaultik) collectSnapshotMetadata() (map[string]*SnapshotMetadataInfo, [
return snapshotMetadata, snapshotIDs, nil
}
// collectReferencedBlobsFromManifests downloads manifests and returns referenced blob hashes with sizes
func (v *Vaultik) collectReferencedBlobsFromManifests(snapshotIDs []string, snapshotMetadata map[string]*SnapshotMetadataInfo) map[string]int64 {
// collectReferencedBlobsFromManifests downloads manifests and returns
// referenced blob hashes with sizes.
func (v *Vaultik) collectReferencedBlobsFromManifests(
snapshotIDs []string, snapshotMetadata map[string]*SnapshotMetadataInfo,
) map[string]int64 {
referencedBlobs := make(map[string]int64)
for _, snapshotID := range snapshotIDs {
@@ -307,8 +346,14 @@ func (v *Vaultik) collectReferencedBlobsFromManifests(snapshotIDs []string, snap
return referencedBlobs
}
// populateRemoteInfoResult fills in the result's snapshot and referenced blob stats
func (v *Vaultik) populateRemoteInfoResult(result *RemoteInfoResult, snapshotMetadata map[string]*SnapshotMetadataInfo, snapshotIDs []string, referencedBlobs map[string]int64) {
// populateRemoteInfoResult fills in the result's snapshot and
// referenced blob stats.
func (v *Vaultik) populateRemoteInfoResult(
result *RemoteInfoResult,
snapshotMetadata map[string]*SnapshotMetadataInfo,
snapshotIDs []string,
referencedBlobs map[string]int64,
) {
var totalMetadataSize int64
for _, id := range snapshotIDs {
@@ -327,9 +372,11 @@ func (v *Vaultik) populateRemoteInfoResult(result *RemoteInfoResult, snapshotMet
}
// scanRemoteBlobStorage lists all blobs on remote and computes orphan stats
func (v *Vaultik) scanRemoteBlobStorage(result *RemoteInfoResult, referencedBlobs map[string]int64, jsonOutput bool) error {
func (v *Vaultik) scanRemoteBlobStorage(
result *RemoteInfoResult, referencedBlobs map[string]int64, jsonOutput bool,
) error {
if !jsonOutput {
v.printfStdout("Scanning blobs...\n")
v.stdoutf("Scanning blobs...\n")
}
blobCh := v.Storage.ListStream(v.ctx, "blobs/")
@@ -341,7 +388,7 @@ func (v *Vaultik) scanRemoteBlobStorage(result *RemoteInfoResult, referencedBlob
}
parts := strings.Split(obj.Key, "/")
if len(parts) < 4 {
if len(parts) < blobKeyParts {
continue
}
@@ -363,51 +410,74 @@ func (v *Vaultik) scanRemoteBlobStorage(result *RemoteInfoResult, referencedBlob
// printRemoteInfoTable renders the human-readable remote info output
func (v *Vaultik) printRemoteInfoTable(result *RemoteInfoResult) {
v.printfStdout("\n=== Snapshot Metadata ===\n")
const (
rowFormat = "%-45s %12s %12s %12s %10s %12s\n"
sizeColWidth = 12
countColWidth = 10
)
v.stdoutf("\n=== Snapshot Metadata ===\n")
if len(result.Snapshots) == 0 {
v.printfStdout("No snapshots found\n")
v.stdoutf("No snapshots found\n")
} else {
v.printfStdout("%-45s %12s %12s %12s %10s %12s\n", "SNAPSHOT", "MANIFEST", "DATABASE", "TOTAL", "BLOBS", "BLOB SIZE")
v.printfStdout("%-45s %12s %12s %12s %10s %12s\n", strings.Repeat("-", 45), strings.Repeat("-", 12), strings.Repeat("-", 12), strings.Repeat("-", 12), strings.Repeat("-", 10), strings.Repeat("-", 12))
separator := fmt.Sprintf(rowFormat,
strings.Repeat("-", snapshotIDColWidth),
strings.Repeat("-", sizeColWidth),
strings.Repeat("-", sizeColWidth),
strings.Repeat("-", sizeColWidth),
strings.Repeat("-", countColWidth),
strings.Repeat("-", sizeColWidth))
v.stdoutf(rowFormat,
"SNAPSHOT", "MANIFEST", "DATABASE", "TOTAL", "BLOBS", "BLOB SIZE")
v.stdoutf("%s", separator)
for _, info := range result.Snapshots {
v.printfStdout("%-45s %12s %12s %12s %10s %12s\n",
truncateString(info.SnapshotID, 45),
humanize.Bytes(uint64(info.ManifestSize)),
humanize.Bytes(uint64(info.DatabaseSize)),
humanize.Bytes(uint64(info.TotalSize)),
v.stdoutf(rowFormat,
truncateString(info.SnapshotID, snapshotIDColWidth),
ubytes(info.ManifestSize),
ubytes(info.DatabaseSize),
ubytes(info.TotalSize),
humanize.Comma(int64(info.BlobCount)),
humanize.Bytes(uint64(info.BlobsSize)),
ubytes(info.BlobsSize),
)
}
v.printfStdout("%-45s %12s %12s %12s %10s %12s\n", strings.Repeat("-", 45), strings.Repeat("-", 12), strings.Repeat("-", 12), strings.Repeat("-", 12), strings.Repeat("-", 10), strings.Repeat("-", 12))
v.printfStdout("%-45s %12s %12s %12s\n", fmt.Sprintf("Total (%d snapshots)", result.TotalMetadataCount), "", "", humanize.Bytes(uint64(result.TotalMetadataSize)))
v.stdoutf("%s", separator)
v.stdoutf("%-45s %12s %12s %12s\n",
fmt.Sprintf("Total (%d snapshots)", result.TotalMetadataCount),
"", "", ubytes(result.TotalMetadataSize))
}
v.printfStdout("\n=== Blob Storage ===\n")
v.printfStdout("Total blobs on remote: %s (%s)\n",
humanize.Comma(int64(result.TotalBlobCount)), humanize.Bytes(uint64(result.TotalBlobSize)))
v.printfStdout("Referenced by snapshots: %s (%s)\n",
humanize.Comma(int64(result.ReferencedBlobCount)), humanize.Bytes(uint64(result.ReferencedBlobSize)))
v.printfStdout("Orphaned (unreferenced): %s (%s)\n",
humanize.Comma(int64(result.OrphanedBlobCount)), humanize.Bytes(uint64(result.OrphanedBlobSize)))
v.stdoutf("\n=== Blob Storage ===\n")
v.stdoutf("Total blobs on remote: %s (%s)\n",
humanize.Comma(int64(result.TotalBlobCount)),
ubytes(result.TotalBlobSize))
v.stdoutf("Referenced by snapshots: %s (%s)\n",
humanize.Comma(int64(result.ReferencedBlobCount)),
ubytes(result.ReferencedBlobSize))
v.stdoutf("Orphaned (unreferenced): %s (%s)\n",
humanize.Comma(int64(result.OrphanedBlobCount)),
ubytes(result.OrphanedBlobSize))
if result.OrphanedBlobCount > 0 {
v.printfStdout("\nRun 'vaultik prune' to remove orphaned blobs.\n")
v.stdoutf("\nRun 'vaultik prune' to remove orphaned blobs.\n")
}
}
// ellipsis is appended by truncateString when it shortens a string.
const ellipsis = "..."
// truncateString truncates a string to maxLen, adding "..." if truncated
func truncateString(s string, maxLen int) string {
if len(s) <= maxLen {
return s
}
if maxLen <= 3 {
if maxLen <= len(ellipsis) {
return s[:maxLen]
}
return s[:maxLen-3] + "..."
return s[:maxLen-len(ellipsis)] + ellipsis
}

693
internal/vaultik/integration_test.go
View File

@@ -24,6 +24,16 @@ import (
"sneak.berlin/go/vaultik/internal/vaultik"
)
// Shared fixture values used across the vaultik integration tests.
const (
testLabel = "test"
testHostname = "test-host"
testAgePublicKey = "age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05g" +
"l0sjq9q9wjg"
testAgeSecretKey = "AGE-SECRET-KEY-19CR5YSFW59HM4TLD6GXVEDMZFTVVF7PPHKU" +
"T68TXSFPK7APHXA2QS2NJA5"
)
// MockStorer implements storage.Storer for testing
type MockStorer struct {
mu sync.Mutex
@@ -38,7 +48,7 @@ func NewMockStorer() *MockStorer {
}
}
func (m *MockStorer) Put(ctx context.Context, key string, reader io.Reader) error {
func (m *MockStorer) Put(_ context.Context, key string, reader io.Reader) error {
m.mu.Lock()
defer m.mu.Unlock()
@@ -54,11 +64,14 @@ func (m *MockStorer) Put(ctx context.Context, key string, reader io.Reader) erro
return nil
}
func (m *MockStorer) PutWithProgress(ctx context.Context, key string, reader io.Reader, size int64, progress storage.ProgressCallback) error {
func (m *MockStorer) PutWithProgress(
ctx context.Context, key string, reader io.Reader,
_ int64, _ storage.ProgressCallback,
) error {
return m.Put(ctx, key, reader)
}
func (m *MockStorer) Get(ctx context.Context, key string) (io.ReadCloser, error) {
func (m *MockStorer) Get(_ context.Context, key string) (io.ReadCloser, error) {
m.mu.Lock()
defer m.mu.Unlock()
@@ -72,7 +85,7 @@ func (m *MockStorer) Get(ctx context.Context, key string) (io.ReadCloser, error)
return io.NopCloser(bytes.NewReader(data)), nil
}
func (m *MockStorer) Stat(ctx context.Context, key string) (*storage.ObjectInfo, error) {
func (m *MockStorer) Stat(_ context.Context, key string) (*storage.ObjectInfo, error) {
m.mu.Lock()
defer m.mu.Unlock()
@@ -89,7 +102,7 @@ func (m *MockStorer) Stat(ctx context.Context, key string) (*storage.ObjectInfo,
}, nil
}
func (m *MockStorer) Delete(ctx context.Context, key string) error {
func (m *MockStorer) Delete(_ context.Context, key string) error {
m.mu.Lock()
defer m.mu.Unlock()
@@ -99,7 +112,7 @@ func (m *MockStorer) Delete(ctx context.Context, key string) error {
return nil
}
func (m *MockStorer) List(ctx context.Context, prefix string) ([]string, error) {
func (m *MockStorer) List(_ context.Context, prefix string) ([]string, error) {
m.mu.Lock()
defer m.mu.Unlock()
@@ -108,7 +121,8 @@ func (m *MockStorer) List(ctx context.Context, prefix string) ([]string, error)
var keys []string
for key := range m.data {
if len(prefix) == 0 || (len(key) >= len(prefix) && key[:len(prefix)] == prefix) {
if len(prefix) == 0 ||
(len(key) >= len(prefix) && key[:len(prefix)] == prefix) {
keys = append(keys, key)
}
}
@@ -116,7 +130,9 @@ func (m *MockStorer) List(ctx context.Context, prefix string) ([]string, error)
return keys, nil
}
func (m *MockStorer) ListStream(ctx context.Context, prefix string) <-chan storage.ObjectInfo {
func (m *MockStorer) ListStream(
_ context.Context, prefix string,
) <-chan storage.ObjectInfo {
ch := make(chan storage.ObjectInfo)
go func() {
defer close(ch)
@@ -125,7 +141,8 @@ func (m *MockStorer) ListStream(ctx context.Context, prefix string) <-chan stora
defer m.mu.Unlock()
for key, data := range m.data {
if len(prefix) == 0 || (len(key) >= len(prefix) && key[:len(prefix)] == prefix) {
if len(prefix) == 0 ||
(len(key) >= len(prefix) && key[:len(prefix)] == prefix) {
ch <- storage.ObjectInfo{
Key: key,
Size: int64(len(data)),
@@ -137,8 +154,8 @@ func (m *MockStorer) ListStream(ctx context.Context, prefix string) <-chan stora
return ch
}
func (m *MockStorer) Info() storage.StorageInfo {
return storage.StorageInfo{
func (m *MockStorer) Info() storage.Info {
return storage.Info{
Type: "mock",
Location: "memory",
}
@@ -163,123 +180,55 @@ func (m *MockStorer) GetStorageSize() int {
return len(m.data)
}
// TestEndToEndBackup tests the full backup workflow with mocked dependencies
func TestEndToEndBackup(t *testing.T) {
// Initialize logger
log.Initialize(log.Config{})
// writeTestFileTree creates each file (and its parent directory) in fs.
func writeTestFileTree(t *testing.T, fs afero.Fs, files map[string]string) {
t.Helper()
// Create in-memory filesystem
fs := afero.NewMemMapFs()
for path, content := range files {
dir := filepath.Dir(path)
// Create test directory structure and files
testFiles := map[string]string{
"/home/user/documents/file1.txt": "This is file 1 content",
"/home/user/documents/file2.txt": "This is file 2 content with more data",
"/home/user/pictures/photo1.jpg": "Binary photo data here...",
"/home/user/code/main.go": "package main\n\nfunc main() {\n\tprintln(\"Hello, World!\")\n}",
}
// Create all directories first
dirs := []string{
"/home/user/documents",
"/home/user/pictures",
"/home/user/code",
}
for _, dir := range dirs {
err := fs.MkdirAll(dir, 0755)
if err != nil {
t.Fatalf("failed to create directory %s: %v", dir, err)
}
}
// Create test files
for path, content := range testFiles {
err := afero.WriteFile(fs, path, []byte(content), 0644)
err = afero.WriteFile(fs, path, []byte(content), 0644)
if err != nil {
t.Fatalf("failed to create test file %s: %v", path, err)
}
}
}
// Create mock storage
mockStorage := NewMockStorer()
// createTestSnapshotRecord inserts a snapshot row so scans have a
// snapshot to attach to.
func createTestSnapshotRecord(
t *testing.T, repos *database.Repositories, snapshotID string,
) {
t.Helper()
// Create test configuration
cfg := &config.Config{
Snapshots: map[string]config.SnapshotConfig{
"test": {
Paths: []string{"/home/user"},
},
},
Exclude: []string{"*.tmp", "*.log"},
ChunkSize: config.Size(16 * 1024), // 16KB chunks
BlobSizeLimit: config.Size(100 * 1024), // 100KB blobs
CompressionLevel: 3,
AgeRecipients: []string{"age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05gl0sjq9q9wjg"}, // Test public key
AgeSecretKey: "AGE-SECRET-KEY-19CR5YSFW59HM4TLD6GXVEDMZFTVVF7PPHKUT68TXSFPK7APHXA2QS2NJA5", // Test private key
S3: config.S3Config{
Endpoint: "http://localhost:9000", // MinIO endpoint for testing
Region: "us-east-1",
Bucket: "test-bucket",
AccessKeyID: "test-access",
SecretAccessKey: "test-secret",
},
IndexPath: ":memory:", // In-memory SQLite database
}
// For a true end-to-end test, we'll create a simpler test that focuses on
// the core backup logic using the scanner directly with our mock storage
ctx := context.Background()
// Create in-memory database
db, err := database.New(ctx, ":memory:")
require.NoError(t, err)
defer func() {
err := db.Close()
if err != nil {
t.Errorf("failed to close database: %v", err)
}
}()
repos := database.NewRepositories(db)
// Create scanner with mock storage
scanner := snapshot.NewScanner(snapshot.ScannerConfig{
FS: fs,
ChunkSize: cfg.ChunkSize.Int64(),
Repositories: repos,
Storage: mockStorage,
MaxBlobSize: cfg.BlobSizeLimit.Int64(),
CompressionLevel: cfg.CompressionLevel,
AgeRecipients: cfg.AgeRecipients,
EnableProgress: false,
})
// Create a snapshot record
snapshotID := "test-snapshot-001"
err = repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
snapshot := &database.Snapshot{
err := repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
snap := &database.Snapshot{
ID: types.SnapshotID(snapshotID),
Hostname: "test-host",
Hostname: testHostname,
VaultikVersion: "test-version",
StartedAt: time.Now(),
}
return repos.Snapshots.Create(ctx, tx, snapshot)
return repos.Snapshots.Create(ctx, tx, snap)
})
require.NoError(t, err)
}
// Run the backup scan
result, err := scanner.Scan(ctx, "/home/user", snapshotID)
require.NoError(t, err)
// Verify scan results
// The scanner counts both files and directories, so we have:
// 4 files + 4 directories (/home, /home/user, /home/user/documents, /home/user/pictures, /home/user/code)
assert.GreaterOrEqual(t, result.FilesScanned, 4, "Should scan at least 4 files")
assert.Positive(t, result.BytesScanned, "Should scan some bytes")
assert.Positive(t, result.ChunksCreated, "Should create chunks")
assert.Positive(t, result.BlobsCreated, "Should create blobs")
// TestEndToEndBackup tests the full backup workflow with mocked dependencies
// verifyEndToEndBackupState checks storage upload calls, database file
// rows, and chunk mappings after the end-to-end backup scan.
func verifyEndToEndBackupState(
ctx context.Context, t *testing.T,
repos *database.Repositories, mockStorage *MockStorer,
) {
t.Helper()
// Verify storage operations
calls := mockStorage.GetCalls()
@@ -313,9 +262,108 @@ func TestEndToEndBackup(t *testing.T) {
assert.Equal(t, 4, regularFiles, "Should have 4 regular files in database")
// Verify chunks were created by checking a specific file
fileChunks, err := repos.FileChunks.GetByPath(ctx, "/home/user/documents/file1.txt")
fileChunks, err := repos.FileChunks.GetByPath(ctx,
"/home/user/documents/file1.txt")
require.NoError(t, err)
assert.NotEmpty(t, fileChunks, "Should have chunks for file1.txt")
}
// newEndToEndTestConfig builds the standard config used by the
// end-to-end backup test.
func newEndToEndTestConfig() *config.Config {
return &config.Config{
Snapshots: map[string]config.SnapshotConfig{
testLabel: {
Paths: []string{"/home/user"},
},
},
Exclude: []string{"*.tmp", "*.log"},
ChunkSize: config.Size(16 * 1024), // 16KB chunks
BlobSizeLimit: config.Size(100 * 1024), // 100KB blobs
CompressionLevel: 3,
AgeRecipients: []string{testAgePublicKey},
AgeSecretKey: testAgeSecretKey,
S3: config.S3Config{
Endpoint: "http://localhost:9000", // MinIO endpoint for testing
Region: "us-east-1",
Bucket: "test-bucket",
AccessKeyID: "test-access",
SecretAccessKey: "test-secret",
},
IndexPath: ":memory:", // In-memory SQLite database
}
}
func TestEndToEndBackup(t *testing.T) {
// Initialize logger
log.Initialize(log.Config{})
t.Parallel()
// Create in-memory filesystem
fs := afero.NewMemMapFs()
// Create test directory structure and files
testFiles := map[string]string{
"/home/user/documents/file1.txt": "This is file 1 content",
"/home/user/documents/file2.txt": "This is file 2 content with more data",
"/home/user/pictures/photo1.jpg": "Binary photo data here...",
"/home/user/code/main.go": "package main\n\nfunc main() {\n" +
"\tprintln(\"Hello, World!\")\n}",
}
writeTestFileTree(t, fs, testFiles)
// Create mock storage
mockStorage := NewMockStorer()
cfg := newEndToEndTestConfig()
// For a true end-to-end test, we'll create a simpler test that focuses on
// the core backup logic using the scanner directly with our mock storage
ctx := context.Background()
// Create in-memory database
db, err := database.New(ctx, ":memory:")
require.NoError(t, err)
defer func() {
err := db.Close()
if err != nil {
t.Errorf("failed to close database: %v", err)
}
}()
repos := database.NewRepositories(db)
// Create scanner with mock storage
scanner := snapshot.NewScanner(snapshot.ScannerConfig{
FS: fs,
ChunkSize: cfg.ChunkSize.Int64(),
Repositories: repos,
Storage: mockStorage,
MaxBlobSize: cfg.BlobSizeLimit.Int64(),
CompressionLevel: cfg.CompressionLevel,
AgeRecipients: cfg.AgeRecipients,
EnableProgress: false,
})
// Create a snapshot record
snapshotID := "test-snapshot-001"
createTestSnapshotRecord(t, repos, snapshotID)
// Run the backup scan
result, err := scanner.Scan(ctx, "/home/user", snapshotID)
require.NoError(t, err)
// Verify scan results. The scanner counts both files and
// directories: 4 files + directories (/home, /home/user,
// /home/user/documents, /home/user/pictures, /home/user/code).
assert.GreaterOrEqual(t, result.FilesScanned, 4, "Should scan at least 4 files")
assert.Positive(t, result.BytesScanned, "Should scan some bytes")
assert.Positive(t, result.ChunksCreated, "Should create chunks")
assert.Positive(t, result.BlobsCreated, "Should create blobs")
verifyEndToEndBackupState(ctx, t, repos, mockStorage)
// Verify blobs were uploaded to storage
assert.Positive(t, mockStorage.GetStorageSize(), "Should have blobs in storage")
@@ -343,6 +391,7 @@ func TestEndToEndBackup(t *testing.T) {
func TestBackupAndVerify(t *testing.T) {
// Initialize logger
log.Initialize(log.Config{})
t.Parallel()
// Create in-memory filesystem
fs := afero.NewMemMapFs()
@@ -379,22 +428,12 @@ func TestBackupAndVerify(t *testing.T) {
Storage: mockStorage,
MaxBlobSize: int64(1024 * 1024), // 1MB blobs
CompressionLevel: 3,
AgeRecipients: []string{"age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05gl0sjq9q9wjg"}, // Test public key
AgeRecipients: []string{testAgePublicKey},
})
// Create a snapshot
snapshotID := "test-snapshot-001"
err = repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
snapshot := &database.Snapshot{
ID: types.SnapshotID(snapshotID),
Hostname: "test-host",
VaultikVersion: "test-version",
StartedAt: time.Now(),
}
return repos.Snapshots.Create(ctx, tx, snapshot)
})
require.NoError(t, err)
createTestSnapshotRecord(t, repos, snapshotID)
// Run the backup
result, err := scanner.Scan(ctx, "/data", snapshotID)
@@ -402,12 +441,14 @@ func TestBackupAndVerify(t *testing.T) {
// Verify backup created blobs
assert.Positive(t, result.BlobsCreated, "Should create at least one blob")
assert.Equal(t, mockStorage.GetStorageSize(), result.BlobsCreated, "Storage should have the blobs")
assert.Equal(t, mockStorage.GetStorageSize(), result.BlobsCreated,
"Storage should have the blobs")
// Verify we can retrieve the blob from storage
objects, err := mockStorage.List(ctx, "blobs/")
require.NoError(t, err)
assert.Len(t, objects, result.BlobsCreated, "Should have correct number of blobs in storage")
assert.Len(t, objects, result.BlobsCreated,
"Should have correct number of blobs in storage")
// Get the first blob and verify it exists
if len(objects) > 0 {
@@ -438,66 +479,15 @@ func TestBackupAndVerify(t *testing.T) {
// TestBackupAndRestore tests the full backup and restore workflow
// This test verifies that the restore code correctly handles the binary SQLite
// database format that is exported by the snapshot manager.
func TestBackupAndRestore(t *testing.T) {
// Initialize logger
log.Initialize(log.Config{})
// Create real temp directory for the database (SQLite needs real filesystem)
realTempDir, err := os.MkdirTemp("", "vaultik-test-")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(realTempDir) }()
// Use real OS filesystem for this test
fs := afero.NewOsFs()
// Create test directory structure and files
dataDir := filepath.Join(realTempDir, "data")
testFiles := map[string]string{
filepath.Join(dataDir, "file1.txt"): "This is file 1 content",
filepath.Join(dataDir, "file2.txt"): "This is file 2 content with more data",
filepath.Join(dataDir, "subdir", "file3.txt"): "This is file 3 in a subdirectory",
}
// Create directories and files
for path, content := range testFiles {
dir := filepath.Dir(path)
err := fs.MkdirAll(dir, 0755)
if err != nil {
t.Fatalf("failed to create directory %s: %v", dir, err)
}
err = afero.WriteFile(fs, path, []byte(content), 0644)
if err != nil {
t.Fatalf("failed to create test file %s: %v", path, err)
}
}
ctx := context.Background()
// Create mock storage
mockStorage := NewMockStorer()
// Test keypair
agePublicKey := "age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05gl0sjq9q9wjg"
ageSecretKey := "AGE-SECRET-KEY-19CR5YSFW59HM4TLD6GXVEDMZFTVVF7PPHKUT68TXSFPK7APHXA2QS2NJA5"
// Create database file
dbPath := filepath.Join(realTempDir, "test.db")
db, err := database.New(ctx, dbPath)
require.NoError(t, err)
defer func() { _ = db.Close() }()
repos := database.NewRepositories(db)
// Create config for snapshot manager
cfg := &config.Config{
AgeSecretKey: ageSecretKey,
AgeRecipients: []string{agePublicKey},
CompressionLevel: 3,
}
// runBackupPhase performs the backup half of the round-trip test:
// create the snapshot, scan the data directory, complete the snapshot,
// and export its metadata, verifying the metadata upload.
func runBackupPhase(
ctx context.Context, t *testing.T, fs afero.Fs,
repos *database.Repositories, mockStorage *MockStorer,
cfg *config.Config, dataDir, dbPath, agePublicKey string,
) string {
t.Helper()
// Create snapshot manager
sm := snapshot.NewSnapshotManager(snapshot.SnapshotManagerParams{
@@ -519,14 +509,16 @@ func TestBackupAndRestore(t *testing.T) {
})
// Create a snapshot
snapshotID, err := sm.CreateSnapshot(ctx, "test-host", "test-version", "test-git")
snapshotID, err := sm.CreateSnapshotWithName(
ctx, testHostname, "", "test-version", "test-git")
require.NoError(t, err)
t.Logf("Created snapshot: %s", snapshotID)
// Run the backup (scan)
result, err := scanner.Scan(ctx, dataDir, snapshotID)
require.NoError(t, err)
t.Logf("Scan complete: %d files, %d blobs", result.FilesScanned, result.BlobsCreated)
t.Logf("Scan complete: %d files, %d blobs",
result.FilesScanned, result.BlobsCreated)
// Complete the snapshot
err = sm.CompleteSnapshot(ctx, snapshotID)
@@ -541,7 +533,63 @@ func TestBackupAndRestore(t *testing.T) {
keys, err := mockStorage.List(ctx, "metadata/")
require.NoError(t, err)
t.Logf("Metadata keys: %v", keys)
assert.GreaterOrEqual(t, len(keys), 2, "Should have at least db.zst.age and manifest.json.zst")
assert.GreaterOrEqual(t, len(keys), 2,
"Should have at least db.zst.age and manifest.json.zst")
return snapshotID
}
func TestBackupAndRestore(t *testing.T) {
// Initialize logger
log.Initialize(log.Config{})
t.Parallel()
// Create real temp directory for the database (SQLite needs real filesystem)
realTempDir := t.TempDir()
// Use real OS filesystem for this test
fs := afero.NewOsFs()
// Create test directory structure and files
dataDir := filepath.Join(realTempDir, "data")
testFiles := map[string]string{
filepath.Join(dataDir, "file1.txt"): "This is file 1 content",
filepath.Join(dataDir, "file2.txt"): "This is file 2 content " +
"with more data",
filepath.Join(dataDir, "subdir", "file3.txt"): "This is file 3 " +
"in a subdirectory",
}
// Create directories and files
writeTestFileTree(t, fs, testFiles)
ctx := context.Background()
// Create mock storage
mockStorage := NewMockStorer()
// Test keypair
agePublicKey := testAgePublicKey
ageSecretKey := testAgeSecretKey
// Create database file
dbPath := filepath.Join(realTempDir, "test.db")
db, err := database.New(ctx, dbPath)
require.NoError(t, err)
defer func() { _ = db.Close() }()
repos := database.NewRepositories(db)
// Create config for snapshot manager
cfg := &config.Config{
AgeSecretKey: ageSecretKey,
AgeRecipients: []string{agePublicKey},
CompressionLevel: 3,
}
snapshotID := runBackupPhase(
ctx, t, fs, repos, mockStorage, cfg, dataDir, dbPath, agePublicKey)
// Close the source database
err = db.Close()
@@ -570,49 +618,32 @@ func TestBackupAndRestore(t *testing.T) {
for origPath, expectedContent := range testFiles {
restoredPath := filepath.Join(restoreDir, origPath)
restoredContent, err := afero.ReadFile(fs, restoredPath)
require.NoError(t, err, "Should be able to read restored file: %s", restoredPath)
assert.Equal(t, expectedContent, string(restoredContent), "Restored content should match original for: %s", origPath)
require.NoError(t, err,
"Should be able to read restored file: %s", restoredPath)
assert.Equal(t, expectedContent, string(restoredContent),
"Restored content should match original for: %s", origPath)
}
t.Log("Backup and restore test completed successfully")
}
// TestEndToEndFileStorage exercises the full backup → restore loop against the
// real `file://` storage backend (FileStorer) on a real OS filesystem. This is
// the closest local approximation of a production backup: encrypted blobs get
// written to disk, the metadata SQLite database is exported through the same
// blobgen pipeline as a real backup, and restoration reads them back through
// the public Vaultik.Restore entrypoint. It is the canonical end-to-end smoke
// test for 1.0.
func TestEndToEndFileStorage(t *testing.T) {
log.Initialize(log.Config{})
// Real OS filesystem (SQLite + FileStorer both need it).
fs := afero.NewOsFs()
tempDir, err := os.MkdirTemp("", "vaultik-e2e-")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(tempDir) }()
dataDir := filepath.Join(tempDir, "source")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
// Write a representative mix of file sizes:
// - empty file
// - tiny text file
// - file just under chunk boundary
// - file forcing multiple chunks
// - nested subdirectories
chunkSize := int64(64 * 1024)
maxBlobSize := int64(512 * 1024)
// setupE2ESourceTree writes a representative mix of file sizes (empty
// file, tiny text file, file under the chunk boundary, multi-chunk
// file, nested subdirectories), a permission-restricted file, an empty
// directory, and a symlink. It returns the content map keyed by path
// plus the restricted-file, empty-dir, and symlink paths.
func setupE2ESourceTree(
t *testing.T, fs afero.Fs, dataDir string, chunkSize int64,
) (map[string][]byte, string, string, string) {
t.Helper()
testFiles := map[string][]byte{
filepath.Join(dataDir, "empty.txt"): {},
filepath.Join(dataDir, "small.txt"): []byte("hello vaultik"),
filepath.Join(dataDir, "subdir", "medium.bin"): bytesPattern("medium-", int(chunkSize/2)),
filepath.Join(dataDir, "subdir", "large.bin"): bytesPattern("large-", int(chunkSize*4)),
filepath.Join(dataDir, "empty.txt"): {},
filepath.Join(dataDir, "small.txt"): []byte("hello vaultik"),
filepath.Join(dataDir, "subdir", "medium.bin"): bytesPattern(
"medium-", int(chunkSize/2)),
filepath.Join(dataDir, "subdir", "large.bin"): bytesPattern(
"large-", int(chunkSize*4)),
filepath.Join(dataDir, "deep", "nest", "leaf.txt"): []byte("leaf"),
}
@@ -624,6 +655,7 @@ func TestEndToEndFileStorage(t *testing.T) {
// Create a file with non-default permissions.
restrictedPath := filepath.Join(dataDir, "restricted.txt")
require.NoError(t, afero.WriteFile(fs, restrictedPath, []byte("secret"), 0o600))
testFiles[restrictedPath] = []byte("secret")
// Create an empty directory (should survive round-trip).
@@ -634,22 +666,38 @@ func TestEndToEndFileStorage(t *testing.T) {
symlinkPath := filepath.Join(dataDir, "link-to-small")
require.NoError(t, os.Symlink("small.txt", symlinkPath))
return testFiles, restrictedPath, emptyDir, symlinkPath
}
// TestEndToEndFileStorage exercises the full backup → restore loop against the
// real `file://` storage backend (FileStorer) on a real OS filesystem. This is
// the closest local approximation of a production backup: encrypted blobs get
// written to disk, the metadata SQLite database is exported through the same
// blobgen pipeline as a real backup, and restoration reads them back through
// the public Vaultik.Restore entrypoint. It is the canonical end-to-end smoke
// test for 1.0.
// runFileStorageBackup performs the backup half of the file-storage
// end-to-end test against a real on-disk FileStorer, verifies the
// on-disk layout, and closes the index database so the restore half
// runs from remote bytes only.
func runFileStorageBackup(
ctx context.Context, t *testing.T, fs afero.Fs,
dataDir, storeDir, dbPath string,
chunkSize, maxBlobSize int64,
) (*config.Config, *storage.FileStorer, string) {
t.Helper()
// FileStorer is the real-world local-disk backend.
storer, err := storage.NewFileStorer(storeDir)
require.NoError(t, err)
agePublicKey := "age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05gl0sjq9q9wjg"
ageSecretKey := "AGE-SECRET-KEY-19CR5YSFW59HM4TLD6GXVEDMZFTVVF7PPHKUT68TXSFPK7APHXA2QS2NJA5"
cfg := &config.Config{
AgeRecipients: []string{agePublicKey},
AgeSecretKey: ageSecretKey,
AgeRecipients: []string{testAgePublicKey},
AgeSecretKey: testAgeSecretKey,
CompressionLevel: 3,
Hostname: "test-host",
Hostname: testHostname,
}
ctx := context.Background()
db, err := database.New(ctx, dbPath)
require.NoError(t, err)
@@ -674,7 +722,8 @@ func TestEndToEndFileStorage(t *testing.T) {
Repositories: repos,
})
snapshotID, err := sm.CreateSnapshotWithName(ctx, cfg.Hostname, "e2e", "test-version", "test-git")
snapshotID, err := sm.CreateSnapshotWithName(
ctx, cfg.Hostname, "e2e", "test-version", "test-git")
require.NoError(t, err)
scanResult, err := scanner.Scan(ctx, dataDir, snapshotID)
@@ -692,7 +741,8 @@ func TestEndToEndFileStorage(t *testing.T) {
require.NoError(t, err)
require.True(t, blobInfo.IsDir())
metaInfo, err := os.Stat(filepath.Join(storeDir, "metadata", snapshot.RemoteSnapshotKey(snapshotID)))
metaInfo, err := os.Stat(filepath.Join(
storeDir, "metadata", snapshot.RemoteSnapshotKey(snapshotID)))
require.NoError(t, err)
require.True(t, metaInfo.IsDir())
@@ -700,6 +750,39 @@ func TestEndToEndFileStorage(t *testing.T) {
// the remote bytes plus the secret key, with no help from the local index.
require.NoError(t, db.Close())
return cfg, storer, snapshotID
}
func TestEndToEndFileStorage(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
// Real OS filesystem (SQLite + FileStorer both need it).
fs := afero.NewOsFs()
tempDir := t.TempDir()
dataDir := filepath.Join(tempDir, "source")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
// Write a representative mix of file sizes:
// - empty file
// - tiny text file
// - file just under chunk boundary
// - file forcing multiple chunks
// - nested subdirectories
chunkSize := int64(64 * 1024)
maxBlobSize := int64(512 * 1024)
testFiles, restrictedPath, emptyDir, symlinkPath :=
setupE2ESourceTree(t, fs, dataDir, chunkSize)
ctx := context.Background()
cfg, storer, snapshotID := runFileStorageBackup(
ctx, t, fs, dataDir, storeDir, dbPath, chunkSize, maxBlobSize)
restoreVaultik := &vaultik.Vaultik{
Config: cfg,
Storage: storer,
@@ -716,6 +799,19 @@ func TestEndToEndFileStorage(t *testing.T) {
Verify: true,
}))
verifyE2ERestoredTree(t, fs, restoreDir, testFiles,
restrictedPath, emptyDir, symlinkPath)
}
// verifyE2ERestoredTree byte-compares every restored file and checks the
// restricted-permission file, empty directory, and symlink special cases.
func verifyE2ERestoredTree(
t *testing.T, fs afero.Fs, restoreDir string,
testFiles map[string][]byte,
restrictedPath, emptyDir, symlinkPath string,
) {
t.Helper()
// Byte-equality compare every original against its restored copy.
for origPath, expected := range testFiles {
restoredPath := filepath.Join(restoreDir, origPath)
@@ -751,22 +847,22 @@ func TestEndToEndFileStorage(t *testing.T) {
// regression where snapshot_blobs was populated only for blobs uploaded
// during the snapshot, leaving fully-deduplicated snapshots unrestorable
// with "chunk X not found in any blob" errors.
func TestDedupOnlySnapshotRestores(t *testing.T) {
log.Initialize(log.Config{})
// dedupBackupEnv bundles the moving parts of the dedup round-trip test.
type dedupBackupEnv struct {
cfg *config.Config
storer *storage.FileStorer
db *database.DB
repos *database.Repositories
sm *snapshot.SnapshotManager
makeScanner func() *snapshot.Scanner
}
fs := afero.NewOsFs()
tempDir, err := os.MkdirTemp("", "vaultik-dedup-")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(tempDir) }()
dataDir := filepath.Join(tempDir, "source")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
chunkSize := int64(64 * 1024)
maxBlobSize := int64(512 * 1024)
// writeDedupSourceFiles writes the two deterministic source files used
// by the dedup test and returns their expected contents by path.
func writeDedupSourceFiles(
t *testing.T, fs afero.Fs, dataDir string, chunkSize int64,
) map[string][]byte {
t.Helper()
testFiles := map[string][]byte{
filepath.Join(dataDir, "a.bin"): bytesPattern("a-", int(chunkSize*3)),
@@ -777,25 +873,30 @@ func TestDedupOnlySnapshotRestores(t *testing.T) {
require.NoError(t, afero.WriteFile(fs, path, content, 0o644))
}
return testFiles
}
// setupDedupBackupEnv creates the storer, config, database, snapshot
// manager, and scanner factory for the dedup round-trip test.
func setupDedupBackupEnv(
ctx context.Context, t *testing.T, fs afero.Fs,
storeDir, dbPath string, chunkSize, maxBlobSize int64,
) *dedupBackupEnv {
t.Helper()
storer, err := storage.NewFileStorer(storeDir)
require.NoError(t, err)
agePublicKey := "age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05gl0sjq9q9wjg"
ageSecretKey := "AGE-SECRET-KEY-19CR5YSFW59HM4TLD6GXVEDMZFTVVF7PPHKUT68TXSFPK7APHXA2QS2NJA5"
cfg := &config.Config{
AgeRecipients: []string{agePublicKey},
AgeSecretKey: ageSecretKey,
AgeRecipients: []string{testAgePublicKey},
AgeSecretKey: testAgeSecretKey,
CompressionLevel: 3,
Hostname: "test-host",
Hostname: testHostname,
}
ctx := context.Background()
db, err := database.New(ctx, dbPath)
require.NoError(t, err)
defer func() { _ = db.Close() }()
repos := database.NewRepositories(db)
makeScanner := func() *snapshot.Scanner {
@@ -814,31 +915,85 @@ func TestDedupOnlySnapshotRestores(t *testing.T) {
})
sm.SetFilesystem(fs)
return &dedupBackupEnv{
cfg: cfg,
storer: storer,
db: db,
repos: repos,
sm: sm,
makeScanner: makeScanner,
}
}
// runDedupSnapshot creates a "dedup" snapshot, scans dataDir into it,
// completes it, and exports its metadata, returning the snapshot ID and
// scan result.
func runDedupSnapshot(
ctx context.Context, t *testing.T,
sm *snapshot.SnapshotManager, scanner *snapshot.Scanner,
hostname, dataDir, dbPath string,
) (string, *snapshot.ScanResult) {
t.Helper()
id, err := sm.CreateSnapshotWithName(ctx, hostname, "dedup", "v", "g")
require.NoError(t, err)
result, err := scanner.Scan(ctx, dataDir, id)
require.NoError(t, err)
require.NoError(t, sm.CompleteSnapshot(ctx, id))
require.NoError(t, sm.ExportSnapshotMetadata(ctx, dbPath, id))
return id, result
}
func TestDedupOnlySnapshotRestores(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
dataDir := filepath.Join(tempDir, "source")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
chunkSize := int64(64 * 1024)
maxBlobSize := int64(512 * 1024)
ctx := context.Background()
testFiles := writeDedupSourceFiles(t, fs, dataDir, chunkSize)
env := setupDedupBackupEnv(
ctx, t, fs, storeDir, dbPath, chunkSize, maxBlobSize)
defer func() { _ = env.db.Close() }()
cfg, storer, repos, sm := env.cfg, env.storer, env.repos, env.sm
makeScanner := env.makeScanner
db := env.db
// First snapshot — uploads all blobs.
id1, err := sm.CreateSnapshotWithName(ctx, cfg.Hostname, "dedup", "v", "g")
require.NoError(t, err)
r1, err := makeScanner().Scan(ctx, dataDir, id1)
require.NoError(t, err)
require.Positive(t, r1.BlobsCreated, "first snapshot should upload at least one blob")
require.NoError(t, sm.CompleteSnapshot(ctx, id1))
require.NoError(t, sm.ExportSnapshotMetadata(ctx, dbPath, id1))
_, r1 := runDedupSnapshot(ctx, t, sm, makeScanner(),
cfg.Hostname, dataDir, dbPath)
require.Positive(t, r1.BlobsCreated,
"first snapshot should upload at least one blob")
// Second snapshot — same data, every chunk dedups. Sleep past the
// second-precision timestamp so the snapshot IDs differ.
time.Sleep(1100 * time.Millisecond)
id2, err := sm.CreateSnapshotWithName(ctx, cfg.Hostname, "dedup", "v", "g")
require.NoError(t, err)
r2, err := makeScanner().Scan(ctx, dataDir, id2)
require.NoError(t, err)
require.Equal(t, 0, r2.BlobsCreated, "second snapshot should upload zero new blobs (fully dedup'd)")
require.NoError(t, sm.CompleteSnapshot(ctx, id2))
require.NoError(t, sm.ExportSnapshotMetadata(ctx, dbPath, id2))
id2, r2 := runDedupSnapshot(ctx, t, sm, makeScanner(),
cfg.Hostname, dataDir, dbPath)
require.Equal(t, 0, r2.BlobsCreated,
"second snapshot should upload zero new blobs (fully dedup'd)")
// snapshot_blobs for id2 must be populated despite no uploads.
blobHashes, err := repos.Snapshots.GetBlobHashes(ctx, id2)
require.NoError(t, err)
require.NotEmpty(t, blobHashes, "snapshot_blobs for fully-dedup'd snapshot must reference blobs uploaded by prior snapshot")
require.NotEmpty(t, blobHashes, "snapshot_blobs for fully-dedup'd snapshot "+
"must reference blobs uploaded by prior snapshot")
require.NoError(t, db.Close())
@@ -871,7 +1026,7 @@ func TestDedupOnlySnapshotRestores(t *testing.T) {
func bytesPattern(tag string, n int) []byte {
out := make([]byte, n)
for i := range out {
out[i] = byte(tag[i%len(tag)] ^ byte(i&0xff))
out[i] = tag[i%len(tag)] ^ byte(i&0xff)
}
return out

69
internal/vaultik/prune.go
View File

@@ -6,7 +6,6 @@ import (
"fmt"
"strings"
"github.com/dustin/go-humanize"
"sneak.berlin/go/vaultik/internal/log"
)
@@ -16,6 +15,14 @@ type PruneOptions struct {
JSON bool
}
// errNukeRequiresForce guards the destructive remote nuke operation.
var errNukeRequiresForce = errors.New(
"nuke requires --force (this deletes ALL remote snapshots and blobs)")
// metadataDirName is the top-level remote directory holding snapshot
// metadata.
const metadataDirName = "metadata"
// NukeRemote deletes every snapshot's metadata and every blob from remote
// storage. After this returns successfully the bucket prefix is empty and
// the next backup starts from scratch.
@@ -24,29 +31,31 @@ type PruneOptions struct {
// confirming with the user.
func (v *Vaultik) NukeRemote(force bool) error {
if !force {
return errors.New("nuke requires --force (this deletes ALL remote snapshots and blobs)")
return errNukeRequiresForce
}
v.UI.Begin("Removing all snapshot metadata from backup destination store.")
v.UI.Beginf("Removing all snapshot metadata from backup destination store.")
_, err := v.RemoveAllSnapshots(&RemoveOptions{Force: true})
if err != nil {
return fmt.Errorf("removing all snapshots: %w", err)
}
v.UI.Begin("Removing any blobs still present in backup destination store.")
v.UI.Beginf("Removing any blobs still present in backup destination store.")
err = v.PruneBlobs(&PruneOptions{Force: true})
if err != nil {
return fmt.Errorf("pruning blobs: %w", err)
}
v.UI.Complete("Backup destination store is now empty.")
v.UI.Completef("Backup destination store is now empty.")
return nil
}
// PruneBlobsResult contains the result of a blob prune operation
//
//nolint:tagliatelle // snake_case is the established JSON output format
type PruneBlobsResult struct {
BlobsFound int `json:"blobs_found"`
BlobsDeleted int `json:"blobs_deleted"`
@@ -113,14 +122,16 @@ func (v *Vaultik) PruneBlobs(opts *PruneOptions) error {
return nil
}
log.Info("Found unreferenced blobs", "count", len(unreferencedBlobs), "total_size", humanize.Bytes(uint64(totalSize)))
log.Info("Found unreferenced blobs",
"count", len(unreferencedBlobs), "total_size", ubytes(totalSize))
if !opts.JSON {
v.printfStdout("Found %d unreferenced blob(s) totaling %s\n", len(unreferencedBlobs), humanize.Bytes(uint64(totalSize)))
v.stdoutf("Found %d unreferenced blob(s) totaling %s\n",
len(unreferencedBlobs), ubytes(totalSize))
}
if !opts.Force && !opts.JSON {
v.printfStdout("\nDelete %d unreferenced blob(s)? [y/N] ", len(unreferencedBlobs))
v.stdoutf("\nDelete %d unreferenced blob(s)? [y/N] ", len(unreferencedBlobs))
var confirm string
@@ -128,7 +139,7 @@ func (v *Vaultik) PruneBlobs(opts *PruneOptions) error {
if err != nil {
v.printlnStdout("Cancelled")
return nil
return nil //nolint:nilerr // read failure means no confirmation
}
if strings.ToLower(confirm) != "y" {
@@ -144,16 +155,18 @@ func (v *Vaultik) PruneBlobs(opts *PruneOptions) error {
return v.outputPruneBlobsJSON(result)
}
v.printfStdout("\nDeleted %d blob(s) totaling %s\n", result.BlobsDeleted, humanize.Bytes(uint64(result.BytesFreed)))
v.stdoutf("\nDeleted %d blob(s) totaling %s\n",
result.BlobsDeleted, ubytes(result.BytesFreed))
if result.BlobsFailed > 0 {
v.printfStdout("Failed to delete %d blob(s)\n", result.BlobsFailed)
v.stdoutf("Failed to delete %d blob(s)\n", result.BlobsFailed)
}
return nil
}
// collectReferencedBlobs downloads all manifests and returns the set of referenced blob hashes
// collectReferencedBlobs downloads all manifests and returns the set of
// referenced blob hashes.
func (v *Vaultik) collectReferencedBlobs() (map[string]bool, error) {
log.Info("Listing remote snapshots")
// IDs returned by listUniqueSnapshotIDs are remote keys (hashed
@@ -185,7 +198,8 @@ func (v *Vaultik) collectReferencedBlobs() (map[string]bool, error) {
manifestCount++
}
log.Info("Processed manifests", "count", manifestCount, "unique_blobs_referenced", len(allBlobsReferenced))
log.Info("Processed manifests",
"count", manifestCount, "unique_blobs_referenced", len(allBlobsReferenced))
return allBlobsReferenced, nil
}
@@ -203,8 +217,10 @@ func (v *Vaultik) listUniqueSnapshotIDs() ([]string, error) {
}
parts := strings.Split(object.Key, "/")
if len(parts) >= 2 && parts[0] == "metadata" && parts[1] != "" {
if strings.HasSuffix(object.Key, "/") || strings.Contains(object.Key, "/manifest.json.zst") {
if len(parts) >= minSnapshotIDParts &&
parts[0] == metadataDirName && parts[1] != "" {
if strings.HasSuffix(object.Key, "/") ||
strings.Contains(object.Key, "/manifest.json.zst") {
snapshotID := parts[1]
if !seen[snapshotID] {
seen[snapshotID] = true
@@ -230,7 +246,7 @@ func (v *Vaultik) listAllRemoteBlobs() (map[string]int64, error) {
}
parts := strings.Split(object.Key, "/")
if len(parts) == 4 && parts[0] == "blobs" {
if len(parts) == blobKeyParts && parts[0] == "blobs" {
allBlobs[parts[3]] = object.Size
}
}
@@ -240,8 +256,11 @@ func (v *Vaultik) listAllRemoteBlobs() (map[string]int64, error) {
return allBlobs, nil
}
// findUnreferencedBlobs returns blob hashes not referenced by any manifest and their total size
func (v *Vaultik) findUnreferencedBlobs(allBlobs map[string]int64, referenced map[string]bool) ([]string, int64) {
// findUnreferencedBlobs returns blob hashes not referenced by any
// manifest and their total size.
func (v *Vaultik) findUnreferencedBlobs(
allBlobs map[string]int64, referenced map[string]bool,
) ([]string, int64) {
var (
unreferenced []string
totalSize int64
@@ -257,8 +276,11 @@ func (v *Vaultik) findUnreferencedBlobs(allBlobs map[string]int64, referenced ma
return unreferenced, totalSize
}
// deleteUnreferencedBlobs deletes the given blobs from storage and populates the result
func (v *Vaultik) deleteUnreferencedBlobs(unreferencedBlobs []string, allBlobs map[string]int64, result *PruneBlobsResult) {
// deleteUnreferencedBlobs deletes the given blobs from storage and
// populates the result.
func (v *Vaultik) deleteUnreferencedBlobs(
unreferencedBlobs []string, allBlobs map[string]int64, result *PruneBlobsResult,
) {
log.Info("Deleting unreferenced blobs")
for i, hash := range unreferencedBlobs {
@@ -274,11 +296,12 @@ func (v *Vaultik) deleteUnreferencedBlobs(unreferencedBlobs []string, allBlobs m
result.BlobsDeleted++
result.BytesFreed += allBlobs[hash]
if (i+1)%100 == 0 || i == len(unreferencedBlobs)-1 {
if (i+1)%progressLogEvery == 0 || i == len(unreferencedBlobs)-1 {
log.Info("Deletion progress",
"deleted", i+1,
"total", len(unreferencedBlobs),
"percent", fmt.Sprintf("%.1f%%", float64(i+1)/float64(len(unreferencedBlobs))*100),
"percent", fmt.Sprintf("%.1f%%",
float64(i+1)/float64(len(unreferencedBlobs))*percentScale),
)
}
}
@@ -287,7 +310,7 @@ func (v *Vaultik) deleteUnreferencedBlobs(unreferencedBlobs []string, allBlobs m
log.Info("Prune complete",
"deleted_count", result.BlobsDeleted,
"deleted_size", humanize.Bytes(uint64(result.BytesFreed)),
"deleted_size", ubytes(result.BytesFreed),
"failed", result.BlobsFailed,
)
}

70
internal/vaultik/purge_per_name_test.go
View File

@@ -16,12 +16,19 @@ import (
"sneak.berlin/go/vaultik/internal/vaultik"
)
// Snapshot IDs reused across the purge tests.
const (
snapSystemT0 = "testhost_system_2026-01-01T00:00:00Z"
snapHomeT0 = "testhost_home_2026-01-01T00:00:00Z"
snapHomeT1 = "testhost_home_2026-01-01T01:00:00Z"
snapHomeT3 = "testhost_home_2026-01-01T03:00:00Z"
)
// setupPurgeTest creates a Vaultik instance with an in-memory database and mock
// storage pre-populated with the given snapshot IDs. Each snapshot is marked as
// completed. Remote metadata stubs are created so syncWithRemote keeps them.
func setupPurgeTest(t *testing.T, snapshotIDs []string) *vaultik.Vaultik {
t.Helper()
log.Initialize(log.Config{})
ctx := context.Background()
db, err := database.New(ctx, ":memory:")
@@ -44,7 +51,7 @@ func setupPurgeTest(t *testing.T, snapshotIDs []string) *vaultik.Vaultik {
snap := &database.Snapshot{
ID: types.SnapshotID(id),
Hostname: "testhost",
VaultikVersion: "test",
VaultikVersion: testLabel,
StartedAt: startedAt,
CompletedAt: &completedAt,
}
@@ -96,13 +103,16 @@ func listRemainingSnapshots(t *testing.T, v *vaultik.Vaultik) []string {
}
func TestPurgeKeepLatest_PerName(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
// Create snapshots for two different names: "home" and "system".
// With per-name --keep-latest, the latest of each should be kept.
snapshotIDs := []string{
"testhost_system_2026-01-01T00:00:00Z",
"testhost_home_2026-01-01T01:00:00Z",
snapSystemT0,
snapHomeT1,
"testhost_system_2026-01-01T02:00:00Z",
"testhost_home_2026-01-01T03:00:00Z",
snapHomeT3,
"testhost_system_2026-01-01T04:00:00Z",
}
@@ -118,15 +128,19 @@ func TestPurgeKeepLatest_PerName(t *testing.T) {
// Should keep the latest of each name
assert.Len(t, remaining, 2, "should keep exactly 2 snapshots (one per name)")
assert.Contains(t, remaining, "testhost_system_2026-01-01T04:00:00Z", "should keep latest system")
assert.Contains(t, remaining, "testhost_home_2026-01-01T03:00:00Z", "should keep latest home")
assert.Contains(t, remaining, "testhost_system_2026-01-01T04:00:00Z",
"should keep latest system")
assert.Contains(t, remaining, snapHomeT3, "should keep latest home")
}
func TestPurgeKeepLatest_SingleName(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
// All snapshots have the same name — keep-latest should keep exactly one.
snapshotIDs := []string{
"testhost_home_2026-01-01T00:00:00Z",
"testhost_home_2026-01-01T01:00:00Z",
snapHomeT0,
snapHomeT1,
"testhost_home_2026-01-01T02:00:00Z",
}
@@ -140,17 +154,21 @@ func TestPurgeKeepLatest_SingleName(t *testing.T) {
remaining := listRemainingSnapshots(t, v)
assert.Len(t, remaining, 1)
assert.Contains(t, remaining, "testhost_home_2026-01-01T02:00:00Z", "should keep the newest")
assert.Contains(t, remaining, "testhost_home_2026-01-01T02:00:00Z",
"should keep the newest")
}
func TestPurgeKeepLatest_WithNameFilter(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
// Use --name to filter purge to only "home" snapshots.
// "system" snapshots should be untouched.
snapshotIDs := []string{
"testhost_system_2026-01-01T00:00:00Z",
"testhost_home_2026-01-01T01:00:00Z",
snapSystemT0,
snapHomeT1,
"testhost_system_2026-01-01T02:00:00Z",
"testhost_home_2026-01-01T03:00:00Z",
snapHomeT3,
"testhost_home_2026-01-01T04:00:00Z",
}
@@ -167,12 +185,15 @@ func TestPurgeKeepLatest_WithNameFilter(t *testing.T) {
// 2 system snapshots untouched + 1 latest home = 3
assert.Len(t, remaining, 3)
assert.Contains(t, remaining, "testhost_system_2026-01-01T00:00:00Z")
assert.Contains(t, remaining, snapSystemT0)
assert.Contains(t, remaining, "testhost_system_2026-01-01T02:00:00Z")
assert.Contains(t, remaining, "testhost_home_2026-01-01T04:00:00Z")
}
func TestPurgeKeepLatest_NoSnapshots(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
v := setupPurgeTest(t, nil)
err := v.PurgeSnapshotsWithOptions(&vaultik.SnapshotPurgeOptions{
@@ -183,8 +204,11 @@ func TestPurgeKeepLatest_NoSnapshots(t *testing.T) {
}
func TestPurgeKeepLatest_NameFilterNoMatch(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
snapshotIDs := []string{
"testhost_system_2026-01-01T00:00:00Z",
snapSystemT0,
"testhost_system_2026-01-01T01:00:00Z",
}
@@ -203,13 +227,16 @@ func TestPurgeKeepLatest_NameFilterNoMatch(t *testing.T) {
}
func TestPurgeOlderThan_WithNameFilter(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
// Snapshots with different names and timestamps.
// --older-than should apply only to the named subset when --name is used.
snapshotIDs := []string{
"testhost_system_2020-01-01T00:00:00Z",
"testhost_home_2020-01-01T00:00:00Z",
"testhost_system_2026-01-01T00:00:00Z",
"testhost_home_2026-01-01T00:00:00Z",
snapSystemT0,
snapHomeT0,
}
v := setupPurgeTest(t, snapshotIDs)
@@ -227,14 +254,17 @@ func TestPurgeOlderThan_WithNameFilter(t *testing.T) {
// Old system stays (not filtered by name), old home deleted, recent ones stay
assert.Len(t, remaining, 3)
assert.Contains(t, remaining, "testhost_system_2020-01-01T00:00:00Z")
assert.Contains(t, remaining, "testhost_system_2026-01-01T00:00:00Z")
assert.Contains(t, remaining, "testhost_home_2026-01-01T00:00:00Z")
assert.Contains(t, remaining, snapSystemT0)
assert.Contains(t, remaining, snapHomeT0)
}
func TestPurgeKeepLatest_ThreeNames(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
// Three different snapshot names with multiple snapshots each.
snapshotIDs := []string{
"testhost_home_2026-01-01T00:00:00Z",
snapHomeT0,
"testhost_system_2026-01-01T01:00:00Z",
"testhost_media_2026-01-01T02:00:00Z",
"testhost_home_2026-01-01T03:00:00Z",

82
internal/vaultik/remove_snapshot_test.go
View File

@@ -29,7 +29,7 @@ func newTestStorer() *testStorer {
}
}
func (s *testStorer) Put(ctx context.Context, key string, reader io.Reader) error {
func (s *testStorer) Put(_ context.Context, key string, reader io.Reader) error {
s.mu.Lock()
defer s.mu.Unlock()
@@ -43,11 +43,14 @@ func (s *testStorer) Put(ctx context.Context, key string, reader io.Reader) erro
return nil
}
func (s *testStorer) PutWithProgress(ctx context.Context, key string, reader io.Reader, size int64, progress storage.ProgressCallback) error {
func (s *testStorer) PutWithProgress(
ctx context.Context, key string, reader io.Reader,
_ int64, _ storage.ProgressCallback,
) error {
return s.Put(ctx, key, reader)
}
func (s *testStorer) Get(ctx context.Context, key string) (io.ReadCloser, error) {
func (s *testStorer) Get(_ context.Context, key string) (io.ReadCloser, error) {
s.mu.Lock()
defer s.mu.Unlock()
@@ -59,7 +62,7 @@ func (s *testStorer) Get(ctx context.Context, key string) (io.ReadCloser, error)
return io.NopCloser(bytes.NewReader(data)), nil
}
func (s *testStorer) Stat(ctx context.Context, key string) (*storage.ObjectInfo, error) {
func (s *testStorer) Stat(_ context.Context, key string) (*storage.ObjectInfo, error) {
s.mu.Lock()
defer s.mu.Unlock()
@@ -74,7 +77,7 @@ func (s *testStorer) Stat(ctx context.Context, key string) (*storage.ObjectInfo,
}, nil
}
func (s *testStorer) Delete(ctx context.Context, key string) error {
func (s *testStorer) Delete(_ context.Context, key string) error {
s.mu.Lock()
defer s.mu.Unlock()
@@ -83,7 +86,7 @@ func (s *testStorer) Delete(ctx context.Context, key string) error {
return nil
}
func (s *testStorer) List(ctx context.Context, prefix string) ([]string, error) {
func (s *testStorer) List(_ context.Context, prefix string) ([]string, error) {
s.mu.Lock()
defer s.mu.Unlock()
@@ -98,7 +101,9 @@ func (s *testStorer) List(ctx context.Context, prefix string) ([]string, error)
return keys, nil
}
func (s *testStorer) ListStream(ctx context.Context, prefix string) <-chan storage.ObjectInfo {
func (s *testStorer) ListStream(
_ context.Context, prefix string,
) <-chan storage.ObjectInfo {
ch := make(chan storage.ObjectInfo)
go func() {
@@ -120,6 +125,13 @@ func (s *testStorer) ListStream(ctx context.Context, prefix string) <-chan stora
return ch
}
func (s *testStorer) Info() storage.Info {
return storage.Info{
Type: testLabel,
Location: "memory",
}
}
func (s *testStorer) hasKey(key string) bool {
s.mu.Lock()
defer s.mu.Unlock()
@@ -136,17 +148,16 @@ func (s *testStorer) keyCount() int {
return len(s.data)
}
func (s *testStorer) Info() storage.StorageInfo {
return storage.StorageInfo{
Type: "test",
Location: "memory",
}
}
// testBlobHashA is a fixture blob hash reused across the remove tests.
const testBlobHashA = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" +
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
// addManifest creates a compressed manifest in storage at the same
// hashed path the production code uses. snapshotID is the human ID;
// the storage path uses RemoteSnapshotKey(id).
func addManifest(t *testing.T, store *testStorer, snapshotID string, blobHashes []string) {
func addManifest(
t *testing.T, store *testStorer, snapshotID string, blobHashes []string,
) {
t.Helper()
blobs := make([]snapshot.BlobInfo, len(blobHashes))
@@ -173,11 +184,12 @@ func addManifest(t *testing.T, store *testStorer, snapshotID string, blobHashes
}
// remoteKeyPath returns the storage-relative path to a snapshot's
// metadata directory or manifest under the hashed remote-key scheme.
// Tests use this in hasKey/asserts to avoid scattering RemoteSnapshotKey
// calls throughout.
func remoteKeyPath(snapshotID, suffix string) string {
return "metadata/" + snapshot.RemoteSnapshotKey(snapshotID) + "/" + suffix
// manifest under the hashed remote-key scheme. Tests use this in
// hasKey/asserts to avoid scattering RemoteSnapshotKey calls
// throughout.
func remoteKeyPath(snapshotID string) string {
return "metadata/" + snapshot.RemoteSnapshotKey(snapshotID) +
"/manifest.json.zst"
}
// addBlob adds a fake blob to storage
@@ -206,10 +218,11 @@ func addBlob(t *testing.T, store *testStorer, hash string) {
// untouched.
func TestRemoveSnapshot_LocalOnly_PreservesRemote(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
blobA := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
blobA := testBlobHashA
addManifest(t, store, "snapshot-001", []string{blobA})
addBlob(t, store, blobA)
@@ -223,9 +236,10 @@ func TestRemoveSnapshot_LocalOnly_PreservesRemote(t *testing.T) {
assert.False(t, result.RemoteRemoved)
assert.True(t, store.hasKey("blobs/aa/aa/"+blobA))
assert.True(t, store.hasKey(remoteKeyPath("snapshot-001", "manifest.json.zst")))
assert.True(t, store.hasKey(remoteKeyPath("snapshot-001")))
assert.Contains(t, tv.Stdout.String(), "Removed snapshot 'snapshot-001' from local database")
assert.Contains(t, tv.Stdout.String(),
"Removed snapshot 'snapshot-001' from local database")
}
// TestRemoveSnapshot_DefaultRemovesMetadataNotBlobs is the canonical
@@ -235,10 +249,11 @@ func TestRemoveSnapshot_LocalOnly_PreservesRemote(t *testing.T) {
// remaining remote manifest, and the output prints that exact command.
func TestRemoveSnapshot_DefaultRemovesMetadataNotBlobs(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
blobUnique := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
blobUnique := testBlobHashA
blobShared := "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
addManifest(t, store, "snapshot-001", []string{blobUnique, blobShared})
@@ -255,8 +270,8 @@ func TestRemoveSnapshot_DefaultRemovesMetadataNotBlobs(t *testing.T) {
assert.Equal(t, "snapshot-001", result.SnapshotID)
assert.True(t, result.RemoteRemoved)
assert.False(t, store.hasKey(remoteKeyPath("snapshot-001", "manifest.json.zst")))
assert.True(t, store.hasKey(remoteKeyPath("snapshot-002", "manifest.json.zst")))
assert.False(t, store.hasKey(remoteKeyPath("snapshot-001")))
assert.True(t, store.hasKey(remoteKeyPath("snapshot-002")))
// Blobs are intentionally NOT touched — that's what `vaultik prune`
// is for.
assert.True(t, store.hasKey("blobs/aa/aa/"+blobUnique))
@@ -270,10 +285,11 @@ func TestRemoveSnapshot_DefaultRemovesMetadataNotBlobs(t *testing.T) {
func TestRemoveSnapshot_DryRun(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
blobA := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
blobA := testBlobHashA
addManifest(t, store, "snapshot-001", []string{blobA})
addBlob(t, store, blobA)
@@ -289,13 +305,14 @@ func TestRemoveSnapshot_DryRun(t *testing.T) {
assert.Equal(t, initialCount, store.keyCount())
assert.True(t, store.hasKey("blobs/aa/aa/"+blobA))
assert.True(t, store.hasKey(remoteKeyPath("snapshot-001", "manifest.json.zst")))
assert.True(t, store.hasKey(remoteKeyPath("snapshot-001")))
assert.Contains(t, tv.Stdout.String(), "[Dry run - no changes made]")
}
func TestRemoveAllSnapshots_RequiresForce(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
addManifest(t, store, "snapshot-001", []string{})
@@ -306,16 +323,17 @@ func TestRemoveAllSnapshots_RequiresForce(t *testing.T) {
opts := &vaultik.RemoveOptions{} // No Force
_, err := tv.RemoveAllSnapshots(opts)
assert.Error(t, err)
require.Error(t, err)
assert.Contains(t, err.Error(), "--all requires --force")
}
func TestRemoveAllSnapshots_WithForce(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
blobA := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
blobA := testBlobHashA
addManifest(t, store, "snapshot-001", []string{blobA})
addManifest(t, store, "snapshot-002", []string{blobA})
addBlob(t, store, blobA)
@@ -331,8 +349,8 @@ func TestRemoveAllSnapshots_WithForce(t *testing.T) {
// Blobs intentionally preserved — that's prune's job.
assert.True(t, store.hasKey("blobs/aa/aa/"+blobA))
assert.False(t, store.hasKey(remoteKeyPath("snapshot-001", "manifest.json.zst")))
assert.False(t, store.hasKey(remoteKeyPath("snapshot-002", "manifest.json.zst")))
assert.False(t, store.hasKey(remoteKeyPath("snapshot-001")))
assert.False(t, store.hasKey(remoteKeyPath("snapshot-002")))
out := tv.Stdout.String()
assert.Contains(t, out, "Removed 2 snapshot(s)")
@@ -342,6 +360,7 @@ func TestRemoveAllSnapshots_WithForce(t *testing.T) {
func TestRemoveAllSnapshots_DryRun(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
addManifest(t, store, "snapshot-001", []string{})
@@ -368,6 +387,7 @@ func TestRemoveAllSnapshots_DryRun(t *testing.T) {
func TestRemoveAllSnapshots_NoSnapshots(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
store := newTestStorer()
// No snapshots added

624
internal/vaultik/restore.go
View File

@@ -14,7 +14,6 @@ import (
"time"
"filippo.io/age"
"github.com/dustin/go-humanize"
"github.com/spf13/afero"
"sneak.berlin/go/vaultik/internal/blobgen"
"sneak.berlin/go/vaultik/internal/database"
@@ -23,6 +22,34 @@ import (
"sneak.berlin/go/vaultik/internal/types"
)
// Sentinel errors for restore failures.
var (
errFilesFailedRestore = errors.New("file(s) failed to restore")
errFilesFailedVerify = errors.New("files failed verification")
errDecryptionKeyRequired = errors.New(
"decryption key required for restore\n\n" +
"Set the VAULTIK_AGE_SECRET_KEY environment variable to your " +
"age private key:\n" +
" export VAULTIK_AGE_SECRET_KEY='AGE-SECRET-KEY-...'")
errBlobMissingFromIndex = errors.New("blob hash missing from blob index")
errChunkNotInAnyBlob = errors.New("chunk not found in any blob")
errBlobIDNotInHashIndex = errors.New("blob id missing from hash index")
errShortChunkRead = errors.New("short read")
)
// restoreDirMode is the permission mode for directories created while
// restoring (parent directories and the target root; restored
// directories themselves get their stored mode).
const restoreDirMode = 0o755
// sweepIntervalDivisor sets the sweeper threshold to one N-th of the
// configured blob size limit.
const sweepIntervalDivisor = 100
// restoreStatusInterval is how often periodic progress lines are
// printed during restore and verify.
const restoreStatusInterval = 15 * time.Second
// RestoreOptions contains options for the restore operation
type RestoreOptions struct {
SnapshotID string
@@ -91,16 +118,17 @@ func (v *Vaultik) Restore(opts *RestoreOptions) error {
if len(files) == 0 {
log.Warn("No files found to restore")
v.UI.Warning("No files found to restore.")
v.UI.Warningf("No files found to restore.")
return nil
}
log.Info("Found files to restore", "count", len(files))
v.UI.Info("Found %s files to restore.", v.UI.Count(len(files)))
v.UI.Infof("Found %s files to restore.", v.UI.Count(len(files)))
// Step 3: Create target directory
if err := v.Fs.MkdirAll(opts.TargetDir, 0755); err != nil {
err = v.Fs.MkdirAll(opts.TargetDir, restoreDirMode)
if err != nil {
return fmt.Errorf("creating target directory: %w", err)
}
@@ -120,27 +148,40 @@ func (v *Vaultik) Restore(opts *RestoreOptions) error {
log.Info("Restore complete",
"files_restored", result.FilesRestored,
"bytes_restored", humanize.Bytes(uint64(result.BytesRestored)),
"bytes_restored", ubytes(result.BytesRestored),
"blobs_downloaded", result.BlobsDownloaded,
"bytes_downloaded", humanize.Bytes(uint64(result.BytesDownloaded)),
"bytes_downloaded", ubytes(result.BytesDownloaded),
"duration", result.Duration,
)
v.UI.Complete("Restored %s files (%s) in %s.",
v.UI.Completef("Restored %s files (%s) in %s.",
v.UI.Count(result.FilesRestored),
v.UI.Size(result.BytesRestored),
v.UI.Duration(result.Duration),
)
return v.finishRestore(repos, files, opts, result)
}
// finishRestore emits the post-restore warnings, runs optional
// verification, and converts any failed-file count into an error.
func (v *Vaultik) finishRestore(
repos *database.Repositories,
files []*database.File,
opts *RestoreOptions,
result *RestoreResult,
) error {
if os.Geteuid() != 0 {
v.UI.Warning("Restore did not preserve file ownership: chown(2) requires root. Re-run as root (e.g. with sudo) if you need original UID/GID preserved.")
v.UI.Warningf("Restore did not preserve file ownership: chown(2) " +
"requires root. Re-run as root (e.g. with sudo) if you need " +
"original UID/GID preserved.")
}
if result.FilesFailed > 0 {
v.UI.Warning("%d file(s) failed to restore:", result.FilesFailed)
v.UI.Warningf("%d file(s) failed to restore:", result.FilesFailed)
for _, path := range result.FailedFiles {
v.UI.Detail("%s", v.UI.Path(path))
v.UI.Detailf("%s", v.UI.Path(path))
}
}
@@ -153,16 +194,19 @@ func (v *Vaultik) Restore(opts *RestoreOptions) error {
}
if result.FilesFailed > 0 {
return fmt.Errorf("%d file(s) failed to restore", result.FilesFailed)
return fmt.Errorf("%d %w", result.FilesFailed, errFilesFailedRestore)
}
return nil
}
// prepareRestoreIdentity validates that an age secret key is configured and parses it
// prepareRestoreIdentity validates that an age secret key is configured
// and parses it.
//
//nolint:ireturn // age.Identity is the decryption abstraction by design
func (v *Vaultik) prepareRestoreIdentity() (age.Identity, error) {
if v.Config.AgeSecretKey == "" {
return nil, errors.New("decryption key required for restore\n\nSet the VAULTIK_AGE_SECRET_KEY environment variable to your age private key:\n export VAULTIK_AGE_SECRET_KEY='AGE-SECRET-KEY-...'")
return nil, errDecryptionKeyRequired
}
identity, err := age.ParseX25519Identity(v.Config.AgeSecretKey)
@@ -212,22 +256,12 @@ func (v *Vaultik) restoreAllFiles(
// Per-restore sweep state: every blob_size_limit/100 bytes written,
// scan the cache and delete any blob whose remaining file references
// are all already restored.
sweeper := newRestoreSweeper(v.ctx, repos, blobCache, v.Config.BlobSizeLimit.Int64()/100)
sweeper := newRestoreSweeper(v.ctx, repos, blobCache,
v.Config.BlobSizeLimit.Int64()/sweepIntervalDivisor)
// Pre-fetch every blob row once so chunk extraction can map a
// blob_id to its hash without a DB round-trip per chunk.
blobsByID, err := repos.Blobs.GetAll(v.ctx)
blobByHash, blobIDToHash, err := v.buildBlobIndexes(repos)
if err != nil {
return nil, fmt.Errorf("fetching blob index: %w", err)
}
blobIDToHash := make(map[string]string, len(blobsByID))
blobByHash := make(map[string]*database.Blob, len(blobsByID))
for id, blob := range blobsByID {
hash := blob.Hash.String()
blobIDToHash[id] = hash
blobByHash[hash] = blob
return nil, err
}
plan, err := newRestorePlan(v.ctx, repos, files, chunkToBlobMap, blobIDToHash)
@@ -235,20 +269,9 @@ func (v *Vaultik) restoreAllFiles(
return nil, fmt.Errorf("building restore plan: %w", err)
}
// Index files by ID so the loop can look them up by the IDs the
// plan hands back.
filesByID := make(map[types.FileID]*database.File, len(files))
for _, f := range files {
filesByID[f.ID] = f
}
filesByID, totalBytesExpected := indexRestoreFiles(files)
// Calculate total bytes expected for percentage / ETA arithmetic.
var totalBytesExpected int64
for _, file := range files {
totalBytesExpected += file.Size
}
v.UI.Begin("Restoring %s files (%s) to %s.",
v.UI.Beginf("Restoring %s files (%s) to %s.",
v.UI.Count(len(files)),
v.UI.Size(totalBytesExpected),
v.UI.Path(opts.TargetDir))
@@ -268,52 +291,41 @@ func (v *Vaultik) restoreAllFiles(
runningAsRoot: os.Geteuid() == 0,
}
err = v.runRestoreLoop(session, plan, filesByID, totalBytesExpected)
if err != nil {
return nil, err
}
return result, nil
}
// runRestoreLoop drains the restore plan: restore files as their blobs
// become available, download the next blob set when nothing is ready,
// and emit periodic progress.
func (v *Vaultik) runRestoreLoop(
session *restoreSession, plan *restorePlan,
filesByID map[types.FileID]*database.File, totalBytesExpected int64,
) error {
// Periodic progress output, matching the snapshot create cadence.
startTime := time.Now()
lastStatusTime := startTime
const statusInterval = 15 * time.Second
processed := 0
totalFiles := len(filesByID)
for plan.hasPending() {
if v.ctx.Err() != nil {
return nil, v.ctx.Err()
return v.ctx.Err()
}
fileID, ready := plan.popReady()
if !ready {
// No file is fully cache-served. First free any blobs
// whose file sets are exhausted — without this, the
// blob whose last file we just finished would still be
// cached when we Put the next one, briefly pushing
// peak occupancy from 1 to 2.
sweeper.sweep()
// Pick the pending file with the smallest uncached
// blob set and download its blobs. After each blob
// lands, the plan moves any pending file whose set
// just emptied onto the ready queue.
next := plan.pickNextDownload()
if next.IsZero() {
break
downloaded, err := session.downloadNextBlobSet(plan)
if err != nil {
return err
}
for _, hash := range plan.blobsNeeded(next) {
blob, ok := blobByHash[hash]
if !ok {
return nil, fmt.Errorf("blob hash %s missing from blob index", hash[:16])
}
err := session.downloadBlobToCache(hash, blob.CompressedSize)
if err != nil {
return nil, fmt.Errorf("downloading blob %s: %w", hash[:16], err)
}
result.BlobsDownloaded++
result.BytesDownloaded += blob.CompressedSize
plan.markBlobCached(hash)
if !downloaded {
break
}
continue
@@ -323,54 +335,173 @@ func (v *Vaultik) restoreAllFiles(
err := session.restoreFile(file)
if err != nil {
log.Error("Failed to restore file", "path", file.Path, "error", err)
if !opts.SkipErrors {
return nil, fmt.Errorf("restoring %s: %w (pass --skip-errors to continue past restore failures)", file.Path, err)
err = v.handleRestoreFileError(
plan, session.opts, session.result, file, fileID, err)
if err != nil {
return err
}
v.UI.Error("Failed to restore %s: %v. Skipping (--skip-errors).", v.UI.Path(file.Path.String()), err)
result.FilesFailed++
result.FailedFiles = append(result.FailedFiles, file.Path.String())
plan.finishFile(fileID)
continue
}
// Record the file as restored so the sweeper can free blobs
// once all referencing files are done, and drop it from the
// plan's indexes so future picks ignore it.
sweeper.fileRestored(fileID.String())
session.sweeper.fileRestored(fileID.String())
plan.finishFile(fileID)
processed++
if time.Since(lastStatusTime) >= statusInterval {
v.printRestoreProgress(processed, len(files), result.BytesRestored, totalBytesExpected, startTime)
lastStatusTime = time.Now()
}
// Structured progress log for --verbose / JSON consumers.
if processed%100 == 0 || processed == len(files) {
log.Info("Restore progress",
"files", fmt.Sprintf("%d/%d", processed, len(files)),
"bytes", humanize.Bytes(uint64(result.BytesRestored)),
)
}
v.restoreProgressTick(processed, totalFiles,
session.result.BytesRestored,
totalBytesExpected, startTime, &lastStatusTime)
}
return result, nil
return nil
}
// downloadNextBlobSet is invoked when no file is fully cache-served.
// It first frees any blobs whose file sets are exhausted — without
// this, the blob whose last file we just finished would still be
// cached when we Put the next one, briefly pushing peak occupancy from
// 1 to 2. It then picks the pending file with the smallest uncached
// blob set and downloads its blobs; after each blob lands, the plan
// moves any pending file whose set just emptied onto the ready queue.
// Returns false when nothing is pending download (the caller stops).
func (s *restoreSession) downloadNextBlobSet(plan *restorePlan) (bool, error) {
s.sweeper.sweep()
next := plan.pickNextDownload()
if next.IsZero() {
return false, nil
}
for _, hash := range plan.blobsNeeded(next) {
blob, ok := s.blobByHash[hash]
if !ok {
return false, fmt.Errorf("%w: %s", errBlobMissingFromIndex, hash[:16])
}
err := s.downloadBlobToCache(hash, blob.CompressedSize)
if err != nil {
return false, fmt.Errorf("downloading blob %s: %w", hash[:16], err)
}
s.result.BlobsDownloaded++
s.result.BytesDownloaded += blob.CompressedSize
plan.markBlobCached(hash)
}
return true, nil
}
// indexRestoreFiles indexes files by ID for plan lookups and sums the
// expected byte total for percentage / ETA arithmetic.
func indexRestoreFiles(
files []*database.File,
) (map[types.FileID]*database.File, int64) {
filesByID := make(map[types.FileID]*database.File, len(files))
var totalBytesExpected int64
for _, f := range files {
filesByID[f.ID] = f
totalBytesExpected += f.Size
}
return filesByID, totalBytesExpected
}
// buildBlobIndexes pre-fetches every blob row once so chunk extraction
// can map a blob_id to its hash without a DB round-trip per chunk.
func (v *Vaultik) buildBlobIndexes(
repos *database.Repositories,
) (map[string]*database.Blob, map[string]string, error) {
blobsByID, err := repos.Blobs.GetAll(v.ctx)
if err != nil {
return nil, nil, fmt.Errorf("fetching blob index: %w", err)
}
blobIDToHash := make(map[string]string, len(blobsByID))
blobByHash := make(map[string]*database.Blob, len(blobsByID))
for id, blob := range blobsByID {
hash := blob.Hash.String()
blobIDToHash[id] = hash
blobByHash[hash] = blob
}
return blobByHash, blobIDToHash, nil
}
// restoreProgressTick emits the periodic UI status line and structured
// progress log during the restore loop.
func (v *Vaultik) restoreProgressTick(
processed, totalFiles int, bytesRestored, totalBytesExpected int64,
startTime time.Time, lastStatusTime *time.Time,
) {
if time.Since(*lastStatusTime) >= restoreStatusInterval {
v.printRestoreProgress(
processed, totalFiles, bytesRestored,
totalBytesExpected, startTime)
*lastStatusTime = time.Now()
}
// Structured progress log for --verbose / JSON consumers.
if processed%progressLogEvery == 0 || processed == totalFiles {
log.Info("Restore progress",
"files", fmt.Sprintf("%d/%d", processed, totalFiles),
"bytes", ubytes(bytesRestored),
)
}
}
// handleRestoreFileError records a per-file restore failure: fatal unless
// --skip-errors is set, in which case the file is counted as failed and
// dropped from the plan.
func (v *Vaultik) handleRestoreFileError(
plan *restorePlan, opts *RestoreOptions, result *RestoreResult,
file *database.File, fileID types.FileID, err error,
) error {
log.Error("Failed to restore file", "path", file.Path, "error", err)
if !opts.SkipErrors {
return fmt.Errorf(
"restoring %s: %w (pass --skip-errors to continue past "+
"restore failures)", file.Path, err)
}
v.UI.Errorf("Failed to restore %s: %v. Skipping (--skip-errors).",
v.UI.Path(file.Path.String()), err)
result.FilesFailed++
result.FailedFiles = append(result.FailedFiles, file.Path.String())
plan.finishFile(fileID)
return nil
}
// printRestoreProgress emits a periodic restore-phase status line via
// the UI writer, mirroring scanner.printProcessingProgress so the two
// long-running commands have the same on-screen rhythm.
func (v *Vaultik) printRestoreProgress(filesDone, totalFiles int, bytesDone, totalBytes int64, startTime time.Time) {
func (v *Vaultik) printRestoreProgress(
filesDone, totalFiles int, bytesDone, totalBytes int64, startTime time.Time,
) {
v.printPhaseProgress("Restore", "restore",
filesDone, totalFiles, bytesDone, totalBytes, startTime)
}
// printPhaseProgress emits a periodic status line for a long-running
// phase (restore or verify) so user-facing pacing is uniform.
func (v *Vaultik) printPhaseProgress(
title, phase string,
filesDone, totalFiles int, bytesDone, totalBytes int64, startTime time.Time,
) {
elapsed := time.Since(startTime)
pct := float64(bytesDone) / float64(totalBytes) * 100
pct := float64(bytesDone) / float64(totalBytes) * percentScale
byteRate := float64(bytesDone) / elapsed.Seconds()
fileRate := float64(filesDone) / elapsed.Seconds()
@@ -382,7 +513,9 @@ func (v *Vaultik) printRestoreProgress(filesDone, totalFiles int, bytesDone, tot
}
if eta > 0 {
v.UI.Progress("Restore: %s/%s files (%s), %s/%s, %s, %.0f files/sec, restore elapsed: %s, restore ETA: %s (est remain %s).",
v.UI.Progressf("%s: %s/%s files (%s), %s/%s, %s, %.0f files/sec, "+
"%s elapsed: %s, %s ETA: %s (est remain %s).",
title,
v.UI.Count(filesDone),
v.UI.Count(totalFiles),
v.UI.Percent(pct),
@@ -390,14 +523,18 @@ func (v *Vaultik) printRestoreProgress(filesDone, totalFiles int, bytesDone, tot
v.UI.Size(totalBytes),
v.UI.Speed(byteRate),
fileRate,
phase,
v.UI.Duration(elapsed),
phase,
v.UI.Time(time.Now().Add(eta)),
v.UI.Duration(eta))
return
}
v.UI.Progress("Restore: %s/%s files (%s), %s/%s, %s, %.0f files/sec, restore elapsed: %s.",
v.UI.Progressf("%s: %s/%s files (%s), %s/%s, %s, %.0f files/sec, "+
"%s elapsed: %s.",
title,
v.UI.Count(filesDone),
v.UI.Count(totalFiles),
v.UI.Percent(pct),
@@ -405,6 +542,7 @@ func (v *Vaultik) printRestoreProgress(filesDone, totalFiles int, bytesDone, tot
v.UI.Size(totalBytes),
v.UI.Speed(byteRate),
fileRate,
phase,
v.UI.Duration(elapsed))
}
@@ -421,17 +559,17 @@ func (v *Vaultik) handleRestoreVerification(
}
if result.FilesFailed > 0 {
v.UI.Error("Verification failed: %s files did not match expected checksums.",
v.UI.Errorf("Verification failed: %s files did not match expected checksums.",
v.UI.Count(result.FilesFailed))
for _, path := range result.FailedFiles {
v.UI.Detail("%s", v.UI.Path(path))
v.UI.Detailf("%s", v.UI.Path(path))
}
return fmt.Errorf("%d files failed verification", result.FilesFailed)
return fmt.Errorf("%d %w", result.FilesFailed, errFilesFailedVerify)
}
v.UI.Complete("Verified %s files (%s).",
v.UI.Completef("Verified %s files (%s).",
v.UI.Count(result.FilesVerified),
v.UI.Size(result.BytesVerified))
@@ -441,9 +579,12 @@ func (v *Vaultik) handleRestoreVerification(
// downloadSnapshotDB downloads and decrypts the snapshot metadata
// database. The snapshotID is the human ID; we hash it to the remote
// key for the storage path.
func (v *Vaultik) downloadSnapshotDB(snapshotID string, identity age.Identity) (*database.DB, error) {
func (v *Vaultik) downloadSnapshotDB(
snapshotID string, identity age.Identity,
) (*database.DB, error) {
// Download encrypted database from storage
dbKey := fmt.Sprintf("metadata/%s/db.zst.age", snapshot.RemoteSnapshotKey(snapshotID))
dbKey := fmt.Sprintf("metadata/%s/db.zst.age",
snapshot.RemoteSnapshotKey(snapshotID))
reader, err := v.Storage.Get(v.ctx, dbKey)
if err != nil {
@@ -457,7 +598,8 @@ func (v *Vaultik) downloadSnapshotDB(snapshotID string, identity age.Identity) (
return nil, fmt.Errorf("reading encrypted data: %w", err)
}
log.Debug("Downloaded encrypted database", "size", humanize.Bytes(uint64(len(encryptedData))))
log.Debug("Downloaded encrypted database",
"size", ubytes(int64(len(encryptedData))))
// Decrypt and decompress using blobgen.Reader
blobReader, err := blobgen.NewReader(bytes.NewReader(encryptedData), identity)
@@ -472,7 +614,7 @@ func (v *Vaultik) downloadSnapshotDB(snapshotID string, identity age.Identity) (
return nil, fmt.Errorf("decrypting and decompressing: %w", err)
}
log.Debug("Decrypted database", "size", humanize.Bytes(uint64(len(dbData))))
log.Debug("Decrypted database", "size", ubytes(int64(len(dbData))))
// Create a temporary database file and write the binary SQLite data directly
tempFile, err := afero.TempFile(v.Fs, "", "vaultik-restore-*.db")
@@ -510,7 +652,9 @@ func (v *Vaultik) downloadSnapshotDB(snapshotID string, identity age.Identity) (
}
// getFilesToRestore returns the list of files to restore based on path filters
func (v *Vaultik) getFilesToRestore(ctx context.Context, repos *database.Repositories, pathFilters []string) ([]*database.File, error) {
func (v *Vaultik) getFilesToRestore(
ctx context.Context, repos *database.Repositories, pathFilters []string,
) ([]*database.File, error) {
// If no filters, get all files
if len(pathFilters) == 0 {
return repos.Files.ListAll(ctx)
@@ -543,7 +687,9 @@ func (v *Vaultik) getFilesToRestore(ctx context.Context, repos *database.Reposit
}
// buildChunkToBlobMap creates a mapping from chunk hash to blob information
func (v *Vaultik) buildChunkToBlobMap(ctx context.Context, repos *database.Repositories) (map[string]*database.BlobChunk, error) {
func (v *Vaultik) buildChunkToBlobMap(
ctx context.Context, repos *database.Repositories,
) (map[string]*database.BlobChunk, error) {
// Query all blob_chunks
query := `SELECT blob_id, chunk_hash, offset, length FROM blob_chunks`
@@ -588,7 +734,7 @@ func (v *Vaultik) buildChunkToBlobMap(ctx context.Context, repos *database.Repos
// readable: restoreFile(file) instead of a ten-argument helper.
type restoreSession struct {
v *Vaultik
ctx context.Context
ctx context.Context //nolint:containedctx // per-restore state by design
repos *database.Repositories
opts *RestoreOptions
identity age.Identity
@@ -613,7 +759,7 @@ func (s *restoreSession) restoreFile(file *database.File) error {
parentDir := filepath.Dir(targetPath)
err := s.v.Fs.MkdirAll(parentDir, 0755)
err := s.v.Fs.MkdirAll(parentDir, restoreDirMode)
if err != nil {
return fmt.Errorf("creating parent directory: %w", err)
}
@@ -640,7 +786,8 @@ func (s *restoreSession) restoreSymlink(file *database.File, targetPath string)
return fmt.Errorf("creating symlink: %w", err)
}
} else {
log.Debug("Symlink creation not supported on this filesystem", "path", file.Path, "target", file.LinkTarget)
log.Debug("Symlink creation not supported on this filesystem",
"path", file.Path, "target", file.LinkTarget)
}
s.result.FilesRestored++
@@ -652,34 +799,51 @@ func (s *restoreSession) restoreSymlink(file *database.File, targetPath string)
// restoreDirectory restores a directory with its permissions, mtime,
// and (on real filesystems, with sufficient privileges) ownership.
func (s *restoreSession) restoreDirectory(file *database.File, targetPath string) error {
func (s *restoreSession) restoreDirectory(
file *database.File, targetPath string,
) error {
err := s.v.Fs.MkdirAll(targetPath, os.FileMode(file.Mode))
if err != nil {
return fmt.Errorf("creating directory: %w", err)
}
err = s.v.Fs.Chmod(targetPath, os.FileMode(file.Mode))
s.applyFileMetadata(file, targetPath)
s.result.FilesRestored++
return nil
}
// applyFileMetadata applies stored permissions, ownership (when running
// as root on a real filesystem), and mtime to a restored path. Failures
// are logged at debug level and do not abort the restore.
func (s *restoreSession) applyFileMetadata(file *database.File, targetPath string) {
err := s.v.Fs.Chmod(targetPath, os.FileMode(file.Mode))
if err != nil {
log.Debug("Failed to set directory permissions", "path", targetPath, "error", err)
log.Debug("Failed to set permissions", "path", targetPath, "error", err)
}
if s.runningAsRoot {
if _, ok := s.v.Fs.(*afero.OsFs); ok {
err := os.Chown(targetPath, int(file.UID), int(file.GID))
err = os.Chown(targetPath, int(file.UID), int(file.GID))
if err != nil {
log.Debug("Failed to set directory ownership", "path", targetPath, "error", err)
log.Debug("Failed to set ownership", "path", targetPath, "error", err)
}
}
}
err = s.v.Fs.Chtimes(targetPath, file.MTime, file.MTime)
if err != nil {
log.Debug("Failed to set directory mtime", "path", targetPath, "error", err)
log.Debug("Failed to set mtime", "path", targetPath, "error", err)
}
}
s.result.FilesRestored++
return nil
// chunkWriteTimings accumulates per-phase durations while writing a
// file's chunks out of the blob cache. Debug instrumentation only.
type chunkWriteTimings struct {
readAt time.Duration
write time.Duration
sweeper time.Duration
}
// restoreRegularFile reconstructs a regular file by reading chunks
@@ -687,7 +851,9 @@ func (s *restoreSession) restoreDirectory(file *database.File, targetPath string
// method runs is that every blob this file needs is already in the
// disk cache — the planner guarantees that by only marking files
// "ready" once their full blob set is on disk.
func (s *restoreSession) restoreRegularFile(file *database.File, targetPath string) error {
func (s *restoreSession) restoreRegularFile(
file *database.File, targetPath string,
) error {
fileStart := time.Now()
t0 := time.Now()
@@ -709,49 +875,9 @@ func (s *restoreSession) restoreRegularFile(file *database.File, targetPath stri
defer func() { _ = outFile.Close() }()
var (
readAtDur time.Duration
writeDur time.Duration
sweeperDur time.Duration
bytesWritten int64
)
for _, fc := range fileChunks {
chunkHashStr := fc.ChunkHash.String()
blobChunk, ok := s.chunkToBlobMap[chunkHashStr]
if !ok {
return fmt.Errorf("chunk %s not found in any blob", chunkHashStr[:16])
}
blobHash, ok := s.blobIDToHash[blobChunk.BlobID.String()]
if !ok {
return fmt.Errorf("blob id %s missing from hash index", blobChunk.BlobID)
}
t0 = time.Now()
chunkData, err := s.blobCache.ReadAt(blobHash, blobChunk.Offset, blobChunk.Length)
readAtDur += time.Since(t0)
if err != nil {
return fmt.Errorf("reading chunk %s from cached blob %s: %w", fc.ChunkHash[:16], blobHash[:16], err)
}
t0 = time.Now()
n, err := outFile.Write(chunkData)
writeDur += time.Since(t0)
if err != nil {
return fmt.Errorf("writing chunk: %w", err)
}
bytesWritten += int64(n)
t0 = time.Now()
s.sweeper.chunkRestored(int64(n))
sweeperDur += time.Since(t0)
bytesWritten, timings, err := s.writeFileChunks(outFile, fileChunks)
if err != nil {
return err
}
log.Debug("Restored regular file (timings)",
@@ -761,9 +887,9 @@ func (s *restoreSession) restoreRegularFile(file *database.File, targetPath stri
"ms_total", time.Since(fileStart).Milliseconds(),
"ms_file_chunks_query", fileChunksQueryDur.Milliseconds(),
"ms_create", createDur.Milliseconds(),
"ms_readat", readAtDur.Milliseconds(),
"ms_writes", writeDur.Milliseconds(),
"ms_sweeper", sweeperDur.Milliseconds(),
"ms_readat", timings.readAt.Milliseconds(),
"ms_writes", timings.write.Milliseconds(),
"ms_sweeper", timings.sweeper.Milliseconds(),
)
err = outFile.Close()
@@ -771,40 +897,82 @@ func (s *restoreSession) restoreRegularFile(file *database.File, targetPath stri
return fmt.Errorf("closing output file: %w", err)
}
err = s.v.Fs.Chmod(targetPath, os.FileMode(file.Mode))
if err != nil {
log.Debug("Failed to set file permissions", "path", targetPath, "error", err)
}
if s.runningAsRoot {
if _, ok := s.v.Fs.(*afero.OsFs); ok {
err := os.Chown(targetPath, int(file.UID), int(file.GID))
if err != nil {
log.Debug("Failed to set file ownership", "path", targetPath, "error", err)
}
}
}
err = s.v.Fs.Chtimes(targetPath, file.MTime, file.MTime)
if err != nil {
log.Debug("Failed to set file mtime", "path", targetPath, "error", err)
}
s.applyFileMetadata(file, targetPath)
s.result.FilesRestored++
s.result.BytesRestored += bytesWritten
log.Debug("Restored file", "path", file.Path, "size", humanize.Bytes(uint64(bytesWritten)))
log.Debug("Restored file", "path", file.Path, "size", ubytes(bytesWritten))
return nil
}
// writeFileChunks streams each of the file's chunks from the blob disk
// cache into outFile, crediting restored bytes to the sweeper as it
// goes. Returns the bytes written plus per-phase timing accumulators.
func (s *restoreSession) writeFileChunks(
outFile afero.File, fileChunks []*database.FileChunk,
) (int64, chunkWriteTimings, error) {
var (
timings chunkWriteTimings
bytesWritten int64
)
for _, fc := range fileChunks {
chunkHashStr := fc.ChunkHash.String()
blobChunk, ok := s.chunkToBlobMap[chunkHashStr]
if !ok {
return bytesWritten, timings, fmt.Errorf(
"%w: %s", errChunkNotInAnyBlob, chunkHashStr[:16])
}
blobHash, ok := s.blobIDToHash[blobChunk.BlobID.String()]
if !ok {
return bytesWritten, timings, fmt.Errorf(
"%w: %s", errBlobIDNotInHashIndex, blobChunk.BlobID)
}
t0 := time.Now()
chunkData, err := s.blobCache.ReadAt(
blobHash, blobChunk.Offset, blobChunk.Length)
timings.readAt += time.Since(t0)
if err != nil {
return bytesWritten, timings, fmt.Errorf(
"reading chunk %s from cached blob %s: %w",
fc.ChunkHash[:16], blobHash[:16], err)
}
t0 = time.Now()
n, err := outFile.Write(chunkData)
timings.write += time.Since(t0)
if err != nil {
return bytesWritten, timings, fmt.Errorf("writing chunk: %w", err)
}
bytesWritten += int64(n)
t0 = time.Now()
s.sweeper.chunkRestored(int64(n))
timings.sweeper += time.Since(t0)
}
return bytesWritten, timings, nil
}
// downloadBlobToCache streams a blob from remote storage straight into
// the disk cache, decrypting and decompressing on the fly. The
// plaintext never lives fully in memory — io.Copy through
// blobDiskCache.PutFromReader uses a 32 KiB buffer regardless of blob
// size, which is what makes multi-GB blobs tractable on machines with
// less RAM than the blob.
func (s *restoreSession) downloadBlobToCache(blobHash string, expectedSize int64) error {
func (s *restoreSession) downloadBlobToCache(
blobHash string, expectedSize int64,
) error {
start := time.Now()
t0 := time.Now()
@@ -840,7 +1008,8 @@ func (s *restoreSession) downloadBlobToCache(blobHash string, expectedSize int64
return nil
}
// verifyRestoredFiles verifies that all restored files match their expected chunk hashes
// verifyRestoredFiles verifies that all restored files match their
// expected chunk hashes.
func (v *Vaultik) verifyRestoredFiles(
ctx context.Context,
repos *database.Repositories,
@@ -870,17 +1039,15 @@ func (v *Vaultik) verifyRestoredFiles(
log.Info("Verifying restored files",
"files", len(regularFiles),
"bytes", humanize.Bytes(uint64(totalBytes)),
"bytes", ubytes(totalBytes),
)
v.UI.Begin("Verifying %s files (%s).",
v.UI.Beginf("Verifying %s files (%s).",
v.UI.Count(len(regularFiles)),
v.UI.Size(totalBytes))
startTime := time.Now()
lastStatusTime := startTime
const statusInterval = 15 * time.Second
var bytesProcessed int64
for i, file := range regularFiles {
@@ -903,8 +1070,9 @@ func (v *Vaultik) verifyRestoredFiles(
bytesProcessed += file.Size
if time.Since(lastStatusTime) >= statusInterval {
v.printVerifyProgress(i+1, len(regularFiles), bytesProcessed, totalBytes, startTime)
if time.Since(lastStatusTime) >= restoreStatusInterval {
v.printVerifyProgress(
i+1, len(regularFiles), bytesProcessed, totalBytes, startTime)
lastStatusTime = time.Now()
}
@@ -912,7 +1080,7 @@ func (v *Vaultik) verifyRestoredFiles(
log.Info("Verification complete",
"files_verified", result.FilesVerified,
"bytes_verified", humanize.Bytes(uint64(result.BytesVerified)),
"bytes_verified", ubytes(result.BytesVerified),
"files_failed", result.FilesFailed,
)
@@ -922,44 +1090,11 @@ func (v *Vaultik) verifyRestoredFiles(
// printVerifyProgress emits a periodic verify-phase status line. Same
// shape as the restore progress line so user-facing pacing is uniform
// across the two phases.
func (v *Vaultik) printVerifyProgress(filesDone, totalFiles int, bytesDone, totalBytes int64, startTime time.Time) {
elapsed := time.Since(startTime)
pct := float64(bytesDone) / float64(totalBytes) * 100
byteRate := float64(bytesDone) / elapsed.Seconds()
fileRate := float64(filesDone) / elapsed.Seconds()
remainingBytes := totalBytes - bytesDone
var eta time.Duration
if byteRate > 0 && remainingBytes > 0 {
eta = time.Duration(float64(remainingBytes)/byteRate) * time.Second
}
if eta > 0 {
v.UI.Progress("Verify: %s/%s files (%s), %s/%s, %s, %.0f files/sec, verify elapsed: %s, verify ETA: %s (est remain %s).",
v.UI.Count(filesDone),
v.UI.Count(totalFiles),
v.UI.Percent(pct),
v.UI.Size(bytesDone),
v.UI.Size(totalBytes),
v.UI.Speed(byteRate),
fileRate,
v.UI.Duration(elapsed),
v.UI.Time(time.Now().Add(eta)),
v.UI.Duration(eta))
return
}
v.UI.Progress("Verify: %s/%s files (%s), %s/%s, %s, %.0f files/sec, verify elapsed: %s.",
v.UI.Count(filesDone),
v.UI.Count(totalFiles),
v.UI.Percent(pct),
v.UI.Size(bytesDone),
v.UI.Size(totalBytes),
v.UI.Speed(byteRate),
fileRate,
v.UI.Duration(elapsed))
func (v *Vaultik) printVerifyProgress(
filesDone, totalFiles int, bytesDone, totalBytes int64, startTime time.Time,
) {
v.printPhaseProgress("Verify", "verify",
filesDone, totalFiles, bytesDone, totalBytes, startTime)
}
// verifyFile verifies a single restored file by checking its chunk hashes
@@ -989,7 +1124,8 @@ func (v *Vaultik) verifyFile(
// Get chunk size from database
chunk, err := repos.Chunks.GetByHash(ctx, fc.ChunkHash.String())
if err != nil {
return bytesVerified, fmt.Errorf("getting chunk %s: %w", fc.ChunkHash.String()[:16], err)
return bytesVerified, fmt.Errorf("getting chunk %s: %w",
fc.ChunkHash.String()[:16], err)
}
// Read chunk data from file
@@ -1001,7 +1137,8 @@ func (v *Vaultik) verifyFile(
}
if int64(n) != chunk.Size {
return bytesVerified, fmt.Errorf("short read: expected %d bytes, got %d", chunk.Size, n)
return bytesVerified, fmt.Errorf("%w: expected %d bytes, got %d",
errShortChunkRead, chunk.Size, n)
}
// Calculate hash and compare
@@ -1010,14 +1147,15 @@ func (v *Vaultik) verifyFile(
expectedHash := fc.ChunkHash.String()
if actualHash != expectedHash {
return bytesVerified, fmt.Errorf("chunk %d hash mismatch: expected %s, got %s",
fc.Idx, expectedHash[:16], actualHash[:16])
return bytesVerified, fmt.Errorf("%w: chunk %d: expected %s, got %s",
errChunkHashMismatch, fc.Idx, expectedHash[:16], actualHash[:16])
}
bytesVerified += int64(n)
}
log.Debug("File verified", "path", file.Path, "bytes", bytesVerified, "chunks", len(fileChunks))
log.Debug("File verified",
"path", file.Path, "bytes", bytesVerified, "chunks", len(fileChunks))
return bytesVerified, nil
}

173
internal/vaultik/restore_locality_test.go
View File

@@ -1,4 +1,4 @@
package vaultik
package vaultik //nolint:testpackage // inspects unexported cache internals
import (
"bytes"
@@ -50,33 +50,28 @@ import (
// path-ordered names that interleave the blobs (a1, b1, c1, a2, b2,
// c2, …) so naive path-order processing would touch every blob before
// finishing any of them.
func TestRestoreLocalityAndReadAt(t *testing.T) {
log.Initialize(log.Config{})
// localitySource is one 1 MiB fixture file used by the locality test.
type localitySource struct {
path string
data []byte
}
fs := afero.NewOsFs()
tempDir, err := os.MkdirTemp("", "vaultik-locality-")
require.NoError(t, err)
// localityCopy is a byte-for-byte clone of one fixture source with an
// interleaved name.
type localityCopy struct {
path string
data []byte
}
defer func() { _ = os.RemoveAll(tempDir) }()
// buildLocalityFixture writes the adversarial source layout described
// in TestRestoreLocalityAndReadAt: 15 sources of 1 MiB each (which the
// backup packs into 3 blobs of 5 chunks) plus 9 interleaved-name copies
// (3 per blob group).
func buildLocalityFixture(
t *testing.T, fs afero.Fs, dataDir string,
) ([]*localitySource, []localityCopy) {
t.Helper()
dataDir := filepath.Join(tempDir, "source")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
require.NoError(t, fs.MkdirAll(dataDir, 0o755))
// Layout: 15 source files of exactly 1 MiB each. With
// chunkSize (avg) = 4 MiB the chunker's minSize is 1 MiB, so any
// file of 1 MiB becomes a single chunk. With a 5 MiB blob limit
// the packer fits exactly 5 chunks per blob, producing 3 blobs
// containing src-001..005, src-006..010, src-011..015.
//
// Then add 9 "copy" files — byte-for-byte clones of three of the
// sources (one from each blob group) — with interleaved names
// (cp-001-A, cp-002-B, cp-003-C, cp-004-A, …) so a naive
// path-ordered restore would touch all three blobs before
// finishing any of them.
const (
srcBytes = 1024 * 1024
srcCount = 15
@@ -84,35 +79,24 @@ func TestRestoreLocalityAndReadAt(t *testing.T) {
perBlob = srcCount / blobsCount
)
type source struct {
path string
data []byte
}
sources := make([]*source, srcCount)
sources := make([]*localitySource, srcCount)
for i := range srcCount {
s := &source{
s := &localitySource{
path: fmt.Sprintf("src-%03d.bin", i+1),
data: randomBytes(t, srcBytes),
}
sources[i] = s
require.NoError(t, afero.WriteFile(fs, filepath.Join(dataDir, s.path), s.data, 0o644))
require.NoError(t, afero.WriteFile(
fs, filepath.Join(dataDir, s.path), s.data, 0o644))
}
// Pick one representative source per blob group (src-001 → blob
// 1, src-006 → blob 2, src-011 → blob 3) and create 3 copies of
// each with interleaved alphabetical names.
type copyFile struct {
path string
data []byte
sourceBlob int // 0, 1, or 2
sourceIndex int // index into sources slice
}
groupReps := []int{0, perBlob, 2 * perBlob} // 0, 5, 10
letters := []byte{'A', 'B', 'C'}
var copies []copyFile
copies := make([]localityCopy, 0, blobsCount*3)
for i := range 3 {
for j := range blobsCount {
@@ -121,10 +105,24 @@ func TestRestoreLocalityAndReadAt(t *testing.T) {
path := filepath.Join(dataDir, name)
src := sources[groupReps[j]]
require.NoError(t, afero.WriteFile(fs, path, src.data, 0o644))
copies = append(copies, copyFile{path: path, data: src.data, sourceBlob: j, sourceIndex: groupReps[j]})
copies = append(copies, localityCopy{path: path, data: src.data})
}
}
return sources, copies
}
// setupLocalityBackup runs the backup half of the locality test: a
// snapshot of dataDir into a file storer with sizes tuned so 15 one-chunk
// files pack into 3 blobs. The index database is closed before returning
// so the restore half runs from remote bytes only.
func setupLocalityBackup(
ctx context.Context, t *testing.T, fs afero.Fs,
dataDir, storeDir, dbPath string,
) (*config.Config, *storage.FileStorer, string) {
t.Helper()
// chunkSize avg = 4 MiB makes minSize = 1 MiB, so a 1 MiB file
// becomes one chunk. maxBlobSize = 5 MiB packs exactly 5 chunks
// per blob, yielding 3 blobs from 15 source files.
@@ -134,8 +132,10 @@ func TestRestoreLocalityAndReadAt(t *testing.T) {
storer, err := storage.NewFileStorer(storeDir)
require.NoError(t, err)
agePublicKey := "age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05gl0sjq9q9wjg"
ageSecretKey := "AGE-SECRET-KEY-19CR5YSFW59HM4TLD6GXVEDMZFTVVF7PPHKUT68TXSFPK7APHXA2QS2NJA5"
agePublicKey := "age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05g" +
"l0sjq9q9wjg"
ageSecretKey := "AGE-SECRET-KEY-19CR5YSFW59HM4TLD6GXVEDMZFTVVF7PPHKU" +
"T68TXSFPK7APHXA2QS2NJA5"
cfg := &config.Config{
AgeRecipients: []string{agePublicKey},
@@ -145,8 +145,6 @@ func TestRestoreLocalityAndReadAt(t *testing.T) {
BlobSizeLimit: config.Size(maxBlobSize),
}
ctx := context.Background()
db, err := database.New(ctx, dbPath)
require.NoError(t, err)
@@ -171,7 +169,8 @@ func TestRestoreLocalityAndReadAt(t *testing.T) {
Repositories: repos,
})
snapshotID, err := sm.CreateSnapshotWithName(ctx, cfg.Hostname, "locality", "test-version", "test-git")
snapshotID, err := sm.CreateSnapshotWithName(
ctx, cfg.Hostname, "locality", "test-version", "test-git")
require.NoError(t, err)
_, err = scanner.Scan(ctx, dataDir, snapshotID)
@@ -182,10 +181,35 @@ func TestRestoreLocalityAndReadAt(t *testing.T) {
blobsOnDisk := listBlobKeys(t, storeDir)
t.Logf("backup produced %d blobs", len(blobsOnDisk))
require.GreaterOrEqual(t, len(blobsOnDisk), 3, "expected at least 3 blobs from 3 filler groups")
require.GreaterOrEqual(t, len(blobsOnDisk), 3,
"expected at least 3 blobs from 3 filler groups")
require.NoError(t, db.Close())
return cfg, storer, snapshotID
}
func TestRestoreLocalityAndReadAt(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
dataDir := filepath.Join(tempDir, "source")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
require.NoError(t, fs.MkdirAll(dataDir, 0o755))
sources, copies := buildLocalityFixture(t, fs, dataDir)
ctx := context.Background()
cfg, storer, snapshotID := setupLocalityBackup(
ctx, t, fs, dataDir, storeDir, dbPath)
// Wrap the storer so we can count downloads per blob key.
counter := newCountingStorer(storer)
@@ -215,20 +239,7 @@ func TestRestoreLocalityAndReadAt(t *testing.T) {
require.NotNil(t, cacheRef, "restoreCacheObserver must fire during restore")
// Verify restored content matches.
for _, s := range sources {
restored := filepath.Join(restoreDir, dataDir, s.path)
got, err := afero.ReadFile(fs, restored)
require.NoErrorf(t, err, "source missing after restore: %s", s.path)
require.Truef(t, bytes.Equal(got, s.data), "byte mismatch for source %s", s.path)
}
for _, c := range copies {
restored := filepath.Join(restoreDir, c.path)
got, err := afero.ReadFile(fs, restored)
require.NoErrorf(t, err, "copy missing after restore: %s", c.path)
require.Truef(t, bytes.Equal(got, c.data), "byte mismatch for copy %s", c.path)
}
verifyLocalityRestore(t, fs, restoreDir, dataDir, sources, copies)
// (1) Each blob fetched exactly once.
for key, n := range counter.snapshot() {
@@ -242,18 +253,48 @@ func TestRestoreLocalityAndReadAt(t *testing.T) {
// (2) Peak cache size ≤ 1. The sweeper plus locality-aware
// ordering should free each blob before the next one downloads.
assert.LessOrEqualf(t, cacheRef.PeakLen(), 1,
"peak cached blobs was %d; expected ≤ 1 with locality-ordered restore", cacheRef.PeakLen())
"peak cached blobs was %d; expected ≤ 1 with locality-ordered restore",
cacheRef.PeakLen())
// (3) Cache.Get must never be called during restore — chunk
// extraction has to go through ReadAt so we never read the whole
// blob from disk to grab a few KB slice.
assert.Equalf(t, 0, cacheRef.GetCalls(),
"blobDiskCache.Get was called %d times during restore; restore must use ReadAt exclusively", cacheRef.GetCalls())
"blobDiskCache.Get was called %d times during restore; "+
"restore must use ReadAt exclusively", cacheRef.GetCalls())
t.Logf("blob cache stats: peak_len=%d get_calls=%d readat_calls=%d",
cacheRef.PeakLen(), cacheRef.GetCalls(), cacheRef.ReadAtCalls())
}
// verifyLocalityRestore byte-compares every restored source and copy
// against its original content.
func verifyLocalityRestore(
t *testing.T,
fs afero.Fs,
restoreDir, dataDir string,
sources []*localitySource,
copies []localityCopy,
) {
t.Helper()
for _, s := range sources {
restored := filepath.Join(restoreDir, dataDir, s.path)
got, err := afero.ReadFile(fs, restored)
require.NoErrorf(t, err, "source missing after restore: %s", s.path)
require.Truef(t, bytes.Equal(got, s.data),
"byte mismatch for source %s", s.path)
}
for _, c := range copies {
restored := filepath.Join(restoreDir, c.path)
got, err := afero.ReadFile(fs, restored)
require.NoErrorf(t, err, "copy missing after restore: %s", c.path)
require.Truef(t, bytes.Equal(got, c.data),
"byte mismatch for copy %s", c.path)
}
}
// randomBytes returns n bytes of random data. Used to make sure the
// chunker picks non-degenerate FastCDC boundaries.
func randomBytes(t *testing.T, n int) []byte {
@@ -316,7 +357,9 @@ func newCountingStorer(inner storage.Storer) *countingStorerInternal {
return &countingStorerInternal{Storer: inner, counts: make(map[string]int)}
}
func (c *countingStorerInternal) Get(ctx context.Context, key string) (io.ReadCloser, error) {
func (c *countingStorerInternal) Get(
ctx context.Context, key string,
) (io.ReadCloser, error) {
c.mu.Lock()
c.counts[key]++
c.mu.Unlock()

15
internal/vaultik/restore_plan.go
View File

@@ -2,6 +2,7 @@ package vaultik
import (
"context"
"errors"
"fmt"
"math"
"os"
@@ -10,6 +11,12 @@ import (
"sneak.berlin/go/vaultik/internal/types"
)
// Sentinel errors for restore planning index lookups.
var (
errPlanChunkMissing = errors.New("chunk missing from blob map")
errPlanBlobIDMissing = errors.New("blob id missing from id-to-hash map")
)
// restorePlan orders restore-time file processing by blob locality. The
// goal is to keep the blob disk cache occupancy as small as possible:
// download one blob, drain every file referencing only that blob, let
@@ -67,14 +74,14 @@ func newRestorePlan(
for _, fc := range fileChunks {
bc, ok := chunkToBlobMap[fc.ChunkHash.String()]
if !ok {
return nil, fmt.Errorf("planning %s: chunk %s missing from blob map",
f.Path, fc.ChunkHash.String()[:16])
return nil, fmt.Errorf("planning %s: %w: %s",
f.Path, errPlanChunkMissing, fc.ChunkHash.String()[:16])
}
hash, ok := blobIDToHash[bc.BlobID.String()]
if !ok {
return nil, fmt.Errorf("planning %s: blob id %s missing from id-to-hash map",
f.Path, bc.BlobID)
return nil, fmt.Errorf("planning %s: %w: %s",
f.Path, errPlanBlobIDMissing, bc.BlobID)
}
blobs[hash] = struct{}{}

12
internal/vaultik/restore_sweeper.go
View File

@@ -28,7 +28,7 @@ import (
// which is local, indexed, and not under contention — the queries are
// cheap and run at most once per blob per sweep interval.
type restoreSweeper struct {
ctx context.Context
ctx context.Context //nolint:containedctx // ctx bound at construction by design
repos *database.Repositories
cache *blobDiskCache
threshold int64
@@ -38,7 +38,12 @@ type restoreSweeper struct {
// newRestoreSweeper returns a sweeper that triggers eviction every
// `threshold` bytes restored. Callers should pass blob_size_limit/100.
func newRestoreSweeper(ctx context.Context, repos *database.Repositories, cache *blobDiskCache, threshold int64) *restoreSweeper {
func newRestoreSweeper(
ctx context.Context,
repos *database.Repositories,
cache *blobDiskCache,
threshold int64,
) *restoreSweeper {
if threshold <= 0 {
threshold = 1
}
@@ -78,7 +83,8 @@ func (s *restoreSweeper) sweep() {
for _, blobHash := range s.cache.Keys() {
needed, err := s.blobStillNeeded(blobHash)
if err != nil {
log.Debug("sweeper referencing-files query failed", "blob_hash", blobHash[:16], "error", err)
log.Debug("sweeper referencing-files query failed",
"blob_hash", blobHash[:16], "error", err)
continue
}

166
internal/vaultik/restore_sweeper_integration_test.go
View File

@@ -40,33 +40,20 @@ import (
// originals; the sweeper must keep each blob alive until BOTH the
// original AND every duplicate referencing its chunks have been
// restored.
func TestRestoreSweeperEvictsBlobs(t *testing.T) {
log.Initialize(log.Config{})
// buildSweeperFixture writes the source layout for the sweeper test: 30
// unique 1 MB random files plus 10 duplicates of a random subset. The
// PRNG seed is fixed so failures are reproducible; the entropy is what
// matters here — the FastCDC chunker needs realistic-looking data to
// pick chunk boundaries naturally. Returns the expected content by
// path.
func buildSweeperFixture(
t *testing.T, fs afero.Fs, dataDir string, uniqueFiles, duplicateFiles int,
) map[string][]byte {
t.Helper()
fs := afero.NewOsFs()
tempDir, err := os.MkdirTemp("", "vaultik-sweeper-")
require.NoError(t, err)
const fileSize = 1 * 1024 * 1024
defer func() { _ = os.RemoveAll(tempDir) }()
dataDir := filepath.Join(tempDir, "source")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
require.NoError(t, fs.MkdirAll(dataDir, 0o755))
// Generate 30 unique 1 MB random files. The PRNG seed is fixed so
// failures are reproducible; the entropy is what matters here — the
// FastCDC chunker needs realistic-looking data to pick chunk
// boundaries naturally.
const (
uniqueFiles = 30
duplicateFiles = 10
fileSize = 1 * 1024 * 1024
)
rng := rand.New(rand.NewSource(42))
rng := rand.New(rand.NewSource(42)) //nolint:gosec // G404: fixture data only
type sourceFile struct {
path string
@@ -87,33 +74,58 @@ func TestRestoreSweeperEvictsBlobs(t *testing.T) {
expected[path] = data
}
// Pick 10 of the originals and copy each to a fresh path so the
// chunker dedups them against the originals' blobs.
// Copy a subset of the originals to fresh paths so the chunker
// dedups them against the originals' blobs.
for i, idx := range rng.Perm(uniqueFiles)[:duplicateFiles] {
src := uniques[idx]
dstPath := filepath.Join(dataDir, fmt.Sprintf("dup-%02d.bin", i))
require.NoError(t, afero.WriteFile(fs, dstPath, src.data, 0o644))
expected[dstPath] = src.data
}
chunkSize := int64(64 * 1024)
maxBlobSize := int64(10 * 1024 * 1024)
return expected
}
storer, err := storage.NewFileStorer(storeDir)
require.NoError(t, err)
// verifySingleDownloadPerBlob asserts each blob on disk was fetched
// exactly once. >1 means the sweeper evicted a still-needed blob; 0
// means the cache silently stopped being consulted.
func verifySingleDownloadPerBlob(
t *testing.T, counter *countingStorer, storeDir string,
) {
t.Helper()
agePublicKey := "age1ezrjmfpwsc95svdg0y54mums3zevgzu0x0ecq2f7tp8a05gl0sjq9q9wjg"
ageSecretKey := "AGE-SECRET-KEY-19CR5YSFW59HM4TLD6GXVEDMZFTVVF7PPHKUT68TXSFPK7APHXA2QS2NJA5"
blobDownloads := 0
cfg := &config.Config{
AgeRecipients: []string{agePublicKey},
AgeSecretKey: ageSecretKey,
CompressionLevel: 3,
Hostname: "test-host",
BlobSizeLimit: config.Size(maxBlobSize),
for key, count := range counter.snapshot() {
if !strings.HasPrefix(key, "blobs/") {
continue
}
assert.Equalf(t, 1, count,
"blob %s should have been downloaded exactly once during "+
"restore, got %d", key, count)
blobDownloads++
}
ctx := context.Background()
blobCount := countBlobsOnDisk(t, storeDir)
assert.Equal(t, blobCount, blobDownloads,
"every blob on disk should have been fetched exactly once during restore")
t.Logf("restore downloaded %d blobs, each exactly once", blobDownloads)
}
// runSweeperBackup performs the backup half of the sweeper test and
// closes the index database so the restore half runs from remote bytes
// only. Returns the snapshot ID.
func runSweeperBackup(
ctx context.Context, t *testing.T, fs afero.Fs,
storer *storage.FileStorer, cfg *config.Config,
dataDir, storeDir, dbPath string,
chunkSize, maxBlobSize int64,
uniqueFiles, duplicateFiles int,
) string {
t.Helper()
db, err := database.New(ctx, dbPath)
require.NoError(t, err)
@@ -139,13 +151,15 @@ func TestRestoreSweeperEvictsBlobs(t *testing.T) {
Repositories: repos,
})
snapshotID, err := sm.CreateSnapshotWithName(ctx, cfg.Hostname, "sweeper", "test-version", "test-git")
snapshotID, err := sm.CreateSnapshotWithName(
ctx, cfg.Hostname, "sweeper", "test-version", "test-git")
require.NoError(t, err)
scanResult, err := scanner.Scan(ctx, dataDir, snapshotID)
require.NoError(t, err)
require.Equal(t, uniqueFiles+duplicateFiles, scanResult.FilesScanned)
require.Greater(t, scanResult.BlobsCreated, 1, "30 MB of unique data at 10 MB blob size should yield multiple blobs")
require.Greater(t, scanResult.BlobsCreated, 1,
"30 MB of unique data at 10 MB blob size should yield multiple blobs")
require.NoError(t, sm.CompleteSnapshot(ctx, snapshotID))
require.NoError(t, sm.ExportSnapshotMetadata(ctx, dbPath, snapshotID))
@@ -161,6 +175,50 @@ func TestRestoreSweeperEvictsBlobs(t *testing.T) {
// as a real restore on a fresh machine would.
require.NoError(t, db.Close())
return snapshotID
}
func TestRestoreSweeperEvictsBlobs(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
dataDir := filepath.Join(tempDir, "source")
storeDir := filepath.Join(tempDir, "remote")
restoreDir := filepath.Join(tempDir, "restored")
dbPath := filepath.Join(tempDir, "index.sqlite")
require.NoError(t, fs.MkdirAll(dataDir, 0o755))
const (
uniqueFiles = 30
duplicateFiles = 10
)
expected := buildSweeperFixture(t, fs, dataDir, uniqueFiles, duplicateFiles)
chunkSize := int64(64 * 1024)
maxBlobSize := int64(10 * 1024 * 1024)
storer, err := storage.NewFileStorer(storeDir)
require.NoError(t, err)
cfg := &config.Config{
AgeRecipients: []string{testAgePublicKey},
AgeSecretKey: testAgeSecretKey,
CompressionLevel: 3,
Hostname: testHostname,
BlobSizeLimit: config.Size(maxBlobSize),
}
ctx := context.Background()
snapshotID := runSweeperBackup(ctx, t, fs, storer, cfg,
dataDir, storeDir, dbPath, chunkSize, maxBlobSize,
uniqueFiles, duplicateFiles)
counter := newCountingStorer(storer)
restoreVaultik := &vaultik.Vaultik{
@@ -186,25 +244,7 @@ func TestRestoreSweeperEvictsBlobs(t *testing.T) {
require.Equalf(t, want, got, "byte mismatch for %s", origPath)
}
// Each blob must have been downloaded exactly once. >1 means the
// sweeper evicted a still-needed blob; 0 means the cache silently
// stopped being consulted.
blobDownloads := 0
for key, count := range counter.snapshot() {
if !strings.HasPrefix(key, "blobs/") {
continue
}
assert.Equalf(t, 1, count,
"blob %s should have been downloaded exactly once during restore, got %d", key, count)
blobDownloads++
}
assert.Equal(t, blobCount, blobDownloads,
"every blob on disk should have been fetched exactly once during restore")
t.Logf("restore downloaded %d blobs, each exactly once", blobDownloads)
verifySingleDownloadPerBlob(t, counter, storeDir)
}
// countingStorer wraps a Storer and records the number of Get calls per
@@ -221,7 +261,9 @@ func newCountingStorer(inner storage.Storer) *countingStorer {
return &countingStorer{Storer: inner, counts: make(map[string]int)}
}
func (c *countingStorer) Get(ctx context.Context, key string) (io.ReadCloser, error) {
func (c *countingStorer) Get(
ctx context.Context, key string,
) (io.ReadCloser, error) {
c.mu.Lock()
c.counts[key]++
c.mu.Unlock()

769
internal/vaultik/snapshot.go
View File

File diff suppressed because it is too large Load Diff

10
internal/vaultik/snapshot_prune_test.go
View File

@@ -1,20 +1,24 @@
package vaultik
package vaultik_test
import (
"testing"
"sneak.berlin/go/vaultik/internal/vaultik"
)
// TestSnapshotCreateOptions_PruneFlag verifies the Prune field exists on
// SnapshotCreateOptions and can be set.
func TestSnapshotCreateOptions_PruneFlag(t *testing.T) {
opts := &SnapshotCreateOptions{
t.Parallel()
opts := &vaultik.SnapshotCreateOptions{
Prune: true,
}
if !opts.Prune {
t.Error("Expected Prune to be true")
}
opts2 := &SnapshotCreateOptions{
opts2 := &vaultik.SnapshotCreateOptions{
Prune: false,
}
if opts2.Prune {

24
internal/vaultik/storage_bind.go
View File

@@ -1,12 +1,18 @@
package vaultik
import (
"errors"
"fmt"
"sneak.berlin/go/vaultik/internal/database"
"sneak.berlin/go/vaultik/internal/log"
)
// errStorageBindingMismatch is returned when the local index database is
// bound to a different destination than the configured storage_url.
var errStorageBindingMismatch = errors.New(
"local index is bound to a different backup destination")
// EnsureStorageBinding guarantees that the local index database is
// bound to the currently-configured storage destination. Every mutating
// command must call this before touching either the local index or the
@@ -54,7 +60,8 @@ func (v *Vaultik) EnsureStorageBinding() error {
}
if stored == "" {
err := v.Repositories.LocalMeta.Set(v.ctx, database.LocalMetaKeyStorageURL, configured)
err = v.Repositories.LocalMeta.Set(
v.ctx, database.LocalMetaKeyStorageURL, configured)
if err != nil {
return fmt.Errorf("recording local storage binding: %w", err)
}
@@ -68,18 +75,19 @@ func (v *Vaultik) EnsureStorageBinding() error {
return nil
}
return fmt.Errorf("%s", buildBindingMismatchMessage(stored, configured))
return fmt.Errorf("%w\n%s",
errStorageBindingMismatch, buildBindingMismatchMessage(stored, configured))
}
// buildBindingMismatchMessage assembles the multi-line explanation
// shown when the local index is bound to a different destination than
// the currently-configured one. Kept as a separate function so the
// lint-flagged multi-line format string is expressed as a plain string
// literal rather than a fmt.Errorf argument (staticcheck ST1005
// disallows trailing punctuation on error format strings).
// the currently-configured one (the first line lives in the
// errStorageBindingMismatch sentinel). Kept as a separate function so
// the multi-line text is expressed as a plain string literal rather
// than a fmt.Errorf argument (staticcheck ST1005 disallows trailing
// punctuation on error format strings).
func buildBindingMismatchMessage(stored, configured string) string {
return "local index is bound to a different backup destination\n" +
" local index bound to: " + stored + "\n" +
return " local index bound to: " + stored + "\n" +
" currently configured: " + configured + "\n" +
"\n" +
"The local index database tracks which chunks and blobs already exist at the\n" +

10
internal/vaultik/storage_bind_test.go
View File

@@ -15,7 +15,9 @@ import (
// buildBindTestVaultik returns a minimal Vaultik wired with a real
// in-memory DB and a config carrying the given StorageURL. Enough to
// exercise EnsureStorageBinding without spinning up storage or fx.
func buildBindTestVaultik(t *testing.T, storageURL string) (*vaultik.TestVaultik, *database.Repositories) {
func buildBindTestVaultik(
t *testing.T, storageURL string,
) (*vaultik.TestVaultik, *database.Repositories) {
t.Helper()
db, err := database.NewTestDB()
@@ -35,6 +37,7 @@ func buildBindTestVaultik(t *testing.T, storageURL string) (*vaultik.TestVaultik
func TestEnsureStorageBinding_FreshDBRecordsURL(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
tv, repos := buildBindTestVaultik(t, "file:///mnt/backups/new")
@@ -42,11 +45,13 @@ func TestEnsureStorageBinding_FreshDBRecordsURL(t *testing.T) {
got, err := repos.LocalMeta.Get(context.Background(), database.LocalMetaKeyStorageURL)
require.NoError(t, err)
assert.Equal(t, "file:///mnt/backups/new", got, "first call must record the configured URL")
assert.Equal(t, "file:///mnt/backups/new", got,
"first call must record the configured URL")
}
func TestEnsureStorageBinding_MatchingURLPasses(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
tv, repos := buildBindTestVaultik(t, "s3://bucket/prefix")
@@ -58,6 +63,7 @@ func TestEnsureStorageBinding_MatchingURLPasses(t *testing.T) {
func TestEnsureStorageBinding_MismatchRefuses(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
tv, repos := buildBindTestVaultik(t, "file:///mnt/backups/new")

26
internal/vaultik/vaultik.go
View File

@@ -1,3 +1,5 @@
// Package vaultik implements the core backup, restore, verify, prune,
// and snapshot-management operations behind the vaultik CLI.
package vaultik
import (
@@ -19,6 +21,12 @@ import (
"sneak.berlin/go/vaultik/internal/ui"
)
// Sentinel errors for misconfigured encryption settings.
var (
errNoAgeRecipients = errors.New("no age recipients configured")
errNoAgeSecretKey = errors.New("no age secret key configured")
)
// Vaultik contains all dependencies needed for vaultik operations
type Vaultik struct {
Globals *globals.Globals
@@ -32,7 +40,7 @@ type Vaultik struct {
Fs afero.Fs
// Context management
ctx context.Context
ctx context.Context //nolint:containedctx // ctx bound at construction by design
cancel context.CancelFunc
// IO
@@ -54,8 +62,8 @@ type Vaultik struct {
restoreCacheObserver func(*blobDiskCache)
}
// VaultikParams contains all parameters for New that can be provided by fx
type VaultikParams struct {
// Params contains all parameters for New that can be provided by fx
type Params struct {
fx.In
Globals *globals.Globals
@@ -71,7 +79,7 @@ type VaultikParams struct {
// New creates a new Vaultik instance with proper context management
// It automatically includes crypto capabilities if age_secret_key is configured
func New(params VaultikParams) *Vaultik {
func New(params Params) *Vaultik {
ctx, cancel := context.WithCancel(context.Background())
// Use provided filesystem or default to OS filesystem
@@ -126,7 +134,7 @@ func (v *Vaultik) CanDecrypt() bool {
// Returns an error if no recipients are configured
func (v *Vaultik) GetEncryptor() (*crypto.Encryptor, error) {
if len(v.Config.AgeRecipients) == 0 {
return nil, errors.New("no age recipients configured")
return nil, errNoAgeRecipients
}
return crypto.NewEncryptor(v.Config.AgeRecipients)
@@ -136,19 +144,21 @@ func (v *Vaultik) GetEncryptor() (*crypto.Encryptor, error) {
// Returns an error if no secret key is configured
func (v *Vaultik) GetDecryptor() (*crypto.Decryptor, error) {
if v.Config.AgeSecretKey == "" {
return nil, errors.New("no age secret key configured")
return nil, errNoAgeSecretKey
}
return crypto.NewDecryptor(v.Config.AgeSecretKey)
}
// GetFilesystem returns the filesystem instance used by Vaultik
//
//nolint:ireturn // afero.Fs is the filesystem abstraction by design
func (v *Vaultik) GetFilesystem() afero.Fs {
return v.Fs
}
// printfStdout writes formatted output to stdout.
func (v *Vaultik) printfStdout(format string, args ...any) {
// stdoutf writes formatted output to stdout.
func (v *Vaultik) stdoutf(format string, args ...any) {
_, _ = fmt.Fprintf(v.Stdout, format, args...)
}

179
internal/vaultik/verify.go
View File

@@ -4,19 +4,37 @@ import (
"crypto/sha256"
"database/sql"
"encoding/hex"
"errors"
"fmt"
"hash"
"io"
"os"
"time"
"github.com/dustin/go-humanize"
"github.com/klauspost/compress/zstd"
// Blank import registers the pure-Go sqlite driver for database/sql.
_ "modernc.org/sqlite"
"sneak.berlin/go/vaultik/internal/log"
"sneak.berlin/go/vaultik/internal/snapshot"
)
// Sentinel errors for snapshot verification failures.
var (
errVerificationFailed = errors.New("verification failed")
errSecretKeyRequired = errors.New(
"VAULTIK_AGE_SECRET_KEY not set; required for deep verification")
errChunksOutOfOrder = errors.New("chunks out of order")
errChunkHashMismatch = errors.New("chunk hash mismatch")
errTrailingBlobData = errors.New(
"blob has unexpected trailing bytes not covered by chunk list")
errManifestExtraBlob = errors.New("manifest contains blob not in database")
errBlobSizeMismatch = errors.New("blob size mismatch")
)
// verifyStatusFailed is the JSON status value for a failed verification.
const verifyStatusFailed = "failed"
// VerifyOptions contains options for the verify command
type VerifyOptions struct {
Deep bool
@@ -24,6 +42,8 @@ type VerifyOptions struct {
}
// VerifyResult contains the result of a snapshot verification
//
//nolint:tagliatelle // snake_case is the established JSON output format
type VerifyResult struct {
SnapshotID string `json:"snapshot_id"`
Status string `json:"status"` // "ok" or "failed"
@@ -37,8 +57,10 @@ type VerifyResult struct {
}
// deepVerifyFailure records a failure in the result and returns it appropriately
func (v *Vaultik) deepVerifyFailure(result *VerifyResult, opts *VerifyOptions, msg string, err error) error {
result.Status = "failed"
func (v *Vaultik) deepVerifyFailure(
result *VerifyResult, opts *VerifyOptions, msg string, err error,
) error {
result.Status = verifyStatusFailed
result.ErrorMessage = msg
if opts.JSON {
@@ -49,7 +71,7 @@ func (v *Vaultik) deepVerifyFailure(result *VerifyResult, opts *VerifyOptions, m
return err
}
return fmt.Errorf("%s", msg)
return fmt.Errorf("%w: %s", errVerificationFailed, msg)
}
// RunDeepVerify executes deep verification operation
@@ -60,15 +82,14 @@ func (v *Vaultik) RunDeepVerify(snapshotID string, opts *VerifyOptions) error {
}
if !v.CanDecrypt() {
msg := "VAULTIK_AGE_SECRET_KEY not set; required for deep verification"
return v.deepVerifyFailure(result, opts, msg, fmt.Errorf("%s", msg))
return v.deepVerifyFailure(result, opts,
errSecretKeyRequired.Error(), errSecretKeyRequired)
}
log.Info("Starting snapshot verification", "snapshot_id", snapshotID, "mode", "deep")
if !opts.JSON {
v.printfStdout("Deep verification of snapshot: %s\n\n", snapshotID)
v.stdoutf("Deep verification of snapshot: %s\n\n", snapshotID)
}
manifest, tempDB, dbBlobs, err := v.loadVerificationData(snapshotID, opts, result)
@@ -104,16 +125,18 @@ func (v *Vaultik) RunDeepVerify(snapshotID string, opts *VerifyOptions) error {
log.Info("✓ Verification completed successfully",
"snapshot_id", snapshotID, "mode", "deep", "blobs_verified", len(dbBlobs))
v.printfStdout("\n✓ Verification completed successfully\n")
v.printfStdout(" Snapshot: %s\n", snapshotID)
v.printfStdout(" Blobs verified: %d\n", len(dbBlobs))
v.printfStdout(" Total size: %s\n", humanize.Bytes(uint64(totalSize)))
v.stdoutf("\n✓ Verification completed successfully\n")
v.stdoutf(" Snapshot: %s\n", snapshotID)
v.stdoutf(" Blobs verified: %d\n", len(dbBlobs))
v.stdoutf(" Total size: %s\n", ubytes(totalSize))
return nil
}
// loadVerificationData downloads manifest, database, and blob list for verification
func (v *Vaultik) loadVerificationData(snapshotID string, opts *VerifyOptions, result *VerifyResult) (*snapshot.Manifest, *tempDB, []snapshot.BlobInfo, error) {
func (v *Vaultik) loadVerificationData(
snapshotID string, opts *VerifyOptions, result *VerifyResult,
) (*snapshot.Manifest, *tempDB, []snapshot.BlobInfo, error) {
// All remote paths use the hashed key derived from the human ID.
remoteKey := snapshot.RemoteSnapshotKey(snapshotID)
@@ -122,7 +145,7 @@ func (v *Vaultik) loadVerificationData(snapshotID string, opts *VerifyOptions, r
log.Info("Downloading manifest", "path", manifestPath)
if !opts.JSON {
v.printfStdout("Downloading manifest...\n")
v.stdoutf("Downloading manifest...\n")
}
manifestReader, err := v.Storage.Get(v.ctx, manifestPath)
@@ -143,11 +166,12 @@ func (v *Vaultik) loadVerificationData(snapshotID string, opts *VerifyOptions, r
log.Info("Manifest loaded",
"manifest_blob_count", manifest.BlobCount,
"manifest_total_size", humanize.Bytes(uint64(manifest.TotalCompressedSize)))
"manifest_total_size", ubytes(manifest.TotalCompressedSize))
if !opts.JSON {
v.printfStdout("Manifest loaded: %d blobs (%s)\n", manifest.BlobCount, humanize.Bytes(uint64(manifest.TotalCompressedSize)))
v.printfStdout("Downloading and decrypting database...\n")
v.stdoutf("Manifest loaded: %d blobs (%s)\n",
manifest.BlobCount, ubytes(manifest.TotalCompressedSize))
v.stdoutf("Downloading and decrypting database...\n")
}
// Download and decrypt database
@@ -163,7 +187,7 @@ func (v *Vaultik) loadVerificationData(snapshotID string, opts *VerifyOptions, r
defer func() { _ = dbReader.Close() }()
tdb, err := v.decryptAndLoadDatabase(dbReader, v.Config.AgeSecretKey)
tdb, err := v.decryptAndLoadDatabase(dbReader)
if err != nil {
return nil, nil, nil, v.deepVerifyFailure(result, opts,
fmt.Sprintf("failed to decrypt database: %v", err),
@@ -186,19 +210,28 @@ func (v *Vaultik) loadVerificationData(snapshotID string, opts *VerifyOptions, r
log.Info("Database loaded",
"db_blob_count", len(dbBlobs),
"db_total_size", humanize.Bytes(uint64(dbTotalSize)))
"db_total_size", ubytes(dbTotalSize))
if !opts.JSON {
v.printfStdout("Database loaded: %d blobs (%s)\n", len(dbBlobs), humanize.Bytes(uint64(dbTotalSize)))
v.stdoutf("Database loaded: %d blobs (%s)\n",
len(dbBlobs), ubytes(dbTotalSize))
}
return manifest, tdb, dbBlobs, nil
}
// runVerificationSteps executes manifest verification, blob existence check, and deep content verification
func (v *Vaultik) runVerificationSteps(manifest *snapshot.Manifest, dbBlobs []snapshot.BlobInfo, tdb *tempDB, opts *VerifyOptions, result *VerifyResult, totalSize int64) error {
// runVerificationSteps executes manifest verification, blob existence
// check, and deep content verification.
func (v *Vaultik) runVerificationSteps(
manifest *snapshot.Manifest,
dbBlobs []snapshot.BlobInfo,
tdb *tempDB,
opts *VerifyOptions,
result *VerifyResult,
totalSize int64,
) error {
if !opts.JSON {
v.printfStdout("Verifying manifest against database...\n")
v.stdoutf("Verifying manifest against database...\n")
}
err := v.verifyManifestAgainstDatabase(manifest, dbBlobs)
@@ -207,8 +240,8 @@ func (v *Vaultik) runVerificationSteps(manifest *snapshot.Manifest, dbBlobs []sn
}
if !opts.JSON {
v.printfStdout("Manifest verified.\n")
v.printfStdout("Checking blob existence in remote storage...\n")
v.stdoutf("Manifest verified.\n")
v.stdoutf("Checking blob existence in remote storage...\n")
}
err = v.verifyBlobExistenceFromDB(dbBlobs)
@@ -217,8 +250,9 @@ func (v *Vaultik) runVerificationSteps(manifest *snapshot.Manifest, dbBlobs []sn
}
if !opts.JSON {
v.printfStdout("All blobs exist.\n")
v.printfStdout("Downloading and verifying blob contents (%d blobs, %s)...\n", len(dbBlobs), humanize.Bytes(uint64(totalSize)))
v.stdoutf("All blobs exist.\n")
v.stdoutf("Downloading and verifying blob contents (%d blobs, %s)...\n",
len(dbBlobs), ubytes(totalSize))
}
err = v.performDeepVerificationFromDB(dbBlobs, tdb.DB, opts)
@@ -243,8 +277,9 @@ func (t *tempDB) Close() error {
return err
}
// decryptAndLoadDatabase decrypts and loads the binary SQLite database from the encrypted stream
func (v *Vaultik) decryptAndLoadDatabase(reader io.ReadCloser, secretKey string) (*tempDB, error) {
// decryptAndLoadDatabase decrypts and loads the binary SQLite database
// from the encrypted stream.
func (v *Vaultik) decryptAndLoadDatabase(reader io.ReadCloser) (*tempDB, error) {
// Get decryptor
decryptor, err := v.GetDecryptor()
if err != nil {
@@ -285,7 +320,7 @@ func (v *Vaultik) decryptAndLoadDatabase(reader io.ReadCloser, secretKey string)
_ = tempFile.Close()
log.Info("Database decompressed", "size", humanize.Bytes(uint64(written)))
log.Info("Database decompressed", "size", ubytes(written))
// Open the database
db, err := sql.Open("sqlite", tempPath)
@@ -346,15 +381,17 @@ func (v *Vaultik) verifyBlob(blobInfo snapshot.BlobInfo, db *sql.DB) error {
log.Info("Blob verified",
"hash", blobInfo.Hash[:16]+"...",
"chunks", chunkCount,
"size", humanize.Bytes(uint64(blobInfo.CompressedSize)),
"size", ubytes(blobInfo.CompressedSize),
)
return nil
}
// verifyBlobChunks queries blob chunks from the database and verifies each chunk's hash
// against the decompressed blob stream
func (v *Vaultik) verifyBlobChunks(db *sql.DB, blobHash string, decompressor io.Reader) (int, error) {
// verifyBlobChunks queries blob chunks from the database and verifies
// each chunk's hash against the decompressed blob stream.
func (v *Vaultik) verifyBlobChunks(
db *sql.DB, blobHash string, decompressor io.Reader,
) (int, error) {
query := `
SELECT bc.chunk_hash, bc.offset, bc.length
FROM blob_chunks bc
@@ -389,7 +426,8 @@ func (v *Vaultik) verifyBlobChunks(db *sql.DB, blobHash string, decompressor io.
// Verify chunk ordering
if offset <= lastOffset {
return 0, fmt.Errorf("chunks out of order: offset %d after %d", offset, lastOffset)
return 0, fmt.Errorf("%w: offset %d after %d",
errChunksOutOfOrder, offset, lastOffset)
}
lastOffset = offset
@@ -423,8 +461,8 @@ func (v *Vaultik) verifyBlobChunks(db *sql.DB, blobHash string, decompressor io.
calculatedHash := hex.EncodeToString(hasher.Sum(nil))
if calculatedHash != chunkHash {
return 0, fmt.Errorf("chunk hash mismatch at offset %d: calculated %s, expected %s",
offset, calculatedHash, chunkHash)
return 0, fmt.Errorf("%w at offset %d: calculated %s, expected %s",
errChunkHashMismatch, offset, calculatedHash, chunkHash)
}
chunkCount++
@@ -438,31 +476,37 @@ func (v *Vaultik) verifyBlobChunks(db *sql.DB, blobHash string, decompressor io.
return chunkCount, nil
}
// verifyBlobFinalIntegrity checks that no trailing data exists in the decompressed stream
// and that the encrypted blob hash matches the expected value
func (v *Vaultik) verifyBlobFinalIntegrity(decompressor io.Reader, blobHasher hash.Hash, expectedHash string) error {
// Verify no remaining data in blob - if chunk list is accurate, blob should be fully consumed
// verifyBlobFinalIntegrity checks that no trailing data exists in the
// decompressed stream and that the encrypted blob hash matches the
// expected value.
func (v *Vaultik) verifyBlobFinalIntegrity(
decompressor io.Reader, blobHasher hash.Hash, expectedHash string,
) error {
// Verify no remaining data in blob - if the chunk list is accurate,
// the blob should be fully consumed.
remaining, err := io.Copy(io.Discard, decompressor)
if err != nil {
return fmt.Errorf("failed to check for remaining blob data: %w", err)
}
if remaining > 0 {
return fmt.Errorf("blob has %d unexpected trailing bytes not covered by chunk list", remaining)
return fmt.Errorf("%w: %d bytes", errTrailingBlobData, remaining)
}
// Verify blob hash matches the encrypted data we downloaded
calculatedBlobHash := hex.EncodeToString(blobHasher.Sum(nil))
if calculatedBlobHash != expectedHash {
return fmt.Errorf("blob hash mismatch: calculated %s, expected %s",
calculatedBlobHash, expectedHash)
return fmt.Errorf("%w: calculated %s, expected %s",
errBlobHashMismatch, calculatedBlobHash, expectedHash)
}
return nil
}
// getBlobsFromDatabase gets all blobs for the snapshot from the database
func (v *Vaultik) getBlobsFromDatabase(snapshotID string, db *sql.DB) ([]snapshot.BlobInfo, error) {
func (v *Vaultik) getBlobsFromDatabase(
snapshotID string, db *sql.DB,
) ([]snapshot.BlobInfo, error) {
query := `
SELECT b.blob_hash, b.compressed_size
FROM snapshot_blobs sb
@@ -505,8 +549,11 @@ func (v *Vaultik) getBlobsFromDatabase(snapshotID string, db *sql.DB) ([]snapsho
return blobs, nil
}
// verifyManifestAgainstDatabase verifies the manifest matches the authoritative database
func (v *Vaultik) verifyManifestAgainstDatabase(manifest *snapshot.Manifest, dbBlobs []snapshot.BlobInfo) error {
// verifyManifestAgainstDatabase verifies the manifest matches the
// authoritative database.
func (v *Vaultik) verifyManifestAgainstDatabase(
manifest *snapshot.Manifest, dbBlobs []snapshot.BlobInfo,
) error {
log.Info("Verifying manifest against database")
// Build map of database blobs
@@ -534,12 +581,13 @@ func (v *Vaultik) verifyManifestAgainstDatabase(manifest *snapshot.Manifest, dbB
for hash, manifestSize := range manifestBlobMap {
dbSize, exists := dbBlobMap[hash]
if !exists {
return fmt.Errorf("manifest contains blob %s not in database", hash)
return fmt.Errorf("%w: %s", errManifestExtraBlob, hash)
}
if dbSize != manifestSize {
return fmt.Errorf("blob %s size mismatch: database has %d bytes, manifest has %d bytes",
hash, dbSize, manifestSize)
return fmt.Errorf(
"%w: blob %s: database has %d bytes, manifest has %d bytes",
errBlobSizeMismatch, hash, dbSize, manifestSize)
}
}
@@ -567,16 +615,18 @@ func (v *Vaultik) verifyBlobExistenceFromDB(blobs []snapshot.BlobInfo) error {
// Verify size matches
if stat.Size != blob.CompressedSize {
return fmt.Errorf("blob %s size mismatch: S3 has %d bytes, database has %d bytes",
blob.Hash, stat.Size, blob.CompressedSize)
return fmt.Errorf(
"%w: blob %s: S3 has %d bytes, database has %d bytes",
errBlobSizeMismatch, blob.Hash, stat.Size, blob.CompressedSize)
}
// Progress update every 100 blobs
if (i+1)%100 == 0 || i == len(blobs)-1 {
if (i+1)%progressLogEvery == 0 || i == len(blobs)-1 {
log.Info("Blob existence check progress",
"checked", i+1,
"total", len(blobs),
"percent", fmt.Sprintf("%.1f%%", float64(i+1)/float64(len(blobs))*100),
"percent", fmt.Sprintf("%.1f%%",
float64(i+1)/float64(len(blobs))*percentScale),
)
}
}
@@ -586,8 +636,11 @@ func (v *Vaultik) verifyBlobExistenceFromDB(blobs []snapshot.BlobInfo) error {
return nil
}
// performDeepVerificationFromDB downloads and verifies the content of each blob using database as source
func (v *Vaultik) performDeepVerificationFromDB(blobs []snapshot.BlobInfo, db *sql.DB, opts *VerifyOptions) error {
// performDeepVerificationFromDB downloads and verifies the content of
// each blob using the database as source.
func (v *Vaultik) performDeepVerificationFromDB(
blobs []snapshot.BlobInfo, db *sql.DB, opts *VerifyOptions,
) error {
// Calculate total bytes for ETA
var totalBytesExpected int64
for _, b := range blobs {
@@ -596,7 +649,7 @@ func (v *Vaultik) performDeepVerificationFromDB(blobs []snapshot.BlobInfo, db *s
log.Info("Starting deep verification - downloading and verifying all blobs",
"blob_count", len(blobs),
"total_size", humanize.Bytes(uint64(totalBytesExpected)),
"total_size", ubytes(totalBytesExpected),
)
startTime := time.Now()
@@ -630,18 +683,18 @@ func (v *Vaultik) performDeepVerificationFromDB(blobs []snapshot.BlobInfo, db *s
"blobs_total", len(blobs),
"blobs_remaining", remaining,
"bytes_done", bytesProcessed,
"bytes_done_human", humanize.Bytes(uint64(bytesProcessed)),
"bytes_done_human", ubytes(bytesProcessed),
"bytes_total", totalBytesExpected,
"bytes_total_human", humanize.Bytes(uint64(totalBytesExpected)),
"bytes_total_human", ubytes(totalBytesExpected),
"elapsed", elapsed.Round(time.Second),
"eta", eta.Round(time.Second),
)
if !opts.JSON {
v.printfStdout(" Verified %d/%d blobs (%d remaining) - %s/%s - elapsed %s, eta %s\n",
v.stdoutf(" Verified %d/%d blobs (%d remaining) - %s/%s - elapsed %s, eta %s\n",
i+1, len(blobs), remaining,
humanize.Bytes(uint64(bytesProcessed)),
humanize.Bytes(uint64(totalBytesExpected)),
ubytes(bytesProcessed),
ubytes(totalBytesExpected),
elapsed.Round(time.Second),
eta.Round(time.Second))
}
@@ -651,7 +704,7 @@ func (v *Vaultik) performDeepVerificationFromDB(blobs []snapshot.BlobInfo, db *s
log.Info("✓ Deep verification completed successfully",
"blobs_verified", len(blobs),
"total_bytes", bytesProcessed,
"total_bytes_human", humanize.Bytes(uint64(bytesProcessed)),
"total_bytes_human", ubytes(bytesProcessed),
"duration", totalElapsed.Round(time.Second),
)

11
internal/vaultik/verify_test.go
View File

@@ -18,6 +18,8 @@ import (
// bytes when streaming through age decryption and zstd decompression.
// This validates the verification path: hash encrypted blob -> decrypt -> decompress.
func TestTeeReaderWithDecryption(t *testing.T) {
t.Parallel()
// Test data - use random data that doesn't compress well (5MB)
testData := make([]byte, 5*1024*1024)
_, err := rand.Read(testData)
@@ -26,7 +28,8 @@ func TestTeeReaderWithDecryption(t *testing.T) {
// Compress the data
var compressedBuf bytes.Buffer
compressor, err := zstd.NewWriter(&compressedBuf, zstd.WithEncoderLevel(zstd.SpeedDefault))
compressor, err := zstd.NewWriter(&compressedBuf,
zstd.WithEncoderLevel(zstd.SpeedDefault))
require.NoError(t, err)
_, err = compressor.Write(testData)
require.NoError(t, err)
@@ -34,8 +37,10 @@ func TestTeeReaderWithDecryption(t *testing.T) {
require.NoError(t, err)
// Encrypt the compressed data
testRecipient := "age1cplgrwj77ta54dnmydvvmzn64ltk83ankxl5sww04mrtmu62kv3s89gmvv"
testSecretKey := "AGE-SECRET-KEY-1C77PYNTHXSHNNC6EYR2W52UWYXACXA5JT00J9CCW9986M3XY87PSGP89AQ"
testRecipient := "age1cplgrwj77ta54dnmydvvmzn64ltk83ankxl5sww04mrt" +
"mu62kv3s89gmvv"
testSecretKey := "AGE-SECRET-KEY-1C77PYNTHXSHNNC6EYR2W52UWYXACXA5J" +
"T00J9CCW9986M3XY87PSGP89AQ"
encryptor, err := crypto.NewEncryptor([]string{testRecipient})
require.NoError(t, err)