List a file:// destination through a symlinked directory (closes #296)

List and ListStream walked the destination with afero.Walk, which uses
Lstat and so reported a symlinked metadata/ or blobs/ directory as one
file, while Put, Get and Stat follow the link. The store then listed as
empty, and prune removed the local snapshot records and deleted every
blob. Both listings now walk a view of the filesystem without its Lstat
method, so afero falls back to Stat and enters the link. A listed prefix
that is a symlink whose target is missing, such as a link to a volume
that is not mounted, fails the listing, as a missing destination
directory does.

A symlink loop is not detected: it is followed until the system refuses
the path for too many levels of links, and the listing fails.

Model: opus-5-5

README.md

@@ -286,15 +286,13 @@ needed.
   remove orphaned blobs from remote storage. By default keeps only the latest
   snapshot per name; use `--keep-newer-than` for a rolling window.
 * `--keep-newer-than <duration>`: With `--prune`, keep snapshots newer than
-  this duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`).
-  Without `--prune`, or with a value that is not a valid duration, the
-  command fails before the backup starts.
+  this duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`)
 
 **`snapshot list`**: Show every snapshot known to this host — the union
 of the local index and the backup destination store — with timestamps
 and three sizes per snapshot (compressed remote size; total
-uncompressed chunk size; size of the chunks that snapshot stored that
-were not stored before, recorded when its backup ran).
+uncompressed chunk size; size of chunks newly referenced by that
+snapshot).
 
 Listing the destination store does **not** require the age secret key,
 so it works in vaultik's intended configuration, where the backed-up
@@ -307,8 +305,8 @@ only the hash, so hostname and snapshot name exist solely in the local
 index and in the encrypted per-snapshot database. Snapshots found only
 on the destination store are therefore listed as
 `<remote only:<abbreviated remote key>>` and show `<remote only>` in
-the uncompressed and "new chunk" columns, which come only from the
-local index. Their timestamp and compressed size are real,
+the uncompressed and "new chunk" columns, which can only be computed
+from the local index. Their timestamp and compressed size are real,
 read from the manifest.
 
 Snapshots in the local index with no counterpart on the destination
@@ -433,10 +431,7 @@ object store. Credentials are read from `s3.access_key_id` and
 `s3.secret_access_key` in the config file.
 
 **Local filesystem** (`file:///path/to/backup`): Stores blobs and metadata on
-a local or mounted filesystem. Useful for testing or backing up to a NAS. The
-path must be absolute; a relative one is refused when the config loads.
-Percent-escapes in the path are decoded, so `my%20stick` and `my stick` name
-the same directory.
+a local or mounted filesystem. Useful for testing or backing up to a NAS.
 
 **Rclone** (`rclone://remote/path`): Uses rclone's 70+ supported cloud
 providers. Requires rclone to be configured separately (`rclone config`).

TODO.md

@@ -22,6 +22,17 @@ the tag exists and is exercised; what is left is merging `next` to
 
 # Completed Steps
 
+- 2026-10-13: Made a `file://` destination list through a symlinked
+  directory ([issue #296](https://git.eeqj.de/sneak/vaultik/issues/296)).
+  A `metadata/` or `blobs/` directory that was a symlink to a directory
+  elsewhere was listed as one file, so the destination store listed as
+  empty while backups and restores still worked through the link.
+  `prune` then removed the local snapshot records and deleted every blob,
+  and `remote info` and `remote nuke` found no blobs. The listings now
+  read through the link as `Get` and `Stat` do. A link whose target is
+  missing, such as one to a volume that is not mounted, or a link
+  pointing back up the tree makes them fail.
+
 - 2026-10-13: Stopped `remote info` counting a file under `blobs/` that
   is not a blob as an orphaned blob
   ([issue #295](https://git.eeqj.de/sneak/vaultik/issues/295)). A stray

docs/DATAMODEL.md

@@ -119,7 +119,6 @@ Tracks backup snapshots.
 - `completed_at` (INTEGER) - Completion timestamp (NULL if in progress)
 - `file_count` (INTEGER) - Number of files in snapshot
 - `chunk_count` (INTEGER) - Number of chunks this snapshot stored that were not stored before
-- `new_chunk_size` (INTEGER) - Total size of those chunks before compression; `snapshot list` shows it as NEW CHUNK SIZE
 - `blob_count` (INTEGER) - Number of blobs this snapshot created
 - `total_size` (INTEGER) - Total size of all files
 - `blob_size` (INTEGER) - Total compressed size of all referenced blobs

internal/blob/packer.go

@@ -245,17 +245,6 @@ func (p *Packer) Flush(ctx context.Context) error {
 	return nil
 }
 
-// DiscardCurrentBlob closes and removes the temp file of the blob being
-// packed, if any, and drops that blob. A caller that stops before Flush
-// must call it, or the temp file is left behind. Thread-safe.
-func (p *Packer) DiscardCurrentBlob() {
-	p.mu.Lock()
-	defer p.mu.Unlock()
-
-	p.cleanupTempFile()
-	p.currentBlob = nil
-}
-
 // FinalizeBlob finalizes the current blob being assembled.
 // This compresses the accumulated chunks, encrypts the result, and computes
 // the content-addressed hash. The finalized blob is either passed to the

internal/cli/config.go

@@ -251,12 +251,6 @@ func NewConfigCommand() *cobra.Command {
 		Use:   "config",
 		Short: "Manage the configuration file",
 		Long:  "Commands for creating, editing, and querying the vaultik config file.",
-		// cobra checks Args only on a command that has a Run: without
-		// one, a mistyped subcommand printed help and exited 0.
-		Args: cobra.NoArgs,
-		Run: func(cmd *cobra.Command, _ []string) {
-			_ = cmd.Help()
-		},
 	}
 
 	cmd.AddCommand(newConfigInitCommand())

internal/cli/database.go

@@ -15,12 +15,6 @@ func NewDatabaseCommand() *cobra.Command {
 		Use:   "database",
 		Short: "Manage the local state database",
 		Long:  `Commands for managing the local SQLite state database.`,
-		// cobra checks Args only on a command that has a Run: without
-		// one, a mistyped subcommand printed help and exited 0.
-		Args: cobra.NoArgs,
-		Run: func(cmd *cobra.Command, _ []string) {
-			_ = cmd.Help()
-		},
 	}
 
 	cmd.AddCommand(

internal/cli/entry_snapshot_create_prune_test.go

@@ -1,108 +0,0 @@
-package cli //nolint:testpackage // shares runEntry and the hermetic config helpers
-
-import (
-	"os"
-	"path/filepath"
-	"strings"
-	"testing"
-
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-)
-
-// Flags of `snapshot create` used to build argument vectors below.
-const (
-	flagPrune         = "--prune"
-	flagKeepNewerThan = "--keep-newer-than"
-)
-
-// TestEntrySnapshotCreateRejectsKeepNewerThanBeforeBackup checks that a
-// --keep-newer-than the post-backup prune could not use stops `snapshot
-// create` before the backup: the command fails and the destination
-// store stays empty. It used to be read only by the prune, after the
-// snapshot was uploaded, and without --prune it was ignored.
-//
-//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
-func TestEntrySnapshotCreateRejectsKeepNewerThanBeforeBackup(t *testing.T) {
-	for _, testCase := range []struct {
-		name string
-		args []string
-	}{
-		{
-			name: "decimal in a day count",
-			args: []string{flagPrune, flagKeepNewerThan, "1.5d"},
-		},
-		{
-			name: "unknown unit",
-			args: []string{flagPrune, flagKeepNewerThan, "4x"},
-		},
-		{
-			name: "without --prune",
-			args: []string{flagKeepNewerThan, "4w"},
-		},
-	} {
-		t.Run(testCase.name, func(t *testing.T) {
-			configPath, storeDir := writeBackupConfig(t)
-
-			code, _, _ := runEntry(t, append(
-				[]string{flagConfig, configPath, cmdSnapshot, cmdCreate},
-				testCase.args...)...)
-
-			assert.Equal(t, 1, code)
-
-			stored, err := os.ReadDir(storeDir)
-			require.NoError(t, err)
-			assert.Empty(t, stored, "nothing may be uploaded")
-		})
-	}
-}
-
-// TestEntrySnapshotCreateReportsPostBackupPruneFailure makes the
-// post-backup prune fail after a good backup, by planting a manifest
-// that cannot be decoded, and checks that the failure is reported as the
-// prune's and not as a failed snapshot creation.
-//
-//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
-func TestEntrySnapshotCreateReportsPostBackupPruneFailure(t *testing.T) {
-	configPath, storeDir := writeBackupConfig(t)
-
-	// A remote key is 64 lowercase hex characters. The prune reads every
-	// manifest on the store and fails on one it cannot decode.
-	plantedDir := filepath.Join(storeDir, "metadata", strings.Repeat("a", 64))
-	require.NoError(t, os.MkdirAll(plantedDir, 0o750))
-	require.NoError(t, os.WriteFile(
-		filepath.Join(plantedDir, "manifest.json.zst"),
-		[]byte("not a valid manifest"), 0o600))
-
-	code, _, stderr := runEntry(t,
-		flagConfig, configPath, cmdSnapshot, cmdCreate, flagPrune)
-
-	assert.Equal(t, 1, code)
-	assert.Contains(t, stderr, "Backup completed, but post-backup prune failed")
-	assert.NotContains(t, stderr, "Snapshot creation failed")
-
-	snapshots, err := os.ReadDir(filepath.Join(storeDir, "metadata"))
-	require.NoError(t, err)
-	assert.Len(t, snapshots, 2,
-		"the new snapshot is on the store beside the planted one")
-}
-
-// writeBackupConfig builds a hermetic config with one small file in its
-// source directory and an empty destination store directory. Returns the
-// config path and the store directory.
-func writeBackupConfig(t *testing.T) (string, string) {
-	t.Helper()
-
-	configPath := writeHermeticPruneConfig(t, false)
-
-	// writeHermeticPruneConfig puts the source and store directories
-	// beside the config file.
-	dir := filepath.Dir(configPath)
-	sourceDir := filepath.Join(dir, "source")
-
-	require.NoError(t, os.Mkdir(sourceDir, 0o750))
-	require.NoError(t, os.WriteFile(
-		filepath.Join(sourceDir, "file.txt"), []byte("hello"), 0o600))
-
-	return configPath, filepath.Join(dir, "store")
-}

internal/cli/entry_unknown_subcommand_test.go

@@ -1,43 +0,0 @@
-package cli //nolint:testpackage // shares runEntry and the command-word constants
-
-import (
-	"testing"
-
-	"github.com/stretchr/testify/assert"
-)
-
-// TestEntryGroupRejectsUnknownSubcommand runs each group command with a
-// mistyped subcommand, as a crontab line might carry one. It must fail
-// with exit 1 and name the word, as `vaultik bogus` does at the top
-// level, instead of printing the group's help and exiting 0 having done
-// nothing. The group name alone still prints its help.
-//
-//nolint:paralleltest // replaces os.Args, os.Stdout and os.Stderr
-func TestEntryGroupRejectsUnknownSubcommand(t *testing.T) {
-	for _, testCase := range []struct {
-		group    string
-		mistyped string
-	}{
-		{group: cmdSnapshot, mistyped: "craete"},
-		{group: "config", mistyped: "inti"},
-		{group: cmdRemote, mistyped: "ifno"},
-		{group: "database", mistyped: "delte"},
-		{group: "completion", mistyped: "nosuchshell"},
-	} {
-		t.Run(testCase.group, func(t *testing.T) {
-			code, stdout, stderr := runEntry(t,
-				testCase.group, testCase.mistyped)
-
-			assert.Equal(t, 1, code)
-			assert.Empty(t, stdout)
-			assert.Contains(t, stderr,
-				`unknown command "`+testCase.mistyped+`"`)
-
-			code, stdout, _ = runEntry(t, testCase.group)
-
-			assert.Equal(t, 0, code)
-			assert.Contains(t, stdout,
-				programName+" "+testCase.group+" [command]")
-		})
-	}
-}

internal/cli/remote.go

@@ -18,12 +18,6 @@ func NewRemoteCommand() *cobra.Command {
 		Use:   "remote",
 		Short: "Remote storage management commands",
 		Long:  "Commands for inspecting and managing remote storage",
-		// cobra checks Args only on a command that has a Run: without
-		// one, a mistyped subcommand printed help and exited 0.
-		Args: cobra.NoArgs,
-		Run: func(cmd *cobra.Command, _ []string) {
-			_ = cmd.Help()
-		},
 	}
 
 	// Add subcommands

internal/cli/root.go

@@ -74,21 +74,6 @@ on the source system.`,
 		NewDatabaseCommand(),
 	)
 
-	// cobra adds the completion command only when Execute runs, and with
-	// no Run, so it skipped its own cobra.NoArgs and a mistyped shell name
-	// printed help and exited 0. Adding it here lets it have a Run. Its
-	// scripts go to os.Stdout as it is now: a later SetOut does not
-	// redirect them.
-	cmd.InitDefaultCompletionCmd()
-
-	for _, sub := range cmd.Commands() {
-		if sub.Name() == "completion" {
-			sub.Run = func(completion *cobra.Command, _ []string) {
-				_ = completion.Help()
-			}
-		}
-	}
-
 	return cmd
 }
 

internal/cli/snapshot.go

@@ -41,12 +41,6 @@ func NewSnapshotCommand() *cobra.Command {
 		Use:   "snapshot",
 		Short: "Snapshot management commands",
 		Long:  "Commands for creating, listing, and managing snapshots",
-		// cobra checks Args only on a command that has a Run: without
-		// one, a mistyped subcommand printed help and exited 0.
-		Args: cobra.NoArgs,
-		Run: func(cmd *cobra.Command, _ []string) {
-			_ = cmd.Help()
-		},
 	}
 
 	// Add subcommands
@@ -77,11 +71,6 @@ otherwise from the platform config directory (~/.config/vaultik/config.yml
 on Linux), then /etc/vaultik/config.yml.`,
 		Args: cobra.ArbitraryArgs,
 		RunE: func(cmd *cobra.Command, args []string) error {
-			err := opts.Validate()
-			if err != nil {
-				return err
-			}
-
 			// Pass snapshot names from args
 			opts.Snapshots = args
 			// --skip-errors is a global flag on the root command.
@@ -108,13 +97,6 @@ on Linux), then /etc/vaultik/config.yml.`,
 			}, func(v *vaultik.Vaultik) error {
 				return v.CreateSnapshot(opts)
 			}, func(err error) {
-				if errors.Is(err, vaultik.ErrPostBackupPruneFailed) {
-					log.Error("Post-backup prune failed", "error", err)
-					ReportErrorf("Backup completed, but %v", err)
-
-					return
-				}
-
 				log.Error("Snapshot creation failed", "error", err)
 				ReportErrorf("Snapshot creation failed: %v", err)
 			})

internal/config/config.go

@@ -61,9 +61,6 @@ var (
 	errBadS3PartSize    = errors.New("s3.part_size must be between 5MiB and 5GiB")
 	errBadStorageScheme = errors.New(
 		"storage_url must start with s3://, file://, or rclone://")
-	errFileURLNotAbsolute = errors.New(
-		"a file:// storage_url needs an absolute path, " +
-			"as in file:///path/to/backup")
 	errStorageNotConfigured = errors.New(
 		"storage not configured; set storage_url or provide s3.endpoint + " +
 			"s3.bucket + credentials")
@@ -464,18 +461,11 @@ func (c *Config) validateStorage() error {
 
 // validateStorageURL validates URL-based storage configuration. File and
 // rclone URLs need no credentials; S3 URLs require the legacy s3.*
-// credential fields. A file URL must name an absolute path: a relative one
-// would resolve against the working directory, so the destination would
-// move with it.
+// credential fields.
 func (c *Config) validateStorageURL() error {
 	switch {
 	case strings.HasPrefix(c.StorageURL, "file://"):
-		// storage.ParseStorageURL applies the same rule, but config
-		// cannot import storage, which imports config.
-		if !strings.HasPrefix(c.StorageURL, "file:///") {
-			return errFileURLNotAbsolute
-		}
-
+		// File storage doesn't need S3 credentials
 		return nil
 	case strings.HasPrefix(c.StorageURL, "rclone://"):
 		// Rclone storage uses rclone's own config

internal/config/config_test.go

@@ -337,35 +337,6 @@ func TestLoadS3PartSize(t *testing.T) {
 	})
 }
 
-// TestLoadRejectsRelativeFileURL checks that a file:// storage_url with a
-// relative path fails at load. Accepted, it would resolve against the
-// working directory, so a backup run from another directory would write
-// to a different destination and still succeed.
-func TestLoadRejectsRelativeFileURL(t *testing.T) {
-	t.Parallel()
-
-	const relativeConfig = "snapshots:\n" +
-		"  test:\n" +
-		"    paths: [/tmp/vaultik-test-source]\n" +
-		"storage_url: file://dest\n"
-
-	path := filepath.Join(t.TempDir(), "config.yml")
-
-	err := os.WriteFile(path, []byte(relativeConfig), 0o600)
-	if err != nil {
-		t.Fatalf("write config: %v", err)
-	}
-
-	_, err = Load(path)
-	if !errors.Is(err, errFileURLNotAbsolute) {
-		t.Fatalf("Load() error = %v, want errFileURLNotAbsolute", err)
-	}
-
-	if !strings.Contains(err.Error(), "absolute path") {
-		t.Errorf("error %q does not say to use an absolute path", err.Error())
-	}
-}
-
 // TestValidateAgeRecipients checks that recipients are parsed at config load
 // (a bad entry fails immediately, not mid-backup) and that no invalid entry —
 // least of all a pasted secret key — is echoed in the error. An empty list

internal/database/repositories.go

@@ -3,7 +3,6 @@ package database
 import (
 	"context"
 	"database/sql"
-	"errors"
 	"fmt"
 )
 
@@ -61,24 +60,22 @@ func (r *Repositories) WithTx(ctx context.Context, fn TxFunc) error {
 
 	defer func() {
 		if p := recover(); p != nil {
-			// The panic is the failure to report, so a rollback error
-			// is dropped.
-			_ = tx.Rollback()
+			rollbackErr := tx.Rollback()
+			if rollbackErr != nil {
+				Fatalf("failed to rollback transaction: %v", rollbackErr)
+			}
 
 			panic(p)
+		} else if err != nil {
+			rollbackErr := tx.Rollback()
+			if rollbackErr != nil {
+				Fatalf("failed to rollback transaction: %v", rollbackErr)
+			}
 		}
 	}()
 
 	err = fn(ctx, tx)
 	if err != nil {
-		// Once ctx is cancelled, database/sql rolls the transaction
-		// back itself and Rollback returns sql.ErrTxDone.
-		rollbackErr := tx.Rollback()
-		if rollbackErr != nil && !errors.Is(rollbackErr, sql.ErrTxDone) {
-			return fmt.Errorf("%w; rolling back transaction: %w",
-				err, rollbackErr)
-		}
-
 		return err
 	}
 

internal/database/repositories_test.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"database/sql"
 	"errors"
-	"strings"
 	"testing"
 	"time"
 
@@ -236,125 +235,6 @@ func TestRepositoriesTransactionRollback(t *testing.T) {
 	}
 }
 
-// When the context is cancelled during the body, database/sql rolls the
-// transaction back itself, so WithTx's own rollback finds it already
-// done. WithTx must return the body's error alone, without that rollback.
-func TestRepositoriesTransactionContextCancelled(t *testing.T) {
-	t.Parallel()
-
-	db, cleanup := setupTestDB(t)
-	defer cleanup()
-
-	ctx, cancel := context.WithCancel(context.Background())
-	defer cancel()
-
-	repos := database.NewRepositories(db)
-
-	err := repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
-		file := &database.File{
-			Path:  "/test/cancelled_file.txt",
-			MTime: time.Now().Truncate(time.Second),
-			Size:  1024,
-			Mode:  0644,
-			UID:   1000,
-			GID:   1000,
-		}
-
-		err := repos.Files.Create(ctx, tx, file)
-		if err != nil {
-			return err
-		}
-
-		cancel()
-
-		// database/sql rolls back from a goroutine of its own; wait
-		// until it has, so the test does not depend on that race.
-		for {
-			_, err = tx.ExecContext(context.WithoutCancel(ctx), "SELECT 1")
-			if errors.Is(err, sql.ErrTxDone) {
-				return ctx.Err()
-			}
-
-			time.Sleep(time.Millisecond)
-		}
-	})
-	if !errors.Is(err, context.Canceled) {
-		t.Fatalf("expected context.Canceled, got: %v", err)
-	}
-
-	if errors.Is(err, sql.ErrTxDone) {
-		t.Fatalf("the already-done rollback was reported: %v", err)
-	}
-
-	file, err := repos.Files.GetByPath(
-		context.Background(), "/test/cancelled_file.txt",
-	)
-	if err != nil {
-		t.Fatalf("error checking for file: %v", err)
-	}
-
-	if file != nil {
-		t.Error("file should not exist after the cancelled transaction")
-	}
-}
-
-// A body that ends the transaction itself with ROLLBACK makes WithTx's
-// own rollback fail. WithTx must return that failure together with the
-// body's error.
-func TestRepositoriesTransactionRollbackFails(t *testing.T) {
-	t.Parallel()
-
-	db, cleanup := setupTestDB(t)
-	defer cleanup()
-
-	ctx := context.Background()
-	repos := database.NewRepositories(db)
-
-	err := repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
-		_, err := tx.ExecContext(ctx, "ROLLBACK")
-		if err != nil {
-			return err
-		}
-
-		return errIntentionalRollback
-	})
-	if !errors.Is(err, errIntentionalRollback) {
-		t.Fatalf("expected the body's error, got: %v", err)
-	}
-
-	if !strings.Contains(err.Error(), "rolling back transaction") {
-		t.Errorf("expected the failed rollback in the error, got: %v", err)
-	}
-}
-
-// When the body ends the transaction itself with ROLLBACK and then
-// panics, WithTx's own rollback fails; the panic must still reach the
-// caller.
-func TestRepositoriesTransactionPanicRollbackFails(t *testing.T) {
-	t.Parallel()
-
-	db, cleanup := setupTestDB(t)
-	defer cleanup()
-
-	ctx := context.Background()
-	repos := database.NewRepositories(db)
-
-	defer func() {
-		if recover() == nil {
-			t.Error("expected the body's panic to reach the caller")
-		}
-	}()
-
-	_ = repos.WithTx(ctx, func(ctx context.Context, tx *sql.Tx) error {
-		_, err := tx.ExecContext(ctx, "ROLLBACK")
-		if err != nil {
-			return err
-		}
-
-		panic(errIntentionalRollback)
-	})
-}
-
 func TestRepositoriesReadTransaction(t *testing.T) {
 	t.Parallel()
 

internal/database/schema/001.sql

@@ -86,7 +86,6 @@ CREATE TABLE IF NOT EXISTS snapshots (
     completed_at INTEGER,
     file_count INTEGER NOT NULL DEFAULT 0,
     chunk_count INTEGER NOT NULL DEFAULT 0,
-    new_chunk_size INTEGER NOT NULL DEFAULT 0,
     blob_count INTEGER NOT NULL DEFAULT 0,
     total_size INTEGER NOT NULL DEFAULT 0,
     blob_size INTEGER NOT NULL DEFAULT 0,

internal/database/snapshots.go

@@ -129,7 +129,6 @@ func (r *SnapshotRepository) UpdateExtendedStats(
 	compressionLevel int,
 	uploadBytes int64,
 	uploadDurationMs int64,
-	newChunkSize int64,
 ) error {
 	compressionRatio, err := r.extendedCompressionRatio(
 		ctx, tx, snapshotID, blobUncompressedSize,
@@ -144,19 +143,18 @@ func (r *SnapshotRepository) UpdateExtendedStats(
 		    compression_ratio = ?,
 		    compression_level = ?,
 		    upload_bytes = ?,
-		    upload_duration_ms = ?,
-		    new_chunk_size = ?
+		    upload_duration_ms = ?
 		WHERE id = ?
 	`
 
 	if tx != nil {
 		_, err = tx.ExecContext(ctx, query,
 			blobUncompressedSize, compressionRatio, compressionLevel,
-			uploadBytes, uploadDurationMs, newChunkSize, snapshotID)
+			uploadBytes, uploadDurationMs, snapshotID)
 	} else {
 		_, err = r.db.ExecWithLog(ctx, query,
 			blobUncompressedSize, compressionRatio, compressionLevel,
-			uploadBytes, uploadDurationMs, newChunkSize, snapshotID)
+			uploadBytes, uploadDurationMs, snapshotID)
 	}
 
 	if err != nil {
@@ -596,24 +594,45 @@ func (r *SnapshotRepository) GetSnapshotUncompressedChunkSize(
 	return totalSize, nil
 }
 
-// GetSnapshotNewChunkSize returns the total plaintext size of the chunks the
-// snapshot's backup stored that were not stored before, as the backup
-// recorded it. It is not derived from the snapshot's files: a file's row is
-// shared by every snapshot that lists it and is rewritten when the file
-// changes, so it shows only the file's latest chunks.
+// GetSnapshotNewChunkSize returns the sum of plaintext sizes of chunks that are
+// referenced by this snapshot but not by any earlier completed snapshot known to
+// the local database. The result is the marginal uncompressed data this snapshot
+// added to the dedup pool — i.e., the delta from prior snapshots.
 func (r *SnapshotRepository) GetSnapshotNewChunkSize(
 	ctx context.Context, snapshotID string,
 ) (int64, error) {
-	query := `SELECT new_chunk_size FROM snapshots WHERE id = ?`
+	query := `
+		WITH this_snap_chunks AS (
+			SELECT DISTINCT fc.chunk_hash
+			FROM snapshot_files sf
+			JOIN file_chunks fc ON sf.file_id = fc.file_id
+			WHERE sf.snapshot_id = ?
+		),
+		prior_chunks AS (
+			SELECT DISTINCT fc.chunk_hash
+			FROM snapshots s
+			JOIN snapshot_files sf ON sf.snapshot_id = s.id
+			JOIN file_chunks fc ON fc.file_id = sf.file_id
+			WHERE s.completed_at IS NOT NULL
+			  AND s.id != ?
+			  AND s.started_at < (SELECT started_at FROM snapshots WHERE id = ?)
+		)
+		SELECT COALESCE(SUM(c.size), 0)
+		FROM chunks c
+		JOIN this_snap_chunks t ON c.chunk_hash = t.chunk_hash
+		WHERE c.chunk_hash NOT IN (SELECT chunk_hash FROM prior_chunks)
+	`
 
-	var newChunkSize int64
+	var totalSize int64
 
-	err := r.db.conn.QueryRowContext(ctx, query, snapshotID).Scan(&newChunkSize)
+	err := r.db.conn.QueryRowContext(
+		ctx, query, snapshotID, snapshotID, snapshotID,
+	).Scan(&totalSize)
 	if err != nil {
 		return 0, fmt.Errorf("querying new chunk size: %w", err)
 	}
 
-	return newChunkSize, nil
+	return totalSize, nil
 }
 
 // GetIncompleteSnapshots returns all snapshots that haven't been completed

internal/log/tty_handler.go

@@ -10,8 +10,6 @@ import (
 	"sync"
 	"time"
 	"unicode"
-
-	"github.com/dustin/go-humanize"
 )
 
 // groupSeparator joins an open group path to an attribute key. This
@@ -306,13 +304,18 @@ func formatDuration(d time.Duration) string {
 	}
 }
 
-// formatBytes formats a byte count with humanize.Bytes, as every command
-// prints one. humanize.Bytes takes no negative count, so one is printed
-// as it is.
+// formatBytes formats bytes in a human-readable way
 func formatBytes(b int64) string {
-	if b < 0 {
+	const unit = 1024
+	if b < unit {
 		return fmt.Sprintf("%d B", b)
 	}
 
-	return humanize.Bytes(uint64(b))
+	div, exp := int64(unit), 0
+	for n := b / unit; n >= unit; n /= unit {
+		div *= unit
+		exp++
+	}
+
+	return fmt.Sprintf("%.1f %cB", float64(b)/float64(div), "KMGTPE"[exp])
 }

internal/log/tty_handler_test.go

@@ -200,8 +200,7 @@ func TestTTYHandlerWithGroupQualifiesKeys(t *testing.T) {
 func TestTTYHandlerByteFormattingSurvivesGrouping(t *testing.T) {
 	t.Parallel()
 
-	// Printed as "1.0 MB", in powers of 1000 like every other byte count.
-	const oneMegabyte = 1_000_000
+	const oneAndAHalfKiB = 1536
 
 	for name, testCase := range map[string]struct {
 		derive func(*slog.Logger) *slog.Logger
@@ -224,13 +223,13 @@ func TestTTYHandlerByteFormattingSurvivesGrouping(t *testing.T) {
 			var buf bytes.Buffer
 
 			logger := slog.New(log.NewTTYHandler(&buf, debugHandlerOptions()))
-			testCase.derive(logger).Info("uploaded", "bytes", oneMegabyte)
+			testCase.derive(logger).Info("uploaded", "bytes", oneAndAHalfKiB)
 
 			line := ansiEscape.ReplaceAllString(buf.String(), "")
 
-			assert.Contains(t, line, testCase.key+"=1.0 MB",
+			assert.Contains(t, line, testCase.key+"=1.5 KB",
 				"a byte count must be human-readable however it is qualified")
-			assert.NotContains(t, line, strconv.Itoa(oneMegabyte),
+			assert.NotContains(t, line, strconv.Itoa(oneAndAHalfKiB),
 				"the raw number must not survive the formatting")
 		})
 	}

internal/snapshot/exclude_test.go

@@ -531,50 +531,3 @@ func TestExcludePatterns_UnanchoredPatternFile(t *testing.T) {
 	require.Equal(t, 1, result.FilesScanned,
 		"Unanchored pattern for file should match anywhere")
 }
-
-func TestExcludePatterns_DoubleStarDirectoryAtEveryDepth(t *testing.T) {
-	t.Parallel()
-
-	// As in gitignore, a leading **/ also matches directly under the source dir
-	fs := setupAnchoredTestFS(t)
-
-	scanner, repos, cleanup := createTestScanner(t, fs, []string{"**/projectname"})
-	defer cleanup()
-
-	ctx := context.Background()
-	createSnapshotRecord(ctx, t, repos, "test-snapshot")
-
-	result, err := scanner.Scan(ctx, "/backup", "test-snapshot")
-	require.NoError(t, err)
-
-	// **/projectname should exclude BOTH:
-	// - /backup/projectname/file.txt
-	// - /backup/otherproject/projectname/file.txt
-	// Total files: 4, excluded: 2
-	require.Equal(t, 2, result.FilesScanned,
-		"**/ directory pattern should match at the top level and nested")
-}
-
-func TestExcludePatterns_DoubleStarFileAtEveryDepth(t *testing.T) {
-	t.Parallel()
-
-	// As in gitignore, a leading **/ also matches directly under the source dir
-	fs := setupAnchoredTestFS(t)
-
-	scanner, repos, cleanup := createTestScanner(t, fs, []string{"**/*.txt"})
-	defer cleanup()
-
-	ctx := context.Background()
-	createSnapshotRecord(ctx, t, repos, "test-snapshot")
-
-	result, err := scanner.Scan(ctx, "/backup", "test-snapshot")
-	require.NoError(t, err)
-
-	// **/*.txt should exclude ALL .txt files:
-	// - /backup/file.txt
-	// - /backup/projectname/file.txt
-	// - /backup/otherproject/projectname/file.txt
-	// Total files: 4, excluded: 3
-	require.Equal(t, 1, result.FilesScanned,
-		"**/ file pattern should match at the top level and nested")
-}

internal/snapshot/scanner.go

@@ -135,9 +135,7 @@ type ScannerConfig struct {
 // are counted per file: BytesScanned is the size of the new and changed
 // files, BytesSkipped that of the unchanged ones. FilesFailed counts the
 // new and changed files that phase 2 could not store; FilesScanned
-// includes them and BytesScanned does not. ChunksCreated counts the
-// chunks that were not already stored, and NewChunkSize is their total
-// size.
+// includes them and BytesScanned does not.
 type ScanResult struct {
 	FilesScanned   int
 	FilesSkipped   int
@@ -147,7 +145,6 @@ type ScanResult struct {
 	BytesSkipped   int64
 	BytesDeleted   int64
 	ChunksCreated  int
-	NewChunkSize   int64
 	BlobsCreated   int
 	BlobsUploaded  int
 	BytesUploaded  int64
@@ -1298,11 +1295,6 @@ func (s *Scanner) batchAddFilesToSnapshot(
 func (s *Scanner) processPhase(
 	ctx context.Context, filesToProcess []*FileToProcess, result *ScanResult,
 ) error {
-	// On an error or cancellation the blob being packed is never
-	// finalized, so its temp file must be removed here. After a
-	// successful flush there is no such blob and this does nothing.
-	defer s.packer.DiscardCurrentBlob()
-
 	// Calculate total bytes to process
 	var totalBytes int64
 	for _, f := range filesToProcess {
@@ -1869,12 +1861,11 @@ func (s *Scanner) processFileStreaming(
 	return nil
 }
 
-// updateChunkStats counts a chunk that was not already stored, and its
-// size. The scan result's file counts, BytesScanned and BytesSkipped are
-// not touched here: the scan phase counts each file once.
+// updateChunkStats counts a chunk that was not already stored. The scan
+// result's file counts, BytesScanned and BytesSkipped are not touched
+// here: the scan phase counts each file once.
 func (s *Scanner) updateChunkStats(chunkSize int64, result *ScanResult) {
 	result.ChunksCreated++
-	result.NewChunkSize += chunkSize
 
 	if s.progress != nil {
 		s.progress.GetStats().ChunksCreated.Add(1)
@@ -2022,14 +2013,6 @@ func compileExcludePatterns(patterns []string) []compiledPattern {
 		// Remove trailing slash if present (directory indicator)
 		pattern = strings.TrimSuffix(pattern, "/")
 
-		// A leading "**/" demands a "/" before the name and so misses an
-		// entry directly under the source path. An unanchored pattern
-		// already matches at every depth, so drop the prefix: as in
-		// gitignore, "**/x" then also matches a top-level "x".
-		if !anchored {
-			pattern = strings.TrimPrefix(pattern, "**/")
-		}
-
 		// Compile the glob pattern
 		// For patterns without path separators, we need to match them as components
 		// e.g., ".git" should match ".git" anywhere in the path

internal/snapshot/skip_errors_test.go

@@ -381,60 +381,3 @@ func TestScannerCancelStopsSkipErrorsRun(t *testing.T) {
 		})
 	}
 }
-
-// TestScannerRemovesBlobTempFileWhenRunStops checks that a run stopped
-// part-way, by a file it cannot read or by a cancellation, removes the
-// temp file of the blob it was packing.
-func TestScannerRemovesBlobTempFileWhenRunStops(t *testing.T) {
-	t.Parallel()
-
-	// a.txt is backed up first and starts a blob; the run stops at b.txt.
-	const target = "/source/b.txt"
-
-	tests := []struct {
-		name  string
-		newFs func(cancel context.CancelFunc) afero.Fs
-	}{
-		{
-			name: "file cannot be read",
-			newFs: func(context.CancelFunc) afero.Fs {
-				return &readFailFs{Fs: afero.NewMemMapFs(), target: target}
-			},
-		},
-		{
-			name: "cancelled",
-			newFs: func(cancel context.CancelFunc) afero.Fs {
-				return &cancelOnOpenFs{
-					Fs: afero.NewMemMapFs(), target: target, cancel: cancel,
-				}
-			},
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			ctx, cancel := context.WithCancel(context.Background())
-			defer cancel()
-
-			fs := tt.newFs(cancel)
-			writeSkipErrorTestFile(t, fs, "/source/a.txt", "first file content")
-			writeSkipErrorTestFile(t, fs, target, "second file content")
-
-			_, err := runSkipErrorScan(ctx, t, fs, "/source", false, nil)
-			if err == nil {
-				t.Fatal("expected the scan to stop with an error, got nil")
-			}
-
-			left, err := afero.Glob(fs, filepath.Join(os.TempDir(), "vaultik-blob-*"))
-			if err != nil {
-				t.Fatalf("listing blob temp files: %v", err)
-			}
-
-			if len(left) != 0 {
-				t.Fatalf("expected no blob temp file left, got %v", left)
-			}
-		})
-	}
-}

internal/snapshot/snapshot.go

@@ -199,7 +199,6 @@ func (sm *SnapshotManager) UpdateSnapshotStatsExtended(
 			stats.CompressionLevel,
 			stats.BytesUploaded,
 			stats.UploadDurationMs,
-			stats.NewChunkSize,
 		)
 	})
 }
@@ -907,7 +906,6 @@ type ExtendedBackupStats struct {
 	BlobUncompressedSize int64 // Total uncompressed size of all referenced blobs
 	CompressionLevel     int   // Compression level used for this snapshot
 	UploadDurationMs     int64 // Total milliseconds spent uploading to S3
-	NewChunkSize         int64 // Total size of the chunks counted in ChunksCreated
 }
 
 // deleteOtherSnapshots deletes all snapshots except the current one

internal/storage/file.go

@@ -120,9 +120,10 @@ func (f *FileStorer) Delete(_ context.Context, key string) error {
 	return nil
 }
 
-// List returns all keys with the given prefix. It fails when the
-// destination directory is missing; a missing prefix under it is an
-// empty listing.
+// List returns all keys with the given prefix, reading through symlinked
+// directories. It fails when the destination directory is missing or the
+// prefix is a symlink whose target is missing; a missing prefix under the
+// destination directory is an empty listing.
 func (f *FileStorer) List(ctx context.Context, prefix string) ([]string, error) {
 	var keys []string
 
@@ -133,8 +134,7 @@ func (f *FileStorer) List(ctx context.Context, prefix string) ([]string, error)
 
 	basePath := f.fullPath(prefix)
 
-	// Check if the prefix exists
-	exists, err := afero.Exists(f.fs, basePath)
+	exists, err := f.prefixExists(basePath)
 	if err != nil {
 		return nil, fmt.Errorf("checking path: %w", err)
 	}
@@ -143,7 +143,7 @@ func (f *FileStorer) List(ctx context.Context, prefix string) ([]string, error)
 		return keys, nil // Empty list for non-existent prefix
 	}
 
-	err = afero.Walk(f.fs, basePath, func(path string, info os.FileInfo, err error) error {
+	err = f.walk(basePath, func(path string, info os.FileInfo, err error) error {
 		if err != nil {
 			return err
 		}
@@ -176,7 +176,9 @@ func (f *FileStorer) List(ctx context.Context, prefix string) ([]string, error)
 }
 
 // ListStream returns a channel of ObjectInfo for large result sets. Like
-// List, it sends an error when the destination directory is missing.
+// List, it reads through symlinked directories and sends an error when
+// the destination directory is missing or the prefix is a symlink whose
+// target is missing.
 func (f *FileStorer) ListStream(ctx context.Context, prefix string) <-chan ObjectInfo {
 	ch := make(chan ObjectInfo)
 
@@ -192,8 +194,7 @@ func (f *FileStorer) ListStream(ctx context.Context, prefix string) <-chan Objec
 
 		basePath := f.fullPath(prefix)
 
-		// Check if the prefix exists
-		exists, err := afero.Exists(f.fs, basePath)
+		exists, err := f.prefixExists(basePath)
 		if err != nil {
 			ch <- ObjectInfo{Err: fmt.Errorf("checking path: %w", err)}
 
@@ -204,7 +205,7 @@ func (f *FileStorer) ListStream(ctx context.Context, prefix string) <-chan Objec
 			return // Empty channel for non-existent prefix
 		}
 
-		_ = afero.Walk(f.fs, basePath, func(path string, info os.FileInfo, err error) error {
+		_ = f.walk(basePath, func(path string, info os.FileInfo, err error) error {
 			// Check context cancellation
 			select {
 			case <-ctx.Done():
@@ -352,6 +353,40 @@ func (f *FileStorer) writeAtomic(
 	return nil
 }
 
+// walk is afero.Walk, except that it reads through a symlinked directory
+// the way Get and Stat do, where afero.Walk reports the link as one file.
+// afero.Walk uses Lstat, which does not follow a link, whenever the
+// filesystem has one; the struct below carries only the afero.Fs methods,
+// so the walk uses Stat instead. A link back up the tree is followed until
+// the operating system refuses the path for too many levels of links, and
+// the walk then fails.
+func (f *FileStorer) walk(root string, fn filepath.WalkFunc) error {
+	return afero.Walk(struct{ afero.Fs }{f.fs}, root, fn)
+}
+
+// prefixExists is afero.Exists, except that it does not follow a symlink
+// at path. A symlinked prefix whose target is missing, such as a link to
+// a volume that is not mounted, therefore exists, and walk then fails on
+// it; with afero.Exists it would list as empty, and prune would delete
+// every blob.
+func (f *FileStorer) prefixExists(path string) (bool, error) {
+	lstater, ok := f.fs.(afero.Lstater)
+	if !ok {
+		return afero.Exists(f.fs, path)
+	}
+
+	_, _, err := lstater.LstatIfPossible(path)
+	if err == nil {
+		return true, nil
+	}
+
+	if os.IsNotExist(err) {
+		return false, nil
+	}
+
+	return false, err
+}
+
 // fullPath returns the full filesystem path for a key.
 func (f *FileStorer) fullPath(key string) string {
 	return filepath.Join(f.basePath, key)

internal/storage/file_backend_test.go

@@ -4,7 +4,10 @@ import (
 	"context"
 	"errors"
 	"io/fs"
+	"os"
 	"path/filepath"
+	"reflect"
+	"strings"
 	"testing"
 
 	"sneak.berlin/go/vaultik/internal/storage"
@@ -60,3 +63,151 @@ func TestFileStorerListMissingDestination(t *testing.T) {
 		t.Errorf("ListStream error = %v, want a not-exist error", streamErr)
 	}
 }
+
+// TestFileStorerListsThroughSymlinkedDirectory checks that List and
+// ListStream read through a symlinked metadata/ or blobs/ directory, as Get
+// and Stat do, instead of listing the link as one key. Listing the whole
+// destination meets the link part-way through the walk; listing its prefix
+// starts the walk at the link.
+func TestFileStorerListsThroughSymlinkedDirectory(t *testing.T) {
+	t.Parallel()
+
+	for _, dir := range []string{"metadata", "blobs"} {
+		t.Run(dir, func(t *testing.T) {
+			t.Parallel()
+
+			base := t.TempDir()
+			key := dir + "/aa/bb/object"
+
+			s, err := storage.NewFileStorer(base)
+			if err != nil {
+				t.Fatalf("NewFileStorer: %v", err)
+			}
+
+			err = s.Put(context.Background(), key, strings.NewReader("data"))
+			if err != nil {
+				t.Fatalf("Put: %v", err)
+			}
+
+			linkPath := filepath.Join(base, dir)
+			movedPath := filepath.Join(t.TempDir(), dir)
+
+			err = os.Rename(linkPath, movedPath)
+			if err != nil {
+				t.Fatalf("moving %s away: %v", dir, err)
+			}
+
+			err = os.Symlink(movedPath, linkPath)
+			if err != nil {
+				t.Fatalf("linking %s: %v", dir, err)
+			}
+
+			want := []string{key}
+
+			for _, prefix := range []string{"", dir + "/"} {
+				got := listSorted(t, s, prefix)
+				if !reflect.DeepEqual(got, want) {
+					t.Errorf("List(%q) = %v, want %v", prefix, got, want)
+				}
+
+				got = listStreamKeys(t, s, prefix)
+				if !reflect.DeepEqual(got, want) {
+					t.Errorf("ListStream(%q) = %v, want %v", prefix, got, want)
+				}
+			}
+		})
+	}
+}
+
+// TestFileStorerListFailsOnSymlinkWithMissingTarget checks that List and
+// ListStream fail on a symlinked metadata/ or blobs/ directory whose target
+// is missing, such as a link to a volume that is not mounted, as they do on
+// a missing destination directory, instead of listing it as empty.
+func TestFileStorerListFailsOnSymlinkWithMissingTarget(t *testing.T) {
+	t.Parallel()
+
+	ctx := context.Background()
+
+	for _, dir := range []string{"metadata", "blobs"} {
+		t.Run(dir, func(t *testing.T) {
+			t.Parallel()
+
+			base := t.TempDir()
+
+			s, err := storage.NewFileStorer(base)
+			if err != nil {
+				t.Fatalf("NewFileStorer: %v", err)
+			}
+
+			unmounted := filepath.Join(t.TempDir(), "unmounted")
+
+			err = os.Symlink(unmounted, filepath.Join(base, dir))
+			if err != nil {
+				t.Fatalf("Symlink: %v", err)
+			}
+
+			for _, prefix := range []string{"", dir + "/"} {
+				keys, err := s.List(ctx, prefix)
+				if !errors.Is(err, fs.ErrNotExist) {
+					t.Errorf("List(%q) = %v, %v; want a not-exist error",
+						prefix, keys, err)
+				}
+
+				var streamErr error
+
+				for object := range s.ListStream(ctx, prefix) {
+					if object.Err != nil {
+						streamErr = object.Err
+					}
+				}
+
+				if !errors.Is(streamErr, fs.ErrNotExist) {
+					t.Errorf("ListStream(%q) error = %v, want a not-exist error",
+						prefix, streamErr)
+				}
+			}
+		})
+	}
+}
+
+// TestFileStorerListFailsOnSymlinkLoop checks that a symlink pointing back
+// at the destination makes List and ListStream fail instead of walking it
+// forever.
+func TestFileStorerListFailsOnSymlinkLoop(t *testing.T) {
+	t.Parallel()
+
+	ctx := context.Background()
+	base := t.TempDir()
+
+	s, err := storage.NewFileStorer(base)
+	if err != nil {
+		t.Fatalf("NewFileStorer: %v", err)
+	}
+
+	err = s.Put(ctx, testBlobKey, strings.NewReader("data"))
+	if err != nil {
+		t.Fatalf("Put: %v", err)
+	}
+
+	err = os.Symlink(base, filepath.Join(base, "blobs", "loop"))
+	if err != nil {
+		t.Fatalf("Symlink: %v", err)
+	}
+
+	keys, err := s.List(ctx, "blobs/")
+	if err == nil {
+		t.Errorf("List = %d keys and no error, want an error", len(keys))
+	}
+
+	var streamErr error
+
+	for object := range s.ListStream(ctx, "blobs/") {
+		if object.Err != nil {
+			streamErr = object.Err
+		}
+	}
+
+	if streamErr == nil {
+		t.Error("ListStream sent no error, want one")
+	}
+}

internal/storage/url.go

@@ -17,10 +17,8 @@ const (
 
 // Sentinel errors for storage URL parsing.
 var (
-	ErrEmptyStorageURL  = errors.New("storage URL is empty")
-	ErrEmptyFilePath    = errors.New("file URL path is empty")
-	ErrRelativeFilePath = errors.New(
-		"file URL needs an absolute path, as in file:///path/to/backup")
+	ErrEmptyStorageURL   = errors.New("storage URL is empty")
+	ErrEmptyFilePath     = errors.New("file URL path is empty")
 	ErrMissingBucket     = errors.New("s3 URL missing bucket name")
 	ErrMissingRemote     = errors.New("rclone URL missing remote name")
 	ErrUnsupportedScheme = errors.New(
@@ -53,8 +51,17 @@ func ParseStorageURL(rawURL string) (*URL, error) {
 		return nil, ErrEmptyStorageURL
 	}
 
+	// Handle file:// URLs
 	if after, ok := strings.CutPrefix(rawURL, "file://"); ok {
-		return parseFileURL(after)
+		path := after
+		if path == "" {
+			return nil, ErrEmptyFilePath
+		}
+
+		return &URL{
+			Scheme: schemeFile,
+			Prefix: path,
+		}, nil
 	}
 
 	if strings.HasPrefix(rawURL, "s3://") {
@@ -68,27 +75,6 @@ func ParseStorageURL(rawURL string) (*URL, error) {
 	return nil, ErrUnsupportedScheme
 }
 
-// parseFileURL parses the path that follows file://. The path must be
-// absolute: a relative one would resolve against the working directory,
-// so the destination would move with it. Percent-escapes are decoded, so
-// my%20stick names the directory "my stick"; a literal space is kept.
-func parseFileURL(rawPath string) (*URL, error) {
-	if rawPath == "" {
-		return nil, ErrEmptyFilePath
-	}
-
-	if !strings.HasPrefix(rawPath, "/") {
-		return nil, ErrRelativeFilePath
-	}
-
-	path, err := url.PathUnescape(rawPath)
-	if err != nil {
-		return nil, fmt.Errorf("invalid file URL path: %w", err)
-	}
-
-	return &URL{Scheme: schemeFile, Prefix: path}, nil
-}
-
 // parseS3URL parses an s3://bucket/prefix URL. It rejects credentials in
 // the userinfo and any query parameter other than endpoint, region and
 // ssl, so a credential-bearing URL is never stored or echoed.

internal/storage/url_parse_test.go

@@ -14,10 +14,7 @@ import (
 func TestParseStorageURLValid(t *testing.T) {
 	t.Parallel()
 
-	const (
-		bucket     = "mybucket"
-		fileScheme = "file"
-	)
+	const bucket = "mybucket"
 
 	cases := []struct {
 		name string
@@ -27,17 +24,7 @@ func TestParseStorageURLValid(t *testing.T) {
 		{
 			name: "file absolute path",
 			raw:  "file:///var/backups/vaultik",
-			want: &storage.URL{Scheme: fileScheme, Prefix: "/var/backups/vaultik"},
-		},
-		{
-			name: "file percent-escaped path is decoded",
-			raw:  "file:///Volumes/my%20stick/backup",
-			want: &storage.URL{Scheme: fileScheme, Prefix: "/Volumes/my stick/backup"},
-		},
-		{
-			name: "file path with a literal space",
-			raw:  "file:///Volumes/my stick/backup",
-			want: &storage.URL{Scheme: fileScheme, Prefix: "/Volumes/my stick/backup"},
+			want: &storage.URL{Scheme: "file", Prefix: "/var/backups/vaultik"},
 		},
 		{
 			name: "s3 bucket and prefix, ssl defaults on",
@@ -103,7 +90,6 @@ func TestParseStorageURLErrors(t *testing.T) {
 	}{
 		{"empty url", "", storage.ErrEmptyStorageURL},
 		{"file empty path", "file://", storage.ErrEmptyFilePath},
-		{"file relative path", "file://dest", storage.ErrRelativeFilePath},
 		{"s3 missing bucket", "s3://", storage.ErrMissingBucket},
 		{"s3 missing bucket with path", "s3:///justprefix", storage.ErrMissingBucket},
 		{"rclone missing remote", "rclone://", storage.ErrMissingRemote},

internal/vaultik/blob_fetch.go

@@ -103,8 +103,6 @@ func (v *Vaultik) FetchAndDecryptBlob(
 }
 
 // FetchBlob downloads a blob and returns a reader for the encrypted data.
-// The reader is closed as soon as ctx is cancelled, which ends a read
-// blocked inside it.
 func (v *Vaultik) FetchBlob(
 	ctx context.Context, blobHash string,
 ) (io.ReadCloser, error) {
@@ -122,47 +120,5 @@ func (v *Vaultik) FetchBlob(
 		return nil, fmt.Errorf("downloading blob %s: %w", shortHash(blobHash), err)
 	}
 
-	return newCloseOnCancelReader(ctx, rc), nil
-}
-
-// closeOnCancelReader closes the reader it wraps when ctx is cancelled.
-// Not every backend's reader stops on cancel by itself (the file
-// backend's does not), and a read stalled on a slow or hung source would
-// otherwise outlast the shutdown timeout, so the process would exit
-// before restore or deep verify removed its decrypted temporary files.
-type closeOnCancelReader struct {
-	io.ReadCloser
-
-	ctx  context.Context //nolint:containedctx // Read reports its error
-	stop func() bool
-}
-
-func newCloseOnCancelReader(
-	ctx context.Context, rc io.ReadCloser,
-) *closeOnCancelReader {
-	return &closeOnCancelReader{
-		ReadCloser: rc,
-		ctx:        ctx,
-		stop:       context.AfterFunc(ctx, func() { _ = rc.Close() }),
-	}
-}
-
-// Read returns the context's error, not the closed reader's, once ctx
-// has been cancelled.
-func (r *closeOnCancelReader) Read(p []byte) (int, error) {
-	n, err := r.ReadCloser.Read(p)
-	if err != nil && r.ctx.Err() != nil {
-		return n, r.ctx.Err()
-	}
-
-	return n, err
-}
-
-// Close closes the wrapped reader, unless the cancel already closed it.
-func (r *closeOnCancelReader) Close() error {
-	if !r.stop() {
-		return nil
-	}
-
-	return r.ReadCloser.Close()
+	return rc, nil
 }

internal/vaultik/destination_validation_test.go

@@ -161,14 +161,13 @@ func TestVerifyManifestBlobsRejectsShortHash(t *testing.T) {
 		Blobs: []snapshot.BlobInfo{{Hash: "abc", CompressedSize: 1}},
 	}
 
-	verified, missing, mismatched, missingSize, unchecked, err :=
+	verified, missing, mismatched, missingSize, err :=
 		v.verifyManifestBlobs(manifest, &VerifyOptions{JSON: true})
 	require.ErrorIs(t, err, errInvalidBlobHash)
 	require.Zero(t, verified)
 	require.Zero(t, missing)
 	require.Zero(t, mismatched)
 	require.Zero(t, missingSize)
-	require.Empty(t, unchecked)
 }
 
 // TestVerifyBlobChunksRejectsNegativeLength proves a blob_chunks row with a

internal/vaultik/helpers.go

@@ -89,6 +89,22 @@ type SnapshotInfo struct {
 	RemotePresent    *bool            `json:"remote_present"`
 }
 
+// formatBytes formats bytes in a human-readable format
+func formatBytes(bytes int64) string {
+	const unit = 1024
+	if bytes < unit {
+		return fmt.Sprintf("%d B", bytes)
+	}
+
+	div, exp := int64(unit), 0
+	for n := bytes / unit; n >= unit; n /= unit {
+		div *= unit
+		exp++
+	}
+
+	return fmt.Sprintf("%.1f %cB", float64(bytes)/float64(div), "KMGTPE"[exp])
+}
+
 // parseSnapshotTimestamp extracts the timestamp from a snapshot ID
 // Format: hostname_snapshotname_2026-01-12T14:41:15Z
 func parseSnapshotTimestamp(snapshotID string) (time.Time, error) {

internal/vaultik/new_chunk_size_test.go

@@ -1,133 +0,0 @@
-package vaultik_test
-
-import (
-	"bytes"
-	"context"
-	"maps"
-	"path/filepath"
-	"testing"
-
-	"github.com/spf13/afero"
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-	"sneak.berlin/go/vaultik/internal/chunker"
-	"sneak.berlin/go/vaultik/internal/database"
-	"sneak.berlin/go/vaultik/internal/log"
-	"sneak.berlin/go/vaultik/internal/storage"
-	"sneak.berlin/go/vaultik/internal/vaultik"
-)
-
-// TestNewChunkSizeOfChangedFileStaysFixed covers
-// https://git.eeqj.de/sneak/vaultik/issues/311. A backup whose only change
-// is a modified file lists the size of the chunks that change added as its
-// NEW CHUNK SIZE, and a later backup that changes the file again leaves
-// every earlier snapshot's figure as it was. The file's row in the local
-// index is shared by all three snapshots and rewritten by each backup.
-func TestNewChunkSizeOfChangedFileStaysFixed(t *testing.T) {
-	log.Initialize(log.Config{})
-	t.Parallel()
-
-	fs := afero.NewOsFs()
-	tempDir := t.TempDir()
-	dataDir := filepath.Join(tempDir, "src")
-	dbPath := filepath.Join(tempDir, "index.sqlite")
-	changedPath := filepath.Join(dataDir, "changed.bin")
-
-	ctx := context.Background()
-	files := writeFaultSourceTree(t, fs, dataDir)
-	cfg := changedFileConfig(dataDir, dbPath)
-
-	appendRandomBytes(t, fs, files, changedPath, appendedFileSize)
-
-	store, err := storage.NewFileStorer(filepath.Join(tempDir, "remote"))
-	require.NoError(t, err)
-
-	db, err := database.New(ctx, dbPath)
-	require.NoError(t, err)
-	t.Cleanup(func() { _ = db.Close() })
-
-	repos := database.NewRepositories(db)
-	v := newBackupVaultik(ctx, cfg, store, repos, db, fs)
-
-	require.NoError(t, backUp(v, "first"))
-
-	firstFiles := maps.Clone(files)
-	firstID := localSnapshotID(ctx, t, repos, "first")
-
-	appendRandomBytes(t, fs, files, changedPath, faultChunkSize)
-	require.NoError(t, backUp(v, "second"))
-
-	secondID := localSnapshotID(ctx, t, repos, "second")
-	secondAdded := sizeOfNewChunks(t, files, firstFiles)
-	require.Positive(t, secondAdded)
-	assert.Equal(t, secondAdded, listedNewChunkSizes(t, v)[secondID],
-		"the second snapshot must count the chunks the change added")
-
-	appendRandomBytes(t, fs, files, changedPath, faultChunkSize)
-	require.NoError(t, backUp(v, "third"))
-
-	sizes := listedNewChunkSizes(t, v)
-	assert.Equal(t, sizeOfNewChunks(t, firstFiles, nil), sizes[firstID],
-		"a later backup must not change the first snapshot's figure")
-	assert.Equal(t, secondAdded, sizes[secondID],
-		"a later backup must not change the second snapshot's figure")
-}
-
-// sizeOfNewChunks returns the total size of the distinct chunks of the
-// files in after that no file in before has, cut the way the backup cuts
-// them.
-func sizeOfNewChunks(t *testing.T, after, before map[string][]byte) int64 {
-	t.Helper()
-
-	known := make(map[string]bool)
-	for _, chunk := range chunksOf(t, before) {
-		known[chunk.Hash] = true
-	}
-
-	var total int64
-
-	for _, chunk := range chunksOf(t, after) {
-		if !known[chunk.Hash] {
-			known[chunk.Hash] = true
-			total += chunk.Size
-		}
-	}
-
-	return total
-}
-
-// chunksOf returns the chunks of every file in files.
-func chunksOf(t *testing.T, files map[string][]byte) []chunker.Chunk {
-	t.Helper()
-
-	c := chunker.NewChunker(faultChunkSize)
-
-	var all []chunker.Chunk
-
-	for _, content := range files {
-		chunks, err := c.ChunkReader(bytes.NewReader(content))
-		require.NoError(t, err)
-
-		all = append(all, chunks...)
-	}
-
-	return all
-}
-
-// listedNewChunkSizes runs `snapshot list --json` and returns each listed
-// snapshot's new_chunk_size by snapshot ID.
-func listedNewChunkSizes(t *testing.T, v *vaultik.Vaultik) map[string]int64 {
-	t.Helper()
-
-	var out bytes.Buffer
-
-	v.Stdout = &out
-	require.NoError(t, v.ListSnapshots(true))
-
-	sizes := make(map[string]int64)
-	for _, row := range decodeListJSON(t, out.String()) {
-		sizes[row.ID] = row.NewChunkSize
-	}
-
-	return sizes
-}

internal/vaultik/restore_interrupt_test.go

@@ -142,14 +142,14 @@ func TestRestoreCleansTempDirOnInterrupt(t *testing.T) {
 }
 
 // scratchEntries returns the vaultik blob-cache and snapshot-database
-// scratch entries (restore's and deep verify's) currently present in dir.
+// scratch entries currently present in dir.
 func scratchEntries(t *testing.T, dir string) []string {
 	t.Helper()
 
 	var matches []string
 
 	for _, pattern := range []string{
-		"vaultik-blobcache-*", "vaultik-restore-*", "vaultik-verify-*",
+		"vaultik-blobcache-*", "vaultik-restore-*",
 	} {
 		found, err := filepath.Glob(filepath.Join(dir, pattern))
 		require.NoError(t, err)
@@ -206,119 +206,3 @@ func TestRestoreSkipErrorsCancelDuringBlobDownload(t *testing.T) {
 	assert.NotContains(t, out.String(), srcPath,
 		"the cancel was reported as a failed file")
 }
-
-// stallingBlobStorer wraps a Storer so that every blob download hands
-// over the first half of the blob and then stalls, ignoring the context,
-// as a file:// blob on a hung USB stick or network share does. Only
-// closing the returned reader ends the stall.
-type stallingBlobStorer struct {
-	storage.Storer
-
-	once    sync.Once
-	stalled chan struct{}
-}
-
-func newStallingBlobStorer(inner storage.Storer) *stallingBlobStorer {
-	return &stallingBlobStorer{Storer: inner, stalled: make(chan struct{})}
-}
-
-func (s *stallingBlobStorer) Get(
-	ctx context.Context, key string,
-) (io.ReadCloser, error) {
-	rc, err := s.Storer.Get(ctx, key)
-	if err != nil || !strings.HasPrefix(key, "blobs/") {
-		return rc, err
-	}
-
-	blob, err := io.ReadAll(rc)
-	_ = rc.Close()
-
-	if err != nil {
-		return nil, err
-	}
-
-	// The writer is never closed, so once the first half has been read
-	// the next Read blocks until the reader is closed.
-	pr, pw := io.Pipe()
-
-	go func() {
-		_, _ = pw.Write(blob[:len(blob)/2])
-
-		s.once.Do(func() { close(s.stalled) })
-	}()
-
-	return pr, nil
-}
-
-// TestInterruptDuringStalledBlobReadCleansTempDir stops a restore and a
-// deep verify while a blob read is stalled on a source that ignores the
-// context. Stopping must end that read, so the operation unwinds and
-// removes its decrypted scratch files well within the shutdown timeout
-// (issue #325).
-//
-// Not parallel: it points TMPDIR at a private directory (via t.Setenv)
-// so it can assert on exactly the scratch files the operation created.
-func TestInterruptDuringStalledBlobReadCleansTempDir(t *testing.T) {
-	log.Initialize(log.Config{})
-
-	operations := map[string]func(v *Vaultik, snapshotID, targetDir string) error{
-		"restore": func(v *Vaultik, snapshotID, targetDir string) error {
-			return v.Restore(&RestoreOptions{
-				SnapshotID: snapshotID,
-				TargetDir:  targetDir,
-			})
-		},
-		"deep verify": func(v *Vaultik, snapshotID, _ string) error {
-			return v.RunDeepVerify(snapshotID, &VerifyOptions{Deep: true})
-		},
-	}
-
-	for name, op := range operations {
-		t.Run(name, func(t *testing.T) {
-			fs := afero.NewOsFs()
-			root := t.TempDir()
-
-			// Random content, so the blob stays large enough to stall
-			// part-way through its compressed, encrypted body.
-			cfg, storer, snapshotID, _ := backupOneFile(context.Background(),
-				t, fs, root, "a.bin", randomBytes(t, 256*1024), 0o644)
-
-			scratch := filepath.Join(root, "scratch")
-			require.NoError(t, fs.MkdirAll(scratch, 0o755))
-			t.Setenv("TMPDIR", scratch)
-
-			stalling := newStallingBlobStorer(storer)
-
-			ctx, cancel := context.WithCancel(context.Background())
-			defer cancel()
-
-			v := newRestoreVaultik(ctx, cfg, stalling, fs)
-			v.cancel = cancel
-
-			var opErr error
-
-			stop := v.StartOperation(func() {
-				opErr = op(v, snapshotID, filepath.Join(root, "restored"))
-			})
-
-			select {
-			case <-stalling.stalled:
-			case <-time.After(30 * time.Second):
-				t.Fatal("the operation never reached a blob read")
-			}
-
-			require.NotEmpty(t, scratchEntries(t, scratch),
-				"expected decrypted scratch files to exist mid-read")
-
-			stopCtx, stopCancel := context.WithTimeout(
-				context.Background(), 10*time.Second)
-			defer stopCancel()
-
-			require.True(t, stop(stopCtx),
-				"stop timed out; the stalled blob read was not interrupted")
-			require.ErrorIs(t, opErr, context.Canceled)
-			require.Empty(t, scratchEntries(t, scratch),
-				"decrypted scratch files remained after the interrupt")
-		})
-	}
-}

internal/vaultik/shallow_verify_test.go

@@ -4,20 +4,15 @@ import (
 	"bytes"
 	"context"
 	"encoding/json"
-	"errors"
-	"fmt"
 	"io"
 	"os"
-	"path"
 	"path/filepath"
-	"slices"
 	"testing"
 
 	"github.com/spf13/afero"
 	"github.com/stretchr/testify/require"
 	"sneak.berlin/go/vaultik/internal/log"
 	"sneak.berlin/go/vaultik/internal/snapshot"
-	"sneak.berlin/go/vaultik/internal/storage"
 	"sneak.berlin/go/vaultik/internal/ui"
 	"sneak.berlin/go/vaultik/internal/vaultik"
 )
@@ -156,224 +151,6 @@ func TestShallowVerifyDetectsMissingDatabase(t *testing.T) {
 		"shallow verify must fail when db.zst.age is absent")
 }
 
-// errStatDenied stands in for any storage error other than not-found: a
-// permission error, an S3 403, a dropped connection.
-var errStatDenied = errors.New("injected stat failure")
-
-// statFailingStorer fails Stat with errStatDenied for the keys in
-// failKeys and passes every other call to the wrapped Storer.
-type statFailingStorer struct {
-	storage.Storer
-
-	failKeys []string
-}
-
-func (s statFailingStorer) Stat(
-	ctx context.Context, key string,
-) (*storage.ObjectInfo, error) {
-	if slices.Contains(s.failKeys, key) {
-		return nil, fmt.Errorf("stat %q: %w", key, errStatDenied)
-	}
-
-	return s.Storer.Stat(ctx, key)
-}
-
-// TestShallowVerifyReportsStatErrorsApartFromMissing backs up a real
-// snapshot, deletes one blob, and makes storage fail Stat with an error
-// other than not-found for a second blob and for the encrypted database.
-// Only the deleted blob may be counted as missing. The second blob and
-// the database must be reported as could not be checked, with the error
-// storage returned, and verify must fail, in the text output and under
-// --json.
-func TestShallowVerifyReportsStatErrorsApartFromMissing(t *testing.T) {
-	log.Initialize(log.Config{})
-	t.Parallel()
-
-	fs := afero.NewOsFs()
-	tempDir := t.TempDir()
-
-	dataDir := filepath.Join(tempDir, "source")
-	storeDir := filepath.Join(tempDir, "remote")
-	dbPath := filepath.Join(tempDir, "index.sqlite")
-
-	chunkSize := int64(32 * 1024)
-	maxBlobSize := int64(128 * 1024)
-
-	// Random content, so the snapshot spans several blobs.
-	require.NoError(t, fs.MkdirAll(dataDir, 0o755))
-	appendRandomBytes(t, fs, map[string][]byte{},
-		filepath.Join(dataDir, "data.bin"), maxBlobSize*4)
-
-	ctx := context.Background()
-
-	cfg, storer, snapshotID := runFileStorageBackup(
-		ctx, t, fs, dataDir, storeDir, dbPath, chunkSize, maxBlobSize)
-
-	blobKeys := storedBlobKeys(t, storeDir)
-	require.GreaterOrEqual(t, len(blobKeys), 2, "expected at least two blobs")
-
-	missingKey, uncheckedKey := blobKeys[0], blobKeys[1]
-	missingHash, uncheckedHash := path.Base(missingKey), path.Base(uncheckedKey)
-
-	require.NoError(t, os.Remove(filepath.Join(storeDir, missingKey)))
-
-	dbKey := "metadata/" + snapshot.RemoteSnapshotKey(snapshotID) + "/db.zst.age"
-	failing := statFailingStorer{
-		Storer:   storer,
-		failKeys: []string{uncheckedKey, dbKey},
-	}
-
-	newVerifier := func(out io.Writer) *vaultik.Vaultik {
-		v := &vaultik.Vaultik{
-			Config:  cfg,
-			Storage: failing,
-			Fs:      fs,
-			Stdout:  out,
-			Stderr:  io.Discard,
-			UI:      ui.NewWithColor(io.Discard, false),
-		}
-		v.SetContext(ctx)
-
-		return v
-	}
-
-	var out bytes.Buffer
-
-	err := newVerifier(&out).VerifySnapshotWithOptions(
-		snapshotID, &vaultik.VerifyOptions{JSON: true})
-	require.Error(t, err)
-
-	result := decodeVerifyResult(t, out.Bytes())
-	require.Equal(t, "failed", result.Status)
-	require.Equal(t, 1, result.Missing, "only the deleted blob is missing")
-	require.Equal(t, result.BlobCount-2, result.Verified)
-	require.Len(t, result.Unchecked, 1)
-	require.Equal(t, uncheckedHash, result.Unchecked[0].Hash)
-	require.Contains(t, result.Unchecked[0].Error, errStatDenied.Error())
-	require.False(t, result.DatabaseMissing,
-		"a database that could not be checked is not missing")
-	require.Contains(t, result.DatabaseCheckError, errStatDenied.Error())
-
-	out.Reset()
-	err = newVerifier(&out).VerifySnapshotWithOptions(
-		snapshotID, &vaultik.VerifyOptions{})
-	require.Error(t, err)
-
-	text := out.String()
-	require.Contains(t, text, "Missing: "+missingHash)
-	require.NotContains(t, text, "Missing: "+uncheckedHash)
-	require.Contains(t, text, "Could not check: "+uncheckedHash)
-	require.Contains(t, text, "Encrypted database:       could not be checked")
-	require.Contains(t, text, errStatDenied.Error())
-	require.Contains(t, text, "FAILED")
-}
-
-// TestShallowVerifyQuietFailsOnStatErrorAlone backs up a healthy snapshot,
-// then makes storage fail Stat, with an error other than not-found, for
-// one blob alone and then for the encrypted database alone. Each run must
-// fail verify, and under --quiet, which drops the report, the error
-// storage returned must still reach stderr.
-//
-//nolint:paralleltest // captureProcessStderr replaces os.Stderr
-func TestShallowVerifyQuietFailsOnStatErrorAlone(t *testing.T) {
-	log.Initialize(log.Config{})
-
-	fs := afero.NewOsFs()
-	tempDir := t.TempDir()
-
-	dataDir := filepath.Join(tempDir, "source")
-	storeDir := filepath.Join(tempDir, "remote")
-	dbPath := filepath.Join(tempDir, "index.sqlite")
-
-	chunkSize := int64(32 * 1024)
-	maxBlobSize := int64(128 * 1024)
-
-	require.NoError(t, fs.MkdirAll(dataDir, 0o755))
-	require.NoError(t, afero.WriteFile(fs,
-		filepath.Join(dataDir, "data.bin"),
-		bytesPattern("shallow-quiet-", int(maxBlobSize*2)), 0o644))
-
-	ctx := context.Background()
-
-	cfg, storer, snapshotID := runFileStorageBackup(
-		ctx, t, fs, dataDir, storeDir, dbPath, chunkSize, maxBlobSize)
-
-	newQuietVerifier := func(s storage.Storer) *vaultik.Vaultik {
-		v := &vaultik.Vaultik{
-			Config:  cfg,
-			Storage: s,
-			Fs:      fs,
-			Stdout:  io.Discard,
-			Stderr:  io.Discard,
-			UI:      ui.NewWithColor(io.Discard, false),
-		}
-		v.SetContext(ctx)
-		v.UI.SetQuiet(true)
-
-		return v
-	}
-
-	require.NoError(t,
-		newQuietVerifier(storer).VerifySnapshotWithOptions(
-			snapshotID, &vaultik.VerifyOptions{}),
-		"shallow verify should pass on a healthy snapshot")
-
-	blobKeys := storedBlobKeys(t, storeDir)
-	require.NotEmpty(t, blobKeys)
-
-	dbKey := "metadata/" + snapshot.RemoteSnapshotKey(snapshotID) + "/db.zst.age"
-
-	for _, failKey := range []string{blobKeys[0], dbKey} {
-		v := newQuietVerifier(statFailingStorer{
-			Storer:   storer,
-			failKeys: []string{failKey},
-		})
-
-		var err error
-
-		stderr := captureProcessStderr(t, func() {
-			err = v.VerifySnapshotWithOptions(
-				snapshotID, &vaultik.VerifyOptions{})
-		})
-
-		require.Error(t, err,
-			"verify must fail when %s could not be checked", failKey)
-		require.Contains(t, stderr, errStatDenied.Error(),
-			"under --quiet the error for %s must still reach stderr", failKey)
-	}
-}
-
-// storedBlobKeys returns the storage key of every blob file under
-// storeDir, sorted.
-func storedBlobKeys(t *testing.T, storeDir string) []string {
-	t.Helper()
-
-	var keys []string
-
-	err := filepath.Walk(filepath.Join(storeDir, "blobs"),
-		func(p string, info os.FileInfo, err error) error {
-			if err != nil {
-				return err
-			}
-
-			if info.IsDir() {
-				return nil
-			}
-
-			rel, err := filepath.Rel(storeDir, p)
-			if err != nil {
-				return err
-			}
-
-			keys = append(keys, filepath.ToSlash(rel))
-
-			return nil
-		})
-	require.NoError(t, err)
-
-	return keys
-}
-
 // growOneBlob appends bytes to the first blob file found under blobsDir,
 // changing its on-disk size so it no longer matches the manifest.
 func growOneBlob(t *testing.T, fs afero.Fs, blobsDir string) {

internal/vaultik/snapshot.go

@@ -14,7 +14,6 @@ import (
 
 	"sneak.berlin/go/vaultik/internal/log"
 	"sneak.berlin/go/vaultik/internal/snapshot"
-	"sneak.berlin/go/vaultik/internal/storage"
 	"sneak.berlin/go/vaultik/internal/types"
 )
 
@@ -29,14 +28,8 @@ var (
 	errNoAgeRecipients      = errors.New(
 		"creating a snapshot needs at least one public key in " +
 			"age_recipients (generate a keypair with: age-keygen)")
-	errKeepNewerThanNeedsPrune = errors.New("--keep-newer-than requires --prune")
 )
 
-// ErrPostBackupPruneFailed wraps a failure of the post-backup prune,
-// which runs only after every snapshot was created and uploaded, so the
-// caller can report it apart from a failed backup.
-var ErrPostBackupPruneFailed = errors.New("post-backup prune failed")
-
 // listRecentLimit caps how many snapshot rows are fetched from the
 // local index for listing and purge operations.
 const listRecentLimit = 10000
@@ -52,26 +45,6 @@ type SnapshotCreateOptions struct {
 	Snapshots     []string // Optional list of snapshot names to process (empty = all)
 }
 
-// Validate fails when KeepNewerThan is set without Prune or is not a
-// valid duration. Only the post-backup prune reads KeepNewerThan, after
-// the backup, so the caller checks it before starting one.
-func (opts *SnapshotCreateOptions) Validate() error {
-	if opts.KeepNewerThan == "" {
-		return nil
-	}
-
-	if !opts.Prune {
-		return errKeepNewerThanNeedsPrune
-	}
-
-	_, err := parseDuration(opts.KeepNewerThan)
-	if err != nil {
-		return fmt.Errorf("--keep-newer-than: %w", err)
-	}
-
-	return nil
-}
-
 // CreateSnapshot executes the snapshot creation operation
 func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
 	// config.Load accepts an empty list, since listing, verifying and
@@ -150,7 +123,7 @@ func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
 	if opts.Prune {
 		err = v.runPostBackupPrune(snapshotNames, opts.KeepNewerThan)
 		if err != nil {
-			return fmt.Errorf("%w: %w", ErrPostBackupPruneFailed, err)
+			return fmt.Errorf("post-backup prune: %w", err)
 		}
 	}
 
@@ -210,7 +183,6 @@ type snapshotStats struct {
 	totalFiles         int
 	totalBytes         int64
 	totalChunks        int
-	newChunkSize       int64
 	totalBlobs         int
 	totalBytesSkipped  int64
 	totalFilesSkipped  int
@@ -343,7 +315,6 @@ func (v *Vaultik) scanAllDirectories(
 		stats.totalFiles += result.FilesScanned
 		stats.totalBytes += result.BytesScanned
 		stats.totalChunks += result.ChunksCreated
-		stats.newChunkSize += result.NewChunkSize
 		stats.totalBlobs += result.BlobsCreated
 		stats.totalFilesSkipped += result.FilesSkipped
 		stats.totalFilesFailed += result.FilesFailed
@@ -406,7 +377,6 @@ func (v *Vaultik) finalizeSnapshotMetadata(
 		BlobUncompressedSize: stats.blobUncompressedSize,
 		CompressionLevel:     v.Config.CompressionLevel,
 		UploadDurationMs:     stats.uploadDuration.Milliseconds(),
-		NewChunkSize:         stats.newChunkSize,
 	}
 
 	err = v.SnapshotManager.UpdateSnapshotStatsExtended(v.ctx, snapshotID, extStats)
@@ -637,7 +607,7 @@ func (v *Vaultik) confirmAndExecutePurge(
 			v.stdoutf("  %s (%s, %s)\n",
 				snap.ID,
 				snap.Timestamp.Format("2006-01-02 15:04:05"),
-				ubytes(snap.CompressedSize))
+				formatBytes(snap.CompressedSize))
 		}
 	}
 
@@ -778,10 +748,18 @@ func (v *Vaultik) VerifySnapshotWithOptions(
 		v.stdoutf("Checking blob presence and sizes...\n")
 	}
 
-	v.checkSnapshotDatabase(remoteKey, result)
+	// A snapshot is only restorable if its encrypted database is present
+	// alongside the blobs. Shallow verify checks that the object exists; it
+	// does not decrypt it (that is deep verify's job).
+	dbPath := fmt.Sprintf("metadata/%s/db.zst.age", remoteKey)
 
-	result.Verified, result.Missing, result.Mismatched, result.MissingSize,
-		result.Unchecked, err = v.verifyManifestBlobs(manifest, opts)
+	_, dbErr := v.Storage.Stat(v.ctx, dbPath)
+	if dbErr != nil {
+		result.DatabaseMissing = true
+	}
+
+	result.Verified, result.Missing, result.Mismatched, result.MissingSize, err =
+		v.verifyManifestBlobs(manifest, opts)
 	if err != nil {
 		if opts.JSON {
 			result.Status = verifyStatusFailed
@@ -818,46 +796,25 @@ func (v *Vaultik) printVerifyHeader(snapshotID string, opts *VerifyOptions) {
 	}
 }
 
-// checkSnapshotDatabase records in result whether the snapshot's
-// encrypted database is missing or could not be checked. A snapshot is
-// only restorable if its database is present alongside the blobs.
-// Shallow verify checks that the object exists; it does not decrypt it
-// (that is deep verify's job).
-func (v *Vaultik) checkSnapshotDatabase(remoteKey string, result *VerifyResult) {
-	dbPath := fmt.Sprintf("metadata/%s/db.zst.age", remoteKey)
-
-	_, err := v.Storage.Stat(v.ctx, dbPath)
-
-	switch {
-	case errors.Is(err, storage.ErrNotFound):
-		result.DatabaseMissing = true
-	case err != nil:
-		result.DatabaseCheckError = err.Error()
-	}
-}
-
 // verifyManifestBlobs checks that each blob in the manifest is present in
 // storage with the size the manifest records, returning the counts of
 // blobs that were present with the right size, absent, and present but the
-// wrong size, the total bytes of the absent blobs, and the blobs it could
-// not check because storage returned an error other than not-found. It
-// does not read blob contents; deep verification (RunDeepVerify) does
-// that. The size comparison matches the deep path (see
-// verifyBlobExistenceFromDB).
+// wrong size, plus the total bytes of the absent blobs. It does not read
+// blob contents; deep verification (RunDeepVerify) does that. The size
+// comparison matches the deep path (see verifyBlobExistenceFromDB).
 func (v *Vaultik) verifyManifestBlobs(
 	manifest *snapshot.Manifest, opts *VerifyOptions,
-) (int, int, int, int64, []UncheckedBlob, error) {
+) (int, int, int, int64, error) {
 	var (
 		verified, missing, mismatched int
 		missingSize                   int64
-		unchecked                     []UncheckedBlob
 	)
 
 	for _, blob := range manifest.Blobs {
 		// The manifest is unauthenticated, so its blob hashes are checked
 		// before being spliced into a storage path.
 		if !isBlobHash(blob.Hash) {
-			return 0, 0, 0, 0, nil, fmt.Errorf("%w: %s",
+			return 0, 0, 0, 0, fmt.Errorf("%w: %s",
 				errInvalidBlobHash, shortHash(blob.Hash))
 		}
 
@@ -866,7 +823,7 @@ func (v *Vaultik) verifyManifestBlobs(
 
 		stat, err := v.Storage.Stat(v.ctx, blobPath)
 		switch {
-		case errors.Is(err, storage.ErrNotFound):
+		case err != nil:
 			if !opts.JSON && !v.UI.Quiet() {
 				v.stdoutf("  Missing: %s (%s)\n",
 					blob.Hash, ubytes(blob.CompressedSize))
@@ -874,13 +831,6 @@ func (v *Vaultik) verifyManifestBlobs(
 
 			missing++
 			missingSize += blob.CompressedSize
-		case err != nil:
-			if !opts.JSON && !v.UI.Quiet() {
-				v.stdoutf("  Could not check: %s (%v)\n", blob.Hash, err)
-			}
-
-			unchecked = append(unchecked,
-				UncheckedBlob{Hash: blob.Hash, Error: err.Error()})
 		case stat.Size != blob.CompressedSize:
 			if !opts.JSON && !v.UI.Quiet() {
 				v.stdoutf("  Wrong size: %s (store has %s, manifest lists %s)\n",
@@ -893,7 +843,7 @@ func (v *Vaultik) verifyManifestBlobs(
 		}
 	}
 
-	return verified, missing, mismatched, missingSize, unchecked, nil
+	return verified, missing, mismatched, missingSize, nil
 }
 
 // formatVerifyResult outputs the final verification results as JSON or
@@ -914,19 +864,9 @@ func (v *Vaultik) formatVerifyResult(
 		return v.outputVerifyJSON(result)
 	}
 
-	// --quiet drops the report but not the errors storage returned, which
-	// are logged on stderr. A failure is still returned either way, and
-	// the cli layer prints it on stderr.
-	if v.UI.Quiet() {
-		for _, blob := range result.Unchecked {
-			log.Warn("Could not check blob", "blob", blob.Hash, "error", blob.Error)
-		}
-
-		if result.DatabaseCheckError != "" {
-			log.Warn("Could not check the encrypted database",
-				"snapshot", result.SnapshotID, "error", result.DatabaseCheckError)
-		}
-	} else {
+	// Under --quiet a failure is still returned, and the cli layer
+	// prints it on stderr.
+	if !v.UI.Quiet() {
 		v.printVerifySummary(result, failure)
 	}
 
@@ -952,19 +892,10 @@ func (v *Vaultik) printVerifySummary(result *VerifyResult, failure string) {
 		v.stdoutf("  Wrong size:               %d blobs\n", result.Mismatched)
 	}
 
-	if len(result.Unchecked) > 0 {
-		v.stdoutf("  Could not check:          %d blobs\n", len(result.Unchecked))
-	}
-
 	if result.DatabaseMissing {
 		v.stdoutf("  Encrypted database:       missing\n")
 	}
 
-	if result.DatabaseCheckError != "" {
-		v.stdoutf("  Encrypted database:       could not be checked (%s)\n",
-			result.DatabaseCheckError)
-	}
-
 	v.stdoutf("  Status:       ")
 
 	if failure != "" {
@@ -992,19 +923,10 @@ func shallowVerifyFailure(result *VerifyResult) string {
 			fmt.Sprintf("%d blobs have the wrong size", result.Mismatched))
 	}
 
-	if len(result.Unchecked) > 0 {
-		parts = append(parts, fmt.Sprintf("%d blobs could not be checked",
-			len(result.Unchecked)))
-	}
-
 	if result.DatabaseMissing {
 		parts = append(parts, "the encrypted database is missing")
 	}
 
-	if result.DatabaseCheckError != "" {
-		parts = append(parts, "the encrypted database could not be checked")
-	}
-
 	return strings.Join(parts, "; ")
 }
 

internal/vaultik/snapshot_list.go

@@ -553,8 +553,8 @@ func (v *Vaultik) printSnapshotTable(snapshots []SnapshotInfo) error {
 
 		if snap.LocallyTracked {
 			id = snap.ID.String()
-			uncompressed = ubytes(snap.UncompressedSize)
-			newChunks = ubytes(snap.NewChunkSize)
+			uncompressed = formatBytes(snap.UncompressedSize)
+			newChunks = formatBytes(snap.NewChunkSize)
 		} else {
 			id = formatRemoteOnlyID(snap.RemoteKey)
 			uncompressed = remoteOnlyCell
@@ -564,7 +564,7 @@ func (v *Vaultik) printSnapshotTable(snapshots []SnapshotInfo) error {
 		_, err = fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n",
 			id,
 			snap.Timestamp.Format("2006-01-02 15:04:05"),
-			ubytes(snap.CompressedSize),
+			formatBytes(snap.CompressedSize),
 			uncompressed,
 			newChunks)
 		if err != nil {

internal/vaultik/snapshot_list_test.go

@@ -212,8 +212,8 @@ const (
 	listConfiguredName = "home"
 	listLocalID        = "testhost_home_2026-03-01T10:00:00Z"
 	listRemoteID       = "otherhost_media_2026-03-02T11:22:33Z"
-	// fiveMegabytes is printed as "5.0 MB" by every command.
-	fiveMegabytes = 5_000_000
+	// fiveMegabytes formats as "5.0 MB" through formatBytes.
+	fiveMegabytes = 5 * 1024 * 1024
 )
 
 // TestListSnapshots_RemoteWithoutSecretKey is the regression guard for
@@ -306,48 +306,6 @@ func TestListSnapshots_RemoteOnlyRowRendering(t *testing.T) {
 		"are not in the local index")
 }
 
-// TestSnapshotListAndPurgePrintSizesLikeRemoteInfo checks that the
-// `snapshot list` table and the purge listing print a byte count the way
-// `remote info` does: in powers of 1000, as the MB label says.
-func TestSnapshotListAndPurgePrintSizesLikeRemoteInfo(t *testing.T) {
-	log.Initialize(log.Config{})
-	t.Parallel()
-
-	const wantSize = "5.0 MB" // fiveMegabytes
-
-	env := newListEnv(t)
-
-	// A remote-only snapshot: `snapshot list` and `remote info` both take
-	// its size from the manifest.
-	remoteKey := env.addRemote(t, listRemoteID,
-		time.Date(2026, 3, 2, 11, 22, 33, 0, time.UTC))
-
-	// A locally tracked snapshot of the same size for the purge to list.
-	// The purge prints the snapshot's blob size, the last argument here.
-	start := time.Date(2026, 3, 1, 10, 0, 0, 0, time.UTC)
-	env.addLocal(t, listLocalID, start)
-	env.addRemote(t, listLocalID, start)
-	require.NoError(t, env.v.Repositories.Snapshots.UpdateCounts(
-		context.Background(), nil, listLocalID, 0, 0, 0, 0, fiveMegabytes))
-
-	require.NoError(t, env.v.RemoteInfo(false))
-	assert.Contains(t, findTableRow(t, env.stdout.String(), remoteKey[:16]),
-		wantSize, "remote info")
-
-	env.stdout.Reset()
-	require.NoError(t, env.v.ListSnapshots(false))
-	assert.Contains(t, findTableRow(t, env.stdout.String(),
-		"<remote only:"+remoteKey[:12]+">"), wantSize, "snapshot list")
-
-	// Without --force and with nothing on stdin the purge lists what it
-	// would delete, reads no confirmation, and deletes nothing.
-	env.stdout.Reset()
-	require.NoError(t, env.v.PurgeSnapshotsWithOptions(
-		&vaultik.SnapshotPurgeOptions{OlderThan: "1d"}))
-	assert.Contains(t, findTableRow(t, env.stdout.String(), listLocalID),
-		wantSize, "snapshot purge")
-}
-
 // findTableRow returns the single output line containing needle.
 func findTableRow(t *testing.T, out, needle string) string {
 	t.Helper()
@@ -487,7 +445,6 @@ type listJSONRow struct {
 	RemoteKey      string `json:"remote_key"`
 	Timestamp      string `json:"timestamp"`
 	CompressedSize int64  `json:"compressed_size"`
-	NewChunkSize   int64  `json:"new_chunk_size"`
 	LocallyTracked bool   `json:"locally_tracked"`
 	RemotePresent  *bool  `json:"remote_present"`
 }

internal/vaultik/symlinked_destination_test.go

@@ -0,0 +1,121 @@
+package vaultik_test
+
+import (
+	"context"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"sneak.berlin/go/vaultik/internal/log"
+	"sneak.berlin/go/vaultik/internal/vaultik"
+)
+
+// TestPruneDeletesNothingThroughSymlinkedDirectory checks that prune keeps
+// every blob and the local snapshot record when the metadata/ or blobs/
+// directory of a file:// destination is a symlink to a directory elsewhere
+// (https://git.eeqj.de/sneak/vaultik/issues/296). A backup writes through
+// such a link, so prune has to list through it too.
+func TestPruneDeletesNothingThroughSymlinkedDirectory(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	for _, dir := range []string{"metadata", "blobs"} {
+		t.Run(dir, func(t *testing.T) {
+			t.Parallel()
+
+			ctx := context.Background()
+			storeDir := filepath.Join(t.TempDir(), "store")
+			v, repos, _ := backUpToFileDestination(ctx, t, storeDir)
+			blobFiles := blobFilesIn(t, storeDir)
+			moveBehindSymlink(t, storeDir, dir)
+
+			require.NoError(t, v.Prune(&vaultik.PruneOptions{Force: true}))
+
+			// When blobs/ is the link, these paths run through it.
+			for _, blobFile := range blobFiles {
+				assert.FileExists(t, blobFile, "prune must delete no blob")
+			}
+
+			snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
+			require.NoError(t, err)
+			assert.Len(t, snapshots, 1, "prune must delete no local snapshot record")
+		})
+	}
+}
+
+// TestPruneFailsOnSymlinkedDirectoryWithMissingTarget checks that prune
+// fails, deleting no blob and keeping the local snapshot record, when the
+// metadata/ or blobs/ directory of a file:// destination is a symlink
+// whose target is missing, as a link to a volume that is not mounted is.
+func TestPruneFailsOnSymlinkedDirectoryWithMissingTarget(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	for _, dir := range []string{"metadata", "blobs"} {
+		t.Run(dir, func(t *testing.T) {
+			t.Parallel()
+
+			ctx := context.Background()
+			storeDir := filepath.Join(t.TempDir(), "store")
+			v, repos, _ := backUpToFileDestination(ctx, t, storeDir)
+			blobFiles := blobFilesIn(t, storeDir)
+			targetPath := moveBehindSymlink(t, storeDir, dir)
+
+			// Unmount the volume the link points to, and mount it again
+			// once prune has run.
+			unmountedPath := targetPath + "-unmounted"
+			require.NoError(t, os.Rename(targetPath, unmountedPath))
+
+			err := v.Prune(&vaultik.PruneOptions{Force: true})
+			require.ErrorIs(t, err, fs.ErrNotExist)
+
+			require.NoError(t, os.Rename(unmountedPath, targetPath))
+
+			// When blobs/ is the link, these paths run through it.
+			for _, blobFile := range blobFiles {
+				assert.FileExists(t, blobFile, "prune must delete no blob")
+			}
+
+			snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
+			require.NoError(t, err)
+			assert.Len(t, snapshots, 1, "prune must delete no local snapshot record")
+		})
+	}
+}
+
+// blobFilesIn returns the paths of the blob files under storeDir/blobs, and
+// fails the test when there are none.
+func blobFilesIn(t *testing.T, storeDir string) []string {
+	t.Helper()
+
+	var blobFiles []string
+
+	err := filepath.WalkDir(filepath.Join(storeDir, "blobs"),
+		func(path string, entry fs.DirEntry, err error) error {
+			if err == nil && !entry.IsDir() {
+				blobFiles = append(blobFiles, path)
+			}
+
+			return err
+		})
+	require.NoError(t, err)
+	require.NotEmpty(t, blobFiles)
+
+	return blobFiles
+}
+
+// moveBehindSymlink moves storeDir/dir to a directory elsewhere, puts a
+// symlink to it in its place, and returns where it went.
+func moveBehindSymlink(t *testing.T, storeDir, dir string) string {
+	t.Helper()
+
+	linkPath := filepath.Join(storeDir, dir)
+	targetPath := filepath.Join(t.TempDir(), dir)
+	require.NoError(t, os.Rename(linkPath, targetPath))
+	require.NoError(t, os.Symlink(targetPath, linkPath))
+
+	return targetPath
+}

internal/vaultik/verify.go

@@ -57,25 +57,11 @@ type VerifyResult struct {
 	Missing     int    `json:"missing"`
 	MissingSize int64  `json:"missing_size,omitempty"`
 	Mismatched  int    `json:"mismatched,omitempty"`
-	// Unchecked lists the blobs shallow verify could not check, because
-	// storage answered with an error other than not-found, so whether
-	// they are present is unknown.
-	Unchecked []UncheckedBlob `json:"unchecked,omitempty"`
 	// DatabaseMissing is set by shallow verify when the snapshot's
 	// encrypted database (metadata/<key>/db.zst.age) is absent, which
 	// makes the snapshot unrestorable regardless of the blobs.
-	DatabaseMissing bool `json:"database_missing,omitempty"`
-	// DatabaseCheckError is set by shallow verify to the error storage
-	// returned when the encrypted database could not be checked.
-	DatabaseCheckError string `json:"database_check_error,omitempty"`
-	ErrorMessage       string `json:"error,omitempty"`
-}
-
-// UncheckedBlob is a blob shallow verify could not check, with the error
-// storage returned for it.
-type UncheckedBlob struct {
-	Hash  string `json:"hash"`
-	Error string `json:"error"`
+	DatabaseMissing bool   `json:"database_missing,omitempty"`
+	ErrorMessage    string `json:"error,omitempty"`
 }
 
 // deepVerifyFailure records a failure in the result and returns it appropriately