Record the defect-hunt fixes in TODO.md Completed Steps (closes #334)

Adds a Completed Steps entry, newest first, for every issue from the
defect hunt (#288) that a merged
fix closed and that had none: the hunt's own issues, those filed while
fixing them, and the owner's ruling on
#239. Each entry says in one or
two sentences what was wrong and what changed; the detail is in each
fix's commit. Next Step now says the hunt's fixes have landed.

Judgement call: #332, closed as
covered by the fix for #296, is
named in that entry rather than given its own.
Judgement call: #351, a Docker
build fix filed while gating these fixes, gets an entry.

Model: opus-5-5

ARCHITECTURE.md

@@ -359,7 +359,7 @@ CreateSnapshot(opts)
 
 ## Deduplication Strategy
 
-1. **File-level**: Files unchanged since last backup are skipped (metadata comparison: size, mtime, mode, uid, gid), unless the file lists a chunk that no uploaded blob holds; such a file is re-chunked
+1. **File-level**: Files and symlinks unchanged since last backup are skipped (metadata comparison: size, mtime, mode, uid, gid, and a symlink's target), unless the file lists a chunk that no uploaded blob holds; such a file is re-chunked
 
 2. **Chunk-level**: Chunks are content-addressed by SHA256 hash. If a chunk hash already exists in the database, the chunk data is not re-uploaded.
 
@@ -385,11 +385,12 @@ bucket/
 A destination without `vaultik-destination.txt` is treated as not present
 while the local index holds snapshots stored there (`checkDestinationMarker`
 in `internal/vaultik/destination_marker.go`): `snapshot create` refuses to
-write to it, `prune` and `snapshot purge` fail, and `snapshot list` and
+write to it, `prune`, `snapshot purge` and `remote nuke` fail, and
-`snapshot remove` behave as for a destination that cannot be listed. This
+`snapshot list` and `snapshot remove` behave as for a destination that cannot
-keeps those commands from taking an unplugged volume's empty mount point for
+be listed. This keeps those commands from taking an unplugged volume's empty
-a destination whose snapshots are all gone. The `file://` backend never
+mount point for a destination whose snapshots are all gone. The `file://`
-creates its destination directory, so a write to a missing one fails.
+backend never creates its destination directory, so a write to a missing one
+fails.
 
 The `{remote-key}` directory name is a one-way double SHA-256 hash of the human
 snapshot ID, so the human ID (hostname, snapshot name, timestamp) is never

README.md

@@ -182,7 +182,7 @@ vaultik version
 * `--verbose`, `-v`: Enable verbose output (on stderr — see below)
 * `--debug`: Enable debug output (on stderr — see below)
 * `--quiet`, `-q`: Suppress non-error output (also suppresses startup banner)
-* `--skip-errors`: Skip files that cannot be read when creating a snapshot, or that cannot be restored when restoring, instead of aborting. Packing and storage errors while creating a snapshot (which would leave a chunk recorded but not stored) still abort the run.
+* `--skip-errors`: Skip files that cannot be read when creating a snapshot, or that cannot be restored when restoring, instead of aborting. Packing and storage errors while creating a snapshot (which would leave a chunk recorded but not stored) still abort the run. A run that skipped anything still finishes but exits non-zero: `snapshot create` keeps the snapshot and counts each file or directory it skipped as failed. A file removed during the backup is also counted as failed but does not change the exit status.
 
 ### locking
 
@@ -297,7 +297,11 @@ value where a list belongs or a word where a number belongs), is refused.
   (for crontab)
 * `--prune`: After backup, drop older snapshots of each backed-up name and
   remove orphaned blobs from remote storage. By default keeps only the latest
-  snapshot per name; use `--keep-newer-than` for a rolling window.
+  snapshot per name; use `--keep-newer-than` for a rolling window. When
+  `--skip-errors` skipped a file or directory it could not read or list, the
+  prune does not run and every older snapshot is kept, since one of them may
+  be the only one that holds what was skipped; the command says so and exits
+  non-zero. A file removed during the backup does not stop the prune.
 * `--keep-newer-than <duration>`: With `--prune`, keep snapshots newer than
   this duration instead of only the latest (e.g. `4w`, `30d`, `6mo`, `1y`).
   Without `--prune`, or with a value that is not a valid duration, the
@@ -454,9 +458,14 @@ since `prune` never deletes it.
 the backup destination store, whatever its name, and then
 `vaultik-destination.txt`, leaving the bucket prefix empty. Destructive
 and irreversible. This is the single supported way to wipe the entire
-destination store. If a listing afterwards still finds anything under
+destination store. A snapshot is removed from the local index only once
-`metadata/` or `blobs/`, the command names each object left and exits
+its metadata is deleted from the destination store. If the destination
-non-zero; running it again deletes what is left.
+store cannot be listed, or the local index holds snapshots and the
+destination has no `vaultik-destination.txt` (an unplugged volume's
+empty mount point), the command fails before it removes anything. If a
+listing afterwards still finds anything under `metadata/` or `blobs/`,
+the command names each object left and exits non-zero; running it again
+deletes what is left.
 * `--force`: Required to confirm destruction.
 
 **`database delete`**: Delete the local SQLite state database file
@@ -529,10 +538,11 @@ For an implementation-level view of the internals — the data model, the
 * `vaultik-destination.txt` holds one fixed line of text, with no host
   name, path or time. A destination without it is treated as not
   present while the local index holds snapshots stored there:
-  `snapshot create` refuses to write to it, `prune` and `snapshot purge`
+  `snapshot create` refuses to write to it, `prune`, `snapshot purge`
-  fail, and `snapshot list` and `snapshot remove` behave as for a
+  and `remote nuke` fail, and `snapshot list` and `snapshot remove`
-  destination that cannot be listed. This keeps an unplugged volume's empty mount point
+  behave as for a destination that cannot be listed. This keeps an
-  from being taken for a destination whose snapshots are all gone. Mount
+  unplugged volume's empty mount point from being taken for a
+  destination whose snapshots are all gone. Mount
   the volume if it is removable; if the destination was emptied on
   purpose, run `vaultik database delete` and back up again.
 * Blobs are two-level directory sharded using the first 4 hex chars of the blob hash

TODO.md

@@ -15,13 +15,229 @@ pre-1.0
 # Next Step
 
 The 1.0 work is complete on `next`: the scope settled on
-[issue #125](https://git.eeqj.de/sneak/vaultik/issues/125) was the work
+[issue #125](https://git.eeqj.de/sneak/vaultik/issues/125) has landed,
-already planned for 1.0, and all of it has landed. The mechanism to cut
+and so has every fix from the defect hunt before the release candidate
-the tag exists and is exercised; what is left is merging `next` to
+([issue #288](https://git.eeqj.de/sneak/vaultik/issues/288)). What is
-`main` and tagging, both the owner's.
+left is merging `next` to `main` and tagging the release, both the
+owner's; a `v*` tag runs the release build in
+`.gitea/workflows/release.yml`.
 
 # Completed Steps
 
+- 2026-10-14: Stopped `snapshot create --prune` pruning after a backup
+  that skipped files it could not read
+  ([issue #382](https://git.eeqj.de/sneak/vaultik/issues/382)). Such a
+  prune could delete the only snapshot still holding those files.
+
+- 2026-10-14: Made `snapshot create --skip-errors` exit non-zero,
+  instead of 0, when it skips anything it cannot read
+  ([issue #310](https://git.eeqj.de/sneak/vaultik/issues/310)).
+
+- 2026-10-14: Kept the local index when `remote nuke` cannot list or
+  find the destination store
+  ([issue #292](https://git.eeqj.de/sneak/vaultik/issues/292)). Nuke
+  had cleared the local snapshot records anyway; it now fails before
+  removing anything.
+
+- 2026-10-14: Stopped counting directories and unchanged symlinks as
+  files to back up
+  ([issue #305](https://git.eeqj.de/sneak/vaultik/issues/305)).
+
+- 2026-10-14: Refused a backup destination that an unplugged volume
+  left empty, as the owner ruled
+  ([issue #239](https://git.eeqj.de/sneak/vaultik/issues/239)). Every
+  backup now writes `vaultik-destination.txt` at the destination root,
+  and while the local index holds snapshots stored there, a destination
+  without it is treated as not present.
+
+- 2026-10-14: Warned about restored ownership only when an entry's
+  owner or group differs from the recorded one
+  ([issue #316](https://git.eeqj.de/sneak/vaultik/issues/316)); a
+  restore by a non-root user had warned even when none did.
+
+- 2026-10-14: Made `snapshot list` report a changed destination instead
+  of advising `prune`, which refuses in that case
+  ([issue #317](https://git.eeqj.de/sneak/vaultik/issues/317)).
+
+- 2026-10-14: Counted a file as deleted by a backup only if the
+  previous snapshot of the same name listed it
+  ([issue #304](https://git.eeqj.de/sneak/vaultik/issues/304)); files
+  that snapshot did not list had been counted too.
+
+- 2026-10-14: Named each prefix once in restore, verify and purge
+  errors, some of which repeated it
+  ([issue #327](https://git.eeqj.de/sneak/vaultik/issues/327)). A deep
+  verify log line that named S3 whatever the destination now names
+  storage.
+
+- 2026-10-14: Refused credentials in a storage URL's endpoint or rclone
+  remote name, which were accepted and could then be printed
+  ([issue #322](https://git.eeqj.de/sneak/vaultik/issues/322)).
+
+- 2026-10-14: Ran `$EDITOR` through the shell in `config edit`, so a
+  value with arguments, such as `code --wait`, no longer fails
+  ([issue #315](https://git.eeqj.de/sneak/vaultik/issues/315)).
+
+- 2026-10-14: Made `snapshot remove` fail, instead of exiting 0, when
+  it cannot delete the metadata of a snapshot that is only on the
+  destination store
+  ([issue #340](https://git.eeqj.de/sneak/vaultik/issues/340)). The
+  README and the help now say when it prints the `vaultik prune`
+  follow-up.
+
+- 2026-10-14: Labelled the totals in the local index cleanup before
+  each backup as totals
+  ([issue #306](https://git.eeqj.de/sneak/vaultik/issues/306)). Each
+  was shown as `(N remain)`, which read as rows left to clean up.
+
+- 2026-10-13: Freed a cached blob once no file still to restore needs
+  it ([issue #324](https://git.eeqj.de/sneak/vaultik/issues/324)); one
+  also used by a file outside the selected paths had stayed in the
+  temporary cache until the restore ended.
+
+- 2026-10-13: Kept the whole startup error when it contains `): `
+  ([issue #320](https://git.eeqj.de/sneak/vaultik/issues/320)). Such
+  an error was cut at its last `): `, so the `blob_size_limit` one lost
+  the setting's name.
+
+- 2026-10-13: Silenced the status lines of `prune` and snapshot removal
+  under `--cron` and `--quiet`
+  ([issue #294](https://git.eeqj.de/sneak/vaultik/issues/294)).
+
+- 2026-10-13: Read the rclone config file that `RCLONE_CONFIG` names
+  ([issue #321](https://git.eeqj.de/sneak/vaultik/issues/321)); a
+  remote defined only there had been reported as not found.
+
+- 2026-10-13: Made `/**/` in an exclude pattern match zero directories,
+  as in gitignore
+  ([issue #338](https://git.eeqj.de/sneak/vaultik/issues/338));
+  `a/**/*.log` had missed `a/x.log`.
+
+- 2026-10-13: Made the `chunk_size` and `blob_size_limit` defaults and
+  the `chunk_size` minimum 1000-based, like the size parser and the
+  docs ([issue #313](https://git.eeqj.de/sneak/vaultik/issues/313)).
+
+- 2026-10-13: Kept a snapshot's remote metadata when `snapshot purge`
+  cannot remove it from the local index
+  ([issue #337](https://git.eeqj.de/sneak/vaultik/issues/337)); the
+  purge had deleted the metadata anyway and exited 0.
+
+- 2026-10-13: Made `snapshot restore` fail for a requested path that
+  matches nothing, which it had ignored
+  ([issue #309](https://git.eeqj.de/sneak/vaultik/issues/309)).
+
+- 2026-10-13: Made `remote nuke` delete every object under `metadata/`
+  and `blobs/`
+  ([issue #291](https://git.eeqj.de/sneak/vaultik/issues/291)); it had
+  left some, such as a stray `.DS_Store`, while reporting the store
+  empty.
+
+- 2026-10-13: Made `config set` create a list for an index under a
+  missing key
+  ([issue #308](https://git.eeqj.de/sneak/vaultik/issues/308)); it had
+  written a map, after which no command could load the config.
+
+- 2026-10-13: Listed a `file://` destination through a symlinked
+  `metadata/` or `blobs/`, which had listed as empty
+  ([issue #296](https://git.eeqj.de/sneak/vaultik/issues/296)). A link
+  to a missing target now fails the listing
+  ([issue #332](https://git.eeqj.de/sneak/vaultik/issues/332)).
+
+- 2026-10-13: Made `snapshot verify --deep --json` report its blob
+  counts ([issue #314](https://git.eeqj.de/sneak/vaultik/issues/314));
+  a failed run had reported 0 missing and 0 verified.
+
+- 2026-10-13: Limited and retried the module download in the Docker
+  build ([issue #351](https://git.eeqj.de/sneak/vaultik/issues/351)),
+  which had hung for as long as a stalled connection to the module
+  proxy lasted.
+
+- 2026-10-13: Recorded each snapshot's uncompressed size when its
+  backup runs
+  ([issue #360](https://git.eeqj.de/sneak/vaultik/issues/360)). A later
+  backup could change an older snapshot's UNCOMPRESSED SIZE in
+  `snapshot list`.
+
+- 2026-10-13: Stopped `snapshot restore --skip-errors --verify`
+  reporting a file that failed to restore again as a checksum failure
+  ([issue #323](https://git.eeqj.de/sneak/vaultik/issues/323)).
+
+- 2026-10-13: Recorded the stored size of a blob a backup finds already
+  stored, not the size of its local copy, which can differ, so
+  `snapshot verify` could fail on a good snapshot
+  ([issue #300](https://git.eeqj.de/sneak/vaultik/issues/300)).
+
+- 2026-10-13: Stopped the local index and the blob upload step ending
+  the process on a failed close of query results or a failed temp file
+  cleanup ([issue #345](https://git.eeqj.de/sneak/vaultik/issues/345));
+  both are now returned as errors.
+
+- 2026-10-13: Closed the snapshot database's storage reader too when
+  restore or deep verify is interrupted
+  ([issue #356](https://git.eeqj.de/sneak/vaultik/issues/356)).
+
+- 2026-10-13: Made `prune` and `remote nuke` fail when a delete on the
+  backup destination store fails
+  ([issue #290](https://git.eeqj.de/sneak/vaultik/issues/290)).
+
+- 2026-10-13: Made an S3 listing that the server cuts short an error
+  ([issue #297](https://git.eeqj.de/sneak/vaultik/issues/297)); `prune`
+  had taken it as complete and could delete blobs still in use.
+
+- 2026-10-13: Made shallow `snapshot verify` report a blob it could not
+  check apart from a missing one
+  ([issue #318](https://git.eeqj.de/sneak/vaultik/issues/318)); any
+  error checking a blob had counted as missing, its cause dropped.
+
+- 2026-10-13: Restored the other files when a file's chunk is in no
+  blob under `--skip-errors`
+  ([issue #326](https://git.eeqj.de/sneak/vaultik/issues/326)); that
+  file had stopped the restore before anything was restored.
+
+- 2026-10-13: Recorded each snapshot's new chunk size when its backup
+  runs ([issue #311](https://git.eeqj.de/sneak/vaultik/issues/311)).
+  `snapshot list` could show a NEW CHUNK SIZE of 0 B for a snapshot
+  whose change was a modified file.
+
+- 2026-10-13: Made a backup interrupted during a local index write exit
+  as interrupted
+  ([issue #301](https://git.eeqj.de/sneak/vaultik/issues/301)); it could
+  end with `FATAL: failed to rollback transaction` and exit 1 instead of
+  130.
+
+- 2026-10-13: Printed `snapshot list` and `snapshot purge` sizes like
+  every other command
+  ([issue #312](https://git.eeqj.de/sneak/vaultik/issues/312)). They,
+  and the `bytes` value in terminal log lines, were 1024-based but
+  labelled KB or MB, and now use 1000-based units.
+
+- 2026-10-13: Closed each blob's storage reader when restore or deep
+  verify is interrupted, which ends a stalled read unless it is stuck in
+  the kernel ([issue #325](https://git.eeqj.de/sneak/vaultik/issues/325)).
+
+- 2026-10-13: Made a mistyped subcommand under `snapshot`, `config`,
+  `remote`, `database` or `completion` an error
+  ([issue #319](https://git.eeqj.de/sneak/vaultik/issues/319)); each had
+  printed its help and exited 0.
+
+- 2026-10-13: Made an exclude pattern starting with `**/` match
+  directly under the snapshot path too, as in gitignore
+  ([issue #303](https://git.eeqj.de/sneak/vaultik/issues/303)).
+
+- 2026-10-13: Refused a relative `file://` storage URL and decoded its
+  percent-escapes
+  ([issue #299](https://git.eeqj.de/sneak/vaultik/issues/299)); a
+  relative path had moved with the working directory.
+
+- 2026-10-13: Removed the blob temp file when a backup stops part-way
+  ([issue #302](https://git.eeqj.de/sneak/vaultik/issues/302)); the
+  half-written blob had stayed in `$TMPDIR`.
+
+- 2026-10-13: Checked `--keep-newer-than` before a backup starts
+  ([issue #307](https://git.eeqj.de/sneak/vaultik/issues/307)); an
+  invalid value had been read only by the prune after the backup, so
+  retention never ran.
+
 - 2026-10-13: Stopped `remote info` counting a file under `blobs/` that
   is not a blob as an orphaned blob
   ([issue #295](https://git.eeqj.de/sneak/vaultik/issues/295)). A stray

docs/REPOSTRUCTURE.md

@@ -27,7 +27,7 @@ The metadata subdirectory is named with the **remote key**, a one-way hash of th
 
 ## Marker File (`vaultik-destination.txt`)
 
-Every backup writes this file at the root before it uploads anything. It holds one fixed line of plain text and names no host, path or time. A destination without it is treated as not present while the local index holds snapshots stored there: `snapshot create` refuses to write to it, and `prune`, `snapshot purge`, `snapshot remove` and `snapshot list` behave as for a destination that cannot be listed. An unplugged volume leaves an empty directory at its mount point, and without the marker that directory would read as a destination whose snapshots are all gone. `remote nuke` deletes the marker last.
+Every backup writes this file at the root before it uploads anything. It holds one fixed line of plain text and names no host, path or time. A destination without it is treated as not present while the local index holds snapshots stored there: `snapshot create` refuses to write to it, and `prune`, `snapshot purge`, `snapshot remove`, `snapshot list` and `remote nuke` behave as for a destination that cannot be listed. An unplugged volume leaves an empty directory at its mount point, and without the marker that directory would read as a destination whose snapshots are all gone. `remote nuke` deletes the marker last.
 
 ## Blobs Directory (`blobs/`)
 

internal/cli/entry_snapshot_create_skip_errors_test.go

@@ -0,0 +1,30 @@
+package cli //nolint:testpackage // shares runEntry and the hermetic config helpers
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+// TestEntrySnapshotCreateSkipErrorsExitsNonZeroWhenItSkips backs up,
+// under --skip-errors, a source directory that does not exist, which the
+// walk skips. The command exits non-zero and reports the backup as
+// completed, not as a failed snapshot creation. See
+// https://git.eeqj.de/sneak/vaultik/issues/310.
+//
+//nolint:paralleltest // replaces os.Args, os.Stdout, os.Stderr and the xdg globals
+func TestEntrySnapshotCreateSkipErrorsExitsNonZeroWhenItSkips(t *testing.T) {
+	// writeBackupConfig would create the source directory.
+	configPath := writeHermeticPruneConfig(t, false)
+
+	code, stdout, stderr := runEntry(t,
+		flagConfig, configPath, "--skip-errors", cmdSnapshot, cmdCreate)
+
+	assert.Equal(t, 1, code)
+	assert.Contains(t, stdout,
+		"Files: 1 examined, 0 backed up, 0 unchanged, 1 failed.")
+	assert.NotContains(t, stdout, "Finished successfully.")
+	assert.Contains(t, stderr,
+		"Backup completed, but 1 file(s) could not be read")
+	assert.NotContains(t, stderr, "Snapshot creation failed")
+}

internal/cli/snapshot.go

@@ -115,6 +115,13 @@ on Linux), then /etc/vaultik/config.yml.`,
 					return
 				}
 
+				if errors.Is(err, vaultik.ErrFilesUnreadable) {
+					log.Error("Backup completed with unreadable files", "error", err)
+					ReportErrorf("Backup completed, but %v", err)
+
+					return
+				}
+
 				log.Error("Snapshot creation failed", "error", err)
 				ReportErrorf("Snapshot creation failed: %v", err)
 			})
@@ -125,7 +132,8 @@ on Linux), then /etc/vaultik/config.yml.`,
 		"Run in cron mode (silent unless warning or error)")
 	cmd.Flags().BoolVar(&opts.Prune, "prune", false,
 		"After backup, drop older snapshots of the same name and remove "+
-			"orphaned blobs")
+			"orphaned blobs (not after a backup that skipped unreadable "+
+			"files)")
 	cmd.Flags().StringVar(&opts.KeepNewerThan, "keep-newer-than", "",
 		"With --prune: keep snapshots newer than this duration "+
 			"(e.g. 30d, 4w, 6mo, 1y; m is minutes, mo is months) "+

internal/snapshot/progress_test.go

@@ -78,8 +78,7 @@ func TestProgressPercentWithSecondPathSmaller(t *testing.T) {
 
 	stats := progress.GetStats()
 
-	// Directories count toward the total size but produce no chunks, so
+	// With both paths counted, the percentage must not exceed 100%.
-	// the percentage ends just under 100%.
 	percent := float64(stats.BytesProcessed.Load()) /
 		float64(stats.TotalSize.Load()) * 100
 	if percent > 100 {

internal/snapshot/scanner.go

@@ -141,10 +141,12 @@ type ScannerConfig struct {
 // ScanResult contains the results of a scan operation. Files and bytes
 // are counted per file: BytesScanned is the size of the new and changed
 // files, BytesSkipped that of the unchanged ones. FilesFailed counts the
-// new and changed files that phase 2 could not store; FilesScanned
+// new and changed files that phase 2 could not store, each entry removed
-// includes them and BytesScanned does not. ChunksCreated counts the
+// after the walk listed its directory and, under --skip-errors, each
-// chunks that were not already stored, and NewChunkSize is their total
+// entry the walk could not read, such as a directory it could not list;
-// size.
+// FilesScanned includes them and BytesScanned does not. ChunksCreated
+// counts the chunks that were not already stored, and NewChunkSize is
+// their total size.
 type ScanResult struct {
 	FilesScanned   int
 	FilesSkipped   int
@@ -161,6 +163,11 @@ type ScanResult struct {
 	UploadDuration time.Duration
 	StartTime      time.Time
 	EndTime        time.Time
+
+	// FilesUnreadable is the part of FilesFailed that --skip-errors
+	// skipped because it could not be read or listed; the rest are files
+	// removed after the walk listed them.
+	FilesUnreadable int
 }
 
 // NewScanner creates a new scanner instance
@@ -284,6 +291,11 @@ func (s *Scanner) Scan(
 		}
 	}
 
+	err = s.recordDirectories(ctx, scanResult.Directories)
+	if err != nil {
+		return nil, err
+	}
+
 	// Summarize scan phase results and update progress
 	s.summarizeScanPhase(result, filesToProcess)
 
@@ -855,14 +867,15 @@ func (s *Scanner) batchInsertFileData(
 // ScanPhaseResult contains the results of the scan phase
 type ScanPhaseResult struct {
 	FilesToProcess   []*FileToProcess
-	UnchangedFileIDs []types.FileID // IDs of unchanged files to associate with snapshot
+	UnchangedFileIDs []types.FileID   // IDs of unchanged files to associate with snapshot
+	Directories      []*FileToProcess // Every directory found, to record
 }
 
 // scanPhase performs the initial directory scan to identify files to process
 // It uses the pre-loaded knownFiles map for fast change detection without DB
 // queries. It also populates existingFiles map for deletion detection.
-// Returns files needing processing and IDs of unchanged files for snapshot
+// Returns files needing processing, IDs of unchanged files for snapshot
-// association.
+// association, and the directories found.
 func (s *Scanner) scanPhase(
 	ctx context.Context,
 	path string,
@@ -870,8 +883,10 @@ func (s *Scanner) scanPhase(
 	existingFiles map[string]struct{},
 	knownFiles map[string]*database.File,
 ) (*ScanPhaseResult, error) {
-	// Use known file count as estimate for progress (accurate for subsequent backups)
+	// Use known file count as estimate for progress (accurate for
-	estimatedTotal := int64(len(knownFiles))
+	// subsequent backups). The walk does not count directories, so the
+	// estimate leaves them out.
+	estimatedTotal := countNonDirectories(knownFiles)
 
 	collector := &scanCollector{}
 
@@ -882,8 +897,28 @@ func (s *Scanner) scanPhase(
 	log.Debug("Starting directory walk", "path", path)
 
 	walkFn := func(filePath string, info os.FileInfo, err error) error {
+		// Check exclude patterns first, so an excluded entry that is gone
+		// or cannot be looked up is neither counted nor reported. For
+		// directories, skip the entire subtree. info is nil when the
+		// walk could not look the entry up.
+		if s.shouldExclude(filePath, path) {
+			if info != nil && info.IsDir() {
+				return filepath.SkipDir
+			}
+
+			return nil
+		}
+
+		// The source path is in no directory listing, so its absence is
+		// an error rather than a removal.
+		if errors.Is(err, os.ErrNotExist) && filePath != path {
+			countRemovedDuringWalk(filePath, existingFiles, result)
+
+			return nil
+		}
+
 		if err != nil {
-			return s.handleWalkError(filePath, err)
+			return s.handleWalkError(filePath, err, result)
 		}
 
 		// Check context cancellation
@@ -893,18 +928,9 @@ func (s *Scanner) scanPhase(
 		default:
 		}
 
-		// Check exclude patterns - for directories, skip the entire subtree
-		if s.shouldExclude(filePath, path) {
-			if info.IsDir() {
-				return filepath.SkipDir
-			}
-
-			return nil
-		}
-
 		// Handle symlinks and directories
 		handled, err := s.recordSpecialEntry(
-			filePath, info, existingFiles, collector, result)
+			filePath, info, existingFiles, knownFiles, collector, result)
 		if handled {
 			return err
 		}
@@ -918,9 +944,10 @@ func (s *Scanner) scanPhase(
 		existingFiles[filePath] = struct{}{}
 
 		// Check file against in-memory map (no DB query!)
-		file, needsProcessing := s.checkFileInMemory(filePath, info, knownFiles)
+		file := s.buildRegularFileEntry(filePath, info)
+		needsProcessing := s.checkFileInMemory(file, knownFiles)
 
-		filesScanned, changedCount := collector.recordRegularFile(
+		filesScanned, changedCount := collector.recordFile(
 			filePath, info, file, needsProcessing)
 
 		// Update result stats
@@ -944,27 +971,45 @@ func (s *Scanner) scanPhase(
 	return &ScanPhaseResult{
 		FilesToProcess:   collector.filesToProcess,
 		UnchangedFileIDs: collector.unchangedFileIDs,
+		Directories:      collector.directories,
 	}, nil
 }
 
+// countNonDirectories returns how many of files are not directories.
+func countNonDirectories(files map[string]*database.File) int64 {
+	var count int64
+
+	for _, f := range files {
+		if f.Mode&uint32(os.ModeDir) == 0 {
+			count++
+		}
+	}
+
+	return count
+}
+
 // recordSpecialEntry records symlinks and directories (which have no
-// data to chunk) and reports whether it handled the entry. For a symlink
+// data to chunk) and reports whether it handled the entry. A symlink is
-// whose target cannot be read it returns handleWalkError's result.
+// counted and checked for changes as a regular file is; a directory is
+// set aside to be recorded and is not counted. For a symlink whose
+// target cannot be read it returns handleWalkError's result.
 func (s *Scanner) recordSpecialEntry(
 	filePath string, info os.FileInfo,
 	existingFiles map[string]struct{},
+	knownFiles map[string]*database.File,
 	collector *scanCollector, result *ScanResult,
 ) (bool, error) {
 	// Handle symlinks
 	if info.Mode()&os.ModeSymlink != 0 {
 		file, err := s.buildSymlinkEntry(filePath, info)
 		if err != nil {
-			return true, s.handleWalkError(filePath, err)
+			return true, s.handleWalkError(filePath, err, result)
 		}
 
 		existingFiles[filePath] = struct{}{}
-		collector.addToProcess(filePath, info, file)
+		needsProcessing := s.checkFileInMemory(file, knownFiles)
-		s.updateScanEntryStats(result, true, info)
+		collector.recordFile(filePath, info, file, needsProcessing)
+		s.updateScanEntryStats(result, needsProcessing, info)
 
 		return true, nil
 	}
@@ -974,7 +1019,7 @@ func (s *Scanner) recordSpecialEntry(
 	if info.IsDir() {
 		file := s.buildDirectoryEntry(filePath, info)
 		existingFiles[filePath] = struct{}{}
-		collector.addToProcess(filePath, info, file)
+		collector.addDirectory(filePath, info, file)
 
 		return true, nil
 	}
@@ -982,15 +1027,38 @@ func (s *Scanner) recordSpecialEntry(
 	return false, nil
 }
 
+// countRemovedDuringWalk counts an entry that was in its directory's
+// listing but was gone when the walk looked it up, like a file removed
+// before phase 2 reads it: as examined and failed, and not as an error,
+// with or without --skip-errors.
+func countRemovedDuringWalk(
+	filePath string, existingFiles map[string]struct{}, result *ScanResult,
+) {
+	log.Warn("File was deleted during the directory walk, skipping",
+		"path", filePath)
+
+	// The walk found it, so it is not also counted as deleted.
+	existingFiles[filePath] = struct{}{}
+	result.FilesScanned++
+	result.FilesFailed++
+}
+
 // handleWalkError deals with a filesystem error surfaced by the walk:
-// skipped (with a UI notice) under --skip-errors, fatal otherwise.
+// under --skip-errors the entry is skipped with a UI notice and counted
-func (s *Scanner) handleWalkError(filePath string, err error) error {
+// as examined, failed and unreadable, otherwise the error is fatal.
+func (s *Scanner) handleWalkError(
+	filePath string, err error, result *ScanResult,
+) error {
 	if s.skipErrors {
 		log.Error("Failed to access file (skipping due to --skip-errors)",
 			"path", filePath, "error", err)
 		s.ui.Errorf("Failed to access %s: %v. Skipping (--skip-errors).",
 			s.ui.Path(filePath), err)
 
+		result.FilesScanned++
+		result.FilesFailed++
+		result.FilesUnreadable++
+
 		return nil // Continue scanning
 	}
 
@@ -1004,28 +1072,29 @@ type scanCollector struct {
 	mu               sync.Mutex
 	filesToProcess   []*FileToProcess
 	unchangedFileIDs []types.FileID // Just IDs - no new records needed
+	directories      []*FileToProcess
 	filesScanned     int64
 }
 
-// addToProcess queues a file entry for processing.
+// addDirectory sets a directory aside to be recorded. It is not counted
-func (c *scanCollector) addToProcess(
+// as a file.
+func (c *scanCollector) addDirectory(
 	path string, info os.FileInfo, file *database.File,
 ) {
 	c.mu.Lock()
 	defer c.mu.Unlock()
 
-	c.filesToProcess = append(c.filesToProcess, &FileToProcess{
+	c.directories = append(c.directories, &FileToProcess{
 		Path:     path,
 		FileInfo: info,
 		File:     file,
 	})
-	c.filesScanned++
 }
 
-// recordRegularFile records a regular file as changed (to process) or
+// recordFile records a regular file or symlink as changed (to process)
-// unchanged (snapshot association only) and returns the running scanned
+// or unchanged (snapshot association only) and returns the running
-// and changed counts.
+// scanned and changed counts.
-func (c *scanCollector) recordRegularFile(
+func (c *scanCollector) recordFile(
 	path string, info os.FileInfo, file *database.File, needsProcessing bool,
 ) (int64, int) {
 	c.mu.Lock()
@@ -1128,7 +1197,28 @@ func (s *Scanner) printScanProgressLine(
 	}
 }
 
-// buildSymlinkEntry creates a File record for a symlink.
+// buildRegularFileEntry creates a File record for a regular file, with
+// no ID: checkFileInMemory sets it.
+func (s *Scanner) buildRegularFileEntry(path string, info os.FileInfo) *database.File {
+	var uid, gid uint32
+	if stat, ok := info.Sys().(*syscall.Stat_t); ok {
+		uid = stat.Uid
+		gid = stat.Gid
+	}
+
+	return &database.File{
+		Path:       types.FilePath(path),
+		SourcePath: types.SourcePath(s.currentSourcePath),
+		MTime:      info.ModTime(),
+		Size:       info.Size(),
+		Mode:       uint32(info.Mode()),
+		UID:        uid,
+		GID:        gid,
+	}
+}
+
+// buildSymlinkEntry creates a File record for a symlink, with no ID:
+// checkFileInMemory sets it.
 func (s *Scanner) buildSymlinkEntry(
 	path string, info os.FileInfo,
 ) (*database.File, error) {
@@ -1144,7 +1234,6 @@ func (s *Scanner) buildSymlinkEntry(
 	}
 
 	return &database.File{
-		ID:         types.NewFileID(),
 		Path:       types.FilePath(path),
 		SourcePath: types.SourcePath(s.currentSourcePath),
 		MTime:      info.ModTime(),
@@ -1176,6 +1265,22 @@ func (s *Scanner) buildDirectoryEntry(path string, info os.FileInfo) *database.F
 	}
 }
 
+// recordDirectories records every directory the walk found. A directory
+// has no data to back up, so it is not among the files counted and
+// backed up.
+func (s *Scanner) recordDirectories(
+	ctx context.Context, directories []*FileToProcess,
+) error {
+	for _, dir := range directories {
+		err := s.recordNonRegularFile(ctx, dir)
+		if err != nil {
+			return fmt.Errorf("recording directory %s: %w", dir.Path, err)
+		}
+	}
+
+	return nil
+}
+
 // recordNonRegularFile writes a symlink or directory entry to the database
 // and associates it with the current snapshot. No chunking is performed.
 func (s *Scanner) recordNonRegularFile(ctx context.Context, ftp *FileToProcess) error {
@@ -1189,62 +1294,36 @@ func (s *Scanner) recordNonRegularFile(ctx context.Context, ftp *FileToProcess)
 	})
 }
 
-// checkFileInMemory checks if a file needs processing using the in-memory map
+// checkFileInMemory reports whether a regular file or symlink needs
+// processing, and gives file the ID of its known record, or a new ID for
+// a new path.
 // No database access is performed - this is purely CPU/memory work
 func (s *Scanner) checkFileInMemory(
-	path string, info os.FileInfo, knownFiles map[string]*database.File,
+	file *database.File, knownFiles map[string]*database.File,
-) (*database.File, bool) {
+) bool {
-	var uid, gid uint32
+	existingFile, exists := knownFiles[file.Path.String()]
-	if stat, ok := info.Sys().(*syscall.Stat_t); ok {
-		uid = stat.Uid
-		gid = stat.Gid
-	}
-
-	// Check against in-memory map first to get existing ID if available
-	existingFile, exists := knownFiles[path]
-
-	// Create file record with ID set upfront
-	// For new files, generate UUID immediately so it's available for chunk associations
-	// For existing files, reuse the existing ID
-	var fileID types.FileID
-	if exists {
-		fileID = existingFile.ID
-	} else {
-		fileID = types.NewFileID()
-	}
-
-	file := &database.File{
-		ID:         fileID,
-		Path:       types.FilePath(path),
-		SourcePath: types.SourcePath(s.currentSourcePath),
-		MTime:      info.ModTime(),
-		Size:       info.Size(),
-		Mode:       uint32(info.Mode()),
-		UID:        uid,
-		GID:        gid,
-	}
 
 	// New file - needs processing
 	if !exists {
-		return file, true
+		file.ID = types.NewFileID()
+
+		return true
 	}
 
+	file.ID = existingFile.ID
+
 	// No uploaded blob holds one of its chunks (see loadFilesToRechunk)
-	if _, rechunk := s.filesToRechunk[fileID]; rechunk {
+	if _, rechunk := s.filesToRechunk[file.ID]; rechunk {
-		return file, true
+		return true
 	}
 
 	// Check if file has changed
-	if existingFile.Size != file.Size ||
+	return existingFile.Size != file.Size ||
 		!existingFile.MTime.Equal(file.MTime) ||
 		existingFile.Mode != file.Mode ||
 		existingFile.UID != file.UID ||
-		existingFile.GID != file.GID {
+		existingFile.GID != file.GID ||
-		return file, true
+		existingFile.LinkTarget != file.LinkTarget
-	}
-
-	// File unchanged
-	return file, false
 }
 
 // batchAddFilesToSnapshot adds existing file IDs to the snapshot association table
@@ -1371,17 +1450,16 @@ func (s *Scanner) processPhase(
 	return s.finalizeProcessPhase(ctx, result)
 }
 
-// processFileWithErrorHandling records a directory or symlink, or wraps
+// processFileWithErrorHandling records a symlink, or wraps
 // processFileStreaming for a regular file with error recovery for deleted
 // files and skip-errors mode. Returns (skipped, error).
 func (s *Scanner) processFileWithErrorHandling(
 	ctx context.Context, fileToProcess *FileToProcess, result *ScanResult,
 ) (bool, error) {
-	// A directory or symlink has no data to open or read; it is only
+	// A symlink has no data to open or read; it is only recorded in the
-	// recorded in the local index. An error recording it stops the run
+	// local index. An error recording it stops the run even under
-	// even under --skip-errors, or the snapshot would complete without it.
+	// --skip-errors, or the snapshot would complete without it.
-	mode := os.FileMode(fileToProcess.File.Mode)
+	if os.FileMode(fileToProcess.File.Mode)&os.ModeSymlink != 0 {
-	if mode&os.ModeSymlink != 0 || mode.IsDir() {
 		err := s.recordNonRegularFile(ctx, fileToProcess)
 		if err != nil {
 			return false, fmt.Errorf("processing file %s: %w", fileToProcess.Path, err)
@@ -1422,6 +1500,7 @@ func (s *Scanner) processFileWithErrorHandling(
 				s.ui.Path(fileToProcess.Path), err)
 
 			countFailedFile(fileToProcess, result)
+			result.FilesUnreadable++
 
 			return true, nil
 		}

internal/snapshot/scanner_estimate_test.go

@@ -0,0 +1,28 @@
+//nolint:testpackage // exercises the unexported countNonDirectories helper
+package snapshot
+
+import (
+	"os"
+	"testing"
+
+	"sneak.berlin/go/vaultik/internal/database"
+)
+
+// The walk counts regular files and symlinks, not directories, so the
+// estimate of its count made from the local index's records leaves the
+// directories out. See https://git.eeqj.de/sneak/vaultik/issues/305.
+func TestCountNonDirectories(t *testing.T) {
+	t.Parallel()
+
+	known := map[string]*database.File{
+		"/src":       {Mode: uint32(os.ModeDir | 0o755)},
+		"/src/sub":   {Mode: uint32(os.ModeDir | 0o755)},
+		"/src/a.txt": {Mode: 0o644},
+		"/src/link":  {Mode: uint32(os.ModeSymlink | 0o777), LinkTarget: "a.txt"},
+	}
+
+	got := countNonDirectories(known)
+	if got != 2 {
+		t.Errorf("countNonDirectories = %d, want 2", got)
+	}
+}

internal/vaultik/destination_marker_test.go

@@ -152,6 +152,25 @@ func TestPurgeRefusesDestinationWithoutMarker(t *testing.T) {
 	assertUntouched(ctx, t, repos, storeDir)
 }
 
+// TestNukeRemoteRefusesDestinationWithoutMarker checks that remote nuke
+// fails, deletes no local snapshot record, and neither reports the
+// destination empty nor advises running nuke again when the destination
+// has no marker.
+func TestNukeRemoteRefusesDestinationWithoutMarker(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	ctx := context.Background()
+	v, repos, out, storeDir := backUpThenUnplugMountPoint(ctx, t)
+
+	err := v.NukeRemote(true)
+	require.ErrorContains(t, err, markerFile)
+	assert.NotContains(t, out.String(), "now empty")
+	assert.NotContains(t, out.String(), "vaultik remote nuke --force")
+
+	assertUntouched(ctx, t, repos, storeDir)
+}
+
 // TestRemoveSnapshotWarnsWhenDestinationHasNoMarker checks that snapshot
 // remove warns that the metadata could not be removed from a destination
 // without a marker, instead of reporting that it was.

internal/vaultik/missing_destination_test.go

@@ -188,3 +188,41 @@ func TestPurgeSaysListingFailedOnceWhenDestinationMissing(t *testing.T) {
 	assert.Equal(t, 1, strings.Count(err.Error(), "listing remote snapshots"),
 		err.Error())
 }
+
+// TestNukeRemoteKeepsLocalIndexWhenDestinationMissing checks that remote
+// nuke fails on a destination it cannot list, leaves the local index
+// unchanged, and neither reports the destination empty nor advises
+// running nuke again.
+func TestNukeRemoteKeepsLocalIndexWhenDestinationMissing(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	ctx := context.Background()
+	v, repos, out := backUpThenUnplug(ctx, t)
+
+	filesBefore, err := repos.Files.ListAll(ctx)
+	require.NoError(t, err)
+	require.NotEmpty(t, filesBefore)
+
+	blobsBefore, err := repos.Blobs.GetAll(ctx)
+	require.NoError(t, err)
+	require.NotEmpty(t, blobsBefore)
+
+	err = v.NukeRemote(true)
+	require.ErrorIs(t, err, fs.ErrNotExist)
+	require.ErrorContains(t, err, "listing remote snapshots")
+	assert.NotContains(t, out.String(), "now empty")
+	assert.NotContains(t, out.String(), "vaultik remote nuke --force")
+
+	snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
+	require.NoError(t, err)
+	assert.Len(t, snapshots, 1, "nuke must delete no local snapshot record")
+
+	files, err := repos.Files.ListAll(ctx)
+	require.NoError(t, err)
+	assert.Len(t, files, len(filesBefore), "nuke must delete no local file record")
+
+	blobs, err := repos.Blobs.GetAll(ctx)
+	require.NoError(t, err)
+	assert.Len(t, blobs, len(blobsBefore), "nuke must delete no local blob record")
+}

internal/vaultik/nuke_remote_test.go

@@ -104,6 +104,32 @@ func TestNukeRemoteFailsWhenADeleteFails(t *testing.T) {
 	}
 }
 
+// TestNukeRemoteKeepsLocalSnapshotWhoseMetadataIsNotDeleted checks that
+// remote nuke keeps a snapshot in the local index while its metadata
+// cannot be deleted from the destination store: dropped from the local
+// index, it would be left there as a remote-only snapshot without its
+// name.
+func TestNukeRemoteKeepsLocalSnapshotWhoseMetadataIsNotDeleted(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	ctx := context.Background()
+	storeDir := filepath.Join(t.TempDir(), "store")
+	v, repos, _ := backUpToFileDestination(ctx, t, storeDir)
+
+	store := faultstore.New(v.Storage)
+	store.OnDelete = func(key string) bool {
+		return strings.HasPrefix(key, "metadata/")
+	}
+	v.Storage = store
+
+	require.Error(t, v.NukeRemote(true))
+
+	snapshots, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
+	require.NoError(t, err)
+	assert.Len(t, snapshots, 1)
+}
+
 // TestNukeRemoteNamesWhatIsLeft checks that remote nuke, when an object
 // it does not recognise cannot be deleted, does not report the store
 // empty but names that object and fails.
@@ -154,8 +180,7 @@ func TestNukeRemoteRefusesAnotherDestinationWithoutRetryAdvice(t *testing.T) {
 
 // TestNukeRemoteFailsWhenALocalSnapshotCannotBeRemoved checks that a
 // snapshot remote nuke cannot remove from the local index fails the
-// run: nuke then leaves that snapshot's metadata on the destination
+// run, and that nuke then does not report the destination store empty.
-// store.
 func TestNukeRemoteFailsWhenALocalSnapshotCannotBeRemoved(t *testing.T) {
 	log.Initialize(log.Config{})
 	t.Parallel()

internal/vaultik/prune.go

@@ -55,12 +55,19 @@ func (v *Vaultik) NukeRemote(force bool) error {
 	}
 
 	// Checked before anything is deleted: running nuke again would fail
-	// the same way, so the advice below must not follow this error.
+	// the same way, so the advice below must not follow these errors.
+	// The listing also fails on a destination without the marker while
+	// the local index holds snapshots.
 	err := v.EnsureStorageBinding()
 	if err != nil {
 		return err
 	}
 
+	_, err = v.listAllRemoteSnapshotKeys()
+	if err != nil {
+		return err
+	}
+
 	err = v.emptyDestinationStore()
 	if err != nil {
 		v.UI.Warningf("Run '%s' again to delete what is left on the "+

internal/vaultik/snapshot.go

@@ -39,6 +39,11 @@ var (
 // caller can report it apart from a failed backup.
 var ErrPostBackupPruneFailed = errors.New("post-backup prune failed")
 
+// ErrFilesUnreadable is returned when --skip-errors skipped files or
+// directories it could not read or list. The snapshot is complete and
+// kept, so the caller can report it apart from a failed backup.
+var ErrFilesUnreadable = errors.New("file(s) could not be read")
+
 // listRecentLimit caps how many snapshot rows are fetched from the
 // local index for listing and purge operations.
 const listRecentLimit = 10000
@@ -130,12 +135,16 @@ func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
 	}
 
 	// Process each named snapshot
+	filesUnreadable := 0
+
 	for snapIdx, snapName := range snapshotNames {
-		err = v.createNamedSnapshot(
+		unreadable, err := v.createNamedSnapshot(
 			opts, hostname, snapName, snapIdx+1, len(snapshotNames))
 		if err != nil {
 			return err
 		}
+
+		filesUnreadable += unreadable
 	}
 
 	// Print overall summary if multiple snapshots
@@ -144,13 +153,34 @@ func (v *Vaultik) CreateSnapshot(opts *SnapshotCreateOptions) error {
 			len(snapshotNames), v.UI.Duration(time.Since(overallStartTime)))
 	}
 
-	if opts.Prune {
+	switch {
+	case opts.Prune && filesUnreadable > 0:
+		// An older snapshot may be the only one that still holds a file
+		// this run could not read.
+		log.Warn("Skipping post-backup prune",
+			"files_unreadable", filesUnreadable)
+		v.UI.Warningf("Skipped the post-backup prune because %d file(s) "+
+			"could not be read; older snapshots are kept.", filesUnreadable)
+	case opts.Prune:
 		err = v.runPostBackupPrune(snapshotNames, opts.KeepNewerThan)
 		if err != nil {
 			return fmt.Errorf("%w: %w", ErrPostBackupPruneFailed, err)
 		}
 	}
 
+	return v.finishCreateSnapshot(filesUnreadable)
+}
+
+// finishCreateSnapshot ends a run whose snapshots are all complete and
+// kept. Like finishRestore, it returns an error when --skip-errors
+// skipped anything it could not read or list; otherwise it prints the
+// closing line. A file removed during the run is counted as failed in
+// the summary but is not an error, with or without --skip-errors.
+func (v *Vaultik) finishCreateSnapshot(filesUnreadable int) error {
+	if filesUnreadable > 0 {
+		return fmt.Errorf("%d %w", filesUnreadable, ErrFilesUnreadable)
+	}
+
 	// Terminus must obey the --cron invariant: silent on total
 	// success only. UI.Complete is dropped in cron/quiet mode (that's
 	// the success path), but if any warnings fired during the run we
@@ -240,16 +270,22 @@ type snapshotStats struct {
 	totalBlobsUploaded int
 	uploadDuration     time.Duration
 
+	// The part of totalFilesFailed that --skip-errors skipped because it
+	// could not be read or listed.
+	totalFilesUnreadable int
+
 	// The sizes of all blobs the snapshot references, set by
 	// finalizeSnapshotMetadata once snapshot_blobs is populated.
 	blobSize             int64
 	blobUncompressedSize int64
 }
 
-// createNamedSnapshot creates a single named snapshot
+// createNamedSnapshot creates a single named snapshot and returns the
+// number of entries --skip-errors skipped because it could not read or
+// list them.
 func (v *Vaultik) createNamedSnapshot(
 	opts *SnapshotCreateOptions, hostname, snapName string, idx, total int,
-) error {
+) (int, error) {
 	snapshotStartTime := time.Now()
 
 	if total > 1 {
@@ -258,12 +294,12 @@ func (v *Vaultik) createNamedSnapshot(
 
 	resolvedDirs, err := v.resolveSnapshotPaths(snapName)
 	if err != nil {
-		return err
+		return 0, err
 	}
 
 	previousSnapshotID, err := v.previousSnapshotID(snapName)
 	if err != nil {
-		return err
+		return 0, err
 	}
 
 	scanner := v.ScannerFactory(snapshot.ScannerParams{
@@ -278,7 +314,7 @@ func (v *Vaultik) createNamedSnapshot(
 	snapshotID, err := v.SnapshotManager.CreateSnapshotWithName(
 		v.ctx, hostname, snapName, v.Globals.Version, v.Globals.Commit)
 	if err != nil {
-		return fmt.Errorf("creating snapshot: %w", err)
+		return 0, fmt.Errorf("creating snapshot: %w", err)
 	}
 
 	log.Info("Beginning snapshot", "snapshot_id", snapshotID, "name", snapName)
@@ -286,12 +322,12 @@ func (v *Vaultik) createNamedSnapshot(
 
 	stats, err := v.scanAllDirectories(scanner, resolvedDirs, snapshotID)
 	if err != nil {
-		return err
+		return 0, err
 	}
 
 	err = v.finalizeSnapshotMetadata(snapshotID, stats)
 	if err != nil {
-		return err
+		return 0, err
 	}
 
 	log.Info("Snapshot complete",
@@ -304,7 +340,7 @@ func (v *Vaultik) createNamedSnapshot(
 
 	v.printSnapshotSummary(snapshotID, snapshotStartTime, stats)
 
-	return nil
+	return stats.totalFilesUnreadable, nil
 }
 
 // previousSnapshotID returns the ID of the most recent completed snapshot
@@ -390,6 +426,7 @@ func (v *Vaultik) scanAllDirectories(
 		stats.totalBlobs += result.BlobsCreated
 		stats.totalFilesSkipped += result.FilesSkipped
 		stats.totalFilesFailed += result.FilesFailed
+		stats.totalFilesUnreadable += result.FilesUnreadable
 		stats.totalBytesSkipped += result.BytesSkipped
 		stats.totalFilesDeleted += result.FilesDeleted
 		stats.totalBytesDeleted += result.BytesDeleted
@@ -1516,7 +1553,9 @@ func (v *Vaultik) removeSnapshotRemote(
 // database from the local index, and (with --remote) every snapshot
 // metadata directory in remote storage. Both sides are processed so a
 // "remove --all" leaves nothing behind, even when the local DB and
-// remote storage have diverged.
+// remote storage have diverged. Unless LocalOnly is set, it fails
+// before it removes anything when the destination store cannot be
+// listed or is not present by checkDestinationMarker.
 func (v *Vaultik) RemoveAllSnapshots(opts *RemoveOptions) (*RemoveResult, error) {
 	err := v.EnsureStorageBinding()
 	if err != nil {
@@ -1529,11 +1568,15 @@ func (v *Vaultik) RemoveAllSnapshots(opts *RemoveOptions) (*RemoveResult, error)
 	}
 
 	// remoteKeys is the set of metadata/<key>/ subdirectories on the
-	// destination store; failures are downgraded to a warning so a
+	// destination store. Only for a local-only remove is a failure
-	// permission-denied or unreachable remote can't block a local-only
+	// downgraded to a warning, so a permission-denied or unreachable
-	// remove.
+	// remote can't block it.
 	remoteKeys, remoteErr := v.listAllRemoteSnapshotKeys()
 	if remoteErr != nil {
+		if !opts.LocalOnly {
+			return nil, remoteErr
+		}
+
 		log.Warn("Could not list remote snapshots", "error", remoteErr)
 		v.UI.Warningf("Could not list remote snapshots: %v.", remoteErr)
 	}
@@ -1595,8 +1638,7 @@ func (v *Vaultik) localSnapshotIDs() ([]string, error) {
 // listAllRemoteSnapshotKeys collects the hashed remote keys
 // (subdirectories under metadata/) currently present in the
 // destination store. Returns (nil, err) when the store cannot be
-// listed, or is not present by checkDestinationMarker; callers must
+// listed, or is not present by checkDestinationMarker.
-// treat that as "no remote info available," not fatal.
 func (v *Vaultik) listAllRemoteSnapshotKeys() ([]string, error) {
 	log.Info("Listing all remote snapshots")
 
@@ -1780,10 +1822,12 @@ func (v *Vaultik) deleteIncompleteSnapshot(snapshotIDStr string, id any) bool {
 	return true
 }
 
-// removeAllLocalSnapshots removes each known local snapshot (and, unless
+// removeAllLocalSnapshots removes each known local snapshot's remote
-// --local-only, its remote metadata), returning the number of snapshots
+// metadata, unless --local-only, and then its local record, returning
-// it could not remove. A snapshot left in the local index keeps its
+// the number of snapshots it could not remove. The local record goes
-// remote metadata too, since it is not an orphan remote key.
+// only once the metadata is deleted: dropped first, it would leave that
+// metadata on the destination store as a remote-only snapshot whose name
+// is lost.
 func (v *Vaultik) removeAllLocalSnapshots(
 	localSnaps []string, opts *RemoveOptions, result *RemoveResult,
 ) int {
@@ -1792,6 +1836,19 @@ func (v *Vaultik) removeAllLocalSnapshots(
 	for _, snapshotID := range localSnaps {
 		log.Info("Removing snapshot", "snapshot_id", snapshotID)
 
+		if !opts.LocalOnly {
+			_, err := v.deleteRemoteSnapshotByKey(
+				snapshot.RemoteSnapshotKey(snapshotID))
+			if err != nil {
+				log.Warn("Failed to remove snapshot metadata from remote",
+					"snapshot_id", snapshotID, "error", err)
+
+				failures++
+
+				continue
+			}
+		}
+
 		err := v.deleteSnapshotFromLocalDB(snapshotID)
 		if err != nil {
 			log.Error("Failed to remove from local database",
@@ -1802,17 +1859,6 @@ func (v *Vaultik) removeAllLocalSnapshots(
 			continue
 		}
 
-		if !opts.LocalOnly {
-			_, err = v.deleteRemoteSnapshotByKey(
-				snapshot.RemoteSnapshotKey(snapshotID))
-			if err != nil {
-				log.Warn("Failed to remove snapshot metadata from remote",
-					"snapshot_id", snapshotID, "error", err)
-
-				failures++
-			}
-		}
-
 		result.SnapshotsRemoved = append(result.SnapshotsRemoved, snapshotID)
 	}
 

internal/vaultik/snapshot_deleted_count_test.go

@@ -178,11 +178,13 @@ func TestBackupDoesNotCountFileInUnopenableDirectoryAsDeleted(t *testing.T) {
 	}
 
 	env.out.Reset()
-	require.NoError(t, env.v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
+	err := env.v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
 		SkipErrors: true,
 		Snapshots:  []string{"work"},
-	}))
+	})
-	assert.Contains(t, env.out.String(), filesLine(1, 0, 1))
+	require.ErrorIs(t, err, vaultik.ErrFilesUnreadable)
+	assert.Contains(t, env.out.String(),
+		"Files: 2 examined, 0 backed up, 1 unchanged, 1 failed.")
 }
 
 // A backup counts only files that the previous snapshot of its own name

internal/vaultik/snapshot_failed_file_test.go

@@ -49,13 +49,19 @@ func TestSnapshotSummaryCountsFileNotStoredAsFailed(t *testing.T) {
 		name       string
 		openErr    error
 		skipErrors bool
+		wantErr    error // nil for none
 	}{
 		// What a normal user gets opening a file with mode 000.
 		{name: "unopenable under skip-errors",
-			openErr: os.ErrPermission, skipErrors: true},
+			openErr: os.ErrPermission, skipErrors: true,
+			wantErr: vaultik.ErrFilesUnreadable},
 		// What opening a file removed after phase 1 gives.
 		{name: "removed between the phases",
 			openErr: os.ErrNotExist, skipErrors: false},
+		// A removal is not an error, so --skip-errors does not make the
+		// run fail. See https://git.eeqj.de/sneak/vaultik/issues/310.
+		{name: "removed between the phases under skip-errors",
+			openErr: os.ErrNotExist, skipErrors: true},
 	}
 
 	for _, tt := range tests {
@@ -99,10 +105,11 @@ func TestSnapshotSummaryCountsFileNotStoredAsFailed(t *testing.T) {
 			v := newBackupVaultik(ctx, cfg, store, repos, db, fs)
 			v.UI = ui.NewWithColor(out, false)
 
-			require.NoError(t, v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
+			err = v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
 				SkipErrors: tt.skipErrors,
 				Snapshots:  []string{"src"},
-			}))
+			})
+			require.ErrorIs(t, err, tt.wantErr)
 
 			summary := out.String()
 			assert.Contains(t, summary,

internal/vaultik/snapshot_skip_errors_test.go

@@ -3,6 +3,7 @@ package vaultik_test
 import (
 	"context"
 	"fmt"
+	"os"
 	"path/filepath"
 	"testing"
 
@@ -73,3 +74,210 @@ func TestSkipErrorsBackupStopsOnIndexErrorRecordingDirectory(t *testing.T) {
 	require.Len(t, snapshots, 1)
 	assert.Nil(t, snapshots[0].CompletedAt)
 }
+
+// A backup under --skip-errors that skips a file it cannot open, or a
+// directory it cannot list, keeps the snapshot, counts what it skipped
+// as failed and returns an error instead of finishing successfully. See
+// https://git.eeqj.de/sneak/vaultik/issues/310.
+func TestSkipErrorsBackupKeepsSnapshotAndFailsWhenItSkips(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	tests := []struct {
+		name    string
+		skipped string // relative to the source directory
+	}{
+		{name: "file it cannot open", skipped: "skipdir/file.txt"},
+		{name: "directory it cannot list", skipped: "skipdir"},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			env := newDeletedCountEnv(t, map[string][]byte{
+				"stored.txt":       []byte("stored"),
+				"skipdir/file.txt": []byte("in skipdir"),
+			})
+			env.setSnapshot("work", "")
+
+			// A mode does not lock out root, who runs the tests, so
+			// opening the skipped path is made to fail instead.
+			env.v.Fs = &openFailFs{
+				path: filepath.Join(env.src, tt.skipped),
+				err:  os.ErrPermission,
+			}
+
+			err := env.v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
+				SkipErrors: true,
+				Snapshots:  []string{"work"},
+			})
+			require.ErrorIs(t, err, vaultik.ErrFilesUnreadable)
+
+			assert.Contains(t, env.out.String(),
+				"Files: 2 examined, 1 backed up, 0 unchanged, 1 failed.")
+			assert.NotContains(t, env.out.String(), "Finished successfully.")
+
+			snapshots, err := env.v.Repositories.Snapshots.ListRecent(
+				context.Background(), listRecentTestLimit)
+			require.NoError(t, err)
+			require.Len(t, snapshots, 1)
+			assert.NotNil(t, snapshots[0].CompletedAt, "the snapshot is kept")
+		})
+	}
+}
+
+// removeOnLookupFs is the real filesystem, except that path is removed
+// just before it is looked up. The walk looks an entry up after listing
+// its directory, so path is listed but gone when the walk reaches it.
+type removeOnLookupFs struct {
+	afero.OsFs
+
+	path string
+}
+
+func (f *removeOnLookupFs) LstatIfPossible(
+	name string,
+) (os.FileInfo, bool, error) {
+	if name == f.path {
+		_ = os.Remove(name)
+	}
+
+	return f.OsFs.LstatIfPossible(name)
+}
+
+// A file removed before the walk reaches it is counted once, as failed,
+// like a file removed before its data is read, and the backup finishes
+// successfully, with or without --skip-errors. See
+// https://git.eeqj.de/sneak/vaultik/issues/310.
+func TestBackupCountsFileRemovedBeforeWalkReachesItAsFailed(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	tests := []struct {
+		name       string
+		skipErrors bool
+	}{
+		{name: "under skip-errors", skipErrors: true},
+		{name: "without skip-errors", skipErrors: false},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			env := newDeletedCountEnv(t, map[string][]byte{
+				"stays.txt":   []byte("stays"),
+				"removed.txt": []byte("removed"),
+			})
+			env.setSnapshot("walk", "")
+
+			// The previous snapshot has the file, so a removal the walk
+			// also counted as deleted would show.
+			env.backUp(t, "walk")
+
+			env.v.Fs = &removeOnLookupFs{
+				path: filepath.Join(env.src, "removed.txt"),
+			}
+
+			env.out.Reset()
+			require.NoError(t, env.v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
+				SkipErrors: tt.skipErrors,
+				Snapshots:  []string{"walk"},
+			}))
+			assert.Contains(t, env.out.String(),
+				"Files: 2 examined, 0 backed up, 1 unchanged, 1 failed.")
+			assert.Contains(t, env.out.String(), "Finished successfully.")
+		})
+	}
+}
+
+// A file an exclude pattern matches is not counted when it is removed
+// before the walk reaches it. See
+// https://git.eeqj.de/sneak/vaultik/issues/310.
+func TestBackupIgnoresExcludedFileRemovedBeforeWalkReachesIt(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	env := newDeletedCountEnv(t, map[string][]byte{
+		"stays.txt":   []byte("stays"),
+		"removed.tmp": []byte("removed"),
+	})
+	env.setSnapshot("walk", "", "*.tmp")
+	env.v.Fs = &removeOnLookupFs{
+		path: filepath.Join(env.src, "removed.tmp"),
+	}
+
+	require.NoError(t, env.v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
+		SkipErrors: true,
+		Snapshots:  []string{"walk"},
+	}))
+	assert.Contains(t, env.out.String(), filesLine(1, 1, 0))
+	assert.Contains(t, env.out.String(), "Finished successfully.")
+}
+
+// A backup under --skip-errors --prune that skipped a file it could not
+// read does not prune: the previous snapshot, the only one that holds
+// the file, is kept, and the output says the prune was skipped. See
+// https://git.eeqj.de/sneak/vaultik/issues/382.
+func TestSkipErrorsBackupDoesNotPruneWhenItSkips(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	env := newDeletedCountEnv(t, map[string][]byte{
+		"other.txt":   []byte("other"),
+		"skipped.txt": []byte("skipped"),
+	})
+	env.setSnapshot("photos", "")
+	env.backUp(t, "photos")
+
+	// An unchanged file is not read again, so the file is changed before
+	// it becomes unreadable.
+	skipped := filepath.Join(env.src, "skipped.txt")
+	require.NoError(t, os.WriteFile(skipped, []byte("changed"), 0o600))
+	env.v.Fs = &openFailFs{path: skipped, err: os.ErrPermission}
+
+	env.out.Reset()
+	err := env.v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
+		Prune:      true,
+		SkipErrors: true,
+		Snapshots:  []string{"photos"},
+	})
+	require.ErrorIs(t, err, vaultik.ErrFilesUnreadable)
+	assert.Contains(t, env.out.String(),
+		"Skipped the post-backup prune because 1 file(s) could not be read")
+	assert.NotContains(t, env.out.String(), "Running post-backup prune.")
+
+	snapshots, err := env.v.Repositories.Snapshots.ListRecent(
+		context.Background(), listRecentTestLimit)
+	require.NoError(t, err)
+	assert.Len(t, snapshots, 2, "the previous snapshot is kept")
+}
+
+// A file removed during a backup under --skip-errors --prune is not a
+// skipped file, so the backup prunes as usual. See
+// https://git.eeqj.de/sneak/vaultik/issues/382.
+func TestSkipErrorsBackupPrunesWhenAFileIsRemovedDuringIt(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	env := newDeletedCountEnv(t, map[string][]byte{
+		"other.txt":   []byte("other"),
+		"removed.txt": []byte("removed"),
+	})
+	env.setSnapshot("photos", "")
+	env.backUp(t, "photos")
+
+	env.v.Fs = &removeOnLookupFs{path: filepath.Join(env.src, "removed.txt")}
+
+	require.NoError(t, env.v.CreateSnapshot(&vaultik.SnapshotCreateOptions{
+		Prune:      true,
+		SkipErrors: true,
+		Snapshots:  []string{"photos"},
+	}))
+
+	snapshots, err := env.v.Repositories.Snapshots.ListRecent(
+		context.Background(), listRecentTestLimit)
+	require.NoError(t, err)
+	assert.Len(t, snapshots, 1, "the previous snapshot is pruned")
+}

internal/vaultik/snapshot_summary_test.go

@@ -13,6 +13,7 @@ import (
 	"github.com/spf13/afero"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
+	"golang.org/x/sys/unix"
 	"sneak.berlin/go/vaultik/internal/config"
 	"sneak.berlin/go/vaultik/internal/database"
 	"sneak.berlin/go/vaultik/internal/log"
@@ -188,6 +189,42 @@ func (e *summaryEnv) dataLine(total, backedUp int64) string {
 		e.v.UI.Size(total), e.v.UI.Size(backedUp))
 }
 
+// enumeratedLine returns the line printed when the walk of a source path
+// ends.
+func (e *summaryEnv) enumeratedLine(
+	files int, total int64, toBackUp int, toBackUpSize int64,
+) string {
+	return fmt.Sprintf(
+		"Enumerated %d snapshot source files (%s total), %d to back up (%s).",
+		files, e.v.UI.Size(total), toBackUp, e.v.UI.Size(toBackUpSize))
+}
+
+// snapshotPaths returns the paths the snapshot lists, sorted.
+func (e *summaryEnv) snapshotPaths(t *testing.T, snapshotID string) []string {
+	t.Helper()
+
+	rows, err := e.db.Conn().QueryContext(context.Background(), `
+		SELECT f.path FROM snapshot_files sf JOIN files f ON f.id = sf.file_id
+		WHERE sf.snapshot_id = ? ORDER BY f.path`, snapshotID)
+	require.NoError(t, err)
+
+	defer func() { _ = rows.Close() }()
+
+	var paths []string
+
+	for rows.Next() {
+		var path string
+
+		require.NoError(t, rows.Scan(&path))
+
+		paths = append(paths, path)
+	}
+
+	require.NoError(t, rows.Err())
+
+	return paths
+}
+
 // A first backup stores copy.bin's chunks while backing up a.bin, so
 // copy.bin's chunks are deduplicated within the run. Each file and byte
 // is still counted once.
@@ -248,6 +285,85 @@ func TestSnapshotSummaryIncrementalRunWithDeduplicatedChunks(t *testing.T) {
 	assert.Equal(t, uncompressed, snap.BlobUncompressedSize)
 }
 
+// A backup of a tree with directories and a symlink, none of which
+// changed since the last backup, backs up nothing, and the snapshot
+// still lists every one of them. Both sides of the line that ends the
+// walk count the same entries, the files and the symlink, and no
+// directory's size. See https://git.eeqj.de/sneak/vaultik/issues/305.
+func TestSnapshotSummaryUnchangedRunWithDirectoriesAndSymlink(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	env := newSummaryEnv(t)
+
+	const linkTarget = "a.bin"
+
+	require.NoError(t, os.Symlink(linkTarget,
+		filepath.Join(filepath.Dir(env.aPath), "link")))
+
+	// A symlink's size is the length of its target.
+	total := env.totalSize + int64(len(linkTarget))
+
+	first := env.backUp(t, "first", false)
+	assert.Contains(t, first, env.enumeratedLine(4, total, 4, total))
+	assert.Contains(t, first, filesLine(4, 4, 0))
+
+	second := env.backUp(t, "second", false)
+	assert.Contains(t, second, env.enumeratedLine(4, total, 0, 0))
+	assert.Contains(t, second, filesLine(4, 0, 4))
+	assert.Contains(t, second, env.dataLine(total, 0))
+
+	assert.Equal(t,
+		env.snapshotPaths(t, env.snapshot(t, "first").ID.String()),
+		env.snapshotPaths(t, env.snapshot(t, "second").ID.String()))
+}
+
+// A directory whose mtime changed and a symlink whose target changed are
+// recorded again by the next backup, which counts only the symlink as
+// backed up.
+func TestSnapshotSummaryChangedDirectoryAndSymlink(t *testing.T) {
+	log.Initialize(log.Config{})
+	t.Parallel()
+
+	env := newSummaryEnv(t)
+
+	// The local index keeps paths with symlinks resolved.
+	dir, err := filepath.EvalSymlinks(filepath.Dir(env.aPath))
+	require.NoError(t, err)
+
+	// The new link gets the old one's mtime, and a target, which need not
+	// exist, of the old target's length, a symlink's size: only the
+	// target differs.
+	linkTime := unix.NsecToTimeval(
+		time.Date(2029, time.March, 4, 5, 6, 7, 0, time.UTC).UnixNano())
+	link := filepath.Join(dir, "link")
+	require.NoError(t, os.Symlink("a.bin", link))
+	require.NoError(t, unix.Lutimes(link, []unix.Timeval{linkTime, linkTime}))
+
+	env.backUp(t, "first", false)
+
+	require.NoError(t, os.Remove(link))
+	require.NoError(t, os.Symlink("b.bin", link))
+	require.NoError(t, unix.Lutimes(link, []unix.Timeval{linkTime, linkTime}))
+
+	mtime := time.Date(2030, time.January, 2, 3, 4, 5, 0, time.UTC)
+	require.NoError(t, os.Chtimes(dir, mtime, mtime))
+
+	assert.Contains(t, env.backUp(t, "second", false), filesLine(4, 1, 3))
+
+	ctx := context.Background()
+
+	dirRow, err := env.repos.Files.GetByPath(ctx, dir)
+	require.NoError(t, err)
+	require.NotNil(t, dirRow)
+	assert.Equal(t, mtime.Unix(), dirRow.MTime.Unix())
+
+	linkRow, err := env.repos.Files.GetByPath(ctx, link)
+	require.NoError(t, err)
+	require.NotNil(t, linkRow)
+	assert.Equal(t, "b.bin", linkRow.LinkTarget.String())
+}
+
 // Under --cron the progress reporter is off; the upload figures must
 // still reach the summary and the snapshots row. The snapshot has two
 // paths, each backed up by its own scan.