Add scan --exclude PATH (closes #163)

The walk neither enters nor records an --exclude path or anything
below it, and prints "scan: skipped PATH (--exclude)" for each one it
reaches. The flag repeats. Each value is resolved to its real path, as
an operand is. The records below an excluded path are kept, as for a
default exclude. An empty value, which would name the working
directory, is a usage error. An --exclude path whose real path scan
cannot find, or an operand at or below one, is a fatal error before
the scan starts. The line naming the database moved out of runScan
into printDatabasePath, which keeps runScan under the lint length
limit.

Model: opus-5-5

README.md

@@ -299,8 +299,11 @@ The reference below is also the program's specification.
   such as `/var/tmp` on Linux (see "Rules for the walk").
 - `--exclude PATH` (`scan`) keeps the walk out of `PATH` and everything below
   it. `PATH` may be a directory or a file. The flag may be given more than once.
-  `PATH` is resolved to its real path, as an operand is. A `PATH` that does not
-  exist is a fatal error, and so is an operand at or below one (see "Operands").
+  `PATH` is resolved to its real path, as an operand is. An empty `PATH`, as in
+  `--exclude ""`, is a usage error, reported in one line on stderr with exit 2
+  before anything is scanned. A `PATH` that does not exist, or whose real path
+  `scan` cannot find, is a fatal error. So is an operand at or below one (see
+  "Operands").
 - `--version`, `-v` prints `sfdupes VERSION` (see "Install") to stdout and
   exits 0. It writes nothing to stderr.
 - `--help`, `-h`, alone or after a subcommand, prints the help text to stderr
@@ -684,6 +687,7 @@ Processing:
 - Records are grouped by signature, the four values
   `(size, head, tail, content)`.
 - Every group with two or more paths is a duplicate group.
+- Empty files form no group, because removing one frees nothing.
 - Within a group, paths are sorted in byte order. The first is `first`, and
   every other path is a `dupe`.
 - Groups are ordered by size, largest first, then by `first` path. The output is
@@ -732,7 +736,7 @@ The summary line on stderr gives:
 - the reclaimable bytes, the sum of `size` over all rows.
 
 ```
-report: 14 records read, 5 duplicate groups, 8 dupe files, 10.0 KiB reclaimable
+report: 14 records read, 4 duplicate groups, 7 dupe files, 10.0 KiB reclaimable
 ```
 
 ### `trees` mode
@@ -760,6 +764,9 @@ Processing:
 - `trees` builds the directory tree and computes every directory's digest. It
   then groups the directories by digest. Every group with two or more
   directories is a duplicate-tree group.
+- `trees` leaves out a group whose trees hold 0 bytes, because removing one
+  frees nothing. An empty file still counts toward the digest of a tree that
+  holds other files.
 - `trees` leaves out a group that a group of its members' parents already
   implies. A group is left out when its members' parents are distinct
   directories that all have the same digest. The parents' group, or an
@@ -1194,7 +1201,9 @@ showing the count it stopped at.
 - `1`: a fatal error, a `scan` stopped by `SIGINT` or `SIGTERM`, or a `dwim`
   stopped by either during its scan. A fatal error is one of these:
     - an operand that does not exist, or whose real path `scan` cannot find;
-    - an `--exclude` path that does not exist, or an operand at or below one;
+    - an `--exclude` path that does not exist, or whose real path `scan` cannot
+      find;
+    - an operand at or below an `--exclude` path;
     - a list of mounted filesystems that `scan` cannot read;
     - another `scan` holding the lock (see "The scan lock");
     - a database, its directory or its lock file that cannot be created or
@@ -1214,6 +1223,7 @@ showing the count it stopped at.
       `--workers abc`;
     - `scan` with no operand;
     - `--workers` below 1;
+    - an empty `--exclude` value, as in `--exclude ""`;
     - `report`, `trees`, `info` or `dwim` with a positional argument.
 
 A fatal error is reported on stderr, starting with `sfdupes:`. A failed write to
@@ -1402,7 +1412,7 @@ All of the following, run in this directory, must pass:
     head -c 2000 /dev/urandom > "$d/a/unique.bin"     # same size, different content
     printf 'x' > "$d/tiny1"; printf 'x' > "$d/tiny2"  # 1-byte duplicates
     printf 'y' > "$d/tiny3"                           # 1-byte non-duplicate
-    : > "$d/empty1"; : > "$d/empty2"                  # empty duplicates
+    : > "$d/empty1"; : > "$d/empty2"                  # empty files
     # duplicate trees: t1 and t2 are identical; t3 differs by one filename
     mkdir -p "$d/t1/sub" "$d/t2/sub" "$d/t3/sub"
     head -c 3000 /dev/urandom > "$d/t1/f1"
@@ -1427,8 +1437,8 @@ All of the following, run in this directory, must pass:
       is the path that sorts first.
     - `t1/f1`, `t2/f1` and `t3/f1` form one group.
     - `t1/sub/f2`, `t2/sub/f2` and `t3/sub/f2renamed` form one group.
-    - `tiny1` and `tiny2` form a pair, and so do `empty1` and `empty2`.
-    - `unique.bin` and `tiny3` appear nowhere.
+    - `tiny1` and `tiny2` form a pair.
+    - `unique.bin`, `tiny3`, `empty1` and `empty2` appear nowhere.
     - The groups come in order of size, largest first.
 
     `trees` prints exactly one row, `$d/t1` as `first` and `$d/t2` as `dupe`.

TODO.md

@@ -31,9 +31,13 @@ are known.
 
 - New `scan` flag `--exclude PATH`, which may be repeated. The walk neither
   enters nor records an excluded path, and keeps the records below it. An
-  operand at or below one is a fatal error (2026-11-10,
+  operand at or below one is a fatal error (2026-11-11,
   https://git.eeqj.de/sneak/sfdupes/issues/163)
 
+- `report` and `trees` leave out empty files. `report` writes no group of size
+  0, and `trees` writes no group whose trees hold 0 bytes. An empty file still
+  counts toward a tree's digest (2026-11-11,
+  https://git.eeqj.de/sneak/sfdupes/issues/178)
 - The Go code moved out of the repository root. `cmd/sfdupes/main.go` is the
   command, and it calls `Main` in the package `internal/sfdupes`. `main.go`
   became `internal/sfdupes/cli.go`. Every other file kept its name. The install

internal/sfdupes/cli.go

@@ -10,6 +10,7 @@ import (
 	"os"
 	"runtime"
 	"runtime/debug"
+	"slices"
 
 	"github.com/spf13/cobra"
 )
@@ -40,6 +41,11 @@ var errNoSubcommand = errors.New("no subcommand")
 // below 1.
 var errWorkersBelowOne = errors.New("--workers must be at least 1")
 
+// errEmptyExcludeValue is the usage error for an empty scan --exclude
+// value, as from --exclude "$DIR" with DIR unset. Taken as a path, it
+// would name the working directory.
+var errEmptyExcludeValue = errors.New("--exclude must not be empty")
+
 // Version is the build version, injected at link time via -ldflags
 // (see the Makefile); empty when the linker set none, as for a binary
 // built by go install. buildVersion turns it into what --version prints.
@@ -222,7 +228,12 @@ func newScanCommand() *cobra.Command {
 		Long:  short + ".\n\n" + defaultExcludesHelp(runtime.GOOS),
 		Args:  cobra.MinimumNArgs(1),
 		PreRunE: func(cmd *cobra.Command, _ []string) error {
-			return checkScanWorkers(cmd, scanWorkers)
+			err := checkScanWorkers(cmd, scanWorkers)
+			if err != nil {
+				return err
+			}
+
+			return rejectEmptyExcludeValue(cmd, scanExclude)
 		},
 		RunE: runE(func(ctx context.Context, args []string) error {
 			ctx, stop := interruptContext(ctx)
@@ -259,6 +270,18 @@ func checkScanWorkers(cmd *cobra.Command, workers int) error {
 	return fmt.Errorf("%w, got %d", errWorkersBelowOne, workers)
 }
 
+// rejectEmptyExcludeValue rejects an empty scan --exclude value. That
+// is a usage error reported in one line, as for --workers below 1.
+func rejectEmptyExcludeValue(cmd *cobra.Command, excluded []string) error {
+	if !slices.Contains(excluded, "") {
+		return nil
+	}
+
+	cmd.SilenceUsage = true
+
+	return errEmptyExcludeValue
+}
+
 // runE adapts a subcommand implementation, or the version print, to
 // cobra's RunE. Cobra prints the error and the command's usage text for
 // every error RunE returns, but a subcommand that ran and failed has no

internal/sfdupes/exclude.go

@@ -104,24 +104,19 @@ func isDefaultExclude(path string, excludes []string) bool {
 	return false
 }
 
-// resolveExcluded returns the real paths of excluded, the --exclude
-// values. One that cannot be resolved is a fatal error, and so is one
-// of roots at or below one of them. roots are real paths already.
-func resolveExcluded(roots, excluded []string) ([]string, error) {
-	excluded, err := resolveRoots(excluded)
-	if err != nil {
-		return nil, fmt.Errorf("--exclude: %w", err)
-	}
-
+// checkOperandsNotExcluded returns an error when one of roots, the
+// operands, is at or below one of excluded, the --exclude values. The
+// error names both. Both lists hold real paths.
+func checkOperandsNotExcluded(roots, excluded []string) error {
 	for _, root := range roots {
 		exclude := deepestRoot(root, excluded)
 		if exclude != "" {
-			return nil, fmt.Errorf("scan %s: %w %s", root,
-				errExcludedOperand, exclude)
+			return fmt.Errorf("scan %s: %w %s", root, errExcludedOperand,
+				exclude)
 		}
 	}
 
-	return excluded, nil
+	return nil
 }
 
 // excludedLine returns the line the scan prints for path when it is one

internal/sfdupes/exclude_test.go

@@ -474,6 +474,47 @@ func TestRunScanExcludedOperand(t *testing.T) {
 	}
 }
 
+// TestRunScanRejectsEmptyExclude checks that an empty --exclude value,
+// given as its own argument, after "=" or after another --exclude, is
+// a usage error reported in one line before anything is scanned
+// (README "Settings").
+func TestRunScanRejectsEmptyExclude(t *testing.T) {
+	dir := t.TempDir()
+
+	cases := map[string][]string{
+		"separate":     {excludeFlag, "", dir},
+		"joined by =":  {excludeFlag + "=", dir},
+		"after a path": {excludeFlag, t.TempDir(), excludeFlag, "", dir},
+	}
+
+	for name, args := range cases {
+		t.Run(name, func(t *testing.T) {
+			path := testDBPath(t)
+			t.Setenv(databaseEnv, path)
+
+			var stdout, stderr bytes.Buffer
+
+			code := run(append([]string{cmdScan}, args...), &stdout,
+				&stderr)
+			if code != exitUsage {
+				t.Errorf("run(scan %q) = %d, want %d", args, code,
+					exitUsage)
+			}
+
+			want := "Error: --exclude must not be empty\n"
+			if stdout.String() != "" || stderr.String() != want {
+				t.Errorf("stdout, stderr = %q, %q, want \"\", %q",
+					stdout.String(), stderr.String(), want)
+			}
+
+			_, err := os.Lstat(path)
+			if !errors.Is(err, os.ErrNotExist) {
+				t.Errorf("lstat %s: %v, want no database", path, err)
+			}
+		})
+	}
+}
+
 // TestRunScanSkipsNonPersistentOperand scans, through the command, a
 // directory on a filesystem that is not persistent, with and without
 // --no-default-excludes: either way it is skipped, with a line saying

internal/sfdupes/report.go

@@ -195,10 +195,11 @@ func (w *reportWriter) add(r scanRec) error {
 }
 
 // endGroup ends the paths read with w.sig: two or more are a duplicate
-// group, held with its paths sorted until writeGroups writes it.
+// group, held with its paths sorted until writeGroups writes it. Empty
+// files form no group, since removing one frees nothing.
 func (w *reportWriter) endGroup() {
-	if len(w.paths) < minGroupSize {
-		// A path with no duplicate is dropped, and its slice reused.
+	if len(w.paths) < minGroupSize || w.sig.size == 0 {
+		// The paths are dropped, and their slice reused.
 		w.paths = w.paths[:0]
 
 		return

internal/sfdupes/report_test.go

@@ -434,6 +434,27 @@ func TestDupeGroups(t *testing.T) {
 	}
 }
 
+// TestDupeGroupsLeavesOutEmptyFiles checks that empty files form no
+// duplicate group, while 1-byte files still do.
+func TestDupeGroupsLeavesOutEmptyFiles(t *testing.T) {
+	t.Parallel()
+
+	e, x := sha256.Sum256(nil), testHash("x")
+	recs := []scanRec{
+		{size: 0, head: e, tail: e, content: e, path: "/a/.gitkeep"},
+		{size: 0, head: e, tail: e, content: e, path: "/b/.gitkeep"},
+		{size: 0, head: e, tail: e, content: e, path: "/c/__init__.py"},
+		{size: 1, head: x, tail: x, content: x, path: "/tiny1"},
+		{size: 1, head: x, tail: x, content: x, path: "/tiny2"},
+	}
+
+	groups := dupeGroupsOf(t, recs)
+	if len(groups) != 1 || groups[0].size != 1 ||
+		!slices.Equal(groups[0].paths, []string{"/tiny1", "/tiny2"}) {
+		t.Fatalf("groups = %+v, want only /tiny1 /tiny2 of size 1", groups)
+	}
+}
+
 func TestDupeGroupsContentSeparates(t *testing.T) {
 	t.Parallel()
 

internal/sfdupes/scan.go

@@ -114,23 +114,19 @@ func runScan(ctx context.Context, roots []string, workers int,
 		return err
 	}
 
-	// A database file missing now is created by lockScanDatabase below.
-	what := "database"
+	printDatabasePath(dbPath, source)
 
-	_, err = os.Stat(dbPath)
-	if errors.Is(err, fs.ErrNotExist) {
-		what = "new database"
-	}
-
-	fmt.Fprintf(os.Stderr, "scan: %s %s (%s)\n", what, escapePath(dbPath),
-		source)
-
-	roots, err = resolveRoots(roots)
+	roots, err = realPaths(roots)
 	if err != nil {
 		return err
 	}
 
-	excluded, err = resolveExcluded(roots, excluded)
+	excluded, err = realPaths(excluded)
+	if err != nil {
+		return fmt.Errorf("--exclude: %w", err)
+	}
+
+	err = checkOperandsNotExcluded(roots, excluded)
 	if err != nil {
 		return err
 	}
@@ -188,6 +184,21 @@ func runScan(ctx context.Context, roots []string, workers int,
 	return nil
 }
 
+// printDatabasePath prints on stderr the line that names the database
+// at dbPath, with source, saying whether it is new. A database file
+// missing now is new: lockScanDatabase creates it.
+func printDatabasePath(dbPath, source string) {
+	what := "database"
+
+	_, err := os.Stat(dbPath)
+	if errors.Is(err, fs.ErrNotExist) {
+		what = "new database"
+	}
+
+	fmt.Fprintf(os.Stderr, "scan: %s %s (%s)\n", what, escapePath(dbPath),
+		source)
+}
+
 // interruptContext returns a copy of ctx that the first SIGINT or
 // SIGTERM cancels; the scan command runs the scan under it. stop
 // releases the signals.
@@ -216,24 +227,25 @@ func interrupted(walked int) error {
 	return errInterrupted
 }
 
-// resolveRoots converts each PATH operand, or each --exclude value, to
-// its real path: absolute, with every symlink in it followed. Database
-// records are keyed by that path, so scan results depend neither on the
-// working directory nor on the symlinks an operand was given through.
-// A path that cannot be resolved, because it or a symlink's target does
-// not exist or cannot be read, is a fatal error before any scanning.
-func resolveRoots(roots []string) ([]string, error) {
-	resolved := make([]string, 0, len(roots))
+// realPaths returns the real path of each of paths, the PATH operands
+// or the --exclude values: absolute, with every symlink in it followed.
+// Database records are keyed by that path, so scan results depend
+// neither on the working directory nor on the symlinks an operand was
+// given through. A path that cannot be resolved, because it or a
+// symlink's target does not exist or cannot be read, is a fatal error
+// before any scanning.
+func realPaths(paths []string) ([]string, error) {
+	resolved := make([]string, 0, len(paths))
 
-	for _, root := range roots {
-		a, err := absolutePath(root)
+	for _, path := range paths {
+		a, err := absolutePath(path)
 		if err != nil {
-			return nil, fmt.Errorf("resolve %s: %w", root, err)
+			return nil, fmt.Errorf("resolve %s: %w", path, err)
 		}
 
 		r, err := filepath.EvalSymlinks(a)
 		if err != nil {
-			return nil, fmt.Errorf("resolve %s: %w", root, err)
+			return nil, fmt.Errorf("resolve %s: %w", path, err)
 		}
 
 		resolved = append(resolved, r)
@@ -425,7 +437,7 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
 
 // walkableRoots returns the operands the walk can start from: regular
 // files, and directories not named .zfs, none of them inside a
-// directory named .zfs. The operands are real paths from resolveRoots,
+// directory named .zfs. The operands are real paths from realPaths,
 // so these checks apply to what a symlink operand points at. Every
 // other operand is warned about, counted as skipped, and dropped. A
 // dropped operand is no longer a root, so the records stored beneath it
@@ -905,7 +917,7 @@ func sendEvent(ctx context.Context, events chan<- walkEvent,
 // operandWarning returns the one-line warning for an operand the walk
 // does not start from, naming the path and what it is, or "" for one it
 // does: a regular file, or a directory not named .zfs, that is not
-// inside a directory named .zfs. resolveRoots has already followed the
+// inside a directory named .zfs. realPaths has already followed the
 // operand's symlinks, so a symlink here is an operand replaced by one
 // after it was resolved, and is not followed.
 func operandWarning(root string, fi fs.FileInfo) string {

internal/sfdupes/scan_test.go

@@ -1185,11 +1185,12 @@ func assertSmokeDupeGroups(t *testing.T, dir string, db *sql.DB) {
 	t.Helper()
 
 	groups := dupeGroups(t, db)
-	if len(groups) != 5 {
-		t.Fatalf("len(groups) = %d, want 5", len(groups))
+	if len(groups) != 4 {
+		t.Fatalf("len(groups) = %d, want 4", len(groups))
 	}
 
-	wantSizes := []int64{3000, 2000, 100, 1, 0}
+	// The empty files empty1 and empty2 form no group.
+	wantSizes := []int64{3000, 2000, 100, 1}
 	for i, g := range groups {
 		if g.size != wantSizes[i] {
 			t.Errorf("groups[%d].size = %d, want %d",
@@ -2367,8 +2368,8 @@ func TestRunScanPassesOverDatabase(t *testing.T) {
 			report := runStdout(t, cmdReport)
 			if strings.Contains(report, "db.sqlite") ||
 				strings.Contains(report, hardLink) ||
-				!strings.Contains(report, filepath.Join(tree, "empty2")) {
-				t.Errorf("report = %q, want the empty files and none of "+
+				!strings.Contains(report, filepath.Join(tree, "b")) {
+				t.Errorf("report = %q, want a and b and none of "+
 					"the database's files", report)
 			}
 		})

internal/sfdupes/trees.go

@@ -301,7 +301,9 @@ func fileEntry(name string, r scanRec) string {
 // collectTreeGroups groups directories by digest, counting each one on
 // prog as it is grouped, and returns every maximal group with two or
 // more members, each group's members sorted by path, groups ordered by
-// tree size descending then by first path ascending.
+// tree size descending then by first path ascending. A group whose
+// trees hold 0 bytes is left out, since removing one frees nothing. The
+// groups it implies hold 0 bytes too, so leaving it out hides nothing.
 func collectTreeGroups(allDirs []*treeNode, super *treeNode,
 	prog *progress,
 ) [][]*treeNode {
@@ -315,7 +317,9 @@ func collectTreeGroups(allDirs []*treeNode, super *treeNode,
 	var dupes [][]*treeNode
 
 	for _, g := range groups {
-		if len(g) < minGroupSize || suppressed(g, super) {
+		// The members share a digest, so they share a total size.
+		if len(g) < minGroupSize || g[0].totalSize == 0 ||
+			suppressed(g, super) {
 			continue
 		}
 

internal/sfdupes/trees_test.go

@@ -2,6 +2,7 @@ package sfdupes //nolint:testpackage // checks the unexported treeNode results
 
 import (
 	"bytes"
+	"crypto/sha256"
 	"database/sql"
 	"os"
 	"slices"
@@ -435,6 +436,35 @@ func TestCollectTreeGroupsSiblings(t *testing.T) {
 	}
 }
 
+// TestCollectTreeGroupsLeavesOutEmptyTrees checks that directories
+// holding only empty files form no group, and that an empty file still
+// counts toward the digest of a tree that holds other files.
+func TestCollectTreeGroupsLeavesOutEmptyTrees(t *testing.T) {
+	t.Parallel()
+
+	h, tl, c, e := testHash("h"), testHash("t"), testHash("c"),
+		sha256.Sum256(nil)
+	recs := []scanRec{
+		{size: 0, head: e, tail: e, content: e, path: "/e/a/.gitkeep"},
+		{size: 0, head: e, tail: e, content: e, path: "/e/b/.gitkeep"},
+		{size: 10, head: h, tail: tl, content: c, path: "/p/x1/f"},
+		{size: 0, head: e, tail: e, content: e, path: "/p/x1/.gitkeep"},
+		{size: 10, head: h, tail: tl, content: c, path: "/p/x2/f"},
+		{size: 0, head: e, tail: e, content: e, path: "/p/x2/.gitkeep"},
+		// Without the empty file, /p/x3 is no copy of /p/x1.
+		{size: 10, head: h, tail: tl, content: c, path: "/p/x3/f"},
+	}
+
+	super, dirs := treeOf(t, recs)
+
+	got := groupPaths(collectTreeGroups(dirs, super, nil))
+
+	want := [][]string{{"/p/x1", "/p/x2"}}
+	if !slices.EqualFunc(got, want, slices.Equal) {
+		t.Fatalf("groups = %v, want %v", got, want)
+	}
+}
+
 func TestCollectTreeGroupsDifferingParents(t *testing.T) {
 	t.Parallel()