trees: count a nested dupe tree once in the reclaimable total (closes #168)

The total summed `size` over every row, so a `dupe` inside another
row's `dupe` was counted twice. It now leaves out a row whose `dupe`
lies inside another row's `dupe`. The rows are unchanged. README
defines the total to match.

Model: opus-5-5

README.md

@@ -22,8 +22,9 @@ records the files it finds in an SQLite database. SQLite keeps a whole database
 in one file. The database is kept between runs. A rescan reads only new and
 changed files. That makes `scan` cheap enough to run every day from `cron`, the
 system's job scheduler. `report`, `trees` and `info` work from the database and
-can run at any time. `dwim` scans `/` and then prints the `trees` report, for a
+can run at any time. `report` and `trees` leave out a copy that was deleted
-first look at a new machine with one command.
+after the scan. `dwim` scans `/` and then prints the `trees` report, for a first
+look at a new machine with one command.
 
 ## Getting Started
 
@@ -316,11 +317,9 @@ The reference below is also the program's specification.
 - `--version`, `-v` prints `sfdupes VERSION` (see "Install") to stdout and
   exits 0. It writes nothing to stderr.
 - `--help`, `-h`, alone or after a subcommand, prints the help text to stderr
-  and exits 0. It writes nothing to stdout. The help of `sfdupes` and of `scan`
+  and exits 0. It writes nothing to stdout. `sfdupes scan --help` also lists the
-  names the database. The help of `report`, `trees`, `info` and `dwim` says what
+  default excludes of the system it runs on, and the filesystem types the walk
-  each prints. `sfdupes scan --help` also lists the default excludes of the
+  never records anything on.
-  system it runs on, and the filesystem types the walk never records anything
-  on.
 
 ### Duplicate detection
 
@@ -849,7 +848,8 @@ The summary line on stderr gives:
 - the records read;
 - the duplicate-tree groups;
 - the dupe trees, which is the number of rows;
-- the reclaimable bytes, the sum of `size` over all rows.
+- the reclaimable bytes, the sum of `size` over all rows, leaving out a row
+  whose `dupe` lies inside another row's `dupe`.
 
 ```
 trees: 14 records read, 1 duplicate tree groups, 1 dupe trees, 3.0 KiB reclaimable

TODO.md

@@ -29,11 +29,10 @@
 Newest first. An entry ends with its date and its issue or branch, where those
 are known.
 
-- The help of `sfdupes` and of `scan` names the database, its default path and
+- The reclaimable total of `trees` is the sum of `size` over all rows, leaving
-  `SFDUPES_DATABASE`. The help of `report` and `trees` lists the columns, the
+  out a row whose `dupe` lies inside another row's `dupe`. Before, the bytes of
-  order of the rows and the escapes. The help of `info` lists its lines. The
+  such a row were counted twice (2026-11-11,
-  help of `dwim` says that it scans `/`, should run as root and keeps the scan
+  https://git.eeqj.de/sneak/sfdupes/issues/168)
-  in the database (2026-11-11, https://git.eeqj.de/sneak/sfdupes/issues/177)
 - New `scan` flag `--exclude PATH`, which may be repeated. The walk neither
   enters nor records an excluded path, and keeps the records below it. An
   operand at or below one is a fatal error (2026-11-11,

internal/sfdupes/cli.go

@@ -32,69 +32,6 @@ const (
 	cmdDwim   = "dwim"
 )
 
-// databaseHelp is the paragraph of the root and scan help that names
-// the database (README "Location").
-const databaseHelp = "scan and dwim store what they find in a database " +
-	"file. report, trees\nand info read it. The file is $" + databaseEnv +
-	" when that is set and\nnot empty, and otherwise " +
-	defaultDatabasePath + ", where creating it\nneeds root.\n"
-
-// escapesHelp is the paragraph of the report and trees help on how
-// paths are escaped (README "Report output format").
-const escapesHelp = `Four characters in a path are escaped, so each row stays one line:
-
-  \\  a backslash
-  \t  a tab
-  \n  a newline
-  \r  a carriage return
-`
-
-// reportHelp, treesHelp, infoHelp and dwimHelp follow the one-line
-// summary in the help of their subcommands.
-const (
-	reportHelp = `The report is tab-separated text on stdout: a header line, then one
-row per duplicate file. The columns are:
-
-  first   the path that sorts first, byte by byte, in its group of copies
-  dupe    another copy of that file
-  size    the file's size in bytes
-
-Rows are ordered by size, largest first, then by path.
-
-` + escapesHelp
-
-	treesHelp = `The report is tab-separated text on stdout: a header line, then one
-row per duplicate directory tree. The columns are:
-
-  first      the path that sorts first, byte by byte, in its group of copies
-  dupe       another copy of that tree
-  files      the number of regular files in one copy
-  size       their total size in bytes
-  disk_size  the bytes the dupe copy takes on disk
-
-Rows are ordered by size, largest first, then by path.
-
-` + escapesHelp
-
-	infoHelp = `The details go to stdout, one per line. Each line holds a name and its
-value, separated by a tab:
-
-  database                the path, and where it came from
-  database file size      in bytes, then rounded, as 2147483648 (2.0 GiB)
-  -wal file size          in bytes, then rounded; only if a -wal file exists
-  modified                when the database file last changed, in UTC
-  schema version          the version of the database's layout
-  records                 the number of records
-  size of recorded files  their total size, in bytes, then rounded
-`
-
-	dwimHelp = `dwim runs what "sfdupes scan /" runs, then what "sfdupes trees" runs.
-Run it as root, so that the scan can read every file. The scan is stored
-in the database, as scan stores it, so "sfdupes trees" prints the report
-again later.
-`
-)
-
 // errNoSubcommand is returned by the root command when it is invoked
 // without a subcommand. That is a usage error, and the usage text
 // cobra prints for it is the whole message.
@@ -265,21 +202,15 @@ func newRootCommand(stdout, stderr io.Writer) *cobra.Command {
 		}),
 	}
 
-	root.Long = root.Short + ".\n\n" + databaseHelp
-	reportCmd.Long = reportCmd.Short + ".\n\n" + reportHelp
-	treesCmd.Long = treesCmd.Short + ".\n\n" + treesHelp
-	infoCmd.Long = infoCmd.Short + ".\n\n" + infoHelp
-	dwimCmd.Long = dwimCmd.Short + ".\n\n" + dwimHelp
-
 	root.AddCommand(newScanCommand(), reportCmd, treesCmd, infoCmd, dwimCmd)
 
 	return root
 }
 
 // newScanCommand builds the scan subcommand, with its --workers, -x,
-// --no-default-excludes and --exclude flags. Its help names the
+// --no-default-excludes and --exclude flags. Its help lists the default
-// database, and lists the default excludes of the system it runs on and
+// excludes of the system it runs on, and the filesystem types it never
-// the filesystem types it never records anything on.
+// records anything on.
 func newScanCommand() *cobra.Command {
 	var (
 		scanWorkers           int
@@ -297,9 +228,8 @@ func newScanCommand() *cobra.Command {
 		// append its generic "[flags]" to it.
 		DisableFlagsInUseLine: true,
 		Short:                 short,
-		Long: short + ".\n\n" + databaseHelp + "\n" +
+		Long:                  short + ".\n\n" + defaultExcludesHelp(runtime.GOOS),
-			defaultExcludesHelp(runtime.GOOS),
+		Args:                  cobra.MinimumNArgs(1),
-		Args: cobra.MinimumNArgs(1),
 		PreRunE: func(cmd *cobra.Command, _ []string) error {
 			err := checkScanWorkers(cmd, scanWorkers)
 			if err != nil {

internal/sfdupes/cli_test.go

@@ -22,9 +22,6 @@ import (
 // error prints and a runtime failure must not.
 const usageMarker = "Usage:"
 
-// helpFlag asks for the help text.
-const helpFlag = "--help"
-
 // TestMain stops the tests at once, saying why, when their temporary
 // directory is not on a persistent filesystem: scan records nothing
 // there (README "Rules for the walk"), so most scan tests would fail
@@ -696,7 +693,7 @@ func TestRunHelp(t *testing.T) {
 
 	// README §Settings: help goes to stderr, exits 0, and leaves stdout
 	// empty.
-	cases := [][]string{{helpFlag}, {"-h"}, {cmdScan, helpFlag}}
+	cases := [][]string{{"--help"}, {"-h"}, {cmdScan, "--help"}}
 
 	for _, args := range cases {
 		var stdout, stderr bytes.Buffer
@@ -757,73 +754,6 @@ func TestRunScanUsageLine(t *testing.T) {
 	}
 }
 
-func TestRunHelpDescribesDatabaseAndOutput(t *testing.T) {
-	t.Parallel()
-
-	// README §Settings: the help names the database, and says what
-	// report, trees, info and dwim print.
-	database := []string{
-		"scan and dwim store", databaseEnv, defaultDatabasePath, "needs root",
-	}
-	escapes := []string{`\\`, `\t`, `\n`, `\r`}
-
-	// The subcommand whose help is printed; none for the root.
-	cases := []struct {
-		subcommand []string
-		want       []string
-	}{
-		{nil, database},
-		{[]string{cmdScan}, database},
-		{
-			[]string{cmdReport},
-			append([]string{
-				"tab-separated", "\n  first ", "\n  dupe ", "\n  size ",
-				"largest first",
-			}, escapes...),
-		},
-		{
-			[]string{cmdTrees},
-			append([]string{
-				"tab-separated", "\n  first ", "\n  dupe ", "\n  files ",
-				"\n  size ", "\n  disk_size ", "largest first",
-			}, escapes...),
-		},
-		{
-			[]string{cmdInfo},
-			[]string{
-				"\n  database ", "\n  database file size ",
-				"\n  -wal file size ", "\n  modified ",
-				"\n  schema version ", "\n  records ",
-				"\n  size of recorded files ",
-			},
-		},
-		{
-			[]string{cmdDwim},
-			[]string{
-				"sfdupes scan /", "as root", "sfdupes trees", "database",
-			},
-		},
-	}
-
-	for _, tc := range cases {
-		var stdout, stderr bytes.Buffer
-
-		args := slices.Concat(tc.subcommand, []string{helpFlag})
-
-		code := run(args, &stdout, &stderr)
-		if code != exitOK {
-			t.Fatalf("run(%v) = %d, want %d", args, code, exitOK)
-		}
-
-		for _, want := range tc.want {
-			if !strings.Contains(stderr.String(), want) {
-				t.Errorf("run(%v) help lacks %q:\n%s",
-					args, want, stderr.String())
-			}
-		}
-	}
-}
-
 func TestRunVersion(t *testing.T) {
 	t.Parallel()
 

internal/sfdupes/trees.go

@@ -101,7 +101,7 @@ func runTrees(ctx context.Context, stdout io.Writer) error {
 		label: treesPhase("write"), unit: "groups", total: int64(len(dupes)),
 		stalledOn: "out",
 	})
-	dupeTrees, reclaimable, err := writeTreeGroups(stdout, dupes, prog)
+	dupeTrees, err := writeTreeGroups(stdout, dupes, prog)
 
 	prog.finish()
 
@@ -112,7 +112,7 @@ func runTrees(ctx context.Context, stdout io.Writer) error {
 	fmt.Fprintf(os.Stderr,
 		"trees: %d records read, %d duplicate tree groups, %d dupe trees, "+
 			"%s reclaimable\n",
-		records, len(dupes), dupeTrees, humanBytes(reclaimable))
+		records, len(dupes), dupeTrees, humanBytes(reclaimableBytes(dupes)))
 
 	return nil
 }
@@ -146,22 +146,19 @@ func readTree(ctx context.Context, db *sql.DB,
 // a row for every member of each group but its first, counting each
 // group on prog once its rows are written. The disk_size column is the
 // dupe's own. stdout may be the terminal prog is drawn on, so the lines
-// go through prog's writer. It returns the number of rows (dupe trees)
+// go through prog's writer. It returns the number of rows (dupe trees).
-// and the bytes they could reclaim.
 func writeTreeGroups(stdout io.Writer, dupes [][]*treeNode,
 	prog *progress,
-) (int, int64, error) {
+) (int, error) {
 	out := prog.writer(stdout)
 
 	_, err := fmt.Fprintln(out, "first\tdupe\tfiles\tsize\tdisk_size")
 	if err != nil {
-		return 0, 0, fmt.Errorf("write stdout: %w", err)
+		return 0, fmt.Errorf("write stdout: %w", err)
 	}
 
 	dupeTrees := 0
 
-	var reclaimable int64
-
 	for _, g := range dupes {
 		first := g[0]
 		for _, n := range g[1:] {
@@ -169,11 +166,10 @@ func writeTreeGroups(stdout io.Writer, dupes [][]*treeNode,
 				escapePath(first.path), escapePath(n.path),
 				first.fileCount, first.totalSize, n.diskSize)
 			if err != nil {
-				return 0, 0, fmt.Errorf("write stdout: %w", err)
+				return 0, fmt.Errorf("write stdout: %w", err)
 			}
 
 			dupeTrees++
-			reclaimable += first.totalSize
 		}
 
 		prog.increment()
@@ -181,10 +177,44 @@ func writeTreeGroups(stdout io.Writer, dupes [][]*treeNode,
 
 	err = out.Flush()
 	if err != nil {
-		return 0, 0, fmt.Errorf("write stdout: %w", err)
+		return 0, fmt.Errorf("write stdout: %w", err)
 	}
 
-	return dupeTrees, reclaimable, nil
+	return dupeTrees, nil
+}
+
+// reclaimableBytes returns the sum of the size column over the rows
+// that writeTreeGroups writes for dupes, leaving out a row whose dupe
+// lies inside another row's dupe.
+func reclaimableBytes(dupes [][]*treeNode) int64 {
+	isDupe := make(map[*treeNode]bool)
+
+	for _, g := range dupes {
+		for _, n := range g[1:] {
+			isDupe[n] = true
+		}
+	}
+
+	var total int64
+
+	for n := range isDupe {
+		if !insideDupe(n, isDupe) {
+			total += n.totalSize
+		}
+	}
+
+	return total
+}
+
+// insideDupe reports whether a directory above n is in isDupe.
+func insideDupe(n *treeNode, isDupe map[*treeNode]bool) bool {
+	for p := n.parent; p != nil; p = p.parent {
+		if isDupe[p] {
+			return true
+		}
+	}
+
+	return false
 }
 
 // treeBuilder reconstructs the directory hierarchy from records added

internal/sfdupes/trees_test.go

@@ -313,7 +313,7 @@ func TestWriteTreeGroupsOnTerminal(t *testing.T) {
 		label: treesPhase("write"), unit: "groups", total: int64(len(dupes)),
 	}, terminalWidth, time.Hour, time.Now)
 
-	_, _, err := writeTreeGroups(os.Stderr, dupes, p)
+	_, err := writeTreeGroups(os.Stderr, dupes, p)
 
 	p.finish()
 
@@ -587,3 +587,64 @@ func TestCollectTreeGroupsKeepsGroupLeftWithTwoCopies(t *testing.T) {
 		t.Errorf("groups = %v, want %v", got, want)
 	}
 }
+
+// TestReclaimableBytesCountsNestedDupeOnce checks that a dupe tree
+// inside another dupe tree adds nothing to the reclaimable bytes. /B/x
+// lies in /B, so the total adds the sizes of /B and /C/x, 1500 + 1000.
+func TestReclaimableBytesCountsNestedDupeOnce(t *testing.T) {
+	t.Parallel()
+
+	fh, ft, fc := testHash("fh"), testHash("ft"), testHash("fc")
+	gh, gt, gc := testHash("gh"), testHash("gt"), testHash("gc")
+	recs := []scanRec{
+		{size: 1000, head: fh, tail: ft, content: fc, path: "/A/x/f"},
+		{size: 500, head: gh, tail: gt, content: gc, path: "/A/y/g"},
+		{size: 1000, head: fh, tail: ft, content: fc, path: "/B/x/f"},
+		{size: 500, head: gh, tail: gt, content: gc, path: "/B/y/g"},
+		{size: 1000, head: fh, tail: ft, content: fc, path: "/C/x/f"},
+	}
+
+	super, dirs := treeOf(t, recs)
+	dupes := collectTreeGroups(dirs, super, everyPathExists, nil)
+
+	wantGroups := [][]string{{"/A", "/B"}, {"/A/x", "/B/x", "/C/x"}}
+	if got := groupPaths(dupes); !slices.EqualFunc(got, wantGroups,
+		slices.Equal) {
+		t.Fatalf("groups = %v, want %v", got, wantGroups)
+	}
+
+	if got := reclaimableBytes(dupes); got != 2500 {
+		t.Errorf("reclaimableBytes = %d, want 2500", got)
+	}
+}
+
+// TestReclaimableBytesCountsDupeInsideGrandparentDupeOnce checks that a
+// dupe tree adds nothing when a dupe tree above its parent holds it.
+// /B/m/x lies in /B, and /B/m is no dupe, since the group of /A and /B
+// implies that of /A/m and /B/m. The total adds the sizes of /B and
+// /C/x, 1300 + 1000.
+func TestReclaimableBytesCountsDupeInsideGrandparentDupeOnce(t *testing.T) {
+	t.Parallel()
+
+	f, w := testHash("f"), testHash("w")
+	recs := []scanRec{
+		{size: 1000, head: f, tail: f, content: f, path: "/A/m/x/f"},
+		{size: 300, head: w, tail: w, content: w, path: "/A/m/w"},
+		{size: 1000, head: f, tail: f, content: f, path: "/B/m/x/f"},
+		{size: 300, head: w, tail: w, content: w, path: "/B/m/w"},
+		{size: 1000, head: f, tail: f, content: f, path: "/C/x/f"},
+	}
+
+	super, dirs := treeOf(t, recs)
+	dupes := collectTreeGroups(dirs, super, everyPathExists, nil)
+
+	wantGroups := [][]string{{"/A", "/B"}, {"/A/m/x", "/B/m/x", "/C/x"}}
+	if got := groupPaths(dupes); !slices.EqualFunc(got, wantGroups,
+		slices.Equal) {
+		t.Fatalf("groups = %v, want %v", got, wantGroups)
+	}
+
+	if got := reclaimableBytes(dupes); got != 2300 {
+		t.Errorf("reclaimableBytes = %d, want 2300", got)
+	}
+}