trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113)

trees now runs in four phases, open, read, group and write, each under
the progress display scan uses, from the line naming the database to
the summary line. open covers a wait on a busy database. read counts
the records first, as scan's load does, then reads them against that
count; group counts the directories, and write the groups. The display
gains a unit so that each rate names what its phase counts; every scan
phase passes files and prints as before. The tests that pinned trees'
stderr now allow the phase lines between the database line and the
summary.

Judgement call: opening the database is a phase of its own, so there are four phases, not the three the issue sketches.
Judgement call: the count and the read are separate queries, as in scan's load, so during a scan the read can end slightly past or short of its total.

Model: opus-5-5

README.md

@@ -695,6 +695,12 @@ Processing:
 - Order groups by total tree size descending, tie-broken by `first` path
   ascending. Output must be fully deterministic for a given input.
 
+`trees` runs in four phases, each with its own progress display on stderr (see
+"Progress"): `open` opens the database; `read` counts the records, then reads
+them in path order, building the hierarchy and computing the digests as it goes;
+`group` groups the directories by digest and picks the maximal groups; `write`
+writes the groups to stdout.
+
 #### Trees output format
 
 TSV on stdout: a header line, then one row per duplicate tree (N-1 rows for a
@@ -715,13 +721,16 @@ units.
 
 ### Progress
 
-Use the progress-bar library for all scan progress; rendering in the style of
+Use the progress-bar library for all progress of `scan` and `trees`; rendering
-`pv` is the model. All progress goes to stderr.
+in the style of `pv` is the model. All progress goes to stderr.
 
-A scan must always show that it is working and not hung. Each phase gets its own
+`scan` and `trees` must always show that they are working and not hung. Each
-display from the moment it starts, naming the phase and its place among the five
+phase gets its own display from the moment it starts, naming the phase and its
-(`[1/5] load`, `[2/5] walk`, `[3/5] hash`, `[4/5] update`, `[5/5] content`) and
+place among the command's phases and showing its progress, its elapsed time and
-showing its progress, its elapsed time and its estimated time remaining (ETA):
+its estimated time remaining (ETA).
+
+A scan has five phases (`[1/5] load`, `[2/5] walk`, `[3/5] hash`,
+`[4/5] update`, `[5/5] content`):
 
 - `load` counts the records first, then reads them against that count. While the
   count runs, the display says it is counting the records.
@@ -738,12 +747,23 @@ showing its progress, its elapsed time and its estimated time remaining (ETA):
   and ETA, start there. The files to read then get a display of their own, with
   their count as its total.
 
+`trees` has four phases (`[1/4] open`, `[2/4] read`, `[3/4] group`,
+`[4/4] write`, see "`trees` mode"):
+
+- `open` has no total. While it opens the database, waiting on a busy one
+  included, the display says it is opening the database.
+- `read` counts the records first, then reads them against that count. While the
+  count runs, the display says it is counting the records.
+- `group` has the directories `read` found as its total, and `write` the
+  duplicate-tree groups.
+
 Each display is one line holding:
 
 - the phase and its place among the phases
 - a `[m/n] x%` display (items processed / total items, percent); before a phase
   has a total, the items processed, or what the phase is waiting for
-- current rate (items/s)
+- current rate (items/s), naming the items the phase counts: files, records,
+  directories or groups
 - elapsed time
 - estimated time remaining, or `unknown`
 
@@ -756,13 +776,13 @@ Example shape (exact layout is flexible, content is not):
 
 Additional requirements:
 
-- On a terminal, a phase with an exact total (every phase but `walk`, once it
+- On a terminal, a phase with an exact total (every phase but `scan`'s `walk`
-  has counted what it will do) is drawn as its line followed by the progress-bar
+  and `trees`' `open`, once it has counted what it will do) is drawn as its line
-  library's bar, 10 columns wide, which shows the percent in place of the line's
+  followed by the progress-bar library's bar, 10 columns wide, which shows the
-  own. Any other phase, and a phase whose bar has no room, is drawn as the
+  percent in place of the line's own. Any other phase, and a phase whose bar has
-  library's spinner followed by its line. Each display is drawn again several
+  no room, is drawn as the library's spinner followed by its line. Each display
-  times a second, so its elapsed time keeps moving while a phase waits for its
+  is drawn again several times a second, so its elapsed time keeps moving while
-  next item.
+  a phase waits for its next item.
 - On a terminal no line reaches the terminal's last column, however narrow the
   terminal is. The phase and its place, the progress, the elapsed time and the
   ETA are kept whole: a line too wide for the terminal leaves out its rate first
@@ -784,8 +804,7 @@ Additional requirements:
   inside the progress display.
 - A phase that stops short of its total, as an interrupted one does, is left
   showing the count it stopped at rather than filled up.
-- `report` and `trees` modes need no progress display, only their stderr
+- `report` mode needs no progress display, only its stderr summary.
-  summaries.
 
 ### Error handling and exit codes
 

TODO.md

@@ -29,6 +29,15 @@
 
 # Completed Steps
 
+- `trees` shows its work on stderr in four phases, `[1/4] open`, `[2/4] read`,
+  `[3/4] group` and `[4/4] write`, each with the progress display `scan` uses,
+  from the moment it starts until the summary line: `open` says it is opening
+  the database, also while it waits on a busy one; `read` counts the records,
+  then reads them against that count; `group` counts the directories it groups,
+  and `write` the groups it writes. A display's rate names what its phase
+  counts: files, records, directories or groups. stdout and the summary line are
+  unchanged (2026-10-16, https://git.eeqj.de/sneak/sfdupes/issues/113)
+
 - each `scan` phase's progress display names the phase and its place among the
   five, as `[2/5] walk`, from the moment it starts, with its progress, elapsed
   time and ETA: `load` counts the records first, the walk's total is estimated

main_test.go

@@ -303,8 +303,15 @@ func TestRunMissingDatabaseIsFatal(t *testing.T) {
 				t.Errorf("run(%s) = %d, want %d", name, code, exitFatal)
 			}
 
+			// trees shows its open phase, as it starts and as it ends,
+			// before the error.
+			opening := ""
+			if name == cmdTrees {
+				opening = treesOpenLine + "\n" + treesOpenLine + "\n"
+			}
+
 			want := name + ": database " + path + " (from " + databaseEnv +
-				")\n" +
+				")\n" + opening +
 				"sfdupes: " + path + ": no database (run \"sfdupes " +
 				"scan\" first, or set " + databaseEnv + ")\n"
 			if got := stderr(); got != want {
@@ -506,7 +513,7 @@ func TestRunNamesDefaultDatabase(t *testing.T) {
 	// nothing here may create or read one: scan gets an operand that
 	// does not exist, which stops it before it takes the lock, and
 	// report and trees a cancelled context, which stops them before
-	// they read anything.
+	// they read anything. trees shows its open phase after the line.
 	t.Setenv(databaseEnv, "")
 
 	// Whether scan calls the database new depends on that machine too.
@@ -540,7 +547,8 @@ func TestRunNamesDefaultDatabase(t *testing.T) {
 			run: func() error {
 				return runTrees(cancelledContext(t), io.Discard)
 			},
-			want: "trees: database " + defaultDatabasePath + " (default)\n",
+			want: "trees: database " + defaultDatabasePath + " (default)\n" +
+				treesOpenLine + "\n" + treesOpenLine + "\n",
 		},
 	}
 
@@ -942,11 +950,16 @@ func TestRunTreesSucceeds(t *testing.T) {
 		t.Errorf("stdout = %q, want %q", got, want)
 	}
 
-	want = "trees: database " + path + " (from " + databaseEnv + ")\n" +
+	// Its phases come between the two lines; see
-		"trees: 2 records read, 1 duplicate tree groups, 1 dupe trees, " +
+	// TestRunTreesProgressNamesEachPhase.
-		"300 B reclaimable\n"
+	first := "trees: database " + path + " (from " + databaseEnv + ")\n"
-	if got := stderr(); got != want {
+	last := "\ntrees: 2 records read, 1 duplicate tree groups, " +
-		t.Errorf("stderr = %q, want %q", got, want)
+		"1 dupe trees, 300 B reclaimable\n"
+
+	if got := stderr(); !strings.HasPrefix(got, first) ||
+		!strings.HasSuffix(got, last) {
+		t.Errorf("stderr = %q, want it to start %q and end %q",
+			got, first, last)
 	}
 
 	assertNoSidecars(t, path)

progress.go

@@ -72,6 +72,7 @@ func phaseLabel(phases []string, phase string) string {
 // progressParams describe one phase's progress display as it starts.
 type progressParams struct {
 	label      string // the phase and its place, from phaseLabel
+	unit       string // what the phase counts, as "files"
 	total      int64  // the items the phase will complete; -1 when not known
 	estimate   bool   // total is an estimate, which the count may pass
 	waitingFor string // what the phase waits for until setTotal is called
@@ -101,6 +102,7 @@ type progress struct {
 	withBar    bool                     // bar is the library's bar, not its spinner
 	widest     int                      // the widest line bar has drawn, in columns
 	label      string
+	unit       string
 	total      int64
 	estimate   bool
 	waitingFor string
@@ -128,8 +130,9 @@ func startProgress(params progressParams, width func() int,
 	now := time.Now()
 	p := &progress{
 		width: width, stop: make(chan struct{}), done: make(chan struct{}),
-		label: params.label, total: params.total, estimate: params.estimate,
+		label: params.label, unit: params.unit, total: params.total,
-		waitingFor: params.waitingFor, start: now, rateStart: now,
+		estimate: params.estimate, waitingFor: params.waitingFor,
+		start: now, rateStart: now,
 	}
 
 	// Show the zero state at once: the first item may take minutes,
@@ -311,9 +314,9 @@ func (p *progress) finish() {
 //
 //	[3/5] hash: [1234/5000] 25%, 300 files/s, elapsed 4s, eta 13s
 //
-// without the percent when percent is false, as before the library's
+// where "files" is the phase's unit, without the percent when percent
-// bar, which shows its own. A phase with no total shows the items done
+// is false, as before the library's bar, which shows its own. A phase
-// instead, or what it waits for.
+// with no total shows the items done instead, or what it waits for.
 //
 // The phase and its place, the progress, the elapsed time and the ETA
 // are kept whole. A line wider than room leaves out the rate, or what
@@ -337,7 +340,7 @@ func (p *progress) line(room int, percent bool) (string, bool) {
 		}
 
 		head = p.label + ": " + p.itemsDone(percent) + ", "
-		extra = fmt.Sprintf("%.0f files/s", rate)
+		extra = fmt.Sprintf("%.0f %s/s", rate, p.unit)
 		tail = fmt.Sprintf("elapsed %s, eta %s", elapsed, p.eta(rate))
 	}
 
@@ -355,11 +358,11 @@ func (p *progress) line(room int, percent bool) (string, bool) {
 
 // itemsDone formats the items done, as "[1234/5000] 25%", with "~"
 // before a total that is an estimate and without the percent when
-// percent is false, or as "1234 files" when the phase has no total.
+// percent is false, or as "1234 files", in the phase's unit, when the
-// p.mu must be held.
+// phase has no total. p.mu must be held.
 func (p *progress) itemsDone(percent bool) string {
 	if p.total < 0 {
-		return fmt.Sprintf("%d files", p.count)
+		return fmt.Sprintf("%d %s", p.count, p.unit)
 	}
 
 	approx := ""

progress_test.go

@@ -86,30 +86,44 @@ func TestProgressLine(t *testing.T) {
 		},
 		want: "[1/5] load: counting the records, elapsed 10s, eta unknown",
 	}, {
-		p:    &progress{label: scanPhase("hash"), total: 100, count: 20},
+		p: &progress{
+			label: scanPhase("hash"), unit: scanUnit, total: 100, count: 20,
+		},
 		want: "[3/5] hash: [20/100] 20%, 2 files/s, elapsed 10s, eta 40s",
 	}, {
-		p:    &progress{label: scanPhase("hash"), total: 100},
+		p:    &progress{label: scanPhase("hash"), unit: scanUnit, total: 100},
 		want: "[3/5] hash: [0/100] 0%, 0 files/s, elapsed 10s, eta unknown",
 	}, {
-		p:    &progress{label: scanPhase("update"), total: 0},
+		p:    &progress{label: scanPhase("update"), unit: scanUnit, total: 0},
 		want: "[4/5] update: [0/0] 100%, 0 files/s, elapsed 10s, eta 0s",
 	}, {
 		p: &progress{
-			label: scanPhase("walk"), total: 100, estimate: true, count: 20,
+			label: scanPhase("walk"), unit: scanUnit, total: 100,
+			estimate: true, count: 20,
 		},
 		want: "[2/5] walk: [20/~100] 20%, 2 files/s, elapsed 10s, eta 40s",
 	}, {
 		// Past the estimate the walk cannot tell how long it has left.
 		p: &progress{
-			label: scanPhase("walk"), total: 100, estimate: true, count: 120,
+			label: scanPhase("walk"), unit: scanUnit, total: 100,
+			estimate: true, count: 120,
 		},
 		want: "[2/5] walk: [120/~100] 120%, 12 files/s, elapsed 10s, " +
 			"eta unknown",
 	}, {
 		// A first scan's walk has no estimate.
-		p:    &progress{label: scanPhase("walk"), total: -1, count: 20},
+		p: &progress{
+			label: scanPhase("walk"), unit: scanUnit, total: -1, count: 20,
+		},
 		want: "[2/5] walk: 20 files, 2 files/s, elapsed 10s, eta unknown",
+	}, {
+		// The rate is in what the phase counts.
+		p: &progress{
+			label: treesPhase("group"), unit: "directories", total: 100,
+			count: 20,
+		},
+		want: "[3/4] group: [20/100] 20%, 2 directories/s, elapsed 10s, " +
+			"eta 40s",
 	}}
 
 	for _, tc := range cases {
@@ -129,7 +143,7 @@ func TestLineFitsRoom(t *testing.T) {
 
 	ago := time.Now().Add(-10 * time.Second)
 	hash := &progress{
-		label: scanPhase("hash"), total: 100, count: 20,
+		label: scanPhase("hash"), unit: scanUnit, total: 100, count: 20,
 		start: ago, rateStart: ago,
 	}
 	content := &progress{
@@ -183,7 +197,8 @@ func TestSetTotalStartsRate(t *testing.T) {
 
 	waited := time.Now().Add(-20 * time.Second)
 	p := &progress{
-		label: scanPhase("load"), total: -1, start: waited, rateStart: waited,
+		label: scanPhase("load"), unit: scanUnit, total: -1, start: waited,
+		rateStart: waited,
 	}
 
 	p.setTotal(100)
@@ -226,7 +241,9 @@ func awaitOutput(t *testing.T, stderr func() string, want string,
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestPlainProgressPrintsAtStartAndEnd(t *testing.T) {
 	stderr := captureStderr(t)
-	p := newProgress(progressParams{label: scanPhase("hash"), total: 1000})
+	p := newProgress(progressParams{
+		label: scanPhase("hash"), unit: scanUnit, total: 1000,
+	})
 
 	for range 1000 {
 		p.increment()
@@ -289,8 +306,14 @@ func TestTerminalRedrawsWhileWaiting(t *testing.T) {
 		params progressParams
 		want   string // how the line shows a count of 50
 	}{
-		{progressParams{label: scanPhase("walk"), total: -1}, "50 files,"},
+		{
-		{progressParams{label: scanPhase("hash"), total: 100}, "[50/100],"},
+			progressParams{label: scanPhase("walk"), unit: scanUnit, total: -1},
+			"50 files,",
+		},
+		{
+			progressParams{label: scanPhase("hash"), unit: scanUnit, total: 100},
+			"[50/100],",
+		},
 	}
 
 	for _, tc := range cases {
@@ -341,8 +364,9 @@ func drawLate(stderr func() string, params progressParams, width int,
 }
 
 // TestTerminalLinesKeepETA checks the line each display shows on an
-// 80-column terminal at a large scan's counts and times: the rate is
+// 80-column terminal at a large scan's counts and times, and the line
-// left out, so that the elapsed time and the ETA stay whole.
+// of trees' read phase at the same counts: the rate is left out, so
+// that the elapsed time and the ETA stay whole.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestTerminalLinesKeepETA(t *testing.T) {
@@ -354,18 +378,29 @@ func TestTerminalLinesKeepETA(t *testing.T) {
 	}{{
 		// Past its estimate, the walk cannot tell how long it has left.
 		params: progressParams{
-			label: scanPhase("walk"), total: 1200000, estimate: true,
+			label: scanPhase("walk"), unit: scanUnit, total: 1200000,
+			estimate: true,
 		},
 		want: "[2/5] walk: [1234567/~1200000] 103%, elapsed 2h30m0s, " +
 			"eta unknown",
 	}, {
-		params: progressParams{label: scanPhase("hash"), total: 9876543},
+		params: progressParams{
+			label: scanPhase("hash"), unit: scanUnit, total: 9876543,
+		},
 		want: "[3/5] hash: [1234567/9876543] 12%, elapsed 2h30m0s, " +
 			"eta 17h30m0s",
 	}, {
-		params: progressParams{label: scanPhase("content"), total: 9876543},
+		params: progressParams{
+			label: scanPhase("content"), unit: scanUnit, total: 9876543,
+		},
 		want: "[5/5] content: [1234567/9876543] 12%, elapsed 2h30m0s, " +
 			"eta 17h30m0s",
+	}, {
+		params: progressParams{
+			label: treesPhase("read"), unit: "records", total: 9876543,
+		},
+		want: "[2/4] read: [1234567/9876543] 12%, elapsed 2h30m0s, " +
+			"eta 17h30m0s",
 	}}
 
 	for _, tc := range cases {
@@ -389,7 +424,9 @@ func TestTerminalLinesFitWidth(t *testing.T) {
 
 	for _, width := range []int{120, 100, 80, 72, 60, 50, 40} {
 		for _, total := range []int64{9876543, 20000} {
-			params := progressParams{label: scanPhase("content"), total: total}
+			params := progressParams{
+				label: scanPhase("content"), unit: scanUnit, total: total,
+			}
 			out := drawLate(stderr, params, width, total/8)
 
 			if width == 120 && !strings.Contains(out, "% |") {
@@ -426,8 +463,16 @@ func TestTerminalNarrowedRedrawsInPlace(t *testing.T) {
 		params progressParams
 		narrow int
 	}{
-		{progressParams{label: scanPhase("walk"), total: -1}, 60},
+		{
-		{progressParams{label: scanPhase("hash"), total: 9876543}, 90},
+			progressParams{label: scanPhase("walk"), unit: scanUnit, total: -1},
+			60,
+		},
+		{
+			progressParams{
+				label: scanPhase("hash"), unit: scanUnit, total: 9876543,
+			},
+			90,
+		},
 	}
 
 	for _, tc := range cases {

scan.go

@@ -84,6 +84,10 @@ type fileMeta struct {
 	hashed bool
 }
 
+// scanUnit is what every scan phase counts, as its progress display
+// names it.
+const scanUnit = "files"
+
 // scanPhase names one phase of a scan with its place among all of them,
 // as "[2/5] walk", for the phase's progress display. The list here is
 // the one place the phases are counted.
@@ -441,7 +445,7 @@ func (s *scanState) loadIndex(ctx context.Context, roots []string) error {
 	// display the scan looks hung before the walk begins. Counting them
 	// first gives the display its total.
 	prog := newProgress(progressParams{
-		label: scanPhase("load"), total: -1,
+		label: scanPhase("load"), unit: scanUnit, total: -1,
 		waitingFor: "counting the records",
 	})
 	defer prog.finish()
@@ -492,7 +496,8 @@ func (s *scanState) walkPhase(
 	}
 
 	prog := newProgress(progressParams{
-		label: scanPhase("walk"), total: estimate, estimate: true,
+		label: scanPhase("walk"), unit: scanUnit, total: estimate,
+		estimate: true,
 	})
 
 	var (
@@ -683,7 +688,7 @@ func (s *scanState) readRuns(ctx context.Context, workers int,
 	defer pool.stop()
 
 	prog := newProgress(progressParams{
-		label: scanPhase(label), total: int64(len(runs)),
+		label: scanPhase(label), unit: scanUnit, total: int64(len(runs)),
 	})
 	defer prog.finish()
 
@@ -772,7 +777,7 @@ func (s *scanState) updatePhase(ctx context.Context) error {
 
 	total := len(s.batch) + len(s.sentinels) + len(deletes)
 	prog := newProgress(progressParams{
-		label: scanPhase("update"), total: int64(total),
+		label: scanPhase("update"), unit: scanUnit, total: int64(total),
 	})
 
 	defer prog.finish()
@@ -860,7 +865,7 @@ func (s *scanState) contentCandidates(
 	// record. The checks start there, against a count of the records
 	// the search returns, and so do their rate and ETA.
 	prog := newProgress(progressParams{
-		label: scanPhase("content"), total: -1,
+		label: scanPhase("content"), unit: scanUnit, total: -1,
 		waitingFor: "searching the database",
 	})
 	defer prog.finish()

scan_test.go

@@ -1550,7 +1550,7 @@ func assertSmokeTreeGroups(t *testing.T, dir string, db *sql.DB) {
 
 	super, dirs := dbTree(t, db)
 
-	tg := collectTreeGroups(dirs, super)
+	tg := collectTreeGroups(dirs, super, nil)
 	if len(tg) != 1 {
 		t.Fatalf("len(tree groups) = %d, want 1", len(tg))
 	}
@@ -2092,7 +2092,7 @@ func TestTreesUnhashedNeverEqual(t *testing.T) {
 	for name, unknown := range cases {
 		super, dirs := treeOf(t, append(slices.Clone(shared), unknown...))
 
-		if tg := collectTreeGroups(dirs, super); len(tg) != 0 {
+		if tg := collectTreeGroups(dirs, super, nil); len(tg) != 0 {
 			t.Errorf("%s: tree groups = %d, want 0 (the files may differ)",
 				name, len(tg))
 		}

trees.go

@@ -4,6 +4,7 @@ import (
 	"bufio"
 	"context"
 	"crypto/sha256"
+	"database/sql"
 	"fmt"
 	"io"
 	"os"
@@ -24,6 +25,13 @@ type treeNode struct {
 	totalSize int64
 }
 
+// treesPhase names one phase of trees with its place among all of them,
+// as "[2/4] read", for the phase's progress display. The list here is
+// the one place the phases are counted.
+func treesPhase(name string) string {
+	return phaseLabel([]string{"open", "read", "group", "write"}, name)
+}
+
 // runTrees implements the trees subcommand: it reads every record from
 // the database in path order, reconstructs the directory hierarchy from
 // the record paths, computes a Merkle-style digest per directory, and
@@ -31,6 +39,13 @@ type treeNode struct {
 // touches the scanned filesystem; its only I/O is the database, stdout,
 // and stderr. It names the database on stderr before it opens it. Any
 // database problem, including a missing database, is fatal.
+//
+// The work runs in four phases, each under a progress display of its
+// own on stderr: open (open the database), read (count the records,
+// then read them into the hierarchy), group (group the directories by
+// digest) and write (write the groups to stdout). Each display ends
+// before the next phase starts, and before an error is returned to be
+// printed.
 func runTrees(ctx context.Context, stdout io.Writer) error {
 	dbPath, source, err := databasePath()
 	if err != nil {
@@ -40,34 +55,105 @@ func runTrees(ctx context.Context, stdout io.Writer) error {
 	fmt.Fprintf(os.Stderr, "trees: database %s (%s)\n",
 		escapePath(dbPath), source)
 
+	// Opening can wait on a busy database for as long as the busy
+	// timeout.
+	prog := newProgress(progressParams{
+		label: treesPhase("open"), total: -1,
+		waitingFor: "opening the database",
+	})
 	db, err := openReportDatabase(ctx, dbPath)
+
+	prog.finish()
+
 	if err != nil {
 		return err
 	}
 
 	defer func() { _ = db.Close() }()
 
+	prog = newProgress(progressParams{
+		label: treesPhase("read"), unit: "records", total: -1,
+		waitingFor: "counting the records",
+	})
+	records, super, allDirs, err := readTree(ctx, db, prog)
+
+	prog.finish()
+
+	if err != nil {
+		return fmt.Errorf("database %s: %w", dbPath, err)
+	}
+
+	prog = newProgress(progressParams{
+		label: treesPhase("group"), unit: "directories",
+		total: int64(len(allDirs)),
+	})
+	dupes := collectTreeGroups(allDirs, super, prog)
+
+	prog.finish()
+
+	prog = newProgress(progressParams{
+		label: treesPhase("write"), unit: "groups", total: int64(len(dupes)),
+	})
+	dupeTrees, reclaimable, err := writeTreeGroups(stdout, dupes, prog)
+
+	prog.finish()
+
+	if err != nil {
+		return err
+	}
+
+	fmt.Fprintf(os.Stderr,
+		"trees: %d records read, %d duplicate tree groups, %d dupe trees, "+
+			"%s reclaimable\n",
+		records, len(dupes), dupeTrees, humanBytes(reclaimable))
+
+	return nil
+}
+
+// readTree counts the records in db and gives prog the count as its
+// total, then reads every record in path order into the directory
+// hierarchy, counting each on prog. It returns the number of records
+// read, the super-root and every directory.
+func readTree(ctx context.Context, db *sql.DB,
+	prog *progress,
+) (int, *treeNode, []*treeNode, error) {
+	total, err := countRecords(ctx, db)
+	if err != nil {
+		return 0, nil, nil, err
+	}
+
+	prog.setTotal(total)
+
 	records := 0
 	tree := newTreeBuilder()
 
 	err = loadFileRows(ctx, db, func(r scanRec) {
 		records++
 
+		prog.increment()
 		tree.add(r)
 	})
 	if err != nil {
-		return fmt.Errorf("database %s: %w", dbPath, err)
+		return 0, nil, nil, err
 	}
 
 	super, allDirs := tree.finish()
 
-	dupes := collectTreeGroups(allDirs, super)
+	return records, super, allDirs, nil
+}
 
+// writeTreeGroups writes dupes to stdout as TSV, a header line and then
+// a row for every member of each group but its first, counting each
+// group on prog once its rows are written. It returns the number of
+// rows (dupe trees) and the bytes they could reclaim.
+func writeTreeGroups(stdout io.Writer, dupes [][]*treeNode,
+	prog *progress,
+) (int, int64, error) {
 	out := bufio.NewWriterSize(stdout, ioBufSize)
 
-	_, err = fmt.Fprintln(out, "first\tdupe\tfiles\tsize")
+	_, err := fmt.Fprintln(out, "first\tdupe\tfiles\tsize")
 	if err != nil {
-		return fmt.Errorf("write stdout: %w", err)
+		return 0, 0, fmt.Errorf("write stdout: %w", err)
 	}
 
 	dupeTrees := 0
@@ -81,25 +167,22 @@ func runTrees(ctx context.Context, stdout io.Writer) error {
 				escapePath(first.path), escapePath(n.path),
 				first.fileCount, first.totalSize)
 			if err != nil {
-				return fmt.Errorf("write stdout: %w", err)
+				return 0, 0, fmt.Errorf("write stdout: %w", err)
 			}
 
 			dupeTrees++
 			reclaimable += first.totalSize
 		}
+
+		prog.increment()
 	}
 
 	err = out.Flush()
 	if err != nil {
-		return fmt.Errorf("write stdout: %w", err)
+		return 0, 0, fmt.Errorf("write stdout: %w", err)
 	}
 
-	fmt.Fprintf(os.Stderr,
+	return dupeTrees, reclaimable, nil
-		"trees: %d records read, %d duplicate tree groups, %d dupe trees, "+
-			"%s reclaimable\n",
-		records, len(dupes), dupeTrees, humanBytes(reclaimable))
-
-	return nil
 }
 
 // treeBuilder reconstructs the directory hierarchy from records added
@@ -221,14 +304,18 @@ func fileEntry(name string, r scanRec) string {
 		"\x00" + r.head + "\x00" + r.tail + "\x00" + content
 }
 
-// collectTreeGroups groups directories by digest and returns every
+// collectTreeGroups groups directories by digest, counting each one on
-// maximal group with two or more members, each group's members sorted
+// prog as it is grouped, and returns every maximal group with two or
-// by path, groups ordered by tree size descending then by first path
+// more members, each group's members sorted by path, groups ordered by
-// ascending.
+// tree size descending then by first path ascending.
-func collectTreeGroups(allDirs []*treeNode, super *treeNode) [][]*treeNode {
+func collectTreeGroups(allDirs []*treeNode, super *treeNode,
+	prog *progress,
+) [][]*treeNode {
 	groups := make(map[[sha256.Size]byte][]*treeNode)
 	for _, d := range allDirs {
 		groups[d.digest] = append(groups[d.digest], d)
+
+		prog.increment()
 	}
 
 	var dupes [][]*treeNode

trees_test.go

@@ -3,7 +3,9 @@ package main
 import (
 	"bytes"
 	"database/sql"
+	"os"
 	"slices"
+	"strings"
 	"testing"
 )
 
@@ -17,6 +19,11 @@ const (
 	f2Content = "f2c"
 )
 
+// treesOpenLine is the plain line of trees' open phase when it opens
+// the database at once; it prints it as the phase starts and as it ends.
+const treesOpenLine = "[1/4] open: opening the database, elapsed 0s, " +
+	"eta unknown"
+
 // smokeTreeRecs mirrors the README smoke-test tree layout: /d/t1 and
 // /d/t2 are identical, /d/t3 differs from them only by one filename.
 func smokeTreeRecs() []scanRec {
@@ -170,7 +177,7 @@ func TestTreeNamesSortingBeforeSlash(t *testing.T) {
 		t.Fatalf("directory paths = %q, want %q", got, want)
 	}
 
-	groups := collectTreeGroups(dirs, super)
+	groups := collectTreeGroups(dirs, super, nil)
 
 	gotGroups := groupPaths(groups)
 	wantGroups := [][]string{{"/a", "/c"}}
@@ -203,6 +210,60 @@ func TestRunTreesEscapesPaths(t *testing.T) {
 	}
 }
 
+// TestRunTreesProgressNamesEachPhase runs trees on the smoke-test
+// records with stderr not a terminal, and checks every line it prints
+// there. After the line naming the database, each phase prints a line
+// naming it with its place among the four as it starts and another as
+// it ends. read counts the 6 records before it reads them, group counts
+// the 8 directories, and write the 1 group. The summary comes last, and
+// stdout holds the report alone.
+func TestRunTreesProgressNamesEachPhase(t *testing.T) {
+	path := seedDatabase(t, smokeTreeRecs())
+	t.Setenv(databaseEnv, path)
+
+	stderr := captureStderr(t)
+
+	var stdout bytes.Buffer
+
+	code := run([]string{cmdTrees}, &stdout, os.Stderr)
+	if code != exitOK {
+		t.Fatalf("run(trees) = %d, want %d; stderr: %s",
+			code, exitOK, stderr())
+	}
+
+	report := "first\tdupe\tfiles\tsize\n/d/t1\t/d/t2\t2\t3100\n"
+	if got := stdout.String(); got != report {
+		t.Errorf("stdout = %q, want %q", got, report)
+	}
+
+	// How each line starts; the rate at the end of a phase varies.
+	want := []string{
+		"trees: database " + path + " (from " + databaseEnv + ")",
+		treesOpenLine,
+		treesOpenLine,
+		"[2/4] read: counting the records, elapsed 0s, eta unknown",
+		"[2/4] read: [6/6] 100%, ",
+		"[3/4] group: [0/8] 0%, 0 directories/s, elapsed 0s, eta unknown",
+		"[3/4] group: [8/8] 100%, ",
+		"[4/4] write: [0/1] 0%, 0 groups/s, elapsed 0s, eta unknown",
+		"[4/4] write: [1/1] 100%, ",
+		"trees: 6 records read, 1 duplicate tree groups, 1 dupe trees, ",
+	}
+
+	lines := strings.Split(strings.TrimSuffix(stderr(), "\n"), "\n")
+	if len(lines) != len(want) {
+		t.Fatalf("stderr lines = %q, want %d lines starting %q",
+			lines, len(want), want)
+	}
+
+	for i := range want {
+		if !strings.HasPrefix(lines[i], want[i]) {
+			t.Errorf("stderr line %d = %q, want it to start %q",
+				i+1, lines[i], want[i])
+		}
+	}
+}
+
 func TestTreeDigests(t *testing.T) {
 	t.Parallel()
 
@@ -254,7 +315,7 @@ func TestCollectTreeGroupsMaximal(t *testing.T) {
 
 	super, dirs := treeOf(t, smokeTreeRecs())
 
-	groups := collectTreeGroups(dirs, super)
+	groups := collectTreeGroups(dirs, super, nil)
 
 	want := [][]string{{"/d/t1", "/d/t2"}}
 	if got := groupPaths(groups); !slices.EqualFunc(got, want,
@@ -278,14 +339,14 @@ func TestCollectTreeGroupsDeterministic(t *testing.T) {
 
 	super, dirs := treeOf(t, recs)
 
-	forward := groupPaths(collectTreeGroups(dirs, super))
+	forward := groupPaths(collectTreeGroups(dirs, super, nil))
 
 	reversed := slices.Clone(recs)
 	slices.Reverse(reversed)
 
 	superR, dirsR := treeOf(t, reversed)
 
-	backward := groupPaths(collectTreeGroups(dirsR, superR))
+	backward := groupPaths(collectTreeGroups(dirsR, superR, nil))
 	if !slices.EqualFunc(forward, backward, slices.Equal) {
 		t.Fatalf("output depends on record order: %v vs %v",
 			forward, backward)
@@ -304,7 +365,7 @@ func TestCollectTreeGroupsSiblings(t *testing.T) {
 
 	super, dirs := treeOf(t, recs)
 
-	got := groupPaths(collectTreeGroups(dirs, super))
+	got := groupPaths(collectTreeGroups(dirs, super, nil))
 
 	want := [][]string{{"/p/x1", "/p/x2"}}
 	if !slices.EqualFunc(got, want, slices.Equal) {
@@ -326,7 +387,7 @@ func TestCollectTreeGroupsDifferingParents(t *testing.T) {
 
 	super, dirs := treeOf(t, recs)
 
-	got := groupPaths(collectTreeGroups(dirs, super))
+	got := groupPaths(collectTreeGroups(dirs, super, nil))
 
 	want := [][]string{{"/p/a/x", "/q/b/x"}}
 	if !slices.EqualFunc(got, want, slices.Equal) {