trees: never sit silent; show its phases as N of M with progress, elapsed time and ETA #113

New Issue

2026-10-16T13:51:04+02:00

clawbot commented

2026-10-16 13:51:04 +02:00

> > root@las1stor1:~# go/bin/sfdupes trees > <hangs> >
>
> The tool should always give the user some feedback to show it is doing something and isn't hung. It should show what phase it's on, out of how many phases, and show phase progress, elapsed, and eta.

and on #56 (comment):

> Took far too long, strace on the pid was hanging too.

His hang was the old v0.0.1 waiting on stdin (#56 (comment)). On next, trees still prints nothing until its summary line. It reads every record in path order, builds the directory tree, groups directories by digest, sorts the groups and writes them. That is about a minute or more of silence at 10 million records. README "Progress" says "report and trees modes need no progress display"; sneak's words above replace that.

Definition of done

From the moment trees starts until its summary line, stderr always shows which phase it is in out of how many, with that phase's progress, elapsed time and ETA. This holds on a terminal and in plain lines (stderr not a terminal: a line as each phase starts, then at least every 5 seconds whether or not an item completed, and no more often). That includes opening the database and waiting on a busy one.
The phases are the implementer's to name. A plain split: read the records (total from a count of the records taken first, which already shows elapsed time while it runs), group the directories (total: the directories found), write the report (total: the groups). Every phase whose total is known shows [m/n], percent, rate, elapsed and ETA.
It uses the same display and phase numbering as scan (#112), which lands first. Build on what that change adds, without a second display.
stdout stays exactly as it is: same rows, same order, same summary line on stderr.
README "Progress" and "trees mode" updated: trees shows progress, and what its phases are.
Tests: the plain lines name each phase with its number and count; stdout unchanged for the existing cases. Keep each test well under a second (#109).
TODO.md Completed Steps entry in the same commit; script/cibuild green on a fresh clone.

Notes for the implementer

Package main (the repo's only package). The phase work stays in trees.go, the display in progress.go, and the record count query in db.go, beside loadFileRows.
Small and plain; no new flags, no new dependencies.

Model: opus-5-5

sneak, on https://git.eeqj.de/sneak/sfdupes/issues/56#issuecomment-145000: > ``` > root@las1stor1:~# go/bin/sfdupes trees > <hangs> > ``` > > The tool should always give the user some feedback to show it is doing something and isn't hung. It should show what phase it's on, out of how many phases, and show phase progress, elapsed, and eta. and on https://git.eeqj.de/sneak/sfdupes/issues/56#issuecomment-145001: > Took far too long, strace on the pid was hanging too. His hang was the old `v0.0.1` waiting on stdin (https://git.eeqj.de/sneak/sfdupes/issues/56#issuecomment-145003). On `next`, `trees` still prints nothing until its summary line. It reads every record in path order, builds the directory tree, groups directories by digest, sorts the groups and writes them. That is about a minute or more of silence at 10 million records. README "Progress" says "`report` and `trees` modes need no progress display"; sneak's words above replace that. ## Definition of done - From the moment `trees` starts until its summary line, stderr always shows which phase it is in out of how many, with that phase's progress, elapsed time and ETA. This holds on a terminal and in plain lines (stderr not a terminal: a line as each phase starts, then at least every 5 seconds whether or not an item completed, and no more often). That includes opening the database and waiting on a busy one. - The phases are the implementer's to name. A plain split: read the records (total from a count of the records taken first, which already shows elapsed time while it runs), group the directories (total: the directories found), write the report (total: the groups). Every phase whose total is known shows `[m/n]`, percent, rate, elapsed and ETA. - It uses the same display and phase numbering as `scan` (https://git.eeqj.de/sneak/sfdupes/issues/112), which lands first. Build on what that change adds, without a second display. - stdout stays exactly as it is: same rows, same order, same summary line on stderr. - README "Progress" and "`trees` mode" updated: `trees` shows progress, and what its phases are. - Tests: the plain lines name each phase with its number and count; stdout unchanged for the existing cases. Keep each test well under a second (https://git.eeqj.de/sneak/sfdupes/issues/109). - `TODO.md` Completed Steps entry in the same commit; `script/cibuild` green on a fresh clone. ## Notes for the implementer - Package `main` (the repo's only package). The phase work stays in `trees.go`, the display in `progress.go`, and the record count query in `db.go`, beside `loadFileRows`. - Small and plain; no new flags, no new dependencies. Model: opus-5-5

clawbot added this to the 1.0.0 milestone 2026-10-16 13:51:04 +02:00

clawbot referenced this issue from a commit

2026-10-16 19:23:57 +02:00

trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113)

clawbot referenced a pull request that will close this issue

2026-10-16 19:25:52 +02:00

trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113) #121

clawbot commented

2026-10-16 19:26:01 +02:00

Built in #121: trees runs in four phases, [1/4] open, [2/4] read, [3/4] group and [4/4] write, on the display scan uses. The PR body says what each phase counts and gives the judgement call.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/sfdupes/pulls/121: `trees` runs in four phases, `[1/4] open`, `[2/4] read`, `[3/4] group` and `[4/4] write`, on the display `scan` uses. The PR body says what each phase counts and gives the judgement call. Model: opus-5-5

clawbot referenced this issue from a commit

2026-10-16 19:54:50 +02:00

trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113)

clawbot referenced this issue from a commit

2026-10-16 19:58:06 +02:00

trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113)

clawbot referenced this issue

2026-10-16 20:22:30 +02:00

trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113) #121

clawbot referenced this issue from a commit

2026-10-16 20:36:40 +02:00

trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113)