1 Commits
Author SHA1 Message Date
clawbot 510b545ae5 trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113)
check / check (push) Canceled after 0s
trees now runs in four phases, open, read, group and write, each under
the progress display scan uses, until the summary line. open covers a
wait on a busy database. read counts the records and reads them in one
read-only transaction, so the two agree during a scan; group counts the
directories, and write the groups. The rows reach stdout through a
writer that clears the display and holds its lock while writing, so no
row shares a line with the display on a shared terminal. Each rate
names what its phase counts.

Judgement call: opening the database is a phase of its own, so there are four phases, not the three the issue sketches.

Model: opus-5-5
2026-10-16 17:58:04 +00:00
4 changed files with 16 additions and 93 deletions
+1 -4
View File
@@ -802,10 +802,7 @@ Additional requirements:
its own.
- A warning printed during a phase always lands on a line of its own, never
inside the progress display. So does every line written to stdout when stdout
and stderr are the same file (the same device and inode): one terminal, as in
a plain `sfdupes trees`, or one pipe or file that both are sent to. When they
are not, stdout is written without waiting on the display, and the display
keeps being drawn while a slow reader of stdout holds up a write.
is the same terminal, as in a plain `sfdupes trees`.
- A phase that stops short of its total, as an interrupted one does, is left
showing the count it stopped at rather than filled up.
- `report` mode needs no progress display, only its stderr summary.
+2 -3
View File
@@ -36,9 +36,8 @@
then reads them against that count; `group` counts the directories it groups,
and `write` the groups it writes. A display's rate names what its phase
counts: files, records, directories or groups. stdout and the summary line are
unchanged. When stdout and stderr are the same file, as one terminal is, none
of stdout's lines shares a line with the display; when they are not, the
display keeps being drawn while stdout waits on a slow reader (2026-10-16,
unchanged, and when stdout is the same terminal, none of its lines shares a
line with the display (2026-10-16,
https://git.eeqj.de/sneak/sfdupes/issues/113)
- each `scan` phase's progress display names the phase and its place among the
+13 -30
View File
@@ -292,42 +292,25 @@ func (p *progress) warnf(format string, args ...any) {
fmt.Fprintln(os.Stderr, msg)
}
// progressWriter writes a phase's output to out, which may be the file
// the phase's display is drawn on, as stdout is in a plain "sfdupes
// trees" on a terminal. It holds what is written to it and writes that
// to out in pieces of about ioBufSize bytes. When out is the file
// stderr is, each piece is written under the display's lock with the
// display cleared first, as warnf does for a warning: no draw lands in
// the middle of a piece, and the next draw shows the display again, on
// the line below. Every write to it must end with a newline, as a row
// written with one fmt.Fprintf does, so that each piece ends a line.
// Flush writes what it holds once the last row is written.
// progressWriter writes a phase's output to out, which may be the
// terminal the phase's display is drawn on, as stdout is in a plain
// "sfdupes trees". It holds what is written to it and writes that to
// out in pieces of about ioBufSize bytes, each under the display's
// lock with the display cleared first, as warnf does for a warning: no
// draw lands in the middle of a piece, and the next draw shows the
// display again, on the line below. Every write to it must end with a
// newline, as a row written with one fmt.Fprintf does, so that each
// piece ends a line. Flush writes what it holds once the last row is
// written.
type progressWriter struct {
// p is the display to clear, nil unless out is the file stderr is.
p *progress
out io.Writer
buf []byte
}
// writer returns a progressWriter writing to out around p's display.
// It clears the display and takes its lock only when out is the file
// stderr is: the same device and inode, as os.SameFile compares them,
// which is one terminal, or one pipe or file that both are sent to.
// Otherwise it writes to out without the lock, so the display keeps
// being drawn while a write to out waits for a slow reader.
func (p *progress) writer(out io.Writer) *progressWriter {
w := &progressWriter{out: out}
if f, ok := out.(*os.File); ok {
outInfo, outErr := f.Stat()
errInfo, errErr := os.Stderr.Stat()
if outErr == nil && errErr == nil && os.SameFile(outInfo, errInfo) {
w.p = p
}
}
return w
return &progressWriter{p: p, out: out}
}
// Write holds b, and writes what it holds to out once that comes to
@@ -341,8 +324,8 @@ func (w *progressWriter) Write(b []byte) (int, error) {
return len(b), w.Flush()
}
// Flush writes what w holds to out. When out is the file stderr is, the
// display is cleared first and not drawn again until it is written.
// Flush writes what w holds to out, with the display cleared first and
// not drawn again until it is written.
func (w *progressWriter) Flush() error {
if w.p != nil {
w.p.mu.Lock()
-56
View File
@@ -2,7 +2,6 @@ package main
import (
"fmt"
"io"
"math"
"os"
"path/filepath"
@@ -624,61 +623,6 @@ func TestProgressWriterRowsOnOwnLines(t *testing.T) {
}
}
// TestProgressWriterStalledStdoutKeepsLines writes a row larger than a
// pipe holds through the display's writer to a pipe nobody reads yet,
// as stdout is when its reader has stopped while stderr goes to a file:
// while the write waits, the display must go on printing its line.
//
//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
func TestProgressWriterStalledStdoutKeepsLines(t *testing.T) {
stderr := captureStderr(t)
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
_ = w.Close()
_ = r.Close()
})
p := startProgress(progressParams{label: treesPhase("write"), total: 1},
nil, testRedraw)
out := p.writer(w)
written := make(chan error, 1)
go func() {
_, writeErr := fmt.Fprintln(out, strings.Repeat("x", ioBufSize))
written <- writeErr
}()
// Pass or fail, the pipe is then read so that the write ends, and
// the display ends before stderr is put back.
t.Cleanup(func() {
go func() {
_, _ = io.Copy(io.Discard, r)
}()
writeErr := <-written
if writeErr != nil {
t.Error(writeErr)
}
p.finish()
})
// The first byte read shows that the write has started; the rest
// of the row waits until the pipe is read.
_, err = r.Read(make([]byte, 1))
if err != nil {
t.Fatal(err)
}
line := "[4/4] write: "
awaitOutput(t, stderr, line, strings.Count(stderr(), line)+3)
}
// TestTerminalStoppedShortKeepsCount checks that the terminal display
// of a phase that stops before its total, as an interrupted one does,
// is left showing the count it stopped at rather than a full bar.