1 Commits
Author SHA1 Message Date
clawbot 510b545ae5 trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113)
check / check (push) Canceled after 0s
trees now runs in four phases, open, read, group and write, each under
the progress display scan uses, until the summary line. open covers a
wait on a busy database. read counts the records and reads them in one
read-only transaction, so the two agree during a scan; group counts the
directories, and write the groups. The rows reach stdout through a
writer that clears the display and holds its lock while writing, so no
row shares a line with the display on a shared terminal. Each rate
names what its phase counts.

Judgement call: opening the database is a phase of its own, so there are four phases, not the three the issue sketches.

Model: opus-5-5
2026-10-16 17:58:04 +00:00
4 changed files with 16 additions and 93 deletions
+1 -4
View File
@@ -802,10 +802,7 @@ Additional requirements:
its own. its own.
- A warning printed during a phase always lands on a line of its own, never - A warning printed during a phase always lands on a line of its own, never
inside the progress display. So does every line written to stdout when stdout inside the progress display. So does every line written to stdout when stdout
and stderr are the same file (the same device and inode): one terminal, as in is the same terminal, as in a plain `sfdupes trees`.
a plain `sfdupes trees`, or one pipe or file that both are sent to. When they
are not, stdout is written without waiting on the display, and the display
keeps being drawn while a slow reader of stdout holds up a write.
- A phase that stops short of its total, as an interrupted one does, is left - A phase that stops short of its total, as an interrupted one does, is left
showing the count it stopped at rather than filled up. showing the count it stopped at rather than filled up.
- `report` mode needs no progress display, only its stderr summary. - `report` mode needs no progress display, only its stderr summary.
+2 -3
View File
@@ -36,9 +36,8 @@
then reads them against that count; `group` counts the directories it groups, then reads them against that count; `group` counts the directories it groups,
and `write` the groups it writes. A display's rate names what its phase and `write` the groups it writes. A display's rate names what its phase
counts: files, records, directories or groups. stdout and the summary line are counts: files, records, directories or groups. stdout and the summary line are
unchanged. When stdout and stderr are the same file, as one terminal is, none unchanged, and when stdout is the same terminal, none of its lines shares a
of stdout's lines shares a line with the display; when they are not, the line with the display (2026-10-16,
display keeps being drawn while stdout waits on a slow reader (2026-10-16,
https://git.eeqj.de/sneak/sfdupes/issues/113) https://git.eeqj.de/sneak/sfdupes/issues/113)
- each `scan` phase's progress display names the phase and its place among the - each `scan` phase's progress display names the phase and its place among the
+13 -30
View File
@@ -292,42 +292,25 @@ func (p *progress) warnf(format string, args ...any) {
fmt.Fprintln(os.Stderr, msg) fmt.Fprintln(os.Stderr, msg)
} }
// progressWriter writes a phase's output to out, which may be the file // progressWriter writes a phase's output to out, which may be the
// the phase's display is drawn on, as stdout is in a plain "sfdupes // terminal the phase's display is drawn on, as stdout is in a plain
// trees" on a terminal. It holds what is written to it and writes that // "sfdupes trees". It holds what is written to it and writes that to
// to out in pieces of about ioBufSize bytes. When out is the file // out in pieces of about ioBufSize bytes, each under the display's
// stderr is, each piece is written under the display's lock with the // lock with the display cleared first, as warnf does for a warning: no
// display cleared first, as warnf does for a warning: no draw lands in // draw lands in the middle of a piece, and the next draw shows the
// the middle of a piece, and the next draw shows the display again, on // display again, on the line below. Every write to it must end with a
// the line below. Every write to it must end with a newline, as a row // newline, as a row written with one fmt.Fprintf does, so that each
// written with one fmt.Fprintf does, so that each piece ends a line. // piece ends a line. Flush writes what it holds once the last row is
// Flush writes what it holds once the last row is written. // written.
type progressWriter struct { type progressWriter struct {
// p is the display to clear, nil unless out is the file stderr is.
p *progress p *progress
out io.Writer out io.Writer
buf []byte buf []byte
} }
// writer returns a progressWriter writing to out around p's display. // writer returns a progressWriter writing to out around p's display.
// It clears the display and takes its lock only when out is the file
// stderr is: the same device and inode, as os.SameFile compares them,
// which is one terminal, or one pipe or file that both are sent to.
// Otherwise it writes to out without the lock, so the display keeps
// being drawn while a write to out waits for a slow reader.
func (p *progress) writer(out io.Writer) *progressWriter { func (p *progress) writer(out io.Writer) *progressWriter {
w := &progressWriter{out: out} return &progressWriter{p: p, out: out}
if f, ok := out.(*os.File); ok {
outInfo, outErr := f.Stat()
errInfo, errErr := os.Stderr.Stat()
if outErr == nil && errErr == nil && os.SameFile(outInfo, errInfo) {
w.p = p
}
}
return w
} }
// Write holds b, and writes what it holds to out once that comes to // Write holds b, and writes what it holds to out once that comes to
@@ -341,8 +324,8 @@ func (w *progressWriter) Write(b []byte) (int, error) {
return len(b), w.Flush() return len(b), w.Flush()
} }
// Flush writes what w holds to out. When out is the file stderr is, the // Flush writes what w holds to out, with the display cleared first and
// display is cleared first and not drawn again until it is written. // not drawn again until it is written.
func (w *progressWriter) Flush() error { func (w *progressWriter) Flush() error {
if w.p != nil { if w.p != nil {
w.p.mu.Lock() w.p.mu.Lock()
-56
View File
@@ -2,7 +2,6 @@ package main
import ( import (
"fmt" "fmt"
"io"
"math" "math"
"os" "os"
"path/filepath" "path/filepath"
@@ -624,61 +623,6 @@ func TestProgressWriterRowsOnOwnLines(t *testing.T) {
} }
} }
// TestProgressWriterStalledStdoutKeepsLines writes a row larger than a
// pipe holds through the display's writer to a pipe nobody reads yet,
// as stdout is when its reader has stopped while stderr goes to a file:
// while the write waits, the display must go on printing its line.
//
//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
func TestProgressWriterStalledStdoutKeepsLines(t *testing.T) {
stderr := captureStderr(t)
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
_ = w.Close()
_ = r.Close()
})
p := startProgress(progressParams{label: treesPhase("write"), total: 1},
nil, testRedraw)
out := p.writer(w)
written := make(chan error, 1)
go func() {
_, writeErr := fmt.Fprintln(out, strings.Repeat("x", ioBufSize))
written <- writeErr
}()
// Pass or fail, the pipe is then read so that the write ends, and
// the display ends before stderr is put back.
t.Cleanup(func() {
go func() {
_, _ = io.Copy(io.Discard, r)
}()
writeErr := <-written
if writeErr != nil {
t.Error(writeErr)
}
p.finish()
})
// The first byte read shows that the write has started; the rest
// of the row waits until the pipe is read.
_, err = r.Read(make([]byte, 1))
if err != nil {
t.Fatal(err)
}
line := "[4/4] write: "
awaitOutput(t, stderr, line, strings.Count(stderr(), line)+3)
}
// TestTerminalStoppedShortKeepsCount checks that the terminal display // TestTerminalStoppedShortKeepsCount checks that the terminal display
// of a phase that stops before its total, as an interrupted one does, // of a phase that stops before its total, as an interrupted one does,
// is left showing the count it stopped at rather than a full bar. // is left showing the count it stopped at rather than a full bar.