Compare commits

..
1 Commits
Author SHA1 Message Date
clawbot 73f9fbb40a Show the walk's path on the line below its display (closes #160)
check / check (push) Canceled after 0s
On a terminal, each draw of the walk shows the path of the last file
it found on the line below, cut short before the last column with
wide characters counted as two columns. The progressbar library's
detail row draws it. The line is erased before a warning and when the
walk ends, so the next phase is drawn on it. Nothing changes when
stderr is not a terminal.

github.com/rivo/uniseg, already in the module graph through the
progressbar library, becomes a direct dependency for the column count.

Model: opus-5-5
2026-11-10 23:49:26 +00:00
9 changed files with 21 additions and 168 deletions

No files matched your search

+6 -13
View File
@@ -665,7 +665,6 @@ Processing:
- Records are grouped by signature, the four values
`(size, head, tail, content)`.
- Every group with two or more paths is a duplicate group.
- Empty files form no group, because removing one frees nothing.
- Within a group, paths are sorted in byte order. The first is `first`, and
every other path is a `dupe`.
- Groups are ordered by size, largest first, then by `first` path. The output is
@@ -714,7 +713,7 @@ The summary line on stderr gives:
- the reclaimable bytes, the sum of `size` over all rows.
```
report: 14 records read, 4 duplicate groups, 7 dupe files, 10.0 KiB reclaimable
report: 14 records read, 5 duplicate groups, 8 dupe files, 10.0 KiB reclaimable
```
### `trees` mode
@@ -742,9 +741,6 @@ Processing:
- `trees` builds the directory tree and computes every directory's digest. It
then groups the directories by digest. Every group with two or more
directories is a duplicate-tree group.
- `trees` leaves out a group whose trees hold 0 bytes, because removing one
frees nothing. An empty file still counts toward the digest of a tree that
holds other files.
- `trees` leaves out a group that a group of its members' parents already
implies. A group is left out when its members' parents are distinct
directories that all have the same digest. The parents' group, or an
@@ -1145,11 +1141,8 @@ On a terminal:
- During `scan`'s `walk`, the line below the display shows the path of the last
file the walk found. Each draw shows the path of that moment, cut short before
the terminal's last column. The path is escaped as in the reports (see "Report
output format"). Each control character left after that is written as `\x` and
its code in two hexadecimal digits, such as `\x1b` for the escape character. A
control character is one that a terminal acts on instead of showing. The line
is erased when the walk ends, and the next phase is drawn on it.
the terminal's last column. The line is erased when the walk ends, and the
next phase is drawn on it.
- No line reaches the terminal's last column. A line too wide for the terminal
first leaves out its rate, the words of a stalled phase, or what the phase is
@@ -1394,7 +1387,7 @@ All of the following, run in this directory, must pass:
head -c 2000 /dev/urandom > "$d/a/unique.bin" # same size, different content
printf 'x' > "$d/tiny1"; printf 'x' > "$d/tiny2" # 1-byte duplicates
printf 'y' > "$d/tiny3" # 1-byte non-duplicate
: > "$d/empty1"; : > "$d/empty2" # empty files
: > "$d/empty1"; : > "$d/empty2" # empty duplicates
# duplicate trees: t1 and t2 are identical; t3 differs by one filename
mkdir -p "$d/t1/sub" "$d/t2/sub" "$d/t3/sub"
head -c 3000 /dev/urandom > "$d/t1/f1"
@@ -1419,8 +1412,8 @@ All of the following, run in this directory, must pass:
is the path that sorts first.
- `t1/f1`, `t2/f1` and `t3/f1` form one group.
- `t1/sub/f2`, `t2/sub/f2` and `t3/sub/f2renamed` form one group.
- `tiny1` and `tiny2` form a pair.
- `unique.bin`, `tiny3`, `empty1` and `empty2` appear nowhere.
- `tiny1` and `tiny2` form a pair, and so do `empty1` and `empty2`.
- `unique.bin` and `tiny3` appear nowhere.
- The groups come in order of size, largest first.
`trees` prints exactly one row, `$d/t1` as `first` and `$d/t2` as `dupe`.
+2 -8
View File
@@ -30,16 +30,10 @@ Newest first. An entry ends with its date and its issue or branch, where those
are known.
- On a terminal, the line below the `walk` display shows the path of the last
file the walk found, at each draw. Its control characters are written out, as
`\t` or `\x1b`, so that the terminal does not act on them. The line is erased
when the walk ends, and the next phase is drawn on it (2026-11-11,
file the walk found, at each draw. The line is erased when the walk ends, and
the next phase is drawn on it (2026-11-10,
https://git.eeqj.de/sneak/sfdupes/issues/160)
- `report` and `trees` leave out empty files. `report` writes no group of size
0, and `trees` writes no group whose trees hold 0 bytes. An empty file still
counts toward a tree's digest (2026-11-11,
https://git.eeqj.de/sneak/sfdupes/issues/178)
- The Go code moved out of the repository root. `cmd/sfdupes/main.go` is the
command, and it calls `Main` in the package `internal/sfdupes`. `main.go`
became `internal/sfdupes/cli.go`. Every other file kept its name. The install
+3 -29
View File
@@ -9,7 +9,6 @@ import (
"strings"
"sync"
"time"
"unicode"
"unicode/utf8"
"github.com/rivo/uniseg"
@@ -246,37 +245,12 @@ func (p *progress) draw() {
if p.path != "" {
// The library draws the path on the line below and returns to
// the phase's line. The path is escaped as warnings are, and its
// other control characters are written out, so that the terminal
// shows them instead of acting on them. It stays clear of the
// last column, so that no terminal wraps it.
shown := escapeControlChars(escapePath(p.path))
_ = p.bar.AddDetail(cutToColumns(shown, width-1))
// the phase's line. The path stays clear of the last column, so
// that no terminal wraps it.
_ = p.bar.AddDetail(cutToColumns(escapePath(p.path), width-1))
}
}
// escapeControlChars returns s with each control character written as
// \x and its code in two hex digits, as \x1b for the escape character.
// The control characters are U+0000 to U+001F, U+007F and U+0080 to
// U+009F, as unicode.IsControl reports them. Every other byte is kept
// as it is.
func escapeControlChars(s string) string {
var b strings.Builder
for rest := s; rest != ""; {
char, size := utf8.DecodeRuneInString(rest)
if unicode.IsControl(char) {
fmt.Fprintf(&b, `\x%02x`, char)
} else {
b.WriteString(rest[:size])
}
rest = rest[size:]
}
return b.String()
}
// cutToColumns returns s cut to at most columns terminal columns, at a
// character boundary. A wide character, as in Chinese, takes two
// columns.
-51
View File
@@ -1166,25 +1166,6 @@ func TestCutToColumns(t *testing.T) {
}
}
func TestEscapeControlChars(t *testing.T) {
t.Parallel()
cases := map[string]string{
"/srv/music": "/srv/music",
"/a\x1b[2Jb": `/a\x1b[2Jb`,
"/a\x00\a\x7fb": `/a\x00\x07\x7fb`,
// U+009B takes two bytes in UTF-8.
"/a\u009b2Jb": `/a\x9b2Jb`,
"/café/AB": "/café/AB",
"/latin1-caf\xe9": "/latin1-caf\xe9",
}
for in, want := range cases {
if got := escapeControlChars(in); got != want {
t.Errorf("escapeControlChars(%q) = %q, want %q", in, got, want)
}
}
}
// pathLines returns the text of each line the library wrote below a
// display's line. It writes one as an erase to the end of the line, a
// carriage return, the text and a newline.
@@ -1243,38 +1224,6 @@ func TestTerminalWalkShowsPathBelow(t *testing.T) {
}
}
// TestTerminalWalkEscapesControlChars checks that a walked path holding
// escape characters, which would clear the screen and set the window
// title, puts none on the line below the walk's display. The line shows
// each control character written out, as \x1b.
//
//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
func TestTerminalWalkEscapesControlChars(t *testing.T) {
stderr := captureStderr(t)
p := startProgress(progressParams{
label: scanPhase("walk"), unit: scanUnit, total: -1,
}, func() int { return 80 }, testRedraw, time.Now)
p.setPath("/srv/\x1b[2J\x1b]0;title\a/beach.jpg")
p.increment()
awaitOutput(t, stderr, "beach.jpg", 1)
p.finish()
lines := pathLines(stderr())
for _, line := range lines {
if strings.Contains(line, "\x1b") {
t.Errorf("line below the display = %q, want no escape "+
"character", line)
}
}
const shown = `/srv/\x1b[2J\x1b]0;title\x07/beach.jpg`
if !slices.Contains(lines, shown) {
t.Errorf("lines below the display = %q, want %q among them",
lines, shown)
}
}
// TestPlainWalkShowsNoPath checks that the walk's display prints no path
// when stderr is not a terminal.
//
+3 -4
View File
@@ -195,11 +195,10 @@ func (w *reportWriter) add(r scanRec) error {
}
// endGroup ends the paths read with w.sig: two or more are a duplicate
// group, held with its paths sorted until writeGroups writes it. Empty
// files form no group, since removing one frees nothing.
// group, held with its paths sorted until writeGroups writes it.
func (w *reportWriter) endGroup() {
if len(w.paths) < minGroupSize || w.sig.size == 0 {
// The paths are dropped, and their slice reused.
if len(w.paths) < minGroupSize {
// A path with no duplicate is dropped, and its slice reused.
w.paths = w.paths[:0]
return
-21
View File
@@ -434,27 +434,6 @@ func TestDupeGroups(t *testing.T) {
}
}
// TestDupeGroupsLeavesOutEmptyFiles checks that empty files form no
// duplicate group, while 1-byte files still do.
func TestDupeGroupsLeavesOutEmptyFiles(t *testing.T) {
t.Parallel()
e, x := sha256.Sum256(nil), testHash("x")
recs := []scanRec{
{size: 0, head: e, tail: e, content: e, path: "/a/.gitkeep"},
{size: 0, head: e, tail: e, content: e, path: "/b/.gitkeep"},
{size: 0, head: e, tail: e, content: e, path: "/c/__init__.py"},
{size: 1, head: x, tail: x, content: x, path: "/tiny1"},
{size: 1, head: x, tail: x, content: x, path: "/tiny2"},
}
groups := dupeGroupsOf(t, recs)
if len(groups) != 1 || groups[0].size != 1 ||
!slices.Equal(groups[0].paths, []string{"/tiny1", "/tiny2"}) {
t.Fatalf("groups = %+v, want only /tiny1 /tiny2 of size 1", groups)
}
}
func TestDupeGroupsContentSeparates(t *testing.T) {
t.Parallel()
+5 -6
View File
@@ -1185,12 +1185,11 @@ func assertSmokeDupeGroups(t *testing.T, dir string, db *sql.DB) {
t.Helper()
groups := dupeGroups(t, db)
if len(groups) != 4 {
t.Fatalf("len(groups) = %d, want 4", len(groups))
if len(groups) != 5 {
t.Fatalf("len(groups) = %d, want 5", len(groups))
}
// The empty files empty1 and empty2 form no group.
wantSizes := []int64{3000, 2000, 100, 1}
wantSizes := []int64{3000, 2000, 100, 1, 0}
for i, g := range groups {
if g.size != wantSizes[i] {
t.Errorf("groups[%d].size = %d, want %d",
@@ -2368,8 +2367,8 @@ func TestRunScanPassesOverDatabase(t *testing.T) {
report := runStdout(t, cmdReport)
if strings.Contains(report, "db.sqlite") ||
strings.Contains(report, hardLink) ||
!strings.Contains(report, filepath.Join(tree, "b")) {
t.Errorf("report = %q, want a and b and none of "+
!strings.Contains(report, filepath.Join(tree, "empty2")) {
t.Errorf("report = %q, want the empty files and none of "+
"the database's files", report)
}
})
+2 -6
View File
@@ -301,9 +301,7 @@ func fileEntry(name string, r scanRec) string {
// collectTreeGroups groups directories by digest, counting each one on
// prog as it is grouped, and returns every maximal group with two or
// more members, each group's members sorted by path, groups ordered by
// tree size descending then by first path ascending. A group whose
// trees hold 0 bytes is left out, since removing one frees nothing. The
// groups it implies hold 0 bytes too, so leaving it out hides nothing.
// tree size descending then by first path ascending.
func collectTreeGroups(allDirs []*treeNode, super *treeNode,
prog *progress,
) [][]*treeNode {
@@ -317,9 +315,7 @@ func collectTreeGroups(allDirs []*treeNode, super *treeNode,
var dupes [][]*treeNode
for _, g := range groups {
// The members share a digest, so they share a total size.
if len(g) < minGroupSize || g[0].totalSize == 0 ||
suppressed(g, super) {
if len(g) < minGroupSize || suppressed(g, super) {
continue
}
-30
View File
@@ -2,7 +2,6 @@ package sfdupes //nolint:testpackage // checks the unexported treeNode results
import (
"bytes"
"crypto/sha256"
"database/sql"
"os"
"slices"
@@ -436,35 +435,6 @@ func TestCollectTreeGroupsSiblings(t *testing.T) {
}
}
// TestCollectTreeGroupsLeavesOutEmptyTrees checks that directories
// holding only empty files form no group, and that an empty file still
// counts toward the digest of a tree that holds other files.
func TestCollectTreeGroupsLeavesOutEmptyTrees(t *testing.T) {
t.Parallel()
h, tl, c, e := testHash("h"), testHash("t"), testHash("c"),
sha256.Sum256(nil)
recs := []scanRec{
{size: 0, head: e, tail: e, content: e, path: "/e/a/.gitkeep"},
{size: 0, head: e, tail: e, content: e, path: "/e/b/.gitkeep"},
{size: 10, head: h, tail: tl, content: c, path: "/p/x1/f"},
{size: 0, head: e, tail: e, content: e, path: "/p/x1/.gitkeep"},
{size: 10, head: h, tail: tl, content: c, path: "/p/x2/f"},
{size: 0, head: e, tail: e, content: e, path: "/p/x2/.gitkeep"},
// Without the empty file, /p/x3 is no copy of /p/x1.
{size: 10, head: h, tail: tl, content: c, path: "/p/x3/f"},
}
super, dirs := treeOf(t, recs)
got := groupPaths(collectTreeGroups(dirs, super, nil))
want := [][]string{{"/p/x1", "/p/x2"}}
if !slices.EqualFunc(got, want, slices.Equal) {
t.Fatalf("groups = %v, want %v", got, want)
}
}
func TestCollectTreeGroupsDifferingParents(t *testing.T) {
t.Parallel()