1 Commits
Author SHA1 Message Date
sneak e80c6541d2 Skip removed paths silently and fix the progress line (closes #133)
check / check (push) Canceled after 0s
A path removed while the scan runs gets no warning and no skipped
count, in the walk or the hash phase. If the first of several hard
links is gone, the next is read. Any other error is still warned about.

On a terminal every display is the library's spinner, then the line. A
phase with an exact total ends the padded line with its percent and a
bar at a fixed column. sfdupes draws that bar: the library shows no
spinner beside its own.

The time remaining is labelled remain, not eta.

Judgement call: an operand removed after the scan starts is passed over
silently too.
Deviation: README's 80-column promise for stalled lines is narrowed to
its example.

Model: opus-5-5
2026-10-26 04:06:11 +00:00
3 changed files with 74 additions and 17 deletions
+7 -9
View File
@@ -633,10 +633,10 @@ Rules for the walk:
is accepted: the database mirrors what the latest scan could actually verify.
The exception is an operand dropped before the database was read, whose record
counts as outside the scanned operands.
- A file or directory removed before the scan reaches it is not an error. The
scan prints no line for it and does not count it as skipped. That holds in the
walk, and in the hash phase for a file removed since the walk. Such a path
gets no record, and the records an earlier scan left at or below it are
- A file or directory removed while the scan runs is not an error. The scan
prints no line for it and does not count it as skipped. A path gone when the
walk reaches it gets no record. A file gone when the hash phase opens it gets
none either. The records an earlier scan left at or below such a path are
deleted.
Concurrency: the walk phase, which also stats files, and the hash phase each use
@@ -938,11 +938,9 @@ Additional requirements:
too wide, and then the end of the time remaining is lost first. That happens
hours into a scan of millions of files, on a terminal narrower than about 75
columns. On an 80-column terminal, the stalled line of the example above
keeps its words. Where the words and the time remaining do not both fit, as
at millions of files and times in hours, the words give way. A terminal made
narrower while a phase runs gets the same: the next draw erases the line
within the new width and fits the line to it, and the display goes on
redrawing in place.
keeps its words. The next draw fits the line to a terminal made narrower
while a phase runs. It erases the old line within the new width, and the
display goes on redrawing in place.
- When stderr is not a terminal (a pipe, a file, `/dev/null`), do not emit ANSI
redraws: print the line the moment each phase starts, every 5 seconds while it
+8 -8
View File
@@ -29,14 +29,14 @@
# Completed Steps
- a file or directory removed before the scan reaches it is not an error, in the
walk or in the hash phase. The scan prints no line for it, does not count it
as skipped, and gives it no record. When the first of several hard links to a
file is gone, the hash phase reads the next one instead. On a terminal every
progress display starts with the spinner, the hash phase's included. A phase
with an exact total ends its line with its percent and a 10-column bar at the
terminal's right edge, which no longer moves as the line's length changes. The
time remaining is labelled `remain`, not `eta` (2026-10-26,
- a file or directory removed while the scan runs is not an error, in the walk
or in the hash phase. The scan prints no line for it and does not count it as
skipped. When the first of several hard links to a file is gone, the hash
phase reads the next one instead. On a terminal every progress display starts
with the spinner, the hash phase's included. A phase with an exact total ends
its line with its percent and a 10-column bar at the terminal's right edge,
which no longer moves as the line's length changes. The time remaining is
labelled `remain`, not `eta` (2026-10-26,
https://git.eeqj.de/sneak/sfdupes/issues/133)
- the `head`, `tail` and `content` columns are BLOBs, each holding the raw 32
+59
View File
@@ -726,6 +726,65 @@ func TestWalkPassesOverRemovedPaths(t *testing.T) {
}
}
// TestScanWarnsAboutUnreadablePaths checks that a permission error is
// still warned about and counted as skipped, unlike a removed path. The
// walk reaches an unreadable directory, and a file in a directory that
// can be read but not searched. Another such file is an operand.
//
//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
func TestScanWarnsAboutUnreadablePaths(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("root ignores file permissions")
}
stderr := captureStderr(t)
dir := t.TempDir()
tree := filepath.Join(dir, "tree")
walked := writeFile(t, tree, "unsearchable/f", []byte("x"))
operand := writeFile(t, dir, "unsearchable/f", []byte("y"))
unreadable := filepath.Join(tree, "unreadable")
err := os.Mkdir(unreadable, 0o750)
if err != nil {
t.Fatal(err)
}
modes := map[string]fs.FileMode{
unreadable: 0,
filepath.Dir(walked): 0o400,
filepath.Dir(operand): 0o400,
}
for p, mode := range modes {
err = os.Chmod(p, mode)
if err != nil {
t.Fatal(err)
}
// Runs before t.TempDir's own cleanup, which must delete the files.
t.Cleanup(func() {
//nolint:gosec // removing the files needs the search bit back
_ = os.Chmod(p, 0o700)
})
}
st := syncTree(t, openTestDB(t), tree, operand)
if st != (scanStats{skipped: 3}) {
t.Errorf("stats = %+v, want 3 skipped and nothing else", st)
}
got := stderr()
for _, warning := range []string{
"walk " + unreadable + ": open " + unreadable + ": permission denied\n",
"stat " + walked + ": lstat " + walked + ": permission denied\n",
"walk " + operand + ": lstat " + operand + ": permission denied\n",
} {
if !strings.Contains(got, warning) {
t.Errorf("stderr = %q, want %q", got, warning)
}
}
}
// syncOneFilesystem scans roots into db with -x and returns the scan
// stats. Every device pruneRoots looks up, and the device each
// operand's walk compares its subdirectories with, comes from devices