Skip removed paths silently and fix the progress line (closes #133)

A path removed while the scan runs gets no warning and no skipped
count, in the walk or the hash phase. If the first of several hard
links is gone, the next is read. Any other error is still warned about.

On a terminal every display is the library's spinner, then the line. A
phase with an exact total ends the padded line with its percent and a
bar at a fixed column. sfdupes draws that bar: the library shows no
spinner beside its own.

The time remaining is labelled remain, not eta.

Judgement call: an operand removed after the scan starts is passed over
silently too.
Deviation: README's 80-column promise for stalled lines is narrowed to
its example.

Model: opus-5-5

README.md

@@ -525,11 +525,11 @@ hash phase:
     Files are hashed in **inode order**, which minimizes seeks on spinning
     disks. Paths that are hard links to the same inode are **read once**, and
     all share the one result. A hard-link backup farm therefore costs one read
-    per inode, not per path. The phase total counts actual reads, so progress
+    per inode, not per path. The phase total counts actual reads, so its
-    and ETA are meaningful. Completed records are committed in batched
+    progress and its time remaining are meaningful. Completed records are
-    transactions **while hashing runs**. A scan interrupted after hours
+    committed in batched transactions **while hashing runs**. A scan interrupted
-    therefore keeps everything hashed so far, and the next scan resumes cheaply,
+    after hours therefore keeps everything hashed so far, and the next scan
-    skipping records already written.
+    resumes cheaply, skipping records already written.
 4.  **update** — commit the final partial batch of hashed records. Then delete
     the records the scan did not verify: those of vanished files, and of paths
     that failed to stat or hash.
@@ -625,14 +625,19 @@ Rules for the walk:
     as described in "`scan` mode" above: they are not deleted. That does not
     hold below an operand at or inside it.
 
-- On any per-path error (permission denied, file vanished between passes,
+- On a per-path error, such as permission denied or an unreadable file: print a
-  unreadable): print a one-line warning to stderr, skip the path, and continue.
+  one-line warning to stderr, skip the path, and continue. Per-file errors never
-  Per-file errors never abort the run; the final summary reports how many were
+  abort the run; the final summary reports how many were skipped. As specified
-  skipped. As specified above, a skipped path that has a database record from an
+  above, a skipped path that has a database record from an earlier scan loses
-  earlier scan loses that record. An unreadable directory subtree likewise loses
+  that record. An unreadable directory subtree likewise loses its records. This
-  its records. This is accepted: the database mirrors what the latest scan could
+  is accepted: the database mirrors what the latest scan could actually verify.
-  actually verify. The exception is an operand dropped before the database was
+  The exception is an operand dropped before the database was read, whose record
-  read, whose record counts as outside the scanned operands.
+  counts as outside the scanned operands.
+- A file or directory removed while the scan runs is not an error. The scan
+  prints no line for it and does not count it as skipped. A path gone when the
+  walk reaches it gets no record. A file gone when the hash phase opens it gets
+  none either. The records an earlier scan left at or below such a path are
+  deleted.
 
 Concurrency: the walk phase, which also stats files, and the hash phase each use
 a worker pool of `--workers` workers (default `runtime.NumCPU()`). The walk
@@ -822,9 +827,9 @@ Use the progress-bar library for all progress of `scan`, `report`, `trees` and
 stderr.
 
 `scan`, `report`, `trees` and `info` must always show that they are working and
-not hung. Each phase gets its own display from the moment it starts, naming the
+not hung. Each phase gets its own display from the moment it starts. The display
-phase and its place among the command's phases and showing its progress, its
+names the phase and its place among the command's phases. It shows the phase's
-elapsed time and its estimated time remaining (ETA).
+progress, its elapsed time and its time remaining.
 
 A scan has four phases (`[1/4] load`, `[2/4] walk`, `[3/4] hash`,
 `[4/4] update`):
@@ -834,9 +839,9 @@ A scan has four phases (`[1/4] load`, `[2/4] walk`, `[3/4] hash`,
 - `walk` has no exact total. Its estimate is the number of records the last scan
   left under the operands, known once `load` is done and shown with a `~`. Once
   the walk passes the estimate, or when there are no such records, as on a first
-  scan, the ETA is unknown.
+  scan, the time remaining is unknown.
 - `hash` and `update` have exact totals. Only files that actually need hashing
-  appear in the hash total, so its ETA is meaningful.
+  appear in the hash total, so its time remaining is meaningful.
 
 `trees` has four phases (`[1/4] open`, `[2/4] read`, `[3/4] group`,
 `[4/4] write`, see "`trees` mode"):
@@ -861,8 +866,8 @@ A scan has four phases (`[1/4] load`, `[2/4] walk`, `[3/4] hash`,
 - `open` has no total. While it opens the database, waiting on a busy one
   included, the display says it is opening the database.
 - `count` has no total either, because one query counts everything: the display
-  says it is counting the records until the query returns, and its ETA is
+  says it is counting the records until the query returns, and its time
-  unknown.
+  remaining is unknown.
 
 Each display is one line holding:
 
@@ -875,16 +880,17 @@ Each display is one line holding:
   while nothing completes, and a burst of items after a wait shows as a burst
   rather than as a steady rate. Once no item has completed for 10 seconds, the
   word `stalled` and where the phase waits take the rate's place (see below).
-- elapsed time
+- elapsed time, labelled `elapsed`
-- estimated time remaining, or `unknown`. It comes from the phase's average rate
+- time remaining, labelled `remain`: a duration such as `12m9s`, or `unknown`.
-  since it started counting, so that it does not swing with each burst.
+  It comes from the phase's average rate since it started counting, so that it
+  does not swing with each burst.
 
-Example shape (exact layout is flexible, content is not):
+Example lines, as printed when stderr is not a terminal:
 
 ```
-[3/4] hash: [12345/98765] 12% |████      | 92 files/s elapsed 2:32 eta 17:54
+[3/4] hash: [12345/98765] 12%, 92 files/s, elapsed 2m32s, remain 17m44s
-[2/4] walk: [1234567/~1200000] 103%, 5120 files/s, elapsed 4m1s, eta unknown
+[2/4] walk: [1234567/~1200000] 103%, 5120 files/s, elapsed 4m1s, remain unknown
-[3/4] hash: [12345/45678] 27%, stalled disk/db, elapsed 4m30s, eta 12m9s
+[3/4] hash: [12345/45678] 27%, stalled disk/db, elapsed 4m30s, remain 12m9s
 ```
 
 When no item has completed for 10 seconds, as while one large file is read, a
@@ -900,27 +906,42 @@ for its total already says what it waits for, and says nothing more.
 
 Additional requirements:
 
-- On a terminal, a phase with an exact total (every phase but `scan`'s `walk`,
+- On a terminal, every display starts with the progress-bar library's spinner in
-  the `open` phase of `report` and `trees`, and both phases of `info`, once it
+  its first column, followed by its line. Each display is drawn again several
-  has counted what it will do) is drawn as its line followed by the progress-bar
+  times a second, so its spinner turns and its elapsed time keeps moving while a
-  library's bar, 10 columns wide, which shows the percent in place of the line's
+  phase waits for its next item.
-  own. Any other phase, and a phase whose bar has no room, is drawn as the
+- On a terminal, a phase with an exact total ends its line with its percent and
-  library's spinner followed by its line. Each display is drawn again several
+  a bar 10 columns wide. This percent takes the place of the line's own. The bar
-  times a second, so its elapsed time keeps moving while a phase waits for its
+  stays at the terminal's right edge, so it does not move as the line grows or
-  next item.
+  shrinks. Every phase has an exact total once it has counted what it will do,
+  except these:
+    - `scan`'s `walk`;
+    - the `open` phase of `report` and `trees`;
+    - both phases of `info`.
+
+    On a terminal 100 columns wide:
+
+    ```
+    ⠹ [3/4] hash: [12345/98765], 92 files/s, elapsed 2m32s, remain 17m44s            12% |█         |
+    ```
+
 - On a terminal no line reaches the terminal's last column, however narrow the
-  terminal is. A line too wide for the terminal first loses its bar, which gives
+  terminal is. A line too wide for the terminal first loses its bar. Next it
-  way to the spinner, then its rate, the words of a stalled phase, or what the
+  loses whichever of these it shows: its rate, the words of a stalled phase, or
-  phase is waiting for. The phase and its place, the progress, the elapsed time
+  what the phase is waiting for. The other four parts are kept whole:
-  and the ETA are kept whole. On an 80-column terminal, at counts like those in
+    - the phase and its place;
-  the example above, a stalled line keeps its words whenever the elapsed time
+    - the progress;
-  and the ETA are both in minutes. Where the words and the ETA do not both fit,
+    - the elapsed time;
-  as at millions of files and times in hours, the words give way. Only when
+    - the time remaining.
-  those four alone are too wide, as on a terminal narrower than about 70 columns
+
-  hours into a scan of millions of files, is the line cut short at the
+    The line is cut short at the terminal's edge only when those four alone are
-  terminal's edge, so the end of the ETA is lost first. A terminal made narrower
+    too wide, and then the end of the time remaining is lost first. That happens
-  while a phase runs gets the same: the next draw erases the line within the new
+    hours into a scan of millions of files, on a terminal narrower than about 75
-  width and fits the line to it, and the display goes on redrawing in place.
+    columns. On an 80-column terminal, the stalled line of the example above
+    keeps its words. The next draw fits the line to a terminal made narrower
+    while a phase runs. It erases the old line within the new width, and the
+    display goes on redrawing in place.
+
 - When stderr is not a terminal (a pipe, a file, `/dev/null`), do not emit ANSI
   redraws: print the line the moment each phase starts, every 5 seconds while it
   runs (whether or not an item has completed, and never more often), and once

TODO.md

@@ -29,6 +29,16 @@
 
 # Completed Steps
 
+- a file or directory removed while the scan runs is not an error, in the walk
+  or in the hash phase. The scan prints no line for it and does not count it as
+  skipped. When the first of several hard links to a file is gone, the hash
+  phase reads the next one instead. On a terminal every progress display starts
+  with the spinner, the hash phase's included. A phase with an exact total ends
+  its line with its percent and a 10-column bar at the terminal's right edge,
+  which no longer moves as the line's length changes. The time remaining is
+  labelled `remain`, not `eta` (2026-10-26,
+  https://git.eeqj.de/sneak/sfdupes/issues/133)
+
 - the `head`, `tail` and `content` columns are BLOBs, each holding the raw 32
   bytes of a SHA-256, half the size of the hex text they held. The hashes stay
   raw from the hash phase through the database to `report` and `trees`. Reading

info_test.go

@@ -12,13 +12,13 @@ import (
 // infoOpenLine is the plain line of info's open phase when it opens the
 // database at once; it prints it as the phase starts and as it ends.
 const infoOpenLine = "[1/2] open: opening the database, elapsed 0s, " +
-	"eta unknown"
+	"remain unknown"
 
 // infoCountLine is the plain line of info's count phase when the count
 // takes under a second; it prints it as the phase starts and as it
 // ends.
 const infoCountLine = "[2/2] count: counting the records, elapsed 0s, " +
-	"eta unknown"
+	"remain unknown"
 
 // infoRecs are four records: a duplicate pair, a file of 20 MiB and
 // another small file.

progress.go

@@ -9,6 +9,7 @@ import (
 	"strings"
 	"sync"
 	"time"
+	"unicode/utf8"
 
 	"github.com/schollz/progressbar/v3"
 	"golang.org/x/term"
@@ -34,10 +35,17 @@ const stallTime = 10 * time.Second
 // spinnerType selects the progressbar library's spinner style.
 const spinnerType = 14
 
-// barWidth is the width of the progressbar library's bar between its
+// barWidth is the width of a phase's bar between its two ends, as in
-// two ends, as in the example line in README "Progress".
+// the example line in README "Progress".
 const barWidth = 10
 
+// barCell is one column of the filled part of a bar.
+const barCell = "█"
+
+// barColumns is the width of what percentBar returns: the percent in
+// four columns and its sign, a space, and the bar with its two ends.
+const barColumns = barWidth + 8
+
 // percentScale converts a fraction to a percentage.
 const percentScale = 100
 
@@ -45,17 +53,12 @@ const percentScale = 100
 // read, as the progressbar library takes it.
 const fallbackWidth = 80
 
-// spinnerMargin is the columns of a spinner's line besides its text:
+// spinnerMargin is the columns of a line besides its text: the spinner
-// the spinner and the three spaces the library draws around the text,
+// and the three spaces the library draws around the text, and the
-// and the terminal's last column, which every line stays clear of so
+// terminal's last column, which every line stays clear of so that no
-// that no terminal wraps it.
+// terminal wraps it.
 const spinnerMargin = 5
 
-// barMargin is the columns of a bar's line besides its text: the
-// percent and the bar the library draws after the text, the spaces
-// around them, and the terminal's last column.
-const barMargin = barWidth + 11
-
 // countingRecords is what the display of a phase that reads the records
 // says while it counts them, before it has its total.
 const countingRecords = "counting the records"
@@ -109,13 +112,13 @@ type drawnCount struct {
 }
 
 // progress renders one phase's progress on stderr. On a TTY the
-// progressbar library draws line, fitted to the terminal's width: after
+// progressbar library draws its spinner and then line, fitted to the
-// the library's spinner, or, for a phase with an exact total, before the
+// terminal's width. A phase with an exact total ends the line with its
-// library's bar when the terminal has room for the bar beside it. It is
+// percent and bar, at the terminal's right edge, when the terminal has
-// drawn again every redrawInterval, so its elapsed time keeps moving
+// room for them. It is drawn again every redrawInterval, so its elapsed
-// while the phase waits for its next item. When stderr is not a TTY it
+// time keeps moving while the phase waits for its next item. When
-// emits no ANSI redraws: it prints line as the phase starts, every
+// stderr is not a TTY it emits no ANSI redraws: it prints line as the
-// plainInterval while it runs, and as it ends.
+// phase starts, every plainInterval while it runs, and as it ends.
 //
 // The draws come from a goroutine of their own, so mu guards the fields
 // below it. Every method must be called from the main goroutine. A nil
@@ -131,7 +134,6 @@ type progress struct {
 
 	mu         sync.Mutex
 	bar        *progressbar.ProgressBar // what the library draws; nil until then
-	withBar    bool                     // bar is the library's bar, not its spinner
 	widest     int                      // the widest line bar has drawn, in columns
 	label      string
 	unit       string
@@ -141,7 +143,7 @@ type progress struct {
 	stalledOn  string
 	count      int64
 	start      time.Time // when the phase started, for the elapsed time
-	rateStart  time.Time // when the count started, for the ETA
+	rateStart  time.Time // when the count started, for the time remaining
 	lastItem   time.Time // when the last item was completed, or the count started
 	// drawn holds the count at each draw over the last rateWindow, oldest
 	// first; the rate counts from the first. It keeps the draw before the
@@ -221,49 +223,42 @@ func (p *progress) draw() {
 	}
 
 	width := p.width()
+	room := width - spinnerMargin
+	text := p.line(now, room, true)
 
-	// A phase with an exact total gets the library's bar when its whole
+	// A phase with an exact total ends its text with its percent and bar
-	// line, without the percent the bar shows, fits beside the bar: the
+	// when its whole line, without the percent the bar shows, fits beside
-	// bar gives way before any part of the line. A total of 0 leaves the
+	// them: the bar gives way before any part of the line. The line is
-	// bar nothing to fill.
+	// padded to the room the bar leaves it, so the bar stays at the
-	text, withBar := "", false
+	// terminal's right edge however long the line is. A total of 0 leaves
+	// the bar nothing to fill.
 	if p.total > 0 && !p.estimate {
-		text = p.line(now, math.MaxInt, false)
+		line := p.line(now, math.MaxInt, false)
-		withBar = len(text) <= width-barMargin
+		if len(line) <= room-barColumns {
+			text = fmt.Sprintf("%-*s%s", room-barColumns, line, p.percentBar())
+		}
 	}
 
-	margin := barMargin
+	// Before each draw the library erases its line with as many spaces as
-	if !withBar {
+	// the widest line it has drawn. A display gets a new one when the
-		margin = spinnerMargin
+	// terminal has been made too narrow for that erase, which would wrap
-		text = p.line(now, width-margin, true)
+	// and leave a line behind at every draw.
-	}
+	if p.bar == nil || p.widest >= width {
-
+		p.newBar(width)
-	// The library makes a bar or a spinner, and before each draw erases
-	// its line with as many spaces as the widest line it has drawn. A
-	// display gets a new one when it turns from one to the other, as its
-	// line grows or the terminal is resized, and when the terminal has
-	// been made too narrow for that erase, which would wrap and leave a
-	// line behind at every draw.
-	if p.bar == nil || p.withBar != withBar || p.widest >= width {
-		p.newBar(withBar, width)
-	}
-
-	if withBar {
-		_ = p.bar.Set64(p.count)
 	}
 
 	p.bar.Describe(text)
 
-	// The library drew the text and its margin, less the terminal's last
+	// The library drew the spinner, the text and the spaces around it,
-	// column, which the margin keeps clear.
+	// less the terminal's last column, which spinnerMargin keeps clear. A
-	p.widest = max(p.widest, len(text)+margin-1)
+	// bar's cells take one column each but more than one byte.
+	p.widest = max(p.widest, utf8.RuneCountInString(text)+spinnerMargin-1)
 }
 
-// newBar has the library make what draw draws: a bar for the phase's
+// newBar has the library make what draw draws: its spinner, followed by
-// total when withBar is set, and a spinner when not. Either shows the
+// the text draw gives it and nothing of the library's own. p.mu must be
-// text draw gives it, and nothing of its own besides the spinner, or
+// held.
-// the bar and its percent. p.mu must be held.
+func (p *progress) newBar(width int) {
-func (p *progress) newBar(withBar bool, width int) {
 	if p.bar != nil {
 		// Erase what was drawn before, short of the last column of a
 		// terminal that may have been made narrower since: the library's
@@ -272,29 +267,36 @@ func (p *progress) newBar(withBar bool, width int) {
 		fmt.Fprint(os.Stderr, "\r"+blank+"\r")
 	}
 
-	total := int64(-1) // the library's spinner
+	// A total of -1 gives the library's spinner.
-	if withBar {
+	p.bar = progressbar.NewOptions64(-1,
-		total = p.total
-	}
-
-	p.bar = progressbar.NewOptions64(total,
 		progressbar.OptionSetWriter(os.Stderr),
-		progressbar.OptionSetWidth(barWidth),
 		progressbar.OptionSpinnerType(spinnerType),
 		// The spinner turns at each draw, not on a timer of the
 		// library's own.
 		progressbar.OptionSetSpinnerChangeInterval(0),
-		// The text carries the elapsed time and the ETA.
+		// The text carries the elapsed time and the time remaining.
 		progressbar.OptionSetElapsedTime(false),
 		progressbar.OptionSetPredictTime(false),
 	)
-	p.withBar, p.widest = withBar, 0
+	p.widest = 0
+}
+
+// percentBar returns the percent and the bar that end the text of a
+// phase with an exact total, as "  25% |██        |": the bar's
+// barWidth columns fill in proportion to the items done. A count past
+// the total, as when a scan adds records while trees reads them, shows
+// as 100%. p.mu must be held.
+func (p *progress) percentBar() string {
+	done := min(float64(p.count)/float64(p.total), 1)
+	filled := strings.Repeat(barCell, int(done*barWidth))
+
+	return fmt.Sprintf("%4.0f%% |%-*s|", done*percentScale, barWidth, filled)
 }
 
 // setTotal gives a phase that started without a total its total, once
-// it is known. The rate, the ETA and the time with no item completed
+// it is known. The rate, the time remaining and the time with no item
-// count from then on: the time spent waiting for the total says nothing
+// completed count from then on: the time spent waiting for the total
-// about how fast items go.
+// says nothing about how fast items go.
 func (p *progress) setTotal(total int64) {
 	if p == nil {
 		return
@@ -438,21 +440,21 @@ func (p *progress) finish() {
 // line formats the phase's progress as of now in at most room columns,
 // as
 //
-//	[3/4] hash: [1234/5000] 25%, 300 files/s, elapsed 4s, eta 13s
+//	[3/4] hash: [1234/5000] 25%, 300 files/s, elapsed 4s, remain 13s
 //
 // where "files" is the phase's unit, without the percent when percent
-// is false, as before the library's bar, which shows its own. A phase
+// is false, as before the bar, which shows its own. A phase with no
-// with no total shows the items done instead, or what it waits for. The
+// total shows the items done instead, or what it waits for. The rate is
-// rate is that since the first draw in drawn, and the ETA comes from
+// that since the first draw in drawn, and the time remaining comes from
 // the rate since the count started. Once no item has been completed for
 // stallTime, the rate, which would read 0, gives way to the word
 // "stalled" and where the phase waits, as "stalled disk/db".
 //
-// The phase and its place, the progress, the elapsed time and the ETA
+// The phase and its place, the progress, the elapsed time and the time
-// are kept whole. A line wider than room leaves out the rate, the words
+// remaining are kept whole. A line wider than room leaves out the rate,
-// of a stalled phase, or what the phase waits for; only if it is still
+// the words of a stalled phase, or what the phase waits for; only if it
-// wider is it cut at room. Every part is ASCII, so its length is its
+// is still wider is it cut at room. Every part is ASCII, so its length
-// width in columns. p.mu must be held.
+// is its width in columns. p.mu must be held.
 func (p *progress) line(now time.Time, room int, percent bool) string {
 	elapsed := now.Sub(p.start).Round(time.Second)
 
@@ -462,7 +464,7 @@ func (p *progress) line(now time.Time, room int, percent bool) string {
 
 	if p.total < 0 && p.waitingFor != "" {
 		head, extra = p.label+": ", p.waitingFor+", "
-		tail = fmt.Sprintf("elapsed %s, eta unknown", elapsed)
+		tail = fmt.Sprintf("elapsed %s, remain unknown", elapsed)
 	} else {
 		since := p.drawn[0]
 		rate := perSecond(p.count-since.count, now.Sub(since.at))
@@ -470,7 +472,7 @@ func (p *progress) line(now time.Time, room int, percent bool) string {
 
 		head = p.label + ": " + p.itemsDone(percent) + ", "
 		extra = fmt.Sprintf("%.0f %s/s, ", rate, p.unit)
-		tail = fmt.Sprintf("elapsed %s, eta %s", elapsed, p.eta(average))
+		tail = fmt.Sprintf("elapsed %s, remain %s", elapsed, p.remain(average))
 
 		if now.Sub(p.lastItem) >= stallTime {
 			extra = "stalled " + p.stalledOn + ", "
@@ -526,10 +528,10 @@ func (p *progress) itemsDone(percent bool) string {
 	return fmt.Sprintf("%s %.0f%%", done, pct)
 }
 
-// eta returns the time the phase has left at rate items a second, or
+// remain returns the time the phase has left at rate items a second, or
 // "unknown": with no total, before the first item, and once the count
 // passes an estimate. p.mu must be held.
-func (p *progress) eta(rate float64) string {
+func (p *progress) remain(rate float64) string {
 	switch {
 	case p.total < 0, p.estimate && p.count > p.total:
 		return "unknown"
@@ -539,8 +541,8 @@ func (p *progress) eta(rate float64) string {
 		return "unknown"
 	}
 
-	remaining := time.Duration(
+	left := time.Duration(
 		float64(p.total-p.count) / rate * float64(time.Second))
 
-	return remaining.Round(time.Second).String()
+	return left.Round(time.Second).String()
 }

progress_test.go

@@ -117,24 +117,24 @@ func TestProgressLine(t *testing.T) {
 			label: scanPhase("load"), total: -1,
 			waitingFor: countingRecords,
 		},
-		want: "[1/4] load: counting the records, elapsed 10s, eta unknown",
+		want: "[1/4] load: counting the records, elapsed 10s, remain unknown",
 	}, {
 		p: &progress{
 			label: scanPhase("hash"), unit: scanUnit, total: 100, count: 20,
 		},
-		want: "[3/4] hash: [20/100] 20%, 2 files/s, elapsed 10s, eta 40s",
+		want: "[3/4] hash: [20/100] 20%, 2 files/s, elapsed 10s, remain 40s",
 	}, {
 		p:    &progress{label: scanPhase("hash"), unit: scanUnit, total: 100},
-		want: "[3/4] hash: [0/100] 0%, 0 files/s, elapsed 10s, eta unknown",
+		want: "[3/4] hash: [0/100] 0%, 0 files/s, elapsed 10s, remain unknown",
 	}, {
 		p:    &progress{label: scanPhase("update"), unit: scanUnit, total: 0},
-		want: "[4/4] update: [0/0] 100%, 0 files/s, elapsed 10s, eta 0s",
+		want: "[4/4] update: [0/0] 100%, 0 files/s, elapsed 10s, remain 0s",
 	}, {
 		p: &progress{
 			label: scanPhase("walk"), unit: scanUnit, total: 100,
 			estimate: true, count: 20,
 		},
-		want: "[2/4] walk: [20/~100] 20%, 2 files/s, elapsed 10s, eta 40s",
+		want: "[2/4] walk: [20/~100] 20%, 2 files/s, elapsed 10s, remain 40s",
 	}, {
 		// Past the estimate the walk cannot tell how long it has left.
 		p: &progress{
@@ -142,13 +142,13 @@ func TestProgressLine(t *testing.T) {
 			estimate: true, count: 120,
 		},
 		want: "[2/4] walk: [120/~100] 120%, 12 files/s, elapsed 10s, " +
-			"eta unknown",
+			"remain unknown",
 	}, {
 		// A first scan's walk has no estimate.
 		p: &progress{
 			label: scanPhase("walk"), unit: scanUnit, total: -1, count: 20,
 		},
-		want: "[2/4] walk: 20 files, 2 files/s, elapsed 10s, eta unknown",
+		want: "[2/4] walk: 20 files, 2 files/s, elapsed 10s, remain unknown",
 	}, {
 		// The rate is in what the phase counts.
 		p: &progress{
@@ -156,7 +156,7 @@ func TestProgressLine(t *testing.T) {
 			count: 20,
 		},
 		want: "[3/4] group: [20/100] 20%, 2 directories/s, elapsed 10s, " +
-			"eta 40s",
+			"remain 40s",
 	}}
 
 	for _, tc := range cases {
@@ -203,28 +203,28 @@ func TestLineFitsRoom(t *testing.T) {
 		percent bool
 		want    string
 	}{{
-		p: hash, room: 57, percent: true,
+		p: hash, room: 60, percent: true,
-		want: "[3/4] hash: [20/100] 20%, 2 files/s, elapsed 10s, eta 40s",
+		want: "[3/4] hash: [20/100] 20%, 2 files/s, elapsed 10s, remain 40s",
 	}, {
-		p: hash, room: 56, percent: true,
+		p: hash, room: 59, percent: true,
-		want: "[3/4] hash: [20/100] 20%, elapsed 10s, eta 40s",
+		want: "[3/4] hash: [20/100] 20%, elapsed 10s, remain 40s",
 	}, {
-		p: hash, room: 45, percent: true,
+		p: hash, room: 48, percent: true,
-		want: "[3/4] hash: [20/100] 20%, elapsed 10s, eta 40",
+		want: "[3/4] hash: [20/100] 20%, elapsed 10s, remain 40",
 	}, {
-		// Before the library's bar, which shows the percent.
+		// Before the bar, which shows the percent.
-		p: hash, room: 57, percent: false,
+		p: hash, room: 60, percent: false,
-		want: "[3/4] hash: [20/100], 2 files/s, elapsed 10s, eta 40s",
+		want: "[3/4] hash: [20/100], 2 files/s, elapsed 10s, remain 40s",
 	}, {
-		p: load, room: 40, percent: true,
+		p: load, room: 43, percent: true,
-		want: "[1/4] load: elapsed 10s, eta unknown",
+		want: "[1/4] load: elapsed 10s, remain unknown",
 	}, {
-		p: stalled, room: 65, percent: true,
+		p: stalled, room: 68, percent: true,
 		want: "[3/4] hash: [20/100] 20%, stalled disk/db, elapsed 50s, " +
-			"eta 3m20s",
+			"remain 3m20s",
 	}, {
-		p: stalled, room: 64, percent: true,
+		p: stalled, room: 67, percent: true,
-		want: "[3/4] hash: [20/100] 20%, elapsed 50s, eta 3m20s",
+		want: "[3/4] hash: [20/100] 20%, elapsed 50s, remain 3m20s",
 	}}
 
 	for _, tc := range cases {
@@ -236,10 +236,10 @@ func TestLineFitsRoom(t *testing.T) {
 }
 
 // TestSetTotalStartsRate checks that setTotal starts a phase's rate,
-// its ETA and its time with no record read from when the total becomes
+// its time remaining and its time with no record read from when the
-// known, not from when the phase started waiting for it: after 20
+// total becomes known, not from when the phase started waiting for it:
-// seconds of waiting for the total, the phase has not stalled 5 seconds
+// after 20 seconds of waiting for the total, the phase has not stalled 5
-// after it, and has 10 seconds after it with no record read.
+// seconds after it, and has 10 seconds after it with no record read.
 func TestSetTotalStartsRate(t *testing.T) {
 	t.Parallel()
 
@@ -255,14 +255,14 @@ func TestSetTotalStartsRate(t *testing.T) {
 
 	clock.Add(5 * time.Second)
 
-	want := "[1/4] load: [0/100] 0%, 0 files/s, elapsed 25s, eta unknown"
+	want := "[1/4] load: [0/100] 0%, 0 files/s, elapsed 25s, remain unknown"
 	if got := p.line(clock.Now(), math.MaxInt, true); got != want {
 		t.Errorf("line = %q, want %q", got, want)
 	}
 
 	clock.Add(5 * time.Second)
 
-	want = "[1/4] load: [0/100] 0%, stalled db, elapsed 30s, eta unknown"
+	want = "[1/4] load: [0/100] 0%, stalled db, elapsed 30s, remain unknown"
 	if got := p.line(clock.Now(), math.MaxInt, true); got != want {
 		t.Errorf("line = %q, want %q", got, want)
 	}
@@ -272,7 +272,7 @@ func TestSetTotalStartsRate(t *testing.T) {
 		p.increment()
 	}
 
-	want = "[1/4] load: [20/100] 20%, 2 files/s, elapsed 30s, eta 40s"
+	want = "[1/4] load: [20/100] 20%, 2 files/s, elapsed 30s, remain 40s"
 	if got := p.line(clock.Now(), math.MaxInt, true); got != want {
 		t.Errorf("line = %q, want %q", got, want)
 	}
@@ -316,7 +316,7 @@ func TestPlainProgressPrintsAtStartAndEnd(t *testing.T) {
 
 	lines := strings.Split(strings.TrimSuffix(stderr(), "\n"), "\n")
 
-	want := "[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 0s, eta unknown"
+	want := "[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 0s, remain unknown"
 	if len(lines) != 2 || lines[0] != want ||
 		!strings.HasPrefix(lines[1], "[3/4] hash: [1000/1000] 100%, ") {
 		t.Errorf("stderr lines = %q, want %q, then the line at 1000",
@@ -336,7 +336,7 @@ func TestPlainProgressPrintsWhileWaiting(t *testing.T) {
 	}, nil, testRedraw, time.Now)
 
 	awaitOutput(t, stderr,
-		"[1/4] load: counting the records, elapsed 0s, eta unknown\n", 3)
+		"[1/4] load: counting the records, elapsed 0s, remain unknown\n", 3)
 	p.finish()
 }
 
@@ -366,23 +366,23 @@ func TestPlainProgressStalledProducer(t *testing.T) {
 
 	clock.Add(5 * time.Second)
 	awaitOutput(t, stderr,
-		"[3/4] hash: [20/100] 20%, 4 files/s, elapsed 5s, eta 20s\n", 3)
+		"[3/4] hash: [20/100] 20%, 4 files/s, elapsed 5s, remain 20s\n", 3)
 
 	clock.Add(time.Second)
 	awaitOutput(t, stderr,
-		"[3/4] hash: [20/100] 20%, 0 files/s, elapsed 6s, eta 24s\n", 3)
+		"[3/4] hash: [20/100] 20%, 0 files/s, elapsed 6s, remain 24s\n", 3)
 
 	clock.Add(4 * time.Second)
 	awaitOutput(t, stderr,
-		"[3/4] hash: [20/100] 20%, stalled disk/db, elapsed 10s, eta 40s\n",
+		"[3/4] hash: [20/100] 20%, stalled disk/db, elapsed 10s, remain 40s\n",
 		3)
 }
 
 // TestPlainProgressRateAfterStall draws a phase's plain line by hand, a
 // little over plainInterval after the line before, as a late tick
 // does, through a stall and a burst of files after it: the rate covers
-// the files read since the line before, while the ETA comes from the
+// the files read since the line before, while the time remaining comes
-// phase's average.
+// from the phase's average.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestPlainProgressRateAfterStall(t *testing.T) {
@@ -410,10 +410,10 @@ func TestPlainProgressRateAfterStall(t *testing.T) {
 	p.finish()
 
 	want := []string{
-		"[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 0s, eta unknown",
+		"[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 0s, remain unknown",
-		"[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 5s, eta unknown",
+		"[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 5s, remain unknown",
-		"[3/4] hash: [0/1000] 0%, stalled disk/db, elapsed 10s, eta unknown",
+		"[3/4] hash: [0/1000] 0%, stalled disk/db, elapsed 10s, remain unknown",
-		"[3/4] hash: [100/1000] 10%, 20 files/s, elapsed 15s, eta 2m15s",
+		"[3/4] hash: [100/1000] 10%, 20 files/s, elapsed 15s, remain 2m15s",
 	}
 
 	lines := strings.Split(strings.TrimSuffix(stderr(), "\n"), "\n")
@@ -426,8 +426,8 @@ func TestPlainProgressRateAfterStall(t *testing.T) {
 // redrawInterval, as a terminal does, through a stall of stallTime and
 // a burst of files after it: the rate covers the files read over the
 // last rateWindow, and falls to 0 once the burst is older than that,
-// while the ETA comes from the phase's average. Plain lines show each
+// while the time remaining comes from the phase's average. Plain lines
-// draw.
+// show each draw.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestProgressRateAfterStallRedrawn(t *testing.T) {
@@ -457,8 +457,8 @@ func TestProgressRateAfterStallRedrawn(t *testing.T) {
 	redraw(rateWindow + redrawInterval)
 	p.finish()
 
-	want := "[3/4] hash: [50/1000] 5%, 10 files/s, elapsed 15s, eta 4m45s\n" +
+	want := "[3/4] hash: [50/1000] 5%, 10 files/s, elapsed 15s, remain 4m45s\n" +
-		"[3/4] hash: [50/1000] 5%, 0 files/s, elapsed 15s, eta 4m47s\n"
+		"[3/4] hash: [50/1000] 5%, 0 files/s, elapsed 15s, remain 4m47s\n"
 	if !strings.Contains(stderr(), want) {
 		t.Errorf("stderr = %q, want the lines %q", stderr(), want)
 	}
@@ -480,10 +480,10 @@ func lastFrame(out string) string {
 }
 
 // TestTerminalRedrawsWhileWaiting checks that the terminal display of a
-// phase, spinner or bar, keeps being drawn while the phase waits for its
+// phase, with a bar or without, keeps being drawn while the phase waits
-// next item, with the elapsed time as of now, and shows every item
+// for its next item, with the elapsed time as of now, and shows every
-// completed before the wait. stderr is not a terminal here, so the
+// item completed before the wait. The terminal is 120 columns wide, so
-// displays take it to be fallbackWidth wide.
+// the hash phase's display has room for its bar.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestTerminalRedrawsWhileWaiting(t *testing.T) {
@@ -506,7 +506,8 @@ func TestTerminalRedrawsWhileWaiting(t *testing.T) {
 	for _, tc := range cases {
 		before := len(stderr())
 		since := func() string { return stderr()[before:] }
-		p := startProgress(tc.params, terminalWidth, testRedraw, time.Now)
+		p := startProgress(tc.params, func() int { return 120 }, testRedraw,
+			time.Now)
 
 		for range 50 {
 			p.increment()
@@ -552,13 +553,13 @@ func drawLate(stderr func() string, params progressParams, width int,
 	return stderr()[before:]
 }
 
-// TestTerminalLinesKeepETA checks the line each display shows on an
+// TestTerminalLinesKeepTimeRemaining checks the line each display shows
-// 80-column terminal at a large scan's counts and times, and the line
+// on an 80-column terminal at a large scan's counts and times, and the
-// of trees' read phase at the same counts: the rate is left out, so
+// line of trees' read phase at the same counts: the rate is left out,
-// that the elapsed time and the ETA stay whole.
+// so that the elapsed time and the time remaining stay whole.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
-func TestTerminalLinesKeepETA(t *testing.T) {
+func TestTerminalLinesKeepTimeRemaining(t *testing.T) {
 	stderr := captureStderr(t)
 
 	cases := []struct {
@@ -571,19 +572,19 @@ func TestTerminalLinesKeepETA(t *testing.T) {
 			estimate: true,
 		},
 		want: "[2/4] walk: [1234567/~1200000] 103%, elapsed 2h30m0s, " +
-			"eta unknown",
+			"remain unknown",
 	}, {
 		params: progressParams{
 			label: scanPhase("hash"), unit: scanUnit, total: 9876543,
 		},
 		want: "[3/4] hash: [1234567/9876543] 12%, elapsed 2h30m0s, " +
-			"eta 17h30m0s",
+			"remain 17h30m0s",
 	}, {
 		params: progressParams{
 			label: treesPhase("read"), unit: "records", total: 9876543,
 		},
 		want: "[2/4] read: [1234567/9876543] 12%, elapsed 2h30m0s, " +
-			"eta 17h30m0s",
+			"remain 17h30m0s",
 	}}
 
 	for _, tc := range cases {
@@ -598,9 +599,10 @@ func TestTerminalLinesKeepETA(t *testing.T) {
 // terminal once the phase has stalled, its last item done 42 seconds
 // before. At an ordinary scan's counts and times, 12345 of 45678 items
 // done four and a half minutes in, the line shows where the phase waits
-// whole, beside the phase, progress, elapsed time and ETA. At a large
+// whole, beside the phase, progress, elapsed time and time remaining.
-// scan's, 1234567 of 9876543 two and a half hours in, those four still
+// At a large scan's, 1234567 of 9876543 two and a half hours in, those
-// show whole, as the words give way first where they do not fit.
+// four still show whole, as the words give way first where they do not
+// fit.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestTerminalStalledLines(t *testing.T) {
@@ -652,7 +654,7 @@ func TestTerminalStalledLines(t *testing.T) {
 		shown := drawStalled(params, 45678, 12345,
 			4*time.Minute+30*time.Second)
 		want := params.label + ": [12345/" + approx + "45678] 27%, stalled " +
-			params.stalledOn + ", elapsed 4m30s, eta 12m9s"
+			params.stalledOn + ", elapsed 4m30s, remain 12m9s"
 
 		if !strings.Contains(shown, want) {
 			t.Errorf("terminal shows %q, want %q", shown, want)
@@ -661,7 +663,7 @@ func TestTerminalStalledLines(t *testing.T) {
 		shown = drawStalled(params, 9876543, 1234567, 150*time.Minute)
 		for _, want := range []string{
 			params.label + ": [1234567/" + approx + "9876543] 12%, ",
-			", elapsed 2h30m0s, eta 17h30m0s",
+			", elapsed 2h30m0s, remain 17h30m0s",
 		} {
 			if !strings.Contains(shown, want) {
 				t.Errorf("terminal shows %q, want %q", shown, want)
@@ -706,13 +708,78 @@ func TestTerminalLinesFitWidth(t *testing.T) {
 	}
 }
 
+// TestTerminalLinesStartWithSpinner draws a display with no total and
+// one with a bar, two and a half hours into the phase, on a terminal 120
+// columns wide. Each line must start with the library's spinner in the
+// first column, then a space and the phase's label.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestTerminalLinesStartWithSpinner(t *testing.T) {
+	stderr := captureStderr(t)
+
+	for _, params := range []progressParams{
+		{label: scanPhase("walk"), unit: scanUnit, total: -1},
+		{label: scanPhase("hash"), unit: scanUnit, total: 9876543},
+	} {
+		shown := lastFrame(drawLate(stderr, params, 120, 1234567))
+		spinner, rest, _ := strings.Cut(shown, " ")
+
+		if utf8.RuneCountInString(spinner) != 1 ||
+			!strings.HasPrefix(rest, params.label+": ") {
+			t.Errorf("terminal shows %q, want the spinner, a space and %q",
+				shown, params.label)
+		}
+
+		if params.total > 0 && !strings.Contains(shown, "% |") {
+			t.Errorf("terminal shows %q, want a bar", shown)
+		}
+	}
+}
+
+// TestTerminalBarStaysAtRightEdge draws the hash phase's display two
+// and a half hours in, at two counts whose lines differ in length, on
+// terminals 110 and 120 columns wide. Its bar must start the same number
+// of columns from the terminal's right edge each time: a line that grows
+// or shrinks, as its time remaining does, never moves the bar.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestTerminalBarStaysAtRightEdge(t *testing.T) {
+	stderr := captureStderr(t)
+	params := progressParams{
+		label: scanPhase("hash"), unit: scanUnit, total: 9876543,
+	}
+
+	widths := []int{110, 120}
+	counts := []int64{1234567, 9876543}
+	fromEdge := make([]int, 0, len(widths)*len(counts))
+
+	for _, width := range widths {
+		for _, count := range counts {
+			shown := lastFrame(drawLate(stderr, params, width, count))
+
+			beforeBar, _, found := strings.Cut(shown, "% |")
+			if !found {
+				t.Fatalf("terminal %d wide shows %q, want a bar", width, shown)
+			}
+
+			fromEdge = append(fromEdge,
+				width-utf8.RuneCountInString(beforeBar))
+		}
+	}
+
+	if slices.Min(fromEdge) != slices.Max(fromEdge) {
+		t.Errorf("bar starts %v columns from the right edge, want the same "+
+			"every time", fromEdge)
+	}
+}
+
 // TestTerminalNarrowedRedrawsInPlace draws a large scan's display, two
 // and a half hours into the phase, on a terminal 120 columns wide, then
 // makes the terminal narrower than that line and draws it again a few
 // times. Nothing written after that, the erase before each redraw
 // included, may reach the narrower terminal's last column: it would
-// wrap, and every redraw would leave a line behind. At these widths the
+// wrap, and every redraw would leave a line behind. At the narrower
-// spinner stays a spinner and the bar gives way to the spinner.
+// width the hash phase's bar gives way.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestTerminalNarrowedRedrawsInPlace(t *testing.T) {
@@ -770,10 +837,10 @@ func TestTerminalNarrowedRedrawsInPlace(t *testing.T) {
 	}
 }
 
-// TestProgressWarningsOnOwnLines drives the terminal displays, spinner
+// TestProgressWarningsOnOwnLines drives the terminal displays, with a
-// and bar, through a run of warnings with no items between them, as when
+// bar and without, through a run of warnings with no items between
-// the walk meets many unreadable paths, for many redraws: every warning
+// them, as when the walk meets many unreadable paths, for many redraws:
-// must land on a line of its own, never inside a redraw.
+// every warning must land on a line of its own, never inside a redraw.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestProgressWarningsOnOwnLines(t *testing.T) {
@@ -819,11 +886,12 @@ func TestProgressWarningsOnOwnLines(t *testing.T) {
 	}
 }
 
-// TestProgressWriterRowsOnOwnLines drives the terminal displays,
+// TestProgressWriterRowsOnOwnLines drives the terminal displays, with a
-// spinner and bar, through a run of rows written through the display's
+// bar and without, through a run of rows written through the display's
 // writer to the same terminal, as stdout is in a plain "sfdupes trees",
 // each written out at once, for many redraws: the terminal must show
-// every row whole, in order, on a line of its own.
+// every row whole, in order, on a line of its own. The terminal is 120
+// columns wide, so the write phase's display has room for its bar.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestProgressWriterRowsOnOwnLines(t *testing.T) {
@@ -840,7 +908,8 @@ func TestProgressWriterRowsOnOwnLines(t *testing.T) {
 	var want []string
 
 	for _, params := range cases {
-		p := startProgress(params, terminalWidth, testRedraw, time.Now)
+		p := startProgress(params, func() int { return 120 }, testRedraw,
+			time.Now)
 		out := p.writer(os.Stderr)
 
 		// No pause between rows: one is garbled only if a redraw lands

report_test.go

@@ -187,7 +187,7 @@ func TestReportStdoutFailsWhileReading(t *testing.T) {
 // reportOpenLine is the plain line of report's open phase when it opens
 // the database at once; it prints it as the phase starts and as it ends.
 const reportOpenLine = "[1/2] open: opening the database, elapsed 0s, " +
-	"eta unknown"
+	"remain unknown"
 
 // smokeReport is report's stdout for the smoke-test records.
 const smokeReport = "first\tdupe\tsize\n" +
@@ -225,7 +225,7 @@ func TestRunReportProgressNamesEachPhase(t *testing.T) {
 		"report: database " + path + " (from " + databaseEnv + ")",
 		reportOpenLine,
 		reportOpenLine,
-		"[2/2] read: counting the records, elapsed 0s, eta unknown",
+		"[2/2] read: counting the records, elapsed 0s, remain unknown",
 		"[2/2] read: [6/6] 100%, ",
 		"report: 6 records read, 2 duplicate groups, 4 dupe files, " +
 			"6.1 KiB reclaimable",

scan.go

@@ -418,7 +418,8 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
 // roots and are not deleted as unverified. An operand on a filesystem
 // in rules that is not persistent is dropped the same way, with a line
 // saying so that is not a warning, and is not counted as skipped. An
-// operand that fails lstat here is kept, and the walk warns about it.
+// operand that fails lstat here is kept, and the walk warns about it
+// unless it no longer exists.
 func (s *scanState) walkableRoots(roots []string, rules walkRules) []string {
 	kept := make([]string, 0, len(roots))
 
@@ -629,8 +630,9 @@ func isDatabaseFile(f fileRec, rules walkRules) bool {
 // (an interrupted scan resumes cheaply: the next run skips everything
 // already recorded). A run that fails to hash is warned about and
 // counted as skipped; stale records for its paths, if any, are deleted
-// by the update phase. The total counts actual reads, so the display
+// by the update phase. A run whose file no longer exists gets no record
-// shows a real ETA.
+// and no warning, and is not counted as skipped. The total counts actual
+// reads, so the display's time remaining is meaningful.
 //
 // Returning early — a failed database write, or a cancelled scan — must
 // not strand the pool: the feeder would park forever on a full jobs
@@ -662,6 +664,11 @@ func (s *scanState) hashPhase(ctx context.Context, workers int) error {
 
 		prog.increment()
 
+		// A file removed since the walk is no error: it gets no record.
+		if errors.Is(r.err, fs.ErrNotExist) {
+			continue
+		}
+
 		if r.err != nil {
 			s.st.skipped += len(r.run)
 
@@ -829,10 +836,12 @@ type walkEvent struct {
 // startWalk seeds every root into the shared walk worker pool and
 // returns the event stream: one record per regular file, one warning
 // event per per-path error, and one event per path -x or rules kept the
-// walk from entering or recording. The channel is closed when the walk
+// walk from entering or recording. A path that no longer exists when
-// completes, and also when ctx is cancelled — every goroutine in the
+// the walk reaches it is no error, and gets no event. The channel is
-// pool abandons its blocking send in that case, so the consumer sees a
+// closed when the walk completes, and also when ctx is cancelled —
-// truncated but properly terminated stream instead of a stalled one.
+// every goroutine in the pool abandons its blocking send in that case,
+// so the consumer sees a truncated but properly terminated stream
+// instead of a stalled one.
 func startWalk(ctx context.Context, roots []string, oneFS bool,
 	rules walkRules, workers int,
 ) <-chan walkEvent {
@@ -899,13 +908,19 @@ func operandWarning(root string, fi fs.FileInfo) string {
 // one of the scan's own database files in rules, and a directory
 // operand becomes an initial job. walkableRoots has already dropped
 // every other operand. One that has changed into something else since
-// is warned about and skipped here; it is still a root, so the records
+// is warned about and skipped here, and one removed since is skipped
-// stored beneath it, other than those under another root, are deleted
+// without a warning. Either is still a root, so the records stored
-// as unverified.
+// beneath it, other than those under another root, are deleted as
+// unverified.
 func seedRoot(ctx context.Context, root string, rules walkRules,
 	events chan<- walkEvent,
 ) []dirJob {
 	fi, err := os.Lstat(root)
+	if errors.Is(err, fs.ErrNotExist) {
+		// Removed since the scan started: no error, and no files.
+		return nil
+	}
+
 	if err != nil {
 		sendEvent(ctx, events, walkEvent{
 			warn: fmt.Sprintf("walk %s: %v", root, err),
@@ -1023,11 +1038,17 @@ func dispatchDirs(ctx context.Context, initial []dirJob,
 
 // walkOneDir reads one directory, emitting an event per regular-file
 // entry and a warning event per unreadable one, and returns the
-// subdirectories to descend into.
+// subdirectories to descend into. A directory or entry removed since
+// the walk found it gets no event.
 func walkOneDir(ctx context.Context, job dirJob, oneFS bool,
 	rules walkRules, events chan<- walkEvent,
 ) []dirJob {
 	entries, err := os.ReadDir(job.path)
+	if errors.Is(err, fs.ErrNotExist) {
+		// Removed since its parent was read: no error, and no files.
+		return nil
+	}
+
 	if err != nil {
 		sendEvent(ctx, events, walkEvent{
 			warn: fmt.Sprintf("walk %s: %v", job.path, err),
@@ -1070,16 +1091,22 @@ func walkOneDir(ctx context.Context, job dirJob, oneFS bool,
 
 // emitFile stats one regular directory entry and emits its record.
 // The lstat happens here in the walk worker, while the directory's
-// metadata is still hot; a path that fails to stat (or stops being a
+// metadata is still hot; a path that fails to stat between the
-// regular file) between the directory read and the lstat is warned
+// directory read and the lstat is warned about and skipped, unless it
-// about and skipped. A file on a filesystem in rules that is not
+// no longer exists, and one that has stopped being a regular file is
-// persistent, as through a bind mount of one file, is not recorded: it
+// skipped. A file on a filesystem in rules that is not persistent, as
-// gets an event naming it instead. One of the scan's own database files
+// through a bind mount of one file, is not recorded: it gets an event
-// in rules gets no event at all.
+// naming it instead. One of the scan's own database files in rules gets
+// no event at all.
 func emitFile(ctx context.Context, p string, e fs.DirEntry,
 	rules walkRules, events chan<- walkEvent,
 ) {
 	info, err := e.Info()
+	if errors.Is(err, fs.ErrNotExist) {
+		// Removed since its directory was read: no error.
+		return
+	}
+
 	if err != nil {
 		sendEvent(ctx, events, walkEvent{
 			warn: fmt.Sprintf("stat %s: %v", p, err),
@@ -1131,6 +1158,11 @@ func subdirJob(ctx context.Context, p string, e fs.DirEntry,
 	}
 
 	info, err := e.Info()
+	if errors.Is(err, fs.ErrNotExist) {
+		// Removed since its parent was read: no error, and no files.
+		return dirJob{}, false
+	}
+
 	if err != nil {
 		sendEvent(ctx, events, walkEvent{
 			warn: fmt.Sprintf("walk %s: %v", p, err),
@@ -1195,7 +1227,7 @@ func inodeOfInfo(fi fs.FileInfo) (uint64, uint64) {
 
 // hashResult carries the hashes computed for one inode run (or the
 // error that prevented computing them) from the pool's workers to the
-// hash phase.
+// hash phase. Its run leaves out the paths found removed.
 type hashResult struct {
 	run     []fileRec
 	head    [sha256.Size]byte
@@ -1296,6 +1328,13 @@ func hashWorker(ctx context.Context, jobs <-chan []fileRec,
 
 		head, tail, content, err := hash(run[0].path, run[0].size)
 
+		// A path removed since the walk is dropped from its run. The
+		// next path is a hard link to the same file, and is read instead.
+		for errors.Is(err, fs.ErrNotExist) && len(run) > 1 {
+			run = run[1:]
+			head, tail, content, err = hash(run[0].path, run[0].size)
+		}
+
 		select {
 		case results <- hashResult{
 			run: run, head: head, tail: tail, content: content, err: err,

scan_test.go

@@ -683,6 +683,108 @@ func TestSubdirJobStatError(t *testing.T) {
 	}
 }
 
+// TestWalkPassesOverRemovedPaths removes a file and a directory after
+// their parent was read, as when they are deleted while a scan runs.
+// The walk then reaches them, the directory also as one to read and as
+// an operand. None of that is an error: the walk sends no event.
+func TestWalkPassesOverRemovedPaths(t *testing.T) {
+	t.Parallel()
+
+	dir := t.TempDir()
+	file := writeFile(t, dir, "file", []byte("x"))
+	writeFile(t, dir, "sub/f", []byte("y"))
+	sub := filepath.Join(dir, "sub")
+	fileEntry := dirEntryFor(t, dir, "file")
+	subEntry := dirEntryFor(t, dir, "sub")
+
+	for _, p := range []string{file, sub} {
+		err := os.RemoveAll(p)
+		if err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	ctx := t.Context()
+	events := make(chan walkEvent, 4)
+
+	emitFile(ctx, file, fileEntry, walkRules{}, events)
+
+	_, descend := subdirJob(ctx, sub, subEntry, dirJob{path: dir}, false,
+		walkRules{}, events)
+	subs := walkOneDir(ctx, dirJob{path: sub}, false, walkRules{}, events)
+	jobs := seedRoot(ctx, sub, walkRules{}, events)
+
+	close(events)
+
+	for ev := range events {
+		t.Errorf("event %+v for a removed path, want none", ev)
+	}
+
+	if descend || len(subs) != 0 || len(jobs) != 0 {
+		t.Errorf("descend = %t, subdirectories %+v, jobs %+v, want none",
+			descend, subs, jobs)
+	}
+}
+
+// TestScanWarnsAboutUnreadablePaths checks that a permission error is
+// still warned about and counted as skipped, unlike a removed path. The
+// walk reaches an unreadable directory, and a file in a directory that
+// can be read but not searched. Another such file is an operand.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestScanWarnsAboutUnreadablePaths(t *testing.T) {
+	if os.Geteuid() == 0 {
+		t.Skip("root ignores file permissions")
+	}
+
+	stderr := captureStderr(t)
+	dir := t.TempDir()
+	tree := filepath.Join(dir, "tree")
+	walked := writeFile(t, tree, "unsearchable/f", []byte("x"))
+	operand := writeFile(t, dir, "unsearchable/f", []byte("y"))
+	unreadable := filepath.Join(tree, "unreadable")
+
+	err := os.Mkdir(unreadable, 0o750)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	modes := map[string]fs.FileMode{
+		unreadable:            0,
+		filepath.Dir(walked):  0o400,
+		filepath.Dir(operand): 0o400,
+	}
+
+	for p, mode := range modes {
+		err = os.Chmod(p, mode)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		// Runs before t.TempDir's own cleanup, which must delete the files.
+		t.Cleanup(func() {
+			//nolint:gosec // removing the files needs the search bit back
+			_ = os.Chmod(p, 0o700)
+		})
+	}
+
+	st := syncTree(t, openTestDB(t), tree, operand)
+	if st != (scanStats{skipped: 3}) {
+		t.Errorf("stats = %+v, want 3 skipped and nothing else", st)
+	}
+
+	got := stderr()
+	for _, warning := range []string{
+		"walk " + unreadable + ": open " + unreadable + ": permission denied\n",
+		"stat " + walked + ": lstat " + walked + ": permission denied\n",
+		"walk " + operand + ": lstat " + operand + ": permission denied\n",
+	} {
+		if !strings.Contains(got, warning) {
+			t.Errorf("stderr = %q, want %q", got, warning)
+		}
+	}
+}
+
 // syncOneFilesystem scans roots into db with -x and returns the scan
 // stats. Every device pruneRoots looks up, and the device each
 // operand's walk compares its subdirectories with, comes from devices
@@ -1128,7 +1230,7 @@ func TestScanPipeline(t *testing.T) {
 // database's records. Each phase's first line names it with its place
 // among the four: load counts the records before reading them, and the
 // walk estimates its total from the records under its operand, or says
-// its ETA is unknown when there are none.
+// its time remaining is unknown when there are none.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestScanProgressNamesEachPhase(t *testing.T) {
@@ -1143,14 +1245,14 @@ func TestScanProgressNamesEachPhase(t *testing.T) {
 	syncTree(t, db, filepath.Join(dir, "a"))
 
 	rescan := stderr()[len(first):]
-	load := "[1/4] load: counting the records, elapsed 0s, eta unknown"
+	load := "[1/4] load: counting the records, elapsed 0s, remain unknown"
 	cases := map[string][]string{
 		first: {
-			load, "[2/4] walk: 0 files, 0 files/s, elapsed 0s, eta unknown",
+			load, "[2/4] walk: 0 files, 0 files/s, elapsed 0s, remain unknown",
 			"[3/4] hash: [0/", "[4/4] update: [0/",
 		},
 		rescan: {
-			load, "[2/4] walk: [0/~2] 0%, 0 files/s, elapsed 0s, eta unknown",
+			load, "[2/4] walk: [0/~2] 0%, 0 files/s, elapsed 0s, remain unknown",
 			"[3/4] hash: [0/", "[4/4] update: [0/",
 		},
 	}
@@ -1685,6 +1787,55 @@ func TestHashPhaseSkipsFileReplacedByFIFO(t *testing.T) {
 	}
 }
 
+// TestHashPhasePassesOverRemovedFiles removes two walked files before
+// the hash phase reads them: one with no other path, and the first of
+// two hard links to one file. Neither is an error. The phase prints
+// nothing but its progress, counts nothing as skipped, and records only
+// the hard link still there.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestHashPhasePassesOverRemovedFiles(t *testing.T) {
+	stderr := captureStderr(t)
+	dir := t.TempDir()
+	alone := writeFile(t, dir, "alone.bin", pattern(1, 300))
+	a := writeFile(t, dir, "a.bin", pattern(2, 300))
+	b := filepath.Join(dir, "b.bin")
+
+	err := os.Link(a, b)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	walked, _ := collectWalk(t, []string{dir}, false, 1)
+
+	// a.bin sorts before b.bin, so the run of the two reads it first.
+	for _, p := range []string{alone, a} {
+		err = os.Remove(p)
+		if err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	s := &scanState{db: openTestDB(t), toHash: walked}
+
+	err = s.hashPhase(t.Context(), 1)
+	if err != nil {
+		t.Fatalf("hashPhase = %v, want nil", err)
+	}
+
+	if s.st.skipped != 0 || len(s.batch) != 1 || s.batch[0].path != b {
+		t.Errorf("skipped = %d, records %+v, want none skipped and a "+
+			"record for %s alone", s.st.skipped, s.batch, b)
+	}
+
+	for line := range strings.Lines(stderr()) {
+		if !strings.HasPrefix(line, scanPhase("hash")+": ") {
+			t.Errorf("stderr holds %q, want the hash phase's progress alone",
+				line)
+		}
+	}
+}
+
 // injectedWriteFailure is the message the injected database trigger
 // aborts with, so the test can recognize its own failure in the error
 // the scan reports.

trees_test.go

@@ -13,7 +13,7 @@ import (
 // treesOpenLine is the plain line of trees' open phase when it opens
 // the database at once; it prints it as the phase starts and as it ends.
 const treesOpenLine = "[1/4] open: opening the database, elapsed 0s, " +
-	"eta unknown"
+	"remain unknown"
 
 // smokeTreeRecs mirrors the README smoke-test tree layout: /d/t1 and
 // /d/t2 are identical, /d/t3 differs from them only by one filename.
@@ -236,11 +236,11 @@ func TestRunTreesProgressNamesEachPhase(t *testing.T) {
 		"trees: database " + path + " (from " + databaseEnv + ")",
 		treesOpenLine,
 		treesOpenLine,
-		"[2/4] read: counting the records, elapsed 0s, eta unknown",
+		"[2/4] read: counting the records, elapsed 0s, remain unknown",
 		"[2/4] read: [6/6] 100%, ",
-		"[3/4] group: [0/8] 0%, 0 directories/s, elapsed 0s, eta unknown",
+		"[3/4] group: [0/8] 0%, 0 directories/s, elapsed 0s, remain unknown",
 		"[3/4] group: [8/8] 100%, ",
-		"[4/4] write: [0/1] 0%, 0 groups/s, elapsed 0s, eta unknown",
+		"[4/4] write: [0/1] 0%, 0 groups/s, elapsed 0s, remain unknown",
 		"[4/4] write: [1/1] 100%, ",
 		"trees: 6 records read, 1 duplicate tree groups, 1 dupe trees, ",
 	}