Keep the record count and total size in a totals table (closes #146)

info counted the records and added up their sizes with a query that
reads every record, which takes minutes on a large database. The
schema now has a totals table of one row, kept current by three
triggers on files, in the transaction of each insert, delete and size
update. info, scan's load phase and the read phases of report and
trees read that row instead of counting over files. scan writes files
with the same statements as before. Each of those writes now fires a
trigger and is slower, so scan keeps SQLite's temporary data in
memory, which wins back part of that time.

Model: opus-5-5

README.md

@@ -393,6 +393,20 @@ All four subcommands operate on a single SQLite database file:
         content    BLOB NOT NULL      -- raw 32-byte SHA-256; whole file under 100 MiB, else samples
     ) WITHOUT ROWID;
     CREATE INDEX files_signature ON files (size, head, tail, content);
+    CREATE TABLE totals (
+        records INTEGER NOT NULL,  -- number of records in files
+        size    INTEGER NOT NULL   -- sizes of all records in files added up
+    );
+    INSERT INTO totals (records, size) VALUES (0, 0);
+    CREATE TRIGGER totals_after_insert AFTER INSERT ON files BEGIN
+        UPDATE totals SET records = records + 1, size = size + NEW.size;
+    END;
+    CREATE TRIGGER totals_after_delete AFTER DELETE ON files BEGIN
+        UPDATE totals SET records = records - 1, size = size - OLD.size;
+    END;
+    CREATE TRIGGER totals_after_update AFTER UPDATE OF size ON files BEGIN
+        UPDATE totals SET size = size + NEW.size - OLD.size;
+    END;
     ```
 
     Paths are stored as BLOBs because Unix paths are raw bytes, not guaranteed
@@ -406,6 +420,17 @@ All four subcommands operate on a single SQLite database file:
     `files_signature` index lets `report` read the records in signature order
     without sorting the table.
 
+    `totals` holds one row: the number of records in `files` and the sum of
+    their sizes. Three triggers on `files` keep that row current. Each runs in
+    the transaction of the change to `files` that fires it, so the row and the
+    records always agree. `info` reads both numbers from this row. `scan`,
+    `report` and `trees` read the number of records from it as the total of
+    their progress displays. Counting the records in `files` would read every
+    record, which takes minutes on a database of tens of millions. The triggers
+    make each write to `files` slower, so a first scan of many small files takes
+    about two fifths longer. On large files, where reading them takes most of
+    the time, the difference is small.
+
 ### Duplicate detection
 
 Two files are duplicates when they have the same size and the same three hashes.
@@ -461,17 +486,26 @@ exclude, is not on a persistent filesystem (see "Rules for the walk"), or, with
 `-x`, is on a different filesystem than that operand, because that operand's
 walk does not enter such a directory and so does not reach it.
 
-An operand that is a symlink (never followed, not even as an operand), socket,
-FIFO, or device node, or a directory named `.zfs`, is not scanned. `scan` prints
-a one-line warning naming the path and what it is, counts it as skipped, and
-drops it from the scanned operands before reading the database. Another operand
-beneath it is still scanned. The records stored beneath it are not deleted: they
-are treated like any other record outside the scanned operands. If it lies under
-another operand, they are under that operand instead, and are treated like any
-other record there. This is not an error: a scan whose every operand is dropped
-walks nothing and exits 0. An operand that is not on a persistent filesystem
-(see "Rules for the walk") is dropped the same way. The line `scan` prints for
-it is not a warning, though, and it is not counted as skipped.
+An operand is not scanned when it is:
+
+- a symlink, which `scan` never follows, not even as an operand;
+- a socket, FIFO or device node;
+- at or inside a directory named `.zfs`.
+
+An operand reached through a symlink into a `.zfs` directory is not scanned
+either.
+
+This holds even when the operand lies under another operand. `scan` prints a
+one-line warning naming the path and what it is, counts it as skipped, and drops
+it from the scanned operands before reading the database. An operand beneath a
+symlink operand is still scanned. The records stored beneath the dropped operand
+are not deleted: they are treated like any other record outside the scanned
+operands. If it lies under another operand, they are under that operand instead,
+and are treated like any other record there. This is not an error: a scan whose
+every operand is dropped walks nothing and exits 0. An operand that is not on a
+persistent filesystem (see "Rules for the walk") is dropped the same way. The
+line `scan` prints for it is not a warning, though, and it is not counted as
+skipped.
 
 `scan` synchronizes the database with the filesystem state under the scanned
 operands:
@@ -525,11 +559,11 @@ hash phase:
     Files are hashed in **inode order**, which minimizes seeks on spinning
     disks. Paths that are hard links to the same inode are **read once**, and
     all share the one result. A hard-link backup farm therefore costs one read
-    per inode, not per path. The phase total counts actual reads, so progress
-    and ETA are meaningful. Completed records are committed in batched
-    transactions **while hashing runs**. A scan interrupted after hours
-    therefore keeps everything hashed so far, and the next scan resumes cheaply,
-    skipping records already written.
+    per inode, not per path. The phase total counts actual reads, so its
+    progress and its time remaining are meaningful. Completed records are
+    committed in batched transactions **while hashing runs**. A scan interrupted
+    after hours therefore keeps everything hashed so far, and the next scan
+    resumes cheaply, skipping records already written.
 4.  **update** — commit the final partial batch of hashed records. Then delete
     the records the scan did not verify: those of vanished files, and of paths
     that failed to stat or hash.
@@ -625,14 +659,19 @@ Rules for the walk:
     as described in "`scan` mode" above: they are not deleted. That does not
     hold below an operand at or inside it.
 
-- On any per-path error (permission denied, file vanished between passes,
-  unreadable): print a one-line warning to stderr, skip the path, and continue.
-  Per-file errors never abort the run; the final summary reports how many were
-  skipped. As specified above, a skipped path that has a database record from an
-  earlier scan loses that record. An unreadable directory subtree likewise loses
-  its records. This is accepted: the database mirrors what the latest scan could
-  actually verify. The exception is an operand dropped before the database was
-  read, whose record counts as outside the scanned operands.
+- On a per-path error, such as permission denied or an unreadable file: print a
+  one-line warning to stderr, skip the path, and continue. Per-file errors never
+  abort the run; the final summary reports how many were skipped. As specified
+  above, a skipped path that has a database record from an earlier scan loses
+  that record. An unreadable directory subtree likewise loses its records. This
+  is accepted: the database mirrors what the latest scan could actually verify.
+  The exception is an operand dropped before the database was read, whose record
+  counts as outside the scanned operands.
+- A file or directory removed while the scan runs is not an error. The scan
+  prints no line for it and does not count it as skipped. A path gone when the
+  walk reaches it gets no record. A file gone when the hash phase opens it gets
+  none either. The records an earlier scan left at or below such a path are
+  deleted.
 
 Concurrency: the walk phase, which also stats files, and the hash phase each use
 a worker pool of `--workers` workers (default `runtime.NumCPU()`). The walk
@@ -674,14 +713,18 @@ Processing:
   database state.
 
 `report` runs in two phases, each with its own progress display on stderr (see
-"Progress"): `open` opens the database; `read` counts the records, then reads
-them in signature order, largest size first. That is the order of the
-`files_signature` index read backwards, so SQLite sorts nothing and returns the
-first record at once. The records sharing a signature come together, and so do
-the records of one size, so `report` holds only the duplicate groups of the size
-it is reading, and writes them, ordered by `first` path, once it reaches a
-smaller size. Its memory holds at most the duplicate paths of one size at a
-time, never every record.
+"Progress"):
+
+- `open` opens the database.
+- `read` gets the number of records from `totals`, then reads the records in
+  signature order, largest size first.
+
+Signature order, largest size first, is the `files_signature` index read
+backwards, so SQLite sorts nothing and returns the first record at once. The
+records sharing a signature come together, and so do the records of one size, so
+`report` holds only the duplicate groups of the size it is reading, and writes
+them, ordered by `first` path, once it reaches a smaller size. Its memory holds
+at most the duplicate paths of one size at a time, never every record.
 
 #### Report output format
 
@@ -755,10 +798,13 @@ Processing:
   ascending. Output must be fully deterministic for a given input.
 
 `trees` runs in four phases, each with its own progress display on stderr (see
-"Progress"): `open` opens the database; `read` counts the records, then reads
-them in path order, building the hierarchy and computing the digests as it goes;
-`group` groups the directories by digest and picks the maximal groups; `write`
-writes the groups to stdout.
+"Progress"):
+
+- `open` opens the database.
+- `read` gets the number of records from `totals`, then reads the records in
+  path order, building the hierarchy and computing the digests as it goes.
+- `group` groups the directories by digest and picks the maximal groups.
+- `write` writes the groups to stdout.
 
 #### Trees output format
 
@@ -805,15 +851,19 @@ size of recorded files	150000000000000 (136.4 TiB)
   database is in WAL mode, the newest changes may be in the `-wal` file alone.
 - `schema version` is the database's `PRAGMA user_version`. A database with
   another version than this build's is refused, as by `report`.
-- `records` is the number of records. `size of recorded files` adds up the sizes
-  of all records, in bytes, then in human units.
+- `records` is the number of records. `size of recorded files` is the sum of the
+  sizes of all records, in bytes, then in human units.
 
 `info` runs in two phases, each with its own progress display on stderr (see
-"Progress"): `open` opens the database; `count` reads the schema version and
-counts the records with one query, a single pass over the table, both in one
-read-only transaction, so the numbers agree with each other while a scan is
-committing. `info` writes its lines once the `count` display has ended, so on a
-terminal they come after the displays.
+"Progress"):
+
+- `open` opens the database.
+- `count` reads the schema version and the row of `totals` (see "Database").
+  Both reads run in one read-only transaction, so the numbers agree with each
+  other while a scan is committing.
+
+`info` writes its lines once the `count` display has ended, so on a terminal
+they come after the displays.
 
 ### Progress
 
@@ -822,29 +872,31 @@ Use the progress-bar library for all progress of `scan`, `report`, `trees` and
 stderr.
 
 `scan`, `report`, `trees` and `info` must always show that they are working and
-not hung. Each phase gets its own display from the moment it starts, naming the
-phase and its place among the command's phases and showing its progress, its
-elapsed time and its estimated time remaining (ETA).
+not hung. Each phase gets its own display from the moment it starts. The display
+names the phase and its place among the command's phases. It shows the phase's
+progress, its elapsed time and its time remaining.
 
 A scan has four phases (`[1/4] load`, `[2/4] walk`, `[3/4] hash`,
 `[4/4] update`):
 
-- `load` counts the records first, then reads them against that count. While the
-  count runs, the display says it is counting the records.
+- `load` gets the number of records from `totals` first, then reads the records
+  against that number. Until it has the number, the display says it is counting
+  the records.
 - `walk` has no exact total. Its estimate is the number of records the last scan
-  left under the operands, known once `load` is done and shown with a `~`. Once
-  the walk passes the estimate, or when there are no such records, as on a first
-  scan, the ETA is unknown.
+  left under the operands, known once `load` is done and shown with a `~`. The
+  time remaining is unknown once the walk passes the estimate, and when there
+  are no such records, as on a first scan.
 - `hash` and `update` have exact totals. Only files that actually need hashing
-  appear in the hash total, so its ETA is meaningful.
+  appear in the hash total, so its time remaining is meaningful.
 
 `trees` has four phases (`[1/4] open`, `[2/4] read`, `[3/4] group`,
 `[4/4] write`, see "`trees` mode"):
 
 - `open` has no total. While it opens the database, waiting on a busy one
   included, the display says it is opening the database.
-- `read` counts the records first, then reads them against that count. While the
-  count runs, the display says it is counting the records.
+- `read` gets the number of records from `totals` first, then reads the records
+  against that number. Until it has the number, the display says it is counting
+  the records.
 - `group` has the directories `read` found as its total, and `write` the
   duplicate-tree groups.
 
@@ -852,17 +904,16 @@ A scan has four phases (`[1/4] load`, `[2/4] walk`, `[3/4] hash`,
 
 - `open` has no total. While it opens the database, waiting on a busy one
   included, the display says it is opening the database.
-- `read` counts the records first, then reads them against that count, writing
-  the rows to stdout as it goes. While the count runs, the display says it is
-  counting the records.
+- `read` gets the number of records from `totals` first, then reads the records
+  against that number, writing the rows to stdout as it goes. Until it has the
+  number, the display says it is counting the records.
 
 `info` has two phases (`[1/2] open`, `[2/2] count`, see "`info` mode"):
 
 - `open` has no total. While it opens the database, waiting on a busy one
   included, the display says it is opening the database.
-- `count` has no total either, because one query counts everything: the display
-  says it is counting the records until the query returns, and its ETA is
-  unknown.
+- `count` has no total either, because it reads one row: the display says it is
+  counting the records until it has the row, and its time remaining is unknown.
 
 Each display is one line holding:
 
@@ -875,16 +926,17 @@ Each display is one line holding:
   while nothing completes, and a burst of items after a wait shows as a burst
   rather than as a steady rate. Once no item has completed for 10 seconds, the
   word `stalled` and where the phase waits take the rate's place (see below).
-- elapsed time
-- estimated time remaining, or `unknown`. It comes from the phase's average rate
-  since it started counting, so that it does not swing with each burst.
+- elapsed time, labelled `elapsed`
+- time remaining, labelled `remain`: a duration such as `12m9s`, or `unknown`.
+  It comes from the phase's average rate since it started counting, so that it
+  does not swing with each burst.
 
-Example shape (exact layout is flexible, content is not):
+Example lines, as printed when stderr is not a terminal:
 
 ```
-[3/4] hash: [12345/98765] 12% |████      | 92 files/s elapsed 2:32 eta 17:54
-[2/4] walk: [1234567/~1200000] 103%, 5120 files/s, elapsed 4m1s, eta unknown
-[3/4] hash: [12345/45678] 27%, stalled disk/db, elapsed 4m30s, eta 12m9s
+[3/4] hash: [12345/98765] 12%, 92 files/s, elapsed 2m32s, remain 17m44s
+[2/4] walk: [1234567/~1200000] 103%, 5120 files/s, elapsed 4m1s, remain unknown
+[3/4] hash: [12345/45678] 27%, stalled disk/db, elapsed 4m30s, remain 12m9s
 ```
 
 When no item has completed for 10 seconds, as while one large file is read, a
@@ -900,27 +952,43 @@ for its total already says what it waits for, and says nothing more.
 
 Additional requirements:
 
-- On a terminal, a phase with an exact total (every phase but `scan`'s `walk`,
-  the `open` phase of `report` and `trees`, and both phases of `info`, once it
-  has counted what it will do) is drawn as its line followed by the progress-bar
-  library's bar, 10 columns wide, which shows the percent in place of the line's
-  own. Any other phase, and a phase whose bar has no room, is drawn as the
-  library's spinner followed by its line. Each display is drawn again several
-  times a second, so its elapsed time keeps moving while a phase waits for its
-  next item.
+- On a terminal, every display starts with the progress-bar library's spinner in
+  its first column, followed by its line. Each display is drawn again several
+  times a second, so its spinner turns and its elapsed time keeps moving while a
+  phase waits for its next item.
+- On a terminal, a phase with an exact total ends its line with its percent and
+  a bar 10 columns wide. This percent takes the place of the line's own. The bar
+  stays at the terminal's right edge, so it does not move as the line grows or
+  shrinks. A phase with nothing to do shows no bar. Every phase has an exact
+  total once it has counted what it will do, except these:
+    - `scan`'s `walk`;
+    - the `open` phase of `report` and `trees`;
+    - both phases of `info`.
+
+    On a terminal 100 columns wide:
+
+    ```
+    ⠹ [3/4] hash: [12345/98765], 92 files/s, elapsed 2m32s, remain 17m44s            12% |█         |
+    ```
+
 - On a terminal no line reaches the terminal's last column, however narrow the
-  terminal is. A line too wide for the terminal first loses its bar, which gives
-  way to the spinner, then its rate, the words of a stalled phase, or what the
-  phase is waiting for. The phase and its place, the progress, the elapsed time
-  and the ETA are kept whole. On an 80-column terminal, at counts like those in
-  the example above, a stalled line keeps its words whenever the elapsed time
-  and the ETA are both in minutes. Where the words and the ETA do not both fit,
-  as at millions of files and times in hours, the words give way. Only when
-  those four alone are too wide, as on a terminal narrower than about 70 columns
-  hours into a scan of millions of files, is the line cut short at the
-  terminal's edge, so the end of the ETA is lost first. A terminal made narrower
-  while a phase runs gets the same: the next draw erases the line within the new
-  width and fits the line to it, and the display goes on redrawing in place.
+  terminal is. A line too wide for the terminal first loses whichever of these
+  it shows: its rate, the words of a stalled phase, or what the phase is waiting
+  for. The other four parts are kept whole:
+    - the phase and its place;
+    - the progress;
+    - the elapsed time;
+    - the time remaining.
+
+    The line loses its bar only when those four and the bar do not fit. It is
+    cut short at the terminal's edge only when those four alone are too wide,
+    and then the end of the time remaining is lost first. That happens hours
+    into a scan of millions of files, on a terminal narrower than about 75
+    columns. On a terminal 100 columns wide, the stalled line of the example
+    above keeps its words and its bar. The next draw fits the line to a terminal
+    made narrower while a phase runs. It erases the old line within the new
+    width, and the display goes on redrawing in place.
+
 - When stderr is not a terminal (a pipe, a file, `/dev/null`), do not emit ANSI
   redraws: print the line the moment each phase starts, every 5 seconds while it
   runs (whether or not an item has completed, and never more often), and once

TODO.md

@@ -29,6 +29,29 @@
 
 # Completed Steps
 
+- the database keeps the number of records and the sum of their sizes in the one
+  row of a `totals` table, which three triggers on `files` keep current. `info`
+  reads that row, and so do `scan`'s `load` phase and the `read` phases of
+  `report` and `trees` for their totals; no count reads every record any more.
+  The triggers make each write to `files` slower. `scan` keeps SQLite's
+  temporary data in memory, which wins back part of that time (2026-10-27,
+  https://git.eeqj.de/sneak/sfdupes/issues/146)
+
+- an operand at or inside a directory named `.zfs` is never scanned, whether
+  given alone or under another operand. `scan` warns about it, counts it as
+  skipped and does not walk it (2026-10-26,
+  https://git.eeqj.de/sneak/sfdupes/issues/142)
+
+- a file or directory removed while the scan runs is not an error, in the walk
+  or in the hash phase. The scan prints no line for it and does not count it as
+  skipped. When the first of several hard links to a file is gone, the hash
+  phase reads the next one instead. On a terminal every progress display starts
+  with the spinner, the hash phase's included. A phase with an exact total ends
+  its line with its percent and a 10-column bar at the terminal's right edge,
+  which no longer moves as the line's length changes. A line too wide for the
+  terminal now loses its rate before its bar. The time remaining is labelled
+  `remain`, not `eta` (2026-10-26, https://git.eeqj.de/sneak/sfdupes/issues/133)
+
 - the `head`, `tail` and `content` columns are BLOBs, each holding the raw 32
   bytes of a SHA-256, half the size of the hex text they held. The hashes stay
   raw from the hash phase through the database to `report` and `trees`. Reading

db.go

@@ -69,6 +69,49 @@ const createIndexSQL = `
 CREATE INDEX files_signature ON files (size, head, tail, content)
 `
 
+// createTotalsSQL creates the totals table, which holds one row: the
+// number of records and the sizes of all records added up. Counting
+// them over files would read every record. The three triggers below
+// keep the row current, in the transaction of each change to files.
+const createTotalsSQL = `
+CREATE TABLE totals (
+    records INTEGER NOT NULL,
+    size    INTEGER NOT NULL
+)
+`
+
+// insertTotalsSQL adds the one row of totals, for an empty files table.
+const insertTotalsSQL = `
+INSERT INTO totals (records, size) VALUES (0, 0)
+`
+
+// totalsAfterInsertSQL counts each inserted record into totals.
+const totalsAfterInsertSQL = `
+CREATE TRIGGER totals_after_insert AFTER INSERT ON files BEGIN
+    UPDATE totals SET records = records + 1, size = size + NEW.size;
+END
+`
+
+// totalsAfterDeleteSQL takes each deleted record out of totals.
+const totalsAfterDeleteSQL = `
+CREATE TRIGGER totals_after_delete AFTER DELETE ON files BEGIN
+    UPDATE totals SET records = records - 1, size = size - OLD.size;
+END
+`
+
+// totalsAfterUpdateSQL replaces a record's old size in totals with its
+// new one. scan's upsert of an existing record is such an update.
+const totalsAfterUpdateSQL = `
+CREATE TRIGGER totals_after_update AFTER UPDATE OF size ON files BEGIN
+    UPDATE totals SET size = size + NEW.size - OLD.size;
+END
+`
+
+// recordCountSQL reads the number of records from totals.
+const recordCountSQL = `
+SELECT records FROM totals
+`
+
 // upsertSQL inserts one file record, replacing any existing record for
 // the same path.
 const upsertSQL = `
@@ -150,9 +193,14 @@ func createDatabaseDirectory(path string) error {
 // scanParams are the connection parameters for scan: read-write, with
 // WAL journaling and a busy timeout, so a report can run while a cron
 // scan is in progress. closeScanDatabase leaves WAL mode again.
+//
+// Each write to files fires a trigger, so SQLite keeps a statement
+// journal for it. temp_store(MEMORY) keeps that journal in memory,
+// which costs far less than a temporary file.
 const scanParams = "_pragma=busy_timeout(10000)" +
 	"&_pragma=journal_mode(WAL)" +
-	"&_pragma=synchronous(NORMAL)"
+	"&_pragma=synchronous(NORMAL)" +
+	"&_pragma=temp_store(MEMORY)"
 
 // reportParams are the connection parameters for report, trees and
 // info: read-only, with the same busy timeout. They set no journal
@@ -426,14 +474,14 @@ func createSchema(ctx context.Context, db *sql.DB) error {
 
 	defer func() { _ = tx.Rollback() }()
 
-	_, err = tx.ExecContext(ctx, createTableSQL)
-	if err != nil {
-		return fmt.Errorf("create schema: %w", err)
-	}
-
-	_, err = tx.ExecContext(ctx, createIndexSQL)
-	if err != nil {
-		return fmt.Errorf("create schema: %w", err)
+	for _, stmt := range []string{
+		createTableSQL, createIndexSQL, createTotalsSQL, insertTotalsSQL,
+		totalsAfterInsertSQL, totalsAfterDeleteSQL, totalsAfterUpdateSQL,
+	} {
+		_, err = tx.ExecContext(ctx, stmt)
+		if err != nil {
+			return fmt.Errorf("create schema: %w", err)
+		}
 	}
 
 	_, err = tx.ExecContext(ctx,
@@ -480,7 +528,7 @@ func loadFileRows(ctx context.Context, db *sql.DB, total func(n int64),
 
 	var n int64
 
-	err = tx.QueryRowContext(ctx, "SELECT COUNT(*) FROM files").Scan(&n)
+	err = tx.QueryRowContext(ctx, recordCountSQL).Scan(&n)
 	if err != nil {
 		return fmt.Errorf("count records: %w", err)
 	}
@@ -556,7 +604,7 @@ func loadSignatureRows(ctx context.Context, db *sql.DB,
 
 	var n int64
 
-	err = tx.QueryRowContext(ctx, "SELECT COUNT(*) FROM files").Scan(&n)
+	err = tx.QueryRowContext(ctx, recordCountSQL).Scan(&n)
 	if err != nil {
 		return fmt.Errorf("count records: %w", err)
 	}
@@ -622,7 +670,7 @@ func (r *scanRec) setHashes(head, tail, content []byte) error {
 func countRecords(ctx context.Context, db *sql.DB) (int64, error) {
 	var n int64
 
-	err := db.QueryRowContext(ctx, "SELECT COUNT(*) FROM files").Scan(&n)
+	err := db.QueryRowContext(ctx, recordCountSQL).Scan(&n)
 	if err != nil {
 		return 0, fmt.Errorf("count records: %w", err)
 	}
@@ -630,12 +678,10 @@ func countRecords(ctx context.Context, db *sql.DB) (int64, error) {
 	return n, nil
 }
 
-// databaseCountsSQL counts the records and adds up their sizes, in one
-// pass over the table. A SUM over no rows is NULL, so on an empty table
-// the SUM falls back to 0.
+// databaseCountsSQL reads the number of records and the sizes of all
+// records added up from totals.
 const databaseCountsSQL = `
-SELECT COUNT(*), COALESCE(SUM(size), 0)
-FROM files
+SELECT records, size FROM totals
 `
 
 // databaseCounts is what info reads from the database: its schema

db_test.go

@@ -535,17 +535,12 @@ func TestLoadRowsRefuseWrongSizeHash(t *testing.T) {
 	}
 }
 
-// TestSignatureRowsNeedNoSort checks that SQLite reads signatureRowsSQL
-// from the files_signature index alone, with nothing to sort: the
-// first record comes at once, so report's read phase counts each
-// record as it comes rather than waiting on SQLite for all of them.
-func TestSignatureRowsNeedNoSort(t *testing.T) {
-	t.Parallel()
+// queryPlan returns the detail of each line of SQLite's EXPLAIN QUERY
+// PLAN for query.
+func queryPlan(t *testing.T, db *sql.DB, query string) []string {
+	t.Helper()
 
-	db := openTestDB(t)
-
-	rows, err := db.QueryContext(t.Context(),
-		"EXPLAIN QUERY PLAN "+signatureRowsSQL)
+	rows, err := db.QueryContext(t.Context(), "EXPLAIN QUERY PLAN "+query)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -573,12 +568,164 @@ func TestSignatureRowsNeedNoSort(t *testing.T) {
 		t.Fatal(err)
 	}
 
+	return plan
+}
+
+// TestSignatureRowsNeedNoSort checks that SQLite reads signatureRowsSQL
+// from the files_signature index alone, with nothing to sort: the
+// first record comes at once, so report's read phase counts each
+// record as it comes rather than waiting on SQLite for all of them.
+func TestSignatureRowsNeedNoSort(t *testing.T) {
+	t.Parallel()
+
+	db := openTestDB(t)
+
+	plan := queryPlan(t, db, signatureRowsSQL)
+
 	want := []string{"SCAN files USING COVERING INDEX files_signature"}
 	if !slices.Equal(plan, want) {
 		t.Errorf("query plan = %q, want %q", plan, want)
 	}
 }
 
+// TestCountsReadOnlyTotals checks that SQLite answers the queries for
+// the number of records and their total size from the one row of
+// totals. A plan that reads files or its index reads every record.
+func TestCountsReadOnlyTotals(t *testing.T) {
+	t.Parallel()
+
+	db := openTestDB(t)
+
+	for _, query := range []string{recordCountSQL, databaseCountsSQL} {
+		plan := queryPlan(t, db, query)
+
+		want := []string{"SCAN totals"}
+		if !slices.Equal(plan, want) {
+			t.Errorf("query plan of %q = %q, want %q", query, plan, want)
+		}
+	}
+}
+
+// TestCountsComeFromTotals sets the row of totals to numbers that files
+// does not hold, then checks that scan's load phase, the read phases of
+// trees and report, and info each report the numbers from totals.
+func TestCountsComeFromTotals(t *testing.T) {
+	t.Parallel()
+
+	// files is empty, so a count over files gives 0 records of 0 bytes.
+	const records, size = 7, 9000
+
+	db := openTestDB(t)
+
+	_, err := db.ExecContext(t.Context(),
+		"UPDATE totals SET records = ?, size = ?", records, size)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	scanTotal, err := countRecords(t.Context(), db)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	var treesTotal, reportTotal int64
+
+	err = loadFileRows(t.Context(), db,
+		func(n int64) { treesTotal = n }, func(scanRec) {})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = loadSignatureRows(t.Context(), db,
+		func(n int64) { reportTotal = n },
+		func(scanRec) error { return nil })
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	info, err := readDatabaseCounts(t.Context(), db)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	for name, got := range map[string]int64{
+		"scan's load":   scanTotal,
+		"trees' read":   treesTotal,
+		"report's read": reportTotal,
+		"info":          info.records,
+	} {
+		if got != records {
+			t.Errorf("%s: %d records, want %d from totals",
+				name, got, records)
+		}
+	}
+
+	if info.size != size {
+		t.Errorf("info: size %d, want %d from totals", info.size, size)
+	}
+}
+
+// assertTotalsMatchFiles fails unless the row of totals holds the
+// number of records and the sizes of all records added up, as counted
+// over files.
+func assertTotalsMatchFiles(t *testing.T, db *sql.DB) {
+	t.Helper()
+
+	got, err := readDatabaseCounts(t.Context(), db)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	var records, size int64
+
+	err = db.QueryRowContext(t.Context(),
+		"SELECT COUNT(*), COALESCE(SUM(size), 0) FROM files").
+		Scan(&records, &size)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if got.records != records || got.size != size {
+		t.Errorf("totals = %d records of %d bytes, files holds %d of %d",
+			got.records, got.size, records, size)
+	}
+}
+
+// TestTotalsMatchFiles scans the smoke tree, rescans it after two files
+// change size, and rescans it after two files are removed. After each
+// scan the row of totals must agree with files.
+func TestTotalsMatchFiles(t *testing.T) {
+	t.Parallel()
+
+	dir := buildSmokeTree(t)
+	db := openTestDB(t)
+
+	syncTree(t, db, dir)
+	assertTotalsMatchFiles(t, db)
+
+	writeFile(t, dir, "a/one.bin", pattern(10, 2500))
+	writeFile(t, dir, "t1/f1", pattern(30, 1000))
+
+	if st := syncTree(t, db, dir); st.updated != 2 {
+		t.Fatalf("rescan stats = %+v, want 2 updated", st)
+	}
+
+	assertTotalsMatchFiles(t, db)
+
+	for _, name := range []string{"a/unique.bin", "t2/sub/f2"} {
+		err := os.Remove(filepath.Join(dir, name))
+		if err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	if st := syncTree(t, db, dir); st.removed != 2 {
+		t.Fatalf("rescan stats = %+v, want 2 removed", st)
+	}
+
+	assertTotalsMatchFiles(t, db)
+}
+
 func TestApplyChangesRoundTrip(t *testing.T) {
 	t.Parallel()
 

info.go

@@ -26,11 +26,11 @@ func infoPhase(name string) string {
 // it. Any database problem, including a missing database, is fatal.
 //
 // The work runs in two phases, each under a progress display of its
-// own on stderr: open (open the database) and count (count the records,
-// with one query). Each display ends before the next phase starts, and
-// before an error is returned to be printed. The details are written
-// once the last display has ended, so no display is drawn while they
-// are written.
+// own on stderr: open (open the database) and count (read the number of
+// records and their total size from the totals table). Each display
+// ends before the next phase starts, and before an error is returned
+// to be printed. The details are written once the last display has
+// ended, so no display is drawn while they are written.
 func runInfo(ctx context.Context, stdout io.Writer) error {
 	dbPath, source, err := databasePath()
 	if err != nil {
@@ -56,7 +56,7 @@ func runInfo(ctx context.Context, stdout io.Writer) error {
 
 	defer func() { _ = db.Close() }()
 
-	// One query counts everything, so the phase never has a total.
+	// One row holds both counts, so the phase never has a total.
 	prog = newProgress(progressParams{
 		label: infoPhase("count"), total: -1,
 		waitingFor: countingRecords,

info_test.go

@@ -12,13 +12,13 @@ import (
 // infoOpenLine is the plain line of info's open phase when it opens the
 // database at once; it prints it as the phase starts and as it ends.
 const infoOpenLine = "[1/2] open: opening the database, elapsed 0s, " +
-	"eta unknown"
+	"remain unknown"
 
 // infoCountLine is the plain line of info's count phase when the count
 // takes under a second; it prints it as the phase starts and as it
 // ends.
 const infoCountLine = "[2/2] count: counting the records, elapsed 0s, " +
-	"eta unknown"
+	"remain unknown"
 
 // infoRecs are four records: a duplicate pair, a file of 20 MiB and
 // another small file.

main_test.go

@@ -180,9 +180,9 @@ func capture(t *testing.T, std **os.File) func() string {
 }
 
 // brokenDatabase writes a database that opens cleanly and passes the
-// schema-version check but has no files table, so the first query
-// fails with the database already open: a fatal error on a path that
-// owns an open database. It closes the database the way scan does.
+// schema-version check but has no tables, so the first query fails
+// with the database already open: a fatal error on a path that owns an
+// open database. It closes the database the way scan does.
 func brokenDatabase(t *testing.T) string {
 	t.Helper()
 
@@ -265,8 +265,9 @@ func TestRunFatalAfterOpenClosesDatabase(t *testing.T) {
 			assertFatalOutput(t, stderr.String(), stdout())
 
 			// Proof that the failure happened after the open: only a
-			// query against the opened database can report this.
-			if !strings.Contains(stderr.String(), "no such table: files") {
+			// query against the opened database can report this. Each
+			// subcommand's first query reads totals.
+			if !strings.Contains(stderr.String(), "no such table: totals") {
 				t.Errorf("stderr = %q, want the failure to come from a "+
 					"query on the open database", stderr.String())
 			}
@@ -928,24 +929,21 @@ func TestRunScanWalksOperandUnderSymlinkOperand(t *testing.T) {
 }
 
 func TestRunScanSkipsZFSOperand(t *testing.T) {
-	path := testDBPath(t)
-	t.Setenv(databaseEnv, path)
-
 	zfs := filepath.Join(t.TempDir(), ".zfs")
-	snapshot := filepath.Join(zfs, "snapshot", "hourly")
-	f := writeFile(t, snapshot, "f", pattern(1, 10))
+	f := writeFile(t, zfs, "snapshot/hourly/f", pattern(1, 10))
 
-	// An operand beneath a .zfs directory is walked, because it is not
-	// itself named .zfs.
-	scanOK(t, snapshot)
+	// No scan records a file inside a .zfs directory, so the record
+	// beneath the operand is written straight into the database.
+	path := seedDatabase(t, []scanRec{{size: 10, path: f}})
+	t.Setenv(databaseEnv, path)
 
 	assertOperandSkipped(t, path, zfs, ".zfs directory", f)
 }
 
 // assertOperandSkipped scans operand alone and checks that it is skipped
 // as kind: a warning naming it, one skip in the summary, exit 0, and the
-// record for kept, which an earlier scan stored beneath operand, still
-// in the database at dbPath.
+// record for kept, stored beneath operand before this scan, still in the
+// database at dbPath.
 func assertOperandSkipped(t *testing.T, dbPath, operand, kind,
 	kept string,
 ) {
@@ -974,6 +972,121 @@ func assertOperandSkipped(t *testing.T, dbPath, operand, kind,
 	recordByPath(t, dbRecords(t, db), kept)
 }
 
+// TestRunScanSkipsOperandInsideZFS scans a directory inside a .zfs
+// directory, first alone and then under another operand. Each time it
+// is warned about and counted as skipped, and nothing in it is walked.
+// The other operand is scanned as usual.
+func TestRunScanSkipsOperandInsideZFS(t *testing.T) {
+	path := testDBPath(t)
+	t.Setenv(databaseEnv, path)
+
+	dir := t.TempDir()
+	hourly := filepath.Join(dir, ".zfs", "snapshot", "hourly")
+	writeFile(t, hourly, "f", pattern(1, 10))
+	f := writeFile(t, dir, "f", pattern(2, 10))
+
+	warning := "walk " + hourly + ": skipping operand inside a .zfs directory\n"
+
+	for _, tc := range []struct {
+		operands []string
+		summary  string
+	}{
+		{
+			[]string{hourly},
+			"scan: 0 files seen (0 added, 0 updated, 0 removed, " +
+				"0 unchanged), 1 skipped\n",
+		},
+		{
+			[]string{dir, hourly},
+			"scan: 1 files seen (1 added, 0 updated, 0 removed, " +
+				"0 unchanged), 1 skipped\n",
+		},
+	} {
+		stderr := scanOK(t, tc.operands...)
+		if !strings.Contains(stderr, warning) ||
+			!strings.Contains(stderr, tc.summary) {
+			t.Errorf("scan %q: stderr = %q, want %q and %q", tc.operands,
+				stderr, warning, tc.summary)
+		}
+	}
+
+	db, err := openDB(path, reportParams)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	t.Cleanup(func() { _ = db.Close() })
+
+	if got := recordPaths(dbRecords(t, db)); !slices.Equal(got, []string{f}) {
+		t.Errorf("records = %q, want %q", got, []string{f})
+	}
+}
+
+// TestRunScanSkipsOperandThroughSymlinkIntoZFS scans snaps/daily, where
+// snaps is a symlink to pool/.zfs/snapshot. The operand is warned about
+// by the path given, counted as skipped, and not walked.
+func TestRunScanSkipsOperandThroughSymlinkIntoZFS(t *testing.T) {
+	t.Setenv(databaseEnv, testDBPath(t))
+
+	dir := t.TempDir()
+	writeFile(t, dir, "pool/.zfs/snapshot/daily/f", pattern(1, 10))
+
+	snaps := filepath.Join(dir, "snaps")
+
+	err := os.Symlink(filepath.Join(dir, "pool", ".zfs", "snapshot"), snaps)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	daily := filepath.Join(snaps, "daily")
+	stderr := scanOK(t, daily)
+
+	warning := "walk " + daily + ": skipping operand inside a .zfs directory\n"
+	summary := "scan: 0 files seen (0 added, 0 updated, 0 removed, " +
+		"0 unchanged), 1 skipped\n"
+
+	if !strings.Contains(stderr, warning) ||
+		!strings.Contains(stderr, summary) {
+		t.Errorf("stderr = %q, want %q and %q", stderr, warning, summary)
+	}
+}
+
+// TestRunScanWalksOperandThroughSymlink scans data/daily, where data is
+// a symlink to a directory outside any .zfs directory. The operand is
+// scanned as usual, and its file is recorded by the path given.
+func TestRunScanWalksOperandThroughSymlink(t *testing.T) {
+	path := testDBPath(t)
+	t.Setenv(databaseEnv, path)
+
+	dir := t.TempDir()
+	writeFile(t, dir, "pool/data/daily/f", pattern(1, 10))
+
+	data := filepath.Join(dir, "data")
+
+	err := os.Symlink(filepath.Join(dir, "pool", "data"), data)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	daily := filepath.Join(data, "daily")
+	stderr := scanOK(t, daily)
+
+	summary := "scan: 1 files seen (1 added, 0 updated, 0 removed, " +
+		"0 unchanged), 0 skipped\n"
+	if !strings.Contains(stderr, summary) {
+		t.Errorf("stderr = %q, want %q", stderr, summary)
+	}
+
+	db, err := openDB(path, reportParams)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	t.Cleanup(func() { _ = db.Close() })
+
+	recordByPath(t, dbRecords(t, db), filepath.Join(daily, "f"))
+}
+
 func TestRunReportSucceeds(t *testing.T) {
 	path := testDBPath(t)
 	t.Setenv(databaseEnv, path)

progress.go

@@ -9,6 +9,7 @@ import (
 	"strings"
 	"sync"
 	"time"
+	"unicode/utf8"
 
 	"github.com/schollz/progressbar/v3"
 	"golang.org/x/term"
@@ -34,10 +35,17 @@ const stallTime = 10 * time.Second
 // spinnerType selects the progressbar library's spinner style.
 const spinnerType = 14
 
-// barWidth is the width of the progressbar library's bar between its
-// two ends, as in the example line in README "Progress".
+// barWidth is the width of a phase's bar between its two ends, as in
+// the example line in README "Progress".
 const barWidth = 10
 
+// barCell is one column of the filled part of a bar.
+const barCell = "█"
+
+// barColumns is the width of what percentBar returns: the percent in
+// four columns and its sign, a space, and the bar with its two ends.
+const barColumns = barWidth + 8
+
 // percentScale converts a fraction to a percentage.
 const percentScale = 100
 
@@ -45,19 +53,14 @@ const percentScale = 100
 // read, as the progressbar library takes it.
 const fallbackWidth = 80
 
-// spinnerMargin is the columns of a spinner's line besides its text:
-// the spinner and the three spaces the library draws around the text,
-// and the terminal's last column, which every line stays clear of so
-// that no terminal wraps it.
+// spinnerMargin is the columns of a line besides its text: the spinner
+// and the three spaces the library draws around the text, and the
+// terminal's last column, which every line stays clear of so that no
+// terminal wraps it.
 const spinnerMargin = 5
 
-// barMargin is the columns of a bar's line besides its text: the
-// percent and the bar the library draws after the text, the spaces
-// around them, and the terminal's last column.
-const barMargin = barWidth + 11
-
 // countingRecords is what the display of a phase that reads the records
-// says while it counts them, before it has its total.
+// says until it has the number of records, its total.
 const countingRecords = "counting the records"
 
 // openPhase is the name of the phase in which report, trees and info
@@ -109,13 +112,13 @@ type drawnCount struct {
 }
 
 // progress renders one phase's progress on stderr. On a TTY the
-// progressbar library draws line, fitted to the terminal's width: after
-// the library's spinner, or, for a phase with an exact total, before the
-// library's bar when the terminal has room for the bar beside it. It is
-// drawn again every redrawInterval, so its elapsed time keeps moving
-// while the phase waits for its next item. When stderr is not a TTY it
-// emits no ANSI redraws: it prints line as the phase starts, every
-// plainInterval while it runs, and as it ends.
+// progressbar library draws its spinner and then line, fitted to the
+// terminal's width. A phase with an exact total ends the line with its
+// percent and bar, at the terminal's right edge, when the terminal has
+// room for them. It is drawn again every redrawInterval, so its elapsed
+// time keeps moving while the phase waits for its next item. When
+// stderr is not a TTY it emits no ANSI redraws: it prints line as the
+// phase starts, every plainInterval while it runs, and as it ends.
 //
 // The draws come from a goroutine of their own, so mu guards the fields
 // below it. Every method must be called from the main goroutine. A nil
@@ -131,7 +134,6 @@ type progress struct {
 
 	mu         sync.Mutex
 	bar        *progressbar.ProgressBar // what the library draws; nil until then
-	withBar    bool                     // bar is the library's bar, not its spinner
 	widest     int                      // the widest line bar has drawn, in columns
 	label      string
 	unit       string
@@ -141,7 +143,7 @@ type progress struct {
 	stalledOn  string
 	count      int64
 	start      time.Time // when the phase started, for the elapsed time
-	rateStart  time.Time // when the count started, for the ETA
+	rateStart  time.Time // when the count started, for the time remaining
 	lastItem   time.Time // when the last item was completed, or the count started
 	// drawn holds the count at each draw over the last rateWindow, oldest
 	// first; the rate counts from the first. It keeps the draw before the
@@ -215,55 +217,34 @@ func (p *progress) draw() {
 	}
 
 	if p.width == nil {
-		fmt.Fprintln(os.Stderr, p.line(now, math.MaxInt, true))
+		fmt.Fprintln(os.Stderr, p.line(now, math.MaxInt, false))
 
 		return
 	}
 
 	width := p.width()
+	text := p.line(now, width-spinnerMargin, true)
 
-	// A phase with an exact total gets the library's bar when its whole
-	// line, without the percent the bar shows, fits beside the bar: the
-	// bar gives way before any part of the line. A total of 0 leaves the
-	// bar nothing to fill.
-	text, withBar := "", false
-	if p.total > 0 && !p.estimate {
-		text = p.line(now, math.MaxInt, false)
-		withBar = len(text) <= width-barMargin
-	}
-
-	margin := barMargin
-	if !withBar {
-		margin = spinnerMargin
-		text = p.line(now, width-margin, true)
-	}
-
-	// The library makes a bar or a spinner, and before each draw erases
-	// its line with as many spaces as the widest line it has drawn. A
-	// display gets a new one when it turns from one to the other, as its
-	// line grows or the terminal is resized, and when the terminal has
-	// been made too narrow for that erase, which would wrap and leave a
-	// line behind at every draw.
-	if p.bar == nil || p.withBar != withBar || p.widest >= width {
-		p.newBar(withBar, width)
-	}
-
-	if withBar {
-		_ = p.bar.Set64(p.count)
+	// Before each draw the library erases its line with as many spaces as
+	// the widest line it has drawn. A display gets a new one when the
+	// terminal has been made too narrow for that erase, which would wrap
+	// and leave a line behind at every draw.
+	if p.bar == nil || p.widest >= width {
+		p.newBar(width)
 	}
 
 	p.bar.Describe(text)
 
-	// The library drew the text and its margin, less the terminal's last
-	// column, which the margin keeps clear.
-	p.widest = max(p.widest, len(text)+margin-1)
+	// The library drew the spinner, the text and the spaces around it,
+	// less the terminal's last column, which spinnerMargin keeps clear. A
+	// bar's cells take one column each but more than one byte.
+	p.widest = max(p.widest, utf8.RuneCountInString(text)+spinnerMargin-1)
 }
 
-// newBar has the library make what draw draws: a bar for the phase's
-// total when withBar is set, and a spinner when not. Either shows the
-// text draw gives it, and nothing of its own besides the spinner, or
-// the bar and its percent. p.mu must be held.
-func (p *progress) newBar(withBar bool, width int) {
+// newBar has the library make what draw draws: its spinner, followed by
+// the text draw gives it and nothing of the library's own. p.mu must be
+// held.
+func (p *progress) newBar(width int) {
 	if p.bar != nil {
 		// Erase what was drawn before, short of the last column of a
 		// terminal that may have been made narrower since: the library's
@@ -272,29 +253,36 @@ func (p *progress) newBar(withBar bool, width int) {
 		fmt.Fprint(os.Stderr, "\r"+blank+"\r")
 	}
 
-	total := int64(-1) // the library's spinner
-	if withBar {
-		total = p.total
-	}
-
-	p.bar = progressbar.NewOptions64(total,
+	// A total of -1 gives the library's spinner.
+	p.bar = progressbar.NewOptions64(-1,
 		progressbar.OptionSetWriter(os.Stderr),
-		progressbar.OptionSetWidth(barWidth),
 		progressbar.OptionSpinnerType(spinnerType),
 		// The spinner turns at each draw, not on a timer of the
 		// library's own.
 		progressbar.OptionSetSpinnerChangeInterval(0),
-		// The text carries the elapsed time and the ETA.
+		// The text carries the elapsed time and the time remaining.
 		progressbar.OptionSetElapsedTime(false),
 		progressbar.OptionSetPredictTime(false),
 	)
-	p.withBar, p.widest = withBar, 0
+	p.widest = 0
+}
+
+// percentBar returns the percent and the bar that end the text of a
+// phase with an exact total, as "  25% |██        |": the bar's
+// barWidth columns fill in proportion to the items done. A count past
+// the total, as when a scan adds records while trees reads them, shows
+// as 100%. p.mu must be held.
+func (p *progress) percentBar() string {
+	done := min(float64(p.count)/float64(p.total), 1)
+	filled := strings.Repeat(barCell, int(done*barWidth))
+
+	return fmt.Sprintf("%4.0f%% |%-*s|", done*percentScale, barWidth, filled)
 }
 
 // setTotal gives a phase that started without a total its total, once
-// it is known. The rate, the ETA and the time with no item completed
-// count from then on: the time spent waiting for the total says nothing
-// about how fast items go.
+// it is known. The rate, the time remaining and the time with no item
+// completed count from then on: the time spent waiting for the total
+// says nothing about how fast items go.
 func (p *progress) setTotal(total int64) {
 	if p == nil {
 		return
@@ -438,22 +426,26 @@ func (p *progress) finish() {
 // line formats the phase's progress as of now in at most room columns,
 // as
 //
-//	[3/4] hash: [1234/5000] 25%, 300 files/s, elapsed 4s, eta 13s
+//	[3/4] hash: [1234/5000] 25%, 300 files/s, elapsed 4s, remain 13s
 //
-// where "files" is the phase's unit, without the percent when percent
-// is false, as before the library's bar, which shows its own. A phase
-// with no total shows the items done instead, or what it waits for. The
-// rate is that since the first draw in drawn, and the ETA comes from
-// the rate since the count started. Once no item has been completed for
-// stallTime, the rate, which would read 0, gives way to the word
-// "stalled" and where the phase waits, as "stalled disk/db".
+// where "files" is the phase's unit. A phase with no total shows the
+// items done instead, or what it waits for. The rate is that since the
+// first draw in drawn, and the time remaining comes from the rate since
+// the count started. Once no item has been completed for stallTime, the
+// rate, which would read 0, gives way to the word "stalled" and where
+// the phase waits, as "stalled disk/db".
 //
-// The phase and its place, the progress, the elapsed time and the ETA
-// are kept whole. A line wider than room leaves out the rate, the words
-// of a stalled phase, or what the phase waits for; only if it is still
-// wider is it cut at room. Every part is ASCII, so its length is its
-// width in columns. p.mu must be held.
-func (p *progress) line(now time.Time, room int, percent bool) string {
+// When bar is true, as on a terminal, a phase with an exact total ends
+// the line with its percent and bar, in place of the percent after its
+// count. A total of 0 leaves the bar nothing to fill.
+//
+// The phase and its place, the progress, the elapsed time and the time
+// remaining are kept whole. A line wider than room first leaves out the
+// rate, the words of a stalled phase, or what the phase waits for. Next
+// it leaves out the bar. Only if it is still wider is it cut at room.
+// Every part but the bar is ASCII, so its length is its width in
+// columns. p.mu must be held.
+func (p *progress) line(now time.Time, room int, bar bool) string {
 	elapsed := now.Sub(p.start).Round(time.Second)
 
 	// head and tail hold the parts kept whole; extra, between them, is
@@ -462,21 +454,41 @@ func (p *progress) line(now time.Time, room int, percent bool) string {
 
 	if p.total < 0 && p.waitingFor != "" {
 		head, extra = p.label+": ", p.waitingFor+", "
-		tail = fmt.Sprintf("elapsed %s, eta unknown", elapsed)
+		tail = fmt.Sprintf("elapsed %s, remain unknown", elapsed)
 	} else {
 		since := p.drawn[0]
 		rate := perSecond(p.count-since.count, now.Sub(since.at))
 		average := perSecond(p.count, now.Sub(p.rateStart))
 
-		head = p.label + ": " + p.itemsDone(percent) + ", "
+		head = p.label + ": " + p.itemsDone(true) + ", "
 		extra = fmt.Sprintf("%.0f %s/s, ", rate, p.unit)
-		tail = fmt.Sprintf("elapsed %s, eta %s", elapsed, p.eta(average))
+		tail = fmt.Sprintf("elapsed %s, remain %s", elapsed, p.remain(average))
 
 		if now.Sub(p.lastItem) >= stallTime {
 			extra = "stalled " + p.stalledOn + ", "
 		}
 	}
 
+	if bar && p.total > 0 && !p.estimate {
+		// Beside the bar, which shows the percent, the count shows none.
+		// The line is padded to the room the bar leaves it, so the bar
+		// stays at room's right edge however long the line is.
+		beside := room - barColumns
+		barHead := p.label + ": " + p.itemsDone(false) + ", "
+
+		switch {
+		case len(barHead+extra+tail) <= beside:
+			return fmt.Sprintf("%-*s%s", beside, barHead+extra+tail,
+				p.percentBar())
+		case len(barHead+tail) <= beside:
+			return fmt.Sprintf("%-*s%s", beside, barHead+tail, p.percentBar())
+		}
+
+		// The bar gives way. The rate, or the words of a stalled phase,
+		// gave way before it and stay out.
+		extra = ""
+	}
+
 	full, kept := head+extra+tail, head+tail
 
 	switch {
@@ -526,10 +538,10 @@ func (p *progress) itemsDone(percent bool) string {
 	return fmt.Sprintf("%s %.0f%%", done, pct)
 }
 
-// eta returns the time the phase has left at rate items a second, or
+// remain returns the time the phase has left at rate items a second, or
 // "unknown": with no total, before the first item, and once the count
 // passes an estimate. p.mu must be held.
-func (p *progress) eta(rate float64) string {
+func (p *progress) remain(rate float64) string {
 	switch {
 	case p.total < 0, p.estimate && p.count > p.total:
 		return "unknown"
@@ -539,8 +551,8 @@ func (p *progress) eta(rate float64) string {
 		return "unknown"
 	}
 
-	remaining := time.Duration(
+	left := time.Duration(
 		float64(p.total-p.count) / rate * float64(time.Second))
 
-	return remaining.Round(time.Second).String()
+	return left.Round(time.Second).String()
 }

progress_test.go

@@ -117,24 +117,24 @@ func TestProgressLine(t *testing.T) {
 			label: scanPhase("load"), total: -1,
 			waitingFor: countingRecords,
 		},
-		want: "[1/4] load: counting the records, elapsed 10s, eta unknown",
+		want: "[1/4] load: counting the records, elapsed 10s, remain unknown",
 	}, {
 		p: &progress{
 			label: scanPhase("hash"), unit: scanUnit, total: 100, count: 20,
 		},
-		want: "[3/4] hash: [20/100] 20%, 2 files/s, elapsed 10s, eta 40s",
+		want: "[3/4] hash: [20/100] 20%, 2 files/s, elapsed 10s, remain 40s",
 	}, {
 		p:    &progress{label: scanPhase("hash"), unit: scanUnit, total: 100},
-		want: "[3/4] hash: [0/100] 0%, 0 files/s, elapsed 10s, eta unknown",
+		want: "[3/4] hash: [0/100] 0%, 0 files/s, elapsed 10s, remain unknown",
 	}, {
 		p:    &progress{label: scanPhase("update"), unit: scanUnit, total: 0},
-		want: "[4/4] update: [0/0] 100%, 0 files/s, elapsed 10s, eta 0s",
+		want: "[4/4] update: [0/0] 100%, 0 files/s, elapsed 10s, remain 0s",
 	}, {
 		p: &progress{
 			label: scanPhase("walk"), unit: scanUnit, total: 100,
 			estimate: true, count: 20,
 		},
-		want: "[2/4] walk: [20/~100] 20%, 2 files/s, elapsed 10s, eta 40s",
+		want: "[2/4] walk: [20/~100] 20%, 2 files/s, elapsed 10s, remain 40s",
 	}, {
 		// Past the estimate the walk cannot tell how long it has left.
 		p: &progress{
@@ -142,13 +142,13 @@ func TestProgressLine(t *testing.T) {
 			estimate: true, count: 120,
 		},
 		want: "[2/4] walk: [120/~100] 120%, 12 files/s, elapsed 10s, " +
-			"eta unknown",
+			"remain unknown",
 	}, {
 		// A first scan's walk has no estimate.
 		p: &progress{
 			label: scanPhase("walk"), unit: scanUnit, total: -1, count: 20,
 		},
-		want: "[2/4] walk: 20 files, 2 files/s, elapsed 10s, eta unknown",
+		want: "[2/4] walk: 20 files, 2 files/s, elapsed 10s, remain unknown",
 	}, {
 		// The rate is in what the phase counts.
 		p: &progress{
@@ -156,14 +156,14 @@ func TestProgressLine(t *testing.T) {
 			count: 20,
 		},
 		want: "[3/4] group: [20/100] 20%, 2 directories/s, elapsed 10s, " +
-			"eta 40s",
+			"remain 40s",
 	}}
 
 	for _, tc := range cases {
 		tc.p.start, tc.p.rateStart, tc.p.lastItem = ago, ago, now
 		tc.p.drawn = []drawnCount{{at: ago}}
 
-		if got := tc.p.line(now, math.MaxInt, true); got != tc.want {
+		if got := tc.p.line(now, math.MaxInt, false); got != tc.want {
 			t.Errorf("line = %q, want %q", got, tc.want)
 		}
 	}
@@ -198,48 +198,49 @@ func TestLineFitsRoom(t *testing.T) {
 	}
 
 	cases := []struct {
-		p       *progress
-		room    int
-		percent bool
-		want    string
+		p    *progress
+		room int
+		bar  bool
+		want string
 	}{{
-		p: hash, room: 57, percent: true,
-		want: "[3/4] hash: [20/100] 20%, 2 files/s, elapsed 10s, eta 40s",
+		p: hash, room: 60,
+		want: "[3/4] hash: [20/100] 20%, 2 files/s, elapsed 10s, remain 40s",
 	}, {
-		p: hash, room: 56, percent: true,
-		want: "[3/4] hash: [20/100] 20%, elapsed 10s, eta 40s",
+		p: hash, room: 59,
+		want: "[3/4] hash: [20/100] 20%, elapsed 10s, remain 40s",
 	}, {
-		p: hash, room: 45, percent: true,
-		want: "[3/4] hash: [20/100] 20%, elapsed 10s, eta 40",
+		p: hash, room: 48,
+		want: "[3/4] hash: [20/100] 20%, elapsed 10s, remain 40",
 	}, {
-		// Before the library's bar, which shows the percent.
-		p: hash, room: 57, percent: false,
-		want: "[3/4] hash: [20/100], 2 files/s, elapsed 10s, eta 40s",
+		// The bar shows the percent.
+		p: hash, room: 74, bar: true,
+		want: "[3/4] hash: [20/100], 2 files/s, elapsed 10s, remain 40s" +
+			"  20% |██        |",
 	}, {
-		p: load, room: 40, percent: true,
-		want: "[1/4] load: elapsed 10s, eta unknown",
+		p: load, room: 43,
+		want: "[1/4] load: elapsed 10s, remain unknown",
 	}, {
-		p: stalled, room: 65, percent: true,
+		p: stalled, room: 68,
 		want: "[3/4] hash: [20/100] 20%, stalled disk/db, elapsed 50s, " +
-			"eta 3m20s",
+			"remain 3m20s",
 	}, {
-		p: stalled, room: 64, percent: true,
-		want: "[3/4] hash: [20/100] 20%, elapsed 50s, eta 3m20s",
+		p: stalled, room: 67,
+		want: "[3/4] hash: [20/100] 20%, elapsed 50s, remain 3m20s",
 	}}
 
 	for _, tc := range cases {
-		if got := tc.p.line(now, tc.room, tc.percent); got != tc.want {
-			t.Errorf("line(%d, %t) = %q, want %q",
-				tc.room, tc.percent, got, tc.want)
+		if got := tc.p.line(now, tc.room, tc.bar); got != tc.want {
+			t.Errorf("line(%d, %t) = %q, want %q", tc.room, tc.bar, got,
+				tc.want)
 		}
 	}
 }
 
 // TestSetTotalStartsRate checks that setTotal starts a phase's rate,
-// its ETA and its time with no record read from when the total becomes
-// known, not from when the phase started waiting for it: after 20
-// seconds of waiting for the total, the phase has not stalled 5 seconds
-// after it, and has 10 seconds after it with no record read.
+// its time remaining and its time with no record read from when the
+// total becomes known, not from when the phase started waiting for it:
+// after 20 seconds of waiting for the total, the phase has not stalled 5
+// seconds after it, and has 10 seconds after it with no record read.
 func TestSetTotalStartsRate(t *testing.T) {
 	t.Parallel()
 
@@ -255,15 +256,15 @@ func TestSetTotalStartsRate(t *testing.T) {
 
 	clock.Add(5 * time.Second)
 
-	want := "[1/4] load: [0/100] 0%, 0 files/s, elapsed 25s, eta unknown"
-	if got := p.line(clock.Now(), math.MaxInt, true); got != want {
+	want := "[1/4] load: [0/100] 0%, 0 files/s, elapsed 25s, remain unknown"
+	if got := p.line(clock.Now(), math.MaxInt, false); got != want {
 		t.Errorf("line = %q, want %q", got, want)
 	}
 
 	clock.Add(5 * time.Second)
 
-	want = "[1/4] load: [0/100] 0%, stalled db, elapsed 30s, eta unknown"
-	if got := p.line(clock.Now(), math.MaxInt, true); got != want {
+	want = "[1/4] load: [0/100] 0%, stalled db, elapsed 30s, remain unknown"
+	if got := p.line(clock.Now(), math.MaxInt, false); got != want {
 		t.Errorf("line = %q, want %q", got, want)
 	}
 
@@ -272,8 +273,8 @@ func TestSetTotalStartsRate(t *testing.T) {
 		p.increment()
 	}
 
-	want = "[1/4] load: [20/100] 20%, 2 files/s, elapsed 30s, eta 40s"
-	if got := p.line(clock.Now(), math.MaxInt, true); got != want {
+	want = "[1/4] load: [20/100] 20%, 2 files/s, elapsed 30s, remain 40s"
+	if got := p.line(clock.Now(), math.MaxInt, false); got != want {
 		t.Errorf("line = %q, want %q", got, want)
 	}
 }
@@ -316,7 +317,7 @@ func TestPlainProgressPrintsAtStartAndEnd(t *testing.T) {
 
 	lines := strings.Split(strings.TrimSuffix(stderr(), "\n"), "\n")
 
-	want := "[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 0s, eta unknown"
+	want := "[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 0s, remain unknown"
 	if len(lines) != 2 || lines[0] != want ||
 		!strings.HasPrefix(lines[1], "[3/4] hash: [1000/1000] 100%, ") {
 		t.Errorf("stderr lines = %q, want %q, then the line at 1000",
@@ -336,7 +337,7 @@ func TestPlainProgressPrintsWhileWaiting(t *testing.T) {
 	}, nil, testRedraw, time.Now)
 
 	awaitOutput(t, stderr,
-		"[1/4] load: counting the records, elapsed 0s, eta unknown\n", 3)
+		"[1/4] load: counting the records, elapsed 0s, remain unknown\n", 3)
 	p.finish()
 }
 
@@ -366,23 +367,23 @@ func TestPlainProgressStalledProducer(t *testing.T) {
 
 	clock.Add(5 * time.Second)
 	awaitOutput(t, stderr,
-		"[3/4] hash: [20/100] 20%, 4 files/s, elapsed 5s, eta 20s\n", 3)
+		"[3/4] hash: [20/100] 20%, 4 files/s, elapsed 5s, remain 20s\n", 3)
 
 	clock.Add(time.Second)
 	awaitOutput(t, stderr,
-		"[3/4] hash: [20/100] 20%, 0 files/s, elapsed 6s, eta 24s\n", 3)
+		"[3/4] hash: [20/100] 20%, 0 files/s, elapsed 6s, remain 24s\n", 3)
 
 	clock.Add(4 * time.Second)
 	awaitOutput(t, stderr,
-		"[3/4] hash: [20/100] 20%, stalled disk/db, elapsed 10s, eta 40s\n",
+		"[3/4] hash: [20/100] 20%, stalled disk/db, elapsed 10s, remain 40s\n",
 		3)
 }
 
 // TestPlainProgressRateAfterStall draws a phase's plain line by hand, a
 // little over plainInterval after the line before, as a late tick
 // does, through a stall and a burst of files after it: the rate covers
-// the files read since the line before, while the ETA comes from the
-// phase's average.
+// the files read since the line before, while the time remaining comes
+// from the phase's average.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestPlainProgressRateAfterStall(t *testing.T) {
@@ -410,10 +411,10 @@ func TestPlainProgressRateAfterStall(t *testing.T) {
 	p.finish()
 
 	want := []string{
-		"[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 0s, eta unknown",
-		"[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 5s, eta unknown",
-		"[3/4] hash: [0/1000] 0%, stalled disk/db, elapsed 10s, eta unknown",
-		"[3/4] hash: [100/1000] 10%, 20 files/s, elapsed 15s, eta 2m15s",
+		"[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 0s, remain unknown",
+		"[3/4] hash: [0/1000] 0%, 0 files/s, elapsed 5s, remain unknown",
+		"[3/4] hash: [0/1000] 0%, stalled disk/db, elapsed 10s, remain unknown",
+		"[3/4] hash: [100/1000] 10%, 20 files/s, elapsed 15s, remain 2m15s",
 	}
 
 	lines := strings.Split(strings.TrimSuffix(stderr(), "\n"), "\n")
@@ -426,8 +427,8 @@ func TestPlainProgressRateAfterStall(t *testing.T) {
 // redrawInterval, as a terminal does, through a stall of stallTime and
 // a burst of files after it: the rate covers the files read over the
 // last rateWindow, and falls to 0 once the burst is older than that,
-// while the ETA comes from the phase's average. Plain lines show each
-// draw.
+// while the time remaining comes from the phase's average. Plain lines
+// show each draw.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestProgressRateAfterStallRedrawn(t *testing.T) {
@@ -457,8 +458,8 @@ func TestProgressRateAfterStallRedrawn(t *testing.T) {
 	redraw(rateWindow + redrawInterval)
 	p.finish()
 
-	want := "[3/4] hash: [50/1000] 5%, 10 files/s, elapsed 15s, eta 4m45s\n" +
-		"[3/4] hash: [50/1000] 5%, 0 files/s, elapsed 15s, eta 4m47s\n"
+	want := "[3/4] hash: [50/1000] 5%, 10 files/s, elapsed 15s, remain 4m45s\n" +
+		"[3/4] hash: [50/1000] 5%, 0 files/s, elapsed 15s, remain 4m47s\n"
 	if !strings.Contains(stderr(), want) {
 		t.Errorf("stderr = %q, want the lines %q", stderr(), want)
 	}
@@ -480,10 +481,10 @@ func lastFrame(out string) string {
 }
 
 // TestTerminalRedrawsWhileWaiting checks that the terminal display of a
-// phase, spinner or bar, keeps being drawn while the phase waits for its
-// next item, with the elapsed time as of now, and shows every item
-// completed before the wait. stderr is not a terminal here, so the
-// displays take it to be fallbackWidth wide.
+// phase, with a bar or without, keeps being drawn while the phase waits
+// for its next item, with the elapsed time as of now, and shows every
+// item completed before the wait. The terminal is 120 columns wide, so
+// the hash phase's display has room for its bar.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestTerminalRedrawsWhileWaiting(t *testing.T) {
@@ -506,7 +507,8 @@ func TestTerminalRedrawsWhileWaiting(t *testing.T) {
 	for _, tc := range cases {
 		before := len(stderr())
 		since := func() string { return stderr()[before:] }
-		p := startProgress(tc.params, terminalWidth, testRedraw, time.Now)
+		p := startProgress(tc.params, func() int { return 120 }, testRedraw,
+			time.Now)
 
 		for range 50 {
 			p.increment()
@@ -552,13 +554,13 @@ func drawLate(stderr func() string, params progressParams, width int,
 	return stderr()[before:]
 }
 
-// TestTerminalLinesKeepETA checks the line each display shows on an
-// 80-column terminal at a large scan's counts and times, and the line
-// of trees' read phase at the same counts: the rate is left out, so
-// that the elapsed time and the ETA stay whole.
+// TestTerminalLinesKeepTimeRemaining checks the line each display shows
+// on an 80-column terminal at a large scan's counts and times, and the
+// line of trees' read phase at the same counts: the rate is left out,
+// so that the elapsed time and the time remaining stay whole.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
-func TestTerminalLinesKeepETA(t *testing.T) {
+func TestTerminalLinesKeepTimeRemaining(t *testing.T) {
 	stderr := captureStderr(t)
 
 	cases := []struct {
@@ -571,19 +573,19 @@ func TestTerminalLinesKeepETA(t *testing.T) {
 			estimate: true,
 		},
 		want: "[2/4] walk: [1234567/~1200000] 103%, elapsed 2h30m0s, " +
-			"eta unknown",
+			"remain unknown",
 	}, {
 		params: progressParams{
 			label: scanPhase("hash"), unit: scanUnit, total: 9876543,
 		},
 		want: "[3/4] hash: [1234567/9876543] 12%, elapsed 2h30m0s, " +
-			"eta 17h30m0s",
+			"remain 17h30m0s",
 	}, {
 		params: progressParams{
 			label: treesPhase("read"), unit: "records", total: 9876543,
 		},
 		want: "[2/4] read: [1234567/9876543] 12%, elapsed 2h30m0s, " +
-			"eta 17h30m0s",
+			"remain 17h30m0s",
 	}}
 
 	for _, tc := range cases {
@@ -594,30 +596,32 @@ func TestTerminalLinesKeepETA(t *testing.T) {
 	}
 }
 
-// TestTerminalStalledLines draws every phase's line on an 80-column
-// terminal once the phase has stalled, its last item done 42 seconds
-// before. At an ordinary scan's counts and times, 12345 of 45678 items
-// done four and a half minutes in, the line shows where the phase waits
-// whole, beside the phase, progress, elapsed time and ETA. At a large
-// scan's, 1234567 of 9876543 two and a half hours in, those four still
-// show whole, as the words give way first where they do not fit.
+// TestTerminalStalledLines draws every phase's line once the phase has
+// stalled, its last item done 42 seconds before. At an ordinary scan's
+// counts and times, 12345 of 45678 items done four and a half minutes
+// in, on a terminal 100 columns wide, the line shows where the phase
+// waits whole, beside the phase, progress, elapsed time and time
+// remaining. At a large scan's, 1234567 of 9876543 two and a half hours
+// in, on an 80-column terminal, those four still show whole, as the
+// words give way first where they do not fit.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestTerminalStalledLines(t *testing.T) {
 	stderr := captureStderr(t)
 
-	// drawStalled draws the display params describes as it starts and
-	// once more as it ends, with count of total items done elapsed into
-	// the phase, and returns what the terminal shows.
-	drawStalled := func(params progressParams, total, count int64,
-		elapsed time.Duration,
+	// drawStalled draws the display params describes on a terminal width
+	// columns wide as it starts and once more as it ends, with count of
+	// total items done elapsed into the phase, and returns what the
+	// terminal shows.
+	drawStalled := func(params progressParams, width int, total,
+		count int64, elapsed time.Duration,
 	) string {
 		before := len(stderr())
 		clock := &testClock{now: time.Now()}
 		params.total = total
 
 		// No timed draws come between the two.
-		p := startProgress(params, func() int { return 80 }, time.Hour,
+		p := startProgress(params, func() int { return width }, time.Hour,
 			clock.Now)
 
 		clock.Add(elapsed - 42*time.Second)
@@ -649,19 +653,22 @@ func TestTerminalStalledLines(t *testing.T) {
 			approx = "~"
 		}
 
-		shown := drawStalled(params, 45678, 12345,
+		// The percent is in the line, or beside the bar.
+		shown := drawStalled(params, 100, 45678, 12345,
 			4*time.Minute+30*time.Second)
-		want := params.label + ": [12345/" + approx + "45678] 27%, stalled " +
-			params.stalledOn + ", elapsed 4m30s, eta 12m9s"
-
-		if !strings.Contains(shown, want) {
-			t.Errorf("terminal shows %q, want %q", shown, want)
+		for _, want := range []string{
+			params.label + ": [12345/" + approx + "45678]",
+			", stalled " + params.stalledOn + ", elapsed 4m30s, remain 12m9s",
+		} {
+			if !strings.Contains(shown, want) {
+				t.Errorf("terminal shows %q, want %q", shown, want)
+			}
 		}
 
-		shown = drawStalled(params, 9876543, 1234567, 150*time.Minute)
+		shown = drawStalled(params, 80, 9876543, 1234567, 150*time.Minute)
 		for _, want := range []string{
 			params.label + ": [1234567/" + approx + "9876543] 12%, ",
-			", elapsed 2h30m0s, eta 17h30m0s",
+			", elapsed 2h30m0s, remain 17h30m0s",
 		} {
 			if !strings.Contains(shown, want) {
 				t.Errorf("terminal shows %q, want %q", shown, want)
@@ -706,13 +713,164 @@ func TestTerminalLinesFitWidth(t *testing.T) {
 	}
 }
 
+// TestTerminalLinesStartWithSpinner draws a display with no total and
+// one with a bar, two and a half hours into the phase, on a terminal 120
+// columns wide. Each line must start with the library's spinner in the
+// first column, then a space and the phase's label.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestTerminalLinesStartWithSpinner(t *testing.T) {
+	stderr := captureStderr(t)
+
+	for _, params := range []progressParams{
+		{label: scanPhase("walk"), unit: scanUnit, total: -1},
+		{label: scanPhase("hash"), unit: scanUnit, total: 9876543},
+	} {
+		shown := lastFrame(drawLate(stderr, params, 120, 1234567))
+		spinner, rest, _ := strings.Cut(shown, " ")
+
+		if utf8.RuneCountInString(spinner) != 1 ||
+			!strings.HasPrefix(rest, params.label+": ") {
+			t.Errorf("terminal shows %q, want the spinner, a space and %q",
+				shown, params.label)
+		}
+
+		if params.total > 0 && !strings.Contains(shown, "% |") {
+			t.Errorf("terminal shows %q, want a bar", shown)
+		}
+	}
+}
+
+// TestTerminalBarStaysAtRightEdge draws the hash phase's display two
+// and a half hours in, at two counts whose lines differ in length, on
+// terminals 110 and 120 columns wide. The bar is partly filled at the
+// first count and full at the second. It must start the same number of
+// columns from the terminal's right edge each time, and end the same
+// number of columns from it: a line that grows or shrinks, as its time
+// remaining does, never moves the bar, and nor do the bar's filled
+// cells.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestTerminalBarStaysAtRightEdge(t *testing.T) {
+	stderr := captureStderr(t)
+	params := progressParams{
+		label: scanPhase("hash"), unit: scanUnit, total: 9876543,
+	}
+
+	widths := []int{110, 120}
+	counts := []int64{1234567, 9876543}
+	startFromEdge := make([]int, 0, len(widths)*len(counts))
+	endFromEdge := make([]int, 0, len(widths)*len(counts))
+
+	for _, width := range widths {
+		for _, count := range counts {
+			shown := lastFrame(drawLate(stderr, params, width, count))
+
+			beforeBar, _, found := strings.Cut(shown, "% |")
+			if !found {
+				t.Fatalf("terminal %d wide shows %q, want a bar", width, shown)
+			}
+
+			// The bar ends with the last "|".
+			throughBar := shown[:strings.LastIndex(shown, "|")+1]
+
+			startFromEdge = append(startFromEdge,
+				width-utf8.RuneCountInString(beforeBar))
+			endFromEdge = append(endFromEdge,
+				width-utf8.RuneCountInString(throughBar))
+		}
+	}
+
+	if slices.Min(startFromEdge) != slices.Max(startFromEdge) {
+		t.Errorf("bar starts %v columns from the right edge, want the same "+
+			"every time", startFromEdge)
+	}
+
+	if slices.Min(endFromEdge) != slices.Max(endFromEdge) {
+		t.Errorf("bar ends %v columns from the right edge, want the same "+
+			"every time", endFromEdge)
+	}
+}
+
+// TestTerminalRateGivesWayBeforeBar draws the hash phase's display two
+// and a half hours in. On a terminal 90 columns wide the line fits
+// beside the bar without its rate, but not with it: the bar is drawn
+// and the rate is not. On a terminal 85 columns wide the line does not
+// fit beside the bar even without its rate: neither is drawn, though
+// the rate would fit where the bar was.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestTerminalRateGivesWayBeforeBar(t *testing.T) {
+	stderr := captureStderr(t)
+	params := progressParams{
+		label: scanPhase("hash"), unit: scanUnit, total: 9876543,
+	}
+
+	shown := lastFrame(drawLate(stderr, params, 90, 1234567))
+	if !strings.Contains(shown, "% |") || strings.Contains(shown, "files/s") {
+		t.Errorf("terminal 90 wide shows %q, want the bar and no rate", shown)
+	}
+
+	shown = lastFrame(drawLate(stderr, params, 85, 1234567))
+	if strings.Contains(shown, "% |") || strings.Contains(shown, "files/s") {
+		t.Errorf("terminal 85 wide shows %q, want neither bar nor rate", shown)
+	}
+}
+
+// TestTerminalSpinnerTurnsBesideBar draws the hash phase's display, its
+// bar partly filled, several times on a terminal 120 columns wide. The
+// spinner must turn at every draw. A filled cell of the bar takes one
+// column but three bytes. A line measured in bytes would seem too wide
+// for the terminal, and every draw would start the display over, its
+// spinner included.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestTerminalSpinnerTurnsBesideBar(t *testing.T) {
+	stderr := captureStderr(t)
+	params := progressParams{
+		label: scanPhase("hash"), unit: scanUnit, total: 9876543,
+	}
+
+	// No timed draws: the test draws.
+	p := startProgress(params, func() int { return 120 }, time.Hour,
+		time.Now)
+
+	p.mu.Lock()
+	p.count = 1234567
+	p.mu.Unlock()
+
+	const draws = 4
+
+	spinners := make([]string, 0, draws)
+
+	for range draws {
+		before := len(stderr())
+
+		p.draw()
+
+		spinner, _, _ := strings.Cut(lastFrame(stderr()[before:]), " ")
+		spinners = append(spinners, spinner)
+	}
+
+	p.finish()
+
+	for i := 1; i < len(spinners); i++ {
+		if spinners[i] == spinners[i-1] {
+			t.Errorf("spinners at successive draws are %q, want each to "+
+				"differ from the one before", spinners)
+
+			break
+		}
+	}
+}
+
 // TestTerminalNarrowedRedrawsInPlace draws a large scan's display, two
 // and a half hours into the phase, on a terminal 120 columns wide, then
 // makes the terminal narrower than that line and draws it again a few
 // times. Nothing written after that, the erase before each redraw
 // included, may reach the narrower terminal's last column: it would
-// wrap, and every redraw would leave a line behind. At these widths the
-// spinner stays a spinner and the bar gives way to the spinner.
+// wrap, and every redraw would leave a line behind. At the narrower
+// width the hash phase's bar gives way.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestTerminalNarrowedRedrawsInPlace(t *testing.T) {
@@ -730,7 +888,7 @@ func TestTerminalNarrowedRedrawsInPlace(t *testing.T) {
 			progressParams{
 				label: scanPhase("hash"), unit: scanUnit, total: 9876543,
 			},
-			90,
+			80,
 		},
 	}
 
@@ -770,10 +928,10 @@ func TestTerminalNarrowedRedrawsInPlace(t *testing.T) {
 	}
 }
 
-// TestProgressWarningsOnOwnLines drives the terminal displays, spinner
-// and bar, through a run of warnings with no items between them, as when
-// the walk meets many unreadable paths, for many redraws: every warning
-// must land on a line of its own, never inside a redraw.
+// TestProgressWarningsOnOwnLines drives the terminal displays, with a
+// bar and without, through a run of warnings with no items between
+// them, as when the walk meets many unreadable paths, for many redraws:
+// every warning must land on a line of its own, never inside a redraw.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestProgressWarningsOnOwnLines(t *testing.T) {
@@ -819,11 +977,12 @@ func TestProgressWarningsOnOwnLines(t *testing.T) {
 	}
 }
 
-// TestProgressWriterRowsOnOwnLines drives the terminal displays,
-// spinner and bar, through a run of rows written through the display's
+// TestProgressWriterRowsOnOwnLines drives the terminal displays, with a
+// bar and without, through a run of rows written through the display's
 // writer to the same terminal, as stdout is in a plain "sfdupes trees",
 // each written out at once, for many redraws: the terminal must show
-// every row whole, in order, on a line of its own.
+// every row whole, in order, on a line of its own. The terminal is 120
+// columns wide, so the write phase's display has room for its bar.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestProgressWriterRowsOnOwnLines(t *testing.T) {
@@ -840,7 +999,8 @@ func TestProgressWriterRowsOnOwnLines(t *testing.T) {
 	var want []string
 
 	for _, params := range cases {
-		p := startProgress(params, terminalWidth, testRedraw, time.Now)
+		p := startProgress(params, func() int { return 120 }, testRedraw,
+			time.Now)
 		out := p.writer(os.Stderr)
 
 		// No pause between rows: one is garbled only if a redraw lands

report.go

@@ -45,11 +45,11 @@ func reportPhase(name string) string {
 // problem, including a missing database, is fatal.
 //
 // The work runs in two phases, each under a progress display of its
-// own on stderr: open (open the database) and read (count the records,
-// then read them in signature order, writing each size's duplicate
-// groups to stdout once that size is read; see readReport). Each
-// display ends before the next phase starts, and before an error is
-// returned to be printed.
+// own on stderr: open (open the database) and read (read the number of
+// records from the totals table, then read the records in signature
+// order, writing each size's duplicate groups to stdout once that size
+// is read; see readReport). Each display ends before the next phase
+// starts, and before an error is returned to be printed.
 func runReport(ctx context.Context, stdout io.Writer) error {
 	dbPath, source, err := databasePath()
 	if err != nil {

report_test.go

@@ -187,7 +187,7 @@ func TestReportStdoutFailsWhileReading(t *testing.T) {
 // reportOpenLine is the plain line of report's open phase when it opens
 // the database at once; it prints it as the phase starts and as it ends.
 const reportOpenLine = "[1/2] open: opening the database, elapsed 0s, " +
-	"eta unknown"
+	"remain unknown"
 
 // smokeReport is report's stdout for the smoke-test records.
 const smokeReport = "first\tdupe\tsize\n" +
@@ -225,7 +225,7 @@ func TestRunReportProgressNamesEachPhase(t *testing.T) {
 		"report: database " + path + " (from " + databaseEnv + ")",
 		reportOpenLine,
 		reportOpenLine,
-		"[2/2] read: counting the records, elapsed 0s, eta unknown",
+		"[2/2] read: counting the records, elapsed 0s, remain unknown",
 		"[2/2] read: [6/6] 100%, ",
 		"report: 6 records read, 2 duplicate groups, 4 dupe files, " +
 			"6.1 KiB reclaimable",

scan.go

@@ -383,7 +383,8 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
 ) error {
 	// Types and filesystems are checked before pruning so that an
 	// operand under a dropped one is still scanned, not dropped as lying
-	// under it.
+	// under it, and so that an operand inside a .zfs directory gets its
+	// warning even when it lies under another operand.
 	roots = pruneRoots(s.walkableRoots(roots, rules), oneFS, rules,
 		deviceOfPath)
 
@@ -411,14 +412,15 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
 }
 
 // walkableRoots returns the operands the walk can start from: regular
-// files, and directories not named .zfs. Every other operand is warned
-// about, counted as skipped, and dropped. A dropped operand is no
-// longer a root, so the records stored beneath it are treated as if it
-// had not been given: those under no other root count as outside the
-// roots and are not deleted as unverified. An operand on a filesystem
-// in rules that is not persistent is dropped the same way, with a line
-// saying so that is not a warning, and is not counted as skipped. An
-// operand that fails lstat here is kept, and the walk warns about it.
+// files, and directories not named .zfs, none of them inside a
+// directory named .zfs. Every other operand is warned about, counted as
+// skipped, and dropped. A dropped operand is no longer a root, so the
+// records stored beneath it are treated as if it had not been given:
+// those under no other root count as outside the roots and are not
+// deleted as unverified. An operand on a filesystem in rules that is not
+// persistent is dropped the same way, with a line saying so that is not
+// a warning, and is not counted as skipped. An operand that fails lstat
+// here is kept, and the walk warns about it unless it no longer exists.
 func (s *scanState) walkableRoots(roots []string, rules walkRules) []string {
 	kept := make([]string, 0, len(roots))
 
@@ -453,8 +455,8 @@ func (s *scanState) walkableRoots(roots []string, rules walkRules) []string {
 // included, so its display counts against the number of records.
 func (s *scanState) loadIndex(ctx context.Context, roots []string) error {
 	// Indexing tens of millions of records takes real time; without a
-	// display the scan looks hung before the walk begins. Counting them
-	// first gives the display its total.
+	// display the scan looks hung before the walk begins. The number of
+	// records, read first, gives the display its total.
 	prog := newProgress(progressParams{
 		label: scanPhase("load"), unit: scanUnit, total: -1,
 		waitingFor: countingRecords, stalledOn: "db",
@@ -629,8 +631,9 @@ func isDatabaseFile(f fileRec, rules walkRules) bool {
 // (an interrupted scan resumes cheaply: the next run skips everything
 // already recorded). A run that fails to hash is warned about and
 // counted as skipped; stale records for its paths, if any, are deleted
-// by the update phase. The total counts actual reads, so the display
-// shows a real ETA.
+// by the update phase. A run whose file no longer exists gets no record
+// and no warning, and is not counted as skipped. The total counts actual
+// reads, so the display's time remaining is meaningful.
 //
 // Returning early — a failed database write, or a cancelled scan — must
 // not strand the pool: the feeder would park forever on a full jobs
@@ -662,6 +665,11 @@ func (s *scanState) hashPhase(ctx context.Context, workers int) error {
 
 		prog.increment()
 
+		// A file removed since the walk is no error: it gets no record.
+		if errors.Is(r.err, fs.ErrNotExist) {
+			continue
+		}
+
 		if r.err != nil {
 			s.st.skipped += len(r.run)
 
@@ -829,10 +837,12 @@ type walkEvent struct {
 // startWalk seeds every root into the shared walk worker pool and
 // returns the event stream: one record per regular file, one warning
 // event per per-path error, and one event per path -x or rules kept the
-// walk from entering or recording. The channel is closed when the walk
-// completes, and also when ctx is cancelled — every goroutine in the
-// pool abandons its blocking send in that case, so the consumer sees a
-// truncated but properly terminated stream instead of a stalled one.
+// walk from entering or recording. A path that no longer exists when
+// the walk reaches it is no error, and gets no event. The channel is
+// closed when the walk completes, and also when ctx is cancelled —
+// every goroutine in the pool abandons its blocking send in that case,
+// so the consumer sees a truncated but properly terminated stream
+// instead of a stalled one.
 func startWalk(ctx context.Context, roots []string, oneFS bool,
 	rules walkRules, workers int,
 ) <-chan walkEvent {
@@ -865,9 +875,24 @@ func sendEvent(ctx context.Context, events chan<- walkEvent,
 
 // operandWarning returns the one-line warning for an operand the walk
 // does not start from, naming the path and what it is, or "" for one it
-// does: a regular file, or a directory not named .zfs. Symlinks are
-// never followed, including as operands.
+// does: a regular file, or a directory not named .zfs, that is not
+// inside a directory named .zfs, neither as given nor with its symlinks
+// resolved. Symlinks are never followed, including as operands.
 func operandWarning(root string, fi fs.FileInfo) string {
+	// The walk never enters a .zfs directory, so it starts from nothing
+	// inside one either, not even through a symlink. An operand whose
+	// symlinks cannot be resolved is checked as given.
+	resolved, err := filepath.EvalSymlinks(root)
+	if err != nil {
+		resolved = root
+	}
+
+	if slices.Contains(strings.Split(filepath.Dir(root), "/"), ".zfs") ||
+		slices.Contains(strings.Split(filepath.Dir(resolved), "/"), ".zfs") {
+		return fmt.Sprintf("walk %s: skipping operand inside a .zfs directory",
+			root)
+	}
+
 	var kind string
 
 	switch mode := fi.Mode(); {
@@ -899,13 +924,19 @@ func operandWarning(root string, fi fs.FileInfo) string {
 // one of the scan's own database files in rules, and a directory
 // operand becomes an initial job. walkableRoots has already dropped
 // every other operand. One that has changed into something else since
-// is warned about and skipped here; it is still a root, so the records
-// stored beneath it, other than those under another root, are deleted
-// as unverified.
+// is warned about and skipped here, and one removed since is skipped
+// without a warning. Either is still a root, so the records stored
+// beneath it, other than those under another root, are deleted as
+// unverified.
 func seedRoot(ctx context.Context, root string, rules walkRules,
 	events chan<- walkEvent,
 ) []dirJob {
 	fi, err := os.Lstat(root)
+	if errors.Is(err, fs.ErrNotExist) {
+		// Removed since the scan started: no error, and no files.
+		return nil
+	}
+
 	if err != nil {
 		sendEvent(ctx, events, walkEvent{
 			warn: fmt.Sprintf("walk %s: %v", root, err),
@@ -1023,11 +1054,17 @@ func dispatchDirs(ctx context.Context, initial []dirJob,
 
 // walkOneDir reads one directory, emitting an event per regular-file
 // entry and a warning event per unreadable one, and returns the
-// subdirectories to descend into.
+// subdirectories to descend into. A directory or entry removed since
+// the walk found it gets no event.
 func walkOneDir(ctx context.Context, job dirJob, oneFS bool,
 	rules walkRules, events chan<- walkEvent,
 ) []dirJob {
 	entries, err := os.ReadDir(job.path)
+	if errors.Is(err, fs.ErrNotExist) {
+		// Removed since its parent was read: no error, and no files.
+		return nil
+	}
+
 	if err != nil {
 		sendEvent(ctx, events, walkEvent{
 			warn: fmt.Sprintf("walk %s: %v", job.path, err),
@@ -1070,16 +1107,22 @@ func walkOneDir(ctx context.Context, job dirJob, oneFS bool,
 
 // emitFile stats one regular directory entry and emits its record.
 // The lstat happens here in the walk worker, while the directory's
-// metadata is still hot; a path that fails to stat (or stops being a
-// regular file) between the directory read and the lstat is warned
-// about and skipped. A file on a filesystem in rules that is not
-// persistent, as through a bind mount of one file, is not recorded: it
-// gets an event naming it instead. One of the scan's own database files
-// in rules gets no event at all.
+// metadata is still hot; a path that fails to stat between the
+// directory read and the lstat is warned about and skipped, unless it
+// no longer exists, and one that has stopped being a regular file is
+// skipped. A file on a filesystem in rules that is not persistent, as
+// through a bind mount of one file, is not recorded: it gets an event
+// naming it instead. One of the scan's own database files in rules gets
+// no event at all.
 func emitFile(ctx context.Context, p string, e fs.DirEntry,
 	rules walkRules, events chan<- walkEvent,
 ) {
 	info, err := e.Info()
+	if errors.Is(err, fs.ErrNotExist) {
+		// Removed since its directory was read: no error.
+		return
+	}
+
 	if err != nil {
 		sendEvent(ctx, events, walkEvent{
 			warn: fmt.Sprintf("stat %s: %v", p, err),
@@ -1131,6 +1174,11 @@ func subdirJob(ctx context.Context, p string, e fs.DirEntry,
 	}
 
 	info, err := e.Info()
+	if errors.Is(err, fs.ErrNotExist) {
+		// Removed since its parent was read: no error, and no files.
+		return dirJob{}, false
+	}
+
 	if err != nil {
 		sendEvent(ctx, events, walkEvent{
 			warn: fmt.Sprintf("walk %s: %v", p, err),
@@ -1195,7 +1243,7 @@ func inodeOfInfo(fi fs.FileInfo) (uint64, uint64) {
 
 // hashResult carries the hashes computed for one inode run (or the
 // error that prevented computing them) from the pool's workers to the
-// hash phase.
+// hash phase. Its run leaves out the paths found removed.
 type hashResult struct {
 	run     []fileRec
 	head    [sha256.Size]byte
@@ -1296,6 +1344,13 @@ func hashWorker(ctx context.Context, jobs <-chan []fileRec,
 
 		head, tail, content, err := hash(run[0].path, run[0].size)
 
+		// A path removed since the walk is dropped from its run. The
+		// next path is a hard link to the same file, and is read instead.
+		for errors.Is(err, fs.ErrNotExist) && len(run) > 1 {
+			run = run[1:]
+			head, tail, content, err = hash(run[0].path, run[0].size)
+		}
+
 		select {
 		case results <- hashResult{
 			run: run, head: head, tail: tail, content: content, err: err,

scan_test.go

@@ -683,6 +683,108 @@ func TestSubdirJobStatError(t *testing.T) {
 	}
 }
 
+// TestWalkPassesOverRemovedPaths removes a file and a directory after
+// their parent was read, as when they are deleted while a scan runs.
+// The walk then reaches them, the directory also as one to read and as
+// an operand. None of that is an error: the walk sends no event.
+func TestWalkPassesOverRemovedPaths(t *testing.T) {
+	t.Parallel()
+
+	dir := t.TempDir()
+	file := writeFile(t, dir, "file", []byte("x"))
+	writeFile(t, dir, "sub/f", []byte("y"))
+	sub := filepath.Join(dir, "sub")
+	fileEntry := dirEntryFor(t, dir, "file")
+	subEntry := dirEntryFor(t, dir, "sub")
+
+	for _, p := range []string{file, sub} {
+		err := os.RemoveAll(p)
+		if err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	ctx := t.Context()
+	events := make(chan walkEvent, 4)
+
+	emitFile(ctx, file, fileEntry, walkRules{}, events)
+
+	_, descend := subdirJob(ctx, sub, subEntry, dirJob{path: dir}, false,
+		walkRules{}, events)
+	subs := walkOneDir(ctx, dirJob{path: sub}, false, walkRules{}, events)
+	jobs := seedRoot(ctx, sub, walkRules{}, events)
+
+	close(events)
+
+	for ev := range events {
+		t.Errorf("event %+v for a removed path, want none", ev)
+	}
+
+	if descend || len(subs) != 0 || len(jobs) != 0 {
+		t.Errorf("descend = %t, subdirectories %+v, jobs %+v, want none",
+			descend, subs, jobs)
+	}
+}
+
+// TestScanWarnsAboutUnreadablePaths checks that a permission error is
+// still warned about and counted as skipped, unlike a removed path. The
+// walk reaches an unreadable directory, and a file in a directory that
+// can be read but not searched. Another such file is an operand.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestScanWarnsAboutUnreadablePaths(t *testing.T) {
+	if os.Geteuid() == 0 {
+		t.Skip("root ignores file permissions")
+	}
+
+	stderr := captureStderr(t)
+	dir := t.TempDir()
+	tree := filepath.Join(dir, "tree")
+	walked := writeFile(t, tree, "unsearchable/f", []byte("x"))
+	operand := writeFile(t, dir, "unsearchable/f", []byte("y"))
+	unreadable := filepath.Join(tree, "unreadable")
+
+	err := os.Mkdir(unreadable, 0o750)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	modes := map[string]fs.FileMode{
+		unreadable:            0,
+		filepath.Dir(walked):  0o400,
+		filepath.Dir(operand): 0o400,
+	}
+
+	for p, mode := range modes {
+		err = os.Chmod(p, mode)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		// Runs before t.TempDir's own cleanup, which must delete the files.
+		t.Cleanup(func() {
+			//nolint:gosec // removing the files needs the search bit back
+			_ = os.Chmod(p, 0o700)
+		})
+	}
+
+	st := syncTree(t, openTestDB(t), tree, operand)
+	if st != (scanStats{skipped: 3}) {
+		t.Errorf("stats = %+v, want 3 skipped and nothing else", st)
+	}
+
+	got := stderr()
+	for _, warning := range []string{
+		"walk " + unreadable + ": open " + unreadable + ": permission denied\n",
+		"stat " + walked + ": lstat " + walked + ": permission denied\n",
+		"walk " + operand + ": lstat " + operand + ": permission denied\n",
+	} {
+		if !strings.Contains(got, warning) {
+			t.Errorf("stderr = %q, want %q", got, warning)
+		}
+	}
+}
+
 // syncOneFilesystem scans roots into db with -x and returns the scan
 // stats. Every device pruneRoots looks up, and the device each
 // operand's walk compares its subdirectories with, comes from devices
@@ -1126,9 +1228,9 @@ func TestScanPipeline(t *testing.T) {
 // TestScanProgressNamesEachPhase checks the plain progress lines of a
 // first scan, and of a rescan of one operand holding two of the
 // database's records. Each phase's first line names it with its place
-// among the four: load counts the records before reading them, and the
-// walk estimates its total from the records under its operand, or says
-// its ETA is unknown when there are none.
+// among the four: load gets the number of records before reading them,
+// and the walk estimates its total from the records under its operand,
+// or says its time remaining is unknown when there are none.
 //
 //nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
 func TestScanProgressNamesEachPhase(t *testing.T) {
@@ -1143,14 +1245,14 @@ func TestScanProgressNamesEachPhase(t *testing.T) {
 	syncTree(t, db, filepath.Join(dir, "a"))
 
 	rescan := stderr()[len(first):]
-	load := "[1/4] load: counting the records, elapsed 0s, eta unknown"
+	load := "[1/4] load: counting the records, elapsed 0s, remain unknown"
 	cases := map[string][]string{
 		first: {
-			load, "[2/4] walk: 0 files, 0 files/s, elapsed 0s, eta unknown",
+			load, "[2/4] walk: 0 files, 0 files/s, elapsed 0s, remain unknown",
 			"[3/4] hash: [0/", "[4/4] update: [0/",
 		},
 		rescan: {
-			load, "[2/4] walk: [0/~2] 0%, 0 files/s, elapsed 0s, eta unknown",
+			load, "[2/4] walk: [0/~2] 0%, 0 files/s, elapsed 0s, remain unknown",
 			"[3/4] hash: [0/", "[4/4] update: [0/",
 		},
 	}
@@ -1685,6 +1787,55 @@ func TestHashPhaseSkipsFileReplacedByFIFO(t *testing.T) {
 	}
 }
 
+// TestHashPhasePassesOverRemovedFiles removes two walked files before
+// the hash phase reads them: one with no other path, and the first of
+// two hard links to one file. Neither is an error. The phase prints
+// nothing but its progress, counts nothing as skipped, and records only
+// the hard link still there.
+//
+//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
+func TestHashPhasePassesOverRemovedFiles(t *testing.T) {
+	stderr := captureStderr(t)
+	dir := t.TempDir()
+	alone := writeFile(t, dir, "alone.bin", pattern(1, 300))
+	a := writeFile(t, dir, "a.bin", pattern(2, 300))
+	b := filepath.Join(dir, "b.bin")
+
+	err := os.Link(a, b)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	walked, _ := collectWalk(t, []string{dir}, false, 1)
+
+	// a.bin sorts before b.bin, so the run of the two reads it first.
+	for _, p := range []string{alone, a} {
+		err = os.Remove(p)
+		if err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	s := &scanState{db: openTestDB(t), toHash: walked}
+
+	err = s.hashPhase(t.Context(), 1)
+	if err != nil {
+		t.Fatalf("hashPhase = %v, want nil", err)
+	}
+
+	if s.st.skipped != 0 || len(s.batch) != 1 || s.batch[0].path != b {
+		t.Errorf("skipped = %d, records %+v, want none skipped and a "+
+			"record for %s alone", s.st.skipped, s.batch, b)
+	}
+
+	for line := range strings.Lines(stderr()) {
+		if !strings.HasPrefix(line, scanPhase("hash")+": ") {
+			t.Errorf("stderr holds %q, want the hash phase's progress alone",
+				line)
+		}
+	}
+}
+
 // injectedWriteFailure is the message the injected database trigger
 // aborts with, so the test can recognize its own failure in the error
 // the scan reports.

trees.go

@@ -40,11 +40,11 @@ func treesPhase(name string) string {
 // database problem, including a missing database, is fatal.
 //
 // The work runs in four phases, each under a progress display of its
-// own on stderr: open (open the database), read (count the records,
-// then read them into the hierarchy), group (group the directories by
-// digest) and write (write the groups to stdout). Each display ends
-// before the next phase starts, and before an error is returned to be
-// printed.
+// own on stderr: open (open the database), read (read the number of
+// records from the totals table, then read the records into the
+// hierarchy), group (group the directories by digest) and write (write
+// the groups to stdout). Each display ends before the next phase
+// starts, and before an error is returned to be printed.
 func runTrees(ctx context.Context, stdout io.Writer) error {
 	dbPath, source, err := databasePath()
 	if err != nil {

trees_test.go

@@ -13,7 +13,7 @@ import (
 // treesOpenLine is the plain line of trees' open phase when it opens
 // the database at once; it prints it as the phase starts and as it ends.
 const treesOpenLine = "[1/4] open: opening the database, elapsed 0s, " +
-	"eta unknown"
+	"remain unknown"
 
 // smokeTreeRecs mirrors the README smoke-test tree layout: /d/t1 and
 // /d/t2 are identical, /d/t3 differs from them only by one filename.
@@ -236,11 +236,11 @@ func TestRunTreesProgressNamesEachPhase(t *testing.T) {
 		"trees: database " + path + " (from " + databaseEnv + ")",
 		treesOpenLine,
 		treesOpenLine,
-		"[2/4] read: counting the records, elapsed 0s, eta unknown",
+		"[2/4] read: counting the records, elapsed 0s, remain unknown",
 		"[2/4] read: [6/6] 100%, ",
-		"[3/4] group: [0/8] 0%, 0 directories/s, elapsed 0s, eta unknown",
+		"[3/4] group: [0/8] 0%, 0 directories/s, elapsed 0s, remain unknown",
 		"[3/4] group: [8/8] 100%, ",
-		"[4/4] write: [0/1] 0%, 0 groups/s, elapsed 0s, eta unknown",
+		"[4/4] write: [0/1] 0%, 0 groups/s, elapsed 0s, remain unknown",
 		"[4/4] write: [1/1] 100%, ",
 		"trees: 6 records read, 1 duplicate tree groups, 1 dupe trees, ",
 	}