report: show its phases as N of M, with progress, elapsed time and ETA (closes #114)

report now runs in two phases, open and read, under the progress
display scan and trees use, until the summary line. open covers a wait
on a busy database. read counts the records, then reads them in the
same read-only transaction in signature order, the files_signature
index read backwards, so SQLite sorts nothing and the read ends at
exactly its count. The duplicate groups are formed in Go from runs of
equal signatures; only those of the size being read are held, and they
are written, ordered by first path, before the next size. The rows go
through the display's stdout writer. stdout and the summary line are
unchanged.

Judgement call: writing is part of read, not a phase of its own, since each size's rows are written while the read goes on.

Model: opus-5-5

README.md

@@ -199,12 +199,13 @@ Goals, in order:
     under resilver). Holding one small record (path, size, mtime) per file in
     memory during a scan is acceptable; holding every file's hashes is not (they
     stay in the database). The reporting commands do not hold every file's
-    hashes either: `report` lets SQLite group and order the records and writes
-    each row as it reads it, so its memory does not grow with the database, and
-    `trees` reads the records in path order and keeps each directory's path,
-    digest and totals, plus the hashes of only the files in the directories
-    holding the record being read, so its memory grows with the number of
-    directories and with the size of the largest directory.
+    hashes either: `report` reads the records in signature order and holds the
+    duplicate paths of one size at a time, so its memory grows with the
+    duplicates of the size that has the most of them, and `trees` reads the
+    records in path order and keeps each directory's path, digest and totals,
+    plus the hashes of only the files in the directories holding the record
+    being read, so its memory grows with the number of directories and with the
+    size of the largest directory.
 3.  **Scan incrementally, analyze offline.** The expensive filesystem scan
     maintains a persistent database; an unchanged file is never read again on a
     rescan, except to compute its content hash once a file of 10 MiB or more
@@ -323,9 +324,9 @@ All three subcommands operate on a single SQLite database file:
   needs the database to itself: a `scan` that starts while a report is still
   reading waits for it up to the 10-second busy timeout, then fails; a `scan`
   that ends while a report has the database open warns and leaves the database
-  in WAL mode until the next scan. `report` writes each row as it reads it, so
-  it is still reading while its output is paused (a pager, a stalled pipe), and
-  a `scan` started then fails after the busy timeout.
+  in WAL mode until the next scan. `report` writes its rows while it reads the
+  records, so it is still reading while its output is paused (a pager, a stalled
+  pipe), and a `scan` started then fails after the busy timeout.
 - A `scan` writes to the database and the files beside it at these points (its
   phases are described under "`scan` mode"):
     - As it starts, before the walk: the directory, an empty database file and
@@ -379,8 +380,8 @@ All three subcommands operate on a single SQLite database file:
     a file of 10 MiB or more, `content` stays empty until the content phase of a
     scan (see "`scan` mode" below) has read the file. A record with an empty
     `content` is never part of a duplicate group, though it still defines the
-    file for tree reconstruction. The `files_signature` index lets SQLite group
-    the records by signature for `report` without sorting the whole table.
+    file for tree reconstruction. The `files_signature` index lets `report` read
+    the records in signature order without sorting the table.
 
 ### Duplicate detection
 
@@ -604,11 +605,8 @@ total.)
 
 **`report` must never touch the filesystem being analyzed.** It does not stat,
 open, or otherwise access any path that appears in the records; its only I/O is
-reading the database, writing stdout/stderr, and the temporary file SQLite sorts
-in when the duplicate rows do not fit in memory. SQLite puts that file in
-`$SQLITE_TMPDIR` or `$TMPDIR` when set, otherwise in `/var/tmp` (or `/tmp`), and
-deletes it as soon as it has opened it. `report` must produce identical output
-whether or not the scanned filesystem is still mounted.
+reading the database and writing stdout/stderr. `report` must produce identical
+output whether or not the scanned filesystem is still mounted.
 
 Processing:
 
@@ -622,6 +620,16 @@ Processing:
   by `first` path ascending. Output must be fully deterministic for a given
   database state.
 
+`report` runs in two phases, each with its own progress display on stderr (see
+"Progress"): `open` opens the database; `read` counts the records, then reads
+them in signature order, largest size first. That is the order of the
+`files_signature` index read backwards, so SQLite sorts nothing and returns the
+first record at once. The records sharing a signature come together, and so do
+the records of one size, so `report` holds only the duplicate groups of the size
+it is reading, and writes them, ordered by `first` path, once it reaches a
+smaller size. Its memory holds at most the duplicate paths of one size at a
+time, never every record.
+
 #### Report output format
 
 TSV on stdout: a header line, then one row per duplicate file (N-1 rows for a
@@ -721,13 +729,13 @@ units.
 
 ### Progress
 
-Use the progress-bar library for all progress of `scan` and `trees`; rendering
-in the style of `pv` is the model. All progress goes to stderr.
+Use the progress-bar library for all progress of `scan`, `report` and `trees`;
+rendering in the style of `pv` is the model. All progress goes to stderr.
 
-`scan` and `trees` must always show that they are working and not hung. Each
-phase gets its own display from the moment it starts, naming the phase and its
-place among the command's phases and showing its progress, its elapsed time and
-its estimated time remaining (ETA).
+`scan`, `report` and `trees` must always show that they are working and not
+hung. Each phase gets its own display from the moment it starts, naming the
+phase and its place among the command's phases and showing its progress, its
+elapsed time and its estimated time remaining (ETA).
 
 A scan has five phases (`[1/5] load`, `[2/5] walk`, `[3/5] hash`,
 `[4/5] update`, `[5/5] content`):
@@ -757,6 +765,14 @@ A scan has five phases (`[1/5] load`, `[2/5] walk`, `[3/5] hash`,
 - `group` has the directories `read` found as its total, and `write` the
   duplicate-tree groups.
 
+`report` has two phases (`[1/2] open`, `[2/2] read`, see "`report` mode"):
+
+- `open` has no total. While it opens the database, waiting on a busy one
+  included, the display says it is opening the database.
+- `read` counts the records first, then reads them against that count, writing
+  the rows to stdout as it goes. While the count runs, the display says it is
+  counting the records.
+
 Each display is one line holding:
 
 - the phase and its place among the phases
@@ -777,12 +793,12 @@ Example shape (exact layout is flexible, content is not):
 Additional requirements:
 
 - On a terminal, a phase with an exact total (every phase but `scan`'s `walk`
-  and `trees`' `open`, once it has counted what it will do) is drawn as its line
-  followed by the progress-bar library's bar, 10 columns wide, which shows the
-  percent in place of the line's own. Any other phase, and a phase whose bar has
-  no room, is drawn as the library's spinner followed by its line. Each display
-  is drawn again several times a second, so its elapsed time keeps moving while
-  a phase waits for its next item.
+  and the `open` phase of `report` and `trees`, once it has counted what it will
+  do) is drawn as its line followed by the progress-bar library's bar, 10
+  columns wide, which shows the percent in place of the line's own. Any other
+  phase, and a phase whose bar has no room, is drawn as the library's spinner
+  followed by its line. Each display is drawn again several times a second, so
+  its elapsed time keeps moving while a phase waits for its next item.
 - On a terminal no line reaches the terminal's last column, however narrow the
   terminal is. The phase and its place, the progress, the elapsed time and the
   ETA are kept whole: a line too wide for the terminal leaves out its rate first
@@ -803,12 +819,12 @@ Additional requirements:
 - A warning printed during a phase always lands on a line of its own, never
   inside the progress display. So does every line written to stdout when stdout
   and stderr are the same file (the same device and inode): one terminal, as in
-  a plain `sfdupes trees`, or one pipe or file that both are sent to. When they
-  are not, stdout is written without waiting on the display, and the display
-  keeps being drawn while a slow reader of stdout holds up a write.
+  a plain `sfdupes report` or `sfdupes trees`, or one pipe or file that both are
+  sent to. When they are not, stdout is written without waiting on the display,
+  and the display keeps being drawn while a slow reader of stdout holds up a
+  write.
 - A phase that stops short of its total, as an interrupted one does, is left
   showing the count it stopped at rather than filled up.
-- `report` mode needs no progress display, only its stderr summary.
 
 ### Error handling and exit codes