Show the progress rate over the last 5 seconds, and where a phase waits after 10 seconds with no item done (closes #125)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
The rate was the phase's average, so a burst after a long wait read as over 1,000 files a second. It now covers the items completed over the last 5 seconds, or since the line before off a terminal; the ETA still comes from the average. Once no item has completed for 10 seconds, the rate gives way to "stalled" and where the phase waits, as "stalled disk/db": disk, db, out (stdout) or cpu, or two of them. A line too wide for the terminal loses its bar first, then its rate or those words, so the phase, progress, elapsed time and ETA stay whole. The content phase's last partial batch is now written under a display of its own. The tests set the display's clock by hand. Deviation: the stall words hold no time and name where the phase waits, not what it does. Judgement call: trees' group phase, which works in memory, says "cpu". Model: opus-5-5
This commit is contained in:
@@ -799,7 +799,8 @@ A scan has five phases (`[1/5] load`, `[2/5] walk`, `[3/5] hash`,
|
||||
`head`, and `tail`, without fetching the records. It then says it is searching
|
||||
the database until SQLite returns the first record; the checks, and their rate
|
||||
and ETA, start there. The files to read then get a display of their own, with
|
||||
their count as its total.
|
||||
their count as its total, and so does the writing of the last of their records
|
||||
to the database, with the count of those records as its total.
|
||||
|
||||
`trees` has four phases (`[1/4] open`, `[2/4] read`, `[3/4] group`,
|
||||
`[4/4] write`, see "`trees` mode"):
|
||||
@@ -833,17 +834,36 @@ Each display is one line holding:
|
||||
- a `[m/n] x%` display (items processed / total items, percent); before a phase
|
||||
has a total, the items processed, or what the phase is waiting for
|
||||
- current rate (items/s), naming the items the phase counts: files, records,
|
||||
directories or groups
|
||||
directories or groups. It covers the items completed over the last 5 seconds,
|
||||
or, when stderr is not a terminal, since the line before, so it falls to 0
|
||||
while nothing completes, and a burst of items after a wait shows as a burst
|
||||
rather than as a steady rate. Once no item has completed for 10 seconds, the
|
||||
word `stalled` and where the phase waits take the rate's place (see below).
|
||||
- elapsed time
|
||||
- estimated time remaining, or `unknown`
|
||||
- estimated time remaining, or `unknown`. It comes from the phase's average rate
|
||||
since it started counting, so that it does not swing with each burst.
|
||||
|
||||
Example shape (exact layout is flexible, content is not):
|
||||
|
||||
```
|
||||
[3/5] hash: [12345/98765] 12% |████ | 92 files/s elapsed 2:32 eta 17:54
|
||||
[2/5] walk: [1234567/~1200000] 103%, 5120 files/s, elapsed 4m1s, eta unknown
|
||||
[3/5] hash: [12345/45678] 27%, stalled disk/db, elapsed 4m30s, eta 12m9s
|
||||
```
|
||||
|
||||
When no item has completed for 10 seconds, as while one large file is read, a
|
||||
disk is slow or the database is busy, the line says `stalled` and where the
|
||||
phase waits in place of the rate, which would read 0: `disk` for files, `db` for
|
||||
the database, `out` for stdout, and two of them, as `disk/db`, for a phase that
|
||||
may wait on either. `walk` says `disk`. `hash` and the content phase's reads say
|
||||
`disk/db`, as they read files and write their records to the database between
|
||||
them, and so do the content phase's checks, which `lstat` each record's file as
|
||||
the database search returns it. `load`, `update`, the content phase's last write
|
||||
and the `read` phase of `trees` say `db`. `trees`' `write` says `out`, and its
|
||||
`group`, which works in memory, says `cpu`. The `read` phase of `report` writes
|
||||
the report while it reads the records, so it says `db/out`. A phase still
|
||||
waiting for its total already says what it waits for, and says nothing more.
|
||||
|
||||
Additional requirements:
|
||||
|
||||
- On a terminal, a phase with an exact total (every phase but `scan`'s `walk`,
|
||||
@@ -855,15 +875,18 @@ Additional requirements:
|
||||
times a second, so its elapsed time keeps moving while a phase waits for its
|
||||
next item.
|
||||
- On a terminal no line reaches the terminal's last column, however narrow the
|
||||
terminal is. The phase and its place, the progress, the elapsed time and the
|
||||
ETA are kept whole: a line too wide for the terminal leaves out its rate first
|
||||
(or what the phase is waiting for), and a bar with no room beside those four
|
||||
gives way to the spinner. Only when those four alone are too wide, as on a
|
||||
terminal narrower than about 70 columns hours into a scan of millions of
|
||||
files, is the line cut short at the terminal's edge, so the end of the ETA is
|
||||
lost first. A terminal made narrower while a phase runs gets the same: the
|
||||
next draw erases the line within the new width and fits the line to it, and
|
||||
the display goes on redrawing in place.
|
||||
terminal is. A line too wide for the terminal first loses its bar, which gives
|
||||
way to the spinner, then its rate, the words of a stalled phase, or what the
|
||||
phase is waiting for. The phase and its place, the progress, the elapsed time
|
||||
and the ETA are kept whole. On an 80-column terminal, a stalled line at counts
|
||||
and times like those of the stalled line in the example above keeps its words
|
||||
whole in every phase; where the words and the ETA do not both fit, as at
|
||||
millions of files and times in hours, the words give way. Only when those four
|
||||
alone are too wide, as on a terminal narrower than about 70 columns hours into
|
||||
a scan of millions of files, is the line cut short at the terminal's edge, so
|
||||
the end of the ETA is lost first. A terminal made narrower while a phase runs
|
||||
gets the same: the next draw erases the line within the new width and fits the
|
||||
line to it, and the display goes on redrawing in place.
|
||||
- When stderr is not a terminal (a pipe, a file, `/dev/null`), do not emit ANSI
|
||||
redraws: print the line the moment each phase starts, every 5 seconds while it
|
||||
runs (whether or not an item has completed, and never more often), and once
|
||||
|
||||
Reference in New Issue
Block a user