Show the progress rate over the last 5 seconds, and where a phase waits after 10 seconds with no item done (closes #125)
check / check (push) Waiting to run

The rate was the phase's average, so a burst after a long wait read as
over 1,000 files a second. It now covers the items completed over the
last 5 seconds, or since the line before off a terminal; the ETA still
comes from the average. Once no item has completed for 10 seconds, the
rate gives way to "stalled" and where the phase waits, as "stalled
disk/db": disk, db, out (stdout) or cpu, or two of them. A line too
wide for the terminal loses its bar first, then its rate or those
words, so the phase, progress, elapsed time and ETA stay whole. The
content phase's last partial batch is now written under a display of
its own. The tests set the display's clock by hand.

Deviation: the stall words hold no time and name where the phase waits, not what it does.
Judgement call: trees' group phase, which works in memory, says "cpu".

Model: opus-5-5
This commit is contained in:
2026-10-16 22:47:32 +00:00
parent e9e09effc6
commit ab9ca809b3
9 changed files with 501 additions and 112 deletions
+18 -5
View File
@@ -446,7 +446,7 @@ func (s *scanState) loadIndex(ctx context.Context, roots []string) error {
// first gives the display its total.
prog := newProgress(progressParams{
label: scanPhase("load"), unit: scanUnit, total: -1,
waitingFor: countingRecords,
waitingFor: countingRecords, stalledOn: "db",
})
defer prog.finish()
@@ -497,7 +497,7 @@ func (s *scanState) walkPhase(
prog := newProgress(progressParams{
label: scanPhase("walk"), unit: scanUnit, total: estimate,
estimate: true,
estimate: true, stalledOn: "disk",
})
var (
@@ -687,8 +687,11 @@ func (s *scanState) readRuns(ctx context.Context, workers int,
pool := startHashPool(ctx, runs, workers, hash)
defer pool.stop()
// Between results the phase may wait on a file being read or on a
// batch of records being written.
prog := newProgress(progressParams{
label: scanPhase(label), unit: scanUnit, total: int64(len(runs)),
stalledOn: "disk/db",
})
defer prog.finish()
@@ -778,6 +781,7 @@ func (s *scanState) updatePhase(ctx context.Context) error {
total := len(s.batch) + len(s.sentinels) + len(deletes)
prog := newProgress(progressParams{
label: scanPhase("update"), unit: scanUnit, total: int64(total),
stalledOn: "db",
})
defer prog.finish()
@@ -840,7 +844,15 @@ func (s *scanState) contentPhase(ctx context.Context, workers int) error {
return err
}
return applyChanges(ctx, s.db, s.batch, nil, nil)
// The last partial batch is written under a display of its own, as
// the update phase writes the hash phase's.
prog := newProgress(progressParams{
label: scanPhase("content"), unit: scanUnit,
total: int64(len(s.batch)), stalledOn: "db",
})
defer prog.finish()
return applyChanges(ctx, s.db, s.batch, nil, prog)
}
// contentCandidates returns the files the content phase reads, and
@@ -863,10 +875,11 @@ func (s *scanState) contentCandidates(
// without a display the scan looks hung before the reads begin.
// The display says it is searching until SQLite returns the first
// record. The checks start there, against a count of the records
// the search returns, and so do their rate and ETA.
// the search returns, and so do their rate and ETA. Between records
// they may wait on a file's lstat or on the search.
prog := newProgress(progressParams{
label: scanPhase("content"), unit: scanUnit, total: -1,
waitingFor: "searching the database",
waitingFor: "searching the database", stalledOn: "disk/db",
})
defer prog.finish()