2 Commits

Author SHA1 Message Date
a0f0050ada Announce each operand on stderr before its passes
All checks were successful
check / check (push) Successful in 6s
With per-operand walk/hash/update cycles, a multi-operand invocation
(e.g. scan /srv/*) showed pass totals that looked like the whole
run's: an operator watching operand 3 of 14 hash 300k files concluded
the other 20M files were being skipped. Print the operand path and
its position before each cycle.
2026-07-24 07:52:54 +07:00
6a15b879de Merge branch 'parallel-walk': parallel per-directory walk, per-operand commits
All checks were successful
check / check (push) Successful in 1m10s
2026-07-24 07:44:28 +07:00
3 changed files with 21 additions and 1 deletions

View File

@@ -209,6 +209,15 @@ operand before starting the next:
3. **update** — apply the operand's insertions, updates, and
deletions to the database in a single transaction.
Because every operand runs its own three passes with its own totals,
`scan` announces each operand on stderr before starting it, so a
multi-operand invocation (e.g. a shell glob) is not mistaken for a
nearly-finished run:
```
scan: /srv/media (operand 3 of 14)
```
Rules for the walk:
- Only regular files. Skip directories, symlinks (do not follow,

View File

@@ -19,6 +19,11 @@
# Completed Steps
- announce each operand on stderr before its passes (2026-07-24,
branch `scan-operand-progress`): with per-operand walk/hash/update
cycles, a multi-operand run (e.g. `scan /srv/*`) showed pass totals
that looked like the whole run's — an operator watching operand 3 of
14 hash 300k files concluded 20M files were being skipped
- parallel walk (2026-07-24, branch `parallel-walk`): the walk pass
was a single goroutine and took hours at ~20M files on a busy pool
(observed: 22M files in 4h on a ZFS server); it is now a

View File

@@ -108,7 +108,13 @@ func syncScan(db *sql.DB, roots []string, workers int,
) (scanStats, error) {
var st scanStats
for _, root := range roots {
for i, root := range roots {
// Each operand runs its own walk/hash/update sequence, so
// announce it: without this, the per-operand pass totals look
// like the whole run's.
fmt.Fprintf(os.Stderr, "scan: %s (operand %d of %d)\n",
root, i+1, len(roots))
err := syncRoot(db, root, workers, oneFS, &st)
if err != nil {
return st, err