Make phases scan-wide, walk operands concurrently, batch updates
All checks were successful
check / check (push) Successful in 5s
All checks were successful
check / check (push) Successful in 5s
All PATH operands belong to a single scan: every operand seeds the shared walk worker pool, and each pass (walk, stat, hash, update) runs exactly once over the whole scan, so pass totals, percentages, and ETAs are scan-global. The per-operand walk/hash/update cycles and their stderr operand announcements are gone; duplicate paths from overlapping operands are deduplicated before stat. The update pass now commits in batched transactions (10k changes per batch) instead of one scan-wide transaction: the filesystem is authoritative and the database is an eventually-consistent reflection of it, so scan-level atomicity buys nothing, while batches keep the WAL small and let concurrent reports observe progress.
This commit is contained in:
8
TODO.md
8
TODO.md
@@ -19,6 +19,14 @@
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- scan-wide phases, concurrent operands, batched updates (2026-07-24,
|
||||
branch `scan-wide-phases`): all operands seed the shared walk pool
|
||||
and every pass runs once over the whole scan, so totals and ETAs
|
||||
are scan-global; the per-operand walk/hash/update cycles and their
|
||||
stderr announcements are gone; the update pass commits in batched
|
||||
transactions — the filesystem is authoritative and the database an
|
||||
eventually-consistent reflection, so scan-level atomicity is not
|
||||
required
|
||||
- split the stat pass back out of the walk (2026-07-24, branch
|
||||
`parallel-phases`): phases are strictly sequential again — walk,
|
||||
stat, hash, update per operand — with parallelism only inside each
|
||||
|
||||
Reference in New Issue
Block a user