Stream report and trees instead of loading every record (closes #14)
check / check (push) Successful in 1m46s
check / check (push) Successful in 1m46s
report now has SQLite group the records and put the rows in report order, helped by a new files_signature index on (size, head, tail, content), and writes each row as it reads it. trees reads the records in path order, where all the paths under a directory come together, so it computes each directory's digest as soon as the stream leaves it and keeps only its path, parent, digest and totals. Output is unchanged. The tests that called the removed in-memory grouping functions now group records stored in a database. New tests check that both commands give the same output whatever order the records were inserted in, and that a stdout failure partway through a long report is reported as one. Model: opus-5-5
This commit was merged in pull request #76.
This commit is contained in:
@@ -29,6 +29,10 @@
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- `report` and `trees` stream the records instead of holding them all in
|
||||
memory; the schema gains the `files_signature` index (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/14)
|
||||
|
||||
- progress prints at once on a non-terminal, uses a real terminal test,
|
||||
and prints warnings through a spinner instead of racing its redraw
|
||||
(2026-10-03, https://git.eeqj.de/sneak/sfdupes/issues/13)
|
||||
|
||||
Reference in New Issue
Block a user