Keep the record count and total size in a totals table (closes #146)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
info counted the records and added up their sizes with a query that reads every record, which takes minutes on a large database. The schema now has a totals table of one row, kept current by three triggers on files, in the transaction of each insert, delete and size update. info, scan's load phase and the read phases of report and trees read that row instead of counting over files. scan writes files with the same statements as before. Each of those writes now fires a trigger and is slower, so scan keeps SQLite's temporary data in memory, which wins back part of that time. Model: opus-5-5
This commit was merged in pull request #148.
This commit is contained in:
@@ -29,6 +29,14 @@
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- the database keeps the number of records and the sum of their sizes in the one
|
||||
row of a `totals` table, which three triggers on `files` keep current. `info`
|
||||
reads that row, and so do `scan`'s `load` phase and the `read` phases of
|
||||
`report` and `trees` for their totals; no count reads every record any more.
|
||||
The triggers make each write to `files` slower. `scan` keeps SQLite's
|
||||
temporary data in memory, which wins back part of that time (2026-10-27,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/146)
|
||||
|
||||
- an operand at or inside a directory named `.zfs` is never scanned, whether
|
||||
given alone or under another operand. `scan` warns about it, counts it as
|
||||
skipped and does not walk it (2026-10-26,
|
||||
|
||||
Reference in New Issue
Block a user