Keep the record count and total size in a totals table (closes #146)
check / check (push) Waiting to run

info counted the records and added up their sizes with a query that
reads every record, which takes minutes on a large database. The
schema now has a totals table of one row, kept current by three
triggers on files, in the transaction of each insert, delete and size
update. info, scan's load phase and the read phases of report and
trees read that row instead of counting over files. scan writes files
with the same statements as before. Each of those writes now fires a
trigger and is slower, so scan keeps SQLite's temporary data in
memory, which wins back part of that time.

Model: opus-5-5
This commit is contained in:
2026-10-27 17:38:37 +00:00
committed by sneak
parent ee09cda355
commit 61ee49a905
11 changed files with 322 additions and 83 deletions
+3 -3
View File
@@ -1228,9 +1228,9 @@ func TestScanPipeline(t *testing.T) {
// TestScanProgressNamesEachPhase checks the plain progress lines of a
// first scan, and of a rescan of one operand holding two of the
// database's records. Each phase's first line names it with its place
// among the four: load counts the records before reading them, and the
// walk estimates its total from the records under its operand, or says
// its time remaining is unknown when there are none.
// among the four: load gets the number of records before reading them,
// and the walk estimates its total from the records under its operand,
// or says its time remaining is unknown when there are none.
//
//nolint:paralleltest // captureStderr replaces the process-wide os.Stderr
func TestScanProgressNamesEachPhase(t *testing.T) {