Keep the record count and total size in a totals table (closes #146)
check / check (push) Canceled after 0s

info counted the records and added up their sizes with a query that
reads every record, which takes minutes on a large database. The
schema now has a totals table of one row, kept current by three
triggers on files, in the transaction of each insert, delete and size
update. info, scan's load phase and the read phases of report and
trees read that row instead of counting over files. scan writes files
with the same statements as before. Each of those writes now fires a
trigger and is slower, so scan keeps SQLite's temporary data in
memory, which wins back part of that time.

Model: opus-5-5
This commit was merged in pull request #148.
This commit is contained in:
2026-10-27 18:59:37 +01:00
parent ee09cda355
commit 98193b2896
11 changed files with 322 additions and 83 deletions
+6 -5
View File
@@ -180,9 +180,9 @@ func capture(t *testing.T, std **os.File) func() string {
}
// brokenDatabase writes a database that opens cleanly and passes the
// schema-version check but has no files table, so the first query
// fails with the database already open: a fatal error on a path that
// owns an open database. It closes the database the way scan does.
// schema-version check but has no tables, so the first query fails
// with the database already open: a fatal error on a path that owns an
// open database. It closes the database the way scan does.
func brokenDatabase(t *testing.T) string {
t.Helper()
@@ -265,8 +265,9 @@ func TestRunFatalAfterOpenClosesDatabase(t *testing.T) {
assertFatalOutput(t, stderr.String(), stdout())
// Proof that the failure happened after the open: only a
// query against the opened database can report this.
if !strings.Contains(stderr.String(), "no such table: files") {
// query against the opened database can report this. Each
// subcommand's first query reads totals.
if !strings.Contains(stderr.String(), "no such table: totals") {
t.Errorf("stderr = %q, want the failure to come from a "+
"query on the open database", stderr.String())
}