Keep the record count and total size in a totals table (closes #146)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
info counted the records and added up their sizes with a query that reads every record, which takes minutes on a large database. The schema now has a totals table of one row, kept current by three triggers on files, in the transaction of each insert, delete and size update. info, scan's load phase and the read phases of report and trees read that row instead of counting over files. scan writes files with the same statements as before. Each of those writes now fires a trigger and is slower, so scan keeps SQLite's temporary data in memory, which wins back part of that time. Model: opus-5-5
This commit was merged in pull request #148.
This commit is contained in:
@@ -26,11 +26,11 @@ func infoPhase(name string) string {
|
||||
// it. Any database problem, including a missing database, is fatal.
|
||||
//
|
||||
// The work runs in two phases, each under a progress display of its
|
||||
// own on stderr: open (open the database) and count (count the records,
|
||||
// with one query). Each display ends before the next phase starts, and
|
||||
// before an error is returned to be printed. The details are written
|
||||
// once the last display has ended, so no display is drawn while they
|
||||
// are written.
|
||||
// own on stderr: open (open the database) and count (read the number of
|
||||
// records and their total size from the totals table). Each display
|
||||
// ends before the next phase starts, and before an error is returned
|
||||
// to be printed. The details are written once the last display has
|
||||
// ended, so no display is drawn while they are written.
|
||||
func runInfo(ctx context.Context, stdout io.Writer) error {
|
||||
dbPath, source, err := databasePath()
|
||||
if err != nil {
|
||||
@@ -56,7 +56,7 @@ func runInfo(ctx context.Context, stdout io.Writer) error {
|
||||
|
||||
defer func() { _ = db.Close() }()
|
||||
|
||||
// One query counts everything, so the phase never has a total.
|
||||
// One row holds both counts, so the phase never has a total.
|
||||
prog = newProgress(progressParams{
|
||||
label: infoPhase("count"), total: -1,
|
||||
waitingFor: countingRecords,
|
||||
|
||||
Reference in New Issue
Block a user