Keep the record count and total size in a totals table (closes #146)
check / check (push) Waiting to run
check / check (push) Waiting to run
info counted the records and added up their sizes with a query that reads every record, which takes minutes on a large database. The schema now has a totals table of one row, kept current by three triggers on files, in the transaction of each insert, delete and size update. info, scan's load phase and the read phases of report and trees read that row instead of counting over files. scan writes files with the same statements as before. Each of those writes now fires a trigger and is slower, so scan keeps SQLite's temporary data in memory, which wins back part of that time. Model: opus-5-5
This commit is contained in:
@@ -40,11 +40,11 @@ func treesPhase(name string) string {
|
||||
// database problem, including a missing database, is fatal.
|
||||
//
|
||||
// The work runs in four phases, each under a progress display of its
|
||||
// own on stderr: open (open the database), read (count the records,
|
||||
// then read them into the hierarchy), group (group the directories by
|
||||
// digest) and write (write the groups to stdout). Each display ends
|
||||
// before the next phase starts, and before an error is returned to be
|
||||
// printed.
|
||||
// own on stderr: open (open the database), read (read the number of
|
||||
// records from the totals table, then read the records into the
|
||||
// hierarchy), group (group the directories by digest) and write (write
|
||||
// the groups to stdout). Each display ends before the next phase
|
||||
// starts, and before an error is returned to be printed.
|
||||
func runTrees(ctx context.Context, stdout io.Writer) error {
|
||||
dbPath, source, err := databasePath()
|
||||
if err != nil {
|
||||
|
||||
Reference in New Issue
Block a user