Vacuum, Analyze and Checkpoint now take Database.lock, as every batch write does, so maintenance and batch writes wait for each other instead of one failing with "database is locked" after SQLite's 5-second busy timeout (#94).
ANALYZE on 4 million routes (a 1.7 GB file, in the page cache) took about 4 s with no limit and 1.1 s with analysis_limit=1000. Even limited, it grows with the file: SQLite counts every index entry. So Analyze runs PRAGMA optimize=0x10002 with that limit, which analyzes only tables without statistics or whose size changed tenfold: about 1 ms on that file when nothing needed it, 1 to 2 s on the first run.
Not visible in the diff:
A wait on a sync.Mutex cannot be abandoned: a maintenance call whose context ends while it waits returns the context's error once it gets the lock, and releases it at once.
Checkpoints every 5 seconds cannot starve batch writes: Go's mutex serves goroutines waiting over 1 ms in arrival order. TestBatchWriteDuringCheckpoint now covers this.
The tests learn that an operation waits for or holds the lock from the lock's debug messages.
Disclosures:
Judgement call: PRAGMA optimize instead of an hourly ANALYZE of every table.
Removed PRAGMA analysis_limit=0 from Initialize: it set no limit.
Timings from the sqlite3 shell 3.53.3; the driver bundles 3.50.3.
Model: opus-5-5
`Vacuum`, `Analyze` and `Checkpoint` now take `Database.lock`, as every batch write does, so maintenance and batch writes wait for each other instead of one failing with "database is locked" after SQLite's 5-second busy timeout (https://git.eeqj.de/sneak/routewatch/issues/94).
ANALYZE on 4 million routes (a 1.7 GB file, in the page cache) took about 4 s with no limit and 1.1 s with `analysis_limit=1000`. Even limited, it grows with the file: SQLite counts every index entry. So `Analyze` runs `PRAGMA optimize=0x10002` with that limit, which analyzes only tables without statistics or whose size changed tenfold: about 1 ms on that file when nothing needed it, 1 to 2 s on the first run.
Not visible in the diff:
- A wait on a `sync.Mutex` cannot be abandoned: a maintenance call whose context ends while it waits returns the context's error once it gets the lock, and releases it at once.
- Checkpoints every 5 seconds cannot starve batch writes: Go's mutex serves goroutines waiting over 1 ms in arrival order. `TestBatchWriteDuringCheckpoint` now covers this.
- The tests learn that an operation waits for or holds the lock from the lock's debug messages.
Disclosures:
- Judgement call: `PRAGMA optimize` instead of an hourly ANALYZE of every table.
- Removed `PRAGMA analysis_limit=0` from `Initialize`: it set no limit.
- Timings from the `sqlite3` shell 3.53.3; the driver bundles 3.50.3.
Model: opus-5-5
Vacuum, Analyze and Checkpoint now take the database lock the batch
writes hold, so maintenance and batch writes wait for each other
instead of meeting at SQLite's write lock, where whichever waited past
the 5-second busy timeout failed with "database is locked". Analyze
runs ANALYZE through PRAGMA optimize, only on tables without statistics
or whose size changed tenfold, reading about 1000 rows of each index:
even limited, ANALYZE counts every index entry, so an hourly ANALYZE of
every table would hold the lock longer as the tables grow. The
analysis_limit=0 in Initialize, which set no limit, is gone.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Vacuum,AnalyzeandCheckpointnow takeDatabase.lock, as every batch write does, so maintenance and batch writes wait for each other instead of one failing with "database is locked" after SQLite's 5-second busy timeout (#94).ANALYZE on 4 million routes (a 1.7 GB file, in the page cache) took about 4 s with no limit and 1.1 s with
analysis_limit=1000. Even limited, it grows with the file: SQLite counts every index entry. SoAnalyzerunsPRAGMA optimize=0x10002with that limit, which analyzes only tables without statistics or whose size changed tenfold: about 1 ms on that file when nothing needed it, 1 to 2 s on the first run.Not visible in the diff:
sync.Mutexcannot be abandoned: a maintenance call whose context ends while it waits returns the context's error once it gets the lock, and releases it at once.TestBatchWriteDuringCheckpointnow covers this.Disclosures:
PRAGMA optimizeinstead of an hourly ANALYZE of every table.PRAGMA analysis_limit=0fromInitialize: it set no limit.sqlite3shell 3.53.3; the driver bundles 3.50.3.Model: opus-5-5
Review passed.
Model: opus-5-5