Create new databases with auto_vacuum incremental (closes #43)
check / check (push) Successful in 2m50s
check / check (push) Successful in 2m50s
SQLite only accepts auto_vacuum before the database file is first written. The connection switched to WAL first, which writes the file, so the PRAGMA auto_vacuum in Initialize came too late and was ignored. The setting now goes in the connection string, which the driver applies on open before the journal mode, and the late PRAGMA is removed. Vacuum now reads every row PRAGMA incremental_vacuum returns: SQLite frees one page per row, and the single step ExecContext took freed only one page per call. Tests check that every pooled connection sees auto_vacuum incremental on a new database and that one Vacuum call frees every page left by deleting routes. Model: opus-5-5
This commit was merged in pull request #49.
This commit is contained in:
@@ -91,8 +91,13 @@ func New(cfg *config.Config, logger *logger.Logger) (*Database, error) {
|
||||
// a transaction that reads before writing starts as a reader and, when it
|
||||
// then writes while another connection holds the write lock, fails at once
|
||||
// with "database is locked" without waiting for _busy_timeout.
|
||||
// _auto_vacuum=incremental lets Vacuum return free pages to the filesystem.
|
||||
// SQLite only accepts it before the database file is first written, and the
|
||||
// switch to WAL writes it, so it must be here: the driver applies it on open,
|
||||
// before _journal_mode. On an existing file it changes nothing.
|
||||
dsn := fmt.Sprintf(
|
||||
"file:%s?_cache_size=%d&_synchronous=OFF&_busy_timeout=%d&_journal_mode=WAL&_txlock=immediate",
|
||||
"file:%s?_cache_size=%d&_synchronous=OFF&_busy_timeout=%d"+
|
||||
"&_auto_vacuum=incremental&_journal_mode=WAL&_txlock=immediate",
|
||||
dbPath,
|
||||
sqliteCacheSizeKiB,
|
||||
sqliteBusyTimeoutMs,
|
||||
@@ -137,9 +142,8 @@ func (d *Database) Initialize() error {
|
||||
// heap. The heap limits below are process-wide, so setting them once here is
|
||||
// enough for the whole pool.
|
||||
pragmas := []string{
|
||||
"PRAGMA journal_mode=WAL", // Write-Ahead Logging
|
||||
"PRAGMA analysis_limit=0", // Disable automatic ANALYZE
|
||||
"PRAGMA auto_vacuum=INCREMENTAL", // Enable incremental vacuum
|
||||
"PRAGMA journal_mode=WAL", // Write-Ahead Logging
|
||||
"PRAGMA analysis_limit=0", // Disable automatic ANALYZE
|
||||
fmt.Sprintf("PRAGMA soft_heap_limit=%d", sqliteSoftHeapLimitBytes),
|
||||
fmt.Sprintf("PRAGMA hard_heap_limit=%d", sqliteHardHeapLimitBytes),
|
||||
}
|
||||
@@ -1857,10 +1861,20 @@ func (d *Database) Vacuum(ctx context.Context) error {
|
||||
// Free up to 1000 pages per call (~4MB with default 4KB page size)
|
||||
// This keeps each vacuum operation quick and non-blocking
|
||||
const pagesToFree = 1000
|
||||
_, err := d.db.ExecContext(ctx, fmt.Sprintf("PRAGMA incremental_vacuum(%d)", pagesToFree))
|
||||
rows, err := d.db.QueryContext(ctx, fmt.Sprintf("PRAGMA incremental_vacuum(%d)", pagesToFree))
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to run incremental vacuum: %w", err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
// SQLite frees one page each time the statement steps, and each step
|
||||
// returns a row, so every row must be read for the pragma to free more
|
||||
// than one page.
|
||||
for rows.Next() {
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return fmt.Errorf("failed to run incremental vacuum: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user