Wait on a busy SQLite database and turn on WAL mode (closes #198) #200

Merged
clawbot merged 2 commits from issue-198-sqlite-busy-timeout-wal into next 2026-10-04 20:58:37 +02:00
Collaborator

SQLite writes were lost under load with "database is locked". Requests and the eviction pass write on separate connections, nothing set a busy timeout, and the default db_url's _journal_mode=WAL is not a parameter modernc.org/sqlite reads, so the database was never in WAL mode.

  • internal/database adds _pragma=busy_timeout(5000) to whatever db_url it opens, the default or one the operator sets, so a write that finds another in progress waits up to five seconds for it. The driver runs each _pragma on every connection it opens.
  • The default db_url is now state.sqlite3 in the state directory with ?_pragma=journal_mode(WAL).
  • README.md and config.example.yml say what pixa adds to db_url, and that WAL mode comes only from the URL.
  • The first commit holds the tests alone, so they can be seen failing on next: writes like one request's and the eviction pass's, made from several goroutines through the code pixad opens the database with, must all succeed for a db_url with and without parameters; and the default db_url must open the database in WAL mode.

Disclosures:

  • Judgement call, per the plan: WAL is set only in the default db_url, not forced on one the operator sets; the busy timeout goes on every db_url.
  • A busy_timeout the operator also puts in db_url is not detected; both are passed to the driver.
  • The expected default in the existing TestOmittedValuesUseDefaults changes with the default.

This lands before #195, whose integration test exposed the defect.

Closes #198

Model: opus-5-5

SQLite writes were lost under load with "database is locked". Requests and the eviction pass write on separate connections, nothing set a busy timeout, and the default `db_url`'s `_journal_mode=WAL` is not a parameter `modernc.org/sqlite` reads, so the database was never in WAL mode. - `internal/database` adds `_pragma=busy_timeout(5000)` to whatever `db_url` it opens, the default or one the operator sets, so a write that finds another in progress waits up to five seconds for it. The driver runs each `_pragma` on every connection it opens. - The default `db_url` is now `state.sqlite3` in the state directory with `?_pragma=journal_mode(WAL)`. - `README.md` and `config.example.yml` say what pixa adds to `db_url`, and that WAL mode comes only from the URL. - The first commit holds the tests alone, so they can be seen failing on `next`: writes like one request's and the eviction pass's, made from several goroutines through the code pixad opens the database with, must all succeed for a `db_url` with and without parameters; and the default `db_url` must open the database in WAL mode. Disclosures: - Judgement call, per the plan: WAL is set only in the default `db_url`, not forced on one the operator sets; the busy timeout goes on every `db_url`. - A `busy_timeout` the operator also puts in `db_url` is not detected; both are passed to the driver. - The expected default in the existing `TestOmittedValuesUseDefaults` changes with the default. This lands before https://git.eeqj.de/sneak/pixa/pulls/195, whose integration test exposed the defect. Closes https://git.eeqj.de/sneak/pixa/issues/198 Model: opus-5-5
clawbot added the needs-review label 2026-10-04 20:38:18 +02:00
clawbot self-assigned this 2026-10-04 20:38:18 +02:00
clawbot added 2 commits 2026-10-04 20:38:18 +02:00
Adds two tests that fail before the fix. One opens the database the way
pixad does and makes writes like one request's and the eviction pass's
from several goroutines at once; on separate connections with no busy
timeout, some fail with "database is locked". The other opens the db_url
derived from state_dir and checks the database is in WAL mode, which the
current default's _journal_mode=WAL does not do because the driver
ignores that parameter. The expected default db_url in
TestOmittedValuesUseDefaults changes to the new one.

Model: opus-5-5
Requests and the eviction pass write on separate connections, and with
no busy timeout a write that met another one failed at once with
"database is locked" and was lost. internal/database now adds
_pragma=busy_timeout(5000) to every db_url, the default or one the
operator sets, so such a write waits up to five seconds. The default
db_url's _journal_mode=WAL is not a parameter the driver reads, so it
is now _pragma=journal_mode(WAL). README.md and config.example.yml say
what pixa adds to db_url.

Model: opus-5-5
Author
Collaborator

PASS c33713c85f1093dab05b70af8fef30d5860cbd5a, rebased onto next at 66e71b42072139c39c6c78dbf38a606fad3a227c.

Model: opus-5-5

**PASS** `c33713c85f1093dab05b70af8fef30d5860cbd5a`, rebased onto `next` at `66e71b42072139c39c6c78dbf38a606fad3a227c`. Model: opus-5-5
clawbot merged commit 625fd42ace into next 2026-10-04 20:58:37 +02:00
clawbot deleted branch issue-198-sqlite-busy-timeout-wal 2026-10-04 20:58:37 +02:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/pixa#200