The local index ignores its WAL and busy-timeout settings, so a read-only command can abort a running backup #217

Closed
opened 2026-10-06 01:49:41 +02:00 by clawbot · 1 comment
Collaborator

database.New opens the index with ?_journal_mode=WAL&_synchronous=NORMAL&_busy_timeout=10000&_locking_mode=NORMAL&_foreign_keys=ON (internal/database/database.go:113-117; the recovery path at :179-183 uses the same string). The modernc.org/sqlite driver (v1.38.0) honours only _pragma=, _time_format, _txlock and vfs, and drops every other parameter without an error. Measured on an index opened by database.New on next at 0700901: journal_mode=delete and busy_timeout=0. Foreign keys are on only because finishOpen issues an explicit PRAGMA.

With a reader holding a statement open on one handle, a write on another handle fails at once with database is locked (5) (SQLITE_BUSY).

Trigger: run vaultik snapshot list or vaultik info while snapshot create is running. The README (:192-197) says these commands are never blocked and run even while a backup is in progress. In fact either side can fail, and a failed write aborts the backup.

There is a trap in the fix. Once the index is in WAL mode, the metadata export's file copy (internal/snapshot/snapshot.go:388-397) copies only the main database file. That copy misses every commit not yet checkpointed, including the snapshot_blobs rows written just before the export. openWithRecovery already sets WAL mode persistently (database.go:210), so any index that ever went through recovery is exposed today.

Definition of done

  1. The connection settings are passed in the driver's _pragma= form, at minimum journal_mode(WAL), busy_timeout(10000), synchronous(NORMAL) and foreign_keys(1), on both open paths.
  2. The export checkpoints the WAL (or copies through SQLite) before copying, so the exported database holds every committed row.
  3. One test asserts PRAGMA journal_mode and PRAGMA busy_timeout on a database.New handle. Another holds a read open on one handle while a second handle commits a write, and asserts that the write succeeds.
  4. The database.New comment, the vacuumDatabase comment and docs/DATAMODEL.md:282 describe what the code now does.
  5. make check passes.

Model: fable-5-1 (audit); opus-5-5 (issue)

`database.New` opens the index with `?_journal_mode=WAL&_synchronous=NORMAL&_busy_timeout=10000&_locking_mode=NORMAL&_foreign_keys=ON` (`internal/database/database.go:113-117`; the recovery path at `:179-183` uses the same string). The `modernc.org/sqlite` driver (v1.38.0) honours only `_pragma=`, `_time_format`, `_txlock` and `vfs`, and drops every other parameter without an error. Measured on an index opened by `database.New` on `next` at `0700901`: `journal_mode=delete` and `busy_timeout=0`. Foreign keys are on only because `finishOpen` issues an explicit `PRAGMA`. With a reader holding a statement open on one handle, a write on another handle fails at once with `database is locked (5) (SQLITE_BUSY)`. Trigger: run `vaultik snapshot list` or `vaultik info` while `snapshot create` is running. The README (`:192-197`) says these commands are never blocked and run even while a backup is in progress. In fact either side can fail, and a failed write aborts the backup. There is a trap in the fix. Once the index is in WAL mode, the metadata export's file copy (`internal/snapshot/snapshot.go:388-397`) copies only the main database file. That copy misses every commit not yet checkpointed, including the `snapshot_blobs` rows written just before the export. `openWithRecovery` already sets WAL mode persistently (`database.go:210`), so any index that ever went through recovery is exposed today. ## Definition of done 1. The connection settings are passed in the driver's `_pragma=` form, at minimum `journal_mode(WAL)`, `busy_timeout(10000)`, `synchronous(NORMAL)` and `foreign_keys(1)`, on both open paths. 2. The export checkpoints the WAL (or copies through SQLite) before copying, so the exported database holds every committed row. 3. One test asserts `PRAGMA journal_mode` and `PRAGMA busy_timeout` on a `database.New` handle. Another holds a read open on one handle while a second handle commits a write, and asserts that the write succeeds. 4. The `database.New` comment, the `vacuumDatabase` comment and `docs/DATAMODEL.md:282` describe what the code now does. 5. `make check` passes. Model: fable-5-1 (audit); opus-5-5 (issue)
clawbot self-assigned this 2026-10-06 01:49:41 +02:00
Author
Collaborator

Fixed in #242. Reconfirmed on next at 4a167e1 before the fix: journal_mode read back delete, busy_timeout read back 0, and a write beside an open read on another handle failed with SQLITE_BUSY. The export trap is real too: with WAL on and the old file copy, the exported database had no snapshot row and no restore worked. The export now copies the open index with VACUUM INTO.

Model: opus-5-5

Fixed in https://git.eeqj.de/sneak/vaultik/pulls/242. Reconfirmed on `next` at `4a167e1` before the fix: `journal_mode` read back `delete`, `busy_timeout` read back 0, and a write beside an open read on another handle failed with `SQLITE_BUSY`. The export trap is real too: with WAL on and the old file copy, the exported database had no snapshot row and no restore worked. The export now copies the open index with `VACUUM INTO`. Model: opus-5-5
Sign in to join this conversation.