Queue SQLite writes on one connection so none fails as locked (closes #223)
check / check (push) Waiting to run
check / check (push) Waiting to run
internal/database now opens the database with one connection. pixa's own reads and writes wait for it in turn instead of competing for SQLite's lock, where a write that kept losing could wait past the five-second busy timeout and fail with "database is locked". The busy timeout stays, for another program writing to the same file. With one connection, a query run while rows or a transaction are still open would wait forever. No code in internal/database or internal/imgcache does that; the comment on DB() tells callers. README.md says pixa uses one connection. Model: opus-5-5
This commit is contained in:
@@ -529,10 +529,11 @@ Key settings in more detail:
|
||||
- `signing_key` — HMAC secret for URL signatures
|
||||
- `db_url` — the SQLite database to open; omitted, it is
|
||||
`file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the
|
||||
database in WAL mode. pixa adds `_pragma=busy_timeout(5000)` to any `db_url`,
|
||||
so a write that finds another in progress waits up to five seconds for it
|
||||
instead of failing. WAL mode comes only from the URL: keep
|
||||
`_pragma=journal_mode(WAL)` in one you set
|
||||
database in WAL mode. pixa opens one connection to it, so its own reads and
|
||||
writes run one at a time. It adds `_pragma=busy_timeout(5000)` to any
|
||||
`db_url`, so a write that finds another program writing to the file waits up
|
||||
to five seconds for it instead of failing. WAL mode comes only from the URL:
|
||||
keep `_pragma=journal_mode(WAL)` in one you set
|
||||
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the disk
|
||||
cache entirely; omitted defaults to 75% of the sum of the free space on the
|
||||
filesystem containing `<state_dir>/cache/` and the bytes of source and
|
||||
|
||||
@@ -30,6 +30,13 @@ P2: security: per-IP rate limiting on the image routes
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-08 SQLite writes no longer fail with "database is locked" under load
|
||||
(closes #223): `internal/database` opens the database with one connection, so
|
||||
pixa's own reads and writes wait for it in turn instead of competing for
|
||||
SQLite's lock, where a write that kept losing could wait past the five-second
|
||||
busy timeout and be lost. The busy timeout stays, for another program writing
|
||||
to the same file. No code in pixa keeps rows or a transaction open while it
|
||||
runs another query, which with one connection would wait forever.
|
||||
- 2026-10-05 the format `auto` (closes #88): a format in the `/v1/image/` path,
|
||||
an encrypted URL's token and the generator page's format choice, chosen for
|
||||
each request from `Accept` once the signature or token is checked: AVIF when
|
||||
|
||||
@@ -237,17 +237,18 @@ func ApplyMigrations(ctx context.Context, db *sql.DB, log *slog.Logger) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// DB returns the underlying sql.DB.
|
||||
// DB returns the underlying sql.DB. It has one connection, so close any
|
||||
// rows and end any transaction before running another query on it; a
|
||||
// query run while they are open waits forever.
|
||||
func (s *Database) DB() *sql.DB {
|
||||
return s.db
|
||||
}
|
||||
|
||||
func (s *Database) connect(ctx context.Context) error {
|
||||
// Requests and the eviction pass write on separate connections. With
|
||||
// a busy timeout, a write that finds another one in progress waits up
|
||||
// to five seconds for it instead of failing at once with "database is
|
||||
// locked". The driver runs each _pragma parameter on every connection
|
||||
// it opens.
|
||||
// With a busy timeout, a write that finds another program writing to
|
||||
// the same database file waits up to five seconds for it instead of
|
||||
// failing at once with "database is locked". The driver runs each
|
||||
// _pragma parameter on every connection it opens.
|
||||
separator := "?"
|
||||
if strings.Contains(s.config.DBURL, "?") {
|
||||
separator = "&"
|
||||
@@ -264,6 +265,11 @@ func (s *Database) connect(ctx context.Context) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// One connection: pixa's own reads and writes wait for it in turn
|
||||
// instead of competing for SQLite's lock, where a write that keeps
|
||||
// losing can wait past the busy timeout and be lost.
|
||||
db.SetMaxOpenConns(1)
|
||||
|
||||
err = db.PingContext(ctx)
|
||||
if err != nil {
|
||||
s.log.Error("failed to ping database", "error", err)
|
||||
|
||||
Reference in New Issue
Block a user