Queue SQLite writes on one connection so none fails as locked (closes #223)
check / check (push) Waiting to run
check / check (push) Waiting to run
internal/database now opens the database with one connection. pixa's own reads and writes run on it one at a time instead of competing for SQLite's lock, where a write that kept losing could wait past the five-second busy timeout and fail with "database is locked". The busy timeout stays, for another program writing to the same file. With one connection, a query run while rows or a transaction are still open would wait forever. No code in internal/database or internal/imgcache does that; the comment on DB() tells callers. README.md says pixa uses one connection and that requests wait while eviction runs one of its queries. Model: opus-5-5
This commit was merged in pull request #225.
This commit is contained in:
@@ -13,11 +13,10 @@ import (
|
||||
)
|
||||
|
||||
// TestConcurrentWritesAllSucceed opens a database the way pixad does and
|
||||
// writes to it from several goroutines at once, so the writes run on
|
||||
// separate connections, as one request's writes and the background eviction
|
||||
// pass do. Every write must succeed, none failing with "database is locked",
|
||||
// whether or not db_url already has parameters, and the parameters it has
|
||||
// must still apply.
|
||||
// writes to it from several goroutines at once, as one request's writes and
|
||||
// the background eviction pass do. Every write must succeed, none failing
|
||||
// with "database is locked", whether or not db_url already has parameters,
|
||||
// and the parameters it has must still apply.
|
||||
func TestConcurrentWritesAllSucceed(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -237,17 +237,18 @@ func ApplyMigrations(ctx context.Context, db *sql.DB, log *slog.Logger) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// DB returns the underlying sql.DB.
|
||||
// DB returns the underlying sql.DB. It has one connection, so close any
|
||||
// rows and end any transaction before running another query on it; a
|
||||
// query run while they are open waits forever.
|
||||
func (s *Database) DB() *sql.DB {
|
||||
return s.db
|
||||
}
|
||||
|
||||
func (s *Database) connect(ctx context.Context) error {
|
||||
// Requests and the eviction pass write on separate connections. With
|
||||
// a busy timeout, a write that finds another one in progress waits up
|
||||
// to five seconds for it instead of failing at once with "database is
|
||||
// locked". The driver runs each _pragma parameter on every connection
|
||||
// it opens.
|
||||
// With a busy timeout, a write that finds another program writing to
|
||||
// the same database file waits up to five seconds for it instead of
|
||||
// failing at once with "database is locked". The driver runs each
|
||||
// _pragma parameter on every connection it opens.
|
||||
separator := "?"
|
||||
if strings.Contains(s.config.DBURL, "?") {
|
||||
separator = "&"
|
||||
@@ -264,6 +265,11 @@ func (s *Database) connect(ctx context.Context) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// One connection: pixa's own reads and writes run on it one at a
|
||||
// time instead of competing for SQLite's lock, where a write that
|
||||
// keeps losing can wait past the busy timeout and be lost.
|
||||
db.SetMaxOpenConns(1)
|
||||
|
||||
err = db.PingContext(ctx)
|
||||
if err != nil {
|
||||
s.log.Error("failed to ping database", "error", err)
|
||||
|
||||
Reference in New Issue
Block a user