Make SQLite durable under concurrent readers and stop re-delivering stranded webhooks (closes #256)

An operator running `sqlite3 <db> .dump` against their own per-webhook
database wedged it: 60 of 60 inbound webhooks rejected with HTTP 500,
206 delivered webhooks stranded at `pending`, and every one of them
POSTed a second time on the next restart while the event log recorded a
single attempt.

Durability. Every SQLite file — main, per-webhook, and archive — now
opens through one path, `internal/database/sqlite_open.go`, in WAL
journal mode with a 10-second busy timeout, `BEGIN IMMEDIATE`
transactions, and a bounded connection pool. WAL is what stops a
reader blocking writers at all. `_txlock=immediate` is what stops a
`COMMIT` failing while its transaction stays open on a pooled
connection, which is how four `database is locked` errors became 593
`cannot start a transaction within a transaction`: a deferred
transaction that upgrades to a write lock mid-flight gets SQLITE_BUSY
without the busy handler being consulted. `cache=shared` is gone,
because under it an in-process conflict is SQLITE_LOCKED, which the
busy handler does not retry.

Delivery. `recordResult` and `updateDeliveryStatus` return their
errors instead of logging and dropping them, and a caller whose
bookkeeping write failed writes nothing at all — the delivery keeps
whichever non-terminal status it already held, and both sweeps recover
it. Recovery and the sweep now reconcile before re-sending: a pending
delivery that already holds a successful `DeliveryResult` is marked
delivered rather than sent again, which is the state that did not
previously exist. A delivery handed back out is claimed by
compare-and-set so successive sweeps cannot send it repeatedly, and it
continues its own attempt numbering instead of restarting at 1. The
sweep gains a `pending`-with-age-bound arm, so a stranded delivery no
longer waits for a restart.

Docs. WAL produces `-wal`/`-shm` sidecars, so the backup and restore
procedures in README.md are corrected: both documented procedures were
re-run against a live instance, and a `-wal` left by a crash carries
data the `.db` alone does not.

Verified by reproducing the failure on unmodified `next` first — 6
targets, 60 events at 5/s, a concurrent `.dump` reader — which gave 38
HTTP 500s and 112 duplicate POSTs at the sinks across a restart. Both
arms of the matched pair now show 0 inbound 500s, 0 engine write
errors, and 0 new requests at the sinks after a restart, counted by
payload.

README.md

@@ -566,9 +566,18 @@ is both the simplest and the only complete rule:
 `events-3f2a1c9e-....db`. The only other file is `webhooker.lock`, the
 always-empty [single-instance lock](#single-instance-lock); it holds no
 state and is not part of the backup set — a copied one is stale and
-blocks nothing. No `-wal` or `-shm` files are produced (see below); a
-transient `{name}.db-journal` may exist beside a database while a write
-is in flight and is not part of the backup set either.
+blocks nothing.
+
+**`-wal` and `-shm` sidecars.** Every database runs in WAL journal mode,
+so while the service is running each `{name}.db` has a `{name}.db-wal`
+and a `{name}.db-shm` beside it. **`-wal` is part of the database, not a
+scratch file**: it holds committed transactions that are not yet in the
+`.db`, so a copy of the `.db` without its `-wal` is missing data and may
+have no readable schema at all. A clean shutdown checkpoints and removes
+both sidecars, so a stopped deployment has none; a killed or crashed one
+leaves them, and they must be carried with the `.db`. `-shm` is
+regenerable, but there is no reason to separate the two — copy the
+directory.
 
 Configuration is **not** in `DATA_DIR` — it comes from the environment
 and from a `.env` file read out of the process working directory. Back
@@ -576,17 +585,19 @@ that up with your deployment config, separately.
 
 ### A hot copy is not safe
 
-No `journal_mode` pragma is ever issued on any database webhooker opens,
-so all of them run on SQLite's default rollback journal. There is no
-WAL. The main and event databases are also held open for the entire
-process lifetime — `WebhookDBManager` caches event database handles and
-closes them only on webhook deletion or shutdown — so "it looked idle"
-is not a guarantee that nothing was mid-transaction.
+Every database webhooker opens runs in WAL journal mode. The main and
+event databases are also held open for the entire process lifetime —
+`WebhookDBManager` caches event database handles and closes them only on
+webhook deletion or shutdown — so "it looked idle" is not a guarantee
+that nothing was mid-transaction.
 
 That means `cp`, `rsync`, `tar` or a filesystem snapshot taken against a
-running instance can capture a database mid-transaction and yield a file
-that is corrupt or missing state the journal would have rolled back. Use
-one of the two procedures below instead.
+running instance can capture a database and its `-wal` at two different
+instants and yield a file that is corrupt or missing state. Copying a
+`.db` on its own is worse and fails loudly: recently written pages,
+including the schema itself on a young database, live in the `-wal`, so
+the copy reads back as an empty or table-less database. Use one of the
+two procedures below instead.
 
 **Stop, copy, start.** The simplest, needs no extra tooling, and the
 only one that gives a single point in time across every file:
@@ -605,8 +616,9 @@ for db in /path/to/data/*.db; do
 done
 ```
 
-`.backup` takes the proper locks and writes a consistent file. Two
-caveats. First, the runtime image is `alpine:3.21` with only
+`.backup` reads through the WAL and writes a single consistent file with
+no sidecars of its own, so the destination is complete as it stands.
+Two caveats. First, the runtime image is `alpine:3.21` with only
 `ca-certificates` added — the `sqlite3` CLI is **not** in it, so run
 this on the host against the volume path, or from a throwaway container
 that mounts the volume. Second, each file is captured at its own
@@ -614,12 +626,21 @@ instant, so a webhook created or an event delivered between two files
 being copied lands in one and not the other. If you need the whole set
 coherent as of a single moment, stop the service.
 
+Note that `sqlite3 <db> .dump` is **not** one of these procedures: it is
+an export, it holds a read transaction open for as long as it runs, and
+it pins the WAL against checkpointing for that whole time. It is safe to
+run — it does not block ingestion — but back up with `.backup` or a
+stopped copy.
+
 Archive databases are the one exception the service is built for: the
 archive writer closes its handle after each write (debounced to at most
 one reopen per second), so an operator can move `archive-{uuid}.db`
 away for offline retention while the service runs, and it is recreated
-on the next write (see
-[Database Architecture](#database-architecture)). That is a
+on the next write. Closing the handle checkpoints and removes that
+file's sidecars, so what is left to move is a single self-contained
+`.db` — but if a `-wal` is there, a write is in flight, and it has to
+move with it. See
+[Database Architecture](#database-architecture). That is a
 move-the-file-away workflow, not a substitute for the backup procedures
 above.
 
@@ -636,14 +657,18 @@ above.
    restored without `webhooker.db` are simply orphaned; nothing
    references their UUIDs.
 
-3. Do not carry `*.db-journal` files into the restore. Backups taken by
-   either procedure above are self-consistent and do not need one.
+3. Carry any `*.db-wal` and `*.db-shm` files that are in the backup.
+   They are part of the database, and dropping a `-wal` silently
+   discards every transaction it still holds. Backups taken by either
+   procedure above will not contain them — `.backup` writes a single
+   consolidated file, and a clean stop checkpoints the sidecars away —
+   but a copy salvaged from a crashed instance will, and it needs them.
 
 4. **Fix ownership.** The container runs as the non-root `webhooker`
    user, UID 1000 / GID 1000. Restored files must be owned by (or
    writable by) that UID, and so must the directory itself — SQLite
-   creates the rollback journal beside the database, so a writable file
-   inside a directory it cannot write is not enough:
+   creates the `-wal` and `-shm` sidecars beside the database, so a
+   writable file inside a directory it cannot write is not enough:
 
     ```bash
     chown -R 1000:1000 /path/to/data
@@ -1391,10 +1416,12 @@ This separation provides:
   only, or disables cleanup entirely when set to `0` (retain forever).
 - **Performance** — each webhook's database has its own page cache and
   its own lock, so concurrent event ingestion across webhooks won't
-  contend. No write-ahead log is involved: both DSNs are
-  `file:{path}?cache=shared&mode=rwc` and no `journal_mode` pragma is
-  ever issued, so every database runs on SQLite's default rollback
-  journal.
+  contend. Every database — main, per-webhook, and archive — is opened
+  through one code path (`internal/database/sqlite_open.go`) in WAL
+  journal mode, with a 10-second busy timeout, `BEGIN IMMEDIATE`
+  transactions, and a bounded connection pool. Under WAL a reader never
+  blocks a writer, so an operator reading a database does not stall
+  event ingestion into it.
 
 The **database target type** builds on this architecture to provide
 long-term archiving, separate from the per-webhook event database (which