9a70afb8b7
All checks were successful
check / check (push) Successful in 3m3s
An operator running `sqlite3 <db> .dump` against their own per-webhook database wedged it: inbound webhooks rejected with HTTP 500, delivered webhooks stranded at `pending`, and every one of them POSTed a second time on the next restart while the event log recorded a single attempt. Durability. Every SQLite file — main, per-webhook, and archive — now opens through one path, `internal/database/sqlite_open.go`, in WAL journal mode with a 10-second busy timeout, `BEGIN IMMEDIATE` transactions, and a bounded connection pool. WAL is what stops a reader blocking writers at all. `_txlock=immediate` is what stops a `COMMIT` failing while its transaction stays open on a pooled connection, which is how four `database is locked` errors became 593 `cannot start a transaction within a transaction`. `cache=shared` is gone, because under it an in-process conflict is SQLITE_LOCKED, which the busy handler does not retry. The busy timeout is applied before journal_mode: the driver runs DSN pragmas in order on every new connection, and `PRAGMA journal_mode` takes a lock, so the reverse order leaves the one pragma that can block uncovered by the handler meant to cover it. Eligibility. `internal/delivery/inflight.go` holds the set of deliveries the engine owns — taken when a task is queued, when a target schedules a retry, and by every recovery path before it re-dispatches; dropped when the worker that ran the task returns. Recovery and both sweep arms re-dispatch only what the set does not hold. Nothing decides that from a row's age: a delivery waiting in a 10000-deep channel is arbitrarily old and perfectly healthy, and reasoning from age re-sends it. `takeForRedispatch` is the single gate every re-dispatch goes through — ownership first, then a conditional update confirming the row is still in the status the batch read. Bookkeeping. `recordResult` and `updateDeliveryStatus` return their errors instead of logging and dropping them, and a caller whose bookkeeping write failed writes nothing at all: the delivery keeps whichever non-terminal status it already held, and the sweeps recover it. Every recovery path — pending and retrying alike — first settles any delivery that already holds a successful `DeliveryResult` rather than sending it again. Recovery continues each delivery's own attempt numbering instead of restarting at 1. The sweep gains a `pending`-with-age-bound arm, so a stranded delivery no longer waits for a restart. Docs. WAL produces `-wal`/`-shm` sidecars, so the backup and restore procedures in README.md are corrected against measurement: both documented procedures were re-run against a live instance, a `-wal` left by a crash carries data the `.db` alone does not, and an archive file normally holds its rows in a `-wal` rather than in the `.db`.
73 lines
1.9 KiB
Go
73 lines
1.9 KiB
Go
package delivery
|
|
|
|
import (
|
|
"context"
|
|
"time"
|
|
|
|
"gorm.io/gorm"
|
|
"sneak.berlin/go/webhooker/internal/database"
|
|
)
|
|
|
|
// logTarget is a fire-and-forget target that logs the entire
|
|
// inbound webhook — the full request body and headers, plus
|
|
// the method, content type, and the webhook and entrypoint
|
|
// ids — then records a single successful attempt.
|
|
//
|
|
// This is the one log call in the service that deliberately writes
|
|
// unbounded client-chosen bytes, so it is the one exception to the
|
|
// per-field budgets in internal/logfield and to the ceiling stated on
|
|
// middleware.MaxAccessLogLineBytes. Capping here would defeat the
|
|
// target: emitting the payload IS the delivery. It costs nothing by
|
|
// default — an authenticated operator has to create a target of this
|
|
// type on a specific webhook before a single line is written — and the
|
|
// bytes it writes are bounded per event by maxWebhookBodySize (1 MB).
|
|
// An operator who adds one is choosing to spend log volume on the
|
|
// payloads that webhook receives.
|
|
type logTarget struct {
|
|
eng *Engine
|
|
}
|
|
|
|
// Deliver implements Target.
|
|
func (t *logTarget) Deliver(
|
|
_ context.Context,
|
|
webhookDB *gorm.DB,
|
|
d *database.Delivery,
|
|
_ *Task,
|
|
_ Scheduler,
|
|
) {
|
|
start := time.Now()
|
|
|
|
t.eng.log.Info(
|
|
"webhook event delivered to log target",
|
|
"delivery_id", d.ID,
|
|
"event_id", d.EventID,
|
|
"target_id", d.TargetID,
|
|
"target_name", d.Target.Name,
|
|
"webhook_id", d.Event.WebhookID,
|
|
"entrypoint_id", d.Event.EntrypointID,
|
|
"method", d.Event.Method,
|
|
"content_type", d.Event.ContentType,
|
|
"headers", d.Event.Headers,
|
|
"body", d.Event.Body,
|
|
)
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
t.eng.observeAttempt(d.Target.Type, elapsed)
|
|
|
|
err := t.eng.recordResult(
|
|
webhookDB, d, 1, true, 0, "", "",
|
|
elapsed.Milliseconds(),
|
|
)
|
|
if err != nil {
|
|
t.eng.bookkeepingFailed(d, err)
|
|
|
|
return
|
|
}
|
|
|
|
t.eng.settleStatus(
|
|
webhookDB, d, d.Target.Type,
|
|
database.DeliveryStatusDelivered,
|
|
)
|
|
}
|