Store events and create deliveries in the delivery engine (closes #544)
check / check (push) Canceled after 0s

The receiver, resubmit and replay handlers now parse the request and
make one call. Engine.StoreEvent stores an event with a pending
delivery for each active target and the totals in one transaction,
then queues them; Engine.CreateReplay does the same for one replay
delivery. The handlers take *delivery.Engine, so the Notifier
interface and its bridge are gone. Archives and CircuitBreakers stay
because the handler tests fake them. The tests that read a queued
task moved to the engine's tests; the handler tests use an engine
that is never started.

Model: opus-5-5
This commit is contained in:
2026-10-15 21:32:25 +00:00
parent 8406c77949
commit 60e787d7a5
15 changed files with 815 additions and 702 deletions
+30 -17
View File
@@ -2153,12 +2153,8 @@ External Service
1. Look up Entrypoint by UUID — 404 if unknown,
410 if inactive
2. Read the body under the 1 MB cap
3. Capture full request as Event
4. Create Delivery records for each active Target
5. Build self-contained delivery.Task structs
(target settings + event data inline for
bodies < 16 KiB)
6. Notify Engine via channel (no DB read needed)
3. Hand the full request, as an Event, to
Engine.StoreEvent
│
▼
┌──────────────┐
@@ -2167,6 +2163,13 @@ External Service
│ (worker │
│ pool) │
└──────┬───────┘
4. Store the Event, a pending Delivery for each
active Target, and the totals, in one
transaction
5. Build a delivery.Task for each Delivery
(target settings + event data, the body
inline when it is under 16 KiB)
6. Queue the tasks on the delivery channel
│
┌── bounded worker pool (N workers) ──┐
▼ ▼ ▼
@@ -2180,6 +2183,12 @@ External Service
└────────────┘
```
A **Resubmit** stores its copy of an event through the same `Engine.StoreEvent`,
and a **Replay** adds one delivery of a stored event through
`Engine.CreateReplay`, which writes the delivery and its target's totals in one
transaction and queues it. Those two methods are the only code that creates a
delivery; the handlers parse the request and make the one call.
### Bounded Worker Pool
The delivery engine uses a **fixed-size worker pool** (default: 10 workers) to
@@ -2189,8 +2198,8 @@ preventing goroutine explosions regardless of queue depth.
**Architecture:**
- **Channels as queues:** Two buffered channels serve as bounded queues: a
delivery channel (new tasks from the webhook handler) and a retry channel
(tasks from backoff timers). Both are buffered to 10,000.
delivery channel (the tasks of deliveries the engine has just stored) and a
retry channel (tasks from backoff timers). Both are buffered to 10,000.
- **Fan-out via channel, not goroutines:** When an event arrives with multiple
targets, each `delivery.Task` is sent to the delivery channel. Workers pick
them up and process them — no goroutine-per-target.
@@ -3054,6 +3063,7 @@ webhooker/
│ │ └── logfield.go # Encoded-byte budget for client-supplied log values
│ ├── delivery/
│ │ ├── engine.go # Event-driven delivery engine (channel + timer based)
│ │ ├── store.go # Storing an event with its deliveries, and a replay, then queueing them
│ │ ├── circuit_breaker.go # Per-target circuit breaker for http/slack targets with retries
│ │ ├── target.go # Target interface, Task, Scheduler
│ │ ├── target_http.go # HTTP target (retries, circuit breaker)
@@ -3150,23 +3160,26 @@ Components are wired via Uber fx in this order:
10. `metrics.NewRegistry` — The registry `/metrics` serves
11. `metrics.New` — The delivery collectors, registered on that registry
12. `middleware.New` — HTTP middleware
13. `delivery.New` — Event-driven delivery engine
13. `delivery.New` — Event-driven delivery engine, which also stores each event
with its deliveries
14. `archive.NewWriters` — The database targets' archive writers, and the lock
every rename of their files runs under
15. `archive.NewSweeper` — Periodic pruning of idle archives
16. `delivery.Engine` → `delivery.Notifier` — interface bridge
17. `archive.Writers` → `archive.Archives` — interface bridge so deleting or
16. `archive.Writers` → `archive.Archives` — interface bridge so deleting or
renaming a webhook or target reaches its archive files
17. `delivery.Engine` → `delivery.CircuitBreakers` — interface bridge so the
pages can show a target whose deliveries are paused
18. `server.New` — HTTP server and router
The server starts via
`fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *archive.Sweeper) {})`,
which triggers the fx lifecycle hooks in dependency order. The
`delivery.Notifier` interface allows the webhook handler to send self-contained
`delivery.Task` slices to the engine without a direct package dependency. Each
task carries all target settings and event data inline (for bodies under 16 KiB,
`delivery.MaxInlineBodySize`), so the engine can deliver without reading from
any database — it only writes to record results.
which triggers the fx lifecycle hooks in dependency order. The handlers take
`*delivery.Engine` itself: the receiver and resubmit handlers store an event
through `StoreEvent`, and the replay handler adds a delivery through
`CreateReplay`. Each task the engine queues carries the target's settings and
the event's data, and the body too when it is under 16 KiB
(`delivery.MaxInlineBodySize`); a larger body is read from the event row when
the attempt runs.
### Middleware Stack