refactor: event-driven delivery engine with channel notifications and timer-based retries
All checks were successful
check / check (push) Successful in 58s

Replace the polling-based delivery engine with a fully event-driven
architecture using Go channels and goroutines:

- Webhook handler notifies engine via buffered channel after creating
  delivery records, with inline event data for payloads < 16KB
- Large payloads (>= 16KB) use pointer semantics (Body *string = nil)
  and are fetched from DB on demand, keeping channel memory bounded
- Failed retry-target deliveries schedule Go timers with exponential
  backoff; timers fire into a separate retry channel when ready
- On startup, engine scans DB once to recover interrupted deliveries
  (pending processed immediately, retrying get timers for remaining
  backoff)
- DB stores delivery status for crash recovery only, not for
  inter-component communication during normal operation
- delivery.Notifier interface decouples handlers from engine; fx wires
  *Engine as Notifier

No more periodic polling. No more wasted cycles when idle.
This commit is contained in:
clawbot
2026-03-01 21:46:16 -08:00
parent 8f62fde8e9
commit 5e683af2a4
6 changed files with 404 additions and 53 deletions

View File

@@ -463,11 +463,12 @@ External Service
1. Look up Entrypoint by UUID
2. Capture full request as Event
3. Queue Delivery to each active Target
4. Notify Engine via channel
┌──────────────┐
│ Delivery │
│ Engine │
│ Delivery │◄── retry timers
│ Engine │ (backoff)
└──────┬───────┘
┌────────────────────┼────────────────────┐
@@ -577,7 +578,7 @@ webhooker/
│ ├── globals/
│ │ └── globals.go # Build-time variables (appname, version, arch)
│ ├── delivery/
│ │ └── engine.go # Background delivery engine (fx lifecycle)
│ │ └── engine.go # Event-driven delivery engine (channel + timer based)
│ ├── handlers/
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
│ │ ├── auth.go # Login, logout handlers
@@ -627,11 +628,14 @@ Components are wired via Uber fx in this order:
7. `session.New` — Cookie-based session manager
8. `handlers.New` — HTTP handlers
9. `middleware.New` — HTTP middleware
10. `delivery.New`Background delivery engine
11. `server.New` — HTTP server and router
10. `delivery.New`Event-driven delivery engine
11. `delivery.Engine``handlers.DeliveryNotifier` — interface bridge
12. `server.New` — HTTP server and router
The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine)
{})` which triggers the fx lifecycle hooks in dependency order.
{})` which triggers the fx lifecycle hooks in dependency order. The
`DeliveryNotifier` interface allows the webhook handler to notify the
delivery engine of new work without a direct package dependency.
### Middleware Stack
@@ -720,7 +724,7 @@ linted, tested, and compiled.
- [x] Per-webhook database lifecycle management (create on webhook
creation, delete on webhook removal)
- [x] `WebhookDBManager` component with lazy connection pooling
- [x] Delivery engine polls all per-webhook DBs for pending deliveries
- [x] Event-driven delivery engine (channel notifications + timer-based retries)
- [x] Database target type marks delivery as immediately successful
(events are already in the per-webhook DB)