Files
webhooker/internal/handlers/webhook.go
clawbot 81413c56e9
All checks were successful
check / check (push) Successful in 2m42s
Refactor delivery targets to a Target interface (closes #77) (#81)
Refactors the delivery engine so each target TYPE is an implementation of a `Target` interface, dispatched from a registry, with each target owning its full delivery including durable retries. Implements the authoritative design from issue #77 (the corrected "hand the DB + Scheduler to the target" design).

## The new interface

```go
type Scheduler interface {
    ScheduleRetry(task Task, delay time.Duration)
}

type Target interface {
    Deliver(ctx context.Context, webhookDB *gorm.DB,
        d *database.Delivery, task *Task, sched Scheduler)
}
```

`Deliver` receives everything a target needs to be autonomous and durable: the request context, the per-webhook `*gorm.DB`, the `*database.Delivery`, the attempt `*Task`, and a `Scheduler` (the engine) for durable re-enqueue. The target makes one attempt, writes the `DeliveryResult`, updates `DeliveryStatus`, and — for retry targets — decides whether to retry, computes its own backoff, gates with its own circuit breaker, and reschedules via the injected `Scheduler`.

`processDelivery` collapses to a registry lookup (`map[database.TargetType]Target`) and a `Deliver` call; an unknown target type still fails the delivery as before.

## Per-target ownership

- `httpTarget` and `slackTarget` share a retry core (`httpCore`) that owns retry, exponential backoff, and the per-target circuit breaker. The core is fire-and-forget when `MaxRetries == 0` and adds breaker-gated backed-off retries when `MaxRetries > 0`. The per-attempt request differs (HTTP forwards the body + filtered headers; Slack posts a formatted message) and is supplied as a closure, so each keeps its exact recording semantics (e.g. HTTP records no error string for a non-2xx, Slack records `HTTP <code>`).
- `databaseTarget` and `logTarget` are fire-and-forget: they record a single successful attempt.

Moved wholesale into the http/slack targets: `deliverHTTP*`, `handleHTTPRetry`, `circuitBreakerBlock`, `calcBackoff` / `calcRemainingBackoff` / `backoffElapsed`, the circuit-breaker `sync.Map` + `getCircuitBreaker`, `clientForConfig`, `doHTTPRequest`, `applyRequestHeaders`, and the config parsers. The engine keeps `recordResult`, `updateDeliveryStatus`, and `ScheduleRetry`.

## Slack MaxRetries gating

Slack is now on the same shared core as HTTP, with retry + breaker gated on `MaxRetries`. A `MaxRetries` of 0 stays single-attempt fire-and-forget, so **every existing Slack target is unchanged**; a Slack target configured with retries gets backoff + circuit breaker.

## Log-target full content

`logTarget` now logs the ENTIRE inbound webhook — full request body and full request headers, plus method, content type, and the webhook id and entrypoint id — rather than a summary line. This supersedes the smaller log-summary work (#70).

## `Task.EntrypointID`

To carry the entrypoint id to the log target, `Task` gains an `EntrypointID` field, populated in the webhook handler's `buildDeliveryTasks`, the engine's recovery-task builder, and `buildEventFromTask`.

## Durability / recovery

The crash-durable async retry model is preserved unchanged: one attempt per worker turn; on failure the status is set `retrying`, backoff is computed, and the task is re-enqueued via `ScheduleRetry` (a `time.AfterFunc` onto the retry channel). On restart, `recoverRetryingDeliveries` and the 60s sweep hand each orphaned `retrying` delivery back to its target to recompute the remaining backoff and reschedule (targets that own retries implement an internal `rescheduler`; fire-and-forget targets, which never produce `retrying` deliveries, are skipped).

## How behaviour is preserved

No external behaviour changes except the two called out above (log target full content; Slack gaining `MaxRetries`-gated retries). All existing delivery tests pass with only their `export_test.go` wrappers re-pointed at the new structure — `ExportDeliverHTTP/Slack/Database/Log` now call the targets, `ExportGetCircuitBreaker` / `ExportClient` / `ExportClientForConfig` / `ExportDoHTTPRequest` resolve against the HTTP target's shared client and breaker map, and `ExportParseHTTPConfig` / `ExportParseSlackConfig` call the relocated free functions. Added: a `logTarget` test asserting the log line contains the full body, headers, and ids, and a Slack `MaxRetries`-gated retry test.

`docker build .` is green (fmt-check, lint, test, static build all pass).

Closes #77

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #81
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 17:07:49 +02:00

348 lines
7.0 KiB
Go

package handlers
import (
"encoding/json"
"io"
"net/http"
"github.com/go-chi/chi"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
const (
// maxWebhookBodySize is the maximum allowed webhook
// request body (1 MB).
maxWebhookBodySize = 1 << maxBodyShift
)
// HandleWebhook handles incoming webhook requests at entrypoint
// URLs.
func (h *Handlers) HandleWebhook() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
w.Header().Set("Allow", "POST")
http.Error(
w,
"Method Not Allowed",
http.StatusMethodNotAllowed,
)
return
}
entrypointUUID := chi.URLParam(r, "uuid")
if entrypointUUID == "" {
http.NotFound(w, r)
return
}
h.log.Info("webhook request received",
"entrypoint_uuid", entrypointUUID,
"method", r.Method,
"remote_addr", r.RemoteAddr,
)
entrypoint, ok := h.lookupEntrypoint(
w, r, entrypointUUID,
)
if !ok {
return
}
if !entrypoint.Active {
http.Error(w, "Gone", http.StatusGone)
return
}
h.processWebhookRequest(w, r, entrypoint)
}
}
// processWebhookRequest reads the body, serializes headers,
// loads targets, and delivers the event.
func (h *Handlers) processWebhookRequest(
w http.ResponseWriter,
r *http.Request,
entrypoint database.Entrypoint,
) {
body, ok := h.readWebhookBody(w, r)
if !ok {
return
}
headersJSON, err := json.Marshal(r.Header)
if err != nil {
h.serverError(w, "failed to serialize headers", err)
return
}
targets, err := h.loadActiveTargets(entrypoint.WebhookID)
if err != nil {
h.serverError(w, "failed to query targets", err)
return
}
h.createAndDeliverEvent(
w, r, entrypoint, body, headersJSON, targets,
)
}
// loadActiveTargets returns all active targets for a webhook.
func (h *Handlers) loadActiveTargets(
webhookID string,
) ([]database.Target, error) {
var targets []database.Target
err := h.db.DB().Where(
"webhook_id = ? AND active = ?",
webhookID, true,
).Find(&targets).Error
return targets, err
}
// lookupEntrypoint finds an entrypoint by UUID path.
func (h *Handlers) lookupEntrypoint(
w http.ResponseWriter,
r *http.Request,
entrypointUUID string,
) (database.Entrypoint, bool) {
var entrypoint database.Entrypoint
result := h.db.DB().Where(
"path = ?", entrypointUUID,
).First(&entrypoint)
if result.Error != nil {
h.log.Debug(
"entrypoint not found",
"path", entrypointUUID,
)
http.NotFound(w, r)
return entrypoint, false
}
return entrypoint, true
}
// readWebhookBody reads and validates the request body size.
func (h *Handlers) readWebhookBody(
w http.ResponseWriter,
r *http.Request,
) ([]byte, bool) {
body, err := io.ReadAll(
io.LimitReader(r.Body, maxWebhookBodySize+1),
)
if err != nil {
h.log.Error(
"failed to read request body", "error", err,
)
http.Error(
w, "Bad request", http.StatusBadRequest,
)
return nil, false
}
if len(body) > maxWebhookBodySize {
http.Error(
w,
"Request body too large",
http.StatusRequestEntityTooLarge,
)
return nil, false
}
return body, true
}
// createAndDeliverEvent creates the event and delivery records
// then notifies the delivery engine.
func (h *Handlers) createAndDeliverEvent(
w http.ResponseWriter,
r *http.Request,
entrypoint database.Entrypoint,
body, headersJSON []byte,
targets []database.Target,
) {
tx, err := h.beginWebhookTx(w, entrypoint.WebhookID)
if err != nil {
return
}
event := h.buildEvent(r, entrypoint, headersJSON, body)
err = tx.Create(event).Error
if err != nil {
tx.Rollback()
h.serverError(w, "failed to create event", err)
return
}
bodyPtr := inlineBody(body)
tasks := h.buildDeliveryTasks(
w, tx, event, entrypoint, targets, bodyPtr,
)
if tasks == nil {
return
}
err = tx.Commit().Error
if err != nil {
h.serverError(w, "failed to commit transaction", err)
return
}
h.finishWebhookResponse(w, event, entrypoint, tasks)
}
// beginWebhookTx opens a transaction on the per-webhook DB.
func (h *Handlers) beginWebhookTx(
w http.ResponseWriter,
webhookID string,
) (*gorm.DB, error) {
webhookDB, err := h.dbMgr.GetDB(webhookID)
if err != nil {
h.serverError(
w, "failed to get webhook database", err,
)
return nil, err
}
tx := webhookDB.Begin()
if tx.Error != nil {
h.serverError(
w, "failed to begin transaction", tx.Error,
)
return nil, tx.Error
}
return tx, nil
}
// inlineBody returns a pointer to body as a string if it fits
// within the inline size limit, or nil otherwise.
func inlineBody(body []byte) *string {
if len(body) < delivery.MaxInlineBodySize {
s := string(body)
return &s
}
return nil
}
// finishWebhookResponse notifies the delivery engine, logs the
// event, and writes the HTTP response.
func (h *Handlers) finishWebhookResponse(
w http.ResponseWriter,
event *database.Event,
entrypoint database.Entrypoint,
tasks []delivery.Task,
) {
if len(tasks) > 0 {
h.notifier.Notify(tasks)
}
h.log.Info("webhook event created",
"event_id", event.ID,
"webhook_id", entrypoint.WebhookID,
"entrypoint_id", entrypoint.ID,
"target_count", len(tasks),
)
w.WriteHeader(http.StatusOK)
_, err := w.Write([]byte(`{"status":"ok"}`))
if err != nil {
h.log.Error(
"failed to write response", "error", err,
)
}
}
// buildEvent creates a new Event struct from request data.
func (h *Handlers) buildEvent(
r *http.Request,
entrypoint database.Entrypoint,
headersJSON, body []byte,
) *database.Event {
return &database.Event{
WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID,
Method: r.Method,
Headers: string(headersJSON),
Body: string(body),
ContentType: r.Header.Get("Content-Type"),
}
}
// buildDeliveryTasks creates delivery records in the
// transaction and returns tasks for the delivery engine.
// Returns nil if an error occurred.
func (h *Handlers) buildDeliveryTasks(
w http.ResponseWriter,
tx *gorm.DB,
event *database.Event,
entrypoint database.Entrypoint,
targets []database.Target,
bodyPtr *string,
) []delivery.Task {
tasks := make([]delivery.Task, 0, len(targets))
for i := range targets {
dlv := &database.Delivery{
EventID: event.ID,
TargetID: targets[i].ID,
Status: database.DeliveryStatusPending,
}
err := tx.Create(dlv).Error
if err != nil {
tx.Rollback()
h.log.Error(
"failed to create delivery",
"target_id", targets[i].ID,
"error", err,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return nil
}
tasks = append(tasks, delivery.Task{
DeliveryID: dlv.ID,
EventID: event.ID,
WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID,
TargetID: targets[i].ID,
TargetName: targets[i].Name,
TargetType: targets[i].Type,
TargetConfig: targets[i].Config,
MaxRetries: targets[i].MaxRetries,
Method: event.Method,
Headers: event.Headers,
ContentType: event.ContentType,
Body: bodyPtr,
AttemptNum: 1,
})
}
return tasks
}