All checks were successful
check / check (push) Successful in 2m42s
Refactors the delivery engine so each target TYPE is an implementation of a `Target` interface, dispatched from a registry, with each target owning its full delivery including durable retries. Implements the authoritative design from issue #77 (the corrected "hand the DB + Scheduler to the target" design). ## The new interface ```go type Scheduler interface { ScheduleRetry(task Task, delay time.Duration) } type Target interface { Deliver(ctx context.Context, webhookDB *gorm.DB, d *database.Delivery, task *Task, sched Scheduler) } ``` `Deliver` receives everything a target needs to be autonomous and durable: the request context, the per-webhook `*gorm.DB`, the `*database.Delivery`, the attempt `*Task`, and a `Scheduler` (the engine) for durable re-enqueue. The target makes one attempt, writes the `DeliveryResult`, updates `DeliveryStatus`, and — for retry targets — decides whether to retry, computes its own backoff, gates with its own circuit breaker, and reschedules via the injected `Scheduler`. `processDelivery` collapses to a registry lookup (`map[database.TargetType]Target`) and a `Deliver` call; an unknown target type still fails the delivery as before. ## Per-target ownership - `httpTarget` and `slackTarget` share a retry core (`httpCore`) that owns retry, exponential backoff, and the per-target circuit breaker. The core is fire-and-forget when `MaxRetries == 0` and adds breaker-gated backed-off retries when `MaxRetries > 0`. The per-attempt request differs (HTTP forwards the body + filtered headers; Slack posts a formatted message) and is supplied as a closure, so each keeps its exact recording semantics (e.g. HTTP records no error string for a non-2xx, Slack records `HTTP <code>`). - `databaseTarget` and `logTarget` are fire-and-forget: they record a single successful attempt. Moved wholesale into the http/slack targets: `deliverHTTP*`, `handleHTTPRetry`, `circuitBreakerBlock`, `calcBackoff` / `calcRemainingBackoff` / `backoffElapsed`, the circuit-breaker `sync.Map` + `getCircuitBreaker`, `clientForConfig`, `doHTTPRequest`, `applyRequestHeaders`, and the config parsers. The engine keeps `recordResult`, `updateDeliveryStatus`, and `ScheduleRetry`. ## Slack MaxRetries gating Slack is now on the same shared core as HTTP, with retry + breaker gated on `MaxRetries`. A `MaxRetries` of 0 stays single-attempt fire-and-forget, so **every existing Slack target is unchanged**; a Slack target configured with retries gets backoff + circuit breaker. ## Log-target full content `logTarget` now logs the ENTIRE inbound webhook — full request body and full request headers, plus method, content type, and the webhook id and entrypoint id — rather than a summary line. This supersedes the smaller log-summary work (#70). ## `Task.EntrypointID` To carry the entrypoint id to the log target, `Task` gains an `EntrypointID` field, populated in the webhook handler's `buildDeliveryTasks`, the engine's recovery-task builder, and `buildEventFromTask`. ## Durability / recovery The crash-durable async retry model is preserved unchanged: one attempt per worker turn; on failure the status is set `retrying`, backoff is computed, and the task is re-enqueued via `ScheduleRetry` (a `time.AfterFunc` onto the retry channel). On restart, `recoverRetryingDeliveries` and the 60s sweep hand each orphaned `retrying` delivery back to its target to recompute the remaining backoff and reschedule (targets that own retries implement an internal `rescheduler`; fire-and-forget targets, which never produce `retrying` deliveries, are skipped). ## How behaviour is preserved No external behaviour changes except the two called out above (log target full content; Slack gaining `MaxRetries`-gated retries). All existing delivery tests pass with only their `export_test.go` wrappers re-pointed at the new structure — `ExportDeliverHTTP/Slack/Database/Log` now call the targets, `ExportGetCircuitBreaker` / `ExportClient` / `ExportClientForConfig` / `ExportDoHTTPRequest` resolve against the HTTP target's shared client and breaker map, and `ExportParseHTTPConfig` / `ExportParseSlackConfig` call the relocated free functions. Added: a `logTarget` test asserting the log line contains the full body, headers, and ids, and a Slack `MaxRetries`-gated retry test. `docker build .` is green (fmt-check, lint, test, static build all pass). Closes #77 Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #81 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org>
300 lines
5.6 KiB
Go
300 lines
5.6 KiB
Go
package delivery
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"net/http"
|
|
"strings"
|
|
"time"
|
|
|
|
"gorm.io/gorm"
|
|
"sneak.berlin/go/webhooker/internal/database"
|
|
)
|
|
|
|
// errMissingWebhookURL is returned when a Slack target config
|
|
// omits its webhook URL.
|
|
var errMissingWebhookURL = errors.New(
|
|
"webhook_url is required",
|
|
)
|
|
|
|
// SlackTargetConfig holds configuration for slack target
|
|
// types.
|
|
type SlackTargetConfig struct {
|
|
WebhookURL string `json:"webhookUrl"`
|
|
}
|
|
|
|
// slackTarget delivers events to Slack incoming webhooks. It
|
|
// formats the event into a Slack message and posts it as
|
|
// JSON. It shares the retry core with the HTTP target: a
|
|
// MaxRetries of 0 stays single-attempt fire-and-forget
|
|
// (preserving existing Slack targets), while a positive
|
|
// MaxRetries adds backoff and circuit breaking.
|
|
type slackTarget struct {
|
|
*httpCore
|
|
|
|
client *http.Client
|
|
}
|
|
|
|
// Deliver implements Target.
|
|
func (t *slackTarget) Deliver(
|
|
ctx context.Context,
|
|
webhookDB *gorm.DB,
|
|
d *database.Delivery,
|
|
task *Task,
|
|
sched Scheduler,
|
|
) {
|
|
cfg, err := parseSlackConfig(d.Target.Config)
|
|
if err != nil {
|
|
t.eng.log.Error(
|
|
"invalid Slack target config",
|
|
"target_id", d.TargetID,
|
|
"error", err,
|
|
)
|
|
|
|
t.failConfig(webhookDB, d, err)
|
|
|
|
return
|
|
}
|
|
|
|
msg := FormatSlackMessage(&d.Event)
|
|
|
|
payload, err := json.Marshal(
|
|
map[string]string{"text": msg},
|
|
)
|
|
if err != nil {
|
|
t.eng.log.Error(
|
|
"failed to marshal Slack payload",
|
|
"target_id", d.TargetID,
|
|
"error", err,
|
|
)
|
|
|
|
t.failConfig(webhookDB, d, err)
|
|
|
|
return
|
|
}
|
|
|
|
attempt := func() attemptResult {
|
|
return t.attempt(ctx, cfg, payload)
|
|
}
|
|
|
|
t.deliver(
|
|
webhookDB, d, task, sched,
|
|
d.Target.MaxRetries, attempt,
|
|
)
|
|
}
|
|
|
|
// failConfig records a first-attempt failure for a delivery
|
|
// that could not be prepared (bad config or unmarshalable
|
|
// payload) and marks it failed.
|
|
func (t *slackTarget) failConfig(
|
|
webhookDB *gorm.DB,
|
|
d *database.Delivery,
|
|
err error,
|
|
) {
|
|
t.eng.recordResult(
|
|
webhookDB, d, 1,
|
|
false, 0, "", err.Error(), 0,
|
|
)
|
|
|
|
t.eng.updateDeliveryStatus(
|
|
webhookDB, d, database.DeliveryStatusFailed,
|
|
)
|
|
}
|
|
|
|
// attempt performs a single Slack POST and derives its
|
|
// outcome, preserving the engine's original semantics: a
|
|
// non-2xx response records an "HTTP <code>" error string and
|
|
// a transport error records a "sending request" error.
|
|
func (t *slackTarget) attempt(
|
|
ctx context.Context,
|
|
cfg *SlackTargetConfig,
|
|
payload []byte,
|
|
) attemptResult {
|
|
start := time.Now()
|
|
|
|
req, err := http.NewRequestWithContext(
|
|
ctx,
|
|
http.MethodPost,
|
|
cfg.WebhookURL,
|
|
bytes.NewReader(payload),
|
|
)
|
|
if err != nil {
|
|
return attemptResult{
|
|
success: false,
|
|
errMsg: err.Error(),
|
|
}
|
|
}
|
|
|
|
req.Header.Set("Content-Type", "application/json")
|
|
req.Header.Set("User-Agent", "webhooker/1.0")
|
|
|
|
resp, doErr := executeHTTPRequest(t.client, req)
|
|
durationMs := time.Since(start).Milliseconds()
|
|
|
|
if doErr != nil {
|
|
return attemptResult{
|
|
success: false,
|
|
duration: durationMs,
|
|
errMsg: fmt.Errorf(
|
|
"sending request: %w", doErr,
|
|
).Error(),
|
|
}
|
|
}
|
|
|
|
defer func() { _ = resp.Body.Close() }()
|
|
|
|
return t.readSlackResponse(resp, durationMs)
|
|
}
|
|
|
|
func (t *slackTarget) readSlackResponse(
|
|
resp *http.Response,
|
|
durationMs int64,
|
|
) attemptResult {
|
|
body, readErr := io.ReadAll(
|
|
io.LimitReader(resp.Body, maxBodyLog),
|
|
)
|
|
if readErr != nil {
|
|
t.eng.log.Error(
|
|
"failed to read Slack response body",
|
|
"error", readErr,
|
|
)
|
|
}
|
|
|
|
success := resp.StatusCode >= httpSuccessMin &&
|
|
resp.StatusCode < httpSuccessMax
|
|
|
|
errMsg := ""
|
|
if !success {
|
|
errMsg = fmt.Sprintf("HTTP %d", resp.StatusCode)
|
|
}
|
|
|
|
return attemptResult{
|
|
statusCode: resp.StatusCode,
|
|
respBody: string(body),
|
|
duration: durationMs,
|
|
success: success,
|
|
errMsg: errMsg,
|
|
}
|
|
}
|
|
|
|
func parseSlackConfig(
|
|
configJSON string,
|
|
) (*SlackTargetConfig, error) {
|
|
if configJSON == "" {
|
|
return nil, errEmptyTargetConfig
|
|
}
|
|
|
|
var cfg SlackTargetConfig
|
|
|
|
err := json.Unmarshal(
|
|
[]byte(configJSON), &cfg,
|
|
)
|
|
if err != nil {
|
|
return nil, fmt.Errorf(
|
|
"parsing config JSON: %w", err,
|
|
)
|
|
}
|
|
|
|
if cfg.WebhookURL == "" {
|
|
return nil, errMissingWebhookURL
|
|
}
|
|
|
|
return &cfg, nil
|
|
}
|
|
|
|
// FormatSlackMessage builds a Slack-compatible message
|
|
// string from a webhook event.
|
|
func FormatSlackMessage(
|
|
event *database.Event,
|
|
) string {
|
|
var b strings.Builder
|
|
|
|
b.WriteString("*Webhook Event Received*\n")
|
|
|
|
fmt.Fprintf(
|
|
&b, "*Method:* `%s`\n", event.Method,
|
|
)
|
|
|
|
fmt.Fprintf(
|
|
&b,
|
|
"*Content-Type:* `%s`\n",
|
|
event.ContentType,
|
|
)
|
|
|
|
fmt.Fprintf(
|
|
&b,
|
|
"*Timestamp:* `%s`\n",
|
|
event.CreatedAt.UTC().Format(time.RFC3339),
|
|
)
|
|
|
|
fmt.Fprintf(
|
|
&b,
|
|
"*Body Size:* %d bytes\n",
|
|
len(event.Body),
|
|
)
|
|
|
|
if event.Body == "" {
|
|
b.WriteString("\n_(empty body)_\n")
|
|
|
|
return b.String()
|
|
}
|
|
|
|
if formatted := formatJSONBody(event.Body); formatted != "" {
|
|
b.WriteString(formatted)
|
|
|
|
return b.String()
|
|
}
|
|
|
|
formatRawBody(&b, event.Body)
|
|
|
|
return b.String()
|
|
}
|
|
|
|
func formatJSONBody(body string) string {
|
|
var parsed json.RawMessage
|
|
if json.Unmarshal([]byte(body), &parsed) != nil {
|
|
return ""
|
|
}
|
|
|
|
var pretty bytes.Buffer
|
|
if json.Indent(&pretty, parsed, "", " ") != nil {
|
|
return ""
|
|
}
|
|
|
|
var b strings.Builder
|
|
|
|
b.WriteString("\n```\n")
|
|
|
|
prettyStr := pretty.String()
|
|
|
|
const maxPayloadDisplay = 3500
|
|
if len(prettyStr) > maxPayloadDisplay {
|
|
b.WriteString(prettyStr[:maxPayloadDisplay])
|
|
b.WriteString("\n... (truncated)")
|
|
} else {
|
|
b.WriteString(prettyStr)
|
|
}
|
|
|
|
b.WriteString("\n```\n")
|
|
|
|
return b.String()
|
|
}
|
|
|
|
func formatRawBody(b *strings.Builder, body string) {
|
|
b.WriteString("\n```\n")
|
|
|
|
const maxRawDisplay = 3500
|
|
if len(body) > maxRawDisplay {
|
|
b.WriteString(body[:maxRawDisplay])
|
|
b.WriteString("\n... (truncated)")
|
|
} else {
|
|
b.WriteString(body)
|
|
}
|
|
|
|
b.WriteString("\n```\n")
|
|
}
|