All checks were successful
check / check (push) Successful in 2m42s
Refactors the delivery engine so each target TYPE is an implementation of a `Target` interface, dispatched from a registry, with each target owning its full delivery including durable retries. Implements the authoritative design from issue #77 (the corrected "hand the DB + Scheduler to the target" design). ## The new interface ```go type Scheduler interface { ScheduleRetry(task Task, delay time.Duration) } type Target interface { Deliver(ctx context.Context, webhookDB *gorm.DB, d *database.Delivery, task *Task, sched Scheduler) } ``` `Deliver` receives everything a target needs to be autonomous and durable: the request context, the per-webhook `*gorm.DB`, the `*database.Delivery`, the attempt `*Task`, and a `Scheduler` (the engine) for durable re-enqueue. The target makes one attempt, writes the `DeliveryResult`, updates `DeliveryStatus`, and — for retry targets — decides whether to retry, computes its own backoff, gates with its own circuit breaker, and reschedules via the injected `Scheduler`. `processDelivery` collapses to a registry lookup (`map[database.TargetType]Target`) and a `Deliver` call; an unknown target type still fails the delivery as before. ## Per-target ownership - `httpTarget` and `slackTarget` share a retry core (`httpCore`) that owns retry, exponential backoff, and the per-target circuit breaker. The core is fire-and-forget when `MaxRetries == 0` and adds breaker-gated backed-off retries when `MaxRetries > 0`. The per-attempt request differs (HTTP forwards the body + filtered headers; Slack posts a formatted message) and is supplied as a closure, so each keeps its exact recording semantics (e.g. HTTP records no error string for a non-2xx, Slack records `HTTP <code>`). - `databaseTarget` and `logTarget` are fire-and-forget: they record a single successful attempt. Moved wholesale into the http/slack targets: `deliverHTTP*`, `handleHTTPRetry`, `circuitBreakerBlock`, `calcBackoff` / `calcRemainingBackoff` / `backoffElapsed`, the circuit-breaker `sync.Map` + `getCircuitBreaker`, `clientForConfig`, `doHTTPRequest`, `applyRequestHeaders`, and the config parsers. The engine keeps `recordResult`, `updateDeliveryStatus`, and `ScheduleRetry`. ## Slack MaxRetries gating Slack is now on the same shared core as HTTP, with retry + breaker gated on `MaxRetries`. A `MaxRetries` of 0 stays single-attempt fire-and-forget, so **every existing Slack target is unchanged**; a Slack target configured with retries gets backoff + circuit breaker. ## Log-target full content `logTarget` now logs the ENTIRE inbound webhook — full request body and full request headers, plus method, content type, and the webhook id and entrypoint id — rather than a summary line. This supersedes the smaller log-summary work (#70). ## `Task.EntrypointID` To carry the entrypoint id to the log target, `Task` gains an `EntrypointID` field, populated in the webhook handler's `buildDeliveryTasks`, the engine's recovery-task builder, and `buildEventFromTask`. ## Durability / recovery The crash-durable async retry model is preserved unchanged: one attempt per worker turn; on failure the status is set `retrying`, backoff is computed, and the task is re-enqueued via `ScheduleRetry` (a `time.AfterFunc` onto the retry channel). On restart, `recoverRetryingDeliveries` and the 60s sweep hand each orphaned `retrying` delivery back to its target to recompute the remaining backoff and reschedule (targets that own retries implement an internal `rescheduler`; fire-and-forget targets, which never produce `retrying` deliveries, are skipped). ## How behaviour is preserved No external behaviour changes except the two called out above (log target full content; Slack gaining `MaxRetries`-gated retries). All existing delivery tests pass with only their `export_test.go` wrappers re-pointed at the new structure — `ExportDeliverHTTP/Slack/Database/Log` now call the targets, `ExportGetCircuitBreaker` / `ExportClient` / `ExportClientForConfig` / `ExportDoHTTPRequest` resolve against the HTTP target's shared client and breaker map, and `ExportParseHTTPConfig` / `ExportParseSlackConfig` call the relocated free functions. Added: a `logTarget` test asserting the log line contains the full body, headers, and ids, and a Slack `MaxRetries`-gated retry test. `docker build .` is green (fmt-check, lint, test, static build all pass). Closes #77 Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #81 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org>
276 lines
6.1 KiB
Go
276 lines
6.1 KiB
Go
package delivery
|
|
|
|
import (
|
|
"context"
|
|
"log/slog"
|
|
"net"
|
|
"net/http"
|
|
"time"
|
|
|
|
"gorm.io/gorm"
|
|
"sneak.berlin/go/webhooker/internal/database"
|
|
)
|
|
|
|
// Exported constants for test access.
|
|
const (
|
|
ExportDeliveryChannelSize = deliveryChannelSize
|
|
ExportRetryChannelSize = retryChannelSize
|
|
ExportDefaultFailureThreshold = defaultFailureThreshold
|
|
ExportDefaultCooldown = defaultCooldown
|
|
)
|
|
|
|
// ExportIsBlockedIP exposes isBlockedIP for testing.
|
|
func ExportIsBlockedIP(ip net.IP) bool {
|
|
return isBlockedIP(ip)
|
|
}
|
|
|
|
// ExportBlockedNetworks exposes blockedNetworks.
|
|
func ExportBlockedNetworks() []*net.IPNet {
|
|
return blockedNetworks
|
|
}
|
|
|
|
// ExportIsForwardableHeader exposes isForwardableHeader.
|
|
func ExportIsForwardableHeader(name string) bool {
|
|
return isForwardableHeader(name)
|
|
}
|
|
|
|
// ExportTruncate exposes truncate for testing.
|
|
func ExportTruncate(s string, maxLen int) string {
|
|
return truncate(s, maxLen)
|
|
}
|
|
|
|
// ExportDeliverHTTP delivers via the http target for testing.
|
|
func (e *Engine) ExportDeliverHTTP(
|
|
ctx context.Context,
|
|
webhookDB *gorm.DB,
|
|
d *database.Delivery,
|
|
task *Task,
|
|
) {
|
|
e.httpTarget.Deliver(ctx, webhookDB, d, task, e)
|
|
}
|
|
|
|
// ExportDeliverDatabase delivers via the database target.
|
|
func (e *Engine) ExportDeliverDatabase(
|
|
webhookDB *gorm.DB, d *database.Delivery,
|
|
) {
|
|
e.targets[database.TargetTypeDatabase].Deliver(
|
|
context.Background(), webhookDB, d, &Task{}, e,
|
|
)
|
|
}
|
|
|
|
// ExportDeliverLog delivers via the log target for testing.
|
|
func (e *Engine) ExportDeliverLog(
|
|
webhookDB *gorm.DB, d *database.Delivery,
|
|
) {
|
|
e.targets[database.TargetTypeLog].Deliver(
|
|
context.Background(), webhookDB, d, &Task{}, e,
|
|
)
|
|
}
|
|
|
|
// ExportDeliverSlack delivers via the slack target for
|
|
// testing.
|
|
func (e *Engine) ExportDeliverSlack(
|
|
ctx context.Context,
|
|
webhookDB *gorm.DB,
|
|
d *database.Delivery,
|
|
) {
|
|
task := &Task{
|
|
DeliveryID: d.ID,
|
|
TargetID: d.TargetID,
|
|
AttemptNum: 1,
|
|
}
|
|
|
|
e.targets[database.TargetTypeSlack].Deliver(
|
|
ctx, webhookDB, d, task, e,
|
|
)
|
|
}
|
|
|
|
// ExportProcessNewTask exposes processNewTask.
|
|
func (e *Engine) ExportProcessNewTask(
|
|
ctx context.Context, task *Task,
|
|
) {
|
|
e.processNewTask(ctx, task)
|
|
}
|
|
|
|
// ExportProcessRetryTask exposes processRetryTask.
|
|
func (e *Engine) ExportProcessRetryTask(
|
|
ctx context.Context, task *Task,
|
|
) {
|
|
e.processRetryTask(ctx, task)
|
|
}
|
|
|
|
// ExportProcessDelivery exposes processDelivery.
|
|
func (e *Engine) ExportProcessDelivery(
|
|
ctx context.Context,
|
|
webhookDB *gorm.DB,
|
|
d *database.Delivery,
|
|
task *Task,
|
|
) {
|
|
e.processDelivery(ctx, webhookDB, d, task)
|
|
}
|
|
|
|
// ExportGetCircuitBreaker exposes the http target's
|
|
// getCircuitBreaker.
|
|
func (e *Engine) ExportGetCircuitBreaker(
|
|
targetID string,
|
|
) *CircuitBreaker {
|
|
return e.httpTarget.getCircuitBreaker(targetID)
|
|
}
|
|
|
|
// ExportParseHTTPConfig exposes parseHTTPConfig.
|
|
func (e *Engine) ExportParseHTTPConfig(
|
|
configJSON string,
|
|
) (*HTTPTargetConfig, error) {
|
|
return parseHTTPConfig(configJSON)
|
|
}
|
|
|
|
// ExportParseSlackConfig exposes parseSlackConfig.
|
|
func (e *Engine) ExportParseSlackConfig(
|
|
configJSON string,
|
|
) (*SlackTargetConfig, error) {
|
|
return parseSlackConfig(configJSON)
|
|
}
|
|
|
|
// ExportDoHTTPRequest exposes the http target's
|
|
// doHTTPRequest.
|
|
func (e *Engine) ExportDoHTTPRequest(
|
|
ctx context.Context,
|
|
cfg *HTTPTargetConfig,
|
|
event *database.Event,
|
|
) (int, string, int64, error) {
|
|
return e.httpTarget.doHTTPRequest(ctx, cfg, event)
|
|
}
|
|
|
|
// ExportClientForConfig exposes the http target's
|
|
// clientForConfig.
|
|
func (e *Engine) ExportClientForConfig(
|
|
cfg *HTTPTargetConfig,
|
|
) *http.Client {
|
|
return e.httpTarget.clientForConfig(cfg)
|
|
}
|
|
|
|
// ExportClient returns the http target's shared HTTP client.
|
|
func (e *Engine) ExportClient() *http.Client {
|
|
return e.httpTarget.client
|
|
}
|
|
|
|
// ExportScheduleRetry exposes ScheduleRetry.
|
|
func (e *Engine) ExportScheduleRetry(
|
|
task Task, delay time.Duration,
|
|
) {
|
|
e.ScheduleRetry(task, delay)
|
|
}
|
|
|
|
// ExportRecoverPendingDeliveries exposes
|
|
// recoverPendingDeliveries.
|
|
func (e *Engine) ExportRecoverPendingDeliveries(
|
|
ctx context.Context,
|
|
webhookDB *gorm.DB,
|
|
webhookID string,
|
|
) {
|
|
e.recoverPendingDeliveries(
|
|
ctx, webhookDB, webhookID,
|
|
)
|
|
}
|
|
|
|
// ExportRecoverWebhookDeliveries exposes
|
|
// recoverWebhookDeliveries.
|
|
func (e *Engine) ExportRecoverWebhookDeliveries(
|
|
ctx context.Context, webhookID string,
|
|
) {
|
|
e.recoverWebhookDeliveries(ctx, webhookID)
|
|
}
|
|
|
|
// ExportRecoverInFlight exposes recoverInFlight.
|
|
func (e *Engine) ExportRecoverInFlight(
|
|
ctx context.Context,
|
|
) {
|
|
e.recoverInFlight(ctx)
|
|
}
|
|
|
|
// ExportStart exposes start for testing.
|
|
func (e *Engine) ExportStart(ctx context.Context) {
|
|
e.start(ctx)
|
|
}
|
|
|
|
// ExportStop exposes stop for testing.
|
|
func (e *Engine) ExportStop() {
|
|
e.stop()
|
|
}
|
|
|
|
// ExportDeliveryCh returns the delivery channel.
|
|
func (e *Engine) ExportDeliveryCh() chan Task {
|
|
return e.deliveryCh
|
|
}
|
|
|
|
// ExportRetryCh returns the retry channel.
|
|
func (e *Engine) ExportRetryCh() chan Task {
|
|
return e.retryCh
|
|
}
|
|
|
|
// NewTestEngine creates an Engine for unit tests without
|
|
// database dependencies.
|
|
func NewTestEngine(
|
|
log *slog.Logger,
|
|
client *http.Client,
|
|
workers int,
|
|
) *Engine {
|
|
e := &Engine{
|
|
log: log,
|
|
deliveryCh: make(chan Task, deliveryChannelSize),
|
|
retryCh: make(chan Task, retryChannelSize),
|
|
workers: workers,
|
|
}
|
|
e.initTargets(client)
|
|
|
|
return e
|
|
}
|
|
|
|
// NewTestEngineSmallRetry creates an Engine with a tiny
|
|
// retry channel buffer for overflow testing.
|
|
func NewTestEngineSmallRetry(
|
|
log *slog.Logger,
|
|
) *Engine {
|
|
e := &Engine{
|
|
log: log,
|
|
retryCh: make(chan Task, 1),
|
|
}
|
|
e.initTargets(nil)
|
|
|
|
return e
|
|
}
|
|
|
|
// NewTestEngineWithDB creates an Engine with a real
|
|
// database and dbManager for integration tests.
|
|
func NewTestEngineWithDB(
|
|
db *database.Database,
|
|
dbMgr *database.WebhookDBManager,
|
|
log *slog.Logger,
|
|
client *http.Client,
|
|
workers int,
|
|
) *Engine {
|
|
e := &Engine{
|
|
database: db,
|
|
dbManager: dbMgr,
|
|
log: log,
|
|
deliveryCh: make(chan Task, deliveryChannelSize),
|
|
retryCh: make(chan Task, retryChannelSize),
|
|
workers: workers,
|
|
}
|
|
e.initTargets(client)
|
|
|
|
return e
|
|
}
|
|
|
|
// NewTestCircuitBreaker creates a CircuitBreaker with
|
|
// custom settings for testing.
|
|
func NewTestCircuitBreaker(
|
|
threshold int, cooldown time.Duration,
|
|
) *CircuitBreaker {
|
|
return &CircuitBreaker{
|
|
state: CircuitClosed,
|
|
threshold: threshold,
|
|
cooldown: cooldown,
|
|
}
|
|
}
|