Refactor delivery targets to a Target interface (closes #77) (#81)
All checks were successful
check / check (push) Successful in 2m42s
All checks were successful
check / check (push) Successful in 2m42s
Refactors the delivery engine so each target TYPE is an implementation of a `Target` interface, dispatched from a registry, with each target owning its full delivery including durable retries. Implements the authoritative design from issue #77 (the corrected "hand the DB + Scheduler to the target" design). ## The new interface ```go type Scheduler interface { ScheduleRetry(task Task, delay time.Duration) } type Target interface { Deliver(ctx context.Context, webhookDB *gorm.DB, d *database.Delivery, task *Task, sched Scheduler) } ``` `Deliver` receives everything a target needs to be autonomous and durable: the request context, the per-webhook `*gorm.DB`, the `*database.Delivery`, the attempt `*Task`, and a `Scheduler` (the engine) for durable re-enqueue. The target makes one attempt, writes the `DeliveryResult`, updates `DeliveryStatus`, and — for retry targets — decides whether to retry, computes its own backoff, gates with its own circuit breaker, and reschedules via the injected `Scheduler`. `processDelivery` collapses to a registry lookup (`map[database.TargetType]Target`) and a `Deliver` call; an unknown target type still fails the delivery as before. ## Per-target ownership - `httpTarget` and `slackTarget` share a retry core (`httpCore`) that owns retry, exponential backoff, and the per-target circuit breaker. The core is fire-and-forget when `MaxRetries == 0` and adds breaker-gated backed-off retries when `MaxRetries > 0`. The per-attempt request differs (HTTP forwards the body + filtered headers; Slack posts a formatted message) and is supplied as a closure, so each keeps its exact recording semantics (e.g. HTTP records no error string for a non-2xx, Slack records `HTTP <code>`). - `databaseTarget` and `logTarget` are fire-and-forget: they record a single successful attempt. Moved wholesale into the http/slack targets: `deliverHTTP*`, `handleHTTPRetry`, `circuitBreakerBlock`, `calcBackoff` / `calcRemainingBackoff` / `backoffElapsed`, the circuit-breaker `sync.Map` + `getCircuitBreaker`, `clientForConfig`, `doHTTPRequest`, `applyRequestHeaders`, and the config parsers. The engine keeps `recordResult`, `updateDeliveryStatus`, and `ScheduleRetry`. ## Slack MaxRetries gating Slack is now on the same shared core as HTTP, with retry + breaker gated on `MaxRetries`. A `MaxRetries` of 0 stays single-attempt fire-and-forget, so **every existing Slack target is unchanged**; a Slack target configured with retries gets backoff + circuit breaker. ## Log-target full content `logTarget` now logs the ENTIRE inbound webhook — full request body and full request headers, plus method, content type, and the webhook id and entrypoint id — rather than a summary line. This supersedes the smaller log-summary work (#70). ## `Task.EntrypointID` To carry the entrypoint id to the log target, `Task` gains an `EntrypointID` field, populated in the webhook handler's `buildDeliveryTasks`, the engine's recovery-task builder, and `buildEventFromTask`. ## Durability / recovery The crash-durable async retry model is preserved unchanged: one attempt per worker turn; on failure the status is set `retrying`, backoff is computed, and the task is re-enqueued via `ScheduleRetry` (a `time.AfterFunc` onto the retry channel). On restart, `recoverRetryingDeliveries` and the 60s sweep hand each orphaned `retrying` delivery back to its target to recompute the remaining backoff and reschedule (targets that own retries implement an internal `rescheduler`; fire-and-forget targets, which never produce `retrying` deliveries, are skipped). ## How behaviour is preserved No external behaviour changes except the two called out above (log target full content; Slack gaining `MaxRetries`-gated retries). All existing delivery tests pass with only their `export_test.go` wrappers re-pointed at the new structure — `ExportDeliverHTTP/Slack/Database/Log` now call the targets, `ExportGetCircuitBreaker` / `ExportClient` / `ExportClientForConfig` / `ExportDoHTTPRequest` resolve against the HTTP target's shared client and breaker map, and `ExportParseHTTPConfig` / `ExportParseSlackConfig` call the relocated free functions. Added: a `logTarget` test asserting the log line contains the full body, headers, and ids, and a Slack `MaxRetries`-gated retry test. `docker build .` is green (fmt-check, lint, test, static build all pass). Closes #77 Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #81 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org>
This commit was merged in pull request #81.
This commit is contained in:
@@ -1,6 +1,7 @@
|
||||
package delivery_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
@@ -1652,6 +1653,179 @@ func TestProcessDelivery_RoutesToSlack(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
// newLogCaptureEngine builds a test engine whose logger
|
||||
// writes to the returned buffer, for inspecting log output.
|
||||
func newLogCaptureEngine(
|
||||
t *testing.T,
|
||||
) (*delivery.Engine, *bytes.Buffer) {
|
||||
t.Helper()
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
log := slog.New(slog.NewTextHandler(
|
||||
&buf,
|
||||
&slog.HandlerOptions{Level: slog.LevelDebug},
|
||||
))
|
||||
|
||||
e := delivery.NewTestEngine(
|
||||
log, &http.Client{Timeout: 5 * time.Second}, 1,
|
||||
)
|
||||
|
||||
return e, &buf
|
||||
}
|
||||
|
||||
// assertLogLineComplete asserts the captured log output
|
||||
// carries the full inbound webhook content and ids.
|
||||
func assertLogLineComplete(
|
||||
t *testing.T, out string, event database.Event,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
assert.Contains(t, out, "log-body-marker",
|
||||
"log line must contain the full request body",
|
||||
)
|
||||
|
||||
assert.Contains(t, out, "Content-Type",
|
||||
"log line must contain the full request headers",
|
||||
)
|
||||
|
||||
assert.Contains(t, out, event.EntrypointID,
|
||||
"log line must contain the entrypoint id",
|
||||
)
|
||||
|
||||
assert.Contains(t, out, event.WebhookID,
|
||||
"log line must contain the webhook id",
|
||||
)
|
||||
|
||||
assert.Contains(t, out, "application/json",
|
||||
"log line must contain the content type",
|
||||
)
|
||||
}
|
||||
|
||||
func TestDeliverLog_LogsFullContent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
db := testWebhookDB(t)
|
||||
e, buf := newLogCaptureEngine(t)
|
||||
|
||||
event := seedEvent(
|
||||
t, db, `{"log-body-marker":"abc123"}`,
|
||||
)
|
||||
|
||||
dlv := seedDelivery(
|
||||
t, db, event.ID, uuid.New().String(),
|
||||
database.DeliveryStatusPending,
|
||||
)
|
||||
|
||||
d := &database.Delivery{
|
||||
EventID: event.ID,
|
||||
TargetID: dlv.TargetID,
|
||||
Status: database.DeliveryStatusPending,
|
||||
Event: event,
|
||||
Target: database.Target{
|
||||
Name: "test-log-full",
|
||||
Type: database.TargetTypeLog,
|
||||
},
|
||||
}
|
||||
d.ID = dlv.ID
|
||||
|
||||
e.ExportDeliverLog(db, d)
|
||||
|
||||
assertLogLineComplete(t, buf.String(), event)
|
||||
|
||||
assertDeliveryStatus(t, db, dlv.ID,
|
||||
database.DeliveryStatusDelivered,
|
||||
)
|
||||
}
|
||||
|
||||
// buildSlackRetryDelivery builds a Slack delivery whose
|
||||
// target is configured with retries enabled.
|
||||
func buildSlackRetryDelivery(
|
||||
dlv database.Delivery,
|
||||
event database.Event,
|
||||
targetID, cfg string,
|
||||
) *database.Delivery {
|
||||
d := &database.Delivery{
|
||||
EventID: event.ID,
|
||||
TargetID: targetID,
|
||||
Status: database.DeliveryStatusPending,
|
||||
Event: event,
|
||||
Target: database.Target{
|
||||
Name: "test-slack-retry",
|
||||
Type: database.TargetTypeSlack,
|
||||
Config: cfg,
|
||||
MaxRetries: 5,
|
||||
},
|
||||
}
|
||||
d.ID = dlv.ID
|
||||
|
||||
return d
|
||||
}
|
||||
|
||||
func TestDeliverSlack_WithRetries_SchedulesRetry(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
db := testWebhookDB(t)
|
||||
ts := newStatusServer(t, http.StatusServiceUnavailable)
|
||||
e := testEngine(t, 1)
|
||||
targetID := uuid.New().String()
|
||||
|
||||
slackCfg, err := json.Marshal(
|
||||
delivery.SlackTargetConfig{WebhookURL: ts.URL},
|
||||
)
|
||||
require.NoError(t, err)
|
||||
|
||||
event := seedEvent(t, db, `{"slack":"retry"}`)
|
||||
|
||||
dlv := seedDelivery(
|
||||
t, db, event.ID, targetID,
|
||||
database.DeliveryStatusPending,
|
||||
)
|
||||
|
||||
d := buildSlackRetryDelivery(
|
||||
dlv, event, targetID, string(slackCfg),
|
||||
)
|
||||
|
||||
task := &delivery.Task{
|
||||
DeliveryID: dlv.ID,
|
||||
TargetID: targetID,
|
||||
TargetType: database.TargetTypeSlack,
|
||||
MaxRetries: 5,
|
||||
AttemptNum: 1,
|
||||
}
|
||||
|
||||
e.ExportProcessDelivery(context.TODO(), db, d, task)
|
||||
|
||||
assertDeliveryStatus(t, db, dlv.ID,
|
||||
database.DeliveryStatusRetrying,
|
||||
)
|
||||
|
||||
assertDeliveryResult(
|
||||
t, db, dlv.ID, false,
|
||||
http.StatusServiceUnavailable,
|
||||
)
|
||||
}
|
||||
|
||||
// newStatusServer starts a test server that always responds
|
||||
// with the given status code.
|
||||
func newStatusServer(
|
||||
t *testing.T, code int,
|
||||
) *httptest.Server {
|
||||
t.Helper()
|
||||
|
||||
ts := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(code)
|
||||
},
|
||||
))
|
||||
|
||||
t.Cleanup(ts.Close)
|
||||
|
||||
return ts
|
||||
}
|
||||
|
||||
// readAll is a small helper to avoid importing io in
|
||||
// a test handler inline.
|
||||
func readAll(r interface {
|
||||
|
||||
Reference in New Issue
Block a user