Close the two remaining delivery terminal-state gaps (closes #107)
All checks were successful
check / check (push) Successful in 3m11s

A delivery could reach a bad end without the engine recording why, and
a retrying delivery could fail to reach an end at all.

An unknown target type marked the delivery failed and wrote no
DeliveryResult, so the event log showed "failed" with no attempts and
the only account of why was one line in the server log. It now records
a result naming the type before failing the delivery.

A deleted target left its retrying deliveries stranded. Both recovery
and the sweep began with a scoped loadTarget, which cannot see a soft
deleted row, so both logged and returned: the delivery stayed retrying
for the life of the database while the sweep repeated the same error
every minute. Both now terminalise it with a recorded reason.

Deleting a target also did not stop deliveries to it. A scheduled
retry is a time.AfterFunc holding the target's configuration from when
the chain began, and nothing on that path read the target row, so the
timer kept firing and kept sending to the destination the operator had
removed for the rest of the backoff chain; terminalising in recovery
and the sweep alone would only have caught it after a restart.
processRetryTask now confirms the target still exists before it
attempts, and abandons the chain when it does not.

Only a target confirmed gone stops anything. A lookup that fails for
any other reason is the main database being unreadable, which is
transient, and every path leaves the delivery exactly as it was rather
than failing it.

The reason text comes from one Unscoped lookup confined to these
terminal paths, because a soft deleted row is what distinguishes a
target the operator deleted from an id that never named one. The
engine's normal target loading stays scoped, or deleting a target
would stop nothing.

Terminal writes reached from recovery keep going through the existing
retainIdle ownership gate; the retry path writes directly, as a
target's own Deliver does, because the worker already holds that
delivery. Nothing was added to either sweep dispatch arm.

Retry fixtures that drove processRetryTask for a target id with no row
in the main database now create one. That state is not reachable in
service: the handler reads the target to build the task.
This commit is contained in:
2026-08-24 02:30:51 +00:00
parent 322d9a6d6b
commit 3f429f9a4f
5 changed files with 789 additions and 10 deletions

View File

@@ -377,6 +377,17 @@ func TestProcessRetryTask_SuccessfulRetry(t *testing.T) {
bodyStr := event.Body
cfg := iHTTPConfig(ts.URL)
// The target row exists because the engine confirms a scheduled
// retry's target has not been deleted before it runs it. A retry
// task whose target id names no row at all is a state the service
// does not produce: the handler read that target to build the
// task. See https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-target",
database.TargetTypeHTTP, cfg, 5,
)
task := iTask(
d, event, s.WebhookID, targetID,
"retry-target", cfg, 5, 2, &bodyStr,
@@ -456,6 +467,12 @@ func TestProcessRetryTask_LargeBody_FetchFromDB(
)
cfg := iHTTPConfig(ts.URL)
iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-large",
database.TargetTypeHTTP, cfg, 5,
)
task := iTask(
d, event, s.WebhookID, targetID,
"retry-large", cfg, 5, 2, nil,
@@ -558,6 +575,12 @@ func TestWorkerLifecycle_ProcessesRetryChannel(
bodyStr := event.Body
cfg := iHTTPConfig(ts.URL)
iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-chan-test",
database.TargetTypeHTTP, cfg, 5,
)
task := iTask(
d, event, s.WebhookID, targetID,
"retry-chan-test", cfg, 5, 2, &bodyStr,