Terminally fail retrying deliveries with a non-retry target type (closes #82)
All checks were successful
check / check (push) Successful in 4m4s
All checks were successful
check / check (push) Successful in 4m4s
Restart recovery and the 60s retry sweep both looked an orphaned `retrying` delivery's target up in the registry and silently returned when it did not implement `rescheduler`. If a target's type was edited from a retry type (`http`/`slack`) to a fire-and-forget type (`database`/`log`) or an unknown one while a delivery was still retrying, that delivery stayed `retrying` forever. Both sites now hand the delivery to one shared helper, `failUnretryableRetry`, which records a `DeliveryResult` naming the current target type as the reason and marks the delivery `failed`. It logs at warn, not error: this is operator-caused state, not a system fault. Re-dispatching under the new type was rejected as it would perform a delivery the operator never asked for; the event itself stays in the per-webhook event database, so manual redelivery can recover it deliberately. Fire-and-forget targets never set status `retrying` under normal operation, so this path stays unreachable for them in practice.
This commit is contained in:
12
README.md
12
README.md
@@ -636,6 +636,18 @@ This means:
|
||||
durable fallback that ensures no retry is permanently lost, even under
|
||||
extreme backpressure.
|
||||
|
||||
**Changing a target's type does not migrate in-flight deliveries.** Only
|
||||
`http` and `slack` targets own durable retries; `database` and `log`
|
||||
targets are fire-and-forget and never produce a `retrying` delivery. If a
|
||||
target's `type` is edited from a retrying type to a non-retrying (or
|
||||
unknown) one while one of its deliveries is still `retrying`, both
|
||||
recovery paths above terminally mark that delivery `failed` and record a
|
||||
`DeliveryResult` naming the current target type as the reason, logging it
|
||||
at warn level. The delivery is not re-dispatched under the new type — the
|
||||
operator never asked for that delivery — and the event itself remains
|
||||
stored in the per-webhook event database, so it can be redelivered
|
||||
manually.
|
||||
|
||||
### Circuit Breaker (HTTP Targets with Retries)
|
||||
|
||||
HTTP targets with `max_retries` > 0 are protected by a **per-target circuit breaker** that
|
||||
|
||||
Reference in New Issue
Block a user