Show a target paused by its circuit breaker (closes #385)
check / check (push) Successful in 3m21s

While an http or slack target's circuit breaker turns its deliveries
away, the target's row on the webhook page says its deliveries are
paused and until when, in UTC and from now. Each of its retrying
deliveries shows as waiting, with the reason and the same time, in the
event log and on the event's page, instead of a bare "retrying".

The engine gains one read, CooldownRemaining(targetID), over the
breakers it already keeps; the handlers reach it through a one-method
interface wired like Archives.

Model: opus-5-5
This commit is contained in:
2026-10-02 22:04:53 +00:00
parent 9305af4f85
commit f531cf9cdf
16 changed files with 358 additions and 22 deletions
+34 -3
View File
@@ -143,6 +143,15 @@ type Archives interface {
Rename(targetID, webhookName, targetName string) error
}
// CircuitBreakers is how the handlers read a target's circuit
// breaker, so the webhook page and the event log can say that
// deliveries to the target are paused and until when. Like Archives,
// it keeps the handlers free of the engine's internals and is
// trivially faked in tests.
type CircuitBreakers interface {
CooldownRemaining(targetID string) time.Duration
}
// EngineParams are the fx dependencies for the delivery
// engine.
type EngineParams struct {
@@ -186,9 +195,11 @@ type Engine struct {
// targets maps each target type to its implementation.
targets map[database.TargetType]Target
// httpTarget is retained so tests can reach the HTTP
// target's shared client and circuit breakers.
httpTarget *httpTarget
// httpTarget and slackTarget are retained so CooldownRemaining
// can read their circuit breakers, and so tests can reach the
// HTTP target's shared client.
httpTarget *httpTarget
slackTarget *slackTarget
// dbTarget is retained so the engine can reach the archive
// writer registry for eviction, renames and the idle sweep.
@@ -300,6 +311,26 @@ func (e *Engine) Rename(
return e.dbTarget.rename(targetID, webhookName, targetName)
}
// CooldownRemaining implements CircuitBreakers. It is
// CircuitBreaker.CooldownRemaining for the target's breaker: how long
// the breaker will keep turning the target's deliveries away, and
// zero when it lets them through. A target with no breaker gets zero,
// and reading never creates one.
func (e *Engine) CooldownRemaining(targetID string) time.Duration {
for _, core := range []*httpCore{
e.httpTarget.httpCore, e.slackTarget.httpCore,
} {
val, ok := core.circuitBreakers.Load(targetID)
if ok {
cb, _ := val.(*CircuitBreaker)
return cb.CooldownRemaining()
}
}
return 0
}
// ScheduleRetry schedules a task to be re-enqueued onto the
// retry channel after delay. It implements the Scheduler
// interface the targets use to own their durable retries.