c1fce4326c
Some checks failed
check / check (push) Failing after 2m17s
A delivery that exhausted max_retries was failed forever. The event body is durably stored, so the only way to get it delivered was to download it and re-POST by hand. The event log now offers a Replay action on any finished delivery. Replay creates a NEW pending delivery for the same event and target and hands it to the delivery engine through the same Notifier the receiver uses, so it is retried, SSRF-guarded and circuit-broken exactly as a first attempt. The original delivery's status, timestamps and recorded attempts are never touched, and what is re-sent is the stored event body, not the response the original attempt received. The target is read as it stands now, including soft-deleted rows so that a deleted target refuses the replay with a message on the page instead of erroring or delivering from stale configuration. A deactivated target and a target id that names nothing refuse the same way, as does a replay of a delivery the engine has not finished. Two bounds on replay storms: the route carries a per-client POST rate limit of 30 per minute, and the handler refuses a replay while an earlier one for the same event and target is still pending or retrying. One new metric, webhooker_delivery_replays_total, on the existing target_type label. A replay is a real delivery and moves the attempt, outcome and duration series like any other; this counter is what separates it from ordinary traffic without adding a dimension to every existing series. The delivery row is written with associations omitted and with neither Event nor Target populated, so no target row reaches the per-webhook event database.
44 lines
1.4 KiB
Go
44 lines
1.4 KiB
Go
package database
|
|
|
|
// DeliveryStatus represents the status of a delivery
|
|
type DeliveryStatus string
|
|
|
|
// Delivery status values.
|
|
const (
|
|
DeliveryStatusPending DeliveryStatus = "pending"
|
|
DeliveryStatusDelivered DeliveryStatus = "delivered"
|
|
DeliveryStatusFailed DeliveryStatus = "failed"
|
|
DeliveryStatusRetrying DeliveryStatus = "retrying"
|
|
)
|
|
|
|
// Terminal reports whether a delivery in this status has finished, so
|
|
// the delivery engine will make no further attempt of its own.
|
|
//
|
|
// It is what decides which deliveries the event log offers to replay:
|
|
// a pending or retrying delivery is still the engine's, and replaying
|
|
// one would race it.
|
|
func (s DeliveryStatus) Terminal() bool {
|
|
switch s {
|
|
case DeliveryStatusDelivered, DeliveryStatusFailed:
|
|
return true
|
|
case DeliveryStatusPending, DeliveryStatusRetrying:
|
|
return false
|
|
default:
|
|
return false
|
|
}
|
|
}
|
|
|
|
// Delivery represents a delivery attempt for an event to a target
|
|
type Delivery struct {
|
|
BaseModel
|
|
|
|
EventID string `gorm:"type:uuid;not null" json:"eventId"`
|
|
TargetID string `gorm:"type:uuid;not null" json:"targetId"`
|
|
Status DeliveryStatus `gorm:"not null;default:'pending'" json:"status"`
|
|
|
|
// Relations
|
|
Event Event `json:"event,omitzero"`
|
|
Target Target `json:"target,omitzero"`
|
|
DeliveryResults []DeliveryResult `json:"deliveryResults,omitempty"`
|
|
}
|