Add per-delivery replay to the event log (closes #203)
Some checks failed
check / check (push) Failing after 2m17s

A delivery that exhausted max_retries was failed forever. The event
body is durably stored, so the only way to get it delivered was to
download it and re-POST by hand.

The event log now offers a Replay action on any finished delivery.
Replay creates a NEW pending delivery for the same event and target
and hands it to the delivery engine through the same Notifier the
receiver uses, so it is retried, SSRF-guarded and circuit-broken
exactly as a first attempt. The original delivery's status,
timestamps and recorded attempts are never touched, and what is
re-sent is the stored event body, not the response the original
attempt received.

The target is read as it stands now, including soft-deleted rows so
that a deleted target refuses the replay with a message on the page
instead of erroring or delivering from stale configuration. A
deactivated target and a target id that names nothing refuse the same
way, as does a replay of a delivery the engine has not finished.

Two bounds on replay storms: the route carries a per-client POST rate
limit of 30 per minute, and the handler refuses a replay while an
earlier one for the same event and target is still pending or
retrying.

One new metric, webhooker_delivery_replays_total, on the existing
target_type label. A replay is a real delivery and moves the attempt,
outcome and duration series like any other; this counter is what
separates it from ordinary traffic without adding a dimension to
every existing series.

The delivery row is written with associations omitted and with
neither Event nor Target populated, so no target row reaches the
per-webhook event database.
This commit is contained in:
clawbot
2026-08-20 05:45:04 +00:00
parent aba02bc509
commit c1fce4326c
12 changed files with 1240 additions and 14 deletions

View File

@@ -12,6 +12,10 @@
</div>
</div>
{{if .ReplayMessage}}
<div class="{{if .ReplayQueued}}alert-success{{else}}alert-error{{end}}">{{.ReplayMessage}}</div>
{{end}}
<div class="card">
<div class="divide-y divide-gray-100">
{{range .Events}}
@@ -40,6 +44,23 @@
{{if .BodyTruncated}}
<p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged &mdash; <a href="/source/{{$.Webhook.ID}}/logs/{{.ID}}/body" class="text-primary-600 hover:text-primary-700 underline">download the full body</a>.</p>
{{end}}
{{if .Deliveries}}
<div class="mt-3 pt-4 border-t border-gray-200">
{{range .Deliveries}}
<div class="flex items-center justify-between py-2 text-xs text-gray-700">
<span><span class="font-medium">{{.Target.Name}}</span>: {{.Status}}</span>
{{if .Status.Terminal}}
<form method="POST" action="/source/{{$.Webhook.ID}}/deliveries/{{.ID}}/replay" class="inline">
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<input type="hidden" name="page" value="{{$.Page}}">
<button type="submit" class="text-xs text-primary-600 hover:text-primary-700" title="Send this event to the target again">Replay</button>
</form>
{{end}}
</div>
{{end}}
</div>
{{end}}
</div>
</div>
{{else}}