f3cb56345ff53a15f682c7d986a65a68d47a94b6
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f3cb56345f |
Resubmit a stored event as a new undelivered event (closes #250)
check / check (push) Successful in 3m3s
Capturing real webhook traffic and firing it repeatedly at a backend under development is a primary function of this service, and per-delivery replay cannot do it: it only ever resolves the delivery's own original target, so a target created for a dev backend has no prior delivery and nothing can be replayed to it. The event log now offers a per-event Resubmit action. It stores a NEW event copying the stored one's method, headers, body and content type verbatim, and fans it out to the webhook's currently ACTIVE targets, resolved fresh by the query the receiver uses -- so a target created long after the original event arrived receives it. The original event's deliveries have no bearing on where the copy goes, inactive targets are skipped as the receiver skips them, and the action is repeatable: replay's in-flight refusal is deliberately not ported, because firing one captured event over and over is the point. The receiver and the resubmit path share one construction and one fan-out site. An eventSource value carries where the fields came from, live request or stored event, and createAndFanOut writes the event and its pending deliveries in one transaction and hands the tasks to the same Notifier, so a resubmitted delivery is retried, SSRF-guarded and circuit-broken exactly as a first one is. buildDeliveryTasks returns an error instead of writing a response, which is what lets both callers share it. The stored event is read once, before the write transaction, with a cast to blob, so a body over delivery.MaxInlineBodySize is copied byte for byte and the engine loads it from the new event row. A nullable resubmitted_from_id records provenance -- empty for an event that arrived on the receiver -- and the event log reports the relationship in both directions, without which the log is unreadable after a few resubmits of one event. The route sits in the owned-source group, so auth, CSRF and the body cap apply, with its own rate limit bucket and an events_resubmitted_total counter. Inbound signature verification is not re-run: there is no inbound signature to check on a copy an authenticated, CSRF-protected operator action submits. Per-delivery replay is unchanged; it serves recovery, which resubmit does not replace. The README claimed in four places that replay was unimplemented, one of them telling the operator that a delivery stranded by a target type change was lost; all four are corrected and resubmit is documented beside replay. |
||
|
|
f0512f1c3c |
Render delivery attempt detail in the event log (closes #202) (#219)
check / check (push) Successful in 2m55s
delivery_results stored status_code, response_body, error, duration and attempt_num, and no template rendered any of it, so a failure read as "target: failed" and diagnosing it meant opening the per-webhook SQLite file by hand. An expanded delivery now lists its attempts with attempt number, status code, duration, error and response body. The body is bounded in the query rather than read whole and truncated in Go (#135), and a body the engine itself cut is no longer presented as complete. The response body and error are untrusted remote content, so target credentials are removed before rendering. Two cases needed care: a secret severed by the 4096-byte cut matches nothing as a whole string, and the engine's io.LimitReader cuts at the same constant the renderer uses, so the guard keys on the body reaching the cap rather than on the stored size exceeding it. Empty secrets are filtered where the secret list is built, because an empty string passed to strings.ReplaceAll inserts the marker at every byte boundary. loadTargetMap builds the redactor half unscoped, so a soft-deleted target's historical deliveries still render redacted. Also regenerates static/css/tailwind.css, which had drifted from the templates: hover:text-red-700, text-red-500, underline and w-28 were in use but absent from the served stylesheet (#236). |
||
|
|
3b0ed826bc |
Add per-delivery replay to the event log (closes #203) (#240)
check / check (push) Successful in 3m21s
There was no redelivery path anywhere: once a delivery exhausted
max_retries it was failed permanently, even though the event body is
durably stored. Storing an event and being unable to re-send it defeats
the reason it is stored, and the ordinary case is a destination that was
down longer than the backoff ladder.
Adds POST /source/{sourceID}/deliveries/{deliveryID}/replay, inside the
authenticated group so it inherits MaxBodySize, CSRF, NoCache and
RequireAuth. Replay creates a NEW pending delivery against the target's
CURRENT config and hands it to the engine through the same notifier the
receiver uses, so it runs the normal path with the retry ladder, the
SSRF-guarded transport and the circuit breaker. The original delivery's
rows are never touched, and the stored event body is re-sent, never the
recorded response.
Replay is refused, with a distinct message, for a non-terminal delivery, a
deleted target, a deactivated target, and when an earlier replay of the
same event and target is still in flight. Bounded by a per-client rate
limit and by that in-flight check.
The new delivery row is written with Omit(clause.Associations) and with
neither Event nor Target populated, so it cannot upsert a targets row into
the per-webhook event database (#206).
Counted by webhooker_delivery_replays_total on the existing target_type
label. A replay also moves the ordinary attempt, outcome and duration
series, because it is a real delivery.
|
||
|
|
41ff16a817 |
Serve an event's full stored body over HTTP (closes #157)
check / check (push) Successful in 2m44s
The 8 KB render cap from #135 left storage untouched but no route served the rest, so a body over the cap was reachable only with filesystem access to the SQLite files — in a product whose purpose is storing webhooks so they can be inspected. GET /source/{sourceID}/logs/{eventID}/body serves the whole body to the webhook's owner, as application/octet-stream with an attachment disposition and nosniff. Those are a security control, not formatting: the bytes come from the public receiver and are handed back inside the operator's authenticated origin, and the existing CSP would not stop a stored HTML payload executing there. The truncation marker links to it only when a body was actually cut. Accepted deviation, documented rather than glossed: #157's definition of done asks the route to stream from the row. It buffers whole instead, because database/sql exposes no incremental handle on a SQLite BLOB and substr range reads re-materialise the entire column per call — an earlier revision chunked at 64 KiB and was 11-15x slower for a worse bound. Three independent reviewers confirmed no streaming path exists. Independently reviewed three times. Two earlier revisions each asserted a memory bound the code did not have; the final reviewer measured 2.057x at the ingest cap and pinned the two overlapping allocations from source — the driver's column buffer and database/sql's convertAssign clone — confirming the stated "roughly two bodies, and 2x is a floor not a ceiling" is now accurate, since SQLite's own materialisation sits outside the Go heap. |
||
|
|
279effb4c2 |
Bound the event log's rendered bodies in the query (closes #135)
check / check (push) Successful in 3m0s
The event log rendered stored bodies untruncated. Since buffered rendering landed (#123) that became resident memory per concurrent viewer, up to tens of MB, driven by payloads unauthenticated clients supply to the public receiver. Bound in the query rather than the template, via substr(cast(body as blob), 1, ?) plus length(cast(body as blob)), so an oversized body never becomes a Go string at all. Adds an EventLogView projection carrying the true byte count, and trims a partial UTF-8 tail without rewriting bodies that are merely invalid UTF-8. Independently reviewed. The generated SQL was dumped under GORM DryRun to confirm the cap is a bound parameter, both casts are present, and no other path selects the full column; soft-delete scope, ordering and pagination are unchanged. Correction to the PR body: its quoted mutation output was produced by removing the bound from eventLogColumns, not by raising the cap to 1<<30 as the text claimed. The reviewer reproduced the real mutation and confirmed the tests do catch removal of the bound. Follow-up #157 restores in-app retrieval of bodies above the cap. |
||
|
|
7f8469a0f2 |
feat: implement core webhook engine, delivery system, and management UI (Phase 2)
check / check (push) Successful in 1m49s
- Webhook reception handler: look up entrypoint by UUID, verify active,
capture full HTTP request (method, headers, body, content-type), create
Event record, queue Delivery records for each active Target, return 200 OK.
Handles edge cases: unknown UUID → 404, inactive → 410, oversized → 413.
- Delivery engine (internal/delivery): fx-managed background goroutine that
polls for pending/retrying deliveries and dispatches to target type handlers.
Graceful shutdown via context cancellation.
- Target type implementations:
- HTTP: fire-and-forget POST with original headers forwarding
- Retry: exponential backoff (1s, 2s, 4s...) up to max_retries
- Database: immediate success (event already stored)
- Log: slog output with event details
- Webhook management pages with Tailwind CSS + Alpine.js:
- List (/sources): webhooks with entrypoint/target/event counts
- Create (/sources/new): form with auto-created default entrypoint
- Detail (/source/{id}): config, entrypoints, targets, recent events
- Edit (/source/{id}/edit): name, description, retention_days
- Delete (/source/{id}/delete): soft-delete with child records
- Add Entrypoint (/source/{id}/entrypoints): inline form
- Add Target (/source/{id}/targets): type-aware form
- Event Log (/source/{id}/logs): paginated with delivery status
- Updated README: marked completed items, updated naming conventions
table, added delivery engine to package layout and DI docs, updated
column names to reflect entity rename.
- Rebuilt Tailwind CSS for new template classes.
Part of: #15
|