03c8e4616cd88214c910ff6d1d4d05f8bb2bc3c4
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
03c8e4616c |
Render delivery attempt detail in the event log (closes #202)
check / check (push) Successful in 3m35s
Expanding a delivery on the event log page now shows each recorded attempt: attempt number, outcome, status code, duration, error and response body. Previously a failure rendered as "target: failed" and diagnosing it meant opening the per-webhook SQLite file by hand. The response body is cut by SQLite via substr over a blob cast, the same projection the event body uses, so an oversized stored response never becomes a Go string. The page reports the cut with a marker. Response bodies and errors are remote content, so both go through a new delivery.Redactor that strips the target's own destination URL, path, query and userinfo, plus the values of credential-shaped request headers, before rendering. Target configuration keeps reaching the template only as a TargetView. A body that reaches the cap is treated as cut whether or not SQLite is what cut it. The delivery engine stops reading a response at its own cap, which is the same number of bytes this page renders, and the row it writes records that cut length as the whole length, so nothing in the row separates a response that ended at the cap from one severed there. Such a body goes through RedactCut, which drops any tail that is a proper prefix of a secret: the remote chooses the padding in front of a credential it echoes, so it chooses where the cut falls inside that credential. Its marker says the response reached the recording limit rather than quoting a total the row does not know. Redactors are built from an unscoped target load. Deleting a target only soft deletes the row while its deliveries survive, and a scoped load would leave exactly those deliveries rendering unredacted. The views the page lists stay scoped. Attempt loading is chunked so the IN clause cannot exceed SQLite's bound-parameter limit, and a chunk that fails fails the page rather than rendering the deliveries it covered as never having run. The page renders at most 20 attempts per delivery, counting what it leaves out. static/css/tailwind.css is regenerated with tailwindcss for the utility classes the new markup uses. |
||
|
|
3b0ed826bc |
Add per-delivery replay to the event log (closes #203) (#240)
check / check (push) Successful in 3m21s
There was no redelivery path anywhere: once a delivery exhausted
max_retries it was failed permanently, even though the event body is
durably stored. Storing an event and being unable to re-send it defeats
the reason it is stored, and the ordinary case is a destination that was
down longer than the backoff ladder.
Adds POST /source/{sourceID}/deliveries/{deliveryID}/replay, inside the
authenticated group so it inherits MaxBodySize, CSRF, NoCache and
RequireAuth. Replay creates a NEW pending delivery against the target's
CURRENT config and hands it to the engine through the same notifier the
receiver uses, so it runs the normal path with the retry ladder, the
SSRF-guarded transport and the circuit breaker. The original delivery's
rows are never touched, and the stored event body is re-sent, never the
recorded response.
Replay is refused, with a distinct message, for a non-terminal delivery, a
deleted target, a deactivated target, and when an earlier replay of the
same event and target is still in flight. Bounded by a per-client rate
limit and by that in-flight check.
The new delivery row is written with Omit(clause.Associations) and with
neither Event nor Target populated, so it cannot upsert a targets row into
the per-webhook event database (#206).
Counted by webhooker_delivery_replays_total on the existing target_type
label. A replay also moves the ordinary attempt, outcome and duration
series, because it is a real delivery.
|
||
|
|
41ff16a817 |
Serve an event's full stored body over HTTP (closes #157)
check / check (push) Successful in 2m44s
The 8 KB render cap from #135 left storage untouched but no route served the rest, so a body over the cap was reachable only with filesystem access to the SQLite files — in a product whose purpose is storing webhooks so they can be inspected. GET /source/{sourceID}/logs/{eventID}/body serves the whole body to the webhook's owner, as application/octet-stream with an attachment disposition and nosniff. Those are a security control, not formatting: the bytes come from the public receiver and are handed back inside the operator's authenticated origin, and the existing CSP would not stop a stored HTML payload executing there. The truncation marker links to it only when a body was actually cut. Accepted deviation, documented rather than glossed: #157's definition of done asks the route to stream from the row. It buffers whole instead, because database/sql exposes no incremental handle on a SQLite BLOB and substr range reads re-materialise the entire column per call — an earlier revision chunked at 64 KiB and was 11-15x slower for a worse bound. Three independent reviewers confirmed no streaming path exists. Independently reviewed three times. Two earlier revisions each asserted a memory bound the code did not have; the final reviewer measured 2.057x at the ingest cap and pinned the two overlapping allocations from source — the driver's column buffer and database/sql's convertAssign clone — confirming the stated "roughly two bodies, and 2x is a floor not a ceiling" is now accurate, since SQLite's own materialisation sits outside the Go heap. |
||
|
|
279effb4c2 |
Bound the event log's rendered bodies in the query (closes #135)
check / check (push) Successful in 3m0s
The event log rendered stored bodies untruncated. Since buffered rendering landed (#123) that became resident memory per concurrent viewer, up to tens of MB, driven by payloads unauthenticated clients supply to the public receiver. Bound in the query rather than the template, via substr(cast(body as blob), 1, ?) plus length(cast(body as blob)), so an oversized body never becomes a Go string at all. Adds an EventLogView projection carrying the true byte count, and trims a partial UTF-8 tail without rewriting bodies that are merely invalid UTF-8. Independently reviewed. The generated SQL was dumped under GORM DryRun to confirm the cap is a bound parameter, both casts are present, and no other path selects the full column; soft-delete scope, ordering and pagination are unchanged. Correction to the PR body: its quoted mutation output was produced by removing the bound from eventLogColumns, not by raising the cap to 1<<30 as the text claimed. The reviewer reproduced the real mutation and confirmed the tests do catch removal of the bound. Follow-up #157 restores in-app retrieval of bodies above the cap. |
||
|
|
7f8469a0f2 |
feat: implement core webhook engine, delivery system, and management UI (Phase 2)
check / check (push) Successful in 1m49s
- Webhook reception handler: look up entrypoint by UUID, verify active,
capture full HTTP request (method, headers, body, content-type), create
Event record, queue Delivery records for each active Target, return 200 OK.
Handles edge cases: unknown UUID → 404, inactive → 410, oversized → 413.
- Delivery engine (internal/delivery): fx-managed background goroutine that
polls for pending/retrying deliveries and dispatches to target type handlers.
Graceful shutdown via context cancellation.
- Target type implementations:
- HTTP: fire-and-forget POST with original headers forwarding
- Retry: exponential backoff (1s, 2s, 4s...) up to max_retries
- Database: immediate success (event already stored)
- Log: slog output with event details
- Webhook management pages with Tailwind CSS + Alpine.js:
- List (/sources): webhooks with entrypoint/target/event counts
- Create (/sources/new): form with auto-created default entrypoint
- Detail (/source/{id}): config, entrypoints, targets, recent events
- Edit (/source/{id}/edit): name, description, retention_days
- Delete (/source/{id}/delete): soft-delete with child records
- Add Entrypoint (/source/{id}/entrypoints): inline form
- Add Target (/source/{id}/targets): type-aware form
- Event Log (/source/{id}/logs): paginated with delivery status
- Updated README: marked completed items, updated naming conventions
table, added delivery engine to package layout and DI docs, updated
column names to reflect entity rename.
- Rebuilt Tailwind CSS for new template classes.
Part of: #15
|