5 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
clawbot	24af4b4200	Harden operator-set target headers (closes #233) ... Three findings from the review of the per-target request headers feature, plus the follow-up they raised about the inbound headers the same delivery path forwards. One rule now governs every header a delivery carries on someone else's behalf: a redirect hop that leaves the origin the target names carries none of them. That covers the operator's configured headers and the inbound event headers forwarded from the sender alike. net/http withholds only Authorization and Cookie across a host change, so an operator's X-Api-Key or a sender's X-Hub-Signature would otherwise follow a 302 to a host nobody configured. Redirects are still followed — refusing them would break every destination that legitimately redirects and would record the 3xx as the delivery's result — but a hop to another host, another port, or down from https to http drops the lot. The shared SSRF-safe transport is kept on that client, so each hop is still dialled through the private-IP guard. The set to strip is not a name list. applyRequestHeaders now returns the canonical names of everything it applied on the sender's or operator's behalf, and the redirect policy strips exactly that, so a header added to the forward set is covered without a second edit. Content-Type and User-Agent are the delivery path's own rather than anyone else's, and both are excluded from that set so they always travel: Content-Type is set from the event and a 307 preserves the body across hosts, so it has to stay typed, and User-Agent is overwritten with this delivery path's own after the forwarded headers are applied, so the sender's never reaches the wire and stripping it off-origin would only substitute net/http's default. The origin comparison no longer collapses two IPv6 origins into one. Hostname() unwraps a literal's brackets, so re-appending the port with a bare colon rendered https://[2001:db8::1]:8080 and https://[2001:db8::1:8080] identically — a different address on a different port passing as the same origin. The port is joined with net.JoinHostPort, and both spellings are in TestSameDeliveryOrigin. The ten-hop cap gains a regression test. Installing a CheckRedirect is precisely what discards net/http's own limit, so a self-redirecting destination is driven through the policy and asserted to stop after exactly ten requests with the sentinel surfacing to the caller. Trailer joins the reserved names. net/http strips it from the request it writes, so a configured one was accepted, stored, and provably never sent. The invalid-header-name error no longer quotes the text before the first colon. That text is only a name if it parses as one; when it does not, a pasted value whose own colon split the line put half a token into a 400 body. TestParseTargetHeaders_ErrorsNeverQuoteAValue asserted this invariant while only exercising the after-the-colon case, and now covers the before-the-colon one. README documents the http target's config keys, the 300-second timeout ceiling, the reserved-header list and the redirect behaviour as one rule over both header classes, including that the drop is per hop rather than permanent: net/http re-copies the initial request's headers each hop, so a chain returning to the configured origin carries them again, exactly as it treats Authorization. It also records what following a 301, 302 or 303 costs, since that is net/http's own behaviour and the decision to follow redirects is what buys it: the POST becomes a GET and the event body and its Content-Type are dropped, so the destination the chain ends at receives no event while the delivery is still recorded Delivered. The edit form's hint gains Trailer and the redirect note. Closes #243	2026-08-20 08:39:21 +00:00
clawbot	d2cebb5783	Correct TODO.md milestone state and record seventeen landed units ... The Status section claimed next held the completed 1.0.0 milestone with every issue closed. Four are open (#176, #178, #186, #187), so a merge of next to main would have shipped that claim to main. Next Step still named #115 and #125 as open owner decisions; both landed. It now names the real open items, #150 and #112, and the forced merge order for the remaining four. Completed Steps was seventeen units behind, back to 2026-08-12.	2026-08-18 02:06:43 +00:00
clawbot	4dd4dfa5eb	chore: consolidate DBURL into DATA_DIR, codebase audit for 1.0.0 ... DBURL → DATA_DIR consolidation: - Remove DBURL env var entirely; main DB now lives at {DATA_DIR}/webhooker.db - database.go constructs DB path from config.DataDir, ensures dir exists - Update DATA_DIR prod default from /data/events to /data - Update all tests to use DataDir instead of DBURL - Update Dockerfile: /data (not /data/events) for all SQLite databases - Update README configuration table, Docker examples, architecture docs Dead code removal: - Remove unused IndexResponse struct (handlers/index.go) - Remove unused TemplateData struct (handlers/handlers.go) Stale comment cleanup: - Remove TODO in server.go (DB cleanup handled by fx lifecycle) - Fix nolint:golint → nolint:revive on ServerParams for consistency - Clean up verbose middleware/routing comments in routes.go - Fix TODO fan-out description (worker pool, not goroutine-per-target) .gitignore fixes: - Add data/ directory to gitignore - Remove stale config.yaml entry (env-only config since rework)	2026-03-01 23:33:20 -08:00
clawbot	32bd40b313	refactor: self-contained delivery tasks — engine delivers without DB reads in happy path ... The webhook handler now builds DeliveryTask structs carrying all target config and event data inline (for bodies ≤16KB) and sends them through the delivery channel. In the happy path, the engine delivers without reading from any database — it only writes to record delivery results. For large bodies (≥16KB), Body is nil and the engine fetches it from the per-webhook database on demand. Retry timers also carry the full DeliveryTask, so retries avoid unnecessary DB reads. The database is used for crash recovery only: on startup the engine scans for interrupted pending/retrying deliveries and re-queues them. Implements owner feedback from issue #15: > the message in the <=16KB case should have everything it needs to do > its delivery. it shouldn't touch the db until it has a success or > failure to record.	2026-03-01 22:09:41 -08:00
clawbot	43c22a9e9a	feat: implement per-webhook event databases ... Split data storage into main application DB (config only) and per-webhook event databases (one SQLite file per webhook). Architecture changes: - New WebhookDBManager component manages per-webhook DB lifecycle (create, open, cache, delete) with lazy connection pooling via sync.Map - Main DB (DBURL) stores only config: Users, Webhooks, Entrypoints, Targets, APIKeys - Per-webhook DBs (DATA_DIR) store Events, Deliveries, DeliveryResults in files named events-{webhook_uuid}.db - New DATA_DIR env var (default: ./data dev, /data/events prod) Behavioral changes: - Webhook creation creates per-webhook DB file - Webhook deletion hard-deletes per-webhook DB file (config soft-deleted) - Event ingestion writes to per-webhook DB, not main DB - Delivery engine polls all per-webhook DBs for pending deliveries - Database target type marks delivery as immediately successful (events are already in the dedicated per-webhook DB) - Event log UI reads from per-webhook DBs with targets from main DB - Existing webhooks without DB files get them created lazily Removed: - ArchivedEvent model (was a half-measure, replaced by per-webhook DBs) - Event/Delivery/DeliveryResult removed from main DB migrations Added: - Comprehensive tests for WebhookDBManager (create, delete, lazy creation, delivery workflow, multiple webhooks, close all) - Dockerfile creates /data/events directory README updates: - Per-webhook event databases documented as implemented (was Phase 2) - DATA_DIR added to configuration table - Docker instructions updated with data volume mount - Data model diagram updated - TODO updated (database separation moved to completed) Closes #15	2026-03-01 17:06:43 -08:00