13de7cd290d59423b441814038824a0b8442963a
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
clawbot
|
13de7cd290 |
Allow retention_days of 0 to mean retain forever (closes #79)
All checks were successful
check / check (push) Successful in 2m43s
RetentionDays carried gorm:"default:30", so GORM substituted 30 for a zero value while building the insert. A webhook could therefore never be configured to keep its events indefinitely: the reaper's retain-forever branch existed but was unreachable from the normal create and edit flows. Introduce database.RetentionForeverDays = 365 * 1000 as the sentinel for "retain forever" and a Webhook.BeforeSave hook that rewrites any non-positive RetentionDays to it. The rewrite has to live in the hook rather than at the call sites: GORM applies the column default while converting the model to insert values, which happens after BeforeSave, so anything later loses that race. Putting it on the model also means a future call site, such as the planned REST API, cannot bypass it. The reaper now skips a webhook when Webhook.RetainsForever reports true, which recognises the sentinel and keeps honouring the old <= 0 values for rows written before it existed. Without this the sentinel, being positive, would have produced a cutoff a thousand years in the past and a DELETE matching nothing on every sweep. Bound the finite retention range, which was previously unbounded on the server. The reaper computes its cutoff as a time.Duration, an int64 nanosecond count, so a day count above 106751 overflows, wraps the span negative, and moves the cutoff into the far future — where it matches every row and the sweep deletes every event, delivery, and delivery result the webhook has, including ones created seconds ago. Nothing rejected such a value: parseRetention accepted any v > 0, and max="365" was a client-side attribute a direct POST ignored, so the wipe was already reachable on main and removing that attribute would have made it reachable by ordinary use. The bound is database.MaxFiniteRetentionDays, derived from the arithmetic itself as math.MaxInt64 / time.Hour / hoursPerDay rather than picked as a round number, and a finite value above it is now a 400 that names the ceiling. retentionCutoff additionally clamps the day count it is given and reports whether any cutoff applies at all, so a row written by an older version, a migration, or a future call site cannot reach the overflow either. A value at or above the retain-forever sentinel stays accepted, because that is what the edit form pre-fills for a retain-forever webhook. Form handling is shared by create and edit through parseRetentionDays so the two cannot drift: an empty field keeps the previous behaviour (default on create, unchanged on edit), 0 is honoured, and an unparseable, negative, or out-of-range value is a 400 that re-renders the form rather than a silently substituted default. The two rejection reasons are distinct sentinel errors so the message can name the ceiling, and the create form now carries the submitted name and description back into the re-rendered inputs, which the edit form already did. The retention inputs drop max="365". That cap was not cosmetic: the edit form pre-fills the stored value, so a retain-forever webhook rendered 365000 into an input capped at 365 and browser validation would have blocked saving any edit to it. min becomes 0 with a hint explaining what 0 does, and the list and detail views render a RetentionLabel of "forever" instead of a raw day count. All three Webhook methods take pointer receivers, so there is no receiver mix and no lint suppression: BeforeSave must take a pointer to mutate the record, and the handlers hand templates a *Webhook because html/template cannot call a pointer method on a value held in a map. The 30-day default is consolidated into database.DefaultRetentionDays, referenced from the handler and from the create form's pre-filled value, with a test asserting it agrees with the struct tag that cannot reference it. |
||
| ee7c626071 |
Implement the database archiving target (closes #43) (#84)
All checks were successful
check / check (push) Successful in 4s
Implements the `databaseTarget` as a real archiving target, replacing the always-successful stub. Delivering to a `database` target now writes the full event into a per-webhook archive SQLite file for long-term storage.
## Archive-writer semantics
- **Separate file:** each webhook's full events are written as rows into `archive-{webhookID}.db` under the data dir, distinct from the per-webhook event DB (`events-{webhookID}.db`). The file and its schema are created on first write if missing. Each row carries the full event: body, headers, method, content type, webhook id, entrypoint id, event id, and an archived-at timestamp.
- **Close/reopen with debounce:** after each write the archive handle is closed and reopened, unless the last (re)open was less than one second ago. This lets an operator move the archive file away for offline archiving while bounding file churn under load. A per-webhook `archiveWriter` owns this debounce state and serialises writes.
- **Auto-recreate:** the file is opened create-if-missing (`mode=rwc`) and its schema re-migrated on every open, so if the archive was moved or removed since the last open, the next write recreates it. The writer also detects a missing file before writing and reopens first, so a moved-away file is recreated rather than lost.
- **Optional expiry, validated at creation:** an optional `expiry` in the target's config JSON (e.g. `{"expiry":"720h"}`) is validated when the target is created (`ValidateArchiveExpiry`; bad values are rejected with a 400 at the add-target form, the Slack URL precedent). The default (missing, empty, or `"never"`) keeps rows forever with no pruning. When a positive duration is set, rows older than it (measured from each row's archived-at time) are pruned on every (re)open; because the file is reopened after writes, prune-on-open keeps the archive swept without a separate background sweeper. A set-but-invalid expiry in a stored config (unparseable, zero, or negative) is an error at delivery time too — never a silent default.
- **No-retry, fail-loud:** the target performs a single attempt with no retries. On success it records one successful attempt and marks the delivery delivered. If the archive write fails, the attempt is recorded as failed with the error and the delivery is marked failed — archiving errors never report success.
## Scope
- `internal/delivery/target_database.go` — the `databaseTarget` (no-retry) archives via a per-webhook writer registry; an archive error records a failed attempt and marks the delivery failed.
- `internal/delivery/target_database_archive.go` (new) — the `archiveWriter`, the archived-row model, config/expiry parsing (fail-loud on set-but-invalid values), `ValidateArchiveExpiry`, and prune-on-open.
- `internal/handlers/source_management.go` — database targets get a creation-validated `expiry` config (`buildDatabaseTargetConfig`); the expiry form value is read where the request body is bounded and bad values are rejected with a 400 at target creation.
- `templates/source_detail.html` — the add-target form shows an expiry field for database targets.
- `README.md` — the database-target documentation describes the archiving semantics.
- `internal/delivery/export_test.go`, `internal/delivery/target_database_test.go`, `internal/handlers` tests — tests and their exported shims.
No changes to the `Target` interface or other targets.
## Tests
- a row is archived (both at the writer level and end-to-end through `Deliver`)
- a forced archive failure (bad stored expiry config) yields a `Failed` delivery with a non-success `DeliveryResult` carrying the error and no archive file created
- the file is recreated after removal, with only the post-removal row
- the one-second reopen debounce (rapid writes reopen once; a write after the window reopens again)
- expiry pruning removes rows older than the configured expiry
- expiry config parsing (empty / `never` / duration accepted; unparseable, zero, and negative values error)
- expiry validation at target creation (`TestValidateArchiveExpiry`; valid values build the config, bad values get a 400)
## Validation
`docker build .` exits 0 (fmt-check, lint, test, build all pass).
Closes #43
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #84
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
|
|||
| 60786c5019 |
feat: add CSRF protection, SSRF prevention, and login rate limiting (#42)
All checks were successful
check / check (push) Successful in 4s
## Security Hardening This PR implements three security hardening issues: ### CSRF Protection (closes #35) - Session-based CSRF tokens with cryptographically random 256-bit generation - Constant-time token comparison to prevent timing attacks - CSRF middleware applied to `/pages`, `/sources`, `/source`, and `/user` routes - Hidden `csrf_token` field added to all 12+ POST forms in templates - Excluded from `/webhook` (inbound webhook POSTs) and `/api` (stateless API) ### SSRF Prevention (closes #36) - `ValidateTargetURL()` blocks private/reserved IP ranges at target creation time - Blocked ranges: `127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`, `::1`, `fc00::/7`, `fe80::/10`, plus multicast, reserved, test-net, and CGN ranges - SSRF-safe HTTP transport with custom `DialContext` in the delivery engine for defense-in-depth (prevents DNS rebinding attacks) - Only `http` and `https` schemes allowed ### Login Rate Limiting (closes #37) - Per-IP rate limiter using `golang.org/x/time/rate` - 5 attempts per minute per IP on `POST /pages/login` - GET requests (form rendering) pass through unaffected - Automatic cleanup of stale per-IP limiter entries every 5 minutes - `X-Forwarded-For` and `X-Real-IP` header support for reverse proxies ### Files Changed **New files:** - `internal/middleware/csrf.go` + tests — CSRF middleware - `internal/middleware/ratelimit.go` + tests — Login rate limiter - `internal/delivery/ssrf.go` + tests — SSRF validation + safe transport **Modified files:** - `internal/server/routes.go` — Wire CSRF and rate limit middleware - `internal/handlers/handlers.go` — Inject CSRF token into template data - `internal/handlers/source_management.go` — SSRF validation on target creation - `internal/delivery/engine.go` — SSRF-safe HTTP transport for production - All form templates — Added hidden `csrf_token` fields - `README.md` — Updated Security section and TODO checklist `docker build .` passes (lint + tests + build). Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Co-authored-by: clawbot <clawbot@eeqj.de> Co-authored-by: Jeffrey Paul <sneak@noreply.example.org> Reviewed-on: #42 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
|||
| 8d702a16c6 |
feat: add Slack target type for incoming webhook notifications (#47)
All checks were successful
check / check (push) Successful in 4s
## Summary Adds a new `slack` target type that sends webhook events as formatted messages to any Slack-compatible incoming webhook URL (Slack, Mattermost, and other compatible services). closes #44 ## What it does When a webhook event is received, the Slack target: 1. Formats a human-readable message with event metadata (HTTP method, content type, timestamp, body size) 2. Pretty-prints the payload in a code block — JSON payloads get indented formatting, non-JSON payloads are shown as raw text 3. Truncates large payloads at 3500 characters to keep Slack messages reasonable 4. POSTs the message as a `{"text": "..."}` JSON payload to the configured webhook URL ## Changes - **`internal/database/model_target.go`** — Add `TargetTypeSlack` constant - **`internal/delivery/engine.go`** — Add `SlackTargetConfig` struct, `deliverSlack` method, `FormatSlackMessage` function (exported), `parseSlackConfig` helper. Route slack targets in `processDelivery` switch. - **`internal/handlers/source_management.go`** — Handle `slack` type in `HandleTargetCreate`, building `webhook_url` config from the URL form field - **`templates/source_detail.html`** — Add "Slack" option to target type dropdown with URL field and helper text - **`README.md`** — Document the new target type, update roadmap ## Tests - `TestParseSlackConfig_Valid` / `_Empty` / `_MissingWebhookURL` — Config parsing - `TestFormatSlackMessage_JSONBody` / `_NonJSONBody` / `_EmptyBody` / `_LargeJSONTruncated` — Message formatting - `TestDeliverSlack_Success` / `_Failure` / `_InvalidConfig` — End-to-end delivery - `TestProcessDelivery_RoutesToSlack` — Routing from processDelivery switch All existing tests continue to pass. `docker build .` (which runs `make check`) passes clean. Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #47 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
|||
clawbot
|
25e27cc57f |
refactor: merge retry target type into http (max_retries=0 = fire-and-forget)
All checks were successful
check / check (push) Successful in 1m46s
|
||
|
clawbot
|
f21a007a3c |
feat: add entrypoint/target management controls (closes #25)
Add toggle (activate/deactivate) and delete buttons for individual
entrypoints and targets on the webhook detail page. Each action is a
POST form submission with ownership verification.
New routes:
POST /source/{id}/entrypoints/{entrypointID}/delete
POST /source/{id}/entrypoints/{entrypointID}/toggle
POST /source/{id}/targets/{targetID}/delete
POST /source/{id}/targets/{targetID}/toggle
|
||
|
clawbot
|
7f8469a0f2 |
feat: implement core webhook engine, delivery system, and management UI (Phase 2)
All checks were successful
check / check (push) Successful in 1m49s
- Webhook reception handler: look up entrypoint by UUID, verify active,
capture full HTTP request (method, headers, body, content-type), create
Event record, queue Delivery records for each active Target, return 200 OK.
Handles edge cases: unknown UUID → 404, inactive → 410, oversized → 413.
- Delivery engine (internal/delivery): fx-managed background goroutine that
polls for pending/retrying deliveries and dispatches to target type handlers.
Graceful shutdown via context cancellation.
- Target type implementations:
- HTTP: fire-and-forget POST with original headers forwarding
- Retry: exponential backoff (1s, 2s, 4s...) up to max_retries
- Database: immediate success (event already stored)
- Log: slog output with event details
- Webhook management pages with Tailwind CSS + Alpine.js:
- List (/sources): webhooks with entrypoint/target/event counts
- Create (/sources/new): form with auto-created default entrypoint
- Detail (/source/{id}): config, entrypoints, targets, recent events
- Edit (/source/{id}/edit): name, description, retention_days
- Delete (/source/{id}/delete): soft-delete with child records
- Add Entrypoint (/source/{id}/entrypoints): inline form
- Add Target (/source/{id}/targets): type-aware form
- Event Log (/source/{id}/logs): paginated with delivery status
- Updated README: marked completed items, updated naming conventions
table, added delivery engine to package layout and DI docs, updated
column names to reflect entity rename.
- Rebuilt Tailwind CSS for new template classes.
Part of: #15
|
