62 Commits

Author SHA1 Message Date
e83eb2977e Bound shutdown hooks by their stop context (closes #102)
All checks were successful
check / check (push) Successful in 3m45s
fx hands OnStop a context carrying the application's stop timeout,
and the delivery engine, the retention reaper, and the archive
sweeper all discarded it and called wg.Wait() bare. A worker wedged
inside a delivery target that never returns, or a sweep blocked on a
locked SQLite database, hung the process forever instead of letting
it exit when the timeout expired.

All three now wait through internal/lifecycle.WaitForShutdown, which
selects the drained WaitGroup against the stop context and, on
timeout, logs at error naming the component and returns an error
rather than reporting a clean stop.

Engine.stop also gains the cancel != nil guard its two mirrored
components already had.
2026-08-12 09:45:25 +00:00
d19e33671c Gate forwarded-header trust behind trusted-proxy config (closes #88)
All checks were successful
check / check (push) Successful in 6s
All three rate limiters (receiver, login, password change) now key on the connection's own address unless the direct peer is inside the new TRUSTED_PROXIES CIDR list, in which case X-Forwarded-For is walked right to left for the first non-proxy hop. Default is the empty list, which trusts nothing. A set-but-unparseable value aborts startup.
2026-08-12 11:36:10 +02:00
aab448b076 Clarify web UI terminology, copy, and the entrypoint URL (closes #57)
All checks were successful
check / check (push) Successful in 4s
Unifies user-visible copy on "Webhook" (routes and URLs unchanged), drops
the placeholder Profile settings section, and adds a copy-to-clipboard
affordance for the entrypoint URL as progressive enhancement — the button
stays hidden unless both the target element and the Clipboard API resolve,
so no dead control appears without JavaScript and the URL stays selectable.

Retention copy now matches what the code does: deletion is permanent, 0
retains forever, and a blank field means the default on create or the
current value on edit. The permanent-deletion sentence is suppressed for a
retain-forever webhook, which the reaper exempts before computing a cutoff.

Template tests gained a render-completed assertion. Without it, a page that
aborted mid-render still satisfied assertions matching the already-flushed
prefix, because renderTemplate streams to the ResponseWriter (#123).
2026-08-11 15:42:08 +02:00
7c43e095a6 Mask the webhook credential in delivery errors and logs (closes #118)
All checks were successful
check / check (push) Successful in 9s
Go embeds the request URL in *url.Error, so any transport failure — DNS,
TLS, refused, timeout, SSRF dial block — persisted the full Slack webhook
URL into the per-webhook SQLite database via DeliveryResult.Error. That
field is tagged json:"error,omitempty", so a future REST API would have
served it.

maskURLError rebuilds the error preserving Op and the wrapped cause, so DNS
vs TLS vs timeout still read differently and errors.Is/As and Timeout()
keep working; only path, query and userinfo are dropped. Applied where the
errors are born, which covers both the Slack and HTTP targets. url.Parse
embeds the URL too, so ValidateTargetURL's parse branch gets the same
treatment.

The SSRF rejection log now logs the masked URL, and source_logs.html
receives view types rather than raw rows, so no config blob is reachable
from that template.

MaskURL is now the single masker for the whole tree.
2026-08-11 15:11:57 +02:00
84b758b785 Rate-limit the public webhook receiver endpoint (closes #64)
All checks were successful
check / check (push) Successful in 5s
The receiver was the one unauthenticated, internet-facing endpoint with no
rate limit, so a misbehaving or hostile sender could flood a webhook
without bound. RECEIVER_RATE_LIMIT (default 120/min) now caps it, keyed on
client IP plus entrypoint path so one entrypoint cannot exhaust another's
budget. Over-limit requests get 429 with Retry-After.

The limiter deliberately does not reuse postRateLimit: that helper is
POST-only and keys on IP alone, whereas the receiver must count every
method. A test locks that property in.

Config parsing follows the fail-loudly idiom: a set-but-unparseable or
non-positive value aborts startup rather than falling back to the default.

Known limitation, tracked in #88: the key still trusts forwarded headers
unconditionally, so the limit is evadable by rotating X-Forwarded-For until
trusted-proxy gating lands.
2026-08-11 14:47:21 +02:00
d51cd0fd29 Enforce the body size limit before CSRF parses the form (closes #90)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
CSRF ran before MaxBodySize, so the CSRF middleware parsed the form body
before any cap applied and an oversized request was read in full before
being rejected. MaxBodySize is now the first middleware in all four route
groups that parse forms, ahead of CSRF and RequireAuth.

An oversize request therefore gets 413 without the handler running and
without state changing, including the password-change route.

Note the ordering trade: an unauthenticated client now receives 413 rather
than an auth redirect on /user/{username}/password.
2026-08-11 14:37:38 +02:00
15a61173fc Mask target config on the source detail page (closes #113)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
The page rendered the stored target config verbatim, exposing the Slack
incoming-webhook URL, which is a bearer credential: anyone holding it can
post to the channel indefinitely, and it cannot be scoped or revoked
per-holder.

Target config now reaches the template only as a TargetView carrying
labelled fields, so no code path can render the raw blob. maskURL keeps
scheme and host and elides the path, and drops query, fragment and
userinfo; every parse failure yields a neutral placeholder rather than
falling back to the stored string. HTTP header values are never rendered,
only a count.

Rendering change only: the stored config format and the delivery path are
unchanged.
2026-08-11 14:37:09 +02:00
e50a79ced9 Allow retention_days of 0 to mean retain forever (closes #79)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
Rewrites retention_days=0 to the RetentionForeverDays sentinel (365 * 1000)
in Webhook.BeforeSave, so the GORM column default cannot win the race. The
reaper skips retain-forever webhooks before building any query.

Also bounds the reaper's cutoff arithmetic: a time.Duration is int64
nanoseconds, so day counts above MaxFiniteRetentionDays (106751) overflowed
and wrapped the cutoff into the future, where created_at < cutoff matched
every row and the sweep deleted everything. parseRetentionDays now rejects
finite values above the ceiling, and retentionCutoff saturates so rows
written by older versions cannot reach it either.

Views render RetentionLabel() rather than the raw sentinel.
2026-08-11 14:35:34 +02:00
c2cd2c440b Add inactivity-based session timeout (closes #66) (#105)
All checks were successful
check / check (push) Successful in 4s
Sessions now carry a server-enforced idle deadline (SESSION_IDLE_TIMEOUT,
default 24h) alongside the 7-day absolute cap, refreshed on authenticated
activity. Activity never extends the absolute cap.
2026-08-10 16:12:40 +02:00
45890d4f82 Fail loudly on set-but-unparseable env config values (closes #80) (#92)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
Defaults now apply only to unset or empty environment variables; a set-but-
unparseable value aborts startup with an error naming the key and the value.
envInt is gone, envBool parses with strconv.ParseBool, and PORT is bounded.
2026-08-10 16:06:12 +02:00
0ce8565f51 Terminally fail retrying deliveries with a non-retry target type (closes #82) (#104)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
A delivery left in `retrying` whose target type was edited to a fire-and-forget
or unknown type was skipped forever by both restart recovery and the retry
sweep. Both paths now record a result row and mark it `failed`.
2026-08-10 16:00:03 +02:00
3e261d2f01 Evict archive writers on deletion and sweep idle archives (closes #89) (#95)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
Per-webhook archive writers are now evicted when the webhook or its last
database target is deleted, and a background sweeper prunes expired rows from
idle archives that no longer receive writes. Archive files themselves are never
deleted.
2026-08-10 15:52:20 +02:00
62481a6f1a Root background loops at context.Background() (closes #97) (#100)
All checks were successful
check / check (push) Successful in 5s
The delivery engine worker pool and the retention reaper both rooted their
goroutines in the fx OnStart hook context, which fx cancels 15s into startup.
Both now use context.WithCancel(context.Background()), bounded by OnStop.
2026-08-10 15:44:56 +02:00
4f5ecb18e5 Add admin password change flow (closes #65) (#83)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
Adds an authenticated, CSRF-protected flow that lets a user change their own password from the profile page.

## Route

- New `POST /password` under the `/user/{username}` group in `setupUserRoutes` (`internal/server/routes.go`). That group already applies `CSRF`, `NoCache`, and `RequireAuth`, so the new endpoint inherits all three.

## Handler (`internal/handlers/profile.go`)

- `HandlePasswordChange` enforces own-user access: the `{username}` path parameter must equal the session username (same 403 rule `HandleProfile` uses). This check plus the session lookup is factored into a shared `profileOwnerOrDeny` helper now used by both handlers.
- Parses `current_password`, `new_password`, and `confirm_password` (body size limited via `http.MaxBytesReader`).
- Verifies the current password with `database.VerifyPassword` against the stored hash.
- Requires the new password to be non-empty and equal to the confirmation.
- Hashes the new password with `database.HashPassword` — the same Argon2id helper used to bootstrap the admin user — and persists it on the user row. No new crypto.
- Re-renders the profile page with a clear success or error message. Wrong current password, empty new password, and mismatched confirmation are each rejected with their own message and leave the stored hash unchanged.

## Template (`templates/profile.html`)

- Adds a "Change Password" card with current / new / confirm password fields plus the hidden `csrf_token` (matching the login form's CSRF embedding).
- Renders success/error alerts using the existing `alert-success` / `alert-error` styles. No new CSS classes, so no Tailwind rebuild is required.

## Tests (`internal/handlers/profile_test.go`)

- `TestHandlePasswordChange_Success`: seeds a user, posts a valid change, asserts success message and that the stored hash changed and verifies against the new password.
- `TestHandlePasswordChange_WrongCurrentPassword`: posts a wrong current password, asserts the rejection message and that the stored hash is unchanged.

Validated with `docker build .` (fmt-check, lint, test, build) — exit 0.

Closes #65

Co-authored-by: sneak <sneak@sneak.berlin>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #83
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 23:23:05 +02:00
734606b7af Update golangci-lint to v2.12.2 with canonical config (#86)
All checks were successful
check / check (push) Successful in 3s
Bumps golangci-lint from v2.11.3 to v2.12.2 and adopts the canonical lint config.

## Version pins

- `Dockerfile`: `golangci/golangci-lint:v2.12.2` Debian image, pinned by digest, dated `2026-08-07`
- `script/bootstrap`: `GOLANGCI_LINT_VERSION=2.12.2` with updated sha256 pins for the `linux-amd64` and `linux-arm64` release archives

## Config

`.golangci.yml` replaced with the canonical config. The previous file kept `lll`/`funlen`/`cyclop`/`dupl` settings under the top-level `linters-settings` key, which the v2 schema ignores; the canonical config nests them under `linters.settings`, so those thresholds now actually apply. The unsupported `issues.exclude-use-default` key was dropped.

## Lint fixes (32 findings)

- `lll` (7): wrapped or shortened over-length lines (struct tag comments moved above fields, test logger construction split, `session.NewForTest` signature wrapped, shortened a `#nosec` comment)
- `goconst` (17): replaced repeated `"POST"`/`"PUT"` literals with `http.MethodPost`/`http.MethodPut`, added shared test constants for `webhooker-test`/`test`/`application/json`, and added `tmplKeyError`/`tmplKeyWebhook` constants for template data keys in `internal/handlers`
- `dupl` (8): merged `buildHTTPTargetConfig` and `buildSlackTargetConfig` into a parameterized `buildURLTargetConfig`; removed the duplicate `iWebhookDB` test helper in favor of `testWebhookDB`; extracted shared helpers in middleware and session tests

No `//nolint` directives were added and behavior is unchanged. `make check` (fmt-check, tests, lint) passes.

Note: golangci-lint v2.12 deprecates the `gomodguard` linter in favor of `gomodguard_v2`; the canonical config change for that is left for a future coordinated update.
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #86
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 23:18:49 +02:00
ee7c626071 Implement the database archiving target (closes #43) (#84)
All checks were successful
check / check (push) Successful in 4s
Implements the `databaseTarget` as a real archiving target, replacing the always-successful stub. Delivering to a `database` target now writes the full event into a per-webhook archive SQLite file for long-term storage.

## Archive-writer semantics

- **Separate file:** each webhook's full events are written as rows into `archive-{webhookID}.db` under the data dir, distinct from the per-webhook event DB (`events-{webhookID}.db`). The file and its schema are created on first write if missing. Each row carries the full event: body, headers, method, content type, webhook id, entrypoint id, event id, and an archived-at timestamp.
- **Close/reopen with debounce:** after each write the archive handle is closed and reopened, unless the last (re)open was less than one second ago. This lets an operator move the archive file away for offline archiving while bounding file churn under load. A per-webhook `archiveWriter` owns this debounce state and serialises writes.
- **Auto-recreate:** the file is opened create-if-missing (`mode=rwc`) and its schema re-migrated on every open, so if the archive was moved or removed since the last open, the next write recreates it. The writer also detects a missing file before writing and reopens first, so a moved-away file is recreated rather than lost.
- **Optional expiry, validated at creation:** an optional `expiry` in the target's config JSON (e.g. `{"expiry":"720h"}`) is validated when the target is created (`ValidateArchiveExpiry`; bad values are rejected with a 400 at the add-target form, the Slack URL precedent). The default (missing, empty, or `"never"`) keeps rows forever with no pruning. When a positive duration is set, rows older than it (measured from each row's archived-at time) are pruned on every (re)open; because the file is reopened after writes, prune-on-open keeps the archive swept without a separate background sweeper. A set-but-invalid expiry in a stored config (unparseable, zero, or negative) is an error at delivery time too — never a silent default.
- **No-retry, fail-loud:** the target performs a single attempt with no retries. On success it records one successful attempt and marks the delivery delivered. If the archive write fails, the attempt is recorded as failed with the error and the delivery is marked failed — archiving errors never report success.

## Scope

- `internal/delivery/target_database.go` — the `databaseTarget` (no-retry) archives via a per-webhook writer registry; an archive error records a failed attempt and marks the delivery failed.
- `internal/delivery/target_database_archive.go` (new) — the `archiveWriter`, the archived-row model, config/expiry parsing (fail-loud on set-but-invalid values), `ValidateArchiveExpiry`, and prune-on-open.
- `internal/handlers/source_management.go` — database targets get a creation-validated `expiry` config (`buildDatabaseTargetConfig`); the expiry form value is read where the request body is bounded and bad values are rejected with a 400 at target creation.
- `templates/source_detail.html` — the add-target form shows an expiry field for database targets.
- `README.md` — the database-target documentation describes the archiving semantics.
- `internal/delivery/export_test.go`, `internal/delivery/target_database_test.go`, `internal/handlers` tests — tests and their exported shims.

No changes to the `Target` interface or other targets.

## Tests

- a row is archived (both at the writer level and end-to-end through `Deliver`)
- a forced archive failure (bad stored expiry config) yields a `Failed` delivery with a non-success `DeliveryResult` carrying the error and no archive file created
- the file is recreated after removal, with only the post-removal row
- the one-second reopen debounce (rapid writes reopen once; a write after the window reopens again)
- expiry pruning removes rows older than the configured expiry
- expiry config parsing (empty / `never` / duration accepted; unparseable, zero, and negative values error)
- expiry validation at target creation (`TestValidateArchiveExpiry`; valid values build the config, bad values get a 400)

## Validation

`docker build .` exits 0 (fmt-check, lint, test, build all pass).

Closes #43

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #84
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 22:50:08 +02:00
81413c56e9 Refactor delivery targets to a Target interface (closes #77) (#81)
All checks were successful
check / check (push) Successful in 2m42s
Refactors the delivery engine so each target TYPE is an implementation of a `Target` interface, dispatched from a registry, with each target owning its full delivery including durable retries. Implements the authoritative design from issue #77 (the corrected "hand the DB + Scheduler to the target" design).

## The new interface

```go
type Scheduler interface {
    ScheduleRetry(task Task, delay time.Duration)
}

type Target interface {
    Deliver(ctx context.Context, webhookDB *gorm.DB,
        d *database.Delivery, task *Task, sched Scheduler)
}
```

`Deliver` receives everything a target needs to be autonomous and durable: the request context, the per-webhook `*gorm.DB`, the `*database.Delivery`, the attempt `*Task`, and a `Scheduler` (the engine) for durable re-enqueue. The target makes one attempt, writes the `DeliveryResult`, updates `DeliveryStatus`, and — for retry targets — decides whether to retry, computes its own backoff, gates with its own circuit breaker, and reschedules via the injected `Scheduler`.

`processDelivery` collapses to a registry lookup (`map[database.TargetType]Target`) and a `Deliver` call; an unknown target type still fails the delivery as before.

## Per-target ownership

- `httpTarget` and `slackTarget` share a retry core (`httpCore`) that owns retry, exponential backoff, and the per-target circuit breaker. The core is fire-and-forget when `MaxRetries == 0` and adds breaker-gated backed-off retries when `MaxRetries > 0`. The per-attempt request differs (HTTP forwards the body + filtered headers; Slack posts a formatted message) and is supplied as a closure, so each keeps its exact recording semantics (e.g. HTTP records no error string for a non-2xx, Slack records `HTTP <code>`).
- `databaseTarget` and `logTarget` are fire-and-forget: they record a single successful attempt.

Moved wholesale into the http/slack targets: `deliverHTTP*`, `handleHTTPRetry`, `circuitBreakerBlock`, `calcBackoff` / `calcRemainingBackoff` / `backoffElapsed`, the circuit-breaker `sync.Map` + `getCircuitBreaker`, `clientForConfig`, `doHTTPRequest`, `applyRequestHeaders`, and the config parsers. The engine keeps `recordResult`, `updateDeliveryStatus`, and `ScheduleRetry`.

## Slack MaxRetries gating

Slack is now on the same shared core as HTTP, with retry + breaker gated on `MaxRetries`. A `MaxRetries` of 0 stays single-attempt fire-and-forget, so **every existing Slack target is unchanged**; a Slack target configured with retries gets backoff + circuit breaker.

## Log-target full content

`logTarget` now logs the ENTIRE inbound webhook — full request body and full request headers, plus method, content type, and the webhook id and entrypoint id — rather than a summary line. This supersedes the smaller log-summary work (#70).

## `Task.EntrypointID`

To carry the entrypoint id to the log target, `Task` gains an `EntrypointID` field, populated in the webhook handler's `buildDeliveryTasks`, the engine's recovery-task builder, and `buildEventFromTask`.

## Durability / recovery

The crash-durable async retry model is preserved unchanged: one attempt per worker turn; on failure the status is set `retrying`, backoff is computed, and the task is re-enqueued via `ScheduleRetry` (a `time.AfterFunc` onto the retry channel). On restart, `recoverRetryingDeliveries` and the 60s sweep hand each orphaned `retrying` delivery back to its target to recompute the remaining backoff and reschedule (targets that own retries implement an internal `rescheduler`; fire-and-forget targets, which never produce `retrying` deliveries, are skipped).

## How behaviour is preserved

No external behaviour changes except the two called out above (log target full content; Slack gaining `MaxRetries`-gated retries). All existing delivery tests pass with only their `export_test.go` wrappers re-pointed at the new structure — `ExportDeliverHTTP/Slack/Database/Log` now call the targets, `ExportGetCircuitBreaker` / `ExportClient` / `ExportClientForConfig` / `ExportDoHTTPRequest` resolve against the HTTP target's shared client and breaker map, and `ExportParseHTTPConfig` / `ExportParseSlackConfig` call the relocated free functions. Added: a `logTarget` test asserting the log line contains the full body, headers, and ids, and a Slack `MaxRetries`-gated retry test.

`docker build .` is green (fmt-check, lint, test, static build all pass).

Closes #77

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #81
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 17:07:49 +02:00
f6b929f2d7 Add per-webhook event retention reaper (closes #63) (#78)
All checks were successful
check / check (push) Successful in 2m42s
Enforces each webhook's `RetentionDays` so per-webhook SQLite files no longer grow without bound.

## Reaper

New `RetentionReaper` in `internal/database/retention.go`. A background ticker runs each sweep: it lists all webhooks from the main DB and, for each webhook with a positive `RetentionDays`, opens its per-webhook DB via `WebhookDBManager.GetDB` and deletes every `Event` (and its dependent `Delivery` and `DeliveryResult` rows) whose `CreatedAt` is older than `RetentionDays` days.

- Deletions run in foreign-key-safe order: delivery results, then deliveries, then events.
- Deletes are unscoped (hard deletes) so rows are physically removed and disk is reclaimed, rather than GORM soft-deleting them.
- `RetentionDays <= 0` means retain forever; those webhooks are skipped.
- Webhooks whose per-webhook DB does not yet exist are skipped.

## Config

`internal/config/config.go` gains `RetentionSweepInterval` (env `RETENTION_SWEEP_INTERVAL`, parsed as a Go duration, default `1h`) via a new `envDuration` helper, following the existing env-helper conventions.

## Wiring

`cmd/webhooker/main.go` registers `database.NewRetentionReaper` as an fx provider and forces its construction in `fx.Invoke`. The reaper starts its sweep loop on an fx `OnStart` hook and stops cleanly on `OnStop` via context cancellation, matching the existing lifecycle components.

## Test

`internal/database/retention_test.go` seeds an old event chain (event + delivery + result, 40 days old) and a recent one (1 day old) in a real per-webhook DB and asserts a single sweep removes only the expired chain while keeping the recent one. A second test forces a non-positive `RetentionDays` and asserts an ancient event is retained.

Note: the `Webhook.RetentionDays` column carries `gorm:"default:30"`, so a `0` passed to a GORM `Create` is replaced by the default; the test forces the value with an explicit column update to exercise the retain-forever path. No model changes were made.

Validated with `docker build .` (fmt-check, lint, test, build) exit 0.

Closes #63

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #78
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 16:15:13 +02:00
8ea7f76540 Add NoCache middleware for authenticated pages (closes #61) (#75)
All checks were successful
check / check (push) Successful in 6s
Adds a `NoCache()` middleware that sets `Cache-Control: no-store` and `Pragma: no-cache`, and wires it onto the dynamic app route groups (`/pages`, `/user/{username}`, `/sources`, `/source/{sourceID}`) adjacent to their existing `CSRF()` call. The static `/s` mount, `/metrics`, `/webhook/{uuid}`, and `/.well-known/healthcheck` are intentionally left untouched (static assets are safe to cache; the others are not authenticated pages).

A middleware unit test asserts both headers are set.

Closes #61

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #75
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 15:33:44 +02:00
752d6beead Validate Slack target URLs at creation time (closes #68) (#73)
All checks were successful
check / check (push) Successful in 4s
Slack delivery targets were only checked by the request-time dialer guard, not at creation, giving them a weaker SSRF gate than HTTP targets.

This validates the Slack incoming-webhook URL with `delivery.ValidateTargetURL` in the Slack target creation path (`buildSlackTargetConfig`), before persisting, mirroring the existing HTTP-target path. On failure the create is rejected with the same clear, non-leaking user-facing error the HTTP path uses.

Adds handlers-package tests covering both an accepted public URL and a rejected private/reserved URL. Confined to `internal/handlers/`; `internal/delivery/` is unchanged.

Closes #68

Co-authored-by: sneak <sneak@sneak.berlin>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #73
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 14:03:56 +02:00
b1f43c9520 Keep the SSRF-safe transport in clientForConfig (closes #69) (#74)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
`clientForConfig()` in `internal/delivery/engine.go` built a fresh `http.Client` without a Transport when a per-target timeout was configured, dropping the request-time private-IP guard for that path.

It now reuses the shared client's SSRF-safe transport (`e.client.Transport`, the same `NewSSRFSafeTransport` instance), overriding only the `Timeout`. Behaviour is unchanged when no per-target timeout is set (the shared client is returned as before), so no engine code path makes an outbound target request with a client lacking the SSRF-safe transport.

Adds a delivery-package test proving a client from `clientForConfig()` with a per-target timeout still refuses private/reserved/link-local destinations, that the timeout is applied, that the SSRF-safe transport is reused (not duplicated), and that the no-timeout path returns the shared client unchanged.

Confined to `internal/delivery/` only; handlers and server code untouched.

Closes #69

Co-authored-by: sneak <sneak@sneak.berlin>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #74
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 14:03:38 +02:00
07fc63d9fa Wrap /user/{username} in RequireAuth middleware (closes #60) (#71)
All checks were successful
check / check (push) Successful in 4s
Enforces authentication for the `/user/{username}` route group at the middleware layer, matching every other authenticated route group.

## Changes

- **`internal/server/routes.go`** (`setupUserRoutes`): added `r.Use(s.mw.RequireAuth())` immediately after the existing `r.Use(s.mw.CSRF())` on the `/user/{username}` group, so auth is enforced by design (CSRF first, then RequireAuth) — consistent with `/sources` and `/source/{sourceID}`.
- **`internal/handlers/profile.go`** (`HandleProfile`): removed the now-dead unauthenticated-redirect branch (RequireAuth guarantees an authenticated session before the handler runs). The handler still reads the username and user id from the session for the own-profile-only check; a request for another user's profile still returns 403. The session-retrieval error is now handled as a 500.

## Tests (`internal/handlers/profile_test.go`)

- own profile returns 200
- another user's profile returns 403
- an unauthenticated request to `/user/{username}` is redirected to `/pages/login` at the middleware layer and never reaches the endpoint handler (routing-level test replicating the CSRF + RequireAuth chain)

## Validation

`docker build .` (fmt-check, lint, test, build) passes.

Closes #60

Co-authored-by: sneak <sneak@sneak.berlin>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #71
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 14:00:16 +02:00
0c9c885d51 Raise HTTP WriteTimeout above the request middleware timeout (closes #62) (#72)
All checks were successful
check / check (push) Successful in 4s
Raise `httpWriteTimeout` in `internal/server/http.go` from 10s to `65 * time.Second` so it comfortably exceeds the router's 60s `requestTimeout`. This makes the `middleware.Timeout(60s)` the effective request limit — a slow response now returns a clean 503 from the middleware instead of being cut at the socket write deadline by the transport.

`httpReadTimeout` stays at 10s. A comment on `httpWriteTimeout` documents that it must remain above the 60s request timeout. Change is confined to `internal/server/http.go`; `routes.go` is untouched.

Closes #62

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #72
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 13:58:28 +02:00
2cc8723997 scripts-to-rule-them-all (#59)
All checks were successful
check / check (push) Successful in 4s
Reviewed-on: #59
Co-authored-by: sneak <sneak@sneak.berlin>
Co-committed-by: sneak <sneak@sneak.berlin>
2026-07-07 02:14:09 +02:00
e0b1e7cf54 Restore TODO.md and move TODO content out of README (#58)
All checks were successful
check / check (push) Successful in 4s
Reviewed-on: #58
Co-authored-by: sneak <sneak@sneak.berlin>
Co-committed-by: sneak <sneak@sneak.berlin>
2026-07-06 21:14:03 +02:00
afe88c601a refactor: use pinned golangci-lint Docker image for linting (#55)
All checks were successful
check / check (push) Successful in 5s
Closes [issue #50](#50)

## Summary

Refactors the Dockerfile to use a separate lint stage with a pinned golangci-lint Docker image, following the pattern used by [sneak/pixa](https://git.eeqj.de/sneak/pixa). This replaces the previous approach of installing golangci-lint via curl in the builder stage.

## Changes

### Dockerfile
- **New `lint` stage** using `golangci/golangci-lint:v2.11.3` (Debian-based, pinned by sha256 digest) as a separate build stage
- **Builder stage** depends on lint via `COPY --from=lint /src/go.sum /dev/null` — build won't proceed unless linting passes
- **Go bumped** from 1.24 to 1.26.1 (`golang:1.26.1-bookworm`, pinned by sha256)
- **golangci-lint bumped** from v1.64.8 to v2.11.3
- All three Docker images (golangci-lint, golang, alpine) pinned by sha256 digest
- Debian-based golangci-lint image used (not Alpine) because mattn/go-sqlite3 CGO does not compile on musl (off64_t)

### Linter Config (.golangci.yml)
- Migrated from v1 to v2 format (`version: "2"` added)
- Removed linters no longer available in v2: `gofmt` (handled by `make fmt-check`), `gosimple` (merged into `staticcheck`), `typecheck` (always-on in v2)
- Same set of linters enabled — no rules weakened

### Code Fixes (all lint issues from v2 upgrade)
- Added package comments to all packages
- Added doc comments to all exported types, functions, and methods
- Fixed unchecked errors flagged by `errcheck` (sqlDB.Close, os.Setenv in tests, resp.Body.Close, fmt.Fprint)
- Fixed unused parameters flagged by `revive` (renamed to `_`)
- Fixed `gosec` G120 warnings: added `http.MaxBytesReader` before `r.ParseForm()` calls
- Fixed `staticcheck` QF1012: replaced `WriteString(fmt.Sprintf(...))` with `fmt.Fprintf`
- Fixed `staticcheck` QF1003: converted if/else chain to tagged switch
- Renamed `DeliveryTask` → `Task` to avoid package stutter (`delivery.Task` instead of `delivery.DeliveryTask`)
- Renamed shadowed builtin `max` parameter to `upperBound` in `cryptoRandInt`
- Used `t.Setenv` instead of `os.Setenv` in tests (auto-restores)

### README.md
- Updated version requirements: Go 1.26+, golangci-lint v2.11+
- Updated Dockerfile description in project structure

## Verification

`docker build .` passes cleanly — formatting check, linting, all tests, and build all succeed.

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #55
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-25 02:16:38 +01:00
d771fe14df fix: detect TLS per-request in CSRF middleware to fix login (#54)
All checks were successful
check / check (push) Successful in 1m55s
## Problem

After the security hardening in PR #42, login fails with `Forbidden - invalid CSRF token` in production deployments.

The CSRF middleware tied its `PlaintextHTTPRequest` wrapping and cookie `Secure` flag to the `IsDev()` environment check. This meant production mode always assumed HTTPS via gorilla/csrf's strict mode, which broke login in common deployment scenarios:

1. **Production behind a TLS-terminating reverse proxy**: gorilla/csrf assumed HTTPS but `r.TLS` was nil (the Go server receives HTTP from the proxy). Origin/Referer scheme mismatches caused `referer not supplied` or `origin invalid` errors.

2. **Production over direct HTTP** (testing/staging with prod config): the `Secure` cookie flag prevented the browser from sending the CSRF cookie back over HTTP, causing `CSRF token invalid` errors.

## Root Cause

gorilla/csrf v1.7.3 defaults to HTTPS-strict mode unless `PlaintextHTTPRequest()` is called. In strict mode it:
- Forces `requestURL.Scheme = "https"` for Origin/Referer comparisons
- Requires a `Referer` header on POST and rejects `http://` Referer schemes
- The `csrf.Secure(true)` option makes the browser refuse to send the CSRF cookie over HTTP

The old code only called `PlaintextHTTPRequest()` in dev mode, leaving prod mode permanently stuck in HTTPS-strict mode regardless of the actual transport.

## Fix

Detect the actual transport protocol **per-request** using:
- `r.TLS != nil` — direct TLS connection to the Go server
- `X-Forwarded-Proto: https` header — TLS-terminating reverse proxy

Two gorilla/csrf middleware instances are maintained (one with `Secure: true`, one with `Secure: false`) since `csrf.Secure()` is a creation-time option. Both use the same signing key, so cookies are interchangeable.

| Scenario | Cookie Secure | Origin/Referer Mode |
|---|---|---|
| Direct TLS (`r.TLS != nil`) |  Secure | Strict (HTTPS scheme) |
| Behind TLS proxy (`X-Forwarded-Proto: https`) |  Secure | Strict (HTTPS scheme) |
| Plaintext HTTP |  Non-Secure | Relaxed (PlaintextHTTPRequest) |

CSRF token validation (cookie + form double-submit) is always enforced regardless of mode.

## Testing

- Added `TestCSRF_ProdMode_PlaintextHTTP_POSTWithValidToken` — prod mode over plaintext HTTP
- Added `TestCSRF_ProdMode_BehindProxy_POSTWithValidToken` — prod mode behind TLS proxy
- Added `TestCSRF_ProdMode_DirectTLS_POSTWithValidToken` — prod mode with direct TLS
- Added `TestCSRF_ProdMode_PlaintextHTTP_POSTWithoutToken` — token still required
- Added `TestIsClientTLS_*` — TLS detection unit tests
- All existing CSRF tests pass unchanged
- `docker build .` passes (includes `make check`)
- Manual verification: built and ran the container in both `dev` and `prod` modes, confirmed login succeeds in both

Closes #53

Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #54
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-18 04:30:57 +01:00
33e2140a5a feat: redirect root path based on auth state (#52)
All checks were successful
check / check (push) Successful in 1m54s
Closes #51

The root path `/` now checks for an authenticated session and redirects accordingly:

- **Authenticated users** → `303 See Other` redirect to `/sources` (the webhook dashboard)
- **Unauthenticated users** → `303 See Other` redirect to `/pages/login`

### Changes

- **`internal/handlers/index.go`** — Replaced the template-rendering `HandleIndex()` with a session-checking redirect handler. Removed `formatUptime` helper (dead code after this change).
- **`internal/handlers/handlers.go`** — Removed `index.html` from the template map (no longer rendered).
- **`internal/handlers/handlers_test.go`** — Replaced the old "handler is not nil" test with two proper redirect tests:
  - `unauthenticated redirects to login` — verifies 303 to `/pages/login`
  - `authenticated redirects to sources` — sets up an authenticated session cookie, verifies 303 to `/sources`
  - Removed `TestFormatUptime` (tested dead code).
- **`README.md`** — Updated the API endpoints table to describe the new redirect behavior.

### How it works

The handler calls `session.Get(r)` and `session.IsAuthenticated(sess)` — the same pattern used by the `RequireAuth` middleware and `HandleLoginPage`. No new dependencies or session logic introduced.

The login flow is unaffected: `HandleLoginSubmit` redirects to `/` after successful login, which now forwards to `/sources` (one extra redirect hop, but correct and clean).

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Co-authored-by: clawbot <clawbot@eeqj.de>
Reviewed-on: #52
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 16:35:08 +01:00
f003ec7141 fix: use absolute path for dev DATA_DIR default, clarify env docs (#46)
All checks were successful
check / check (push) Successful in 1m9s
Closes #45.

## Problem

1. The README didn't clearly explain what `WEBHOOKER_ENVIRONMENT=dev` vs `prod` actually changes.
2. The dev-mode default for `DATA_DIR` was `./data` — a relative path whose meaning depends on the working directory. There's no reason to use a relative path even in development.

## Changes

### Code (`internal/config/config.go`)

- Replace the dev default `DATA_DIR` from `./data` to `$XDG_DATA_HOME/webhooker` (falling back to `$HOME/.local/share/webhooker`). This follows the XDG Base Directory Specification and ensures the data directory is always an absolute path regardless of the working directory.
- Add `devDataDir()` helper that resolves the XDG path, with a `/tmp/webhooker` last-resort fallback if `$HOME` can't be determined.

### Tests (`internal/config/config_test.go`)

- `TestDevDataDir`: verifies XDG_DATA_HOME is respected, HOME fallback works, and the result is always absolute.
- `TestDevDefaultDataDirIsAbsolute`: integration test that creates a full Config via fx and asserts the dev default DataDir is absolute.

### README

- Add a table documenting exactly what `dev` vs `prod` changes: DATA_DIR default, CORS policy, and session cookie Secure flag.
- Clarify that log format and security headers are independent of the environment setting.
- Update the DATA_DIR default in the configuration variable table.

Co-authored-by: clawbot <clawbot@eeqj.de>
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #46
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 12:48:52 +01:00
17e740a45f fix: use absolute paths and static linking in Dockerfile (#49)
Some checks failed
check / check (push) Has been cancelled
Closes #48

## Problem

The Docker container failed to start with:
```
exec ./webhooker: no such file or directory
```

Two root causes:

1. **Relative paths**: `COPY` destination and `CMD` used relative paths (`./webhooker`), depending on `WORKDIR` context.

2. **Dynamic linking** (the actual root cause): The binary was built with CGO enabled on Debian (glibc) via `make build`, but deployed to an Alpine runtime (musl). The kernel couldn't find the glibc dynamic linker (`/lib64/ld-linux-x86-64.so.2`), producing the misleading "no such file or directory" error — even though the file existed on disk.

## Fix

- **Absolute paths throughout**: `COPY --from=builder /build/bin/webhooker /app/webhooker` and `CMD ["/app/webhooker"]` — no reliance on WORKDIR.

- **Static rebuild for Alpine**: Added a `RUN CGO_ENABLED=1 go build -ldflags '-extldflags "-static"' -o bin/webhooker ./cmd/webhooker` step after `make check`. This rebuilds the binary with static linking so it runs on Alpine without glibc. The `make check` step still runs normally (formatting, linting, tests, dynamic build) — the static rebuild is only for the deployment binary.

## Verification

- `docker build .` passes (all checks green)
- Container starts successfully and initializes the Fx dependency graph
- The README already stated "The runtime binary is statically linked and runs on Alpine" — this fix makes that claim actually true.

Co-authored-by: clawbot <clawbot@users.noreply.git.eeqj.de>
Reviewed-on: #49
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 12:48:13 +01:00
60786c5019 feat: add CSRF protection, SSRF prevention, and login rate limiting (#42)
All checks were successful
check / check (push) Successful in 4s
## Security Hardening

This PR implements three security hardening issues:

### CSRF Protection (closes #35)

- Session-based CSRF tokens with cryptographically random 256-bit generation
- Constant-time token comparison to prevent timing attacks
- CSRF middleware applied to `/pages`, `/sources`, `/source`, and `/user` routes
- Hidden `csrf_token` field added to all 12+ POST forms in templates
- Excluded from `/webhook` (inbound webhook POSTs) and `/api` (stateless API)

### SSRF Prevention (closes #36)

- `ValidateTargetURL()` blocks private/reserved IP ranges at target creation time
- Blocked ranges: `127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`, `::1`, `fc00::/7`, `fe80::/10`, plus multicast, reserved, test-net, and CGN ranges
- SSRF-safe HTTP transport with custom `DialContext` in the delivery engine for defense-in-depth (prevents DNS rebinding attacks)
- Only `http` and `https` schemes allowed

### Login Rate Limiting (closes #37)

- Per-IP rate limiter using `golang.org/x/time/rate`
- 5 attempts per minute per IP on `POST /pages/login`
- GET requests (form rendering) pass through unaffected
- Automatic cleanup of stale per-IP limiter entries every 5 minutes
- `X-Forwarded-For` and `X-Real-IP` header support for reverse proxies

### Files Changed

**New files:**
- `internal/middleware/csrf.go` + tests — CSRF middleware
- `internal/middleware/ratelimit.go` + tests — Login rate limiter
- `internal/delivery/ssrf.go` + tests — SSRF validation + safe transport

**Modified files:**
- `internal/server/routes.go` — Wire CSRF and rate limit middleware
- `internal/handlers/handlers.go` — Inject CSRF token into template data
- `internal/handlers/source_management.go` — SSRF validation on target creation
- `internal/delivery/engine.go` — SSRF-safe HTTP transport for production
- All form templates — Added hidden `csrf_token` fields
- `README.md` — Updated Security section and TODO checklist

`docker build .` passes (lint + tests + build).

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Co-authored-by: clawbot <clawbot@eeqj.de>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #42
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 12:38:45 +01:00
8d702a16c6 feat: add Slack target type for incoming webhook notifications (#47)
All checks were successful
check / check (push) Successful in 4s
## Summary

Adds a new `slack` target type that sends webhook events as formatted messages to any Slack-compatible incoming webhook URL (Slack, Mattermost, and other compatible services).

closes #44

## What it does

When a webhook event is received, the Slack target:

1. Formats a human-readable message with event metadata (HTTP method, content type, timestamp, body size)
2. Pretty-prints the payload in a code block — JSON payloads get indented formatting, non-JSON payloads are shown as raw text
3. Truncates large payloads at 3500 characters to keep Slack messages reasonable
4. POSTs the message as a `{"text": "..."}` JSON payload to the configured webhook URL

## Changes

- **`internal/database/model_target.go`** — Add `TargetTypeSlack` constant
- **`internal/delivery/engine.go`** — Add `SlackTargetConfig` struct, `deliverSlack` method, `FormatSlackMessage` function (exported), `parseSlackConfig` helper. Route slack targets in `processDelivery` switch.
- **`internal/handlers/source_management.go`** — Handle `slack` type in `HandleTargetCreate`, building `webhook_url` config from the URL form field
- **`templates/source_detail.html`** — Add "Slack" option to target type dropdown with URL field and helper text
- **`README.md`** — Document the new target type, update roadmap

## Tests

- `TestParseSlackConfig_Valid` / `_Empty` / `_MissingWebhookURL` — Config parsing
- `TestFormatSlackMessage_JSONBody` / `_NonJSONBody` / `_EmptyBody` / `_LargeJSONTruncated` — Message formatting
- `TestDeliverSlack_Success` / `_Failure` / `_InvalidConfig` — End-to-end delivery
- `TestProcessDelivery_RoutesToSlack` — Routing from processDelivery switch

All existing tests continue to pass. `docker build .` (which runs `make check`) passes clean.

Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #47
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 12:30:50 +01:00
1fbcf96581 security: add headers middleware, session regeneration, and body size limits (#41)
All checks were successful
check / check (push) Successful in 1m47s
## Summary

This PR implements three security hardening measures:

### Security Headers Middleware (closes #34)

Adds a `SecurityHeaders()` middleware applied globally to all routes. Every response now includes:
- `Strict-Transport-Security: max-age=63072000; includeSubDomains; preload`
- `X-Content-Type-Options: nosniff`
- `X-Frame-Options: DENY`
- `Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'`
- `Referrer-Policy: strict-origin-when-cross-origin`
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`

### Session Fixation Prevention (closes #38)

Adds a `Regenerate()` method to the session manager that destroys the old session and creates a new one with a fresh ID, copying all session values. Called after successful login to prevent session fixation attacks.

### Request Body Size Limits (closes #39)

Adds a `MaxBodySize()` middleware using `http.MaxBytesReader` to limit POST/PUT/PATCH request bodies to 1 MB. Applied to all form endpoints (`/pages`, `/sources`, `/source/*`).

## Files Changed

- `internal/middleware/middleware.go` — Added `SecurityHeaders()` and `MaxBodySize()` middleware
- `internal/session/session.go` — Added `Regenerate()` method for session fixation prevention
- `internal/handlers/auth.go` — Updated login handler to regenerate session after authentication
- `internal/server/routes.go` — Added SecurityHeaders globally, MaxBodySize to form route groups
- `README.md` — Documented new middleware in stack, updated Security section, moved items to completed TODO

closes #34, closes #38, closes #39

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #41
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-05 12:32:56 +01:00
a51e863017 Remove globals.Buildarch from codebase (#31)
All checks were successful
check / check (push) Successful in 1m0s
Remove the `Buildarch` field from the globals package and all references throughout the codebase.

**Changes:**
- Removed `Buildarch` package-level var and struct field from `internal/globals/globals.go`
- Removed `Buildarch` from the `New()` constructor
- Removed `globals.Buildarch = runtime.GOARCH` and unused `runtime` import from `cmd/webhooker/main.go`
- Removed `buildarch` from logger startup output in `internal/logger/logger.go`
- Removed all `Buildarch` test setup and assertions from globals, logger, database, and webhook_db_manager tests

All tests pass, `make check` passes, `docker build .` succeeds.

closes [issue #30](#30)

<!-- session: agent:sdlc-manager:subagent:5cae6803-6bdf-467d-9a56-43f135521e5f -->

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #31
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-04 12:07:58 +01:00
289f479772 test: add tests for delivery, middleware, and session packages (#32)
Some checks failed
check / check (push) Has been cancelled
## Summary

Add comprehensive test coverage for three previously-untested packages, addressing [issue #28](#28).

## Coverage Improvements

| Package | Before | After |
|---------|--------|-------|
| `internal/delivery` | 37.1% | 74.5% |
| `internal/middleware` | 0.0% | 70.2% |
| `internal/session` | 0.0% | 51.5% |

## What's Tested

### delivery (37% → 75%)
- `processNewTask` with inline and large (DB-fetched) bodies
- `processRetryTask` success, skip non-retrying, large body fetch
- Worker lifecycle start/stop, retry channel processing
- `processDelivery` unknown target type handling
- `recoverPendingDeliveries`, `recoverWebhookDeliveries`, `recoverInFlight`
- HTTP delivery with custom headers, timeout, invalid config
- `Notify` batching

### middleware (0% → 70%)
- Logging middleware status code capture and pass-through
- `LoggingResponseWriter` delegation
- CORS dev mode (allow-all) and prod mode (no-op)
- `RequireAuth` redirect for unauthenticated, pass-through for authenticated
- `MetricsAuth` basic auth validation
- `ipFromHostPort` helper

### session (0% → 52%)
- `Get`/`Save` round-trip with real cookie store
- `SetUser`, `GetUserID`, `GetUsername`, `IsAuthenticated`
- `ClearUser` removes all keys
- `Destroy` invalidates session (MaxAge -1)
- Session persistence across requests
- Edge cases: overwrite user, wrong type, constants

## Test Helpers Added
- `database.NewTestDatabase` / `NewTestWebhookDBManager` — cross-package test helpers for delivery integration tests
- `session.NewForTest` — creates session manager without fx lifecycle for middleware tests

## Notes
- No production code modified
- All tests use `httptest`, SQLite in-memory, and real cookie stores — no external network calls
- Full test suite completes in ~3.5s within the 30s timeout
- `docker build .` passes (lint + test + build)

closes #28

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #32
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-04 12:07:23 +01:00
687655ed49 Merge pull request 'feat: webhooker 1.0 MVP — entity rename, core engine, delivery, management UI' (#16) from feature/mvp-1.0 into main
All checks were successful
check / check (push) Successful in 4s
Reviewed-on: #16
2026-03-04 01:19:41 +01:00
clawbot
8e00e40008 docs: fix stale references to development mode and retry target type
All checks were successful
check / check (push) Successful in 5s
- README.md: remove 'in development mode' from admin user creation
  description (admin user creation is unconditional)
- internal/delivery/engine.go: remove 'and retry' from HTTPTargetConfig
  comment (retry was merged into http target type)
- internal/delivery/engine_test.go: remove '/retry' from
  newHTTPTargetConfig comment for consistency
2026-03-03 16:12:43 -08:00
clawbot
3588facfff remove unnecessary data migration and dead DevelopmentMode config
All checks were successful
check / check (push) Successful in 1m52s
- Remove retry→http data migration from migrate() — no databases exist pre-1.0
- Remove unused DevelopmentMode field and DEVELOPMENT_MODE env var from config
- Remove DevelopmentMode from config log output (dead code cleanup)
2026-03-03 09:16:03 -08:00
clawbot
25e27cc57f refactor: merge retry target type into http (max_retries=0 = fire-and-forget)
All checks were successful
check / check (push) Successful in 1m46s
2026-03-01 23:51:55 -08:00
clawbot
4dd4dfa5eb chore: consolidate DBURL into DATA_DIR, codebase audit for 1.0.0
All checks were successful
check / check (push) Successful in 56s
DBURL → DATA_DIR consolidation:
- Remove DBURL env var entirely; main DB now lives at {DATA_DIR}/webhooker.db
- database.go constructs DB path from config.DataDir, ensures dir exists
- Update DATA_DIR prod default from /data/events to /data
- Update all tests to use DataDir instead of DBURL
- Update Dockerfile: /data (not /data/events) for all SQLite databases
- Update README configuration table, Docker examples, architecture docs

Dead code removal:
- Remove unused IndexResponse struct (handlers/index.go)
- Remove unused TemplateData struct (handlers/handlers.go)

Stale comment cleanup:
- Remove TODO in server.go (DB cleanup handled by fx lifecycle)
- Fix nolint:golint → nolint:revive on ServerParams for consistency
- Clean up verbose middleware/routing comments in routes.go
- Fix TODO fan-out description (worker pool, not goroutine-per-target)

.gitignore fixes:
- Add data/ directory to gitignore
- Remove stale config.yaml entry (env-only config since rework)
2026-03-01 23:33:20 -08:00
clawbot
536e5682d6 test: add comprehensive delivery engine and circuit breaker tests
All checks were successful
check / check (push) Successful in 1m48s
Add unit tests for internal/delivery/ package covering:

Circuit breaker tests (circuit_breaker_test.go):
- Closed state allows deliveries
- Failure counting below threshold
- Open transition after threshold failures
- Cooldown blocks during cooldown period
- Half-open transition after cooldown expires
- Probe success closes circuit
- Probe failure reopens circuit
- Success resets failure counter
- Concurrent access safety (race-safe)
- CooldownRemaining for all states
- CircuitState String() output

Engine tests (engine_test.go):
- Non-blocking Notify when channel is full
- HTTP target success and failure delivery
- Database target immediate success
- Log target immediate success
- Retry target success with circuit breaker
- Max retries exhausted marks delivery failed
- Retry scheduling on failure
- Exponential backoff duration verification
- Backoff cap at shift 30
- Body pointer semantics (inline <16KB, nil >=16KB)
- Worker pool bounded concurrency
- Circuit breaker blocks delivery attempts
- Circuit breaker per-target creation
- HTTP config parsing (valid, empty, missing URL)
- scheduleRetry sends to retry channel
- scheduleRetry drops when channel full
- Header forwarding (forwardable vs hop-by-hop)
- processDelivery routing to correct handler
- Truncate helper function

All tests use real SQLite databases and httptest servers.
All tests pass with -race flag.
2026-03-01 23:16:30 -08:00
clawbot
49852e7506 refactor: remove file-based configuration, use env vars only
All checks were successful
check / check (push) Successful in 1m0s
Remove the entire pkg/config package (Viper-based YAML config file
loader) and simplify internal/config to read all settings directly from
environment variables via os.Getenv(). This eliminates the spurious
"Failed to load config" log messages that appeared when no config.yaml
file was present.

- Delete pkg/config/ (YAML loader, resolver, manager, tests)
- Delete configs/config.yaml.example
- Simplify internal/config helper functions to use os.Getenv() with
  defaults instead of falling back to pkgconfig
- Update tests to set env vars directly instead of creating in-memory
  YAML config files via afero
- Remove afero, cloud.google.com/*, aws-sdk-go dependencies from go.mod
- Update README: document env-var-only configuration, remove YAML/Viper
  references
- Keep godotenv/autoload for .env file convenience in local development

closes #27
2026-03-01 23:04:49 -08:00
clawbot
10db6c5b84 refactor: bounded worker pool with DB-mediated retry fallback
All checks were successful
check / check (push) Successful in 58s
Replace unbounded goroutine-per-delivery fan-out with a fixed-size
worker pool (10 workers). Channels serve as bounded queues (10,000
buffer). Workers are the only goroutines doing HTTP delivery.

When retry channel overflows, timers are dropped instead of re-armed.
The delivery stays in 'retrying' status in the DB and a periodic sweep
(every 60s) recovers orphaned retries. The database is the durable
fallback — same path used on startup recovery.

Addresses owner feedback on circuit breaker recovery goroutine flood.
2026-03-01 22:52:27 -08:00
clawbot
9b4ae41c44 feat: parallel fan-out delivery + circuit breaker for retry targets
All checks were successful
check / check (push) Successful in 1m52s
- Fan out all targets for an event in parallel goroutines (fire-and-forget)
- Add per-target circuit breaker for retry targets (closed/open/half-open)
- Circuit breaker trips after 5 consecutive failures, 30s cooldown
- Open circuit skips delivery and reschedules after cooldown
- Half-open allows one probe delivery to test recovery
- HTTP/database/log targets unaffected (no circuit breaker)
- Recovery path also fans out in parallel
- Update README with parallel delivery and circuit breaker docs
2026-03-01 22:20:33 -08:00
clawbot
32bd40b313 refactor: self-contained delivery tasks — engine delivers without DB reads in happy path
All checks were successful
check / check (push) Successful in 58s
The webhook handler now builds DeliveryTask structs carrying all target
config and event data inline (for bodies ≤16KB) and sends them through
the delivery channel. In the happy path, the engine delivers without
reading from any database — it only writes to record delivery results.

For large bodies (≥16KB), Body is nil and the engine fetches it from the
per-webhook database on demand. Retry timers also carry the full
DeliveryTask, so retries avoid unnecessary DB reads.

The database is used for crash recovery only: on startup the engine scans
for interrupted pending/retrying deliveries and re-queues them.

Implements owner feedback from issue #15:
> the message in the <=16KB case should have everything it needs to do
> its delivery. it shouldn't touch the db until it has a success or
> failure to record.
2026-03-01 22:09:41 -08:00
9b9ee1718a refactor: auto-generate session key and store in database
All checks were successful
check / check (push) Successful in 57s
Remove SESSION_KEY env var requirement. On first startup, a
cryptographically secure 32-byte key is generated and stored in a new
settings table. Subsequent startups load the key from the database.

- Add Setting model (key-value table) for application config
- Add Database.GetOrCreateSessionKey() method
- Session manager initializes in OnStart after database is connected
- Remove DevSessionKey constant and SESSION_KEY env var handling
- Remove prod validation requiring SESSION_KEY
- Update README: config table, Docker instructions, security notes
- Update config.yaml.example
- Update all tests to remove SessionKey references

Addresses owner feedback on issue #15.
2026-03-01 21:57:19 -08:00
clawbot
5e683af2a4 refactor: event-driven delivery engine with channel notifications and timer-based retries
All checks were successful
check / check (push) Successful in 58s
Replace the polling-based delivery engine with a fully event-driven
architecture using Go channels and goroutines:

- Webhook handler notifies engine via buffered channel after creating
  delivery records, with inline event data for payloads < 16KB
- Large payloads (>= 16KB) use pointer semantics (Body *string = nil)
  and are fetched from DB on demand, keeping channel memory bounded
- Failed retry-target deliveries schedule Go timers with exponential
  backoff; timers fire into a separate retry channel when ready
- On startup, engine scans DB once to recover interrupted deliveries
  (pending processed immediately, retrying get timers for remaining
  backoff)
- DB stores delivery status for crash recovery only, not for
  inter-component communication during normal operation
- delivery.Notifier interface decouples handlers from engine; fx wires
  *Engine as Notifier

No more periodic polling. No more wasted cycles when idle.
2026-03-01 21:46:16 -08:00
clawbot
8f62fde8e9 revert admin password logging to slog.Info (closes #26)
All checks were successful
check / check (push) Successful in 1m58s
2026-03-01 21:26:31 -08:00
clawbot
43c22a9e9a feat: implement per-webhook event databases
All checks were successful
check / check (push) Successful in 1m50s
Split data storage into main application DB (config only) and
per-webhook event databases (one SQLite file per webhook).

Architecture changes:
- New WebhookDBManager component manages per-webhook DB lifecycle
  (create, open, cache, delete) with lazy connection pooling via sync.Map
- Main DB (DBURL) stores only config: Users, Webhooks, Entrypoints,
  Targets, APIKeys
- Per-webhook DBs (DATA_DIR) store Events, Deliveries, DeliveryResults
  in files named events-{webhook_uuid}.db
- New DATA_DIR env var (default: ./data dev, /data/events prod)

Behavioral changes:
- Webhook creation creates per-webhook DB file
- Webhook deletion hard-deletes per-webhook DB file (config soft-deleted)
- Event ingestion writes to per-webhook DB, not main DB
- Delivery engine polls all per-webhook DBs for pending deliveries
- Database target type marks delivery as immediately successful (events
  are already in the dedicated per-webhook DB)
- Event log UI reads from per-webhook DBs with targets from main DB
- Existing webhooks without DB files get them created lazily

Removed:
- ArchivedEvent model (was a half-measure, replaced by per-webhook DBs)
- Event/Delivery/DeliveryResult removed from main DB migrations

Added:
- Comprehensive tests for WebhookDBManager (create, delete, lazy
  creation, delivery workflow, multiple webhooks, close all)
- Dockerfile creates /data/events directory

README updates:
- Per-webhook event databases documented as implemented (was Phase 2)
- DATA_DIR added to configuration table
- Docker instructions updated with data volume mount
- Data model diagram updated
- TODO updated (database separation moved to completed)

Closes #15
2026-03-01 17:06:43 -08:00
clawbot
6c393ccb78 fix: database target writes to dedicated archive table
All checks were successful
check / check (push) Successful in 1m43s
The "database" target type now writes events to a separate
archived_events table instead of just marking the delivery as done.
This table persists independently of internal event retention/pruning,
allowing the data to be consumed by external systems or preserved
indefinitely.

New ArchivedEvent model copies the full event payload (method, headers,
body, content_type) along with webhook/entrypoint/event/target IDs.
2026-03-01 16:40:27 -08:00
clawbot
418d3da97e fix: remove spurious config load log message (closes #27)
When no config.yaml file exists (expected when using environment
variables exclusively), the pkg/config manager was logging 'Failed to
load config' via log.Printf, which is confusing during normal operation.
Suppress these messages since missing config file is a valid state.
2026-03-01 16:39:26 -08:00
clawbot
7bac22bdfd fix: don't log admin password via slog (closes #26)
Replace slog.Info (which outputs structured JSON in prod and ends up in
log aggregation) with a plain fmt.Fprintf to stderr. The password is
printed once on first startup in a clearly-delimited banner that won't
be parsed as a structured log field.
2026-03-01 16:38:38 -08:00
clawbot
f21a007a3c feat: add entrypoint/target management controls (closes #25)
Add toggle (activate/deactivate) and delete buttons for individual
entrypoints and targets on the webhook detail page. Each action is a
POST form submission with ownership verification.

New routes:
  POST /source/{id}/entrypoints/{entrypointID}/delete
  POST /source/{id}/entrypoints/{entrypointID}/toggle
  POST /source/{id}/targets/{targetID}/delete
  POST /source/{id}/targets/{targetID}/toggle
2026-03-01 16:38:14 -08:00
clawbot
2606d41c60 fix: cascade soft-delete for webhook deletion (closes #24)
When deleting a webhook, also soft-delete all related deliveries and
delivery results (not just entrypoints, targets, and events). Query
event IDs, then delivery IDs, then cascade delete delivery results,
deliveries, events, entrypoints, targets, and finally the webhook
itself — all within a single transaction.
2026-03-01 16:37:21 -08:00
clawbot
45228d9e99 fix: restrict CORS to same-origin (closes #23)
In dev mode, keep the wildcard origin for local testing convenience.
In production, skip CORS headers entirely since the web UI is
server-rendered and cross-origin requests are not expected.
2026-03-01 16:36:56 -08:00
clawbot
348fd81fe6 fix: remove dead DevAdminUsername/Password config (closes #22)
Remove DevAdminUsername and DevAdminPassword fields from the Config
struct and their loading code. These fields were never referenced
anywhere else in the codebase.
2026-03-01 16:36:36 -08:00
clawbot
36824046fb fix: remove double cleanShutdown call (closes #21)
The serve() method called cleanShutdown() after ctx.Done(), and the fx
OnStop hook also called cleanShutdown(). Remove the call in serve() so
shutdown happens exactly once via the fx lifecycle.
2026-03-01 16:35:55 -08:00
clawbot
e2ac30287b fix: restrict webhook endpoint to POST only (closes #20)
Add method check at the top of HandleWebhook, returning 405 Method Not
Allowed with an Allow: POST header for any non-POST request. This
prevents GET, PUT, DELETE, etc. from being accepted at entrypoint URLs.
2026-03-01 16:35:38 -08:00
clawbot
49ab1a6147 fix: DevSessionKey wrong length (closes #19)
Replace the old 35-byte dev session key with a proper randomly-generated
32-byte key. Also ensure dev mode actually falls back to DevSessionKey
when SESSION_KEY is not set in the environment, rather than leaving
SessionKey empty and failing at session creation.

Update tests to remove the old key references.
2026-03-01 16:35:16 -08:00
clawbot
d65480c5ec fix: template rendering returns empty pages (closes #18)
Reorder template.ParseFS arguments so the page template file is listed
first. Go's template package names the template set after the first file
parsed. When htmlheader.html was first, its content (entirely a
{{define}} block) became the root template, which is empty. By putting
the page file first, its {{template "base" .}} invocation becomes the
root action and the page renders correctly.
2026-03-01 16:34:33 -08:00
clawbot
d4fbd6c110 fix: delivery engine nil pointer crash on startup (closes #17)
Store the *database.Database wrapper instead of calling .DB() eagerly
at construction time. The GORM *gorm.DB is only available after the
database's OnStart hook runs, but the engine constructor runs during
fx resolution (before OnStart). Accessing .DB() lazily via the wrapper
avoids the nil pointer panic.
2026-03-01 16:34:16 -08:00
clawbot
7f8469a0f2 feat: implement core webhook engine, delivery system, and management UI (Phase 2)
All checks were successful
check / check (push) Successful in 1m49s
- Webhook reception handler: look up entrypoint by UUID, verify active,
  capture full HTTP request (method, headers, body, content-type), create
  Event record, queue Delivery records for each active Target, return 200 OK.
  Handles edge cases: unknown UUID → 404, inactive → 410, oversized → 413.

- Delivery engine (internal/delivery): fx-managed background goroutine that
  polls for pending/retrying deliveries and dispatches to target type handlers.
  Graceful shutdown via context cancellation.

- Target type implementations:
  - HTTP: fire-and-forget POST with original headers forwarding
  - Retry: exponential backoff (1s, 2s, 4s...) up to max_retries
  - Database: immediate success (event already stored)
  - Log: slog output with event details

- Webhook management pages with Tailwind CSS + Alpine.js:
  - List (/sources): webhooks with entrypoint/target/event counts
  - Create (/sources/new): form with auto-created default entrypoint
  - Detail (/source/{id}): config, entrypoints, targets, recent events
  - Edit (/source/{id}/edit): name, description, retention_days
  - Delete (/source/{id}/delete): soft-delete with child records
  - Add Entrypoint (/source/{id}/entrypoints): inline form
  - Add Target (/source/{id}/targets): type-aware form
  - Event Log (/source/{id}/logs): paginated with delivery status

- Updated README: marked completed items, updated naming conventions
  table, added delivery engine to package layout and DI docs, updated
  column names to reflect entity rename.

- Rebuilt Tailwind CSS for new template classes.

Part of: #15
2026-03-01 16:14:28 -08:00
137 changed files with 25352 additions and 3683 deletions

View File

@@ -12,4 +12,4 @@ jobs:
- name: Checkout - name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 2024-10-23 uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 2024-10-23
- name: Build Docker image (runs make check) - name: Build Docker image (runs make check)
run: docker build . run: script/cibuild

4
.gitignore vendored
View File

@@ -29,9 +29,9 @@ Thumbs.db
# Environment and config files # Environment and config files
.env .env
.env.local .env.local
config.yaml
# Database files # Data directory (SQLite databases)
data/
*.db *.db
*.sqlite *.sqlite
*.sqlite3 *.sqlite3

View File

@@ -1,46 +1,34 @@
version: "2"
# Config schema uses the golangci-lint v2 layout (settings live under
# linters.settings, not top-level linters-settings) so that the
# thresholds below are actually applied by golangci-lint >= v2.
run: run:
timeout: 5m timeout: 5m
tests: true modules-download-mode: readonly
linters: linters:
enable: default: all
- gofmt disable:
- revive # Genuinely incompatible with project patterns
- govet - exhaustruct # Requires all struct fields
- errcheck - depguard # Dependency allow/block lists
- staticcheck - godot # Requires comments to end with periods
- unused - wsl # Deprecated, replaced by wsl_v5
- gosimple - wrapcheck # Too verbose for internal packages
- ineffassign - varnamelen # Short names like db, id are idiomatic Go
- typecheck settings:
- gosec lll:
- misspell line-length: 88
- unparam funlen:
- prealloc lines: 80
- copyloopvar statements: 50
- gocritic cyclop:
- gochecknoinits max-complexity: 15
- gochecknoglobals dupl:
threshold: 100
linters-settings:
gofmt:
simplify: true
revive:
confidence: 0.8
govet:
enable:
- shadow
errcheck:
check-type-assertions: true
check-blank: true
issues: issues:
exclude-rules: max-issues-per-linter: 0
# Exclude globals check for version variables in main max-same-issues: 0
- path: cmd/webhooker/main.go
linters:
- gochecknoglobals
# Exclude globals check for version variables in globals package
- path: internal/globals/globals.go
linters:
- gochecknoglobals

View File

@@ -1,50 +1,58 @@
# golang:1.24 (bookworm) — 2026-03-01 # Lint stage
# Using Debian-based image because gorm.io/driver/sqlite pulls in # golangci/golangci-lint:v2.12.2 (Debian-based), 2026-08-07
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl. # Using Debian-based image because mattn/go-sqlite3 (CGO) does not
FROM golang@sha256:d2d2bc1c84f7e60d7d2438a3836ae7d0c847f4888464e7ec9ba3a1339a1ee804 AS builder # compile on Alpine musl (off64_t is a glibc type).
FROM golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint
# gcc is pre-installed in the Debian-based golang image
RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/* RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/*
WORKDIR /build WORKDIR /src
# Install golangci-lint v1.64.8 — 2026-03-01 # Copy go mod files first for better layer caching
# Using v1.x because the repo's .golangci.yml uses v1 config format.
RUN set -eux; \
GOLANGCI_VERSION="1.64.8"; \
ARCH="$(uname -m)"; \
case "${ARCH}" in \
x86_64) \
GOARCH="amd64"; \
GOLANGCI_SHA256="b6270687afb143d019f387c791cd2a6f1cb383be9b3124d241ca11bd3ce2e54e"; \
;; \
aarch64) \
GOARCH="arm64"; \
GOLANGCI_SHA256="a6ab58ebcb1c48572622146cdaec2956f56871038a54ed1149f1386e287789a5"; \
;; \
*) echo "unsupported architecture: ${ARCH}" && exit 1 ;; \
esac; \
wget -q "https://github.com/golangci/golangci-lint/releases/download/v${GOLANGCI_VERSION}/golangci-lint-${GOLANGCI_VERSION}-linux-${GOARCH}.tar.gz" \
-O /tmp/golangci-lint.tar.gz; \
echo "${GOLANGCI_SHA256} /tmp/golangci-lint.tar.gz" | sha256sum -c -; \
tar -xzf /tmp/golangci-lint.tar.gz -C /tmp; \
mv "/tmp/golangci-lint-${GOLANGCI_VERSION}-linux-${GOARCH}/golangci-lint" /usr/local/bin/; \
rm -rf /tmp/golangci-lint*; \
golangci-lint --version
# Copy go module files and download dependencies
COPY go.mod go.sum ./ COPY go.mod go.sum ./
COPY pkg/config/go.mod pkg/config/go.sum ./pkg/config/
RUN go mod download RUN go mod download
# Copy source code # Copy source code
COPY . . COPY . .
# Run all checks (fmt-check, lint, test, build) # Run formatting check and linter
RUN make check RUN make fmt-check
RUN make lint
# alpine:3.21 — 2026-03-01 # Build stage
FROM alpine@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709 # golang:1.26.1-bookworm (Debian-based), 2026-03-17
# Using Debian-based image because gorm.io/driver/sqlite pulls in
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl.
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder
# Depend on lint stage passing
COPY --from=lint /src/go.sum /dev/null
RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/*
WORKDIR /build
# Copy go mod files first for better layer caching
COPY go.mod go.sum ./
RUN go mod download
# Copy source code
COPY . .
# Run tests and build
RUN make test
RUN make build
# Rebuild with static linking for Alpine runtime.
# make build already verified compilation.
# The CGO binary from `make build` is dynamically linked against glibc,
# which doesn't exist on Alpine (musl). Rebuild with static linking so
# the binary runs on Alpine without glibc.
RUN CGO_ENABLED=1 go build -ldflags '-extldflags "-static"' -o bin/webhooker ./cmd/webhooker
# Runtime stage
# alpine:3.21, 2026-03-17
FROM alpine:3.21@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709
RUN apk --no-cache add ca-certificates RUN apk --no-cache add ca-certificates
@@ -55,9 +63,13 @@ RUN addgroup -g 1000 -S webhooker && \
WORKDIR /app WORKDIR /app
# Copy binary from builder # Copy binary from builder
COPY --from=builder /build/bin/webhooker . COPY --from=builder /build/bin/webhooker /app/webhooker
RUN chown -R webhooker:webhooker /app # Create data directory for all SQLite databases (main app DB +
# per-webhook event DBs). DATA_DIR defaults to /var/lib/webhooker.
RUN mkdir -p /var/lib/webhooker
RUN chown -R webhooker:webhooker /app /var/lib/webhooker
USER webhooker USER webhooker
@@ -66,4 +78,4 @@ EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:8080/.well-known/healthcheck || exit 1 CMD wget --no-verbose --tries=1 --spider http://localhost:8080/.well-known/healthcheck || exit 1
CMD ["./webhooker"] CMD ["/app/webhooker"]

View File

@@ -1,22 +1,28 @@
.PHONY: test lint fmt fmt-check check build run dev deps docker clean hooks css .PHONY: bootstrap setup test lint fmt fmt-check check build run dev deps docker clean hooks css
# Default target # Default target
.DEFAULT_GOAL := check .DEFAULT_GOAL := check
bootstrap:
@script/bootstrap
setup:
@script/setup
test: test:
go test -v -race -timeout 30s ./... @script/test
lint: lint:
golangci-lint run --config .golangci.yml ./... @script/lint
fmt: fmt:
gofmt -s -w . @script/fmt
@command -v goimports >/dev/null 2>&1 && goimports -w . || true
fmt-check: fmt-check:
@test -z "$$(gofmt -s -l .)" || { echo "gofmt needed on:"; gofmt -s -l .; exit 1; } @script/fmt-check
check: fmt-check lint test build check:
@script/check
build: build:
go build -o bin/webhooker ./cmd/webhooker go build -o bin/webhooker ./cmd/webhooker
@@ -32,15 +38,13 @@ deps:
go mod tidy go mod tidy
docker: docker:
docker build -t webhooker:latest . @script/docker
clean: clean:
rm -rf bin/ rm -rf bin/
hooks: hooks:
@printf '#!/bin/sh\nmake check\n' > .git/hooks/pre-commit @script/install-precommit
@chmod +x .git/hooks/pre-commit
@echo "pre-commit hook installed"
css: css:
tailwindcss -i static/css/input.css -o static/css/tailwind.css --minify tailwindcss -i static/css/input.css -o static/css/tailwind.css --minify

741
README.md
View File

@@ -11,8 +11,8 @@ with retry support, logging, and observability. Category: infrastructure
### Prerequisites ### Prerequisites
- Go 1.24+ - Go 1.26+
- golangci-lint v1.64+ - golangci-lint v2.11+
- Docker (for containerized deployment) - Docker (for containerized deployment)
### Quick Start ### Quick Start
@@ -38,42 +38,155 @@ make docker
### Development Commands ### Development Commands
```bash ```bash
make bootstrap # Install all dependencies (idempotent)
make setup # Bootstrap + install git pre-commit hook
make fmt # Format code (gofmt + goimports) make fmt # Format code (gofmt + goimports)
make lint # Run golangci-lint make lint # Run golangci-lint
make test # Run tests with race detection make test # Run tests with race detection
make check # fmt-check + lint + test + build (CI gate) make check # test + lint + fmt-check (CI gate)
make build # Build binary to bin/webhooker make build # Build binary to bin/webhooker
make dev # go run ./cmd/webhooker make dev # go run ./cmd/webhooker
make docker # Build Docker image make docker # Build Docker image
make hooks # Install git pre-commit hook that runs make check make hooks # Install git pre-commit hook that runs script/precommit
``` ```
### Configuration ### Configuration
webhooker uses a YAML configuration file with environment-specific All configuration is via environment variables. For local development,
overrides, loaded via the `pkg/config` library (Viper-based). The you can place variables in a `.env` file in the project root (loaded
environment is selected by setting `WEBHOOKER_ENVIRONMENT` to `dev` or automatically via `godotenv/autoload`).
`prod` (default: `dev`).
Configuration is resolved in this order (highest priority first): The environment is selected by setting `WEBHOOKER_ENVIRONMENT` to `dev`
or `prod` (default: `dev`). The setting controls several behaviors:
1. Environment variables | Behavior | `dev` | `prod` |
2. `.env` file (loaded via `godotenv/autoload`) | --------------------- | -------------------------------- | ------------------------------- |
3. Config file values for the active environment | CORS | Allows any origin (`*`) | Disabled (no-op) |
4. Config file defaults | Session cookie Secure | `false` (works over plain HTTP) | `true` (requires HTTPS) |
The CSRF cookie's `Secure` flag and Origin/Referer validation mode are
determined per-request based on the actual transport protocol, not the
environment setting. The middleware checks `r.TLS` (direct TLS) and the
`X-Forwarded-Proto` header (TLS-terminating reverse proxy) to decide:
- **Direct TLS or `X-Forwarded-Proto: https`**: Secure cookies, strict
Origin/Referer validation.
- **Plaintext HTTP**: Non-Secure cookies, relaxed Origin/Referer
checks (token validation still enforced).
This means CSRF protection works correctly in all deployment scenarios:
behind a TLS-terminating reverse proxy, with direct TLS, or over plain
HTTP during development. When running behind a reverse proxy, ensure it
sets the `X-Forwarded-Proto: https` header.
All other differences (log format, security headers, etc.) are
independent of the environment setting — log format is determined by
TTY detection, and security headers are always applied.
| Variable | Description | Default | | Variable | Description | Default |
| ----------------------- | ----------------------------------- | -------- | | ----------------------- | ----------------------------------- | -------- |
| `WEBHOOKER_ENVIRONMENT` | `dev` or `prod` | `dev` | | `WEBHOOKER_ENVIRONMENT` | `dev` or `prod` | `dev` |
| `PORT` | HTTP listen port | `8080` | | `PORT` | HTTP listen port | `8080` |
| `DBURL` | SQLite database connection string | *(required)* | | `DATA_DIR` | Directory for all SQLite databases | `/var/lib/webhooker` |
| `SESSION_KEY` | Base64-encoded 32-byte session key | *(required in prod)* |
| `DEBUG` | Enable debug logging | `false` | | `DEBUG` | Enable debug logging | `false` |
| `MAINTENANCE_MODE` | Serve the maintenance page | `false` |
| `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` | | `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` |
| `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` | | `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` |
| `SENTRY_DSN` | Sentry error reporting DSN | `""` | | `SENTRY_DSN` | Sentry error reporting DSN | `""` |
| `SESSION_IDLE_TIMEOUT` | Idle session timeout (Go duration) | `24h` |
| `RECEIVER_RATE_LIMIT` | Receiver requests/minute per IP per entrypoint | `120` |
| `TRUSTED_PROXIES` | CIDRs whose forwarded headers are trusted | `""` (none) |
On first startup in development mode, webhooker creates an `admin` user #### Trusted proxies
`TRUSTED_PROXIES` is a comma-separated list of CIDR blocks (a bare
address such as `192.168.1.7` is accepted and treated as a single
host), for example `192.168.1.7, 2001:db8::5`. It decides whose
`X-Forwarded-For` header the rate limiters believe, so it should name
the addresses of your reverse proxies and nothing else.
`X-Forwarded-For` is honoured **only** when the connecting peer is
inside one of these blocks; for every other peer the client identity is
the connection's own address and the header is ignored. The default is
the empty list, which trusts nobody — anything else would let any
client pick its own rate limit bucket, minting a fresh one per request
or draining someone else's. Set it to the address of your reverse
proxy, and to nothing wider. A set but unparseable value aborts
startup.
`X-Real-IP` and `True-Client-IP` are **never** read, from any peer.
Reverse proxies append to `X-Forwarded-For` but forward other client
headers verbatim, so a single-valued header is client-controlled even
behind a trusted proxy.
Within a trusted request, `X-Forwarded-For` is read right to left,
because the rightmost entry is the one the nearest proxy appended and
everything left of it may have been written by the client. The first
hop that is not itself a trusted proxy is taken as the client. A hop
that is not a bare IP address — `ip:port`, a bracketed IPv6 literal,
the token `unknown` — ends the walk and the peer address is used
instead, since past such an entry the chain is not the shape assumed
here. The peer address is likewise used when the header is absent or
every hop in it is a trusted proxy.
Two operator requirements follow:
- Your proxy must **append** the peer address to `X-Forwarded-For`
(nginx `$proxy_add_x_forwarded_for`, HAProxy `option forwardfor`,
Caddy and AWS ALB by default), and must append a bare address with
no port.
- List proxy hosts **only**. Any address inside `TRUSTED_PROXIES`
chooses its own rate-limit key: its `X-Forwarded-For` is walked, so
it can name a different address on every request to get a fresh
bucket each time, or name another client's address to drain that
client's bucket. Never list a block that also covers clients — a
broad `10.0.0.0/8` on a network where clients live in the same range
makes all three limits, including the unauthenticated webhook
receiver, silently bypassable by every client in the block.
Sessions are bounded by two independent clocks, and end at whichever
one runs out first:
- **Idle expiry** (`SESSION_IDLE_TIMEOUT`, default `24h`) is a sliding
window. Every authenticated request pushes it forward, so a session
in continuous use never hits it, while an abandoned one expires a day
after its last use. Set it to `0` to disable idle expiry entirely;
the absolute cap below still applies. A set-but-unparseable value
aborts startup rather than silently falling back to the default.
- **Absolute expiry** is a fixed 7 days from login. Activity does
**not** extend it: after a week, every session ends and the user
authenticates again.
Only requests that authenticate with the session count as activity, so
an unauthenticated request carrying the cookie cannot keep a session
alive. The idle timestamp is rewritten at most once per tenth of the
idle window rather than on every request, which means a session may
expire up to 10% early relative to the user's true last request, but
never late.
#### Invalid values abort startup
The defaults above apply **only** to variables that are unset (or set
to an empty string). A variable that is set but cannot be parsed is a
fatal configuration error: webhooker logs the offending variable and
its value and refuses to start, rather than silently running with a
substituted default. `PORT=eighty`, `DEBUG=ture`, and
`RETENTION_SWEEP_INTERVAL=1 hour` all abort startup. `PORT` must
additionally be a number in the range 165535,
`RECEIVER_RATE_LIMIT` must be at least 1, and every entry in
`TRUSTED_PROXIES` must be a CIDR block or a bare IP address.
Boolean variables (`DEBUG`, `MAINTENANCE_MODE`) accept exactly the
spellings Go's `strconv.ParseBool` accepts — `1`, `t`, `T`, `TRUE`,
`true`, `True`, `0`, `f`, `F`, `FALSE`, `false`, `False` — and nothing
else. `yes`, `on`, and `off` are rejected rather than quietly treated
as false.
On first startup, webhooker automatically generates a cryptographically
secure session encryption key and stores it in the database. This key
persists across restarts — no manual key management is needed.
On first startup, webhooker creates an `admin` user
with a randomly generated password and logs it to stdout. This password with a randomly generated password and logs it to stdout. This password
is only displayed once. is only displayed once.
@@ -82,16 +195,42 @@ is only displayed once.
```bash ```bash
docker run -d \ docker run -d \
-p 8080:8080 \ -p 8080:8080 \
-v /path/to/data:/data \ -v /path/to/data:/var/lib/webhooker \
-e DBURL="file:/data/webhooker.db?cache=shared&mode=rwc" \
-e SESSION_KEY="<base64-encoded-32-byte-key>" \
-e WEBHOOKER_ENVIRONMENT=prod \ -e WEBHOOKER_ENVIRONMENT=prod \
webhooker:latest webhooker:latest
``` ```
The container runs as a non-root user (`webhooker`, UID 1000), exposes The container runs as a non-root user (`webhooker`, UID 1000), exposes
port 8080, and includes a health check against port 8080, and includes a health check against
`/.well-known/healthcheck`. `/.well-known/healthcheck`. The `/var/lib/webhooker` volume holds all
SQLite databases: the main application database (`webhooker.db`) and
the per-webhook event databases (`events-{uuid}.db`). Mount this as a
persistent volume to preserve data across container restarts.
## Entrypoints
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call
them. We provide:
- `script/bootstrap` — install all dependencies (idempotent)
- `script/setup` — make a fresh clone ready for development
(bootstrap, then install-precommit)
- `script/projectname` — output the project name ("webhooker")
- `script/test` — run the test suite
- `script/lint` — run golangci-lint
- `script/fmt` — format all code (writes)
- `script/fmt-check` — check formatting (read-only)
- `script/check` — run test, lint, and fmt-check
- `script/docker` — build the Docker image tagged via `script/projectname`
- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile
runs the checks, so a green build implies a green repo)
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
`script/check`)
- `script/install-precommit` — install the git pre-commit hook that
runs `script/precommit`
## Rationale ## Rationale
@@ -158,25 +297,24 @@ It uses:
logging with TTY detection (text for dev, JSON for prod) logging with TTY detection (text for dev, JSON for prod)
- **[gorilla/sessions](https://github.com/gorilla/sessions)** for - **[gorilla/sessions](https://github.com/gorilla/sessions)** for
encrypted cookie-based session management encrypted cookie-based session management
- **[gorilla/csrf](https://github.com/gorilla/csrf)** for CSRF
protection (cookie-based double-submit tokens)
- **[go-chi/httprate](https://github.com/go-chi/httprate)** for
per-IP login rate limiting (sliding window counter)
- **[Prometheus](https://prometheus.io)** for metrics, served at - **[Prometheus](https://prometheus.io)** for metrics, served at
`/metrics` behind basic auth `/metrics` behind basic auth
- **[Sentry](https://sentry.io)** for optional error reporting - **[Sentry](https://sentry.io)** for optional error reporting
### Naming Conventions ### Naming Conventions
This README uses the target naming scheme for the application's core The codebase uses consistent naming throughout (rename completed in
entities. The current codebase uses older names that will be updated in
a future refactor (see
[issue #12](https://git.eeqj.de/sneak/webhooker/issues/12)): [issue #12](https://git.eeqj.de/sneak/webhooker/issues/12)):
| README (target name) | Current code name | Description | | Entity | Description |
| --------------------- | ----------------- | ----------- | | ---------------- | ----------- |
| **Webhook** | `Processor` | Top-level configuration entity grouping entrypoints and targets | | **Webhook** | Top-level configuration entity grouping entrypoints and targets |
| **Entrypoint** | `Webhook` | A receiver URL where external services POST events | | **Entrypoint** | A receiver URL where external services POST events |
| **Target** | `Target` | A delivery destination for events | | **Target** | A delivery destination for events |
Throughout this document, the target names are used. The code rename is
tracked separately.
### Data Model ### Data Model
@@ -196,11 +334,15 @@ tier** (event ingestion, delivery, and logging).
│ │ │ └──────────┘ └──────────────┘ │ │ │ │ └──────────┘ └──────────────┘ │
│ │ │──1:N──│ APIKey │ │ │ │ │──1:N──│ APIKey │ │
│ └──────────┘ └──────────┘ │ │ └──────────┘ └──────────┘ │
│ │
│ ┌──────────┐ │
│ │ Setting │ (key-value application config) │
│ └──────────┘ │
└─────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────────────┐
│ EVENT TIER │ │ EVENT TIER │
(planned: per-webhook dedicated database) (per-webhook dedicated databases)
│ │ │ │
│ ┌──────────┐ ┌──────────┐ ┌─────────────────┐ │ │ ┌──────────┐ ┌──────────┐ ┌─────────────────┐ │
│ │ Event │──1:N──│ Delivery │──1:N──│ DeliveryResult │ │ │ │ Event │──1:N──│ Delivery │──1:N──│ DeliveryResult │ │
@@ -208,6 +350,22 @@ tier** (event ingestion, delivery, and logging).
└─────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────┘
``` ```
#### Setting
A key-value pair for application-level configuration that is
auto-managed rather than user-provided. Used to store the session
encryption key and any future auto-generated settings.
| Field | Type | Description |
| ------- | ------ | ----------- |
| `key` | string | Primary key (setting name) |
| `value` | text | Setting value |
Currently stored settings:
- **`session_key`** — Base64-encoded 32-byte session encryption key,
auto-generated on first startup.
#### User #### User
A registered user of the webhooker service. A registered user of the webhooker service.
@@ -227,10 +385,10 @@ password logged to stdout.
#### Webhook #### Webhook
The top-level configuration entity (currently called "Processor" in The top-level configuration entity. A webhook groups together one or
code). A webhook groups together one or more entrypoints (receiver URLs) more entrypoints (receiver URLs) and one or more targets (delivery
and one or more targets (delivery destinations) into a logical unit. A destinations) into a logical unit. A user creates a webhook to set up
user creates a webhook to set up event routing. event routing.
| Field | Type | Description | | Field | Type | Description |
| ---------------- | ------- | ----------- | | ---------------- | ------- | ----------- |
@@ -238,24 +396,40 @@ user creates a webhook to set up event routing.
| `user_id` | UUID | Foreign key → User | | `user_id` | UUID | Foreign key → User |
| `name` | string | Human-readable name | | `name` | string | Human-readable name |
| `description` | string | Optional description | | `description` | string | Optional description |
| `retention_days` | integer | Days to retain events (default: 30) | | `retention_days` | integer | Days to retain events (default: 30; 0 means retain forever) |
**Relations:** Belongs to User. Has many Entrypoints. Has many Targets. **Relations:** Belongs to User. Has many Entrypoints. Has many Targets.
The `retention_days` field controls how long event data is kept in the The `retention_days` field controls how long event data is kept in the
webhook's dedicated database before automatic cleanup. webhook's dedicated database before automatic cleanup.
Setting `retention_days` to `0` means "retain events forever". Because
the column carries a default of 30, a literal zero cannot survive an
insert, so a zero is rewritten on save to a sentinel of `365 * 1000`
days (`database.RetentionForeverDays`). The retention reaper recognises
that sentinel and skips the webhook entirely, and the web UI displays
such a webhook's retention as "forever" rather than as a day count.
A *finite* retention is capped at `database.MaxFiniteRetentionDays`
(106751 days, about 292 years), and a larger one is rejected with a
400. The cap is not arbitrary: the reaper computes its cutoff as a
`time.Duration`, an int64 nanosecond count, and a longer period
overflows it. An overflowed cutoff lands in the future, where it
matches every row, so the sweep would delete every event the webhook
has instead of none. The reaper also clamps the value it is given, so a
row written by an older version cannot trigger that either.
#### Entrypoint #### Entrypoint
A receiver URL where external services POST webhook events (currently A receiver URL where external services POST webhook events. Each
called "Webhook" in code). Each entrypoint has a unique UUID-based path. entrypoint has a unique UUID-based path.
When an HTTP request arrives at an entrypoint's path, webhooker captures When an HTTP request arrives at an entrypoint's path, webhooker captures
the full request and creates an Event. the full request and creates an Event.
| Field | Type | Description | | Field | Type | Description |
| -------------- | ------- | ----------- | | -------------- | ------- | ----------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `path` | string | Unique URL path (UUID-based, e.g. `/webhook/{uuid}`) | | `path` | string | Unique URL path (UUID-based, e.g. `/webhook/{uuid}`) |
| `description` | string | Optional description | | `description` | string | Optional description |
| `active` | boolean | Whether this entrypoint accepts events (default: true) | | `active` | boolean | Whether this entrypoint accepts events (default: true) |
@@ -275,24 +449,31 @@ events should be forwarded.
| Field | Type | Description | | Field | Type | Description |
| ---------------- | ---------- | ----------- | | ---------------- | ---------- | ----------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `name` | string | Human-readable name | | `name` | string | Human-readable name |
| `type` | TargetType | One of: `http`, `retry`, `database`, `log` | | `type` | TargetType | One of: `http`, `slack`, `database`, `log` |
| `active` | boolean | Whether deliveries are enabled (default: true) | | `active` | boolean | Whether deliveries are enabled (default: true) |
| `config` | JSON text | Type-specific configuration | | `config` | JSON text | Type-specific configuration |
| `max_retries` | integer | Maximum retry attempts (for retry targets) | | `max_retries` | integer | Maximum retry attempts for HTTP targets (0 = fire-and-forget, >0 = retries with backoff) |
| `max_queue_size` | integer | Maximum queued deliveries (for retry targets) | | `max_queue_size` | integer | Maximum queued deliveries (for HTTP targets with retries) |
**Relations:** Belongs to Webhook. Has many Deliveries. **Relations:** Belongs to Webhook. Has many Deliveries.
**Target types:** **Target types:**
- **`http`** — Forward the event as an HTTP POST to a configured URL. - **`http`** — Forward the event as an HTTP POST to a configured URL.
Fire-and-forget: a single attempt with no retries. Behavior depends on `max_retries`: when `max_retries` is 0 (the
- **`retry`** — Forward the event via HTTP POST with automatic retry on default), the target operates in fire-and-forget mode — a single
failure. Uses exponential backoff up to `max_retries` attempts. attempt with no retries and no circuit breaker. When `max_retries` is
- **`database`** — Store the event in the webhook's database only (no greater than 0, failed deliveries are retried with exponential backoff
external delivery). Useful for pure logging/archival. up to `max_retries` attempts, protected by a per-target circuit
breaker.
- **`database`** — Archive the full event as a row into a separate
per-webhook archive database (`archive-{webhookID}.db`) for long-term
retention, with an optional creation-validated expiry (default: keep
forever). No external delivery and no retries; an archive write
failure fails the delivery. See the database target section under
"Per-Webhook Event Databases" for the full semantics.
- **`log`** — Write the event to the application log (stdout). Useful - **`log`** — Write the event to the application log (stdout). Useful
for debugging. for debugging.
@@ -321,8 +502,8 @@ data for replay and auditing.
| Field | Type | Description | | Field | Type | Description |
| -------------- | ------ | ----------- | | -------------- | ------ | ----------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `webhook_id` | UUID | Foreign key → Entrypoint | | `entrypoint_id` | UUID | Foreign key → Entrypoint |
| `method` | string | HTTP method (POST, PUT, etc.) | | `method` | string | HTTP method (POST, PUT, etc.) |
| `headers` | JSON | Complete request headers | | `headers` | JSON | Complete request headers |
| `body` | text | Raw request body | | `body` | text | Raw request body |
@@ -389,36 +570,39 @@ All entities include these fields from `BaseModel`:
### Database Architecture ### Database Architecture
#### Current Implementation #### Per-Webhook Event Databases
webhooker currently uses a **single SQLite database** for all data — webhooker uses **separate SQLite database files**: a main application
application configuration, user accounts, and (once implemented) event database for configuration data and per-webhook databases for event
storage. The database connection is managed by GORM with a single storage. All database files live in the `DATA_DIR` directory.
connection string configured via `DBURL`. On first startup the database
is auto-migrated and an `admin` user is created.
#### Planned: Per-Webhook Event Databases (Phase 2) **Main Application Database** (`{DATA_DIR}/webhooker.db`) — stores
configuration and application state:
In a future phase (see TODO Phase 2 below), webhooker will split into
**separate SQLite database files**: a main application database for
configuration data and per-webhook databases for event storage.
**Main Application Database** — will store:
- **Settings** — auto-managed key-value config (e.g. session encryption
key)
- **Users** — accounts and Argon2id password hashes - **Users** — accounts and Argon2id password hashes
- **Webhooks** (Processors) — webhook configurations - **Webhooks** — webhook configurations
- **Entrypoints** (Webhooks) — receiver URL definitions - **Entrypoints** — receiver URL definitions
- **Targets** — delivery destination configurations - **Targets** — delivery destination configurations
- **APIKeys** — programmatic access credentials - **APIKeys** — programmatic access credentials
**Per-Webhook Event Databases** — each webhook will get its own On first startup the main database is auto-migrated, a session
dedicated SQLite file containing: encryption key is generated and stored, and an `admin` user is created.
**Per-Webhook Event Databases** (`{DATA_DIR}/events-{webhook_uuid}.db`)
— each webhook gets its own dedicated SQLite file containing:
- **Events** — captured incoming webhook payloads - **Events** — captured incoming webhook payloads
- **Deliveries** — event-to-target pairings and their status - **Deliveries** — event-to-target pairings and their status
- **DeliveryResults** — individual delivery attempt logs - **DeliveryResults** — individual delivery attempt logs
This planned separation will provide: Per-webhook databases are created automatically when a webhook is
created (and lazily on first access for webhooks that predate this
feature). They are managed by the `WebhookDBManager` component, which
handles connection pooling, lazy opening, migrations, and cleanup.
This separation provides:
- **Isolation** — a high-volume webhook won't cause lock contention or - **Isolation** — a high-volume webhook won't cause lock contention or
WAL bloat affecting the main application or other webhooks. WAL bloat affecting the main application or other webhooks.
@@ -426,14 +610,72 @@ This planned separation will provide:
backed up, archived, rotated, or size-limited without impacting the backed up, archived, rotated, or size-limited without impacting the
application. application.
- **Clean deletion** — removing a webhook and all its history is as - **Clean deletion** — removing a webhook and all its history is as
simple as deleting one file. simple as deleting one file. Configuration is soft-deleted in the main
DB; the event database file is hard-deleted (permanently removed).
- **Per-webhook retention** — the `retention_days` field on each webhook - **Per-webhook retention** — the `retention_days` field on each webhook
will control automatic cleanup of old events in that webhook's controls automatic cleanup of old events in that webhook's database
database only. only, or disables cleanup entirely when set to `0` (retain forever).
- **Performance** — each webhook's database will have its own WAL, its - **Performance** — each webhook's database has its own WAL, its own
own page cache, and its own lock, so concurrent event ingestion across page cache, and its own lock, so concurrent event ingestion across
webhooks won't contend. webhooks won't contend.
The **database target type** builds on this architecture to provide
long-term archiving, separate from the per-webhook event database (which
may prune events under its own retention). Delivering to a database
target writes the full event — body, headers, method, content type, and
webhook/entrypoint/event identifiers — as a row into a dedicated archive
database, `archive-{webhookID}.db`, stored under the data directory
beside the event database. After each write the archive handle is closed
and reopened, debounced to at most once per second, so an operator can
move the archive file away for offline archiving without stopping the
service; a moved or removed archive file is recreated automatically on
the next write. An optional `expiry` in the target's config JSON (e.g.
`{"expiry":"720h"}`) is validated when the target is created — the
default (unset or the literal `never`) keeps rows forever — and rows
older than the expiry are pruned each time the archive is (re)opened. An
archive write failure is never silent success: the delivery records a
failed attempt with the error and is marked failed.
Because reopens only happen on writes, an archive belonging to a webhook
that has stopped receiving events would never be pruned. A background
**archive sweeper** closes that gap: on the same interval as the event
retention reaper (`RETENTION_SWEEP_INTERVAL`) it prunes every archive
whose database target declares a positive expiry, whether or not the
webhook is still receiving traffic. The sweep never creates an archive —
a webhook whose archive file does not yet exist is skipped, not
initialised — it takes the same per-webhook lock the write path uses, so
it can never interleave with a write, and it leaves the archive closed
afterwards so the move-the-file-away workflow keeps working. Archives
with no expiry, or the expiry `never`, are not touched by the sweep at
all.
Note that a webhook has one archive file but may carry more than one
`database` target, each with its own `expiry`. The shortest expiry
configured on any of them therefore governs the whole archive, and the
sweep applies it whether or not the webhook is still receiving events.
Configure a single `database` target per webhook unless you intend that.
Deleting a webhook releases its archive: the delivery engine's cached
archive writer is dropped and its file handle closed, so nothing lingers
after the webhook is gone. The archive **file itself is deliberately
left on disk**. Unlike the event database — per-webhook working storage
that is hard-deleted with the webhook — an archive is long-term storage
an operator may still want to keep or move away for offline retention,
and destroying it as a side effect of deleting a webhook would be
unrecoverable. Removing `archive-{webhookID}.db` is the operator's call.
Deleting a webhook's last `database` target releases the writer the same
way, and for the same reason leaves the file alone.
The **Slack target type** sends webhook events as formatted messages to
any Slack-compatible incoming webhook URL (works with Slack, Mattermost,
and other compatible services). Each message includes event metadata
(HTTP method, content type, timestamp, body size) and the payload
pretty-printed in a code block. JSON payloads are automatically
formatted with indentation for readability; non-JSON payloads are shown
as raw text. Large payloads are truncated to keep messages reasonable.
Config stores `webhook_url` — the Slack/Mattermost incoming webhook
endpoint.
The database uses the The database uses the
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at [modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at
runtime, though CGO is required at build time due to the transitive runtime, though CGO is required at build time due to the transitive
@@ -453,35 +695,175 @@ External Service
1. Look up Entrypoint by UUID 1. Look up Entrypoint by UUID
2. Capture full request as Event 2. Capture full request as Event
3. Queue Delivery to each active Target 3. Create Delivery records for each active Target
4. Build self-contained DeliveryTask structs
(target config + event data inline for ≤16KB)
5. Notify Engine via channel (no DB read needed)
┌──────────────┐ ┌──────────────┐
│ Delivery │ │ Delivery │◄── retry timers
│ Engine │ │ Engine │ (backoff)
│ (worker │
│ pool) │
└──────┬───────┘ └──────┬───────┘
┌────────────────────┼────────────────────┐ ┌── bounded worker pool (N workers) ──┐
▼ ▼ ▼ ▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐
│ HTTP Target│ │Retry Target│ │ Log Target │ │ HTTP Target│ │ HTTP Target│ │ Log Target │
│ (1 attempt)│ │ (backoff) │ │ (stdout) │ │(max_retries│ │(max_retries│ │ (stdout) │
└────────────┘ └────────────┘ └────────────┘ == 0) │ │ > 0, │ └────────────┘
│ fire+forget│ │ backoff + │
└────────────┘ │ circuit │
│ breaker) │
└────────────┘
``` ```
### Bounded Worker Pool
The delivery engine uses a **fixed-size worker pool** (default: 10
workers) to process all deliveries. At most N deliveries are in-flight
at any time, preventing goroutine explosions regardless of queue depth.
**Architecture:**
- **Channels as queues:** Two buffered channels serve as bounded queues:
a delivery channel (new tasks from the webhook handler) and a retry
channel (tasks from backoff timers). Both are buffered to 10,000.
- **Fan-out via channel, not goroutines:** When an event arrives with
multiple targets, each `DeliveryTask` is sent to the delivery channel.
Workers pick them up and process them — no goroutine-per-target.
- **Worker goroutines:** A fixed number of worker goroutines select from
both channels. Each worker processes one task at a time, then picks up
the next. Workers are the ONLY goroutines doing actual HTTP delivery.
- **Retry backpressure with DB fallback:** When a retry timer fires and
the retry channel is full, the timer is dropped — the delivery stays
in `retrying` status in the database. A periodic sweep (every 60s)
scans for these "orphaned" retries and re-queues them. No blocked
goroutines, no unbounded timer chains.
- **Bounded concurrency:** At most N deliveries (N = number of workers)
are in-flight simultaneously. Even if a circuit breaker is open for
hours and thousands of retries queue up in the channels, the workers
drain them at a controlled rate when the circuit closes.
This means:
- **No goroutine explosion** — even with 10,000 queued retries, only
N worker goroutines exist.
- **Natural backpressure** — if workers are busy, new tasks wait in the
channel buffer rather than spawning more goroutines.
- **Independent results** — each worker records its own delivery result
in the per-webhook database without coordination.
- **Graceful shutdown** — cancel the context, workers finish their
current task and exit. `WaitGroup.Wait()` ensures clean shutdown.
**Recovery paths:**
1. **Startup recovery:** When the engine starts, it scans all per-webhook
databases for `pending` and `retrying` deliveries. Pending deliveries
are sent to the delivery channel; retrying deliveries get backoff
timers scheduled.
2. **Periodic retry sweep (DB-mediated fallback):** Every 60 seconds the
engine scans for `retrying` deliveries whose backoff period has
elapsed. This catches "orphaned" retries — ones whose in-memory timer
was dropped because the retry channel was full. The database is the
durable fallback that ensures no retry is permanently lost, even under
extreme backpressure.
**Changing a target's type does not migrate in-flight deliveries.** Only
`http` and `slack` targets own durable retries; `database` and `log`
targets are fire-and-forget and never produce a `retrying` delivery. If a
target's `type` is edited from a retrying type to a non-retrying (or
unknown) one while one of its deliveries is still `retrying`, both
recovery paths above terminally mark that delivery `failed` and record a
`DeliveryResult` naming the current target type as the reason, logging it
at warn level. The delivery is not re-dispatched under the new type — the
operator never asked for that delivery — and the event itself remains
stored in the per-webhook event database, so it can be redelivered
manually.
### Circuit Breaker (HTTP Targets with Retries)
HTTP targets with `max_retries` > 0 are protected by a **per-target circuit breaker** that
prevents hammering a down target with repeated failed delivery attempts.
The circuit breaker is in-memory only and resets on restart (which is
fine — startup recovery rescans the database anyway).
**States:**
| State | Behavior |
| ----------- | -------- |
| **Closed** | Normal operation. Deliveries flow through. Consecutive failures are counted. |
| **Open** | Target appears down. Deliveries are skipped and rescheduled for after the cooldown. |
| **Half-Open** | Cooldown expired. One probe delivery is allowed to test if the target has recovered. |
**Transitions:**
```
success ┌──────────┐
┌────────────────────► │ Closed │ ◄─── probe succeeds
│ │ (normal) │
│ └────┬─────┘
│ │ N consecutive failures
│ ▼
│ ┌──────────┐
│ │ Open │ ◄─── probe fails
│ │(tripped) │
│ └────┬─────┘
│ │ cooldown expires
│ ▼
│ ┌──────────┐
└──────────────────────│Half-Open │
│ (probe) │
└──────────┘
```
**Defaults:**
- **Failure threshold:** 5 consecutive failures before opening
- **Cooldown:** 30 seconds in open state before probing
**Scope:** Circuit breakers only apply to **HTTP targets with
`max_retries` > 0**. Fire-and-forget HTTP targets (`max_retries` == 0),
Slack targets, database targets (local operations), and log
targets (stdout) do not use circuit breakers.
When a circuit is open and a new delivery arrives, the engine marks the
delivery as `retrying` and schedules a retry timer for after the
remaining cooldown period. This ensures no deliveries are lost — they're
just delayed until the target is healthy again.
### Rate Limiting ### Rate Limiting
Global rate limiting middleware (e.g., per-IP throttling applied at the Global blanket rate limiting middleware (e.g., a per-IP throttle shared
router level) **must not** apply to webhook receiver endpoints. Webhook with the web UI) **must not** apply to webhook receiver endpoints.
endpoints receive automated traffic from external services at Webhook endpoints receive automated traffic from external services at
unpredictable rates, and blanket rate limits would cause legitimate unpredictable rates, and blanket limits shared with other routes would
deliveries to be dropped. cause legitimate deliveries to be dropped.
Instead, each webhook has its own individually configurable rate limit, The receiver instead has its own dedicated abuse limit, scoped to the
applied within the webhook handler itself. By default, no rate limit is `/webhook/{uuid}` route only and keyed per client IP per entrypoint: one
applied — webhook endpoints accept traffic as fast as it arrives. Rate misbehaving sender is throttled without affecting other senders of the
limits can be configured per-webhook when needed (e.g., to protect same entrypoint or the same sender's other entrypoints. The limit is
against a misbehaving sender). `RECEIVER_RATE_LIMIT` requests per minute (default 120, generous for
legitimate webhook senders). Requests over the limit receive HTTP 429
with a `Retry-After` header. A set-but-invalid `RECEIVER_RATE_LIMIT`
value aborts startup rather than silently falling back to the default.
Every limiter here — receiver, login, and password change — identifies
the client the same way, through one shared key function: the
connection's own address, unless the peer is listed in
`TRUSTED_PROXIES`, in which case the forwarded client address is used
instead. See [Trusted proxies](#trusted-proxies). Deployed without that
variable set, a client behind a reverse proxy shares one bucket with
every other client behind the same proxy, which is the safe direction
to be wrong in: set `TRUSTED_PROXIES` to the proxy's address to get
per-client limits back.
Finer-grained per-webhook rate limits (configured in the web UI and
enforced in the webhook handler) can layer on top of this env-level
abuse limit later; they are tracked as future work.
### API Endpoints ### API Endpoints
@@ -489,7 +871,7 @@ against a misbehaving sender).
| Method | Path | Description | | Method | Path | Description |
| ------ | --------------------------- | ----------- | | ------ | --------------------------- | ----------- |
| `GET` | `/` | Web UI index page (server-rendered) | | `GET` | `/` | Root redirect (authenticated → `/sources`, unauthenticated → `/pages/login`) |
| `GET` | `/.well-known/healthcheck` | Health check (JSON: status, uptime, version) | | `GET` | `/.well-known/healthcheck` | Health check (JSON: status, uptime, version) |
| `GET` | `/s/*` | Static file serving (embedded CSS, JS) | | `GET` | `/s/*` | Static file serving (embedded CSS, JS) |
| `ANY` | `/webhook/{uuid}` | Webhook receiver endpoint (accepts all methods) | | `ANY` | `/webhook/{uuid}` | Webhook receiver endpoint (accepts all methods) |
@@ -515,6 +897,8 @@ against a misbehaving sender).
| `POST` | `/source/{id}/edit` | Edit webhook submission | | `POST` | `/source/{id}/edit` | Edit webhook submission |
| `POST` | `/source/{id}/delete` | Delete webhook | | `POST` | `/source/{id}/delete` | Delete webhook |
| `GET` | `/source/{id}/logs` | Webhook event logs | | `GET` | `/source/{id}/logs` | Webhook event logs |
| `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook |
| `POST` | `/source/{id}/targets` | Add target to webhook |
#### Infrastructure Endpoints #### Infrastructure Endpoints
@@ -548,51 +932,56 @@ webhooker/
│ └── main.go # Entry point: sets globals, wires fx │ └── main.go # Entry point: sets globals, wires fx
├── internal/ ├── internal/
│ ├── config/ │ ├── config/
│ │ └── config.go # Configuration loading via pkg/config │ │ └── config.go # Configuration loading from environment variables
│ ├── database/ │ ├── database/
│ │ ├── base_model.go # BaseModel with UUID primary keys │ │ ├── base_model.go # BaseModel with UUID primary keys
│ │ ├── database.go # GORM connection, migrations, admin seed │ │ ├── database.go # GORM connection, migrations, admin seed
│ │ ├── models.go # AutoMigrate for all models │ │ ├── models.go # AutoMigrate for config-tier models
│ │ ├── model_setting.go # Setting entity (key-value app config)
│ │ ├── model_user.go # User entity │ │ ├── model_user.go # User entity
│ │ ├── model_processor.go # Webhook entity (to be renamed) │ │ ├── model_webhook.go # Webhook entity
│ │ ├── model_webhook.go # Entrypoint entity (to be renamed) │ │ ├── model_entrypoint.go # Entrypoint entity
│ │ ├── model_target.go # Target entity and TargetType enum │ │ ├── model_target.go # Target entity and TargetType enum
│ │ ├── model_event.go # Event entity │ │ ├── model_event.go # Event entity (per-webhook DB)
│ │ ├── model_delivery.go # Delivery entity and DeliveryStatus enum │ │ ├── model_delivery.go # Delivery entity (per-webhook DB)
│ │ ├── model_delivery_result.go # DeliveryResult entity │ │ ├── model_delivery_result.go # DeliveryResult entity (per-webhook DB)
│ │ ├── model_apikey.go # APIKey entity │ │ ├── model_apikey.go # APIKey entity
│ │ ── password.go # Argon2id hashing and verification │ │ ── password.go # Argon2id hashing and verification
│ │ └── webhook_db_manager.go # Per-webhook DB lifecycle manager
│ ├── globals/ │ ├── globals/
│ │ └── globals.go # Build-time variables (appname, version, arch) │ │ └── globals.go # Build-time variables (appname, version, arch)
│ ├── delivery/
│ │ ├── engine.go # Event-driven delivery engine (channel + timer based)
│ │ ├── circuit_breaker.go # Per-target circuit breaker for HTTP targets with retries
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
│ ├── handlers/ │ ├── handlers/
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering │ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
│ │ ├── auth.go # Login, logout handlers │ │ ├── auth.go # Login, logout handlers
│ │ ├── healthcheck.go # Health check handler │ │ ├── healthcheck.go # Health check handler
│ │ ├── index.go # Index page handler │ │ ├── index.go # Index page handler
│ │ ├── profile.go # User profile handler │ │ ├── profile.go # User profile handler
│ │ ├── source_management.go # Webhook CRUD handlers (stubs) │ │ ├── source_management.go # Webhook CRUD handlers
│ │ └── webhook.go # Webhook receiver handler │ │ └── webhook.go # Webhook receiver handler
│ ├── healthcheck/ │ ├── healthcheck/
│ │ └── healthcheck.go # Health check service (uptime, version) │ │ └── healthcheck.go # Health check service (uptime, version)
│ ├── logger/ │ ├── logger/
│ │ └── logger.go # slog setup with TTY detection │ │ └── logger.go # slog setup with TTY detection
│ ├── middleware/ │ ├── middleware/
│ │ ── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth │ │ ── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth, SecurityHeaders, MaxBodySize
│ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf)
│ │ └── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate)
│ ├── server/ │ ├── server/
│ │ ├── server.go # Server struct, fx lifecycle, signal handling │ │ ├── server.go # Server struct, fx lifecycle, signal handling
│ │ ├── http.go # HTTP server setup with timeouts │ │ ├── http.go # HTTP server setup with timeouts
│ │ └── routes.go # All route definitions │ │ └── routes.go # All route definitions
│ └── session/ │ └── session/
│ └── session.go # Cookie-based session management │ └── session.go # Cookie-based session management
├── pkg/config/ # Reusable multi-environment config library
├── static/ ├── static/
│ ├── static.go # //go:embed directive │ ├── static.go # //go:embed directive
│ ├── css/style.css # Custom stylesheet (system font stack, card effects, layout) │ ├── css/style.css # Custom stylesheet (system font stack, card effects, layout)
│ └── js/app.js # Client-side JavaScript (minimal bootstrap) │ └── js/app.js # Client-side JavaScript (minimal bootstrap)
├── templates/ # Go HTML templates (base, index, login, etc.) ├── templates/ # Go HTML templates (base, index, login, etc.)
├── configs/ ├── Dockerfile # Multi-stage: lint, build+test, then Alpine runtime
│ └── config.yaml.example # Example configuration file
├── Dockerfile # Multi-stage: build + check, then Alpine runtime
├── Makefile # fmt, lint, test, check, build, docker targets ├── Makefile # fmt, lint, test, check, build, docker targets
├── go.mod / go.sum ├── go.mod / go.sum
└── .golangci.yml # Linter configuration └── .golangci.yml # Linter configuration
@@ -604,16 +993,26 @@ Components are wired via Uber fx in this order:
1. `globals.New` — Build-time variables (appname, version, arch) 1. `globals.New` — Build-time variables (appname, version, arch)
2. `logger.New` — Structured logging (slog with TTY detection) 2. `logger.New` — Structured logging (slog with TTY detection)
3. `config.New` — Configuration loading (pkg/config + environment) 3. `config.New` — Configuration loading (environment variables)
4. `database.New` — SQLite connection, migrations, admin user seed 4. `database.New` Main SQLite connection, config migrations, admin
5. `healthcheck.New` — Health check service user seed
6. `session.New` — Cookie-based session manager 5. `database.NewWebhookDBManager` — Per-webhook event database
7. `handlers.New` — HTTP handlers lifecycle manager
8. `middleware.New` — HTTP middleware 6. `healthcheck.New` — Health check service
9. `server.New`HTTP server and router 7. `session.New`Cookie-based session manager (key from database)
8. `handlers.New` — HTTP handlers
9. `middleware.New` — HTTP middleware
10. `delivery.New` — Event-driven delivery engine
11. `delivery.Engine``handlers.DeliveryNotifier` — interface bridge
12. `server.New` — HTTP server and router
The server starts via `fx.Invoke(func(*server.Server) {})` which The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine)
triggers the fx lifecycle hooks in dependency order. {})` which triggers the fx lifecycle hooks in dependency order. The
`DeliveryNotifier` interface allows the webhook handler to send
self-contained `DeliveryTask` slices to the engine without a direct
package dependency. Each task carries all target config and event data
inline (for bodies ≤16KB), so the engine can deliver without reading
from any database — it only writes to record results.
### Middleware Stack ### Middleware Stack
@@ -621,14 +1020,29 @@ Applied to all routes in this order:
1. **Recoverer** — Panic recovery (chi built-in) 1. **Recoverer** — Panic recovery (chi built-in)
2. **RequestID** — Generate unique request IDs (chi built-in) 2. **RequestID** — Generate unique request IDs (chi built-in)
3. **Logging** — Structured request logging (method, URL, status, 3. **SecurityHeaders** — Production security headers on every response
(HSTS, X-Content-Type-Options, X-Frame-Options, CSP, Referrer-Policy,
Permissions-Policy)
4. **Logging** — Structured request logging (method, URL, status,
latency, remote IP, user agent, request ID) latency, remote IP, user agent, request ID)
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set) 5. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
5. **CORS** — Cross-origin resource sharing headers 6. **CORS** — Cross-origin resource sharing headers
6. **Timeout** — 60-second request timeout 7. **Timeout** — 60-second request timeout
7. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set; 8. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set;
configured with `Repanic: true` so panics still reach Recoverer) configured with `Repanic: true` so panics still reach Recoverer)
Additionally, form endpoints (`/pages`, `/user/*`, `/sources`,
`/source/*`) apply a **MaxBodySize** middleware that limits
POST/PUT/PATCH request bodies to 1 MB. It is registered ahead of the
CSRF middleware in every one of those route groups, because
gorilla/csrf parses the form; if the cap were installed after it, form
parsing would run under net/http's 10 MB default and the 1 MB limit
would never apply. A request that declares a `Content-Length` over the
limit is answered with `413 Request Entity Too Large` before any other
middleware or handler runs; a chunked request, or one that lies about
its length, is hard-capped by `http.MaxBytesReader` and fails
downstream at form-parse time.
### Authentication ### Authentication
- **Web UI:** Cookie-based sessions using gorilla/sessions with - **Web UI:** Cookie-based sessions using gorilla/sessions with
@@ -643,7 +1057,29 @@ Applied to all routes in this order:
- Passwords hashed with Argon2id (64 MB memory cost) - Passwords hashed with Argon2id (64 MB memory cost)
- Session cookies are HttpOnly, SameSite Lax, Secure (prod only) - Session cookies are HttpOnly, SameSite Lax, Secure (prod only)
- Session key must be a 32-byte base64-encoded value - Session regeneration on login to prevent session fixation attacks
- Session key is a 32-byte value auto-generated on first startup and
stored in the database
- Production security headers on all responses: HSTS, X-Content-Type-Options
(`nosniff`), X-Frame-Options (`DENY`), Content-Security-Policy, Referrer-Policy,
and Permissions-Policy
- Request body size limits (1 MB) on all form POST endpoints, enforced
by middleware that runs before CSRF parses the form
- **CSRF protection** via [gorilla/csrf](https://github.com/gorilla/csrf)
on all state-changing forms (cookie-based double-submit tokens with
HMAC authentication). Applied to `/pages`, `/sources`, `/source`, and
`/user` routes. Excluded from `/webhook` (inbound webhook POSTs) and
`/api` (stateless API). The middleware auto-detects TLS status
per-request (via `r.TLS` and `X-Forwarded-Proto`) to set appropriate
cookie security flags and Origin/Referer validation mode
- **SSRF prevention** for HTTP delivery targets: private/reserved IP
ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked
both at target creation time (URL validation) and at delivery time
(custom HTTP transport with SSRF-safe dialer that validates resolved
IPs before connecting, preventing DNS rebinding attacks)
- **Login rate limiting** via [go-chi/httprate](https://github.com/go-chi/httprate):
per-IP sliding-window rate limiter on the login endpoint (5 POST
attempts per minute per IP) to prevent brute-force attacks
- Prometheus metrics behind basic auth - Prometheus metrics behind basic auth
- Static assets embedded in binary (no filesystem access needed at - Static assets embedded in binary (no filesystem access needed at
runtime) runtime)
@@ -657,7 +1093,8 @@ The Dockerfile uses a multi-stage build:
1. **Builder stage** (Debian-based `golang:1.24`) — installs 1. **Builder stage** (Debian-based `golang:1.24`) — installs
golangci-lint, downloads dependencies, copies source, runs `make golangci-lint, downloads dependencies, copies source, runs `make
check` (format verification, linting, tests, compilation). check` (format verification, linting, tests, compilation).
2. **Runtime stage** (`alpine:3.21`) — copies the binary, runs as 2. **Runtime stage** (`alpine:3.21`) — copies the binary, creates the
`/var/lib/webhooker` directory for all SQLite databases, runs as
non-root user, exposes port 8080, includes a health check. non-root user, exposes port 8080, includes a health check.
The builder uses Debian rather than Alpine because GORM's SQLite The builder uses Debian rather than Alpine because GORM's SQLite
@@ -669,65 +1106,7 @@ linted, tested, and compiled.
## TODO ## TODO
### Phase 1: Core Webhook Engine See [TODO.md](TODO.md).
- [ ] Implement webhook reception and event storage at `/webhook/{uuid}`
- [ ] Build event processing and target delivery engine
- [ ] Implement HTTP target type (fire-and-forget POST)
- [ ] Implement retry target type (exponential backoff)
- [ ] Implement database target type (store only)
- [ ] Implement log target type (console output)
- [ ] Per-webhook rate limiting in the receiver handler
- [ ] Webhook signature verification (GitHub, Stripe formats)
### Phase 2: Database Separation
- [ ] Split into main application DB + per-webhook event DBs
- [ ] Automatic event retention cleanup based on `retention_days`
- [ ] Per-webhook database lifecycle management (create on webhook
creation, delete on webhook removal)
### Phase 3: Security & Infrastructure
- [ ] Implement authentication middleware for protected routes
([#9](https://git.eeqj.de/sneak/webhooker/issues/9))
- [ ] Security headers (HSTS, CSP, X-Frame-Options)
- [ ] CSRF protection for forms
- [ ] Session expiration and "remember me"
- [ ] Password change/reset flow
- [ ] API key authentication for programmatic access
### Phase 4: Web UI
- [ ] Webhook management pages (list, create, edit, delete)
- [ ] Webhook request log viewer with filtering
- [ ] Delivery status and retry management UI
- [ ] Manual event redelivery
- [ ] Analytics dashboard (success rates, response times)
- [ ] Replace Bootstrap with Tailwind CSS + Alpine.js
([#4](https://git.eeqj.de/sneak/webhooker/issues/4))
### Phase 5: REST API
- [ ] RESTful CRUD for webhooks, entrypoints, targets
- [ ] Event viewing and filtering endpoints
- [ ] Event redelivery endpoint
- [ ] OpenAPI specification
### Phase 6: Code Quality
- [ ] Rename Processor → Webhook, Webhook → Entrypoint in code
([#12](https://git.eeqj.de/sneak/webhooker/issues/12))
- [ ] Embed templates via `//go:embed`
([#7](https://git.eeqj.de/sneak/webhooker/issues/7))
- [ ] Use `slog.LevelVar` for dynamic log level switching
([#8](https://git.eeqj.de/sneak/webhooker/issues/8))
- [ ] Simplify configuration to prefer environment variables
([#10](https://git.eeqj.de/sneak/webhooker/issues/10))
- [ ] Remove redundant `godotenv/autoload` import
([#11](https://git.eeqj.de/sneak/webhooker/issues/11))
### Future
- [ ] Email delivery target type
- [ ] SNS, S3, Slack delivery targets
- [ ] Data transformations (e.g., webhook-to-Slack message formatting)
- [ ] JSONL file delivery with periodic S3 upload
- [ ] Webhook event search and filtering
- [ ] Multi-user with role-based access
## License ## License

View File

@@ -1,6 +1,6 @@
--- ---
title: Repository Policies title: Repository Policies
last_modified: 2026-02-22 last_modified: 2026-07-06
--- ---
This document covers repository structure, tooling, and workflow standards. Code This document covers repository structure, tooling, and workflow standards. Code
@@ -34,10 +34,46 @@ style conventions are in separate documents:
every file before committing. There are zero exceptions to this rule. every file before committing. There are zero exceptions to this rule.
- Every repo with software must have a root `Makefile` with these targets: - Every repo with software must have a root `Makefile` with these targets:
`make test`, `make lint`, `make fmt` (writes), `make fmt-check` (read-only), `make bootstrap`, `make setup`, `make test`, `make lint`, `make fmt` (writes),
`make check` (prereqs: `test`, `lint`, `fmt-check`), `make docker`, and `make fmt-check` (read-only), `make check` (runs `test`, `lint`, `fmt-check`),
`make hooks` (installs pre-commit hook). A model Makefile is at `make docker`, and `make hooks` (installs pre-commit hook). A model Makefile
`https://git.eeqj.de/sneak/prompts/raw/branch/main/Makefile`. is at `https://git.eeqj.de/sneak/prompts/raw/branch/main/Makefile`.
- Repos follow the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
pattern: the implementation of each Makefile target lives in an executable
script in `script/` (`script/bootstrap`, `script/setup`, `script/test`,
`script/lint`, `script/fmt`, `script/fmt-check`, `script/check`,
`script/docker`), and the Makefile targets are thin shims that call them. The
scripts must be POSIX sh (`#!/bin/sh`, `set -eu`, no bashisms) so they run in
minimal containers (e.g. alpine images have no bash); locate the repo root
with `$(cd "$(dirname "$0")/.." && pwd -P)` and `cd` there before acting. From
the standard's canonical set we use `bootstrap`, `setup` (make the repo ready
for development after a fresh clone: runs `bootstrap`, then
`install-precommit`, plus any repo-specific initialization), `test`, and
`cibuild`. `script/bootstrap` installs all dependencies idempotently and
assumes nothing is present: base tools come from nix, apt, brew, or apk
(detected in that order; apt runs noninteractive). For node it uses the
installed node if present; otherwise it installs a PINNED node version via
nvm, first installing nvm itself if missing — from a hash-verified GitHub
release archive (never `curl | sh`), with bash installed as an explicit
prerequisite since nvm requires bash. yarn is then pinned via
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
always exact versions. `script/cibuild` runs the CI build: it changes to the
repo root and runs `docker build .`; the Gitea workflow calls it. Four further
scripts are our own extensions to the standard: `script/check` runs
`script/test`, `script/lint`, and `script/fmt-check`; `script/precommit` is
what the git pre-commit hook runs, and it calls `script/check`;
`script/install-precommit` installs the git pre-commit hook (the `make hooks`
target shims to it); and `script/projectname` (literally that filename) simply
outputs the project's name. Scripts that need the name call
`script/projectname` — e.g. `script/docker` assembles its image tag from it —
so those scripts stay byte-identical across all repos. Repo-type-specific
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
`script/precommit`, not in the hook itself. Model scripts are at
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
must document the provided scripts in an **Entrypoints** section (see the
README requirements below).
- Always use Makefile targets (`make fmt`, `make test`, `make lint`, etc.) - Always use Makefile targets (`make fmt`, `make test`, `make lint`, etc.)
instead of invoking the underlying tools directly. The Makefile is the single instead of invoking the underlying tools directly. The Makefile is the single
@@ -57,11 +93,83 @@ style conventions are in separate documents:
as a build step so the build fails if the branch is not green. For non-server as a build step so the build fails if the branch is not green. For non-server
repos, the Dockerfile should bring up a development environment and run repos, the Dockerfile should bring up a development environment and run
`make check`. For server repos, `make check` should run as an early build `make check`. For server repos, `make check` should run as an early build
stage before the final image is assembled. stage before the final image is assembled. Dockerfiles install development
prerequisites by running `script/bootstrap` rather than duplicating installs
inline; COPY `script/` and the dependency manifests (`package.json` +
`yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap
layer stays cached until dependencies change.
- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go
repos use a multistage build where linting runs in an independent stage based
on the `golangci/golangci-lint` image (pinned by hash). This stage runs
`make fmt-check` and `make lint` before the full build begins. The build stage
then declares an explicit dependency on the lint stage via
`COPY --from=lint /src/go.sum /dev/null`, which forces BuildKit to complete
linting before proceeding to compilation and tests. This ensures lint failures
surface in seconds rather than minutes, without blocking on dependency
download or compilation in the build stage.
The standard pattern for a Go repo Dockerfile is:
```dockerfile
# Lint stage — fast feedback on formatting and lint issues
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN make fmt-check
RUN make lint
# Build stage
# golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder
WORKDIR /src
# Force BuildKit to run the lint stage before proceeding
COPY --from=lint /src/go.sum /dev/null
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN make test
ARG VERSION=dev
RUN CGO_ENABLED=0 go build -trimpath \
-ldflags="-s -w -X main.Version=${VERSION}" \
-o /app ./cmd/app/
# Runtime stage
FROM alpine@sha256:...
COPY --from=builder /app /usr/local/bin/app
ENTRYPOINT ["app"]
```
Key points:
- The lint stage uses the `golangci/golangci-lint` image directly (it
includes both Go and the linter), so there is no need to install the
linter separately.
- `COPY --from=lint /src/go.sum /dev/null` is a no-op file copy that creates
a stage dependency. BuildKit runs stages in parallel by default; without
this line, the build stage would not wait for lint to finish and a lint
failure might not fail the overall build.
- If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint stage must
create placeholder files so the embed directives resolve. Example:
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
The lint stage should not depend on the actual build output — it exists to
fail fast.
- If the project requires CGO or system libraries for linting (e.g.
`vips-dev`), install them in the lint stage with `apk add`.
- The build stage runs `make test` after compilation setup. Tests run in the
build stage, not the lint stage, because they may require compiled
artifacts or heavier dependencies.
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that - Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `docker build .` on push. Since the Dockerfile already runs `make check`, runs `script/cibuild` (which runs `docker build .`) on push. Since the
a successful build implies all checks pass. Dockerfile already runs `make check`, a successful build implies all checks
pass.
- Use platform-standard formatters: `black` for Python, `prettier` for - Use platform-standard formatters: `black` for Python, `prettier` for
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
@@ -69,9 +177,11 @@ style conventions are in separate documents:
Markdown (hard-wrap at 80 columns). Documentation and writing repos (Markdown, Markdown (hard-wrap at 80 columns). Documentation and writing repos (Markdown,
HTML, CSS) should also have `.prettierrc` and `.prettierignore`. HTML, CSS) should also have `.prettierrc` and `.prettierignore`.
- Pre-commit hook: `make check` if local testing is possible, otherwise - Pre-commit hook: runs `script/precommit`, which calls `script/check`. If local
`make lint && make fmt-check`. The Makefile should provide a `make hooks` testing is not possible in the repo, `script/precommit` may skip `script/test`
target to install the pre-commit hook. and run only `script/lint` and `script/fmt-check`. The hook is installed by
`script/install-precommit`; the Makefile must provide a `make hooks` target
that shims to it.
- All repos with software must have tests that run via the platform-standard - All repos with software must have tests that run via the platform-standard
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful
@@ -82,6 +192,42 @@ style conventions are in separate documents:
- `make test` must complete in under 20 seconds. Add a 30-second timeout in the - `make test` must complete in under 20 seconds. Add a 30-second timeout in the
Makefile. Makefile.
- **`make test` should use the conditional verbose rerun pattern.** Run tests
without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to
show full output. This keeps CI logs and `docker build` output clean on
success (just package/suite summaries) while providing full diagnostic detail
on failure (every test case, every assertion). The general shell pattern:
```makefile
test:
@<test-command> || \
{ echo "--- Rerunning with -v for details ---"; \
<test-command-with-v>; exit 1; }
```
Go example:
```makefile
test:
@go test -timeout 30s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -timeout 30s -race -v ./...; exit 1; }
```
Python example:
```makefile
test:
@python -m pytest || \
{ echo "--- Rerunning with -v for details ---"; \
python -m pytest -v; exit 1; }
```
The `exit 1` ensures the target always fails after a rerun — the first run
already proved the tests are broken, so the build must not pass even if a
flaky test happens to succeed on the second attempt. The rerun exists solely
for diagnostic output.
- Docker builds must complete in under 5 minutes. - Docker builds must complete in under 5 minutes.
- `make check` must not modify any files in the repo. Tests may use temporary - `make check` must not modify any files in the repo. Tests may use temporary
@@ -98,6 +244,13 @@ style conventions are in separate documents:
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
a new repo. a new repo.
- **No build artifacts in version control.** Code-derived data (compiled
bundles, minified output, generated assets) must never be committed to the
repository if it can be avoided. The build process (e.g. Dockerfile, Makefile)
should generate these at build time. Notable exception: Go protobuf generated
files (`.pb.go`) ARE committed because repos need to work with `go get`, which
downloads code but does not execute code generation.
- Never use `git add -A` or `git add .`. Always stage files explicitly by name. - Never use `git add -A` or `git add .`. Always stage files explicitly by name.
- Never force-push to `main`. - Never force-push to `main`.
@@ -121,12 +274,76 @@ style conventions are in separate documents:
- Dockerized web services listen on port 8080 by default, overridable with - Dockerized web services listen on port 8080 by default, overridable with
`PORT`. `PORT`.
- **HTTP/web services must be hardened for production internet exposure before
tagging 1.0.** This means full compliance with security best practices
including, without limitation, all of the following:
- **Security headers** on every response:
- `Strict-Transport-Security` (HSTS) with `max-age` of at least one year
and `includeSubDomains`.
- `Content-Security-Policy` (CSP) with a restrictive default policy
(`default-src 'self'` as a baseline, tightened per-resource as
needed). Never use `unsafe-inline` or `unsafe-eval` unless
unavoidable, and document the reason.
- `X-Frame-Options: DENY` (or `SAMEORIGIN` if framing is required).
Prefer the `frame-ancestors` CSP directive as the primary control.
- `X-Content-Type-Options: nosniff`.
- `Referrer-Policy: strict-origin-when-cross-origin` (or stricter).
- `Permissions-Policy` restricting access to browser features the
application does not use (camera, microphone, geolocation, etc.).
- **Request and response limits:**
- Maximum request body size enforced on all endpoints (e.g. Go
`http.MaxBytesReader`). Choose a sane default per-route; never accept
unbounded input.
- Maximum response body size where applicable (e.g. paginated APIs).
- `ReadTimeout` and `ReadHeaderTimeout` on the `http.Server` to defend
against slowloris attacks.
- `WriteTimeout` on the `http.Server`.
- `IdleTimeout` on the `http.Server`.
- Per-handler execution time limits via `context.WithTimeout` or
chi/stdlib `middleware.Timeout`.
- **Authentication and session security:**
- Rate limiting on password-based authentication endpoints. API keys are
high-entropy and not susceptible to brute force, so they are exempt.
- CSRF tokens on all state-mutating HTML forms. API endpoints
authenticated via `Authorization` header (Bearer token, API key) are
exempt because the browser does not attach these automatically.
- Passwords stored using bcrypt, scrypt, or argon2 — never plain-text,
MD5, or SHA.
- Session cookies set with `HttpOnly`, `Secure`, and `SameSite=Lax` (or
`Strict`) attributes.
- **Reverse proxy awareness:**
- True client IP detection when behind a reverse proxy
(`X-Forwarded-For`, `X-Real-IP`). The application must accept
forwarded headers only from a configured set of trusted proxy
addresses — never trust `X-Forwarded-For` unconditionally.
- **CORS:**
- Authenticated endpoints must restrict `Access-Control-Allow-Origin` to
an explicit allowlist of known origins. Wildcard (`*`) is acceptable
only for public, unauthenticated read-only APIs.
- **Error handling:**
- Internal errors must never leak stack traces, SQL queries, file paths,
or other implementation details to the client. Return generic error
messages in production; detailed errors only when `DEBUG` is enabled.
- **TLS:**
- Services never terminate TLS directly. They are always deployed behind
a TLS-terminating reverse proxy. The service itself listens on plain
HTTP. However, HSTS headers and `Secure` cookie flags must still be
set by the application so that the browser enforces HTTPS end-to-end.
This list is non-exhaustive. Apply defense-in-depth: if a standard security
hardening measure exists for HTTP services and is not listed here, it is
still expected. When in doubt, harden.
- `README.md` is the primary documentation. Required sections: - `README.md` is the primary documentation. Required sections:
- **Description**: First line must include the project name, purpose, - **Description**: First line must include the project name, purpose,
category (web server, SPA, CLI tool, etc.), license, and author. Example: category (web server, SPA, CLI tool, etc.), license, and author. Example:
"µPaaS is an MIT-licensed Go web application by @sneak that receives "µPaaS is an MIT-licensed Go web application by @sneak that receives
git-frontend webhooks and deploys applications via Docker in realtime." git-frontend webhooks and deploys applications via Docker in realtime."
- **Getting Started**: Copy-pasteable install/usage code block. - **Getting Started**: Copy-pasteable install/usage code block.
- **Entrypoints**: Opens by stating that the repo adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard (with that link), then documents each provided `script/`
entrypoint and its purpose.
- **Rationale**: Why does this exist? - **Rationale**: Why does this exist?
- **Design**: How is the program structured? - **Design**: How is the program structured?
- **TODO**: Update meticulously, even between commits. When planning, put - **TODO**: Update meticulously, even between commits. When planning, put
@@ -145,11 +362,11 @@ style conventions are in separate documents:
- Database migrations live in `internal/db/migrations/` and must be embedded in - Database migrations live in `internal/db/migrations/` and must be embedded in
the binary. the binary.
- `000_migration.sql` — contains ONLY the creation of the migrations tracking - `000_migration.sql` — contains ONLY the creation of the migrations
table itself. Nothing else. tracking table itself. Nothing else.
- `001_schema.sql` — the full application schema. - `001_schema.sql` — the full application schema.
- **Pre-1.0.0:** never add additional migration files (002, 003, etc.). There - **Pre-1.0.0:** never add additional migration files (002, 003, etc.).
is no installed base to migrate. Edit `001_schema.sql` directly. There is no installed base to migrate. Edit `001_schema.sql` directly.
- **Post-1.0.0:** add new numbered migration files for each schema change. - **Post-1.0.0:** add new numbered migration files for each schema change.
Never edit existing migrations after release. Never edit existing migrations after release.
@@ -181,6 +398,9 @@ style conventions are in separate documents:
- `README.md`, `.git`, `.gitignore`, `.editorconfig` - `README.md`, `.git`, `.gitignore`, `.editorconfig`
- `LICENSE`, `REPO_POLICIES.md` (copy from the `prompts` repo) - `LICENSE`, `REPO_POLICIES.md` (copy from the `prompts` repo)
- `Makefile` - `Makefile`
- `script/` entrypoints (`bootstrap`, `setup`, `projectname`, `test`,
`lint`, `fmt`, `fmt-check`, `check`, `docker`, `cibuild`, `precommit`,
`install-precommit`)
- `Dockerfile`, `.dockerignore` - `Dockerfile`, `.dockerignore`
- `.gitea/workflows/check.yml` - `.gitea/workflows/check.yml`
- Go: `go.mod`, `go.sum`, `.golangci.yml` - Go: `go.mod`, `go.sum`, `.golangci.yml`

108
TODO.md Normal file
View File

@@ -0,0 +1,108 @@
# Workflow
* branch (from `main`)
* do the work in Next Step
* move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work)
* merge to `main` if the branch is not protected, otherwise open a PR
* push
# Status
pre-1.0. No git tags exist. main (4f5ecb1) is a working webhook proxy
with auth, CSRF/SSRF protections, login rate limiting, Slack target,
event retention (#63), the database archiving target (#43), the admin
password change flow (#65), policy compliance (#6), pinned lint tooling
(#55), and fail-loud configuration parsing (#80). Note: TODO.md was
deliberately deleted from this repo in f9a9569 (2026-03-01, #6); its
content was folded into the README TODO section, which this draft
reconstructs as of 2026-07-06.
# Next Step
Manual event redelivery from the web UI (replay is a core promised
capability in the README rationale).
# Completed Steps
- 2026-08-11 Web UI cleanup: nav terminology unified on Webhooks, the
Profile settings placeholder removed, a progressive-enhancement copy
button for the entrypoint URL, and retention form copy that states the
actual policy (deletion by the reaper, 0 retains forever) (#57)
- 2026-08-09 Inactivity-based session timeout: sliding idle expiry
(`SESSION_IDLE_TIMEOUT`, default `24h`) refreshed on authenticated
requests, with the 7-day absolute cap kept as an independent
backstop that activity never extends (#66)
- 2026-08-09 Restart recovery and the 60s retry sweep terminally fail an
orphaned `retrying` delivery whose target type no longer supports
retries, recording a `DeliveryResult` with the reason instead of
leaving the delivery stuck forever (#82)
- 2026-08-09 Root the delivery engine's worker pool and the retention
reaper's sweep loop at `context.Background()` rather than the fx
`OnStart` hook context (#97), which carries fx's 15s start timeout and
killed both roughly fifteen seconds after boot: the proxy silently
stopped delivering webhooks entirely, and the reaper never ran a
single sweep under its default one-hour interval
- 2026-08-09 Archive writer lifecycle (#89): deleting a webhook (or its
last `database` target) evicts the cached archive writer and closes
its handle while deliberately leaving `archive-{webhookID}.db` on
disk, and a new `ArchiveSweeper` prunes idle archives on the existing
`RETENTION_SWEEP_INTERVAL` without ever creating an archive file
- 2026-08-09 Configuration parsing fails loudly on set-but-unparseable
environment values: `envInt` removed in favour of `envPositiveInt`
plus a `PORT` range check, `envBool` now parses with
`strconv.ParseBool`, and defaults apply only to unset variables (#80)
- 2026-08-07 Automatic event retention cleanup based on
`retention_days`, deleting expired events, deliveries, and delivery
results from each per-webhook event database (#63)
- 2026-08-07 Update golangci-lint to v2.12.2 (Docker image digest in
`Dockerfile`, release-archive sha256 pins in `script/bootstrap`),
adopt the canonical `.golangci.yml` (v2 `linters.settings` layout so
`lll`/`funlen`/`cyclop`/`dupl` thresholds actually apply), and fix
all newly surfaced lint findings
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
Makefile shims, README Entrypoints section
- 2026-03-25 pin golangci-lint Docker image for linting (#55)
- 2026-03-18 CSRF middleware detects TLS per-request, fixing login over
plain HTTP and behind reverse proxies (#54)
- 2026-03-17 root path redirects based on auth state (#52)
- 2026-03-17 CSRF protection, SSRF prevention for HTTP delivery targets
with DNS rebinding defense, and per-IP login rate limiting (#42)
- 2026-03-17 Slack target type for incoming webhook notifications (#47)
- 2026-03-17 Dockerfile absolute paths and static linking (#49);
absolute dev DATA_DIR default and clarified env docs (#46)
- 2026-03-05 security headers middleware, session regeneration on
login, request body size limits (#41)
- 2026-03-04 tests for delivery, middleware, and session packages
(#32); removed globals.Buildarch (#31)
- 2026-03-04 1.0 MVP merge: Webhook/Entrypoint/Target rename, core
delivery engine with bounded worker pool and circuit breaker,
parallel fan-out, per-webhook event databases, management UI (#16)
- 2026-03-01 repo brought to REPO_POLICIES standards; TODO.md folded
into README (#6)
# Future Steps
- Delivery status and retry management UI
- Per-webhook rate limiting in the receiver handler (per-webhook config
plus handler enforcement; global limits must not apply to receiver
endpoints)
- Webhook signature verification for GitHub and Stripe HMAC formats
- API key authentication for programmatic access (APIKey model exists;
Bearer token middleware does not)
- REST API v1
- CRUD for webhooks, entrypoints, targets
- event viewing and filtering endpoints
- event redelivery endpoint
- OpenAPI specification
- Analytics dashboard: success rates, response times, volume
- A remember-me option at login
- Password change and reset flow
- Later, nice to have
- email delivery target type
- SNS and S3 delivery targets
- data transformations (e.g. webhook to Slack message formatting)
- JSONL file delivery with periodic S3 upload
- webhook event search and filtering
- multi-user with role-based access control

View File

@@ -1,11 +1,11 @@
// Package main is the entry point for the webhooker application.
package main package main
import ( import (
"runtime"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/handlers" "sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/healthcheck" "sneak.berlin/go/webhooker/internal/healthcheck"
@@ -16,6 +16,8 @@ import (
) )
// Build-time variables set via -ldflags. // Build-time variables set via -ldflags.
//
//nolint:gochecknoglobals // Build-time variables injected by the linker.
var ( var (
version = "dev" version = "dev"
appname = "webhooker" appname = "webhooker"
@@ -24,7 +26,6 @@ var (
func main() { func main() {
globals.Appname = appname globals.Appname = appname
globals.Version = version globals.Version = version
globals.Buildarch = runtime.GOARCH
fx.New( fx.New(
fx.Provide( fx.Provide(
@@ -32,12 +33,32 @@ func main() {
logger.New, logger.New,
config.New, config.New,
database.New, database.New,
database.NewWebhookDBManager,
database.NewRetentionReaper,
healthcheck.New, healthcheck.New,
session.New, session.New,
handlers.New, handlers.New,
middleware.New, middleware.New,
delivery.New,
delivery.NewArchiveSweeper,
// Wire *delivery.Engine as delivery.Notifier so the
// webhook handler can notify the engine of new deliveries.
func(e *delivery.Engine) delivery.Notifier { return e },
// Wire *delivery.Engine as delivery.WebhookEvictor so
// deleting a webhook releases its archive writer.
func(e *delivery.Engine) delivery.WebhookEvictor {
return e
},
server.New, server.New,
), ),
fx.Invoke(func(*server.Server) {}), fx.Invoke(
func(
*server.Server,
*delivery.Engine,
*database.RetentionReaper,
*delivery.ArchiveSweeper,
) {
},
),
).Run() ).Run()
} }

View File

@@ -1,50 +0,0 @@
environments:
dev:
config:
port: 8080
debug: true
maintenanceMode: false
developmentMode: true
environment: dev
# Database URL for local development
dburl: postgres://webhooker:webhooker@localhost:5432/webhooker_dev?sslmode=disable
# Basic auth for metrics endpoint in dev
metricsUsername: admin
metricsPassword: admin
# Dev admin credentials for testing
devAdminUsername: devadmin
devAdminPassword: devpassword
secrets:
# Use default insecure session key for development
sessionKey: d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=
# Sentry DSN - usually not needed in dev
sentryDSN: ""
prod:
config:
port: $ENV:PORT
debug: $ENV:DEBUG
maintenanceMode: $ENV:MAINTENANCE_MODE
developmentMode: false
environment: prod
dburl: $ENV:DBURL
metricsUsername: $ENV:METRICS_USERNAME
metricsPassword: $ENV:METRICS_PASSWORD
# Dev admin credentials should not be set in production
devAdminUsername: ""
devAdminPassword: ""
secrets:
sessionKey: $ENV:SESSION_KEY
sentryDSN: $ENV:SENTRY_DSN
configDefaults:
# These defaults apply to all environments unless overridden
port: 8080
debug: false
maintenanceMode: false
developmentMode: false
environment: dev
metricsUsername: ""
metricsPassword: ""
devAdminUsername: ""
devAdminPassword: ""

33
go.mod
View File

@@ -1,49 +1,37 @@
module sneak.berlin/go/webhooker module sneak.berlin/go/webhooker
go 1.23.0 go 1.26.1
toolchain go1.24.1
require ( require (
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8 github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
github.com/getsentry/sentry-go v0.25.0 github.com/getsentry/sentry-go v0.25.0
github.com/go-chi/chi v1.5.5 github.com/go-chi/chi v1.5.5
github.com/go-chi/cors v1.2.1 github.com/go-chi/cors v1.2.1
github.com/go-chi/httprate v0.15.0
github.com/google/uuid v1.6.0 github.com/google/uuid v1.6.0
github.com/gorilla/csrf v1.7.3
github.com/gorilla/sessions v1.4.0 github.com/gorilla/sessions v1.4.0
github.com/joho/godotenv v1.5.1 github.com/joho/godotenv v1.5.1
github.com/prometheus/client_golang v1.18.0 github.com/prometheus/client_golang v1.18.0
github.com/slok/go-http-metrics v0.11.0 github.com/slok/go-http-metrics v0.11.0
github.com/spf13/afero v1.14.0
github.com/stretchr/testify v1.8.4 github.com/stretchr/testify v1.8.4
go.uber.org/fx v1.20.1 go.uber.org/fx v1.20.1
golang.org/x/crypto v0.38.0 golang.org/x/crypto v0.38.0
gorm.io/driver/sqlite v1.5.4 gorm.io/driver/sqlite v1.5.4
gorm.io/gorm v1.25.5 gorm.io/gorm v1.25.5
modernc.org/sqlite v1.28.0 modernc.org/sqlite v1.28.0
sneak.berlin/go/webhooker/pkg/config v0.0.0-00010101000000-000000000000
) )
require ( require (
cloud.google.com/go/compute v1.23.3 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/iam v1.1.5 // indirect
cloud.google.com/go/secretmanager v1.11.4 // indirect
github.com/aws/aws-sdk-go v1.50.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dustin/go-humanize v1.0.1 // indirect github.com/dustin/go-humanize v1.0.1 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/gorilla/securecookie v1.1.2 // indirect github.com/gorilla/securecookie v1.1.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect github.com/jinzhu/now v1.1.5 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/cpuid/v2 v2.2.10 // indirect
github.com/kr/text v0.2.0 // indirect github.com/kr/text v0.2.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-sqlite3 v1.14.17 // indirect github.com/mattn/go-sqlite3 v1.14.17 // indirect
@@ -53,25 +41,16 @@ require (
github.com/prometheus/common v0.45.0 // indirect github.com/prometheus/common v0.45.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect github.com/prometheus/procfs v0.12.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
go.opencensus.io v0.24.0 // indirect github.com/zeebo/xxh3 v1.0.2 // indirect
go.uber.org/atomic v1.9.0 // indirect go.uber.org/atomic v1.9.0 // indirect
go.uber.org/dig v1.17.0 // indirect go.uber.org/dig v1.17.0 // indirect
go.uber.org/multierr v1.9.0 // indirect go.uber.org/multierr v1.9.0 // indirect
go.uber.org/zap v1.23.0 // indirect go.uber.org/zap v1.23.0 // indirect
golang.org/x/mod v0.17.0 // indirect golang.org/x/mod v0.17.0 // indirect
golang.org/x/net v0.25.0 // indirect
golang.org/x/oauth2 v0.15.0 // indirect
golang.org/x/sync v0.14.0 // indirect golang.org/x/sync v0.14.0 // indirect
golang.org/x/sys v0.33.0 // indirect golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.25.0 // indirect golang.org/x/text v0.25.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
google.golang.org/api v0.153.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20231106174013-bbf56f31fb17 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20231106174013-bbf56f31fb17 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f // indirect
google.golang.org/grpc v1.59.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/uint128 v1.2.0 // indirect lukechampine.com/uint128 v1.2.0 // indirect
@@ -84,5 +63,3 @@ require (
modernc.org/strutil v1.1.3 // indirect modernc.org/strutil v1.1.3 // indirect
modernc.org/token v1.0.1 // indirect modernc.org/token v1.0.1 // indirect
) )
replace sneak.berlin/go/webhooker/pkg/config => ./pkg/config

148
go.sum
View File

@@ -1,28 +1,11 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.110.10 h1:LXy9GEO+timppncPIAZoOj3l58LIU9k+kn48AN7IO3Y=
cloud.google.com/go v0.110.10/go.mod h1:v1OoFqYxiBkUrruItNM3eT4lLByNjxmJSV/xDKJNnic=
cloud.google.com/go/compute v1.23.3 h1:6sVlXXBmbd7jNX0Ipq0trII3e4n1/MsADLK6a+aiVlk=
cloud.google.com/go/compute v1.23.3/go.mod h1:VCgBUoMnIVIR0CscqQiPJLAG25E3ZRZMzcFZeQ+h8CI=
cloud.google.com/go/compute/metadata v0.2.3 h1:mg4jlk7mCAj6xXp9UJ4fjI9VUI5rubuGBW5aJ7UnBMY=
cloud.google.com/go/compute/metadata v0.2.3/go.mod h1:VAV5nSsACxMJvgaAuX6Pk2AawlZn8kiOGuCv6gTkwuA=
cloud.google.com/go/iam v1.1.5 h1:1jTsCu4bcsNsE4iiqNT5SHwrDRCfRmIaaaVFhRveTJI=
cloud.google.com/go/iam v1.1.5/go.mod h1:rB6P/Ic3mykPbFio+vo7403drjlgvoWfYpJhMXEbzv8=
cloud.google.com/go/secretmanager v1.11.4 h1:krnX9qpG2kR2fJ+u+uNyNo+ACVhplIAS4Pu7u+4gd+k=
cloud.google.com/go/secretmanager v1.11.4/go.mod h1:wreJlbS9Zdq21lMzWmJ0XhWW2ZxgPeahsqeV/vZoJ3w=
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8 h1:nMpu1t4amK3vJWBibQ5X/Nv0aXL+b69TQf2uK5PH7Go= github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8 h1:nMpu1t4amK3vJWBibQ5X/Nv0aXL+b69TQf2uK5PH7Go=
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8/go.mod h1:3cARGAK9CfW3HoxCy1a0G4TKrdiKke8ftOMEOHyySYs= github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8/go.mod h1:3cARGAK9CfW3HoxCy1a0G4TKrdiKke8ftOMEOHyySYs=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/aws/aws-sdk-go v1.50.0 h1:HBtrLeO+QyDKnc3t1+5DR1RxodOHCGr8ZcrHudpv7jI=
github.com/aws/aws-sdk-go v1.50.0/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk=
github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A= github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A=
github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -30,42 +13,17 @@ github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/getsentry/sentry-go v0.25.0 h1:q6Eo+hS+yoJlTO3uu/azhQadsD8V+jQn2D8VvX1eOyI= github.com/getsentry/sentry-go v0.25.0 h1:q6Eo+hS+yoJlTO3uu/azhQadsD8V+jQn2D8VvX1eOyI=
github.com/getsentry/sentry-go v0.25.0/go.mod h1:lc76E2QywIyW8WuBnwl8Lc4bkmQH4+w1gwTf25trprY= github.com/getsentry/sentry-go v0.25.0/go.mod h1:lc76E2QywIyW8WuBnwl8Lc4bkmQH4+w1gwTf25trprY=
github.com/go-chi/chi v1.5.5 h1:vOB/HbEMt9QqBqErz07QehcOKHaWFtuj87tTDVz2qXE= github.com/go-chi/chi v1.5.5 h1:vOB/HbEMt9QqBqErz07QehcOKHaWFtuj87tTDVz2qXE=
github.com/go-chi/chi v1.5.5/go.mod h1:C9JqLr3tIYjDOZpzn+BCuxY8z8vmca43EeMgyZt7irw= github.com/go-chi/chi v1.5.5/go.mod h1:C9JqLr3tIYjDOZpzn+BCuxY8z8vmca43EeMgyZt7irw=
github.com/go-chi/cors v1.2.1 h1:xEC8UT3Rlp2QuWNEr4Fs/c2EAGVKBwy/1vHx3bppil4= github.com/go-chi/cors v1.2.1 h1:xEC8UT3Rlp2QuWNEr4Fs/c2EAGVKBwy/1vHx3bppil4=
github.com/go-chi/cors v1.2.1/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58= github.com/go-chi/cors v1.2.1/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-chi/httprate v0.15.0 h1:j54xcWV9KGmPf/X4H32/aTH+wBlrvxL7P+SdnRqxh5g=
github.com/go-chi/httprate v0.15.0/go.mod h1:rzGHhVrsBn3IMLYDOZQsSU4fJNWcjui4fWKJcCId1R4=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA= github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og= github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
@@ -73,15 +31,10 @@ github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ= github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo= github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo=
github.com/google/s2a-go v0.1.7 h1:60BLSyTrOV4/haCDW4zb1guZItoSq8foHCXrAnjBo/o=
github.com/google/s2a-go v0.1.7/go.mod h1:50CgR4k1jNlWBu4UfS4AcfhVe1r6pdZPygJ3R8F0Qdw=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.2 h1:Vie5ybvEvT75RniqhfFxPRy3Bf7vr3h0cechB90XaQs= github.com/gorilla/csrf v1.7.3 h1:BHWt6FTLZAb2HtWT5KDBf6qgpZzvtbp9QWDRKZMXJC0=
github.com/googleapis/enterprise-certificate-proxy v0.3.2/go.mod h1:VLSiSSBs/ksPL8kq3OBOQ6WRI2QnaFynd1DCjZ62+V0= github.com/gorilla/csrf v1.7.3/go.mod h1:F1Fj3KG23WYHE6gozCmBAezKookxbIvUJT+121wTuLk=
github.com/googleapis/gax-go/v2 v2.12.0 h1:A+gCJKdRfqXkr+BIRGtZLibNXf0m1f9E4HG56etFpas=
github.com/googleapis/gax-go/v2 v2.12.0/go.mod h1:y+aIqrI5eb1YGMVJfuV3185Ts/D7qKpsEkdD5+I6QGU=
github.com/gorilla/securecookie v1.1.2 h1:YCIWL56dvtr73r6715mJs5ZvhtnY73hBvEF8kXD8ePA= github.com/gorilla/securecookie v1.1.2 h1:YCIWL56dvtr73r6715mJs5ZvhtnY73hBvEF8kXD8ePA=
github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pwzwo4h3eOamfo= github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pwzwo4h3eOamfo=
github.com/gorilla/sessions v1.4.0 h1:kpIYOp/oi6MG/p5PgxApU8srsSw9tuFbt46Lt7auzqQ= github.com/gorilla/sessions v1.4.0 h1:kpIYOp/oi6MG/p5PgxApU8srsSw9tuFbt46Lt7auzqQ=
@@ -90,14 +43,12 @@ github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc= github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ= github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0= github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4= github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs= github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8= github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
@@ -117,7 +68,6 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.18.0 h1:HzFfmkOzH5Q8L8G+kSJKUx5dtG87sewO+FoDDqP5Tbk= github.com/prometheus/client_golang v1.18.0 h1:HzFfmkOzH5Q8L8G+kSJKUx5dtG87sewO+FoDDqP5Tbk=
github.com/prometheus/client_golang v1.18.0/go.mod h1:T+GXkCk5wSJyOqMIzVgvvjFDlkOQntgjkJWKrN5txjA= github.com/prometheus/client_golang v1.18.0/go.mod h1:T+GXkCk5wSJyOqMIzVgvvjFDlkOQntgjkJWKrN5txjA=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw= github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw=
github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI= github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI=
github.com/prometheus/common v0.45.0 h1:2BGz0eBc2hdMDLnO/8n0jeB3oPrt2D08CekT0lneoxM= github.com/prometheus/common v0.45.0 h1:2BGz0eBc2hdMDLnO/8n0jeB3oPrt2D08CekT0lneoxM=
@@ -130,21 +80,16 @@ github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjR
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/slok/go-http-metrics v0.11.0 h1:ABJUpekCZSkQT1wQrFvS4kGbhea/w6ndFJaWJeh3zL0= github.com/slok/go-http-metrics v0.11.0 h1:ABJUpekCZSkQT1wQrFvS4kGbhea/w6ndFJaWJeh3zL0=
github.com/slok/go-http-metrics v0.11.0/go.mod h1:ZGKeYG1ET6TEJpQx18BqAJAvxw9jBAZXCHU7bWQqqAc= github.com/slok/go-http-metrics v0.11.0/go.mod h1:ZGKeYG1ET6TEJpQx18BqAJAvxw9jBAZXCHU7bWQqqAc=
github.com/spf13/afero v1.14.0 h1:9tH6MapGnn/j0eb0yIXiLjERO8RB6xIVZRDCX7PtqWA=
github.com/spf13/afero v1.14.0/go.mod h1:acJQ8t0ohCGuMN3O+Pv0V0hgMxNYDlvdk+VTfyZmbYo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.1 h1:4VhoImhV/Bm0ToFkXFi8hXNXwpDRZ/ynw3amt82mzq0= github.com/stretchr/objx v0.5.1 h1:4VhoImhV/Bm0ToFkXFi8hXNXwpDRZ/ynw3amt82mzq0=
github.com/stretchr/objx v0.5.1/go.mod h1:/iHQpkQwBD6DLUmQ4pE+s1TXdob1mORJ4/UFdrifcy0= github.com/stretchr/objx v0.5.1/go.mod h1:/iHQpkQwBD6DLUmQ4pE+s1TXdob1mORJ4/UFdrifcy0=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/dig v1.17.0 h1:5Chju+tUvcC+N7N6EV08BJz41UZuO3BmHcN4A287ZLI= go.uber.org/dig v1.17.0 h1:5Chju+tUvcC+N7N6EV08BJz41UZuO3BmHcN4A287ZLI=
@@ -157,105 +102,32 @@ go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ= go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
go.uber.org/zap v1.23.0 h1:OjGQ5KQDEUawVHxNwQgPpiypGHOxo2mNZsOqTak4fFY= go.uber.org/zap v1.23.0 h1:OjGQ5KQDEUawVHxNwQgPpiypGHOxo2mNZsOqTak4fFY=
go.uber.org/zap v1.23.0/go.mod h1:D+nX8jyLsMHMYrln8A0rJjFt/T/9/bGgIhAqxv5URuY= go.uber.org/zap v1.23.0/go.mod h1:D+nX8jyLsMHMYrln8A0rJjFt/T/9/bGgIhAqxv5URuY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8= golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8=
golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw= golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA= golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.15.0 h1:s8pnnxNVzjWyrvYdFUQq5llS1PX2zhPXmccZv99h7uQ=
golang.org/x/oauth2 v0.15.0/go.mod h1:q48ptWNTY5XWf+JNten23lcvHpLJ0ZSxF5ttTHKVCAM=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ= golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ=
golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw= golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4= golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA= golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg= golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.153.0 h1:N1AwGhielyKFaUqH07/ZSIQR3uNPcV7NVw0vj+j4iR4=
google.golang.org/api v0.153.0/go.mod h1:3qNJX5eOmhiWYc67jRA/3GsDw97UFb5ivv7Y2PrriAY=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20231106174013-bbf56f31fb17 h1:wpZ8pe2x1Q3f2KyT5f8oP/fa9rHAKgFPr/HZdNuS+PQ=
google.golang.org/genproto v0.0.0-20231106174013-bbf56f31fb17/go.mod h1:J7XzRzVy1+IPwWHZUzoD0IccYZIrXILAQpc+Qy9CMhY=
google.golang.org/genproto/googleapis/api v0.0.0-20231106174013-bbf56f31fb17 h1:JpwMPBpFN3uKhdaekDpiNlImDdkUAyiJ6ez/uxGaUSo=
google.golang.org/genproto/googleapis/api v0.0.0-20231106174013-bbf56f31fb17/go.mod h1:0xJLfVdJqpAPl8tDg1ujOCGzx6LFLttXT5NhllGOXY4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f h1:ultW7fxlIvee4HYrtnaRPon9HpEgFk5zYpmfMgtKB5I=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f/go.mod h1:L9KNLi232K1/xB6f7AlSX692koaRnKaWSR0stBki0Yc=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.59.0 h1:Z5Iec2pjwb+LEOqzpB2MR12/eKFhDPhuqW91O+4bwUk=
google.golang.org/grpc v1.59.0/go.mod h1:aUPDwccQo6OTjy7Hct4AfBPD1GptF4fyUjIkQ9YtF98=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gorm.io/driver/sqlite v1.5.4 h1:IqXwXi8M/ZlPzH/947tn5uik3aYQslP9BVveoax0nV0= gorm.io/driver/sqlite v1.5.4 h1:IqXwXi8M/ZlPzH/947tn5uik3aYQslP9BVveoax0nV0=
gorm.io/driver/sqlite v1.5.4/go.mod h1:qxAuCol+2r6PannQDpOP1FP6ag3mKi4esLnB/jHed+4= gorm.io/driver/sqlite v1.5.4/go.mod h1:qxAuCol+2r6PannQDpOP1FP6ag3mKi4esLnB/jHed+4=
gorm.io/gorm v1.25.5 h1:zR9lOiiYf09VNh5Q1gphfyia1JpiClIWG9hQaxB/mls= gorm.io/gorm v1.25.5 h1:zR9lOiiYf09VNh5Q1gphfyia1JpiClIWG9hQaxB/mls=
gorm.io/gorm v1.25.5/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8= gorm.io/gorm v1.25.5/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI= lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI=
lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk= lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk=
modernc.org/cc/v3 v3.40.0 h1:P3g79IUS/93SYhtoeaHW+kRCIrYaxJ27MFPv+7kaTOw= modernc.org/cc/v3 v3.40.0 h1:P3g79IUS/93SYhtoeaHW+kRCIrYaxJ27MFPv+7kaTOw=

View File

@@ -1,16 +1,19 @@
// Package config loads application configuration from environment variables.
package config package config
import ( import (
"errors"
"fmt" "fmt"
"log/slog" "log/slog"
"net/netip"
"os" "os"
"strconv" "strconv"
"strings" "strings"
"time"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
pkgconfig "sneak.berlin/go/webhooker/pkg/config"
// Populates the environment from a ./.env file automatically for // Populates the environment from a ./.env file automatically for
// development configuration. Kept in one place only (here). // development configuration. Kept in one place only (here).
@@ -18,133 +21,399 @@ import (
) )
const ( const (
// EnvironmentDev represents development environment // EnvironmentDev represents development environment.
EnvironmentDev = "dev" EnvironmentDev = "dev"
// EnvironmentProd represents production environment // EnvironmentProd represents production environment.
EnvironmentProd = "prod" EnvironmentProd = "prod"
// DevSessionKey is an insecure default session key for development
// This is "webhooker-dev-session-key-insecure!" base64 encoded // defaultPort is the default HTTP listen port.
DevSessionKey = "d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=" defaultPort = 8080
// defaultRetentionSweepInterval is how often the retention
// reaper deletes events older than each webhook's RetentionDays.
defaultRetentionSweepInterval = time.Hour
// defaultSessionIdleTimeout is how long a session may go without
// authenticated activity before it expires.
defaultSessionIdleTimeout = 24 * time.Hour
// defaultReceiverRateLimit is the default number of requests
// per minute each client IP may send to a single webhook
// receiver entrypoint. Generous for legitimate webhook
// senders while bounding abuse of the one unauthenticated,
// internet-exposed endpoint.
defaultReceiverRateLimit = 120
// maxPort is the highest valid TCP port number. The lower
// bound (at least 1) is enforced by envPositiveInt.
maxPort = 65535
// mappedV4Offset is the number of leading bits an IPv4-mapped
// IPv6 prefix spends on the ::ffff:0:0/96 wrapper, so a /104
// covers the same addresses as an IPv4 /8.
mappedV4Offset = 96
) )
// nolint:revive // ConfigParams is a standard fx naming convention // ErrInvalidEnvironment is returned when WEBHOOKER_ENVIRONMENT
// contains an unrecognised value.
var ErrInvalidEnvironment = errors.New("invalid environment")
// ErrNonPositiveValue is returned when an environment variable that
// requires a positive integer is set to zero or a negative number.
var ErrNonPositiveValue = errors.New("value must be positive")
// ErrInvalidPort is returned when an environment variable holding a
// TCP port number is set above the valid port range.
var ErrInvalidPort = errors.New("invalid port")
// ErrInvalidCIDR is returned when an environment variable holding a
// list of CIDR blocks contains an entry that is neither a CIDR block
// nor a bare IP address.
var ErrInvalidCIDR = errors.New("invalid CIDR")
//nolint:revive // ConfigParams is a standard fx naming convention.
type ConfigParams struct { type ConfigParams struct {
fx.In fx.In
Globals *globals.Globals Globals *globals.Globals
Logger *logger.Logger Logger *logger.Logger
} }
// Config holds all application configuration loaded from
// environment variables.
type Config struct { type Config struct {
DBURL string DataDir string
Debug bool Debug bool
MaintenanceMode bool MaintenanceMode bool
DevelopmentMode bool
DevAdminUsername string
DevAdminPassword string
Environment string Environment string
MetricsPassword string MetricsPassword string
MetricsUsername string MetricsUsername string
Port int Port int
SentryDSN string SentryDSN string
SessionKey string
// RetentionSweepInterval is how often the retention reaper runs.
RetentionSweepInterval time.Duration
// SessionIdleTimeout is the sliding inactivity window after
// which a session expires. Non-positive disables idle expiry.
SessionIdleTimeout time.Duration
// ReceiverRateLimit is the number of requests per minute each
// client IP may send to a single webhook receiver entrypoint.
ReceiverRateLimit int
// TrustedProxies is the set of networks whose members are
// allowed to speak for the client with X-Forwarded-For, the
// only forwarded header read. It is empty unless
// TRUSTED_PROXIES is set, and empty means no peer is
// trusted: forwarded headers are then ignored entirely and
// clients are identified by the connection's own address.
// Members can choose their own rate-limit key, so this must
// name proxy hosts only, never a block that also covers
// clients.
TrustedProxies []netip.Prefix
params *ConfigParams params *ConfigParams
log *slog.Logger log *slog.Logger
} }
// IsDev returns true if running in development environment // IsDev returns true if running in development environment.
func (c *Config) IsDev() bool { func (c *Config) IsDev() bool {
return c.Environment == EnvironmentDev return c.Environment == EnvironmentDev
} }
// IsProd returns true if running in production environment // IsProd returns true if running in production environment.
func (c *Config) IsProd() bool { func (c *Config) IsProd() bool {
return c.Environment == EnvironmentProd return c.Environment == EnvironmentProd
} }
// envString returns the env var value if set, otherwise falls back to pkgconfig. // envString returns the value of the named environment variable,
func envString(envKey, configKey string) string { // or an empty string if not set.
if v := os.Getenv(envKey); v != "" { func envString(key string) string {
return v return os.Getenv(key)
}
return pkgconfig.GetString(configKey)
} }
// envSecretString returns the env var value if set, otherwise falls back to pkgconfig secrets. // envBool returns the value of the named environment variable
func envSecretString(envKey, configKey string) string { // parsed as a boolean. Returns defaultValue if not set. If the
if v := os.Getenv(envKey); v != "" { // variable is set but cannot be parsed, it returns a wrapped error
return v // naming the key and the bad value, so startup fails loudly rather
// than silently falling back to the default.
//
// Parsing is strconv.ParseBool, which accepts 1, t, T, TRUE, true,
// True, 0, f, F, FALSE, false and False. Anything else — "yes",
// "on", or a typo like "ture" — is an error rather than a silent
// false.
func envBool(key string, defaultValue bool) (bool, error) {
v := os.Getenv(key)
if v == "" {
return defaultValue, nil
} }
return pkgconfig.GetSecretString(configKey)
b, err := strconv.ParseBool(v)
if err != nil {
return false, fmt.Errorf(
"invalid boolean for %s: %q: %w", key, v, err,
)
}
return b, nil
} }
// envBool returns the env var value parsed as bool, otherwise falls back to pkgconfig. // envPositiveInt returns the value of the named environment variable
func envBool(envKey, configKey string) bool { // parsed as a positive integer. Returns defaultValue if not set. If
if v := os.Getenv(envKey); v != "" { // the variable is set but cannot be parsed, or parses to less than
return strings.EqualFold(v, "true") || v == "1" // one, it returns a wrapped error naming the key and the bad value,
// so startup fails loudly rather than silently falling back to the
// default.
func envPositiveInt(
key string,
defaultValue int,
) (int, error) {
v := os.Getenv(key)
if v == "" {
return defaultValue, nil
} }
return pkgconfig.GetBool(configKey)
i, err := strconv.Atoi(v)
if err != nil {
return 0, fmt.Errorf(
"invalid integer for %s: %q: %w", key, v, err,
)
}
if i < 1 {
return 0, fmt.Errorf(
"%w: %s must be at least 1, got %q",
ErrNonPositiveValue, key, v,
)
}
return i, nil
} }
// envInt returns the env var value parsed as int, otherwise falls back to pkgconfig. // envPort returns the value of the named environment variable parsed
func envInt(envKey, configKey string, defaultValue ...int) int { // as a TCP port number. Returns defaultValue if not set. A set value
if v := os.Getenv(envKey); v != "" { // that is unparseable, below 1, or above maxPort is a hard error
if i, err := strconv.Atoi(v); err == nil { // naming the key and the bad value.
return i func envPort(key string, defaultValue int) (int, error) {
port, err := envPositiveInt(key, defaultValue)
if err != nil {
return 0, err
} }
if port > maxPort {
return 0, fmt.Errorf(
"%w: %s must be at most %d, got %d",
ErrInvalidPort, key, maxPort, port,
)
} }
return pkgconfig.GetInt(configKey, defaultValue...)
return port, nil
} }
// nolint:revive // lc parameter is required by fx even if unused // envDuration returns the value of the named environment variable
func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) { // parsed as a Go duration (e.g. "1h", "30m"). Returns defaultValue if
log := params.Logger.Get() // not set. If the variable is set but cannot be parsed, it returns a
// wrapped error naming the key and the bad value, so startup fails
// loudly rather than silently falling back to the default.
func envDuration(
key string,
defaultValue time.Duration,
) (time.Duration, error) {
v := os.Getenv(key)
if v == "" {
return defaultValue, nil
}
// Determine environment from WEBHOOKER_ENVIRONMENT env var, default to dev d, err := time.ParseDuration(v)
if err != nil {
return 0, fmt.Errorf(
"invalid duration for %s: %q: %w", key, v, err,
)
}
return d, nil
}
// parseCIDR parses one trusted-proxy list entry, which may be a
// CIDR block ("10.0.0.0/8") or a bare address ("10.0.0.1", treated
// as a single-host block).
//
// Both forms are unmapped, because peer addresses are unmapped
// before they are matched against the list: an IPv4-mapped prefix
// left in that form would silently never match.
func parseCIDR(entry string) (netip.Prefix, error) {
if strings.Contains(entry, "/") {
prefix, err := netip.ParsePrefix(entry)
if err != nil {
return netip.Prefix{}, err //nolint:wrapcheck // wrapped by caller
}
if addr := prefix.Addr(); addr.Is4In6() &&
prefix.Bits() >= mappedV4Offset {
prefix = netip.PrefixFrom(
addr.Unmap(), prefix.Bits()-mappedV4Offset,
)
}
return prefix.Masked(), nil
}
addr, err := netip.ParseAddr(entry)
if err != nil {
return netip.Prefix{}, err //nolint:wrapcheck // wrapped by caller
}
return netip.PrefixFrom(addr.Unmap(), addr.Unmap().BitLen()), nil
}
// envPrefixList returns the value of the named environment variable
// parsed as a comma-separated list of CIDR blocks (bare addresses
// allowed). An unset, empty, or blank value yields an empty list. A
// set value containing an unparseable entry is a hard error naming
// the key and the bad entry, so startup fails loudly rather than
// silently running with a list the operator did not intend.
func envPrefixList(key string) ([]netip.Prefix, error) {
v := strings.TrimSpace(os.Getenv(key))
if v == "" {
return nil, nil
}
var prefixes []netip.Prefix
for entry := range strings.SplitSeq(v, ",") {
entry = strings.TrimSpace(entry)
if entry == "" {
continue
}
prefix, err := parseCIDR(entry)
if err != nil {
return nil, fmt.Errorf(
"%w: %s: %q: %w", ErrInvalidCIDR, key, entry, err,
)
}
prefixes = append(prefixes, prefix)
}
return prefixes, nil
}
// resolveEnvironment reads WEBHOOKER_ENVIRONMENT, defaulting to
// dev, and rejects unrecognised values.
func resolveEnvironment() (string, error) {
environment := os.Getenv("WEBHOOKER_ENVIRONMENT") environment := os.Getenv("WEBHOOKER_ENVIRONMENT")
if environment == "" { if environment == "" {
environment = EnvironmentDev environment = EnvironmentDev
} }
// Validate environment if environment != EnvironmentDev &&
if environment != EnvironmentDev && environment != EnvironmentProd { environment != EnvironmentProd {
return nil, fmt.Errorf("WEBHOOKER_ENVIRONMENT must be either '%s' or '%s', got '%s'", return "", fmt.Errorf(
EnvironmentDev, EnvironmentProd, environment) "%w: WEBHOOKER_ENVIRONMENT must be '%s' or '%s', got '%s'",
ErrInvalidEnvironment,
EnvironmentDev, EnvironmentProd, environment,
)
} }
// Set the environment in the config package (for fallback resolution) return environment, nil
pkgconfig.SetEnvironment(environment) }
// Load configuration values — env vars take precedence over config.yaml // loadFromEnv builds a Config from the environment. Every value that
s := &Config{ // needs parsing fails loudly when it is set but unparseable: the
DBURL: envString("DBURL", "dburl"), // documented defaults apply only to variables that are unset (or
Debug: envBool("DEBUG", "debug"), // empty), never as a substitute for a value the operator actually
MaintenanceMode: envBool("MAINTENANCE_MODE", "maintenanceMode"), // provided.
DevelopmentMode: envBool("DEVELOPMENT_MODE", "developmentMode"), func loadFromEnv() (*Config, error) {
DevAdminUsername: envString("DEV_ADMIN_USERNAME", "devAdminUsername"), environment, err := resolveEnvironment()
DevAdminPassword: envString("DEV_ADMIN_PASSWORD", "devAdminPassword"), if err != nil {
return nil, err
}
port, err := envPort("PORT", defaultPort)
if err != nil {
return nil, err
}
debug, err := envBool("DEBUG", false)
if err != nil {
return nil, err
}
maintenanceMode, err := envBool("MAINTENANCE_MODE", false)
if err != nil {
return nil, err
}
retentionSweepInterval, err := envDuration(
"RETENTION_SWEEP_INTERVAL",
defaultRetentionSweepInterval,
)
if err != nil {
return nil, err
}
sessionIdleTimeout, err := envDuration(
"SESSION_IDLE_TIMEOUT",
defaultSessionIdleTimeout,
)
if err != nil {
return nil, err
}
receiverRateLimit, err := envPositiveInt(
"RECEIVER_RATE_LIMIT",
defaultReceiverRateLimit,
)
if err != nil {
return nil, err
}
trustedProxies, err := envPrefixList("TRUSTED_PROXIES")
if err != nil {
return nil, err
}
return &Config{
DataDir: envString("DATA_DIR"),
Debug: debug,
MaintenanceMode: maintenanceMode,
Environment: environment, Environment: environment,
MetricsUsername: envString("METRICS_USERNAME", "metricsUsername"), MetricsUsername: envString("METRICS_USERNAME"),
MetricsPassword: envString("METRICS_PASSWORD", "metricsPassword"), MetricsPassword: envString("METRICS_PASSWORD"),
Port: envInt("PORT", "port", 8080), Port: port,
SentryDSN: envSecretString("SENTRY_DSN", "sentryDSN"), SentryDSN: envString("SENTRY_DSN"),
SessionKey: envSecretString("SESSION_KEY", "sessionKey"), RetentionSweepInterval: retentionSweepInterval,
log: log, SessionIdleTimeout: sessionIdleTimeout,
params: &params, ReceiverRateLimit: receiverRateLimit,
TrustedProxies: trustedProxies,
}, nil
}
// New creates a Config by reading environment variables.
//
//nolint:revive // lc parameter is required by fx even if unused.
func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) {
log := params.Logger.Get()
// A set-but-unparseable value anywhere in the environment is a
// hard error, so fx aborts startup rather than running with a
// silently substituted default.
s, err := loadFromEnv()
if err != nil {
return nil, err
} }
// Validate database URL s.log = log
if s.DBURL == "" { s.params = &params
return nil, fmt.Errorf("database URL (DBURL) is required")
}
// In production, require session key // Set default DataDir. All SQLite databases (main application
if s.IsProd() && s.SessionKey == "" { // DB and per-webhook event DBs) live here. The same default is
return nil, fmt.Errorf("SESSION_KEY is required in production environment") // used regardless of environment; override with DATA_DIR if
} // needed.
if s.DataDir == "" {
// In development mode, warn if using default session key s.DataDir = "/var/lib/webhooker"
if s.IsDev() && s.SessionKey == DevSessionKey {
log.Warn("Using insecure default session key for development mode")
} }
if s.Debug { if s.Debug {
@@ -157,10 +426,13 @@ func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) {
"port", s.Port, "port", s.Port,
"debug", s.Debug, "debug", s.Debug,
"maintenanceMode", s.MaintenanceMode, "maintenanceMode", s.MaintenanceMode,
"developmentMode", s.DevelopmentMode, "dataDir", s.DataDir,
"hasSessionKey", s.SessionKey != "", "retentionSweepInterval", s.RetentionSweepInterval.String(),
"receiverRateLimit", s.ReceiverRateLimit,
"trustedProxies", len(s.TrustedProxies),
"hasSentryDSN", s.SentryDSN != "", "hasSentryDSN", s.SentryDSN != "",
"hasMetricsAuth", s.MetricsUsername != "" && s.MetricsPassword != "", "hasMetricsAuth",
s.MetricsUsername != "" && s.MetricsPassword != "",
) )
return s, nil return s, nil

View File

@@ -1,68 +1,30 @@
package config package config_test
import ( import (
"os" "os"
"testing" "testing"
"time"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx" "go.uber.org/fx"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
pkgconfig "sneak.berlin/go/webhooker/pkg/config"
) )
// createTestConfig creates a test configuration file in memory // Shared subtest names for the env-parsing tables below, which all
func createTestConfig(fs afero.Fs) error { // exercise the same three cases against different variables.
configYAML := ` const (
environments: caseUnsetUsesDefault = "unset uses default"
dev: caseValidValueParsed = "valid value is parsed"
config: caseUnparseableFails = "unparseable value fails startup"
port: 8080
debug: true
maintenanceMode: false
developmentMode: true
environment: dev
dburl: postgres://test:test@localhost:5432/test_dev?sslmode=disable
metricsUsername: testuser
metricsPassword: testpass
devAdminUsername: devadmin
devAdminPassword: devpass
secrets:
sessionKey: d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=
sentryDSN: ""
prod: // cidrPrivateV4 is the sample trusted-proxy block the
config: // TRUSTED_PROXIES cases are built from.
port: $ENV:PORT cidrPrivateV4 = "10.0.0.0/8"
debug: $ENV:DEBUG )
maintenanceMode: $ENV:MAINTENANCE_MODE
developmentMode: false
environment: prod
dburl: $ENV:DBURL
metricsUsername: $ENV:METRICS_USERNAME
metricsPassword: $ENV:METRICS_PASSWORD
devAdminUsername: ""
devAdminPassword: ""
secrets:
sessionKey: $ENV:SESSION_KEY
sentryDSN: $ENV:SENTRY_DSN
configDefaults:
port: 8080
debug: false
maintenanceMode: false
developmentMode: false
environment: dev
metricsUsername: ""
metricsPassword: ""
devAdminUsername: ""
devAdminPassword: ""
`
return afero.WriteFile(fs, "config.yaml", []byte(configYAML), 0644)
}
func TestEnvironmentConfig(t *testing.T) { func TestEnvironmentConfig(t *testing.T) {
tests := []struct { tests := []struct {
@@ -75,26 +37,18 @@ func TestEnvironmentConfig(t *testing.T) {
}{ }{
{ {
name: "default is dev", name: "default is dev",
envValue: "",
expectError: false,
isDev: true, isDev: true,
isProd: false, isProd: false,
}, },
{ {
name: "explicit dev", name: "explicit dev",
envValue: "dev", envValue: "dev",
expectError: false,
isDev: true, isDev: true,
isProd: false, isProd: false,
}, },
{ {
name: "explicit prod with session key", name: "explicit prod",
envValue: "prod", envValue: "prod",
envVars: map[string]string{
"SESSION_KEY": "cHJvZC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
"DBURL": "postgres://prod:prod@localhost:5432/prod?sslmode=require",
},
expectError: false,
isDev: false, isDev: false,
isProd: true, isProd: true,
}, },
@@ -107,194 +61,526 @@ func TestEnvironmentConfig(t *testing.T) {
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Create in-memory filesystem with test config // Cannot use t.Parallel() here because t.Setenv
fs := afero.NewMemMapFs() // is incompatible with parallel subtests.
require.NoError(t, createTestConfig(fs))
pkgconfig.SetFs(fs)
// Set environment variable if specified
if tt.envValue != "" { if tt.envValue != "" {
os.Setenv("WEBHOOKER_ENVIRONMENT", tt.envValue) t.Setenv(
defer os.Unsetenv("WEBHOOKER_ENVIRONMENT") "WEBHOOKER_ENVIRONMENT", tt.envValue,
)
} else {
require.NoError(t, os.Unsetenv(
"WEBHOOKER_ENVIRONMENT",
))
} }
// Set additional environment variables
for k, v := range tt.envVars { for k, v := range tt.envVars {
os.Setenv(k, v) t.Setenv(k, v)
defer os.Unsetenv(k)
} }
if tt.expectError { if tt.expectError {
// Use regular fx.New for error cases since fxtest doesn't expose errors the same way testEnvironmentConfigError(t)
var cfg *Config
app := fx.New(
fx.NopLogger, // Suppress fx logs in tests
fx.Provide(
globals.New,
logger.New,
New,
),
fx.Populate(&cfg),
)
assert.Error(t, app.Err())
} else { } else {
// Use fxtest for success cases testEnvironmentConfigSuccess(
var cfg *Config t, tt.isDev, tt.isProd,
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
New,
),
fx.Populate(&cfg),
) )
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(t, tt.isDev, cfg.IsDev())
assert.Equal(t, tt.isProd, cfg.IsProd())
} }
}) })
} }
} }
func TestSessionKeyDefaults(t *testing.T) { func testEnvironmentConfigError(t *testing.T) {
t.Helper()
var cfg *config.Config
app := fx.New(
fx.NopLogger,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
assert.Error(t, app.Err())
}
func testEnvironmentConfigSuccess(
t *testing.T,
isDev, isProd bool,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(t, isDev, cfg.IsDev())
assert.Equal(t, isProd, cfg.IsProd())
}
func TestRetentionSweepInterval(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
environment string set bool
sessionKey string value string
dburl string
expectError bool expectError bool
expectedKey string expected time.Duration
}{ }{
{ {
name: "dev mode with default session key", name: caseUnsetUsesDefault,
environment: "dev", set: false,
sessionKey: "", expected: time.Hour,
expectError: false,
expectedKey: DevSessionKey,
}, },
{ {
name: "dev mode with custom session key", name: caseValidValueParsed,
environment: "dev", set: true,
sessionKey: "Y3VzdG9tLXNlc3Npb24ta2V5LTMyLWJ5dGVzLWxvbmchIQ==", value: "15m",
expectError: false, expected: 15 * time.Minute,
expectedKey: "Y3VzdG9tLXNlc3Npb24ta2V5LTMyLWJ5dGVzLWxvbmchIQ==",
}, },
{ {
name: "prod mode with no session key fails", name: caseUnparseableFails,
environment: "prod", set: true,
sessionKey: "", value: "not-a-duration",
dburl: "postgres://prod:prod@localhost:5432/prod",
expectError: true, expectError: true,
}, },
{
name: "prod mode with session key succeeds",
environment: "prod",
sessionKey: "cHJvZC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
dburl: "postgres://prod:prod@localhost:5432/prod",
expectError: false,
expectedKey: "cHJvZC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Create in-memory filesystem with test config // Cannot use t.Parallel() here because t.Setenv
fs := afero.NewMemMapFs() // is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
// Create custom config for session key tests if tt.set {
configYAML := ` t.Setenv("RETENTION_SWEEP_INTERVAL", tt.value)
environments: } else {
dev: require.NoError(t, os.Unsetenv(
config: "RETENTION_SWEEP_INTERVAL",
environment: dev ))
developmentMode: true
dburl: postgres://test:test@localhost:5432/test_dev
secrets:`
// Only add sessionKey line if it's not empty
if tt.sessionKey != "" {
configYAML += `
sessionKey: ` + tt.sessionKey
} else if tt.environment == "dev" {
// For dev mode with no session key, use the default
configYAML += `
sessionKey: d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=`
}
// Add prod config if testing prod
if tt.environment == "prod" {
configYAML += `
prod:
config:
environment: prod
developmentMode: false
dburl: $ENV:DBURL
secrets:
sessionKey: $ENV:SESSION_KEY`
}
require.NoError(t, afero.WriteFile(fs, "config.yaml", []byte(configYAML), 0644))
pkgconfig.SetFs(fs)
// Clean up any existing env vars
os.Unsetenv("WEBHOOKER_ENVIRONMENT")
os.Unsetenv("SESSION_KEY")
os.Unsetenv("DBURL")
// Set environment variables
os.Setenv("WEBHOOKER_ENVIRONMENT", tt.environment)
defer os.Unsetenv("WEBHOOKER_ENVIRONMENT")
if tt.sessionKey != "" && tt.environment == "prod" {
os.Setenv("SESSION_KEY", tt.sessionKey)
defer os.Unsetenv("SESSION_KEY")
}
if tt.dburl != "" {
os.Setenv("DBURL", tt.dburl)
defer os.Unsetenv("DBURL")
} }
if tt.expectError { if tt.expectError {
// Use regular fx.New for error cases expectStartupError(t)
var cfg *Config } else {
testRetentionSweepIntervalSuccess(t, tt.expected)
}
})
}
}
// startupError builds the app config.New belongs to and returns
// the error fx reports, which is non-nil whenever an environment
// value is set but invalid.
func startupError(t *testing.T) error {
t.Helper()
var cfg *config.Config
app := fx.New( app := fx.New(
fx.NopLogger, // Suppress fx logs in tests fx.NopLogger,
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
New, config.New,
), ),
fx.Populate(&cfg), fx.Populate(&cfg),
) )
assert.Error(t, app.Err())
} else { return app.Err()
// Use fxtest for success cases }
var cfg *Config
// expectStartupError asserts that fx refuses to build the app,
// which is what a set-but-invalid environment value must cause.
func expectStartupError(t *testing.T) {
t.Helper()
assert.Error(t, startupError(t))
}
// expectStartupErrorFor asserts that startup fails, that the error
// names the offending variable so an operator can find it, and,
// when sentinel is non-nil, that it wraps that sentinel.
func expectStartupErrorFor(
t *testing.T,
key string,
sentinel error,
) {
t.Helper()
err := startupError(t)
require.ErrorContains(t, err, key)
if sentinel != nil {
require.ErrorIs(t, err, sentinel)
}
}
func testRetentionSweepIntervalSuccess(
t *testing.T,
expected time.Duration,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New( app := fxtest.New(
t, t,
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
New, config.New,
), ),
fx.Populate(&cfg), fx.Populate(&cfg),
) )
require.NoError(t, app.Err()) require.NoError(t, app.Err())
app.RequireStart() app.RequireStart()
defer app.RequireStop() defer app.RequireStop()
if tt.environment == "dev" && tt.sessionKey == "" { assert.Equal(t, expected, cfg.RetentionSweepInterval)
// Dev mode with no session key uses default }
assert.Equal(t, DevSessionKey, cfg.SessionKey)
} else { func TestSessionIdleTimeout(t *testing.T) {
assert.Equal(t, tt.expectedKey, cfg.SessionKey) tests := []struct {
name string
set bool
value string
expectError bool
expected time.Duration
}{
{
name: caseUnsetUsesDefault,
set: false,
expected: 24 * time.Hour,
},
{
name: caseValidValueParsed,
set: true,
value: "30m",
expected: 30 * time.Minute,
},
{
name: caseUnparseableFails,
set: true,
value: "not-a-duration",
expectError: true,
},
} }
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set {
t.Setenv("SESSION_IDLE_TIMEOUT", tt.value)
} else {
require.NoError(t, os.Unsetenv(
"SESSION_IDLE_TIMEOUT",
))
}
if tt.expectError {
expectStartupError(t)
} else {
testSessionIdleTimeoutSuccess(t, tt.expected)
} }
}) })
} }
} }
func testSessionIdleTimeoutSuccess(
t *testing.T,
expected time.Duration,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(t, expected, cfg.SessionIdleTimeout)
}
func TestDefaultDataDir(t *testing.T) {
for _, env := range []string{"", "dev", "prod"} {
name := env
if name == "" {
name = "unset"
}
t.Run("env="+name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
if env != "" {
t.Setenv("WEBHOOKER_ENVIRONMENT", env)
} else {
require.NoError(t, os.Unsetenv(
"WEBHOOKER_ENVIRONMENT",
))
}
require.NoError(t, os.Unsetenv("DATA_DIR"))
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(
t, "/var/lib/webhooker", cfg.DataDir,
)
})
}
}
func TestReceiverRateLimit(t *testing.T) {
tests := []struct {
name string
set bool
value string
expectError bool
// sentinel, when set, must be wrapped by the startup
// error; every error case must additionally name the
// variable in its message.
sentinel error
expected int
}{
{
name: caseUnsetUsesDefault,
set: false,
expected: 120,
},
{
name: caseValidValueParsed,
set: true,
value: "30",
expected: 30,
},
{
name: caseUnparseableFails,
set: true,
value: "not-a-number",
expectError: true,
},
{
name: "zero fails startup",
set: true,
value: "0",
expectError: true,
sentinel: config.ErrNonPositiveValue,
},
{
name: "negative fails startup",
set: true,
value: "-5",
expectError: true,
sentinel: config.ErrNonPositiveValue,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set {
t.Setenv("RECEIVER_RATE_LIMIT", tt.value)
} else {
require.NoError(t, os.Unsetenv(
"RECEIVER_RATE_LIMIT",
))
}
if tt.expectError {
expectStartupErrorFor(
t, "RECEIVER_RATE_LIMIT", tt.sentinel,
)
} else {
testReceiverRateLimitSuccess(t, tt.expected)
}
})
}
}
func testReceiverRateLimitSuccess(
t *testing.T,
expected int,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(t, expected, cfg.ReceiverRateLimit)
}
func TestTrustedProxies(t *testing.T) {
tests := []struct {
name string
set bool
value string
expectError bool
expected []string
}{
{
// The default must be "trust nobody": an empty list
// means forwarded headers are ignored, never that
// every peer may speak for the client.
name: caseUnsetUsesDefault,
set: false,
expected: []string{},
},
{
name: "blank value trusts nothing",
set: true,
value: " ",
expected: []string{},
},
{
name: caseValidValueParsed,
set: true,
value: cidrPrivateV4 + ", 192.168.1.7 ,2001:db8::/32",
expected: []string{
cidrPrivateV4, "192.168.1.7/32", "2001:db8::/32",
},
},
{
name: "host bits are masked off",
set: true,
value: "10.1.2.3/8",
expected: []string{cidrPrivateV4},
},
{
// Peer addresses are unmapped before they are
// matched, so an IPv4-mapped prefix kept in that
// form could never match anything.
name: "IPv4-mapped prefix is unmapped",
set: true,
value: "::ffff:10.0.0.0/104",
expected: []string{cidrPrivateV4},
},
{
name: caseUnparseableFails,
set: true,
value: cidrPrivateV4 + ",not-an-address",
expectError: true,
},
{
name: "out-of-range prefix length fails startup",
set: true,
value: "10.0.0.0/33",
expectError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set {
t.Setenv("TRUSTED_PROXIES", tt.value)
} else {
require.NoError(t, os.Unsetenv("TRUSTED_PROXIES"))
}
if tt.expectError {
expectStartupErrorFor(
t, "TRUSTED_PROXIES", config.ErrInvalidCIDR,
)
} else {
testTrustedProxiesSuccess(t, tt.expected)
}
})
}
}
func testTrustedProxiesSuccess(
t *testing.T,
expected []string,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
got := make([]string, 0, len(cfg.TrustedProxies))
for _, prefix := range cfg.TrustedProxies {
got = append(got, prefix.String())
}
assert.Equal(t, expected, got)
}

409
internal/config/env_test.go Normal file
View File

@@ -0,0 +1,409 @@
package config_test
import (
"os"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger"
)
// testEnvKey is a throwaway variable name used only by the helper
// tables below, so they cannot disturb real configuration.
const testEnvKey = "WEBHOOKER_TEST_VALUE"
// Real configuration variables exercised by the config.New tests.
const (
envKeyPort = "PORT"
envKeyDebug = "DEBUG"
envKeyMaintenanceMode = "MAINTENANCE_MODE"
)
// envBoolCase is one row of the envBool table.
type envBoolCase struct {
name string
set bool
value string
defaultValue bool
expectError bool
expected bool
}
// envBoolCases is the envBool table, kept out of the test body so
// the test itself stays readable.
func envBoolCases() []envBoolCase {
return []envBoolCase{
{
name: "unset uses default false",
defaultValue: false,
expected: false,
},
{
name: "unset uses default true",
defaultValue: true,
expected: true,
},
{
name: "empty uses default true",
set: true,
value: "",
defaultValue: true,
expected: true,
},
{
name: "true is parsed",
set: true,
value: "true",
expected: true,
},
{
name: "one is parsed",
set: true,
value: "1",
expected: true,
},
{
name: "False is parsed",
set: true,
value: "False",
defaultValue: true,
expected: false,
},
{
name: "zero is parsed",
set: true,
value: "0",
defaultValue: true,
expected: false,
},
{
name: "yes is rejected",
set: true,
value: "yes",
expectError: true,
},
{
name: "on is rejected",
set: true,
value: "on",
expectError: true,
},
{
name: "typo is rejected",
set: true,
value: "ture",
expectError: true,
},
}
}
func TestEnvBool(t *testing.T) {
for _, tt := range envBoolCases() {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
if tt.set {
t.Setenv(testEnvKey, tt.value)
} else {
require.NoError(t, os.Unsetenv(testEnvKey))
}
got, err := config.EnvBoolForTest(
testEnvKey, tt.defaultValue,
)
if tt.expectError {
require.Error(t, err)
assert.Contains(t, err.Error(), testEnvKey)
assert.Contains(t, err.Error(), tt.value)
return
}
require.NoError(t, err)
assert.Equal(t, tt.expected, got)
})
}
}
func TestEnvPositiveInt(t *testing.T) {
const defaultValue = 7
tests := []struct {
name string
set bool
value string
expectError bool
errIs error
expected int
}{
{
name: "unset returns the default integer",
expected: defaultValue,
},
{
name: "empty returns the default integer",
set: true,
value: "",
expected: defaultValue,
},
{
name: "positive value is parsed",
set: true,
value: "42",
expected: 42,
},
{
name: "unparseable value is rejected",
set: true,
value: "not-a-number",
expectError: true,
},
{
name: "zero is rejected",
set: true,
value: "0",
expectError: true,
errIs: config.ErrNonPositiveValue,
},
{
name: "negative is rejected",
set: true,
value: "-5",
expectError: true,
errIs: config.ErrNonPositiveValue,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
if tt.set {
t.Setenv(testEnvKey, tt.value)
} else {
require.NoError(t, os.Unsetenv(testEnvKey))
}
got, err := config.EnvPositiveIntForTest(
testEnvKey, defaultValue,
)
if tt.expectError {
require.Error(t, err)
assert.Contains(t, err.Error(), testEnvKey)
assert.Contains(t, err.Error(), tt.value)
if tt.errIs != nil {
require.ErrorIs(t, err, tt.errIs)
}
return
}
require.NoError(t, err)
assert.Equal(t, tt.expected, got)
})
}
}
func TestEnvPort(t *testing.T) {
const defaultValue = 8080
tests := []struct {
name string
set bool
value string
expectError bool
errIs error
expected int
}{
{
name: "unset returns the default port",
expected: defaultValue,
},
{
name: "valid port is parsed",
set: true,
value: "9000",
expected: 9000,
},
{
name: "highest port is accepted",
set: true,
value: "65535",
expected: 65535,
},
{
name: "unparseable value is rejected",
set: true,
value: "not-a-port",
expectError: true,
},
{
name: "zero is rejected",
set: true,
value: "0",
expectError: true,
errIs: config.ErrNonPositiveValue,
},
{
name: "above the port range is rejected",
set: true,
value: "65536",
expectError: true,
errIs: config.ErrInvalidPort,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
if tt.set {
t.Setenv(testEnvKey, tt.value)
} else {
require.NoError(t, os.Unsetenv(testEnvKey))
}
got, err := config.EnvPortForTest(
testEnvKey, defaultValue,
)
if tt.expectError {
require.Error(t, err)
assert.Contains(t, err.Error(), testEnvKey)
if tt.errIs != nil {
require.ErrorIs(t, err, tt.errIs)
}
return
}
require.NoError(t, err)
assert.Equal(t, tt.expected, got)
})
}
}
// buildConfig constructs a Config through fx exactly as the
// application does, returning the config and any construction error.
func buildConfig(t *testing.T) (*config.Config, error) {
t.Helper()
var cfg *config.Config
app := fx.New(
fx.NopLogger,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
return cfg, app.Err()
}
func TestNewRejectsBadEnvValues(t *testing.T) {
tests := []struct {
name string
key string
value string
expectError bool
check func(t *testing.T, cfg *config.Config)
}{
{
name: "valid PORT is used",
key: envKeyPort,
value: "9001",
check: func(t *testing.T, cfg *config.Config) {
t.Helper()
assert.Equal(t, 9001, cfg.Port)
},
},
{
name: "unparseable PORT aborts startup",
key: envKeyPort,
value: "eighty-eighty",
expectError: true,
},
{
name: "out-of-range PORT aborts startup",
key: envKeyPort,
value: "70000",
expectError: true,
},
{
name: "valid DEBUG is used",
key: envKeyDebug,
value: "true",
check: func(t *testing.T, cfg *config.Config) {
t.Helper()
assert.True(t, cfg.Debug)
},
},
{
name: "unparseable DEBUG aborts startup",
key: envKeyDebug,
value: "ture",
expectError: true,
},
{
name: "unparseable MAINTENANCE_MODE aborts startup",
key: envKeyMaintenanceMode,
value: "sometimes",
expectError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
t.Setenv(tt.key, tt.value)
cfg, err := buildConfig(t)
if tt.expectError {
require.Error(t, err)
assert.Contains(t, err.Error(), tt.key)
assert.Contains(t, err.Error(), tt.value)
return
}
require.NoError(t, err)
require.NotNil(t, cfg)
tt.check(t, cfg)
})
}
}
// TestNewUsesDefaultsWhenUnset proves the fail-loud behaviour did not
// break the legitimate unset case: absent variables still get their
// documented defaults.
func TestNewUsesDefaultsWhenUnset(t *testing.T) {
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
for _, key := range []string{
envKeyPort, envKeyDebug, envKeyMaintenanceMode,
} {
require.NoError(t, os.Unsetenv(key))
}
cfg, err := buildConfig(t)
require.NoError(t, err)
require.NotNil(t, cfg)
assert.Equal(t, 8080, cfg.Port)
assert.False(t, cfg.Debug)
assert.False(t, cfg.MaintenanceMode)
}

View File

@@ -0,0 +1,20 @@
package config
// This file exposes the unexported environment parsing helpers to
// the external config_test package so each helper can be covered by
// its own table-driven test without weakening the package API.
// EnvBoolForTest exposes envBool.
func EnvBoolForTest(key string, defaultValue bool) (bool, error) {
return envBool(key, defaultValue)
}
// EnvPositiveIntForTest exposes envPositiveInt.
func EnvPositiveIntForTest(key string, defaultValue int) (int, error) {
return envPositiveInt(key, defaultValue)
}
// EnvPortForTest exposes envPort.
func EnvPortForTest(key string, defaultValue int) (int, error) {
return envPort(key, defaultValue)
}

View File

@@ -11,15 +11,16 @@ import (
// This replaces gorm.Model but uses UUID instead of uint for ID // This replaces gorm.Model but uses UUID instead of uint for ID
type BaseModel struct { type BaseModel struct {
ID string `gorm:"type:uuid;primary_key" json:"id"` ID string `gorm:"type:uuid;primary_key" json:"id"`
CreatedAt time.Time `json:"created_at"` CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updated_at"` UpdatedAt time.Time `json:"updatedAt"`
DeletedAt gorm.DeletedAt `gorm:"index" json:"deleted_at,omitempty"` DeletedAt gorm.DeletedAt `gorm:"index" json:"deletedAt,omitzero"`
} }
// BeforeCreate hook to set UUID before creating a record // BeforeCreate hook to set UUID before creating a record.
func (b *BaseModel) BeforeCreate(tx *gorm.DB) error { func (b *BaseModel) BeforeCreate(_ *gorm.DB) error {
if b.ID == "" { if b.ID == "" {
b.ID = uuid.New().String() b.ID = uuid.New().String()
} }
return nil return nil
} }

View File

@@ -1,9 +1,16 @@
// Package database provides SQLite persistence for webhooks, events, and users.
package database package database
import ( import (
"context" "context"
"crypto/rand"
"database/sql" "database/sql"
"encoding/base64"
"errors"
"fmt"
"log/slog" "log/slog"
"os"
"path/filepath"
"go.uber.org/fx" "go.uber.org/fx"
"gorm.io/driver/sqlite" "gorm.io/driver/sqlite"
@@ -13,30 +20,42 @@ import (
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
// nolint:revive // DatabaseParams is a standard fx naming convention const (
dataDirPerm = 0750
randomPasswordLen = 16
sessionKeyLen = 32
)
//nolint:revive // DatabaseParams is a standard fx naming convention.
type DatabaseParams struct { type DatabaseParams struct {
fx.In fx.In
Config *config.Config Config *config.Config
Logger *logger.Logger Logger *logger.Logger
} }
// Database manages the main SQLite connection and schema migrations.
type Database struct { type Database struct {
db *gorm.DB db *gorm.DB
log *slog.Logger log *slog.Logger
params *DatabaseParams params *DatabaseParams
} }
func New(lc fx.Lifecycle, params DatabaseParams) (*Database, error) { // New creates a Database that connects on fx start and disconnects on stop.
func New(
lc fx.Lifecycle,
params DatabaseParams,
) (*Database, error) {
d := &Database{ d := &Database{
params: &params, params: &params,
log: params.Logger.Get(), log: params.Logger.Get(),
} }
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(_ context.Context) error { // nolint:revive // ctx unused but required by fx OnStart: func(_ context.Context) error {
return d.connect() return d.connect()
}, },
OnStop: func(_ context.Context) error { // nolint:revive // ctx unused but required by fx OnStop: func(_ context.Context) error {
return d.close() return d.close()
}, },
}) })
@@ -44,17 +63,92 @@ func New(lc fx.Lifecycle, params DatabaseParams) (*Database, error) {
return d, nil return d, nil
} }
func (d *Database) connect() error { // DB returns the underlying GORM database handle.
dbURL := d.params.Config.DBURL func (d *Database) DB() *gorm.DB {
if dbURL == "" { return d.db
// Default to SQLite for development }
dbURL = "file:webhooker.db?cache=shared&mode=rwc"
// GetOrCreateSessionKey retrieves the session encryption key from the
// settings table. If no key exists, a cryptographically secure random
// 32-byte key is generated, base64-encoded, and stored for future use.
func (d *Database) GetOrCreateSessionKey() (string, error) {
var setting Setting
result := d.db.Where(
&Setting{Key: "session_key"},
).First(&setting)
if result.Error == nil {
return setting.Value, nil
} }
// First, open the database with the pure Go driver if !errors.Is(result.Error, gorm.ErrRecordNotFound) {
return "", fmt.Errorf(
"failed to query session key: %w",
result.Error,
)
}
// Generate a new cryptographically secure 32-byte key
keyBytes := make([]byte, sessionKeyLen)
_, err := rand.Read(keyBytes)
if err != nil {
return "", fmt.Errorf(
"failed to generate session key: %w",
err,
)
}
encoded := base64.StdEncoding.EncodeToString(keyBytes)
setting = Setting{
Key: "session_key",
Value: encoded,
}
err = d.db.Create(&setting).Error
if err != nil {
return "", fmt.Errorf(
"failed to store session key: %w",
err,
)
}
d.log.Info(
"generated new session key and stored in database",
)
return encoded, nil
}
func (d *Database) connect() error {
// Ensure the data directory exists before opening the database.
dataDir := d.params.Config.DataDir
err := os.MkdirAll(dataDir, dataDirPerm)
if err != nil {
return fmt.Errorf(
"creating data directory %s: %w",
dataDir,
err,
)
}
// Construct the main application database path inside DATA_DIR.
dbPath := filepath.Join(dataDir, "webhooker.db")
dbURL := fmt.Sprintf(
"file:%s?cache=shared&mode=rwc",
dbPath,
)
// Open the database with the pure Go SQLite driver
sqlDB, err := sql.Open("sqlite", dbURL) sqlDB, err := sql.Open("sqlite", dbURL)
if err != nil { if err != nil {
d.log.Error("failed to open database", "error", err) d.log.Error(
"failed to open database",
"error", err,
)
return err return err
} }
@@ -63,12 +157,16 @@ func (d *Database) connect() error {
Conn: sqlDB, Conn: sqlDB,
}, &gorm.Config{}) }, &gorm.Config{})
if err != nil { if err != nil {
d.log.Error("failed to connect to database", "error", err) d.log.Error(
"failed to connect to database",
"error", err,
)
return err return err
} }
d.db = db d.db = db
d.log.Info("connected to database", "database", dbURL) d.log.Info("connected to database", "path", dbPath)
// Run migrations // Run migrations
return d.migrate() return d.migrate()
@@ -76,34 +174,62 @@ func (d *Database) connect() error {
func (d *Database) migrate() error { func (d *Database) migrate() error {
// Run GORM auto-migrations // Run GORM auto-migrations
if err := d.Migrate(); err != nil { err := d.Migrate()
d.log.Error("failed to run database migrations", "error", err) if err != nil {
d.log.Error(
"failed to run database migrations",
"error", err,
)
return err return err
} }
d.log.Info("database migrations completed") d.log.Info("database migrations completed")
// Check if admin user exists // Check if admin user exists
var userCount int64 var userCount int64
if err := d.db.Model(&User{}).Count(&userCount).Error; err != nil {
d.log.Error("failed to count users", "error", err) err = d.db.Model(&User{}).Count(&userCount).Error
if err != nil {
d.log.Error(
"failed to count users",
"error", err,
)
return err return err
} }
if userCount == 0 { if userCount == 0 {
// Create admin user return d.createAdminUser()
}
return nil
}
func (d *Database) createAdminUser() error {
d.log.Info("no users found, creating admin user") d.log.Info("no users found, creating admin user")
// Generate random password // Generate random password
password, err := GenerateRandomPassword(16) password, err := GenerateRandomPassword(
randomPasswordLen,
)
if err != nil { if err != nil {
d.log.Error("failed to generate random password", "error", err) d.log.Error(
"failed to generate random password",
"error", err,
)
return err return err
} }
// Hash the password // Hash the password
hashedPassword, err := HashPassword(password) hashedPassword, err := HashPassword(password)
if err != nil { if err != nil {
d.log.Error("failed to hash password", "error", err) d.log.Error(
"failed to hash password",
"error", err,
)
return err return err
} }
@@ -113,17 +239,22 @@ func (d *Database) migrate() error {
Password: hashedPassword, Password: hashedPassword,
} }
if err := d.db.Create(adminUser).Error; err != nil { err = d.db.Create(adminUser).Error
d.log.Error("failed to create admin user", "error", err) if err != nil {
d.log.Error(
"failed to create admin user",
"error", err,
)
return err return err
} }
// Log the password - this will only happen once on first startup
d.log.Info("admin user created", d.log.Info("admin user created",
"username", "admin", "username", "admin",
"password", password, "password", password,
"message", "SAVE THIS PASSWORD - it will not be shown again!") "message",
} "SAVE THIS PASSWORD - it will not be shown again!",
)
return nil return nil
} }
@@ -134,11 +265,9 @@ func (d *Database) close() error {
if err != nil { if err != nil {
return err return err
} }
return sqlDB.Close() return sqlDB.Close()
} }
return nil return nil
} }
func (d *Database) DB() *gorm.DB {
return d.db
}

View File

@@ -1,74 +1,56 @@
package database package database_test
import ( import (
"context" "context"
"testing" "testing"
"github.com/spf13/afero"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
pkgconfig "sneak.berlin/go/webhooker/pkg/config"
) )
func TestDatabaseConnection(t *testing.T) { const (
// Set up in-memory config so the test does not depend on config.yaml on disk // testAppname is the Globals.Appname used in tests.
fs := afero.NewMemMapFs() testAppname = "webhooker-test"
testConfigYAML := ` // testVersion is the Globals.Version used in tests.
environments: testVersion = "test"
dev: // testContentType is the event content type used in tests.
config: testContentType = "application/json"
port: 8080 // testWebhookName is the Webhook.Name used in tests.
debug: false testWebhookName = "test-webhook"
maintenanceMode: false // testForeverLabel is Webhook.RetentionLabel for a retain-forever
developmentMode: true // webhook.
environment: dev testForeverLabel = "forever"
dburl: "file::memory:?cache=shared" )
secrets:
sessionKey: d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE= func setupTestDB(
sentryDSN: "" t *testing.T,
configDefaults: ) (*database.Database, *fxtest.Lifecycle) {
port: 8080 t.Helper()
`
if err := afero.WriteFile(fs, "config.yaml", []byte(testConfigYAML), 0644); err != nil {
t.Fatalf("Failed to write test config: %v", err)
}
pkgconfig.SetFs(fs)
// Set up test dependencies
lc := fxtest.NewLifecycle(t) lc := fxtest.NewLifecycle(t)
// Create globals g := &globals.Globals{
globals.Appname = "webhooker-test" Appname: testAppname,
globals.Version = "test" Version: testVersion,
globals.Buildarch = "test"
g, err := globals.New(lc)
if err != nil {
t.Fatalf("Failed to create globals: %v", err)
} }
// Create logger l, err := logger.New(
l, err := logger.New(lc, logger.LoggerParams{Globals: g}) lc,
logger.LoggerParams{Globals: g},
)
if err != nil { if err != nil {
t.Fatalf("Failed to create logger: %v", err) t.Fatalf("Failed to create logger: %v", err)
} }
// Create config c := &config.Config{
c, err := config.New(lc, config.ConfigParams{ DataDir: t.TempDir(),
Globals: g, Environment: "dev",
Logger: l,
})
if err != nil {
t.Fatalf("Failed to create config: %v", err)
} }
// Override DBURL to use a temp file-based SQLite (in-memory doesn't persist across connections) db, err := database.New(lc, database.DatabaseParams{
c.DBURL = "file:" + t.TempDir() + "/test.db?cache=shared&mode=rwc"
// Create database
db, err := New(lc, DatabaseParams{
Config: c, Config: c,
Logger: l, Logger: l,
}) })
@@ -76,31 +58,45 @@ configDefaults:
t.Fatalf("Failed to create database: %v", err) t.Fatalf("Failed to create database: %v", err)
} }
// Start lifecycle (this will trigger the connection) return db, lc
}
func TestDatabaseConnection(t *testing.T) {
t.Parallel()
db, lc := setupTestDB(t)
ctx := context.Background() ctx := context.Background()
err = lc.Start(ctx)
err := lc.Start(ctx)
if err != nil { if err != nil {
t.Fatalf("Failed to connect to database: %v", err) t.Fatalf("Failed to connect to database: %v", err)
} }
defer func() { defer func() {
if stopErr := lc.Stop(ctx); stopErr != nil { stopErr := lc.Stop(ctx)
t.Errorf("Failed to stop lifecycle: %v", stopErr) if stopErr != nil {
t.Errorf(
"Failed to stop lifecycle: %v",
stopErr,
)
} }
}() }()
// Verify we can get the DB instance
if db.DB() == nil { if db.DB() == nil {
t.Error("Expected non-nil database connection") t.Error("Expected non-nil database connection")
} }
// Test that we can perform a simple query
var result int var result int
err = db.DB().Raw("SELECT 1").Scan(&result).Error err = db.DB().Raw("SELECT 1").Scan(&result).Error
if err != nil { if err != nil {
t.Fatalf("Failed to execute test query: %v", err) t.Fatalf("Failed to execute test query: %v", err)
} }
if result != 1 { if result != 1 {
t.Errorf("Expected query result to be 1, got %d", result) t.Errorf(
"Expected query result to be 1, got %d",
result,
)
} }
} }

View File

@@ -0,0 +1,67 @@
package database
import (
"context"
"log/slog"
"os"
"time"
"go.uber.org/fx"
)
// NewTestRetentionReaper builds a RetentionReaper backed by the given
// main database and per-webhook database manager, without the fx
// lifecycle. Intended for tests.
func NewTestRetentionReaper(
db *Database,
mgr *WebhookDBManager,
) *RetentionReaper {
return &RetentionReaper{
db: db,
dbManager: mgr,
log: slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
)),
interval: time.Hour,
}
}
// ExportSweep runs a single retention sweep synchronously for tests.
func (r *RetentionReaper) ExportSweep(ctx context.Context) {
r.sweep(ctx)
}
// ExportRegisterHooks registers the reaper's real fx lifecycle hooks
// on a lifecycle supplied by a test, so a test can drive the exact
// OnStart/OnStop functions the application runs and hand OnStart the
// kind of context fx actually supplies.
func (r *RetentionReaper) ExportRegisterHooks(lc fx.Lifecycle) {
r.registerHooks(lc)
}
// ExportStart starts the reaper's background loop for tests.
func (r *RetentionReaper) ExportStart() {
r.start()
}
// ExportStop stops the reaper's background loop for tests.
func (r *RetentionReaper) ExportStop(ctx context.Context) error {
return r.stop(ctx)
}
// ExportWedgeLoop adds a goroutine to the reaper's WaitGroup that
// never observes cancellation and returns only when release is
// closed. It stands in for a sweep stuck on a locked database.
func (r *RetentionReaper) ExportWedgeLoop(
release <-chan struct{},
) {
r.wg.Go(func() {
<-release
})
}
// ExportSetInterval overrides the sweep interval for tests.
func (r *RetentionReaper) ExportSetInterval(d time.Duration) {
r.interval = d
}

View File

@@ -6,11 +6,11 @@ import "time"
type APIKey struct { type APIKey struct {
BaseModel BaseModel
UserID string `gorm:"type:uuid;not null" json:"user_id"` UserID string `gorm:"type:uuid;not null" json:"userId"`
Key string `gorm:"uniqueIndex;not null" json:"key"` Key string `gorm:"uniqueIndex;not null" json:"key"`
Description string `json:"description"` Description string `json:"description"`
LastUsedAt *time.Time `json:"last_used_at,omitempty"` LastUsedAt *time.Time `json:"lastUsedAt,omitempty"`
// Relations // Relations
User User `json:"user,omitempty"` User User `json:"user,omitzero"`
} }

View File

@@ -3,6 +3,7 @@ package database
// DeliveryStatus represents the status of a delivery // DeliveryStatus represents the status of a delivery
type DeliveryStatus string type DeliveryStatus string
// Delivery status values.
const ( const (
DeliveryStatusPending DeliveryStatus = "pending" DeliveryStatusPending DeliveryStatus = "pending"
DeliveryStatusDelivered DeliveryStatus = "delivered" DeliveryStatusDelivered DeliveryStatus = "delivered"
@@ -14,12 +15,12 @@ const (
type Delivery struct { type Delivery struct {
BaseModel BaseModel
EventID string `gorm:"type:uuid;not null" json:"event_id"` EventID string `gorm:"type:uuid;not null" json:"eventId"`
TargetID string `gorm:"type:uuid;not null" json:"target_id"` TargetID string `gorm:"type:uuid;not null" json:"targetId"`
Status DeliveryStatus `gorm:"not null;default:'pending'" json:"status"` Status DeliveryStatus `gorm:"not null;default:'pending'" json:"status"`
// Relations // Relations
Event Event `json:"event,omitempty"` Event Event `json:"event,omitzero"`
Target Target `json:"target,omitempty"` Target Target `json:"target,omitzero"`
DeliveryResults []DeliveryResult `json:"delivery_results,omitempty"` DeliveryResults []DeliveryResult `json:"deliveryResults,omitempty"`
} }

View File

@@ -4,14 +4,14 @@ package database
type DeliveryResult struct { type DeliveryResult struct {
BaseModel BaseModel
DeliveryID string `gorm:"type:uuid;not null" json:"delivery_id"` DeliveryID string `gorm:"type:uuid;not null" json:"deliveryId"`
AttemptNum int `gorm:"not null" json:"attempt_num"` AttemptNum int `gorm:"not null" json:"attemptNum"`
Success bool `json:"success"` Success bool `json:"success"`
StatusCode int `json:"status_code,omitempty"` StatusCode int `json:"statusCode,omitempty"`
ResponseBody string `gorm:"type:text" json:"response_body,omitempty"` ResponseBody string `gorm:"type:text" json:"responseBody,omitempty"`
Error string `json:"error,omitempty"` Error string `json:"error,omitempty"`
Duration int64 `json:"duration_ms"` // Duration in milliseconds Duration int64 `json:"durationMs"` // Duration in milliseconds
// Relations // Relations
Delivery Delivery `json:"delivery,omitempty"` Delivery Delivery `json:"delivery,omitzero"`
} }

View File

@@ -4,11 +4,14 @@ package database
type Entrypoint struct { type Entrypoint struct {
BaseModel BaseModel
WebhookID string `gorm:"type:uuid;not null" json:"webhook_id"` WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
Path string `gorm:"uniqueIndex;not null" json:"path"` // URL path for this entrypoint
// Path is the URL path for this entrypoint.
Path string `gorm:"uniqueIndex;not null" json:"path"`
Description string `json:"description"` Description string `json:"description"`
Active bool `gorm:"default:true" json:"active"` Active bool `gorm:"default:true" json:"active"`
// Relations // Relations
Webhook Webhook `json:"webhook,omitempty"` Webhook Webhook `json:"webhook,omitzero"`
} }

View File

@@ -4,17 +4,17 @@ package database
type Event struct { type Event struct {
BaseModel BaseModel
WebhookID string `gorm:"type:uuid;not null" json:"webhook_id"` WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
EntrypointID string `gorm:"type:uuid;not null" json:"entrypoint_id"` EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
// Request data // Request data
Method string `gorm:"not null" json:"method"` Method string `gorm:"not null" json:"method"`
Headers string `gorm:"type:text" json:"headers"` // JSON Headers string `gorm:"type:text" json:"headers"` // JSON
Body string `gorm:"type:text" json:"body"` Body string `gorm:"type:text" json:"body"`
ContentType string `json:"content_type"` ContentType string `json:"contentType"`
// Relations // Relations
Webhook Webhook `json:"webhook,omitempty"` Webhook Webhook `json:"webhook,omitzero"`
Entrypoint Entrypoint `json:"entrypoint,omitempty"` Entrypoint Entrypoint `json:"entrypoint,omitzero"`
Deliveries []Delivery `json:"deliveries,omitempty"` Deliveries []Delivery `json:"deliveries,omitempty"`
} }

View File

@@ -0,0 +1,8 @@
package database
// Setting stores application-level key-value configuration.
// Used for auto-generated values like the session encryption key.
type Setting struct {
Key string `gorm:"primaryKey" json:"key"`
Value string `gorm:"type:text;not null" json:"value"`
}

View File

@@ -3,18 +3,19 @@ package database
// TargetType represents the type of delivery target // TargetType represents the type of delivery target
type TargetType string type TargetType string
// Target type values.
const ( const (
TargetTypeHTTP TargetType = "http" TargetTypeHTTP TargetType = "http"
TargetTypeRetry TargetType = "retry"
TargetTypeDatabase TargetType = "database" TargetTypeDatabase TargetType = "database"
TargetTypeLog TargetType = "log" TargetTypeLog TargetType = "log"
TargetTypeSlack TargetType = "slack"
) )
// Target represents a delivery target for a webhook // Target represents a delivery target for a webhook
type Target struct { type Target struct {
BaseModel BaseModel
WebhookID string `gorm:"type:uuid;not null" json:"webhook_id"` WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
Name string `gorm:"not null" json:"name"` Name string `gorm:"not null" json:"name"`
Type TargetType `gorm:"not null" json:"type"` Type TargetType `gorm:"not null" json:"type"`
Active bool `gorm:"default:true" json:"active"` Active bool `gorm:"default:true" json:"active"`
@@ -22,11 +23,12 @@ type Target struct {
// Configuration fields (JSON stored based on type) // Configuration fields (JSON stored based on type)
Config string `gorm:"type:text" json:"config"` // JSON configuration Config string `gorm:"type:text" json:"config"` // JSON configuration
// For retry targets // For HTTP targets (max_retries=0 means fire-and-forget,
MaxRetries int `json:"max_retries,omitempty"` // >0 enables retries with backoff)
MaxQueueSize int `json:"max_queue_size,omitempty"` MaxRetries int `json:"maxRetries,omitempty"`
MaxQueueSize int `json:"maxQueueSize,omitempty"`
// Relations // Relations
Webhook Webhook `json:"webhook,omitempty"` Webhook Webhook `json:"webhook,omitzero"`
Deliveries []Delivery `json:"deliveries,omitempty"` Deliveries []Delivery `json:"deliveries,omitempty"`
} }

View File

@@ -9,5 +9,5 @@ type User struct {
// Relations // Relations
Webhooks []Webhook `json:"webhooks,omitempty"` Webhooks []Webhook `json:"webhooks,omitempty"`
APIKeys []APIKey `json:"api_keys,omitempty"` APIKeys []APIKey `json:"apiKeys,omitempty"`
} }

View File

@@ -1,16 +1,125 @@
package database package database
import (
"math"
"strconv"
"time"
"gorm.io/gorm"
)
const (
// DefaultRetentionDays is the event retention period applied to a
// webhook created without an explicit retention value. It is the
// single source of truth for that policy and must stay in sync
// with the `gorm:"default:30"` column default on
// Webhook.RetentionDays below; a struct tag cannot reference a
// constant, so a test asserts the two agree.
DefaultRetentionDays = 30
// RetentionForeverDays is the sentinel RetentionDays value meaning
// "retain events forever". Users express that intent as 0, which
// Webhook.BeforeSave rewrites to this value: the column default
// substitutes DefaultRetentionDays for a zero value at insert
// time, so a zero can never survive a round trip to the database.
// Nothing outside this file may hardcode the number.
RetentionForeverDays = 365 * 1000
// MaxFiniteRetentionDays is the largest finite retention period the
// reaper's cutoff arithmetic can represent, and therefore the
// largest one a caller may request. It is derived from that
// arithmetic rather than picked: retentionCutoff computes
// retentionDays * hoursPerDay * time.Hour, and a time.Duration is
// an int64 nanosecond count, so math.MaxInt64 nanoseconds divided
// by an hour and then by a day is the exact ceiling — 106751 days,
// a little over 292 years.
//
// One day more overflows int64, wraps the product negative, and
// turns the cutoff into a timestamp in the far future that matches
// every row in the webhook's database. That is why this bound is
// enforced on input and why retentionCutoff saturates underneath
// it. Note that RetentionForeverDays deliberately sits above this
// ceiling: such webhooks are skipped before any cutoff is
// computed, and never reach the arithmetic at all.
MaxFiniteRetentionDays = int(
math.MaxInt64 / int64(time.Hour) / hoursPerDay,
)
)
// Webhook represents a webhook processing unit that groups entrypoints and targets // Webhook represents a webhook processing unit that groups entrypoints and targets
//
// Every method below takes a pointer receiver. BeforeSave has to,
// because it mutates the record and GORM only invokes hooks declared
// that way; the display helpers follow suit so the receiver kinds do
// not mix. Handlers therefore put a *Webhook into template data:
// html/template cannot call a pointer method on a value held in a map,
// because a map element is not addressable.
type Webhook struct { type Webhook struct {
BaseModel BaseModel
UserID string `gorm:"type:uuid;not null" json:"user_id"` UserID string `gorm:"type:uuid;not null" json:"userId"`
Name string `gorm:"not null" json:"name"` Name string `gorm:"not null" json:"name"`
Description string `json:"description"` Description string `json:"description"`
RetentionDays int `gorm:"default:30" json:"retention_days"` // Days to retain events
// RetentionDays is the number of days to retain events. A value of
// RetentionForeverDays means retain forever. The column default
// must equal DefaultRetentionDays.
RetentionDays int `gorm:"default:30" json:"retentionDays"`
// Relations // Relations
User User `json:"user,omitempty"` User User `json:"user,omitzero"`
Entrypoints []Entrypoint `json:"entrypoints,omitempty"` Entrypoints []Entrypoint `json:"entrypoints,omitempty"`
Targets []Target `json:"targets,omitempty"` Targets []Target `json:"targets,omitempty"`
} }
// BeforeSave normalises RetentionDays on every insert and update. A
// non-positive value is the user's way of asking for "retain forever",
// which is stored as the RetentionForeverDays sentinel.
//
// This has to happen in a hook rather than at the call sites. GORM
// substitutes the column default (DefaultRetentionDays) for a zero
// value while building the insert statement, which runs after
// BeforeSave; rewriting any later than this loses that race and the
// row lands at 30 days. Living on the model also means a future call
// site — a REST API, a fixture, a migration — cannot bypass it.
func (w *Webhook) BeforeSave(_ *gorm.DB) error {
if w.RetentionDays <= 0 {
w.RetentionDays = RetentionForeverDays
}
return nil
}
// retainsForever reports whether a stored RetentionDays value means
// "keep events indefinitely". It is the single definition of that
// question, shared by Webhook.RetainsForever and by the reaper's
// cutoff computation so the two cannot disagree about which webhooks
// are exempt from reaping.
//
// It accepts the RetentionForeverDays sentinel written by BeforeSave
// and, defensively, the non-positive values that rows written before
// the sentinel existed may still carry.
func retainsForever(retentionDays int) bool {
return retentionDays <= 0 ||
retentionDays >= RetentionForeverDays
}
// RetainsForever reports whether this webhook's events are kept
// indefinitely.
func (w *Webhook) RetainsForever() bool {
return retainsForever(w.RetentionDays)
}
// RetentionLabel returns the webhook's retention policy as display
// text, so that no template has to know about the sentinel value.
func (w *Webhook) RetentionLabel() string {
if w.RetainsForever() {
return "forever"
}
if w.RetentionDays == 1 {
return "1 day"
}
return strconv.Itoa(w.RetentionDays) + " days"
}

View File

@@ -0,0 +1,222 @@
package database_test
import (
"context"
"reflect"
"strconv"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
)
// startedTestDB returns a started main database for model-level tests.
func startedTestDB(t *testing.T) *gorm.DB {
t.Helper()
db, lc := setupTestDB(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
t.Cleanup(func() { require.NoError(t, lc.Stop(ctx)) })
return db.DB()
}
// storedRetention reads the retention_days column straight out of the
// row, so the assertion is about what was persisted rather than about
// whatever the in-memory struct happens to hold.
func storedRetention(t *testing.T, db *gorm.DB, id string) int {
t.Helper()
var got int
require.NoError(
t,
db.Model(&database.Webhook{}).
Where("id = ?", id).
Pluck("retention_days", &got).Error,
)
return got
}
// newWebhookWithRetention creates a webhook through the ordinary Create
// path, so the BeforeSave hook and the GORM column default both apply
// exactly as they do in production.
func newWebhookWithRetention(
t *testing.T,
db *gorm.DB,
wh *database.Webhook,
) string {
t.Helper()
wh.UserID = uuid.New().String()
wh.Name = testWebhookName
require.NoError(
t,
db.Omit(clause.Associations).Create(wh).Error,
)
return wh.ID
}
func TestWebhookBeforeSave_ZeroBecomesForeverSentinel(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
wh := &database.Webhook{RetentionDays: 0}
id := newWebhookWithRetention(t, db, wh)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetention(t, db, id),
"a zero retention must be stored as the sentinel, "+
"not replaced by the column default",
)
}
func TestWebhookBeforeSave_NegativeBecomesForeverSentinel(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
wh := &database.Webhook{RetentionDays: -5}
id := newWebhookWithRetention(t, db, wh)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetention(t, db, id),
)
}
func TestWebhookBeforeSave_PositiveIsPreserved(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
wh := &database.Webhook{RetentionDays: 7}
id := newWebhookWithRetention(t, db, wh)
assert.Equal(t, 7, storedRetention(t, db, id))
}
// TestWebhookBeforeSave_UpdateToZeroBecomesSentinel proves the hook
// fires on update as well as insert, via the same Save call the edit
// handler makes.
func TestWebhookBeforeSave_UpdateToZeroBecomesSentinel(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
wh := &database.Webhook{RetentionDays: 30}
id := newWebhookWithRetention(t, db, wh)
require.Equal(t, 30, storedRetention(t, db, id))
wh.RetentionDays = 0
require.NoError(t, db.Omit(clause.Associations).Save(wh).Error)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetention(t, db, id),
)
}
// TestWebhookRetentionColumnDefaultMatchesConstant guards the one place
// the default lives twice: a struct tag cannot reference a constant, so
// this asserts the tag and DefaultRetentionDays agree.
func TestWebhookRetentionColumnDefaultMatchesConstant(t *testing.T) {
t.Parallel()
field, ok := reflect.TypeFor[database.Webhook]().
FieldByName("RetentionDays")
require.True(t, ok, "Webhook.RetentionDays must exist")
assert.Equal(
t,
"default:"+strconv.Itoa(database.DefaultRetentionDays),
field.Tag.Get("gorm"),
)
}
// TestMaxFiniteRetentionDaysIsTheOverflowCeiling asserts that the
// constant is exactly where the cutoff arithmetic stops working, which
// is what makes it a derived bound rather than a round number someone
// liked. One day more wraps the int64 nanosecond count negative, and a
// negative span is precisely what turned a cutoff into a future
// timestamp that matched — and deleted — every row.
//
// The multiplications are done through variables on purpose: as
// constant expressions the overflowing one would not compile.
func TestMaxFiniteRetentionDaysIsTheOverflowCeiling(t *testing.T) {
t.Parallel()
const hoursPerDay = 24
atCeiling := database.MaxFiniteRetentionDays
overCeiling := database.MaxFiniteRetentionDays + 1
assert.Positive(
t,
time.Duration(atCeiling*hoursPerDay)*time.Hour,
"the ceiling itself must still be representable",
)
assert.Negative(
t,
time.Duration(overCeiling*hoursPerDay)*time.Hour,
"one day past the ceiling must overflow",
)
assert.Less(
t,
database.MaxFiniteRetentionDays,
database.RetentionForeverDays,
"the sentinel sits above the ceiling and is only safe "+
"because retain-forever webhooks skip the arithmetic",
)
}
func TestWebhookRetainsForeverAndLabel(t *testing.T) {
t.Parallel()
cases := []struct {
name string
days int
forever bool
label string
}{
{
"sentinel",
database.RetentionForeverDays, true, testForeverLabel,
},
{
"above sentinel",
database.RetentionForeverDays + 1, true, testForeverLabel,
},
{"legacy zero", 0, true, testForeverLabel},
{"legacy negative", -1, true, testForeverLabel},
{"default", database.DefaultRetentionDays, false, "30 days"},
{"one day", 1, false, "1 day"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
wh := database.Webhook{RetentionDays: tc.days}
assert.Equal(t, tc.forever, wh.RetainsForever())
assert.Equal(t, tc.label, wh.RetentionLabel())
})
}
}

View File

@@ -1,15 +1,16 @@
package database package database
// Migrate runs database migrations for all models // Migrate runs database migrations for the main application database.
// Only configuration-tier models are stored in the main database.
// Event-tier models (Event, Delivery, DeliveryResult) live in
// per-webhook dedicated databases managed by WebhookDBManager.
func (d *Database) Migrate() error { func (d *Database) Migrate() error {
return d.db.AutoMigrate( return d.db.AutoMigrate(
&Setting{},
&User{}, &User{},
&APIKey{}, &APIKey{},
&Webhook{}, &Webhook{},
&Entrypoint{}, &Entrypoint{},
&Target{}, &Target{},
&Event{},
&Delivery{},
&DeliveryResult{},
) )
} }

View File

@@ -4,6 +4,7 @@ import (
"crypto/rand" "crypto/rand"
"crypto/subtle" "crypto/subtle"
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"math/big" "math/big"
"strings" "strings"
@@ -20,6 +21,23 @@ const (
argon2SaltLen = 16 argon2SaltLen = 16
) )
// hashParts is the expected number of $-separated segments
// in an encoded Argon2id hash string.
const hashParts = 6
// minPasswordComplexityLen is the minimum password length that
// triggers per-character-class complexity enforcement.
const minPasswordComplexityLen = 4
// Sentinel errors returned by decodeHash.
var (
errInvalidHashFormat = errors.New("invalid hash format")
errInvalidAlgorithm = errors.New("invalid algorithm")
errIncompatibleVersion = errors.New("incompatible argon2 version")
errSaltLengthOutOfRange = errors.New("salt length out of range")
errHashLengthOutOfRange = errors.New("hash length out of range")
)
// PasswordConfig holds Argon2 configuration // PasswordConfig holds Argon2 configuration
type PasswordConfig struct { type PasswordConfig struct {
Time uint32 Time uint32
@@ -46,26 +64,44 @@ func HashPassword(password string) (string, error) {
// Generate a salt // Generate a salt
salt := make([]byte, config.SaltLen) salt := make([]byte, config.SaltLen)
if _, err := rand.Read(salt); err != nil {
_, err := rand.Read(salt)
if err != nil {
return "", err return "", err
} }
// Generate the hash // Generate the hash
hash := argon2.IDKey([]byte(password), salt, config.Time, config.Memory, config.Threads, config.KeyLen) hash := argon2.IDKey(
[]byte(password),
salt,
config.Time,
config.Memory,
config.Threads,
config.KeyLen,
)
// Encode the hash and parameters // Encode the hash and parameters
b64Salt := base64.RawStdEncoding.EncodeToString(salt) b64Salt := base64.RawStdEncoding.EncodeToString(salt)
b64Hash := base64.RawStdEncoding.EncodeToString(hash) b64Hash := base64.RawStdEncoding.EncodeToString(hash)
// Format: $argon2id$v=19$m=65536,t=1,p=4$salt$hash // Format: $argon2id$v=19$m=65536,t=1,p=4$salt$hash
encoded := fmt.Sprintf("$argon2id$v=%d$m=%d,t=%d,p=%d$%s$%s", encoded := fmt.Sprintf(
argon2.Version, config.Memory, config.Time, config.Threads, b64Salt, b64Hash) "$argon2id$v=%d$m=%d,t=%d,p=%d$%s$%s",
argon2.Version,
config.Memory,
config.Time,
config.Threads,
b64Salt,
b64Hash,
)
return encoded, nil return encoded, nil
} }
// VerifyPassword checks if the provided password matches the hash // VerifyPassword checks if the provided password matches the hash
func VerifyPassword(password, encodedHash string) (bool, error) { func VerifyPassword(
password, encodedHash string,
) (bool, error) {
// Extract parameters and hash from encoded string // Extract parameters and hash from encoded string
config, salt, hash, err := decodeHash(encodedHash) config, salt, hash, err := decodeHash(encodedHash)
if err != nil { if err != nil {
@@ -73,60 +109,119 @@ func VerifyPassword(password, encodedHash string) (bool, error) {
} }
// Generate hash of the provided password // Generate hash of the provided password
otherHash := argon2.IDKey([]byte(password), salt, config.Time, config.Memory, config.Threads, config.KeyLen) otherHash := argon2.IDKey(
[]byte(password),
salt,
config.Time,
config.Memory,
config.Threads,
config.KeyLen,
)
// Compare hashes using constant time comparison // Compare hashes using constant time comparison
return subtle.ConstantTimeCompare(hash, otherHash) == 1, nil return subtle.ConstantTimeCompare(hash, otherHash) == 1, nil
} }
// decodeHash extracts parameters, salt, and hash from an encoded hash string // decodeHash extracts parameters, salt, and hash from an
func decodeHash(encodedHash string) (*PasswordConfig, []byte, []byte, error) { // encoded hash string.
func decodeHash(
encodedHash string,
) (*PasswordConfig, []byte, []byte, error) {
parts := strings.Split(encodedHash, "$") parts := strings.Split(encodedHash, "$")
if len(parts) != 6 { if len(parts) != hashParts {
return nil, nil, nil, fmt.Errorf("invalid hash format") return nil, nil, nil, errInvalidHashFormat
} }
if parts[1] != "argon2id" { if parts[1] != "argon2id" {
return nil, nil, nil, fmt.Errorf("invalid algorithm") return nil, nil, nil, errInvalidAlgorithm
} }
var version int version, err := parseVersion(parts[2])
if _, err := fmt.Sscanf(parts[2], "v=%d", &version); err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, err
} }
if version != argon2.Version { if version != argon2.Version {
return nil, nil, nil, fmt.Errorf("incompatible argon2 version") return nil, nil, nil, errIncompatibleVersion
} }
config := &PasswordConfig{} config, err := parseParams(parts[3])
if _, err := fmt.Sscanf(parts[3], "m=%d,t=%d,p=%d", &config.Memory, &config.Time, &config.Threads); err != nil {
return nil, nil, nil, err
}
salt, err := base64.RawStdEncoding.DecodeString(parts[4])
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, err
} }
saltLen := len(salt)
if saltLen < 0 || saltLen > int(^uint32(0)) {
return nil, nil, nil, fmt.Errorf("salt length out of range")
}
config.SaltLen = uint32(saltLen) // nolint:gosec // checked above
hash, err := base64.RawStdEncoding.DecodeString(parts[5]) salt, err := decodeSalt(parts[4])
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, err
} }
hashLen := len(hash)
if hashLen < 0 || hashLen > int(^uint32(0)) { config.SaltLen = uint32(len(salt)) //nolint:gosec // validated in decodeSalt
return nil, nil, nil, fmt.Errorf("hash length out of range")
hash, err := decodeHashBytes(parts[5])
if err != nil {
return nil, nil, nil, err
} }
config.KeyLen = uint32(hashLen) // nolint:gosec // checked above
config.KeyLen = uint32(len(hash)) //nolint:gosec // validated in decodeHashBytes
return config, salt, hash, nil return config, salt, hash, nil
} }
// GenerateRandomPassword generates a cryptographically secure random password func parseVersion(s string) (int, error) {
var version int
_, err := fmt.Sscanf(s, "v=%d", &version)
if err != nil {
return 0, fmt.Errorf("parsing version: %w", err)
}
return version, nil
}
func parseParams(s string) (*PasswordConfig, error) {
config := &PasswordConfig{}
_, err := fmt.Sscanf(
s, "m=%d,t=%d,p=%d",
&config.Memory, &config.Time, &config.Threads,
)
if err != nil {
return nil, fmt.Errorf("parsing params: %w", err)
}
return config, nil
}
func decodeSalt(s string) ([]byte, error) {
salt, err := base64.RawStdEncoding.DecodeString(s)
if err != nil {
return nil, fmt.Errorf("decoding salt: %w", err)
}
saltLen := len(salt)
if saltLen < 0 || saltLen > int(^uint32(0)) {
return nil, errSaltLengthOutOfRange
}
return salt, nil
}
func decodeHashBytes(s string) ([]byte, error) {
hash, err := base64.RawStdEncoding.DecodeString(s)
if err != nil {
return nil, fmt.Errorf("decoding hash: %w", err)
}
hashLen := len(hash)
if hashLen < 0 || hashLen > int(^uint32(0)) {
return nil, errHashLengthOutOfRange
}
return hash, nil
}
// GenerateRandomPassword generates a cryptographically secure
// random password.
func GenerateRandomPassword(length int) (string, error) { func GenerateRandomPassword(length int) (string, error) {
const ( const (
uppercase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" uppercase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
@@ -141,27 +236,27 @@ func GenerateRandomPassword(length int) (string, error) {
// Create password slice // Create password slice
password := make([]byte, length) password := make([]byte, length)
// Ensure at least one character from each set for password complexity // Ensure at least one character from each set
if length >= 4 { if length >= minPasswordComplexityLen {
// Get one character from each set
password[0] = uppercase[cryptoRandInt(len(uppercase))] password[0] = uppercase[cryptoRandInt(len(uppercase))]
password[1] = lowercase[cryptoRandInt(len(lowercase))] password[1] = lowercase[cryptoRandInt(len(lowercase))]
password[2] = digits[cryptoRandInt(len(digits))] password[2] = digits[cryptoRandInt(len(digits))]
password[3] = special[cryptoRandInt(len(special))] password[3] = special[cryptoRandInt(len(special))]
// Fill the rest randomly from all characters // Fill the rest randomly from all characters
for i := 4; i < length; i++ { for i := minPasswordComplexityLen; i < length; i++ {
password[i] = allChars[cryptoRandInt(len(allChars))] password[i] = allChars[cryptoRandInt(len(allChars))]
} }
// Shuffle the password to avoid predictable pattern // Shuffle the password to avoid predictable pattern
for i := len(password) - 1; i > 0; i-- { for i := range len(password) - 1 {
j := cryptoRandInt(i + 1) j := cryptoRandInt(len(password) - i)
password[i], password[j] = password[j], password[i] idx := len(password) - 1 - i
password[idx], password[j] = password[j], password[idx]
} }
} else { } else {
// For very short passwords, just use all characters // For very short passwords, just use all characters
for i := 0; i < length; i++ { for i := range length {
password[i] = allChars[cryptoRandInt(len(allChars))] password[i] = allChars[cryptoRandInt(len(allChars))]
} }
} }
@@ -169,16 +264,17 @@ func GenerateRandomPassword(length int) (string, error) {
return string(password), nil return string(password), nil
} }
// cryptoRandInt generates a cryptographically secure random integer in [0, max) // cryptoRandInt generates a cryptographically secure random
func cryptoRandInt(max int) int { // integer in [0, upperBound).
if max <= 0 { func cryptoRandInt(upperBound int) int {
panic("max must be positive") if upperBound <= 0 {
panic("upperBound must be positive")
} }
// Calculate the maximum valid value to avoid modulo bias nBig, err := rand.Int(
// For example, if max=200 and we have 256 possible values, rand.Reader,
// we only accept values 0-199 (reject 200-255) big.NewInt(int64(upperBound)),
nBig, err := rand.Int(rand.Reader, big.NewInt(int64(max))) )
if err != nil { if err != nil {
panic(fmt.Sprintf("crypto/rand error: %v", err)) panic(fmt.Sprintf("crypto/rand error: %v", err))
} }

View File

@@ -1,11 +1,15 @@
package database package database_test
import ( import (
"strings" "strings"
"testing" "testing"
"sneak.berlin/go/webhooker/internal/database"
) )
func TestGenerateRandomPassword(t *testing.T) { func TestGenerateRandomPassword(t *testing.T) {
t.Parallel()
tests := []struct { tests := []struct {
name string name string
length int length int
@@ -18,109 +22,172 @@ func TestGenerateRandomPassword(t *testing.T) {
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
password, err := GenerateRandomPassword(tt.length) t.Parallel()
password, err := database.GenerateRandomPassword(
tt.length,
)
if err != nil { if err != nil {
t.Fatalf("GenerateRandomPassword() error = %v", err) t.Fatalf(
"GenerateRandomPassword() error = %v",
err,
)
} }
if len(password) != tt.length { if len(password) != tt.length {
t.Errorf("Password length = %v, want %v", len(password), tt.length) t.Errorf(
"Password length = %v, want %v",
len(password), tt.length,
)
} }
// For passwords >= 4 chars, check complexity checkPasswordComplexity(
if tt.length >= 4 { t, password, tt.length,
hasUpper := false )
hasLower := false
hasDigit := false
hasSpecial := false
for _, char := range password {
switch {
case char >= 'A' && char <= 'Z':
hasUpper = true
case char >= 'a' && char <= 'z':
hasLower = true
case char >= '0' && char <= '9':
hasDigit = true
case strings.ContainsRune("!@#$%^&*()_+-=[]{}|;:,.<>?", char):
hasSpecial = true
}
}
if !hasUpper || !hasLower || !hasDigit || !hasSpecial {
t.Errorf("Password lacks required complexity: upper=%v, lower=%v, digit=%v, special=%v",
hasUpper, hasLower, hasDigit, hasSpecial)
}
}
}) })
} }
} }
func checkPasswordComplexity(
t *testing.T,
password string,
length int,
) {
t.Helper()
// For passwords >= 4 chars, check complexity
if length < 4 {
return
}
flags := classifyChars(password)
if !flags[0] || !flags[1] || !flags[2] || !flags[3] {
t.Errorf(
"Password lacks required complexity: "+
"upper=%v, lower=%v, digit=%v, special=%v",
flags[0], flags[1], flags[2], flags[3],
)
}
}
func classifyChars(s string) [4]bool {
var flags [4]bool // upper, lower, digit, special
for _, char := range s {
switch {
case char >= 'A' && char <= 'Z':
flags[0] = true
case char >= 'a' && char <= 'z':
flags[1] = true
case char >= '0' && char <= '9':
flags[2] = true
case strings.ContainsRune(
"!@#$%^&*()_+-=[]{}|;:,.<>?",
char,
):
flags[3] = true
}
}
return flags
}
func TestGenerateRandomPasswordUniqueness(t *testing.T) { func TestGenerateRandomPasswordUniqueness(t *testing.T) {
t.Parallel()
// Generate multiple passwords and ensure they're different // Generate multiple passwords and ensure they're different
passwords := make(map[string]bool) passwords := make(map[string]bool)
const numPasswords = 100 const numPasswords = 100
for i := 0; i < numPasswords; i++ { for range numPasswords {
password, err := GenerateRandomPassword(16) password, err := database.GenerateRandomPassword(16)
if err != nil { if err != nil {
t.Fatalf("GenerateRandomPassword() error = %v", err) t.Fatalf(
"GenerateRandomPassword() error = %v",
err,
)
} }
if passwords[password] { if passwords[password] {
t.Errorf("Duplicate password generated: %s", password) t.Errorf(
"Duplicate password generated: %s",
password,
)
} }
passwords[password] = true passwords[password] = true
} }
} }
func TestHashPassword(t *testing.T) { func TestHashPassword(t *testing.T) {
t.Parallel()
password := "testPassword123!" password := "testPassword123!"
hash, err := HashPassword(password) hash, err := database.HashPassword(password)
if err != nil { if err != nil {
t.Fatalf("HashPassword() error = %v", err) t.Fatalf("HashPassword() error = %v", err)
} }
// Check that hash has correct format // Check that hash has correct format
if !strings.HasPrefix(hash, "$argon2id$") { if !strings.HasPrefix(hash, "$argon2id$") {
t.Errorf("Hash doesn't have correct prefix: %s", hash) t.Errorf(
"Hash doesn't have correct prefix: %s",
hash,
)
} }
// Verify password // Verify password
valid, err := VerifyPassword(password, hash) valid, err := database.VerifyPassword(password, hash)
if err != nil { if err != nil {
t.Fatalf("VerifyPassword() error = %v", err) t.Fatalf("VerifyPassword() error = %v", err)
} }
if !valid { if !valid {
t.Error("VerifyPassword() returned false for correct password") t.Error(
"VerifyPassword() returned false " +
"for correct password",
)
} }
// Verify wrong password fails // Verify wrong password fails
valid, err = VerifyPassword("wrongPassword", hash) valid, err = database.VerifyPassword(
"wrongPassword", hash,
)
if err != nil { if err != nil {
t.Fatalf("VerifyPassword() error = %v", err) t.Fatalf("VerifyPassword() error = %v", err)
} }
if valid { if valid {
t.Error("VerifyPassword() returned true for wrong password") t.Error(
"VerifyPassword() returned true " +
"for wrong password",
)
} }
} }
func TestHashPasswordUniqueness(t *testing.T) { func TestHashPasswordUniqueness(t *testing.T) {
t.Parallel()
password := "testPassword123!" password := "testPassword123!"
// Same password should produce different hashes due to salt // Same password should produce different hashes
hash1, err := HashPassword(password) hash1, err := database.HashPassword(password)
if err != nil { if err != nil {
t.Fatalf("HashPassword() error = %v", err) t.Fatalf("HashPassword() error = %v", err)
} }
hash2, err := HashPassword(password) hash2, err := database.HashPassword(password)
if err != nil { if err != nil {
t.Fatalf("HashPassword() error = %v", err) t.Fatalf("HashPassword() error = %v", err)
} }
if hash1 == hash2 { if hash1 == hash2 {
t.Error("Same password produced identical hashes (salt not working)") t.Error(
"Same password produced identical hashes " +
"(salt not working)",
)
} }
} }

View File

@@ -0,0 +1,321 @@
package database
import (
"context"
"fmt"
"log/slog"
"sync"
"time"
"go.uber.org/fx"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/lifecycle"
"sneak.berlin/go/webhooker/internal/logger"
)
// hoursPerDay converts a RetentionDays count into hours for cutoff
// computation.
const hoursPerDay = 24
// RetentionReaperParams holds the fx dependencies for the
// RetentionReaper.
type RetentionReaperParams struct {
fx.In
Config *config.Config
Database *Database
DBManager *WebhookDBManager
Logger *logger.Logger
}
// RetentionReaper periodically deletes expired events (and their
// dependent deliveries and delivery results) from each per-webhook
// database, enforcing every webhook's RetentionDays. Rows are removed
// permanently so that per-webhook SQLite files do not grow without
// bound.
type RetentionReaper struct {
db *Database
dbManager *WebhookDBManager
log *slog.Logger
interval time.Duration
cancel context.CancelFunc
wg sync.WaitGroup
}
// NewRetentionReaper creates the retention reaper and registers its
// fx lifecycle hooks. The background sweep loop starts on OnStart and
// stops cleanly on OnStop via context cancellation.
func NewRetentionReaper(
lc fx.Lifecycle,
params RetentionReaperParams,
) *RetentionReaper {
r := &RetentionReaper{
db: params.Database,
dbManager: params.DBManager,
log: params.Logger.Get(),
interval: params.Config.RetentionSweepInterval,
}
r.registerHooks(lc)
return r
}
// registerHooks wires the reaper's start and stop into the fx
// lifecycle. The start hook's context is deliberately ignored (see
// start for why the sweep loop must not inherit it); the stop hook's
// context is honoured (see stop).
func (r *RetentionReaper) registerHooks(lc fx.Lifecycle) {
lc.Append(fx.Hook{
//nolint:contextcheck // Not inheriting the hook context is
// the point: see start.
OnStart: func(_ context.Context) error {
r.start()
return nil
},
OnStop: func(ctx context.Context) error {
return r.stop(ctx)
},
})
}
// start launches the background sweep loop.
//
// The loop's context is derived from context.Background(), NOT from
// the fx OnStart hook context. The hook context carries fx's start
// timeout (15s by default) and is cancelled once the start phase
// completes, so a loop derived from it dies 45 minutes before its
// first tick under the default one-hour sweep interval, leaving a
// reaper that never reaps. A long-lived goroutine must outlive the
// startup phase, so its lifetime is bounded by OnStop instead: stop
// cancels this context and waits on the WaitGroup.
func (r *RetentionReaper) start() {
ctx, cancel := context.WithCancel(context.Background())
r.cancel = cancel
r.wg.Add(1)
go r.run(ctx)
r.log.Info(
"retention reaper started",
"interval", r.interval.String(),
)
}
// stop cancels the sweep loop's context and waits for it to
// exit, bounded by the stop hook's context: a sweep wedged on a
// locked database must not hang the process past fx's stop
// timeout.
func (r *RetentionReaper) stop(ctx context.Context) error {
r.log.Info("retention reaper stopping")
if r.cancel != nil {
r.cancel()
}
err := lifecycle.WaitForShutdown(
ctx, r.log, "retention reaper", &r.wg,
)
if err != nil {
return err
}
r.log.Info("retention reaper stopped")
return nil
}
func (r *RetentionReaper) run(ctx context.Context) {
defer r.wg.Done()
ticker := time.NewTicker(r.interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
r.sweep(ctx)
}
}
}
// sweep lists every webhook from the main database and reaps expired
// rows from each per-webhook database that has a finite retention
// policy. Webhooks set to retain forever are skipped entirely.
func (r *RetentionReaper) sweep(ctx context.Context) {
var webhooks []Webhook
err := r.db.DB().
Model(&Webhook{}).
Find(&webhooks).Error
if err != nil {
r.log.Error(
"retention sweep: failed to list webhooks",
"error", err,
)
return
}
for i := range webhooks {
select {
case <-ctx.Done():
return
default:
}
wh := webhooks[i]
// Skip retain-forever webhooks before building any query.
// RetainsForever covers both the RetentionForeverDays
// sentinel and the non-positive values that predate it: the
// sentinel is a positive number, so without this the reaper
// would compute a cutoff a thousand years in the past and
// issue a DELETE matching nothing on every single sweep.
if wh.RetainsForever() {
continue
}
// Nothing to reap if the per-webhook database has never
// been created.
if !r.dbManager.DBExists(wh.ID) {
continue
}
r.reapWebhook(wh.ID, wh.RetentionDays)
}
}
// reapWebhook removes every expired event (and its dependents) from a
// single webhook's database.
func (r *RetentionReaper) reapWebhook(
webhookID string,
retentionDays int,
) {
db, err := r.dbManager.GetDB(webhookID)
if err != nil {
r.log.Error(
"retention sweep: failed to open webhook database",
"webhook_id", webhookID,
"error", err,
)
return
}
cutoff, ok := retentionCutoff(time.Now(), retentionDays)
if !ok {
return
}
deleted, err := reapExpired(db, cutoff)
if err != nil {
r.log.Error(
"retention sweep: failed to reap expired events",
"webhook_id", webhookID,
"error", err,
)
return
}
if deleted > 0 {
r.log.Info(
"retention sweep: reaped expired events",
"webhook_id", webhookID,
"retention_days", retentionDays,
"events_deleted", deleted,
)
}
}
// retentionCutoff returns the timestamp before which a webhook's
// events have expired, and whether any cutoff applies at all. It
// reports false for a retain-forever policy, so no DELETE is issued.
//
// The day count is clamped to MaxFiniteRetentionDays first. This is
// defense in depth rather than decoration: a time.Duration is an int64
// nanosecond count, so an unclamped multiplication overflows above
// that ceiling and wraps the span negative. Subtracting a negative
// span moves the cutoff into the far future, where it matches every
// row in the database: the sweep then deletes every event, delivery,
// and delivery result, including ones created seconds ago. Rejecting
// out-of-range input at the form is the primary guard; saturating here
// means an old row, a migration, or a future call site cannot turn a
// too-large retention into total data loss.
func retentionCutoff(
now time.Time,
retentionDays int,
) (time.Time, bool) {
if retainsForever(retentionDays) {
return time.Time{}, false
}
if retentionDays > MaxFiniteRetentionDays {
retentionDays = MaxFiniteRetentionDays
}
return now.Add(
-time.Duration(retentionDays*hoursPerDay) * time.Hour,
), true
}
// reapExpired hard-deletes, in foreign-key-safe order, the delivery
// results, deliveries, and events associated with events older than
// cutoff. Deletes are unscoped so rows are physically removed rather
// than soft-deleted, reclaiming disk. It returns the number of events
// deleted.
func reapExpired(db *gorm.DB, cutoff time.Time) (int64, error) {
// Fresh subqueries are built per statement to avoid reusing a
// mutated builder across executions.
expiredEventIDs := func() *gorm.DB {
return db.Model(&Event{}).
Select("id").
Where("created_at < ?", cutoff)
}
expiredDeliveryIDs := func() *gorm.DB {
return db.Model(&Delivery{}).
Select("id").
Where("event_id IN (?)", expiredEventIDs())
}
// 1. Delivery results whose delivery belongs to an expired event.
res := db.Unscoped().
Where("delivery_id IN (?)", expiredDeliveryIDs()).
Delete(&DeliveryResult{})
if res.Error != nil {
return 0, fmt.Errorf(
"deleting expired delivery results: %w",
res.Error,
)
}
// 2. Deliveries belonging to an expired event.
del := db.Unscoped().
Where("event_id IN (?)", expiredEventIDs()).
Delete(&Delivery{})
if del.Error != nil {
return 0, fmt.Errorf(
"deleting expired deliveries: %w",
del.Error,
)
}
// 3. The expired events themselves.
ev := db.Unscoped().
Where("created_at < ?", cutoff).
Delete(&Event{})
if ev.Error != nil {
return 0, fmt.Errorf(
"deleting expired events: %w",
ev.Error,
)
}
return ev.RowsAffected, nil
}

View File

@@ -0,0 +1,272 @@
package database_test
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
const (
// reaperTestInterval is the sweep interval a lifecycle test
// runs the reaper at, so a loop that survives startup produces
// an observable sweep quickly.
reaperTestInterval = 10 * time.Millisecond
// reaperStopTimeout bounds how long a lifecycle test waits for
// the reaper's OnStop hook to return before declaring the
// shutdown hung.
reaperStopTimeout = 10 * time.Second
// reaperTestRetentionDays is the retention policy the lifecycle
// tests give their webhook.
reaperTestRetentionDays = 30
// reaperWedgeStopTimeout is the stop timeout the wedged-shutdown
// test hands OnStop, standing in for fx's StopTimeout. The test
// asserts only that the hook returns at all, and allows it
// reaperStopTimeout — forty times this budget — to do so, so no
// assertion races the wall clock.
reaperWedgeStopTimeout = 250 * time.Millisecond
)
// recordingLifecycle is a minimal fx.Lifecycle that records the
// hooks a component registers, so a test can invoke the real
// OnStart/OnStop functions with a context of its choosing.
type recordingLifecycle struct {
hooks []fx.Hook
}
func (l *recordingLifecycle) Append(h fx.Hook) {
l.hooks = append(l.hooks, h)
}
// startReaperViaHook drives the genuine fx hooks the application
// registers for the reaper, handing OnStart a context that is
// already done. It returns the recorded lifecycle so the caller
// can drive OnStop too.
func startReaperViaHook(
t *testing.T, r *database.RetentionReaper,
) *recordingLifecycle {
t.Helper()
lc := &recordingLifecycle{}
r.ExportRegisterHooks(lc)
require.Len(t, lc.hooks, 1)
// fx hands OnStart a context carrying the application start
// timeout, and cancels it when the start phase ends. An
// already-cancelled context is that same defect taken to its
// limit, and unlike a plain context.Background() it actually
// distinguishes a correctly rooted loop from a broken one.
hookCtx, cancel := context.WithCancel(context.Background())
cancel()
require.NoError(t, lc.hooks[0].OnStart(hookCtx))
return lc
}
// eventGone reports whether an event row has been removed. It
// takes no *testing.T because it is polled from an
// assert.Eventually condition, which runs off the test goroutine
// where testify assertions must not be used.
func eventGone(db *gorm.DB, eventID string) bool {
var n int64
err := db.Unscoped().Model(&database.Event{}).
Where("id = ?", eventID).Count(&n).Error
if err != nil {
return false
}
return n == 0
}
// seedExpiredWebhook creates a webhook with a finite retention
// policy plus one long-expired event chain, and returns the
// webhook's database and the chain's event ID.
func seedExpiredWebhook(
t *testing.T, env *retentionTestEnv,
) (*gorm.DB, string) {
t.Helper()
webhookID := createWebhook(
t, env.mainDB.DB(), reaperTestRetentionDays,
)
db, err := env.mgr.GetDB(webhookID)
require.NoError(t, err)
chain := seedEventChain(
t, db, webhookID,
time.Now().Add(-365*24*time.Hour),
)
return db, chain.eventID
}
// TestRetentionReaper_LoopOutlivesStartHookContext is the
// regression test for a reaper that never reaped. fx calls
// OnStart with a context carrying the application's start timeout
// (15s by default) and cancels it when the start phase ends, so a
// sweep loop rooted in it is dead three quarters of an hour
// before its first tick under the default one-hour interval, and
// per-webhook event databases grow without bound exactly as they
// did before retention existed.
//
// Driving OnStart with an already-cancelled context is that
// defect taken to its limit: a loop that inherits the hook
// context never ticks once, while a correctly rooted loop keeps
// sweeping for as long as the process lives.
func TestRetentionReaper_LoopOutlivesStartHookContext(
t *testing.T,
) {
t.Parallel()
env := setupRetentionTest(t)
db, eventID := seedExpiredWebhook(t, env)
env.reaper.ExportSetInterval(reaperTestInterval)
lc := startReaperViaHook(t, env.reaper)
t.Cleanup(func() {
_ = lc.hooks[0].OnStop(context.Background())
})
assert.Eventually(
t,
func() bool { return eventGone(db, eventID) },
5*time.Second,
reaperTestInterval,
"the sweep loop must keep running after the start "+
"hook's context is done; it reaped nothing, so it "+
"inherited the hook context and died",
)
}
// TestRetentionReaper_StopHookStopsLoop proves the fix did not
// trade a startup bug for a shutdown hang: now that the sweep
// loop no longer observes the start hook's cancellation, OnStop
// is the only thing that can stop it, and it must both return
// promptly and actually leave the loop stopped.
func TestRetentionReaper_StopHookStopsLoop(t *testing.T) {
t.Parallel()
env := setupRetentionTest(t)
db, eventID := seedExpiredWebhook(t, env)
env.reaper.ExportSetInterval(reaperTestInterval)
lc := startReaperViaHook(t, env.reaper)
// Let the loop prove it is running before stopping it, so a
// fast OnStop cannot pass by stopping something already dead.
require.Eventually(
t,
func() bool { return eventGone(db, eventID) },
5*time.Second,
reaperTestInterval,
)
var stopErr error
stopped := make(chan struct{})
go func() {
defer close(stopped)
// stop blocks on the loop's WaitGroup, so returning at all
// proves the goroutine observed the cancellation.
stopErr = lc.hooks[0].OnStop(context.Background())
}()
select {
case <-stopped:
case <-time.After(reaperStopTimeout):
t.Fatal(
"OnStop did not return: the retention reaper's " +
"WaitGroup is still waiting on a loop that never " +
"observed cancellation",
)
}
require.NoError(t, stopErr)
// With the loop gone, a newly expired chain must survive.
survivor := seedEventChain(
t, db, "stopped-webhook",
time.Now().Add(-365*24*time.Hour),
)
time.Sleep(20 * reaperTestInterval)
assert.False(
t,
eventGone(db, survivor.eventID),
"a stopped reaper must not sweep anything",
)
}
// TestRetentionReaper_StopHookHonoursStopTimeout is the
// regression test for a shutdown that could never complete. fx
// hands OnStop a context carrying the application's stop timeout;
// an OnStop that discards it and calls wg.Wait() bare hangs the
// process forever on a sweep blocked on a locked SQLite database
// — precisely when a bounded shutdown matters most.
//
// The wedged goroutine here never observes cancellation, so the
// hook can only return by honouring its context, and it must say
// so rather than reporting a clean stop.
func TestRetentionReaper_StopHookHonoursStopTimeout(
t *testing.T,
) {
t.Parallel()
env := setupRetentionTest(t)
env.reaper.ExportSetInterval(reaperTestInterval)
lc := startReaperViaHook(t, env.reaper)
release := make(chan struct{})
t.Cleanup(func() { close(release) })
env.reaper.ExportWedgeLoop(release)
stopCtx, cancel := context.WithTimeout(
context.Background(), reaperWedgeStopTimeout,
)
defer cancel()
var stopErr error
stopped := make(chan struct{})
go func() {
defer close(stopped)
stopErr = lc.hooks[0].OnStop(stopCtx)
}()
select {
case <-stopped:
case <-time.After(reaperStopTimeout):
t.Fatal(
"OnStop did not return: it discarded the stop " +
"context and is waiting on a wedged goroutine " +
"that will never observe cancellation",
)
}
require.ErrorIs(t, stopErr, context.DeadlineExceeded)
require.ErrorContains(t, stopErr, "retention reaper")
}

View File

@@ -0,0 +1,402 @@
package database_test
import (
"context"
"net/http"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"gorm.io/gorm"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger"
)
// retentionTestEnv bundles the pieces a retention test drives.
type retentionTestEnv struct {
reaper *database.RetentionReaper
mainDB *database.Database
mgr *database.WebhookDBManager
}
func setupRetentionTest(t *testing.T) *retentionTestEnv {
t.Helper()
lc := fxtest.NewLifecycle(t)
g := &globals.Globals{
Appname: testAppname,
Version: testVersion,
}
l, err := logger.New(lc, logger.LoggerParams{Globals: g})
require.NoError(t, err)
cfg := &config.Config{
DataDir: t.TempDir(),
Environment: "dev",
}
mainDB, err := database.New(lc, database.DatabaseParams{
Config: cfg,
Logger: l,
})
require.NoError(t, err)
mgr, err := database.NewWebhookDBManager(
lc,
database.WebhookDBManagerParams{Config: cfg, Logger: l},
)
require.NoError(t, err)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
t.Cleanup(func() { require.NoError(t, lc.Stop(ctx)) })
return &retentionTestEnv{
reaper: database.NewTestRetentionReaper(mainDB, mgr),
mainDB: mainDB,
mgr: mgr,
}
}
// createWebhook inserts a webhook row into the main database with the
// given retention policy and returns its ID.
func createWebhook(
t *testing.T,
db *gorm.DB,
retentionDays int,
) string {
t.Helper()
wh := &database.Webhook{
UserID: uuid.New().String(),
Name: testWebhookName,
RetentionDays: retentionDays,
}
require.NoError(
t,
db.Omit(clause.Associations).Create(wh).Error,
)
// Webhook.BeforeSave rewrites a non-positive RetentionDays to the
// retain-forever sentinel, and the column's GORM default would
// otherwise substitute 30. Force the requested value with a
// column-level update so tests can plant legacy rows that predate
// the sentinel and still carry a literal 0 or negative value.
require.NoError(
t,
db.Model(wh).
Update("retention_days", retentionDays).Error,
)
return wh.ID
}
// createWebhookNormally inserts a webhook through the ordinary Create
// path, with no column-level forcing, so Webhook.BeforeSave applies
// exactly as it does in production. Passing 0 therefore yields a row
// holding the RetentionForeverDays sentinel.
func createWebhookNormally(
t *testing.T,
db *gorm.DB,
retentionDays int,
) string {
t.Helper()
wh := &database.Webhook{
UserID: uuid.New().String(),
Name: testWebhookName,
RetentionDays: retentionDays,
}
require.NoError(
t,
db.Omit(clause.Associations).Create(wh).Error,
)
return wh.ID
}
// eventChain is the set of row IDs seeded for a single event.
type eventChain struct {
eventID string
deliveryID string
resultID string
}
// seedEventChain creates an event with one delivery and one delivery
// result, all stamped with createdAt, and returns their IDs.
func seedEventChain(
t *testing.T,
db *gorm.DB,
webhookID string,
createdAt time.Time,
) eventChain {
t.Helper()
event := &database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: http.MethodPost,
Body: `{"seed": true}`,
ContentType: testContentType,
}
event.CreatedAt = createdAt
require.NoError(t, db.Create(event).Error)
delivery := &database.Delivery{
EventID: event.ID,
TargetID: uuid.New().String(),
Status: database.DeliveryStatusDelivered,
}
delivery.CreatedAt = createdAt
require.NoError(t, db.Create(delivery).Error)
result := &database.DeliveryResult{
DeliveryID: delivery.ID,
AttemptNum: 1,
Success: true,
StatusCode: 200,
Duration: 10,
}
result.CreatedAt = createdAt
require.NoError(t, db.Create(result).Error)
return eventChain{
eventID: event.ID,
deliveryID: delivery.ID,
resultID: result.ID,
}
}
// countByID returns how many rows of model match the given id,
// counting even hard-deletable rows via Unscoped.
func countByID(
t *testing.T,
db *gorm.DB,
model any,
id string,
) int64 {
t.Helper()
var n int64
require.NoError(
t,
db.Unscoped().Model(model).
Where("id = ?", id).Count(&n).Error,
)
return n
}
func assertChainGone(
t *testing.T,
db *gorm.DB,
chain eventChain,
) {
t.Helper()
assert.Zero(
t,
countByID(t, db, &database.Event{}, chain.eventID),
"expired event should be removed",
)
assert.Zero(
t,
countByID(t, db, &database.Delivery{}, chain.deliveryID),
"expired delivery should be removed",
)
assert.Zero(
t,
countByID(
t, db, &database.DeliveryResult{}, chain.resultID,
),
"expired delivery result should be removed",
)
}
func assertChainPresent(
t *testing.T,
db *gorm.DB,
chain eventChain,
) {
t.Helper()
assert.Equal(
t,
int64(1),
countByID(t, db, &database.Event{}, chain.eventID),
"recent event should be retained",
)
assert.Equal(
t,
int64(1),
countByID(t, db, &database.Delivery{}, chain.deliveryID),
"recent delivery should be retained",
)
assert.Equal(
t,
int64(1),
countByID(
t, db, &database.DeliveryResult{}, chain.resultID,
),
"recent delivery result should be retained",
)
}
func TestRetentionReaper_ReapsExpiredKeepsRecent(t *testing.T) {
t.Parallel()
env := setupRetentionTest(t)
const retentionDays = 30
webhookID := createWebhook(
t, env.mainDB.DB(), retentionDays,
)
db, err := env.mgr.GetDB(webhookID)
require.NoError(t, err)
now := time.Now()
old := seedEventChain(
t, db, webhookID,
now.Add(-40*24*time.Hour),
)
recent := seedEventChain(
t, db, webhookID,
now.Add(-1*24*time.Hour),
)
env.reaper.ExportSweep(context.Background())
assertChainGone(t, db, old)
assertChainPresent(t, db, recent)
}
// TestRetentionReaper_SkipsSentinelReapsFiniteInSameSweep covers the
// end-to-end retain-forever path: a webhook created the normal way with
// a requested retention of 0 lands on the RetentionForeverDays
// sentinel, and the reaper leaves its ancient events alone while still
// reaping a finite-retention webhook in the very same sweep.
func TestRetentionReaper_SkipsSentinelReapsFiniteInSameSweep(
t *testing.T,
) {
t.Parallel()
env := setupRetentionTest(t)
foreverID := createWebhookNormally(t, env.mainDB.DB(), 0)
var stored database.Webhook
require.NoError(
t,
env.mainDB.DB().Where("id = ?", foreverID).
First(&stored).Error,
)
require.Equal(
t,
database.RetentionForeverDays,
stored.RetentionDays,
"a requested retention of 0 must persist as the sentinel",
)
finiteID := createWebhookNormally(t, env.mainDB.DB(), 30)
foreverDB, err := env.mgr.GetDB(foreverID)
require.NoError(t, err)
finiteDB, err := env.mgr.GetDB(finiteID)
require.NoError(t, err)
ancient := time.Now().Add(-365 * 24 * time.Hour)
kept := seedEventChain(t, foreverDB, foreverID, ancient)
doomed := seedEventChain(t, finiteDB, finiteID, ancient)
env.reaper.ExportSweep(context.Background())
assertChainPresent(t, foreverDB, kept)
assertChainGone(t, finiteDB, doomed)
}
// TestRetentionReaper_HugeFiniteRetentionRetainsRecentEvents pins the
// overflow that made a large finite retention destroy everything.
//
// The cutoff is a time.Duration, an int64 nanosecond count. A day
// count above MaxFiniteRetentionDays multiplied out unclamped wraps
// negative, so subtracting it moves the cutoff into the far future,
// where "created_at < cutoff" matches every row: an event created a
// moment ago, and its delivery and delivery result, were all deleted
// on the first sweep. 200000 is inside that band and below the
// retain-forever sentinel, so it is treated as a finite policy and
// really does reach the arithmetic.
//
// The row is planted at the column level because such a value can no
// longer be submitted through the form; the point of the test is that
// a row from an older version, or a future call site, still cannot
// trigger the wipe.
func TestRetentionReaper_HugeFiniteRetentionRetainsRecentEvents(
t *testing.T,
) {
t.Parallel()
env := setupRetentionTest(t)
const overflowingRetentionDays = 200000
require.Greater(
t,
overflowingRetentionDays,
database.MaxFiniteRetentionDays,
"the test value must exceed what the cutoff can represent",
)
require.Less(
t,
overflowingRetentionDays,
database.RetentionForeverDays,
"the test value must not be rescued by the forever skip",
)
webhookID := createWebhook(
t, env.mainDB.DB(), overflowingRetentionDays,
)
db, err := env.mgr.GetDB(webhookID)
require.NoError(t, err)
fresh := seedEventChain(t, db, webhookID, time.Now())
env.reaper.ExportSweep(context.Background())
assertChainPresent(t, db, fresh)
}
func TestRetentionReaper_RetainsForeverWhenNonPositive(t *testing.T) {
t.Parallel()
env := setupRetentionTest(t)
// A legacy row written before the sentinel existed still carries a
// literal 0; the <= 0 guard must keep honouring it.
webhookID := createWebhook(t, env.mainDB.DB(), 0)
db, err := env.mgr.GetDB(webhookID)
require.NoError(t, err)
ancient := seedEventChain(
t, db, webhookID,
time.Now().Add(-365*24*time.Hour),
)
env.reaper.ExportSweep(context.Background())
assertChainPresent(t, db, ancient)
}

View File

@@ -0,0 +1,34 @@
package database
import (
"log/slog"
"os"
"gorm.io/gorm"
)
// NewTestDatabase creates a Database wrapper around a pre-opened *gorm.DB.
// Intended for use in tests that need a *database.Database without the
// full fx lifecycle. The caller is responsible for closing the underlying
// sql.DB connection.
func NewTestDatabase(db *gorm.DB) *Database {
return &Database{
db: db,
log: slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
)),
}
}
// NewTestWebhookDBManager creates a WebhookDBManager backed by the given
// data directory. Intended for use in tests without the fx lifecycle.
func NewTestWebhookDBManager(dataDir string) *WebhookDBManager {
return &WebhookDBManager{
dataDir: dataDir,
log: slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
)),
}
}

View File

@@ -0,0 +1,281 @@
package database
import (
"context"
"database/sql"
"errors"
"fmt"
"log/slog"
"os"
"path/filepath"
"sync"
"go.uber.org/fx"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/logger"
)
// WebhookDBManagerParams holds the fx dependencies for
// WebhookDBManager.
type WebhookDBManagerParams struct {
fx.In
Config *config.Config
Logger *logger.Logger
}
// errInvalidCachedDBType indicates a type assertion failure
// when retrieving a cached database connection.
var errInvalidCachedDBType = errors.New(
"invalid cached database type",
)
// WebhookDBManager manages per-webhook SQLite database files
// for event storage. Each webhook gets its own dedicated
// database containing Events, Deliveries, and DeliveryResults.
// Database connections are opened lazily and cached.
type WebhookDBManager struct {
dataDir string
dbs sync.Map // map[webhookID]*gorm.DB
log *slog.Logger
}
// NewWebhookDBManager creates a new WebhookDBManager and
// registers lifecycle hooks.
func NewWebhookDBManager(
lc fx.Lifecycle,
params WebhookDBManagerParams,
) (*WebhookDBManager, error) {
m := &WebhookDBManager{
dataDir: params.Config.DataDir,
log: params.Logger.Get(),
}
// Create data directory if it doesn't exist
err := os.MkdirAll(m.dataDir, dataDirPerm)
if err != nil {
return nil, fmt.Errorf(
"creating data directory %s: %w",
m.dataDir,
err,
)
}
lc.Append(fx.Hook{
OnStop: func(_ context.Context) error {
return m.CloseAll()
},
})
m.log.Info(
"webhook database manager initialized",
"data_dir", m.dataDir,
)
return m, nil
}
// GetDB returns the database connection for a webhook,
// creating the database file lazily if it doesn't exist.
func (m *WebhookDBManager) GetDB(
webhookID string,
) (*gorm.DB, error) {
// Fast path: already open
if val, ok := m.dbs.Load(webhookID); ok {
cachedDB, castOK := val.(*gorm.DB)
if !castOK {
return nil, fmt.Errorf(
"%w for webhook %s",
errInvalidCachedDBType,
webhookID,
)
}
return cachedDB, nil
}
// Slow path: open/create the database
db, err := m.openDB(webhookID)
if err != nil {
return nil, err
}
// Store it; if another goroutine beat us, close ours
actual, loaded := m.dbs.LoadOrStore(webhookID, db)
if loaded {
// Another goroutine created it first; close our duplicate
sqlDB, closeErr := db.DB()
if closeErr == nil {
_ = sqlDB.Close()
}
existingDB, castOK := actual.(*gorm.DB)
if !castOK {
return nil, fmt.Errorf(
"%w for webhook %s",
errInvalidCachedDBType,
webhookID,
)
}
return existingDB, nil
}
return db, nil
}
// CreateDB explicitly creates a new per-webhook database file
// and runs migrations.
func (m *WebhookDBManager) CreateDB(
webhookID string,
) error {
_, err := m.GetDB(webhookID)
return err
}
// DBExists checks if a per-webhook database file exists on
// disk.
func (m *WebhookDBManager) DBExists(
webhookID string,
) bool {
_, err := os.Stat(m.dbPath(webhookID))
return err == nil
}
// DeleteDB closes the connection and deletes the database file
// for a webhook. The file is permanently removed.
func (m *WebhookDBManager) DeleteDB(
webhookID string,
) error {
// Close and remove from cache
if val, ok := m.dbs.LoadAndDelete(webhookID); ok {
if gormDB, castOK := val.(*gorm.DB); castOK {
sqlDB, err := gormDB.DB()
if err == nil {
_ = sqlDB.Close()
}
}
}
// Delete the main DB file and WAL/SHM files
path := m.dbPath(webhookID)
for _, suffix := range []string{"", "-wal", "-shm"} {
err := os.Remove(path + suffix)
if err != nil && !os.IsNotExist(err) {
return fmt.Errorf(
"deleting webhook database file %s%s: %w",
path, suffix, err,
)
}
}
m.log.Info(
"deleted per-webhook database",
"webhook_id", webhookID,
)
return nil
}
// CloseAll closes all open per-webhook database connections.
// Called during application shutdown.
func (m *WebhookDBManager) CloseAll() error {
var lastErr error
m.dbs.Range(func(key, value any) bool {
if gormDB, castOK := value.(*gorm.DB); castOK {
sqlDB, err := gormDB.DB()
if err == nil {
closeErr := sqlDB.Close()
if closeErr != nil {
lastErr = closeErr
m.log.Error(
"failed to close webhook database",
"webhook_id", key,
"error", closeErr,
)
}
}
}
m.dbs.Delete(key)
return true
})
return lastErr
}
// DBPath returns the filesystem path for a webhook's database
// file.
func (m *WebhookDBManager) DBPath(
webhookID string,
) string {
return m.dbPath(webhookID)
}
func (m *WebhookDBManager) dbPath(
webhookID string,
) string {
return filepath.Join(
m.dataDir,
fmt.Sprintf("events-%s.db", webhookID),
)
}
// openDB opens (or creates) a per-webhook SQLite database and
// runs migrations.
func (m *WebhookDBManager) openDB(
webhookID string,
) (*gorm.DB, error) {
path := m.dbPath(webhookID)
dbURL := fmt.Sprintf(
"file:%s?cache=shared&mode=rwc",
path,
)
sqlDB, err := sql.Open("sqlite", dbURL)
if err != nil {
return nil, fmt.Errorf(
"opening webhook database %s: %w",
webhookID, err,
)
}
db, err := gorm.Open(sqlite.Dialector{
Conn: sqlDB,
}, &gorm.Config{})
if err != nil {
_ = sqlDB.Close()
return nil, fmt.Errorf(
"connecting to webhook database %s: %w",
webhookID, err,
)
}
// Run migrations for event-tier models only
err = db.AutoMigrate(
&Event{}, &Delivery{}, &DeliveryResult{},
)
if err != nil {
_ = sqlDB.Close()
return nil, fmt.Errorf(
"migrating webhook database %s: %w",
webhookID, err,
)
}
m.log.Info(
"opened per-webhook database",
"webhook_id", webhookID,
"path", path,
)
return db, nil
}

View File

@@ -0,0 +1,366 @@
package database_test
import (
"context"
"net/http"
"os"
"path/filepath"
"testing"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger"
)
func setupTestWebhookDBManager(
t *testing.T,
) (*database.WebhookDBManager, *fxtest.Lifecycle) {
t.Helper()
lc := fxtest.NewLifecycle(t)
g := &globals.Globals{
Appname: testAppname,
Version: testVersion,
}
l, err := logger.New(
lc,
logger.LoggerParams{Globals: g},
)
require.NoError(t, err)
dataDir := filepath.Join(t.TempDir(), "events")
cfg := &config.Config{
DataDir: dataDir,
}
mgr, err := database.NewWebhookDBManager(
lc,
database.WebhookDBManagerParams{
Config: cfg,
Logger: l,
},
)
require.NoError(t, err)
return mgr, lc
}
func TestWebhookDBManager_CreateAndGetDB(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
// DB should not exist yet
assert.False(t, mgr.DBExists(webhookID))
// Create the DB
err := mgr.CreateDB(webhookID)
require.NoError(t, err)
// DB file should now exist
assert.True(t, mgr.DBExists(webhookID))
// Get the DB again (should use cached connection)
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
require.NotNil(t, db)
// Verify we can write an event
event := &database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: http.MethodPost,
Headers: `{"Content-Type":["application/json"]}`,
Body: `{"test": true}`,
ContentType: testContentType,
}
require.NoError(t, db.Create(event).Error)
assert.NotEmpty(t, event.ID)
// Verify we can read it back
var readEvent database.Event
require.NoError(
t,
db.First(&readEvent, "id = ?", event.ID).Error,
)
assert.Equal(t, webhookID, readEvent.WebhookID)
assert.Equal(t, http.MethodPost, readEvent.Method)
assert.Equal(t, `{"test": true}`, readEvent.Body)
}
func TestWebhookDBManager_DeleteDB(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
// Create the DB and write some data
require.NoError(t, mgr.CreateDB(webhookID))
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
event := &database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: http.MethodPost,
Body: `{"test": true}`,
ContentType: testContentType,
}
require.NoError(t, db.Create(event).Error)
// Delete the DB
require.NoError(t, mgr.DeleteDB(webhookID))
// File should no longer exist
assert.False(t, mgr.DBExists(webhookID))
// Verify the file is actually gone from disk
dbPath := mgr.DBPath(webhookID)
_, err = os.Stat(dbPath)
assert.True(t, os.IsNotExist(err))
}
func TestWebhookDBManager_LazyCreation(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
// GetDB should lazily create the database
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
require.NotNil(t, db)
// File should now exist
assert.True(t, mgr.DBExists(webhookID))
}
func TestWebhookDBManager_DeliveryWorkflow(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
targetID := uuid.New().String()
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
event, delivery := seedDeliveryWorkflow(
t, db, webhookID, targetID,
)
verifyPendingDeliveries(t, db, event)
completeDelivery(t, db, delivery)
verifyNoPending(t, db)
}
func seedDeliveryWorkflow(
t *testing.T,
db *gorm.DB,
webhookID, targetID string,
) (*database.Event, *database.Delivery) {
t.Helper()
event := &database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: http.MethodPost,
Headers: `{"Content-Type":["application/json"]}`,
Body: `{"payload": "test"}`,
ContentType: testContentType,
}
require.NoError(t, db.Create(event).Error)
delivery := &database.Delivery{
EventID: event.ID,
TargetID: targetID,
Status: database.DeliveryStatusPending,
}
require.NoError(t, db.Create(delivery).Error)
return event, delivery
}
func verifyPendingDeliveries(
t *testing.T,
db *gorm.DB,
event *database.Event,
) {
t.Helper()
var pending []database.Delivery
require.NoError(
t,
db.Where(
"status = ?",
database.DeliveryStatusPending,
).Preload("Event").Find(&pending).Error,
)
require.Len(t, pending, 1)
assert.Equal(t, event.ID, pending[0].EventID)
assert.Equal(t, http.MethodPost, pending[0].Event.Method)
}
func completeDelivery(
t *testing.T,
db *gorm.DB,
delivery *database.Delivery,
) {
t.Helper()
result := &database.DeliveryResult{
DeliveryID: delivery.ID,
AttemptNum: 1,
Success: true,
StatusCode: 200,
Duration: 42,
}
require.NoError(t, db.Create(result).Error)
require.NoError(
t,
db.Model(delivery).Update(
"status",
database.DeliveryStatusDelivered,
).Error,
)
}
func verifyNoPending(
t *testing.T,
db *gorm.DB,
) {
t.Helper()
var stillPending []database.Delivery
require.NoError(
t,
db.Where(
"status = ?",
database.DeliveryStatusPending,
).Find(&stillPending).Error,
)
assert.Empty(t, stillPending)
}
func TestWebhookDBManager_MultipleWebhooks(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhook1 := uuid.New().String()
webhook2 := uuid.New().String()
// Create DBs for two webhooks
require.NoError(t, mgr.CreateDB(webhook1))
require.NoError(t, mgr.CreateDB(webhook2))
db1, err := mgr.GetDB(webhook1)
require.NoError(t, err)
db2, err := mgr.GetDB(webhook2)
require.NoError(t, err)
// Write events to each webhook's DB
event1 := &database.Event{
WebhookID: webhook1,
EntrypointID: uuid.New().String(),
Method: http.MethodPost,
Body: `{"webhook": 1}`,
ContentType: testContentType,
}
event2 := &database.Event{
WebhookID: webhook2,
EntrypointID: uuid.New().String(),
Method: http.MethodPut,
Body: `{"webhook": 2}`,
ContentType: testContentType,
}
require.NoError(t, db1.Create(event1).Error)
require.NoError(t, db2.Create(event2).Error)
// Verify isolation: each DB only has its own events
var count1 int64
db1.Model(&database.Event{}).Count(&count1)
assert.Equal(t, int64(1), count1)
var count2 int64
db2.Model(&database.Event{}).Count(&count2)
assert.Equal(t, int64(1), count2)
// Delete webhook1's DB, webhook2 should be unaffected
require.NoError(t, mgr.DeleteDB(webhook1))
assert.False(t, mgr.DBExists(webhook1))
assert.True(t, mgr.DBExists(webhook2))
// webhook2's data should still be accessible
var events []database.Event
require.NoError(t, db2.Find(&events).Error)
assert.Len(t, events, 1)
assert.Equal(t, "PUT", events[0].Method)
}
func TestWebhookDBManager_CloseAll(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
// Create a few DBs
for range 3 {
require.NoError(
t,
mgr.CreateDB(uuid.New().String()),
)
}
// CloseAll should close all connections without error
require.NoError(t, mgr.CloseAll())
// Stop lifecycle (CloseAll already called)
require.NoError(t, lc.Stop(ctx))
}

View File

@@ -0,0 +1,239 @@
package delivery
import (
"context"
"errors"
"log/slog"
"sync"
"time"
"go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/lifecycle"
"sneak.berlin/go/webhooker/internal/logger"
)
// ArchiveSweeperParams holds the fx dependencies for the
// ArchiveSweeper.
type ArchiveSweeperParams struct {
fx.In
Config *config.Config
Database *database.Database
Engine *Engine
Logger *logger.Logger
}
// ArchiveSweeper periodically prunes expired rows from
// per-webhook archive databases whose database target carries a
// positive expiry.
//
// Without it, pruning happens only when an archive is
// (re)opened, and archives are only ever reopened by writes: an
// archive belonging to a webhook that has stopped receiving
// events would keep its expired rows forever. The sweep closes
// that gap without changing anything for archives whose expiry
// is unset or "never".
//
// It reuses Config.RetentionSweepInterval rather than
// introducing a second interval: this is a retention sweep with
// the same semantics as the event retention reaper.
type ArchiveSweeper struct {
db *database.Database
eng *Engine
log *slog.Logger
interval time.Duration
cancel context.CancelFunc
wg sync.WaitGroup
}
// NewArchiveSweeper creates the archive sweeper and registers
// its fx lifecycle hooks. The background sweep loop starts on
// OnStart and stops cleanly on OnStop via context cancellation.
func NewArchiveSweeper(
lc fx.Lifecycle,
params ArchiveSweeperParams,
) *ArchiveSweeper {
s := &ArchiveSweeper{
db: params.Database,
eng: params.Engine,
log: params.Logger.Get(),
interval: params.Config.RetentionSweepInterval,
}
s.registerHooks(lc)
return s
}
// registerHooks wires the sweeper's start and stop into the fx
// lifecycle. The start hook's context is deliberately ignored
// (see start for why the background loop must not inherit it);
// the stop hook's context is honoured (see stop).
func (s *ArchiveSweeper) registerHooks(lc fx.Lifecycle) {
lc.Append(fx.Hook{
//nolint:contextcheck // Not passing the hook context is
// the point: see start.
OnStart: func(_ context.Context) error {
s.start()
return nil
},
OnStop: func(ctx context.Context) error {
return s.stop(ctx)
},
})
}
// start launches the background sweep loop.
//
// The loop's context is derived from context.Background(), NOT
// from the fx OnStart hook context. The hook context carries
// fx's start timeout (15s by default), so a loop derived from it
// is cancelled 15 seconds after the application starts — long
// before the first tick under the default one-hour sweep
// interval, leaving a sweeper that never sweeps. A long-lived
// goroutine must outlive the startup phase, so its lifetime is
// bounded by OnStop instead: stop cancels this context and waits
// on the WaitGroup.
func (s *ArchiveSweeper) start() {
ctx, cancel := context.WithCancel(context.Background())
s.cancel = cancel
s.wg.Add(1)
go s.run(ctx)
s.log.Info(
"archive sweeper started",
"interval", s.interval.String(),
)
}
// stop cancels the sweep loop's context and waits for it to
// exit, bounded by the stop hook's context: a prune wedged on a
// locked archive must not hang the process past fx's stop
// timeout.
func (s *ArchiveSweeper) stop(ctx context.Context) error {
s.log.Info("archive sweeper stopping")
if s.cancel != nil {
s.cancel()
}
err := lifecycle.WaitForShutdown(
ctx, s.log, "archive sweeper", &s.wg,
)
if err != nil {
return err
}
s.log.Info("archive sweeper stopped")
return nil
}
func (s *ArchiveSweeper) run(ctx context.Context) {
defer s.wg.Done()
ticker := time.NewTicker(s.interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
s.sweep(ctx)
}
}
}
// sweep prunes every archive whose database target declares a
// positive expiry. Targets belonging to a deleted webhook are
// soft-deleted along with it, so GORM's default scope already
// excludes them.
//
// A failure for one webhook is logged and the sweep continues,
// matching how the write path already treats a prune error as
// non-fatal.
func (s *ArchiveSweeper) sweep(ctx context.Context) {
var targets []database.Target
err := s.db.DB().
Model(&database.Target{}).
Where("type = ?", database.TargetTypeDatabase).
Find(&targets).Error
if err != nil {
s.log.Error(
"archive sweep: failed to list database targets",
"error", err,
)
return
}
for i := range targets {
select {
case <-ctx.Done():
return
default:
}
s.sweepTarget(&targets[i])
}
}
// sweepTarget prunes the archive of a single database target.
// A missing, empty, or "never" expiry parses as a zero duration
// and is skipped entirely, so those archives keep exactly the
// behaviour they had before the sweep existed.
func (s *ArchiveSweeper) sweepTarget(target *database.Target) {
expiry, err := parseArchiveExpiry(target.Config)
if err != nil {
s.log.Error(
"archive sweep: invalid database target config",
"webhook_id", target.WebhookID,
"target_id", target.ID,
"error", err,
)
return
}
if expiry <= 0 {
return
}
if s.eng == nil || s.eng.dbTarget == nil {
return
}
err = s.eng.dbTarget.sweepWebhook(target.WebhookID, expiry)
if err == nil {
return
}
// A writer evicted underneath the sweep means the operator
// deleted the webhook (or its last database target) while the
// sweep was walking the target list. That is an ordinary
// interleaving, not a failure, so it must not produce an
// error line.
if errors.Is(err, errArchiveWriterEvicted) {
s.log.Debug(
"archive sweep: writer evicted mid-sweep",
"webhook_id", target.WebhookID,
"target_id", target.ID,
)
return
}
s.log.Error(
"archive sweep: failed to prune archive",
"webhook_id", target.WebhookID,
"target_id", target.ID,
"error", err,
)
}

View File

@@ -0,0 +1,947 @@
package delivery_test
import (
"context"
"database/sql"
"fmt"
"net/http"
"os"
"path/filepath"
"sync"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/clause"
_ "modernc.org/sqlite" // Pure Go SQLite driver.
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
const (
// sweepRowOld and sweepRowNew are the event ids
// seedArchiveRows assigns to the first and second seeded
// rows.
sweepRowOld = "ev-0"
sweepRowNew = "ev-1"
// sweepConcurrentWrites is how many deliveries the
// concurrent write-plus-sweep test races against the sweep.
sweepConcurrentWrites = 20
)
// sweeperEnv bundles the pieces an archive sweep test drives:
// a main configuration database holding webhooks and targets, a
// delivery engine owning the archive writer registry, and the
// data directory the archive files live in.
type sweeperEnv struct {
sweeper *delivery.ArchiveSweeper
eng *delivery.Engine
mainDB *database.Database
dataDir string
}
func setupSweeperTest(t *testing.T) *sweeperEnv {
t.Helper()
dataDir := t.TempDir()
log := archiveTestLogger()
sqlDB, err := sql.Open(
"sqlite",
fmt.Sprintf(
"file:%s?mode=rwc",
filepath.Join(dataDir, "main.db"),
),
)
require.NoError(t, err)
t.Cleanup(func() { _ = sqlDB.Close() })
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB}, &gorm.Config{},
)
require.NoError(t, err)
mainDB := database.NewTestDatabase(gdb)
require.NoError(t, mainDB.Migrate())
eng := delivery.NewTestEngineWithDB(
mainDB,
database.NewTestWebhookDBManager(dataDir),
log,
&http.Client{Timeout: 5 * time.Second},
1,
)
return &sweeperEnv{
sweeper: delivery.NewTestArchiveSweeper(
mainDB, eng, log,
),
eng: eng,
mainDB: mainDB,
dataDir: dataDir,
}
}
// archivePath returns where the engine keeps a webhook's
// archive file.
func (env *sweeperEnv) archivePath(webhookID string) string {
return filepath.Join(
env.dataDir, fmt.Sprintf("archive-%s.db", webhookID),
)
}
// seedDatabaseTarget creates a webhook with one database target
// carrying the given target config JSON, and returns the
// webhook id.
func (env *sweeperEnv) seedDatabaseTarget(
t *testing.T, configJSON string,
) string {
t.Helper()
wh := &database.Webhook{
UserID: uuid.New().String(),
Name: "sweep-test",
}
require.NoError(
t,
env.mainDB.DB().
Omit(clause.Associations).
Create(wh).Error,
)
tgt := &database.Target{
WebhookID: wh.ID,
Name: "archive",
Type: database.TargetTypeDatabase,
Active: true,
Config: configJSON,
}
require.NoError(
t,
env.mainDB.DB().
Omit(clause.Associations).
Create(tgt).Error,
)
return wh.ID
}
// seedArchiveRows creates the archive file for a webhook and
// inserts one row per supplied archived-at timestamp, returning
// the archive path. The handle is closed before returning, so
// the archive is idle exactly as it would be with no traffic.
func (env *sweeperEnv) seedArchiveRows(
t *testing.T, webhookID string, archivedAt ...time.Time,
) string {
t.Helper()
path := env.archivePath(webhookID)
sqlDB, err := sql.Open(
"sqlite", fmt.Sprintf("file:%s?mode=rwc", path),
)
require.NoError(t, err)
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB}, &gorm.Config{},
)
require.NoError(t, err)
require.NoError(
t, gdb.AutoMigrate(&delivery.ExportArchivedEvent{}),
)
for i, at := range archivedAt {
row := delivery.ExportArchivedEvent{
EventID: fmt.Sprintf("ev-%d", i),
WebhookID: webhookID,
Method: http.MethodPost,
Body: `{"seeded":true}`,
ArchivedAt: at,
}
require.NoError(t, gdb.Create(&row).Error)
}
require.NoError(t, sqlDB.Close())
return path
}
// archivedEventIDs returns the event ids currently stored in an
// archive file, read through a separate read-only handle.
func archivedEventIDs(
t *testing.T, path string,
) []string {
t.Helper()
var rows []delivery.ExportArchivedEvent
rdb := openArchiveDBForRead(t, path)
require.NoError(t, rdb.Order("event_id").Find(&rows).Error)
ids := make([]string, 0, len(rows))
for i := range rows {
ids = append(ids, rows[i].EventID)
}
return ids
}
// countArchivedRows counts the rows in an archive file without
// asserting anything, so it is safe to poll from an
// assert.Eventually condition (which runs off the test
// goroutine, where testify assertions must not be used).
func countArchivedRows(path string) (int64, error) {
sqlDB, err := sql.Open(
"sqlite", fmt.Sprintf("file:%s?mode=ro", path),
)
if err != nil {
return 0, err
}
defer func() { _ = sqlDB.Close() }()
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB}, &gorm.Config{},
)
if err != nil {
return 0, err
}
var count int64
err = gdb.Model(&delivery.ExportArchivedEvent{}).
Count(&count).Error
if err != nil {
return 0, err
}
return count, nil
}
// TestArchiveSweeper_LoopOutlivesStartHookContext is the
// regression test for a sweeper that never swept. fx calls
// OnStart with a context carrying the application's start
// timeout (15 seconds by default), so a background loop whose
// context is derived from it is cancelled 15 seconds into the
// process — three quarters of an hour before the first tick
// under the default one-hour sweep interval.
//
// The hook context here is already cancelled, which is the same
// defect taken to its limit: a loop that inherits it never runs
// a single tick, while a correctly rooted loop keeps sweeping
// for as long as the process lives. Handing the hook a plain
// context.Background() would assert nothing at all.
func TestArchiveSweeper_LoopOutlivesStartHookContext(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
now := time.Now()
path := env.seedArchiveRows(
t, webhookID,
now.Add(-48*time.Hour),
now.Add(-time.Minute),
)
env.sweeper.ExportSetInterval(10 * time.Millisecond)
// Drive the genuine fx hooks the application registers,
// rather than a test-only entry point.
lc := &recordingLifecycle{}
env.sweeper.ExportRegisterHooks(lc)
require.Len(t, lc.hooks, 1)
hookCtx, cancel := context.WithCancel(context.Background())
cancel()
require.NoError(t, lc.hooks[0].OnStart(hookCtx))
t.Cleanup(func() {
_ = lc.hooks[0].OnStop(context.Background())
})
assert.Eventually(
t,
func() bool {
count, err := countArchivedRows(path)
return err == nil && count == 1
},
5*time.Second,
10*time.Millisecond,
"the sweep loop must keep running after the start "+
"hook's context is done; it pruned nothing, so it "+
"inherited the hook context and died",
)
}
// TestArchiveSweep_DoesNotResurrectEvictedWriter covers the
// interleaving where a sweep tick has already listed a webhook's
// target when the webhook is deleted and its writer evicted. The
// sweep must not put a writer back into the registry: nothing
// would ever evict it again, which is precisely the leak this
// change exists to close.
func TestArchiveSweep_DoesNotResurrectEvictedWriter(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows(
t, webhookID, time.Now().Add(-48*time.Hour),
)
// Prime the registry the way a delivery would, then evict as
// the deletion path does. The target row is deliberately left
// in place: this is the tick that listed the webhook before
// the deletion committed.
_, err := env.eng.ExportEnsureArchiveWriter(webhookID)
require.NoError(t, err)
env.eng.EvictWebhook(webhookID)
require.False(t, env.eng.ExportHasArchiveWriter(webhookID))
env.sweeper.ExportSweep(context.Background())
assert.False(
t, env.eng.ExportHasArchiveWriter(webhookID),
"a sweep must never re-register a writer for a webhook "+
"whose registry entry has already been released",
)
}
// TestArchiveSweep_LeavesNoRegistryEntry states the same
// invariant in its general form: sweeping an archive whose
// webhook has no cached writer must not leave one behind, so the
// registry keeps holding only writers a delivery created and an
// eviction can reach.
func TestArchiveSweep_LeavesNoRegistryEntry(t *testing.T) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path := env.seedArchiveRows(
t, webhookID,
time.Now().Add(-48*time.Hour),
time.Now().Add(-time.Minute),
)
require.False(t, env.eng.ExportHasArchiveWriter(webhookID))
env.sweeper.ExportSweep(context.Background())
assert.Equal(
t, []string{sweepRowNew}, archivedEventIDs(t, path),
"the sweep must still prune an idle archive",
)
assert.False(
t, env.eng.ExportHasArchiveWriter(webhookID),
"the sweep must release the registry entry it created",
)
}
// TestArchiveSweep_KeepsWriterAdoptedByDelivery is the other
// half of that invariant: an entry the sweep created but a
// delivery then claimed belongs to the registry and must survive
// the sweep, or the delivery would be left holding a detached
// writer with an open handle that no eviction can reach.
func TestArchiveSweep_KeepsWriterAdoptedByDelivery(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows(
t, webhookID, time.Now().Add(-48*time.Hour),
)
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"n":1}`)
event.WebhookID = webhookID
d := seedDatabaseTargetDelivery(
t, webhookDB, event, `{"expiry":"1h"}`,
)
env.sweeper.ExportSweep(context.Background())
require.False(t, env.eng.ExportHasArchiveWriter(webhookID))
env.eng.ExportDeliverDatabase(webhookDB, d)
assert.True(
t, env.eng.ExportHasArchiveWriter(webhookID),
"a delivery's writer must stay registered",
)
env.sweeper.ExportSweep(context.Background())
assert.True(
t, env.eng.ExportHasArchiveWriter(webhookID),
"a sweep must not drop a writer a delivery owns",
)
}
// TestArchiveSweep_KeepsWriterAdoptedDuringSweep covers the one
// interleaving the sweepOwned flag exists for, which
// TestArchiveSweep_KeepsWriterAdoptedByDelivery cannot reach: a
// delivery adopting the sweep's own entry WHILE that sweep is
// still running.
//
// The registry operations are driven directly, in the order the
// sweep and a concurrent delivery perform them, so the window is
// exercised deterministically rather than hoped for:
//
// 1. the sweep finds no cached writer and registers one of its
// own, marked sweep-owned;
// 2. a delivery arrives, is handed that very writer, clears the
// flag and opens the archive handle;
// 3. the sweep finishes and releases what it created.
//
// Step 3 must leave the entry alone. Dropping it would detach a
// writer that is holding an open archive handle inside its
// debounce window, and no eviction could ever reach it again —
// exactly the process-lifetime handle leak this change exists to
// close. The eviction at the end proves the entry is still
// reachable.
func TestArchiveSweep_KeepsWriterAdoptedDuringSweep(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows(
t, webhookID, time.Now().Add(-48*time.Hour),
)
sweepWriter, created, err := env.eng.ExportSweepWriterFor(
webhookID,
)
require.NoError(t, err)
require.True(
t, created,
"the sweep must have created the registry entry itself",
)
// The delivery lands mid-sweep and adopts the entry.
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"n":1}`)
event.WebhookID = webhookID
d := seedDatabaseTargetDelivery(
t, webhookDB, event, `{"expiry":"1h"}`,
)
env.eng.ExportDeliverDatabase(webhookDB, d)
adopted := env.eng.ExportArchiveWriterFor(webhookID)
require.NotNil(t, adopted)
require.True(
t, sweepWriter.Same(adopted),
"the delivery must have adopted the sweep's writer",
)
require.True(
t, env.eng.ExportArchiveHandleOpen(webhookID),
"the delivery leaves the archive handle open",
)
// The sweep finishes.
env.eng.ExportReleaseSweepWriter(webhookID, sweepWriter)
require.True(
t, env.eng.ExportHasArchiveWriter(webhookID),
"a writer adopted by a delivery during a sweep must "+
"stay registered, or its open handle is unreachable",
)
env.eng.EvictWebhook(webhookID)
assert.False(
t, env.eng.ExportHasArchiveWriter(webhookID),
"the adopted writer must still be evictable",
)
assert.False(
t, sweepWriter.HandleOpen(),
"eviction must have closed the adopted writer's handle",
)
}
// TestArchiveSweep_ContinuesAfterPerWebhookFailure proves a
// failure for one webhook does not abort the sweep for the
// others: an unparseable expiry and an unreadable archive both
// have to be logged and stepped over.
func TestArchiveSweep_ContinuesAfterPerWebhookFailure(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
// Seeded first so the sweep reaches them before the healthy
// webhook: targets come back in insertion order.
badConfigID := env.seedDatabaseTarget(t, `{"expiry":"!!!"}`)
env.seedArchiveRows(
t, badConfigID, time.Now().Add(-48*time.Hour),
)
corruptID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
require.NoError(t, os.WriteFile(
env.archivePath(corruptID),
[]byte("this is not a sqlite database"),
0o600,
))
healthyID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
healthyPath := env.seedArchiveRows(
t, healthyID,
time.Now().Add(-48*time.Hour),
time.Now().Add(-time.Minute),
)
env.sweeper.ExportSweep(context.Background())
assert.Equal(
t, []string{sweepRowNew},
archivedEventIDs(t, healthyPath),
"a failure for an earlier webhook must not stop the "+
"sweep from pruning the ones after it",
)
}
// TestArchiveSweep_OpenExistingDoesNotCreateFile pins the second
// of the two no-create guards. The first is the stat in
// sweepWebhook; this one is the SQLite open mode, which is what
// protects the window between that stat and the open. Flipping
// the sweep's mode to create-if-missing makes this fail.
func TestArchiveSweep_OpenExistingDoesNotCreateFile(
t *testing.T,
) {
t.Parallel()
dir := t.TempDir()
path := filepath.Join(dir, "archive-absent.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
err := w.OpenExisting(time.Hour)
require.Error(
t, err,
"opening a missing archive without create permission "+
"must fail rather than conjure the file",
)
for _, suffix := range archiveFileSuffixes() {
assert.NoFileExists(t, path+suffix)
}
}
// TestArchiveSweep_PrunesIdleArchive is the core regression
// test for this issue: an archive that receives no further
// writes must still lose its expired rows. Before the sweeper
// existed, pruning only ever ran on a write-triggered reopen,
// so an idle archive kept expired rows forever.
func TestArchiveSweep_PrunesIdleArchive(t *testing.T) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
now := time.Now()
path := env.seedArchiveRows(
t, webhookID,
now.Add(-48*time.Hour),
now.Add(-time.Minute),
)
require.Equal(
t, []string{sweepRowOld, sweepRowNew},
archivedEventIDs(t, path),
)
env.sweeper.ExportSweep(context.Background())
assert.Equal(
t, []string{sweepRowNew}, archivedEventIDs(t, path),
"the sweep should prune rows older than the expiry "+
"from an idle archive and keep the rest",
)
}
// TestArchiveSweep_LeavesArchiveClosed proves the sweep does
// not hold the archive open afterwards, so an operator can
// still move the file away for offline retention.
//
// The assertion is made on a writer the test holds a reference
// to, and the handle is proven OPEN before the sweep runs, so the
// test observes the sweep closing it rather than a writer that
// merely never opened anything. Asking the registry instead would
// be vacuous here: the sweep releases an entry it created, and a
// missing entry reports "not open" whether or not anything was
// closed.
func TestArchiveSweep_LeavesArchiveClosed(t *testing.T) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path := env.seedArchiveRows(
t, webhookID, time.Now().Add(-48*time.Hour),
)
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.OpenExisting(time.Hour))
require.True(
t, w.HandleOpen(),
"the writer must hold an open handle before the sweep",
)
require.NoError(t, w.SweepExpired(time.Hour))
assert.False(
t, w.HandleOpen(),
"an idle archive must end the sweep closed",
)
}
// TestArchiveSweep_ClosesHandleOfRegisteredWriter states the same
// guarantee end to end, through the real sweeper and a writer the
// registry keeps.
//
// The delivery leaves the archive handle open inside its debounce
// window and makes the entry delivery-owned, so the sweep finds a
// cached writer (created is false, nothing is released) and the
// registry query afterwards is answered by a writer that really
// exists. A handle left open here would be doubly wrong: it also
// blocks the operator's move-the-file-away workflow.
func TestArchiveSweep_ClosesHandleOfRegisteredWriter(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows(
t, webhookID, time.Now().Add(-48*time.Hour),
)
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"n":1}`)
event.WebhookID = webhookID
d := seedDatabaseTargetDelivery(
t, webhookDB, event, `{"expiry":"1h"}`,
)
env.eng.ExportDeliverDatabase(webhookDB, d)
require.True(
t, env.eng.ExportArchiveHandleOpen(webhookID),
"the delivery must leave the archive handle open",
)
env.sweeper.ExportSweep(context.Background())
require.True(
t, env.eng.ExportHasArchiveWriter(webhookID),
"the delivery's registry entry must survive the sweep",
)
assert.False(
t, env.eng.ExportArchiveHandleOpen(webhookID),
"the sweep must leave the archive closed",
)
}
// TestArchiveSweep_NeverExpiryUntouched proves the sweep is a
// no-op for the default retention policy, so archives with no
// expiry (or the literal "never") behave exactly as before.
func TestArchiveSweep_NeverExpiryUntouched(t *testing.T) {
t.Parallel()
for _, configJSON := range []string{
`{"expiry":"never"}`,
`{"expiry":""}`,
"",
} {
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, configJSON)
path := env.seedArchiveRows(
t, webhookID,
time.Now().Add(-10000*time.Hour),
)
env.sweeper.ExportSweep(context.Background())
assert.Equal(
t, []string{sweepRowOld}, archivedEventIDs(t, path),
"config %q must keep rows forever", configJSON,
)
assert.False(
t, env.eng.ExportHasArchiveWriter(webhookID),
"config %q must leave no registry entry behind",
configJSON,
)
}
}
// TestArchiveSweep_NeverExpirySkipsBeforeOpening pins the
// expiry <= 0 boundary in sweepTarget, which the row assertions
// above cannot reach: pruning is separately gated on a positive
// expiry, so a "never" archive keeps its rows even if the sweep
// does open it.
//
// The spec is stronger than that — a "never" archive is skipped
// before any file is touched — so the archive here exists but has
// never been migrated. Opening it at all would run AutoMigrate
// and create the archive table, which is exactly what must not
// happen.
func TestArchiveSweep_NeverExpirySkipsBeforeOpening(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"never"}`)
path := env.archivePath(webhookID)
seedUnmigratedArchive(t, path)
require.False(t, archiveTableExists(t, path))
env.sweeper.ExportSweep(context.Background())
assert.False(
t, archiveTableExists(t, path),
"a never-expiry archive must not be opened at all",
)
}
// seedUnmigratedArchive creates an archive file that exists but
// carries no archive schema, so any open of it is observable: the
// archive table appears only if something ran AutoMigrate.
func seedUnmigratedArchive(t *testing.T, path string) {
t.Helper()
sqlDB, err := sql.Open(
"sqlite", fmt.Sprintf("file:%s?mode=rwc", path),
)
require.NoError(t, err)
_, err = sqlDB.ExecContext(
t.Context(), "CREATE TABLE placeholder (id INTEGER)",
)
require.NoError(t, err)
require.NoError(t, sqlDB.Close())
}
// archiveTableExists reports whether an archive file has had the
// archive schema migrated into it.
func archiveTableExists(t *testing.T, path string) bool {
t.Helper()
return openArchiveDBForRead(t, path).
Migrator().
HasTable(&delivery.ExportArchivedEvent{})
}
// TestArchiveSweep_DoesNotCreateArchiveFile proves the sweep
// never conjures an archive: a webhook with a database target
// that has never received an event must still have no archive
// file (nor SQLite sidecar) after a sweep.
func TestArchiveSweep_DoesNotCreateArchiveFile(t *testing.T) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path := env.archivePath(webhookID)
require.NoFileExists(t, path)
env.sweeper.ExportSweep(context.Background())
for _, suffix := range archiveFileSuffixes() {
assert.NoFileExists(
t, path+suffix,
"the sweep must not create an archive file",
)
}
}
// TestArchiveSweep_DoesNotCreateAfterWriterExists covers the
// same guarantee once a writer is cached in the registry but
// the file itself is still absent (for instance because the
// operator moved the archive away).
func TestArchiveSweep_DoesNotCreateAfterWriterExists(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path, err := env.eng.ExportEnsureArchiveWriter(webhookID)
require.NoError(t, err)
require.NoFileExists(t, path)
env.sweeper.ExportSweep(context.Background())
assert.NoFileExists(t, path)
}
// TestArchiveSweep_SkipsDeletedWebhookTargets proves that the
// sweep ignores targets soft-deleted along with their webhook,
// so a deleted webhook's archive is never reopened.
func TestArchiveSweep_SkipsDeletedWebhookTargets(t *testing.T) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path := env.seedArchiveRows(
t, webhookID, time.Now().Add(-48*time.Hour),
)
require.NoError(
t,
env.mainDB.DB().
Where("webhook_id = ?", webhookID).
Delete(&database.Target{}).Error,
)
env.sweeper.ExportSweep(context.Background())
assert.Equal(
t, []string{sweepRowOld}, archivedEventIDs(t, path),
"a deleted target's archive must be left alone",
)
}
// TestArchiveSweep_ConcurrentWrites proves the sweep serialises
// against writes through the per-webhook writer mutex. Run
// under -race, an unsynchronised sweep would be caught here.
func TestArchiveSweep_ConcurrentWrites(t *testing.T) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
webhookDB := testWebhookDB(t)
// The deliveries are seeded up front, on the test's own
// goroutine: the seed helpers assert, and testify assertions
// must not run off the test goroutine.
deliveries := make(
[]*database.Delivery, 0, sweepConcurrentWrites,
)
for range sweepConcurrentWrites {
event := seedEvent(t, webhookDB, `{"n":1}`)
event.WebhookID = webhookID
deliveries = append(
deliveries,
seedDatabaseTargetDelivery(
t, webhookDB, event, `{"expiry":"1h"}`,
),
)
}
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
for _, d := range deliveries {
env.eng.ExportDeliverDatabase(webhookDB, d)
}
}()
go func() {
defer wg.Done()
for range sweepConcurrentWrites {
env.sweeper.ExportSweep(context.Background())
}
}()
wg.Wait()
assert.FileExists(t, env.archivePath(webhookID))
}
// TestArchiveSweeper_StopsCleanly proves the background loop
// exits on OnStop rather than leaking a goroutine.
func TestArchiveSweeper_StopsCleanly(t *testing.T) {
t.Parallel()
env := setupSweeperTest(t)
webhookID := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows(
t, webhookID, time.Now().Add(-48*time.Hour),
)
env.sweeper.ExportSetInterval(time.Millisecond)
env.sweeper.ExportStart()
// stop blocks on the loop's WaitGroup, so returning without
// error proves the loop observed the cancellation and exited
// well inside the stop context.
require.NoError(
t, env.sweeper.ExportStop(context.Background()),
)
}
// TestArchiveSweeper_StopHookHonoursStopTimeout is the sweeper's
// half of the same shutdown defect the engine and the retention
// reaper carried: an OnStop that discards its context and waits
// on the WaitGroup bare hangs the process forever on a prune
// wedged inside a locked archive.
func TestArchiveSweeper_StopHookHonoursStopTimeout(
t *testing.T,
) {
t.Parallel()
env := setupSweeperTest(t)
lc := &recordingLifecycle{}
env.sweeper.ExportRegisterHooks(lc)
require.Len(t, lc.hooks, 1)
require.NoError(t, lc.hooks[0].OnStart(context.Background()))
release := make(chan struct{})
t.Cleanup(func() { close(release) })
env.sweeper.ExportWedgeLoop(release)
requireStopHookExpires(t, lc.hooks[0], "archive sweeper")
}

View File

@@ -0,0 +1,152 @@
package delivery
import (
"sync"
"time"
)
// CircuitState represents the current state of a circuit
// breaker.
type CircuitState int
const (
// CircuitClosed is the normal operating state.
CircuitClosed CircuitState = iota
// CircuitOpen means the circuit has tripped.
CircuitOpen
// CircuitHalfOpen allows a single probe delivery to
// test whether the target has recovered.
CircuitHalfOpen
)
const (
// defaultFailureThreshold is the number of consecutive
// failures before a circuit breaker trips open.
defaultFailureThreshold = 5
// defaultCooldown is how long a circuit stays open
// before transitioning to half-open.
defaultCooldown = 30 * time.Second
)
// CircuitBreaker implements the circuit breaker pattern
// for a single delivery target.
type CircuitBreaker struct {
mu sync.Mutex
state CircuitState
failures int
threshold int
cooldown time.Duration
lastFailure time.Time
}
// NewCircuitBreaker creates a circuit breaker with default
// settings.
func NewCircuitBreaker() *CircuitBreaker {
return &CircuitBreaker{
state: CircuitClosed,
threshold: defaultFailureThreshold,
cooldown: defaultCooldown,
}
}
// Allow checks whether a delivery attempt should proceed.
func (cb *CircuitBreaker) Allow() bool {
cb.mu.Lock()
defer cb.mu.Unlock()
switch cb.state {
case CircuitClosed:
return true
case CircuitOpen:
if time.Since(cb.lastFailure) >= cb.cooldown {
cb.state = CircuitHalfOpen
return true
}
return false
case CircuitHalfOpen:
return false
default:
return true
}
}
// CooldownRemaining returns how much time is left before
// an open circuit transitions to half-open.
func (cb *CircuitBreaker) CooldownRemaining() time.Duration {
cb.mu.Lock()
defer cb.mu.Unlock()
if cb.state != CircuitOpen {
return 0
}
remaining := cb.cooldown - time.Since(cb.lastFailure)
if remaining < 0 {
return 0
}
return remaining
}
// RecordSuccess records a successful delivery and resets
// the circuit breaker to closed state.
func (cb *CircuitBreaker) RecordSuccess() {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.failures = 0
cb.state = CircuitClosed
}
// RecordFailure records a failed delivery. If the failure
// count reaches the threshold, the circuit trips open.
func (cb *CircuitBreaker) RecordFailure() {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.failures++
cb.lastFailure = time.Now()
switch cb.state {
case CircuitClosed:
if cb.failures >= cb.threshold {
cb.state = CircuitOpen
}
case CircuitOpen:
// Already open; no state change needed.
case CircuitHalfOpen:
// Probe failed -- reopen immediately.
cb.state = CircuitOpen
}
}
// State returns the current circuit state.
func (cb *CircuitBreaker) State() CircuitState {
cb.mu.Lock()
defer cb.mu.Unlock()
return cb.state
}
// String returns the human-readable name of a circuit
// state.
func (s CircuitState) String() string {
switch s {
case CircuitClosed:
return "closed"
case CircuitOpen:
return "open"
case CircuitHalfOpen:
return "half-open"
default:
return "unknown"
}
}

View File

@@ -0,0 +1,310 @@
package delivery_test
import (
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/delivery"
)
func TestCircuitBreaker_ClosedState_AllowsDeliveries(
t *testing.T,
) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
assert.Equal(t, delivery.CircuitClosed, cb.State())
assert.True(t, cb.Allow(),
"closed circuit should allow deliveries",
)
for range 10 {
assert.True(t, cb.Allow())
}
}
func TestCircuitBreaker_FailureCounting(t *testing.T) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
for i := range delivery.ExportDefaultFailureThreshold - 1 {
cb.RecordFailure()
assert.Equal(t,
delivery.CircuitClosed, cb.State(),
"circuit should remain closed after %d failures",
i+1,
)
assert.True(t, cb.Allow(),
"should still allow after %d failures",
i+1,
)
}
}
func TestCircuitBreaker_OpenTransition(t *testing.T) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
assert.Equal(t, delivery.CircuitOpen, cb.State(),
"circuit should be open after threshold failures",
)
assert.False(t, cb.Allow(),
"open circuit should reject deliveries",
)
}
func TestCircuitBreaker_Cooldown_StaysOpen(t *testing.T) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
require.Equal(t, delivery.CircuitOpen, cb.State())
assert.False(t, cb.Allow(),
"should be blocked during cooldown",
)
remaining := cb.CooldownRemaining()
assert.Greater(t, remaining, time.Duration(0),
"cooldown should have remaining time",
)
}
func TestCircuitBreaker_HalfOpen_AfterCooldown(
t *testing.T,
) {
t.Parallel()
cb := newShortCooldownCB(t)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
require.Equal(t, delivery.CircuitOpen, cb.State())
time.Sleep(60 * time.Millisecond)
assert.Equal(t, time.Duration(0),
cb.CooldownRemaining(),
)
assert.True(t, cb.Allow(),
"should allow one probe after cooldown",
)
assert.Equal(t,
delivery.CircuitHalfOpen, cb.State(),
"should be half-open after probe allowed",
)
assert.False(t, cb.Allow(),
"should reject additional probes while half-open",
)
}
func TestCircuitBreaker_ProbeSuccess_ClosesCircuit(
t *testing.T,
) {
t.Parallel()
cb := newShortCooldownCB(t)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
time.Sleep(60 * time.Millisecond)
require.True(t, cb.Allow())
cb.RecordSuccess()
assert.Equal(t, delivery.CircuitClosed, cb.State(),
"successful probe should close circuit",
)
assert.True(t, cb.Allow(),
"closed circuit should allow deliveries",
)
}
func TestCircuitBreaker_ProbeFailure_ReopensCircuit(
t *testing.T,
) {
t.Parallel()
cb := newShortCooldownCB(t)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
time.Sleep(60 * time.Millisecond)
require.True(t, cb.Allow())
cb.RecordFailure()
assert.Equal(t, delivery.CircuitOpen, cb.State(),
"failed probe should reopen circuit",
)
assert.False(t, cb.Allow(),
"reopened circuit should reject deliveries",
)
}
func TestCircuitBreaker_SuccessResetsFailures(
t *testing.T,
) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
for range delivery.ExportDefaultFailureThreshold - 1 {
cb.RecordFailure()
}
require.Equal(t, delivery.CircuitClosed, cb.State())
cb.RecordSuccess()
assert.Equal(t, delivery.CircuitClosed, cb.State())
for range delivery.ExportDefaultFailureThreshold - 1 {
cb.RecordFailure()
}
assert.Equal(t, delivery.CircuitClosed, cb.State(),
"circuit should still be closed -- "+
"success reset the counter",
)
cb.RecordFailure()
assert.Equal(t, delivery.CircuitOpen, cb.State())
}
func TestCircuitBreaker_ConcurrentAccess(t *testing.T) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
const goroutines = 100
var wg sync.WaitGroup
wg.Add(goroutines * 3)
for range goroutines {
go func() {
defer wg.Done()
cb.Allow()
}()
}
for range goroutines {
go func() {
defer wg.Done()
cb.RecordFailure()
}()
}
for range goroutines {
go func() {
defer wg.Done()
cb.RecordSuccess()
}()
}
wg.Wait()
state := cb.State()
assert.Contains(t,
[]delivery.CircuitState{
delivery.CircuitClosed,
delivery.CircuitOpen,
delivery.CircuitHalfOpen,
},
state,
"state should be valid after concurrent access",
)
}
func TestCircuitBreaker_CooldownRemaining_ClosedReturnsZero(
t *testing.T,
) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
assert.Equal(t, time.Duration(0),
cb.CooldownRemaining(),
"closed circuit should have zero cooldown remaining",
)
}
func TestCircuitBreaker_CooldownRemaining_HalfOpenReturnsZero(
t *testing.T,
) {
t.Parallel()
cb := newShortCooldownCB(t)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
time.Sleep(60 * time.Millisecond)
require.True(t, cb.Allow())
assert.Equal(t, time.Duration(0),
cb.CooldownRemaining(),
"half-open circuit should have zero cooldown remaining",
)
}
func TestCircuitState_String(t *testing.T) {
t.Parallel()
assert.Equal(t, "closed", delivery.CircuitClosed.String())
assert.Equal(t, "open", delivery.CircuitOpen.String())
assert.Equal(t, "half-open", delivery.CircuitHalfOpen.String())
assert.Equal(t, "unknown", delivery.CircuitState(99).String())
}
// newShortCooldownCB creates a CircuitBreaker with a short
// cooldown for testing. We use NewCircuitBreaker and
// manipulate through the public API.
func newShortCooldownCB(t *testing.T) *delivery.CircuitBreaker {
t.Helper()
return delivery.NewTestCircuitBreaker(
delivery.ExportDefaultFailureThreshold,
50*time.Millisecond,
)
}

View File

@@ -0,0 +1,112 @@
package delivery_test
import (
"context"
"log/slog"
"net/http"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/delivery"
)
// newSSRFTestEngine builds an Engine whose shared client
// carries the SSRF-safe transport, mirroring production.
func newSSRFTestEngine() *delivery.Engine {
log := slog.New(slog.DiscardHandler)
client := &http.Client{
Timeout: 30 * time.Second,
Transport: delivery.NewSSRFSafeTransport(),
}
return delivery.NewTestEngine(log, client, 1)
}
// TestClientForConfig_TimeoutKeepsSSRFGuard asserts that a
// client returned by clientForConfig for a config with a
// per-target timeout still refuses connections to
// private/reserved addresses (the timeout must not drop the
// SSRF-safe transport).
func TestClientForConfig_TimeoutKeepsSSRFGuard(t *testing.T) {
t.Parallel()
engine := newSSRFTestEngine()
blocked := []string{
"http://127.0.0.1/hook",
"http://169.254.169.254/latest/meta-data/",
"http://[fe80::1]/hook",
}
for _, target := range blocked {
t.Run(target, func(t *testing.T) {
t.Parallel()
cfg := &delivery.HTTPTargetConfig{
URL: target,
Timeout: 5,
}
client := engine.ExportClientForConfig(cfg)
require.NotSame(t, engine.ExportClient(), client,
"a per-target timeout must yield a "+
"distinct client",
)
assert.Equal(t,
5*time.Second, client.Timeout,
"the per-target timeout must be applied",
)
assert.Same(t,
engine.ExportClient().Transport,
client.Transport,
"the SSRF-safe transport must be reused, "+
"not dropped",
)
req, err := http.NewRequestWithContext(
context.Background(),
http.MethodPost, target, nil,
)
require.NoError(t, err)
resp, doErr := client.Do(req)
if resp != nil {
_ = resp.Body.Close()
}
require.Error(t, doErr,
"request to %s must be blocked", target,
)
assert.Contains(t, doErr.Error(), "blocked",
"error must come from the SSRF guard",
)
})
}
}
// TestClientForConfig_NoTimeoutUnchanged asserts that with
// no per-target timeout the shared SSRF-safe client is
// returned unchanged.
func TestClientForConfig_NoTimeoutUnchanged(t *testing.T) {
t.Parallel()
engine := newSSRFTestEngine()
cfg := &delivery.HTTPTargetConfig{
URL: "https://example.com/hook",
}
client := engine.ExportClientForConfig(cfg)
assert.Same(t, engine.ExportClient(), client,
"without a per-target timeout the shared client "+
"must be returned unchanged",
)
}

1158
internal/delivery/engine.go Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,271 @@
package delivery_test
import (
"context"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/require"
"go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
const (
// hookStopTimeout bounds how long a lifecycle test waits for
// the engine's OnStop hook to return before declaring the
// shutdown hung.
hookStopTimeout = 10 * time.Second
// hookSettleDelay is how long startEngineViaHook waits after
// OnStart before the caller may enqueue work. A worker pool
// wrongly rooted in the already-done hook context has nothing
// but ctx.Done() ready in its select, so it is deterministically
// gone by the end of this window. Without the wait, Notify would
// race the pool's very first select, in which a ready ctx.Done()
// and a ready deliveryCh are chosen between at random and a
// doomed pool still delivers.
hookSettleDelay = 250 * time.Millisecond
// wedgeStopTimeout is the stop timeout a wedged-shutdown test
// hands OnStop, standing in for fx's StopTimeout. The test
// asserts only that the hook returns at all, and allows it
// hookStopTimeout — forty times this budget — to do so, so no
// assertion here races the wall clock.
wedgeStopTimeout = 250 * time.Millisecond
)
// recordingLifecycle is a minimal fx.Lifecycle that records the
// hooks a component registers, so a test can invoke the real
// OnStart/OnStop functions with a context of its choosing.
type recordingLifecycle struct {
hooks []fx.Hook
}
func (l *recordingLifecycle) Append(h fx.Hook) {
l.hooks = append(l.hooks, h)
}
// requireStopHookExpires drives hook.OnStop with a stop context
// that expires while a wedged goroutine is still running, and
// requires the hook to return the deadline error naming
// component instead of blocking on the WaitGroup forever.
func requireStopHookExpires(
t *testing.T, hook fx.Hook, component string,
) {
t.Helper()
stopCtx, cancel := context.WithTimeout(
context.Background(), wedgeStopTimeout,
)
defer cancel()
var stopErr error
stopped := make(chan struct{})
go func() {
defer close(stopped)
stopErr = hook.OnStop(stopCtx)
}()
select {
case <-stopped:
case <-time.After(hookStopTimeout):
t.Fatal(
"OnStop did not return: it discarded the stop " +
"context and is waiting on a wedged goroutine " +
"that will never observe cancellation",
)
}
require.ErrorIs(t, stopErr, context.DeadlineExceeded)
require.ErrorContains(t, stopErr, component)
}
// startEngineViaHook drives the genuine fx hooks the application
// registers for the engine, handing OnStart a context that is
// already done, and returns only once a pool that inherited that
// context would have exited. It returns the recorded lifecycle so
// the caller can drive OnStop too.
//
// Callers must not seed pending or retrying deliveries before
// calling this: restart recovery enqueues those during startup,
// which would put work in the queue while the pool is still
// racing its first select.
func startEngineViaHook(
t *testing.T, eng *delivery.Engine,
) *recordingLifecycle {
t.Helper()
lc := &recordingLifecycle{}
eng.ExportRegisterHooks(lc)
require.Len(t, lc.hooks, 1)
// fx hands OnStart a context carrying the application start
// timeout, and cancels it when the start phase ends. An
// already-cancelled context is that same defect taken to its
// limit, and unlike a plain context.Background() it actually
// distinguishes a correctly rooted loop from a broken one.
hookCtx, cancel := context.WithCancel(context.Background())
cancel()
require.NoError(t, lc.hooks[0].OnStart(hookCtx))
time.Sleep(hookSettleDelay)
return lc
}
// seedLogTask seeds a pending delivery for a log target and
// returns its ID together with the task that drives it. The log
// target needs no network, so a delivery completing proves only
// that a worker picked the task up.
func seedLogTask(
t *testing.T, s iSetup,
) (string, delivery.Task) {
t.Helper()
event := iSeedEvent(
t, s.WebhookDB, s.WebhookID,
`{"lifecycle":"hook-context"}`,
)
targetID := uuid.New().String()
d := iSeedDelivery(
t, s.WebhookDB, event.ID, targetID,
database.DeliveryStatusPending,
)
bodyStr := event.Body
task := iTask(
d, event, s.WebhookID, targetID,
"hook-context-test", "", 0, 1, &bodyStr,
)
task.TargetType = database.TargetTypeLog
return d.ID, task
}
// TestEngine_WorkersOutliveStartHookContext is the regression
// test for a delivery engine that stopped delivering roughly
// fifteen seconds after boot. fx calls OnStart with a context
// carrying the application's start timeout (15s by default) and
// cancels it when the start phase ends, so a worker pool rooted
// in it exits shortly after startup: the process keeps accepting
// and persisting events while nothing at all forwards them.
//
// Driving OnStart with an already-cancelled context is that
// defect taken to its limit. A pool that inherits the hook
// context is gone before the task is even enqueued; a correctly
// rooted pool keeps working for as long as the process lives.
func TestEngine_WorkersOutliveStartHookContext(t *testing.T) {
t.Parallel()
s := newISetup(t)
lc := startEngineViaHook(t, s.Engine)
t.Cleanup(func() {
_ = lc.hooks[0].OnStop(context.Background())
})
// Seeded only after the pool has settled, so restart recovery
// cannot enqueue it during startup.
deliveryID, task := seedLogTask(t, s)
s.Engine.Notify([]delivery.Task{task})
iWaitForDelivered(t, s.WebhookDB, deliveryID)
}
// TestEngine_StopHookStopsWorkers proves the fix did not trade a
// startup bug for a shutdown hang: now that the worker pool no
// longer observes the start hook's cancellation, OnStop is the
// only thing that can stop it, and it must both return promptly
// and actually leave the pool drained.
func TestEngine_StopHookStopsWorkers(t *testing.T) {
t.Parallel()
s := newISetup(t)
lc := startEngineViaHook(t, s.Engine)
// Let the pool prove it is running before stopping it, so a
// fast OnStop cannot pass by stopping something already dead.
firstID, firstTask := seedLogTask(t, s)
s.Engine.Notify([]delivery.Task{firstTask})
iWaitForDelivered(t, s.WebhookDB, firstID)
var stopErr error
stopped := make(chan struct{})
go func() {
defer close(stopped)
// stop blocks on the workers' WaitGroup, so returning at
// all proves every goroutine observed the cancellation.
stopErr = lc.hooks[0].OnStop(context.Background())
}()
select {
case <-stopped:
case <-time.After(hookStopTimeout):
t.Fatal(
"OnStop did not return: the delivery engine's " +
"WaitGroup is still waiting on a goroutine that " +
"never observed cancellation",
)
}
require.NoError(t, stopErr)
// With every worker gone, a freshly notified task must sit
// untouched in the queue rather than being delivered.
secondID, secondTask := seedLogTask(t, s)
s.Engine.Notify([]delivery.Task{secondTask})
time.Sleep(200 * time.Millisecond)
var after database.Delivery
require.NoError(
t,
s.WebhookDB.First(&after, "id = ?", secondID).Error,
)
require.Equal(
t,
database.DeliveryStatusPending,
after.Status,
"a stopped engine must not deliver anything",
)
}
// TestEngine_StopHookHonoursStopTimeout is the regression test
// for a shutdown that could never complete. fx hands OnStop a
// context carrying the application's stop timeout; an OnStop
// that discards it and calls wg.Wait() bare hangs the process
// forever on a single worker stuck inside a delivery target that
// never returns — precisely when a bounded shutdown matters
// most.
//
// The wedged goroutine here never observes cancellation, so the
// hook can only return by honouring its context, and it must say
// so rather than reporting a clean stop.
func TestEngine_StopHookHonoursStopTimeout(t *testing.T) {
t.Parallel()
s := newISetup(t)
lc := startEngineViaHook(t, s.Engine)
release := make(chan struct{})
t.Cleanup(func() { close(release) })
s.Engine.ExportWedgeWorker(release)
requireStopHookExpires(t, lc.hooks[0], "delivery engine")
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,557 @@
package delivery
import (
"context"
"log/slog"
"net"
"net/http"
"time"
"go.uber.org/fx"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// ErrExportArchiveWriterEvicted exposes the sentinel returned by
// an evicted archive writer. It carries the Err prefix rather
// than this file's usual Export one because it is a sentinel
// error.
var ErrExportArchiveWriterEvicted = errArchiveWriterEvicted
// Exported constants for test access.
const (
ExportDeliveryChannelSize = deliveryChannelSize
ExportRetryChannelSize = retryChannelSize
ExportDefaultFailureThreshold = defaultFailureThreshold
ExportDefaultCooldown = defaultCooldown
)
// ExportIsBlockedIP exposes isBlockedIP for testing.
func ExportIsBlockedIP(ip net.IP) bool {
return isBlockedIP(ip)
}
// ExportBlockedNetworks exposes blockedNetworks.
func ExportBlockedNetworks() []*net.IPNet {
return blockedNetworks
}
// ExportIsForwardableHeader exposes isForwardableHeader.
func ExportIsForwardableHeader(name string) bool {
return isForwardableHeader(name)
}
// ExportTruncate exposes truncate for testing.
func ExportTruncate(s string, maxLen int) string {
return truncate(s, maxLen)
}
// ExportDeliverHTTP delivers via the http target for testing.
func (e *Engine) ExportDeliverHTTP(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
) {
e.httpTarget.Deliver(ctx, webhookDB, d, task, e)
}
// ExportDeliverDatabase delivers via the database target.
func (e *Engine) ExportDeliverDatabase(
webhookDB *gorm.DB, d *database.Delivery,
) {
e.targets[database.TargetTypeDatabase].Deliver(
context.Background(), webhookDB, d, &Task{}, e,
)
}
// ExportDeliverLog delivers via the log target for testing.
func (e *Engine) ExportDeliverLog(
webhookDB *gorm.DB, d *database.Delivery,
) {
e.targets[database.TargetTypeLog].Deliver(
context.Background(), webhookDB, d, &Task{}, e,
)
}
// ExportDeliverSlack delivers via the slack target for
// testing.
func (e *Engine) ExportDeliverSlack(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
) {
task := &Task{
DeliveryID: d.ID,
TargetID: d.TargetID,
AttemptNum: 1,
}
e.targets[database.TargetTypeSlack].Deliver(
ctx, webhookDB, d, task, e,
)
}
// ExportProcessNewTask exposes processNewTask.
func (e *Engine) ExportProcessNewTask(
ctx context.Context, task *Task,
) {
e.processNewTask(ctx, task)
}
// ExportProcessRetryTask exposes processRetryTask.
func (e *Engine) ExportProcessRetryTask(
ctx context.Context, task *Task,
) {
e.processRetryTask(ctx, task)
}
// ExportProcessDelivery exposes processDelivery.
func (e *Engine) ExportProcessDelivery(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
) {
e.processDelivery(ctx, webhookDB, d, task)
}
// ExportGetCircuitBreaker exposes the http target's
// getCircuitBreaker.
func (e *Engine) ExportGetCircuitBreaker(
targetID string,
) *CircuitBreaker {
return e.httpTarget.getCircuitBreaker(targetID)
}
// ExportParseHTTPConfig exposes parseHTTPConfig.
func (e *Engine) ExportParseHTTPConfig(
configJSON string,
) (*HTTPTargetConfig, error) {
return parseHTTPConfig(configJSON)
}
// ExportParseSlackConfig exposes parseSlackConfig.
func (e *Engine) ExportParseSlackConfig(
configJSON string,
) (*SlackTargetConfig, error) {
return parseSlackConfig(configJSON)
}
// ExportDoHTTPRequest exposes the http target's
// doHTTPRequest.
func (e *Engine) ExportDoHTTPRequest(
ctx context.Context,
cfg *HTTPTargetConfig,
event *database.Event,
) (int, string, int64, error) {
return e.httpTarget.doHTTPRequest(ctx, cfg, event)
}
// ExportClientForConfig exposes the http target's
// clientForConfig.
func (e *Engine) ExportClientForConfig(
cfg *HTTPTargetConfig,
) *http.Client {
return e.httpTarget.clientForConfig(cfg)
}
// ExportClient returns the http target's shared HTTP client.
func (e *Engine) ExportClient() *http.Client {
return e.httpTarget.client
}
// ExportScheduleRetry exposes ScheduleRetry.
func (e *Engine) ExportScheduleRetry(
task Task, delay time.Duration,
) {
e.ScheduleRetry(task, delay)
}
// ExportRecoverPendingDeliveries exposes
// recoverPendingDeliveries.
func (e *Engine) ExportRecoverPendingDeliveries(
ctx context.Context,
webhookDB *gorm.DB,
webhookID string,
) {
e.recoverPendingDeliveries(
ctx, webhookDB, webhookID,
)
}
// ExportRecoverWebhookDeliveries exposes
// recoverWebhookDeliveries.
func (e *Engine) ExportRecoverWebhookDeliveries(
ctx context.Context, webhookID string,
) {
e.recoverWebhookDeliveries(ctx, webhookID)
}
// ExportRecoverInFlight exposes recoverInFlight.
func (e *Engine) ExportRecoverInFlight(
ctx context.Context,
) {
e.recoverInFlight(ctx)
}
// ExportSweepWebhookRetries exposes sweepWebhookRetries.
func (e *Engine) ExportSweepWebhookRetries(
ctx context.Context, webhookID string,
) {
e.sweepWebhookRetries(ctx, webhookID)
}
// ExportStart exposes start for testing.
func (e *Engine) ExportStart() {
e.start()
}
// ExportRegisterHooks registers the engine's real fx lifecycle
// hooks on a lifecycle supplied by a test, so a test can drive
// the exact OnStart/OnStop functions the application runs and
// hand OnStart the kind of context fx actually supplies.
func (e *Engine) ExportRegisterHooks(lc fx.Lifecycle) {
e.registerHooks(lc)
}
// ExportStop exposes stop for testing.
func (e *Engine) ExportStop(ctx context.Context) error {
return e.stop(ctx)
}
// ExportWedgeWorker adds a goroutine to the engine's WaitGroup
// that never observes cancellation and returns only when release
// is closed. It stands in for a worker stuck inside a delivery
// target that never returns, which is the only way stop can be
// made to outlast its context.
func (e *Engine) ExportWedgeWorker(release <-chan struct{}) {
e.wg.Go(func() {
<-release
})
}
// ExportDeliveryCh returns the delivery channel.
func (e *Engine) ExportDeliveryCh() chan Task {
return e.deliveryCh
}
// ExportRetryCh returns the retry channel.
func (e *Engine) ExportRetryCh() chan Task {
return e.retryCh
}
// NewTestEngine creates an Engine for unit tests without
// database dependencies.
func NewTestEngine(
log *slog.Logger,
client *http.Client,
workers int,
) *Engine {
e := &Engine{
log: log,
deliveryCh: make(chan Task, deliveryChannelSize),
retryCh: make(chan Task, retryChannelSize),
workers: workers,
}
e.initTargets(client)
return e
}
// NewTestEngineSmallRetry creates an Engine with a tiny
// retry channel buffer for overflow testing.
func NewTestEngineSmallRetry(
log *slog.Logger,
) *Engine {
e := &Engine{
log: log,
retryCh: make(chan Task, 1),
}
e.initTargets(nil)
return e
}
// NewTestEngineWithDB creates an Engine with a real
// database and dbManager for integration tests.
func NewTestEngineWithDB(
db *database.Database,
dbMgr *database.WebhookDBManager,
log *slog.Logger,
client *http.Client,
workers int,
) *Engine {
e := &Engine{
database: db,
dbManager: dbMgr,
log: log,
deliveryCh: make(chan Task, deliveryChannelSize),
retryCh: make(chan Task, retryChannelSize),
workers: workers,
}
e.initTargets(client)
return e
}
// NewTestCircuitBreaker creates a CircuitBreaker with
// custom settings for testing.
func NewTestCircuitBreaker(
threshold int, cooldown time.Duration,
) *CircuitBreaker {
return &CircuitBreaker{
state: CircuitClosed,
threshold: threshold,
cooldown: cooldown,
}
}
// ExportArchivedEvent aliases the archive row type so black-box
// tests can construct and read archive rows.
type ExportArchivedEvent = archivedEvent
// ExportArchiveWriter wraps an archiveWriter so black-box tests
// can exercise the per-webhook archive file mechanics.
type ExportArchiveWriter struct {
w *archiveWriter
}
// NewExportArchiveWriter builds an archive writer for tests,
// optionally overriding the reopen debounce (a non-positive
// debounce keeps the production default).
func NewExportArchiveWriter(
path string, log *slog.Logger, debounce time.Duration,
) *ExportArchiveWriter {
w := newArchiveWriter(path, log)
if debounce > 0 {
w.debounce = debounce
}
return &ExportArchiveWriter{w: w}
}
// Write archives a row through the writer.
func (e *ExportArchiveWriter) Write(
row ExportArchivedEvent, expiry time.Duration,
) error {
return e.w.write(row, expiry)
}
// Open opens the archive file, pruning when expiry is positive.
func (e *ExportArchiveWriter) Open(expiry time.Duration) error {
return e.w.open(expiry)
}
// Reopen closes and reopens the archive file.
func (e *ExportArchiveWriter) Reopen(
expiry time.Duration,
) error {
return e.w.reopen(expiry)
}
// Reopens reports how many times the file has been opened.
func (e *ExportArchiveWriter) Reopens() int {
return e.w.reopens
}
// DB returns the writer's current open handle for row
// inspection in tests.
func (e *ExportArchiveWriter) DB() *gorm.DB {
return e.w.db
}
// Path returns the archive file the writer owns.
func (e *ExportArchiveWriter) Path() string {
return e.w.path
}
// OpenExisting opens the archive without permitting creation,
// the way the idle sweep does.
func (e *ExportArchiveWriter) OpenExisting(
expiry time.Duration,
) error {
return e.w.openMode(archiveModeExisting, expiry)
}
// SweepExpired runs an idle sweep of the archive.
func (e *ExportArchiveWriter) SweepExpired(
expiry time.Duration,
) error {
return e.w.sweepExpired(expiry)
}
// Evict marks the writer evicted and closes its handle, exactly
// as leaving the registry does.
func (e *ExportArchiveWriter) Evict() {
e.w.evict()
}
// HandleOpen reports whether the writer currently holds an open
// archive handle.
func (e *ExportArchiveWriter) HandleOpen() bool {
e.w.mu.Lock()
defer e.w.mu.Unlock()
return e.w.db != nil
}
// Same reports whether both wrappers refer to the very same
// underlying archive writer, so a test can prove a registry entry
// is the writer it was handed rather than a replacement.
func (e *ExportArchiveWriter) Same(
other *ExportArchiveWriter,
) bool {
return other != nil && e.w == other.w
}
// ExportArchiveWriterFor returns the archive writer the registry
// currently caches for a webhook, or nil when none is cached. It
// never creates one, so a test can hold a reference to the very
// writer an eviction is about to detach.
func (e *Engine) ExportArchiveWriterFor(
webhookID string,
) *ExportArchiveWriter {
e.dbTarget.mu.Lock()
defer e.dbTarget.mu.Unlock()
w, ok := e.dbTarget.writers[webhookID]
if !ok {
return nil
}
return &ExportArchiveWriter{w: w}
}
// ExportHasArchiveWriter reports whether the database target
// currently caches an archive writer for a webhook.
func (e *Engine) ExportHasArchiveWriter(
webhookID string,
) bool {
e.dbTarget.mu.Lock()
defer e.dbTarget.mu.Unlock()
_, ok := e.dbTarget.writers[webhookID]
return ok
}
// ExportArchiveHandleOpen reports whether the cached archive
// writer for a webhook holds an open database handle. It
// returns false when no writer is cached.
func (e *Engine) ExportArchiveHandleOpen(
webhookID string,
) bool {
e.dbTarget.mu.Lock()
w, ok := e.dbTarget.writers[webhookID]
e.dbTarget.mu.Unlock()
if !ok {
return false
}
w.mu.Lock()
defer w.mu.Unlock()
return w.db != nil
}
// ExportEnsureArchiveWriter creates (if needed) and returns the
// archive file path of the cached writer for a webhook, so a
// test can prime the registry the way a delivery would.
func (e *Engine) ExportEnsureArchiveWriter(
webhookID string,
) (string, error) {
w, err := e.dbTarget.writerFor(webhookID)
if err != nil {
return "", err
}
return w.path, nil
}
// ExportSweepWriterFor takes a webhook's registry writer exactly
// as the idle sweep does, reporting whether the sweep had to
// create the entry. It lets a test drive the registry through the
// sweep's own entry point instead of choreographing goroutines.
func (e *Engine) ExportSweepWriterFor(
webhookID string,
) (*ExportArchiveWriter, bool, error) {
w, created, err := e.dbTarget.sweepWriterFor(webhookID)
if err != nil {
return nil, false, err
}
return &ExportArchiveWriter{w: w}, created, nil
}
// ExportReleaseSweepWriter releases a sweep-created registry entry
// exactly as a finished sweep does.
func (e *Engine) ExportReleaseSweepWriter(
webhookID string, w *ExportArchiveWriter,
) {
e.dbTarget.releaseSweepWriter(webhookID, w.w)
}
// NewTestArchiveSweeper builds an ArchiveSweeper backed by the
// given main database and engine, without the fx lifecycle.
// Intended for tests.
func NewTestArchiveSweeper(
db *database.Database,
eng *Engine,
log *slog.Logger,
) *ArchiveSweeper {
return &ArchiveSweeper{
db: db,
eng: eng,
log: log,
interval: time.Hour,
}
}
// ExportSweep runs a single archive sweep synchronously for
// tests.
func (s *ArchiveSweeper) ExportSweep(ctx context.Context) {
s.sweep(ctx)
}
// ExportStart starts the sweeper's background loop for tests.
func (s *ArchiveSweeper) ExportStart() {
s.start()
}
// ExportRegisterHooks registers the sweeper's real fx lifecycle
// hooks on a lifecycle supplied by a test, so a test can drive
// the exact OnStart/OnStop functions the application runs and
// hand OnStart the kind of context fx actually supplies.
func (s *ArchiveSweeper) ExportRegisterHooks(lc fx.Lifecycle) {
s.registerHooks(lc)
}
// ExportStop stops the sweeper's background loop for tests.
func (s *ArchiveSweeper) ExportStop(ctx context.Context) error {
return s.stop(ctx)
}
// ExportWedgeLoop adds a goroutine to the sweeper's WaitGroup
// that never observes cancellation and returns only when release
// is closed. It stands in for a prune stuck on a locked archive.
func (s *ArchiveSweeper) ExportWedgeLoop(
release <-chan struct{},
) {
s.wg.Go(func() {
<-release
})
}
// ExportSetInterval overrides the sweep interval for tests.
func (s *ArchiveSweeper) ExportSetInterval(d time.Duration) {
s.interval = d
}
// ExportParseArchiveExpiry exposes parseArchiveExpiry.
func ExportParseArchiveExpiry(
configJSON string,
) (time.Duration, error) {
return parseArchiveExpiry(configJSON)
}

227
internal/delivery/ssrf.go Normal file
View File

@@ -0,0 +1,227 @@
package delivery
import (
"context"
"errors"
"fmt"
"net"
"net/http"
"net/url"
"time"
)
const (
// dnsResolutionTimeout is the maximum time to wait for
// DNS resolution during SSRF validation.
dnsResolutionTimeout = 5 * time.Second
)
// Sentinel errors for SSRF validation.
var (
errNoHostname = errors.New("URL has no hostname")
errNoIPs = errors.New(
"hostname resolved to no IP addresses",
)
errBlockedIP = errors.New(
"blocked private/reserved IP range",
)
errInvalidScheme = errors.New(
"only http and https are allowed",
)
)
// blockedNetworks contains all private/reserved IP ranges
// that should be blocked to prevent SSRF attacks.
//
//nolint:gochecknoglobals // package-level network list is appropriate here
var blockedNetworks []*net.IPNet
//nolint:gochecknoinits // init is the idiomatic way to parse CIDRs once at startup
func init() {
cidrs := []string{
"127.0.0.0/8",
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16",
"169.254.0.0/16",
"0.0.0.0/8",
"100.64.0.0/10",
"192.0.0.0/24",
"192.0.2.0/24",
"198.18.0.0/15",
"198.51.100.0/24",
"203.0.113.0/24",
"224.0.0.0/4",
"240.0.0.0/4",
"::1/128",
"fc00::/7",
"fe80::/10",
}
for _, cidr := range cidrs {
_, network, err := net.ParseCIDR(cidr)
if err != nil {
panic(fmt.Sprintf(
"ssrf: failed to parse CIDR %q: %v",
cidr, err,
))
}
blockedNetworks = append(
blockedNetworks, network,
)
}
}
// isBlockedIP checks whether an IP address falls within
// any blocked private/reserved network range.
func isBlockedIP(ip net.IP) bool {
for _, network := range blockedNetworks {
if network.Contains(ip) {
return true
}
}
return false
}
// ValidateTargetURL checks that an HTTP delivery target
// URL is safe from SSRF attacks.
func ValidateTargetURL(
ctx context.Context, targetURL string,
) error {
parsed, err := url.Parse(targetURL)
if err != nil {
// url.Parse embeds the whole URL in its error, and
// this one is logged and shown; mask it. Every other
// branch below reports only the hostname.
return fmt.Errorf(
"invalid URL: %w", maskURLError(err),
)
}
err = validateScheme(parsed.Scheme)
if err != nil {
return err
}
host := parsed.Hostname()
if host == "" {
return errNoHostname
}
if ip := net.ParseIP(host); ip != nil {
return checkBlockedIP(ip)
}
return validateHostname(ctx, host)
}
func validateScheme(scheme string) error {
if scheme != "http" && scheme != "https" {
return fmt.Errorf(
"unsupported URL scheme %q: %w",
scheme, errInvalidScheme,
)
}
return nil
}
func checkBlockedIP(ip net.IP) error {
if isBlockedIP(ip) {
return fmt.Errorf(
"target IP %s is in a blocked "+
"private/reserved range: %w",
ip, errBlockedIP,
)
}
return nil
}
func validateHostname(
ctx context.Context, host string,
) error {
dnsCtx, cancel := context.WithTimeout(
ctx, dnsResolutionTimeout,
)
defer cancel()
ips, err := net.DefaultResolver.LookupIPAddr(
dnsCtx, host,
)
if err != nil {
return fmt.Errorf(
"failed to resolve hostname %q: %w",
host, err,
)
}
if len(ips) == 0 {
return fmt.Errorf(
"hostname %q: %w", host, errNoIPs,
)
}
for _, ipAddr := range ips {
if isBlockedIP(ipAddr.IP) {
return fmt.Errorf(
"hostname %q resolves to blocked "+
"IP %s: %w",
host, ipAddr.IP, errBlockedIP,
)
}
}
return nil
}
// NewSSRFSafeTransport creates an http.Transport with a
// custom DialContext that blocks connections to
// private/reserved IP addresses.
func NewSSRFSafeTransport() *http.Transport {
return &http.Transport{
DialContext: ssrfDialContext,
}
}
func ssrfDialContext(
ctx context.Context,
network, addr string,
) (net.Conn, error) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
return nil, fmt.Errorf(
"ssrf: invalid address %q: %w",
addr, err,
)
}
ips, err := net.DefaultResolver.LookupIPAddr(
ctx, host,
)
if err != nil {
return nil, fmt.Errorf(
"ssrf: DNS resolution failed for %q: %w",
host, err,
)
}
for _, ipAddr := range ips {
if isBlockedIP(ipAddr.IP) {
return nil, fmt.Errorf(
"ssrf: connection to %s (%s) "+
"blocked: %w",
host, ipAddr.IP, errBlockedIP,
)
}
}
var dialer net.Dialer
return dialer.DialContext(
ctx, network,
net.JoinHostPort(ips[0].IP.String(), port),
)
}

View File

@@ -0,0 +1,172 @@
package delivery_test
import (
"context"
"net"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/delivery"
)
func TestIsBlockedIP_PrivateRanges(t *testing.T) {
t.Parallel()
tests := []struct {
name string
ip string
blocked bool
}{
{"loopback 127.0.0.1", "127.0.0.1", true},
{"loopback 127.0.0.2", "127.0.0.2", true},
{"loopback 127.255.255.255", "127.255.255.255", true},
{"10.0.0.0", "10.0.0.0", true},
{"10.0.0.1", "10.0.0.1", true},
{"10.255.255.255", "10.255.255.255", true},
{"172.16.0.1", "172.16.0.1", true},
{"172.31.255.255", "172.31.255.255", true},
{"172.15.255.255", "172.15.255.255", false},
{"172.32.0.0", "172.32.0.0", false},
{"192.168.0.1", "192.168.0.1", true},
{"192.168.255.255", "192.168.255.255", true},
{"169.254.0.1", "169.254.0.1", true},
{"169.254.169.254", "169.254.169.254", true},
{"8.8.8.8", "8.8.8.8", false},
{"1.1.1.1", "1.1.1.1", false},
{"93.184.216.34", "93.184.216.34", false},
{"::1", "::1", true},
{"fd00::1", "fd00::1", true},
{"fc00::1", "fc00::1", true},
{"fe80::1", "fe80::1", true},
{
"2607:f8b0:4004:800::200e",
"2607:f8b0:4004:800::200e",
false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
ip := net.ParseIP(tt.ip)
require.NotNil(t, ip,
"failed to parse IP %s", tt.ip,
)
assert.Equal(t,
tt.blocked,
delivery.ExportIsBlockedIP(ip),
"isBlockedIP(%s) = %v, want %v",
tt.ip,
delivery.ExportIsBlockedIP(ip),
tt.blocked,
)
})
}
}
func TestValidateTargetURL_Blocked(t *testing.T) {
t.Parallel()
blockedURLs := []string{
"http://127.0.0.1/hook",
"http://127.0.0.1:8080/hook",
"https://10.0.0.1/hook",
"http://192.168.1.1/webhook",
"http://172.16.0.1/api",
"http://169.254.169.254/latest/meta-data/",
"http://[::1]/hook",
"http://[fc00::1]/hook",
"http://[fe80::1]/hook",
"http://0.0.0.0/hook",
}
for _, u := range blockedURLs {
t.Run(u, func(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), u,
)
assert.Error(t, err,
"URL %s should be blocked", u,
)
})
}
}
func TestValidateTargetURL_Allowed(t *testing.T) {
t.Parallel()
allowedURLs := []string{
"https://example.com/hook",
"http://93.184.216.34/webhook",
"https://hooks.slack.com/services/T00/B00/xxx",
}
for _, u := range allowedURLs {
t.Run(u, func(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), u,
)
assert.NoError(t, err,
"URL %s should be allowed", u,
)
})
}
}
func TestValidateTargetURL_InvalidScheme(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), "ftp://example.com/hook",
)
require.Error(t, err)
assert.Contains(t, err.Error(),
"unsupported URL scheme",
)
}
func TestValidateTargetURL_EmptyHost(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), "http:///path",
)
assert.Error(t, err)
}
func TestValidateTargetURL_InvalidURL(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), "://invalid",
)
assert.Error(t, err)
}
func TestBlockedNetworks_Initialized(t *testing.T) {
t.Parallel()
nets := delivery.ExportBlockedNetworks()
assert.NotEmpty(t, nets,
"blockedNetworks should be initialized",
)
assert.GreaterOrEqual(t, len(nets), 8,
"should have at least 8 blocked network ranges",
)
}

104
internal/delivery/target.go Normal file
View File

@@ -0,0 +1,104 @@
package delivery
import (
"context"
"net/http"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// Scheduler re-enqueues a task for a future delivery attempt.
// The engine provides one to each target so a target can own
// its retries durably: it records the attempt, marks the
// delivery retrying, and asks the Scheduler to deliver the
// next attempt after delay — exactly what the engine does for
// its own restart recovery.
type Scheduler interface {
ScheduleRetry(task Task, delay time.Duration)
}
// Target delivers an event to one target type. Each type is
// an implementation. A Target owns its whole delivery: it
// makes the attempt, records the DeliveryResult and updates
// the DeliveryStatus, and — for targets that retry — decides
// whether to retry, computes its own backoff, gates with its
// own circuit breaker, and reschedules via the injected
// Scheduler. Fire-and-forget targets simply record a single
// attempt.
type Target interface {
Deliver(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
)
}
// rescheduler is implemented by targets that own durable
// retries. The engine's restart recovery and periodic sweep
// use it to let the target recompute the schedule for an
// orphaned retrying delivery, keeping the retry schedule
// target-owned. Fire-and-forget targets do not implement it
// and their (never-occurring) retrying deliveries are
// skipped.
type rescheduler interface {
// remainingBackoff returns how long to wait before the
// next attempt of a recovered retrying delivery.
remainingBackoff(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) time.Duration
// backoffElapsed reports whether the backoff window for
// the last attempt has already passed, so the periodic
// sweep can re-enqueue the delivery now.
backoffElapsed(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) bool
}
// attemptResult is the outcome of a single delivery attempt,
// as reported by a target's per-attempt function to the
// shared retry core.
type attemptResult struct {
statusCode int
respBody string
duration int64
success bool
errMsg string
}
// initTargets builds the target registry, wiring each target
// to the engine's persistence helpers and giving the HTTP and
// Slack targets the shared SSRF-safe client. It is called by
// both New and the test constructors so the registry is
// always populated.
func (e *Engine) initTargets(client *http.Client) {
httpT := &httpTarget{
httpCore: &httpCore{eng: e},
client: client,
}
slackT := &slackTarget{
httpCore: &httpCore{eng: e},
client: client,
}
dbT := &databaseTarget{eng: e}
e.httpTarget = httpT
e.dbTarget = dbT
e.targets = map[database.TargetType]Target{
database.TargetTypeHTTP: httpT,
database.TargetTypeSlack: slackT,
database.TargetTypeDatabase: dbT,
database.TargetTypeLog: &logTarget{eng: e},
}
}

View File

@@ -0,0 +1,202 @@
package delivery
import (
"encoding/json"
"fmt"
"strconv"
"sneak.berlin/go/webhooker/internal/database"
)
// configUnavailable is what a target's configuration renders
// as when it is absent, of an unknown type, or does not
// parse. The stored blob is never shown as a fallback: it can
// hold a credential (a Slack incoming webhook URL is a bearer
// token) and a UI that prints it leaks that credential into
// browser history, screenshots and screen shares.
const configUnavailable = "(unavailable)"
// ConfigField is one labelled, display-safe value derived
// from a target's stored configuration.
type ConfigField struct {
Label string
Value string
}
// TargetView is the display-safe projection of a target for
// the UI. It deliberately has no raw configuration field, so
// no template — present or future — can render the stored
// blob.
type TargetView struct {
ID string
Name string
Type database.TargetType
Active bool
Config []ConfigField
}
// NewTargetViews projects targets for rendering, replacing
// each stored configuration blob with named, display-safe
// fields.
func NewTargetViews(
targets []database.Target,
) []TargetView {
views := make([]TargetView, 0, len(targets))
for i := range targets {
t := &targets[i]
views = append(views, TargetView{
ID: t.ID,
Name: t.Name,
Type: t.Type,
Active: t.Active,
Config: targetConfigFields(t),
})
}
return views
}
// targetConfigFields returns the display-safe fields for a
// target's configuration. Anything it cannot parse becomes
// the neutral placeholder.
func targetConfigFields(
t *database.Target,
) []ConfigField {
switch t.Type {
case database.TargetTypeSlack:
return slackConfigFields(t.Config)
case database.TargetTypeHTTP:
return httpConfigFields(t)
case database.TargetTypeDatabase:
return databaseConfigFields(t.Config)
case database.TargetTypeLog:
// The log target takes no configuration.
return nil
default:
return unavailableConfigFields()
}
}
// unavailableConfigFields is the neutral placeholder shown
// for a configuration that could not be presented.
func unavailableConfigFields() []ConfigField {
return []ConfigField{{
Label: "Configuration",
Value: configUnavailable,
}}
}
// slackConfigFields describes a Slack target. Only the masked
// webhook URL is shown; the full URL is the credential.
func slackConfigFields(configJSON string) []ConfigField {
cfg, err := parseSlackConfig(configJSON)
if err != nil {
return unavailableConfigFields()
}
return []ConfigField{{
Label: "Webhook URL",
Value: cfg.MaskedWebhookURL(),
}}
}
// httpConfigFields describes an HTTP target: its destination
// and its retry settings. Header values are not shown — they
// routinely carry authorization tokens — only how many are
// configured.
func httpConfigFields(t *database.Target) []ConfigField {
cfg, err := parseHTTPConfig(t.Config)
if err != nil {
return unavailableConfigFields()
}
fields := []ConfigField{{
Label: "Destination URL",
Value: cfg.URL,
}}
if cfg.Timeout > 0 {
fields = append(fields, ConfigField{
Label: "Timeout",
Value: strconv.Itoa(cfg.Timeout) + "s",
})
}
if len(cfg.Headers) > 0 {
fields = append(fields, ConfigField{
Label: "Headers",
Value: fmt.Sprintf(
"%d configured", len(cfg.Headers),
),
})
}
return append(fields, retryFields(t)...)
}
// retryFields describes a target's retry settings, which live
// on the target row rather than in its configuration blob.
func retryFields(t *database.Target) []ConfigField {
retries := strconv.Itoa(t.MaxRetries)
if t.MaxRetries == 0 {
retries += " (fire-and-forget)"
}
fields := []ConfigField{{
Label: "Max Retries",
Value: retries,
}}
if t.MaxQueueSize > 0 {
fields = append(fields, ConfigField{
Label: "Max Queue Size",
Value: strconv.Itoa(t.MaxQueueSize),
})
}
return fields
}
// databaseConfigFields describes an archive target. Its
// configuration is optional, and an absent or empty expiry
// means the archive is kept forever. An expiry that is set
// but not a valid duration is reported as unavailable rather
// than echoed back.
func databaseConfigFields(configJSON string) []ConfigField {
expiry := archiveExpiryNever
if configJSON != "" {
var cfg databaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return unavailableConfigFields()
}
if cfg.Expiry != "" {
if ValidateArchiveExpiry(cfg.Expiry) != nil {
return unavailableConfigFields()
}
expiry = cfg.Expiry
}
}
return []ConfigField{{
Label: "Archive Expiry",
Value: expiry,
}}
}
// MaskedWebhookURL returns the Slack webhook URL reduced to
// its scheme and host, with the path, query and any userinfo
// elided. The path segments are the credential, so none of
// them is shown: the field accepts an arbitrary URL, so no
// segment can be assumed non-secret. A URL that does not
// parse into a scheme and host yields the neutral
// placeholder, never the raw string.
func (c *SlackTargetConfig) MaskedWebhookURL() string {
return MaskURL(c.WebhookURL)
}

View File

@@ -0,0 +1,299 @@
package delivery_test
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
const (
// slackSecretPath is the credential-bearing part of a
// Slack incoming webhook URL: everything after the host.
slackSecretPath = "/services/T00000000/B00000000/" +
"XXXXXXXXXXXXXXXXXXXXXXXX"
slackWebhookURL = "https://hooks.slack.com" +
slackSecretPath
viewExampleOrigin = "https://example.com"
viewExampleHook = viewExampleOrigin + "/hook"
viewUnavailable = "(unavailable)"
viewExpiryNever = "never"
)
func TestMaskedWebhookURL(t *testing.T) {
t.Parallel()
tests := map[string]struct {
url string
want string
}{
"slack webhook": {
url: slackWebhookURL,
want: "https://hooks.slack.com/...",
},
"query string dropped": {
url: viewExampleOrigin + "/a?token=secret",
want: viewExampleOrigin + "/...",
},
// Fabricated userinfo in a test URL, not a real
// credential.
//nolint:gosec // G101
"userinfo dropped": {
url: "https://user:pw@example.com/a/b",
want: viewExampleOrigin + "/...",
},
"no path": {
url: viewExampleOrigin,
want: viewExampleOrigin,
},
"root path": {
url: viewExampleOrigin + "/",
want: viewExampleOrigin,
},
"not a url": {
url: "definitely not a url",
want: viewUnavailable,
},
"empty": {
url: "",
want: viewUnavailable,
},
}
for name, tc := range tests {
t.Run(name, func(t *testing.T) {
t.Parallel()
cfg := &delivery.SlackTargetConfig{
WebhookURL: tc.url,
}
assert.Equal(
t, tc.want, cfg.MaskedWebhookURL(),
)
})
}
}
// TestMaskedWebhookURL_NeverLeaksPath is the direct
// expression of the rule: whatever the input, the masked
// value never contains a path segment of it.
func TestMaskedWebhookURL_NeverLeaksPath(t *testing.T) {
t.Parallel()
cfg := &delivery.SlackTargetConfig{
WebhookURL: slackWebhookURL,
}
masked := cfg.MaskedWebhookURL()
assert.NotContains(t, masked, "T00000000")
assert.NotContains(t, masked, "B00000000")
assert.NotContains(
t, masked, "XXXXXXXXXXXXXXXXXXXXXXXX",
)
assert.NotContains(t, masked, slackSecretPath)
}
// fieldMap turns a view's config fields into a lookup so
// assertions read by label.
func fieldMap(fields []delivery.ConfigField) map[string]string {
out := make(map[string]string, len(fields))
for _, f := range fields {
out[f.Label] = f.Value
}
return out
}
// viewFor projects a single target and returns its view.
func viewFor(
t *testing.T,
target database.Target,
) delivery.TargetView {
t.Helper()
views := delivery.NewTargetViews(
[]database.Target{target},
)
require.Len(t, views, 1)
return views[0]
}
func TestNewTargetViews_Slack(t *testing.T) {
t.Parallel()
view := viewFor(t, database.Target{
Name: "slack-target",
Type: database.TargetTypeSlack,
Active: true,
Config: `{"webhookUrl":"` +
slackWebhookURL + `"}`,
})
assert.Equal(t, "slack-target", view.Name)
assert.Equal(
t,
map[string]string{
"Webhook URL": "https://hooks.slack.com/...",
},
fieldMap(view.Config),
)
}
func TestNewTargetViews_HTTP(t *testing.T) {
t.Parallel()
view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"` + viewExampleHook + `",` +
`"timeout":30,` +
`"headers":{"Authorization":"Bearer sekrit"}}`,
MaxRetries: 5,
MaxQueueSize: 100,
})
fields := fieldMap(view.Config)
assert.Equal(
t,
map[string]string{
"Destination URL": viewExampleHook,
"Timeout": "30s",
"Headers": "1 configured",
"Max Retries": "5",
"Max Queue Size": "100",
},
fields,
)
// Header values can be credentials and are never shown.
for _, v := range fields {
assert.NotContains(t, v, "sekrit")
}
}
func TestNewTargetViews_HTTPFireAndForget(t *testing.T) {
t.Parallel()
view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"` + viewExampleHook + `"}`,
})
assert.Equal(
t,
map[string]string{
"Destination URL": viewExampleHook,
"Max Retries": "0 (fire-and-forget)",
},
fieldMap(view.Config),
)
}
func TestNewTargetViews_Database(t *testing.T) {
t.Parallel()
tests := map[string]struct {
config string
want string
}{
"empty config": {config: "", want: viewExpiryNever},
"empty expiry": {config: `{}`, want: viewExpiryNever},
"explicit": {
config: `{"expiry":"720h"}`,
want: "720h",
},
"never literal": {
config: `{"expiry":"` + viewExpiryNever + `"}`,
want: viewExpiryNever,
},
}
for name, tc := range tests {
t.Run(name, func(t *testing.T) {
t.Parallel()
view := viewFor(t, database.Target{
Type: database.TargetTypeDatabase,
Config: tc.config,
})
assert.Equal(
t,
map[string]string{"Archive Expiry": tc.want},
fieldMap(view.Config),
)
})
}
}
func TestNewTargetViews_Log(t *testing.T) {
t.Parallel()
view := viewFor(t, database.Target{
Type: database.TargetTypeLog,
Config: "",
})
assert.Empty(t, view.Config)
}
// TestNewTargetViews_Unpresentable proves that no config the
// view cannot present falls back to the stored blob.
func TestNewTargetViews_Unpresentable(t *testing.T) {
t.Parallel()
const blob = `{"webhookUrl":"https://hooks.slack.com` +
slackSecretPath + `"`
tests := map[string]database.Target{
"unknown target type": {
Type: database.TargetType("carrier-pigeon"),
Config: blob,
},
"unparseable json": {
Type: database.TargetTypeSlack,
Config: blob,
},
"empty slack config": {
Type: database.TargetTypeSlack,
},
"slack config without url": {
Type: database.TargetTypeSlack,
Config: `{}`,
},
"unparseable http json": {
Type: database.TargetTypeHTTP,
Config: `{"url":`,
},
"unparseable archive json": {
Type: database.TargetTypeDatabase,
Config: `{"expiry":`,
},
"invalid archive expiry": {
Type: database.TargetTypeDatabase,
Config: `{"expiry":"a fortnight"}`,
},
}
for name, target := range tests {
t.Run(name, func(t *testing.T) {
t.Parallel()
view := viewFor(t, target)
assert.Equal(
t,
map[string]string{
"Configuration": viewUnavailable,
},
fieldMap(view.Config),
)
})
}
}

View File

@@ -0,0 +1,295 @@
package delivery
import (
"context"
"fmt"
"path/filepath"
"sync"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// databaseTarget is a no-retry target that archives the
// full inbound event into a per-webhook archive SQLite file,
// separate from the per-webhook event database. The event is
// already persisted in the per-webhook event DB by the time
// delivery runs; the database target additionally writes a
// durable long-term copy into archive-{webhookID}.db and then
// records a single attempt whose outcome reflects whether the
// archive write succeeded. See archiveWriter for the
// close/reopen, auto-recreate, and expiry semantics.
type databaseTarget struct {
eng *Engine
mu sync.Mutex
writers map[string]*archiveWriter
}
// Deliver implements Target. It archives the event, then
// records one successful attempt and marks the delivery
// delivered. An archiving error fails the delivery: the
// attempt is recorded as failed with the error and the
// delivery is marked failed, so a target that could not do
// its one job (archiving) never reports success. The target
// does not retry; the event remains durably stored in the
// per-webhook event database.
func (t *databaseTarget) Deliver(
_ context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
_ *Task,
_ Scheduler,
) {
err := t.archive(d)
if err != nil {
t.eng.log.Error(
"failed to archive event to database target",
"delivery_id", d.ID,
"event_id", d.EventID,
"error", err,
)
t.eng.recordResult(
webhookDB, d, 1, false, 0, "",
err.Error(), 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusFailed,
)
return
}
t.eng.recordResult(
webhookDB, d, 1, true, 0, "", "", 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusDelivered,
)
}
// archive writes the full event as a row into the webhook's
// archive database, honouring the optional per-target expiry
// parsed from the target config JSON.
func (t *databaseTarget) archive(d *database.Delivery) error {
webhookID := d.Event.WebhookID
if webhookID == "" {
return errArchiveMissingWebhookID
}
expiry, err := parseArchiveExpiry(d.Target.Config)
if err != nil {
return err
}
w, err := t.writerFor(webhookID)
if err != nil {
return err
}
row := archivedEvent{
EventID: d.Event.ID,
WebhookID: webhookID,
EntrypointID: d.Event.EntrypointID,
Method: d.Event.Method,
Headers: d.Event.Headers,
Body: d.Event.Body,
ContentType: d.Event.ContentType,
}
return w.write(row, expiry)
}
// writerFor returns the archiveWriter for a webhook, creating
// and caching it on first use. Each webhook has one writer so
// its close/reopen debounce state is shared across concurrent
// deliveries. The archive file lives beside the per-webhook
// event database in the data directory.
func (t *databaseTarget) writerFor(
webhookID string,
) (*archiveWriter, error) {
path, err := t.archivePath(webhookID)
if err != nil {
return nil, err
}
t.mu.Lock()
defer t.mu.Unlock()
if t.writers == nil {
t.writers = make(map[string]*archiveWriter)
}
w, ok := t.writers[webhookID]
if !ok {
w = newArchiveWriter(path, t.eng.log)
t.writers[webhookID] = w
}
// A delivery claims the entry: even if the idle sweep created
// it moments ago, it now belongs to the registry proper and
// the sweep must leave it in place when it finishes.
w.sweepOwned = false
return w, nil
}
// sweepWriterFor returns the archive writer the idle sweep should
// prune a webhook through, together with whether the sweep itself
// created the registry entry.
//
// The sweep must route its prune through the registered writer so
// the writer's mutex orders it against concurrent writes, but it
// must never leave a registry entry behind: a sweep that ran
// concurrently with the webhook's deletion would otherwise
// re-create an entry that nothing will ever evict again, which is
// exactly the leak eviction exists to prevent. An entry the sweep
// creates is therefore marked sweep-owned and handed back to
// releaseSweepWriter when the sweep is done.
func (t *databaseTarget) sweepWriterFor(
webhookID string,
) (*archiveWriter, bool, error) {
path, err := t.archivePath(webhookID)
if err != nil {
return nil, false, err
}
t.mu.Lock()
defer t.mu.Unlock()
if t.writers == nil {
t.writers = make(map[string]*archiveWriter)
}
w, ok := t.writers[webhookID]
if ok {
return w, false, nil
}
w = newArchiveWriter(path, t.eng.log)
w.sweepOwned = true
t.writers[webhookID] = w
return w, true, nil
}
// releaseSweepWriter drops a registry entry that the idle sweep
// created, so a sweep leaves the registry exactly as it found it.
//
// The entry is removed only if it is still the very writer the
// sweep installed and no delivery has claimed it in the meantime
// (writerFor clears sweepOwned when it hands a writer to the
// write path). Both conditions are evaluated under the registry
// lock, so an eviction that raced the sweep — which removes the
// entry outright — simply finds nothing left to do here, and a
// delivery that adopted the writer keeps a registered, evictable
// one.
func (t *databaseTarget) releaseSweepWriter(
webhookID string, w *archiveWriter,
) {
t.mu.Lock()
defer t.mu.Unlock()
cur, ok := t.writers[webhookID]
if !ok || cur != w || !cur.sweepOwned {
return
}
delete(t.writers, webhookID)
}
// archivePath returns the archive file path for a webhook: it
// lives beside the per-webhook event database in the data
// directory. It does not touch the filesystem.
func (t *databaseTarget) archivePath(
webhookID string,
) (string, error) {
if t.eng.dbManager == nil {
return "", errArchiveNoDataDir
}
dir := filepath.Dir(t.eng.dbManager.DBPath(webhookID))
return filepath.Join(
dir, fmt.Sprintf("archive-%s.db", webhookID),
), nil
}
// evict drops a webhook's archive writer from the registry and
// closes its handle, so a deleted webhook does not leave a
// writer (and an open archive handle within its debounce
// window) alive for the process lifetime.
//
// The map entry is removed under the registry lock, which is
// then released before the handle is closed under the writer's
// own lock: that ordering keeps the registry available to other
// webhooks while an in-flight write on this one drains, and
// closing under the writer's lock means eviction can never race
// a write.
//
// Eviction is idempotent and silent for a webhook with no
// writer, which is the common case: a webhook with no database
// target never creates one. It never deletes the archive file.
func (t *databaseTarget) evict(webhookID string) {
t.mu.Lock()
w, ok := t.writers[webhookID]
if ok {
delete(t.writers, webhookID)
}
t.mu.Unlock()
if !ok {
return
}
w.evict()
t.eng.log.Info(
"evicted archive writer",
"webhook_id", webhookID,
"path", w.path,
)
}
// sweepWebhook prunes one webhook's archive of rows older than
// expiry, without requiring a write. It returns nil (nothing to
// do) when the archive file does not exist, so a sweep never
// creates an archive for a webhook that has a database target
// but has never received an event.
//
// It also never leaves a registry entry behind: an entry it had
// to create to reach the writer's mutex is released again once
// the prune is done, so a sweep racing a webhook deletion cannot
// resurrect the writer the eviction just dropped.
func (t *databaseTarget) sweepWebhook(
webhookID string, expiry time.Duration,
) error {
path, err := t.archivePath(webhookID)
if err != nil {
return err
}
// Check before taking a writer at all: a webhook whose
// archive has never been created gets no writer, no handle,
// and no file.
if !fileExists(path) {
return nil
}
w, created, err := t.sweepWriterFor(webhookID)
if err != nil {
return err
}
if created {
defer t.releaseSweepWriter(webhookID, w)
}
return w.sweepExpired(expiry)
}

View File

@@ -0,0 +1,431 @@
package delivery
import (
"database/sql"
"encoding/json"
"errors"
"fmt"
"log/slog"
"os"
"sync"
"time"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
// archiveExpiryNever is the expiry sentinel (and default) that
// disables pruning so archived rows are kept forever.
const archiveExpiryNever = "never"
// archiveReopenDebounce bounds how often an archive file is
// closed and reopened. After each write the handle is closed
// and reopened so an operator can move the file away for
// offline archiving, but never more than once per this window.
const archiveReopenDebounce = time.Second
const (
// archiveModeCreate is the SQLite URI mode used by the write
// path: open the archive file, creating it if missing, so a
// first write (or a write after the operator moved the file
// away) recreates it.
archiveModeCreate = "rwc"
// archiveModeExisting is the SQLite URI mode used by the idle
// sweep: open read-write but never create. A sweep must never
// conjure an empty archive file for a webhook that has a
// database target but has never received an event.
archiveModeExisting = "rw"
)
var (
// errArchiveMissingWebhookID is returned when an event to
// archive has no webhook id to key its archive file on.
errArchiveMissingWebhookID = errors.New(
"cannot archive event without a webhook id",
)
// errArchiveNoDataDir is returned when the database target
// has no webhook database manager and so cannot locate the
// data directory for archive files.
errArchiveNoDataDir = errors.New(
"database target has no data directory",
)
// errArchiveExpiryNotPositive is returned when a
// user-supplied archive expiry parses as a duration but is
// zero or negative; "never" is the way to disable pruning.
errArchiveExpiryNotPositive = errors.New(
"expiry must be a positive duration or \"never\"",
)
// errArchiveWriterEvicted is returned when a writer that has
// been evicted (its webhook was deleted, or its last database
// target was removed) is used again. An evicted writer is no
// longer in the registry, so reopening its file would leak a
// handle nothing owns.
errArchiveWriterEvicted = errors.New(
"archive writer has been evicted",
)
)
// databaseTargetConfig is the optional per-target JSON config
// for a database (archive) target.
type databaseTargetConfig struct {
// Expiry is a Go duration (e.g. "720h") after which
// archived rows are pruned, or "never" (the default) to
// keep them forever.
Expiry string `json:"expiry"`
}
// archivedEvent is one fully captured webhook event stored in a
// per-webhook archive database for long-term retention. It is a
// self-contained copy — independent of the per-webhook event
// database, which may prune events under its own retention.
type archivedEvent struct {
ID uint `gorm:"primaryKey;autoIncrement"`
EventID string `gorm:"index"`
WebhookID string
EntrypointID string
Method string
Headers string
Body string
ContentType string
// ArchivedAt is when the row was archived and is the age
// basis for expiry pruning.
ArchivedAt time.Time `gorm:"index"`
}
// parseArchiveExpiry reads the optional expiry from a database
// target's config JSON. An empty config, an empty expiry, or
// the literal "never" all mean keep forever, returned as a zero
// duration. Any other value must parse as a positive Go
// duration; a set-but-invalid value (unparseable, zero, or
// negative) is an error rather than a silent default, matching
// ValidateArchiveExpiry at target creation.
func parseArchiveExpiry(
configJSON string,
) (time.Duration, error) {
if configJSON == "" {
return 0, nil
}
var cfg databaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return 0, fmt.Errorf(
"parsing database target config: %w", err,
)
}
if cfg.Expiry == "" || cfg.Expiry == archiveExpiryNever {
return 0, nil
}
dur, err := time.ParseDuration(cfg.Expiry)
if err != nil {
return 0, fmt.Errorf(
"parsing archive expiry %q: %w", cfg.Expiry, err,
)
}
if dur <= 0 {
return 0, fmt.Errorf(
"%w: %q", errArchiveExpiryNotPositive, cfg.Expiry,
)
}
return dur, nil
}
// ValidateArchiveExpiry checks a user-supplied archive expiry
// for a database target at configuration time. Valid values are
// empty, "never" (both meaning keep forever), or a positive Go
// duration such as "720h". Anything else is an error, so a bad
// expiry is rejected when the target is created rather than
// failing every subsequent delivery.
func ValidateArchiveExpiry(expiry string) error {
if expiry == "" || expiry == archiveExpiryNever {
return nil
}
dur, err := time.ParseDuration(expiry)
if err != nil {
return fmt.Errorf(
"expiry must be %q or a Go duration "+
"such as \"720h\": %w",
archiveExpiryNever, err,
)
}
if dur <= 0 {
return fmt.Errorf(
"%w: %q", errArchiveExpiryNotPositive, expiry,
)
}
return nil
}
// archiveWriter owns one per-webhook archive SQLite file. It
// serialises writes, and after each write closes and reopens
// the file (debounced to at most once per debounce window) so
// an operator can move the file away for offline archiving. The
// next write recreates a moved or removed file, because the
// file is opened create-if-missing and its schema is migrated
// on every open.
type archiveWriter struct {
mu sync.Mutex
path string
log *slog.Logger
debounce time.Duration
db *gorm.DB
lastReopen time.Time
reopens int
// evicted marks a writer that has been removed from the
// per-webhook registry. Its handle is closed and it must
// never open the file again: nothing holds it any more, so a
// reopen would leak the handle for the process lifetime.
evicted bool
// sweepOwned marks a registry entry that the idle sweep
// created because no writer was cached for the webhook. The
// sweep removes such an entry again when it is done, so a
// sweep can never leave — or resurrect — a registry entry
// for a webhook that has been deleted. A delivery that adopts
// the writer clears the flag, handing the entry to the
// registry proper.
//
// Unlike every other field here it is guarded by
// databaseTarget.mu, not by this writer's mu: it describes the
// registry entry rather than the file.
sweepOwned bool
}
// newArchiveWriter builds an archiveWriter for a file path with
// the default reopen debounce.
func newArchiveWriter(
path string, log *slog.Logger,
) *archiveWriter {
return &archiveWriter{
path: path,
log: log,
debounce: archiveReopenDebounce,
}
}
// write appends the event as a row, then applies the debounced
// close/reopen. It recreates the archive file if it was moved
// or removed since the last open. A positive expiry prunes rows
// older than it on each (re)open.
func (w *archiveWriter) write(
row archivedEvent, expiry time.Duration,
) error {
w.mu.Lock()
defer w.mu.Unlock()
if w.evicted {
return fmt.Errorf(
"%w: %s", errArchiveWriterEvicted, w.path,
)
}
if w.db == nil || !fileExists(w.path) {
err := w.reopen(expiry)
if err != nil {
return err
}
}
row.ArchivedAt = time.Now()
err := w.db.Create(&row).Error
if err != nil {
return fmt.Errorf(
"archiving event to %s: %w", w.path, err,
)
}
if time.Since(w.lastReopen) >= w.debounce {
return w.reopen(expiry)
}
return nil
}
// open opens (creating if missing) the archive file, migrates
// its schema, records the reopen time, and prunes expired rows
// when expiry is positive.
func (w *archiveWriter) open(expiry time.Duration) error {
return w.openMode(archiveModeCreate, expiry)
}
// openMode opens the archive file with the given SQLite URI
// mode, migrates its schema, records the reopen time, and
// prunes expired rows when expiry is positive. The write path
// passes archiveModeCreate so a missing file is recreated; the
// idle sweep passes archiveModeExisting so a missing file is an
// error rather than a newly conjured empty archive.
func (w *archiveWriter) openMode(
mode string, expiry time.Duration,
) error {
dbURL := fmt.Sprintf("file:%s?mode=%s", w.path, mode)
sqlDB, err := sql.Open("sqlite", dbURL)
if err != nil {
return fmt.Errorf(
"opening archive database %s: %w", w.path, err,
)
}
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB}, &gorm.Config{},
)
if err != nil {
_ = sqlDB.Close()
return fmt.Errorf(
"connecting to archive database %s: %w",
w.path, err,
)
}
err = gdb.AutoMigrate(&archivedEvent{})
if err != nil {
_ = sqlDB.Close()
return fmt.Errorf(
"migrating archive database %s: %w", w.path, err,
)
}
w.db = gdb
w.lastReopen = time.Now()
w.reopens++
if expiry > 0 {
w.prune(expiry)
}
return nil
}
// reopen closes any open handle and opens the file afresh. The
// fresh open recreates the file if it was moved away.
func (w *archiveWriter) reopen(expiry time.Duration) error {
w.close()
return w.open(expiry)
}
// close closes the underlying handle, if any.
func (w *archiveWriter) close() {
if w.db == nil {
return
}
sqlDB, err := w.db.DB()
if err == nil {
_ = sqlDB.Close()
}
w.db = nil
}
// sweepExpired prunes an archive that may have gone idle, with
// no write to trigger the usual on-reopen prune. It takes the
// writer's own mutex for the whole operation, so a sweep is
// ordered against concurrent writes rather than reaching around
// them to the file.
//
// It never creates the archive file: a missing file is skipped,
// and the reopen uses archiveModeExisting so SQLite itself
// refuses to create one if the file disappears between the
// check and the open.
//
// The archive is left CLOSED afterwards. An idle archive holding
// no handle is what keeps the operator's move-the-file-away
// workflow working; the next write reopens (and recreates) the
// file as it always has.
func (w *archiveWriter) sweepExpired(expiry time.Duration) error {
w.mu.Lock()
defer w.mu.Unlock()
if w.evicted {
return fmt.Errorf(
"%w: %s", errArchiveWriterEvicted, w.path,
)
}
if !fileExists(w.path) {
return nil
}
// Drop any live handle first so the prune runs against a
// freshly opened file, matching the write path's semantics.
w.close()
err := w.openMode(archiveModeExisting, expiry)
if err != nil {
return err
}
w.close()
return nil
}
// evict closes the writer's handle and marks it unusable. It is
// called when the writer leaves the registry, either because the
// webhook was deleted or because its last database target was
// removed. The archive FILE is deliberately left on disk: it is
// long-term storage an operator may still want.
func (w *archiveWriter) evict() {
w.mu.Lock()
defer w.mu.Unlock()
w.evicted = true
w.close()
}
// prune deletes archived rows older than expiry, measured from
// each row's archived time. It runs on every (re)open, so a
// steadily written archive is swept by its own write traffic. An
// archive that goes idle receives no further reopens, which is
// why ArchiveSweeper exists to drive sweepExpired on a timer.
// Failures are logged, not fatal: a prune error must not stop
// archiving.
func (w *archiveWriter) prune(expiry time.Duration) {
cutoff := time.Now().Add(-expiry)
res := w.db.Where("archived_at < ?", cutoff).
Delete(&archivedEvent{})
if res.Error != nil {
w.log.Error(
"failed to prune expired archive rows",
"path", w.path,
"error", res.Error,
)
return
}
if res.RowsAffected > 0 {
w.log.Info(
"pruned expired archive rows",
"path", w.path,
"rows_deleted", res.RowsAffected,
)
}
}
// fileExists reports whether a path currently exists.
func fileExists(path string) bool {
_, err := os.Stat(path)
return err == nil
}

View File

@@ -0,0 +1,363 @@
package delivery_test
import (
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// evictTestEngine builds an engine backed by a temporary data
// directory and returns it along with that directory.
func evictTestEngine(t *testing.T) (*delivery.Engine, string) {
t.Helper()
dataDir := t.TempDir()
eng := delivery.NewTestEngineWithDB(
nil,
database.NewTestWebhookDBManager(dataDir),
archiveTestLogger(),
&http.Client{Timeout: 5 * time.Second},
1,
)
return eng, dataDir
}
// TestEvictWebhook_ClosesAndRemovesWriter proves that evicting
// a webhook drops its archive writer from the registry and
// closes the open archive handle, rather than leaving both
// alive for the process lifetime.
func TestEvictWebhook_ClosesAndRemovesWriter(t *testing.T) {
t.Parallel()
eng, dataDir := evictTestEngine(t)
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":true}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, "")
eng.ExportDeliverDatabase(webhookDB, d)
webhookID := event.WebhookID
require.True(
t, eng.ExportHasArchiveWriter(webhookID),
"a delivery should have cached an archive writer",
)
require.True(
t, eng.ExportArchiveHandleOpen(webhookID),
"the writer should hold an open handle after a write",
)
eng.EvictWebhook(webhookID)
assert.False(
t, eng.ExportHasArchiveWriter(webhookID),
"eviction should remove the registry entry",
)
assert.False(
t, eng.ExportArchiveHandleOpen(webhookID),
"eviction should close the archive handle",
)
archivePath := filepath.Join(
dataDir, fmt.Sprintf("archive-%s.db", webhookID),
)
assert.FileExists(
t, archivePath,
"eviction must not delete the archive file",
)
}
// TestEvictWebhook_UnknownWebhookIsNoOp proves eviction is safe
// for the common case of a webhook that never had a database
// target, and that repeating it does not panic.
func TestEvictWebhook_UnknownWebhookIsNoOp(t *testing.T) {
t.Parallel()
eng, _ := evictTestEngine(t)
assert.NotPanics(t, func() {
eng.EvictWebhook("no-such-webhook")
eng.EvictWebhook("no-such-webhook")
})
assert.False(
t, eng.ExportHasArchiveWriter("no-such-webhook"),
"eviction must not create a writer",
)
}
// evictTestRow builds an archive row for the eviction tests.
func evictTestRow(eventID string) delivery.ExportArchivedEvent {
return delivery.ExportArchivedEvent{
EventID: eventID,
WebhookID: "wh-evict",
Method: http.MethodPost,
Body: `{"seeded":true}`,
}
}
// TestEvictedWriter_WriteDoesNotReopenFile is the direct test of
// the evicted guard on the write path. A writer that has left
// the registry is held by nobody, so a handle it opened could
// never be closed again: it must refuse the write outright
// rather than recreate the archive behind the registry's back.
//
// The archive file is removed before the eviction, so an
// unguarded write is unmistakable — it recreates the file.
func TestEvictedWriter_WriteDoesNotReopenFile(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-evicted.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Write(evictTestRow("ev-1"), 0))
require.FileExists(t, path)
// The operator moves the archive away for offline retention,
// which the write path would ordinarily undo on the next
// write by recreating the file.
require.NoError(t, os.Remove(path))
w.Evict()
err := w.Write(evictTestRow("ev-2"), 0)
require.ErrorIs(
t, err, delivery.ErrExportArchiveWriterEvicted,
"an evicted writer must refuse writes",
)
assert.NoFileExists(
t, path,
"an evicted writer must not reopen (or recreate) the "+
"archive file",
)
assert.False(
t, w.HandleOpen(),
"an evicted writer must hold no handle",
)
}
// TestEvictedWriter_SweepDoesNotReopenFile is the same test for
// the sweep path: an idle sweep that reaches a writer already
// evicted underneath it must return the sentinel rather than
// reopen a file nothing owns.
func TestEvictedWriter_SweepDoesNotReopenFile(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-evicted.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Write(evictTestRow("ev-1"), 0))
require.FileExists(t, path)
w.Evict()
err := w.SweepExpired(time.Hour)
require.ErrorIs(
t, err, delivery.ErrExportArchiveWriterEvicted,
"an evicted writer must refuse an idle sweep",
)
assert.False(
t, w.HandleOpen(),
"a refused sweep must not leave a handle open",
)
}
// racingWrites drives a pack of goroutines writing to one
// archive writer until each is refused, so an eviction on the
// test goroutine has to take the writer's mutex away from writes
// that are already contending for it.
type racingWrites struct {
wg sync.WaitGroup
mu sync.Mutex
sawEvicted bool
otherErr error
started chan struct{}
}
// racingWriteGoroutines is how many goroutines contend for the
// writer's mutex while the eviction lands.
const racingWriteGoroutines = 4
// startRacingWrites launches the writing goroutines. Each writes
// in a loop and stops at its first error, recording whether that
// error was the eviction sentinel. The deadline is a backstop
// against a hang, not a timing assumption: the first write after
// the eviction is refused.
func startRacingWrites(
w *delivery.ExportArchiveWriter,
) *racingWrites {
r := &racingWrites{
started: make(chan struct{}, racingWriteGoroutines),
}
deadline := time.Now().Add(10 * time.Second)
r.wg.Add(racingWriteGoroutines)
for i := range racingWriteGoroutines {
go func() {
defer r.wg.Done()
first := true
for time.Now().Before(deadline) {
err := w.Write(
evictTestRow(fmt.Sprintf("ev-%d", i)), 0,
)
if first {
r.started <- struct{}{}
first = false
}
if err == nil {
continue
}
r.record(err)
return
}
}()
}
return r
}
// record classifies the error that stopped one goroutine.
func (r *racingWrites) record(err error) {
r.mu.Lock()
defer r.mu.Unlock()
if errors.Is(err, delivery.ErrExportArchiveWriterEvicted) {
r.sawEvicted = true
return
}
r.otherErr = err
}
// awaitFirstWrite blocks until at least one write has run, so
// the eviction that follows is a genuine race.
func (r *racingWrites) awaitFirstWrite() {
<-r.started
}
// wait joins the goroutines and reports whether any write was
// refused with the eviction sentinel, plus any unexpected error.
func (r *racingWrites) wait() (bool, error) {
r.wg.Wait()
r.mu.Lock()
defer r.mu.Unlock()
return r.sawEvicted, r.otherErr
}
// TestEvictWebhook_RacingWriteDoesNotReopenHandle exercises the
// interleaving the evicted flag exists for: writes already
// contending for the writer's mutex when the eviction takes it.
// The write that wins the mutex after the eviction must abandon
// its work rather than reopen the archive, leaving the writer
// permanently handle-free. Run under -race.
func TestEvictWebhook_RacingWriteDoesNotReopenHandle(
t *testing.T,
) {
t.Parallel()
eng, _ := evictTestEngine(t)
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":true}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, "")
// Prime the registry so the test can hold the very writer the
// eviction is about to detach.
eng.ExportDeliverDatabase(webhookDB, d)
w := eng.ExportArchiveWriterFor(event.WebhookID)
require.NotNil(t, w)
require.True(t, w.HandleOpen())
race := startRacingWrites(w)
// Evict only once writes are genuinely in flight, so the
// eviction has to contend for the writer's mutex.
race.awaitFirstWrite()
eng.EvictWebhook(event.WebhookID)
sawEvicted, otherErr := race.wait()
require.NoError(t, otherErr)
assert.True(
t, sawEvicted,
"a write after eviction must be refused",
)
assert.False(
t, w.HandleOpen(),
"no write may reopen the archive once the writer has "+
"been evicted",
)
assert.False(
t, eng.ExportHasArchiveWriter(event.WebhookID),
"the registry entry must stay gone",
)
}
// TestEvictWebhook_LaterDeliveryRecreatesWriter proves eviction
// does not break archiving for a webhook that is still alive: a
// subsequent delivery gets a brand new writer from the registry.
// It says nothing about the evicted writer itself — that is what
// TestEvictedWriter_WriteDoesNotReopenFile covers.
func TestEvictWebhook_LaterDeliveryRecreatesWriter(t *testing.T) {
t.Parallel()
eng, _ := evictTestEngine(t)
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":true}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, "")
eng.ExportDeliverDatabase(webhookDB, d)
require.True(
t, eng.ExportHasArchiveWriter(event.WebhookID),
)
eng.EvictWebhook(event.WebhookID)
// A fresh delivery for the same webhook gets a brand new
// writer from the registry, so archiving keeps working.
second := seedDatabaseTargetDelivery(
t, webhookDB, event, "",
)
eng.ExportDeliverDatabase(webhookDB, second)
assert.True(
t, eng.ExportHasArchiveWriter(event.WebhookID),
"a later delivery should recreate the writer",
)
}

View File

@@ -0,0 +1,402 @@
package delivery_test
import (
"database/sql"
"fmt"
"log/slog"
"net/http"
"os"
"path/filepath"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
_ "modernc.org/sqlite" // Pure Go SQLite driver.
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
func archiveTestLogger() *slog.Logger {
return slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
}
// openArchiveDBForRead opens an archive file read-only so a
// test can inspect the rows the writer persisted.
func openArchiveDBForRead(
t *testing.T, path string,
) *gorm.DB {
t.Helper()
sqlDB, err := sql.Open(
"sqlite",
fmt.Sprintf("file:%s?mode=ro", path),
)
require.NoError(t, err)
t.Cleanup(func() { _ = sqlDB.Close() })
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB}, &gorm.Config{},
)
require.NoError(t, err)
return gdb
}
// archiveFileSuffixes returns the archive file itself and the
// SQLite sidecars that accompany an open database. A test that
// asserts no archive was created has to check all of them.
func archiveFileSuffixes() []string {
return []string{"", "-wal", "-shm"}
}
// removeArchiveFiles simulates an operator moving the archive
// away by deleting the SQLite file and its sidecar files.
func removeArchiveFiles(t *testing.T, path string) {
t.Helper()
for _, suffix := range []string{
"", "-wal", "-shm", "-journal",
} {
err := os.Remove(path + suffix)
if err != nil && !os.IsNotExist(err) {
t.Fatalf("removing %s%s: %v", path, suffix, err)
}
}
}
// TestDeliverDatabase_ArchivesEvent verifies that delivering to
// a database target marks the delivery delivered and archives
// the full event into a separate per-webhook archive file.
func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
t.Parallel()
dataDir := t.TempDir()
dbMgr := database.NewTestWebhookDBManager(dataDir)
e := delivery.NewTestEngineWithDB(
nil, dbMgr,
archiveTestLogger(),
&http.Client{Timeout: 5 * time.Second},
1,
)
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":true}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, "")
e.ExportDeliverDatabase(webhookDB, d)
var updated database.Delivery
require.NoError(t, webhookDB.First(
&updated, "id = ?", d.ID,
).Error)
assert.Equal(t,
database.DeliveryStatusDelivered, updated.Status,
"database target should mark the delivery delivered",
)
archivePath := filepath.Join(
dataDir,
fmt.Sprintf("archive-%s.db", event.WebhookID),
)
assert.FileExists(t, archivePath)
rdb := openArchiveDBForRead(t, archivePath)
var rows []delivery.ExportArchivedEvent
require.NoError(t, rdb.Find(&rows).Error)
require.Len(t, rows, 1)
assert.Equal(t, event.ID, rows[0].EventID)
assert.Equal(t, event.WebhookID, rows[0].WebhookID)
assert.Equal(t, event.Method, rows[0].Method)
assert.JSONEq(t, `{"archived":true}`, rows[0].Body)
}
func TestArchiveWriter_WritesRow(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
row := delivery.ExportArchivedEvent{
EventID: "ev-1",
WebhookID: "wh-1",
EntrypointID: "ep-1",
Method: "POST",
Headers: `{"X":"Y"}`,
Body: `{"hello":"world"}`,
ContentType: "application/json",
}
require.NoError(t, w.Write(row, 0))
assert.FileExists(t, path)
var got []delivery.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "ev-1", got[0].EventID)
assert.Equal(t, "wh-1", got[0].WebhookID)
assert.Equal(t, "ep-1", got[0].EntrypointID)
assert.Equal(t, row.Method, got[0].Method)
assert.Equal(t, row.ContentType, got[0].ContentType)
assert.JSONEq(t, `{"hello":"world"}`, got[0].Body)
assert.False(t, got[0].ArchivedAt.IsZero())
}
func TestArchiveWriter_RecreatesAfterRemoval(
t *testing.T,
) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "a"}, 0,
))
assert.FileExists(t, path)
// The operator moves the archive away while the handle is
// still open.
removeArchiveFiles(t, path)
require.NoFileExists(t, path)
// The next write recreates the file with a fresh schema and
// only the new row.
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "b"}, 0,
))
assert.FileExists(t, path)
var got []delivery.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "b", got[0].EventID)
}
func TestArchiveWriter_ReopenDebounce(t *testing.T) {
t.Parallel()
// A generous debounce keeps the two rapid writes inside
// the window even on a heavily loaded test machine.
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 2*time.Second,
)
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "a"}, 0,
))
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "b"}, 0,
))
// Two writes inside the debounce window trigger only the
// initial open — no extra close/reopen.
assert.Equal(t, 1, w.Reopens())
time.Sleep(2100 * time.Millisecond)
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "c"}, 0,
))
// A write after the window elapses closes and reopens once.
assert.Equal(t, 2, w.Reopens())
}
func TestArchiveWriter_ExpiryPrune(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Open(0))
old := delivery.ExportArchivedEvent{
EventID: "old",
ArchivedAt: time.Now().Add(-2 * time.Hour),
}
fresh := delivery.ExportArchivedEvent{
EventID: "fresh",
ArchivedAt: time.Now(),
}
require.NoError(t, w.DB().Create(&old).Error)
require.NoError(t, w.DB().Create(&fresh).Error)
// Reopening with a one-hour expiry prunes the old row.
require.NoError(t, w.Reopen(time.Hour))
var got []delivery.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "fresh", got[0].EventID)
}
func TestParseArchiveExpiry(t *testing.T) {
t.Parallel()
cases := []struct {
name string
in string
want time.Duration
wantErr bool
}{
{"empty config", "", 0, false},
{"explicit never", `{"expiry":"never"}`, 0, false},
{"empty expiry", `{"expiry":""}`, 0, false},
{"duration", `{"expiry":"1h"}`, time.Hour, false},
{"unparseable", `{"expiry":"nonsense"}`, 0, true},
{"zero duration", `{"expiry":"0s"}`, 0, true},
{"negative duration", `{"expiry":"-5h"}`, 0, true},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
got, err := delivery.ExportParseArchiveExpiry(tc.in)
if tc.wantErr {
require.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, tc.want, got)
})
}
}
// seedDatabaseTargetDelivery seeds a pending delivery for a
// database target with the given config JSON and returns the
// in-memory delivery the target handler is invoked with.
func seedDatabaseTargetDelivery(
t *testing.T,
webhookDB *gorm.DB,
event database.Event,
config string,
) *database.Delivery {
t.Helper()
dlv := seedDelivery(
t, webhookDB, event.ID, uuid.New().String(),
database.DeliveryStatusPending,
)
d := &database.Delivery{
EventID: event.ID,
TargetID: dlv.TargetID,
Status: database.DeliveryStatusPending,
Event: event,
Target: database.Target{
Name: "test-db",
Type: database.TargetTypeDatabase,
Config: config,
},
}
d.ID = dlv.ID
return d
}
// TestDeliverDatabase_ArchiveFailureFailsDelivery verifies that
// an archive error (here: an unparseable expiry in the target
// config) fails the delivery loudly: the attempt is recorded as
// failed with the error and the delivery is marked failed, not
// delivered.
func TestDeliverDatabase_ArchiveFailureFailsDelivery(
t *testing.T,
) {
t.Parallel()
dataDir := t.TempDir()
e := delivery.NewTestEngineWithDB(
nil, database.NewTestWebhookDBManager(dataDir),
archiveTestLogger(),
&http.Client{Timeout: 5 * time.Second},
1,
)
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":false}`)
d := seedDatabaseTargetDelivery(
t, webhookDB, event, `{"expiry":"nonsense"}`,
)
e.ExportDeliverDatabase(webhookDB, d)
var updated database.Delivery
require.NoError(t, webhookDB.First(
&updated, "id = ?", d.ID,
).Error)
assert.Equal(t,
database.DeliveryStatusFailed, updated.Status,
"archive failure must mark the delivery failed",
)
var results []database.DeliveryResult
require.NoError(t, webhookDB.Where(
"delivery_id = ?", d.ID,
).Find(&results).Error)
require.Len(t, results, 1)
assert.False(t,
results[0].Success,
"the attempt must be recorded as failed",
)
assert.Contains(t,
results[0].Error, "nonsense",
"the archive error must be recorded on the attempt",
)
assert.NoFileExists(t,
filepath.Join(
dataDir,
fmt.Sprintf("archive-%s.db", event.WebhookID),
),
"no archive file should exist for a failed config",
)
}
func TestValidateArchiveExpiry(t *testing.T) {
t.Parallel()
valid := []string{"", "never", "1h", "720h", "30m"}
for _, in := range valid {
require.NoError(t,
delivery.ValidateArchiveExpiry(in),
"expiry %q should be accepted", in,
)
}
invalid := []string{"nonsense", "7d", "-5h", "0s", "0"}
for _, in := range invalid {
require.Error(t,
delivery.ValidateArchiveExpiry(in),
"expiry %q should be rejected", in,
)
}
}

View File

@@ -0,0 +1,511 @@
package delivery
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"sync"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// Sentinel errors returned by the config parsers.
var (
errEmptyTargetConfig = errors.New(
"empty target config",
)
errMissingTargetURL = errors.New(
"target URL is required",
)
)
// HTTPTargetConfig holds configuration for http target
// types.
type HTTPTargetConfig struct {
URL string `json:"url"`
Headers map[string]string `json:"headers,omitempty"`
Timeout int `json:"timeout,omitempty"`
}
// httpCore holds the retry, backoff, and circuit-breaker
// machinery shared by the HTTP and Slack targets. Each of
// those targets owns its own httpCore instance (and thus its
// own circuit breakers); the per-attempt request differs
// between them and is supplied as a closure.
type httpCore struct {
eng *Engine
// circuitBreakers stores a *CircuitBreaker per target ID.
circuitBreakers sync.Map
}
// deliver runs one delivery attempt through the retry core.
// A maxRetries of 0 is fire-and-forget: a single attempt is
// recorded and no circuit breaker is consulted. A positive
// maxRetries gates the attempt on the circuit breaker and
// schedules a backed-off retry on failure.
func (c *httpCore) deliver(
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
maxRetries int,
attempt func() attemptResult,
) {
if maxRetries == 0 {
c.fireAndForget(webhookDB, d, attempt())
return
}
c.withRetry(
webhookDB, d, task, sched, maxRetries, attempt,
)
}
func (c *httpCore) fireAndForget(
webhookDB *gorm.DB,
d *database.Delivery,
res attemptResult,
) {
c.eng.recordResult(
webhookDB, d, 1, res.success,
res.statusCode, res.respBody, res.errMsg,
res.duration,
)
if res.success {
c.eng.updateDeliveryStatus(
webhookDB, d,
database.DeliveryStatusDelivered,
)
return
}
c.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusFailed,
)
}
func (c *httpCore) withRetry(
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
maxRetries int,
attempt func() attemptResult,
) {
cb := c.getCircuitBreaker(task.TargetID)
if c.circuitBreakerBlock(webhookDB, d, task, sched, cb) {
return
}
attemptNum := task.AttemptNum
res := attempt()
c.eng.recordResult(
webhookDB, d, attemptNum, res.success,
res.statusCode, res.respBody, res.errMsg,
res.duration,
)
if res.success {
cb.RecordSuccess()
c.eng.updateDeliveryStatus(
webhookDB, d,
database.DeliveryStatusDelivered,
)
return
}
cb.RecordFailure()
c.handleRetry(
webhookDB, d, task, sched, maxRetries, attemptNum,
)
}
func (c *httpCore) circuitBreakerBlock(
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
cb *CircuitBreaker,
) bool {
if cb.Allow() {
return false
}
remaining := cb.CooldownRemaining()
c.eng.log.Info(
"circuit breaker open, skipping delivery",
"target_id", task.TargetID,
"target_name", task.TargetName,
"delivery_id", d.ID,
"cooldown_remaining", remaining,
)
c.eng.updateDeliveryStatus(
webhookDB, d,
database.DeliveryStatusRetrying,
)
retryTask := *task
sched.ScheduleRetry(retryTask, remaining)
return true
}
func (c *httpCore) handleRetry(
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
maxRetries int,
attemptNum int,
) {
if attemptNum >= maxRetries {
c.eng.updateDeliveryStatus(
webhookDB, d,
database.DeliveryStatusFailed,
)
return
}
c.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusRetrying,
)
backoff := calcBackoff(attemptNum)
retryTask := *task
retryTask.AttemptNum = attemptNum + 1
sched.ScheduleRetry(retryTask, backoff)
}
func (c *httpCore) getCircuitBreaker(
targetID string,
) *CircuitBreaker {
if val, ok := c.circuitBreakers.Load(targetID); ok {
cb, _ := val.(*CircuitBreaker)
return cb
}
fresh := NewCircuitBreaker()
actual, _ := c.circuitBreakers.LoadOrStore(
targetID, fresh,
)
cb, _ := actual.(*CircuitBreaker)
return cb
}
// remainingBackoff returns how long remains of the backoff
// window for the last attempt of a recovered retrying
// delivery. It implements rescheduler.
func (c *httpCore) remainingBackoff(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) time.Duration {
var lastResult database.DeliveryResult
err := webhookDB.
Where("delivery_id = ?", deliveryID).
Order("created_at DESC").
First(&lastResult).Error
if err != nil {
return 0
}
backoff := calcBackoff(attemptNum)
elapsed := time.Since(lastResult.CreatedAt)
remaining := backoff - elapsed
return max(remaining, 0)
}
// backoffElapsed reports whether the backoff window for the
// last attempt of a retrying delivery has passed. It
// implements rescheduler.
func (c *httpCore) backoffElapsed(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) bool {
var lastResult database.DeliveryResult
err := webhookDB.
Where("delivery_id = ?", deliveryID).
Order("created_at DESC").
First(&lastResult).Error
if err != nil {
return true
}
backoff := calcBackoff(attemptNum)
return time.Since(lastResult.CreatedAt) >= backoff
}
func calcBackoff(attemptNum int) time.Duration {
shift := max(attemptNum-1, 0)
shift = min(shift, maxBackoffShift)
return time.Duration(1<<uint(shift)) * time.Second
}
// httpTarget delivers events to http targets. It forwards the
// event body and (filtered) request headers to the configured
// URL and owns retry, backoff, and circuit breaking through
// the shared httpCore.
type httpTarget struct {
*httpCore
client *http.Client
}
// Deliver implements Target.
func (t *httpTarget) Deliver(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
) {
cfg, err := parseHTTPConfig(d.Target.Config)
if err != nil {
t.eng.log.Error(
"invalid HTTP target config",
"target_id", d.TargetID,
"error", err,
)
t.eng.recordResult(
webhookDB, d, task.AttemptNum,
false, 0, "", err.Error(), 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusFailed,
)
return
}
attempt := func() attemptResult {
return t.attempt(ctx, cfg, &d.Event)
}
t.deliver(
webhookDB, d, task, sched,
d.Target.MaxRetries, attempt,
)
}
// attempt performs a single HTTP delivery attempt and derives
// the success flag and error message the same way the engine
// did: a non-2xx response is a failure but carries no error
// string; only a transport-level error does.
func (t *httpTarget) attempt(
ctx context.Context,
cfg *HTTPTargetConfig,
event *database.Event,
) attemptResult {
statusCode, respBody, duration, reqErr :=
t.doHTTPRequest(ctx, cfg, event)
success := reqErr == nil &&
statusCode >= httpSuccessMin &&
statusCode < httpSuccessMax
errMsg := ""
if reqErr != nil {
errMsg = reqErr.Error()
}
return attemptResult{
statusCode: statusCode,
respBody: respBody,
duration: duration,
success: success,
errMsg: errMsg,
}
}
func (t *httpTarget) doHTTPRequest(
ctx context.Context,
cfg *HTTPTargetConfig,
event *database.Event,
) (int, string, int64, error) {
start := time.Now()
req, reqErr := http.NewRequestWithContext(
ctx,
http.MethodPost,
cfg.URL,
bytes.NewReader([]byte(event.Body)),
)
if reqErr != nil {
return 0, "", 0, fmt.Errorf(
"creating request: %w",
maskURLError(reqErr),
)
}
applyRequestHeaders(req, event, cfg)
client := t.clientForConfig(cfg)
resp, doErr := executeHTTPRequest(client, req)
dur := time.Since(start).Milliseconds()
if doErr != nil {
return 0, "", dur, fmt.Errorf(
"sending request: %w", doErr,
)
}
defer func() { _ = resp.Body.Close() }()
body, readErr := io.ReadAll(
io.LimitReader(resp.Body, maxBodyLog),
)
if readErr != nil {
return resp.StatusCode, "", dur,
fmt.Errorf(
"reading response body: %w", readErr,
)
}
return resp.StatusCode, string(body), dur, nil
}
func (t *httpTarget) clientForConfig(
cfg *HTTPTargetConfig,
) *http.Client {
if cfg.Timeout > 0 {
// Reuse the shared client's SSRF-safe transport so
// a per-target timeout does not drop the
// request-time private-IP guard. Only the timeout
// is overridden.
return &http.Client{
Timeout: time.Duration(
cfg.Timeout,
) * time.Second,
Transport: t.client.Transport,
}
}
return t.client
}
func parseHTTPConfig(
configJSON string,
) (*HTTPTargetConfig, error) {
if configJSON == "" {
return nil, errEmptyTargetConfig
}
var cfg HTTPTargetConfig
err := json.Unmarshal(
[]byte(configJSON), &cfg,
)
if err != nil {
return nil, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
if cfg.URL == "" {
return nil, errMissingTargetURL
}
return &cfg, nil
}
// isForwardableHeader returns true if the header should
// be forwarded to targets.
func isForwardableHeader(name string) bool {
switch http.CanonicalHeaderKey(name) {
case "Host", "Connection", "Keep-Alive",
"Transfer-Encoding", "Te", "Trailer",
"Upgrade", "Proxy-Authorization",
"Proxy-Connection", "Content-Length":
return false
default:
return true
}
}
func applyRequestHeaders(
req *http.Request,
event *database.Event,
cfg *HTTPTargetConfig,
) {
if event.ContentType != "" {
req.Header.Set(
"Content-Type", event.ContentType,
)
}
var originalHeaders map[string][]string
if event.Headers != "" {
jsonErr := json.Unmarshal(
[]byte(event.Headers),
&originalHeaders,
)
if jsonErr == nil {
for k, vals := range originalHeaders {
if isForwardableHeader(k) {
for _, v := range vals {
req.Header.Add(k, v)
}
}
}
}
}
for k, v := range cfg.Headers {
req.Header.Set(k, v)
}
req.Header.Set("User-Agent", "webhooker/1.0")
}
// executeHTTPRequest sends an HTTP request using the provided
// client. URLs are validated by the config parsers and the
// SSRF-safe transport before reaching here.
//
// Transport failures are masked here, at the single point
// where every target's request errors are born, because the
// caller stores them in DeliveryResult.Error: an unmasked
// *url.Error would write the target URL — the credential for
// a Slack incoming webhook — into the per-webhook database.
func executeHTTPRequest(
client *http.Client, req *http.Request,
) (*http.Response, error) {
resp, err := client.Do(req) //#nosec G704 -- validated URL, SSRF-safe transport
if err != nil {
return nil, maskURLError(err)
}
return resp, nil
}

View File

@@ -0,0 +1,47 @@
package delivery
import (
"context"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// logTarget is a fire-and-forget target that logs the entire
// inbound webhook — the full request body and headers, plus
// the method, content type, and the webhook and entrypoint
// ids — then records a single successful attempt.
type logTarget struct {
eng *Engine
}
// Deliver implements Target.
func (t *logTarget) Deliver(
_ context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
_ *Task,
_ Scheduler,
) {
t.eng.log.Info(
"webhook event delivered to log target",
"delivery_id", d.ID,
"event_id", d.EventID,
"target_id", d.TargetID,
"target_name", d.Target.Name,
"webhook_id", d.Event.WebhookID,
"entrypoint_id", d.Event.EntrypointID,
"method", d.Event.Method,
"content_type", d.Event.ContentType,
"headers", d.Event.Headers,
"body", d.Event.Body,
)
t.eng.recordResult(
webhookDB, d, 1, true, 0, "", "", 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusDelivered,
)
}

View File

@@ -0,0 +1,299 @@
package delivery
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// errMissingWebhookURL is returned when a Slack target config
// omits its webhook URL.
var errMissingWebhookURL = errors.New(
"webhook_url is required",
)
// SlackTargetConfig holds configuration for slack target
// types.
type SlackTargetConfig struct {
WebhookURL string `json:"webhookUrl"`
}
// slackTarget delivers events to Slack incoming webhooks. It
// formats the event into a Slack message and posts it as
// JSON. It shares the retry core with the HTTP target: a
// MaxRetries of 0 stays single-attempt fire-and-forget
// (preserving existing Slack targets), while a positive
// MaxRetries adds backoff and circuit breaking.
type slackTarget struct {
*httpCore
client *http.Client
}
// Deliver implements Target.
func (t *slackTarget) Deliver(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
) {
cfg, err := parseSlackConfig(d.Target.Config)
if err != nil {
t.eng.log.Error(
"invalid Slack target config",
"target_id", d.TargetID,
"error", err,
)
t.failConfig(webhookDB, d, err)
return
}
msg := FormatSlackMessage(&d.Event)
payload, err := json.Marshal(
map[string]string{"text": msg},
)
if err != nil {
t.eng.log.Error(
"failed to marshal Slack payload",
"target_id", d.TargetID,
"error", err,
)
t.failConfig(webhookDB, d, err)
return
}
attempt := func() attemptResult {
return t.attempt(ctx, cfg, payload)
}
t.deliver(
webhookDB, d, task, sched,
d.Target.MaxRetries, attempt,
)
}
// failConfig records a first-attempt failure for a delivery
// that could not be prepared (bad config or unmarshalable
// payload) and marks it failed.
func (t *slackTarget) failConfig(
webhookDB *gorm.DB,
d *database.Delivery,
err error,
) {
t.eng.recordResult(
webhookDB, d, 1,
false, 0, "", err.Error(), 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusFailed,
)
}
// attempt performs a single Slack POST and derives its
// outcome, preserving the engine's original semantics: a
// non-2xx response records an "HTTP <code>" error string and
// a transport error records a "sending request" error.
func (t *slackTarget) attempt(
ctx context.Context,
cfg *SlackTargetConfig,
payload []byte,
) attemptResult {
start := time.Now()
req, err := http.NewRequestWithContext(
ctx,
http.MethodPost,
cfg.WebhookURL,
bytes.NewReader(payload),
)
if err != nil {
return attemptResult{
success: false,
errMsg: maskURLError(err).Error(),
}
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("User-Agent", "webhooker/1.0")
resp, doErr := executeHTTPRequest(t.client, req)
durationMs := time.Since(start).Milliseconds()
if doErr != nil {
return attemptResult{
success: false,
duration: durationMs,
errMsg: fmt.Errorf(
"sending request: %w", doErr,
).Error(),
}
}
defer func() { _ = resp.Body.Close() }()
return t.readSlackResponse(resp, durationMs)
}
func (t *slackTarget) readSlackResponse(
resp *http.Response,
durationMs int64,
) attemptResult {
body, readErr := io.ReadAll(
io.LimitReader(resp.Body, maxBodyLog),
)
if readErr != nil {
t.eng.log.Error(
"failed to read Slack response body",
"error", readErr,
)
}
success := resp.StatusCode >= httpSuccessMin &&
resp.StatusCode < httpSuccessMax
errMsg := ""
if !success {
errMsg = fmt.Sprintf("HTTP %d", resp.StatusCode)
}
return attemptResult{
statusCode: resp.StatusCode,
respBody: string(body),
duration: durationMs,
success: success,
errMsg: errMsg,
}
}
func parseSlackConfig(
configJSON string,
) (*SlackTargetConfig, error) {
if configJSON == "" {
return nil, errEmptyTargetConfig
}
var cfg SlackTargetConfig
err := json.Unmarshal(
[]byte(configJSON), &cfg,
)
if err != nil {
return nil, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
if cfg.WebhookURL == "" {
return nil, errMissingWebhookURL
}
return &cfg, nil
}
// FormatSlackMessage builds a Slack-compatible message
// string from a webhook event.
func FormatSlackMessage(
event *database.Event,
) string {
var b strings.Builder
b.WriteString("*Webhook Event Received*\n")
fmt.Fprintf(
&b, "*Method:* `%s`\n", event.Method,
)
fmt.Fprintf(
&b,
"*Content-Type:* `%s`\n",
event.ContentType,
)
fmt.Fprintf(
&b,
"*Timestamp:* `%s`\n",
event.CreatedAt.UTC().Format(time.RFC3339),
)
fmt.Fprintf(
&b,
"*Body Size:* %d bytes\n",
len(event.Body),
)
if event.Body == "" {
b.WriteString("\n_(empty body)_\n")
return b.String()
}
if formatted := formatJSONBody(event.Body); formatted != "" {
b.WriteString(formatted)
return b.String()
}
formatRawBody(&b, event.Body)
return b.String()
}
func formatJSONBody(body string) string {
var parsed json.RawMessage
if json.Unmarshal([]byte(body), &parsed) != nil {
return ""
}
var pretty bytes.Buffer
if json.Indent(&pretty, parsed, "", " ") != nil {
return ""
}
var b strings.Builder
b.WriteString("\n```\n")
prettyStr := pretty.String()
const maxPayloadDisplay = 3500
if len(prettyStr) > maxPayloadDisplay {
b.WriteString(prettyStr[:maxPayloadDisplay])
b.WriteString("\n... (truncated)")
} else {
b.WriteString(prettyStr)
}
b.WriteString("\n```\n")
return b.String()
}
func formatRawBody(b *strings.Builder, body string) {
b.WriteString("\n```\n")
const maxRawDisplay = 3500
if len(body) > maxRawDisplay {
b.WriteString(body[:maxRawDisplay])
b.WriteString("\n... (truncated)")
} else {
b.WriteString(body)
}
b.WriteString("\n```\n")
}

View File

@@ -0,0 +1,61 @@
package delivery
import (
"errors"
"net/url"
)
// urlPathElision stands in for a URL's elided path.
const urlPathElision = "/..."
// MaskURL renders a URL as scheme plus host with everything
// that can carry a secret removed. A delivery target URL is
// itself a credential — a Slack incoming webhook URL is a
// bearer token — so the path, query and userinfo are never
// reproduced, in a page, a log line or a stored error. A URL
// that does not parse into a scheme and host yields the
// neutral placeholder, never the raw string.
func MaskURL(raw string) string {
parsed, err := url.Parse(raw)
if err != nil || parsed.Scheme == "" ||
parsed.Host == "" {
return configUnavailable
}
masked := parsed.Scheme + "://" + parsed.Host
if parsed.Path != "" && parsed.Path != "/" {
masked += urlPathElision
}
return masked
}
// maskURLError strips the credential from an error raised
// against a request URL. The net/http and net/url packages
// embed the full request URL in every *url.Error they return,
// so an unmodified transport error persisted into
// DeliveryResult.Error writes the credential to disk.
//
// The masked error keeps the operation and the wrapped cause,
// so a DNS failure still reads differently from a refused
// connection, a TLS handshake failure or a timeout, and Is,
// As, Timeout and Temporary keep working on it. Only the
// path, query and userinfo of the URL are dropped. Errors
// that carry no URL are returned unchanged.
//
// Call it where the error is raised, before any wrapping: it
// replaces the *url.Error itself, so any context wrapped
// around it first would be discarded.
func maskURLError(err error) error {
var urlErr *url.Error
if !errors.As(err, &urlErr) {
return err
}
return &url.Error{
Op: urlErr.Op,
URL: MaskURL(urlErr.URL),
Err: urlErr.Err,
}
}

View File

@@ -0,0 +1,196 @@
package delivery_test
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// The path of a Slack incoming webhook URL is the credential:
// whoever holds these segments can post to the channel
// forever. None of them may reach a stored delivery error,
// which lives on disk in the per-webhook database and is
// serialized by the JSON tag on DeliveryResult.Error.
const (
maskSecretPath = "/services/T00000000/B00000000/" +
"XXXXXXXXXXXXXXXXXXXXXXXX"
)
// assertNoCredential fails if the whole path or any single
// segment of it survived into the message, so a partial leak
// fails the test too.
func assertNoCredential(t *testing.T, msg string) {
t.Helper()
segments := []string{
maskSecretPath,
"services",
"T00000000",
"B00000000",
"XXXXXXXXXXXXXXXXXXXXXXXX",
}
for _, segment := range segments {
assert.NotContains(t, msg, segment)
}
}
// storedDeliveryError returns the error string persisted for a
// delivery, which is what an operator and any future API read.
func storedDeliveryError(
t *testing.T, db *gorm.DB, deliveryID string,
) string {
t.Helper()
var result database.DeliveryResult
require.NoError(t, db.Where(
"delivery_id = ?", deliveryID,
).First(&result).Error)
return result.Error
}
// deliverSlackTo runs a Slack delivery against webhookURL and
// returns the error string it persisted.
func deliverSlackTo(
t *testing.T, webhookURL string,
) string {
t.Helper()
db := testWebhookDB(t)
e := testEngine(t, 1)
targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: webhookURL,
},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"test":true}`)
dlv := seedDelivery(
t, db, event.ID, targetID,
database.DeliveryStatusPending,
)
d := buildSlackDelivery(
dlv, event, targetID,
"test-slack-mask", string(slackCfg),
)
e.ExportDeliverSlack(context.TODO(), db, d)
assertDeliveryStatus(t, db, dlv.ID,
database.DeliveryStatusFailed,
)
return storedDeliveryError(t, db, dlv.ID)
}
// TestDeliverSlack_TransportErrorMasksWebhookURL is the
// load-bearing regression test: a transport failure must not
// persist the webhook URL's credential into the database, and
// must still say what went wrong and where.
func TestDeliverSlack_TransportErrorMasksWebhookURL(
t *testing.T,
) {
t.Parallel()
// A server closed before use gives a deterministic
// transport failure against a known host.
ts := httptest.NewServer(http.NewServeMux())
host := ts.URL
ts.Close()
errMsg := deliverSlackTo(t, host+maskSecretPath)
require.NotEmpty(t, errMsg)
assertNoCredential(t, errMsg)
// The diagnostic value survives: the operation, the host
// and the transport failure are all still reported, and
// only the path is elided.
assert.Contains(t, errMsg, "sending request")
assert.Contains(t, errMsg, "Post")
assert.Contains(t, errMsg, host+"/...")
assert.Contains(t, errMsg, "connection refused")
}
// TestDeliverSlack_UnparsableURLMasksWebhookURL covers the
// other error path out of a Slack attempt: url.Parse also
// embeds the whole URL in the error it returns.
func TestDeliverSlack_UnparsableURLMasksWebhookURL(
t *testing.T,
) {
t.Parallel()
errMsg := deliverSlackTo(
t,
"https://hooks.slack.com"+maskSecretPath+"\n",
)
require.NotEmpty(t, errMsg)
assertNoCredential(t, errMsg)
assert.Contains(t, errMsg, "invalid control character")
}
// TestDoHTTPRequest_TransportErrorMasksURL proves the HTTP
// target's transport errors are masked too; its destination
// URL can carry a token in a query string.
func TestDoHTTPRequest_TransportErrorMasksURL(t *testing.T) {
t.Parallel()
ts := httptest.NewServer(http.NewServeMux())
host := ts.URL
ts.Close()
e := testEngine(t, 1)
cfg, err := e.ExportParseHTTPConfig(
newHTTPTargetConfig(host + maskSecretPath),
)
require.NoError(t, err)
statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
context.TODO(), cfg,
&database.Event{Body: `{"test":true}`},
)
require.Error(t, reqErr)
assert.Zero(t, statusCode)
assertNoCredential(t, reqErr.Error())
assert.Contains(t, reqErr.Error(), host+"/...")
assert.Contains(
t, reqErr.Error(), "connection refused",
)
}
// TestValidateTargetURL_UnparsableURLIsMasked proves the SSRF
// validator's error does not carry the submitted URL, which
// the handler both logs and shows.
func TestValidateTargetURL_UnparsableURLIsMasked(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.TODO(),
"https://hooks.slack.com"+maskSecretPath+"\n",
)
require.Error(t, err)
assertNoCredential(t, err.Error())
assert.Contains(t, err.Error(), "invalid URL")
}

View File

@@ -1,28 +1,34 @@
// Package globals provides build-time variables injected via ldflags.
package globals package globals
import ( import (
"go.uber.org/fx" "go.uber.org/fx"
) )
// these get populated from main() and copied into the Globals object. // Build-time variables populated from main() and copied into the
// Globals object.
//
//nolint:gochecknoglobals // Build-time variables set by main().
var ( var (
Appname string Appname string
Version string Version string
Buildarch string
) )
// Globals holds build-time metadata about the application.
type Globals struct { type Globals struct {
Appname string Appname string
Version string Version string
Buildarch string
} }
// nolint:revive // lc parameter is required by fx even if unused // New creates a Globals instance from the package-level
// build-time variables.
//
//nolint:revive // lc parameter is required by fx even if unused.
func New(lc fx.Lifecycle) (*Globals, error) { func New(lc fx.Lifecycle) (*Globals, error) {
n := &Globals{ n := &Globals{
Appname: Appname, Appname: Appname,
Buildarch: Buildarch,
Version: Version, Version: Version,
} }
return n, nil return n, nil
} }

View File

@@ -1,30 +1,30 @@
package globals package globals_test
import ( import (
"testing" "testing"
"go.uber.org/fx/fxtest" "sneak.berlin/go/webhooker/internal/globals"
) )
func TestNew(t *testing.T) { func TestGlobalsFields(t *testing.T) {
// Set test values t.Parallel()
Appname = "test-app"
Version = "1.0.0"
Buildarch = "test-arch"
lc := fxtest.NewLifecycle(t) g := &globals.Globals{
globals, err := New(lc) Appname: "test-app",
if err != nil { Version: "1.0.0",
t.Fatalf("New() error = %v", err)
} }
if globals.Appname != "test-app" { if g.Appname != "test-app" {
t.Errorf("Appname = %v, want %v", globals.Appname, "test-app") t.Errorf(
"Appname = %v, want %v",
g.Appname, "test-app",
)
} }
if globals.Version != "1.0.0" {
t.Errorf("Version = %v, want %v", globals.Version, "1.0.0") if g.Version != "1.0.0" {
} t.Errorf(
if globals.Buildarch != "test-arch" { "Version = %v, want %v",
t.Errorf("Buildarch = %v, want %v", globals.Buildarch, "test-arch") g.Version, "1.0.0",
)
} }
} }

View File

@@ -13,12 +13,13 @@ func (h *Handlers) HandleLoginPage() http.HandlerFunc {
sess, err := h.session.Get(r) sess, err := h.session.Get(r)
if err == nil && h.session.IsAuthenticated(sess) { if err == nil && h.session.IsAuthenticated(sess) {
http.Redirect(w, r, "/", http.StatusSeeOther) http.Redirect(w, r, "/", http.StatusSeeOther)
return return
} }
// Render login page // Render login page
data := map[string]interface{}{ data := map[string]any{
"Error": "", tmplKeyError: "",
} }
h.renderTemplate(w, r, "login.html", data) h.renderTemplate(w, r, "login.html", data)
@@ -28,10 +29,13 @@ func (h *Handlers) HandleLoginPage() http.HandlerFunc {
// HandleLoginSubmit handles the login form submission (POST) // HandleLoginSubmit handles the login form submission (POST)
func (h *Handlers) HandleLoginSubmit() http.HandlerFunc { func (h *Handlers) HandleLoginSubmit() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
// Parse form data // The body size cap is enforced by the MaxBodySize
if err := r.ParseForm(); err != nil { // middleware, which runs before CSRF parses the form.
err := r.ParseForm()
if err != nil {
h.log.Error("failed to parse form", "error", err) h.log.Error("failed to parse form", "error", err)
http.Error(w, "Bad request", http.StatusBadRequest) http.Error(w, "Bad request", http.StatusBadRequest)
return return
} }
@@ -40,76 +44,159 @@ func (h *Handlers) HandleLoginSubmit() http.HandlerFunc {
// Validate input // Validate input
if username == "" || password == "" { if username == "" || password == "" {
data := map[string]interface{}{ h.renderLoginError(
"Error": "Username and password are required", w, r,
} "Username and password are required",
w.WriteHeader(http.StatusBadRequest) http.StatusBadRequest,
h.renderTemplate(w, r, "login.html", data) )
return return
} }
// Find user in database user, err := h.authenticateUser(
var user database.User w, r, username, password,
if err := h.db.DB().Where("username = ?", username).First(&user).Error; err != nil { )
h.log.Debug("user not found", "username", username)
data := map[string]interface{}{
"Error": "Invalid username or password",
}
w.WriteHeader(http.StatusUnauthorized)
h.renderTemplate(w, r, "login.html", data)
return
}
// Verify password
valid, err := database.VerifyPassword(password, user.Password)
if err != nil { if err != nil {
h.log.Error("failed to verify password", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
return return
} }
if !valid { err = h.createAuthenticatedSession(w, r, user)
h.log.Debug("invalid password", "username", username)
data := map[string]interface{}{
"Error": "Invalid username or password",
}
w.WriteHeader(http.StatusUnauthorized)
h.renderTemplate(w, r, "login.html", data)
return
}
// Create session
sess, err := h.session.Get(r)
if err != nil { if err != nil {
h.log.Error("failed to get session", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
return return
} }
// Set user in session h.log.Info(
h.session.SetUser(sess, user.ID, user.Username) "user logged in",
"username", username,
// Save session "user_id", user.ID,
if err := h.session.Save(r, w, sess); err != nil { )
h.log.Error("failed to save session", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
return
}
h.log.Info("user logged in", "username", username, "user_id", user.ID)
// Redirect to home page // Redirect to home page
http.Redirect(w, r, "/", http.StatusSeeOther) http.Redirect(w, r, "/", http.StatusSeeOther)
} }
} }
// renderLoginError renders the login page with an error message.
func (h *Handlers) renderLoginError(
w http.ResponseWriter,
r *http.Request,
msg string,
status int,
) {
data := map[string]any{
tmplKeyError: msg,
}
w.WriteHeader(status)
h.renderTemplate(w, r, "login.html", data)
}
// authenticateUser looks up and verifies a user's credentials.
// On failure it writes an HTTP response and returns an error.
func (h *Handlers) authenticateUser(
w http.ResponseWriter,
r *http.Request,
username, password string,
) (database.User, error) {
var user database.User
err := h.db.DB().Where(
"username = ?", username,
).First(&user).Error
if err != nil {
h.log.Debug("user not found", "username", username)
h.renderLoginError(
w, r,
"Invalid username or password",
http.StatusUnauthorized,
)
return user, err
}
valid, err := database.VerifyPassword(password, user.Password)
if err != nil {
h.log.Error("failed to verify password", "error", err)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return user, err
}
if !valid {
h.log.Debug("invalid password", "username", username)
h.renderLoginError(
w, r,
"Invalid username or password",
http.StatusUnauthorized,
)
return user, errInvalidPassword
}
return user, nil
}
// createAuthenticatedSession regenerates the session and stores
// user info. On failure it writes an HTTP response and returns
// an error.
func (h *Handlers) createAuthenticatedSession(
w http.ResponseWriter,
r *http.Request,
user database.User,
) error {
oldSess, err := h.session.Get(r)
if err != nil {
h.log.Error("failed to get session", "error", err)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return err
}
sess, err := h.session.Regenerate(r, w, oldSess)
if err != nil {
h.log.Error(
"failed to regenerate session", "error", err,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return err
}
h.session.SetUser(sess, user.ID, user.Username)
err = h.session.Save(r, w, sess)
if err != nil {
h.log.Error("failed to save session", "error", err)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return err
}
return nil
}
// HandleLogout handles user logout // HandleLogout handles user logout
func (h *Handlers) HandleLogout() http.HandlerFunc { func (h *Handlers) HandleLogout() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
sess, err := h.session.Get(r) sess, err := h.session.Get(r)
if err != nil { if err != nil {
h.log.Error("failed to get session", "error", err) h.log.Error("failed to get session", "error", err)
http.Redirect(w, r, "/pages/login", http.StatusSeeOther) http.Redirect(
w, r, "/pages/login", http.StatusSeeOther,
)
return return
} }
@@ -117,8 +204,12 @@ func (h *Handlers) HandleLogout() http.HandlerFunc {
h.session.Destroy(sess) h.session.Destroy(sess)
// Save the destroyed session // Save the destroyed session
if err := h.session.Save(r, w, sess); err != nil { err = h.session.Save(r, w, sess)
h.log.Error("failed to save destroyed session", "error", err) if err != nil {
h.log.Error(
"failed to save destroyed session",
"error", err,
)
} }
// Redirect to login page // Redirect to login page

View File

@@ -0,0 +1,38 @@
package handlers
import "net/http"
// RenderTemplateForTest exposes renderTemplate for use in the
// handlers_test package.
func (s *Handlers) RenderTemplateForTest(
w http.ResponseWriter,
r *http.Request,
pageTemplate string,
data any,
) {
s.renderTemplate(w, r, pageTemplate, data)
}
// BuildSlackTargetConfigForTest exposes buildURLTargetConfig
// with the Slack target parameters for use in the
// handlers_test package.
func (s *Handlers) BuildSlackTargetConfigForTest(
w http.ResponseWriter,
r *http.Request,
targetURL string,
) (string, error) {
return s.buildURLTargetConfig(
w, r, targetURL, "webhookUrl",
"Webhook URL is required for Slack targets",
)
}
// BuildDatabaseTargetConfigForTest exposes
// buildDatabaseTargetConfig for use in the handlers_test
// package.
func (s *Handlers) BuildDatabaseTargetConfigForTest(
w http.ResponseWriter,
expiry string,
) (string, error) {
return s.buildDatabaseTargetConfig(w, expiry)
}

View File

@@ -1,75 +1,133 @@
// Package handlers provides HTTP request handlers for the
// webhooker web UI and API.
package handlers package handlers
import ( import (
"context" "context"
"encoding/json" "encoding/json"
"errors"
"html/template" "html/template"
"log/slog" "log/slog"
"net/http" "net/http"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/healthcheck" "sneak.berlin/go/webhooker/internal/healthcheck"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
"sneak.berlin/go/webhooker/templates" "sneak.berlin/go/webhooker/templates"
) )
// nolint:revive // HandlersParams is a standard fx naming convention const (
// maxBodyShift is the bit shift for 1 MB body limit.
maxBodyShift = 20
// recentEventLimit is the number of recent events to show.
recentEventLimit = 20
// paginationPerPage is the number of items per page.
paginationPerPage = 25
// tmplKeyError is the template data key for an error message.
tmplKeyError = "Error"
// tmplKeyWebhook is the template data key for a webhook.
tmplKeyWebhook = "Webhook"
)
// errInvalidPassword is returned when a password does not match.
var errInvalidPassword = errors.New("invalid password")
//nolint:revive // HandlersParams is a standard fx naming convention.
type HandlersParams struct { type HandlersParams struct {
fx.In fx.In
Logger *logger.Logger Logger *logger.Logger
Globals *globals.Globals Globals *globals.Globals
Database *database.Database Database *database.Database
WebhookDBMgr *database.WebhookDBManager
Healthcheck *healthcheck.Healthcheck Healthcheck *healthcheck.Healthcheck
Session *session.Session Session *session.Session
Notifier delivery.Notifier
Evictor delivery.WebhookEvictor
} }
// Handlers provides HTTP handler methods for all application
// routes.
type Handlers struct { type Handlers struct {
params *HandlersParams params *HandlersParams
log *slog.Logger log *slog.Logger
hc *healthcheck.Healthcheck hc *healthcheck.Healthcheck
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager
session *session.Session session *session.Session
notifier delivery.Notifier
evictor delivery.WebhookEvictor
templates map[string]*template.Template templates map[string]*template.Template
} }
// parsePageTemplate parses a page-specific template set from the embedded FS. // parsePageTemplate parses a page-specific template set from the
// Each page template is combined with the shared base, htmlheader, and navbar templates. // embedded FS. Each page template is combined with the shared
// base, htmlheader, and navbar templates. The page file must be
// listed first so that its root action ({{template "base" .}})
// becomes the template set's entry point.
func parsePageTemplate(pageFile string) *template.Template { func parsePageTemplate(pageFile string) *template.Template {
return template.Must( return template.Must(
template.ParseFS(templates.Templates, "htmlheader.html", "navbar.html", "base.html", pageFile), template.ParseFS(
templates.Templates,
pageFile,
"base.html",
"htmlheader.html",
"navbar.html",
),
) )
} }
func New(lc fx.Lifecycle, params HandlersParams) (*Handlers, error) { // New creates a Handlers instance, parsing all page templates at
// startup.
func New(
lc fx.Lifecycle,
params HandlersParams,
) (*Handlers, error) {
s := new(Handlers) s := new(Handlers)
s.params = &params s.params = &params
s.log = params.Logger.Get() s.log = params.Logger.Get()
s.hc = params.Healthcheck s.hc = params.Healthcheck
s.db = params.Database s.db = params.Database
s.dbMgr = params.WebhookDBMgr
s.session = params.Session s.session = params.Session
s.notifier = params.Notifier
s.evictor = params.Evictor
// Parse all page templates once at startup // Parse all page templates once at startup
s.templates = map[string]*template.Template{ s.templates = map[string]*template.Template{
"index.html": parsePageTemplate("index.html"),
"login.html": parsePageTemplate("login.html"), "login.html": parsePageTemplate("login.html"),
"profile.html": parsePageTemplate("profile.html"), "profile.html": parsePageTemplate("profile.html"),
"sources_list.html": parsePageTemplate("sources_list.html"),
"sources_new.html": parsePageTemplate("sources_new.html"),
"source_detail.html": parsePageTemplate("source_detail.html"),
"source_edit.html": parsePageTemplate("source_edit.html"),
"source_logs.html": parsePageTemplate("source_logs.html"),
} }
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error { OnStart: func(_ context.Context) error {
return nil return nil
}, },
}) })
return s, nil return s, nil
} }
//nolint:unparam // r parameter will be used in the future for request context func (s *Handlers) respondJSON(
func (s *Handlers) respondJSON(w http.ResponseWriter, r *http.Request, data interface{}, status int) { w http.ResponseWriter,
_ *http.Request,
data any,
status int,
) {
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status) w.WriteHeader(status)
if data != nil { if data != nil {
err := json.NewEncoder(w).Encode(data) err := json.NewEncoder(w).Encode(data)
if err != nil { if err != nil {
@@ -78,17 +136,15 @@ func (s *Handlers) respondJSON(w http.ResponseWriter, r *http.Request, data inte
} }
} }
//nolint:unparam,unused // will be used for handling JSON requests // serverError logs an error and sends a 500 response.
func (s *Handlers) decodeJSON(w http.ResponseWriter, r *http.Request, v interface{}) error { func (s *Handlers) serverError(
return json.NewDecoder(r.Body).Decode(v) w http.ResponseWriter, msg string, err error,
} ) {
s.log.Error(msg, "error", err)
// TemplateData represents the common data passed to templates http.Error(
type TemplateData struct { w, "Internal server error",
User *UserInfo http.StatusInternalServerError,
Version string )
UserCount int64
Uptime string
} }
// UserInfo represents user information for templates // UserInfo represents user information for templates
@@ -97,52 +153,91 @@ type UserInfo struct {
Username string Username string
} }
// renderTemplate renders a pre-parsed template with common data // templateDataWrapper wraps non-map data with common fields.
func (s *Handlers) renderTemplate(w http.ResponseWriter, r *http.Request, pageTemplate string, data interface{}) { type templateDataWrapper struct {
User *UserInfo
CSRFToken string
Data any
}
// getUserInfo extracts user info from the session.
func (s *Handlers) getUserInfo(
r *http.Request,
) *UserInfo {
sess, err := s.session.Get(r)
if err != nil || !s.session.IsAuthenticated(sess) {
return nil
}
username, ok := s.session.GetUsername(sess)
if !ok {
return nil
}
userID, ok := s.session.GetUserID(sess)
if !ok {
return nil
}
return &UserInfo{ID: userID, Username: username}
}
// renderTemplate renders a pre-parsed template with common
// data
func (s *Handlers) renderTemplate(
w http.ResponseWriter,
r *http.Request,
pageTemplate string,
data any,
) {
tmpl, ok := s.templates[pageTemplate] tmpl, ok := s.templates[pageTemplate]
if !ok { if !ok {
s.log.Error("template not found", "template", pageTemplate) s.log.Error(
http.Error(w, "Internal server error", http.StatusInternalServerError) "template not found",
"template", pageTemplate,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return return
} }
// Get user from session if available userInfo := s.getUserInfo(r)
var userInfo *UserInfo csrfToken := middleware.CSRFToken(r)
sess, err := s.session.Get(r)
if err == nil && s.session.IsAuthenticated(sess) {
if username, ok := s.session.GetUsername(sess); ok {
if userID, ok := s.session.GetUserID(sess); ok {
userInfo = &UserInfo{
ID: userID,
Username: username,
}
}
}
}
// If data is a map, merge user info into it if m, ok := data.(map[string]any); ok {
if m, ok := data.(map[string]interface{}); ok {
m["User"] = userInfo m["User"] = userInfo
if err := tmpl.Execute(w, m); err != nil { m["CSRFToken"] = csrfToken
s.log.Error("failed to execute template", "error", err) s.executeTemplate(w, tmpl, m)
http.Error(w, "Internal server error", http.StatusInternalServerError)
}
return
}
// Wrap data with base template data return
type templateDataWrapper struct {
User *UserInfo
Data interface{}
} }
wrapper := templateDataWrapper{ wrapper := templateDataWrapper{
User: userInfo, User: userInfo,
CSRFToken: csrfToken,
Data: data, Data: data,
} }
if err := tmpl.Execute(w, wrapper); err != nil { s.executeTemplate(w, tmpl, wrapper)
s.log.Error("failed to execute template", "error", err) }
http.Error(w, "Internal server error", http.StatusInternalServerError)
// executeTemplate runs the template and handles errors.
func (s *Handlers) executeTemplate(
w http.ResponseWriter,
tmpl *template.Template,
data any,
) {
err := tmpl.Execute(w, data)
if err != nil {
s.log.Error(
"failed to execute template", "error", err,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
} }
} }

View File

@@ -1,10 +1,11 @@
package handlers package handlers_test
import ( import (
"context"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"sync"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -12,120 +13,263 @@ import (
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/healthcheck" "sneak.berlin/go/webhooker/internal/healthcheck"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
func TestHandleIndex(t *testing.T) { type noopNotifier struct{}
var h *Handlers
app := fxtest.New( func (n *noopNotifier) Notify([]delivery.Task) {}
// recordingEvictor is a delivery.WebhookEvictor that records
// the webhook ids it was asked to evict, so a test can prove
// that a deletion path reached the delivery engine.
type recordingEvictor struct {
mu sync.Mutex
evicted []string
}
func (r *recordingEvictor) EvictWebhook(webhookID string) {
r.mu.Lock()
defer r.mu.Unlock()
r.evicted = append(r.evicted, webhookID)
}
// Evicted returns a copy of the recorded webhook ids.
func (r *recordingEvictor) Evicted() []string {
r.mu.Lock()
defer r.mu.Unlock()
out := make([]string, len(r.evicted))
copy(out, r.evicted)
return out
}
func newTestApp(
t *testing.T,
targets ...any,
) *fxtest.App {
t.Helper()
return fxtest.New(
t, t,
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
func() *config.Config { func() *config.Config {
return &config.Config{ return &config.Config{
// This is a base64 encoded 32-byte key: "test-session-key-32-bytes-long!!" DataDir: t.TempDir(),
SessionKey: "dGVzdC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
} }
}, },
func() *database.Database { database.New,
// Mock database with a mock DB method database.NewWebhookDBManager,
db := &database.Database{}
return db
},
healthcheck.New, healthcheck.New,
session.New, session.New,
New, func() delivery.Notifier {
return &noopNotifier{}
},
func() *recordingEvictor {
return &recordingEvictor{}
},
func(r *recordingEvictor) delivery.WebhookEvictor {
return r
},
handlers.New,
), ),
fx.Populate(&h), fx.Populate(targets...),
) )
app.RequireStart() }
defer app.RequireStop()
func TestHandleIndex_Unauthenticated(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder()
// Since we can't test actual template rendering without templates,
// let's test that the handler is created and doesn't panic
handler := h.HandleIndex() handler := h.HandleIndex()
assert.NotNil(t, handler) handler.ServeHTTP(w, req)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(
t, "/pages/login", w.Header().Get("Location"),
)
}
func TestHandleIndex_Authenticated(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder()
s, err := sess.Get(req)
require.NoError(t, err)
sess.SetUser(s, "test-user-id", "testuser")
err = sess.Save(req, w, s)
require.NoError(t, err)
req2 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
for _, cookie := range w.Result().Cookies() {
req2.AddCookie(cookie)
}
w2 := httptest.NewRecorder()
h.HandleIndex().ServeHTTP(w2, req2)
assert.Equal(t, http.StatusSeeOther, w2.Code)
assert.Equal(
t, "/sources", w2.Header().Get("Location"),
)
}
func TestBuildSlackTargetConfig_AcceptsPublicURL(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, "/", nil)
w := httptest.NewRecorder()
cfg, err := h.BuildSlackTargetConfigForTest(
w, req, "http://93.184.216.34/services/T00/B00/xxx",
)
require.NoError(t, err)
assert.Equal(t, http.StatusOK, w.Code)
assert.Contains(t, cfg, "webhookUrl")
}
func TestBuildSlackTargetConfig_RejectsReservedURL(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, "/", nil)
w := httptest.NewRecorder()
cfg, err := h.BuildSlackTargetConfigForTest(
w, req, "http://169.254.169.254/latest/meta-data/",
)
require.Error(t, err)
assert.Empty(t, cfg)
assert.Equal(t, http.StatusBadRequest, w.Code)
} }
func TestRenderTemplate(t *testing.T) { func TestRenderTemplate(t *testing.T) {
var h *Handlers t.Parallel()
app := fxtest.New( var h *handlers.Handlers
t,
fx.Provide( app := newTestApp(t, &h)
globals.New,
logger.New,
func() *config.Config {
return &config.Config{
// This is a base64 encoded 32-byte key: "test-session-key-32-bytes-long!!"
SessionKey: "dGVzdC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
}
},
func() *database.Database {
// Mock database
return &database.Database{}
},
healthcheck.New,
session.New,
New,
),
fx.Populate(&h),
)
app.RequireStart() app.RequireStart()
defer app.RequireStop()
t.Run("handles missing templates gracefully", func(t *testing.T) { t.Cleanup(app.RequireStop)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder() w := httptest.NewRecorder()
data := map[string]interface{}{ data := map[string]any{"Version": "1.0.0"}
"Version": "1.0.0",
}
// When a non-existent template name is requested, renderTemplate h.RenderTemplateForTest(
// should return an internal server error w, req, "nonexistent.html", data,
h.renderTemplate(w, req, "nonexistent.html", data) )
// Should return internal server error when template is not found assert.Equal(
assert.Equal(t, http.StatusInternalServerError, w.Code) t, http.StatusInternalServerError, w.Code,
}) )
} }
func TestFormatUptime(t *testing.T) { func TestBuildDatabaseTargetConfig_Valid(t *testing.T) {
tests := []struct { t.Parallel()
name string
duration string
expected string
}{
{
name: "minutes only",
duration: "45m",
expected: "45m",
},
{
name: "hours and minutes",
duration: "2h30m",
expected: "2h 30m",
},
{
name: "days, hours and minutes",
duration: "25h45m",
expected: "1d 1h 45m",
},
}
for _, tt := range tests { var h *handlers.Handlers
t.Run(tt.name, func(t *testing.T) {
d, err := time.ParseDuration(tt.duration) app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
// Empty expiry: the keep-forever default, empty config.
w := httptest.NewRecorder()
cfg, err := h.BuildDatabaseTargetConfigForTest(w, "")
require.NoError(t, err) require.NoError(t, err)
assert.Empty(t, cfg)
result := formatUptime(d) // Explicit never is stored as config.
assert.Equal(t, tt.expected, result) w = httptest.NewRecorder()
}) cfg, err = h.BuildDatabaseTargetConfigForTest(w, "never")
require.NoError(t, err)
assert.JSONEq(t, `{"expiry":"never"}`, cfg)
// A positive duration is stored as config.
w = httptest.NewRecorder()
cfg, err = h.BuildDatabaseTargetConfigForTest(w, "720h")
require.NoError(t, err)
assert.JSONEq(t, `{"expiry":"720h"}`, cfg)
}
func TestBuildDatabaseTargetConfig_RejectsBadExpiry(
t *testing.T,
) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
for _, bad := range []string{"nonsense", "7d", "-5h"} {
w := httptest.NewRecorder()
cfg, err := h.BuildDatabaseTargetConfigForTest(w, bad)
require.Error(t, err, "expiry %q", bad)
assert.Empty(t, cfg)
assert.Equal(
t, http.StatusBadRequest, w.Code,
"expiry %q should be rejected with 400", bad,
)
} }
} }

View File

@@ -4,9 +4,13 @@ import (
"net/http" "net/http"
) )
const httpStatusOK = 200
// HandleHealthCheck returns an HTTP handler that reports
// application health.
func (s *Handlers) HandleHealthCheck() http.HandlerFunc { func (s *Handlers) HandleHealthCheck() http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) { return func(w http.ResponseWriter, req *http.Request) {
resp := s.hc.Healthcheck() resp := s.hc.Healthcheck()
s.respondJSON(w, req, resp, 200) s.respondJSON(w, req, resp, httpStatusOK)
} }
} }

View File

@@ -1,54 +1,21 @@
package handlers package handlers
import ( import (
"fmt"
"net/http" "net/http"
"time"
"sneak.berlin/go/webhooker/internal/database"
) )
type IndexResponse struct { // HandleIndex returns a handler for the root path that redirects
Message string `json:"message"` // based on authentication state: authenticated users go to /sources
Version string `json:"version"` // (the dashboard), unauthenticated users go to the login page.
}
func (s *Handlers) HandleIndex() http.HandlerFunc { func (s *Handlers) HandleIndex() http.HandlerFunc {
// Calculate server start time return func(w http.ResponseWriter, r *http.Request) {
startTime := time.Now() sess, err := s.session.Get(r)
if err == nil && s.session.IsAuthenticated(sess) {
http.Redirect(w, r, "/sources", http.StatusSeeOther)
return func(w http.ResponseWriter, req *http.Request) { return
// Calculate uptime
uptime := time.Since(startTime)
uptimeStr := formatUptime(uptime)
// Get user count from database
var userCount int64
s.db.DB().Model(&database.User{}).Count(&userCount)
// Prepare template data
data := map[string]interface{}{
"Version": s.params.Globals.Version,
"Uptime": uptimeStr,
"UserCount": userCount,
} }
// Render the template http.Redirect(w, r, "/pages/login", http.StatusSeeOther)
s.renderTemplate(w, req, "index.html", data)
} }
} }
// formatUptime formats a duration into a human-readable string
func formatUptime(d time.Duration) string {
days := int(d.Hours()) / 24
hours := int(d.Hours()) % 24
minutes := int(d.Minutes()) % 60
if days > 0 {
return fmt.Sprintf("%dd %dh %dm", days, hours, minutes)
}
if hours > 0 {
return fmt.Sprintf("%dh %dm", hours, minutes)
}
return fmt.Sprintf("%dm", minutes)
}

View File

@@ -4,56 +4,201 @@ import (
"net/http" "net/http"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"sneak.berlin/go/webhooker/internal/database"
) )
// HandleProfile returns a handler for the user profile page // HandleProfile returns a handler for the user profile page
func (h *Handlers) HandleProfile() http.HandlerFunc { func (h *Handlers) HandleProfile() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
// Get username from URL sessionUserID, sessionUsername, ok :=
h.profileOwnerOrDeny(w, r)
if !ok {
return
}
h.renderProfile(w, r, sessionUserID, sessionUsername, "", "")
}
}
// HandlePasswordChange returns a handler that lets an authenticated
// user change their own password. It is served by the CSRF- and
// auth-protected POST /password route under /user/{username}.
func (h *Handlers) HandlePasswordChange() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
sessionUserID, sessionUsername, ok :=
h.profileOwnerOrDeny(w, r)
if !ok {
return
}
// The body size cap is enforced by the MaxBodySize
// middleware, which runs before CSRF parses the form.
err := r.ParseForm()
if err != nil {
h.log.Error("failed to parse form", "error", err)
http.Error(w, "Bad request", http.StatusBadRequest)
return
}
successMessage, errorMessage, handled := h.applyPasswordChange(
w,
sessionUsername,
r.FormValue("current_password"),
r.FormValue("new_password"),
r.FormValue("confirm_password"),
)
if !handled {
return
}
h.renderProfile(
w, r, sessionUserID, sessionUsername,
successMessage, errorMessage,
)
}
}
// applyPasswordChange verifies the current password and, on success,
// persists a fresh hash for the user, reusing the same helpers that
// bootstrap the admin user. It returns the success and error messages
// to display on the profile page. On an internal failure it writes a
// 500 response itself and returns handled=false, signalling the caller
// to stop without re-rendering the page.
func (h *Handlers) applyPasswordChange(
w http.ResponseWriter,
username, currentPassword, newPassword, confirmPassword string,
) (string, string, bool) {
// Load the user row so we can verify the current password and
// persist the new hash.
var user database.User
err := h.db.DB().Where(
"username = ?", username,
).First(&user).Error
if err != nil {
h.serverError(
w, "failed to load user for password change", err,
)
return "", "", false
}
valid, err := database.VerifyPassword(
currentPassword, user.Password,
)
if err != nil {
h.serverError(w, "failed to verify password", err)
return "", "", false
}
if !valid {
return "", "Current password is incorrect.", true
}
if newPassword == "" {
return "", "New password must not be empty.", true
}
if newPassword != confirmPassword {
return "", "New password and confirmation do not match.", true
}
hashedPassword, err := database.HashPassword(newPassword)
if err != nil {
h.serverError(w, "failed to hash new password", err)
return "", "", false
}
err = h.db.DB().Model(&user).Update(
"password", hashedPassword,
).Error
if err != nil {
h.serverError(w, "failed to update password", err)
return "", "", false
}
h.log.Info("user changed password", "username", username)
return "Password changed successfully.", "", true
}
// profileOwnerOrDeny resolves the session identity and enforces that a
// user may only act on their own profile (the requested username in the
// URL must equal the session username). On any failure it writes the
// appropriate HTTP response and returns ok=false; callers must stop
// when ok is false.
func (h *Handlers) profileOwnerOrDeny(
w http.ResponseWriter,
r *http.Request,
) (string, string, bool) {
requestedUsername := chi.URLParam(r, "username") requestedUsername := chi.URLParam(r, "username")
if requestedUsername == "" { if requestedUsername == "" {
http.NotFound(w, r) http.NotFound(w, r)
return
return "", "", false
} }
// Get session // RequireAuth middleware guarantees an authenticated session
// before this handler runs, so we only need to guard against an
// unexpected retrieval error.
sess, err := h.session.Get(r) sess, err := h.session.Get(r)
if err != nil || !h.session.IsAuthenticated(sess) { if err != nil {
// Redirect to login if not authenticated h.serverError(w, "failed to get session", err)
http.Redirect(w, r, "/pages/login", http.StatusSeeOther)
return return "", "", false
} }
// Get user info from session
sessionUsername, ok := h.session.GetUsername(sess) sessionUsername, ok := h.session.GetUsername(sess)
if !ok { if !ok {
h.log.Error("authenticated session missing username") h.log.Error("authenticated session missing username")
http.Error(w, "Internal server error", http.StatusInternalServerError) http.Error(
return w, "Internal server error",
http.StatusInternalServerError,
)
return "", "", false
} }
sessionUserID, ok := h.session.GetUserID(sess) sessionUserID, ok := h.session.GetUserID(sess)
if !ok { if !ok {
h.log.Error("authenticated session missing user ID") h.log.Error("authenticated session missing user ID")
http.Error(w, "Internal server error", http.StatusInternalServerError) http.Error(
return w, "Internal server error",
http.StatusInternalServerError,
)
return "", "", false
} }
// For now, only allow users to view their own profile // Only allow users to act on their own profile.
if requestedUsername != sessionUsername { if requestedUsername != sessionUsername {
http.Error(w, "Forbidden", http.StatusForbidden) http.Error(w, "Forbidden", http.StatusForbidden)
return
return "", "", false
} }
// Prepare data for template return sessionUserID, sessionUsername, true
data := map[string]interface{}{ }
"User": &UserInfo{
ID: sessionUserID, // renderProfile renders the profile page for the given user,
Username: sessionUsername, // optionally including a success or error message.
}, func (h *Handlers) renderProfile(
} w http.ResponseWriter,
r *http.Request,
// Render the profile page userID, username, successMessage, errorMessage string,
h.renderTemplate(w, r, "profile.html", data) ) {
} data := map[string]any{
"User": &UserInfo{
ID: userID,
Username: username,
},
"SuccessMessage": successMessage,
"ErrorMessage": errorMessage,
}
h.renderTemplate(w, r, "profile.html", data)
} }

View File

@@ -0,0 +1,293 @@
package handlers_test
import (
"context"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/go-chi/chi"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/session"
)
// authenticatedCookies creates an authenticated session for the given
// user and returns the resulting cookies for use on a later request.
func authenticatedCookies(
t *testing.T,
sess *session.Session,
userID, username string,
) []*http.Cookie {
t.Helper()
setupReq := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/setup", nil,
)
setupW := httptest.NewRecorder()
s, err := sess.Get(setupReq)
require.NoError(t, err)
sess.SetUser(s, userID, username)
require.NoError(t, sess.Save(setupReq, setupW, s))
cookies := setupW.Result().Cookies()
require.NotEmpty(t, cookies, "session cookie should be set")
return cookies
}
// profileRequest builds a GET request for the given profile username,
// attaching the supplied cookies and the chi URL parameter that the
// handler reads via chi.URLParam.
func profileRequest(
username string,
cookies []*http.Cookie,
) *http.Request {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/user/"+username, nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
rctx := chi.NewRouteContext()
rctx.URLParams.Add("username", username)
return req.WithContext(
context.WithValue(req.Context(), chi.RouteCtxKey, rctx),
)
}
func TestHandleProfile_OwnProfile_OK(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
cookies := authenticatedCookies(t, sess, "test-user-id", "testuser")
req := profileRequest("testuser", cookies)
w := httptest.NewRecorder()
h.HandleProfile().ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
}
func TestHandleProfile_OtherProfile_Forbidden(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
cookies := authenticatedCookies(t, sess, "test-user-id", "testuser")
req := profileRequest("otheruser", cookies)
w := httptest.NewRecorder()
h.HandleProfile().ServeHTTP(w, req)
assert.Equal(t, http.StatusForbidden, w.Code)
}
// TestUserRoute_Unauthenticated_RedirectedByMiddleware exercises the
// /user/{username} route group's middleware chain (CSRF then
// RequireAuth, matching setupUserRoutes) and proves that an
// unauthenticated request is redirected to /pages/login at the
// middleware layer, never reaching the endpoint handler.
func TestUserRoute_Unauthenticated_RedirectedByMiddleware(t *testing.T) {
t.Parallel()
var log *logger.Logger
var cfg *config.Config
var sess *session.Session
app := newTestApp(t, &log, &cfg, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
mw := middleware.NewForTest(log.Get(), cfg, sess)
var handlerReached bool
router := chi.NewRouter()
router.Route("/user/{username}", func(r chi.Router) {
r.Use(mw.CSRF())
r.Use(mw.RequireAuth())
r.Get("/", func(w http.ResponseWriter, _ *http.Request) {
handlerReached = true
w.WriteHeader(http.StatusOK)
})
})
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/user/testuser", nil,
)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
assert.False(
t, handlerReached,
"handler must not be reached for unauthenticated request",
)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(t, "/pages/login", w.Header().Get("Location"))
}
// passwordChangeRequest builds a POST request to the password-change
// endpoint for the given username, attaching the supplied cookies, an
// urlencoded form body, and the chi URL parameter the handler reads.
func passwordChangeRequest(
username string,
cookies []*http.Cookie,
form url.Values,
) *http.Request {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost,
"/user/"+username+"/password",
strings.NewReader(form.Encode()),
)
req.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
req.AddCookie(c)
}
rctx := chi.NewRouteContext()
rctx.URLParams.Add("username", username)
return req.WithContext(
context.WithValue(req.Context(), chi.RouteCtxKey, rctx),
)
}
func TestHandlePasswordChange_Success(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
var db *database.Database
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
oldHash, err := database.HashPassword("oldpassword")
require.NoError(t, err)
user := &database.User{Username: "pwuser", Password: oldHash}
require.NoError(t, db.DB().Create(user).Error)
cookies := authenticatedCookies(t, sess, user.ID, "pwuser")
form := url.Values{}
form.Set("current_password", "oldpassword")
form.Set("new_password", "newpassword")
form.Set("confirm_password", "newpassword")
req := passwordChangeRequest("pwuser", cookies, form)
w := httptest.NewRecorder()
h.HandlePasswordChange().ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
assert.Contains(
t, w.Body.String(), "Password changed successfully.",
)
var updated database.User
require.NoError(t,
db.DB().Where("username = ?", "pwuser").First(&updated).Error,
)
assert.NotEqual(t, oldHash, updated.Password)
valid, err := database.VerifyPassword(
"newpassword", updated.Password,
)
require.NoError(t, err)
assert.True(t, valid, "new password should verify against new hash")
}
func TestHandlePasswordChange_WrongCurrentPassword(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
var db *database.Database
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
oldHash, err := database.HashPassword("oldpassword")
require.NoError(t, err)
user := &database.User{Username: "pwuser2", Password: oldHash}
require.NoError(t, db.DB().Create(user).Error)
cookies := authenticatedCookies(t, sess, user.ID, "pwuser2")
form := url.Values{}
form.Set("current_password", "wrongpassword")
form.Set("new_password", "newpassword")
form.Set("confirm_password", "newpassword")
req := passwordChangeRequest("pwuser2", cookies, form)
w := httptest.NewRecorder()
h.HandlePasswordChange().ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
assert.Contains(
t, w.Body.String(), "Current password is incorrect.",
)
var unchanged database.User
require.NoError(t,
db.DB().Where(
"username = ?", "pwuser2",
).First(&unchanged).Error,
)
assert.Equal(
t, oldHash, unchanged.Password,
"stored hash must be unchanged after a rejected change",
)
}

View File

@@ -0,0 +1,356 @@
package handlers_test
import (
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/go-chi/chi"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
const (
deleteTestUserID = "test-user-id"
deleteTestUsername = "testuser"
// paramSourceID and paramTargetID are the chi URL parameter
// names the deletion handlers read.
paramSourceID = "sourceID"
paramTargetID = "targetID"
)
// seedWebhook inserts a webhook owned by the test user and
// returns it.
func seedWebhook(
t *testing.T,
db *database.Database,
) *database.Webhook {
t.Helper()
wh := &database.Webhook{
UserID: deleteTestUserID,
Name: "delete-me",
}
require.NoError(
t,
db.DB().Omit(clause.Associations).Create(wh).Error,
)
return wh
}
// seedTarget inserts a target of the given type for a webhook
// and returns it.
func seedTarget(
t *testing.T,
db *database.Database,
webhookID string,
targetType database.TargetType,
) *database.Target {
t.Helper()
tgt := &database.Target{
WebhookID: webhookID,
Name: "t-" + string(targetType),
Type: targetType,
Active: true,
}
require.NoError(
t,
db.DB().Omit(clause.Associations).Create(tgt).Error,
)
return tgt
}
// archivePathFor returns the archive database path the
// delivery engine would use for a webhook: beside the webhook's
// event database in the data directory.
func archivePathFor(
t *testing.T,
mgr *database.WebhookDBManager,
webhookID string,
) string {
t.Helper()
return filepath.Join(
filepath.Dir(mgr.DBPath(webhookID)),
"archive-"+webhookID+".db",
)
}
// writeArchivePlaceholder creates a stand-in archive file so a
// test can assert the file survives webhook deletion.
func writeArchivePlaceholder(path string) error {
return os.WriteFile(path, []byte("archive"), 0o600)
}
// postRequest builds an authenticated POST request carrying the
// given chi URL parameters.
func postRequest(
path string,
cookies []*http.Cookie,
params map[string]string,
) *http.Request {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, path, nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
rctx := chi.NewRouteContext()
for k, v := range params {
rctx.URLParams.Add(k, v)
}
return req.WithContext(
context.WithValue(req.Context(), chi.RouteCtxKey, rctx),
)
}
// TestHandleSourceDelete_EvictsArchiveWriter proves that
// deleting a webhook reaches the delivery engine and releases
// the webhook's archive writer, exercised through the real
// deletion handler rather than by calling the evictor directly.
func TestHandleSourceDelete_EvictsArchiveWriter(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
ev *recordingEvictor
)
app := newTestApp(t, &h, &sess, &db, &ev)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedTarget(t, db, wh.ID, database.TargetTypeDatabase)
cookies := authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
)
req := postRequest(
"/source/"+wh.ID+"/delete",
cookies,
map[string]string{paramSourceID: wh.ID},
)
w := httptest.NewRecorder()
h.HandleSourceDelete().ServeHTTP(w, req)
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(
t, []string{wh.ID}, ev.Evicted(),
"deleting a webhook should evict its archive writer",
)
}
// TestHandleSourceDelete_KeepsArchiveFile proves that deleting
// a webhook does not remove its archive database file: the
// archive is long-term storage the operator owns.
func TestHandleSourceDelete_KeepsArchiveFile(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
mgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &mgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
// Place an archive file where the delivery engine would.
archivePath := archivePathFor(t, mgr, wh.ID)
require.NoError(
t,
writeArchivePlaceholder(archivePath),
)
cookies := authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
)
req := postRequest(
"/source/"+wh.ID+"/delete",
cookies,
map[string]string{paramSourceID: wh.ID},
)
w := httptest.NewRecorder()
h.HandleSourceDelete().ServeHTTP(w, req)
require.Equal(t, http.StatusSeeOther, w.Code)
assert.FileExists(
t, archivePath,
"webhook deletion must not destroy the archive file",
)
}
// TestHandleTargetDelete_EvictsWhenLastDatabaseTargetGone
// proves that removing the last database target releases the
// archive writer.
func TestHandleTargetDelete_EvictsWhenLastDatabaseTargetGone(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
ev *recordingEvictor
)
app := newTestApp(t, &h, &sess, &db, &ev)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedTarget(
t, db, wh.ID, database.TargetTypeDatabase,
)
cookies := authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
)
req := postRequest(
"/source/"+wh.ID+"/targets/"+tgt.ID+"/delete",
cookies,
map[string]string{
paramSourceID: wh.ID,
paramTargetID: tgt.ID,
},
)
w := httptest.NewRecorder()
h.HandleTargetDelete().ServeHTTP(w, req)
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(
t, []string{wh.ID}, ev.Evicted(),
"removing the last database target should evict",
)
}
// TestHandleTargetDelete_KeepsWriterWhenDatabaseTargetRemains
// proves that deleting one of several database targets leaves
// the still-needed archive writer alone: the surviving target
// keeps archiving to the same file, so the writer must stay.
func TestHandleTargetDelete_KeepsWriterWhenDatabaseTargetRemains(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
ev *recordingEvictor
)
app := newTestApp(t, &h, &sess, &db, &ev)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
doomed := seedTarget(
t, db, wh.ID, database.TargetTypeDatabase,
)
seedTarget(t, db, wh.ID, database.TargetTypeDatabase)
cookies := authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
)
req := postRequest(
"/source/"+wh.ID+"/targets/"+doomed.ID+"/delete",
cookies,
map[string]string{
paramSourceID: wh.ID,
paramTargetID: doomed.ID,
},
)
w := httptest.NewRecorder()
h.HandleTargetDelete().ServeHTTP(w, req)
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Empty(
t, ev.Evicted(),
"a second database target still needs the writer",
)
}
// TestHandleTargetDelete_KeepsWriterWhenOtherTypeDeleted proves
// that deleting a target of an unrelated type leaves a
// still-needed archive writer alone: the webhook's database
// target is untouched, so its writer must stay.
func TestHandleTargetDelete_KeepsWriterWhenOtherTypeDeleted(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
ev *recordingEvictor
)
app := newTestApp(t, &h, &sess, &db, &ev)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedTarget(t, db, wh.ID, database.TargetTypeDatabase)
other := seedTarget(t, db, wh.ID, database.TargetTypeLog)
cookies := authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
)
req := postRequest(
"/source/"+wh.ID+"/targets/"+other.ID+"/delete",
cookies,
map[string]string{
paramSourceID: wh.ID,
paramTargetID: other.ID,
},
)
w := httptest.NewRecorder()
h.HandleTargetDelete().ServeHTTP(w, req)
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Empty(
t, ev.Evicted(),
"a surviving database target must keep its writer",
)
}

View File

@@ -0,0 +1,187 @@
package handlers_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
// The secret path segments of a Slack incoming webhook URL.
// Holding them is enough to post to the channel forever, so
// they must never reach the rendered page.
const (
slackSecretPath = "/services/T00000000/B00000000/" +
"XXXXXXXXXXXXXXXXXXXXXXXX"
slackWebhookURL = "https://hooks.slack.com" +
slackSecretPath
)
// seedConfiguredTarget inserts a target with a stored config
// blob and returns it.
func seedConfiguredTarget(
t *testing.T,
db *database.Database,
webhookID string,
targetType database.TargetType,
config string,
) *database.Target {
t.Helper()
tgt := &database.Target{
WebhookID: webhookID,
Name: "t-" + string(targetType),
Type: targetType,
Active: true,
Config: config,
}
require.NoError(
t,
db.DB().Omit(clause.Associations).Create(tgt).Error,
)
return tgt
}
// renderSourceDetailPage runs the real source detail handler
// for a webhook and returns the rendered HTML.
func renderSourceDetailPage(
t *testing.T,
h *handlers.Handlers,
sess *session.Session,
webhookID string,
) string {
t.Helper()
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet,
"/source/"+webhookID,
nil,
)
for _, c := range authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
) {
req.AddCookie(c)
}
rctx := chi.NewRouteContext()
rctx.URLParams.Add(paramSourceID, webhookID)
req = req.WithContext(
context.WithValue(
req.Context(), chi.RouteCtxKey, rctx,
),
)
w := httptest.NewRecorder()
h.HandleSourceDetail().ServeHTTP(w, req)
require.Equal(t, http.StatusOK, w.Code)
return w.Body.String()
}
// TestHandleSourceDetail_MasksSlackWebhookURL is the
// load-bearing regression test for the credential leak: the
// rendered page must show the Slack target without any of the
// secret path segments of its webhook URL.
func TestHandleSourceDetail_MasksSlackWebhookURL(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
)
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
)
body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.NotContains(t, body, slackSecretPath)
assert.NotContains(t, body, "T00000000")
assert.NotContains(t, body, "B00000000")
assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
)
assert.NotContains(t, body, "webhookUrl")
assert.Contains(t, body, "Webhook URL")
assert.Contains(t, body, "https://hooks.slack.com/...")
}
// TestHandleSourceDetail_RendersNamedTargetFields proves the
// other target types render labelled fields rather than the
// stored blob.
func TestHandleSourceDetail_RendersNamedTargetFields(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
)
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeHTTP,
`{"url":"https://example.com/hook","timeout":30,`+
`"headers":{"Authorization":"Bearer sekrit"}}`,
)
seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeDatabase,
`{"expiry":"720h"}`,
)
seedConfiguredTarget(
t, db, wh.ID,
database.TargetType("carrier-pigeon"),
`{"beak":"sharp"}`,
)
body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.Contains(t, body, "Destination URL")
assert.Contains(t, body, "https://example.com/hook")
assert.Contains(t, body, "Timeout")
assert.Contains(t, body, "1 configured")
assert.NotContains(t, body, "sekrit")
assert.Contains(t, body, "Archive Expiry")
assert.Contains(t, body, "720h")
// An unknown type gets the neutral placeholder, never the
// stored blob.
assert.Contains(t, body, "(unavailable)")
assert.NotContains(t, body, "beak")
}

View File

@@ -0,0 +1,134 @@
package handlers_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
// seedDeliveredEvent records an event and a delivery for it in
// the webhook's own database, so the log page has a delivery
// to render against the target.
func seedDeliveredEvent(
t *testing.T,
dbMgr *database.WebhookDBManager,
webhookID, targetID string,
) {
t.Helper()
webhookDB, err := dbMgr.GetDB(webhookID)
require.NoError(t, err)
event := &database.Event{
WebhookID: webhookID,
Method: http.MethodPost,
Body: `{"test":true}`,
ContentType: "application/json",
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(event).Error)
dlv := &database.Delivery{
EventID: event.ID,
TargetID: targetID,
Status: database.DeliveryStatusDelivered,
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(dlv).Error)
}
// renderSourceLogsPage runs the real event log handler for a
// webhook and returns the rendered HTML.
func renderSourceLogsPage(
t *testing.T,
h *handlers.Handlers,
sess *session.Session,
webhookID string,
) string {
t.Helper()
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet,
"/source/"+webhookID+"/logs",
nil,
)
for _, c := range authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
) {
req.AddCookie(c)
}
rctx := chi.NewRouteContext()
rctx.URLParams.Add(paramSourceID, webhookID)
req = req.WithContext(
context.WithValue(
req.Context(), chi.RouteCtxKey, rctx,
),
)
w := httptest.NewRecorder()
h.HandleSourceLogs().ServeHTTP(w, req)
require.Equal(t, http.StatusOK, w.Code)
return w.Body.String()
}
// TestHandleSourceLogs_MasksSlackWebhookURL proves the event
// log page is handed a display-safe projection of each target
// rather than the stored row, so the credential cannot be
// rendered from its template data.
func TestHandleSourceLogs_MasksSlackWebhookURL(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
)
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
body := renderSourceLogsPage(t, h, sess, wh.ID)
assert.NotContains(t, body, slackSecretPath)
assert.NotContains(t, body, "T00000000")
assert.NotContains(t, body, "B00000000")
assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
)
assert.NotContains(t, body, "webhookUrl")
// The page still identifies the delivery's target.
assert.Contains(t, body, tgt.Name)
assert.Contains(t, body, "delivered")
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,589 @@
package handlers_test
import (
"context"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"strings"
"testing"
"github.com/go-chi/chi"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
const (
// sourceTestUserID is the session user id used by the webhook
// management tests.
sourceTestUserID = "source-test-user"
// sourceIDParam is the chi URL parameter naming a webhook.
sourceIDParam = "sourceID"
)
// formRequest builds an urlencoded POST to path carrying the given
// cookies, plus any chi URL parameters the handler reads.
func formRequest(
path string,
cookies []*http.Cookie,
form url.Values,
urlParams map[string]string,
) *http.Request {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost,
path,
strings.NewReader(form.Encode()),
)
req.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
req.AddCookie(c)
}
rctx := chi.NewRouteContext()
for k, v := range urlParams {
rctx.URLParams.Add(k, v)
}
return req.WithContext(
context.WithValue(req.Context(), chi.RouteCtxKey, rctx),
)
}
// getRequest builds a GET to path carrying the given cookies, plus any
// chi URL parameters the handler reads.
func getRequest(
t *testing.T,
path string,
cookies []*http.Cookie,
urlParams map[string]string,
) *http.Request {
t.Helper()
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, path, nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
rctx := chi.NewRouteContext()
for k, v := range urlParams {
rctx.URLParams.Add(k, v)
}
return req.WithContext(
context.WithValue(req.Context(), chi.RouteCtxKey, rctx),
)
}
// submitCreate posts the webhook creation form with the given
// retention_days value (omitted entirely when retention is nil) and
// returns the recorder.
func submitCreate(
t *testing.T,
h *handlers.Handlers,
cookies []*http.Cookie,
name string,
retention *string,
) *httptest.ResponseRecorder {
t.Helper()
form := url.Values{}
form.Set("name", name)
if retention != nil {
form.Set("retention_days", *retention)
}
req := formRequest("/sources/new", cookies, form, nil)
w := httptest.NewRecorder()
h.HandleSourceCreateSubmit().ServeHTTP(w, req)
return w
}
// onlyWebhook loads the single webhook belonging to the test user.
func onlyWebhook(
t *testing.T,
db *database.Database,
) database.Webhook {
t.Helper()
var webhooks []database.Webhook
require.NoError(
t,
db.DB().Where("user_id = ?", sourceTestUserID).
Find(&webhooks).Error,
)
require.Len(t, webhooks, 1)
return webhooks[0]
}
// seedWebhookWithRetention inserts a webhook owned by the test user
// with an exact stored retention value, bypassing Webhook.BeforeSave
// via a column-level update so that legacy rows can be planted too.
func seedWebhookWithRetention(
t *testing.T,
db *database.Database,
retentionDays int,
) database.Webhook {
t.Helper()
wh := &database.Webhook{
UserID: sourceTestUserID,
Name: "seeded",
RetentionDays: retentionDays,
}
require.NoError(
t,
db.DB().Omit(clause.Associations).Create(wh).Error,
)
require.NoError(
t,
db.DB().Model(wh).
Update("retention_days", retentionDays).Error,
)
wh.RetentionDays = retentionDays
return *wh
}
// storedRetentionDays reads the retention_days column for a webhook.
func storedRetentionDays(
t *testing.T,
db *database.Database,
id string,
) int {
t.Helper()
var got int
require.NoError(
t,
db.DB().Model(&database.Webhook{}).
Where("id = ?", id).
Pluck("retention_days", &got).Error,
)
return got
}
// sourceTestEnv bundles the handler, session, and database a webhook
// management test drives.
type sourceTestEnv struct {
handlers *handlers.Handlers
db *database.Database
cookies []*http.Cookie
}
func setupSourceTest(t *testing.T) *sourceTestEnv {
t.Helper()
var h *handlers.Handlers
var sess *session.Session
var db *database.Database
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
return &sourceTestEnv{
handlers: h,
db: db,
cookies: authenticatedCookies(
t, sess, sourceTestUserID, "sourceuser",
),
}
}
// TestHandleSourceCreateSubmit_ZeroRetentionPersistsForever is the core
// regression test for the bug: the create form's 0 must reach the
// database as the retain-forever sentinel rather than being replaced by
// the column's default of 30.
func TestHandleSourceCreateSubmit_ZeroRetentionPersistsForever(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
zero := "0"
w := submitCreate(t, env.handlers, env.cookies, "forever", &zero)
require.Equal(t, http.StatusSeeOther, w.Code)
wh := onlyWebhook(t, env.db)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetentionDays(t, env.db, wh.ID),
)
assert.True(t, wh.RetainsForever())
}
func TestHandleSourceCreateSubmit_OmittedRetentionUsesDefault(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
w := submitCreate(t, env.handlers, env.cookies, "defaulted", nil)
require.Equal(t, http.StatusSeeOther, w.Code)
wh := onlyWebhook(t, env.db)
assert.Equal(
t,
database.DefaultRetentionDays,
storedRetentionDays(t, env.db, wh.ID),
)
}
// TestHandleSourceCreate_PrefillsDefaultFromConstant keeps the create
// form's pre-filled retention from becoming a third hardcoded copy of
// the 30-day policy.
func TestHandleSourceCreate_PrefillsDefaultFromConstant(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
w := httptest.NewRecorder()
env.handlers.HandleSourceCreate().ServeHTTP(
w, getRequest(t, "/sources/new", env.cookies, nil),
)
require.Equal(t, http.StatusOK, w.Code)
body := w.Body.String()
assert.Contains(
t, body,
`value="`+strconv.Itoa(database.DefaultRetentionDays)+`"`,
)
assert.NotContains(
t, body, `max="365"`,
"a max below the sentinel would block retain-forever",
)
assert.Contains(t, body, `min="0"`)
}
func TestHandleSourceCreateSubmit_InvalidRetentionIsRejected(
t *testing.T,
) {
t.Parallel()
for _, raw := range []string{"abc", "-1", "3.5"} {
t.Run(raw, func(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
w := submitCreate(
t, env.handlers, env.cookies, "bad", &raw,
)
assert.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(
t, w.Body.String(), "Retention must be",
)
var count int64
require.NoError(
t,
env.db.DB().Model(&database.Webhook{}).
Where("user_id = ?", sourceTestUserID).
Count(&count).Error,
)
assert.Zero(
t, count,
"no webhook may be created from a rejected form",
)
})
}
}
// TestHandleSourceCreateSubmit_OverflowingRetentionIsRejected covers
// the data-loss path directly: a finite retention above the largest one
// the reaper's cutoff arithmetic can represent must never reach the
// database, because the sweep would compute a future cutoff and delete
// every event the webhook has.
func TestHandleSourceCreateSubmit_OverflowingRetentionIsRejected(
t *testing.T,
) {
t.Parallel()
tooBig := strconv.Itoa(database.MaxFiniteRetentionDays + 1)
env := setupSourceTest(t)
w := submitCreate(t, env.handlers, env.cookies, "huge", &tooBig)
assert.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(
t, w.Body.String(),
strconv.Itoa(database.MaxFiniteRetentionDays),
"the form tells the user the actual ceiling",
)
var count int64
require.NoError(
t,
env.db.DB().Model(&database.Webhook{}).
Where("user_id = ?", sourceTestUserID).
Count(&count).Error,
)
assert.Zero(
t, count,
"no webhook may be created from a rejected form",
)
}
// TestHandleSourceCreateSubmit_SentinelIsAcceptedAsForever guards the
// boundary between "too large to represent" and "retain forever": the
// sentinel is above MaxFiniteRetentionDays, but it is the value the
// edit form pre-fills, so it must be accepted rather than rejected as
// out of range.
func TestHandleSourceCreateSubmit_SentinelIsAcceptedAsForever(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
sentinel := strconv.Itoa(database.RetentionForeverDays)
w := submitCreate(t, env.handlers, env.cookies, "forever", &sentinel)
require.Equal(t, http.StatusSeeOther, w.Code)
wh := onlyWebhook(t, env.db)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetentionDays(t, env.db, wh.ID),
)
}
// TestHandleSourceCreateSubmit_RejectedFormKeepsUserInput checks that a
// validation failure hands the user's typing back, matching what the
// edit form already does. Losing a long description to a mistyped
// retention value is the kind of thing that makes people give up on a
// form.
func TestHandleSourceCreateSubmit_RejectedFormKeepsUserInput(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
const (
name = "kept-name"
description = "a description worth not losing"
)
form := url.Values{}
form.Set("name", name)
form.Set("description", description)
form.Set("retention_days", "nonsense")
req := formRequest("/sources/new", env.cookies, form, nil)
w := httptest.NewRecorder()
env.handlers.HandleSourceCreateSubmit().ServeHTTP(w, req)
require.Equal(t, http.StatusBadRequest, w.Code)
body := w.Body.String()
assert.Contains(t, body, `value="`+name+`"`)
assert.Contains(t, body, description)
}
// submitEdit posts the webhook edit form for the given webhook.
func submitEdit(
t *testing.T,
env *sourceTestEnv,
wh database.Webhook,
retention string,
) *httptest.ResponseRecorder {
t.Helper()
form := url.Values{}
form.Set("name", wh.Name)
form.Set("description", wh.Description)
form.Set("retention_days", retention)
req := formRequest(
"/source/"+wh.ID+"/edit",
env.cookies,
form,
map[string]string{sourceIDParam: wh.ID},
)
w := httptest.NewRecorder()
env.handlers.HandleSourceEditSubmit().ServeHTTP(w, req)
return w
}
func TestHandleSourceEditSubmit_ZeroRetentionPersistsForever(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(
t, env.db, database.DefaultRetentionDays,
)
w := submitEdit(t, env, wh, "0")
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetentionDays(t, env.db, wh.ID),
)
}
func TestHandleSourceEditSubmit_InvalidRetentionIsRejected(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(
t, env.db, database.DefaultRetentionDays,
)
w := submitEdit(t, env, wh, "not-a-number")
assert.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(t, w.Body.String(), "Retention must be")
assert.Equal(
t,
database.DefaultRetentionDays,
storedRetentionDays(t, env.db, wh.ID),
"a rejected form must not change the stored retention",
)
}
func TestHandleSourceEditSubmit_EmptyRetentionLeavesValueUnchanged(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(t, env.db, 7)
w := submitEdit(t, env, wh, "")
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(t, 7, storedRetentionDays(t, env.db, wh.ID))
}
// TestSourceEditForm_ForeverWebhookRoundTrips walks the exact path that
// the removed max="365" cap used to break: render the edit form for a
// retain-forever webhook, confirm the pre-filled sentinel is not capped
// by browser validation, then submit that pre-filled value straight
// back and confirm the retention policy survives untouched.
func TestSourceEditForm_ForeverWebhookRoundTrips(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(
t, env.db, database.RetentionForeverDays,
)
req := getRequest(
t, "/source/"+wh.ID+"/edit", env.cookies,
map[string]string{sourceIDParam: wh.ID},
)
w := httptest.NewRecorder()
env.handlers.HandleSourceEdit().ServeHTTP(w, req)
require.Equal(t, http.StatusOK, w.Code)
sentinel := strconv.Itoa(database.RetentionForeverDays)
body := w.Body.String()
assert.Contains(
t, body, `value="`+sentinel+`"`,
"the edit form pre-fills the stored retention",
)
assert.NotContains(
t, body, `max="365"`,
"a max below the sentinel would block saving any edit",
)
// "Currently forever." is the rendered RetentionLabel, not the
// static hint below the input, which says "Enter 0 to retain events
// forever." A bare Contains of "forever" would pass for any
// webhook and would assert nothing about this one.
assert.Contains(
t, body, "Currently forever.",
"the form reports this webhook's policy as forever",
)
// Submit the pre-filled value back, exactly as a browser would.
post := submitEdit(t, env, wh, sentinel)
require.Equal(t, http.StatusSeeOther, post.Code)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetentionDays(t, env.db, wh.ID),
)
}
// TestSourceListAndDetail_ShowForeverNotTheSentinelNumber checks that
// the retain-forever value is never rendered to the user as a raw day
// count on either read-only view.
func TestSourceListAndDetail_ShowForeverNotTheSentinelNumber(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(
t, env.db, database.RetentionForeverDays,
)
sentinel := strconv.Itoa(database.RetentionForeverDays)
listW := httptest.NewRecorder()
env.handlers.HandleSourceList().ServeHTTP(
listW, getRequest(t, "/sources", env.cookies, nil),
)
require.Equal(t, http.StatusOK, listW.Code)
assert.Contains(t, listW.Body.String(), "Retention: forever")
assert.NotContains(t, listW.Body.String(), sentinel)
detailW := httptest.NewRecorder()
env.handlers.HandleSourceDetail().ServeHTTP(
detailW,
getRequest(
t, "/source/"+wh.ID, env.cookies,
map[string]string{sourceIDParam: wh.ID},
),
)
require.Equal(t, http.StatusOK, detailW.Code)
assert.Contains(t, detailW.Body.String(), "Retention: forever")
assert.NotContains(t, detailW.Body.String(), sentinel)
}

View File

@@ -0,0 +1,299 @@
package handlers_test
import (
"context"
"net/http"
"net/http/httptest"
"strconv"
"testing"
"time"
"github.com/stretchr/testify/assert"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
// Template data keys the page templates read. The handlers package has
// its own unexported constants for these; this is the external test
// package, so it needs its own.
const (
dataKeyWebhook = "Webhook"
dataKeyError = "Error"
)
// testWebhookID is the identifier given to the webhook under test on
// pages that render one.
const testWebhookID = "wh-1"
// renderPage renders a page template through the real template set as
// an authenticated user and returns the resulting HTML.
func renderPage(
t *testing.T,
h *handlers.Handlers,
sess *session.Session,
page string,
data map[string]any,
) string {
t.Helper()
cookies := authenticatedCookies(t, sess, "test-user-id", "testuser")
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
w := httptest.NewRecorder()
h.RenderTemplateForTest(w, req, page, data)
return w.Body.String()
}
// TestNavbarUsesWebhookTerminology pins the user-visible navigation
// label to "Webhooks". The /sources route is deliberately unchanged, so
// the assertion targets the link text rather than the href.
func TestNavbarUsesWebhookTerminology(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
// One item, so the list body renders too: it calls
// WebhookListItem.RetentionLabel, promoted from the embedded
// Webhook and therefore a pointer method. An empty list would
// skip that call and hide a template error behind the
// navigation assertions below.
item := handlers.WebhookListItem{}
item.Name = "wh"
item.ID = testWebhookID
item.RetentionDays = 14
body := renderPage(t, h, sess, "sources_list.html", map[string]any{
"Webhooks": []handlers.WebhookListItem{item},
})
assert.Contains(t, body, "Retention: 14 days")
assert.Contains(t, body, `class="btn-text">Webhooks</a>`)
assert.Contains(
t, body, `class="btn-text w-full text-left">Webhooks</a>`,
)
assert.Contains(
t, body,
`<h1 class="text-2xl font-medium text-gray-900">Webhooks</h1>`,
)
assert.NotContains(
t, body, ">Sources<",
"no user-visible element may still be labelled Sources",
)
assert.Contains(
t, body, `href="/sources"`,
"the /sources route itself must not change",
)
}
// TestEditPageUsesWebhookTerminology pins the edit page's heading and
// its back link. The link's href still points at /source/{id}, which is
// intentional: only user-visible copy changes.
func TestEditPageUsesWebhookTerminology(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
// The webhook goes in as a pointer because source_edit.html calls
// Webhook.RetentionLabel, a pointer method: a map element is not
// addressable, so a value here renders an error instead of the
// page.
webhook := &database.Webhook{Name: "wh", RetentionDays: 14}
webhook.ID = testWebhookID
body := renderPage(t, h, sess, "source_edit.html", map[string]any{
dataKeyWebhook: webhook,
dataKeyError: "",
})
assert.Contains(t, body, "Edit Webhook")
assert.NotContains(t, body, ">Sources<")
assert.Contains(t, body, `href="/source/wh-1"`)
}
// TestCreateFormRetentionCopyMatchesBehaviour pins the create form's
// retention copy to what the code does: the reaper permanently deletes
// events past the cutoff, an empty field falls back to
// DefaultRetentionDays, and 0 is rewritten to the retain-forever
// sentinel by Webhook.BeforeSave.
func TestCreateFormRetentionCopyMatchesBehaviour(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
body := renderPage(t, h, sess, "sources_new.html", map[string]any{
"Name": "",
"Description": "",
"DefaultRetentionDays": database.DefaultRetentionDays,
dataKeyError: "",
})
assert.Contains(
t, body,
"permanently deletes events older than this",
"the form must say retention is enforced by deletion",
)
assert.Contains(t, body, "Enter 0 to retain events forever")
assert.Contains(
t, body,
"leave blank to use the default of "+
strconv.Itoa(database.DefaultRetentionDays)+" days",
"blank means the default, not forever",
)
}
// TestEditFormRetentionCopyMatchesBehaviour pins the edit form's
// retention copy, including that it states the stored policy via
// RetentionLabel and that an empty field leaves that policy unchanged
// rather than meaning forever.
func TestEditFormRetentionCopyMatchesBehaviour(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
finite := &database.Webhook{Name: "wh", RetentionDays: 14}
finite.ID = testWebhookID
body := renderPage(t, h, sess, "source_edit.html", map[string]any{
dataKeyWebhook: finite,
dataKeyError: "",
})
assert.Contains(t, body, "Currently 14 days.")
assert.Contains(
t, body,
"permanently deletes events older than this",
)
assert.Contains(t, body, "Enter 0 to retain events forever")
assert.Contains(
t, body,
"leave blank to keep the current setting",
"blank means unchanged, not forever",
)
forever := &database.Webhook{
Name: "wh",
RetentionDays: database.RetentionForeverDays,
}
forever.ID = "wh-2"
foreverBody := renderPage(
t, h, sess, "source_edit.html", map[string]any{
dataKeyWebhook: forever,
dataKeyError: "",
},
)
assert.Contains(
t, foreverBody, "Currently forever.",
"a retain-forever webhook must not read as a day count",
)
assert.Contains(
t, foreverBody,
"No events are deleted while retention is set to forever",
)
assert.NotContains(
t, foreverBody,
"permanently deletes events older than this",
"the reaper skips retain-forever webhooks, so the form "+
"must not claim it deletes their events",
)
}
// TestEntrypointCopyButtonIsProgressiveEnhancement proves the copy
// affordance degrades: the button ships with the hidden attribute, so a
// browser that never runs app.js shows no dead control, and the URL is
// rendered as ordinary selectable text either way.
func TestEntrypointCopyButtonIsProgressiveEnhancement(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
entrypoint := database.Entrypoint{Path: "abc123"}
entrypoint.ID = "ep-1"
// The webhook goes in as a pointer because source_detail.html
// calls Webhook.RetentionLabel, a pointer method: a map element
// is not addressable, so a value here aborts execution partway
// down the page, after the copy button has already been flushed
// to the response.
webhook := &database.Webhook{Name: "wh", RetentionDays: 14}
webhook.ID = testWebhookID
webhook.CreatedAt = time.Date(
2026, time.January, 2, 3, 4, 5, 0, time.UTC,
)
body := renderPage(t, h, sess, "source_detail.html", map[string]any{
dataKeyWebhook: webhook,
"Entrypoints": []database.Entrypoint{entrypoint},
// The handler passes delivery.NewTargetViews(targets), never
// raw targets, so the test data has to have that same shape.
"Targets": delivery.NewTargetViews(nil),
"Events": []database.Event{},
"BaseURL": "https://hooks.example.com",
})
assert.Contains(
t, body,
`<code id="entrypoint-url-ep-1"`,
)
assert.Contains(t, body, "https://hooks.example.com/webhook/abc123")
assert.Contains(
t, body,
`hidden data-copy-target="entrypoint-url-ep-1"`,
"the button must start hidden and be revealed by script",
)
// renderTemplate streams to the ResponseWriter, so an abort
// midway still leaves everything above it in the body. This pins
// content from the last line of the template, which is below the
// assertions above: without it, a page that renders the copy
// button and then 500s passes.
assert.Contains(
t, body, "Retention: 14 days",
"the page must render to completion, not abort partway",
)
}

View File

@@ -1,42 +1,347 @@
package handlers package handlers
import ( import (
"encoding/json"
"io"
"net/http" "net/http"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// HandleWebhook handles incoming webhook requests at entrypoint URLs const (
// maxWebhookBodySize is the maximum allowed webhook
// request body (1 MB).
maxWebhookBodySize = 1 << maxBodyShift
)
// HandleWebhook handles incoming webhook requests at entrypoint
// URLs.
func (h *Handlers) HandleWebhook() http.HandlerFunc { func (h *Handlers) HandleWebhook() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
// Get entrypoint UUID from URL if r.Method != http.MethodPost {
entrypointUUID := chi.URLParam(r, "uuid") w.Header().Set("Allow", "POST")
if entrypointUUID == "" { http.Error(
http.NotFound(w, r) w,
"Method Not Allowed",
http.StatusMethodNotAllowed,
)
return
}
entrypointUUID := chi.URLParam(r, "uuid")
if entrypointUUID == "" {
http.NotFound(w, r)
return return
} }
// Log the incoming webhook request
h.log.Info("webhook request received", h.log.Info("webhook request received",
"entrypoint_uuid", entrypointUUID, "entrypoint_uuid", entrypointUUID,
"method", r.Method, "method", r.Method,
"remote_addr", r.RemoteAddr, "remote_addr", r.RemoteAddr,
"user_agent", r.UserAgent(),
) )
// Only POST methods are allowed for webhooks entrypoint, ok := h.lookupEntrypoint(
if r.Method != http.MethodPost { w, r, entrypointUUID,
w.Header().Set("Allow", "POST") )
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) if !ok {
return return
} }
// TODO: Implement webhook handling logic if !entrypoint.Active {
// Look up entrypoint by UUID, find parent webhook, fan out to targets http.Error(w, "Gone", http.StatusGone)
w.WriteHeader(http.StatusNotFound)
_, err := w.Write([]byte("unimplemented")) return
if err != nil {
h.log.Error("failed to write response", "error", err)
} }
h.processWebhookRequest(w, r, entrypoint)
} }
} }
// processWebhookRequest reads the body, serializes headers,
// loads targets, and delivers the event.
func (h *Handlers) processWebhookRequest(
w http.ResponseWriter,
r *http.Request,
entrypoint database.Entrypoint,
) {
body, ok := h.readWebhookBody(w, r)
if !ok {
return
}
headersJSON, err := json.Marshal(r.Header)
if err != nil {
h.serverError(w, "failed to serialize headers", err)
return
}
targets, err := h.loadActiveTargets(entrypoint.WebhookID)
if err != nil {
h.serverError(w, "failed to query targets", err)
return
}
h.createAndDeliverEvent(
w, r, entrypoint, body, headersJSON, targets,
)
}
// loadActiveTargets returns all active targets for a webhook.
func (h *Handlers) loadActiveTargets(
webhookID string,
) ([]database.Target, error) {
var targets []database.Target
err := h.db.DB().Where(
"webhook_id = ? AND active = ?",
webhookID, true,
).Find(&targets).Error
return targets, err
}
// lookupEntrypoint finds an entrypoint by UUID path.
func (h *Handlers) lookupEntrypoint(
w http.ResponseWriter,
r *http.Request,
entrypointUUID string,
) (database.Entrypoint, bool) {
var entrypoint database.Entrypoint
result := h.db.DB().Where(
"path = ?", entrypointUUID,
).First(&entrypoint)
if result.Error != nil {
h.log.Debug(
"entrypoint not found",
"path", entrypointUUID,
)
http.NotFound(w, r)
return entrypoint, false
}
return entrypoint, true
}
// readWebhookBody reads and validates the request body size.
func (h *Handlers) readWebhookBody(
w http.ResponseWriter,
r *http.Request,
) ([]byte, bool) {
body, err := io.ReadAll(
io.LimitReader(r.Body, maxWebhookBodySize+1),
)
if err != nil {
h.log.Error(
"failed to read request body", "error", err,
)
http.Error(
w, "Bad request", http.StatusBadRequest,
)
return nil, false
}
if len(body) > maxWebhookBodySize {
http.Error(
w,
"Request body too large",
http.StatusRequestEntityTooLarge,
)
return nil, false
}
return body, true
}
// createAndDeliverEvent creates the event and delivery records
// then notifies the delivery engine.
func (h *Handlers) createAndDeliverEvent(
w http.ResponseWriter,
r *http.Request,
entrypoint database.Entrypoint,
body, headersJSON []byte,
targets []database.Target,
) {
tx, err := h.beginWebhookTx(w, entrypoint.WebhookID)
if err != nil {
return
}
event := h.buildEvent(r, entrypoint, headersJSON, body)
err = tx.Create(event).Error
if err != nil {
tx.Rollback()
h.serverError(w, "failed to create event", err)
return
}
bodyPtr := inlineBody(body)
tasks := h.buildDeliveryTasks(
w, tx, event, entrypoint, targets, bodyPtr,
)
if tasks == nil {
return
}
err = tx.Commit().Error
if err != nil {
h.serverError(w, "failed to commit transaction", err)
return
}
h.finishWebhookResponse(w, event, entrypoint, tasks)
}
// beginWebhookTx opens a transaction on the per-webhook DB.
func (h *Handlers) beginWebhookTx(
w http.ResponseWriter,
webhookID string,
) (*gorm.DB, error) {
webhookDB, err := h.dbMgr.GetDB(webhookID)
if err != nil {
h.serverError(
w, "failed to get webhook database", err,
)
return nil, err
}
tx := webhookDB.Begin()
if tx.Error != nil {
h.serverError(
w, "failed to begin transaction", tx.Error,
)
return nil, tx.Error
}
return tx, nil
}
// inlineBody returns a pointer to body as a string if it fits
// within the inline size limit, or nil otherwise.
func inlineBody(body []byte) *string {
if len(body) < delivery.MaxInlineBodySize {
s := string(body)
return &s
}
return nil
}
// finishWebhookResponse notifies the delivery engine, logs the
// event, and writes the HTTP response.
func (h *Handlers) finishWebhookResponse(
w http.ResponseWriter,
event *database.Event,
entrypoint database.Entrypoint,
tasks []delivery.Task,
) {
if len(tasks) > 0 {
h.notifier.Notify(tasks)
}
h.log.Info("webhook event created",
"event_id", event.ID,
"webhook_id", entrypoint.WebhookID,
"entrypoint_id", entrypoint.ID,
"target_count", len(tasks),
)
w.WriteHeader(http.StatusOK)
_, err := w.Write([]byte(`{"status":"ok"}`))
if err != nil {
h.log.Error(
"failed to write response", "error", err,
)
}
}
// buildEvent creates a new Event struct from request data.
func (h *Handlers) buildEvent(
r *http.Request,
entrypoint database.Entrypoint,
headersJSON, body []byte,
) *database.Event {
return &database.Event{
WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID,
Method: r.Method,
Headers: string(headersJSON),
Body: string(body),
ContentType: r.Header.Get("Content-Type"),
}
}
// buildDeliveryTasks creates delivery records in the
// transaction and returns tasks for the delivery engine.
// Returns nil if an error occurred.
func (h *Handlers) buildDeliveryTasks(
w http.ResponseWriter,
tx *gorm.DB,
event *database.Event,
entrypoint database.Entrypoint,
targets []database.Target,
bodyPtr *string,
) []delivery.Task {
tasks := make([]delivery.Task, 0, len(targets))
for i := range targets {
dlv := &database.Delivery{
EventID: event.ID,
TargetID: targets[i].ID,
Status: database.DeliveryStatusPending,
}
err := tx.Create(dlv).Error
if err != nil {
tx.Rollback()
h.log.Error(
"failed to create delivery",
"target_id", targets[i].ID,
"error", err,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return nil
}
tasks = append(tasks, delivery.Task{
DeliveryID: dlv.ID,
EventID: event.ID,
WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID,
TargetID: targets[i].ID,
TargetName: targets[i].Name,
TargetType: targets[i].Type,
TargetConfig: targets[i].Config,
MaxRetries: targets[i].MaxRetries,
Method: event.Method,
Headers: event.Headers,
ContentType: event.ContentType,
Body: bodyPtr,
AttemptNum: 1,
})
}
return tasks
}

View File

@@ -1,3 +1,4 @@
// Package healthcheck provides application health status reporting.
package healthcheck package healthcheck
import ( import (
@@ -12,55 +13,51 @@ import (
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
// nolint:revive // HealthcheckParams is a standard fx naming convention //nolint:revive // HealthcheckParams is a standard fx naming convention.
type HealthcheckParams struct { type HealthcheckParams struct {
fx.In fx.In
Globals *globals.Globals Globals *globals.Globals
Config *config.Config Config *config.Config
Logger *logger.Logger Logger *logger.Logger
Database *database.Database Database *database.Database
} }
// Healthcheck tracks application uptime and reports health status.
type Healthcheck struct { type Healthcheck struct {
StartupTime time.Time StartupTime time.Time
log *slog.Logger log *slog.Logger
params *HealthcheckParams params *HealthcheckParams
} }
func New(lc fx.Lifecycle, params HealthcheckParams) (*Healthcheck, error) { // New creates a Healthcheck that records the startup time on fx
// start.
func New(
lc fx.Lifecycle,
params HealthcheckParams,
) (*Healthcheck, error) {
s := new(Healthcheck) s := new(Healthcheck)
s.params = &params s.params = &params
s.log = params.Logger.Get() s.log = params.Logger.Get()
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(_ context.Context) error { // nolint:revive // ctx unused but required by fx OnStart: func(_ context.Context) error {
s.StartupTime = time.Now() s.StartupTime = time.Now()
return nil return nil
}, },
OnStop: func(ctx context.Context) error { OnStop: func(_ context.Context) error {
return nil return nil
}, },
}) })
return s, nil return s, nil
} }
// nolint:revive // HealthcheckResponse is a clear, descriptive name // Healthcheck returns the current health status of the
type HealthcheckResponse struct { // application.
Status string `json:"status"` func (s *Healthcheck) Healthcheck() *Response {
Now string `json:"now"` resp := &Response{
UptimeSeconds int64 `json:"uptime_seconds"`
UptimeHuman string `json:"uptime_human"`
Version string `json:"version"`
Appname string `json:"appname"`
Maintenance bool `json:"maintenance_mode"`
}
func (s *Healthcheck) uptime() time.Duration {
return time.Since(s.StartupTime)
}
func (s *Healthcheck) Healthcheck() *HealthcheckResponse {
resp := &HealthcheckResponse{
Status: "ok", Status: "ok",
Now: time.Now().UTC().Format(time.RFC3339Nano), Now: time.Now().UTC().Format(time.RFC3339Nano),
UptimeSeconds: int64(s.uptime().Seconds()), UptimeSeconds: int64(s.uptime().Seconds()),
@@ -69,5 +66,21 @@ func (s *Healthcheck) Healthcheck() *HealthcheckResponse {
Version: s.params.Globals.Version, Version: s.params.Globals.Version,
Maintenance: s.params.Config.MaintenanceMode, Maintenance: s.params.Config.MaintenanceMode,
} }
return resp return resp
} }
// Response contains the JSON-serialised health status.
type Response struct {
Status string `json:"status"`
Now string `json:"now"`
UptimeSeconds int64 `json:"uptimeSeconds"`
UptimeHuman string `json:"uptimeHuman"`
Version string `json:"version"`
Appname string `json:"appname"`
Maintenance bool `json:"maintenanceMode"`
}
func (s *Healthcheck) uptime() time.Duration {
return time.Since(s.StartupTime)
}

View File

@@ -0,0 +1,57 @@
// Package lifecycle holds helpers shared by the components that
// register fx start and stop hooks.
package lifecycle
import (
"context"
"fmt"
"log/slog"
"sync"
)
// WaitForShutdown waits for wg to drain, bounded by ctx.
//
// fx hands OnStop a context carrying the application's stop
// timeout. A bare wg.Wait() discards that deadline, so a single
// goroutine that never observes cancellation — a delivery target
// that never returns, a SQLite operation blocked on a lock —
// hangs the process forever instead of letting it exit when the
// timeout expires, which is exactly when a clean shutdown matters
// most.
//
// On timeout it logs at error naming component and returns an
// error: the goroutines are still running, and reporting success
// would hide an unclean shutdown from the operator. The waiting
// goroutine outlives this call and exits when (if) wg drains; it
// holds nothing but the channel it closes.
func WaitForShutdown(
ctx context.Context,
log *slog.Logger,
component string,
wg *sync.WaitGroup,
) error {
done := make(chan struct{})
go func() {
defer close(done)
wg.Wait()
}()
select {
case <-done:
return nil
case <-ctx.Done():
log.Error(
"shutdown timed out, goroutines still running",
"component", component,
"error", ctx.Err(),
)
return fmt.Errorf(
"%s: shutdown timed out, "+
"goroutines still running: %w",
component, ctx.Err(),
)
}
}

View File

@@ -0,0 +1,62 @@
package lifecycle_test
import (
"context"
"log/slog"
"sync"
"testing"
"time"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/lifecycle"
)
// waitTimeout is the stop budget the timeout case gives a
// goroutine that never returns. The test's own patience is the
// go test deadline, so the only thing this value affects is how
// long the case takes.
const waitTimeout = 100 * time.Millisecond
func discardLogger() *slog.Logger {
return slog.New(slog.DiscardHandler)
}
func TestWaitForShutdown_DrainedGroup(t *testing.T) {
t.Parallel()
var wg sync.WaitGroup
wg.Go(func() {})
require.NoError(
t,
lifecycle.WaitForShutdown(
context.Background(), discardLogger(),
"test component", &wg,
),
)
}
func TestWaitForShutdown_ContextExpires(t *testing.T) {
t.Parallel()
release := make(chan struct{})
t.Cleanup(func() { close(release) })
var wg sync.WaitGroup
wg.Go(func() { <-release })
ctx, cancel := context.WithTimeout(
context.Background(), waitTimeout,
)
defer cancel()
err := lifecycle.WaitForShutdown(
ctx, discardLogger(), "test component", &wg,
)
require.ErrorIs(t, err, context.DeadlineExceeded)
require.ErrorContains(t, err, "test component")
}

View File

@@ -1,3 +1,5 @@
// Package logger provides structured logging with dynamic level
// control.
package logger package logger
import ( import (
@@ -10,19 +12,25 @@ import (
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
) )
// nolint:revive // LoggerParams is a standard fx naming convention //nolint:revive // LoggerParams is a standard fx naming convention.
type LoggerParams struct { type LoggerParams struct {
fx.In fx.In
Globals *globals.Globals Globals *globals.Globals
} }
// Logger wraps slog with dynamic level control and structured
// output.
type Logger struct { type Logger struct {
logger *slog.Logger logger *slog.Logger
levelVar *slog.LevelVar levelVar *slog.LevelVar
params LoggerParams params LoggerParams
} }
// nolint:revive // lc parameter is required by fx even if unused // New creates a Logger that outputs text (TTY) or JSON (non-TTY)
// to stdout.
//
//nolint:revive // lc parameter is required by fx even if unused.
func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) { func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) {
l := new(Logger) l := new(Logger)
l.params = params l.params = params
@@ -37,17 +45,22 @@ func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) {
tty = true tty = true
} }
replaceAttr := func(_ []string, a slog.Attr) slog.Attr { // nolint:revive // groups unused //nolint:revive // groups param unused but required by slog ReplaceAttr signature.
replaceAttr := func(_ []string, a slog.Attr) slog.Attr {
// Always use UTC for timestamps // Always use UTC for timestamps
if a.Key == slog.TimeKey { if a.Key == slog.TimeKey {
if t, ok := a.Value.Any().(time.Time); ok { if t, ok := a.Value.Any().(time.Time); ok {
return slog.Time(slog.TimeKey, t.UTC()) return slog.Time(slog.TimeKey, t.UTC())
} }
return a
} }
return a return a
} }
var handler slog.Handler var handler slog.Handler
opts := &slog.HandlerOptions{ opts := &slog.HandlerOptions{
Level: l.levelVar, Level: l.levelVar,
ReplaceAttr: replaceAttr, ReplaceAttr: replaceAttr,
@@ -69,24 +82,27 @@ func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) {
return l, nil return l, nil
} }
// EnableDebugLogging switches the log level to debug.
func (l *Logger) EnableDebugLogging() { func (l *Logger) EnableDebugLogging() {
l.levelVar.Set(slog.LevelDebug) l.levelVar.Set(slog.LevelDebug)
l.logger.Debug("debug logging enabled", "debug", true) l.logger.Debug("debug logging enabled", "debug", true)
} }
// Get returns the underlying slog.Logger.
func (l *Logger) Get() *slog.Logger { func (l *Logger) Get() *slog.Logger {
return l.logger return l.logger
} }
// Identify logs the application name and version at startup.
func (l *Logger) Identify() { func (l *Logger) Identify() {
l.logger.Info("starting", l.logger.Info("starting",
"appname", l.params.Globals.Appname, "appname", l.params.Globals.Appname,
"version", l.params.Globals.Version, "version", l.params.Globals.Version,
"buildarch", l.params.Globals.Buildarch,
) )
} }
// Helper methods to maintain compatibility with existing code // Writer returns an io.Writer suitable for standard library
// loggers.
func (l *Logger) Writer() io.Writer { func (l *Logger) Writer() io.Writer {
return os.Stdout return os.Stdout
} }

View File

@@ -1,65 +1,59 @@
package logger package logger_test
import ( import (
"testing" "testing"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger"
) )
func testGlobals() *globals.Globals {
return &globals.Globals{
Appname: "test-app",
Version: "1.0.0",
}
}
func TestNew(t *testing.T) { func TestNew(t *testing.T) {
// Set up globals t.Parallel()
globals.Appname = "test-app"
globals.Version = "1.0.0"
globals.Buildarch = "test-arch"
lc := fxtest.NewLifecycle(t) lc := fxtest.NewLifecycle(t)
g, err := globals.New(lc)
if err != nil { params := logger.LoggerParams{
t.Fatalf("globals.New() error = %v", err) Globals: testGlobals(),
} }
params := LoggerParams{ l, err := logger.New(lc, params)
Globals: g,
}
logger, err := New(lc, params)
if err != nil { if err != nil {
t.Fatalf("New() error = %v", err) t.Fatalf("New() error = %v", err)
} }
if logger.Get() == nil { if l.Get() == nil {
t.Error("Get() returned nil logger") t.Error("Get() returned nil logger")
} }
// Test that we can log without panic // Test that we can log without panic
logger.Get().Info("test message", "key", "value") l.Get().Info("test message", "key", "value")
} }
func TestEnableDebugLogging(t *testing.T) { func TestEnableDebugLogging(t *testing.T) {
// Set up globals t.Parallel()
globals.Appname = "test-app"
globals.Version = "1.0.0"
globals.Buildarch = "test-arch"
lc := fxtest.NewLifecycle(t) lc := fxtest.NewLifecycle(t)
g, err := globals.New(lc)
if err != nil { params := logger.LoggerParams{
t.Fatalf("globals.New() error = %v", err) Globals: testGlobals(),
} }
params := LoggerParams{ l, err := logger.New(lc, params)
Globals: g,
}
logger, err := New(lc, params)
if err != nil { if err != nil {
t.Fatalf("New() error = %v", err) t.Fatalf("New() error = %v", err)
} }
// Enable debug logging should not panic // Enable debug logging should not panic
logger.EnableDebugLogging() l.EnableDebugLogging()
// Test debug logging // Test debug logging
logger.Get().Debug("debug message", "test", true) l.Get().Debug("debug message", "test", true)
} }

View File

@@ -0,0 +1,84 @@
package middleware
import (
"net/http"
"github.com/gorilla/csrf"
)
// CSRFToken retrieves the CSRF token from the request context.
// Returns an empty string if the gorilla/csrf middleware has not run.
func CSRFToken(r *http.Request) string {
return csrf.Token(r)
}
// isClientTLS reports whether the client-facing connection uses TLS.
// It checks for a direct TLS connection (r.TLS) or a TLS-terminating
// reverse proxy that sets the standard X-Forwarded-Proto header.
func isClientTLS(r *http.Request) bool {
return r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https"
}
// CSRF returns middleware that provides CSRF protection using the
// gorilla/csrf library. The middleware uses the session authentication
// key to sign a CSRF cookie and validates a masked token submitted via
// the "csrf_token" form field (or the "X-CSRF-Token" header) on
// POST/PUT/PATCH/DELETE requests. Requests with an invalid or missing
// token receive a 403 Forbidden response.
//
// The middleware detects the client-facing transport protocol per-request
// using r.TLS and the X-Forwarded-Proto header. This allows correct
// behavior in all deployment scenarios:
//
// - Direct HTTPS: strict Referer/Origin checks, Secure cookies.
// - Behind a TLS-terminating reverse proxy: strict checks (the
// browser is on HTTPS, so Origin/Referer headers use https://),
// Secure cookies (the browser sees HTTPS from the proxy).
// - Direct HTTP: relaxed Referer/Origin checks via PlaintextHTTPRequest,
// non-Secure cookies so the browser sends them over HTTP.
//
// Two gorilla/csrf instances are maintained — one with Secure cookies
// (for TLS) and one without (for plaintext HTTP) — because the
// csrf.Secure option is set at creation time, not per-request.
func (m *Middleware) CSRF() func(http.Handler) http.Handler {
csrfErrorHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
m.log.Warn("csrf: token validation failed",
"method", r.Method,
"path", r.URL.Path,
"remote_addr", r.RemoteAddr,
"reason", csrf.FailureReason(r),
)
http.Error(w, "Forbidden - invalid CSRF token", http.StatusForbidden)
})
key := m.session.GetKey()
baseOpts := []csrf.Option{
csrf.FieldName("csrf_token"),
csrf.SameSite(csrf.SameSiteLaxMode),
csrf.Path("/"),
csrf.ErrorHandler(csrfErrorHandler),
}
// Two middleware instances with different Secure flags but the
// same signing key, so cookies are interchangeable between them.
tlsProtect := csrf.Protect(key, append(baseOpts, csrf.Secure(true))...)
httpProtect := csrf.Protect(key, append(baseOpts, csrf.Secure(false))...)
return func(next http.Handler) http.Handler {
tlsCSRF := tlsProtect(next)
httpCSRF := httpProtect(next)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if isClientTLS(r) {
// Client is on TLS (directly or via reverse proxy).
// Use Secure cookies and strict Origin/Referer checks.
tlsCSRF.ServeHTTP(w, r)
} else {
// Plaintext HTTP: use non-Secure cookies and tell
// gorilla/csrf to use "http" for scheme comparisons,
// skipping the strict Referer check that assumes TLS.
httpCSRF.ServeHTTP(w, csrf.PlaintextHTTPRequest(r))
}
})
}
}

View File

@@ -0,0 +1,494 @@
package middleware_test
import (
"context"
"crypto/tls"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware"
)
// csrfCookieName is the gorilla/csrf cookie name.
const csrfCookieName = "_gorilla_csrf"
// csrfGetToken performs a GET request through the CSRF middleware
// and returns the token and cookies.
func csrfGetToken(
t *testing.T,
csrfMW func(http.Handler) http.Handler,
getReq *http.Request,
) (string, []*http.Cookie) {
t.Helper()
var token string
getHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, r *http.Request) {
token = middleware.CSRFToken(r)
},
))
getW := httptest.NewRecorder()
getHandler.ServeHTTP(getW, getReq)
cookies := getW.Result().Cookies()
require.NotEmpty(t, cookies, "CSRF cookie should be set")
require.NotEmpty(t, token, "CSRF token should be set")
return token, cookies
}
// csrfPostWithToken performs a POST request with the given CSRF
// token and cookies through the middleware. Returns whether the
// handler was called and the response code.
func csrfPostWithToken(
t *testing.T,
csrfMW func(http.Handler) http.Handler,
postReq *http.Request,
token string,
cookies []*http.Cookie,
) (bool, int) {
t.Helper()
var called bool
postHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
form := url.Values{"csrf_token": {token}}
postReq.Body = http.NoBody
postReq.Body = nil
// Rebuild the request with the form body
rebuilt := httptest.NewRequestWithContext(
context.Background(),
postReq.Method, postReq.URL.String(),
strings.NewReader(form.Encode()),
)
rebuilt.Header = postReq.Header.Clone()
rebuilt.TLS = postReq.TLS
rebuilt.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
rebuilt.AddCookie(c)
}
postW := httptest.NewRecorder()
postHandler.ServeHTTP(postW, rebuilt)
return called, postW.Code
}
func TestCSRF_GETSetsToken(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var gotToken string
handler := m.CSRF()(http.HandlerFunc(
func(_ http.ResponseWriter, r *http.Request) {
gotToken = middleware.CSRFToken(r)
},
))
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/form", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.NotEmpty(
t, gotToken,
"CSRF token should be set in context on GET",
)
}
func TestCSRF_POSTWithValidToken(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
csrfMW := m.CSRF()
getReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/form", nil,
)
token, cookies := csrfGetToken(t, csrfMW, getReq)
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/form", nil,
)
called, _ := csrfPostWithToken(
t, csrfMW, postReq, token, cookies,
)
assert.True(
t, called,
"handler should be called with valid CSRF token",
)
}
// csrfPOSTWithoutTokenTest is a shared helper for testing POST
// requests without a CSRF token in both dev and prod modes.
func csrfPOSTWithoutTokenTest(
t *testing.T,
env string,
msg string,
) {
t.Helper()
m, _ := testMiddleware(t, env)
csrfMW := m.CSRF()
// GET to establish the CSRF cookie
getHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {},
))
getReq := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/form", nil)
getW := httptest.NewRecorder()
getHandler.ServeHTTP(getW, getReq)
cookies := getW.Result().Cookies()
// POST without CSRF token
var called bool
postHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/form", nil,
)
postReq.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
postReq.AddCookie(c)
}
postW := httptest.NewRecorder()
postHandler.ServeHTTP(postW, postReq)
assert.False(t, called, msg)
assert.Equal(t, http.StatusForbidden, postW.Code)
}
func TestCSRF_POSTWithoutToken(t *testing.T) {
t.Parallel()
csrfPOSTWithoutTokenTest(
t,
config.EnvironmentDev,
"handler should NOT be called without CSRF token",
)
}
func TestCSRF_POSTWithInvalidToken(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
csrfMW := m.CSRF()
// GET to establish the CSRF cookie
getHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {},
))
getReq := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/form", nil)
getW := httptest.NewRecorder()
getHandler.ServeHTTP(getW, getReq)
cookies := getW.Result().Cookies()
// POST with wrong CSRF token
var called bool
postHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
form := url.Values{"csrf_token": {"invalid-token-value"}}
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/form",
strings.NewReader(form.Encode()),
)
postReq.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
postReq.AddCookie(c)
}
postW := httptest.NewRecorder()
postHandler.ServeHTTP(postW, postReq)
assert.False(
t, called,
"handler should NOT be called with invalid CSRF token",
)
assert.Equal(t, http.StatusForbidden, postW.Code)
}
func TestCSRF_GETDoesNotValidate(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.CSRF()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/form", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"GET requests should pass through CSRF middleware",
)
}
func TestCSRFToken_NoMiddleware(t *testing.T) {
t.Parallel()
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
assert.Empty(
t, middleware.CSRFToken(req),
"CSRFToken should return empty string when "+
"middleware has not run",
)
}
// --- TLS Detection Tests ---
func TestIsClientTLS_DirectTLS(t *testing.T) {
t.Parallel()
r := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
r.TLS = &tls.ConnectionState{}
assert.True(
t, middleware.IsClientTLS(r),
"should detect direct TLS connection",
)
}
func TestIsClientTLS_XForwardedProto(t *testing.T) {
t.Parallel()
r := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
r.Header.Set("X-Forwarded-Proto", "https")
assert.True(
t, middleware.IsClientTLS(r),
"should detect TLS via X-Forwarded-Proto",
)
}
func TestIsClientTLS_PlaintextHTTP(t *testing.T) {
t.Parallel()
r := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
assert.False(
t, middleware.IsClientTLS(r),
"should detect plaintext HTTP",
)
}
func TestIsClientTLS_XForwardedProtoHTTP(t *testing.T) {
t.Parallel()
r := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
r.Header.Set("X-Forwarded-Proto", "http")
assert.False(
t, middleware.IsClientTLS(r),
"should detect plaintext when X-Forwarded-Proto is http",
)
}
// --- Production Mode: POST over plaintext HTTP ---
func TestCSRF_ProdMode_PlaintextHTTP_POSTWithValidToken(
t *testing.T,
) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentProd)
csrfMW := m.CSRF()
getReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/form", nil,
)
token, cookies := csrfGetToken(t, csrfMW, getReq)
// Verify cookie is NOT Secure (plaintext HTTP in prod)
for _, c := range cookies {
if c.Name == csrfCookieName {
assert.False(t, c.Secure,
"CSRF cookie should not be Secure "+
"over plaintext HTTP")
}
}
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/form", nil,
)
called, code := csrfPostWithToken(
t, csrfMW, postReq, token, cookies,
)
assert.True(t, called,
"handler should be called -- prod mode over "+
"plaintext HTTP must work")
assert.NotEqual(t, http.StatusForbidden, code,
"should not return 403")
}
// --- Production Mode: POST with X-Forwarded-Proto ---
func TestCSRF_ProdMode_BehindProxy_POSTWithValidToken(
t *testing.T,
) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentProd)
csrfMW := m.CSRF()
getReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "http://example.com/form", nil,
)
getReq.Header.Set("X-Forwarded-Proto", "https")
token, cookies := csrfGetToken(t, csrfMW, getReq)
// Verify cookie IS Secure (X-Forwarded-Proto: https)
for _, c := range cookies {
if c.Name == csrfCookieName {
assert.True(t, c.Secure,
"CSRF cookie should be Secure behind "+
"TLS proxy")
}
}
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "http://example.com/form", nil,
)
postReq.Header.Set("X-Forwarded-Proto", "https")
postReq.Header.Set("Origin", "https://example.com")
called, code := csrfPostWithToken(
t, csrfMW, postReq, token, cookies,
)
assert.True(t, called,
"handler should be called -- prod mode behind "+
"TLS proxy must work")
assert.NotEqual(t, http.StatusForbidden, code,
"should not return 403")
}
// --- Production Mode: direct TLS ---
func TestCSRF_ProdMode_DirectTLS_POSTWithValidToken(
t *testing.T,
) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentProd)
csrfMW := m.CSRF()
getReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "https://example.com/form", nil,
)
getReq.TLS = &tls.ConnectionState{}
token, cookies := csrfGetToken(t, csrfMW, getReq)
// Verify cookie IS Secure (direct TLS)
for _, c := range cookies {
if c.Name == csrfCookieName {
assert.True(t, c.Secure,
"CSRF cookie should be Secure over "+
"direct TLS")
}
}
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "https://example.com/form", nil,
)
postReq.TLS = &tls.ConnectionState{}
postReq.Header.Set("Origin", "https://example.com")
called, code := csrfPostWithToken(
t, csrfMW, postReq, token, cookies,
)
assert.True(t, called,
"handler should be called -- direct TLS must work")
assert.NotEqual(t, http.StatusForbidden, code,
"should not return 403")
}
// --- Production Mode: POST without token still rejects ---
func TestCSRF_ProdMode_PlaintextHTTP_POSTWithoutToken(
t *testing.T,
) {
t.Parallel()
csrfPOSTWithoutTokenTest(
t,
config.EnvironmentProd,
"handler should NOT be called without CSRF token "+
"even in prod+plaintext",
)
}

View File

@@ -0,0 +1,38 @@
package middleware
import (
"net/http"
)
// NewLoggingResponseWriterForTest wraps newLoggingResponseWriter
// for use in external test packages.
func NewLoggingResponseWriterForTest(
w http.ResponseWriter,
) *loggingResponseWriter {
return newLoggingResponseWriter(w)
}
// LoggingResponseWriterStatusCode returns the status code
// captured by the loggingResponseWriter.
func LoggingResponseWriterStatusCode(
lrw *loggingResponseWriter,
) int {
return lrw.statusCode
}
// IPFromHostPort exposes ipFromHostPort for testing.
func IPFromHostPort(hp string) string {
return ipFromHostPort(hp)
}
// IsClientTLS exposes isClientTLS for testing.
func IsClientTLS(r *http.Request) bool {
return isClientTLS(r)
}
// LoginRateLimitConst exposes the loginRateLimit constant.
const LoginRateLimitConst = loginRateLimit
// PasswordChangeRateLimitConst exposes the
// passwordChangeRateLimit constant.
const PasswordChangeRateLimitConst = passwordChangeRateLimit

View File

@@ -1,3 +1,5 @@
// Package middleware provides HTTP middleware for logging, auth,
// CORS, and metrics.
package middleware package middleware
import ( import (
@@ -19,26 +21,42 @@ import (
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
// nolint:revive // MiddlewareParams is a standard fx naming convention const (
// corsMaxAge is the maximum time (in seconds) that a
// preflight response can be cached.
corsMaxAge = 300
)
//nolint:revive // MiddlewareParams is a standard fx naming convention.
type MiddlewareParams struct { type MiddlewareParams struct {
fx.In fx.In
Logger *logger.Logger Logger *logger.Logger
Globals *globals.Globals Globals *globals.Globals
Config *config.Config Config *config.Config
Session *session.Session Session *session.Session
} }
// Middleware provides HTTP middleware for logging, CORS, auth, and
// metrics.
type Middleware struct { type Middleware struct {
log *slog.Logger log *slog.Logger
params *MiddlewareParams params *MiddlewareParams
session *session.Session session *session.Session
} }
func New(lc fx.Lifecycle, params MiddlewareParams) (*Middleware, error) { // New creates a Middleware from the provided fx parameters.
//
//nolint:revive // lc parameter is required by fx even if unused.
func New(
lc fx.Lifecycle,
params MiddlewareParams,
) (*Middleware, error) {
s := new(Middleware) s := new(Middleware)
s.params = &params s.params = &params
s.log = params.Logger.Get() s.log = params.Logger.Get()
s.session = params.Session s.session = params.Session
return s, nil return s, nil
} }
@@ -50,19 +68,24 @@ func ipFromHostPort(hp string) string {
if err != nil { if err != nil {
return "" return ""
} }
if len(h) > 0 && h[0] == '[' { if len(h) > 0 && h[0] == '[' {
return h[1 : len(h)-1] return h[1 : len(h)-1]
} }
return h return h
} }
type loggingResponseWriter struct { type loggingResponseWriter struct {
http.ResponseWriter http.ResponseWriter
statusCode int statusCode int
} }
// nolint:revive // unexported type is only used internally // newLoggingResponseWriter wraps w and records status codes.
func NewLoggingResponseWriter(w http.ResponseWriter) *loggingResponseWriter { func newLoggingResponseWriter(
w http.ResponseWriter,
) *loggingResponseWriter {
return &loggingResponseWriter{w, http.StatusOK} return &loggingResponseWriter{w, http.StatusOK}
} }
@@ -71,23 +94,30 @@ func (lrw *loggingResponseWriter) WriteHeader(code int) {
lrw.ResponseWriter.WriteHeader(code) lrw.ResponseWriter.WriteHeader(code)
} }
// type Middleware func(http.Handler) http.Handler // Logging returns middleware that logs each HTTP request with
// this returns a Middleware that is designed to do every request through the // timing and metadata.
// mux, note the signature:
func (s *Middleware) Logging() func(http.Handler) http.Handler { func (s *Middleware) Logging() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
start := time.Now() start := time.Now()
lrw := NewLoggingResponseWriter(w) lrw := newLoggingResponseWriter(w)
ctx := r.Context() ctx := r.Context()
defer func() { defer func() {
latency := time.Since(start) latency := time.Since(start)
requestID := "" requestID := ""
if reqID := ctx.Value(middleware.RequestIDKey); reqID != nil {
if reqID := ctx.Value(
middleware.RequestIDKey,
); reqID != nil {
if id, ok := reqID.(string); ok { if id, ok := reqID.(string); ok {
requestID = id requestID = id
} }
} }
s.log.Info("http request", s.log.Info("http request",
"request_start", start, "request_start", start,
"method", r.Method, "method", r.Method,
@@ -107,56 +137,110 @@ func (s *Middleware) Logging() func(http.Handler) http.Handler {
} }
} }
// CORS returns middleware that sets CORS headers (permissive in
// dev, no-op in prod).
func (s *Middleware) CORS() func(http.Handler) http.Handler { func (s *Middleware) CORS() func(http.Handler) http.Handler {
if s.params.Config.IsDev() {
// In development, allow any origin for local testing.
return cors.Handler(cors.Options{ return cors.Handler(cors.Options{
// CHANGEME! these are defaults, change them to suit your needs or
// read from environment/viper.
// AllowedOrigins: []string{"https://foo.com"}, // Use this to allow specific origin hosts
AllowedOrigins: []string{"*"}, AllowedOrigins: []string{"*"},
// AllowOriginFunc: func(r *http.Request, origin string) bool { return true }, AllowedMethods: []string{
AllowedMethods: []string{"GET", "POST", "PUT", "DELETE", "OPTIONS"}, "GET", "POST", "PUT", "DELETE", "OPTIONS",
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "X-CSRF-Token"}, },
AllowedHeaders: []string{
"Accept", "Authorization",
"Content-Type", "X-CSRF-Token",
},
ExposedHeaders: []string{"Link"}, ExposedHeaders: []string{"Link"},
AllowCredentials: false, AllowCredentials: false,
MaxAge: 300, // Maximum value not ignored by any of major browsers MaxAge: corsMaxAge,
}) })
}
// In production, the web UI is server-rendered so
// cross-origin requests are not expected. Return a no-op
// middleware.
return func(next http.Handler) http.Handler {
return next
}
} }
// RequireAuth returns middleware that checks for a valid session. // RequireAuth returns middleware that checks for a valid session.
// Unauthenticated users are redirected to the login page. // Unauthenticated users are redirected to the login page.
func (s *Middleware) RequireAuth() func(http.Handler) http.Handler { func (s *Middleware) RequireAuth() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
sess, err := s.session.Get(r) sess, err := s.session.Get(r)
if err != nil { if err != nil {
s.log.Debug("auth middleware: failed to get session", "error", err) s.log.Debug(
http.Redirect(w, r, "/pages/login", http.StatusSeeOther) "auth middleware: failed to get session",
"error", err,
)
http.Redirect(
w, r, "/pages/login", http.StatusSeeOther,
)
return return
} }
// IsAuthenticated also enforces both session expiry
// deadlines, so an idle-expired or absolutely-expired
// session lands here and is sent back to the login
// page.
if !s.session.IsAuthenticated(sess) { if !s.session.IsAuthenticated(sess) {
s.log.Debug("auth middleware: unauthenticated request", s.log.Debug(
"auth middleware: unauthenticated request",
"path", r.URL.Path, "path", r.URL.Path,
"method", r.Method, "method", r.Method,
) )
http.Redirect(w, r, "/pages/login", http.StatusSeeOther) http.Redirect(
w, r, "/pages/login", http.StatusSeeOther,
)
return return
} }
// This request authenticated with the session, so it
// counts as activity: push the idle deadline forward.
// This is the only place sessions are refreshed, which
// is what keeps an unauthenticated request from
// extending someone else's session. Touch advances the
// idle clock only -- the absolute cap is untouched --
// and reports false when nothing changed, so most
// requests do not re-issue the cookie. Save before the
// handler runs, while the headers are still ours to
// write.
if s.session.Touch(sess) {
err = s.session.Save(r, w, sess)
if err != nil {
s.log.Error(
"auth middleware: failed to refresh session",
"error", err,
)
}
}
next.ServeHTTP(w, r) next.ServeHTTP(w, r)
}) })
} }
} }
// Metrics returns middleware that records Prometheus HTTP metrics.
func (s *Middleware) Metrics() func(http.Handler) http.Handler { func (s *Middleware) Metrics() func(http.Handler) http.Handler {
mdlw := ghmm.New(ghmm.Config{ mdlw := ghmm.New(ghmm.Config{
Recorder: metrics.NewRecorder(metrics.Config{}), Recorder: metrics.NewRecorder(metrics.Config{}),
}) })
return func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler {
return std.Handler("", mdlw, next) return std.Handler("", mdlw, next)
} }
} }
// MetricsAuth returns middleware that protects metrics endpoints
// with basic auth.
func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler { func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler {
return basicauth.New( return basicauth.New(
"metrics", "metrics",
@@ -167,3 +251,128 @@ func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler {
}, },
) )
} }
// SecurityHeaders returns middleware that sets production security
// headers on every response: HSTS, X-Content-Type-Options,
// X-Frame-Options, CSP, Referrer-Policy, and Permissions-Policy.
func (s *Middleware) SecurityHeaders() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
w.Header().Set(
"Strict-Transport-Security",
"max-age=63072000; includeSubDomains; preload",
)
w.Header().Set(
"X-Content-Type-Options", "nosniff",
)
w.Header().Set("X-Frame-Options", "DENY")
w.Header().Set(
"Content-Security-Policy",
"default-src 'self'; "+
"script-src 'self' 'unsafe-inline'; "+
"style-src 'self' 'unsafe-inline'",
)
w.Header().Set(
"Referrer-Policy",
"strict-origin-when-cross-origin",
)
w.Header().Set(
"Permissions-Policy",
"camera=(), microphone=(), geolocation=()",
)
next.ServeHTTP(w, r)
})
}
}
// NoCache returns middleware that instructs browsers and
// intermediary proxies not to cache the response. It sets
// Cache-Control: no-store and Pragma: no-cache (the latter for
// older HTTP/1.0 intermediaries). Apply it to authenticated pages
// so webhook configuration and captured event data are not stored
// by caches.
func (s *Middleware) NoCache() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
w.Header().Set("Cache-Control", "no-store")
w.Header().Set("Pragma", "no-cache")
next.ServeHTTP(w, r)
})
}
}
// bodyLimitedMethod reports whether the request method carries a
// body that the MaxBodySize middleware should cap.
func bodyLimitedMethod(method string) bool {
return method == http.MethodPost ||
method == http.MethodPut ||
method == http.MethodPatch
}
// MaxBodySize returns middleware that limits the size of
// POST/PUT/PATCH request bodies to maxBytes. It must be registered
// before any middleware that parses the body — notably CSRF, which
// calls r.PostFormValue — so that form parsing happens under this
// cap rather than net/http's 10 MB default.
//
// Two enforcement paths exist, because http.MaxBytesReader alone
// cannot produce a 413: it reports the overflow as an error from
// Read, by which point the body parser downstream has already
// converted that error into its own response.
//
// - Declared oversize: the request announces a Content-Length
// greater than maxBytes. The middleware answers 413 Request
// Entity Too Large immediately and does not call the next
// handler, so neither CSRF nor the endpoint handler runs.
// - Undeclared oversize: the request is chunked (Content-Length
// of -1) or lies about its Content-Length. There is nothing to
// check up front, so http.MaxBytesReader hard-caps the body at
// maxBytes and the request fails downstream — the form parse
// errors out and CSRF rejects it with 403. The response is less
// precise than a 413, but the body is still never buffered
// beyond the cap, which is the property that matters.
func (s *Middleware) MaxBodySize(
maxBytes int64,
) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
if !bodyLimitedMethod(r.Method) {
next.ServeHTTP(w, r)
return
}
if r.ContentLength > maxBytes {
s.log.Warn(
"request body exceeds limit",
"method", r.Method,
"path", r.URL.Path,
"content_length", r.ContentLength,
"limit", maxBytes,
)
http.Error(
w,
"Request Entity Too Large",
http.StatusRequestEntityTooLarge,
)
return
}
r.Body = http.MaxBytesReader(w, r.Body, maxBytes)
next.ServeHTTP(w, r)
})
}
}

View File

@@ -0,0 +1,991 @@
package middleware_test
import (
"context"
"encoding/base64"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
"github.com/gorilla/sessions"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/session"
)
const testKeySize = 32
// testMiddleware creates a Middleware with minimal dependencies
// for testing. It uses a real session.Session backed by an
// in-memory cookie store.
func testMiddleware(
t *testing.T,
env string,
) (*middleware.Middleware, *session.Session) {
t.Helper()
m, s, _ := testMiddlewareWithSessionClock(t, env, 0, nil)
return m, s
}
// testMiddlewareWithSessionClock is testMiddleware with a
// configurable session idle timeout and a manually advanced clock,
// for the session-expiry tests. A nil clock uses the real one.
func testMiddlewareWithSessionClock(
t *testing.T,
env string,
idleTimeout time.Duration,
clock *fakeClock,
) (*middleware.Middleware, *session.Session, *fakeClock) {
t.Helper()
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
cfg := &config.Config{
Environment: env,
SessionIdleTimeout: idleTimeout,
}
// Create a real session manager with a known key
key := make([]byte, testKeySize)
for i := range key {
key[i] = byte(i)
}
store := sessions.NewCookieStore(key)
store.Options = &sessions.Options{
Path: "/",
MaxAge: 86400 * 7,
HttpOnly: true,
Secure: false,
SameSite: http.SameSiteLaxMode,
}
var now func() time.Time
if clock != nil {
now = clock.Now
}
sessManager := session.NewForTest(store, cfg, log, key, now)
m := middleware.NewForTest(log, cfg, sessManager)
return m, sessManager, clock
}
// fakeClock is a manually advanced clock, so session expiry can be
// tested without sleeping.
type fakeClock struct {
t time.Time
}
func (c *fakeClock) Now() time.Time {
return c.t
}
func (c *fakeClock) Advance(d time.Duration) {
c.t = c.t.Add(d)
}
// newFakeClock returns a clock started at a fixed instant.
func newFakeClock() *fakeClock {
return &fakeClock{
t: time.Date(
2026, time.January, 2, 3, 4, 5, 0, time.UTC,
),
}
}
// --- Logging Middleware Tests ---
func TestLogging_SetsStatusCode(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.Logging()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusCreated)
_, err := w.Write([]byte("created"))
if err != nil {
return
}
},
))
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/test", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(t, http.StatusCreated, w.Code)
assert.Equal(t, "created", w.Body.String())
}
func TestLogging_DefaultStatusOK(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.Logging()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
_, err := w.Write([]byte("ok"))
if err != nil {
return
}
},
))
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
// When no explicit WriteHeader is called, default is 200
assert.Equal(t, http.StatusOK, w.Code)
}
func TestLogging_PassesThroughToNext(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.Logging()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/api/webhook", nil,
)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"logging middleware should call the next handler",
)
}
// --- LoggingResponseWriter Tests ---
func TestLoggingResponseWriter_CapturesStatusCode(t *testing.T) {
t.Parallel()
w := httptest.NewRecorder()
lrw := middleware.NewLoggingResponseWriterForTest(w)
// Default should be 200
assert.Equal(
t, http.StatusOK,
middleware.LoggingResponseWriterStatusCode(lrw),
)
// WriteHeader should capture the status code
lrw.WriteHeader(http.StatusNotFound)
assert.Equal(
t, http.StatusNotFound,
middleware.LoggingResponseWriterStatusCode(lrw),
)
// Underlying writer should also get the status code
assert.Equal(t, http.StatusNotFound, w.Code)
}
func TestLoggingResponseWriter_WriteDelegatesToUnderlying(
t *testing.T,
) {
t.Parallel()
w := httptest.NewRecorder()
lrw := middleware.NewLoggingResponseWriterForTest(w)
n, err := lrw.Write([]byte("hello world"))
require.NoError(t, err)
assert.Equal(t, 11, n)
assert.Equal(t, "hello world", w.Body.String())
}
// --- CORS Middleware Tests ---
func TestCORS_DevMode_AllowsAnyOrigin(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.CORS()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
))
// Preflight request
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodOptions, "/api/test", nil,
)
req.Header.Set("Origin", "http://localhost:3000")
req.Header.Set("Access-Control-Request-Method", "POST")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
// In dev mode, CORS should allow any origin
assert.Equal(
t, "*",
w.Header().Get("Access-Control-Allow-Origin"),
)
}
func TestCORS_ProdMode_NoOp(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentProd)
var called bool
handler := m.CORS()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/api/test", nil,
)
req.Header.Set("Origin", "http://evil.com")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"prod CORS middleware should pass through to handler",
)
// In prod, no CORS headers should be set (no-op middleware)
assert.Empty(
t,
w.Header().Get("Access-Control-Allow-Origin"),
"prod mode should not set CORS headers",
)
}
// --- RequireAuth Middleware Tests ---
func TestRequireAuth_NoSession_RedirectsToLogin(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.RequireAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/dashboard", nil,
)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.False(
t, called,
"handler should not be called for "+
"unauthenticated request",
)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(t, "/pages/login", w.Header().Get("Location"))
}
func TestRequireAuth_AuthenticatedSession_PassesThrough(
t *testing.T,
) {
t.Parallel()
m, sessManager := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.RequireAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
// Create an authenticated session by making a request,
// setting session data, and saving the session cookie
setupReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/setup", nil,
)
setupW := httptest.NewRecorder()
sess, err := sessManager.Get(setupReq)
require.NoError(t, err)
sessManager.SetUser(sess, "user-123", "testuser")
require.NoError(t, sessManager.Save(setupReq, setupW, sess))
// Extract the cookie from the setup response
cookies := setupW.Result().Cookies()
require.NotEmpty(t, cookies, "session cookie should be set")
// Make the actual request with the session cookie
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/dashboard", nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"handler should be called for authenticated request",
)
}
func TestRequireAuth_UnauthenticatedSession_RedirectsToLogin(
t *testing.T,
) {
t.Parallel()
m, sessManager := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.RequireAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
// Create a session but don't authenticate it
setupReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/setup", nil,
)
setupW := httptest.NewRecorder()
sess, err := sessManager.Get(setupReq)
require.NoError(t, err)
// Don't call SetUser -- session exists but is not
// authenticated
require.NoError(t, sessManager.Save(setupReq, setupW, sess))
cookies := setupW.Result().Cookies()
require.NotEmpty(t, cookies)
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/dashboard", nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.False(
t, called,
"handler should not be called for "+
"unauthenticated session",
)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(t, "/pages/login", w.Header().Get("Location"))
}
// --- RequireAuth Session Expiry Tests ---
// loginCookies authenticates a new session and returns the cookies
// a browser would then send back.
func loginCookies(
t *testing.T,
sessManager *session.Session,
) []*http.Cookie {
t.Helper()
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/login", nil)
w := httptest.NewRecorder()
sess, err := sessManager.Get(req)
require.NoError(t, err)
sessManager.SetUser(sess, "user-123", "testuser")
require.NoError(t, sessManager.Save(req, w, sess))
cookies := w.Result().Cookies()
require.NotEmpty(t, cookies, "session cookie should be set")
return cookies
}
// runAuthed sends a request carrying cookies through RequireAuth
// and reports whether the protected handler ran, plus the response.
func runAuthed(
t *testing.T,
m *middleware.Middleware,
cookies []*http.Cookie,
) (bool, *httptest.ResponseRecorder) {
t.Helper()
var called bool
handler := m.RequireAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/dashboard", nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
return called, w
}
// sessionCookies filters a response's cookies down to the session
// cookie, so tests can tell whether the session was re-issued.
func sessionCookies(
w *httptest.ResponseRecorder,
) []*http.Cookie {
var out []*http.Cookie
for _, c := range w.Result().Cookies() {
if c.Name == session.SessionName {
out = append(out, c)
}
}
return out
}
func TestRequireAuth_IdleExpiredSession_RedirectsToLogin(
t *testing.T,
) {
t.Parallel()
idle := time.Hour
m, sessManager, clock := testMiddlewareWithSessionClock(
t, config.EnvironmentDev, idle, newFakeClock(),
)
cookies := loginCookies(t, sessManager)
clock.Advance(idle)
called, w := runAuthed(t, m, cookies)
assert.False(
t, called,
"handler should not run for an idle-expired session",
)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(t, "/pages/login", w.Header().Get("Location"))
assert.Empty(
t, sessionCookies(w),
"an expired session must not be refreshed",
)
}
func TestRequireAuth_RefreshesIdleDeadlineOnActivity(
t *testing.T,
) {
t.Parallel()
idle := time.Hour
m, sessManager, clock := testMiddlewareWithSessionClock(
t, config.EnvironmentDev, idle, newFakeClock(),
)
cookies := loginCookies(t, sessManager)
// Activity halfway through the idle window.
clock.Advance(idle / 2)
called, w := runAuthed(t, m, cookies)
require.True(t, called, "handler should run while valid")
refreshed := sessionCookies(w)
require.NotEmpty(
t, refreshed,
"activity should re-issue the session cookie",
)
// Past the original deadline. The refreshed cookie is still
// good; the original one is not.
clock.Advance(idle - time.Second)
calledRefreshed, _ := runAuthed(t, m, refreshed)
assert.True(
t, calledRefreshed,
"refreshed session should outlive the original deadline",
)
calledStale, staleW := runAuthed(t, m, cookies)
assert.False(
t, calledStale,
"the pre-refresh cookie carries the old idle deadline",
)
assert.Equal(t, http.StatusSeeOther, staleW.Code)
}
func TestRequireAuth_UnauthenticatedRequestDoesNotRefresh(
t *testing.T,
) {
t.Parallel()
m, sessManager, _ := testMiddlewareWithSessionClock(
t, config.EnvironmentDev, time.Hour, newFakeClock(),
)
// A session cookie that exists but was never authenticated.
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/setup", nil)
setupW := httptest.NewRecorder()
sess, err := sessManager.Get(req)
require.NoError(t, err)
require.NoError(t, sessManager.Save(req, setupW, sess))
cookies := setupW.Result().Cookies()
require.NotEmpty(t, cookies)
called, w := runAuthed(t, m, cookies)
assert.False(t, called)
assert.Empty(
t, sessionCookies(w),
"an unauthenticated request must not stamp the session",
)
}
// --- NoCache Middleware Tests ---
func TestNoCache_SetsHeaders(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.NoCache()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/sources", nil,
)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"NoCache middleware should call the next handler",
)
assert.Equal(
t, "no-store",
w.Header().Get("Cache-Control"),
)
assert.Equal(
t, "no-cache",
w.Header().Get("Pragma"),
)
}
// --- MaxBodySize Middleware Tests ---
const testBodyLimit int64 = 64
// maxBodySizeHandler wraps a sentinel handler in MaxBodySize with
// testBodyLimit. The sentinel records whether it ran and how much of
// the body it managed to read, so tests can distinguish "never
// reached" from "reached but truncated".
type maxBodySizeResult struct {
called bool
read int
readErr error
response *httptest.ResponseRecorder
}
func runMaxBodySize(
t *testing.T,
req *http.Request,
) *maxBodySizeResult {
t.Helper()
m, _ := testMiddleware(t, config.EnvironmentDev)
res := &maxBodySizeResult{response: httptest.NewRecorder()}
handler := m.MaxBodySize(testBodyLimit)(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
res.called = true
body, err := io.ReadAll(r.Body)
res.read = len(body)
res.readErr = err
w.WriteHeader(http.StatusOK)
},
))
handler.ServeHTTP(res.response, req)
return res
}
// postWithBody builds a POST request whose Content-Length is
// accurate for the given payload size.
func postWithBody(size int) *http.Request {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login",
strings.NewReader(strings.Repeat("a", size)),
)
req.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
return req
}
func TestMaxBodySize_DeclaredOversize_413AndHandlerNotReached(
t *testing.T,
) {
t.Parallel()
res := runMaxBodySize(t, postWithBody(int(testBodyLimit)+1))
assert.False(
t, res.called,
"handler must not be reached for an oversized body",
)
assert.Equal(
t, http.StatusRequestEntityTooLarge, res.response.Code,
)
}
func TestMaxBodySize_AtLimit_PassesThrough(t *testing.T) {
t.Parallel()
res := runMaxBodySize(t, postWithBody(int(testBodyLimit)))
assert.True(
t, res.called,
"handler should be reached for a body at the limit",
)
require.NoError(t, res.readErr)
assert.Equal(t, int(testBodyLimit), res.read)
assert.Equal(t, http.StatusOK, res.response.Code)
}
func TestMaxBodySize_UnderLimit_PassesThrough(t *testing.T) {
t.Parallel()
res := runMaxBodySize(t, postWithBody(1))
assert.True(t, res.called)
require.NoError(t, res.readErr)
assert.Equal(t, 1, res.read)
assert.Equal(t, http.StatusOK, res.response.Code)
}
func TestMaxBodySize_GetWithOversizeBody_NotCapped(t *testing.T) {
t.Parallel()
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/pages/login",
strings.NewReader(
strings.Repeat("a", int(testBodyLimit)+1),
),
)
res := runMaxBodySize(t, req)
assert.True(
t, res.called,
"GET requests are not subject to the POST body cap",
)
require.NoError(t, res.readErr)
assert.Equal(t, int(testBodyLimit)+1, res.read)
}
// TestMaxBodySize_UndeclaredOversize_TruncatedAtCap covers the
// chunked / lying-Content-Length case: there is nothing to check up
// front, so the request reaches the handler but MaxBytesReader
// hard-caps the body and the read fails at the limit.
func TestMaxBodySize_UndeclaredOversize_TruncatedAtCap(
t *testing.T,
) {
t.Parallel()
req := postWithBody(int(testBodyLimit) + 1)
// Simulate a chunked request: no declared length.
req.ContentLength = -1
res := runMaxBodySize(t, req)
assert.True(
t, res.called,
"an undeclared oversize body cannot be rejected up front",
)
require.Error(
t, res.readErr,
"reading past the cap must fail",
)
assert.Equal(
t, int(testBodyLimit), res.read,
"the handler must not see more than the cap",
)
}
// --- Helper Tests ---
func TestIpFromHostPort(t *testing.T) {
t.Parallel()
tests := []struct {
name string
input string
expected string
}{
{"ipv4 with port", "192.168.1.1:8080", "192.168.1.1"},
{"ipv6 with port", "[::1]:8080", "::1"},
{"invalid format", "not-a-host-port", ""},
{"empty string", "", ""},
{"localhost", "127.0.0.1:80", "127.0.0.1"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
result := middleware.IPFromHostPort(tt.input)
assert.Equal(t, tt.expected, result)
})
}
}
// --- MetricsAuth Tests ---
// metricsAuthMiddleware creates a Middleware configured for
// metrics auth testing. This helper de-duplicates the setup in
// metrics auth test functions.
func metricsAuthMiddleware(
t *testing.T,
) *middleware.Middleware {
t.Helper()
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
cfg := &config.Config{
Environment: config.EnvironmentDev,
MetricsUsername: "admin",
MetricsPassword: "secret",
}
key := make([]byte, testKeySize)
store := sessions.NewCookieStore(key)
store.Options = &sessions.Options{Path: "/", MaxAge: 86400}
sessManager := session.NewForTest(store, cfg, log, key, nil)
return middleware.NewForTest(log, cfg, sessManager)
}
// runMetricsAuthRequest sends a GET /metrics request with the
// given basic-auth password through MetricsAuth and reports
// whether the wrapped handler ran plus the recorded response.
func runMetricsAuthRequest(
t *testing.T, password string,
) (bool, *httptest.ResponseRecorder) {
t.Helper()
m := metricsAuthMiddleware(t)
var called bool
handler := m.MetricsAuth()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/metrics", nil,
)
req.SetBasicAuth("admin", password)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
return called, w
}
func TestMetricsAuth_ValidCredentials(t *testing.T) {
t.Parallel()
called, w := runMetricsAuthRequest(t, "secret")
assert.True(
t, called,
"handler should be called with valid basic auth",
)
assert.Equal(t, http.StatusOK, w.Code)
}
func TestMetricsAuth_InvalidCredentials(t *testing.T) {
t.Parallel()
called, w := runMetricsAuthRequest(t, "wrong-password")
assert.False(
t, called,
"handler should not be called with invalid basic auth",
)
assert.Equal(t, http.StatusUnauthorized, w.Code)
}
func TestMetricsAuth_NoCredentials(t *testing.T) {
t.Parallel()
m := metricsAuthMiddleware(t)
var called bool
handler := m.MetricsAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/metrics", nil,
)
// No basic auth header
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.False(
t, called,
"handler should not be called without credentials",
)
assert.Equal(t, http.StatusUnauthorized, w.Code)
}
// --- CORS Dev Mode Detailed Tests ---
func TestCORS_DevMode_AllowsMethods(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.CORS()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
))
// Preflight for POST
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodOptions, "/api/webhooks", nil,
)
req.Header.Set("Origin", "http://localhost:5173")
req.Header.Set("Access-Control-Request-Method", "POST")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
allowMethods := w.Header().Get("Access-Control-Allow-Methods")
assert.Contains(t, allowMethods, "POST")
}
// --- Base64 key validation for completeness ---
func TestSessionKeyFormat(t *testing.T) {
t.Parallel()
// Verify that the session initialization correctly validates
// key format. A proper 32-byte key encoded as base64 should
// work.
key := make([]byte, testKeySize)
for i := range key {
key[i] = byte(i + 1)
}
encoded := base64.StdEncoding.EncodeToString(key)
decoded, err := base64.StdEncoding.DecodeString(encoded)
require.NoError(t, err)
assert.Len(t, decoded, testKeySize)
}

View File

@@ -0,0 +1,233 @@
package middleware
import (
"net/http"
"net/netip"
"slices"
"strings"
"time"
"github.com/go-chi/httprate"
)
const (
// loginRateLimit is the maximum number of login attempts
// per interval.
loginRateLimit = 5
// loginRateInterval is the time window for the rate limit.
loginRateInterval = 1 * time.Minute
// passwordChangeRateLimit is the maximum number of password
// change attempts per interval. Each attempt verifies the
// current password, so the endpoint must be rate-limited
// like any other password-based authentication endpoint.
passwordChangeRateLimit = 5
// passwordChangeRateInterval is the time window for the
// password change rate limit.
passwordChangeRateInterval = 1 * time.Minute
// receiverRateInterval is the time window for the webhook
// receiver rate limit. The configured limit is expressed in
// requests per minute.
receiverRateInterval = 1 * time.Minute
)
// normalizeAddr strips the IPv4-in-IPv6 wrapper and any zone from
// addr so that comparisons and bucket keys are canonical.
func normalizeAddr(addr netip.Addr) netip.Addr {
return addr.Unmap().WithZone("")
}
// isTrustedProxy reports whether addr belongs to a network the
// operator listed in TRUSTED_PROXIES. The list is empty by default,
// so by default nothing is trusted.
func (m *Middleware) isTrustedProxy(addr netip.Addr) bool {
for _, prefix := range m.params.Config.TrustedProxies {
if prefix.Contains(addr) {
return true
}
}
return false
}
// forwardedClientAddr returns the client address named by this
// request's X-Forwarded-For chain. It is consulted only for requests
// whose direct peer is a trusted proxy.
//
// X-Forwarded-For is the only header read. X-Real-IP and
// True-Client-IP are deliberately ignored: the reverse proxies in
// common use append to X-Forwarded-For and pass any other header the
// client sent through untouched, so believing a single-valued header
// would let a client behind the trusted proxy name its own bucket —
// the very bypass this gating exists to close.
//
// The chain is walked right to left, because the rightmost entry is
// the one the nearest proxy appended and everything to its left may
// have been written by the client. The first hop that is not itself
// a trusted proxy is the client. A hop that cannot be read as a bare
// address ends the walk: past it the chain is not the shape assumed
// here, so the caller falls back to the peer address.
func (m *Middleware) forwardedClientAddr(
r *http.Request,
) (netip.Addr, bool) {
hops := strings.Split(
strings.Join(r.Header.Values("X-Forwarded-For"), ","), ",",
)
for _, hop := range slices.Backward(hops) {
hop = strings.TrimSpace(hop)
if hop == "" {
continue
}
addr, err := netip.ParseAddr(hop)
if err != nil {
return netip.Addr{}, false
}
if addr = normalizeAddr(addr); !m.isTrustedProxy(addr) {
return addr, true
}
}
return netip.Addr{}, false
}
// rateLimitKey is the client identity every rate limiter in this
// package buckets on. Forwarded headers are honoured only when the
// direct peer (RemoteAddr) is inside the configured trusted-proxy
// set; otherwise the peer address itself is the key. Without that
// gate any client could mint a fresh bucket per request, or starve
// another client's bucket, by picking an X-Forwarded-For value —
// which makes every limit here decorative against a deliberate
// attacker.
func (m *Middleware) rateLimitKey(r *http.Request) (string, error) {
return m.clientKey(r), nil
}
// clientKey computes the bucket key described on rateLimitKey.
func (m *Middleware) clientKey(r *http.Request) string {
peer, err := netip.ParseAddr(ipFromHostPort(r.RemoteAddr))
if err != nil {
// Not an address we can reason about; key on the raw
// value rather than collapsing such peers into one
// shared bucket.
return r.RemoteAddr
}
peer = normalizeAddr(peer)
if !m.isTrustedProxy(peer) {
return peer.String()
}
if addr, ok := m.forwardedClientAddr(r); ok {
return addr.String()
}
return peer.String()
}
// tooManyRequests returns the 429 handler shared by every limiter:
// it logs the rejection with logMessage and answers with
// responseMessage. httprate adds the Retry-After header (RFC 6585).
func (m *Middleware) tooManyRequests(
logMessage, responseMessage string,
) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
m.log.Warn(logMessage, "path", r.URL.Path)
http.Error(w, responseMessage, http.StatusTooManyRequests)
}
}
// LoginRateLimit returns middleware that enforces per-IP rate
// limiting on login attempts using go-chi/httprate. Only POST
// requests are rate-limited; GET requests (rendering the login
// form) pass through unaffected. When the rate limit is exceeded,
// a 429 Too Many Requests response is returned. Clients are
// identified by rateLimitKey.
func (m *Middleware) LoginRateLimit() func(http.Handler) http.Handler {
return m.postRateLimit(
loginRateLimit,
loginRateInterval,
"login rate limit exceeded",
"Too many login attempts. Please try again later.",
)
}
// PasswordChangeRateLimit returns middleware that enforces
// per-IP rate limiting on password change attempts. The change
// endpoint verifies the current password, so without a limit a
// stolen session could be used to brute-force it; the limit
// matches the login endpoint's.
func (m *Middleware) PasswordChangeRateLimit() func(http.Handler) http.Handler {
return m.postRateLimit(
passwordChangeRateLimit,
passwordChangeRateInterval,
"password change rate limit exceeded",
"Too many password change attempts. "+
"Please try again later.",
)
}
// postRateLimit builds middleware that enforces a per-IP rate
// limit on POST requests only; all other methods pass through
// unaffected. Requests over the limit receive a 429 with the
// given response message, and each rejection is logged with the
// given log message. Clients are identified by rateLimitKey.
func (m *Middleware) postRateLimit(
limit int,
interval time.Duration,
logMessage, responseMessage string,
) func(http.Handler) http.Handler {
limiter := httprate.Limit(
limit,
interval,
httprate.WithKeyFuncs(m.rateLimitKey),
httprate.WithLimitHandler(
m.tooManyRequests(logMessage, responseMessage),
),
)
return func(next http.Handler) http.Handler {
limited := limiter(next)
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
// Only rate-limit POST requests.
if r.Method != http.MethodPost {
next.ServeHTTP(w, r)
return
}
limited.ServeHTTP(w, r)
})
}
}
// ReceiverRateLimit returns middleware that rate-limits the
// public webhook receiver endpoint per client IP per request
// path (the path contains the entrypoint UUID, so each sender
// is limited per entrypoint without affecting other senders or
// other entrypoints). The limit is Config.ReceiverRateLimit
// requests per minute. Requests over the limit receive a 429.
// Clients are identified by rateLimitKey.
func (m *Middleware) ReceiverRateLimit() func(http.Handler) http.Handler {
return httprate.Limit(
m.params.Config.ReceiverRateLimit,
receiverRateInterval,
httprate.WithKeyFuncs(
m.rateLimitKey,
httprate.KeyByEndpoint,
),
httprate.WithLimitHandler(m.tooManyRequests(
"webhook receiver rate limit exceeded",
"Too many requests. Please slow down.",
)),
)
}

View File

@@ -0,0 +1,611 @@
package middleware_test
import (
"context"
"fmt"
"log/slog"
"net/http"
"net/http/httptest"
"net/netip"
"os"
"testing"
"github.com/stretchr/testify/assert"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware"
)
func TestLoginRateLimit_AllowsGET(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var callCount int
handler := m.LoginRateLimit()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
callCount++
w.WriteHeader(http.StatusOK)
},
))
// GET requests should never be rate-limited
for i := range 20 {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/pages/login", nil,
)
req.RemoteAddr = "192.168.1.1:12345"
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(
t, http.StatusOK, w.Code,
"GET request %d should pass", i,
)
}
assert.Equal(t, 20, callCount)
}
// runPostLimitTest exercises a POST-only rate limit middleware:
// the first limit POSTs to path from ip must pass, and the next
// one must be rejected with 429 without reaching the handler.
func runPostLimitTest(
t *testing.T,
mw func(http.Handler) http.Handler,
limit int,
path, ip string,
) {
t.Helper()
var callCount int
handler := mw(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
callCount++
w.WriteHeader(http.StatusOK)
},
))
// The first limit POST requests should succeed
for i := range limit {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, path, nil,
)
req.RemoteAddr = ip
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(
t, http.StatusOK, w.Code,
"POST request %d should pass", i,
)
}
// Next POST should be rate-limited
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, path, nil,
)
req.RemoteAddr = ip
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(
t, http.StatusTooManyRequests, w.Code,
"POST after limit should be 429",
)
assert.Equal(t, limit, callCount)
}
func TestLoginRateLimit_LimitsPOST(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
runPostLimitTest(
t,
m.LoginRateLimit(),
middleware.LoginRateLimitConst,
"/pages/login",
"10.0.0.1:12345",
)
}
func TestPasswordChangeRateLimit_LimitsPOST(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
runPostLimitTest(
t,
m.PasswordChangeRateLimit(),
middleware.PasswordChangeRateLimitConst,
"/user/admin/password",
"10.0.0.2:12345",
)
}
func TestLoginRateLimit_IndependentPerIP(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.LoginRateLimit()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
))
// Exhaust limit for IP1
for range middleware.LoginRateLimitConst {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login", nil,
)
req.RemoteAddr = "1.2.3.4:12345"
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
}
// IP1 should be rate-limited
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login", nil,
)
req.RemoteAddr = "1.2.3.4:12345"
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(t, http.StatusTooManyRequests, w.Code)
// IP2 should still be allowed
req2 := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login", nil,
)
req2.RemoteAddr = "5.6.7.8:12345"
w2 := httptest.NewRecorder()
handler.ServeHTTP(w2, req2)
assert.Equal(
t, http.StatusOK, w2.Code,
"different IP should not be affected",
)
}
// okHandler is the terminal handler the limiter middleware wraps
// in these tests: it answers 200 to anything that reaches it.
func okHandler() http.Handler {
return http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
)
}
// rateLimitMiddleware builds a Middleware around cfg, whose
// TrustedProxies field is what the rate limit key function gates
// forwarded-header trust on.
func rateLimitMiddleware(
t *testing.T, cfg *config.Config,
) *middleware.Middleware {
t.Helper()
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
return middleware.NewForTest(log, cfg, nil)
}
// trustedProxies parses CIDR strings for a test Config.
func trustedProxies(cidrs ...string) []netip.Prefix {
prefixes := make([]netip.Prefix, 0, len(cidrs))
for _, cidr := range cidrs {
prefixes = append(prefixes, netip.MustParsePrefix(cidr))
}
return prefixes
}
// postWithHeaders sends one POST to the handler from peer with the
// given headers set and returns the recorder.
func postWithHeaders(
handler http.Handler,
peer, path string,
headers map[string]string,
) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, path, nil,
)
req.RemoteAddr = peer
for name, value := range headers {
req.Header.Set(name, value)
}
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
return w
}
// receiverLimitedHandler builds a ReceiverRateLimit-wrapped
// handler with the given per-minute limit and no trusted proxies.
func receiverLimitedHandler(
t *testing.T, limit int,
) http.Handler {
t.Helper()
m := rateLimitMiddleware(
t, &config.Config{ReceiverRateLimit: limit},
)
return m.ReceiverRateLimit()(okHandler())
}
// receiverPost sends one POST to the handler from the given IP
// and path and returns the recorder.
func receiverPost(
handler http.Handler, ip, path string,
) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, path, nil,
)
req.RemoteAddr = ip
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
return w
}
func TestReceiverRateLimit_LimitsPerIPAndPath(t *testing.T) {
t.Parallel()
const limit = 3
handler := receiverLimitedHandler(t, limit)
// The first limit requests from one IP to one entrypoint
// pass.
for i := range limit {
w := receiverPost(
handler, "9.9.9.9:1234", "/webhook/uuid-a",
)
assert.Equal(
t, http.StatusOK, w.Code,
"request %d should pass", i,
)
}
// The next request over the limit is rejected with a 429
// carrying a Retry-After header.
w := receiverPost(
handler, "9.9.9.9:1234", "/webhook/uuid-a",
)
assert.Equal(t, http.StatusTooManyRequests, w.Code)
assert.NotEmpty(
t, w.Header().Get("Retry-After"),
"429 must carry a Retry-After header",
)
// The same IP is not limited on a different entrypoint.
w = receiverPost(
handler, "9.9.9.9:1234", "/webhook/uuid-b",
)
assert.Equal(
t, http.StatusOK, w.Code,
"a different entrypoint must not be affected",
)
// A different IP is not limited on the same entrypoint.
w = receiverPost(
handler, "8.8.8.8:1234", "/webhook/uuid-a",
)
assert.Equal(
t, http.StatusOK, w.Code,
"a different client IP must not be affected",
)
}
// TestReceiverRateLimit_CountsEveryMethod proves the receiver
// limit counts non-POST requests too: a GET shares the bucket
// with a POST and is itself rejected once over the limit.
func TestReceiverRateLimit_CountsEveryMethod(t *testing.T) {
t.Parallel()
const (
limit = 2
ip = "7.7.7.7:1234"
path = "/webhook/uuid-c"
)
handler := receiverLimitedHandler(t, limit)
get := func() *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, path, nil,
)
req.RemoteAddr = ip
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
return w
}
// One POST plus one GET fill the bucket, so the GET must
// have been counted.
assert.Equal(
t, http.StatusOK, receiverPost(handler, ip, path).Code,
)
assert.Equal(t, http.StatusOK, get().Code)
assert.Equal(
t, http.StatusTooManyRequests, get().Code,
"a GET over the limit must be rate-limited",
)
}
const (
loginPath = "/pages/login"
headerXFF = "X-Forwarded-For"
headerReal = "X-Real-IP"
headerTrue = "True-Client-IP"
)
// assertSharedBucket drives the login limiter from peer with the
// trusted-proxy set proxies, sending one more request than the limit
// allows and varying the headers on each with headers(i). Every
// request must land in the same bucket, so the last one is rejected:
// if any of the varying header values reached the key, the run would
// have minted fresh buckets and nothing would be rejected.
func assertSharedBucket(
t *testing.T,
proxies []netip.Prefix,
peer string,
headers func(i int) map[string]string,
msg string,
) {
t.Helper()
m := rateLimitMiddleware(
t, &config.Config{TrustedProxies: proxies},
)
handler := m.LoginRateLimit()(okHandler())
for i := range middleware.LoginRateLimitConst {
w := postWithHeaders(handler, peer, loginPath, headers(i))
assert.Equal(
t, http.StatusOK, w.Code,
"request %d should pass", i,
)
}
w := postWithHeaders(
handler, peer, loginPath,
headers(middleware.LoginRateLimitConst),
)
assert.Equal(t, http.StatusTooManyRequests, w.Code, msg)
}
// TestRateLimitKey_SpoofedForwardedFromUntrustedPeer is the test
// this gating exists for: with no trusted proxies configured (the
// default), a client that rotates a forwarded header on every
// request must stay in one bucket. If forwarded headers were
// trusted unconditionally, each spoofed value would mint a fresh
// bucket and the limit would stop no one.
func TestRateLimitKey_SpoofedForwardedFromUntrustedPeer(
t *testing.T,
) {
t.Parallel()
for _, header := range []string{
headerXFF, headerReal, headerTrue,
} {
t.Run(header, func(t *testing.T) {
t.Parallel()
assertSharedBucket(
t, nil, "203.0.113.9:44444",
func(i int) map[string]string {
return map[string]string{
header: fmt.Sprintf(
"198.51.100.%d", i+1,
),
}
},
"a spoofed "+header+" from an untrusted peer "+
"must not mint a fresh bucket",
)
})
}
}
// TestRateLimitKey_SingleValuedHeadersIgnoredFromTrustedPeer is the
// regression test for the bypass hiding inside the trusted case.
// Real reverse proxies (nginx, HAProxy, Caddy, ALB) set only
// X-Forwarded-For and pass every other client header through
// verbatim, so a client behind the configured proxy can send its own
// X-Real-IP or True-Client-IP. Reading either would hand that client
// a fresh bucket per request from inside exactly the deployment
// TRUSTED_PROXIES exists to serve, so neither header is read at all.
func TestRateLimitKey_SingleValuedHeadersIgnoredFromTrustedPeer(
t *testing.T,
) {
t.Parallel()
for _, header := range []string{headerReal, headerTrue} {
t.Run(header, func(t *testing.T) {
t.Parallel()
assertSharedBucket(
t, trustedProxies("10.0.0.0/8"),
"10.0.0.1:44444",
func(i int) map[string]string {
return map[string]string{
header: fmt.Sprintf(
"198.51.100.%d", i+1,
),
}
},
header+" from a trusted peer must not mint a "+
"fresh bucket: only X-Forwarded-For is read",
)
})
}
}
// TestRateLimitKey_MalformedRightmostHopFallsBackToPeer covers the
// other end of the chain walk. The rightmost X-Forwarded-For entry
// is the one the trusted proxy appended; if it cannot be read as an
// address the chain is not the shape the walk assumes, and every
// entry to its left may have come from the client. The walk must
// stop and fall back to the peer rather than select one of them.
func TestRateLimitKey_MalformedRightmostHopFallsBackToPeer(
t *testing.T,
) {
t.Parallel()
// Forms seen in the wild: host:port (Azure Application
// Gateway, IIS ARR), a bracketed IPv6 literal, and the
// RFC 7239 placeholder token.
for _, tail := range []string{
"198.51.100.7:1234", "[2001:db8::1]", "unknown",
} {
t.Run(tail, func(t *testing.T) {
t.Parallel()
assertSharedBucket(
t, trustedProxies("10.0.0.0/8"),
"10.0.0.1:44444",
func(i int) map[string]string {
return map[string]string{
headerXFF: fmt.Sprintf(
"9.9.9.%d, %s", i+1, tail,
),
}
},
"an unparseable rightmost hop must fall back "+
"to the peer address, not select a "+
"client-controlled entry",
)
})
}
}
// TestRateLimitKey_ForwardedHonouredFromTrustedPeer checks the
// other half: when the direct peer is a configured trusted proxy,
// the forwarded client address is what buckets are keyed on, so
// one sender behind the proxy cannot exhaust another's limit.
func TestRateLimitKey_ForwardedHonouredFromTrustedPeer(
t *testing.T,
) {
t.Parallel()
m := rateLimitMiddleware(t, &config.Config{
TrustedProxies: trustedProxies("10.0.0.0/8"),
})
handler := m.LoginRateLimit()(okHandler())
const peer = "10.0.0.1:44444"
first := map[string]string{headerXFF: "198.51.100.7"}
for range middleware.LoginRateLimitConst {
postWithHeaders(handler, peer, loginPath, first)
}
w := postWithHeaders(handler, peer, loginPath, first)
assert.Equal(
t, http.StatusTooManyRequests, w.Code,
"the forwarded client's own bucket must fill up",
)
w = postWithHeaders(
handler, peer, loginPath,
map[string]string{headerXFF: "198.51.100.8"},
)
assert.Equal(
t, http.StatusOK, w.Code,
"a forwarded header from a trusted peer must be honoured",
)
}
// TestRateLimitKey_ChainWalkSkipsClientPrepended covers the
// residual spoofing route behind a trusted proxy: the client
// controls the leftmost X-Forwarded-For entries, so the key is the
// rightmost hop that is not itself trusted. Rotating the prepended
// entry must not create new buckets.
func TestRateLimitKey_ChainWalkSkipsClientPrepended(t *testing.T) {
t.Parallel()
assertSharedBucket(
t, trustedProxies("10.0.0.0/8"), "10.0.0.1:44444",
func(i int) map[string]string {
return map[string]string{
headerXFF: fmt.Sprintf(
"9.9.9.%d, 198.51.100.7, 10.0.0.2", i+1,
),
}
},
"a client-prepended X-Forwarded-For entry must not "+
"mint a fresh bucket",
)
}
// TestReceiverRateLimit_IgnoresForwardedFromUntrustedPeer proves
// the receiver limiter uses the same gated key function as the
// POST limiters.
func TestReceiverRateLimit_IgnoresForwardedFromUntrustedPeer(
t *testing.T,
) {
t.Parallel()
const (
limit = 3
peer = "203.0.113.10:44444"
path = "/webhook/uuid-d"
)
handler := receiverLimitedHandler(t, limit)
for i := range limit {
w := postWithHeaders(
handler, peer, path,
map[string]string{
headerXFF: fmt.Sprintf(
"198.51.100.%d", i+1,
),
},
)
assert.Equal(
t, http.StatusOK, w.Code,
"request %d should pass", i,
)
}
w := postWithHeaders(
handler, peer, path,
map[string]string{headerXFF: "198.51.100.200"},
)
assert.Equal(
t, http.StatusTooManyRequests, w.Code,
"a spoofed X-Forwarded-For from an untrusted peer must "+
"not mint a fresh receiver bucket",
)
}

View File

@@ -0,0 +1,24 @@
package middleware
import (
"log/slog"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/session"
)
// NewForTest creates a Middleware with the minimum dependencies
// needed for testing. This bypasses the fx lifecycle.
func NewForTest(
log *slog.Logger,
cfg *config.Config,
sess *session.Session,
) *Middleware {
return &Middleware{
log: log,
params: &MiddlewareParams{
Config: cfg,
},
session: sess,
}
}

View File

@@ -0,0 +1,36 @@
package server
import (
"log/slog"
"net/http"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/middleware"
)
// MaxFormBodySizeForTest exposes the form body cap so tests can
// build requests that sit exactly at, below, and above it.
const MaxFormBodySizeForTest = maxFormBodySize
// NewRouterForTest builds the real route tree via SetupRoutes with
// the supplied middleware and handlers, bypassing the fx lifecycle
// and the HTTP listener. Tests use it so that route-group middleware
// registration order is exercised exactly as it ships, rather than
// against a hand-rebuilt chain that could drift from routes.go.
func NewRouterForTest(
log *slog.Logger,
cfg *config.Config,
mw *middleware.Middleware,
h *handlers.Handlers,
) http.Handler {
s := &Server{
log: log,
mw: mw,
h: h,
params: ServerParams{Config: cfg},
}
s.SetupRoutes()
return s.router
}

View File

@@ -1,18 +1,36 @@
package server package server
import ( import (
"errors"
"fmt" "fmt"
"net/http" "net/http"
"time" "time"
) )
const (
// httpReadTimeout is the maximum duration for reading the
// entire request, including the body.
httpReadTimeout = 10 * time.Second
// httpWriteTimeout is the maximum duration before timing out
// writes of the response. It must stay above the router's
// requestTimeout (60s, in routes.go) so the middleware timeout
// fires first and returns a clean 503, rather than the transport
// cutting the connection at the socket write deadline.
httpWriteTimeout = 65 * time.Second
// httpMaxHeaderBytes is the maximum number of bytes the
// server will read parsing the request headers.
httpMaxHeaderBytes = 1 << 20
)
func (s *Server) serveUntilShutdown() { func (s *Server) serveUntilShutdown() {
listenAddr := fmt.Sprintf(":%d", s.params.Config.Port) listenAddr := fmt.Sprintf(":%d", s.params.Config.Port)
s.httpServer = &http.Server{ s.httpServer = &http.Server{
Addr: listenAddr, Addr: listenAddr,
ReadTimeout: 10 * time.Second, ReadTimeout: httpReadTimeout,
WriteTimeout: 10 * time.Second, WriteTimeout: httpWriteTimeout,
MaxHeaderBytes: 1 << 20, MaxHeaderBytes: httpMaxHeaderBytes,
Handler: s, Handler: s,
} }
@@ -21,14 +39,21 @@ func (s *Server) serveUntilShutdown() {
s.SetupRoutes() s.SetupRoutes()
s.log.Info("http begin listen", "listenaddr", listenAddr) s.log.Info("http begin listen", "listenaddr", listenAddr)
if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
err := s.httpServer.ListenAndServe()
if err != nil && !errors.Is(err, http.ErrServerClosed) {
s.log.Error("listen error", "error", err) s.log.Error("listen error", "error", err)
if s.cancelFunc != nil { if s.cancelFunc != nil {
s.cancelFunc() s.cancelFunc()
} }
} }
} }
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { // ServeHTTP delegates to the router.
func (s *Server) ServeHTTP(
w http.ResponseWriter,
r *http.Request,
) {
s.router.ServeHTTP(w, r) s.router.ServeHTTP(w, r)
} }

View File

@@ -11,55 +11,57 @@ import (
"sneak.berlin/go/webhooker/static" "sneak.berlin/go/webhooker/static"
) )
// maxFormBodySize is the maximum allowed request body size (in
// bytes) for form POST endpoints. 1 MB is generous for any form
// submission while preventing abuse from oversized payloads.
const maxFormBodySize int64 = 1 * 1024 * 1024 // 1 MB
// requestTimeout is the maximum time allowed for a single HTTP
// request.
const requestTimeout = 60 * time.Second
// SetupRoutes configures all HTTP routes and middleware on the
// server's router.
func (s *Server) SetupRoutes() { func (s *Server) SetupRoutes() {
s.router = chi.NewRouter() s.router = chi.NewRouter()
s.setupGlobalMiddleware()
s.setupRoutes()
}
// the mux .Use() takes a http.Handler wrapper func, like most func (s *Server) setupGlobalMiddleware() {
// things that deal with "middlewares" like alice et c, and will
// call ServeHTTP on it. These middlewares applied by the mux (you
// can .Use() more than one) will be applied to every request into
// the service.
s.router.Use(middleware.Recoverer) s.router.Use(middleware.Recoverer)
s.router.Use(middleware.RequestID) s.router.Use(middleware.RequestID)
s.router.Use(s.mw.SecurityHeaders())
s.router.Use(s.mw.Logging()) s.router.Use(s.mw.Logging())
// add metrics middleware only if we can serve them behind auth // Metrics middleware (only if credentials are configured)
if s.params.Config.MetricsUsername != "" { if s.params.Config.MetricsUsername != "" {
s.router.Use(s.mw.Metrics()) s.router.Use(s.mw.Metrics())
} }
// set up CORS headers
s.router.Use(s.mw.CORS()) s.router.Use(s.mw.CORS())
s.router.Use(middleware.Timeout(requestTimeout))
// timeout for request context; your handlers must finish within // Sentry error reporting (if SENTRY_DSN is set). Repanic is
// this window: // true so panics still bubble up to the Recoverer middleware.
s.router.Use(middleware.Timeout(60 * time.Second))
// this adds a sentry reporting middleware if and only if sentry is
// enabled via setting of SENTRY_DSN in env.
if s.sentryEnabled { if s.sentryEnabled {
// Options docs at
// https://docs.sentry.io/platforms/go/guides/http/
// we set sentry to repanic so that all panics bubble up to the
// Recoverer chi middleware above.
sentryHandler := sentryhttp.New(sentryhttp.Options{ sentryHandler := sentryhttp.New(sentryhttp.Options{
Repanic: true, Repanic: true,
}) })
s.router.Use(sentryHandler.Handle) s.router.Use(sentryHandler.Handle)
} }
}
//////////////////////////////////////////////////////////////////////// func (s *Server) setupRoutes() {
// ROUTES
// complete docs: https://github.com/go-chi/chi
////////////////////////////////////////////////////////////////////////
s.router.Get("/", s.h.HandleIndex()) s.router.Get("/", s.h.HandleIndex())
s.router.Mount("/s", http.StripPrefix("/s", http.FileServer(http.FS(static.Static)))) s.router.Mount(
"/s",
http.StripPrefix("/s", http.FileServer(http.FS(static.Static))),
)
s.router.Route("/api/v1", func(_ chi.Router) { s.router.Route("/api/v1", func(_ chi.Router) {
// TODO: Add API routes here // API routes will be added here.
}) })
s.router.Get( s.router.Get(
@@ -71,42 +73,106 @@ func (s *Server) SetupRoutes() {
if s.params.Config.MetricsUsername != "" { if s.params.Config.MetricsUsername != "" {
s.router.Group(func(r chi.Router) { s.router.Group(func(r chi.Router) {
r.Use(s.mw.MetricsAuth()) r.Use(s.mw.MetricsAuth())
r.Get("/metrics", http.HandlerFunc(promhttp.Handler().ServeHTTP)) r.Get(
"/metrics",
http.HandlerFunc(
promhttp.Handler().ServeHTTP,
),
)
}) })
} }
// pages that are rendered server-side s.setupPageRoutes()
s.setupUserRoutes()
s.setupSourceRoutes()
s.setupWebhookRoutes()
}
func (s *Server) setupPageRoutes() {
s.router.Route("/pages", func(r chi.Router) { s.router.Route("/pages", func(r chi.Router) {
// Login page (no auth required) // MaxBodySize must precede CSRF: gorilla/csrf parses the
// form, so the cap has to be installed before it runs.
r.Use(s.mw.MaxBodySize(maxFormBodySize))
r.Use(s.mw.CSRF())
r.Use(s.mw.NoCache())
r.Group(func(r chi.Router) {
r.Use(s.mw.LoginRateLimit())
r.Get("/login", s.h.HandleLoginPage()) r.Get("/login", s.h.HandleLoginPage())
r.Post("/login", s.h.HandleLoginSubmit()) r.Post("/login", s.h.HandleLoginSubmit())
})
// Logout (auth required)
r.Post("/logout", s.h.HandleLogout()) r.Post("/logout", s.h.HandleLogout())
}) })
}
// User profile routes func (s *Server) setupUserRoutes() {
s.router.Route("/user/{username}", func(r chi.Router) { s.router.Route("/user/{username}", func(r chi.Router) {
r.Get("/", s.h.HandleProfile()) // MaxBodySize must precede CSRF: gorilla/csrf parses the
}) // form, so the cap has to be installed before it runs.
r.Use(s.mw.MaxBodySize(maxFormBodySize))
// Webhook management routes (require authentication) r.Use(s.mw.CSRF())
s.router.Route("/sources", func(r chi.Router) { r.Use(s.mw.NoCache())
r.Use(s.mw.RequireAuth()) r.Use(s.mw.RequireAuth())
r.Get("/", s.h.HandleSourceList()) // List all webhooks r.Get("/", s.h.HandleProfile())
r.Get("/new", s.h.HandleSourceCreate()) // Show create form r.With(s.mw.PasswordChangeRateLimit()).Post(
r.Post("/new", s.h.HandleSourceCreateSubmit()) // Handle create submission "/password", s.h.HandlePasswordChange(),
)
})
}
func (s *Server) setupSourceRoutes() {
s.router.Route("/sources", func(r chi.Router) {
// MaxBodySize must precede CSRF: gorilla/csrf parses the
// form, so the cap has to be installed before it runs.
r.Use(s.mw.MaxBodySize(maxFormBodySize))
r.Use(s.mw.CSRF())
r.Use(s.mw.NoCache())
r.Use(s.mw.RequireAuth())
r.Get("/", s.h.HandleSourceList())
r.Get("/new", s.h.HandleSourceCreate())
r.Post("/new", s.h.HandleSourceCreateSubmit())
}) })
s.router.Route("/source/{sourceID}", func(r chi.Router) { s.router.Route("/source/{sourceID}", func(r chi.Router) {
// MaxBodySize must precede CSRF: gorilla/csrf parses the
// form, so the cap has to be installed before it runs.
r.Use(s.mw.MaxBodySize(maxFormBodySize))
r.Use(s.mw.CSRF())
r.Use(s.mw.NoCache())
r.Use(s.mw.RequireAuth()) r.Use(s.mw.RequireAuth())
r.Get("/", s.h.HandleSourceDetail()) // View webhook details r.Get("/", s.h.HandleSourceDetail())
r.Get("/edit", s.h.HandleSourceEdit()) // Show edit form r.Get("/edit", s.h.HandleSourceEdit())
r.Post("/edit", s.h.HandleSourceEditSubmit()) // Handle edit submission r.Post("/edit", s.h.HandleSourceEditSubmit())
r.Post("/delete", s.h.HandleSourceDelete()) // Delete webhook r.Post("/delete", s.h.HandleSourceDelete())
r.Get("/logs", s.h.HandleSourceLogs()) // View webhook logs r.Get("/logs", s.h.HandleSourceLogs())
r.Post(
"/entrypoints",
s.h.HandleEntrypointCreate(),
)
r.Post(
"/entrypoints/{entrypointID}/delete",
s.h.HandleEntrypointDelete(),
)
r.Post(
"/entrypoints/{entrypointID}/toggle",
s.h.HandleEntrypointToggle(),
)
r.Post("/targets", s.h.HandleTargetCreate())
r.Post(
"/targets/{targetID}/delete",
s.h.HandleTargetDelete(),
)
r.Post(
"/targets/{targetID}/toggle",
s.h.HandleTargetToggle(),
)
}) })
}
// Entrypoint endpoint - accepts incoming webhook POST requests
s.router.HandleFunc("/webhook/{uuid}", s.h.HandleWebhook()) func (s *Server) setupWebhookRoutes() {
s.router.With(s.mw.ReceiverRateLimit()).HandleFunc(
"/webhook/{uuid}",
s.h.HandleWebhook(),
)
} }

View File

@@ -0,0 +1,383 @@
package server_test
import (
"context"
"html"
"net/http"
"net/http/httptest"
"net/url"
"regexp"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/healthcheck"
"sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/server"
"sneak.berlin/go/webhooker/internal/session"
)
// csrfCookieName is the cookie gorilla/csrf issues when it runs. Its
// presence or absence on a response is how these tests tell whether
// the CSRF middleware executed.
const csrfCookieName = "_gorilla_csrf"
type noopNotifier struct{}
func (n *noopNotifier) Notify([]delivery.Task) {}
// noopEvictor satisfies handlers.New's delivery.WebhookEvictor
// dependency. These tests never delete a webhook, so there is
// nothing to record.
type noopEvictor struct{}
func (e *noopEvictor) EvictWebhook(string) {}
// testEnv is the real router from routes.go plus the collaborators
// tests need to seed users and forge sessions.
type testEnv struct {
router http.Handler
sess *session.Session
db *database.Database
}
// newTestEnv wires the dependency graph with fx and builds the
// production route tree, so middleware registration order is
// exercised exactly as it ships.
func newTestEnv(t *testing.T) *testEnv {
t.Helper()
var (
log *logger.Logger
cfg *config.Config
mw *middleware.Middleware
hnd *handlers.Handlers
sess *session.Session
db *database.Database
)
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
func() *config.Config {
return &config.Config{
DataDir: t.TempDir(),
Environment: config.EnvironmentDev,
}
},
database.New,
database.NewWebhookDBManager,
healthcheck.New,
session.New,
func() delivery.Notifier { return &noopNotifier{} },
func() delivery.WebhookEvictor { return &noopEvictor{} },
middleware.New,
handlers.New,
),
fx.Populate(&log, &cfg, &mw, &hnd, &sess, &db),
)
app.RequireStart()
t.Cleanup(app.RequireStop)
return &testEnv{
router: server.NewRouterForTest(log.Get(), cfg, mw, hnd),
sess: sess,
db: db,
}
}
// oversizeValue returns a form value one byte past the route-group
// body cap, so an encoded form containing it is guaranteed oversize.
func oversizeValue() string {
return strings.Repeat("a", int(server.MaxFormBodySizeForTest)+1)
}
// csrfCookieSet reports whether the response issued a gorilla/csrf
// cookie, which only happens if the CSRF middleware ran.
func csrfCookieSet(w *httptest.ResponseRecorder) bool {
for _, c := range w.Result().Cookies() {
if c.Name == csrfCookieName {
return true
}
}
return false
}
// get issues a GET through the router with the supplied cookies.
func (e *testEnv) get(
path string,
cookies []*http.Cookie,
) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, path, nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
w := httptest.NewRecorder()
e.router.ServeHTTP(w, req)
return w
}
// post issues a urlencoded form POST through the router. The body is
// a strings.Reader, so the request carries an accurate
// Content-Length — the signal MaxBodySize checks up front.
func (e *testEnv) post(
path string,
form url.Values,
cookies []*http.Cookie,
) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, path,
strings.NewReader(form.Encode()),
)
req.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
req.AddCookie(c)
}
w := httptest.NewRecorder()
e.router.ServeHTTP(w, req)
return w
}
// csrfFrom renders the page at path and returns the CSRF token from
// its form together with every cookie needed for the follow-up POST.
func (e *testEnv) csrfFrom(
t *testing.T,
path string,
cookies []*http.Cookie,
) (string, []*http.Cookie) {
t.Helper()
w := e.get(path, cookies)
require.Equal(t, http.StatusOK, w.Code)
pattern := regexp.MustCompile(
`name="csrf_token" value="([^"]+)"`,
)
match := pattern.FindStringSubmatch(w.Body.String())
require.Len(t, match, 2, "form must embed a CSRF token")
// html/template escapes "+" and "=" in attribute values, and
// gorilla/csrf tokens are standard base64, so the value read
// out of the markup has to be unescaped before it is submitted.
token := html.UnescapeString(match[1])
combined := make([]*http.Cookie, 0, len(cookies))
combined = append(combined, cookies...)
combined = append(combined, w.Result().Cookies()...)
return token, combined
}
// authCookies forges an authenticated session for the given user.
func (e *testEnv) authCookies(
t *testing.T,
userID, username string,
) []*http.Cookie {
t.Helper()
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/setup", nil,
)
w := httptest.NewRecorder()
s, err := e.sess.Get(req)
require.NoError(t, err)
e.sess.SetUser(s, userID, username)
require.NoError(t, e.sess.Save(req, w, s))
cookies := w.Result().Cookies()
require.NotEmpty(t, cookies, "session cookie should be set")
return cookies
}
// seedUser creates a user with the given password and returns the
// stored hash so tests can assert whether it later changed.
func (e *testEnv) seedUser(
t *testing.T,
username, password string,
) (string, string) {
t.Helper()
hash, err := database.HashPassword(password)
require.NoError(t, err)
user := &database.User{Username: username, Password: hash}
require.NoError(t, e.db.DB().Create(user).Error)
return user.ID, hash
}
// storedHash reads the current password hash for a username.
func (e *testEnv) storedHash(t *testing.T, username string) string {
t.Helper()
var user database.User
require.NoError(t,
e.db.DB().Where("username = ?", username).
First(&user).Error,
)
return user.Password
}
// --- /pages group ---
// TestPagesLogin_OversizeBody_RejectedBeforeCSRF proves the cap runs
// ahead of gorilla/csrf: the response is a clean 413 and no CSRF
// cookie was issued, so neither the CSRF middleware nor the login
// handler ran.
func TestPagesLogin_OversizeBody_RejectedBeforeCSRF(t *testing.T) {
t.Parallel()
env := newTestEnv(t)
form := url.Values{}
form.Set("username", oversizeValue())
form.Set("password", "irrelevant")
w := env.post("/pages/login", form, nil)
assert.Equal(
t, http.StatusRequestEntityTooLarge, w.Code,
)
assert.False(
t, csrfCookieSet(w),
"CSRF middleware must not run for an oversized body",
)
}
// TestPagesLogin_UnderLimit_NoToken_CSRFRejects is the control for
// the test above: an identically shaped but under-limit POST does
// reach gorilla/csrf, which rejects it and issues its cookie. Without
// this, the missing-cookie assertion above would prove nothing.
func TestPagesLogin_UnderLimit_NoToken_CSRFRejects(t *testing.T) {
t.Parallel()
env := newTestEnv(t)
form := url.Values{}
form.Set("username", "someone")
form.Set("password", "irrelevant")
w := env.post("/pages/login", form, nil)
assert.Equal(t, http.StatusForbidden, w.Code)
assert.True(
t, csrfCookieSet(w),
"CSRF middleware should run for an under-limit body",
)
}
// TestPagesLogin_UnderLimit_ValidToken_ReachesHandler proves the
// reorder did not break CSRF token handling: a token harvested from
// the rendered login form is still accepted and the request lands in
// the handler.
func TestPagesLogin_UnderLimit_ValidToken_ReachesHandler(
t *testing.T,
) {
t.Parallel()
env := newTestEnv(t)
token, cookies := env.csrfFrom(t, "/pages/login", nil)
form := url.Values{}
form.Set("csrf_token", token)
form.Set("username", "nosuchuser")
form.Set("password", "wrongpassword")
w := env.post("/pages/login", form, cookies)
assert.Equal(t, http.StatusUnauthorized, w.Code)
assert.Contains(
t, w.Body.String(), "Invalid username or password",
"request should reach the login handler",
)
}
// --- /user/{username} group ---
// TestPasswordChange_OversizeBody_RejectedAndPasswordUnchanged
// covers the route that previously had no middleware body cap at
// all. The request carries a valid session and a valid CSRF token,
// so the only thing that can stop it is the size cap; the unchanged
// password hash is the observable proof the handler never ran.
func TestPasswordChange_OversizeBody_RejectedAndPasswordUnchanged(
t *testing.T,
) {
t.Parallel()
env := newTestEnv(t)
userID, originalHash := env.seedUser(t, "pwuser", "oldpassword")
cookies := env.authCookies(t, userID, "pwuser")
token, cookies := env.csrfFrom(t, "/user/pwuser/", cookies)
form := url.Values{}
form.Set("csrf_token", token)
form.Set("current_password", "oldpassword")
form.Set("new_password", oversizeValue())
form.Set("confirm_password", oversizeValue())
w := env.post("/user/pwuser/password", form, cookies)
assert.Equal(
t, http.StatusRequestEntityTooLarge, w.Code,
)
assert.Equal(
t, originalHash, env.storedHash(t, "pwuser"),
"handler must not run, so the password must be unchanged",
)
}
// TestPasswordChange_UnderLimit_Succeeds proves that adding the cap
// to the /user/{username} group did not break the route it guards.
func TestPasswordChange_UnderLimit_Succeeds(t *testing.T) {
t.Parallel()
env := newTestEnv(t)
userID, originalHash := env.seedUser(t, "okuser", "oldpassword")
cookies := env.authCookies(t, userID, "okuser")
token, cookies := env.csrfFrom(t, "/user/okuser/", cookies)
form := url.Values{}
form.Set("csrf_token", token)
form.Set("current_password", "oldpassword")
form.Set("new_password", "brandnewpassword")
form.Set("confirm_password", "brandnewpassword")
w := env.post("/user/okuser/password", form, cookies)
assert.Equal(t, http.StatusOK, w.Code)
assert.NotEqual(
t, originalHash, env.storedHash(t, "okuser"),
"an under-limit password change should still apply",
)
}

View File

@@ -1,3 +1,5 @@
// Package server wires up HTTP routes and manages the
// application lifecycle.
package server package server
import ( import (
@@ -21,10 +23,20 @@ import (
"github.com/go-chi/chi" "github.com/go-chi/chi"
) )
// ServerParams is a standard fx naming convention for dependency injection const (
// nolint:golint // shutdownTimeout is the maximum time to wait for the HTTP
// server to finish in-flight requests during shutdown.
shutdownTimeout = 5 * time.Second
// sentryFlushTimeout is the maximum time to wait for Sentry
// to flush pending events during shutdown.
sentryFlushTimeout = 2 * time.Second
)
//nolint:revive // ServerParams is a standard fx naming convention.
type ServerParams struct { type ServerParams struct {
fx.In fx.In
Logger *logger.Logger Logger *logger.Logger
Globals *globals.Globals Globals *globals.Globals
Config *config.Config Config *config.Config
@@ -32,12 +44,13 @@ type ServerParams struct {
Handlers *handlers.Handlers Handlers *handlers.Handlers
} }
// Server is the main HTTP server that wires up routes and manages
// graceful shutdown.
type Server struct { type Server struct {
startupTime time.Time startupTime time.Time
exitCode int exitCode int
sentryEnabled bool sentryEnabled bool
log *slog.Logger log *slog.Logger
ctx context.Context
cancelFunc context.CancelFunc cancelFunc context.CancelFunc
httpServer *http.Server httpServer *http.Server
router *chi.Mux router *chi.Mux
@@ -46,6 +59,8 @@ type Server struct {
h *handlers.Handlers h *handlers.Handlers
} }
// New creates a Server that starts the HTTP listener on fx start
// and stops it gracefully.
func New(lc fx.Lifecycle, params ServerParams) (*Server, error) { func New(lc fx.Lifecycle, params ServerParams) (*Server, error) {
s := new(Server) s := new(Server)
s.params = params s.params = params
@@ -54,19 +69,23 @@ func New(lc fx.Lifecycle, params ServerParams) (*Server, error) {
s.log = params.Logger.Get() s.log = params.Logger.Get()
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error { OnStart: func(_ context.Context) error {
s.startupTime = time.Now() s.startupTime = time.Now()
go s.Run() go s.Run()
return nil return nil
}, },
OnStop: func(ctx context.Context) error { OnStop: func(ctx context.Context) error {
s.cleanShutdown() s.cleanShutdown(ctx)
return nil return nil
}, },
}) })
return s, nil return s, nil
} }
// Run configures Sentry and starts serving HTTP requests.
func (s *Server) Run() { func (s *Server) Run() {
s.configure() s.configure()
@@ -76,6 +95,12 @@ func (s *Server) Run() {
s.serve() s.serve()
} }
// MaintenanceMode returns whether the server is in maintenance
// mode.
func (s *Server) MaintenanceMode() bool {
return s.params.Config.MaintenanceMode
}
func (s *Server) enableSentry() { func (s *Server) enableSentry() {
s.sentryEnabled = false s.sentryEnabled = false
@@ -85,68 +110,78 @@ func (s *Server) enableSentry() {
err := sentry.Init(sentry.ClientOptions{ err := sentry.Init(sentry.ClientOptions{
Dsn: s.params.Config.SentryDSN, Dsn: s.params.Config.SentryDSN,
Release: fmt.Sprintf("%s-%s", s.params.Globals.Appname, s.params.Globals.Version), Release: fmt.Sprintf(
"%s-%s",
s.params.Globals.Appname,
s.params.Globals.Version,
),
}) })
if err != nil { if err != nil {
s.log.Error("sentry init failure", "error", err) s.log.Error("sentry init failure", "error", err)
// Don't use fatal since we still want the service to run // Don't use fatal since we still want the service to run
return return
} }
s.log.Info("sentry error reporting activated") s.log.Info("sentry error reporting activated")
s.sentryEnabled = true s.sentryEnabled = true
} }
func (s *Server) serve() int { func (s *Server) serve() int {
s.ctx, s.cancelFunc = context.WithCancel(context.Background()) ctx, cancelFunc := context.WithCancel(context.Background())
s.cancelFunc = cancelFunc
// signal watcher // signal watcher
go func() { go func() {
c := make(chan os.Signal, 1) c := make(chan os.Signal, 1)
signal.Ignore(syscall.SIGPIPE) signal.Ignore(syscall.SIGPIPE)
signal.Notify(c, os.Interrupt, syscall.SIGTERM) signal.Notify(c, os.Interrupt, syscall.SIGTERM)
// block and wait for signal // block and wait for signal
sig := <-c sig := <-c
s.log.Info("signal received", "signal", sig.String()) s.log.Info("signal received", "signal", sig.String())
if s.cancelFunc != nil { if s.cancelFunc != nil {
// cancelling the main context will trigger a clean // cancelling the main context will trigger a clean
// shutdown. // shutdown via the fx OnStop hook.
s.cancelFunc() s.cancelFunc()
} }
}() }()
go s.serveUntilShutdown() go s.serveUntilShutdown()
<-s.ctx.Done() <-ctx.Done()
s.cleanShutdown() // Shutdown is handled by the fx OnStop hook (cleanShutdown).
// Do not call cleanShutdown() here to avoid double invocation.
return s.exitCode return s.exitCode
} }
func (s *Server) cleanupForExit() { func (s *Server) cleanupForExit() {
s.log.Info("cleaning up") s.log.Info("cleaning up")
// TODO: close database connections, flush buffers, etc.
} }
func (s *Server) cleanShutdown() { func (s *Server) cleanShutdown(ctx context.Context) {
// initiate clean shutdown // initiate clean shutdown
s.exitCode = 0 s.exitCode = 0
ctxShutdown, shutdownCancel := context.WithTimeout(context.Background(), 5*time.Second)
ctxShutdown, shutdownCancel := context.WithTimeout(
ctx, shutdownTimeout,
)
defer shutdownCancel() defer shutdownCancel()
if err := s.httpServer.Shutdown(ctxShutdown); err != nil { err := s.httpServer.Shutdown(ctxShutdown)
s.log.Error("server clean shutdown failed", "error", err) if err != nil {
s.log.Error(
"server clean shutdown failed", "error", err,
)
} }
s.cleanupForExit() s.cleanupForExit()
if s.sentryEnabled { if s.sentryEnabled {
sentry.Flush(2 * time.Second) sentry.Flush(sentryFlushTimeout)
} }
} }
func (s *Server) MaintenanceMode() bool {
return s.params.Config.MaintenanceMode
}
func (s *Server) configure() { func (s *Server) configure() {
// identify ourselves in the logs // identify ourselves in the logs
s.params.Logger.Identify() s.params.Logger.Identify()

View File

@@ -1,59 +1,145 @@
// Package session manages HTTP session storage and authentication
// state.
package session package session
import ( import (
"context"
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"log/slog" "log/slog"
"maps"
"net/http" "net/http"
"time"
"github.com/gorilla/sessions" "github.com/gorilla/sessions"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
const ( const (
// SessionName is the name of the session cookie // SessionName is the name of the session cookie.
SessionName = "webhooker_session" SessionName = "webhooker_session"
// UserIDKey is the session key for user ID // UserIDKey is the session key for user ID.
UserIDKey = "user_id" UserIDKey = "user_id"
// UsernameKey is the session key for username // UsernameKey is the session key for username.
UsernameKey = "username" UsernameKey = "username"
// AuthenticatedKey is the session key for authentication status // AuthenticatedKey is the session key for authentication
// status.
AuthenticatedKey = "authenticated" AuthenticatedKey = "authenticated"
// CreatedAtKey is the session key holding the Unix timestamp at
// which the session was authenticated. It anchors the ABSOLUTE
// expiry clock and is written exactly once, by SetUser. Nothing
// refreshes it: an absolute deadline that moved with activity
// would not be a cap at all.
CreatedAtKey = "created_at"
// LastSeenKey is the session key holding the Unix timestamp of
// the most recent authenticated request. It anchors the IDLE
// expiry clock and is pushed forward by Touch.
LastSeenKey = "last_seen"
// sessionKeyLength is the required length in bytes for the
// session authentication key.
sessionKeyLength = 32
// sessionMaxAgeDays is the session cookie lifetime in days.
sessionMaxAgeDays = 7
// secondsPerDay is the number of seconds in a day.
secondsPerDay = 86400
// sessionAbsoluteMaxAge is the hard upper bound on how long a
// session may live, measured from CreatedAtKey. Activity never
// extends it, so even a continuously used session ends here and
// the user has to authenticate again.
sessionAbsoluteMaxAge = sessionMaxAgeDays * secondsPerDay * time.Second
// idleRefreshDivisor rate-limits idle-deadline refreshes. Touch
// only rewrites LastSeenKey once the stored value is older than
// idleTimeout/idleRefreshDivisor, so an active session is
// re-saved at most this many times per idle window instead of
// once per request. See Touch for the tradeoff this buys.
idleRefreshDivisor = 10
) )
// nolint:revive // SessionParams is a standard fx naming convention // ErrSessionKeyLength is returned when the decoded session key
type SessionParams struct { // does not have the expected length.
var ErrSessionKeyLength = errors.New("session key length mismatch")
// Params holds dependencies injected by fx.
type Params struct {
fx.In fx.In
Config *config.Config Config *config.Config
Database *database.Database
Logger *logger.Logger Logger *logger.Logger
} }
// Session manages encrypted session storage // Session manages encrypted session storage.
type Session struct { type Session struct {
store *sessions.CookieStore store *sessions.CookieStore
key []byte // raw 32-byte auth key, also used for CSRF cookie signing
log *slog.Logger log *slog.Logger
config *config.Config config *config.Config
// idleTimeout is the sliding inactivity window. A session that
// sees no authenticated request within this window expires,
// independently of the absolute cap. Non-positive disables idle
// expiry and leaves sessionAbsoluteMaxAge as the only bound.
idleTimeout time.Duration
// now reads the current time. Injected so expiry can be tested
// without sleeping.
now func() time.Time
} }
// New creates a new session manager // New creates a new session manager. The cookie store is
func New(lc fx.Lifecycle, params SessionParams) (*Session, error) { // initialized during the fx OnStart phase after the database is
if params.Config.SessionKey == "" { // connected, using a session key that is auto-generated and stored
return nil, fmt.Errorf("SESSION_KEY environment variable is required") // in the database.
func New(
lc fx.Lifecycle,
params Params,
) (*Session, error) {
s := &Session{
log: params.Logger.Get(),
config: params.Config,
idleTimeout: params.Config.SessionIdleTimeout,
now: time.Now,
} }
// Decode the base64 session key lc.Append(fx.Hook{
keyBytes, err := base64.StdEncoding.DecodeString(params.Config.SessionKey) OnStart: func(_ context.Context) error {
sessionKey, err := params.Database.GetOrCreateSessionKey()
if err != nil { if err != nil {
return nil, fmt.Errorf("invalid SESSION_KEY format: %w", err) return fmt.Errorf(
"failed to get session key: %w", err,
)
} }
if len(keyBytes) != 32 { keyBytes, err := base64.StdEncoding.DecodeString(
return nil, fmt.Errorf("SESSION_KEY must be 32 bytes (got %d)", len(keyBytes)) sessionKey,
)
if err != nil {
return fmt.Errorf(
"invalid session key format: %w", err,
)
}
if len(keyBytes) != sessionKeyLength {
return fmt.Errorf(
"%w: want %d, got %d",
ErrSessionKeyLength,
sessionKeyLength,
len(keyBytes),
)
} }
store := sessions.NewCookieStore(keyBytes) store := sessions.NewCookieStore(keyBytes)
@@ -61,65 +147,254 @@ func New(lc fx.Lifecycle, params SessionParams) (*Session, error) {
// Configure cookie options for security // Configure cookie options for security
store.Options = &sessions.Options{ store.Options = &sessions.Options{
Path: "/", Path: "/",
MaxAge: 86400 * 7, // 7 days MaxAge: secondsPerDay * sessionMaxAgeDays,
HttpOnly: true, HttpOnly: true,
Secure: !params.Config.IsDev(), // HTTPS in production Secure: !params.Config.IsDev(),
SameSite: http.SameSiteLaxMode, SameSite: http.SameSiteLaxMode,
} }
s := &Session{ s.key = keyBytes
store: store, s.store = store
log: params.Logger.Get(), s.log.Info("session manager initialized")
config: params.Config,
} return nil
},
})
return s, nil return s, nil
} }
// Get retrieves a session for the request // Get retrieves a session for the request.
func (s *Session) Get(r *http.Request) (*sessions.Session, error) { func (s *Session) Get(
r *http.Request,
) (*sessions.Session, error) {
return s.store.Get(r, SessionName) return s.store.Get(r, SessionName)
} }
// Save saves the session // GetKey returns the raw 32-byte authentication key used for
func (s *Session) Save(r *http.Request, w http.ResponseWriter, sess *sessions.Session) error { // session encryption. This key is also suitable for CSRF cookie
// signing.
func (s *Session) GetKey() []byte {
return s.key
}
// Save saves the session.
func (s *Session) Save(
r *http.Request,
w http.ResponseWriter,
sess *sessions.Session,
) error {
return sess.Save(r, w) return sess.Save(r, w)
} }
// SetUser sets the user information in the session // SetUser sets the user information in the session. It starts both
func (s *Session) SetUser(sess *sessions.Session, userID, username string) { // expiry clocks: CreatedAtKey (absolute, never refreshed again) and
// LastSeenKey (idle, refreshed by Touch).
func (s *Session) SetUser(
sess *sessions.Session,
userID, username string,
) {
now := s.now().Unix()
sess.Values[UserIDKey] = userID sess.Values[UserIDKey] = userID
sess.Values[UsernameKey] = username sess.Values[UsernameKey] = username
sess.Values[AuthenticatedKey] = true sess.Values[AuthenticatedKey] = true
sess.Values[CreatedAtKey] = now
sess.Values[LastSeenKey] = now
} }
// ClearUser removes user information from the session // ClearUser removes user information from the session, including
// both expiry timestamps.
func (s *Session) ClearUser(sess *sessions.Session) { func (s *Session) ClearUser(sess *sessions.Session) {
delete(sess.Values, UserIDKey) delete(sess.Values, UserIDKey)
delete(sess.Values, UsernameKey) delete(sess.Values, UsernameKey)
delete(sess.Values, AuthenticatedKey) delete(sess.Values, AuthenticatedKey)
delete(sess.Values, CreatedAtKey)
delete(sess.Values, LastSeenKey)
}
// sessionTime reads a Unix-second timestamp stored under key.
func sessionTime(
sess *sessions.Session,
key string,
) (time.Time, bool) {
secs, ok := sess.Values[key].(int64)
if !ok {
return time.Time{}, false
}
return time.Unix(secs, 0), true
} }
// IsAuthenticated checks if the session has an authenticated user // IsAuthenticated checks if the session has an authenticated user
// whose session has not passed either expiry deadline. Every
// authentication decision goes through here, so neither clock can
// be bypassed by a caller that forgets to check it.
func (s *Session) IsAuthenticated(sess *sessions.Session) bool { func (s *Session) IsAuthenticated(sess *sessions.Session) bool {
auth, ok := sess.Values[AuthenticatedKey].(bool) auth, ok := sess.Values[AuthenticatedKey].(bool)
return ok && auth if !ok || !auth {
return false
}
return !s.expired(sess)
} }
// GetUserID retrieves the user ID from the session // Touch records authenticated activity by pushing the IDLE deadline
func (s *Session) GetUserID(sess *sessions.Session) (string, bool) { // forward. It writes LastSeenKey only; CreatedAtKey is left alone so
// the absolute cap keeps counting down even for a user who never
// stops clicking.
//
// Callers must only invoke Touch for a request that authenticated
// with this session. Refreshing on an unauthenticated request would
// let anyone holding a stolen or abandoned cookie keep the session
// alive by polling a public endpoint. Touch enforces that itself by
// returning false for any session that is not currently
// authenticated and unexpired.
//
// To avoid re-encrypting and re-emitting the session cookie on every
// single request, the timestamp is advanced only once it is older
// than idleTimeout/idleRefreshDivisor. The tradeoff is that
// LastSeenKey lags real activity by up to that much, so a session
// can expire slightly early relative to the user's true last
// request -- never late.
//
// Touch reports whether it changed the session; only then does the
// caller need to save it.
func (s *Session) Touch(sess *sessions.Session) bool {
if s.idleTimeout <= 0 {
return false
}
if !s.IsAuthenticated(sess) {
return false
}
now := s.now()
lastSeen, ok := sessionTime(sess, LastSeenKey)
if ok && now.Sub(lastSeen) < s.idleTimeout/idleRefreshDivisor {
return false
}
sess.Values[LastSeenKey] = now.Unix()
return true
}
// GetUserID retrieves the user ID from the session.
func (s *Session) GetUserID(
sess *sessions.Session,
) (string, bool) {
userID, ok := sess.Values[UserIDKey].(string) userID, ok := sess.Values[UserIDKey].(string)
return userID, ok return userID, ok
} }
// GetUsername retrieves the username from the session // GetUsername retrieves the username from the session.
func (s *Session) GetUsername(sess *sessions.Session) (string, bool) { func (s *Session) GetUsername(
sess *sessions.Session,
) (string, bool) {
username, ok := sess.Values[UsernameKey].(string) username, ok := sess.Values[UsernameKey].(string)
return username, ok return username, ok
} }
// Destroy invalidates the session // Destroy invalidates the session.
func (s *Session) Destroy(sess *sessions.Session) { func (s *Session) Destroy(sess *sessions.Session) {
sess.Options.MaxAge = -1 sess.Options.MaxAge = -1
s.ClearUser(sess) s.ClearUser(sess)
} }
// Regenerate creates a new session with the same values but a
// fresh ID. The old session is destroyed (MaxAge = -1) and saved,
// then a new session is created. This prevents session fixation
// attacks by ensuring the session ID changes after privilege
// escalation (e.g. login).
func (s *Session) Regenerate(
r *http.Request,
w http.ResponseWriter,
oldSess *sessions.Session,
) (*sessions.Session, error) {
// Copy the values from the old session
oldValues := make(map[any]any)
maps.Copy(oldValues, oldSess.Values)
// Destroy the old session
oldSess.Options.MaxAge = -1
s.ClearUser(oldSess)
err := oldSess.Save(r, w)
if err != nil {
return nil, fmt.Errorf(
"failed to destroy old session: %w", err,
)
}
// Create a new session (gorilla/sessions generates a new ID)
newSess, err := s.store.New(r, SessionName)
if err != nil {
// store.New may return an error alongside a new empty
// session if the old cookie is now invalid. That is
// expected after we destroyed it above. Only fail on a
// nil session.
if newSess == nil {
return nil, fmt.Errorf(
"failed to create new session: %w", err,
)
}
}
// Restore the copied values into the new session
maps.Copy(newSess.Values, oldValues)
// Apply the standard session options (the destroyed old
// session had MaxAge = -1, which store.New might inherit
// from the cookie).
newSess.Options = &sessions.Options{
Path: "/",
MaxAge: secondsPerDay * sessionMaxAgeDays,
HttpOnly: true,
Secure: !s.config.IsDev(),
SameSite: http.SameSiteLaxMode,
}
return newSess, nil
}
// expired reports whether the session has passed either of its two
// independent deadlines. They are deliberately kept apart:
//
// - the ABSOLUTE deadline is CreatedAtKey + sessionAbsoluteMaxAge.
// It is fixed at login and no amount of activity moves it.
// - the IDLE deadline is LastSeenKey + idleTimeout. Activity moves
// it forward via Touch.
//
// Whichever comes first ends the session.
//
// A session that claims to be authenticated but carries no
// timestamps predates this check; it is treated as expired so the
// user re-authenticates rather than being granted an unbounded
// session.
func (s *Session) expired(sess *sessions.Session) bool {
now := s.now()
createdAt, ok := sessionTime(sess, CreatedAtKey)
if !ok {
return true
}
if !now.Before(createdAt.Add(sessionAbsoluteMaxAge)) {
return true
}
if s.idleTimeout <= 0 {
return false
}
lastSeen, ok := sessionTime(sess, LastSeenKey)
if !ok {
return true
}
return !now.Before(lastSeen.Add(s.idleTimeout))
}

View File

@@ -0,0 +1,853 @@
package session_test
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"testing"
"time"
"github.com/gorilla/sessions"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/session"
)
const testKeySize = 32
// testIdleTimeout is the idle window used by the expiry tests.
const testIdleTimeout = time.Hour
// testAbsoluteMaxAge restates the documented absolute session cap
// independently of the implementation constant.
const testAbsoluteMaxAge = 7 * 24 * time.Hour
// fakeClock is a manually advanced clock, so expiry can be tested
// without sleeping.
type fakeClock struct {
t time.Time
}
func (c *fakeClock) Now() time.Time {
return c.t
}
func (c *fakeClock) Advance(d time.Duration) {
c.t = c.t.Add(d)
}
// testSession creates a Session with a real cookie store and the
// real clock.
func testSession(t *testing.T) *session.Session {
t.Helper()
s, _ := testSessionWithClock(t, testIdleTimeout, nil)
return s
}
// testSessionWithClock creates a Session with a real cookie store,
// the given idle timeout, and a manually advanced clock. Passing a
// nil clock uses the real one.
func testSessionWithClock(
t *testing.T,
idleTimeout time.Duration,
clock *fakeClock,
) (*session.Session, *fakeClock) {
t.Helper()
key := make([]byte, testKeySize)
for i := range key {
key[i] = byte(i + 42)
}
store := sessions.NewCookieStore(key)
store.Options = &sessions.Options{
Path: "/",
MaxAge: 86400 * 7,
HttpOnly: true,
Secure: false,
SameSite: http.SameSiteLaxMode,
}
cfg := &config.Config{
Environment: config.EnvironmentDev,
SessionIdleTimeout: idleTimeout,
}
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
var now func() time.Time
if clock != nil {
now = clock.Now
}
return session.NewForTest(store, cfg, log, key, now), clock
}
// newFakeClock returns a clock started at a fixed instant.
func newFakeClock() *fakeClock {
return &fakeClock{
t: time.Date(
2026, time.January, 2, 3, 4, 5, 0, time.UTC,
),
}
}
// authenticatedSession returns a fresh session that has just been
// logged in, along with its manager and clock.
func authenticatedSession(
t *testing.T,
idleTimeout time.Duration,
) (*session.Session, *sessions.Session, *fakeClock) {
t.Helper()
s, clock := testSessionWithClock(
t, idleTimeout, newFakeClock(),
)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
require.True(t, s.IsAuthenticated(sess))
return s, sess, clock
}
// --- Get and Save Tests ---
func TestGet_NewSession(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
require.NotNil(t, sess)
assert.True(
t, sess.IsNew,
"session should be new when no cookie is present",
)
}
func TestGet_ExistingSession(t *testing.T) {
t.Parallel()
s := testSession(t)
// Create and save a session
req1 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w1 := httptest.NewRecorder()
sess1, err := s.Get(req1)
require.NoError(t, err)
sess1.Values["test_key"] = "test_value"
require.NoError(t, s.Save(req1, w1, sess1))
// Extract cookies
cookies := w1.Result().Cookies()
require.NotEmpty(t, cookies)
// Make a new request with the session cookie
req2 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
for _, c := range cookies {
req2.AddCookie(c)
}
sess2, err := s.Get(req2)
require.NoError(t, err)
assert.False(
t, sess2.IsNew,
"session should not be new when cookie is present",
)
assert.Equal(t, "test_value", sess2.Values["test_key"])
}
func TestSave_SetsCookie(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder()
sess, err := s.Get(req)
require.NoError(t, err)
sess.Values["key"] = "value"
err = s.Save(req, w, sess)
require.NoError(t, err)
cookies := w.Result().Cookies()
require.NotEmpty(t, cookies, "Save should set a cookie")
// Verify the cookie has the expected name
var found bool
for _, c := range cookies {
if c.Name == session.SessionName {
found = true
assert.True(
t, c.HttpOnly,
"session cookie should be HTTP-only",
)
break
}
}
assert.True(
t, found,
"should find a cookie named %s", session.SessionName,
)
}
// --- SetUser and User Retrieval Tests ---
func TestSetUser_SetsAllFields(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-abc-123", "alice")
assert.Equal(
t, "user-abc-123", sess.Values[session.UserIDKey],
)
assert.Equal(
t, "alice", sess.Values[session.UsernameKey],
)
assert.Equal(
t, true, sess.Values[session.AuthenticatedKey],
)
}
// testSessionGetter exercises a session string getter before and
// after SetUser: it must report false with an empty value on a
// fresh session, then true with the expected value once
// SetUser(sess, "user-xyz", "bob") has run.
func testSessionGetter(
t *testing.T,
get func(
*session.Session, *sessions.Session,
) (string, bool),
expected string,
) {
t.Helper()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
// Before setting user
val, ok := get(s, sess)
assert.False(
t, ok, "should return false before SetUser",
)
assert.Empty(t, val)
// After setting user
s.SetUser(sess, "user-xyz", "bob")
val, ok = get(s, sess)
assert.True(t, ok)
assert.Equal(t, expected, val)
}
func TestGetUserID(t *testing.T) {
t.Parallel()
testSessionGetter(
t,
func(
s *session.Session, sess *sessions.Session,
) (string, bool) {
return s.GetUserID(sess)
},
"user-xyz",
)
}
func TestGetUsername(t *testing.T) {
t.Parallel()
testSessionGetter(
t,
func(
s *session.Session, sess *sessions.Session,
) (string, bool) {
return s.GetUsername(sess)
},
"bob",
)
}
// --- IsAuthenticated Tests ---
func TestIsAuthenticated_NoSession(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
assert.False(
t, s.IsAuthenticated(sess),
"new session should not be authenticated",
)
}
func TestIsAuthenticated_AfterSetUser(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
assert.True(t, s.IsAuthenticated(sess))
}
func TestIsAuthenticated_AfterClearUser(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
require.True(t, s.IsAuthenticated(sess))
s.ClearUser(sess)
assert.False(
t, s.IsAuthenticated(sess),
"should not be authenticated after ClearUser",
)
}
func TestIsAuthenticated_WrongType(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
// Set authenticated to a non-bool value
sess.Values[session.AuthenticatedKey] = "yes"
assert.False(
t, s.IsAuthenticated(sess),
"should return false for non-bool authenticated value",
)
}
// --- ClearUser Tests ---
func TestClearUser_RemovesAllKeys(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
s.ClearUser(sess)
_, hasUserID := sess.Values[session.UserIDKey]
assert.False(t, hasUserID, "UserIDKey should be removed")
_, hasUsername := sess.Values[session.UsernameKey]
assert.False(t, hasUsername, "UsernameKey should be removed")
_, hasAuth := sess.Values[session.AuthenticatedKey]
assert.False(
t, hasAuth, "AuthenticatedKey should be removed",
)
}
// --- Destroy Tests ---
func TestDestroy_InvalidatesSession(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
s.Destroy(sess)
// After Destroy: MaxAge should be -1 (delete cookie) and
// user data cleared
assert.Equal(
t, -1, sess.Options.MaxAge,
"Destroy should set MaxAge to -1",
)
assert.False(
t, s.IsAuthenticated(sess),
"should not be authenticated after Destroy",
)
_, hasUserID := sess.Values[session.UserIDKey]
assert.False(t, hasUserID, "Destroy should clear user ID")
}
// --- Session Persistence Round-Trip ---
func TestSessionPersistence_RoundTrip(t *testing.T) {
t.Parallel()
s := testSession(t)
// Step 1: Create session, set user, save
req1 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w1 := httptest.NewRecorder()
sess1, err := s.Get(req1)
require.NoError(t, err)
s.SetUser(sess1, "user-round-trip", "charlie")
require.NoError(t, s.Save(req1, w1, sess1))
cookies := w1.Result().Cookies()
require.NotEmpty(t, cookies)
// Step 2: New request with cookies -- session data should
// persist
req2 := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/profile", nil,
)
for _, c := range cookies {
req2.AddCookie(c)
}
sess2, err := s.Get(req2)
require.NoError(t, err)
assert.True(
t, s.IsAuthenticated(sess2),
"session should be authenticated after round-trip",
)
userID, ok := s.GetUserID(sess2)
assert.True(t, ok)
assert.Equal(t, "user-round-trip", userID)
username, ok := s.GetUsername(sess2)
assert.True(t, ok)
assert.Equal(t, "charlie", username)
}
// --- Constants Tests ---
func TestSessionConstants(t *testing.T) {
t.Parallel()
assert.Equal(t, "webhooker_session", session.SessionName)
assert.Equal(t, "user_id", session.UserIDKey)
assert.Equal(t, "username", session.UsernameKey)
assert.Equal(t, "authenticated", session.AuthenticatedKey)
assert.Equal(t, "created_at", session.CreatedAtKey)
assert.Equal(t, "last_seen", session.LastSeenKey)
}
// --- Expiry Tests ---
func TestSetUser_StartsBothClocks(t *testing.T) {
t.Parallel()
_, sess, clock := authenticatedSession(t, testIdleTimeout)
assert.Equal(
t, clock.Now().Unix(), sess.Values[session.CreatedAtKey],
"SetUser should anchor the absolute clock",
)
assert.Equal(
t, clock.Now().Unix(), sess.Values[session.LastSeenKey],
"SetUser should anchor the idle clock",
)
}
func TestIsAuthenticated_WithinIdleWindow(t *testing.T) {
t.Parallel()
s, sess, clock := authenticatedSession(t, testIdleTimeout)
clock.Advance(testIdleTimeout - time.Second)
assert.True(
t, s.IsAuthenticated(sess),
"session should still be valid just inside the idle window",
)
}
func TestIsAuthenticated_IdleExpired(t *testing.T) {
t.Parallel()
s, sess, clock := authenticatedSession(t, testIdleTimeout)
clock.Advance(testIdleTimeout)
assert.False(
t, s.IsAuthenticated(sess),
"session should expire once the idle window lapses",
)
}
// TestTouch_DoesNotExtendAbsoluteCap is the regression test for the
// refresh-the-wrong-clock bug: a session that is used continuously
// must survive well past the idle window and still die at the
// absolute cap.
func TestTouch_DoesNotExtendAbsoluteCap(t *testing.T) {
t.Parallel()
s, sess, clock := authenticatedSession(t, testIdleTimeout)
createdAt := sess.Values[session.CreatedAtKey]
// Stay active: a request every half idle window, right up to
// the absolute cap.
step := testIdleTimeout / 2
steps := int(testAbsoluteMaxAge/step) - 1
for i := range steps {
clock.Advance(step)
s.Touch(sess)
require.True(
t, s.IsAuthenticated(sess),
"active session should survive the idle window "+
"(step %d of %d)", i+1, steps,
)
}
// One more step of activity takes the session to exactly the
// absolute cap, measured from login. Nothing that happened in
// the loop may have moved that deadline.
clock.Advance(step)
s.Touch(sess)
assert.False(
t, s.IsAuthenticated(sess),
"activity must not extend the absolute cap",
)
assert.Equal(
t, createdAt, sess.Values[session.CreatedAtKey],
"Touch must never rewrite the absolute-clock anchor",
)
}
func TestTouch_RefreshesIdleDeadline(t *testing.T) {
t.Parallel()
s, sess, clock := authenticatedSession(t, testIdleTimeout)
// Halfway through the window, activity happens.
clock.Advance(testIdleTimeout / 2)
assert.True(
t, s.Touch(sess),
"Touch should refresh once past the lazy-refresh threshold",
)
// Past the original deadline, but inside the refreshed one.
clock.Advance(testIdleTimeout - time.Second)
assert.True(
t, s.IsAuthenticated(sess),
"refreshed session should outlive the original deadline",
)
// And it still expires an idle window after that activity.
clock.Advance(time.Second)
assert.False(
t, s.IsAuthenticated(sess),
"refreshed session should expire one window after activity",
)
}
func TestTouch_LazyBelowRefreshThreshold(t *testing.T) {
t.Parallel()
s, sess, clock := authenticatedSession(t, testIdleTimeout)
before := sess.Values[session.LastSeenKey]
// A request arriving almost immediately is not worth a cookie
// rewrite.
clock.Advance(time.Second)
assert.False(
t, s.Touch(sess),
"Touch should not rewrite the session below the threshold",
)
assert.Equal(
t, before, sess.Values[session.LastSeenKey],
"last-seen should be unchanged below the threshold",
)
}
func TestTouch_UnauthenticatedSessionIsNotRefreshed(t *testing.T) {
t.Parallel()
s, clock := testSessionWithClock(
t, testIdleTimeout, newFakeClock(),
)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
clock.Advance(testIdleTimeout / 2)
assert.False(
t, s.Touch(sess),
"an unauthenticated session must not be refreshed",
)
_, hasLastSeen := sess.Values[session.LastSeenKey]
assert.False(
t, hasLastSeen,
"Touch must not stamp an unauthenticated session",
)
}
func TestTouch_IdleExpiredSessionIsNotRevived(t *testing.T) {
t.Parallel()
s, sess, clock := authenticatedSession(t, testIdleTimeout)
clock.Advance(testIdleTimeout)
require.False(t, s.IsAuthenticated(sess))
assert.False(
t, s.Touch(sess),
"an already expired session must not be refreshed",
)
assert.False(
t, s.IsAuthenticated(sess),
"Touch must not revive an expired session",
)
}
func TestIsAuthenticated_MissingTimestamps(t *testing.T) {
t.Parallel()
s, _ := testSessionWithClock(
t, testIdleTimeout, newFakeClock(),
)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
// A session from before idle expiry existed: authenticated,
// but with no timestamps. Fail closed.
sess.Values[session.AuthenticatedKey] = true
assert.False(
t, s.IsAuthenticated(sess),
"a session with no timestamps should be rejected",
)
}
func TestIsAuthenticated_MissingLastSeen(t *testing.T) {
t.Parallel()
s, sess, _ := authenticatedSession(t, testIdleTimeout)
delete(sess.Values, session.LastSeenKey)
assert.False(
t, s.IsAuthenticated(sess),
"a session with no idle anchor should be rejected",
)
}
func TestIdleTimeoutDisabled_AbsoluteCapStillApplies(t *testing.T) {
t.Parallel()
s, sess, clock := authenticatedSession(t, 0)
// Idle expiry is off, so an untouched session survives an
// arbitrary idle stretch.
clock.Advance(testAbsoluteMaxAge - time.Second)
assert.True(
t, s.IsAuthenticated(sess),
"idle expiry should be disabled by a non-positive timeout",
)
assert.False(
t, s.Touch(sess),
"Touch should be a no-op when idle expiry is disabled",
)
// The absolute cap still ends it.
clock.Advance(time.Second)
assert.False(
t, s.IsAuthenticated(sess),
"the absolute cap must still apply with idle expiry off",
)
}
func TestClearUser_RemovesTimestamps(t *testing.T) {
t.Parallel()
s, sess, _ := authenticatedSession(t, testIdleTimeout)
s.ClearUser(sess)
_, hasCreatedAt := sess.Values[session.CreatedAtKey]
assert.False(t, hasCreatedAt, "CreatedAtKey should be removed")
_, hasLastSeen := sess.Values[session.LastSeenKey]
assert.False(t, hasLastSeen, "LastSeenKey should be removed")
}
// --- Edge Cases ---
func TestSetUser_OverwritesPreviousUser(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-1", "alice")
assert.True(t, s.IsAuthenticated(sess))
// Overwrite with a different user
s.SetUser(sess, "user-2", "bob")
userID, ok := s.GetUserID(sess)
assert.True(t, ok)
assert.Equal(t, "user-2", userID)
username, ok := s.GetUsername(sess)
assert.True(t, ok)
assert.Equal(t, "bob", username)
}
func TestDestroy_ThenSave_DeletesCookie(t *testing.T) {
t.Parallel()
s := testSession(t)
// Create a session
req1 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w1 := httptest.NewRecorder()
sess, err := s.Get(req1)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
require.NoError(t, s.Save(req1, w1, sess))
cookies := w1.Result().Cookies()
require.NotEmpty(t, cookies)
// Destroy and save
req2 := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/logout", nil,
)
for _, c := range cookies {
req2.AddCookie(c)
}
w2 := httptest.NewRecorder()
sess2, err := s.Get(req2)
require.NoError(t, err)
s.Destroy(sess2)
require.NoError(t, s.Save(req2, w2, sess2))
// The cookie should have MaxAge = -1 (browser should delete)
responseCookies := w2.Result().Cookies()
var sessionCookie *http.Cookie
for _, c := range responseCookies {
if c.Name == session.SessionName {
sessionCookie = c
break
}
}
require.NotNil(
t, sessionCookie,
"should have a session cookie in response",
)
assert.Negative(
t, sessionCookie.MaxAge,
"destroyed session cookie should have negative MaxAge",
)
}

Some files were not shown because too many files have changed in this diff Show More