47 Commits

Author SHA1 Message Date
8cf9d0525a feat: add receiver rate limiting (refs #64)
Some checks failed
check / check (push) Failing after 59s
2026-08-07 18:32:00 +00:00
81413c56e9 Refactor delivery targets to a Target interface (closes #77) (#81)
All checks were successful
check / check (push) Successful in 2m42s
Refactors the delivery engine so each target TYPE is an implementation of a `Target` interface, dispatched from a registry, with each target owning its full delivery including durable retries. Implements the authoritative design from issue #77 (the corrected "hand the DB + Scheduler to the target" design).

## The new interface

```go
type Scheduler interface {
    ScheduleRetry(task Task, delay time.Duration)
}

type Target interface {
    Deliver(ctx context.Context, webhookDB *gorm.DB,
        d *database.Delivery, task *Task, sched Scheduler)
}
```

`Deliver` receives everything a target needs to be autonomous and durable: the request context, the per-webhook `*gorm.DB`, the `*database.Delivery`, the attempt `*Task`, and a `Scheduler` (the engine) for durable re-enqueue. The target makes one attempt, writes the `DeliveryResult`, updates `DeliveryStatus`, and — for retry targets — decides whether to retry, computes its own backoff, gates with its own circuit breaker, and reschedules via the injected `Scheduler`.

`processDelivery` collapses to a registry lookup (`map[database.TargetType]Target`) and a `Deliver` call; an unknown target type still fails the delivery as before.

## Per-target ownership

- `httpTarget` and `slackTarget` share a retry core (`httpCore`) that owns retry, exponential backoff, and the per-target circuit breaker. The core is fire-and-forget when `MaxRetries == 0` and adds breaker-gated backed-off retries when `MaxRetries > 0`. The per-attempt request differs (HTTP forwards the body + filtered headers; Slack posts a formatted message) and is supplied as a closure, so each keeps its exact recording semantics (e.g. HTTP records no error string for a non-2xx, Slack records `HTTP <code>`).
- `databaseTarget` and `logTarget` are fire-and-forget: they record a single successful attempt.

Moved wholesale into the http/slack targets: `deliverHTTP*`, `handleHTTPRetry`, `circuitBreakerBlock`, `calcBackoff` / `calcRemainingBackoff` / `backoffElapsed`, the circuit-breaker `sync.Map` + `getCircuitBreaker`, `clientForConfig`, `doHTTPRequest`, `applyRequestHeaders`, and the config parsers. The engine keeps `recordResult`, `updateDeliveryStatus`, and `ScheduleRetry`.

## Slack MaxRetries gating

Slack is now on the same shared core as HTTP, with retry + breaker gated on `MaxRetries`. A `MaxRetries` of 0 stays single-attempt fire-and-forget, so **every existing Slack target is unchanged**; a Slack target configured with retries gets backoff + circuit breaker.

## Log-target full content

`logTarget` now logs the ENTIRE inbound webhook — full request body and full request headers, plus method, content type, and the webhook id and entrypoint id — rather than a summary line. This supersedes the smaller log-summary work (#70).

## `Task.EntrypointID`

To carry the entrypoint id to the log target, `Task` gains an `EntrypointID` field, populated in the webhook handler's `buildDeliveryTasks`, the engine's recovery-task builder, and `buildEventFromTask`.

## Durability / recovery

The crash-durable async retry model is preserved unchanged: one attempt per worker turn; on failure the status is set `retrying`, backoff is computed, and the task is re-enqueued via `ScheduleRetry` (a `time.AfterFunc` onto the retry channel). On restart, `recoverRetryingDeliveries` and the 60s sweep hand each orphaned `retrying` delivery back to its target to recompute the remaining backoff and reschedule (targets that own retries implement an internal `rescheduler`; fire-and-forget targets, which never produce `retrying` deliveries, are skipped).

## How behaviour is preserved

No external behaviour changes except the two called out above (log target full content; Slack gaining `MaxRetries`-gated retries). All existing delivery tests pass with only their `export_test.go` wrappers re-pointed at the new structure — `ExportDeliverHTTP/Slack/Database/Log` now call the targets, `ExportGetCircuitBreaker` / `ExportClient` / `ExportClientForConfig` / `ExportDoHTTPRequest` resolve against the HTTP target's shared client and breaker map, and `ExportParseHTTPConfig` / `ExportParseSlackConfig` call the relocated free functions. Added: a `logTarget` test asserting the log line contains the full body, headers, and ids, and a Slack `MaxRetries`-gated retry test.

`docker build .` is green (fmt-check, lint, test, static build all pass).

Closes #77

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #81
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 17:07:49 +02:00
f6b929f2d7 Add per-webhook event retention reaper (closes #63) (#78)
All checks were successful
check / check (push) Successful in 2m42s
Enforces each webhook's `RetentionDays` so per-webhook SQLite files no longer grow without bound.

## Reaper

New `RetentionReaper` in `internal/database/retention.go`. A background ticker runs each sweep: it lists all webhooks from the main DB and, for each webhook with a positive `RetentionDays`, opens its per-webhook DB via `WebhookDBManager.GetDB` and deletes every `Event` (and its dependent `Delivery` and `DeliveryResult` rows) whose `CreatedAt` is older than `RetentionDays` days.

- Deletions run in foreign-key-safe order: delivery results, then deliveries, then events.
- Deletes are unscoped (hard deletes) so rows are physically removed and disk is reclaimed, rather than GORM soft-deleting them.
- `RetentionDays <= 0` means retain forever; those webhooks are skipped.
- Webhooks whose per-webhook DB does not yet exist are skipped.

## Config

`internal/config/config.go` gains `RetentionSweepInterval` (env `RETENTION_SWEEP_INTERVAL`, parsed as a Go duration, default `1h`) via a new `envDuration` helper, following the existing env-helper conventions.

## Wiring

`cmd/webhooker/main.go` registers `database.NewRetentionReaper` as an fx provider and forces its construction in `fx.Invoke`. The reaper starts its sweep loop on an fx `OnStart` hook and stops cleanly on `OnStop` via context cancellation, matching the existing lifecycle components.

## Test

`internal/database/retention_test.go` seeds an old event chain (event + delivery + result, 40 days old) and a recent one (1 day old) in a real per-webhook DB and asserts a single sweep removes only the expired chain while keeping the recent one. A second test forces a non-positive `RetentionDays` and asserts an ancient event is retained.

Note: the `Webhook.RetentionDays` column carries `gorm:"default:30"`, so a `0` passed to a GORM `Create` is replaced by the default; the test forces the value with an explicit column update to exercise the retain-forever path. No model changes were made.

Validated with `docker build .` (fmt-check, lint, test, build) exit 0.

Closes #63

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #78
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 16:15:13 +02:00
8ea7f76540 Add NoCache middleware for authenticated pages (closes #61) (#75)
All checks were successful
check / check (push) Successful in 6s
Adds a `NoCache()` middleware that sets `Cache-Control: no-store` and `Pragma: no-cache`, and wires it onto the dynamic app route groups (`/pages`, `/user/{username}`, `/sources`, `/source/{sourceID}`) adjacent to their existing `CSRF()` call. The static `/s` mount, `/metrics`, `/webhook/{uuid}`, and `/.well-known/healthcheck` are intentionally left untouched (static assets are safe to cache; the others are not authenticated pages).

A middleware unit test asserts both headers are set.

Closes #61

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #75
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 15:33:44 +02:00
752d6beead Validate Slack target URLs at creation time (closes #68) (#73)
All checks were successful
check / check (push) Successful in 4s
Slack delivery targets were only checked by the request-time dialer guard, not at creation, giving them a weaker SSRF gate than HTTP targets.

This validates the Slack incoming-webhook URL with `delivery.ValidateTargetURL` in the Slack target creation path (`buildSlackTargetConfig`), before persisting, mirroring the existing HTTP-target path. On failure the create is rejected with the same clear, non-leaking user-facing error the HTTP path uses.

Adds handlers-package tests covering both an accepted public URL and a rejected private/reserved URL. Confined to `internal/handlers/`; `internal/delivery/` is unchanged.

Closes #68

Co-authored-by: sneak <sneak@sneak.berlin>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #73
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 14:03:56 +02:00
b1f43c9520 Keep the SSRF-safe transport in clientForConfig (closes #69) (#74)
Some checks failed
check / check (push) Has been cancelled
`clientForConfig()` in `internal/delivery/engine.go` built a fresh `http.Client` without a Transport when a per-target timeout was configured, dropping the request-time private-IP guard for that path.

It now reuses the shared client's SSRF-safe transport (`e.client.Transport`, the same `NewSSRFSafeTransport` instance), overriding only the `Timeout`. Behaviour is unchanged when no per-target timeout is set (the shared client is returned as before), so no engine code path makes an outbound target request with a client lacking the SSRF-safe transport.

Adds a delivery-package test proving a client from `clientForConfig()` with a per-target timeout still refuses private/reserved/link-local destinations, that the timeout is applied, that the SSRF-safe transport is reused (not duplicated), and that the no-timeout path returns the shared client unchanged.

Confined to `internal/delivery/` only; handlers and server code untouched.

Closes #69

Co-authored-by: sneak <sneak@sneak.berlin>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #74
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 14:03:38 +02:00
07fc63d9fa Wrap /user/{username} in RequireAuth middleware (closes #60) (#71)
All checks were successful
check / check (push) Successful in 4s
Enforces authentication for the `/user/{username}` route group at the middleware layer, matching every other authenticated route group.

## Changes

- **`internal/server/routes.go`** (`setupUserRoutes`): added `r.Use(s.mw.RequireAuth())` immediately after the existing `r.Use(s.mw.CSRF())` on the `/user/{username}` group, so auth is enforced by design (CSRF first, then RequireAuth) — consistent with `/sources` and `/source/{sourceID}`.
- **`internal/handlers/profile.go`** (`HandleProfile`): removed the now-dead unauthenticated-redirect branch (RequireAuth guarantees an authenticated session before the handler runs). The handler still reads the username and user id from the session for the own-profile-only check; a request for another user's profile still returns 403. The session-retrieval error is now handled as a 500.

## Tests (`internal/handlers/profile_test.go`)

- own profile returns 200
- another user's profile returns 403
- an unauthenticated request to `/user/{username}` is redirected to `/pages/login` at the middleware layer and never reaches the endpoint handler (routing-level test replicating the CSRF + RequireAuth chain)

## Validation

`docker build .` (fmt-check, lint, test, build) passes.

Closes #60

Co-authored-by: sneak <sneak@sneak.berlin>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #71
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 14:00:16 +02:00
0c9c885d51 Raise HTTP WriteTimeout above the request middleware timeout (closes #62) (#72)
All checks were successful
check / check (push) Successful in 4s
Raise `httpWriteTimeout` in `internal/server/http.go` from 10s to `65 * time.Second` so it comfortably exceeds the router's 60s `requestTimeout`. This makes the `middleware.Timeout(60s)` the effective request limit — a slow response now returns a clean 503 from the middleware instead of being cut at the socket write deadline by the transport.

`httpReadTimeout` stays at 10s. A comment on `httpWriteTimeout` documents that it must remain above the 60s request timeout. Change is confined to `internal/server/http.go`; `routes.go` is untouched.

Closes #62

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #72
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 13:58:28 +02:00
2cc8723997 scripts-to-rule-them-all (#59)
All checks were successful
check / check (push) Successful in 4s
Reviewed-on: #59
Co-authored-by: sneak <sneak@sneak.berlin>
Co-committed-by: sneak <sneak@sneak.berlin>
2026-07-07 02:14:09 +02:00
e0b1e7cf54 Restore TODO.md and move TODO content out of README (#58)
All checks were successful
check / check (push) Successful in 4s
Reviewed-on: #58
Co-authored-by: sneak <sneak@sneak.berlin>
Co-committed-by: sneak <sneak@sneak.berlin>
2026-07-06 21:14:03 +02:00
afe88c601a refactor: use pinned golangci-lint Docker image for linting (#55)
All checks were successful
check / check (push) Successful in 5s
Closes [issue #50](#50)

## Summary

Refactors the Dockerfile to use a separate lint stage with a pinned golangci-lint Docker image, following the pattern used by [sneak/pixa](https://git.eeqj.de/sneak/pixa). This replaces the previous approach of installing golangci-lint via curl in the builder stage.

## Changes

### Dockerfile
- **New `lint` stage** using `golangci/golangci-lint:v2.11.3` (Debian-based, pinned by sha256 digest) as a separate build stage
- **Builder stage** depends on lint via `COPY --from=lint /src/go.sum /dev/null` — build won't proceed unless linting passes
- **Go bumped** from 1.24 to 1.26.1 (`golang:1.26.1-bookworm`, pinned by sha256)
- **golangci-lint bumped** from v1.64.8 to v2.11.3
- All three Docker images (golangci-lint, golang, alpine) pinned by sha256 digest
- Debian-based golangci-lint image used (not Alpine) because mattn/go-sqlite3 CGO does not compile on musl (off64_t)

### Linter Config (.golangci.yml)
- Migrated from v1 to v2 format (`version: "2"` added)
- Removed linters no longer available in v2: `gofmt` (handled by `make fmt-check`), `gosimple` (merged into `staticcheck`), `typecheck` (always-on in v2)
- Same set of linters enabled — no rules weakened

### Code Fixes (all lint issues from v2 upgrade)
- Added package comments to all packages
- Added doc comments to all exported types, functions, and methods
- Fixed unchecked errors flagged by `errcheck` (sqlDB.Close, os.Setenv in tests, resp.Body.Close, fmt.Fprint)
- Fixed unused parameters flagged by `revive` (renamed to `_`)
- Fixed `gosec` G120 warnings: added `http.MaxBytesReader` before `r.ParseForm()` calls
- Fixed `staticcheck` QF1012: replaced `WriteString(fmt.Sprintf(...))` with `fmt.Fprintf`
- Fixed `staticcheck` QF1003: converted if/else chain to tagged switch
- Renamed `DeliveryTask` → `Task` to avoid package stutter (`delivery.Task` instead of `delivery.DeliveryTask`)
- Renamed shadowed builtin `max` parameter to `upperBound` in `cryptoRandInt`
- Used `t.Setenv` instead of `os.Setenv` in tests (auto-restores)

### README.md
- Updated version requirements: Go 1.26+, golangci-lint v2.11+
- Updated Dockerfile description in project structure

## Verification

`docker build .` passes cleanly — formatting check, linting, all tests, and build all succeed.

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #55
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-25 02:16:38 +01:00
d771fe14df fix: detect TLS per-request in CSRF middleware to fix login (#54)
All checks were successful
check / check (push) Successful in 1m55s
## Problem

After the security hardening in PR #42, login fails with `Forbidden - invalid CSRF token` in production deployments.

The CSRF middleware tied its `PlaintextHTTPRequest` wrapping and cookie `Secure` flag to the `IsDev()` environment check. This meant production mode always assumed HTTPS via gorilla/csrf's strict mode, which broke login in common deployment scenarios:

1. **Production behind a TLS-terminating reverse proxy**: gorilla/csrf assumed HTTPS but `r.TLS` was nil (the Go server receives HTTP from the proxy). Origin/Referer scheme mismatches caused `referer not supplied` or `origin invalid` errors.

2. **Production over direct HTTP** (testing/staging with prod config): the `Secure` cookie flag prevented the browser from sending the CSRF cookie back over HTTP, causing `CSRF token invalid` errors.

## Root Cause

gorilla/csrf v1.7.3 defaults to HTTPS-strict mode unless `PlaintextHTTPRequest()` is called. In strict mode it:
- Forces `requestURL.Scheme = "https"` for Origin/Referer comparisons
- Requires a `Referer` header on POST and rejects `http://` Referer schemes
- The `csrf.Secure(true)` option makes the browser refuse to send the CSRF cookie over HTTP

The old code only called `PlaintextHTTPRequest()` in dev mode, leaving prod mode permanently stuck in HTTPS-strict mode regardless of the actual transport.

## Fix

Detect the actual transport protocol **per-request** using:
- `r.TLS != nil` — direct TLS connection to the Go server
- `X-Forwarded-Proto: https` header — TLS-terminating reverse proxy

Two gorilla/csrf middleware instances are maintained (one with `Secure: true`, one with `Secure: false`) since `csrf.Secure()` is a creation-time option. Both use the same signing key, so cookies are interchangeable.

| Scenario | Cookie Secure | Origin/Referer Mode |
|---|---|---|
| Direct TLS (`r.TLS != nil`) |  Secure | Strict (HTTPS scheme) |
| Behind TLS proxy (`X-Forwarded-Proto: https`) |  Secure | Strict (HTTPS scheme) |
| Plaintext HTTP |  Non-Secure | Relaxed (PlaintextHTTPRequest) |

CSRF token validation (cookie + form double-submit) is always enforced regardless of mode.

## Testing

- Added `TestCSRF_ProdMode_PlaintextHTTP_POSTWithValidToken` — prod mode over plaintext HTTP
- Added `TestCSRF_ProdMode_BehindProxy_POSTWithValidToken` — prod mode behind TLS proxy
- Added `TestCSRF_ProdMode_DirectTLS_POSTWithValidToken` — prod mode with direct TLS
- Added `TestCSRF_ProdMode_PlaintextHTTP_POSTWithoutToken` — token still required
- Added `TestIsClientTLS_*` — TLS detection unit tests
- All existing CSRF tests pass unchanged
- `docker build .` passes (includes `make check`)
- Manual verification: built and ran the container in both `dev` and `prod` modes, confirmed login succeeds in both

Closes #53

Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #54
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-18 04:30:57 +01:00
33e2140a5a feat: redirect root path based on auth state (#52)
All checks were successful
check / check (push) Successful in 1m54s
Closes #51

The root path `/` now checks for an authenticated session and redirects accordingly:

- **Authenticated users** → `303 See Other` redirect to `/sources` (the webhook dashboard)
- **Unauthenticated users** → `303 See Other` redirect to `/pages/login`

### Changes

- **`internal/handlers/index.go`** — Replaced the template-rendering `HandleIndex()` with a session-checking redirect handler. Removed `formatUptime` helper (dead code after this change).
- **`internal/handlers/handlers.go`** — Removed `index.html` from the template map (no longer rendered).
- **`internal/handlers/handlers_test.go`** — Replaced the old "handler is not nil" test with two proper redirect tests:
  - `unauthenticated redirects to login` — verifies 303 to `/pages/login`
  - `authenticated redirects to sources` — sets up an authenticated session cookie, verifies 303 to `/sources`
  - Removed `TestFormatUptime` (tested dead code).
- **`README.md`** — Updated the API endpoints table to describe the new redirect behavior.

### How it works

The handler calls `session.Get(r)` and `session.IsAuthenticated(sess)` — the same pattern used by the `RequireAuth` middleware and `HandleLoginPage`. No new dependencies or session logic introduced.

The login flow is unaffected: `HandleLoginSubmit` redirects to `/` after successful login, which now forwards to `/sources` (one extra redirect hop, but correct and clean).

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Co-authored-by: clawbot <clawbot@eeqj.de>
Reviewed-on: #52
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 16:35:08 +01:00
f003ec7141 fix: use absolute path for dev DATA_DIR default, clarify env docs (#46)
All checks were successful
check / check (push) Successful in 1m9s
Closes #45.

## Problem

1. The README didn't clearly explain what `WEBHOOKER_ENVIRONMENT=dev` vs `prod` actually changes.
2. The dev-mode default for `DATA_DIR` was `./data` — a relative path whose meaning depends on the working directory. There's no reason to use a relative path even in development.

## Changes

### Code (`internal/config/config.go`)

- Replace the dev default `DATA_DIR` from `./data` to `$XDG_DATA_HOME/webhooker` (falling back to `$HOME/.local/share/webhooker`). This follows the XDG Base Directory Specification and ensures the data directory is always an absolute path regardless of the working directory.
- Add `devDataDir()` helper that resolves the XDG path, with a `/tmp/webhooker` last-resort fallback if `$HOME` can't be determined.

### Tests (`internal/config/config_test.go`)

- `TestDevDataDir`: verifies XDG_DATA_HOME is respected, HOME fallback works, and the result is always absolute.
- `TestDevDefaultDataDirIsAbsolute`: integration test that creates a full Config via fx and asserts the dev default DataDir is absolute.

### README

- Add a table documenting exactly what `dev` vs `prod` changes: DATA_DIR default, CORS policy, and session cookie Secure flag.
- Clarify that log format and security headers are independent of the environment setting.
- Update the DATA_DIR default in the configuration variable table.

Co-authored-by: clawbot <clawbot@eeqj.de>
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #46
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 12:48:52 +01:00
17e740a45f fix: use absolute paths and static linking in Dockerfile (#49)
Some checks failed
check / check (push) Has been cancelled
Closes #48

## Problem

The Docker container failed to start with:
```
exec ./webhooker: no such file or directory
```

Two root causes:

1. **Relative paths**: `COPY` destination and `CMD` used relative paths (`./webhooker`), depending on `WORKDIR` context.

2. **Dynamic linking** (the actual root cause): The binary was built with CGO enabled on Debian (glibc) via `make build`, but deployed to an Alpine runtime (musl). The kernel couldn't find the glibc dynamic linker (`/lib64/ld-linux-x86-64.so.2`), producing the misleading "no such file or directory" error — even though the file existed on disk.

## Fix

- **Absolute paths throughout**: `COPY --from=builder /build/bin/webhooker /app/webhooker` and `CMD ["/app/webhooker"]` — no reliance on WORKDIR.

- **Static rebuild for Alpine**: Added a `RUN CGO_ENABLED=1 go build -ldflags '-extldflags "-static"' -o bin/webhooker ./cmd/webhooker` step after `make check`. This rebuilds the binary with static linking so it runs on Alpine without glibc. The `make check` step still runs normally (formatting, linting, tests, dynamic build) — the static rebuild is only for the deployment binary.

## Verification

- `docker build .` passes (all checks green)
- Container starts successfully and initializes the Fx dependency graph
- The README already stated "The runtime binary is statically linked and runs on Alpine" — this fix makes that claim actually true.

Co-authored-by: clawbot <clawbot@users.noreply.git.eeqj.de>
Reviewed-on: #49
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 12:48:13 +01:00
60786c5019 feat: add CSRF protection, SSRF prevention, and login rate limiting (#42)
All checks were successful
check / check (push) Successful in 4s
## Security Hardening

This PR implements three security hardening issues:

### CSRF Protection (closes #35)

- Session-based CSRF tokens with cryptographically random 256-bit generation
- Constant-time token comparison to prevent timing attacks
- CSRF middleware applied to `/pages`, `/sources`, `/source`, and `/user` routes
- Hidden `csrf_token` field added to all 12+ POST forms in templates
- Excluded from `/webhook` (inbound webhook POSTs) and `/api` (stateless API)

### SSRF Prevention (closes #36)

- `ValidateTargetURL()` blocks private/reserved IP ranges at target creation time
- Blocked ranges: `127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`, `::1`, `fc00::/7`, `fe80::/10`, plus multicast, reserved, test-net, and CGN ranges
- SSRF-safe HTTP transport with custom `DialContext` in the delivery engine for defense-in-depth (prevents DNS rebinding attacks)
- Only `http` and `https` schemes allowed

### Login Rate Limiting (closes #37)

- Per-IP rate limiter using `golang.org/x/time/rate`
- 5 attempts per minute per IP on `POST /pages/login`
- GET requests (form rendering) pass through unaffected
- Automatic cleanup of stale per-IP limiter entries every 5 minutes
- `X-Forwarded-For` and `X-Real-IP` header support for reverse proxies

### Files Changed

**New files:**
- `internal/middleware/csrf.go` + tests — CSRF middleware
- `internal/middleware/ratelimit.go` + tests — Login rate limiter
- `internal/delivery/ssrf.go` + tests — SSRF validation + safe transport

**Modified files:**
- `internal/server/routes.go` — Wire CSRF and rate limit middleware
- `internal/handlers/handlers.go` — Inject CSRF token into template data
- `internal/handlers/source_management.go` — SSRF validation on target creation
- `internal/delivery/engine.go` — SSRF-safe HTTP transport for production
- All form templates — Added hidden `csrf_token` fields
- `README.md` — Updated Security section and TODO checklist

`docker build .` passes (lint + tests + build).

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Co-authored-by: clawbot <clawbot@eeqj.de>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #42
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 12:38:45 +01:00
8d702a16c6 feat: add Slack target type for incoming webhook notifications (#47)
All checks were successful
check / check (push) Successful in 4s
## Summary

Adds a new `slack` target type that sends webhook events as formatted messages to any Slack-compatible incoming webhook URL (Slack, Mattermost, and other compatible services).

closes #44

## What it does

When a webhook event is received, the Slack target:

1. Formats a human-readable message with event metadata (HTTP method, content type, timestamp, body size)
2. Pretty-prints the payload in a code block — JSON payloads get indented formatting, non-JSON payloads are shown as raw text
3. Truncates large payloads at 3500 characters to keep Slack messages reasonable
4. POSTs the message as a `{"text": "..."}` JSON payload to the configured webhook URL

## Changes

- **`internal/database/model_target.go`** — Add `TargetTypeSlack` constant
- **`internal/delivery/engine.go`** — Add `SlackTargetConfig` struct, `deliverSlack` method, `FormatSlackMessage` function (exported), `parseSlackConfig` helper. Route slack targets in `processDelivery` switch.
- **`internal/handlers/source_management.go`** — Handle `slack` type in `HandleTargetCreate`, building `webhook_url` config from the URL form field
- **`templates/source_detail.html`** — Add "Slack" option to target type dropdown with URL field and helper text
- **`README.md`** — Document the new target type, update roadmap

## Tests

- `TestParseSlackConfig_Valid` / `_Empty` / `_MissingWebhookURL` — Config parsing
- `TestFormatSlackMessage_JSONBody` / `_NonJSONBody` / `_EmptyBody` / `_LargeJSONTruncated` — Message formatting
- `TestDeliverSlack_Success` / `_Failure` / `_InvalidConfig` — End-to-end delivery
- `TestProcessDelivery_RoutesToSlack` — Routing from processDelivery switch

All existing tests continue to pass. `docker build .` (which runs `make check`) passes clean.

Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #47
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-17 12:30:50 +01:00
1fbcf96581 security: add headers middleware, session regeneration, and body size limits (#41)
All checks were successful
check / check (push) Successful in 1m47s
## Summary

This PR implements three security hardening measures:

### Security Headers Middleware (closes #34)

Adds a `SecurityHeaders()` middleware applied globally to all routes. Every response now includes:
- `Strict-Transport-Security: max-age=63072000; includeSubDomains; preload`
- `X-Content-Type-Options: nosniff`
- `X-Frame-Options: DENY`
- `Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'`
- `Referrer-Policy: strict-origin-when-cross-origin`
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`

### Session Fixation Prevention (closes #38)

Adds a `Regenerate()` method to the session manager that destroys the old session and creates a new one with a fresh ID, copying all session values. Called after successful login to prevent session fixation attacks.

### Request Body Size Limits (closes #39)

Adds a `MaxBodySize()` middleware using `http.MaxBytesReader` to limit POST/PUT/PATCH request bodies to 1 MB. Applied to all form endpoints (`/pages`, `/sources`, `/source/*`).

## Files Changed

- `internal/middleware/middleware.go` — Added `SecurityHeaders()` and `MaxBodySize()` middleware
- `internal/session/session.go` — Added `Regenerate()` method for session fixation prevention
- `internal/handlers/auth.go` — Updated login handler to regenerate session after authentication
- `internal/server/routes.go` — Added SecurityHeaders globally, MaxBodySize to form route groups
- `README.md` — Documented new middleware in stack, updated Security section, moved items to completed TODO

closes #34, closes #38, closes #39

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #41
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-05 12:32:56 +01:00
a51e863017 Remove globals.Buildarch from codebase (#31)
All checks were successful
check / check (push) Successful in 1m0s
Remove the `Buildarch` field from the globals package and all references throughout the codebase.

**Changes:**
- Removed `Buildarch` package-level var and struct field from `internal/globals/globals.go`
- Removed `Buildarch` from the `New()` constructor
- Removed `globals.Buildarch = runtime.GOARCH` and unused `runtime` import from `cmd/webhooker/main.go`
- Removed `buildarch` from logger startup output in `internal/logger/logger.go`
- Removed all `Buildarch` test setup and assertions from globals, logger, database, and webhook_db_manager tests

All tests pass, `make check` passes, `docker build .` succeeds.

closes [issue #30](#30)

<!-- session: agent:sdlc-manager:subagent:5cae6803-6bdf-467d-9a56-43f135521e5f -->

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #31
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-04 12:07:58 +01:00
289f479772 test: add tests for delivery, middleware, and session packages (#32)
Some checks failed
check / check (push) Has been cancelled
## Summary

Add comprehensive test coverage for three previously-untested packages, addressing [issue #28](#28).

## Coverage Improvements

| Package | Before | After |
|---------|--------|-------|
| `internal/delivery` | 37.1% | 74.5% |
| `internal/middleware` | 0.0% | 70.2% |
| `internal/session` | 0.0% | 51.5% |

## What's Tested

### delivery (37% → 75%)
- `processNewTask` with inline and large (DB-fetched) bodies
- `processRetryTask` success, skip non-retrying, large body fetch
- Worker lifecycle start/stop, retry channel processing
- `processDelivery` unknown target type handling
- `recoverPendingDeliveries`, `recoverWebhookDeliveries`, `recoverInFlight`
- HTTP delivery with custom headers, timeout, invalid config
- `Notify` batching

### middleware (0% → 70%)
- Logging middleware status code capture and pass-through
- `LoggingResponseWriter` delegation
- CORS dev mode (allow-all) and prod mode (no-op)
- `RequireAuth` redirect for unauthenticated, pass-through for authenticated
- `MetricsAuth` basic auth validation
- `ipFromHostPort` helper

### session (0% → 52%)
- `Get`/`Save` round-trip with real cookie store
- `SetUser`, `GetUserID`, `GetUsername`, `IsAuthenticated`
- `ClearUser` removes all keys
- `Destroy` invalidates session (MaxAge -1)
- Session persistence across requests
- Edge cases: overwrite user, wrong type, constants

## Test Helpers Added
- `database.NewTestDatabase` / `NewTestWebhookDBManager` — cross-package test helpers for delivery integration tests
- `session.NewForTest` — creates session manager without fx lifecycle for middleware tests

## Notes
- No production code modified
- All tests use `httptest`, SQLite in-memory, and real cookie stores — no external network calls
- Full test suite completes in ~3.5s within the 30s timeout
- `docker build .` passes (lint + test + build)

closes #28

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #32
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-04 12:07:23 +01:00
687655ed49 Merge pull request 'feat: webhooker 1.0 MVP — entity rename, core engine, delivery, management UI' (#16) from feature/mvp-1.0 into main
All checks were successful
check / check (push) Successful in 4s
Reviewed-on: #16
2026-03-04 01:19:41 +01:00
clawbot
8e00e40008 docs: fix stale references to development mode and retry target type
All checks were successful
check / check (push) Successful in 5s
- README.md: remove 'in development mode' from admin user creation
  description (admin user creation is unconditional)
- internal/delivery/engine.go: remove 'and retry' from HTTPTargetConfig
  comment (retry was merged into http target type)
- internal/delivery/engine_test.go: remove '/retry' from
  newHTTPTargetConfig comment for consistency
2026-03-03 16:12:43 -08:00
clawbot
3588facfff remove unnecessary data migration and dead DevelopmentMode config
All checks were successful
check / check (push) Successful in 1m52s
- Remove retry→http data migration from migrate() — no databases exist pre-1.0
- Remove unused DevelopmentMode field and DEVELOPMENT_MODE env var from config
- Remove DevelopmentMode from config log output (dead code cleanup)
2026-03-03 09:16:03 -08:00
clawbot
25e27cc57f refactor: merge retry target type into http (max_retries=0 = fire-and-forget)
All checks were successful
check / check (push) Successful in 1m46s
2026-03-01 23:51:55 -08:00
clawbot
4dd4dfa5eb chore: consolidate DBURL into DATA_DIR, codebase audit for 1.0.0
All checks were successful
check / check (push) Successful in 56s
DBURL → DATA_DIR consolidation:
- Remove DBURL env var entirely; main DB now lives at {DATA_DIR}/webhooker.db
- database.go constructs DB path from config.DataDir, ensures dir exists
- Update DATA_DIR prod default from /data/events to /data
- Update all tests to use DataDir instead of DBURL
- Update Dockerfile: /data (not /data/events) for all SQLite databases
- Update README configuration table, Docker examples, architecture docs

Dead code removal:
- Remove unused IndexResponse struct (handlers/index.go)
- Remove unused TemplateData struct (handlers/handlers.go)

Stale comment cleanup:
- Remove TODO in server.go (DB cleanup handled by fx lifecycle)
- Fix nolint:golint → nolint:revive on ServerParams for consistency
- Clean up verbose middleware/routing comments in routes.go
- Fix TODO fan-out description (worker pool, not goroutine-per-target)

.gitignore fixes:
- Add data/ directory to gitignore
- Remove stale config.yaml entry (env-only config since rework)
2026-03-01 23:33:20 -08:00
clawbot
536e5682d6 test: add comprehensive delivery engine and circuit breaker tests
All checks were successful
check / check (push) Successful in 1m48s
Add unit tests for internal/delivery/ package covering:

Circuit breaker tests (circuit_breaker_test.go):
- Closed state allows deliveries
- Failure counting below threshold
- Open transition after threshold failures
- Cooldown blocks during cooldown period
- Half-open transition after cooldown expires
- Probe success closes circuit
- Probe failure reopens circuit
- Success resets failure counter
- Concurrent access safety (race-safe)
- CooldownRemaining for all states
- CircuitState String() output

Engine tests (engine_test.go):
- Non-blocking Notify when channel is full
- HTTP target success and failure delivery
- Database target immediate success
- Log target immediate success
- Retry target success with circuit breaker
- Max retries exhausted marks delivery failed
- Retry scheduling on failure
- Exponential backoff duration verification
- Backoff cap at shift 30
- Body pointer semantics (inline <16KB, nil >=16KB)
- Worker pool bounded concurrency
- Circuit breaker blocks delivery attempts
- Circuit breaker per-target creation
- HTTP config parsing (valid, empty, missing URL)
- scheduleRetry sends to retry channel
- scheduleRetry drops when channel full
- Header forwarding (forwardable vs hop-by-hop)
- processDelivery routing to correct handler
- Truncate helper function

All tests use real SQLite databases and httptest servers.
All tests pass with -race flag.
2026-03-01 23:16:30 -08:00
clawbot
49852e7506 refactor: remove file-based configuration, use env vars only
All checks were successful
check / check (push) Successful in 1m0s
Remove the entire pkg/config package (Viper-based YAML config file
loader) and simplify internal/config to read all settings directly from
environment variables via os.Getenv(). This eliminates the spurious
"Failed to load config" log messages that appeared when no config.yaml
file was present.

- Delete pkg/config/ (YAML loader, resolver, manager, tests)
- Delete configs/config.yaml.example
- Simplify internal/config helper functions to use os.Getenv() with
  defaults instead of falling back to pkgconfig
- Update tests to set env vars directly instead of creating in-memory
  YAML config files via afero
- Remove afero, cloud.google.com/*, aws-sdk-go dependencies from go.mod
- Update README: document env-var-only configuration, remove YAML/Viper
  references
- Keep godotenv/autoload for .env file convenience in local development

closes #27
2026-03-01 23:04:49 -08:00
clawbot
10db6c5b84 refactor: bounded worker pool with DB-mediated retry fallback
All checks were successful
check / check (push) Successful in 58s
Replace unbounded goroutine-per-delivery fan-out with a fixed-size
worker pool (10 workers). Channels serve as bounded queues (10,000
buffer). Workers are the only goroutines doing HTTP delivery.

When retry channel overflows, timers are dropped instead of re-armed.
The delivery stays in 'retrying' status in the DB and a periodic sweep
(every 60s) recovers orphaned retries. The database is the durable
fallback — same path used on startup recovery.

Addresses owner feedback on circuit breaker recovery goroutine flood.
2026-03-01 22:52:27 -08:00
clawbot
9b4ae41c44 feat: parallel fan-out delivery + circuit breaker for retry targets
All checks were successful
check / check (push) Successful in 1m52s
- Fan out all targets for an event in parallel goroutines (fire-and-forget)
- Add per-target circuit breaker for retry targets (closed/open/half-open)
- Circuit breaker trips after 5 consecutive failures, 30s cooldown
- Open circuit skips delivery and reschedules after cooldown
- Half-open allows one probe delivery to test recovery
- HTTP/database/log targets unaffected (no circuit breaker)
- Recovery path also fans out in parallel
- Update README with parallel delivery and circuit breaker docs
2026-03-01 22:20:33 -08:00
clawbot
32bd40b313 refactor: self-contained delivery tasks — engine delivers without DB reads in happy path
All checks were successful
check / check (push) Successful in 58s
The webhook handler now builds DeliveryTask structs carrying all target
config and event data inline (for bodies ≤16KB) and sends them through
the delivery channel. In the happy path, the engine delivers without
reading from any database — it only writes to record delivery results.

For large bodies (≥16KB), Body is nil and the engine fetches it from the
per-webhook database on demand. Retry timers also carry the full
DeliveryTask, so retries avoid unnecessary DB reads.

The database is used for crash recovery only: on startup the engine scans
for interrupted pending/retrying deliveries and re-queues them.

Implements owner feedback from issue #15:
> the message in the <=16KB case should have everything it needs to do
> its delivery. it shouldn't touch the db until it has a success or
> failure to record.
2026-03-01 22:09:41 -08:00
9b9ee1718a refactor: auto-generate session key and store in database
All checks were successful
check / check (push) Successful in 57s
Remove SESSION_KEY env var requirement. On first startup, a
cryptographically secure 32-byte key is generated and stored in a new
settings table. Subsequent startups load the key from the database.

- Add Setting model (key-value table) for application config
- Add Database.GetOrCreateSessionKey() method
- Session manager initializes in OnStart after database is connected
- Remove DevSessionKey constant and SESSION_KEY env var handling
- Remove prod validation requiring SESSION_KEY
- Update README: config table, Docker instructions, security notes
- Update config.yaml.example
- Update all tests to remove SessionKey references

Addresses owner feedback on issue #15.
2026-03-01 21:57:19 -08:00
clawbot
5e683af2a4 refactor: event-driven delivery engine with channel notifications and timer-based retries
All checks were successful
check / check (push) Successful in 58s
Replace the polling-based delivery engine with a fully event-driven
architecture using Go channels and goroutines:

- Webhook handler notifies engine via buffered channel after creating
  delivery records, with inline event data for payloads < 16KB
- Large payloads (>= 16KB) use pointer semantics (Body *string = nil)
  and are fetched from DB on demand, keeping channel memory bounded
- Failed retry-target deliveries schedule Go timers with exponential
  backoff; timers fire into a separate retry channel when ready
- On startup, engine scans DB once to recover interrupted deliveries
  (pending processed immediately, retrying get timers for remaining
  backoff)
- DB stores delivery status for crash recovery only, not for
  inter-component communication during normal operation
- delivery.Notifier interface decouples handlers from engine; fx wires
  *Engine as Notifier

No more periodic polling. No more wasted cycles when idle.
2026-03-01 21:46:16 -08:00
clawbot
8f62fde8e9 revert admin password logging to slog.Info (closes #26)
All checks were successful
check / check (push) Successful in 1m58s
2026-03-01 21:26:31 -08:00
clawbot
43c22a9e9a feat: implement per-webhook event databases
All checks were successful
check / check (push) Successful in 1m50s
Split data storage into main application DB (config only) and
per-webhook event databases (one SQLite file per webhook).

Architecture changes:
- New WebhookDBManager component manages per-webhook DB lifecycle
  (create, open, cache, delete) with lazy connection pooling via sync.Map
- Main DB (DBURL) stores only config: Users, Webhooks, Entrypoints,
  Targets, APIKeys
- Per-webhook DBs (DATA_DIR) store Events, Deliveries, DeliveryResults
  in files named events-{webhook_uuid}.db
- New DATA_DIR env var (default: ./data dev, /data/events prod)

Behavioral changes:
- Webhook creation creates per-webhook DB file
- Webhook deletion hard-deletes per-webhook DB file (config soft-deleted)
- Event ingestion writes to per-webhook DB, not main DB
- Delivery engine polls all per-webhook DBs for pending deliveries
- Database target type marks delivery as immediately successful (events
  are already in the dedicated per-webhook DB)
- Event log UI reads from per-webhook DBs with targets from main DB
- Existing webhooks without DB files get them created lazily

Removed:
- ArchivedEvent model (was a half-measure, replaced by per-webhook DBs)
- Event/Delivery/DeliveryResult removed from main DB migrations

Added:
- Comprehensive tests for WebhookDBManager (create, delete, lazy
  creation, delivery workflow, multiple webhooks, close all)
- Dockerfile creates /data/events directory

README updates:
- Per-webhook event databases documented as implemented (was Phase 2)
- DATA_DIR added to configuration table
- Docker instructions updated with data volume mount
- Data model diagram updated
- TODO updated (database separation moved to completed)

Closes #15
2026-03-01 17:06:43 -08:00
clawbot
6c393ccb78 fix: database target writes to dedicated archive table
All checks were successful
check / check (push) Successful in 1m43s
The "database" target type now writes events to a separate
archived_events table instead of just marking the delivery as done.
This table persists independently of internal event retention/pruning,
allowing the data to be consumed by external systems or preserved
indefinitely.

New ArchivedEvent model copies the full event payload (method, headers,
body, content_type) along with webhook/entrypoint/event/target IDs.
2026-03-01 16:40:27 -08:00
clawbot
418d3da97e fix: remove spurious config load log message (closes #27)
When no config.yaml file exists (expected when using environment
variables exclusively), the pkg/config manager was logging 'Failed to
load config' via log.Printf, which is confusing during normal operation.
Suppress these messages since missing config file is a valid state.
2026-03-01 16:39:26 -08:00
clawbot
7bac22bdfd fix: don't log admin password via slog (closes #26)
Replace slog.Info (which outputs structured JSON in prod and ends up in
log aggregation) with a plain fmt.Fprintf to stderr. The password is
printed once on first startup in a clearly-delimited banner that won't
be parsed as a structured log field.
2026-03-01 16:38:38 -08:00
clawbot
f21a007a3c feat: add entrypoint/target management controls (closes #25)
Add toggle (activate/deactivate) and delete buttons for individual
entrypoints and targets on the webhook detail page. Each action is a
POST form submission with ownership verification.

New routes:
  POST /source/{id}/entrypoints/{entrypointID}/delete
  POST /source/{id}/entrypoints/{entrypointID}/toggle
  POST /source/{id}/targets/{targetID}/delete
  POST /source/{id}/targets/{targetID}/toggle
2026-03-01 16:38:14 -08:00
clawbot
2606d41c60 fix: cascade soft-delete for webhook deletion (closes #24)
When deleting a webhook, also soft-delete all related deliveries and
delivery results (not just entrypoints, targets, and events). Query
event IDs, then delivery IDs, then cascade delete delivery results,
deliveries, events, entrypoints, targets, and finally the webhook
itself — all within a single transaction.
2026-03-01 16:37:21 -08:00
clawbot
45228d9e99 fix: restrict CORS to same-origin (closes #23)
In dev mode, keep the wildcard origin for local testing convenience.
In production, skip CORS headers entirely since the web UI is
server-rendered and cross-origin requests are not expected.
2026-03-01 16:36:56 -08:00
clawbot
348fd81fe6 fix: remove dead DevAdminUsername/Password config (closes #22)
Remove DevAdminUsername and DevAdminPassword fields from the Config
struct and their loading code. These fields were never referenced
anywhere else in the codebase.
2026-03-01 16:36:36 -08:00
clawbot
36824046fb fix: remove double cleanShutdown call (closes #21)
The serve() method called cleanShutdown() after ctx.Done(), and the fx
OnStop hook also called cleanShutdown(). Remove the call in serve() so
shutdown happens exactly once via the fx lifecycle.
2026-03-01 16:35:55 -08:00
clawbot
e2ac30287b fix: restrict webhook endpoint to POST only (closes #20)
Add method check at the top of HandleWebhook, returning 405 Method Not
Allowed with an Allow: POST header for any non-POST request. This
prevents GET, PUT, DELETE, etc. from being accepted at entrypoint URLs.
2026-03-01 16:35:38 -08:00
clawbot
49ab1a6147 fix: DevSessionKey wrong length (closes #19)
Replace the old 35-byte dev session key with a proper randomly-generated
32-byte key. Also ensure dev mode actually falls back to DevSessionKey
when SESSION_KEY is not set in the environment, rather than leaving
SessionKey empty and failing at session creation.

Update tests to remove the old key references.
2026-03-01 16:35:16 -08:00
clawbot
d65480c5ec fix: template rendering returns empty pages (closes #18)
Reorder template.ParseFS arguments so the page template file is listed
first. Go's template package names the template set after the first file
parsed. When htmlheader.html was first, its content (entirely a
{{define}} block) became the root template, which is empty. By putting
the page file first, its {{template "base" .}} invocation becomes the
root action and the page renders correctly.
2026-03-01 16:34:33 -08:00
clawbot
d4fbd6c110 fix: delivery engine nil pointer crash on startup (closes #17)
Store the *database.Database wrapper instead of calling .DB() eagerly
at construction time. The GORM *gorm.DB is only available after the
database's OnStart hook runs, but the engine constructor runs during
fx resolution (before OnStart). Accessing .DB() lazily via the wrapper
avoids the nil pointer panic.
2026-03-01 16:34:16 -08:00
clawbot
7f8469a0f2 feat: implement core webhook engine, delivery system, and management UI (Phase 2)
All checks were successful
check / check (push) Successful in 1m49s
- Webhook reception handler: look up entrypoint by UUID, verify active,
  capture full HTTP request (method, headers, body, content-type), create
  Event record, queue Delivery records for each active Target, return 200 OK.
  Handles edge cases: unknown UUID → 404, inactive → 410, oversized → 413.

- Delivery engine (internal/delivery): fx-managed background goroutine that
  polls for pending/retrying deliveries and dispatches to target type handlers.
  Graceful shutdown via context cancellation.

- Target type implementations:
  - HTTP: fire-and-forget POST with original headers forwarding
  - Retry: exponential backoff (1s, 2s, 4s...) up to max_retries
  - Database: immediate success (event already stored)
  - Log: slog output with event details

- Webhook management pages with Tailwind CSS + Alpine.js:
  - List (/sources): webhooks with entrypoint/target/event counts
  - Create (/sources/new): form with auto-created default entrypoint
  - Detail (/source/{id}): config, entrypoints, targets, recent events
  - Edit (/source/{id}/edit): name, description, retention_days
  - Delete (/source/{id}/delete): soft-delete with child records
  - Add Entrypoint (/source/{id}/entrypoints): inline form
  - Add Target (/source/{id}/targets): type-aware form
  - Event Log (/source/{id}/logs): paginated with delivery status

- Updated README: marked completed items, updated naming conventions
  table, added delivery engine to package layout and DI docs, updated
  column names to reflect entity rename.

- Rebuilt Tailwind CSS for new template classes.

Part of: #15
2026-03-01 16:14:28 -08:00
112 changed files with 14858 additions and 3617 deletions

View File

@@ -12,4 +12,4 @@ jobs:
- name: Checkout - name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 2024-10-23 uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 2024-10-23
- name: Build Docker image (runs make check) - name: Build Docker image (runs make check)
run: docker build . run: script/cibuild

4
.gitignore vendored
View File

@@ -29,9 +29,9 @@ Thumbs.db
# Environment and config files # Environment and config files
.env .env
.env.local .env.local
config.yaml
# Database files # Data directory (SQLite databases)
data/
*.db *.db
*.sqlite *.sqlite
*.sqlite3 *.sqlite3

View File

@@ -1,46 +1,32 @@
version: "2"
run: run:
timeout: 5m timeout: 5m
tests: true modules-download-mode: readonly
linters: linters:
enable: default: all
- gofmt disable:
- revive # Genuinely incompatible with project patterns
- govet - exhaustruct # Requires all struct fields
- errcheck - depguard # Dependency allow/block lists
- staticcheck - godot # Requires comments to end with periods
- unused - wsl # Deprecated, replaced by wsl_v5
- gosimple - wrapcheck # Too verbose for internal packages
- ineffassign - varnamelen # Short names like db, id are idiomatic Go
- typecheck
- gosec
- misspell
- unparam
- prealloc
- copyloopvar
- gocritic
- gochecknoinits
- gochecknoglobals
linters-settings: linters-settings:
gofmt: lll:
simplify: true line-length: 88
revive: funlen:
confidence: 0.8 lines: 80
govet: statements: 50
enable: cyclop:
- shadow max-complexity: 15
errcheck: dupl:
check-type-assertions: true threshold: 100
check-blank: true
issues: issues:
exclude-rules: exclude-use-default: false
# Exclude globals check for version variables in main max-issues-per-linter: 0
- path: cmd/webhooker/main.go max-same-issues: 0
linters:
- gochecknoglobals
# Exclude globals check for version variables in globals package
- path: internal/globals/globals.go
linters:
- gochecknoglobals

View File

@@ -1,50 +1,58 @@
# golang:1.24 (bookworm) — 2026-03-01 # Lint stage
# Using Debian-based image because gorm.io/driver/sqlite pulls in # golangci/golangci-lint:v2.11.3 (Debian-based), 2026-03-17
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl. # Using Debian-based image because mattn/go-sqlite3 (CGO) does not
FROM golang@sha256:d2d2bc1c84f7e60d7d2438a3836ae7d0c847f4888464e7ec9ba3a1339a1ee804 AS builder # compile on Alpine musl (off64_t is a glibc type).
FROM golangci/golangci-lint:v2.11.3@sha256:e838e8ab68aaefe83e2408691510867ade9329c0e0b895a3fb35eb93d1c2a4ba AS lint
# gcc is pre-installed in the Debian-based golang image
RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/* RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/*
WORKDIR /build WORKDIR /src
# Install golangci-lint v1.64.8 — 2026-03-01 # Copy go mod files first for better layer caching
# Using v1.x because the repo's .golangci.yml uses v1 config format.
RUN set -eux; \
GOLANGCI_VERSION="1.64.8"; \
ARCH="$(uname -m)"; \
case "${ARCH}" in \
x86_64) \
GOARCH="amd64"; \
GOLANGCI_SHA256="b6270687afb143d019f387c791cd2a6f1cb383be9b3124d241ca11bd3ce2e54e"; \
;; \
aarch64) \
GOARCH="arm64"; \
GOLANGCI_SHA256="a6ab58ebcb1c48572622146cdaec2956f56871038a54ed1149f1386e287789a5"; \
;; \
*) echo "unsupported architecture: ${ARCH}" && exit 1 ;; \
esac; \
wget -q "https://github.com/golangci/golangci-lint/releases/download/v${GOLANGCI_VERSION}/golangci-lint-${GOLANGCI_VERSION}-linux-${GOARCH}.tar.gz" \
-O /tmp/golangci-lint.tar.gz; \
echo "${GOLANGCI_SHA256} /tmp/golangci-lint.tar.gz" | sha256sum -c -; \
tar -xzf /tmp/golangci-lint.tar.gz -C /tmp; \
mv "/tmp/golangci-lint-${GOLANGCI_VERSION}-linux-${GOARCH}/golangci-lint" /usr/local/bin/; \
rm -rf /tmp/golangci-lint*; \
golangci-lint --version
# Copy go module files and download dependencies
COPY go.mod go.sum ./ COPY go.mod go.sum ./
COPY pkg/config/go.mod pkg/config/go.sum ./pkg/config/
RUN go mod download RUN go mod download
# Copy source code # Copy source code
COPY . . COPY . .
# Run all checks (fmt-check, lint, test, build) # Run formatting check and linter
RUN make check RUN make fmt-check
RUN make lint
# alpine:3.21 — 2026-03-01 # Build stage
FROM alpine@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709 # golang:1.26.1-bookworm (Debian-based), 2026-03-17
# Using Debian-based image because gorm.io/driver/sqlite pulls in
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl.
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder
# Depend on lint stage passing
COPY --from=lint /src/go.sum /dev/null
RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/*
WORKDIR /build
# Copy go mod files first for better layer caching
COPY go.mod go.sum ./
RUN go mod download
# Copy source code
COPY . .
# Run tests and build
RUN make test
RUN make build
# Rebuild with static linking for Alpine runtime.
# make build already verified compilation.
# The CGO binary from `make build` is dynamically linked against glibc,
# which doesn't exist on Alpine (musl). Rebuild with static linking so
# the binary runs on Alpine without glibc.
RUN CGO_ENABLED=1 go build -ldflags '-extldflags "-static"' -o bin/webhooker ./cmd/webhooker
# Runtime stage
# alpine:3.21, 2026-03-17
FROM alpine:3.21@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709
RUN apk --no-cache add ca-certificates RUN apk --no-cache add ca-certificates
@@ -55,9 +63,13 @@ RUN addgroup -g 1000 -S webhooker && \
WORKDIR /app WORKDIR /app
# Copy binary from builder # Copy binary from builder
COPY --from=builder /build/bin/webhooker . COPY --from=builder /build/bin/webhooker /app/webhooker
RUN chown -R webhooker:webhooker /app # Create data directory for all SQLite databases (main app DB +
# per-webhook event DBs). DATA_DIR defaults to /var/lib/webhooker.
RUN mkdir -p /var/lib/webhooker
RUN chown -R webhooker:webhooker /app /var/lib/webhooker
USER webhooker USER webhooker
@@ -66,4 +78,4 @@ EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:8080/.well-known/healthcheck || exit 1 CMD wget --no-verbose --tries=1 --spider http://localhost:8080/.well-known/healthcheck || exit 1
CMD ["./webhooker"] CMD ["/app/webhooker"]

View File

@@ -1,22 +1,28 @@
.PHONY: test lint fmt fmt-check check build run dev deps docker clean hooks css .PHONY: bootstrap setup test lint fmt fmt-check check build run dev deps docker clean hooks css
# Default target # Default target
.DEFAULT_GOAL := check .DEFAULT_GOAL := check
bootstrap:
@script/bootstrap
setup:
@script/setup
test: test:
go test -v -race -timeout 30s ./... @script/test
lint: lint:
golangci-lint run --config .golangci.yml ./... @script/lint
fmt: fmt:
gofmt -s -w . @script/fmt
@command -v goimports >/dev/null 2>&1 && goimports -w . || true
fmt-check: fmt-check:
@test -z "$$(gofmt -s -l .)" || { echo "gofmt needed on:"; gofmt -s -l .; exit 1; } @script/fmt-check
check: fmt-check lint test build check:
@script/check
build: build:
go build -o bin/webhooker ./cmd/webhooker go build -o bin/webhooker ./cmd/webhooker
@@ -32,15 +38,13 @@ deps:
go mod tidy go mod tidy
docker: docker:
docker build -t webhooker:latest . @script/docker
clean: clean:
rm -rf bin/ rm -rf bin/
hooks: hooks:
@printf '#!/bin/sh\nmake check\n' > .git/hooks/pre-commit @script/install-precommit
@chmod +x .git/hooks/pre-commit
@echo "pre-commit hook installed"
css: css:
tailwindcss -i static/css/input.css -o static/css/tailwind.css --minify tailwindcss -i static/css/input.css -o static/css/tailwind.css --minify

561
README.md
View File

@@ -11,8 +11,8 @@ with retry support, logging, and observability. Category: infrastructure
### Prerequisites ### Prerequisites
- Go 1.24+ - Go 1.26+
- golangci-lint v1.64+ - golangci-lint v2.11+
- Docker (for containerized deployment) - Docker (for containerized deployment)
### Quick Start ### Quick Start
@@ -38,42 +38,67 @@ make docker
### Development Commands ### Development Commands
```bash ```bash
make bootstrap # Install all dependencies (idempotent)
make setup # Bootstrap + install git pre-commit hook
make fmt # Format code (gofmt + goimports) make fmt # Format code (gofmt + goimports)
make lint # Run golangci-lint make lint # Run golangci-lint
make test # Run tests with race detection make test # Run tests with race detection
make check # fmt-check + lint + test + build (CI gate) make check # test + lint + fmt-check (CI gate)
make build # Build binary to bin/webhooker make build # Build binary to bin/webhooker
make dev # go run ./cmd/webhooker make dev # go run ./cmd/webhooker
make docker # Build Docker image make docker # Build Docker image
make hooks # Install git pre-commit hook that runs make check make hooks # Install git pre-commit hook that runs script/precommit
``` ```
### Configuration ### Configuration
webhooker uses a YAML configuration file with environment-specific All configuration is via environment variables. For local development,
overrides, loaded via the `pkg/config` library (Viper-based). The you can place variables in a `.env` file in the project root (loaded
environment is selected by setting `WEBHOOKER_ENVIRONMENT` to `dev` or automatically via `godotenv/autoload`).
`prod` (default: `dev`).
Configuration is resolved in this order (highest priority first): The environment is selected by setting `WEBHOOKER_ENVIRONMENT` to `dev`
or `prod` (default: `dev`). The setting controls several behaviors:
1. Environment variables | Behavior | `dev` | `prod` |
2. `.env` file (loaded via `godotenv/autoload`) | --------------------- | -------------------------------- | ------------------------------- |
3. Config file values for the active environment | CORS | Allows any origin (`*`) | Disabled (no-op) |
4. Config file defaults | Session cookie Secure | `false` (works over plain HTTP) | `true` (requires HTTPS) |
The CSRF cookie's `Secure` flag and Origin/Referer validation mode are
determined per-request based on the actual transport protocol, not the
environment setting. The middleware checks `r.TLS` (direct TLS) and the
`X-Forwarded-Proto` header (TLS-terminating reverse proxy) to decide:
- **Direct TLS or `X-Forwarded-Proto: https`**: Secure cookies, strict
Origin/Referer validation.
- **Plaintext HTTP**: Non-Secure cookies, relaxed Origin/Referer
checks (token validation still enforced).
This means CSRF protection works correctly in all deployment scenarios:
behind a TLS-terminating reverse proxy, with direct TLS, or over plain
HTTP during development. When running behind a reverse proxy, ensure it
sets the `X-Forwarded-Proto: https` header.
All other differences (log format, security headers, etc.) are
independent of the environment setting — log format is determined by
TTY detection, and security headers are always applied.
| Variable | Description | Default | | Variable | Description | Default |
| ----------------------- | ----------------------------------- | -------- | | ----------------------- | ----------------------------------- | -------- |
| `WEBHOOKER_ENVIRONMENT` | `dev` or `prod` | `dev` | | `WEBHOOKER_ENVIRONMENT` | `dev` or `prod` | `dev` |
| `PORT` | HTTP listen port | `8080` | | `PORT` | HTTP listen port | `8080` |
| `DBURL` | SQLite database connection string | *(required)* | | `DATA_DIR` | Directory for all SQLite databases | `/var/lib/webhooker` |
| `SESSION_KEY` | Base64-encoded 32-byte session key | *(required in prod)* |
| `DEBUG` | Enable debug logging | `false` | | `DEBUG` | Enable debug logging | `false` |
| `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` | | `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` |
| `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` | | `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` |
| `SENTRY_DSN` | Sentry error reporting DSN | `""` | | `SENTRY_DSN` | Sentry error reporting DSN | `""` |
| `RECEIVER_RATE_LIMIT` | Receiver requests/minute per IP per entrypoint | `120` |
On first startup in development mode, webhooker creates an `admin` user On first startup, webhooker automatically generates a cryptographically
secure session encryption key and stores it in the database. This key
persists across restarts — no manual key management is needed.
On first startup, webhooker creates an `admin` user
with a randomly generated password and logs it to stdout. This password with a randomly generated password and logs it to stdout. This password
is only displayed once. is only displayed once.
@@ -82,16 +107,42 @@ is only displayed once.
```bash ```bash
docker run -d \ docker run -d \
-p 8080:8080 \ -p 8080:8080 \
-v /path/to/data:/data \ -v /path/to/data:/var/lib/webhooker \
-e DBURL="file:/data/webhooker.db?cache=shared&mode=rwc" \
-e SESSION_KEY="<base64-encoded-32-byte-key>" \
-e WEBHOOKER_ENVIRONMENT=prod \ -e WEBHOOKER_ENVIRONMENT=prod \
webhooker:latest webhooker:latest
``` ```
The container runs as a non-root user (`webhooker`, UID 1000), exposes The container runs as a non-root user (`webhooker`, UID 1000), exposes
port 8080, and includes a health check against port 8080, and includes a health check against
`/.well-known/healthcheck`. `/.well-known/healthcheck`. The `/var/lib/webhooker` volume holds all
SQLite databases: the main application database (`webhooker.db`) and
the per-webhook event databases (`events-{uuid}.db`). Mount this as a
persistent volume to preserve data across container restarts.
## Entrypoints
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call
them. We provide:
- `script/bootstrap` — install all dependencies (idempotent)
- `script/setup` — make a fresh clone ready for development
(bootstrap, then install-precommit)
- `script/projectname` — output the project name ("webhooker")
- `script/test` — run the test suite
- `script/lint` — run golangci-lint
- `script/fmt` — format all code (writes)
- `script/fmt-check` — check formatting (read-only)
- `script/check` — run test, lint, and fmt-check
- `script/docker` — build the Docker image tagged via `script/projectname`
- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile
runs the checks, so a green build implies a green repo)
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
`script/check`)
- `script/install-precommit` — install the git pre-commit hook that
runs `script/precommit`
## Rationale ## Rationale
@@ -158,25 +209,24 @@ It uses:
logging with TTY detection (text for dev, JSON for prod) logging with TTY detection (text for dev, JSON for prod)
- **[gorilla/sessions](https://github.com/gorilla/sessions)** for - **[gorilla/sessions](https://github.com/gorilla/sessions)** for
encrypted cookie-based session management encrypted cookie-based session management
- **[gorilla/csrf](https://github.com/gorilla/csrf)** for CSRF
protection (cookie-based double-submit tokens)
- **[go-chi/httprate](https://github.com/go-chi/httprate)** for
per-IP login rate limiting (sliding window counter)
- **[Prometheus](https://prometheus.io)** for metrics, served at - **[Prometheus](https://prometheus.io)** for metrics, served at
`/metrics` behind basic auth `/metrics` behind basic auth
- **[Sentry](https://sentry.io)** for optional error reporting - **[Sentry](https://sentry.io)** for optional error reporting
### Naming Conventions ### Naming Conventions
This README uses the target naming scheme for the application's core The codebase uses consistent naming throughout (rename completed in
entities. The current codebase uses older names that will be updated in
a future refactor (see
[issue #12](https://git.eeqj.de/sneak/webhooker/issues/12)): [issue #12](https://git.eeqj.de/sneak/webhooker/issues/12)):
| README (target name) | Current code name | Description | | Entity | Description |
| --------------------- | ----------------- | ----------- | | ---------------- | ----------- |
| **Webhook** | `Processor` | Top-level configuration entity grouping entrypoints and targets | | **Webhook** | Top-level configuration entity grouping entrypoints and targets |
| **Entrypoint** | `Webhook` | A receiver URL where external services POST events | | **Entrypoint** | A receiver URL where external services POST events |
| **Target** | `Target` | A delivery destination for events | | **Target** | A delivery destination for events |
Throughout this document, the target names are used. The code rename is
tracked separately.
### Data Model ### Data Model
@@ -196,11 +246,15 @@ tier** (event ingestion, delivery, and logging).
│ │ │ └──────────┘ └──────────────┘ │ │ │ │ └──────────┘ └──────────────┘ │
│ │ │──1:N──│ APIKey │ │ │ │ │──1:N──│ APIKey │ │
│ └──────────┘ └──────────┘ │ │ └──────────┘ └──────────┘ │
│ │
│ ┌──────────┐ │
│ │ Setting │ (key-value application config) │
│ └──────────┘ │
└─────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────────────┐
│ EVENT TIER │ │ EVENT TIER │
(planned: per-webhook dedicated database) (per-webhook dedicated databases)
│ │ │ │
│ ┌──────────┐ ┌──────────┐ ┌─────────────────┐ │ │ ┌──────────┐ ┌──────────┐ ┌─────────────────┐ │
│ │ Event │──1:N──│ Delivery │──1:N──│ DeliveryResult │ │ │ │ Event │──1:N──│ Delivery │──1:N──│ DeliveryResult │ │
@@ -208,6 +262,22 @@ tier** (event ingestion, delivery, and logging).
└─────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────┘
``` ```
#### Setting
A key-value pair for application-level configuration that is
auto-managed rather than user-provided. Used to store the session
encryption key and any future auto-generated settings.
| Field | Type | Description |
| ------- | ------ | ----------- |
| `key` | string | Primary key (setting name) |
| `value` | text | Setting value |
Currently stored settings:
- **`session_key`** — Base64-encoded 32-byte session encryption key,
auto-generated on first startup.
#### User #### User
A registered user of the webhooker service. A registered user of the webhooker service.
@@ -227,10 +297,10 @@ password logged to stdout.
#### Webhook #### Webhook
The top-level configuration entity (currently called "Processor" in The top-level configuration entity. A webhook groups together one or
code). A webhook groups together one or more entrypoints (receiver URLs) more entrypoints (receiver URLs) and one or more targets (delivery
and one or more targets (delivery destinations) into a logical unit. A destinations) into a logical unit. A user creates a webhook to set up
user creates a webhook to set up event routing. event routing.
| Field | Type | Description | | Field | Type | Description |
| ---------------- | ------- | ----------- | | ---------------- | ------- | ----------- |
@@ -247,15 +317,15 @@ webhook's dedicated database before automatic cleanup.
#### Entrypoint #### Entrypoint
A receiver URL where external services POST webhook events (currently A receiver URL where external services POST webhook events. Each
called "Webhook" in code). Each entrypoint has a unique UUID-based path. entrypoint has a unique UUID-based path.
When an HTTP request arrives at an entrypoint's path, webhooker captures When an HTTP request arrives at an entrypoint's path, webhooker captures
the full request and creates an Event. the full request and creates an Event.
| Field | Type | Description | | Field | Type | Description |
| -------------- | ------- | ----------- | | -------------- | ------- | ----------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `path` | string | Unique URL path (UUID-based, e.g. `/webhook/{uuid}`) | | `path` | string | Unique URL path (UUID-based, e.g. `/webhook/{uuid}`) |
| `description` | string | Optional description | | `description` | string | Optional description |
| `active` | boolean | Whether this entrypoint accepts events (default: true) | | `active` | boolean | Whether this entrypoint accepts events (default: true) |
@@ -275,24 +345,29 @@ events should be forwarded.
| Field | Type | Description | | Field | Type | Description |
| ---------------- | ---------- | ----------- | | ---------------- | ---------- | ----------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `name` | string | Human-readable name | | `name` | string | Human-readable name |
| `type` | TargetType | One of: `http`, `retry`, `database`, `log` | | `type` | TargetType | One of: `http`, `slack`, `database`, `log` |
| `active` | boolean | Whether deliveries are enabled (default: true) | | `active` | boolean | Whether deliveries are enabled (default: true) |
| `config` | JSON text | Type-specific configuration | | `config` | JSON text | Type-specific configuration |
| `max_retries` | integer | Maximum retry attempts (for retry targets) | | `max_retries` | integer | Maximum retry attempts for HTTP targets (0 = fire-and-forget, >0 = retries with backoff) |
| `max_queue_size` | integer | Maximum queued deliveries (for retry targets) | | `max_queue_size` | integer | Maximum queued deliveries (for HTTP targets with retries) |
**Relations:** Belongs to Webhook. Has many Deliveries. **Relations:** Belongs to Webhook. Has many Deliveries.
**Target types:** **Target types:**
- **`http`** — Forward the event as an HTTP POST to a configured URL. - **`http`** — Forward the event as an HTTP POST to a configured URL.
Fire-and-forget: a single attempt with no retries. Behavior depends on `max_retries`: when `max_retries` is 0 (the
- **`retry`** — Forward the event via HTTP POST with automatic retry on default), the target operates in fire-and-forget mode — a single
failure. Uses exponential backoff up to `max_retries` attempts. attempt with no retries and no circuit breaker. When `max_retries` is
- **`database`** — Store the event in the webhook's database only (no greater than 0, failed deliveries are retried with exponential backoff
external delivery). Useful for pure logging/archival. up to `max_retries` attempts, protected by a per-target circuit
breaker.
- **`database`** — Confirm the event is stored in the webhook's
per-webhook database (no external delivery). Since events are always
written to the per-webhook DB on ingestion, this target marks delivery
as immediately successful. Useful for ensuring durable event archival.
- **`log`** — Write the event to the application log (stdout). Useful - **`log`** — Write the event to the application log (stdout). Useful
for debugging. for debugging.
@@ -321,8 +396,8 @@ data for replay and auditing.
| Field | Type | Description | | Field | Type | Description |
| -------------- | ------ | ----------- | | -------------- | ------ | ----------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `webhook_id` | UUID | Foreign key → Entrypoint | | `entrypoint_id` | UUID | Foreign key → Entrypoint |
| `method` | string | HTTP method (POST, PUT, etc.) | | `method` | string | HTTP method (POST, PUT, etc.) |
| `headers` | JSON | Complete request headers | | `headers` | JSON | Complete request headers |
| `body` | text | Raw request body | | `body` | text | Raw request body |
@@ -389,36 +464,39 @@ All entities include these fields from `BaseModel`:
### Database Architecture ### Database Architecture
#### Current Implementation #### Per-Webhook Event Databases
webhooker currently uses a **single SQLite database** for all data — webhooker uses **separate SQLite database files**: a main application
application configuration, user accounts, and (once implemented) event database for configuration data and per-webhook databases for event
storage. The database connection is managed by GORM with a single storage. All database files live in the `DATA_DIR` directory.
connection string configured via `DBURL`. On first startup the database
is auto-migrated and an `admin` user is created.
#### Planned: Per-Webhook Event Databases (Phase 2) **Main Application Database** (`{DATA_DIR}/webhooker.db`) — stores
configuration and application state:
In a future phase (see TODO Phase 2 below), webhooker will split into
**separate SQLite database files**: a main application database for
configuration data and per-webhook databases for event storage.
**Main Application Database** — will store:
- **Settings** — auto-managed key-value config (e.g. session encryption
key)
- **Users** — accounts and Argon2id password hashes - **Users** — accounts and Argon2id password hashes
- **Webhooks** (Processors) — webhook configurations - **Webhooks** — webhook configurations
- **Entrypoints** (Webhooks) — receiver URL definitions - **Entrypoints** — receiver URL definitions
- **Targets** — delivery destination configurations - **Targets** — delivery destination configurations
- **APIKeys** — programmatic access credentials - **APIKeys** — programmatic access credentials
**Per-Webhook Event Databases** — each webhook will get its own On first startup the main database is auto-migrated, a session
dedicated SQLite file containing: encryption key is generated and stored, and an `admin` user is created.
**Per-Webhook Event Databases** (`{DATA_DIR}/events-{webhook_uuid}.db`)
— each webhook gets its own dedicated SQLite file containing:
- **Events** — captured incoming webhook payloads - **Events** — captured incoming webhook payloads
- **Deliveries** — event-to-target pairings and their status - **Deliveries** — event-to-target pairings and their status
- **DeliveryResults** — individual delivery attempt logs - **DeliveryResults** — individual delivery attempt logs
This planned separation will provide: Per-webhook databases are created automatically when a webhook is
created (and lazily on first access for webhooks that predate this
feature). They are managed by the `WebhookDBManager` component, which
handles connection pooling, lazy opening, migrations, and cleanup.
This separation provides:
- **Isolation** — a high-volume webhook won't cause lock contention or - **Isolation** — a high-volume webhook won't cause lock contention or
WAL bloat affecting the main application or other webhooks. WAL bloat affecting the main application or other webhooks.
@@ -426,14 +504,31 @@ This planned separation will provide:
backed up, archived, rotated, or size-limited without impacting the backed up, archived, rotated, or size-limited without impacting the
application. application.
- **Clean deletion** — removing a webhook and all its history is as - **Clean deletion** — removing a webhook and all its history is as
simple as deleting one file. simple as deleting one file. Configuration is soft-deleted in the main
DB; the event database file is hard-deleted (permanently removed).
- **Per-webhook retention** — the `retention_days` field on each webhook - **Per-webhook retention** — the `retention_days` field on each webhook
will control automatic cleanup of old events in that webhook's controls automatic cleanup of old events in that webhook's database
database only. only.
- **Performance** — each webhook's database will have its own WAL, its - **Performance** — each webhook's database has its own WAL, its own
own page cache, and its own lock, so concurrent event ingestion across page cache, and its own lock, so concurrent event ingestion across
webhooks won't contend. webhooks won't contend.
The **database target type** leverages this architecture: since events
are already stored in the per-webhook database by design, the database
target simply marks the delivery as immediately successful. The
per-webhook DB IS the dedicated event database — that's the whole point
of the database target type.
The **Slack target type** sends webhook events as formatted messages to
any Slack-compatible incoming webhook URL (works with Slack, Mattermost,
and other compatible services). Each message includes event metadata
(HTTP method, content type, timestamp, body size) and the payload
pretty-printed in a code block. JSON payloads are automatically
formatted with indentation for readability; non-JSON payloads are shown
as raw text. Large payloads are truncated to keep messages reasonable.
Config stores `webhook_url` — the Slack/Mattermost incoming webhook
endpoint.
The database uses the The database uses the
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at [modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at
runtime, though CGO is required at build time due to the transitive runtime, though CGO is required at build time due to the transitive
@@ -453,35 +548,153 @@ External Service
1. Look up Entrypoint by UUID 1. Look up Entrypoint by UUID
2. Capture full request as Event 2. Capture full request as Event
3. Queue Delivery to each active Target 3. Create Delivery records for each active Target
4. Build self-contained DeliveryTask structs
(target config + event data inline for ≤16KB)
5. Notify Engine via channel (no DB read needed)
┌──────────────┐ ┌──────────────┐
│ Delivery │ │ Delivery │◄── retry timers
│ Engine │ │ Engine │ (backoff)
│ (worker │
│ pool) │
└──────┬───────┘ └──────┬───────┘
┌────────────────────┼────────────────────┐ ┌── bounded worker pool (N workers) ──┐
▼ ▼ ▼ ▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐
│ HTTP Target│ │Retry Target│ │ Log Target │ │ HTTP Target│ │ HTTP Target│ │ Log Target │
│ (1 attempt)│ │ (backoff) │ │ (stdout) │ │(max_retries│ │(max_retries│ │ (stdout) │
└────────────┘ └────────────┘ └────────────┘ == 0) │ │ > 0, │ └────────────┘
│ fire+forget│ │ backoff + │
└────────────┘ │ circuit │
│ breaker) │
└────────────┘
``` ```
### Bounded Worker Pool
The delivery engine uses a **fixed-size worker pool** (default: 10
workers) to process all deliveries. At most N deliveries are in-flight
at any time, preventing goroutine explosions regardless of queue depth.
**Architecture:**
- **Channels as queues:** Two buffered channels serve as bounded queues:
a delivery channel (new tasks from the webhook handler) and a retry
channel (tasks from backoff timers). Both are buffered to 10,000.
- **Fan-out via channel, not goroutines:** When an event arrives with
multiple targets, each `DeliveryTask` is sent to the delivery channel.
Workers pick them up and process them — no goroutine-per-target.
- **Worker goroutines:** A fixed number of worker goroutines select from
both channels. Each worker processes one task at a time, then picks up
the next. Workers are the ONLY goroutines doing actual HTTP delivery.
- **Retry backpressure with DB fallback:** When a retry timer fires and
the retry channel is full, the timer is dropped — the delivery stays
in `retrying` status in the database. A periodic sweep (every 60s)
scans for these "orphaned" retries and re-queues them. No blocked
goroutines, no unbounded timer chains.
- **Bounded concurrency:** At most N deliveries (N = number of workers)
are in-flight simultaneously. Even if a circuit breaker is open for
hours and thousands of retries queue up in the channels, the workers
drain them at a controlled rate when the circuit closes.
This means:
- **No goroutine explosion** — even with 10,000 queued retries, only
N worker goroutines exist.
- **Natural backpressure** — if workers are busy, new tasks wait in the
channel buffer rather than spawning more goroutines.
- **Independent results** — each worker records its own delivery result
in the per-webhook database without coordination.
- **Graceful shutdown** — cancel the context, workers finish their
current task and exit. `WaitGroup.Wait()` ensures clean shutdown.
**Recovery paths:**
1. **Startup recovery:** When the engine starts, it scans all per-webhook
databases for `pending` and `retrying` deliveries. Pending deliveries
are sent to the delivery channel; retrying deliveries get backoff
timers scheduled.
2. **Periodic retry sweep (DB-mediated fallback):** Every 60 seconds the
engine scans for `retrying` deliveries whose backoff period has
elapsed. This catches "orphaned" retries — ones whose in-memory timer
was dropped because the retry channel was full. The database is the
durable fallback that ensures no retry is permanently lost, even under
extreme backpressure.
### Circuit Breaker (HTTP Targets with Retries)
HTTP targets with `max_retries` > 0 are protected by a **per-target circuit breaker** that
prevents hammering a down target with repeated failed delivery attempts.
The circuit breaker is in-memory only and resets on restart (which is
fine — startup recovery rescans the database anyway).
**States:**
| State | Behavior |
| ----------- | -------- |
| **Closed** | Normal operation. Deliveries flow through. Consecutive failures are counted. |
| **Open** | Target appears down. Deliveries are skipped and rescheduled for after the cooldown. |
| **Half-Open** | Cooldown expired. One probe delivery is allowed to test if the target has recovered. |
**Transitions:**
```
success ┌──────────┐
┌────────────────────► │ Closed │ ◄─── probe succeeds
│ │ (normal) │
│ └────┬─────┘
│ │ N consecutive failures
│ ▼
│ ┌──────────┐
│ │ Open │ ◄─── probe fails
│ │(tripped) │
│ └────┬─────┘
│ │ cooldown expires
│ ▼
│ ┌──────────┐
└──────────────────────│Half-Open │
│ (probe) │
└──────────┘
```
**Defaults:**
- **Failure threshold:** 5 consecutive failures before opening
- **Cooldown:** 30 seconds in open state before probing
**Scope:** Circuit breakers only apply to **HTTP targets with
`max_retries` > 0**. Fire-and-forget HTTP targets (`max_retries` == 0),
Slack targets, database targets (local operations), and log
targets (stdout) do not use circuit breakers.
When a circuit is open and a new delivery arrives, the engine marks the
delivery as `retrying` and schedules a retry timer for after the
remaining cooldown period. This ensures no deliveries are lost — they're
just delayed until the target is healthy again.
### Rate Limiting ### Rate Limiting
Global rate limiting middleware (e.g., per-IP throttling applied at the Global blanket rate limiting middleware (e.g., a per-IP throttle shared
router level) **must not** apply to webhook receiver endpoints. Webhook with the web UI) **must not** apply to webhook receiver endpoints.
endpoints receive automated traffic from external services at Webhook endpoints receive automated traffic from external services at
unpredictable rates, and blanket rate limits would cause legitimate unpredictable rates, and blanket limits shared with other routes would
deliveries to be dropped. cause legitimate deliveries to be dropped.
Instead, each webhook has its own individually configurable rate limit, The receiver instead has its own dedicated abuse limit, scoped to the
applied within the webhook handler itself. By default, no rate limit is `/webhook/{uuid}` route only and keyed per client IP per entrypoint: one
applied — webhook endpoints accept traffic as fast as it arrives. Rate misbehaving sender is throttled without affecting other senders of the
limits can be configured per-webhook when needed (e.g., to protect same entrypoint or the same sender's other entrypoints. The limit is
against a misbehaving sender). `RECEIVER_RATE_LIMIT` requests per minute (default 120, generous for
legitimate webhook senders). Requests over the limit receive HTTP 429
with a `Retry-After` header. A set-but-invalid `RECEIVER_RATE_LIMIT`
value aborts startup rather than silently falling back to the default.
Finer-grained per-webhook rate limits (configured in the web UI and
enforced in the webhook handler) can layer on top of this env-level
abuse limit later; they are tracked as future work.
### API Endpoints ### API Endpoints
@@ -489,7 +702,7 @@ against a misbehaving sender).
| Method | Path | Description | | Method | Path | Description |
| ------ | --------------------------- | ----------- | | ------ | --------------------------- | ----------- |
| `GET` | `/` | Web UI index page (server-rendered) | | `GET` | `/` | Root redirect (authenticated → `/sources`, unauthenticated → `/pages/login`) |
| `GET` | `/.well-known/healthcheck` | Health check (JSON: status, uptime, version) | | `GET` | `/.well-known/healthcheck` | Health check (JSON: status, uptime, version) |
| `GET` | `/s/*` | Static file serving (embedded CSS, JS) | | `GET` | `/s/*` | Static file serving (embedded CSS, JS) |
| `ANY` | `/webhook/{uuid}` | Webhook receiver endpoint (accepts all methods) | | `ANY` | `/webhook/{uuid}` | Webhook receiver endpoint (accepts all methods) |
@@ -515,6 +728,8 @@ against a misbehaving sender).
| `POST` | `/source/{id}/edit` | Edit webhook submission | | `POST` | `/source/{id}/edit` | Edit webhook submission |
| `POST` | `/source/{id}/delete` | Delete webhook | | `POST` | `/source/{id}/delete` | Delete webhook |
| `GET` | `/source/{id}/logs` | Webhook event logs | | `GET` | `/source/{id}/logs` | Webhook event logs |
| `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook |
| `POST` | `/source/{id}/targets` | Add target to webhook |
#### Infrastructure Endpoints #### Infrastructure Endpoints
@@ -548,51 +763,56 @@ webhooker/
│ └── main.go # Entry point: sets globals, wires fx │ └── main.go # Entry point: sets globals, wires fx
├── internal/ ├── internal/
│ ├── config/ │ ├── config/
│ │ └── config.go # Configuration loading via pkg/config │ │ └── config.go # Configuration loading from environment variables
│ ├── database/ │ ├── database/
│ │ ├── base_model.go # BaseModel with UUID primary keys │ │ ├── base_model.go # BaseModel with UUID primary keys
│ │ ├── database.go # GORM connection, migrations, admin seed │ │ ├── database.go # GORM connection, migrations, admin seed
│ │ ├── models.go # AutoMigrate for all models │ │ ├── models.go # AutoMigrate for config-tier models
│ │ ├── model_setting.go # Setting entity (key-value app config)
│ │ ├── model_user.go # User entity │ │ ├── model_user.go # User entity
│ │ ├── model_processor.go # Webhook entity (to be renamed) │ │ ├── model_webhook.go # Webhook entity
│ │ ├── model_webhook.go # Entrypoint entity (to be renamed) │ │ ├── model_entrypoint.go # Entrypoint entity
│ │ ├── model_target.go # Target entity and TargetType enum │ │ ├── model_target.go # Target entity and TargetType enum
│ │ ├── model_event.go # Event entity │ │ ├── model_event.go # Event entity (per-webhook DB)
│ │ ├── model_delivery.go # Delivery entity and DeliveryStatus enum │ │ ├── model_delivery.go # Delivery entity (per-webhook DB)
│ │ ├── model_delivery_result.go # DeliveryResult entity │ │ ├── model_delivery_result.go # DeliveryResult entity (per-webhook DB)
│ │ ├── model_apikey.go # APIKey entity │ │ ├── model_apikey.go # APIKey entity
│ │ ── password.go # Argon2id hashing and verification │ │ ── password.go # Argon2id hashing and verification
│ │ └── webhook_db_manager.go # Per-webhook DB lifecycle manager
│ ├── globals/ │ ├── globals/
│ │ └── globals.go # Build-time variables (appname, version, arch) │ │ └── globals.go # Build-time variables (appname, version, arch)
│ ├── delivery/
│ │ ├── engine.go # Event-driven delivery engine (channel + timer based)
│ │ ├── circuit_breaker.go # Per-target circuit breaker for HTTP targets with retries
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
│ ├── handlers/ │ ├── handlers/
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering │ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
│ │ ├── auth.go # Login, logout handlers │ │ ├── auth.go # Login, logout handlers
│ │ ├── healthcheck.go # Health check handler │ │ ├── healthcheck.go # Health check handler
│ │ ├── index.go # Index page handler │ │ ├── index.go # Index page handler
│ │ ├── profile.go # User profile handler │ │ ├── profile.go # User profile handler
│ │ ├── source_management.go # Webhook CRUD handlers (stubs) │ │ ├── source_management.go # Webhook CRUD handlers
│ │ └── webhook.go # Webhook receiver handler │ │ └── webhook.go # Webhook receiver handler
│ ├── healthcheck/ │ ├── healthcheck/
│ │ └── healthcheck.go # Health check service (uptime, version) │ │ └── healthcheck.go # Health check service (uptime, version)
│ ├── logger/ │ ├── logger/
│ │ └── logger.go # slog setup with TTY detection │ │ └── logger.go # slog setup with TTY detection
│ ├── middleware/ │ ├── middleware/
│ │ ── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth │ │ ── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth, SecurityHeaders, MaxBodySize
│ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf)
│ │ └── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate)
│ ├── server/ │ ├── server/
│ │ ├── server.go # Server struct, fx lifecycle, signal handling │ │ ├── server.go # Server struct, fx lifecycle, signal handling
│ │ ├── http.go # HTTP server setup with timeouts │ │ ├── http.go # HTTP server setup with timeouts
│ │ └── routes.go # All route definitions │ │ └── routes.go # All route definitions
│ └── session/ │ └── session/
│ └── session.go # Cookie-based session management │ └── session.go # Cookie-based session management
├── pkg/config/ # Reusable multi-environment config library
├── static/ ├── static/
│ ├── static.go # //go:embed directive │ ├── static.go # //go:embed directive
│ ├── css/style.css # Custom stylesheet (system font stack, card effects, layout) │ ├── css/style.css # Custom stylesheet (system font stack, card effects, layout)
│ └── js/app.js # Client-side JavaScript (minimal bootstrap) │ └── js/app.js # Client-side JavaScript (minimal bootstrap)
├── templates/ # Go HTML templates (base, index, login, etc.) ├── templates/ # Go HTML templates (base, index, login, etc.)
├── configs/ ├── Dockerfile # Multi-stage: lint, build+test, then Alpine runtime
│ └── config.yaml.example # Example configuration file
├── Dockerfile # Multi-stage: build + check, then Alpine runtime
├── Makefile # fmt, lint, test, check, build, docker targets ├── Makefile # fmt, lint, test, check, build, docker targets
├── go.mod / go.sum ├── go.mod / go.sum
└── .golangci.yml # Linter configuration └── .golangci.yml # Linter configuration
@@ -604,16 +824,26 @@ Components are wired via Uber fx in this order:
1. `globals.New` — Build-time variables (appname, version, arch) 1. `globals.New` — Build-time variables (appname, version, arch)
2. `logger.New` — Structured logging (slog with TTY detection) 2. `logger.New` — Structured logging (slog with TTY detection)
3. `config.New` — Configuration loading (pkg/config + environment) 3. `config.New` — Configuration loading (environment variables)
4. `database.New` — SQLite connection, migrations, admin user seed 4. `database.New` Main SQLite connection, config migrations, admin
5. `healthcheck.New` — Health check service user seed
6. `session.New` — Cookie-based session manager 5. `database.NewWebhookDBManager` — Per-webhook event database
7. `handlers.New` — HTTP handlers lifecycle manager
8. `middleware.New` — HTTP middleware 6. `healthcheck.New` — Health check service
9. `server.New`HTTP server and router 7. `session.New`Cookie-based session manager (key from database)
8. `handlers.New` — HTTP handlers
9. `middleware.New` — HTTP middleware
10. `delivery.New` — Event-driven delivery engine
11. `delivery.Engine``handlers.DeliveryNotifier` — interface bridge
12. `server.New` — HTTP server and router
The server starts via `fx.Invoke(func(*server.Server) {})` which The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine)
triggers the fx lifecycle hooks in dependency order. {})` which triggers the fx lifecycle hooks in dependency order. The
`DeliveryNotifier` interface allows the webhook handler to send
self-contained `DeliveryTask` slices to the engine without a direct
package dependency. Each task carries all target config and event data
inline (for bodies ≤16KB), so the engine can deliver without reading
from any database — it only writes to record results.
### Middleware Stack ### Middleware Stack
@@ -621,14 +851,21 @@ Applied to all routes in this order:
1. **Recoverer** — Panic recovery (chi built-in) 1. **Recoverer** — Panic recovery (chi built-in)
2. **RequestID** — Generate unique request IDs (chi built-in) 2. **RequestID** — Generate unique request IDs (chi built-in)
3. **Logging** — Structured request logging (method, URL, status, 3. **SecurityHeaders** — Production security headers on every response
(HSTS, X-Content-Type-Options, X-Frame-Options, CSP, Referrer-Policy,
Permissions-Policy)
4. **Logging** — Structured request logging (method, URL, status,
latency, remote IP, user agent, request ID) latency, remote IP, user agent, request ID)
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set) 5. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
5. **CORS** — Cross-origin resource sharing headers 6. **CORS** — Cross-origin resource sharing headers
6. **Timeout** — 60-second request timeout 7. **Timeout** — 60-second request timeout
7. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set; 8. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set;
configured with `Repanic: true` so panics still reach Recoverer) configured with `Repanic: true` so panics still reach Recoverer)
Additionally, form endpoints (`/pages`, `/sources`, `/source/*`) apply a
**MaxBodySize** middleware that limits POST/PUT/PATCH request bodies to
1 MB using `http.MaxBytesReader`, preventing oversized form submissions.
### Authentication ### Authentication
- **Web UI:** Cookie-based sessions using gorilla/sessions with - **Web UI:** Cookie-based sessions using gorilla/sessions with
@@ -643,7 +880,28 @@ Applied to all routes in this order:
- Passwords hashed with Argon2id (64 MB memory cost) - Passwords hashed with Argon2id (64 MB memory cost)
- Session cookies are HttpOnly, SameSite Lax, Secure (prod only) - Session cookies are HttpOnly, SameSite Lax, Secure (prod only)
- Session key must be a 32-byte base64-encoded value - Session regeneration on login to prevent session fixation attacks
- Session key is a 32-byte value auto-generated on first startup and
stored in the database
- Production security headers on all responses: HSTS, X-Content-Type-Options
(`nosniff`), X-Frame-Options (`DENY`), Content-Security-Policy, Referrer-Policy,
and Permissions-Policy
- Request body size limits (1 MB) on all form POST endpoints
- **CSRF protection** via [gorilla/csrf](https://github.com/gorilla/csrf)
on all state-changing forms (cookie-based double-submit tokens with
HMAC authentication). Applied to `/pages`, `/sources`, `/source`, and
`/user` routes. Excluded from `/webhook` (inbound webhook POSTs) and
`/api` (stateless API). The middleware auto-detects TLS status
per-request (via `r.TLS` and `X-Forwarded-Proto`) to set appropriate
cookie security flags and Origin/Referer validation mode
- **SSRF prevention** for HTTP delivery targets: private/reserved IP
ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked
both at target creation time (URL validation) and at delivery time
(custom HTTP transport with SSRF-safe dialer that validates resolved
IPs before connecting, preventing DNS rebinding attacks)
- **Login rate limiting** via [go-chi/httprate](https://github.com/go-chi/httprate):
per-IP sliding-window rate limiter on the login endpoint (5 POST
attempts per minute per IP) to prevent brute-force attacks
- Prometheus metrics behind basic auth - Prometheus metrics behind basic auth
- Static assets embedded in binary (no filesystem access needed at - Static assets embedded in binary (no filesystem access needed at
runtime) runtime)
@@ -657,7 +915,8 @@ The Dockerfile uses a multi-stage build:
1. **Builder stage** (Debian-based `golang:1.24`) — installs 1. **Builder stage** (Debian-based `golang:1.24`) — installs
golangci-lint, downloads dependencies, copies source, runs `make golangci-lint, downloads dependencies, copies source, runs `make
check` (format verification, linting, tests, compilation). check` (format verification, linting, tests, compilation).
2. **Runtime stage** (`alpine:3.21`) — copies the binary, runs as 2. **Runtime stage** (`alpine:3.21`) — copies the binary, creates the
`/var/lib/webhooker` directory for all SQLite databases, runs as
non-root user, exposes port 8080, includes a health check. non-root user, exposes port 8080, includes a health check.
The builder uses Debian rather than Alpine because GORM's SQLite The builder uses Debian rather than Alpine because GORM's SQLite
@@ -669,65 +928,7 @@ linted, tested, and compiled.
## TODO ## TODO
### Phase 1: Core Webhook Engine See [TODO.md](TODO.md).
- [ ] Implement webhook reception and event storage at `/webhook/{uuid}`
- [ ] Build event processing and target delivery engine
- [ ] Implement HTTP target type (fire-and-forget POST)
- [ ] Implement retry target type (exponential backoff)
- [ ] Implement database target type (store only)
- [ ] Implement log target type (console output)
- [ ] Per-webhook rate limiting in the receiver handler
- [ ] Webhook signature verification (GitHub, Stripe formats)
### Phase 2: Database Separation
- [ ] Split into main application DB + per-webhook event DBs
- [ ] Automatic event retention cleanup based on `retention_days`
- [ ] Per-webhook database lifecycle management (create on webhook
creation, delete on webhook removal)
### Phase 3: Security & Infrastructure
- [ ] Implement authentication middleware for protected routes
([#9](https://git.eeqj.de/sneak/webhooker/issues/9))
- [ ] Security headers (HSTS, CSP, X-Frame-Options)
- [ ] CSRF protection for forms
- [ ] Session expiration and "remember me"
- [ ] Password change/reset flow
- [ ] API key authentication for programmatic access
### Phase 4: Web UI
- [ ] Webhook management pages (list, create, edit, delete)
- [ ] Webhook request log viewer with filtering
- [ ] Delivery status and retry management UI
- [ ] Manual event redelivery
- [ ] Analytics dashboard (success rates, response times)
- [ ] Replace Bootstrap with Tailwind CSS + Alpine.js
([#4](https://git.eeqj.de/sneak/webhooker/issues/4))
### Phase 5: REST API
- [ ] RESTful CRUD for webhooks, entrypoints, targets
- [ ] Event viewing and filtering endpoints
- [ ] Event redelivery endpoint
- [ ] OpenAPI specification
### Phase 6: Code Quality
- [ ] Rename Processor → Webhook, Webhook → Entrypoint in code
([#12](https://git.eeqj.de/sneak/webhooker/issues/12))
- [ ] Embed templates via `//go:embed`
([#7](https://git.eeqj.de/sneak/webhooker/issues/7))
- [ ] Use `slog.LevelVar` for dynamic log level switching
([#8](https://git.eeqj.de/sneak/webhooker/issues/8))
- [ ] Simplify configuration to prefer environment variables
([#10](https://git.eeqj.de/sneak/webhooker/issues/10))
- [ ] Remove redundant `godotenv/autoload` import
([#11](https://git.eeqj.de/sneak/webhooker/issues/11))
### Future
- [ ] Email delivery target type
- [ ] SNS, S3, Slack delivery targets
- [ ] Data transformations (e.g., webhook-to-Slack message formatting)
- [ ] JSONL file delivery with periodic S3 upload
- [ ] Webhook event search and filtering
- [ ] Multi-user with role-based access
## License ## License

View File

@@ -1,6 +1,6 @@
--- ---
title: Repository Policies title: Repository Policies
last_modified: 2026-02-22 last_modified: 2026-07-06
--- ---
This document covers repository structure, tooling, and workflow standards. Code This document covers repository structure, tooling, and workflow standards. Code
@@ -34,10 +34,46 @@ style conventions are in separate documents:
every file before committing. There are zero exceptions to this rule. every file before committing. There are zero exceptions to this rule.
- Every repo with software must have a root `Makefile` with these targets: - Every repo with software must have a root `Makefile` with these targets:
`make test`, `make lint`, `make fmt` (writes), `make fmt-check` (read-only), `make bootstrap`, `make setup`, `make test`, `make lint`, `make fmt` (writes),
`make check` (prereqs: `test`, `lint`, `fmt-check`), `make docker`, and `make fmt-check` (read-only), `make check` (runs `test`, `lint`, `fmt-check`),
`make hooks` (installs pre-commit hook). A model Makefile is at `make docker`, and `make hooks` (installs pre-commit hook). A model Makefile
`https://git.eeqj.de/sneak/prompts/raw/branch/main/Makefile`. is at `https://git.eeqj.de/sneak/prompts/raw/branch/main/Makefile`.
- Repos follow the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
pattern: the implementation of each Makefile target lives in an executable
script in `script/` (`script/bootstrap`, `script/setup`, `script/test`,
`script/lint`, `script/fmt`, `script/fmt-check`, `script/check`,
`script/docker`), and the Makefile targets are thin shims that call them. The
scripts must be POSIX sh (`#!/bin/sh`, `set -eu`, no bashisms) so they run in
minimal containers (e.g. alpine images have no bash); locate the repo root
with `$(cd "$(dirname "$0")/.." && pwd -P)` and `cd` there before acting. From
the standard's canonical set we use `bootstrap`, `setup` (make the repo ready
for development after a fresh clone: runs `bootstrap`, then
`install-precommit`, plus any repo-specific initialization), `test`, and
`cibuild`. `script/bootstrap` installs all dependencies idempotently and
assumes nothing is present: base tools come from nix, apt, brew, or apk
(detected in that order; apt runs noninteractive). For node it uses the
installed node if present; otherwise it installs a PINNED node version via
nvm, first installing nvm itself if missing — from a hash-verified GitHub
release archive (never `curl | sh`), with bash installed as an explicit
prerequisite since nvm requires bash. yarn is then pinned via
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
always exact versions. `script/cibuild` runs the CI build: it changes to the
repo root and runs `docker build .`; the Gitea workflow calls it. Four further
scripts are our own extensions to the standard: `script/check` runs
`script/test`, `script/lint`, and `script/fmt-check`; `script/precommit` is
what the git pre-commit hook runs, and it calls `script/check`;
`script/install-precommit` installs the git pre-commit hook (the `make hooks`
target shims to it); and `script/projectname` (literally that filename) simply
outputs the project's name. Scripts that need the name call
`script/projectname` — e.g. `script/docker` assembles its image tag from it —
so those scripts stay byte-identical across all repos. Repo-type-specific
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
`script/precommit`, not in the hook itself. Model scripts are at
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
must document the provided scripts in an **Entrypoints** section (see the
README requirements below).
- Always use Makefile targets (`make fmt`, `make test`, `make lint`, etc.) - Always use Makefile targets (`make fmt`, `make test`, `make lint`, etc.)
instead of invoking the underlying tools directly. The Makefile is the single instead of invoking the underlying tools directly. The Makefile is the single
@@ -57,11 +93,83 @@ style conventions are in separate documents:
as a build step so the build fails if the branch is not green. For non-server as a build step so the build fails if the branch is not green. For non-server
repos, the Dockerfile should bring up a development environment and run repos, the Dockerfile should bring up a development environment and run
`make check`. For server repos, `make check` should run as an early build `make check`. For server repos, `make check` should run as an early build
stage before the final image is assembled. stage before the final image is assembled. Dockerfiles install development
prerequisites by running `script/bootstrap` rather than duplicating installs
inline; COPY `script/` and the dependency manifests (`package.json` +
`yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap
layer stays cached until dependencies change.
- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go
repos use a multistage build where linting runs in an independent stage based
on the `golangci/golangci-lint` image (pinned by hash). This stage runs
`make fmt-check` and `make lint` before the full build begins. The build stage
then declares an explicit dependency on the lint stage via
`COPY --from=lint /src/go.sum /dev/null`, which forces BuildKit to complete
linting before proceeding to compilation and tests. This ensures lint failures
surface in seconds rather than minutes, without blocking on dependency
download or compilation in the build stage.
The standard pattern for a Go repo Dockerfile is:
```dockerfile
# Lint stage — fast feedback on formatting and lint issues
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN make fmt-check
RUN make lint
# Build stage
# golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder
WORKDIR /src
# Force BuildKit to run the lint stage before proceeding
COPY --from=lint /src/go.sum /dev/null
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN make test
ARG VERSION=dev
RUN CGO_ENABLED=0 go build -trimpath \
-ldflags="-s -w -X main.Version=${VERSION}" \
-o /app ./cmd/app/
# Runtime stage
FROM alpine@sha256:...
COPY --from=builder /app /usr/local/bin/app
ENTRYPOINT ["app"]
```
Key points:
- The lint stage uses the `golangci/golangci-lint` image directly (it
includes both Go and the linter), so there is no need to install the
linter separately.
- `COPY --from=lint /src/go.sum /dev/null` is a no-op file copy that creates
a stage dependency. BuildKit runs stages in parallel by default; without
this line, the build stage would not wait for lint to finish and a lint
failure might not fail the overall build.
- If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint stage must
create placeholder files so the embed directives resolve. Example:
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
The lint stage should not depend on the actual build output — it exists to
fail fast.
- If the project requires CGO or system libraries for linting (e.g.
`vips-dev`), install them in the lint stage with `apk add`.
- The build stage runs `make test` after compilation setup. Tests run in the
build stage, not the lint stage, because they may require compiled
artifacts or heavier dependencies.
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that - Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `docker build .` on push. Since the Dockerfile already runs `make check`, runs `script/cibuild` (which runs `docker build .`) on push. Since the
a successful build implies all checks pass. Dockerfile already runs `make check`, a successful build implies all checks
pass.
- Use platform-standard formatters: `black` for Python, `prettier` for - Use platform-standard formatters: `black` for Python, `prettier` for
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
@@ -69,9 +177,11 @@ style conventions are in separate documents:
Markdown (hard-wrap at 80 columns). Documentation and writing repos (Markdown, Markdown (hard-wrap at 80 columns). Documentation and writing repos (Markdown,
HTML, CSS) should also have `.prettierrc` and `.prettierignore`. HTML, CSS) should also have `.prettierrc` and `.prettierignore`.
- Pre-commit hook: `make check` if local testing is possible, otherwise - Pre-commit hook: runs `script/precommit`, which calls `script/check`. If local
`make lint && make fmt-check`. The Makefile should provide a `make hooks` testing is not possible in the repo, `script/precommit` may skip `script/test`
target to install the pre-commit hook. and run only `script/lint` and `script/fmt-check`. The hook is installed by
`script/install-precommit`; the Makefile must provide a `make hooks` target
that shims to it.
- All repos with software must have tests that run via the platform-standard - All repos with software must have tests that run via the platform-standard
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful
@@ -82,6 +192,42 @@ style conventions are in separate documents:
- `make test` must complete in under 20 seconds. Add a 30-second timeout in the - `make test` must complete in under 20 seconds. Add a 30-second timeout in the
Makefile. Makefile.
- **`make test` should use the conditional verbose rerun pattern.** Run tests
without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to
show full output. This keeps CI logs and `docker build` output clean on
success (just package/suite summaries) while providing full diagnostic detail
on failure (every test case, every assertion). The general shell pattern:
```makefile
test:
@<test-command> || \
{ echo "--- Rerunning with -v for details ---"; \
<test-command-with-v>; exit 1; }
```
Go example:
```makefile
test:
@go test -timeout 30s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -timeout 30s -race -v ./...; exit 1; }
```
Python example:
```makefile
test:
@python -m pytest || \
{ echo "--- Rerunning with -v for details ---"; \
python -m pytest -v; exit 1; }
```
The `exit 1` ensures the target always fails after a rerun — the first run
already proved the tests are broken, so the build must not pass even if a
flaky test happens to succeed on the second attempt. The rerun exists solely
for diagnostic output.
- Docker builds must complete in under 5 minutes. - Docker builds must complete in under 5 minutes.
- `make check` must not modify any files in the repo. Tests may use temporary - `make check` must not modify any files in the repo. Tests may use temporary
@@ -98,6 +244,13 @@ style conventions are in separate documents:
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
a new repo. a new repo.
- **No build artifacts in version control.** Code-derived data (compiled
bundles, minified output, generated assets) must never be committed to the
repository if it can be avoided. The build process (e.g. Dockerfile, Makefile)
should generate these at build time. Notable exception: Go protobuf generated
files (`.pb.go`) ARE committed because repos need to work with `go get`, which
downloads code but does not execute code generation.
- Never use `git add -A` or `git add .`. Always stage files explicitly by name. - Never use `git add -A` or `git add .`. Always stage files explicitly by name.
- Never force-push to `main`. - Never force-push to `main`.
@@ -121,12 +274,76 @@ style conventions are in separate documents:
- Dockerized web services listen on port 8080 by default, overridable with - Dockerized web services listen on port 8080 by default, overridable with
`PORT`. `PORT`.
- **HTTP/web services must be hardened for production internet exposure before
tagging 1.0.** This means full compliance with security best practices
including, without limitation, all of the following:
- **Security headers** on every response:
- `Strict-Transport-Security` (HSTS) with `max-age` of at least one year
and `includeSubDomains`.
- `Content-Security-Policy` (CSP) with a restrictive default policy
(`default-src 'self'` as a baseline, tightened per-resource as
needed). Never use `unsafe-inline` or `unsafe-eval` unless
unavoidable, and document the reason.
- `X-Frame-Options: DENY` (or `SAMEORIGIN` if framing is required).
Prefer the `frame-ancestors` CSP directive as the primary control.
- `X-Content-Type-Options: nosniff`.
- `Referrer-Policy: strict-origin-when-cross-origin` (or stricter).
- `Permissions-Policy` restricting access to browser features the
application does not use (camera, microphone, geolocation, etc.).
- **Request and response limits:**
- Maximum request body size enforced on all endpoints (e.g. Go
`http.MaxBytesReader`). Choose a sane default per-route; never accept
unbounded input.
- Maximum response body size where applicable (e.g. paginated APIs).
- `ReadTimeout` and `ReadHeaderTimeout` on the `http.Server` to defend
against slowloris attacks.
- `WriteTimeout` on the `http.Server`.
- `IdleTimeout` on the `http.Server`.
- Per-handler execution time limits via `context.WithTimeout` or
chi/stdlib `middleware.Timeout`.
- **Authentication and session security:**
- Rate limiting on password-based authentication endpoints. API keys are
high-entropy and not susceptible to brute force, so they are exempt.
- CSRF tokens on all state-mutating HTML forms. API endpoints
authenticated via `Authorization` header (Bearer token, API key) are
exempt because the browser does not attach these automatically.
- Passwords stored using bcrypt, scrypt, or argon2 — never plain-text,
MD5, or SHA.
- Session cookies set with `HttpOnly`, `Secure`, and `SameSite=Lax` (or
`Strict`) attributes.
- **Reverse proxy awareness:**
- True client IP detection when behind a reverse proxy
(`X-Forwarded-For`, `X-Real-IP`). The application must accept
forwarded headers only from a configured set of trusted proxy
addresses — never trust `X-Forwarded-For` unconditionally.
- **CORS:**
- Authenticated endpoints must restrict `Access-Control-Allow-Origin` to
an explicit allowlist of known origins. Wildcard (`*`) is acceptable
only for public, unauthenticated read-only APIs.
- **Error handling:**
- Internal errors must never leak stack traces, SQL queries, file paths,
or other implementation details to the client. Return generic error
messages in production; detailed errors only when `DEBUG` is enabled.
- **TLS:**
- Services never terminate TLS directly. They are always deployed behind
a TLS-terminating reverse proxy. The service itself listens on plain
HTTP. However, HSTS headers and `Secure` cookie flags must still be
set by the application so that the browser enforces HTTPS end-to-end.
This list is non-exhaustive. Apply defense-in-depth: if a standard security
hardening measure exists for HTTP services and is not listed here, it is
still expected. When in doubt, harden.
- `README.md` is the primary documentation. Required sections: - `README.md` is the primary documentation. Required sections:
- **Description**: First line must include the project name, purpose, - **Description**: First line must include the project name, purpose,
category (web server, SPA, CLI tool, etc.), license, and author. Example: category (web server, SPA, CLI tool, etc.), license, and author. Example:
"µPaaS is an MIT-licensed Go web application by @sneak that receives "µPaaS is an MIT-licensed Go web application by @sneak that receives
git-frontend webhooks and deploys applications via Docker in realtime." git-frontend webhooks and deploys applications via Docker in realtime."
- **Getting Started**: Copy-pasteable install/usage code block. - **Getting Started**: Copy-pasteable install/usage code block.
- **Entrypoints**: Opens by stating that the repo adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard (with that link), then documents each provided `script/`
entrypoint and its purpose.
- **Rationale**: Why does this exist? - **Rationale**: Why does this exist?
- **Design**: How is the program structured? - **Design**: How is the program structured?
- **TODO**: Update meticulously, even between commits. When planning, put - **TODO**: Update meticulously, even between commits. When planning, put
@@ -145,11 +362,11 @@ style conventions are in separate documents:
- Database migrations live in `internal/db/migrations/` and must be embedded in - Database migrations live in `internal/db/migrations/` and must be embedded in
the binary. the binary.
- `000_migration.sql` — contains ONLY the creation of the migrations tracking - `000_migration.sql` — contains ONLY the creation of the migrations
table itself. Nothing else. tracking table itself. Nothing else.
- `001_schema.sql` — the full application schema. - `001_schema.sql` — the full application schema.
- **Pre-1.0.0:** never add additional migration files (002, 003, etc.). There - **Pre-1.0.0:** never add additional migration files (002, 003, etc.).
is no installed base to migrate. Edit `001_schema.sql` directly. There is no installed base to migrate. Edit `001_schema.sql` directly.
- **Post-1.0.0:** add new numbered migration files for each schema change. - **Post-1.0.0:** add new numbered migration files for each schema change.
Never edit existing migrations after release. Never edit existing migrations after release.
@@ -181,6 +398,9 @@ style conventions are in separate documents:
- `README.md`, `.git`, `.gitignore`, `.editorconfig` - `README.md`, `.git`, `.gitignore`, `.editorconfig`
- `LICENSE`, `REPO_POLICIES.md` (copy from the `prompts` repo) - `LICENSE`, `REPO_POLICIES.md` (copy from the `prompts` repo)
- `Makefile` - `Makefile`
- `script/` entrypoints (`bootstrap`, `setup`, `projectname`, `test`,
`lint`, `fmt`, `fmt-check`, `check`, `docker`, `cibuild`, `precommit`,
`install-precommit`)
- `Dockerfile`, `.dockerignore` - `Dockerfile`, `.dockerignore`
- `.gitea/workflows/check.yml` - `.gitea/workflows/check.yml`
- Go: `go.mod`, `go.sum`, `.golangci.yml` - Go: `go.mod`, `go.sum`, `.golangci.yml`

78
TODO.md Normal file
View File

@@ -0,0 +1,78 @@
# Workflow
* branch (from `main`)
* do the work in Next Step
* move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work)
* merge to `main` if the branch is not protected, otherwise open a PR
* push
# Status
pre-1.0. No git tags exist. main (81413c5) is a working webhook proxy
with auth, CSRF/SSRF protections, login rate limiting, Slack target,
policy compliance (#6), pinned lint tooling (#55), a per-webhook event
retention reaper (#63), and delivery targets behind a Target interface
(#77). Work is tracked as Gitea issues (the authoritative TODO); this
file is a summary. Note: TODO.md was deliberately deleted from this
repo in f9a9569 (2026-03-01, #6); its content was folded into the
README TODO section, which this draft reconstructs as of 2026-07-06.
# Next Step
Manual event redelivery from the web UI (replay is a core promised
capability in the README rationale).
# Completed Steps
- 2026-08-07 Rate-limit the public webhook receiver per client IP per
entrypoint, env-configurable with fail-loud parsing (#64)
- 2026-08-07 Per-webhook event retention reaper (#63); NoCache
middleware for authenticated pages (#61); Target interface refactor
(#77)
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
Makefile shims, README Entrypoints section
- 2026-03-25 pin golangci-lint Docker image for linting (#55)
- 2026-03-18 CSRF middleware detects TLS per-request, fixing login over
plain HTTP and behind reverse proxies (#54)
- 2026-03-17 root path redirects based on auth state (#52)
- 2026-03-17 CSRF protection, SSRF prevention for HTTP delivery targets
with DNS rebinding defense, and per-IP login rate limiting (#42)
- 2026-03-17 Slack target type for incoming webhook notifications (#47)
- 2026-03-17 Dockerfile absolute paths and static linking (#49);
absolute dev DATA_DIR default and clarified env docs (#46)
- 2026-03-05 security headers middleware, session regeneration on
login, request body size limits (#41)
- 2026-03-04 tests for delivery, middleware, and session packages
(#32); removed globals.Buildarch (#31)
- 2026-03-04 1.0 MVP merge: Webhook/Entrypoint/Target rename, core
delivery engine with bounded worker pool and circuit breaker,
parallel fan-out, per-webhook event databases, management UI (#16)
- 2026-03-01 repo brought to REPO_POLICIES standards; TODO.md folded
into README (#6)
# Future Steps
- Delivery status and retry management UI
- Per-webhook rate limiting in the receiver handler (per-webhook config
plus handler enforcement, layered on the env-level receiver limit
from #64; global limits must not apply to receiver endpoints)
- Webhook signature verification for GitHub and Stripe HMAC formats
- API key authentication for programmatic access (APIKey model exists;
Bearer token middleware does not)
- REST API v1
- CRUD for webhooks, entrypoints, targets
- event viewing and filtering endpoints
- event redelivery endpoint
- OpenAPI specification
- Analytics dashboard: success rates, response times, volume
- Session expiration tuning and a remember-me option
- Password change and reset flow
- Later, nice to have
- email delivery target type
- SNS and S3 delivery targets
- data transformations (e.g. webhook to Slack message formatting)
- JSONL file delivery with periodic S3 upload
- webhook event search and filtering
- multi-user with role-based access control

View File

@@ -1,11 +1,11 @@
// Package main is the entry point for the webhooker application.
package main package main
import ( import (
"runtime"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/handlers" "sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/healthcheck" "sneak.berlin/go/webhooker/internal/healthcheck"
@@ -16,6 +16,8 @@ import (
) )
// Build-time variables set via -ldflags. // Build-time variables set via -ldflags.
//
//nolint:gochecknoglobals // Build-time variables injected by the linker.
var ( var (
version = "dev" version = "dev"
appname = "webhooker" appname = "webhooker"
@@ -24,7 +26,6 @@ var (
func main() { func main() {
globals.Appname = appname globals.Appname = appname
globals.Version = version globals.Version = version
globals.Buildarch = runtime.GOARCH
fx.New( fx.New(
fx.Provide( fx.Provide(
@@ -32,12 +33,25 @@ func main() {
logger.New, logger.New,
config.New, config.New,
database.New, database.New,
database.NewWebhookDBManager,
database.NewRetentionReaper,
healthcheck.New, healthcheck.New,
session.New, session.New,
handlers.New, handlers.New,
middleware.New, middleware.New,
delivery.New,
// Wire *delivery.Engine as delivery.Notifier so the
// webhook handler can notify the engine of new deliveries.
func(e *delivery.Engine) delivery.Notifier { return e },
server.New, server.New,
), ),
fx.Invoke(func(*server.Server) {}), fx.Invoke(
func(
*server.Server,
*delivery.Engine,
*database.RetentionReaper,
) {
},
),
).Run() ).Run()
} }

View File

@@ -1,50 +0,0 @@
environments:
dev:
config:
port: 8080
debug: true
maintenanceMode: false
developmentMode: true
environment: dev
# Database URL for local development
dburl: postgres://webhooker:webhooker@localhost:5432/webhooker_dev?sslmode=disable
# Basic auth for metrics endpoint in dev
metricsUsername: admin
metricsPassword: admin
# Dev admin credentials for testing
devAdminUsername: devadmin
devAdminPassword: devpassword
secrets:
# Use default insecure session key for development
sessionKey: d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=
# Sentry DSN - usually not needed in dev
sentryDSN: ""
prod:
config:
port: $ENV:PORT
debug: $ENV:DEBUG
maintenanceMode: $ENV:MAINTENANCE_MODE
developmentMode: false
environment: prod
dburl: $ENV:DBURL
metricsUsername: $ENV:METRICS_USERNAME
metricsPassword: $ENV:METRICS_PASSWORD
# Dev admin credentials should not be set in production
devAdminUsername: ""
devAdminPassword: ""
secrets:
sessionKey: $ENV:SESSION_KEY
sentryDSN: $ENV:SENTRY_DSN
configDefaults:
# These defaults apply to all environments unless overridden
port: 8080
debug: false
maintenanceMode: false
developmentMode: false
environment: dev
metricsUsername: ""
metricsPassword: ""
devAdminUsername: ""
devAdminPassword: ""

33
go.mod
View File

@@ -1,49 +1,37 @@
module sneak.berlin/go/webhooker module sneak.berlin/go/webhooker
go 1.23.0 go 1.26.1
toolchain go1.24.1
require ( require (
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8 github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
github.com/getsentry/sentry-go v0.25.0 github.com/getsentry/sentry-go v0.25.0
github.com/go-chi/chi v1.5.5 github.com/go-chi/chi v1.5.5
github.com/go-chi/cors v1.2.1 github.com/go-chi/cors v1.2.1
github.com/go-chi/httprate v0.15.0
github.com/google/uuid v1.6.0 github.com/google/uuid v1.6.0
github.com/gorilla/csrf v1.7.3
github.com/gorilla/sessions v1.4.0 github.com/gorilla/sessions v1.4.0
github.com/joho/godotenv v1.5.1 github.com/joho/godotenv v1.5.1
github.com/prometheus/client_golang v1.18.0 github.com/prometheus/client_golang v1.18.0
github.com/slok/go-http-metrics v0.11.0 github.com/slok/go-http-metrics v0.11.0
github.com/spf13/afero v1.14.0
github.com/stretchr/testify v1.8.4 github.com/stretchr/testify v1.8.4
go.uber.org/fx v1.20.1 go.uber.org/fx v1.20.1
golang.org/x/crypto v0.38.0 golang.org/x/crypto v0.38.0
gorm.io/driver/sqlite v1.5.4 gorm.io/driver/sqlite v1.5.4
gorm.io/gorm v1.25.5 gorm.io/gorm v1.25.5
modernc.org/sqlite v1.28.0 modernc.org/sqlite v1.28.0
sneak.berlin/go/webhooker/pkg/config v0.0.0-00010101000000-000000000000
) )
require ( require (
cloud.google.com/go/compute v1.23.3 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/iam v1.1.5 // indirect
cloud.google.com/go/secretmanager v1.11.4 // indirect
github.com/aws/aws-sdk-go v1.50.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dustin/go-humanize v1.0.1 // indirect github.com/dustin/go-humanize v1.0.1 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/gorilla/securecookie v1.1.2 // indirect github.com/gorilla/securecookie v1.1.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect github.com/jinzhu/now v1.1.5 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/cpuid/v2 v2.2.10 // indirect
github.com/kr/text v0.2.0 // indirect github.com/kr/text v0.2.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-sqlite3 v1.14.17 // indirect github.com/mattn/go-sqlite3 v1.14.17 // indirect
@@ -53,25 +41,16 @@ require (
github.com/prometheus/common v0.45.0 // indirect github.com/prometheus/common v0.45.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect github.com/prometheus/procfs v0.12.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
go.opencensus.io v0.24.0 // indirect github.com/zeebo/xxh3 v1.0.2 // indirect
go.uber.org/atomic v1.9.0 // indirect go.uber.org/atomic v1.9.0 // indirect
go.uber.org/dig v1.17.0 // indirect go.uber.org/dig v1.17.0 // indirect
go.uber.org/multierr v1.9.0 // indirect go.uber.org/multierr v1.9.0 // indirect
go.uber.org/zap v1.23.0 // indirect go.uber.org/zap v1.23.0 // indirect
golang.org/x/mod v0.17.0 // indirect golang.org/x/mod v0.17.0 // indirect
golang.org/x/net v0.25.0 // indirect
golang.org/x/oauth2 v0.15.0 // indirect
golang.org/x/sync v0.14.0 // indirect golang.org/x/sync v0.14.0 // indirect
golang.org/x/sys v0.33.0 // indirect golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.25.0 // indirect golang.org/x/text v0.25.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
google.golang.org/api v0.153.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20231106174013-bbf56f31fb17 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20231106174013-bbf56f31fb17 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f // indirect
google.golang.org/grpc v1.59.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/uint128 v1.2.0 // indirect lukechampine.com/uint128 v1.2.0 // indirect
@@ -84,5 +63,3 @@ require (
modernc.org/strutil v1.1.3 // indirect modernc.org/strutil v1.1.3 // indirect
modernc.org/token v1.0.1 // indirect modernc.org/token v1.0.1 // indirect
) )
replace sneak.berlin/go/webhooker/pkg/config => ./pkg/config

148
go.sum
View File

@@ -1,28 +1,11 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.110.10 h1:LXy9GEO+timppncPIAZoOj3l58LIU9k+kn48AN7IO3Y=
cloud.google.com/go v0.110.10/go.mod h1:v1OoFqYxiBkUrruItNM3eT4lLByNjxmJSV/xDKJNnic=
cloud.google.com/go/compute v1.23.3 h1:6sVlXXBmbd7jNX0Ipq0trII3e4n1/MsADLK6a+aiVlk=
cloud.google.com/go/compute v1.23.3/go.mod h1:VCgBUoMnIVIR0CscqQiPJLAG25E3ZRZMzcFZeQ+h8CI=
cloud.google.com/go/compute/metadata v0.2.3 h1:mg4jlk7mCAj6xXp9UJ4fjI9VUI5rubuGBW5aJ7UnBMY=
cloud.google.com/go/compute/metadata v0.2.3/go.mod h1:VAV5nSsACxMJvgaAuX6Pk2AawlZn8kiOGuCv6gTkwuA=
cloud.google.com/go/iam v1.1.5 h1:1jTsCu4bcsNsE4iiqNT5SHwrDRCfRmIaaaVFhRveTJI=
cloud.google.com/go/iam v1.1.5/go.mod h1:rB6P/Ic3mykPbFio+vo7403drjlgvoWfYpJhMXEbzv8=
cloud.google.com/go/secretmanager v1.11.4 h1:krnX9qpG2kR2fJ+u+uNyNo+ACVhplIAS4Pu7u+4gd+k=
cloud.google.com/go/secretmanager v1.11.4/go.mod h1:wreJlbS9Zdq21lMzWmJ0XhWW2ZxgPeahsqeV/vZoJ3w=
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8 h1:nMpu1t4amK3vJWBibQ5X/Nv0aXL+b69TQf2uK5PH7Go= github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8 h1:nMpu1t4amK3vJWBibQ5X/Nv0aXL+b69TQf2uK5PH7Go=
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8/go.mod h1:3cARGAK9CfW3HoxCy1a0G4TKrdiKke8ftOMEOHyySYs= github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8/go.mod h1:3cARGAK9CfW3HoxCy1a0G4TKrdiKke8ftOMEOHyySYs=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/aws/aws-sdk-go v1.50.0 h1:HBtrLeO+QyDKnc3t1+5DR1RxodOHCGr8ZcrHudpv7jI=
github.com/aws/aws-sdk-go v1.50.0/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk=
github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A= github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A=
github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -30,42 +13,17 @@ github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/getsentry/sentry-go v0.25.0 h1:q6Eo+hS+yoJlTO3uu/azhQadsD8V+jQn2D8VvX1eOyI= github.com/getsentry/sentry-go v0.25.0 h1:q6Eo+hS+yoJlTO3uu/azhQadsD8V+jQn2D8VvX1eOyI=
github.com/getsentry/sentry-go v0.25.0/go.mod h1:lc76E2QywIyW8WuBnwl8Lc4bkmQH4+w1gwTf25trprY= github.com/getsentry/sentry-go v0.25.0/go.mod h1:lc76E2QywIyW8WuBnwl8Lc4bkmQH4+w1gwTf25trprY=
github.com/go-chi/chi v1.5.5 h1:vOB/HbEMt9QqBqErz07QehcOKHaWFtuj87tTDVz2qXE= github.com/go-chi/chi v1.5.5 h1:vOB/HbEMt9QqBqErz07QehcOKHaWFtuj87tTDVz2qXE=
github.com/go-chi/chi v1.5.5/go.mod h1:C9JqLr3tIYjDOZpzn+BCuxY8z8vmca43EeMgyZt7irw= github.com/go-chi/chi v1.5.5/go.mod h1:C9JqLr3tIYjDOZpzn+BCuxY8z8vmca43EeMgyZt7irw=
github.com/go-chi/cors v1.2.1 h1:xEC8UT3Rlp2QuWNEr4Fs/c2EAGVKBwy/1vHx3bppil4= github.com/go-chi/cors v1.2.1 h1:xEC8UT3Rlp2QuWNEr4Fs/c2EAGVKBwy/1vHx3bppil4=
github.com/go-chi/cors v1.2.1/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58= github.com/go-chi/cors v1.2.1/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-chi/httprate v0.15.0 h1:j54xcWV9KGmPf/X4H32/aTH+wBlrvxL7P+SdnRqxh5g=
github.com/go-chi/httprate v0.15.0/go.mod h1:rzGHhVrsBn3IMLYDOZQsSU4fJNWcjui4fWKJcCId1R4=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA= github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og= github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
@@ -73,15 +31,10 @@ github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ= github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo= github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo=
github.com/google/s2a-go v0.1.7 h1:60BLSyTrOV4/haCDW4zb1guZItoSq8foHCXrAnjBo/o=
github.com/google/s2a-go v0.1.7/go.mod h1:50CgR4k1jNlWBu4UfS4AcfhVe1r6pdZPygJ3R8F0Qdw=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.2 h1:Vie5ybvEvT75RniqhfFxPRy3Bf7vr3h0cechB90XaQs= github.com/gorilla/csrf v1.7.3 h1:BHWt6FTLZAb2HtWT5KDBf6qgpZzvtbp9QWDRKZMXJC0=
github.com/googleapis/enterprise-certificate-proxy v0.3.2/go.mod h1:VLSiSSBs/ksPL8kq3OBOQ6WRI2QnaFynd1DCjZ62+V0= github.com/gorilla/csrf v1.7.3/go.mod h1:F1Fj3KG23WYHE6gozCmBAezKookxbIvUJT+121wTuLk=
github.com/googleapis/gax-go/v2 v2.12.0 h1:A+gCJKdRfqXkr+BIRGtZLibNXf0m1f9E4HG56etFpas=
github.com/googleapis/gax-go/v2 v2.12.0/go.mod h1:y+aIqrI5eb1YGMVJfuV3185Ts/D7qKpsEkdD5+I6QGU=
github.com/gorilla/securecookie v1.1.2 h1:YCIWL56dvtr73r6715mJs5ZvhtnY73hBvEF8kXD8ePA= github.com/gorilla/securecookie v1.1.2 h1:YCIWL56dvtr73r6715mJs5ZvhtnY73hBvEF8kXD8ePA=
github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pwzwo4h3eOamfo= github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pwzwo4h3eOamfo=
github.com/gorilla/sessions v1.4.0 h1:kpIYOp/oi6MG/p5PgxApU8srsSw9tuFbt46Lt7auzqQ= github.com/gorilla/sessions v1.4.0 h1:kpIYOp/oi6MG/p5PgxApU8srsSw9tuFbt46Lt7auzqQ=
@@ -90,14 +43,12 @@ github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc= github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ= github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0= github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4= github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs= github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8= github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
@@ -117,7 +68,6 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.18.0 h1:HzFfmkOzH5Q8L8G+kSJKUx5dtG87sewO+FoDDqP5Tbk= github.com/prometheus/client_golang v1.18.0 h1:HzFfmkOzH5Q8L8G+kSJKUx5dtG87sewO+FoDDqP5Tbk=
github.com/prometheus/client_golang v1.18.0/go.mod h1:T+GXkCk5wSJyOqMIzVgvvjFDlkOQntgjkJWKrN5txjA= github.com/prometheus/client_golang v1.18.0/go.mod h1:T+GXkCk5wSJyOqMIzVgvvjFDlkOQntgjkJWKrN5txjA=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw= github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw=
github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI= github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI=
github.com/prometheus/common v0.45.0 h1:2BGz0eBc2hdMDLnO/8n0jeB3oPrt2D08CekT0lneoxM= github.com/prometheus/common v0.45.0 h1:2BGz0eBc2hdMDLnO/8n0jeB3oPrt2D08CekT0lneoxM=
@@ -130,21 +80,16 @@ github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjR
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/slok/go-http-metrics v0.11.0 h1:ABJUpekCZSkQT1wQrFvS4kGbhea/w6ndFJaWJeh3zL0= github.com/slok/go-http-metrics v0.11.0 h1:ABJUpekCZSkQT1wQrFvS4kGbhea/w6ndFJaWJeh3zL0=
github.com/slok/go-http-metrics v0.11.0/go.mod h1:ZGKeYG1ET6TEJpQx18BqAJAvxw9jBAZXCHU7bWQqqAc= github.com/slok/go-http-metrics v0.11.0/go.mod h1:ZGKeYG1ET6TEJpQx18BqAJAvxw9jBAZXCHU7bWQqqAc=
github.com/spf13/afero v1.14.0 h1:9tH6MapGnn/j0eb0yIXiLjERO8RB6xIVZRDCX7PtqWA=
github.com/spf13/afero v1.14.0/go.mod h1:acJQ8t0ohCGuMN3O+Pv0V0hgMxNYDlvdk+VTfyZmbYo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.1 h1:4VhoImhV/Bm0ToFkXFi8hXNXwpDRZ/ynw3amt82mzq0= github.com/stretchr/objx v0.5.1 h1:4VhoImhV/Bm0ToFkXFi8hXNXwpDRZ/ynw3amt82mzq0=
github.com/stretchr/objx v0.5.1/go.mod h1:/iHQpkQwBD6DLUmQ4pE+s1TXdob1mORJ4/UFdrifcy0= github.com/stretchr/objx v0.5.1/go.mod h1:/iHQpkQwBD6DLUmQ4pE+s1TXdob1mORJ4/UFdrifcy0=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/dig v1.17.0 h1:5Chju+tUvcC+N7N6EV08BJz41UZuO3BmHcN4A287ZLI= go.uber.org/dig v1.17.0 h1:5Chju+tUvcC+N7N6EV08BJz41UZuO3BmHcN4A287ZLI=
@@ -157,105 +102,32 @@ go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ= go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
go.uber.org/zap v1.23.0 h1:OjGQ5KQDEUawVHxNwQgPpiypGHOxo2mNZsOqTak4fFY= go.uber.org/zap v1.23.0 h1:OjGQ5KQDEUawVHxNwQgPpiypGHOxo2mNZsOqTak4fFY=
go.uber.org/zap v1.23.0/go.mod h1:D+nX8jyLsMHMYrln8A0rJjFt/T/9/bGgIhAqxv5URuY= go.uber.org/zap v1.23.0/go.mod h1:D+nX8jyLsMHMYrln8A0rJjFt/T/9/bGgIhAqxv5URuY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8= golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8=
golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw= golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA= golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.15.0 h1:s8pnnxNVzjWyrvYdFUQq5llS1PX2zhPXmccZv99h7uQ=
golang.org/x/oauth2 v0.15.0/go.mod h1:q48ptWNTY5XWf+JNten23lcvHpLJ0ZSxF5ttTHKVCAM=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ= golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ=
golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw= golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4= golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA= golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg= golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.153.0 h1:N1AwGhielyKFaUqH07/ZSIQR3uNPcV7NVw0vj+j4iR4=
google.golang.org/api v0.153.0/go.mod h1:3qNJX5eOmhiWYc67jRA/3GsDw97UFb5ivv7Y2PrriAY=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20231106174013-bbf56f31fb17 h1:wpZ8pe2x1Q3f2KyT5f8oP/fa9rHAKgFPr/HZdNuS+PQ=
google.golang.org/genproto v0.0.0-20231106174013-bbf56f31fb17/go.mod h1:J7XzRzVy1+IPwWHZUzoD0IccYZIrXILAQpc+Qy9CMhY=
google.golang.org/genproto/googleapis/api v0.0.0-20231106174013-bbf56f31fb17 h1:JpwMPBpFN3uKhdaekDpiNlImDdkUAyiJ6ez/uxGaUSo=
google.golang.org/genproto/googleapis/api v0.0.0-20231106174013-bbf56f31fb17/go.mod h1:0xJLfVdJqpAPl8tDg1ujOCGzx6LFLttXT5NhllGOXY4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f h1:ultW7fxlIvee4HYrtnaRPon9HpEgFk5zYpmfMgtKB5I=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f/go.mod h1:L9KNLi232K1/xB6f7AlSX692koaRnKaWSR0stBki0Yc=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.59.0 h1:Z5Iec2pjwb+LEOqzpB2MR12/eKFhDPhuqW91O+4bwUk=
google.golang.org/grpc v1.59.0/go.mod h1:aUPDwccQo6OTjy7Hct4AfBPD1GptF4fyUjIkQ9YtF98=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gorm.io/driver/sqlite v1.5.4 h1:IqXwXi8M/ZlPzH/947tn5uik3aYQslP9BVveoax0nV0= gorm.io/driver/sqlite v1.5.4 h1:IqXwXi8M/ZlPzH/947tn5uik3aYQslP9BVveoax0nV0=
gorm.io/driver/sqlite v1.5.4/go.mod h1:qxAuCol+2r6PannQDpOP1FP6ag3mKi4esLnB/jHed+4= gorm.io/driver/sqlite v1.5.4/go.mod h1:qxAuCol+2r6PannQDpOP1FP6ag3mKi4esLnB/jHed+4=
gorm.io/gorm v1.25.5 h1:zR9lOiiYf09VNh5Q1gphfyia1JpiClIWG9hQaxB/mls= gorm.io/gorm v1.25.5 h1:zR9lOiiYf09VNh5Q1gphfyia1JpiClIWG9hQaxB/mls=
gorm.io/gorm v1.25.5/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8= gorm.io/gorm v1.25.5/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI= lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI=
lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk= lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk=
modernc.org/cc/v3 v3.40.0 h1:P3g79IUS/93SYhtoeaHW+kRCIrYaxJ27MFPv+7kaTOw= modernc.org/cc/v3 v3.40.0 h1:P3g79IUS/93SYhtoeaHW+kRCIrYaxJ27MFPv+7kaTOw=

View File

@@ -1,16 +1,18 @@
// Package config loads application configuration from environment variables.
package config package config
import ( import (
"errors"
"fmt" "fmt"
"log/slog" "log/slog"
"os" "os"
"strconv" "strconv"
"strings" "strings"
"time"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
pkgconfig "sneak.berlin/go/webhooker/pkg/config"
// Populates the environment from a ./.env file automatically for // Populates the environment from a ./.env file automatically for
// development configuration. Kept in one place only (here). // development configuration. Kept in one place only (here).
@@ -18,133 +20,228 @@ import (
) )
const ( const (
// EnvironmentDev represents development environment // EnvironmentDev represents development environment.
EnvironmentDev = "dev" EnvironmentDev = "dev"
// EnvironmentProd represents production environment // EnvironmentProd represents production environment.
EnvironmentProd = "prod" EnvironmentProd = "prod"
// DevSessionKey is an insecure default session key for development
// This is "webhooker-dev-session-key-insecure!" base64 encoded // defaultPort is the default HTTP listen port.
DevSessionKey = "d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=" defaultPort = 8080
// defaultRetentionSweepInterval is how often the retention
// reaper deletes events older than each webhook's RetentionDays.
defaultRetentionSweepInterval = time.Hour
// defaultReceiverRateLimit is the default number of requests
// per minute each client IP may send to a single webhook
// receiver entrypoint. Generous for legitimate webhook
// senders while bounding abuse of the one unauthenticated,
// internet-exposed endpoint.
defaultReceiverRateLimit = 120
) )
// nolint:revive // ConfigParams is a standard fx naming convention // ErrInvalidEnvironment is returned when WEBHOOKER_ENVIRONMENT
// contains an unrecognised value.
var ErrInvalidEnvironment = errors.New("invalid environment")
// ErrNonPositiveValue is returned when an environment variable that
// requires a positive integer is set to zero or a negative number.
var ErrNonPositiveValue = errors.New("value must be positive")
//nolint:revive // ConfigParams is a standard fx naming convention.
type ConfigParams struct { type ConfigParams struct {
fx.In fx.In
Globals *globals.Globals Globals *globals.Globals
Logger *logger.Logger Logger *logger.Logger
} }
// Config holds all application configuration loaded from
// environment variables.
type Config struct { type Config struct {
DBURL string DataDir string
Debug bool Debug bool
MaintenanceMode bool MaintenanceMode bool
DevelopmentMode bool
DevAdminUsername string
DevAdminPassword string
Environment string Environment string
MetricsPassword string MetricsPassword string
MetricsUsername string MetricsUsername string
Port int Port int
SentryDSN string SentryDSN string
SessionKey string
// RetentionSweepInterval is how often the retention reaper runs.
RetentionSweepInterval time.Duration
// ReceiverRateLimit is the number of requests per minute each
// client IP may send to a single webhook receiver entrypoint.
ReceiverRateLimit int
params *ConfigParams params *ConfigParams
log *slog.Logger log *slog.Logger
} }
// IsDev returns true if running in development environment // IsDev returns true if running in development environment.
func (c *Config) IsDev() bool { func (c *Config) IsDev() bool {
return c.Environment == EnvironmentDev return c.Environment == EnvironmentDev
} }
// IsProd returns true if running in production environment // IsProd returns true if running in production environment.
func (c *Config) IsProd() bool { func (c *Config) IsProd() bool {
return c.Environment == EnvironmentProd return c.Environment == EnvironmentProd
} }
// envString returns the env var value if set, otherwise falls back to pkgconfig. // envString returns the value of the named environment variable,
func envString(envKey, configKey string) string { // or an empty string if not set.
if v := os.Getenv(envKey); v != "" { func envString(key string) string {
return v return os.Getenv(key)
}
return pkgconfig.GetString(configKey)
} }
// envSecretString returns the env var value if set, otherwise falls back to pkgconfig secrets. // envBool returns the value of the named environment variable
func envSecretString(envKey, configKey string) string { // parsed as a boolean. Returns defaultValue if not set.
if v := os.Getenv(envKey); v != "" { func envBool(key string, defaultValue bool) bool {
return v if v := os.Getenv(key); v != "" {
}
return pkgconfig.GetSecretString(configKey)
}
// envBool returns the env var value parsed as bool, otherwise falls back to pkgconfig.
func envBool(envKey, configKey string) bool {
if v := os.Getenv(envKey); v != "" {
return strings.EqualFold(v, "true") || v == "1" return strings.EqualFold(v, "true") || v == "1"
} }
return pkgconfig.GetBool(configKey)
return defaultValue
} }
// envInt returns the env var value parsed as int, otherwise falls back to pkgconfig. // envInt returns the value of the named environment variable
func envInt(envKey, configKey string, defaultValue ...int) int { // parsed as an integer. Returns defaultValue if not set or
if v := os.Getenv(envKey); v != "" { // unparseable.
if i, err := strconv.Atoi(v); err == nil { func envInt(key string, defaultValue int) int {
if v := os.Getenv(key); v != "" {
i, err := strconv.Atoi(v)
if err == nil {
return i return i
} }
} }
return pkgconfig.GetInt(configKey, defaultValue...)
return defaultValue
} }
// nolint:revive // lc parameter is required by fx even if unused // envPositiveInt returns the value of the named environment variable
// parsed as a positive integer. Returns defaultValue if not set. If
// the variable is set but cannot be parsed, or parses to less than
// one, it returns a wrapped error naming the key and the bad value,
// so startup fails loudly rather than silently falling back to the
// default.
func envPositiveInt(
key string,
defaultValue int,
) (int, error) {
v := os.Getenv(key)
if v == "" {
return defaultValue, nil
}
i, err := strconv.Atoi(v)
if err != nil {
return 0, fmt.Errorf(
"invalid integer for %s: %q: %w", key, v, err,
)
}
if i < 1 {
return 0, fmt.Errorf(
"%w: %s must be at least 1, got %q",
ErrNonPositiveValue, key, v,
)
}
return i, nil
}
// envDuration returns the value of the named environment variable
// parsed as a Go duration (e.g. "1h", "30m"). Returns defaultValue if
// not set. If the variable is set but cannot be parsed, it returns a
// wrapped error naming the key and the bad value, so startup fails
// loudly rather than silently falling back to the default.
func envDuration(
key string,
defaultValue time.Duration,
) (time.Duration, error) {
v := os.Getenv(key)
if v == "" {
return defaultValue, nil
}
d, err := time.ParseDuration(v)
if err != nil {
return 0, fmt.Errorf(
"invalid duration for %s: %q: %w", key, v, err,
)
}
return d, nil
}
// New creates a Config by reading environment variables.
//
//nolint:revive // lc parameter is required by fx even if unused.
func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) { func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) {
log := params.Logger.Get() log := params.Logger.Get()
// Determine environment from WEBHOOKER_ENVIRONMENT env var, default to dev // Determine environment from WEBHOOKER_ENVIRONMENT env var,
// default to dev
environment := os.Getenv("WEBHOOKER_ENVIRONMENT") environment := os.Getenv("WEBHOOKER_ENVIRONMENT")
if environment == "" { if environment == "" {
environment = EnvironmentDev environment = EnvironmentDev
} }
// Validate environment // Validate environment
if environment != EnvironmentDev && environment != EnvironmentProd { if environment != EnvironmentDev &&
return nil, fmt.Errorf("WEBHOOKER_ENVIRONMENT must be either '%s' or '%s', got '%s'", environment != EnvironmentProd {
EnvironmentDev, EnvironmentProd, environment) return nil, fmt.Errorf(
"%w: WEBHOOKER_ENVIRONMENT must be '%s' or '%s', got '%s'",
ErrInvalidEnvironment,
EnvironmentDev, EnvironmentProd, environment,
)
} }
// Set the environment in the config package (for fallback resolution) // Parse the retention sweep interval; a set-but-unparseable value
pkgconfig.SetEnvironment(environment) // is a hard error so fx aborts startup rather than silently using
// the default.
retentionSweepInterval, err := envDuration(
"RETENTION_SWEEP_INTERVAL",
defaultRetentionSweepInterval,
)
if err != nil {
return nil, err
}
// Load configuration values — env vars take precedence over config.yaml // Parse the receiver rate limit; a set-but-unparseable or
// non-positive value is a hard error so fx aborts startup
// rather than silently using the default.
receiverRateLimit, err := envPositiveInt(
"RECEIVER_RATE_LIMIT",
defaultReceiverRateLimit,
)
if err != nil {
return nil, err
}
// Load configuration values from environment variables
s := &Config{ s := &Config{
DBURL: envString("DBURL", "dburl"), DataDir: envString("DATA_DIR"),
Debug: envBool("DEBUG", "debug"), Debug: envBool("DEBUG", false),
MaintenanceMode: envBool("MAINTENANCE_MODE", "maintenanceMode"), MaintenanceMode: envBool("MAINTENANCE_MODE", false),
DevelopmentMode: envBool("DEVELOPMENT_MODE", "developmentMode"),
DevAdminUsername: envString("DEV_ADMIN_USERNAME", "devAdminUsername"),
DevAdminPassword: envString("DEV_ADMIN_PASSWORD", "devAdminPassword"),
Environment: environment, Environment: environment,
MetricsUsername: envString("METRICS_USERNAME", "metricsUsername"), MetricsUsername: envString("METRICS_USERNAME"),
MetricsPassword: envString("METRICS_PASSWORD", "metricsPassword"), MetricsPassword: envString("METRICS_PASSWORD"),
Port: envInt("PORT", "port", 8080), Port: envInt("PORT", defaultPort),
SentryDSN: envSecretString("SENTRY_DSN", "sentryDSN"), SentryDSN: envString("SENTRY_DSN"),
SessionKey: envSecretString("SESSION_KEY", "sessionKey"), RetentionSweepInterval: retentionSweepInterval,
ReceiverRateLimit: receiverRateLimit,
log: log, log: log,
params: &params, params: &params,
} }
// Validate database URL // Set default DataDir. All SQLite databases (main application
if s.DBURL == "" { // DB and per-webhook event DBs) live here. The same default is
return nil, fmt.Errorf("database URL (DBURL) is required") // used regardless of environment; override with DATA_DIR if
} // needed.
if s.DataDir == "" {
// In production, require session key s.DataDir = "/var/lib/webhooker"
if s.IsProd() && s.SessionKey == "" {
return nil, fmt.Errorf("SESSION_KEY is required in production environment")
}
// In development mode, warn if using default session key
if s.IsDev() && s.SessionKey == DevSessionKey {
log.Warn("Using insecure default session key for development mode")
} }
if s.Debug { if s.Debug {
@@ -157,10 +254,12 @@ func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) {
"port", s.Port, "port", s.Port,
"debug", s.Debug, "debug", s.Debug,
"maintenanceMode", s.MaintenanceMode, "maintenanceMode", s.MaintenanceMode,
"developmentMode", s.DevelopmentMode, "dataDir", s.DataDir,
"hasSessionKey", s.SessionKey != "", "retentionSweepInterval", s.RetentionSweepInterval.String(),
"receiverRateLimit", s.ReceiverRateLimit,
"hasSentryDSN", s.SentryDSN != "", "hasSentryDSN", s.SentryDSN != "",
"hasMetricsAuth", s.MetricsUsername != "" && s.MetricsPassword != "", "hasMetricsAuth",
s.MetricsUsername != "" && s.MetricsPassword != "",
) )
return s, nil return s, nil

View File

@@ -1,69 +1,19 @@
package config package config_test
import ( import (
"os" "os"
"testing" "testing"
"time"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx" "go.uber.org/fx"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
pkgconfig "sneak.berlin/go/webhooker/pkg/config"
) )
// createTestConfig creates a test configuration file in memory
func createTestConfig(fs afero.Fs) error {
configYAML := `
environments:
dev:
config:
port: 8080
debug: true
maintenanceMode: false
developmentMode: true
environment: dev
dburl: postgres://test:test@localhost:5432/test_dev?sslmode=disable
metricsUsername: testuser
metricsPassword: testpass
devAdminUsername: devadmin
devAdminPassword: devpass
secrets:
sessionKey: d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=
sentryDSN: ""
prod:
config:
port: $ENV:PORT
debug: $ENV:DEBUG
maintenanceMode: $ENV:MAINTENANCE_MODE
developmentMode: false
environment: prod
dburl: $ENV:DBURL
metricsUsername: $ENV:METRICS_USERNAME
metricsPassword: $ENV:METRICS_PASSWORD
devAdminUsername: ""
devAdminPassword: ""
secrets:
sessionKey: $ENV:SESSION_KEY
sentryDSN: $ENV:SENTRY_DSN
configDefaults:
port: 8080
debug: false
maintenanceMode: false
developmentMode: false
environment: dev
metricsUsername: ""
metricsPassword: ""
devAdminUsername: ""
devAdminPassword: ""
`
return afero.WriteFile(fs, "config.yaml", []byte(configYAML), 0644)
}
func TestEnvironmentConfig(t *testing.T) { func TestEnvironmentConfig(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -75,26 +25,18 @@ func TestEnvironmentConfig(t *testing.T) {
}{ }{
{ {
name: "default is dev", name: "default is dev",
envValue: "",
expectError: false,
isDev: true, isDev: true,
isProd: false, isProd: false,
}, },
{ {
name: "explicit dev", name: "explicit dev",
envValue: "dev", envValue: "dev",
expectError: false,
isDev: true, isDev: true,
isProd: false, isProd: false,
}, },
{ {
name: "explicit prod with session key", name: "explicit prod",
envValue: "prod", envValue: "prod",
envVars: map[string]string{
"SESSION_KEY": "cHJvZC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
"DBURL": "postgres://prod:prod@localhost:5432/prod?sslmode=require",
},
expectError: false,
isDev: false, isDev: false,
isProd: true, isProd: true,
}, },
@@ -107,194 +49,318 @@ func TestEnvironmentConfig(t *testing.T) {
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Create in-memory filesystem with test config // Cannot use t.Parallel() here because t.Setenv
fs := afero.NewMemMapFs() // is incompatible with parallel subtests.
require.NoError(t, createTestConfig(fs))
pkgconfig.SetFs(fs)
// Set environment variable if specified
if tt.envValue != "" { if tt.envValue != "" {
os.Setenv("WEBHOOKER_ENVIRONMENT", tt.envValue) t.Setenv(
defer os.Unsetenv("WEBHOOKER_ENVIRONMENT") "WEBHOOKER_ENVIRONMENT", tt.envValue,
)
} else {
require.NoError(t, os.Unsetenv(
"WEBHOOKER_ENVIRONMENT",
))
} }
// Set additional environment variables
for k, v := range tt.envVars { for k, v := range tt.envVars {
os.Setenv(k, v) t.Setenv(k, v)
defer os.Unsetenv(k)
} }
if tt.expectError { if tt.expectError {
// Use regular fx.New for error cases since fxtest doesn't expose errors the same way testEnvironmentConfigError(t)
var cfg *Config
app := fx.New(
fx.NopLogger, // Suppress fx logs in tests
fx.Provide(
globals.New,
logger.New,
New,
),
fx.Populate(&cfg),
)
assert.Error(t, app.Err())
} else { } else {
// Use fxtest for success cases testEnvironmentConfigSuccess(
var cfg *Config t, tt.isDev, tt.isProd,
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
New,
),
fx.Populate(&cfg),
) )
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(t, tt.isDev, cfg.IsDev())
assert.Equal(t, tt.isProd, cfg.IsProd())
} }
}) })
} }
} }
func TestSessionKeyDefaults(t *testing.T) { func testEnvironmentConfigError(t *testing.T) {
t.Helper()
var cfg *config.Config
app := fx.New(
fx.NopLogger,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
assert.Error(t, app.Err())
}
func testEnvironmentConfigSuccess(
t *testing.T,
isDev, isProd bool,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(t, isDev, cfg.IsDev())
assert.Equal(t, isProd, cfg.IsProd())
}
func TestRetentionSweepInterval(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
environment string set bool
sessionKey string value string
dburl string
expectError bool expectError bool
expectedKey string expected time.Duration
}{ }{
{ {
name: "dev mode with default session key", name: "unset uses default",
environment: "dev", set: false,
sessionKey: "", expected: time.Hour,
expectError: false,
expectedKey: DevSessionKey,
}, },
{ {
name: "dev mode with custom session key", name: "valid value is parsed",
environment: "dev", set: true,
sessionKey: "Y3VzdG9tLXNlc3Npb24ta2V5LTMyLWJ5dGVzLWxvbmchIQ==", value: "15m",
expectError: false, expected: 15 * time.Minute,
expectedKey: "Y3VzdG9tLXNlc3Npb24ta2V5LTMyLWJ5dGVzLWxvbmchIQ==",
}, },
{ {
name: "prod mode with no session key fails", name: "unparseable value fails startup",
environment: "prod", set: true,
sessionKey: "", value: "not-a-duration",
dburl: "postgres://prod:prod@localhost:5432/prod",
expectError: true, expectError: true,
}, },
{
name: "prod mode with session key succeeds",
environment: "prod",
sessionKey: "cHJvZC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
dburl: "postgres://prod:prod@localhost:5432/prod",
expectError: false,
expectedKey: "cHJvZC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Create in-memory filesystem with test config // Cannot use t.Parallel() here because t.Setenv
fs := afero.NewMemMapFs() // is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
// Create custom config for session key tests if tt.set {
configYAML := ` t.Setenv("RETENTION_SWEEP_INTERVAL", tt.value)
environments: } else {
dev: require.NoError(t, os.Unsetenv(
config: "RETENTION_SWEEP_INTERVAL",
environment: dev ))
developmentMode: true
dburl: postgres://test:test@localhost:5432/test_dev
secrets:`
// Only add sessionKey line if it's not empty
if tt.sessionKey != "" {
configYAML += `
sessionKey: ` + tt.sessionKey
} else if tt.environment == "dev" {
// For dev mode with no session key, use the default
configYAML += `
sessionKey: d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=`
}
// Add prod config if testing prod
if tt.environment == "prod" {
configYAML += `
prod:
config:
environment: prod
developmentMode: false
dburl: $ENV:DBURL
secrets:
sessionKey: $ENV:SESSION_KEY`
}
require.NoError(t, afero.WriteFile(fs, "config.yaml", []byte(configYAML), 0644))
pkgconfig.SetFs(fs)
// Clean up any existing env vars
os.Unsetenv("WEBHOOKER_ENVIRONMENT")
os.Unsetenv("SESSION_KEY")
os.Unsetenv("DBURL")
// Set environment variables
os.Setenv("WEBHOOKER_ENVIRONMENT", tt.environment)
defer os.Unsetenv("WEBHOOKER_ENVIRONMENT")
if tt.sessionKey != "" && tt.environment == "prod" {
os.Setenv("SESSION_KEY", tt.sessionKey)
defer os.Unsetenv("SESSION_KEY")
}
if tt.dburl != "" {
os.Setenv("DBURL", tt.dburl)
defer os.Unsetenv("DBURL")
} }
if tt.expectError { if tt.expectError {
// Use regular fx.New for error cases testRetentionSweepIntervalError(t)
var cfg *Config } else {
testRetentionSweepIntervalSuccess(t, tt.expected)
}
})
}
}
func testRetentionSweepIntervalError(t *testing.T) {
t.Helper()
var cfg *config.Config
app := fx.New( app := fx.New(
fx.NopLogger, // Suppress fx logs in tests fx.NopLogger,
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
New, config.New,
), ),
fx.Populate(&cfg), fx.Populate(&cfg),
) )
assert.Error(t, app.Err()) assert.Error(t, app.Err())
} else { }
// Use fxtest for success cases
var cfg *Config func testRetentionSweepIntervalSuccess(
t *testing.T,
expected time.Duration,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New( app := fxtest.New(
t, t,
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
New, config.New,
), ),
fx.Populate(&cfg), fx.Populate(&cfg),
) )
require.NoError(t, app.Err()) require.NoError(t, app.Err())
app.RequireStart() app.RequireStart()
defer app.RequireStop() defer app.RequireStop()
if tt.environment == "dev" && tt.sessionKey == "" { assert.Equal(t, expected, cfg.RetentionSweepInterval)
// Dev mode with no session key uses default }
assert.Equal(t, DevSessionKey, cfg.SessionKey)
} else { func TestDefaultDataDir(t *testing.T) {
assert.Equal(t, tt.expectedKey, cfg.SessionKey) for _, env := range []string{"", "dev", "prod"} {
name := env
if name == "" {
name = "unset"
} }
t.Run("env="+name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
if env != "" {
t.Setenv("WEBHOOKER_ENVIRONMENT", env)
} else {
require.NoError(t, os.Unsetenv(
"WEBHOOKER_ENVIRONMENT",
))
}
require.NoError(t, os.Unsetenv("DATA_DIR"))
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(
t, "/var/lib/webhooker", cfg.DataDir,
)
})
}
}
func TestReceiverRateLimit(t *testing.T) {
tests := []struct {
name string
set bool
value string
expectError bool
expected int
}{
{
name: "unset uses default",
set: false,
expected: 120,
},
{
name: "valid value is parsed",
set: true,
value: "30",
expected: 30,
},
{
name: "unparseable value fails startup",
set: true,
value: "not-a-number",
expectError: true,
},
{
name: "zero fails startup",
set: true,
value: "0",
expectError: true,
},
{
name: "negative fails startup",
set: true,
value: "-5",
expectError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set {
t.Setenv("RECEIVER_RATE_LIMIT", tt.value)
} else {
require.NoError(t, os.Unsetenv(
"RECEIVER_RATE_LIMIT",
))
}
if tt.expectError {
testReceiverRateLimitError(t)
} else {
testReceiverRateLimitSuccess(t, tt.expected)
} }
}) })
} }
} }
func testReceiverRateLimitError(t *testing.T) {
t.Helper()
var cfg *config.Config
app := fx.New(
fx.NopLogger,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
assert.Error(t, app.Err())
}
func testReceiverRateLimitSuccess(
t *testing.T,
expected int,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(t, expected, cfg.ReceiverRateLimit)
}

View File

@@ -11,15 +11,16 @@ import (
// This replaces gorm.Model but uses UUID instead of uint for ID // This replaces gorm.Model but uses UUID instead of uint for ID
type BaseModel struct { type BaseModel struct {
ID string `gorm:"type:uuid;primary_key" json:"id"` ID string `gorm:"type:uuid;primary_key" json:"id"`
CreatedAt time.Time `json:"created_at"` CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updated_at"` UpdatedAt time.Time `json:"updatedAt"`
DeletedAt gorm.DeletedAt `gorm:"index" json:"deleted_at,omitempty"` DeletedAt gorm.DeletedAt `gorm:"index" json:"deletedAt,omitzero"`
} }
// BeforeCreate hook to set UUID before creating a record // BeforeCreate hook to set UUID before creating a record.
func (b *BaseModel) BeforeCreate(tx *gorm.DB) error { func (b *BaseModel) BeforeCreate(_ *gorm.DB) error {
if b.ID == "" { if b.ID == "" {
b.ID = uuid.New().String() b.ID = uuid.New().String()
} }
return nil return nil
} }

View File

@@ -1,9 +1,16 @@
// Package database provides SQLite persistence for webhooks, events, and users.
package database package database
import ( import (
"context" "context"
"crypto/rand"
"database/sql" "database/sql"
"encoding/base64"
"errors"
"fmt"
"log/slog" "log/slog"
"os"
"path/filepath"
"go.uber.org/fx" "go.uber.org/fx"
"gorm.io/driver/sqlite" "gorm.io/driver/sqlite"
@@ -13,30 +20,42 @@ import (
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
// nolint:revive // DatabaseParams is a standard fx naming convention const (
dataDirPerm = 0750
randomPasswordLen = 16
sessionKeyLen = 32
)
//nolint:revive // DatabaseParams is a standard fx naming convention.
type DatabaseParams struct { type DatabaseParams struct {
fx.In fx.In
Config *config.Config Config *config.Config
Logger *logger.Logger Logger *logger.Logger
} }
// Database manages the main SQLite connection and schema migrations.
type Database struct { type Database struct {
db *gorm.DB db *gorm.DB
log *slog.Logger log *slog.Logger
params *DatabaseParams params *DatabaseParams
} }
func New(lc fx.Lifecycle, params DatabaseParams) (*Database, error) { // New creates a Database that connects on fx start and disconnects on stop.
func New(
lc fx.Lifecycle,
params DatabaseParams,
) (*Database, error) {
d := &Database{ d := &Database{
params: &params, params: &params,
log: params.Logger.Get(), log: params.Logger.Get(),
} }
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(_ context.Context) error { // nolint:revive // ctx unused but required by fx OnStart: func(_ context.Context) error {
return d.connect() return d.connect()
}, },
OnStop: func(_ context.Context) error { // nolint:revive // ctx unused but required by fx OnStop: func(_ context.Context) error {
return d.close() return d.close()
}, },
}) })
@@ -44,17 +63,92 @@ func New(lc fx.Lifecycle, params DatabaseParams) (*Database, error) {
return d, nil return d, nil
} }
func (d *Database) connect() error { // DB returns the underlying GORM database handle.
dbURL := d.params.Config.DBURL func (d *Database) DB() *gorm.DB {
if dbURL == "" { return d.db
// Default to SQLite for development }
dbURL = "file:webhooker.db?cache=shared&mode=rwc"
// GetOrCreateSessionKey retrieves the session encryption key from the
// settings table. If no key exists, a cryptographically secure random
// 32-byte key is generated, base64-encoded, and stored for future use.
func (d *Database) GetOrCreateSessionKey() (string, error) {
var setting Setting
result := d.db.Where(
&Setting{Key: "session_key"},
).First(&setting)
if result.Error == nil {
return setting.Value, nil
} }
// First, open the database with the pure Go driver if !errors.Is(result.Error, gorm.ErrRecordNotFound) {
return "", fmt.Errorf(
"failed to query session key: %w",
result.Error,
)
}
// Generate a new cryptographically secure 32-byte key
keyBytes := make([]byte, sessionKeyLen)
_, err := rand.Read(keyBytes)
if err != nil {
return "", fmt.Errorf(
"failed to generate session key: %w",
err,
)
}
encoded := base64.StdEncoding.EncodeToString(keyBytes)
setting = Setting{
Key: "session_key",
Value: encoded,
}
err = d.db.Create(&setting).Error
if err != nil {
return "", fmt.Errorf(
"failed to store session key: %w",
err,
)
}
d.log.Info(
"generated new session key and stored in database",
)
return encoded, nil
}
func (d *Database) connect() error {
// Ensure the data directory exists before opening the database.
dataDir := d.params.Config.DataDir
err := os.MkdirAll(dataDir, dataDirPerm)
if err != nil {
return fmt.Errorf(
"creating data directory %s: %w",
dataDir,
err,
)
}
// Construct the main application database path inside DATA_DIR.
dbPath := filepath.Join(dataDir, "webhooker.db")
dbURL := fmt.Sprintf(
"file:%s?cache=shared&mode=rwc",
dbPath,
)
// Open the database with the pure Go SQLite driver
sqlDB, err := sql.Open("sqlite", dbURL) sqlDB, err := sql.Open("sqlite", dbURL)
if err != nil { if err != nil {
d.log.Error("failed to open database", "error", err) d.log.Error(
"failed to open database",
"error", err,
)
return err return err
} }
@@ -63,12 +157,16 @@ func (d *Database) connect() error {
Conn: sqlDB, Conn: sqlDB,
}, &gorm.Config{}) }, &gorm.Config{})
if err != nil { if err != nil {
d.log.Error("failed to connect to database", "error", err) d.log.Error(
"failed to connect to database",
"error", err,
)
return err return err
} }
d.db = db d.db = db
d.log.Info("connected to database", "database", dbURL) d.log.Info("connected to database", "path", dbPath)
// Run migrations // Run migrations
return d.migrate() return d.migrate()
@@ -76,34 +174,62 @@ func (d *Database) connect() error {
func (d *Database) migrate() error { func (d *Database) migrate() error {
// Run GORM auto-migrations // Run GORM auto-migrations
if err := d.Migrate(); err != nil { err := d.Migrate()
d.log.Error("failed to run database migrations", "error", err) if err != nil {
d.log.Error(
"failed to run database migrations",
"error", err,
)
return err return err
} }
d.log.Info("database migrations completed") d.log.Info("database migrations completed")
// Check if admin user exists // Check if admin user exists
var userCount int64 var userCount int64
if err := d.db.Model(&User{}).Count(&userCount).Error; err != nil {
d.log.Error("failed to count users", "error", err) err = d.db.Model(&User{}).Count(&userCount).Error
if err != nil {
d.log.Error(
"failed to count users",
"error", err,
)
return err return err
} }
if userCount == 0 { if userCount == 0 {
// Create admin user return d.createAdminUser()
}
return nil
}
func (d *Database) createAdminUser() error {
d.log.Info("no users found, creating admin user") d.log.Info("no users found, creating admin user")
// Generate random password // Generate random password
password, err := GenerateRandomPassword(16) password, err := GenerateRandomPassword(
randomPasswordLen,
)
if err != nil { if err != nil {
d.log.Error("failed to generate random password", "error", err) d.log.Error(
"failed to generate random password",
"error", err,
)
return err return err
} }
// Hash the password // Hash the password
hashedPassword, err := HashPassword(password) hashedPassword, err := HashPassword(password)
if err != nil { if err != nil {
d.log.Error("failed to hash password", "error", err) d.log.Error(
"failed to hash password",
"error", err,
)
return err return err
} }
@@ -113,17 +239,22 @@ func (d *Database) migrate() error {
Password: hashedPassword, Password: hashedPassword,
} }
if err := d.db.Create(adminUser).Error; err != nil { err = d.db.Create(adminUser).Error
d.log.Error("failed to create admin user", "error", err) if err != nil {
d.log.Error(
"failed to create admin user",
"error", err,
)
return err return err
} }
// Log the password - this will only happen once on first startup
d.log.Info("admin user created", d.log.Info("admin user created",
"username", "admin", "username", "admin",
"password", password, "password", password,
"message", "SAVE THIS PASSWORD - it will not be shown again!") "message",
} "SAVE THIS PASSWORD - it will not be shown again!",
)
return nil return nil
} }
@@ -134,11 +265,9 @@ func (d *Database) close() error {
if err != nil { if err != nil {
return err return err
} }
return sqlDB.Close() return sqlDB.Close()
} }
return nil return nil
} }
func (d *Database) DB() *gorm.DB {
return d.db
}

View File

@@ -1,74 +1,42 @@
package database package database_test
import ( import (
"context" "context"
"testing" "testing"
"github.com/spf13/afero"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
pkgconfig "sneak.berlin/go/webhooker/pkg/config"
) )
func TestDatabaseConnection(t *testing.T) { func setupTestDB(
// Set up in-memory config so the test does not depend on config.yaml on disk t *testing.T,
fs := afero.NewMemMapFs() ) (*database.Database, *fxtest.Lifecycle) {
testConfigYAML := ` t.Helper()
environments:
dev:
config:
port: 8080
debug: false
maintenanceMode: false
developmentMode: true
environment: dev
dburl: "file::memory:?cache=shared"
secrets:
sessionKey: d2ViaG9va2VyLWRldi1zZXNzaW9uLWtleS1pbnNlY3VyZSE=
sentryDSN: ""
configDefaults:
port: 8080
`
if err := afero.WriteFile(fs, "config.yaml", []byte(testConfigYAML), 0644); err != nil {
t.Fatalf("Failed to write test config: %v", err)
}
pkgconfig.SetFs(fs)
// Set up test dependencies
lc := fxtest.NewLifecycle(t) lc := fxtest.NewLifecycle(t)
// Create globals g := &globals.Globals{
globals.Appname = "webhooker-test" Appname: "webhooker-test",
globals.Version = "test" Version: "test",
globals.Buildarch = "test"
g, err := globals.New(lc)
if err != nil {
t.Fatalf("Failed to create globals: %v", err)
} }
// Create logger l, err := logger.New(
l, err := logger.New(lc, logger.LoggerParams{Globals: g}) lc,
logger.LoggerParams{Globals: g},
)
if err != nil { if err != nil {
t.Fatalf("Failed to create logger: %v", err) t.Fatalf("Failed to create logger: %v", err)
} }
// Create config c := &config.Config{
c, err := config.New(lc, config.ConfigParams{ DataDir: t.TempDir(),
Globals: g, Environment: "dev",
Logger: l,
})
if err != nil {
t.Fatalf("Failed to create config: %v", err)
} }
// Override DBURL to use a temp file-based SQLite (in-memory doesn't persist across connections) db, err := database.New(lc, database.DatabaseParams{
c.DBURL = "file:" + t.TempDir() + "/test.db?cache=shared&mode=rwc"
// Create database
db, err := New(lc, DatabaseParams{
Config: c, Config: c,
Logger: l, Logger: l,
}) })
@@ -76,31 +44,45 @@ configDefaults:
t.Fatalf("Failed to create database: %v", err) t.Fatalf("Failed to create database: %v", err)
} }
// Start lifecycle (this will trigger the connection) return db, lc
}
func TestDatabaseConnection(t *testing.T) {
t.Parallel()
db, lc := setupTestDB(t)
ctx := context.Background() ctx := context.Background()
err = lc.Start(ctx)
err := lc.Start(ctx)
if err != nil { if err != nil {
t.Fatalf("Failed to connect to database: %v", err) t.Fatalf("Failed to connect to database: %v", err)
} }
defer func() { defer func() {
if stopErr := lc.Stop(ctx); stopErr != nil { stopErr := lc.Stop(ctx)
t.Errorf("Failed to stop lifecycle: %v", stopErr) if stopErr != nil {
t.Errorf(
"Failed to stop lifecycle: %v",
stopErr,
)
} }
}() }()
// Verify we can get the DB instance
if db.DB() == nil { if db.DB() == nil {
t.Error("Expected non-nil database connection") t.Error("Expected non-nil database connection")
} }
// Test that we can perform a simple query
var result int var result int
err = db.DB().Raw("SELECT 1").Scan(&result).Error err = db.DB().Raw("SELECT 1").Scan(&result).Error
if err != nil { if err != nil {
t.Fatalf("Failed to execute test query: %v", err) t.Fatalf("Failed to execute test query: %v", err)
} }
if result != 1 { if result != 1 {
t.Errorf("Expected query result to be 1, got %d", result) t.Errorf(
"Expected query result to be 1, got %d",
result,
)
} }
} }

View File

@@ -0,0 +1,31 @@
package database
import (
"context"
"log/slog"
"os"
"time"
)
// NewTestRetentionReaper builds a RetentionReaper backed by the given
// main database and per-webhook database manager, without the fx
// lifecycle. Intended for tests.
func NewTestRetentionReaper(
db *Database,
mgr *WebhookDBManager,
) *RetentionReaper {
return &RetentionReaper{
db: db,
dbManager: mgr,
log: slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
)),
interval: time.Hour,
}
}
// ExportSweep runs a single retention sweep synchronously for tests.
func (r *RetentionReaper) ExportSweep(ctx context.Context) {
r.sweep(ctx)
}

View File

@@ -6,11 +6,11 @@ import "time"
type APIKey struct { type APIKey struct {
BaseModel BaseModel
UserID string `gorm:"type:uuid;not null" json:"user_id"` UserID string `gorm:"type:uuid;not null" json:"userId"`
Key string `gorm:"uniqueIndex;not null" json:"key"` Key string `gorm:"uniqueIndex;not null" json:"key"`
Description string `json:"description"` Description string `json:"description"`
LastUsedAt *time.Time `json:"last_used_at,omitempty"` LastUsedAt *time.Time `json:"lastUsedAt,omitempty"`
// Relations // Relations
User User `json:"user,omitempty"` User User `json:"user,omitzero"`
} }

View File

@@ -3,6 +3,7 @@ package database
// DeliveryStatus represents the status of a delivery // DeliveryStatus represents the status of a delivery
type DeliveryStatus string type DeliveryStatus string
// Delivery status values.
const ( const (
DeliveryStatusPending DeliveryStatus = "pending" DeliveryStatusPending DeliveryStatus = "pending"
DeliveryStatusDelivered DeliveryStatus = "delivered" DeliveryStatusDelivered DeliveryStatus = "delivered"
@@ -14,12 +15,12 @@ const (
type Delivery struct { type Delivery struct {
BaseModel BaseModel
EventID string `gorm:"type:uuid;not null" json:"event_id"` EventID string `gorm:"type:uuid;not null" json:"eventId"`
TargetID string `gorm:"type:uuid;not null" json:"target_id"` TargetID string `gorm:"type:uuid;not null" json:"targetId"`
Status DeliveryStatus `gorm:"not null;default:'pending'" json:"status"` Status DeliveryStatus `gorm:"not null;default:'pending'" json:"status"`
// Relations // Relations
Event Event `json:"event,omitempty"` Event Event `json:"event,omitzero"`
Target Target `json:"target,omitempty"` Target Target `json:"target,omitzero"`
DeliveryResults []DeliveryResult `json:"delivery_results,omitempty"` DeliveryResults []DeliveryResult `json:"deliveryResults,omitempty"`
} }

View File

@@ -4,14 +4,14 @@ package database
type DeliveryResult struct { type DeliveryResult struct {
BaseModel BaseModel
DeliveryID string `gorm:"type:uuid;not null" json:"delivery_id"` DeliveryID string `gorm:"type:uuid;not null" json:"deliveryId"`
AttemptNum int `gorm:"not null" json:"attempt_num"` AttemptNum int `gorm:"not null" json:"attemptNum"`
Success bool `json:"success"` Success bool `json:"success"`
StatusCode int `json:"status_code,omitempty"` StatusCode int `json:"statusCode,omitempty"`
ResponseBody string `gorm:"type:text" json:"response_body,omitempty"` ResponseBody string `gorm:"type:text" json:"responseBody,omitempty"`
Error string `json:"error,omitempty"` Error string `json:"error,omitempty"`
Duration int64 `json:"duration_ms"` // Duration in milliseconds Duration int64 `json:"durationMs"` // Duration in milliseconds
// Relations // Relations
Delivery Delivery `json:"delivery,omitempty"` Delivery Delivery `json:"delivery,omitzero"`
} }

View File

@@ -4,11 +4,11 @@ package database
type Entrypoint struct { type Entrypoint struct {
BaseModel BaseModel
WebhookID string `gorm:"type:uuid;not null" json:"webhook_id"` WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
Path string `gorm:"uniqueIndex;not null" json:"path"` // URL path for this entrypoint Path string `gorm:"uniqueIndex;not null" json:"path"` // URL path for this entrypoint
Description string `json:"description"` Description string `json:"description"`
Active bool `gorm:"default:true" json:"active"` Active bool `gorm:"default:true" json:"active"`
// Relations // Relations
Webhook Webhook `json:"webhook,omitempty"` Webhook Webhook `json:"webhook,omitzero"`
} }

View File

@@ -4,17 +4,17 @@ package database
type Event struct { type Event struct {
BaseModel BaseModel
WebhookID string `gorm:"type:uuid;not null" json:"webhook_id"` WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
EntrypointID string `gorm:"type:uuid;not null" json:"entrypoint_id"` EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
// Request data // Request data
Method string `gorm:"not null" json:"method"` Method string `gorm:"not null" json:"method"`
Headers string `gorm:"type:text" json:"headers"` // JSON Headers string `gorm:"type:text" json:"headers"` // JSON
Body string `gorm:"type:text" json:"body"` Body string `gorm:"type:text" json:"body"`
ContentType string `json:"content_type"` ContentType string `json:"contentType"`
// Relations // Relations
Webhook Webhook `json:"webhook,omitempty"` Webhook Webhook `json:"webhook,omitzero"`
Entrypoint Entrypoint `json:"entrypoint,omitempty"` Entrypoint Entrypoint `json:"entrypoint,omitzero"`
Deliveries []Delivery `json:"deliveries,omitempty"` Deliveries []Delivery `json:"deliveries,omitempty"`
} }

View File

@@ -0,0 +1,8 @@
package database
// Setting stores application-level key-value configuration.
// Used for auto-generated values like the session encryption key.
type Setting struct {
Key string `gorm:"primaryKey" json:"key"`
Value string `gorm:"type:text;not null" json:"value"`
}

View File

@@ -3,18 +3,19 @@ package database
// TargetType represents the type of delivery target // TargetType represents the type of delivery target
type TargetType string type TargetType string
// Target type values.
const ( const (
TargetTypeHTTP TargetType = "http" TargetTypeHTTP TargetType = "http"
TargetTypeRetry TargetType = "retry"
TargetTypeDatabase TargetType = "database" TargetTypeDatabase TargetType = "database"
TargetTypeLog TargetType = "log" TargetTypeLog TargetType = "log"
TargetTypeSlack TargetType = "slack"
) )
// Target represents a delivery target for a webhook // Target represents a delivery target for a webhook
type Target struct { type Target struct {
BaseModel BaseModel
WebhookID string `gorm:"type:uuid;not null" json:"webhook_id"` WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
Name string `gorm:"not null" json:"name"` Name string `gorm:"not null" json:"name"`
Type TargetType `gorm:"not null" json:"type"` Type TargetType `gorm:"not null" json:"type"`
Active bool `gorm:"default:true" json:"active"` Active bool `gorm:"default:true" json:"active"`
@@ -22,11 +23,11 @@ type Target struct {
// Configuration fields (JSON stored based on type) // Configuration fields (JSON stored based on type)
Config string `gorm:"type:text" json:"config"` // JSON configuration Config string `gorm:"type:text" json:"config"` // JSON configuration
// For retry targets // For HTTP targets (max_retries=0 means fire-and-forget, >0 enables retries with backoff)
MaxRetries int `json:"max_retries,omitempty"` MaxRetries int `json:"maxRetries,omitempty"`
MaxQueueSize int `json:"max_queue_size,omitempty"` MaxQueueSize int `json:"maxQueueSize,omitempty"`
// Relations // Relations
Webhook Webhook `json:"webhook,omitempty"` Webhook Webhook `json:"webhook,omitzero"`
Deliveries []Delivery `json:"deliveries,omitempty"` Deliveries []Delivery `json:"deliveries,omitempty"`
} }

View File

@@ -9,5 +9,5 @@ type User struct {
// Relations // Relations
Webhooks []Webhook `json:"webhooks,omitempty"` Webhooks []Webhook `json:"webhooks,omitempty"`
APIKeys []APIKey `json:"api_keys,omitempty"` APIKeys []APIKey `json:"apiKeys,omitempty"`
} }

View File

@@ -4,13 +4,13 @@ package database
type Webhook struct { type Webhook struct {
BaseModel BaseModel
UserID string `gorm:"type:uuid;not null" json:"user_id"` UserID string `gorm:"type:uuid;not null" json:"userId"`
Name string `gorm:"not null" json:"name"` Name string `gorm:"not null" json:"name"`
Description string `json:"description"` Description string `json:"description"`
RetentionDays int `gorm:"default:30" json:"retention_days"` // Days to retain events RetentionDays int `gorm:"default:30" json:"retentionDays"` // Days to retain events
// Relations // Relations
User User `json:"user,omitempty"` User User `json:"user,omitzero"`
Entrypoints []Entrypoint `json:"entrypoints,omitempty"` Entrypoints []Entrypoint `json:"entrypoints,omitempty"`
Targets []Target `json:"targets,omitempty"` Targets []Target `json:"targets,omitempty"`
} }

View File

@@ -1,15 +1,16 @@
package database package database
// Migrate runs database migrations for all models // Migrate runs database migrations for the main application database.
// Only configuration-tier models are stored in the main database.
// Event-tier models (Event, Delivery, DeliveryResult) live in
// per-webhook dedicated databases managed by WebhookDBManager.
func (d *Database) Migrate() error { func (d *Database) Migrate() error {
return d.db.AutoMigrate( return d.db.AutoMigrate(
&Setting{},
&User{}, &User{},
&APIKey{}, &APIKey{},
&Webhook{}, &Webhook{},
&Entrypoint{}, &Entrypoint{},
&Target{}, &Target{},
&Event{},
&Delivery{},
&DeliveryResult{},
) )
} }

View File

@@ -4,6 +4,7 @@ import (
"crypto/rand" "crypto/rand"
"crypto/subtle" "crypto/subtle"
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"math/big" "math/big"
"strings" "strings"
@@ -20,6 +21,23 @@ const (
argon2SaltLen = 16 argon2SaltLen = 16
) )
// hashParts is the expected number of $-separated segments
// in an encoded Argon2id hash string.
const hashParts = 6
// minPasswordComplexityLen is the minimum password length that
// triggers per-character-class complexity enforcement.
const minPasswordComplexityLen = 4
// Sentinel errors returned by decodeHash.
var (
errInvalidHashFormat = errors.New("invalid hash format")
errInvalidAlgorithm = errors.New("invalid algorithm")
errIncompatibleVersion = errors.New("incompatible argon2 version")
errSaltLengthOutOfRange = errors.New("salt length out of range")
errHashLengthOutOfRange = errors.New("hash length out of range")
)
// PasswordConfig holds Argon2 configuration // PasswordConfig holds Argon2 configuration
type PasswordConfig struct { type PasswordConfig struct {
Time uint32 Time uint32
@@ -46,26 +64,44 @@ func HashPassword(password string) (string, error) {
// Generate a salt // Generate a salt
salt := make([]byte, config.SaltLen) salt := make([]byte, config.SaltLen)
if _, err := rand.Read(salt); err != nil {
_, err := rand.Read(salt)
if err != nil {
return "", err return "", err
} }
// Generate the hash // Generate the hash
hash := argon2.IDKey([]byte(password), salt, config.Time, config.Memory, config.Threads, config.KeyLen) hash := argon2.IDKey(
[]byte(password),
salt,
config.Time,
config.Memory,
config.Threads,
config.KeyLen,
)
// Encode the hash and parameters // Encode the hash and parameters
b64Salt := base64.RawStdEncoding.EncodeToString(salt) b64Salt := base64.RawStdEncoding.EncodeToString(salt)
b64Hash := base64.RawStdEncoding.EncodeToString(hash) b64Hash := base64.RawStdEncoding.EncodeToString(hash)
// Format: $argon2id$v=19$m=65536,t=1,p=4$salt$hash // Format: $argon2id$v=19$m=65536,t=1,p=4$salt$hash
encoded := fmt.Sprintf("$argon2id$v=%d$m=%d,t=%d,p=%d$%s$%s", encoded := fmt.Sprintf(
argon2.Version, config.Memory, config.Time, config.Threads, b64Salt, b64Hash) "$argon2id$v=%d$m=%d,t=%d,p=%d$%s$%s",
argon2.Version,
config.Memory,
config.Time,
config.Threads,
b64Salt,
b64Hash,
)
return encoded, nil return encoded, nil
} }
// VerifyPassword checks if the provided password matches the hash // VerifyPassword checks if the provided password matches the hash
func VerifyPassword(password, encodedHash string) (bool, error) { func VerifyPassword(
password, encodedHash string,
) (bool, error) {
// Extract parameters and hash from encoded string // Extract parameters and hash from encoded string
config, salt, hash, err := decodeHash(encodedHash) config, salt, hash, err := decodeHash(encodedHash)
if err != nil { if err != nil {
@@ -73,60 +109,119 @@ func VerifyPassword(password, encodedHash string) (bool, error) {
} }
// Generate hash of the provided password // Generate hash of the provided password
otherHash := argon2.IDKey([]byte(password), salt, config.Time, config.Memory, config.Threads, config.KeyLen) otherHash := argon2.IDKey(
[]byte(password),
salt,
config.Time,
config.Memory,
config.Threads,
config.KeyLen,
)
// Compare hashes using constant time comparison // Compare hashes using constant time comparison
return subtle.ConstantTimeCompare(hash, otherHash) == 1, nil return subtle.ConstantTimeCompare(hash, otherHash) == 1, nil
} }
// decodeHash extracts parameters, salt, and hash from an encoded hash string // decodeHash extracts parameters, salt, and hash from an
func decodeHash(encodedHash string) (*PasswordConfig, []byte, []byte, error) { // encoded hash string.
func decodeHash(
encodedHash string,
) (*PasswordConfig, []byte, []byte, error) {
parts := strings.Split(encodedHash, "$") parts := strings.Split(encodedHash, "$")
if len(parts) != 6 { if len(parts) != hashParts {
return nil, nil, nil, fmt.Errorf("invalid hash format") return nil, nil, nil, errInvalidHashFormat
} }
if parts[1] != "argon2id" { if parts[1] != "argon2id" {
return nil, nil, nil, fmt.Errorf("invalid algorithm") return nil, nil, nil, errInvalidAlgorithm
} }
var version int version, err := parseVersion(parts[2])
if _, err := fmt.Sscanf(parts[2], "v=%d", &version); err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, err
} }
if version != argon2.Version { if version != argon2.Version {
return nil, nil, nil, fmt.Errorf("incompatible argon2 version") return nil, nil, nil, errIncompatibleVersion
} }
config := &PasswordConfig{} config, err := parseParams(parts[3])
if _, err := fmt.Sscanf(parts[3], "m=%d,t=%d,p=%d", &config.Memory, &config.Time, &config.Threads); err != nil {
return nil, nil, nil, err
}
salt, err := base64.RawStdEncoding.DecodeString(parts[4])
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, err
} }
saltLen := len(salt)
if saltLen < 0 || saltLen > int(^uint32(0)) {
return nil, nil, nil, fmt.Errorf("salt length out of range")
}
config.SaltLen = uint32(saltLen) // nolint:gosec // checked above
hash, err := base64.RawStdEncoding.DecodeString(parts[5]) salt, err := decodeSalt(parts[4])
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, err
} }
hashLen := len(hash)
if hashLen < 0 || hashLen > int(^uint32(0)) { config.SaltLen = uint32(len(salt)) //nolint:gosec // validated in decodeSalt
return nil, nil, nil, fmt.Errorf("hash length out of range")
hash, err := decodeHashBytes(parts[5])
if err != nil {
return nil, nil, nil, err
} }
config.KeyLen = uint32(hashLen) // nolint:gosec // checked above
config.KeyLen = uint32(len(hash)) //nolint:gosec // validated in decodeHashBytes
return config, salt, hash, nil return config, salt, hash, nil
} }
// GenerateRandomPassword generates a cryptographically secure random password func parseVersion(s string) (int, error) {
var version int
_, err := fmt.Sscanf(s, "v=%d", &version)
if err != nil {
return 0, fmt.Errorf("parsing version: %w", err)
}
return version, nil
}
func parseParams(s string) (*PasswordConfig, error) {
config := &PasswordConfig{}
_, err := fmt.Sscanf(
s, "m=%d,t=%d,p=%d",
&config.Memory, &config.Time, &config.Threads,
)
if err != nil {
return nil, fmt.Errorf("parsing params: %w", err)
}
return config, nil
}
func decodeSalt(s string) ([]byte, error) {
salt, err := base64.RawStdEncoding.DecodeString(s)
if err != nil {
return nil, fmt.Errorf("decoding salt: %w", err)
}
saltLen := len(salt)
if saltLen < 0 || saltLen > int(^uint32(0)) {
return nil, errSaltLengthOutOfRange
}
return salt, nil
}
func decodeHashBytes(s string) ([]byte, error) {
hash, err := base64.RawStdEncoding.DecodeString(s)
if err != nil {
return nil, fmt.Errorf("decoding hash: %w", err)
}
hashLen := len(hash)
if hashLen < 0 || hashLen > int(^uint32(0)) {
return nil, errHashLengthOutOfRange
}
return hash, nil
}
// GenerateRandomPassword generates a cryptographically secure
// random password.
func GenerateRandomPassword(length int) (string, error) { func GenerateRandomPassword(length int) (string, error) {
const ( const (
uppercase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" uppercase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
@@ -141,27 +236,27 @@ func GenerateRandomPassword(length int) (string, error) {
// Create password slice // Create password slice
password := make([]byte, length) password := make([]byte, length)
// Ensure at least one character from each set for password complexity // Ensure at least one character from each set
if length >= 4 { if length >= minPasswordComplexityLen {
// Get one character from each set
password[0] = uppercase[cryptoRandInt(len(uppercase))] password[0] = uppercase[cryptoRandInt(len(uppercase))]
password[1] = lowercase[cryptoRandInt(len(lowercase))] password[1] = lowercase[cryptoRandInt(len(lowercase))]
password[2] = digits[cryptoRandInt(len(digits))] password[2] = digits[cryptoRandInt(len(digits))]
password[3] = special[cryptoRandInt(len(special))] password[3] = special[cryptoRandInt(len(special))]
// Fill the rest randomly from all characters // Fill the rest randomly from all characters
for i := 4; i < length; i++ { for i := minPasswordComplexityLen; i < length; i++ {
password[i] = allChars[cryptoRandInt(len(allChars))] password[i] = allChars[cryptoRandInt(len(allChars))]
} }
// Shuffle the password to avoid predictable pattern // Shuffle the password to avoid predictable pattern
for i := len(password) - 1; i > 0; i-- { for i := range len(password) - 1 {
j := cryptoRandInt(i + 1) j := cryptoRandInt(len(password) - i)
password[i], password[j] = password[j], password[i] idx := len(password) - 1 - i
password[idx], password[j] = password[j], password[idx]
} }
} else { } else {
// For very short passwords, just use all characters // For very short passwords, just use all characters
for i := 0; i < length; i++ { for i := range length {
password[i] = allChars[cryptoRandInt(len(allChars))] password[i] = allChars[cryptoRandInt(len(allChars))]
} }
} }
@@ -169,16 +264,17 @@ func GenerateRandomPassword(length int) (string, error) {
return string(password), nil return string(password), nil
} }
// cryptoRandInt generates a cryptographically secure random integer in [0, max) // cryptoRandInt generates a cryptographically secure random
func cryptoRandInt(max int) int { // integer in [0, upperBound).
if max <= 0 { func cryptoRandInt(upperBound int) int {
panic("max must be positive") if upperBound <= 0 {
panic("upperBound must be positive")
} }
// Calculate the maximum valid value to avoid modulo bias nBig, err := rand.Int(
// For example, if max=200 and we have 256 possible values, rand.Reader,
// we only accept values 0-199 (reject 200-255) big.NewInt(int64(upperBound)),
nBig, err := rand.Int(rand.Reader, big.NewInt(int64(max))) )
if err != nil { if err != nil {
panic(fmt.Sprintf("crypto/rand error: %v", err)) panic(fmt.Sprintf("crypto/rand error: %v", err))
} }

View File

@@ -1,11 +1,15 @@
package database package database_test
import ( import (
"strings" "strings"
"testing" "testing"
"sneak.berlin/go/webhooker/internal/database"
) )
func TestGenerateRandomPassword(t *testing.T) { func TestGenerateRandomPassword(t *testing.T) {
t.Parallel()
tests := []struct { tests := []struct {
name string name string
length int length int
@@ -18,109 +22,172 @@ func TestGenerateRandomPassword(t *testing.T) {
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
password, err := GenerateRandomPassword(tt.length) t.Parallel()
password, err := database.GenerateRandomPassword(
tt.length,
)
if err != nil { if err != nil {
t.Fatalf("GenerateRandomPassword() error = %v", err) t.Fatalf(
"GenerateRandomPassword() error = %v",
err,
)
} }
if len(password) != tt.length { if len(password) != tt.length {
t.Errorf("Password length = %v, want %v", len(password), tt.length) t.Errorf(
"Password length = %v, want %v",
len(password), tt.length,
)
} }
// For passwords >= 4 chars, check complexity checkPasswordComplexity(
if tt.length >= 4 { t, password, tt.length,
hasUpper := false )
hasLower := false
hasDigit := false
hasSpecial := false
for _, char := range password {
switch {
case char >= 'A' && char <= 'Z':
hasUpper = true
case char >= 'a' && char <= 'z':
hasLower = true
case char >= '0' && char <= '9':
hasDigit = true
case strings.ContainsRune("!@#$%^&*()_+-=[]{}|;:,.<>?", char):
hasSpecial = true
}
}
if !hasUpper || !hasLower || !hasDigit || !hasSpecial {
t.Errorf("Password lacks required complexity: upper=%v, lower=%v, digit=%v, special=%v",
hasUpper, hasLower, hasDigit, hasSpecial)
}
}
}) })
} }
} }
func checkPasswordComplexity(
t *testing.T,
password string,
length int,
) {
t.Helper()
// For passwords >= 4 chars, check complexity
if length < 4 {
return
}
flags := classifyChars(password)
if !flags[0] || !flags[1] || !flags[2] || !flags[3] {
t.Errorf(
"Password lacks required complexity: "+
"upper=%v, lower=%v, digit=%v, special=%v",
flags[0], flags[1], flags[2], flags[3],
)
}
}
func classifyChars(s string) [4]bool {
var flags [4]bool // upper, lower, digit, special
for _, char := range s {
switch {
case char >= 'A' && char <= 'Z':
flags[0] = true
case char >= 'a' && char <= 'z':
flags[1] = true
case char >= '0' && char <= '9':
flags[2] = true
case strings.ContainsRune(
"!@#$%^&*()_+-=[]{}|;:,.<>?",
char,
):
flags[3] = true
}
}
return flags
}
func TestGenerateRandomPasswordUniqueness(t *testing.T) { func TestGenerateRandomPasswordUniqueness(t *testing.T) {
t.Parallel()
// Generate multiple passwords and ensure they're different // Generate multiple passwords and ensure they're different
passwords := make(map[string]bool) passwords := make(map[string]bool)
const numPasswords = 100 const numPasswords = 100
for i := 0; i < numPasswords; i++ { for range numPasswords {
password, err := GenerateRandomPassword(16) password, err := database.GenerateRandomPassword(16)
if err != nil { if err != nil {
t.Fatalf("GenerateRandomPassword() error = %v", err) t.Fatalf(
"GenerateRandomPassword() error = %v",
err,
)
} }
if passwords[password] { if passwords[password] {
t.Errorf("Duplicate password generated: %s", password) t.Errorf(
"Duplicate password generated: %s",
password,
)
} }
passwords[password] = true passwords[password] = true
} }
} }
func TestHashPassword(t *testing.T) { func TestHashPassword(t *testing.T) {
t.Parallel()
password := "testPassword123!" password := "testPassword123!"
hash, err := HashPassword(password) hash, err := database.HashPassword(password)
if err != nil { if err != nil {
t.Fatalf("HashPassword() error = %v", err) t.Fatalf("HashPassword() error = %v", err)
} }
// Check that hash has correct format // Check that hash has correct format
if !strings.HasPrefix(hash, "$argon2id$") { if !strings.HasPrefix(hash, "$argon2id$") {
t.Errorf("Hash doesn't have correct prefix: %s", hash) t.Errorf(
"Hash doesn't have correct prefix: %s",
hash,
)
} }
// Verify password // Verify password
valid, err := VerifyPassword(password, hash) valid, err := database.VerifyPassword(password, hash)
if err != nil { if err != nil {
t.Fatalf("VerifyPassword() error = %v", err) t.Fatalf("VerifyPassword() error = %v", err)
} }
if !valid { if !valid {
t.Error("VerifyPassword() returned false for correct password") t.Error(
"VerifyPassword() returned false " +
"for correct password",
)
} }
// Verify wrong password fails // Verify wrong password fails
valid, err = VerifyPassword("wrongPassword", hash) valid, err = database.VerifyPassword(
"wrongPassword", hash,
)
if err != nil { if err != nil {
t.Fatalf("VerifyPassword() error = %v", err) t.Fatalf("VerifyPassword() error = %v", err)
} }
if valid { if valid {
t.Error("VerifyPassword() returned true for wrong password") t.Error(
"VerifyPassword() returned true " +
"for wrong password",
)
} }
} }
func TestHashPasswordUniqueness(t *testing.T) { func TestHashPasswordUniqueness(t *testing.T) {
t.Parallel()
password := "testPassword123!" password := "testPassword123!"
// Same password should produce different hashes due to salt // Same password should produce different hashes
hash1, err := HashPassword(password) hash1, err := database.HashPassword(password)
if err != nil { if err != nil {
t.Fatalf("HashPassword() error = %v", err) t.Fatalf("HashPassword() error = %v", err)
} }
hash2, err := HashPassword(password) hash2, err := database.HashPassword(password)
if err != nil { if err != nil {
t.Fatalf("HashPassword() error = %v", err) t.Fatalf("HashPassword() error = %v", err)
} }
if hash1 == hash2 { if hash1 == hash2 {
t.Error("Same password produced identical hashes (salt not working)") t.Error(
"Same password produced identical hashes " +
"(salt not working)",
)
} }
} }

View File

@@ -0,0 +1,252 @@
package database
import (
"context"
"fmt"
"log/slog"
"sync"
"time"
"go.uber.org/fx"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/logger"
)
// hoursPerDay converts a RetentionDays count into hours for cutoff
// computation.
const hoursPerDay = 24
// RetentionReaperParams holds the fx dependencies for the
// RetentionReaper.
type RetentionReaperParams struct {
fx.In
Config *config.Config
Database *Database
DBManager *WebhookDBManager
Logger *logger.Logger
}
// RetentionReaper periodically deletes expired events (and their
// dependent deliveries and delivery results) from each per-webhook
// database, enforcing every webhook's RetentionDays. Rows are removed
// permanently so that per-webhook SQLite files do not grow without
// bound.
type RetentionReaper struct {
db *Database
dbManager *WebhookDBManager
log *slog.Logger
interval time.Duration
cancel context.CancelFunc
wg sync.WaitGroup
}
// NewRetentionReaper creates the retention reaper and registers its
// fx lifecycle hooks. The background sweep loop starts on OnStart and
// stops cleanly on OnStop via context cancellation.
func NewRetentionReaper(
lc fx.Lifecycle,
params RetentionReaperParams,
) *RetentionReaper {
r := &RetentionReaper{
db: params.Database,
dbManager: params.DBManager,
log: params.Logger.Get(),
interval: params.Config.RetentionSweepInterval,
}
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
r.start(ctx)
return nil
},
OnStop: func(_ context.Context) error {
r.stop()
return nil
},
})
return r
}
func (r *RetentionReaper) start(ctx context.Context) {
ctx, cancel := context.WithCancel(ctx)
r.cancel = cancel
r.wg.Add(1)
go r.run(ctx)
r.log.Info(
"retention reaper started",
"interval", r.interval.String(),
)
}
func (r *RetentionReaper) stop() {
r.log.Info("retention reaper stopping")
if r.cancel != nil {
r.cancel()
}
r.wg.Wait()
r.log.Info("retention reaper stopped")
}
func (r *RetentionReaper) run(ctx context.Context) {
defer r.wg.Done()
ticker := time.NewTicker(r.interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
r.sweep(ctx)
}
}
}
// sweep lists every webhook from the main database and reaps expired
// rows from each per-webhook database whose RetentionDays is positive.
func (r *RetentionReaper) sweep(ctx context.Context) {
var webhooks []Webhook
err := r.db.DB().
Model(&Webhook{}).
Find(&webhooks).Error
if err != nil {
r.log.Error(
"retention sweep: failed to list webhooks",
"error", err,
)
return
}
for i := range webhooks {
select {
case <-ctx.Done():
return
default:
}
wh := webhooks[i]
// RetentionDays of zero or less means retain forever.
if wh.RetentionDays <= 0 {
continue
}
// Nothing to reap if the per-webhook database has never
// been created.
if !r.dbManager.DBExists(wh.ID) {
continue
}
r.reapWebhook(wh.ID, wh.RetentionDays)
}
}
// reapWebhook removes every expired event (and its dependents) from a
// single webhook's database.
func (r *RetentionReaper) reapWebhook(
webhookID string,
retentionDays int,
) {
db, err := r.dbManager.GetDB(webhookID)
if err != nil {
r.log.Error(
"retention sweep: failed to open webhook database",
"webhook_id", webhookID,
"error", err,
)
return
}
cutoff := time.Now().Add(
-time.Duration(retentionDays*hoursPerDay) * time.Hour,
)
deleted, err := reapExpired(db, cutoff)
if err != nil {
r.log.Error(
"retention sweep: failed to reap expired events",
"webhook_id", webhookID,
"error", err,
)
return
}
if deleted > 0 {
r.log.Info(
"retention sweep: reaped expired events",
"webhook_id", webhookID,
"retention_days", retentionDays,
"events_deleted", deleted,
)
}
}
// reapExpired hard-deletes, in foreign-key-safe order, the delivery
// results, deliveries, and events associated with events older than
// cutoff. Deletes are unscoped so rows are physically removed rather
// than soft-deleted, reclaiming disk. It returns the number of events
// deleted.
func reapExpired(db *gorm.DB, cutoff time.Time) (int64, error) {
// Fresh subqueries are built per statement to avoid reusing a
// mutated builder across executions.
expiredEventIDs := func() *gorm.DB {
return db.Model(&Event{}).
Select("id").
Where("created_at < ?", cutoff)
}
expiredDeliveryIDs := func() *gorm.DB {
return db.Model(&Delivery{}).
Select("id").
Where("event_id IN (?)", expiredEventIDs())
}
// 1. Delivery results whose delivery belongs to an expired event.
res := db.Unscoped().
Where("delivery_id IN (?)", expiredDeliveryIDs()).
Delete(&DeliveryResult{})
if res.Error != nil {
return 0, fmt.Errorf(
"deleting expired delivery results: %w",
res.Error,
)
}
// 2. Deliveries belonging to an expired event.
del := db.Unscoped().
Where("event_id IN (?)", expiredEventIDs()).
Delete(&Delivery{})
if del.Error != nil {
return 0, fmt.Errorf(
"deleting expired deliveries: %w",
del.Error,
)
}
// 3. The expired events themselves.
ev := db.Unscoped().
Where("created_at < ?", cutoff).
Delete(&Event{})
if ev.Error != nil {
return 0, fmt.Errorf(
"deleting expired events: %w",
ev.Error,
)
}
return ev.RowsAffected, nil
}

View File

@@ -0,0 +1,277 @@
package database_test
import (
"context"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"gorm.io/gorm"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger"
)
// retentionTestEnv bundles the pieces a retention test drives.
type retentionTestEnv struct {
reaper *database.RetentionReaper
mainDB *database.Database
mgr *database.WebhookDBManager
}
func setupRetentionTest(t *testing.T) *retentionTestEnv {
t.Helper()
lc := fxtest.NewLifecycle(t)
g := &globals.Globals{
Appname: "webhooker-test",
Version: "test",
}
l, err := logger.New(lc, logger.LoggerParams{Globals: g})
require.NoError(t, err)
cfg := &config.Config{
DataDir: t.TempDir(),
Environment: "dev",
}
mainDB, err := database.New(lc, database.DatabaseParams{
Config: cfg,
Logger: l,
})
require.NoError(t, err)
mgr, err := database.NewWebhookDBManager(
lc,
database.WebhookDBManagerParams{Config: cfg, Logger: l},
)
require.NoError(t, err)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
t.Cleanup(func() { require.NoError(t, lc.Stop(ctx)) })
return &retentionTestEnv{
reaper: database.NewTestRetentionReaper(mainDB, mgr),
mainDB: mainDB,
mgr: mgr,
}
}
// createWebhook inserts a webhook row into the main database with the
// given retention policy and returns its ID.
func createWebhook(
t *testing.T,
db *gorm.DB,
retentionDays int,
) string {
t.Helper()
wh := &database.Webhook{
UserID: uuid.New().String(),
Name: "test-webhook",
RetentionDays: retentionDays,
}
require.NoError(
t,
db.Omit(clause.Associations).Create(wh).Error,
)
// The RetentionDays column carries a GORM default of 30, so a
// zero (or negative) value passed to Create is replaced by that
// default. Force the requested value explicitly so the
// retain-forever (<= 0) path can be exercised.
require.NoError(
t,
db.Model(wh).
Update("retention_days", retentionDays).Error,
)
return wh.ID
}
// eventChain is the set of row IDs seeded for a single event.
type eventChain struct {
eventID string
deliveryID string
resultID string
}
// seedEventChain creates an event with one delivery and one delivery
// result, all stamped with createdAt, and returns their IDs.
func seedEventChain(
t *testing.T,
db *gorm.DB,
webhookID string,
createdAt time.Time,
) eventChain {
t.Helper()
event := &database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: "POST",
Body: `{"seed": true}`,
ContentType: "application/json",
}
event.CreatedAt = createdAt
require.NoError(t, db.Create(event).Error)
delivery := &database.Delivery{
EventID: event.ID,
TargetID: uuid.New().String(),
Status: database.DeliveryStatusDelivered,
}
delivery.CreatedAt = createdAt
require.NoError(t, db.Create(delivery).Error)
result := &database.DeliveryResult{
DeliveryID: delivery.ID,
AttemptNum: 1,
Success: true,
StatusCode: 200,
Duration: 10,
}
result.CreatedAt = createdAt
require.NoError(t, db.Create(result).Error)
return eventChain{
eventID: event.ID,
deliveryID: delivery.ID,
resultID: result.ID,
}
}
// countByID returns how many rows of model match the given id,
// counting even hard-deletable rows via Unscoped.
func countByID(
t *testing.T,
db *gorm.DB,
model any,
id string,
) int64 {
t.Helper()
var n int64
require.NoError(
t,
db.Unscoped().Model(model).
Where("id = ?", id).Count(&n).Error,
)
return n
}
func assertChainGone(
t *testing.T,
db *gorm.DB,
chain eventChain,
) {
t.Helper()
assert.Zero(
t,
countByID(t, db, &database.Event{}, chain.eventID),
"expired event should be removed",
)
assert.Zero(
t,
countByID(t, db, &database.Delivery{}, chain.deliveryID),
"expired delivery should be removed",
)
assert.Zero(
t,
countByID(
t, db, &database.DeliveryResult{}, chain.resultID,
),
"expired delivery result should be removed",
)
}
func assertChainPresent(
t *testing.T,
db *gorm.DB,
chain eventChain,
) {
t.Helper()
assert.Equal(
t,
int64(1),
countByID(t, db, &database.Event{}, chain.eventID),
"recent event should be retained",
)
assert.Equal(
t,
int64(1),
countByID(t, db, &database.Delivery{}, chain.deliveryID),
"recent delivery should be retained",
)
assert.Equal(
t,
int64(1),
countByID(
t, db, &database.DeliveryResult{}, chain.resultID,
),
"recent delivery result should be retained",
)
}
func TestRetentionReaper_ReapsExpiredKeepsRecent(t *testing.T) {
t.Parallel()
env := setupRetentionTest(t)
const retentionDays = 30
webhookID := createWebhook(
t, env.mainDB.DB(), retentionDays,
)
db, err := env.mgr.GetDB(webhookID)
require.NoError(t, err)
now := time.Now()
old := seedEventChain(
t, db, webhookID,
now.Add(-40*24*time.Hour),
)
recent := seedEventChain(
t, db, webhookID,
now.Add(-1*24*time.Hour),
)
env.reaper.ExportSweep(context.Background())
assertChainGone(t, db, old)
assertChainPresent(t, db, recent)
}
func TestRetentionReaper_RetainsForeverWhenNonPositive(t *testing.T) {
t.Parallel()
env := setupRetentionTest(t)
// RetentionDays of zero means retain forever.
webhookID := createWebhook(t, env.mainDB.DB(), 0)
db, err := env.mgr.GetDB(webhookID)
require.NoError(t, err)
ancient := seedEventChain(
t, db, webhookID,
time.Now().Add(-365*24*time.Hour),
)
env.reaper.ExportSweep(context.Background())
assertChainPresent(t, db, ancient)
}

View File

@@ -0,0 +1,28 @@
package database
import (
"log/slog"
"os"
"gorm.io/gorm"
)
// NewTestDatabase creates a Database wrapper around a pre-opened *gorm.DB.
// Intended for use in tests that need a *database.Database without the
// full fx lifecycle. The caller is responsible for closing the underlying
// sql.DB connection.
func NewTestDatabase(db *gorm.DB) *Database {
return &Database{
db: db,
log: slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: slog.LevelDebug})),
}
}
// NewTestWebhookDBManager creates a WebhookDBManager backed by the given
// data directory. Intended for use in tests without the fx lifecycle.
func NewTestWebhookDBManager(dataDir string) *WebhookDBManager {
return &WebhookDBManager{
dataDir: dataDir,
log: slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: slog.LevelDebug})),
}
}

View File

@@ -0,0 +1,281 @@
package database
import (
"context"
"database/sql"
"errors"
"fmt"
"log/slog"
"os"
"path/filepath"
"sync"
"go.uber.org/fx"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/logger"
)
// WebhookDBManagerParams holds the fx dependencies for
// WebhookDBManager.
type WebhookDBManagerParams struct {
fx.In
Config *config.Config
Logger *logger.Logger
}
// errInvalidCachedDBType indicates a type assertion failure
// when retrieving a cached database connection.
var errInvalidCachedDBType = errors.New(
"invalid cached database type",
)
// WebhookDBManager manages per-webhook SQLite database files
// for event storage. Each webhook gets its own dedicated
// database containing Events, Deliveries, and DeliveryResults.
// Database connections are opened lazily and cached.
type WebhookDBManager struct {
dataDir string
dbs sync.Map // map[webhookID]*gorm.DB
log *slog.Logger
}
// NewWebhookDBManager creates a new WebhookDBManager and
// registers lifecycle hooks.
func NewWebhookDBManager(
lc fx.Lifecycle,
params WebhookDBManagerParams,
) (*WebhookDBManager, error) {
m := &WebhookDBManager{
dataDir: params.Config.DataDir,
log: params.Logger.Get(),
}
// Create data directory if it doesn't exist
err := os.MkdirAll(m.dataDir, dataDirPerm)
if err != nil {
return nil, fmt.Errorf(
"creating data directory %s: %w",
m.dataDir,
err,
)
}
lc.Append(fx.Hook{
OnStop: func(_ context.Context) error {
return m.CloseAll()
},
})
m.log.Info(
"webhook database manager initialized",
"data_dir", m.dataDir,
)
return m, nil
}
// GetDB returns the database connection for a webhook,
// creating the database file lazily if it doesn't exist.
func (m *WebhookDBManager) GetDB(
webhookID string,
) (*gorm.DB, error) {
// Fast path: already open
if val, ok := m.dbs.Load(webhookID); ok {
cachedDB, castOK := val.(*gorm.DB)
if !castOK {
return nil, fmt.Errorf(
"%w for webhook %s",
errInvalidCachedDBType,
webhookID,
)
}
return cachedDB, nil
}
// Slow path: open/create the database
db, err := m.openDB(webhookID)
if err != nil {
return nil, err
}
// Store it; if another goroutine beat us, close ours
actual, loaded := m.dbs.LoadOrStore(webhookID, db)
if loaded {
// Another goroutine created it first; close our duplicate
sqlDB, closeErr := db.DB()
if closeErr == nil {
_ = sqlDB.Close()
}
existingDB, castOK := actual.(*gorm.DB)
if !castOK {
return nil, fmt.Errorf(
"%w for webhook %s",
errInvalidCachedDBType,
webhookID,
)
}
return existingDB, nil
}
return db, nil
}
// CreateDB explicitly creates a new per-webhook database file
// and runs migrations.
func (m *WebhookDBManager) CreateDB(
webhookID string,
) error {
_, err := m.GetDB(webhookID)
return err
}
// DBExists checks if a per-webhook database file exists on
// disk.
func (m *WebhookDBManager) DBExists(
webhookID string,
) bool {
_, err := os.Stat(m.dbPath(webhookID))
return err == nil
}
// DeleteDB closes the connection and deletes the database file
// for a webhook. The file is permanently removed.
func (m *WebhookDBManager) DeleteDB(
webhookID string,
) error {
// Close and remove from cache
if val, ok := m.dbs.LoadAndDelete(webhookID); ok {
if gormDB, castOK := val.(*gorm.DB); castOK {
sqlDB, err := gormDB.DB()
if err == nil {
_ = sqlDB.Close()
}
}
}
// Delete the main DB file and WAL/SHM files
path := m.dbPath(webhookID)
for _, suffix := range []string{"", "-wal", "-shm"} {
err := os.Remove(path + suffix)
if err != nil && !os.IsNotExist(err) {
return fmt.Errorf(
"deleting webhook database file %s%s: %w",
path, suffix, err,
)
}
}
m.log.Info(
"deleted per-webhook database",
"webhook_id", webhookID,
)
return nil
}
// CloseAll closes all open per-webhook database connections.
// Called during application shutdown.
func (m *WebhookDBManager) CloseAll() error {
var lastErr error
m.dbs.Range(func(key, value any) bool {
if gormDB, castOK := value.(*gorm.DB); castOK {
sqlDB, err := gormDB.DB()
if err == nil {
closeErr := sqlDB.Close()
if closeErr != nil {
lastErr = closeErr
m.log.Error(
"failed to close webhook database",
"webhook_id", key,
"error", closeErr,
)
}
}
}
m.dbs.Delete(key)
return true
})
return lastErr
}
// DBPath returns the filesystem path for a webhook's database
// file.
func (m *WebhookDBManager) DBPath(
webhookID string,
) string {
return m.dbPath(webhookID)
}
func (m *WebhookDBManager) dbPath(
webhookID string,
) string {
return filepath.Join(
m.dataDir,
fmt.Sprintf("events-%s.db", webhookID),
)
}
// openDB opens (or creates) a per-webhook SQLite database and
// runs migrations.
func (m *WebhookDBManager) openDB(
webhookID string,
) (*gorm.DB, error) {
path := m.dbPath(webhookID)
dbURL := fmt.Sprintf(
"file:%s?cache=shared&mode=rwc",
path,
)
sqlDB, err := sql.Open("sqlite", dbURL)
if err != nil {
return nil, fmt.Errorf(
"opening webhook database %s: %w",
webhookID, err,
)
}
db, err := gorm.Open(sqlite.Dialector{
Conn: sqlDB,
}, &gorm.Config{})
if err != nil {
_ = sqlDB.Close()
return nil, fmt.Errorf(
"connecting to webhook database %s: %w",
webhookID, err,
)
}
// Run migrations for event-tier models only
err = db.AutoMigrate(
&Event{}, &Delivery{}, &DeliveryResult{},
)
if err != nil {
_ = sqlDB.Close()
return nil, fmt.Errorf(
"migrating webhook database %s: %w",
webhookID, err,
)
}
m.log.Info(
"opened per-webhook database",
"webhook_id", webhookID,
"path", path,
)
return db, nil
}

View File

@@ -0,0 +1,365 @@
package database_test
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger"
)
func setupTestWebhookDBManager(
t *testing.T,
) (*database.WebhookDBManager, *fxtest.Lifecycle) {
t.Helper()
lc := fxtest.NewLifecycle(t)
g := &globals.Globals{
Appname: "webhooker-test",
Version: "test",
}
l, err := logger.New(
lc,
logger.LoggerParams{Globals: g},
)
require.NoError(t, err)
dataDir := filepath.Join(t.TempDir(), "events")
cfg := &config.Config{
DataDir: dataDir,
}
mgr, err := database.NewWebhookDBManager(
lc,
database.WebhookDBManagerParams{
Config: cfg,
Logger: l,
},
)
require.NoError(t, err)
return mgr, lc
}
func TestWebhookDBManager_CreateAndGetDB(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
// DB should not exist yet
assert.False(t, mgr.DBExists(webhookID))
// Create the DB
err := mgr.CreateDB(webhookID)
require.NoError(t, err)
// DB file should now exist
assert.True(t, mgr.DBExists(webhookID))
// Get the DB again (should use cached connection)
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
require.NotNil(t, db)
// Verify we can write an event
event := &database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: "POST",
Headers: `{"Content-Type":["application/json"]}`,
Body: `{"test": true}`,
ContentType: "application/json",
}
require.NoError(t, db.Create(event).Error)
assert.NotEmpty(t, event.ID)
// Verify we can read it back
var readEvent database.Event
require.NoError(
t,
db.First(&readEvent, "id = ?", event.ID).Error,
)
assert.Equal(t, webhookID, readEvent.WebhookID)
assert.Equal(t, "POST", readEvent.Method)
assert.Equal(t, `{"test": true}`, readEvent.Body)
}
func TestWebhookDBManager_DeleteDB(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
// Create the DB and write some data
require.NoError(t, mgr.CreateDB(webhookID))
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
event := &database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: "POST",
Body: `{"test": true}`,
ContentType: "application/json",
}
require.NoError(t, db.Create(event).Error)
// Delete the DB
require.NoError(t, mgr.DeleteDB(webhookID))
// File should no longer exist
assert.False(t, mgr.DBExists(webhookID))
// Verify the file is actually gone from disk
dbPath := mgr.DBPath(webhookID)
_, err = os.Stat(dbPath)
assert.True(t, os.IsNotExist(err))
}
func TestWebhookDBManager_LazyCreation(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
// GetDB should lazily create the database
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
require.NotNil(t, db)
// File should now exist
assert.True(t, mgr.DBExists(webhookID))
}
func TestWebhookDBManager_DeliveryWorkflow(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
targetID := uuid.New().String()
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
event, delivery := seedDeliveryWorkflow(
t, db, webhookID, targetID,
)
verifyPendingDeliveries(t, db, event)
completeDelivery(t, db, delivery)
verifyNoPending(t, db)
}
func seedDeliveryWorkflow(
t *testing.T,
db *gorm.DB,
webhookID, targetID string,
) (*database.Event, *database.Delivery) {
t.Helper()
event := &database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: "POST",
Headers: `{"Content-Type":["application/json"]}`,
Body: `{"payload": "test"}`,
ContentType: "application/json",
}
require.NoError(t, db.Create(event).Error)
delivery := &database.Delivery{
EventID: event.ID,
TargetID: targetID,
Status: database.DeliveryStatusPending,
}
require.NoError(t, db.Create(delivery).Error)
return event, delivery
}
func verifyPendingDeliveries(
t *testing.T,
db *gorm.DB,
event *database.Event,
) {
t.Helper()
var pending []database.Delivery
require.NoError(
t,
db.Where(
"status = ?",
database.DeliveryStatusPending,
).Preload("Event").Find(&pending).Error,
)
require.Len(t, pending, 1)
assert.Equal(t, event.ID, pending[0].EventID)
assert.Equal(t, "POST", pending[0].Event.Method)
}
func completeDelivery(
t *testing.T,
db *gorm.DB,
delivery *database.Delivery,
) {
t.Helper()
result := &database.DeliveryResult{
DeliveryID: delivery.ID,
AttemptNum: 1,
Success: true,
StatusCode: 200,
Duration: 42,
}
require.NoError(t, db.Create(result).Error)
require.NoError(
t,
db.Model(delivery).Update(
"status",
database.DeliveryStatusDelivered,
).Error,
)
}
func verifyNoPending(
t *testing.T,
db *gorm.DB,
) {
t.Helper()
var stillPending []database.Delivery
require.NoError(
t,
db.Where(
"status = ?",
database.DeliveryStatusPending,
).Find(&stillPending).Error,
)
assert.Empty(t, stillPending)
}
func TestWebhookDBManager_MultipleWebhooks(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhook1 := uuid.New().String()
webhook2 := uuid.New().String()
// Create DBs for two webhooks
require.NoError(t, mgr.CreateDB(webhook1))
require.NoError(t, mgr.CreateDB(webhook2))
db1, err := mgr.GetDB(webhook1)
require.NoError(t, err)
db2, err := mgr.GetDB(webhook2)
require.NoError(t, err)
// Write events to each webhook's DB
event1 := &database.Event{
WebhookID: webhook1,
EntrypointID: uuid.New().String(),
Method: "POST",
Body: `{"webhook": 1}`,
ContentType: "application/json",
}
event2 := &database.Event{
WebhookID: webhook2,
EntrypointID: uuid.New().String(),
Method: "PUT",
Body: `{"webhook": 2}`,
ContentType: "application/json",
}
require.NoError(t, db1.Create(event1).Error)
require.NoError(t, db2.Create(event2).Error)
// Verify isolation: each DB only has its own events
var count1 int64
db1.Model(&database.Event{}).Count(&count1)
assert.Equal(t, int64(1), count1)
var count2 int64
db2.Model(&database.Event{}).Count(&count2)
assert.Equal(t, int64(1), count2)
// Delete webhook1's DB, webhook2 should be unaffected
require.NoError(t, mgr.DeleteDB(webhook1))
assert.False(t, mgr.DBExists(webhook1))
assert.True(t, mgr.DBExists(webhook2))
// webhook2's data should still be accessible
var events []database.Event
require.NoError(t, db2.Find(&events).Error)
assert.Len(t, events, 1)
assert.Equal(t, "PUT", events[0].Method)
}
func TestWebhookDBManager_CloseAll(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
// Create a few DBs
for range 3 {
require.NoError(
t,
mgr.CreateDB(uuid.New().String()),
)
}
// CloseAll should close all connections without error
require.NoError(t, mgr.CloseAll())
// Stop lifecycle (CloseAll already called)
require.NoError(t, lc.Stop(ctx))
}

View File

@@ -0,0 +1,152 @@
package delivery
import (
"sync"
"time"
)
// CircuitState represents the current state of a circuit
// breaker.
type CircuitState int
const (
// CircuitClosed is the normal operating state.
CircuitClosed CircuitState = iota
// CircuitOpen means the circuit has tripped.
CircuitOpen
// CircuitHalfOpen allows a single probe delivery to
// test whether the target has recovered.
CircuitHalfOpen
)
const (
// defaultFailureThreshold is the number of consecutive
// failures before a circuit breaker trips open.
defaultFailureThreshold = 5
// defaultCooldown is how long a circuit stays open
// before transitioning to half-open.
defaultCooldown = 30 * time.Second
)
// CircuitBreaker implements the circuit breaker pattern
// for a single delivery target.
type CircuitBreaker struct {
mu sync.Mutex
state CircuitState
failures int
threshold int
cooldown time.Duration
lastFailure time.Time
}
// NewCircuitBreaker creates a circuit breaker with default
// settings.
func NewCircuitBreaker() *CircuitBreaker {
return &CircuitBreaker{
state: CircuitClosed,
threshold: defaultFailureThreshold,
cooldown: defaultCooldown,
}
}
// Allow checks whether a delivery attempt should proceed.
func (cb *CircuitBreaker) Allow() bool {
cb.mu.Lock()
defer cb.mu.Unlock()
switch cb.state {
case CircuitClosed:
return true
case CircuitOpen:
if time.Since(cb.lastFailure) >= cb.cooldown {
cb.state = CircuitHalfOpen
return true
}
return false
case CircuitHalfOpen:
return false
default:
return true
}
}
// CooldownRemaining returns how much time is left before
// an open circuit transitions to half-open.
func (cb *CircuitBreaker) CooldownRemaining() time.Duration {
cb.mu.Lock()
defer cb.mu.Unlock()
if cb.state != CircuitOpen {
return 0
}
remaining := cb.cooldown - time.Since(cb.lastFailure)
if remaining < 0 {
return 0
}
return remaining
}
// RecordSuccess records a successful delivery and resets
// the circuit breaker to closed state.
func (cb *CircuitBreaker) RecordSuccess() {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.failures = 0
cb.state = CircuitClosed
}
// RecordFailure records a failed delivery. If the failure
// count reaches the threshold, the circuit trips open.
func (cb *CircuitBreaker) RecordFailure() {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.failures++
cb.lastFailure = time.Now()
switch cb.state {
case CircuitClosed:
if cb.failures >= cb.threshold {
cb.state = CircuitOpen
}
case CircuitOpen:
// Already open; no state change needed.
case CircuitHalfOpen:
// Probe failed -- reopen immediately.
cb.state = CircuitOpen
}
}
// State returns the current circuit state.
func (cb *CircuitBreaker) State() CircuitState {
cb.mu.Lock()
defer cb.mu.Unlock()
return cb.state
}
// String returns the human-readable name of a circuit
// state.
func (s CircuitState) String() string {
switch s {
case CircuitClosed:
return "closed"
case CircuitOpen:
return "open"
case CircuitHalfOpen:
return "half-open"
default:
return "unknown"
}
}

View File

@@ -0,0 +1,310 @@
package delivery_test
import (
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/delivery"
)
func TestCircuitBreaker_ClosedState_AllowsDeliveries(
t *testing.T,
) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
assert.Equal(t, delivery.CircuitClosed, cb.State())
assert.True(t, cb.Allow(),
"closed circuit should allow deliveries",
)
for range 10 {
assert.True(t, cb.Allow())
}
}
func TestCircuitBreaker_FailureCounting(t *testing.T) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
for i := range delivery.ExportDefaultFailureThreshold - 1 {
cb.RecordFailure()
assert.Equal(t,
delivery.CircuitClosed, cb.State(),
"circuit should remain closed after %d failures",
i+1,
)
assert.True(t, cb.Allow(),
"should still allow after %d failures",
i+1,
)
}
}
func TestCircuitBreaker_OpenTransition(t *testing.T) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
assert.Equal(t, delivery.CircuitOpen, cb.State(),
"circuit should be open after threshold failures",
)
assert.False(t, cb.Allow(),
"open circuit should reject deliveries",
)
}
func TestCircuitBreaker_Cooldown_StaysOpen(t *testing.T) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
require.Equal(t, delivery.CircuitOpen, cb.State())
assert.False(t, cb.Allow(),
"should be blocked during cooldown",
)
remaining := cb.CooldownRemaining()
assert.Greater(t, remaining, time.Duration(0),
"cooldown should have remaining time",
)
}
func TestCircuitBreaker_HalfOpen_AfterCooldown(
t *testing.T,
) {
t.Parallel()
cb := newShortCooldownCB(t)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
require.Equal(t, delivery.CircuitOpen, cb.State())
time.Sleep(60 * time.Millisecond)
assert.Equal(t, time.Duration(0),
cb.CooldownRemaining(),
)
assert.True(t, cb.Allow(),
"should allow one probe after cooldown",
)
assert.Equal(t,
delivery.CircuitHalfOpen, cb.State(),
"should be half-open after probe allowed",
)
assert.False(t, cb.Allow(),
"should reject additional probes while half-open",
)
}
func TestCircuitBreaker_ProbeSuccess_ClosesCircuit(
t *testing.T,
) {
t.Parallel()
cb := newShortCooldownCB(t)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
time.Sleep(60 * time.Millisecond)
require.True(t, cb.Allow())
cb.RecordSuccess()
assert.Equal(t, delivery.CircuitClosed, cb.State(),
"successful probe should close circuit",
)
assert.True(t, cb.Allow(),
"closed circuit should allow deliveries",
)
}
func TestCircuitBreaker_ProbeFailure_ReopensCircuit(
t *testing.T,
) {
t.Parallel()
cb := newShortCooldownCB(t)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
time.Sleep(60 * time.Millisecond)
require.True(t, cb.Allow())
cb.RecordFailure()
assert.Equal(t, delivery.CircuitOpen, cb.State(),
"failed probe should reopen circuit",
)
assert.False(t, cb.Allow(),
"reopened circuit should reject deliveries",
)
}
func TestCircuitBreaker_SuccessResetsFailures(
t *testing.T,
) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
for range delivery.ExportDefaultFailureThreshold - 1 {
cb.RecordFailure()
}
require.Equal(t, delivery.CircuitClosed, cb.State())
cb.RecordSuccess()
assert.Equal(t, delivery.CircuitClosed, cb.State())
for range delivery.ExportDefaultFailureThreshold - 1 {
cb.RecordFailure()
}
assert.Equal(t, delivery.CircuitClosed, cb.State(),
"circuit should still be closed -- "+
"success reset the counter",
)
cb.RecordFailure()
assert.Equal(t, delivery.CircuitOpen, cb.State())
}
func TestCircuitBreaker_ConcurrentAccess(t *testing.T) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
const goroutines = 100
var wg sync.WaitGroup
wg.Add(goroutines * 3)
for range goroutines {
go func() {
defer wg.Done()
cb.Allow()
}()
}
for range goroutines {
go func() {
defer wg.Done()
cb.RecordFailure()
}()
}
for range goroutines {
go func() {
defer wg.Done()
cb.RecordSuccess()
}()
}
wg.Wait()
state := cb.State()
assert.Contains(t,
[]delivery.CircuitState{
delivery.CircuitClosed,
delivery.CircuitOpen,
delivery.CircuitHalfOpen,
},
state,
"state should be valid after concurrent access",
)
}
func TestCircuitBreaker_CooldownRemaining_ClosedReturnsZero(
t *testing.T,
) {
t.Parallel()
cb := delivery.NewCircuitBreaker()
assert.Equal(t, time.Duration(0),
cb.CooldownRemaining(),
"closed circuit should have zero cooldown remaining",
)
}
func TestCircuitBreaker_CooldownRemaining_HalfOpenReturnsZero(
t *testing.T,
) {
t.Parallel()
cb := newShortCooldownCB(t)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
time.Sleep(60 * time.Millisecond)
require.True(t, cb.Allow())
assert.Equal(t, time.Duration(0),
cb.CooldownRemaining(),
"half-open circuit should have zero cooldown remaining",
)
}
func TestCircuitState_String(t *testing.T) {
t.Parallel()
assert.Equal(t, "closed", delivery.CircuitClosed.String())
assert.Equal(t, "open", delivery.CircuitOpen.String())
assert.Equal(t, "half-open", delivery.CircuitHalfOpen.String())
assert.Equal(t, "unknown", delivery.CircuitState(99).String())
}
// newShortCooldownCB creates a CircuitBreaker with a short
// cooldown for testing. We use NewCircuitBreaker and
// manipulate through the public API.
func newShortCooldownCB(t *testing.T) *delivery.CircuitBreaker {
t.Helper()
return delivery.NewTestCircuitBreaker(
delivery.ExportDefaultFailureThreshold,
50*time.Millisecond,
)
}

View File

@@ -0,0 +1,112 @@
package delivery_test
import (
"context"
"log/slog"
"net/http"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/delivery"
)
// newSSRFTestEngine builds an Engine whose shared client
// carries the SSRF-safe transport, mirroring production.
func newSSRFTestEngine() *delivery.Engine {
log := slog.New(slog.DiscardHandler)
client := &http.Client{
Timeout: 30 * time.Second,
Transport: delivery.NewSSRFSafeTransport(),
}
return delivery.NewTestEngine(log, client, 1)
}
// TestClientForConfig_TimeoutKeepsSSRFGuard asserts that a
// client returned by clientForConfig for a config with a
// per-target timeout still refuses connections to
// private/reserved addresses (the timeout must not drop the
// SSRF-safe transport).
func TestClientForConfig_TimeoutKeepsSSRFGuard(t *testing.T) {
t.Parallel()
engine := newSSRFTestEngine()
blocked := []string{
"http://127.0.0.1/hook",
"http://169.254.169.254/latest/meta-data/",
"http://[fe80::1]/hook",
}
for _, target := range blocked {
t.Run(target, func(t *testing.T) {
t.Parallel()
cfg := &delivery.HTTPTargetConfig{
URL: target,
Timeout: 5,
}
client := engine.ExportClientForConfig(cfg)
require.NotSame(t, engine.ExportClient(), client,
"a per-target timeout must yield a "+
"distinct client",
)
assert.Equal(t,
5*time.Second, client.Timeout,
"the per-target timeout must be applied",
)
assert.Same(t,
engine.ExportClient().Transport,
client.Transport,
"the SSRF-safe transport must be reused, "+
"not dropped",
)
req, err := http.NewRequestWithContext(
context.Background(),
http.MethodPost, target, nil,
)
require.NoError(t, err)
resp, doErr := client.Do(req)
if resp != nil {
_ = resp.Body.Close()
}
require.Error(t, doErr,
"request to %s must be blocked", target,
)
assert.Contains(t, doErr.Error(), "blocked",
"error must come from the SSRF guard",
)
})
}
}
// TestClientForConfig_NoTimeoutUnchanged asserts that with
// no per-target timeout the shared SSRF-safe client is
// returned unchanged.
func TestClientForConfig_NoTimeoutUnchanged(t *testing.T) {
t.Parallel()
engine := newSSRFTestEngine()
cfg := &delivery.HTTPTargetConfig{
URL: "https://example.com/hook",
}
client := engine.ExportClientForConfig(cfg)
assert.Same(t, engine.ExportClient(), client,
"without a per-target timeout the shared client "+
"must be returned unchanged",
)
}

1026
internal/delivery/engine.go Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,275 @@
package delivery
import (
"context"
"log/slog"
"net"
"net/http"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// Exported constants for test access.
const (
ExportDeliveryChannelSize = deliveryChannelSize
ExportRetryChannelSize = retryChannelSize
ExportDefaultFailureThreshold = defaultFailureThreshold
ExportDefaultCooldown = defaultCooldown
)
// ExportIsBlockedIP exposes isBlockedIP for testing.
func ExportIsBlockedIP(ip net.IP) bool {
return isBlockedIP(ip)
}
// ExportBlockedNetworks exposes blockedNetworks.
func ExportBlockedNetworks() []*net.IPNet {
return blockedNetworks
}
// ExportIsForwardableHeader exposes isForwardableHeader.
func ExportIsForwardableHeader(name string) bool {
return isForwardableHeader(name)
}
// ExportTruncate exposes truncate for testing.
func ExportTruncate(s string, maxLen int) string {
return truncate(s, maxLen)
}
// ExportDeliverHTTP delivers via the http target for testing.
func (e *Engine) ExportDeliverHTTP(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
) {
e.httpTarget.Deliver(ctx, webhookDB, d, task, e)
}
// ExportDeliverDatabase delivers via the database target.
func (e *Engine) ExportDeliverDatabase(
webhookDB *gorm.DB, d *database.Delivery,
) {
e.targets[database.TargetTypeDatabase].Deliver(
context.Background(), webhookDB, d, &Task{}, e,
)
}
// ExportDeliverLog delivers via the log target for testing.
func (e *Engine) ExportDeliverLog(
webhookDB *gorm.DB, d *database.Delivery,
) {
e.targets[database.TargetTypeLog].Deliver(
context.Background(), webhookDB, d, &Task{}, e,
)
}
// ExportDeliverSlack delivers via the slack target for
// testing.
func (e *Engine) ExportDeliverSlack(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
) {
task := &Task{
DeliveryID: d.ID,
TargetID: d.TargetID,
AttemptNum: 1,
}
e.targets[database.TargetTypeSlack].Deliver(
ctx, webhookDB, d, task, e,
)
}
// ExportProcessNewTask exposes processNewTask.
func (e *Engine) ExportProcessNewTask(
ctx context.Context, task *Task,
) {
e.processNewTask(ctx, task)
}
// ExportProcessRetryTask exposes processRetryTask.
func (e *Engine) ExportProcessRetryTask(
ctx context.Context, task *Task,
) {
e.processRetryTask(ctx, task)
}
// ExportProcessDelivery exposes processDelivery.
func (e *Engine) ExportProcessDelivery(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
) {
e.processDelivery(ctx, webhookDB, d, task)
}
// ExportGetCircuitBreaker exposes the http target's
// getCircuitBreaker.
func (e *Engine) ExportGetCircuitBreaker(
targetID string,
) *CircuitBreaker {
return e.httpTarget.getCircuitBreaker(targetID)
}
// ExportParseHTTPConfig exposes parseHTTPConfig.
func (e *Engine) ExportParseHTTPConfig(
configJSON string,
) (*HTTPTargetConfig, error) {
return parseHTTPConfig(configJSON)
}
// ExportParseSlackConfig exposes parseSlackConfig.
func (e *Engine) ExportParseSlackConfig(
configJSON string,
) (*SlackTargetConfig, error) {
return parseSlackConfig(configJSON)
}
// ExportDoHTTPRequest exposes the http target's
// doHTTPRequest.
func (e *Engine) ExportDoHTTPRequest(
ctx context.Context,
cfg *HTTPTargetConfig,
event *database.Event,
) (int, string, int64, error) {
return e.httpTarget.doHTTPRequest(ctx, cfg, event)
}
// ExportClientForConfig exposes the http target's
// clientForConfig.
func (e *Engine) ExportClientForConfig(
cfg *HTTPTargetConfig,
) *http.Client {
return e.httpTarget.clientForConfig(cfg)
}
// ExportClient returns the http target's shared HTTP client.
func (e *Engine) ExportClient() *http.Client {
return e.httpTarget.client
}
// ExportScheduleRetry exposes ScheduleRetry.
func (e *Engine) ExportScheduleRetry(
task Task, delay time.Duration,
) {
e.ScheduleRetry(task, delay)
}
// ExportRecoverPendingDeliveries exposes
// recoverPendingDeliveries.
func (e *Engine) ExportRecoverPendingDeliveries(
ctx context.Context,
webhookDB *gorm.DB,
webhookID string,
) {
e.recoverPendingDeliveries(
ctx, webhookDB, webhookID,
)
}
// ExportRecoverWebhookDeliveries exposes
// recoverWebhookDeliveries.
func (e *Engine) ExportRecoverWebhookDeliveries(
ctx context.Context, webhookID string,
) {
e.recoverWebhookDeliveries(ctx, webhookID)
}
// ExportRecoverInFlight exposes recoverInFlight.
func (e *Engine) ExportRecoverInFlight(
ctx context.Context,
) {
e.recoverInFlight(ctx)
}
// ExportStart exposes start for testing.
func (e *Engine) ExportStart(ctx context.Context) {
e.start(ctx)
}
// ExportStop exposes stop for testing.
func (e *Engine) ExportStop() {
e.stop()
}
// ExportDeliveryCh returns the delivery channel.
func (e *Engine) ExportDeliveryCh() chan Task {
return e.deliveryCh
}
// ExportRetryCh returns the retry channel.
func (e *Engine) ExportRetryCh() chan Task {
return e.retryCh
}
// NewTestEngine creates an Engine for unit tests without
// database dependencies.
func NewTestEngine(
log *slog.Logger,
client *http.Client,
workers int,
) *Engine {
e := &Engine{
log: log,
deliveryCh: make(chan Task, deliveryChannelSize),
retryCh: make(chan Task, retryChannelSize),
workers: workers,
}
e.initTargets(client)
return e
}
// NewTestEngineSmallRetry creates an Engine with a tiny
// retry channel buffer for overflow testing.
func NewTestEngineSmallRetry(
log *slog.Logger,
) *Engine {
e := &Engine{
log: log,
retryCh: make(chan Task, 1),
}
e.initTargets(nil)
return e
}
// NewTestEngineWithDB creates an Engine with a real
// database and dbManager for integration tests.
func NewTestEngineWithDB(
db *database.Database,
dbMgr *database.WebhookDBManager,
log *slog.Logger,
client *http.Client,
workers int,
) *Engine {
e := &Engine{
database: db,
dbManager: dbMgr,
log: log,
deliveryCh: make(chan Task, deliveryChannelSize),
retryCh: make(chan Task, retryChannelSize),
workers: workers,
}
e.initTargets(client)
return e
}
// NewTestCircuitBreaker creates a CircuitBreaker with
// custom settings for testing.
func NewTestCircuitBreaker(
threshold int, cooldown time.Duration,
) *CircuitBreaker {
return &CircuitBreaker{
state: CircuitClosed,
threshold: threshold,
cooldown: cooldown,
}
}

222
internal/delivery/ssrf.go Normal file
View File

@@ -0,0 +1,222 @@
package delivery
import (
"context"
"errors"
"fmt"
"net"
"net/http"
"net/url"
"time"
)
const (
// dnsResolutionTimeout is the maximum time to wait for
// DNS resolution during SSRF validation.
dnsResolutionTimeout = 5 * time.Second
)
// Sentinel errors for SSRF validation.
var (
errNoHostname = errors.New("URL has no hostname")
errNoIPs = errors.New(
"hostname resolved to no IP addresses",
)
errBlockedIP = errors.New(
"blocked private/reserved IP range",
)
errInvalidScheme = errors.New(
"only http and https are allowed",
)
)
// blockedNetworks contains all private/reserved IP ranges
// that should be blocked to prevent SSRF attacks.
//
//nolint:gochecknoglobals // package-level network list is appropriate here
var blockedNetworks []*net.IPNet
//nolint:gochecknoinits // init is the idiomatic way to parse CIDRs once at startup
func init() {
cidrs := []string{
"127.0.0.0/8",
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16",
"169.254.0.0/16",
"0.0.0.0/8",
"100.64.0.0/10",
"192.0.0.0/24",
"192.0.2.0/24",
"198.18.0.0/15",
"198.51.100.0/24",
"203.0.113.0/24",
"224.0.0.0/4",
"240.0.0.0/4",
"::1/128",
"fc00::/7",
"fe80::/10",
}
for _, cidr := range cidrs {
_, network, err := net.ParseCIDR(cidr)
if err != nil {
panic(fmt.Sprintf(
"ssrf: failed to parse CIDR %q: %v",
cidr, err,
))
}
blockedNetworks = append(
blockedNetworks, network,
)
}
}
// isBlockedIP checks whether an IP address falls within
// any blocked private/reserved network range.
func isBlockedIP(ip net.IP) bool {
for _, network := range blockedNetworks {
if network.Contains(ip) {
return true
}
}
return false
}
// ValidateTargetURL checks that an HTTP delivery target
// URL is safe from SSRF attacks.
func ValidateTargetURL(
ctx context.Context, targetURL string,
) error {
parsed, err := url.Parse(targetURL)
if err != nil {
return fmt.Errorf("invalid URL: %w", err)
}
err = validateScheme(parsed.Scheme)
if err != nil {
return err
}
host := parsed.Hostname()
if host == "" {
return errNoHostname
}
if ip := net.ParseIP(host); ip != nil {
return checkBlockedIP(ip)
}
return validateHostname(ctx, host)
}
func validateScheme(scheme string) error {
if scheme != "http" && scheme != "https" {
return fmt.Errorf(
"unsupported URL scheme %q: %w",
scheme, errInvalidScheme,
)
}
return nil
}
func checkBlockedIP(ip net.IP) error {
if isBlockedIP(ip) {
return fmt.Errorf(
"target IP %s is in a blocked "+
"private/reserved range: %w",
ip, errBlockedIP,
)
}
return nil
}
func validateHostname(
ctx context.Context, host string,
) error {
dnsCtx, cancel := context.WithTimeout(
ctx, dnsResolutionTimeout,
)
defer cancel()
ips, err := net.DefaultResolver.LookupIPAddr(
dnsCtx, host,
)
if err != nil {
return fmt.Errorf(
"failed to resolve hostname %q: %w",
host, err,
)
}
if len(ips) == 0 {
return fmt.Errorf(
"hostname %q: %w", host, errNoIPs,
)
}
for _, ipAddr := range ips {
if isBlockedIP(ipAddr.IP) {
return fmt.Errorf(
"hostname %q resolves to blocked "+
"IP %s: %w",
host, ipAddr.IP, errBlockedIP,
)
}
}
return nil
}
// NewSSRFSafeTransport creates an http.Transport with a
// custom DialContext that blocks connections to
// private/reserved IP addresses.
func NewSSRFSafeTransport() *http.Transport {
return &http.Transport{
DialContext: ssrfDialContext,
}
}
func ssrfDialContext(
ctx context.Context,
network, addr string,
) (net.Conn, error) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
return nil, fmt.Errorf(
"ssrf: invalid address %q: %w",
addr, err,
)
}
ips, err := net.DefaultResolver.LookupIPAddr(
ctx, host,
)
if err != nil {
return nil, fmt.Errorf(
"ssrf: DNS resolution failed for %q: %w",
host, err,
)
}
for _, ipAddr := range ips {
if isBlockedIP(ipAddr.IP) {
return nil, fmt.Errorf(
"ssrf: connection to %s (%s) "+
"blocked: %w",
host, ipAddr.IP, errBlockedIP,
)
}
}
var dialer net.Dialer
return dialer.DialContext(
ctx, network,
net.JoinHostPort(ips[0].IP.String(), port),
)
}

View File

@@ -0,0 +1,172 @@
package delivery_test
import (
"context"
"net"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/delivery"
)
func TestIsBlockedIP_PrivateRanges(t *testing.T) {
t.Parallel()
tests := []struct {
name string
ip string
blocked bool
}{
{"loopback 127.0.0.1", "127.0.0.1", true},
{"loopback 127.0.0.2", "127.0.0.2", true},
{"loopback 127.255.255.255", "127.255.255.255", true},
{"10.0.0.0", "10.0.0.0", true},
{"10.0.0.1", "10.0.0.1", true},
{"10.255.255.255", "10.255.255.255", true},
{"172.16.0.1", "172.16.0.1", true},
{"172.31.255.255", "172.31.255.255", true},
{"172.15.255.255", "172.15.255.255", false},
{"172.32.0.0", "172.32.0.0", false},
{"192.168.0.1", "192.168.0.1", true},
{"192.168.255.255", "192.168.255.255", true},
{"169.254.0.1", "169.254.0.1", true},
{"169.254.169.254", "169.254.169.254", true},
{"8.8.8.8", "8.8.8.8", false},
{"1.1.1.1", "1.1.1.1", false},
{"93.184.216.34", "93.184.216.34", false},
{"::1", "::1", true},
{"fd00::1", "fd00::1", true},
{"fc00::1", "fc00::1", true},
{"fe80::1", "fe80::1", true},
{
"2607:f8b0:4004:800::200e",
"2607:f8b0:4004:800::200e",
false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
ip := net.ParseIP(tt.ip)
require.NotNil(t, ip,
"failed to parse IP %s", tt.ip,
)
assert.Equal(t,
tt.blocked,
delivery.ExportIsBlockedIP(ip),
"isBlockedIP(%s) = %v, want %v",
tt.ip,
delivery.ExportIsBlockedIP(ip),
tt.blocked,
)
})
}
}
func TestValidateTargetURL_Blocked(t *testing.T) {
t.Parallel()
blockedURLs := []string{
"http://127.0.0.1/hook",
"http://127.0.0.1:8080/hook",
"https://10.0.0.1/hook",
"http://192.168.1.1/webhook",
"http://172.16.0.1/api",
"http://169.254.169.254/latest/meta-data/",
"http://[::1]/hook",
"http://[fc00::1]/hook",
"http://[fe80::1]/hook",
"http://0.0.0.0/hook",
}
for _, u := range blockedURLs {
t.Run(u, func(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), u,
)
assert.Error(t, err,
"URL %s should be blocked", u,
)
})
}
}
func TestValidateTargetURL_Allowed(t *testing.T) {
t.Parallel()
allowedURLs := []string{
"https://example.com/hook",
"http://93.184.216.34/webhook",
"https://hooks.slack.com/services/T00/B00/xxx",
}
for _, u := range allowedURLs {
t.Run(u, func(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), u,
)
assert.NoError(t, err,
"URL %s should be allowed", u,
)
})
}
}
func TestValidateTargetURL_InvalidScheme(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), "ftp://example.com/hook",
)
require.Error(t, err)
assert.Contains(t, err.Error(),
"unsupported URL scheme",
)
}
func TestValidateTargetURL_EmptyHost(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), "http:///path",
)
assert.Error(t, err)
}
func TestValidateTargetURL_InvalidURL(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
context.Background(), "://invalid",
)
assert.Error(t, err)
}
func TestBlockedNetworks_Initialized(t *testing.T) {
t.Parallel()
nets := delivery.ExportBlockedNetworks()
assert.NotEmpty(t, nets,
"blockedNetworks should be initialized",
)
assert.GreaterOrEqual(t, len(nets), 8,
"should have at least 8 blocked network ranges",
)
}

101
internal/delivery/target.go Normal file
View File

@@ -0,0 +1,101 @@
package delivery
import (
"context"
"net/http"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// Scheduler re-enqueues a task for a future delivery attempt.
// The engine provides one to each target so a target can own
// its retries durably: it records the attempt, marks the
// delivery retrying, and asks the Scheduler to deliver the
// next attempt after delay — exactly what the engine does for
// its own restart recovery.
type Scheduler interface {
ScheduleRetry(task Task, delay time.Duration)
}
// Target delivers an event to one target type. Each type is
// an implementation. A Target owns its whole delivery: it
// makes the attempt, records the DeliveryResult and updates
// the DeliveryStatus, and — for targets that retry — decides
// whether to retry, computes its own backoff, gates with its
// own circuit breaker, and reschedules via the injected
// Scheduler. Fire-and-forget targets simply record a single
// attempt.
type Target interface {
Deliver(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
)
}
// rescheduler is implemented by targets that own durable
// retries. The engine's restart recovery and periodic sweep
// use it to let the target recompute the schedule for an
// orphaned retrying delivery, keeping the retry schedule
// target-owned. Fire-and-forget targets do not implement it
// and their (never-occurring) retrying deliveries are
// skipped.
type rescheduler interface {
// remainingBackoff returns how long to wait before the
// next attempt of a recovered retrying delivery.
remainingBackoff(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) time.Duration
// backoffElapsed reports whether the backoff window for
// the last attempt has already passed, so the periodic
// sweep can re-enqueue the delivery now.
backoffElapsed(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) bool
}
// attemptResult is the outcome of a single delivery attempt,
// as reported by a target's per-attempt function to the
// shared retry core.
type attemptResult struct {
statusCode int
respBody string
duration int64
success bool
errMsg string
}
// initTargets builds the target registry, wiring each target
// to the engine's persistence helpers and giving the HTTP and
// Slack targets the shared SSRF-safe client. It is called by
// both New and the test constructors so the registry is
// always populated.
func (e *Engine) initTargets(client *http.Client) {
httpT := &httpTarget{
httpCore: &httpCore{eng: e},
client: client,
}
slackT := &slackTarget{
httpCore: &httpCore{eng: e},
client: client,
}
e.httpTarget = httpT
e.targets = map[database.TargetType]Target{
database.TargetTypeHTTP: httpT,
database.TargetTypeSlack: slackT,
database.TargetTypeDatabase: &databaseTarget{eng: e},
database.TargetTypeLog: &logTarget{eng: e},
}
}

View File

@@ -0,0 +1,34 @@
package delivery
import (
"context"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// databaseTarget is a fire-and-forget target: the event is
// already persisted in the per-webhook database by the time
// delivery runs, so the target records a single successful
// attempt. (Durable archiving to a separate store is tracked
// as its own work.)
type databaseTarget struct {
eng *Engine
}
// Deliver implements Target.
func (t *databaseTarget) Deliver(
_ context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
_ *Task,
_ Scheduler,
) {
t.eng.recordResult(
webhookDB, d, 1, true, 0, "", "", 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusDelivered,
)
}

View File

@@ -0,0 +1,499 @@
package delivery
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"sync"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// Sentinel errors returned by the config parsers.
var (
errEmptyTargetConfig = errors.New(
"empty target config",
)
errMissingTargetURL = errors.New(
"target URL is required",
)
)
// HTTPTargetConfig holds configuration for http target
// types.
type HTTPTargetConfig struct {
URL string `json:"url"`
Headers map[string]string `json:"headers,omitempty"`
Timeout int `json:"timeout,omitempty"`
}
// httpCore holds the retry, backoff, and circuit-breaker
// machinery shared by the HTTP and Slack targets. Each of
// those targets owns its own httpCore instance (and thus its
// own circuit breakers); the per-attempt request differs
// between them and is supplied as a closure.
type httpCore struct {
eng *Engine
// circuitBreakers stores a *CircuitBreaker per target ID.
circuitBreakers sync.Map
}
// deliver runs one delivery attempt through the retry core.
// A maxRetries of 0 is fire-and-forget: a single attempt is
// recorded and no circuit breaker is consulted. A positive
// maxRetries gates the attempt on the circuit breaker and
// schedules a backed-off retry on failure.
func (c *httpCore) deliver(
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
maxRetries int,
attempt func() attemptResult,
) {
if maxRetries == 0 {
c.fireAndForget(webhookDB, d, attempt())
return
}
c.withRetry(
webhookDB, d, task, sched, maxRetries, attempt,
)
}
func (c *httpCore) fireAndForget(
webhookDB *gorm.DB,
d *database.Delivery,
res attemptResult,
) {
c.eng.recordResult(
webhookDB, d, 1, res.success,
res.statusCode, res.respBody, res.errMsg,
res.duration,
)
if res.success {
c.eng.updateDeliveryStatus(
webhookDB, d,
database.DeliveryStatusDelivered,
)
return
}
c.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusFailed,
)
}
func (c *httpCore) withRetry(
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
maxRetries int,
attempt func() attemptResult,
) {
cb := c.getCircuitBreaker(task.TargetID)
if c.circuitBreakerBlock(webhookDB, d, task, sched, cb) {
return
}
attemptNum := task.AttemptNum
res := attempt()
c.eng.recordResult(
webhookDB, d, attemptNum, res.success,
res.statusCode, res.respBody, res.errMsg,
res.duration,
)
if res.success {
cb.RecordSuccess()
c.eng.updateDeliveryStatus(
webhookDB, d,
database.DeliveryStatusDelivered,
)
return
}
cb.RecordFailure()
c.handleRetry(
webhookDB, d, task, sched, maxRetries, attemptNum,
)
}
func (c *httpCore) circuitBreakerBlock(
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
cb *CircuitBreaker,
) bool {
if cb.Allow() {
return false
}
remaining := cb.CooldownRemaining()
c.eng.log.Info(
"circuit breaker open, skipping delivery",
"target_id", task.TargetID,
"target_name", task.TargetName,
"delivery_id", d.ID,
"cooldown_remaining", remaining,
)
c.eng.updateDeliveryStatus(
webhookDB, d,
database.DeliveryStatusRetrying,
)
retryTask := *task
sched.ScheduleRetry(retryTask, remaining)
return true
}
func (c *httpCore) handleRetry(
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
maxRetries int,
attemptNum int,
) {
if attemptNum >= maxRetries {
c.eng.updateDeliveryStatus(
webhookDB, d,
database.DeliveryStatusFailed,
)
return
}
c.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusRetrying,
)
backoff := calcBackoff(attemptNum)
retryTask := *task
retryTask.AttemptNum = attemptNum + 1
sched.ScheduleRetry(retryTask, backoff)
}
func (c *httpCore) getCircuitBreaker(
targetID string,
) *CircuitBreaker {
if val, ok := c.circuitBreakers.Load(targetID); ok {
cb, _ := val.(*CircuitBreaker)
return cb
}
fresh := NewCircuitBreaker()
actual, _ := c.circuitBreakers.LoadOrStore(
targetID, fresh,
)
cb, _ := actual.(*CircuitBreaker)
return cb
}
// remainingBackoff returns how long remains of the backoff
// window for the last attempt of a recovered retrying
// delivery. It implements rescheduler.
func (c *httpCore) remainingBackoff(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) time.Duration {
var lastResult database.DeliveryResult
err := webhookDB.
Where("delivery_id = ?", deliveryID).
Order("created_at DESC").
First(&lastResult).Error
if err != nil {
return 0
}
backoff := calcBackoff(attemptNum)
elapsed := time.Since(lastResult.CreatedAt)
remaining := backoff - elapsed
return max(remaining, 0)
}
// backoffElapsed reports whether the backoff window for the
// last attempt of a retrying delivery has passed. It
// implements rescheduler.
func (c *httpCore) backoffElapsed(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) bool {
var lastResult database.DeliveryResult
err := webhookDB.
Where("delivery_id = ?", deliveryID).
Order("created_at DESC").
First(&lastResult).Error
if err != nil {
return true
}
backoff := calcBackoff(attemptNum)
return time.Since(lastResult.CreatedAt) >= backoff
}
func calcBackoff(attemptNum int) time.Duration {
shift := max(attemptNum-1, 0)
shift = min(shift, maxBackoffShift)
return time.Duration(1<<uint(shift)) * time.Second
}
// httpTarget delivers events to http targets. It forwards the
// event body and (filtered) request headers to the configured
// URL and owns retry, backoff, and circuit breaking through
// the shared httpCore.
type httpTarget struct {
*httpCore
client *http.Client
}
// Deliver implements Target.
func (t *httpTarget) Deliver(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
) {
cfg, err := parseHTTPConfig(d.Target.Config)
if err != nil {
t.eng.log.Error(
"invalid HTTP target config",
"target_id", d.TargetID,
"error", err,
)
t.eng.recordResult(
webhookDB, d, task.AttemptNum,
false, 0, "", err.Error(), 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusFailed,
)
return
}
attempt := func() attemptResult {
return t.attempt(ctx, cfg, &d.Event)
}
t.deliver(
webhookDB, d, task, sched,
d.Target.MaxRetries, attempt,
)
}
// attempt performs a single HTTP delivery attempt and derives
// the success flag and error message the same way the engine
// did: a non-2xx response is a failure but carries no error
// string; only a transport-level error does.
func (t *httpTarget) attempt(
ctx context.Context,
cfg *HTTPTargetConfig,
event *database.Event,
) attemptResult {
statusCode, respBody, duration, reqErr :=
t.doHTTPRequest(ctx, cfg, event)
success := reqErr == nil &&
statusCode >= httpSuccessMin &&
statusCode < httpSuccessMax
errMsg := ""
if reqErr != nil {
errMsg = reqErr.Error()
}
return attemptResult{
statusCode: statusCode,
respBody: respBody,
duration: duration,
success: success,
errMsg: errMsg,
}
}
func (t *httpTarget) doHTTPRequest(
ctx context.Context,
cfg *HTTPTargetConfig,
event *database.Event,
) (int, string, int64, error) {
start := time.Now()
req, reqErr := http.NewRequestWithContext(
ctx,
http.MethodPost,
cfg.URL,
bytes.NewReader([]byte(event.Body)),
)
if reqErr != nil {
return 0, "", 0, fmt.Errorf(
"creating request: %w", reqErr,
)
}
applyRequestHeaders(req, event, cfg)
client := t.clientForConfig(cfg)
resp, doErr := executeHTTPRequest(client, req)
dur := time.Since(start).Milliseconds()
if doErr != nil {
return 0, "", dur, fmt.Errorf(
"sending request: %w", doErr,
)
}
defer func() { _ = resp.Body.Close() }()
body, readErr := io.ReadAll(
io.LimitReader(resp.Body, maxBodyLog),
)
if readErr != nil {
return resp.StatusCode, "", dur,
fmt.Errorf(
"reading response body: %w", readErr,
)
}
return resp.StatusCode, string(body), dur, nil
}
func (t *httpTarget) clientForConfig(
cfg *HTTPTargetConfig,
) *http.Client {
if cfg.Timeout > 0 {
// Reuse the shared client's SSRF-safe transport so
// a per-target timeout does not drop the
// request-time private-IP guard. Only the timeout
// is overridden.
return &http.Client{
Timeout: time.Duration(
cfg.Timeout,
) * time.Second,
Transport: t.client.Transport,
}
}
return t.client
}
func parseHTTPConfig(
configJSON string,
) (*HTTPTargetConfig, error) {
if configJSON == "" {
return nil, errEmptyTargetConfig
}
var cfg HTTPTargetConfig
err := json.Unmarshal(
[]byte(configJSON), &cfg,
)
if err != nil {
return nil, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
if cfg.URL == "" {
return nil, errMissingTargetURL
}
return &cfg, nil
}
// isForwardableHeader returns true if the header should
// be forwarded to targets.
func isForwardableHeader(name string) bool {
switch http.CanonicalHeaderKey(name) {
case "Host", "Connection", "Keep-Alive",
"Transfer-Encoding", "Te", "Trailer",
"Upgrade", "Proxy-Authorization",
"Proxy-Connection", "Content-Length":
return false
default:
return true
}
}
func applyRequestHeaders(
req *http.Request,
event *database.Event,
cfg *HTTPTargetConfig,
) {
if event.ContentType != "" {
req.Header.Set(
"Content-Type", event.ContentType,
)
}
var originalHeaders map[string][]string
if event.Headers != "" {
jsonErr := json.Unmarshal(
[]byte(event.Headers),
&originalHeaders,
)
if jsonErr == nil {
for k, vals := range originalHeaders {
if isForwardableHeader(k) {
for _, v := range vals {
req.Header.Add(k, v)
}
}
}
}
}
for k, v := range cfg.Headers {
req.Header.Set(k, v)
}
req.Header.Set("User-Agent", "webhooker/1.0")
}
// executeHTTPRequest sends an HTTP request using the provided
// client. URLs are validated by the config parsers and the
// SSRF-safe transport before reaching here.
func executeHTTPRequest(
client *http.Client, req *http.Request,
) (*http.Response, error) {
return client.Do(req) //#nosec G704 -- URL validated by parseHTTPConfig/parseSlackConfig and SSRF-safe transport
}

View File

@@ -0,0 +1,47 @@
package delivery
import (
"context"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// logTarget is a fire-and-forget target that logs the entire
// inbound webhook — the full request body and headers, plus
// the method, content type, and the webhook and entrypoint
// ids — then records a single successful attempt.
type logTarget struct {
eng *Engine
}
// Deliver implements Target.
func (t *logTarget) Deliver(
_ context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
_ *Task,
_ Scheduler,
) {
t.eng.log.Info(
"webhook event delivered to log target",
"delivery_id", d.ID,
"event_id", d.EventID,
"target_id", d.TargetID,
"target_name", d.Target.Name,
"webhook_id", d.Event.WebhookID,
"entrypoint_id", d.Event.EntrypointID,
"method", d.Event.Method,
"content_type", d.Event.ContentType,
"headers", d.Event.Headers,
"body", d.Event.Body,
)
t.eng.recordResult(
webhookDB, d, 1, true, 0, "", "", 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusDelivered,
)
}

View File

@@ -0,0 +1,299 @@
package delivery
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// errMissingWebhookURL is returned when a Slack target config
// omits its webhook URL.
var errMissingWebhookURL = errors.New(
"webhook_url is required",
)
// SlackTargetConfig holds configuration for slack target
// types.
type SlackTargetConfig struct {
WebhookURL string `json:"webhookUrl"`
}
// slackTarget delivers events to Slack incoming webhooks. It
// formats the event into a Slack message and posts it as
// JSON. It shares the retry core with the HTTP target: a
// MaxRetries of 0 stays single-attempt fire-and-forget
// (preserving existing Slack targets), while a positive
// MaxRetries adds backoff and circuit breaking.
type slackTarget struct {
*httpCore
client *http.Client
}
// Deliver implements Target.
func (t *slackTarget) Deliver(
ctx context.Context,
webhookDB *gorm.DB,
d *database.Delivery,
task *Task,
sched Scheduler,
) {
cfg, err := parseSlackConfig(d.Target.Config)
if err != nil {
t.eng.log.Error(
"invalid Slack target config",
"target_id", d.TargetID,
"error", err,
)
t.failConfig(webhookDB, d, err)
return
}
msg := FormatSlackMessage(&d.Event)
payload, err := json.Marshal(
map[string]string{"text": msg},
)
if err != nil {
t.eng.log.Error(
"failed to marshal Slack payload",
"target_id", d.TargetID,
"error", err,
)
t.failConfig(webhookDB, d, err)
return
}
attempt := func() attemptResult {
return t.attempt(ctx, cfg, payload)
}
t.deliver(
webhookDB, d, task, sched,
d.Target.MaxRetries, attempt,
)
}
// failConfig records a first-attempt failure for a delivery
// that could not be prepared (bad config or unmarshalable
// payload) and marks it failed.
func (t *slackTarget) failConfig(
webhookDB *gorm.DB,
d *database.Delivery,
err error,
) {
t.eng.recordResult(
webhookDB, d, 1,
false, 0, "", err.Error(), 0,
)
t.eng.updateDeliveryStatus(
webhookDB, d, database.DeliveryStatusFailed,
)
}
// attempt performs a single Slack POST and derives its
// outcome, preserving the engine's original semantics: a
// non-2xx response records an "HTTP <code>" error string and
// a transport error records a "sending request" error.
func (t *slackTarget) attempt(
ctx context.Context,
cfg *SlackTargetConfig,
payload []byte,
) attemptResult {
start := time.Now()
req, err := http.NewRequestWithContext(
ctx,
http.MethodPost,
cfg.WebhookURL,
bytes.NewReader(payload),
)
if err != nil {
return attemptResult{
success: false,
errMsg: err.Error(),
}
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("User-Agent", "webhooker/1.0")
resp, doErr := executeHTTPRequest(t.client, req)
durationMs := time.Since(start).Milliseconds()
if doErr != nil {
return attemptResult{
success: false,
duration: durationMs,
errMsg: fmt.Errorf(
"sending request: %w", doErr,
).Error(),
}
}
defer func() { _ = resp.Body.Close() }()
return t.readSlackResponse(resp, durationMs)
}
func (t *slackTarget) readSlackResponse(
resp *http.Response,
durationMs int64,
) attemptResult {
body, readErr := io.ReadAll(
io.LimitReader(resp.Body, maxBodyLog),
)
if readErr != nil {
t.eng.log.Error(
"failed to read Slack response body",
"error", readErr,
)
}
success := resp.StatusCode >= httpSuccessMin &&
resp.StatusCode < httpSuccessMax
errMsg := ""
if !success {
errMsg = fmt.Sprintf("HTTP %d", resp.StatusCode)
}
return attemptResult{
statusCode: resp.StatusCode,
respBody: string(body),
duration: durationMs,
success: success,
errMsg: errMsg,
}
}
func parseSlackConfig(
configJSON string,
) (*SlackTargetConfig, error) {
if configJSON == "" {
return nil, errEmptyTargetConfig
}
var cfg SlackTargetConfig
err := json.Unmarshal(
[]byte(configJSON), &cfg,
)
if err != nil {
return nil, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
if cfg.WebhookURL == "" {
return nil, errMissingWebhookURL
}
return &cfg, nil
}
// FormatSlackMessage builds a Slack-compatible message
// string from a webhook event.
func FormatSlackMessage(
event *database.Event,
) string {
var b strings.Builder
b.WriteString("*Webhook Event Received*\n")
fmt.Fprintf(
&b, "*Method:* `%s`\n", event.Method,
)
fmt.Fprintf(
&b,
"*Content-Type:* `%s`\n",
event.ContentType,
)
fmt.Fprintf(
&b,
"*Timestamp:* `%s`\n",
event.CreatedAt.UTC().Format(time.RFC3339),
)
fmt.Fprintf(
&b,
"*Body Size:* %d bytes\n",
len(event.Body),
)
if event.Body == "" {
b.WriteString("\n_(empty body)_\n")
return b.String()
}
if formatted := formatJSONBody(event.Body); formatted != "" {
b.WriteString(formatted)
return b.String()
}
formatRawBody(&b, event.Body)
return b.String()
}
func formatJSONBody(body string) string {
var parsed json.RawMessage
if json.Unmarshal([]byte(body), &parsed) != nil {
return ""
}
var pretty bytes.Buffer
if json.Indent(&pretty, parsed, "", " ") != nil {
return ""
}
var b strings.Builder
b.WriteString("\n```\n")
prettyStr := pretty.String()
const maxPayloadDisplay = 3500
if len(prettyStr) > maxPayloadDisplay {
b.WriteString(prettyStr[:maxPayloadDisplay])
b.WriteString("\n... (truncated)")
} else {
b.WriteString(prettyStr)
}
b.WriteString("\n```\n")
return b.String()
}
func formatRawBody(b *strings.Builder, body string) {
b.WriteString("\n```\n")
const maxRawDisplay = 3500
if len(body) > maxRawDisplay {
b.WriteString(body[:maxRawDisplay])
b.WriteString("\n... (truncated)")
} else {
b.WriteString(body)
}
b.WriteString("\n```\n")
}

View File

@@ -1,28 +1,34 @@
// Package globals provides build-time variables injected via ldflags.
package globals package globals
import ( import (
"go.uber.org/fx" "go.uber.org/fx"
) )
// these get populated from main() and copied into the Globals object. // Build-time variables populated from main() and copied into the
// Globals object.
//
//nolint:gochecknoglobals // Build-time variables set by main().
var ( var (
Appname string Appname string
Version string Version string
Buildarch string
) )
// Globals holds build-time metadata about the application.
type Globals struct { type Globals struct {
Appname string Appname string
Version string Version string
Buildarch string
} }
// nolint:revive // lc parameter is required by fx even if unused // New creates a Globals instance from the package-level
// build-time variables.
//
//nolint:revive // lc parameter is required by fx even if unused.
func New(lc fx.Lifecycle) (*Globals, error) { func New(lc fx.Lifecycle) (*Globals, error) {
n := &Globals{ n := &Globals{
Appname: Appname, Appname: Appname,
Buildarch: Buildarch,
Version: Version, Version: Version,
} }
return n, nil return n, nil
} }

View File

@@ -1,30 +1,30 @@
package globals package globals_test
import ( import (
"testing" "testing"
"go.uber.org/fx/fxtest" "sneak.berlin/go/webhooker/internal/globals"
) )
func TestNew(t *testing.T) { func TestGlobalsFields(t *testing.T) {
// Set test values t.Parallel()
Appname = "test-app"
Version = "1.0.0"
Buildarch = "test-arch"
lc := fxtest.NewLifecycle(t) g := &globals.Globals{
globals, err := New(lc) Appname: "test-app",
if err != nil { Version: "1.0.0",
t.Fatalf("New() error = %v", err)
} }
if globals.Appname != "test-app" { if g.Appname != "test-app" {
t.Errorf("Appname = %v, want %v", globals.Appname, "test-app") t.Errorf(
"Appname = %v, want %v",
g.Appname, "test-app",
)
} }
if globals.Version != "1.0.0" {
t.Errorf("Version = %v, want %v", globals.Version, "1.0.0") if g.Version != "1.0.0" {
} t.Errorf(
if globals.Buildarch != "test-arch" { "Version = %v, want %v",
t.Errorf("Buildarch = %v, want %v", globals.Buildarch, "test-arch") g.Version, "1.0.0",
)
} }
} }

View File

@@ -13,11 +13,12 @@ func (h *Handlers) HandleLoginPage() http.HandlerFunc {
sess, err := h.session.Get(r) sess, err := h.session.Get(r)
if err == nil && h.session.IsAuthenticated(sess) { if err == nil && h.session.IsAuthenticated(sess) {
http.Redirect(w, r, "/", http.StatusSeeOther) http.Redirect(w, r, "/", http.StatusSeeOther)
return return
} }
// Render login page // Render login page
data := map[string]interface{}{ data := map[string]any{
"Error": "", "Error": "",
} }
@@ -28,10 +29,15 @@ func (h *Handlers) HandleLoginPage() http.HandlerFunc {
// HandleLoginSubmit handles the login form submission (POST) // HandleLoginSubmit handles the login form submission (POST)
func (h *Handlers) HandleLoginSubmit() http.HandlerFunc { func (h *Handlers) HandleLoginSubmit() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
// Limit request body to prevent memory exhaustion
r.Body = http.MaxBytesReader(w, r.Body, 1<<maxBodyShift)
// Parse form data // Parse form data
if err := r.ParseForm(); err != nil { err := r.ParseForm()
if err != nil {
h.log.Error("failed to parse form", "error", err) h.log.Error("failed to parse form", "error", err)
http.Error(w, "Bad request", http.StatusBadRequest) http.Error(w, "Bad request", http.StatusBadRequest)
return return
} }
@@ -40,76 +46,159 @@ func (h *Handlers) HandleLoginSubmit() http.HandlerFunc {
// Validate input // Validate input
if username == "" || password == "" { if username == "" || password == "" {
data := map[string]interface{}{ h.renderLoginError(
"Error": "Username and password are required", w, r,
} "Username and password are required",
w.WriteHeader(http.StatusBadRequest) http.StatusBadRequest,
h.renderTemplate(w, r, "login.html", data) )
return return
} }
// Find user in database user, err := h.authenticateUser(
var user database.User w, r, username, password,
if err := h.db.DB().Where("username = ?", username).First(&user).Error; err != nil { )
h.log.Debug("user not found", "username", username)
data := map[string]interface{}{
"Error": "Invalid username or password",
}
w.WriteHeader(http.StatusUnauthorized)
h.renderTemplate(w, r, "login.html", data)
return
}
// Verify password
valid, err := database.VerifyPassword(password, user.Password)
if err != nil { if err != nil {
h.log.Error("failed to verify password", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
return return
} }
if !valid { err = h.createAuthenticatedSession(w, r, user)
h.log.Debug("invalid password", "username", username)
data := map[string]interface{}{
"Error": "Invalid username or password",
}
w.WriteHeader(http.StatusUnauthorized)
h.renderTemplate(w, r, "login.html", data)
return
}
// Create session
sess, err := h.session.Get(r)
if err != nil { if err != nil {
h.log.Error("failed to get session", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
return return
} }
// Set user in session h.log.Info(
h.session.SetUser(sess, user.ID, user.Username) "user logged in",
"username", username,
// Save session "user_id", user.ID,
if err := h.session.Save(r, w, sess); err != nil { )
h.log.Error("failed to save session", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
return
}
h.log.Info("user logged in", "username", username, "user_id", user.ID)
// Redirect to home page // Redirect to home page
http.Redirect(w, r, "/", http.StatusSeeOther) http.Redirect(w, r, "/", http.StatusSeeOther)
} }
} }
// renderLoginError renders the login page with an error message.
func (h *Handlers) renderLoginError(
w http.ResponseWriter,
r *http.Request,
msg string,
status int,
) {
data := map[string]any{
"Error": msg,
}
w.WriteHeader(status)
h.renderTemplate(w, r, "login.html", data)
}
// authenticateUser looks up and verifies a user's credentials.
// On failure it writes an HTTP response and returns an error.
func (h *Handlers) authenticateUser(
w http.ResponseWriter,
r *http.Request,
username, password string,
) (database.User, error) {
var user database.User
err := h.db.DB().Where(
"username = ?", username,
).First(&user).Error
if err != nil {
h.log.Debug("user not found", "username", username)
h.renderLoginError(
w, r,
"Invalid username or password",
http.StatusUnauthorized,
)
return user, err
}
valid, err := database.VerifyPassword(password, user.Password)
if err != nil {
h.log.Error("failed to verify password", "error", err)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return user, err
}
if !valid {
h.log.Debug("invalid password", "username", username)
h.renderLoginError(
w, r,
"Invalid username or password",
http.StatusUnauthorized,
)
return user, errInvalidPassword
}
return user, nil
}
// createAuthenticatedSession regenerates the session and stores
// user info. On failure it writes an HTTP response and returns
// an error.
func (h *Handlers) createAuthenticatedSession(
w http.ResponseWriter,
r *http.Request,
user database.User,
) error {
oldSess, err := h.session.Get(r)
if err != nil {
h.log.Error("failed to get session", "error", err)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return err
}
sess, err := h.session.Regenerate(r, w, oldSess)
if err != nil {
h.log.Error(
"failed to regenerate session", "error", err,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return err
}
h.session.SetUser(sess, user.ID, user.Username)
err = h.session.Save(r, w, sess)
if err != nil {
h.log.Error("failed to save session", "error", err)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return err
}
return nil
}
// HandleLogout handles user logout // HandleLogout handles user logout
func (h *Handlers) HandleLogout() http.HandlerFunc { func (h *Handlers) HandleLogout() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
sess, err := h.session.Get(r) sess, err := h.session.Get(r)
if err != nil { if err != nil {
h.log.Error("failed to get session", "error", err) h.log.Error("failed to get session", "error", err)
http.Redirect(w, r, "/pages/login", http.StatusSeeOther) http.Redirect(
w, r, "/pages/login", http.StatusSeeOther,
)
return return
} }
@@ -117,8 +206,12 @@ func (h *Handlers) HandleLogout() http.HandlerFunc {
h.session.Destroy(sess) h.session.Destroy(sess)
// Save the destroyed session // Save the destroyed session
if err := h.session.Save(r, w, sess); err != nil { err = h.session.Save(r, w, sess)
h.log.Error("failed to save destroyed session", "error", err) if err != nil {
h.log.Error(
"failed to save destroyed session",
"error", err,
)
} }
// Redirect to login page // Redirect to login page

View File

@@ -0,0 +1,24 @@
package handlers
import "net/http"
// RenderTemplateForTest exposes renderTemplate for use in the
// handlers_test package.
func (s *Handlers) RenderTemplateForTest(
w http.ResponseWriter,
r *http.Request,
pageTemplate string,
data any,
) {
s.renderTemplate(w, r, pageTemplate, data)
}
// BuildSlackTargetConfigForTest exposes buildSlackTargetConfig
// for use in the handlers_test package.
func (s *Handlers) BuildSlackTargetConfigForTest(
w http.ResponseWriter,
r *http.Request,
targetURL string,
) (string, error) {
return s.buildSlackTargetConfig(w, r, targetURL)
}

View File

@@ -1,75 +1,127 @@
// Package handlers provides HTTP request handlers for the
// webhooker web UI and API.
package handlers package handlers
import ( import (
"context" "context"
"encoding/json" "encoding/json"
"errors"
"html/template" "html/template"
"log/slog" "log/slog"
"net/http" "net/http"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/healthcheck" "sneak.berlin/go/webhooker/internal/healthcheck"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
"sneak.berlin/go/webhooker/templates" "sneak.berlin/go/webhooker/templates"
) )
// nolint:revive // HandlersParams is a standard fx naming convention const (
// maxBodyShift is the bit shift for 1 MB body limit.
maxBodyShift = 20
// recentEventLimit is the number of recent events to show.
recentEventLimit = 20
// defaultRetentionDays is the default event retention period.
defaultRetentionDays = 30
// paginationPerPage is the number of items per page.
paginationPerPage = 25
)
// errInvalidPassword is returned when a password does not match.
var errInvalidPassword = errors.New("invalid password")
//nolint:revive // HandlersParams is a standard fx naming convention.
type HandlersParams struct { type HandlersParams struct {
fx.In fx.In
Logger *logger.Logger Logger *logger.Logger
Globals *globals.Globals Globals *globals.Globals
Database *database.Database Database *database.Database
WebhookDBMgr *database.WebhookDBManager
Healthcheck *healthcheck.Healthcheck Healthcheck *healthcheck.Healthcheck
Session *session.Session Session *session.Session
Notifier delivery.Notifier
} }
// Handlers provides HTTP handler methods for all application
// routes.
type Handlers struct { type Handlers struct {
params *HandlersParams params *HandlersParams
log *slog.Logger log *slog.Logger
hc *healthcheck.Healthcheck hc *healthcheck.Healthcheck
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager
session *session.Session session *session.Session
notifier delivery.Notifier
templates map[string]*template.Template templates map[string]*template.Template
} }
// parsePageTemplate parses a page-specific template set from the embedded FS. // parsePageTemplate parses a page-specific template set from the
// Each page template is combined with the shared base, htmlheader, and navbar templates. // embedded FS. Each page template is combined with the shared
// base, htmlheader, and navbar templates. The page file must be
// listed first so that its root action ({{template "base" .}})
// becomes the template set's entry point.
func parsePageTemplate(pageFile string) *template.Template { func parsePageTemplate(pageFile string) *template.Template {
return template.Must( return template.Must(
template.ParseFS(templates.Templates, "htmlheader.html", "navbar.html", "base.html", pageFile), template.ParseFS(
templates.Templates,
pageFile,
"base.html",
"htmlheader.html",
"navbar.html",
),
) )
} }
func New(lc fx.Lifecycle, params HandlersParams) (*Handlers, error) { // New creates a Handlers instance, parsing all page templates at
// startup.
func New(
lc fx.Lifecycle,
params HandlersParams,
) (*Handlers, error) {
s := new(Handlers) s := new(Handlers)
s.params = &params s.params = &params
s.log = params.Logger.Get() s.log = params.Logger.Get()
s.hc = params.Healthcheck s.hc = params.Healthcheck
s.db = params.Database s.db = params.Database
s.dbMgr = params.WebhookDBMgr
s.session = params.Session s.session = params.Session
s.notifier = params.Notifier
// Parse all page templates once at startup // Parse all page templates once at startup
s.templates = map[string]*template.Template{ s.templates = map[string]*template.Template{
"index.html": parsePageTemplate("index.html"),
"login.html": parsePageTemplate("login.html"), "login.html": parsePageTemplate("login.html"),
"profile.html": parsePageTemplate("profile.html"), "profile.html": parsePageTemplate("profile.html"),
"sources_list.html": parsePageTemplate("sources_list.html"),
"sources_new.html": parsePageTemplate("sources_new.html"),
"source_detail.html": parsePageTemplate("source_detail.html"),
"source_edit.html": parsePageTemplate("source_edit.html"),
"source_logs.html": parsePageTemplate("source_logs.html"),
} }
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error { OnStart: func(_ context.Context) error {
return nil return nil
}, },
}) })
return s, nil return s, nil
} }
//nolint:unparam // r parameter will be used in the future for request context func (s *Handlers) respondJSON(
func (s *Handlers) respondJSON(w http.ResponseWriter, r *http.Request, data interface{}, status int) { w http.ResponseWriter,
_ *http.Request,
data any,
status int,
) {
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status) w.WriteHeader(status)
if data != nil { if data != nil {
err := json.NewEncoder(w).Encode(data) err := json.NewEncoder(w).Encode(data)
if err != nil { if err != nil {
@@ -78,17 +130,15 @@ func (s *Handlers) respondJSON(w http.ResponseWriter, r *http.Request, data inte
} }
} }
//nolint:unparam,unused // will be used for handling JSON requests // serverError logs an error and sends a 500 response.
func (s *Handlers) decodeJSON(w http.ResponseWriter, r *http.Request, v interface{}) error { func (s *Handlers) serverError(
return json.NewDecoder(r.Body).Decode(v) w http.ResponseWriter, msg string, err error,
} ) {
s.log.Error(msg, "error", err)
// TemplateData represents the common data passed to templates http.Error(
type TemplateData struct { w, "Internal server error",
User *UserInfo http.StatusInternalServerError,
Version string )
UserCount int64
Uptime string
} }
// UserInfo represents user information for templates // UserInfo represents user information for templates
@@ -97,52 +147,91 @@ type UserInfo struct {
Username string Username string
} }
// renderTemplate renders a pre-parsed template with common data // templateDataWrapper wraps non-map data with common fields.
func (s *Handlers) renderTemplate(w http.ResponseWriter, r *http.Request, pageTemplate string, data interface{}) { type templateDataWrapper struct {
User *UserInfo
CSRFToken string
Data any
}
// getUserInfo extracts user info from the session.
func (s *Handlers) getUserInfo(
r *http.Request,
) *UserInfo {
sess, err := s.session.Get(r)
if err != nil || !s.session.IsAuthenticated(sess) {
return nil
}
username, ok := s.session.GetUsername(sess)
if !ok {
return nil
}
userID, ok := s.session.GetUserID(sess)
if !ok {
return nil
}
return &UserInfo{ID: userID, Username: username}
}
// renderTemplate renders a pre-parsed template with common
// data
func (s *Handlers) renderTemplate(
w http.ResponseWriter,
r *http.Request,
pageTemplate string,
data any,
) {
tmpl, ok := s.templates[pageTemplate] tmpl, ok := s.templates[pageTemplate]
if !ok { if !ok {
s.log.Error("template not found", "template", pageTemplate) s.log.Error(
http.Error(w, "Internal server error", http.StatusInternalServerError) "template not found",
"template", pageTemplate,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return return
} }
// Get user from session if available userInfo := s.getUserInfo(r)
var userInfo *UserInfo csrfToken := middleware.CSRFToken(r)
sess, err := s.session.Get(r)
if err == nil && s.session.IsAuthenticated(sess) {
if username, ok := s.session.GetUsername(sess); ok {
if userID, ok := s.session.GetUserID(sess); ok {
userInfo = &UserInfo{
ID: userID,
Username: username,
}
}
}
}
// If data is a map, merge user info into it if m, ok := data.(map[string]any); ok {
if m, ok := data.(map[string]interface{}); ok {
m["User"] = userInfo m["User"] = userInfo
if err := tmpl.Execute(w, m); err != nil { m["CSRFToken"] = csrfToken
s.log.Error("failed to execute template", "error", err) s.executeTemplate(w, tmpl, m)
http.Error(w, "Internal server error", http.StatusInternalServerError)
}
return
}
// Wrap data with base template data return
type templateDataWrapper struct {
User *UserInfo
Data interface{}
} }
wrapper := templateDataWrapper{ wrapper := templateDataWrapper{
User: userInfo, User: userInfo,
CSRFToken: csrfToken,
Data: data, Data: data,
} }
if err := tmpl.Execute(w, wrapper); err != nil { s.executeTemplate(w, tmpl, wrapper)
s.log.Error("failed to execute template", "error", err) }
http.Error(w, "Internal server error", http.StatusInternalServerError)
// executeTemplate runs the template and handles errors.
func (s *Handlers) executeTemplate(
w http.ResponseWriter,
tmpl *template.Template,
data any,
) {
err := tmpl.Execute(w, data)
if err != nil {
s.log.Error(
"failed to execute template", "error", err,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
} }
} }

View File

@@ -1,10 +1,10 @@
package handlers package handlers_test
import ( import (
"context"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -12,120 +12,177 @@ import (
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/healthcheck" "sneak.berlin/go/webhooker/internal/healthcheck"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
func TestHandleIndex(t *testing.T) { type noopNotifier struct{}
var h *Handlers
app := fxtest.New( func (n *noopNotifier) Notify([]delivery.Task) {}
func newTestApp(
t *testing.T,
targets ...any,
) *fxtest.App {
t.Helper()
return fxtest.New(
t, t,
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
func() *config.Config { func() *config.Config {
return &config.Config{ return &config.Config{
// This is a base64 encoded 32-byte key: "test-session-key-32-bytes-long!!" DataDir: t.TempDir(),
SessionKey: "dGVzdC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
} }
}, },
func() *database.Database { database.New,
// Mock database with a mock DB method database.NewWebhookDBManager,
db := &database.Database{}
return db
},
healthcheck.New, healthcheck.New,
session.New, session.New,
New, func() delivery.Notifier {
return &noopNotifier{}
},
handlers.New,
), ),
fx.Populate(&h), fx.Populate(targets...),
) )
app.RequireStart() }
defer app.RequireStop()
func TestHandleIndex_Unauthenticated(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder()
// Since we can't test actual template rendering without templates,
// let's test that the handler is created and doesn't panic
handler := h.HandleIndex() handler := h.HandleIndex()
assert.NotNil(t, handler) handler.ServeHTTP(w, req)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(
t, "/pages/login", w.Header().Get("Location"),
)
}
func TestHandleIndex_Authenticated(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder()
s, err := sess.Get(req)
require.NoError(t, err)
sess.SetUser(s, "test-user-id", "testuser")
err = sess.Save(req, w, s)
require.NoError(t, err)
req2 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
for _, cookie := range w.Result().Cookies() {
req2.AddCookie(cookie)
}
w2 := httptest.NewRecorder()
h.HandleIndex().ServeHTTP(w2, req2)
assert.Equal(t, http.StatusSeeOther, w2.Code)
assert.Equal(
t, "/sources", w2.Header().Get("Location"),
)
}
func TestBuildSlackTargetConfig_AcceptsPublicURL(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, "/", nil)
w := httptest.NewRecorder()
cfg, err := h.BuildSlackTargetConfigForTest(
w, req, "http://93.184.216.34/services/T00/B00/xxx",
)
require.NoError(t, err)
assert.Equal(t, http.StatusOK, w.Code)
assert.Contains(t, cfg, "webhookUrl")
}
func TestBuildSlackTargetConfig_RejectsReservedURL(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, "/", nil)
w := httptest.NewRecorder()
cfg, err := h.BuildSlackTargetConfigForTest(
w, req, "http://169.254.169.254/latest/meta-data/",
)
require.Error(t, err)
assert.Empty(t, cfg)
assert.Equal(t, http.StatusBadRequest, w.Code)
} }
func TestRenderTemplate(t *testing.T) { func TestRenderTemplate(t *testing.T) {
var h *Handlers t.Parallel()
app := fxtest.New( var h *handlers.Handlers
t,
fx.Provide( app := newTestApp(t, &h)
globals.New,
logger.New,
func() *config.Config {
return &config.Config{
// This is a base64 encoded 32-byte key: "test-session-key-32-bytes-long!!"
SessionKey: "dGVzdC1zZXNzaW9uLWtleS0zMi1ieXRlcy1sb25nISE=",
}
},
func() *database.Database {
// Mock database
return &database.Database{}
},
healthcheck.New,
session.New,
New,
),
fx.Populate(&h),
)
app.RequireStart() app.RequireStart()
defer app.RequireStop()
t.Run("handles missing templates gracefully", func(t *testing.T) { t.Cleanup(app.RequireStop)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder() w := httptest.NewRecorder()
data := map[string]interface{}{ data := map[string]any{"Version": "1.0.0"}
"Version": "1.0.0",
}
// When a non-existent template name is requested, renderTemplate h.RenderTemplateForTest(
// should return an internal server error w, req, "nonexistent.html", data,
h.renderTemplate(w, req, "nonexistent.html", data) )
// Should return internal server error when template is not found assert.Equal(
assert.Equal(t, http.StatusInternalServerError, w.Code) t, http.StatusInternalServerError, w.Code,
}) )
}
func TestFormatUptime(t *testing.T) {
tests := []struct {
name string
duration string
expected string
}{
{
name: "minutes only",
duration: "45m",
expected: "45m",
},
{
name: "hours and minutes",
duration: "2h30m",
expected: "2h 30m",
},
{
name: "days, hours and minutes",
duration: "25h45m",
expected: "1d 1h 45m",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
d, err := time.ParseDuration(tt.duration)
require.NoError(t, err)
result := formatUptime(d)
assert.Equal(t, tt.expected, result)
})
}
} }

View File

@@ -4,9 +4,13 @@ import (
"net/http" "net/http"
) )
const httpStatusOK = 200
// HandleHealthCheck returns an HTTP handler that reports
// application health.
func (s *Handlers) HandleHealthCheck() http.HandlerFunc { func (s *Handlers) HandleHealthCheck() http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) { return func(w http.ResponseWriter, req *http.Request) {
resp := s.hc.Healthcheck() resp := s.hc.Healthcheck()
s.respondJSON(w, req, resp, 200) s.respondJSON(w, req, resp, httpStatusOK)
} }
} }

View File

@@ -1,54 +1,21 @@
package handlers package handlers
import ( import (
"fmt"
"net/http" "net/http"
"time"
"sneak.berlin/go/webhooker/internal/database"
) )
type IndexResponse struct { // HandleIndex returns a handler for the root path that redirects
Message string `json:"message"` // based on authentication state: authenticated users go to /sources
Version string `json:"version"` // (the dashboard), unauthenticated users go to the login page.
}
func (s *Handlers) HandleIndex() http.HandlerFunc { func (s *Handlers) HandleIndex() http.HandlerFunc {
// Calculate server start time return func(w http.ResponseWriter, r *http.Request) {
startTime := time.Now() sess, err := s.session.Get(r)
if err == nil && s.session.IsAuthenticated(sess) {
http.Redirect(w, r, "/sources", http.StatusSeeOther)
return func(w http.ResponseWriter, req *http.Request) { return
// Calculate uptime
uptime := time.Since(startTime)
uptimeStr := formatUptime(uptime)
// Get user count from database
var userCount int64
s.db.DB().Model(&database.User{}).Count(&userCount)
// Prepare template data
data := map[string]interface{}{
"Version": s.params.Globals.Version,
"Uptime": uptimeStr,
"UserCount": userCount,
} }
// Render the template http.Redirect(w, r, "/pages/login", http.StatusSeeOther)
s.renderTemplate(w, req, "index.html", data)
} }
} }
// formatUptime formats a duration into a human-readable string
func formatUptime(d time.Duration) string {
days := int(d.Hours()) / 24
hours := int(d.Hours()) % 24
minutes := int(d.Minutes()) % 60
if days > 0 {
return fmt.Sprintf("%dd %dh %dm", days, hours, minutes)
}
if hours > 0 {
return fmt.Sprintf("%dh %dm", hours, minutes)
}
return fmt.Sprintf("%dm", minutes)
}

View File

@@ -13,14 +13,18 @@ func (h *Handlers) HandleProfile() http.HandlerFunc {
requestedUsername := chi.URLParam(r, "username") requestedUsername := chi.URLParam(r, "username")
if requestedUsername == "" { if requestedUsername == "" {
http.NotFound(w, r) http.NotFound(w, r)
return return
} }
// Get session // Get session. RequireAuth middleware guarantees an
// authenticated session before this handler runs, so we
// only need to guard against an unexpected retrieval error.
sess, err := h.session.Get(r) sess, err := h.session.Get(r)
if err != nil || !h.session.IsAuthenticated(sess) { if err != nil {
// Redirect to login if not authenticated h.log.Error("failed to get session", "error", err)
http.Redirect(w, r, "/pages/login", http.StatusSeeOther) http.Error(w, "Internal server error", http.StatusInternalServerError)
return return
} }
@@ -29,6 +33,7 @@ func (h *Handlers) HandleProfile() http.HandlerFunc {
if !ok { if !ok {
h.log.Error("authenticated session missing username") h.log.Error("authenticated session missing username")
http.Error(w, "Internal server error", http.StatusInternalServerError) http.Error(w, "Internal server error", http.StatusInternalServerError)
return return
} }
@@ -36,17 +41,19 @@ func (h *Handlers) HandleProfile() http.HandlerFunc {
if !ok { if !ok {
h.log.Error("authenticated session missing user ID") h.log.Error("authenticated session missing user ID")
http.Error(w, "Internal server error", http.StatusInternalServerError) http.Error(w, "Internal server error", http.StatusInternalServerError)
return return
} }
// For now, only allow users to view their own profile // For now, only allow users to view their own profile
if requestedUsername != sessionUsername { if requestedUsername != sessionUsername {
http.Error(w, "Forbidden", http.StatusForbidden) http.Error(w, "Forbidden", http.StatusForbidden)
return return
} }
// Prepare data for template // Prepare data for template
data := map[string]interface{}{ data := map[string]any{
"User": &UserInfo{ "User": &UserInfo{
ID: sessionUserID, ID: sessionUserID,
Username: sessionUsername, Username: sessionUsername,

View File

@@ -0,0 +1,159 @@
package handlers_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/session"
)
// authenticatedCookies creates an authenticated session for the given
// user and returns the resulting cookies for use on a later request.
func authenticatedCookies(
t *testing.T,
sess *session.Session,
userID, username string,
) []*http.Cookie {
t.Helper()
setupReq := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/setup", nil,
)
setupW := httptest.NewRecorder()
s, err := sess.Get(setupReq)
require.NoError(t, err)
sess.SetUser(s, userID, username)
require.NoError(t, sess.Save(setupReq, setupW, s))
cookies := setupW.Result().Cookies()
require.NotEmpty(t, cookies, "session cookie should be set")
return cookies
}
// profileRequest builds a GET request for the given profile username,
// attaching the supplied cookies and the chi URL parameter that the
// handler reads via chi.URLParam.
func profileRequest(
username string,
cookies []*http.Cookie,
) *http.Request {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/user/"+username, nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
rctx := chi.NewRouteContext()
rctx.URLParams.Add("username", username)
return req.WithContext(
context.WithValue(req.Context(), chi.RouteCtxKey, rctx),
)
}
func TestHandleProfile_OwnProfile_OK(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
cookies := authenticatedCookies(t, sess, "test-user-id", "testuser")
req := profileRequest("testuser", cookies)
w := httptest.NewRecorder()
h.HandleProfile().ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
}
func TestHandleProfile_OtherProfile_Forbidden(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
var sess *session.Session
app := newTestApp(t, &h, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
cookies := authenticatedCookies(t, sess, "test-user-id", "testuser")
req := profileRequest("otheruser", cookies)
w := httptest.NewRecorder()
h.HandleProfile().ServeHTTP(w, req)
assert.Equal(t, http.StatusForbidden, w.Code)
}
// TestUserRoute_Unauthenticated_RedirectedByMiddleware exercises the
// /user/{username} route group's middleware chain (CSRF then
// RequireAuth, matching setupUserRoutes) and proves that an
// unauthenticated request is redirected to /pages/login at the
// middleware layer, never reaching the endpoint handler.
func TestUserRoute_Unauthenticated_RedirectedByMiddleware(t *testing.T) {
t.Parallel()
var log *logger.Logger
var cfg *config.Config
var sess *session.Session
app := newTestApp(t, &log, &cfg, &sess)
app.RequireStart()
t.Cleanup(app.RequireStop)
mw := middleware.NewForTest(log.Get(), cfg, sess)
var handlerReached bool
router := chi.NewRouter()
router.Route("/user/{username}", func(r chi.Router) {
r.Use(mw.CSRF())
r.Use(mw.RequireAuth())
r.Get("/", func(w http.ResponseWriter, _ *http.Request) {
handlerReached = true
w.WriteHeader(http.StatusOK)
})
})
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/user/testuser", nil,
)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
assert.False(
t, handlerReached,
"handler must not be reached for unauthenticated request",
)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(t, "/pages/login", w.Header().Get("Location"))
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,42 +1,347 @@
package handlers package handlers
import ( import (
"encoding/json"
"io"
"net/http" "net/http"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// HandleWebhook handles incoming webhook requests at entrypoint URLs const (
// maxWebhookBodySize is the maximum allowed webhook
// request body (1 MB).
maxWebhookBodySize = 1 << maxBodyShift
)
// HandleWebhook handles incoming webhook requests at entrypoint
// URLs.
func (h *Handlers) HandleWebhook() http.HandlerFunc { func (h *Handlers) HandleWebhook() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
// Get entrypoint UUID from URL if r.Method != http.MethodPost {
entrypointUUID := chi.URLParam(r, "uuid") w.Header().Set("Allow", "POST")
if entrypointUUID == "" { http.Error(
http.NotFound(w, r) w,
"Method Not Allowed",
http.StatusMethodNotAllowed,
)
return
}
entrypointUUID := chi.URLParam(r, "uuid")
if entrypointUUID == "" {
http.NotFound(w, r)
return return
} }
// Log the incoming webhook request
h.log.Info("webhook request received", h.log.Info("webhook request received",
"entrypoint_uuid", entrypointUUID, "entrypoint_uuid", entrypointUUID,
"method", r.Method, "method", r.Method,
"remote_addr", r.RemoteAddr, "remote_addr", r.RemoteAddr,
"user_agent", r.UserAgent(),
) )
// Only POST methods are allowed for webhooks entrypoint, ok := h.lookupEntrypoint(
if r.Method != http.MethodPost { w, r, entrypointUUID,
w.Header().Set("Allow", "POST") )
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) if !ok {
return return
} }
// TODO: Implement webhook handling logic if !entrypoint.Active {
// Look up entrypoint by UUID, find parent webhook, fan out to targets http.Error(w, "Gone", http.StatusGone)
w.WriteHeader(http.StatusNotFound)
_, err := w.Write([]byte("unimplemented")) return
if err != nil {
h.log.Error("failed to write response", "error", err)
} }
h.processWebhookRequest(w, r, entrypoint)
} }
} }
// processWebhookRequest reads the body, serializes headers,
// loads targets, and delivers the event.
func (h *Handlers) processWebhookRequest(
w http.ResponseWriter,
r *http.Request,
entrypoint database.Entrypoint,
) {
body, ok := h.readWebhookBody(w, r)
if !ok {
return
}
headersJSON, err := json.Marshal(r.Header)
if err != nil {
h.serverError(w, "failed to serialize headers", err)
return
}
targets, err := h.loadActiveTargets(entrypoint.WebhookID)
if err != nil {
h.serverError(w, "failed to query targets", err)
return
}
h.createAndDeliverEvent(
w, r, entrypoint, body, headersJSON, targets,
)
}
// loadActiveTargets returns all active targets for a webhook.
func (h *Handlers) loadActiveTargets(
webhookID string,
) ([]database.Target, error) {
var targets []database.Target
err := h.db.DB().Where(
"webhook_id = ? AND active = ?",
webhookID, true,
).Find(&targets).Error
return targets, err
}
// lookupEntrypoint finds an entrypoint by UUID path.
func (h *Handlers) lookupEntrypoint(
w http.ResponseWriter,
r *http.Request,
entrypointUUID string,
) (database.Entrypoint, bool) {
var entrypoint database.Entrypoint
result := h.db.DB().Where(
"path = ?", entrypointUUID,
).First(&entrypoint)
if result.Error != nil {
h.log.Debug(
"entrypoint not found",
"path", entrypointUUID,
)
http.NotFound(w, r)
return entrypoint, false
}
return entrypoint, true
}
// readWebhookBody reads and validates the request body size.
func (h *Handlers) readWebhookBody(
w http.ResponseWriter,
r *http.Request,
) ([]byte, bool) {
body, err := io.ReadAll(
io.LimitReader(r.Body, maxWebhookBodySize+1),
)
if err != nil {
h.log.Error(
"failed to read request body", "error", err,
)
http.Error(
w, "Bad request", http.StatusBadRequest,
)
return nil, false
}
if len(body) > maxWebhookBodySize {
http.Error(
w,
"Request body too large",
http.StatusRequestEntityTooLarge,
)
return nil, false
}
return body, true
}
// createAndDeliverEvent creates the event and delivery records
// then notifies the delivery engine.
func (h *Handlers) createAndDeliverEvent(
w http.ResponseWriter,
r *http.Request,
entrypoint database.Entrypoint,
body, headersJSON []byte,
targets []database.Target,
) {
tx, err := h.beginWebhookTx(w, entrypoint.WebhookID)
if err != nil {
return
}
event := h.buildEvent(r, entrypoint, headersJSON, body)
err = tx.Create(event).Error
if err != nil {
tx.Rollback()
h.serverError(w, "failed to create event", err)
return
}
bodyPtr := inlineBody(body)
tasks := h.buildDeliveryTasks(
w, tx, event, entrypoint, targets, bodyPtr,
)
if tasks == nil {
return
}
err = tx.Commit().Error
if err != nil {
h.serverError(w, "failed to commit transaction", err)
return
}
h.finishWebhookResponse(w, event, entrypoint, tasks)
}
// beginWebhookTx opens a transaction on the per-webhook DB.
func (h *Handlers) beginWebhookTx(
w http.ResponseWriter,
webhookID string,
) (*gorm.DB, error) {
webhookDB, err := h.dbMgr.GetDB(webhookID)
if err != nil {
h.serverError(
w, "failed to get webhook database", err,
)
return nil, err
}
tx := webhookDB.Begin()
if tx.Error != nil {
h.serverError(
w, "failed to begin transaction", tx.Error,
)
return nil, tx.Error
}
return tx, nil
}
// inlineBody returns a pointer to body as a string if it fits
// within the inline size limit, or nil otherwise.
func inlineBody(body []byte) *string {
if len(body) < delivery.MaxInlineBodySize {
s := string(body)
return &s
}
return nil
}
// finishWebhookResponse notifies the delivery engine, logs the
// event, and writes the HTTP response.
func (h *Handlers) finishWebhookResponse(
w http.ResponseWriter,
event *database.Event,
entrypoint database.Entrypoint,
tasks []delivery.Task,
) {
if len(tasks) > 0 {
h.notifier.Notify(tasks)
}
h.log.Info("webhook event created",
"event_id", event.ID,
"webhook_id", entrypoint.WebhookID,
"entrypoint_id", entrypoint.ID,
"target_count", len(tasks),
)
w.WriteHeader(http.StatusOK)
_, err := w.Write([]byte(`{"status":"ok"}`))
if err != nil {
h.log.Error(
"failed to write response", "error", err,
)
}
}
// buildEvent creates a new Event struct from request data.
func (h *Handlers) buildEvent(
r *http.Request,
entrypoint database.Entrypoint,
headersJSON, body []byte,
) *database.Event {
return &database.Event{
WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID,
Method: r.Method,
Headers: string(headersJSON),
Body: string(body),
ContentType: r.Header.Get("Content-Type"),
}
}
// buildDeliveryTasks creates delivery records in the
// transaction and returns tasks for the delivery engine.
// Returns nil if an error occurred.
func (h *Handlers) buildDeliveryTasks(
w http.ResponseWriter,
tx *gorm.DB,
event *database.Event,
entrypoint database.Entrypoint,
targets []database.Target,
bodyPtr *string,
) []delivery.Task {
tasks := make([]delivery.Task, 0, len(targets))
for i := range targets {
dlv := &database.Delivery{
EventID: event.ID,
TargetID: targets[i].ID,
Status: database.DeliveryStatusPending,
}
err := tx.Create(dlv).Error
if err != nil {
tx.Rollback()
h.log.Error(
"failed to create delivery",
"target_id", targets[i].ID,
"error", err,
)
http.Error(
w, "Internal server error",
http.StatusInternalServerError,
)
return nil
}
tasks = append(tasks, delivery.Task{
DeliveryID: dlv.ID,
EventID: event.ID,
WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID,
TargetID: targets[i].ID,
TargetName: targets[i].Name,
TargetType: targets[i].Type,
TargetConfig: targets[i].Config,
MaxRetries: targets[i].MaxRetries,
Method: event.Method,
Headers: event.Headers,
ContentType: event.ContentType,
Body: bodyPtr,
AttemptNum: 1,
})
}
return tasks
}

View File

@@ -1,3 +1,4 @@
// Package healthcheck provides application health status reporting.
package healthcheck package healthcheck
import ( import (
@@ -12,55 +13,51 @@ import (
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
// nolint:revive // HealthcheckParams is a standard fx naming convention //nolint:revive // HealthcheckParams is a standard fx naming convention.
type HealthcheckParams struct { type HealthcheckParams struct {
fx.In fx.In
Globals *globals.Globals Globals *globals.Globals
Config *config.Config Config *config.Config
Logger *logger.Logger Logger *logger.Logger
Database *database.Database Database *database.Database
} }
// Healthcheck tracks application uptime and reports health status.
type Healthcheck struct { type Healthcheck struct {
StartupTime time.Time StartupTime time.Time
log *slog.Logger log *slog.Logger
params *HealthcheckParams params *HealthcheckParams
} }
func New(lc fx.Lifecycle, params HealthcheckParams) (*Healthcheck, error) { // New creates a Healthcheck that records the startup time on fx
// start.
func New(
lc fx.Lifecycle,
params HealthcheckParams,
) (*Healthcheck, error) {
s := new(Healthcheck) s := new(Healthcheck)
s.params = &params s.params = &params
s.log = params.Logger.Get() s.log = params.Logger.Get()
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(_ context.Context) error { // nolint:revive // ctx unused but required by fx OnStart: func(_ context.Context) error {
s.StartupTime = time.Now() s.StartupTime = time.Now()
return nil return nil
}, },
OnStop: func(ctx context.Context) error { OnStop: func(_ context.Context) error {
return nil return nil
}, },
}) })
return s, nil return s, nil
} }
// nolint:revive // HealthcheckResponse is a clear, descriptive name // Healthcheck returns the current health status of the
type HealthcheckResponse struct { // application.
Status string `json:"status"` func (s *Healthcheck) Healthcheck() *Response {
Now string `json:"now"` resp := &Response{
UptimeSeconds int64 `json:"uptime_seconds"`
UptimeHuman string `json:"uptime_human"`
Version string `json:"version"`
Appname string `json:"appname"`
Maintenance bool `json:"maintenance_mode"`
}
func (s *Healthcheck) uptime() time.Duration {
return time.Since(s.StartupTime)
}
func (s *Healthcheck) Healthcheck() *HealthcheckResponse {
resp := &HealthcheckResponse{
Status: "ok", Status: "ok",
Now: time.Now().UTC().Format(time.RFC3339Nano), Now: time.Now().UTC().Format(time.RFC3339Nano),
UptimeSeconds: int64(s.uptime().Seconds()), UptimeSeconds: int64(s.uptime().Seconds()),
@@ -69,5 +66,21 @@ func (s *Healthcheck) Healthcheck() *HealthcheckResponse {
Version: s.params.Globals.Version, Version: s.params.Globals.Version,
Maintenance: s.params.Config.MaintenanceMode, Maintenance: s.params.Config.MaintenanceMode,
} }
return resp return resp
} }
// Response contains the JSON-serialised health status.
type Response struct {
Status string `json:"status"`
Now string `json:"now"`
UptimeSeconds int64 `json:"uptimeSeconds"`
UptimeHuman string `json:"uptimeHuman"`
Version string `json:"version"`
Appname string `json:"appname"`
Maintenance bool `json:"maintenanceMode"`
}
func (s *Healthcheck) uptime() time.Duration {
return time.Since(s.StartupTime)
}

View File

@@ -1,3 +1,5 @@
// Package logger provides structured logging with dynamic level
// control.
package logger package logger
import ( import (
@@ -10,19 +12,25 @@ import (
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
) )
// nolint:revive // LoggerParams is a standard fx naming convention //nolint:revive // LoggerParams is a standard fx naming convention.
type LoggerParams struct { type LoggerParams struct {
fx.In fx.In
Globals *globals.Globals Globals *globals.Globals
} }
// Logger wraps slog with dynamic level control and structured
// output.
type Logger struct { type Logger struct {
logger *slog.Logger logger *slog.Logger
levelVar *slog.LevelVar levelVar *slog.LevelVar
params LoggerParams params LoggerParams
} }
// nolint:revive // lc parameter is required by fx even if unused // New creates a Logger that outputs text (TTY) or JSON (non-TTY)
// to stdout.
//
//nolint:revive // lc parameter is required by fx even if unused.
func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) { func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) {
l := new(Logger) l := new(Logger)
l.params = params l.params = params
@@ -37,17 +45,22 @@ func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) {
tty = true tty = true
} }
replaceAttr := func(_ []string, a slog.Attr) slog.Attr { // nolint:revive // groups unused //nolint:revive // groups param unused but required by slog ReplaceAttr signature.
replaceAttr := func(_ []string, a slog.Attr) slog.Attr {
// Always use UTC for timestamps // Always use UTC for timestamps
if a.Key == slog.TimeKey { if a.Key == slog.TimeKey {
if t, ok := a.Value.Any().(time.Time); ok { if t, ok := a.Value.Any().(time.Time); ok {
return slog.Time(slog.TimeKey, t.UTC()) return slog.Time(slog.TimeKey, t.UTC())
} }
return a
} }
return a return a
} }
var handler slog.Handler var handler slog.Handler
opts := &slog.HandlerOptions{ opts := &slog.HandlerOptions{
Level: l.levelVar, Level: l.levelVar,
ReplaceAttr: replaceAttr, ReplaceAttr: replaceAttr,
@@ -69,24 +82,27 @@ func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) {
return l, nil return l, nil
} }
// EnableDebugLogging switches the log level to debug.
func (l *Logger) EnableDebugLogging() { func (l *Logger) EnableDebugLogging() {
l.levelVar.Set(slog.LevelDebug) l.levelVar.Set(slog.LevelDebug)
l.logger.Debug("debug logging enabled", "debug", true) l.logger.Debug("debug logging enabled", "debug", true)
} }
// Get returns the underlying slog.Logger.
func (l *Logger) Get() *slog.Logger { func (l *Logger) Get() *slog.Logger {
return l.logger return l.logger
} }
// Identify logs the application name and version at startup.
func (l *Logger) Identify() { func (l *Logger) Identify() {
l.logger.Info("starting", l.logger.Info("starting",
"appname", l.params.Globals.Appname, "appname", l.params.Globals.Appname,
"version", l.params.Globals.Version, "version", l.params.Globals.Version,
"buildarch", l.params.Globals.Buildarch,
) )
} }
// Helper methods to maintain compatibility with existing code // Writer returns an io.Writer suitable for standard library
// loggers.
func (l *Logger) Writer() io.Writer { func (l *Logger) Writer() io.Writer {
return os.Stdout return os.Stdout
} }

View File

@@ -1,65 +1,59 @@
package logger package logger_test
import ( import (
"testing" "testing"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger"
) )
func testGlobals() *globals.Globals {
return &globals.Globals{
Appname: "test-app",
Version: "1.0.0",
}
}
func TestNew(t *testing.T) { func TestNew(t *testing.T) {
// Set up globals t.Parallel()
globals.Appname = "test-app"
globals.Version = "1.0.0"
globals.Buildarch = "test-arch"
lc := fxtest.NewLifecycle(t) lc := fxtest.NewLifecycle(t)
g, err := globals.New(lc)
if err != nil { params := logger.LoggerParams{
t.Fatalf("globals.New() error = %v", err) Globals: testGlobals(),
} }
params := LoggerParams{ l, err := logger.New(lc, params)
Globals: g,
}
logger, err := New(lc, params)
if err != nil { if err != nil {
t.Fatalf("New() error = %v", err) t.Fatalf("New() error = %v", err)
} }
if logger.Get() == nil { if l.Get() == nil {
t.Error("Get() returned nil logger") t.Error("Get() returned nil logger")
} }
// Test that we can log without panic // Test that we can log without panic
logger.Get().Info("test message", "key", "value") l.Get().Info("test message", "key", "value")
} }
func TestEnableDebugLogging(t *testing.T) { func TestEnableDebugLogging(t *testing.T) {
// Set up globals t.Parallel()
globals.Appname = "test-app"
globals.Version = "1.0.0"
globals.Buildarch = "test-arch"
lc := fxtest.NewLifecycle(t) lc := fxtest.NewLifecycle(t)
g, err := globals.New(lc)
if err != nil { params := logger.LoggerParams{
t.Fatalf("globals.New() error = %v", err) Globals: testGlobals(),
} }
params := LoggerParams{ l, err := logger.New(lc, params)
Globals: g,
}
logger, err := New(lc, params)
if err != nil { if err != nil {
t.Fatalf("New() error = %v", err) t.Fatalf("New() error = %v", err)
} }
// Enable debug logging should not panic // Enable debug logging should not panic
logger.EnableDebugLogging() l.EnableDebugLogging()
// Test debug logging // Test debug logging
logger.Get().Debug("debug message", "test", true) l.Get().Debug("debug message", "test", true)
} }

View File

@@ -0,0 +1,84 @@
package middleware
import (
"net/http"
"github.com/gorilla/csrf"
)
// CSRFToken retrieves the CSRF token from the request context.
// Returns an empty string if the gorilla/csrf middleware has not run.
func CSRFToken(r *http.Request) string {
return csrf.Token(r)
}
// isClientTLS reports whether the client-facing connection uses TLS.
// It checks for a direct TLS connection (r.TLS) or a TLS-terminating
// reverse proxy that sets the standard X-Forwarded-Proto header.
func isClientTLS(r *http.Request) bool {
return r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https"
}
// CSRF returns middleware that provides CSRF protection using the
// gorilla/csrf library. The middleware uses the session authentication
// key to sign a CSRF cookie and validates a masked token submitted via
// the "csrf_token" form field (or the "X-CSRF-Token" header) on
// POST/PUT/PATCH/DELETE requests. Requests with an invalid or missing
// token receive a 403 Forbidden response.
//
// The middleware detects the client-facing transport protocol per-request
// using r.TLS and the X-Forwarded-Proto header. This allows correct
// behavior in all deployment scenarios:
//
// - Direct HTTPS: strict Referer/Origin checks, Secure cookies.
// - Behind a TLS-terminating reverse proxy: strict checks (the
// browser is on HTTPS, so Origin/Referer headers use https://),
// Secure cookies (the browser sees HTTPS from the proxy).
// - Direct HTTP: relaxed Referer/Origin checks via PlaintextHTTPRequest,
// non-Secure cookies so the browser sends them over HTTP.
//
// Two gorilla/csrf instances are maintained — one with Secure cookies
// (for TLS) and one without (for plaintext HTTP) — because the
// csrf.Secure option is set at creation time, not per-request.
func (m *Middleware) CSRF() func(http.Handler) http.Handler {
csrfErrorHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
m.log.Warn("csrf: token validation failed",
"method", r.Method,
"path", r.URL.Path,
"remote_addr", r.RemoteAddr,
"reason", csrf.FailureReason(r),
)
http.Error(w, "Forbidden - invalid CSRF token", http.StatusForbidden)
})
key := m.session.GetKey()
baseOpts := []csrf.Option{
csrf.FieldName("csrf_token"),
csrf.SameSite(csrf.SameSiteLaxMode),
csrf.Path("/"),
csrf.ErrorHandler(csrfErrorHandler),
}
// Two middleware instances with different Secure flags but the
// same signing key, so cookies are interchangeable between them.
tlsProtect := csrf.Protect(key, append(baseOpts, csrf.Secure(true))...)
httpProtect := csrf.Protect(key, append(baseOpts, csrf.Secure(false))...)
return func(next http.Handler) http.Handler {
tlsCSRF := tlsProtect(next)
httpCSRF := httpProtect(next)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if isClientTLS(r) {
// Client is on TLS (directly or via reverse proxy).
// Use Secure cookies and strict Origin/Referer checks.
tlsCSRF.ServeHTTP(w, r)
} else {
// Plaintext HTTP: use non-Secure cookies and tell
// gorilla/csrf to use "http" for scheme comparisons,
// skipping the strict Referer check that assumes TLS.
httpCSRF.ServeHTTP(w, csrf.PlaintextHTTPRequest(r))
}
})
}
}

View File

@@ -0,0 +1,494 @@
package middleware_test
import (
"context"
"crypto/tls"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware"
)
// csrfCookieName is the gorilla/csrf cookie name.
const csrfCookieName = "_gorilla_csrf"
// csrfGetToken performs a GET request through the CSRF middleware
// and returns the token and cookies.
func csrfGetToken(
t *testing.T,
csrfMW func(http.Handler) http.Handler,
getReq *http.Request,
) (string, []*http.Cookie) {
t.Helper()
var token string
getHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, r *http.Request) {
token = middleware.CSRFToken(r)
},
))
getW := httptest.NewRecorder()
getHandler.ServeHTTP(getW, getReq)
cookies := getW.Result().Cookies()
require.NotEmpty(t, cookies, "CSRF cookie should be set")
require.NotEmpty(t, token, "CSRF token should be set")
return token, cookies
}
// csrfPostWithToken performs a POST request with the given CSRF
// token and cookies through the middleware. Returns whether the
// handler was called and the response code.
func csrfPostWithToken(
t *testing.T,
csrfMW func(http.Handler) http.Handler,
postReq *http.Request,
token string,
cookies []*http.Cookie,
) (bool, int) {
t.Helper()
var called bool
postHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
form := url.Values{"csrf_token": {token}}
postReq.Body = http.NoBody
postReq.Body = nil
// Rebuild the request with the form body
rebuilt := httptest.NewRequestWithContext(
context.Background(),
postReq.Method, postReq.URL.String(),
strings.NewReader(form.Encode()),
)
rebuilt.Header = postReq.Header.Clone()
rebuilt.TLS = postReq.TLS
rebuilt.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
rebuilt.AddCookie(c)
}
postW := httptest.NewRecorder()
postHandler.ServeHTTP(postW, rebuilt)
return called, postW.Code
}
func TestCSRF_GETSetsToken(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var gotToken string
handler := m.CSRF()(http.HandlerFunc(
func(_ http.ResponseWriter, r *http.Request) {
gotToken = middleware.CSRFToken(r)
},
))
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/form", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.NotEmpty(
t, gotToken,
"CSRF token should be set in context on GET",
)
}
func TestCSRF_POSTWithValidToken(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
csrfMW := m.CSRF()
getReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/form", nil,
)
token, cookies := csrfGetToken(t, csrfMW, getReq)
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/form", nil,
)
called, _ := csrfPostWithToken(
t, csrfMW, postReq, token, cookies,
)
assert.True(
t, called,
"handler should be called with valid CSRF token",
)
}
// csrfPOSTWithoutTokenTest is a shared helper for testing POST
// requests without a CSRF token in both dev and prod modes.
func csrfPOSTWithoutTokenTest(
t *testing.T,
env string,
msg string,
) {
t.Helper()
m, _ := testMiddleware(t, env)
csrfMW := m.CSRF()
// GET to establish the CSRF cookie
getHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {},
))
getReq := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/form", nil)
getW := httptest.NewRecorder()
getHandler.ServeHTTP(getW, getReq)
cookies := getW.Result().Cookies()
// POST without CSRF token
var called bool
postHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/form", nil,
)
postReq.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
postReq.AddCookie(c)
}
postW := httptest.NewRecorder()
postHandler.ServeHTTP(postW, postReq)
assert.False(t, called, msg)
assert.Equal(t, http.StatusForbidden, postW.Code)
}
func TestCSRF_POSTWithoutToken(t *testing.T) {
t.Parallel()
csrfPOSTWithoutTokenTest(
t,
config.EnvironmentDev,
"handler should NOT be called without CSRF token",
)
}
func TestCSRF_POSTWithInvalidToken(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
csrfMW := m.CSRF()
// GET to establish the CSRF cookie
getHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {},
))
getReq := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/form", nil)
getW := httptest.NewRecorder()
getHandler.ServeHTTP(getW, getReq)
cookies := getW.Result().Cookies()
// POST with wrong CSRF token
var called bool
postHandler := csrfMW(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
form := url.Values{"csrf_token": {"invalid-token-value"}}
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/form",
strings.NewReader(form.Encode()),
)
postReq.Header.Set(
"Content-Type", "application/x-www-form-urlencoded",
)
for _, c := range cookies {
postReq.AddCookie(c)
}
postW := httptest.NewRecorder()
postHandler.ServeHTTP(postW, postReq)
assert.False(
t, called,
"handler should NOT be called with invalid CSRF token",
)
assert.Equal(t, http.StatusForbidden, postW.Code)
}
func TestCSRF_GETDoesNotValidate(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.CSRF()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/form", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"GET requests should pass through CSRF middleware",
)
}
func TestCSRFToken_NoMiddleware(t *testing.T) {
t.Parallel()
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
assert.Empty(
t, middleware.CSRFToken(req),
"CSRFToken should return empty string when "+
"middleware has not run",
)
}
// --- TLS Detection Tests ---
func TestIsClientTLS_DirectTLS(t *testing.T) {
t.Parallel()
r := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
r.TLS = &tls.ConnectionState{}
assert.True(
t, middleware.IsClientTLS(r),
"should detect direct TLS connection",
)
}
func TestIsClientTLS_XForwardedProto(t *testing.T) {
t.Parallel()
r := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
r.Header.Set("X-Forwarded-Proto", "https")
assert.True(
t, middleware.IsClientTLS(r),
"should detect TLS via X-Forwarded-Proto",
)
}
func TestIsClientTLS_PlaintextHTTP(t *testing.T) {
t.Parallel()
r := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
assert.False(
t, middleware.IsClientTLS(r),
"should detect plaintext HTTP",
)
}
func TestIsClientTLS_XForwardedProtoHTTP(t *testing.T) {
t.Parallel()
r := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
r.Header.Set("X-Forwarded-Proto", "http")
assert.False(
t, middleware.IsClientTLS(r),
"should detect plaintext when X-Forwarded-Proto is http",
)
}
// --- Production Mode: POST over plaintext HTTP ---
func TestCSRF_ProdMode_PlaintextHTTP_POSTWithValidToken(
t *testing.T,
) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentProd)
csrfMW := m.CSRF()
getReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/form", nil,
)
token, cookies := csrfGetToken(t, csrfMW, getReq)
// Verify cookie is NOT Secure (plaintext HTTP in prod)
for _, c := range cookies {
if c.Name == csrfCookieName {
assert.False(t, c.Secure,
"CSRF cookie should not be Secure "+
"over plaintext HTTP")
}
}
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/form", nil,
)
called, code := csrfPostWithToken(
t, csrfMW, postReq, token, cookies,
)
assert.True(t, called,
"handler should be called -- prod mode over "+
"plaintext HTTP must work")
assert.NotEqual(t, http.StatusForbidden, code,
"should not return 403")
}
// --- Production Mode: POST with X-Forwarded-Proto ---
func TestCSRF_ProdMode_BehindProxy_POSTWithValidToken(
t *testing.T,
) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentProd)
csrfMW := m.CSRF()
getReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "http://example.com/form", nil,
)
getReq.Header.Set("X-Forwarded-Proto", "https")
token, cookies := csrfGetToken(t, csrfMW, getReq)
// Verify cookie IS Secure (X-Forwarded-Proto: https)
for _, c := range cookies {
if c.Name == csrfCookieName {
assert.True(t, c.Secure,
"CSRF cookie should be Secure behind "+
"TLS proxy")
}
}
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "http://example.com/form", nil,
)
postReq.Header.Set("X-Forwarded-Proto", "https")
postReq.Header.Set("Origin", "https://example.com")
called, code := csrfPostWithToken(
t, csrfMW, postReq, token, cookies,
)
assert.True(t, called,
"handler should be called -- prod mode behind "+
"TLS proxy must work")
assert.NotEqual(t, http.StatusForbidden, code,
"should not return 403")
}
// --- Production Mode: direct TLS ---
func TestCSRF_ProdMode_DirectTLS_POSTWithValidToken(
t *testing.T,
) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentProd)
csrfMW := m.CSRF()
getReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "https://example.com/form", nil,
)
getReq.TLS = &tls.ConnectionState{}
token, cookies := csrfGetToken(t, csrfMW, getReq)
// Verify cookie IS Secure (direct TLS)
for _, c := range cookies {
if c.Name == csrfCookieName {
assert.True(t, c.Secure,
"CSRF cookie should be Secure over "+
"direct TLS")
}
}
postReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "https://example.com/form", nil,
)
postReq.TLS = &tls.ConnectionState{}
postReq.Header.Set("Origin", "https://example.com")
called, code := csrfPostWithToken(
t, csrfMW, postReq, token, cookies,
)
assert.True(t, called,
"handler should be called -- direct TLS must work")
assert.NotEqual(t, http.StatusForbidden, code,
"should not return 403")
}
// --- Production Mode: POST without token still rejects ---
func TestCSRF_ProdMode_PlaintextHTTP_POSTWithoutToken(
t *testing.T,
) {
t.Parallel()
csrfPOSTWithoutTokenTest(
t,
config.EnvironmentProd,
"handler should NOT be called without CSRF token "+
"even in prod+plaintext",
)
}

View File

@@ -0,0 +1,34 @@
package middleware
import (
"net/http"
)
// NewLoggingResponseWriterForTest wraps newLoggingResponseWriter
// for use in external test packages.
func NewLoggingResponseWriterForTest(
w http.ResponseWriter,
) *loggingResponseWriter {
return newLoggingResponseWriter(w)
}
// LoggingResponseWriterStatusCode returns the status code
// captured by the loggingResponseWriter.
func LoggingResponseWriterStatusCode(
lrw *loggingResponseWriter,
) int {
return lrw.statusCode
}
// IPFromHostPort exposes ipFromHostPort for testing.
func IPFromHostPort(hp string) string {
return ipFromHostPort(hp)
}
// IsClientTLS exposes isClientTLS for testing.
func IsClientTLS(r *http.Request) bool {
return isClientTLS(r)
}
// LoginRateLimitConst exposes the loginRateLimit constant.
const LoginRateLimitConst = loginRateLimit

View File

@@ -1,3 +1,5 @@
// Package middleware provides HTTP middleware for logging, auth,
// CORS, and metrics.
package middleware package middleware
import ( import (
@@ -19,26 +21,42 @@ import (
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
// nolint:revive // MiddlewareParams is a standard fx naming convention const (
// corsMaxAge is the maximum time (in seconds) that a
// preflight response can be cached.
corsMaxAge = 300
)
//nolint:revive // MiddlewareParams is a standard fx naming convention.
type MiddlewareParams struct { type MiddlewareParams struct {
fx.In fx.In
Logger *logger.Logger Logger *logger.Logger
Globals *globals.Globals Globals *globals.Globals
Config *config.Config Config *config.Config
Session *session.Session Session *session.Session
} }
// Middleware provides HTTP middleware for logging, CORS, auth, and
// metrics.
type Middleware struct { type Middleware struct {
log *slog.Logger log *slog.Logger
params *MiddlewareParams params *MiddlewareParams
session *session.Session session *session.Session
} }
func New(lc fx.Lifecycle, params MiddlewareParams) (*Middleware, error) { // New creates a Middleware from the provided fx parameters.
//
//nolint:revive // lc parameter is required by fx even if unused.
func New(
lc fx.Lifecycle,
params MiddlewareParams,
) (*Middleware, error) {
s := new(Middleware) s := new(Middleware)
s.params = &params s.params = &params
s.log = params.Logger.Get() s.log = params.Logger.Get()
s.session = params.Session s.session = params.Session
return s, nil return s, nil
} }
@@ -50,19 +68,24 @@ func ipFromHostPort(hp string) string {
if err != nil { if err != nil {
return "" return ""
} }
if len(h) > 0 && h[0] == '[' { if len(h) > 0 && h[0] == '[' {
return h[1 : len(h)-1] return h[1 : len(h)-1]
} }
return h return h
} }
type loggingResponseWriter struct { type loggingResponseWriter struct {
http.ResponseWriter http.ResponseWriter
statusCode int statusCode int
} }
// nolint:revive // unexported type is only used internally // newLoggingResponseWriter wraps w and records status codes.
func NewLoggingResponseWriter(w http.ResponseWriter) *loggingResponseWriter { func newLoggingResponseWriter(
w http.ResponseWriter,
) *loggingResponseWriter {
return &loggingResponseWriter{w, http.StatusOK} return &loggingResponseWriter{w, http.StatusOK}
} }
@@ -71,23 +94,30 @@ func (lrw *loggingResponseWriter) WriteHeader(code int) {
lrw.ResponseWriter.WriteHeader(code) lrw.ResponseWriter.WriteHeader(code)
} }
// type Middleware func(http.Handler) http.Handler // Logging returns middleware that logs each HTTP request with
// this returns a Middleware that is designed to do every request through the // timing and metadata.
// mux, note the signature:
func (s *Middleware) Logging() func(http.Handler) http.Handler { func (s *Middleware) Logging() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
start := time.Now() start := time.Now()
lrw := NewLoggingResponseWriter(w) lrw := newLoggingResponseWriter(w)
ctx := r.Context() ctx := r.Context()
defer func() { defer func() {
latency := time.Since(start) latency := time.Since(start)
requestID := "" requestID := ""
if reqID := ctx.Value(middleware.RequestIDKey); reqID != nil {
if reqID := ctx.Value(
middleware.RequestIDKey,
); reqID != nil {
if id, ok := reqID.(string); ok { if id, ok := reqID.(string); ok {
requestID = id requestID = id
} }
} }
s.log.Info("http request", s.log.Info("http request",
"request_start", start, "request_start", start,
"method", r.Method, "method", r.Method,
@@ -107,39 +137,65 @@ func (s *Middleware) Logging() func(http.Handler) http.Handler {
} }
} }
// CORS returns middleware that sets CORS headers (permissive in
// dev, no-op in prod).
func (s *Middleware) CORS() func(http.Handler) http.Handler { func (s *Middleware) CORS() func(http.Handler) http.Handler {
if s.params.Config.IsDev() {
// In development, allow any origin for local testing.
return cors.Handler(cors.Options{ return cors.Handler(cors.Options{
// CHANGEME! these are defaults, change them to suit your needs or
// read from environment/viper.
// AllowedOrigins: []string{"https://foo.com"}, // Use this to allow specific origin hosts
AllowedOrigins: []string{"*"}, AllowedOrigins: []string{"*"},
// AllowOriginFunc: func(r *http.Request, origin string) bool { return true }, AllowedMethods: []string{
AllowedMethods: []string{"GET", "POST", "PUT", "DELETE", "OPTIONS"}, "GET", "POST", "PUT", "DELETE", "OPTIONS",
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "X-CSRF-Token"}, },
AllowedHeaders: []string{
"Accept", "Authorization",
"Content-Type", "X-CSRF-Token",
},
ExposedHeaders: []string{"Link"}, ExposedHeaders: []string{"Link"},
AllowCredentials: false, AllowCredentials: false,
MaxAge: 300, // Maximum value not ignored by any of major browsers MaxAge: corsMaxAge,
}) })
}
// In production, the web UI is server-rendered so
// cross-origin requests are not expected. Return a no-op
// middleware.
return func(next http.Handler) http.Handler {
return next
}
} }
// RequireAuth returns middleware that checks for a valid session. // RequireAuth returns middleware that checks for a valid session.
// Unauthenticated users are redirected to the login page. // Unauthenticated users are redirected to the login page.
func (s *Middleware) RequireAuth() func(http.Handler) http.Handler { func (s *Middleware) RequireAuth() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
sess, err := s.session.Get(r) sess, err := s.session.Get(r)
if err != nil { if err != nil {
s.log.Debug("auth middleware: failed to get session", "error", err) s.log.Debug(
http.Redirect(w, r, "/pages/login", http.StatusSeeOther) "auth middleware: failed to get session",
"error", err,
)
http.Redirect(
w, r, "/pages/login", http.StatusSeeOther,
)
return return
} }
if !s.session.IsAuthenticated(sess) { if !s.session.IsAuthenticated(sess) {
s.log.Debug("auth middleware: unauthenticated request", s.log.Debug(
"auth middleware: unauthenticated request",
"path", r.URL.Path, "path", r.URL.Path,
"method", r.Method, "method", r.Method,
) )
http.Redirect(w, r, "/pages/login", http.StatusSeeOther) http.Redirect(
w, r, "/pages/login", http.StatusSeeOther,
)
return return
} }
@@ -148,15 +204,19 @@ func (s *Middleware) RequireAuth() func(http.Handler) http.Handler {
} }
} }
// Metrics returns middleware that records Prometheus HTTP metrics.
func (s *Middleware) Metrics() func(http.Handler) http.Handler { func (s *Middleware) Metrics() func(http.Handler) http.Handler {
mdlw := ghmm.New(ghmm.Config{ mdlw := ghmm.New(ghmm.Config{
Recorder: metrics.NewRecorder(metrics.Config{}), Recorder: metrics.NewRecorder(metrics.Config{}),
}) })
return func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler {
return std.Handler("", mdlw, next) return std.Handler("", mdlw, next)
} }
} }
// MetricsAuth returns middleware that protects metrics endpoints
// with basic auth.
func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler { func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler {
return basicauth.New( return basicauth.New(
"metrics", "metrics",
@@ -167,3 +227,85 @@ func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler {
}, },
) )
} }
// SecurityHeaders returns middleware that sets production security
// headers on every response: HSTS, X-Content-Type-Options,
// X-Frame-Options, CSP, Referrer-Policy, and Permissions-Policy.
func (s *Middleware) SecurityHeaders() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
w.Header().Set(
"Strict-Transport-Security",
"max-age=63072000; includeSubDomains; preload",
)
w.Header().Set(
"X-Content-Type-Options", "nosniff",
)
w.Header().Set("X-Frame-Options", "DENY")
w.Header().Set(
"Content-Security-Policy",
"default-src 'self'; "+
"script-src 'self' 'unsafe-inline'; "+
"style-src 'self' 'unsafe-inline'",
)
w.Header().Set(
"Referrer-Policy",
"strict-origin-when-cross-origin",
)
w.Header().Set(
"Permissions-Policy",
"camera=(), microphone=(), geolocation=()",
)
next.ServeHTTP(w, r)
})
}
}
// NoCache returns middleware that instructs browsers and
// intermediary proxies not to cache the response. It sets
// Cache-Control: no-store and Pragma: no-cache (the latter for
// older HTTP/1.0 intermediaries). Apply it to authenticated pages
// so webhook configuration and captured event data are not stored
// by caches.
func (s *Middleware) NoCache() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
w.Header().Set("Cache-Control", "no-store")
w.Header().Set("Pragma", "no-cache")
next.ServeHTTP(w, r)
})
}
}
// MaxBodySize returns middleware that limits the request body size
// for POST requests. If the body exceeds the given limit in
// bytes, the server returns 413 Request Entity Too Large. This
// prevents clients from sending arbitrarily large form bodies.
func (s *Middleware) MaxBodySize(
maxBytes int64,
) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
if r.Method == http.MethodPost ||
r.Method == http.MethodPut ||
r.Method == http.MethodPatch {
r.Body = http.MaxBytesReader(
w, r.Body, maxBytes,
)
}
next.ServeHTTP(w, r)
})
}
}

View File

@@ -0,0 +1,627 @@
package middleware_test
import (
"context"
"encoding/base64"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"testing"
"github.com/gorilla/sessions"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/session"
)
const testKeySize = 32
// testMiddleware creates a Middleware with minimal dependencies
// for testing. It uses a real session.Session backed by an
// in-memory cookie store.
func testMiddleware(
t *testing.T,
env string,
) (*middleware.Middleware, *session.Session) {
t.Helper()
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
cfg := &config.Config{
Environment: env,
}
// Create a real session manager with a known key
key := make([]byte, testKeySize)
for i := range key {
key[i] = byte(i)
}
store := sessions.NewCookieStore(key)
store.Options = &sessions.Options{
Path: "/",
MaxAge: 86400 * 7,
HttpOnly: true,
Secure: false,
SameSite: http.SameSiteLaxMode,
}
sessManager := session.NewForTest(store, cfg, log, key)
m := middleware.NewForTest(log, cfg, sessManager)
return m, sessManager
}
// --- Logging Middleware Tests ---
func TestLogging_SetsStatusCode(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.Logging()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusCreated)
_, err := w.Write([]byte("created"))
if err != nil {
return
}
},
))
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/test", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(t, http.StatusCreated, w.Code)
assert.Equal(t, "created", w.Body.String())
}
func TestLogging_DefaultStatusOK(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.Logging()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
_, err := w.Write([]byte("ok"))
if err != nil {
return
}
},
))
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
// When no explicit WriteHeader is called, default is 200
assert.Equal(t, http.StatusOK, w.Code)
}
func TestLogging_PassesThroughToNext(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.Logging()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/api/webhook", nil,
)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"logging middleware should call the next handler",
)
}
// --- LoggingResponseWriter Tests ---
func TestLoggingResponseWriter_CapturesStatusCode(t *testing.T) {
t.Parallel()
w := httptest.NewRecorder()
lrw := middleware.NewLoggingResponseWriterForTest(w)
// Default should be 200
assert.Equal(
t, http.StatusOK,
middleware.LoggingResponseWriterStatusCode(lrw),
)
// WriteHeader should capture the status code
lrw.WriteHeader(http.StatusNotFound)
assert.Equal(
t, http.StatusNotFound,
middleware.LoggingResponseWriterStatusCode(lrw),
)
// Underlying writer should also get the status code
assert.Equal(t, http.StatusNotFound, w.Code)
}
func TestLoggingResponseWriter_WriteDelegatesToUnderlying(
t *testing.T,
) {
t.Parallel()
w := httptest.NewRecorder()
lrw := middleware.NewLoggingResponseWriterForTest(w)
n, err := lrw.Write([]byte("hello world"))
require.NoError(t, err)
assert.Equal(t, 11, n)
assert.Equal(t, "hello world", w.Body.String())
}
// --- CORS Middleware Tests ---
func TestCORS_DevMode_AllowsAnyOrigin(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.CORS()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
))
// Preflight request
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodOptions, "/api/test", nil,
)
req.Header.Set("Origin", "http://localhost:3000")
req.Header.Set("Access-Control-Request-Method", "POST")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
// In dev mode, CORS should allow any origin
assert.Equal(
t, "*",
w.Header().Get("Access-Control-Allow-Origin"),
)
}
func TestCORS_ProdMode_NoOp(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentProd)
var called bool
handler := m.CORS()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/api/test", nil,
)
req.Header.Set("Origin", "http://evil.com")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"prod CORS middleware should pass through to handler",
)
// In prod, no CORS headers should be set (no-op middleware)
assert.Empty(
t,
w.Header().Get("Access-Control-Allow-Origin"),
"prod mode should not set CORS headers",
)
}
// --- RequireAuth Middleware Tests ---
func TestRequireAuth_NoSession_RedirectsToLogin(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.RequireAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/dashboard", nil,
)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.False(
t, called,
"handler should not be called for "+
"unauthenticated request",
)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(t, "/pages/login", w.Header().Get("Location"))
}
func TestRequireAuth_AuthenticatedSession_PassesThrough(
t *testing.T,
) {
t.Parallel()
m, sessManager := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.RequireAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
// Create an authenticated session by making a request,
// setting session data, and saving the session cookie
setupReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/setup", nil,
)
setupW := httptest.NewRecorder()
sess, err := sessManager.Get(setupReq)
require.NoError(t, err)
sessManager.SetUser(sess, "user-123", "testuser")
require.NoError(t, sessManager.Save(setupReq, setupW, sess))
// Extract the cookie from the setup response
cookies := setupW.Result().Cookies()
require.NotEmpty(t, cookies, "session cookie should be set")
// Make the actual request with the session cookie
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/dashboard", nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"handler should be called for authenticated request",
)
}
func TestRequireAuth_UnauthenticatedSession_RedirectsToLogin(
t *testing.T,
) {
t.Parallel()
m, sessManager := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.RequireAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
// Create a session but don't authenticate it
setupReq := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/setup", nil,
)
setupW := httptest.NewRecorder()
sess, err := sessManager.Get(setupReq)
require.NoError(t, err)
// Don't call SetUser -- session exists but is not
// authenticated
require.NoError(t, sessManager.Save(setupReq, setupW, sess))
cookies := setupW.Result().Cookies()
require.NotEmpty(t, cookies)
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/dashboard", nil,
)
for _, c := range cookies {
req.AddCookie(c)
}
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.False(
t, called,
"handler should not be called for "+
"unauthenticated session",
)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(t, "/pages/login", w.Header().Get("Location"))
}
// --- NoCache Middleware Tests ---
func TestNoCache_SetsHeaders(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var called bool
handler := m.NoCache()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/sources", nil,
)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"NoCache middleware should call the next handler",
)
assert.Equal(
t, "no-store",
w.Header().Get("Cache-Control"),
)
assert.Equal(
t, "no-cache",
w.Header().Get("Pragma"),
)
}
// --- Helper Tests ---
func TestIpFromHostPort(t *testing.T) {
t.Parallel()
tests := []struct {
name string
input string
expected string
}{
{"ipv4 with port", "192.168.1.1:8080", "192.168.1.1"},
{"ipv6 with port", "[::1]:8080", "::1"},
{"invalid format", "not-a-host-port", ""},
{"empty string", "", ""},
{"localhost", "127.0.0.1:80", "127.0.0.1"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
result := middleware.IPFromHostPort(tt.input)
assert.Equal(t, tt.expected, result)
})
}
}
// --- MetricsAuth Tests ---
// metricsAuthMiddleware creates a Middleware configured for
// metrics auth testing. This helper de-duplicates the setup in
// metrics auth test functions.
func metricsAuthMiddleware(
t *testing.T,
) *middleware.Middleware {
t.Helper()
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
cfg := &config.Config{
Environment: config.EnvironmentDev,
MetricsUsername: "admin",
MetricsPassword: "secret",
}
key := make([]byte, testKeySize)
store := sessions.NewCookieStore(key)
store.Options = &sessions.Options{Path: "/", MaxAge: 86400}
sessManager := session.NewForTest(store, cfg, log, key)
return middleware.NewForTest(log, cfg, sessManager)
}
func TestMetricsAuth_ValidCredentials(t *testing.T) {
t.Parallel()
m := metricsAuthMiddleware(t)
var called bool
handler := m.MetricsAuth()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/metrics", nil,
)
req.SetBasicAuth("admin", "secret")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.True(
t, called,
"handler should be called with valid basic auth",
)
assert.Equal(t, http.StatusOK, w.Code)
}
func TestMetricsAuth_InvalidCredentials(t *testing.T) {
t.Parallel()
m := metricsAuthMiddleware(t)
var called bool
handler := m.MetricsAuth()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/metrics", nil,
)
req.SetBasicAuth("admin", "wrong-password")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.False(
t, called,
"handler should not be called with invalid basic auth",
)
assert.Equal(t, http.StatusUnauthorized, w.Code)
}
func TestMetricsAuth_NoCredentials(t *testing.T) {
t.Parallel()
m := metricsAuthMiddleware(t)
var called bool
handler := m.MetricsAuth()(http.HandlerFunc(
func(_ http.ResponseWriter, _ *http.Request) {
called = true
},
))
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/metrics", nil,
)
// No basic auth header
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.False(
t, called,
"handler should not be called without credentials",
)
assert.Equal(t, http.StatusUnauthorized, w.Code)
}
// --- CORS Dev Mode Detailed Tests ---
func TestCORS_DevMode_AllowsMethods(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.CORS()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
))
// Preflight for POST
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodOptions, "/api/webhooks", nil,
)
req.Header.Set("Origin", "http://localhost:5173")
req.Header.Set("Access-Control-Request-Method", "POST")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
allowMethods := w.Header().Get("Access-Control-Allow-Methods")
assert.Contains(t, allowMethods, "POST")
}
// --- Base64 key validation for completeness ---
func TestSessionKeyFormat(t *testing.T) {
t.Parallel()
// Verify that the session initialization correctly validates
// key format. A proper 32-byte key encoded as base64 should
// work.
key := make([]byte, testKeySize)
for i := range key {
key[i] = byte(i + 1)
}
encoded := base64.StdEncoding.EncodeToString(key)
decoded, err := base64.StdEncoding.DecodeString(encoded)
require.NoError(t, err)
assert.Len(t, decoded, testKeySize)
}

View File

@@ -0,0 +1,103 @@
package middleware
import (
"net/http"
"time"
"github.com/go-chi/httprate"
)
const (
// loginRateLimit is the maximum number of login attempts
// per interval.
loginRateLimit = 5
// loginRateInterval is the time window for the rate limit.
loginRateInterval = 1 * time.Minute
// receiverRateInterval is the time window for the webhook
// receiver rate limit. The configured limit is expressed in
// requests per minute.
receiverRateInterval = 1 * time.Minute
)
// LoginRateLimit returns middleware that enforces per-IP rate
// limiting on login attempts using go-chi/httprate. Only POST
// requests are rate-limited; GET requests (rendering the login
// form) pass through unaffected. When the rate limit is exceeded,
// a 429 Too Many Requests response is returned. IP extraction
// honours X-Forwarded-For, X-Real-IP, and True-Client-IP headers
// for reverse-proxy setups.
func (m *Middleware) LoginRateLimit() func(http.Handler) http.Handler {
limiter := httprate.Limit(
loginRateLimit,
loginRateInterval,
httprate.WithKeyFuncs(httprate.KeyByRealIP),
httprate.WithLimitHandler(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
m.log.Warn("login rate limit exceeded",
"path", r.URL.Path,
)
http.Error(
w,
"Too many login attempts. "+
"Please try again later.",
http.StatusTooManyRequests,
)
},
)),
)
return func(next http.Handler) http.Handler {
limited := limiter(next)
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
// Only rate-limit POST requests (actual login
// attempts)
if r.Method != http.MethodPost {
next.ServeHTTP(w, r)
return
}
limited.ServeHTTP(w, r)
})
}
}
// ReceiverRateLimit returns middleware that rate-limits the
// public webhook receiver endpoint per client IP per request
// path (the path contains the entrypoint UUID, so each sender
// is limited per entrypoint without affecting other senders or
// other entrypoints). The limit is Config.ReceiverRateLimit
// requests per minute. Requests over the limit receive a 429;
// httprate adds the Retry-After header (RFC 6585). IP
// extraction honours X-Forwarded-For, X-Real-IP, and
// True-Client-IP headers for reverse-proxy setups.
func (m *Middleware) ReceiverRateLimit() func(http.Handler) http.Handler {
return httprate.Limit(
m.params.Config.ReceiverRateLimit,
receiverRateInterval,
httprate.WithKeyFuncs(
httprate.KeyByRealIP,
httprate.KeyByEndpoint,
),
httprate.WithLimitHandler(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
m.log.Warn(
"webhook receiver rate limit exceeded",
"path", r.URL.Path,
)
http.Error(
w,
"Too many requests. "+
"Please slow down.",
http.StatusTooManyRequests,
)
},
)),
)
}

View File

@@ -0,0 +1,240 @@
package middleware_test
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"testing"
"github.com/stretchr/testify/assert"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware"
)
func TestLoginRateLimit_AllowsGET(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var callCount int
handler := m.LoginRateLimit()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
callCount++
w.WriteHeader(http.StatusOK)
},
))
// GET requests should never be rate-limited
for i := range 20 {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/pages/login", nil,
)
req.RemoteAddr = "192.168.1.1:12345"
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(
t, http.StatusOK, w.Code,
"GET request %d should pass", i,
)
}
assert.Equal(t, 20, callCount)
}
func TestLoginRateLimit_LimitsPOST(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
var callCount int
handler := m.LoginRateLimit()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
callCount++
w.WriteHeader(http.StatusOK)
},
))
// First loginRateLimit POST requests should succeed
for i := range middleware.LoginRateLimitConst {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login", nil,
)
req.RemoteAddr = "10.0.0.1:12345"
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(
t, http.StatusOK, w.Code,
"POST request %d should pass", i,
)
}
// Next POST should be rate-limited
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login", nil,
)
req.RemoteAddr = "10.0.0.1:12345"
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(
t, http.StatusTooManyRequests, w.Code,
"POST after limit should be 429",
)
assert.Equal(t, middleware.LoginRateLimitConst, callCount)
}
func TestLoginRateLimit_IndependentPerIP(t *testing.T) {
t.Parallel()
m, _ := testMiddleware(t, config.EnvironmentDev)
handler := m.LoginRateLimit()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
))
// Exhaust limit for IP1
for range middleware.LoginRateLimitConst {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login", nil,
)
req.RemoteAddr = "1.2.3.4:12345"
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
}
// IP1 should be rate-limited
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login", nil,
)
req.RemoteAddr = "1.2.3.4:12345"
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
assert.Equal(t, http.StatusTooManyRequests, w.Code)
// IP2 should still be allowed
req2 := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, "/pages/login", nil,
)
req2.RemoteAddr = "5.6.7.8:12345"
w2 := httptest.NewRecorder()
handler.ServeHTTP(w2, req2)
assert.Equal(
t, http.StatusOK, w2.Code,
"different IP should not be affected",
)
}
// receiverLimitedHandler builds a ReceiverRateLimit-wrapped
// handler with the given per-minute limit.
func receiverLimitedHandler(
t *testing.T, limit int,
) http.Handler {
t.Helper()
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
m := middleware.NewForTest(
log,
&config.Config{ReceiverRateLimit: limit},
nil,
)
return m.ReceiverRateLimit()(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
))
}
// receiverPost sends one POST to the handler from the given IP
// and path and returns the recorder.
func receiverPost(
handler http.Handler, ip, path string,
) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(),
http.MethodPost, path, nil,
)
req.RemoteAddr = ip
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
return w
}
func TestReceiverRateLimit_LimitsPerIPAndPath(t *testing.T) {
t.Parallel()
const limit = 3
handler := receiverLimitedHandler(t, limit)
// The first limit requests from one IP to one entrypoint
// pass.
for i := range limit {
w := receiverPost(
handler, "9.9.9.9:1234", "/webhook/uuid-a",
)
assert.Equal(
t, http.StatusOK, w.Code,
"request %d should pass", i,
)
}
// The next request over the limit is rejected with a 429
// carrying a Retry-After header.
w := receiverPost(
handler, "9.9.9.9:1234", "/webhook/uuid-a",
)
assert.Equal(t, http.StatusTooManyRequests, w.Code)
assert.NotEmpty(
t, w.Header().Get("Retry-After"),
"429 must carry a Retry-After header",
)
// The same IP is not limited on a different entrypoint.
w = receiverPost(
handler, "9.9.9.9:1234", "/webhook/uuid-b",
)
assert.Equal(
t, http.StatusOK, w.Code,
"a different entrypoint must not be affected",
)
// A different IP is not limited on the same entrypoint.
w = receiverPost(
handler, "8.8.8.8:1234", "/webhook/uuid-a",
)
assert.Equal(
t, http.StatusOK, w.Code,
"a different client IP must not be affected",
)
}

View File

@@ -0,0 +1,24 @@
package middleware
import (
"log/slog"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/session"
)
// NewForTest creates a Middleware with the minimum dependencies
// needed for testing. This bypasses the fx lifecycle.
func NewForTest(
log *slog.Logger,
cfg *config.Config,
sess *session.Session,
) *Middleware {
return &Middleware{
log: log,
params: &MiddlewareParams{
Config: cfg,
},
session: sess,
}
}

View File

@@ -1,18 +1,36 @@
package server package server
import ( import (
"errors"
"fmt" "fmt"
"net/http" "net/http"
"time" "time"
) )
const (
// httpReadTimeout is the maximum duration for reading the
// entire request, including the body.
httpReadTimeout = 10 * time.Second
// httpWriteTimeout is the maximum duration before timing out
// writes of the response. It must stay above the router's
// requestTimeout (60s, in routes.go) so the middleware timeout
// fires first and returns a clean 503, rather than the transport
// cutting the connection at the socket write deadline.
httpWriteTimeout = 65 * time.Second
// httpMaxHeaderBytes is the maximum number of bytes the
// server will read parsing the request headers.
httpMaxHeaderBytes = 1 << 20
)
func (s *Server) serveUntilShutdown() { func (s *Server) serveUntilShutdown() {
listenAddr := fmt.Sprintf(":%d", s.params.Config.Port) listenAddr := fmt.Sprintf(":%d", s.params.Config.Port)
s.httpServer = &http.Server{ s.httpServer = &http.Server{
Addr: listenAddr, Addr: listenAddr,
ReadTimeout: 10 * time.Second, ReadTimeout: httpReadTimeout,
WriteTimeout: 10 * time.Second, WriteTimeout: httpWriteTimeout,
MaxHeaderBytes: 1 << 20, MaxHeaderBytes: httpMaxHeaderBytes,
Handler: s, Handler: s,
} }
@@ -21,14 +39,21 @@ func (s *Server) serveUntilShutdown() {
s.SetupRoutes() s.SetupRoutes()
s.log.Info("http begin listen", "listenaddr", listenAddr) s.log.Info("http begin listen", "listenaddr", listenAddr)
if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
err := s.httpServer.ListenAndServe()
if err != nil && !errors.Is(err, http.ErrServerClosed) {
s.log.Error("listen error", "error", err) s.log.Error("listen error", "error", err)
if s.cancelFunc != nil { if s.cancelFunc != nil {
s.cancelFunc() s.cancelFunc()
} }
} }
} }
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { // ServeHTTP delegates to the router.
func (s *Server) ServeHTTP(
w http.ResponseWriter,
r *http.Request,
) {
s.router.ServeHTTP(w, r) s.router.ServeHTTP(w, r)
} }

View File

@@ -11,55 +11,57 @@ import (
"sneak.berlin/go/webhooker/static" "sneak.berlin/go/webhooker/static"
) )
// maxFormBodySize is the maximum allowed request body size (in
// bytes) for form POST endpoints. 1 MB is generous for any form
// submission while preventing abuse from oversized payloads.
const maxFormBodySize int64 = 1 * 1024 * 1024 // 1 MB
// requestTimeout is the maximum time allowed for a single HTTP
// request.
const requestTimeout = 60 * time.Second
// SetupRoutes configures all HTTP routes and middleware on the
// server's router.
func (s *Server) SetupRoutes() { func (s *Server) SetupRoutes() {
s.router = chi.NewRouter() s.router = chi.NewRouter()
s.setupGlobalMiddleware()
s.setupRoutes()
}
// the mux .Use() takes a http.Handler wrapper func, like most func (s *Server) setupGlobalMiddleware() {
// things that deal with "middlewares" like alice et c, and will
// call ServeHTTP on it. These middlewares applied by the mux (you
// can .Use() more than one) will be applied to every request into
// the service.
s.router.Use(middleware.Recoverer) s.router.Use(middleware.Recoverer)
s.router.Use(middleware.RequestID) s.router.Use(middleware.RequestID)
s.router.Use(s.mw.SecurityHeaders())
s.router.Use(s.mw.Logging()) s.router.Use(s.mw.Logging())
// add metrics middleware only if we can serve them behind auth // Metrics middleware (only if credentials are configured)
if s.params.Config.MetricsUsername != "" { if s.params.Config.MetricsUsername != "" {
s.router.Use(s.mw.Metrics()) s.router.Use(s.mw.Metrics())
} }
// set up CORS headers
s.router.Use(s.mw.CORS()) s.router.Use(s.mw.CORS())
s.router.Use(middleware.Timeout(requestTimeout))
// timeout for request context; your handlers must finish within // Sentry error reporting (if SENTRY_DSN is set). Repanic is
// this window: // true so panics still bubble up to the Recoverer middleware.
s.router.Use(middleware.Timeout(60 * time.Second))
// this adds a sentry reporting middleware if and only if sentry is
// enabled via setting of SENTRY_DSN in env.
if s.sentryEnabled { if s.sentryEnabled {
// Options docs at
// https://docs.sentry.io/platforms/go/guides/http/
// we set sentry to repanic so that all panics bubble up to the
// Recoverer chi middleware above.
sentryHandler := sentryhttp.New(sentryhttp.Options{ sentryHandler := sentryhttp.New(sentryhttp.Options{
Repanic: true, Repanic: true,
}) })
s.router.Use(sentryHandler.Handle) s.router.Use(sentryHandler.Handle)
} }
}
//////////////////////////////////////////////////////////////////////// func (s *Server) setupRoutes() {
// ROUTES
// complete docs: https://github.com/go-chi/chi
////////////////////////////////////////////////////////////////////////
s.router.Get("/", s.h.HandleIndex()) s.router.Get("/", s.h.HandleIndex())
s.router.Mount("/s", http.StripPrefix("/s", http.FileServer(http.FS(static.Static)))) s.router.Mount(
"/s",
http.StripPrefix("/s", http.FileServer(http.FS(static.Static))),
)
s.router.Route("/api/v1", func(_ chi.Router) { s.router.Route("/api/v1", func(_ chi.Router) {
// TODO: Add API routes here // API routes will be added here.
}) })
s.router.Get( s.router.Get(
@@ -71,42 +73,94 @@ func (s *Server) SetupRoutes() {
if s.params.Config.MetricsUsername != "" { if s.params.Config.MetricsUsername != "" {
s.router.Group(func(r chi.Router) { s.router.Group(func(r chi.Router) {
r.Use(s.mw.MetricsAuth()) r.Use(s.mw.MetricsAuth())
r.Get("/metrics", http.HandlerFunc(promhttp.Handler().ServeHTTP)) r.Get(
"/metrics",
http.HandlerFunc(
promhttp.Handler().ServeHTTP,
),
)
}) })
} }
// pages that are rendered server-side s.setupPageRoutes()
s.setupUserRoutes()
s.setupSourceRoutes()
s.setupWebhookRoutes()
}
func (s *Server) setupPageRoutes() {
s.router.Route("/pages", func(r chi.Router) { s.router.Route("/pages", func(r chi.Router) {
// Login page (no auth required) r.Use(s.mw.CSRF())
r.Use(s.mw.NoCache())
r.Use(s.mw.MaxBodySize(maxFormBodySize))
r.Group(func(r chi.Router) {
r.Use(s.mw.LoginRateLimit())
r.Get("/login", s.h.HandleLoginPage()) r.Get("/login", s.h.HandleLoginPage())
r.Post("/login", s.h.HandleLoginSubmit()) r.Post("/login", s.h.HandleLoginSubmit())
})
// Logout (auth required)
r.Post("/logout", s.h.HandleLogout()) r.Post("/logout", s.h.HandleLogout())
}) })
}
// User profile routes func (s *Server) setupUserRoutes() {
s.router.Route("/user/{username}", func(r chi.Router) { s.router.Route("/user/{username}", func(r chi.Router) {
r.Use(s.mw.CSRF())
r.Use(s.mw.NoCache())
r.Use(s.mw.RequireAuth())
r.Get("/", s.h.HandleProfile()) r.Get("/", s.h.HandleProfile())
}) })
}
// Webhook management routes (require authentication) func (s *Server) setupSourceRoutes() {
s.router.Route("/sources", func(r chi.Router) { s.router.Route("/sources", func(r chi.Router) {
r.Use(s.mw.CSRF())
r.Use(s.mw.NoCache())
r.Use(s.mw.RequireAuth()) r.Use(s.mw.RequireAuth())
r.Get("/", s.h.HandleSourceList()) // List all webhooks r.Use(s.mw.MaxBodySize(maxFormBodySize))
r.Get("/new", s.h.HandleSourceCreate()) // Show create form r.Get("/", s.h.HandleSourceList())
r.Post("/new", s.h.HandleSourceCreateSubmit()) // Handle create submission r.Get("/new", s.h.HandleSourceCreate())
r.Post("/new", s.h.HandleSourceCreateSubmit())
}) })
s.router.Route("/source/{sourceID}", func(r chi.Router) { s.router.Route("/source/{sourceID}", func(r chi.Router) {
r.Use(s.mw.CSRF())
r.Use(s.mw.NoCache())
r.Use(s.mw.RequireAuth()) r.Use(s.mw.RequireAuth())
r.Get("/", s.h.HandleSourceDetail()) // View webhook details r.Use(s.mw.MaxBodySize(maxFormBodySize))
r.Get("/edit", s.h.HandleSourceEdit()) // Show edit form r.Get("/", s.h.HandleSourceDetail())
r.Post("/edit", s.h.HandleSourceEditSubmit()) // Handle edit submission r.Get("/edit", s.h.HandleSourceEdit())
r.Post("/delete", s.h.HandleSourceDelete()) // Delete webhook r.Post("/edit", s.h.HandleSourceEditSubmit())
r.Get("/logs", s.h.HandleSourceLogs()) // View webhook logs r.Post("/delete", s.h.HandleSourceDelete())
r.Get("/logs", s.h.HandleSourceLogs())
r.Post(
"/entrypoints",
s.h.HandleEntrypointCreate(),
)
r.Post(
"/entrypoints/{entrypointID}/delete",
s.h.HandleEntrypointDelete(),
)
r.Post(
"/entrypoints/{entrypointID}/toggle",
s.h.HandleEntrypointToggle(),
)
r.Post("/targets", s.h.HandleTargetCreate())
r.Post(
"/targets/{targetID}/delete",
s.h.HandleTargetDelete(),
)
r.Post(
"/targets/{targetID}/toggle",
s.h.HandleTargetToggle(),
)
}) })
}
// Entrypoint endpoint - accepts incoming webhook POST requests
s.router.HandleFunc("/webhook/{uuid}", s.h.HandleWebhook()) func (s *Server) setupWebhookRoutes() {
s.router.With(s.mw.ReceiverRateLimit()).HandleFunc(
"/webhook/{uuid}",
s.h.HandleWebhook(),
)
} }

View File

@@ -1,3 +1,5 @@
// Package server wires up HTTP routes and manages the
// application lifecycle.
package server package server
import ( import (
@@ -21,10 +23,20 @@ import (
"github.com/go-chi/chi" "github.com/go-chi/chi"
) )
// ServerParams is a standard fx naming convention for dependency injection const (
// nolint:golint // shutdownTimeout is the maximum time to wait for the HTTP
// server to finish in-flight requests during shutdown.
shutdownTimeout = 5 * time.Second
// sentryFlushTimeout is the maximum time to wait for Sentry
// to flush pending events during shutdown.
sentryFlushTimeout = 2 * time.Second
)
//nolint:revive // ServerParams is a standard fx naming convention.
type ServerParams struct { type ServerParams struct {
fx.In fx.In
Logger *logger.Logger Logger *logger.Logger
Globals *globals.Globals Globals *globals.Globals
Config *config.Config Config *config.Config
@@ -32,12 +44,13 @@ type ServerParams struct {
Handlers *handlers.Handlers Handlers *handlers.Handlers
} }
// Server is the main HTTP server that wires up routes and manages
// graceful shutdown.
type Server struct { type Server struct {
startupTime time.Time startupTime time.Time
exitCode int exitCode int
sentryEnabled bool sentryEnabled bool
log *slog.Logger log *slog.Logger
ctx context.Context
cancelFunc context.CancelFunc cancelFunc context.CancelFunc
httpServer *http.Server httpServer *http.Server
router *chi.Mux router *chi.Mux
@@ -46,6 +59,8 @@ type Server struct {
h *handlers.Handlers h *handlers.Handlers
} }
// New creates a Server that starts the HTTP listener on fx start
// and stops it gracefully.
func New(lc fx.Lifecycle, params ServerParams) (*Server, error) { func New(lc fx.Lifecycle, params ServerParams) (*Server, error) {
s := new(Server) s := new(Server)
s.params = params s.params = params
@@ -54,19 +69,23 @@ func New(lc fx.Lifecycle, params ServerParams) (*Server, error) {
s.log = params.Logger.Get() s.log = params.Logger.Get()
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error { OnStart: func(_ context.Context) error {
s.startupTime = time.Now() s.startupTime = time.Now()
go s.Run() go s.Run()
return nil return nil
}, },
OnStop: func(ctx context.Context) error { OnStop: func(ctx context.Context) error {
s.cleanShutdown() s.cleanShutdown(ctx)
return nil return nil
}, },
}) })
return s, nil return s, nil
} }
// Run configures Sentry and starts serving HTTP requests.
func (s *Server) Run() { func (s *Server) Run() {
s.configure() s.configure()
@@ -76,6 +95,12 @@ func (s *Server) Run() {
s.serve() s.serve()
} }
// MaintenanceMode returns whether the server is in maintenance
// mode.
func (s *Server) MaintenanceMode() bool {
return s.params.Config.MaintenanceMode
}
func (s *Server) enableSentry() { func (s *Server) enableSentry() {
s.sentryEnabled = false s.sentryEnabled = false
@@ -85,68 +110,78 @@ func (s *Server) enableSentry() {
err := sentry.Init(sentry.ClientOptions{ err := sentry.Init(sentry.ClientOptions{
Dsn: s.params.Config.SentryDSN, Dsn: s.params.Config.SentryDSN,
Release: fmt.Sprintf("%s-%s", s.params.Globals.Appname, s.params.Globals.Version), Release: fmt.Sprintf(
"%s-%s",
s.params.Globals.Appname,
s.params.Globals.Version,
),
}) })
if err != nil { if err != nil {
s.log.Error("sentry init failure", "error", err) s.log.Error("sentry init failure", "error", err)
// Don't use fatal since we still want the service to run // Don't use fatal since we still want the service to run
return return
} }
s.log.Info("sentry error reporting activated") s.log.Info("sentry error reporting activated")
s.sentryEnabled = true s.sentryEnabled = true
} }
func (s *Server) serve() int { func (s *Server) serve() int {
s.ctx, s.cancelFunc = context.WithCancel(context.Background()) ctx, cancelFunc := context.WithCancel(context.Background())
s.cancelFunc = cancelFunc
// signal watcher // signal watcher
go func() { go func() {
c := make(chan os.Signal, 1) c := make(chan os.Signal, 1)
signal.Ignore(syscall.SIGPIPE) signal.Ignore(syscall.SIGPIPE)
signal.Notify(c, os.Interrupt, syscall.SIGTERM) signal.Notify(c, os.Interrupt, syscall.SIGTERM)
// block and wait for signal // block and wait for signal
sig := <-c sig := <-c
s.log.Info("signal received", "signal", sig.String()) s.log.Info("signal received", "signal", sig.String())
if s.cancelFunc != nil { if s.cancelFunc != nil {
// cancelling the main context will trigger a clean // cancelling the main context will trigger a clean
// shutdown. // shutdown via the fx OnStop hook.
s.cancelFunc() s.cancelFunc()
} }
}() }()
go s.serveUntilShutdown() go s.serveUntilShutdown()
<-s.ctx.Done() <-ctx.Done()
s.cleanShutdown() // Shutdown is handled by the fx OnStop hook (cleanShutdown).
// Do not call cleanShutdown() here to avoid double invocation.
return s.exitCode return s.exitCode
} }
func (s *Server) cleanupForExit() { func (s *Server) cleanupForExit() {
s.log.Info("cleaning up") s.log.Info("cleaning up")
// TODO: close database connections, flush buffers, etc.
} }
func (s *Server) cleanShutdown() { func (s *Server) cleanShutdown(ctx context.Context) {
// initiate clean shutdown // initiate clean shutdown
s.exitCode = 0 s.exitCode = 0
ctxShutdown, shutdownCancel := context.WithTimeout(context.Background(), 5*time.Second)
ctxShutdown, shutdownCancel := context.WithTimeout(
ctx, shutdownTimeout,
)
defer shutdownCancel() defer shutdownCancel()
if err := s.httpServer.Shutdown(ctxShutdown); err != nil { err := s.httpServer.Shutdown(ctxShutdown)
s.log.Error("server clean shutdown failed", "error", err) if err != nil {
s.log.Error(
"server clean shutdown failed", "error", err,
)
} }
s.cleanupForExit() s.cleanupForExit()
if s.sentryEnabled { if s.sentryEnabled {
sentry.Flush(2 * time.Second) sentry.Flush(sentryFlushTimeout)
} }
} }
func (s *Server) MaintenanceMode() bool {
return s.params.Config.MaintenanceMode
}
func (s *Server) configure() { func (s *Server) configure() {
// identify ourselves in the logs // identify ourselves in the logs
s.params.Logger.Identify() s.params.Logger.Identify()

View File

@@ -1,59 +1,107 @@
// Package session manages HTTP session storage and authentication
// state.
package session package session
import ( import (
"context"
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"log/slog" "log/slog"
"maps"
"net/http" "net/http"
"github.com/gorilla/sessions" "github.com/gorilla/sessions"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
const ( const (
// SessionName is the name of the session cookie // SessionName is the name of the session cookie.
SessionName = "webhooker_session" SessionName = "webhooker_session"
// UserIDKey is the session key for user ID // UserIDKey is the session key for user ID.
UserIDKey = "user_id" UserIDKey = "user_id"
// UsernameKey is the session key for username // UsernameKey is the session key for username.
UsernameKey = "username" UsernameKey = "username"
// AuthenticatedKey is the session key for authentication status // AuthenticatedKey is the session key for authentication
// status.
AuthenticatedKey = "authenticated" AuthenticatedKey = "authenticated"
// sessionKeyLength is the required length in bytes for the
// session authentication key.
sessionKeyLength = 32
// sessionMaxAgeDays is the session cookie lifetime in days.
sessionMaxAgeDays = 7
// secondsPerDay is the number of seconds in a day.
secondsPerDay = 86400
) )
// nolint:revive // SessionParams is a standard fx naming convention // ErrSessionKeyLength is returned when the decoded session key
type SessionParams struct { // does not have the expected length.
var ErrSessionKeyLength = errors.New("session key length mismatch")
// Params holds dependencies injected by fx.
type Params struct {
fx.In fx.In
Config *config.Config Config *config.Config
Database *database.Database
Logger *logger.Logger Logger *logger.Logger
} }
// Session manages encrypted session storage // Session manages encrypted session storage.
type Session struct { type Session struct {
store *sessions.CookieStore store *sessions.CookieStore
key []byte // raw 32-byte auth key, also used for CSRF cookie signing
log *slog.Logger log *slog.Logger
config *config.Config config *config.Config
} }
// New creates a new session manager // New creates a new session manager. The cookie store is
func New(lc fx.Lifecycle, params SessionParams) (*Session, error) { // initialized during the fx OnStart phase after the database is
if params.Config.SessionKey == "" { // connected, using a session key that is auto-generated and stored
return nil, fmt.Errorf("SESSION_KEY environment variable is required") // in the database.
func New(
lc fx.Lifecycle,
params Params,
) (*Session, error) {
s := &Session{
log: params.Logger.Get(),
config: params.Config,
} }
// Decode the base64 session key lc.Append(fx.Hook{
keyBytes, err := base64.StdEncoding.DecodeString(params.Config.SessionKey) OnStart: func(_ context.Context) error {
sessionKey, err := params.Database.GetOrCreateSessionKey()
if err != nil { if err != nil {
return nil, fmt.Errorf("invalid SESSION_KEY format: %w", err) return fmt.Errorf(
"failed to get session key: %w", err,
)
} }
if len(keyBytes) != 32 { keyBytes, err := base64.StdEncoding.DecodeString(
return nil, fmt.Errorf("SESSION_KEY must be 32 bytes (got %d)", len(keyBytes)) sessionKey,
)
if err != nil {
return fmt.Errorf(
"invalid session key format: %w", err,
)
}
if len(keyBytes) != sessionKeyLength {
return fmt.Errorf(
"%w: want %d, got %d",
ErrSessionKeyLength,
sessionKeyLength,
len(keyBytes),
)
} }
store := sessions.NewCookieStore(keyBytes) store := sessions.NewCookieStore(keyBytes)
@@ -61,65 +109,147 @@ func New(lc fx.Lifecycle, params SessionParams) (*Session, error) {
// Configure cookie options for security // Configure cookie options for security
store.Options = &sessions.Options{ store.Options = &sessions.Options{
Path: "/", Path: "/",
MaxAge: 86400 * 7, // 7 days MaxAge: secondsPerDay * sessionMaxAgeDays,
HttpOnly: true, HttpOnly: true,
Secure: !params.Config.IsDev(), // HTTPS in production Secure: !params.Config.IsDev(),
SameSite: http.SameSiteLaxMode, SameSite: http.SameSiteLaxMode,
} }
s := &Session{ s.key = keyBytes
store: store, s.store = store
log: params.Logger.Get(), s.log.Info("session manager initialized")
config: params.Config,
} return nil
},
})
return s, nil return s, nil
} }
// Get retrieves a session for the request // Get retrieves a session for the request.
func (s *Session) Get(r *http.Request) (*sessions.Session, error) { func (s *Session) Get(
r *http.Request,
) (*sessions.Session, error) {
return s.store.Get(r, SessionName) return s.store.Get(r, SessionName)
} }
// Save saves the session // GetKey returns the raw 32-byte authentication key used for
func (s *Session) Save(r *http.Request, w http.ResponseWriter, sess *sessions.Session) error { // session encryption. This key is also suitable for CSRF cookie
// signing.
func (s *Session) GetKey() []byte {
return s.key
}
// Save saves the session.
func (s *Session) Save(
r *http.Request,
w http.ResponseWriter,
sess *sessions.Session,
) error {
return sess.Save(r, w) return sess.Save(r, w)
} }
// SetUser sets the user information in the session // SetUser sets the user information in the session.
func (s *Session) SetUser(sess *sessions.Session, userID, username string) { func (s *Session) SetUser(
sess *sessions.Session,
userID, username string,
) {
sess.Values[UserIDKey] = userID sess.Values[UserIDKey] = userID
sess.Values[UsernameKey] = username sess.Values[UsernameKey] = username
sess.Values[AuthenticatedKey] = true sess.Values[AuthenticatedKey] = true
} }
// ClearUser removes user information from the session // ClearUser removes user information from the session.
func (s *Session) ClearUser(sess *sessions.Session) { func (s *Session) ClearUser(sess *sessions.Session) {
delete(sess.Values, UserIDKey) delete(sess.Values, UserIDKey)
delete(sess.Values, UsernameKey) delete(sess.Values, UsernameKey)
delete(sess.Values, AuthenticatedKey) delete(sess.Values, AuthenticatedKey)
} }
// IsAuthenticated checks if the session has an authenticated user // IsAuthenticated checks if the session has an authenticated
// user.
func (s *Session) IsAuthenticated(sess *sessions.Session) bool { func (s *Session) IsAuthenticated(sess *sessions.Session) bool {
auth, ok := sess.Values[AuthenticatedKey].(bool) auth, ok := sess.Values[AuthenticatedKey].(bool)
return ok && auth return ok && auth
} }
// GetUserID retrieves the user ID from the session // GetUserID retrieves the user ID from the session.
func (s *Session) GetUserID(sess *sessions.Session) (string, bool) { func (s *Session) GetUserID(
sess *sessions.Session,
) (string, bool) {
userID, ok := sess.Values[UserIDKey].(string) userID, ok := sess.Values[UserIDKey].(string)
return userID, ok return userID, ok
} }
// GetUsername retrieves the username from the session // GetUsername retrieves the username from the session.
func (s *Session) GetUsername(sess *sessions.Session) (string, bool) { func (s *Session) GetUsername(
sess *sessions.Session,
) (string, bool) {
username, ok := sess.Values[UsernameKey].(string) username, ok := sess.Values[UsernameKey].(string)
return username, ok return username, ok
} }
// Destroy invalidates the session // Destroy invalidates the session.
func (s *Session) Destroy(sess *sessions.Session) { func (s *Session) Destroy(sess *sessions.Session) {
sess.Options.MaxAge = -1 sess.Options.MaxAge = -1
s.ClearUser(sess) s.ClearUser(sess)
} }
// Regenerate creates a new session with the same values but a
// fresh ID. The old session is destroyed (MaxAge = -1) and saved,
// then a new session is created. This prevents session fixation
// attacks by ensuring the session ID changes after privilege
// escalation (e.g. login).
func (s *Session) Regenerate(
r *http.Request,
w http.ResponseWriter,
oldSess *sessions.Session,
) (*sessions.Session, error) {
// Copy the values from the old session
oldValues := make(map[any]any)
maps.Copy(oldValues, oldSess.Values)
// Destroy the old session
oldSess.Options.MaxAge = -1
s.ClearUser(oldSess)
err := oldSess.Save(r, w)
if err != nil {
return nil, fmt.Errorf(
"failed to destroy old session: %w", err,
)
}
// Create a new session (gorilla/sessions generates a new ID)
newSess, err := s.store.New(r, SessionName)
if err != nil {
// store.New may return an error alongside a new empty
// session if the old cookie is now invalid. That is
// expected after we destroyed it above. Only fail on a
// nil session.
if newSess == nil {
return nil, fmt.Errorf(
"failed to create new session: %w", err,
)
}
}
// Restore the copied values into the new session
maps.Copy(newSess.Values, oldValues)
// Apply the standard session options (the destroyed old
// session had MaxAge = -1, which store.New might inherit
// from the cookie).
newSess.Options = &sessions.Options{
Path: "/",
MaxAge: secondsPerDay * sessionMaxAgeDays,
HttpOnly: true,
Secure: !s.config.IsDev(),
SameSite: http.SameSiteLaxMode,
}
return newSess, nil
}

View File

@@ -0,0 +1,507 @@
package session_test
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"testing"
"github.com/gorilla/sessions"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/session"
)
const testKeySize = 32
// testSession creates a Session with a real cookie store for
// testing.
func testSession(t *testing.T) *session.Session {
t.Helper()
key := make([]byte, testKeySize)
for i := range key {
key[i] = byte(i + 42)
}
store := sessions.NewCookieStore(key)
store.Options = &sessions.Options{
Path: "/",
MaxAge: 86400 * 7,
HttpOnly: true,
Secure: false,
SameSite: http.SameSiteLaxMode,
}
cfg := &config.Config{
Environment: config.EnvironmentDev,
}
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
return session.NewForTest(store, cfg, log, key)
}
// --- Get and Save Tests ---
func TestGet_NewSession(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
require.NotNil(t, sess)
assert.True(
t, sess.IsNew,
"session should be new when no cookie is present",
)
}
func TestGet_ExistingSession(t *testing.T) {
t.Parallel()
s := testSession(t)
// Create and save a session
req1 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w1 := httptest.NewRecorder()
sess1, err := s.Get(req1)
require.NoError(t, err)
sess1.Values["test_key"] = "test_value"
require.NoError(t, s.Save(req1, w1, sess1))
// Extract cookies
cookies := w1.Result().Cookies()
require.NotEmpty(t, cookies)
// Make a new request with the session cookie
req2 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
for _, c := range cookies {
req2.AddCookie(c)
}
sess2, err := s.Get(req2)
require.NoError(t, err)
assert.False(
t, sess2.IsNew,
"session should not be new when cookie is present",
)
assert.Equal(t, "test_value", sess2.Values["test_key"])
}
func TestSave_SetsCookie(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w := httptest.NewRecorder()
sess, err := s.Get(req)
require.NoError(t, err)
sess.Values["key"] = "value"
err = s.Save(req, w, sess)
require.NoError(t, err)
cookies := w.Result().Cookies()
require.NotEmpty(t, cookies, "Save should set a cookie")
// Verify the cookie has the expected name
var found bool
for _, c := range cookies {
if c.Name == session.SessionName {
found = true
assert.True(
t, c.HttpOnly,
"session cookie should be HTTP-only",
)
break
}
}
assert.True(
t, found,
"should find a cookie named %s", session.SessionName,
)
}
// --- SetUser and User Retrieval Tests ---
func TestSetUser_SetsAllFields(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-abc-123", "alice")
assert.Equal(
t, "user-abc-123", sess.Values[session.UserIDKey],
)
assert.Equal(
t, "alice", sess.Values[session.UsernameKey],
)
assert.Equal(
t, true, sess.Values[session.AuthenticatedKey],
)
}
func TestGetUserID(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
// Before setting user
userID, ok := s.GetUserID(sess)
assert.False(
t, ok, "should return false when no user ID is set",
)
assert.Empty(t, userID)
// After setting user
s.SetUser(sess, "user-xyz", "bob")
userID, ok = s.GetUserID(sess)
assert.True(t, ok)
assert.Equal(t, "user-xyz", userID)
}
func TestGetUsername(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
// Before setting user
username, ok := s.GetUsername(sess)
assert.False(
t, ok, "should return false when no username is set",
)
assert.Empty(t, username)
// After setting user
s.SetUser(sess, "user-xyz", "bob")
username, ok = s.GetUsername(sess)
assert.True(t, ok)
assert.Equal(t, "bob", username)
}
// --- IsAuthenticated Tests ---
func TestIsAuthenticated_NoSession(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
assert.False(
t, s.IsAuthenticated(sess),
"new session should not be authenticated",
)
}
func TestIsAuthenticated_AfterSetUser(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
assert.True(t, s.IsAuthenticated(sess))
}
func TestIsAuthenticated_AfterClearUser(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
require.True(t, s.IsAuthenticated(sess))
s.ClearUser(sess)
assert.False(
t, s.IsAuthenticated(sess),
"should not be authenticated after ClearUser",
)
}
func TestIsAuthenticated_WrongType(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
// Set authenticated to a non-bool value
sess.Values[session.AuthenticatedKey] = "yes"
assert.False(
t, s.IsAuthenticated(sess),
"should return false for non-bool authenticated value",
)
}
// --- ClearUser Tests ---
func TestClearUser_RemovesAllKeys(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
s.ClearUser(sess)
_, hasUserID := sess.Values[session.UserIDKey]
assert.False(t, hasUserID, "UserIDKey should be removed")
_, hasUsername := sess.Values[session.UsernameKey]
assert.False(t, hasUsername, "UsernameKey should be removed")
_, hasAuth := sess.Values[session.AuthenticatedKey]
assert.False(
t, hasAuth, "AuthenticatedKey should be removed",
)
}
// --- Destroy Tests ---
func TestDestroy_InvalidatesSession(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
s.Destroy(sess)
// After Destroy: MaxAge should be -1 (delete cookie) and
// user data cleared
assert.Equal(
t, -1, sess.Options.MaxAge,
"Destroy should set MaxAge to -1",
)
assert.False(
t, s.IsAuthenticated(sess),
"should not be authenticated after Destroy",
)
_, hasUserID := sess.Values[session.UserIDKey]
assert.False(t, hasUserID, "Destroy should clear user ID")
}
// --- Session Persistence Round-Trip ---
func TestSessionPersistence_RoundTrip(t *testing.T) {
t.Parallel()
s := testSession(t)
// Step 1: Create session, set user, save
req1 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w1 := httptest.NewRecorder()
sess1, err := s.Get(req1)
require.NoError(t, err)
s.SetUser(sess1, "user-round-trip", "charlie")
require.NoError(t, s.Save(req1, w1, sess1))
cookies := w1.Result().Cookies()
require.NotEmpty(t, cookies)
// Step 2: New request with cookies -- session data should
// persist
req2 := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/profile", nil,
)
for _, c := range cookies {
req2.AddCookie(c)
}
sess2, err := s.Get(req2)
require.NoError(t, err)
assert.True(
t, s.IsAuthenticated(sess2),
"session should be authenticated after round-trip",
)
userID, ok := s.GetUserID(sess2)
assert.True(t, ok)
assert.Equal(t, "user-round-trip", userID)
username, ok := s.GetUsername(sess2)
assert.True(t, ok)
assert.Equal(t, "charlie", username)
}
// --- Constants Tests ---
func TestSessionConstants(t *testing.T) {
t.Parallel()
assert.Equal(t, "webhooker_session", session.SessionName)
assert.Equal(t, "user_id", session.UserIDKey)
assert.Equal(t, "username", session.UsernameKey)
assert.Equal(t, "authenticated", session.AuthenticatedKey)
}
// --- Edge Cases ---
func TestSetUser_OverwritesPreviousUser(t *testing.T) {
t.Parallel()
s := testSession(t)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
sess, err := s.Get(req)
require.NoError(t, err)
s.SetUser(sess, "user-1", "alice")
assert.True(t, s.IsAuthenticated(sess))
// Overwrite with a different user
s.SetUser(sess, "user-2", "bob")
userID, ok := s.GetUserID(sess)
assert.True(t, ok)
assert.Equal(t, "user-2", userID)
username, ok := s.GetUsername(sess)
assert.True(t, ok)
assert.Equal(t, "bob", username)
}
func TestDestroy_ThenSave_DeletesCookie(t *testing.T) {
t.Parallel()
s := testSession(t)
// Create a session
req1 := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
w1 := httptest.NewRecorder()
sess, err := s.Get(req1)
require.NoError(t, err)
s.SetUser(sess, "user-123", "alice")
require.NoError(t, s.Save(req1, w1, sess))
cookies := w1.Result().Cookies()
require.NotEmpty(t, cookies)
// Destroy and save
req2 := httptest.NewRequestWithContext(
context.Background(),
http.MethodGet, "/logout", nil,
)
for _, c := range cookies {
req2.AddCookie(c)
}
w2 := httptest.NewRecorder()
sess2, err := s.Get(req2)
require.NoError(t, err)
s.Destroy(sess2)
require.NoError(t, s.Save(req2, w2, sess2))
// The cookie should have MaxAge = -1 (browser should delete)
responseCookies := w2.Result().Cookies()
var sessionCookie *http.Cookie
for _, c := range responseCookies {
if c.Name == session.SessionName {
sessionCookie = c
break
}
}
require.NotNil(
t, sessionCookie,
"should have a session cookie in response",
)
assert.Negative(
t, sessionCookie.MaxAge,
"destroyed session cookie should have negative MaxAge",
)
}

View File

@@ -0,0 +1,22 @@
package session
import (
"log/slog"
"github.com/gorilla/sessions"
"sneak.berlin/go/webhooker/internal/config"
)
// NewForTest creates a Session with a pre-configured cookie store for use
// in tests. This bypasses the fx lifecycle and database dependency, allowing
// middleware and handler tests to use real session functionality. The key
// parameter is the raw 32-byte authentication key used for session encryption
// and CSRF cookie signing.
func NewForTest(store *sessions.CookieStore, cfg *config.Config, log *slog.Logger, key []byte) *Session {
return &Session{
store: store,
key: key,
config: cfg,
log: log,
}
}

View File

@@ -1 +0,0 @@

View File

@@ -1,303 +0,0 @@
# Configuration Module (Go)
A simple, clean, and generic configuration management system that supports multiple environments and automatic value resolution. This module is completely standalone and can be used in any Go project.
## Features
- **Simple API**: Just `config.Get()` and `config.GetSecret()`
- **Type-safe helpers**: `config.GetString()`, `config.GetInt()`, `config.GetBool()`
- **Environment Support**: Separate configs for different environments (dev/prod/staging/etc)
- **Value Resolution**: Automatic resolution of special values:
- `$ENV:VARIABLE` - Read from environment variable
- `$GSM:secret-name` - Read from Google Secret Manager
- `$ASM:secret-name` - Read from AWS Secrets Manager
- `$FILE:/path/to/file` - Read from file contents
- **Hierarchical Defaults**: Environment-specific values override defaults
- **YAML-based**: Easy to read and edit configuration files
- **Thread-safe**: Safe for concurrent use
- **Testable**: Uses afero filesystem abstraction for easy testing
- **Minimal Dependencies**: Only requires YAML parser and cloud SDKs (optional)
## Installation
```bash
go get git.eeqj.de/sneak/webhooker/pkg/config
```
## Usage
```go
package main
import (
"fmt"
"git.eeqj.de/sneak/webhooker/pkg/config"
)
func main() {
// Set the environment explicitly
config.SetEnvironment("prod")
// Get configuration values
baseURL := config.GetString("baseURL")
apiTimeout := config.GetInt("timeout", 30)
debugMode := config.GetBool("debugMode", false)
// Get secret values
apiKey := config.GetSecretString("api_key")
dbPassword := config.GetSecretString("db_password", "default")
// Get all values (for debugging)
allConfig := config.GetAllConfig()
allSecrets := config.GetAllSecrets()
// Reload configuration from file
if err := config.Reload(); err != nil {
fmt.Printf("Failed to reload config: %v\n", err)
}
}
```
## Configuration File Structure
Create a `config.yaml` file in your project root:
```yaml
environments:
dev:
config:
baseURL: https://dev.example.com
debugMode: true
timeout: 30
secrets:
api_key: dev-key-12345
db_password: $ENV:DEV_DB_PASSWORD
prod:
config:
baseURL: https://prod.example.com
debugMode: false
timeout: 10
GCPProject: my-project-123
AWSRegion: us-west-2
secrets:
api_key: $GSM:prod-api-key
db_password: $ASM:prod/db/password
configDefaults:
app_name: my-app
timeout: 30
log_level: INFO
port: 8080
```
## How It Works
1. **Environment Selection**: Call `config.SetEnvironment("prod")` to select which environment to use
2. **Value Lookup**: When you call `config.Get("key")`:
- First checks `environments.<env>.config.key`
- Falls back to `configDefaults.key`
- Returns the default value if not found
3. **Secret Lookup**: When you call `config.GetSecret("key")`:
- Looks in `environments.<env>.secrets.key`
- Returns the default value if not found
4. **Value Resolution**: If a value starts with a special prefix:
- `$ENV:` - Reads from environment variable
- `$GSM:` - Fetches from Google Secret Manager (requires GCPProject to be set in config)
- `$ASM:` - Fetches from AWS Secrets Manager (uses AWSRegion from config or defaults to us-east-1)
- `$FILE:` - Reads from file (supports `~` expansion)
## Type-Safe Access
The module provides type-safe helper functions:
```go
// String values
baseURL := config.GetString("baseURL", "http://localhost")
// Integer values
port := config.GetInt("port", 8080)
// Boolean values
debug := config.GetBool("debug", false)
// Secret string values
apiKey := config.GetSecretString("api_key", "default-key")
```
## Local Development
For local development, you can:
1. Use environment variables:
```yaml
secrets:
api_key: $ENV:LOCAL_API_KEY
```
2. Use local files:
```yaml
secrets:
api_key: $FILE:~/.secrets/api-key.txt
```
3. Create a `config.local.yaml` (gitignored) with literal values for testing
## Cloud Provider Support
### Google Secret Manager
To use GSM resolution (`$GSM:` prefix):
1. Set `GCPProject` in your config
2. Ensure proper authentication (e.g., `GOOGLE_APPLICATION_CREDENTIALS` environment variable)
3. The module will automatically initialize the GSM client when needed
### AWS Secrets Manager
To use ASM resolution (`$ASM:` prefix):
1. Optionally set `AWSRegion` in your config (defaults to us-east-1)
2. Ensure proper authentication (e.g., AWS credentials in environment or IAM role)
3. The module will automatically initialize the ASM client when needed
## Advanced Usage
### Loading from a Specific File
```go
// Load configuration from a specific file
if err := config.LoadFile("/path/to/config.yaml"); err != nil {
log.Fatal(err)
}
```
### Checking Configuration Values
```go
// Get all configuration for current environment
allConfig := config.GetAllConfig()
for key, value := range allConfig {
fmt.Printf("%s: %v\n", key, value)
}
// Get all secrets (be careful with logging!)
allSecrets := config.GetAllSecrets()
```
## Testing
The module uses the [afero](https://github.com/spf13/afero) filesystem abstraction, making it easy to test without real files:
```go
package myapp_test
import (
"testing"
"github.com/spf13/afero"
"git.eeqj.de/sneak/webhooker/pkg/config"
)
func TestMyApp(t *testing.T) {
// Create an in-memory filesystem for testing
fs := afero.NewMemMapFs()
// Write a test config file
testConfig := `
environments:
test:
config:
apiURL: http://test.example.com
secrets:
apiKey: test-key-123
`
afero.WriteFile(fs, "config.yaml", []byte(testConfig), 0644)
// Use the test filesystem
config.SetFs(fs)
config.SetEnvironment("test")
// Now your tests use the in-memory config
if url := config.GetString("apiURL"); url != "http://test.example.com" {
t.Errorf("Expected test URL, got %s", url)
}
}
```
### Unit Testing with Isolated Config
For unit tests, you can create isolated configuration managers:
```go
func TestMyComponent(t *testing.T) {
// Create a test-specific manager
manager := config.NewManager()
// Use in-memory filesystem
fs := afero.NewMemMapFs()
afero.WriteFile(fs, "config.yaml", []byte(testConfig), 0644)
manager.SetFs(fs)
// Test with isolated configuration
manager.SetEnvironment("test")
value := manager.Get("someKey", "default")
}
```
## Error Handling
- If a config file is not found when using the default loader, an error is returned
- If a key is not found, the default value is returned
- If a special value cannot be resolved (e.g., env var not set, file not found), `nil` is returned
- Cloud provider errors are logged but return `nil` to allow graceful degradation
## Thread Safety
All operations are thread-safe. The module uses read-write mutexes to ensure safe concurrent access to configuration data.
## Example Integration
```go
package main
import (
"log"
"os"
"git.eeqj.de/sneak/webhooker/pkg/config"
)
func main() {
// Read environment from your app-specific env var
environment := os.Getenv("APP_ENV")
if environment == "" {
environment = "dev"
}
config.SetEnvironment(environment)
// Now use configuration throughout your app
databaseURL := config.GetString("database_url")
apiKey := config.GetSecretString("api_key")
log.Printf("Running in %s environment", environment)
log.Printf("Database URL: %s", databaseURL)
}
```
## Migration from Python Version
The Go version maintains API compatibility with the Python version where possible:
| Python | Go |
|--------|-----|
| `config.get('key')` | `config.Get("key")` or `config.GetString("key")` |
| `config.getSecret('key')` | `config.GetSecret("key")` or `config.GetSecretString("key")` |
| `config.set_environment('prod')` | `config.SetEnvironment("prod")` |
| `config.reload()` | `config.Reload()` |
| `config.get_all_config()` | `config.GetAllConfig()` |
| `config.get_all_secrets()` | `config.GetAllSecrets()` |
## License
This module is designed to be standalone and can be extracted into its own repository with your preferred license.

View File

@@ -1,180 +0,0 @@
// Package config provides a simple, clean, and generic configuration management system
// that supports multiple environments and automatic value resolution.
//
// Features:
// - Simple API: Just config.Get() and config.GetSecret()
// - Environment Support: Separate configs for different environments (dev/prod/staging/etc)
// - Value Resolution: Automatic resolution of special values:
// - $ENV:VARIABLE - Read from environment variable
// - $GSM:secret-name - Read from Google Secret Manager
// - $ASM:secret-name - Read from AWS Secrets Manager
// - $FILE:/path/to/file - Read from file contents
// - Hierarchical Defaults: Environment-specific values override defaults
// - YAML-based: Easy to read and edit configuration files
// - Zero Dependencies: Only depends on yaml and cloud provider SDKs (optional)
//
// Usage:
//
// import "sneak.berlin/go/webhooker/pkg/config"
//
// // Set the environment explicitly
// config.SetEnvironment("prod")
//
// // Get configuration values
// baseURL := config.Get("baseURL")
// apiTimeout := config.GetInt("timeout", 30)
//
// // Get secret values
// apiKey := config.GetSecret("api_key")
// dbPassword := config.GetSecret("db_password", "default")
package config
import (
"sync"
"github.com/spf13/afero"
)
// Global configuration manager instance
var (
globalManager *Manager
mu sync.Mutex // Protect global manager updates
)
// getManager returns the global configuration manager, creating it if necessary
func getManager() *Manager {
mu.Lock()
defer mu.Unlock()
if globalManager == nil {
globalManager = NewManager()
}
return globalManager
}
// SetEnvironment sets the active environment.
func SetEnvironment(environment string) {
getManager().SetEnvironment(environment)
}
// SetFs sets the filesystem to use for all file operations.
// This is primarily useful for testing with an in-memory filesystem.
func SetFs(fs afero.Fs) {
mu.Lock()
defer mu.Unlock()
// Create a new manager with the specified filesystem
newManager := NewManager()
newManager.SetFs(fs)
// Replace the global manager
globalManager = newManager
}
// Get retrieves a configuration value.
//
// This looks for values in the following order:
// 1. Environment-specific config (environments.<env>.config.<key>)
// 2. Config defaults (configDefaults.<key>)
//
// Values are resolved if they contain special prefixes:
// - $ENV:VARIABLE_NAME - reads from environment variable
// - $GSM:secret-name - reads from Google Secret Manager
// - $ASM:secret-name - reads from AWS Secrets Manager
// - $FILE:/path/to/file - reads from file
func Get(key string, defaultValue ...interface{}) interface{} {
var def interface{}
if len(defaultValue) > 0 {
def = defaultValue[0]
}
return getManager().Get(key, def)
}
// GetString retrieves a configuration value as a string.
func GetString(key string, defaultValue ...string) string {
var def string
if len(defaultValue) > 0 {
def = defaultValue[0]
}
val := Get(key, def)
if s, ok := val.(string); ok {
return s
}
return def
}
// GetInt retrieves a configuration value as an integer.
func GetInt(key string, defaultValue ...int) int {
var def int
if len(defaultValue) > 0 {
def = defaultValue[0]
}
val := Get(key, def)
switch v := val.(type) {
case int:
return v
case int64:
return int(v)
case float64:
return int(v)
default:
return def
}
}
// GetBool retrieves a configuration value as a boolean.
func GetBool(key string, defaultValue ...bool) bool {
var def bool
if len(defaultValue) > 0 {
def = defaultValue[0]
}
val := Get(key, def)
if b, ok := val.(bool); ok {
return b
}
return def
}
// GetSecret retrieves a secret value.
//
// This looks for secrets defined in environments.<env>.secrets.<key>
func GetSecret(key string, defaultValue ...interface{}) interface{} {
var def interface{}
if len(defaultValue) > 0 {
def = defaultValue[0]
}
return getManager().GetSecret(key, def)
}
// GetSecretString retrieves a secret value as a string.
func GetSecretString(key string, defaultValue ...string) string {
var def string
if len(defaultValue) > 0 {
def = defaultValue[0]
}
val := GetSecret(key, def)
if s, ok := val.(string); ok {
return s
}
return def
}
// Reload reloads the configuration from file.
func Reload() error {
return getManager().Reload()
}
// GetAllConfig returns all configuration values for the current environment.
func GetAllConfig() map[string]interface{} {
return getManager().GetAllConfig()
}
// GetAllSecrets returns all secrets for the current environment.
func GetAllSecrets() map[string]interface{} {
return getManager().GetAllSecrets()
}
// LoadFile loads configuration from a specific file.
func LoadFile(configFile string) error {
return getManager().LoadFile(configFile)
}

View File

@@ -1,306 +0,0 @@
package config
import (
"os"
"testing"
"github.com/spf13/afero"
)
func TestNewManager(t *testing.T) {
manager := NewManager()
if manager == nil {
t.Fatal("NewManager returned nil")
}
if manager.config == nil {
t.Error("Manager config map is nil")
}
if manager.loader == nil {
t.Error("Manager loader is nil")
}
if manager.resolvedCache == nil {
t.Error("Manager resolvedCache is nil")
}
if manager.fs == nil {
t.Error("Manager fs is nil")
}
}
func TestLoader_FindConfigFile(t *testing.T) {
// Create an in-memory filesystem for testing
fs := afero.NewMemMapFs()
loader := NewLoader(fs)
// Create a config file in the filesystem
configContent := `
environments:
test:
config:
testKey: testValue
secrets:
testSecret: secretValue
configDefaults:
defaultKey: defaultValue
`
// Create the file in the current directory
if err := afero.WriteFile(fs, "config.yaml", []byte(configContent), 0644); err != nil {
t.Fatalf("Failed to write test config: %v", err)
}
// Test finding the config file
foundPath, err := loader.FindConfigFile("config.yaml")
if err != nil {
t.Errorf("FindConfigFile failed: %v", err)
}
// In memory fs, the path should be exactly what we created
if foundPath != "config.yaml" {
t.Errorf("Expected config.yaml, got %s", foundPath)
}
}
func TestLoader_LoadYAML(t *testing.T) {
fs := afero.NewMemMapFs()
loader := NewLoader(fs)
// Create a test config file
testConfig := `
environments:
test:
config:
testKey: testValue
configDefaults:
defaultKey: defaultValue
`
if err := afero.WriteFile(fs, "test-config.yaml", []byte(testConfig), 0644); err != nil {
t.Fatalf("Failed to write test config: %v", err)
}
// Load the YAML
config, err := loader.LoadYAML("test-config.yaml")
if err != nil {
t.Fatalf("LoadYAML failed: %v", err)
}
// Verify the structure
envs, ok := config["environments"].(map[string]interface{})
if !ok {
t.Fatal("environments not found or wrong type")
}
testEnv, ok := envs["test"].(map[string]interface{})
if !ok {
t.Fatal("test environment not found")
}
testConfig2, ok := testEnv["config"].(map[string]interface{})
if !ok {
t.Fatal("test config not found")
}
if testConfig2["testKey"] != "testValue" {
t.Errorf("Expected testKey=testValue, got %v", testConfig2["testKey"])
}
}
func TestResolver_ResolveEnv(t *testing.T) {
fs := afero.NewMemMapFs()
resolver := NewResolver("", "", fs)
// Set a test environment variable
os.Setenv("TEST_CONFIG_VAR", "test-value")
defer os.Unsetenv("TEST_CONFIG_VAR")
// Test resolving environment variable
result := resolver.Resolve("$ENV:TEST_CONFIG_VAR")
if result != "test-value" {
t.Errorf("Expected 'test-value', got %v", result)
}
// Test non-existent env var
result = resolver.Resolve("$ENV:NON_EXISTENT_VAR")
if result != nil {
t.Errorf("Expected nil for non-existent env var, got %v", result)
}
}
func TestResolver_ResolveFile(t *testing.T) {
fs := afero.NewMemMapFs()
resolver := NewResolver("", "", fs)
// Create a test file
secretContent := "my-secret-value"
if err := afero.WriteFile(fs, "/test-secret.txt", []byte(secretContent+"\n"), 0644); err != nil {
t.Fatalf("Failed to write test file: %v", err)
}
// Test resolving file
result := resolver.Resolve("$FILE:/test-secret.txt")
if result != secretContent {
t.Errorf("Expected '%s', got %v", secretContent, result)
}
// Test non-existent file
result = resolver.Resolve("$FILE:/non/existent/file")
if result != nil {
t.Errorf("Expected nil for non-existent file, got %v", result)
}
}
func TestManager_GetAndSet(t *testing.T) {
// Create an in-memory filesystem
fs := afero.NewMemMapFs()
// Create a test config file
testConfig := `
environments:
dev:
config:
apiURL: http://dev.example.com
timeout: 30
debug: true
secrets:
apiKey: dev-key-123
prod:
config:
apiURL: https://prod.example.com
timeout: 10
debug: false
secrets:
apiKey: $ENV:PROD_API_KEY
configDefaults:
appName: TestApp
timeout: 20
port: 8080
`
if err := afero.WriteFile(fs, "config.yaml", []byte(testConfig), 0644); err != nil {
t.Fatalf("Failed to write test config: %v", err)
}
// Create manager and set the filesystem
manager := NewManager()
manager.SetFs(fs)
// Load config should find the file automatically
manager.SetEnvironment("dev")
// Test getting config values
if v := manager.Get("apiURL", ""); v != "http://dev.example.com" {
t.Errorf("Expected dev apiURL, got %v", v)
}
if v := manager.Get("timeout", 0); v != 30 {
t.Errorf("Expected timeout=30, got %v", v)
}
if v := manager.Get("debug", false); v != true {
t.Errorf("Expected debug=true, got %v", v)
}
// Test default values
if v := manager.Get("appName", ""); v != "TestApp" {
t.Errorf("Expected appName from defaults, got %v", v)
}
// Test getting secrets
if v := manager.GetSecret("apiKey", ""); v != "dev-key-123" {
t.Errorf("Expected dev apiKey, got %v", v)
}
// Switch to prod environment
manager.SetEnvironment("prod")
if v := manager.Get("apiURL", ""); v != "https://prod.example.com" {
t.Errorf("Expected prod apiURL, got %v", v)
}
// Test environment variable resolution in secrets
os.Setenv("PROD_API_KEY", "prod-key-456")
defer os.Unsetenv("PROD_API_KEY")
if v := manager.GetSecret("apiKey", ""); v != "prod-key-456" {
t.Errorf("Expected resolved env var for apiKey, got %v", v)
}
}
func TestGlobalAPI(t *testing.T) {
// Create an in-memory filesystem
fs := afero.NewMemMapFs()
// Create a test config file
testConfig := `
environments:
test:
config:
stringVal: hello
intVal: 42
boolVal: true
secrets:
secret1: test-secret
configDefaults:
defaultString: world
`
if err := afero.WriteFile(fs, "config.yaml", []byte(testConfig), 0644); err != nil {
t.Fatalf("Failed to write test config: %v", err)
}
// Use the global API with the test filesystem
SetFs(fs)
SetEnvironment("test")
// Test type-safe getters
if v := GetString("stringVal"); v != "hello" {
t.Errorf("Expected 'hello', got %v", v)
}
if v := GetInt("intVal"); v != 42 {
t.Errorf("Expected 42, got %v", v)
}
if v := GetBool("boolVal"); v != true {
t.Errorf("Expected true, got %v", v)
}
if v := GetSecretString("secret1"); v != "test-secret" {
t.Errorf("Expected 'test-secret', got %v", v)
}
// Test defaults
if v := GetString("defaultString"); v != "world" {
t.Errorf("Expected 'world', got %v", v)
}
}
func TestManager_SetFs(t *testing.T) {
// Create manager with default OS filesystem
manager := NewManager()
// Create an in-memory filesystem
memFs := afero.NewMemMapFs()
// Write a config file to the memory fs
testConfig := `
environments:
test:
config:
testKey: fromMemory
configDefaults:
defaultKey: memoryDefault
`
if err := afero.WriteFile(memFs, "config.yaml", []byte(testConfig), 0644); err != nil {
t.Fatalf("Failed to write test config: %v", err)
}
// Set the filesystem
manager.SetFs(memFs)
manager.SetEnvironment("test")
// Test that it reads from the memory filesystem
if v := manager.Get("testKey", ""); v != "fromMemory" {
t.Errorf("Expected 'fromMemory', got %v", v)
}
if v := manager.Get("defaultKey", ""); v != "memoryDefault" {
t.Errorf("Expected 'memoryDefault', got %v", v)
}
}

View File

@@ -1,146 +0,0 @@
package config_test
import (
"fmt"
"testing"
"github.com/spf13/afero"
"sneak.berlin/go/webhooker/pkg/config"
)
// ExampleSetFs demonstrates how to use an in-memory filesystem for testing
func ExampleSetFs() {
// Create an in-memory filesystem
fs := afero.NewMemMapFs()
// Create a test configuration file
configYAML := `
environments:
test:
config:
baseURL: https://test.example.com
debugMode: true
secrets:
apiKey: test-key-12345
production:
config:
baseURL: https://api.example.com
debugMode: false
configDefaults:
appName: Test Application
timeout: 30
`
// Write the config to the in-memory filesystem
if err := afero.WriteFile(fs, "config.yaml", []byte(configYAML), 0644); err != nil {
panic(err)
}
// Use the in-memory filesystem
config.SetFs(fs)
config.SetEnvironment("test")
// Now all config operations use the in-memory filesystem
fmt.Printf("Base URL: %s\n", config.GetString("baseURL"))
fmt.Printf("Debug Mode: %v\n", config.GetBool("debugMode"))
fmt.Printf("App Name: %s\n", config.GetString("appName"))
// Output:
// Base URL: https://test.example.com
// Debug Mode: true
// App Name: Test Application
}
// TestWithAferoFilesystem shows how to test with different filesystem implementations
func TestWithAferoFilesystem(t *testing.T) {
tests := []struct {
name string
setupFs func() afero.Fs
environment string
key string
expected string
}{
{
name: "in-memory filesystem",
setupFs: func() afero.Fs {
fs := afero.NewMemMapFs()
config := `
environments:
dev:
config:
apiURL: http://localhost:8080
`
afero.WriteFile(fs, "config.yaml", []byte(config), 0644)
return fs
},
environment: "dev",
key: "apiURL",
expected: "http://localhost:8080",
},
{
name: "readonly filesystem",
setupFs: func() afero.Fs {
memFs := afero.NewMemMapFs()
config := `
environments:
staging:
config:
apiURL: https://staging.example.com
`
afero.WriteFile(memFs, "config.yaml", []byte(config), 0644)
// Wrap in a read-only filesystem
return afero.NewReadOnlyFs(memFs)
},
environment: "staging",
key: "apiURL",
expected: "https://staging.example.com",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Create a new manager for each test to ensure isolation
manager := config.NewManager()
manager.SetFs(tt.setupFs())
manager.SetEnvironment(tt.environment)
result := manager.Get(tt.key, "")
if result != tt.expected {
t.Errorf("Expected %s, got %v", tt.expected, result)
}
})
}
}
// TestFileResolution shows how $FILE: resolution works with afero
func TestFileResolution(t *testing.T) {
// Create an in-memory filesystem
fs := afero.NewMemMapFs()
// Create a secret file
secretContent := "super-secret-api-key"
if err := afero.WriteFile(fs, "/secrets/api-key.txt", []byte(secretContent), 0600); err != nil {
t.Fatal(err)
}
// Create a config that references the file
configYAML := `
environments:
prod:
secrets:
apiKey: $FILE:/secrets/api-key.txt
`
if err := afero.WriteFile(fs, "config.yaml", []byte(configYAML), 0644); err != nil {
t.Fatal(err)
}
// Use the filesystem
config.SetFs(fs)
config.SetEnvironment("prod")
// Get the secret - it should resolve from the file
apiKey := config.GetSecretString("apiKey")
if apiKey != secretContent {
t.Errorf("Expected %s, got %s", secretContent, apiKey)
}
}

View File

@@ -1,139 +0,0 @@
package config_test
import (
"fmt"
"log"
"os"
"sneak.berlin/go/webhooker/pkg/config"
)
func Example() {
// Set the environment explicitly
config.SetEnvironment("dev")
// Get configuration values
baseURL := config.GetString("baseURL")
timeout := config.GetInt("timeout", 30)
debugMode := config.GetBool("debugMode", false)
fmt.Printf("Base URL: %s\n", baseURL)
fmt.Printf("Timeout: %d\n", timeout)
fmt.Printf("Debug Mode: %v\n", debugMode)
// Get secret values
apiKey := config.GetSecretString("api_key")
if apiKey != "" {
fmt.Printf("API Key: %s...\n", apiKey[:8])
}
}
func ExampleSetEnvironment() {
// Your application determines which environment to use
// This could come from command line args, env vars, etc.
environment := os.Getenv("APP_ENV")
if environment == "" {
environment = "development"
}
// Set the environment explicitly
config.SetEnvironment(environment)
// Now use configuration throughout your application
fmt.Printf("Environment: %s\n", environment)
fmt.Printf("App Name: %s\n", config.GetString("app_name"))
}
func ExampleGetString() {
config.SetEnvironment("prod")
// Get a string configuration value with a default
baseURL := config.GetString("baseURL", "http://localhost:8080")
fmt.Printf("Base URL: %s\n", baseURL)
}
func ExampleGetInt() {
config.SetEnvironment("prod")
// Get an integer configuration value with a default
port := config.GetInt("port", 8080)
fmt.Printf("Port: %d\n", port)
}
func ExampleGetBool() {
config.SetEnvironment("dev")
// Get a boolean configuration value with a default
debugMode := config.GetBool("debugMode", false)
fmt.Printf("Debug Mode: %v\n", debugMode)
}
func ExampleGetSecretString() {
config.SetEnvironment("prod")
// Get a secret string value
apiKey := config.GetSecretString("api_key")
if apiKey != "" {
// Be careful not to log the full secret!
fmt.Printf("API Key configured: yes\n")
}
}
func ExampleLoadFile() {
// Load configuration from a specific file
if err := config.LoadFile("/path/to/config.yaml"); err != nil {
log.Printf("Failed to load config: %v", err)
return
}
config.SetEnvironment("staging")
fmt.Printf("Loaded configuration from custom file\n")
}
func ExampleReload() {
config.SetEnvironment("dev")
// Get initial value
oldValue := config.GetString("some_key")
// ... config file might have been updated ...
// Reload configuration from file
if err := config.Reload(); err != nil {
log.Printf("Failed to reload config: %v", err)
return
}
// Get potentially updated value
newValue := config.GetString("some_key")
fmt.Printf("Value changed: %v\n", oldValue != newValue)
}
// Example config.yaml structure:
/*
environments:
development:
config:
baseURL: http://localhost:8000
debugMode: true
port: 8000
secrets:
api_key: dev-key-12345
production:
config:
baseURL: https://api.example.com
debugMode: false
port: 443
GCPProject: my-project-123
AWSRegion: us-west-2
secrets:
api_key: $GSM:prod-api-key
db_password: $ASM:prod/db/password
configDefaults:
app_name: My Application
timeout: 30
log_level: INFO
port: 8080
*/

View File

@@ -1,41 +0,0 @@
module sneak.berlin/go/webhooker/pkg/config
go 1.23.0
toolchain go1.24.1
require (
github.com/aws/aws-sdk-go v1.50.0
github.com/spf13/afero v1.14.0
gopkg.in/yaml.v3 v3.0.1
)
require (
cloud.google.com/go/compute v1.23.1 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/iam v1.1.3 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/crypto v0.14.0 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/oauth2 v0.13.0 // indirect
golang.org/x/sync v0.12.0 // indirect
golang.org/x/sys v0.13.0 // indirect
golang.org/x/text v0.23.0 // indirect
google.golang.org/api v0.149.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20231016165738-49dd2c1f3d0b // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231016165738-49dd2c1f3d0b // indirect
google.golang.org/grpc v1.59.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect
)
require (
cloud.google.com/go/secretmanager v1.11.4
github.com/jmespath/go-jmespath v0.4.0 // indirect
)

View File

@@ -1,161 +0,0 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.110.8 h1:tyNdfIxjzaWctIiLYOTalaLKZ17SI44SKFW26QbOhME=
cloud.google.com/go v0.110.8/go.mod h1:Iz8AkXJf1qmxC3Oxoep8R1T36w8B92yU29PcBhHO5fk=
cloud.google.com/go/compute v1.23.1 h1:V97tBoDaZHb6leicZ1G6DLK2BAaZLJ/7+9BB/En3hR0=
cloud.google.com/go/compute v1.23.1/go.mod h1:CqB3xpmPKKt3OJpW2ndFIXnA9A4xAy/F3Xp1ixncW78=
cloud.google.com/go/compute/metadata v0.2.3 h1:mg4jlk7mCAj6xXp9UJ4fjI9VUI5rubuGBW5aJ7UnBMY=
cloud.google.com/go/compute/metadata v0.2.3/go.mod h1:VAV5nSsACxMJvgaAuX6Pk2AawlZn8kiOGuCv6gTkwuA=
cloud.google.com/go/iam v1.1.3 h1:18tKG7DzydKWUnLjonWcJO6wjSCAtzh4GcRKlH/Hrzc=
cloud.google.com/go/iam v1.1.3/go.mod h1:3khUlaBXfPKKe7huYgEpDn6FtgRyMEqbkvBxrQyY5SE=
cloud.google.com/go/secretmanager v1.11.4 h1:krnX9qpG2kR2fJ+u+uNyNo+ACVhplIAS4Pu7u+4gd+k=
cloud.google.com/go/secretmanager v1.11.4/go.mod h1:wreJlbS9Zdq21lMzWmJ0XhWW2ZxgPeahsqeV/vZoJ3w=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/aws/aws-sdk-go v1.50.0 h1:HBtrLeO+QyDKnc3t1+5DR1RxodOHCGr8ZcrHudpv7jI=
github.com/aws/aws-sdk-go v1.50.0/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/s2a-go v0.1.7 h1:60BLSyTrOV4/haCDW4zb1guZItoSq8foHCXrAnjBo/o=
github.com/google/s2a-go v0.1.7/go.mod h1:50CgR4k1jNlWBu4UfS4AcfhVe1r6pdZPygJ3R8F0Qdw=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.2 h1:Vie5ybvEvT75RniqhfFxPRy3Bf7vr3h0cechB90XaQs=
github.com/googleapis/enterprise-certificate-proxy v0.3.2/go.mod h1:VLSiSSBs/ksPL8kq3OBOQ6WRI2QnaFynd1DCjZ62+V0=
github.com/googleapis/gax-go/v2 v2.12.0 h1:A+gCJKdRfqXkr+BIRGtZLibNXf0m1f9E4HG56etFpas=
github.com/googleapis/gax-go/v2 v2.12.0/go.mod h1:y+aIqrI5eb1YGMVJfuV3185Ts/D7qKpsEkdD5+I6QGU=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/spf13/afero v1.14.0 h1:9tH6MapGnn/j0eb0yIXiLjERO8RB6xIVZRDCX7PtqWA=
github.com/spf13/afero v1.14.0/go.mod h1:acJQ8t0ohCGuMN3O+Pv0V0hgMxNYDlvdk+VTfyZmbYo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc=
golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.13.0 h1:jDDenyj+WgFtmV3zYVoi8aE2BwtXFLWOA67ZfNWftiY=
golang.org/x/oauth2 v0.13.0/go.mod h1:/JMhi4ZRXAf4HG9LiNmxvk+45+96RUlVThiH8FzNBn0=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.12.0 h1:MHc5BpPuC30uJk597Ri8TV3CNZcTLu6B6z4lJy+g6Jw=
golang.org/x/sync v0.12.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY=
golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.149.0 h1:b2CqT6kG+zqJIVKRQ3ELJVLN1PwHZ6DJ3dW8yl82rgY=
google.golang.org/api v0.149.0/go.mod h1:Mwn1B7JTXrzXtnvmzQE2BD6bYZQ8DShKZDZbeN9I7qI=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b h1:+YaDE2r2OG8t/z5qmsh7Y+XXwCbvadxxZ0YY6mTdrVA=
google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b/go.mod h1:CgAqfJo+Xmu0GwA0411Ht3OU3OntXwsGmrmjI8ioGXI=
google.golang.org/genproto/googleapis/api v0.0.0-20231016165738-49dd2c1f3d0b h1:CIC2YMXmIhYw6evmhPxBKJ4fmLbOFtXQN/GV3XOZR8k=
google.golang.org/genproto/googleapis/api v0.0.0-20231016165738-49dd2c1f3d0b/go.mod h1:IBQ646DjkDkvUIsVq/cc03FUFQ9wbZu7yE396YcL870=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231016165738-49dd2c1f3d0b h1:ZlWIi1wSK56/8hn4QcBp/j9M7Gt3U/3hZw3mC7vDICo=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231016165738-49dd2c1f3d0b/go.mod h1:swOH3j0KzcDDgGUWr+SNpyTen5YrXjS3eyPzFYKc6lc=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.59.0 h1:Z5Iec2pjwb+LEOqzpB2MR12/eKFhDPhuqW91O+4bwUk=
google.golang.org/grpc v1.59.0/go.mod h1:aUPDwccQo6OTjy7Hct4AfBPD1GptF4fyUjIkQ9YtF98=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

View File

@@ -1,104 +0,0 @@
package config
import (
"fmt"
"path/filepath"
"github.com/spf13/afero"
"gopkg.in/yaml.v3"
)
// Loader handles loading configuration from YAML files.
type Loader struct {
fs afero.Fs
}
// NewLoader creates a new configuration loader.
func NewLoader(fs afero.Fs) *Loader {
return &Loader{
fs: fs,
}
}
// FindConfigFile searches for a configuration file by looking up the directory tree.
func (l *Loader) FindConfigFile(filename string) (string, error) {
if filename == "" {
filename = "config.yaml"
}
// First check if the file exists in the current directory (simple case)
if _, err := l.fs.Stat(filename); err == nil {
return filename, nil
}
// For more complex cases, try to walk up the directory tree
// Start from current directory or root for in-memory filesystems
currentDir := "."
// Try to get the absolute path, but if it fails (e.g., in-memory fs),
// just use the current directory
if absPath, err := filepath.Abs("."); err == nil {
currentDir = absPath
}
// Search up the directory tree
for {
configPath := filepath.Join(currentDir, filename)
if _, err := l.fs.Stat(configPath); err == nil {
return configPath, nil
}
// Move up one directory
parentDir := filepath.Dir(currentDir)
if parentDir == currentDir || currentDir == "." || currentDir == "/" {
// Reached the root directory or can't go up further
break
}
currentDir = parentDir
}
return "", fmt.Errorf("configuration file %s not found in directory tree", filename)
}
// LoadYAML loads a YAML file and returns the parsed configuration.
func (l *Loader) LoadYAML(filePath string) (map[string]interface{}, error) {
data, err := afero.ReadFile(l.fs, filePath)
if err != nil {
return nil, fmt.Errorf("failed to read file %s: %w", filePath, err)
}
var config map[string]interface{}
if err := yaml.Unmarshal(data, &config); err != nil {
return nil, fmt.Errorf("failed to parse YAML from %s: %w", filePath, err)
}
if config == nil {
config = make(map[string]interface{})
}
return config, nil
}
// MergeConfigs performs a deep merge of two configuration maps.
// The override map values take precedence over the base map.
func (l *Loader) MergeConfigs(base, override map[string]interface{}) map[string]interface{} {
if base == nil {
base = make(map[string]interface{})
}
for key, value := range override {
if baseValue, exists := base[key]; exists {
// If both values are maps, merge them recursively
if baseMap, baseOk := baseValue.(map[string]interface{}); baseOk {
if overrideMap, overrideOk := value.(map[string]interface{}); overrideOk {
base[key] = l.MergeConfigs(baseMap, overrideMap)
continue
}
}
}
// Otherwise, override the value
base[key] = value
}
return base
}

View File

@@ -1,373 +0,0 @@
package config
import (
"fmt"
"log"
"strings"
"sync"
"github.com/spf13/afero"
)
// Manager manages application configuration with value resolution.
type Manager struct {
mu sync.RWMutex
config map[string]interface{}
environment string
resolver *Resolver
loader *Loader
configFile string
resolvedCache map[string]interface{}
fs afero.Fs
}
// NewManager creates a new configuration manager.
func NewManager() *Manager {
fs := afero.NewOsFs()
return &Manager{
config: make(map[string]interface{}),
loader: NewLoader(fs),
resolvedCache: make(map[string]interface{}),
fs: fs,
}
}
// SetFs sets the filesystem to use for all file operations.
// This is primarily useful for testing with an in-memory filesystem.
func (m *Manager) SetFs(fs afero.Fs) {
m.mu.Lock()
defer m.mu.Unlock()
m.fs = fs
m.loader = NewLoader(fs)
// If we have a resolver, recreate it with the new fs
if m.resolver != nil {
gcpProject := ""
awsRegion := "us-east-1"
// Try to get the current settings
if gcpProj := m.getConfigValue("GCPProject", ""); gcpProj != nil {
if str, ok := gcpProj.(string); ok {
gcpProject = str
}
}
if awsReg := m.getConfigValue("AWSRegion", "us-east-1"); awsReg != nil {
if str, ok := awsReg.(string); ok {
awsRegion = str
}
}
m.resolver = NewResolver(gcpProject, awsRegion, fs)
}
// Clear caches as filesystem changed
m.resolvedCache = make(map[string]interface{})
}
// LoadFile loads configuration from a specific file.
func (m *Manager) LoadFile(configFile string) error {
m.mu.Lock()
defer m.mu.Unlock()
config, err := m.loader.LoadYAML(configFile)
if err != nil {
return err
}
m.config = config
m.configFile = configFile
m.resolvedCache = make(map[string]interface{}) // Clear cache
return nil
}
// loadConfig loads the configuration from file.
func (m *Manager) loadConfig() error {
if m.configFile == "" {
// Try to find config.yaml
configPath, err := m.loader.FindConfigFile("config.yaml")
if err != nil {
return err
}
m.configFile = configPath
}
config, err := m.loader.LoadYAML(m.configFile)
if err != nil {
return err
}
m.config = config
m.resolvedCache = make(map[string]interface{}) // Clear cache
return nil
}
// SetEnvironment sets the active environment.
func (m *Manager) SetEnvironment(environment string) {
m.mu.Lock()
defer m.mu.Unlock()
m.environment = strings.ToLower(environment)
// Create resolver with GCP project and AWS region if available
gcpProject := m.getConfigValue("GCPProject", "")
awsRegion := m.getConfigValue("AWSRegion", "us-east-1")
if gcpProjectStr, ok := gcpProject.(string); ok {
if awsRegionStr, ok := awsRegion.(string); ok {
m.resolver = NewResolver(gcpProjectStr, awsRegionStr, m.fs)
}
}
// Clear resolved cache when environment changes
m.resolvedCache = make(map[string]interface{})
}
// Get retrieves a configuration value.
func (m *Manager) Get(key string, defaultValue interface{}) interface{} {
m.mu.RLock()
// Ensure config is loaded
if m.config == nil || len(m.config) == 0 {
// Need to upgrade to write lock to load config
m.mu.RUnlock()
m.mu.Lock()
// Double-check after acquiring write lock
if m.config == nil || len(m.config) == 0 {
if err := m.loadConfig(); err != nil {
log.Printf("Failed to load config: %v", err)
m.mu.Unlock()
return defaultValue
}
}
// Downgrade back to read lock
m.mu.Unlock()
m.mu.RLock()
}
defer m.mu.RUnlock()
// Check cache first
cacheKey := fmt.Sprintf("config.%s", key)
if cached, ok := m.resolvedCache[cacheKey]; ok {
return cached
}
// Try environment-specific config first
var rawValue interface{}
if m.environment != "" {
envMap, ok := m.config["environments"].(map[string]interface{})
if ok {
if env, ok := envMap[m.environment].(map[string]interface{}); ok {
if config, ok := env["config"].(map[string]interface{}); ok {
if val, exists := config[key]; exists {
rawValue = val
}
}
}
}
}
// Fall back to configDefaults
if rawValue == nil {
if defaults, ok := m.config["configDefaults"].(map[string]interface{}); ok {
if val, exists := defaults[key]; exists {
rawValue = val
}
}
}
if rawValue == nil {
return defaultValue
}
// Resolve the value if we have a resolver
var resolvedValue interface{}
if m.resolver != nil {
resolvedValue = m.resolver.Resolve(rawValue)
} else {
resolvedValue = rawValue
}
// Cache the resolved value
m.resolvedCache[cacheKey] = resolvedValue
return resolvedValue
}
// GetSecret retrieves a secret value for the current environment.
func (m *Manager) GetSecret(key string, defaultValue interface{}) interface{} {
m.mu.RLock()
// Ensure config is loaded
if m.config == nil || len(m.config) == 0 {
// Need to upgrade to write lock to load config
m.mu.RUnlock()
m.mu.Lock()
// Double-check after acquiring write lock
if m.config == nil || len(m.config) == 0 {
if err := m.loadConfig(); err != nil {
log.Printf("Failed to load config: %v", err)
m.mu.Unlock()
return defaultValue
}
}
// Downgrade back to read lock
m.mu.Unlock()
m.mu.RLock()
}
defer m.mu.RUnlock()
if m.environment == "" {
log.Printf("No environment set when getting secret '%s'", key)
return defaultValue
}
// Get the current environment's config
envMap, ok := m.config["environments"].(map[string]interface{})
if !ok {
return defaultValue
}
env, ok := envMap[m.environment].(map[string]interface{})
if !ok {
return defaultValue
}
secrets, ok := env["secrets"].(map[string]interface{})
if !ok {
return defaultValue
}
secretValue, exists := secrets[key]
if !exists {
return defaultValue
}
// Resolve the value
if m.resolver != nil {
resolved := m.resolver.Resolve(secretValue)
if resolved == nil {
return defaultValue
}
return resolved
}
return secretValue
}
// getConfigValue is an internal helper to get config values without locking.
func (m *Manager) getConfigValue(key string, defaultValue interface{}) interface{} {
// Try environment-specific config first
var rawValue interface{}
if m.environment != "" {
envMap, ok := m.config["environments"].(map[string]interface{})
if ok {
if env, ok := envMap[m.environment].(map[string]interface{}); ok {
if config, ok := env["config"].(map[string]interface{}); ok {
if val, exists := config[key]; exists {
rawValue = val
}
}
}
}
}
// Fall back to configDefaults
if rawValue == nil {
if defaults, ok := m.config["configDefaults"].(map[string]interface{}); ok {
if val, exists := defaults[key]; exists {
rawValue = val
}
}
}
if rawValue == nil {
return defaultValue
}
return rawValue
}
// Reload reloads the configuration from file.
func (m *Manager) Reload() error {
m.mu.Lock()
defer m.mu.Unlock()
return m.loadConfig()
}
// GetAllConfig returns all configuration values for the current environment.
func (m *Manager) GetAllConfig() map[string]interface{} {
m.mu.RLock()
defer m.mu.RUnlock()
result := make(map[string]interface{})
// Start with configDefaults
if defaults, ok := m.config["configDefaults"].(map[string]interface{}); ok {
for k, v := range defaults {
if m.resolver != nil {
result[k] = m.resolver.Resolve(v)
} else {
result[k] = v
}
}
}
// Override with environment-specific config
if m.environment != "" {
envMap, ok := m.config["environments"].(map[string]interface{})
if ok {
if env, ok := envMap[m.environment].(map[string]interface{}); ok {
if config, ok := env["config"].(map[string]interface{}); ok {
for k, v := range config {
if m.resolver != nil {
result[k] = m.resolver.Resolve(v)
} else {
result[k] = v
}
}
}
}
}
}
return result
}
// GetAllSecrets returns all secrets for the current environment.
func (m *Manager) GetAllSecrets() map[string]interface{} {
m.mu.RLock()
defer m.mu.RUnlock()
if m.environment == "" {
return make(map[string]interface{})
}
envMap, ok := m.config["environments"].(map[string]interface{})
if !ok {
return make(map[string]interface{})
}
env, ok := envMap[m.environment].(map[string]interface{})
if !ok {
return make(map[string]interface{})
}
secrets, ok := env["secrets"].(map[string]interface{})
if !ok {
return make(map[string]interface{})
}
// Resolve all secrets
result := make(map[string]interface{})
for k, v := range secrets {
if m.resolver != nil {
result[k] = m.resolver.Resolve(v)
} else {
result[k] = v
}
}
return result
}

View File

@@ -1,204 +0,0 @@
package config
import (
"context"
"fmt"
"log"
"os"
"path/filepath"
"regexp"
"strings"
secretmanager "cloud.google.com/go/secretmanager/apiv1"
"cloud.google.com/go/secretmanager/apiv1/secretmanagerpb"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/secretsmanager"
"github.com/spf13/afero"
)
// Resolver handles resolution of configuration values with special prefixes.
type Resolver struct {
gcpProject string
awsRegion string
gsmClient *secretmanager.Client
asmClient *secretsmanager.SecretsManager
awsSession *session.Session
specialValue *regexp.Regexp
fs afero.Fs
}
// NewResolver creates a new value resolver.
func NewResolver(gcpProject, awsRegion string, fs afero.Fs) *Resolver {
return &Resolver{
gcpProject: gcpProject,
awsRegion: awsRegion,
specialValue: regexp.MustCompile(`^\$([A-Z]+):(.+)$`),
fs: fs,
}
}
// Resolve resolves a configuration value that may contain special prefixes.
func (r *Resolver) Resolve(value interface{}) interface{} {
switch v := value.(type) {
case string:
return r.resolveString(v)
case map[string]interface{}:
// Recursively resolve map values
result := make(map[string]interface{})
for k, val := range v {
result[k] = r.Resolve(val)
}
return result
case []interface{}:
// Recursively resolve slice items
result := make([]interface{}, len(v))
for i, val := range v {
result[i] = r.Resolve(val)
}
return result
default:
// Return non-string values as-is
return value
}
}
// resolveString resolves a string value that may contain a special prefix.
func (r *Resolver) resolveString(value string) interface{} {
matches := r.specialValue.FindStringSubmatch(value)
if matches == nil {
return value
}
resolverType := matches[1]
resolverValue := matches[2]
switch resolverType {
case "ENV":
return r.resolveEnv(resolverValue)
case "GSM":
return r.resolveGSM(resolverValue)
case "ASM":
return r.resolveASM(resolverValue)
case "FILE":
return r.resolveFile(resolverValue)
default:
log.Printf("Unknown resolver type: %s", resolverType)
return value
}
}
// resolveEnv resolves an environment variable.
func (r *Resolver) resolveEnv(envVar string) interface{} {
value := os.Getenv(envVar)
if value == "" {
return nil
}
return value
}
// resolveGSM resolves a Google Secret Manager secret.
func (r *Resolver) resolveGSM(secretName string) interface{} {
if r.gcpProject == "" {
log.Printf("GCP project not configured for GSM resolution")
return nil
}
// Initialize GSM client if needed
if r.gsmClient == nil {
ctx := context.Background()
client, err := secretmanager.NewClient(ctx)
if err != nil {
log.Printf("Failed to create GSM client: %v", err)
return nil
}
r.gsmClient = client
}
// Build the resource name
name := fmt.Sprintf("projects/%s/secrets/%s/versions/latest", r.gcpProject, secretName)
// Access the secret
ctx := context.Background()
req := &secretmanagerpb.AccessSecretVersionRequest{
Name: name,
}
result, err := r.gsmClient.AccessSecretVersion(ctx, req)
if err != nil {
log.Printf("Failed to access GSM secret %s: %v", secretName, err)
return nil
}
return string(result.Payload.Data)
}
// resolveASM resolves an AWS Secrets Manager secret.
func (r *Resolver) resolveASM(secretName string) interface{} {
// Initialize AWS session if needed
if r.awsSession == nil {
sess, err := session.NewSession(&aws.Config{
Region: aws.String(r.awsRegion),
})
if err != nil {
log.Printf("Failed to create AWS session: %v", err)
return nil
}
r.awsSession = sess
}
// Initialize ASM client if needed
if r.asmClient == nil {
r.asmClient = secretsmanager.New(r.awsSession)
}
// Get the secret value
input := &secretsmanager.GetSecretValueInput{
SecretId: aws.String(secretName),
}
result, err := r.asmClient.GetSecretValue(input)
if err != nil {
log.Printf("Failed to access ASM secret %s: %v", secretName, err)
return nil
}
// Return the secret string
if result.SecretString != nil {
return *result.SecretString
}
// If it's binary data, we can't handle it as a string config value
log.Printf("ASM secret %s contains binary data, which is not supported", secretName)
return nil
}
// resolveFile resolves a file's contents.
func (r *Resolver) resolveFile(filePath string) interface{} {
// Expand user home directory if present
if strings.HasPrefix(filePath, "~/") {
home, err := os.UserHomeDir()
if err != nil {
log.Printf("Failed to get user home directory: %v", err)
return nil
}
filePath = filepath.Join(home, filePath[2:])
}
data, err := afero.ReadFile(r.fs, filePath)
if err != nil {
log.Printf("Failed to read file %s: %v", filePath, err)
return nil
}
// Strip whitespace/newlines from file contents
return strings.TrimSpace(string(data))
}
// Close closes any open clients.
func (r *Resolver) Close() error {
if r.gsmClient != nil {
return r.gsmClient.Close()
}
return nil
}

121
script/bootstrap Executable file
View File

@@ -0,0 +1,121 @@
#!/bin/sh
# script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes NOTHING is present (not git,
# make, or go). golangci-lint is packaged in nix, brew, and apk; on apt
# it is installed from a hash-verified GitHub release archive (never
# curl | sh).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-07. Never "latest"; exact versions only.
GOLANGCI_LINT_VERSION="2.11.3"
# sha256 of golangci-lint-2.11.3-linux-<arch>.tar.gz release archives
GOLANGCI_LINT_SHA256_AMD64="87bb8cddbcc825d5778b64e8a91b46c0526b247f4e2f2904dea74ec7450475d1"
GOLANGCI_LINT_SHA256_ARM64="ee3d95f301359e7d578e6d99c8ad5aeadbabc5a13009a30b2b0df11c8058afe9"
PKGMGR=""
SUDO=""
detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0
if command -v nix-env >/dev/null 2>&1; then
PKGMGR="nix"
elif command -v apt-get >/dev/null 2>&1; then
PKGMGR="apt"
elif command -v brew >/dev/null 2>&1; then
PKGMGR="brew"
elif command -v apk >/dev/null 2>&1; then
PKGMGR="apk"
else
echo "bootstrap: no supported package manager (nix, apt, brew, apk)" >&2
exit 1
fi
if [ "$PKGMGR" = "apt" ]; then
export DEBIAN_FRONTEND=noninteractive
if [ "$(id -u)" != "0" ]; then
SUDO="sudo"
fi
fi
}
# pkg_install <nix-attr> <apt-pkg> <brew-formula> <apk-pkg>
pkg_install() {
detect_pkgmgr
case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;;
apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;;
brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;;
esac
}
missing() {
! command -v "$1" >/dev/null 2>&1
}
# verify_sha256 <file> <expected-hash>
verify_sha256() {
if command -v sha256sum >/dev/null 2>&1; then
actual="$(sha256sum "$1" | cut -d' ' -f1)"
else
actual="$(shasum -a 256 "$1" | cut -d' ' -f1)"
fi
if [ "$actual" != "$2" ]; then
echo "bootstrap: sha256 mismatch for $1" >&2
echo " expected: $2" >&2
echo " actual: $actual" >&2
exit 1
fi
}
# apt has no golangci-lint package: install a pinned release archive
# from GitHub, verified by hardcoded sha256 (never curl | sh).
install_golangci_lint_release() {
case "$(uname -m)" in
x86_64) goarch="amd64"; sha="$GOLANGCI_LINT_SHA256_AMD64" ;;
aarch64|arm64) goarch="arm64"; sha="$GOLANGCI_LINT_SHA256_ARM64" ;;
*)
echo "bootstrap: unsupported architecture $(uname -m)" >&2
exit 1
;;
esac
if missing curl; then pkg_install curl curl curl curl; fi
name="golangci-lint-${GOLANGCI_LINT_VERSION}-linux-${goarch}"
tmp="$(mktemp -d)"
curl -fsSL -o "$tmp/$name.tar.gz" \
"https://github.com/golangci/golangci-lint/releases/download/v${GOLANGCI_LINT_VERSION}/${name}.tar.gz"
verify_sha256 "$tmp/$name.tar.gz" "$sha"
tar -xzf "$tmp/$name.tar.gz" -C "$tmp"
$SUDO install -m 0755 "$tmp/$name/golangci-lint" /usr/local/bin/golangci-lint
rm -rf "$tmp"
}
ensure_golangci_lint() {
if ! missing golangci-lint; then return 0; fi
detect_pkgmgr
case "$PKGMGR" in
apt) install_golangci_lint_release ;;
*) pkg_install golangci-lint golangci-lint golangci-lint golangci-lint ;;
esac
}
main() {
cd "$ROOT"
# Base tooling
if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi
# Go toolchain and linter
if missing go; then pkg_install go golang go go; fi
ensure_golangci_lint
go mod download
echo "bootstrap complete"
}
main "$@"

15
script/check Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/sh
# script/check: run all checks (test, lint, fmt-check). Our own
# extension to scripts-to-rule-them-all. Must not modify any files.
# Generic: usually needs no adaptation.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
main() {
"$SCRIPT_DIR/test"
"$SCRIPT_DIR/lint"
"$SCRIPT_DIR/fmt-check"
}
main "$@"

15
script/cibuild Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/sh
# script/cibuild: run the CI build. The Dockerfile runs the checks
# (make fmt-check, lint, test), so a successful build implies a green
# repo. Generic: needs no adaptation. The Gitea workflow runs this on
# push.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
docker build .
}
main "$@"

15
script/docker Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/sh
# script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname.
# Generic: needs no adaptation.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build -t "$("$SCRIPT_DIR/projectname")" .
}
main "$@"

15
script/fmt Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/sh
# script/fmt: format all files (writes).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
gofmt -s -w .
if command -v goimports >/dev/null 2>&1; then
goimports -w .
fi
}
main "$@"

17
script/fmt-check Executable file
View File

@@ -0,0 +1,17 @@
#!/bin/sh
# script/fmt-check: check formatting (read-only). Same scope as
# script/fmt, but fails instead of writing.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
if [ -n "$(gofmt -s -l .)" ]; then
echo "gofmt needed on:"
gofmt -s -l .
exit 1
fi
}
main "$@"

16
script/install-precommit Executable file
View File

@@ -0,0 +1,16 @@
#!/bin/sh
# script/install-precommit: install the git pre-commit hook that runs
# script/precommit. Our own extension to scripts-to-rule-them-all.
# Generic: needs no adaptation.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
printf '#!/bin/sh\nset -e\nscript/precommit\n' > .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
echo "pre-commit hook installed: runs script/precommit"
}
main "$@"

12
script/lint Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/sh
# script/lint: run the linter.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
golangci-lint run --config .golangci.yml ./...
}
main "$@"

21
script/precommit Executable file
View File

@@ -0,0 +1,21 @@
#!/bin/sh
# script/precommit: run by the git pre-commit hook; fails the commit if
# checks fail. Our own extension to scripts-to-rule-them-all. Go repo
# extras: go mod tidy must not change go.mod/go.sum.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
go mod tidy
git diff --exit-code -- go.mod go.sum || {
echo "precommit: go mod tidy changed go.mod/go.sum;" \
"stage the changes and retry" >&2
exit 1
}
"$SCRIPT_DIR/check"
}
main "$@"

12
script/projectname Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/sh
# script/projectname: output the name of this project. Our own
# extension to scripts-to-rule-them-all. Other scripts that need the
# name (e.g. script/docker) call this, so they can stay identical
# across all repos.
set -eu
main() {
echo "webhooker"
}
main "$@"

14
script/setup Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/sh
# script/setup: set up the repo for development after a fresh clone:
# installs dependencies (script/bootstrap) and the git pre-commit hook.
# Add any repo-specific initialization (db init, .env template) here.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
main() {
"$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/install-precommit"
}
main "$@"

Some files were not shown because too many files have changed in this diff Show More