webhooker/README.md

# webhooker

webhooker is a self-hosted webhook proxy and store-and-forward service
written in [Go](https://golang.org) by
[@sneak](https://sneak.berlin). It receives webhooks from external
services, durably stores them, and delivers them to configured targets
with retry support, logging, and observability. Category: infrastructure
/ web service. License: MIT.

## Getting Started

### Prerequisites

- Go 1.26+
- golangci-lint v2.11+
- Docker (for containerized deployment)

### Quick Start

```bash
# Clone the repo
git clone https://git.eeqj.de/sneak/webhooker.git
cd webhooker

# Install Go dependencies
make deps

# Run all checks (format, lint, test, build)
make check

# Run in development mode (uses SQLite in current directory)
make dev

# Build Docker image
make docker
```

### Development Commands

```bash
make bootstrap  # Install all dependencies (idempotent)
make setup      # Bootstrap + install git pre-commit hook
make fmt        # Format code (gofmt + goimports)
make lint       # Run golangci-lint
make test       # Run tests with race detection
make check      # test + lint + fmt-check (CI gate)
make build      # Build binary to bin/webhooker
make dev        # go run ./cmd/webhooker
make docker     # Build Docker image
make hooks      # Install git pre-commit hook that runs script/precommit
```

### Configuration

All configuration is via environment variables. For local development,
you can place variables in a `.env` file in the project root (loaded
automatically via `godotenv/autoload`).

The environment is selected by setting `WEBHOOKER_ENVIRONMENT` to `dev`
or `prod` (default: `dev`). The setting controls several behaviors:

| Behavior              | `dev`                            | `prod`                          |
| --------------------- | -------------------------------- | ------------------------------- |
| CORS                  | Allows any origin (`*`)          | Disabled (no-op)                |
| Session cookie Secure | `false` (works over plain HTTP)  | `true` (requires HTTPS)         |

The CSRF cookie's `Secure` flag and Origin/Referer validation mode are
determined per-request based on the actual transport protocol, not the
environment setting. The middleware checks `r.TLS` (direct TLS) and the
`X-Forwarded-Proto` header (TLS-terminating reverse proxy) to decide:

- **Direct TLS or `X-Forwarded-Proto: https`**: Secure cookies, strict
  Origin/Referer validation.
- **Plaintext HTTP**: Non-Secure cookies, relaxed Origin/Referer
  checks (token validation still enforced).

This means CSRF protection works correctly in all deployment scenarios:
behind a TLS-terminating reverse proxy, with direct TLS, or over plain
HTTP during development. When running behind a reverse proxy, ensure it
sets the `X-Forwarded-Proto: https` header.

All other differences (log format, security headers, etc.) are
independent of the environment setting — log format is determined by
TTY detection, and security headers are always applied.

| Variable                | Description                         | Default  |
| ----------------------- | ----------------------------------- | -------- |
| `WEBHOOKER_ENVIRONMENT` | `dev` or `prod`                     | `dev`    |
| `PORT`                  | HTTP listen port                    | `8080`   |
| `DATA_DIR`              | Directory for all SQLite databases  | `/var/lib/webhooker` |
| `DEBUG`                 | Enable debug logging                | `false`  |
| `MAINTENANCE_MODE`      | Serve the maintenance page          | `false`  |
| `METRICS_USERNAME`      | Basic auth username for `/metrics`  | `""`     |
| `METRICS_PASSWORD`      | Basic auth password for `/metrics`  | `""`     |
| `SENTRY_DSN`            | Sentry error reporting DSN          | `""`     |
| `SESSION_IDLE_TIMEOUT`  | Idle session timeout (Go duration)  | `24h`    |
| `RECEIVER_RATE_LIMIT`   | Receiver requests/minute per IP per entrypoint | `120` |
| `TRUSTED_PROXIES`       | CIDRs whose forwarded headers are trusted | `""` (none) |

#### Trusted proxies

`TRUSTED_PROXIES` is a comma-separated list of CIDR blocks (a bare
address such as `192.168.1.7` is accepted and treated as a single
host), for example `192.168.1.7, 2001:db8::5`. It decides whose
`X-Forwarded-For` header the rate limiters believe, so it should name
the addresses of your reverse proxies and nothing else.

`X-Forwarded-For` is honoured **only** when the connecting peer is
inside one of these blocks; for every other peer the client identity is
the connection's own address and the header is ignored. The default is
the empty list, which trusts nobody — anything else would let any
client pick its own rate limit bucket, minting a fresh one per request
or draining someone else's. Set it to the address of your reverse
proxy, and to nothing wider. A set but unparseable value aborts
startup.

`X-Real-IP` and `True-Client-IP` are **never** read, from any peer.
Reverse proxies append to `X-Forwarded-For` but forward other client
headers verbatim, so a single-valued header is client-controlled even
behind a trusted proxy.

Within a trusted request, `X-Forwarded-For` is read right to left,
because the rightmost entry is the one the nearest proxy appended and
everything left of it may have been written by the client. The first
hop that is not itself a trusted proxy is taken as the client. A hop
that is not a bare IP address — `ip:port`, a bracketed IPv6 literal,
the token `unknown` — ends the walk and the peer address is used
instead, since past such an entry the chain is not the shape assumed
here. The peer address is likewise used when the header is absent or
every hop in it is a trusted proxy.

Two operator requirements follow:

- Your proxy must **append** the peer address to `X-Forwarded-For`
  (nginx `$proxy_add_x_forwarded_for`, HAProxy `option forwardfor`,
  Caddy and AWS ALB by default), and must append a bare address with
  no port.
- List proxy hosts **only**. Any address inside `TRUSTED_PROXIES`
  chooses its own rate-limit key: its `X-Forwarded-For` is walked, so
  it can name a different address on every request to get a fresh
  bucket each time, or name another client's address to drain that
  client's bucket. Never list a block that also covers clients — a
  broad `10.0.0.0/8` on a network where clients live in the same range
  makes all three limits, including the unauthenticated webhook
  receiver, silently bypassable by every client in the block.

Sessions are bounded by two independent clocks, and end at whichever
one runs out first:

- **Idle expiry** (`SESSION_IDLE_TIMEOUT`, default `24h`) is a sliding
  window. Every authenticated request pushes it forward, so a session
  in continuous use never hits it, while an abandoned one expires a day
  after its last use. Any non-positive value (`0`, or a negative
  duration such as `-1s`) disables idle expiry entirely; the absolute
  cap below still applies. A set-but-unparseable value aborts startup
  rather than silently falling back to the default.
- **Absolute expiry** is a fixed 7 days from login. Activity does
  **not** extend it: after a week, every session ends and the user
  authenticates again.

Only requests that authenticate with the session count as activity, so
an unauthenticated request carrying the cookie cannot keep a session
alive. The idle timestamp is rewritten at most once per tenth of the
idle window rather than on every request, which means a session may
expire up to 10% early relative to the user's true last request, but
never late.

Both clocks are anchored by timestamps stored in the session cookie.
Sessions issued before this feature existed carry neither, so they are
treated as expired: upgrading to a build that has it logs every
existing session out once, and those users sign in again.

#### Invalid values abort startup

The defaults above apply **only** to variables that are unset (or set
to an empty string). A variable that is set but cannot be parsed is a
fatal configuration error: webhooker logs the offending variable and
its value and refuses to start, rather than silently running with a
substituted default. `PORT=eighty`, `DEBUG=ture`, and
`RETENTION_SWEEP_INTERVAL=1 hour` all abort startup. `PORT` must
additionally be a number in the range 1–65535,
`RECEIVER_RATE_LIMIT` must be at least 1, and every entry in
`TRUSTED_PROXIES` must be a CIDR block or a bare IP address.

Boolean variables (`DEBUG`, `MAINTENANCE_MODE`) accept exactly the
spellings Go's `strconv.ParseBool` accepts — `1`, `t`, `T`, `TRUE`,
`true`, `True`, `0`, `f`, `F`, `FALSE`, `false`, `False` — and nothing
else. `yes`, `on`, and `off` are rejected rather than quietly treated
as false.

On first startup, webhooker automatically generates a cryptographically
secure session encryption key and stores it in the database. This key
persists across restarts — no manual key management is needed.

On first startup, webhooker creates an `admin` user
with a randomly generated password and logs it to stdout. This password
is only displayed once.

### Running with Docker

```bash
docker run -d \
  -p 8080:8080 \
  -v /path/to/data:/var/lib/webhooker \
  -e WEBHOOKER_ENVIRONMENT=prod \
  webhooker:latest
```

The container runs as a non-root user (`webhooker`, UID 1000), exposes
port 8080, and includes a health check against
`/.well-known/healthcheck`. The `/var/lib/webhooker` volume holds all
SQLite databases: the main application database (`webhooker.db`) and
the per-webhook event databases (`events-{uuid}.db`). Mount this as a
persistent volume to preserve data across container restarts.

## Entrypoints

This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call
them. We provide:

- `script/bootstrap` — install all dependencies (idempotent)
- `script/setup` — make a fresh clone ready for development
  (bootstrap, then install-precommit)
- `script/projectname` — output the project name ("webhooker")
- `script/test` — run the test suite
- `script/lint` — run golangci-lint
- `script/fmt` — format all code (writes)
- `script/fmt-check` — check formatting (read-only)
- `script/check` — run test, lint, and fmt-check
- `script/docker` — build the Docker image tagged via `script/projectname`
- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile
  runs the checks, so a green build implies a green repo)
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
  `script/check`)
- `script/install-precommit` — install the git pre-commit hook that
  runs `script/precommit`

## Rationale

Webhook integrations between services are inherently fragile. The
receiving service must be online when the webhook fires, most webhook
senders provide no built-in retry mechanism, and there is no standard
way to inspect what was sent, when it was sent, or whether delivery
succeeded.

webhooker solves this by acting as a durable intermediary:

1. **Reliable ingestion** — webhooker is always ready to accept incoming
   webhooks. It stores every received event before attempting any
   delivery, so nothing is lost if downstream targets are unavailable.

2. **Guaranteed delivery** — Events are queued for delivery to each
   configured target. Failed deliveries are retried with configurable
   backoff. Every delivery attempt is logged with status codes, response
   bodies, and timing.

3. **Observability** — Full request/response logging for every webhook
   received and every delivery attempted. Prometheus metrics expose
   volume, latency, and error rates. The web UI provides real-time
   visibility into event flow.

4. **Fan-out** — A single incoming webhook can be delivered to multiple
   targets simultaneously. This enables patterns like forwarding a
   GitHub webhook to both a deployment service and a Slack channel.

5. **Replay** — Stored events can be manually redelivered for debugging
   or testing, without requiring the original sender to fire the webhook
   again.

### Use Cases

- **Store-and-forward** with configurable retries for unreliable
  receivers
- **Observability** via Prometheus metrics on webhook frequency, payload
  size, and delivery performance
- **Debugging** and introspection of webhook payloads in the web UI
- **Replay** of webhook events for application testing and development
- **Fan-out** delivery of a single webhook to multiple downstream
  targets
- **High-availability ingestion** for delivery to less reliable backend
  systems

## Design

### Architecture Overview

webhooker is structured as a standard Go HTTP server following the
[sneak/prompts GO_HTTP_SERVER_CONVENTIONS](https://git.eeqj.de/sneak/prompts/src/branch/main/prompts/GO_HTTP_SERVER_CONVENTIONS.md).
It uses:

- **[Uber fx](https://go.uber.org/fx)** for dependency injection and
  lifecycle management
- **[go-chi](https://github.com/go-chi/chi)** for HTTP routing
- **[GORM](https://gorm.io)** for database access with
  **[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite)** as
  the runtime SQLite driver. Note: `gorm.io/driver/sqlite` transitively
  depends on `mattn/go-sqlite3`, which requires CGO at build time (see
  [Docker](#docker) section)
- **[slog](https://pkg.go.dev/log/slog)** (stdlib) for structured
  logging with TTY detection (text for dev, JSON for prod)
- **[gorilla/sessions](https://github.com/gorilla/sessions)** for
  encrypted cookie-based session management
- **[gorilla/csrf](https://github.com/gorilla/csrf)** for CSRF
  protection (cookie-based double-submit tokens)
- **[go-chi/httprate](https://github.com/go-chi/httprate)** for
  per-IP login rate limiting (sliding window counter)
- **[Prometheus](https://prometheus.io)** for metrics, served at
  `/metrics` behind basic auth
- **[Sentry](https://sentry.io)** for optional error reporting

### Naming Conventions

The codebase uses consistent naming throughout (rename completed in
[issue #12](https://git.eeqj.de/sneak/webhooker/issues/12)):

| Entity           | Description |
| ---------------- | ----------- |
| **Webhook**      | Top-level configuration entity grouping entrypoints and targets |
| **Entrypoint**   | A receiver URL where external services POST events |
| **Target**       | A delivery destination for events |

### Data Model

webhooker's data model has eight entities organized into two tiers: the
**application tier** (user and webhook configuration) and the **event
tier** (event ingestion, delivery, and logging).

```
┌─────────────────────────────────────────────────────────────┐
│                    APPLICATION TIER                          │
│                  (main application database)                 │
│                                                             │
│  ┌──────────┐       ┌──────────┐       ┌──────────────┐     │
│  │   User   │──1:N──│  Webhook │──1:N──│  Entrypoint  │     │
│  │          │       │          │       │              │     │
│  │          │       │          │──1:N──│   Target     │     │
│  │          │       └──────────┘       └──────────────┘     │
│  │          │──1:N──│  APIKey  │                             │
│  └──────────┘       └──────────┘                             │
│                                                             │
│  ┌──────────┐                                               │
│  │ Setting  │  (key-value application config)               │
│  └──────────┘                                               │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│                      EVENT TIER                              │
│            (per-webhook dedicated databases)                 │
│                                                             │
│  ┌──────────┐       ┌──────────┐       ┌─────────────────┐  │
│  │  Event   │──1:N──│ Delivery │──1:N──│ DeliveryResult  │  │
│  └──────────┘       └──────────┘       └─────────────────┘  │
└─────────────────────────────────────────────────────────────┘
```

#### Setting

A key-value pair for application-level configuration that is
auto-managed rather than user-provided. Used to store the session
encryption key and any future auto-generated settings.

| Field   | Type   | Description |
| ------- | ------ | ----------- |
| `key`   | string | Primary key (setting name) |
| `value` | text   | Setting value |

Currently stored settings:

- **`session_key`** — Base64-encoded 32-byte session encryption key,
  auto-generated on first startup.

#### User

A registered user of the webhooker service.

| Field      | Type     | Description |
| ---------- | -------- | ----------- |
| `id`       | UUID     | Primary key |
| `username` | string   | Unique login name |
| `password` | string   | Argon2id hash (never exposed via API) |

**Relations:** Has many Webhooks. Has many APIKeys.

Passwords are hashed with Argon2id using secure defaults (64 MB memory,
1 iteration, 4 threads, 32-byte key, 16-byte salt). On first startup,
an `admin` user is created with a randomly generated 16-character
password logged to stdout.

#### Webhook

The top-level configuration entity. A webhook groups together one or
more entrypoints (receiver URLs) and one or more targets (delivery
destinations) into a logical unit. A user creates a webhook to set up
event routing.

| Field            | Type    | Description |
| ---------------- | ------- | ----------- |
| `id`             | UUID    | Primary key |
| `user_id`        | UUID    | Foreign key → User |
| `name`           | string  | Human-readable name |
| `description`    | string  | Optional description |
| `retention_days` | integer | Days to retain events (default: 30; 0 means retain forever) |

**Relations:** Belongs to User. Has many Entrypoints. Has many Targets.

The `retention_days` field controls how long event data is kept in the
webhook's dedicated database before automatic cleanup.

Setting `retention_days` to `0` means "retain events forever". Because
the column carries a default of 30, a literal zero cannot survive an
insert, so a zero is rewritten on save to a sentinel of `365 * 1000`
days (`database.RetentionForeverDays`). The retention reaper recognises
that sentinel and skips the webhook entirely, and the web UI displays
such a webhook's retention as "forever" rather than as a day count.

A *finite* retention is capped at `database.MaxFiniteRetentionDays`
(106751 days, about 292 years), and a larger one is rejected with a
400. The cap is not arbitrary: the reaper computes its cutoff as a
`time.Duration`, an int64 nanosecond count, and a longer period
overflows it. An overflowed cutoff lands in the future, where it
matches every row, so the sweep would delete every event the webhook
has instead of none. The reaper also clamps the value it is given, so a
row written by an older version cannot trigger that either.

#### Entrypoint

A receiver URL where external services POST webhook events. Each
entrypoint has a unique UUID-based path.
When an HTTP request arrives at an entrypoint's path, webhooker captures
the full request and creates an Event.

| Field          | Type    | Description |
| -------------- | ------- | ----------- |
| `id`           | UUID    | Primary key |
| `webhook_id`   | UUID    | Foreign key → Webhook |
| `path`         | string  | Unique URL path (UUID-based, e.g. `/webhook/{uuid}`) |
| `description`  | string  | Optional description |
| `active`       | boolean | Whether this entrypoint accepts events (default: true) |

**Relations:** Belongs to Webhook.

A webhook can have multiple entrypoints. This allows separate URLs for
different event sources that all feed into the same processing pipeline
(e.g., one entrypoint for GitHub, another for Stripe, both routing to
the same targets).

#### Target

A delivery destination for events. Each target defines where and how
events should be forwarded.

| Field            | Type       | Description |
| ---------------- | ---------- | ----------- |
| `id`             | UUID       | Primary key |
| `webhook_id`     | UUID       | Foreign key → Webhook |
| `name`           | string     | Human-readable name |
| `type`           | TargetType | One of: `http`, `slack`, `database`, `log` |
| `active`         | boolean    | Whether deliveries are enabled (default: true) |
| `config`         | JSON text  | Type-specific configuration |
| `max_retries`    | integer    | Maximum retry attempts for HTTP targets (0 = fire-and-forget, >0 = retries with backoff) |
| `max_queue_size` | integer    | Maximum queued deliveries (for HTTP targets with retries) |

**Relations:** Belongs to Webhook. Has many Deliveries.

**Target types:**

- **`http`** — Forward the event as an HTTP POST to a configured URL.
  Behavior depends on `max_retries`: when `max_retries` is 0 (the
  default), the target operates in fire-and-forget mode — a single
  attempt with no retries and no circuit breaker. When `max_retries` is
  greater than 0, failed deliveries are retried with exponential backoff
  up to `max_retries` attempts, protected by a per-target circuit
  breaker.
- **`database`** — Archive the full event as a row into a separate
  per-webhook archive database (`archive-{webhookID}.db`) for long-term
  retention, with an optional creation-validated expiry (default: keep
  forever). No external delivery and no retries; an archive write
  failure fails the delivery. See the database target section under
  "Per-Webhook Event Databases" for the full semantics.
- **`log`** — Write the event to the application log (stdout). Useful
  for debugging.

The `config` field stores type-specific configuration as JSON (e.g.,
destination URL, custom headers, timeout settings).

#### APIKey

A programmatic access credential for API authentication.

| Field          | Type      | Description |
| -------------- | --------- | ----------- |
| `id`           | UUID      | Primary key |
| `user_id`      | UUID      | Foreign key → User |
| `key`          | string    | Unique API key value |
| `description`  | string    | Optional description |
| `last_used_at` | timestamp | Last time this key was used (nullable) |

**Relations:** Belongs to User.

#### Event

A captured incoming webhook request. Stores the complete HTTP request
data for replay and auditing.

| Field          | Type   | Description |
| -------------- | ------ | ----------- |
| `id`             | UUID   | Primary key |
| `webhook_id`     | UUID   | Foreign key → Webhook |
| `entrypoint_id`  | UUID   | Foreign key → Entrypoint |
| `method`       | string | HTTP method (POST, PUT, etc.) |
| `headers`      | JSON   | Complete request headers |
| `body`         | text   | Raw request body |
| `content_type` | string | Content-Type header value |

**Relations:** Belongs to Webhook. Belongs to Entrypoint. Has many
Deliveries.

When a request arrives at an entrypoint, the full request (method,
headers, body) is captured as an Event. The event is then queued for
delivery to every active target configured on the parent webhook.

#### Delivery

The pairing of an event with a target. Tracks the overall delivery
status across potentially multiple attempts.

| Field      | Type           | Description |
| ---------- | -------------- | ----------- |
| `id`       | UUID           | Primary key |
| `event_id` | UUID           | Foreign key → Event |
| `target_id`| UUID           | Foreign key → Target |
| `status`   | DeliveryStatus | One of: `pending`, `delivered`, `failed`, `retrying` |

**Relations:** Belongs to Event. Belongs to Target. Has many
DeliveryResults.

**Delivery statuses:**

- **`pending`** — Created but not yet attempted.
- **`retrying`** — At least one attempt failed; more attempts remain.
- **`delivered`** — Successfully delivered (at least one attempt
  succeeded).
- **`failed`** — All retry attempts exhausted without success.

#### DeliveryResult

The result of a single delivery attempt. Every attempt (including
retries) is individually logged for full observability.

| Field           | Type    | Description |
| --------------- | ------- | ----------- |
| `id`            | UUID    | Primary key |
| `delivery_id`   | UUID    | Foreign key → Delivery |
| `attempt_num`   | integer | Attempt number (1-based) |
| `success`       | boolean | Whether this attempt succeeded |
| `status_code`   | integer | HTTP response status code (if applicable) |
| `response_body` | text    | Response body (if applicable) |
| `error`         | string  | Error message (on failure) |
| `duration`      | integer | Request duration in milliseconds |

**Relations:** Belongs to Delivery.

#### Common Fields

All entities include these fields from `BaseModel`:

| Field        | Type      | Description |
| ------------ | --------- | ----------- |
| `id`         | UUID      | Auto-generated UUIDv4 primary key |
| `created_at` | timestamp | Record creation time |
| `updated_at` | timestamp | Last modification time |
| `deleted_at` | timestamp | Soft-delete timestamp (nullable; GORM soft deletes) |

### Database Architecture

#### Per-Webhook Event Databases

webhooker uses **separate SQLite database files**: a main application
database for configuration data and per-webhook databases for event
storage. All database files live in the `DATA_DIR` directory.

**Main Application Database** (`{DATA_DIR}/webhooker.db`) — stores
configuration and application state:

- **Settings** — auto-managed key-value config (e.g. session encryption
  key)
- **Users** — accounts and Argon2id password hashes
- **Webhooks** — webhook configurations
- **Entrypoints** — receiver URL definitions
- **Targets** — delivery destination configurations
- **APIKeys** — programmatic access credentials

On first startup the main database is auto-migrated, a session
encryption key is generated and stored, and an `admin` user is created.

**Per-Webhook Event Databases** (`{DATA_DIR}/events-{webhook_uuid}.db`)
— each webhook gets its own dedicated SQLite file containing:

- **Events** — captured incoming webhook payloads
- **Deliveries** — event-to-target pairings and their status
- **DeliveryResults** — individual delivery attempt logs

Per-webhook databases are created automatically when a webhook is
created (and lazily on first access for webhooks that predate this
feature). They are managed by the `WebhookDBManager` component, which
handles connection pooling, lazy opening, migrations, and cleanup.

This separation provides:

- **Isolation** — a high-volume webhook won't cause lock contention or
  WAL bloat affecting the main application or other webhooks.
- **Independent lifecycle** — event databases can be independently
  backed up, archived, rotated, or size-limited without impacting the
  application.
- **Clean deletion** — removing a webhook and all its history is as
  simple as deleting one file. Configuration is soft-deleted in the main
  DB; the event database file is hard-deleted (permanently removed).
- **Per-webhook retention** — the `retention_days` field on each webhook
  controls automatic cleanup of old events in that webhook's database
  only, or disables cleanup entirely when set to `0` (retain forever).
- **Performance** — each webhook's database has its own WAL, its own
  page cache, and its own lock, so concurrent event ingestion across
  webhooks won't contend.

The **database target type** builds on this architecture to provide
long-term archiving, separate from the per-webhook event database (which
may prune events under its own retention). Delivering to a database
target writes the full event — body, headers, method, content type, and
webhook/entrypoint/event identifiers — as a row into a dedicated archive
database, `archive-{webhookID}.db`, stored under the data directory
beside the event database. After each write the archive handle is closed
and reopened, debounced to at most once per second, so an operator can
move the archive file away for offline archiving without stopping the
service; a moved or removed archive file is recreated automatically on
the next write. An optional `expiry` in the target's config JSON (e.g.
`{"expiry":"720h"}`) is validated when the target is created — the
default (unset or the literal `never`) keeps rows forever — and rows
older than the expiry are pruned each time the archive is (re)opened. An
archive write failure is never silent success: the delivery records a
failed attempt with the error and is marked failed.

Because reopens only happen on writes, an archive belonging to a webhook
that has stopped receiving events would never be pruned. A background
**archive sweeper** closes that gap: on the same interval as the event
retention reaper (`RETENTION_SWEEP_INTERVAL`) it prunes every archive
whose database target declares a positive expiry, whether or not the
webhook is still receiving traffic. The sweep never creates an archive —
a webhook whose archive file does not yet exist is skipped, not
initialised — it takes the same per-webhook lock the write path uses, so
it can never interleave with a write, and it leaves the archive closed
afterwards so the move-the-file-away workflow keeps working. Archives
with no expiry, or the expiry `never`, are not touched by the sweep at
all.

Note that a webhook has one archive file but may carry more than one
`database` target, each with its own `expiry`. The shortest expiry
configured on any of them therefore governs the whole archive, and the
sweep applies it whether or not the webhook is still receiving events.
Configure a single `database` target per webhook unless you intend that.

Deleting a webhook releases its archive: the delivery engine's cached
archive writer is dropped and its file handle closed, so nothing lingers
after the webhook is gone. The archive **file itself is deliberately
left on disk**. Unlike the event database — per-webhook working storage
that is hard-deleted with the webhook — an archive is long-term storage
an operator may still want to keep or move away for offline retention,
and destroying it as a side effect of deleting a webhook would be
unrecoverable. Removing `archive-{webhookID}.db` is the operator's call.
Deleting a webhook's last `database` target releases the writer the same
way, and for the same reason leaves the file alone.

The **Slack target type** sends webhook events as formatted messages to
any Slack-compatible incoming webhook URL (works with Slack, Mattermost,
and other compatible services). Each message includes event metadata
(HTTP method, content type, timestamp, body size) and the payload
pretty-printed in a code block. JSON payloads are automatically
formatted with indentation for readability; non-JSON payloads are shown
as raw text. Large payloads are truncated to keep messages reasonable.
Config stores `webhook_url` — the Slack/Mattermost incoming webhook
endpoint.

The database uses the
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at
runtime, though CGO is required at build time due to the transitive
`mattn/go-sqlite3` dependency from `gorm.io/driver/sqlite`.

### Request Flow

```
External Service
       │
       │  POST /webhook/{uuid}
       ▼
┌─────────────┐     ┌──────────────┐     ┌──────────────┐
│  chi Router │────►│  Middleware   │────►│   Webhook    │
│             │     │  Stack       │     │   Handler    │
└─────────────┘     └──────────────┘     └──────┬───────┘
                                                │
                     1. Look up Entrypoint by UUID
                     2. Capture full request as Event
                     3. Create Delivery records for each active Target
                     4. Build self-contained DeliveryTask structs
                        (target config + event data inline for ≤16KB)
                     5. Notify Engine via channel (no DB read needed)
                                                │
                                                ▼
                                        ┌──────────────┐
                                        │  Delivery    │◄── retry timers
                                        │  Engine      │    (backoff)
                                        │  (worker     │
                                        │   pool)      │
                                        └──────┬───────┘
                                               │
                              ┌── bounded worker pool (N workers) ──┐
                              ▼                ▼                    ▼
                       ┌────────────┐   ┌────────────┐      ┌────────────┐
                       │ HTTP Target│   │ HTTP Target│      │ Log Target │
                       │(max_retries│   │(max_retries│      │ (stdout)   │
                       │   == 0)    │   │   > 0,     │      └────────────┘
                       │ fire+forget│   │ backoff +  │
                       └────────────┘   │ circuit    │
                                        │ breaker)   │
                                        └────────────┘
```

### Bounded Worker Pool

The delivery engine uses a **fixed-size worker pool** (default: 10
workers) to process all deliveries. At most N deliveries are in-flight
at any time, preventing goroutine explosions regardless of queue depth.

**Architecture:**

- **Channels as queues:** Two buffered channels serve as bounded queues:
  a delivery channel (new tasks from the webhook handler) and a retry
  channel (tasks from backoff timers). Both are buffered to 10,000.
- **Fan-out via channel, not goroutines:** When an event arrives with
  multiple targets, each `DeliveryTask` is sent to the delivery channel.
  Workers pick them up and process them — no goroutine-per-target.
- **Worker goroutines:** A fixed number of worker goroutines select from
  both channels. Each worker processes one task at a time, then picks up
  the next. Workers are the ONLY goroutines doing actual HTTP delivery.
- **Retry backpressure with DB fallback:** When a retry timer fires and
  the retry channel is full, the timer is dropped — the delivery stays
  in `retrying` status in the database. A periodic sweep (every 60s)
  scans for these "orphaned" retries and re-queues them. No blocked
  goroutines, no unbounded timer chains.
- **Bounded concurrency:** At most N deliveries (N = number of workers)
  are in-flight simultaneously. Even if a circuit breaker is open for
  hours and thousands of retries queue up in the channels, the workers
  drain them at a controlled rate when the circuit closes.

This means:

- **No goroutine explosion** — even with 10,000 queued retries, only
  N worker goroutines exist.
- **Natural backpressure** — if workers are busy, new tasks wait in the
  channel buffer rather than spawning more goroutines.
- **Independent results** — each worker records its own delivery result
  in the per-webhook database without coordination.
- **Graceful shutdown** — cancel the context, workers finish their
  current task and exit. `WaitGroup.Wait()` ensures clean shutdown.

**Recovery paths:**

1. **Startup recovery:** When the engine starts, it scans all per-webhook
   databases for `pending` and `retrying` deliveries. Pending deliveries
   are sent to the delivery channel; retrying deliveries get backoff
   timers scheduled.
2. **Periodic retry sweep (DB-mediated fallback):** Every 60 seconds the
   engine scans for `retrying` deliveries whose backoff period has
   elapsed. This catches "orphaned" retries — ones whose in-memory timer
   was dropped because the retry channel was full. The database is the
   durable fallback that ensures no retry is permanently lost, even under
   extreme backpressure.

**Changing a target's type does not migrate in-flight deliveries.** Only
`http` and `slack` targets own durable retries; `database` and `log`
targets are fire-and-forget and never produce a `retrying` delivery. If a
target's `type` is edited from a retrying type to a non-retrying (or
unknown) one while one of its deliveries is still `retrying`, both
recovery paths above terminally mark that delivery `failed` and record a
`DeliveryResult` naming the current target type as the reason, logging it
at warn level. The delivery is not re-dispatched under the new type — the
operator never asked for that delivery — and the event itself remains
stored in the per-webhook event database, so it can be redelivered
manually.

### Circuit Breaker (HTTP Targets with Retries)

HTTP targets with `max_retries` > 0 are protected by a **per-target circuit breaker** that
prevents hammering a down target with repeated failed delivery attempts.
The circuit breaker is in-memory only and resets on restart (which is
fine — startup recovery rescans the database anyway).

**States:**

| State       | Behavior |
| ----------- | -------- |
| **Closed**  | Normal operation. Deliveries flow through. Consecutive failures are counted. |
| **Open**    | Target appears down. Deliveries are skipped and rescheduled for after the cooldown. |
| **Half-Open** | Cooldown expired. One probe delivery is allowed to test if the target has recovered. |

**Transitions:**

```
     success              ┌──────────┐
   ┌────────────────────► │  Closed  │ ◄─── probe succeeds
   │                      │ (normal) │
   │                      └────┬─────┘
   │                           │ N consecutive failures
   │                           ▼
   │                      ┌──────────┐
   │                      │   Open   │ ◄─── probe fails
   │                      │(tripped) │
   │                      └────┬─────┘
   │                           │ cooldown expires
   │                           ▼
   │                      ┌──────────┐
   └──────────────────────│Half-Open │
                          │ (probe)  │
                          └──────────┘
```

**Defaults:**

- **Failure threshold:** 5 consecutive failures before opening
- **Cooldown:** 30 seconds in open state before probing

**Scope:** Circuit breakers only apply to **HTTP targets with
`max_retries` > 0**. Fire-and-forget HTTP targets (`max_retries` == 0),
Slack targets, database targets (local operations), and log
targets (stdout) do not use circuit breakers.

When a circuit is open and a new delivery arrives, the engine marks the
delivery as `retrying` and schedules a retry timer for after the
remaining cooldown period. This ensures no deliveries are lost — they're
just delayed until the target is healthy again.

### Rate Limiting

Global blanket rate limiting middleware (e.g., a per-IP throttle shared
with the web UI) **must not** apply to webhook receiver endpoints.
Webhook endpoints receive automated traffic from external services at
unpredictable rates, and blanket limits shared with other routes would
cause legitimate deliveries to be dropped.

The receiver instead has its own dedicated abuse limit, scoped to the
`/webhook/{uuid}` route only and keyed per client IP per entrypoint: one
misbehaving sender is throttled without affecting other senders of the
same entrypoint or the same sender's other entrypoints. The limit is
`RECEIVER_RATE_LIMIT` requests per minute (default 120, generous for
legitimate webhook senders). Requests over the limit receive HTTP 429
with a `Retry-After` header. A set-but-invalid `RECEIVER_RATE_LIMIT`
value aborts startup rather than silently falling back to the default.

Every limiter here — receiver, login, and password change — identifies
the client the same way, through one shared key function: the
connection's own address, unless the peer is listed in
`TRUSTED_PROXIES`, in which case the forwarded client address is used
instead. See [Trusted proxies](#trusted-proxies). Deployed without that
variable set, a client behind a reverse proxy shares one bucket with
every other client behind the same proxy, which is the safe direction
to be wrong in: set `TRUSTED_PROXIES` to the proxy's address to get
per-client limits back.

Finer-grained per-webhook rate limits (configured in the web UI and
enforced in the webhook handler) can layer on top of this env-level
abuse limit later; they are tracked as future work.

### API Endpoints

#### Public Endpoints

| Method | Path                        | Description |
| ------ | --------------------------- | ----------- |
| `GET`  | `/`                         | Root redirect (authenticated → `/sources`, unauthenticated → `/pages/login`) |
| `GET`  | `/.well-known/healthcheck`  | Health check (JSON: status, uptime, version) |
| `GET`  | `/s/*`                      | Static file serving (embedded CSS, JS) |
| `ANY`  | `/webhook/{uuid}`           | Webhook receiver endpoint (accepts all methods) |

#### Authentication Endpoints

| Method | Path            | Description |
| ------ | --------------- | ----------- |
| `GET`  | `/pages/login`  | Login page |
| `POST` | `/pages/login`  | Login form submission |
| `POST` | `/pages/logout` | Logout (destroys session) |

#### Authenticated Endpoints

| Method | Path                     | Description |
| ------ | ------------------------ | ----------- |
| `GET`  | `/user/{username}`       | User profile page |
| `GET`  | `/sources`               | List user's webhooks |
| `GET`  | `/sources/new`           | Create webhook form |
| `POST` | `/sources/new`           | Create webhook submission |
| `GET`  | `/source/{id}`           | Webhook detail view |
| `GET`  | `/source/{id}/edit`      | Edit webhook form |
| `POST` | `/source/{id}/edit`      | Edit webhook submission |
| `POST` | `/source/{id}/delete`    | Delete webhook |
| `GET`  | `/source/{id}/logs`      | Webhook event logs |
| `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook |
| `POST` | `/source/{id}/targets`   | Add target to webhook |

#### Infrastructure Endpoints

| Method | Path       | Description |
| ------ | ---------- | ----------- |
| `GET`  | `/metrics` | Prometheus metrics (requires basic auth) |

#### API (Planned)

| Method   | Path                           | Description |
| -------- | ------------------------------ | ----------- |
| `GET`    | `/api/v1/webhooks`             | List webhooks |
| `POST`   | `/api/v1/webhooks`             | Create webhook |
| `GET`    | `/api/v1/webhooks/{id}`        | Get webhook details |
| `PUT`    | `/api/v1/webhooks/{id}`        | Update webhook |
| `DELETE` | `/api/v1/webhooks/{id}`        | Delete webhook |
| `GET`    | `/api/v1/webhooks/{id}/events` | List events for webhook |
| `POST`   | `/api/v1/events/{id}/redeliver`| Redeliver an event |

API authentication will use API keys passed via `Authorization: Bearer
<key>` header.

### Package Layout

All application code lives under `internal/` to prevent external
imports. The entry point is `cmd/webhooker/main.go`.

```
webhooker/
├── cmd/webhooker/
│   └── main.go               # Entry point: sets globals, wires fx
├── internal/
│   ├── config/
│   │   └── config.go          # Configuration loading from environment variables
│   ├── database/
│   │   ├── base_model.go      # BaseModel with UUID primary keys
│   │   ├── database.go        # GORM connection, migrations, admin seed
│   │   ├── models.go          # AutoMigrate for config-tier models
│   │   ├── model_setting.go   # Setting entity (key-value app config)
│   │   ├── model_user.go      # User entity
│   │   ├── model_webhook.go    # Webhook entity
│   │   ├── model_entrypoint.go # Entrypoint entity
│   │   ├── model_target.go    # Target entity and TargetType enum
│   │   ├── model_event.go     # Event entity (per-webhook DB)
│   │   ├── model_delivery.go  # Delivery entity (per-webhook DB)
│   │   ├── model_delivery_result.go  # DeliveryResult entity (per-webhook DB)
│   │   ├── model_apikey.go    # APIKey entity
│   │   ├── password.go        # Argon2id hashing and verification
│   │   └── webhook_db_manager.go  # Per-webhook DB lifecycle manager
│   ├── globals/
│   │   └── globals.go         # Build-time variables (appname, version, arch)
│   ├── delivery/
│   │   ├── engine.go          # Event-driven delivery engine (channel + timer based)
│   │   ├── circuit_breaker.go # Per-target circuit breaker for HTTP targets with retries
│   │   └── ssrf.go            # SSRF prevention (IP validation, safe HTTP transport)
│   ├── handlers/
│   │   ├── handlers.go        # Base handler struct, JSON helpers, template rendering
│   │   ├── auth.go            # Login, logout handlers
│   │   ├── healthcheck.go     # Health check handler
│   │   ├── index.go           # Index page handler
│   │   ├── profile.go         # User profile handler
│   │   ├── source_management.go  # Webhook CRUD handlers
│   │   └── webhook.go         # Webhook receiver handler
│   ├── healthcheck/
│   │   └── healthcheck.go     # Health check service (uptime, version)
│   ├── logger/
│   │   └── logger.go          # slog setup with TTY detection
│   ├── middleware/
│   │   ├── middleware.go      # Logging, CORS, Auth, Metrics, MetricsAuth, SecurityHeaders, MaxBodySize
│   │   ├── csrf.go            # CSRF protection middleware (gorilla/csrf)
│   │   └── ratelimit.go       # Per-IP rate limiting middleware (go-chi/httprate)
│   ├── server/
│   │   ├── server.go          # Server struct, fx lifecycle, signal handling
│   │   ├── http.go            # HTTP server setup with timeouts
│   │   └── routes.go          # All route definitions
│   └── session/
│       └── session.go         # Cookie-based session management
├── static/
│   ├── static.go              # //go:embed directive
│   ├── css/style.css          # Custom stylesheet (system font stack, card effects, layout)
│   └── js/app.js              # Client-side JavaScript (minimal bootstrap)
├── templates/                 # Go HTML templates (base, index, login, etc.)
├── Dockerfile                 # Multi-stage: lint, build+test, then Alpine runtime
├── Makefile                   # fmt, lint, test, check, build, docker targets
├── go.mod / go.sum
└── .golangci.yml              # Linter configuration
```

### Dependency Injection

Components are wired via Uber fx in this order:

1. `globals.New` — Build-time variables (appname, version, arch)
2. `logger.New` — Structured logging (slog with TTY detection)
3. `config.New` — Configuration loading (environment variables)
4. `database.New` — Main SQLite connection, config migrations, admin
   user seed
5. `database.NewWebhookDBManager` — Per-webhook event database
   lifecycle manager
6. `healthcheck.New` — Health check service
7. `session.New` — Cookie-based session manager (key from database)
8. `handlers.New` — HTTP handlers
9. `middleware.New` — HTTP middleware
10. `delivery.New` — Event-driven delivery engine
11. `delivery.Engine` → `handlers.DeliveryNotifier` — interface bridge
12. `server.New` — HTTP server and router

The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine)
{})` which triggers the fx lifecycle hooks in dependency order. The
`DeliveryNotifier` interface allows the webhook handler to send
self-contained `DeliveryTask` slices to the engine without a direct
package dependency. Each task carries all target config and event data
inline (for bodies ≤16KB), so the engine can deliver without reading
from any database — it only writes to record results.

### Middleware Stack

Applied to all routes in this order:

1. **Recoverer** — Panic recovery (chi built-in)
2. **RequestID** — Generate unique request IDs (chi built-in)
3. **SecurityHeaders** — Production security headers on every response
   (HSTS, X-Content-Type-Options, X-Frame-Options, CSP, Referrer-Policy,
   Permissions-Policy)
4. **Logging** — Structured request logging (method, URL, status,
   latency, remote IP, user agent, request ID)
5. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
6. **CORS** — Cross-origin resource sharing headers
7. **Timeout** — 60-second request timeout
8. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set;
   configured with `Repanic: true` so panics still reach Recoverer)

Additionally, form endpoints (`/pages`, `/user/*`, `/sources`,
`/source/*`) apply a **MaxBodySize** middleware that limits
POST/PUT/PATCH request bodies to 1 MB. It is registered ahead of the
CSRF middleware in every one of those route groups, because
gorilla/csrf parses the form; if the cap were installed after it, form
parsing would run under net/http's 10 MB default and the 1 MB limit
would never apply. A request that declares a `Content-Length` over the
limit is answered with `413 Request Entity Too Large` before any other
middleware or handler runs; a chunked request, or one that lies about
its length, is hard-capped by `http.MaxBytesReader` and fails
downstream at form-parse time.

### Authentication

- **Web UI:** Cookie-based sessions using gorilla/sessions with
  encrypted cookies. Sessions are configured with HttpOnly, SameSite
  Lax, and Secure (in production). Session lifetime is 7 days.
- **API (planned):** API key authentication via `Authorization: Bearer`
  header. API keys are stored per-user with usage tracking
  (`last_used_at`).
- **Metrics:** Basic authentication protecting the `/metrics` endpoint.

### Security

- Passwords hashed with Argon2id (64 MB memory cost)
- Session cookies are HttpOnly, SameSite Lax, Secure (prod only)
- Session regeneration on login to prevent session fixation attacks
- Session key is a 32-byte value auto-generated on first startup and
  stored in the database
- Production security headers on all responses: HSTS, X-Content-Type-Options
  (`nosniff`), X-Frame-Options (`DENY`), Content-Security-Policy, Referrer-Policy,
  and Permissions-Policy
- Request body size limits (1 MB) on all form POST endpoints, enforced
  by middleware that runs before CSRF parses the form
- **CSRF protection** via [gorilla/csrf](https://github.com/gorilla/csrf)
  on all state-changing forms (cookie-based double-submit tokens with
  HMAC authentication). Applied to `/pages`, `/sources`, `/source`, and
  `/user` routes. Excluded from `/webhook` (inbound webhook POSTs) and
  `/api` (stateless API). The middleware auto-detects TLS status
  per-request (via `r.TLS` and `X-Forwarded-Proto`) to set appropriate
  cookie security flags and Origin/Referer validation mode
- **SSRF prevention** for HTTP delivery targets: private/reserved IP
  ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked
  both at target creation time (URL validation) and at delivery time
  (custom HTTP transport with SSRF-safe dialer that validates resolved
  IPs before connecting, preventing DNS rebinding attacks)
- **Login rate limiting** via [go-chi/httprate](https://github.com/go-chi/httprate):
  per-IP sliding-window rate limiter on the login endpoint (5 POST
  attempts per minute per IP) to prevent brute-force attacks
- Prometheus metrics behind basic auth
- Static assets embedded in binary (no filesystem access needed at
  runtime)
- Container runs as non-root user (UID 1000)
- GORM soft deletes on all entities (data preserved for audit)

### Docker

The Dockerfile uses a multi-stage build:

1. **Builder stage** (Debian-based `golang:1.24`) — installs
   golangci-lint, downloads dependencies, copies source, runs `make
   check` (format verification, linting, tests, compilation).
2. **Runtime stage** (`alpine:3.21`) — copies the binary, creates the
   `/var/lib/webhooker` directory for all SQLite databases, runs as
   non-root user, exposes port 8080, includes a health check.

The builder uses Debian rather than Alpine because GORM's SQLite
dialect pulls in CGO-dependent headers at compile time. The runtime
binary is statically linked and runs on Alpine.

`docker build .` is the CI gate — if it passes, the code is formatted,
linted, tested, and compiled.

## TODO

See [TODO.md](TODO.md).

## License

MIT

## Author

[@sneak](https://sneak.berlin)