All checks were successful
check / check (push) Successful in 1m49s
- Webhook reception handler: look up entrypoint by UUID, verify active,
capture full HTTP request (method, headers, body, content-type), create
Event record, queue Delivery records for each active Target, return 200 OK.
Handles edge cases: unknown UUID → 404, inactive → 410, oversized → 413.
- Delivery engine (internal/delivery): fx-managed background goroutine that
polls for pending/retrying deliveries and dispatches to target type handlers.
Graceful shutdown via context cancellation.
- Target type implementations:
- HTTP: fire-and-forget POST with original headers forwarding
- Retry: exponential backoff (1s, 2s, 4s...) up to max_retries
- Database: immediate success (event already stored)
- Log: slog output with event details
- Webhook management pages with Tailwind CSS + Alpine.js:
- List (/sources): webhooks with entrypoint/target/event counts
- Create (/sources/new): form with auto-created default entrypoint
- Detail (/source/{id}): config, entrypoints, targets, recent events
- Edit (/source/{id}/edit): name, description, retention_days
- Delete (/source/{id}/delete): soft-delete with child records
- Add Entrypoint (/source/{id}/entrypoints): inline form
- Add Target (/source/{id}/targets): type-aware form
- Event Log (/source/{id}/logs): paginated with delivery status
- Updated README: marked completed items, updated naming conventions
table, added delivery engine to package layout and DI docs, updated
column names to reflect entity rename.
- Rebuilt Tailwind CSS for new template classes.
Part of: #15
738 lines
32 KiB
Markdown
738 lines
32 KiB
Markdown
# webhooker
|
|
|
|
webhooker is a self-hosted webhook proxy and store-and-forward service
|
|
written in [Go](https://golang.org) by
|
|
[@sneak](https://sneak.berlin). It receives webhooks from external
|
|
services, durably stores them, and delivers them to configured targets
|
|
with retry support, logging, and observability. Category: infrastructure
|
|
/ web service. License: MIT.
|
|
|
|
## Getting Started
|
|
|
|
### Prerequisites
|
|
|
|
- Go 1.24+
|
|
- golangci-lint v1.64+
|
|
- Docker (for containerized deployment)
|
|
|
|
### Quick Start
|
|
|
|
```bash
|
|
# Clone the repo
|
|
git clone https://git.eeqj.de/sneak/webhooker.git
|
|
cd webhooker
|
|
|
|
# Install Go dependencies
|
|
make deps
|
|
|
|
# Run all checks (format, lint, test, build)
|
|
make check
|
|
|
|
# Run in development mode (uses SQLite in current directory)
|
|
make dev
|
|
|
|
# Build Docker image
|
|
make docker
|
|
```
|
|
|
|
### Development Commands
|
|
|
|
```bash
|
|
make fmt # Format code (gofmt + goimports)
|
|
make lint # Run golangci-lint
|
|
make test # Run tests with race detection
|
|
make check # fmt-check + lint + test + build (CI gate)
|
|
make build # Build binary to bin/webhooker
|
|
make dev # go run ./cmd/webhooker
|
|
make docker # Build Docker image
|
|
make hooks # Install git pre-commit hook that runs make check
|
|
```
|
|
|
|
### Configuration
|
|
|
|
webhooker uses a YAML configuration file with environment-specific
|
|
overrides, loaded via the `pkg/config` library (Viper-based). The
|
|
environment is selected by setting `WEBHOOKER_ENVIRONMENT` to `dev` or
|
|
`prod` (default: `dev`).
|
|
|
|
Configuration is resolved in this order (highest priority first):
|
|
|
|
1. Environment variables
|
|
2. `.env` file (loaded via `godotenv/autoload`)
|
|
3. Config file values for the active environment
|
|
4. Config file defaults
|
|
|
|
| Variable | Description | Default |
|
|
| ----------------------- | ----------------------------------- | -------- |
|
|
| `WEBHOOKER_ENVIRONMENT` | `dev` or `prod` | `dev` |
|
|
| `PORT` | HTTP listen port | `8080` |
|
|
| `DBURL` | SQLite database connection string | *(required)* |
|
|
| `SESSION_KEY` | Base64-encoded 32-byte session key | *(required in prod)* |
|
|
| `DEBUG` | Enable debug logging | `false` |
|
|
| `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` |
|
|
| `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` |
|
|
| `SENTRY_DSN` | Sentry error reporting DSN | `""` |
|
|
|
|
On first startup in development mode, webhooker creates an `admin` user
|
|
with a randomly generated password and logs it to stdout. This password
|
|
is only displayed once.
|
|
|
|
### Running with Docker
|
|
|
|
```bash
|
|
docker run -d \
|
|
-p 8080:8080 \
|
|
-v /path/to/data:/data \
|
|
-e DBURL="file:/data/webhooker.db?cache=shared&mode=rwc" \
|
|
-e SESSION_KEY="<base64-encoded-32-byte-key>" \
|
|
-e WEBHOOKER_ENVIRONMENT=prod \
|
|
webhooker:latest
|
|
```
|
|
|
|
The container runs as a non-root user (`webhooker`, UID 1000), exposes
|
|
port 8080, and includes a health check against
|
|
`/.well-known/healthcheck`.
|
|
|
|
## Rationale
|
|
|
|
Webhook integrations between services are inherently fragile. The
|
|
receiving service must be online when the webhook fires, most webhook
|
|
senders provide no built-in retry mechanism, and there is no standard
|
|
way to inspect what was sent, when it was sent, or whether delivery
|
|
succeeded.
|
|
|
|
webhooker solves this by acting as a durable intermediary:
|
|
|
|
1. **Reliable ingestion** — webhooker is always ready to accept incoming
|
|
webhooks. It stores every received event before attempting any
|
|
delivery, so nothing is lost if downstream targets are unavailable.
|
|
|
|
2. **Guaranteed delivery** — Events are queued for delivery to each
|
|
configured target. Failed deliveries are retried with configurable
|
|
backoff. Every delivery attempt is logged with status codes, response
|
|
bodies, and timing.
|
|
|
|
3. **Observability** — Full request/response logging for every webhook
|
|
received and every delivery attempted. Prometheus metrics expose
|
|
volume, latency, and error rates. The web UI provides real-time
|
|
visibility into event flow.
|
|
|
|
4. **Fan-out** — A single incoming webhook can be delivered to multiple
|
|
targets simultaneously. This enables patterns like forwarding a
|
|
GitHub webhook to both a deployment service and a Slack channel.
|
|
|
|
5. **Replay** — Stored events can be manually redelivered for debugging
|
|
or testing, without requiring the original sender to fire the webhook
|
|
again.
|
|
|
|
### Use Cases
|
|
|
|
- **Store-and-forward** with configurable retries for unreliable
|
|
receivers
|
|
- **Observability** via Prometheus metrics on webhook frequency, payload
|
|
size, and delivery performance
|
|
- **Debugging** and introspection of webhook payloads in the web UI
|
|
- **Replay** of webhook events for application testing and development
|
|
- **Fan-out** delivery of a single webhook to multiple downstream
|
|
targets
|
|
- **High-availability ingestion** for delivery to less reliable backend
|
|
systems
|
|
|
|
## Design
|
|
|
|
### Architecture Overview
|
|
|
|
webhooker is structured as a standard Go HTTP server following the
|
|
[sneak/prompts GO_HTTP_SERVER_CONVENTIONS](https://git.eeqj.de/sneak/prompts/src/branch/main/prompts/GO_HTTP_SERVER_CONVENTIONS.md).
|
|
It uses:
|
|
|
|
- **[Uber fx](https://go.uber.org/fx)** for dependency injection and
|
|
lifecycle management
|
|
- **[go-chi](https://github.com/go-chi/chi)** for HTTP routing
|
|
- **[GORM](https://gorm.io)** for database access with
|
|
**[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite)** as
|
|
the runtime SQLite driver. Note: `gorm.io/driver/sqlite` transitively
|
|
depends on `mattn/go-sqlite3`, which requires CGO at build time (see
|
|
[Docker](#docker) section)
|
|
- **[slog](https://pkg.go.dev/log/slog)** (stdlib) for structured
|
|
logging with TTY detection (text for dev, JSON for prod)
|
|
- **[gorilla/sessions](https://github.com/gorilla/sessions)** for
|
|
encrypted cookie-based session management
|
|
- **[Prometheus](https://prometheus.io)** for metrics, served at
|
|
`/metrics` behind basic auth
|
|
- **[Sentry](https://sentry.io)** for optional error reporting
|
|
|
|
### Naming Conventions
|
|
|
|
The codebase uses consistent naming throughout (rename completed in
|
|
[issue #12](https://git.eeqj.de/sneak/webhooker/issues/12)):
|
|
|
|
| Entity | Description |
|
|
| ---------------- | ----------- |
|
|
| **Webhook** | Top-level configuration entity grouping entrypoints and targets |
|
|
| **Entrypoint** | A receiver URL where external services POST events |
|
|
| **Target** | A delivery destination for events |
|
|
|
|
### Data Model
|
|
|
|
webhooker's data model has eight entities organized into two tiers: the
|
|
**application tier** (user and webhook configuration) and the **event
|
|
tier** (event ingestion, delivery, and logging).
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ APPLICATION TIER │
|
|
│ (main application database) │
|
|
│ │
|
|
│ ┌──────────┐ ┌──────────┐ ┌──────────────┐ │
|
|
│ │ User │──1:N──│ Webhook │──1:N──│ Entrypoint │ │
|
|
│ │ │ │ │ │ │ │
|
|
│ │ │ │ │──1:N──│ Target │ │
|
|
│ │ │ └──────────┘ └──────────────┘ │
|
|
│ │ │──1:N──│ APIKey │ │
|
|
│ └──────────┘ └──────────┘ │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ EVENT TIER │
|
|
│ (planned: per-webhook dedicated database) │
|
|
│ │
|
|
│ ┌──────────┐ ┌──────────┐ ┌─────────────────┐ │
|
|
│ │ Event │──1:N──│ Delivery │──1:N──│ DeliveryResult │ │
|
|
│ └──────────┘ └──────────┘ └─────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
#### User
|
|
|
|
A registered user of the webhooker service.
|
|
|
|
| Field | Type | Description |
|
|
| ---------- | -------- | ----------- |
|
|
| `id` | UUID | Primary key |
|
|
| `username` | string | Unique login name |
|
|
| `password` | string | Argon2id hash (never exposed via API) |
|
|
|
|
**Relations:** Has many Webhooks. Has many APIKeys.
|
|
|
|
Passwords are hashed with Argon2id using secure defaults (64 MB memory,
|
|
1 iteration, 4 threads, 32-byte key, 16-byte salt). On first startup,
|
|
an `admin` user is created with a randomly generated 16-character
|
|
password logged to stdout.
|
|
|
|
#### Webhook
|
|
|
|
The top-level configuration entity. A webhook groups together one or
|
|
more entrypoints (receiver URLs) and one or more targets (delivery
|
|
destinations) into a logical unit. A user creates a webhook to set up
|
|
event routing.
|
|
|
|
| Field | Type | Description |
|
|
| ---------------- | ------- | ----------- |
|
|
| `id` | UUID | Primary key |
|
|
| `user_id` | UUID | Foreign key → User |
|
|
| `name` | string | Human-readable name |
|
|
| `description` | string | Optional description |
|
|
| `retention_days` | integer | Days to retain events (default: 30) |
|
|
|
|
**Relations:** Belongs to User. Has many Entrypoints. Has many Targets.
|
|
|
|
The `retention_days` field controls how long event data is kept in the
|
|
webhook's dedicated database before automatic cleanup.
|
|
|
|
#### Entrypoint
|
|
|
|
A receiver URL where external services POST webhook events. Each
|
|
entrypoint has a unique UUID-based path.
|
|
When an HTTP request arrives at an entrypoint's path, webhooker captures
|
|
the full request and creates an Event.
|
|
|
|
| Field | Type | Description |
|
|
| -------------- | ------- | ----------- |
|
|
| `id` | UUID | Primary key |
|
|
| `webhook_id` | UUID | Foreign key → Webhook |
|
|
| `path` | string | Unique URL path (UUID-based, e.g. `/webhook/{uuid}`) |
|
|
| `description` | string | Optional description |
|
|
| `active` | boolean | Whether this entrypoint accepts events (default: true) |
|
|
|
|
**Relations:** Belongs to Webhook.
|
|
|
|
A webhook can have multiple entrypoints. This allows separate URLs for
|
|
different event sources that all feed into the same processing pipeline
|
|
(e.g., one entrypoint for GitHub, another for Stripe, both routing to
|
|
the same targets).
|
|
|
|
#### Target
|
|
|
|
A delivery destination for events. Each target defines where and how
|
|
events should be forwarded.
|
|
|
|
| Field | Type | Description |
|
|
| ---------------- | ---------- | ----------- |
|
|
| `id` | UUID | Primary key |
|
|
| `webhook_id` | UUID | Foreign key → Webhook |
|
|
| `name` | string | Human-readable name |
|
|
| `type` | TargetType | One of: `http`, `retry`, `database`, `log` |
|
|
| `active` | boolean | Whether deliveries are enabled (default: true) |
|
|
| `config` | JSON text | Type-specific configuration |
|
|
| `max_retries` | integer | Maximum retry attempts (for retry targets) |
|
|
| `max_queue_size` | integer | Maximum queued deliveries (for retry targets) |
|
|
|
|
**Relations:** Belongs to Webhook. Has many Deliveries.
|
|
|
|
**Target types:**
|
|
|
|
- **`http`** — Forward the event as an HTTP POST to a configured URL.
|
|
Fire-and-forget: a single attempt with no retries.
|
|
- **`retry`** — Forward the event via HTTP POST with automatic retry on
|
|
failure. Uses exponential backoff up to `max_retries` attempts.
|
|
- **`database`** — Store the event in the webhook's database only (no
|
|
external delivery). Useful for pure logging/archival.
|
|
- **`log`** — Write the event to the application log (stdout). Useful
|
|
for debugging.
|
|
|
|
The `config` field stores type-specific configuration as JSON (e.g.,
|
|
destination URL, custom headers, timeout settings).
|
|
|
|
#### APIKey
|
|
|
|
A programmatic access credential for API authentication.
|
|
|
|
| Field | Type | Description |
|
|
| -------------- | --------- | ----------- |
|
|
| `id` | UUID | Primary key |
|
|
| `user_id` | UUID | Foreign key → User |
|
|
| `key` | string | Unique API key value |
|
|
| `description` | string | Optional description |
|
|
| `last_used_at` | timestamp | Last time this key was used (nullable) |
|
|
|
|
**Relations:** Belongs to User.
|
|
|
|
#### Event
|
|
|
|
A captured incoming webhook request. Stores the complete HTTP request
|
|
data for replay and auditing.
|
|
|
|
| Field | Type | Description |
|
|
| -------------- | ------ | ----------- |
|
|
| `id` | UUID | Primary key |
|
|
| `webhook_id` | UUID | Foreign key → Webhook |
|
|
| `entrypoint_id` | UUID | Foreign key → Entrypoint |
|
|
| `method` | string | HTTP method (POST, PUT, etc.) |
|
|
| `headers` | JSON | Complete request headers |
|
|
| `body` | text | Raw request body |
|
|
| `content_type` | string | Content-Type header value |
|
|
|
|
**Relations:** Belongs to Webhook. Belongs to Entrypoint. Has many
|
|
Deliveries.
|
|
|
|
When a request arrives at an entrypoint, the full request (method,
|
|
headers, body) is captured as an Event. The event is then queued for
|
|
delivery to every active target configured on the parent webhook.
|
|
|
|
#### Delivery
|
|
|
|
The pairing of an event with a target. Tracks the overall delivery
|
|
status across potentially multiple attempts.
|
|
|
|
| Field | Type | Description |
|
|
| ---------- | -------------- | ----------- |
|
|
| `id` | UUID | Primary key |
|
|
| `event_id` | UUID | Foreign key → Event |
|
|
| `target_id`| UUID | Foreign key → Target |
|
|
| `status` | DeliveryStatus | One of: `pending`, `delivered`, `failed`, `retrying` |
|
|
|
|
**Relations:** Belongs to Event. Belongs to Target. Has many
|
|
DeliveryResults.
|
|
|
|
**Delivery statuses:**
|
|
|
|
- **`pending`** — Created but not yet attempted.
|
|
- **`retrying`** — At least one attempt failed; more attempts remain.
|
|
- **`delivered`** — Successfully delivered (at least one attempt
|
|
succeeded).
|
|
- **`failed`** — All retry attempts exhausted without success.
|
|
|
|
#### DeliveryResult
|
|
|
|
The result of a single delivery attempt. Every attempt (including
|
|
retries) is individually logged for full observability.
|
|
|
|
| Field | Type | Description |
|
|
| --------------- | ------- | ----------- |
|
|
| `id` | UUID | Primary key |
|
|
| `delivery_id` | UUID | Foreign key → Delivery |
|
|
| `attempt_num` | integer | Attempt number (1-based) |
|
|
| `success` | boolean | Whether this attempt succeeded |
|
|
| `status_code` | integer | HTTP response status code (if applicable) |
|
|
| `response_body` | text | Response body (if applicable) |
|
|
| `error` | string | Error message (on failure) |
|
|
| `duration` | integer | Request duration in milliseconds |
|
|
|
|
**Relations:** Belongs to Delivery.
|
|
|
|
#### Common Fields
|
|
|
|
All entities include these fields from `BaseModel`:
|
|
|
|
| Field | Type | Description |
|
|
| ------------ | --------- | ----------- |
|
|
| `id` | UUID | Auto-generated UUIDv4 primary key |
|
|
| `created_at` | timestamp | Record creation time |
|
|
| `updated_at` | timestamp | Last modification time |
|
|
| `deleted_at` | timestamp | Soft-delete timestamp (nullable; GORM soft deletes) |
|
|
|
|
### Database Architecture
|
|
|
|
#### Current Implementation
|
|
|
|
webhooker currently uses a **single SQLite database** for all data —
|
|
application configuration, user accounts, and (once implemented) event
|
|
storage. The database connection is managed by GORM with a single
|
|
connection string configured via `DBURL`. On first startup the database
|
|
is auto-migrated and an `admin` user is created.
|
|
|
|
#### Planned: Per-Webhook Event Databases (Phase 2)
|
|
|
|
In a future phase (see TODO Phase 2 below), webhooker will split into
|
|
**separate SQLite database files**: a main application database for
|
|
configuration data and per-webhook databases for event storage.
|
|
|
|
**Main Application Database** — will store:
|
|
|
|
- **Users** — accounts and Argon2id password hashes
|
|
- **Webhooks** — webhook configurations
|
|
- **Entrypoints** — receiver URL definitions
|
|
- **Targets** — delivery destination configurations
|
|
- **APIKeys** — programmatic access credentials
|
|
|
|
**Per-Webhook Event Databases** — each webhook will get its own
|
|
dedicated SQLite file containing:
|
|
|
|
- **Events** — captured incoming webhook payloads
|
|
- **Deliveries** — event-to-target pairings and their status
|
|
- **DeliveryResults** — individual delivery attempt logs
|
|
|
|
This planned separation will provide:
|
|
|
|
- **Isolation** — a high-volume webhook won't cause lock contention or
|
|
WAL bloat affecting the main application or other webhooks.
|
|
- **Independent lifecycle** — event databases can be independently
|
|
backed up, archived, rotated, or size-limited without impacting the
|
|
application.
|
|
- **Clean deletion** — removing a webhook and all its history is as
|
|
simple as deleting one file.
|
|
- **Per-webhook retention** — the `retention_days` field on each webhook
|
|
will control automatic cleanup of old events in that webhook's
|
|
database only.
|
|
- **Performance** — each webhook's database will have its own WAL, its
|
|
own page cache, and its own lock, so concurrent event ingestion across
|
|
webhooks won't contend.
|
|
|
|
The database uses the
|
|
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at
|
|
runtime, though CGO is required at build time due to the transitive
|
|
`mattn/go-sqlite3` dependency from `gorm.io/driver/sqlite`.
|
|
|
|
### Request Flow
|
|
|
|
```
|
|
External Service
|
|
│
|
|
│ POST /webhook/{uuid}
|
|
▼
|
|
┌─────────────┐ ┌──────────────┐ ┌──────────────┐
|
|
│ chi Router │────►│ Middleware │────►│ Webhook │
|
|
│ │ │ Stack │ │ Handler │
|
|
└─────────────┘ └──────────────┘ └──────┬───────┘
|
|
│
|
|
1. Look up Entrypoint by UUID
|
|
2. Capture full request as Event
|
|
3. Queue Delivery to each active Target
|
|
│
|
|
▼
|
|
┌──────────────┐
|
|
│ Delivery │
|
|
│ Engine │
|
|
└──────┬───────┘
|
|
│
|
|
┌────────────────────┼────────────────────┐
|
|
▼ ▼ ▼
|
|
┌────────────┐ ┌────────────┐ ┌────────────┐
|
|
│ HTTP Target│ │Retry Target│ │ Log Target │
|
|
│ (1 attempt)│ │ (backoff) │ │ (stdout) │
|
|
└────────────┘ └────────────┘ └────────────┘
|
|
```
|
|
|
|
### Rate Limiting
|
|
|
|
Global rate limiting middleware (e.g., per-IP throttling applied at the
|
|
router level) **must not** apply to webhook receiver endpoints. Webhook
|
|
endpoints receive automated traffic from external services at
|
|
unpredictable rates, and blanket rate limits would cause legitimate
|
|
deliveries to be dropped.
|
|
|
|
Instead, each webhook has its own individually configurable rate limit,
|
|
applied within the webhook handler itself. By default, no rate limit is
|
|
applied — webhook endpoints accept traffic as fast as it arrives. Rate
|
|
limits can be configured per-webhook when needed (e.g., to protect
|
|
against a misbehaving sender).
|
|
|
|
### API Endpoints
|
|
|
|
#### Public Endpoints
|
|
|
|
| Method | Path | Description |
|
|
| ------ | --------------------------- | ----------- |
|
|
| `GET` | `/` | Web UI index page (server-rendered) |
|
|
| `GET` | `/.well-known/healthcheck` | Health check (JSON: status, uptime, version) |
|
|
| `GET` | `/s/*` | Static file serving (embedded CSS, JS) |
|
|
| `ANY` | `/webhook/{uuid}` | Webhook receiver endpoint (accepts all methods) |
|
|
|
|
#### Authentication Endpoints
|
|
|
|
| Method | Path | Description |
|
|
| ------ | --------------- | ----------- |
|
|
| `GET` | `/pages/login` | Login page |
|
|
| `POST` | `/pages/login` | Login form submission |
|
|
| `POST` | `/pages/logout` | Logout (destroys session) |
|
|
|
|
#### Authenticated Endpoints
|
|
|
|
| Method | Path | Description |
|
|
| ------ | ------------------------ | ----------- |
|
|
| `GET` | `/user/{username}` | User profile page |
|
|
| `GET` | `/sources` | List user's webhooks |
|
|
| `GET` | `/sources/new` | Create webhook form |
|
|
| `POST` | `/sources/new` | Create webhook submission |
|
|
| `GET` | `/source/{id}` | Webhook detail view |
|
|
| `GET` | `/source/{id}/edit` | Edit webhook form |
|
|
| `POST` | `/source/{id}/edit` | Edit webhook submission |
|
|
| `POST` | `/source/{id}/delete` | Delete webhook |
|
|
| `GET` | `/source/{id}/logs` | Webhook event logs |
|
|
| `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook |
|
|
| `POST` | `/source/{id}/targets` | Add target to webhook |
|
|
|
|
#### Infrastructure Endpoints
|
|
|
|
| Method | Path | Description |
|
|
| ------ | ---------- | ----------- |
|
|
| `GET` | `/metrics` | Prometheus metrics (requires basic auth) |
|
|
|
|
#### API (Planned)
|
|
|
|
| Method | Path | Description |
|
|
| -------- | ------------------------------ | ----------- |
|
|
| `GET` | `/api/v1/webhooks` | List webhooks |
|
|
| `POST` | `/api/v1/webhooks` | Create webhook |
|
|
| `GET` | `/api/v1/webhooks/{id}` | Get webhook details |
|
|
| `PUT` | `/api/v1/webhooks/{id}` | Update webhook |
|
|
| `DELETE` | `/api/v1/webhooks/{id}` | Delete webhook |
|
|
| `GET` | `/api/v1/webhooks/{id}/events` | List events for webhook |
|
|
| `POST` | `/api/v1/events/{id}/redeliver`| Redeliver an event |
|
|
|
|
API authentication will use API keys passed via `Authorization: Bearer
|
|
<key>` header.
|
|
|
|
### Package Layout
|
|
|
|
All application code lives under `internal/` to prevent external
|
|
imports. The entry point is `cmd/webhooker/main.go`.
|
|
|
|
```
|
|
webhooker/
|
|
├── cmd/webhooker/
|
|
│ └── main.go # Entry point: sets globals, wires fx
|
|
├── internal/
|
|
│ ├── config/
|
|
│ │ └── config.go # Configuration loading via pkg/config
|
|
│ ├── database/
|
|
│ │ ├── base_model.go # BaseModel with UUID primary keys
|
|
│ │ ├── database.go # GORM connection, migrations, admin seed
|
|
│ │ ├── models.go # AutoMigrate for all models
|
|
│ │ ├── model_user.go # User entity
|
|
│ │ ├── model_webhook.go # Webhook entity
|
|
│ │ ├── model_entrypoint.go # Entrypoint entity
|
|
│ │ ├── model_target.go # Target entity and TargetType enum
|
|
│ │ ├── model_event.go # Event entity
|
|
│ │ ├── model_delivery.go # Delivery entity and DeliveryStatus enum
|
|
│ │ ├── model_delivery_result.go # DeliveryResult entity
|
|
│ │ ├── model_apikey.go # APIKey entity
|
|
│ │ └── password.go # Argon2id hashing and verification
|
|
│ ├── globals/
|
|
│ │ └── globals.go # Build-time variables (appname, version, arch)
|
|
│ ├── delivery/
|
|
│ │ └── engine.go # Background delivery engine (fx lifecycle)
|
|
│ ├── handlers/
|
|
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
|
|
│ │ ├── auth.go # Login, logout handlers
|
|
│ │ ├── healthcheck.go # Health check handler
|
|
│ │ ├── index.go # Index page handler
|
|
│ │ ├── profile.go # User profile handler
|
|
│ │ ├── source_management.go # Webhook CRUD handlers
|
|
│ │ └── webhook.go # Webhook receiver handler
|
|
│ ├── healthcheck/
|
|
│ │ └── healthcheck.go # Health check service (uptime, version)
|
|
│ ├── logger/
|
|
│ │ └── logger.go # slog setup with TTY detection
|
|
│ ├── middleware/
|
|
│ │ └── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth
|
|
│ ├── server/
|
|
│ │ ├── server.go # Server struct, fx lifecycle, signal handling
|
|
│ │ ├── http.go # HTTP server setup with timeouts
|
|
│ │ └── routes.go # All route definitions
|
|
│ └── session/
|
|
│ └── session.go # Cookie-based session management
|
|
├── pkg/config/ # Reusable multi-environment config library
|
|
├── static/
|
|
│ ├── static.go # //go:embed directive
|
|
│ ├── css/style.css # Custom stylesheet (system font stack, card effects, layout)
|
|
│ └── js/app.js # Client-side JavaScript (minimal bootstrap)
|
|
├── templates/ # Go HTML templates (base, index, login, etc.)
|
|
├── configs/
|
|
│ └── config.yaml.example # Example configuration file
|
|
├── Dockerfile # Multi-stage: build + check, then Alpine runtime
|
|
├── Makefile # fmt, lint, test, check, build, docker targets
|
|
├── go.mod / go.sum
|
|
└── .golangci.yml # Linter configuration
|
|
```
|
|
|
|
### Dependency Injection
|
|
|
|
Components are wired via Uber fx in this order:
|
|
|
|
1. `globals.New` — Build-time variables (appname, version, arch)
|
|
2. `logger.New` — Structured logging (slog with TTY detection)
|
|
3. `config.New` — Configuration loading (pkg/config + environment)
|
|
4. `database.New` — SQLite connection, migrations, admin user seed
|
|
5. `healthcheck.New` — Health check service
|
|
6. `session.New` — Cookie-based session manager
|
|
7. `handlers.New` — HTTP handlers
|
|
8. `middleware.New` — HTTP middleware
|
|
9. `delivery.New` — Background delivery engine
|
|
10. `server.New` — HTTP server and router
|
|
|
|
The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine)
|
|
{})` which triggers the fx lifecycle hooks in dependency order.
|
|
|
|
### Middleware Stack
|
|
|
|
Applied to all routes in this order:
|
|
|
|
1. **Recoverer** — Panic recovery (chi built-in)
|
|
2. **RequestID** — Generate unique request IDs (chi built-in)
|
|
3. **Logging** — Structured request logging (method, URL, status,
|
|
latency, remote IP, user agent, request ID)
|
|
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
|
|
5. **CORS** — Cross-origin resource sharing headers
|
|
6. **Timeout** — 60-second request timeout
|
|
7. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set;
|
|
configured with `Repanic: true` so panics still reach Recoverer)
|
|
|
|
### Authentication
|
|
|
|
- **Web UI:** Cookie-based sessions using gorilla/sessions with
|
|
encrypted cookies. Sessions are configured with HttpOnly, SameSite
|
|
Lax, and Secure (in production). Session lifetime is 7 days.
|
|
- **API (planned):** API key authentication via `Authorization: Bearer`
|
|
header. API keys are stored per-user with usage tracking
|
|
(`last_used_at`).
|
|
- **Metrics:** Basic authentication protecting the `/metrics` endpoint.
|
|
|
|
### Security
|
|
|
|
- Passwords hashed with Argon2id (64 MB memory cost)
|
|
- Session cookies are HttpOnly, SameSite Lax, Secure (prod only)
|
|
- Session key must be a 32-byte base64-encoded value
|
|
- Prometheus metrics behind basic auth
|
|
- Static assets embedded in binary (no filesystem access needed at
|
|
runtime)
|
|
- Container runs as non-root user (UID 1000)
|
|
- GORM soft deletes on all entities (data preserved for audit)
|
|
|
|
### Docker
|
|
|
|
The Dockerfile uses a multi-stage build:
|
|
|
|
1. **Builder stage** (Debian-based `golang:1.24`) — installs
|
|
golangci-lint, downloads dependencies, copies source, runs `make
|
|
check` (format verification, linting, tests, compilation).
|
|
2. **Runtime stage** (`alpine:3.21`) — copies the binary, runs as
|
|
non-root user, exposes port 8080, includes a health check.
|
|
|
|
The builder uses Debian rather than Alpine because GORM's SQLite
|
|
dialect pulls in CGO-dependent headers at compile time. The runtime
|
|
binary is statically linked and runs on Alpine.
|
|
|
|
`docker build .` is the CI gate — if it passes, the code is formatted,
|
|
linted, tested, and compiled.
|
|
|
|
## TODO
|
|
|
|
### Completed: Code Quality (Phase 1 of MVP)
|
|
- [x] Rename Processor → Webhook, Webhook → Entrypoint in code
|
|
([#12](https://git.eeqj.de/sneak/webhooker/issues/12))
|
|
- [x] Embed templates via `//go:embed`
|
|
([#7](https://git.eeqj.de/sneak/webhooker/issues/7))
|
|
- [x] Use `slog.LevelVar` for dynamic log level switching
|
|
([#8](https://git.eeqj.de/sneak/webhooker/issues/8))
|
|
- [x] Simplify configuration to prefer environment variables
|
|
([#10](https://git.eeqj.de/sneak/webhooker/issues/10))
|
|
- [x] Remove redundant `godotenv/autoload` import
|
|
([#11](https://git.eeqj.de/sneak/webhooker/issues/11))
|
|
- [x] Implement authentication middleware for protected routes
|
|
([#9](https://git.eeqj.de/sneak/webhooker/issues/9))
|
|
- [x] Replace Bootstrap with Tailwind CSS + Alpine.js
|
|
([#4](https://git.eeqj.de/sneak/webhooker/issues/4))
|
|
|
|
### Completed: Core Webhook Engine (Phase 2 of MVP)
|
|
- [x] Implement webhook reception and event storage at `/webhook/{uuid}`
|
|
- [x] Build event processing and target delivery engine
|
|
- [x] Implement HTTP target type (fire-and-forget POST)
|
|
- [x] Implement retry target type (exponential backoff)
|
|
- [x] Implement database target type (store only)
|
|
- [x] Implement log target type (console output)
|
|
- [x] Webhook management pages (list, create, edit, delete)
|
|
- [x] Webhook request log viewer with pagination
|
|
- [x] Entrypoint and target management UI
|
|
|
|
### Remaining: Core Features
|
|
- [ ] Per-webhook rate limiting in the receiver handler
|
|
- [ ] Webhook signature verification (GitHub, Stripe formats)
|
|
- [ ] Security headers (HSTS, CSP, X-Frame-Options)
|
|
- [ ] CSRF protection for forms
|
|
- [ ] Session expiration and "remember me"
|
|
- [ ] Password change/reset flow
|
|
- [ ] API key authentication for programmatic access
|
|
- [ ] Manual event redelivery
|
|
- [ ] Analytics dashboard (success rates, response times)
|
|
- [ ] Delivery status and retry management UI
|
|
|
|
### Remaining: Database Separation
|
|
- [ ] Split into main application DB + per-webhook event DBs
|
|
- [ ] Automatic event retention cleanup based on `retention_days`
|
|
- [ ] Per-webhook database lifecycle management (create on webhook
|
|
creation, delete on webhook removal)
|
|
|
|
### Remaining: REST API
|
|
- [ ] RESTful CRUD for webhooks, entrypoints, targets
|
|
- [ ] Event viewing and filtering endpoints
|
|
- [ ] Event redelivery endpoint
|
|
- [ ] OpenAPI specification
|
|
|
|
### Future
|
|
- [ ] Email delivery target type
|
|
- [ ] SNS, S3, Slack delivery targets
|
|
- [ ] Data transformations (e.g., webhook-to-Slack message formatting)
|
|
- [ ] JSONL file delivery with periodic S3 upload
|
|
- [ ] Webhook event search and filtering
|
|
- [ ] Multi-user with role-based access
|
|
|
|
## License
|
|
|
|
MIT
|
|
|
|
## Author
|
|
|
|
[@sneak](https://sneak.berlin)
|