sneak/webhooker
1
0
Fork 0
Code Issues 8 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: comprehensive README rewrite with complete service specification #13

Merged
sneak merged 3 commits from feature/comprehensive-readme-rewrite into main 2026-03-02 00:43:56 +01:00
Conversation 3 Commits 3 Files Changed 1 +675 -126
1 changed files with 675 additions and 126 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

801
README.md
View File

@ -1,188 +1,737 @@
# webhooker
webhooker is a Go web application by [@sneak](https://sneak.berlin) that
receives, stores, and proxies webhooks to configured targets with retry
support, observability, and a management web UI. License: pending.
webhooker is a self-hosted webhook proxy and store-and-forward service
written in [Go](https://golang.org) by
[@sneak](https://sneak.berlin). It receives webhooks from external
services, durably stores them, and delivers them to configured targets
with retry support, logging, and observability. Category: infrastructure
/ web service. License: MIT.
## Getting Started
### Prerequisites
- Go 1.24+
- golangci-lint v1.64+
- Docker (for containerized deployment)
### Quick Start
```bash
# Clone the repo
git clone https://git.eeqj.de/sneak/webhooker.git
cd webhooker
# Install dependencies
# Install Go dependencies
make deps
# Copy example config
cp configs/config.yaml.example config.yaml
# Run in development mode
make dev
# Run all checks (format, lint, test, build)
make check
# Run in development mode (uses SQLite in current directory)
make dev
# Build Docker image
make docker
```
### Environment Variables
### Development Commands
- `WEBHOOKER_ENVIRONMENT` — `dev` or `prod` (default: `dev`)
- `DEBUG` — Enable debug logging
- `PORT` — Server port (default: `8080`)
- `DBURL` — Database connection string
- `SESSION_KEY` — Base64-encoded 32-byte session key (required in prod)
- `METRICS_USERNAME` — Username for metrics endpoint
- `METRICS_PASSWORD` — Password for metrics endpoint
- `SENTRY_DSN` — Sentry error reporting (optional)
```bash
make fmt # Format code (gofmt + goimports)
make lint # Run golangci-lint
make test # Run tests with race detection
make check # fmt-check + lint + test + build (CI gate)
make build # Build binary to bin/webhooker
make dev # go run ./cmd/webhooker
make docker # Build Docker image
make hooks # Install git pre-commit hook that runs make check
```
### Configuration
webhooker uses a YAML configuration file with environment-specific
overrides, loaded via the `pkg/config` library (Viper-based). The
environment is selected by setting `WEBHOOKER_ENVIRONMENT` to `dev` or
`prod` (default: `dev`).
Configuration is resolved in this order (highest priority first):
1. Environment variables
2. `.env` file (loaded via `godotenv/autoload`)
3. Config file values for the active environment
4. Config file defaults
| Variable | Description | Default |
| ----------------------- | ----------------------------------- | -------- |
| `WEBHOOKER_ENVIRONMENT` | `dev` or `prod` | `dev` |
| `PORT` | HTTP listen port | `8080` |
| `DBURL` | SQLite database connection string | *(required)* |
| `SESSION_KEY` | Base64-encoded 32-byte session key | *(required in prod)* |
| `DEBUG` | Enable debug logging | `false` |
| `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` |
| `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` |
| `SENTRY_DSN` | Sentry error reporting DSN | `""` |
On first startup in development mode, webhooker creates an `admin` user
with a randomly generated password and logs it to stdout. This password
is only displayed once.
### Running with Docker
```bash
docker run -d \
-p 8080:8080 \
-v /path/to/data:/data \
-e DBURL="file:/data/webhooker.db?cache=shared&mode=rwc" \
-e SESSION_KEY="<base64-encoded-32-byte-key>" \
-e WEBHOOKER_ENVIRONMENT=prod \
webhooker:latest
```
The container runs as a non-root user (`webhooker`, UID 1000), exposes
port 8080, and includes a health check against
`/.well-known/healthcheck`.
## Rationale
Webhook integrations between services are fragile: the receiving service
must be up when the webhook fires, there is no built-in retry for most
webhook senders, and there is no visibility into what was sent or when.
webhooker solves this by acting as a reliable intermediary that receives
webhooks, stores them, and delivers them to configured targets — with
optional retries, logging, and Prometheus metrics for observability.
Webhook integrations between services are inherently fragile. The
receiving service must be online when the webhook fires, most webhook
senders provide no built-in retry mechanism, and there is no standard
way to inspect what was sent, when it was sent, or whether delivery
succeeded.
Use cases include:
webhooker solves this by acting as a durable intermediary:
- Store-and-forward with unlimited retries for unreliable receivers
- Prometheus/Grafana metric analysis of webhook frequency, size, and
handler performance
- Introspection and debugging of webhook payloads
- Redelivery of webhook events for application testing
- Fan-out delivery of webhooks to multiple targets
- HA ingestion endpoint for delivery to less reliable systems
1. **Reliable ingestion** — webhooker is always ready to accept incoming
webhooks. It stores every received event before attempting any
delivery, so nothing is lost if downstream targets are unavailable.
2. **Guaranteed delivery** — Events are queued for delivery to each
configured target. Failed deliveries are retried with configurable
backoff. Every delivery attempt is logged with status codes, response
bodies, and timing.
3. **Observability** — Full request/response logging for every webhook
received and every delivery attempted. Prometheus metrics expose
volume, latency, and error rates. The web UI provides real-time
visibility into event flow.
4. **Fan-out** — A single incoming webhook can be delivered to multiple
targets simultaneously. This enables patterns like forwarding a
GitHub webhook to both a deployment service and a Slack channel.
5. **Replay** — Stored events can be manually redelivered for debugging
or testing, without requiring the original sender to fire the webhook
again.
### Use Cases
- **Store-and-forward** with configurable retries for unreliable
receivers
- **Observability** via Prometheus metrics on webhook frequency, payload
size, and delivery performance
- **Debugging** and introspection of webhook payloads in the web UI
- **Replay** of webhook events for application testing and development
- **Fan-out** delivery of a single webhook to multiple downstream
targets
- **High-availability ingestion** for delivery to less reliable backend
systems
## Design
### Architecture
### Architecture Overview
webhooker uses Uber's fx dependency injection library for managing
application lifecycle. It uses `log/slog` for structured logging, GORM
for database access, and SQLite (via `modernc.org/sqlite`, pure Go, no
CGO) for storage. HTTP routing uses chi.
webhooker is structured as a standard Go HTTP server following the
[sneak/prompts GO_HTTP_SERVER_CONVENTIONS](https://git.eeqj.de/sneak/prompts/src/branch/main/prompts/GO_HTTP_SERVER_CONVENTIONS.md).
It uses:
### Rate Limiting
- **[Uber fx](https://go.uber.org/fx)** for dependency injection and
lifecycle management
- **[go-chi](https://github.com/go-chi/chi)** for HTTP routing
- **[GORM](https://gorm.io)** for database access with
**[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite)** as
the runtime SQLite driver. Note: `gorm.io/driver/sqlite` transitively
depends on `mattn/go-sqlite3`, which requires CGO at build time (see
[Docker](#docker) section)
- **[slog](https://pkg.go.dev/log/slog)** (stdlib) for structured
logging with TTY detection (text for dev, JSON for prod)
- **[gorilla/sessions](https://github.com/gorilla/sessions)** for
encrypted cookie-based session management
- **[Prometheus](https://prometheus.io)** for metrics, served at
`/metrics` behind basic auth
- **[Sentry](https://sentry.io)** for optional error reporting
Global rate limiting middleware (e.g. per-IP throttling applied at the
router level) must **not** apply to webhook receiver endpoints
(`/webhook/{uuid}`). Webhook endpoints receive automated traffic from
external services at unpredictable rates, and blanket rate limits would
cause legitimate webhook deliveries to be dropped.
### Naming Conventions
Instead, each webhook endpoint has its own individually configurable rate
limit, applied within the webhook handler itself. By default, no rate
limit is applied — webhook endpoints accept traffic as fast as it arrives.
Rate limits can be configured on a per-webhook basis in the application
when needed (e.g. to protect against a misbehaving sender).
This README uses the target naming scheme for the application's core
entities. The current codebase uses older names that will be updated in
a future refactor (see
[issue #12](https://git.eeqj.de/sneak/webhooker/issues/12)):
### Database Architecture
| README (target name) | Current code name | Description |
| --------------------- | ----------------- | ----------- |
| **Webhook** | `Processor` | Top-level configuration entity grouping entrypoints and targets |
| **Entrypoint** | `Webhook` | A receiver URL where external services POST events |
| **Target** | `Target` | A delivery destination for events |
webhooker uses separate SQLite database files rather than a single
monolithic database:
- **Main application database** — Stores application configuration and
all standard webapp data: users, sessions, API keys, and global
settings.
- **Per-processor databases** — Each processor (working name — a better
term is needed) gets its own dedicated SQLite database file containing:
input logs, processor logs, and all output queues for that specific
processor.
This separation provides several benefits: processor databases can be
independently backed up, rotated, or archived; a high-volume processor
won't cause lock contention or bloat affecting the main application; and
individual processor data can be cleanly deleted when a processor is
removed.
### Package Layout
All application code lives under `internal/` to prevent external imports.
The main entry point is `cmd/webhooker/main.go`.
- `internal/config` — Configuration management via `pkg/config` (Viper-based)
- `internal/database` — GORM database connection, migrations, and models
- `internal/globals` — Global application metadata (version, build info)
- `internal/handlers` — HTTP handlers using the closure pattern
- `internal/healthcheck` — Health check endpoint logic
- `internal/logger` — Structured logging setup (`log/slog`)
- `internal/middleware` — HTTP middleware (auth, CORS, logging, metrics)
- `internal/server` — HTTP server setup and routing
- `internal/session` — Session management (gorilla/sessions)
- `pkg/config` — Reusable multi-environment configuration library
- `static/` — Embedded static assets (CSS, JS)
- `templates/` — Go HTML templates
Throughout this document, the target names are used. The code rename is
tracked separately.
### Data Model
- **Users** — Service users with username/password (Argon2id hashing)
- **Processors** — Webhook processing units, many-to-one with users
- **Webhooks** — Inbound URL endpoints feeding into processors
- **Targets** — Delivery destinations per processor (HTTP, retry, database, log)
- **Events** — Captured webhook payloads
- **Deliveries** — Pairing of events with targets
- **Delivery Results** — Outcome of each delivery attempt
- **API Keys** — Programmatic access credentials per user
webhooker's data model has eight entities organized into two tiers: the
**application tier** (user and webhook configuration) and the **event
tier** (event ingestion, delivery, and logging).
```
┌─────────────────────────────────────────────────────────────┐
│ APPLICATION TIER │
│ (main application database) │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────────┐ │
│ │ User │──1:N──│ Webhook │──1:N──│ Entrypoint │ │
│ │ │ │ │ │ │ │
│ │ │ │ │──1:N──│ Target │ │
│ │ │ └──────────┘ └──────────────┘ │
│ │ │──1:N──│ APIKey │ │
│ └──────────┘ └──────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ EVENT TIER │
│ (planned: per-webhook dedicated database) │
│ │
│ ┌──────────┐ ┌──────────┐ ┌─────────────────┐ │
│ │ Event │──1:N──│ Delivery │──1:N──│ DeliveryResult │ │
│ └──────────┘ └──────────┘ └─────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
#### User
A registered user of the webhooker service.
| Field | Type | Description |
| ---------- | -------- | ----------- |
| `id` | UUID | Primary key |
| `username` | string | Unique login name |
| `password` | string | Argon2id hash (never exposed via API) |
**Relations:** Has many Webhooks. Has many APIKeys.
Passwords are hashed with Argon2id using secure defaults (64 MB memory,
1 iteration, 4 threads, 32-byte key, 16-byte salt). On first startup,
an `admin` user is created with a randomly generated 16-character
password logged to stdout.
#### Webhook
The top-level configuration entity (currently called "Processor" in
code). A webhook groups together one or more entrypoints (receiver URLs)
and one or more targets (delivery destinations) into a logical unit. A
user creates a webhook to set up event routing.
| Field | Type | Description |
| ---------------- | ------- | ----------- |
| `id` | UUID | Primary key |
| `user_id` | UUID | Foreign key → User |
| `name` | string | Human-readable name |
| `description` | string | Optional description |
| `retention_days` | integer | Days to retain events (default: 30) |
**Relations:** Belongs to User. Has many Entrypoints. Has many Targets.
The `retention_days` field controls how long event data is kept in the
webhook's dedicated database before automatic cleanup.
#### Entrypoint
A receiver URL where external services POST webhook events (currently
called "Webhook" in code). Each entrypoint has a unique UUID-based path.
When an HTTP request arrives at an entrypoint's path, webhooker captures
the full request and creates an Event.
| Field | Type | Description |
| -------------- | ------- | ----------- |
| `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook |
| `path` | string | Unique URL path (UUID-based, e.g. `/webhook/{uuid}`) |
| `description` | string | Optional description |
| `active` | boolean | Whether this entrypoint accepts events (default: true) |
**Relations:** Belongs to Webhook.
A webhook can have multiple entrypoints. This allows separate URLs for
different event sources that all feed into the same processing pipeline
(e.g., one entrypoint for GitHub, another for Stripe, both routing to
the same targets).
#### Target
A delivery destination for events. Each target defines where and how
events should be forwarded.
| Field | Type | Description |
| ---------------- | ---------- | ----------- |
| `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook |
| `name` | string | Human-readable name |
| `type` | TargetType | One of: `http`, `retry`, `database`, `log` |
| `active` | boolean | Whether deliveries are enabled (default: true) |
| `config` | JSON text | Type-specific configuration |
| `max_retries` | integer | Maximum retry attempts (for retry targets) |
| `max_queue_size` | integer | Maximum queued deliveries (for retry targets) |
**Relations:** Belongs to Webhook. Has many Deliveries.
**Target types:**
- **`http`** — Forward the event as an HTTP POST to a configured URL.
Fire-and-forget: a single attempt with no retries.
- **`retry`** — Forward the event via HTTP POST with automatic retry on
failure. Uses exponential backoff up to `max_retries` attempts.
- **`database`** — Store the event in the webhook's database only (no
external delivery). Useful for pure logging/archival.
- **`log`** — Write the event to the application log (stdout). Useful
for debugging.
The `config` field stores type-specific configuration as JSON (e.g.,
destination URL, custom headers, timeout settings).
#### APIKey
A programmatic access credential for API authentication.
| Field | Type | Description |
| -------------- | --------- | ----------- |
| `id` | UUID | Primary key |
| `user_id` | UUID | Foreign key → User |
| `key` | string | Unique API key value |
| `description` | string | Optional description |
| `last_used_at` | timestamp | Last time this key was used (nullable) |
**Relations:** Belongs to User.
#### Event
A captured incoming webhook request. Stores the complete HTTP request
data for replay and auditing.
| Field | Type | Description |
| -------------- | ------ | ----------- |
| `id` | UUID | Primary key |
| `processor_id` | UUID | Foreign key → Webhook |
| `webhook_id` | UUID | Foreign key → Entrypoint |
| `method` | string | HTTP method (POST, PUT, etc.) |
| `headers` | JSON | Complete request headers |
| `body` | text | Raw request body |
| `content_type` | string | Content-Type header value |
**Relations:** Belongs to Webhook. Belongs to Entrypoint. Has many
Deliveries.
When a request arrives at an entrypoint, the full request (method,
headers, body) is captured as an Event. The event is then queued for
delivery to every active target configured on the parent webhook.
#### Delivery
The pairing of an event with a target. Tracks the overall delivery
status across potentially multiple attempts.
| Field | Type | Description |
| ---------- | -------------- | ----------- |
| `id` | UUID | Primary key |
| `event_id` | UUID | Foreign key → Event |
| `target_id`| UUID | Foreign key → Target |
| `status` | DeliveryStatus | One of: `pending`, `delivered`, `failed`, `retrying` |
**Relations:** Belongs to Event. Belongs to Target. Has many
DeliveryResults.
**Delivery statuses:**
- **`pending`** — Created but not yet attempted.
- **`retrying`** — At least one attempt failed; more attempts remain.
- **`delivered`** — Successfully delivered (at least one attempt
succeeded).
- **`failed`** — All retry attempts exhausted without success.
#### DeliveryResult
The result of a single delivery attempt. Every attempt (including
retries) is individually logged for full observability.
| Field | Type | Description |
| --------------- | ------- | ----------- |
| `id` | UUID | Primary key |
| `delivery_id` | UUID | Foreign key → Delivery |
| `attempt_num` | integer | Attempt number (1-based) |
| `success` | boolean | Whether this attempt succeeded |
| `status_code` | integer | HTTP response status code (if applicable) |
| `response_body` | text | Response body (if applicable) |
| `error` | string | Error message (on failure) |
| `duration` | integer | Request duration in milliseconds |
**Relations:** Belongs to Delivery.
#### Common Fields
All entities include these fields from `BaseModel`:
| Field | Type | Description |
| ------------ | --------- | ----------- |
| `id` | UUID | Auto-generated UUIDv4 primary key |
| `created_at` | timestamp | Record creation time |
| `updated_at` | timestamp | Last modification time |
| `deleted_at` | timestamp | Soft-delete timestamp (nullable; GORM soft deletes) |
### Database Architecture
#### Current Implementation
webhooker currently uses a **single SQLite database** for all data —
application configuration, user accounts, and (once implemented) event
storage. The database connection is managed by GORM with a single
connection string configured via `DBURL`. On first startup the database
is auto-migrated and an `admin` user is created.
#### Planned: Per-Webhook Event Databases (Phase 2)
In a future phase (see TODO Phase 2 below), webhooker will split into
**separate SQLite database files**: a main application database for
configuration data and per-webhook databases for event storage.
**Main Application Database** — will store:
- **Users** — accounts and Argon2id password hashes
- **Webhooks** (Processors) — webhook configurations
- **Entrypoints** (Webhooks) — receiver URL definitions
- **Targets** — delivery destination configurations
- **APIKeys** — programmatic access credentials
**Per-Webhook Event Databases** — each webhook will get its own
dedicated SQLite file containing:
- **Events** — captured incoming webhook payloads
- **Deliveries** — event-to-target pairings and their status
- **DeliveryResults** — individual delivery attempt logs
This planned separation will provide:
- **Isolation** — a high-volume webhook won't cause lock contention or
WAL bloat affecting the main application or other webhooks.
- **Independent lifecycle** — event databases can be independently
backed up, archived, rotated, or size-limited without impacting the
application.
- **Clean deletion** — removing a webhook and all its history is as
simple as deleting one file.
- **Per-webhook retention** — the `retention_days` field on each webhook
will control automatic cleanup of old events in that webhook's
database only.
- **Performance** — each webhook's database will have its own WAL, its
own page cache, and its own lock, so concurrent event ingestion across
webhooks won't contend.
The database uses the
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at
runtime, though CGO is required at build time due to the transitive
`mattn/go-sqlite3` dependency from `gorm.io/driver/sqlite`.
### Request Flow
```
External Service
│
│ POST /webhook/{uuid}
▼
┌─────────────┐ ┌──────────────┐ ┌──────────────┐
│ chi Router │────►│ Middleware │────►│ Webhook │
│ │ │ Stack │ │ Handler │
└─────────────┘ └──────────────┘ └──────┬───────┘
│
1. Look up Entrypoint by UUID
2. Capture full request as Event
3. Queue Delivery to each active Target
│
▼
┌──────────────┐
│ Delivery │
│ Engine │
└──────┬───────┘
│
┌────────────────────┼────────────────────┐
▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐
│ HTTP Target│ │Retry Target│ │ Log Target │
│ (1 attempt)│ │ (backoff) │ │ (stdout) │
└────────────┘ └────────────┘ └────────────┘
```
### Rate Limiting
Global rate limiting middleware (e.g., per-IP throttling applied at the
router level) **must not** apply to webhook receiver endpoints. Webhook
endpoints receive automated traffic from external services at
unpredictable rates, and blanket rate limits would cause legitimate
deliveries to be dropped.
Instead, each webhook has its own individually configurable rate limit,
applied within the webhook handler itself. By default, no rate limit is
applied — webhook endpoints accept traffic as fast as it arrives. Rate
limits can be configured per-webhook when needed (e.g., to protect
against a misbehaving sender).
### API Endpoints
- `GET /` — Web UI index page
- `GET /.well-known/healthcheck` — Health check with uptime, version
- `GET /s/*` — Static file serving (CSS, JS)
- `GET /metrics` — Prometheus metrics (requires basic auth)
- `POST /webhook/{uuid}` — Webhook receiver endpoint
- `/pages/login`, `/pages/logout` — Authentication
- `/user/{username}` — User profile
- `/sources/*` — Webhook source management
#### Public Endpoints
| Method | Path | Description |
| ------ | --------------------------- | ----------- |
| `GET` | `/` | Web UI index page (server-rendered) |
| `GET` | `/.well-known/healthcheck` | Health check (JSON: status, uptime, version) |
| `GET` | `/s/*` | Static file serving (embedded CSS, JS) |
| `ANY` | `/webhook/{uuid}` | Webhook receiver endpoint (accepts all methods) |
#### Authentication Endpoints
| Method | Path | Description |
| ------ | --------------- | ----------- |
| `GET` | `/pages/login` | Login page |
| `POST` | `/pages/login` | Login form submission |
| `POST` | `/pages/logout` | Logout (destroys session) |
#### Authenticated Endpoints
| Method | Path | Description |
| ------ | ------------------------ | ----------- |
| `GET` | `/user/{username}` | User profile page |
| `GET` | `/sources` | List user's webhooks |
| `GET` | `/sources/new` | Create webhook form |
| `POST` | `/sources/new` | Create webhook submission |
| `GET` | `/source/{id}` | Webhook detail view |
| `GET` | `/source/{id}/edit` | Edit webhook form |
| `POST` | `/source/{id}/edit` | Edit webhook submission |
| `POST` | `/source/{id}/delete` | Delete webhook |
| `GET` | `/source/{id}/logs` | Webhook event logs |
#### Infrastructure Endpoints
| Method | Path | Description |
| ------ | ---------- | ----------- |
| `GET` | `/metrics` | Prometheus metrics (requires basic auth) |
#### API (Planned)
| Method | Path | Description |
| -------- | ------------------------------ | ----------- |
| `GET` | `/api/v1/webhooks` | List webhooks |
| `POST` | `/api/v1/webhooks` | Create webhook |
| `GET` | `/api/v1/webhooks/{id}` | Get webhook details |
| `PUT` | `/api/v1/webhooks/{id}` | Update webhook |
| `DELETE` | `/api/v1/webhooks/{id}` | Delete webhook |
| `GET` | `/api/v1/webhooks/{id}/events` | List events for webhook |
| `POST` | `/api/v1/events/{id}/redeliver`| Redeliver an event |
API authentication will use API keys passed via `Authorization: Bearer
<key>` header.
### Package Layout
All application code lives under `internal/` to prevent external
imports. The entry point is `cmd/webhooker/main.go`.
```
webhooker/
├── cmd/webhooker/
│ └── main.go # Entry point: sets globals, wires fx
├── internal/
│ ├── config/
│ │ └── config.go # Configuration loading via pkg/config
│ ├── database/
│ │ ├── base_model.go # BaseModel with UUID primary keys
│ │ ├── database.go # GORM connection, migrations, admin seed
│ │ ├── models.go # AutoMigrate for all models
│ │ ├── model_user.go # User entity
│ │ ├── model_processor.go # Webhook entity (to be renamed)
│ │ ├── model_webhook.go # Entrypoint entity (to be renamed)
│ │ ├── model_target.go # Target entity and TargetType enum
│ │ ├── model_event.go # Event entity
│ │ ├── model_delivery.go # Delivery entity and DeliveryStatus enum
│ │ ├── model_delivery_result.go # DeliveryResult entity
│ │ ├── model_apikey.go # APIKey entity
│ │ └── password.go # Argon2id hashing and verification
│ ├── globals/
│ │ └── globals.go # Build-time variables (appname, version, arch)
│ ├── handlers/
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
│ │ ├── auth.go # Login, logout handlers
│ │ ├── healthcheck.go # Health check handler
│ │ ├── index.go # Index page handler
│ │ ├── profile.go # User profile handler
│ │ ├── source_management.go # Webhook CRUD handlers (stubs)
│ │ └── webhook.go # Webhook receiver handler
│ ├── healthcheck/
│ │ └── healthcheck.go # Health check service (uptime, version)
│ ├── logger/
│ │ └── logger.go # slog setup with TTY detection
│ ├── middleware/
│ │ └── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth
│ ├── server/
│ │ ├── server.go # Server struct, fx lifecycle, signal handling
│ │ ├── http.go # HTTP server setup with timeouts
│ │ └── routes.go # All route definitions
│ └── session/
│ └── session.go # Cookie-based session management
├── pkg/config/ # Reusable multi-environment config library
├── static/
│ ├── static.go # //go:embed directive
│ ├── css/style.css # Custom stylesheet (system font stack, card effects, layout)
│ └── js/app.js # Client-side JavaScript (minimal bootstrap)
├── templates/ # Go HTML templates (base, index, login, etc.)
├── configs/
│ └── config.yaml.example # Example configuration file
├── Dockerfile # Multi-stage: build + check, then Alpine runtime
├── Makefile # fmt, lint, test, check, build, docker targets
├── go.mod / go.sum
└── .golangci.yml # Linter configuration
```
### Dependency Injection
Components are wired via Uber fx in this order:
1. `globals.New` — Build-time variables (appname, version, arch)
2. `logger.New` — Structured logging (slog with TTY detection)
3. `config.New` — Configuration loading (pkg/config + environment)
4. `database.New` — SQLite connection, migrations, admin user seed
5. `healthcheck.New` — Health check service
6. `session.New` — Cookie-based session manager
7. `handlers.New` — HTTP handlers
8. `middleware.New` — HTTP middleware
9. `server.New` — HTTP server and router
The server starts via `fx.Invoke(func(*server.Server) {})` which
triggers the fx lifecycle hooks in dependency order.
### Middleware Stack
Applied to all routes in this order:
1. **Recoverer** — Panic recovery (chi built-in)
2. **RequestID** — Generate unique request IDs (chi built-in)
3. **Logging** — Structured request logging (method, URL, status,
latency, remote IP, user agent, request ID)
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
5. **CORS** — Cross-origin resource sharing headers
6. **Timeout** — 60-second request timeout
7. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set;
configured with `Repanic: true` so panics still reach Recoverer)
### Authentication
- **Web UI:** Cookie-based sessions using gorilla/sessions with
encrypted cookies. Sessions are configured with HttpOnly, SameSite
Lax, and Secure (in production). Session lifetime is 7 days.
- **API (planned):** API key authentication via `Authorization: Bearer`
header. API keys are stored per-user with usage tracking
(`last_used_at`).
- **Metrics:** Basic authentication protecting the `/metrics` endpoint.
### Security
- Passwords hashed with Argon2id (64 MB memory cost)
- Session cookies are HttpOnly, SameSite Lax, Secure (prod only)
- Session key must be a 32-byte base64-encoded value
- Prometheus metrics behind basic auth
- Static assets embedded in binary (no filesystem access needed at
runtime)
- Container runs as non-root user (UID 1000)
- GORM soft deletes on all entities (data preserved for audit)
### Docker
The Dockerfile uses a multi-stage build:
1. **Builder stage** (Debian-based `golang:1.24`) — installs
golangci-lint, downloads dependencies, copies source, runs `make
check` (format verification, linting, tests, compilation).
2. **Runtime stage** (`alpine:3.21`) — copies the binary, runs as
non-root user, exposes port 8080, includes a health check.
The builder uses Debian rather than Alpine because GORM's SQLite
dialect pulls in CGO-dependent headers at compile time. The runtime
binary is statically linked and runs on Alpine.
`docker build .` is the CI gate — if it passes, the code is formatted,
linted, tested, and compiled.
## TODO
### Phase 1: Security & Infrastructure
- [ ] Security headers (HSTS, CSP, X-Frame-Options)
- [ ] Rate limiting middleware
- [ ] CSRF protection for forms
- [ ] Request ID tracking through entire lifecycle
### Phase 2: Authentication & Authorization
- [ ] Authentication middleware for protected routes
- [ ] Session expiration and "remember me"
- [ ] Password reset flow
- [ ] API key authentication for programmatic access
### Phase 3: Core Webhook Features
- [ ] Webhook reception and event storage at `/webhook/{uuid}`
- [ ] Event processing and target delivery engine
- [ ] HTTP target type (fire-and-forget POST)
- [ ] Retry target type (exponential backoff)
- [ ] Database target type (store only)
- [ ] Log target type (console output)
### Phase 1: Core Webhook Engine
- [ ] Implement webhook reception and event storage at `/webhook/{uuid}`
- [ ] Build event processing and target delivery engine
- [ ] Implement HTTP target type (fire-and-forget POST)
- [ ] Implement retry target type (exponential backoff)
- [ ] Implement database target type (store only)
- [ ] Implement log target type (console output)
- [ ] Per-webhook rate limiting in the receiver handler
- [ ] Webhook signature verification (GitHub, Stripe formats)
### Phase 2: Database Separation
- [ ] Split into main application DB + per-webhook event DBs
- [ ] Automatic event retention cleanup based on `retention_days`
- [ ] Per-webhook database lifecycle management (create on webhook
creation, delete on webhook removal)
### Phase 3: Security & Infrastructure
- [ ] Implement authentication middleware for protected routes
([#9](https://git.eeqj.de/sneak/webhooker/issues/9))
- [ ] Security headers (HSTS, CSP, X-Frame-Options)
- [ ] CSRF protection for forms
- [ ] Session expiration and "remember me"
- [ ] Password change/reset flow
- [ ] API key authentication for programmatic access
### Phase 4: Web UI
- [ ] Webhook source management pages (list, create, edit, delete)
- [ ] Webhook management pages (list, create, edit, delete)
- [ ] Webhook request log viewer with filtering
- [ ] Delivery status and retry management UI
- [ ] Manual event redelivery
- [ ] Analytics dashboard (success rates, response times)
- [ ] Replace Bootstrap with Tailwind CSS + Alpine.js
([#4](https://git.eeqj.de/sneak/webhooker/issues/4))
### Phase 5: API
- [ ] RESTful CRUD for processors, webhooks, targets
### Phase 5: REST API
- [ ] RESTful CRUD for webhooks, entrypoints, targets
- [ ] Event viewing and filtering endpoints
- [ ] API documentation (OpenAPI)
- [ ] Event redelivery endpoint
- [ ] OpenAPI specification
### Phase 6: Code Quality
- [ ] Rename Processor → Webhook, Webhook → Entrypoint in code
([#12](https://git.eeqj.de/sneak/webhooker/issues/12))
- [ ] Embed templates via `//go:embed`
([#7](https://git.eeqj.de/sneak/webhooker/issues/7))
- [ ] Use `slog.LevelVar` for dynamic log level switching
([#8](https://git.eeqj.de/sneak/webhooker/issues/8))
- [ ] Simplify configuration to prefer environment variables
([#10](https://git.eeqj.de/sneak/webhooker/issues/10))
- [ ] Remove redundant `godotenv/autoload` import
([#11](https://git.eeqj.de/sneak/webhooker/issues/11))
### Future
- [ ] Email source and delivery target types
- [ ] Email delivery target type
- [ ] SNS, S3, Slack delivery targets
- [ ] Data transformations (e.g. webhook-to-Slack message)
- [ ] Data transformations (e.g., webhook-to-Slack message formatting)
- [ ] JSONL file delivery with periodic S3 upload
- [ ] Webhook event search and filtering
- [ ] Multi-user with role-based access
## License
Pending — to be determined by the author (MIT, GPL, or WTFPL).
MIT
## Author