webhooker

webhooker is a self-hosted webhook proxy and store-and-forward service written in Go by @sneak. It receives webhooks from external services, durably stores them, and delivers them to configured targets with retry support, logging, and observability. Category: infrastructure / web service. License: MIT.

Getting Started

Prerequisites

• Go 1.24+
• golangci-lint v1.64+
• Docker (for containerized deployment)

Quick Start

# Clone the repo
git clone https://git.eeqj.de/sneak/webhooker.git
cd webhooker

# Install Go dependencies
make deps

# Run all checks (format, lint, test, build)
make check

# Run in development mode (uses SQLite in current directory)
make dev

# Build Docker image
make docker

Development Commands

make fmt        # Format code (gofmt + goimports)
make lint       # Run golangci-lint
make test       # Run tests with race detection
make check      # fmt-check + lint + test + build (CI gate)
make build      # Build binary to bin/webhooker
make dev        # go run ./cmd/webhooker
make docker     # Build Docker image
make hooks      # Install git pre-commit hook that runs make check

Configuration

webhooker uses a YAML configuration file with environment-specific overrides, loaded via the pkg/config library (Viper-based). The environment is selected by setting WEBHOOKER_ENVIRONMENT to dev or prod (default: dev).

Configuration is resolved in this order (highest priority first):

1. Environment variables
2. .env file (loaded via godotenv/autoload)
3. Config file values for the active environment
4. Config file defaults

Variable	Description	Default
WEBHOOKER_ENVIRONMENT	dev or prod	dev
PORT	HTTP listen port	8080
DBURL	SQLite database connection string	(required)
SESSION_KEY	Base64-encoded 32-byte session key	(required in prod)
DEBUG	Enable debug logging	false
METRICS_USERNAME	Basic auth username for /metrics	""
METRICS_PASSWORD	Basic auth password for /metrics	""
SENTRY_DSN	Sentry error reporting DSN	""

On first startup in development mode, webhooker creates an admin user with a randomly generated password and logs it to stdout. This password is only displayed once.

Running with Docker

docker run -d \
  -p 8080:8080 \
  -v /path/to/data:/data \
  -e DBURL="file:/data/webhooker.db?cache=shared&mode=rwc" \
  -e SESSION_KEY="<base64-encoded-32-byte-key>" \
  -e WEBHOOKER_ENVIRONMENT=prod \
  webhooker:latest

The container runs as a non-root user (webhooker, UID 1000), exposes port 8080, and includes a health check against /.well-known/healthcheck.

Rationale

Webhook integrations between services are inherently fragile. The receiving service must be online when the webhook fires, most webhook senders provide no built-in retry mechanism, and there is no standard way to inspect what was sent, when it was sent, or whether delivery succeeded.

webhooker solves this by acting as a durable intermediary:

1. Reliable ingestion — webhooker is always ready to accept incoming webhooks. It stores every received event before attempting any delivery, so nothing is lost if downstream targets are unavailable.

2. Guaranteed delivery — Events are queued for delivery to each configured target. Failed deliveries are retried with configurable backoff. Every delivery attempt is logged with status codes, response bodies, and timing.

3. Observability — Full request/response logging for every webhook received and every delivery attempted. Prometheus metrics expose volume, latency, and error rates. The web UI provides real-time visibility into event flow.

4. Fan-out — A single incoming webhook can be delivered to multiple targets simultaneously. This enables patterns like forwarding a GitHub webhook to both a deployment service and a Slack channel.

5. Replay — Stored events can be manually redelivered for debugging or testing, without requiring the original sender to fire the webhook again.

Use Cases

• Store-and-forward with configurable retries for unreliable receivers
• Observability via Prometheus metrics on webhook frequency, payload size, and delivery performance
• Debugging and introspection of webhook payloads in the web UI
• Replay of webhook events for application testing and development
• Fan-out delivery of a single webhook to multiple downstream targets
• High-availability ingestion for delivery to less reliable backend systems

Design

Architecture Overview

webhooker is structured as a standard Go HTTP server following the sneak/prompts GO_HTTP_SERVER_CONVENTIONS. It uses:

• Uber fx for dependency injection and lifecycle management
• go-chi for HTTP routing
• GORM for database access with modernc.org/sqlite as the runtime SQLite driver. Note: gorm.io/driver/sqlite transitively depends on mattn/go-sqlite3, which requires CGO at build time (see Docker section)
• slog (stdlib) for structured logging with TTY detection (text for dev, JSON for prod)
• gorilla/sessions for encrypted cookie-based session management
• Prometheus for metrics, served at /metrics behind basic auth
• Sentry for optional error reporting

Naming Conventions

This README uses the target naming scheme for the application's core entities. The current codebase uses older names that will be updated in a future refactor (see issue #12):

README (target name)	Current code name	Description
Webhook	Processor	Top-level configuration entity grouping entrypoints and targets
Entrypoint	Webhook	A receiver URL where external services POST events
Target	Target	A delivery destination for events

Throughout this document, the target names are used. The code rename is tracked separately.

Data Model

webhooker's data model has eight entities organized into two tiers: the application tier (user and webhook configuration) and the event tier (event ingestion, delivery, and logging).

┌─────────────────────────────────────────────────────────────┐
│                    APPLICATION TIER                          │
│                  (main application database)                 │
│                                                             │
│  ┌──────────┐       ┌──────────┐       ┌──────────────┐     │
│  │   User   │──1:N──│  Webhook │──1:N──│  Entrypoint  │     │
│  │          │       │          │       │              │     │
│  │          │       │          │──1:N──│   Target     │     │
│  │          │       └──────────┘       └──────────────┘     │
│  │          │──1:N──│  APIKey  │                             │
│  └──────────┘       └──────────┘                             │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│                      EVENT TIER                              │
│        (planned: per-webhook dedicated database)            │
│                                                             │
│  ┌──────────┐       ┌──────────┐       ┌─────────────────┐  │
│  │  Event   │──1:N──│ Delivery │──1:N──│ DeliveryResult  │  │
│  └──────────┘       └──────────┘       └─────────────────┘  │
└─────────────────────────────────────────────────────────────┘

User

A registered user of the webhooker service.

Field	Type	Description
id	UUID	Primary key
username	string	Unique login name
password	string	Argon2id hash (never exposed via API)

Relations: Has many Webhooks. Has many APIKeys.

Passwords are hashed with Argon2id using secure defaults (64 MB memory, 1 iteration, 4 threads, 32-byte key, 16-byte salt). On first startup, an admin user is created with a randomly generated 16-character password logged to stdout.

Webhook

The top-level configuration entity (currently called "Processor" in code). A webhook groups together one or more entrypoints (receiver URLs) and one or more targets (delivery destinations) into a logical unit. A user creates a webhook to set up event routing.

Field	Type	Description
id	UUID	Primary key
user_id	UUID	Foreign key → User
name	string	Human-readable name
description	string	Optional description
retention_days	integer	Days to retain events (default: 30)

Relations: Belongs to User. Has many Entrypoints. Has many Targets.

The retention_days field controls how long event data is kept in the webhook's dedicated database before automatic cleanup.

Entrypoint

A receiver URL where external services POST webhook events (currently called "Webhook" in code). Each entrypoint has a unique UUID-based path. When an HTTP request arrives at an entrypoint's path, webhooker captures the full request and creates an Event.

Field	Type	Description
id	UUID	Primary key
processor_id	UUID	Foreign key → Webhook
path	string	Unique URL path (UUID-based, e.g. /webhook/{uuid})
description	string	Optional description
active	boolean	Whether this entrypoint accepts events (default: true)

Relations: Belongs to Webhook.

A webhook can have multiple entrypoints. This allows separate URLs for different event sources that all feed into the same processing pipeline (e.g., one entrypoint for GitHub, another for Stripe, both routing to the same targets).

Target

A delivery destination for events. Each target defines where and how events should be forwarded.

Field	Type	Description
id	UUID	Primary key
processor_id	UUID	Foreign key → Webhook
name	string	Human-readable name
type	TargetType	One of: http, retry, database, log
active	boolean	Whether deliveries are enabled (default: true)
config	JSON text	Type-specific configuration
max_retries	integer	Maximum retry attempts (for retry targets)
max_queue_size	integer	Maximum queued deliveries (for retry targets)

Relations: Belongs to Webhook. Has many Deliveries.

Target types:

• http — Forward the event as an HTTP POST to a configured URL. Fire-and-forget: a single attempt with no retries.
• retry — Forward the event via HTTP POST with automatic retry on failure. Uses exponential backoff up to max_retries attempts.
• database — Store the event in the webhook's database only (no external delivery). Useful for pure logging/archival.
• log — Write the event to the application log (stdout). Useful for debugging.

The config field stores type-specific configuration as JSON (e.g., destination URL, custom headers, timeout settings).

APIKey

A programmatic access credential for API authentication.

Field	Type	Description
id	UUID	Primary key
user_id	UUID	Foreign key → User
key	string	Unique API key value
description	string	Optional description
last_used_at	timestamp	Last time this key was used (nullable)

Relations: Belongs to User.

Event

A captured incoming webhook request. Stores the complete HTTP request data for replay and auditing.

Field	Type	Description
id	UUID	Primary key
processor_id	UUID	Foreign key → Webhook
webhook_id	UUID	Foreign key → Entrypoint
method	string	HTTP method (POST, PUT, etc.)
headers	JSON	Complete request headers
body	text	Raw request body
content_type	string	Content-Type header value

Relations: Belongs to Webhook. Belongs to Entrypoint. Has many Deliveries.

When a request arrives at an entrypoint, the full request (method, headers, body) is captured as an Event. The event is then queued for delivery to every active target configured on the parent webhook.

Delivery

The pairing of an event with a target. Tracks the overall delivery status across potentially multiple attempts.

Field	Type	Description
id	UUID	Primary key
event_id	UUID	Foreign key → Event
target_id	UUID	Foreign key → Target
status	DeliveryStatus	One of: pending, delivered, failed, retrying

Relations: Belongs to Event. Belongs to Target. Has many DeliveryResults.

Delivery statuses:

• pending — Created but not yet attempted.
• retrying — At least one attempt failed; more attempts remain.
• delivered — Successfully delivered (at least one attempt succeeded).
• failed — All retry attempts exhausted without success.

DeliveryResult

The result of a single delivery attempt. Every attempt (including retries) is individually logged for full observability.

Field	Type	Description
id	UUID	Primary key
delivery_id	UUID	Foreign key → Delivery
attempt_num	integer	Attempt number (1-based)
success	boolean	Whether this attempt succeeded
status_code	integer	HTTP response status code (if applicable)
response_body	text	Response body (if applicable)
error	string	Error message (on failure)
duration	integer	Request duration in milliseconds

Relations: Belongs to Delivery.

Common Fields

All entities include these fields from BaseModel:

Field	Type	Description
id	UUID	Auto-generated UUIDv4 primary key
created_at	timestamp	Record creation time
updated_at	timestamp	Last modification time
deleted_at	timestamp	Soft-delete timestamp (nullable; GORM soft deletes)

Database Architecture

Current Implementation

webhooker currently uses a single SQLite database for all data — application configuration, user accounts, and (once implemented) event storage. The database connection is managed by GORM with a single connection string configured via DBURL. On first startup the database is auto-migrated and an admin user is created.

Planned: Per-Webhook Event Databases (Phase 2)

In a future phase (see TODO Phase 2 below), webhooker will split into separate SQLite database files: a main application database for configuration data and per-webhook databases for event storage.

Main Application Database — will store:

• Users — accounts and Argon2id password hashes
• Webhooks (Processors) — webhook configurations
• Entrypoints (Webhooks) — receiver URL definitions
• Targets — delivery destination configurations
• APIKeys — programmatic access credentials

Per-Webhook Event Databases — each webhook will get its own dedicated SQLite file containing:

• Events — captured incoming webhook payloads
• Deliveries — event-to-target pairings and their status
• DeliveryResults — individual delivery attempt logs

This planned separation will provide:

• Isolation — a high-volume webhook won't cause lock contention or WAL bloat affecting the main application or other webhooks.
• Independent lifecycle — event databases can be independently backed up, archived, rotated, or size-limited without impacting the application.
• Clean deletion — removing a webhook and all its history is as simple as deleting one file.
• Per-webhook retention — the retention_days field on each webhook will control automatic cleanup of old events in that webhook's database only.
• Performance — each webhook's database will have its own WAL, its own page cache, and its own lock, so concurrent event ingestion across webhooks won't contend.

The database uses the modernc.org/sqlite driver at runtime, though CGO is required at build time due to the transitive mattn/go-sqlite3 dependency from gorm.io/driver/sqlite.

Request Flow

External Service
       │
       │  POST /webhook/{uuid}
       ▼
┌─────────────┐     ┌──────────────┐     ┌──────────────┐
│  chi Router │────►│  Middleware   │────►│   Webhook    │
│             │     │  Stack       │     │   Handler    │
└─────────────┘     └──────────────┘     └──────┬───────┘
                                                │
                     1. Look up Entrypoint by UUID
                     2. Capture full request as Event
                     3. Queue Delivery to each active Target
                                                │
                                                ▼
                                        ┌──────────────┐
                                        │  Delivery    │
                                        │  Engine      │
                                        └──────┬───────┘
                                               │
                          ┌────────────────────┼────────────────────┐
                          ▼                    ▼                    ▼
                   ┌────────────┐       ┌────────────┐      ┌────────────┐
                   │ HTTP Target│       │Retry Target│      │ Log Target │
                   │ (1 attempt)│       │ (backoff)  │      │ (stdout)   │
                   └────────────┘       └────────────┘      └────────────┘

Rate Limiting

Global rate limiting middleware (e.g., per-IP throttling applied at the router level) must not apply to webhook receiver endpoints. Webhook endpoints receive automated traffic from external services at unpredictable rates, and blanket rate limits would cause legitimate deliveries to be dropped.

Instead, each webhook has its own individually configurable rate limit, applied within the webhook handler itself. By default, no rate limit is applied — webhook endpoints accept traffic as fast as it arrives. Rate limits can be configured per-webhook when needed (e.g., to protect against a misbehaving sender).

API Endpoints

Public Endpoints

Method	Path	Description
GET	/	Web UI index page (server-rendered)
GET	/.well-known/healthcheck	Health check (JSON: status, uptime, version)
GET	/s/*	Static file serving (embedded CSS, JS)
ANY	/webhook/{uuid}	Webhook receiver endpoint (accepts all methods)

Authentication Endpoints

Method	Path	Description
GET	/pages/login	Login page
POST	/pages/login	Login form submission
POST	/pages/logout	Logout (destroys session)

Authenticated Endpoints

Method	Path	Description
GET	/user/{username}	User profile page
GET	/sources	List user's webhooks
GET	/sources/new	Create webhook form
POST	/sources/new	Create webhook submission
GET	/source/{id}	Webhook detail view
GET	/source/{id}/edit	Edit webhook form
POST	/source/{id}/edit	Edit webhook submission
POST	/source/{id}/delete	Delete webhook
GET	/source/{id}/logs	Webhook event logs

Infrastructure Endpoints

Method	Path	Description
GET	/metrics	Prometheus metrics (requires basic auth)

API (Planned)

Method	Path	Description
GET	/api/v1/webhooks	List webhooks
POST	/api/v1/webhooks	Create webhook
GET	/api/v1/webhooks/{id}	Get webhook details
PUT	/api/v1/webhooks/{id}	Update webhook
DELETE	/api/v1/webhooks/{id}	Delete webhook
GET	/api/v1/webhooks/{id}/events	List events for webhook
POST	/api/v1/events/{id}/redeliver	Redeliver an event

API authentication will use API keys passed via Authorization: Bearer <key> header.

Package Layout

All application code lives under internal/ to prevent external imports. The entry point is cmd/webhooker/main.go.

webhooker/
├── cmd/webhooker/
│   └── main.go               # Entry point: sets globals, wires fx
├── internal/
│   ├── config/
│   │   └── config.go          # Configuration loading via pkg/config
│   ├── database/
│   │   ├── base_model.go      # BaseModel with UUID primary keys
│   │   ├── database.go        # GORM connection, migrations, admin seed
│   │   ├── models.go          # AutoMigrate for all models
│   │   ├── model_user.go      # User entity
│   │   ├── model_processor.go # Webhook entity (to be renamed)
│   │   ├── model_webhook.go   # Entrypoint entity (to be renamed)
│   │   ├── model_target.go    # Target entity and TargetType enum
│   │   ├── model_event.go     # Event entity
│   │   ├── model_delivery.go  # Delivery entity and DeliveryStatus enum
│   │   ├── model_delivery_result.go  # DeliveryResult entity
│   │   ├── model_apikey.go    # APIKey entity
│   │   └── password.go        # Argon2id hashing and verification
│   ├── globals/
│   │   └── globals.go         # Build-time variables (appname, version, arch)
│   ├── handlers/
│   │   ├── handlers.go        # Base handler struct, JSON helpers, template rendering
│   │   ├── auth.go            # Login, logout handlers
│   │   ├── healthcheck.go     # Health check handler
│   │   ├── index.go           # Index page handler
│   │   ├── profile.go         # User profile handler
│   │   ├── source_management.go  # Webhook CRUD handlers (stubs)
│   │   └── webhook.go         # Webhook receiver handler
│   ├── healthcheck/
│   │   └── healthcheck.go     # Health check service (uptime, version)
│   ├── logger/
│   │   └── logger.go          # slog setup with TTY detection
│   ├── middleware/
│   │   └── middleware.go      # Logging, CORS, Auth, Metrics, MetricsAuth
│   ├── server/
│   │   ├── server.go          # Server struct, fx lifecycle, signal handling
│   │   ├── http.go            # HTTP server setup with timeouts
│   │   └── routes.go          # All route definitions
│   └── session/
│       └── session.go         # Cookie-based session management
├── pkg/config/                # Reusable multi-environment config library
├── static/
│   ├── static.go              # //go:embed directive
│   ├── css/style.css          # Custom stylesheet (system font stack, card effects, layout)
│   └── js/app.js              # Client-side JavaScript (minimal bootstrap)
├── templates/                 # Go HTML templates (base, index, login, etc.)
├── configs/
│   └── config.yaml.example    # Example configuration file
├── Dockerfile                 # Multi-stage: build + check, then Alpine runtime
├── Makefile                   # fmt, lint, test, check, build, docker targets
├── go.mod / go.sum
└── .golangci.yml              # Linter configuration

Dependency Injection

Components are wired via Uber fx in this order:

1. globals.New — Build-time variables (appname, version, arch)
2. logger.New — Structured logging (slog with TTY detection)
3. config.New — Configuration loading (pkg/config + environment)
4. database.New — SQLite connection, migrations, admin user seed
5. healthcheck.New — Health check service
6. session.New — Cookie-based session manager
7. handlers.New — HTTP handlers
8. middleware.New — HTTP middleware
9. server.New — HTTP server and router

The server starts via fx.Invoke(func(*server.Server) {}) which triggers the fx lifecycle hooks in dependency order.

Middleware Stack

Applied to all routes in this order:

1. Recoverer — Panic recovery (chi built-in)
2. RequestID — Generate unique request IDs (chi built-in)
3. Logging — Structured request logging (method, URL, status, latency, remote IP, user agent, request ID)
4. Metrics — Prometheus HTTP metrics (if METRICS_USERNAME is set)
5. CORS — Cross-origin resource sharing headers
6. Timeout — 60-second request timeout
7. Sentry — Error reporting to Sentry (if SENTRY_DSN is set; configured with Repanic: true so panics still reach Recoverer)

Authentication

• Web UI: Cookie-based sessions using gorilla/sessions with encrypted cookies. Sessions are configured with HttpOnly, SameSite Lax, and Secure (in production). Session lifetime is 7 days.
• API (planned): API key authentication via Authorization: Bearer header. API keys are stored per-user with usage tracking (last_used_at).
• Metrics: Basic authentication protecting the /metrics endpoint.

Security

• Passwords hashed with Argon2id (64 MB memory cost)
• Session cookies are HttpOnly, SameSite Lax, Secure (prod only)
• Session key must be a 32-byte base64-encoded value
• Prometheus metrics behind basic auth
• Static assets embedded in binary (no filesystem access needed at runtime)
• Container runs as non-root user (UID 1000)
• GORM soft deletes on all entities (data preserved for audit)

Docker

The Dockerfile uses a multi-stage build:

1. Builder stage (Debian-based golang:1.24) — installs golangci-lint, downloads dependencies, copies source, runs make check (format verification, linting, tests, compilation).
2. Runtime stage (alpine:3.21) — copies the binary, runs as non-root user, exposes port 8080, includes a health check.

The builder uses Debian rather than Alpine because GORM's SQLite dialect pulls in CGO-dependent headers at compile time. The runtime binary is statically linked and runs on Alpine.

docker build . is the CI gate — if it passes, the code is formatted, linted, tested, and compiled.

TODO

Phase 1: Core Webhook Engine

• Implement webhook reception and event storage at /webhook/{uuid}
• Build event processing and target delivery engine
• Implement HTTP target type (fire-and-forget POST)
• Implement retry target type (exponential backoff)
• Implement database target type (store only)
• Implement log target type (console output)
• Per-webhook rate limiting in the receiver handler
• Webhook signature verification (GitHub, Stripe formats)

Phase 2: Database Separation

• Split into main application DB + per-webhook event DBs
• Automatic event retention cleanup based on retention_days
• Per-webhook database lifecycle management (create on webhook creation, delete on webhook removal)

Phase 3: Security & Infrastructure

• Implement authentication middleware for protected routes (#9)
• Security headers (HSTS, CSP, X-Frame-Options)
• CSRF protection for forms
• Session expiration and "remember me"
• Password change/reset flow
• API key authentication for programmatic access

Phase 4: Web UI

• Webhook management pages (list, create, edit, delete)
• Webhook request log viewer with filtering
• Delivery status and retry management UI
• Manual event redelivery
• Analytics dashboard (success rates, response times)
• Replace Bootstrap with Tailwind CSS + Alpine.js (#4)

Phase 5: REST API

• RESTful CRUD for webhooks, entrypoints, targets
• Event viewing and filtering endpoints
• Event redelivery endpoint
• OpenAPI specification

Phase 6: Code Quality

• Rename Processor → Webhook, Webhook → Entrypoint in code (#12)
• Embed templates via //go:embed (#7)
• Use slog.LevelVar for dynamic log level switching (#8)
• Simplify configuration to prefer environment variables (#10)
• Remove redundant godotenv/autoload import (#11)

Future

• Email delivery target type
• SNS, S3, Slack delivery targets
• Data transformations (e.g., webhook-to-Slack message formatting)
• JSONL file delivery with periodic S3 upload
• Webhook event search and filtering
• Multi-user with role-based access

License

MIT

Author

@sneak