webhooker/README.md

webhooker

webhooker is a self-hosted webhook proxy and store-and-forward service written in Go by @sneak. It receives webhooks from external services, durably stores them, and delivers them to configured targets with retry support, logging, and observability. Category: infrastructure / web service. License: MIT.

Each entrypoint is a version 4 UUID served at /h/{uuid}, and that UUID is the entrypoint's only credential. webhooker does not use shared secrets, HMAC signatures or token headers on the receiver, and will not add them — read The entrypoint URL is the authentication secret before deploying one.

Getting Started

Prerequisites

• Go 1.26.1+ (the version in go.mod)
• Docker (for make lint, make fmt and make css, and so for make check, for the browser test in make test-browser, for the CI gate, and for containerized deployment)

golangci-lint is not a prerequisite and must not be installed on the host: script/bootstrap does not install it, and make lint runs the digest-pinned linter image via Dockerfile.lint. The same holds for tailwindcss (see Stylesheet). ESLint, prettier, node and yarn are not prerequisites either, and make lint and make fmt never use a host copy of them (see Linting).

Quick Start

# Clone the repo
git clone https://git.eeqj.de/sneak/webhooker.git
cd webhooker

# Install the Go toolchain if missing, and the Go dependencies
make bootstrap

# Run all checks (test, lint, format check, stylesheet check)
make check

# Run the server from the clone. DATA_DIR defaults to
# /var/lib/webhooker in every environment, so set it (in .env or the
# shell) to a writable directory.
DATA_DIR=./data make dev

# Build Docker image
make docker

Development Commands

make bootstrap  # Install all dependencies (idempotent)
make setup      # Bootstrap + install git pre-commit hook
make assets     # Extract Alpine.js from 3p/ (test, check, build, dev run it)
make fmt        # Format Go (gofmt + goimports) and Markdown (prettier, in Docker)
make fmt-check  # Fail if gofmt or prettier would change anything (writes nothing)
make lint       # Run golangci-lint and ESLint in Docker
make test       # Run tests with race detection
make test-browser # Run the browser test in Docker (Dockerfile.browser)
make check      # test + lint + fmt-check + css-check (CI gate)
make build      # Build binary to bin/webhooker (version-stamped)
make version    # Print the version this checkout would stamp
make run        # build, then run ./bin/webhooker
make dev        # go run ./cmd/webhooker
make deps       # go mod download + go mod tidy
make docker     # Build Docker image
make hooks      # Install git pre-commit hook that runs script/precommit
make css        # Regenerate static/css/tailwind.css (tailwindcss in Docker)
make css-check  # Fail if static/css/tailwind.css is stale (writes nothing)
make clean      # Remove bin/

Configuration

All configuration is via environment variables. For local development, you can place variables in a .env file in the process working directory, read once at startup before anything else looks at the environment.

The file is optional and having none is the normal case for a deployment. An empty file is the same as none: it has nothing in it to apply. A file that is there but cannot be parsed aborts startup with a message naming it, because a single malformed line makes none of the file apply: every variable in it silently reverts to its default, which is exactly the failure Invalid values abort startup exists to prevent, for all of them at once. A variable already present in the real environment wins over the file's value for the same name.

The environment is selected by setting WEBHOOKER_ENVIRONMENT to dev or prod (default: prod; dev must be set explicitly). The setting controls exactly one behavior:

Behavior	dev	prod
CORS	Allows any origin (*)	Disabled (no-op)

The environment setting does not control cookie security. Both the session cookie and the CSRF cookie get their Secure flag, and the CSRF middleware its Origin/Referer validation mode, from the transport of each individual request, decided by one predicate — internal/reqtls.IsTLS. It reports TLS for a direct TLS connection (r.TLS) or for a TLS-terminating reverse proxy that reports one in X-Forwarded-Proto:

• Direct TLS or X-Forwarded-Proto: https: Secure cookies, strict Origin/Referer validation.
• Plaintext HTTP: Non-Secure cookies, relaxed Origin/Referer checks (token validation still enforced).

The X-Forwarded-Proto value is matched case-insensitively on its first comma-separated element, trimmed, so HTTPS and the appended chains a proxy behind another proxy emits (https, http) are all read as TLS.

This means both cookie security and CSRF protection work correctly in all deployment scenarios: behind a TLS-terminating reverse proxy, with direct TLS, or over plain HTTP during development — a plain-HTTP local run gets non-Secure cookies and remains usable, and a proxied deployment gets Secure ones without the operator setting anything. When running behind a reverse proxy, ensure it sets the X-Forwarded-Proto: https header. Unlike X-Forwarded-For, this header is read from any peer and is not gated by TRUSTED_PROXIES; a correctly configured proxy overwrites whatever a client sent. On a listener exposed directly to clients, any client can assert it, so do not run one without a proxy in front.

All other differences (log format, security headers, etc.) are independent of the environment setting — log format is determined by TTY detection, and security headers are always applied.

Variable	Description	Default
WEBHOOKER_ENVIRONMENT	dev or prod	prod
PORT	HTTP listen port	8080
BIND_ADDRESS	IP address the HTTP listener binds. Loopback by default, so the cleartext listener is not published on every interface. The Docker image ships 0.0.0.0 instead. See Bind address	127.0.0.1 (image: 0.0.0.0)
DATA_DIR	Directory for all SQLite databases	/var/lib/webhooker
DEBUG	Enable debug logging	false
METRICS_USERNAME	Basic auth username for /metrics. Must be set together with METRICS_PASSWORD; one without the other fails startup	""
METRICS_PASSWORD	Basic auth password for /metrics. Must be set together with METRICS_USERNAME; one without the other fails startup	""
SENTRY_DSN	Sentry error reporting DSN. Unset leaves error reporting off; a value the Sentry SDK cannot parse fails startup rather than serving with reporting silently off	""
RETENTION_SWEEP_INTERVAL	How often the retention reaper and archive sweeper run (Go duration, must be positive). A value that does not parse, or is zero or negative, fails startup	1h
SESSION_IDLE_TIMEOUT	Idle session timeout (Go duration)	24h
RECEIVER_RATE_LIMIT	Receiver requests/minute per IP per entrypoint (10x that per IP across the route)	120
TRUSTED_PROXIES	CIDRs whose forwarded headers are trusted. A set value replaces the default. If any client can reach webhooker, or the proxy in front of it, from an RFC 1918 source address, set it to the proxy's address alone. See Trusted proxies	10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 (RFC 1918)
ALLOWED_EGRESS_CIDRS	CIDRs that delivery targets may reach despite the SSRF blocklist. Read Allowing egress to your own network before setting it	"" (none)

The Settings page of the web UI (/settings, behind the login) lists every one of these with the value the running server loaded. It is read-only, and it shows METRICS_PASSWORD and SENTRY_DSN only as set or not set, never their values.

Allowing egress to your own network

By default every delivery target must resolve to a public address. The private and reserved ranges — RFC 1918, loopback, CGNAT, link-local and the rest — are refused, which stops a target from being used to make webhooker probe the network it sits in.

Besides the private and reserved ranges, the default blocklist refuses public cloud metadata addresses: currently only 168.63.129.16, Azure's WireServer, which serves an Azure VM its credentials. Because it is a public address, listing it in ALLOWED_EGRESS_CIDRS reopens it.

That is all the default blocklist covers: the IPv4 private and reserved ranges; of IPv6, only loopback (::1), the unspecified address (::), unique local addresses (fc00::/7), link-local addresses (fe80::/10), multicast (ff00::/8) and documentation space (2001:db8::/32); and certain public addresses. A public address belongs on the default blocklist only if it hands credentials, user data or bootstrap material to whatever can reach it, without the caller presenting anything. A provider's other public addresses are not refused. IBM Cloud, for example, serves its package mirrors, time servers and object storage on 161.26.0.0/16, and the private endpoints of its own cloud services on 166.8.0.0/14. Neither range hands out credentials that way: the token service among those endpoints issues a token only in exchange for something the caller presents, such as an API key. Reaching these services can be a legitimate delivery, and every cloud has some, so a partial list would promise coverage it does not give.

That default is also inconvenient for the thing webhooker is mostly for: taking a public webhook and forwarding it to something on your own network. A container on the same Docker network, a box on 10.x, a service on 127.0.0.1 — all refused, until you name them.

ALLOWED_EGRESS_CIDRS is a comma-separated list of CIDR blocks (a bare address such as 10.0.0.7 is accepted and treated as a single host), for example 10.0.0.0/8, 172.17.0.0/16. Addresses inside those blocks become valid delivery destinations. Everything outside them keeps the default answer, so this only ever adds destinations — it never removes any, and it cannot narrow what was already reachable.

The risk, plainly. Each block you list is a network that anyone who can create a delivery target can now make this process issue requests into, and read the response body back out of via the delivery log. That is server-side request forgery, deliberately enabled and scoped by you. A webhooker admin account is therefore as trusted as the narrowest thing on those networks: an unauthenticated admin panel, a database listening without a password, or an internal API that trusts its network position is reachable through it. List the smallest blocks that cover the destinations you actually deliver to — prefer 10.1.2.3/32 over 10.0.0.0/8 — and never list a block wider than the network you are willing to expose.

Listing 0.0.0.0/0 or ::/0 opens every other private and reserved range at once — loopback, RFC 1918, CGNAT, ULA, the lot. It is a functional off switch for everything except the addresses listed as unconditionally blocked below, and it makes any delivery target a probe into your entire network and this host's own loopback services. Do not list it.

Two things this setting cannot do:

• It cannot turn the guard off. There is no boolean, and no value that disables SSRF protection wholesale. The guard is always on and the list is always an allowlist; an empty list (the default) means every private and reserved range stays refused. Note that 0.0.0.0/0 gets you most of the way there anyway, per above.

• It cannot open link-local, the unspecified addresses, or a cloud metadata endpoint at a non-public address that discloses credentials or user data. A metadata address is on the list below when it is not a public address and both of these hold: the provider fixes it, so it cannot collide with anything you run; and reaching it hands out credentials, user data or bootstrap material. Those stay blocked no matter what you list, including when you list them outright or list a supernet such as 0.0.0.0/0, ::/0, fd00::/8 or 100.64.0.0/10. Treat this as best effort rather than a guarantee — it is a hand-maintained list and the caveat below the table applies:

  Blocked unconditionally	What it is
  169.254.0.0/16	IPv4 link-local, carrying 169.254.169.254 (AWS, Azure, DigitalOcean, Hetzner, OpenStack and others — not Alibaba, which uses 100.100.100.200 below)
  fe80::/10	IPv6 link-local
  fd00:ec2::254/128	AWS IPv6 IMDS
  fd00:ec2::23/128	AWS EKS Pod Identity Agent
  fd20:ce::254/128	GCP metadata for IPv6-only instances
  fd00:c1::a9fe:a9fe/128	Oracle OCI IMDS over IPv6
  fd00:42::42/128	Scaleway metadata over IPv6
  fd00:a9fe:a9fe::1/128	Linode/Akamai metadata over IPv6
  100.100.100.200/32	Alibaba Cloud metadata, inside CGNAT
  192.0.0.192/32	Oracle Cloud Classic metadata
  0.0.0.0/32	IPv4 unspecified address, which reaches this host's loopback on Linux
  ::/128	IPv6 unspecified address, which reaches this host's loopback on Linux
  ::a9fe:a9fe/128	169.254.169.254 as an IPv4-compatible IPv6 address
  64:ff9b::a9fe:a9fe/128	169.254.169.254 behind the NAT64 well-known prefix

  The IPv4-mapped form ::ffff:169.254.169.254 is covered by the 169.254.0.0/16 entry. Reaching any of these but the two unspecified addresses is credential or user-data theft rather than delivery to an internal service. Every entry outside the two link-local blocks is a single address, so blocking it costs you nothing else on the network around it.

  The unspecified addresses 0.0.0.0 and :: hand out nothing themselves, but no host can have either, and on Linux a connection to one reaches this host's own loopback. They are listed so that an allowlist reaches loopback only through an entry that covers a loopback address, such as 127.0.0.0/8, ::1 or 0.0.0.0/0, never through one that covers only 0.0.0.0 or ::; 0.0.0.0/8, for example, does not open loopback.

  The six ULA entries, all inside fd00::/8, are why this matters in practice: fd00::/8 is an ordinary block to allowlist for your own IPv6 network, and without those host routes that one line would hand out cloud credentials on five providers at once. There is only one /8 involved — fd20:ce::254 masks into fd00::/8 as well — and the six endpoints are five providers because AWS appears twice, IMDS and EKS Pod Identity. Several of them are described as "link-local" — or even "localhost" — in their own vendor's documentation, but they are ULAs and fe80::/10 does not cover them.

  Every entry above is reserved space. All but the last two are already refused with no allowlist set, and listing them here is only what stops an allowlist from reopening them; the last two are the alternate encodings, which the default blocklist does not match. A publicly routable metadata address is not listed here, because nothing on this list can be reopened and blocking one that way would leave you no escape hatch at all; Azure's 168.63.129.16 is refused by the default blocklist instead, as described above.

  This list is not exhaustive of every cloud's metadata address — if yours is not here, do not allowlist the block that contains it.

The list is applied at one place in the code, which both target creation and delivery consult, so a URL that the target form accepts is one that delivery will actually attempt — the two cannot disagree. Delivery re-resolves and re-checks the destination at dial time, so a hostname that resolves to an allowed address during validation and a different one later (DNS rebinding) is still refused unless the new address is also allowed.

A set but unparseable value aborts startup. When the list is non-empty webhooker logs it at startup, blocks and all, so the hole is visible in the log of any deployment that has one.

Bind address

BIND_ADDRESS is the IP address the HTTP listener binds. The binary defaults to 127.0.0.1, so a bare webhooker is reachable only from the host it runs on. The Docker image ships ENV BIND_ADDRESS=0.0.0.0 instead — see below for why the two differ.

That listener speaks cleartext, and it serves both the admin UI and the unauthenticated webhook receiver. webhooker terminates no TLS itself; a production deployment puts a reverse proxy in front of it (see Deployment behind a reverse proxy), and the proxy reaches it over loopback. A default that bound every interface would leave that cleartext port answering the internet alongside the proxy — the admin sign-in form and the receiver, in the clear, on a port nobody chose to publish. Reaching webhooker from another host is therefore something you configure, not something you get by default.

In a container the answer is 0.0.0.0, which is why the image ships that. A container's network namespace is already the boundary the loopback default is reaching for: nothing outside the container gets to 0.0.0.0:8080 because of the namespace, whatever the process bound. Exposure is decided at the publish flag instead — -p 127.0.0.1:8080:8080 rather than -p 8080:8080 — which is the operator's to choose and is what Running with Docker shows. A loopback bind inside a container buys nothing and makes the process unreachable through its own published port.

The value must be an IP address literal:

• 127.0.0.1 — loopback only (the binary's default). Use this with a reverse proxy on the same host.
• 0.0.0.0 — every IPv4 address. The image's default; on a bare host, only behind a firewall on the port.
• :: — every address, IPv6 and (on Linux, with the default net.ipv6.bindv6only=0) IPv4 as well.
• A specific address such as 10.0.0.5 — that interface only.

An empty value is treated as unset, as everywhere else here, and takes the default. In a container that matters: BIND_ADDRESS= throws away the image's 0.0.0.0 and falls back to the binary's 127.0.0.1, which is the one quiet failure this setting has — see Running with Docker.

Hostnames are not accepted. localhost aborts startup rather than being resolved: which of 127.0.0.1 and ::1 it means differs by host, a name can resolve to several addresses of which only one could be bound, and the answer can change under a running process. A value carrying a port (127.0.0.1:8080) is likewise rejected — the port is PORT's business. Any unparseable value aborts startup; see Invalid values abort startup.

An address that parses but is not assigned to this host — say 10.0.0.5 on a machine that has no such interface — is a valid literal, so it reaches the listener and fails there. The process logs the bind error and exits non-zero rather than staying up with nothing listening. The effective value is in the bindAddress field of the startup log line, which is the way to check what a running deployment actually bound.

Metrics credentials

METRICS_USERNAME and METRICS_PASSWORD are set together or not at all. With both set, /metrics is served behind basic auth. With neither set, the route is not registered and returns 404. With one set and the other empty or unset, the process refuses to start and exits non-zero with an error naming both variables — mounting the endpoint on the username alone would publish it behind a password that is the empty string, and quietly withholding it would deny an endpoint that was asked for. The hasMetricsAuth field in the startup log and the existence of the route are the same value, so they cannot disagree.

Single-instance lock

Exactly one webhooker process may use a DATA_DIR at a time. Two processes sharing one open the same databases and each run delivery recovery over the same rows, so every pending delivery goes out twice — duplicate delivery to your endpoints, from nothing worse than an overlapping deploy or a double start.

At startup, before anything opens a database, the process takes an exclusive advisory lock (flock(2)) on {DATA_DIR}/webhooker.lock and holds it for its lifetime. A second process pointed at the same directory prints a message naming it and exits non-zero:

webhooker: data directory is already in use by another instance: /var/lib/webhooker (/var/lib/webhooker/webhooker.lock). Only one webhooker may use a data directory: two both run delivery recovery over the same rows and both deliver

The lock is the kernel's, not the file's: it is released when the process exits, including kill -9, so a leftover webhooker.lock never blocks a restart and must not be deleted by hand. The file is also left in place on a clean shutdown, deliberately — unlinking it would let the next process lock a fresh inode while a third still held the old one.

To run two webhookers on one host, give each its own DATA_DIR.

flock(2) is host-local and per-inode: it arbitrates between processes and containers sharing a volume or bind mount on one machine, but not between hosts on a network filesystem, and a DATA_DIR inside a container's own writable layer is not shared with anything. On a filesystem that refuses flock outright, startup fails closed — the process reports the error and refuses to start rather than running unlocked.

Trusted proxies

TRUSTED_PROXIES is a comma-separated list of CIDR blocks (a bare address such as 192.168.1.7 is accepted and treated as a single host), for example 192.168.1.7, 2001:db8::5. It decides whose X-Forwarded-For header the rate limiters believe, so it should cover the addresses of your reverse proxies.

X-Forwarded-For is honoured only when the connecting peer is inside one of these blocks; for every other peer the client identity is the connection's own address and the header is ignored. Unset (or empty), the list is the RFC 1918 private ranges: 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16. A set value replaces the default entirely. A set but unparseable value aborts startup.

If any client can reach webhooker, or the proxy in front of it, from an RFC 1918 source address (directly, or through anything that can rewrite source addresses, such as NAT or a published container port), set TRUSTED_PROXIES to the proxy's address alone, or every rate limit, the webhook receiver's included, can be bypassed by those clients. The address to set is the remoteIP field of the http request log line for a request that came through the proxy.

Behind a proxy the list does not cover, every request keys on the proxy's own address and all clients share a single bucket per limit. The receiver limits become service-wide ceilings, and the login endpoint's failure counting collapses onto one key, so a stranger's wrong passwords throttle every other client's wrong passwords. Set TRUSTED_PROXIES to that proxy's address to restore per-client buckets.

What it cannot do is lock the operator out. The login endpoint verifies credentials before it consults any limit and charges only failures, so a correct password is never throttled no matter how full the bucket is. See Rate Limiting.

X-Real-IP and True-Client-IP are never read, from any peer. Reverse proxies append to X-Forwarded-For but forward other client headers verbatim, so a single-valued header is client-controlled even behind a trusted proxy.

Within a trusted request, X-Forwarded-For is read right to left, because the rightmost entry is the one the nearest proxy appended and everything left of it may have been written by the client. The first hop that is not itself a trusted proxy is taken as the client. A hop that is not a bare IP address — ip:port, a bracketed IPv6 literal, the token unknown — ends the walk and the peer address is used instead, since past such an entry the chain is not the shape assumed here. The peer address is likewise used when the header is absent or every hop in it is a trusted proxy.

Your proxy must therefore append the peer address to X-Forwarded-For (nginx $proxy_add_x_forwarded_for, HAProxy option forwardfor, Caddy and AWS ALB by default), and must append a bare address with no port.

Every log line that names a client carries two addresses: remoteIP, the connecting peer, which behind a proxy is the proxy; and clientIP, the client the rate limiters identify by the rules above, which is the field to read when tracing who sent what. Those lines are the http request access log line, the rate-limit rejection lines (login failure limit exceeded among them), the csrf: token validation failed warning and the receiver's webhook request received line. clientIP is only as trustworthy as TRUSTED_PROXIES: for a request from a peer inside the list, it is read out of the X-Forwarded-For that peer sent, so a peer that does not belong in the list can make it name any address it likes. For a request from any other peer, both fields name the peer.

Sessions

Sessions are bounded by two independent clocks, and end at whichever one runs out first:

• Idle expiry (SESSION_IDLE_TIMEOUT, default 24h) is a sliding window. Every authenticated request pushes it forward, so a session in continuous use never hits it, while an abandoned one expires a day after its last use. Any non-positive value (0, or a negative duration such as -1s) disables idle expiry entirely; the absolute cap below still applies. A set-but-unparseable value aborts startup rather than silently falling back to the default.
• Absolute expiry is a fixed 7 days from login. Activity does not extend it: after a week, every session ends and the user authenticates again.

Only requests that authenticate with the session count as activity, so an unauthenticated request carrying the cookie cannot keep a session alive. The idle timestamp is rewritten at most once per tenth of the idle window rather than on every request, which means a session may expire up to 10% early relative to the user's true last request, but never late.

Both clocks are anchored by timestamps stored in the session cookie. Sessions issued before this feature existed carry neither, so they are treated as expired: upgrading to a build that has it logs every existing session out once, and those users sign in again.

Invalid values abort startup

The defaults above apply only to variables that are unset (or set to an empty string). A variable that is set but cannot be parsed is a fatal configuration error: webhooker logs the offending variable and its value and refuses to start, rather than silently running with a substituted default. PORT=eighty, DEBUG=ture, and RETENTION_SWEEP_INTERVAL=1 hour all abort startup. PORT must additionally be a number in the range 1–65535, RECEIVER_RATE_LIMIT must be at least 1, RETENTION_SWEEP_INTERVAL must be greater than zero (it is a ticker period, so 0s or a negative value would crash the reaper after startup), every entry in TRUSTED_PROXIES and ALLOWED_EGRESS_CIDRS must be a CIDR block or a bare IP address, and BIND_ADDRESS must be an IP address literal — localhost, 127.0.0.1:8080 and 10.0.0.0/8 are each rejected rather than resolved, split, or narrowed to something they do not say — and SENTRY_DSN must parse as a Sentry DSN. SESSION_IDLE_TIMEOUT is the exception: a non-positive value there means idle expiry is disabled, not invalid.

SENTRY_DSN is checked with the Sentry SDK's own parser, the same call the SDK makes on the DSN it is later handed, so what configuration accepts is exactly what will initialise. A typo in it is the one configuration mistake nothing downstream can ever notice — the variable is still set, so every later signal reports error reporting as on while no report is being sent — which is why it aborts rather than starting with reporting off. Leaving it unset is not a mistake and not affected: error reporting is simply off and startup is normal.

The boolean variable DEBUG accepts exactly the spellings Go's strconv.ParseBool accepts — 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False — and nothing else. yes, on, and off are rejected rather than quietly treated as false.

On first startup, webhooker automatically generates a cryptographically secure session encryption key and stores it in the database. This key persists across restarts — no manual key management is needed.

The admin account

On first startup — a DATA_DIR with no accounts in it — webhooker creates an admin user with a randomly generated password and prints it to standard output as a ruled banner:

========================================================================
WEBHOOKER FIRST BOOT: an admin account has been created.

    username: admin
    password: 3xamPl3-p4ssw0rd

Save this password now: it is shown only here, and only once.
If it is lost, run `webhooker resetpw admin` on a stopped deployment.
========================================================================

It is a banner rather than a log line because that is the only time it is ever shown: as one INFO record it would sit among the records fx writes as each start hook runs, and under docker run -d it is one line in a log subject to rotation. The database stores only its Argon2id hash. There is no second account and no forgot-password flow, so the banner and the reset command below are the only two ways in.

A start that finds no webhooker.db in DATA_DIR, or a zero-length one (which SQLite opens as an empty database), also logs created a new, empty database at WARN, with the file's path, shortly before the banner. On a deployment that has run before, that line means webhooker.db was lost: either the file was missing, most often because the volume holding DATA_DIR is not mounted, or it was zero-length, as a truncated copy leaves it.

Recovering a lost admin password

webhooker resetpw sets an existing account's password from the command line:

# Generate a new password and print it.
DATA_DIR=/var/lib/webhooker webhooker resetpw -generate admin

# Or supply one on standard input (minimum 8 characters).
printf '%s' "$NEW_PASSWORD" | \
    DATA_DIR=/var/lib/webhooker webhooker resetpw admin

In a container it is the same binary. The image's CMD is /app/webhooker, and a command given to docker run replaces all of it, so the whole command has to be given:

docker run --rm -v webhooker-data:/var/lib/webhooker \
    webhooker /app/webhooker resetpw -generate admin

Stop the service first — with the volume still attached to a running container, the command refuses.

The password is never taken as a command-line argument: on Linux argv is readable through /proc by every account on the host for as long as the process lives. Standard input is echoed when it is a terminal — the prompt says so — so -generate or a pipe is preferable on a shared machine.

What it will not do:

• Run against a live deployment. It takes the same exclusive DATA_DIR lock the server does (see Single-instance lock) and refuses while a running instance holds it, naming the directory and exiting non-zero. A running process keeps serving every session that authenticated with the old password, so a reset underneath it would report a change the service does not honour.
• Create anything. A DATA_DIR that does not exist, or that holds no webhooker.db or a zero-length one, is an error naming the path rather than a new empty deployment — a mistyped path must not be built out and then reported as a success.
• Create an account. A username that does not exist is an error. resetpw changes an existing account's password and nothing else.

DATA_DIR selects the deployment exactly as it does for the server. A password that changes on disk takes effect at the next login; sessions that are already authenticated are unaffected either way.

Changing a password you still know needs none of this — use POST /user/{username}/password in the web UI.

What DEBUG=true exposes

DEBUG=true lowers the log level to DEBUG, which turns on every statement GORM runs, the two by-design lookup misses on the unauthenticated routes, the rate limiter's own rejections, and fx's records of building the dependency graph at startup. It is meant to be safe to turn on while diagnosing a live service and safe to paste the output of into a bug report.

What it does not put in the log:

• Values bound to a SQL statement. Statements are logged with their placeholders, never with the values substituted into them, at every level. That is what keeps the session encryption key out of the first boot's INSERT INTO settings and the admin account's Argon2id password hash out of its INSERT INTO users — the two statements that made a debug log worth stealing. It applies to every table and every statement rather than to a list of tables known to hold a secret, so a table added later is covered without anyone remembering to add it. The cost is that a failing statement can no longer be replayed from the log alone: the statement, the table, the driver error and the row count are all still there, but its values have to come from the database. internal/gormlog/firstboot_test.go boots the real graph with DEBUG=true against an empty DATA_DIR and asserts that neither secret appears in what that boot wrote to stdout. The one exception is (*gorm.DB).Scan, which GORM logs through its own trace recorder rather than through this filter. No production code path calls it, and internal/gormlog/scan_guard_test.go fails if a non-test file adds one.
• Session cookies, API keys or target credentials. None of these is logged at any level.

What is in the log regardless of DEBUG, and is not a debug-logging decision:

• The initial admin password, in the clear, once, on the first boot that creates the account — as the banner described under The admin account, written straight to standard output rather than through the logger. That banner is the only place it is ever shown; the database stores the hash. A first boot's output is not safe to paste anywhere until that account's password has been changed. The same applies to webhooker resetpw -generate, which prints the password it generated in the same form.
• An authenticated operator's own configuration, echoed back untruncated — webhook names, target hostnames. See the logging section under Security for the full list and for the per-line size bound that covers unauthenticated traffic.

Running with Docker

docker run -d \
  -p 127.0.0.1:8080:8080 \
  -v /path/to/data:/var/lib/webhooker \
  -e BIND_ADDRESS=0.0.0.0 \
  webhooker:latest

The image and the bare binary default BIND_ADDRESS differently, on purpose. The binary defaults to 127.0.0.1; the image ships ENV BIND_ADDRESS=0.0.0.0, so the -e BIND_ADDRESS=0.0.0.0 above is belt-and-braces and the command works without it.

The two cases are not the same question. On a bare host, 0.0.0.0 puts the cleartext admin UI and the unauthenticated receiver on every interface of the machine, which is what the loopback default exists to prevent. In a container, the network namespace is already that boundary: nothing outside reaches 0.0.0.0:8080 because of the namespace, not because of the bind. What decides exposure there is the publish flag, and that is the line to get right.

So publish to 127.0.0.1:8080 rather than 8080. A bare -p 8080:8080 opens the port on every interface of the host — through firewall rules too, since Docker's forwarding rules are inserted ahead of most host firewalls. Publish to the host address your reverse proxy connects from, and nothing wider.

An empty BIND_ADDRESS is treated as unset, like every other variable here, so -e BIND_ADDRESS= does not mean "keep the image default" — it discards the image's 0.0.0.0 and falls back to the binary's 127.0.0.1. In a container that is the failure below, and nothing in the logs names the variable. A templated Compose file or a .env line with an empty value is the usual way in. Either set a literal or leave the variable out entirely.

Overriding BIND_ADDRESS to a loopback address in a container — by that route or deliberately — makes the container unreachable from outside its namespace even with -p. The published port answers nothing, and the health check fails too: it requests http://localhost:8080, localhost resolves to ::1 first, and a 127.0.0.1 bind is not listening there. The container then goes unhealthy about 65 seconds after start — from HEALTHCHECK --start-period=5s --interval=30s --retries=3, so failing probes at 5s, 35s and 65s, and unhealthy on the third. (Docker's probe cadence during the start period has changed between versions; re-derive from those three values rather than trusting the figure. Measured at 65s on Docker 29.7.2.) A container unhealthy with connection refused in its health log, or a published port that resets connections, is this.

The app runs as a non-root user (webhooker, UID 1000), exposes port 8080, and includes a health check against /.well-known/healthcheck. The /var/lib/webhooker volume holds all SQLite databases: the main application database (webhooker.db), the per-webhook event databases (events-{uuid}.db), and any archive databases written by database targets (archive-{webhook_name}-{target_name}-{target_uuid}.db, with -{period} before .db for a target that rotates). Mount this as a persistent volume to preserve data across container restarts.

The container sets its data directory's owner and mode itself before the app starts, so a host directory can be mounted as it is, whoever owns it. The image's ENTRYPOINT, deploy/docker-entrypoint.sh, starts as root, creates DATA_DIR if it is missing, gives the directory and anything in it that belongs to another user to webhooker, sets the directory to 0750, and only then runs the app as webhooker. Started with --user, it changes nothing and runs the app as that user.

The file modes are not yours to set, and do not depend on the directory. webhooker.db holds target configuration in plaintext — bearer tokens, API keys, Slack webhook URLs — along with the session encryption key, so webhooker creates every SQLite file it owns 0600: each database and both of its -wal and -shm sidecars, across all three tiers. Files an earlier build left 0644 are tightened when they are opened. The directory's 0750 is defence in depth — it stops other local users listing the directory and learning your webhook UUIDs from the events-{uuid}.db filenames — not the barrier protecting the credentials.

Running under upaas

upaas builds the image from this repository's Dockerfile and runs it. The app needs:

• Network and port: add no port mapping in upaas. upaas publishes every mapped port on all interfaces of the host (upaas issue 113), which would put the plain-HTTP admin UI and receiver there. Instead, set the app's Docker Network in upaas to your reverse proxy's Docker network; the proxy then reaches the app at upaas- followed by the app name, port 8080. Leave PORT unset: the image's health check probes 8080.

• Volume: one host directory mounted at /var/lib/webhooker.

• Environment variables:

  • WEBHOOKER_ENVIRONMENT=prod
  • TRUSTED_PROXIES: unset, it is the RFC 1918 ranges. Set it to your reverse proxy's address alone if that address is outside those ranges, or if any client can reach webhooker, or the proxy, from an RFC 1918 source address (directly, or through anything that can rewrite source addresses, such as NAT or a published container port). The remoteIP field of the http request log line for a request that came through the proxy shows that address; the health check's own lines show ::1. See Trusted proxies.
  • Leave BIND_ADDRESS and DATA_DIR unset: the image sets BIND_ADDRESS to 0.0.0.0, and DATA_DIR defaults to /var/lib/webhooker.
  • Everything else is optional; see Configuration.

• Health check: the image's own, which requests /.well-known/healthcheck. upaas reads the container's health 60 seconds after a deploy and marks the deploy failed unless it is healthy.

• First run: the first start prints the admin password once, in the banner described under The admin account, to the container's log. upaas names the container upaas- followed by the app name, so for an app named webhooker:

  docker logs upaas-webhooker
  

  If the password is lost, stop the container, set a new password with the app's own image and volume, and start it again (see Recovering a lost admin password):

  docker stop upaas-webhooker
  docker run --rm --volumes-from upaas-webhooker \
      "$(docker inspect -f '{{.Image}}' upaas-webhooker)" \
      /app/webhooker resetpw -generate admin
  docker start upaas-webhooker
  

Deployment behind a reverse proxy

webhooker terminates no TLS of its own. It serves plaintext HTTP and expects a reverse proxy in front of it, which is the deployment it is built for: the proxy holds the certificate, and webhooker binds loopback where only the proxy can reach it.

Five things have to be right. Each one is silent when it is wrong — the service comes up, serves pages, and is broken in a way nothing reports.

1. Bind or firewall the app port. The binary binds 127.0.0.1 by default, so the cleartext listener is not published beside the proxy. The image binds 0.0.0.0 inside its own network namespace and relies on the publish address instead — -p 127.0.0.1:8080:8080. Either way the port must reach the proxy and nothing else; widen it only with a firewall or a publish address in front of it. A cleartext port answering the internet serves the admin sign-in form and the unauthenticated receiver with no TLS at all, and the proxy in front of it changes nothing about that.
2. Make sure the environment is not dev (leave WEBHOOKER_ENVIRONMENT unset or set it to prod), and make sure the proxy sends X-Forwarded-Proto. These are two requirements, not one. The environment setting decides CORS and nothing else: dev answers every origin with Access-Control-Allow-Origin: * (without credentials), which a server-rendered production deployment has no use for, and prod — the default — disables it. Cookie Secure and the strict Origin/Referer mode are not tied to it — they are decided per request from the transport, which behind a proxy means the X-Forwarded-Proto header. The block below sets it; without it every request is read as plaintext and cookies ship without Secure. See Configuration.
3. Make sure TRUSTED_PROXIES covers the proxy's address. For a proxy it does not cover, every rate limiter keys on the proxy, so all clients share one bucket per limit. Unset, the list is the RFC 1918 ranges, which do not cover a proxy that reaches the binary itself over loopback (the binary bound to 127.0.0.1). With the image, the address to check is the remoteIP field of the http request log line for a request that came through the proxy. If any client can reach webhooker, or the proxy, from an RFC 1918 source address, set the list to the proxy's address alone. See Trusted proxies.
4. Send Host as $http_host, not $host. $host strips the port. webhooker's Origin/Referer check compares against the host it was given, so on any port other than 443 $host makes every form POST — including login — fail with 403 origin invalid, with nothing in the error naming the cause.
5. Keep the proxy's access log. webhooker's own access log names the client in its clientIP field only while TRUSTED_PROXIES covers the proxy; the proxy's log names it regardless. nginx's default combined format already logs $remote_addr; do not replace it with one that drops the client address, and retain those logs as long as you would want to answer a question about traffic.

nginx

Complete server block. Replace the server_name and the two certificate paths.

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;  # nginx 1.25.1+; older: listen 443 ssl http2;

    server_name webhooker.example.com;

    ssl_certificate     /etc/ssl/certs/webhooker.example.com.crt;
    ssl_certificate_key /etc/ssl/private/webhooker.example.com.key;
    ssl_protocols       TLSv1.2 TLSv1.3;

    # webhooker caps form POST bodies at 1 MB. nginx's default happens
    # to match, so leaving this out breaks nothing today — but if you
    # ever raise webhooker's cap, this is the limit you will still be
    # hitting, and the rejection is nginx's HTML page rather than
    # webhooker's message.
    client_max_body_size 1m;

    # $remote_addr is the client. webhooker's own log names it, as
    # clientIP, only while TRUSTED_PROXIES covers this proxy.
    access_log /var/log/nginx/webhooker.access.log combined;

    location / {
        # A literal address, not localhost: with BIND_ADDRESS at its
        # 127.0.0.1 default, a localhost that resolves to ::1 first
        # gets connection refused.
        proxy_pass http://127.0.0.1:8080;

        # $http_host, NOT $host. $host drops the port and every form
        # POST fails with 403 origin invalid on any port but 443.
        proxy_set_header Host              $http_host;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Above webhooker's own 60s request timeout, so its 503
        # reaches the client instead of nginx cutting the connection
        # first and answering 504.
        proxy_read_timeout 70s;
    }
}

server {
    listen 80;
    listen [::]:80;
    server_name webhooker.example.com;
    return 308 https://$host$request_uri;
}

X-Forwarded-For must be appended to, which $proxy_add_x_forwarded_for does. webhooker reads no other forwarded client header: X-Real-IP and True-Client-IP are ignored from every peer, so setting them has no effect. See Trusted proxies for how the chain is walked.

X-Forwarded-Proto: https is what tells webhooker the request arrived over TLS, which decides the Secure flag on both the session and CSRF cookies and the strict Origin/Referer mode. Without it, requests are treated as plaintext and the cookies ship without Secure. Unlike X-Forwarded-For, this header is read from any peer and is not gated by TRUSTED_PROXIES, so the proxy must overwrite whatever a client sent — $scheme above does.

With that block, webhooker's environment is:

BIND_ADDRESS=127.0.0.1   # the default; stated here to be explicit
TRUSTED_PROXIES=127.0.0.1

If nginx runs on another host, BIND_ADDRESS becomes the address it connects to, TRUSTED_PROXIES becomes nginx's address, and the port must be firewalled to that address — the traffic between them is cleartext.

HSTS is always sent, and is not configurable

Every response carries Strict-Transport-Security: max-age=63072000; includeSubDomains; preload. Two years, every subdomain, and a preload token. There is no setting that changes or suppresses it.

This is worth knowing before the first request reaches a browser: a client that sees it once will refuse plaintext HTTP to that hostname — and to every subdomain of it — for two years, whatever else is served there. Terminate TLS on a hostname you are prepared to keep on HTTPS.

Backup, Restore, and Upgrades

What to back up

All persistent state lives in DATA_DIR (/var/lib/webhooker by default). Back up that directory in full — the state is spread across several files whose names depend on your data, so copying the directory is both the simplest and the only complete rule:

• webhooker.db — one per install. Settings (including the session encryption key), users, API keys, webhooks, entrypoints, targets.
• events-{webhook_uuid}.db — one per webhook. Events, deliveries, delivery results.
• archive-{webhook_name}-{target_name}-{target_uuid}.db — one per database target, or one per month, day or hour for a target that rotates, with -{period} before .db. Archived events. The two names are made safe for a file name, and the files are renamed when the webhook or the target is (see Database Architecture).

{webhook_uuid} and {target_uuid} are UUID primary keys in their canonical 36-character hyphenated form, so a real filename looks like events-3f2a1c9e-....db. The only other file is webhooker.lock, the always-empty single-instance lock; it holds no state and is not part of the backup set — a copied one is stale and blocks nothing.

-wal and -shm sidecars. Every database runs in WAL journal mode, so while the service is running each {name}.db has a {name}.db-wal and a {name}.db-shm beside it. -wal is part of the database, not a scratch file: it holds committed transactions that are not yet in the .db, so a copy of the .db without its -wal is missing data and may have no readable schema at all. -shm is regenerable, but there is no reason to separate the two — copy the directory and you have them.

A clean shutdown closes every database, which checkpoints and removes its sidecars; a killed or crashed instance leaves them, and they must be carried with the .db. An archive the service has not opened since a crash keeps that crash's sidecars, even across a later clean stop.

A missing sidecar is therefore normal, and SQLite makes new ones, so a -wal lost from a copy cannot be reported: the transactions it held are simply gone. SQLite reads a -wal up to its first damaged frame, as after a crash, and rebuilds a damaged -shm. A sidecar with the wrong mode is set back to 0600 when its database is opened. A directory in place of either is refused then, with an error naming it: for webhooker.db the server and webhooker resetpw stop, and an event or archive database fails as a damaged one does (see Database Architecture).

Configuration is not in DATA_DIR — it comes from the environment and from a .env file read out of the process working directory. Back that up with your deployment config, separately.

A hot copy is not safe

Every database webhooker opens runs in WAL journal mode. The main and event databases are also held open for the entire process lifetime — WebhookDBManager caches event database handles and closes them only on webhook deletion or shutdown — so "it looked idle" is not a guarantee that nothing was mid-transaction.

That means cp, rsync, tar or a filesystem snapshot taken against a running instance can capture a database and its -wal at two different instants and yield a file that is corrupt or missing state. Copying a .db on its own is worse and fails loudly: recently written pages, including the schema itself on a young database, live in the -wal, so the copy reads back as an empty or table-less database. Use one of the two procedures below instead.

Stop, copy, start. The simplest, needs no extra tooling, and the only one that gives a single point in time across every file:

docker stop webhooker
cp -a /path/to/data /path/to/backup-$(date -u +%Y%m%dT%H%M%SZ)
docker start webhooker

SQLite online backup. No downtime, one file at a time:

for db in /path/to/data/*.db; do
    sqlite3 "$db" ".backup '/path/to/backup/$(basename "$db")'"
done

.backup reads through the WAL and writes a single consistent file with no sidecars of its own, so the destination is complete as it stands. Two caveats. First, the runtime image is alpine:3.21 with only ca-certificates and su-exec added — the sqlite3 CLI is not in it, so run this on the host against the volume path, or from a throwaway container that mounts the volume. Second, each file is captured at its own instant, so a webhook created or an event delivered between two files being copied lands in one and not the other. If you need the whole set coherent as of a single moment, stop the service.

Note that sqlite3 <db> .dump is not one of these procedures: it is an export, it holds a read transaction open for as long as it runs, and it pins the WAL against checkpointing for that whole time. It is safe to run — it does not block ingestion — but back up with .backup or a stopped copy.

Archive databases are the one exception the service is built for: the archive writer closes and reopens its handle around writes (debounced to at most one reopen per second), so an operator can move an archive-….db away for offline retention while the service runs, and it is recreated on the next write. The webhook page names each database target's archive file. See Database Architecture. That is a move-the-file-away workflow, not a substitute for the backup procedures above.

Move the sidecars with it. Under WAL that workflow is no longer a single file, and the common case is the dangerous one. The reopen happens on the next write after the debounce window elapses, so after the last write of a burst nothing checkpoints: measured, 20 s after ten events the archive-….db was 4096 bytes — a header, no table — with all ten rows sitting in a 189 KB -wal. Copying the .db alone at that moment yields a file that opens with no such table: archived_events. The file becomes self-contained again when the handle closes, which happens on the next write past the debounce window, when the connection pool retires the idle connection (about a minute after the last write), or at the idle archive sweep — measured, the same file was a complete 20 KB .db with no sidecars about a minute after its last write. A clean stop closes it too. So either move the archive-….db together with any -wal/-shm beside it, or wait until there are none.

Restore

1. Stop the service.

2. Restore the whole set together: webhooker.db and every events-*.db and every archive-*.db. A restore that leaves out webhooker.db or an events-*.db is reported, not refused; one that leaves out an archive-*.db or a -wal (step 3) is not reported at all. Every database is opened mode=rwc, so a missing events-{uuid}.db is created empty: the webhook comes back with its configuration intact and its entire event history gone. The first start after the restore logs created a new, empty database at WARN for each such file, with its path, as it does for a missing webhooker.db. A missing archive-*.db is recreated at its target's next delivery without a warning, since moving one away is a supported workflow. Event databases restored without webhooker.db are simply orphaned; nothing references their UUIDs.

3. Carry any *.db-wal and *.db-shm files that are in the backup. They are part of the database, and dropping a -wal silently discards every transaction it still holds. An .backup set will not contain any: it writes a single consolidated file per database. A stop-and-copy set normally has none, because a clean stop closes every database and checkpoints its sidecars away; the exception is an archive not opened since a crash. A copy salvaged from a crashed instance has them for everything, and needs all of them.

4. Start the service. The container gives the directory and the restored files to the webhooker user before the app starts, whoever restored them (see Running with Docker). AutoMigrate runs against each restored database as it is opened.

Upgrades

Back up before every upgrade. Every start runs GORM AutoMigrate unconditionally against whatever files it finds:

• the main database on connect — Setting, User, APIKey, Webhook, Entrypoint, Target
• each event database when it is lazily opened — Event, Delivery, DeliveryResult, EventTotals, TargetTotals, EntrypointTotals
• each archive database on every open and reopen

There is no schema version table, no migration ledger, and no down migrations. Nothing in the files records which version wrote them, and no code path undoes a migration.

Upgrade procedure:

1. Stop the service.
2. Back up DATA_DIR using one of the procedures above.
3. Pull the new image and start it.
4. Confirm database migrations completed in the logs before putting traffic back on it.
5. Confirm the new build is the one running: curl -s http://host:8080/.well-known/healthcheck reports the version it was stamped with (see Version stamping).

Upgrading past the introduction of BIND_ADDRESS: earlier versions always bound every interface. Container deployments are unaffected — the image ships ENV BIND_ADDRESS=0.0.0.0, so a docker run or Compose service that worked before still works with nothing changed.

A bare binary is the case that changes: the listener now binds 127.0.0.1 unless BIND_ADDRESS says otherwise, so a deployment that relied on reaching it from another host becomes unreachable until it sets the address the proxy connects to. Check the bindAddress field of the startup log to see what a running process bound. See Bind address.

Downgrade is unsupported. Once a newer binary has migrated the files there is no way to move them back. AutoMigrate is additive — it adds tables, columns and indexes and never drops or rewrites them — so an older binary will generally open migrated files and appear to work while writing against a schema it does not know about. The failure mode is silent divergence, not a startup error. The only supported way back to an older version is restoring the pre-upgrade backup, which discards everything received since that backup was taken.

Version stamping

The binary reports its version at /.well-known/healthcheck (the version field), in the UI footer, and in the startup log line (msg=starting, version=...). It is also the Sentry release name, as webhooker-{version}. The value is stamped in at build time by the linker; it is not read from a file at runtime, so it identifies the build itself.

script/version produces the value and both build paths use it:

Build	What it reports
Clean checkout at a tag	exactly that tag, e.g. v1.0.0
Commits past a tag	v1.0.0-3-g1a2b3c4 — tag, commits since, short SHA
No tag reachable	the short SHA, e.g. 1a2b3c4
Uncommitted changes	the above with a -dirty suffix
No git metadata	unknown

The image derives it the same way, from the .git that the build context carries, so any docker build . of a clone, with no build arguments, stamps the commit it was built from; a shallow clone of one branch has no tags and stamps the short SHA. .dockerignore must therefore leave out neither .git nor any tracked file, which git in the build would see as deleted, marking the version -dirty. It does leave .git/config, which can hold a remote URL carrying a credential and which git describe does not need, out of a directory context. A context sent as a tar is not filtered by .dockerignore, so it carries .git/config unless its sender leaves it out; for upaas, that is sneak/upaas#274. git in the build reads the checkout whoever owns its files, since a context sent as a tar archive keeps the sender's owners and git otherwise refuses a checkout owned by another user. A VERSION build arg (--build-arg VERSION=...) takes precedence; script/docker (and so make docker) passes the one script/version resolves on the host. The image build fails if its context carries .git and the version still comes out unknown, which means git is missing from the build or could not read the checkout.

unknown is what a source tarball, or a docker build with no .git in its context and no VERSION build arg, reports. A build that reports unknown is a build nobody told what it was; it is not a failure, but it cannot be traced back to a commit.

make version prints what the current checkout would stamp, and make build VERSION=v1.2.3 overrides it. An empty override — from make build VERSION= or from --build-arg VERSION= — means unset rather than "", and resolves the way an absent one does.

Nothing that varies between two builds of the same commit is stamped — no timestamp, no hostname, no builder identity — so two builds of one commit still produce a byte-identical binary.

Backups contain secrets

Treat a backup with the same care as the credentials inside it. Encrypt backups at rest and restrict who can read them.

• events-{uuid}.db and archive-….db hold the full payload body and headers of every event as received, including whatever the sending service put in them — tokens, signatures, personal data.
• Event databases written before issue #206 was fixed also contain target credentials: a GORM association upsert on the delivery and retry write path copied targets rows, config included, into the per-webhook database. For a Slack target the webhookUrl is the bearer credential, and an http target's URL can embed userinfo. This version never writes those rows; the first time it opens such a file it deletes them and vacuums the file, which removes the credential bytes rather than only unlinking the rows. Deleting alone would not: the bytes stay readable in the file's free pages until it is rewritten. The sweep is recorded in the file's user_version only once the vacuum returns, so a sweep that fails or is interrupted fails the open and is retried on the next one, and a file this version has opened without error holds no leaked rows and no recoverable bytes from them. On upgrade this rewrites each existing events-{uuid}.db once, on its first open. Two cases still hand over live delivery destinations: a backup taken from an older build, and a backup of a file this version has not yet opened successfully. Copies already made stay affected — the sweep only rewrites the file it opens, and freed blocks may persist in filesystem snapshots and on the underlying storage. Rotate any target credential that was in a backup you cannot account for.
• webhooker.db stores target config unencrypted, tracked at issue #212, next to the session encryption key and the Argon2id password hashes.

The entrypoint URL is the authentication secret

The entrypoint UUID is the credential, and it is the only one. webhooker mints a version 4 UUID per entrypoint and serves it at /h/{uuid}. Possession of that URL is the authentication: anyone who holds it can submit events to the entrypoint, and the receiver verifies nothing else about the sender.

There is no shared secret, no HMAC signature, no bearer token and no second factor on the receiver, and none will be added. This was considered and rejected; the implementation that existed was removed in PR #279, closing issue #67 and issue #241. A proposal to reintroduce any of them — including as "defence in depth" alongside the UUID — is answered by this section. Inbound signature headers a sender sends anyway (X-Hub-Signature and its per-provider equivalents) are stored and forwarded as ordinary headers; nothing checks them.

What that means for an operator:

• The URL is a capability, so treat it as a secret. Keep it out of logs, ticket bodies, chat messages and screenshots. Anyone who reads it anywhere can post events as that sender.
• Rotating means minting a new entrypoint, not changing a key. There is no way to rotate the UUID in place. To retire one, delete the entrypoint (or deactivate it, which answers 410) and create a new one, then point the sender at the new URL.
• A sender that cannot be given a secret URL is a constraint on that integration, not a reason to change this. If a service only supports signed payloads to a well-known URL, raise it as its own problem — pick a different integration path, or accept that it cannot be used. It is not grounds to reintroduce shared secrets.

Entrypoints

This repository adheres to the Scripts to Rule Them All standard: normalized scripts in script/ are the entrypoints for the development workflow. Thirteen of the Makefile's nineteen targets are thin shims that call them; build, run, dev, deps, clean and version are inline commands with no script behind them, though build, run and dev first run script/assets, and build and version both take their value from script/version.

script/test, make build and make dev each run script/assets first, which writes the ignored static/js/alpine.min.js (see Third-party browser assets), so make test, make check and the pre-commit hook work on a fresh clone without a separate step.

We provide:

• script/bootstrap — install all dependencies (idempotent)
• script/setup — make a fresh clone ready for development (bootstrap, then install-precommit)
• script/projectname — output the project name ("webhooker")
• script/assets — extract Alpine.js from its tarball in 3p/ (see Third-party browser assets)
• script/test — run the test suite
• script/test-browser — run the browser test in Docker (see Third-party browser assets)
• script/lint — run golangci-lint and ESLint in Docker (see Linting below)
• script/fmt — format the Go code and, in Docker, the Markdown (writes)
• script/fmt-check — check formatting (read-only)
• script/css — regenerate static/css/tailwind.css in Docker (writes; see Stylesheet)
• script/css-check — fail if static/css/tailwind.css differs from what script/css would generate (read-only)
• script/check — run test, lint, fmt-check, and css-check
• script/version — output the version to stamp into the binary (see Version stamping)
• script/docker — build the Docker image tagged via script/projectname, passing script/version's output in as the VERSION build arg
• script/cibuild — CI entrypoint: docker build . (the Dockerfile runs the checks, so a green build implies a green repo)
• script/ci-mark-superseded — CI helper: mark the commits whose run a newer push cancelled (see CI gate honesty)
• script/precommit — pre-commit checks (go mod tidy guard, then script/check)
• script/install-precommit — install the git pre-commit hook that runs script/precommit

Third-party browser assets

The web UI serves one third-party script, Alpine.js, in its CSP build: the npm package @alpinejs/csp. The pages' Content-Security-Policy forbids eval, which the standard alpinejs build needs to run the expressions written in the markup. The CSP build runs no expressions, so every Alpine directive in templates/ only names a property or method of a component registered in static/js/app.js: x-data="collapsible" and @click="toggle", never x-data="{ open: false }" or @click="open = !open".

A browser test in internal/server loads the webhook page, its edit pages and the event log under the real policy and checks that: the add entrypoint form stays hidden until Add is clicked; for every target type, the targets section's Add shows only a choice of type with Next and Cancel, Next shows only that type's fields (no url field for database or log), Cancel at either step closes the form, and saving adds the target; a refused target comes back with its form open, the values entered and the reason, and after Cancel the next Add starts with an empty form and no reason; a refused save on the target edit page and on the webhook edit page comes back with the reason and every value entered; the Copy button beside an entrypoint URL reads "Copied" once clicked; an entrypoint's Edit button shows its edit form in place of its description and hides until the form closes, Cancel hides the form and drops what was typed, as does leaving the page and going back to it, and Save changes the description; of the recent events on the webhook page only the newest starts expanded, each expands and collapses, and Open leads to the event's own page; of the events in the event log only the newest starts expanded, and an event there expands and collapses when its row's caret or its ID is clicked, and from the keyboard, but not when its ID is selected with the mouse, and a delivery's attempts inside it expand and collapse; and at phone width the menu button opens and closes the mobile menu. It also fails if the browser reports a console warning or error, an uncaught exception, or anything the policy refused. make check and the image build lint it but do not run it, and make test leaves it out (its file is built only with the browser build tag). Run it with make test-browser after changing templates/ or static/js/: that builds Dockerfile.browser, which runs the test in a digest-pinned headless browser image, so the host needs no browser.

The package's tarball is committed as 3p/alpinejs-csp-3.14.9.tgz, byte for byte as the npm registry publishes it. It is a dependency, not this repo's build output, so REPO_POLICIES.md's rule against committed build artifacts does not apply. The directory is 3p/ rather than vendor/ because Go treats a root vendor/ directory as its module vendor directory.

script/assets (make assets) extracts the browser build, package/dist/cdn.min.js, from the tarball to static/js/alpine.min.js, where go:embed picks it up. script/test, make build and make dev run it first, and the Dockerfile builds through make test and make build, so nothing downloads Alpine.js. The extracted file is not committed, and .dockerignore keeps any host copy out of the build context. static/static.go names every file it embeds, so a build that skips the extraction, such as a bare go build, fails with an error naming js/alpine.min.js.

To move to a new version: download https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz, check it against the dist.integrity hash listed at https://registry.npmjs.org/@alpinejs/csp/<version>, replace the tarball in 3p/ with it as alpinejs-csp-<version>.tgz, update its file name in script/assets, and run make check and make test-browser.

Stylesheet

static/css/tailwind.css is generated by Tailwind and committed. To change the styles, edit the templates, static/js/app.js, internal/handlers/recent_events.go or static/css/input.css, run make css, and commit the regenerated file with the change. Tailwind takes classes only from the files that input.css names in its @source lines; a class written in any other file is not generated until that file is named there too. make check and the image build fail when the committed file differs from what make css generates. static/css/style.css is hand-written and is not generated.

make css runs the Tailwind standalone CLI in Docker, at the version and sha256 pinned in the Dockerfile's stylesheet stages; it is never installed on the host. To move to a new version, change the version in both download URLs and both sha256 sums, taken from the release's sha256sums.txt, then run make css and commit the result.

Rationale

Webhook integrations between services are inherently fragile. The receiving service must be online when the webhook fires, most webhook senders provide no built-in retry mechanism, and there is no standard way to inspect what was sent, when it was sent, or whether delivery succeeded.

webhooker solves this by acting as a durable intermediary:

1. Reliable ingestion — webhooker is always ready to accept incoming webhooks. It stores every received event before attempting any delivery, so nothing is lost if downstream targets are unavailable.

2. Guaranteed delivery — Events are queued for delivery to each configured target. Failed deliveries are retried with configurable backoff. Every delivery attempt is logged with status codes, response bodies, and timing.

   That guarantee is at-least-once, not exactly-once. When a send reaches its target but the write recording that outcome fails, the delivery is deliberately left in a recoverable state rather than marked done — losing a delivery is the worse failure — so the pending sweep picks it up about fifteen minutes later, or the next restart does, and the target receives a payload it already got. webhooker adds no delivery identifier of its own to an outbound request, so make your receiver idempotent against whatever the payload itself carries.

3. Observability — Full request/response logging for every webhook received and every delivery attempted. Prometheus metrics expose volume, latency, and error rates. The web UI provides real-time visibility into event flow.

4. Fan-out — A single incoming webhook can be delivered to multiple targets simultaneously. This enables patterns like forwarding a GitHub webhook to both a deployment service and a Slack channel.

5. Replay and resubmit — Every received event is stored in full, and the event log offers two redelivery actions built on that. Replay re-sends one finished delivery to its own target, for recovering a delivery that failed. Resubmit re-injects the stored event as a new undelivered event and fans it out to every currently active target, for firing captured traffic at a backend under development. Both are web UI actions; there is no API for either.

Use Cases

• Store-and-forward with configurable retries for unreliable receivers
• Observability via Prometheus metrics on webhook frequency, payload size, and delivery performance
• Debugging and introspection of webhook payloads in the web UI
• Resubmit of captured webhook events for application testing and development, and replay of a single failed delivery for recovery
• Fan-out delivery of a single webhook to multiple downstream targets
• High-availability ingestion for delivery to less reliable backend systems

Design

Architecture Overview

webhooker is structured as a standard Go HTTP server following the sneak/prompts GO_HTTP_SERVER_CONVENTIONS. It uses:

• Uber fx for dependency injection and lifecycle management
• go-chi for HTTP routing
• GORM for database access with modernc.org/sqlite as the runtime SQLite driver. Note: gorm.io/driver/sqlite transitively depends on mattn/go-sqlite3, which requires CGO at build time (see Docker section)
• slog (stdlib) for structured logging with TTY detection (text for dev, JSON for prod)
• gorilla/sessions for encrypted cookie-based session management
• gorilla/csrf for CSRF protection (cookie-based double-submit tokens)
• go-chi/httprate for sliding-window rate limiting of the password-change and webhook receiver endpoints. The bucket is per client IP only when TRUSTED_PROXIES covers the reverse proxy (by default it covers the RFC 1918 private ranges); otherwise every client behind that proxy shares one bucket per limit. The login endpoint counts failed attempts itself instead, so that a correct password is never throttled (see Rate Limiting)
• Prometheus for metrics, served at /metrics behind basic auth
• Sentry for optional error reporting

Naming Conventions

The codebase uses consistent naming throughout (rename completed in issue #12):

Entity	Description
Webhook	Top-level configuration entity grouping entrypoints and targets
Entrypoint	A receiver URL where external services POST events
Target	A delivery destination for events

Data Model

webhooker's data model has eleven entities organized into two tiers: the application tier (user and webhook configuration) and the event tier (event ingestion, delivery, and logging).

┌─────────────────────────────────────────────────────────────┐
│                    APPLICATION TIER                          │
│                  (main application database)                 │
│                                                             │
│  ┌──────────┐       ┌──────────┐       ┌──────────────┐     │
│  │   User   │──1:N──│  Webhook │──1:N──│  Entrypoint  │     │
│  │          │       │          │       │              │     │
│  │          │       │          │──1:N──│   Target     │     │
│  │          │       └──────────┘       └──────────────┘     │
│  │          │──1:N──│  APIKey  │                             │
│  └──────────┘       └──────────┘                             │
│                                                             │
│  ┌──────────┐                                               │
│  │ Setting  │  (key-value application config)               │
│  └──────────┘                                               │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│                      EVENT TIER                              │
│            (per-webhook dedicated databases)                 │
│                                                             │
│  ┌──────────┐       ┌──────────┐       ┌─────────────────┐  │
│  │  Event   │──1:N──│ Delivery │──1:N──│ DeliveryResult  │  │
│  └──────────┘       └──────────┘       └─────────────────┘  │
│                                                             │
│  ┌──────────────┐  (one row: running counts of events)      │
│  │ EventTotals  │                                           │
│  └──────────────┘                                           │
│  ┌──────────────┐  (one row per target: running counts      │
│  │ TargetTotals │   of its deliveries)                      │
│  └──────────────┘                                           │
│  ┌──────────────────┐  (one row per entrypoint: when the    │
│  │ EntrypointTotals │   last event arrived on its URL)      │
│  └──────────────────┘                                       │
└─────────────────────────────────────────────────────────────┘

Setting

A key-value pair for application-level configuration that is auto-managed rather than user-provided. Used to store the session encryption key and any future auto-generated settings.

Field	Type	Description
key	string	Primary key (setting name)
value	text	Setting value

Currently stored settings:

• session_key — Base64-encoded 32-byte session encryption key, auto-generated on first startup.

User

A registered user of the webhooker service.

Field	Type	Description
id	UUID	Primary key
username	string	Unique login name, at most 1024 bytes so that it fits in the session cookie
password	string	Argon2id hash (never exposed via API)

Relations: Has many Webhooks. Has many APIKeys.

Passwords are hashed with Argon2id using secure defaults (64 MB memory, 1 iteration, 4 threads, 32-byte key, 16-byte salt). On first startup, an admin user is created with a randomly generated 16-character password printed once to stdout; webhooker resetpw sets it again if it is lost (see The admin account). Every one of those paths hashes through the same internal/database code, so the parameters cannot drift between them.

Webhook

The top-level configuration entity. A webhook groups together one or more entrypoints (receiver URLs) and one or more targets (delivery destinations) into a logical unit. A user creates a webhook to set up event routing.

The new webhook form can also give the webhook its first targets: an optional HTTP target URL creates an http target named HTTP, and the archive checkbox creates a database target named Archive whose expiry is the archive expiry chosen beside it (never, 1h, 12h, 24h, 30d, 90d or 365d) and whose rotation is the rotation chosen below that (none, monthly, daily or hourly). Both are validated as on the add target form, and the webhook and its targets are created together or not at all.

Field	Type	Description
id	UUID	Primary key
user_id	UUID	Foreign key → User
name	string	Human-readable name
description	string	Optional description
retention_days	integer	Days to retain events (default: 30; 0 means retain forever)

Relations: Belongs to User. Has many Entrypoints. Has many Targets.

The retention_days field controls how long event data is kept in the webhook's dedicated database before automatic cleanup.

Setting retention_days to 0 means "retain events forever". Because the column carries a default of 30, a literal zero cannot survive an insert, so a zero is rewritten on save to a sentinel of 365 * 1000 days (database.RetentionForeverDays). The retention reaper recognises that sentinel and skips the webhook entirely, and the web UI displays such a webhook's retention as "forever" rather than as a day count.

Submitted retention_days values therefore fall into three bands, not two:

• 1 up to database.MaxFiniteRetentionDays (106751 days, about 292 years) is accepted as a finite retention.
• Above that ceiling but below the retain-forever sentinel of 365000 (database.RetentionForeverDays) is rejected with a 400. This is the band the cap exists for.
• 0, and 365000 or above, are accepted and mean retain forever, collapsing to the sentinel — 0 in Webhook.BeforeSave, the large values in parseRetentionDays. The large values are not out of range: the edit form pre-fills the sentinel for a retain-forever webhook, so submitting that form back unchanged has to keep meaning "forever".

A negative value is in none of the three: parseRetentionDays rejects it with a 400 before BeforeSave ever sees it.

The cap is not arbitrary: the reaper computes its cutoff as a time.Duration, an int64 nanosecond count, and a longer period overflows it. An overflowed cutoff lands in the future, where it matches every row, so the sweep would delete every event the webhook has instead of none. The reaper also clamps the value it is given, so a row written by an older version cannot trigger that either.

Entrypoint

A receiver URL where external services POST webhook events. Each entrypoint has a unique UUID-based path. When an HTTP request arrives at an entrypoint's path, webhooker captures the full request and creates an Event.

Field	Type	Description
id	UUID	Primary key
webhook_id	UUID	Foreign key → Webhook
path	string	Unique bare UUID, generated at creation. The /h/ prefix is route only and is not stored: the receiver matches this column against the raw {uuid} path segment. It is also the entrypoint's credential; see The entrypoint URL is the authentication secret
description	string	Optional description
active	boolean	Whether this entrypoint accepts events (default: true)

Relations: Belongs to Webhook.

A webhook can have multiple entrypoints. This allows separate URLs for different event sources that all feed into the same processing pipeline (e.g., one entrypoint for GitHub, another for Stripe, both routing to the same targets).

The webhook page shows, for each entrypoint, when the last event arrived on its URL, which retention leaves in place, or "never" if none ever has, and how many events arrived on it within the webhook's retention period. A resubmitted event did not arrive on the URL and counts in neither.

Target

A delivery destination for events. Each target defines where and how events should be forwarded.

Field	Type	Description
id	UUID	Primary key
webhook_id	UUID	Foreign key → Webhook
name	string	Human-readable name
type	TargetType	One of: http, slack, database, log
active	boolean	Whether deliveries are enabled (default: true)
config	JSON text	Type-specific configuration
max_retries	integer	Total delivery attempts for http and slack targets, not retries on top of the first: 0 is a single fire-and-forget attempt with no retries and no circuit breaker, and a value of N makes N attempts in all, with exponential backoff and a per-target circuit breaker. Ignored by database and log targets. The web UI labels it Delivery attempts

Relations: Belongs to Webhook. Has many Deliveries.

Target types:

• http — Forward the event as an HTTP POST to a configured URL. max_retries is the total number of delivery attempts, not retries on top of the first: when max_retries is 0 (the default), the target operates in fire-and-forget mode, a single attempt with no retries and no circuit breaker; a value of N makes up to N attempts in all, retrying failed deliveries with exponential backoff and protecting them with a per-target circuit breaker.
• slack — Post the event as a formatted message to a Slack-compatible incoming webhook URL (webhookUrl in config). It is built on the same HTTP core as http and honours max_retries identically, circuit breaker included. See the Slack target section under "Per-Webhook Event Databases" for the message format.
• database — Archive the full event as a row into the target's own archive database (archive-{webhook_name}-{target_name}-{target_uuid}.db) for long-term retention, with an optional creation-validated expiry (default: keep forever) and rotation (default: none, one file). The new webhook form, the add target form and the target edit form all offer the same expiries: never, 1h, 12h, 24h, 30d, 90d or 365d, and the same rotations: none, monthly, daily or hourly. The target list shows the expiry in plain units, such as "30 days", and the rotation. No external delivery and no retries; an archive write failure fails the delivery. See the database target section under "Per-Webhook Event Databases" for the full semantics. The web UI calls this type an archive: its badge, the add target form's type list and the target edit page say so, and its settings are labelled Archive expiry and Archive rotation.
• log — Write the event to the application log (stdout). Useful for debugging.

The config field stores type-specific configuration as JSON (e.g., destination URL, custom headers, timeout settings).

http target configuration:

Key	Type	Description
url	string	Destination the event is POSTed to
headers	object	Extra request headers, applied last so they win over the event's own forwarded headers
timeout	integer (sec)	Per-target request timeout; unset (or 0) uses the shared 30-second client timeout

timeout is capped at 300 seconds, and the form rejects anything above it rather than substituting the cap. A delivery attempt holds one of the bounded pool's workers for its whole duration, so an unbounded timeout would let a single unresponsive destination stall the queue.

headers rejects the names the delivery path or net/http writes regardless of what is configured: Host, Content-Length, Transfer-Encoding, Connection, Trailer and User-Agent. These are refused at the form rather than accepted and ignored, because a stored header that provably never reaches the wire tells the operator their configuration took effect when it did not. Content-Type is not reserved: a configured one deliberately overrides the event's.

Redirects. A redirect from an http target's destination is followed, up to ten hops, and the delivery's recorded status and body come from the final hop. One rule governs every header the delivery carries for someone else — the configured headers and the inbound event headers forwarded from the sender alike: a hop that leaves the origin the target names carries none of them. Leaving the origin means a different host, a different port, or a step down from https to http. Both classes routinely carry a secret — a configured X-Api-Key or PRIVATE-TOKEN, an inbound X-Hub-Signature — and an open redirect at the destination would otherwise hand it to a host the operator never chose. net/http already does this for Authorization and Cookie. The delivery path's own headers (Content-Type, User-Agent) are not origin-scoped and always travel, so a body preserved across a 307 is still typed. A 301, 302 or 303 is a different matter, and this is net/http's behaviour rather than webhooker's: the POST becomes a GET and the event body and its Content-Type are dropped, so the destination the chain ends at receives no event at all — and the delivery is still recorded Delivered on that hop's 2xx. Redirects within the target's own origin keep everything, so a destination that redirects its own paths is unaffected; the drop is per hop rather than permanent, so a chain that returns to the configured origin carries the headers again, exactly as net/http treats Authorization. Each hop is dialled through the same SSRF guard as the first, so a redirect aimed at a private or reserved address is refused at connect time.

APIKey

A programmatic access credential for API authentication.

Field	Type	Description
id	UUID	Primary key
user_id	UUID	Foreign key → User
key	string	Unique API key value
description	string	Optional description
last_used_at	timestamp	Last time this key was used (nullable)

Relations: Belongs to User.

Event

A captured incoming webhook request. Stores the complete HTTP request data for auditing, for replay, and for resubmission.

Field	Type	Description
id	UUID	Primary key
webhook_id	UUID	Foreign key → Webhook
entrypoint_id	UUID	Foreign key → Entrypoint
method	string	HTTP method of the captured request. Always POST: the receiver answers every other method with 405 before an Event is created
headers	JSON	Complete request headers
body	text	Raw request body
content_type	string	Content-Type header value
body_bytes	integer	The body's size in bytes, recorded when the event is stored, on receipt and on resubmit
resubmitted_from_id	UUID	The event this one was copied from by a resubmit (nullable; empty for an event that arrived on the receiver). Not a foreign key: the source event can be reaped by retention while its copies remain

Relations: Belongs to Webhook. Belongs to Entrypoint. Has many Deliveries.

When a request arrives at an entrypoint, the full request (method, headers, body) is captured as an Event. The event is then queued for delivery to every active target configured on the parent webhook.

Delivery

The pairing of an event with a target. Tracks the overall delivery status across potentially multiple attempts.

Field	Type	Description
id	UUID	Primary key
event_id	UUID	Foreign key → Event
target_id	UUID	Foreign key → Target
status	DeliveryStatus	One of: pending, delivered, failed, retrying
finished_at	timestamp	When the delivery became delivered or failed (nullable; empty while pending or retrying)
replay	boolean	Whether the delivery was created by Replay

Relations: Belongs to Event. Belongs to Target. Has many DeliveryResults.

Delivery statuses:

• pending — Created but not yet attempted.
• retrying — At least one attempt failed; more attempts remain.
• delivered — Successfully delivered (at least one attempt succeeded).
• failed — All retry attempts exhausted without success.

Replay. A delivered or failed delivery is finished as far as the engine is concerned, but the event is still stored, so the event log offers a per-delivery Replay action for it. Replay creates a NEW pending delivery for the same event and target and hands it to the engine on the ordinary path — same retries, same SSRF guard, same circuit breaker as a first attempt. It never touches the delivery it repeats: that row's status, timestamps and recorded attempts stand as the record of what happened. The new delivery records replay, and the event log and the event's page label it a replay.

What is re-sent is the stored event body, against the target's configuration as it stands now — the point of a replay is to deliver where the destination has since been fixed. A target that has been deleted or deactivated therefore refuses the replay with a message on the event log rather than delivering from stale configuration, and a replay is refused while an earlier one for the same event and target is still pending or retrying. A delivery whose target has been deleted shows no Replay action at all: recreating the target makes a new one that the old delivery does not name, so Resubmit is how that event reaches the webhook's currently active targets.

Resubmit. Replay recovers one delivery; resubmit re-injects one EVENT. The event log offers a per-event Resubmit action that stores a NEW event copying the stored one's method, headers, body and content_type verbatim, then fans it out to the webhook's currently active targets — resolved fresh by the same query the receiver uses, so a target created long after the original event arrived receives it. That is the difference that matters: a target added to test a backend under development has no prior delivery, so there is nothing to replay to it, while a resubmit reaches it like any other active target. Inactive targets are skipped, exactly as the receiver skips them.

The new event is a first-class event in the log with its own deliveries, not a marker on the one it came from, and the original's deliveries are left untouched. It records resubmitted_from_id, and the event log shows the relationship both ways, so a captured event fired twenty times at a backend stays traceable. Resubmitting the same event repeatedly is supported and is the point of the action — there is no in-flight refusal; the route's own rate limit is what bounds it.

DeliveryResult

The result of a single delivery attempt. Every attempt (including retries) is individually logged for full observability.

Field	Type	Description
id	UUID	Primary key
delivery_id	UUID	Foreign key → Delivery
attempt_num	integer	Attempt number (1-based)
success	boolean	Whether this attempt succeeded
status_code	integer	HTTP response status code (if applicable)
response_body	text	Response body (if applicable)
error	string	Error message (on failure)
duration	integer	Request duration in milliseconds

A database or log target sends no HTTP request, so in the event log and on the event's page its attempts show no status: a successful one reads "archived" or "written to the log".

The event log and the event's page show when each attempt was recorded and when each delivery was created, as the recent events list shows when an event arrived: how long ago, with the full UTC time on hover.

Relations: Belongs to Delivery.

EventTotals, TargetTotals and EntrypointTotals

Running counts in each event database, read by the statistics pane at the top of the webhook page and by the webhook list, and each entrypoint's last event, read by the webhook page's entrypoint list. EventTotals is one row:

Field	Type	Description
events	integer	Events ever stored, resubmitted copies included
events_removed	integer	Events retention has deleted
last_event_at	timestamp	When the newest event arrived (nullable; empty before the first); retention leaves it as it is

TargetTotals is one row per target, created by the first delivery to it:

Field	Type	Description
target_id	UUID	The target (primary key)
deliveries	integer	Deliveries to it ever created, replays included
delivered	integer	Of those, how many became delivered
failed	integer	Of those, how many became failed
deliveries_removed	integer	Its deliveries retention has deleted
failed_removed	integer	Its failed deliveries retention has deleted

EntrypointTotals is one row per entrypoint, created by the first event that arrives on its URL:

Field	Type	Description
entrypoint_id	UUID	The entrypoint (primary key)
last_event_at	timestamp	When the newest event arrived on its URL; a resubmitted event leaves it as it is, and so does retention

Each count changes in the transaction that writes or deletes the rows it counts, and each last_event_at in the transaction that stores the event. The pane's lifetime events are events, and its lifetime deliveries and failures are deliveries and failed summed over the targets; each figure within retention is the same less what retention removed, so neither needs the rows themselves. Its last event is last_event_at in EventTotals, so it still shows once retention has removed every event; each entrypoint's last event, from EntrypointTotals, does too. Its last-10-minutes and last-24-hours figures are counted from the events and deliveries indexes over just that window, the deliveries in one query grouped by target. Its failure percentage for a window is the deliveries that became failed in it out of all that became delivered or failed in it, and a dash when none did.

The webhook list at /hooks shows three of the pane's figures for each webhook: its events within retention and its last event, both from EventTotals, and its deliveries that failed in the last 24 hours, counted with the pane's query. It opens each webhook's event database once (the handle stays open) and runs those two reads there, so its cost grows with the number of webhooks and, for each, with the deliveries that finished in the last 24 hours, never with the events stored.

The target list on the webhook page shows, for each target, its delivered and failed totals, which retention does not reduce, and its deliveries that became delivered and failed in the last 24 hours, counted with the pane's query. Deliveries still pending or retrying count in neither.

Event-tier indexes

These indexes on the per-webhook event databases are declared in the model tags, so AutoMigrate creates them on a fresh database:

Table	Columns	Serves
deliveries	status, deleted_at, finished_at, target_id	Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, the webhook page's statistics and target list and the webhook list, which count each target's deliveries by status and when they finished, and the event log, which lists and counts the events with a failed delivery or one pending or retrying
deliveries	event_id, deleted_at	The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events
delivery_results	delivery_id, deleted_at	The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events
events	deleted_at, created_at	The webhook page's statistics, which count recent events
events	resubmitted_from_id, deleted_at	The event log, which counts the events resubmitted from each event on a page
events	entrypoint_id, deleted_at, resubmitted_from_id, created_at	The webhook page's entrypoint list, which counts the events that arrived on each entrypoint's URL within the retention period
events	created_at	Retention, which selects expired events by age

GORM's soft delete adds deleted_at IS NULL to these queries; retention leaves it out. SQLite keeps no statistics on these tables, and without them it rates the deleted_at index, which every live row matches, above an index on a column matched against several values or compared with a range. So every index but the last also covers deleted_at. It comes second in the event_id and delivery_id indexes, so that retention can use them without it. The event log's count, the one query on the resubmitted_from_id index, always carries deleted_at IS NULL and uses both columns. The entrypoint list's count, the one query on the entrypoint_id index, uses all four, resubmitted_from_id IS NULL leaving out resubmitted copies and created_at last because it compares it with a range (>=). In the statistics' events index deleted_at comes first, because they compare created_at with a range (>=) and SQLite narrows by a range only on the last column it uses.

Common Fields

Every entity except Setting, EventTotals, TargetTotals and EntrypointTotals includes these fields from BaseModel. Setting is a bare key-value row with no id, no timestamps and no soft delete. Of the three totals tables, event_totals holds counts and last_event_at, keyed by a numeric id; target_totals holds counts, keyed by target_id; and entrypoint_totals holds last_event_at, keyed by entrypoint_id:

Field	Type	Description
id	UUID	Auto-generated UUIDv4 primary key
created_at	timestamp	Record creation time
updated_at	timestamp	Last modification time
deleted_at	timestamp	Soft-delete timestamp (nullable; GORM soft deletes)

Database Architecture

Per-Webhook Event Databases

webhooker uses separate SQLite database files: a main application database for configuration data and per-webhook databases for event storage. All database files live in the DATA_DIR directory.

Every one of them is created 0600, and so is each -wal and -shm sidecar. See Running with Docker for what that does and does not protect.

Main Application Database ({DATA_DIR}/webhooker.db) — stores configuration and application state:

• Settings — auto-managed key-value config (e.g. session encryption key)
• Users — accounts and Argon2id password hashes
• Webhooks — webhook configurations
• Entrypoints — receiver URL definitions
• Targets — delivery destination configurations
• APIKeys — programmatic access credentials

On first startup the main database is auto-migrated, a session encryption key is generated and stored, and an admin user is created.

Per-Webhook Event Databases ({DATA_DIR}/events-{webhook_uuid}.db) — each webhook gets its own dedicated SQLite file containing:

• Events — captured incoming webhook payloads
• Deliveries — event-to-target pairings and their status
• DeliveryResults — individual delivery attempt logs
• EventTotals, TargetTotals and EntrypointTotals — running counts of the above, the deliveries per target, and each entrypoint's last event, kept through retention

Per-webhook databases are created automatically when a webhook is created. They are managed by the WebhookDBManager component, which handles connection pooling, lazy opening, migrations, and cleanup.

A per-webhook database that is missing or zero-length later means its webhook's events and pending deliveries are gone. The next time it is opened, an empty one is created in its place, so the webhook keeps receiving, and created a new, empty database is logged at WARN with the file's path. Every webhook's database is opened when the service starts, so this appears at the latest at the first start after the file was lost. A file there that SQLite cannot open fails that webhook alone, with an ERROR naming the webhook on every access and a 500 to its senders, so one damaged file does not stop the others.

This separation provides:

• Isolation — a high-volume webhook won't cause lock contention or journal growth affecting the main application or other webhooks.
• Independent lifecycle — event databases can be independently backed up, archived, rotated, or size-limited without impacting the application.
• Clean deletion — removing a webhook and all its history is as simple as deleting one file. Configuration is soft-deleted in the main DB; the event database file is hard-deleted (permanently removed).
• Per-webhook retention — the retention_days field on each webhook controls automatic cleanup of old events in that webhook's database only, or disables cleanup entirely when set to 0 (retain forever).
• Performance — each webhook's database has its own page cache and its own lock, so concurrent event ingestion across webhooks won't contend. Every database — main, per-webhook, and archive — is opened through one code path (internal/database/sqlite_open.go) in WAL journal mode, with a 10-second busy timeout, BEGIN IMMEDIATE transactions, and a bounded connection pool. Under WAL a reader never blocks a writer, so an operator reading a database does not stall event ingestion into it.

The database target type builds on this architecture to provide long-term archiving, separate from the per-webhook event database (which may prune events under its own retention). Delivering to a database target writes the full event — body, headers, method, content type, and webhook/entrypoint/event identifiers — as a row into the target's own archive database, archive-{webhook_name}-{target_name}-{target_uuid}.db, stored under the data directory beside the event database. Each database target has its own archive file, even when one webhook has several.

Both names are made safe for a file name the same way: lowercased, ASCII letters and digits kept, every other run of characters turned into a single -, no - at either end, cut to 40 characters, and unnamed when nothing is left. The target UUID keeps the file name unique. A webhook named Orders (EU) with a target named Long-term archive archives into archive-orders-eu-long-term-archive-{target_uuid}.db.

An optional rotation in the target's config JSON (e.g. {"rotation":"daily"}) is none, the default, which keeps the one file, or monthly, daily or hourly. A target that rotates writes each event to a file named for the period of the event's receive time, in UTC, put before the .db: archive-orders-eu-long-term-archive-{target_uuid}-2026-10.db monthly, …-2026-10-01.db daily and …-2026-10-01-19.db hourly. Each file holds exactly its period's events, and the first event of a new period starts the next file, so a finished period's file can be moved away like any archive. A changed rotation applies from the next event: the files already written keep their names and stay, pruned, shown and downloaded with the rest, since every file named for the target is its archive, whichever rotation wrote it.

Renaming the webhook or the target renames every one of the target's files, each keeping its period, under the same lock the archive writes and the archive sweeper take. Webhook edits, target edits and target creation run one at a time, so no edit can rename the files between another's rename and save, and the names on disk match the UI. A rename never replaces a file: if one already has a new name, the edit is refused with an error naming that file, nothing is moved, and the stored name stays. If the archive is not there (the operator moved it away), the rename is not an error, and the next write creates the file under the new name.

The file is moved just before the new name is saved. If the process stops between the two, the archive is left under the new name while the UI still shows the old one, and the next delivery starts a second archive under the name shown. To bring them back together, stop the service before moving anything, and move each archive as its .db together with any -wal and -shm beside it, since the -wal can hold rows that are not yet in the .db. If no file has the name shown, move the archive under the new name back to it. If a second archive already has the name shown, move the archive under the new name out of the data directory instead and keep it as you would any archive moved away. Then start the service again.

After each write the archive handle is closed and reopened, debounced to at most once per second, so an operator can move the archive file away for offline archiving without stopping the service; a moved or removed archive file is recreated automatically on the next write. A zero-length archive file is written to as a new archive: SQLite opens it as an empty database, so it holds nothing to lose. An optional expiry in the target's config JSON (e.g. {"expiry":"720h"}) is validated when the target is created — the default (unset or the literal never) keeps rows forever — and rows older than the expiry are pruned each time the archive is (re)opened. An archive write failure is never silent success: the delivery records a failed attempt with the error and is marked failed.

Because reopens only happen on writes, an archive whose target has stopped receiving events would never be pruned. A background archive sweeper closes that gap: on the same interval as the event retention reaper (RETENTION_SWEEP_INTERVAL) it prunes every archive whose database target declares a positive expiry, whether or not the target is still receiving traffic. The sweep never creates an archive — a target whose archive file does not yet exist is skipped, not initialised — it takes the same per-target lock the write path uses, so it can never interleave with a write, and it leaves the archive closed afterwards so the move-the-file-away workflow keeps working. Archives with no expiry, or the expiry never, are not touched by the sweep at all.

For a target with several files, the write path prunes only the file it writes to, and the sweep prunes every one of them, taking the target's lock for one file at a time, so a write to the target waits for at most one file's prune. A file that is gone by the time the sweep reaches it is skipped. A file named for a period that the sweep leaves empty is deleted, with any -wal and -shm beside it; the file without a period is kept even when empty, as it always has been.

Because each database target has its own archive file, a target's expiry governs only its own archive. Two database targets on one webhook with different expiries keep two archives, each pruned on its own schedule.

The webhook page shows, for each database target, the name of the archive file an event received now would go to, the size on disk of all the target's archive files together, how many there are when there is more than one, and when the latest of them was last written. The size counts each .db and its -wal together, and the last write is the latest of their modification times, since a write lands in the -wal first. All are read from the files' metadata; the archive is never opened. While the named file does not exist — before the first write, before the first event of a new period, and after the file has been moved away — the page shows not created yet beside the name.

Each database target on the webhook page has a Download button, which returns its archive as one gzipped JSON file, archive-{webhook_name}-{target_name}-{YYYYMMDDTHHMMSSZ}.json.gz, the names made safe as above and the time in UTC. The file holds one object: webhook and target, each an id and a name; exported_at; and archived_events, one object per archived row with every column, keyed by column name. The rows come from every one of the target's files: the file without a period first, then the others in the order of their periods, oldest first, and each row from a file named for a period has that period beside its columns. A body that is not valid UTF-8 is written in base64, with "body_encoding": "base64" beside it. An archive that does not exist yet, or was moved away, downloads with an empty archived_events; the download never creates a file.

The download streams: each row is read and written out compressed before the next is read, so neither the archive nor the JSON is held in memory. When it starts it lists the target's files by the stored names, under the lock that webhook edits, target edits and target creation hold, and lets go. It then opens one file at a time, only when its rows are about to be written out, and closes it before it opens the next, so it never has more than one of the target's files open. To open each, it takes the lock again just long enough to find the file by its period under the names stored then, so a rename during the download loses no file; a file that is gone by then, emptied by the sweep or moved away, is skipped. Each file is read on a connection of its own inside one read-only transaction, so it is written out as it stood when it was opened, and archive writes go on meanwhile, since under WAL a reader never blocks a writer. Until the open file is closed its -wal cannot be checkpointed past what the download reads, so a long download lets that -wal grow.

Deleting a webhook releases its archives: the delivery engine's cached archive writers are dropped and their file handles closed, so nothing lingers after the webhook is gone. The archive files themselves are deliberately left on disk. Unlike the event database — per-webhook working storage that is hard-deleted with the webhook — an archive is long-term storage an operator may still want to keep or move away for offline retention, and destroying it as a side effect of deleting a webhook would be unrecoverable. Removing an archive-….db is the operator's call. Deleting a database target releases its writer the same way, and for the same reason leaves its file alone.

The Slack target type sends webhook events as formatted messages to any Slack-compatible incoming webhook URL (works with Slack, Mattermost, and other compatible services). Each message includes event metadata (HTTP method, content type, timestamp, body size) and the payload pretty-printed in a code block. JSON payloads are automatically formatted with indentation for readability; non-JSON payloads are shown as raw text. Large payloads are truncated to keep messages reasonable. Config stores webhookUrl — the Slack/Mattermost incoming webhook endpoint. That is the JSON key; the error text for a missing one reads webhook_url is required, which is the message, not the key.

The database uses the modernc.org/sqlite driver at runtime, though CGO is required at build time due to the transitive mattn/go-sqlite3 dependency from gorm.io/driver/sqlite.

Request Flow

External Service
       │
       │  POST /h/{uuid}
       ▼
┌─────────────┐     ┌──────────────┐     ┌──────────────┐
│  chi Router │────►│  Middleware   │────►│   Webhook    │
│             │     │  Stack       │     │   Handler    │
└─────────────┘     └──────────────┘     └──────┬───────┘
                                                │
                     1. Look up Entrypoint by UUID — 404 if unknown,
                        410 if inactive
                     2. Read the body under the 1 MB cap
                     3. Capture full request as Event
                     4. Create Delivery records for each active Target
                     5. Build self-contained delivery.Task structs
                        (target config + event data inline for
                         bodies < 16 KiB)
                     6. Notify Engine via channel (no DB read needed)
                                                │
                                                ▼
                                        ┌──────────────┐
                                        │  Delivery    │◄── retry timers
                                        │  Engine      │    (backoff)
                                        │  (worker     │
                                        │   pool)      │
                                        └──────┬───────┘
                                               │
                              ┌── bounded worker pool (N workers) ──┐
                              ▼                ▼                    ▼
                       ┌────────────┐   ┌────────────┐      ┌────────────┐
                       │ HTTP Target│   │ HTTP Target│      │ Log Target │
                       │(max_retries│   │(max_retries│      │ (stdout)   │
                       │   == 0)    │   │   > 0,     │      └────────────┘
                       │ fire+forget│   │ backoff +  │
                       └────────────┘   │ circuit    │
                                        │ breaker)   │
                                        └────────────┘

Bounded Worker Pool

The delivery engine uses a fixed-size worker pool (default: 10 workers) to process all deliveries. At most N deliveries are in-flight at any time, preventing goroutine explosions regardless of queue depth.

Architecture:

• Channels as queues: Two buffered channels serve as bounded queues: a delivery channel (new tasks from the webhook handler) and a retry channel (tasks from backoff timers). Both are buffered to 10,000.
• Fan-out via channel, not goroutines: When an event arrives with multiple targets, each delivery.Task is sent to the delivery channel. Workers pick them up and process them — no goroutine-per-target.
• Worker goroutines: A fixed number of worker goroutines select from both channels. Each worker processes one task at a time, then picks up the next. Workers are the ONLY goroutines doing actual HTTP delivery.
• Retry backpressure with DB fallback: When a retry timer fires and the retry channel is full, the timer is dropped — the delivery stays in retrying status in the database. A periodic sweep (every 60s) scans for these "orphaned" retries and re-queues them. No blocked goroutines, no unbounded timer chains.
• Bounded concurrency: At most N deliveries (N = number of workers) are in-flight simultaneously. Even if a circuit breaker is open for hours and thousands of retries queue up in the channels, the workers drain them at a controlled rate when the circuit closes.

This means:

• No goroutine explosion — even with 10,000 queued retries, only N worker goroutines exist.
• Natural backpressure — if workers are busy, new tasks wait in the channel buffer rather than spawning more goroutines.
• Independent results — each worker records its own delivery result in the per-webhook database without coordination.
• Graceful shutdown — cancel the context, workers finish their current task and exit. The stop hook waits for the pool via lifecycle.WaitForShutdown, which bounds that wait by fx's stop timeout rather than blocking forever on a wedged worker. On timeout it logs at ERROR and returns an error, and the goroutines that did not finish are still running — an unclean shutdown is reported rather than hidden.

Recovery paths:

1. Startup recovery: When the engine starts, it scans all per-webhook databases for pending and retrying deliveries. Pending deliveries are sent to the delivery channel; retrying deliveries get backoff timers scheduled.
2. Periodic retry sweep (DB-mediated fallback): Every 60 seconds the engine scans for retrying deliveries whose backoff period has elapsed. This catches "orphaned" retries — ones whose in-memory timer was dropped because the retry channel was full. The database is the durable fallback that ensures no retry is permanently lost, even under extreme backpressure.

Changing a target's type does not migrate in-flight deliveries. Only http and slack targets own durable retries; database and log targets are fire-and-forget and never produce a retrying delivery. If a target's type is edited from a retrying type to a non-retrying (or unknown) one while one of its deliveries is still retrying, both recovery paths above terminally mark that delivery failed and record a DeliveryResult naming the current target type as the reason, logging it at warn level. The delivery is not re-dispatched under the new type — the operator never asked for that delivery — and the event itself remains stored in the per-webhook event database. Nothing is lost: the delivery is failed, which is terminal, so the event log offers Replay on it to re-send it to that target under the new type, and Resubmit on the event to re-inject it to every currently active target.

Circuit Breaker (HTTP and Slack Targets with Retries)

http and slack targets with max_retries > 0 are protected by a per-target circuit breaker that prevents hammering a down target with repeated failed delivery attempts. The circuit breaker is in-memory only and resets on restart (which is fine — startup recovery rescans the database anyway).

States:

State	Behavior
Closed	Normal operation. Deliveries flow through. Consecutive failures are counted.
Open	Target appears down. Deliveries are skipped and rescheduled for after the cooldown.
Half-Open	Cooldown expired. One probe delivery is allowed to test if the target has recovered. Other deliveries are rescheduled for one whole cooldown later.

Transitions:

     success              ┌──────────┐
   ┌────────────────────► │  Closed  │ ◄─── probe succeeds
   │                      │ (normal) │
   │                      └────┬─────┘
   │                           │ N consecutive failures
   │                           ▼
   │                      ┌──────────┐
   │                      │   Open   │ ◄─── probe fails
   │                      │(tripped) │
   │                      └────┬─────┘
   │                           │ cooldown expires
   │                           ▼
   │                      ┌──────────┐
   └──────────────────────│Half-Open │
                          │ (probe)  │
                          └──────────┘

Defaults:

• Failure threshold: 5 consecutive failures before opening
• Cooldown: 30 seconds in open state before probing

Scope: Circuit breakers apply to http and slack targets with max_retries > 0. The Slack target is built on the same HTTP core and hands its own max_retries to the same retry path, so it gets a breaker with the same 5-failure / 30-second defaults. Fire-and-forget targets of either type (max_retries == 0), database targets (local operations), and log targets (stdout) do not use circuit breakers.

When a circuit is open and a new delivery arrives, the engine marks the delivery as retrying and schedules a retry timer for after the remaining cooldown period. This ensures no deliveries are lost — they're just delayed until the target is healthy again. A delivery already in retrying keeps that status without another database write each time the breaker turns it away.

While a target's breaker is open, the target's row on the webhook page says its deliveries are paused until the cooldown ends, in UTC and as a time from now. Each of its retrying deliveries shows as waiting in the event log and on the event's page, with the earliest it can be tried next: the later of the cooldown's end and the end of its own backoff after its last attempt. It is only the earliest: when the cooldown ends, one of the target's waiting deliveries is sent to test it while the others wait at least one more cooldown, as the row also says. A time not on the current UTC day is shown with its date. While the breaker is half-open, the row says instead that deliveries are held while one delivery tests whether the target has recovered, with no time, and the target's deliveries show their plain status, since any of them may be the one being sent.

Metrics

/metrics serves one Prometheus registry behind basic auth (see Infrastructure Endpoints). Alongside the inbound HTTP metrics recorded by the middleware, it exposes the delivery pipeline — the part of the service that can be failing while the receive side looks perfectly healthy, because it is: events are arriving and being stored, they are just not getting anywhere.

Metric	Type	Meaning
webhooker_events_received_total	counter	Events received and durably stored. Compare against the delivery counters on one dashboard
webhooker_delivery_attempts_total	counter	Delivery attempts actually dispatched to a target. A delivery a circuit breaker refused is not one: it is counted as a retry instead, but only when the refusal moves it into retrying
webhooker_deliveries_succeeded_total	counter	Deliveries that reached delivered
webhooker_deliveries_failed_total	counter	Deliveries that failed terminally and will not be retried
webhooker_delivery_retries_total	counter	Deliveries put back into retrying
webhooker_delivery_replays_total	counter	Deliveries an operator replayed from the event log. A replay runs the ordinary engine path, so it also moves the attempt, outcome and duration series; this is the only one that separates it from ordinary traffic
webhooker_events_resubmitted_total	counter	Stored events an operator re-injected from the event log. The new event also moves webhooker_events_received_total, since it is a stored event the delivery side is compared against; this counter is what separates the two. Unlabelled: the target types it fans out to belong to the delivery series
webhooker_delivery_duration_seconds	histogram	Wall time of a single dispatched delivery attempt, the same duration the attempt's DeliveryResult records
webhooker_deliveries_pending	gauge	Deliveries currently in pending
webhooker_deliveries_retrying	gauge	Deliveries currently in retrying
webhooker_circuit_breakers_open	gauge	Circuit breakers currently open

Every delivery metric carries exactly one label, target_type, and cardinality is the whole reason for that restriction. The two event-level counters carry no label at all — an event is not the property of any one target type. A target type is one of four compile-time constants, so the label domain is bounded by construction; a value outside that set collapses to unknown rather than minting a series of its own. Target ids, event ids and entrypoint ids are deliberately not labels: they are UUIDs minted per operator action or per inbound request, a series is never reclaimed once it exists, and labelling by any of them would make /metrics a memory leak that grows with traffic.

The two queue-depth gauges are counted out of the databases by a sampler that runs every 30 seconds for as long as the delivery engine does, rather than tracked as deltas alongside the status transitions: a delta would have to be seeded at startup from rows a previous process wrote, and would drift permanently on any transition that failed to persist.

Those two gauges also publish an unknown series, from startup rather than on first occurrence. Deliveries queued against a target that has since been deleted are counted there: that backlog is the one nobody is watching, so it is the one that must not silently vanish from the gauge. The outcome counters move only after the status change has been written, so a transition the database rejected is never reported as an outcome that happened.

Inbound HTTP metrics

The middleware records three more on the same registry:

Metric	Type	Labels
http_request_duration_seconds	histogram	service, handler, method, code
http_response_size_bytes	histogram	service, handler, method, code
http_requests_inflight	gauge	service, handler

Two of those labels are written once per request from bytes the client chose, so both are bounded to something this service registers:

• handler is the chi route pattern — /h/{uuid}, never the concrete path. A request matching no route carries (unmatched), and no entrypoint UUID ever reaches a label.
• method is the request method when the router can route it, and (unmatched) otherwise. net/http accepts any RFC 9110 token as a method, so the raw value bounds the label at nothing; the nine chi matches routes for stay distinguishable, and a token that could only ever have produced a 405 does not get a series of its own.

The other two are not request-controlled: code is the status one of this service's own handlers wrote, and service is a fixed empty string.

http_requests_inflight is deliberately aggregate — its handler is always (all), one series counting the requests in flight across the whole service. The gauge is incremented before routing and decremented after the handler returns, and the route pattern exists only between those two moments, so labelling it by pattern would increment one series and decrement another, leaving every pattern permanently off by the number of requests it served.

Rate Limiting

Global blanket rate limiting middleware (e.g., a per-IP throttle shared with the web UI) must not apply to webhook receiver endpoints. Webhook endpoints receive automated traffic from external services at unpredictable rates, and blanket limits shared with other routes would cause legitimate deliveries to be dropped.

The receiver instead has its own dedicated abuse limit, scoped to the /h/{uuid} route only and keyed per client IP per request path (httprate.KeyByEndpoint): one misbehaving sender is throttled without affecting other senders of the same entrypoint or the same sender's other entrypoints. Keying on the path rather than on the entrypoint matters — see the aggregate limit below. The limit is RECEIVER_RATE_LIMIT requests per minute (default 120, generous for legitimate webhook senders). Requests over the limit receive HTTP 429 with a Retry-After header. A set-but-invalid RECEIVER_RATE_LIMIT value aborts startup rather than silently falling back to the default.

A second limit sits in front of that one, keyed on the client IP alone and covering the whole route at ten times RECEIVER_RATE_LIMIT requests per minute (default 1200). The per-entrypoint limit needs it: the route pattern matches any single path segment, so a client that invents a fresh path per request gets a fresh per-entrypoint bucket every time and would otherwise have no aggregate limit at all — while each of those requests still costs an entrypoint lookup before it 404s. The aggregate limit leaves room for one address to drive several entrypoints at their full rate, and it is not configurable separately.

What that aggregate limit bounds is the database work an invented path costs; log volume it caps rather than eliminates. A path that names no entrypoint is recorded by the handler at DEBUG, and the aggregate limiter logs its own rejections at DEBUG and without the path, so neither appears at all under the default level. The per-entrypoint limiter is the loud one: it logs every rejection at WARN with the request path, which on this route is attacker-controlled text. A client hammering a single invented path is served RECEIVER_RATE_LIMIT requests and has the rest of its aggregate budget rejected there, so the aggregate limit is what bounds the number of those WARN lines — to under ten times RECEIVER_RATE_LIMIT per minute per client IP, 1080 at the defaults, where before it there was no bound at all. Their width is bounded by the field budgets below, the same ones the access log spends. The access log is bounded by neither limit: every request is recorded once at INFO, served or rejected alike.

What the access log does bound is the content of those lines. A 3xx or 4xx response logs the chi route pattern — /h/{uuid}, /user/{username}//, or the literal (unmatched) when the request hit no route at all — in place of the concrete URL. Those are the outcomes an unauthenticated client can drive for free: 404 and 429 on any invented receiver path, a login redirect on any invented profile path. Logging the URL there would let a flood write text of its own choosing, at a length of its own choosing, into the log. 2xx and 5xx responses keep the concrete path — a success resolved against a static route or against the operator's own data (on the receiver, against a stored entrypoint UUID), and a 5xx is a bug in this service, where the exact path is the evidence and no client can provoke one at will.

The query string is never logged; it is replaced by the fixed marker ?(redacted). It is client-chosen on every route, and /.well-known/healthcheck and /s/* answer 200 to anyone with no rate limiter in front of them, so a query on a fixed 200 URL would otherwise buy the same amplification as an invented path. Nothing debuggable is lost: the only query parameters this service reads are the sign-in page's next, the page to return to, notice, which names the line a page shows after an action, and the event log's show, which picks the events it lists.

Client-supplied request content does not leave the host by the other route either. The Sentry SDK attaches the request to every event it captures, independently of the access log, and SendDefaultPII=false does not cover all of what it copies: the raw query string and the first 10 KiB of the request body are both taken unconditionally, the body precisely because these handlers call ParseForm. A BeforeSend hook therefore replaces the query string and the body with (redacted), drops cookies and the remote-address environment, and reduces the headers to a fixed allowlist — Accept, Content-Length, Content-Type, Host, Origin, Referer, User-Agent and X-Request-Id.

The same hook rewrites the request URL. The SDK builds it as scheme://host/path from the concrete path, which on the receiver route is /h/<uuid> in full — and that UUID is a write capability, not an identifier: anyone holding it can post events this service accepts and its targets then deliver. A tracker has its own retention, access control and deletion policy, so the rule the access log follows above does not carry across that boundary. What is sent is the chi route pattern instead: http://host/h/{uuid}.

The scheme and the host are kept, and everything else in the URL is discarded rather than edited, so a future SDK version that starts appending a query string cannot widen this. The scheme has to survive for the reason given below. The host is whatever the request's Host header carried — this service validates no hostname, so on a directly exposed deployment a client sets it — and that same header is on the allowlist above, so scrubbing the host out of the URL would withhold nothing that is not sent anyway.

The body, the query string and the URL are all handled on every route rather than filtered by route. For the URL that is also what keeps the event locatable: an error event is grouped by its exception and stack trace, not by its URL, so replacing the path with the pattern costs no grouping and the pattern still names the route in the UI. And an unconditional rule cannot leak on a route somebody forgets to add to it, which a route-conditional one can. For the body there is a second reason: nothing debuggable is lost, because every handler reads its fields with PostFormValue, so the body is exactly where the credentials are — the target destination URL, the login password, both password-change fields — and the one route whose body is genuine signal is the receiver, whose body is already stored on the event and served from the UI, so a tracker is not where anyone reads it.

The route is reachable from the hook only on the error dispatch. sentryhttp's recover path puts the request on the context it hands to RecoverWithContext, and the SDK carries that context through to BeforeSend as hint.Context, so hint.Context.Value(sentry.RequestContextKey) yields the live request and chi's RoutePattern() yields the matched pattern off it. The transaction dispatch has no such request: a finished span captures with a nil hint, which the client replaces with an empty one, so BeforeSendTransaction sees no context at all. Tracing is off in this service, so no transaction event is produced today, but the hook is installed on both dispatches as a floor.

Where the pattern is out of reach — the transaction dispatch, an event captured outside the router, or a request that matched no route — the fallback is never the concrete path. The path becomes the literal /(redacted), so the URL reads http://host/(redacted); a URL the rewrite cannot parse into a scheme is withheld whole. A transaction event additionally carries the SDK's own METHOD /path name, built from the concrete path as well; it is rewritten on the same terms, to POST /h/{uuid} where the pattern is known and POST /(redacted) where it is not.

The headers are an allowlist for the same reason the rules above are unconditional: the SDK's own filter removes four names and passes everything else, which would ship X-CSRF-Token and the shared secrets senders put on the receiver route. What survives still names the failing route — scheme, host, route pattern, method — and X-Request-Id ties the event to the local access log line that holds the rest. Nothing dropped is needed for the likeliest use, debugging a CSRF rejection. Its three inputs are the TLS decision, Origin and Referer; the latter two are kept, and the first is the scheme of the retained URL, because the SDK derives that scheme from r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https". That predicate is the SDK's own and is stricter than internal/reqtls.IsTLS, which this service now uses everywhere it decides transport: the SDK reports http for the HTTPS and https, http spellings reqtls accepts. Only a reported scheme is affected, no decision is, so it is left to the SDK rather than reimplemented. That is what the rewrite above preserves it for, and it is why dropping X-Forwarded-Proto costs nothing. The dropped provider headers (X-GitHub-Event, X-Gitlab-Event and the like) are real signal but are recorded locally on the event, and Sentry-Trace/Baggage are already reflected in the event's trace context.

The remaining client-supplied fields are truncated rather than dropped, each to a fixed budget: 512 bytes for url, useragent and referer, 128 for request_id (chi passes an inbound X-Request-Id header through), and 32 for method. A truncated User-Agent is still worth reading; an absent one is not. A cut value ends in [truncated], which is charged on top of the budget rather than inside it.

Each budget is spent in encoded bytes, not in the bytes the client sent. Every rune is charged what the wider of the two log handlers emits for it: two bytes for a quotation mark, a backslash or a tab; six for a non-printable rune below U+10000; ten for one at or above it, which the text handler spells \UXXXXXXXX. Go's header parser accepts all of them in a header value, so a budget counted raw would buy a field several times its nominal size — and the line, not the header, is what an operator has to store. Plain ASCII encodes one byte for one, so a real browser's User-Agent still fits whole; a value built out of escapes keeps a proportionally shorter prefix, which is the right trade.

Net: one INFO line per request, of at most 2,560 bytes. That ceiling is arithmetic, not an observation: 3 × (512 + 11) for url, useragent and referer, plus 128 + 11 for request_id, plus 32 + 11 for method, plus a 405-byte fixed portion (the field names, the punctuation, both timestamps at their longest, remoteIP and clientIP each charged as an IPv6 address with a zone, the status and the latency) — 2,156 bytes, stated at 2,560 so the figure has headroom. internal/middleware/accesslog_test.go asserts it against 8 KB of client-chosen text in the path, in the query, and in each of User-Agent, Referer, X-Request-Id and X-Forwarded-For, including cases built from the characters the handlers escape, and against a 5xx that keeps its concrete path while all three header fields are also at their budget and an X-Forwarded-For sent from a trusted proxy ends in an IPv6 client address at its longest followed by an 8 KB zone, where clientIP must name the address without the zone. Every case runs through both handlers internal/logger can select — the JSON one and the text one it installs on a tty — since the two do not escape alike and the ceiling is quoted unqualified.

Multiply that ceiling by the request rate to size log storage. Note that the rate is not bounded by the limits above on every route: /.well-known/healthcheck and /s/* sit behind no limiter, so there the multiplier is whatever the deployment will serve.

The same ceiling covers every other line the service writes through slog that carries text an unauthenticated client supplies, with one exception stated below it: the recovered-panic record, which carries a whole goroutine stack alongside its client-supplied fields and so has its own wider ceiling. The access log is not the only line a client can put its own text into, and a budget that held for one line and not the others would be worse than no stated budget at all. Every slog call an unauthenticated request can reach spends the same per-field budget through internal/logfield, and each carries strictly fewer client-supplied fields than the access log does, so none of them can be wider than it:

Log line	Level	Client-chosen value	Reachable unauthenticated
request body exceeds limit (413)	WARN	path, method	yes — MaxBodySize precedes RequireAuth
csrf: token validation failed (403)	WARN	path, method	yes — CSRF precedes RequireAuth
... rate limit exceeded (429)	WARN	path	yes, on the receiver
auth middleware: unauthenticated request	DEBUG	path, method	yes, by definition
entrypoint not found	DEBUG	entrypoint UUID	yes, on the receiver
user not found / invalid password	DEBUG	username	yes, on the sign-in form
login failure limit exceeded (429)	WARN	path	yes, on the sign-in form
password verification capacity exhausted	WARN	path	yes, on the sign-in form

DEBUG being off by default is not a bound. An operator turning it on to diagnose a flood must not thereby hand the flood an unbounded write, so those lines are capped too.

The last two rows are capped defensively rather than against a demonstrated width: chi routes POST /pages/login on a static pattern, so r.URL.Path there is the 12-byte constant /pages/login and each line lands near 120 bytes. RecordLoginFailure is nonetheless an exported method taking any *http.Request, and a future caller on a route with a URL parameter would widen the line. Since no request through the mux can, both caps are pinned by tests that call those two entry points directly with the path such a caller would supply. Removing either cap fails 14 subtests.

internal/middleware/logbound_test.go and internal/handlers/logbound_test.go drive 8 KB of client-chosen text at each of these — just under 1 KB at invalid password, whose accounts are shared with the successful-login line and so must stay within the 1024-byte username limit — through both handlers, and through seven fills: plain text as the baseline, and then the quotation mark, backslash, tab, newline, C0 control and astral non-printable, six characters the wider of the two handlers spends more on than the client spent sending them. Every case holds each line to the 2,560-byte ceiling. That per-line ceiling is what the figure above states, and every row establishes it.

Three of the sites go further and bound the whole flood's output — the total bytes a run of distinct invented values wrote, which is the shape an operator sizing storage cares about. They are request body exceeds limit (TestMaxBodySize_FloodOfOversizePathsDoesNotGrowTheLog), entrypoint not found and user not found (the last two through assertBoundedFlood). The other rows carry no aggregate assertion; the per-line ceiling is what they establish.

internal/logfield/logfield_test.go measures the per-rune charge against what the handlers really emit, over roughly 3,000 code points on each, so an undercharged rune fails a test rather than quietly falsifying the ceiling.

It covers GORM's statement logging as well. GORM's own default logger printed the fully interpolated SQL — parameters and all — to standard output on every statement that returned an error, including a plain record-not-found, at a level no operator setting reached. Two of this service's lookups miss by design on unauthenticated routes: the entrypoint lookup behind /h/{uuid} and the user lookup behind the sign-in form, whose path segment and submitted username the client picks outright. Every gorm.Open in the service now installs the adapter in internal/gormlog instead. It writes through the same slog logger as everything else, so its lines take the level the operator set and the handler internal/logger selected, and every value it emits is spent through the same internal/logfield budget. A record-not-found is not logged as an error: it is the expected outcome on both of those paths, and each handler already records its own miss at DEBUG — bounded, per the table above — without the SQL. Slow statements are kept, at WARN, above the same 200 ms threshold GORM used and with the statement bounded, because that report is the one thing GORM's logger gave an operator that nothing else here does. The adapter orders its cases exactly as GORM's own Trace orders them, so a statement that both missed and ran slow is still reported as slow, and dropping the miss costs an operator no report IgnoreRecordNotFoundError would have kept. A GORM line spends at most two of those budgets — the statement and the driver error — against a smaller fixed portion than the access log's, and internal/gormlog/gormlog_test.go asserts each line against MaxAccessLogLineBytes directly rather than leaving it as arithmetic.

The adapter also logs no bound value at all: it implements gorm.ParamsFilter and discards the parameters, so GORM renders the statement with its placeholders intact instead of substituting the values into it. That is a separate property from the size bound and it is what a bound is no substitute for — the session encryption key is 44 base64 characters and an Argon2id hash under 100, so both fit inside every budget above and a truncated secret is still a secret. It holds on all three arms of Trace, including the routine one an operator reaches at DEBUG, which is the only level at which a successful INSERT is written at all. One GORM path does not consult the filter — (*gorm.DB).Scan, which records the statement through GORM's own trace recorder. No production code path calls it; only tests do, and what a test binds is fixture data. internal/gormlog/scan_guard_test.go fails if a non-test file calls it. Pluck, Row and Raw all run through the normal callback processor and are filtered. See #### What DEBUG=true exposes under Configuration.

What that ceiling does not cover, stated here so the figure is not read as more than it is:

• Lines carrying an authenticated operator's own input, which are not truncated at all. webhook created logs the submitted name verbatim (internal/handlers/webhook_create.go) and target URL blocked by SSRF protection logs the target host (internal/handlers/target_create.go), as do the target_name lines in internal/delivery/engine.go and internal/delivery/target_http.go. The only bound on any of them is the 1 MB form body cap, so a 100 KB name writes a single line of roughly 600 KB — measured. This is deliberate: every one of these requires an authenticated operator on a service with no self-registration, and truncating the operator's own configuration echoed back would cost debuggability against no adversary. It does mean the 2,560-byte figure sizes unauthenticated traffic, not the operator's own administrative requests.
• The log delivery target, which writes the whole inbound event — headers and body — to the log. This one is deliberate: capping it would defeat the target, since emitting the payload is the delivery. It costs nothing unless an authenticated operator creates a target of that type on a specific webhook, and each line it writes is bounded per event by the 1 MB receiver body cap. Adding one is a decision to spend log volume on that webhook's payloads.
• The Go runtime, which does not go through internal/logger. The runtime writes an unrecovered panic or a fatal error itself, as plain text on standard error, and that output cannot be redirected. A panic in a background worker rather than in a request handler is the case that reaches it, since nothing recovers those. It carries no client-chosen value at a client-chosen length: the service's own panic calls are invariant guards over constants and over crypto/rand, apart from the one that hands http.ErrAbortHandler back to net/http, described below.
• A failure before fx's logger is built, such as an invalid configuration value. fx's logger takes the configuration, so when that fails fx's own console logger still prints the failure as plain text on standard error. Its values come from the operator's environment, not from a client.
• net/http's own faults, which are not a separate writer. internal/server/http.go builds its server with a nil ErrorLog, so net/http falls back to the log package's default logger — and internal/logger calls slog.SetDefault, which redirects that logger into whichever handler it installed. Those lines therefore arrive on standard output, shaped like every other line, at INFO. They are not truncated. A handler panic is no longer one of them: the recover middleware below answers it and writes it as the bounded record described there instead, and internal/server/recoverer_test.go requires that http: panic serving appear in neither of the process's two streams when a panic is driven through the production router. The one panic still handed back to net/http is http.ErrAbortHandler, which it special-cases and does not log at all. What is left on this path is net/http's own diagnostics, whose values are the runtime's, not a client's.

Wider than that 2,560-byte ceiling, and stated separately rather than carved out of it: the record a recovered panic produces. The recover middleware in internal/middleware answers 500 and writes one ERROR record through internal/logger carrying the panic value, the stack and the request id — the same request_id the access log line for that request carries, which is how the two are joined. It replaced chi's middleware.Recoverer, which on a current Go release crashed inside its own stack pretty-printer: the connection was dropped rather than answered, and what reached the operator described that crash rather than the fault behind it.

That record is bounded the same way, in the same encoded bytes and through the same internal/logfield budget: 512 for the panic value, because a handler is free to build one out of the request, 128 for the request id, which a client supplies outright through X-Request-Id, and 8,192 for the stack, cut at its far end so that the panic site survives a cut and net/http's accept frames are what is lost. Net: at most 10,240 bytes, once per recovered panic — 9,121 by the arithmetic (523 + 8,203 + 139 + a 256-byte fixed portion), stated at 10,240 for headroom.

Those two numbers are the claim; the measurements below only illustrate it. internal/middleware/recoverer_test.go drives all three growable fields past their budgets on one record, over both handlers, and measured 9,009 bytes on the JSON handler and 8,982–8,983 on the text one in one checkout. Neither is an invariant: the stack's own content decides where its cut lands, so the figures move by a byte or so between runs. The real case is far below both — through the shipped middleware chain the whole record measures roughly 3,960 bytes over a roughly 3,690-byte stack, taken by internal/server/recoverer_test.go from the process's own file descriptors while driving a panic through the production router over a real server in a subprocess. That pair moves further still, since debug.Stack() embeds absolute source paths and so depends on where the tree is checked out: four checkouts have reported 3,959, 3,961, 3,984 and 4,026. What the tests assert is the ceiling, that every client-supplied field was cut, and that the shipped chain's stack arrived uncut — never the numbers.

Every limiter here — receiver, login, password change, delivery replay and event resubmit — identifies the client the same way, through one shared key function: the connection's own address, unless the peer is inside TRUSTED_PROXIES, in which case the forwarded client address is used instead. That address becomes a bucket by family: IPv4 keys on the full address, IPv6 on its /64 prefix. A routed /64 is the normal residential and mobile IPv6 allocation, so keying IPv6 per address would let one subscriber rotate source addresses and mint a fresh bucket per request, evading these limits at the network layer without spoofing anything; the cost is that distinct clients inside one /64 share a bucket. IPv4-mapped addresses (::ffff:1.2.3.4) key as the IPv4 address they carry. See Trusted proxies. When that variable does not cover the reverse proxy, a client behind it shares one bucket with every other client behind the same proxy. Set TRUSTED_PROXIES to the proxy's address to get per-client limits back. What the shared bucket costs is not the same for every limiter, and the two cases pull in opposite directions:

• For the receiver limits it costs throughput, which is the safe direction to be wrong in: sharing can only make a limit bind sooner, never let a sender past it. It matters more for the aggregate limit than for the per-entrypoint one: with every request keyed on the proxy, the aggregate limit becomes a service-wide ceiling of 1200 requests per minute across all senders and all entrypoints, where the per-entrypoint limit's capacity still grows with the number of entrypoints.
• For the login and password-change limits it costs precision, not availability. Login failures from every client land in one counter, so a stranger's wrong passwords make the operator's own wrong passwords answer 429 sooner; the operator's correct password is never affected, because it is never counted.

The login endpoint

The login POST is the one endpoint with no pre-emptive limiter in front of it, and that is deliberate. A limiter that spends budget on arrival is a lockout wherever clients share one bucket, as they do behind a reverse proxy that TRUSTED_PROXIES does not cover: a stranger sending five POSTs a minute — about 0.08 requests per second, from anywhere — keeps it permanently full, and the operator has no second administrative path. So the handler inverts the order:

1. Credentials are verified first, and only a failed attempt spends budget. A correct password is never rate-limited, whatever the counters hold. This is what guarantees the admin UI stays reachable.
2. Failures are counted per (client bucket, submitted username), five per minute, after which further failures from that pair are answered 429 with a Retry-After. That 429 is a label on the response, not a gate in front of the work: the credential check has already run by the time the counter is consulted, so a throttled client's guess is still evaluated. See the guessing rate below. A successful login clears the counter, so mistyping a few times and then getting it right leaves you unthrottled. Because the submitted username is attacker-controlled, at most 1024 username counters and 1024 fallback address counters are tracked; past the first cap failures fall back to the address counter, and past both they are answered as throttled without being recorded. Total tracked state is under half a megabyte and does not grow with the number of usernames an attacker invents.
3. Concurrent password verifications are capped at two, and the queue for them at 16. Verifying before counting means every login request costs an Argon2id hash, and Argon2id here is 64 MB per hash — two slots is a 128 MB ceiling on password hashing. Every endpoint that hashes a password takes a slot, including the password-change endpoint, which holds one across both the verification and the new hash. A request that waits five seconds without getting a slot is answered 503 Service Unavailable and no hash is computed for it. The wait alone does not bound memory, only how long one request holds some, so the number of waiters is capped as well. Size the queue from what a parked waiter actually retains, not from the 1 MB body cap: that caps the raw body read, while the body-cap, CSRF and form-parsing middleware all run before the guard, so a waiter holds its parsed form plus its request header block for the whole wait. Measured on the pinned Go 1.26.1 toolchain, as the heap delta with 64 waiters parked in the handler, an ordinary two-field login form retains ~0 MB, a 1 MB urlencoded body at Go's 10,000-parameter parse cap retains 2.82 MB (3.09 MB with %41 escapes), and the ~0.9 MB of headers the 1 MB header cap allows takes it to 4.18 MB — the retained parse and the headers dominate, not the raw body. So the cap is 16 waiters: 16 x 4.18 MB is about 67 MB of committed queue memory, and two slots drain a full 16-deep queue in roughly 0.6 s, far inside the five-second deadline. A request arriving past the cap is shed with 503 immediately instead of joining the queue. Peak commitment for the endpoint is therefore about 203 MB: 128 MB of Argon2id, plus the 18 requests holding a parsed form — 16 queued and the 2 being hashed — at about 75 MB. That 203 MB is live commitment, not resident size: the Go collector lets the heap reach roughly twice the live set before collecting, with transient parse garbage on top of it. The independent review of this endpoint fired 18 adversarial requests at an idle guard and measured a peak HeapAlloc of 392 MB. Provision on the order of 400 MB, not for the 203 MB itemised here and not for the hashing budget alone.

An unknown username is verified against a dummy hash rather than rejected early, so a nonexistent account costs the same time as a real one and the response cannot be used to enumerate usernames.

This raises online guessing throughput by about 300x, and that is the trade. Because the credential check always precedes the counter, what bounds online brute force is the semaphore, not the failure counter. Two slots at the cost of one Argon2id verification is on the order of 27 guesses per second, about 2.3 million per day, against 5 per minute under the pre-emptive limiter this replaced. Treat that figure as a lower bound rather than a ceiling: it was measured with Go's race detector enabled, so real hardware verifies faster and guesses faster. Choose the admin password to survive millions of online guesses per day — a long random passphrase, not a memorable one. Rate-limiting POST /pages/login at the reverse proxy, where the real client address is visible, is the way to put a cheaper bound back on top.

The residual exposure is a bounded, self-clearing loss of login availability — not merely of latency. A flood can keep both verification slots busy, and a request that neither gets a slot within five seconds nor finds room in the queue is answered 503. Above roughly 27 requests per second the operator is not served slowly, it is shed: its chance per attempt is about the ratio of service rate to flood rate, so at 400 requests per second it is roughly one attempt in fourteen. A sufficiently determined flood still denies login for as long as it runs.

What changed is the price and the aftermath. Denying login used to cost an attacker 0.08 requests per second from anywhere; it now costs 30 or more sustained, about 400 times as much. Nothing accumulates while the flood runs, nothing needs resetting when it stops, and the operator's correct password succeeds on the first attempt afterwards. Restarting the service is not a remedy: a restart clears the failure counters, which are not what is saturated, and the flood re-fills both verification slots on its first two requests. The remedies are to block the source at the reverse proxy, or to rate-limit POST /pages/login there — the one place a limit can be applied without reintroducing the lockout, because the proxy sees the real client address. TRUSTED_PROXIES does not stop the saturation. The flood's source is in the clientIP field of webhooker's access log while TRUSTED_PROXIES covers the proxy, and in the proxy's own access log either way (see Trusted proxies).

Finer-grained per-webhook rate limits (configured in the web UI and enforced in the webhook handler) can layer on top of this env-level abuse limit later; they are tracked as future work.

API Endpoints

Public Endpoints

Method	Path	Description
GET	/	Root redirect, 303 (authenticated → /hooks, unauthenticated → /pages/login)
GET	/.well-known/healthcheck	Health check (JSON: status, now, uptimeSeconds, uptimeHuman, version, appname)
GET, HEAD	/s/*	Static file serving (embedded CSS, JS). GET and HEAD only — POST, PUT, PATCH, DELETE, OPTIONS, TRACE and CONNECT are answered 405 Method Not Allowed with Allow: GET, HEAD. Any other method (such as PROPFIND) is refused by chi before it reaches this route, and gets 405 without an Allow header. Pinned by TestStaticServesOnlyGetAndHead
POST	/h/{uuid}	Webhook receiver endpoint. POST only — every other method is answered 405 Method Not Allowed with Allow: POST. Rate limited (see Rate Limiting)

Authentication Endpoints

Method	Path	Description
GET	/pages/login	Sign-in page (not rate limited). Its next parameter names the page to return to after signing in; anything but a path on this site is replaced with /
POST	/pages/login	Sign-in form submission. On success, redirects to the form's next when it is a path on this site, otherwise to /. Credentials are verified before any limit is consulted, so a correct password is never throttled; 5 FAILED attempts per minute per bucket per submitted username, then 429. 503 if no verification slot frees up within 5s, or immediately if 16 requests are already queued for one (see Rate Limiting)
POST	/pages/logout	Sign out (destroys session)

Authenticated Endpoints

A signed-out GET of any of these is redirected to /pages/login with its path and query as next when they fit in 2048 bytes, so signing in returns to the page that was asked for.

Method	Path	Description
GET	/user/{username}	User profile page
POST	/user/{username}/password	Change the user's password (5 per minute per bucket, then 429; 503 if no verification slot frees up within 5s, or immediately if 16 requests are already queued for one)
GET	/settings	Read-only list of the configuration the server is running with; METRICS_PASSWORD and SENTRY_DSN show only as set or not set
GET	/hooks	List user's webhooks
GET	/hooks/new	Create webhook form
POST	/hooks/new	Create webhook submission
GET	/hook/{id}	Webhook detail view
GET	/hook/{id}/edit	Edit webhook form
POST	/hook/{id}/edit	Edit webhook submission
POST	/hook/{id}/delete	Delete webhook
GET	/hook/{id}/events	Full Event Log. ?show=failed lists only the events with a failed delivery, and ?show=pending only those with a delivery pending or retrying
GET	/hook/{id}/events/{eventID}	One event's own page: its details, the entrypoint it arrived at (for a resubmitted copy, the one the request it copies arrived at), its request headers, its whole body and every delivery of it
GET	/hook/{id}/events/{eventID}/body	Download an event's stored body. The pages show a body as text, cut at 32 KiB in the recent events and the event log, and leave a binary one out, so this is the only route that serves the stored bytes; it is offered wherever a body is cut or binary
POST	/hook/{id}/deliveries/{deliveryID}/replay	Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then 429)
POST	/hook/{id}/events/{eventID}/resubmit	Resubmit a stored event: creates a new event copying it and fans that out to every currently active target (30 per minute per bucket, then 429)
POST	/hook/{id}/entrypoints	Add entrypoint to webhook
POST	/hook/{id}/entrypoints/{entrypointID}/edit	Change an entrypoint's description; its URL stays the same
POST	/hook/{id}/entrypoints/{entrypointID}/delete	Delete an entrypoint
POST	/hook/{id}/entrypoints/{entrypointID}/toggle	Enable or disable an entrypoint
POST	/hook/{id}/targets	Add target to webhook
GET	/hook/{id}/targets/{targetID}/edit	Edit target form. The one page that renders a target's destination URL and header values in full, rather than masked
POST	/hook/{id}/targets/{targetID}/edit	Edit target submission
GET	/hook/{id}/targets/{targetID}/download	Download a database target's archive as one gzipped JSON file. See Database Architecture
POST	/hook/{id}/targets/{targetID}/delete	Delete a target
POST	/hook/{id}/targets/{targetID}/toggle	Enable or disable a target

Infrastructure Endpoints

Method	Path	Description
GET	/metrics	Prometheus metrics, behind basic auth. The route is registered only when METRICS_USERNAME and METRICS_PASSWORD are both set; with neither set it does not exist and returns 404, and with only one set the process refuses to start

API (Planned)

Method	Path	Description
GET	/api/v1/webhooks	List webhooks
POST	/api/v1/webhooks	Create webhook
GET	/api/v1/webhooks/{id}	Get webhook details
PUT	/api/v1/webhooks/{id}	Update webhook
DELETE	/api/v1/webhooks/{id}	Delete webhook
GET	/api/v1/webhooks/{id}/events	List events for webhook
POST	/api/v1/events/{id}/redeliver	Redeliver an event

None of these exist yet. /api/v1 is mounted with no routes, so every path under it returns 404 today. API authentication will use API keys passed via Authorization: Bearer <key> header; no Bearer middleware is implemented either.

Package Layout

All application code lives under internal/ to prevent external imports. The entry point is cmd/webhooker/main.go.

webhooker/
├── 3p/
│   └── alpinejs-csp-3.14.9.tgz  # Alpine.js CSP build npm package, extracted by make assets
├── cmd/webhooker/
│   └── main.go               # Entry point: subcommand dispatch; no args locks DATA_DIR and wires fx
├── internal/
│   ├── banner/
│   │   └── banner.go          # Ruled block for the one credential shown in the clear
│   ├── ciscript/
│   │   └── doc.go             # Tests for the CI shell scripts in script/; no runtime code
│   ├── resetpw/
│   │   └── resetpw.go         # `webhooker resetpw`: set an account's password, stopped deployments only
│   ├── config/
│   │   ├── config.go          # Configuration loading from environment variables
│   │   └── testing.go         # ClearEnvForTest: an empty environment for one test
│   ├── database/
│   │   ├── base_model.go      # BaseModel with UUID primary keys
│   │   ├── database.go        # GORM connection, migrations, admin seed
│   │   ├── models.go          # AutoMigrate for config-tier models
│   │   ├── model_setting.go   # Setting entity (key-value app config)
│   │   ├── model_user.go      # User entity
│   │   ├── model_webhook.go    # Webhook entity
│   │   ├── model_entrypoint.go # Entrypoint entity
│   │   ├── model_target.go    # Target entity and TargetType enum
│   │   ├── model_event.go     # Event entity (per-webhook DB)
│   │   ├── model_delivery.go  # Delivery entity (per-webhook DB)
│   │   ├── model_delivery_result.go  # DeliveryResult entity (per-webhook DB)
│   │   ├── model_totals.go    # EventTotals, TargetTotals and EntrypointTotals (per-webhook DB)
│   │   ├── model_apikey.go    # APIKey entity
│   │   ├── password.go        # Argon2id hashing and verification
│   │   ├── retention.go       # Retention reaper (per-webhook event expiry)
│   │   ├── testing.go         # NewTestDatabase: wrapper for tests, no fx lifecycle
│   │   └── webhook_db_manager.go  # Per-webhook DB lifecycle manager
│   ├── datadir/
│   │   └── lock.go            # Exclusive advisory lock on DATA_DIR (one instance)
│   ├── globals/
│   │   └── globals.go         # Build-time variables (appname, version, arch)
│   ├── gormlog/
│   │   └── gormlog.go         # GORM's logger.Interface on top of slog, bounded
│   ├── logfield/
│   │   └── logfield.go        # Encoded-byte budget for client-supplied log values
│   ├── delivery/
│   │   ├── engine.go          # Event-driven delivery engine (channel + timer based)
│   │   ├── circuit_breaker.go # Per-target circuit breaker for http/slack targets with retries
│   │   ├── target.go          # Target interface, Task, Scheduler
│   │   ├── target_http.go     # HTTP target (retries, circuit breaker)
│   │   ├── target_slack.go    # Slack/Mattermost incoming-webhook target
│   │   ├── target_database.go # Database archive target
│   │   ├── target_database_archive.go  # Archive file lifecycle and pruning
│   │   ├── target_database_rotation.go # Archive rotation and file names
│   │   ├── target_database_export.go   # Archive download as gzipped JSON
│   │   ├── target_log.go      # Log target (stdout)
│   │   ├── target_config_view.go  # Masked target config for templates
│   │   ├── archive_sweeper.go # Periodic pruning of idle archives
│   │   ├── queue_depth.go     # Periodic sampler behind the queue-depth gauges
│   │   ├── url_mask.go        # Strips credentials from *url.Error
│   │   └── ssrf.go            # SSRF prevention (IP validation, safe HTTP transport)
│   ├── handlers/
│   │   ├── handlers.go        # Base handler struct, JSON helpers, template rendering
│   │   ├── auth.go            # Login, logout handlers
│   │   ├── delivery_replay.go # Per-delivery replay: new delivery, current target config
│   │   ├── event_resubmit.go  # Event resubmit: new event, all currently active targets
│   │   ├── entrypoint_view.go # Masked entrypoint view for templates
│   │   ├── event_log_view.go  # Event log projection, byte-capped in SQL
│   │   ├── healthcheck.go     # Health check handler
│   │   ├── index.go           # Index page handler
│   │   ├── profile.go         # User profile handler
│   │   ├── settings.go        # Read-only Settings page handler
│   │   ├── webhook_list.go    # Webhook list page
│   │   ├── webhook_create.go  # Webhook create
│   │   ├── webhook_detail.go  # Webhook detail page
│   │   ├── webhook_edit.go    # Webhook edit, archive renaming
│   │   ├── webhook_delete.go  # Webhook delete, event database and archive writer removal
│   │   ├── event_log.go       # Event log page: loaders, filters, delivery views
│   │   ├── entrypoint.go      # Entrypoint create, edit, delete and toggle
│   │   ├── target_create.go   # Target create, per-type config builders
│   │   ├── target_delete.go   # Target delete
│   │   ├── target_toggle.go   # Target toggle
│   │   ├── shared.go          # Helpers shared by several handlers
│   │   └── webhook.go         # Webhook receiver handler
│   ├── healthcheck/
│   │   └── healthcheck.go     # Health check service (uptime, version)
│   ├── lifecycle/
│   │   └── lifecycle.go       # Shared stop-hook waiter, bounded by the stop context
│   ├── logger/
│   │   └── logger.go          # slog setup with TTY detection; fx's event logger
│   ├── metrics/
│   │   └── metrics.go         # Delivery Prometheus collectors, labelled by target type
│   ├── middleware/
│   │   ├── middleware.go      # Logging, CORS, Auth, Metrics, MetricsAuth, SecurityHeaders, MaxBodySize
│   │   ├── csrf.go            # CSRF protection middleware (gorilla/csrf)
│   │   ├── ratelimit.go       # Per-IP rate limiting middleware (go-chi/httprate)
│   │   ├── loginguard.go      # Login failure counters and the Argon2id verification semaphore
│   │   └── testing.go         # NewForTest: Middleware without the fx lifecycle
│   ├── reqtls/
│   │   └── reqtls.go          # IsTLS: the one TLS predicate, r.TLS or X-Forwarded-Proto
│   ├── server/
│   │   ├── server.go          # Server struct, fx lifecycle, signal handling
│   │   ├── http.go            # HTTP server setup with timeouts
│   │   └── routes.go          # All route definitions
│   ├── session/
│   │   ├── session.go         # Cookie-based session management
│   │   └── testing.go         # NewForTest: Session without the fx lifecycle
│   └── versionscript/
│       └── doc.go             # Tests for script/version and the build files that use it
├── static/
│   ├── static.go              # //go:embed directive
│   ├── css/input.css          # Tailwind input, source for tailwind.css (make css)
│   ├── css/tailwind.css       # Generated stylesheet the pages load
│   ├── css/style.css          # Hand-written, loaded after tailwind.css: btn-small, the pointer cursor for input.css's buttons, the webhook list cards' focus outline
│   ├── js/app.js              # Copy-to-clipboard, and the Alpine.js components
│   └── js/alpine.min.js       # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
├── templates/                 # Go HTML templates (base, login, sources, etc.)
├── script/                    # Scripts to Rule Them All entrypoints
├── Dockerfile                 # Stages: lint, stylesheet, JavaScript lint, Markdown, test+build, Alpine runtime
├── Dockerfile.lint            # Lint-only image built by script/lint
├── Dockerfile.browser         # Browser test image built by script/test-browser
├── Makefile                   # 13 of 19 targets shim script/; 6 are inline
├── go.mod / go.sum
├── package.json / yarn.lock   # ESLint and prettier, pinned, for the JavaScript lint and Markdown stages
├── eslint.config.mjs          # ESLint configuration for static/js/
├── .prettierrc                # prettier settings for the Markdown
└── .golangci.yml              # golangci-lint configuration

Dependency Injection

Components are wired via Uber fx in this order:

1. globals.New — Build-time variables (appname, version, arch)
2. logger.New — Structured logging (slog with TTY detection)
3. config.New — Configuration loading (environment variables)
4. database.New — Main SQLite connection, config migrations, admin user seed
5. database.NewWebhookDBManager — Per-webhook event database lifecycle manager
6. database.NewRetentionReaper — Per-webhook event retention sweep
7. healthcheck.New — Health check service
8. session.New — Cookie-based session manager (key from database)
9. handlers.New — HTTP handlers
10. metrics.NewRegistry — The registry /metrics serves
11. metrics.New — The delivery collectors, registered on that registry
12. middleware.New — HTTP middleware
13. delivery.New — Event-driven delivery engine
14. delivery.NewArchiveSweeper — Periodic pruning of idle archives
15. delivery.Engine → delivery.Notifier — interface bridge
16. delivery.Engine → delivery.Archives — interface bridge so deleting or renaming a webhook or target reaches its archive files
17. server.New — HTTP server and router

The server starts via fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *delivery.ArchiveSweeper) {}), which triggers the fx lifecycle hooks in dependency order. The delivery.Notifier interface allows the webhook handler to send self-contained delivery.Task slices to the engine without a direct package dependency. Each task carries all target config and event data inline (for bodies under 16 KiB, delivery.MaxInlineBodySize), so the engine can deliver without reading from any database — it only writes to record results.

Middleware Stack

Applied to all routes in this order:

1. RequestID — Generate unique request IDs (chi built-in)
2. SecurityHeaders — Production security headers on every response (HSTS, X-Content-Type-Options, X-Frame-Options, CSP, Referrer-Policy, Permissions-Policy)
3. Logging — Structured request logging (method, URL, status, latency, remote IP, client IP, user agent, request ID)
4. Metrics — Prometheus HTTP metrics (if METRICS_USERNAME and METRICS_PASSWORD are both set)
5. CORS — Cross-origin resource sharing headers
6. Timeout — 60-second request timeout
7. Recoverer — Panic recovery: one ERROR record through internal/logger and a 500
8. Sentry — Error reporting to Sentry (if SENTRY_DSN is set; configured with Repanic: true so panics still reach Recoverer)

Recoverer sits seventh rather than first, and both neighbours are the reason. It runs inside everything that observes the response, so the 500 it writes for a panicking handler is the status the access log records and the metrics count; registered first, as chi's own middleware.Recoverer was, the same request was logged as a 200 that the client never received. It runs outside the Sentry handler, so Repanic: true has something to re-raise into: an operator with SENTRY_DSN set keeps the report, and one without it now gets the local record instead of nothing. What that placement gives up is recovery of a panic in the six entries above it, none of which does more than set a header or start a timer.

Each admin page route group (/pages, /user/*, /settings, /hooks, /hook/*) starts with its own Recoverer and, if SENTRY_DSN is set, its own Sentry error reporting. That Recoverer answers a panic with the 500 error page in the normal layout; the global one keeps the plain-text 500 for every other route.

Additionally, form endpoints (/pages, /user/*, /settings, /hooks, /hook/*) apply a MaxBodySize middleware that limits POST/PUT/PATCH request bodies to 1 MB. It is registered ahead of the CSRF middleware in every one of those route groups, because gorilla/csrf parses the form; if the cap were installed after it, form parsing would run under net/http's 10 MB default and the 1 MB limit would never apply. A request that declares a Content-Length over the limit is answered with 413 Request Entity Too Large without its body being read and without reaching CSRF, the route group's remaining middleware, or the handler. It is not rejected before any other middleware, though: the global entries listed above all run first, so such a request is still logged and given the security headers — and counted in the metrics, on a deployment where the /metrics credentials are set and the Metrics middleware is therefore registered at all. The rejection itself is logged at WARN with the method, path and declared length. A chunked request, or one that lies about its length, is hard-capped by http.MaxBytesReader and fails downstream at form-parse time.

Those same five route groups then apply CSRF and NoCache (Cache-Control: no-store, Pragma: no-cache), and every group except /pages applies RequireAuth. The rate limiters are per-route rather than global: PasswordChangeRateLimit on /user/{username}/password and ReceiverRateLimit on /h/{uuid}. There is deliberately none on /pages/login — that endpoint counts failures inside the handler, after the credential check, see The login endpoint.

Authentication

• Webhook receiver: the entrypoint UUID in the URL, and nothing else. No shared secret, no HMAC signature, no token header, and none will be added — see The entrypoint URL is the authentication secret.
• Web UI: Cookie-based sessions using gorilla/sessions with encrypted cookies. Sessions are configured with HttpOnly, SameSite Lax, and Secure whenever the request is on TLS — the flag follows the request's transport, not the environment. Absolute session lifetime is 7 days, with a sliding idle timeout on top of it (see Sessions).
• API (planned): API key authentication via Authorization: Bearer header. API keys are stored per-user with usage tracking (last_used_at).
• Metrics: Basic authentication protecting the /metrics endpoint.
• Recovery: webhooker resetpw <username> on a stopped deployment is the only way back into an account whose password was lost (see Recovering a lost admin password).

Security

• Passwords hashed with Argon2id (64 MB memory cost)
• Session cookies are HttpOnly, SameSite Lax, and Secure on any request that arrived over TLS (directly or through a reverse proxy reporting it), decided per-request by internal/reqtls.IsTLS rather than by the configured environment
• Session regeneration on login to prevent session fixation attacks
• Session key is a 32-byte value auto-generated on first startup and stored in the database
• Production security headers on all responses: HSTS, X-Content-Type-Options (nosniff), X-Frame-Options (DENY), Content-Security-Policy, Referrer-Policy, and Permissions-Policy
• Request body size limits (1 MB) on all form POST endpoints, enforced by middleware that runs before CSRF parses the form
• CSRF protection via gorilla/csrf on all state-changing forms (cookie-based double-submit tokens with HMAC authentication). Applied to /pages, /hooks, /hook, /settings, and /user routes. Excluded from /h (inbound webhook POSTs) and /api (stateless API). The middleware detects TLS per-request through internal/reqtls.IsTLS — the same predicate the session cookie uses — to set appropriate cookie security flags and Origin/Referer validation mode
• The entrypoint URL is the receiver's only credential. Nothing about an inbound request is verified; possession of the UUID authorises submission, and no shared secret or signature check will be added alongside it (see The entrypoint URL is the authentication secret)
• SSRF prevention for HTTP delivery targets: private/reserved IP ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked both at target creation time (URL validation) and at delivery time (custom HTTP transport with SSRF-safe dialer that validates resolved IPs before connecting, preventing DNS rebinding attacks). Both paths route through a single decision function, so they cannot disagree about a destination. An operator can permit specific blocks with ALLOWED_EGRESS_CIDRS; the guard cannot be switched off, and link-local, the unspecified addresses 0.0.0.0 and ::, and a pinned set of known cloud metadata endpoints — several of which are ULAs outside link-local — stay blocked whatever is listed, though listing 0.0.0.0/0 or ::/0 does open every other private range
• Login limiting is inverted, deliberately. The login POST has no pre-emptive rate limiter in front of it. Credentials are verified first and only a failed attempt spends budget, so a correct password is never throttled and no flood of wrong ones can deny the operator the only administrative path. Failures are counted per (bucket, submitted username), five per minute, after which further failures are answered 429 with a Retry-After. What bounds brute force is not that counter but the cap of two concurrent Argon2id verifications: a throttled client's guess is still evaluated, so roughly 27 guesses a second get through and the admin password has to carry that load (see The login endpoint). GET requests to the sign-in page are not limited
• Password-change rate limiting via go-chi/httprate: sliding-window rate limiter, 5 POST attempts per minute per bucket. It runs behind session auth, so only a client already holding a valid session reaches it, and an operator throttled out of changing a password can still log in. The bucket is per client IP only when TRUSTED_PROXIES covers the reverse proxy; otherwise every client shares one bucket, which costs precision rather than availability (see Rate Limiting)
• Prometheus metrics behind basic auth
• Static assets embedded in binary (no filesystem access needed at runtime)
• The app runs as the non-root webhooker user (UID 1000) in the container. The image sets no USER, so these run as root: the ENTRYPOINT script, which sets the data directory's owner and mode before the app starts; the image's health check; and docker exec, unless given --user
• GORM soft deletes on every entity that carries BaseModel, which is all of them but Setting, EventTotals, TargetTotals and EntrypointTotals (data preserved for audit)

Shutdown

On SIGINT or SIGTERM, fx runs the registered stop hooks in reverse dependency order under a 5 second budget (fx.StopTimeout in cmd/webhooker/main.go). That budget covers the whole sequence, not each hook. The order, read off the fx stop-hook log:

1. ArchiveSweeper
2. RetentionReaper
3. server — the HTTP drain, bounded by server.ShutdownTimeout (3 seconds) and by what the hooks before it left, then a Sentry flush if SENTRY_DSN is set
4. delivery.Engine — waits for its workers, then closes the archive databases
5. healthcheck
6. WebhookDBManager
7. the database close

The two components that can realistically hold the budget run first: a retention sweep or an archive prune caught mid-tick each waits on its WaitGroup bounded by the stop context, so a wedge there consumes the 5 seconds before the HTTP server hook is ever entered. The hooks after the server are microsecond-scale in normal operation.

The HTTP drain budget is deliberately shorter than the sequence budget. Were the two equal, a drain that used its whole budget would exhaust the sequence budget at the instant it finished, and every later hook — the delivery engine, the healthcheck, the webhook DB manager and the database close — would be skipped in exactly the case where the drain mattered. 3 seconds leaves 2 seconds (server.TailHookReserve) for the tail. The reserve is that remainder, not a figure sized to the tail, which takes about a millisecond.

That reserve belongs to the tail hooks, not to the server hook, and the server hook could take it in two ways. The hooks before it may already have spent part of the budget, so a full 3-second drain would come out of the reserve; the drain is therefore also bounded by whatever is left on the stop context minus the reserve. And the Sentry flush runs after the drain inside the same hook, and sentry.Flush takes a bare duration and honours no context, so an unreachable Sentry endpoint would add its own timeout on top of a full-length drain and consume the whole sequence budget by itself. It is clamped the same way, and skipped when that leaves too little to be worth attempting — so a full-length drain means Sentry events are dropped rather than the database close being skipped.

This does not make the database close unconditional. A slow ArchiveSweeper or RetentionReaper is enough to cut the shutdown short, not only one that consumes the whole budget: what they spend comes out of the drain first, so after 2 seconds of theirs a request still in flight gets 1 second to finish, and after 3 it gets none. Past 3 seconds they spend the reserve itself, and one that takes the whole budget skips every hook after it, the database close included.

The value is chosen to sit inside the container stop grace period. Docker's default docker stop grace is 10 seconds and the Dockerfile sets no STOPSIGNAL or grace override, so the process must be gone before that. fx's own default is 15 seconds, which is past the grace: the container would be SIGKILLed (exit 137) before the bound could fire, and nothing that depends on it — including the shutdown timed out, goroutines still running error log that tells an operator a component is wedged — would ever be reached.

Two operational consequences follow from bounding the sequence:

• A wedged component aborts the rest of the shutdown. fx checks the stop context before each remaining hook and returns outright once it has expired, skipping the hooks it has not reached. If the first-stopped component consumes the whole budget, the later hooks never run — the database close among them. SQLite is crash-safe, so this is not corruption, but it is not a clean close either.
• Lowering the grace below 5 seconds reintroduces the silent truncation. docker stop --time, Compose's stop_grace_period, or Kubernetes' terminationGracePeriodSeconds set under 5 seconds put SIGKILL back in front of the bound, and the process dies with no shutdown diagnostics at all. Keep the deployment's grace above the stop timeout.

Linting

golangci-lint never runs on the host. script/lint builds Dockerfile.lint, which copies the repo into the digest-pinned golangci-lint image and lints as a build step, so a successful build is a clean lint. A host binary would share one cache and one lock with every other checkout on the machine, which has produced both invented findings attributed to other worktrees and unearned passes.

Three properties are load-bearing:

• script/lint passes --no-cache-filter=lint. Without it an unchanged tree replays the lint layer from cache and the build exits 0 in under a second having linted nothing. The deps stage stays cacheable, so module downloads are not repeated. Invalidation is scoped to the one stage; never prune the shared build cache.
• script/lint does not trust that flag. Docker silently ignores --no-cache-filter for a stage name that does not match, so a stage rename or a one-character typo would restore the cached false green with no warning and a fast exit 0. The script therefore tees the build output and treats a run as a pass only if golangci-lint's own summary line (N issues. / N issues:) appears in it: no summary, no lint, whatever the exit code says.
• Both lint steps use RUN --network=none. golangci-lint config verify is documented as fetching its JSON schema over HTTPS, which would be an unpinned remote dependency; the pinned image resolves the schema without network access, and --network=none enforces that instead of trusting it. Verify is worth keeping because golangci-lint run silently ignores config keys it does not recognize, so a typo would disable a setting with no warning.

ESLint never runs on the host either. It lints static/js/ (not the extracted Alpine.js) in the Dockerfile's js-lint stage, which script/lint builds after Dockerfile.lint and the image build runs before the builder stage. Its version is pinned in package.json and every package's hash in yarn.lock. The js-deps stage before it installs ESLint and stays cached until either file changes, so only the lint step re-runs and ESLint is not downloaded again. eslint.config.mjs turns on the rules of the JavaScript styleguide REPO_POLICIES.md links to that a linter can check: no-var and prefer-const. ESLint prints nothing on a pass, so script/lint has no summary line to look for; it names the stage once for both --target and --no-cache-filter, and --target fails on a name that matches no stage.

prettier formats the Markdown, and it never runs on the host either. It is pinned in package.json and yarn.lock beside ESLint, installed by the same js-deps stage, and reads its settings from .prettierrc. make fmt builds the markdown-output stage and writes the formatted files back into the tree; make fmt-check and the image build run the markdown-check stage, which fails on any Markdown file prettier would change.

Docker

The Dockerfile uses a multi-stage build. Each stage is pinned by digest, and the lint and builder stages are separate images so the linter's version is fixed independently of the compiler's:

1. Lint stage (golangci/golangci-lint:v2.12.2, Debian-based) — downloads dependencies, copies the source, and runs the gofmt check, then script/assets to extract Alpine.js from 3p/, then golangci-lint config verify and golangci-lint run, both with --network=none.
2. Stylesheet stages (debian:bookworm-slim, with the Tailwind standalone CLI pinned by version and sha256, one binary per architecture) — generate static/css/tailwind.css from static/css/input.css and the files its @source lines name. css-check fails when the committed file differs from the generated one, and make css writes the generated file out from css-output (see Stylesheet).
3. JavaScript lint stages (node:24.21.0-alpine, with yarn) — js-deps installs ESLint and prettier from yarn.lock and js-lint runs ESLint over static/js/ (see Linting).
4. Markdown stages (on js-deps) — markdown-check runs prettier over the Markdown and fails on any file it would change, and make fmt writes the formatted files out from markdown-output.
5. Builder stage (golang:1.26.1-bookworm) — depends on the lint, css-check, js-lint and markdown-check stages passing (it copies a file from each), runs make test and make build (both extract Alpine.js from 3p/ first), and finally rebuilds the binary with CGO_ENABLED=1 and static linking so it runs on musl. Both builds go through make build, the relink adding its -extldflags via GO_LDFLAGS, so neither can drop the -X that stamps the version. The version is the VERSION build arg if one is given, otherwise derived from the .git in the context, and the stage fails if a context with .git would stamp unknown (see Version stamping).
6. Runtime stage (alpine:3.21) — copies the static binary and deploy/docker-entrypoint.sh, creates the /var/lib/webhooker directory for all SQLite databases, exposes port 8080, and includes a health check against /.well-known/healthcheck. It sets no USER: the ENTRYPOINT script starts as root, sets the data directory's owner and mode, and runs the app as the non-root webhooker user (UID 1000) through su-exec.

The lint stage invokes gofmt and golangci-lint directly rather than make fmt-check and make lint: it is already the pinned linter image, and both targets build docker stages, which would need a docker daemon inside this build.

The lint and builder stages use Debian rather than Alpine because gorm.io/driver/sqlite pulls in mattn/go-sqlite3, which needs CGO and does not compile against musl. Only the final binary is statically linked, which is what lets it run on the Alpine runtime image.

script/cibuild — docker build . — is the CI gate: the checks run inside the image, so a build that succeeds is a repo that is formatted, linted, tested and compiled, with a current stylesheet. script/lint also uses Docker (Dockerfile.lint and the js-lint stage, see Linting above), so make lint and make check run the same pinned linter versions the gate does; of the steps make check runs, only script/test and the gofmt check in script/fmt-check run on the host.

CI gate honesty

A layer cache lets docker build . exit 0 in seconds with the lint and test stages replayed rather than executed, which would make a green check meaningless. The check workflow therefore writes .ci-fingerprint into the build context before building. Its value is the hash of the commit being checked, so every commit, docs-only ones and a squash merge whose tree matches an already-built branch included, gets a new fingerprint, invalidates the COPY . . layer of every check stage, and really runs the gofmt check, golangci-lint, the stylesheet check, ESLint, the Markdown check, make test, and make build. A run that reports success ran them.

The module download layer sits above COPY . . and stays cached.

A separate workflow step, run before the fingerprint is written, covers a second way the gate lied: Gitea cancels an in-flight run when a newer commit lands on the same branch and records that cancellation as a failure status, so a commit nothing ever tested reads as a test result. Cancellation is unconditional server-side for push events, so the superseding run calls script/ci-mark-superseded, which rewrites that exact status to failure / Superseded by a newer commit; never tested.

The state stays failure on purpose: Gitea's combined status folds skipped into success, so marking a never-tested commit skipped made the status API report green for it, indistinguishable from a commit that passed. Reading a commit's status on this repo therefore goes:

• success / Successful in ... — the checks ran and passed.
• failure / Failing after ... — the checks ran and failed.
• failure / Superseded by a newer commit; never tested — the run was cancelled, by a newer push or by hand, and nothing was verified about this commit. Test the commit itself before concluding anything about it.

Genuine failures and successes are never touched, and no status is left pending, which would block the commit indefinitely. The step derives its context string from the workflow name, the job id and the event. That is deliberately not byte-identical to Gitea's own rule, which uses the job's display name: where the runner exports the id, so giving the job a name: — or renaming the workflow — makes the derived context stop matching. The step fails loudly when no status on the commit carries that context, so no rename can silently disable the rewrite.

TODO

See TODO.md.

License

MIT

Author

@sneak