Bind the app port deliberately and document the proxy deployment (closes #268)
All checks were successful
check / check (push) Successful in 3m1s
All checks were successful
check / check (push) Successful in 3m1s
The plaintext listener bound `:PORT`, so it answered on every interface with no way to say otherwise. That published the admin UI and the unauthenticated receiver in cleartext beside whatever TLS proxy was in front of them, reachable from any host that could route to the machine. BIND_ADDRESS now selects the address, defaulting to 127.0.0.1. Loopback is the only default that does not silently expose that listener; reaching webhooker from another host becomes a deliberate act. A container must set BIND_ADDRESS=0.0.0.0, since a loopback bind inside a network namespace is unreachable even with -p, and the documented `docker run` does. Only IP address literals are accepted: hostnames, host:port and CIDR blocks abort startup naming the variable and the value, and a literal that is not an address of this host fails at listen and exits non-zero. The http.Server is now built in New rather than in the serving goroutine. Two goroutines reached that field with nothing ordering them — the serving goroutine assigning it, the fx stop hook calling Shutdown on it — which is a data race the race detector reports as soon as anything starts and stops the server, and a nil dereference if a stop arrived first. README gains a "Deployment behind a reverse proxy" section: a working nginx server block, and the five things that are silent when wrong — bind or firewall the app port, WEBHOOKER_ENVIRONMENT=prod, TRUSTED_PROXIES, Host as $http_host rather than $host, and keeping the proxy's access log because webhooker's own records only the proxy.
This commit is contained in:
223
README.md
223
README.md
@@ -104,6 +104,7 @@ TTY detection, and security headers are always applied.
|
||||
| ----------------------- | ----------------------------------- | -------- |
|
||||
| `WEBHOOKER_ENVIRONMENT` | `dev` or `prod` | `dev` |
|
||||
| `PORT` | HTTP listen port | `8080` |
|
||||
| `BIND_ADDRESS` | IP address the HTTP listener binds. Loopback by default, so the cleartext listener is not published on every interface. Containers must set `0.0.0.0`. See [Bind address](#bind-address) | `127.0.0.1` |
|
||||
| `DATA_DIR` | Directory for all SQLite databases | `/var/lib/webhooker` |
|
||||
| `DEBUG` | Enable debug logging | `false` |
|
||||
| `MAINTENANCE_MODE` | Report `maintenanceMode: true` in the healthcheck JSON. It does not change how any request is served — no maintenance page exists | `false` |
|
||||
@@ -225,6 +226,58 @@ A set but unparseable value aborts startup. When the list is non-empty
|
||||
webhooker logs it at startup, blocks and all, so the hole is visible in
|
||||
the log of any deployment that has one.
|
||||
|
||||
#### Bind address
|
||||
|
||||
`BIND_ADDRESS` is the IP address the HTTP listener binds. It defaults
|
||||
to `127.0.0.1`, so out of the box webhooker is reachable only from the
|
||||
host it runs on.
|
||||
|
||||
That listener speaks **cleartext**, and it serves both the admin UI and
|
||||
the unauthenticated webhook receiver. webhooker terminates no TLS
|
||||
itself; a production deployment puts a reverse proxy in front of it
|
||||
(see
|
||||
[Deployment behind a reverse proxy](#deployment-behind-a-reverse-proxy)),
|
||||
and the proxy reaches it over loopback. A default that bound every
|
||||
interface would leave that cleartext port answering the internet
|
||||
alongside the proxy — the admin login form and the receiver, in the
|
||||
clear, on a port nobody chose to publish. Reaching webhooker from
|
||||
another host is therefore something you configure, not something you
|
||||
get by default.
|
||||
|
||||
**A container must set `BIND_ADDRESS=0.0.0.0`.** A process bound to
|
||||
loopback inside a container is unreachable from outside its network
|
||||
namespace even with `-p`, because the published port maps to the
|
||||
container's external address and nothing is listening there. The
|
||||
container's namespace is its own boundary, and `-p` is the exposure
|
||||
decision. The `docker run` command in
|
||||
[Running with Docker](#running-with-docker) sets it.
|
||||
|
||||
The value must be an IP address literal:
|
||||
|
||||
- `127.0.0.1` — loopback only (the default). Use this with a reverse
|
||||
proxy on the same host.
|
||||
- `0.0.0.0` — every IPv4 address. Required in a container; on a bare
|
||||
host, only with a firewall in front of the port.
|
||||
- `::` — every address, IPv6 and (on Linux, with the default
|
||||
`net.ipv6.bindv6only=0`) IPv4 as well.
|
||||
- A specific address such as `10.0.0.5` — that interface only.
|
||||
|
||||
Hostnames are **not** accepted. `localhost` aborts startup rather than
|
||||
being resolved: which of `127.0.0.1` and `::1` it means differs by
|
||||
host, a name can resolve to several addresses of which only one could
|
||||
be bound, and the answer can change under a running process. A value
|
||||
carrying a port (`127.0.0.1:8080`) is likewise rejected — the port is
|
||||
`PORT`'s business. Any unparseable value aborts startup; see
|
||||
[Invalid values abort startup](#invalid-values-abort-startup).
|
||||
|
||||
An address that parses but is not assigned to this host — say
|
||||
`10.0.0.5` on a machine that has no such interface — is a valid
|
||||
literal, so it reaches the listener and fails there. The process logs
|
||||
the bind error and exits non-zero rather than staying up with nothing
|
||||
listening. The effective value is in the `bindAddress` field of the
|
||||
startup log line, which is the way to check what a running deployment
|
||||
actually bound.
|
||||
|
||||
#### Metrics credentials
|
||||
|
||||
`METRICS_USERNAME` and `METRICS_PASSWORD` are set together or not at
|
||||
@@ -382,8 +435,11 @@ additionally be a number in the range 1–65535,
|
||||
`RECEIVER_RATE_LIMIT` must be at least 1,
|
||||
`RETENTION_SWEEP_INTERVAL` must be greater than zero (it is a ticker
|
||||
period, so `0s` or a negative value would crash the reaper after
|
||||
startup), and every entry in `TRUSTED_PROXIES` and
|
||||
`ALLOWED_EGRESS_CIDRS` must be a CIDR block or a bare IP address.
|
||||
startup), every entry in `TRUSTED_PROXIES` and
|
||||
`ALLOWED_EGRESS_CIDRS` must be a CIDR block or a bare IP address, and
|
||||
`BIND_ADDRESS` must be an IP address literal — `localhost`,
|
||||
`127.0.0.1:8080` and `10.0.0.0/8` are each rejected rather than
|
||||
resolved, split, or narrowed to something they do not say.
|
||||
`SESSION_IDLE_TIMEOUT` is the exception: a
|
||||
non-positive value there means idle expiry is disabled, not invalid.
|
||||
|
||||
@@ -528,12 +584,32 @@ decision:
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
-p 8080:8080 \
|
||||
-p 127.0.0.1:8080:8080 \
|
||||
-v /path/to/data:/var/lib/webhooker \
|
||||
-e WEBHOOKER_ENVIRONMENT=prod \
|
||||
-e BIND_ADDRESS=0.0.0.0 \
|
||||
webhooker:latest
|
||||
```
|
||||
|
||||
`BIND_ADDRESS=0.0.0.0` is **required** in a container and is not the
|
||||
default. webhooker binds loopback unless told otherwise (see
|
||||
[Bind address](#bind-address)), and a loopback-bound process inside a
|
||||
container is unreachable from outside its network namespace even with
|
||||
`-p`: the port is published and nothing answers on it. The container's
|
||||
health check fails too — it requests `http://localhost:8080`, and
|
||||
`localhost` resolves to `::1` first, which a `127.0.0.1` bind is not
|
||||
listening on — so the container goes `unhealthy` about 95 seconds
|
||||
after start. A container that is `unhealthy` with `connection refused`
|
||||
in its health log, or a published port that resets connections, is
|
||||
this.
|
||||
|
||||
Publishing to `127.0.0.1:8080` rather than `8080` keeps Docker from
|
||||
opening the cleartext port on every interface of the host, which is
|
||||
what a bare `-p 8080:8080` does — including through firewall rules,
|
||||
since Docker's forwarding rules are inserted ahead of most host
|
||||
firewalls. Bind it to the host address your reverse proxy connects
|
||||
from, and nothing wider.
|
||||
|
||||
The container runs as a non-root user (`webhooker`, UID 1000), exposes
|
||||
port 8080, and includes a health check against
|
||||
`/.well-known/healthcheck`. The `/var/lib/webhooker` volume holds all
|
||||
@@ -543,6 +619,137 @@ databases written by `database` targets (`archive-{uuid}.db`). Mount
|
||||
this as a persistent volume to preserve data across container
|
||||
restarts.
|
||||
|
||||
## Deployment behind a reverse proxy
|
||||
|
||||
webhooker terminates no TLS of its own. It serves plaintext HTTP and
|
||||
expects a reverse proxy in front of it, which is the deployment it is
|
||||
built for: the proxy holds the certificate, and webhooker binds
|
||||
loopback where only the proxy can reach it.
|
||||
|
||||
Five things have to be right. Each one is silent when it is wrong —
|
||||
the service comes up, serves pages, and is broken in a way nothing
|
||||
reports.
|
||||
|
||||
1. **Bind or firewall the app port.** `BIND_ADDRESS` defaults to
|
||||
`127.0.0.1` so the cleartext listener is not published beside the
|
||||
proxy. If you must widen it — a container, or a proxy on another
|
||||
host — firewall the port to the proxy's address. A cleartext port
|
||||
answering the internet serves the admin login form and the
|
||||
unauthenticated receiver with no TLS at all, and the proxy in front
|
||||
of it changes nothing about that.
|
||||
2. **Set `WEBHOOKER_ENVIRONMENT=prod`.** It defaults to `dev`, and the
|
||||
session cookie's `Secure` flag depends on it — see the table in
|
||||
[Configuration](#configuration). Left at the default, a session
|
||||
cookie can be sent over plaintext HTTP.
|
||||
3. **Set `TRUSTED_PROXIES` to the proxy's address.** Unset, every rate
|
||||
limiter keys on the connecting peer, which behind a proxy is the
|
||||
proxy on every request: all clients collapse into one global bucket
|
||||
per limit and the receiver's per-IP limits become service-wide
|
||||
ceilings. See [Trusted proxies](#trusted-proxies). List the proxy
|
||||
and nothing else.
|
||||
4. **Send `Host` as `$http_host`, not `$host`.** `$host` strips the
|
||||
port. webhooker's Origin/Referer check compares against the host it
|
||||
was given, so on any port other than 443 `$host` makes every form
|
||||
POST — including login — fail with `403 origin invalid`, with
|
||||
nothing in the error naming the cause.
|
||||
5. **Keep the proxy's access log.** webhooker's own access log records
|
||||
the peer address, which behind a proxy is always the proxy. The
|
||||
proxy's log is the only record of which client sent what. nginx's
|
||||
default `combined` format already logs `$remote_addr`; do not
|
||||
replace it with one that drops the client address, and retain those
|
||||
logs as long as you would want to answer a question about traffic.
|
||||
|
||||
### nginx
|
||||
|
||||
Complete server block. Replace the `server_name` and the two
|
||||
certificate paths.
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
listen [::]:443 ssl;
|
||||
http2 on; # nginx 1.25.1+; older: listen 443 ssl http2;
|
||||
|
||||
server_name webhooker.example.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/webhooker.example.com.crt;
|
||||
ssl_certificate_key /etc/ssl/private/webhooker.example.com.key;
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
|
||||
# webhooker caps form POST bodies at 1 MB. nginx's default happens
|
||||
# to match, so leaving this out breaks nothing today — but if you
|
||||
# ever raise webhooker's cap, this is the limit you will still be
|
||||
# hitting, and the rejection is nginx's HTML page rather than
|
||||
# webhooker's message.
|
||||
client_max_body_size 1m;
|
||||
|
||||
# $remote_addr is the client. webhooker's own log records this
|
||||
# proxy and nothing else, so this file is the only place the
|
||||
# client's address is written down.
|
||||
access_log /var/log/nginx/webhooker.access.log combined;
|
||||
|
||||
location / {
|
||||
# A literal address, not localhost: with BIND_ADDRESS at its
|
||||
# 127.0.0.1 default, a localhost that resolves to ::1 first
|
||||
# gets connection refused.
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
|
||||
# $http_host, NOT $host. $host drops the port and every form
|
||||
# POST fails with 403 origin invalid on any port but 443.
|
||||
proxy_set_header Host $http_host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Above webhooker's own 60s request timeout, so its 503
|
||||
# reaches the client instead of nginx cutting the connection
|
||||
# first and answering 504.
|
||||
proxy_read_timeout 70s;
|
||||
}
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name webhooker.example.com;
|
||||
return 308 https://$host$request_uri;
|
||||
}
|
||||
```
|
||||
|
||||
`X-Forwarded-For` must be **appended** to, which
|
||||
`$proxy_add_x_forwarded_for` does. webhooker reads no other forwarded
|
||||
client header: `X-Real-IP` and `True-Client-IP` are ignored from every
|
||||
peer, so setting them has no effect. See
|
||||
[Trusted proxies](#trusted-proxies) for how the chain is walked.
|
||||
|
||||
`X-Forwarded-Proto: https` is what tells webhooker the request arrived
|
||||
over TLS, which decides the CSRF cookie's `Secure` flag and the strict
|
||||
Origin/Referer mode. Without it, requests are treated as plaintext.
|
||||
|
||||
With that block, webhooker's environment is:
|
||||
|
||||
```sh
|
||||
WEBHOOKER_ENVIRONMENT=prod
|
||||
BIND_ADDRESS=127.0.0.1 # the default; stated here to be explicit
|
||||
TRUSTED_PROXIES=127.0.0.1
|
||||
```
|
||||
|
||||
If nginx runs on another host, `BIND_ADDRESS` becomes the address it
|
||||
connects to, `TRUSTED_PROXIES` becomes nginx's address, and the port
|
||||
must be firewalled to that address — the traffic between them is
|
||||
cleartext.
|
||||
|
||||
### HSTS is always sent, and is not configurable
|
||||
|
||||
Every response carries
|
||||
`Strict-Transport-Security: max-age=63072000; includeSubDomains; preload`.
|
||||
Two years, every subdomain, and a `preload` token. There is no setting
|
||||
that changes or suppresses it.
|
||||
|
||||
This is worth knowing before the first request reaches a browser: a
|
||||
client that sees it once will refuse plaintext HTTP to that hostname —
|
||||
and to every subdomain of it — for two years, whatever else is served
|
||||
there. Terminate TLS on a hostname you are prepared to keep on HTTPS.
|
||||
|
||||
## Backup, Restore, and Upgrades
|
||||
|
||||
### What to back up
|
||||
@@ -678,6 +885,16 @@ Upgrade procedure:
|
||||
4. Confirm `database migrations completed` in the logs before putting
|
||||
traffic back on it.
|
||||
|
||||
**Upgrading past the introduction of `BIND_ADDRESS`:** earlier versions
|
||||
always bound every interface. The listener now binds `127.0.0.1` unless
|
||||
`BIND_ADDRESS` says otherwise, so a deployment that relied on the old
|
||||
behaviour becomes unreachable from other hosts until it sets one. In a
|
||||
container that means `BIND_ADDRESS=0.0.0.0`; without it the container
|
||||
also goes `unhealthy`, since its health check reaches the app over
|
||||
`localhost`, which resolves to `::1` before `127.0.0.1`. On a bare
|
||||
host, set the address the proxy connects to. See
|
||||
[Bind address](#bind-address).
|
||||
|
||||
**Downgrade is unsupported.** Once a newer binary has migrated the files
|
||||
there is no way to move them back. `AutoMigrate` is additive — it adds
|
||||
tables, columns and indexes and never drops or rewrites them — so an
|
||||
|
||||
Reference in New Issue
Block a user