Bind the app port deliberately and document the proxy deployment (closes #268)
All checks were successful
check / check (push) Successful in 3m46s
All checks were successful
check / check (push) Successful in 3m46s
The plaintext listener bound `:PORT`, so it answered on every interface with no way to say otherwise. That published the admin UI and the unauthenticated receiver in cleartext beside whatever TLS proxy was in front of them, reachable from any host that could route to the machine. BIND_ADDRESS now selects the address. The binary defaults to 127.0.0.1, which is the safe answer for a bare host: reaching webhooker from elsewhere becomes a deliberate act. The image sets 0.0.0.0, which is the correct answer inside a container, where the network namespace is already the boundary and exposure is decided by the publish flag instead — so `-p 127.0.0.1:8080:8080` is what the README shows. Existing container deployments are unaffected. Only IP address literals are accepted: hostnames, host:port and CIDR blocks abort startup naming the variable and the value, and a literal that is not an address of this host fails at listen and exits non-zero. The http.Server is now built in New rather than in the serving goroutine, and sentryEnabled is atomic. Both fields were written by the serving goroutine and read by the fx stop hook with nothing ordering them, and the OnStart hook returns before that goroutine has necessarily run: cleanShutdown could dereference a nil httpServer on an early SIGTERM, and both reads raced. No test started and stopped the server, so nothing observed it. Closes #226. README gains a "Deployment behind a reverse proxy" section: a working nginx server block, and the five things that are silent when wrong — bind or firewall the app port, WEBHOOKER_ENVIRONMENT=prod, TRUSTED_PROXIES, Host as $http_host rather than $host, and keeping the proxy's access log because webhooker's own records only the proxy.
This commit is contained in:
238
README.md
238
README.md
@@ -104,6 +104,7 @@ TTY detection, and security headers are always applied.
|
||||
| ----------------------- | ----------------------------------- | -------- |
|
||||
| `WEBHOOKER_ENVIRONMENT` | `dev` or `prod` | `dev` |
|
||||
| `PORT` | HTTP listen port | `8080` |
|
||||
| `BIND_ADDRESS` | IP address the HTTP listener binds. Loopback by default, so the cleartext listener is not published on every interface. The Docker image ships `0.0.0.0` instead. See [Bind address](#bind-address) | `127.0.0.1` (image: `0.0.0.0`) |
|
||||
| `DATA_DIR` | Directory for all SQLite databases | `/var/lib/webhooker` |
|
||||
| `DEBUG` | Enable debug logging | `false` |
|
||||
| `MAINTENANCE_MODE` | Report `maintenanceMode: true` in the healthcheck JSON. It does not change how any request is served — no maintenance page exists | `false` |
|
||||
@@ -225,6 +226,62 @@ A set but unparseable value aborts startup. When the list is non-empty
|
||||
webhooker logs it at startup, blocks and all, so the hole is visible in
|
||||
the log of any deployment that has one.
|
||||
|
||||
#### Bind address
|
||||
|
||||
`BIND_ADDRESS` is the IP address the HTTP listener binds. The binary
|
||||
defaults to `127.0.0.1`, so a bare webhooker is reachable only from the
|
||||
host it runs on. The Docker image ships `ENV BIND_ADDRESS=0.0.0.0`
|
||||
instead — see below for why the two differ.
|
||||
|
||||
That listener speaks **cleartext**, and it serves both the admin UI and
|
||||
the unauthenticated webhook receiver. webhooker terminates no TLS
|
||||
itself; a production deployment puts a reverse proxy in front of it
|
||||
(see
|
||||
[Deployment behind a reverse proxy](#deployment-behind-a-reverse-proxy)),
|
||||
and the proxy reaches it over loopback. A default that bound every
|
||||
interface would leave that cleartext port answering the internet
|
||||
alongside the proxy — the admin login form and the receiver, in the
|
||||
clear, on a port nobody chose to publish. Reaching webhooker from
|
||||
another host is therefore something you configure, not something you
|
||||
get by default.
|
||||
|
||||
**In a container the answer is `0.0.0.0`, which is why the image ships
|
||||
that.** A container's network namespace is already the boundary the
|
||||
loopback default is reaching for: nothing outside the container gets to
|
||||
`0.0.0.0:8080` because of the namespace, whatever the process bound.
|
||||
Exposure is decided at the publish flag instead — `-p
|
||||
127.0.0.1:8080:8080` rather than `-p 8080:8080` — which is the
|
||||
operator's to choose and is what
|
||||
[Running with Docker](#running-with-docker) shows. A loopback bind
|
||||
inside a container buys nothing and makes the process unreachable
|
||||
through its own published port.
|
||||
|
||||
The value must be an IP address literal:
|
||||
|
||||
- `127.0.0.1` — loopback only (the binary's default). Use this with a
|
||||
reverse proxy on the same host.
|
||||
- `0.0.0.0` — every IPv4 address. The image's default; on a bare host,
|
||||
only behind a firewall on the port.
|
||||
- `::` — every address, IPv6 and (on Linux, with the default
|
||||
`net.ipv6.bindv6only=0`) IPv4 as well.
|
||||
- A specific address such as `10.0.0.5` — that interface only.
|
||||
|
||||
Hostnames are **not** accepted. `localhost` aborts startup rather than
|
||||
being resolved: which of `127.0.0.1` and `::1` it means differs by
|
||||
host, a name can resolve to several addresses of which only one could
|
||||
be bound, and the answer can change under a running process. A value
|
||||
carrying a port (`127.0.0.1:8080`) is likewise rejected — the port is
|
||||
`PORT`'s business. Any unparseable value aborts startup; see
|
||||
[Invalid values abort startup](#invalid-values-abort-startup).
|
||||
|
||||
An address that parses but is not assigned to this host — say
|
||||
`10.0.0.5` on a machine that has no such interface — is a valid
|
||||
literal, so it reaches the listener and fails there. The process logs
|
||||
the bind error and exits non-zero rather than staying up with nothing
|
||||
listening. The effective value is in the `bindAddress` field of the
|
||||
startup log line, which is the way to check what a running deployment
|
||||
actually bound.
|
||||
|
||||
#### Metrics credentials
|
||||
|
||||
`METRICS_USERNAME` and `METRICS_PASSWORD` are set together or not at
|
||||
@@ -382,8 +439,11 @@ additionally be a number in the range 1–65535,
|
||||
`RECEIVER_RATE_LIMIT` must be at least 1,
|
||||
`RETENTION_SWEEP_INTERVAL` must be greater than zero (it is a ticker
|
||||
period, so `0s` or a negative value would crash the reaper after
|
||||
startup), and every entry in `TRUSTED_PROXIES` and
|
||||
`ALLOWED_EGRESS_CIDRS` must be a CIDR block or a bare IP address.
|
||||
startup), every entry in `TRUSTED_PROXIES` and
|
||||
`ALLOWED_EGRESS_CIDRS` must be a CIDR block or a bare IP address, and
|
||||
`BIND_ADDRESS` must be an IP address literal — `localhost`,
|
||||
`127.0.0.1:8080` and `10.0.0.0/8` are each rejected rather than
|
||||
resolved, split, or narrowed to something they do not say.
|
||||
`SESSION_IDLE_TIMEOUT` is the exception: a
|
||||
non-positive value there means idle expiry is disabled, not invalid.
|
||||
|
||||
@@ -528,12 +588,41 @@ decision:
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
-p 8080:8080 \
|
||||
-p 127.0.0.1:8080:8080 \
|
||||
-v /path/to/data:/var/lib/webhooker \
|
||||
-e WEBHOOKER_ENVIRONMENT=prod \
|
||||
-e BIND_ADDRESS=0.0.0.0 \
|
||||
webhooker:latest
|
||||
```
|
||||
|
||||
**The image and the bare binary default `BIND_ADDRESS` differently, on
|
||||
purpose.** The binary defaults to `127.0.0.1`; the image ships
|
||||
`ENV BIND_ADDRESS=0.0.0.0`, so the `-e BIND_ADDRESS=0.0.0.0` above is
|
||||
belt-and-braces and the command works without it.
|
||||
|
||||
The two cases are not the same question. On a bare host, `0.0.0.0`
|
||||
puts the cleartext admin UI and the unauthenticated receiver on every
|
||||
interface of the machine, which is what the loopback default exists to
|
||||
prevent. In a container, the network namespace is already that
|
||||
boundary: nothing outside reaches `0.0.0.0:8080` because of the
|
||||
namespace, not because of the bind. What decides exposure there is the
|
||||
**publish flag**, and that is the line to get right.
|
||||
|
||||
So publish to `127.0.0.1:8080` rather than `8080`. A bare
|
||||
`-p 8080:8080` opens the port on every interface of the host — through
|
||||
firewall rules too, since Docker's forwarding rules are inserted ahead
|
||||
of most host firewalls. Publish to the host address your reverse proxy
|
||||
connects from, and nothing wider.
|
||||
|
||||
If you override `BIND_ADDRESS` to a loopback address in a container,
|
||||
the container is unreachable from outside its namespace even with
|
||||
`-p`: the published port answers nothing, and the health check fails
|
||||
as well — it requests `http://localhost:8080`, `localhost` resolves to
|
||||
`::1` first, and a `127.0.0.1` bind is not listening there, so the
|
||||
container goes `unhealthy` about 95 seconds after start. A container
|
||||
`unhealthy` with `connection refused` in its health log, or a
|
||||
published port that resets connections, is this.
|
||||
|
||||
The container runs as a non-root user (`webhooker`, UID 1000), exposes
|
||||
port 8080, and includes a health check against
|
||||
`/.well-known/healthcheck`. The `/var/lib/webhooker` volume holds all
|
||||
@@ -543,6 +632,137 @@ databases written by `database` targets (`archive-{uuid}.db`). Mount
|
||||
this as a persistent volume to preserve data across container
|
||||
restarts.
|
||||
|
||||
## Deployment behind a reverse proxy
|
||||
|
||||
webhooker terminates no TLS of its own. It serves plaintext HTTP and
|
||||
expects a reverse proxy in front of it, which is the deployment it is
|
||||
built for: the proxy holds the certificate, and webhooker binds
|
||||
loopback where only the proxy can reach it.
|
||||
|
||||
Five things have to be right. Each one is silent when it is wrong —
|
||||
the service comes up, serves pages, and is broken in a way nothing
|
||||
reports.
|
||||
|
||||
1. **Bind or firewall the app port.** `BIND_ADDRESS` defaults to
|
||||
`127.0.0.1` so the cleartext listener is not published beside the
|
||||
proxy. If you must widen it — a container, or a proxy on another
|
||||
host — firewall the port to the proxy's address. A cleartext port
|
||||
answering the internet serves the admin login form and the
|
||||
unauthenticated receiver with no TLS at all, and the proxy in front
|
||||
of it changes nothing about that.
|
||||
2. **Set `WEBHOOKER_ENVIRONMENT=prod`.** It defaults to `dev`, and the
|
||||
session cookie's `Secure` flag depends on it — see the table in
|
||||
[Configuration](#configuration). Left at the default, a session
|
||||
cookie can be sent over plaintext HTTP.
|
||||
3. **Set `TRUSTED_PROXIES` to the proxy's address.** Unset, every rate
|
||||
limiter keys on the connecting peer, which behind a proxy is the
|
||||
proxy on every request: all clients collapse into one global bucket
|
||||
per limit and the receiver's per-IP limits become service-wide
|
||||
ceilings. See [Trusted proxies](#trusted-proxies). List the proxy
|
||||
and nothing else.
|
||||
4. **Send `Host` as `$http_host`, not `$host`.** `$host` strips the
|
||||
port. webhooker's Origin/Referer check compares against the host it
|
||||
was given, so on any port other than 443 `$host` makes every form
|
||||
POST — including login — fail with `403 origin invalid`, with
|
||||
nothing in the error naming the cause.
|
||||
5. **Keep the proxy's access log.** webhooker's own access log records
|
||||
the peer address, which behind a proxy is always the proxy. The
|
||||
proxy's log is the only record of which client sent what. nginx's
|
||||
default `combined` format already logs `$remote_addr`; do not
|
||||
replace it with one that drops the client address, and retain those
|
||||
logs as long as you would want to answer a question about traffic.
|
||||
|
||||
### nginx
|
||||
|
||||
Complete server block. Replace the `server_name` and the two
|
||||
certificate paths.
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
listen [::]:443 ssl;
|
||||
http2 on; # nginx 1.25.1+; older: listen 443 ssl http2;
|
||||
|
||||
server_name webhooker.example.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/webhooker.example.com.crt;
|
||||
ssl_certificate_key /etc/ssl/private/webhooker.example.com.key;
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
|
||||
# webhooker caps form POST bodies at 1 MB. nginx's default happens
|
||||
# to match, so leaving this out breaks nothing today — but if you
|
||||
# ever raise webhooker's cap, this is the limit you will still be
|
||||
# hitting, and the rejection is nginx's HTML page rather than
|
||||
# webhooker's message.
|
||||
client_max_body_size 1m;
|
||||
|
||||
# $remote_addr is the client. webhooker's own log records this
|
||||
# proxy and nothing else, so this file is the only place the
|
||||
# client's address is written down.
|
||||
access_log /var/log/nginx/webhooker.access.log combined;
|
||||
|
||||
location / {
|
||||
# A literal address, not localhost: with BIND_ADDRESS at its
|
||||
# 127.0.0.1 default, a localhost that resolves to ::1 first
|
||||
# gets connection refused.
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
|
||||
# $http_host, NOT $host. $host drops the port and every form
|
||||
# POST fails with 403 origin invalid on any port but 443.
|
||||
proxy_set_header Host $http_host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Above webhooker's own 60s request timeout, so its 503
|
||||
# reaches the client instead of nginx cutting the connection
|
||||
# first and answering 504.
|
||||
proxy_read_timeout 70s;
|
||||
}
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name webhooker.example.com;
|
||||
return 308 https://$host$request_uri;
|
||||
}
|
||||
```
|
||||
|
||||
`X-Forwarded-For` must be **appended** to, which
|
||||
`$proxy_add_x_forwarded_for` does. webhooker reads no other forwarded
|
||||
client header: `X-Real-IP` and `True-Client-IP` are ignored from every
|
||||
peer, so setting them has no effect. See
|
||||
[Trusted proxies](#trusted-proxies) for how the chain is walked.
|
||||
|
||||
`X-Forwarded-Proto: https` is what tells webhooker the request arrived
|
||||
over TLS, which decides the CSRF cookie's `Secure` flag and the strict
|
||||
Origin/Referer mode. Without it, requests are treated as plaintext.
|
||||
|
||||
With that block, webhooker's environment is:
|
||||
|
||||
```sh
|
||||
WEBHOOKER_ENVIRONMENT=prod
|
||||
BIND_ADDRESS=127.0.0.1 # the default; stated here to be explicit
|
||||
TRUSTED_PROXIES=127.0.0.1
|
||||
```
|
||||
|
||||
If nginx runs on another host, `BIND_ADDRESS` becomes the address it
|
||||
connects to, `TRUSTED_PROXIES` becomes nginx's address, and the port
|
||||
must be firewalled to that address — the traffic between them is
|
||||
cleartext.
|
||||
|
||||
### HSTS is always sent, and is not configurable
|
||||
|
||||
Every response carries
|
||||
`Strict-Transport-Security: max-age=63072000; includeSubDomains; preload`.
|
||||
Two years, every subdomain, and a `preload` token. There is no setting
|
||||
that changes or suppresses it.
|
||||
|
||||
This is worth knowing before the first request reaches a browser: a
|
||||
client that sees it once will refuse plaintext HTTP to that hostname —
|
||||
and to every subdomain of it — for two years, whatever else is served
|
||||
there. Terminate TLS on a hostname you are prepared to keep on HTTPS.
|
||||
|
||||
## Backup, Restore, and Upgrades
|
||||
|
||||
### What to back up
|
||||
@@ -678,6 +898,18 @@ Upgrade procedure:
|
||||
4. Confirm `database migrations completed` in the logs before putting
|
||||
traffic back on it.
|
||||
|
||||
**Upgrading past the introduction of `BIND_ADDRESS`:** earlier versions
|
||||
always bound every interface. **Container deployments are unaffected**
|
||||
— the image ships `ENV BIND_ADDRESS=0.0.0.0`, so a `docker run` or
|
||||
Compose service that worked before still works with nothing changed.
|
||||
|
||||
A **bare binary** is the case that changes: the listener now binds
|
||||
`127.0.0.1` unless `BIND_ADDRESS` says otherwise, so a deployment that
|
||||
relied on reaching it from another host becomes unreachable until it
|
||||
sets the address the proxy connects to. Check the `bindAddress` field
|
||||
of the startup log to see what a running process bound. See
|
||||
[Bind address](#bind-address).
|
||||
|
||||
**Downgrade is unsupported.** Once a newer binary has migrated the files
|
||||
there is no way to move them back. `AutoMigrate` is additive — it adds
|
||||
tables, columns and indexes and never drops or rewrites them — so an
|
||||
|
||||
Reference in New Issue
Block a user