|
|
|
@@ -1,8 +1,8 @@
|
|
|
|
|
# webhooker
|
|
|
|
|
|
|
|
|
|
webhooker is an MIT-licensed web service, written in [Go](https://golang.org) by
|
|
|
|
|
[@sneak](https://sneak.berlin), that receives the HTTP requests other services
|
|
|
|
|
send when something happens and forwards them to your own systems.
|
|
|
|
|
webhooker is an MIT-licensed [Go](https://golang.org) web service by
|
|
|
|
|
[@sneak](https://sneak.berlin) that receives the HTTP requests other services
|
|
|
|
|
send and forwards them to your own systems.
|
|
|
|
|
|
|
|
|
|
Many services send an HTTP request to a URL you give them when something
|
|
|
|
|
happens: a code push, a payment, a new ticket. webhooker receives those
|
|
|
|
@@ -30,8 +30,8 @@ Five words name the moving parts.
|
|
|
|
|
it. The UUID is a random 36-character identifier, and it is the only thing
|
|
|
|
|
that authorises a sender.
|
|
|
|
|
- A **target** is a place events are delivered to. There are four kinds: an HTTP
|
|
|
|
|
URL, a Slack or Mattermost message-posting URL, an archive file, and the
|
|
|
|
|
application log.
|
|
|
|
|
URL, the message-posting URL of a chat service (Slack or Mattermost), an
|
|
|
|
|
archive file, and the application log.
|
|
|
|
|
- An **event** is one stored request: its method, query string, headers and
|
|
|
|
|
body.
|
|
|
|
|
- A **delivery** is one event paired with one target. It records every attempt.
|
|
|
|
@@ -39,15 +39,15 @@ Five words name the moving parts.
|
|
|
|
|
The part of webhooker that answers requests to `/h/{uuid}` is the receiver. It
|
|
|
|
|
stores the event and one pending delivery per active target, then answers `200`.
|
|
|
|
|
|
|
|
|
|
Delivery is at least once, not exactly once. If a delivery reaches the target
|
|
|
|
|
but webhooker cannot record that outcome, it sends the delivery again later. A
|
|
|
|
|
delivery in flight when the service stops is also sent again at the next start.
|
|
|
|
|
webhooker adds no delivery identifier of its own to the request, so make your
|
|
|
|
|
targets tolerate a duplicate.
|
|
|
|
|
Delivery is at least once, not exactly once. A delivery can reach its target
|
|
|
|
|
more than once. If a delivery reaches the target but webhooker cannot record
|
|
|
|
|
that outcome, it sends the delivery again later. A delivery in flight when the
|
|
|
|
|
service stops is also sent again at the next start. webhooker adds no delivery
|
|
|
|
|
identifier of its own to the request, so make your targets tolerate a duplicate.
|
|
|
|
|
|
|
|
|
|
## Getting Started
|
|
|
|
|
|
|
|
|
|
You need the Go version named in `go.mod`, and Docker for the checks.
|
|
|
|
|
You need the Go version named in `go.mod`.
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
git clone https://git.eeqj.de/sneak/webhooker.git
|
|
|
|
@@ -57,7 +57,8 @@ DATA_DIR=../webhooker-data make dev
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`DATA_DIR` is where webhooker keeps its databases. It defaults to
|
|
|
|
|
`/var/lib/webhooker`, so point it at a directory you can write to.
|
|
|
|
|
`/var/lib/webhooker`, so point it at a directory you can write to. `make build`
|
|
|
|
|
writes the program to `bin/webhooker`. Running that file starts webhooker.
|
|
|
|
|
|
|
|
|
|
The first start creates the `admin` account and prints its password once, in a
|
|
|
|
|
ruled box on standard output:
|
|
|
|
@@ -87,6 +88,9 @@ docker run -d \
|
|
|
|
|
webhooker
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The first-boot password goes to the container's log. Read it with `docker logs`
|
|
|
|
|
and the container's name.
|
|
|
|
|
|
|
|
|
|
In production, put a reverse proxy in front of webhooker: a server that accepts
|
|
|
|
|
the public connections and passes them on. Read
|
|
|
|
|
[Behind a reverse proxy](#behind-a-reverse-proxy) first.
|
|
|
|
@@ -99,12 +103,12 @@ sooner after 24 hours without use, unless `SESSION_IDLE_TIMEOUT` says otherwise.
|
|
|
|
|
|
|
|
|
|
### Webhooks
|
|
|
|
|
|
|
|
|
|
**New Webhook** on the webhook list at `/hooks` asks for **Name**, an optional
|
|
|
|
|
On the webhook list at `/hooks`, **New Webhook** asks for **Name**, an optional
|
|
|
|
|
**Description** and **Retention (days)**. Events older than the retention period
|
|
|
|
|
are deleted, along with their deliveries. The default is 30 days. `0` keeps
|
|
|
|
|
events forever. The form also has an **HTTP target URL** field and an
|
|
|
|
|
**Archive** checkbox; each creates that target with the webhook. Every new
|
|
|
|
|
webhook gets one entrypoint, described as `Default entrypoint`.
|
|
|
|
|
events forever. Filling in **HTTP target URL** creates an `http` target with the
|
|
|
|
|
webhook. Ticking **Archive** creates an archive target. Every new webhook gets
|
|
|
|
|
one entrypoint, described as `Default entrypoint`.
|
|
|
|
|
|
|
|
|
|
### Entrypoint URLs
|
|
|
|
|
|
|
|
|
@@ -120,7 +124,7 @@ deactivated entrypoint answers `410 Gone`. A deleted or unknown one answers
|
|
|
|
|
|
|
|
|
|
### Targets
|
|
|
|
|
|
|
|
|
|
**Add** under **Targets** asks for the type first, then that type's settings.
|
|
|
|
|
Under **Targets**, **Add** asks for the type first, then that type's settings.
|
|
|
|
|
|
|
|
|
|
- An `http` target posts each event to a URL, with optional **Headers** and a
|
|
|
|
|
**Timeout**.
|
|
|
|
@@ -135,6 +139,8 @@ deactivated entrypoint answers `410 Gone`. A deleted or unknown one answers
|
|
|
|
|
`http` and `slack` targets also take **Delivery attempts**: how many times each
|
|
|
|
|
delivery is tried in all. The default is 1. The most is 20. A target's type
|
|
|
|
|
cannot be changed. Deleting an archive target leaves its files on disk.
|
|
|
|
|
**Deactivate** and **Activate** on the webhook page switch a target off and on,
|
|
|
|
|
and new events skip a target that is off.
|
|
|
|
|
|
|
|
|
|
### Events
|
|
|
|
|
|
|
|
|
@@ -144,11 +150,11 @@ body** link. **Full Event Log** shows the 50 newest events with their deliveries
|
|
|
|
|
and every attempt. **Failed** and **Pending** narrow it to events with a failed
|
|
|
|
|
delivery, or with one still pending or retrying.
|
|
|
|
|
|
|
|
|
|
**Replay** sits beside each finished delivery. It creates a new delivery of the
|
|
|
|
|
same event to the same target, with the target's settings as they stand now.
|
|
|
|
|
**Resubmit** sits beside each event. It stores a copy of the event as a new
|
|
|
|
|
event and delivers it to every target that is active now. Both actions are
|
|
|
|
|
limited to 30 per minute per client.
|
|
|
|
|
A finished delivery has a **Replay** button while its target exists. **Replay**
|
|
|
|
|
creates a new delivery of the same event to the same target, with the target's
|
|
|
|
|
settings as they stand now. **Resubmit** sits beside each event. It stores a
|
|
|
|
|
copy of the event as a new event and delivers it to every target that is active
|
|
|
|
|
now. Both actions are limited to 30 per minute per client.
|
|
|
|
|
|
|
|
|
|
### Profile and settings
|
|
|
|
|
|
|
|
|
@@ -171,45 +177,51 @@ Do not set `BIND_ADDRESS` to an empty value in a container. An empty value means
|
|
|
|
|
unset, and unset falls back to `127.0.0.1`, so the container answers nothing on
|
|
|
|
|
its published port.
|
|
|
|
|
|
|
|
|
|
The app runs as user `webhooker`. The image's start-up script,
|
|
|
|
|
`deploy/docker-entrypoint.sh`, gives `DATA_DIR` and its contents to that user
|
|
|
|
|
The app runs as user `webhooker`. The image's start-up script is
|
|
|
|
|
`deploy/docker-entrypoint.sh`. It gives `DATA_DIR` and its contents to that user
|
|
|
|
|
before the app starts. So a host directory can be mounted as it is. webhooker
|
|
|
|
|
creates every SQLite file with mode `0600`.
|
|
|
|
|
creates every SQLite file readable and writable by that user alone (mode
|
|
|
|
|
`0600`).
|
|
|
|
|
|
|
|
|
|
### Running under upaas
|
|
|
|
|
|
|
|
|
|
[upaas](https://git.eeqj.de/sneak/upaas), a deployment tool, builds the image
|
|
|
|
|
from this repository's `Dockerfile` and runs it. Add no port mapping, because
|
|
|
|
|
upaas publishes every mapped port on all interfaces of the host. Put the app on
|
|
|
|
|
your reverse proxy's Docker network instead, and mount one host directory at
|
|
|
|
|
`/var/lib/webhooker`.
|
|
|
|
|
[upaas](https://git.eeqj.de/sneak/upaas) is a deployment tool. It builds the
|
|
|
|
|
image from this repository's `Dockerfile` and runs it. Add no port mapping,
|
|
|
|
|
because upaas publishes every mapped port on all interfaces of the host. Put the
|
|
|
|
|
app on your reverse proxy's Docker network instead, and mount one host directory
|
|
|
|
|
at `/var/lib/webhooker`. The first-boot password goes to the container's log.
|
|
|
|
|
Read it with `docker logs` and the name upaas gave the container.
|
|
|
|
|
|
|
|
|
|
### Behind a reverse proxy
|
|
|
|
|
|
|
|
|
|
webhooker speaks plain HTTP. It does not handle TLS, the encryption behind
|
|
|
|
|
HTTPS: it holds no certificate and accepts no HTTPS connections. The reverse
|
|
|
|
|
proxy holds the certificate and reaches webhooker over a private address. On a
|
|
|
|
|
bare host that is loopback, the address only this host can reach. For the image
|
|
|
|
|
it is the Docker bridge.
|
|
|
|
|
proxy holds the certificate and reaches webhooker over a private address. A bare
|
|
|
|
|
host is a host without Docker. On a bare host the private address is loopback
|
|
|
|
|
(`127.0.0.1`), which only this host can reach. For the image it is the Docker
|
|
|
|
|
bridge, the private network Docker puts containers on.
|
|
|
|
|
|
|
|
|
|
Four things must be right.
|
|
|
|
|
|
|
|
|
|
1. **Keep the app port private.** Only the proxy may reach it: `127.0.0.1` on a
|
|
|
|
|
bare host, or a port published to `127.0.0.1` for the image.
|
|
|
|
|
2. **Send `X-Forwarded-Proto: https`.** This header tells webhooker the request
|
|
|
|
|
arrived over TLS. It decides the `Secure` flag on cookies, and whether a form
|
|
|
|
|
submission must carry a matching `Origin` or `Referer`. webhooker reads it
|
|
|
|
|
from any connecting machine, so the proxy must overwrite whatever the client
|
|
|
|
|
sent.
|
|
|
|
|
arrived over TLS. It decides the `Secure` flag on cookies, the flag that
|
|
|
|
|
makes a browser send them only over HTTPS. It also decides whether a form
|
|
|
|
|
submission must carry a matching `Origin` or `Referer` header. A browser
|
|
|
|
|
sends those two headers to say which site or page a request came from.
|
|
|
|
|
webhooker reads `X-Forwarded-Proto` from any connecting machine, so the proxy
|
|
|
|
|
must overwrite whatever the client sent.
|
|
|
|
|
3. **Cover the proxy in `TRUSTED_PROXIES`.** Otherwise every client shares one
|
|
|
|
|
rate-limit bucket, the count a limit keeps per client. The default list
|
|
|
|
|
covers the Docker bridge address, such as `172.17.0.1`, that the image sees a
|
|
|
|
|
proxy at. It does not cover loopback, so on a bare host set
|
|
|
|
|
`TRUSTED_PROXIES=127.0.0.1`.
|
|
|
|
|
4. **Pass on the client's `Host` header with its port.** webhooker checks a
|
|
|
|
|
form's `Origin` against the `Host` header it receives. A `Host` without the
|
|
|
|
|
port makes every form submission fail with `403` on any port other than 443.
|
|
|
|
|
In nginx that means `$http_host`, not `$host`.
|
|
|
|
|
rate-limit bucket, the count a limit keeps per client. A proxy on the host
|
|
|
|
|
reaches the container from the Docker bridge address, such as `172.17.0.1`.
|
|
|
|
|
The default list covers it. The list does not cover loopback, so on a bare
|
|
|
|
|
host set `TRUSTED_PROXIES=127.0.0.1`.
|
|
|
|
|
4. **Pass on the client's `Host` header with its port.** `Host` is the header
|
|
|
|
|
that names the site the client asked for. webhooker checks a form's `Origin`
|
|
|
|
|
against the `Host` header it receives. A `Host` without the port makes every
|
|
|
|
|
form submission fail with `403` on any port other than `443`. For the nginx
|
|
|
|
|
reverse proxy that means `$http_host`, not `$host`.
|
|
|
|
|
|
|
|
|
|
Protect the proxy's access log: its request paths hold the entrypoint UUIDs.
|
|
|
|
|
|
|
|
|
@@ -233,14 +245,19 @@ server {
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The block matches webhooker's 1 MB body cap and its 60 second request timeout.
|
|
|
|
|
The block matches webhooker's 1 MB body cap. Its `proxy_read_timeout` is longer
|
|
|
|
|
than webhooker's 60 second request timeout, so webhooker's own `504` reaches the
|
|
|
|
|
client first.
|
|
|
|
|
|
|
|
|
|
### HSTS
|
|
|
|
|
|
|
|
|
|
HSTS is a response header that tells browsers to use HTTPS only for this
|
|
|
|
|
hostname and every name under it, for `max-age` seconds. Every response carries
|
|
|
|
|
it unless `HSTS_MAX_AGE` is `0`. A browser keeps the rule until `max-age` runs
|
|
|
|
|
out, so serve webhooker on a hostname you will keep on HTTPS.
|
|
|
|
|
out, so serve webhooker on a hostname you will keep on HTTPS. The header always
|
|
|
|
|
carries `preload`. On a whole domain such as `example.com`, that lets the domain
|
|
|
|
|
be added to the HTTPS-only list that browsers ship with. That listing is hard to
|
|
|
|
|
undo.
|
|
|
|
|
|
|
|
|
|
## Recovering a lost admin password
|
|
|
|
|
|
|
|
|
@@ -252,10 +269,11 @@ the server does and refuses while a running instance holds it.
|
|
|
|
|
DATA_DIR=/var/lib/webhooker webhooker resetpw -generate admin
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Without `-generate`, the new password is read from standard input. It must be at
|
|
|
|
|
least 8 characters. It is never taken as a command-line argument, because other
|
|
|
|
|
accounts on the host can read those. The command creates nothing: a missing
|
|
|
|
|
`DATA_DIR`, a missing `webhooker.db` or an unknown username is an error.
|
|
|
|
|
`-generate` makes up a random password and prints it. Without `-generate`, the
|
|
|
|
|
new password is read from standard input. It must be at least 8 bytes long. It
|
|
|
|
|
is never taken as a command-line argument, because other accounts on the host
|
|
|
|
|
can read those. The command creates nothing: a missing `DATA_DIR`, a missing
|
|
|
|
|
`webhooker.db` or an unknown username is an error.
|
|
|
|
|
|
|
|
|
|
In a container, run the binary at `/app/webhooker` with the stopped container's
|
|
|
|
|
volume mounted:
|
|
|
|
@@ -269,18 +287,19 @@ docker run --rm -v /path/to/data:/var/lib/webhooker \
|
|
|
|
|
|
|
|
|
|
All state lives in `DATA_DIR`. Back up the whole directory. It holds:
|
|
|
|
|
|
|
|
|
|
- the main database, `webhooker.db`: the session key, the `admin` account's
|
|
|
|
|
password hash, and the webhooks, entrypoints and targets;
|
|
|
|
|
- the main database, `webhooker.db`: the session key (which signs the session
|
|
|
|
|
cookies), the `admin` account's password hash, and the webhooks, entrypoints
|
|
|
|
|
and targets;
|
|
|
|
|
- one event database per webhook, `events-{webhook_uuid}.db`: its events,
|
|
|
|
|
deliveries and attempts;
|
|
|
|
|
- the files of each archive target;
|
|
|
|
|
- the lock file, `webhooker.lock`, which is always empty.
|
|
|
|
|
|
|
|
|
|
Every database runs in WAL mode. WAL is SQLite's write-ahead log: finished
|
|
|
|
|
Every database runs in WAL mode. WAL is SQLite's write-ahead log. Finished
|
|
|
|
|
writes land in a `-wal` file beside the database before they reach the `.db`
|
|
|
|
|
itself, and a `-shm` file beside it indexes that log. A copy of the `.db`
|
|
|
|
|
without its `-wal` is missing data. A clean shutdown removes the `-wal` and
|
|
|
|
|
`-shm` files; a crash leaves them, and they must be carried with the `.db`. Keep
|
|
|
|
|
itself. A `-shm` file beside it indexes that log. A copy of the `.db` without
|
|
|
|
|
its `-wal` is missing data. A clean shutdown removes the `-wal` and `-shm`
|
|
|
|
|
files; a crash leaves them, and they must be carried with the `.db`. Keep
|
|
|
|
|
`DATA_DIR` on a local disk: WAL mode is not reliable on network filesystems, and
|
|
|
|
|
webhooker does not check for one.
|
|
|
|
|
|
|
|
|
@@ -296,7 +315,7 @@ done
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Each file is captured at its own instant. A webhook created during the backup
|
|
|
|
|
can be in the copy of `webhooker.db` with no event database in the backup; a
|
|
|
|
|
can be in the copy of `webhooker.db` with no event database in the backup. A
|
|
|
|
|
restore then creates that event database empty.
|
|
|
|
|
|
|
|
|
|
To restore, stop the service and put back every file together, with any `-wal`
|
|
|
|
@@ -314,11 +333,11 @@ plain text. Protect it accordingly.
|
|
|
|
|
|
|
|
|
|
### The entrypoint URL is the credential
|
|
|
|
|
|
|
|
|
|
Each entrypoint is a random version 4 UUID served at `/h/{uuid}`. Holding that
|
|
|
|
|
URL is what authorises a sender. The receiver checks nothing else: no shared
|
|
|
|
|
secret, no HMAC signature (a hash of the body made with a shared key), no token
|
|
|
|
|
header. Signature headers a sender adds anyway are stored and forwarded as
|
|
|
|
|
ordinary headers.
|
|
|
|
|
Each entrypoint is a random UUID served at `/h/{uuid}`. Holding that URL is what
|
|
|
|
|
authorises a sender. The receiver checks nothing else: no shared secret, no HMAC
|
|
|
|
|
signature (a hash of the body made with a shared key), no token header. If a
|
|
|
|
|
sender adds signature headers anyway, they are stored and forwarded as ordinary
|
|
|
|
|
headers.
|
|
|
|
|
|
|
|
|
|
Keep the URL out of tickets, chat and screenshots. webhooker's own log never
|
|
|
|
|
writes it. The reverse proxy's access log does hold it, in the request path.
|
|
|
|
@@ -327,33 +346,37 @@ writes it. The reverse proxy's access log does hold it, in the request path.
|
|
|
|
|
|
|
|
|
|
- Passwords are hashed with Argon2id, a slow password hashing function.
|
|
|
|
|
- Session cookies are signed, `HttpOnly` (hidden from page scripts) and
|
|
|
|
|
`SameSite=Lax` (not sent with cross-site form posts). They carry `Secure` when
|
|
|
|
|
the request carries `X-Forwarded-Proto: https`.
|
|
|
|
|
`SameSite=Lax` (not sent with cross-site form posts). They carry `Secure`
|
|
|
|
|
(sent only over HTTPS) when the request carries `X-Forwarded-Proto: https`.
|
|
|
|
|
- Every form is protected against cross-site request forgery (CSRF). CSRF is
|
|
|
|
|
when another site makes your browser submit a form here. Each form carries a
|
|
|
|
|
token tied to a cookie, and a submission's `Origin` must name this host.
|
|
|
|
|
- Form bodies are capped at 1 MB.
|
|
|
|
|
- Every response carries these headers:
|
|
|
|
|
- `Strict-Transport-Security`, unless `HSTS_MAX_AGE` is `0`;
|
|
|
|
|
- `X-Content-Type-Options: nosniff`;
|
|
|
|
|
- `X-Frame-Options: DENY`;
|
|
|
|
|
- a Content-Security-Policy (CSP), the header that tells the browser where
|
|
|
|
|
scripts and styles may come from, set to
|
|
|
|
|
`default-src 'self'; script-src 'self'; style-src 'self'`;
|
|
|
|
|
- `Referrer-Policy: strict-origin-when-cross-origin`;
|
|
|
|
|
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`.
|
|
|
|
|
- `Strict-Transport-Security`, the HSTS header, unless `HSTS_MAX_AGE` is
|
|
|
|
|
`0`;
|
|
|
|
|
- `X-Content-Type-Options: nosniff`, which stops the browser guessing a
|
|
|
|
|
file's type;
|
|
|
|
|
- `X-Frame-Options: DENY`, which stops other sites showing these pages in a
|
|
|
|
|
frame;
|
|
|
|
|
- `Content-Security-Policy: default-src 'self'; script-src 'self'; style-src 'self'`,
|
|
|
|
|
which tells the browser where scripts and styles may come from;
|
|
|
|
|
- `Referrer-Policy: strict-origin-when-cross-origin`, which tells another
|
|
|
|
|
site only which site a visitor came from, not which page;
|
|
|
|
|
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`, which
|
|
|
|
|
denies the pages the camera, microphone and location.
|
|
|
|
|
- `/metrics` sits behind HTTP basic authentication, a username and password sent
|
|
|
|
|
with each request.
|
|
|
|
|
|
|
|
|
|
### Sign-in limits
|
|
|
|
|
|
|
|
|
|
The password is checked first, and only a failed sign-in is counted. A correct
|
|
|
|
|
password is never answered `429`. Failures are counted per client and submitted
|
|
|
|
|
username. From the fifth failure within a minute, each failure is answered
|
|
|
|
|
`429`. At most two password checks run at once; a request that gets no turn
|
|
|
|
|
within 5 seconds is answered `503`. So those two checks, not the failure count,
|
|
|
|
|
bound how fast passwords can be guessed. Choose a long random admin password, or
|
|
|
|
|
rate-limit `POST /pages/login` at the reverse proxy.
|
|
|
|
|
From the fifth failure within a minute, each failure is answered `429`. Only
|
|
|
|
|
failures are counted. A correct password is never answered `429`. Failures are
|
|
|
|
|
counted per client and submitted username. At most two password checks run at
|
|
|
|
|
once; a request that gets no turn within 5 seconds is answered `503`. So
|
|
|
|
|
guessing speed is bounded by the limit of two checks at a time, not by the
|
|
|
|
|
failure count. Choose a long random `admin` password, or rate-limit
|
|
|
|
|
`POST /pages/login` at the reverse proxy.
|
|
|
|
|
|
|
|
|
|
### Delivery destinations
|
|
|
|
|
|
|
|
|
@@ -361,8 +384,9 @@ A delivery target could be used to make webhooker send requests into the network
|
|
|
|
|
it sits in and read the answers back. This is called server-side request forgery
|
|
|
|
|
(SSRF). webhooker refuses it with the SSRF guard.
|
|
|
|
|
|
|
|
|
|
Every target URL must use `http` or `https` and resolve to a public address. The
|
|
|
|
|
SSRF guard refuses these ranges:
|
|
|
|
|
Every target URL must use `http` or `https`. By default its hostname must also
|
|
|
|
|
point at a public address, one reachable from the internet. The SSRF guard
|
|
|
|
|
refuses these ranges:
|
|
|
|
|
|
|
|
|
|
- loopback, `127.0.0.0/8` and `::1`;
|
|
|
|
|
- the RFC 1918 private ranges, `10.0.0.0/8`, `172.16.0.0/12` and
|
|
|
|
@@ -370,17 +394,20 @@ SSRF guard refuses these ranges:
|
|
|
|
|
- link-local space, which is valid only on one network segment;
|
|
|
|
|
- carrier-grade NAT space, which internet providers use inside their own
|
|
|
|
|
networks;
|
|
|
|
|
- multicast, the IPv6 unique local range, and the other reserved ranges.
|
|
|
|
|
- multicast, the addresses that reach a group of hosts at once;
|
|
|
|
|
- the IPv6 unique local range, `fc00::/7`, the private space of IPv6;
|
|
|
|
|
- the other reserved ranges.
|
|
|
|
|
|
|
|
|
|
The SSRF guard runs when a target is saved and again when each delivery
|
|
|
|
|
connects, so a hostname that changes its address later is still checked.
|
|
|
|
|
connects. So a hostname that changes its address later is still checked.
|
|
|
|
|
Redirects are followed up to ten hops, and each hop is checked. A hop to another
|
|
|
|
|
host or port, or from `https` to `http`, carries none of the configured or
|
|
|
|
|
forwarded headers, so a redirect cannot pass a credential to another host.
|
|
|
|
|
host or port, or from `https` to `http`, drops every configured or forwarded
|
|
|
|
|
header except `Content-Type`, which names the body's format. So a redirect
|
|
|
|
|
cannot pass a credential to another host.
|
|
|
|
|
|
|
|
|
|
Egress means the connections webhooker opens outward to deliver.
|
|
|
|
|
[`ALLOWED_EGRESS_CIDRS`](#allowing-egress-to-your-own-network) opens named
|
|
|
|
|
networks to it.
|
|
|
|
|
[`ALLOWED_EGRESS_CIDRS`](#allowing-egress-to-your-own-network) lets deliveries
|
|
|
|
|
reach networks you name.
|
|
|
|
|
|
|
|
|
|
### What the logs contain
|
|
|
|
|
|
|
|
|
@@ -394,8 +421,9 @@ The `admin` password is printed in the clear on the first boot and by
|
|
|
|
|
query string, headers and body.
|
|
|
|
|
|
|
|
|
|
When `SENTRY_DSN` is set, error reports go to Sentry, an error reporting
|
|
|
|
|
service. A report carries no query string, body or cookies, and the route
|
|
|
|
|
pattern (the route as registered, such as `/h/{uuid}`) in place of the path.
|
|
|
|
|
service. A report carries no query string, body or cookies. It carries the route
|
|
|
|
|
pattern in place of the path. A route pattern is the path with its variable part
|
|
|
|
|
left as a name, such as `/h/{uuid}`.
|
|
|
|
|
|
|
|
|
|
## Design
|
|
|
|
|
|
|
|
|
@@ -404,21 +432,23 @@ sends them on in the background, and SQLite holds everything in between.
|
|
|
|
|
|
|
|
|
|
The receiver stores the event and one pending delivery per active target in one
|
|
|
|
|
database write, then queues the deliveries. Background workers run them and
|
|
|
|
|
record each attempt. At start and then every minute, a recovery routine queues
|
|
|
|
|
again any delivery left `pending` or `retrying`, so a stop or a crash loses no
|
|
|
|
|
delivery.
|
|
|
|
|
record each attempt. At start, a recovery routine resumes every delivery left
|
|
|
|
|
`pending` or `retrying`. It then runs every minute. Each run resumes every
|
|
|
|
|
`retrying` delivery that the engine is not already handling. It also resumes a
|
|
|
|
|
`pending` delivery that has sat untouched for 15 minutes. So a stop or a crash
|
|
|
|
|
loses no delivery.
|
|
|
|
|
|
|
|
|
|
The main database holds configuration. Each webhook has its own event database,
|
|
|
|
|
so writes for one webhook never wait on another webhook's database. The program
|
|
|
|
|
starts in `cmd/webhooker/main.go`, and its packages are under `internal/`. The
|
|
|
|
|
pages' HTML, CSS and JavaScript are embedded in the binary.
|
|
|
|
|
starts in `cmd/webhooker/main.go`, and its logic is under `internal/`. The
|
|
|
|
|
`static` and `templates` packages embed the pages' HTML, CSS and JavaScript in
|
|
|
|
|
the binary.
|
|
|
|
|
|
|
|
|
|
## Configuration reference
|
|
|
|
|
|
|
|
|
|
All configuration comes from environment variables. For development, variables
|
|
|
|
|
can also go in a `.env` file in the working directory. A variable set in the
|
|
|
|
|
real environment wins over the file. A `.env` file that cannot be parsed aborts
|
|
|
|
|
startup.
|
|
|
|
|
All configuration comes from environment variables. Variables can also go in a
|
|
|
|
|
`.env` file in the working directory. A variable set in the real environment
|
|
|
|
|
wins over the file. A `.env` file that cannot be parsed aborts startup.
|
|
|
|
|
|
|
|
|
|
A variable that is unset or empty takes its default. A variable that is set but
|
|
|
|
|
cannot be parsed aborts startup with a message naming it. A Go duration is a
|
|
|
|
@@ -427,16 +457,20 @@ number with a unit, such as `30s`, `15m` or `24h`.
|
|
|
|
|
### `WEBHOOKER_ENVIRONMENT`
|
|
|
|
|
|
|
|
|
|
`dev` or `prod`. Default `prod`. This controls one thing: CORS, the headers that
|
|
|
|
|
let a page on another site call this one from a browser. In `dev`, every origin
|
|
|
|
|
is allowed. In `prod`, no CORS headers are sent.
|
|
|
|
|
let a page on another site call this one from a browser. An origin is a site as
|
|
|
|
|
a browser names it: scheme, host and port. In `dev`, every origin is allowed. In
|
|
|
|
|
`prod`, no CORS headers are sent.
|
|
|
|
|
|
|
|
|
|
### `PORT`
|
|
|
|
|
|
|
|
|
|
The TCP port the listener binds. Default `8080`.
|
|
|
|
|
The port webhooker listens on. Default `8080`. Docker marks the image's
|
|
|
|
|
container healthy by a check every 30 seconds. The check requests port `8080`
|
|
|
|
|
whatever `PORT` says, so a container started with another `PORT` reports
|
|
|
|
|
unhealthy.
|
|
|
|
|
|
|
|
|
|
### `BIND_ADDRESS`
|
|
|
|
|
|
|
|
|
|
The IP address the listener binds. Default `127.0.0.1`. The Docker image sets
|
|
|
|
|
The IP address webhooker listens on. Default `127.0.0.1`. The Docker image sets
|
|
|
|
|
`0.0.0.0`. It must be an IP address literal: `localhost` and a value with a port
|
|
|
|
|
are refused. `0.0.0.0` listens on IPv4 only; `::` listens on IPv6 and IPv4.
|
|
|
|
|
|
|
|
|
@@ -445,17 +479,17 @@ are refused. `0.0.0.0` listens on IPv4 only; `::` listens on IPv6 and IPv4.
|
|
|
|
|
The directory for every SQLite database and the lock file. Default
|
|
|
|
|
`/var/lib/webhooker`. It is created if it is missing.
|
|
|
|
|
|
|
|
|
|
Exactly one webhooker process may use a `DATA_DIR` at a time; two would both run
|
|
|
|
|
the recovery routine and send every pending delivery twice. At startup the
|
|
|
|
|
process locks `{DATA_DIR}/webhooker.lock` with `flock`, the kernel's file lock,
|
|
|
|
|
and a second process pointed at the same directory exits with a message naming
|
|
|
|
|
it.
|
|
|
|
|
Exactly one webhooker process may use a `DATA_DIR` at a time. Two would both run
|
|
|
|
|
the recovery routine and could send one delivery twice. At startup the process
|
|
|
|
|
locks `{DATA_DIR}/webhooker.lock` with `flock`, the kernel's file lock. A second
|
|
|
|
|
process pointed at the same directory exits with a message that names the
|
|
|
|
|
directory and the lock file.
|
|
|
|
|
|
|
|
|
|
### `DEBUG`
|
|
|
|
|
|
|
|
|
|
`true` or `false`. Default `false`. It accepts the spellings Go's
|
|
|
|
|
`strconv.ParseBool` accepts; `yes` and `on` are refused. `DEBUG=true` lowers the
|
|
|
|
|
log level to `DEBUG`, which adds every SQL statement, with `?` in place of each
|
|
|
|
|
log level to `DEBUG`, which adds SQL statements, with `?` in place of each
|
|
|
|
|
value, and other diagnostic lines.
|
|
|
|
|
|
|
|
|
|
### `METRICS_USERNAME` and `METRICS_PASSWORD`
|
|
|
|
@@ -504,11 +538,11 @@ private ranges. A set value replaces the default.
|
|
|
|
|
|
|
|
|
|
The list decides whose `X-Forwarded-For` header is believed. `X-Forwarded-For`
|
|
|
|
|
is the header a proxy uses to pass on the address of the client it is serving.
|
|
|
|
|
It is read only when the connecting machine is inside the list. `X-Real-IP` and
|
|
|
|
|
`True-Client-IP` are never read. If any client can reach webhooker, or the proxy
|
|
|
|
|
in front of it, from a private address, set the list to the proxy's address
|
|
|
|
|
alone. Otherwise those clients can choose their own `X-Forwarded-For` and bypass
|
|
|
|
|
every rate limit.
|
|
|
|
|
It is read only when the connecting machine is inside the list. webhooker never
|
|
|
|
|
reads `X-Real-IP` or `True-Client-IP`, two other headers with the same job. Some
|
|
|
|
|
clients may reach webhooker, or the proxy in front of it, from a private
|
|
|
|
|
address. If so, set the list to the proxy's address alone. Otherwise those
|
|
|
|
|
clients can choose their own `X-Forwarded-For` and bypass every rate limit.
|
|
|
|
|
|
|
|
|
|
### Allowing egress to your own network
|
|
|
|
|
|
|
|
|
@@ -517,13 +551,15 @@ that delivery targets may reach although the SSRF guard refuses them by default.
|
|
|
|
|
Default empty. Example: `10.0.0.0/8, 172.17.0.0/16`. When the list is set,
|
|
|
|
|
startup logs it at `WARN`.
|
|
|
|
|
|
|
|
|
|
Anyone with the admin account can make webhooker send requests to every network
|
|
|
|
|
you list, and read the responses in the event log. List the smallest blocks that
|
|
|
|
|
cover your destinations, and never `0.0.0.0/0` or `::/0`.
|
|
|
|
|
Anyone with the `admin` account can make webhooker send requests to every
|
|
|
|
|
network you list, and read the responses in **Full Event Log**. List the
|
|
|
|
|
smallest networks that cover your destinations, and never `0.0.0.0/0` or `::/0`.
|
|
|
|
|
|
|
|
|
|
These addresses stay blocked whatever you list. They are link-local space, the
|
|
|
|
|
unspecified addresses, and cloud metadata endpoints, the addresses a cloud gives
|
|
|
|
|
each virtual machine for its own credentials:
|
|
|
|
|
unspecified addresses and cloud metadata endpoints. The unspecified addresses
|
|
|
|
|
are `0.0.0.0` and `::`, which name no host. A cloud metadata endpoint is the
|
|
|
|
|
address a cloud gives each virtual machine for its own credentials. The blocked
|
|
|
|
|
addresses are:
|
|
|
|
|
|
|
|
|
|
- `169.254.0.0/16` and `fe80::/10`;
|
|
|
|
|
- `fd00:ec2::254/128` and `fd00:ec2::23/128`;
|
|
|
|
@@ -536,25 +572,25 @@ each virtual machine for its own credentials:
|
|
|
|
|
- `0.0.0.0/32` and `::/128`;
|
|
|
|
|
- `::a9fe:a9fe/128` and `64:ff9b::a9fe:a9fe/128`.
|
|
|
|
|
|
|
|
|
|
This list is not every cloud's metadata address. If yours is not here, do not
|
|
|
|
|
list the block that contains it. Azure's public `168.63.129.16` is refused by
|
|
|
|
|
default but is reopened by a list that covers it.
|
|
|
|
|
Some clouds' metadata addresses are not on this list. If yours is not here, do
|
|
|
|
|
not list the network that contains it. Azure's public metadata address,
|
|
|
|
|
`168.63.129.16`, is refused by default but is reopened by a list that covers it.
|
|
|
|
|
|
|
|
|
|
## Delivery reference
|
|
|
|
|
|
|
|
|
|
An `http` target posts the event body to its URL with the event's `Content-Type`
|
|
|
|
|
and headers, then the target's configured **Headers**, which replace event
|
|
|
|
|
headers of the same name. Every `http` and `slack` request carries
|
|
|
|
|
`User-Agent: webhooker/{version}`, replacing the sender's `User-Agent` and any
|
|
|
|
|
configured one. A `2xx` response is a success. The first 4 KiB of the response
|
|
|
|
|
body is stored with the attempt. **Timeout** is at most 300 seconds, and blank
|
|
|
|
|
means the 30 second default. **Pass the query string on to this target** appends
|
|
|
|
|
the event's query string to the URL.
|
|
|
|
|
An `http` target posts the event body to its URL. The request carries the
|
|
|
|
|
event's `Content-Type` and headers. The target's configured **Headers** are
|
|
|
|
|
added next and replace event headers of the same name. Every `http` and `slack`
|
|
|
|
|
request carries `User-Agent: webhooker/{version}`. `User-Agent` is the header
|
|
|
|
|
that names the sending program. The **Headers** field refuses `User-Agent`. A
|
|
|
|
|
response with a status from `200` to `299` is a success. The first 4 KiB of the
|
|
|
|
|
response body is stored with the attempt. **Timeout** is at most 300 seconds,
|
|
|
|
|
and blank means the 30 second default. **Pass the query string on to this
|
|
|
|
|
target** appends the event's query string to the URL.
|
|
|
|
|
|
|
|
|
|
A `slack` target posts a message that describes the event and shows its body in
|
|
|
|
|
a code block, cut at 3500 bytes. An archive target, whose stored type is
|
|
|
|
|
`database`, appends the event as a row to its own SQLite file. A `log` target
|
|
|
|
|
writes the event as one `INFO` line.
|
|
|
|
|
a code block, cut at 3500 bytes. An archive target appends the event as a row to
|
|
|
|
|
its own SQLite file. A `log` target writes the event as one `INFO` line.
|
|
|
|
|
|
|
|
|
|
**Delivery attempts** is from 1 to 20. A failed attempt marks the delivery
|
|
|
|
|
`retrying` and schedules the next attempt; the wait doubles each time, starting
|
|
|
|
@@ -572,11 +608,12 @@ paused.
|
|
|
|
|
An archive target writes to
|
|
|
|
|
`archive-{webhook_name}-{target_name}-{target_uuid}.db` in `DATA_DIR`, the two
|
|
|
|
|
names made safe for a file name. A rotation of `monthly`, `daily` or `hourly`
|
|
|
|
|
starts a new file for each period, in UTC, with the period before `.db`, such as
|
|
|
|
|
`-2026-10-01.db`. Rows older than the expiry are deleted on each write and by
|
|
|
|
|
the archive sweep. Renaming the webhook or the target renames the files.
|
|
|
|
|
**Download** on the webhook page returns the target's whole archive as one
|
|
|
|
|
gzipped JSON file.
|
|
|
|
|
starts a new file for each period. Periods follow UTC, the universal time that
|
|
|
|
|
ignores local time zones. The period goes before `.db`, such as
|
|
|
|
|
`-2026-10-01.db`. Rows older than the expiry are deleted each time a file is
|
|
|
|
|
opened and by the archive sweep. Renaming the webhook or the target renames the
|
|
|
|
|
files. **Download** on the webhook page returns the target's whole archive as
|
|
|
|
|
one JSON file compressed with gzip.
|
|
|
|
|
|
|
|
|
|
## HTTP routes reference
|
|
|
|
|
|
|
|
|
@@ -587,6 +624,8 @@ gzipped JSON file.
|
|
|
|
|
- an unknown UUID gets `404`;
|
|
|
|
|
- a deactivated entrypoint gets `410`;
|
|
|
|
|
- a body over 1 MB gets `413`;
|
|
|
|
|
- a body that cannot be read in full gets `400`, for example one still
|
|
|
|
|
arriving after the server's 10 second read limit;
|
|
|
|
|
- a failure to store the event gets `500`;
|
|
|
|
|
- a stored event gets `200` with `{"status":"ok"}`.
|
|
|
|
|
- `GET /pages/login` shows the sign-in form, `POST /pages/login` signs in, and
|
|
|
|
@@ -600,9 +639,9 @@ gzipped JSON file.
|
|
|
|
|
## Metrics reference
|
|
|
|
|
|
|
|
|
|
`/metrics` serves these metrics for Prometheus, the monitoring system, plus the
|
|
|
|
|
Go runtime and process metrics the Prometheus library provides. Each delivery
|
|
|
|
|
metric carries one label, `target_type`, which is `http`, `slack`, `database`,
|
|
|
|
|
`log` or `unknown`.
|
|
|
|
|
Go runtime and process metrics the Prometheus library provides. A label is a
|
|
|
|
|
name and value attached to a metric. Each delivery metric carries one label,
|
|
|
|
|
`target_type`, which is `http`, `slack`, `database`, `log` or `unknown`.
|
|
|
|
|
|
|
|
|
|
- `webhooker_events_received_total` and `webhooker_events_resubmitted_total`,
|
|
|
|
|
two counters (totals that only grow) with no label;
|
|
|
|
@@ -616,17 +655,17 @@ metric carries one label, `target_type`, which is `http`, `slack`, `database`,
|
|
|
|
|
(current values) sampled every 30 seconds;
|
|
|
|
|
- `webhooker_circuit_breakers_open`.
|
|
|
|
|
|
|
|
|
|
Two inbound HTTP metrics, `http_request_duration_seconds` and
|
|
|
|
|
`http_response_size_bytes`, carry the route pattern as their `handler` label,
|
|
|
|
|
never the real path. A third, `http_requests_inflight`, counts all routes
|
|
|
|
|
together.
|
|
|
|
|
`http_request_duration_seconds` and `http_response_size_bytes` measure inbound
|
|
|
|
|
requests. Their `handler` label is the route pattern, never the real path.
|
|
|
|
|
`http_requests_inflight` counts all routes together.
|
|
|
|
|
|
|
|
|
|
## Development
|
|
|
|
|
|
|
|
|
|
You need the Go version named in `go.mod`, and Docker. The tests, the linters,
|
|
|
|
|
prettier and the stylesheet build run in Docker, at versions pinned (fixed to
|
|
|
|
|
one exact release) in the `Dockerfile`, `package.json` and `yarn.lock`. `gofmt`
|
|
|
|
|
and `goimports` run on the host.
|
|
|
|
|
`prettier` and the stylesheet build run in Docker. Their versions are pinned
|
|
|
|
|
(fixed to one exact release) in the `Dockerfile`, `package.json` and
|
|
|
|
|
`yarn.lock`. `gofmt` runs on the host. `goimports` runs on the host too, when it
|
|
|
|
|
is installed. `script/bootstrap` does not install it.
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
make bootstrap # Install git, make, Go and the Go modules
|
|
|
|
@@ -639,25 +678,27 @@ make build # Build bin/webhooker, version-stamped
|
|
|
|
|
make docker # Build the Docker image
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Each `make` target except `build`, `run`, `dev`, `deps` and `clean` runs a
|
|
|
|
|
script in `script/`, listed under [Entrypoints](#entrypoints). Every
|
|
|
|
|
`docker build` in `script/` passes `--no-cache`, so a check that passes has
|
|
|
|
|
really run.
|
|
|
|
|
Most `make` targets run a script in `script/`, listed under
|
|
|
|
|
[Entrypoints](#entrypoints). `build`, `run`, `dev`, `deps` and `clean` run their
|
|
|
|
|
own commands. Every `docker build` in `script/` passes `--no-cache`, so a check
|
|
|
|
|
that passes has really run.
|
|
|
|
|
|
|
|
|
|
The binary's version comes from `script/version`: `git describe` on the
|
|
|
|
|
checkout, or `VERSION` when that is set. It is shown at
|
|
|
|
|
`/.well-known/healthcheck` and in the page footer.
|
|
|
|
|
|
|
|
|
|
The pages use one third-party script, Alpine.js, in its CSP build
|
|
|
|
|
`@alpinejs/csp`, which runs under a Content-Security-Policy that forbids `eval`.
|
|
|
|
|
The npm registry is where JavaScript packages are published. The package file in
|
|
|
|
|
`3p/` is byte for byte the one it publishes.
|
|
|
|
|
The pages use one third-party script, Alpine.js. They load `@alpinejs/csp`, the
|
|
|
|
|
version of it that works when the Content-Security-Policy forbids running text
|
|
|
|
|
as code. The npm registry is where JavaScript packages are published.
|
|
|
|
|
`static/js/alpine.min.js` is committed byte for byte as the npm registry
|
|
|
|
|
publishes it.
|
|
|
|
|
|
|
|
|
|
A browser test in `internal/server` loads the pages in a headless browser, one
|
|
|
|
|
without a window, under the real Content-Security-Policy. `make test` leaves it
|
|
|
|
|
out; run `make test-browser` after changing `templates/` or `static/js/`.
|
|
|
|
|
`static/css/tailwind.css` is generated by Tailwind and committed; `make check`
|
|
|
|
|
fails when it is out of date, and `make css` regenerates it.
|
|
|
|
|
A headless browser is a browser without a window. A browser test in
|
|
|
|
|
`internal/server` loads the pages in one under the real Content-Security-Policy.
|
|
|
|
|
`make test` leaves it out; run `make test-browser` after changing `templates/`
|
|
|
|
|
or `static/js/`. `static/css/tailwind.css` is generated by Tailwind, a CSS
|
|
|
|
|
generator. The file is committed. `make check` fails when it is out of date, and
|
|
|
|
|
`make css` regenerates it.
|
|
|
|
|
|
|
|
|
|
## Entrypoints
|
|
|
|
|
|
|
|
|
@@ -670,26 +711,25 @@ application are under [Entrypoint URLs](#entrypoint-urls).
|
|
|
|
|
- `script/bootstrap` installs git, make, Go and the Go modules.
|
|
|
|
|
- `script/setup` runs `script/bootstrap`, then `script/install-precommit`.
|
|
|
|
|
- `script/projectname` prints `webhooker`.
|
|
|
|
|
- `script/assets` extracts Alpine.js from `3p/` to `static/js/alpine.min.js`.
|
|
|
|
|
- `script/test` runs the tests in Docker.
|
|
|
|
|
- `script/test-browser` runs the browser test in Docker.
|
|
|
|
|
- `script/lint` runs `gofmt`, `golangci-lint` and ESLint in Docker.
|
|
|
|
|
- `script/fmt` formats the Go code with `gofmt` and `goimports` on the host, and
|
|
|
|
|
the Markdown with prettier in Docker.
|
|
|
|
|
- `script/fmt-check` fails when `gofmt` or prettier would change a file. It does
|
|
|
|
|
not run `goimports`, and it writes nothing.
|
|
|
|
|
- `script/fmt` formats the Go code with `gofmt` on the host and the Markdown
|
|
|
|
|
with `prettier` in Docker. It also runs `goimports` when that is installed.
|
|
|
|
|
- `script/fmt-check` fails when `gofmt` or `prettier` would change a file. It
|
|
|
|
|
does not run `goimports`, and it writes nothing.
|
|
|
|
|
- `script/css` regenerates `static/css/tailwind.css` in Docker.
|
|
|
|
|
- `script/css-check` fails when `static/css/tailwind.css` is out of date.
|
|
|
|
|
- `script/check` runs `script/test`, `script/lint`, `script/fmt-check` and
|
|
|
|
|
`script/css-check`.
|
|
|
|
|
- `script/version` prints the version to stamp into the binary.
|
|
|
|
|
- `script/docker` builds the image tagged `webhooker`.
|
|
|
|
|
- `script/cibuild` is what CI, the check run on every push, runs:
|
|
|
|
|
- CI is the check that runs on every push. It runs `script/cibuild`:
|
|
|
|
|
`script/bootstrap`, `script/check`, then the image build.
|
|
|
|
|
- `script/precommit` checks that `go mod tidy` changes nothing, then runs
|
|
|
|
|
`script/check`.
|
|
|
|
|
- `script/install-precommit` installs the git pre-commit hook that runs
|
|
|
|
|
`script/precommit`.
|
|
|
|
|
- `script/install-precommit` installs the git hook that runs `script/precommit`
|
|
|
|
|
before each commit.
|
|
|
|
|
|
|
|
|
|
## TODO
|
|
|
|
|
|
|
|
|
|