1 Commits
Author SHA1 Message Date
sneak 6eedfc96d1 Rewrite README.md sentence by sentence
check / check (push) Canceled after 0s
Rewritten from scratch against the code on next. The old text was
the list of what a reader needs covered.

Model: fable-5-1
2026-10-25 07:42:39 +00:00
+60 -79
View File
@@ -99,12 +99,11 @@ sooner after 24 hours without use, unless `SESSION_IDLE_TIMEOUT` says otherwise.
### Webhooks ### Webhooks
**New Webhook** on the webhook list at `/hooks` asks for **Name**, an optional **New Webhook** on the webhook list at `/hooks` asks for a name, an optional
**Description** and **Retention (days)**. Events older than the retention period description and a retention period in days. Events older than the retention
are deleted, along with their deliveries. The default is 30 days. `0` keeps period are deleted, along with their deliveries. The default is 30 days. `0`
events forever. The form also has an **HTTP target URL** field and an keeps events forever. Every new webhook gets one entrypoint, described as
**Archive** checkbox; each creates that target with the webhook. Every new `Default entrypoint`.
webhook gets one entrypoint, described as `Default entrypoint`.
### Entrypoint URLs ### Entrypoint URLs
@@ -122,14 +121,14 @@ deactivated entrypoint answers `410 Gone`. A deleted or unknown one answers
**Add** under **Targets** asks for the type first, then that type's settings. **Add** under **Targets** asks for the type first, then that type's settings.
- An `http` target posts each event to a URL, with optional **Headers** and a - An `http` target posts each event to a URL, with optional request headers and
**Timeout**. a timeout.
- A `slack` target posts each event as a message. Its **Webhook URL** is the - A `slack` target posts each event as a message to the message-posting URL
message-posting URL Slack or Mattermost give out, which they call an incoming Slack or Mattermost give out, which they call an incoming webhook.
webhook.
- An archive target writes each event as a row in a SQLite file of its own. Its - An archive target writes each event as a row in a SQLite file of its own. Its
stored type is `database`. **Archive expiry** is how long archived events are stored type is `database`. It takes an expiry, which is how long archived
kept, and **Archive rotation** is how often a new archive file is started. events are kept, and a rotation, which is how often a new archive file is
started.
- A `log` target writes each event to the application log. - A `log` target writes each event to the application log.
`http` and `slack` targets also take **Delivery attempts**: how many times each `http` and `slack` targets also take **Delivery attempts**: how many times each
@@ -188,9 +187,8 @@ your reverse proxy's Docker network instead, and mount one host directory at
webhooker speaks plain HTTP. It does not handle TLS, the encryption behind webhooker speaks plain HTTP. It does not handle TLS, the encryption behind
HTTPS: it holds no certificate and accepts no HTTPS connections. The reverse HTTPS: it holds no certificate and accepts no HTTPS connections. The reverse
proxy holds the certificate and reaches webhooker over a private address. On a proxy holds the certificate and reaches webhooker over loopback, the address
bare host that is loopback, the address only this host can reach. For the image only this host can reach.
it is the Docker bridge.
Four things must be right. Four things must be right.
@@ -202,18 +200,16 @@ Four things must be right.
from any connecting machine, so the proxy must overwrite whatever the client from any connecting machine, so the proxy must overwrite whatever the client
sent. sent.
3. **Cover the proxy in `TRUSTED_PROXIES`.** Otherwise every client shares one 3. **Cover the proxy in `TRUSTED_PROXIES`.** Otherwise every client shares one
rate-limit bucket, the count a limit keeps per client. The default list rate-limit bucket, the count a limit keeps per client. The default list does
covers the Docker bridge address, such as `172.17.0.1`, that the image sees a not cover a proxy that connects over loopback.
proxy at. It does not cover loopback, so on a bare host set 4. **Send `Host` as `$http_host`, not `$host`.** webhooker checks a form's
`TRUSTED_PROXIES=127.0.0.1`. `Origin` against the `Host` header it receives. `$host` drops the port, so on
4. **Pass on the client's `Host` header with its port.** webhooker checks a any port other than 443 every form submission fails with `403`.
form's `Origin` against the `Host` header it receives. A `Host` without the
port makes every form submission fail with `403` on any port other than 443.
In nginx that means `$http_host`, not `$host`.
Protect the proxy's access log: its request paths hold the entrypoint UUIDs. Protect the proxy's access log: its request paths hold the entrypoint UUIDs.
An nginx server block for a bare host: An nginx server block for this, with `TRUSTED_PROXIES=127.0.0.1` on webhooker's
side:
```nginx ```nginx
server { server {
@@ -252,18 +248,12 @@ the server does and refuses while a running instance holds it.
DATA_DIR=/var/lib/webhooker webhooker resetpw -generate admin DATA_DIR=/var/lib/webhooker webhooker resetpw -generate admin
``` ```
Without `-generate`, the new password is read from standard input. It must be at Without `-generate`, the new password is read from standard input, at least 8
least 8 characters. It is never taken as a command-line argument, because other characters. It is never taken as a command-line argument, because other accounts
accounts on the host can read those. The command creates nothing: a missing on the host can read those. In a container, run the same command with
`DATA_DIR`, a missing `webhooker.db` or an unknown username is an error. `docker run --rm` and the stopped container's volume mounted. The command
creates nothing: a missing `DATA_DIR`, a missing `webhooker.db` or an unknown
In a container, run the binary at `/app/webhooker` with the stopped container's username is an error.
volume mounted:
```bash
docker run --rm -v /path/to/data:/var/lib/webhooker \
webhooker /app/webhooker resetpw -generate admin
```
## Backup and restore ## Backup and restore
@@ -273,16 +263,15 @@ All state lives in `DATA_DIR`. Back up the whole directory. It holds:
password hash, and the webhooks, entrypoints and targets; password hash, and the webhooks, entrypoints and targets;
- one event database per webhook, `events-{webhook_uuid}.db`: its events, - one event database per webhook, `events-{webhook_uuid}.db`: its events,
deliveries and attempts; deliveries and attempts;
- the files of each archive target; - the archive files, one per archive target;
- the lock file, `webhooker.lock`, which is always empty. - the lock file, `webhooker.lock`, which is always empty.
Every database runs in WAL mode. WAL is SQLite's write-ahead log: finished Every database runs in WAL mode. WAL is SQLite's write-ahead log: finished
writes land in a `-wal` file beside the database before they reach the `.db` writes land in a `-wal` file beside the database before they reach the `.db`
itself, and a `-shm` file beside it indexes that log. A copy of the `.db` itself. A copy of the `.db` without its `-wal` is missing data. A clean shutdown
without its `-wal` is missing data. A clean shutdown removes the `-wal` and removes the `-wal` and `-shm` files; a crash leaves them, and they must be
`-shm` files; a crash leaves them, and they must be carried with the `.db`. Keep carried with the `.db`. Keep `DATA_DIR` on a local disk: WAL mode is not
`DATA_DIR` on a local disk: WAL mode is not reliable on network filesystems, and reliable on network filesystems, and webhooker does not check for one.
webhooker does not check for one.
A copy taken while the service runs can catch a `.db` and its `-wal` at two A copy taken while the service runs can catch a `.db` and its `-wal` at two
different instants. Stop the service, copy the directory, and start it again. different instants. Stop the service, copy the directory, and start it again.
@@ -329,9 +318,9 @@ writes it. The reverse proxy's access log does hold it, in the request path.
- Session cookies are signed, `HttpOnly` (hidden from page scripts) and - Session cookies are signed, `HttpOnly` (hidden from page scripts) and
`SameSite=Lax` (not sent with cross-site form posts). They carry `Secure` when `SameSite=Lax` (not sent with cross-site form posts). They carry `Secure` when
the request carries `X-Forwarded-Proto: https`. the request carries `X-Forwarded-Proto: https`.
- Every form is protected against cross-site request forgery (CSRF). CSRF is - Every form is protected against cross-site request forgery (CSRF), where
when another site makes your browser submit a form here. Each form carries a another site makes your browser submit a form here: each form carries a token
token tied to a cookie, and a submission's `Origin` must name this host. tied to a cookie, and a submission's `Origin` must name this host.
- Form bodies are capped at 1 MB. - Form bodies are capped at 1 MB.
- Every response carries these headers: - Every response carries these headers:
- `Strict-Transport-Security`, unless `HSTS_MAX_AGE` is `0`; - `Strict-Transport-Security`, unless `HSTS_MAX_AGE` is `0`;
@@ -339,16 +328,14 @@ writes it. The reverse proxy's access log does hold it, in the request path.
- `X-Frame-Options: DENY`; - `X-Frame-Options: DENY`;
- a Content-Security-Policy (CSP), the header that tells the browser where - a Content-Security-Policy (CSP), the header that tells the browser where
scripts and styles may come from, set to scripts and styles may come from, set to
`default-src 'self'; script-src 'self'; style-src 'self'`; `default-src 'self'; script-src 'self'; style-src 'self'`.
- `Referrer-Policy: strict-origin-when-cross-origin`;
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`.
- `/metrics` sits behind HTTP basic authentication, a username and password sent - `/metrics` sits behind HTTP basic authentication, a username and password sent
with each request. with each request.
### Sign-in limits ### Sign-in limits
The password is checked first, and only a failed sign-in is counted. A correct The password is checked first, and only a failed sign-in is counted. A correct
password is never answered `429`. Failures are counted per client and submitted password is never throttled. Failures are counted per client and submitted
username. From the fifth failure within a minute, each failure is answered username. From the fifth failure within a minute, each failure is answered
`429`. At most two password checks run at once; a request that gets no turn `429`. At most two password checks run at once; a request that gets no turn
within 5 seconds is answered `503`. So those two checks, not the failure count, within 5 seconds is answered `503`. So those two checks, not the failure count,
@@ -409,9 +396,9 @@ again any delivery left `pending` or `retrying`, so a stop or a crash loses no
delivery. delivery.
The main database holds configuration. Each webhook has its own event database, The main database holds configuration. Each webhook has its own event database,
so writes for one webhook never wait on another webhook's database. The program so writes for one webhook never wait on another webhook's database. The Go code
starts in `cmd/webhooker/main.go`, and its packages are under `internal/`. The is under `internal/`, one package per concern, and the pages' HTML, CSS and
pages' HTML, CSS and JavaScript are embedded in the binary. JavaScript are embedded in the binary.
## Configuration reference ## Configuration reference
@@ -543,25 +530,23 @@ default but is reopened by a list that covers it.
## Delivery reference ## Delivery reference
An `http` target posts the event body to its URL with the event's `Content-Type` An `http` target posts the event body to its URL with the event's `Content-Type`
and headers, then the target's configured **Headers**, which replace event and headers, then the target's configured headers, which replace event headers
headers of the same name. Every `http` and `slack` request carries of the same name. A `2xx` response is a success. The first 4 KiB of the response
`User-Agent: webhooker/{version}`, replacing the sender's `User-Agent` and any body is stored with the attempt. `timeout_seconds` is at most 300, and blank
configured one. A `2xx` response is a success. The first 4 KiB of the response means the 30 second default. `forward_query` appends the event's query string to
body is stored with the attempt. **Timeout** is at most 300 seconds, and blank the URL.
means the 30 second default. **Pass the query string on to this target** appends
the event's query string to the URL.
A `slack` target posts a message that describes the event and shows its body in A `slack` target posts a message that describes the event and shows its body in
a code block, cut at 3500 bytes. An archive target, whose stored type is a code block, cut at 3500 bytes. An archive target, whose stored type is
`database`, appends the event as a row to its own SQLite file. A `log` target `database`, appends the event as a row to its own SQLite file. A `log` target
writes the event as one `INFO` line. writes the event as one `INFO` line.
**Delivery attempts** is from 1 to 20. A failed attempt marks the delivery `http` and `slack` targets take `max_attempts`, from 1 to 20. A failed attempt
`retrying` and schedules the next attempt; the wait doubles each time, starting marks the delivery `retrying` and schedules the next attempt; the wait doubles
at one second. After the last allowed attempt fails, the delivery is marked each time, starting at one second. After the last allowed attempt fails, the
`failed`. delivery is marked `failed`.
A target with more than one delivery attempt has a circuit breaker, which stops A target with `max_attempts` above 1 has a circuit breaker, which stops
hammering a destination that is down. After 5 failures in a row, deliveries to hammering a destination that is down. After 5 failures in a row, deliveries to
the target pause for 30 seconds. Then one delivery is sent as a probe; the the target pause for 30 seconds. Then one delivery is sent as a probe; the
others wait until a probe succeeds. The webhook page says when a target is others wait until a probe succeeds. The webhook page says when a target is
@@ -610,23 +595,21 @@ metric carries one label, `target_type`, which is `http`, `slack`, `database`,
- `webhooker_deliveries_succeeded_total` and - `webhooker_deliveries_succeeded_total` and
`webhooker_deliveries_failed_total`; `webhooker_deliveries_failed_total`;
- `webhooker_delivery_retries_total` and `webhooker_delivery_replays_total`; - `webhooker_delivery_retries_total` and `webhooker_delivery_replays_total`;
- `webhooker_delivery_duration_seconds`, a histogram (a distribution) of how - `webhooker_delivery_duration_seconds`, a histogram (a distribution) of one
long one attempt took; attempt's wall time;
- `webhooker_deliveries_pending` and `webhooker_deliveries_retrying`, gauges - `webhooker_deliveries_pending` and `webhooker_deliveries_retrying`, gauges
(current values) sampled every 30 seconds; (current values) sampled every 30 seconds;
- `webhooker_circuit_breakers_open`. - `webhooker_circuit_breakers_open`.
Two inbound HTTP metrics, `http_request_duration_seconds` and Three inbound HTTP metrics, named `http_...`, carry the route pattern as their
`http_response_size_bytes`, carry the route pattern as their `handler` label, `handler` label, never the real path.
never the real path. A third, `http_requests_inflight`, counts all routes
together.
## Development ## Development
You need the Go version named in `go.mod`, and Docker. The tests, the linters, You need the Go version named in `go.mod`, and Docker. The tests, the linters,
prettier and the stylesheet build run in Docker, at versions pinned (fixed to prettier and the stylesheet build run in Docker, at versions pinned (fixed to
one exact release) in the `Dockerfile`, `package.json` and `yarn.lock`. `gofmt` one exact release) in the `Dockerfile`, `package.json` and `yarn.lock`. Only
and `goimports` run on the host. `gofmt` and `goimports` run on the host.
```bash ```bash
make bootstrap # Install git, make, Go and the Go modules make bootstrap # Install git, make, Go and the Go modules
@@ -650,8 +633,8 @@ checkout, or `VERSION` when that is set. It is shown at
The pages use one third-party script, Alpine.js, in its CSP build The pages use one third-party script, Alpine.js, in its CSP build
`@alpinejs/csp`, which runs under a Content-Security-Policy that forbids `eval`. `@alpinejs/csp`, which runs under a Content-Security-Policy that forbids `eval`.
The npm registry is where JavaScript packages are published. The package file in The package file in `3p/` is byte for byte the one the npm registry, the
`3p/` is byte for byte the one it publishes. JavaScript package registry, publishes.
A browser test in `internal/server` loads the pages in a headless browser, one A browser test in `internal/server` loads the pages in a headless browser, one
without a window, under the real Content-Security-Policy. `make test` leaves it without a window, under the real Content-Security-Policy. `make test` leaves it
@@ -674,10 +657,8 @@ application are under [Entrypoint URLs](#entrypoint-urls).
- `script/test` runs the tests in Docker. - `script/test` runs the tests in Docker.
- `script/test-browser` runs the browser test in Docker. - `script/test-browser` runs the browser test in Docker.
- `script/lint` runs `gofmt`, `golangci-lint` and ESLint in Docker. - `script/lint` runs `gofmt`, `golangci-lint` and ESLint in Docker.
- `script/fmt` formats the Go code with `gofmt` and `goimports` on the host, and - `script/fmt` formats the Go code on the host and the Markdown in Docker.
the Markdown with prettier in Docker. - `script/fmt-check` checks the same and fails instead of writing.
- `script/fmt-check` fails when `gofmt` or prettier would change a file. It does
not run `goimports`, and it writes nothing.
- `script/css` regenerates `static/css/tailwind.css` in Docker. - `script/css` regenerates `static/css/tailwind.css` in Docker.
- `script/css-check` fails when `static/css/tailwind.css` is out of date. - `script/css-check` fails when `static/css/tailwind.css` is out of date.
- `script/check` runs `script/test`, `script/lint`, `script/fmt-check` and - `script/check` runs `script/test`, `script/lint`, `script/fmt-check` and