1 Commits
Author SHA1 Message Date
sneak 17bd4770dd Rewrite README.md sentence by sentence
check / check (push) Canceled after 0s
Rewritten from scratch against the code on next. The old text was
the list of what a reader needs covered.

Model: fable-5-1
2026-10-25 08:24:52 +00:00
+79 -60
View File
@@ -99,11 +99,12 @@ sooner after 24 hours without use, unless `SESSION_IDLE_TIMEOUT` says otherwise.
### Webhooks ### Webhooks
**New Webhook** on the webhook list at `/hooks` asks for a name, an optional **New Webhook** on the webhook list at `/hooks` asks for **Name**, an optional
description and a retention period in days. Events older than the retention **Description** and **Retention (days)**. Events older than the retention period
period are deleted, along with their deliveries. The default is 30 days. `0` are deleted, along with their deliveries. The default is 30 days. `0` keeps
keeps events forever. Every new webhook gets one entrypoint, described as events forever. The form also has an **HTTP target URL** field and an
`Default entrypoint`. **Archive** checkbox; each creates that target with the webhook. Every new
webhook gets one entrypoint, described as `Default entrypoint`.
### Entrypoint URLs ### Entrypoint URLs
@@ -121,14 +122,14 @@ deactivated entrypoint answers `410 Gone`. A deleted or unknown one answers
**Add** under **Targets** asks for the type first, then that type's settings. **Add** under **Targets** asks for the type first, then that type's settings.
- An `http` target posts each event to a URL, with optional request headers and - An `http` target posts each event to a URL, with optional **Headers** and a
a timeout. **Timeout**.
- A `slack` target posts each event as a message to the message-posting URL - A `slack` target posts each event as a message. Its **Webhook URL** is the
Slack or Mattermost give out, which they call an incoming webhook. message-posting URL Slack or Mattermost give out, which they call an incoming
webhook.
- An archive target writes each event as a row in a SQLite file of its own. Its - An archive target writes each event as a row in a SQLite file of its own. Its
stored type is `database`. It takes an expiry, which is how long archived stored type is `database`. **Archive expiry** is how long archived events are
events are kept, and a rotation, which is how often a new archive file is kept, and **Archive rotation** is how often a new archive file is started.
started.
- A `log` target writes each event to the application log. - A `log` target writes each event to the application log.
`http` and `slack` targets also take **Delivery attempts**: how many times each `http` and `slack` targets also take **Delivery attempts**: how many times each
@@ -187,8 +188,9 @@ your reverse proxy's Docker network instead, and mount one host directory at
webhooker speaks plain HTTP. It does not handle TLS, the encryption behind webhooker speaks plain HTTP. It does not handle TLS, the encryption behind
HTTPS: it holds no certificate and accepts no HTTPS connections. The reverse HTTPS: it holds no certificate and accepts no HTTPS connections. The reverse
proxy holds the certificate and reaches webhooker over loopback, the address proxy holds the certificate and reaches webhooker over a private address. On a
only this host can reach. bare host that is loopback, the address only this host can reach. For the image
it is the Docker bridge.
Four things must be right. Four things must be right.
@@ -200,16 +202,18 @@ Four things must be right.
from any connecting machine, so the proxy must overwrite whatever the client from any connecting machine, so the proxy must overwrite whatever the client
sent. sent.
3. **Cover the proxy in `TRUSTED_PROXIES`.** Otherwise every client shares one 3. **Cover the proxy in `TRUSTED_PROXIES`.** Otherwise every client shares one
rate-limit bucket, the count a limit keeps per client. The default list does rate-limit bucket, the count a limit keeps per client. The default list
not cover a proxy that connects over loopback. covers the Docker bridge address, such as `172.17.0.1`, that the image sees a
4. **Send `Host` as `$http_host`, not `$host`.** webhooker checks a form's proxy at. It does not cover loopback, so on a bare host set
`Origin` against the `Host` header it receives. `$host` drops the port, so on `TRUSTED_PROXIES=127.0.0.1`.
any port other than 443 every form submission fails with `403`. 4. **Pass on the client's `Host` header with its port.** webhooker checks a
form's `Origin` against the `Host` header it receives. A `Host` without the
port makes every form submission fail with `403` on any port other than 443.
In nginx that means `$http_host`, not `$host`.
Protect the proxy's access log: its request paths hold the entrypoint UUIDs. Protect the proxy's access log: its request paths hold the entrypoint UUIDs.
An nginx server block for this, with `TRUSTED_PROXIES=127.0.0.1` on webhooker's An nginx server block for a bare host:
side:
```nginx ```nginx
server { server {
@@ -248,12 +252,18 @@ the server does and refuses while a running instance holds it.
DATA_DIR=/var/lib/webhooker webhooker resetpw -generate admin DATA_DIR=/var/lib/webhooker webhooker resetpw -generate admin
``` ```
Without `-generate`, the new password is read from standard input, at least 8 Without `-generate`, the new password is read from standard input. It must be at
characters. It is never taken as a command-line argument, because other accounts least 8 characters. It is never taken as a command-line argument, because other
on the host can read those. In a container, run the same command with accounts on the host can read those. The command creates nothing: a missing
`docker run --rm` and the stopped container's volume mounted. The command `DATA_DIR`, a missing `webhooker.db` or an unknown username is an error.
creates nothing: a missing `DATA_DIR`, a missing `webhooker.db` or an unknown
username is an error. In a container, run the binary at `/app/webhooker` with the stopped container's
volume mounted:
```bash
docker run --rm -v /path/to/data:/var/lib/webhooker \
webhooker /app/webhooker resetpw -generate admin
```
## Backup and restore ## Backup and restore
@@ -263,15 +273,16 @@ All state lives in `DATA_DIR`. Back up the whole directory. It holds:
password hash, and the webhooks, entrypoints and targets; password hash, and the webhooks, entrypoints and targets;
- one event database per webhook, `events-{webhook_uuid}.db`: its events, - one event database per webhook, `events-{webhook_uuid}.db`: its events,
deliveries and attempts; deliveries and attempts;
- the archive files, one per archive target; - the files of each archive target;
- the lock file, `webhooker.lock`, which is always empty. - the lock file, `webhooker.lock`, which is always empty.
Every database runs in WAL mode. WAL is SQLite's write-ahead log: finished Every database runs in WAL mode. WAL is SQLite's write-ahead log: finished
writes land in a `-wal` file beside the database before they reach the `.db` writes land in a `-wal` file beside the database before they reach the `.db`
itself. A copy of the `.db` without its `-wal` is missing data. A clean shutdown itself, and a `-shm` file beside it indexes that log. A copy of the `.db`
removes the `-wal` and `-shm` files; a crash leaves them, and they must be without its `-wal` is missing data. A clean shutdown removes the `-wal` and
carried with the `.db`. Keep `DATA_DIR` on a local disk: WAL mode is not `-shm` files; a crash leaves them, and they must be carried with the `.db`. Keep
reliable on network filesystems, and webhooker does not check for one. `DATA_DIR` on a local disk: WAL mode is not reliable on network filesystems, and
webhooker does not check for one.
A copy taken while the service runs can catch a `.db` and its `-wal` at two A copy taken while the service runs can catch a `.db` and its `-wal` at two
different instants. Stop the service, copy the directory, and start it again. different instants. Stop the service, copy the directory, and start it again.
@@ -318,9 +329,9 @@ writes it. The reverse proxy's access log does hold it, in the request path.
- Session cookies are signed, `HttpOnly` (hidden from page scripts) and - Session cookies are signed, `HttpOnly` (hidden from page scripts) and
`SameSite=Lax` (not sent with cross-site form posts). They carry `Secure` when `SameSite=Lax` (not sent with cross-site form posts). They carry `Secure` when
the request carries `X-Forwarded-Proto: https`. the request carries `X-Forwarded-Proto: https`.
- Every form is protected against cross-site request forgery (CSRF), where - Every form is protected against cross-site request forgery (CSRF). CSRF is
another site makes your browser submit a form here: each form carries a token when another site makes your browser submit a form here. Each form carries a
tied to a cookie, and a submission's `Origin` must name this host. token tied to a cookie, and a submission's `Origin` must name this host.
- Form bodies are capped at 1 MB. - Form bodies are capped at 1 MB.
- Every response carries these headers: - Every response carries these headers:
- `Strict-Transport-Security`, unless `HSTS_MAX_AGE` is `0`; - `Strict-Transport-Security`, unless `HSTS_MAX_AGE` is `0`;
@@ -328,14 +339,16 @@ writes it. The reverse proxy's access log does hold it, in the request path.
- `X-Frame-Options: DENY`; - `X-Frame-Options: DENY`;
- a Content-Security-Policy (CSP), the header that tells the browser where - a Content-Security-Policy (CSP), the header that tells the browser where
scripts and styles may come from, set to scripts and styles may come from, set to
`default-src 'self'; script-src 'self'; style-src 'self'`. `default-src 'self'; script-src 'self'; style-src 'self'`;
- `Referrer-Policy: strict-origin-when-cross-origin`;
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`.
- `/metrics` sits behind HTTP basic authentication, a username and password sent - `/metrics` sits behind HTTP basic authentication, a username and password sent
with each request. with each request.
### Sign-in limits ### Sign-in limits
The password is checked first, and only a failed sign-in is counted. A correct The password is checked first, and only a failed sign-in is counted. A correct
password is never throttled. Failures are counted per client and submitted password is never answered `429`. Failures are counted per client and submitted
username. From the fifth failure within a minute, each failure is answered username. From the fifth failure within a minute, each failure is answered
`429`. At most two password checks run at once; a request that gets no turn `429`. At most two password checks run at once; a request that gets no turn
within 5 seconds is answered `503`. So those two checks, not the failure count, within 5 seconds is answered `503`. So those two checks, not the failure count,
@@ -396,9 +409,9 @@ again any delivery left `pending` or `retrying`, so a stop or a crash loses no
delivery. delivery.
The main database holds configuration. Each webhook has its own event database, The main database holds configuration. Each webhook has its own event database,
so writes for one webhook never wait on another webhook's database. The Go code so writes for one webhook never wait on another webhook's database. The program
is under `internal/`, one package per concern, and the pages' HTML, CSS and starts in `cmd/webhooker/main.go`, and its packages are under `internal/`. The
JavaScript are embedded in the binary. pages' HTML, CSS and JavaScript are embedded in the binary.
## Configuration reference ## Configuration reference
@@ -530,23 +543,25 @@ default but is reopened by a list that covers it.
## Delivery reference ## Delivery reference
An `http` target posts the event body to its URL with the event's `Content-Type` An `http` target posts the event body to its URL with the event's `Content-Type`
and headers, then the target's configured headers, which replace event headers and headers, then the target's configured **Headers**, which replace event
of the same name. A `2xx` response is a success. The first 4 KiB of the response headers of the same name. Every `http` and `slack` request carries
body is stored with the attempt. `timeout_seconds` is at most 300, and blank `User-Agent: webhooker/{version}`, replacing the sender's `User-Agent` and any
means the 30 second default. `forward_query` appends the event's query string to configured one. A `2xx` response is a success. The first 4 KiB of the response
the URL. body is stored with the attempt. **Timeout** is at most 300 seconds, and blank
means the 30 second default. **Pass the query string on to this target** appends
the event's query string to the URL.
A `slack` target posts a message that describes the event and shows its body in A `slack` target posts a message that describes the event and shows its body in
a code block, cut at 3500 bytes. An archive target, whose stored type is a code block, cut at 3500 bytes. An archive target, whose stored type is
`database`, appends the event as a row to its own SQLite file. A `log` target `database`, appends the event as a row to its own SQLite file. A `log` target
writes the event as one `INFO` line. writes the event as one `INFO` line.
`http` and `slack` targets take `max_attempts`, from 1 to 20. A failed attempt **Delivery attempts** is from 1 to 20. A failed attempt marks the delivery
marks the delivery `retrying` and schedules the next attempt; the wait doubles `retrying` and schedules the next attempt; the wait doubles each time, starting
each time, starting at one second. After the last allowed attempt fails, the at one second. After the last allowed attempt fails, the delivery is marked
delivery is marked `failed`. `failed`.
A target with `max_attempts` above 1 has a circuit breaker, which stops A target with more than one delivery attempt has a circuit breaker, which stops
hammering a destination that is down. After 5 failures in a row, deliveries to hammering a destination that is down. After 5 failures in a row, deliveries to
the target pause for 30 seconds. Then one delivery is sent as a probe; the the target pause for 30 seconds. Then one delivery is sent as a probe; the
others wait until a probe succeeds. The webhook page says when a target is others wait until a probe succeeds. The webhook page says when a target is
@@ -595,21 +610,23 @@ metric carries one label, `target_type`, which is `http`, `slack`, `database`,
- `webhooker_deliveries_succeeded_total` and - `webhooker_deliveries_succeeded_total` and
`webhooker_deliveries_failed_total`; `webhooker_deliveries_failed_total`;
- `webhooker_delivery_retries_total` and `webhooker_delivery_replays_total`; - `webhooker_delivery_retries_total` and `webhooker_delivery_replays_total`;
- `webhooker_delivery_duration_seconds`, a histogram (a distribution) of one - `webhooker_delivery_duration_seconds`, a histogram (a distribution) of how
attempt's wall time; long one attempt took;
- `webhooker_deliveries_pending` and `webhooker_deliveries_retrying`, gauges - `webhooker_deliveries_pending` and `webhooker_deliveries_retrying`, gauges
(current values) sampled every 30 seconds; (current values) sampled every 30 seconds;
- `webhooker_circuit_breakers_open`. - `webhooker_circuit_breakers_open`.
Three inbound HTTP metrics, named `http_...`, carry the route pattern as their Two inbound HTTP metrics, `http_request_duration_seconds` and
`handler` label, never the real path. `http_response_size_bytes`, carry the route pattern as their `handler` label,
never the real path. A third, `http_requests_inflight`, counts all routes
together.
## Development ## Development
You need the Go version named in `go.mod`, and Docker. The tests, the linters, You need the Go version named in `go.mod`, and Docker. The tests, the linters,
prettier and the stylesheet build run in Docker, at versions pinned (fixed to prettier and the stylesheet build run in Docker, at versions pinned (fixed to
one exact release) in the `Dockerfile`, `package.json` and `yarn.lock`. Only one exact release) in the `Dockerfile`, `package.json` and `yarn.lock`. `gofmt`
`gofmt` and `goimports` run on the host. and `goimports` run on the host.
```bash ```bash
make bootstrap # Install git, make, Go and the Go modules make bootstrap # Install git, make, Go and the Go modules
@@ -633,8 +650,8 @@ checkout, or `VERSION` when that is set. It is shown at
The pages use one third-party script, Alpine.js, in its CSP build The pages use one third-party script, Alpine.js, in its CSP build
`@alpinejs/csp`, which runs under a Content-Security-Policy that forbids `eval`. `@alpinejs/csp`, which runs under a Content-Security-Policy that forbids `eval`.
The package file in `3p/` is byte for byte the one the npm registry, the The npm registry is where JavaScript packages are published. The package file in
JavaScript package registry, publishes. `3p/` is byte for byte the one it publishes.
A browser test in `internal/server` loads the pages in a headless browser, one A browser test in `internal/server` loads the pages in a headless browser, one
without a window, under the real Content-Security-Policy. `make test` leaves it without a window, under the real Content-Security-Policy. `make test` leaves it
@@ -657,8 +674,10 @@ application are under [Entrypoint URLs](#entrypoint-urls).
- `script/test` runs the tests in Docker. - `script/test` runs the tests in Docker.
- `script/test-browser` runs the browser test in Docker. - `script/test-browser` runs the browser test in Docker.
- `script/lint` runs `gofmt`, `golangci-lint` and ESLint in Docker. - `script/lint` runs `gofmt`, `golangci-lint` and ESLint in Docker.
- `script/fmt` formats the Go code on the host and the Markdown in Docker. - `script/fmt` formats the Go code with `gofmt` and `goimports` on the host, and
- `script/fmt-check` checks the same and fails instead of writing. the Markdown with prettier in Docker.
- `script/fmt-check` fails when `gofmt` or prettier would change a file. It does
not run `goimports`, and it writes nothing.
- `script/css` regenerates `static/css/tailwind.css` in Docker. - `script/css` regenerates `static/css/tailwind.css` in Docker.
- `script/css-check` fails when `static/css/tailwind.css` is out of date. - `script/css-check` fails when `static/css/tailwind.css` is out of date.
- `script/check` runs `script/test`, `script/lint`, `script/fmt-check` and - `script/check` runs `script/test`, `script/lint`, `script/fmt-check` and