diff --git a/README.md b/README.md index 25298d5..b0dd81e 100644 --- a/README.md +++ b/README.md @@ -4,89 +4,71 @@ [@sneak](https://sneak.berlin), for self-hosted web apps. It refuses the requests of scrapers, scanners and other abusers before they reach an app. +## What it does + A web application firewall reads each request on its way to a web app and -refuses the ones that look like abuse. `smallwebwaf` is one binary with two -commands that do this together: +refuses the ones that look like abuse. `smallwebwaf` is one binary, shipped in +one image, with two commands that do this together: -- `smallwebwaf rproxy` is a small reverse proxy in front of one app. It accepts - requests on the app's behalf and asks `servewaf` about each one. It passes the - allowed ones to the app, and the app's answers back. It keeps nothing, and - never touches disk. -- `smallwebwaf servewaf` decides. It runs the checks, and keeps the bans, - counters and history. One `servewaf` serves many sites. +- `smallwebwaf rproxy` is a small reverse proxy in front of one app. traefik + routes to it. It asks `servewaf` about each request, passes the allowed ones + to the app, and passes the app's answers back. It keeps nothing and never + touches disk. +- `smallwebwaf servewaf` decides. It runs the checks and keeps the bans, the + counters and each client's history. One `servewaf` serves many sites: a client + banned through one site is refused on all of them, and its requests on all of + them count toward one set of limits. -The site keeps answering while `servewaf` is down, slow or restarting. `rproxy` -waits at most 2 seconds for `servewaf`'s answer, and without one passes the -request to the app. +The site keeps answering while `servewaf` is down, slow or restarting: without +an answer in time, `rproxy` passes the request to the app. -`rproxy` requires one setting, `SWRP_WAF_BACKEND`, which says where `servewaf` -is. In a container of its own, it also needs `SWRP_UPSTREAM_URL`, which says -where the app is. Every other setting has a default chosen for a service on the -open internet. [`SPEC.md`](SPEC.md) says how `smallwebwaf` behaves and why. The -reference documents under [`docs/`](docs/) are listed under "Documents" below. +It protects against: -## What it protects against - -- Scrapers that fetch every page of a site, all day. Rate limiting gives each - client 500 requests a minute, 20,000 an hour and 300,000 a day. That leaves - room for programmatic use of an API, such as an automation fleet on another - host. A fleet of about 20 agents makes about 120 requests a minute at its - busiest. A gitea page opened with an empty cache loads about 40 files, so a - few such first visits fit in the minute limit too. The minute limit catches, - within seconds, a client firing requests as fast as it can. The hour limit - catches a client that keeps up more than about 5.5 a second for an hour. The - day limit catches a client that keeps up 4 a second around the clock. Counting - cannot catch a client that stays under all three limits, such as one that - keeps up 3 a second; only the protections below can. Byte limits give each - client 4 GiB a minute, 20 GiB an hour and 200 GiB a day. A client over a limit - is refused and banned. The first ban lasts an hour. Each repeat within a day - makes the next ban three times as long. A ban that would last more than seven - days is permanent. +- Scrapers that fetch every page of a site, all day. Each client gets a limit on + requests and on bytes per minute, per hour and per day. The defaults leave + room for programmatic use of an API, such as an automation fleet, and catch a + client firing requests as fast as it can within seconds. A client over a limit + is refused and banned: for a while at first, for three times as long at each + repeat, and permanently when it keeps at it. - Scanners that probe for secrets, such as `/.env` or `/.git/config`. The - default rule file bans such a client for seven days at its first probe. Its - next request during the ban makes the ban permanent. The default rule file - does not cover probes for other apps, such as a WordPress login. A `ban` rule - of your own, or an entry in `SWWAF_TRAP_PATHS`, bans those the same way. + default rule file bans such a client at its first probe, and its next request + during the ban makes the ban permanent. Your own rules, or a list of trap + paths, ban probes for other apps, such as a WordPress login, the same way. - Attacks carried in a request's URL or headers, such as SQL injection and path - traversal. The OWASP Core Rule Set is a published set of rules that recognise - common web attacks. Coraza, a Go engine, runs those rules (see "Attack - detection" in [`SPEC.md`](SPEC.md)). A request the rules take for an attack is - refused. -- Abuse from networks and countries you do not want. You can: - - refuse whole netblocks and countries; - - give the clients of chosen networks and countries a fraction of every - limit, so that they are banned sooner; - - refuse or limit clients on a blocklist or a DNS blocklist, or with a high - AbuseIPDB score; - - ban clients named by your own CrowdSec engine. + traversal. The OWASP Core Rule Set, a published set of rules that recognise + common web attacks, runs on every request through Coraza, a Go engine. A + request the rules take for an attack is refused. +- Abuse from networks and countries you do not want. You can refuse whole + netblocks and countries; give the clients of chosen networks and countries a + fraction of every limit, so that they are banned sooner; refuse or limit + clients on a blocklist, a DNS blocklist, or with a high AbuseIPDB score; and + ban clients named by your own CrowdSec engine. - Slow and oversized requests. Every request and every answer has a time limit and a size limit. For the operator, it: -- logs each request as JSON lines, saying what it did and why (see - [`docs/request-log.md`](docs/request-log.md)); -- sends alerts to a webhook, Slack or ntfy (see - [`docs/alerts.md`](docs/alerts.md)); -- serves Prometheus metrics (see [`docs/metrics.md`](docs/metrics.md)); +- logs each request as JSON lines, saying what it did and why; +- sends alerts to a webhook, Slack or ntfy; +- serves Prometheus metrics; - keeps its bans, counters and history in JSON files you can read and edit, so - that a restart loses nothing and a ban can be lifted with an editor (see - [`docs/state-files.md`](docs/state-files.md)). + that a restart loses nothing and a ban can be lifted with an editor. Not built, and not planned: - TLS and routing by host name. Both stay with traefik, the reverse proxy in front of `rproxy`. - Browser challenges such as captchas. -- A web console. +- A web console, or a configuration file. - Defence against floods large enough to fill the host's network link. ## Getting started -Build the image from a clone. `make docker` runs the tests and the linter, then -builds the image, tagged `smallwebwaf`. It builds for amd64, on an amd64 host -only: the `Dockerfile` checks Ubuntu's package lists against the hashes of the -amd64 archive. +### The image + +No image is published. Build it from a clone. `make docker` runs the tests and +the linter, then builds the image, tagged `smallwebwaf`. It builds for amd64, on +an amd64 host only. ```sh git clone https://git.eeqj.de/sneak/smallwebwaf.git @@ -94,62 +76,38 @@ cd smallwebwaf make docker ``` -Push the image to a registry your hosts pull from. +Push the image to a registry your hosts pull from, and name it there by digest. The image runs `rproxy` by default. To run `servewaf` from the same image, set -the container's `SMALLWEBWAF_COMMAND` to `servewaf`. +the container's `SMALLWEBWAF_COMMAND` to `servewaf`. The image's health check +follows the command: in `rproxy`'s container it passes once `rproxy` answers and +the app accepts connections, whether or not `servewaf` answers; in `servewaf`'s +container, once `servewaf` answers. ### The usual setup: three containers The usual setup is three containers on one private network: - the app; -- `rproxy`, in front of the app, which traefik routes to; -- `servewaf`, with a volume for its state files. +- `rproxy`, in front of the app, which traefik routes to: `SWRP_WAF_BACKEND` + names `servewaf` and `SWRP_UPSTREAM_URL` names the app; +- `servewaf`, with `SMALLWEBWAF_COMMAND=servewaf` and a volume at + `/var/lib/smallwebwaf` for its state files. -```sh -docker network create site - -docker run --detach --name servewaf --network site \ - --env SMALLWEBWAF_COMMAND=servewaf \ - --volume servewaf-state:/var/lib/smallwebwaf \ - /smallwebwaf@sha256: - -docker run --detach --name app --network site \ - /app@sha256: - -docker run --detach --name rproxy --network site \ - --env SWRP_WAF_BACKEND=http://servewaf:8080 \ - --env SWRP_UPSTREAM_URL=http://app:8081 \ - /smallwebwaf@sha256: -``` - -Here the app listens on port 8081, at its container's address on that network. Point traefik's labels at port 8080 of `rproxy` (`traefik.http.services..loadbalancer.server.port=8080`). Publish no port of the app or of `servewaf`. Anything on that network can reach the app around -`rproxy`, and can ask `servewaf` questions. - -With no setting, `servewaf` sends the address of every new visitor to GeoJS, a -web service, to find its country and network. `SWWAF_LOOKUP_SOURCE=off` sends -nothing. `SWWAF_LOOKUP_SOURCE=file`, with `SWWAF_LOOKUP_DB_PATH` naming a -database file on your own host, looks addresses up in that file (see "Lookup of -AS number and country" in [`SPEC.md`](SPEC.md)). - -### Many sites, one `servewaf` - -Run one `rproxy` for each app, all with `SWRP_WAF_BACKEND` naming the same -`servewaf`. It keeps one table of clients and bans for all of them: - -- A client banned through one site is refused on all of them. -- A client's requests on all of them count toward one set of limits. -- Log lines, alerts and ban notes name the site by its host. +`rproxy`, and can ask `servewaf` questions, so the network holds only what you +trust. Each further app adds its own container and its own `rproxy`, whose +`SWRP_WAF_BACKEND` names the same `servewaf`. The "Smallest setup" and "In front +of gitea" examples below show this shape. ### `rproxy` in the app's container -An app's image can also be built on the `smallwebwaf` image, so that `rproxy` -runs beside the app. `servewaf` still runs in a container of its own. The app's -`Dockerfile` starts `FROM` the `smallwebwaf` image, named by digest, and adds: +An app's image can instead be built on the `smallwebwaf` image, so that `rproxy` +runs beside the app in one container. `servewaf` still runs in a container of +its own. The app's `Dockerfile` starts `FROM` the `smallwebwaf` image, named by +digest, and adds: - the app's binary; - the packages the app needs, from the nixpkgs in the image; @@ -191,110 +149,21 @@ main "$@" Leave `ENTRYPOINT` and `USER` out of the app's `Dockerfile`. The image's own entrypoint starts `rproxy` and the app side by side, each as its own user. Run -the app's image with `SWRP_WAF_BACKEND` set. +the app's image with `SWRP_WAF_BACKEND` set. `deploy/example-app` is such an +app, and the "`rproxy` beside the app" example below runs one. -`deploy/example-app` is such an app. `make example-app` builds it and checks it -(see "Checking that it works" below). To run `rproxy` and `servewaf` on the host -while working on them, see `script/run` under "Entrypoints" below. +The app sees the headers traefik sent, with traefik's address added to +`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust +`rproxy`'s address for forwarded headers. In a container of its own, `rproxy` is +on the private network, which the usual default of a trusted-proxy setting +covers. Beside the app, its address is `127.0.0.1` or `::1`, which the private +ranges leave out, so the app's own setting must name them too, as `app.run` +does. -### When `servewaf` is down, slow or overloaded - -`SWRP_DEFAULT_DISPOSITION` decides what `rproxy` does with a request when -`servewaf`: - -- cannot be connected to; -- does not answer at all; -- does not answer within `SWRP_TIMEOUT_MS`, 2 seconds by default. - -Its default, `allow`, passes the request to the app without `servewaf`'s answer. -`deny` answers `503` instead. Either way the request gets no report, so its -bytes are not counted and it is not added to its client's history. When -`servewaf` does answer, a `deny` or an error goes to the visitor as `servewaf` -gave it, and `SWRP_DEFAULT_DISPOSITION` plays no part. - -## What happens to a request - -A request goes through fifteen steps, in this order. Once `servewaf` refuses a -request, the steps after that are skipped, apart from the report of step 15, and -nothing of the request reaches the app. "What happens to a request" in -[`SPEC.md`](SPEC.md) gives each step in full, under the same numbers. - -`rproxy` takes the first two: - -1. The health check, `GET /_smallwebwaf/healthz`, is answered at once. -2. The visitor's address is worked out, and `servewaf` is asked about the - request. - -`servewaf` takes the checks: - -3. The static lists, `SWWAF_ALLOW_NETS` and `SWWAF_DENY_NETS`. -4. The bans. -5. The lookup of the client's AS number and country. -6. The country lists. -7. The reputation sources. -8. The client's limit percentage. -9. The rate limits. -10. The trap paths. -11. The rule files. -12. The header limits, and then the Core Rule Set. -13. A path under `/_smallwebwaf/` is answered `404`, so that the app never sees - that prefix. - -`rproxy` takes the rest, with `servewaf` counting at the end: - -14. The request is passed to the app. -15. After the answer, `rproxy` sends `servewaf` a report. For an allowed - request, `servewaf` counts the byte limits and adds the request to the - client's history. - -A refused visitor is answered with `SWWAF_BAN_RESPONSE`, `403` by default (see -"Ban lengths, response and scope" in [`SPEC.md`](SPEC.md)). A `block` rule or a -Core Rule Set match answers `403`. `rproxy` itself answers `405` to a method it -does not pass on, and the status for a size or time limit, such as `413` for a -body too large (see "`rproxy`" in [`SPEC.md`](SPEC.md)). - -## Settings you are likely to touch - -Each setting is an environment variable, read at start. Those of `servewaf` -start with `SWWAF_`, and those of `rproxy` with `SWRP_`. -[`docs/settings.md`](docs/settings.md) lists every setting, its default and the -values it takes. The ones most deployments touch: - -- `SWWAF_MODE=observe`: `servewaf` refuses nothing and makes no ban of its own. - A request it would have refused is passed to the app, and its log line says - what `enforce` mode would have done. For trying a configuration before - enforcing it. The limits of `rproxy` still apply. -- `SWWAF_ALLOW_NETS`: netblocks you trust, such as your monitoring. Their - clients skip `servewaf`'s checks. -- `SWWAF_RATE_LIMIT_EXEMPT_NETS`: netblocks exempt from rate limiting and the - byte limits, such as a machine that talks to the app all day. -- `SWWAF_TRAP_PATHS`: paths only scanners ask for, such as - `/wp-login.php,/xmlrpc.php` in front of a code forge. A request for one bans - the client. -- `SWWAF_WAF_DISABLED_RULES`: Core Rule Set rule ids to switch off when a rule - refuses your app's ordinary requests. The log line names the rules a request - matched. Keep the six default ids in a list you give (see - [`docs/settings.md`](docs/settings.md)). -- `SWRP_MAX_REQUEST_SIZE` and `SWRP_MAX_REQUEST_TIME_MS`: raise them for an app - that takes uploads over 100 MiB or slower than 60 seconds. -- `SWRP_DEFAULT_DISPOSITION`: `allow` (the default) or `deny`, as above. -- `SWRP_TRUSTED_PROXIES`: the netblocks whose `X-Forwarded-For` `rproxy` - believes, by default the private address ranges. -- `SWWAF_LOOKUP_SOURCE`: `geojs` (the default), `file` or `off`, as above. -- `SWWAF_ADMIN_TOKEN` and `SWWAF_METRICS_TOKEN`: switch on the ban endpoints and - the metrics, at `servewaf`'s own address. -- `SWWAF_ALERT_WEBHOOK_URL`, `SWWAF_ALERT_SLACK_WEBHOOK_URL` or - `SWWAF_ALERT_NTFY_URL`: where alerts go. -- `SWWAF_INSTANCE_NAME`: the name in every line `servewaf` logs, and in every - alert and metric, such as `fsn1/servewaf`. - -## Checking that it works +### Checking that it works - `docker inspect --format '{{.State.Health.Status}}' ` says - `healthy` for each container of `smallwebwaf`: - - for `rproxy`, once it answers and the app accepts connections, whether or - not `servewaf` answers; - - for `servewaf`, once it answers. + `healthy` for each container of `smallwebwaf`. - A log line with `"msg":"starting"` gives the version, the address and every setting in effect, with secrets masked. - A request to the site through traefik gets the app's answer. @@ -304,23 +173,522 @@ values it takes. The ones most deployments touch: `SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app. `servewaf`'s log lines have `"would_action":"banned"` and `"rule_ids":["env-file"]`. In `enforce` mode the same request bans your - address for seven days, and your next request makes the ban permanent. Lift - the ban by editing `bans.json` (see "Bans" in [`SPEC.md`](SPEC.md)). + address, and your next request makes the ban permanent. Lift the ban as + "Clients and bans" below describes. - With `SWWAF_METRICS_TOKEN` set, this answers with the metrics, from a host on `servewaf`'s network: `curl -H "Authorization: Bearer $TOKEN" http://servewaf:8080/_smallwebwaf/metrics`. -- `make example-app` builds the image and the example app on it, runs the app - with `rproxy` in its container and `servewaf` in another, and checks them. It - needs network access. It checks: - - the health checks; - - a request passed to the app; - - a ban for a broken limit, and a ban for a probe made permanent; - - that only `servewaf`'s container changes the state directory's owner; - - with `servewaf` stopped, the app still answering through `rproxy`; - - with `servewaf` started again, bans applying again; - - orderly stops with `sv stop` and `docker stop`; - - bans surviving a new container on the same volume; - - a setting given as a file. +- `make example-app` builds the image and an app on it, runs them, and checks + the whole exchange (see "Entrypoints"). + +## Configuration + +Every setting is an environment variable, read once at start; changing one means +restarting the container. `rproxy` reads only the `SWRP_` settings, and +`servewaf` only the `SWWAF_` ones. Every setting but `SWRP_WAF_BACKEND` has a +default, chosen for a service on the open internet. A setting that needs an +account, a key, a token or an alert destination stays off until you give one. + +- Any `SWWAF_` setting can be given as a file instead, for secrets and long + lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. + The file's contents are the value, less one newline at the end. Setting both + forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already names a file, so + it has no `_FILE` form, and no `SWRP_` setting has one, since `rproxy` never + touches disk. +- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. +- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of + 1024: `512K`, `100M`, `5G`. +- A count is a whole number of requests, such as `1000`. +- A time in milliseconds is a whole number above zero, such as `2000`. +- A list is comma separated. A list set to an empty value is an empty list, and + replaces the default. +- A netblock is an address and a prefix length, such as `10.0.0.0/8`; a bare + address stands for itself alone. A country is a two-letter code in either + case, such as `de`. An AS number is `AS` and the number, such as `AS64496`. A + percentage is a whole number from 0 to 100. A path prefix starts with `/`. +- `off` switches a limit, a threshold or a timeout off where the table says a + setting takes it. +- An invalid setting stops the process at start with one log line naming the + variable and what is wrong with the value. The settings in effect are logged + on one line at start, with tokens, keys and webhook paths shown as `********`. + +Every setting the code reads, with its default: + +| Variable | Default | What it does | +| ------------------------------------------ | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `SMALLWEBWAF_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `servewaf`. | +| `SWRP_WAF_BACKEND` | required | `servewaf`'s URL, such as `http://servewaf:8080`. | +| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. | +| `SWRP_LISTEN_ADDR` | `:8080` | Where `rproxy` listens; leave the host part empty. | +| `SWRP_TRUSTED_PROXIES` | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none. | +| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `servewaf`'s answer to a question and to a report, in milliseconds. | +| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `servewaf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. | +| `SWRP_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. | +| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`; a larger one is answered `502`, or cut off once it has started. | +| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | How long the visitor may take to send its request line and headers, and then the whole request to reach the app, in milliseconds or `off`. | +| `SWRP_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. | +| `SWWAF_LISTEN_ADDR` | `:8080` | Where `servewaf` listens; leave the host part empty. | +| `SWWAF_INSTANCE_NAME` | the container's host name | The name in every line `servewaf` logs, and in every alert and metric, such as `fsn1/servewaf`. | +| `SWWAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. | +| `SWWAF_IPV6_GROUP_PREFIX` | `64` | The prefix length of the IPv6 netblock that counts as one client, from 32 to 128. | +| `SWWAF_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. | +| `SWWAF_MAX_BANS` | `5000` | The most bans `servewaf` keeps of its own; bans an admin made do not count. | +| `SWWAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. | +| `SWWAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. | +| `SWWAF_DENY_NETS` | empty | Netblocks whose clients are refused. | +| `SWWAF_RATE_LIMIT_PER_MINUTE` | `500` | The most requests a client may make in a minute, a count or `off`; one more bans it. | +| `SWWAF_RATE_LIMIT_PER_HOUR` | `20000` | The most requests a client may make in an hour, a count or `off`; one more bans it. | +| `SWWAF_RATE_LIMIT_PER_DAY` | `300000` | The most requests a client may make in a day, a count or `off`; one more bans it. | +| `SWWAF_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes whose requests the request and byte limits neither count nor refuse, such as `/assets/`. | +| `SWWAF_BYTES_LIMIT_PER_MINUTE` | `4G` | The most bytes a client's requests may carry in a minute, a size or `off`; the request that takes a client past any byte limit bans it. | +| `SWWAF_BYTES_LIMIT_PER_HOUR` | `20G` | The most bytes a client's requests may carry in an hour, a size or `off`. | +| `SWWAF_BYTES_LIMIT_PER_DAY` | `200G` | The most bytes a client's requests may carry in a day, a size or `off`. | +| `SWWAF_BYTES_COUNT` | `both` | Which body bytes the byte limits count: `response`, `request` or `both`. | +| `SWWAF_ERROR_BURST_THRESHOLD` | `30` | The most refusals by a rule file, a trap path or the Core Rule Set a client may get in a minute, a count or `off`; one more bans it. | +| `SWWAF_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. | +| `SWWAF_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. | +| `SWWAF_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A limit broken again within this time after the last such ban ended gets a ban three times as long. | +| `SWWAF_MAX_BAN_DURATION` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. | +| `SWWAF_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. | +| `SWWAF_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. | +| `SWWAF_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs`, a web service, `file`, or `off`. | +| `SWWAF_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. | +| `SWWAF_LOOKUP_TIMEOUT` | `800ms` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take. | +| `SWWAF_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. | +| `SWWAF_DENIED_COUNTRIES` | empty | Countries whose clients are refused, such as `cn,ru`. | +| `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. | +| `SWWAF_ASN_LIMIT_PERCENT` | empty | AS numbers whose clients get a percentage of every request and byte limit, such as `AS14061:50,AS16276:25`. | +| `SWWAF_COUNTRY_LIMIT_PERCENT` | empty | The same by country, such as `cn:25`. | +| `SWWAF_ASN_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the AS numbers listed, in place of their `SWWAF_ASN_LIMIT_PERCENT`. | +| `SWWAF_COUNTRY_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the countries listed, in place of their `SWWAF_COUNTRY_LIMIT_PERCENT`. | +| `SWWAF_UNKNOWN_LIMIT_PERCENT` | `100` | The percentage of every limit a client without a country gets. | +| `SWWAF_ASN_LIMIT_PERCENT_URL` | unset | The URL of a text file of `AS:` lines, fetched like a blocklist; where it and `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. | +| `SWWAF_RULES_DIR` | `/etc/smallwebwaf/rules.d` | The directory of the rule files, read at start and again when a file there changes. | +| `SWWAF_RULES_ENABLED` | `true` | `false` reads no rule file. | +| `SWWAF_TRAP_PATHS` | empty | Paths only scanners ask for, such as `/wp-login.php,/xmlrpc.php`; a request for one bans the client. | +| `SWWAF_WAF_MODE` | `block` | What the Core Rule Set does with a match: `block` refuses it with `403`, `detect` only logs and alerts it, `off` runs no rule. | +| `SWWAF_WAF_PARANOIA_LEVEL` | `1` | The Core Rule Set's paranoia level, from 1 to 4; each level up finds more attacks and takes more ordinary requests for attacks. | +| `SWWAF_WAF_ANOMALY_THRESHOLD` | `5` | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5. | +| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. | +| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. | +| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. | +| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. | +| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. | +| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:` of every limit, or `log`. | +| `SWWAF_DNSBL_ZONES` | empty | DNS blocklists asked about each new client in the background, such as `dnsbl.dronebl.org`. | +| `SWWAF_DNSBL_RESOLVER` | unset | The resolver the zones are asked through, an IP address with an optional port; unset, the host's resolver is used. | +| `SWWAF_ABUSEIPDB_KEY` | unset | The key of your AbuseIPDB account; set, a client that has committed an offence is checked in the background. | +| `SWWAF_ABUSEIPDB_MIN_SCORE` | `75` | The least AbuseIPDB abuse confidence score, from 0 to 100, that is a hit. | +| `SWWAF_ABUSEIPDB_DAILY_BUDGET` | `900` | The most AbuseIPDB checks made in a day. | +| `SWWAF_CROWDSEC_LAPI_URL` | unset | The local API of a CrowdSec engine you run, such as `http://172.17.0.1:8080`; its decision list is fetched every minute, and a listed client is banned. | +| `SWWAF_CROWDSEC_LAPI_KEY` | unset | The key the engine gave for `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it; set with the URL, never alone. | +| `SWWAF_REPUTATION_ACTION` | `limit:25` | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:` or `log`. | +| `SWWAF_REPUTATION_CACHE_TTL` | `24h` | How long a DNS blocklist verdict or an AbuseIPDB score is used. | +| `SWWAF_REPUTATION_TIMEOUT` | `2s` | How long a DNS blocklist query or an AbuseIPDB check may take. | +| `SWWAF_STATE_DIR` | `/var/lib/smallwebwaf` | The directory of the state files, an absolute path. | +| `SWWAF_STATE_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. | +| `SWWAF_STATE_COUNTER_INTERVAL` | `15m` | How often every state file is written. | +| `SWWAF_LOG_LEVEL` | `info` | The least severe of `servewaf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. | +| `SWWAF_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. | +| `SWWAF_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `servewaf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. | +| `SWWAF_LOG_REMOTE_TLS_CA_FILE` | unset | A file of PEM certificates a `syslog+tls` server's certificate must chain to, instead of the host's own. | +| `SWWAF_LOG_REMOTE_BUFFER` | `10000` | The most lines held while they wait to be sent; past it, the oldest is dropped. | +| `SWWAF_LOG_REMOTE_FACILITY` | `local0` | The syslog facility of each record sent. | +| `SWWAF_LOG_REMOTE_APP_NAME` | `SWWAF_INSTANCE_NAME` | The app name in each syslog record, 1 to 48 printable ASCII characters without a space. | +| `SWWAF_ADMIN_TOKEN` | unset | The bearer token for the ban and client endpoints, at least 32 characters; unset, they answer `404`. | +| `SWWAF_METRICS_TOKEN` | unset | The bearer token for `/_smallwebwaf/metrics`, at least 32 characters; unset, the metrics are off. | +| `SWWAF_METRICS_TOP_N` | `50` | How many AS numbers and how many countries get metric series of their own; the rest are summed as `other`. | +| `SWWAF_ALERT_WEBHOOK_URL` | unset | A webhook each alert is posted to as a JSON object. | +| `SWWAF_ALERT_WEBHOOK_HEADERS` | empty | Headers sent with each post, a list of `Name:value`, such as `Authorization:Bearer `. | +| `SWWAF_ALERT_SLACK_WEBHOOK_URL` | unset | A Slack incoming webhook each alert is posted to as a message. | +| `SWWAF_ALERT_NTFY_URL` | unset | An ntfy topic each alert is published to, as its full URL, such as `https://ntfy.sh/my-alerts`. | +| `SWWAF_ALERT_NTFY_TOKEN` | unset | An ntfy access token sent with each alert. | +| `SWWAF_ALERT_EVENTS` | `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error` | The events alerts are sent for; the hourly `summary` is sent whatever it says. | +| `SWWAF_ALERT_COOLDOWN` | `15m` | How long after an alert its repeats are held back and counted, a duration or `off`. | +| `SWWAF_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. | +| `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. | +| `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. | +| `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when one client's requests carry more bytes than this in a minute, a size. | +| `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when one client's requests carry more bytes than this in an hour, a size. | +| `SWWAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. | +| `SWWAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. | +| `SWWAF_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when the netblock around a client carries more bytes than this in a minute, a size. | +| `SWWAF_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when the netblock around a client carries more bytes than this in an hour, a size. | +| `SWWAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWWAF_ANOMALY_NET_` thresholds count, from 0 to 32. | +| `SWWAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWWAF_ANOMALY_NET_` thresholds count, from 0 to 128. | +| `SWWAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. | +| `SWWAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. | +| `SWWAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when one AS number carries more bytes than this in a minute, a size. | +| `SWWAF_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when one AS number carries more bytes than this in an hour, a size. | +| `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. | +| `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. | +| `SWWAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when the whole service carries more bytes than this in a minute, a size. | +| `SWWAF_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when the whole service carries more bytes than this in an hour, a size. | +| `SWWAF_WATCH_NETS` | empty | Named netblocks the `SWWAF_WATCH_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. | +| `SWWAF_WATCH_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. | +| `SWWAF_WATCH_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. | +| `SWWAF_WATCH_BYTES_PER_MINUTE` | `off` | Alert when a watched netblock carries more bytes than this in a minute, a size. | +| `SWWAF_WATCH_BYTES_PER_HOUR` | `off` | Alert when a watched netblock carries more bytes than this in an hour, a size. | + +[`docs/settings.md`](docs/settings.md) gives the values each setting takes in +full, and the limits that are fixed rather than settings. + +## docker-compose examples + +Each example is a complete `compose.yaml`. `smallwebwaf` is the tag +`make docker` gives the image; on another host, use the name you pushed it +under, by digest. Docker Compose puts the services on a private network of its +own, where each reaches the others by service name. The first two examples +publish `rproxy`'s port on the host, for trying it out; the others leave that to +traefik. + +### Smallest setup + +One app, `rproxy` in front of it, and `servewaf` with a volume for its state +files, every setting at its default. + +```yaml +services: + app: + # Your app, which listens on port 8081. + image: registry.example/app@sha256:0000000000000000000000000000000000000000000000000000000000000000 + + rproxy: + image: smallwebwaf + environment: + SWRP_WAF_BACKEND: http://servewaf:8080 + SWRP_UPSTREAM_URL: http://app:8081 + ports: + - "8080:8080" + + servewaf: + image: smallwebwaf + environment: + SMALLWEBWAF_COMMAND: servewaf + volumes: + - servewaf-state:/var/lib/smallwebwaf + +volumes: + servewaf-state: +``` + +### Custom per-address limits + +A site with no API: tighter limits for everyone, no limits for the office and +the automation host, no checks for monitoring, a refused netblock, a smaller +share of the limits for two hosting networks and one country, trap paths, and +`429` for a refused client. + +```yaml +services: + app: + image: registry.example/app@sha256:0000000000000000000000000000000000000000000000000000000000000000 + + rproxy: + image: smallwebwaf + environment: + SWRP_WAF_BACKEND: http://servewaf:8080 + SWRP_UPSTREAM_URL: http://app:8081 + ports: + - "8080:8080" + + servewaf: + image: smallwebwaf + environment: + SMALLWEBWAF_COMMAND: servewaf + # Monitoring skips every check. + SWWAF_ALLOW_NETS: 192.0.2.10 + # The office and the automation host are never rate limited. + SWWAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7 + # A netblock that sends nothing wanted. + SWWAF_DENY_NETS: 198.51.100.128/25 + # Tighter limits for everyone else: a site with no API. + SWWAF_RATE_LIMIT_PER_MINUTE: "120" + SWWAF_RATE_LIMIT_PER_HOUR: "3000" + SWWAF_RATE_LIMIT_PER_DAY: "30000" + # Hosting networks and one country get a share of those limits. + SWWAF_ASN_LIMIT_PERCENT: AS14061:50,AS16276:25 + SWWAF_COUNTRY_LIMIT_PERCENT: cn:25 + # Paths only scanners ask for; one request bans the client. + SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php + # Answer a refused client with 429 rather than 403. + SWWAF_BAN_RESPONSE: "429" + SWWAF_LIMIT_BAN_DURATION: 30m + volumes: + - servewaf-state:/var/lib/smallwebwaf + +volumes: + servewaf-state: +``` + +### In front of gitea + +gitea behind traefik, with the upload and byte limits raised for a code forge, +the Actions runners exempt from counting, and the default request limits, which +already leave room for API clients, kept. `traefik` is the network traefik is +on, made outside this file. + +```yaml +services: + gitea: + # gitea 1.27.3, 2026-08-29 + image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef + environment: + GITEA__server__ROOT_URL: https://git.example/ + # gitea's TCP peer is rproxy, on the private network. + GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 + volumes: + - gitea-data:/data + networks: + - site + ports: + # SSH does not pass through smallwebwaf. + - "2222:22" + + rproxy: + image: smallwebwaf + environment: + SWRP_WAF_BACKEND: http://servewaf:8080 + SWRP_UPSTREAM_URL: http://gitea:3000 + # Pushes, LFS objects and image layers over 100 MiB, or slower + # than a minute, are cut off at the defaults. + SWRP_MAX_REQUEST_SIZE: 2G + SWRP_MAX_REQUEST_TIME_MS: "600000" + networks: + - site + - traefik + labels: + traefik.enable: "true" + traefik.http.routers.git.rule: Host(`git.example`) + traefik.http.routers.git.entrypoints: websecure + traefik.http.routers.git.tls.certresolver: letsencrypt + traefik.http.services.git.loadbalancer.server.port: "8080" + + servewaf: + image: smallwebwaf + environment: + SMALLWEBWAF_COMMAND: servewaf + SWWAF_INSTANCE_NAME: fsn1/servewaf + # The Actions runners ask for work all day; never count them. + SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24 + # A clone larger than 1 GiB needs the response limit and the + # byte limits raised together. + SWWAF_BYTES_LIMIT_PER_MINUTE: 16G + SWWAF_BYTES_LIMIT_PER_HOUR: 80G + SWWAF_BYTES_LIMIT_PER_DAY: 800G + # gitea never serves these; a request for one bans the client. + SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/ + volumes: + - servewaf-state:/var/lib/smallwebwaf + networks: + - site + +networks: + site: + traefik: + external: true + +volumes: + gitea-data: + servewaf-state: +``` + +[`docs/gitea.md`](docs/gitea.md) records what the defaults do in front of gitea, +request by request. + +### `rproxy` beside the app + +The app built on the `smallwebwaf` image, with `rproxy` inside its container and +`503` for every request while `servewaf` is down; and a `servewaf` that observes +before it enforces, looks addresses up in a local database rather than GeoJS, +sends alerts to ntfy, and reads its token from a file. `app/` holds the +`Dockerfile` and `app.run` from "`rproxy` in the app's container". + +```yaml +services: + app: + # A Dockerfile built FROM the smallwebwaf image, with rproxy inside. + build: ./app + environment: + SWRP_WAF_BACKEND: http://servewaf:8080 + # Answer 503 rather than pass a request on while servewaf is down. + SWRP_DEFAULT_DISPOSITION: deny + ports: + - "8080:8080" + + servewaf: + image: smallwebwaf + environment: + SMALLWEBWAF_COMMAND: servewaf + SWWAF_INSTANCE_NAME: fsn1/servewaf + # Refuse nothing yet; log what enforce mode would have done. + SWWAF_MODE: observe + # Look addresses up in a local IPinfo Lite database, never GeoJS. + SWWAF_LOOKUP_SOURCE: file + SWWAF_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb + # Alerts to an ntfy topic, with its token read from a file. + SWWAF_ALERT_NTFY_URL: https://ntfy.example/smallwebwaf + SWWAF_ALERT_NTFY_TOKEN_FILE: /etc/smallwebwaf/tokens/ntfy + SWWAF_ALERT_EVENTS: ban,permanent_ban,waf_block,source_failure,file_error + volumes: + - servewaf-state:/var/lib/smallwebwaf + - /srv/ipinfo:/var/lib/ipinfo:ro + - /srv/smallwebwaf/tokens:/etc/smallwebwaf/tokens:ro + +volumes: + servewaf-state: +``` + +The database and the token are mounted by their directories, not by file, so +that a replaced file is seen. A token file belongs to uid 65532, the +`smallwebwaf` user, with mode `0400`: + +```sh +mkdir -p /srv/smallwebwaf/tokens +openssl rand -hex 32 > /srv/smallwebwaf/tokens/ntfy +chown 65532:65532 /srv/smallwebwaf/tokens/ntfy +chmod 0400 /srv/smallwebwaf/tokens/ntfy +``` + +## Design + +How it works inside. [`SPEC.md`](SPEC.md) gives every rule in full, and why. + +### What happens to a request + +A request goes through these steps, in this order. Once `servewaf` refuses a +request, the steps after that are skipped, apart from the report, and nothing of +the request reaches the app. + +`rproxy` takes the first two: + +1. The health check, `GET /_smallwebwaf/healthz`, is answered at once. +2. The visitor's address is worked out from `X-Forwarded-For`, believed only + from `SWRP_TRUSTED_PROXIES`. `rproxy` sends `servewaf` the request as the + visitor sent it, with the first 64 KiB of its body. + +`servewaf` takes the checks: + +3. The static lists, `SWWAF_ALLOW_NETS` and `SWWAF_DENY_NETS`. +4. The bans. +5. The lookup of the client's AS number and country. +6. The country lists. +7. The reputation sources. +8. The client's limit percentage, from its AS number, its country or a + reputation hit. +9. The rate limits. +10. The trap paths. +11. The rule files. +12. The header limits, and then the Core Rule Set. +13. A path under `/_smallwebwaf/` is answered `404`, so that the app never sees + that prefix. + +`rproxy` takes the rest, with `servewaf` counting at the end: + +14. The request is passed to the app, streaming. +15. After the answer, `rproxy` sends `servewaf` a report with the status and the + bytes each way. For an allowed request, `servewaf` counts the bytes toward + the byte limits and adds the request to the client's history. + +A refused visitor is answered with `SWWAF_BAN_RESPONSE`. A `block` rule or a +Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has +false positives; such refusals count toward `SWWAF_ERROR_BURST_THRESHOLD` +instead. `rproxy` itself answers `405` to a method it does not pass on, and the +status for a size or time limit, such as `413` for a body too large. + +### Clients and bans + +- A client is one IPv4 address or one IPv6 netblock of + `SWWAF_IPV6_GROUP_PREFIX`, since one abuser usually controls a whole /64. +- Each limit is counted over a sliding window with a few integers per client. + The table of clients and the table of bans each have a bound + (`SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_MAX_BANS`), so memory is bounded too. +- A ban refuses every request from its netblock until it ends. Requests refused + under a ban count toward no limit. A permanent ban ends only when an admin + lifts it. +- Every ban carries notes: the client's AS number and country, the limit it + broke or the rule it matched, the request that caused it, and its earlier + bans. An admin adds, keeps or lifts a ban by editing `bans.json` or through + the ban endpoints (see [`docs/admin-endpoints.md`](docs/admin-endpoints.md)). + +### Attack detection + +- The Core Rule Set reads the method, the URL with its query string, and the + headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In + front of a code forge, bodies carry what people write and publish, which the + rules take for attacks. +- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary + requests to a code forge through, such as git over HTTP and sign-in through + Git Credential Manager. No setting undoes them; "Attack detection" in + [`SPEC.md`](SPEC.md) lists them. +- Rule files are plain text, one rule to a line, edited by hand and read again + while `servewaf` runs. The image ships one file of probes no real visitor + sends (see [`docs/rules.md`](docs/rules.md)). + +### State files + +`servewaf` keeps everything in memory, and the five JSON files in +`SWWAF_STATE_DIR` hold a copy: `bans.json`, `clients.json`, `lookups.json`, +`reputation.json` and `alerts.json`. Each is written whole, to a temporary file +renamed over the real one, so a crash leaves a complete file. `servewaf` notices +a file edited on disk and takes the edit in; an edit that does not parse is set +aside as `.bad`, and never stops the running process (see +[`docs/state-files.md`](docs/state-files.md)). + +### The image and its processes + +- The image is built on Ubuntu 26.04 LTS, pinned by digest, with its packages + from a snapshot of Ubuntu's archive and nixpkgs fixed at one release. An app's + `Dockerfile` installs a package with `nix-env -iA nixpkgs.` and gets the + same package each time it is built (see [`docs/image.md`](docs/image.md)). +- runit, a service supervisor, runs `smallwebwaf` as the `smallwebwaf` user (uid + and gid 65532) and starts it again when it exits. In an app's image built on + the `smallwebwaf` image, it runs `rproxy` and the app side by side, each as + its own user. The `run` script of `servewaf` gives the state directory to the + `smallwebwaf` user at each start, so a volume needs no change of owner. +- `rproxy` and `servewaf` use no token between them. `servewaf`'s own endpoints, + under `/_smallwebwaf/`, are for your admin tools and your metrics collector, + at its own address on the private network; nothing routes the internet there. +- `docker stop` gives requests in progress five seconds to finish. With `rproxy` + in the app's container, the app must keep answering for five seconds after + `SIGTERM` for that to help them. +- Only faults found at start stop a process: an invalid setting, a rule file or + state file that does not parse, a lookup database that cannot be read, or an + address it cannot listen on. Once running, a failing lookup service, + reputation source or alert destination never takes `servewaf` down. + +### Packages + +- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`. +- `internal/smallwebwaf`: the process, as the command it is given, and the + health check. +- `internal/config`: reads the settings, the one place they are read. +- `internal/rproxy`: the reverse proxy in front of one app. +- `internal/proxy`: `servewaf`'s server, which runs the checks and answers its + own endpoints. +- `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`, + `internal/lookup`, `internal/reputation`, `internal/anomaly` and + `internal/alerts`: one check or source each. +- `internal/state`, `internal/requestlog`, `internal/remotelog` and + `internal/metrics`: the state files, the log lines, the syslog sender and the + metrics. + +## Rationale + +Small self-hosted sites receive a great deal of traffic nobody asked for: +scrapers that ignore `robots.txt` and crawl every commit of every repository on +a public git server, scanners walking through lists of WordPress and `.env` +paths, and credential-guessing bots. Most of it comes from a small number of +hosting networks and countries. A single-person operation has no abuse desk and +no CDN contract. It needs something small that can be put in front of its +services and left alone. + +The existing tools each solve part of this. Rule-based firewalls catch attack +payloads but do not limit request rates. Rate limiters count requests but cannot +tell a residential visitor from a rented server farm. The products that do most +of it want a database and a web console. None of them can say "clients from +these networks are banned after half as many requests as anyone else". +[`EVALUATION.md`](EVALUATION.md) goes through the candidates one by one. ## Entrypoints @@ -332,10 +700,8 @@ The tests and the linter run in Docker, as phases of the `Dockerfile`, and never on the host. - `script/bootstrap`: installs on the host what the other scripts need, with - `make bootstrap`: - - `make`, `git` and `curl`; - - Go, for `gofmt`; - - node, yarn and prettier. + `make bootstrap`: `make`, `git` and `curl`; Go, for `gofmt`; node, yarn and + prettier. - `script/setup`: readies a fresh clone: `script/bootstrap`, then `script/install-precommit`. `make setup`. - `script/projectname`: prints `smallwebwaf`, which the other scripts tag their @@ -362,89 +728,26 @@ on the host. `make build`. - `script/run`: builds `bin/smallwebwaf` and runs `servewaf` and `rproxy` on the host, with the settings in your environment. `make run`. Unless your settings - say otherwise: - - `rproxy` listens on port 8080, in front of an app at - `http://127.0.0.1:8081`; - - `servewaf` listens on port 8090; - - its state files go in `bin/state`; - - its rule files are those of `share/rules.d`. -- `script/example-app`: builds the image and the example app on it, runs them - with a volume, and checks them as "Checking that it works" above describes. - Then it removes the containers, the network, the volumes and the images. Needs - network access. `make example-app`. + say otherwise, `rproxy` listens on port 8080, in front of an app at + `http://127.0.0.1:8081`; `servewaf` listens on port 8090; its state files go + in `bin/state`; its rule files are those of `share/rules.d`. +- `script/example-app`: builds the image and the example app + (`deploy/example-app`) on it, runs the app with `rproxy` in its container and + `servewaf` in another, and checks them: the health checks; a request passed to + the app; a ban for a broken limit, and a ban for a probe made permanent; that + only `servewaf`'s container changes the state directory's owner; with + `servewaf` stopped, the app still answering through `rproxy`; with `servewaf` + started again, bans applying again; orderly stops with `sv stop` and + `docker stop`; bans surviving a new container on the same volume; a setting + given as a file. Then it removes the containers, the network, the volumes and + the images. Needs network access. `make example-app`. - `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind - `rproxy` and `servewaf`, three containers on one network. Every setting is at - its default, but for those that join the containers and the header that labels - each request. It sends through them a sample of gitea's ordinary use and the - requests the Core Rule Set is known to refuse, and prints what `servewaf` did - with each. Needs network access, and a docker daemon that pushes to a registry - on `localhost` over plain HTTP. [`docs/gitea.md`](docs/gitea.md) records the - results. `make gitea-requests`. - -## Rationale - -Small self-hosted sites receive a great deal of traffic nobody asked for: - -- scrapers that ignore `robots.txt` and crawl every commit of every repository - on a public git server; -- scanners walking through lists of WordPress and `.env` paths; -- credential-guessing bots. - -Most of it comes from a small number of hosting networks and countries. A -single-person operation has no abuse desk and no CDN contract. It needs -something small that can be put in front of its services and left alone. - -The existing tools each solve part of this. Rule-based firewalls catch attack -payloads but do not limit request rates. Rate limiters count requests but cannot -tell a residential visitor from a rented server farm. The products that do most -of it want a database and a web console. None of them can say "clients from -these networks are banned after half as many requests as anyone else". That is -the useful thing to say when nearly all abuse comes from a known list of AS -numbers. [`EVALUATION.md`](EVALUATION.md) goes through the candidates one by -one. - -`smallwebwaf` fills that gap with: - -- per-client limits that ban; -- lower limits for the networks and countries abuse comes from; -- bans that grow with repetition; -- a log that is easy to search and ship; -- a site that keeps answering while the part that decides is down; -- one binary, with no database and one required setting. - -## Design - -The `smallwebwaf` image is built on Ubuntu 26.04 LTS, pinned by digest, with -nixpkgs fixed at one release. An app's `Dockerfile` installs a package with -`nix-env -iA nixpkgs.` and gets the same package each time it is built. -[`docs/image.md`](docs/image.md) lists what the image holds and how each part is -pinned. - -- runit, a service supervisor, runs `smallwebwaf` and starts it again when it - exits. In an app's image built on the `smallwebwaf` image, it runs `rproxy` - and the app side by side, each as its own user. -- `rproxy` listens on port 8080 and passes requests to the app at - `SWRP_UPSTREAM_URL`. It answers the health check itself, and the refusals and - errors "`rproxy`" in [`SPEC.md`](SPEC.md) lists. -- `servewaf` listens on port 8080 of its own container. Nothing should route the - internet there. Its own endpoints, under `/_smallwebwaf/`, are for your admin - tools and your metrics collector. -- `rproxy` and `servewaf` use no token between them. Anything on their private - network can ask `servewaf` a question, or send it a report. -- The app sees the headers traefik sent, with traefik's address added to - `X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust - `rproxy`'s address for forwarded headers. With `rproxy` in the app's - container, that address is `127.0.0.1` or `::1`, which the private address - ranges leave out. -- The state files live in `/var/lib/smallwebwaf` in `servewaf`'s container. A - volume mounted there keeps them across a deploy. -- `docker stop` gives requests in progress five seconds to finish. With `rproxy` - in the app's container, the app must keep answering for five seconds after - `SIGTERM` for that to help them. - -"Deployment" in [`SPEC.md`](SPEC.md) has the rest: the health checks, what -happens when a process stops, moving a port, and the forwarded headers. -"Architecture" in [`SPEC.md`](SPEC.md) lists the packages and what each does. + `rproxy` and `servewaf`, three containers on one network, every setting at its + default but those that join the containers. It sends through them a sample of + gitea's ordinary use and the requests the Core Rule Set is known to refuse, + and prints what `servewaf` did with each. Needs network access, and a docker + daemon that pushes to a registry on `localhost` over plain HTTP. + `make gitea-requests`. ## TODO @@ -457,8 +760,7 @@ happens when a process stops, moving a port, and the forwarded headers. - [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers and misses, and why none was adopted. - [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows. -- [`docs/settings.md`](docs/settings.md): every setting, its default and the - values it takes. +- [`docs/settings.md`](docs/settings.md): every setting and the values it takes. - [`docs/rules.md`](docs/rules.md): the rule file format and the default file. - [`docs/admin-endpoints.md`](docs/admin-endpoints.md): the endpoints under `/_smallwebwaf/`, with examples.