Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 clawbot
|
ac1a69e863 |
@@ -1,13 +1,12 @@
|
|||||||
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
|
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
|
||||||
run example-app gitea-requests gitea-setup
|
run example-app gitea-requests
|
||||||
|
|
||||||
# Makefile targets are thin shims; the implementations live in script/
|
# Makefile targets are thin shims; the implementations live in script/
|
||||||
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
||||||
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
|
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
|
||||||
# which test checks. build and run are for working on the code by hand;
|
# which test checks. build and run are for working on the code by hand;
|
||||||
# example-app checks the image with an app built on it; gitea-requests
|
# example-app checks the image with an app built on it; gitea-requests
|
||||||
# reports the gitea requests the Core Rule Set refuses at the defaults;
|
# reports the gitea requests the Core Rule Set refuses at the defaults.
|
||||||
# gitea-setup checks the setup in deploy/gitea.
|
|
||||||
|
|
||||||
bootstrap:
|
bootstrap:
|
||||||
@script/bootstrap
|
@script/bootstrap
|
||||||
@@ -50,6 +49,3 @@ example-app:
|
|||||||
|
|
||||||
gitea-requests:
|
gitea-requests:
|
||||||
@script/gitea-requests
|
@script/gitea-requests
|
||||||
|
|
||||||
gitea-setup:
|
|
||||||
@script/gitea-setup
|
|
||||||
|
|||||||
@@ -4,45 +4,59 @@
|
|||||||
[@sneak](https://sneak.berlin), for self-hosted web apps. It refuses the
|
[@sneak](https://sneak.berlin), for self-hosted web apps. It refuses the
|
||||||
requests of scrapers, scanners and other abusers before they reach an app.
|
requests of scrapers, scanners and other abusers before they reach an app.
|
||||||
|
|
||||||
## What it does
|
|
||||||
|
|
||||||
A web application firewall reads each request on its way to a web app and
|
A web application firewall reads each request on its way to a web app and
|
||||||
refuses the ones that look like abuse. `smallwebwaf` is one binary, shipped in
|
refuses the ones that look like abuse. `smallwebwaf` is one binary with two
|
||||||
one image, with two commands that do this together:
|
commands that do this together:
|
||||||
|
|
||||||
- `smallwebwaf rproxy` is a small reverse proxy in front of one app. traefik
|
- `smallwebwaf rproxy` is a small reverse proxy in front of one app. It accepts
|
||||||
routes to it. It asks `servewaf` about each request, passes the allowed ones
|
requests on the app's behalf and asks `servewaf` about each one. It passes the
|
||||||
to the app, and passes the app's answers back. It keeps nothing and never
|
allowed ones to the app, and the app's answers back. It keeps nothing, and
|
||||||
touches disk.
|
never touches disk.
|
||||||
- `smallwebwaf servewaf` decides. It runs the checks and keeps the bans, the
|
- `smallwebwaf servewaf` decides. It runs the checks, and keeps the bans,
|
||||||
counters and each client's history. One `servewaf` serves many sites: a client
|
counters and history. One `servewaf` serves many sites.
|
||||||
banned through one site is refused on all of them, and its requests on all of
|
|
||||||
them count toward one set of limits.
|
|
||||||
|
|
||||||
The site keeps answering while `servewaf` is down, slow or restarting: without
|
The site keeps answering while `servewaf` is down, slow or restarting. `rproxy`
|
||||||
an answer in time, `rproxy` passes the request to the app.
|
waits at most 2 seconds for `servewaf`'s answer, and without one passes the
|
||||||
|
request to the app.
|
||||||
|
|
||||||
It protects against:
|
`rproxy` requires one setting, `SWRP_WAF_BACKEND`, which says where `servewaf`
|
||||||
|
is. In a container of its own, it also needs `SWRP_UPSTREAM_URL`, which says
|
||||||
|
where the app is. Every other setting has a default chosen for a service on the
|
||||||
|
open internet. [`SPEC.md`](SPEC.md) says how `smallwebwaf` behaves and why. The
|
||||||
|
reference documents under [`docs/`](docs/) are listed under "Documents" below.
|
||||||
|
|
||||||
- Scrapers that fetch every page of a site, all day. Each client gets a limit on
|
## What it protects against
|
||||||
requests and on bytes per minute, per hour and per day. The defaults leave
|
|
||||||
room for programmatic use of an API, such as an automation fleet, and catch a
|
- Scrapers that fetch every page of a site, all day. Rate limiting gives each
|
||||||
client firing requests as fast as it can within seconds. A client over a limit
|
client 500 requests a minute, 20,000 an hour and 300,000 a day. That leaves
|
||||||
is refused and banned: for a while at first, for three times as long at each
|
room for programmatic use of an API, such as an automation fleet on another
|
||||||
repeat, and permanently when it keeps at it.
|
host. A fleet of about 20 agents makes about 120 requests a minute at its
|
||||||
|
busiest. A gitea page opened with an empty cache loads about 40 files, so a
|
||||||
|
few such first visits fit in the minute limit too. The minute limit catches,
|
||||||
|
within seconds, a client firing requests as fast as it can. The hour limit
|
||||||
|
catches a client that keeps up more than about 5.5 a second for an hour. The
|
||||||
|
day limit catches a client that keeps up 4 a second around the clock. Counting
|
||||||
|
cannot catch a client that stays under all three limits, such as one that
|
||||||
|
keeps up 3 a second; only the protections below can. Byte limits give each
|
||||||
|
client 4 GiB a minute, 20 GiB an hour and 200 GiB a day. A client over a limit
|
||||||
|
is refused and banned. The first ban lasts an hour. Each repeat within a day
|
||||||
|
makes the next ban three times as long. A ban that would last more than seven
|
||||||
|
days is permanent.
|
||||||
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
||||||
default rule file bans such a client at its first probe, and its next request
|
default rule file bans such a client for seven days at its first probe. Its
|
||||||
during the ban makes the ban permanent. Your own rules, or a list of trap
|
next request during the ban makes the ban permanent. The default rule file
|
||||||
paths, ban probes for other apps, such as a WordPress login, the same way.
|
does not cover probes for other apps, such as a WordPress login. A `ban` rule
|
||||||
|
of your own, or an entry in `SWWAF_TRAP_PATHS`, bans those the same way.
|
||||||
- Attacks carried in a request's URL or headers, such as SQL injection and path
|
- Attacks carried in a request's URL or headers, such as SQL injection and path
|
||||||
traversal. The OWASP Core Rule Set, a published set of rules that recognise
|
traversal. The OWASP Core Rule Set is a published set of rules that recognise
|
||||||
common web attacks, runs on every request through Coraza, a Go engine. A
|
common web attacks. Coraza, a Go engine, runs those rules (see "Attack
|
||||||
request the rules take for an attack is refused.
|
detection" in [`SPEC.md`](SPEC.md)). A request the rules take for an attack is
|
||||||
|
refused.
|
||||||
- Abuse from networks and countries you do not want. You can:
|
- Abuse from networks and countries you do not want. You can:
|
||||||
- refuse whole netblocks and countries;
|
- refuse whole netblocks and countries;
|
||||||
- give the clients of chosen networks and countries a fraction of every
|
- give the clients of chosen networks and countries a fraction of every
|
||||||
limit, so that they are banned sooner;
|
limit, so that they are banned sooner;
|
||||||
- refuse or limit clients on a blocklist, a DNS blocklist, or with a high
|
- refuse or limit clients on a blocklist or a DNS blocklist, or with a high
|
||||||
AbuseIPDB score;
|
AbuseIPDB score;
|
||||||
- ban clients named by your own CrowdSec engine.
|
- ban clients named by your own CrowdSec engine.
|
||||||
- Slow and oversized requests. Every request and every answer has a time limit
|
- Slow and oversized requests. Every request and every answer has a time limit
|
||||||
@@ -50,27 +64,29 @@ It protects against:
|
|||||||
|
|
||||||
For the operator, it:
|
For the operator, it:
|
||||||
|
|
||||||
- logs each request as JSON lines, saying what it did and why;
|
- logs each request as JSON lines, saying what it did and why (see
|
||||||
- sends alerts to a webhook, Slack or ntfy;
|
[`docs/request-log.md`](docs/request-log.md));
|
||||||
- serves Prometheus metrics;
|
- sends alerts to a webhook, Slack or ntfy (see
|
||||||
|
[`docs/alerts.md`](docs/alerts.md));
|
||||||
|
- serves Prometheus metrics (see [`docs/metrics.md`](docs/metrics.md));
|
||||||
- keeps its bans, counters and history in JSON files you can read and edit, so
|
- keeps its bans, counters and history in JSON files you can read and edit, so
|
||||||
that a restart loses nothing and a ban can be lifted with an editor.
|
that a restart loses nothing and a ban can be lifted with an editor (see
|
||||||
|
[`docs/state-files.md`](docs/state-files.md)).
|
||||||
|
|
||||||
Not built, and not planned:
|
Not built, and not planned:
|
||||||
|
|
||||||
- TLS and routing by host name. Both stay with traefik, the reverse proxy in
|
- TLS and routing by host name. Both stay with traefik, the reverse proxy in
|
||||||
front of `rproxy`.
|
front of `rproxy`.
|
||||||
- Browser challenges such as captchas.
|
- Browser challenges such as captchas.
|
||||||
- A web console, or a configuration file.
|
- A web console.
|
||||||
- Defence against floods large enough to fill the host's network link.
|
- Defence against floods large enough to fill the host's network link.
|
||||||
|
|
||||||
## Getting started
|
## Getting started
|
||||||
|
|
||||||
### The image
|
Build the image from a clone. `make docker` runs the tests and the linter, then
|
||||||
|
builds the image, tagged `smallwebwaf`. It builds for amd64, on an amd64 host
|
||||||
No image is published. Build it from a clone. `make docker` runs the tests and
|
only: the `Dockerfile` checks Ubuntu's package lists against the hashes of the
|
||||||
the linter, then builds the image, tagged `smallwebwaf`. It builds for amd64, on
|
amd64 archive.
|
||||||
an amd64 host only.
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
git clone https://git.eeqj.de/sneak/smallwebwaf.git
|
git clone https://git.eeqj.de/sneak/smallwebwaf.git
|
||||||
@@ -78,38 +94,62 @@ cd smallwebwaf
|
|||||||
make docker
|
make docker
|
||||||
```
|
```
|
||||||
|
|
||||||
Push the image to a registry your hosts pull from, and name it there by digest.
|
Push the image to a registry your hosts pull from.
|
||||||
|
|
||||||
The image runs `rproxy` by default. To run `servewaf` from the same image, set
|
The image runs `rproxy` by default. To run `servewaf` from the same image, set
|
||||||
the container's `SMALLWEBWAF_COMMAND` to `servewaf`. The image's health check
|
the container's `SMALLWEBWAF_COMMAND` to `servewaf`.
|
||||||
follows the command: in `rproxy`'s container it passes once `rproxy` answers and
|
|
||||||
the app accepts connections, whether or not `servewaf` answers; in `servewaf`'s
|
|
||||||
container, once `servewaf` answers.
|
|
||||||
|
|
||||||
### The usual setup: three containers
|
### The usual setup: three containers
|
||||||
|
|
||||||
The usual setup is three containers on one private network:
|
The usual setup is three containers on one private network:
|
||||||
|
|
||||||
- the app;
|
- the app;
|
||||||
- `rproxy`, in front of the app, which traefik routes to: `SWRP_WAF_BACKEND`
|
- `rproxy`, in front of the app, which traefik routes to;
|
||||||
names `servewaf` and `SWRP_UPSTREAM_URL` names the app;
|
- `servewaf`, with a volume for its state files.
|
||||||
- `servewaf`, with `SMALLWEBWAF_COMMAND=servewaf` and a volume at
|
|
||||||
`/var/lib/smallwebwaf` for its state files.
|
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker network create site
|
||||||
|
|
||||||
|
docker run --detach --name servewaf --network site \
|
||||||
|
--env SMALLWEBWAF_COMMAND=servewaf \
|
||||||
|
--volume servewaf-state:/var/lib/smallwebwaf \
|
||||||
|
<registry>/smallwebwaf@sha256:<digest>
|
||||||
|
|
||||||
|
docker run --detach --name app --network site \
|
||||||
|
<registry>/app@sha256:<digest>
|
||||||
|
|
||||||
|
docker run --detach --name rproxy --network site \
|
||||||
|
--env SWRP_WAF_BACKEND=http://servewaf:8080 \
|
||||||
|
--env SWRP_UPSTREAM_URL=http://app:8081 \
|
||||||
|
<registry>/smallwebwaf@sha256:<digest>
|
||||||
|
```
|
||||||
|
|
||||||
|
Here the app listens on port 8081, at its container's address on that network.
|
||||||
Point traefik's labels at port 8080 of `rproxy`
|
Point traefik's labels at port 8080 of `rproxy`
|
||||||
(`traefik.http.services.<name>.loadbalancer.server.port=8080`). Publish no port
|
(`traefik.http.services.<name>.loadbalancer.server.port=8080`). Publish no port
|
||||||
of the app or of `servewaf`. Anything on that network can reach the app around
|
of the app or of `servewaf`. Anything on that network can reach the app around
|
||||||
`rproxy`, and can ask `servewaf` questions, so the network holds only what you
|
`rproxy`, and can ask `servewaf` questions.
|
||||||
trust. Each further app adds its own container and its own `rproxy`, whose
|
|
||||||
`SWRP_WAF_BACKEND` names the same `servewaf`. The "Smallest setup" and "In front
|
With no setting, `servewaf` sends the address of every new visitor to GeoJS, a
|
||||||
of gitea" examples below show this shape.
|
web service, to find its country and network. `SWWAF_LOOKUP_SOURCE=off` sends
|
||||||
|
nothing. `SWWAF_LOOKUP_SOURCE=file`, with `SWWAF_LOOKUP_DB_PATH` naming a
|
||||||
|
database file on your own host, looks addresses up in that file (see "Lookup of
|
||||||
|
AS number and country" in [`SPEC.md`](SPEC.md)).
|
||||||
|
|
||||||
|
### Many sites, one `servewaf`
|
||||||
|
|
||||||
|
Run one `rproxy` for each app, all with `SWRP_WAF_BACKEND` naming the same
|
||||||
|
`servewaf`. It keeps one table of clients and bans for all of them:
|
||||||
|
|
||||||
|
- A client banned through one site is refused on all of them.
|
||||||
|
- A client's requests on all of them count toward one set of limits.
|
||||||
|
- Log lines, alerts and ban notes name the site by its host.
|
||||||
|
|
||||||
### `rproxy` in the app's container
|
### `rproxy` in the app's container
|
||||||
|
|
||||||
An app's image can instead be built on the `smallwebwaf` image, so that `rproxy`
|
An app's image can also be built on the `smallwebwaf` image, so that `rproxy`
|
||||||
runs beside the app in one container. `servewaf` still runs in a container of
|
runs beside the app. `servewaf` still runs in a container of its own. The app's
|
||||||
its own. The app's `Dockerfile` starts `FROM` the `smallwebwaf` image, named by
|
`Dockerfile` starts `FROM` the `smallwebwaf` image, named by digest, and adds:
|
||||||
digest, and adds:
|
|
||||||
|
|
||||||
- the app's binary;
|
- the app's binary;
|
||||||
- the packages the app needs, from the nixpkgs in the image;
|
- the packages the app needs, from the nixpkgs in the image;
|
||||||
@@ -151,446 +191,39 @@ main "$@"
|
|||||||
|
|
||||||
Leave `ENTRYPOINT` and `USER` out of the app's `Dockerfile`. The image's own
|
Leave `ENTRYPOINT` and `USER` out of the app's `Dockerfile`. The image's own
|
||||||
entrypoint starts `rproxy` and the app side by side, each as its own user. Run
|
entrypoint starts `rproxy` and the app side by side, each as its own user. Run
|
||||||
the app's image with `SWRP_WAF_BACKEND` set. `deploy/example-app` is such an
|
the app's image with `SWRP_WAF_BACKEND` set.
|
||||||
app, and the "`rproxy` beside the app" example below runs one.
|
|
||||||
|
|
||||||
The app sees the headers traefik sent, with traefik's address added to
|
`deploy/example-app` is such an app. `make example-app` builds it and checks it
|
||||||
`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
|
(see "Checking that it works" below). To run `rproxy` and `servewaf` on the host
|
||||||
`rproxy`'s address for forwarded headers. In a container of its own, `rproxy` is
|
while working on them, see `script/run` under "Entrypoints" below.
|
||||||
on the private network, which the usual default of a trusted-proxy setting
|
|
||||||
covers. Beside the app, its address is `127.0.0.1` or `::1`, which the private
|
|
||||||
ranges leave out, so the app's own setting must name them too, as `app.run`
|
|
||||||
does.
|
|
||||||
|
|
||||||
### Checking that it works
|
### When `servewaf` is down, slow or overloaded
|
||||||
|
|
||||||
- `docker inspect --format '{{.State.Health.Status}}' <container>` says
|
`SWRP_DEFAULT_DISPOSITION` decides what `rproxy` does with a request when
|
||||||
`healthy` for each container of `smallwebwaf`.
|
`servewaf`:
|
||||||
- A log line with `"msg":"starting"` gives the version, the address and every
|
|
||||||
setting in effect, with secrets masked.
|
|
||||||
- A request to the site through traefik gets the app's answer.
|
|
||||||
`https://<site>/_smallwebwaf/healthz` answers `ok`. `servewaf`'s log lines
|
|
||||||
about the request have `"action":"forward"`.
|
|
||||||
- To see a refusal without banning yourself, run `servewaf` with
|
|
||||||
`SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app.
|
|
||||||
`servewaf`'s log lines have `"would_action":"banned"` and
|
|
||||||
`"rule_ids":["env-file"]`. A ban you get in `enforce` mode is lifted as
|
|
||||||
"Clients and bans" below describes.
|
|
||||||
- With `SWWAF_METRICS_TOKEN` set, this answers with the metrics, from a host on
|
|
||||||
`servewaf`'s network:
|
|
||||||
`curl -H "Authorization: Bearer $TOKEN" http://servewaf:8080/_smallwebwaf/metrics`.
|
|
||||||
- `make example-app` builds the image and an app on it, runs them, and checks
|
|
||||||
the whole exchange (see "Entrypoints").
|
|
||||||
|
|
||||||
## Configuration
|
- cannot be connected to;
|
||||||
|
- does not answer at all;
|
||||||
|
- does not answer within `SWRP_TIMEOUT_MS`, 2 seconds by default.
|
||||||
|
|
||||||
Every setting is an environment variable, read once at start; changing one means
|
Its default, `allow`, passes the request to the app without `servewaf`'s answer.
|
||||||
restarting the container. `rproxy` reads only the `SWRP_` settings, and
|
`deny` answers `503` instead. Either way the request gets no report, so its
|
||||||
`servewaf` only the `SWWAF_` ones. Every setting but `SWRP_WAF_BACKEND` has a
|
bytes are not counted and it is not added to its client's history. When
|
||||||
default, chosen for a service on the open internet. A setting that needs an
|
`servewaf` does answer, a `deny` or an error goes to the visitor as `servewaf`
|
||||||
account, a key, a token or an alert destination stays off until you give one.
|
gave it, and `SWRP_DEFAULT_DISPOSITION` plays no part.
|
||||||
|
|
||||||
- Any `SWWAF_` setting can be given as a file instead, for secrets and long
|
## What happens to a request
|
||||||
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
|
||||||
The file's contents are the value, less one newline at the end. Setting both
|
|
||||||
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already names a file, so
|
|
||||||
it has no `_FILE` form, and no `SWRP_` setting has one, since `rproxy` never
|
|
||||||
touches disk.
|
|
||||||
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
|
||||||
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
|
||||||
1024: `512K`, `100M`, `5G`.
|
|
||||||
- A count is a whole number of requests, such as `1000`.
|
|
||||||
- A time in milliseconds is a whole number above zero, such as `2000`.
|
|
||||||
- A list is comma separated. A list set to an empty value is an empty list, and
|
|
||||||
replaces the default.
|
|
||||||
- A netblock is an address and a prefix length, such as `10.0.0.0/8`; a bare
|
|
||||||
address stands for itself alone. A country is a two-letter code in either
|
|
||||||
case, such as `de`. An AS number is `AS` and the number, such as `AS64496`. A
|
|
||||||
percentage is a whole number from 0 to 100. A path prefix starts with `/`.
|
|
||||||
- `off` switches a limit, a threshold or a timeout off where the table says a
|
|
||||||
setting takes it.
|
|
||||||
- An invalid setting stops the process at start with one log line naming the
|
|
||||||
variable and what is wrong with the value. The settings in effect are logged
|
|
||||||
on one line at start, with tokens, keys and webhook paths shown as `********`.
|
|
||||||
|
|
||||||
Every setting the code reads, with its default:
|
A request goes through fifteen steps, in this order. Once `servewaf` refuses a
|
||||||
|
request, the steps after that are skipped, apart from the report of step 15, and
|
||||||
| Variable | Default | What it does |
|
nothing of the request reaches the app. "What happens to a request" in
|
||||||
| ------------------------------------------ | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
[`SPEC.md`](SPEC.md) gives each step in full, under the same numbers.
|
||||||
| `SMALLWEBWAF_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `servewaf`. |
|
|
||||||
| `SWRP_WAF_BACKEND` | required | `servewaf`'s URL, such as `http://servewaf:8080`. |
|
|
||||||
| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
|
||||||
| `SWRP_LISTEN_ADDR` | `:8080` | Where `rproxy` listens; leave the host part empty. |
|
|
||||||
| `SWRP_TRUSTED_PROXIES` | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none. |
|
|
||||||
| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `servewaf`'s answer to a question and to a report, in milliseconds. |
|
|
||||||
| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `servewaf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
|
||||||
| `SWRP_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
|
||||||
| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`; a larger one is answered `502`, or cut off once it has started. |
|
|
||||||
| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | How long the visitor may take to send its request line and headers, and then the whole request to reach the app, in milliseconds or `off`. |
|
|
||||||
| `SWRP_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
|
||||||
| `SWWAF_LISTEN_ADDR` | `:8080` | Where `servewaf` listens; leave the host part empty. |
|
|
||||||
| `SWWAF_INSTANCE_NAME` | the container's host name | The name in every line `servewaf` logs, and in every alert and metric, such as `fsn1/servewaf`. |
|
|
||||||
| `SWWAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
|
||||||
| `SWWAF_IPV6_GROUP_PREFIX` | `64` | The prefix length of the IPv6 netblock that counts as one client, from 32 to 128. |
|
|
||||||
| `SWWAF_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
|
||||||
| `SWWAF_MAX_BANS` | `5000` | The most bans `servewaf` keeps of its own; bans an admin made do not count. |
|
|
||||||
| `SWWAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
|
||||||
| `SWWAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. |
|
|
||||||
| `SWWAF_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
|
||||||
| `SWWAF_RATE_LIMIT_PER_MINUTE` | `500` | The most requests a client may make in a minute, a count or `off`; one more bans it. |
|
|
||||||
| `SWWAF_RATE_LIMIT_PER_HOUR` | `20000` | The most requests a client may make in an hour, a count or `off`; one more bans it. |
|
|
||||||
| `SWWAF_RATE_LIMIT_PER_DAY` | `300000` | The most requests a client may make in a day, a count or `off`; one more bans it. |
|
|
||||||
| `SWWAF_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes whose requests the request and byte limits neither count nor refuse, such as `/assets/`. |
|
|
||||||
| `SWWAF_BYTES_LIMIT_PER_MINUTE` | `4G` | The most bytes a client's requests may carry in a minute, a size or `off`; the request that takes a client past any byte limit bans it. |
|
|
||||||
| `SWWAF_BYTES_LIMIT_PER_HOUR` | `20G` | The most bytes a client's requests may carry in an hour, a size or `off`. |
|
|
||||||
| `SWWAF_BYTES_LIMIT_PER_DAY` | `200G` | The most bytes a client's requests may carry in a day, a size or `off`. |
|
|
||||||
| `SWWAF_BYTES_COUNT` | `both` | Which body bytes the byte limits count: `response`, `request` or `both`. |
|
|
||||||
| `SWWAF_ERROR_BURST_THRESHOLD` | `30` | The most refusals by a rule file, a trap path or the Core Rule Set a client may get in a minute, a count or `off`; one more bans it. |
|
|
||||||
| `SWWAF_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
|
||||||
| `SWWAF_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. |
|
|
||||||
| `SWWAF_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A limit broken again within this time after the last such ban ended gets a ban three times as long. |
|
|
||||||
| `SWWAF_MAX_BAN_DURATION` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. |
|
|
||||||
| `SWWAF_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. |
|
|
||||||
| `SWWAF_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
|
||||||
| `SWWAF_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs`, a web service, `file`, or `off`. |
|
|
||||||
| `SWWAF_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
|
||||||
| `SWWAF_LOOKUP_TIMEOUT` | `800ms` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take. |
|
|
||||||
| `SWWAF_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
|
||||||
| `SWWAF_DENIED_COUNTRIES` | empty | Countries whose clients are refused, such as `cn,ru`. |
|
|
||||||
| `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. |
|
|
||||||
| `SWWAF_ASN_LIMIT_PERCENT` | empty | AS numbers whose clients get a percentage of every request and byte limit, such as `AS14061:50,AS16276:25`. |
|
|
||||||
| `SWWAF_COUNTRY_LIMIT_PERCENT` | empty | The same by country, such as `cn:25`. |
|
|
||||||
| `SWWAF_ASN_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the AS numbers listed, in place of their `SWWAF_ASN_LIMIT_PERCENT`. |
|
|
||||||
| `SWWAF_COUNTRY_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the countries listed, in place of their `SWWAF_COUNTRY_LIMIT_PERCENT`. |
|
|
||||||
| `SWWAF_UNKNOWN_LIMIT_PERCENT` | `100` | The percentage of every limit a client without a country gets. |
|
|
||||||
| `SWWAF_ASN_LIMIT_PERCENT_URL` | unset | The URL of a text file of `AS<number>:<percent>` lines, fetched like a blocklist; where it and `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. |
|
|
||||||
| `SWWAF_RULES_DIR` | `/etc/smallwebwaf/rules.d` | The directory of the rule files, read at start and again when a file there changes. |
|
|
||||||
| `SWWAF_RULES_ENABLED` | `true` | `false` reads no rule file. |
|
|
||||||
| `SWWAF_TRAP_PATHS` | empty | Paths only scanners ask for, such as `/wp-login.php,/xmlrpc.php`; a request for one bans the client. |
|
|
||||||
| `SWWAF_WAF_MODE` | `block` | What the Core Rule Set does with a match: `block` refuses it with `403`, `detect` only logs and alerts it, `off` runs no rule. |
|
|
||||||
| `SWWAF_WAF_PARANOIA_LEVEL` | `1` | The Core Rule Set's paranoia level, from 1 to 4; each level up finds more attacks and takes more ordinary requests for attacks. |
|
|
||||||
| `SWWAF_WAF_ANOMALY_THRESHOLD` | `5` | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5. |
|
|
||||||
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
|
||||||
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
|
||||||
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
|
||||||
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
|
||||||
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
|
||||||
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
|
||||||
| `SWWAF_DNSBL_ZONES` | empty | DNS blocklists asked about each new client in the background, such as `dnsbl.dronebl.org`. |
|
|
||||||
| `SWWAF_DNSBL_RESOLVER` | unset | The resolver the zones are asked through, an IP address with an optional port; unset, the host's resolver is used. |
|
|
||||||
| `SWWAF_ABUSEIPDB_KEY` | unset | The key of your AbuseIPDB account; set, a client that has committed an offence is checked in the background. |
|
|
||||||
| `SWWAF_ABUSEIPDB_MIN_SCORE` | `75` | The least AbuseIPDB abuse confidence score, from 0 to 100, that is a hit. |
|
|
||||||
| `SWWAF_ABUSEIPDB_DAILY_BUDGET` | `900` | The most AbuseIPDB checks made in a day. |
|
|
||||||
| `SWWAF_CROWDSEC_LAPI_URL` | unset | The local API of a CrowdSec engine you run, such as `http://172.17.0.1:8080`; its decision list is fetched every minute, and a listed client is banned. |
|
|
||||||
| `SWWAF_CROWDSEC_LAPI_KEY` | unset | The key the engine gave for `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it; set with the URL, never alone. |
|
|
||||||
| `SWWAF_REPUTATION_ACTION` | `limit:25` | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:<percent>` or `log`. |
|
|
||||||
| `SWWAF_REPUTATION_CACHE_TTL` | `24h` | How long a DNS blocklist verdict or an AbuseIPDB score is used. |
|
|
||||||
| `SWWAF_REPUTATION_TIMEOUT` | `2s` | How long a DNS blocklist query or an AbuseIPDB check may take. |
|
|
||||||
| `SWWAF_STATE_DIR` | `/var/lib/smallwebwaf` | The directory of the state files, an absolute path. |
|
|
||||||
| `SWWAF_STATE_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. |
|
|
||||||
| `SWWAF_STATE_COUNTER_INTERVAL` | `15m` | How often every state file is written. |
|
|
||||||
| `SWWAF_LOG_LEVEL` | `info` | The least severe of `servewaf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. |
|
|
||||||
| `SWWAF_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. |
|
|
||||||
| `SWWAF_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `servewaf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. |
|
|
||||||
| `SWWAF_LOG_REMOTE_TLS_CA_FILE` | unset | A file of PEM certificates a `syslog+tls` server's certificate must chain to, instead of the host's own. |
|
|
||||||
| `SWWAF_LOG_REMOTE_BUFFER` | `10000` | The most lines held while they wait to be sent; past it, the oldest is dropped. |
|
|
||||||
| `SWWAF_LOG_REMOTE_FACILITY` | `local0` | The syslog facility of each record sent. |
|
|
||||||
| `SWWAF_LOG_REMOTE_APP_NAME` | `SWWAF_INSTANCE_NAME` | The app name in each syslog record, 1 to 48 printable ASCII characters without a space. |
|
|
||||||
| `SWWAF_ADMIN_TOKEN` | unset | The bearer token for the ban and client endpoints, at least 32 characters; unset, they answer `404`. |
|
|
||||||
| `SWWAF_METRICS_TOKEN` | unset | The bearer token for `/_smallwebwaf/metrics`, at least 32 characters; unset, the metrics are off. |
|
|
||||||
| `SWWAF_METRICS_TOP_N` | `50` | How many AS numbers and how many countries get metric series of their own; the rest are summed as `other`. |
|
|
||||||
| `SWWAF_ALERT_WEBHOOK_URL` | unset | A webhook each alert is posted to as a JSON object. |
|
|
||||||
| `SWWAF_ALERT_WEBHOOK_HEADERS` | empty | Headers sent with each post, a list of `Name:value`, such as `Authorization:Bearer <token>`. |
|
|
||||||
| `SWWAF_ALERT_SLACK_WEBHOOK_URL` | unset | A Slack incoming webhook each alert is posted to as a message. |
|
|
||||||
| `SWWAF_ALERT_NTFY_URL` | unset | An ntfy topic each alert is published to, as its full URL, such as `https://ntfy.sh/my-alerts`. |
|
|
||||||
| `SWWAF_ALERT_NTFY_TOKEN` | unset | An ntfy access token sent with each alert. |
|
|
||||||
| `SWWAF_ALERT_EVENTS` | `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error` | The events alerts are sent for; the hourly `summary` is sent whatever it says. |
|
|
||||||
| `SWWAF_ALERT_COOLDOWN` | `15m` | How long after an alert its repeats are held back and counted, a duration or `off`. |
|
|
||||||
| `SWWAF_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
|
|
||||||
| `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
|
|
||||||
| `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
|
|
||||||
| `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when one client's requests carry more bytes than this in a minute, a size. |
|
|
||||||
| `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when one client's requests carry more bytes than this in an hour, a size. |
|
|
||||||
| `SWWAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
|
|
||||||
| `SWWAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
|
|
||||||
| `SWWAF_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when the netblock around a client carries more bytes than this in a minute, a size. |
|
|
||||||
| `SWWAF_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when the netblock around a client carries more bytes than this in an hour, a size. |
|
|
||||||
| `SWWAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWWAF_ANOMALY_NET_` thresholds count, from 0 to 32. |
|
|
||||||
| `SWWAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWWAF_ANOMALY_NET_` thresholds count, from 0 to 128. |
|
|
||||||
| `SWWAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
|
|
||||||
| `SWWAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
|
|
||||||
| `SWWAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when one AS number carries more bytes than this in a minute, a size. |
|
|
||||||
| `SWWAF_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when one AS number carries more bytes than this in an hour, a size. |
|
|
||||||
| `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
|
|
||||||
| `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
|
|
||||||
| `SWWAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when the whole service carries more bytes than this in a minute, a size. |
|
|
||||||
| `SWWAF_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when the whole service carries more bytes than this in an hour, a size. |
|
|
||||||
| `SWWAF_WATCH_NETS` | empty | Named netblocks the `SWWAF_WATCH_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
|
|
||||||
| `SWWAF_WATCH_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
|
|
||||||
| `SWWAF_WATCH_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
|
|
||||||
| `SWWAF_WATCH_BYTES_PER_MINUTE` | `off` | Alert when a watched netblock carries more bytes than this in a minute, a size. |
|
|
||||||
| `SWWAF_WATCH_BYTES_PER_HOUR` | `off` | Alert when a watched netblock carries more bytes than this in an hour, a size. |
|
|
||||||
|
|
||||||
[`docs/settings.md`](docs/settings.md) gives the values each setting takes in
|
|
||||||
full, and the limits that are fixed rather than settings.
|
|
||||||
|
|
||||||
## docker-compose examples
|
|
||||||
|
|
||||||
Each example is a complete `compose.yaml`. `smallwebwaf` is the tag
|
|
||||||
`make docker` gives the image; on another host, use the name you pushed it
|
|
||||||
under, by digest. Docker Compose puts the services on a private network of its
|
|
||||||
own, where each reaches the others by service name. Only "In front of gitea"
|
|
||||||
uses traefik. The other three publish `rproxy`'s port on the host, for trying it
|
|
||||||
out.
|
|
||||||
|
|
||||||
### Smallest setup
|
|
||||||
|
|
||||||
One app, `rproxy` in front of it, and `servewaf` with a volume for its state
|
|
||||||
files, every setting at its default.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
# Your app, which listens on port 8081.
|
|
||||||
image: registry.example/app@sha256:0000000000000000000000000000000000000000000000000000000000000000
|
|
||||||
|
|
||||||
rproxy:
|
|
||||||
image: smallwebwaf
|
|
||||||
environment:
|
|
||||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
|
||||||
SWRP_UPSTREAM_URL: http://app:8081
|
|
||||||
ports:
|
|
||||||
- "8080:8080"
|
|
||||||
|
|
||||||
servewaf:
|
|
||||||
image: smallwebwaf
|
|
||||||
environment:
|
|
||||||
SMALLWEBWAF_COMMAND: servewaf
|
|
||||||
volumes:
|
|
||||||
- servewaf-state:/var/lib/smallwebwaf
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
servewaf-state:
|
|
||||||
```
|
|
||||||
|
|
||||||
### Custom per-address limits
|
|
||||||
|
|
||||||
For a site with no API:
|
|
||||||
|
|
||||||
- tighter limits for everyone;
|
|
||||||
- no limits for the office and the automation host;
|
|
||||||
- no checks for monitoring;
|
|
||||||
- a refused netblock;
|
|
||||||
- a smaller share of the limits for two hosting networks and one country;
|
|
||||||
- trap paths;
|
|
||||||
- `429` for a refused client.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
image: registry.example/app@sha256:0000000000000000000000000000000000000000000000000000000000000000
|
|
||||||
|
|
||||||
rproxy:
|
|
||||||
image: smallwebwaf
|
|
||||||
environment:
|
|
||||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
|
||||||
SWRP_UPSTREAM_URL: http://app:8081
|
|
||||||
ports:
|
|
||||||
- "8080:8080"
|
|
||||||
|
|
||||||
servewaf:
|
|
||||||
image: smallwebwaf
|
|
||||||
environment:
|
|
||||||
SMALLWEBWAF_COMMAND: servewaf
|
|
||||||
# Monitoring skips every check.
|
|
||||||
SWWAF_ALLOW_NETS: 192.0.2.10
|
|
||||||
# The office and the automation host are never rate limited.
|
|
||||||
SWWAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
|
||||||
# A netblock that sends nothing wanted.
|
|
||||||
SWWAF_DENY_NETS: 198.51.100.128/25
|
|
||||||
# Tighter limits for everyone else: a site with no API.
|
|
||||||
SWWAF_RATE_LIMIT_PER_MINUTE: "120"
|
|
||||||
SWWAF_RATE_LIMIT_PER_HOUR: "3000"
|
|
||||||
SWWAF_RATE_LIMIT_PER_DAY: "30000"
|
|
||||||
# Hosting networks and one country get a share of those limits.
|
|
||||||
SWWAF_ASN_LIMIT_PERCENT: AS14061:50,AS16276:25
|
|
||||||
SWWAF_COUNTRY_LIMIT_PERCENT: cn:25
|
|
||||||
# Paths only scanners ask for; one request bans the client.
|
|
||||||
SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php
|
|
||||||
# Answer a refused client with 429 rather than 403.
|
|
||||||
SWWAF_BAN_RESPONSE: "429"
|
|
||||||
SWWAF_LIMIT_BAN_DURATION: 30m
|
|
||||||
volumes:
|
|
||||||
- servewaf-state:/var/lib/smallwebwaf
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
servewaf-state:
|
|
||||||
```
|
|
||||||
|
|
||||||
### In front of gitea
|
|
||||||
|
|
||||||
gitea behind traefik, set up for a code forge:
|
|
||||||
|
|
||||||
- the upload limits raised, in size and in time;
|
|
||||||
- the response size limit and the byte limits raised, for clones over 1 GiB;
|
|
||||||
- the Actions runners exempt from counting;
|
|
||||||
- the default request limits kept, since they already leave room for API
|
|
||||||
clients.
|
|
||||||
|
|
||||||
`traefik` is the network traefik is on, made outside this file.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
gitea:
|
|
||||||
# gitea 1.27.3, 2026-08-29
|
|
||||||
image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef
|
|
||||||
environment:
|
|
||||||
GITEA__server__ROOT_URL: https://git.example/
|
|
||||||
# The host and the port of the SSH clone URLs gitea shows.
|
|
||||||
GITEA__server__SSH_DOMAIN: git.example
|
|
||||||
GITEA__server__SSH_PORT: "2222"
|
|
||||||
# gitea's TCP peer is rproxy, on the private network.
|
|
||||||
GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
|
|
||||||
volumes:
|
|
||||||
- gitea-data:/data
|
|
||||||
networks:
|
|
||||||
- site
|
|
||||||
ports:
|
|
||||||
# SSH does not pass through smallwebwaf.
|
|
||||||
- "2222:22"
|
|
||||||
|
|
||||||
rproxy:
|
|
||||||
image: smallwebwaf
|
|
||||||
environment:
|
|
||||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
|
||||||
SWRP_UPSTREAM_URL: http://gitea:3000
|
|
||||||
# Pushes, LFS objects and image layers over 100 MiB, or slower
|
|
||||||
# than a minute, are cut off at the defaults.
|
|
||||||
SWRP_MAX_REQUEST_SIZE: 2G
|
|
||||||
SWRP_MAX_REQUEST_TIME_MS: "600000"
|
|
||||||
# A clone larger than 1 GiB is cut off at the default.
|
|
||||||
SWRP_MAX_RESPONSE_SIZE: 4G
|
|
||||||
networks:
|
|
||||||
- site
|
|
||||||
- traefik
|
|
||||||
labels:
|
|
||||||
traefik.enable: "true"
|
|
||||||
# rproxy is on two networks; traefik reaches it on this one.
|
|
||||||
traefik.docker.network: traefik
|
|
||||||
traefik.http.routers.git.rule: Host(`git.example`)
|
|
||||||
traefik.http.routers.git.entrypoints: websecure
|
|
||||||
traefik.http.routers.git.tls.certresolver: letsencrypt
|
|
||||||
traefik.http.services.git.loadbalancer.server.port: "8080"
|
|
||||||
|
|
||||||
servewaf:
|
|
||||||
image: smallwebwaf
|
|
||||||
environment:
|
|
||||||
SMALLWEBWAF_COMMAND: servewaf
|
|
||||||
SWWAF_INSTANCE_NAME: fsn1/servewaf
|
|
||||||
# The Actions runners ask for work all day; never count them.
|
|
||||||
SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24
|
|
||||||
# Raised with rproxy's response limit, to four times the default
|
|
||||||
# each, so that one clone of that size breaks no byte limit.
|
|
||||||
SWWAF_BYTES_LIMIT_PER_MINUTE: 16G
|
|
||||||
SWWAF_BYTES_LIMIT_PER_HOUR: 80G
|
|
||||||
SWWAF_BYTES_LIMIT_PER_DAY: 800G
|
|
||||||
# gitea never serves these; a request for one bans the client.
|
|
||||||
SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/
|
|
||||||
volumes:
|
|
||||||
- servewaf-state:/var/lib/smallwebwaf
|
|
||||||
networks:
|
|
||||||
- site
|
|
||||||
|
|
||||||
networks:
|
|
||||||
site:
|
|
||||||
traefik:
|
|
||||||
external: true
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
gitea-data:
|
|
||||||
servewaf-state:
|
|
||||||
```
|
|
||||||
|
|
||||||
`deploy/gitea` holds a complete setup of gitea, traefik included, as a Docker
|
|
||||||
Compose file. [`docs/gitea.md`](docs/gitea.md) explains it, and records what the
|
|
||||||
defaults do in front of gitea, request by request.
|
|
||||||
|
|
||||||
### `rproxy` beside the app
|
|
||||||
|
|
||||||
The app is built on the `smallwebwaf` image, with `rproxy` inside its container.
|
|
||||||
`app/` holds the `Dockerfile` and `app.run` from "`rproxy` in the app's
|
|
||||||
container". `rproxy` answers `503` to every request while `servewaf` is down,
|
|
||||||
and `servewaf`:
|
|
||||||
|
|
||||||
- observes before it enforces;
|
|
||||||
- looks addresses up in a local database rather than GeoJS;
|
|
||||||
- sends alerts to ntfy;
|
|
||||||
- reads its ntfy token from a file.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
# A Dockerfile built FROM the smallwebwaf image, with rproxy inside.
|
|
||||||
build: ./app
|
|
||||||
environment:
|
|
||||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
|
||||||
# Answer 503 rather than pass a request on while servewaf is down.
|
|
||||||
SWRP_DEFAULT_DISPOSITION: deny
|
|
||||||
ports:
|
|
||||||
- "8080:8080"
|
|
||||||
|
|
||||||
servewaf:
|
|
||||||
image: smallwebwaf
|
|
||||||
environment:
|
|
||||||
SMALLWEBWAF_COMMAND: servewaf
|
|
||||||
SWWAF_INSTANCE_NAME: fsn1/servewaf
|
|
||||||
# Refuse nothing yet; log what enforce mode would have done.
|
|
||||||
SWWAF_MODE: observe
|
|
||||||
# Look addresses up in a local IPinfo Lite database, never GeoJS.
|
|
||||||
SWWAF_LOOKUP_SOURCE: file
|
|
||||||
SWWAF_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
|
|
||||||
# Alerts to an ntfy topic, with its token read from a file.
|
|
||||||
SWWAF_ALERT_NTFY_URL: https://ntfy.example/smallwebwaf
|
|
||||||
SWWAF_ALERT_NTFY_TOKEN_FILE: /etc/smallwebwaf/tokens/ntfy
|
|
||||||
SWWAF_ALERT_EVENTS: ban,permanent_ban,waf_block,source_failure,file_error
|
|
||||||
volumes:
|
|
||||||
- servewaf-state:/var/lib/smallwebwaf
|
|
||||||
- /srv/ipinfo:/var/lib/ipinfo:ro
|
|
||||||
- /srv/smallwebwaf/tokens:/etc/smallwebwaf/tokens:ro
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
servewaf-state:
|
|
||||||
```
|
|
||||||
|
|
||||||
The database is mounted by its directory, not by file, so that a replaced
|
|
||||||
database is seen. A token file belongs to uid 65532, the `smallwebwaf` user,
|
|
||||||
with mode `0400`. This one holds an access token the ntfy server issued, which
|
|
||||||
`ntfy token add <user>` prints there:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
mkdir -p /srv/smallwebwaf/tokens
|
|
||||||
# <token> is the access token that ntfy printed.
|
|
||||||
echo '<token>' > /srv/smallwebwaf/tokens/ntfy
|
|
||||||
chown 65532:65532 /srv/smallwebwaf/tokens/ntfy
|
|
||||||
chmod 0400 /srv/smallwebwaf/tokens/ntfy
|
|
||||||
```
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
How it works inside. [`SPEC.md`](SPEC.md) gives every rule in full, and why.
|
|
||||||
|
|
||||||
### What happens to a request
|
|
||||||
|
|
||||||
A request goes through these steps, in this order. Once `servewaf` refuses a
|
|
||||||
request, the steps after that are skipped, apart from the report, and nothing of
|
|
||||||
the request reaches the app.
|
|
||||||
|
|
||||||
`rproxy` takes the first two:
|
`rproxy` takes the first two:
|
||||||
|
|
||||||
1. The health check, `GET /_smallwebwaf/healthz`, is answered at once.
|
1. The health check, `GET /_smallwebwaf/healthz`, is answered at once.
|
||||||
2. The visitor's address is worked out from `X-Forwarded-For`, believed only
|
2. The visitor's address is worked out, and `servewaf` is asked about the
|
||||||
from `SWRP_TRUSTED_PROXIES`. `rproxy` sends `servewaf` the request as the
|
request.
|
||||||
visitor sent it, with the first 64 KiB of its body.
|
|
||||||
|
|
||||||
`servewaf` takes the checks:
|
`servewaf` takes the checks:
|
||||||
|
|
||||||
@@ -599,8 +232,7 @@ the request reaches the app.
|
|||||||
5. The lookup of the client's AS number and country.
|
5. The lookup of the client's AS number and country.
|
||||||
6. The country lists.
|
6. The country lists.
|
||||||
7. The reputation sources.
|
7. The reputation sources.
|
||||||
8. The client's limit percentage, from its AS number, its country or a
|
8. The client's limit percentage.
|
||||||
reputation hit.
|
|
||||||
9. The rate limits.
|
9. The rate limits.
|
||||||
10. The trap paths.
|
10. The trap paths.
|
||||||
11. The rule files.
|
11. The rule files.
|
||||||
@@ -610,113 +242,85 @@ the request reaches the app.
|
|||||||
|
|
||||||
`rproxy` takes the rest, with `servewaf` counting at the end:
|
`rproxy` takes the rest, with `servewaf` counting at the end:
|
||||||
|
|
||||||
14. The request is passed to the app, streaming.
|
14. The request is passed to the app.
|
||||||
15. After the answer, `rproxy` sends `servewaf` a report with the status and the
|
15. After the answer, `rproxy` sends `servewaf` a report. For an allowed
|
||||||
bytes each way. For an allowed request, `servewaf` counts the bytes toward
|
request, `servewaf` counts the byte limits and adds the request to the
|
||||||
the byte limits and adds the request to the client's history.
|
client's history.
|
||||||
|
|
||||||
A refused visitor is answered with `SWWAF_BAN_RESPONSE`. A `block` rule or a
|
A refused visitor is answered with `SWWAF_BAN_RESPONSE`, `403` by default (see
|
||||||
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
|
"Ban lengths, response and scope" in [`SPEC.md`](SPEC.md)). A `block` rule or a
|
||||||
false positives; such refusals count toward `SWWAF_ERROR_BURST_THRESHOLD`
|
Core Rule Set match answers `403`. `rproxy` itself answers `405` to a method it
|
||||||
instead. A request over the header limits of step 12 is answered `431`, and bans
|
does not pass on, and the status for a size or time limit, such as `413` for a
|
||||||
nobody either. `rproxy` itself answers `405` to a method it does not pass on,
|
body too large (see "`rproxy`" in [`SPEC.md`](SPEC.md)).
|
||||||
and the status for a size or time limit, such as `413` for a body too large.
|
|
||||||
|
|
||||||
### Clients and bans
|
## Settings you are likely to touch
|
||||||
|
|
||||||
- A client is one IPv4 address or one IPv6 netblock of
|
Each setting is an environment variable, read at start. Those of `servewaf`
|
||||||
`SWWAF_IPV6_GROUP_PREFIX`, since one abuser usually controls a whole /64.
|
start with `SWWAF_`, and those of `rproxy` with `SWRP_`.
|
||||||
- Each limit is counted over a sliding window with a few integers per client.
|
[`docs/settings.md`](docs/settings.md) lists every setting, its default and the
|
||||||
The table of clients and the table of bans each have a bound
|
values it takes. The ones most deployments touch:
|
||||||
(`SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_MAX_BANS`), so memory is bounded too.
|
|
||||||
- A ban refuses every request from its netblock until it ends. Requests refused
|
|
||||||
under a ban count toward no limit. A permanent ban ends only when an admin
|
|
||||||
lifts it.
|
|
||||||
- Every ban carries notes: the client's AS number and country, the limit it
|
|
||||||
broke or the rule it matched, the request that caused it, and its earlier
|
|
||||||
bans. An admin adds, keeps or lifts a ban by editing `bans.json` or through
|
|
||||||
the ban endpoints (see [`docs/admin-endpoints.md`](docs/admin-endpoints.md)).
|
|
||||||
|
|
||||||
### Attack detection
|
- `SWWAF_MODE=observe`: `servewaf` refuses nothing and makes no ban of its own.
|
||||||
|
A request it would have refused is passed to the app, and its log line says
|
||||||
|
what `enforce` mode would have done. For trying a configuration before
|
||||||
|
enforcing it. The limits of `rproxy` still apply.
|
||||||
|
- `SWWAF_ALLOW_NETS`: netblocks you trust, such as your monitoring. Their
|
||||||
|
clients skip `servewaf`'s checks.
|
||||||
|
- `SWWAF_RATE_LIMIT_EXEMPT_NETS`: netblocks exempt from rate limiting and the
|
||||||
|
byte limits, such as a machine that talks to the app all day.
|
||||||
|
- `SWWAF_TRAP_PATHS`: paths only scanners ask for, such as
|
||||||
|
`/wp-login.php,/xmlrpc.php` in front of a code forge. A request for one bans
|
||||||
|
the client.
|
||||||
|
- `SWWAF_WAF_DISABLED_RULES`: Core Rule Set rule ids to switch off when a rule
|
||||||
|
refuses your app's ordinary requests. The log line names the rules a request
|
||||||
|
matched. Keep the six default ids in a list you give (see
|
||||||
|
[`docs/settings.md`](docs/settings.md)).
|
||||||
|
- `SWRP_MAX_REQUEST_SIZE` and `SWRP_MAX_REQUEST_TIME_MS`: raise them for an app
|
||||||
|
that takes uploads over 100 MiB or slower than 60 seconds.
|
||||||
|
- `SWRP_DEFAULT_DISPOSITION`: `allow` (the default) or `deny`, as above.
|
||||||
|
- `SWRP_TRUSTED_PROXIES`: the netblocks whose `X-Forwarded-For` `rproxy`
|
||||||
|
believes, by default the private address ranges.
|
||||||
|
- `SWWAF_LOOKUP_SOURCE`: `geojs` (the default), `file` or `off`, as above.
|
||||||
|
- `SWWAF_ADMIN_TOKEN` and `SWWAF_METRICS_TOKEN`: switch on the ban endpoints and
|
||||||
|
the metrics, at `servewaf`'s own address.
|
||||||
|
- `SWWAF_ALERT_WEBHOOK_URL`, `SWWAF_ALERT_SLACK_WEBHOOK_URL` or
|
||||||
|
`SWWAF_ALERT_NTFY_URL`: where alerts go.
|
||||||
|
- `SWWAF_INSTANCE_NAME`: the name in every line `servewaf` logs, and in every
|
||||||
|
alert and metric, such as `fsn1/servewaf`.
|
||||||
|
|
||||||
- The Core Rule Set reads the method, the URL with its query string, and the
|
## Checking that it works
|
||||||
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
|
|
||||||
front of a code forge, bodies carry what people write and publish, which the
|
|
||||||
rules take for attacks.
|
|
||||||
- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary
|
|
||||||
requests to a code forge through, such as git over HTTP and sign-in through
|
|
||||||
Git Credential Manager. No setting undoes them; "Attack detection" in
|
|
||||||
[`SPEC.md`](SPEC.md) lists them.
|
|
||||||
- Rule files are plain text, one rule to a line, edited by hand and read again
|
|
||||||
while `servewaf` runs. The image ships one file of probes no real visitor
|
|
||||||
sends (see [`docs/rules.md`](docs/rules.md)).
|
|
||||||
|
|
||||||
### State files
|
- `docker inspect --format '{{.State.Health.Status}}' <container>` says
|
||||||
|
`healthy` for each container of `smallwebwaf`:
|
||||||
`servewaf` keeps everything in memory, and the five JSON files in
|
- for `rproxy`, once it answers and the app accepts connections, whether or
|
||||||
`SWWAF_STATE_DIR` hold a copy: `bans.json`, `clients.json`, `lookups.json`,
|
not `servewaf` answers;
|
||||||
`reputation.json` and `alerts.json`. Each is written whole, to a temporary file
|
- for `servewaf`, once it answers.
|
||||||
renamed over the real one, so a crash leaves a complete file. `servewaf` notices
|
- A log line with `"msg":"starting"` gives the version, the address and every
|
||||||
a file edited on disk and takes the edit in; an edit that does not parse is set
|
setting in effect, with secrets masked.
|
||||||
aside as `<name>.bad`, and never stops the running process (see
|
- A request to the site through traefik gets the app's answer.
|
||||||
[`docs/state-files.md`](docs/state-files.md)).
|
`https://<site>/_smallwebwaf/healthz` answers `ok`. `servewaf`'s log lines
|
||||||
|
about the request have `"action":"forward"`.
|
||||||
### The image and its processes
|
- To see a refusal without banning yourself, run `servewaf` with
|
||||||
|
`SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app.
|
||||||
- The image is built on Ubuntu 26.04 LTS, pinned by digest, with its packages
|
`servewaf`'s log lines have `"would_action":"banned"` and
|
||||||
from a snapshot of Ubuntu's archive and nixpkgs fixed at one release. An app's
|
`"rule_ids":["env-file"]`. In `enforce` mode the same request bans your
|
||||||
`Dockerfile` installs a package with `nix-env -iA nixpkgs.<name>` and gets the
|
address for seven days, and your next request makes the ban permanent. Lift
|
||||||
same package each time it is built (see [`docs/image.md`](docs/image.md)).
|
the ban by editing `bans.json` (see "Bans" in [`SPEC.md`](SPEC.md)).
|
||||||
- runit, a service supervisor, runs `smallwebwaf` as the `smallwebwaf` user (uid
|
- With `SWWAF_METRICS_TOKEN` set, this answers with the metrics, from a host on
|
||||||
and gid 65532) and starts it again when it exits. The `run` script of
|
`servewaf`'s network:
|
||||||
`servewaf` gives the state directory to the `smallwebwaf` user at each start,
|
`curl -H "Authorization: Bearer $TOKEN" http://servewaf:8080/_smallwebwaf/metrics`.
|
||||||
so a volume needs no change of owner.
|
- `make example-app` builds the image and the example app on it, runs the app
|
||||||
- `rproxy` and `servewaf` use no token between them. `servewaf`'s own endpoints,
|
with `rproxy` in its container and `servewaf` in another, and checks them. It
|
||||||
under `/_smallwebwaf/`, are for your admin tools and your metrics collector,
|
needs network access. It checks:
|
||||||
at its own address on the private network; nothing routes the internet there.
|
- the health checks;
|
||||||
- `docker stop` gives requests in progress five seconds to finish. With `rproxy`
|
- a request passed to the app;
|
||||||
in the app's container, the app must keep answering for five seconds after
|
- a ban for a broken limit, and a ban for a probe made permanent;
|
||||||
`SIGTERM` for that to help them.
|
- that only `servewaf`'s container changes the state directory's owner;
|
||||||
- Only faults found at start stop a process: an invalid setting, a rule file or
|
- with `servewaf` stopped, the app still answering through `rproxy`;
|
||||||
state file that does not parse, a lookup database that cannot be read, or an
|
- with `servewaf` started again, bans applying again;
|
||||||
address it cannot listen on. Once running, a failing lookup service,
|
- orderly stops with `sv stop` and `docker stop`;
|
||||||
reputation source or alert destination never takes `servewaf` down.
|
- bans surviving a new container on the same volume;
|
||||||
|
- a setting given as a file.
|
||||||
### Packages
|
|
||||||
|
|
||||||
- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`.
|
|
||||||
- `internal/smallwebwaf`: the process, as the command it is given, and the
|
|
||||||
health check.
|
|
||||||
- `internal/config`: reads the settings, the one place they are read.
|
|
||||||
- `internal/rproxy`: the reverse proxy in front of one app.
|
|
||||||
- `internal/proxy`: `servewaf`'s server, which runs the checks and answers its
|
|
||||||
own endpoints.
|
|
||||||
- `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`,
|
|
||||||
`internal/lookup`, `internal/reputation`, `internal/anomaly` and
|
|
||||||
`internal/alerts`: one check or source each.
|
|
||||||
- `internal/state`, `internal/requestlog`, `internal/remotelog` and
|
|
||||||
`internal/metrics`: the state files, the log lines, the syslog sender and the
|
|
||||||
metrics.
|
|
||||||
|
|
||||||
## Rationale
|
|
||||||
|
|
||||||
Small self-hosted sites receive a great deal of traffic nobody asked for:
|
|
||||||
|
|
||||||
- scrapers that ignore `robots.txt` and crawl every commit of every repository
|
|
||||||
on a public git server;
|
|
||||||
- scanners walking through lists of WordPress and `.env` paths;
|
|
||||||
- credential-guessing bots.
|
|
||||||
|
|
||||||
Most of it comes from a small number of hosting networks and countries. A
|
|
||||||
single-person operation has no abuse desk and no CDN contract. It needs
|
|
||||||
something small that can be put in front of its services and left alone.
|
|
||||||
|
|
||||||
The existing tools each solve part of this. Rule-based firewalls catch attack
|
|
||||||
payloads but do not limit request rates. Rate limiters count requests but cannot
|
|
||||||
tell a residential visitor from a rented server farm. The products that do most
|
|
||||||
of it want a database and a web console. None of them can say "clients from
|
|
||||||
these networks are banned after half as many requests as anyone else".
|
|
||||||
[`EVALUATION.md`](EVALUATION.md) goes through the candidates one by one.
|
|
||||||
|
|
||||||
## Entrypoints
|
## Entrypoints
|
||||||
|
|
||||||
@@ -764,32 +368,83 @@ on the host.
|
|||||||
- `servewaf` listens on port 8090;
|
- `servewaf` listens on port 8090;
|
||||||
- its state files go in `bin/state`;
|
- its state files go in `bin/state`;
|
||||||
- its rule files are those of `share/rules.d`.
|
- its rule files are those of `share/rules.d`.
|
||||||
- `script/example-app`: builds the image and the example app
|
- `script/example-app`: builds the image and the example app on it, runs them
|
||||||
(`deploy/example-app`) on it, runs the app with `rproxy` in its container and
|
with a volume, and checks them as "Checking that it works" above describes.
|
||||||
`servewaf` in another, and checks them. Then it removes the containers, the
|
Then it removes the containers, the network, the volumes and the images. Needs
|
||||||
network, the volumes and the images. Needs network access. `make example-app`.
|
network access. `make example-app`.
|
||||||
It checks:
|
|
||||||
- the health checks;
|
|
||||||
- a request passed to the app;
|
|
||||||
- a ban for a broken limit, and a ban for a probe made permanent;
|
|
||||||
- that only `servewaf`'s container changes the state directory's owner;
|
|
||||||
- with `servewaf` stopped, the app still answering through `rproxy`;
|
|
||||||
- with `servewaf` started again, bans applying again;
|
|
||||||
- orderly stops with `sv stop` and `docker stop`;
|
|
||||||
- bans surviving a new container on the same volume;
|
|
||||||
- a setting given as a file.
|
|
||||||
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind
|
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind
|
||||||
`rproxy` and `servewaf`, three containers on one network. Every setting is at
|
`rproxy` and `servewaf`, three containers on one network. Every setting is at
|
||||||
its default, but for those that join the containers and
|
its default, but for those that join the containers and the header that labels
|
||||||
`SWWAF_LOG_REQUEST_HEADERS`, which adds the header that labels each request.
|
each request. It sends through them a sample of gitea's ordinary use and the
|
||||||
It sends through them a sample of gitea's ordinary use and the requests the
|
requests the Core Rule Set is known to refuse, and prints what `servewaf` did
|
||||||
Core Rule Set is known to refuse, and prints what `servewaf` did with each.
|
with each. Needs network access, and a docker daemon that pushes to a registry
|
||||||
Needs network access, and a docker daemon that pushes to a registry on
|
on `localhost` over plain HTTP. [`docs/gitea.md`](docs/gitea.md) records the
|
||||||
`localhost` over plain HTTP. `make gitea-requests`.
|
results. `make gitea-requests`.
|
||||||
- `script/gitea-setup`: builds the image, brings up the setup in `deploy/gitea`,
|
|
||||||
and checks it as [`docs/gitea.md`](docs/gitea.md) describes. Then it removes
|
## Rationale
|
||||||
the containers, the network, the volumes and the image it built. Needs network
|
|
||||||
access. `make gitea-setup`.
|
Small self-hosted sites receive a great deal of traffic nobody asked for:
|
||||||
|
|
||||||
|
- scrapers that ignore `robots.txt` and crawl every commit of every repository
|
||||||
|
on a public git server;
|
||||||
|
- scanners walking through lists of WordPress and `.env` paths;
|
||||||
|
- credential-guessing bots.
|
||||||
|
|
||||||
|
Most of it comes from a small number of hosting networks and countries. A
|
||||||
|
single-person operation has no abuse desk and no CDN contract. It needs
|
||||||
|
something small that can be put in front of its services and left alone.
|
||||||
|
|
||||||
|
The existing tools each solve part of this. Rule-based firewalls catch attack
|
||||||
|
payloads but do not limit request rates. Rate limiters count requests but cannot
|
||||||
|
tell a residential visitor from a rented server farm. The products that do most
|
||||||
|
of it want a database and a web console. None of them can say "clients from
|
||||||
|
these networks are banned after half as many requests as anyone else". That is
|
||||||
|
the useful thing to say when nearly all abuse comes from a known list of AS
|
||||||
|
numbers. [`EVALUATION.md`](EVALUATION.md) goes through the candidates one by
|
||||||
|
one.
|
||||||
|
|
||||||
|
`smallwebwaf` fills that gap with:
|
||||||
|
|
||||||
|
- per-client limits that ban;
|
||||||
|
- lower limits for the networks and countries abuse comes from;
|
||||||
|
- bans that grow with repetition;
|
||||||
|
- a log that is easy to search and ship;
|
||||||
|
- a site that keeps answering while the part that decides is down;
|
||||||
|
- one binary, with no database and one required setting.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
The `smallwebwaf` image is built on Ubuntu 26.04 LTS, pinned by digest, with
|
||||||
|
nixpkgs fixed at one release. An app's `Dockerfile` installs a package with
|
||||||
|
`nix-env -iA nixpkgs.<name>` and gets the same package each time it is built.
|
||||||
|
[`docs/image.md`](docs/image.md) lists what the image holds and how each part is
|
||||||
|
pinned.
|
||||||
|
|
||||||
|
- runit, a service supervisor, runs `smallwebwaf` and starts it again when it
|
||||||
|
exits. In an app's image built on the `smallwebwaf` image, it runs `rproxy`
|
||||||
|
and the app side by side, each as its own user.
|
||||||
|
- `rproxy` listens on port 8080 and passes requests to the app at
|
||||||
|
`SWRP_UPSTREAM_URL`. It answers the health check itself, and the refusals and
|
||||||
|
errors "`rproxy`" in [`SPEC.md`](SPEC.md) lists.
|
||||||
|
- `servewaf` listens on port 8080 of its own container. Nothing should route the
|
||||||
|
internet there. Its own endpoints, under `/_smallwebwaf/`, are for your admin
|
||||||
|
tools and your metrics collector.
|
||||||
|
- `rproxy` and `servewaf` use no token between them. Anything on their private
|
||||||
|
network can ask `servewaf` a question, or send it a report.
|
||||||
|
- The app sees the headers traefik sent, with traefik's address added to
|
||||||
|
`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
|
||||||
|
`rproxy`'s address for forwarded headers. With `rproxy` in the app's
|
||||||
|
container, that address is `127.0.0.1` or `::1`, which the private address
|
||||||
|
ranges leave out.
|
||||||
|
- The state files live in `/var/lib/smallwebwaf` in `servewaf`'s container. A
|
||||||
|
volume mounted there keeps them across a deploy.
|
||||||
|
- `docker stop` gives requests in progress five seconds to finish. With `rproxy`
|
||||||
|
in the app's container, the app must keep answering for five seconds after
|
||||||
|
`SIGTERM` for that to help them.
|
||||||
|
|
||||||
|
"Deployment" in [`SPEC.md`](SPEC.md) has the rest: the health checks, what
|
||||||
|
happens when a process stops, moving a port, and the forwarded headers.
|
||||||
|
"Architecture" in [`SPEC.md`](SPEC.md) lists the packages and what each does.
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
|
|
||||||
@@ -802,7 +457,8 @@ on the host.
|
|||||||
- [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers
|
- [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers
|
||||||
and misses, and why none was adopted.
|
and misses, and why none was adopted.
|
||||||
- [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows.
|
- [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows.
|
||||||
- [`docs/settings.md`](docs/settings.md): every setting and the values it takes.
|
- [`docs/settings.md`](docs/settings.md): every setting, its default and the
|
||||||
|
values it takes.
|
||||||
- [`docs/rules.md`](docs/rules.md): the rule file format and the default file.
|
- [`docs/rules.md`](docs/rules.md): the rule file format and the default file.
|
||||||
- [`docs/admin-endpoints.md`](docs/admin-endpoints.md): the endpoints under
|
- [`docs/admin-endpoints.md`](docs/admin-endpoints.md): the endpoints under
|
||||||
`/_smallwebwaf/`, with examples.
|
`/_smallwebwaf/`, with examples.
|
||||||
@@ -813,8 +469,7 @@ on the host.
|
|||||||
- [`docs/state-files.md`](docs/state-files.md): the state files and what each
|
- [`docs/state-files.md`](docs/state-files.md): the state files and what each
|
||||||
holds.
|
holds.
|
||||||
- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
|
- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
|
||||||
- [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea: the setup
|
- [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea.
|
||||||
in `deploy/gitea`, and what the defaults do there.
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -1,127 +0,0 @@
|
|||||||
# Gitea behind traefik, with rproxy and servewaf in front of it.
|
|
||||||
# docs/gitea.md says what each setting here does and why. Before starting it:
|
|
||||||
# - set SMALLWEBWAF_IMAGE to the smallwebwaf image, named by digest;
|
|
||||||
# - replace git.example.org with the site's host name;
|
|
||||||
# - replace each 10.99.0 with the first three numbers of a /24 subnet that
|
|
||||||
# no other network on the host uses. Replace each fd10:99 with the first
|
|
||||||
# two hextets of an unused /64 subnet whose third and fourth hextets are
|
|
||||||
# 0, as in fd10:99::/64.
|
|
||||||
# Then run `docker compose up --detach` in this directory.
|
|
||||||
# script/gitea-setup checks this setup.
|
|
||||||
|
|
||||||
name: gitea
|
|
||||||
|
|
||||||
x-logging: &logging
|
|
||||||
driver: json-file
|
|
||||||
options:
|
|
||||||
max-size: 10m
|
|
||||||
max-file: "5"
|
|
||||||
|
|
||||||
services:
|
|
||||||
traefik:
|
|
||||||
# traefik v3.7.17, 2026-11-03
|
|
||||||
image: traefik@sha256:68ae5fe5b64ca166704b746a9e8afcd385e36b3f569a5691401bf652d57b3748
|
|
||||||
# traefik replaces the X-Forwarded-For header a visitor sends. Keep it
|
|
||||||
# so: otherwise a visitor could name an address on the shared network,
|
|
||||||
# and skip servewaf's checks.
|
|
||||||
command:
|
|
||||||
- --providers.docker.exposedbydefault=false
|
|
||||||
- --entrypoints.web.address=:80
|
|
||||||
- --entrypoints.web.http.redirections.entrypoint.to=websecure
|
|
||||||
- --entrypoints.websecure.address=:443
|
|
||||||
# Until a certificate resolver is added here, as for your other
|
|
||||||
# sites, traefik answers with a certificate it made itself.
|
|
||||||
- --entrypoints.websecure.http.tls=true
|
|
||||||
# traefik's own limit of 60 seconds for a whole request would cut
|
|
||||||
# off an upload long before rproxy's limit of 30 minutes.
|
|
||||||
- --entrypoints.websecure.transport.respondingTimeouts.readTimeout=30m
|
|
||||||
ports:
|
|
||||||
- 80:80
|
|
||||||
- 443:443
|
|
||||||
volumes:
|
|
||||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
|
||||||
networks:
|
|
||||||
shared:
|
|
||||||
# Containers on the shared network find traefik under the
|
|
||||||
# site's name, and so reach it from their own address.
|
|
||||||
aliases:
|
|
||||||
- git.example.org
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
gitea:
|
|
||||||
# gitea 1.27.3, 2026-08-29
|
|
||||||
image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef
|
|
||||||
environment:
|
|
||||||
GITEA__server__ROOT_URL: https://git.example.org/
|
|
||||||
# Gitea starts at its defaults, with SQLite, instead of showing its
|
|
||||||
# install page to whoever opens it first.
|
|
||||||
GITEA__security__INSTALL_LOCK: "true"
|
|
||||||
# rproxy's addresses. Gitea then takes each visitor's address from
|
|
||||||
# the X-Real-Ip header traefik sets and rproxy passes on.
|
|
||||||
GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.99.0.2,fd10:99::2
|
|
||||||
volumes:
|
|
||||||
- gitea-data:/data
|
|
||||||
networks:
|
|
||||||
- shared
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
rproxy:
|
|
||||||
image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image}
|
|
||||||
environment:
|
|
||||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
|
||||||
SWRP_UPSTREAM_URL: http://gitea:3000
|
|
||||||
# Pushes, LFS objects, packages and release attachments of up to
|
|
||||||
# 2 GiB, sent in up to 30 minutes.
|
|
||||||
SWRP_MAX_REQUEST_SIZE: 2G
|
|
||||||
SWRP_MAX_REQUEST_TIME_MS: "1800000"
|
|
||||||
# 15 minutes for a clone or a download, half the default.
|
|
||||||
SWRP_MAX_RESPONSE_TIME_MS: "900000"
|
|
||||||
labels:
|
|
||||||
traefik.enable: "true"
|
|
||||||
traefik.http.routers.gitea.rule: Host(`git.example.org`)
|
|
||||||
traefik.http.routers.gitea.entrypoints: websecure
|
|
||||||
traefik.http.services.gitea.loadbalancer.server.port: "8080"
|
|
||||||
networks:
|
|
||||||
shared:
|
|
||||||
ipv4_address: 10.99.0.2
|
|
||||||
ipv6_address: fd10:99::2
|
|
||||||
logging: *logging
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
servewaf:
|
|
||||||
image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image}
|
|
||||||
environment:
|
|
||||||
SMALLWEBWAF_COMMAND: servewaf
|
|
||||||
# The containers on the shared network skip the checks. Add the
|
|
||||||
# address of an automation fleet on another host here too; without
|
|
||||||
# that, the default request limits are sized for such a fleet.
|
|
||||||
SWWAF_ALLOW_NETS: 10.99.0.128/25,fd10:99:0:0:8000::/65
|
|
||||||
# In block mode the Core Rule Set refuses everyday Gitea searches
|
|
||||||
# and file names, and 31 refusals in a minute ban the address.
|
|
||||||
SWWAF_WAF_MODE: detect
|
|
||||||
volumes:
|
|
||||||
- servewaf-state:/var/lib/smallwebwaf
|
|
||||||
networks:
|
|
||||||
- shared
|
|
||||||
logging: *logging
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
networks:
|
|
||||||
# The network traefik and Gitea share. Its containers get addresses in the
|
|
||||||
# upper half of each subnet, which servewaf lets skip the checks. Each
|
|
||||||
# subnet's gateway, the host's own address on it, stays out of that half:
|
|
||||||
# requests docker passes on from a published port can come from it.
|
|
||||||
shared:
|
|
||||||
enable_ipv6: true
|
|
||||||
ipam:
|
|
||||||
config:
|
|
||||||
- subnet: 10.99.0.0/24
|
|
||||||
ip_range: 10.99.0.128/25
|
|
||||||
gateway: 10.99.0.1
|
|
||||||
- subnet: fd10:99::/64
|
|
||||||
ip_range: fd10:99:0:0:8000::/65
|
|
||||||
gateway: fd10:99::1
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
gitea-data:
|
|
||||||
servewaf-state:
|
|
||||||
+42
-163
@@ -1,178 +1,51 @@
|
|||||||
# smallwebwaf in front of gitea
|
# smallwebwaf in front of gitea
|
||||||
|
|
||||||
`deploy/gitea/compose.yaml` runs gitea behind traefik, with `rproxy` and
|
These notes record what `smallwebwaf` does in front of gitea at the default
|
||||||
`servewaf` in front of it. This document says what that setup sets and why, and
|
settings. They are observations, not requirements; the requirements are in
|
||||||
what the defaults would do without it. [`SPEC.md`](../SPEC.md) says how
|
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind `rproxy` and
|
||||||
`smallwebwaf` behaves.
|
`servewaf`, and sends through them a sample of gitea's ordinary use and the
|
||||||
|
requests named below. It reports each request the Core Rule Set refuses, so that
|
||||||
|
these notes can be checked again when gitea or the Core Rule Set changes
|
||||||
|
version.
|
||||||
|
|
||||||
## The setup
|
## What the defaults cover
|
||||||
|
|
||||||
Four containers share one Docker network, the shared network:
|
|
||||||
|
|
||||||
- traefik, which routes the site's HTTPS requests to `rproxy`, and redirects
|
|
||||||
plain HTTP to HTTPS;
|
|
||||||
- `rproxy`, in front of gitea;
|
|
||||||
- gitea 1.27.3, as its image comes;
|
|
||||||
- `servewaf`, with a volume for its state files.
|
|
||||||
|
|
||||||
To start it:
|
|
||||||
|
|
||||||
1. In `deploy/gitea/compose.yaml`, replace `git.example.org` with the site's
|
|
||||||
host name.
|
|
||||||
2. Replace each `10.99.0` with the first three numbers of a `/24` subnet that no
|
|
||||||
other network on the host uses. Replace each `fd10:99` with the first two
|
|
||||||
hextets of an unused `/64` subnet whose third and fourth hextets are 0, as in
|
|
||||||
`fd10:99::/64`.
|
|
||||||
3. Set `SMALLWEBWAF_IMAGE` to the `smallwebwaf` image, named by digest.
|
|
||||||
4. Run `docker compose up --detach` in `deploy/gitea`.
|
|
||||||
5. Make gitea's first user with `gitea admin user create`, in gitea's container,
|
|
||||||
as the user `git`.
|
|
||||||
|
|
||||||
gitea starts at its defaults, with SQLite, instead of showing its install page
|
|
||||||
to whoever opens it first. traefik answers with a certificate it made itself
|
|
||||||
until you add a certificate resolver, as for your other sites. SSH to gitea does
|
|
||||||
not pass through `smallwebwaf`; publish gitea's SSH port yourself if you use it.
|
|
||||||
|
|
||||||
`make gitea-setup` checks the setup. It builds the image, brings the setup up
|
|
||||||
under a name of its own, and checks:
|
|
||||||
|
|
||||||
- that a page load, a push and a clone sent through traefik pass through
|
|
||||||
`rproxy`, and that `servewaf` checks them;
|
|
||||||
- that gitea logs the visitor's address, not `rproxy`'s;
|
|
||||||
- that a request from a container on the shared network skips the checks;
|
|
||||||
- that with `servewaf` stopped, gitea still answers through `rproxy`.
|
|
||||||
|
|
||||||
## Uploads and downloads
|
|
||||||
|
|
||||||
An upload is a request:
|
|
||||||
|
|
||||||
|
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
|
||||||
|
by it.
|
||||||
|
- A clone is a response. It fits the defaults of 30 minutes and 1 GiB unless the
|
||||||
|
repository is large or the link slow. A repository whose clone is larger than
|
||||||
|
1 GiB needs `SWRP_MAX_RESPONSE_SIZE` raised, and the byte limits with it
|
||||||
|
("Byte limits, per client" in `SPEC.md`).
|
||||||
|
- A push is a request, and so is every other upload:
|
||||||
- a git push;
|
- a git push;
|
||||||
- an LFS object;
|
- an LFS object;
|
||||||
- a container image layer;
|
- a container image layer;
|
||||||
- a package file;
|
- a package file;
|
||||||
- a release attachment.
|
- a release attachment.
|
||||||
|
|
||||||
The setup lets an upload of up to 2 GiB, sent in up to 30 minutes, through:
|
At the defaults, an upload larger than 100 MiB or taking more than 60
|
||||||
|
seconds is cut off ("`rproxy`'s limits" in `SPEC.md`). A gitea that takes
|
||||||
|
large uploads needs `SWRP_MAX_REQUEST_SIZE` and `SWRP_MAX_REQUEST_TIME_MS`
|
||||||
|
raised to fit.
|
||||||
|
|
||||||
- `SWRP_MAX_REQUEST_SIZE` is `2G`. At the default, an upload over 100 MiB is cut
|
- Archive download and blame or history pages are what scrapers hammer. The
|
||||||
off.
|
request limits do most of the work there.
|
||||||
- `SWRP_MAX_REQUEST_TIME_MS` is `1800000`, 30 minutes. At the default, an upload
|
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
|
||||||
that takes over 60 seconds is cut off.
|
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
|
||||||
- traefik's `readTimeout` is 30 minutes. traefik's own default of 60 seconds for
|
43,200 requests a day. While it runs a job, it reports the job's log and state
|
||||||
a whole request would cut off an upload first.
|
every second instead, two requests a second. So one runner busy all day sends
|
||||||
|
120 requests a minute and 172,800 a day, under the request limits. Two such
|
||||||
A clone or a download is a response:
|
runners on one address, each busy all day, make 345,600 and pass the day limit
|
||||||
|
of 300,000. Three busy at once make 21,600 an hour and pass the hour limit of
|
||||||
- `SWRP_MAX_RESPONSE_TIME_MS` is `900000`, 15 minutes, half the default. A clone
|
20,000. Five busy at once make 600 a minute and pass the minute limit of 500.
|
||||||
or download that takes longer is cut off.
|
Their address is then banned, their running jobs fail, and each repeat within
|
||||||
- `SWRP_MAX_RESPONSE_SIZE` stays at its default, 1 GiB. A repository whose clone
|
a day makes the ban longer. Newer runners ask for work less often. Put the
|
||||||
is larger needs it raised, and the byte limits with it ("Byte limits, per
|
|
||||||
client" in `SPEC.md`).
|
|
||||||
|
|
||||||
The byte limits stay at their defaults. Two uploads of 2 GiB from one address in
|
|
||||||
one minute pass the minute's limit of 4 GiB, and ban the address.
|
|
||||||
|
|
||||||
## The checks
|
|
||||||
|
|
||||||
`SWWAF_WAF_MODE` is `detect`. The Core Rule Set then logs a match, with its rule
|
|
||||||
ids, and refuses nothing. In `block` mode, the default, it refuses everyday
|
|
||||||
gitea searches and file names. The section "What the Core Rule Set refuses in
|
|
||||||
`block` mode" below lists them. Each such refusal counts toward the error burst,
|
|
||||||
so 31 in one minute ban the address.
|
|
||||||
|
|
||||||
Every other check stays at its default. A client may make 500 requests a minute,
|
|
||||||
20,000 an hour and 300,000 a day. Archive downloads and blame and history pages
|
|
||||||
are what scrapers hammer, and these limits do most of the work there.
|
|
||||||
|
|
||||||
## The shared network
|
|
||||||
|
|
||||||
Containers on the shared network skip `servewaf`'s checks:
|
|
||||||
|
|
||||||
- The shared network's subnets are `10.99.0.0/24` and `fd10:99::/64`. Docker
|
|
||||||
gives its containers addresses in the upper half of each, `10.99.0.128/25` and
|
|
||||||
`fd10:99:0:0:8000::/65`, which `SWWAF_ALLOW_NETS` names.
|
|
||||||
- traefik is known on the shared network by the site's host name. So a container
|
|
||||||
there that asks for the site reaches traefik from its own address. An
|
|
||||||
automation fleet in such a container skips the checks.
|
|
||||||
- Each subnet's gateway, `10.99.0.1` and `fd10:99::1`, is the host's own address
|
|
||||||
on it. Requests docker passes on from a published port can come from the
|
|
||||||
gateway, so the gateways are left out of `SWWAF_ALLOW_NETS`. The network has
|
|
||||||
IPv6, so a visitor over IPv6 reaches traefik from its own address, and
|
|
||||||
servewaf checks it as itself.
|
|
||||||
- A container on the shared network that asks for the site at the host's public
|
|
||||||
address arrives from the gateway. It is checked like any visitor.
|
|
||||||
- A fleet on another host can have its address added to `SWWAF_ALLOW_NETS`.
|
|
||||||
Without that, the default request limits are sized for such a fleet.
|
|
||||||
- traefik replaces the `X-Forwarded-For` a visitor sends, by default. Keep it
|
|
||||||
so: otherwise a visitor could name an address on the shared network, and skip
|
|
||||||
the checks.
|
|
||||||
- Anything on the shared network can reach gitea and `servewaf` around `rproxy`.
|
|
||||||
Put only containers you trust on it.
|
|
||||||
|
|
||||||
## gitea's settings
|
|
||||||
|
|
||||||
- `REVERSE_PROXY_TRUSTED_PROXIES` names `rproxy`'s fixed addresses, `10.99.0.2`
|
|
||||||
and `fd10:99::2`. gitea then takes each visitor's address from the `X-Real-Ip`
|
|
||||||
header, which traefik sets and `rproxy` passes on. Without it, gitea logs
|
|
||||||
every visitor as `rproxy`.
|
|
||||||
- `INSTALL_LOCK` is `true`, as "The setup" above says.
|
|
||||||
|
|
||||||
## Methods
|
|
||||||
|
|
||||||
`rproxy` passes on only these methods: `GET`, `HEAD`, `POST`, `OPTIONS`, `PUT`,
|
|
||||||
`DELETE`, `PATCH` and `QUERY`. gitea 1.27.3 needs no other. Its routes use:
|
|
||||||
|
|
||||||
- pages and the API: `GET`, `HEAD`, `POST`, `PUT`, `PATCH` and `DELETE`;
|
|
||||||
- git's push, clone and fetch over HTTP: `GET`, `POST` and `OPTIONS`;
|
|
||||||
- LFS: `GET`, `POST` and `PUT`;
|
|
||||||
- the container registry and the other package registries: `GET`, `HEAD`,
|
|
||||||
`POST`, `PUT`, `PATCH` and `DELETE`;
|
|
||||||
- Actions runners: `POST`.
|
|
||||||
|
|
||||||
## The log
|
|
||||||
|
|
||||||
`rproxy` and `servewaf` write each query string to the log whole. A token sent
|
|
||||||
in the query string, as `?token=`, is then in the log. Send it in the
|
|
||||||
`Authorization` header instead.
|
|
||||||
|
|
||||||
## Actions runners
|
|
||||||
|
|
||||||
A runner on the shared network skips the checks. One on another host is counted
|
|
||||||
like any client. A runner older than version 0.4 (April 2026) asks for work
|
|
||||||
every 2 seconds, 43,200 requests a day. While it runs a job, it reports the
|
|
||||||
job's log and state every second instead, two requests a second. So one runner
|
|
||||||
busy all day sends 120 requests a minute and 172,800 a day, under the request
|
|
||||||
limits.
|
|
||||||
|
|
||||||
Several runners on one address can pass them:
|
|
||||||
|
|
||||||
- Two busy all day make 345,600 requests, and pass the day limit of 300,000.
|
|
||||||
- Three busy at once make 21,600 an hour, and pass the hour limit of 20,000.
|
|
||||||
- Five busy at once make 600 a minute, and pass the minute limit of 500.
|
|
||||||
|
|
||||||
Their address is then banned, their running jobs fail, and each repeat within a
|
|
||||||
day makes the ban longer. Newer runners ask for work less often. Put the
|
|
||||||
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||||
|
|
||||||
## What the Core Rule Set refuses in `block` mode
|
## What the Core Rule Set lets through
|
||||||
|
|
||||||
These notes record what the Core Rule Set does in front of gitea in `block`
|
At the defaults, the Core Rule Set lets gitea's ordinary use through, apart from
|
||||||
mode, at the other defaults. They are observations, not requirements.
|
the refusals in the next section:
|
||||||
`script/gitea-requests` runs gitea behind `rproxy` and `servewaf` at the
|
|
||||||
defaults. It sends through them a sample of gitea's ordinary use and the
|
|
||||||
requests named below. It reports each request the Core Rule Set refuses, so that
|
|
||||||
these notes can be checked again when gitea or the Core Rule Set changes
|
|
||||||
version. In `detect` mode, as the setup runs it, the same requests are logged
|
|
||||||
with their rule ids, and pass.
|
|
||||||
|
|
||||||
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
|
|
||||||
toward). The request log names the rule in `waf_rule_ids`, which
|
|
||||||
`SWWAF_WAF_DISABLED_RULES` can switch off.
|
|
||||||
|
|
||||||
### What it lets through
|
|
||||||
|
|
||||||
The Core Rule Set lets gitea's ordinary use through, apart from the refusals in
|
|
||||||
the sections after this one:
|
|
||||||
|
|
||||||
- browsing and views of files in a repository, with their history, blame and the
|
- browsing and views of files in a repository, with their history, blame and the
|
||||||
file tree;
|
file tree;
|
||||||
@@ -187,6 +60,12 @@ the sections after this one:
|
|||||||
editor, code included, since no body is read;
|
editor, code included, since no body is read;
|
||||||
- the page gitea shows after a change, with its message naming what was done.
|
- the page gitea shows after a change, with its message naming what was done.
|
||||||
|
|
||||||
|
## What the Core Rule Set still refuses
|
||||||
|
|
||||||
|
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
|
||||||
|
toward). The request log names the rule in `waf_rule_ids`, which
|
||||||
|
`SWWAF_WAF_DISABLED_RULES` can switch off.
|
||||||
|
|
||||||
### A query string that reads as an attack
|
### A query string that reads as an attack
|
||||||
|
|
||||||
Most often this is a search. A search term is a query parameter, which the Core
|
Most often this is a search. A search term is a query parameter, which the Core
|
||||||
@@ -304,7 +183,7 @@ the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
|
|||||||
than 30 such refusals from its address in a minute, as when many jobs upload at
|
than 30 such refusals from its address in a minute, as when many jobs upload at
|
||||||
once. `actions/upload-artifact@v4` does not send the header.
|
once. `actions/upload-artifact@v4` does not send the header.
|
||||||
|
|
||||||
### What body inspection refuses
|
## What body inspection refuses
|
||||||
|
|
||||||
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
|
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
|
||||||
front of gitea it then refuses these, until their rule ids, named in the request
|
front of gitea it then refuses these, until their rule ids, named in the request
|
||||||
@@ -316,7 +195,7 @@ log, are added to `SWWAF_WAF_DISABLED_RULES`:
|
|||||||
- PyPI uploads (922130);
|
- PyPI uploads (922130);
|
||||||
- attachments named like `debug.log` or `config.yml` (932180).
|
- attachments named like `debug.log` or `config.yml` (932180).
|
||||||
|
|
||||||
### What `script/gitea-requests` sends
|
## What `script/gitea-requests` sends
|
||||||
|
|
||||||
Of the requests above, the script sends:
|
Of the requests above, the script sends:
|
||||||
|
|
||||||
|
|||||||
@@ -1,9 +1,7 @@
|
|||||||
package rproxy_test
|
package rproxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
|
||||||
"bytes"
|
"bytes"
|
||||||
"errors"
|
|
||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net"
|
"net"
|
||||||
@@ -604,12 +602,10 @@ func TestAnAppTooSlowToAnswerIsAnswered504(t *testing.T) {
|
|||||||
func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
|
func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const start = "the start"
|
|
||||||
|
|
||||||
release := make(chan struct{})
|
release := make(chan struct{})
|
||||||
servewaf := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
_, _ = io.WriteString(w, start)
|
_, _ = io.WriteString(w, "the start")
|
||||||
_ = http.NewResponseController(w).Flush()
|
_ = http.NewResponseController(w).Flush()
|
||||||
|
|
||||||
<-release
|
<-release
|
||||||
@@ -617,29 +613,17 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
|
|||||||
|
|
||||||
t.Cleanup(func() { close(release) })
|
t.Cleanup(func() { close(release) })
|
||||||
|
|
||||||
// The time limit is long enough for a loaded host to pass on the start of the
|
|
||||||
// answer before it runs out.
|
|
||||||
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseTime: "3000",
|
maxResponseTime: "300",
|
||||||
})
|
})
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||||
|
|
||||||
res, err := http.ReadResponse(bufio.NewReader(conn), nil)
|
// dial's deadline ends this read with an error if the connection stays open.
|
||||||
|
_, err := io.ReadAll(conn)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("read the answer: %v", err)
|
t.Errorf("read: %v, want the connection closed", err)
|
||||||
}
|
|
||||||
|
|
||||||
// The body breaks off when rproxy closes the connection. If the connection
|
|
||||||
// stays open, dial's deadline ends this read with a timeout instead.
|
|
||||||
body, err := io.ReadAll(res.Body)
|
|
||||||
_ = res.Body.Close()
|
|
||||||
|
|
||||||
if res.StatusCode != http.StatusOK || string(body) != start ||
|
|
||||||
!errors.Is(err, io.ErrUnexpectedEOF) {
|
|
||||||
t.Errorf("answered %d %q (%v), want the app's 200 %q, then the connection closed",
|
|
||||||
res.StatusCode, body, err, start)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := servewaf.report(t); report.Status != http.StatusOK {
|
if report := servewaf.report(t); report.Status != http.StatusOK {
|
||||||
@@ -667,10 +651,8 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
// The time limit is long enough for a loaded host to start passing on the
|
|
||||||
// answer before it runs out.
|
|
||||||
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseTime: "3000",
|
maxResponseTime: "500",
|
||||||
})
|
})
|
||||||
|
|
||||||
// The visitor asks, and never reads the answer.
|
// The visitor asks, and never reads the answer.
|
||||||
@@ -811,10 +793,8 @@ func TestAVisitorThatStopsSendingGetsAnAnswerRProxyGivesByTheDeadline(t *testing
|
|||||||
|
|
||||||
servewaf := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
// The time limit is long enough for a loaded host to read the first part of
|
|
||||||
// the body before it runs out.
|
|
||||||
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestTime: "3000",
|
maxRequestTime: "500",
|
||||||
})
|
})
|
||||||
|
|
||||||
// Go's server waits for a little of the body left before it sends an answer,
|
// Go's server waits for a little of the body left before it sends an answer,
|
||||||
|
|||||||
@@ -1353,10 +1353,8 @@ func TestAReportServeWAFLeavesUnansweredIsDroppedAtTheTimeLimit(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
// The time limit is long enough for a loaded host to answer the question before
|
|
||||||
// it runs out.
|
|
||||||
addr, out := startRProxy(t, backend, app.URL, map[string]string{
|
addr, out := startRProxy(t, backend, app.URL, map[string]string{
|
||||||
timeoutSetting: "3000",
|
timeoutSetting: "1000",
|
||||||
})
|
})
|
||||||
|
|
||||||
t.Cleanup(func() { close(release) })
|
t.Cleanup(func() { close(release) })
|
||||||
|
|||||||
@@ -1,212 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
# script/gitea-setup: build the image, bring up deploy/gitea under a project
|
|
||||||
# name of its own, and check it:
|
|
||||||
# - that a page load, a push and a clone sent through traefik pass through
|
|
||||||
# rproxy, and that servewaf checks them;
|
|
||||||
# - that gitea logs the visitor's address, not rproxy's;
|
|
||||||
# - that a request from a container on the shared network skips the checks;
|
|
||||||
# - that with servewaf stopped, gitea still answers through rproxy.
|
|
||||||
# It publishes no port of traefik's, since this host's ports 80 and 443 may be
|
|
||||||
# taken. It reaches traefik at its address on the shared network instead, so
|
|
||||||
# its own requests come from the network's gateway, which servewaf checks like
|
|
||||||
# any visitor. Two runs at once on one docker host cannot both start, since
|
|
||||||
# both would use the subnet compose.yaml names. The containers, the network,
|
|
||||||
# the volumes, the image and the files the script makes are removed however it
|
|
||||||
# ends; the gitea and traefik images stay, as the base images of a build do.
|
|
||||||
# It needs network access. script/check does not run this.
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|
||||||
|
|
||||||
# Named after this run, so that runs in other clones on the same host
|
|
||||||
# never touch each other's.
|
|
||||||
NAME="$("$SCRIPT_DIR/projectname")-gitea-setup-$$"
|
|
||||||
IMAGE="$NAME-image"
|
|
||||||
NETWORK="${NAME}_shared"
|
|
||||||
VISITOR="$NAME-visitor"
|
|
||||||
|
|
||||||
# The site's host name and gitea's image, as deploy/gitea/compose.yaml names
|
|
||||||
# them. gitea's image also runs the container on the shared network.
|
|
||||||
SITE="git.example.org"
|
|
||||||
# gitea 1.27.3, 2026-08-29
|
|
||||||
GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
|
|
||||||
|
|
||||||
# gitea's one user, who owns the repository.
|
|
||||||
PASSWORD="owner-password"
|
|
||||||
|
|
||||||
# The files the script makes, and traefik's and the gateway's addresses on the
|
|
||||||
# shared network once the setup runs.
|
|
||||||
WORK=""
|
|
||||||
traefik=""
|
|
||||||
gateway=""
|
|
||||||
|
|
||||||
cleanup() {
|
|
||||||
# So that a signal cannot stop it partway through.
|
|
||||||
trap '' HUP INT TERM
|
|
||||||
docker rm --force "$VISITOR" >/dev/null 2>&1 || true
|
|
||||||
if [ -n "$WORK" ]; then
|
|
||||||
compose down --volumes --remove-orphans --timeout 5 >/dev/null 2>&1 || true
|
|
||||||
rm -rf "$WORK"
|
|
||||||
fi
|
|
||||||
docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
|
|
||||||
}
|
|
||||||
|
|
||||||
# compose <argument>...: docker compose on the setup, under this run's project
|
|
||||||
# name, with the image built here and no port of traefik's published.
|
|
||||||
compose() {
|
|
||||||
SMALLWEBWAF_IMAGE="$IMAGE" docker compose --project-name "$NAME" \
|
|
||||||
--file "$ROOT/deploy/gitea/compose.yaml" --file "$WORK/no-ports.yaml" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
fail() {
|
|
||||||
echo "gitea-setup: $*; the last lines of each container's output:" >&2
|
|
||||||
compose logs --no-color --tail 30 >&2 || true
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# wait_for <what fails> <command>...: run the command every second until
|
|
||||||
# it succeeds, for at most a minute.
|
|
||||||
wait_for() {
|
|
||||||
failure="$1"
|
|
||||||
shift
|
|
||||||
tries=0
|
|
||||||
until "$@"; do
|
|
||||||
tries=$((tries + 1))
|
|
||||||
[ "$tries" -lt 60 ] || fail "$failure"
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# logged <service> <text>...: a line of the service's output holds every text,
|
|
||||||
# in any order.
|
|
||||||
logged() {
|
|
||||||
lines="$(compose logs --no-color --no-log-prefix "$1" 2>&1)"
|
|
||||||
shift
|
|
||||||
for text in "$@"; do
|
|
||||||
lines="$(printf '%s\n' "$lines" | grep -F -e "$text")" || return 1
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# send <path> <curl option>...: send a request for path to the site, at
|
|
||||||
# traefik's address, and print the status of the answer, 000 for none. traefik
|
|
||||||
# answers with a certificate it made itself, since the setup names no
|
|
||||||
# certificate resolver.
|
|
||||||
send() {
|
|
||||||
path="$1"
|
|
||||||
shift
|
|
||||||
curl --silent --show-error --insecure --max-time 30 --output /dev/null \
|
|
||||||
--write-out '%{http_code}' --resolve "$SITE:443:$traefik" "$@" \
|
|
||||||
"https://$SITE$path" || true
|
|
||||||
}
|
|
||||||
|
|
||||||
# routed: traefik passes a request for the site's front page on to gitea.
|
|
||||||
routed() {
|
|
||||||
[ "$(send / 2>/dev/null)" = 200 ]
|
|
||||||
}
|
|
||||||
|
|
||||||
# site_git <git argument>...: run git as the owner, without the host's own git
|
|
||||||
# settings, reaching the site at traefik's address. git gives up on a transfer
|
|
||||||
# slower than 1000 bytes a second for 30 seconds.
|
|
||||||
site_git() {
|
|
||||||
GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
|
|
||||||
-c user.name=owner -c user.email=owner@example.invalid \
|
|
||||||
-c http.sslVerify=false -c "http.curloptResolve=$SITE:443:$traefik" \
|
|
||||||
-c http.lowSpeedLimit=1000 -c http.lowSpeedTime=30 "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
# address_of <service>: print the service's address on the shared network.
|
|
||||||
address_of() {
|
|
||||||
docker inspect --format "{{(index .NetworkSettings.Networks \"$NETWORK\").IPAddress}}" \
|
|
||||||
"$(compose ps --quiet "$1")"
|
|
||||||
}
|
|
||||||
|
|
||||||
# passed_through <method> <path>: rproxy passed a request from the gateway for
|
|
||||||
# path to gitea on servewaf's allow, and servewaf checked it: its report line
|
|
||||||
# gives the client's counts and the Core Rule Set's score.
|
|
||||||
passed_through() {
|
|
||||||
wait_for "rproxy logged no $1 $2 that servewaf allowed" logged rproxy \
|
|
||||||
"\"method\":\"$1\"" "\"path\":\"$2\"" "\"client_ip\":\"$gateway\"" \
|
|
||||||
'"disposition":"allow"'
|
|
||||||
wait_for "servewaf logged no check of $1 $2" logged servewaf \
|
|
||||||
'"servewaf_line":"report"' "\"method\":\"$1\"" "\"path\":\"$2\"" \
|
|
||||||
"\"client_ip\":\"$gateway\"" '"action":"forward"' '"counts":' '"waf_score":'
|
|
||||||
}
|
|
||||||
|
|
||||||
main() {
|
|
||||||
cd "$ROOT"
|
|
||||||
trap cleanup EXIT
|
|
||||||
trap 'exit 1' HUP INT TERM
|
|
||||||
WORK="$(mktemp -d)"
|
|
||||||
printf 'services:\n traefik:\n ports: !reset []\n' >"$WORK/no-ports.yaml"
|
|
||||||
|
|
||||||
docker build --no-cache -t "$IMAGE" .
|
|
||||||
|
|
||||||
# rproxy is healthy once gitea takes connections.
|
|
||||||
compose up --detach --quiet-pull --wait --wait-timeout 120 ||
|
|
||||||
fail "the setup did not start"
|
|
||||||
traefik="$(address_of traefik)"
|
|
||||||
gateway="$(docker network inspect --format '{{(index .IPAM.Config 0).Gateway}}' "$NETWORK")"
|
|
||||||
wait_for "traefik did not route the site to rproxy" routed
|
|
||||||
|
|
||||||
compose exec --user git gitea gitea admin user create --admin \
|
|
||||||
--username owner --password "$PASSWORD" --email owner@example.invalid \
|
|
||||||
--must-change-password=false >/dev/null || fail "creating gitea's user failed"
|
|
||||||
status="$(send /api/v1/user/repos --user "owner:$PASSWORD" \
|
|
||||||
--header "Content-Type: application/json" --data '{"name":"repo"}')"
|
|
||||||
[ "$status" = 201 ] || fail "creating the repository was answered $status"
|
|
||||||
|
|
||||||
status="$(send /owner/repo)"
|
|
||||||
[ "$status" = 200 ] || fail "the repository's page was answered $status"
|
|
||||||
passed_through GET /owner/repo
|
|
||||||
echo "gitea-setup: a page load passes through rproxy, and servewaf checks it"
|
|
||||||
|
|
||||||
wait_for "gitea did not log the visitor's address" logged gitea \
|
|
||||||
"GET /owner/repo for $gateway:"
|
|
||||||
echo "gitea-setup: gitea logs the visitor's address, not rproxy's"
|
|
||||||
|
|
||||||
site_git init --quiet --initial-branch=main "$WORK/repo"
|
|
||||||
echo "hello" >"$WORK/repo/README.md"
|
|
||||||
site_git -C "$WORK/repo" add README.md
|
|
||||||
site_git -C "$WORK/repo" commit --quiet --message "Add a README"
|
|
||||||
site_git -C "$WORK/repo" push --quiet \
|
|
||||||
"https://owner:$PASSWORD@$SITE/owner/repo.git" main ||
|
|
||||||
fail "the push failed"
|
|
||||||
passed_through POST /owner/repo.git/git-receive-pack
|
|
||||||
echo "gitea-setup: a push passes through rproxy, and servewaf checks it"
|
|
||||||
|
|
||||||
site_git clone --quiet "https://owner:$PASSWORD@$SITE/owner/repo.git" \
|
|
||||||
"$WORK/clone" || fail "the clone failed"
|
|
||||||
[ "$(cat "$WORK/clone/README.md")" = hello ] || fail "the clone lacks the README"
|
|
||||||
passed_through POST /owner/repo.git/git-upload-pack
|
|
||||||
echo "gitea-setup: a clone passes through rproxy, and servewaf checks it"
|
|
||||||
|
|
||||||
# A container on the shared network, as sandcastle is, asks for the site
|
|
||||||
# by its name, and so reaches traefik from its own address.
|
|
||||||
status="$(docker run --rm --name "$VISITOR" --network "$NETWORK" \
|
|
||||||
--entrypoint curl "$GITEA_IMAGE" --silent --insecure --max-time 30 \
|
|
||||||
--output /dev/null --write-out '%{http_code}' "https://$SITE/api/v1/version" ||
|
|
||||||
true)"
|
|
||||||
[ "$status" = 200 ] || fail "the request from the shared network was answered $status"
|
|
||||||
wait_for "servewaf logged no request from the shared network" logged servewaf \
|
|
||||||
'"servewaf_line":"report"' '"path":"/api/v1/version"' '"action":"forward"'
|
|
||||||
line="$(compose logs --no-color --no-log-prefix servewaf |
|
|
||||||
grep -F '"servewaf_line":"report"' | grep -F '"path":"/api/v1/version"')"
|
|
||||||
case "$line" in
|
|
||||||
*"\"client_ip\":\"$gateway\""*)
|
|
||||||
fail "the request from the shared network came from the gateway" ;;
|
|
||||||
*'"counts":'* | *'"waf_score":'*)
|
|
||||||
fail "servewaf checked the request from the shared network" ;;
|
|
||||||
esac
|
|
||||||
echo "gitea-setup: a request from the shared network skips the checks"
|
|
||||||
|
|
||||||
compose stop servewaf >/dev/null 2>&1 || fail "servewaf did not stop"
|
|
||||||
status="$(send /owner/repo)"
|
|
||||||
[ "$status" = 200 ] ||
|
|
||||||
fail "with servewaf stopped, the repository's page was answered $status"
|
|
||||||
wait_for "rproxy logged no request SWRP_DEFAULT_DISPOSITION decided" \
|
|
||||||
logged rproxy '"path":"/owner/repo"' '"default_disposition":true'
|
|
||||||
echo "gitea-setup: with servewaf stopped, gitea answers through rproxy"
|
|
||||||
}
|
|
||||||
|
|
||||||
main "$@"
|
|
||||||
Reference in New Issue
Block a user