Rename the program to sww and its deciding daemon to discriminator
check / check (push) Successful in 5m33s
check / check (push) Successful in 5m33s
The program is now sww, built from cmd/sww, with two modes: sww rproxy and sww discriminator. servewaf becomes discriminator everywhere it names that daemon: the package (internal/discriminator), its types, functions, files, the discriminator_line log field, and its docs and tests. smallwebwaf is left only where sneak's directive allows it: the documents' product name, the git.eeqj.de/sneak/smallwebwaf URLs, and the Go module path sneak.berlin/go/smallwebwaf. Everywhere else a name containing it takes the sww form: the package (internal/sww), metric names (sww_*), the X-Sww-* headers, the /_sww/ admin endpoints, the SWW_COMMAND setting, the OS user, and the state and rules directories. No logic changes. make check passes. Model: deepseek-v4-pro
This commit is contained in:
1 parent
7fed6b6d36
commit
d8f417b00b
97 files changed
+1945
-1912
No files matched your search
@@ -10,17 +10,17 @@ A web application firewall reads each request on its way to a web app and
|
||||
refuses the ones that look like abuse. `smallwebwaf` is one binary, shipped in
|
||||
one image, with two commands that do this together:
|
||||
|
||||
- `smallwebwaf rproxy` is a small reverse proxy in front of one app. traefik
|
||||
routes to it. It asks `servewaf` about each request, passes the allowed ones
|
||||
to the app, and passes the app's answers back. It keeps nothing and never
|
||||
touches disk.
|
||||
- `smallwebwaf servewaf` decides. It runs the checks and keeps the bans, the
|
||||
counters and each client's history. One `servewaf` serves many sites: a client
|
||||
banned through one site is refused on all of them, and its requests on all of
|
||||
them count toward one set of limits.
|
||||
- `sww rproxy` is a small reverse proxy in front of one app. traefik routes to
|
||||
it. It asks `discriminator` about each request, passes the allowed ones to the
|
||||
app, and passes the app's answers back. It keeps nothing and never touches
|
||||
disk.
|
||||
- `sww discriminator` decides. It runs the checks and keeps the bans, the
|
||||
counters and each client's history. One `discriminator` serves many sites: a
|
||||
client banned through one site is refused on all of them, and its requests on
|
||||
all of them count toward one set of limits.
|
||||
|
||||
The site keeps answering while `servewaf` is down, slow or restarting: without
|
||||
an answer in time, `rproxy` passes the request to the app.
|
||||
The site keeps answering while `discriminator` is down, slow or restarting:
|
||||
without an answer in time, `rproxy` passes the request to the app.
|
||||
|
||||
It protects against:
|
||||
|
||||
@@ -79,11 +79,11 @@ make docker
|
||||
|
||||
Push the image to a registry your hosts pull from, and name it there by digest.
|
||||
|
||||
The image runs `rproxy` by default. To run `servewaf` from the same image, set
|
||||
the container's `SMALLWEBWAF_COMMAND` to `servewaf`. The image's health check
|
||||
The image runs `rproxy` by default. To run `discriminator` from the same image,
|
||||
set the container's `SWW_COMMAND` to `discriminator`. The image's health check
|
||||
follows the command: in `rproxy`'s container it passes once `rproxy` answers and
|
||||
the app accepts connections, whether or not `servewaf` answers; in `servewaf`'s
|
||||
container, once `servewaf` answers.
|
||||
the app accepts connections, whether or not `discriminator` answers; in
|
||||
`discriminator`'s container, once `discriminator` answers.
|
||||
|
||||
### The usual setup: three containers
|
||||
|
||||
@@ -91,24 +91,24 @@ The usual setup is three containers on one private network:
|
||||
|
||||
- the app;
|
||||
- `rproxy`, in front of the app, which traefik routes to: `SWRP_WAF_BACKEND`
|
||||
names `servewaf` and `SWRP_UPSTREAM_URL` names the app;
|
||||
- `servewaf`, with `SMALLWEBWAF_COMMAND=servewaf` and a volume at
|
||||
`/var/lib/smallwebwaf` for its state files.
|
||||
names `discriminator` and `SWRP_UPSTREAM_URL` names the app;
|
||||
- `discriminator`, with `SWW_COMMAND=discriminator` and a volume at
|
||||
`/var/lib/sww` for its state files.
|
||||
|
||||
Point traefik's labels at port 8080 of `rproxy`
|
||||
(`traefik.http.services.<name>.loadbalancer.server.port=8080`). Publish no port
|
||||
of the app or of `servewaf`. Anything on that network can reach the app around
|
||||
`rproxy`, and can ask `servewaf` questions, so the network holds only what you
|
||||
trust. Each further app adds its own container and its own `rproxy`, whose
|
||||
`SWRP_WAF_BACKEND` names the same `servewaf`. The "Smallest setup" example below
|
||||
shows this shape.
|
||||
of the app or of `discriminator`. Anything on that network can reach the app
|
||||
around `rproxy`, and can ask `discriminator` questions, so the network holds
|
||||
only what you trust. Each further app adds its own container and its own
|
||||
`rproxy`, whose `SWRP_WAF_BACKEND` names the same `discriminator`. The "Smallest
|
||||
setup" example below shows this shape.
|
||||
|
||||
### `rproxy` in the app's container
|
||||
|
||||
An app's image can instead be built on the `smallwebwaf` image, so that `rproxy`
|
||||
runs beside the app in one container. `servewaf` still runs in a container of
|
||||
its own. The app's `Dockerfile` starts `FROM` the `smallwebwaf` image, named by
|
||||
digest, and adds:
|
||||
runs beside the app in one container. `discriminator` still runs in a container
|
||||
of its own. The app's `Dockerfile` starts `FROM` the `smallwebwaf` image, named
|
||||
by digest, and adds:
|
||||
|
||||
- the app's binary;
|
||||
- the packages the app needs, from the nixpkgs in the image;
|
||||
@@ -168,16 +168,16 @@ does.
|
||||
- A log line with `"msg":"starting"` gives the version, the address and every
|
||||
setting in effect, with secrets masked.
|
||||
- A request to the site through traefik gets the app's answer.
|
||||
`https://<site>/_smallwebwaf/healthz` answers `ok`. `servewaf`'s log lines
|
||||
about the request have `"action":"forward"`.
|
||||
- To see a refusal without banning yourself, run `servewaf` with
|
||||
`https://<site>/_sww/healthz` answers `ok`. `discriminator`'s log lines about
|
||||
the request have `"action":"forward"`.
|
||||
- To see a refusal without banning yourself, run `discriminator` with
|
||||
`SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app.
|
||||
`servewaf`'s log lines have `"would_action":"banned"` and
|
||||
`discriminator`'s log lines have `"would_action":"banned"` and
|
||||
`"rule_ids":["env-file"]`. A ban you get in `enforce` mode is lifted as
|
||||
"Clients and bans" below describes.
|
||||
- With `SWWAF_METRICS_TOKEN` set, this answers with the metrics, from a host on
|
||||
`servewaf`'s network:
|
||||
`curl -H "Authorization: Bearer $TOKEN" http://servewaf:8080/_smallwebwaf/metrics`.
|
||||
`discriminator`'s network:
|
||||
`curl -H "Authorization: Bearer $TOKEN" http://discriminator:8080/_sww/metrics`.
|
||||
- `make example-app` builds the image and an app on it, runs them, and checks
|
||||
the whole exchange (see "Entrypoints").
|
||||
|
||||
@@ -185,8 +185,8 @@ does.
|
||||
|
||||
Every setting is an environment variable, read once at start; changing one means
|
||||
restarting the container. `rproxy` reads only the `SWRP_` settings, and
|
||||
`servewaf` only the `SWWAF_` ones. Every setting but `SWRP_WAF_BACKEND` has a
|
||||
default, chosen for a service on the open internet. A setting that needs an
|
||||
`discriminator` only the `SWWAF_` ones. Every setting but `SWRP_WAF_BACKEND` has
|
||||
a default, chosen for a service on the open internet. A setting that needs an
|
||||
account, a key, a token or an alert destination stays off until you give one.
|
||||
|
||||
- Any `SWWAF_` setting can be given as a file instead, for secrets and long
|
||||
@@ -216,23 +216,23 @@ Every setting the code reads, with its default:
|
||||
|
||||
| Variable | Default | What it does |
|
||||
| ------------------------------------------ | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `SMALLWEBWAF_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `servewaf`. |
|
||||
| `SWRP_WAF_BACKEND` | required | `servewaf`'s URL, such as `http://servewaf:8080`. |
|
||||
| `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `discriminator`. |
|
||||
| `SWRP_WAF_BACKEND` | required | `discriminator`'s URL, such as `http://discriminator:8080`. |
|
||||
| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
||||
| `SWRP_LISTEN_ADDR` | `:8080` | Where `rproxy` listens; leave the host part empty. |
|
||||
| `SWRP_TRUSTED_PROXIES` | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none. |
|
||||
| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `servewaf`'s answer to a question and to a report, in milliseconds. |
|
||||
| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `servewaf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
||||
| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `discriminator`'s answer to a question and to a report, in milliseconds. |
|
||||
| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `discriminator` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
||||
| `SWRP_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
||||
| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`; a larger one is answered `502`, or cut off once it has started. |
|
||||
| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | How long the visitor may take to send its request line and headers, and then the whole request to reach the app, in milliseconds or `off`. |
|
||||
| `SWRP_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||
| `SWWAF_LISTEN_ADDR` | `:8080` | Where `servewaf` listens; leave the host part empty. |
|
||||
| `SWWAF_INSTANCE_NAME` | the container's host name | The name in every line `servewaf` logs, and in every alert and metric, such as `fsn1/servewaf`. |
|
||||
| `SWWAF_LISTEN_ADDR` | `:8080` | Where `discriminator` listens; leave the host part empty. |
|
||||
| `SWWAF_INSTANCE_NAME` | the container's host name | The name in every line `discriminator` logs, and in every alert and metric, such as `fsn1/discriminator`. |
|
||||
| `SWWAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
||||
| `SWWAF_IPV6_GROUP_PREFIX` | `64` | The prefix length of the IPv6 netblock that counts as one client, from 32 to 128. |
|
||||
| `SWWAF_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
||||
| `SWWAF_MAX_BANS` | `5000` | The most bans `servewaf` keeps of its own; bans an admin made do not count. |
|
||||
| `SWWAF_MAX_BANS` | `5000` | The most bans `discriminator` keeps of its own; bans an admin made do not count. |
|
||||
| `SWWAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
||||
| `SWWAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. |
|
||||
| `SWWAF_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
||||
@@ -263,7 +263,7 @@ Every setting the code reads, with its default:
|
||||
| `SWWAF_COUNTRY_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the countries listed, in place of their `SWWAF_COUNTRY_LIMIT_PERCENT`. |
|
||||
| `SWWAF_UNKNOWN_LIMIT_PERCENT` | `100` | The percentage of every limit a client without a country gets. |
|
||||
| `SWWAF_ASN_LIMIT_PERCENT_URL` | unset | The URL of a text file of `AS<number>:<percent>` lines, fetched like a blocklist; where it and `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. |
|
||||
| `SWWAF_RULES_DIR` | `/etc/smallwebwaf/rules.d` | The directory of the rule files, read at start and again when a file there changes. |
|
||||
| `SWWAF_RULES_DIR` | `/etc/sww/rules.d` | The directory of the rule files, read at start and again when a file there changes. |
|
||||
| `SWWAF_RULES_ENABLED` | `true` | `false` reads no rule file. |
|
||||
| `SWWAF_TRAP_PATHS` | empty | Paths only scanners ask for, such as `/wp-login.php,/xmlrpc.php`; a request for one bans the client. |
|
||||
| `SWWAF_WAF_MODE` | `block` | What the Core Rule Set does with a match: `block` refuses it with `403`, `detect` only logs and alerts it, `off` runs no rule. |
|
||||
@@ -285,18 +285,18 @@ Every setting the code reads, with its default:
|
||||
| `SWWAF_REPUTATION_ACTION` | `limit:25` | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:<percent>` or `log`. |
|
||||
| `SWWAF_REPUTATION_CACHE_TTL` | `24h` | How long a DNS blocklist verdict or an AbuseIPDB score is used. |
|
||||
| `SWWAF_REPUTATION_TIMEOUT` | `2s` | How long a DNS blocklist query or an AbuseIPDB check may take. |
|
||||
| `SWWAF_STATE_DIR` | `/var/lib/smallwebwaf` | The directory of the state files, an absolute path. |
|
||||
| `SWWAF_STATE_DIR` | `/var/lib/sww` | The directory of the state files, an absolute path. |
|
||||
| `SWWAF_STATE_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. |
|
||||
| `SWWAF_STATE_COUNTER_INTERVAL` | `15m` | How often every state file is written. |
|
||||
| `SWWAF_LOG_LEVEL` | `info` | The least severe of `servewaf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. |
|
||||
| `SWWAF_LOG_LEVEL` | `info` | The least severe of `discriminator`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. |
|
||||
| `SWWAF_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. |
|
||||
| `SWWAF_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `servewaf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. |
|
||||
| `SWWAF_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `discriminator` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. |
|
||||
| `SWWAF_LOG_REMOTE_TLS_CA_FILE` | unset | A file of PEM certificates a `syslog+tls` server's certificate must chain to, instead of the host's own. |
|
||||
| `SWWAF_LOG_REMOTE_BUFFER` | `10000` | The most lines held while they wait to be sent; past it, the oldest is dropped. |
|
||||
| `SWWAF_LOG_REMOTE_FACILITY` | `local0` | The syslog facility of each record sent. |
|
||||
| `SWWAF_LOG_REMOTE_APP_NAME` | `SWWAF_INSTANCE_NAME` | The app name in each syslog record, 1 to 48 printable ASCII characters without a space. |
|
||||
| `SWWAF_ADMIN_TOKEN` | unset | The bearer token for the ban and client endpoints, at least 32 characters; unset, they answer `404`. |
|
||||
| `SWWAF_METRICS_TOKEN` | unset | The bearer token for `/_smallwebwaf/metrics`, at least 32 characters; unset, the metrics are off. |
|
||||
| `SWWAF_METRICS_TOKEN` | unset | The bearer token for `/_sww/metrics`, at least 32 characters; unset, the metrics are off. |
|
||||
| `SWWAF_METRICS_TOP_N` | `50` | How many AS numbers and how many countries get metric series of their own; the rest are summed as `other`. |
|
||||
| `SWWAF_ALERT_WEBHOOK_URL` | unset | A webhook each alert is posted to as a JSON object. |
|
||||
| `SWWAF_ALERT_WEBHOOK_HEADERS` | empty | Headers sent with each post, a list of `Name:value`, such as `Authorization:Bearer <token>`. |
|
||||
@@ -343,8 +343,8 @@ own, where each reaches the others by service name. Each example publishes
|
||||
|
||||
### Smallest setup
|
||||
|
||||
One app, `rproxy` in front of it, and `servewaf` with a volume for its state
|
||||
files, every setting at its default.
|
||||
One app, `rproxy` in front of it, and `discriminator` with a volume for its
|
||||
state files, every setting at its default.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
@@ -355,20 +355,20 @@ services:
|
||||
rproxy:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
||||
SWRP_WAF_BACKEND: http://discriminator:8080
|
||||
SWRP_UPSTREAM_URL: http://app:8081
|
||||
ports:
|
||||
- "8080:8080"
|
||||
|
||||
servewaf:
|
||||
discriminator:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SMALLWEBWAF_COMMAND: servewaf
|
||||
SWW_COMMAND: discriminator
|
||||
volumes:
|
||||
- servewaf-state:/var/lib/smallwebwaf
|
||||
- discriminator-state:/var/lib/sww
|
||||
|
||||
volumes:
|
||||
servewaf-state:
|
||||
discriminator-state:
|
||||
```
|
||||
|
||||
### Custom per-address limits
|
||||
@@ -391,15 +391,15 @@ services:
|
||||
rproxy:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
||||
SWRP_WAF_BACKEND: http://discriminator:8080
|
||||
SWRP_UPSTREAM_URL: http://app:8081
|
||||
ports:
|
||||
- "8080:8080"
|
||||
|
||||
servewaf:
|
||||
discriminator:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SMALLWEBWAF_COMMAND: servewaf
|
||||
SWW_COMMAND: discriminator
|
||||
# Monitoring skips every check.
|
||||
SWWAF_ALLOW_NETS: 192.0.2.10
|
||||
# The office and the automation host are never rate limited.
|
||||
@@ -419,18 +419,18 @@ services:
|
||||
SWWAF_BAN_RESPONSE: "429"
|
||||
SWWAF_LIMIT_BAN_DURATION: 30m
|
||||
volumes:
|
||||
- servewaf-state:/var/lib/smallwebwaf
|
||||
- discriminator-state:/var/lib/sww
|
||||
|
||||
volumes:
|
||||
servewaf-state:
|
||||
discriminator-state:
|
||||
```
|
||||
|
||||
### `rproxy` beside the app
|
||||
|
||||
The app is built on the `smallwebwaf` image, with `rproxy` inside its container.
|
||||
`app/` holds the `Dockerfile` and `app.run` from "`rproxy` in the app's
|
||||
container". `rproxy` answers `503` to every request while `servewaf` is down,
|
||||
and `servewaf`:
|
||||
container". `rproxy` answers `503` to every request while `discriminator` is
|
||||
down, and `discriminator`:
|
||||
|
||||
- observes before it enforces;
|
||||
- looks addresses up in a local database rather than GeoJS;
|
||||
@@ -443,17 +443,17 @@ services:
|
||||
# A Dockerfile built FROM the smallwebwaf image, with rproxy inside.
|
||||
build: ./app
|
||||
environment:
|
||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
||||
# Answer 503 rather than pass a request on while servewaf is down.
|
||||
SWRP_WAF_BACKEND: http://discriminator:8080
|
||||
# Answer 503 rather than pass a request on while discriminator is down.
|
||||
SWRP_DEFAULT_DISPOSITION: deny
|
||||
ports:
|
||||
- "8080:8080"
|
||||
|
||||
servewaf:
|
||||
discriminator:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SMALLWEBWAF_COMMAND: servewaf
|
||||
SWWAF_INSTANCE_NAME: fsn1/servewaf
|
||||
SWW_COMMAND: discriminator
|
||||
SWWAF_INSTANCE_NAME: fsn1/discriminator
|
||||
# Refuse nothing yet; log what enforce mode would have done.
|
||||
SWWAF_MODE: observe
|
||||
# Look addresses up in a local IPinfo Lite database, never GeoJS.
|
||||
@@ -461,28 +461,28 @@ services:
|
||||
SWWAF_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
|
||||
# Alerts to an ntfy topic, with its token read from a file.
|
||||
SWWAF_ALERT_NTFY_URL: https://ntfy.example/smallwebwaf
|
||||
SWWAF_ALERT_NTFY_TOKEN_FILE: /etc/smallwebwaf/tokens/ntfy
|
||||
SWWAF_ALERT_NTFY_TOKEN_FILE: /etc/sww/tokens/ntfy
|
||||
SWWAF_ALERT_EVENTS: ban,permanent_ban,waf_block,source_failure,file_error
|
||||
volumes:
|
||||
- servewaf-state:/var/lib/smallwebwaf
|
||||
- discriminator-state:/var/lib/sww
|
||||
- /srv/ipinfo:/var/lib/ipinfo:ro
|
||||
- /srv/smallwebwaf/tokens:/etc/smallwebwaf/tokens:ro
|
||||
- /srv/sww/tokens:/etc/sww/tokens:ro
|
||||
|
||||
volumes:
|
||||
servewaf-state:
|
||||
discriminator-state:
|
||||
```
|
||||
|
||||
The database is mounted by its directory, not by file, so that a replaced
|
||||
database is seen. A token file belongs to uid 65532, the `smallwebwaf` user,
|
||||
with mode `0400`. This one holds an access token the ntfy server issued, which
|
||||
database is seen. A token file belongs to uid 65532, the `sww` user, with mode
|
||||
`0400`. This one holds an access token the ntfy server issued, which
|
||||
`ntfy token add <user>` prints there:
|
||||
|
||||
```sh
|
||||
mkdir -p /srv/smallwebwaf/tokens
|
||||
mkdir -p /srv/sww/tokens
|
||||
# <token> is the access token that ntfy printed.
|
||||
echo '<token>' > /srv/smallwebwaf/tokens/ntfy
|
||||
chown 65532:65532 /srv/smallwebwaf/tokens/ntfy
|
||||
chmod 0400 /srv/smallwebwaf/tokens/ntfy
|
||||
echo '<token>' > /srv/sww/tokens/ntfy
|
||||
chown 65532:65532 /srv/sww/tokens/ntfy
|
||||
chmod 0400 /srv/sww/tokens/ntfy
|
||||
```
|
||||
|
||||
## Design
|
||||
@@ -491,18 +491,18 @@ How it works inside. [`SPEC.md`](SPEC.md) gives every rule in full, and why.
|
||||
|
||||
### What happens to a request
|
||||
|
||||
A request goes through these steps, in this order. Once `servewaf` refuses a
|
||||
request, the steps after that are skipped, apart from the report, and nothing of
|
||||
the request reaches the app.
|
||||
A request goes through these steps, in this order. Once `discriminator` refuses
|
||||
a request, the steps after that are skipped, apart from the report, and nothing
|
||||
of the request reaches the app.
|
||||
|
||||
`rproxy` takes the first two:
|
||||
|
||||
1. The health check, `GET /_smallwebwaf/healthz`, is answered at once.
|
||||
1. The health check, `GET /_sww/healthz`, is answered at once.
|
||||
2. The visitor's address is worked out from `X-Forwarded-For`, believed only
|
||||
from `SWRP_TRUSTED_PROXIES`. `rproxy` sends `servewaf` the request as the
|
||||
visitor sent it, with the first 64 KiB of its body.
|
||||
from `SWRP_TRUSTED_PROXIES`. `rproxy` sends `discriminator` the request as
|
||||
the visitor sent it, with the first 64 KiB of its body.
|
||||
|
||||
`servewaf` takes the checks:
|
||||
`discriminator` takes the checks:
|
||||
|
||||
3. The static lists, `SWWAF_ALLOW_NETS` and `SWWAF_DENY_NETS`.
|
||||
4. The bans.
|
||||
@@ -515,15 +515,15 @@ the request reaches the app.
|
||||
10. The trap paths.
|
||||
11. The rule files.
|
||||
12. The header limits, and then the Core Rule Set.
|
||||
13. A path under `/_smallwebwaf/` is answered `404`, so that the app never sees
|
||||
that prefix.
|
||||
13. A path under `/_sww/` is answered `404`, so that the app never sees that
|
||||
prefix.
|
||||
|
||||
`rproxy` takes the rest, with `servewaf` counting at the end:
|
||||
`rproxy` takes the rest, with `discriminator` counting at the end:
|
||||
|
||||
14. The request is passed to the app, streaming.
|
||||
15. After the answer, `rproxy` sends `servewaf` a report with the status and the
|
||||
bytes each way. For an allowed request, `servewaf` counts the bytes toward
|
||||
the byte limits and adds the request to the client's history.
|
||||
15. After the answer, `rproxy` sends `discriminator` a report with the status
|
||||
and the bytes each way. For an allowed request, `discriminator` counts the
|
||||
bytes toward the byte limits and adds the request to the client's history.
|
||||
|
||||
A refused visitor is answered with `SWWAF_BAN_RESPONSE`. A `block` rule or a
|
||||
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
|
||||
@@ -558,17 +558,17 @@ and the status for a size or time limit, such as `413` for a body too large.
|
||||
Git Credential Manager. No setting undoes them; "Attack detection" in
|
||||
[`SPEC.md`](SPEC.md) lists them.
|
||||
- Rule files are plain text, one rule to a line, edited by hand and read again
|
||||
while `servewaf` runs. The image ships one file of probes no real visitor
|
||||
while `discriminator` runs. The image ships one file of probes no real visitor
|
||||
sends (see [`docs/rules.md`](docs/rules.md)).
|
||||
|
||||
### State files
|
||||
|
||||
`servewaf` keeps everything in memory, and the five JSON files in
|
||||
`discriminator` keeps everything in memory, and the five JSON files in
|
||||
`SWWAF_STATE_DIR` hold a copy: `bans.json`, `clients.json`, `lookups.json`,
|
||||
`reputation.json` and `alerts.json`. Each is written whole, to a temporary file
|
||||
renamed over the real one, so a crash leaves a complete file. `servewaf` notices
|
||||
a file edited on disk and takes the edit in; an edit that does not parse is set
|
||||
aside as `<name>.bad`, and never stops the running process (see
|
||||
renamed over the real one, so a crash leaves a complete file. `discriminator`
|
||||
notices a file edited on disk and takes the edit in; an edit that does not parse
|
||||
is set aside as `<name>.bad`, and never stops the running process (see
|
||||
[`docs/state-files.md`](docs/state-files.md)).
|
||||
|
||||
### The image and its processes
|
||||
@@ -577,30 +577,30 @@ aside as `<name>.bad`, and never stops the running process (see
|
||||
come from Ubuntu's archive, and nixpkgs is fixed at one release. An app's
|
||||
`Dockerfile` installs a package with `nix-env -iA nixpkgs.<name>` and gets the
|
||||
same package each time it is built (see [`docs/image.md`](docs/image.md)).
|
||||
- runit, a service supervisor, runs `smallwebwaf` as the `smallwebwaf` user (uid
|
||||
and gid 65532) and starts it again when it exits. The `run` script of
|
||||
`servewaf` gives the state directory to the `smallwebwaf` user at each start,
|
||||
so a volume needs no change of owner.
|
||||
- `rproxy` and `servewaf` use no token between them. `servewaf`'s own endpoints,
|
||||
under `/_smallwebwaf/`, are for your admin tools and your metrics collector,
|
||||
at its own address on the private network; nothing routes the internet there.
|
||||
- runit, a service supervisor, runs `smallwebwaf` as the `sww` user (uid and
|
||||
gid 65532) and starts it again when it exits. The `run` script of
|
||||
`discriminator` gives the state directory to the `sww` user at each start, so
|
||||
a volume needs no change of owner.
|
||||
- `rproxy` and `discriminator` use no token between them. `discriminator`'s own
|
||||
endpoints, under `/_sww/`, are for your admin tools and your metrics
|
||||
collector, at its own address on the private network; nothing routes the
|
||||
internet there.
|
||||
- `docker stop` gives requests in progress five seconds to finish. With `rproxy`
|
||||
in the app's container, the app must keep answering for five seconds after
|
||||
`SIGTERM` for that to help them.
|
||||
- Only faults found at start stop a process: an invalid setting, a rule file or
|
||||
state file that does not parse, a lookup database that cannot be read, or an
|
||||
address it cannot listen on. Once running, a failing lookup service,
|
||||
reputation source or alert destination never takes `servewaf` down.
|
||||
reputation source or alert destination never takes `discriminator` down.
|
||||
|
||||
### Packages
|
||||
|
||||
- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`.
|
||||
- `internal/smallwebwaf`: the process, as the command it is given, and the
|
||||
health check.
|
||||
- `cmd/sww`: the binary, which only calls `internal/sww`.
|
||||
- `internal/sww`: the process, as the command it is given, and the health check.
|
||||
- `internal/config`: reads the settings, the one place they are read.
|
||||
- `internal/rproxy`: the reverse proxy in front of one app.
|
||||
- `internal/servewaf`: `servewaf`'s server, which runs the checks and answers
|
||||
its own endpoints.
|
||||
- `internal/discriminator`: `discriminator`'s server, which runs the checks and
|
||||
answers its own endpoints.
|
||||
- `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`,
|
||||
`internal/lookup`, `internal/reputation`, `internal/anomaly` and
|
||||
`internal/alerts`: one check or source each.
|
||||
@@ -664,27 +664,26 @@ on the host.
|
||||
image build.
|
||||
- `script/precommit`: run by the git pre-commit hook; runs `script/check`.
|
||||
- `script/install-precommit`: installs that hook. `make hooks`.
|
||||
- `script/build`: builds `bin/smallwebwaf` with the Go on your host.
|
||||
`make build`.
|
||||
- `script/run`: builds `bin/smallwebwaf` and runs `servewaf` and `rproxy` on the
|
||||
- `script/build`: builds `bin/sww` with the Go on your host. `make build`.
|
||||
- `script/run`: builds `bin/sww` and runs `discriminator` and `rproxy` on the
|
||||
host, with the settings in your environment. `make run`. Unless your settings
|
||||
say otherwise:
|
||||
- `rproxy` listens on port 8080, in front of an app at
|
||||
`http://127.0.0.1:8081`;
|
||||
- `servewaf` listens on port 8090;
|
||||
- `discriminator` listens on port 8090;
|
||||
- its state files go in `bin/state`;
|
||||
- its rule files are those of `share/rules.d`.
|
||||
- `script/example-app`: builds the image and the example app
|
||||
(`deploy/example-app`) on it, runs the app with `rproxy` in its container and
|
||||
`servewaf` in another, and checks them. Then it removes the containers, the
|
||||
network, the volumes and the images. Needs network access. `make example-app`.
|
||||
It checks:
|
||||
`discriminator` in another, and checks them. Then it removes the containers,
|
||||
the network, the volumes and the images. Needs network access.
|
||||
`make example-app`. It checks:
|
||||
- the health checks;
|
||||
- a request passed to the app;
|
||||
- a ban for a broken limit, and a ban for a probe made permanent;
|
||||
- that only `servewaf`'s container changes the state directory's owner;
|
||||
- with `servewaf` stopped, the app still answering through `rproxy`;
|
||||
- with `servewaf` started again, bans applying again;
|
||||
- that only `discriminator`'s container changes the state directory's owner;
|
||||
- with `discriminator` stopped, the app still answering through `rproxy`;
|
||||
- with `discriminator` started again, bans applying again;
|
||||
- orderly stops with `sv stop` and `docker stop`;
|
||||
- bans surviving a new container on the same volume;
|
||||
- a setting given as a file.
|
||||
@@ -703,7 +702,7 @@ on the host.
|
||||
- [`docs/settings.md`](docs/settings.md): every setting and the values it takes.
|
||||
- [`docs/rules.md`](docs/rules.md): the rule file format and the default file.
|
||||
- [`docs/admin-endpoints.md`](docs/admin-endpoints.md): the endpoints under
|
||||
`/_smallwebwaf/`, with examples.
|
||||
`/_sww/`, with examples.
|
||||
- [`docs/request-log.md`](docs/request-log.md): the request log's fields.
|
||||
- [`docs/alerts.md`](docs/alerts.md): the alert events and what each destination
|
||||
gets.
|
||||
|
||||
Reference in new issue
Block a user