smallwebwaf/README.md

# smallwebwaf

`smallwebwaf` is an MIT-licensed web application firewall, written in Go by
[@sneak](https://sneak.berlin), for one self-hosted web app. It runs inside the
app's own container, in front of the app. It refuses the requests of scrapers,
scanners and other abusers before they reach the app.

A web application firewall reads each request on its way to a web app and
refuses the ones that look like abuse. `smallwebwaf` is also a reverse proxy: it
accepts requests on the app's behalf, passes them to the app, and passes the
app's answers back. A request it does not refuse reaches the app as it came. The
only headers it changes are the ones that tell the app who the client is.

It needs no setting. Every setting has a default chosen for a service on the
open internet. The design is in [`SPEC.md`](SPEC.md). The survey of existing
tools that led to it is in [`EVALUATION.md`](EVALUATION.md).

## What it protects against

- Scrapers that fetch every page of a site, all day. Rate limiting gives each
  client 1000 requests a minute, 10,000 an hour and 50,000 a day. Byte limits
  give it 10 GiB a minute, 20 GiB an hour and 50 GiB a day. A client over a
  limit is refused and banned. The first ban lasts an hour. Each repeat within a
  day makes the next ban three times as long. A ban that would last more than
  seven days is permanent.
- Scanners that probe for secrets and known weak spots: `/.env`, `/.git/config`,
  or a WordPress login on a site that runs no WordPress. The default rule file
  bans such a client for seven days at its first probe. Its next request during
  the ban makes the ban permanent. You can add rules of your own, and name trap
  paths that do the same.
- Attacks carried in a request's URL or headers, such as SQL injection and path
  traversal. The OWASP Core Rule Set is a published set of rules that recognise
  common web attacks. Coraza, a Go engine that runs those rules, inspects every
  request. A request the rules take for an attack is refused.
- Abuse from networks and countries you do not want. You can:
    - refuse whole netblocks and countries;
    - give the clients of chosen networks and countries a fraction of every
      limit, so that they are banned sooner;
    - refuse or limit clients that a blocklist, a DNS blocklist, AbuseIPDB or
      your own CrowdSec engine has already seen misbehave.
- Slow and oversized requests. Every request and every answer has a time limit
  and a size limit.

For the operator, it:

- writes one JSON log line for every request, saying what it did and why;
- sends alerts to a webhook, Slack or ntfy;
- serves Prometheus metrics;
- keeps its bans, counters and history in JSON files you can read and edit, so
  that a restart loses nothing and a ban can be lifted with an editor.

Not built, and not planned:

- TLS and routing for several apps. Both stay with traefik, the reverse proxy in
  front of the container.
- Browser challenges such as captchas.
- A web console.
- Defence against floods large enough to fill the host's network link.

## Getting started

Build the image from a clone. `make docker` runs the tests and the linter, then
builds the image, tagged `smallwebwaf`. It builds for amd64, on an amd64 host
only: the `Dockerfile` checks Ubuntu's package lists against the hashes of the
amd64 archive.

```sh
git clone https://git.eeqj.de/sneak/smallwebwaf.git
cd smallwebwaf
make docker
```

Push the image to a registry your hosts pull from. Then build the app's own
image on it. The app's `Dockerfile` starts `FROM` the `smallwebwaf` image, named
by digest, and adds:

- the app's binary;
- the packages the app needs, from the nixpkgs in the image;
- a user for the app;
- a `run` script that starts the app listening on `127.0.0.1:8081`.

```dockerfile
# The smallwebwaf image, pinned by digest.
FROM <registry>/smallwebwaf@sha256:<digest>

# Packages the app needs, if any, from the nixpkgs in the image.
RUN nix-env -iA nixpkgs.git

# The app's binary, and a user of its own to run it.
COPY app /usr/local/bin/app
RUN useradd --system --no-create-home --shell /usr/sbin/nologin app

# The app's runit service.
COPY --chmod=755 app.run /etc/service/app/run
```

`app.run` sits beside the `Dockerfile`. `--listen` and `--trusted-proxies` stand
for the app's own options:

```bash
#!/usr/bin/env bash
set -euo pipefail

main() {
    sleep 1
    exec chpst -u app:app /usr/local/bin/app \
        --listen 127.0.0.1:8081 \
        --trusted-proxies 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1/32,::1/128
}

main "$@"
```

Leave `ENTRYPOINT` and `USER` out of the app's `Dockerfile`. The image's own
entrypoint starts `smallwebwaf` and the app side by side, each as its own user.

Deploy the app's image as you deploy any app. Point traefik's labels at port
8080 (`traefik.http.services.<name>.loadbalancer.server.port=8080`). Mount a
volume at `/var/lib/smallwebwaf`, so that bans and history survive a deploy. For
example, with no setting:

```sh
docker run --detach --name app \
    --volume app-smallwebwaf:/var/lib/smallwebwaf \
    <registry>/app@sha256:<digest>
```

`deploy/example-app` is such an app. `make example-app` builds it and checks it
(see "Checking that it works" below).

To work on the code, run `smallwebwaf` outside the image. `make build` builds
`bin/smallwebwaf` with the Go on your host. `make run` builds it and runs it
with the settings in your environment. It then listens on port 8080 in front of
an app at `SWWAF_UPSTREAM_URL`, by default `http://127.0.0.1:8081`. Its state
files go in `bin/state` unless `SWWAF_STATE_DIR` is set. Its rule files are
those of `share/rules.d` unless `SWWAF_RULES_DIR` is set.

## How it sits in front of the app

The `smallwebwaf` image is built on Ubuntu 26.04 LTS, pinned by digest. It holds
three packages from a dated snapshot of Ubuntu's archive: `ca-certificates`,
`nix-bin` and `runit`. It also holds nixpkgs, the package collection Nix
installs from, fixed at one commit. An app's `Dockerfile` installs a package
with `nix-env -iA nixpkgs.<name>`, and gets the same package each time it is
built. Such a package lands on the `PATH` of every process in the container,
after Ubuntu's own commands.

- The container's first process is `runsvinit`, which starts runit. runit is a
  service supervisor. It runs the `run` script of each directory under
  `/etc/service`, and runs it again a second after its program exits. The image
  ships `/etc/service/smallwebwaf/run`, which starts `smallwebwaf` as the
  `smallwebwaf` user (uid 65532). The app's `Dockerfile` adds a service
  directory of its own beside it, named after the app. Its `run` script starts
  the app as a user of its own. Neither `smallwebwaf` nor the app runs as root.
  The app's user cannot touch the state files or the process of `smallwebwaf`.
- `smallwebwaf` listens on port 8080, on every address. traefik sends the app's
  requests there. `smallwebwaf` passes them to the app on `127.0.0.1:8081`. The
  app listens on `127.0.0.1` only, so nothing reaches it except through
  `smallwebwaf`. The health check, the metrics and the ban management are on the
  same port, under `/_smallwebwaf/`. A request for that prefix never reaches the
  app.
- The app sees what it would see from traefik directly: the same `Host`, the
  same `X-Forwarded-Proto`, and `X-Forwarded-For` with traefik's address added
  at the end. The app's TCP peer is now `smallwebwaf` on `127.0.0.1`. So the app
  must trust `127.0.0.1` and `::1` for forwarded headers, besides the private
  address ranges. An app left at the usual default, the private ranges alone,
  sees every visitor as `127.0.0.1`. Each request also carries its id in
  `X-Request-ID`, the same id as in its log line.
- Both processes start with the container's environment variables. That is why
  every setting of `smallwebwaf` starts with `SWWAF_`. A token given as an
  environment variable is readable by the app. A token given as a file that only
  the `smallwebwaf` user can read is not (see "Settings given as files" below).
- The state files live in `/var/lib/smallwebwaf`. Without a volume there, they
  survive a restart of the container and are lost when a deploy replaces it. At
  each start, the `run` script gives that directory and every file in it to the
  `smallwebwaf` user. A host directory mounted there needs no change of owner.
- The image's health check runs `smallwebwaf healthcheck`. It passes while
  `smallwebwaf` answers `GET /_smallwebwaf/healthz` on `127.0.0.1` and the app
  accepts connections at `SWWAF_UPSTREAM_URL`. It runs every second from the
  container's start until it first passes, for up to a minute, and every 30
  seconds after that. traefik sends a container no requests until it is healthy.
- `SWWAF_LISTEN_ADDR` can move `smallwebwaf` to another port. The app then
  leaves that port free instead, the health check follows it, and traefik's
  label must name it. Keep the address part empty: `:9000`, never
  `127.0.0.1:9000`. traefik reaches `smallwebwaf` on the container's address,
  and the health check on `127.0.0.1`.
- When `smallwebwaf` or the app exits, runit starts it again a second later. The
  other process keeps running. While `smallwebwaf` is down, traefik answers
  `502`. While the app is down, `smallwebwaf` answers `502`. A `smallwebwaf`
  that stops at start, as on an invalid setting, is started again every second
  until the cause is fixed. It logs its message each time.
- `docker stop` has runit send `SIGTERM` to both processes at once.
  `smallwebwaf` stops taking requests, gives those in progress five seconds to
  finish, and writes its state files. Those requests can finish only while the
  app still answers. An app that exits at once on `SIGTERM` ends them: the
  client gets `502`, or a cut-off answer. An app that should let them finish
  keeps answering for the whole five seconds.

## What happens to a request

`smallwebwaf` handles each request in this order. The first step that refuses it
ends the handling. Nothing of a refused request reaches the app.

1. `GET /_smallwebwaf/healthz` is answered `200` with `ok` at once, before any
   check and without asking the app.
2. The client is worked out. A netblock is a range of addresses, written as an
   address and a prefix length: `203.0.113.0/24`. `SWWAF_TRUSTED_PROXIES` names
   the netblocks traefik comes from, by default the private address ranges.
    - A TCP peer outside those netblocks is the client. Its forwarded headers
      are replaced.
    - For a peer inside them, `X-Forwarded-For` is read from the right. The
      first address outside those netblocks is the client.
    - If every address in the header is inside them, the leftmost is the client.
      With no header, the peer is.
    - An IPv6 client is one whole group: the netblock around its address that
      `SWWAF_IPV6_GROUP_PREFIX` sets, a /64 by default. One abuser usually holds
      a whole /64.
3. A client in `SWWAF_ALLOW_NETS` skips every check below. A client in
   `SWWAF_DENY_NETS` is refused.
4. A client under a ban is refused. If the ban was made for a clear sign of
   attack, this request makes it permanent.
5. The client's AS number and country are looked up, unless
   `SWWAF_LOOKUP_SOURCE` is `off`. An AS number names the network an address
   belongs to, such as `AS64496`. The lookup asks the GeoJS web service by
   default, or a database file you supply.
6. The country lists are checked. A client from a country in
   `SWWAF_DENIED_COUNTRIES` is refused. When
   `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any other country
   is refused, and so is a client without a country.
7. The reputation sources are checked, in this order:
    - the blocklists `SWWAF_BLOCKLIST_URLS` names;
    - the decision list of the CrowdSec engine `SWWAF_CROWDSEC_LAPI_URL` names;
    - the DNS blocklists `SWWAF_DNSBL_ZONES` names;
    - AbuseIPDB, while `SWWAF_ABUSEIPDB_KEY` is set.

    A source that lists the client refuses it, lowers its limits, or only notes
    it, as `SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say. A
    CrowdSec decision bans the client until the decision ends. No request waits
    for a query. DNS blocklists and AbuseIPDB are asked in the background, and
    their answer applies to the client's later requests.

8. Rate limiting counts the request, unless the client is in
   `SWWAF_RATE_LIMIT_EXEMPT_NETS` or the path is under
   `SWWAF_RATE_LIMIT_EXEMPT_PATHS`. The client's percentage of the limits is the
   lowest that applies to it, from its AS number, its country or a reputation
   source; otherwise 100. A request that takes the client over a limit is
   refused and bans the client.
9. A request for a path in `SWWAF_TRAP_PATHS` is refused and bans the client, as
   a clear sign of attack.
10. The rule files are checked, in file name order and line order. A `log` rule
    notes the match. A `block` rule refuses the request. A `ban` rule refuses it
    and bans the client, as a clear sign of attack.
11. The Core Rule Set inspects the request's method, URL and headers. It
    inspects the body too once `SWWAF_WAF_BODY_LIMIT` is set. A request it takes
    for an attack is refused in `block` mode, the default, and let through in
    `detect` mode. It bans no one by itself.
12. A request under `/_smallwebwaf/` is answered here by `smallwebwaf`, after
    its token is checked. Any other request is passed to the app. The app's
    answer is streamed back within the size and time limits. A WebSocket, a
    connection the app switches to another protocol, passes through, and the
    timeouts do not cut it.
13. Once the answer has ended:
    - The request's bytes are counted for the byte limits. A client whose bytes
      pass a limit is banned, so its next request is refused.
    - A refusal by a rule, a trap path, the Core Rule Set or a token check is
      counted for the error burst. More than `SWWAF_ERROR_BURST_THRESHOLD` such
      refusals of one client in a minute ban it.
    - The request goes into the client's history and the anomaly counters, any
      alert due is raised, and the log line is written.

In `observe` mode (`SWWAF_MODE=observe`), steps 3 to 11 refuse nothing and ban
no one. A request they would have refused is passed to the app, and its log line
says what `enforce` mode would have done. The size and time limits still apply,
and so does the token check. Observe mode is for trying a configuration before
enforcing it.

## What a refused visitor sees

- `SWWAF_BAN_RESPONSE` answers most refusals: a banned client, a broken rate
  limit, a `ban` rule, a trap path, `SWWAF_DENY_NETS`, the country lists, and a
  reputation source whose action is `deny`. By default it is `403` with the
  plain text `Forbidden`. It can be `429` instead, or `close`, which closes the
  connection without an answer. Behind traefik, `close` still answers the
  client: traefik sends `502`, as it does whenever its backend drops a
  connection.
- A `block` rule and the Core Rule Set always answer `403`.
- The size and time limits answer with their own statuses:
    - `413` for a request body over `SWWAF_REQUEST_MAX_BYTES`;
    - `408` for a client too slow to send its body;
    - `504` for an app too slow to answer, or too slow to take the request;
    - `502` for an answer over `SWWAF_RESPONSE_MAX_BYTES`, or an app that cannot
      be reached.

    Once an answer has started, a limit can only cut the connection.

- Go's HTTP server answers two cases before `smallwebwaf` sees the request, and
  neither gets a log line. A request line and headers over
  `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` get `431`. A client too slow to send
  them gets an error answer, mostly `400`, or no answer.
- An endpoint of `smallwebwaf`'s own answers `401` to a request without its
  token. While that token is unset, it answers `404`, as does any other path
  under `/_smallwebwaf/`.

## What the operator sees

- Every request gets one JSON line on stdout, which `docker logs` shows. Its
  `action` field says what was done, such as `forward`, `banned`, `rate_limited`
  or `waf_blocked`. Other fields say why: the rules that matched, the Core Rule
  Set's score, the limit broken, the reputation sources that listed the client,
  and when its ban ends. See "Request log" below.
- Every ban is in `bans.json` in the state directory, with notes on why it was
  made. See "Bans" below.
- Every ban, Core Rule Set match, reputation hit, anomaly and failure of a
  source or a file raises an alert. See "Alerts" below.
- With `SWWAF_ADMIN_TOKEN` set, `GET /_smallwebwaf/clients/<ip>` answers with
  everything `smallwebwaf` knows of a client, and `GET /_smallwebwaf/bans` with
  every ban. See "Admin endpoints" below.

## Checking that it works

- `docker inspect --format '{{.State.Health.Status}}' <container>` says
  `healthy` once `smallwebwaf` answers and the app accepts connections. A log
  line with `"msg":"starting"` gives the version, the address and every setting
  in effect, with secrets masked.
- A request to the site through traefik gets the app's answer, and its log line
  has `"action":"forward"`. `https://<site>/_smallwebwaf/healthz` answers `ok`.
- To see a refusal without banning yourself, run the container with
  `SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app. Its log
  line has `"would_action":"banned"` and `"rule_ids":["env-file"]`. In `enforce`
  mode the same request bans your address for seven days, and your next request
  for good. Lift the ban by editing `bans.json` or through the ban endpoint (see
  "Bans" below).
- With `SWWAF_METRICS_TOKEN` set,
  `curl -H "Authorization: Bearer $TOKEN" https://<site>/_smallwebwaf/metrics`
  answers with the metrics (see "Metrics" below).
- `make example-app` builds the image and the example app on it, runs the app,
  and checks it. It needs network access. It checks:
    - the health check passes;
    - a request reaches the app through `smallwebwaf`;
    - a second request in a minute bans the client (the check sets the rate
      limit to one a minute);
    - a probe for `/.env` bans another client, and its next request makes the
      ban permanent;
    - `sv stop` and `docker stop` stop it in order;
    - a new container on the same volume still refuses the banned client;
    - the state directory can be named by a file, and a missing one is reported.

## Settings

Every setting is an environment variable whose name starts with `SWWAF_`. Every
setting has a default, so none has to be set. Settings are read once, at start.
To change one, restart the container.

A setting that is set to a value it cannot take stops the start. The process
writes one line, `"type":"process"` with `"msg":"invalid setting"`, naming the
variable and what is wrong with the value. It exits with status 1, and runit
starts it again every second until the value is fixed. The entries below say
which values each setting takes. Any other value stops the start. At start, the
settings in effect are logged on one line. Tokens, keys and the paths of
webhooks are shown as `********`.

The forms of values:

- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. Zero
  and negative durations are refused.
- A size is a number of bytes with an optional `K`, `M` or `G`, which are powers
  of 1024: `1K` is 1024 bytes, `512K`, `100M`, `5G`.
- A count is a whole number of requests, such as `1000`.
- `off` switches a setting off, where its entry below says it takes `off`.
- A list is comma separated, with spaces around items ignored. An empty item is
  refused. A list set but empty is an empty list, and replaces the default.
- A netblock is an address and a prefix length, such as `10.0.0.0/8`. A bare
  address stands for itself alone.
- A country is a two-letter code of ISO 3166-1, in either case: `de` and `DE`
  are the same. `xk` stands for Kosovo. A code that is not a country is refused,
  such as `nk` (North Korea is `kp`) or the withdrawn `su`.
- An AS number is `AS` and the number, in either case, such as `AS64496`.
- A percentage is a whole number from 0 to 100.
- A path prefix starts with `/`, such as `/assets/`.

### Where it listens and what it protects

- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens, an
  optional host and a port. Keep the host empty.
- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or
  `https` URL with a host and an optional port, and nothing more: no path, user
  or query.
- `SWWAF_INSTANCE_NAME` (default: the container's host name, the first 12
  characters of the container's id): the name every log line, alert and metric
  gives as `instance`. Set it to a name that survives deploys and tells
  instances apart, such as `fsn1app1/gitea`. It must be valid UTF-8. While
  `SWWAF_ALERT_NTFY_URL` is set, it may hold no control character. While
  `SWWAF_LOG_REMOTE_URL` is set and `SWWAF_LOG_REMOTE_APP_NAME` is not, it must
  be 1 to 48 printable ASCII characters without a space.
- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to refuse nothing
  and log what would have been refused (see "What happens to a request" above).
- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`,
  the private address ranges): the netblocks whose `X-Forwarded-For` is
  believed, normally where traefik comes from. Set but empty, it trusts nothing:
  every TCP peer is then the client.
- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the length of the netblock that is
  one IPv6 client, from 32 to 128. After a change, each IPv6 client starts
  afresh. Existing bans keep refusing their netblocks until they end.
- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept in memory
  and in `clients.json`, a whole number above zero. Past it, the least recently
  seen client is dropped first.

### Time and size limits

Each request has limits on both sides: between the client and `smallwebwaf`, and
between `smallwebwaf` and the app. Bodies stream through in both directions and
are never held whole in memory.

- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to
  send its request line and headers, and then, from the end of the headers, its
  body. A duration or `off`.
- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request
  line and headers. A size over `4K`. It cannot be `off`: Go's HTTP server
  always has such a limit.
- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection
  may wait for its next request before it is closed. A duration or `off`. The
  default is longer than the 90 seconds after which traefik closes an unused
  connection, so traefik closes first.
- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the answer may take
  to reach the client, from the end of the request to the last byte. A duration
  or `off`.
- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the
  app and sending it the whole request may take. A duration or `off`.
- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take
  to send its whole answer. A duration or `off`.
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size
  or `off`. A request that announces a larger body is refused before anything
  reaches the app.
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size
  or `off`.

In front of a code forge, a clone is a response and a push is a request. At the
defaults, an upload over 100 MiB or slower than 60 seconds is cut off. A forge
that takes large uploads needs `SWWAF_REQUEST_MAX_BYTES`,
`SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to
fit.

### Netblock lists

- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip every check,
  such as your monitoring or your own network. The size and time limits still
  apply to them. They are not looked up. They still need a token for the admin
  endpoints.
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients rate
  limiting and the byte limits neither count nor refuse, such as a machine that
  talks to the app all day. Everything else still applies to them.
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
  A client in `SWWAF_ALLOW_NETS` too is let through.

### Rate limiting and byte limits

- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
  (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
  requests a client may make in each window. A count or `off`. The defaults are
  several times what one busy person produces. A browser loading a heavy page
  makes a few hundred requests, and several people often share one address.
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes rate limiting
  does not count, such as `/assets/`. Their bytes are not counted either. The
  path is matched as sent, before any query and not percent-decoded, character
  for character. `/assets/` matches `/assets/app.js`. It does not match
  `/Assets/app.js`, `/%61ssets/app.js` or `/static/assets/app.js`. There are no
  wildcards. A path holding `..`, a backslash or an encoded slash (`%2F`) is
  never exempt, since the app may read it as a path outside the prefix.
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
  (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most
  bytes a client may have counted in each window. A size or `off`. Bytes are
  counted once the answer has ended, so a byte limit never cuts an answer short.
  The client's next request is refused instead. Each default is above the
  largest request and answer together, so at the defaults no single download
  breaks a limit.
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
  `response`, `request` or `both`. For a WebSocket, what it carried each way
  counts once it closes.

### Looking up the AS number and country

- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `geojs`
  asks the GeoJS web service, which is then told the address of every new
  visitor. `file` looks clients up in the IPinfo Lite database file
  `SWWAF_LOOKUP_DB_PATH` names, and sends nothing off the host. `off` looks up
  no client. With `off`, a setting that needs lookups stops the start:
    - a country list;
    - `SWWAF_ADD_LOOKUP_HEADERS`;
    - an entry in `SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`,
      `SWWAF_ASN_BYTES_PERCENT` or `SWWAF_COUNTRY_BYTES_PERCENT`;
    - `SWWAF_UNKNOWN_LIMIT_PERCENT` below 100;
    - `SWWAF_ASN_LIMIT_PERCENT_URL`;
    - an anomaly threshold per AS number.
- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
  `.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`. `file` without it stops the
  start, and so does setting it with any other source. A file that is missing or
  cannot be read stops the start too.
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
  client's first answer from GeoJS when a setting needs it, and how long a
  request to GeoJS may take. A duration; it cannot be `off`.
- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the
  client's AS number in `X-Client-ASN` and its country in `X-Client-Country`. A
  header whose value is unknown is left out. Whatever this setting says, any
  such header the client sent is removed.

See "GeoJS and the lookup database" below for what each source sends where, and
how long answers are kept.

### Country lists

- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused,
  such as `cn,ru,kp`.
- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
  countries whose clients get through, such as `us,de`. A client whose country
  cannot be found is refused too. Otherwise every new client would be let in
  whenever GeoJS stops answering. A client on a private address has no country,
  so it is refused unless it is in `SWWAF_ALLOW_NETS`.

A country on both lists stops the start. A refused request is answered as a
banned client is, before its body is read. It is not an offence and makes no
ban.

### Lower limits by AS number and country

Clients of the AS numbers and countries listed here get a percentage of every
rate limit and byte limit. The same rules then ban them after fewer requests. A
lowered limit is rounded down. `0` is a zero allowance: the client's first
request bans it. Each client is counted on its own. No budget is shared by a
whole AS number or country, which one abuser could use up.

- `SWWAF_ASN_LIMIT_PERCENT` (default empty): a list of AS numbers, each with `:`
  and a percentage, such as `AS14061:50,AS16276:50,AS45102:25`. An AS number
  listed twice is refused.
- `SWWAF_COUNTRY_LIMIT_PERCENT` (default empty): the same by country, such as
  `cn:25,ru:50`.
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty):
  the same for the byte limits alone. For the AS numbers and countries they
  list, they take the place of the two settings above.
- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage every client
  without a country gets: one the lookup cannot place, one on a private address,
  and one whose answer from GeoJS has not come in time.
- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
  text file of such entries, one to a line, such as `AS14061:50`. One list can
  then serve several instances. It is fetched and kept as a blocklist is. Where
  it and `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. A
  URL that `SWWAF_BLOCKLIST_URLS` names too stops the start.

A client to which several percentages apply gets the lowest. The log line of
each counted request, and the notes of each ban, say which setting lowered the
limit.

### Reputation sources

None is on by default. A reputation source judges a client by what others saw it
do. The defaults judge it by what it does to your service.

- `SWWAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as
  `https://www.spamhaus.org/drop/drop.txt`. Each is an `http` or `https` URL
  without a user or a fragment. A URL listed twice is refused.
- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long after a list was last
  fetched, or a fetch failed, it is fetched again. A duration of `1h` or more;
  it cannot be `off`. The Spamhaus lists may be fetched no more than once an
  hour.
- `SWWAF_BLOCKLIST_ACTION` (default `deny`): what is done with a client a
  blocklist lists. `deny` refuses its requests. `limit:<percent>`, such as
  `limit:25`, gives it that percentage of every limit. `log` only notes the list
  in the log line and raises an alert.
- `SWWAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask about each
  client, such as `dnsbl.dronebl.org`. A DNS blocklist lists an address by
  answering a DNS query for a name made from it. A zone is a DNS name of at most
  189 characters. A Spamhaus zone carries its key, as in
  `<key>.xbl.dq.spamhaus.net`, and is shown with the key masked. A zone listed
  twice is refused, whatever its case or key. Do not name a zone meant for mail:
  such a zone lists the residential addresses ordinary visitors come from.
- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through,
  such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional
  port, 53 when none is given. Unset, the host's resolver is used. Several zones
  refuse queries that come through a public resolver.
- `SWWAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account.
  While it is set, a client that has committed an offence is checked. A control
  character in it is refused.
- `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score,
  from 0 to 100, that is a hit.
- `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day,
  in UTC, a whole number above zero. AbuseIPDB's free accounts may make 1,000.
- `SWWAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine
  you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a
  user or a fragment. Its decision list is at `v1/decisions` under that URL, and
  is fetched every minute. Set without the key, it stops the start. So does a
  decision list URL that `SWWAF_BLOCKLIST_URLS` or `SWWAF_ASN_LIMIT_PERCENT_URL`
  names too.
- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
  `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
  URL, it stops the start.
- `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client a
  DNS blocklist lists, or whose AbuseIPDB score is a hit. `deny`,
  `limit:<percent>` or `log`, as for `SWWAF_BLOCKLIST_ACTION`. Such a verdict is
  less certain than a list like DROP, so by default the client gets a quarter of
  every limit.
- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or
  AbuseIPDB's score is used. A duration; it cannot be `off`.
- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a
  check with AbuseIPDB may take. A duration; it cannot be `off`.

See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.

### Attack detection

- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the
  rule files (see "Rule files" below). A directory that cannot be read stops the
  start. An empty one is no error.
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file.
- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only
  scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of a code
  forge. A request for one bans the client, as a `ban` rule does. Each starts
  with `/` and holds no `?`. A trap path is matched whole, without the query,
  against the path as sent and once percent-decoded. `/wp-login.php` matches
  `/wp-login.php?x=1` and `/%77p-login.php`. It does not match `/wp-login.php/`
  or `/blog/wp-login.php`.
- `SWWAF_WAF_MODE` (default `block`): what the Core Rule Set does with a request
  it takes for an attack. `block` refuses it with `403`. `detect` lets it
  through, logged and alerted. `off` inspects no request.
- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level,
  from 1 to 4. Each level up runs more rules. Those find more attacks, and take
  more ordinary requests for attacks.
- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the score at which a request is a
  match. Each rule that matches adds to the request's score, and a rule rated
  critical adds 5. So by default one such rule is enough. A whole number above
  zero, or `off`. With `off` no request is a match, and scores are still logged.
- `SWWAF_WAF_DISABLED_RULES` (default
  `920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules
  to switch off. The log line names the rules a request matched, so a rule that
  refuses your app's ordinary requests can be found there. The default switches
  off six rules that get in the way of a code forge:
    - 920340, 920420 and 920640 refuse a request for the type of its body, when
      the type is missing or not on the Core Rule Set's short list;
    - 920440 refuses a request for its file extension, such as `.sh` or `.sql`;
    - 930130 and 930140 refuse a file or directory name in the path, such as
      `.git/`, `.gitignore` or `package.json`.

    In front of a code forge these refuse git over HTTP, uploads and views of
    ordinary files. The default rule file bans those probes at the site root
    instead. A list given replaces the default, so include them. An id must be a
    whole number above zero. An id from 900000 to 900999 is refused: those are
    the rules that set the Core Rule Set up, and `smallwebwaf`'s own.

- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does
  not inspect, matched as `SWWAF_RATE_LIMIT_EXEMPT_PATHS` is.
- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size,
  at most `1G`, switches body inspection on. The Core Rule Set then reads a form
  data or multipart body up to that size, and the rest streams on to the app. It
  reads a JSON or XML body no larger than that size, since those cannot be read
  in part. Any other body, such as a git push, reaches the app uninspected. See
  "The Core Rule Set" below.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
  in a minute after a rule match, a trap path, a Core Rule Set match or a token
  check. One more bans the client, as a broken rate limit does. A count or
  `off`. The app's own `401` and `404` answers are not counted.

### How long bans last and what they cover

- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered: `403`,
  `429`, or `close` to close the connection without an answer. A `block` rule
  and the Core Rule Set always answer `403`.
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate
  limit, byte limit or error burst. A duration; it cannot be `off`.
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a limit broken again within
  this time after a ban ended bans for three times as long as that ban. A
  duration; it cannot be `off`.
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken limit that would
  be longer is permanent instead. A duration; it cannot be `off`.
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
  attack. A duration; it cannot be `off`.
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
  kept, past, active and permanent. A whole number above zero. Bans you make or
  keep are kept besides, and never dropped.
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
  an IPv4 client that a ban covers, from 0 to 32. `24` bans the surrounding /24.
  An IPv6 ban covers the client's group.

### Where and how often the state files are written

- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the directory of the state
  files, an absolute path. A directory that does not exist or cannot be written
  stops the start.
- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long after a ban is made,
  lifted or made permanent `bans.json` is written, with every change in between.
  A duration; it cannot be `off`.
- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is
  written. A duration; it cannot be `off`.

### Logging

- `SWWAF_LOG_REQUEST_HEADERS` (default
  `accept,accept-language,accept-encoding,content-type,origin,range`): the
  request headers whose values the log line gives, in either case.
  `Authorization`, `Cookie` and `Set-Cookie` are never logged, even if listed.
  `Host` and `Transfer-Encoding` are refused: Go's HTTP server takes them out of
  the request. The host is the field `host`.
- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own
  messages that are written: `debug`, `info`, `warn` or `error`. No message is
  at `debug` yet. It holds back no request log line. `warn` and `error` hold
  back the line of settings at start.
- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server every line on stdout
  is also sent to, such as `syslog+tls://logs.example:6514`. The forms are
  `syslog+udp://`, `syslog+tcp://` and `syslog+tls://`, each with a host and a
  port and nothing more.
- `SWWAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates
  that a `syslog+tls` server's certificate must chain to, instead of the host's
  own. A file that cannot be read, or holds no certificate, stops the start.
- `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they
  wait to be sent, a whole number above zero.
- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`,
  `user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, `news`, `uucp`, `cron`,
  `authpriv`, `ftp`, or `local0` to `local7`.
- `SWWAF_LOG_REMOTE_APP_NAME` (default: `SWWAF_INSTANCE_NAME`): the app name the
  records carry, 1 to 48 printable ASCII characters without a space.

### Tokens and metrics

- `SWWAF_ADMIN_TOKEN` (default unset): the token for the ban endpoints and
  `/_smallwebwaf/clients/<ip>`. A long random value of at least 32 characters.
  Unset, those endpoints are off. Give it as a file to keep it from the app (see
  "Settings given as files" below).
- `SWWAF_METRICS_TOKEN` (default unset): the token a scraper sends for
  `/_smallwebwaf/metrics`, at least 32 characters. Unset, the metrics are off.
  It is separate from the admin token, so a scraper cannot manage bans.
- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many
  countries get series of their own in the metrics. The rest count as `other`. A
  whole number above zero.

### Alert destinations

- `SWWAF_ALERT_WEBHOOK_URL` (default unset): a webhook each alert is posted to
  as a JSON object. An `http` or `https` URL without a user or a fragment. The
  settings logged at start show its path and query as `********`, since many
  webhooks carry their secret there.
- `SWWAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post,
  such as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and
  value. A value cannot hold a comma.
- `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): a Slack incoming webhook each
  alert is posted to as a message. It is checked and masked as the webhook URL
  is.
- `SWWAF_ALERT_NTFY_URL` (default unset): an ntfy topic each alert is published
  to, as the topic's full URL, such as `https://ntfy.sh/my-alerts`. It is
  checked and masked as the webhook URL is.
- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token, sent as
  `Authorization: Bearer <token>`. A control character in it is refused.
- `SWWAF_ALERT_EVENTS` (default
  `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`):
  the events alerts are sent for. Any other name is refused.
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held
  back. A duration or `off`.
- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour.
  The rest are rolled into one summary. A whole number above zero, or `off`.

### Anomaly thresholds

These only alert. They refuse and ban nothing. Each scope has four thresholds:
the most requests and the most bytes counted in a minute and in an hour. Their
names are the scope's prefix and `REQUESTS_PER_MINUTE`, `REQUESTS_PER_HOUR`,
`BYTES_PER_MINUTE` or `BYTES_PER_HOUR`. Requests are counts, bytes are sizes.
Each is `off` by default. What is unusual depends on each service's normal
traffic, which the metrics show.

- `SWWAF_ANOMALY_CLIENT_...`: per client.
- `SWWAF_ANOMALY_NET_...`: per netblock around a client. The netblock is
  `SWWAF_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) long for an IPv4
  client, and `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) long
  for an IPv6 one.
- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs lookups on.
- `SWWAF_ANOMALY_TOTAL_...`: the whole service.
- `SWWAF_WATCH_...`: each netblock `SWWAF_WATCH_NETS` names, as a whole.
  `SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
  as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
  name, or with a bad netblock, is refused. So is a name used twice.

Every request but the health check is counted, whatever was done with it,
clients in `SWWAF_ALLOW_NETS` included. At most 20,000 counters are kept.
`alerts.json` keeps them across a restart.

### Settings given as files

Any setting may be given as a file holding its value. The variable named like
the setting with `_FILE` added, such as `SWWAF_METRICS_TOKEN_FILE`, names the
file. The file's contents are the value, less one newline at the end. They are
checked as the value itself would be. Setting both forms stops the start, and so
does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` names a
file already, and has no `_FILE` form.

A token given as a file is out of the app's reach only while the `smallwebwaf`
user alone can read the file. Make it on the host, owned by uid 65532 with mode
`0400`. Mount its directory into the container read-only:

```sh
mkdir -p /srv/app/tokens
openssl rand -hex 32 > /srv/app/tokens/metrics
chown 65532:65532 /srv/app/tokens/metrics
chmod 0400 /srv/app/tokens/metrics
```

and for the container, `-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro` and
`-e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics`.

A few limits are fixed rather than settings:

- at most 100,000 GeoJS answers are kept, for 7 days each;
- at most 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept;
- at most 1,000 DNS blocklist queries are under way at once;
- the CrowdSec decision list is fetched again a minute after the last fetch.

## Bans

A ban refuses every request from a netblock until it ends, with
`SWWAF_BAN_RESPONSE`. The netblock is the client's IPv4 address, or the netblock
around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or the client's IPv6 group. A
banned client is not looked up and not counted by rate limiting.

`smallwebwaf` makes bans for three causes:

- `limit`: the client broke a rate limit, a byte limit or the error burst. The
  first such ban lasts `SWWAF_LIMIT_BAN_DURATION`, an hour. A limit broken again
  within `SWWAF_LIMIT_BAN_REPEAT_WINDOW`, a day, of a ban ending bans for three
  times as long as that ban: 1, 3, 9, 27 and 81 hours. A ban that would last
  longer than `SWWAF_MAX_BAN_DURATION`, seven days, is permanent. Such a ban
  sets the client's counters back to zero.
- `attack`: the client matched a `ban` rule or asked for a trap path. The first
  such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days. Any request from the
  netblock while it lasts makes it permanent. Once it has run out, the netblock
  is served like any other. Its next clear sign of attack bans it permanently at
  once.
- `crowdsec`: a decision of your CrowdSec engine lists the client. The ban lasts
  until the decision ends.

A `block` rule and the Core Rule Set ban no one by themselves, since the Core
Rule Set takes some ordinary requests for attacks. Their refusals count toward
the error burst instead. `SWWAF_DENY_NETS`, the country lists and a reputation
source with the action `deny` make no ban either.

Each ban carries notes for deciding whether to lift it:

- the limit broken, what it was on, its window and the count reached; or the
  rule id and target, or the trap path;
- the client's percentage of the limit and the setting that gave it, when a
  setting lowered it;
- the request that caused the ban;
- the client's AS number, AS name and country;
- the reputation sources that listed the client;
- the netblock's requests since it was first seen, and how many this ban has
  refused;
- how many earlier bans the netblock had, by cause.

At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept. Past that, the
earliest ban of the netblock that has gone longest without a request is dropped.
Bans whose cause is `admin`, those you make or keep, are never dropped.

### Managing bans by editing `bans.json`

`bans.json` in the state directory holds every ban, indented to be read. A
permanent ban's `expires` is `null`. `smallwebwaf` watches the file and takes in
an edit as soon as it is saved.

To ban a netblock, add an entry with its `netblock`, `start` and `expires`.
`expires` is `null` for a ban that never ends. `reason`, `notes` and `cause` may
be left out; the cause is then `admin`:

```json
{
    "version": 1,
    "bans": [
        {
            "netblock": "203.0.113.0/24",
            "start": "2026-10-06T12:00:00Z",
            "expires": null,
            "reason": "probes for logins"
        }
    ]
}
```

To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause`
to `admin`. To lift a ban, add `"lifted"` with the time you lift it, such as
`"lifted": "2026-10-06T13:00:00Z"`. The ban then refuses nothing and does not
make the netblock's next ban longer. It is kept with its notes. To forget a ban,
delete its entry. A ban whose cause is `attack` becomes permanent at the first
request it refuses. One whose cause is `admin` or `crowdsec` does not.

An edit that cannot be read does not stop the running `smallwebwaf`. It keeps
what it holds. At the file's next write it renames your file to `bans.json.bad`,
writes the file again from memory, and logs and alerts the error. Mend the
`.bad` file and move it back. See "State files" below.

### Managing bans through the admin endpoints

With `SWWAF_ADMIN_TOKEN` set, and the token in `$TOKEN`:

```sh
# Every ban.
curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans

# Ban 203.0.113.0/24 for seven days.
curl -H "Authorization: Bearer $TOKEN" \
    --json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \
    https://app.example/_smallwebwaf/bans

# Lift the bans on 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" -X DELETE \
    https://app.example/_smallwebwaf/bans/203.0.113.9

# What smallwebwaf knows of 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" \
    https://app.example/_smallwebwaf/clients/203.0.113.9
```

See "Admin endpoints" below for what each answers.

## Rule files

`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`, in the order of
their names. It checks each request against their rules in that order. A file
whose name starts with `.`, such as an editor's lock file, is not a rule file. A
rule is a line of four fields separated by spaces or tabs: an id, a target, an
action and a regex. The regex runs to the end of the line. Blank lines and lines
starting with `#` are ignored.

```
# id           target      action  regex
env-file       path        ban     (?i)^/\.env(\.[a-z]+)?$
scanner-agent  user_agent  ban     (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
```

- The id is letters, digits, `-` and `_`. No two rules share one. The log line,
  the metrics and a ban's notes name the rule by it.
- The target is what the regex is matched against:
    - `path`: the path without the query;
    - `uri`: the path and the query together;
    - `query`: the query as sent;
    - `method`, `host`, `user_agent` and `referer`;
    - `header:<Name>`: any one request header but `Host` and
      `Transfer-Encoding`.

    `path` and `uri` are matched both as sent and once percent-decoded, so that
    an encoded probe does not slip past. A header sent more than once is matched
    with its values joined by `, `. A header not sent is empty text. No body is
    read.

- The action is `log`, `block` or `ban`. `log` only notes the match in the log
  line. `block` refuses the request with `403` and bans no one. `ban` refuses it
  and bans the client, as a clear sign of attack. Matching stops at the first
  rule that refuses. Keep `ban` for requests no real visitor sends. Anchor a
  path at the site root with `^/`: a file of the same name deeper in a site can
  be ordinary content.
- The regex is in Go's syntax, RE2. It has no backreferences or lookaround, and
  takes time linear in the text it reads. It matches anywhere in the target
  unless anchored with `^` and `$`. `(?i)` at its front makes it ignore case.

A line that is not a rule stops the start, with a message naming the file and
line. So does a bad `header:` name, a rule for `Host` or `Transfer-Encoding`, a
regex that does not compile, or an id used twice. While it runs, `smallwebwaf`
watches the directory. It reads the files again once the directory has had no
change for 2 seconds, so that a file still being written is read whole. If the
files then hold such an error, the rules stay as they were. The log and a
`file_error` alert name the file and line.

The image ships one file, `share/rules.d/00-default.rules`, installed as
`/etc/smallwebwaf/rules.d/00-default.rules`. Its rules:

- ban probes for secrets, version control directories, backups, logs and web
  shells at the site root;
- ban the user agents of common scanners;
- block `../` twice in a row in the path or the query;
- note a request without a user agent.

An app's `Dockerfile` adds rules of its own in a file named to sort after it,
such as `50-gitea.rules`:

```
wp-probe  path  ban  (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
```

```dockerfile
COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules
```

A directory mounted over `/etc/smallwebwaf/rules.d` replaces the default file.
Single files mounted into it add to it. Docker does not show a single mounted
file being replaced, which is how many editors save. So rules to be edited while
`smallwebwaf` runs belong in a mounted directory. To run without rules, mount an
empty directory or set `SWWAF_RULES_ENABLED=false`.

## The Core Rule Set

The OWASP Core Rule Set 4.25.0 runs on each request after the rule files, unless
`SWWAF_WAF_MODE` is `off`. It reads the method, the URL with its query, and the
headers. Once `SWWAF_WAF_BODY_LIMIT` is set, it also reads a body that is form
data, multipart, JSON or XML. No answer is inspected. Each rule that matches
adds to the request's score. A score at or over `SWWAF_WAF_ANOMALY_THRESHOLD` is
a match. A match is refused with `403` in `block` mode, and let through in
`detect` mode. Either way the log line gives the rule ids and the score, and a
`waf_block` alert is sent. A request with more than 1000 query parameters or
body fields adds 5 to its score, since Coraza reads only the first 1000. Three
kinds of request are not inspected: one a rule file refused, one for a path
under `SWWAF_WAF_EXEMPT_PATHS`, and one from a client in `SWWAF_ALLOW_NETS`.

`smallwebwaf` changes the Core Rule Set in six ways, so that a code forge's
ordinary requests get through. The changes hold in front of every app. No
setting undoes them.

- `PUT`, `PATCH` and `DELETE` are allowed besides `GET`, `HEAD`, `POST` and
  `OPTIONS`.
- The headers `Expect` and `Content-Encoding` are allowed. Once
  `SWWAF_WAF_BODY_LIMIT` is set, `Content-Encoding` is refused again on a body
  the Core Rule Set reads, since a compressed body cannot be inspected.
- The query parameter `redirect_uri` is not checked for a URL naming an IP
  address or `localhost` (rules 931100 and 934110). Git credential helpers ask
  to be sent back to such a URL.
- Ten query parameters are not checked against three lists: system files
  (930120), shell paths (932160) and command names (932260). They are `path`,
  `files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`,
  `artifactName` and `redirect_to`. A file such as `.gitignore` or a branch such
  as `docker-build` then gets through there. So does what only those three rules
  refuse, such as `|cat /etc/passwd`. Path traversal and SQL injection are still
  refused there. Names are matched without regard to case. An app that uses one
  of these parameters as a file on the server, or passes it to a shell, has to
  check it itself.
- The cookies `gitea_flash` and `redirect_to` are not read. `Referer` is not
  checked for a Unix command given without arguments (932340), or for Java
  starting a process (944110).
- Responses are not inspected.

Body inspection suits apps whose forms carry no code. In front of a code forge
it refuses issue and comment text, wiki pages and files saved in the web editor
that hold shell commands or code. The log line names the rule ids, which
`SWWAF_WAF_DISABLED_RULES` can switch off. A body the Core Rule Set cannot parse
adds 5 to the score. So does a multipart body the limit cuts in the middle of a
part's header. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to
send the part that is read.

## Admin endpoints

`smallwebwaf` answers these requests itself, on the app's own address and
through traefik like any other request. Apart from the health check, each needs
a token, sent as `Authorization: Bearer <token>`. Each goes through every check
any other request goes through. It is answered where another request would be
passed to the app: a banned client stays refused, and each request counts toward
rate limiting. A missing or wrong token is answered `401`, in `observe` mode
too. Such a refusal counts toward the error burst, so a client guessing tokens
is soon banned. While a token is unset, its endpoints answer `404`.

- `GET /_smallwebwaf/healthz`: `200` and `ok`, to anyone, before any check and
  without asking the app.
- `GET /_smallwebwaf/metrics`: the metrics, for `SWWAF_METRICS_TOKEN` (see
  "Metrics" below).
- `GET /_smallwebwaf/bans`: every ban held, under `bans`, each as `bans.json`
  lists it. Needs `SWWAF_ADMIN_TOKEN`, as do the three below.
- `POST /_smallwebwaf/bans`: bans a netblock. The body is a JSON object of
  `netblock`, `duration` and, if you like, `reason`. `netblock` is a netblock
  such as `203.0.113.0/24`, or a client's address, which bans the netblock a ban
  on that client covers. `duration` is a duration such as `1h` or `7d`, or
  `permanent`. The ban starts at once, with the cause `admin`. A bad body is
  answered `400`, saying what is wrong: one that is not such an object, has
  another field, is longer than 4 KiB, or names an IPv4-mapped netblock or an
  address with a zone.
- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban on a netblock the
  address `<client>` is in. It answers `404` when none is active.
- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client.
  Under `client`, its counters and history as `clients.json` holds them, or
  `null` when it has never been seen. Under `bans`, every ban on a netblock the
  address is in, with its notes.

An admin whose own address is banned lifts that ban by editing `bans.json`.

## Request log

`smallwebwaf` writes one JSON object per line on stdout for every request,
refused ones included:

```
{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
```

A field that does not apply to a request is left out. Every line has `type`, the
fields from `time` to `user_agent`, `request_id`, `peer_ip`, `client_group`,
`asn`, `as_name`, `country`, `action` and `duration_total`.

- `time` is when the request arrived, in UTC. `instance` is
  `SWWAF_INSTANCE_NAME`. `scheme` is the `X-Forwarded-Proto` a trusted proxy
  sent, otherwise `http`. `path` and `query` are as the client sent them.
- `request_id` is the `X-Request-ID` a trusted proxy sent, or a new random one
  of 26 letters and digits. It goes to the app in `X-Request-ID`.
- `peer_ip` is the TCP peer, normally traefik. `forwarded_for` is the
  `X-Forwarded-For` header as received. `client_group` is the client as rate
  limiting counts it: an IPv4 address as a /32, or an IPv6 group.
- `asn`, `as_name` and `country` are the client's, as looked up. Each is empty
  when:
    - lookups are off;
    - the client is not looked up;
    - the source cannot place the client;
    - GeoJS had not answered by the time the request went on.
- `content_type` and `content_length` are the request's own. `request_headers`
  holds the headers `SWWAF_LOG_REQUEST_HEADERS` names, by name in lower case.
  `has_authorization` and `has_cookie` are true when the request had those
  headers; their values are never logged. `websocket` is true when the app
  switched the connection to another protocol.
- `status` is what the client was sent, `0` if nothing. `upstream_status` is
  what the app answered. `response_content_type`, `cache_control` and `location`
  are the answer's headers. `request_bytes` and `response_bytes` count body
  bytes. `aborted` is true when the client went away early.
- `action` is what was done:
    - `forward`: passed to the app;
    - `denied`: refused for `SWWAF_DENY_NETS`, or for a reputation source with
      the action `deny`;
    - `banned`: refused under a ban, or for a `ban` rule, a trap path or a
      CrowdSec decision, each of which bans the client;
    - `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`:
      refused by that check;
    - `too_large` and `timed_out`: a size or time limit was passed;
    - `upstream_error`: the app could not be reached, or its answer broke off;
    - `admin`: answered at one of `smallwebwaf`'s own endpoints.
- `would_action` is there in `observe` mode. It names the action `enforce` mode
  would have taken. `action` then names what was done, normally `forward`.
- `limit_percent` and `limit_percent_setting` give the client's percentage of
  the rate limits and the setting that lowered it, when one did. `bytes_percent`
  and `bytes_percent_setting` do the same for the byte limits.
- `counts` gives the client's requests in the minute, hour and day as rate
  limiting counts them, this request included. `minute_bytes`, `hour_bytes` and
  `day_bytes` give its bytes. Each window is counted in two fixed buckets, the
  earlier one weighted by how much of it the window still covers, so a count can
  have a fraction. `counts` is left out for a request rate limiting does not
  count.
- `rule_ids` lists the rule file rules that matched, in order, up to the one
  that refused. `waf_score` and `waf_rule_ids` give the Core Rule Set's score
  and the rules that matched, for a request it inspected.
- `limit_hit` names the limit broken: `minute`, `hour` or `day` for a rate
  limit; `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or
  `error_burst`. `offence` is then `limit`. A request whose bytes broke a limit
  was passed on whole, so its `action` stays `forward`.
- `reputation` lists the sources that list the client: blocklist URLs, the
  CrowdSec decision list's URL, DNS blocklist zones, then `abuseipdb`.
- `ban_expires` is when the client's ban ends, for a request that made one or
  was refused under one: a time, or `permanent`.
- The timings are in milliseconds. `duration_total` runs from the request's
  headers being read to its log line. `duration_checks` runs to the end of the
  checks, and `duration_waf` is the part of that the Core Rule Set took. The
  three `duration_upstream_` timings run from the hand-over to the app:
  `connect` until there was a connection, `first_byte` until the first byte of
  the answer, and `total` until the end.

No body is logged, and no header but those above. `smallwebwaf`'s own messages
(start, the settings, stop, errors) share the stream as JSON lines marked
`"type":"process"`.

### Sending the log to a syslog server

While `SWWAF_LOG_REMOTE_URL` is set, every line on stdout is also sent to that
server, as the message of an RFC 5424 syslog record. Over UDP each record is one
datagram. Over TCP and TLS each record follows its length in bytes and a space.
The record carries the facility `SWWAF_LOG_REMOTE_FACILITY` names, the severity
informational, the time, the host's name and the app name
`SWWAF_LOG_REMOTE_APP_NAME` gives. stdout is unchanged.

Lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines, so a server that is
slow or down never holds up a request. When the buffer is full, the oldest line
is dropped. A failed connection is tried again a second later, and twice as long
after each further failure, up to a minute. A line too long for one UDP datagram
is dropped alone. As `smallwebwaf` stops, it sends the lines still waiting for
at most two seconds and gives up the rest. stdout has carried them.

## Alerts

Each alert goes to each destination you name. `SWWAF_ALERT_WEBHOOK_URL` gets it
as one JSON object. `SWWAF_ALERT_SLACK_WEBHOOK_URL` and `SWWAF_ALERT_NTFY_URL`
get a message made from it. An alert is one of these events, sent when
`SWWAF_ALERT_EVENTS` names it:

- `ban`: a ban `smallwebwaf` made.
- `permanent_ban`: a permanent ban it made, or a ban a request made permanent.
- `waf_block`: a request the Core Rule Set scored at or over the threshold. In
  `block` mode the request was refused. In `detect` mode it was not, and the
  alert's `detail` gives `mode`.
- `anomaly`: a count over an anomaly threshold.
- `reputation_hit`: a request whose client a blocklist, the CrowdSec decision
  list or a DNS blocklist lists, or whose AbuseIPDB score is a hit. One alert
  per source.
- `source_failure`: GeoJS failing, a list that cannot be fetched, a DNS
  blocklist or AbuseIPDB failing or refusing a query, or the day's AbuseIPDB
  checks used up.
- `file_error`: a rule file or state file edit with an error, a state file that
  cannot be written, or a replacement lookup database that cannot be read.

The bans you make raise no alert. In `observe` mode, a request that would have
made a ban raises the alert `enforce` mode would have, with `mode` set to
`observe` in its `detail`. This is the alert for a ban for a broken rate limit,
shown indented; it is sent on one line:

```json
{
    "instance": "fsn1app1/gitea",
    "time": "2026-10-06T12:00:00.123461Z",
    "event": "ban",
    "client": "203.0.113.9",
    "netblock": "203.0.113.9/32",
    "asn": "",
    "as_name": "",
    "country": "",
    "reason": "requests per minute over the limit of 1000",
    "detail": {
        "ban_expires": "2026-10-06T13:00:00.123Z",
        "cause": "limit",
        "notes": {
            "asn": "",
            "as_name": "",
            "country": "",
            "kind": "requests",
            "limit": 1000,
            "window": "minute",
            "count": 1001,
            "request": {
                "time": "2026-10-06T12:00:00.123456789Z",
                "method": "GET",
                "host": "app.example",
                "path": "/owner/repo/commits/branch/main?page=812",
                "status": 403,
                "user_agent": "scraper/1.0"
            },
            "requests": 5210,
            "refused": 0,
            "earlier_bans": {
                "limit": 0,
                "attack": 0,
                "admin": 0,
                "crowdsec": 0
            }
        }
    },
    "suppressed_repeats": 0
}
```

- `client` is the address whose request raised the alert. `netblock` is the
  ban's netblock, the netblock an anomaly counted, or the client's group. Both
  are empty for `source_failure` and `file_error`. `asn`, `as_name` and
  `country` are the client's as known when the alert was raised. They are empty
  when GeoJS had not answered yet.
- `reason` is a short sentence: a ban's reason, what was counted over which
  threshold, `listed by a blocklist`, and so on.
- `detail` is what is particular to the event:
    - for a ban, its `cause`, `ban_expires` and `notes`;
    - for an anomaly, the `scope`, `window`, `kind`, `count` and `threshold`;
    - for a reputation hit, the `source` and, for AbuseIPDB, the `score`;
    - for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`;
    - for a source failure, the `source` and `error`;
    - for a file error, the `file` and `error`.
- `suppressed_repeats` is how many repeats the cooldown held back before this
  alert.

Slack and ntfy get a message with a title and a text. The title is the instance
and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then a
line for each of the client, netblock, country, file, source, error and mode the
alert has, and the suppressed repeats if any. Slack is posted the title in bold
and the text below it, with `&`, `<` and `>` escaped. ntfy is posted the text,
with the title in `Title`, the token if set, and a priority and a tag by event:

- `ban`: priority `default`, tag `no_entry`.
- `permanent_ban`: priority `high`, tag `no_entry`.
- `waf_block`: priority `default`, tag `shield`.
- `anomaly`: priority `high`, tag `chart_with_upwards_trend`.
- `reputation_hit`: priority `low`, tag `label`.
- `source_failure` and `file_error`: priority `high`, tag `warning`.
- `summary`: priority `default`, tag `bar_chart`.

A repeat is an alert for the same event on the same netblock, source, file or
anomaly scope, less than `SWWAF_ALERT_COOLDOWN` after the last. It is held back
and counted, and the next alert sent gives the count. Past
`SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and counted
by event. Once the hour ends, one alert with the event `summary` sums them up,
with the repeats of the cooldowns that ran out.

Each destination has a queue of at most 1000 alerts, sent one at a time, so a
slow destination holds up nothing. A `2xx` answer takes an alert. A `4xx` other
than `408` and `429` refuses it, and it is dropped. Anything else is a failure:
the alert is sent again a second later, and twice as long after each further
failure, up to a minute. `alerts.json` keeps the queues and cooldowns across a
restart.

## Metrics

`GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request
that sends `SWWAF_METRICS_TOKEN`. No metric carries a client's address. Every
metric carries the label `instance`, set to `SWWAF_INSTANCE_NAME`. Prometheus
keeps it as `exported_instance` unless the scrape sets `honor_labels: true`.

- Traffic:
    - `smallwebwaf_requests_total`, `smallwebwaf_request_bytes_total` and
      `smallwebwaf_response_bytes_total`, by `status_class` such as `2xx` and by
      `action`;
    - `smallwebwaf_request_duration_seconds` and
      `smallwebwaf_upstream_duration_seconds`, as histograms;
    - `smallwebwaf_requests_in_flight`.
- Limits and bans:
    - `smallwebwaf_rate_limit_hits_total`, by `window` and by `kind`
      (`requests`, `bytes` or `refusals`);
    - `smallwebwaf_size_and_time_limit_hits_total`, by `limit`, the setting
      passed;
    - `smallwebwaf_offences_total`, by `kind`;
    - `smallwebwaf_bans_made_total`, by `cause`;
    - `smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
- Attack detection:
    - `smallwebwaf_rule_matches_total`, by `rule_id` and `action`, and
      `smallwebwaf_rules_loaded`;
    - `smallwebwaf_waf_matches_total`, by `mode` and `rule_id`.
- Countries and AS numbers, for the `SWWAF_METRICS_TOP_N` busiest of each, the
  rest as `other`:
    - `smallwebwaf_country_requests_total`,
      `smallwebwaf_country_request_bytes_total`,
      `smallwebwaf_country_response_bytes_total` and
      `smallwebwaf_country_list_refusals_total`, by `country`;
    - `smallwebwaf_asn_requests_total`, `smallwebwaf_asn_request_bytes_total`
      and `smallwebwaf_asn_response_bytes_total`, by `asn`.
- Lookups:
    - `smallwebwaf_geojs_requests_total` and `smallwebwaf_geojs_failures_total`;
    - `smallwebwaf_geojs_unanswered_total`, the requests that needed an answer
      and went on without it;
    - with the file source,
      `smallwebwaf_lookup_database_last_read_timestamp_seconds` and
      `smallwebwaf_lookup_database_read_failures_total`.
- Reputation sources, by `source`: each list's URL, each zone, and `abuseipdb`:
    - `smallwebwaf_reputation_hits_total`,
      `smallwebwaf_reputation_queries_total` and
      `smallwebwaf_reputation_failures_total`;
    - `smallwebwaf_reputation_last_fetch_timestamp_seconds`, for a list;
    - `smallwebwaf_reputation_daily_budget_remaining`, for AbuseIPDB.
- State files, by `file`:
    - `smallwebwaf_state_file_writes_total`,
      `smallwebwaf_state_file_write_failures_total`,
      `smallwebwaf_state_file_last_write_timestamp_seconds` and
      `smallwebwaf_state_file_size_bytes`;
    - `smallwebwaf_state_file_edits_taken_in_total` and
      `smallwebwaf_state_file_edits_set_aside_total`;
    - `smallwebwaf_tracked_clients`, the clients in the table.
- The syslog server, while one is set:
  `smallwebwaf_remote_log_lines_sent_total`,
  `smallwebwaf_remote_log_lines_dropped_total` and
  `smallwebwaf_remote_log_buffer_depth`.
- Alerts, by `destination`: `smallwebwaf_alerts_sent_total`,
  `smallwebwaf_alerts_failed_total`, `smallwebwaf_alerts_suppressed_total` and
  `smallwebwaf_alerts_dropped_total`.
- Go's own `go_` metrics and the process's `process_` metrics.

## State files

`smallwebwaf` keeps its state in memory, and a copy of it in five JSON files in
`SWWAF_STATE_DIR`. Each has a top-level `version` of 1.

- `bans.json`: every ban with its notes, indented to be read (see "Bans" above).
- `clients.json`: each client's counters of requests, bytes and refusals, and
  its history. One client per line, so `grep` shows everything about one. The
  history holds:
    - when the client was first and last seen;
    - its AS number, AS name and country;
    - its requests, and how many were forwarded and refused;
    - its bytes each way;
    - its answers by status class;
    - its offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and
      `token_refused`.
- `lookups.json`: GeoJS's answers, one per line.
- `reputation.json`: the last good copy of each list fetched from a URL, with
  when it was last tried; each DNS blocklist verdict still in use; and
  AbuseIPDB's scores and the checks spent today.
- `alerts.json`: the alert cooldowns, the hour under way, the alerts waiting for
  each destination, and the anomaly counters.

`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted or
made permanent. Every file is written every `SWWAF_STATE_COUNTER_INTERVAL`, and
when `smallwebwaf` stops. Each write goes to a temporary file that then replaces
the file, so a crash leaves the old file or the new one, whole. A write that
fails is logged, raised as a `file_error` alert, and tried again at the next
write. A hard kill loses what changed since the last write.

At start the files are read back. Each client keeps its counts, so a restart
gives no one a fresh allowance. Each ban keeps refusing until it ends. A missing
file is empty state. A file that does not parse, or has another `version`, stops
the start with a message naming the file and the place. So does an entry without
a field it needs, a ban whose `cause` is not `limit`, `attack`, `admin` or
`crowdsec`, and a ban whose `expires` is not after its `start`.

While it runs, `smallwebwaf` watches the directory and takes in your edit of a
file as soon as you save it. What the file then holds replaces what it held. It
tells its own writes from yours by comparing the file with what it last read or
wrote. It takes in your edit before it writes, so your edit is not overwritten.
A change it made after you opened the file, such as a new ban, is lost when you
save over it. An edit with an error that would stop the start does not stop the
running `smallwebwaf`. It keeps what it holds. At the file's next write it
renames your file to `<name>.bad`, writes the file again from memory, and logs
and alerts the file and the place of the error. Mend the `.bad` file and move it
back. A file you remove is written again at its next write.

## GeoJS and the lookup database

By default `smallwebwaf` asks GeoJS, a free web service that needs no account,
about every new visitor's address. It asks whether or not a setting uses the
answer, since the log line, the history, the ban notes and the metrics carry the
AS number and country. Four kinds of visitor are not sent: those in
`SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`, those a ban covers, and those on a
private, loopback or link-local address, which no source can place. An IPv6
visitor is asked about by the first address of its group. Each answer is kept
for seven days, in memory and in `lookups.json`.

A request waits for its client's first answer, up to `SWWAF_LOOKUP_TIMEOUT`,
only while a setting acts on the answer before the request goes on: a country
list, `SWWAF_ADD_LOOKUP_HEADERS`, or any setting under "Lower limits by AS
number and country" above. Otherwise it goes on at once, logged without the
answer. The answer reaches the client's history and ban notes when it comes.
Addresses are asked about together, up to 200 in one request, one request at a
time. While GeoJS fails, new visitors count as from an unknown country, which
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses. GeoJS is then left alone for a
second, and twice as long after each further failure, up to five minutes.

To keep your visitors' addresses on your own host, set
`SWWAF_LOOKUP_SOURCE=off`, or set it to `file`. `file` looks every client up in
the free IPinfo Lite database (`ipinfo_lite.mmdb`), which `SWWAF_LOOKUP_DB_PATH`
names. You download the database with an IPinfo account and mount it into the
container. It answers at once, with nothing sent off the host. A client whose
address it does not hold counts as from an unknown country. `smallwebwaf` reads
the whole file at start. It reads the file again once it has gone 2 seconds
without a change after you replace it. A replacement it cannot read is logged
and alerted, and the file read before stays in use. Mount the directory that
holds the file rather than the file itself: docker does not show a single
mounted file being replaced. IPinfo releases the database under the Creative
Commons Attribution-ShareAlike 4.0 licence. It asks for a link to IPinfo on the
site that uses it, such as "IP address data is powered by IPinfo".

## Blocklists

`SWWAF_BLOCKLIST_URLS` names text files of addresses and netblocks, one to a
line, as the Spamhaus DROP list is written. Anything after a `;` or `#` is left
out. An IPv4-mapped address such as `::ffff:192.0.2.0/120` is read as the IPv4
netblock it stands for.

Each list, and the file `SWWAF_ASN_LIMIT_PERCENT_URL` names, is fetched
`SWWAF_BLOCKLIST_REFRESH` after it was last fetched or tried, one after another.
`reputation.json` keeps when each was last tried, so a restart does not fetch a
list again before its time. A fetch fails when the server answers other than
`200`, does not finish within a minute, or sends more than 16 MiB. It also fails
on a line that is not an address or netblock, or, for the AS number file, not
`AS<number>:<percent>`. The copy fetched before stays in use. The failure is
logged and raised as a `source_failure` alert. Each list is named by its URL in
the log, the alerts, the ban notes and the metrics, so keep a secret out of it.

`deny` suits lists of networks that send nothing legitimate, such as DROP.
`limit:<percent>` suits lists of addresses shared with ordinary visitors, such
as Tor exits. The Spamhaus DROP list is The Spamhaus Project's. Its terms ask
that a product using it credit The Spamhaus Project and keep the list's date and
copyright lines with the data, which the copy in `reputation.json` does. They
also ask that it be fetched no more than once an hour. Each `smallwebwaf`
fetches its own copy, so several on one host, sharing one address, can fetch
within the same hour.

## DNS blocklists

Each zone in `SWWAF_DNSBL_ZONES` is asked about a client's own address in the
background, the first time the client is seen. The request goes on at once, and
so does every request from the client until the zone has answered. The name
asked about is the one RFC 5782 gives: the numbers of an IPv4 address reversed,
or the 32 hex digits of an IPv6 address reversed. `192.0.2.99` is asked about as
`99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` lists the client. A
name that does not exist does not. Any other answer is a failure: an address in
`127.255.255.0/24`, with which Spamhaus refuses a query, an address outside
`127.0.0.0/8`, a resolver error, or no answer within `SWWAF_REPUTATION_TIMEOUT`.
A failure is logged and alerted, and the zone is not asked again for a minute.
Each verdict is kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and in
`reputation.json`.

The key of a Spamhaus zone under `dq.spamhaus.net` is its first label.
`smallwebwaf` shows `********` in its place wherever it names the zone. Only
`reputation.json` keeps the zone with its key.

## AbuseIPDB

While `SWWAF_ABUSEIPDB_KEY` is set, `smallwebwaf` asks AbuseIPDB's check
endpoint for the abuse confidence score of a client, from 0 to 100. Only a
client that has committed an offence is checked: one that broke a limit, matched
a `ban` rule, asked for a trap path, or was refused by a `block` rule, the Core
Rule Set or a token check. The check runs in the background, at the client's
first request after the offence, so no request waits. The score applies to the
client's later requests. An IPv6 group is checked by the address of the request
that has it checked, and the score is used for the whole group. A score at or
over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and `SWWAF_REPUTATION_ACTION` says
what is done. Each score is kept for `SWWAF_REPUTATION_CACHE_TTL`.

At most `SWWAF_ABUSEIPDB_DAILY_BUDGET` checks are made in a day, counted in UTC
and kept in `reputation.json`. A restart does not make the budget whole again.
The check that spends the last of the budget is logged and alerted. No client is
checked until the day ends. A check fails when AbuseIPDB answers other than
`200`, gives no score, or does not answer within `SWWAF_REPUTATION_TIMEOUT`. A
failure is logged and alerted, and no client is checked for a minute. The key is
sent in the `Key` header and nowhere else, and is never logged.

## CrowdSec

While `SWWAF_CROWDSEC_LAPI_URL` names the local API of a CrowdSec engine you
run, `smallwebwaf` fetches its decision list every minute. It sends
`SWWAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key` header. The list is kept as a
blocklist is kept. The decisions of the type `ban` on an address or a netblock
are used. Any other, such as a captcha or a country, is left out. A decision
lasts for the `duration` it had left when the engine answered.

A request from a client a decision in force lists is refused. It bans the
client's netblock until the decision ends, with the cause `crowdsec`. Only a
client that sends a request is banned, so a long list does not fill `bans.json`.
Such a ban is never made permanent, and does not lengthen the netblock's next
ban for a broken limit. A ban you lift while the decision is in force is
followed by another at the client's next request. Delete the decision in
CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and lift
the ban once the list has been fetched again. A fetch fails when the engine
answers other than `200`, a redirect included. It also fails when the fetch does
not finish within a minute, the answer is longer than 16 MiB, or the answer is
not a list of decisions.

## Rationale

Small self-hosted sites now receive a great deal of traffic nobody asked for:

- scrapers that ignore `robots.txt` and crawl every commit of every repository
  on a public git server;
- scanners walking through lists of WordPress and `.env` paths;
- credential-guessing bots.

Most of it comes from a small number of hosting networks and countries. A
single-person operation has no abuse desk and no CDN contract. It needs
something small that can be put in front of one service and left alone.

The existing tools each solve part of this. Rule-based firewalls catch attack
payloads but do not limit request rates. Rate limiters count requests but cannot
tell a residential visitor from a rented server farm. The products that do most
of it want several containers, a database and a web console. None of them can
say "clients from these networks are banned after half as many requests as
anyone else". That is the useful thing to say when nearly all abuse comes from a
known list of AS numbers. [`EVALUATION.md`](EVALUATION.md) goes through the
candidates one by one.

`smallwebwaf` fills that gap with:

- per-client limits that ban;
- lower limits for the networks and countries abuse comes from;
- bans that grow with repetition;
- a log that is easy to search and ship;
- one binary in the app's own container, with no database and no required
  setting.

## Design

- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`.
- `internal/smallwebwaf`: the process. It reads the settings, the rule files,
  the lookup database and the state files, listens, and serves until `SIGTERM`
  or `SIGINT`. Then it stops, writing the state files. Run as
  `smallwebwaf healthcheck`, it is the image's health check instead.
- `internal/config`: reads the settings, the one place they are read.
- `internal/proxy`: what happens to each request. It works out the client, runs
  the checks in `check`, and passes the request to the app with the standard
  library's `httputil.ReverseProxy`, within the timeouts and size limits. It
  answers its own endpoints in `answerAdmin`. Once the request has ended, it
  counts bytes, refusals and anomalies, and writes the log line.
- `internal/ratelimit`: the table of clients. It counts requests, bytes and
  refusals, and keeps each client's history.
- `internal/bans`: the ban ledger. It holds each netblock's bans with their
  notes, decides how long a new ban lasts, and picks the ban to drop past
  `SWWAF_MAX_BANS`.
- `internal/rules`: reads the rule files, and again as they change, and tells
  which rules a request matches.
- `internal/waf`: the Core Rule Set with the six changes, as Coraza directives.
- `internal/lookup`: looks clients up through GeoJS or in the lookup database.
- `internal/reputation`: fetches the lists, asks the DNS blocklists and
  AbuseIPDB in the background, and keeps their answers.
- `internal/anomaly`: the anomaly counters and their alerts.
- `internal/alerts`: holds back repeats and alerts past the hourly limit, and
  sends the rest to the webhook, Slack and ntfy, each from a queue of its own.
- `internal/state`: reads the state files at start, takes in edits while
  running, and writes the files when due and at the stop.
- `internal/requestlog`: the lines on stdout.
- `internal/remotelog`: sends those lines to the syslog server. It is written
  with the standard library alone, since `log/syslog` writes only the older
  syslog format.
- `internal/metrics`: the metrics, served in the Prometheus text format.
- `Dockerfile`: the lint and test phases, then the image. Its last stage
  installs Ubuntu's packages, nixpkgs, `runsvinit` and `smallwebwaf`, with
  `share/smallwebwaf.run` as runit's `run` script and
  `share/rules.d/00-default.rules` as the default rule file.
- `deploy/example-app`: an app built on the image, which `script/example-app`
  checks.

Besides the standard library, it uses:

- `github.com/hashicorp/golang-lru/v2` for the bounded tables;
- `github.com/prometheus/client_golang` for the metrics;
- `github.com/fsnotify/fsnotify` to notice edited files;
- `github.com/oschwald/maxminddb-golang/v2` to read the lookup database;
- `github.com/corazawaf/coraza/v3` to run the Core Rule Set, which
  `github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0` carries;
- `github.com/maxmind/mmdbwriter`, in the tests, to write lookup databases.

## Entrypoints

This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: the scripts in `script/` are the entrypoints for working on it, and
the `Makefile` targets are thin shims that call them. The scripts are POSIX sh.
The tests and the linter run in Docker, as phases of the `Dockerfile`, and never
on the host.

- `script/bootstrap`: installs what the other scripts need on the host: `make`,
  `git`, `curl`, Go for `gofmt`, node and yarn, and prettier. `make bootstrap`.
- `script/setup`: readies a fresh clone: `script/bootstrap`, then
  `script/install-precommit`. `make setup`.
- `script/projectname`: prints `smallwebwaf`, which the other scripts tag their
  images with.
- `script/test`: checks that `go.mod` and `go.sum` are tidy, then runs the
  tests, as the `test` phase of the `Dockerfile`. `make test`.
- `script/tidy`: writes `go.mod` and `go.sum` as `go mod tidy` writes them, in
  the `tidy` stage of the `Dockerfile`. `make tidy`.
- `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`.
  `make lint`.
- `script/fmt`: formats the Go code with `gofmt` and the Markdown with prettier.
  `make fmt`.
- `script/fmt-check`: checks the formatting and changes nothing.
  `make fmt-check`.
- `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`.
  `make check`.
- `script/docker`: builds the image, whose build runs the tests and the linter
  first. `make docker`.
- `script/cibuild`: what CI runs: `script/bootstrap`, `script/check`, then the
  image build.
- `script/precommit`: run by the git pre-commit hook; runs `script/check`.
- `script/install-precommit`: installs that hook. `make hooks`.
- `script/build`: builds `bin/smallwebwaf` on the host, with Go installed.
  `make build`.
- `script/run`: builds `bin/smallwebwaf` and runs it. Its state files go in
  `bin/state` unless `SWWAF_STATE_DIR` is set. Its rule files are those of
  `share/rules.d` unless `SWWAF_RULES_DIR` is set. `make run`.
- `script/example-app`: builds the image and the example app on it, runs it with
  a volume, and checks it as "Checking that it works" above describes. Then it
  removes the containers, volumes and images. Needs network access.
  `make example-app`.
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it at
  the defaults. It sends through it a sample of gitea's ordinary use, and the
  requests the Core Rule Set is known to refuse. It prints what `smallwebwaf`
  did with each and the rule ids that matched, so that those requests can be
  checked again when gitea or the Core Rule Set changes version. Needs network
  access, and a docker daemon that pushes to a registry on `localhost` over
  plain HTTP. `make gitea-requests`.

## TODO

- Everything `SPEC.md` describes is built.
- Two points `SPEC.md` defers until there is experience to go on: sharing bans
  between the instances of several apps, and sending the log over RELP.

## Documents

- [`SPEC.md`](SPEC.md): the design.
- [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers
  and misses, and why none was adopted.
- [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows.

## License

MIT. See [`LICENSE`](LICENSE).

## Author

[@sneak](https://sneak.berlin)