smallwebwaf/README.md

# smallwebwaf

`smallwebwaf` is a simple, fast, logging web application firewall for people who
host their own services. It is one small container that sits between your
reverse proxy (traefik) and one application: traefik points at `smallwebwaf`,
and `smallwebwaf` points at the app. It is configured with environment
variables, keeps its state in memory, and writes a detailed JSON log line for
every request.

Status: design stage. This repository currently holds the documents only; no
code has been written. The design is in [`SPEC.md`](SPEC.md), and the survey of
existing tools that led to it is in [`EVALUATION.md`](EVALUATION.md).

## Why

Small self-hosted sites now receive a great deal of traffic nobody asked for:
scrapers that ignore `robots.txt` and crawl every commit of every repository on
a public git server, vulnerability scanners walking through lists of WordPress
and `.env` paths, and credential-guessing bots. Most of it comes from a small
number of hosting networks and countries. A single-person operation has no
abuse desk and no CDN contract; it needs something small that can be put in
front of one service and left alone.

The existing tools each solve part of this. Rule-based firewalls catch attack
payloads but do not limit request rates. Rate limiters count requests but
cannot tell a residential visitor from a rented server farm. The products that
do most of it want several containers, a database and a web console. None of
them can say "clients from these networks get half the normal allowance", which
is the most useful thing to be able to say when nearly all abuse comes from a
known list of AS numbers. [`EVALUATION.md`](EVALUATION.md) goes through the
candidates one by one.

`smallwebwaf` is meant to fill that gap:

- protect a service from misbehaving scrapers and scanners with per-client
  request and byte limits over a minute, an hour and a day;
- bias those limits against the countries and AS numbers that abuse commonly
  comes from, so their clients get a configured percentage of the normal
  allowance rather than an outright block;
- remember abusers, block them for a while, and ban the ones who keep coming
  back;
- log everything in a form that is easy to search and ship elsewhere;
- stay small enough to understand: one binary, one container, environment
  variables, no database.

## Proposed features

- Reverse proxy for one upstream application, streaming in both directions,
  with WebSocket support. One `smallwebwaf` per app.
- Real client address worked out from `X-Forwarded-For`, trusting only the
  proxy networks you list. IPv6 clients are counted by /64.
- Rate limits per client on requests per minute, per hour and per day, and on
  bytes per minute, per hour and per day.
- Netblocks that bypass rate limiting, netblocks that bypass everything, and
  netblocks that are always refused.
- AS number and country lookup for every client from a local database file.
- Biased limits: listed AS numbers and countries get a percentage of every
  limit, for example 50 percent for common abuse-source networks. Zero percent
  refuses outright.
- Attack detection:
    - a directory of plain text rule files, one regex per line, for catching
      scanning and penetration probes; easy to edit by hand;
    - the OWASP Core Rule Set, run by the Coraza engine, in detect-only or
      blocking mode;
    - trap paths and bursts of error responses.
- Offences add up to a temporary block. Block lengths grow for repeat offenders
  (for example one hour, then a day, then a week) and end in a permanent ban.
- IP reputation: downloadable blocklists, DNS blocklists, AbuseIPDB, and an
  optional feed of decisions from a CrowdSec engine. Lookups happen in the
  background and never delay a request.
- Alerts on attacks and bans to a generic webhook, Slack or ntfy, with a
  cooldown and an hourly cap so a wide attack cannot flood the channel.
- Anomaly alerts when requests or bytes per minute or hour cross a threshold,
  for a single client, its surrounding netblock, an AS number, a named netblock
  or the whole service.
- Observe mode: log and alert on every decision while refusing nothing, for the
  first days in front of a new service.
- Request log: one JSON object per request on stdout with the usual web log
  fields, the decision taken and why, AS number and country, and timings.
  Optionally also sent to a remote syslog or RELP endpoint.
- Prometheus metrics on their own port.
- State (bans, offender history, hour and day counters, reputation cache) held
  in memory and saved as readable, hand-editable JSON files, written atomically.
  Nothing is read from disk while serving a request.
- A small admin endpoint for health checks, listing, adding and lifting bans,
  and asking why a given address was refused.

Not planned: TLS termination, routing for several apps, browser challenges
(captcha or proof of work), a web console, or defence against floods large
enough to fill the host's network link.

## How it works, in short

For each request `smallwebwaf`:

- works out who the client really is;
- lets it straight through if it is on the bypass list, refuses it if it is on
  the deny list or currently banned;
- looks up its AS number and country, and any cached reputation verdict;
- picks the client's limit percentage from those;
- checks the minute, hour and day request counters against the limits, and
  answers 429 if one is exceeded;
- checks the request against the rule files and the Core Rule Set;
- forwards it to the app and streams the response back;
- counts the bytes, records any offence, bans the client if it has collected
  enough of them, sends any alerts that are due, and writes the log line.

A minimal deployment beside an app in docker-compose:

```yaml
services:
    app:
        image: example/app
        networks: [internal]

    waf:
        image: <registry>/smallwebwaf:<pinned digest>
        environment:
            UPSTREAM_URL: http://app:3000
            TRUSTED_PROXIES: 172.18.0.0/16
            MODE: observe
            RATE_LIMIT_PER_MINUTE: "120"
            RATE_LIMIT_PER_HOUR: "2000"
            RATE_LIMIT_PER_DAY: "10000"
            ASN_LIMIT_PERCENT: AS14061:50,AS16276:50
            COUNTRY_LIMIT_PERCENT: CN:25
            ALERT_NTFY_URL: https://ntfy.example.invalid/alerts
        volumes: [waf-state:/data]
        networks: [internal, traefik]
        labels:
            traefik.enable: "true"
            traefik.http.routers.app.rule: Host(`app.example.invalid`)
            traefik.http.services.app.loadbalancer.server.port: "8080"
```

A rule file is one rule per line: a name, what to match against, what to do,
and a regex.

```
env-file       path        offence:3  (?i)/\.env(\.[a-z]+)?$
scanner-agent  user_agent  ban        (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
```

[`SPEC.md`](SPEC.md) has the full design: every environment variable, the rule
file format, the state files, the log fields, the metrics, failure behaviour,
the build order, and the design questions still open for the owner.

## Documents

- [`SPEC.md`](SPEC.md): the design.
- [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers
  and misses, and why none was adopted.