From 7fb8662e69757765c7a89d590b5b9df09dee4a2a Mon Sep 17 00:00:00 2001 From: sneak Date: Thu, 22 Oct 2026 18:52:51 +0000 Subject: [PATCH 1/8] SPEC rewritten from scratch, short and plain, built and planned apart (closes #156) The old SPEC opened with a draft status and milestone narrative, said several requirements twice, and ended with a build order that is all built. The new one keeps every requirement the old one states, said once, in short sentences with bulleted lists in place of comma chains. What the code on next does is in the main sections; the two points the old SPEC deferred, sharing bans between instances and RELP, are in a last section, "Planned, not built", each marked as planned. The status line, the milestones and the build order are dropped as history. Where the old SPEC disagreed with the code, the new one follows the code; the PR body lists each such point. Model: fable-5-1 --- SPEC.md | 2714 +++++++++++++++++++++++++++---------------------------- 1 file changed, 1346 insertions(+), 1368 deletions(-) diff --git a/SPEC.md b/SPEC.md index c2bf8ed..e437903 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,822 +1,815 @@ -# smallwebwaf SPEC (draft): protective reverse proxy for one app +# smallwebwaf specification -Status: fourth draft, with the owner's rulings to date applied. Milestone 1 of -the build order is built. `EVALUATION.md` beside this file explains why no -existing tool was chosen. +`smallwebwaf` is a web application firewall and reverse proxy for one app. It +runs inside the app's container, between traefik and the app. This document says +what it must do and why. Everything in it but the last section, "Planned, not +built", is built. `README.md` says how to run it. `EVALUATION.md` says why no +existing tool was adopted. ## Purpose -A reverse proxy that runs inside the container of the one application it -protects, between traefik and the app. The app's Dockerfile builds `FROM` the -`smallwebwaf` image; in the container, `smallwebwaf` listens on port 8080, where -traefik sends the app's requests, and forwards them to the app on -`127.0.0.1:8081` (see "Deployment"). It limits request and byte rates per -client, bounds the size and duration of every request and response, detects -attacks, bans abusers (briefly at first, for seven days on a clear sign of -attack, permanently when they keep at it), consults IP reputation sources, looks -up the AS number and country of each client, can refuse whole countries, lowers -its limits for listed AS numbers and countries, and sends alerts. - -It is meant to protect an app on the open internet from the first request with -no setting at all: every setting has a default chosen for a service facing the -internet in 2026. Everything is configured by environment variables, each named -with the `SWWAF_` prefix, apart from the attack detection rules, which are read -from a directory of hand-editable text files. +- It protects one self-hosted app on the open internet from scrapers, scanners + and other abusers. +- It protects the app from its first request with no setting at all. Every + setting has a default chosen for a service facing the internet in 2026. +- It limits each client's requests and bytes per minute, hour and day. It bounds + the size and duration of every request and answer. It detects attacks. +- It bans abusers: briefly at first, for seven days on a clear sign of attack, + and permanently when they keep at it. +- It looks up each client's AS number and country. It can refuse whole countries + and netblocks, and lower its limits for listed AS numbers and countries. It + consults reputation sources. It sends alerts. +- Everything is set by environment variables named with the `SWWAF_` prefix. The + only other inputs are files: the rule files, the state files, a lookup + database, and a setting given as a file. ## Non-goals -- TLS termination, certificates, hostname routing: traefik's job. -- More than one application per `smallwebwaf`. Each app's image carries its own. -- Browser challenges (captcha, proof of work). If wanted, chain Anubis between - `smallwebwaf` and the app. -- Defence against traffic floods that saturate the host's network link. That - needs help upstream of the host. -- A web UI or a configuration file. Settings are environment variables. Apart - from settings given as files (the `_FILE` form of a setting, such as - `SWWAF_ADMIN_TOKEN_FILE`, and `SWWAF_LOG_REMOTE_TLS_CA_FILE`), its own state - files and the lookup database, the only files read are the rule files, which - hold one regex per line and nothing more elaborate. -- Sharing bans between the `smallwebwaf` instances of several apps in the first - version: each instance keeps its own. Running one CrowdSec engine per host, - which every instance would report its bans to, is reconsidered once real ban - volumes are known. Reading a CrowdSec decision list is supported (see - Reputation). +- TLS, certificates and routing by host name. Those are traefik's job. +- More than one app per `smallwebwaf`. Each app's image carries its own. +- Browser challenges such as captchas or proof of work. If wanted, chain Anubis + between `smallwebwaf` and the app. +- Defence against floods that saturate the host's network link. That needs help + upstream of the host. +- A web console or a configuration file. +- Sharing bans between the instances of several apps. Each instance keeps its + own. Reading a CrowdSec engine's decisions is supported. ## Architecture -- One statically linked Go binary, shipped in an image built on Ubuntu 26.04 LTS - with nixpkgs installed, which is the base of the app's own image, so that - `smallwebwaf` and the app run in one container (see "Deployment"). There runit - starts `smallwebwaf` as its own non-root user, and `smallwebwaf` writes only +- One statically linked Go binary. It ships in an image built on Ubuntu 26.04 + LTS with nixpkgs installed. That image is the base of the app's own image, so + that `smallwebwaf` and the app run in one container (see "Deployment"). runit + starts `smallwebwaf` as a non-root user of its own. `smallwebwaf` writes only to its state directory. - One listener, on port 8080, which traefik routes to. It forwards requests to - the app on `127.0.0.1:8081`, and it answers the endpoints of `smallwebwaf` - itself for health, metrics and ban management, under `/_smallwebwaf/`, which - never reach the app (see "Admin endpoints"). -- Components inside the process: - - Client identification: works out the real client IP from - `X-Forwarded-For`, trusting only the proxy netblocks in - `SWWAF_TRUSTED_PROXIES`, by default the private address ranges. - - Lookup: AS number and country, by default from the GeoJS web service, - whose answers are kept for seven days, or instead from a database file the - operator supplies, held in memory. - - Reputation: static blocklists fetched on a schedule; DNSBL and reputation - API queries made in the background and cached. - - Counters: per-client request and byte counts per minute, hour and day, - held in memory and written out with the rest of the state, so they survive - a restart. - - Attack detection: regex rules read from a directory of plain text rule - files and read again whenever the files change (see "Rule files"), Coraza - with the OWASP Core Rule Set, and simple signals (requests for listed trap - paths, bursts of requests refused by the rule files or the Core Rule Set). + the app on `127.0.0.1:8081`. It answers its own endpoints, for health, metrics + and ban management, under `/_smallwebwaf/`. Those never reach the app. +- The parts of the process: + - Client identification: the real client address from `X-Forwarded-For`, + trusting only the netblocks in `SWWAF_TRUSTED_PROXIES`. + - Lookup: the AS number and country, from the GeoJS web service by default + or from a database file the operator supplies, held in memory. + - Reputation: blocklists fetched on a schedule; DNS blocklists and AbuseIPDB + asked in the background, with their answers cached. + - Counters: per-client requests and bytes per minute, hour and day, held in + memory and written out with the rest of the state. + - Attack detection: regex rules read from plain text files and read again + when they change; Coraza with the OWASP Core Rule Set; trap paths; and the + error burst, a burst of refusals from one client. - Ban ledger: active and past bans with their notes, and each client's - history, held in memory and kept in JSON files that `smallwebwaf` watches - for an admin's edits (see "Bans" and "Persistent state"). No database of - any kind is used. + history, held in memory and kept in JSON files the operator may edit. - Request log: one JSON object per request on stdout, optionally also sent - to a remote syslog server (see "Request log"). - - Metrics: Prometheus counters and gauges (see "Metrics endpoint"). - - Alerting: a queue with de-duplication feeding webhook, Slack and ntfy - senders. - - Proxy: the standard library's `net/http/httputil.ReverseProxy`, streaming - in both directions within the size and time limits, with WebSocket upgrade - support. -- Proposed libraries (to be checked against the owner's Go dependency defaults - before any are added): - - standard library for the proxy, HTTP clients, DNS, logging (`log/slog`), - state files (`encoding/json`, `os.Rename`); + to a syslog server. + - Metrics: Prometheus counters and gauges. + - Alerting: a queue that holds back repeats and feeds a webhook, Slack and + ntfy. + - Proxy: the standard library's `httputil.ReverseProxy`, streaming in both + directions within the size and time limits, with WebSocket support. +- No database of any kind. Nothing is read from disk while serving a request. +- Libraries, checked against the owner's Go dependency defaults: + - the standard library for the proxy, HTTP clients, DNS, logging + (`log/slog`) and the state files (`encoding/json`, `os.Rename`); - `github.com/corazawaf/coraza/v3` and `github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries - the Core Rule Set 4.25.0, for attack detection. The changes `smallwebwaf` - makes to the Core Rule Set and the default `SWWAF_WAF_DISABLED_RULES` (see - "Configuration surface", attack detection) are built for that version, - since rule ids and what each rule matches change between versions, and are - checked again before the version changes; - - `github.com/oschwald/maxminddb-golang/v2`, the current major version, to - read the lookup database; - - `github.com/fsnotify/fsnotify` to notice files that are edited or added - while `smallwebwaf` runs; - - `github.com/prometheus/client_golang` for metrics; - - remote log sending is written in this project: the standard library's - `log/syslog` is frozen and writes only the older syslog format, not the - current one (RFC 5424). RELP is not in the first release; it is revisited - against the available packages afterwards. + the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to the Core Rule + Set, and the default `SWWAF_WAF_DISABLED_RULES`, are built for that + version. They are checked again before the version changes; + - `github.com/oschwald/maxminddb-golang/v2` to read the lookup database; + - `github.com/fsnotify/fsnotify` to notice edited files; + - `github.com/prometheus/client_golang` for the metrics; + - remote log sending written in this project, since the standard library's + `log/syslog` is frozen and writes only the older syslog format. -## Data flow for one request +## What happens to a request -Steps run in this order; the first step that produces a final answer ends -processing. The size and time limits (see "Configuration surface") apply to the -whole exchange. +The steps run in this order. The first step that produces a final answer ends +the handling. The size and time limits apply to the whole exchange. -- A `GET /_smallwebwaf/healthz` is answered at once (see "Admin endpoints"). -- Identify the client. - - If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, walk `X-Forwarded-For` - from the right and take the first address not inside - `SWWAF_TRUSTED_PROXIES`. If every address in the header is inside - `SWWAF_TRUSTED_PROXIES`, take the leftmost, so a visitor on a private - network who comes through traefik is known by its own address, not - traefik's. If there is no header, as when another container calls - `smallwebwaf` directly, the TCP peer is the client. - - If the TCP peer is outside `SWWAF_TRUSTED_PROXIES`, it is the client, and - the header is ignored. - - IPv6 clients are grouped by prefix (`SWWAF_IPV6_GROUP_PREFIX`, default 64) - for counting and banning, because one abuser usually controls a whole /64. - A client is therefore one IPv4 address or one IPv6 group, a /64 by - default. -- Static lists. - - In `SWWAF_ALLOW_NETS`: skip every check below and forward, or answer a - request under `/_smallwebwaf/` (see "Admin endpoints"). Still counted for - anomaly alerts. - - In `SWWAF_DENY_NETS`: refuse. -- Ban ledger. An active ban on the client's netblock: refuse with - `SWWAF_BAN_RESPONSE`. If a clear sign of attack caused the ban, the ban - becomes permanent (see "Bans"). -- Look up the AS number and country, unless `SWWAF_LOOKUP_SOURCE` is `off`. With - GeoJS, the default, a request waits for its client's first answer, up to - `SWWAF_LOOKUP_TIMEOUT`, only when a setting needs it before the request goes - on: the country lists, the biased thresholds or `SWWAF_ADD_LOOKUP_HEADERS`. - Otherwise the request goes on at once; the answer is added to the client's - history and ban notes when it comes, and a request that ends before then is - logged without it. A client the lookup cannot place, which includes every - private, loopback and link-local address, has an unknown country and AS - number: the exclusive country list refuses it, the biased thresholds give it - `SWWAF_UNKNOWN_LIMIT_PERCENT`, and nothing else treats it differently. -- Country lists. A client whose country is in `SWWAF_DENIED_COUNTRIES`, or, when - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, is not in it, is refused with - `SWWAF_BAN_RESPONSE`. Every step up to here needs only the client's address, - so the request body has not been read yet, and the refusal skips everything - below, the rule files and the Core Rule Set included. It is counted in the - metrics, but it is not an offence and makes no ban. -- Reputation. - - Address inside a fetched blocklist: apply `SWWAF_BLOCKLIST_ACTION`. - - Cached DNSBL or reputation API result: apply `SWWAF_REPUTATION_ACTION`. - - No cached result: queue a background query and carry on. A first request - is never delayed by a reputation query. -- Work out the client's limit percentage: the lowest of the percentages that - apply (AS number, country, reputation), or 100 if none applies. -- Request rate limits. Unless the client is in `SWWAF_RATE_LIMIT_EXEMPT_NETS`, - check the minute, hour and day counters against each limit times the - percentage. A request over any of them breaks that limit: it is refused with - `SWWAF_BAN_RESPONSE` and the client is banned (see "Bans"). -- Rule files. The request is checked against the rules loaded from - `SWWAF_RULES_DIR`, in file name order then line order. Each rule that matches - takes its action: only log, refuse with 403, or refuse and ban. Matching stops - at the first rule that refuses or bans. -- Core Rule Set inspection of the request: its method, its URL with the query - string, and its headers. By default no body is read. When - `SWWAF_WAF_BODY_LIMIT` is set to a size, a body is read up to that size if it - is form data, multipart, JSON or XML, the kinds the Core Rule Set can read. - Any other body, such as a git push or a container image layer, reaches the app - uninspected, and so does a JSON or XML body larger than - `SWWAF_WAF_BODY_LIMIT`, which cannot be read in part. Responses are not - inspected. In `block` mode, the default, a match at or over the anomaly - threshold is refused with 403; in `detect` mode it is only logged and alerted. -- A request under `/_smallwebwaf/` is answered by `smallwebwaf` here and goes no - further (see "Admin endpoints"). -- Forward to `SWWAF_UPSTREAM_URL`, streaming. Add `X-Forwarded-For` and, if - enabled, `X-Client-ASN` and `X-Client-Country` for the app's own logs. -- After the response. - - Add response bytes (and request body bytes) to the byte counters. A client - whose byte total passes a byte limit times the percentage has broken that - limit and is banned. The response in progress is not cut off. - - Count the client's requests that `smallwebwaf` refused after a rule file - or Core Rule Set match, or for a missing or wrong token (see "Admin - endpoints"); answers the app gave are not counted. More than - `SWWAF_ERROR_BURST_THRESHOLD` of them within a minute breaks a limit and - bans the client. - - Update the client's history and the anomaly counters, and evaluate alert - thresholds. +1. `GET /_smallwebwaf/healthz` is answered at once, before any check. +2. The client is identified. + - If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, `X-Forwarded-For` is + walked from the right. The first address outside those netblocks is the + client. If every address is inside them, the leftmost is the client, so + that a visitor on a private network is known by its own address. With no + header, the peer is the client. + - If the peer is outside `SWWAF_TRUSTED_PROXIES`, it is the client, and the + header is ignored. + - An IPv6 client is its group: the netblock of `SWWAF_IPV6_GROUP_PREFIX` + around its address, a /64 by default, since one abuser usually controls a + whole /64. A client is therefore one IPv4 address or one IPv6 group. +3. The static lists. A client in `SWWAF_ALLOW_NETS` skips every check below. It + is still counted for the anomaly thresholds. A client in `SWWAF_DENY_NETS` is + refused. +4. The ban ledger. A client under a ban is refused with `SWWAF_BAN_RESPONSE`. If + a clear sign of attack caused the ban, the ban becomes permanent. +5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off`. + - With GeoJS, a request waits for its client's first answer, up to + `SWWAF_LOOKUP_TIMEOUT`, only when a setting needs it before the request + goes on: a country list, a biased threshold or `SWWAF_ADD_LOOKUP_HEADERS`. + Otherwise the request goes on at once. The answer is added to the client's + history and ban notes when it comes. A request that ends before then is + logged without it. + - A client the lookup cannot place has an unknown country and AS number. + Every private, loopback and link-local address is such a client. The + exclusive country list refuses it, and the biased thresholds give it + `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it differently. +6. The country lists. A client from a country in `SWWAF_DENIED_COUNTRIES` is + refused. When `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any + other country is refused. The answer is `SWWAF_BAN_RESPONSE`. The request + body has not been read yet. The refusal skips everything below, the rule + files and the Core Rule Set included. It is logged with the action + `country_denied` and counted in the metrics. It is not an offence and makes + no ban. +7. The reputation sources. A client inside a fetched blocklist gets + `SWWAF_BLOCKLIST_ACTION`. A client the CrowdSec decision list lists is banned + until the decision ends. A cached DNS blocklist verdict or AbuseIPDB score + gets `SWWAF_REPUTATION_ACTION`. With no cached verdict, a query is queued in + the background and the request goes on. A first request is never delayed by a + reputation query. +8. The client's limit percentage: the lowest of the percentages that apply to + it, from its AS number, its country or a reputation source, or 100 if none + does. +9. Rate limiting, unless the client is in `SWWAF_RATE_LIMIT_EXEMPT_NETS` or the + path is under `SWWAF_RATE_LIMIT_EXEMPT_PATHS`. The minute, hour and day + counters are checked against each limit times the percentage. A request over + any of them breaks that limit: it is refused with `SWWAF_BAN_RESPONSE`, and + the client is banned (see "Bans"). +10. The trap paths. A request for a path in `SWWAF_TRAP_PATHS` is a clear sign + of attack: it is refused and the client is banned. +11. The rule files, in file name order and then line order. A rule that matches + takes its action: only log, refuse with `403`, or refuse and ban. Matching + stops at the first rule that refuses. +12. The Core Rule Set inspects the request's method, its URL with the query + string, and its headers. By default it reads no body. When + `SWWAF_WAF_BODY_LIMIT` is a size, it reads a body that is form data, + multipart, JSON or XML, up to that size. Any other body reaches the app + uninspected. So does a JSON or XML body larger than the limit, which cannot + be read in part. Responses are not inspected. In `block` mode, the default, + a match at or over the anomaly threshold is refused with `403`. In `detect` + mode it is only logged and alerted. +13. A request under `/_smallwebwaf/` is answered here and goes no further. +14. The request is forwarded to `SWWAF_UPSTREAM_URL`, streaming, with + `X-Forwarded-For` added, and `X-Client-ASN` and `X-Client-Country` when + enabled. +15. After the response: + - The response bytes, the request body bytes, or both are added to the byte + counters. A client whose total passes a byte limit times the percentage + has broken that limit and is banned. The response in progress is not cut + off. + - The client's refusals are counted: those `smallwebwaf` refused after a + rule file match, a trap path, a Core Rule Set match, or a missing or wrong + token. Answers the app gave are not counted. More than + `SWWAF_ERROR_BURST_THRESHOLD` of them within a minute break a limit and + ban the client. + - The client's history and the anomaly counters are updated, and the alert + thresholds are evaluated. ## Counting method -- Each window (minute, hour, day) uses two adjacent fixed buckets per client, - with the previous bucket weighted by how much of it still overlaps the sliding - window. This costs a few integers per client per window and avoids the burst +- Each window (minute, hour, day) uses two adjacent fixed buckets per client. + The earlier bucket is weighted by how much of it still overlaps the sliding + window. This costs a few integers per client per window. It avoids the burst at bucket boundaries that a single fixed bucket allows. -- Counters are read and updated in memory; a request never waits on the disk. +- Counters are read and updated in memory. A request never waits on the disk. They are written to `clients.json` every `SWWAF_STATE_COUNTER_INTERVAL` and at - shutdown and loaded again at start (see "Persistent state"), so a restart does - not hand every client a fresh allowance. Buckets whose time has passed are - discarded on load. -- Memory is bounded by `SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_MAX_BANS` and the - number of GeoJS answers kept (see "Configuration surface"). When the table of - clients is full, the least recently seen client is dropped first, with its - history. When `SWWAF_MAX_BANS` bans `smallwebwaf` made are held, the one that - has gone longest without a request from its netblock is dropped first, whether - it is past, active or permanent. Bans an admin made are never dropped, and - there are as many as the admin adds (see "Bans"). + shutdown, and loaded again at start. A restart therefore does not hand every + client a fresh allowance. Buckets whose time has passed are discarded on load. +- Memory is bounded by `SWWAF_MAX_TRACKED_CLIENTS` and `SWWAF_MAX_BANS`, and by + fixed limits on the GeoJS answers, the reputation verdicts and the anomaly + counters kept. When the table of clients is full, the least recently seen + client is dropped first, with its history. When `SWWAF_MAX_BANS` bans + `smallwebwaf` made are held, the one that has gone longest without a request + is dropped first, past, active or permanent. Bans an admin made are never + dropped. ## Bans -A ban refuses every request from a netblock, answering with `SWWAF_BAN_RESPONSE` -(`403` by default), until the ban ends. A permanent ban does not run out: it -ends when an admin lifts it or, if `smallwebwaf` made it, when a new ban is made -while `SWWAF_MAX_BANS` bans `smallwebwaf` made are held and it is the one that -has gone longest without a request. A scanner whose ban was dropped that way is -refused and banned again by its next probe. A ban an admin made, whose cause is -`admin`, is never dropped and does not count toward `SWWAF_MAX_BANS`, so such -bans can never fill the table, however many there are. An admin who wants to -keep a ban `smallwebwaf` made sets its cause to `admin`. +A ban refuses every request from a netblock with `SWWAF_BAN_RESPONSE` until it +ends. A permanent ban does not run out. It ends when an admin lifts it, or, if +`smallwebwaf` made it, when it is dropped past `SWWAF_MAX_BANS`. A scanner whose +ban was dropped that way is refused and banned again by its next probe. A ban an +admin made, whose cause is `admin`, is never dropped and does not count toward +`SWWAF_MAX_BANS`. Such bans can never fill the table. An admin who wants to keep +a ban `smallwebwaf` made sets its cause to `admin`. -The netblock a ban covers is the client: its IPv4 address, or its IPv6 group of -`SWWAF_IPV6_GROUP_PREFIX` (a /64 by default). `SWWAF_BAN_SCOPE_V4_PREFIX` can -widen an IPv4 ban to the surrounding netblock. At the defaults a ban covers one -address or one /64, so a permanent ban does not reach neighbours who did -nothing. +The netblock a ban covers is the client: its IPv4 address, or its IPv6 group. +`SWWAF_BAN_SCOPE_V4_PREFIX` can widen an IPv4 ban to the surrounding netblock. +At the defaults a ban covers one address or one /64, so a permanent ban does not +reach neighbours who did nothing. -An offence is a request `smallwebwaf` holds against the client: one that carries -a clear sign of attack, breaks a limit, or is refused by a rule file, by the -Core Rule Set or for a missing or wrong token. Offences are counted by kind in -the client's history. Two kinds lead to a ban, each by its own rule: +An offence is a request `smallwebwaf` holds against the client. It is one that +carries a clear sign of attack, breaks a limit, or is refused by a rule file, by +the Core Rule Set or for a missing or wrong token. Offences are counted by kind +in the client's history. Two kinds lead to a ban, each by its own rule: -- A clear sign of attack: a request that matches a rule file rule whose action - is `ban`, or asks for a path in `SWWAF_TRAP_PATHS`. Examples are a probe for a - `.env` file or a `.git` directory, and a known scanner's user agent. +- A clear sign of attack: a request that matches a rule whose action is `ban`, + or asks for a path in `SWWAF_TRAP_PATHS`. Examples are a probe for a `.env` + file or a `.git` directory, and a known scanner's user agent. - The first one bans the netblock for `SWWAF_ATTACK_BAN_DURATION` (default `7d`). - Any further request from the netblock while that ban lasts shows it is malicious: the ban becomes permanent. - - Once that ban has run out, the netblock is served like any other, but its - next clear sign of attack bans it permanently at once. -- A broken limit: a request over a request limit, a response that takes the + - Once that ban has run out, the netblock is served like any other. Its next + clear sign of attack bans it permanently at once. +- A broken limit: a request over a rate limit, a response that takes the client's byte total past a byte limit, or more than - `SWWAF_ERROR_BURST_THRESHOLD` requests within a minute refused after a rule - file or Core Rule Set match, or for a missing or wrong token. - - The first such ban, or one that comes more than - `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`) after the last such ban - ended, lasts `SWWAF_LIMIT_BAN_DURATION` (default `1h`). + `SWWAF_ERROR_BURST_THRESHOLD` refusals within a minute. + - The first such ban lasts `SWWAF_LIMIT_BAN_DURATION` (default `1h`). So + does one that comes more than `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default + `24h`) after the last such ban ended. - Breaking a limit again within that window makes the new ban three times as long as the last one: one hour, then 3, 9, 27 and 81 hours. - A ban that would be longer than `SWWAF_MAX_BAN_DURATION` (default `7d`) is - permanent instead; with the defaults, that is the sixth ban in a row. - - Each such ban sets the client's counters for every limit back to zero, so - once it ends only new requests can break a limit again; the client's + permanent instead. At the defaults, that is the sixth ban in a row. + - Each such ban sets the client's counters for every limit back to zero. + Once it ends, only new requests can break a limit again. The client's history keeps its totals. A Core Rule Set match refuses only the request it matched, without a ban, -because the Core Rule Set has false positives; a `block` rule does the same. +because the Core Rule Set has false positives. A `block` rule does the same. Those refusals count toward the error burst, so a client that keeps setting them off is banned under the second rule. Requests refused under a ban are not -counted against limits, but they are counted in the client's history and in the +counted against limits. They are counted in the client's history and in the ban's notes. +A client the CrowdSec decision list lists is banned until the decision ends, +with the cause `crowdsec`. Such a ban is never made permanent. It does not make +the netblock's next ban for a broken limit longer. + Every ban carries notes with what an admin needs to decide whether to lift it (see "Persistent state"). An admin can add or lift a ban at any time by editing -`bans.json`, or through the ban endpoints when `SWWAF_ADMIN_TOKEN` is set (see -"Admin endpoints"). A ban lifted before it ends is kept, marked as lifted, and -does not count toward a longer ban; deleting its entry from `bans.json` forgets -it entirely. +`bans.json`, or through the ban endpoints when `SWWAF_ADMIN_TOKEN` is set. A ban +lifted before it ends is kept, marked as lifted, and does not count toward a +longer ban. Deleting its entry from `bans.json` forgets it entirely. -Clients of the AS numbers and countries listed in the biased thresholds, and -clients listed by a reputation source whose action is `limit:`, get -lower limits (see "Biased thresholds"), so the same rules ban them after fewer -requests. For them the result is always a ban, first a temporary one and then, -for repeated abuse, a permanent one. +Clients of the AS numbers and countries listed in the biased thresholds get +lower limits. So do clients a reputation source lists whose action is +`limit:`. The same rules then ban them after fewer requests. Some refusals make no ban at all. `SWWAF_DENY_NETS`, a blocklist or reputation -hit whose action is `deny`, and the country lists (`SWWAF_DENIED_COUNTRIES`, -`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`) refuse each request they cover and record -nothing in `bans.json`. +hit whose action is `deny`, and the country lists refuse each request they cover +and record nothing in `bans.json`. -## Configuration surface +## Settings -Conventions: lists are comma separated; netblocks are CIDR (a bare address means -/32 or /128); durations use Go syntax plus `d` for days (`90s`, `15m`, `24h`, -`7d`); byte sizes accept `K`, `M`, `G` suffixes, which are powers of 1024: `1K` -is 1024 bytes, `1M` is `1024K` and `1G` is `1024M`. Countries are two-letter ISO -codes in either case (`de` and `DE` are the same); a code that is not a country -code, such as `nk` (North Korea is `kp`), stops the start with a message naming -it. +Conventions: +- Lists are comma separated. Netblocks are CIDR; a bare address means /32 or + /128. Durations use Go's syntax plus `d` for days (`90s`, `15m`, `24h`, `7d`). + Sizes take `K`, `M` and `G`, powers of 1024. Countries are two-letter ISO + codes in either case. A code that is not a country, such as `nk`, stops the + start with a message naming it. - No setting is required. Every setting has a default chosen for a service facing the internet in 2026, or stays off until the operator supplies what it needs: an alert destination, an account key, a token. - Every setting's name starts with `SWWAF_`, since `smallwebwaf` shares its - container, and so its environment variables, with the app it protects. -- Any limit or threshold can be switched off with the value `off`, except - `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`. + container, and so its environment variables, with the app. +- `off` switches off: + - a timeout, a size limit, a rate limit, a byte limit or an anomaly + threshold; + - the Core Rule Set's anomaly threshold and the error burst threshold; + - the alert cooldown and the hourly alert cap. +- `off` is refused by `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, and by every + setting that is not a limit: the lookup timeout, the reputation cache and + timeout, the ban durations, the state write times, and the blocklist refresh. - A list set to an empty value is an empty list, and replaces the default. - Every setting may instead be given as a file holding the value, named by the setting's name with `_FILE` added, such as `SWWAF_ADMIN_TOKEN_FILE`, for - secrets and long lists. `SWWAF_LOG_REMOTE_TLS_CA_FILE`, whose value names a - file already, has no `_FILE` form. -- Settings, including those given as files, are read once at start; changing one - means restarting the container. The files `smallwebwaf` watches while it runs - are its state files, its rule files and the lookup database. -- Invalid configuration stops the process at start with a message naming the - variable. At start the effective configuration is logged with secrets masked. + secrets and long lists. `SWWAF_LOG_REMOTE_TLS_CA_FILE` names a file already + and has no `_FILE` form. +- Settings, files included, are read once at start. Changing one means + restarting the container. The files `smallwebwaf` watches while it runs are + its state files, its rule files and the lookup database. +- An invalid setting stops the process at start with a message naming the + variable. At start the settings in effect are logged, with secrets masked. The settings, by group: -- Core - - `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the application, - which listens there in the same container (see "Deployment"). - - `SWWAF_LISTEN_ADDR` (default `:8080`): the one listener, for the requests - `smallwebwaf` forwards and for its own endpoints (see "Admin endpoints"). - - `SWWAF_ADMIN_TOKEN`: bearer token for the ban endpoints and - `/_smallwebwaf/clients/`, a long random value, since they can be - reached from the internet. A token shorter than 32 characters stops the - start with a message naming the variable. Unset by default, which switches - them off; bans are then managed by editing `bans.json`. - - `SWWAF_INSTANCE_NAME` (default: the container's host name, which docker - sets to the first 12 characters of the container's id unless the - deployment names one): included in every log line, metric and alert. Set - it, for example to `fsn1app1/gitea`, for a name that stays the same when a - deploy replaces the container and that tells instances apart when several - report to one place. - - `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to log and alert - on every decision while refusing nothing. - - `SWWAF_TRUSTED_PROXIES` (default - `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`, the private address ranges): - netblocks whose `X-Forwarded-For` is believed, normally where traefik - reaches `smallwebwaf` from. The header is used only when the TCP peer is - inside this list; otherwise the peer's own address is the client, so a - client that connects directly cannot claim another address. A list given - replaces the default; set but empty, it trusts nothing. - - `SWWAF_IPV6_GROUP_PREFIX` (default `64`). - - `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): clients held in memory and - in `clients.json` (see "Persistent state"). - - `SWWAF_MAX_BANS` (default `5000`): bans `smallwebwaf` made, past, active - and permanent, held in memory and in `bans.json`. Bans an admin made are - kept besides these and never dropped (see "Bans"). -- Persistent state - - `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the JSON state files, - in a directory of their own beside the app's data, belonging to the - `smallwebwaf` user. A volume mounted there keeps the state when a deploy - replaces the container (see "Deployment"). - - `SWWAF_STATE_WRITE_DELAY` (default `10s`): after a ban is made, lifted or - made permanent, `bans.json` is written this long later with every change - made in between, so a burst of changes becomes one write. - - `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often `clients.json`, - `lookups.json`, `reputation.json` and `alerts.json` are written, and - `bans.json` when only the counts in its notes changed. -- Logging - - The request log on stdout is always on and has no switch. - - `SWWAF_LOG_LEVEL` (default `info`): for the process's own messages - (start-up, fetch failures, state writes), which are JSON lines on stdout - too, marked `"type":"process"`. It does not filter the request log. - - `SWWAF_LOG_REQUEST_HEADERS` (default - `accept,accept-language,accept-encoding,content-type,origin,range`): extra - request headers to record. `Authorization`, `Cookie` and `Set-Cookie` - values are never logged, only whether they were present. - - `SWWAF_LOG_REMOTE_URL`: when set, every log line is also sent to this - endpoint. Forms: `syslog+udp://host:514`, `syslog+tcp://host:514`, - `syslog+tls://host:6514`. - - `SWWAF_LOG_REMOTE_TLS_CA_FILE`: optional CA certificate for the `+tls` - forms. - - `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): lines held in memory while - the endpoint is unreachable; when full the oldest are dropped and counted. - - `SWWAF_LOG_REMOTE_FACILITY` (default `local0`), - `SWWAF_LOG_REMOTE_APP_NAME` (default `SWWAF_INSTANCE_NAME`): syslog header - fields. -- Metrics - - `SWWAF_METRICS_TOKEN`: bearer token a scraper sends for - `/_smallwebwaf/metrics`, a long random value. A token shorter than 32 - characters stops the start with a message naming the variable. Unset by - default, which switches the metrics off, since they would otherwise be - open to anyone on the internet. - - `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and countries - get their own series; the rest are summed as `other`. -- Static lists - - `SWWAF_ALLOW_NETS`: bypass everything (monitoring, the owner's own - networks). - - `SWWAF_RATE_LIMIT_EXEMPT_NETS`: bypass request and byte limits only; the - error burst threshold, attack detection and bans still apply. - - `SWWAF_DENY_NETS`: always refused. -- Request rate limits, per client (R1, R2). Breaking one bans the client (see - "Bans"), so the defaults sit several times above what one busy person - produces: a browser loading a heavy page makes a few hundred requests, a git - clone a handful, and several people often share one address. A machine that - talks to the app all day, such as a gitea Actions runner (see the notes - specific to gitea under "Deployment"), can still pass the day limit; its - address belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. - - `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), - `SWWAF_RATE_LIMIT_PER_HOUR` (default `10000`), `SWWAF_RATE_LIMIT_PER_DAY` - (default `50000`). - - `SWWAF_RATE_LIMIT_EXEMPT_PATHS`: path prefixes not counted (static assets, - health checks). -- Byte limits, per client. A response's bytes are counted when it ends, so every - default sits above the largest response allowed (`SWWAF_RESPONSE_MAX_BYTES`, 5 - GiB) and no single download breaks one. - - `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), - `SWWAF_BYTES_LIMIT_PER_HOUR` (default `20G`), `SWWAF_BYTES_LIMIT_PER_DAY` - (default `50G`). - - `SWWAF_BYTES_COUNT` (default `both`): `response`, `request` or `both`. -- Size and time limits, per request, in both directions. The client-facing - timeouts apply between the client and `smallwebwaf`, the app-facing ones - between `smallwebwaf` and the app, since a slow client and a slow app are - separate problems. There is one size limit for request bodies and one for - response bodies: `smallwebwaf` passes bodies through unchanged, so a limit on - each side would bound the same bytes and the lower one would always decide. - Bodies stream straight through, so a request body reaches the app while the - client is still sending it. - - `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take - to send its request line and headers, and then, from the end of the - headers, its body. - - `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest - request line and headers a client may send. Over it, Go's HTTP server - answers `431` and closes the connection, and nothing reaches the app. It - must be more than `4K`, and cannot be `off`: Go's HTTP server always has - such a limit, and reads 4 KiB past the one it is given before it refuses. - - `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open - connection may wait for its next request before `smallwebwaf` closes it. - It is longer than the 90 seconds after which traefik, by default, closes a - connection it is not using, so traefik closes first and never sends a - request on a connection `smallwebwaf` is closing. - - `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long `smallwebwaf` - may take to deliver one response to the client, from the end of the - request to the last byte. - - `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long `smallwebwaf` - may take to connect to the app and send it the whole request. - - `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may - take to send one whole response, from the end of the request to the last - byte. - - `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body, as - the client sends it and the app receives it. - - `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body, as - the app sends it and the client receives it. - - When a limit is passed after a request's line and headers have been read, - but before the response has started, `smallwebwaf` answers itself: `413` - for a request body that is too large, `408` for a client too slow to send - its body, `502` for a response that is too large, `504` for an app that is - too slow. A request that announces a body larger than its limit is refused - before anything reaches the app. Once the response has started it can only - be cut off, and the connection is closed. - - Since a request body streams through, each side can hold up the other: a - slow client slows the send to the app, and an app slow to take the body - slows the client's send. So while a request body is still on its way, a - request timeout that runs out, `SWWAF_CLIENT_REQUEST_TIMEOUT` or - `SWWAF_UPSTREAM_REQUEST_TIMEOUT`, answers `408` if `smallwebwaf` was - waiting for the client to send more at that moment, and `504` if it was - waiting for the app to take what it had. - - Go's HTTP server, on which `smallwebwaf` is built, reads a request's line - and headers before `smallwebwaf` sees the request. A client that takes - longer than `SWWAF_CLIENT_REQUEST_TIMEOUT` to send them gets either no - answer or an error answer from the server, mostly `400 Bad Request`; - either way the server closes its connection and nothing reaches the app. - Headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` are answered `431` by - the server itself. Neither request gets a line in the request log. - - A WebSocket connection leaves these limits behind once it is upgraded: it - stays open until either side closes it. -- Lookup of AS number and country (R7). On by default through GeoJS, which needs - no account or file, so that country lookup works with no setup. The file is - the alternative for a service that keeps its visitors' addresses on its own - host. - - `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. - - `SWWAF_LOOKUP_DB_PATH`: the file for `file`, the IPinfo Lite database in - its `.mmdb` form (`ipinfo_lite.mmdb`), one file that carries both country - and AS number; `smallwebwaf` reads its `asn`, `as_name` and `country_code` - fields. The operator downloads it with a free IPinfo account and mounts it - read-only. `smallwebwaf` never fetches it and holds no account or token - for it. A file that is missing or unreadable at start stops the start. - - Refreshing the file is the operator's business; IPinfo updates it daily. - `smallwebwaf` notices when the file is replaced and reads it again; a - replacement it cannot read is ignored, logged and sent as one `file_error` - alert, and the file it has stays in use. Mount the directory that holds - the file rather than the file itself: docker does not show a single - mounted file being replaced on the host. - - `geojs` asks the free GeoJS web service, which needs no account or key, - about up to 200 addresses in one request - (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), and reads each answer's - `country_code`, `asn` and `organization_name` (the AS name). An answer - without a `country_code`, which GeoJS sends for ranges it cannot place, - and an `asn` of `64512`, which it gives when it knows none, count as - unknown. GeoJS is told the address of every new visitor, whether or not a - setting uses the answer, since the request log, the metrics, the client's - history and ban notes carry the AS number and country. Private, loopback - and link-local addresses, which no source can place, are never sent. - - Each GeoJS answer is kept in memory for 7 days, apart from the table of - clients, so it outlasts a client dropped from that table; after 7 days the - client's next request asks again. Up to 100,000 answers are kept, about 15 - MiB; when that many are held, the answer used longest ago goes first. - Writing the answers to `lookups.json` and reading them back at start, so - that they survive a restart, comes in milestone 3 or later - (https://git.eeqj.de/sneak/smallwebwaf/issues/17); until then a restart - loses them. - - `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its - client's first answer when a setting needs it (see "Data flow for one - request"); GeoJS normally answers in a fraction of that. At most one - request to GeoJS is under way at a time, the addresses that arrive - meanwhile are asked about together in the next one, up to 200 per request - with the rest in the requests after it, and a request that takes longer - than `SWWAF_LOOKUP_TIMEOUT` is abandoned. A client whose answer has not - come in time counts as unknown until it comes: its later requests do not - wait, and its address is asked about again in the background. - - GeoJS publishes no rate limit, but its terms forbid "an excessive amount - of API requests", judged by GeoJS alone, and let it block a caller. While - GeoJS is slow, down or refusing `smallwebwaf`, clients with a kept answer - are unaffected and new clients count as unknown, so - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`, when set, refuses them. - `smallwebwaf` keeps asking with backoff and sends one `source_failure` - alert per cooldown. A service that cannot accept this uses the file. - - One source at a time: `SWWAF_LOOKUP_SOURCE=file` without - `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other - `SWWAF_LOOKUP_SOURCE`, the default `geojs` included, stops the start with - a message naming both. So does a setting that needs lookups while - `SWWAF_LOOKUP_SOURCE` is `off`: the country lists and the biased - thresholds below, `SWWAF_ADD_LOOKUP_HEADERS`, and the per-AS-number - anomaly thresholds. - - `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): pass `X-Client-ASN` and - `X-Client-Country` to the app. -- Country lists. Both are empty by default: the defaults judge a client by what - it does, not by where it comes from. Clients in `SWWAF_ALLOW_NETS` are not - checked. - - `SWWAF_DENIED_COUNTRIES`: for example `cn,ru,kp,ir,ua,by`. Every request - from a listed country is refused. - - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`: for example `us,de`. Only clients - placed in a listed country get through. Every request from any other - country is refused, and so is every request from a client the lookup - cannot place: one whose address is missing from the database, one GeoJS - has not answered for in time, and any client on a private address, such as - a visitor on the local network, another container or internal monitoring. - Those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let - unplaced clients through would let every new client in whenever GeoJS - stops answering. - - Both may be set. `SWWAF_DENIED_COUNTRIES` then adds nothing, since the - exclusive list already refuses every other country, and a code on both - lists stops the start with a message naming it. - - A refused request is answered with `SWWAF_BAN_RESPONSE`, as a banned - client is, as soon as the client's address has been looked up, before its - body is read. It skips the reputation checks, the limits, the rule files - and the Core Rule Set, is logged with the action `country_denied`, and is - counted in the metrics. It is a refusal, not a ban: it is not an offence - and makes no ban record. -- Biased thresholds (R8). The lists are empty by default, for the same reason as - the country lists; the request log and the metrics show which AS numbers and - countries a service's abuse comes from. - - `SWWAF_ASN_LIMIT_PERCENT`: for example `AS14061:50,AS16276:50,AS45102:25`. - Clients in a listed AS number get that percentage of every request and - byte limit, so the rules in "Bans" ban them after fewer requests than - others. `0` is a zero allowance: the client's first request breaks a limit - and bans it. - - `SWWAF_COUNTRY_LIMIT_PERCENT`: same form with ISO country codes, for - example `CN:25,RU:50`. Each client from a listed country gets that - percentage of the normal per-client limits. - - No budget is shared by a whole country or AS number: one abuser could use - it up and lock out everyone else there, a denial of service nobody chose - and the one this tool exists to prevent. Refusing a whole country is left - to the operator's choice, through the country lists. - - When more than one percentage applies to a client (its AS number, its - country, a reputation hit), the lowest applies. - - `SWWAF_ASN_BYTES_PERCENT`, `SWWAF_COUNTRY_BYTES_PERCENT`: optional - overrides applied to byte limits only, when the byte percentage should - differ from the request percentage. - - `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): for clients the lookup - cannot place. - - `SWWAF_ASN_LIMIT_PERCENT_URL`: optional URL of a text file of `AS:percent` - lines, so one abuse-source list can be shared by every `smallwebwaf` in - the fleet. It is fetched and refreshed like the blocklists. -- Attack detection (R9). By default the Core Rule Set reads the method, the URL - with its query string, and the headers of every request, which is where - automated attacks show: injection in query strings, path traversal, attacks - carried in headers, scanners' user agents. It reads no request body. On a code - forge, the example deployment, bodies carry what people write and publish, - such as issue and comment text, wiki pages, files saved in the web editor and - package descriptions, and when these hold shell commands or code the Core Rule - Set takes them for attacks. The default gives up refusing an attack carried in - a body; the rule files, the limits and the bans still apply to the client that - sends it, and `SWWAF_WAF_BODY_LIMIT` switches body inspection on. - - `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): directory of rule - files, read at start and again whenever a file in it changes; format under - "Rule files". - - `SWWAF_RULES_ENABLED` (default `true`): `false` skips rule files entirely. - - `SWWAF_WAF_MODE` (default `block`): `off`, `detect` (log and alert only), - or `block`. - - `SWWAF_WAF_PARANOIA_LEVEL` (default `1`), `SWWAF_WAF_ANOMALY_THRESHOLD` - (default `5`): the Core Rule Set's own two tuning values, at the Core Rule - Set's own defaults. - - `smallwebwaf` also changes the Core Rule Set 4.25.0 in six ways, since in - front of gitea it would otherwise refuse ordinary requests. The changes - hold in front of every app, and no setting undoes them: - - PUT, PATCH and DELETE are allowed methods besides GET, HEAD, POST and - OPTIONS; APIs, container image pushes and package uploads use them. - Other methods stay refused. - - The request header `Expect` is allowed, since curl and git send it - before some large uploads. `Content-Encoding` is allowed on a request - whose body the Core Rule Set does not read, which by default is every - request: git compresses a fetch request over 1 KiB and says so in that - header. On a body the Core Rule Set does read, the header stays - refused, since a compressed body cannot be inspected. The other - headers the Core Rule Set refuses, such as `Proxy`, stay refused. - - The `redirect_uri` parameter is not checked for a URL naming an IP - address or `localhost` (931100, 934110). Git Credential Manager, - git-credential-oauth and tea, which gitea registers for OAuth sign-in - out of the box, ask to be sent back to `http://127.0.0.1` on the - user's own machine, and the server never fetches that address. - - The query parameters in which gitea sends file paths, branch and - workflow names and the page to return to after signing in (`path`, - `files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`, - `artifactName` and `redirect_to`) are not checked against the Core - Rule Set's lists of system files (930120), shell paths (932160) and - command names (932260). In a repository any name on those lists can be - an ordinary file or branch, such as `.gitignore`, `package.json`, - `docker-compose.yml`, `bin/docker-entrypoint` or a branch named - `docker-build`, and gitea reads these values as names within a - repository or its own records, or as a page of its own site. Like the - other changes, this one holds in front of every app, not only gitea, - and no setting restores the three rules in those parameters. What only - these three rules refuse is let through there, and that is more than a - name such as `/etc/passwd` or `whoami` on its own: commands such as - `|cat /etc/passwd`, `wget http://…` and `nc -e /bin/sh …`, and - `file:///etc/passwd`, pass as well. Path traversal (`../`), SQL and - script injection and PHP, Java and Node.js code are still refused - there, and every other parameter keeps all three rules. These names, - and `redirect_uri` in the change before, are matched without regard to - case, as Coraza matches them, so `Path` or `PATH` is treated as - `path`. An app that uses one of these parameters as a file on the - server, or passes it to a shell, gets no help from the three rules - there (see "Risks the design has to handle"). - - The Core Rule Set reads the request without the `gitea_flash` and - `redirect_to` cookies, and does not check `Referer` for a Unix command - given without arguments (932340) or for Java starting a process - (944110). Gitea writes into `gitea_flash` a message for the next page - naming what was just done, such as a file deleted in the web editor or - a branch, milestone or project created, and into `redirect_to` the - page to return to after signing in, often the one the visitor clicked - "Sign in" on; `Referer` is the address of the page a request comes - from. In these cookies the Core Rule Set takes names such as - `package.json`, `document.write.js` or `Get-ChildItem.ps1`, and titles - that hold them, for attacks. In `Referer` it takes the address of a - search for one word, such as `env`, `set` or `last`, when the address - ends in it, and the address of OpenJDK's - `src/java.base/share/classes/java/lang/Runtime.java`, which holds both - `runtime` and `java.`. A browser sends such a cookie with every - request until gitea replaces or removes it, which gitea cannot do - while `smallwebwaf` refuses those requests, so one refusal would keep - the browser out of the whole site. A browser names a page in `Referer` - on everything the page loads and on every link followed from it, so - all of those would be refused. Gitea shows the message with any script - removed and returns only to a page of its own site. Like the other - changes, this one holds in front of every app: an attack sent in a - cookie of either name is not refused by the Core Rule Set, nor is a - Unix command without arguments or Java starting a process sent in - `Referer`. Every other cookie is read in full, and `Referer` keeps the - other rules, script and SQL injection among them. - - Responses are not inspected. A raw file from a repository, such as a - shell script, looks to the response rules like source code leaking - from the server. - - `SWWAF_WAF_DISABLED_RULES` (default - `920340,920420,920440,920640,930130,930140`): rule ids to switch off when - an app trips a false positive. The default switches off the rules that - refuse a request for the type of its body, when it is missing or not on - the Core Rule Set's short list (920340, 920420, 920640); for its file - extension, such as `.sh` or `.sql` (920440); and for a file or directory - name in its path, such as `.git/`, `.gitignore`, `Dockerfile`, - `package.json` or an editor's settings directory (930130, 930140). In - front of a code forge these refuse git over HTTP, container image and - package uploads, and views of ordinary files in a repository. At the site - root, where no app serves such files, the default rule file bans the - common probes these rules caught, such as `/.env`, `/.git/config`, - `/.aws/credentials`, `/.ssh/id_rsa`, `/.htpasswd` and `/wp-config.php.bak` - (see "Rule files"). A list given replaces the default, so include them in - it. - - `SWWAF_WAF_EXEMPT_PATHS`: path prefixes not inspected. - - `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A - size, such as `128K`, has the Core Rule Set read form data and multipart - bodies up to that size, streaming the rest of a longer one on without - holding it in memory, and JSON and XML bodies no larger than it, since - those cannot be read in part. Any other body is not read, since the Core - Rule Set would read it as form data, where binary content such as a git - push trips rules written for text. Body inspection suits apps whose forms - carry no code. In front of gitea it refuses issue and comment text, wiki - pages and files saved in the web editor that hold shell commands or code - (932125, 932235, 932250 and others), package descriptions that show code, - PyPI uploads (922130), and attachments named like `debug.log` or - `config.yml` (932180), until the rule ids the request log names are added - to `SWWAF_WAF_DISABLED_RULES`. - - `SWWAF_TRAP_PATHS`: paths the app never serves and only scanners ask for, - for example `/wp-login.php,/xmlrpc.php` in front of gitea. A request for - one is a clear sign of attack. This is the env-var short form of a `path` - rule with the `ban` action, for deployments that mount no rule files. - - `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): requests per client per - minute that `smallwebwaf` refused after a rule file or Core Rule Set - match, or for a missing or wrong token; more than this breaks a limit (see - "Bans"). A client trying one attack or token after another is refused many - times a minute; a person rarely more than a few times. Answers the app - gives are not counted, since they do not tell a scanner from an ordinary - client: in front of gitea, container image and package clients are - answered 404 by design, for each layer a push checks and each package a - lookup asks about, often hundreds of times a minute, and git and registry - clients are answered 401 at the start of every push, every fetch from a - private repository and every image pull. -- Bans (R5), following the rules under "Bans" - - `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign - of attack. - - `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken - limit. - - `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): breaking a limit again - within this time after a ban for a broken limit ended makes the next ban - three times as long. - - `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban that would be longer is - permanent instead. - - `SWWAF_BAN_RESPONSE` (default `403`): `403`, `429`, or `close` to drop the - connection without an answer. `403` makes a ban easy to recognise when - debugging. Behind traefik, `close` does not leave the client unanswered: - traefik answers `502`, as it does whenever its backend drops a connection. - - `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): widen to for example `24` to - ban the surrounding netblock. -- Reputation (R6). No source is on by default, for the reason the country lists - and the biased thresholds are empty: a source refuses the clients that a list - kept elsewhere names, or lowers their limits, while the defaults judge a - client only by what it does to this service. AbuseIPDB and CrowdSec also need - an account or an engine of the operator's own. Besides, the list best suited - to be on by default, Spamhaus DROP, must not be downloaded automatically more - than once an hour, and Spamhaus may block an address that downloads it too - often. Each `smallwebwaf` fetches its own copy, so on a host that runs several - apps with it, which share one address, downloads would often come less than an - hour apart. - - `SWWAF_BLOCKLIST_URLS`: text files of addresses and netblocks, one per - line; anything after a `;` or `#` on a line is ignored. For example the - Spamhaus DROP list, `https://www.spamhaus.org/drop/drop.txt`. Spamhaus's - terms for it, on its DROP page: automated downloads must be at least one - hour apart, and once a day is more than enough in most cases; a product - that uses it must credit The Spamhaus Project and keep the list's date and - copyright lines with the data. Refreshed every `SWWAF_BLOCKLIST_REFRESH` - (default `24h`); the last good copy is kept whole, comment lines included, - on failure and across restarts. - - `SWWAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:`, or - `log`. `deny` refuses every request from a listed address, which suits - lists of networks that send nothing legitimate, such as DROP. - `limit:` gives listed clients that percentage of every limit, so - they are banned after fewer requests; it suits a list of addresses shared - with ordinary visitors, such as Tor exits. - - `SWWAF_DNSBL_ZONES`: for example `dnsbl.dronebl.org`. Zones meant for mail - (lists of residential ranges) will block ordinary visitors and should not - be used; Spamhaus zones need their keyed query service, given as a zone - name containing the key. - - `SWWAF_DNSBL_RESOLVER`: optional resolver address, since public resolvers - are refused by several list operators. - - `SWWAF_ABUSEIPDB_KEY`, `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`), - `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`; the free tier allows 1000 - checks a day). Only clients that have already committed one offence are - queried, so the budget is spent on suspects. - - `SWWAF_CROWDSEC_LAPI_URL`, `SWWAF_CROWDSEC_LAPI_KEY`: optional. If a - CrowdSec engine exists on the host, pull its decision list on a schedule - and treat listed addresses as banned. This is how a fleet-wide blocklist - can arrive without `smallwebwaf` depending on CrowdSec. The list is kept - like a fetched blocklist, and a listed client's request makes a ban with - the cause `crowdsec` that lasts as long as CrowdSec's decision, so a long - list does not fill `bans.json`. - - `SWWAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:`, - or `log`, for DNSBL and API hits. Such verdicts are less certain than a - blocklist, so by default a listed client gets a quarter of every limit and - is banned after a quarter of the requests. - - `SWWAF_REPUTATION_CACHE_TTL` (default `24h`), `SWWAF_REPUTATION_TIMEOUT` - (default `2s`). -- Alerting (R3) - - `SWWAF_ALERT_WEBHOOK_URL`: JSON POST; schema below. - `SWWAF_ALERT_WEBHOOK_HEADERS`: optional `Name:value` pairs for - authentication. - - `SWWAF_ALERT_SLACK_WEBHOOK_URL`: Slack incoming webhook, formatted - message. - - `SWWAF_ALERT_NTFY_URL` (full topic URL), `SWWAF_ALERT_NTFY_TOKEN`: title, - priority and tags set from the event type. - - `SWWAF_ALERT_EVENTS` (default - `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): - which event types are sent. `source_failure` is a reputation source or - GeoJS failing or refusing `smallwebwaf`. `file_error` is a rule file or - state file edited while running that does not parse, a replacement lookup - database that cannot be read, or a state file that cannot be written. - - `SWWAF_ALERT_COOLDOWN` (default `15m`): the same event type for the same - client or netblock is not repeated within this time; a count of suppressed - repeats is included in the next one. - - `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): beyond this, alerts are rolled - into one summary per hour so a wide attack cannot flood the channel. -- Anomaly thresholds, alert only, nothing is blocked (R4). None is set by - default, and a threshold left unset sends no alert: these only alert, an alert - needs a destination only the operator can supply, and what counts as unusual - depends on each service's normal traffic, which the metrics show. - - Per client: `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`, - `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`, - `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE`, - `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR`. - - Per surrounding netblock (`SWWAF_ANOMALY_NET_V4_PREFIX` default `24`, - `SWWAF_ANOMALY_NET_V6_PREFIX` default `48`): - `SWWAF_ANOMALY_NET_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_NET_BYTES_PER_MINUTE`, `..._PER_HOUR`. - - Per AS number, which needs a lookup source: - `SWWAF_ANOMALY_ASN_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_ASN_BYTES_PER_MINUTE`, `..._PER_HOUR`. - - Whole service: `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_TOTAL_BYTES_PER_MINUTE`, `..._PER_HOUR`. - - Named netblocks: `SWWAF_WATCH_NETS`, for example - `office=203.0.113.0/24,scraper-x=198.51.100.0/22`, with - `SWWAF_WATCH_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_WATCH_BYTES_PER_MINUTE`, `..._PER_HOUR` applied to each named block - as a whole. - - Anomaly counting includes clients in `SWWAF_ALLOW_NETS` and - `SWWAF_RATE_LIMIT_EXEMPT_NETS`, since an exempt client misbehaving is - worth knowing about. +### Core + +- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app, which listens + there in the same container. +- `SWWAF_LISTEN_ADDR` (default `:8080`): the one listener, for the requests + `smallwebwaf` forwards and for its own endpoints. +- `SWWAF_ADMIN_TOKEN`: the bearer token for the ban endpoints and + `/_smallwebwaf/clients/`. A long random value, since the endpoints can be + reached from the internet. One shorter than 32 characters stops the start. + Unset by default, which switches the endpoints off; bans are then managed by + editing `bans.json`. +- `SWWAF_INSTANCE_NAME` (default: the container's host name, which docker sets + to the first 12 characters of the container's id): included in every log line, + metric and alert. Set it, for example to `fsn1app1/gitea`, to a name that + stays the same when a deploy replaces the container and tells instances apart. +- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to log and alert on + every decision while refusing nothing. +- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`, + the private address ranges): the netblocks whose `X-Forwarded-For` is + believed, normally where traefik reaches `smallwebwaf` from. A client that + connects directly cannot claim another address. A list given replaces the + default; set but empty, it trusts nothing. +- `SWWAF_IPV6_GROUP_PREFIX` (default `64`), from 32 to 128. +- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the clients held in memory and + in `clients.json`. +- `SWWAF_MAX_BANS` (default `5000`): the bans `smallwebwaf` made that are held, + past, active and permanent. Bans an admin made are kept besides, and never + dropped. + +### Persistent state + +- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the JSON state files, in a + directory of their own beside the app's data, belonging to the `smallwebwaf` + user. A volume mounted there keeps the state when a deploy replaces the + container. +- `SWWAF_STATE_WRITE_DELAY` (default `10s`): after a ban is made, lifted or made + permanent, `bans.json` is written this long later, with every change made in + between, so a burst of changes becomes one write. +- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is + written. + +### Logging + +- The request log on stdout is always on and has no switch. +- `SWWAF_LOG_LEVEL` (default `info`): for the process's own messages, which are + JSON lines on stdout too, marked `"type":"process"`. It does not filter the + request log. +- `SWWAF_LOG_REQUEST_HEADERS` (default + `accept,accept-language,accept-encoding,content-type,origin,range`): the + request headers to record. `Authorization`, `Cookie` and `Set-Cookie` values + are never logged, only whether they were present. +- `SWWAF_LOG_REMOTE_URL`: when set, every log line is also sent there, as + `syslog+udp://host:514`, `syslog+tcp://host:514` or `syslog+tls://host:6514`. +- `SWWAF_LOG_REMOTE_TLS_CA_FILE`: an optional CA certificate file for the `+tls` + form. +- `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the lines held in memory while + the server is unreachable. When full, the oldest are dropped and counted. +- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`) and `SWWAF_LOG_REMOTE_APP_NAME` + (default `SWWAF_INSTANCE_NAME`): the syslog header fields. + +### Metrics + +- `SWWAF_METRICS_TOKEN`: the bearer token a scraper sends for + `/_smallwebwaf/metrics`, a long random value. One shorter than 32 characters + stops the start. Unset by default, which switches the metrics off, since they + would otherwise be open to anyone on the internet. +- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and countries get + series of their own. The rest are summed as `other`. + +### Static lists + +- `SWWAF_ALLOW_NETS`: bypass everything (monitoring, the owner's own networks). +- `SWWAF_RATE_LIMIT_EXEMPT_NETS`: bypass the request and byte limits only. The + error burst, attack detection and bans still apply. +- `SWWAF_DENY_NETS`: always refused. + +### Request rate limits, per client + +Breaking one bans the client, so the defaults sit several times above what one +busy person produces. A browser loading a heavy page makes a few hundred +requests, a git clone a handful, and several people often share one address. A +machine that talks to the app all day, such as a gitea Actions runner, can still +pass the day limit. Its address belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. + +- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR` + (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`). +- `SWWAF_RATE_LIMIT_EXEMPT_PATHS`: path prefixes not counted, such as static + assets or health checks. + +### Byte limits, per client + +A response's bytes are counted when it ends, so every default sits above the +largest response allowed, and no single download breaks one. + +- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR` + (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`). +- `SWWAF_BYTES_COUNT` (default `both`): `response`, `request` or `both`. + +### Size and time limits + +These apply per request, in both directions. The client-facing timeouts apply +between the client and `smallwebwaf`. The app-facing ones apply between +`smallwebwaf` and the app. A slow client and a slow app are separate problems. +There is one size limit for request bodies and one for response bodies: +`smallwebwaf` passes bodies through unchanged, so a limit on each side would +bound the same bytes, and the lower one would always decide. Bodies stream +straight through, so a request body reaches the app while the client is still +sending it. + +- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to + send its request line and headers, and then, from the end of the headers, its + body. +- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request + line and headers. Over it, Go's HTTP server answers `431` and closes the + connection, and nothing reaches the app. It must be more than `4K` and cannot + be `off`: Go's HTTP server always has such a limit, and reads 4 KiB past the + one it is given before it refuses. +- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection + may wait for its next request before `smallwebwaf` closes it. It is longer + than the 90 seconds after which traefik, by default, closes an unused + connection. So traefik closes first, and never sends a request on a connection + `smallwebwaf` is closing. +- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long one response may + take to reach the client, from the end of the request to the last byte. +- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the + app and sending it the whole request may take. +- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take + to send one whole response, from the end of the request to the last byte. +- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. +- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. + +When a limit is passed after the request line and headers have been read, but +before the response has started, `smallwebwaf` answers itself: + +- `413` for a request body that is too large; +- `408` for a client too slow to send its body; +- `502` for a response that is too large; +- `504` for an app that is too slow. + +A request that announces a body larger than its limit is refused before anything +reaches the app. Once the response has started it can only be cut off, and the +connection is closed. + +Since a request body streams through, each side can hold up the other. While a +request body is still on its way, a request timeout that runs out answers `408` +if `smallwebwaf` was waiting for the client to send more at that moment. It +answers `504` if it was waiting for the app to take what it had. + +Go's HTTP server reads a request's line and headers before `smallwebwaf` sees +the request. A client that takes longer than `SWWAF_CLIENT_REQUEST_TIMEOUT` to +send them gets either no answer or an error answer from the server, mostly +`400`. Either way the server closes the connection and nothing reaches the app. +Headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` are answered `431` by the +server itself. Neither request gets a line in the request log. + +A WebSocket connection leaves these limits behind once it is upgraded. It stays +open until either side closes it. + +### Lookup of AS number and country + +On by default through GeoJS, which needs no account or file, so that country +lookup works with no setup. The file is the alternative for a service that keeps +its visitors' addresses on its own host. + +- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. +- `SWWAF_LOOKUP_DB_PATH`: the file for `file`: the IPinfo Lite database in its + `.mmdb` form (`ipinfo_lite.mmdb`), one file that carries both country and AS + number. `smallwebwaf` reads its `asn`, `as_name` and `country_code` fields. + The operator downloads it with a free IPinfo account and mounts it read-only. + `smallwebwaf` never fetches it and holds no account or token for it. A file + that is missing or unreadable at start stops the start. +- Refreshing the file is the operator's business; IPinfo updates it daily. + `smallwebwaf` notices when the file is replaced and reads it again. A + replacement it cannot read is ignored, logged and sent as one `file_error` + alert, and the file it has stays in use. Mount the directory that holds the + file rather than the file itself: docker does not show a single mounted file + being replaced on the host. +- `geojs` asks the free GeoJS web service about up to 200 addresses in one + request (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`). It reads each + answer's `country_code`, `asn` and `organization_name` (the AS name). An + answer without a `country_code`, which GeoJS sends for ranges it cannot place, + counts as unknown. So does an `asn` of `64512`, which GeoJS gives when it + knows none. GeoJS is told the address of every new visitor, whether or not a + setting uses the answer. The request log, the metrics, the client's history + and ban notes carry the AS number and country, so every answer is used. + Private, loopback and link-local addresses, which no source can place, are + never sent. +- Each GeoJS answer is kept for 7 days, in memory and in `lookups.json`, apart + from the table of clients, so it outlasts a client dropped from that table. + After 7 days the client's next request asks again. Up to 100,000 answers are + kept, about 15 MiB. When that many are held, the answer used longest ago goes + first. +- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its + client's first answer when a setting needs it. GeoJS normally answers in a + fraction of that. At most one request to GeoJS is under way at a time. The + addresses that arrive meanwhile are asked about together in the next one, up + to 200 per request, the rest in the requests after it. A request that takes + longer than `SWWAF_LOOKUP_TIMEOUT` is abandoned. A client whose answer has not + come in time counts as unknown until it comes. Its later requests do not wait, + and its address is asked about again in the background. +- GeoJS publishes no rate limit, but its terms forbid "an excessive amount of + API requests", judged by GeoJS alone, and let it block a caller. While GeoJS + is slow, down or refusing `smallwebwaf`, clients with a kept answer are + unaffected and new clients count as unknown. So + `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`, when set, refuses them. `smallwebwaf` + keeps asking with backoff and sends one `source_failure` alert per cooldown. A + service that cannot accept this uses the file. +- One source at a time. `SWWAF_LOOKUP_SOURCE=file` without + `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other source, stops + the start with a message naming both. So does a setting that needs lookups + while `SWWAF_LOOKUP_SOURCE` is `off`: the country lists, the biased + thresholds, `SWWAF_ADD_LOOKUP_HEADERS`, and the anomaly thresholds per AS + number. +- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): pass `X-Client-ASN` and + `X-Client-Country` to the app. + +### Country lists + +Both are empty by default: the defaults judge a client by what it does, not by +where it comes from. Clients in `SWWAF_ALLOW_NETS` are not checked. + +- `SWWAF_DENIED_COUNTRIES`: for example `cn,ru,kp,ir,ua,by`. Every request from + a listed country is refused. +- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`: for example `us,de`. Only clients + placed in a listed country get through. Every request from any other country + is refused. So is every request from a client the lookup cannot place: + - one whose address is missing from the database; + - one GeoJS has not answered for in time; + - any client on a private address, such as a visitor on the local network, + another container or internal monitoring. + + Those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let + unplaced clients through would let every new client in whenever GeoJS stops + answering. + +- Both may be set. `SWWAF_DENIED_COUNTRIES` then adds nothing, since the + exclusive list already refuses every other country. A code on both lists stops + the start with a message naming it. +- What a refused request gets is under "What happens to a request", step 6. + +### Biased thresholds + +The lists are empty by default, for the same reason as the country lists. The +request log and the metrics show which AS numbers and countries a service's +abuse comes from. + +- `SWWAF_ASN_LIMIT_PERCENT`: for example `AS14061:50,AS16276:50,AS45102:25`. + Clients in a listed AS number get that percentage of every request and byte + limit, so the rules in "Bans" ban them after fewer requests than others. `0` + is a zero allowance: the client's first request breaks a limit and bans it. +- `SWWAF_COUNTRY_LIMIT_PERCENT`: the same form with country codes, for example + `CN:25,RU:50`. +- No budget is shared by a whole country or AS number. One abuser could use it + up and lock out everyone else there: a denial of service nobody chose, and the + one this tool exists to prevent. Refusing a whole country is left to the + operator's choice, through the country lists. +- When more than one percentage applies to a client (its AS number, its country, + a reputation hit), the lowest applies. +- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT`: optional + overrides applied to the byte limits only, when the byte percentage should + differ from the request percentage. +- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): for clients the lookup cannot + place. +- `SWWAF_ASN_LIMIT_PERCENT_URL`: optional URL of a text file of `AS:percent` + lines, so one abuse-source list can be shared by every `smallwebwaf` in the + fleet. It is fetched and refreshed like the blocklists. + +### Attack detection + +By default the Core Rule Set reads the method, the URL with its query string, +and the headers of every request. That is where automated attacks show: +injection in query strings, path traversal, attacks carried in headers, +scanners' user agents. It reads no request body. On a code forge, the example +deployment, bodies carry what people write and publish: issue and comment text, +wiki pages, files saved in the web editor, package descriptions. When these hold +shell commands or code, the Core Rule Set takes them for attacks. The default +gives up refusing an attack carried in a body. The rule files, the limits and +the bans still apply to the client that sends it, and `SWWAF_WAF_BODY_LIMIT` +switches body inspection on. + +- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of rule + files, read at start and again whenever a file in it changes (see "Rule + files"). +- `SWWAF_RULES_ENABLED` (default `true`): `false` skips rule files entirely. +- `SWWAF_WAF_MODE` (default `block`): `off`, `detect` (log and alert only), or + `block`. +- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`) and `SWWAF_WAF_ANOMALY_THRESHOLD` + (default `5`): the Core Rule Set's own two tuning values, at its own defaults. +- `SWWAF_WAF_DISABLED_RULES` (default + `920340,920420,920440,920640,930130,930140`): rule ids to switch off when an + app trips a false positive. The default switches off six rules: + - 920340, 920420 and 920640 refuse a request for the type of its body, when + the type is missing or not on the Core Rule Set's short list; + - 920440 refuses a request for its file extension, such as `.sh` or `.sql`; + - 930130 and 930140 refuse a file or directory name in the path, such as + `.git/`, `.gitignore`, `Dockerfile`, `package.json` or an editor's + settings directory. + + In front of a code forge these refuse git over HTTP, container image and + package uploads, and views of ordinary files in a repository. At the site + root, where no app serves such files, the default rule file bans the common + probes these rules caught, such as `/.env`, `/.git/config`, + `/.aws/credentials`, `/.ssh/id_rsa`, `/.htpasswd` and `/wp-config.php.bak`. + A list given replaces the default, so include them in it. + +- `SWWAF_WAF_EXEMPT_PATHS`: path prefixes not inspected. +- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size, + such as `128K`, switches body inspection on. The Core Rule Set then reads form + data and multipart bodies up to that size, streaming the rest of a longer one + on without holding it in memory. It reads JSON and XML bodies no larger than + the size, since those cannot be read in part. Any other body is not read: the + Core Rule Set would read it as form data, where binary content such as a git + push trips rules written for text. Body inspection suits apps whose forms + carry no code. In front of gitea it refuses, until the rule ids the request + log names are added to `SWWAF_WAF_DISABLED_RULES`: + - issue and comment text, wiki pages and files saved in the web editor that + hold shell commands or code (932125, 932235, 932250 and others); + - package descriptions that show code; + - PyPI uploads (922130); + - attachments named like `debug.log` or `config.yml` (932180). +- `SWWAF_TRAP_PATHS`: paths the app never serves and only scanners ask for, for + example `/wp-login.php,/xmlrpc.php` in front of gitea. A request for one is a + clear sign of attack. This is the short form of a `path` rule with the `ban` + action, for deployments that mount no rule files. +- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the refusals per client per + minute after a rule file match, a trap path, a Core Rule Set match, or a + missing or wrong token. More than this breaks a limit (see "Bans"). A client + trying one attack or token after another is refused many times a minute; a + person rarely more than a few times. Answers the app gives are not counted, + since they do not tell a scanner from an ordinary client. In front of gitea, + container image and package clients are answered `404` by design, often + hundreds of times a minute. git and registry clients are answered `401` at the + start of every push, every fetch from a private repository and every image + pull. + +### The six changes to the Core Rule Set + +`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of +gitea it would otherwise refuse ordinary requests. The changes hold in front of +every app, and no setting undoes them. + +- `PUT`, `PATCH` and `DELETE` are allowed methods besides `GET`, `HEAD`, `POST` + and `OPTIONS`. APIs, container image pushes and package uploads use them. + Other methods stay refused. +- The request header `Expect` is allowed, since curl and git send it before some + large uploads. `Content-Encoding` is allowed on a request whose body the Core + Rule Set does not read, which by default is every request: git compresses a + fetch request over 1 KiB and says so in that header. On a body the Core Rule + Set does read, the header stays refused, since a compressed body cannot be + inspected. The other headers the Core Rule Set refuses, such as `Proxy`, stay + refused. +- The `redirect_uri` parameter is not checked for a URL naming an IP address or + `localhost` (931100, 934110). Git Credential Manager, git-credential-oauth and + tea, which gitea registers for OAuth sign-in, ask to be sent back to + `http://127.0.0.1` on the user's own machine. The server never fetches that + address. +- Ten query parameters are not checked against the Core Rule Set's lists of + system files (930120), shell paths (932160) and command names (932260): + `path`, `files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`, + `artifactName` and `redirect_to`. gitea sends file paths, branch and workflow + names, and the page to return to after signing in, in them. In a repository + any name on those lists can be an ordinary file or branch, such as + `.gitignore`, `package.json`, `docker-compose.yml`, `bin/docker-entrypoint` or + a branch named `docker-build`. gitea reads these values as names within a + repository or its own records, or as a page of its own site. What only these + three rules refuse is let through there. That is more than a name such as + `/etc/passwd` or `whoami` on its own: commands such as `|cat /etc/passwd`, + `wget http://…` and `nc -e /bin/sh …`, and `file:///etc/passwd`, pass as well. + Path traversal (`../`), SQL and script injection, and PHP, Java and Node.js + code are still refused there. Every other parameter keeps all three rules. + These names, and `redirect_uri`, are matched without regard to case, as Coraza + matches them. An app that uses one of these parameters as a file on the + server, or passes it to a shell, gets no help from the three rules there. See + "Risks the design has to handle". +- The Core Rule Set reads the request without the `gitea_flash` and + `redirect_to` cookies. It does not check `Referer` for a Unix command given + without arguments (932340) or for Java starting a process (944110). gitea + writes into `gitea_flash` a message for the next page naming what was just + done, and into `redirect_to` the page to return to after signing in. `Referer` + is the address of the page a request comes from. In these cookies the Core + Rule Set takes names such as `package.json`, `document.write.js` or + `Get-ChildItem.ps1`, and titles that hold them, for attacks. In `Referer` it + takes the address of a search for one word, such as `env`, `set` or `last`, + when the address ends in it, and the address of OpenJDK's + `src/java.base/share/classes/java/lang/Runtime.java`. A browser sends such a + cookie with every request until gitea replaces or removes it. gitea cannot do + that while `smallwebwaf` refuses those requests, so one refusal would keep the + browser out of the whole site. A browser names a page in `Referer` on + everything the page loads and on every link followed from it, so all of those + would be refused. gitea shows the message with any script removed and returns + only to a page of its own site. An attack sent in a cookie of either name is + not refused by the Core Rule Set. Nor is a Unix command without arguments, or + Java starting a process, sent in `Referer`. Every other cookie is read in + full, and `Referer` keeps the other rules, script and SQL injection among + them. +- Responses are not inspected. A raw file from a repository, such as a shell + script, looks to the response rules like source code leaking from the server. + +### Bans + +- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of + attack. +- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken limit. +- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): breaking a limit again within + this time after a ban for a broken limit ended makes the next ban three times + as long. +- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban that would be longer is + permanent instead. +- `SWWAF_BAN_RESPONSE` (default `403`): `403`, `429`, or `close` to drop the + connection without an answer. `403` makes a ban easy to recognise when + debugging. Behind traefik, `close` does not leave the client unanswered: + traefik answers `502`, as it does whenever its backend drops a connection. +- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): widen to for example `24` to ban + the surrounding netblock. + +### Reputation + +No source is on by default, for the reason the country lists and the biased +thresholds are empty. A source refuses the clients that a list kept elsewhere +names, or lowers their limits, while the defaults judge a client only by what it +does to this service. AbuseIPDB and CrowdSec also need an account or an engine +of the operator's own. Besides, the list best suited to be on by default, +Spamhaus DROP, must not be downloaded automatically more than once an hour. +Spamhaus may block an address that downloads it too often. Each `smallwebwaf` +fetches its own copy. On a host that runs several apps with it, which share one +address, downloads would often come less than an hour apart. + +- `SWWAF_BLOCKLIST_URLS`: text files of addresses and netblocks, one per line; + anything after a `;` or `#` on a line is ignored. For example the Spamhaus + DROP list, `https://www.spamhaus.org/drop/drop.txt`. Spamhaus's terms for it, + on its DROP page: automated downloads must be at least one hour apart, and + once a day is more than enough in most cases; a product that uses it must + credit The Spamhaus Project and keep the list's date and copyright lines with + the data. Refreshed every `SWWAF_BLOCKLIST_REFRESH` (default `24h`, never less + than `1h`). The last good copy is kept whole, comment lines included, on + failure and across restarts. +- `SWWAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:` or `log`. + `deny` refuses every request from a listed address, which suits lists of + networks that send nothing legitimate, such as DROP. `limit:` gives + listed clients that percentage of every limit, so they are banned after fewer + requests. It suits a list of addresses shared with ordinary visitors, such as + Tor exits. +- `SWWAF_DNSBL_ZONES`: for example `dnsbl.dronebl.org`. Zones meant for mail + (lists of residential ranges) will block ordinary visitors and should not be + used. Spamhaus zones need their keyed query service, given as a zone name + containing the key. +- `SWWAF_DNSBL_RESOLVER`: an optional resolver address, since public resolvers + are refused by several list operators. +- `SWWAF_ABUSEIPDB_KEY`, `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`) and + `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`; the free tier allows 1000 + checks a day). Only clients that have already committed one offence are + queried, so the budget is spent on suspects. +- `SWWAF_CROWDSEC_LAPI_URL` and `SWWAF_CROWDSEC_LAPI_KEY`: optional. If a + CrowdSec engine exists on the host, its decision list is pulled on a schedule + and listed addresses are treated as banned. This is how a fleet-wide blocklist + can arrive without `smallwebwaf` depending on CrowdSec. The list is kept like + a fetched blocklist. A listed client's request makes a ban with the cause + `crowdsec` that lasts as long as CrowdSec's decision, so a long list does not + fill `bans.json`. +- `SWWAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:` or + `log`, for DNS blocklist and AbuseIPDB hits. Such verdicts are less certain + than a blocklist, so by default a listed client gets a quarter of every limit + and is banned after a quarter of the requests. +- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`) and `SWWAF_REPUTATION_TIMEOUT` + (default `2s`). + +### Alerting + +- `SWWAF_ALERT_WEBHOOK_URL`: a JSON POST; schema under "Alert webhook schema". + `SWWAF_ALERT_WEBHOOK_HEADERS`: optional `Name:value` pairs for authentication. +- `SWWAF_ALERT_SLACK_WEBHOOK_URL`: a Slack incoming webhook, sent a formatted + message. +- `SWWAF_ALERT_NTFY_URL` (the full topic URL) and `SWWAF_ALERT_NTFY_TOKEN`: + title, priority and tags set from the event type. +- `SWWAF_ALERT_EVENTS` (default + `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): + which event types are sent. `source_failure` is a reputation source or GeoJS + failing or refusing `smallwebwaf`. `file_error` is a rule file or state file + edited while running that does not parse, a replacement lookup database that + cannot be read, or a state file that cannot be written. +- `SWWAF_ALERT_COOLDOWN` (default `15m`): the same event for the same netblock, + source, file or anomaly scope is not repeated within this time. A count of + suppressed repeats is included in the next one. +- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): beyond this, alerts are rolled into + one summary per hour, so a wide attack cannot flood the channel. + +### Anomaly thresholds + +These only alert; nothing is blocked. None is set by default, and a threshold +left unset sends no alert. An alert needs a destination only the operator can +supply, and what counts as unusual depends on each service's normal traffic, +which the metrics show. + +- Per client: `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`, + `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`, + `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` and + `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR`. +- Per surrounding netblock (`SWWAF_ANOMALY_NET_V4_PREFIX` default `24`, + `SWWAF_ANOMALY_NET_V6_PREFIX` default `48`): + `SWWAF_ANOMALY_NET_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, + `SWWAF_ANOMALY_NET_BYTES_PER_MINUTE` and `..._PER_HOUR`. +- Per AS number, which needs a lookup source: + `SWWAF_ANOMALY_ASN_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, + `SWWAF_ANOMALY_ASN_BYTES_PER_MINUTE` and `..._PER_HOUR`. +- The whole service: `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, + `SWWAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` and `..._PER_HOUR`. +- Named netblocks: `SWWAF_WATCH_NETS`, for example + `office=203.0.113.0/24,scraper-x=198.51.100.0/22`, with + `SWWAF_WATCH_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, + `SWWAF_WATCH_BYTES_PER_MINUTE` and `..._PER_HOUR` applied to each named + netblock as a whole. +- Anomaly counting includes clients in `SWWAF_ALLOW_NETS` and + `SWWAF_RATE_LIMIT_EXEMPT_NETS`, since an exempt client misbehaving is worth + knowing about. ## Rule files -A required feature: `smallwebwaf` reads every `*.rules` file in -`SWWAF_RULES_DIR` and checks each request against them. The files are plain text -meant to be edited by hand, so that a new scanning pattern seen in the request -log can be turned into a rule in one line. `smallwebwaf` watches the directory: -a file edited, added or removed there takes effect while it runs. +`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR` and checks each +request against them. The files are plain text meant to be edited by hand, so +that a new scanning pattern seen in the request log can be turned into a rule in +one line. `smallwebwaf` watches the directory: a file edited, added or removed +there takes effect while it runs. -- One rule per line, four fields separated by spaces or tabs; the fourth field +- One rule per line, four fields separated by spaces or tabs. The fourth field runs to the end of the line: ``` @@ -825,49 +818,49 @@ a file edited, added or removed there takes effect while it runs. - Blank lines and lines starting with `#` are ignored. - `id`: a short name of letters, digits, `-` and `_`, unique across all files. - It appears in the request log, metrics, alerts and ban notes. + It appears in the request log, the metrics, the alerts and the ban notes. - `target`: what the regex is matched against. - `path`: the URL path without the query, both as received and once percent-decoded, so an encoded probe cannot slip past. - `query`: the raw query string. - - `uri`: path and query together, both as received and once percent-decoded, - so an encoded probe cannot slip past. + - `uri`: the path and query together, both as received and once + percent-decoded. - `method`, `host`, `user_agent`, `referer`. - - `header:`: any one request header. - - Request bodies are not available to rule files; reading them is the Core + - `header:`: any one request header but `Host` and + `Transfer-Encoding`, which Go's HTTP server takes out of every request. + - Request bodies are not available to rule files. Reading them is the Core Rule Set's job, once `SWWAF_WAF_BODY_LIMIT` switches it on. - `action`: - `log`: note the match in the request log and do nothing else. - - `block`: refuse the request with 403. The client is not banned for it, but - the refusal counts toward the error burst (see "Bans"). + - `block`: refuse the request with `403`. The client is not banned for it, + but the refusal counts toward the error burst. - `ban`: the request is a clear sign of attack. Refuse it and ban the - client's netblock for seven days (`SWWAF_ATTACK_BAN_DURATION`); any - further request during those days, or a later clear sign of attack, makes - the ban permanent (see "Bans"). Use it only for requests no real visitor - sends, and anchor a path at the site root (`^/`): a file of the same name - deeper in a site can be ordinary content, such as a file in a gitea - repository. + client's netblock for `SWWAF_ATTACK_BAN_DURATION`. Any further request + during the ban, or a later clear sign of attack, makes the ban permanent. + Use it only for requests no real visitor sends. Anchor a path at the site + root (`^/`): a file of the same name deeper in a site can be ordinary + content, such as a file in a gitea repository. - `regex`: Go regular expression syntax (RE2). It has no backreferences or - lookaround, and in exchange matching time is linear in the input, so no rule - can be made to stall the proxy. `(?i)` at the front makes a rule - case-insensitive. A rule matches if the regex matches anywhere in the target; - anchor with `^` and `$` when that is not wanted. + lookaround. In exchange matching time is linear in the input, so no rule can + be made to stall the proxy. `(?i)` at the front makes a rule ignore case. A + rule matches if the regex matches anywhere in the target; anchor with `^` and + `$` when that is not wanted. - Files are read in name order (`00-default.rules` before `50-gitea.rules`), rules in line order. - Rules are compiled when their file is read and held in memory. At start, a line that does not parse, a regex that does not compile, or a duplicate id stops the process with a message naming the file and line. While running, the same faults leave the rules as they were, including the earlier version of - that file, and the log and one `file_error` alert name the file and line; once - the file is fixed, it is read again. A `SWWAF_RULES_DIR` that does not exist - stops the start with a message naming it. An empty directory is not an error: - the log says that no rules were loaded. To run without rule files, mount an - empty directory or set `SWWAF_RULES_ENABLED=false`. + that file. The log and one `file_error` alert name the file and line. Once the + file is fixed, it is read again. A `SWWAF_RULES_DIR` that does not exist stops + the start with a message naming it. An empty directory is not an error: the + log says that no rules were loaded. To run without rule files, mount an empty + directory or set `SWWAF_RULES_ENABLED=false`. - The image ships a default file in `SWWAF_RULES_DIR`. The app's Dockerfile can copy files of its own into it, beside the default. Mounting a directory over it replaces the defaults; mounting single files into it adds to them. Docker does not show a single mounted file being replaced on the host, which is how - many editors save, so rules meant to be edited while `smallwebwaf` runs belong + many editors save. So rules meant to be edited while `smallwebwaf` runs belong in a mounted directory, with a copy of the default file if the defaults are to stay. - In `SWWAF_MODE=observe` every action is logged as what would have happened and @@ -894,17 +887,16 @@ path-traversal uri block (\.\./){2,} empty-agent user_agent log ^$ ``` -It is kept short and limited to patterns that are wrong for every app, and its -path rules are anchored at the site root: in front of gitea, `/.env` is a probe, +It is kept short and limited to patterns that are wrong for every app. Its path +rules are anchored at the site root. In front of gitea, `/.env` is a probe, while `///src/branch/main/.env.example` is a file in a repository -that any visitor or search crawler may open. Its path rules ban the common -probes for secrets, version control directories, backups and logs. The Core Rule -Set's rules for file names and extensions (930130, 920440) refuse such names -anywhere in a path, which is why the default `SWWAF_WAF_DISABLED_RULES` switches -them off: a code forge serves such files deeper in its paths. Anything -app-specific belongs in a file the app's Dockerfile adds or the deployer mounts: -a request for `/wp-login.php`, for example, is a clear sign of attack in front -of gitea and an ordinary login in front of WordPress. +that any visitor or search crawler may open. The Core Rule Set's rules for file +names and extensions (930130, 920440) refuse such names anywhere in a path, +which is why the default `SWWAF_WAF_DISABLED_RULES` switches them off: a code +forge serves such files deeper in its paths. Anything app-specific belongs in a +file the app's Dockerfile adds or the deployer mounts. A request for +`/wp-login.php`, for example, is a clear sign of attack in front of gitea and an +ordinary login in front of WordPress: ``` # 50-gitea.rules: WordPress probes, which gitea never serves @@ -920,193 +912,210 @@ No database is used, and nothing is read from disk while serving a request. The files are meant for people as well: an admin can read or edit them at any time, and the running `smallwebwaf` takes the edit in. -- Files, each holding one kind of state: - - `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that - `smallwebwaf` made and every ban an admin made. Per entry: the netblock, - start, expiry (`null` for permanent), what caused it (`attack`, `limit`, - `admin` or `crowdsec`), a short reason, when it was lifted if an admin - lifted it, and a `notes` field (below). - - `clients.json`: per client, its minute, hour and day counters and its - history since it was first seen: first and last time seen, the AS number, - AS name and country last looked up and when, total requests and bytes in - each direction, how many requests were forwarded and how many refused, - responses by status class, and offences by kind. Clients that were never - banned are kept too, so every client's history survives a restart; - `GET /_smallwebwaf/clients/` shows it. - - `lookups.json`: GeoJS answers, one per client: the AS number, AS name and - country, when GeoJS was asked and when the answer was last used; up to - 100,000, each for 7 days. This file comes in milestone 3 or later; until - then the answers are kept in memory only. - - `reputation.json`: the last good copy of each list fetched from a URL, - cached DNSBL and reputation API verdicts, each with the time it was - fetched, and the AbuseIPDB checks spent today, so a restart does not reset - the daily budget. - - `alerts.json`: the anomaly counters for surrounding netblocks, AS numbers, - named netblocks and the whole service; for each event type and client or - netblock, when it was last sent and how many repeats the cooldown has held - back since; the alerts sent this hour; and alerts still waiting to be - sent. -- Ban notes. The notes on a ban hold what an admin needs to decide whether to - lift it, drawn from what `smallwebwaf` already knows; nothing is looked up to - fill them: - - the AS number, AS name and country, added when the lookup answers (empty - when lookups are off); - - what was broken: the rule ids and target that matched, or the limit, its - window, the count reached and the client's limit percentage with what set - it; and any reputation sources that listed the client; - - the request that caused the ban, the one that broke the limit or carried - the clear sign of attack: time, method, host, path with its query string, - status and user agent, each text cut to 256 bytes; - - how many requests counted toward the ban, and the time span over which - they came; - - the netblock's total requests since it was first seen, and the requests - refused under this ban so far, kept up to date while the ban lasts; - - how many earlier bans of each kind the netblock has had. -- Size and disk writes. Each file is rewritten whole, so each has a bound: - - `clients.json` takes about 1 KiB per client. Clients are dropped only when - the table is full, so on a public service the file grows to the default - `SWWAF_MAX_TRACKED_CLIENTS` of 20,000, about 20 MiB. Written every 15 - minutes, that is under 2 GiB of disk writes a day. - - `bans.json` takes about 1.2 KiB per ban and at most about 2.5 KiB, since - the notes hold one request and their texts are cut short. At the default - `SWWAF_MAX_BANS` of 5,000 it is about 6 MiB, and never more than about 12 - MiB, plus whatever bans an admin made. It is written when a ban is made, - lifted or made permanent, at most once every 10 seconds, and otherwise - with the 15-minute write, so its writes follow the bans made: with a full - file, a hundred new bans a day come to about 600 MiB of disk writes. - - `lookups.json` takes about 150 bytes per answer, about 15 MiB when full. - Written every 15 minutes, that is under 1.5 GiB of disk writes a day. - - `reputation.json` and `alerts.json` are usually a few MiB or less. - - Writing a file of these sizes takes well under a second on an ordinary - disk, in the background, so rewriting whole files needs nothing cleverer. -- Format: indented JSON with a top-level `version` number, entries sorted by - client address, times in RFC 3339 UTC, durations and sizes as plain numbers - with the unit in the field name. The aim is that a person can open `bans.json` - in an editor, find an address, and remove or add an entry. `clients.json` and - `lookups.json` put each entry on one line instead, which halves their size and - lets `grep` show everything about one client. -- Writing: - - serialise from a snapshot taken under the lock, so requests are not held - up while the file is written; - - write to a temporary file in the same directory, sync it, rename it over - the real name, sync the directory. A crash at any point leaves either the - old complete file or the new complete file, never a partial one; - - `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, - lifted or made permanent. Changes that only update the counts in its notes - wait for the write every `SWWAF_STATE_COUNTER_INTERVAL`, as all of - `clients.json`, `lookups.json`, `reputation.json` and `alerts.json` do. - All five are written on orderly shutdown (SIGTERM); - - before writing a file, `smallwebwaf` checks whether it changed on disk - since it last read or wrote it; if it did, `smallwebwaf` takes that edit - in first (below), so an admin's edit is never overwritten. -- Reading at start: - - Active bans go into an in-memory prefix lookup; everything else into maps. - Counter buckets and cache entries whose time has passed are discarded. - - A missing file means empty state and is normal on first run. - - A file that does not parse, or has an unknown `version`, stops the process - with a message naming the file and position. Starting with empty state - would silently forgive every repeat offender, and since writes are atomic - a broken file can only come from a hand edit, which the editor should hear - about. -- Edits while running: - - `smallwebwaf` watches `SWWAF_STATE_DIR` and notices a file that is edited, - replaced or added. It tells its own writes from an admin's by comparing - the file with what it last wrote. - - An edit that parses is taken in at once: what the file says replaces what - `smallwebwaf` held for that file. Removing a ban's entry from `bans.json` - lifts the ban; adding an entry bans. Changes `smallwebwaf` made after the - admin opened the file, such as a new ban, are lost when the admin saves - over them; most editors warn when a file changed on disk while it was - open. - - An edit that does not parse does not stop the running `smallwebwaf`. It - keeps the state it has, renames the edited file to `.bad` (for - example `bans.json.bad`) so the edit is kept, writes the file again from - memory, logs the file and the position of the error, and sends them in one - `file_error` alert. The admin fixes the `.bad` file and moves it back. -- `SWWAF_STATE_DIR` not writable at start: the process exits. A write that fails - while running: state stays correct in memory, the failure is logged, counted - in metrics, sent as a `file_error` alert once per cooldown, and retried at the - next write. -- What a hard kill can lose: ban changes from the last `SWWAF_STATE_WRITE_DELAY` - (10 seconds), and everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` - (15 minutes): counts and history, the counts in ban notes, GeoJS answers, - reputation verdicts and the AbuseIPDB count, anomaly counters and alert - cooldowns. Losing the volume loses ban history, not service. +The files, each holding one kind of state: + +- `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf` + made and every ban an admin made. Per entry: the netblock, start, expiry + (`null` for permanent), the cause (`attack`, `limit`, `admin` or `crowdsec`), + a short reason, when it was lifted if an admin lifted it, and the notes below. +- `clients.json`: per client, its minute, hour and day counters and its history + since it was first seen. The history holds: + - the first and last time seen; + - the AS number, AS name and country last looked up, and when; + - total requests, and how many were forwarded and how many refused; + - bytes in each direction; + - responses by status class; + - offences by kind. + + Clients that were never banned are kept too, so every client's history + survives a restart. + +- `lookups.json`: GeoJS answers, one per client: the AS number, AS name and + country, when GeoJS was asked and when the answer was last used. Up to + 100,000, each for 7 days. +- `reputation.json`: the last good copy of each list fetched from a URL; the + cached DNS blocklist verdicts and AbuseIPDB scores, each with the time it was + fetched; and the AbuseIPDB checks spent today, so that a restart does not + reset the daily budget. +- `alerts.json`: the anomaly counters; for each event and netblock, source, file + or anomaly scope, when it was last sent and how many repeats the cooldown has + held back since; the alerts sent this hour; and the alerts still waiting to be + sent. + +Ban notes hold what an admin needs to decide whether to lift a ban, drawn from +what `smallwebwaf` already knows. Nothing is looked up to fill them: + +- the AS number, AS name and country, added when the lookup answers (empty when + lookups are off); +- what was broken: the rule id and target that matched, or the trap path, or the + limit, its window, the count reached and the client's limit percentage with + what set it; and any reputation sources that listed the client; +- the request that caused the ban: time, method, host, path with its query + string, status and user agent, each text cut to 256 bytes; +- how many requests counted toward the ban, and the window over which they came; +- the netblock's total requests since it was first seen, and the requests + refused under this ban so far, kept up to date while the ban lasts; +- how many earlier bans of each kind the netblock has had. + +Size and disk writes. Each file is rewritten whole, so each has a bound: + +- `clients.json` takes about 1 KiB per client. On a public service the file + grows to the default `SWWAF_MAX_TRACKED_CLIENTS` of 20,000, about 20 MiB. + Written every 15 minutes, that is under 2 GiB of disk writes a day. +- `bans.json` takes about 1.2 KiB per ban and at most about 2.5 KiB, since the + notes hold one request and their texts are cut short. At the default + `SWWAF_MAX_BANS` of 5,000 it is about 6 MiB, and never more than about 12 MiB, + plus whatever bans an admin made. +- `lookups.json` takes about 150 bytes per answer, about 15 MiB when full. +- `reputation.json` and `alerts.json` are usually a few MiB or less. +- Writing a file of these sizes takes well under a second on an ordinary disk, + in the background, so rewriting whole files needs nothing cleverer. + +Format: indented JSON with a top-level `version` number, entries sorted by +client address, times in RFC 3339 UTC, durations and sizes as plain numbers with +the unit in the field name. The aim is that a person can open `bans.json` in an +editor, find an address, and remove or add an entry. `clients.json` and +`lookups.json` put each entry on one line instead. That halves their size and +lets `grep` show everything about one client. + +Writing: + +- serialise from a snapshot taken under the lock, so requests are not held up + while the file is written; +- write to a temporary file in the same directory, sync it, rename it over the + real name, sync the directory. A crash at any point leaves either the old + complete file or the new complete file, never a partial one; +- `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted + or made permanent. Changes that only update the counts in its notes wait for + the write every `SWWAF_STATE_COUNTER_INTERVAL`, as the other four files do. + All five are written on orderly shutdown (`SIGTERM`); +- before writing a file, `smallwebwaf` checks whether it changed on disk since + it last read or wrote it. If it did, `smallwebwaf` takes that edit in first, + so an admin's edit is never overwritten. + +Reading at start: + +- Active bans go into an in-memory prefix lookup; everything else into maps. + Counter buckets and cache entries whose time has passed are discarded. +- A missing file means empty state, and is normal on first run. +- A file that does not parse, or has an unknown `version`, stops the process + with a message naming the file and position. Starting with empty state would + silently forgive every repeat offender. Since writes are atomic, a broken file + can only come from a hand edit, which the editor should hear about. + +Edits while running: + +- `smallwebwaf` watches `SWWAF_STATE_DIR` and notices a file that is edited, + replaced or added. It tells its own writes from an admin's by comparing the + file with what it last wrote. +- An edit that parses is taken in at once: what the file says replaces what + `smallwebwaf` held for that file. Removing a ban's entry from `bans.json` + lifts the ban; adding an entry bans. Changes `smallwebwaf` made after the + admin opened the file, such as a new ban, are lost when the admin saves over + them. Most editors warn when a file changed on disk while it was open. +- An edit that does not parse does not stop the running `smallwebwaf`. It keeps + the state it has and renames the edited file to `.bad` (for example + `bans.json.bad`), so the edit is kept. It writes the file again from memory, + logs the file and the position of the error, and sends them in one + `file_error` alert. The admin fixes the `.bad` file and moves it back. + +`SWWAF_STATE_DIR` not writable at start: the process exits. A write that fails +while running: state stays correct in memory, the failure is logged, counted in +the metrics, sent as a `file_error` alert once per cooldown, and retried at the +next write. + +What a hard kill can lose: ban changes from the last `SWWAF_STATE_WRITE_DELAY` +(10 seconds), and everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` +(15 minutes): counts and history, the counts in ban notes, GeoJS answers, +reputation verdicts and the AbuseIPDB count, anomaly counters and alert +cooldowns. Losing the volume loses ban history, not service. ## Request log One JSON object per line on stdout for every request, including refused ones, -apart from those Go's HTTP server ends before `smallwebwaf` sees them (see -"Configuration surface", size and time limits). stdout is always on. When -`SWWAF_LOG_REMOTE_URL` is set the same lines are also sent to the remote -endpoint, so a deployment can stop depending on docker's log handling while -`docker logs` keeps working. +apart from those Go's HTTP server ends before `smallwebwaf` sees them (see "Size +and time limits"). stdout is always on. When `SWWAF_LOG_REMOTE_URL` is set the +same lines are also sent there, so a deployment can stop depending on docker's +log handling while `docker logs` keeps working. - Standard web log fields: `time` (RFC 3339 with milliseconds), `instance`, `client_ip`, `method`, `scheme`, `host`, `path`, `query`, `protocol`, `status`, `request_bytes`, `response_bytes`, `referer`, `user_agent`. -- Request detail: `request_id` (generated if traefik did not supply one, and - passed to the app), `peer_ip` (the TCP peer, normally traefik), - `forwarded_for` (the header as received), `client_group` (the /64 or - configured prefix used for counting), `asn`, `as_name`, `country`, - `content_type`, `content_length`, the headers named in - `SWWAF_LOG_REQUEST_HEADERS`, `has_authorization` and `has_cookie` as booleans, - `websocket` when the connection was upgraded. +- Request detail: + - `request_id`, generated if traefik did not supply one, and passed to the + app; + - `peer_ip`, the TCP peer, normally traefik; + - `forwarded_for`, the header as received; + - `client_group`, the /64 or configured prefix used for counting; + - `asn`, `as_name`, `country`; + - `content_type`, `content_length`, and the headers named in + `SWWAF_LOG_REQUEST_HEADERS`; + - `has_authorization` and `has_cookie`, as booleans; + - `websocket`, when the connection was upgraded. - Response detail: `response_content_type`, `upstream_status` (differs from `status` when `smallwebwaf` answered itself), `cache_control`, `location` on redirects, `aborted` when the client went away early. -- Decision: `action` (`forward`, `banned`, `denied`, `country_denied`, - `rate_limited`, `rule_blocked`, `waf_blocked`, `too_large`, `timed_out`, - `upstream_error`, and `admin` for a request `smallwebwaf` answered at one of - its own endpoints), `would_action` in `observe` mode, `limit_percent` and - which rule set it, `counts` (the client's minute, hour and day request and - byte totals after this request), `limit_hit` (which window), `rule_ids` (rule - file rules that matched), `waf_rule_ids`, `waf_score`, `reputation` (sources - that listed the client), `offence` when one was recorded, `ban_expires`. +- Decision: + - `action`: `forward`, `banned`, `denied`, `country_denied`, `rate_limited`, + `rule_blocked`, `waf_blocked`, `too_large`, `timed_out`, `upstream_error`, + or `admin` for a request `smallwebwaf` answered at one of its own + endpoints; + - `would_action`, in `observe` mode; + - `limit_percent`, and which setting gave it; + - `counts`, the client's minute, hour and day request and byte totals after + this request; + - `limit_hit`, which window; + - `rule_ids`, the rule file rules that matched; + - `waf_rule_ids` and `waf_score`; + - `reputation`, the sources that listed the client; + - `offence`, when one was recorded; + - `ban_expires`. - Timings in milliseconds: `duration_total`, `duration_checks` (everything `smallwebwaf` did before forwarding), `duration_waf`, - `duration_upstream_connect`, `duration_upstream_first_byte`, + `duration_upstream_connect`, `duration_upstream_first_byte` and `duration_upstream_total`. - Bodies are never logged. Query strings are logged as received; an app that carries secrets in query strings needs that fixed in the app. - The process's own messages share the stream as JSON lines with `"type":"process"`; request lines carry `"type":"request"`. - Remote sending: - - syslog forms send each line as the message of an RFC 5424 record, with - octet-counted framing on TCP and TLS; + - each line is the message of an RFC 5424 syslog record, with octet-counted + framing on TCP and TLS; - sending happens on its own goroutine from a bounded buffer - (`SWWAF_LOG_REMOTE_BUFFER`). An unreachable or slow endpoint never delays - a request and never stops stdout; it reconnects with backoff, drops the - oldest lines when the buffer is full, and counts the drops in metrics. UDP - gives no delivery signal at all and is offered only for compatibility. + (`SWWAF_LOG_REMOTE_BUFFER`). An unreachable or slow server never delays a + request and never stops stdout. The sender reconnects with backoff, drops + the oldest lines when the buffer is full, and counts the drops in the + metrics. UDP gives no delivery signal at all and is offered only for + compatibility. ## Metrics endpoint Prometheus text format at `/_smallwebwaf/metrics`, for a request carrying -`SWWAF_METRICS_TOKEN` (see "Admin endpoints"). While the token is unset, the -metrics are off. The token is separate from `SWWAF_ADMIN_TOKEN`, so a scraper -that holds it cannot manage bans. +`SWWAF_METRICS_TOKEN`. While the token is unset, the metrics are off. The token +is separate from `SWWAF_ADMIN_TOKEN`, so a scraper that holds it cannot manage +bans. - Traffic: requests and bytes in and out, by status class and `action`; request duration and upstream duration histograms; requests in flight. -- Limits and bans: limit hits by window and kind (requests or bytes), size and - time limit hits by limit, offences by kind, bans created by cause, permanent - bans, active bans (gauge); requests refused by the country lists, by country. +- Limits and bans: limit hits by window and kind (requests, bytes or refusals); + size and time limit hits by limit; offences by kind; bans made by cause; + permanent bans; active bans; requests refused by the country lists, by + country. - Attack detection: rule file matches by rule id and action, and the number of - rules loaded; Core Rule Set matches by mode and rule id (label limited to the - rules that actually fired). + rules loaded; Core Rule Set matches by mode and rule id, limited to the rules + that actually fired. - Lookup and reputation: requests and bytes by AS number and by country, limited - to the `SWWAF_METRICS_TOP_N` (default `50`) busiest of each with the rest - summed as `other`, so the label set stays bounded; reputation queries, hits, - failures and remaining daily budget by source; GeoJS requests and failures, - and clients that counted as unknown because it did not answer in time; age of - each blocklist and lookup database. -- Housekeeping: tracked clients (gauge), state file writes, write failures, last - successful write time and size per file; files read again after an edit, and - edits set aside because they did not parse; alerts sent, failed and suppressed - by destination; remote log lines sent, dropped and buffer depth; the standard - Go runtime and process metrics. -- No metric carries a client IP address as a label; per-address questions are + to the `SWWAF_METRICS_TOP_N` busiest of each with the rest summed as `other`, + so the label set stays bounded; reputation queries, hits, failures and + remaining daily budget by source; GeoJS requests and failures, and clients + that counted as unknown because it did not answer in time; when each list was + last fetched and the lookup database last read. +- Housekeeping: tracked clients; state file writes, write failures, last + successful write time and size per file; edits taken in, and edits set aside + because they did not parse; alerts sent, failed, suppressed and dropped by + destination; remote log lines sent, dropped and buffer depth; the standard Go + runtime and process metrics. +- No metric carries a client IP address as a label. Per-address questions are answered by the request log and `GET /_smallwebwaf/clients/`. ## Admin endpoints @@ -1118,29 +1127,27 @@ Admins and scrapers reach these endpoints through traefik, like any other request. - `GET /_smallwebwaf/healthz`: answers `200` with `ok` to anyone, without a - token, while `smallwebwaf` is running; it does not ask the app, which the - container's health check does (see "Deployment"). It is answered before any - check, so a health checker never needs an exemption and is never refused, for - example by `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`. -- `GET /_smallwebwaf/metrics`: the metrics (see "Metrics endpoint"); needs - `SWWAF_METRICS_TOKEN`. -- `GET /_smallwebwaf/bans`, `POST /_smallwebwaf/bans` (client or netblock, - duration, reason), `DELETE /_smallwebwaf/bans/`: need + token, while `smallwebwaf` is running. It does not ask the app; the + container's health check does. It is answered before any check, so a health + checker never needs an exemption and is never refused, for example by + `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`. +- `GET /_smallwebwaf/metrics`: the metrics; needs `SWWAF_METRICS_TOKEN`. +- `GET /_smallwebwaf/bans`, `POST /_smallwebwaf/bans` (a client or netblock, a + duration, a reason) and `DELETE /_smallwebwaf/bans/`: need `SWWAF_ADMIN_TOKEN`. Editing `bans.json` does the same without a token. -- `GET /_smallwebwaf/clients/`: needs `SWWAF_ADMIN_TOKEN`. Current counters, - history, lookup result, reputation, offences, and bans with their notes; for - answering "why was this address refused" and "what has this netblock been - doing". -- A token is sent as `Authorization: Bearer `, and one set shorter than - 32 characters stops the start (see "Configuration surface"). While a token is - unset, the endpoints that need it answer `404`, as does any other path under - the prefix. +- `GET /_smallwebwaf/clients/`: needs `SWWAF_ADMIN_TOKEN`. The client's + current counters and history, which holds its lookup result and offences, and + its bans with their notes. It answers "why was this address refused" and "what + has this netblock been doing". +- A token is sent as `Authorization: Bearer `. One set shorter than 32 + characters stops the start. While a token is unset, the endpoints that need it + answer `404`, as does any other path under the prefix. - Apart from the health check, these requests go through every check that any - other request goes through, and are answered at the point where another + other request goes through. They are answered at the point where another request would be forwarded to the app: a banned or refused client stays refused, and each request counts toward the client's limits. A missing or wrong token is answered `401`, in `SWWAF_MODE=observe` too, and counts toward - the error burst, so a client guessing tokens is banned once it passes + the error burst. A client guessing tokens is therefore banned once it passes `SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute. A client in `SWWAF_ALLOW_NETS` skips the checks but still needs the token. - An admin whose own address is banned lifts the ban by editing `bans.json`. @@ -1149,23 +1156,25 @@ request. One JSON object per alert: -- `instance`, `time`, `event` (one of the `SWWAF_ALERT_EVENTS` values) -- `client`, `netblock`, `asn`, `as_name`, `country` -- `reason`: short human-readable sentence +- `instance`, `time`, `event` (one of the `SWWAF_ALERT_EVENTS` values, or + `summary` for the hourly summary); +- `client`, `netblock`, `asn`, `as_name`, `country`; +- `reason`: a short human-readable sentence; - `detail`: event-specific fields, for example `window`, `count`, `limit`, `limit_percent`, `rule_ids`, `path`, `ban_expires`, for a ban its notes, and - for `file_error` the file and the position of the error -- `suppressed_repeats`: number of identical alerts held back by the cooldown + for `file_error` the file and the position of the error; +- `suppressed_repeats`: the number of identical alerts held back by the + cooldown. ## Deployment -The recommended deployment puts `smallwebwaf` inside the app's own container: -the app's Dockerfile builds `FROM` the `smallwebwaf` image. Each app remains one -container, deployed as before, and an app is hardened just by changing its base -image and perhaps installing on top of it, from nixpkgs, the packages it needs. -In the container, `smallwebwaf` listens on port 8080, which traefik routes to, -and forwards each request to the app on `127.0.0.1:8081`, its default backend -(`SWWAF_UPSTREAM_URL`). No setting is required. +`smallwebwaf` runs inside the app's own container: the app's Dockerfile builds +`FROM` the `smallwebwaf` image. Each app remains one container, deployed as +before. An app is hardened just by changing its base image, and perhaps +installing on top of it, from nixpkgs, the packages it needs. In the container, +`smallwebwaf` listens on port 8080, which traefik routes to, and forwards each +request to the app on `127.0.0.1:8081`, its default `SWWAF_UPSTREAM_URL`. No +setting is required. The image holds: @@ -1173,16 +1182,16 @@ The image holds: digest. The image moves to the next LTS release when that ships. - Three packages from Ubuntu, `ca-certificates`, `nix-bin` and `runit`, installed from a dated snapshot of Ubuntu's archive and checked by hash (see - "Packages from Ubuntu" below). The Ubuntu image has no CA certificates, and - they come with `nix-bin` only because a library it uses recommends them, so + "Packages from Ubuntu"). The Ubuntu image has no CA certificates. They come + with `nix-bin` only because a library it uses recommends them, so `ca-certificates` is installed by name. Without it Nix cannot download packages, and `smallwebwaf`, unable to reach GeoJS, would count every visitor as coming from an unknown country. - Nix, the package manager, from Ubuntu's own `nix-bin` package, and nixpkgs, the collection of packages Nix installs from, fixed at one commit (see - "Packages from nixpkgs" below). Root uses Nix directly, and no Nix daemon runs - in the container. Nix run by root expects a group of build users, `nixbld`, - which `nix-bin` does not create, so the image writes `build-users-group =` to + "Packages from nixpkgs"). Root uses Nix directly, and no Nix daemon runs in + the container. Nix run by root expects a group of build users, `nixbld`, which + `nix-bin` does not create. So the image writes `build-users-group =` to `/etc/nix/nix.conf`, and root's builds then run without build users. - runit, from Ubuntu's own `runit` package, and `runsvinit` as the entrypoint, as the owner's code style guide asks for service containers. Neither Ubuntu @@ -1190,26 +1199,26 @@ The image holds: (`github.com/peterbourgon/runsvinit`) at a fixed commit hash. Its repository is archived and has not changed since 2015, and its last tag is `v2.0.0`. It has no `go.mod`, and `go build` of its directory needs one, so the build - writes one; since `runsvinit` uses only Go's standard library, that file names + writes one. Since `runsvinit` uses only Go's standard library, that file names nothing else. `runsvinit` starts runit's `runsvdir`, which starts a `runsv` - for each directory under `/etc/service`; each `runsv` runs the `run` script in + for each directory under `/etc/service`. Each `runsv` runs the `run` script in its directory, and runs it again whenever it exits. Ubuntu's runit looks for services in `/etc/service` too, so when `docker stop` has `runsvinit` stop each service with runit's `sv`, `sv` finds it. - The `smallwebwaf` binary, and a user of its own, `smallwebwaf` (uid and gid 65532). -- The service directory `/etc/service/smallwebwaf`, whose `run` script waits one +- The service directory `/etc/service/smallwebwaf`. Its `run` script waits one second (`sleep 1`), makes `SWWAF_STATE_DIR` and every file in it belong to the `smallwebwaf` user, and starts `smallwebwaf` as that user with runit's `chpst`. - The directory `/var/lib/smallwebwaf` for the state files, and - `/etc/smallwebwaf/rules.d` with the default rule file (see "Rule files"). + `/etc/smallwebwaf/rules.d` with the default rule file. - Port 8080 declared (`EXPOSE 8080`), and the health check described below (`HEALTHCHECK`). -Every `run` script, the app's included, is a bash script that starts with -`#!/usr/bin/env bash` and `set -euo pipefail` and puts its code in a `main` -function, called on its last line, as the owner's code style guide asks; Ubuntu +Every `run` script, the app's included, is a bash script. It starts with +`#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main` +function, called on its last line, as the owner's code style guide asks. Ubuntu ships bash. Beyond its `FROM` line, the app's Dockerfile adds: @@ -1217,13 +1226,13 @@ Beyond its `FROM` line, the app's Dockerfile adds: - its binary; - any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.`; - its runit service: a directory under `/etc/service` named after the app, never - `smallwebwaf`, whose `run` script waits one second (`sleep 1`) and then starts + `smallwebwaf`. Its `run` script waits one second (`sleep 1`) and then starts the app with `chpst`, listening on `127.0.0.1:8081`, as a user of its own that the Dockerfile creates with `useradd`. That user is neither root nor `smallwebwaf`, so that the app cannot touch the state files or the process of `smallwebwaf`. -It sets no `ENTRYPOINT` or `USER` of its own: the container must start +It sets no `ENTRYPOINT` or `USER` of its own. The container must start `runsvinit`, as root, so that it can start each service as its own user. An example, for an app whose binary is `app` and which takes the address it @@ -1231,7 +1240,7 @@ listens on and the proxies it trusts as options of its own: ```dockerfile # The smallwebwaf image, pinned by digest. -FROM /smallwebwaf: +FROM /smallwebwaf@sha256: # Packages the app needs, if any, from the nixpkgs in the image. RUN nix-env -iA nixpkgs.git @@ -1260,55 +1269,63 @@ main() { main "$@" ``` -Packages from Ubuntu: the image installs `ca-certificates`, `nix-bin` and -`runit` from Ubuntu's snapshot service, which serves the archive as it was at a -given moment, rather than from the archive itself, whose packages change with -every update. The image's Dockerfile names that moment, in apt's -`--snapshot 20261001T000000Z` form, on both `apt-get update` and -`apt-get install`. That moment is never earlier than the date of the pinned -Ubuntu image, since packages from an older snapshot can need older versions of -packages the Ubuntu image already holds, and it moves forward whenever that -image's digest does. `apt-get update` keeps the snapshot's `InRelease` files, -which apt checks against the archive's signature, in `/var/lib/apt/lists/`; each -lists the SHA-256 hash of the package lists it covers, and each package list the -hash of every package in it. The Dockerfile also names the SHA-256 hash of each -of the snapshot's `InRelease` files, and the build checks them after -`apt-get update` and before `apt-get install`, so every package apt installs is -checked, through those files, against hashes the Dockerfile names. -`apt-get update` also fetches the live archive's `InRelease` files into the same -directory; their hashes change whenever the archive does, and the install does -not use them, so the check leaves them out. The hashes are those of the archive -for amd64, and so the image is built for amd64: other architectures use Ubuntu's -ports archive, whose `InRelease` files differ. The snapshot service is reached -over HTTPS, and the Ubuntu image has no CA certificates of its own, so this one -install uses those of the Go image that `smallwebwaf` is built in, which is -pinned by digest too: apt's `Acquire::https::CaInfo` option names that image's -CA certificate file, `/etc/ssl/certs/ca-certificates.crt`, mounted for that one -step. +### Packages from Ubuntu -Packages from nixpkgs: nixpkgs is fixed at one commit of its newest release -branch, `nixos-26.05` today. For each commit of the branch that has passed its -tests, the Nix project publishes a release on `releases.nixos.org`, such as -`nixos-26.05.11045.774debe7a0d1`, and the image takes nixpkgs from that -release's file `nixexprs.tar.xz`, not from a GitHub archive of the commit, whose -bytes can change. The image's Dockerfile names the release and the SHA-256 hash -of that file, which the release's page lists, and the build checks the hash -before unpacking it. nixpkgs is set up for root under the name `nixpkgs`, so the -app's Dockerfile installs a package with `nix-env -iA nixpkgs.`, and -whatever it installs is on the `PATH` of every service: the image adds root's -Nix profile, `/nix/var/nix/profiles/default/bin`, at the end of the `PATH`, -after Ubuntu's own directories, so that no package hides the image's own -commands. busybox, for one, brings its own `sv`, which looks for services -elsewhere. Because nixpkgs stays at that commit, an app built on the same -`smallwebwaf` image gets the same packages each time it is built. Unpacked, -nixpkgs takes about 500 MiB of disk, more on some filesystems such as ZFS, and -each package an app installs from it adds its own size, with everything it -depends on. A newer commit of the branch, with its security fixes, comes with a -newer `smallwebwaf` image, as do Ubuntu's own fixes; an app takes them by -changing the digest in its `FROM` line. When nixpkgs makes its next release, -every six months, the image moves to that release's branch. +The image installs `ca-certificates`, `nix-bin` and `runit` from Ubuntu's +snapshot service, which serves the archive as it was at a given moment. The +archive itself changes with every update, so its packages are not pinned. The +image's Dockerfile names that moment, in apt's `--snapshot 20261001T000000Z` +form, on both `apt-get update` and `apt-get install`. That moment is never +earlier than the date of the pinned Ubuntu image, since packages from an older +snapshot can need older versions of packages the Ubuntu image already holds. It +moves forward whenever that image's digest does. -The two processes: +`apt-get update` keeps the snapshot's `InRelease` files, which apt checks +against the archive's signature, in `/var/lib/apt/lists/`. Each lists the +SHA-256 hash of the package lists it covers, and each package list the hash of +every package in it. The Dockerfile also names the SHA-256 hash of each of the +snapshot's `InRelease` files. The build checks them after `apt-get update` and +before `apt-get install`, so every package apt installs is checked, through +those files, against hashes the Dockerfile names. `apt-get update` also fetches +the live archive's `InRelease` files into the same directory. Their hashes +change whenever the archive does, and the install does not use them, so the +check leaves them out. The hashes are those of the archive for amd64, and so the +image is built for amd64: other architectures use Ubuntu's ports archive, whose +`InRelease` files differ. + +The snapshot service is reached over HTTPS, and the Ubuntu image has no CA +certificates of its own. So this one install uses those of the Go image that +`smallwebwaf` is built in, which is pinned by digest too: apt's +`Acquire::https::CaInfo` option names that image's CA certificate file, +`/etc/ssl/certs/ca-certificates.crt`, mounted for that one step. + +### Packages from nixpkgs + +nixpkgs is fixed at one commit of its newest release branch, `nixos-26.05` +today. For each commit of the branch that has passed its tests, the Nix project +publishes a release on `releases.nixos.org`, such as +`nixos-26.05.11045.774debe7a0d1`. The image takes nixpkgs from that release's +file `nixexprs.tar.xz`, not from a GitHub archive of the commit, whose bytes can +change. The image's Dockerfile names the release and the SHA-256 hash of that +file, which the release's page lists. The build checks the hash before unpacking +it. + +nixpkgs is set up for root under the name `nixpkgs`, so the app's Dockerfile +installs a package with `nix-env -iA nixpkgs.`. Whatever it installs is on +the `PATH` of every service. The image adds root's Nix profile, +`/nix/var/nix/profiles/default/bin`, at the end of the `PATH`, after Ubuntu's +own directories, so that no package hides the image's own commands. busybox, for +one, brings its own `sv`, which looks for services elsewhere. Because nixpkgs +stays at that commit, an app built on the same `smallwebwaf` image gets the same +packages each time it is built. Unpacked, nixpkgs takes about 500 MiB of disk, +more on some filesystems such as ZFS. Each package an app installs from it adds +its own size, with everything it depends on. A newer commit of the branch, with +its security fixes, comes with a newer `smallwebwaf` image, as do Ubuntu's own +fixes. An app takes them by changing the digest in its `FROM` line. When nixpkgs +makes its next release, every six months, the image moves to that release's +branch. + +### The two processes - `runsvinit` is the container's first process and runs as root, as do runit's `runsvdir` and the `runsv` that watches each service. Each `run` script starts @@ -1317,264 +1334,285 @@ The two processes: - Both start with the container's environment variables, which is why every setting of `smallwebwaf` carries the `SWWAF_` prefix. Both write to the container's output, which `docker logs` shows: the JSON lines of `smallwebwaf` - (see "Request log") and whatever the app writes. + and whatever the app writes. - When `smallwebwaf` or the app exits, for whatever reason, runit runs its `run` script again, which waits one second before starting it. The other process keeps running, and the container does not stop. While `smallwebwaf` is down, - nothing answers on port 8080 and traefik answers `502`; while the app is down, - `smallwebwaf` answers `502` (see "Failure behaviour"). + nothing answers on port 8080 and traefik answers `502`. While the app is down, + `smallwebwaf` answers `502`. - A `smallwebwaf` that stops at start, for example on invalid configuration, is started again every second and stops again, logging its message each time, until the cause is fixed. The health check fails meanwhile. -- `docker stop` sends SIGTERM to `runsvinit`, which has runit stop both services - at once, each with SIGTERM, and exits once they have stopped. `smallwebwaf` - stops taking requests, gives those in progress five seconds to finish, and - writes its state files as it stops (see "Persistent state"). The requests in - progress can finish only while the app still answers. An app that exits at - once on SIGTERM ends them: the client gets `502`, or a response cut short if - the app had already begun to answer. An app that should let them finish keeps - taking and answering requests, not only the ones it already has, for the whole - five seconds after SIGTERM, since `smallwebwaf` can still pass it requests it - took before then. +- `docker stop` sends `SIGTERM` to `runsvinit`, which has runit stop both + services at once, each with `SIGTERM`, and exits once they have stopped. + `smallwebwaf` stops taking requests, gives those in progress five seconds to + finish, and writes its state files as it stops. The requests in progress can + finish only while the app still answers. An app that exits at once on + `SIGTERM` ends them: the client gets `502`, or a response cut short if the app + had already begun to answer. An app that should let them finish keeps taking + and answering requests for the whole five seconds after `SIGTERM`, not only + the ones it already has. `smallwebwaf` can still pass it requests it took + before then. - The container's root filesystem stays writable: runit writes each service's status into its directory under `/etc/service`. -The health check: the image's `HEALTHCHECK` runs `smallwebwaf healthcheck`, -which passes while `smallwebwaf` answers `GET /_smallwebwaf/healthz` on -`127.0.0.1`, at the port in `SWWAF_LISTEN_ADDR`, and the app accepts connections -at the address in `SWWAF_UPSTREAM_URL`, and fails when either does not. The -container therefore shows as healthy only while both processes are up. traefik -sends a container no requests until it shows as healthy, so the check runs every -second from the container's start until it first passes, for up to a minute, and -every 30 seconds after that. An app with a health check of its own can replace -the image's `HEALTHCHECK` with one that checks both. +### The health check -Ports: `smallwebwaf` listens on port 8080 on every address and on no other port; -its health check, metrics and ban management are all on that listener, under -`/_smallwebwaf/` (see "Admin endpoints"). `SWWAF_LISTEN_ADDR` may set another -port: the image's health check takes its port from that setting, and traefik's -port label (`traefik.http.services..loadbalancer.server.port`) must name -the same port, and the app must leave that port free. The address part of +The image's `HEALTHCHECK` runs `smallwebwaf healthcheck`. It passes while +`smallwebwaf` answers `GET /_smallwebwaf/healthz` on `127.0.0.1`, at the port in +`SWWAF_LISTEN_ADDR`, and the app accepts connections at the address in +`SWWAF_UPSTREAM_URL`. It fails when either does not. The container therefore +shows as healthy only while both processes are up. traefik sends a container no +requests until it shows as healthy. So the check runs every second from the +container's start until it first passes, for up to a minute, and every 30 +seconds after that. An app with a health check of its own can replace the +image's `HEALTHCHECK` with one that checks both. + +### Ports + +`smallwebwaf` listens on port 8080 on every address and on no other port. Its +health check, metrics and ban management are all on that listener, under +`/_smallwebwaf/`. `SWWAF_LISTEN_ADDR` may set another port. The image's health +check takes its port from that setting, traefik's port label +(`traefik.http.services..loadbalancer.server.port`) must name the same +port, and the app must leave that port free. The address part of `SWWAF_LISTEN_ADDR` stays empty (for example `:9000`, never `127.0.0.1:9000`), so `smallwebwaf` keeps listening on every address: traefik reaches it on the container's address, and the health check on `127.0.0.1`. The app listens on `127.0.0.1:8081` only, so that nothing outside the container reaches it except -through `smallwebwaf`: an app that listens on every address can be reached +through `smallwebwaf`. An app that listens on every address can be reached around `smallwebwaf` by anything that reaches the container. -State: `smallwebwaf` keeps its state files in `/var/lib/smallwebwaf` +### State + +`smallwebwaf` keeps its state files in `/var/lib/smallwebwaf` (`SWWAF_STATE_DIR`), a directory of its own beside the app's data, which the app keeps in directories of its own, such as `/var/lib/app`. Without a volume there, the files live in the container: they survive a restart of the container and are lost when a deploy replaces it. A volume mounted at `/var/lib/smallwebwaf`, -named or a host directory, keeps them across deploys; the `run` script of -`smallwebwaf` makes it and every file in it belong to the `smallwebwaf` user, so +named or a host directory, keeps them across deploys. The `run` script of +`smallwebwaf` makes it and every file in it belong to the `smallwebwaf` user. So a host directory mounted there needs no change of owner, and files an earlier owner left in it can be read and replaced. It is a volume of its own, separate -from the app's, holds a few tens of MiB at most with the defaults (see -"Persistent state"), and needs no backup beyond whatever the host already does. -The image declares no volume, since every app image built on it would inherit -it. Milestone 2 (https://git.eeqj.de/sneak/smallwebwaf/issues/14) writes no -state files and needs no volume. +from the app's. It holds a few tens of MiB at most with the defaults, and needs +no backup beyond whatever the host already does. The image declares no volume, +since every app image built on it would inherit it. -Tokens: a token given as a file (`SWWAF_ADMIN_TOKEN_FILE`, -`SWWAF_METRICS_TOKEN_FILE`) is out of the app's reach only while the -`smallwebwaf` user alone can read the file. The operator makes the file on the -host, owned by uid 65532, the `smallwebwaf` user, with mode `0400`, and mounts -the directory that holds it into the container read-only; the container sees the -same owner and mode. +### Tokens -Forwarded headers: the app's TCP peer is `smallwebwaf` on `127.0.0.1`, and the -`X-Forwarded-For` the app receives ends with traefik's address, which -`smallwebwaf` adds after the visitor's. The app must therefore trust `127.0.0.1` -and `::1` for forwarded headers, besides the private address ranges traefik is -on. The usual default for a trusted-proxy setting, the private ranges alone, -does not include loopback: an app left at it ignores the header and sees every -visitor as `127.0.0.1`. A list given replaces that default, so the app's own -trusted-proxy setting names them all, -`10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1/32,::1/128`, as in the +A token given as a file (`SWWAF_ADMIN_TOKEN_FILE`, `SWWAF_METRICS_TOKEN_FILE`) +is out of the app's reach only while the `smallwebwaf` user alone can read the +file. The operator makes the file on the host, owned by uid 65532, the +`smallwebwaf` user, with mode `0400`, and mounts the directory that holds it +into the container read-only. The container sees the same owner and mode. + +### Forwarded headers + +The app's TCP peer is `smallwebwaf` on `127.0.0.1`, and the `X-Forwarded-For` +the app receives ends with traefik's address, which `smallwebwaf` adds after the +visitor's. The app must therefore trust `127.0.0.1` and `::1` for forwarded +headers, besides the private address ranges traefik is on. The usual default for +a trusted-proxy setting, the private ranges alone, does not include loopback: an +app left at it ignores the header and sees every visitor as `127.0.0.1`. A list +given replaces that default, so the app's own trusted-proxy setting names them +all, `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1/32,::1/128`, as in the example. -upaas: an app deployed by upaas takes this shape with no change to upaas. upaas -builds the app's Dockerfile, which now starts `FROM` the `smallwebwaf` image, -and runs the one container as it runs any app, with the app's traefik labels, -environment variables and volumes. The labels route to port 8080 -(`traefik.http.services..loadbalancer.server.port=8080`), any `SWWAF_` -settings go with the app's environment variables, and the volume for +### upaas + +An app deployed by upaas takes this shape with no change to upaas. upaas builds +the app's Dockerfile, which now starts `FROM` the `smallwebwaf` image, and runs +the one container as it runs any app, with the app's traefik labels, environment +variables and volumes. The labels route to port 8080 +(`traefik.http.services..loadbalancer.server.port=8080`). Any `SWWAF_` +settings go with the app's environment variables. The volume for `/var/lib/smallwebwaf` goes beside the app's own, as does the directory that holds any token file. -- The endpoints of `smallwebwaf` are reached through traefik like any other - request, for example `https://app.example.invalid/_smallwebwaf/metrics` for a - scraper that sends `SWWAF_METRICS_TOKEN` (see "Admin endpoints"). -- Rollout per service: build the app's image on the `smallwebwaf` image, with no - setting, and deploy it as before. It protects the app from the first request, - and nothing needs tuning first. Afterwards the request log and the ban notes - say why each client was refused. A real visitor refused by mistake is let - through with an exclusion (`SWWAF_WAF_DISABLED_RULES`, - `SWWAF_WAF_EXEMPT_PATHS`, `SWWAF_RATE_LIMIT_EXEMPT_PATHS`) or an exemption - (`SWWAF_RATE_LIMIT_EXEMPT_NETS`, `SWWAF_ALLOW_NETS`), and its ban is lifted in - `bans.json`. Additions to consider at any time: an alert destination, country - lists or biased thresholds, reputation sources, the metrics and admin tokens, - a remote log endpoint, and an app-specific rule file. -- Notes specific to gitea, built on the image like any other app: - - SSH access to gitea does not pass through `smallwebwaf` and is not - protected by it. - - A clone is a response, and fits the defaults of 30 minutes and 5 GiB for - all but the largest repositories and slowest links. A push is a request, - and so is every other upload: at the defaults, one larger than 100 MiB or - taking more than 60 seconds is cut off, whether it is a git push, an LFS - object, a container image layer, a package file or a release attachment. - The client is answered `413` for a body that is too large, before anything - reaches gitea when the request announces its size, or `408` for one the - client sends too slowly (`504` if gitea is too slow to take it); the - upload fails, and no one is banned for it. A gitea that takes large - uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` - and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does - not read an upload's body, which streams through without being held in - memory. - - At the defaults (see "Configuration surface", attack detection), the Core - Rule Set lets gitea's ordinary use through, apart from the refusals in the - next note: browsing and views of files in a repository, with their - history, blame and the file tree; diffs, including their hidden lines and - large files, and pull request review; git's clone, fetch and push over - HTTP; signing in, including the return to the page a visitor came from and - sign-in with Git Credential Manager, git-credential-oauth or tea; the - API's calls for a file and its commits; pushing and pulling container - images and packages; Actions runners, and artifacts uploaded with - `actions/upload-artifact@v4`; and posting issues, pull requests, comments, - wiki pages and files saved in the web editor, code included, since no body - is read, and the page gitea shows after a change with its message naming - what was done, such as a file deleted or a branch, milestone or project - created. - - The Core Rule Set can still refuse the requests below. Each refusal - answers only that request, with 403, and bans no one by itself: it counts - toward the error burst, which a person does not reach this way. The - request log names the rule in `waf_rule_ids`, which - `SWWAF_WAF_DISABLED_RULES` can switch off. - - A query string that reads to it as an attack, most often a search: one - for a name on its lists of system files and commands, such as - `package.json`, `.gitignore` or `docker-compose.yml`, or for text that - starts with a command name, or whose first word has one right after a - `/`, such as `python3`, `ssh key` or `feat/docker-support`; or one - holding a shell command with its options or a system path (`ls -la`, - `sed -i`, `/bin/sh`), a command in backticks, script code (`fetch(`, - `${VAR}`, `process.env`), HTML (`.bad` and written again from memory; the rule file is left as it is. - Logged, one `file_error` alert. +- Alert destination down: alerts are retried with backoff from a bounded queue, + the oldest dropped first, and the drops counted in the metrics. +- App down or restarting: `502` from `smallwebwaf`, not counted as a client + offence. +- Attack detection engine error on a request: the request is forwarded, and the + error is logged and counted. +- Every other failure is covered where its part is specified: + - GeoJS and the lookup database under "Lookup of AS number and country"; + - the remote log server under "Request log"; + - state file writes and edits under "Persistent state"; + - rule file edits under "Rule files"; + - a process exiting under "The two processes". - In short: the only things that stop the process happen at start: invalid configuration, a configured lookup database that is missing or unreadable, a rule file or state file that does not parse, and an unwritable @@ -1587,120 +1625,60 @@ holds any token file. ## Risks the design has to handle -- Forged `X-Forwarded-For`: handled by believing the header only from a peer - inside `SWWAF_TRUSTED_PROXIES` and by walking it from the right. The default - trusts every private address, which in the intended deployment means traefik; - where other containers can reach `smallwebwaf` directly, or visitors on a - private network reach traefik, they can name another address in the header, - and setting `SWWAF_TRUSTED_PROXIES` to traefik's own network closes that gap. +- Forged `X-Forwarded-For`: the header is believed only from a peer inside + `SWWAF_TRUSTED_PROXIES`, and walked from the right. The default trusts every + private address, which in the intended deployment means traefik. Where other + containers can reach `smallwebwaf` directly, or visitors on a private network + reach traefik, they can name another address in the header. Setting + `SWWAF_TRUSTED_PROXIES` to traefik's own network closes that gap. - Many clients behind one address (mobile carriers, offices, Tor): they share - its limits and its bans. The default limits sit several times above what one - busy person produces, `SWWAF_RATE_LIMIT_EXEMPT_NETS` and `SWWAF_ALLOW_NETS` - take known shared addresses out, and the ban notes show an admin what happened - before a ban is lifted. -- A `ban` rule that matches a real visitor bans it for seven days, and the - visitor's next request during those days makes the ban permanent. So the - default rule file keeps to requests no real visitor sends, with its paths - anchored at the site root, where no app serves secrets, version control - directories, backups or web shells. A visitor banned by mistake is let back in - by lifting the ban in `bans.json`. + its limits and its bans. The defaults sit well above one person's use, + `SWWAF_RATE_LIMIT_EXEMPT_NETS` and `SWWAF_ALLOW_NETS` take known shared + addresses out, and the ban notes show an admin what happened before a ban is + lifted. +- A `ban` rule that matches a real visitor bans it for seven days, then + permanently at its next request. So the default rule file keeps to requests no + real visitor sends, anchored at the site root. A visitor banned by mistake is + let back in by lifting the ban in `bans.json`. - IPv6 address rotation inside a /64: handled by grouping. - Widely distributed scrapers using thousands of addresses at low rates each: per-client limits do not see them. The AS number and netblock anomaly alerts reveal them once their thresholds are set, and a low `SWWAF_ASN_LIMIT_PERCENT` - for that AS number is the response: it lowers the limits of each of its - clients. There is no shared budget for a whole AS number or country, which one - abuser could use up and so lock out everyone else there. -- Core Rule Set false positives against real apps (a search for code, and any - body once `SWWAF_WAF_BODY_LIMIT` is set): a match refuses only that request - and bans no one by itself; the request log names the rule, and exclusions by - rule id and path fix it. `script/gitea-requests` runs gitea behind the image - at the defaults and sends through it a short sample of gitea's ordinary use - and some of the requests that three of the notes specific to gitea say are - refused: for each file the note on a file name in a query parameter names, the - API's list of its commits; for `Runtime.java` alone, the commit graph of that - file, the hidden lines of its diff and the API's call for the contents of - files that names it; the sign-in page gitea sends a visitor to on the way to - the edit page of `document.write.js`; the pages of `java.lang.String.md`, - `META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts` - and `.../notes.txt`; and one container image push that reuses a layer, which - it does not try again. It reports each request the Core Rule Set refuses, so - that those requests can be checked again when gitea or the Core Rule Set - changes version. -- Attacks carried in request bodies: not refused by default, since on a code - forge bodies are full of code the Core Rule Set takes for attacks. Most of - what shows in URLs and headers is still refused, the client that sends an - attack is still subject to the rule files, the limits and the bans, and - `SWWAF_WAF_BODY_LIMIT` switches body inspection on for apps whose forms carry - no code. -- An app that uses `path`, `ref` or another of the query parameters left out of - 930120, 932160 and 932260 (see "Configuration surface", attack detection) as a - file on the server, or passes it to a shell: that change holds in front of - every app and no setting restores the three rules, so what only they refuse - reaches the app in those parameters, such as `/etc/passwd`, `|cat /etc/passwd` - or `nc -e /bin/sh …`. Path traversal (`../`) is still refused there. Such an - app has to check those values itself, or refuse what it must never receive in - them with a rule file. So does an app that reads a cookie named `gitea_flash` - or `redirect_to`, which the Core Rule Set does not read, or passes `Referer` - to a shell. + for that AS number lowers the limits of each of its clients. +- Core Rule Set false positives against real apps: a match refuses only that + request and bans no one by itself. The request log names the rule, and + exclusions by rule id and path fix it. `script/gitea-requests` checks the + known cases. +- Attacks carried in request bodies: not refused by default, for the reason + given under "Attack detection". The rule files, the limits and the bans still + apply to the client, and `SWWAF_WAF_BODY_LIMIT` switches body inspection on. +- An app that uses one of the query parameters left out of 930120, 932160 and + 932260 as a file on the server, or passes it to a shell; or that reads a + cookie named `gitea_flash` or `redirect_to`, or passes `Referer` to a shell: + no setting restores those rules. Such an app has to check those values itself, + or refuse what it must never receive in them with a rule file. - The admin endpoints can be reached from the internet: all but the health check - need a token and are off while it is unset, and a missing or wrong token - counts toward the error burst, so a client guessing tokens is soon banned. - Tokens are meant to be long random values, and one shorter than 32 characters - stops the start. The app starts with the same environment variables as - `smallwebwaf`, so it can read a token given as one; a token given as a file - that only the `smallwebwaf` user can read (`SWWAF_ADMIN_TOKEN_FILE`, - `SWWAF_METRICS_TOKEN_FILE`) is out of the app's reach (see "Deployment"). + need a token and are off while it is unset, and token guessing is banned + through the error burst. A token given as a file that only the `smallwebwaf` + user can read is out of the app's reach. - GeoJS, the default lookup source: every new visitor's address goes to a third - party, and a swarm of fresh addresses, when lookups peak, is when GeoJS may - slow down or block `smallwebwaf`. Keeping answers for 7 days and asking about - many addresses in one request keep the number of requests low; the file source - has neither risk. -- Slow-request attacks: `SWWAF_CLIENT_REQUEST_TIMEOUT` bounds how long a request - may take to arrive, `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` how large its - headers may be, and `SWWAF_CLIENT_IDLE_TIMEOUT` closes a kept-open connection - that sends nothing more. -- Alert floods: cooldown and hourly cap. + party, and a swarm of fresh addresses is when GeoJS may slow down or block + `smallwebwaf`. Keeping answers for 7 days and asking about many addresses in + one request keep the number of requests low. The file source has neither risk. +- Slow-request attacks: bounded by `SWWAF_CLIENT_REQUEST_TIMEOUT`, + `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` and `SWWAF_CLIENT_IDLE_TIMEOUT`. +- Alert floods: the cooldown and the hourly cap. -## Build order +## Planned, not built -- Milestone 1 (https://git.eeqj.de/sneak/smallwebwaf/issues/13): a pass-through - proxy with read and write timeouts, size limits and a request log. It - identifies the client and passes requests and answers through unchanged within - the four timeouts and the two size limits; the header size and the idle time - are fixed at their defaults, not settings. Each request gets a line in the - request log, with part of its fields. It writes nothing to disk. -- Milestone 2 (https://git.eeqj.de/sneak/smallwebwaf/issues/14): per-IP rate - limits and country allow and deny lists, ready for production. - - The request rate limits per client, over a minute, an hour and a day. A - client over one is refused with `429` until it is back under every limit; - bans come later. - - The country lists, with each client's country looked up through GeoJS, - only while a list is set. A client on a private, loopback or link-local - address has no country, and neither list checks it. - - Like milestone 1, it writes nothing to disk: the GeoJS answers and the - rate counters are kept in memory only, and a restart loses them. The - header size and the idle time stay fixed at their defaults. - - The container image described under "Deployment", with runit and the - container's health check. The health check calls `/_smallwebwaf/healthz`, - so milestone 2 answers that path, although the other admin endpoints come - later. The image's `/var/lib/smallwebwaf`, which the `run` script gives to - the `smallwebwaf` user, and `/etc/smallwebwaf/rules.d` come with the state - files and the rule files. -- Milestone 3 and later: the rest of the design, in this order: - - static lists, the bans that broken request limits lead to, the ban ledger - and the JSON state files with edits taken in while running, exemptions, - `observe` mode, the rest of the request log's fields, the metrics - endpoint, and the header size and the idle time as settings - (`SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, `SWWAF_CLIENT_IDLE_TIMEOUT`). - With the static lists comes `SWWAF_ALLOW_NETS`, and from then on - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses a client on a private, - loopback or link-local address unless `SWWAF_ALLOW_NETS` lists it; - - rule files, the other admin endpoints, alerting to all three destinations, - remote log sending; - - AS number and country lookup for every client, from the file or GeoJS, - biased thresholds, byte limits, anomaly thresholds; - - blocklists, DNSBL, AbuseIPDB, optional CrowdSec decision feed; - - attack detection with Coraza and the Core Rule Set, trap paths, error - bursts. -- Each stage is usable on its own; milestone 2 is the first to go to production. +Everything above is built. These two points are specified as later work, and +nothing in the code does them yet: + +- Planned: sharing bans between the instances of several apps, by running one + CrowdSec engine per host that every instance reports its bans to. It is + reconsidered once real ban volumes are known. Until then each instance keeps + its own bans, and can read a CrowdSec engine's decisions as described under + "Reputation". +- Planned: sending the log over RELP, the reliable syslog transport. It is + revisited against the available Go packages. Until then the remote log goes + over UDP, TCP or TLS as described under "Request log". -- 2.54.0 From e44ba7ee5e197e8dbc8f15d521974dc99ee77304 Mon Sep 17 00:00:00 2001 From: sneak Date: Thu, 22 Oct 2026 19:43:03 +0000 Subject: [PATCH 2/8] SPEC rework after review: each requirement once, gitea notes to docs/ Every requirement is stated once, where it belongs, and linked from elsewhere. The notes on gitea move to docs/gitea.md, which script/gitea-requests points at. The list of settings that take off is exact, the non-goal that contradicted the planned item is gone, and the sentences the review named are split or reordered. The package comments in internal/waf and internal/anomaly point at the renamed sections. Model: fable-5-1 --- SPEC.md | 1334 ++++++++++++++--------------------- docs/gitea.md | 198 ++++++ internal/anomaly/anomaly.go | 4 +- internal/waf/waf.go | 4 +- script/gitea-requests | 11 +- 5 files changed, 730 insertions(+), 821 deletions(-) create mode 100644 docs/gitea.md diff --git a/SPEC.md b/SPEC.md index e437903..b1b7daa 100644 --- a/SPEC.md +++ b/SPEC.md @@ -2,16 +2,18 @@ `smallwebwaf` is a web application firewall and reverse proxy for one app. It runs inside the app's container, between traefik and the app. This document says -what it must do and why. Everything in it but the last section, "Planned, not -built", is built. `README.md` says how to run it. `EVALUATION.md` says why no -existing tool was adopted. +what it must do and why, each requirement once. Everything in it is built, +except the last section, "Planned, not built". `README.md` says how to run it. +`EVALUATION.md` says why no existing tool was adopted. `docs/gitea.md` records +how it behaves in front of gitea. ## Purpose - It protects one self-hosted app on the open internet from scrapers, scanners - and other abusers. -- It protects the app from its first request with no setting at all. Every - setting has a default chosen for a service facing the internet in 2026. + and other abusers, from the app's first request, with no setting at all. Every + setting has a default chosen for a service facing the internet in 2026. A + setting that needs an alert destination, an account key or a token stays off + until the operator supplies it. - It limits each client's requests and bytes per minute, hour and day. It bounds the size and duration of every request and answer. It detects attacks. - It bans abusers: briefly at first, for seven days on a clear sign of attack, @@ -19,9 +21,10 @@ existing tool was adopted. - It looks up each client's AS number and country. It can refuse whole countries and netblocks, and lower its limits for listed AS numbers and countries. It consults reputation sources. It sends alerts. -- Everything is set by environment variables named with the `SWWAF_` prefix. The - only other inputs are files: the rule files, the state files, a lookup - database, and a setting given as a file. +- Everything is set by environment variables whose names start with `SWWAF_`, + since `smallwebwaf` shares its container, and so its environment, with the + app. The only other inputs are files: the rule files, the state files, a + lookup database, and a setting given as a file. ## Non-goals @@ -32,41 +35,20 @@ existing tool was adopted. - Defence against floods that saturate the host's network link. That needs help upstream of the host. - A web console or a configuration file. -- Sharing bans between the instances of several apps. Each instance keeps its - own. Reading a CrowdSec engine's decisions is supported. ## Architecture - One statically linked Go binary. It ships in an image built on Ubuntu 26.04 LTS with nixpkgs installed. That image is the base of the app's own image, so - that `smallwebwaf` and the app run in one container (see "Deployment"). runit + `smallwebwaf` and the app run in one container (see "Deployment"). runit starts `smallwebwaf` as a non-root user of its own. `smallwebwaf` writes only to its state directory. - One listener, on port 8080, which traefik routes to. It forwards requests to - the app on `127.0.0.1:8081`. It answers its own endpoints, for health, metrics - and ban management, under `/_smallwebwaf/`. Those never reach the app. -- The parts of the process: - - Client identification: the real client address from `X-Forwarded-For`, - trusting only the netblocks in `SWWAF_TRUSTED_PROXIES`. - - Lookup: the AS number and country, from the GeoJS web service by default - or from a database file the operator supplies, held in memory. - - Reputation: blocklists fetched on a schedule; DNS blocklists and AbuseIPDB - asked in the background, with their answers cached. - - Counters: per-client requests and bytes per minute, hour and day, held in - memory and written out with the rest of the state. - - Attack detection: regex rules read from plain text files and read again - when they change; Coraza with the OWASP Core Rule Set; trap paths; and the - error burst, a burst of refusals from one client. - - Ban ledger: active and past bans with their notes, and each client's - history, held in memory and kept in JSON files the operator may edit. - - Request log: one JSON object per request on stdout, optionally also sent - to a syslog server. - - Metrics: Prometheus counters and gauges. - - Alerting: a queue that holds back repeats and feeds a webhook, Slack and - ntfy. - - Proxy: the standard library's `httputil.ReverseProxy`, streaming in both - directions within the size and time limits, with WebSocket support. + the app on `127.0.0.1:8081`. It answers its own endpoints under + `/_smallwebwaf/` (see "Admin endpoints"). Those never reach the app. - No database of any kind. Nothing is read from disk while serving a request. +- The proxy is the standard library's `httputil.ReverseProxy`. The other parts + of the process are each specified below. - Libraries, checked against the owner's Go dependency defaults: - the standard library for the proxy, HTTP clients, DNS, logging (`log/slog`) and the state files (`encoding/json`, `os.Rename`); @@ -84,7 +66,8 @@ existing tool was adopted. ## What happens to a request The steps run in this order. The first step that produces a final answer ends -the handling. The size and time limits apply to the whole exchange. +the handling. The size and time limits apply to the whole exchange. Each step +names the section that holds its rules. 1. `GET /_smallwebwaf/healthz` is answered at once, before any check. 2. The client is identified. @@ -101,107 +84,81 @@ the handling. The size and time limits apply to the whole exchange. 3. The static lists. A client in `SWWAF_ALLOW_NETS` skips every check below. It is still counted for the anomaly thresholds. A client in `SWWAF_DENY_NETS` is refused. -4. The ban ledger. A client under a ban is refused with `SWWAF_BAN_RESPONSE`. If - a clear sign of attack caused the ban, the ban becomes permanent. -5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off`. - - With GeoJS, a request waits for its client's first answer, up to - `SWWAF_LOOKUP_TIMEOUT`, only when a setting needs it before the request - goes on: a country list, a biased threshold or `SWWAF_ADD_LOOKUP_HEADERS`. - Otherwise the request goes on at once. The answer is added to the client's - history and ban notes when it comes. A request that ends before then is - logged without it. - - A client the lookup cannot place has an unknown country and AS number. - Every private, loopback and link-local address is such a client. The - exclusive country list refuses it, and the biased thresholds give it - `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it differently. -6. The country lists. A client from a country in `SWWAF_DENIED_COUNTRIES` is - refused. When `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any - other country is refused. The answer is `SWWAF_BAN_RESPONSE`. The request - body has not been read yet. The refusal skips everything below, the rule - files and the Core Rule Set included. It is logged with the action - `country_denied` and counted in the metrics. It is not an offence and makes - no ban. -7. The reputation sources. A client inside a fetched blocklist gets - `SWWAF_BLOCKLIST_ACTION`. A client the CrowdSec decision list lists is banned - until the decision ends. A cached DNS blocklist verdict or AbuseIPDB score - gets `SWWAF_REPUTATION_ACTION`. With no cached verdict, a query is queued in - the background and the request goes on. A first request is never delayed by a - reputation query. -8. The client's limit percentage: the lowest of the percentages that apply to - it, from its AS number, its country or a reputation source, or 100 if none - does. -9. Rate limiting, unless the client is in `SWWAF_RATE_LIMIT_EXEMPT_NETS` or the - path is under `SWWAF_RATE_LIMIT_EXEMPT_PATHS`. The minute, hour and day - counters are checked against each limit times the percentage. A request over - any of them breaks that limit: it is refused with `SWWAF_BAN_RESPONSE`, and - the client is banned (see "Bans"). -10. The trap paths. A request for a path in `SWWAF_TRAP_PATHS` is a clear sign - of attack: it is refused and the client is banned. -11. The rule files, in file name order and then line order. A rule that matches - takes its action: only log, refuse with `403`, or refuse and ban. Matching - stops at the first rule that refuses. -12. The Core Rule Set inspects the request's method, its URL with the query - string, and its headers. By default it reads no body. When - `SWWAF_WAF_BODY_LIMIT` is a size, it reads a body that is form data, - multipart, JSON or XML, up to that size. Any other body reaches the app - uninspected. So does a JSON or XML body larger than the limit, which cannot - be read in part. Responses are not inspected. In `block` mode, the default, - a match at or over the anomaly threshold is refused with `403`. In `detect` - mode it is only logged and alerted. +4. The ban ledger. A client under a ban is refused (see "Bans"). +5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number + and country"). The request goes on at once. It waits for the first GeoJS + answer, up to `SWWAF_LOOKUP_TIMEOUT`, only when one of these needs it: + - a country list; + - a biased threshold; + - `SWWAF_ADD_LOOKUP_HEADERS`. +6. The country lists (see "Country lists"). A refused request is not an offence. +7. The reputation sources (see "Reputation"). A client on a fetched blocklist + gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is + banned. A client listed by a DNS blocklist, or scored at least + `SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`. + With no cached verdict, a query is queued in the background and the request + goes on. A first request is never delayed by a reputation query. +8. The client's limit percentage (see "Biased thresholds"). +9. Rate limiting (see "Request rate limits, per client"). A request over a limit + times the percentage is refused and bans the client. +10. The trap paths (see `SWWAF_TRAP_PATHS`). A request for one is refused and + bans the client. +11. The rule files (see "Rule files"). Matching stops at the first rule that + refuses. +12. The Core Rule Set (see "Attack detection"). In `block` mode a match is + refused with `403`. In `detect` mode it is only logged and alerted. 13. A request under `/_smallwebwaf/` is answered here and goes no further. 14. The request is forwarded to `SWWAF_UPSTREAM_URL`, streaming, with `X-Forwarded-For` added, and `X-Client-ASN` and `X-Client-Country` when enabled. 15. After the response: - - The response bytes, the request body bytes, or both are added to the byte - counters. A client whose total passes a byte limit times the percentage - has broken that limit and is banned. The response in progress is not cut - off. - - The client's refusals are counted: those `smallwebwaf` refused after a - rule file match, a trap path, a Core Rule Set match, or a missing or wrong - token. Answers the app gave are not counted. More than - `SWWAF_ERROR_BURST_THRESHOLD` of them within a minute break a limit and - ban the client. + - The response bytes, the request bytes or both are added to the byte + counters, as `SWWAF_BYTES_COUNT` says. A client whose total passes a byte + limit times the percentage has broken that limit and is banned. The + response in progress is not cut off. + - The client's refusals are counted for the error burst (see + `SWWAF_ERROR_BURST_THRESHOLD`). - The client's history and the anomaly counters are updated, and the alert thresholds are evaluated. +In `SWWAF_MODE=observe`, every refusal above is logged as what would have +happened, nothing is refused, and no ban is made. The size and time limits and +the token check still apply. + ## Counting method - Each window (minute, hour, day) uses two adjacent fixed buckets per client. The earlier bucket is weighted by how much of it still overlaps the sliding window. This costs a few integers per client per window. It avoids the burst at bucket boundaries that a single fixed bucket allows. -- Counters are read and updated in memory. A request never waits on the disk. - They are written to `clients.json` every `SWWAF_STATE_COUNTER_INTERVAL` and at - shutdown, and loaded again at start. A restart therefore does not hand every - client a fresh allowance. Buckets whose time has passed are discarded on load. -- Memory is bounded by `SWWAF_MAX_TRACKED_CLIENTS` and `SWWAF_MAX_BANS`, and by - fixed limits on the GeoJS answers, the reputation verdicts and the anomaly - counters kept. When the table of clients is full, the least recently seen - client is dropped first, with its history. When `SWWAF_MAX_BANS` bans - `smallwebwaf` made are held, the one that has gone longest without a request - is dropped first, past, active or permanent. Bans an admin made are never - dropped. +- Counters are read and updated in memory. They are written to `clients.json` + and loaded again at start (see "Persistent state"), so a restart does not hand + every client a fresh allowance. Buckets whose time has passed are discarded on + load. +- Memory is bounded. When the table of clients holds + `SWWAF_MAX_TRACKED_CLIENTS`, the least recently seen client is dropped first, + with its history. The bans are bounded by `SWWAF_MAX_BANS` (see "Bans"). The + GeoJS answers, the reputation verdicts and the anomaly counters have fixed + bounds, given where each is described. ## Bans A ban refuses every request from a netblock with `SWWAF_BAN_RESPONSE` until it -ends. A permanent ban does not run out. It ends when an admin lifts it, or, if -`smallwebwaf` made it, when it is dropped past `SWWAF_MAX_BANS`. A scanner whose -ban was dropped that way is refused and banned again by its next probe. A ban an -admin made, whose cause is `admin`, is never dropped and does not count toward -`SWWAF_MAX_BANS`. Such bans can never fill the table. An admin who wants to keep -a ban `smallwebwaf` made sets its cause to `admin`. +ends. The netblock is the client: its IPv4 address, widened to +`SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one +address or one /64, so a permanent ban does not reach neighbours who did +nothing. Requests refused under a ban are not counted against limits. They are +counted in the client's history and in the ban's notes. -The netblock a ban covers is the client: its IPv4 address, or its IPv6 group. -`SWWAF_BAN_SCOPE_V4_PREFIX` can widen an IPv4 ban to the surrounding netblock. -At the defaults a ban covers one address or one /64, so a permanent ban does not -reach neighbours who did nothing. +An offence is a request `smallwebwaf` holds against the client. It is one of: -An offence is a request `smallwebwaf` holds against the client. It is one that -carries a clear sign of attack, breaks a limit, or is refused by a rule file, by -the Core Rule Set or for a missing or wrong token. Offences are counted by kind -in the client's history. Two kinds lead to a ban, each by its own rule: +- a clear sign of attack; +- a broken limit; +- a refusal by a rule file, by the Core Rule Set or for a missing or wrong + token. + +Offences are counted by kind in the client's history. The first two kinds lead +to a ban, each by its own rule: - A clear sign of attack: a request that matches a rule whose action is `ban`, or asks for a path in `SWWAF_TRAP_PATHS`. Examples are a probe for a `.env` @@ -229,13 +186,16 @@ in the client's history. Two kinds lead to a ban, each by its own rule: A Core Rule Set match refuses only the request it matched, without a ban, because the Core Rule Set has false positives. A `block` rule does the same. Those refusals count toward the error burst, so a client that keeps setting them -off is banned under the second rule. Requests refused under a ban are not -counted against limits. They are counted in the client's history and in the -ban's notes. +off is banned under the second rule. -A client the CrowdSec decision list lists is banned until the decision ends, -with the cause `crowdsec`. Such a ban is never made permanent. It does not make -the netblock's next ban for a broken limit longer. +A client on the CrowdSec decision list is banned until the decision ends, with +the cause `crowdsec`. Only a client that sends a request is banned, so a long +list does not fill `bans.json`. Such a ban is never made permanent. It does not +make the netblock's next ban for a broken limit longer. + +Some refusals make no ban at all. `SWWAF_DENY_NETS`, a blocklist or reputation +hit whose action is `deny`, and the country lists refuse each request they cover +and record nothing in `bans.json`. Every ban carries notes with what an admin needs to decide whether to lift it (see "Persistent state"). An admin can add or lift a ban at any time by editing @@ -243,13 +203,14 @@ Every ban carries notes with what an admin needs to decide whether to lift it lifted before it ends is kept, marked as lifted, and does not count toward a longer ban. Deleting its entry from `bans.json` forgets it entirely. -Clients of the AS numbers and countries listed in the biased thresholds get -lower limits. So do clients a reputation source lists whose action is -`limit:`. The same rules then ban them after fewer requests. - -Some refusals make no ban at all. `SWWAF_DENY_NETS`, a blocklist or reputation -hit whose action is `deny`, and the country lists refuse each request they cover -and record nothing in `bans.json`. +A permanent ban does not run out. It ends when an admin lifts it, or when the +table of bans is full. When `smallwebwaf` holds `SWWAF_MAX_BANS` bans of its +own, it drops the one that has gone longest without a request, whether past, +active or permanent. A scanner whose ban was dropped that way is refused and +banned again by its next probe. A ban an admin made, whose cause is `admin`, is +never dropped and does not count toward `SWWAF_MAX_BANS`, so such bans can never +fill the table. An admin who wants to keep a ban `smallwebwaf` made sets its +cause to `admin`. ## Settings @@ -260,24 +221,22 @@ Conventions: Sizes take `K`, `M` and `G`, powers of 1024. Countries are two-letter ISO codes in either case. A code that is not a country, such as `nk`, stops the start with a message naming it. -- No setting is required. Every setting has a default chosen for a service - facing the internet in 2026, or stays off until the operator supplies what it - needs: an alert destination, an account key, a token. -- Every setting's name starts with `SWWAF_`, since `smallwebwaf` shares its - container, and so its environment variables, with the app. -- `off` switches off: - - a timeout, a size limit, a rate limit, a byte limit or an anomaly - threshold; - - the Core Rule Set's anomaly threshold and the error burst threshold; - - the alert cooldown and the hourly alert cap. -- `off` is refused by `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, and by every - setting that is not a limit: the lookup timeout, the reputation cache and - timeout, the ban durations, the state write times, and the blocklist refresh. +- These settings take `off`, which switches them off: + - the five timeouts under "Size and time limits"; + - `SWWAF_REQUEST_MAX_BYTES` and `SWWAF_RESPONSE_MAX_BYTES`; + - the three rate limits and the three byte limits; + - `SWWAF_WAF_ANOMALY_THRESHOLD`, `SWWAF_WAF_BODY_LIMIT` and + `SWWAF_ERROR_BURST_THRESHOLD`; + - `SWWAF_ALERT_COOLDOWN` and `SWWAF_ALERT_MAX_PER_HOUR`; + - every anomaly threshold. + + Every other setting refuses `off`. + - A list set to an empty value is an empty list, and replaces the default. -- Every setting may instead be given as a file holding the value, named by the - setting's name with `_FILE` added, such as `SWWAF_ADMIN_TOKEN_FILE`, for - secrets and long lists. `SWWAF_LOG_REMOTE_TLS_CA_FILE` names a file already - and has no `_FILE` form. +- Any setting can be read from a file instead, for secrets and long lists: + `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. + `SWWAF_LOG_REMOTE_TLS_CA_FILE` already holds a file name, so it has no `_FILE` + form. - Settings, files included, are read once at start. Changing one means restarting the container. The files `smallwebwaf` watches while it runs are its state files, its rule files and the lookup database. @@ -288,19 +247,14 @@ The settings, by group: ### Core -- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app, which listens - there in the same container. +- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. - `SWWAF_LISTEN_ADDR` (default `:8080`): the one listener, for the requests - `smallwebwaf` forwards and for its own endpoints. -- `SWWAF_ADMIN_TOKEN`: the bearer token for the ban endpoints and - `/_smallwebwaf/clients/`. A long random value, since the endpoints can be - reached from the internet. One shorter than 32 characters stops the start. - Unset by default, which switches the endpoints off; bans are then managed by - editing `bans.json`. + `smallwebwaf` forwards and for its own endpoints. The address part stays empty + (see "Ports"). - `SWWAF_INSTANCE_NAME` (default: the container's host name, which docker sets to the first 12 characters of the container's id): included in every log line, - metric and alert. Set it, for example to `fsn1app1/gitea`, to a name that - stays the same when a deploy replaces the container and tells instances apart. + metric and alert. Set it to a name that survives a redeploy and tells + instances apart, such as `fsn1app1/gitea`. - `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to log and alert on every decision while refusing nothing. - `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`, @@ -312,18 +266,31 @@ The settings, by group: - `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the clients held in memory and in `clients.json`. - `SWWAF_MAX_BANS` (default `5000`): the bans `smallwebwaf` made that are held, - past, active and permanent. Bans an admin made are kept besides, and never - dropped. + past, active and permanent (see "Bans"). -### Persistent state +### Tokens + +- `SWWAF_ADMIN_TOKEN`: the bearer token for the ban endpoints and + `/_smallwebwaf/clients/`. Unset by default, which switches them off; bans + are then managed by editing `bans.json`. +- `SWWAF_METRICS_TOKEN`: the bearer token a scraper sends for + `/_smallwebwaf/metrics`. Unset by default, which switches the metrics off, + since they would otherwise be open to anyone on the internet. It is separate + from `SWWAF_ADMIN_TOKEN`, so a scraper that holds it cannot manage bans. +- A token is a long random value, since the endpoints can be reached from the + internet. One shorter than 32 characters stops the start. The app starts with + the same environment variables as `smallwebwaf`, so it can read a token given + as one. The file form keeps a token out of the app's reach (see "Tokens" under + "Deployment"). + +### State files - `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the JSON state files, in a directory of their own beside the app's data, belonging to the `smallwebwaf` - user. A volume mounted there keeps the state when a deploy replaces the - container. -- `SWWAF_STATE_WRITE_DELAY` (default `10s`): after a ban is made, lifted or made - permanent, `bans.json` is written this long later, with every change made in - between, so a burst of changes becomes one write. + user. +- `SWWAF_STATE_WRITE_DELAY` (default `10s`): `bans.json` is written this long + after a ban is made, lifted or made permanent, so a burst of changes becomes + one write. - `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is written. @@ -348,10 +315,6 @@ The settings, by group: ### Metrics -- `SWWAF_METRICS_TOKEN`: the bearer token a scraper sends for - `/_smallwebwaf/metrics`, a long random value. One shorter than 32 characters - stops the start. Unset by default, which switches the metrics off, since they - would otherwise be open to anyone on the internet. - `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and countries get series of their own. The rest are summed as `other`. @@ -387,22 +350,18 @@ largest response allowed, and no single download breaks one. ### Size and time limits These apply per request, in both directions. The client-facing timeouts apply -between the client and `smallwebwaf`. The app-facing ones apply between -`smallwebwaf` and the app. A slow client and a slow app are separate problems. -There is one size limit for request bodies and one for response bodies: -`smallwebwaf` passes bodies through unchanged, so a limit on each side would -bound the same bytes, and the lower one would always decide. Bodies stream -straight through, so a request body reaches the app while the client is still -sending it. +between the client and `smallwebwaf`, the app-facing ones between `smallwebwaf` +and the app, since a slow client and a slow app are separate problems. There is +one size limit for request bodies and one for response bodies, since +`smallwebwaf` passes bodies through unchanged. Bodies stream straight through, +so a request body reaches the app while the client is still sending it. - `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to send its request line and headers, and then, from the end of the headers, its body. - `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request - line and headers. Over it, Go's HTTP server answers `431` and closes the - connection, and nothing reaches the app. It must be more than `4K` and cannot - be `off`: Go's HTTP server always has such a limit, and reads 4 KiB past the - one it is given before it refuses. + line and headers. It must be more than `4K`. Go's HTTP server always has such + a limit, and reads 4 KiB past the one it is given before it refuses. - `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection may wait for its next request before `smallwebwaf` closes it. It is longer than the 90 seconds after which traefik, by default, closes an unused @@ -417,32 +376,30 @@ sending it. - `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. - `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. -When a limit is passed after the request line and headers have been read, but -before the response has started, `smallwebwaf` answers itself: +What the client gets when a limit is passed: -- `413` for a request body that is too large; -- `408` for a client too slow to send its body; -- `502` for a response that is too large; -- `504` for an app that is too slow. - -A request that announces a body larger than its limit is refused before anything -reaches the app. Once the response has started it can only be cut off, and the -connection is closed. - -Since a request body streams through, each side can hold up the other. While a -request body is still on its way, a request timeout that runs out answers `408` -if `smallwebwaf` was waiting for the client to send more at that moment. It -answers `504` if it was waiting for the app to take what it had. - -Go's HTTP server reads a request's line and headers before `smallwebwaf` sees -the request. A client that takes longer than `SWWAF_CLIENT_REQUEST_TIMEOUT` to -send them gets either no answer or an error answer from the server, mostly -`400`. Either way the server closes the connection and nothing reaches the app. -Headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` are answered `431` by the -server itself. Neither request gets a line in the request log. - -A WebSocket connection leaves these limits behind once it is upgraded. It stays -open until either side closes it. +- Go's HTTP server reads a request's line and headers before `smallwebwaf` sees + the request. It answers `431` to headers over + `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`. A client too slow to send them gets + either no answer or an error answer, mostly `400`. Either way the server + closes the connection, nothing reaches the app, and the request log gets no + line. +- A request that announces a body larger than its limit is refused before + anything reaches the app. +- When a limit is passed after the headers have been read, but before the + response has started, `smallwebwaf` answers itself: + - `413` for a request body that is too large; + - `408` for a client too slow to send its body; + - `502` for a response that is too large; + - `504` for an app that is too slow. +- A request body streams through, so each side can hold up the other. A request + timeout that runs out while the body is still on its way answers `408` if + `smallwebwaf` was waiting for the client at that moment. It answers `504` if + it was waiting for the app to take what it had. +- Once the response has started it can only be cut off, and the connection is + closed. +- A WebSocket connection leaves these limits behind once it is upgraded. It + stays open until either side closes it. ### Lookup of AS number and country @@ -450,55 +407,45 @@ On by default through GeoJS, which needs no account or file, so that country lookup works with no setup. The file is the alternative for a service that keeps its visitors' addresses on its own host. -- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. +- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. One source + at a time: `file` without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` + with any other source, stops the start with a message naming both. So does a + setting that needs lookups while the source is `off`: the country lists, the + biased thresholds, `SWWAF_ADD_LOOKUP_HEADERS`, and the anomaly thresholds per + AS number. - `SWWAF_LOOKUP_DB_PATH`: the file for `file`: the IPinfo Lite database in its - `.mmdb` form (`ipinfo_lite.mmdb`), one file that carries both country and AS - number. `smallwebwaf` reads its `asn`, `as_name` and `country_code` fields. - The operator downloads it with a free IPinfo account and mounts it read-only. - `smallwebwaf` never fetches it and holds no account or token for it. A file - that is missing or unreadable at start stops the start. -- Refreshing the file is the operator's business; IPinfo updates it daily. - `smallwebwaf` notices when the file is replaced and reads it again. A - replacement it cannot read is ignored, logged and sent as one `file_error` - alert, and the file it has stays in use. Mount the directory that holds the - file rather than the file itself: docker does not show a single mounted file - being replaced on the host. + `.mmdb` form (`ipinfo_lite.mmdb`), whose `asn`, `as_name` and `country_code` + fields are read. The operator downloads it with a free IPinfo account, mounts + it read-only, and refreshes it; IPinfo updates it daily. `smallwebwaf` never + fetches it and holds no account or token for it. A file that is missing or + unreadable at start stops the start. `smallwebwaf` notices when the file is + replaced and reads it again. A replacement it cannot read is ignored, logged + and sent as one `file_error` alert, and the file it has stays in use. The + directory that holds the file is mounted, not the file itself (see "Rule + files" for why). - `geojs` asks the free GeoJS web service about up to 200 addresses in one - request (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`). It reads each - answer's `country_code`, `asn` and `organization_name` (the AS name). An - answer without a `country_code`, which GeoJS sends for ranges it cannot place, - counts as unknown. So does an `asn` of `64512`, which GeoJS gives when it - knows none. GeoJS is told the address of every new visitor, whether or not a - setting uses the answer. The request log, the metrics, the client's history - and ban notes carry the AS number and country, so every answer is used. - Private, loopback and link-local addresses, which no source can place, are - never sent. + request (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a + time, and reads each answer's `country_code`, `asn` and `organization_name`. + An answer without a `country_code`, or with the `asn` `64512`, counts as + unknown. GeoJS is told the address of every new visitor, whether or not a + setting uses the answer, since the request log, the metrics and the ban notes + carry it. Private, loopback and link-local addresses, which no source can + place, are never sent. +- A client the lookup cannot place has an unknown country and AS number. The + exclusive country list refuses it, and the biased thresholds give it + `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it differently. - Each GeoJS answer is kept for 7 days, in memory and in `lookups.json`, apart - from the table of clients, so it outlasts a client dropped from that table. - After 7 days the client's next request asks again. Up to 100,000 answers are - kept, about 15 MiB. When that many are held, the answer used longest ago goes - first. + from the table of clients. Up to 100,000 answers are kept, about 15 MiB; past + that, the answer used longest ago goes first. - `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its - client's first answer when a setting needs it. GeoJS normally answers in a - fraction of that. At most one request to GeoJS is under way at a time. The - addresses that arrive meanwhile are asked about together in the next one, up - to 200 per request, the rest in the requests after it. A request that takes - longer than `SWWAF_LOOKUP_TIMEOUT` is abandoned. A client whose answer has not - come in time counts as unknown until it comes. Its later requests do not wait, - and its address is asked about again in the background. + client's first answer when a setting needs it, and how long a request to GeoJS + may take. A client whose answer has not come in time counts as unknown until + it comes, and its later requests do not wait. - GeoJS publishes no rate limit, but its terms forbid "an excessive amount of API requests", judged by GeoJS alone, and let it block a caller. While GeoJS - is slow, down or refusing `smallwebwaf`, clients with a kept answer are - unaffected and new clients count as unknown. So - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`, when set, refuses them. `smallwebwaf` - keeps asking with backoff and sends one `source_failure` alert per cooldown. A - service that cannot accept this uses the file. -- One source at a time. `SWWAF_LOOKUP_SOURCE=file` without - `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other source, stops - the start with a message naming both. So does a setting that needs lookups - while `SWWAF_LOOKUP_SOURCE` is `off`: the country lists, the biased - thresholds, `SWWAF_ADD_LOOKUP_HEADERS`, and the anomaly thresholds per AS - number. + is slow, down or refusing `smallwebwaf`, new clients count as unknown, + `smallwebwaf` keeps asking with backoff, and one `source_failure` alert is + sent per cooldown. A service that cannot accept this uses the file. - `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): pass `X-Client-ASN` and `X-Client-Country` to the app. @@ -511,20 +458,20 @@ where it comes from. Clients in `SWWAF_ALLOW_NETS` are not checked. a listed country is refused. - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`: for example `us,de`. Only clients placed in a listed country get through. Every request from any other country - is refused. So is every request from a client the lookup cannot place: - - one whose address is missing from the database; - - one GeoJS has not answered for in time; - - any client on a private address, such as a visitor on the local network, - another container or internal monitoring. - - Those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let - unplaced clients through would let every new client in whenever GeoJS stops - answering. - + is refused. So is every request from a client the lookup cannot place, such as + a visitor on the local network, another container or internal monitoring. + Those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let + unplaced clients through would let every new client in whenever GeoJS stops + answering. That is the nearest a failing helper comes to taking the service + down: GeoJS failing while this list is set refuses every new client. - Both may be set. `SWWAF_DENIED_COUNTRIES` then adds nothing, since the exclusive list already refuses every other country. A code on both lists stops the start with a message naming it. -- What a refused request gets is under "What happens to a request", step 6. +- A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the + client's address has been looked up, before its body is read. It skips the + reputation checks, the limits, the rule files and the Core Rule Set. It is + logged with the action `country_denied`, and counted in the metrics. It is a + refusal, not a ban. ### Biased thresholds @@ -538,12 +485,6 @@ abuse comes from. is a zero allowance: the client's first request breaks a limit and bans it. - `SWWAF_COUNTRY_LIMIT_PERCENT`: the same form with country codes, for example `CN:25,RU:50`. -- No budget is shared by a whole country or AS number. One abuser could use it - up and lock out everyone else there: a denial of service nobody chose, and the - one this tool exists to prevent. Refusing a whole country is left to the - operator's choice, through the country lists. -- When more than one percentage applies to a client (its AS number, its country, - a reputation hit), the lowest applies. - `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT`: optional overrides applied to the byte limits only, when the byte percentage should differ from the request percentage. @@ -552,6 +493,14 @@ abuse comes from. - `SWWAF_ASN_LIMIT_PERCENT_URL`: optional URL of a text file of `AS:percent` lines, so one abuse-source list can be shared by every `smallwebwaf` in the fleet. It is fetched and refreshed like the blocklists. +- A client's limit percentage is the lowest of those that apply to it: its AS + number's, its country's, or a reputation hit's `limit:`. It is 100 if + none applies. Clients given a lower percentage are banned after fewer + requests, first for a while and then, for repeated abuse, permanently. +- No budget is shared by a whole country or AS number. One abuser could use it + up and lock out everyone else there: a denial of service nobody chose, and the + one this tool exists to prevent. Refusing a whole country is left to the + operator's choice, through the country lists. ### Attack detection @@ -567,8 +516,7 @@ the bans still apply to the client that sends it, and `SWWAF_WAF_BODY_LIMIT` switches body inspection on. - `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of rule - files, read at start and again whenever a file in it changes (see "Rule - files"). + files (see "Rule files"). - `SWWAF_RULES_ENABLED` (default `true`): `false` skips rule files entirely. - `SWWAF_WAF_MODE` (default `block`): `off`, `detect` (log and alert only), or `block`. @@ -587,9 +535,8 @@ switches body inspection on. In front of a code forge these refuse git over HTTP, container image and package uploads, and views of ordinary files in a repository. At the site root, where no app serves such files, the default rule file bans the common - probes these rules caught, such as `/.env`, `/.git/config`, - `/.aws/credentials`, `/.ssh/id_rsa`, `/.htpasswd` and `/wp-config.php.bak`. - A list given replaces the default, so include them in it. + probes these rules caught (see "Rule files"). A list given replaces the + default, so include them in it. - `SWWAF_WAF_EXEMPT_PATHS`: path prefixes not inspected. - `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size, @@ -598,9 +545,10 @@ switches body inspection on. on without holding it in memory. It reads JSON and XML bodies no larger than the size, since those cannot be read in part. Any other body is not read: the Core Rule Set would read it as form data, where binary content such as a git - push trips rules written for text. Body inspection suits apps whose forms - carry no code. In front of gitea it refuses, until the rule ids the request - log names are added to `SWWAF_WAF_DISABLED_RULES`: + push trips rules written for text. Responses are never inspected. Body + inspection suits apps whose forms carry no code. In front of gitea it refuses + the requests below, until their rule ids, named in the request log, are added + to `SWWAF_WAF_DISABLED_RULES`: - issue and comment text, wiki pages and files saved in the web editor that hold shell commands or code (932125, 932235, 932250 and others); - package descriptions that show code; @@ -616,23 +564,19 @@ switches body inspection on. trying one attack or token after another is refused many times a minute; a person rarely more than a few times. Answers the app gives are not counted, since they do not tell a scanner from an ordinary client. In front of gitea, - container image and package clients are answered `404` by design, often - hundreds of times a minute. git and registry clients are answered `401` at the - start of every push, every fetch from a private repository and every image - pull. - -### The six changes to the Core Rule Set + registry and git clients are answered `404` and `401` by design, hundreds of + times a minute. `smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of -every app, and no setting undoes them. +every app, and no setting undoes them: - `PUT`, `PATCH` and `DELETE` are allowed methods besides `GET`, `HEAD`, `POST` and `OPTIONS`. APIs, container image pushes and package uploads use them. Other methods stay refused. - The request header `Expect` is allowed, since curl and git send it before some large uploads. `Content-Encoding` is allowed on a request whose body the Core - Rule Set does not read, which by default is every request: git compresses a + Rule Set does not read, which by default is every request. git compresses a fetch request over 1 KiB and says so in that header. On a body the Core Rule Set does read, the header stays refused, since a compressed body cannot be inspected. The other headers the Core Rule Set refuses, such as `Proxy`, stay @@ -644,47 +588,40 @@ every app, and no setting undoes them. address. - Ten query parameters are not checked against the Core Rule Set's lists of system files (930120), shell paths (932160) and command names (932260): - `path`, `files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`, - `artifactName` and `redirect_to`. gitea sends file paths, branch and workflow - names, and the page to return to after signing in, in them. In a repository - any name on those lists can be an ordinary file or branch, such as - `.gitignore`, `package.json`, `docker-compose.yml`, `bin/docker-entrypoint` or - a branch named `docker-build`. gitea reads these values as names within a - repository or its own records, or as a page of its own site. What only these - three rules refuse is let through there. That is more than a name such as - `/etc/passwd` or `whoami` on its own: commands such as `|cat /etc/passwd`, - `wget http://…` and `nc -e /bin/sh …`, and `file:///etc/passwd`, pass as well. - Path traversal (`../`), SQL and script injection, and PHP, Java and Node.js - code are still refused there. Every other parameter keeps all three rules. - These names, and `redirect_uri`, are matched without regard to case, as Coraza - matches them. An app that uses one of these parameters as a file on the - server, or passes it to a shell, gets no help from the three rules there. See - "Risks the design has to handle". + - `path`, `files`, `skip-to` and `sub_path`; + - `ref`, `sha`, `branch` and `workflow`; + - `artifactName` and `redirect_to`. + + gitea puts file paths, branch and workflow names, and the page to return to + after signing in into them. Any name on those lists can be an ordinary file + or branch, such as `.gitignore` or `docker-build`. In these parameters, + anything only these three rules catch reaches the app, commands such as + `|cat /etc/passwd` included. Path traversal, SQL and script injection, and + PHP, Java and Node.js code are still refused there. Every other parameter + keeps all three rules. Parameter names are matched without regard to case, + as Coraza matches them. + - The Core Rule Set reads the request without the `gitea_flash` and `redirect_to` cookies. It does not check `Referer` for a Unix command given - without arguments (932340) or for Java starting a process (944110). gitea - writes into `gitea_flash` a message for the next page naming what was just - done, and into `redirect_to` the page to return to after signing in. `Referer` - is the address of the page a request comes from. In these cookies the Core - Rule Set takes names such as `package.json`, `document.write.js` or - `Get-ChildItem.ps1`, and titles that hold them, for attacks. In `Referer` it - takes the address of a search for one word, such as `env`, `set` or `last`, - when the address ends in it, and the address of OpenJDK's - `src/java.base/share/classes/java/lang/Runtime.java`. A browser sends such a - cookie with every request until gitea replaces or removes it. gitea cannot do - that while `smallwebwaf` refuses those requests, so one refusal would keep the - browser out of the whole site. A browser names a page in `Referer` on - everything the page loads and on every link followed from it, so all of those - would be refused. gitea shows the message with any script removed and returns - only to a page of its own site. An attack sent in a cookie of either name is - not refused by the Core Rule Set. Nor is a Unix command without arguments, or - Java starting a process, sent in `Referer`. Every other cookie is read in - full, and `Referer` keeps the other rules, script and SQL injection among - them. + without arguments (932340) or for Java starting a process (944110). gitea puts + a message naming what was just done into `gitea_flash`. It puts the page to + return to after signing in into `redirect_to`. The Core Rule Set takes file + names in them, such as `package.json`, for attacks. A browser sends such a + cookie with every request until gitea replaces it. gitea cannot replace it + while those requests are refused, so one refusal would keep the browser out of + the whole site. `Referer` names the page a request comes from, on everything + that page loads. The Core Rule Set takes some page addresses for attacks, such + as a search for `env`, or OpenJDK's `Runtime.java`, whose path holds both + `runtime` and `java.`. gitea shows the message with any script removed and + returns only to its own pages. Every other cookie is read in full, and + `Referer` keeps the other rules. - Responses are not inspected. A raw file from a repository, such as a shell script, looks to the response rules like source code leaking from the server. -### Bans +The fourth and fifth changes leave an app on its own in those places (see "Risks +the design has to handle"). + +### Ban lengths and response - `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of attack. @@ -704,24 +641,25 @@ every app, and no setting undoes them. ### Reputation No source is on by default, for the reason the country lists and the biased -thresholds are empty. A source refuses the clients that a list kept elsewhere -names, or lowers their limits, while the defaults judge a client only by what it -does to this service. AbuseIPDB and CrowdSec also need an account or an engine -of the operator's own. Besides, the list best suited to be on by default, -Spamhaus DROP, must not be downloaded automatically more than once an hour. -Spamhaus may block an address that downloads it too often. Each `smallwebwaf` -fetches its own copy. On a host that runs several apps with it, which share one -address, downloads would often come less than an hour apart. +thresholds are empty. A source acts on a list kept elsewhere; the defaults judge +a client only by what it does here. AbuseIPDB and CrowdSec also need an account +or an engine of the operator's own. Besides, the list best suited to be on by +default, Spamhaus DROP, must not be downloaded automatically more than once an +hour. Spamhaus may block an address that downloads it too often. Each +`smallwebwaf` fetches its own copy. On a host that runs several apps with it, +which share one address, downloads would often come less than an hour apart. - `SWWAF_BLOCKLIST_URLS`: text files of addresses and netblocks, one per line; anything after a `;` or `#` on a line is ignored. For example the Spamhaus DROP list, `https://www.spamhaus.org/drop/drop.txt`. Spamhaus's terms for it, - on its DROP page: automated downloads must be at least one hour apart, and - once a day is more than enough in most cases; a product that uses it must - credit The Spamhaus Project and keep the list's date and copyright lines with - the data. Refreshed every `SWWAF_BLOCKLIST_REFRESH` (default `24h`, never less - than `1h`). The last good copy is kept whole, comment lines included, on - failure and across restarts. + on its DROP page: + - automated downloads must be at least one hour apart, and once a day is + more than enough in most cases; + - a product that uses it must credit The Spamhaus Project and keep the + list's date and copyright lines with the data. +- `SWWAF_BLOCKLIST_REFRESH` (default `24h`, never less than `1h`): how often + each list is fetched again. The last good copy is kept whole, comment lines + included, on failure and across restarts. - `SWWAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:` or `log`. `deny` refuses every request from a listed address, which suits lists of networks that send nothing legitimate, such as DROP. `limit:` gives @@ -740,17 +678,16 @@ address, downloads would often come less than an hour apart. queried, so the budget is spent on suspects. - `SWWAF_CROWDSEC_LAPI_URL` and `SWWAF_CROWDSEC_LAPI_KEY`: optional. If a CrowdSec engine exists on the host, its decision list is pulled on a schedule - and listed addresses are treated as banned. This is how a fleet-wide blocklist - can arrive without `smallwebwaf` depending on CrowdSec. The list is kept like - a fetched blocklist. A listed client's request makes a ban with the cause - `crowdsec` that lasts as long as CrowdSec's decision, so a long list does not - fill `bans.json`. + and treated as a blocklist (see "Bans" for the ban it makes). This is how a + fleet-wide blocklist can arrive without `smallwebwaf` depending on CrowdSec. - `SWWAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:` or `log`, for DNS blocklist and AbuseIPDB hits. Such verdicts are less certain than a blocklist, so by default a listed client gets a quarter of every limit and is banned after a quarter of the requests. - `SWWAF_REPUTATION_CACHE_TTL` (default `24h`) and `SWWAF_REPUTATION_TIMEOUT` (default `2s`). +- A source that is down or over quota gives no verdict. The service continues, + with one `source_failure` alert per cooldown. ### Alerting @@ -763,14 +700,17 @@ address, downloads would often come less than an hour apart. - `SWWAF_ALERT_EVENTS` (default `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): which event types are sent. `source_failure` is a reputation source or GeoJS - failing or refusing `smallwebwaf`. `file_error` is a rule file or state file - edited while running that does not parse, a replacement lookup database that - cannot be read, or a state file that cannot be written. + failing or refusing `smallwebwaf`. `file_error` is one of: + - a rule file or state file edited while running that does not parse; + - a replacement lookup database that cannot be read; + - a state file that cannot be written. - `SWWAF_ALERT_COOLDOWN` (default `15m`): the same event for the same netblock, source, file or anomaly scope is not repeated within this time. A count of suppressed repeats is included in the next one. - `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): beyond this, alerts are rolled into one summary per hour, so a wide attack cannot flood the channel. +- A destination that is down is retried with backoff from a bounded queue, the + oldest alert dropped first, and the drops counted in the metrics. ### Anomaly thresholds @@ -799,15 +739,16 @@ which the metrics show. netblock as a whole. - Anomaly counting includes clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS`, since an exempt client misbehaving is worth - knowing about. + knowing about. At most 20,000 counters are kept, in memory and in + `alerts.json`. ## Rule files `smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR` and checks each -request against them. The files are plain text meant to be edited by hand, so -that a new scanning pattern seen in the request log can be turned into a rule in -one line. `smallwebwaf` watches the directory: a file edited, added or removed -there takes effect while it runs. +request against them. The files are plain text meant to be edited by hand. A new +scanning pattern seen in the request log can be turned into a rule in one line. +`smallwebwaf` watches the directory: a file edited, added or removed there takes +effect while it runs. - One rule per line, four fields separated by spaces or tabs. The fourth field runs to the end of the line: @@ -834,12 +775,10 @@ there takes effect while it runs. - `log`: note the match in the request log and do nothing else. - `block`: refuse the request with `403`. The client is not banned for it, but the refusal counts toward the error burst. - - `ban`: the request is a clear sign of attack. Refuse it and ban the - client's netblock for `SWWAF_ATTACK_BAN_DURATION`. Any further request - during the ban, or a later clear sign of attack, makes the ban permanent. - Use it only for requests no real visitor sends. Anchor a path at the site - root (`^/`): a file of the same name deeper in a site can be ordinary - content, such as a file in a gitea repository. + - `ban`: the request is a clear sign of attack (see "Bans"). Use it only for + requests no real visitor sends. Anchor a path at the site root (`^/`): a + file of the same name deeper in a site can be ordinary content, such as a + file in a gitea repository. - `regex`: Go regular expression syntax (RE2). It has no backreferences or lookaround. In exchange matching time is linear in the input, so no rule can be made to stall the proxy. `(?i)` at the front makes a rule ignore case. A @@ -849,22 +788,20 @@ there takes effect while it runs. rules in line order. - Rules are compiled when their file is read and held in memory. At start, a line that does not parse, a regex that does not compile, or a duplicate id - stops the process with a message naming the file and line. While running, the - same faults leave the rules as they were, including the earlier version of - that file. The log and one `file_error` alert name the file and line. Once the - file is fixed, it is read again. A `SWWAF_RULES_DIR` that does not exist stops - the start with a message naming it. An empty directory is not an error: the - log says that no rules were loaded. To run without rule files, mount an empty - directory or set `SWWAF_RULES_ENABLED=false`. + stops the process with a message naming the file and line. So does a + `SWWAF_RULES_DIR` that does not exist. While running, the same faults leave + the rules as they were, and the log and one `file_error` alert name the file + and line. Once the file is fixed, it is read again. An empty directory is not + an error. To run without rule files, mount an empty directory or set + `SWWAF_RULES_ENABLED=false`. - The image ships a default file in `SWWAF_RULES_DIR`. The app's Dockerfile can copy files of its own into it, beside the default. Mounting a directory over - it replaces the defaults; mounting single files into it adds to them. Docker - does not show a single mounted file being replaced on the host, which is how - many editors save. So rules meant to be edited while `smallwebwaf` runs belong - in a mounted directory, with a copy of the default file if the defaults are to - stay. -- In `SWWAF_MODE=observe` every action is logged as what would have happened and - nothing is refused. + it replaces the defaults; mounting single files into it adds to them. Many + editors save a file by replacing it, and a container keeps seeing the old one + when the file alone is mounted. So rules meant to be edited while + `smallwebwaf` runs belong in a mounted directory, with a copy of the default + file if the defaults are to stay. The lookup database is mounted the same way, + for the same reason. - Clients in `SWWAF_ALLOW_NETS` are not checked. The default file the image ships: @@ -887,16 +824,13 @@ path-traversal uri block (\.\./){2,} empty-agent user_agent log ^$ ``` -It is kept short and limited to patterns that are wrong for every app. Its path -rules are anchored at the site root. In front of gitea, `/.env` is a probe, +It is kept short and limited to patterns that are wrong for every app, with its +path rules anchored at the site root. In front of gitea, `/.env` is a probe, while `///src/branch/main/.env.example` is a file in a repository -that any visitor or search crawler may open. The Core Rule Set's rules for file -names and extensions (930130, 920440) refuse such names anywhere in a path, -which is why the default `SWWAF_WAF_DISABLED_RULES` switches them off: a code -forge serves such files deeper in its paths. Anything app-specific belongs in a -file the app's Dockerfile adds or the deployer mounts. A request for -`/wp-login.php`, for example, is a clear sign of attack in front of gitea and an -ordinary login in front of WordPress: +that any visitor may open. Anything app-specific belongs in a file the app's +Dockerfile adds or the deployer mounts. A request for `/wp-login.php`, for +example, is a clear sign of attack in front of gitea and an ordinary login in +front of WordPress: ``` # 50-gitea.rules: WordPress probes, which gitea never serves @@ -908,16 +842,18 @@ wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/) All state lives in memory, and the files in `SWWAF_STATE_DIR` hold a copy of all of it, so an orderly stop loses nothing. The one exception is log lines still waiting to be sent to `SWWAF_LOG_REMOTE_URL`, which stdout has already carried. -No database is used, and nothing is read from disk while serving a request. The -files are meant for people as well: an admin can read or edit them at any time, -and the running `smallwebwaf` takes the edit in. +The files are meant for people as well: an admin can read or edit them at any +time, and the running `smallwebwaf` takes the edit in. The files, each holding one kind of state: - `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf` - made and every ban an admin made. Per entry: the netblock, start, expiry - (`null` for permanent), the cause (`attack`, `limit`, `admin` or `crowdsec`), - a short reason, when it was lifted if an admin lifted it, and the notes below. + made and every ban an admin made. Per entry: + - the netblock, start and expiry (`null` for permanent); + - the cause: `attack`, `limit`, `admin` or `crowdsec`; + - a short reason; + - when it was lifted, if an admin lifted it; + - the notes below. - `clients.json`: per client, its minute, hour and day counters and its history since it was first seen. The history holds: - the first and last time seen; @@ -931,16 +867,17 @@ The files, each holding one kind of state: survives a restart. - `lookups.json`: GeoJS answers, one per client: the AS number, AS name and - country, when GeoJS was asked and when the answer was last used. Up to - 100,000, each for 7 days. + country, when GeoJS was asked and when the answer was last used. - `reputation.json`: the last good copy of each list fetched from a URL; the cached DNS blocklist verdicts and AbuseIPDB scores, each with the time it was fetched; and the AbuseIPDB checks spent today, so that a restart does not reset the daily budget. -- `alerts.json`: the anomaly counters; for each event and netblock, source, file - or anomaly scope, when it was last sent and how many repeats the cooldown has - held back since; the alerts sent this hour; and the alerts still waiting to be - sent. +- `alerts.json`: + - the anomaly counters; + - for each event and netblock, source, file or anomaly scope, when it was + last sent and how many repeats the cooldown has held back since; + - the alerts sent this hour; + - the alerts still waiting to be sent. Ban notes hold what an admin needs to decide whether to lift a ban, drawn from what `smallwebwaf` already knows. Nothing is looked up to fill them: @@ -948,35 +885,37 @@ what `smallwebwaf` already knows. Nothing is looked up to fill them: - the AS number, AS name and country, added when the lookup answers (empty when lookups are off); - what was broken: the rule id and target that matched, or the trap path, or the - limit, its window, the count reached and the client's limit percentage with - what set it; and any reputation sources that listed the client; + limit, its window and the count reached; +- the client's limit percentage, with the setting that gave it; +- the reputation sources that listed the client; - the request that caused the ban: time, method, host, path with its query string, status and user agent, each text cut to 256 bytes; -- how many requests counted toward the ban, and the window over which they came; - the netblock's total requests since it was first seen, and the requests refused under this ban so far, kept up to date while the ban lasts; - how many earlier bans of each kind the netblock has had. -Size and disk writes. Each file is rewritten whole, so each has a bound: +Each file is rewritten whole, so each has a bound. `clients.json` takes about 1 +KiB per client, about 20 MiB at the default `SWWAF_MAX_TRACKED_CLIENTS`, which +written every 15 minutes is under 2 GiB of disk writes a day. `bans.json` takes +about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at the default +`SWWAF_MAX_BANS`. With a full file, a hundred new bans a day come to about 600 +MiB of writes. `lookups.json` takes about 150 bytes per answer, about 15 MiB +when full, under 1.5 GiB of writes a day. The other two files are usually a few +MiB or less. Writing a file of these sizes takes well under a second on an +ordinary disk, in the background, so rewriting whole files needs nothing +cleverer. -- `clients.json` takes about 1 KiB per client. On a public service the file - grows to the default `SWWAF_MAX_TRACKED_CLIENTS` of 20,000, about 20 MiB. - Written every 15 minutes, that is under 2 GiB of disk writes a day. -- `bans.json` takes about 1.2 KiB per ban and at most about 2.5 KiB, since the - notes hold one request and their texts are cut short. At the default - `SWWAF_MAX_BANS` of 5,000 it is about 6 MiB, and never more than about 12 MiB, - plus whatever bans an admin made. -- `lookups.json` takes about 150 bytes per answer, about 15 MiB when full. -- `reputation.json` and `alerts.json` are usually a few MiB or less. -- Writing a file of these sizes takes well under a second on an ordinary disk, - in the background, so rewriting whole files needs nothing cleverer. +Format: -Format: indented JSON with a top-level `version` number, entries sorted by -client address, times in RFC 3339 UTC, durations and sizes as plain numbers with -the unit in the field name. The aim is that a person can open `bans.json` in an -editor, find an address, and remove or add an entry. `clients.json` and -`lookups.json` put each entry on one line instead. That halves their size and -lets `grep` show everything about one client. +- indented JSON with a top-level `version` number; +- entries sorted by client address; +- times in RFC 3339 UTC; +- durations and sizes as plain numbers, with the unit in the field name; +- `clients.json` and `lookups.json` put each entry on one line instead. That + halves their size and lets `grep` show everything about one client. + +The aim is that a person can open `bans.json` in an editor, find an address, and +remove or add an entry. Writing: @@ -985,10 +924,10 @@ Writing: - write to a temporary file in the same directory, sync it, rename it over the real name, sync the directory. A crash at any point leaves either the old complete file or the new complete file, never a partial one; -- `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted - or made permanent. Changes that only update the counts in its notes wait for - the write every `SWWAF_STATE_COUNTER_INTERVAL`, as the other four files do. - All five are written on orderly shutdown (`SIGTERM`); +- `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban changes. Changes + that only update the counts in its notes wait for the write every + `SWWAF_STATE_COUNTER_INTERVAL`, as the other four files do. All five are + written on orderly shutdown (`SIGTERM`); - before writing a file, `smallwebwaf` checks whether it changed on disk since it last read or wrote it. If it did, `smallwebwaf` takes that edit in first, so an admin's edit is never overwritten. @@ -1002,6 +941,7 @@ Reading at start: with a message naming the file and position. Starting with empty state would silently forgive every repeat offender. Since writes are atomic, a broken file can only come from a hand edit, which the editor should hear about. +- `SWWAF_STATE_DIR` not writable at start: the process exits. Edits while running: @@ -1019,16 +959,21 @@ Edits while running: logs the file and the position of the error, and sends them in one `file_error` alert. The admin fixes the `.bad` file and moves it back. -`SWWAF_STATE_DIR` not writable at start: the process exits. A write that fails -while running: state stays correct in memory, the failure is logged, counted in -the metrics, sent as a `file_error` alert once per cooldown, and retried at the -next write. +A write that fails while running: -What a hard kill can lose: ban changes from the last `SWWAF_STATE_WRITE_DELAY` -(10 seconds), and everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` -(15 minutes): counts and history, the counts in ban notes, GeoJS answers, -reputation verdicts and the AbuseIPDB count, anomaly counters and alert -cooldowns. Losing the volume loses ban history, not service. +- state stays correct in memory; +- the failure is logged and counted in the metrics; +- one `file_error` alert is sent per cooldown; +- the write is retried at the next write. + +What a hard kill can lose: + +- ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds); +- everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes): + counts and history, the counts in ban notes, GeoJS answers, reputation + verdicts and the AbuseIPDB count, anomaly counters and alert cooldowns. + +Losing the volume loses ban history, not service. ## Request log @@ -1091,40 +1036,44 @@ log handling while `docker logs` keeps working. ## Metrics endpoint Prometheus text format at `/_smallwebwaf/metrics`, for a request carrying -`SWWAF_METRICS_TOKEN`. While the token is unset, the metrics are off. The token -is separate from `SWWAF_ADMIN_TOKEN`, so a scraper that holds it cannot manage -bans. +`SWWAF_METRICS_TOKEN`. - Traffic: requests and bytes in and out, by status class and `action`; request duration and upstream duration histograms; requests in flight. -- Limits and bans: limit hits by window and kind (requests, bytes or refusals); - size and time limit hits by limit; offences by kind; bans made by cause; - permanent bans; active bans; requests refused by the country lists, by - country. -- Attack detection: rule file matches by rule id and action, and the number of - rules loaded; Core Rule Set matches by mode and rule id, limited to the rules - that actually fired. -- Lookup and reputation: requests and bytes by AS number and by country, limited - to the `SWWAF_METRICS_TOP_N` busiest of each with the rest summed as `other`, - so the label set stays bounded; reputation queries, hits, failures and - remaining daily budget by source; GeoJS requests and failures, and clients - that counted as unknown because it did not answer in time; when each list was - last fetched and the lookup database last read. -- Housekeeping: tracked clients; state file writes, write failures, last - successful write time and size per file; edits taken in, and edits set aside - because they did not parse; alerts sent, failed, suppressed and dropped by - destination; remote log lines sent, dropped and buffer depth; the standard Go - runtime and process metrics. +- Limits and bans: + - limit hits by window and kind (requests, bytes or refusals); + - size and time limit hits by limit; + - offences by kind; + - bans made by cause, permanent bans, active bans; + - requests refused by the country lists, by country. +- Attack detection: + - rule file matches by rule id and action, and the number of rules loaded; + - Core Rule Set matches by mode and rule id, limited to the rules that + fired. +- Lookup and reputation: + - requests and bytes by AS number and by country, limited to the + `SWWAF_METRICS_TOP_N` busiest of each with the rest summed as `other`, so + the label set stays bounded; + - reputation queries, hits, failures and remaining daily budget by source; + - GeoJS requests and failures, and clients that counted as unknown because + it did not answer in time; + - when each list was last fetched, and the lookup database last read. +- Housekeeping: + - tracked clients; + - state file writes, write failures, last successful write time and size per + file; edits taken in, and edits set aside because they did not parse; + - alerts sent, failed, suppressed and dropped by destination; + - remote log lines sent, dropped and buffer depth; + - the standard Go runtime and process metrics. - No metric carries a client IP address as a label. Per-address questions are answered by the request log and `GET /_smallwebwaf/clients/`. ## Admin endpoints -`smallwebwaf` has one listener, on port 8080. A request whose path starts with -`/_smallwebwaf/` is for `smallwebwaf` itself: it answers it and never passes it -to the app. The prefix carries the tool's name, so it takes no path an app uses. -Admins and scrapers reach these endpoints through traefik, like any other -request. +A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself: +it answers it and never passes it to the app. The prefix carries the tool's +name, so it takes no path an app uses. Admins and scrapers reach these endpoints +through traefik, like any other request. - `GET /_smallwebwaf/healthz`: answers `200` with `ok` to anyone, without a token, while `smallwebwaf` is running. It does not ask the app; the @@ -1139,9 +1088,9 @@ request. current counters and history, which holds its lookup result and offences, and its bans with their notes. It answers "why was this address refused" and "what has this netblock been doing". -- A token is sent as `Authorization: Bearer `. One set shorter than 32 - characters stops the start. While a token is unset, the endpoints that need it - answer `404`, as does any other path under the prefix. +- A token is sent as `Authorization: Bearer ` (see "Tokens" under + "Settings"). While a token is unset, the endpoints that need it answer `404`, + as does any other path under the prefix. - Apart from the health check, these requests go through every check that any other request goes through. They are answered at the point where another request would be forwarded to the app: a banned or refused client stays @@ -1169,42 +1118,27 @@ One JSON object per alert: ## Deployment `smallwebwaf` runs inside the app's own container: the app's Dockerfile builds -`FROM` the `smallwebwaf` image. Each app remains one container, deployed as -before. An app is hardened just by changing its base image, and perhaps -installing on top of it, from nixpkgs, the packages it needs. In the container, -`smallwebwaf` listens on port 8080, which traefik routes to, and forwards each -request to the app on `127.0.0.1:8081`, its default `SWWAF_UPSTREAM_URL`. No -setting is required. +`FROM` the `smallwebwaf` image. Each app remains one container. An app is +hardened by changing its base image. It can add packages from nixpkgs. The image holds: - Ubuntu 26.04 LTS, the newest long-term support release of Ubuntu, pinned by digest. The image moves to the next LTS release when that ships. -- Three packages from Ubuntu, `ca-certificates`, `nix-bin` and `runit`, - installed from a dated snapshot of Ubuntu's archive and checked by hash (see - "Packages from Ubuntu"). The Ubuntu image has no CA certificates. They come - with `nix-bin` only because a library it uses recommends them, so - `ca-certificates` is installed by name. Without it Nix cannot download - packages, and `smallwebwaf`, unable to reach GeoJS, would count every visitor - as coming from an unknown country. -- Nix, the package manager, from Ubuntu's own `nix-bin` package, and nixpkgs, - the collection of packages Nix installs from, fixed at one commit (see - "Packages from nixpkgs"). Root uses Nix directly, and no Nix daemon runs in - the container. Nix run by root expects a group of build users, `nixbld`, which - `nix-bin` does not create. So the image writes `build-users-group =` to - `/etc/nix/nix.conf`, and root's builds then run without build users. -- runit, from Ubuntu's own `runit` package, and `runsvinit` as the entrypoint, - as the owner's code style guide asks for service containers. Neither Ubuntu - nor nixpkgs packages `runsvinit`, so the image builds it from its source - (`github.com/peterbourgon/runsvinit`) at a fixed commit hash. Its repository - is archived and has not changed since 2015, and its last tag is `v2.0.0`. It - has no `go.mod`, and `go build` of its directory needs one, so the build - writes one. Since `runsvinit` uses only Go's standard library, that file names - nothing else. `runsvinit` starts runit's `runsvdir`, which starts a `runsv` - for each directory under `/etc/service`. Each `runsv` runs the `run` script in - its directory, and runs it again whenever it exits. Ubuntu's runit looks for - services in `/etc/service` too, so when `docker stop` has `runsvinit` stop - each service with runit's `sv`, `sv` finds it. +- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, installed from a dated + snapshot of its archive and checked by hash (see "Pinning"). The Ubuntu image + has no CA certificates; without them Nix cannot download packages and + `smallwebwaf` cannot reach GeoJS. +- Nix and nixpkgs, fixed at one release (see "Pinning"). Root uses Nix directly; + no Nix daemon runs, and `build-users-group =` in `/etc/nix/nix.conf` lets root + build without the build users `nix-bin` does not create. +- runit, and `runsvinit` as the entrypoint, as the owner's code style guide asks + for service containers. No package carries `runsvinit`, so the image builds it + from its archived source (`github.com/peterbourgon/runsvinit`, unchanged + since 2015) at a fixed commit hash, with a `go.mod` the build writes. + `runsvinit` starts runit's `runsvdir`, which runs the `run` script of each + directory under `/etc/service` and runs it again whenever it exits. Ubuntu's + `sv` looks for services there too. - The `smallwebwaf` binary, and a user of its own, `smallwebwaf` (uid and gid 65532). - The service directory `/etc/service/smallwebwaf`. Its `run` script waits one @@ -1227,10 +1161,10 @@ Beyond its `FROM` line, the app's Dockerfile adds: - any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.`; - its runit service: a directory under `/etc/service` named after the app, never `smallwebwaf`. Its `run` script waits one second (`sleep 1`) and then starts - the app with `chpst`, listening on `127.0.0.1:8081`, as a user of its own that - the Dockerfile creates with `useradd`. That user is neither root nor - `smallwebwaf`, so that the app cannot touch the state files or the process of - `smallwebwaf`. + the app with `chpst`, listening on `127.0.0.1:8081`. The app runs as a user of + its own, which the Dockerfile creates with `useradd`. That user is neither + root nor `smallwebwaf`, so that the app cannot touch the state files or the + process of `smallwebwaf`. It sets no `ENTRYPOINT` or `USER` of its own. The container must start `runsvinit`, as root, so that it can start each service as its own user. @@ -1269,90 +1203,53 @@ main() { main "$@" ``` -### Packages from Ubuntu +### Pinning -The image installs `ca-certificates`, `nix-bin` and `runit` from Ubuntu's -snapshot service, which serves the archive as it was at a given moment. The -archive itself changes with every update, so its packages are not pinned. The -image's Dockerfile names that moment, in apt's `--snapshot 20261001T000000Z` -form, on both `apt-get update` and `apt-get install`. That moment is never -earlier than the date of the pinned Ubuntu image, since packages from an older -snapshot can need older versions of packages the Ubuntu image already holds. It -moves forward whenever that image's digest does. - -`apt-get update` keeps the snapshot's `InRelease` files, which apt checks -against the archive's signature, in `/var/lib/apt/lists/`. Each lists the -SHA-256 hash of the package lists it covers, and each package list the hash of -every package in it. The Dockerfile also names the SHA-256 hash of each of the -snapshot's `InRelease` files. The build checks them after `apt-get update` and -before `apt-get install`, so every package apt installs is checked, through -those files, against hashes the Dockerfile names. `apt-get update` also fetches -the live archive's `InRelease` files into the same directory. Their hashes -change whenever the archive does, and the install does not use them, so the -check leaves them out. The hashes are those of the archive for amd64, and so the -image is built for amd64: other architectures use Ubuntu's ports archive, whose -`InRelease` files differ. - -The snapshot service is reached over HTTPS, and the Ubuntu image has no CA -certificates of its own. So this one install uses those of the Go image that -`smallwebwaf` is built in, which is pinned by digest too: apt's -`Acquire::https::CaInfo` option names that image's CA certificate file, -`/etc/ssl/certs/ca-certificates.crt`, mounted for that one step. - -### Packages from nixpkgs - -nixpkgs is fixed at one commit of its newest release branch, `nixos-26.05` -today. For each commit of the branch that has passed its tests, the Nix project -publishes a release on `releases.nixos.org`, such as -`nixos-26.05.11045.774debe7a0d1`. The image takes nixpkgs from that release's -file `nixexprs.tar.xz`, not from a GitHub archive of the commit, whose bytes can -change. The image's Dockerfile names the release and the SHA-256 hash of that -file, which the release's page lists. The build checks the hash before unpacking -it. - -nixpkgs is set up for root under the name `nixpkgs`, so the app's Dockerfile -installs a package with `nix-env -iA nixpkgs.`. Whatever it installs is on -the `PATH` of every service. The image adds root's Nix profile, -`/nix/var/nix/profiles/default/bin`, at the end of the `PATH`, after Ubuntu's -own directories, so that no package hides the image's own commands. busybox, for -one, brings its own `sv`, which looks for services elsewhere. Because nixpkgs -stays at that commit, an app built on the same `smallwebwaf` image gets the same -packages each time it is built. Unpacked, nixpkgs takes about 500 MiB of disk, -more on some filesystems such as ZFS. Each package an app installs from it adds -its own size, with everything it depends on. A newer commit of the branch, with -its security fixes, comes with a newer `smallwebwaf` image, as do Ubuntu's own -fixes. An app takes them by changing the digest in its `FROM` line. When nixpkgs -makes its next release, every six months, the image moves to that release's -branch. +- Ubuntu's packages come from Ubuntu's snapshot service, the archive as it was + at the moment the Dockerfile names (`--snapshot 20261001T000000Z`), never + earlier than the pinned Ubuntu image's date. The Dockerfile names the SHA-256 + hash of each of the snapshot's `InRelease` files, which cover every package + list and so every package, and the build checks them before installing. The + hashes are the amd64 archive's, so the image is built for amd64 only. The + snapshot is fetched over HTTPS with the CA certificates of the Go image that + builds `smallwebwaf`, itself pinned by digest, since the Ubuntu image has + none. +- nixpkgs comes from a release file on `releases.nixos.org` (`nixexprs.tar.xz` + of `nixos-26.05.11045.774debe7a0d1`), named with its SHA-256 hash, not from a + GitHub archive, whose bytes can change. It is set up for root as `nixpkgs`. + Root's Nix profile, `/nix/var/nix/profiles/default/bin`, is last on the + `PATH`, so that no package hides the image's own commands; busybox's `sv`, for + one, looks for services elsewhere. Unpacked, nixpkgs takes about 500 MiB. +- Security fixes from Ubuntu and nixpkgs come with a newer `smallwebwaf` image, + which an app takes by changing the digest in its `FROM` line. The image moves + to each new nixpkgs release, every six months. ### The two processes -- `runsvinit` is the container's first process and runs as root, as do runit's - `runsvdir` and the `runsv` that watches each service. Each `run` script starts - as root and hands over to its program's user with `chpst`: `smallwebwaf` runs - as `smallwebwaf`, the app as its own user. -- Both start with the container's environment variables, which is why every - setting of `smallwebwaf` carries the `SWWAF_` prefix. Both write to the - container's output, which `docker logs` shows: the JSON lines of `smallwebwaf` - and whatever the app writes. -- When `smallwebwaf` or the app exits, for whatever reason, runit runs its `run` - script again, which waits one second before starting it. The other process - keeps running, and the container does not stop. While `smallwebwaf` is down, - nothing answers on port 8080 and traefik answers `502`. While the app is down, - `smallwebwaf` answers `502`. -- A `smallwebwaf` that stops at start, for example on invalid configuration, is - started again every second and stops again, logging its message each time, - until the cause is fixed. The health check fails meanwhile. +- `runsvinit`, `runsvdir` and each service's `runsv` run as root. Each `run` + script starts as root and hands over to its program's user with `chpst`. Both + services start with the container's environment variables and write to the + container's output, which `docker logs` shows. +- When `smallwebwaf` or the app exits, runit starts it again a second later. The + other process keeps running, and the container does not stop. While + `smallwebwaf` is down, traefik answers `502`. While the app is down, + `smallwebwaf` answers `502`, which is not a client offence. +- Only four things stop `smallwebwaf`, all at start: + - invalid configuration; + - a lookup database that is missing or unreadable; + - a rule file or state file that does not parse; + - an unwritable `SWWAF_STATE_DIR`. +- A `smallwebwaf` that stops at start is started again every second, logging its + message each time, until the cause is fixed. The health check fails meanwhile. + Once running, a broken helper or a broken edit never takes the protected + service down. - `docker stop` sends `SIGTERM` to `runsvinit`, which has runit stop both services at once, each with `SIGTERM`, and exits once they have stopped. `smallwebwaf` stops taking requests, gives those in progress five seconds to - finish, and writes its state files as it stops. The requests in progress can - finish only while the app still answers. An app that exits at once on - `SIGTERM` ends them: the client gets `502`, or a response cut short if the app - had already begun to answer. An app that should let them finish keeps taking - and answering requests for the whole five seconds after `SIGTERM`, not only - the ones it already has. `smallwebwaf` can still pass it requests it took - before then. + finish, and writes its state files. Those requests can finish only while the + app still answers. An app that exits at once on `SIGTERM` ends them: the + client gets `502`, or a response cut short. To let them finish, the app must + keep taking and answering requests for five seconds after `SIGTERM`. - The container's root filesystem stays writable: runit writes each service's status into its directory under `/etc/service`. @@ -1370,33 +1267,27 @@ image's `HEALTHCHECK` with one that checks both. ### Ports -`smallwebwaf` listens on port 8080 on every address and on no other port. Its -health check, metrics and ban management are all on that listener, under -`/_smallwebwaf/`. `SWWAF_LISTEN_ADDR` may set another port. The image's health -check takes its port from that setting, traefik's port label -(`traefik.http.services..loadbalancer.server.port`) must name the same -port, and the app must leave that port free. The address part of -`SWWAF_LISTEN_ADDR` stays empty (for example `:9000`, never `127.0.0.1:9000`), -so `smallwebwaf` keeps listening on every address: traefik reaches it on the -container's address, and the health check on `127.0.0.1`. The app listens on -`127.0.0.1:8081` only, so that nothing outside the container reaches it except -through `smallwebwaf`. An app that listens on every address can be reached +`smallwebwaf` listens on no port but the one in `SWWAF_LISTEN_ADDR`, and its own +endpoints are on that listener. Another port may be set: the health check +follows the setting, traefik's port label +(`traefik.http.services..loadbalancer.server.port`) must name it, and the +app must leave it free. The address part of `SWWAF_LISTEN_ADDR` stays empty +(`:9000`, never `127.0.0.1:9000`), so that traefik reaches `smallwebwaf` on the +container's address and the health check on `127.0.0.1`. The app listens on +`127.0.0.1:8081` only. An app that listens on every address can be reached around `smallwebwaf` by anything that reaches the container. ### State -`smallwebwaf` keeps its state files in `/var/lib/smallwebwaf` -(`SWWAF_STATE_DIR`), a directory of its own beside the app's data, which the app -keeps in directories of its own, such as `/var/lib/app`. Without a volume there, -the files live in the container: they survive a restart of the container and are -lost when a deploy replaces it. A volume mounted at `/var/lib/smallwebwaf`, -named or a host directory, keeps them across deploys. The `run` script of -`smallwebwaf` makes it and every file in it belong to the `smallwebwaf` user. So -a host directory mounted there needs no change of owner, and files an earlier -owner left in it can be read and replaced. It is a volume of its own, separate -from the app's. It holds a few tens of MiB at most with the defaults, and needs -no backup beyond whatever the host already does. The image declares no volume, -since every app image built on it would inherit it. +The app keeps its data in directories of its own, such as `/var/lib/app`, and +`smallwebwaf` keeps its state files beside them in `SWWAF_STATE_DIR`. Without a +volume there, the files survive a restart of the container and are lost when a +deploy replaces it. A volume mounted at `/var/lib/smallwebwaf`, named or a host +directory, keeps them across deploys. The `run` script makes it and every file +in it belong to the `smallwebwaf` user, so a host directory mounted there needs +no change of owner. The volume is separate from the app's, holds a few tens of +MiB at most with the defaults, and needs no backup beyond the host's own. The +image declares no volume, since every app image built on it would inherit it. ### Tokens @@ -1421,250 +1312,69 @@ example. ### upaas An app deployed by upaas takes this shape with no change to upaas. upaas builds -the app's Dockerfile, which now starts `FROM` the `smallwebwaf` image, and runs -the one container as it runs any app, with the app's traefik labels, environment -variables and volumes. The labels route to port 8080 -(`traefik.http.services..loadbalancer.server.port=8080`). Any `SWWAF_` -settings go with the app's environment variables. The volume for -`/var/lib/smallwebwaf` goes beside the app's own, as does the directory that -holds any token file. - -The endpoints of `smallwebwaf` are reached through traefik like any other -request, for example `https://app.example.invalid/_smallwebwaf/metrics` for a -scraper that sends `SWWAF_METRICS_TOKEN`. +the app's Dockerfile and runs the one container as it runs any app. The traefik +labels route to port 8080, and any `SWWAF_` settings go with the app's +environment variables. The volume for `/var/lib/smallwebwaf` and the directory +of any token file go beside the app's own volumes. ### Rollout per service -Build the app's image on the `smallwebwaf` image, with no setting, and deploy it -as before. It protects the app from the first request, and nothing needs tuning -first. Afterwards the request log and the ban notes say why each client was -refused. A real visitor refused by mistake is let through with an exclusion -(`SWWAF_WAF_DISABLED_RULES`, `SWWAF_WAF_EXEMPT_PATHS`, +Build the app's image on the `smallwebwaf` image, with no setting, and deploy +it. Nothing needs tuning first. Afterwards the request log and the ban notes say +why each client was refused. A real visitor refused by mistake is let through +with an exclusion (`SWWAF_WAF_DISABLED_RULES`, `SWWAF_WAF_EXEMPT_PATHS`, `SWWAF_RATE_LIMIT_EXEMPT_PATHS`) or an exemption (`SWWAF_RATE_LIMIT_EXEMPT_NETS`, `SWWAF_ALLOW_NETS`), and its ban is lifted in -`bans.json`. Additions to consider at any time: an alert destination, country -lists or biased thresholds, reputation sources, the metrics and admin tokens, a -remote log endpoint, and an app-specific rule file. +`bans.json`. Additions to consider at any time: -### Notes specific to gitea +- an alert destination; +- country lists or biased thresholds; +- reputation sources; +- the metrics and admin tokens; +- a remote log server; +- an app-specific rule file. -gitea is built on the image like any other app. - -- SSH access to gitea does not pass through `smallwebwaf` and is not protected - by it. -- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all - but the largest repositories and slowest links. A push is a request, and so is - every other upload: a git push, an LFS object, a container image layer, a - package file or a release attachment. At the defaults, an upload larger than - 100 MiB or taking more than 60 seconds is cut off. The client is answered - `413` for a body that is too large, before anything reaches gitea when the - request announces its size. It is answered `408` for a body it sends too - slowly, or `504` if gitea is too slow to take it. The upload fails, and no one - is banned for it. A gitea that takes large uploads needs - `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and - `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does not - read an upload's body, which streams through without being held in memory. -- At the defaults, the Core Rule Set lets gitea's ordinary use through, apart - from the refusals in the next note: - - browsing and views of files in a repository, with their history, blame and - the file tree; - - diffs, including their hidden lines and large files, and pull request - review; - - git's clone, fetch and push over HTTP; - - signing in, including the return to the page a visitor came from, and - sign-in with Git Credential Manager, git-credential-oauth or tea; - - the API's calls for a file and its commits; - - pushing and pulling container images and packages; - - Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`; - - posting issues, pull requests, comments, wiki pages and files saved in the - web editor, code included, since no body is read; - - the page gitea shows after a change, with its message naming what was - done. -- The Core Rule Set can still refuse the requests below. Each refusal answers - only that request, with `403`, and bans no one by itself. It counts toward the - error burst, which a person does not reach this way. The request log names the - rule in `waf_rule_ids`, which `SWWAF_WAF_DISABLED_RULES` can switch off. - - A query string that reads to it as an attack, most often a search. The - lists refuse such a name in any other query parameter too, such as a - release attachment uploaded through the API as `docker-compose.yml`. - Examples: - - a name on its lists of system files and commands, such as - `package.json`, `.gitignore` or `docker-compose.yml`; - - text that starts with a command name, or whose first word has one - right after a `/`, such as `python3`, `ssh key` or - `feat/docker-support`; - - a shell command with its options or a system path (`ls -la`, `sed -i`, - `/bin/sh`), or a command in backticks; - - script code (`fetch(`, `${VAR}`, `process.env`), HTML (` Date: Thu, 22 Oct 2026 20:43:32 +0000 Subject: [PATCH 3/8] SPEC: each requirement once, lists moved to docs/, repeats and sections dropped Every requirement the second review found in two to five places is now stated once, with links from the other places. "Risks the design has to handle", "Rollout per service" and most of "Architecture" are gone, their two points of their own moved to "Attack detection" and "Anomaly thresholds". The request log's fields, the metrics and the image's contents move to docs/request-log.md, docs/metrics.md and docs/image.md, each linked from the spec. The off rule names SWWAF_LOOKUP_SOURCE and SWWAF_WAF_MODE, the reputation bound is given, and the six Core Rule Set changes are a numbered list. Model: fable-5-1 --- SPEC.md | 835 +++++++++++++++++--------------------------- docs/gitea.md | 12 + docs/image.md | 56 +++ docs/metrics.md | 83 +++++ docs/request-log.md | 128 +++++++ 5 files changed, 603 insertions(+), 511 deletions(-) create mode 100644 docs/image.md create mode 100644 docs/metrics.md create mode 100644 docs/request-log.md diff --git a/SPEC.md b/SPEC.md index b1b7daa..f5c683e 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4,8 +4,9 @@ runs inside the app's container, between traefik and the app. This document says what it must do and why, each requirement once. Everything in it is built, except the last section, "Planned, not built". `README.md` says how to run it. -`EVALUATION.md` says why no existing tool was adopted. `docs/gitea.md` records -how it behaves in front of gitea. +`EVALUATION.md` says why no existing tool was adopted. `docs/` holds the lists +this document links to, and `docs/gitea.md` records how `smallwebwaf` behaves in +front of gitea. ## Purpose @@ -19,8 +20,8 @@ how it behaves in front of gitea. - It bans abusers: briefly at first, for seven days on a clear sign of attack, and permanently when they keep at it. - It looks up each client's AS number and country. It can refuse whole countries - and netblocks, and lower its limits for listed AS numbers and countries. It - consults reputation sources. It sends alerts. + and netblocks, lower its limits for listed AS numbers and countries, consult + reputation sources, and send alerts. - Everything is set by environment variables whose names start with `SWWAF_`, since `smallwebwaf` shares its container, and so its environment, with the app. The only other inputs are files: the rule files, the state files, a @@ -38,25 +39,20 @@ how it behaves in front of gitea. ## Architecture -- One statically linked Go binary. It ships in an image built on Ubuntu 26.04 - LTS with nixpkgs installed. That image is the base of the app's own image, so - `smallwebwaf` and the app run in one container (see "Deployment"). runit - starts `smallwebwaf` as a non-root user of its own. `smallwebwaf` writes only - to its state directory. -- One listener, on port 8080, which traefik routes to. It forwards requests to - the app on `127.0.0.1:8081`. It answers its own endpoints under - `/_smallwebwaf/` (see "Admin endpoints"). Those never reach the app. +- One statically linked Go binary. It writes only to its state directory. It + ships in the image that is the base of the app's own image (see "Deployment"). - No database of any kind. Nothing is read from disk while serving a request. -- The proxy is the standard library's `httputil.ReverseProxy`. The other parts - of the process are each specified below. - Libraries, checked against the owner's Go dependency defaults: - - the standard library for the proxy, HTTP clients, DNS, logging - (`log/slog`) and the state files (`encoding/json`, `os.Rename`); + - the standard library for the proxy (`httputil.ReverseProxy`), HTTP + clients, DNS, logging (`log/slog`) and the state files (`encoding/json`, + `os.Rename`); - `github.com/corazawaf/coraza/v3` and `github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries - the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to the Core Rule - Set, and the default `SWWAF_WAF_DISABLED_RULES`, are built for that - version. They are checked again before the version changes; + the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to it, and the + default `SWWAF_WAF_DISABLED_RULES`, are built for that version. They are + checked again before the version changes, since rule ids, and what each + rule matches, change between versions; + - `github.com/hashicorp/golang-lru/v2` for the bounded tables; - `github.com/oschwald/maxminddb-golang/v2` to read the lookup database; - `github.com/fsnotify/fsnotify` to notice edited files; - `github.com/prometheus/client_golang` for the metrics; @@ -72,32 +68,36 @@ names the section that holds its rules. 1. `GET /_smallwebwaf/healthz` is answered at once, before any check. 2. The client is identified. - If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, `X-Forwarded-For` is - walked from the right. The first address outside those netblocks is the - client. If every address is inside them, the leftmost is the client, so - that a visitor on a private network is known by its own address. With no - header, the peer is the client. + walked from the right, and the first address outside those netblocks is + the client. If every address is inside them, the leftmost is, so that a + visitor on a private network is known by its own address. With no header, + the peer is the client. - If the peer is outside `SWWAF_TRUSTED_PROXIES`, it is the client, and the header is ignored. - An IPv6 client is its group: the netblock of `SWWAF_IPV6_GROUP_PREFIX` around its address, a /64 by default, since one abuser usually controls a whole /64. A client is therefore one IPv4 address or one IPv6 group. -3. The static lists. A client in `SWWAF_ALLOW_NETS` skips every check below. It - is still counted for the anomaly thresholds. A client in `SWWAF_DENY_NETS` is - refused. -4. The ban ledger. A client under a ban is refused (see "Bans"). +3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12 and the + byte limits. It is still counted for the anomaly thresholds, and still needs + a token at step 13. A client in `SWWAF_DENY_NETS` is refused; a client in + both lists is let through. +4. Bans. A client under a ban is refused (see "Bans"). 5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number and country"). The request goes on at once. It waits for the first GeoJS answer, up to `SWWAF_LOOKUP_TIMEOUT`, only when one of these needs it: - a country list; - a biased threshold; - `SWWAF_ADD_LOOKUP_HEADERS`. -6. The country lists (see "Country lists"). A refused request is not an offence. + + A request that ends before the answer comes is logged without it. + +6. The country lists (see "Country lists"). 7. The reputation sources (see "Reputation"). A client on a fetched blocklist gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is banned. A client listed by a DNS blocklist, or scored at least `SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`. With no cached verdict, a query is queued in the background and the request - goes on. A first request is never delayed by a reputation query. + goes on, so a first request is never delayed by a reputation query. 8. The client's limit percentage (see "Biased thresholds"). 9. Rate limiting (see "Request rate limits, per client"). A request over a limit times the percentage is refused and bans the client. @@ -105,9 +105,10 @@ names the section that holds its rules. bans the client. 11. The rule files (see "Rule files"). Matching stops at the first rule that refuses. -12. The Core Rule Set (see "Attack detection"). In `block` mode a match is - refused with `403`. In `detect` mode it is only logged and alerted. -13. A request under `/_smallwebwaf/` is answered here and goes no further. +12. The Core Rule Set (see "Attack detection"). A match is refused with `403` + while `SWWAF_WAF_MODE` is `block`, and only logged and alerted while it is + `detect`. +13. A request under `/_smallwebwaf/` is answered here (see "Admin endpoints"). 14. The request is forwarded to `SWWAF_UPSTREAM_URL`, streaming, with `X-Forwarded-For` added, and `X-Client-ASN` and `X-Client-Country` when enabled. @@ -129,7 +130,7 @@ the token check still apply. - Each window (minute, hour, day) uses two adjacent fixed buckets per client. The earlier bucket is weighted by how much of it still overlaps the sliding - window. This costs a few integers per client per window. It avoids the burst + window. This costs a few integers per client per window and avoids the burst at bucket boundaries that a single fixed bucket allows. - Counters are read and updated in memory. They are written to `clients.json` and loaded again at start (see "Persistent state"), so a restart does not hand @@ -137,14 +138,13 @@ the token check still apply. load. - Memory is bounded. When the table of clients holds `SWWAF_MAX_TRACKED_CLIENTS`, the least recently seen client is dropped first, - with its history. The bans are bounded by `SWWAF_MAX_BANS` (see "Bans"). The - GeoJS answers, the reputation verdicts and the anomaly counters have fixed - bounds, given where each is described. + with its history. The bans, the GeoJS answers, the reputation verdicts and the + anomaly counters have bounds of their own, given where each is described. ## Bans -A ban refuses every request from a netblock with `SWWAF_BAN_RESPONSE` until it -ends. The netblock is the client: its IPv4 address, widened to +Until a ban ends, every request from its netblock is refused with +`SWWAF_BAN_RESPONSE`. The netblock is the client: its IPv4 address, widened to `SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one address or one /64, so a permanent ban does not reach neighbours who did nothing. Requests refused under a ban are not counted against limits. They are @@ -193,20 +193,24 @@ the cause `crowdsec`. Only a client that sends a request is banned, so a long list does not fill `bans.json`. Such a ban is never made permanent. It does not make the netblock's next ban for a broken limit longer. -Some refusals make no ban at all. `SWWAF_DENY_NETS`, a blocklist or reputation -hit whose action is `deny`, and the country lists refuse each request they cover -and record nothing in `bans.json`. +These refuse each request they cover, record nothing in `bans.json`, and count +no offence: -Every ban carries notes with what an admin needs to decide whether to lift it -(see "Persistent state"). An admin can add or lift a ban at any time by editing -`bans.json`, or through the ban endpoints when `SWWAF_ADMIN_TOKEN` is set. A ban -lifted before it ends is kept, marked as lifted, and does not count toward a -longer ban. Deleting its entry from `bans.json` forgets it entirely. +- `SWWAF_DENY_NETS`; +- a blocklist or reputation hit whose action is `deny`; +- the country lists. -A permanent ban does not run out. It ends when an admin lifts it, or when the -table of bans is full. When `smallwebwaf` holds `SWWAF_MAX_BANS` bans of its -own, it drops the one that has gone longest without a request, whether past, -active or permanent. A scanner whose ban was dropped that way is refused and +Every ban carries notes with what an admin needs to decide whether to lift it. +An admin adds or lifts a ban by editing `bans.json` (see "Persistent state"), or +through the ban endpoints when `SWWAF_ADMIN_TOKEN` is set. A ban lifted before +it ends is kept, marked as lifted, and does not count toward a longer ban. +Deleting its entry from `bans.json` forgets it entirely. + +A permanent ban does not run out. It ends when an admin lifts it, or when +`smallwebwaf` drops it from a full table, as below. When `smallwebwaf` holds +`SWWAF_MAX_BANS` bans of its own, it drops the earliest ban of one netblock: the +netblock that has gone longest without a request. A past, active or permanent +ban is dropped alike. A scanner whose ban was dropped that way is refused and banned again by its next probe. A ban an admin made, whose cause is `admin`, is never dropped and does not count toward `SWWAF_MAX_BANS`, so such bans can never fill the table. An admin who wants to keep a ban `smallwebwaf` made sets its @@ -222,6 +226,7 @@ Conventions: codes in either case. A code that is not a country, such as `nk`, stops the start with a message naming it. - These settings take `off`, which switches them off: + - `SWWAF_LOOKUP_SOURCE` and `SWWAF_WAF_MODE`; - the five timeouts under "Size and time limits"; - `SWWAF_REQUEST_MAX_BYTES` and `SWWAF_RESPONSE_MAX_BYTES`; - the three rate limits and the three byte limits; @@ -261,7 +266,10 @@ The settings, by group: the private address ranges): the netblocks whose `X-Forwarded-For` is believed, normally where traefik reaches `smallwebwaf` from. A client that connects directly cannot claim another address. A list given replaces the - default; set but empty, it trusts nothing. + default; set but empty, it trusts nothing. Where other containers reach + `smallwebwaf` directly, or visitors on a private network reach traefik, they + can name another address in the header; this setting, narrowed to traefik's + own network, closes that gap. - `SWWAF_IPV6_GROUP_PREFIX` (default `64`), from 32 to 128. - `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the clients held in memory and in `clients.json`. @@ -271,26 +279,25 @@ The settings, by group: ### Tokens - `SWWAF_ADMIN_TOKEN`: the bearer token for the ban endpoints and - `/_smallwebwaf/clients/`. Unset by default, which switches them off; bans - are then managed by editing `bans.json`. + `/_smallwebwaf/clients/`. Unset by default, which switches them off. - `SWWAF_METRICS_TOKEN`: the bearer token a scraper sends for `/_smallwebwaf/metrics`. Unset by default, which switches the metrics off, since they would otherwise be open to anyone on the internet. It is separate from `SWWAF_ADMIN_TOKEN`, so a scraper that holds it cannot manage bans. - A token is a long random value, since the endpoints can be reached from the internet. One shorter than 32 characters stops the start. The app starts with - the same environment variables as `smallwebwaf`, so it can read a token given - as one. The file form keeps a token out of the app's reach (see "Tokens" under - "Deployment"). + the same environment variables as `smallwebwaf`, so it can read a token set as + an environment variable. The file form keeps a token out of the app's reach + (see "Tokens" under "Deployment"). ### State files - `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the JSON state files, in a directory of their own beside the app's data, belonging to the `smallwebwaf` user. -- `SWWAF_STATE_WRITE_DELAY` (default `10s`): `bans.json` is written this long - after a ban is made, lifted or made permanent, so a burst of changes becomes - one write. +- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be + written after a ban is made, lifted or made permanent, so a burst of changes + becomes one write. - `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is written. @@ -316,7 +323,8 @@ The settings, by group: ### Metrics - `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and countries get - series of their own. The rest are summed as `other`. + series of their own. The rest are summed as `other`, so the label set stays + bounded. ### Static lists @@ -325,6 +333,10 @@ The settings, by group: error burst, attack detection and bans still apply. - `SWWAF_DENY_NETS`: always refused. +Many clients share one address (mobile carriers, offices, Tor), and so share its +limits and its bans. The defaults sit well above one person's use, and the first +two lists take known shared addresses out. + ### Request rate limits, per client Breaking one bans the client, so the defaults sit several times above what one @@ -352,9 +364,10 @@ largest response allowed, and no single download breaks one. These apply per request, in both directions. The client-facing timeouts apply between the client and `smallwebwaf`, the app-facing ones between `smallwebwaf` and the app, since a slow client and a slow app are separate problems. There is -one size limit for request bodies and one for response bodies, since -`smallwebwaf` passes bodies through unchanged. Bodies stream straight through, -so a request body reaches the app while the client is still sending it. +one size limit for request bodies and one for response bodies. Bodies pass +through unchanged, so a limit on each side would bound the same bytes, and the +lower one would always decide. Bodies stream straight through, so a request body +reaches the app while the client is still sending it. - `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to send its request line and headers, and then, from the end of the headers, its @@ -380,22 +393,21 @@ What the client gets when a limit is passed: - Go's HTTP server reads a request's line and headers before `smallwebwaf` sees the request. It answers `431` to headers over - `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`. A client too slow to send them gets - either no answer or an error answer, mostly `400`. Either way the server - closes the connection, nothing reaches the app, and the request log gets no - line. + `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, and a client too slow to send them + gets an error answer, mostly `400`, or none. Either way the server closes the + connection, nothing reaches the app, and the request log gets no line. - A request that announces a body larger than its limit is refused before anything reaches the app. -- When a limit is passed after the headers have been read, but before the - response has started, `smallwebwaf` answers itself: +- A limit passed after the headers have been read, but before the response has + started, is answered by `smallwebwaf`: - `413` for a request body that is too large; - `408` for a client too slow to send its body; - `502` for a response that is too large; - `504` for an app that is too slow. - A request body streams through, so each side can hold up the other. A request timeout that runs out while the body is still on its way answers `408` if - `smallwebwaf` was waiting for the client at that moment. It answers `504` if - it was waiting for the app to take what it had. + `smallwebwaf` was waiting for the client at that moment, and `504` if it was + waiting for the app. - Once the response has started it can only be cut off, and the connection is closed. - A WebSocket connection leaves these limits behind once it is upgraded. It @@ -404,25 +416,26 @@ What the client gets when a limit is passed: ### Lookup of AS number and country On by default through GeoJS, which needs no account or file, so that country -lookup works with no setup. The file is the alternative for a service that keeps -its visitors' addresses on its own host. +lookup works with no setup. A local database file is the alternative for a +service that keeps its visitors' addresses on its own host. -- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. One source - at a time: `file` without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` - with any other source, stops the start with a message naming both. So does a - setting that needs lookups while the source is `off`: the country lists, the - biased thresholds, `SWWAF_ADD_LOOKUP_HEADERS`, and the anomaly thresholds per - AS number. -- `SWWAF_LOOKUP_DB_PATH`: the file for `file`: the IPinfo Lite database in its - `.mmdb` form (`ipinfo_lite.mmdb`), whose `asn`, `as_name` and `country_code` +- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. Only one + source is used. The start stops, naming both settings, when `file` is set + without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other + source. The start also stops when the source is `off` and one of these needs + lookups: + - a country list; + - a biased threshold; + - `SWWAF_ADD_LOOKUP_HEADERS`; + - an anomaly threshold per AS number. +- `SWWAF_LOOKUP_DB_PATH`: the database the `file` source reads: the IPinfo Lite + database (`ipinfo_lite.mmdb`), whose `asn`, `as_name` and `country_code` fields are read. The operator downloads it with a free IPinfo account, mounts - it read-only, and refreshes it; IPinfo updates it daily. `smallwebwaf` never - fetches it and holds no account or token for it. A file that is missing or - unreadable at start stops the start. `smallwebwaf` notices when the file is - replaced and reads it again. A replacement it cannot read is ignored, logged - and sent as one `file_error` alert, and the file it has stays in use. The - directory that holds the file is mounted, not the file itself (see "Rule - files" for why). + the directory that holds it read-only (see "Rule files" for why), and + refreshes it. `smallwebwaf` never fetches it. A file that is missing or + unreadable at start stops the start. A replaced file is read again; a + replacement that cannot be read is logged, sent as one `file_error` alert, and + ignored, the file in use staying in use. - `geojs` asks the free GeoJS web service about up to 200 addresses in one request (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a time, and reads each answer's `country_code`, `asn` and `organization_name`. @@ -434,18 +447,21 @@ its visitors' addresses on its own host. - A client the lookup cannot place has an unknown country and AS number. The exclusive country list refuses it, and the biased thresholds give it `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it differently. -- Each GeoJS answer is kept for 7 days, in memory and in `lookups.json`, apart - from the table of clients. Up to 100,000 answers are kept, about 15 MiB; past - that, the answer used longest ago goes first. -- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its - client's first answer when a setting needs it, and how long a request to GeoJS - may take. A client whose answer has not come in time counts as unknown until - it comes, and its later requests do not wait. +- Each GeoJS answer is kept for 7 days, in memory and in `lookups.json`, + separately from the table of clients, so that it outlasts a client dropped + from that table. Up to 100,000 answers are kept, about 15 MiB; past that, the + answer used longest ago goes first. +- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a visitor's request waits for + its client's first answer when a setting needs it. It is also how long a + request to GeoJS may take. A client whose answer has not come in time counts + as unknown until it comes, and its later requests do not wait. - GeoJS publishes no rate limit, but its terms forbid "an excessive amount of - API requests", judged by GeoJS alone, and let it block a caller. While GeoJS - is slow, down or refusing `smallwebwaf`, new clients count as unknown, - `smallwebwaf` keeps asking with backoff, and one `source_failure` alert is - sent per cooldown. A service that cannot accept this uses the file. + API requests", judged by GeoJS alone, and let it block a caller. Kept answers + and batched requests keep the number of requests low. While GeoJS is slow, + down or refusing `smallwebwaf`, new clients count as unknown, `smallwebwaf` + keeps asking with backoff, and one `source_failure` alert is sent per + cooldown. A service that cannot accept this, or that every new visitor's + address goes to a third party, uses the file. - `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): pass `X-Client-ASN` and `X-Client-Country` to the app. @@ -458,20 +474,20 @@ where it comes from. Clients in `SWWAF_ALLOW_NETS` are not checked. a listed country is refused. - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`: for example `us,de`. Only clients placed in a listed country get through. Every request from any other country - is refused. So is every request from a client the lookup cannot place, such as - a visitor on the local network, another container or internal monitoring. - Those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let - unplaced clients through would let every new client in whenever GeoJS stops - answering. That is the nearest a failing helper comes to taking the service - down: GeoJS failing while this list is set refuses every new client. + is refused. So is every request from a client the lookup cannot place (see + "Lookup of AS number and country"), such as internal monitoring; those that + should reach the app go in `SWWAF_ALLOW_NETS`. A list that let unplaced + clients through would let every new client in whenever GeoJS stops answering. + The cost: while GeoJS fails, this list refuses every new client. No other + failure of GeoJS, a reputation source or an alert destination comes this close + to taking the service down. - Both may be set. `SWWAF_DENIED_COUNTRIES` then adds nothing, since the exclusive list already refuses every other country. A code on both lists stops the start with a message naming it. - A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the client's address has been looked up, before its body is read. It skips the reputation checks, the limits, the rule files and the Core Rule Set. It is - logged with the action `country_denied`, and counted in the metrics. It is a - refusal, not a ban. + logged with the action `country_denied`, and counted in the metrics. ### Biased thresholds @@ -481,14 +497,14 @@ abuse comes from. - `SWWAF_ASN_LIMIT_PERCENT`: for example `AS14061:50,AS16276:50,AS45102:25`. Clients in a listed AS number get that percentage of every request and byte - limit, so the rules in "Bans" ban them after fewer requests than others. `0` - is a zero allowance: the client's first request breaks a limit and bans it. + limit. `0` is a zero allowance: the client's first request breaks a limit and + bans it. - `SWWAF_COUNTRY_LIMIT_PERCENT`: the same form with country codes, for example `CN:25,RU:50`. - `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT`: optional overrides applied to the byte limits only, when the byte percentage should differ from the request percentage. -- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): for clients the lookup cannot +- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): for a client the lookup cannot place. - `SWWAF_ASN_LIMIT_PERCENT_URL`: optional URL of a text file of `AS:percent` lines, so one abuse-source list can be shared by every `smallwebwaf` in the @@ -508,12 +524,11 @@ By default the Core Rule Set reads the method, the URL with its query string, and the headers of every request. That is where automated attacks show: injection in query strings, path traversal, attacks carried in headers, scanners' user agents. It reads no request body. On a code forge, the example -deployment, bodies carry what people write and publish: issue and comment text, -wiki pages, files saved in the web editor, package descriptions. When these hold -shell commands or code, the Core Rule Set takes them for attacks. The default -gives up refusing an attack carried in a body. The rule files, the limits and -the bans still apply to the client that sends it, and `SWWAF_WAF_BODY_LIMIT` -switches body inspection on. +deployment, bodies carry what people write and publish, such as issue text and +files saved in the web editor. The Core Rule Set takes the shell commands and +code in them for attacks. The default gives up refusing an attack carried in a +body; the rule files, the limits and the bans still apply to the client that +sends it, and `SWWAF_WAF_BODY_LIMIT` switches body inspection on. - `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of rule files (see "Rule files"). @@ -522,6 +537,9 @@ switches body inspection on. `block`. - `SWWAF_WAF_PARANOIA_LEVEL` (default `1`) and `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the Core Rule Set's own two tuning values, at its own defaults. + Each rule that matches adds its score to the request's. A request whose score + reaches the threshold is a match. With the threshold `off`, nothing is a + match, and scores are still logged. - `SWWAF_WAF_DISABLED_RULES` (default `920340,920420,920440,920640,930130,930140`): rule ids to switch off when an app trips a false positive. The default switches off six rules: @@ -536,7 +554,7 @@ switches body inspection on. package uploads, and views of ordinary files in a repository. At the site root, where no app serves such files, the default rule file bans the common probes these rules caught (see "Rule files"). A list given replaces the - default, so include them in it. + default, so name these six ids in it to keep them off. - `SWWAF_WAF_EXEMPT_PATHS`: path prefixes not inspected. - `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size, @@ -545,15 +563,8 @@ switches body inspection on. on without holding it in memory. It reads JSON and XML bodies no larger than the size, since those cannot be read in part. Any other body is not read: the Core Rule Set would read it as form data, where binary content such as a git - push trips rules written for text. Responses are never inspected. Body - inspection suits apps whose forms carry no code. In front of gitea it refuses - the requests below, until their rule ids, named in the request log, are added - to `SWWAF_WAF_DISABLED_RULES`: - - issue and comment text, wiki pages and files saved in the web editor that - hold shell commands or code (932125, 932235, 932250 and others); - - package descriptions that show code; - - PyPI uploads (922130); - - attachments named like `debug.log` or `config.yml` (932180). + push trips rules written for text. Body inspection suits apps whose forms + carry no code. `docs/gitea.md` records what it refuses in front of gitea. - `SWWAF_TRAP_PATHS`: paths the app never serves and only scanners ask for, for example `/wp-login.php,/xmlrpc.php` in front of gitea. A request for one is a clear sign of attack. This is the short form of a `path` rule with the `ban` @@ -564,30 +575,32 @@ switches body inspection on. trying one attack or token after another is refused many times a minute; a person rarely more than a few times. Answers the app gives are not counted, since they do not tell a scanner from an ordinary client. In front of gitea, - registry and git clients are answered `404` and `401` by design, hundreds of - times a minute. + registry and git clients get `404` and `401` by design, hundreds of times a + minute. +- A Core Rule Set engine error on a request forwards the request, and is logged + and counted. `smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of every app, and no setting undoes them: -- `PUT`, `PATCH` and `DELETE` are allowed methods besides `GET`, `HEAD`, `POST` - and `OPTIONS`. APIs, container image pushes and package uploads use them. - Other methods stay refused. -- The request header `Expect` is allowed, since curl and git send it before some - large uploads. `Content-Encoding` is allowed on a request whose body the Core - Rule Set does not read, which by default is every request. git compresses a - fetch request over 1 KiB and says so in that header. On a body the Core Rule - Set does read, the header stays refused, since a compressed body cannot be - inspected. The other headers the Core Rule Set refuses, such as `Proxy`, stay - refused. -- The `redirect_uri` parameter is not checked for a URL naming an IP address or - `localhost` (931100, 934110). Git Credential Manager, git-credential-oauth and - tea, which gitea registers for OAuth sign-in, ask to be sent back to - `http://127.0.0.1` on the user's own machine. The server never fetches that - address. -- Ten query parameters are not checked against the Core Rule Set's lists of - system files (930120), shell paths (932160) and command names (932260): +1. `PUT`, `PATCH` and `DELETE` are allowed methods besides `GET`, `HEAD`, `POST` + and `OPTIONS`. APIs, container image pushes and package uploads use them. + Other methods stay refused. +2. The request header `Expect` is allowed, since curl and git send it before + some large uploads. `Content-Encoding` is allowed on a request whose body the + Core Rule Set does not read, which by default is every request. git + compresses a fetch request over 1 KiB and says so in that header. On a body + the Core Rule Set does read, the header stays refused, since a compressed + body cannot be inspected. The other headers the Core Rule Set refuses, such + as `Proxy`, stay refused. +3. The `redirect_uri` parameter is not checked for a URL naming an IP address or + `localhost` (931100, 934110). Git Credential Manager, git-credential-oauth + and tea, which gitea registers for OAuth sign-in, ask to be sent back to + `http://127.0.0.1` on the user's own machine. The server never fetches that + address. +4. Ten query parameters are not checked against the Core Rule Set's lists of + system files (930120), shell paths (932160) and command names (932260): - `path`, `files`, `skip-to` and `sub_path`; - `ref`, `sha`, `branch` and `workflow`; - `artifactName` and `redirect_to`. @@ -599,55 +612,53 @@ every app, and no setting undoes them: `|cat /etc/passwd` included. Path traversal, SQL and script injection, and PHP, Java and Node.js code are still refused there. Every other parameter keeps all three rules. Parameter names are matched without regard to case, - as Coraza matches them. + as Coraza matches them. An app that uses one of these parameters as a file + on the server, or passes it to a shell, must check the value itself. A rule + file can refuse it instead. -- The Core Rule Set reads the request without the `gitea_flash` and - `redirect_to` cookies. It does not check `Referer` for a Unix command given - without arguments (932340) or for Java starting a process (944110). gitea puts - a message naming what was just done into `gitea_flash`. It puts the page to - return to after signing in into `redirect_to`. The Core Rule Set takes file - names in them, such as `package.json`, for attacks. A browser sends such a - cookie with every request until gitea replaces it. gitea cannot replace it - while those requests are refused, so one refusal would keep the browser out of - the whole site. `Referer` names the page a request comes from, on everything - that page loads. The Core Rule Set takes some page addresses for attacks, such - as a search for `env`, or OpenJDK's `Runtime.java`, whose path holds both - `runtime` and `java.`. gitea shows the message with any script removed and - returns only to its own pages. Every other cookie is read in full, and - `Referer` keeps the other rules. -- Responses are not inspected. A raw file from a repository, such as a shell - script, looks to the response rules like source code leaking from the server. +5. Two cookies are left out, and `Referer` keeps fewer rules: + - The Core Rule Set reads the request without the `gitea_flash` and + `redirect_to` cookies. gitea puts a message naming what was just done into + `gitea_flash`. It puts the page to return to after signing in into + `redirect_to`. The Core Rule Set takes file names in them, such as + `package.json`, for attacks. A browser sends such a cookie with every + request until gitea replaces it. gitea cannot replace it while those + requests are refused, so one refusal would keep the browser out of the + whole site. gitea shows the message with any script removed and returns + only to its own pages. Every other cookie is read in full. An app that + reads a cookie of either name must check it itself. + - `Referer` is not checked for a Unix command given without arguments + (932340) or for Java starting a process (944110). `Referer` names the page + a request comes from, on everything that page loads and on every link + followed from it. The Core Rule Set takes some page addresses for attacks, + such as a search for `env`, or OpenJDK's `Runtime.java`, whose path holds + both `runtime` and `java.`. All of those requests would be refused. + `Referer` keeps the other rules. An app that passes `Referer` to a shell + must check it itself. +6. Responses are not inspected. A raw file from a repository, such as a shell + script, looks to the response rules like source code leaking from the server. -The fourth and fifth changes leave an app on its own in those places (see "Risks -the design has to handle"). +### Ban response and scope -### Ban lengths and response - -- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of - attack. -- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken limit. -- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): breaking a limit again within - this time after a ban for a broken limit ended makes the next ban three times - as long. -- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban that would be longer is - permanent instead. - `SWWAF_BAN_RESPONSE` (default `403`): `403`, `429`, or `close` to drop the connection without an answer. `403` makes a ban easy to recognise when debugging. Behind traefik, `close` does not leave the client unanswered: traefik answers `502`, as it does whenever its backend drops a connection. - `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): widen to for example `24` to ban the surrounding netblock. +- `SWWAF_ATTACK_BAN_DURATION`, `SWWAF_LIMIT_BAN_DURATION`, + `SWWAF_LIMIT_BAN_REPEAT_WINDOW` and `SWWAF_MAX_BAN_DURATION`: the ban lengths, + with their defaults, under "Bans". ### Reputation -No source is on by default, for the reason the country lists and the biased -thresholds are empty. A source acts on a list kept elsewhere; the defaults judge -a client only by what it does here. AbuseIPDB and CrowdSec also need an account -or an engine of the operator's own. Besides, the list best suited to be on by -default, Spamhaus DROP, must not be downloaded automatically more than once an -hour. Spamhaus may block an address that downloads it too often. Each -`smallwebwaf` fetches its own copy. On a host that runs several apps with it, -which share one address, downloads would often come less than an hour apart. +No source is on by default. A source acts on a list kept elsewhere; the defaults +judge a client only by what it does here. AbuseIPDB and CrowdSec also need an +account or an engine of the operator's own. The list best suited to be on by +default, Spamhaus DROP, must not be fetched more than once an hour, and Spamhaus +may block an address that does. Each `smallwebwaf` fetches its own copy, so +several apps on one host, sharing one address, would often fetch less than an +hour apart. - `SWWAF_BLOCKLIST_URLS`: text files of addresses and netblocks, one per line; anything after a `;` or `#` on a line is ignored. For example the Spamhaus @@ -663,9 +674,8 @@ which share one address, downloads would often come less than an hour apart. - `SWWAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:` or `log`. `deny` refuses every request from a listed address, which suits lists of networks that send nothing legitimate, such as DROP. `limit:` gives - listed clients that percentage of every limit, so they are banned after fewer - requests. It suits a list of addresses shared with ordinary visitors, such as - Tor exits. + listed clients that percentage of every limit. It suits a list of addresses + shared with ordinary visitors, such as Tor exits. - `SWWAF_DNSBL_ZONES`: for example `dnsbl.dronebl.org`. Zones meant for mail (lists of residential ranges) will block ordinary visitors and should not be used. Spamhaus zones need their keyed query service, given as a zone name @@ -677,15 +687,17 @@ which share one address, downloads would often come less than an hour apart. checks a day). Only clients that have already committed one offence are queried, so the budget is spent on suspects. - `SWWAF_CROWDSEC_LAPI_URL` and `SWWAF_CROWDSEC_LAPI_KEY`: optional. If a - CrowdSec engine exists on the host, its decision list is pulled on a schedule - and treated as a blocklist (see "Bans" for the ban it makes). This is how a - fleet-wide blocklist can arrive without `smallwebwaf` depending on CrowdSec. + CrowdSec engine exists on the host, its decision list is pulled on a schedule. + It is kept like a fetched blocklist, and a client on it is banned (see + "Bans"). This is how a fleet-wide blocklist can arrive without `smallwebwaf` + depending on CrowdSec. - `SWWAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:` or `log`, for DNS blocklist and AbuseIPDB hits. Such verdicts are less certain - than a blocklist, so by default a listed client gets a quarter of every limit - and is banned after a quarter of the requests. + than a blocklist, so by default a listed client gets a quarter of every limit. - `SWWAF_REPUTATION_CACHE_TTL` (default `24h`) and `SWWAF_REPUTATION_TIMEOUT` (default `2s`). +- Up to 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept. + Past that, the one fetched longest ago goes first. - A source that is down or over quota gives no verdict. The service continues, with one `source_failure` alert per cooldown. @@ -704,9 +716,10 @@ which share one address, downloads would often come less than an hour apart. - a rule file or state file edited while running that does not parse; - a replacement lookup database that cannot be read; - a state file that cannot be written. -- `SWWAF_ALERT_COOLDOWN` (default `15m`): the same event for the same netblock, - source, file or anomaly scope is not repeated within this time. A count of - suppressed repeats is included in the next one. +- `SWWAF_ALERT_COOLDOWN` (default `15m`): the same event is not repeated within + this time for the same netblock, source or file, or, for an anomaly, for the + same thing counted: one client, netblock, AS number or named netblock, or the + whole service. A count of suppressed repeats is included in the next one. - `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): beyond this, alerts are rolled into one summary per hour, so a wide attack cannot flood the channel. - A destination that is down is retried with backoff from a bounded queue, the @@ -717,7 +730,9 @@ which share one address, downloads would often come less than an hour apart. These only alert; nothing is blocked. None is set by default, and a threshold left unset sends no alert. An alert needs a destination only the operator can supply, and what counts as unusual depends on each service's normal traffic, -which the metrics show. +which the metrics show. They are what reveals a widely distributed scraper, +thousands of addresses at low rates each, which the per-client limits do not +see; a low `SWWAF_ASN_LIMIT_PERCENT` for its AS number is the response. - Per client: `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`, `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`, @@ -745,10 +760,10 @@ which the metrics show. ## Rule files `smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR` and checks each -request against them. The files are plain text meant to be edited by hand. A new -scanning pattern seen in the request log can be turned into a rule in one line. -`smallwebwaf` watches the directory: a file edited, added or removed there takes -effect while it runs. +request against them, unless the client is in `SWWAF_ALLOW_NETS`. The files are +plain text meant to be edited by hand, so that a new scanning pattern seen in +the request log becomes a rule in one line. `smallwebwaf` watches the directory: +a file edited, added or removed there takes effect while it runs. - One rule per line, four fields separated by spaces or tabs. The fourth field runs to the end of the line: @@ -773,12 +788,12 @@ effect while it runs. Rule Set's job, once `SWWAF_WAF_BODY_LIMIT` switches it on. - `action`: - `log`: note the match in the request log and do nothing else. - - `block`: refuse the request with `403`. The client is not banned for it, - but the refusal counts toward the error burst. + - `block`: refuse the request with `403`, without a ban (see "Bans"). - `ban`: the request is a clear sign of attack (see "Bans"). Use it only for requests no real visitor sends. Anchor a path at the site root (`^/`): a file of the same name deeper in a site can be ordinary content, such as a - file in a gitea repository. + file in a gitea repository. A visitor banned by mistake is let back in by + lifting the ban. - `regex`: Go regular expression syntax (RE2). It has no backreferences or lookaround. In exchange matching time is linear in the input, so no rule can be made to stall the proxy. `(?i)` at the front makes a rule ignore case. A @@ -792,17 +807,15 @@ effect while it runs. `SWWAF_RULES_DIR` that does not exist. While running, the same faults leave the rules as they were, and the log and one `file_error` alert name the file and line. Once the file is fixed, it is read again. An empty directory is not - an error. To run without rule files, mount an empty directory or set - `SWWAF_RULES_ENABLED=false`. + an error: the log says that no rules were loaded. To run without rule files, + mount an empty directory or set `SWWAF_RULES_ENABLED=false`. - The image ships a default file in `SWWAF_RULES_DIR`. The app's Dockerfile can copy files of its own into it, beside the default. Mounting a directory over it replaces the defaults; mounting single files into it adds to them. Many editors save a file by replacing it, and a container keeps seeing the old one when the file alone is mounted. So rules meant to be edited while - `smallwebwaf` runs belong in a mounted directory, with a copy of the default - file if the defaults are to stay. The lookup database is mounted the same way, - for the same reason. -- Clients in `SWWAF_ALLOW_NETS` are not checked. + `smallwebwaf` runs belong in a mounted directory. The lookup database is + mounted the same way, for the same reason. The default file the image ships: @@ -824,18 +837,12 @@ path-traversal uri block (\.\./){2,} empty-agent user_agent log ^$ ``` -It is kept short and limited to patterns that are wrong for every app, with its -path rules anchored at the site root. In front of gitea, `/.env` is a probe, -while `///src/branch/main/.env.example` is a file in a repository -that any visitor may open. Anything app-specific belongs in a file the app's -Dockerfile adds or the deployer mounts. A request for `/wp-login.php`, for -example, is a clear sign of attack in front of gitea and an ordinary login in -front of WordPress: - -``` -# 50-gitea.rules: WordPress probes, which gitea never serves -wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/) -``` +It is kept short and limited to patterns that are wrong for every app. In front +of gitea, `/.env` is a probe, while +`///src/branch/main/.env.example` is a file in a repository that +any visitor may open. Anything app-specific, such as a rule banning WordPress +probes in front of gitea, belongs in a file the app's Dockerfile adds or the +deployer mounts. ## Persistent state @@ -868,14 +875,16 @@ The files, each holding one kind of state: - `lookups.json`: GeoJS answers, one per client: the AS number, AS name and country, when GeoJS was asked and when the answer was last used. -- `reputation.json`: the last good copy of each list fetched from a URL; the - cached DNS blocklist verdicts and AbuseIPDB scores, each with the time it was - fetched; and the AbuseIPDB checks spent today, so that a restart does not - reset the daily budget. +- `reputation.json`: + - the last good copy of each list fetched from a URL; + - the cached DNS blocklist verdicts and AbuseIPDB scores, each with the time + it was fetched; + - the AbuseIPDB checks spent today, so that a restart does not reset the + daily budget. - `alerts.json`: - the anomaly counters; - - for each event and netblock, source, file or anomaly scope, when it was - last sent and how many repeats the cooldown has held back since; + - for each event and netblock, source, file or thing counted, when it was + last sent, and the count of repeats held back since; - the alerts sent this hour; - the alerts still waiting to be sent. @@ -895,17 +904,17 @@ what `smallwebwaf` already knows. Nothing is looked up to fill them: - how many earlier bans of each kind the netblock has had. Each file is rewritten whole, so each has a bound. `clients.json` takes about 1 -KiB per client, about 20 MiB at the default `SWWAF_MAX_TRACKED_CLIENTS`, which -written every 15 minutes is under 2 GiB of disk writes a day. `bans.json` takes +KiB per client, about 20 MiB at the default `SWWAF_MAX_TRACKED_CLIENTS`. Written +every 15 minutes, that is under 2 GiB of disk writes a day. `bans.json` takes about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at the default `SWWAF_MAX_BANS`. With a full file, a hundred new bans a day come to about 600 -MiB of writes. `lookups.json` takes about 150 bytes per answer, about 15 MiB -when full, under 1.5 GiB of writes a day. The other two files are usually a few -MiB or less. Writing a file of these sizes takes well under a second on an -ordinary disk, in the background, so rewriting whole files needs nothing -cleverer. +MiB of writes. `lookups.json` takes about 150 bytes per answer, under 1.5 GiB of +writes a day when full. The other two files are usually a few MiB or less. +Writing a file of these sizes takes well under a second, in the background, so +rewriting whole files needs nothing cleverer. -Format: +Format, so that a person can open `bans.json` in an editor, find an address, and +remove or add an entry: - indented JSON with a top-level `version` number; - entries sorted by client address; @@ -914,9 +923,6 @@ Format: - `clients.json` and `lookups.json` put each entry on one line instead. That halves their size and lets `grep` show everything about one client. -The aim is that a person can open `bans.json` in an editor, find an address, and -remove or add an entry. - Writing: - serialise from a snapshot taken under the lock, so requests are not held up @@ -924,8 +930,7 @@ Writing: - write to a temporary file in the same directory, sync it, rename it over the real name, sync the directory. A crash at any point leaves either the old complete file or the new complete file, never a partial one; -- `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban changes. Changes - that only update the counts in its notes wait for the write every +- changes that only update the counts in a ban's notes wait for the write every `SWWAF_STATE_COUNTER_INTERVAL`, as the other four files do. All five are written on orderly shutdown (`SIGTERM`); - before writing a file, `smallwebwaf` checks whether it changed on disk since @@ -950,9 +955,10 @@ Edits while running: file with what it last wrote. - An edit that parses is taken in at once: what the file says replaces what `smallwebwaf` held for that file. Removing a ban's entry from `bans.json` - lifts the ban; adding an entry bans. Changes `smallwebwaf` made after the - admin opened the file, such as a new ban, are lost when the admin saves over - them. Most editors warn when a file changed on disk while it was open. + lifts the ban, an admin's own included; adding an entry bans. Changes + `smallwebwaf` made after the admin opened the file, such as a new ban, are + lost when the admin saves over them. Most editors warn when a file changed on + disk while it was open. - An edit that does not parse does not stop the running `smallwebwaf`. It keeps the state it has and renames the edited file to `.bad` (for example `bans.json.bad`), so the edit is kept. It writes the file again from memory, @@ -970,103 +976,41 @@ What a hard kill can lose: - ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds); - everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes): - counts and history, the counts in ban notes, GeoJS answers, reputation - verdicts and the AbuseIPDB count, anomaly counters and alert cooldowns. + - counts and history, and the counts in ban notes; + - GeoJS answers; + - reputation verdicts and the AbuseIPDB count; + - anomaly counters and alert cooldowns. Losing the volume loses ban history, not service. ## Request log -One JSON object per line on stdout for every request, including refused ones, +Every request gets one JSON object on one line on stdout, refused ones included, apart from those Go's HTTP server ends before `smallwebwaf` sees them (see "Size -and time limits"). stdout is always on. When `SWWAF_LOG_REMOTE_URL` is set the -same lines are also sent there, so a deployment can stop depending on docker's -log handling while `docker logs` keeps working. +and time limits"). `docs/request-log.md` lists the fields. Bodies are never +logged. Query strings are logged as received; an app that carries secrets in +query strings needs that fixed in the app. The process's own messages share the +stream as JSON lines with `"type":"process"`; request lines carry +`"type":"request"`. -- Standard web log fields: `time` (RFC 3339 with milliseconds), `instance`, - `client_ip`, `method`, `scheme`, `host`, `path`, `query`, `protocol`, - `status`, `request_bytes`, `response_bytes`, `referer`, `user_agent`. -- Request detail: - - `request_id`, generated if traefik did not supply one, and passed to the - app; - - `peer_ip`, the TCP peer, normally traefik; - - `forwarded_for`, the header as received; - - `client_group`, the /64 or configured prefix used for counting; - - `asn`, `as_name`, `country`; - - `content_type`, `content_length`, and the headers named in - `SWWAF_LOG_REQUEST_HEADERS`; - - `has_authorization` and `has_cookie`, as booleans; - - `websocket`, when the connection was upgraded. -- Response detail: `response_content_type`, `upstream_status` (differs from - `status` when `smallwebwaf` answered itself), `cache_control`, `location` on - redirects, `aborted` when the client went away early. -- Decision: - - `action`: `forward`, `banned`, `denied`, `country_denied`, `rate_limited`, - `rule_blocked`, `waf_blocked`, `too_large`, `timed_out`, `upstream_error`, - or `admin` for a request `smallwebwaf` answered at one of its own - endpoints; - - `would_action`, in `observe` mode; - - `limit_percent`, and which setting gave it; - - `counts`, the client's minute, hour and day request and byte totals after - this request; - - `limit_hit`, which window; - - `rule_ids`, the rule file rules that matched; - - `waf_rule_ids` and `waf_score`; - - `reputation`, the sources that listed the client; - - `offence`, when one was recorded; - - `ban_expires`. -- Timings in milliseconds: `duration_total`, `duration_checks` (everything - `smallwebwaf` did before forwarding), `duration_waf`, - `duration_upstream_connect`, `duration_upstream_first_byte` and - `duration_upstream_total`. -- Bodies are never logged. Query strings are logged as received; an app that - carries secrets in query strings needs that fixed in the app. -- The process's own messages share the stream as JSON lines with - `"type":"process"`; request lines carry `"type":"request"`. -- Remote sending: - - each line is the message of an RFC 5424 syslog record, with octet-counted - framing on TCP and TLS; - - sending happens on its own goroutine from a bounded buffer - (`SWWAF_LOG_REMOTE_BUFFER`). An unreachable or slow server never delays a - request and never stops stdout. The sender reconnects with backoff, drops - the oldest lines when the buffer is full, and counts the drops in the - metrics. UDP gives no delivery signal at all and is offered only for - compatibility. +When `SWWAF_LOG_REMOTE_URL` is set, the same lines are also sent there, so a +deployment can stop depending on docker's log handling while `docker logs` keeps +working: + +- each line is the message of an RFC 5424 syslog record, with octet-counted + framing on TCP and TLS; +- sending happens on its own goroutine from a bounded buffer + (`SWWAF_LOG_REMOTE_BUFFER`). An unreachable or slow server never delays a + request and never stops stdout. The sender reconnects with backoff, drops the + oldest lines when the buffer is full, and counts the drops in the metrics. UDP + gives no delivery signal at all and is offered only for compatibility. ## Metrics endpoint -Prometheus text format at `/_smallwebwaf/metrics`, for a request carrying -`SWWAF_METRICS_TOKEN`. - -- Traffic: requests and bytes in and out, by status class and `action`; request - duration and upstream duration histograms; requests in flight. -- Limits and bans: - - limit hits by window and kind (requests, bytes or refusals); - - size and time limit hits by limit; - - offences by kind; - - bans made by cause, permanent bans, active bans; - - requests refused by the country lists, by country. -- Attack detection: - - rule file matches by rule id and action, and the number of rules loaded; - - Core Rule Set matches by mode and rule id, limited to the rules that - fired. -- Lookup and reputation: - - requests and bytes by AS number and by country, limited to the - `SWWAF_METRICS_TOP_N` busiest of each with the rest summed as `other`, so - the label set stays bounded; - - reputation queries, hits, failures and remaining daily budget by source; - - GeoJS requests and failures, and clients that counted as unknown because - it did not answer in time; - - when each list was last fetched, and the lookup database last read. -- Housekeeping: - - tracked clients; - - state file writes, write failures, last successful write time and size per - file; edits taken in, and edits set aside because they did not parse; - - alerts sent, failed, suppressed and dropped by destination; - - remote log lines sent, dropped and buffer depth; - - the standard Go runtime and process metrics. -- No metric carries a client IP address as a label. Per-address questions are - answered by the request log and `GET /_smallwebwaf/clients/`. +`GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request +carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. No metric +carries a client IP address as a label. Per-address questions are answered by +the request log and `GET /_smallwebwaf/clients/`. ## Admin endpoints @@ -1083,11 +1027,11 @@ through traefik, like any other request. - `GET /_smallwebwaf/metrics`: the metrics; needs `SWWAF_METRICS_TOKEN`. - `GET /_smallwebwaf/bans`, `POST /_smallwebwaf/bans` (a client or netblock, a duration, a reason) and `DELETE /_smallwebwaf/bans/`: need - `SWWAF_ADMIN_TOKEN`. Editing `bans.json` does the same without a token. -- `GET /_smallwebwaf/clients/`: needs `SWWAF_ADMIN_TOKEN`. The client's - current counters and history, which holds its lookup result and offences, and - its bans with their notes. It answers "why was this address refused" and "what - has this netblock been doing". + `SWWAF_ADMIN_TOKEN`. +- `GET /_smallwebwaf/clients/`: needs `SWWAF_ADMIN_TOKEN`. It answers with + the client's current counters, its history (with its lookup result and + offences), and its bans with their notes. It answers "why was this address + refused" and "what has this netblock been doing". - A token is sent as `Authorization: Bearer ` (see "Tokens" under "Settings"). While a token is unset, the endpoints that need it answer `404`, as does any other path under the prefix. @@ -1099,7 +1043,6 @@ through traefik, like any other request. the error burst. A client guessing tokens is therefore banned once it passes `SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute. A client in `SWWAF_ALLOW_NETS` skips the checks but still needs the token. -- An admin whose own address is banned lifts the ban by editing `bans.json`. ## Alert webhook schema @@ -1123,32 +1066,17 @@ hardened by changing its base image. It can add packages from nixpkgs. The image holds: -- Ubuntu 26.04 LTS, the newest long-term support release of Ubuntu, pinned by - digest. The image moves to the next LTS release when that ships. -- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, installed from a dated - snapshot of its archive and checked by hash (see "Pinning"). The Ubuntu image - has no CA certificates; without them Nix cannot download packages and - `smallwebwaf` cannot reach GeoJS. -- Nix and nixpkgs, fixed at one release (see "Pinning"). Root uses Nix directly; - no Nix daemon runs, and `build-users-group =` in `/etc/nix/nix.conf` lets root - build without the build users `nix-bin` does not create. -- runit, and `runsvinit` as the entrypoint, as the owner's code style guide asks - for service containers. No package carries `runsvinit`, so the image builds it - from its archived source (`github.com/peterbourgon/runsvinit`, unchanged - since 2015) at a fixed commit hash, with a `go.mod` the build writes. - `runsvinit` starts runit's `runsvdir`, which runs the `run` script of each - directory under `/etc/service` and runs it again whenever it exits. Ubuntu's - `sv` looks for services there too. -- The `smallwebwaf` binary, and a user of its own, `smallwebwaf` (uid and gid - 65532). -- The service directory `/etc/service/smallwebwaf`. Its `run` script waits one - second (`sleep 1`), makes `SWWAF_STATE_DIR` and every file in it belong to the - `smallwebwaf` user, and starts `smallwebwaf` as that user with runit's - `chpst`. -- The directory `/var/lib/smallwebwaf` for the state files, and - `/etc/smallwebwaf/rules.d` with the default rule file. -- Port 8080 declared (`EXPOSE 8080`), and the health check described below - (`HEALTHCHECK`). +- Ubuntu 26.04 LTS, with `ca-certificates`, `nix-bin` and `runit` from Ubuntu; +- nixpkgs, fixed at one release; +- `runsvinit` as the entrypoint; +- the `smallwebwaf` binary, a user of its own (uid and gid 65532), and its runit + service; +- its state and rules directories; +- `EXPOSE 8080` and the health check below. + +Every input to the image is pinned by digest or hash, so that an app built on it +gets the same packages each time. `docs/image.md` says how each part is pinned, +and why. Every `run` script, the app's included, is a bash script. It starts with `#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main` @@ -1161,68 +1089,14 @@ Beyond its `FROM` line, the app's Dockerfile adds: - any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.`; - its runit service: a directory under `/etc/service` named after the app, never `smallwebwaf`. Its `run` script waits one second (`sleep 1`) and then starts - the app with `chpst`, listening on `127.0.0.1:8081`. The app runs as a user of - its own, which the Dockerfile creates with `useradd`. That user is neither - root nor `smallwebwaf`, so that the app cannot touch the state files or the - process of `smallwebwaf`. + the app with `chpst`, listening where `SWWAF_UPSTREAM_URL` points (see + "Ports"). The app runs as a user of its own, which the Dockerfile creates with + `useradd`. That user is neither root nor `smallwebwaf`, so that the app cannot + touch the state files or the process of `smallwebwaf`. It sets no `ENTRYPOINT` or `USER` of its own. The container must start `runsvinit`, as root, so that it can start each service as its own user. - -An example, for an app whose binary is `app` and which takes the address it -listens on and the proxies it trusts as options of its own: - -```dockerfile -# The smallwebwaf image, pinned by digest. -FROM /smallwebwaf@sha256: - -# Packages the app needs, if any, from the nixpkgs in the image. -RUN nix-env -iA nixpkgs.git - -# The app's binary, and a user of its own to run it. -COPY app /usr/local/bin/app -RUN useradd --system --no-create-home --shell /usr/sbin/nologin app - -# The app's runit service. -COPY --chmod=755 app.run /etc/service/app/run -``` - -with `app.run` beside the Dockerfile: - -```bash -#!/usr/bin/env bash -set -euo pipefail - -main() { - sleep 1 - exec chpst -u app:app /usr/local/bin/app \ - --listen 127.0.0.1:8081 \ - --trusted-proxies 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1/32,::1/128 -} - -main "$@" -``` - -### Pinning - -- Ubuntu's packages come from Ubuntu's snapshot service, the archive as it was - at the moment the Dockerfile names (`--snapshot 20261001T000000Z`), never - earlier than the pinned Ubuntu image's date. The Dockerfile names the SHA-256 - hash of each of the snapshot's `InRelease` files, which cover every package - list and so every package, and the build checks them before installing. The - hashes are the amd64 archive's, so the image is built for amd64 only. The - snapshot is fetched over HTTPS with the CA certificates of the Go image that - builds `smallwebwaf`, itself pinned by digest, since the Ubuntu image has - none. -- nixpkgs comes from a release file on `releases.nixos.org` (`nixexprs.tar.xz` - of `nixos-26.05.11045.774debe7a0d1`), named with its SHA-256 hash, not from a - GitHub archive, whose bytes can change. It is set up for root as `nixpkgs`. - Root's Nix profile, `/nix/var/nix/profiles/default/bin`, is last on the - `PATH`, so that no package hides the image's own commands; busybox's `sv`, for - one, looks for services elsewhere. Unpacked, nixpkgs takes about 500 MiB. -- Security fixes from Ubuntu and nixpkgs come with a newer `smallwebwaf` image, - which an app takes by changing the digest in its `FROM` line. The image moves - to each new nixpkgs release, every six months. +"Getting started" in `README.md` shows such a Dockerfile and `run` script. ### The two processes @@ -1240,16 +1114,15 @@ main "$@" - a rule file or state file that does not parse; - an unwritable `SWWAF_STATE_DIR`. - A `smallwebwaf` that stops at start is started again every second, logging its - message each time, until the cause is fixed. The health check fails meanwhile. - Once running, a broken helper or a broken edit never takes the protected - service down. + message each time, until the cause is fixed; the health check fails meanwhile. + Once running, a failing GeoJS, reputation source or alert destination, or a + broken edit, never takes the protected service down. - `docker stop` sends `SIGTERM` to `runsvinit`, which has runit stop both services at once, each with `SIGTERM`, and exits once they have stopped. `smallwebwaf` stops taking requests, gives those in progress five seconds to - finish, and writes its state files. Those requests can finish only while the - app still answers. An app that exits at once on `SIGTERM` ends them: the - client gets `502`, or a response cut short. To let them finish, the app must - keep taking and answering requests for five seconds after `SIGTERM`. + finish, and writes its state files. An app that exits at once on `SIGTERM` + ends those requests: the client gets `502`, or a response cut short. To let + them finish, the app must keep answering for five seconds after `SIGTERM`. - The container's root filesystem stays writable: runit writes each service's status into its directory under `/etc/service`. @@ -1257,37 +1130,38 @@ main "$@" The image's `HEALTHCHECK` runs `smallwebwaf healthcheck`. It passes while `smallwebwaf` answers `GET /_smallwebwaf/healthz` on `127.0.0.1`, at the port in -`SWWAF_LISTEN_ADDR`, and the app accepts connections at the address in -`SWWAF_UPSTREAM_URL`. It fails when either does not. The container therefore -shows as healthy only while both processes are up. traefik sends a container no -requests until it shows as healthy. So the check runs every second from the -container's start until it first passes, for up to a minute, and every 30 -seconds after that. An app with a health check of its own can replace the +`SWWAF_LISTEN_ADDR`, and the app accepts connections at `SWWAF_UPSTREAM_URL`. So +the container shows as healthy only while both processes are up. traefik sends a +container no requests until it shows as healthy. So the check runs every second +from the container's start until it first passes, for up to a minute, and every +30 seconds after that. An app with a health check of its own can replace the image's `HEALTHCHECK` with one that checks both. ### Ports -`smallwebwaf` listens on no port but the one in `SWWAF_LISTEN_ADDR`, and its own -endpoints are on that listener. Another port may be set: the health check -follows the setting, traefik's port label +`smallwebwaf` listens on no port but the one in `SWWAF_LISTEN_ADDR`, 8080 by +default, which traefik routes to. Its own endpoints are on that listener, and it +forwards every other request to the app at `SWWAF_UPSTREAM_URL`, +`127.0.0.1:8081` by default. Another port may be set: the health check follows +the setting, traefik's port label (`traefik.http.services..loadbalancer.server.port`) must name it, and the app must leave it free. The address part of `SWWAF_LISTEN_ADDR` stays empty (`:9000`, never `127.0.0.1:9000`), so that traefik reaches `smallwebwaf` on the -container's address and the health check on `127.0.0.1`. The app listens on -`127.0.0.1:8081` only. An app that listens on every address can be reached +container's address and the health check on `127.0.0.1`. The app listens on that +loopback address only. An app that listens on every address can be reached around `smallwebwaf` by anything that reaches the container. ### State -The app keeps its data in directories of its own, such as `/var/lib/app`, and -`smallwebwaf` keeps its state files beside them in `SWWAF_STATE_DIR`. Without a -volume there, the files survive a restart of the container and are lost when a -deploy replaces it. A volume mounted at `/var/lib/smallwebwaf`, named or a host -directory, keeps them across deploys. The `run` script makes it and every file -in it belong to the `smallwebwaf` user, so a host directory mounted there needs -no change of owner. The volume is separate from the app's, holds a few tens of -MiB at most with the defaults, and needs no backup beyond the host's own. The -image declares no volume, since every app image built on it would inherit it. +`smallwebwaf` keeps its state files in `SWWAF_STATE_DIR`, beside the app's own +data directories. Without a volume there, the files survive a restart of the +container and are lost when a deploy replaces it. A volume mounted at +`/var/lib/smallwebwaf`, named or a host directory, keeps them across deploys. +The `run` script makes it and every file in it belong to the `smallwebwaf` user, +so a host directory mounted there needs no change of owner. The volume is +separate from the app's, holds a few tens of MiB at most with the defaults, and +needs no backup beyond the host's own. The image declares no volume, since every +app image built on it would inherit it. ### Tokens @@ -1306,8 +1180,7 @@ headers, besides the private address ranges traefik is on. The usual default for a trusted-proxy setting, the private ranges alone, does not include loopback: an app left at it ignores the header and sees every visitor as `127.0.0.1`. A list given replaces that default, so the app's own trusted-proxy setting names them -all, `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1/32,::1/128`, as in the -example. +all: `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1/32,::1/128`. ### upaas @@ -1315,69 +1188,9 @@ An app deployed by upaas takes this shape with no change to upaas. upaas builds the app's Dockerfile and runs the one container as it runs any app. The traefik labels route to port 8080, and any `SWWAF_` settings go with the app's environment variables. The volume for `/var/lib/smallwebwaf` and the directory -of any token file go beside the app's own volumes. - -### Rollout per service - -Build the app's image on the `smallwebwaf` image, with no setting, and deploy -it. Nothing needs tuning first. Afterwards the request log and the ban notes say -why each client was refused. A real visitor refused by mistake is let through -with an exclusion (`SWWAF_WAF_DISABLED_RULES`, `SWWAF_WAF_EXEMPT_PATHS`, -`SWWAF_RATE_LIMIT_EXEMPT_PATHS`) or an exemption -(`SWWAF_RATE_LIMIT_EXEMPT_NETS`, `SWWAF_ALLOW_NETS`), and its ban is lifted in -`bans.json`. Additions to consider at any time: - -- an alert destination; -- country lists or biased thresholds; -- reputation sources; -- the metrics and admin tokens; -- a remote log server; -- an app-specific rule file. - -gitea is built on the image like any other app. `docs/gitea.md` records what the -defaults do in front of it, and `script/gitea-requests` checks those notes. - -## Risks the design has to handle - -- Forged `X-Forwarded-For`: handled by step 2 of "What happens to a request". - The default trusts every private address, which in the intended deployment - means traefik. Where other containers reach `smallwebwaf` directly, or - visitors on a private network reach traefik, they can name another address in - the header. `SWWAF_TRUSTED_PROXIES` set to traefik's own network closes that - gap. -- Many clients behind one address (mobile carriers, offices, Tor) share its - limits and its bans. The defaults sit well above one person's use, the static - lists take known shared addresses out, and the ban notes show an admin what - happened before a ban is lifted. -- A `ban` rule that matches a real visitor bans it for a long time (see "Bans"). - So the default rule file keeps to requests no real visitor sends, anchored at - the site root, and a ban is lifted in `bans.json`. -- IPv6 address rotation inside a /64: handled by grouping. -- Widely distributed scrapers, thousands of addresses at low rates each: - per-client limits do not see them. The AS number and netblock anomaly alerts - reveal them, and a low `SWWAF_ASN_LIMIT_PERCENT` for that AS number is the - response. -- Core Rule Set false positives: a match refuses only that request and bans no - one by itself. The request log names the rule, and exclusions by rule id and - path fix it. An engine error on a request forwards the request, and is logged - and counted. Attacks in request bodies are not refused by default, for the - reason under "Attack detection". -- The fourth and fifth changes to the Core Rule Set hold in front of every app, - and no setting restores those rules. An app that uses one of those query - parameters as a file on the server, or passes it to a shell, must check the - value itself. A rule file can refuse it instead. The same holds for an app - that reads a cookie named `gitea_flash` or `redirect_to`, or passes `Referer` - to a shell. -- The admin endpoints can be reached from the internet. All but the health check - need a token and are off while it is unset, and token guessing is banned - through the error burst. -- GeoJS, the default lookup source: every new visitor's address goes to a third - party, and a swarm of fresh addresses is when GeoJS may slow down or block - `smallwebwaf`. Kept answers and batched requests keep the number of requests - low. The file source has neither risk. -- Slow-request attacks: bounded by `SWWAF_CLIENT_REQUEST_TIMEOUT`, - `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` and `SWWAF_CLIENT_IDLE_TIMEOUT`. -- Alert floods: the cooldown and the hourly cap. +of any token file go beside the app's own volumes. gitea is built on the image +like any other app; `docs/gitea.md` records what the defaults do in front of it, +and `script/gitea-requests` checks those notes. ## Planned, not built diff --git a/docs/gitea.md b/docs/gitea.md index 85d7a40..d703385 100644 --- a/docs/gitea.md +++ b/docs/gitea.md @@ -172,6 +172,18 @@ the runner. More than 30 such refusals from its address within a minute, as when many jobs upload at once, ban it through the error burst. `actions/upload-artifact@v4` does not send the header. +## What body inspection refuses + +With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In +front of gitea it then refuses these, until their rule ids, named in the request +log, are added to `SWWAF_WAF_DISABLED_RULES`: + +- issue and comment text, wiki pages and files saved in the web editor that hold + shell commands or code (932125, 932235, 932250 and others); +- package descriptions that show code; +- PyPI uploads (922130); +- attachments named like `debug.log` or `config.yml` (932180). + ## What `script/gitea-requests` sends Of the requests above, the script sends: diff --git a/docs/image.md b/docs/image.md new file mode 100644 index 0000000..c208356 --- /dev/null +++ b/docs/image.md @@ -0,0 +1,56 @@ +# What the image holds + +The `smallwebwaf` image is the base of each app's own image. The requirements +are under "Deployment" in [`SPEC.md`](../SPEC.md); this file lists what the +image holds, how each part is pinned, and why. + +## Contents + +- Ubuntu 26.04 LTS, the newest long-term support release of Ubuntu, pinned by + digest. The image moves to the next LTS release when that ships. +- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, installed from a dated + snapshot of its archive and checked by hash. The Ubuntu image has no CA + certificates; without them Nix cannot download packages and `smallwebwaf` + cannot reach GeoJS. +- Nix and nixpkgs, fixed at one release. Root uses Nix directly; no Nix daemon + runs. `nix-bin` creates no build users, so `build-users-group =` in + `/etc/nix/nix.conf` lets root build without them. +- runit, and `runsvinit` as the entrypoint, as the owner's code style guide asks + for service containers. No package carries `runsvinit`, so the image builds it + from its archived source (`github.com/peterbourgon/runsvinit`, unchanged + since 2015) at a fixed commit hash, with a `go.mod` the build writes. + `runsvinit` starts runit's `runsvdir`, which runs the `run` script of each + directory under `/etc/service` and runs it again whenever it exits. Ubuntu's + `sv` looks for services there too. +- The `smallwebwaf` binary, and a user of its own, `smallwebwaf` (uid and gid + 65532). +- The service directory `/etc/service/smallwebwaf`. Its `run` script waits one + second (`sleep 1`), makes `SWWAF_STATE_DIR` and every file in it belong to the + `smallwebwaf` user, and starts `smallwebwaf` as that user with runit's + `chpst`. +- The directory `/var/lib/smallwebwaf` for the state files, and + `/etc/smallwebwaf/rules.d` with the default rule file. +- Port 8080 declared (`EXPOSE 8080`), and the health check (`HEALTHCHECK`) that + "The health check" in `SPEC.md` describes. + +## Pinning + +- Ubuntu's packages come from Ubuntu's snapshot service, the archive as it was + at the moment the Dockerfile names (`--snapshot 20261001T000000Z`), never + earlier than the pinned Ubuntu image's date. The Dockerfile names the SHA-256 + hash of each of the snapshot's `InRelease` files, which cover every package + list and so every package, and the build checks them before installing. The + hashes are the amd64 archive's, so the image is built for amd64 only. The + snapshot is fetched over HTTPS with the CA certificates of the Go image that + builds `smallwebwaf`, itself pinned by digest, since the Ubuntu image has + none. +- nixpkgs comes from a release file on `releases.nixos.org` (`nixexprs.tar.xz` + of `nixos-26.05.11045.774debe7a0d1`), named with its SHA-256 hash, not from a + GitHub archive, whose bytes can change. It is set up for root as `nixpkgs`. + Root's Nix profile, `/nix/var/nix/profiles/default/bin`, is last on the + `PATH`, so that no package hides the image's own commands; busybox's `sv`, for + one, looks for services elsewhere. Unpacked, nixpkgs takes about 500 MiB. +- An app built on the image gets the same packages each time it is built. + Security fixes from Ubuntu and nixpkgs come with a newer `smallwebwaf` image, + which an app takes by changing the digest in its `FROM` line. The image moves + to each new nixpkgs release, every six months. diff --git a/docs/metrics.md b/docs/metrics.md new file mode 100644 index 0000000..10e9f58 --- /dev/null +++ b/docs/metrics.md @@ -0,0 +1,83 @@ +# The metrics + +`GET /_smallwebwaf/metrics` answers in the Prometheus text format. The +requirements are under "Metrics endpoint" in [`SPEC.md`](../SPEC.md); this file +lists the metrics. Every metric carries the label `instance`, set to +`SWWAF_INSTANCE_NAME`; Prometheus keeps it as `exported_instance` unless the +scrape sets `honor_labels: true`. No metric carries a client's address. + +## Traffic + +- `smallwebwaf_requests_total`, `smallwebwaf_request_bytes_total` and + `smallwebwaf_response_bytes_total`, by `status_class` such as `2xx` and by + `action`; +- `smallwebwaf_request_duration_seconds` and + `smallwebwaf_upstream_duration_seconds`, as histograms; +- `smallwebwaf_requests_in_flight`. + +## Limits and bans + +- `smallwebwaf_rate_limit_hits_total`, by `window` and by `kind` (`requests`, + `bytes` or `refusals`); +- `smallwebwaf_size_and_time_limit_hits_total`, by `limit`, the setting passed; +- `smallwebwaf_offences_total`, by `kind`; +- `smallwebwaf_bans_made_total`, by `cause`; +- `smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`. + +## Attack detection + +- `smallwebwaf_rule_matches_total`, by `rule_id` and `action`, and + `smallwebwaf_rules_loaded`; +- `smallwebwaf_waf_matches_total`, by `mode` and `rule_id`, for the rules that + fired. + +## Countries and AS numbers + +For the `SWWAF_METRICS_TOP_N` busiest of each, the rest as `other`: + +- `smallwebwaf_country_requests_total`, + `smallwebwaf_country_request_bytes_total`, + `smallwebwaf_country_response_bytes_total` and + `smallwebwaf_country_list_refusals_total`, by `country`; +- `smallwebwaf_asn_requests_total`, `smallwebwaf_asn_request_bytes_total` and + `smallwebwaf_asn_response_bytes_total`, by `asn`. + +## Lookups + +- `smallwebwaf_geojs_requests_total` and `smallwebwaf_geojs_failures_total`; +- `smallwebwaf_geojs_unanswered_total`: the requests that needed an answer and + went on without it; +- with the file source, + `smallwebwaf_lookup_database_last_read_timestamp_seconds` and + `smallwebwaf_lookup_database_read_failures_total`. + +## Reputation sources + +By `source`: each list's URL, each zone, and `abuseipdb`: + +- `smallwebwaf_reputation_hits_total`, `smallwebwaf_reputation_queries_total` + and `smallwebwaf_reputation_failures_total`; +- `smallwebwaf_reputation_last_fetch_timestamp_seconds`, for a list; +- `smallwebwaf_reputation_daily_budget_remaining`, for AbuseIPDB. + +## State files + +By `file`: + +- `smallwebwaf_state_file_writes_total`, + `smallwebwaf_state_file_write_failures_total`, + `smallwebwaf_state_file_last_write_timestamp_seconds` and + `smallwebwaf_state_file_size_bytes`; +- `smallwebwaf_state_file_edits_taken_in_total` and + `smallwebwaf_state_file_edits_set_aside_total`; +- `smallwebwaf_tracked_clients`: the clients in the table. + +## Remote log, alerts and the process + +- while a syslog server is set, `smallwebwaf_remote_log_lines_sent_total`, + `smallwebwaf_remote_log_lines_dropped_total` and + `smallwebwaf_remote_log_buffer_depth`; +- by `destination`, `smallwebwaf_alerts_sent_total`, + `smallwebwaf_alerts_failed_total`, `smallwebwaf_alerts_suppressed_total` and + `smallwebwaf_alerts_dropped_total`; +- Go's own `go_` metrics and the process's `process_` metrics. diff --git a/docs/request-log.md b/docs/request-log.md new file mode 100644 index 0000000..e0c1922 --- /dev/null +++ b/docs/request-log.md @@ -0,0 +1,128 @@ +# The request log's fields + +`smallwebwaf` writes one JSON object per line on stdout for every request. The +requirements are under "Request log" in [`SPEC.md`](../SPEC.md); this file lists +the fields. A field that does not apply to a request is left out. + +## Example + +``` +{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104} +``` + +## Standard web log fields + +- `time`: when the request arrived, RFC 3339 with milliseconds, in UTC; +- `instance`: `SWWAF_INSTANCE_NAME`; +- `client_ip`, `method`, `scheme`, `host`, `path`, `query`, `protocol`; +- `status`: what the client was sent, `0` if nothing; +- `request_bytes` and `response_bytes`: body bytes each way; +- `referer` and `user_agent`. + +## Request detail + +- `request_id`: the `X-Request-ID` a trusted proxy sent, or a new random one of + 26 letters and digits. It goes to the app in `X-Request-ID`. +- `peer_ip`: the TCP peer, normally traefik. +- `forwarded_for`: the `X-Forwarded-For` header as received. +- `client_group`: the client as rate limiting counts it, an IPv4 address as a + /32 or an IPv6 group. +- `asn`, `as_name` and `country`: the client's, as looked up. Each is empty + when: + - lookups are off; + - the client is not looked up; + - the source cannot place the client; + - GeoJS had not answered by the time the request went on. +- `content_type` and `content_length`: the request's own. +- `request_headers`: the headers listed in `SWWAF_LOG_REQUEST_HEADERS`, by name + in lower case. +- `has_authorization` and `has_cookie`: true when the request had those headers. + Their values are never logged. +- `websocket`: true when the app switched the connection to another protocol. + +## Response detail + +- `upstream_status`: what the app answered. It differs from `status` when + `smallwebwaf` answered itself. +- `response_content_type`, `cache_control` and `location`: the answer's headers. +- `aborted`: true when the client went away early. + +## Decision + +- `action`: what was done: + - `forward`: passed to the app; + - `denied`: refused for `SWWAF_DENY_NETS`, or for a reputation source with + the action `deny`; + - `banned`: refused under a ban, or for a `ban` rule, a trap path or a + CrowdSec decision, each of which bans the client; + - `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`: + refused by that check; + - `too_large` and `timed_out`: a size or time limit was passed; + - `upstream_error`: the app could not be reached, or its answer broke off; + - `admin`: answered at one of `smallwebwaf`'s own endpoints. +- `would_action`: in `observe` mode, what `enforce` mode would have done with a + request it would have refused. `action` then names what was done, normally + `forward`. +- `limit_percent` and `limit_percent_setting`: the client's percentage of the + rate limits and the setting that lowered it, when one did. `bytes_percent` and + `bytes_percent_setting` do the same for the byte limits. +- `counts`: the client's requests in the minute, hour and day as rate limiting + counts them, this request included, as `minute`, `hour` and `day`; its bytes + as `minute_bytes`, `hour_bytes` and `day_bytes`. A count can have a fraction, + since each window is counted in two buckets with the earlier one weighted. + `counts` is left out when rate limiting did not count the request. +- `limit_hit`: the limit broken: `minute`, `hour` or `day` for a rate limit; + `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or + `error_burst`. A request whose bytes broke a limit was passed on whole, so its + `action` stays `forward`. +- `rule_ids`: the rule file rules that matched, in order, up to the one that + refused. +- `waf_score` and `waf_rule_ids`: the Core Rule Set's score and the rules that + matched, for a request it inspected. +- `reputation`: the sources the client is on, in this order: + - blocklist URLs; + - the CrowdSec decision list's URL; + - DNS blocklist zones; + - `abuseipdb`. +- `offence`: the kind of offence recorded, when one was. +- `ban_expires`: when the client's ban ends, for a request that made one or was + refused under one: a time, or `permanent`. + +## Timings + +All in milliseconds: + +- `duration_total`: from the request's headers being read to its log line; +- `duration_checks`: to the end of the checks, before forwarding; +- `duration_waf`: the part of that the Core Rule Set took; +- `duration_upstream_connect`: from the hand-over to the app until there was a + connection; +- `duration_upstream_first_byte`: until the first byte of the answer; +- `duration_upstream_total`: until the end of the answer. + +## Every line + +Every line has `type`, the standard web log fields, `request_id`, `peer_ip`, +`client_group`, `asn`, `as_name`, `country`, `action` and `duration_total`. The +process's own messages share the stream with `"type":"process"`; request lines +carry `"type":"request"`. + +## Sending the log to a syslog server + +While `SWWAF_LOG_REMOTE_URL` is set, every line on stdout is also sent to that +server, as the message of an RFC 5424 syslog record. Over UDP each record is one +datagram. Over TCP and TLS each record follows its length in bytes and a space. +stdout is unchanged. The record carries: + +- the facility from `SWWAF_LOG_REMOTE_FACILITY`; +- the severity informational; +- the time; +- the host's name; +- the app name from `SWWAF_LOG_REMOTE_APP_NAME`. + +Lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines, so a server that is +slow or down never holds up a request. When the buffer is full, the oldest line +is dropped. A failed connection is tried again a second later, and twice as long +after each further failure, up to a minute. A line too long for one UDP datagram +is dropped alone. As `smallwebwaf` stops, it spends at most two seconds sending +the lines still waiting, and drops the rest. stdout has carried them. -- 2.54.0 From d712bb1496dcf21c1d8a3c06ee94e6d54d17f13b Mon Sep 17 00:00:00 2001 From: sneak Date: Thu, 22 Oct 2026 21:08:26 +0000 Subject: [PATCH 4/8] SPEC: requirements and reasons only, reference material in docs/ docs/settings.md is now the one settings reference; the spec keeps the conventions, the off list and each group's requirements and reasons, and links to it for the per-setting entries. The endpoint list, the ban notes, the state file contents, the rule file format and the alert object likewise live in docs/ and are linked from the spec. The package list from the README's Design section joins "Architecture". The nine docs files are byte-identical to those on the README branch. Package comments that named "Alert webhook schema", "Request log" and "Metrics endpoint" for their field lists now name the docs files. Model: fable-5-1 --- SPEC.md | 852 +++++++++++------------------- docs/admin-endpoints.md | 35 ++ docs/alerts.md | 149 ++++++ docs/bans.md | 84 +++ docs/core-rule-set.md | 31 ++ docs/lookup.md | 79 +++ docs/reputation.md | 113 ++++ docs/rules.md | 114 ++++ docs/settings.md | 483 +++++++++++++++++ docs/state-files.md | 102 ++++ internal/alerts/alerts.go | 11 +- internal/metrics/metrics.go | 7 +- internal/ratelimit/ratelimit.go | 2 +- internal/remotelog/remotelog.go | 7 +- internal/requestlog/requestlog.go | 7 +- 15 files changed, 1515 insertions(+), 561 deletions(-) create mode 100644 docs/admin-endpoints.md create mode 100644 docs/alerts.md create mode 100644 docs/bans.md create mode 100644 docs/core-rule-set.md create mode 100644 docs/lookup.md create mode 100644 docs/reputation.md create mode 100644 docs/rules.md create mode 100644 docs/settings.md create mode 100644 docs/state-files.md diff --git a/SPEC.md b/SPEC.md index f5c683e..e7c5586 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4,9 +4,9 @@ runs inside the app's container, between traefik and the app. This document says what it must do and why, each requirement once. Everything in it is built, except the last section, "Planned, not built". `README.md` says how to run it. -`EVALUATION.md` says why no existing tool was adopted. `docs/` holds the lists -this document links to, and `docs/gitea.md` records how `smallwebwaf` behaves in -front of gitea. +`EVALUATION.md` says why no existing tool was adopted. The reference documents +under `docs/`, which this document links to, give the settings, the formats and +the lists; every fact lives in one of these files and is linked from the others. ## Purpose @@ -42,10 +42,47 @@ front of gitea. - One statically linked Go binary. It writes only to its state directory. It ships in the image that is the base of the app's own image (see "Deployment"). - No database of any kind. Nothing is read from disk while serving a request. +- Packages: + - `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`. + - `internal/smallwebwaf`: the process. It reads the settings, the rule + files, the lookup database and the state files, listens, and serves until + `SIGTERM` or `SIGINT`. Then it stops, writing the state files. Run as + `smallwebwaf healthcheck`, it is the image's health check instead. + - `internal/config`: reads the settings, the one place they are read. + - `internal/proxy`: what happens to each request. It works out the client + and runs the checks in `check`. It passes the request to the app with the + standard library's `httputil.ReverseProxy`, within the timeouts and size + limits, and answers its own endpoints in `answerAdmin`. Once the request + has ended, it counts bytes, refusals and anomalies, and writes the log + line. + - `internal/ratelimit`: the table of clients. It counts requests, bytes and + refusals, and keeps each client's history. + - `internal/bans`: the bans. It holds each netblock's bans with their notes, + decides how long a new ban lasts, and picks the ban to drop past + `SWWAF_MAX_BANS`. + - `internal/rules`: reads the rule files, and again as they change, and + tells which rules a request matches. + - `internal/waf`: the Core Rule Set with the six changes, as Coraza + directives. + - `internal/lookup`: looks clients up through GeoJS or in the lookup + database. + - `internal/reputation`: fetches the lists, asks the DNS blocklists and + AbuseIPDB in the background, and keeps their answers. + - `internal/anomaly`: the anomaly counters and their alerts. + - `internal/alerts`: holds back repeats and alerts past the hourly limit, + and sends the rest to the webhook, Slack and ntfy, each from a queue of + its own. + - `internal/state`: reads the state files at start, takes in edits while + running, and writes the files when due and at the stop. + - `internal/requestlog` and `internal/remotelog`: the lines on stdout, and + sending them to the syslog server. + - `internal/metrics`: the metrics, in the Prometheus text format. + - `Dockerfile`: the lint and test phases, then the image. + `deploy/example-app` is an app built on the image, which + `script/example-app` checks. - Libraries, checked against the owner's Go dependency defaults: - - the standard library for the proxy (`httputil.ReverseProxy`), HTTP - clients, DNS, logging (`log/slog`) and the state files (`encoding/json`, - `os.Rename`); + - the standard library for the proxy, HTTP clients, DNS, logging + (`log/slog`) and the state files (`encoding/json`, `os.Rename`); - `github.com/corazawaf/coraza/v3` and `github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to it, and the @@ -56,6 +93,7 @@ front of gitea. - `github.com/oschwald/maxminddb-golang/v2` to read the lookup database; - `github.com/fsnotify/fsnotify` to notice edited files; - `github.com/prometheus/client_golang` for the metrics; + - `github.com/maxmind/mmdbwriter`, in the tests, to write lookup databases; - remote log sending written in this project, since the standard library's `log/syslog` is frozen and writes only the older syslog format. @@ -101,7 +139,7 @@ names the section that holds its rules. 8. The client's limit percentage (see "Biased thresholds"). 9. Rate limiting (see "Request rate limits, per client"). A request over a limit times the percentage is refused and bans the client. -10. The trap paths (see `SWWAF_TRAP_PATHS`). A request for one is refused and +10. The trap paths (see "Attack detection"). A request for one is refused and bans the client. 11. The rule files (see "Rule files"). Matching stops at the first rule that refuses. @@ -117,8 +155,8 @@ names the section that holds its rules. counters, as `SWWAF_BYTES_COUNT` says. A client whose total passes a byte limit times the percentage has broken that limit and is banned. The response in progress is not cut off. - - The client's refusals are counted for the error burst (see - `SWWAF_ERROR_BURST_THRESHOLD`). + - The client's refusals are counted for the error burst (see "Attack + detection"). - The client's history and the anomaly counters are updated, and the alert thresholds are evaluated. @@ -139,7 +177,8 @@ the token check still apply. - Memory is bounded. When the table of clients holds `SWWAF_MAX_TRACKED_CLIENTS`, the least recently seen client is dropped first, with its history. The bans, the GeoJS answers, the reputation verdicts and the - anomaly counters have bounds of their own, given where each is described. + anomaly counters have bounds of their own; `docs/settings.md` gives the fixed + ones under "Fixed limits". ## Bans @@ -200,11 +239,12 @@ no offence: - a blocklist or reputation hit whose action is `deny`; - the country lists. -Every ban carries notes with what an admin needs to decide whether to lift it. -An admin adds or lifts a ban by editing `bans.json` (see "Persistent state"), or -through the ban endpoints when `SWWAF_ADMIN_TOKEN` is set. A ban lifted before -it ends is kept, marked as lifted, and does not count toward a longer ban. -Deleting its entry from `bans.json` forgets it entirely. +Every ban carries notes with what an admin needs to decide whether to lift it, +drawn from what `smallwebwaf` already knows; nothing is looked up to fill them. +`docs/bans.md` lists the notes and says how a ban is made, kept or lifted, by +editing `bans.json` or through the ban endpoints. A ban lifted before it ends is +kept, marked as lifted, and does not count toward a longer ban. Deleting its +entry from `bans.json` forgets it entirely. A permanent ban does not run out. It ends when an admin lifts it, or when `smallwebwaf` drops it from a full table, as below. When `smallwebwaf` holds @@ -248,42 +288,32 @@ Conventions: - An invalid setting stops the process at start with a message naming the variable. At start the settings in effect are logged, with secrets masked. -The settings, by group: +`docs/settings.md` lists every setting with its default and the values it takes, +in the groups below. What each group must do, and why: ### Core -- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. -- `SWWAF_LISTEN_ADDR` (default `:8080`): the one listener, for the requests - `smallwebwaf` forwards and for its own endpoints. The address part stays empty - (see "Ports"). -- `SWWAF_INSTANCE_NAME` (default: the container's host name, which docker sets - to the first 12 characters of the container's id): included in every log line, - metric and alert. Set it to a name that survives a redeploy and tells - instances apart, such as `fsn1app1/gitea`. -- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to log and alert on - every decision while refusing nothing. -- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`, - the private address ranges): the netblocks whose `X-Forwarded-For` is - believed, normally where traefik reaches `smallwebwaf` from. A client that - connects directly cannot claim another address. A list given replaces the - default; set but empty, it trusts nothing. Where other containers reach - `smallwebwaf` directly, or visitors on a private network reach traefik, they - can name another address in the header; this setting, narrowed to traefik's - own network, closes that gap. -- `SWWAF_IPV6_GROUP_PREFIX` (default `64`), from 32 to 128. -- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the clients held in memory and - in `clients.json`. -- `SWWAF_MAX_BANS` (default `5000`): the bans `smallwebwaf` made that are held, - past, active and permanent (see "Bans"). +- One listener, `SWWAF_LISTEN_ADDR`, for the requests `smallwebwaf` forwards to + `SWWAF_UPSTREAM_URL` and for its own endpoints (see "Ports"). +- `SWWAF_INSTANCE_NAME` is in every log line, metric and alert, so that + instances can be told apart. The default, the container's host name, changes + at every redeploy; a name set by the operator survives it. +- `SWWAF_MODE=observe` logs and alerts on every decision while refusing nothing, + so that a configuration can be tried before it is enforced. +- `X-Forwarded-For` is believed only from `SWWAF_TRUSTED_PROXIES`, by default + the private address ranges, so a client that connects directly cannot claim + another address. Where other containers reach `smallwebwaf` directly, or + visitors on a private network reach traefik, they can name another address in + the header; the setting, narrowed to traefik's own network, closes that gap. +- `SWWAF_IPV6_GROUP_PREFIX`, `SWWAF_MAX_TRACKED_CLIENTS` and `SWWAF_MAX_BANS` + set the group size and the bounds under "Counting method" and "Bans". ### Tokens -- `SWWAF_ADMIN_TOKEN`: the bearer token for the ban endpoints and - `/_smallwebwaf/clients/`. Unset by default, which switches them off. -- `SWWAF_METRICS_TOKEN`: the bearer token a scraper sends for - `/_smallwebwaf/metrics`. Unset by default, which switches the metrics off, - since they would otherwise be open to anyone on the internet. It is separate - from `SWWAF_ADMIN_TOKEN`, so a scraper that holds it cannot manage bans. +- `SWWAF_ADMIN_TOKEN` switches on the ban endpoints and + `/_smallwebwaf/clients/`; `SWWAF_METRICS_TOKEN` switches on the metrics, + which would otherwise be open to anyone on the internet. The two are separate, + so a scraper that holds the metrics token cannot manage bans. - A token is a long random value, since the endpoints can be reached from the internet. One shorter than 32 characters stops the start. The app starts with the same environment variables as `smallwebwaf`, so it can read a token set as @@ -292,50 +322,39 @@ The settings, by group: ### State files -- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the JSON state files, in a - directory of their own beside the app's data, belonging to the `smallwebwaf` - user. -- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be - written after a ban is made, lifted or made permanent, so a burst of changes - becomes one write. -- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is - written. +- The state files live in `SWWAF_STATE_DIR`, a directory of their own beside the + app's data, belonging to the `smallwebwaf` user. +- `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted + or made permanent, so a burst of changes becomes one write. Every file is + written every `SWWAF_STATE_COUNTER_INTERVAL`, and at an orderly stop. ### Logging -- The request log on stdout is always on and has no switch. -- `SWWAF_LOG_LEVEL` (default `info`): for the process's own messages, which are - JSON lines on stdout too, marked `"type":"process"`. It does not filter the - request log. -- `SWWAF_LOG_REQUEST_HEADERS` (default - `accept,accept-language,accept-encoding,content-type,origin,range`): the - request headers to record. `Authorization`, `Cookie` and `Set-Cookie` values - are never logged, only whether they were present. -- `SWWAF_LOG_REMOTE_URL`: when set, every log line is also sent there, as - `syslog+udp://host:514`, `syslog+tcp://host:514` or `syslog+tls://host:6514`. -- `SWWAF_LOG_REMOTE_TLS_CA_FILE`: an optional CA certificate file for the `+tls` - form. -- `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the lines held in memory while - the server is unreachable. When full, the oldest are dropped and counted. -- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`) and `SWWAF_LOG_REMOTE_APP_NAME` - (default `SWWAF_INSTANCE_NAME`): the syslog header fields. +- The request log on stdout is always on and has no switch. The process's own + messages are JSON lines on stdout too, marked `"type":"process"`; + `SWWAF_LOG_LEVEL` filters only those. +- `SWWAF_LOG_REQUEST_HEADERS` names the request headers to record. + `Authorization`, `Cookie` and `Set-Cookie` values are never logged, only + whether they were present. +- `SWWAF_LOG_REMOTE_URL` sends every log line to a syslog server as well, over + UDP, TCP or TLS. Lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines; + when it is full, the oldest are dropped and counted. ### Metrics -- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and countries get - series of their own. The rest are summed as `other`, so the label set stays +- `SWWAF_METRICS_TOP_N` bounds the series by AS number and by country: that many + of the busiest each, the rest summed as `other`, so the label set stays bounded. ### Static lists -- `SWWAF_ALLOW_NETS`: bypass everything (monitoring, the owner's own networks). -- `SWWAF_RATE_LIMIT_EXEMPT_NETS`: bypass the request and byte limits only. The - error burst, attack detection and bans still apply. -- `SWWAF_DENY_NETS`: always refused. - -Many clients share one address (mobile carriers, offices, Tor), and so share its -limits and its bans. The defaults sit well above one person's use, and the first -two lists take known shared addresses out. +- `SWWAF_ALLOW_NETS` bypasses everything (monitoring, the owner's own networks). + `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and byte limits only; the + error burst, attack detection and bans still apply. `SWWAF_DENY_NETS` is + always refused. +- Many clients share one address (mobile carriers, offices, Tor), and so share + its limits and its bans. The defaults sit well above one person's use, and the + first two lists take known shared addresses out. ### Request rate limits, per client @@ -344,20 +363,14 @@ busy person produces. A browser loading a heavy page makes a few hundred requests, a git clone a handful, and several people often share one address. A machine that talks to the app all day, such as a gitea Actions runner, can still pass the day limit. Its address belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. - -- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR` - (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`). -- `SWWAF_RATE_LIMIT_EXEMPT_PATHS`: path prefixes not counted, such as static - assets or health checks. +`SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as +static assets or health checks. ### Byte limits, per client A response's bytes are counted when it ends, so every default sits above the -largest response allowed, and no single download breaks one. - -- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR` - (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`). -- `SWWAF_BYTES_COUNT` (default `both`): `response`, `request` or `both`. +largest response allowed, and no single download breaks one. `SWWAF_BYTES_COUNT` +says whether request bytes, response bytes or both are counted. ### Size and time limits @@ -369,25 +382,12 @@ through unchanged, so a limit on each side would bound the same bytes, and the lower one would always decide. Bodies stream straight through, so a request body reaches the app while the client is still sending it. -- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to - send its request line and headers, and then, from the end of the headers, its - body. -- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request - line and headers. It must be more than `4K`. Go's HTTP server always has such - a limit, and reads 4 KiB past the one it is given before it refuses. -- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection - may wait for its next request before `smallwebwaf` closes it. It is longer - than the 90 seconds after which traefik, by default, closes an unused - connection. So traefik closes first, and never sends a request on a connection - `smallwebwaf` is closing. -- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long one response may - take to reach the client, from the end of the request to the last byte. -- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the - app and sending it the whole request may take. -- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take - to send one whole response, from the end of the request to the last byte. -- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. -- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. +- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` must be more than `4K` and cannot be + `off`. Go's HTTP server always has such a limit, and reads 4 KiB past the one + it is given before it refuses. +- `SWWAF_CLIENT_IDLE_TIMEOUT` is longer by default than the 90 seconds after + which traefik, by default, closes an unused connection. So traefik closes + first, and never sends a request on a connection `smallwebwaf` is closing. What the client gets when a limit is passed: @@ -416,71 +416,56 @@ What the client gets when a limit is passed: ### Lookup of AS number and country On by default through GeoJS, which needs no account or file, so that country -lookup works with no setup. A local database file is the alternative for a -service that keeps its visitors' addresses on its own host. +lookup works with no setup. A local database file, the free IPinfo Lite +database, is the alternative for a service that keeps its visitors' addresses on +its own host; the operator downloads and mounts it, and `smallwebwaf` never +fetches it and holds no account or token for it. `docs/lookup.md` says what each +source sends where, how long answers are kept, and when a request waits. -- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. Only one - source is used. The start stops, naming both settings, when `file` is set - without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other - source. The start also stops when the source is `off` and one of these needs - lookups: - - a country list; - - a biased threshold; - - `SWWAF_ADD_LOOKUP_HEADERS`; - - an anomaly threshold per AS number. -- `SWWAF_LOOKUP_DB_PATH`: the database the `file` source reads: the IPinfo Lite - database (`ipinfo_lite.mmdb`), whose `asn`, `as_name` and `country_code` - fields are read. The operator downloads it with a free IPinfo account, mounts - the directory that holds it read-only (see "Rule files" for why), and - refreshes it. `smallwebwaf` never fetches it. A file that is missing or - unreadable at start stops the start. A replaced file is read again; a - replacement that cannot be read is logged, sent as one `file_error` alert, and - ignored, the file in use staying in use. -- `geojs` asks the free GeoJS web service about up to 200 addresses in one - request (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a - time, and reads each answer's `country_code`, `asn` and `organization_name`. - An answer without a `country_code`, or with the `asn` `64512`, counts as - unknown. GeoJS is told the address of every new visitor, whether or not a - setting uses the answer, since the request log, the metrics and the ban notes - carry it. Private, loopback and link-local addresses, which no source can - place, are never sent. +- Only one source is used. The start stops, naming both settings, when `file` is + set without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other + source. The start also stops when the source is `off` and a setting needs + lookups: a country list, a biased threshold, `SWWAF_ADD_LOOKUP_HEADERS` or an + anomaly threshold per AS number. +- GeoJS is told the address of every new visitor, whether or not a setting uses + the answer, since the request log, the metrics and the ban notes carry it. + Private, loopback and link-local addresses, which no source can place, are + never sent. Addresses are asked about in batches, one request at a time, and + answers are kept for days, which keeps the number of requests to GeoJS low. + Answers are kept apart from the table of clients, so that an answer outlasts a + client dropped from that table. - A client the lookup cannot place has an unknown country and AS number. The exclusive country list refuses it, and the biased thresholds give it `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it differently. -- Each GeoJS answer is kept for 7 days, in memory and in `lookups.json`, - separately from the table of clients, so that it outlasts a client dropped - from that table. Up to 100,000 answers are kept, about 15 MiB; past that, the - answer used longest ago goes first. -- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a visitor's request waits for - its client's first answer when a setting needs it. It is also how long a - request to GeoJS may take. A client whose answer has not come in time counts - as unknown until it comes, and its later requests do not wait. +- `SWWAF_LOOKUP_TIMEOUT` bounds both how long a visitor's request waits for its + client's first answer, when a setting needs it, and how long a request to + GeoJS may take. A client whose answer has not come in time counts as unknown + until it comes, and its later requests do not wait. - GeoJS publishes no rate limit, but its terms forbid "an excessive amount of - API requests", judged by GeoJS alone, and let it block a caller. Kept answers - and batched requests keep the number of requests low. While GeoJS is slow, - down or refusing `smallwebwaf`, new clients count as unknown, `smallwebwaf` - keeps asking with backoff, and one `source_failure` alert is sent per - cooldown. A service that cannot accept this, or that every new visitor's - address goes to a third party, uses the file. -- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): pass `X-Client-ASN` and - `X-Client-Country` to the app. + API requests", judged by GeoJS alone, and let it block a caller. While GeoJS + is slow, down or refusing `smallwebwaf`, new clients count as unknown, + `smallwebwaf` keeps asking with backoff, and one `source_failure` alert is + sent per cooldown. A service that cannot accept this, or that every new + visitor's address goes to a third party, uses the file. +- A replaced database file is read again; one that cannot be read is logged, + sent as one `file_error` alert, and ignored, the file in use staying in use. + The directory that holds the file is mounted, not the file itself (see "Rule + files" for why). ### Country lists Both are empty by default: the defaults judge a client by what it does, not by where it comes from. Clients in `SWWAF_ALLOW_NETS` are not checked. -- `SWWAF_DENIED_COUNTRIES`: for example `cn,ru,kp,ir,ua,by`. Every request from - a listed country is refused. -- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`: for example `us,de`. Only clients - placed in a listed country get through. Every request from any other country - is refused. So is every request from a client the lookup cannot place (see - "Lookup of AS number and country"), such as internal monitoring; those that - should reach the app go in `SWWAF_ALLOW_NETS`. A list that let unplaced - clients through would let every new client in whenever GeoJS stops answering. - The cost: while GeoJS fails, this list refuses every new client. No other - failure of GeoJS, a reputation source or an alert destination comes this close - to taking the service down. +- `SWWAF_DENIED_COUNTRIES` refuses every request from a listed country. +- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` lets through only clients placed in a + listed country. Every request from any other country is refused. So is every + request from a client the lookup cannot place, such as internal monitoring; + those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let + unplaced clients through would let every new client in whenever GeoJS stops + answering. The cost: while GeoJS fails, this list refuses every new client. No + other failure of GeoJS, a reputation source or an alert destination comes this + close to taking the service down. - Both may be set. `SWWAF_DENIED_COUNTRIES` then adds nothing, since the exclusive list already refuses every other country. A code on both lists stops the start with a message naming it. @@ -495,20 +480,14 @@ The lists are empty by default, for the same reason as the country lists. The request log and the metrics show which AS numbers and countries a service's abuse comes from. -- `SWWAF_ASN_LIMIT_PERCENT`: for example `AS14061:50,AS16276:50,AS45102:25`. - Clients in a listed AS number get that percentage of every request and byte - limit. `0` is a zero allowance: the client's first request breaks a limit and - bans it. -- `SWWAF_COUNTRY_LIMIT_PERCENT`: the same form with country codes, for example - `CN:25,RU:50`. -- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT`: optional - overrides applied to the byte limits only, when the byte percentage should - differ from the request percentage. -- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): for a client the lookup cannot - place. -- `SWWAF_ASN_LIMIT_PERCENT_URL`: optional URL of a text file of `AS:percent` - lines, so one abuse-source list can be shared by every `smallwebwaf` in the - fleet. It is fetched and refreshed like the blocklists. +- Clients in a listed AS number or country get that percentage of every request + and byte limit (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`), or + of the byte limits alone (`SWWAF_ASN_BYTES_PERCENT`, + `SWWAF_COUNTRY_BYTES_PERCENT`). `0` is a zero allowance: the client's first + request breaks a limit and bans it. +- `SWWAF_ASN_LIMIT_PERCENT_URL` names a text file of `AS:percent` lines, so one + abuse-source list can be shared by every `smallwebwaf` in the fleet. It is + fetched and refreshed like the blocklists. - A client's limit percentage is the lowest of those that apply to it: its AS number's, its country's, or a reputation hit's `limit:`. It is 100 if none applies. Clients given a lower percentage are banned after fewer @@ -529,20 +508,15 @@ files saved in the web editor. The Core Rule Set takes the shell commands and code in them for attacks. The default gives up refusing an attack carried in a body; the rule files, the limits and the bans still apply to the client that sends it, and `SWWAF_WAF_BODY_LIMIT` switches body inspection on. +`docs/core-rule-set.md` says what the Core Rule Set reads and how it scores a +request. -- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of rule - files (see "Rule files"). -- `SWWAF_RULES_ENABLED` (default `true`): `false` skips rule files entirely. -- `SWWAF_WAF_MODE` (default `block`): `off`, `detect` (log and alert only), or - `block`. -- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`) and `SWWAF_WAF_ANOMALY_THRESHOLD` - (default `5`): the Core Rule Set's own two tuning values, at its own defaults. - Each rule that matches adds its score to the request's. A request whose score +- `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule + Set's own two tuning values, at its own defaults. A request whose score reaches the threshold is a match. With the threshold `off`, nothing is a match, and scores are still logged. -- `SWWAF_WAF_DISABLED_RULES` (default - `920340,920420,920440,920640,930130,930140`): rule ids to switch off when an - app trips a false positive. The default switches off six rules: +- `SWWAF_WAF_DISABLED_RULES` switches off rule ids when an app trips a false + positive. The default switches off six rules: - 920340, 920420 and 920640 refuse a request for the type of its body, when the type is missing or not on the Core Rule Set's short list; - 920440 refuses a request for its file extension, such as `.sh` or `.sql`; @@ -553,30 +527,27 @@ sends it, and `SWWAF_WAF_BODY_LIMIT` switches body inspection on. In front of a code forge these refuse git over HTTP, container image and package uploads, and views of ordinary files in a repository. At the site root, where no app serves such files, the default rule file bans the common - probes these rules caught (see "Rule files"). A list given replaces the + probes these rules caught (see "Rule files"). A list you give replaces the default, so name these six ids in it to keep them off. -- `SWWAF_WAF_EXEMPT_PATHS`: path prefixes not inspected. -- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size, - such as `128K`, switches body inspection on. The Core Rule Set then reads form - data and multipart bodies up to that size, streaming the rest of a longer one - on without holding it in memory. It reads JSON and XML bodies no larger than +- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. Form data + and multipart bodies are read up to the size, the rest streaming on without + being held in memory; JSON and XML bodies are read only when no larger than the size, since those cannot be read in part. Any other body is not read: the Core Rule Set would read it as form data, where binary content such as a git push trips rules written for text. Body inspection suits apps whose forms carry no code. `docs/gitea.md` records what it refuses in front of gitea. -- `SWWAF_TRAP_PATHS`: paths the app never serves and only scanners ask for, for - example `/wp-login.php,/xmlrpc.php` in front of gitea. A request for one is a - clear sign of attack. This is the short form of a `path` rule with the `ban` - action, for deployments that mount no rule files. -- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the refusals per client per - minute after a rule file match, a trap path, a Core Rule Set match, or a - missing or wrong token. More than this breaks a limit (see "Bans"). A client - trying one attack or token after another is refused many times a minute; a - person rarely more than a few times. Answers the app gives are not counted, - since they do not tell a scanner from an ordinary client. In front of gitea, - registry and git clients get `404` and `401` by design, hundreds of times a - minute. +- `SWWAF_TRAP_PATHS` names paths the app never serves and only scanners ask for, + such as `/wp-login.php` in front of gitea. A request for one is a clear sign + of attack. This is the short form of a `path` rule with the `ban` action, for + deployments that mount no rule files. +- `SWWAF_ERROR_BURST_THRESHOLD` bounds the refusals per client per minute after + a rule file match, a trap path, a Core Rule Set match, or a missing or wrong + token. More than this breaks a limit (see "Bans"). A client trying one attack + or token after another is refused many times a minute; a person rarely more + than a few times. Answers the app gives are not counted, since they do not + tell a scanner from an ordinary client. In front of gitea, registry and git + clients get `404` and `401` by design, hundreds of times a minute. - A Core Rule Set engine error on a request forwards the request, and is logged and counted. @@ -640,15 +611,11 @@ every app, and no setting undoes them: ### Ban response and scope -- `SWWAF_BAN_RESPONSE` (default `403`): `403`, `429`, or `close` to drop the - connection without an answer. `403` makes a ban easy to recognise when - debugging. Behind traefik, `close` does not leave the client unanswered: - traefik answers `502`, as it does whenever its backend drops a connection. -- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): widen to for example `24` to ban - the surrounding netblock. -- `SWWAF_ATTACK_BAN_DURATION`, `SWWAF_LIMIT_BAN_DURATION`, - `SWWAF_LIMIT_BAN_REPEAT_WINDOW` and `SWWAF_MAX_BAN_DURATION`: the ban lengths, - with their defaults, under "Bans". +`SWWAF_BAN_RESPONSE` is `403` by default, which makes a ban easy to recognise +when debugging; `429` and `close` are the alternatives. Behind traefik, `close` +does not leave the client unanswered: traefik answers `502`, as it does whenever +its backend drops a connection. `SWWAF_BAN_SCOPE_V4_PREFIX` widens an IPv4 ban +to the surrounding netblock. The four ban lengths are under "Bans". ### Reputation @@ -658,70 +625,54 @@ account or an engine of the operator's own. The list best suited to be on by default, Spamhaus DROP, must not be fetched more than once an hour, and Spamhaus may block an address that does. Each `smallwebwaf` fetches its own copy, so several apps on one host, sharing one address, would often fetch less than an -hour apart. +hour apart. `docs/reputation.md` says how each source is asked and what is done +with the answer. -- `SWWAF_BLOCKLIST_URLS`: text files of addresses and netblocks, one per line; - anything after a `;` or `#` on a line is ignored. For example the Spamhaus - DROP list, `https://www.spamhaus.org/drop/drop.txt`. Spamhaus's terms for it, - on its DROP page: +- Blocklists (`SWWAF_BLOCKLIST_URLS`) are text files of addresses and netblocks, + such as the Spamhaus DROP list. Spamhaus's terms for it, on its DROP page: - automated downloads must be at least one hour apart, and once a day is more than enough in most cases; - a product that uses it must credit The Spamhaus Project and keep the list's date and copyright lines with the data. -- `SWWAF_BLOCKLIST_REFRESH` (default `24h`, never less than `1h`): how often - each list is fetched again. The last good copy is kept whole, comment lines - included, on failure and across restarts. -- `SWWAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:` or `log`. - `deny` refuses every request from a listed address, which suits lists of - networks that send nothing legitimate, such as DROP. `limit:` gives - listed clients that percentage of every limit. It suits a list of addresses - shared with ordinary visitors, such as Tor exits. -- `SWWAF_DNSBL_ZONES`: for example `dnsbl.dronebl.org`. Zones meant for mail - (lists of residential ranges) will block ordinary visitors and should not be - used. Spamhaus zones need their keyed query service, given as a zone name - containing the key. -- `SWWAF_DNSBL_RESOLVER`: an optional resolver address, since public resolvers - are refused by several list operators. -- `SWWAF_ABUSEIPDB_KEY`, `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`) and - `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`; the free tier allows 1000 - checks a day). Only clients that have already committed one offence are - queried, so the budget is spent on suspects. -- `SWWAF_CROWDSEC_LAPI_URL` and `SWWAF_CROWDSEC_LAPI_KEY`: optional. If a - CrowdSec engine exists on the host, its decision list is pulled on a schedule. - It is kept like a fetched blocklist, and a client on it is banned (see - "Bans"). This is how a fleet-wide blocklist can arrive without `smallwebwaf` - depending on CrowdSec. -- `SWWAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:` or - `log`, for DNS blocklist and AbuseIPDB hits. Such verdicts are less certain - than a blocklist, so by default a listed client gets a quarter of every limit. -- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`) and `SWWAF_REPUTATION_TIMEOUT` - (default `2s`). -- Up to 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept. - Past that, the one fetched longest ago goes first. -- A source that is down or over quota gives no verdict. The service continues, - with one `source_failure` alert per cooldown. + + So `SWWAF_BLOCKLIST_REFRESH` is never less than `1h`, and the last good copy + is kept whole, comment lines included, on failure and across restarts. + +- `SWWAF_BLOCKLIST_ACTION` is `deny` by default, which refuses every request + from a listed address and suits lists of networks that send nothing + legitimate, such as DROP. `limit:` gives listed clients that + percentage of every limit, and suits a list of addresses shared with ordinary + visitors, such as Tor exits. `log` only notes the hit. +- DNS blocklists (`SWWAF_DNSBL_ZONES`) meant for mail list residential ranges + and would block ordinary visitors, so they should not be used. Spamhaus zones + need their keyed query service, given as a zone name containing the key. + `SWWAF_DNSBL_RESOLVER` names a resolver, since public resolvers are refused by + several list operators. +- AbuseIPDB is asked only about clients that have already committed one offence, + so the daily budget (`SWWAF_ABUSEIPDB_DAILY_BUDGET`, under the free tier's + 1000 checks a day) is spent on suspects. +- A CrowdSec engine on the host (`SWWAF_CROWDSEC_LAPI_URL`) has its decision + list pulled on a schedule. It is kept like a fetched blocklist, and a client + on it is banned (see "Bans"). This is how a fleet-wide blocklist can arrive + without `smallwebwaf` depending on CrowdSec. +- `SWWAF_REPUTATION_ACTION`, for DNS blocklist and AbuseIPDB hits, is `limit:25` + by default. Such verdicts are less certain than a blocklist, so a listed + client gets a quarter of every limit rather than a refusal. +- Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, with a fixed + bound on their number. A source that is down or over quota gives no verdict. + The service continues, with one `source_failure` alert per cooldown. ### Alerting -- `SWWAF_ALERT_WEBHOOK_URL`: a JSON POST; schema under "Alert webhook schema". - `SWWAF_ALERT_WEBHOOK_HEADERS`: optional `Name:value` pairs for authentication. -- `SWWAF_ALERT_SLACK_WEBHOOK_URL`: a Slack incoming webhook, sent a formatted - message. -- `SWWAF_ALERT_NTFY_URL` (the full topic URL) and `SWWAF_ALERT_NTFY_TOKEN`: - title, priority and tags set from the event type. -- `SWWAF_ALERT_EVENTS` (default - `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): - which event types are sent. `source_failure` is a reputation source or GeoJS - failing or refusing `smallwebwaf`. `file_error` is one of: - - a rule file or state file edited while running that does not parse; - - a replacement lookup database that cannot be read; - - a state file that cannot be written. -- `SWWAF_ALERT_COOLDOWN` (default `15m`): the same event is not repeated within - this time for the same netblock, source or file, or, for an anomaly, for the - same thing counted: one client, netblock, AS number or named netblock, or the - whole service. A count of suppressed repeats is included in the next one. -- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): beyond this, alerts are rolled into - one summary per hour, so a wide attack cannot flood the channel. +- Alerts go to a webhook as JSON (`docs/alerts.md` gives the object), and to + Slack and ntfy as formatted messages. `SWWAF_ALERT_EVENTS` chooses the event + types sent. +- `SWWAF_ALERT_COOLDOWN`: the same event is not repeated within this time for + the same netblock, source or file. For an anomaly, it is the same thing + counted that must match: one client, netblock, AS number or named netblock, or + the whole service. A count of suppressed repeats is included in the next one. +- `SWWAF_ALERT_MAX_PER_HOUR`: beyond this, alerts are rolled into one summary + per hour, so a wide attack cannot flood the channel. - A destination that is down is retried with backoff from a bounded queue, the oldest alert dropped first, and the drops counted in the metrics. @@ -734,24 +685,9 @@ which the metrics show. They are what reveals a widely distributed scraper, thousands of addresses at low rates each, which the per-client limits do not see; a low `SWWAF_ASN_LIMIT_PERCENT` for its AS number is the response. -- Per client: `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`, - `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`, - `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` and - `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR`. -- Per surrounding netblock (`SWWAF_ANOMALY_NET_V4_PREFIX` default `24`, - `SWWAF_ANOMALY_NET_V6_PREFIX` default `48`): - `SWWAF_ANOMALY_NET_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_NET_BYTES_PER_MINUTE` and `..._PER_HOUR`. -- Per AS number, which needs a lookup source: - `SWWAF_ANOMALY_ASN_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_ASN_BYTES_PER_MINUTE` and `..._PER_HOUR`. -- The whole service: `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` and `..._PER_HOUR`. -- Named netblocks: `SWWAF_WATCH_NETS`, for example - `office=203.0.113.0/24,scraper-x=198.51.100.0/22`, with - `SWWAF_WATCH_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_WATCH_BYTES_PER_MINUTE` and `..._PER_HOUR` applied to each named - netblock as a whole. +- Requests and bytes are counted per minute and per hour, for each client, each + surrounding netblock, each AS number, the whole service, and each named + netblock in `SWWAF_WATCH_NETS`. - Anomaly counting includes clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS`, since an exempt client misbehaving is worth knowing about. At most 20,000 counters are kept, in memory and in @@ -760,89 +696,37 @@ see; a low `SWWAF_ASN_LIMIT_PERCENT` for its AS number is the response. ## Rule files `smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR` and checks each -request against them, unless the client is in `SWWAF_ALLOW_NETS`. The files are -plain text meant to be edited by hand, so that a new scanning pattern seen in -the request log becomes a rule in one line. `smallwebwaf` watches the directory: -a file edited, added or removed there takes effect while it runs. +request against them, unless the client is in `SWWAF_ALLOW_NETS`. +`docs/rules.md` gives the format, the targets and actions, the errors, and the +default file. -- One rule per line, four fields separated by spaces or tabs. The fourth field - runs to the end of the line: - - ``` - - ``` - -- Blank lines and lines starting with `#` are ignored. -- `id`: a short name of letters, digits, `-` and `_`, unique across all files. - It appears in the request log, the metrics, the alerts and the ban notes. -- `target`: what the regex is matched against. - - `path`: the URL path without the query, both as received and once - percent-decoded, so an encoded probe cannot slip past. - - `query`: the raw query string. - - `uri`: the path and query together, both as received and once - percent-decoded. - - `method`, `host`, `user_agent`, `referer`. - - `header:`: any one request header but `Host` and - `Transfer-Encoding`, which Go's HTTP server takes out of every request. - - Request bodies are not available to rule files. Reading them is the Core - Rule Set's job, once `SWWAF_WAF_BODY_LIMIT` switches it on. -- `action`: - - `log`: note the match in the request log and do nothing else. - - `block`: refuse the request with `403`, without a ban (see "Bans"). - - `ban`: the request is a clear sign of attack (see "Bans"). Use it only for - requests no real visitor sends. Anchor a path at the site root (`^/`): a - file of the same name deeper in a site can be ordinary content, such as a - file in a gitea repository. A visitor banned by mistake is let back in by - lifting the ban. -- `regex`: Go regular expression syntax (RE2). It has no backreferences or - lookaround. In exchange matching time is linear in the input, so no rule can - be made to stall the proxy. `(?i)` at the front makes a rule ignore case. A - rule matches if the regex matches anywhere in the target; anchor with `^` and - `$` when that is not wanted. -- Files are read in name order (`00-default.rules` before `50-gitea.rules`), - rules in line order. -- Rules are compiled when their file is read and held in memory. At start, a - line that does not parse, a regex that does not compile, or a duplicate id - stops the process with a message naming the file and line. So does a - `SWWAF_RULES_DIR` that does not exist. While running, the same faults leave - the rules as they were, and the log and one `file_error` alert name the file - and line. Once the file is fixed, it is read again. An empty directory is not - an error: the log says that no rules were loaded. To run without rule files, - mount an empty directory or set `SWWAF_RULES_ENABLED=false`. -- The image ships a default file in `SWWAF_RULES_DIR`. The app's Dockerfile can - copy files of its own into it, beside the default. Mounting a directory over - it replaces the defaults; mounting single files into it adds to them. Many - editors save a file by replacing it, and a container keeps seeing the old one - when the file alone is mounted. So rules meant to be edited while - `smallwebwaf` runs belong in a mounted directory. The lookup database is - mounted the same way, for the same reason. - -The default file the image ships: - -``` -# 00-default.rules: probes no real visitor sends, anchored at the site root - -# id target action regex -env-file path ban (?i)^/\.env(\.[a-z]+)?$ -vcs-dir path ban (?i)^/\.(git|svn|hg|bzr)(/|$) -secrets-dir path ban (?i)^/\.(aws|ssh|docker|kube)/ -secret-file path ban (?i)^/\.(htpasswd|htaccess|npmrc|netrc|pgpass|git-credentials|bash_history|DS_Store)$ -editor-dir path ban (?i)^/\.(vscode|idea)/ -backup-file path ban (?i)^/[^/]+\.(php(\.[a-z0-9]+|~)|sql(\.[a-z0-9]+)?)$ -log-file path ban (?i)^/(debug|error|access)\.log$ -compose-file path ban (?i)^/(docker-)?compose\.ya?ml$ -php-shell path ban (?i)^/(shell|c99|r57|wso|alfa)\.php$ -scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|masscan|zgrab|wpscan)\b -path-traversal uri block (\.\./){2,} -empty-agent user_agent log ^$ -``` - -It is kept short and limited to patterns that are wrong for every app. In front -of gitea, `/.env` is a probe, while -`///src/branch/main/.env.example` is a file in a repository that -any visitor may open. Anything app-specific, such as a rule banning WordPress -probes in front of gitea, belongs in a file the app's Dockerfile adds or the -deployer mounts. +- The files are plain text meant to be edited by hand, so that a new scanning + pattern seen in the request log becomes a rule in one line. `smallwebwaf` + watches the directory: a file edited, added or removed there takes effect + while it runs. +- A rule's regex is Go's RE2 syntax. It has no backreferences or lookaround; in + exchange matching time is linear in the input, so no rule can be made to stall + the proxy. +- `path` and `uri` are matched both as received and once percent-decoded, so an + encoded probe cannot slip past. Request bodies are not available to rule + files; reading them is the Core Rule Set's job. +- `block` refuses the request without a ban; `ban` makes the request a clear + sign of attack (see "Bans"). `ban` is for requests no real visitor sends, + anchored at the site root. A file of the same name deeper in a site can be + ordinary content, such as a file in a gitea repository. A visitor banned by + mistake is let back in by lifting the ban. +- At start, a file that does not parse, a regex that does not compile, or a + duplicate id stops the process with a message naming the file and line. So + does a `SWWAF_RULES_DIR` that does not exist. While running, the same faults + leave the rules as they were, and the log and one `file_error` alert name the + file and line. An empty directory is not an error. +- The image ships a default file, kept short and limited to patterns that are + wrong for every app, with its path rules anchored at the site root. Anything + app-specific belongs in a file the app's Dockerfile adds or the deployer + mounts. Rules meant to be edited while `smallwebwaf` runs belong in a mounted + directory, not a mounted file. Many editors save a file by replacing it, and a + container keeps seeing the old one when the file alone is mounted. The lookup + database is mounted the same way, for the same reason. ## Persistent state @@ -850,160 +734,61 @@ All state lives in memory, and the files in `SWWAF_STATE_DIR` hold a copy of all of it, so an orderly stop loses nothing. The one exception is log lines still waiting to be sent to `SWWAF_LOG_REMOTE_URL`, which stdout has already carried. The files are meant for people as well: an admin can read or edit them at any -time, and the running `smallwebwaf` takes the edit in. +time, and the running `smallwebwaf` takes the edit in. `docs/state-files.md` +lists the files, what each holds, their sizes, and how an edit is taken in. -The files, each holding one kind of state: - -- `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf` - made and every ban an admin made. Per entry: - - the netblock, start and expiry (`null` for permanent); - - the cause: `attack`, `limit`, `admin` or `crowdsec`; - - a short reason; - - when it was lifted, if an admin lifted it; - - the notes below. -- `clients.json`: per client, its minute, hour and day counters and its history - since it was first seen. The history holds: - - the first and last time seen; - - the AS number, AS name and country last looked up, and when; - - total requests, and how many were forwarded and how many refused; - - bytes in each direction; - - responses by status class; - - offences by kind. - - Clients that were never banned are kept too, so every client's history - survives a restart. - -- `lookups.json`: GeoJS answers, one per client: the AS number, AS name and - country, when GeoJS was asked and when the answer was last used. -- `reputation.json`: - - the last good copy of each list fetched from a URL; - - the cached DNS blocklist verdicts and AbuseIPDB scores, each with the time - it was fetched; - - the AbuseIPDB checks spent today, so that a restart does not reset the - daily budget. -- `alerts.json`: - - the anomaly counters; - - for each event and netblock, source, file or thing counted, when it was - last sent, and the count of repeats held back since; - - the alerts sent this hour; - - the alerts still waiting to be sent. - -Ban notes hold what an admin needs to decide whether to lift a ban, drawn from -what `smallwebwaf` already knows. Nothing is looked up to fill them: - -- the AS number, AS name and country, added when the lookup answers (empty when - lookups are off); -- what was broken: the rule id and target that matched, or the trap path, or the - limit, its window and the count reached; -- the client's limit percentage, with the setting that gave it; -- the reputation sources that listed the client; -- the request that caused the ban: time, method, host, path with its query - string, status and user agent, each text cut to 256 bytes; -- the netblock's total requests since it was first seen, and the requests - refused under this ban so far, kept up to date while the ban lasts; -- how many earlier bans of each kind the netblock has had. - -Each file is rewritten whole, so each has a bound. `clients.json` takes about 1 -KiB per client, about 20 MiB at the default `SWWAF_MAX_TRACKED_CLIENTS`. Written -every 15 minutes, that is under 2 GiB of disk writes a day. `bans.json` takes -about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at the default -`SWWAF_MAX_BANS`. With a full file, a hundred new bans a day come to about 600 -MiB of writes. `lookups.json` takes about 150 bytes per answer, under 1.5 GiB of -writes a day when full. The other two files are usually a few MiB or less. -Writing a file of these sizes takes well under a second, in the background, so -rewriting whole files needs nothing cleverer. - -Format, so that a person can open `bans.json` in an editor, find an address, and -remove or add an entry: - -- indented JSON with a top-level `version` number; -- entries sorted by client address; -- times in RFC 3339 UTC; -- durations and sizes as plain numbers, with the unit in the field name; -- `clients.json` and `lookups.json` put each entry on one line instead. That - halves their size and lets `grep` show everything about one client. - -Writing: - -- serialise from a snapshot taken under the lock, so requests are not held up - while the file is written; -- write to a temporary file in the same directory, sync it, rename it over the - real name, sync the directory. A crash at any point leaves either the old - complete file or the new complete file, never a partial one; -- changes that only update the counts in a ban's notes wait for the write every - `SWWAF_STATE_COUNTER_INTERVAL`, as the other four files do. All five are - written on orderly shutdown (`SIGTERM`); -- before writing a file, `smallwebwaf` checks whether it changed on disk since - it last read or wrote it. If it did, `smallwebwaf` takes that edit in first, - so an admin's edit is never overwritten. - -Reading at start: - -- Active bans go into an in-memory prefix lookup; everything else into maps. - Counter buckets and cache entries whose time has passed are discarded. -- A missing file means empty state, and is normal on first run. -- A file that does not parse, or has an unknown `version`, stops the process - with a message naming the file and position. Starting with empty state would - silently forgive every repeat offender. Since writes are atomic, a broken file - can only come from a hand edit, which the editor should hear about. -- `SWWAF_STATE_DIR` not writable at start: the process exits. - -Edits while running: - -- `smallwebwaf` watches `SWWAF_STATE_DIR` and notices a file that is edited, - replaced or added. It tells its own writes from an admin's by comparing the - file with what it last wrote. -- An edit that parses is taken in at once: what the file says replaces what - `smallwebwaf` held for that file. Removing a ban's entry from `bans.json` - lifts the ban, an admin's own included; adding an entry bans. Changes - `smallwebwaf` made after the admin opened the file, such as a new ban, are - lost when the admin saves over them. Most editors warn when a file changed on - disk while it was open. -- An edit that does not parse does not stop the running `smallwebwaf`. It keeps - the state it has and renames the edited file to `.bad` (for example - `bans.json.bad`), so the edit is kept. It writes the file again from memory, - logs the file and the position of the error, and sends them in one - `file_error` alert. The admin fixes the `.bad` file and moves it back. - -A write that fails while running: - -- state stays correct in memory; -- the failure is logged and counted in the metrics; -- one `file_error` alert is sent per cooldown; -- the write is retried at the next write. - -What a hard kill can lose: - -- ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds); -- everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes): - - counts and history, and the counts in ban notes; - - GeoJS answers; - - reputation verdicts and the AbuseIPDB count; - - anomaly counters and alert cooldowns. - -Losing the volume loses ban history, not service. +- Each file holds one kind of state, as JSON with: + - a top-level `version` number; + - entries sorted by client address; + - times in RFC 3339 UTC; + - durations and sizes as plain numbers, with the unit in the field name. +- `bans.json` is indented, so that a person can open it in an editor, find an + address, and remove or add an entry. `clients.json` and `lookups.json` put + each entry on one line, which halves their size and lets `grep` show + everything about one client. +- Clients that were never banned are kept too, so every client's history + survives a restart. +- Each file is rewritten whole, so each has a bound given by its setting. + Writing a file of these sizes takes well under a second, in the background, so + rewriting whole files needs nothing cleverer. +- A write serialises from a snapshot taken under the lock, so requests are not + held up. It goes to a temporary file in the same directory, which is synced + and renamed over the real name. A crash at any point leaves either the old + complete file or the new complete file, never a partial one. Before writing a + file, `smallwebwaf` takes in any edit made to it on disk, so an admin's edit + is never overwritten. +- At start, a file that does not parse, or has an unknown `version`, stops the + process with a message naming the file and position. Starting with empty state + would silently forgive every repeat offender. Since writes are atomic, a + broken file can only come from a hand edit, which the editor should hear + about. A missing file means empty state, and is normal on first run. +- While running, an edit that parses replaces what `smallwebwaf` held for that + file. An edit that does not parse never stops the running `smallwebwaf`. It + keeps the state it has and sets the edited file aside as `.bad`. It + writes the file again from memory, and names the file and the position of the + error in the log and in one `file_error` alert. +- A write that fails leaves the state correct in memory. It is logged, counted + in the metrics and sent as one `file_error` alert per cooldown, and retried at + the next write. +- A hard kill loses the changes since the last write. Losing the volume loses + ban history, not service. ## Request log Every request gets one JSON object on one line on stdout, refused ones included, apart from those Go's HTTP server ends before `smallwebwaf` sees them (see "Size -and time limits"). `docs/request-log.md` lists the fields. Bodies are never -logged. Query strings are logged as received; an app that carries secrets in -query strings needs that fixed in the app. The process's own messages share the -stream as JSON lines with `"type":"process"`; request lines carry -`"type":"request"`. +and time limits"). `docs/request-log.md` lists the fields and describes the +syslog sending. Bodies are never logged. Query strings are logged as received; +an app that carries secrets in query strings needs that fixed in the app. The +process's own messages share the stream as JSON lines with `"type":"process"`; +request lines carry `"type":"request"`. -When `SWWAF_LOG_REMOTE_URL` is set, the same lines are also sent there, so a -deployment can stop depending on docker's log handling while `docker logs` keeps -working: - -- each line is the message of an RFC 5424 syslog record, with octet-counted - framing on TCP and TLS; -- sending happens on its own goroutine from a bounded buffer - (`SWWAF_LOG_REMOTE_BUFFER`). An unreachable or slow server never delays a - request and never stops stdout. The sender reconnects with backoff, drops the - oldest lines when the buffer is full, and counts the drops in the metrics. UDP - gives no delivery signal at all and is offered only for compatibility. +When `SWWAF_LOG_REMOTE_URL` is set, the same lines are also sent there as RFC +5424 syslog records. So a deployment can stop depending on docker's log handling +while `docker logs` keeps working. Sending happens on its own goroutine from a +bounded buffer: an unreachable or slow server never delays a request and never +stops stdout. UDP gives no delivery signal at all and is offered only for +compatibility. ## Metrics endpoint @@ -1017,24 +802,17 @@ the request log and `GET /_smallwebwaf/clients/`. A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself: it answers it and never passes it to the app. The prefix carries the tool's name, so it takes no path an app uses. Admins and scrapers reach these endpoints -through traefik, like any other request. +through traefik, like any other request. `docs/admin-endpoints.md` lists the +endpoints and what each answers. -- `GET /_smallwebwaf/healthz`: answers `200` with `ok` to anyone, without a +- `GET /_smallwebwaf/healthz` answers `200` with `ok` to anyone, without a token, while `smallwebwaf` is running. It does not ask the app; the container's health check does. It is answered before any check, so a health checker never needs an exemption and is never refused, for example by `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`. -- `GET /_smallwebwaf/metrics`: the metrics; needs `SWWAF_METRICS_TOKEN`. -- `GET /_smallwebwaf/bans`, `POST /_smallwebwaf/bans` (a client or netblock, a - duration, a reason) and `DELETE /_smallwebwaf/bans/`: need - `SWWAF_ADMIN_TOKEN`. -- `GET /_smallwebwaf/clients/`: needs `SWWAF_ADMIN_TOKEN`. It answers with - the client's current counters, its history (with its lookup result and - offences), and its bans with their notes. It answers "why was this address - refused" and "what has this netblock been doing". -- A token is sent as `Authorization: Bearer ` (see "Tokens" under - "Settings"). While a token is unset, the endpoints that need it answer `404`, - as does any other path under the prefix. +- Every other endpoint needs its token, sent as `Authorization: Bearer ` + (see "Tokens" under "Settings"). While a token is unset, the endpoints that + need it answer `404`, as does any other path under the prefix. - Apart from the health check, these requests go through every check that any other request goes through. They are answered at the point where another request would be forwarded to the app: a banned or refused client stays @@ -1044,20 +822,6 @@ through traefik, like any other request. `SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute. A client in `SWWAF_ALLOW_NETS` skips the checks but still needs the token. -## Alert webhook schema - -One JSON object per alert: - -- `instance`, `time`, `event` (one of the `SWWAF_ALERT_EVENTS` values, or - `summary` for the hourly summary); -- `client`, `netblock`, `asn`, `as_name`, `country`; -- `reason`: a short human-readable sentence; -- `detail`: event-specific fields, for example `window`, `count`, `limit`, - `limit_percent`, `rule_ids`, `path`, `ban_expires`, for a ban its notes, and - for `file_error` the file and the position of the error; -- `suppressed_repeats`: the number of identical alerts held back by the - cooldown. - ## Deployment `smallwebwaf` runs inside the app's own container: the app's Dockerfile builds diff --git a/docs/admin-endpoints.md b/docs/admin-endpoints.md new file mode 100644 index 0000000..85563f0 --- /dev/null +++ b/docs/admin-endpoints.md @@ -0,0 +1,35 @@ +# Admin endpoints + +`smallwebwaf` answers these requests itself, on the app's own address and +through traefik like any other request. Apart from the health check, each needs +a token, sent as `Authorization: Bearer `, and goes through every check +any other request goes through. It is answered where another request would be +passed to the app: a banned client stays refused, and each request counts toward +rate limiting. A missing or wrong token is answered `401`, in `observe` mode +too. Such a refusal counts toward the error burst, so a client guessing tokens +is soon banned. While a token is unset, its endpoints answer `404`. + +- `GET /_smallwebwaf/healthz`: `200` and `ok`, to anyone, before any check and + without asking the app. +- `GET /_smallwebwaf/metrics`: the metrics, for `SWWAF_METRICS_TOKEN` (see + [`metrics.md`](metrics.md)). +- `GET /_smallwebwaf/bans`: every ban held, under `bans`, each as `bans.json` + lists it. Needs `SWWAF_ADMIN_TOKEN`, as do the three below. +- `POST /_smallwebwaf/bans`: bans a netblock. The body is a JSON object of + `netblock`, `duration` and, if you like, `reason`. `netblock` is a netblock + such as `203.0.113.0/24`, or a client's address. A client's address bans its + IPv4 netblock (`SWWAF_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a + duration such as `1h` or `7d`, or `permanent`. The ban starts at once, with + the cause `admin`. A bad body is answered `400`, saying what is wrong: + - one that is not such an object, or has another field; + - one longer than 4 KiB; + - one that names an IPv4-mapped netblock, or an address with a zone. +- `DELETE /_smallwebwaf/bans/`: lifts every active ban covering the + address ``. It answers `404` when none is active. +- `GET /_smallwebwaf/clients/`: what `smallwebwaf` knows of the client. + Under `client`, its counters and history as `clients.json` holds them, or + `null` when it has never been seen. Under `bans`, every ban covering the + address, with its notes. + +An admin whose own address is banned lifts that ban by editing `bans.json` (see +[`bans.md`](bans.md)). diff --git a/docs/alerts.md b/docs/alerts.md new file mode 100644 index 0000000..6577753 --- /dev/null +++ b/docs/alerts.md @@ -0,0 +1,149 @@ +# Alerts + +Each alert goes to each destination you name. `SWWAF_ALERT_WEBHOOK_URL` gets it +as one JSON object. `SWWAF_ALERT_SLACK_WEBHOOK_URL` and `SWWAF_ALERT_NTFY_URL` +get a message made from it. The settings are under "Alert destinations" in +[`settings.md`](settings.md). An alert is one of these events, and is sent when +its event is in `SWWAF_ALERT_EVENTS`: + +- `ban`: a ban `smallwebwaf` made. +- `permanent_ban`: a permanent ban it made, or an existing ban made permanent by + a request. +- `waf_block`: the Core Rule Set scored a request at or over the threshold. In + `block` mode the request was refused. In `detect` mode it was not, and the + alert's `detail` gives `mode`. +- `anomaly`: a count over an anomaly threshold. +- `reputation_hit`: a request from a client on a blocklist, the CrowdSec + decision list or a DNS blocklist, or with an AbuseIPDB score that is a hit. + One alert per source. +- `source_failure`: one of: + - GeoJS failing; + - a list that cannot be fetched; + - a DNS blocklist or AbuseIPDB failing or refusing a query; + - the day's AbuseIPDB checks used up. +- `file_error`: one of: + - a rule file or state file edit with an error; + - a state file that cannot be written; + - a replacement lookup database that cannot be read. +- `summary`: the hourly summary described under "Repeats and the hourly cap" + below. + +The bans you make raise no alert. In `observe` mode, a request that would have +made a ban still raises the ban's alert, with `mode` set to `observe` in its +`detail`. + +## The JSON object + +This is the alert for a ban for a broken rate limit, shown indented; it is sent +on one line: + +```json +{ + "instance": "fsn1app1/gitea", + "time": "2026-10-06T12:00:00.123461Z", + "event": "ban", + "client": "203.0.113.9", + "netblock": "203.0.113.9/32", + "asn": "", + "as_name": "", + "country": "", + "reason": "requests per minute over the limit of 1000", + "detail": { + "ban_expires": "2026-10-06T13:00:00.123Z", + "cause": "limit", + "notes": { + "asn": "", + "as_name": "", + "country": "", + "kind": "requests", + "limit": 1000, + "window": "minute", + "count": 1001, + "request": { + "time": "2026-10-06T12:00:00.123456789Z", + "method": "GET", + "host": "app.example", + "path": "/owner/repo/commits/branch/main?page=812", + "status": 403, + "user_agent": "scraper/1.0" + }, + "requests": 5210, + "refused": 0, + "earlier_bans": { + "limit": 0, + "attack": 0, + "admin": 0, + "crowdsec": 0 + } + } + }, + "suppressed_repeats": 0 +} +``` + +- `instance` is `SWWAF_INSTANCE_NAME`. `time` is when the alert was raised. + `event` is one of the events above. +- `client` is the address whose request raised the alert. `netblock` is the + ban's netblock, the netblock an anomaly counted, or the client's group. Both + are empty for `source_failure` and `file_error`. `asn`, `as_name` and + `country` are the client's as known when the alert was raised. They are empty + when GeoJS had not answered yet. +- `reason` is a short sentence, such as + `requests per minute over the limit of 1000` or `listed by a blocklist`. +- `detail` is what is particular to the event: + - for a ban, its `cause`, `ban_expires` and `notes` (see + [`bans.md`](bans.md) for the notes); + - for an anomaly, the `scope`, `window`, `kind`, `count` and `threshold`; + - for a reputation hit, the `source` and, for AbuseIPDB, the `score`; + - for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`; + - for a source failure, the `source` and `error`; + - for a file error, the `file` and `error`. +- `suppressed_repeats` is how many repeats the cooldown held back before this + alert. + +## Slack and ntfy + +Slack and ntfy get a message with a title and a text. The title is the instance +and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then one +line for each of these that the alert has: + +- the client, the netblock and the country; +- the file, the source, the error and the mode from `detail`; +- the suppressed repeats, if any. + +Slack gets the title in bold and the text below it, with `&`, `<` and `>` +escaped. ntfy gets the text as the message, with these headers: + +- `Title`: the title; +- `Authorization`: the token, as `Bearer `, while + `SWWAF_ALERT_NTFY_TOKEN` is set; +- `Priority` and `Tags`: by event, as listed below. + +The priority and tag by event: + +- `ban`: priority `default`, tag `no_entry`. +- `permanent_ban`: priority `high`, tag `no_entry`. +- `waf_block`: priority `default`, tag `shield`. +- `anomaly`: priority `high`, tag `chart_with_upwards_trend`. +- `reputation_hit`: priority `low`, tag `label`. +- `source_failure` and `file_error`: priority `high`, tag `warning`. +- `summary`: priority `default`, tag `bar_chart`. + +## Repeats and the hourly cap + +A repeat is an alert for the same event on the same netblock, source, file or +thing counted, less than `SWWAF_ALERT_COOLDOWN` after the last. It is held back +and counted, and the next alert sent gives the count. Past +`SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and counted +by event. Once the hour ends, one alert with the event `summary` gives those +counts. It also counts the repeats that a cooldown held back and that no later +alert reported. + +## Sending + +Each destination has a queue of at most 1000 alerts, sent one at a time, so a +slow destination holds up nothing. A `2xx` answer takes an alert. A `4xx` other +than `408` and `429` refuses it, and it is dropped. Anything else is a failure: +the alert is sent again a second later, and twice as long after each further +failure, up to a minute. `alerts.json` keeps the queues and cooldowns across a +restart (see [`state-files.md`](state-files.md)). diff --git a/docs/bans.md b/docs/bans.md new file mode 100644 index 0000000..bd0f4e6 --- /dev/null +++ b/docs/bans.md @@ -0,0 +1,84 @@ +# Bans + +How bans are made, what they cover and how long they last is under "Bans" in +[`SPEC.md`](../SPEC.md); the settings are under "How long bans last and what +they cover" in [`settings.md`](settings.md). This file says what a ban's notes +hold, and how to make, keep or lift a ban. + +## The notes + +Each ban carries notes for deciding whether to lift it, drawn from what +`smallwebwaf` already knows. Nothing is looked up to fill them: + +- the AS number, AS name and country, added when the lookup answers (empty when + lookups are off); +- what was broken: the rule id and target that matched, or the trap path, or the + limit, its window and the count reached; +- the client's percentage of the limit, with the setting that gave it; +- the reputation sources the client is on; +- the request that caused the ban: time, method, host, path with its query + string, status and user agent, each text cut to 256 bytes; +- the netblock's requests since it was first seen, and how many this ban has + refused so far, kept up to date while the ban lasts; +- how many earlier bans the netblock had, by cause. + +## Managing bans by editing `bans.json` + +`bans.json` in the state directory holds every ban, indented to be read. A +permanent ban's `expires` is `null`. `smallwebwaf` watches the file and takes in +an edit as soon as it is saved (see [`state-files.md`](state-files.md)). + +To ban a netblock, add an entry with its `netblock`, `start` and `expires`. +`expires` is `null` for a ban that never ends. `reason`, `notes` and `cause` may +be left out; the cause is then `admin`: + +```json +{ + "version": 1, + "bans": [ + { + "netblock": "203.0.113.0/24", + "start": "2026-10-06T12:00:00Z", + "expires": null, + "reason": "probes for logins" + } + ] +} +``` + +To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause` +to `admin`. To lift a ban, add `"lifted"` with the time you lift it, such as +`"lifted": "2026-10-06T13:00:00Z"`. The ban then refuses nothing and does not +make the netblock's next ban longer. It is kept with its notes. To forget a ban, +delete its entry. A ban whose cause is `attack` becomes permanent at the first +request it refuses. One whose cause is `admin` or `crowdsec` does not. + +An edit that cannot be read does not stop the running `smallwebwaf`. It keeps +what it holds. At the file's next write it renames your file to `bans.json.bad`, +writes the file again from memory, and logs and alerts the error. Mend the +`.bad` file and move it back. + +## Managing bans through the admin endpoints + +With `SWWAF_ADMIN_TOKEN` set, and the token in `$TOKEN`: + +```sh +# Every ban. +curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans + +# Ban 203.0.113.0/24 for seven days. +curl -H "Authorization: Bearer $TOKEN" \ + --json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \ + https://app.example/_smallwebwaf/bans + +# Lift the bans on 203.0.113.9. +curl -H "Authorization: Bearer $TOKEN" -X DELETE \ + https://app.example/_smallwebwaf/bans/203.0.113.9 + +# What smallwebwaf knows of 203.0.113.9. +curl -H "Authorization: Bearer $TOKEN" \ + https://app.example/_smallwebwaf/clients/203.0.113.9 +``` + +See [`admin-endpoints.md`](admin-endpoints.md) for what each answers. An admin +whose own address is banned lifts that ban by editing `bans.json`. diff --git a/docs/core-rule-set.md b/docs/core-rule-set.md new file mode 100644 index 0000000..066e749 --- /dev/null +++ b/docs/core-rule-set.md @@ -0,0 +1,31 @@ +# The Core Rule Set + +The OWASP Core Rule Set 4.25.0 runs on each request after the rule files, unless +`SWWAF_WAF_MODE` is `off`. It reads the method, the URL with its query, and the +headers. Once `SWWAF_WAF_BODY_LIMIT` is set, it also reads a body that is form +data, multipart, JSON or XML. No answer is inspected. The settings are under +"Attack detection" in [`settings.md`](settings.md). The same section of +[`SPEC.md`](../SPEC.md) says why bodies are not read by default, and gives the +six changes `smallwebwaf` makes to the Core Rule Set, with the reason for each. + +Each rule that matches adds to the request's score. A score at or over +`SWWAF_WAF_ANOMALY_THRESHOLD` is a match. A match is refused with `403` while +`SWWAF_WAF_MODE` is `block`, and let through while it is `detect`. Either way +the log line gives the rule ids and the score, and a `waf_block` alert is sent. +A request with more than 1000 query parameters or body fields adds 5 to its +score, since Coraza reads only the first 1000. The Core Rule Set inspects only a +request that every earlier check let through. It skips a path under +`SWWAF_WAF_EXEMPT_PATHS` and a client in `SWWAF_ALLOW_NETS`. An engine error on +a request forwards the request, and is logged and counted. + +Body inspection suits apps whose forms carry no code. In front of a code forge +it refuses issue and comment text, wiki pages and files saved in the web editor +that hold shell commands or code. The log line names the rule ids, which +`SWWAF_WAF_DISABLED_RULES` can switch off. A body the Core Rule Set cannot parse +adds 5 to the score. So does a multipart body cut by the limit in the middle of +a part's header. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to +send the part that is read. Any other body, such as a git push, reaches the app +uninspected. + +[`gitea.md`](gitea.md) records what the Core Rule Set still refuses in front of +gitea, with and without body inspection. diff --git a/docs/lookup.md b/docs/lookup.md new file mode 100644 index 0000000..45909d0 --- /dev/null +++ b/docs/lookup.md @@ -0,0 +1,79 @@ +# GeoJS and the lookup database + +`smallwebwaf` looks up each client's AS number and country, through GeoJS by +default or in a database file on your own host. The settings are under "Looking +up the AS number and country" in [`settings.md`](settings.md); the reasons for +the defaults are under "Lookup of AS number and country" in +[`SPEC.md`](../SPEC.md). + +## GeoJS + +By default `smallwebwaf` asks GeoJS, a free web service that needs no account, +about every new visitor's address. It asks whether or not a setting uses the +answer, since the log line, the history, the ban notes and the metrics carry the +AS number and country. These addresses are never sent: + +- one in `SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`; +- one under a ban; +- a private, loopback or link-local address, which no source can place. + +For an IPv6 visitor, GeoJS is asked about the first address of its group. +Addresses are asked about together, up to 200 in one request +(`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a time. Each +answer's `country_code`, `asn` and `organization_name` are read. An answer +without a `country_code`, or with the `asn` `64512`, counts as unknown. Each +answer is kept for seven days, in memory and in `lookups.json`, separately from +the table of clients. Up to 100,000 answers are kept; past that, the answer used +longest ago goes first. + +A request waits for its client's first answer only when a setting needs the +answer before the request goes on. It waits up to `SWWAF_LOOKUP_TIMEOUT`. The +settings that make it wait: + +- a country list; +- `SWWAF_ADD_LOOKUP_HEADERS`; +- any setting under "Lower limits by AS number and country" in + [`settings.md`](settings.md). + +Otherwise the request goes on at once, logged without the answer. The answer +reaches the client's history and ban notes when it comes. A client whose answer +has not come in time counts as from an unknown country until it comes, and its +later requests do not wait. While GeoJS fails, new visitors count as from an +unknown country, which `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses. GeoJS is +then left alone for a second, and twice as long after each further failure, up +to five minutes, and one `source_failure` alert is sent per cooldown. + +## The database file + +To keep your visitors' addresses on your own host, set +`SWWAF_LOOKUP_SOURCE=off`, or set it to `file`. `file` looks every client up in +the free IPinfo Lite database (`ipinfo_lite.mmdb`) at `SWWAF_LOOKUP_DB_PATH`, +reading its `asn`, `as_name` and `country_code` fields. You download the +database with an IPinfo account and mount it into the container; `smallwebwaf` +never fetches it and holds no account or token for it. It answers at once, with +nothing sent off the host. A client whose address it does not hold counts as +from an unknown country. + +`smallwebwaf` reads the whole file at start. It reads the file again once it has +gone 2 seconds without a change after you replace it. A replacement it cannot +read is logged and alerted, and the file read before stays in use. Mount the +directory that holds the file, not the file itself. Many editors and download +tools save a file by replacing it, and a container keeps seeing the old file +when only the file is mounted. + +IPinfo updates the database daily, and releases it under the Creative Commons +Attribution-ShareAlike 4.0 licence. It asks for a link to IPinfo on the site +that uses it, such as "IP address data is powered by IPinfo". + +## A client without a country + +A client the lookup cannot place has an unknown country and AS number. That is +one of: + +- a client on a private address; +- one GeoJS or the database does not know; +- one whose GeoJS answer has not come in time. + +`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses such a client, and the lower +limits give it `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it +differently. diff --git a/docs/reputation.md b/docs/reputation.md new file mode 100644 index 0000000..80c6318 --- /dev/null +++ b/docs/reputation.md @@ -0,0 +1,113 @@ +# Reputation sources + +A reputation source judges a client by what others saw it do. None is on by +default. The settings are under "Reputation sources" in +[`settings.md`](settings.md); the reasons for the defaults are under +"Reputation" in [`SPEC.md`](../SPEC.md). The sources are checked in this order, +and a source the client is on refuses it, lowers its limits, or only notes it, +as `SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say. + +## Blocklists + +`SWWAF_BLOCKLIST_URLS` lists text files of addresses and netblocks, one to a +line, as the Spamhaus DROP list is written. Anything after a `;` or `#` is left +out. An IPv4-mapped address such as `::ffff:192.0.2.0/120` is read as the IPv4 +netblock it stands for. + +Each list is fetched again `SWWAF_BLOCKLIST_REFRESH` after its last try, one +list at a time. The file of `SWWAF_ASN_LIMIT_PERCENT_URL` is too. +`reputation.json` keeps when each was last tried, so a restart does not fetch a +list again before its time. A fetch fails when: + +- the server answers other than `200`; +- the fetch does not finish within a minute; +- the list is longer than 16 MiB; +- a line is not an address or netblock, or, for the AS number file, not + `AS:`. + +The copy fetched before stays in use, comment lines included. The failure is +logged and raised as a `source_failure` alert. The log, the alerts, the ban +notes and the metrics name each list by its URL, so keep a secret out of it. + +The Spamhaus DROP list is The Spamhaus Project's. Its terms, and what they mean +for `SWWAF_BLOCKLIST_REFRESH` and for several instances on one host, are under +"Reputation" in [`SPEC.md`](../SPEC.md). The copy in `reputation.json` keeps the +list's date and copyright lines. + +## CrowdSec + +While `SWWAF_CROWDSEC_LAPI_URL` is set to the local API of a CrowdSec engine you +run, `smallwebwaf` fetches its decision list every minute. It sends +`SWWAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key` header. The list is kept as a +blocklist is kept. The decisions of the type `ban` on an address or a netblock +are used. Any other, such as a captcha or a country, is left out. A decision +lasts for the `duration` it had left when the engine answered. + +A request is refused when a decision still in force names its client. The +client's netblock is banned until the decision ends, with the cause `crowdsec`. +Only a client that sends a request is banned, so a long list does not fill +`bans.json`. Such a ban is never made permanent, and does not lengthen the +netblock's next ban for a broken limit. If you lift a ban while the decision is +in force, the client's next request makes another. Delete the decision in +CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and lift +the ban once the list has been fetched again. A fetch fails when: + +- the engine answers other than `200`, a redirect included; +- the fetch does not finish within a minute; +- the answer is longer than 16 MiB; +- the answer is not a list of decisions. + +## DNS blocklists + +Each zone in `SWWAF_DNSBL_ZONES` is asked about a client's own address in the +background, the first time the client is seen. The request goes on at once, and +so does every request from the client until the zone has answered. The name +asked about is the one RFC 5782 gives: the numbers of an IPv4 address reversed, +or the 32 hex digits of an IPv6 address reversed. `192.0.2.99` is asked about as +`99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` lists the client, but +one in `127.255.255.0/24` is Spamhaus refusing the query. A name that does not +exist does not list the client. These are failures: + +- an answer in `127.255.255.0/24`; +- an answer outside `127.0.0.0/8`; +- a resolver error; +- no answer within `SWWAF_REPUTATION_TIMEOUT`. + +A failure is logged and alerted, and the zone is not asked again for a minute. +Each verdict is kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and in +`reputation.json`. Up to 100,000 verdicts are kept; past that, the one fetched +longest ago goes first. At most 1,000 queries are under way at once. + +The key of a Spamhaus zone under `dq.spamhaus.net` is its first label. +`smallwebwaf` shows `********` in its place wherever it names the zone. Only +`reputation.json` keeps the zone with its key. + +## AbuseIPDB + +While `SWWAF_ABUSEIPDB_KEY` is set, `smallwebwaf` asks AbuseIPDB's check +endpoint for the abuse confidence score of a client, from 0 to 100. Only a +client that has committed an offence is checked. An offence is one of: + +- a broken limit; +- a `ban` rule match or a trap path; +- a refusal by a `block` rule, the Core Rule Set or a token check. + +The check runs in the background, at the client's first request after the +offence that no ban refuses, so no request waits. The score applies to the +client's later requests. For an IPv6 client, AbuseIPDB is asked about the +address of the request that set off the check, and the score is used for the +whole group. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and +`SWWAF_REPUTATION_ACTION` says what is done. Each score is kept for +`SWWAF_REPUTATION_CACHE_TTL`, up to 100,000 scores. + +At most `SWWAF_ABUSEIPDB_DAILY_BUDGET` checks are made in a day, counted in UTC +and kept in `reputation.json`. A restart does not make the budget whole again. +The check that spends the last of the budget is logged and alerted. No client is +checked until the day ends. A check fails when: + +- AbuseIPDB answers other than `200`; +- the answer gives no score; +- no answer comes within `SWWAF_REPUTATION_TIMEOUT`. + +A failure is logged and alerted, and no client is checked for a minute. The key +is sent in the `Key` header and nowhere else, and is never logged. diff --git a/docs/rules.md b/docs/rules.md new file mode 100644 index 0000000..8af131b --- /dev/null +++ b/docs/rules.md @@ -0,0 +1,114 @@ +# Rule files + +`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`, in the order of +their names, and checks each request against their rules in that order, unless +the client is in `SWWAF_ALLOW_NETS`. The requirements the files meet are under +"Rule files" in [`SPEC.md`](../SPEC.md). A file whose name starts with `.`, such +as an editor's lock file, is not a rule file. + +## The format + +A rule is a line of four fields separated by spaces or tabs: an id, a target, an +action and a regex. The regex runs to the end of the line. Blank lines and lines +starting with `#` are ignored. + +``` +# id target action regex +env-file path ban (?i)^/\.env(\.[a-z]+)?$ +scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b +``` + +- The id is letters, digits, `-` and `_`. No two rules share one. The log line, + the metrics, the alerts and a ban's notes name the rule by it. +- The target is what the regex is matched against: + - `path`: the path without the query; + - `uri`: the path and the query together; + - `query`: the query as sent; + - `method`, `host`, `user_agent` and `referer`; + - `header:`: any one request header but `Host` and + `Transfer-Encoding`, which Go's HTTP server takes out of every request. + + `path` and `uri` are matched both as sent and once percent-decoded, so that + an encoded probe does not slip past. A header sent more than once is matched + with its values joined by `, `. A header not sent is empty text. No body is + read; reading bodies is the Core Rule Set's job, once `SWWAF_WAF_BODY_LIMIT` + switches it on. + +- The action is `log`, `block` or `ban`. `log` only notes the match in the log + line. `block` refuses the request with `403` and bans no one; the refusal + counts toward the error burst. `ban` refuses it and bans the client, as a + clear sign of attack. Matching stops at the first rule that refuses. Keep + `ban` for requests no real visitor sends. Anchor a path at the site root with + `^/`: a file of the same name deeper in a site can be ordinary content. +- The regex is in Go's syntax, RE2. It has no backreferences or lookaround, and + takes time linear in the text it reads, so no rule can be made to stall the + proxy. It matches anywhere in the target unless anchored with `^` and `$`. + `(?i)` at its front makes it ignore case. + +## Errors + +These stop the start, with a message naming the file and line: + +- a line that is not a rule; +- a bad `header:` name, or a rule for `Host` or `Transfer-Encoding`; +- a regex that does not compile; +- an id used twice. + +So does a `SWWAF_RULES_DIR` that does not exist. An empty directory is not an +error: the log says that no rules were loaded. + +While it runs, `smallwebwaf` watches the directory. It reads the files again +once the directory has had no change for 2 seconds, so that a file still being +written is read whole. If the files then hold such an error, the rules stay as +they were. The log and a `file_error` alert name the file and line. Once the +file is fixed, it is read again. + +## The default file + +The image ships one file, `share/rules.d/00-default.rules`, installed as +`/etc/smallwebwaf/rules.d/00-default.rules`: + +``` +# 00-default.rules: probes no real visitor sends, anchored at the site root + +# id target action regex +env-file path ban (?i)^/\.env(\.[a-z]+)?$ +vcs-dir path ban (?i)^/\.(git|svn|hg|bzr)(/|$) +secrets-dir path ban (?i)^/\.(aws|ssh|docker|kube)/ +secret-file path ban (?i)^/\.(htpasswd|htaccess|npmrc|netrc|pgpass|git-credentials|bash_history|DS_Store)$ +editor-dir path ban (?i)^/\.(vscode|idea)/ +backup-file path ban (?i)^/[^/]+\.(php(\.[a-z0-9]+|~)|sql(\.[a-z0-9]+)?)$ +log-file path ban (?i)^/(debug|error|access)\.log$ +compose-file path ban (?i)^/(docker-)?compose\.ya?ml$ +php-shell path ban (?i)^/(shell|c99|r57|wso|alfa)\.php$ +scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|masscan|zgrab|wpscan)\b +path-traversal uri block (\.\./){2,} +empty-agent user_agent log ^$ +``` + +It is kept short and limited to patterns that are wrong for every app, with its +path rules anchored at the site root. In front of gitea, `/.env` is a probe, +while `///src/branch/main/.env.example` is a file in a repository +that any visitor may open. Anything app-specific belongs in a file the app's +`Dockerfile` adds or the deployer mounts. A request for `/wp-login.php`, for +example, is a clear sign of attack in front of gitea and an ordinary login in +front of WordPress. Such a rule goes in a file named to sort after the default: + +``` +# 50-gitea.rules: WordPress probes, which gitea never serves +wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/) +``` + +```dockerfile +COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules +``` + +## Mounting rule files + +A directory mounted over `/etc/smallwebwaf/rules.d` replaces the default file; +single files mounted into it add to it. Many editors save a file by replacing +it, and a container keeps seeing the old one when the file alone is mounted. So +rules to be edited while `smallwebwaf` runs belong in a mounted directory, with +a copy of the default file if the defaults are to stay. The lookup database is +mounted the same way, for the same reason. To run without rules, mount an empty +directory or set `SWWAF_RULES_ENABLED=false`. diff --git a/docs/settings.md b/docs/settings.md new file mode 100644 index 0000000..d9088ba --- /dev/null +++ b/docs/settings.md @@ -0,0 +1,483 @@ +# Settings + +Every setting is an environment variable whose name starts with `SWWAF_`. Every +setting has a default, so none has to be set. Settings are read once, at start. +To change one, restart the container. + +A setting that is set to a value it cannot take stops the start. The process +writes one line, `"type":"process"` with `"msg":"invalid setting"`, naming the +variable and what is wrong with the value. It exits with status 1, and runit +starts it again every second until the value is fixed. The entries below say +which values each setting takes. Any other value stops the start. At start, the +settings in effect are logged on one line. Tokens, keys and the paths of +webhooks are shown as `********`. + +The forms of values: + +- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. Zero + and negative durations are refused. +- A size is a number of bytes with an optional `K`, `M` or `G`, which are powers + of 1024: `1K` is 1024 bytes, `512K`, `100M`, `5G`. +- A count is a whole number of requests, such as `1000`. +- `off` switches a setting off, where its entry below says it takes `off`. +- A list is comma separated, with spaces around items ignored. An empty item is + refused. A list set but empty is an empty list, and replaces the default. +- A netblock is an address and a prefix length, such as `10.0.0.0/8`. A bare + address stands for itself alone. +- A country is a two-letter code of ISO 3166-1, in either case: `de` and `DE` + are the same. `xk` stands for Kosovo. A code that is not a country is refused, + such as `nk` (North Korea is `kp`) or the withdrawn `su`. +- An AS number is `AS` and the number, in either case, such as `AS64496`. +- A percentage is a whole number from 0 to 100. +- A path prefix starts with `/`, such as `/assets/`. + +## Where it listens and what it protects + +- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens, an + optional host and a port. Keep the host empty: `:9000`, never + `127.0.0.1:9000`. The app must not listen on that port. The health check + follows the setting, and traefik's label must name the port. +- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or + `https` URL with a host and an optional port, and nothing more: no path, user + or query. +- `SWWAF_INSTANCE_NAME` (default: the container's host name, the first 12 + characters of the container's id): the `instance` name in every log line, + alert and metric. Set it to a name that survives deploys and tells instances + apart, such as `fsn1app1/gitea`. It must be valid UTF-8. While + `SWWAF_ALERT_NTFY_URL` is set, it may hold no control character. While + `SWWAF_LOG_REMOTE_URL` is set and `SWWAF_LOG_REMOTE_APP_NAME` is not, it must + be 1 to 48 printable ASCII characters without a space. +- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to ban no one and + log what `enforce` would have done. Observe mode still refuses a request over + a size or time limit, and one without the right token. +- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`, + the private address ranges): the netblocks whose `X-Forwarded-For` is + believed, normally where traefik comes from. Set but empty, it trusts nothing: + every TCP peer is then the client. +- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the length of the netblock that is + one IPv6 client, from 32 to 128. After a change, each IPv6 client starts + afresh. Existing bans keep refusing their netblocks until they end. +- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept in memory + and in `clients.json`, a whole number above zero. Past it, the least recently + seen client is dropped first. + +## Time and size limits + +Each request has limits on both sides: between the client and `smallwebwaf`, and +between `smallwebwaf` and the app. Bodies stream through in both directions, +except the part the Core Rule Set reads while `SWWAF_WAF_BODY_LIMIT` is set. + +- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to + send its request line and headers, and then, from the end of the headers, its + body. A duration or `off`. +- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request + line and headers. A size over `4K`. It cannot be `off`: Go's HTTP server + always has such a limit. +- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection + may wait for its next request before it is closed. A duration or `off`. The + default is longer than the 90 seconds after which traefik closes an unused + connection, so traefik closes first. +- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the answer may take + to reach the client, from the end of the request to the last byte. A duration + or `off`. +- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the + app and sending it the whole request may take. A duration or `off`. +- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take + to send its whole answer. A duration or `off`. +- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size + or `off`. A request that announces a larger body is refused before anything + reaches the app. +- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size + or `off`. + +In front of a code forge, a clone is a response and a push is a request. At the +defaults, an upload over 100 MiB or slower than 60 seconds is cut off. A forge +that takes large uploads needs `SWWAF_REQUEST_MAX_BYTES`, +`SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to +fit. + +What the client gets when a limit is passed is under "Size and time limits" in +[`SPEC.md`](../SPEC.md). + +## Netblock lists + +- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip every check, + such as your monitoring or your own network. The size and time limits still + apply to them. They are not looked up. They still need a token for the admin + endpoints. +- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are + exempt from rate limiting and the byte limits, such as a machine that talks to + the app all day. Everything else still applies to them. +- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused. + A client in both lists is let through. + +## Rate limiting and byte limits + +- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR` + (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most + requests a client may make in each window. A count or `off`. The defaults are + several times what one busy person produces. A browser loading a heavy page + makes a few hundred requests, and several people often share one address. +- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from + rate limiting, such as `/assets/`. Their bytes are not counted either. The + path is matched exactly as sent, without the query, and is not + percent-decoded. `/assets/` matches `/assets/app.js`. It does not match + `/Assets/app.js`, `/%61ssets/app.js` or `/static/assets/app.js`. There are no + wildcards. A path holding `..`, a backslash or an encoded slash (`%2F`) is + never exempt, since the app may read it as a path outside the prefix. +- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR` + (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most + bytes a client may have counted in each window. A size or `off`. Bytes are + counted once the answer has ended, so a byte limit never cuts an answer short. + The client's next request is refused instead. Each default is above the + largest request and answer together, so at the defaults no single download + breaks a limit. +- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count: + `response`, `request` or `both`. For a WebSocket, what it carried each way + counts once it closes. + +## Looking up the AS number and country + +- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `geojs` + asks the GeoJS web service, which is then told the address of every new + visitor. `file` looks clients up in the IPinfo Lite database file at + `SWWAF_LOOKUP_DB_PATH`, and sends nothing off the host. `off` looks up no + client. With `off`, a setting that needs lookups stops the start: + - a country list; + - `SWWAF_ADD_LOOKUP_HEADERS`; + - an entry in `SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`, + `SWWAF_ASN_BYTES_PERCENT` or `SWWAF_COUNTRY_BYTES_PERCENT`; + - `SWWAF_UNKNOWN_LIMIT_PERCENT` below 100; + - `SWWAF_ASN_LIMIT_PERCENT_URL`; + - an anomaly threshold per AS number. +- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its + `.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`. `file` without it stops the + start, and so does setting it with any other source. A file that is missing or + cannot be read stops the start too. +- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its + client's first answer from GeoJS when a setting needs it, and how long a + request to GeoJS may take. A duration; it cannot be `off`. +- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the + client's AS number in `X-Client-ASN` and its country in `X-Client-Country`. A + header whose value is unknown is left out. Whatever this setting says, any + such header the client sent is removed. + +See [`lookup.md`](lookup.md) for what each source sends where, and how long +answers are kept. + +## Country lists + +- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused, + such as `cn,ru,kp`. +- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only + countries whose clients get through, such as `us,de`. A client whose country + cannot be found is refused too. Otherwise every new client would be let in + whenever GeoJS stops answering. A client on a private address has no country, + so it is refused unless it is in `SWWAF_ALLOW_NETS`. + +A country on both lists stops the start. What a refused request gets, and why +the lists are empty by default, is under "Country lists" in +[`SPEC.md`](../SPEC.md). + +## Lower limits by AS number and country + +Clients of the AS numbers and countries listed here get a percentage of every +rate limit and byte limit. A lowered limit is rounded down. "Biased thresholds" +in [`SPEC.md`](../SPEC.md) says why the lists are empty by default and why no +budget is shared by a whole AS number or country. + +- `SWWAF_ASN_LIMIT_PERCENT` (default empty): a list of AS numbers, each with `:` + and a percentage, such as `AS14061:50,AS16276:50,AS45102:25`. An AS number + listed twice is refused. +- `SWWAF_COUNTRY_LIMIT_PERCENT` (default empty): the same by country, such as + `cn:25,ru:50`. +- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty): + the same for the byte limits alone. For the AS numbers and countries they + list, they take the place of the two settings above. +- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage of every limit + for a client without a country: + - one the lookup cannot place; + - one on a private address; + - one whose answer from GeoJS has not come in time. +- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a + text file of such entries, one to a line, such as `AS14061:50`. One list can + then serve several instances. It is fetched and kept as a blocklist is. Where + it and `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. A + URL that is also in `SWWAF_BLOCKLIST_URLS` stops the start. + +A client to which several percentages apply gets the lowest. The log line of +each counted request, and the notes of each ban, say which setting lowered the +limit. + +## Reputation sources + +None is on by default. A reputation source judges a client by what others saw it +do. The defaults judge it by what it does to your service. + +- `SWWAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as + `https://www.spamhaus.org/drop/drop.txt`. Each is an `http` or `https` URL + without a user or a fragment. A URL listed twice is refused. +- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its + last fetch, or failed try, before it is fetched again. A duration of `1h` or + more; it cannot be `off`. The Spamhaus lists may be fetched no more than once + an hour. +- `SWWAF_BLOCKLIST_ACTION` (default `deny`): what is done with a client on a + blocklist. `deny` refuses its requests. `limit:`, such as `limit:25`, + gives it that percentage of every limit. `log` only notes the list in the log + line and raises an alert. +- `SWWAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask about each + client, such as `dnsbl.dronebl.org`. A DNS blocklist answers a DNS query for a + name made from an address, and so says whether the address is on it. A zone is + a DNS name of at most 189 characters. A Spamhaus zone carries its key, as in + `.xbl.dq.spamhaus.net`, and is shown with the key masked. A zone listed + twice is refused, whatever its case or key. Do not name a zone meant for mail: + such a zone holds the residential addresses ordinary visitors come from. +- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through, + such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional + port, 53 when none is given. Unset, the host's resolver is used. Several zones + refuse queries that come through a public resolver. +- `SWWAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account. + While it is set, a client that has committed an offence is checked. A control + character in it is refused. +- `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score, + from 0 to 100, that is a hit. +- `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day, + in UTC, a whole number above zero. AbuseIPDB's free accounts may make 1,000. +- `SWWAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine + you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a + user or a fragment. Its decision list is at `v1/decisions` under that URL, and + is fetched every minute. Set without the key, it stops the start. So does a + decision list URL that is also in `SWWAF_BLOCKLIST_URLS`, or that is + `SWWAF_ASN_LIMIT_PERCENT_URL`. +- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for + `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the + URL, it stops the start. +- `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client on + a DNS blocklist, or one whose AbuseIPDB score is a hit. `deny`, + `limit:` or `log`, as for `SWWAF_BLOCKLIST_ACTION`. Such a verdict is + less certain than the Spamhaus DROP list, so by default the client gets a + quarter of every limit. +- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or + AbuseIPDB's score is used. A duration; it cannot be `off`. +- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a + check with AbuseIPDB may take. A duration; it cannot be `off`. + +See [`reputation.md`](reputation.md) for how each source is asked and what it +does with the answer. + +## Attack detection + +- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the + rule files (see [`rules.md`](rules.md)). A directory that cannot be read stops + the start. An empty one is no error. +- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file. +- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only + scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of a code + forge. A request for one bans the client, as a `ban` rule does. Each starts + with `/` and holds no `?`. A trap path is matched whole, without the query, + against the path as sent and once percent-decoded. `/wp-login.php` matches + `/wp-login.php?x=1` and `/%77p-login.php`. It does not match `/wp-login.php/` + or `/blog/wp-login.php`. +- `SWWAF_WAF_MODE` (default `block`): what the Core Rule Set does with a request + it takes for an attack. `block` refuses it with `403`. `detect` lets it + through, logged and alerted. `off` inspects no request. +- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level, + from 1 to 4. Each level up runs more rules. Those find more attacks, and take + more ordinary requests for attacks. +- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the score at which a request is a + match. Each rule that matches adds to the request's score, and a rule rated + critical adds 5. So by default one such rule is enough. A whole number above + zero, or `off`. With `off` no request is a match, and scores are still logged. +- `SWWAF_WAF_DISABLED_RULES` (default + `920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules + to switch off. The log line names the rules a request matched, so a rule that + refuses your app's ordinary requests can be found there. "Attack detection" in + [`SPEC.md`](../SPEC.md) says what the six default ids refuse and why they are + off. A list you give replaces the default, so name these six ids in it to keep + them off. An id must be a whole number above zero. An id from 900000 to 900999 + is refused: those are the rules that set the Core Rule Set up, and + `smallwebwaf`'s own. +- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does + not inspect, matched as `SWWAF_RATE_LIMIT_EXEMPT_PATHS` is. +- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size, + at most `1G`, switches body inspection on. The Core Rule Set then reads a form + data or multipart body up to that size, and the rest streams on to the app. It + reads a JSON or XML body no larger than that size, since those cannot be read + in part. Any other body, such as a git push, reaches the app uninspected. See + [`core-rule-set.md`](core-rule-set.md). +- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client + in a minute for a rule match, a trap path, a Core Rule Set match or a failed + token check. One more bans the client, as a broken rate limit does. A count or + `off`. The app's own `401` and `404` answers are not counted. + +## How long bans last and what they cover + +- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered: `403`, + `429`, or `close` to close the connection without an answer. A `block` rule + and the Core Rule Set always answer `403`. +- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate + limit, byte limit or error burst. A duration; it cannot be `off`. +- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a client that breaks a limit + again within this time after its ban ended is banned three times as long. A + duration; it cannot be `off`. +- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken limit that would + be longer is permanent instead. A duration; it cannot be `off`. +- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of + attack. A duration; it cannot be `off`. +- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are + kept, whether past, active or permanent. A whole number above zero. Bans whose + cause is `admin` are kept besides and never dropped (see + [`bans.md`](bans.md)). +- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around + an IPv4 client that a ban covers, from 0 to 32. `24` bans the surrounding /24. + An IPv6 ban covers the client's group. + +## Where and how often the state files are written + +- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the directory of the state + files, an absolute path. A directory that does not exist or cannot be written + stops the start. +- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be + written after a ban is made, lifted or made permanent. Every change in between + goes into the same write. A duration; it cannot be `off`. +- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is + written. A duration; it cannot be `off`. + +## Logging + +- `SWWAF_LOG_REQUEST_HEADERS` (default + `accept,accept-language,accept-encoding,content-type,origin,range`): the + request headers to record in the log line, in either case. `Authorization`, + `Cookie` and `Set-Cookie` are never logged, even if listed. `Host` and + `Transfer-Encoding` are refused: Go's HTTP server takes them out of the + request. The host is the field `host`. +- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own + messages that are written: `debug`, `info`, `warn` or `error`. No message is + at `debug`. It holds back no request log line. `warn` and `error` hold back + the line of settings at start. +- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server every line on stdout + is also sent to, such as `syslog+tls://logs.example:6514`. The forms are + `syslog+udp://`, `syslog+tcp://` and `syslog+tls://`, each with a host and a + port and nothing more. +- `SWWAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates + that a `syslog+tls` server's certificate must chain to, instead of the host's + own. A file that cannot be read, or holds no certificate, stops the start. +- `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they + wait to be sent, a whole number above zero. +- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`, + `user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, `news`, `uucp`, `cron`, + `authpriv`, `ftp`, or `local0` to `local7`. +- `SWWAF_LOG_REMOTE_APP_NAME` (default: `SWWAF_INSTANCE_NAME`): the app name in + each syslog record, 1 to 48 printable ASCII characters without a space. + +## Tokens and metrics + +- `SWWAF_ADMIN_TOKEN` (default unset): the token for the ban endpoints and + `/_smallwebwaf/clients/`. A long random value of at least 32 characters. + Unset, those endpoints are off. Give it as a file to keep it from the app (see + "Settings given as files" below). +- `SWWAF_METRICS_TOKEN` (default unset): the token a scraper sends for + `/_smallwebwaf/metrics`, at least 32 characters. Unset, the metrics are off. + It is separate from the admin token, so a scraper cannot manage bans. +- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many + countries get series of their own in the metrics. The rest count as `other`. A + whole number above zero. + +## Alert destinations + +- `SWWAF_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each + alert as a JSON object. An `http` or `https` URL without a user or a fragment. + The settings logged at start show its path and query as `********`, since many + webhooks carry their secret there. +- `SWWAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post, + such as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and + value. A value cannot hold a comma. +- `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook + that receives each alert as a message. It is checked and masked as the webhook + URL is. +- `SWWAF_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each + alert, as the topic's full URL, such as `https://ntfy.sh/my-alerts`. It is + checked and masked as the webhook URL is. +- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token, sent as + `Authorization: Bearer `. A control character in it is refused. +- `SWWAF_ALERT_EVENTS` (default + `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): + the events alerts are sent for. Any other name is refused. +- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held + back. A duration or `off`. +- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. + The rest are rolled into one summary. A whole number above zero, or `off`. + +See [`alerts.md`](alerts.md) for the events and what each destination gets. + +## Anomaly thresholds + +These only alert. They refuse and ban nothing. Each scope has four thresholds: +the most requests and the most bytes counted in a minute and in an hour. +Requests are counts, bytes are sizes. Each is `off` by default. What is unusual +depends on each service's normal traffic, which the metrics show. The four +thresholds per client are: + +- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`; +- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`; +- `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE`; +- `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR`. + +The other scopes' names have another prefix before the same four endings, such +as `REQUESTS_PER_MINUTE`: + +- `SWWAF_ANOMALY_NET_...`: per netblock around a client. Two settings give the + netblock's length: + - `SWWAF_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) for an IPv4 + client; + - `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6 + client. +- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs lookups on. +- `SWWAF_ANOMALY_TOTAL_...`: the whole service. +- `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole. + `SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such + as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a + name, or with a bad netblock, is refused. So is a name used twice. + +Every request but the health check is counted, whatever was done with it, +clients in `SWWAF_ALLOW_NETS` included. At most 20,000 counters are kept. +`alerts.json` keeps them across a restart. + +## Settings given as files + +Any setting may be given as a file holding its value. Add `_FILE` to the +setting's name: `SWWAF_METRICS_TOKEN_FILE` names the file that holds +`SWWAF_METRICS_TOKEN`. The file's contents are the value, less one newline at +the end. They are checked as the value itself would be. Setting both forms stops +the start, and so does naming a file that cannot be read. +`SWWAF_LOG_REMOTE_TLS_CA_FILE` already holds a file name, so it has no `_FILE` +form. + +A token given as a file is out of the app's reach only while the `smallwebwaf` +user alone can read the file. Make it on the host, owned by uid 65532 with mode +`0400`: + +```sh +mkdir -p /srv/app/tokens +openssl rand -hex 32 > /srv/app/tokens/metrics +chown 65532:65532 /srv/app/tokens/metrics +chmod 0400 /srv/app/tokens/metrics +``` + +Mount its directory into the container read-only, and name the file: + +```sh +docker run \ + -v /srv/app/tokens:/etc/smallwebwaf/tokens:ro \ + -e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics \ + ... +``` + +## Fixed limits + +A few limits are fixed rather than settings: + +- at most 100,000 GeoJS answers are kept, for 7 days each; +- at most 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept; +- at most 1,000 DNS blocklist queries are under way at once; +- the CrowdSec decision list is fetched again a minute after the last fetch. diff --git a/docs/state-files.md b/docs/state-files.md new file mode 100644 index 0000000..dd185cd --- /dev/null +++ b/docs/state-files.md @@ -0,0 +1,102 @@ +# State files + +`smallwebwaf` keeps its state in memory, and a copy of it in five JSON files in +`SWWAF_STATE_DIR`. The files are meant for people as well: you can read or edit +them at any time, and the running `smallwebwaf` takes the edit in. The +requirements they meet are under "Persistent state" in [`SPEC.md`](../SPEC.md). + +## The files + +Each has a top-level `version` of 1. + +- `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf` + made and every ban you made, indented to be read. Per entry: + - the netblock, start and expiry (`null` for permanent); + - the cause: `attack`, `limit`, `admin` or `crowdsec`; + - a short reason; + - when it was lifted, if you lifted it; + - the notes (see [`bans.md`](bans.md)). +- `clients.json`: per client, its minute, hour and day counters and its history + since it was first seen, one client per line, so `grep` shows everything about + one. The history holds: + - the first and last time seen; + - the AS number, AS name and country last looked up, and when; + - total requests, and how many were forwarded and how many refused; + - bytes in each direction; + - responses by status class; + - offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and + `token_refused`. + + Clients that were never banned are kept too, so every client's history + survives a restart. + +- `lookups.json`: GeoJS answers, one per line: the AS number, AS name and + country, when GeoJS was asked and when the answer was last used. +- `reputation.json`: + - the last good copy of each list fetched from a URL, with when it was last + tried; + - each DNS blocklist verdict still in use; + - AbuseIPDB's scores, and the checks spent today. +- `alerts.json`: + - the anomaly counters; + - the alert cooldowns: for each event and netblock, source, file or thing + counted, when it was last sent and the count of repeats held back since; + - the alerts sent and held back in the current hour; + - the alerts waiting for each destination. + +Times are RFC 3339 UTC. Durations and sizes are plain numbers, with the unit in +the field name. `bans.json`, `reputation.json` and `alerts.json` are indented; +`clients.json` and `lookups.json` put each entry on one line. + +## Sizes + +`clients.json` takes about 1 KiB per client, about 20 MiB at the default +`SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of +disk writes a day. `bans.json` takes about 1.2 KiB per ban, at most about 2.5 +KiB, so about 6 MiB at the default `SWWAF_MAX_BANS`. With a full file, a hundred +new bans a day come to about 600 MiB of writes. `lookups.json` takes about 150 +bytes per answer, under 1.5 GiB of writes a day when full. The other two files +are usually a few MiB or less. + +## Writing + +`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted or +made permanent. Every file is written every `SWWAF_STATE_COUNTER_INTERVAL`, and +when `smallwebwaf` stops. Each write goes to a temporary file that then replaces +the file, so a crash leaves the old file or the new one, whole. Before writing a +file, `smallwebwaf` takes in any edit you made to it, so your edit is not +overwritten. A write that fails is logged, raised as a `file_error` alert, and +tried again at the next write. A hard kill loses what changed since the last +write: + +- ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds); +- everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes): + - counts and history, and the counts in ban notes; + - GeoJS answers; + - reputation verdicts and the AbuseIPDB count; + - anomaly counters and alert cooldowns. + +## Reading at start + +The files are read back. Each client keeps its counts, so a restart gives no one +a fresh allowance. Each ban keeps refusing until it ends. Counter buckets and +cache entries whose time has passed are discarded. A missing file is empty +state. These stop the start, with a message naming the file and the place: + +- a file that does not parse, or has another `version`; +- an entry without a field it needs; +- a ban whose `cause` is not `limit`, `attack`, `admin` or `crowdsec`; +- a ban whose `expires` is not after its `start`. + +## Editing while it runs + +`smallwebwaf` watches the directory and takes in your edit of a file as soon as +you save it. What the file then holds replaces what `smallwebwaf` held for it. +It tells its own writes from yours by comparing the file with what it last read +or wrote. A change it made after you opened the file, such as a new ban, is lost +when you save over it; most editors warn when a file changed on disk while it +was open. An edit with an error that would stop the start does not stop the +running `smallwebwaf`. It keeps what it holds. At the file's next write it +renames your file to `.bad`, writes the file again from memory, and logs +and alerts the file and the place of the error. Mend the `.bad` file and move it +back. A file you remove is written again at its next write. diff --git a/internal/alerts/alerts.go b/internal/alerts/alerts.go index 89ea0e9..7864637 100644 --- a/internal/alerts/alerts.go +++ b/internal/alerts/alerts.go @@ -1,8 +1,8 @@ // Package alerts sends alerts on bans, on traffic over an anomaly // threshold, on a source that fails and on a file with an error to each // destination set: to the webhook -// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as the "Alert -// webhook schema" section of SPEC.md describes, to the Slack incoming +// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON +// object" in docs/alerts.md describes, to the Slack incoming // webhook SWWAF_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the // ntfy topic SWWAF_ALERT_NTFY_URL names. A repeat within // SWWAF_ALERT_COOLDOWN is held back, and so is an alert past @@ -149,11 +149,10 @@ type Params struct { } // Alert is one alert, as the webhook is sent it and alerts.json holds it, -// with the fields of the "Alert webhook schema" section of SPEC.md. ASN, -// ASName and Country are, for a ban, the client's as the ban's notes give -// them. +// with the fields of "The JSON object" in docs/alerts.md. ASN, ASName and +// Country are, for a ban, the client's as the ban's notes give them. // -//nolint:tagliatelle // SPEC.md's alert webhook schema names its fields in snake_case +//nolint:tagliatelle // docs/alerts.md names the alert's fields in snake_case type Alert struct { Instance string `json:"instance"` Time time.Time `json:"time"` diff --git a/internal/metrics/metrics.go b/internal/metrics/metrics.go index f1f4fc4..c7f7d96 100644 --- a/internal/metrics/metrics.go +++ b/internal/metrics/metrics.go @@ -1,6 +1,7 @@ -// Package metrics keeps smallwebwaf's Prometheus metrics, as the "Metrics -// endpoint" section of SPEC.md lists them, and serves them in the -// Prometheus text format. No metric carries a client's address. +// Package metrics keeps smallwebwaf's Prometheus metrics, as +// docs/metrics.md lists them, and serves them in the Prometheus text +// format, as the "Metrics endpoint" section of SPEC.md requires. No +// metric carries a client's address. package metrics import ( diff --git a/internal/ratelimit/ratelimit.go b/internal/ratelimit/ratelimit.go index 2413ac6..2318102 100644 --- a/internal/ratelimit/ratelimit.go +++ b/internal/ratelimit/ratelimit.go @@ -206,7 +206,7 @@ type Hit struct { // Counts are a client's requests and bytes in the minute, the hour and // the day that end at a request, that request's included. // -//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case +//nolint:tagliatelle // docs/request-log.md names the fields in snake_case type Counts struct { Minute float64 `json:"minute"` Hour float64 `json:"hour"` diff --git a/internal/remotelog/remotelog.go b/internal/remotelog/remotelog.go index d40239b..9f5f2ee 100644 --- a/internal/remotelog/remotelog.go +++ b/internal/remotelog/remotelog.go @@ -1,8 +1,9 @@ // Package remotelog sends the lines smallwebwaf writes on stdout to the // remote log endpoint, SWWAF_LOG_REMOTE_URL, as the "Request log" section -// of SPEC.md describes: each line as the message of an RFC 5424 syslog -// record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so a slow -// or unreachable endpoint never holds up a request or stdout. +// of SPEC.md requires and "Sending the log to a syslog server" in +// docs/request-log.md describes: each line as the message of an RFC 5424 +// syslog record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so +// a slow or unreachable endpoint never holds up a request or stdout. package remotelog import ( diff --git a/internal/requestlog/requestlog.go b/internal/requestlog/requestlog.go index d9b5f66..829f8a0 100644 --- a/internal/requestlog/requestlog.go +++ b/internal/requestlog/requestlog.go @@ -64,11 +64,10 @@ const LimitHitErrorBurst = "error_burst" const timeLayout = "2006-01-02T15:04:05.000Z07:00" // Line is one request's line in the request log. The field names, and -// their order, are those of the "Request log" section of SPEC.md. A field -// that may not apply to a request is left out of its line when it does -// not. +// their order, are those of docs/request-log.md. A field that may not +// apply to a request is left out of its line when it does not. // -//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case +//nolint:tagliatelle // docs/request-log.md names the fields in snake_case type Line struct { Type string `json:"type"` -- 2.54.0 From 4ea0a975c8c3d0f02d0adc7ff8f0c2ae83a9c162 Mon Sep 17 00:00:00 2001 From: sneak Date: Thu, 22 Oct 2026 21:51:39 +0000 Subject: [PATCH 5/8] SPEC: the one place for behaviour and reasons; docs/ as lookup lists only docs/core-rule-set.md, docs/lookup.md, docs/reputation.md and docs/bans.md restated the spec and are folded back into it and removed. The remaining docs files hold only lists: settings with their defaults and value forms, the request log's fields, the metrics, the endpoints, the state files and their contents, the rule file format and default file, the alert events and messages, the image's contents and pins, and the gitea notes. Every behaviour each of them held moves to the spec, said once. The settings groups have the same names in both files. Stale comments in internal/state and script/example-app point at the right places. Model: fable-5-1 --- SPEC.md | 730 ++++++++++++++++++++------------ docs/admin-endpoints.md | 49 ++- docs/alerts.md | 70 ++- docs/bans.md | 84 ---- docs/core-rule-set.md | 31 -- docs/gitea.md | 39 +- docs/image.md | 78 ++-- docs/lookup.md | 79 ---- docs/reputation.md | 113 ----- docs/request-log.md | 50 +-- docs/rules.md | 71 +--- docs/settings.md | 599 ++++++++++---------------- docs/state-files.md | 182 ++++---- internal/remotelog/remotelog.go | 7 +- internal/state/state.go | 3 +- internal/state/state_test.go | 4 +- script/example-app | 2 +- 17 files changed, 918 insertions(+), 1273 deletions(-) delete mode 100644 docs/bans.md delete mode 100644 docs/core-rule-set.md delete mode 100644 docs/lookup.md delete mode 100644 docs/reputation.md diff --git a/SPEC.md b/SPEC.md index e7c5586..c0f2dbd 100644 --- a/SPEC.md +++ b/SPEC.md @@ -5,8 +5,8 @@ runs inside the app's container, between traefik and the app. This document says what it must do and why, each requirement once. Everything in it is built, except the last section, "Planned, not built". `README.md` says how to run it. `EVALUATION.md` says why no existing tool was adopted. The reference documents -under `docs/`, which this document links to, give the settings, the formats and -the lists; every fact lives in one of these files and is linked from the others. +under `docs/` give the settings, the formats and the lists. This document links +to them. ## Purpose @@ -58,12 +58,12 @@ the lists; every fact lives in one of these files and is linked from the others. - `internal/ratelimit`: the table of clients. It counts requests, bytes and refusals, and keeps each client's history. - `internal/bans`: the bans. It holds each netblock's bans with their notes, - decides how long a new ban lasts, and picks the ban to drop past - `SWWAF_MAX_BANS`. + decides how long a new ban lasts, and picks the ban to drop when the table + of bans is full. - `internal/rules`: reads the rule files, and again as they change, and tells which rules a request matches. - - `internal/waf`: the Core Rule Set with the six changes, as Coraza - directives. + - `internal/waf`: the Core Rule Set, with the six changes listed under + "Attack detection", as Coraza directives. - `internal/lookup`: looks clients up through GeoJS or in the lookup database. - `internal/reputation`: fetches the lists, asks the DNS blocklists and @@ -116,9 +116,9 @@ names the section that holds its rules. around its address, a /64 by default, since one abuser usually controls a whole /64. A client is therefore one IPv4 address or one IPv6 group. 3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12 and the - byte limits. It is still counted for the anomaly thresholds, and still needs - a token at step 13. A client in `SWWAF_DENY_NETS` is refused; a client in - both lists is let through. + byte limits and the error burst at step 15. It is still counted for the + anomaly thresholds, and still needs a token at step 13. A client in + `SWWAF_DENY_NETS` is refused; a client in both lists is let through. 4. Bans. A client under a ban is refused (see "Bans"). 5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number and country"). The request goes on at once. It waits for the first GeoJS @@ -148,15 +148,19 @@ names the section that holds its rules. `detect`. 13. A request under `/_smallwebwaf/` is answered here (see "Admin endpoints"). 14. The request is forwarded to `SWWAF_UPSTREAM_URL`, streaming, with - `X-Forwarded-For` added, and `X-Client-ASN` and `X-Client-Country` when - enabled. + `X-Forwarded-For` added. With `SWWAF_ADD_LOOKUP_HEADERS`, `X-Client-ASN` and + `X-Client-Country` are added for the app's own logs; a header whose value is + unknown is left out. Whatever that setting says, any such header the client + sent is removed. 15. After the response: - The response bytes, the request bytes or both are added to the byte - counters, as `SWWAF_BYTES_COUNT` says. A client whose total passes a byte - limit times the percentage has broken that limit and is banned. The - response in progress is not cut off. - - The client's refusals are counted for the error burst (see "Attack - detection"). + counters, as `SWWAF_BYTES_COUNT` says. For a WebSocket, what it carried + each way counts once it closes. A client whose total passes a byte limit + times the percentage has broken that limit and is banned. The response in + progress is not cut off. + - A refusal by a rule file, a trap path, the Core Rule Set or a token check + is counted for the error burst (see "Attack detection"). No other refusal + is. - The client's history and the anomaly counters are updated, and the alert thresholds are evaluated. @@ -176,9 +180,10 @@ the token check still apply. load. - Memory is bounded. When the table of clients holds `SWWAF_MAX_TRACKED_CLIENTS`, the least recently seen client is dropped first, - with its history. The bans, the GeoJS answers, the reputation verdicts and the - anomaly counters have bounds of their own; `docs/settings.md` gives the fixed - ones under "Fixed limits". + with its history. The bans are bounded by `SWWAF_MAX_BANS` (see "Bans"). The + GeoJS answers, the reputation verdicts and scores, the DNS queries under way + and the anomaly counters have fixed bounds, listed under "Fixed limits" in + `docs/settings.md`; past a bound, the entry used longest ago goes first. ## Bans @@ -187,7 +192,8 @@ Until a ban ends, every request from its netblock is refused with `SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one address or one /64, so a permanent ban does not reach neighbours who did nothing. Requests refused under a ban are not counted against limits. They are -counted in the client's history and in the ban's notes. +counted in the client's history and in the ban's notes. A banned client is not +looked up. An offence is a request `smallwebwaf` holds against the client. It is one of: @@ -203,7 +209,7 @@ to a ban, each by its own rule: or asks for a path in `SWWAF_TRAP_PATHS`. Examples are a probe for a `.env` file or a `.git` directory, and a known scanner's user agent. - The first one bans the netblock for `SWWAF_ATTACK_BAN_DURATION` (default - `7d`). + `7d`), with the cause `attack`. - Any further request from the netblock while that ban lasts shows it is malicious: the ban becomes permanent. - Once that ban has run out, the netblock is served like any other. Its next @@ -211,9 +217,10 @@ to a ban, each by its own rule: - A broken limit: a request over a rate limit, a response that takes the client's byte total past a byte limit, or more than `SWWAF_ERROR_BURST_THRESHOLD` refusals within a minute. - - The first such ban lasts `SWWAF_LIMIT_BAN_DURATION` (default `1h`). So - does one that comes more than `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default - `24h`) after the last such ban ended. + - The first such ban lasts `SWWAF_LIMIT_BAN_DURATION` (default `1h`), with + the cause `limit`. So does one that comes more than + `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`) after the last such ban + ended. - Breaking a limit again within that window makes the new ban three times as long as the last one: one hour, then 3, 9, 27 and 81 hours. - A ban that would be longer than `SWWAF_MAX_BAN_DURATION` (default `7d`) is @@ -222,15 +229,18 @@ to a ban, each by its own rule: Once it ends, only new requests can break a limit again. The client's history keeps its totals. -A Core Rule Set match refuses only the request it matched, without a ban, -because the Core Rule Set has false positives. A `block` rule does the same. -Those refusals count toward the error burst, so a client that keeps setting them -off is banned under the second rule. +A Core Rule Set match refuses only the request it matched, with `403` and +without a ban, because the Core Rule Set has false positives. A `block` rule +does the same. Those refusals count toward the error burst, so a client that +keeps setting them off is banned under the second rule. A client on the CrowdSec decision list is banned until the decision ends, with the cause `crowdsec`. Only a client that sends a request is banned, so a long list does not fill `bans.json`. Such a ban is never made permanent. It does not -make the netblock's next ban for a broken limit longer. +make the netblock's next ban for a broken limit longer. A ban lifted while the +decision is in force is followed by another at the client's next request; the +decision is deleted in CrowdSec first, and the ban lifted once the list has been +fetched again. These refuse each request they cover, record nothing in `bans.json`, and count no offence: @@ -240,31 +250,34 @@ no offence: - the country lists. Every ban carries notes with what an admin needs to decide whether to lift it, -drawn from what `smallwebwaf` already knows; nothing is looked up to fill them. -`docs/bans.md` lists the notes and says how a ban is made, kept or lifted, by -editing `bans.json` or through the ban endpoints. A ban lifted before it ends is -kept, marked as lifted, and does not count toward a longer ban. Deleting its -entry from `bans.json` forgets it entirely. +drawn from what `smallwebwaf` already knows. No lookup is made for the notes +alone. `docs/state-files.md` lists the notes with the rest of a ban's entry. + +An admin adds, keeps or lifts a ban by editing `bans.json` (see "Persistent +state") or through the ban endpoints (see "Admin endpoints"). A ban an admin +makes has the cause `admin`; an admin who wants to keep a ban `smallwebwaf` made +sets its cause to `admin`. A ban lifted before it ends is kept, marked as +lifted, and does not count toward a longer ban; it refuses nothing. Deleting its +entry from `bans.json` forgets it entirely. Only a ban whose cause is `attack` +becomes permanent at the first request it refuses. An admin whose own address is +banned lifts the ban by editing `bans.json`. A permanent ban does not run out. It ends when an admin lifts it, or when `smallwebwaf` drops it from a full table, as below. When `smallwebwaf` holds `SWWAF_MAX_BANS` bans of its own, it drops the earliest ban of one netblock: the netblock that has gone longest without a request. A past, active or permanent ban is dropped alike. A scanner whose ban was dropped that way is refused and -banned again by its next probe. A ban an admin made, whose cause is `admin`, is -never dropped and does not count toward `SWWAF_MAX_BANS`, so such bans can never -fill the table. An admin who wants to keep a ban `smallwebwaf` made sets its -cause to `admin`. +banned again by its next probe. A ban whose cause is `admin` is never dropped +and does not count toward `SWWAF_MAX_BANS`, so such bans can never fill the +table. ## Settings +`docs/settings.md` lists every setting with its default and the values it takes, +in the groups below. It also gives the forms of values: lists, netblocks, +durations, sizes, countries, AS numbers, percentages and path prefixes. Conventions: -- Lists are comma separated. Netblocks are CIDR; a bare address means /32 or - /128. Durations use Go's syntax plus `d` for days (`90s`, `15m`, `24h`, `7d`). - Sizes take `K`, `M` and `G`, powers of 1024. Countries are two-letter ISO - codes in either case. A code that is not a country, such as `nk`, stops the - start with a message naming it. - These settings take `off`, which switches them off: - `SWWAF_LOOKUP_SOURCE` and `SWWAF_WAF_MODE`; - the five timeouts under "Size and time limits"; @@ -275,21 +288,24 @@ Conventions: - `SWWAF_ALERT_COOLDOWN` and `SWWAF_ALERT_MAX_PER_HOUR`; - every anomaly threshold. - Every other setting refuses `off`. + No other setting can be switched off with `off`. - A list set to an empty value is an empty list, and replaces the default. - Any setting can be read from a file instead, for secrets and long lists: - `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. - `SWWAF_LOG_REMOTE_TLS_CA_FILE` already holds a file name, so it has no `_FILE` - form. + `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. The + file's contents are the value, less one newline at the end, and are checked as + the value itself would be. Setting both forms stops the start, and so does + naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already + holds a file name, so it has no `_FILE` form. - Settings, files included, are read once at start. Changing one means restarting the container. The files `smallwebwaf` watches while it runs are its state files, its rule files and the lookup database. -- An invalid setting stops the process at start with a message naming the - variable. At start the settings in effect are logged, with secrets masked. - -`docs/settings.md` lists every setting with its default and the values it takes, -in the groups below. What each group must do, and why: +- An invalid setting stops the process at start with one log line, marked + `"type":"process"` with `"msg":"invalid setting"`, naming the variable and + what is wrong with the value. runit then starts the process again every second + until the value is fixed (see "The two processes"). At start the settings in + effect are logged on one line; tokens, keys and the paths of webhooks, which + often carry a secret, are shown as `********`. ### Core @@ -304,16 +320,19 @@ in the groups below. What each group must do, and why: the private address ranges, so a client that connects directly cannot claim another address. Where other containers reach `smallwebwaf` directly, or visitors on a private network reach traefik, they can name another address in - the header; the setting, narrowed to traefik's own network, closes that gap. -- `SWWAF_IPV6_GROUP_PREFIX`, `SWWAF_MAX_TRACKED_CLIENTS` and `SWWAF_MAX_BANS` - set the group size and the bounds under "Counting method" and "Bans". + the header. Narrowing the setting to traefik's own network closes that gap. +- `SWWAF_IPV6_GROUP_PREFIX` sets the group size. After a change, each IPv6 + client starts afresh; existing bans keep refusing their netblocks until they + end. `SWWAF_MAX_TRACKED_CLIENTS` and `SWWAF_MAX_BANS` set the bounds under + "Counting method" and "Bans". ### Tokens - `SWWAF_ADMIN_TOKEN` switches on the ban endpoints and - `/_smallwebwaf/clients/`; `SWWAF_METRICS_TOKEN` switches on the metrics, - which would otherwise be open to anyone on the internet. The two are separate, - so a scraper that holds the metrics token cannot manage bans. + `/_smallwebwaf/clients/`. The metrics are off until `SWWAF_METRICS_TOKEN` + is set, since without a token they would be open to anyone on the internet. + The two are separate, so a scraper that holds the metrics token cannot manage + bans. - A token is a long random value, since the endpoints can be reached from the internet. One shorter than 32 characters stops the start. The app starts with the same environment variables as `smallwebwaf`, so it can read a token set as @@ -322,36 +341,36 @@ in the groups below. What each group must do, and why: ### State files -- The state files live in `SWWAF_STATE_DIR`, a directory of their own beside the - app's data, belonging to the `smallwebwaf` user. +- `SWWAF_STATE_DIR` names the directory of the state files (see "State" under + "Deployment"). - `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted or made permanent, so a burst of changes becomes one write. Every file is written every `SWWAF_STATE_COUNTER_INTERVAL`, and at an orderly stop. ### Logging -- The request log on stdout is always on and has no switch. The process's own - messages are JSON lines on stdout too, marked `"type":"process"`; - `SWWAF_LOG_LEVEL` filters only those. +- The request log on stdout is always on and has no switch (see "Request log"). + The process's own messages share the stream; `SWWAF_LOG_LEVEL` filters only + those, and no message is at `debug`. `warn` and `error` hold back the line of + settings at start. - `SWWAF_LOG_REQUEST_HEADERS` names the request headers to record. - `Authorization`, `Cookie` and `Set-Cookie` values are never logged, only - whether they were present. + `Authorization`, `Cookie` and `Set-Cookie` values are never logged, even if + listed, only whether they were present. - `SWWAF_LOG_REMOTE_URL` sends every log line to a syslog server as well, over - UDP, TCP or TLS. Lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines; - when it is full, the oldest are dropped and counted. + UDP, TCP or TLS (see "Request log"). ### Metrics -- `SWWAF_METRICS_TOP_N` bounds the series by AS number and by country: that many - of the busiest each, the rest summed as `other`, so the label set stays +- `SWWAF_METRICS_TOP_N` is how many of the busiest AS numbers and countries get + a series of their own. The rest are summed as `other`, so the label set stays bounded. ### Static lists -- `SWWAF_ALLOW_NETS` bypasses everything (monitoring, the owner's own networks). - `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and byte limits only; the - error burst, attack detection and bans still apply. `SWWAF_DENY_NETS` is - always refused. +- `SWWAF_ALLOW_NETS` skips the checks step 3 names, for monitoring or the + owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and + byte limits only; the error burst, attack detection and bans still apply. + `SWWAF_DENY_NETS` is always refused. - Many clients share one address (mobile carriers, offices, Tor), and so share its limits and its bans. The defaults sit well above one person's use, and the first two lists take known shared addresses out. @@ -361,16 +380,24 @@ in the groups below. What each group must do, and why: Breaking one bans the client, so the defaults sit several times above what one busy person produces. A browser loading a heavy page makes a few hundred requests, a git clone a handful, and several people often share one address. A -machine that talks to the app all day, such as a gitea Actions runner, can still -pass the day limit. Its address belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. +machine that talks to the app all day can still pass the day limit. Its address +belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. + `SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as -static assets or health checks. +static assets or health checks; their bytes are not counted either. A prefix is +matched exactly as sent, without the query, and is not percent-decoded: +`/assets/` matches `/assets/app.js`, and does not match `/Assets/app.js`, +`/%61ssets/app.js` or `/static/assets/app.js`. There are no wildcards. A path +holding `..`, a backslash or an encoded slash (`%2F`) is never exempt, since the +app may read it as a path outside the prefix. `SWWAF_WAF_EXEMPT_PATHS` is +matched the same way. ### Byte limits, per client -A response's bytes are counted when it ends, so every default sits above the -largest response allowed, and no single download breaks one. `SWWAF_BYTES_COUNT` -says whether request bytes, response bytes or both are counted. +A response's bytes are counted when it ends, so a byte limit never cuts an +answer short; the client's next request is refused instead. Every default sits +above the largest request and answer together, so at the defaults no single +download breaks a limit. ### Size and time limits @@ -380,7 +407,8 @@ and the app, since a slow client and a slow app are separate problems. There is one size limit for request bodies and one for response bodies. Bodies pass through unchanged, so a limit on each side would bound the same bytes, and the lower one would always decide. Bodies stream straight through, so a request body -reaches the app while the client is still sending it. +reaches the app while the client is still sending it, except the part the Core +Rule Set reads while `SWWAF_WAF_BODY_LIMIT` is set. - `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` must be more than `4K` and cannot be `off`. Go's HTTP server always has such a limit, and reads 4 KiB past the one @@ -402,12 +430,12 @@ What the client gets when a limit is passed: started, is answered by `smallwebwaf`: - `413` for a request body that is too large; - `408` for a client too slow to send its body; - - `502` for a response that is too large; + - `502` for a response that is too large, or an app that cannot be reached; - `504` for an app that is too slow. - A request body streams through, so each side can hold up the other. A request - timeout that runs out while the body is still on its way answers `408` if - `smallwebwaf` was waiting for the client at that moment, and `504` if it was - waiting for the app. + timeout can run out while the body is on its way. The answer is then `408` if + `smallwebwaf` was waiting for the client, and `504` if it was waiting for the + app. - Once the response has started it can only be cut off, and the connection is closed. - A WebSocket connection leaves these limits behind once it is upgraded. It @@ -416,41 +444,61 @@ What the client gets when a limit is passed: ### Lookup of AS number and country On by default through GeoJS, which needs no account or file, so that country -lookup works with no setup. A local database file, the free IPinfo Lite -database, is the alternative for a service that keeps its visitors' addresses on -its own host; the operator downloads and mounts it, and `smallwebwaf` never -fetches it and holds no account or token for it. `docs/lookup.md` says what each -source sends where, how long answers are kept, and when a request waits. +lookup works with no setup. A local database file is the alternative for a +service that keeps its visitors' addresses on its own host. - Only one source is used. The start stops, naming both settings, when `file` is set without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other - source. The start also stops when the source is `off` and a setting needs - lookups: a country list, a biased threshold, `SWWAF_ADD_LOOKUP_HEADERS` or an - anomaly threshold per AS number. + source. The start also stops when the source is `off` and one of these needs + lookups: + - a country list; + - a biased threshold; + - `SWWAF_ADD_LOOKUP_HEADERS`; + - an anomaly threshold per AS number. - GeoJS is told the address of every new visitor, whether or not a setting uses the answer, since the request log, the metrics and the ban notes carry it. - Private, loopback and link-local addresses, which no source can place, are - never sent. Addresses are asked about in batches, one request at a time, and - answers are kept for days, which keeps the number of requests to GeoJS low. - Answers are kept apart from the table of clients, so that an answer outlasts a - client dropped from that table. + These addresses are never sent: + - one in `SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`; + - one under a ban; + - a private, loopback or link-local address, which no source can place. +- For an IPv6 visitor, GeoJS is asked about the first address of its group. + Addresses are asked about together, up to 200 in one request + (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a time, which + keeps the number of requests to GeoJS low. Each answer's `country_code`, `asn` + and `organization_name` are read. An answer without a `country_code`, or with + the `asn` `64512`, counts as unknown. +- Each answer is kept for seven days, in memory and in `lookups.json`. Answers + are kept apart from the table of clients, so that an answer outlasts a client + dropped from that table. The answer reaches the client's history and ban notes + when it comes. +- `SWWAF_LOOKUP_TIMEOUT` is how long a visitor's request waits for its client's + first answer, when a setting needs it (see step 5). It also bounds each + request to GeoJS. A client whose answer has not come in time counts as unknown + until it comes, and its later requests do not wait. Its address is asked about + again in the background. - A client the lookup cannot place has an unknown country and AS number. The - exclusive country list refuses it, and the biased thresholds give it - `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it differently. -- `SWWAF_LOOKUP_TIMEOUT` bounds both how long a visitor's request waits for its - client's first answer, when a setting needs it, and how long a request to - GeoJS may take. A client whose answer has not come in time counts as unknown - until it comes, and its later requests do not wait. + biased thresholds give it `SWWAF_UNKNOWN_LIMIT_PERCENT`, and the exclusive + country list refuses it (see "Country lists"). Nothing else treats it + differently. - GeoJS publishes no rate limit, but its terms forbid "an excessive amount of API requests", judged by GeoJS alone, and let it block a caller. While GeoJS - is slow, down or refusing `smallwebwaf`, new clients count as unknown, - `smallwebwaf` keeps asking with backoff, and one `source_failure` alert is - sent per cooldown. A service that cannot accept this, or that every new - visitor's address goes to a third party, uses the file. -- A replaced database file is read again; one that cannot be read is logged, - sent as one `file_error` alert, and ignored, the file in use staying in use. - The directory that holds the file is mounted, not the file itself (see "Rule - files" for why). + is slow, down or refusing `smallwebwaf`, new clients count as unknown. GeoJS + is then left alone for a second, and twice as long after each further failure, + up to five minutes, and one `source_failure` alert is sent per cooldown. A + service that cannot accept this uses the file. +- The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`, + `as_name` and `country_code` fields are read. The operator downloads it with a + free IPinfo account and mounts it read-only; `smallwebwaf` never fetches it + and holds no account or token for it. IPinfo updates it daily, and releases it + under the Creative Commons Attribution-ShareAlike 4.0 licence, asking for a + link to IPinfo on the site that uses it, such as "IP address data is powered + by IPinfo". +- The file is read whole at start; one that is missing or unreadable stops the + start. It is read again once it has gone 2 seconds without a change after + being replaced. One that cannot be read is logged, sent as one `file_error` + alert and ignored. The database already loaded stays in use. The directory + that holds the file is mounted, not the file itself (see "Rule files" for + why). A client whose address the file does not hold counts as unknown. ### Country lists @@ -460,12 +508,12 @@ where it comes from. Clients in `SWWAF_ALLOW_NETS` are not checked. - `SWWAF_DENIED_COUNTRIES` refuses every request from a listed country. - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` lets through only clients placed in a listed country. Every request from any other country is refused. So is every - request from a client the lookup cannot place, such as internal monitoring; - those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let - unplaced clients through would let every new client in whenever GeoJS stops - answering. The cost: while GeoJS fails, this list refuses every new client. No - other failure of GeoJS, a reputation source or an alert destination comes this - close to taking the service down. + request from a client the lookup cannot place, such as internal monitoring or + a visitor on a private address; those that should reach the app go in + `SWWAF_ALLOW_NETS`. A list that let unplaced clients through would let every + new client in whenever GeoJS stops answering. The cost: while GeoJS fails, + this list refuses every new client. No other failure of GeoJS, a reputation + source or an alert destination comes this close to taking the service down. - Both may be set. `SWWAF_DENIED_COUNTRIES` then adds nothing, since the exclusive list already refuses every other country. A code on both lists stops the start with a message naming it. @@ -481,17 +529,25 @@ request log and the metrics show which AS numbers and countries a service's abuse comes from. - Clients in a listed AS number or country get that percentage of every request - and byte limit (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`), or - of the byte limits alone (`SWWAF_ASN_BYTES_PERCENT`, - `SWWAF_COUNTRY_BYTES_PERCENT`). `0` is a zero allowance: the client's first + and byte limit (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A + lowered limit is rounded down. `0` is a zero allowance: the client's first request breaks a limit and bans it. +- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` set the byte + limits' percentage alone, when it should differ from the request limits'. For + an AS number they list, they replace its percentage from + `SWWAF_ASN_LIMIT_PERCENT` and `SWWAF_ASN_LIMIT_PERCENT_URL`; for a country, + the one from `SWWAF_COUNTRY_LIMIT_PERCENT`. - `SWWAF_ASN_LIMIT_PERCENT_URL` names a text file of `AS:percent` lines, so one abuse-source list can be shared by every `smallwebwaf` in the fleet. It is - fetched and refreshed like the blocklists. + fetched and kept like a blocklist (see "Reputation"). Where it and + `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. - A client's limit percentage is the lowest of those that apply to it: its AS - number's, its country's, or a reputation hit's `limit:`. It is 100 if - none applies. Clients given a lower percentage are banned after fewer - requests, first for a while and then, for repeated abuse, permanently. + number's, its country's, or a reputation hit's `limit:`. A client the + lookup cannot place gets `SWWAF_UNKNOWN_LIMIT_PERCENT` in place of an AS + number's or a country's. The percentage is 100 if none applies. Clients given + a lower percentage are banned after fewer requests, first for a while and + then, for repeated abuse, permanently. The log line of each counted request, + and the notes of each ban, say which setting lowered the limit. - No budget is shared by a whole country or AS number. One abuser could use it up and lock out everyone else there: a denial of service nobody chose, and the one this tool exists to prevent. Refusing a whole country is left to the @@ -506,15 +562,23 @@ scanners' user agents. It reads no request body. On a code forge, the example deployment, bodies carry what people write and publish, such as issue text and files saved in the web editor. The Core Rule Set takes the shell commands and code in them for attacks. The default gives up refusing an attack carried in a -body; the rule files, the limits and the bans still apply to the client that -sends it, and `SWWAF_WAF_BODY_LIMIT` switches body inspection on. -`docs/core-rule-set.md` says what the Core Rule Set reads and how it scores a -request. +body. The rule files, the limits and the bans still apply to the client that +sends it. +- The Core Rule Set inspects only a request that every earlier check let + through. It skips a path under `SWWAF_WAF_EXEMPT_PATHS` and a client in + `SWWAF_ALLOW_NETS`. An engine error on a request forwards the request, and is + logged and counted. - `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule - Set's own two tuning values, at its own defaults. A request whose score - reaches the threshold is a match. With the threshold `off`, nothing is a - match, and scores are still logged. + Set's own two tuning values, at its own defaults. Each level up runs more + rules, which find more attacks and take more ordinary requests for attacks. + Each rule that matches adds its score to the request's, and a rule rated + critical adds 5. A request whose score reaches the threshold is a match, so by + default one critical rule is enough. With the threshold `off`, nothing is a + match, and scores are still logged. A request with more than 1000 query + parameters or body fields adds 5 to its score, since Coraza reads only the + first 1000. Either way the log line gives the rule ids and the score, and a + `waf_block` alert is sent. - `SWWAF_WAF_DISABLED_RULES` switches off rule ids when an app trips a false positive. The default switches off six rules: - 920340, 920420 and 920640 refuse a request for the type of its body, when @@ -531,25 +595,29 @@ request. default, so name these six ids in it to keep them off. - `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. Form data - and multipart bodies are read up to the size, the rest streaming on without - being held in memory; JSON and XML bodies are read only when no larger than + and multipart bodies are read up to the size, and the rest streams on without + being held in memory. JSON and XML bodies are read only when no larger than the size, since those cannot be read in part. Any other body is not read: the Core Rule Set would read it as form data, where binary content such as a git - push trips rules written for text. Body inspection suits apps whose forms - carry no code. `docs/gitea.md` records what it refuses in front of gitea. + push trips rules written for text. A body the Core Rule Set cannot parse adds + 5 to the score, and so does a multipart body cut by the limit in the middle of + a part's header. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out + to send the part that is read. Body inspection suits apps whose forms carry no + code; `docs/gitea.md` records what it refuses in front of gitea. - `SWWAF_TRAP_PATHS` names paths the app never serves and only scanners ask for, such as `/wp-login.php` in front of gitea. A request for one is a clear sign of attack. This is the short form of a `path` rule with the `ban` action, for - deployments that mount no rule files. -- `SWWAF_ERROR_BURST_THRESHOLD` bounds the refusals per client per minute after - a rule file match, a trap path, a Core Rule Set match, or a missing or wrong - token. More than this breaks a limit (see "Bans"). A client trying one attack - or token after another is refused many times a minute; a person rarely more - than a few times. Answers the app gives are not counted, since they do not - tell a scanner from an ordinary client. In front of gitea, registry and git - clients get `404` and `401` by design, hundreds of times a minute. -- A Core Rule Set engine error on a request forwards the request, and is logged - and counted. + deployments that mount no rule files. A trap path is matched whole, without + the query, against the path as sent and once percent-decoded: `/wp-login.php` + matches `/wp-login.php?x=1` and `/%77p-login.php`, and does not match + `/wp-login.php/` or `/blog/wp-login.php`. +- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute for a rule + file match, a trap path, a Core Rule Set match, or a missing or wrong token. + More than this breaks a limit (see "Bans"). A client trying one attack or + token after another is refused many times a minute; a person rarely more than + a few times. Answers the app gives are not counted, since they do not tell a + scanner from an ordinary client. In front of gitea, registry and git clients + get `404` and `401` by design, hundreds of times a minute. `smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of @@ -609,7 +677,7 @@ every app, and no setting undoes them: 6. Responses are not inspected. A raw file from a repository, such as a shell script, looks to the response rules like source code leaking from the server. -### Ban response and scope +### Ban lengths, response and scope `SWWAF_BAN_RESPONSE` is `403` by default, which makes a ban easy to recognise when debugging; `429` and `close` are the alternatives. Behind traefik, `close` @@ -625,11 +693,13 @@ account or an engine of the operator's own. The list best suited to be on by default, Spamhaus DROP, must not be fetched more than once an hour, and Spamhaus may block an address that does. Each `smallwebwaf` fetches its own copy, so several apps on one host, sharing one address, would often fetch less than an -hour apart. `docs/reputation.md` says how each source is asked and what is done -with the answer. +hour apart. - Blocklists (`SWWAF_BLOCKLIST_URLS`) are text files of addresses and netblocks, - such as the Spamhaus DROP list. Spamhaus's terms for it, on its DROP page: + one to a line, as the Spamhaus DROP list is written. Anything after a `;` or + `#` on a line is ignored. An IPv4-mapped address such as + `::ffff:192.0.2.0/120` is read as the IPv4 netblock it stands for. Spamhaus's + terms for DROP, on its DROP page: - automated downloads must be at least one hour apart, and once a day is more than enough in most cases; - a product that uses it must credit The Spamhaus Project and keep the @@ -638,95 +708,174 @@ with the answer. So `SWWAF_BLOCKLIST_REFRESH` is never less than `1h`, and the last good copy is kept whole, comment lines included, on failure and across restarts. +- Each list is fetched again `SWWAF_BLOCKLIST_REFRESH` after its last try, one + list at a time. `reputation.json` keeps when each was last tried, so a restart + does not fetch a list again before its time. A fetch fails when: + - the server answers other than `200`; + - the fetch does not finish within a minute; + - the list is longer than 16 MiB; + - a line is not an address or netblock, or, for the AS number file, not + `AS:`. + + The log, the alerts, the ban notes and the metrics name each list by its + URL, so a secret stays out of it. + - `SWWAF_BLOCKLIST_ACTION` is `deny` by default, which refuses every request from a listed address and suits lists of networks that send nothing legitimate, such as DROP. `limit:` gives listed clients that percentage of every limit, and suits a list of addresses shared with ordinary - visitors, such as Tor exits. `log` only notes the hit. -- DNS blocklists (`SWWAF_DNSBL_ZONES`) meant for mail list residential ranges - and would block ordinary visitors, so they should not be used. Spamhaus zones - need their keyed query service, given as a zone name containing the key. - `SWWAF_DNSBL_RESOLVER` names a resolver, since public resolvers are refused by - several list operators. -- AbuseIPDB is asked only about clients that have already committed one offence, - so the daily budget (`SWWAF_ABUSEIPDB_DAILY_BUDGET`, under the free tier's - 1000 checks a day) is spent on suspects. + visitors, such as Tor exits. `log` only notes the hit in the log line and + raises an alert. - A CrowdSec engine on the host (`SWWAF_CROWDSEC_LAPI_URL`) has its decision - list pulled on a schedule. It is kept like a fetched blocklist, and a client - on it is banned (see "Bans"). This is how a fleet-wide blocklist can arrive - without `smallwebwaf` depending on CrowdSec. + list fetched every minute, with `SWWAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key` + header. The list is kept like a fetched blocklist, and a client on it is + banned (see "Bans"). Only decisions of the type `ban` on an address or a + netblock are used; any other, such as a captcha or a country, is left out. A + decision lasts for the `duration` it had left when the engine answered. A + fetch fails on an answer other than `200`, a redirect included, on one not + finished within a minute, on one longer than 16 MiB, and on one that is not a + list of decisions. This is how a fleet-wide blocklist can arrive without + `smallwebwaf` depending on CrowdSec. +- Each DNS blocklist (`SWWAF_DNSBL_ZONES`) is asked about a client's own address + in the background, the first time the client is seen; the request goes on at + once, and so does every request from the client until the zone has answered. + The name asked about is the one RFC 5782 gives: the numbers of an IPv4 address + reversed, or the 32 hex digits of an IPv6 address reversed, so `192.0.2.99` is + asked about as `99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` + lists the client, but one in `127.255.255.0/24` is Spamhaus refusing the + query. A name that does not exist does not list the client. These are + failures, after which the zone is not asked again for a minute: + - an answer in `127.255.255.0/24`; + - an answer outside `127.0.0.0/8`; + - a resolver error; + - no answer within `SWWAF_REPUTATION_TIMEOUT`. + + DNS blocklists made for mail servers list residential ranges, and would + block ordinary visitors, so they should not be used. Spamhaus zones need + their keyed query service, given as a zone name containing the key, which + `smallwebwaf` shows as `********` wherever it names the zone; only + `reputation.json` keeps the zone with its key. `SWWAF_DNSBL_RESOLVER` names + a resolver, since public resolvers are refused by several list operators. + +- AbuseIPDB (`SWWAF_ABUSEIPDB_KEY`) is asked, at its check endpoint, for the + abuse confidence score of a client, from 0 to 100. Only a client that has + already committed one offence is checked, so the daily budget + (`SWWAF_ABUSEIPDB_DAILY_BUDGET`, under the free tier's 1000 checks a day) is + spent on suspects. The check runs in the background at the client's next + request that no ban refuses, so no request waits; the score applies to the + client's later requests. For an IPv6 client, AbuseIPDB is asked about the + address of the request that set off the check, and the score is used for the + whole group. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit. The + budget is counted in UTC days and kept in `reputation.json`, so a restart does + not make it whole again; the check that spends the last of it is logged and + alerted, and no client is checked until the day ends. A check fails on an + answer other than `200`, on one without a score, and on no answer within + `SWWAF_REPUTATION_TIMEOUT`; after a failure no client is checked for a minute. + The key is sent in the `Key` header and nowhere else, and is never logged. - `SWWAF_REPUTATION_ACTION`, for DNS blocklist and AbuseIPDB hits, is `limit:25` by default. Such verdicts are less certain than a blocklist, so a listed client gets a quarter of every limit rather than a refusal. -- Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, with a fixed - bound on their number. A source that is down or over quota gives no verdict. - The service continues, with one `source_failure` alert per cooldown. +- Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and + in `reputation.json`, with the fixed bounds under "Fixed limits" in + `docs/settings.md`. A source that is down or over quota gives no verdict. The + service continues, with one `source_failure` alert per cooldown. ### Alerting -- Alerts go to a webhook as JSON (`docs/alerts.md` gives the object), and to - Slack and ntfy as formatted messages. `SWWAF_ALERT_EVENTS` chooses the event - types sent. -- `SWWAF_ALERT_COOLDOWN`: the same event is not repeated within this time for - the same netblock, source or file. For an anomaly, it is the same thing - counted that must match: one client, netblock, AS number or named netblock, or - the whole service. A count of suppressed repeats is included in the next one. -- `SWWAF_ALERT_MAX_PER_HOUR`: beyond this, alerts are rolled into one summary - per hour, so a wide attack cannot flood the channel. -- A destination that is down is retried with backoff from a bounded queue, the - oldest alert dropped first, and the drops counted in the metrics. +- Alerts go to a webhook as JSON, and to Slack and ntfy as formatted messages. + `docs/alerts.md` lists the events, the JSON object and the messages. + `SWWAF_ALERT_EVENTS` chooses which of the seven event types are sent; the + hourly `summary` is sent whatever it says. The bans an admin makes raise no + alert. In `observe` mode, a request that would have made a ban still raises + the ban's alert, with `mode` set to `observe` in its `detail`. +- A repeat is an alert with the same event, and the same netblock, source, file + or anomaly scope, as one sent less than `SWWAF_ALERT_COOLDOWN` before. An + anomaly scope is the thing counted: one client, netblock, AS number or named + netblock, or the whole service. A repeat is held back and counted, and the + next such alert sent gives the count. +- Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and + counted by event, so a wide attack cannot flood the channel. Once the hour + ends, one alert with the event `summary` gives those counts. It also counts + the repeats that a cooldown held back and that no later alert reported. +- Each destination has a queue of at most 1000 alerts, sent one at a time, so a + slow destination holds up nothing; when it is full, the oldest alert is + dropped, and the drops are counted in the metrics. A `2xx` answer takes an + alert. A `4xx` other than `408` and `429` refuses it, and it is dropped. + Anything else is a failure: the alert is sent again a second later, and twice + as long after each further failure, up to a minute. `alerts.json` keeps the + queues and cooldowns across a restart. ### Anomaly thresholds These only alert; nothing is blocked. None is set by default, and a threshold left unset sends no alert. An alert needs a destination only the operator can supply, and what counts as unusual depends on each service's normal traffic, -which the metrics show. They are what reveals a widely distributed scraper, -thousands of addresses at low rates each, which the per-client limits do not -see; a low `SWWAF_ASN_LIMIT_PERCENT` for its AS number is the response. +which the metrics show. They reveal a widely distributed scraper, thousands of +addresses at low rates each, which the per-client limits do not see. A low +`SWWAF_ASN_LIMIT_PERCENT` for its AS number is the response. - Requests and bytes are counted per minute and per hour, for each client, each surrounding netblock, each AS number, the whole service, and each named netblock in `SWWAF_WATCH_NETS`. -- Anomaly counting includes clients in `SWWAF_ALLOW_NETS` and - `SWWAF_RATE_LIMIT_EXEMPT_NETS`, since an exempt client misbehaving is worth - knowing about. At most 20,000 counters are kept, in memory and in - `alerts.json`. +- Every request but the health check is counted, whatever was done with it, + clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` included, + since an exempt client misbehaving is worth knowing about. The counters are + kept in memory and in `alerts.json`, with a fixed bound. ## Rule files `smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR` and checks each request against them, unless the client is in `SWWAF_ALLOW_NETS`. -`docs/rules.md` gives the format, the targets and actions, the errors, and the -default file. +`docs/rules.md` gives the format, the targets and actions, and the default file. - The files are plain text meant to be edited by hand, so that a new scanning pattern seen in the request log becomes a rule in one line. `smallwebwaf` watches the directory: a file edited, added or removed there takes effect - while it runs. + while it runs. It reads the files again once the directory has had no change + for 2 seconds, so that a file still being written is read whole. A file whose + name starts with `.`, such as an editor's lock file, is not a rule file. +- Files are read in name order (`00-default.rules` before `50-gitea.rules`), + rules in line order. Matching stops at the first rule that refuses. - A rule's regex is Go's RE2 syntax. It has no backreferences or lookaround; in exchange matching time is linear in the input, so no rule can be made to stall the proxy. - `path` and `uri` are matched both as received and once percent-decoded, so an - encoded probe cannot slip past. Request bodies are not available to rule - files; reading them is the Core Rule Set's job. -- `block` refuses the request without a ban; `ban` makes the request a clear - sign of attack (see "Bans"). `ban` is for requests no real visitor sends, - anchored at the site root. A file of the same name deeper in a site can be - ordinary content, such as a file in a gitea repository. A visitor banned by - mistake is let back in by lifting the ban. -- At start, a file that does not parse, a regex that does not compile, or a - duplicate id stops the process with a message naming the file and line. So - does a `SWWAF_RULES_DIR` that does not exist. While running, the same faults - leave the rules as they were, and the log and one `file_error` alert name the - file and line. An empty directory is not an error. + encoded probe cannot slip past. A header sent more than once is matched with + its values joined by `, `; a header not sent is empty text. `Host` and + `Transfer-Encoding` cannot be targets, since Go's HTTP server takes them out + of every request. Request bodies are not available to rule files; reading them + is the Core Rule Set's job. +- `log` only notes the match in the request log. `block` refuses the request + without a ban; `ban` makes the request a clear sign of attack (see "Bans"). + `ban` is for requests no real visitor sends, anchored at the site root. A file + of the same name deeper in a site can be ordinary content, such as a file in a + gitea repository. A visitor banned by mistake is let back in by lifting the + ban. +- At start, these stop the process with a message naming the file and line: + - a line that does not parse; + - a regex that does not compile; + - a rule for `Host` or `Transfer-Encoding`; + - a duplicate id. + + So does a `SWWAF_RULES_DIR` that does not exist or cannot be read. While + running, the same faults leave the rules as they were, and the log and one + `file_error` alert name the file and line. Once the file is fixed, it is + read again. An empty directory is not an error: the log says that no rules + were loaded. + - The image ships a default file, kept short and limited to patterns that are - wrong for every app, with its path rules anchored at the site root. Anything - app-specific belongs in a file the app's Dockerfile adds or the deployer - mounts. Rules meant to be edited while `smallwebwaf` runs belong in a mounted + wrong for every app, with its path rules anchored at the site root. In front + of gitea, `/.env` is a probe, while + `///src/branch/main/.env.example` is a file in a repository that + any visitor may open. Anything app-specific, such as a rule banning WordPress + probes in front of gitea, belongs in a file the app's Dockerfile adds or the + deployer mounts. A directory mounted over `SWWAF_RULES_DIR` replaces the + default file; single files mounted into it add to it. +- Rules meant to be edited while `smallwebwaf` runs belong in a mounted directory, not a mounted file. Many editors save a file by replacing it, and a container keeps seeing the old one when the file alone is mounted. The lookup - database is mounted the same way, for the same reason. + database is mounted the same way, for the same reason. To run without rule + files, mount an empty directory or set `SWWAF_RULES_ENABLED=false`. ## Persistent state @@ -735,65 +884,94 @@ of it, so an orderly stop loses nothing. The one exception is log lines still waiting to be sent to `SWWAF_LOG_REMOTE_URL`, which stdout has already carried. The files are meant for people as well: an admin can read or edit them at any time, and the running `smallwebwaf` takes the edit in. `docs/state-files.md` -lists the files, what each holds, their sizes, and how an edit is taken in. +lists the files, what each holds, and their sizes. - Each file holds one kind of state, as JSON with: - a top-level `version` number; - entries sorted by client address; - times in RFC 3339 UTC; - durations and sizes as plain numbers, with the unit in the field name. -- `bans.json` is indented, so that a person can open it in an editor, find an - address, and remove or add an entry. `clients.json` and `lookups.json` put - each entry on one line, which halves their size and lets `grep` show - everything about one client. +- `bans.json`, `reputation.json` and `alerts.json` are indented, so that a + person can open `bans.json` in an editor, find an address, and remove or add + an entry. `clients.json` and `lookups.json` put each entry on one line, which + halves their size and lets `grep` show everything about one client. - Clients that were never banned are kept too, so every client's history survives a restart. -- Each file is rewritten whole, so each has a bound given by its setting. - Writing a file of these sizes takes well under a second, in the background, so - rewriting whole files needs nothing cleverer. +- Each file has a bound, so it can be rewritten whole. Writing a file of these + sizes takes well under a second, in the background, so rewriting whole files + needs nothing cleverer. - A write serialises from a snapshot taken under the lock, so requests are not held up. It goes to a temporary file in the same directory, which is synced - and renamed over the real name. A crash at any point leaves either the old - complete file or the new complete file, never a partial one. Before writing a - file, `smallwebwaf` takes in any edit made to it on disk, so an admin's edit - is never overwritten. -- At start, a file that does not parse, or has an unknown `version`, stops the - process with a message naming the file and position. Starting with empty state - would silently forgive every repeat offender. Since writes are atomic, a - broken file can only come from a hand edit, which the editor should hear - about. A missing file means empty state, and is normal on first run. -- While running, an edit that parses replaces what `smallwebwaf` held for that - file. An edit that does not parse never stops the running `smallwebwaf`. It - keeps the state it has and sets the edited file aside as `.bad`. It - writes the file again from memory, and names the file and the position of the - error in the log and in one `file_error` alert. + and renamed over the real name, and the directory is synced. A crash at any + point leaves either the old complete file or the new complete file, never a + partial one. Before writing a file, `smallwebwaf` takes in any edit made to it + on disk, so an admin's edit is never overwritten. +- At start, the files are read back. Active bans go into an in-memory lookup by + netblock, everything else into maps. Counter buckets and cache entries whose + time has passed are discarded. A missing file means empty state, and is normal + on first run. A file that does not parse, or has an unknown `version`, stops + the process with a message naming the file and position; so does an entry + without a field it needs, a ban whose cause is not one of the four, or a ban + whose `expires` is not after its `start`. Starting with empty state would + silently forgive every repeat offender. Since writes are atomic, a broken file + can only come from a hand edit, which the editor should hear about. +- While running, `smallwebwaf` notices a file that is edited, replaced or added. + It tells its own writes from an admin's by comparing the file with what it + last read or wrote. An edit that parses is taken in at once: what the file + says replaces what `smallwebwaf` held for that file. Removing a ban's entry + from `bans.json` lifts the ban; adding an entry bans; setting `lifted` lifts + it and keeps it. A change `smallwebwaf` made after the admin opened the file, + such as a new ban, is lost when the admin saves over it. Most editors warn + when a file changed on disk while it was open. A file that is removed is + written again at its next write. +- An edit that does not parse never stops the running `smallwebwaf`. It keeps + the state it has and sets the edited file aside as `.bad`, so the edit + is kept. It writes the file again from memory, and names the file and the + position of the error in the log and in one `file_error` alert. The admin + mends the `.bad` file and moves it back. - A write that fails leaves the state correct in memory. It is logged, counted in the metrics and sent as one `file_error` alert per cooldown, and retried at the next write. -- A hard kill loses the changes since the last write. Losing the volume loses - ban history, not service. +- A hard kill loses the changes since the last write: ban changes from the last + `SWWAF_STATE_WRITE_DELAY`, everything else from the last + `SWWAF_STATE_COUNTER_INTERVAL`. Losing the volume loses ban history, not + service. ## Request log Every request gets one JSON object on one line on stdout, refused ones included, apart from those Go's HTTP server ends before `smallwebwaf` sees them (see "Size -and time limits"). `docs/request-log.md` lists the fields and describes the -syslog sending. Bodies are never logged. Query strings are logged as received; -an app that carries secrets in query strings needs that fixed in the app. The -process's own messages share the stream as JSON lines with `"type":"process"`; -request lines carry `"type":"request"`. +and time limits"). `docs/request-log.md` lists the fields. Bodies are never +logged, and no header but those listed there. Query strings are logged as +received; an app that carries secrets in query strings needs that fixed in the +app. The process's own messages (start, the settings, stop, errors) share the +stream as JSON lines with `"type":"process"`; request lines carry +`"type":"request"`. When `SWWAF_LOG_REMOTE_URL` is set, the same lines are also sent there as RFC 5424 syslog records. So a deployment can stop depending on docker's log handling -while `docker logs` keeps working. Sending happens on its own goroutine from a -bounded buffer: an unreachable or slow server never delays a request and never -stops stdout. UDP gives no delivery signal at all and is offered only for -compatibility. +while `docker logs` keeps working. + +- Over UDP each record is one datagram; a line too long for one datagram is + dropped alone. Over TCP and TLS each record follows its length in bytes and a + space. UDP gives no delivery signal at all and is offered only for + compatibility. +- The record carries the facility from `SWWAF_LOG_REMOTE_FACILITY`, the severity + informational, the time, the host's name and the app name from + `SWWAF_LOG_REMOTE_APP_NAME`. +- Sending happens on its own goroutine from a buffer of + `SWWAF_LOG_REMOTE_BUFFER` lines, so a server that is slow or down never holds + up a request and never stops stdout. When the buffer is full, the oldest line + is dropped and counted in the metrics. A failed connection is tried again a + second later, and twice as long after each further failure, up to a minute. +- As `smallwebwaf` stops, it spends at most two seconds sending the lines still + waiting, and drops the rest; stdout has carried them. ## Metrics endpoint `GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request -carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. No metric +carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. Every +metric carries the label `instance`, set to `SWWAF_INSTANCE_NAME`. No metric carries a client IP address as a label. Per-address questions are answered by the request log and `GET /_smallwebwaf/clients/`. @@ -803,7 +981,7 @@ A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself: it answers it and never passes it to the app. The prefix carries the tool's name, so it takes no path an app uses. Admins and scrapers reach these endpoints through traefik, like any other request. `docs/admin-endpoints.md` lists the -endpoints and what each answers. +endpoints, what each takes and what each answers. - `GET /_smallwebwaf/healthz` answers `200` with `ok` to anyone, without a token, while `smallwebwaf` is running. It does not ask the app; the @@ -819,28 +997,54 @@ endpoints and what each answers. refused, and each request counts toward the client's limits. A missing or wrong token is answered `401`, in `SWWAF_MODE=observe` too, and counts toward the error burst. A client guessing tokens is therefore banned once it passes - `SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute. A client in - `SWWAF_ALLOW_NETS` skips the checks but still needs the token. + `SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute. +- A ban made through the endpoints starts at once, with the cause `admin`. + Lifting through the endpoints lifts every active ban covering the address + given. ## Deployment `smallwebwaf` runs inside the app's own container: the app's Dockerfile builds -`FROM` the `smallwebwaf` image. Each app remains one container. An app is -hardened by changing its base image. It can add packages from nixpkgs. +`FROM` the `smallwebwaf` image. Each app is one container. An app is hardened by +changing its base image. It can add packages from nixpkgs. `docs/image.md` lists +what the image holds and how each part is pinned. -The image holds: - -- Ubuntu 26.04 LTS, with `ca-certificates`, `nix-bin` and `runit` from Ubuntu; -- nixpkgs, fixed at one release; -- `runsvinit` as the entrypoint; -- the `smallwebwaf` binary, a user of its own (uid and gid 65532), and its runit - service; -- its state and rules directories; -- `EXPOSE 8080` and the health check below. - -Every input to the image is pinned by digest or hash, so that an app built on it -gets the same packages each time. `docs/image.md` says how each part is pinned, -and why. +- The image is Ubuntu's newest long-term support release, pinned by digest, and + moves to the next LTS release when that ships. Its packages come from Ubuntu's + snapshot service, the archive as it was at the moment the Dockerfile names, + rather than the live archive, whose packages change with every update. The + snapshot is never older than the pinned Ubuntu image, since older packages can + need older versions of packages the image holds, and it moves forward with the + image's digest. The Dockerfile names the hash of each of the snapshot's + `InRelease` files, which cover every package list and so every package, and + the build checks them before installing. The hashes are the amd64 archive's, + so the image is built for amd64 only. +- The Ubuntu image has no CA certificates. Without them Nix cannot download + packages and `smallwebwaf` cannot reach GeoJS, so `ca-certificates` is + installed by name. The snapshot itself is fetched over HTTPS with the CA + certificates of the Go image that builds `smallwebwaf`, itself pinned by + digest. +- nixpkgs comes from a release file on `releases.nixos.org`, named with its + hash, not from a GitHub archive, whose bytes can change. Root uses Nix + directly; no Nix daemon runs, and since `nix-bin` creates no build users, + `build-users-group =` in `/etc/nix/nix.conf` lets root build without them. + Root's Nix profile is last on the `PATH`, so that no package hides the image's + own commands; busybox's `sv`, for one, looks for services elsewhere. An app + built on the image gets the same packages each time. Security fixes from + Ubuntu and nixpkgs come with a newer `smallwebwaf` image, which an app takes + by changing the digest in its `FROM` line; the image moves to each new nixpkgs + release, every six months. +- runit supervises the two processes, with `runsvinit` as the entrypoint, as the + owner's code style guide asks for service containers. No package carries + `runsvinit`, so the image builds it from its archived source, unchanged since + 2015, at a fixed commit hash, with a `go.mod` the build writes. `runsvinit` + starts runit's `runsvdir`, which runs the `run` script of each directory under + `/etc/service` and runs it again whenever it exits. Ubuntu's `sv` looks for + services there too. +- The image's `run` script for `smallwebwaf` waits one second and makes + `SWWAF_STATE_DIR` and every file in it belong to the `smallwebwaf` user (uid + and gid 65532). Then it starts `smallwebwaf` as that user with runit's + `chpst`. Every `run` script, the app's included, is a bash script. It starts with `#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main` @@ -917,15 +1121,16 @@ around `smallwebwaf` by anything that reaches the container. ### State -`smallwebwaf` keeps its state files in `SWWAF_STATE_DIR`, beside the app's own -data directories. Without a volume there, the files survive a restart of the -container and are lost when a deploy replaces it. A volume mounted at -`/var/lib/smallwebwaf`, named or a host directory, keeps them across deploys. -The `run` script makes it and every file in it belong to the `smallwebwaf` user, -so a host directory mounted there needs no change of owner. The volume is -separate from the app's, holds a few tens of MiB at most with the defaults, and -needs no backup beyond the host's own. The image declares no volume, since every -app image built on it would inherit it. +`smallwebwaf` keeps its state files in `SWWAF_STATE_DIR`, a directory of its own +beside the app's data directories, belonging to the `smallwebwaf` user. Without +a volume there, the files survive a restart of the container and are lost when a +deploy replaces it. A volume mounted at `/var/lib/smallwebwaf`, named or a host +directory, keeps them across deploys. A host directory mounted there needs no +change of owner, since the `run` script gives it to the `smallwebwaf` user at +each start; files an earlier owner left in it can be read and replaced. The +volume is separate from the app's, holds a few tens of MiB at most with the +defaults, and needs no backup beyond the host's own. The image declares no +volume, since every app image built on it would inherit it. ### Tokens @@ -934,6 +1139,7 @@ is out of the app's reach only while the `smallwebwaf` user alone can read the file. The operator makes the file on the host, owned by uid 65532, the `smallwebwaf` user, with mode `0400`, and mounts the directory that holds it into the container read-only. The container sees the same owner and mode. +"Settings given as files" in `docs/settings.md` shows the commands. ### Forwarded headers diff --git a/docs/admin-endpoints.md b/docs/admin-endpoints.md index 85563f0..dde1d67 100644 --- a/docs/admin-endpoints.md +++ b/docs/admin-endpoints.md @@ -1,26 +1,20 @@ # Admin endpoints -`smallwebwaf` answers these requests itself, on the app's own address and -through traefik like any other request. Apart from the health check, each needs -a token, sent as `Authorization: Bearer `, and goes through every check -any other request goes through. It is answered where another request would be -passed to the app: a banned client stays refused, and each request counts toward -rate limiting. A missing or wrong token is answered `401`, in `observe` mode -too. Such a refusal counts toward the error burst, so a client guessing tokens -is soon banned. While a token is unset, its endpoints answer `404`. +The endpoints under `/_smallwebwaf/`, what each takes and what each answers. +"Admin endpoints" in [`SPEC.md`](../SPEC.md) says which checks they go through +and how their tokens work. -- `GET /_smallwebwaf/healthz`: `200` and `ok`, to anyone, before any check and - without asking the app. +- `GET /_smallwebwaf/healthz`: `200` and `ok`, to anyone. - `GET /_smallwebwaf/metrics`: the metrics, for `SWWAF_METRICS_TOKEN` (see [`metrics.md`](metrics.md)). - `GET /_smallwebwaf/bans`: every ban held, under `bans`, each as `bans.json` lists it. Needs `SWWAF_ADMIN_TOKEN`, as do the three below. - `POST /_smallwebwaf/bans`: bans a netblock. The body is a JSON object of `netblock`, `duration` and, if you like, `reason`. `netblock` is a netblock - such as `203.0.113.0/24`, or a client's address. A client's address bans its - IPv4 netblock (`SWWAF_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a - duration such as `1h` or `7d`, or `permanent`. The ban starts at once, with - the cause `admin`. A bad body is answered `400`, saying what is wrong: + such as `203.0.113.0/24`, or a client's address, which bans its IPv4 netblock + (`SWWAF_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a duration such + as `1h` or `7d`, or `permanent`. A bad body is answered `400`, saying what is + wrong: - one that is not such an object, or has another field; - one longer than 4 KiB; - one that names an IPv4-mapped netblock, or an address with a zone. @@ -28,8 +22,27 @@ is soon banned. While a token is unset, its endpoints answer `404`. address ``. It answers `404` when none is active. - `GET /_smallwebwaf/clients/`: what `smallwebwaf` knows of the client. Under `client`, its counters and history as `clients.json` holds them, or - `null` when it has never been seen. Under `bans`, every ban covering the - address, with its notes. + `null` when `clients.json` does not hold it. Under `bans`, every ban covering + the address, with its notes. -An admin whose own address is banned lifts that ban by editing `bans.json` (see -[`bans.md`](bans.md)). +## Examples + +With `SWWAF_ADMIN_TOKEN` set, and the token in `$TOKEN`: + +```sh +# Every ban. +curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans + +# Ban 203.0.113.0/24 for seven days. +curl -H "Authorization: Bearer $TOKEN" \ + --json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \ + https://app.example/_smallwebwaf/bans + +# Lift the bans on 203.0.113.9. +curl -H "Authorization: Bearer $TOKEN" -X DELETE \ + https://app.example/_smallwebwaf/bans/203.0.113.9 + +# What smallwebwaf knows of 203.0.113.9. +curl -H "Authorization: Bearer $TOKEN" \ + https://app.example/_smallwebwaf/clients/203.0.113.9 +``` diff --git a/docs/alerts.md b/docs/alerts.md index 6577753..1da7174 100644 --- a/docs/alerts.md +++ b/docs/alerts.md @@ -1,17 +1,17 @@ # Alerts -Each alert goes to each destination you name. `SWWAF_ALERT_WEBHOOK_URL` gets it -as one JSON object. `SWWAF_ALERT_SLACK_WEBHOOK_URL` and `SWWAF_ALERT_NTFY_URL` -get a message made from it. The settings are under "Alert destinations" in -[`settings.md`](settings.md). An alert is one of these events, and is sent when -its event is in `SWWAF_ALERT_EVENTS`: +The alert events, the JSON object the webhook gets, and the messages Slack and +ntfy get. "Alerting" in [`SPEC.md`](../SPEC.md) says when an alert is sent, held +back or dropped. + +## Events - `ban`: a ban `smallwebwaf` made. - `permanent_ban`: a permanent ban it made, or an existing ban made permanent by a request. -- `waf_block`: the Core Rule Set scored a request at or over the threshold. In - `block` mode the request was refused. In `detect` mode it was not, and the - alert's `detail` gives `mode`. +- `waf_block`: the Core Rule Set scored a request at or over the threshold. The + request was refused unless the alert's `detail` gives `mode`: `detect` or + `observe`. - `anomaly`: a count over an anomaly threshold. - `reputation_hit`: a request from a client on a blocklist, the CrowdSec decision list or a DNS blocklist, or with an AbuseIPDB score that is a hit. @@ -25,12 +25,7 @@ its event is in `SWWAF_ALERT_EVENTS`: - a rule file or state file edit with an error; - a state file that cannot be written; - a replacement lookup database that cannot be read. -- `summary`: the hourly summary described under "Repeats and the hourly cap" - below. - -The bans you make raise no alert. In `observe` mode, a request that would have -made a ban still raises the ban's alert, with `mode` set to `observe` in its -`detail`. +- `summary`: the hourly summary of alerts held back. ## The JSON object @@ -81,24 +76,26 @@ on one line: } ``` -- `instance` is `SWWAF_INSTANCE_NAME`. `time` is when the alert was raised. - `event` is one of the events above. -- `client` is the address whose request raised the alert. `netblock` is the - ban's netblock, the netblock an anomaly counted, or the client's group. Both - are empty for `source_failure` and `file_error`. `asn`, `as_name` and - `country` are the client's as known when the alert was raised. They are empty - when GeoJS had not answered yet. -- `reason` is a short sentence, such as +- `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`: + one of the events above. +- `client`: the address whose request raised the alert. `netblock`: the ban's + netblock, the netblock an anomaly counted, or, for the other events, the + client as rate limiting counts it: an IPv4 address as a /32, or an IPv6 group. + Both are empty for `source_failure` and `file_error`. +- `asn`, `as_name` and `country`: the client's as known when the alert was + raised. They are empty when GeoJS had not answered yet. +- `reason`: a short sentence, such as `requests per minute over the limit of 1000` or `listed by a blocklist`. -- `detail` is what is particular to the event: - - for a ban, its `cause`, `ban_expires` and `notes` (see - [`bans.md`](bans.md) for the notes); +- `detail`: what is particular to the event: + - for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed + under `bans.json` in [`state-files.md`](state-files.md)); - for an anomaly, the `scope`, `window`, `kind`, `count` and `threshold`; - for a reputation hit, the `source` and, for AbuseIPDB, the `score`; - for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`; - for a source failure, the `source` and `error`; - - for a file error, the `file` and `error`. -- `suppressed_repeats` is how many repeats the cooldown held back before this + - for a file error, the `file` and `error`; + - `mode`, when the request was not refused: `detect` or `observe`. +- `suppressed_repeats`: how many repeats the cooldown held back before this alert. ## Slack and ntfy @@ -128,22 +125,3 @@ The priority and tag by event: - `reputation_hit`: priority `low`, tag `label`. - `source_failure` and `file_error`: priority `high`, tag `warning`. - `summary`: priority `default`, tag `bar_chart`. - -## Repeats and the hourly cap - -A repeat is an alert for the same event on the same netblock, source, file or -thing counted, less than `SWWAF_ALERT_COOLDOWN` after the last. It is held back -and counted, and the next alert sent gives the count. Past -`SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and counted -by event. Once the hour ends, one alert with the event `summary` gives those -counts. It also counts the repeats that a cooldown held back and that no later -alert reported. - -## Sending - -Each destination has a queue of at most 1000 alerts, sent one at a time, so a -slow destination holds up nothing. A `2xx` answer takes an alert. A `4xx` other -than `408` and `429` refuses it, and it is dropped. Anything else is a failure: -the alert is sent again a second later, and twice as long after each further -failure, up to a minute. `alerts.json` keeps the queues and cooldowns across a -restart (see [`state-files.md`](state-files.md)). diff --git a/docs/bans.md b/docs/bans.md deleted file mode 100644 index bd0f4e6..0000000 --- a/docs/bans.md +++ /dev/null @@ -1,84 +0,0 @@ -# Bans - -How bans are made, what they cover and how long they last is under "Bans" in -[`SPEC.md`](../SPEC.md); the settings are under "How long bans last and what -they cover" in [`settings.md`](settings.md). This file says what a ban's notes -hold, and how to make, keep or lift a ban. - -## The notes - -Each ban carries notes for deciding whether to lift it, drawn from what -`smallwebwaf` already knows. Nothing is looked up to fill them: - -- the AS number, AS name and country, added when the lookup answers (empty when - lookups are off); -- what was broken: the rule id and target that matched, or the trap path, or the - limit, its window and the count reached; -- the client's percentage of the limit, with the setting that gave it; -- the reputation sources the client is on; -- the request that caused the ban: time, method, host, path with its query - string, status and user agent, each text cut to 256 bytes; -- the netblock's requests since it was first seen, and how many this ban has - refused so far, kept up to date while the ban lasts; -- how many earlier bans the netblock had, by cause. - -## Managing bans by editing `bans.json` - -`bans.json` in the state directory holds every ban, indented to be read. A -permanent ban's `expires` is `null`. `smallwebwaf` watches the file and takes in -an edit as soon as it is saved (see [`state-files.md`](state-files.md)). - -To ban a netblock, add an entry with its `netblock`, `start` and `expires`. -`expires` is `null` for a ban that never ends. `reason`, `notes` and `cause` may -be left out; the cause is then `admin`: - -```json -{ - "version": 1, - "bans": [ - { - "netblock": "203.0.113.0/24", - "start": "2026-10-06T12:00:00Z", - "expires": null, - "reason": "probes for logins" - } - ] -} -``` - -To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause` -to `admin`. To lift a ban, add `"lifted"` with the time you lift it, such as -`"lifted": "2026-10-06T13:00:00Z"`. The ban then refuses nothing and does not -make the netblock's next ban longer. It is kept with its notes. To forget a ban, -delete its entry. A ban whose cause is `attack` becomes permanent at the first -request it refuses. One whose cause is `admin` or `crowdsec` does not. - -An edit that cannot be read does not stop the running `smallwebwaf`. It keeps -what it holds. At the file's next write it renames your file to `bans.json.bad`, -writes the file again from memory, and logs and alerts the error. Mend the -`.bad` file and move it back. - -## Managing bans through the admin endpoints - -With `SWWAF_ADMIN_TOKEN` set, and the token in `$TOKEN`: - -```sh -# Every ban. -curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans - -# Ban 203.0.113.0/24 for seven days. -curl -H "Authorization: Bearer $TOKEN" \ - --json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \ - https://app.example/_smallwebwaf/bans - -# Lift the bans on 203.0.113.9. -curl -H "Authorization: Bearer $TOKEN" -X DELETE \ - https://app.example/_smallwebwaf/bans/203.0.113.9 - -# What smallwebwaf knows of 203.0.113.9. -curl -H "Authorization: Bearer $TOKEN" \ - https://app.example/_smallwebwaf/clients/203.0.113.9 -``` - -See [`admin-endpoints.md`](admin-endpoints.md) for what each answers. An admin -whose own address is banned lifts that ban by editing `bans.json`. diff --git a/docs/core-rule-set.md b/docs/core-rule-set.md deleted file mode 100644 index 066e749..0000000 --- a/docs/core-rule-set.md +++ /dev/null @@ -1,31 +0,0 @@ -# The Core Rule Set - -The OWASP Core Rule Set 4.25.0 runs on each request after the rule files, unless -`SWWAF_WAF_MODE` is `off`. It reads the method, the URL with its query, and the -headers. Once `SWWAF_WAF_BODY_LIMIT` is set, it also reads a body that is form -data, multipart, JSON or XML. No answer is inspected. The settings are under -"Attack detection" in [`settings.md`](settings.md). The same section of -[`SPEC.md`](../SPEC.md) says why bodies are not read by default, and gives the -six changes `smallwebwaf` makes to the Core Rule Set, with the reason for each. - -Each rule that matches adds to the request's score. A score at or over -`SWWAF_WAF_ANOMALY_THRESHOLD` is a match. A match is refused with `403` while -`SWWAF_WAF_MODE` is `block`, and let through while it is `detect`. Either way -the log line gives the rule ids and the score, and a `waf_block` alert is sent. -A request with more than 1000 query parameters or body fields adds 5 to its -score, since Coraza reads only the first 1000. The Core Rule Set inspects only a -request that every earlier check let through. It skips a path under -`SWWAF_WAF_EXEMPT_PATHS` and a client in `SWWAF_ALLOW_NETS`. An engine error on -a request forwards the request, and is logged and counted. - -Body inspection suits apps whose forms carry no code. In front of a code forge -it refuses issue and comment text, wiki pages and files saved in the web editor -that hold shell commands or code. The log line names the rule ids, which -`SWWAF_WAF_DISABLED_RULES` can switch off. A body the Core Rule Set cannot parse -adds 5 to the score. So does a multipart body cut by the limit in the middle of -a part's header. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to -send the part that is read. Any other body, such as a git push, reaches the app -uninspected. - -[`gitea.md`](gitea.md) records what the Core Rule Set still refuses in front of -gitea, with and without body inspection. diff --git a/docs/gitea.md b/docs/gitea.md index d703385..46b8143 100644 --- a/docs/gitea.md +++ b/docs/gitea.md @@ -13,16 +13,19 @@ checked again when gitea or the Core Rule Set changes version. by it. - A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all but the largest repositories and slowest links. -- A push is a request, and so is every other upload: a git push, an LFS object, - a container image layer, a package file or a release attachment. At the - defaults, an upload larger than 100 MiB or taking more than 60 seconds is cut - off. The client is answered `413` for a body that is too large, before - anything reaches gitea when the request announces its size. It is answered - `408` for a body it sends too slowly, or `504` if gitea is too slow to take - it. The upload fails, and no one is banned for it. A gitea that takes large - uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and - `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does not - read an upload's body, which streams through without being held in memory. +- A push is a request, and so is every other upload: + - a git push; + - an LFS object; + - a container image layer; + - a package file; + - a release attachment. + + At the defaults, an upload larger than 100 MiB or taking more than 60 + seconds is cut off, and no one is banned for it. "Size and time limits" in + `SPEC.md` gives the answers the client gets. A gitea that takes large + uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and + `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. + - Archive download and blame or history pages are what scrapers hammer. The request limits do most of the work there. - An Actions runner that reaches gitea through `smallwebwaf` sends requests all @@ -32,8 +35,7 @@ checked again when gitea or the Core Rule Set changes version. jobs. It is then banned, the job it is running fails, and each repeat within a day makes the ban longer. Newer runners ask less often, but a busy one, or several on one address, can still pass it. Put the runners' addresses in - `SWWAF_RATE_LIMIT_EXEMPT_NETS`. That takes them out of the request and byte - limits, while the error burst, attack detection and bans still apply. + `SWWAF_RATE_LIMIT_EXEMPT_NETS`. ## What the Core Rule Set lets through @@ -55,10 +57,9 @@ the refusals in the next section: ## What the Core Rule Set still refuses -Each refusal answers only that request, with `403`, and bans no one by itself. -It counts toward the error burst, which a person does not reach this way. The -request log names the rule in `waf_rule_ids`, which `SWWAF_WAF_DISABLED_RULES` -can switch off. +Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts +toward). The request log names the rule in `waf_rule_ids`, which +`SWWAF_WAF_DISABLED_RULES` can switch off. ### A query string that reads as an attack @@ -151,9 +152,9 @@ carries it. A push can reuse a layer of an image docker has pushed to that gitea before. docker then asks gitea to take the layer from that image, naming it in the query parameter `from`, which 932260 checks. The request is refused when the image's -owner, or its name after the owner, starts with an entry on the list of -commands. Examples are `owner/python3-app`, `owner/docker-base`, and any image -of an owner named `ansible` or `docker-team`. +owner starts with an entry on the list of commands. So is one whose name after +the owner does. Examples are `owner/python3-app`, `owner/docker-base`, and any +image of an owner named `ansible` or `docker-team`. What happens next depends on docker's image store. A docker that keeps its images in containerd's image store fails the push, and fails each new try the diff --git a/docs/image.md b/docs/image.md index c208356..d2955c0 100644 --- a/docs/image.md +++ b/docs/image.md @@ -1,56 +1,36 @@ # What the image holds -The `smallwebwaf` image is the base of each app's own image. The requirements -are under "Deployment" in [`SPEC.md`](../SPEC.md); this file lists what the -image holds, how each part is pinned, and why. +The contents of the `smallwebwaf` image, and how each part is pinned. +"Deployment" in [`SPEC.md`](../SPEC.md) says why each part is there and why it +is pinned that way. ## Contents -- Ubuntu 26.04 LTS, the newest long-term support release of Ubuntu, pinned by - digest. The image moves to the next LTS release when that ships. -- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, installed from a dated - snapshot of its archive and checked by hash. The Ubuntu image has no CA - certificates; without them Nix cannot download packages and `smallwebwaf` - cannot reach GeoJS. -- Nix and nixpkgs, fixed at one release. Root uses Nix directly; no Nix daemon - runs. `nix-bin` creates no build users, so `build-users-group =` in - `/etc/nix/nix.conf` lets root build without them. -- runit, and `runsvinit` as the entrypoint, as the owner's code style guide asks - for service containers. No package carries `runsvinit`, so the image builds it - from its archived source (`github.com/peterbourgon/runsvinit`, unchanged - since 2015) at a fixed commit hash, with a `go.mod` the build writes. - `runsvinit` starts runit's `runsvdir`, which runs the `run` script of each - directory under `/etc/service` and runs it again whenever it exits. Ubuntu's - `sv` looks for services there too. -- The `smallwebwaf` binary, and a user of its own, `smallwebwaf` (uid and gid - 65532). -- The service directory `/etc/service/smallwebwaf`. Its `run` script waits one - second (`sleep 1`), makes `SWWAF_STATE_DIR` and every file in it belong to the - `smallwebwaf` user, and starts `smallwebwaf` as that user with runit's - `chpst`. -- The directory `/var/lib/smallwebwaf` for the state files, and - `/etc/smallwebwaf/rules.d` with the default rule file. -- Port 8080 declared (`EXPOSE 8080`), and the health check (`HEALTHCHECK`) that - "The health check" in `SPEC.md` describes. +- Ubuntu 26.04 LTS, pinned by digest. +- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, from the snapshot of its + archive at `20261001T000000Z`. +- Nix, with nixpkgs `nixos-26.05.11045.774debe7a0d1` set up for root as + `nixpkgs`. No Nix daemon; `build-users-group =` in `/etc/nix/nix.conf`. Root's + Nix profile, `/nix/var/nix/profiles/default/bin`, last on the `PATH`. + Unpacked, nixpkgs takes about 500 MiB. +- `runsvinit` as the entrypoint, built from `github.com/peterbourgon/runsvinit` + at a fixed commit hash. +- The `smallwebwaf` binary, and the user `smallwebwaf` (uid and gid 65532). +- The service directory `/etc/service/smallwebwaf`, with the `run` script + `share/smallwebwaf.run`. +- `/var/lib/smallwebwaf` for the state files, and `/etc/smallwebwaf/rules.d` + with the default rule file `share/rules.d/00-default.rules`. +- `EXPOSE 8080`, and the `HEALTHCHECK` that runs `smallwebwaf healthcheck`. -## Pinning +## Pins -- Ubuntu's packages come from Ubuntu's snapshot service, the archive as it was - at the moment the Dockerfile names (`--snapshot 20261001T000000Z`), never - earlier than the pinned Ubuntu image's date. The Dockerfile names the SHA-256 - hash of each of the snapshot's `InRelease` files, which cover every package - list and so every package, and the build checks them before installing. The - hashes are the amd64 archive's, so the image is built for amd64 only. The - snapshot is fetched over HTTPS with the CA certificates of the Go image that - builds `smallwebwaf`, itself pinned by digest, since the Ubuntu image has - none. -- nixpkgs comes from a release file on `releases.nixos.org` (`nixexprs.tar.xz` - of `nixos-26.05.11045.774debe7a0d1`), named with its SHA-256 hash, not from a - GitHub archive, whose bytes can change. It is set up for root as `nixpkgs`. - Root's Nix profile, `/nix/var/nix/profiles/default/bin`, is last on the - `PATH`, so that no package hides the image's own commands; busybox's `sv`, for - one, looks for services elsewhere. Unpacked, nixpkgs takes about 500 MiB. -- An app built on the image gets the same packages each time it is built. - Security fixes from Ubuntu and nixpkgs come with a newer `smallwebwaf` image, - which an app takes by changing the digest in its `FROM` line. The image moves - to each new nixpkgs release, every six months. +- Ubuntu's packages: the snapshot service, `--snapshot 20261001T000000Z` on both + `apt-get update` and `apt-get install`. The SHA-256 hash of each of the + snapshot's `InRelease` files is named in the `Dockerfile` and checked after + `apt-get update`. The hashes are the amd64 archive's. +- nixpkgs: the release file `nixexprs.tar.xz` of + `nixos-26.05.11045.774debe7a0d1` on `releases.nixos.org`, named with its + SHA-256 hash. +- `runsvinit`: the archived repository, unchanged since 2015, at tag `v2.0.0`, + by commit hash, with a `go.mod` the build writes. +- The Go image that builds `smallwebwaf`: by digest. diff --git a/docs/lookup.md b/docs/lookup.md deleted file mode 100644 index 45909d0..0000000 --- a/docs/lookup.md +++ /dev/null @@ -1,79 +0,0 @@ -# GeoJS and the lookup database - -`smallwebwaf` looks up each client's AS number and country, through GeoJS by -default or in a database file on your own host. The settings are under "Looking -up the AS number and country" in [`settings.md`](settings.md); the reasons for -the defaults are under "Lookup of AS number and country" in -[`SPEC.md`](../SPEC.md). - -## GeoJS - -By default `smallwebwaf` asks GeoJS, a free web service that needs no account, -about every new visitor's address. It asks whether or not a setting uses the -answer, since the log line, the history, the ban notes and the metrics carry the -AS number and country. These addresses are never sent: - -- one in `SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`; -- one under a ban; -- a private, loopback or link-local address, which no source can place. - -For an IPv6 visitor, GeoJS is asked about the first address of its group. -Addresses are asked about together, up to 200 in one request -(`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a time. Each -answer's `country_code`, `asn` and `organization_name` are read. An answer -without a `country_code`, or with the `asn` `64512`, counts as unknown. Each -answer is kept for seven days, in memory and in `lookups.json`, separately from -the table of clients. Up to 100,000 answers are kept; past that, the answer used -longest ago goes first. - -A request waits for its client's first answer only when a setting needs the -answer before the request goes on. It waits up to `SWWAF_LOOKUP_TIMEOUT`. The -settings that make it wait: - -- a country list; -- `SWWAF_ADD_LOOKUP_HEADERS`; -- any setting under "Lower limits by AS number and country" in - [`settings.md`](settings.md). - -Otherwise the request goes on at once, logged without the answer. The answer -reaches the client's history and ban notes when it comes. A client whose answer -has not come in time counts as from an unknown country until it comes, and its -later requests do not wait. While GeoJS fails, new visitors count as from an -unknown country, which `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses. GeoJS is -then left alone for a second, and twice as long after each further failure, up -to five minutes, and one `source_failure` alert is sent per cooldown. - -## The database file - -To keep your visitors' addresses on your own host, set -`SWWAF_LOOKUP_SOURCE=off`, or set it to `file`. `file` looks every client up in -the free IPinfo Lite database (`ipinfo_lite.mmdb`) at `SWWAF_LOOKUP_DB_PATH`, -reading its `asn`, `as_name` and `country_code` fields. You download the -database with an IPinfo account and mount it into the container; `smallwebwaf` -never fetches it and holds no account or token for it. It answers at once, with -nothing sent off the host. A client whose address it does not hold counts as -from an unknown country. - -`smallwebwaf` reads the whole file at start. It reads the file again once it has -gone 2 seconds without a change after you replace it. A replacement it cannot -read is logged and alerted, and the file read before stays in use. Mount the -directory that holds the file, not the file itself. Many editors and download -tools save a file by replacing it, and a container keeps seeing the old file -when only the file is mounted. - -IPinfo updates the database daily, and releases it under the Creative Commons -Attribution-ShareAlike 4.0 licence. It asks for a link to IPinfo on the site -that uses it, such as "IP address data is powered by IPinfo". - -## A client without a country - -A client the lookup cannot place has an unknown country and AS number. That is -one of: - -- a client on a private address; -- one GeoJS or the database does not know; -- one whose GeoJS answer has not come in time. - -`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses such a client, and the lower -limits give it `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it -differently. diff --git a/docs/reputation.md b/docs/reputation.md deleted file mode 100644 index 80c6318..0000000 --- a/docs/reputation.md +++ /dev/null @@ -1,113 +0,0 @@ -# Reputation sources - -A reputation source judges a client by what others saw it do. None is on by -default. The settings are under "Reputation sources" in -[`settings.md`](settings.md); the reasons for the defaults are under -"Reputation" in [`SPEC.md`](../SPEC.md). The sources are checked in this order, -and a source the client is on refuses it, lowers its limits, or only notes it, -as `SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say. - -## Blocklists - -`SWWAF_BLOCKLIST_URLS` lists text files of addresses and netblocks, one to a -line, as the Spamhaus DROP list is written. Anything after a `;` or `#` is left -out. An IPv4-mapped address such as `::ffff:192.0.2.0/120` is read as the IPv4 -netblock it stands for. - -Each list is fetched again `SWWAF_BLOCKLIST_REFRESH` after its last try, one -list at a time. The file of `SWWAF_ASN_LIMIT_PERCENT_URL` is too. -`reputation.json` keeps when each was last tried, so a restart does not fetch a -list again before its time. A fetch fails when: - -- the server answers other than `200`; -- the fetch does not finish within a minute; -- the list is longer than 16 MiB; -- a line is not an address or netblock, or, for the AS number file, not - `AS:`. - -The copy fetched before stays in use, comment lines included. The failure is -logged and raised as a `source_failure` alert. The log, the alerts, the ban -notes and the metrics name each list by its URL, so keep a secret out of it. - -The Spamhaus DROP list is The Spamhaus Project's. Its terms, and what they mean -for `SWWAF_BLOCKLIST_REFRESH` and for several instances on one host, are under -"Reputation" in [`SPEC.md`](../SPEC.md). The copy in `reputation.json` keeps the -list's date and copyright lines. - -## CrowdSec - -While `SWWAF_CROWDSEC_LAPI_URL` is set to the local API of a CrowdSec engine you -run, `smallwebwaf` fetches its decision list every minute. It sends -`SWWAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key` header. The list is kept as a -blocklist is kept. The decisions of the type `ban` on an address or a netblock -are used. Any other, such as a captcha or a country, is left out. A decision -lasts for the `duration` it had left when the engine answered. - -A request is refused when a decision still in force names its client. The -client's netblock is banned until the decision ends, with the cause `crowdsec`. -Only a client that sends a request is banned, so a long list does not fill -`bans.json`. Such a ban is never made permanent, and does not lengthen the -netblock's next ban for a broken limit. If you lift a ban while the decision is -in force, the client's next request makes another. Delete the decision in -CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and lift -the ban once the list has been fetched again. A fetch fails when: - -- the engine answers other than `200`, a redirect included; -- the fetch does not finish within a minute; -- the answer is longer than 16 MiB; -- the answer is not a list of decisions. - -## DNS blocklists - -Each zone in `SWWAF_DNSBL_ZONES` is asked about a client's own address in the -background, the first time the client is seen. The request goes on at once, and -so does every request from the client until the zone has answered. The name -asked about is the one RFC 5782 gives: the numbers of an IPv4 address reversed, -or the 32 hex digits of an IPv6 address reversed. `192.0.2.99` is asked about as -`99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` lists the client, but -one in `127.255.255.0/24` is Spamhaus refusing the query. A name that does not -exist does not list the client. These are failures: - -- an answer in `127.255.255.0/24`; -- an answer outside `127.0.0.0/8`; -- a resolver error; -- no answer within `SWWAF_REPUTATION_TIMEOUT`. - -A failure is logged and alerted, and the zone is not asked again for a minute. -Each verdict is kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and in -`reputation.json`. Up to 100,000 verdicts are kept; past that, the one fetched -longest ago goes first. At most 1,000 queries are under way at once. - -The key of a Spamhaus zone under `dq.spamhaus.net` is its first label. -`smallwebwaf` shows `********` in its place wherever it names the zone. Only -`reputation.json` keeps the zone with its key. - -## AbuseIPDB - -While `SWWAF_ABUSEIPDB_KEY` is set, `smallwebwaf` asks AbuseIPDB's check -endpoint for the abuse confidence score of a client, from 0 to 100. Only a -client that has committed an offence is checked. An offence is one of: - -- a broken limit; -- a `ban` rule match or a trap path; -- a refusal by a `block` rule, the Core Rule Set or a token check. - -The check runs in the background, at the client's first request after the -offence that no ban refuses, so no request waits. The score applies to the -client's later requests. For an IPv6 client, AbuseIPDB is asked about the -address of the request that set off the check, and the score is used for the -whole group. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and -`SWWAF_REPUTATION_ACTION` says what is done. Each score is kept for -`SWWAF_REPUTATION_CACHE_TTL`, up to 100,000 scores. - -At most `SWWAF_ABUSEIPDB_DAILY_BUDGET` checks are made in a day, counted in UTC -and kept in `reputation.json`. A restart does not make the budget whole again. -The check that spends the last of the budget is logged and alerted. No client is -checked until the day ends. A check fails when: - -- AbuseIPDB answers other than `200`; -- the answer gives no score; -- no answer comes within `SWWAF_REPUTATION_TIMEOUT`. - -A failure is logged and alerted, and no client is checked for a minute. The key -is sent in the `Key` header and nowhere else, and is never logged. diff --git a/docs/request-log.md b/docs/request-log.md index e0c1922..1891448 100644 --- a/docs/request-log.md +++ b/docs/request-log.md @@ -1,8 +1,8 @@ # The request log's fields -`smallwebwaf` writes one JSON object per line on stdout for every request. The -requirements are under "Request log" in [`SPEC.md`](../SPEC.md); this file lists -the fields. A field that does not apply to a request is left out. +The fields of the JSON object `smallwebwaf` writes for each request. "Request +log" in [`SPEC.md`](../SPEC.md) says when a line is written and how the log is +sent to a syslog server. A field that does not apply to a request is left out. ## Example @@ -10,6 +10,12 @@ the fields. A field that does not apply to a request is left out. {"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104} ``` +## Every line + +Every line has `type` (`request`; the process's own messages have `process`), +the standard web log fields, `request_id`, `peer_ip`, `client_group`, `asn`, +`as_name`, `country`, `action` and `duration_total`. + ## Standard web log fields - `time`: when the request arrived, RFC 3339 with milliseconds, in UTC; @@ -66,11 +72,12 @@ the fields. A field that does not apply to a request is left out. - `limit_percent` and `limit_percent_setting`: the client's percentage of the rate limits and the setting that lowered it, when one did. `bytes_percent` and `bytes_percent_setting` do the same for the byte limits. -- `counts`: the client's requests in the minute, hour and day as rate limiting - counts them, this request included, as `minute`, `hour` and `day`; its bytes - as `minute_bytes`, `hour_bytes` and `day_bytes`. A count can have a fraction, - since each window is counted in two buckets with the earlier one weighted. - `counts` is left out when rate limiting did not count the request. +- `counts`: the client's totals as rate limiting counts them, this request + included. It is left out when rate limiting did not count the request. A count + can have a fraction, since each window is counted in two buckets with the + earlier one weighted: + - `minute`, `hour` and `day`: requests; + - `minute_bytes`, `hour_bytes` and `day_bytes`: bytes. - `limit_hit`: the limit broken: `minute`, `hour` or `day` for a rate limit; `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or `error_burst`. A request whose bytes broke a limit was passed on whole, so its @@ -99,30 +106,3 @@ All in milliseconds: connection; - `duration_upstream_first_byte`: until the first byte of the answer; - `duration_upstream_total`: until the end of the answer. - -## Every line - -Every line has `type`, the standard web log fields, `request_id`, `peer_ip`, -`client_group`, `asn`, `as_name`, `country`, `action` and `duration_total`. The -process's own messages share the stream with `"type":"process"`; request lines -carry `"type":"request"`. - -## Sending the log to a syslog server - -While `SWWAF_LOG_REMOTE_URL` is set, every line on stdout is also sent to that -server, as the message of an RFC 5424 syslog record. Over UDP each record is one -datagram. Over TCP and TLS each record follows its length in bytes and a space. -stdout is unchanged. The record carries: - -- the facility from `SWWAF_LOG_REMOTE_FACILITY`; -- the severity informational; -- the time; -- the host's name; -- the app name from `SWWAF_LOG_REMOTE_APP_NAME`. - -Lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines, so a server that is -slow or down never holds up a request. When the buffer is full, the oldest line -is dropped. A failed connection is tried again a second later, and twice as long -after each further failure, up to a minute. A line too long for one UDP datagram -is dropped alone. As `smallwebwaf` stops, it spends at most two seconds sending -the lines still waiting, and drops the rest. stdout has carried them. diff --git a/docs/rules.md b/docs/rules.md index 8af131b..1bd3fd3 100644 --- a/docs/rules.md +++ b/docs/rules.md @@ -1,10 +1,8 @@ # Rule files -`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`, in the order of -their names, and checks each request against their rules in that order, unless -the client is in `SWWAF_ALLOW_NETS`. The requirements the files meet are under -"Rule files" in [`SPEC.md`](../SPEC.md). A file whose name starts with `.`, such -as an editor's lock file, is not a rule file. +The format of a rule file, and the default file the image ships. "Rule files" in +[`SPEC.md`](../SPEC.md) says how the files are read, watched and mounted, and +what each action leads to. ## The format @@ -18,7 +16,7 @@ env-file path ban (?i)^/\.env(\.[a-z]+)?$ scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b ``` -- The id is letters, digits, `-` and `_`. No two rules share one. The log line, +- The id is letters, digits, `-` and `_`, unique across all files. The log line, the metrics, the alerts and a ban's notes name the rule by it. - The target is what the regex is matched against: - `path`: the path without the query; @@ -26,42 +24,10 @@ scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b - `query`: the query as sent; - `method`, `host`, `user_agent` and `referer`; - `header:`: any one request header but `Host` and - `Transfer-Encoding`, which Go's HTTP server takes out of every request. - - `path` and `uri` are matched both as sent and once percent-decoded, so that - an encoded probe does not slip past. A header sent more than once is matched - with its values joined by `, `. A header not sent is empty text. No body is - read; reading bodies is the Core Rule Set's job, once `SWWAF_WAF_BODY_LIMIT` - switches it on. - -- The action is `log`, `block` or `ban`. `log` only notes the match in the log - line. `block` refuses the request with `403` and bans no one; the refusal - counts toward the error burst. `ban` refuses it and bans the client, as a - clear sign of attack. Matching stops at the first rule that refuses. Keep - `ban` for requests no real visitor sends. Anchor a path at the site root with - `^/`: a file of the same name deeper in a site can be ordinary content. -- The regex is in Go's syntax, RE2. It has no backreferences or lookaround, and - takes time linear in the text it reads, so no rule can be made to stall the - proxy. It matches anywhere in the target unless anchored with `^` and `$`. - `(?i)` at its front makes it ignore case. - -## Errors - -These stop the start, with a message naming the file and line: - -- a line that is not a rule; -- a bad `header:` name, or a rule for `Host` or `Transfer-Encoding`; -- a regex that does not compile; -- an id used twice. - -So does a `SWWAF_RULES_DIR` that does not exist. An empty directory is not an -error: the log says that no rules were loaded. - -While it runs, `smallwebwaf` watches the directory. It reads the files again -once the directory has had no change for 2 seconds, so that a file still being -written is read whole. If the files then hold such an error, the rules stay as -they were. The log and a `file_error` alert name the file and line. Once the -file is fixed, it is read again. + `Transfer-Encoding`. +- The action is `log`, `block` or `ban`. +- The regex is in Go's syntax, RE2. It matches anywhere in the target unless + anchored with `^` and `$`. `(?i)` at its front makes it ignore case. ## The default file @@ -86,13 +52,10 @@ path-traversal uri block (\.\./){2,} empty-agent user_agent log ^$ ``` -It is kept short and limited to patterns that are wrong for every app, with its -path rules anchored at the site root. In front of gitea, `/.env` is a probe, -while `///src/branch/main/.env.example` is a file in a repository -that any visitor may open. Anything app-specific belongs in a file the app's -`Dockerfile` adds or the deployer mounts. A request for `/wp-login.php`, for -example, is a clear sign of attack in front of gitea and an ordinary login in -front of WordPress. Such a rule goes in a file named to sort after the default: +## A file of your own + +An app's `Dockerfile` adds rules of its own in a file named to sort after the +default, such as `50-gitea.rules`: ``` # 50-gitea.rules: WordPress probes, which gitea never serves @@ -102,13 +65,3 @@ wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/) ```dockerfile COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules ``` - -## Mounting rule files - -A directory mounted over `/etc/smallwebwaf/rules.d` replaces the default file; -single files mounted into it add to it. Many editors save a file by replacing -it, and a container keeps seeing the old one when the file alone is mounted. So -rules to be edited while `smallwebwaf` runs belong in a mounted directory, with -a copy of the default file if the defaults are to stay. The lookup database is -mounted the same way, for the same reason. To run without rules, mount an empty -directory or set `SWWAF_RULES_ENABLED=false`. diff --git a/docs/settings.md b/docs/settings.md index d9088ba..55990fa 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -1,27 +1,19 @@ # Settings -Every setting is an environment variable whose name starts with `SWWAF_`. Every -setting has a default, so none has to be set. Settings are read once, at start. -To change one, restart the container. +Every setting, its default and the values it takes, in the groups of "Settings" +in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings +that take `off`, the `_FILE` form, and what each group does and why. A setting +set to a value its entry does not allow stops the start. -A setting that is set to a value it cannot take stops the start. The process -writes one line, `"type":"process"` with `"msg":"invalid setting"`, naming the -variable and what is wrong with the value. It exits with status 1, and runit -starts it again every second until the value is fixed. The entries below say -which values each setting takes. Any other value stops the start. At start, the -settings in effect are logged on one line. Tokens, keys and the paths of -webhooks are shown as `********`. - -The forms of values: +## Forms of values - A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. Zero and negative durations are refused. - A size is a number of bytes with an optional `K`, `M` or `G`, which are powers - of 1024: `1K` is 1024 bytes, `512K`, `100M`, `5G`. + of 1024, so `1K` is 1024 bytes. Examples: `512K`, `100M`, `5G`. - A count is a whole number of requests, such as `1000`. -- `off` switches a setting off, where its entry below says it takes `off`. - A list is comma separated, with spaces around items ignored. An empty item is - refused. A list set but empty is an empty list, and replaces the default. + refused. - A netblock is an address and a prefix length, such as `10.0.0.0/8`. A bare address stands for itself alone. - A country is a two-letter code of ISO 3166-1, in either case: `de` and `DE` @@ -31,337 +23,58 @@ The forms of values: - A percentage is a whole number from 0 to 100. - A path prefix starts with `/`, such as `/assets/`. -## Where it listens and what it protects +## Core -- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens, an - optional host and a port. Keep the host empty: `:9000`, never - `127.0.0.1:9000`. The app must not listen on that port. The health check - follows the setting, and traefik's label must name the port. - `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or - `https` URL with a host and an optional port, and nothing more: no path, user - or query. -- `SWWAF_INSTANCE_NAME` (default: the container's host name, the first 12 - characters of the container's id): the `instance` name in every log line, - alert and metric. Set it to a name that survives deploys and tells instances - apart, such as `fsn1app1/gitea`. It must be valid UTF-8. While - `SWWAF_ALERT_NTFY_URL` is set, it may hold no control character. While - `SWWAF_LOG_REMOTE_URL` is set and `SWWAF_LOG_REMOTE_APP_NAME` is not, it must - be 1 to 48 printable ASCII characters without a space. -- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to ban no one and - log what `enforce` would have done. Observe mode still refuses a request over - a size or time limit, and one without the right token. -- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`, - the private address ranges): the netblocks whose `X-Forwarded-For` is - believed, normally where traefik comes from. Set but empty, it trusts nothing: - every TCP peer is then the client. -- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the length of the netblock that is - one IPv6 client, from 32 to 128. After a change, each IPv6 client starts - afresh. Existing bans keep refusing their netblocks until they end. -- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept in memory - and in `clients.json`, a whole number above zero. Past it, the least recently - seen client is dropped first. - -## Time and size limits - -Each request has limits on both sides: between the client and `smallwebwaf`, and -between `smallwebwaf` and the app. Bodies stream through in both directions, -except the part the Core Rule Set reads while `SWWAF_WAF_BODY_LIMIT` is set. - -- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to - send its request line and headers, and then, from the end of the headers, its - body. A duration or `off`. -- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request - line and headers. A size over `4K`. It cannot be `off`: Go's HTTP server - always has such a limit. -- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection - may wait for its next request before it is closed. A duration or `off`. The - default is longer than the 90 seconds after which traefik closes an unused - connection, so traefik closes first. -- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the answer may take - to reach the client, from the end of the request to the last byte. A duration - or `off`. -- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the - app and sending it the whole request may take. A duration or `off`. -- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take - to send its whole answer. A duration or `off`. -- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size - or `off`. A request that announces a larger body is refused before anything - reaches the app. -- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size - or `off`. - -In front of a code forge, a clone is a response and a push is a request. At the -defaults, an upload over 100 MiB or slower than 60 seconds is cut off. A forge -that takes large uploads needs `SWWAF_REQUEST_MAX_BYTES`, -`SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to -fit. - -What the client gets when a limit is passed is under "Size and time limits" in -[`SPEC.md`](../SPEC.md). - -## Netblock lists - -- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip every check, - such as your monitoring or your own network. The size and time limits still - apply to them. They are not looked up. They still need a token for the admin - endpoints. -- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are - exempt from rate limiting and the byte limits, such as a machine that talks to - the app all day. Everything else still applies to them. -- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused. - A client in both lists is let through. - -## Rate limiting and byte limits - -- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR` - (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most - requests a client may make in each window. A count or `off`. The defaults are - several times what one busy person produces. A browser loading a heavy page - makes a few hundred requests, and several people often share one address. -- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from - rate limiting, such as `/assets/`. Their bytes are not counted either. The - path is matched exactly as sent, without the query, and is not - percent-decoded. `/assets/` matches `/assets/app.js`. It does not match - `/Assets/app.js`, `/%61ssets/app.js` or `/static/assets/app.js`. There are no - wildcards. A path holding `..`, a backslash or an encoded slash (`%2F`) is - never exempt, since the app may read it as a path outside the prefix. -- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR` - (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most - bytes a client may have counted in each window. A size or `off`. Bytes are - counted once the answer has ended, so a byte limit never cuts an answer short. - The client's next request is refused instead. Each default is above the - largest request and answer together, so at the defaults no single download - breaks a limit. -- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count: - `response`, `request` or `both`. For a WebSocket, what it carried each way - counts once it closes. - -## Looking up the AS number and country - -- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `geojs` - asks the GeoJS web service, which is then told the address of every new - visitor. `file` looks clients up in the IPinfo Lite database file at - `SWWAF_LOOKUP_DB_PATH`, and sends nothing off the host. `off` looks up no - client. With `off`, a setting that needs lookups stops the start: - - a country list; - - `SWWAF_ADD_LOOKUP_HEADERS`; - - an entry in `SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`, - `SWWAF_ASN_BYTES_PERCENT` or `SWWAF_COUNTRY_BYTES_PERCENT`; - - `SWWAF_UNKNOWN_LIMIT_PERCENT` below 100; - - `SWWAF_ASN_LIMIT_PERCENT_URL`; - - an anomaly threshold per AS number. -- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its - `.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`. `file` without it stops the - start, and so does setting it with any other source. A file that is missing or - cannot be read stops the start too. -- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its - client's first answer from GeoJS when a setting needs it, and how long a - request to GeoJS may take. A duration; it cannot be `off`. -- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the - client's AS number in `X-Client-ASN` and its country in `X-Client-Country`. A - header whose value is unknown is left out. Whatever this setting says, any - such header the client sent is removed. - -See [`lookup.md`](lookup.md) for what each source sends where, and how long -answers are kept. - -## Country lists - -- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused, - such as `cn,ru,kp`. -- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only - countries whose clients get through, such as `us,de`. A client whose country - cannot be found is refused too. Otherwise every new client would be let in - whenever GeoJS stops answering. A client on a private address has no country, - so it is refused unless it is in `SWWAF_ALLOW_NETS`. - -A country on both lists stops the start. What a refused request gets, and why -the lists are empty by default, is under "Country lists" in -[`SPEC.md`](../SPEC.md). - -## Lower limits by AS number and country - -Clients of the AS numbers and countries listed here get a percentage of every -rate limit and byte limit. A lowered limit is rounded down. "Biased thresholds" -in [`SPEC.md`](../SPEC.md) says why the lists are empty by default and why no -budget is shared by a whole AS number or country. - -- `SWWAF_ASN_LIMIT_PERCENT` (default empty): a list of AS numbers, each with `:` - and a percentage, such as `AS14061:50,AS16276:50,AS45102:25`. An AS number - listed twice is refused. -- `SWWAF_COUNTRY_LIMIT_PERCENT` (default empty): the same by country, such as - `cn:25,ru:50`. -- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty): - the same for the byte limits alone. For the AS numbers and countries they - list, they take the place of the two settings above. -- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage of every limit - for a client without a country: - - one the lookup cannot place; - - one on a private address; - - one whose answer from GeoJS has not come in time. -- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a - text file of such entries, one to a line, such as `AS14061:50`. One list can - then serve several instances. It is fetched and kept as a blocklist is. Where - it and `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. A - URL that is also in `SWWAF_BLOCKLIST_URLS` stops the start. - -A client to which several percentages apply gets the lowest. The log line of -each counted request, and the notes of each ban, say which setting lowered the -limit. - -## Reputation sources - -None is on by default. A reputation source judges a client by what others saw it -do. The defaults judge it by what it does to your service. - -- `SWWAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as - `https://www.spamhaus.org/drop/drop.txt`. Each is an `http` or `https` URL - without a user or a fragment. A URL listed twice is refused. -- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its - last fetch, or failed try, before it is fetched again. A duration of `1h` or - more; it cannot be `off`. The Spamhaus lists may be fetched no more than once - an hour. -- `SWWAF_BLOCKLIST_ACTION` (default `deny`): what is done with a client on a - blocklist. `deny` refuses its requests. `limit:`, such as `limit:25`, - gives it that percentage of every limit. `log` only notes the list in the log - line and raises an alert. -- `SWWAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask about each - client, such as `dnsbl.dronebl.org`. A DNS blocklist answers a DNS query for a - name made from an address, and so says whether the address is on it. A zone is - a DNS name of at most 189 characters. A Spamhaus zone carries its key, as in - `.xbl.dq.spamhaus.net`, and is shown with the key masked. A zone listed - twice is refused, whatever its case or key. Do not name a zone meant for mail: - such a zone holds the residential addresses ordinary visitors come from. -- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through, - such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional - port, 53 when none is given. Unset, the host's resolver is used. Several zones - refuse queries that come through a public resolver. -- `SWWAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account. - While it is set, a client that has committed an offence is checked. A control - character in it is refused. -- `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score, - from 0 to 100, that is a hit. -- `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day, - in UTC, a whole number above zero. AbuseIPDB's free accounts may make 1,000. -- `SWWAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine - you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a - user or a fragment. Its decision list is at `v1/decisions` under that URL, and - is fetched every minute. Set without the key, it stops the start. So does a - decision list URL that is also in `SWWAF_BLOCKLIST_URLS`, or that is - `SWWAF_ASN_LIMIT_PERCENT_URL`. -- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for - `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the - URL, it stops the start. -- `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client on - a DNS blocklist, or one whose AbuseIPDB score is a hit. `deny`, - `limit:` or `log`, as for `SWWAF_BLOCKLIST_ACTION`. Such a verdict is - less certain than the Spamhaus DROP list, so by default the client gets a - quarter of every limit. -- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or - AbuseIPDB's score is used. A duration; it cannot be `off`. -- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a - check with AbuseIPDB may take. A duration; it cannot be `off`. - -See [`reputation.md`](reputation.md) for how each source is asked and what it -does with the answer. - -## Attack detection - -- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the - rule files (see [`rules.md`](rules.md)). A directory that cannot be read stops - the start. An empty one is no error. -- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file. -- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only - scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of a code - forge. A request for one bans the client, as a `ban` rule does. Each starts - with `/` and holds no `?`. A trap path is matched whole, without the query, - against the path as sent and once percent-decoded. `/wp-login.php` matches - `/wp-login.php?x=1` and `/%77p-login.php`. It does not match `/wp-login.php/` - or `/blog/wp-login.php`. -- `SWWAF_WAF_MODE` (default `block`): what the Core Rule Set does with a request - it takes for an attack. `block` refuses it with `403`. `detect` lets it - through, logged and alerted. `off` inspects no request. -- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level, - from 1 to 4. Each level up runs more rules. Those find more attacks, and take - more ordinary requests for attacks. -- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the score at which a request is a - match. Each rule that matches adds to the request's score, and a rule rated - critical adds 5. So by default one such rule is enough. A whole number above - zero, or `off`. With `off` no request is a match, and scores are still logged. -- `SWWAF_WAF_DISABLED_RULES` (default - `920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules - to switch off. The log line names the rules a request matched, so a rule that - refuses your app's ordinary requests can be found there. "Attack detection" in - [`SPEC.md`](../SPEC.md) says what the six default ids refuse and why they are - off. A list you give replaces the default, so name these six ids in it to keep - them off. An id must be a whole number above zero. An id from 900000 to 900999 - is refused: those are the rules that set the Core Rule Set up, and - `smallwebwaf`'s own. -- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does - not inspect, matched as `SWWAF_RATE_LIMIT_EXEMPT_PATHS` is. -- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size, - at most `1G`, switches body inspection on. The Core Rule Set then reads a form - data or multipart body up to that size, and the rest streams on to the app. It - reads a JSON or XML body no larger than that size, since those cannot be read - in part. Any other body, such as a git push, reaches the app uninspected. See - [`core-rule-set.md`](core-rule-set.md). -- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client - in a minute for a rule match, a trap path, a Core Rule Set match or a failed - token check. One more bans the client, as a broken rate limit does. A count or - `off`. The app's own `401` and `404` answers are not counted. - -## How long bans last and what they cover - -- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered: `403`, - `429`, or `close` to close the connection without an answer. A `block` rule - and the Core Rule Set always answer `403`. -- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate - limit, byte limit or error burst. A duration; it cannot be `off`. -- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a client that breaks a limit - again within this time after its ban ended is banned three times as long. A - duration; it cannot be `off`. -- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken limit that would - be longer is permanent instead. A duration; it cannot be `off`. -- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of - attack. A duration; it cannot be `off`. + `https` URL with a host and an optional port, and nothing more. +- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens, an + optional host and a port. The host stays empty. +- `SWWAF_INSTANCE_NAME` (default: the container's host name): the `instance` + name in every log line, alert and metric, such as `fsn1app1/gitea`. Valid + UTF-8. No control character while `SWWAF_ALERT_NTFY_URL` is set. 1 to 48 + printable ASCII characters without a space while `SWWAF_LOG_REMOTE_URL` is set + and `SWWAF_LOG_REMOTE_APP_NAME` is not. +- `SWWAF_MODE` (default `enforce`): `enforce` or `observe`. +- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`): + the netblocks whose `X-Forwarded-For` is believed. Set but empty, it trusts + nothing. +- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the prefix length of an IPv6 + client's group, from 32 to 128. +- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a whole + number above zero. - `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are - kept, whether past, active or permanent. A whole number above zero. Bans whose - cause is `admin` are kept besides and never dropped (see - [`bans.md`](bans.md)). -- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around - an IPv4 client that a ban covers, from 0 to 32. `24` bans the surrounding /24. - An IPv6 ban covers the client's group. + kept, a whole number above zero. -## Where and how often the state files are written +## Tokens + +- `SWWAF_ADMIN_TOKEN` (default unset): the token for the ban endpoints and + `/_smallwebwaf/clients/`. At least 32 characters. +- `SWWAF_METRICS_TOKEN` (default unset): the token for `/_smallwebwaf/metrics`. + At least 32 characters. + +## State files - `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the directory of the state - files, an absolute path. A directory that does not exist or cannot be written - stops the start. + files, an absolute path that exists and can be written. - `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be - written after a ban is made, lifted or made permanent. Every change in between - goes into the same write. A duration; it cannot be `off`. + written after a ban is made, lifted or made permanent. A duration. - `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is - written. A duration; it cannot be `off`. + written. A duration. ## Logging +- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own + messages that are written: `debug`, `info`, `warn` or `error`. - `SWWAF_LOG_REQUEST_HEADERS` (default `accept,accept-language,accept-encoding,content-type,origin,range`): the - request headers to record in the log line, in either case. `Authorization`, - `Cookie` and `Set-Cookie` are never logged, even if listed. `Host` and - `Transfer-Encoding` are refused: Go's HTTP server takes them out of the - request. The host is the field `host`. -- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own - messages that are written: `debug`, `info`, `warn` or `error`. No message is - at `debug`. It holds back no request log line. `warn` and `error` hold back - the line of settings at start. -- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server every line on stdout - is also sent to, such as `syslog+tls://logs.example:6514`. The forms are - `syslog+udp://`, `syslog+tcp://` and `syslog+tls://`, each with a host and a - port and nothing more. + request headers recorded in the log line, in either case. `Host` and + `Transfer-Encoding` are refused; the host is the field `host`. +- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server that also gets every + line written to stdout: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`, + each with a host and a port and nothing more. - `SWWAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates that a `syslog+tls` server's certificate must chain to, instead of the host's - own. A file that cannot be read, or holds no certificate, stops the start. + own. A file that cannot be read, or holds no certificate, is refused. - `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they wait to be sent, a whole number above zero. - `SWWAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`, @@ -370,53 +83,193 @@ does with the answer. - `SWWAF_LOG_REMOTE_APP_NAME` (default: `SWWAF_INSTANCE_NAME`): the app name in each syslog record, 1 to 48 printable ASCII characters without a space. -## Tokens and metrics +## Metrics -- `SWWAF_ADMIN_TOKEN` (default unset): the token for the ban endpoints and - `/_smallwebwaf/clients/`. A long random value of at least 32 characters. - Unset, those endpoints are off. Give it as a file to keep it from the app (see - "Settings given as files" below). -- `SWWAF_METRICS_TOKEN` (default unset): the token a scraper sends for - `/_smallwebwaf/metrics`, at least 32 characters. Unset, the metrics are off. - It is separate from the admin token, so a scraper cannot manage bans. - `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many - countries get series of their own in the metrics. The rest count as `other`. A - whole number above zero. + countries get series of their own. A whole number above zero. -## Alert destinations +## Static lists + +- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients are refused only + for a size or time limit or a missing or wrong token, such as your monitoring. +- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are + exempt from rate limiting and the byte limits. +- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused. + +## Request rate limits, per client + +- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR` + (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most + requests a client may make in each window. A count or `off`. +- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from + rate limiting and the byte limits, such as `/assets/`. + +## Byte limits, per client + +- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR` + (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most + bytes counted for a client in each window. A size or `off`. +- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count: + `response`, `request` or `both`. + +## Size and time limits + +- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to + send its request line and headers, and then, from the end of the headers, its + body. A duration or `off`. +- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request + line and headers. A size over `4K`; never `off`. +- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection + may wait for its next request before it is closed. A duration or `off`. +- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the answer may take + to reach the client, from the end of the request to the last byte. A duration + or `off`. +- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the + app and sending it the whole request may take. A duration or `off`. +- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take + to send its whole answer. A duration or `off`. +- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size + or `off`. +- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size + or `off`. + +## Lookup of AS number and country + +- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. +- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its + `.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`. +- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its + client's first answer from GeoJS, and how long a request to GeoJS may take. A + duration; never `off`. +- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the + client's AS number in `X-Client-ASN` and its country in `X-Client-Country`. + +## Country lists + +- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused, + such as `cn,ru,kp`. +- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only + countries whose clients get through, such as `us,de`. A country on both lists + is refused. + +## Biased thresholds + +- `SWWAF_ASN_LIMIT_PERCENT` (default empty): a list of AS numbers, each with `:` + and a percentage, such as `AS14061:50,AS16276:50,AS45102:25`. An AS number + listed twice is refused. +- `SWWAF_COUNTRY_LIMIT_PERCENT` (default empty): the same by country, such as + `cn:25,ru:50`. +- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty): + the same, for the byte limits alone. +- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage for a client the + lookup cannot place. +- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a + text file of `AS:` lines, one to a line. A URL that is also + in `SWWAF_BLOCKLIST_URLS` is refused. + +## Attack detection + +- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the + rule files. +- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file. +- `SWWAF_TRAP_PATHS` (default empty): paths only scanners ask for, such as + `/wp-login.php,/xmlrpc.php`. Each starts with `/` and holds no `?`. +- `SWWAF_WAF_MODE` (default `block`): `off`, `detect` or `block`. +- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level, + from 1 to 4. +- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the score at which a request is a + match. A whole number above zero, or `off`. +- `SWWAF_WAF_DISABLED_RULES` (default + `920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules + to switch off. Each a whole number above zero. An id from 900000 to 900999 is + refused: those are the rules that set the Core Rule Set up, and + `smallwebwaf`'s own. +- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does + not inspect. +- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off`, or how much of a request body + the Core Rule Set reads, a size of at most `1G`. +- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client + in a minute for a rule match, a trap path, a Core Rule Set match or a failed + token check. A count or `off`. + +## Ban lengths, response and scope + +- `SWWAF_BAN_RESPONSE` (default `403`): `403`, `429` or `close`. +- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken limit. A + duration. +- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): the time after a ban ends + within which breaking a limit again lengthens the next ban. A duration. +- `SWWAF_MAX_BAN_DURATION` (default `7d`): the longest ban for a broken limit; + one that would be longer is permanent. A duration. +- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of + attack. A duration. +- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): how much a ban on an IPv4 client + covers: the prefix length of its netblock, from 0 to 32. + +## Reputation + +- `SWWAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as + `https://www.spamhaus.org/drop/drop.txt`. Each an `http` or `https` URL + without a user or a fragment. A URL listed twice is refused. +- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its + last try before it is fetched again. A duration of `1h` or more. +- `SWWAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:` or `log`. +- `SWWAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask, such as + `dnsbl.dronebl.org`. Each a DNS name of at most 189 characters. A Spamhaus + zone carries its key, as in `.xbl.dq.spamhaus.net`. A zone listed twice + is refused, whatever its case or key. +- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through, + such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional + port, 53 when none is given. Unset, the host's resolver is used. +- `SWWAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account. A + control character in it is refused. +- `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score, + from 0 to 100, that is a hit. +- `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day, + a whole number above zero. AbuseIPDB's free accounts may make 1,000. +- `SWWAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine + you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a + user or a fragment; its decision list is at `v1/decisions` under it. Set + without the key, it is refused. So is a decision list URL that is also in + `SWWAF_BLOCKLIST_URLS`, or that is `SWWAF_ASN_LIMIT_PERCENT_URL`. +- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for + `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the + URL, it is refused. +- `SWWAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:` or + `log`, for DNS blocklist and AbuseIPDB hits. +- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or + AbuseIPDB's score is used. A duration. +- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a + check with AbuseIPDB may take. A duration. + +## Alerting - `SWWAF_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each alert as a JSON object. An `http` or `https` URL without a user or a fragment. - The settings logged at start show its path and query as `********`, since many - webhooks carry their secret there. - `SWWAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post, such as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and value. A value cannot hold a comma. - `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook - that receives each alert as a message. It is checked and masked as the webhook - URL is. + that receives each alert as a message, checked as the webhook URL is. - `SWWAF_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each - alert, as the topic's full URL, such as `https://ntfy.sh/my-alerts`. It is - checked and masked as the webhook URL is. -- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token, sent as - `Authorization: Bearer `. A control character in it is refused. + alert, as the topic's full URL, such as `https://ntfy.sh/my-alerts`, checked + as the webhook URL is. +- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token. A control + character in it is refused. - `SWWAF_ALERT_EVENTS` (default `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): - the events alerts are sent for. Any other name is refused. + the event types alerts are sent for, from those in [`alerts.md`](alerts.md) + but `summary`. Any other name is refused. - `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held back. A duration or `off`. -- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. - The rest are rolled into one summary. A whole number above zero, or `off`. - -See [`alerts.md`](alerts.md) for the events and what each destination gets. +- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. A + whole number above zero, or `off`. ## Anomaly thresholds -These only alert. They refuse and ban nothing. Each scope has four thresholds: -the most requests and the most bytes counted in a minute and in an hour. -Requests are counts, bytes are sizes. Each is `off` by default. What is unusual -depends on each service's normal traffic, which the metrics show. The four -thresholds per client are: +Each scope has four thresholds: the most requests and the most bytes counted in +a minute and in an hour. Requests are counts, bytes are sizes. Each is `off` by +default. The four thresholds per client are: - `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`; - `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`; @@ -432,30 +285,19 @@ as `REQUESTS_PER_MINUTE`: client; - `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6 client. -- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs lookups on. +- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs a lookup source. - `SWWAF_ANOMALY_TOTAL_...`: the whole service. - `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole. `SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a name, or with a bad netblock, is refused. So is a name used twice. -Every request but the health check is counted, whatever was done with it, -clients in `SWWAF_ALLOW_NETS` included. At most 20,000 counters are kept. -`alerts.json` keeps them across a restart. - ## Settings given as files -Any setting may be given as a file holding its value. Add `_FILE` to the -setting's name: `SWWAF_METRICS_TOKEN_FILE` names the file that holds -`SWWAF_METRICS_TOKEN`. The file's contents are the value, less one newline at -the end. They are checked as the value itself would be. Setting both forms stops -the start, and so does naming a file that cannot be read. -`SWWAF_LOG_REMOTE_TLS_CA_FILE` already holds a file name, so it has no `_FILE` -form. - -A token given as a file is out of the app's reach only while the `smallwebwaf` -user alone can read the file. Make it on the host, owned by uid 65532 with mode -`0400`: +Add `_FILE` to a setting's name to read its value from a file: +`SWWAF_METRICS_TOKEN_FILE` names the file that holds `SWWAF_METRICS_TOKEN`. To +keep a token from the app, make the file on the host, owned by uid 65532 with +mode `0400`: ```sh mkdir -p /srv/app/tokens @@ -477,7 +319,8 @@ docker run \ A few limits are fixed rather than settings: -- at most 100,000 GeoJS answers are kept, for 7 days each; +- at most 100,000 GeoJS answers are kept; - at most 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept; - at most 1,000 DNS blocklist queries are under way at once; -- the CrowdSec decision list is fetched again a minute after the last fetch. +- at most 20,000 anomaly counters are kept; +- each alert destination's queue holds at most 1,000 alerts. diff --git a/docs/state-files.md b/docs/state-files.md index dd185cd..0c00910 100644 --- a/docs/state-files.md +++ b/docs/state-files.md @@ -1,102 +1,100 @@ # State files -`smallwebwaf` keeps its state in memory, and a copy of it in five JSON files in -`SWWAF_STATE_DIR`. The files are meant for people as well: you can read or edit -them at any time, and the running `smallwebwaf` takes the edit in. The -requirements they meet are under "Persistent state" in [`SPEC.md`](../SPEC.md). - -## The files - +The five JSON files in `SWWAF_STATE_DIR`, and what each holds. "Persistent +state" in [`SPEC.md`](../SPEC.md) says how they are written, read and edited. Each has a top-level `version` of 1. -- `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf` - made and every ban you made, indented to be read. Per entry: - - the netblock, start and expiry (`null` for permanent); - - the cause: `attack`, `limit`, `admin` or `crowdsec`; - - a short reason; - - when it was lifted, if you lifted it; - - the notes (see [`bans.md`](bans.md)). -- `clients.json`: per client, its minute, hour and day counters and its history - since it was first seen, one client per line, so `grep` shows everything about - one. The history holds: - - the first and last time seen; - - the AS number, AS name and country last looked up, and when; - - total requests, and how many were forwarded and how many refused; - - bytes in each direction; - - responses by status class; - - offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and - `token_refused`. +## `bans.json` - Clients that were never banned are kept too, so every client's history - survives a restart. +Active and past bans: every ban an admin made, and up to `SWWAF_MAX_BANS` of +those `smallwebwaf` made. Per entry: -- `lookups.json`: GeoJS answers, one per line: the AS number, AS name and - country, when GeoJS was asked and when the answer was last used. -- `reputation.json`: - - the last good copy of each list fetched from a URL, with when it was last - tried; - - each DNS blocklist verdict still in use; - - AbuseIPDB's scores, and the checks spent today. -- `alerts.json`: - - the anomaly counters; - - the alert cooldowns: for each event and netblock, source, file or thing - counted, when it was last sent and the count of repeats held back since; - - the alerts sent and held back in the current hour; - - the alerts waiting for each destination. +- `netblock`, `start` and `expires` (`null` for a permanent ban); +- `cause`: `attack`, `limit`, `admin` or `crowdsec`; +- `reason`: a short reason; +- `lifted`: when an admin lifted it, if one did; +- `notes`: the notes below. -Times are RFC 3339 UTC. Durations and sizes are plain numbers, with the unit in -the field name. `bans.json`, `reputation.json` and `alerts.json` are indented; -`clients.json` and `lookups.json` put each entry on one line. +To ban a netblock, add an entry with its `netblock`, `start` and `expires`. +`reason`, `notes` and `cause` may be left out; the cause is then `admin`: + +```json +{ + "version": 1, + "bans": [ + { + "netblock": "203.0.113.0/24", + "start": "2026-10-06T12:00:00Z", + "expires": null, + "reason": "probes for logins" + } + ] +} +``` + +To lift a ban, add `"lifted"` with the time you lift it, such as +`"lifted": "2026-10-06T13:00:00Z"`. To keep a ban `smallwebwaf` made, set its +`cause` to `admin`. To forget a ban, delete its entry. + +The notes: + +- `asn`, `as_name` and `country`: the client's, added when the lookup answers + (empty when lookups are off); +- what was broken: the rule id and target that matched, or the trap path, or the + limit, its window and the count reached; +- the client's percentage of the limit, with the setting that gave it; +- the reputation sources the client is on; +- `request`: the request that caused the ban: time, method, host, path with its + query string, status and user agent, each text cut to 256 bytes; +- `requests` and `refused`: the netblock's requests since it was first seen, and + how many this ban has refused so far, kept up to date while the ban lasts; +- `earlier_bans`: how many earlier bans the netblock had, by cause. + +## `clients.json` + +Per client, one per line: its minute, hour and day counters, and its history +since it was first seen: + +- the first and last time seen; +- the AS number, AS name and country last looked up, and when; +- total requests, and how many were forwarded and how many refused; +- bytes in each direction; +- responses by status class; +- offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and + `token_refused`. + +## `lookups.json` + +GeoJS answers, one per line: + +- `client`; +- `asn`, `as_name` and `country`; +- `answered`: when GeoJS answered; +- `used`: when the answer was last used. + +## `reputation.json` + +- the last good copy of each list fetched from a URL, with when it was last + tried; +- each DNS blocklist verdict still in use; +- AbuseIPDB's scores, and the checks spent today. + +## `alerts.json` + +- the anomaly counters; +- the alert cooldowns: for each event and netblock, source, file or anomaly + scope, when it was last sent and the count of repeats held back since; +- the alerts sent and held back in the current hour; +- the alerts waiting for each destination. ## Sizes -`clients.json` takes about 1 KiB per client, about 20 MiB at the default -`SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of -disk writes a day. `bans.json` takes about 1.2 KiB per ban, at most about 2.5 -KiB, so about 6 MiB at the default `SWWAF_MAX_BANS`. With a full file, a hundred -new bans a day come to about 600 MiB of writes. `lookups.json` takes about 150 -bytes per answer, under 1.5 GiB of writes a day when full. The other two files -are usually a few MiB or less. - -## Writing - -`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted or -made permanent. Every file is written every `SWWAF_STATE_COUNTER_INTERVAL`, and -when `smallwebwaf` stops. Each write goes to a temporary file that then replaces -the file, so a crash leaves the old file or the new one, whole. Before writing a -file, `smallwebwaf` takes in any edit you made to it, so your edit is not -overwritten. A write that fails is logged, raised as a `file_error` alert, and -tried again at the next write. A hard kill loses what changed since the last -write: - -- ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds); -- everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes): - - counts and history, and the counts in ban notes; - - GeoJS answers; - - reputation verdicts and the AbuseIPDB count; - - anomaly counters and alert cooldowns. - -## Reading at start - -The files are read back. Each client keeps its counts, so a restart gives no one -a fresh allowance. Each ban keeps refusing until it ends. Counter buckets and -cache entries whose time has passed are discarded. A missing file is empty -state. These stop the start, with a message naming the file and the place: - -- a file that does not parse, or has another `version`; -- an entry without a field it needs; -- a ban whose `cause` is not `limit`, `attack`, `admin` or `crowdsec`; -- a ban whose `expires` is not after its `start`. - -## Editing while it runs - -`smallwebwaf` watches the directory and takes in your edit of a file as soon as -you save it. What the file then holds replaces what `smallwebwaf` held for it. -It tells its own writes from yours by comparing the file with what it last read -or wrote. A change it made after you opened the file, such as a new ban, is lost -when you save over it; most editors warn when a file changed on disk while it -was open. An edit with an error that would stop the start does not stop the -running `smallwebwaf`. It keeps what it holds. At the file's next write it -renames your file to `.bad`, writes the file again from memory, and logs -and alerts the file and the place of the error. Mend the `.bad` file and move it -back. A file you remove is written again at its next write. +- `clients.json`: about 1 KiB per client, about 20 MiB at the default + `SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of + disk writes a day. +- `bans.json`: about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at + the default `SWWAF_MAX_BANS`. With a full file, a hundred new bans a day come + to about 600 MiB of writes. +- `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when + full. +- `reputation.json` and `alerts.json`: usually a few MiB or less. diff --git a/internal/remotelog/remotelog.go b/internal/remotelog/remotelog.go index 9f5f2ee..d40239b 100644 --- a/internal/remotelog/remotelog.go +++ b/internal/remotelog/remotelog.go @@ -1,9 +1,8 @@ // Package remotelog sends the lines smallwebwaf writes on stdout to the // remote log endpoint, SWWAF_LOG_REMOTE_URL, as the "Request log" section -// of SPEC.md requires and "Sending the log to a syslog server" in -// docs/request-log.md describes: each line as the message of an RFC 5424 -// syslog record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so -// a slow or unreachable endpoint never holds up a request or stdout. +// of SPEC.md describes: each line as the message of an RFC 5424 syslog +// record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so a slow +// or unreachable endpoint never holds up a request or stdout. package remotelog import ( diff --git a/internal/state/state.go b/internal/state/state.go index a6321c4..296fde1 100644 --- a/internal/state/state.go +++ b/internal/state/state.go @@ -1,5 +1,6 @@ // Package state keeps smallwebwaf's state in JSON files in -// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md describes: +// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md requires +// and docs/state-files.md lists them: // bans.json holds the bans, clients.json each client's counters and // history, lookups.json GeoJS's answers, reputation.json the last try and // last good copy of each list fetched from a URL, the DNSBL zones' diff --git a/internal/state/state_test.go b/internal/state/state_test.go index 0b98d94..97bfccf 100644 --- a/internal/state/state_test.go +++ b/internal/state/state_test.go @@ -1019,8 +1019,8 @@ func TestEditJustBeforeAScheduledWriteSurvivesIt(t *testing.T) { edit(t, dir, bansJSON, permanentBansJSON) // The write takes the edit in first, and writes it back. The second - // ban, made after the admin opened the file, is lost, as "Edits - // while running" in SPEC.md says. + // ban, made after the admin opened the file, is lost, as the + // "Persistent state" section of SPEC.md says. time.Sleep(time.Nanosecond) synctest.Wait() diff --git a/script/example-app b/script/example-app index 18cea8e..c668d2a 100755 --- a/script/example-app +++ b/script/example-app @@ -87,7 +87,7 @@ start_container() { # start_moved_container: run the app's container with the state files on # a new volume at /srv/state, which SWWAF_STATE_DIR_FILE names in a file # only the smallwebwaf user can read, as under "Settings given as files" -# in README.md, and wait until it is healthy. smallwebwaf can write the +# in docs/settings.md, and wait until it is healthy. smallwebwaf can write the # new volume only once the run script has given it to that user. start_moved_container() { docker volume create "$MOVED_VOLUME" >/dev/null -- 2.54.0 From 23908b0e33c431fdb9ff9c03e843f9e3c17dfd60 Mon Sep 17 00:00:00 2001 From: sneak Date: Thu, 22 Oct 2026 22:35:29 +0000 Subject: [PATCH 6/8] SPEC: false statements fixed, repeats removed, dropped requirements restored The bounds sentence, the blocklist URL sentence, the deny list sentence, the limit action's scope and the observe mode sentence now say what the code does. Each of the twenty-one requirements the fourth review found twice is stated once, with a link from the other place. The request id header, redirect_uri's case, change 4's reason and the live archive's InRelease files are back. The biased thresholds are defined where they are introduced. The docs files apply the fifth README review's findings and stay identical to the README branch. Model: fable-5-1 --- SPEC.md | 384 ++++++++++++++++++++++---------------------- docs/alerts.md | 13 +- docs/gitea.md | 72 +++++---- docs/image.md | 5 +- docs/metrics.md | 8 +- docs/request-log.md | 35 ++-- docs/settings.md | 35 ++-- docs/state-files.md | 34 ++-- 8 files changed, 303 insertions(+), 283 deletions(-) diff --git a/SPEC.md b/SPEC.md index c0f2dbd..13fa229 100644 --- a/SPEC.md +++ b/SPEC.md @@ -11,10 +11,10 @@ to them. ## Purpose - It protects one self-hosted app on the open internet from scrapers, scanners - and other abusers, from the app's first request, with no setting at all. Every - setting has a default chosen for a service facing the internet in 2026. A - setting that needs an alert destination, an account key or a token stays off - until the operator supplies it. + and other abusers. It does so from the app's first request, with no setting at + all. Every setting has a default chosen for a service facing the internet + in 2026. A setting that needs an alert destination, an account key or a token + stays off until the operator supplies it. - It limits each client's requests and bytes per minute, hour and day. It bounds the size and duration of every request and answer. It detects attacks. - It bans abusers: briefly at first, for seven days on a clear sign of attack, @@ -60,8 +60,8 @@ to them. - `internal/bans`: the bans. It holds each netblock's bans with their notes, decides how long a new ban lasts, and picks the ban to drop when the table of bans is full. - - `internal/rules`: reads the rule files, and again as they change, and - tells which rules a request matches. + - `internal/rules`: reads the rule files at start and again when they + change, and tells which rules a request matches. - `internal/waf`: the Core Rule Set, with the six changes listed under "Attack detection", as Coraza directives. - `internal/lookup`: looks clients up through GeoJS or in the lookup @@ -100,8 +100,8 @@ to them. ## What happens to a request The steps run in this order. The first step that produces a final answer ends -the handling. The size and time limits apply to the whole exchange. Each step -names the section that holds its rules. +the checks; step 15 runs for every request. The size and time limits apply to +the whole exchange. Each step names the section that holds its rules. 1. `GET /_smallwebwaf/healthz` is answered at once, before any check. 2. The client is identified. @@ -147,20 +147,19 @@ names the section that holds its rules. while `SWWAF_WAF_MODE` is `block`, and only logged and alerted while it is `detect`. 13. A request under `/_smallwebwaf/` is answered here (see "Admin endpoints"). -14. The request is forwarded to `SWWAF_UPSTREAM_URL`, streaming, with - `X-Forwarded-For` added. With `SWWAF_ADD_LOOKUP_HEADERS`, `X-Client-ASN` and - `X-Client-Country` are added for the app's own logs; a header whose value is - unknown is left out. Whatever that setting says, any such header the client - sent is removed. -15. After the response: - - The response bytes, the request bytes or both are added to the byte - counters, as `SWWAF_BYTES_COUNT` says. For a WebSocket, what it carried - each way counts once it closes. A client whose total passes a byte limit - times the percentage has broken that limit and is banned. The response in - progress is not cut off. - - A refusal by a rule file, a trap path, the Core Rule Set or a token check - is counted for the error burst (see "Attack detection"). No other refusal - is. +14. The request is forwarded to `SWWAF_UPSTREAM_URL`, streaming. + `X-Forwarded-For` is added, and `X-Request-ID` carries the request's id, + generated when traefik did not supply one. With `SWWAF_ADD_LOOKUP_HEADERS`, + `X-Client-ASN` and `X-Client-Country` are added for the app's own logs; a + header whose value is unknown is left out. Whatever that setting says, any + such header the client sent is removed. +15. After the response, or after the refusal: + - For a request forwarded to the app, the response bytes, the request bytes + or both are added to the byte counters, as `SWWAF_BYTES_COUNT` says. For a + WebSocket, what it carried each way counts once it closes. A client whose + total passes a byte limit times the percentage has broken that limit and + is banned. The response in progress is not cut off. + - The refusals the error burst counts (see "Attack detection") are counted. - The client's history and the anomaly counters are updated, and the alert thresholds are evaluated. @@ -183,7 +182,9 @@ the token check still apply. with its history. The bans are bounded by `SWWAF_MAX_BANS` (see "Bans"). The GeoJS answers, the reputation verdicts and scores, the DNS queries under way and the anomaly counters have fixed bounds, listed under "Fixed limits" in - `docs/settings.md`; past a bound, the entry used longest ago goes first. + `docs/settings.md`. Past its bound, the GeoJS answer or anomaly counter used + longest ago goes first, and the verdict or score fetched longest ago. Past + 1,000 DNS queries under way, a zone is asked at the client's next request. ## Bans @@ -211,7 +212,8 @@ to a ban, each by its own rule: - The first one bans the netblock for `SWWAF_ATTACK_BAN_DURATION` (default `7d`), with the cause `attack`. - Any further request from the netblock while that ban lasts shows it is - malicious: the ban becomes permanent. + malicious: the ban becomes permanent. No other cause of ban becomes + permanent this way. - Once that ban has run out, the netblock is served like any other. Its next clear sign of attack bans it permanently at once. - A broken limit: a request over a rate limit, a response that takes the @@ -238,8 +240,8 @@ A client on the CrowdSec decision list is banned until the decision ends, with the cause `crowdsec`. Only a client that sends a request is banned, so a long list does not fill `bans.json`. Such a ban is never made permanent. It does not make the netblock's next ban for a broken limit longer. A ban lifted while the -decision is in force is followed by another at the client's next request; the -decision is deleted in CrowdSec first, and the ban lifted once the list has been +decision is in force is made again at the client's next request. To lift it for +good, delete the decision in CrowdSec, then lift the ban once the list has been fetched again. These refuse each request they cover, record nothing in `bans.json`, and count @@ -258,9 +260,9 @@ state") or through the ban endpoints (see "Admin endpoints"). A ban an admin makes has the cause `admin`; an admin who wants to keep a ban `smallwebwaf` made sets its cause to `admin`. A ban lifted before it ends is kept, marked as lifted, and does not count toward a longer ban; it refuses nothing. Deleting its -entry from `bans.json` forgets it entirely. Only a ban whose cause is `attack` -becomes permanent at the first request it refuses. An admin whose own address is -banned lifts the ban by editing `bans.json`. +entry from `bans.json` forgets it entirely. An admin whose own address is banned +lifts the ban by editing `bans.json`, since the endpoints refuse a banned +client's request like any other. A permanent ban does not run out. It ends when an admin lifts it, or when `smallwebwaf` drops it from a full table, as below. When `smallwebwaf` holds @@ -274,8 +276,7 @@ table. ## Settings `docs/settings.md` lists every setting with its default and the values it takes, -in the groups below. It also gives the forms of values: lists, netblocks, -durations, sizes, countries, AS numbers, percentages and path prefixes. +in the groups below. It also gives the form each kind of value takes. Conventions: - These settings take `off`, which switches them off: @@ -301,21 +302,18 @@ Conventions: restarting the container. The files `smallwebwaf` watches while it runs are its state files, its rule files and the lookup database. - An invalid setting stops the process at start with one log line, marked - `"type":"process"` with `"msg":"invalid setting"`, naming the variable and - what is wrong with the value. runit then starts the process again every second - until the value is fixed (see "The two processes"). At start the settings in - effect are logged on one line; tokens, keys and the paths of webhooks, which - often carry a secret, are shown as `********`. + `"type":"process"` with `"msg":"invalid setting"`. The line names the variable + and what is wrong with the value; "The two processes" says what happens then. + At start the settings in effect are logged on one line; tokens, keys and the + paths of webhooks, which often carry a secret, are shown as `********`. ### Core -- One listener, `SWWAF_LISTEN_ADDR`, for the requests `smallwebwaf` forwards to - `SWWAF_UPSTREAM_URL` and for its own endpoints (see "Ports"). - `SWWAF_INSTANCE_NAME` is in every log line, metric and alert, so that instances can be told apart. The default, the container's host name, changes at every redeploy; a name set by the operator survives it. -- `SWWAF_MODE=observe` logs and alerts on every decision while refusing nothing, - so that a configuration can be tried before it is enforced. +- `SWWAF_MODE=observe` lets a configuration be tried before it is enforced (see + "What happens to a request"). - `X-Forwarded-For` is believed only from `SWWAF_TRUSTED_PROXIES`, by default the private address ranges, so a client that connects directly cannot claim another address. Where other containers reach `smallwebwaf` directly, or @@ -350,14 +348,12 @@ Conventions: ### Logging - The request log on stdout is always on and has no switch (see "Request log"). - The process's own messages share the stream; `SWWAF_LOG_LEVEL` filters only - those, and no message is at `debug`. `warn` and `error` hold back the line of - settings at start. + `SWWAF_LOG_LEVEL` filters only the process's own messages, and no message is + at `debug`. `warn` and `error` hold back the line of settings at start. - `SWWAF_LOG_REQUEST_HEADERS` names the request headers to record. `Authorization`, `Cookie` and `Set-Cookie` values are never logged, even if listed, only whether they were present. -- `SWWAF_LOG_REMOTE_URL` sends every log line to a syslog server as well, over - UDP, TCP or TLS (see "Request log"). +- `SWWAF_LOG_REMOTE_URL` names a syslog server (see "Request log"). ### Metrics @@ -370,7 +366,6 @@ Conventions: - `SWWAF_ALLOW_NETS` skips the checks step 3 names, for monitoring or the owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and byte limits only; the error burst, attack detection and bans still apply. - `SWWAF_DENY_NETS` is always refused. - Many clients share one address (mobile carriers, offices, Tor), and so share its limits and its bans. The defaults sit well above one person's use, and the first two lists take known shared addresses out. @@ -379,9 +374,8 @@ Conventions: Breaking one bans the client, so the defaults sit several times above what one busy person produces. A browser loading a heavy page makes a few hundred -requests, a git clone a handful, and several people often share one address. A -machine that talks to the app all day can still pass the day limit. Its address -belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. +requests, and a git clone a handful. A machine that talks to the app all day can +still pass the day limit. Its address belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. `SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as static assets or health checks; their bytes are not counted either. A prefix is @@ -394,10 +388,9 @@ matched the same way. ### Byte limits, per client -A response's bytes are counted when it ends, so a byte limit never cuts an -answer short; the client's next request is refused instead. Every default sits -above the largest request and answer together, so at the defaults no single -download breaks a limit. +A response's bytes are counted when it ends (see step 15). So every default sits +above the largest request and answer together, and no single download breaks a +limit. ### Size and time limits @@ -476,10 +469,9 @@ service that keeps its visitors' addresses on its own host. request to GeoJS. A client whose answer has not come in time counts as unknown until it comes, and its later requests do not wait. Its address is asked about again in the background. -- A client the lookup cannot place has an unknown country and AS number. The - biased thresholds give it `SWWAF_UNKNOWN_LIMIT_PERCENT`, and the exclusive - country list refuses it (see "Country lists"). Nothing else treats it - differently. +- A client the lookup cannot place has an unknown country and AS number, which + the country lists and the biased thresholds treat as described there. Nothing + else treats it differently. - GeoJS publishes no rate limit, but its terms forbid "an excessive amount of API requests", judged by GeoJS alone, and let it block a caller. While GeoJS is slow, down or refusing `smallwebwaf`, new clients count as unknown. GeoJS @@ -489,21 +481,19 @@ service that keeps its visitors' addresses on its own host. - The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`, `as_name` and `country_code` fields are read. The operator downloads it with a free IPinfo account and mounts it read-only; `smallwebwaf` never fetches it - and holds no account or token for it. IPinfo updates it daily, and releases it - under the Creative Commons Attribution-ShareAlike 4.0 licence, asking for a - link to IPinfo on the site that uses it, such as "IP address data is powered - by IPinfo". + and holds no account or token for it. IPinfo updates it daily. Its licence is + Creative Commons Attribution-ShareAlike 4.0, and IPinfo asks for a link to it + on the site that uses it, such as "IP address data is powered by IPinfo". - The file is read whole at start; one that is missing or unreadable stops the start. It is read again once it has gone 2 seconds without a change after being replaced. One that cannot be read is logged, sent as one `file_error` - alert and ignored. The database already loaded stays in use. The directory - that holds the file is mounted, not the file itself (see "Rule files" for - why). A client whose address the file does not hold counts as unknown. + alert and ignored. The database already loaded stays in use. A client whose + address the file does not hold counts as unknown. ### Country lists Both are empty by default: the defaults judge a client by what it does, not by -where it comes from. Clients in `SWWAF_ALLOW_NETS` are not checked. +where it comes from. - `SWWAF_DENIED_COUNTRIES` refuses every request from a listed country. - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` lets through only clients placed in a @@ -524,9 +514,10 @@ where it comes from. Clients in `SWWAF_ALLOW_NETS` are not checked. ### Biased thresholds -The lists are empty by default, for the same reason as the country lists. The -request log and the metrics show which AS numbers and countries a service's -abuse comes from. +The biased thresholds lower the limits of clients in listed AS numbers and +countries. The lists are empty by default, for the same reason as the country +lists. The request log and the metrics show which AS numbers and countries a +service's abuse comes from. - Clients in a listed AS number or country get that percentage of every request and byte limit (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A @@ -566,9 +557,8 @@ body. The rule files, the limits and the bans still apply to the client that sends it. - The Core Rule Set inspects only a request that every earlier check let - through. It skips a path under `SWWAF_WAF_EXEMPT_PATHS` and a client in - `SWWAF_ALLOW_NETS`. An engine error on a request forwards the request, and is - logged and counted. + through, and skips a path under `SWWAF_WAF_EXEMPT_PATHS`. An engine error on a + request forwards the request, and is logged and counted. - `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule Set's own two tuning values, at its own defaults. Each level up runs more rules, which find more attacks and take more ordinary requests for attacks. @@ -577,8 +567,8 @@ sends it. default one critical rule is enough. With the threshold `off`, nothing is a match, and scores are still logged. A request with more than 1000 query parameters or body fields adds 5 to its score, since Coraza reads only the - first 1000. Either way the log line gives the rule ids and the score, and a - `waf_block` alert is sent. + first 1000. For a match, in `block` and `detect` mode alike, the log line + gives the rule ids and the score, and a `waf_block` alert is sent. - `SWWAF_WAF_DISABLED_RULES` switches off rule ids when an app trips a false positive. The default switches off six rules: - 920340, 920420 and 920640 refuse a request for the type of its body, when @@ -605,19 +595,20 @@ sends it. to send the part that is read. Body inspection suits apps whose forms carry no code; `docs/gitea.md` records what it refuses in front of gitea. - `SWWAF_TRAP_PATHS` names paths the app never serves and only scanners ask for, - such as `/wp-login.php` in front of gitea. A request for one is a clear sign - of attack. This is the short form of a `path` rule with the `ban` action, for - deployments that mount no rule files. A trap path is matched whole, without - the query, against the path as sent and once percent-decoded: `/wp-login.php` - matches `/wp-login.php?x=1` and `/%77p-login.php`, and does not match - `/wp-login.php/` or `/blog/wp-login.php`. + such as `/wp-login.php` in front of gitea. This is the short form of a `path` + rule with the `ban` action, for deployments that mount no rule files. A trap + path is matched whole, without the query, against the path as sent and once + percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and + `/%77p-login.php`, and does not match `/wp-login.php/` or + `/blog/wp-login.php`. - `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute for a rule - file match, a trap path, a Core Rule Set match, or a missing or wrong token. - More than this breaks a limit (see "Bans"). A client trying one attack or - token after another is refused many times a minute; a person rarely more than - a few times. Answers the app gives are not counted, since they do not tell a - scanner from an ordinary client. In front of gitea, registry and git clients - get `404` and `401` by design, hundreds of times a minute. + file match, a trap path, a Core Rule Set match, or a missing or wrong token; + no other refusal counts. More than this breaks a limit (see "Bans"). A client + trying one attack or token after another is refused many times a minute; a + person rarely more than a few times. Answers the app gives are not counted, + since they do not tell a scanner from an ordinary client. In front of gitea, + registry and git clients get `404` and `401` by design, hundreds of times a + minute. `smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of @@ -645,15 +636,15 @@ every app, and no setting undoes them: - `artifactName` and `redirect_to`. gitea puts file paths, branch and workflow names, and the page to return to - after signing in into them. Any name on those lists can be an ordinary file - or branch, such as `.gitignore` or `docker-build`. In these parameters, - anything only these three rules catch reaches the app, commands such as - `|cat /etc/passwd` included. Path traversal, SQL and script injection, and - PHP, Java and Node.js code are still refused there. Every other parameter - keeps all three rules. Parameter names are matched without regard to case, - as Coraza matches them. An app that uses one of these parameters as a file - on the server, or passes it to a shell, must check the value itself. A rule - file can refuse it instead. + after signing in into them. It reads these values as names in a repository + or in its own records, or as a page of its own site, never as a command. Any + name on those lists can be an ordinary file or branch, such as `.gitignore` + or `docker-build`. In these parameters, anything only these three rules + catch reaches the app, commands such as `|cat /etc/passwd` included. Path + traversal, SQL and script injection, and PHP, Java and Node.js code are + still refused there. Every other parameter keeps all three rules. An app + that uses one of these parameters as a file on the server, or passes it to a + shell, must check the value itself. A rule file can refuse it instead. 5. Two cookies are left out, and `Referer` keeps fewer rules: - The Core Rule Set reads the request without the `gitea_flash` and @@ -677,6 +668,9 @@ every app, and no setting undoes them: 6. Responses are not inspected. A raw file from a repository, such as a shell script, looks to the response rules like source code leaking from the server. +The parameter and cookie names in these changes are matched without regard to +case, as Coraza matches them. + ### Ban lengths, response and scope `SWWAF_BAN_RESPONSE` is `403` by default, which makes a ban easy to recognise @@ -690,10 +684,10 @@ to the surrounding netblock. The four ban lengths are under "Bans". No source is on by default. A source acts on a list kept elsewhere; the defaults judge a client only by what it does here. AbuseIPDB and CrowdSec also need an account or an engine of the operator's own. The list best suited to be on by -default, Spamhaus DROP, must not be fetched more than once an hour, and Spamhaus -may block an address that does. Each `smallwebwaf` fetches its own copy, so -several apps on one host, sharing one address, would often fetch less than an -hour apart. +default, Spamhaus DROP, has terms, below, that forbid frequent fetching, and +Spamhaus may block an address that breaks them. Each `smallwebwaf` fetches its +own copy, so several apps on one host, sharing one address, would often fetch +too close together. - Blocklists (`SWWAF_BLOCKLIST_URLS`) are text files of addresses and netblocks, one to a line, as the Spamhaus DROP list is written. Anything after a `;` or @@ -718,33 +712,34 @@ hour apart. `AS:`. The log, the alerts, the ban notes and the metrics name each list by its - URL, so a secret stays out of it. + URL, so a list's URL must not hold a secret. - `SWWAF_BLOCKLIST_ACTION` is `deny` by default, which refuses every request from a listed address and suits lists of networks that send nothing legitimate, such as DROP. `limit:` gives listed clients that - percentage of every limit, and suits a list of addresses shared with ordinary - visitors, such as Tor exits. `log` only notes the hit in the log line and - raises an alert. + percentage of every request and byte limit, and suits a list of addresses + shared with ordinary visitors, such as Tor exits. `log` only notes the hit in + the log line and raises an alert. - A CrowdSec engine on the host (`SWWAF_CROWDSEC_LAPI_URL`) has its decision list fetched every minute, with `SWWAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key` header. The list is kept like a fetched blocklist, and a client on it is banned (see "Bans"). Only decisions of the type `ban` on an address or a netblock are used; any other, such as a captcha or a country, is left out. A - decision lasts for the `duration` it had left when the engine answered. A - fetch fails on an answer other than `200`, a redirect included, on one not - finished within a minute, on one longer than 16 MiB, and on one that is not a - list of decisions. This is how a fleet-wide blocklist can arrive without - `smallwebwaf` depending on CrowdSec. + decision lasts for the `duration` it had left when the engine answered. This + is how a fleet-wide blocklist can arrive without `smallwebwaf` depending on + CrowdSec. A fetch fails when: + - the engine answers other than `200`, a redirect included; + - the fetch does not finish within a minute; + - the answer is longer than 16 MiB; + - the answer is not a list of decisions. - Each DNS blocklist (`SWWAF_DNSBL_ZONES`) is asked about a client's own address - in the background, the first time the client is seen; the request goes on at - once, and so does every request from the client until the zone has answered. - The name asked about is the one RFC 5782 gives: the numbers of an IPv4 address - reversed, or the 32 hex digits of an IPv6 address reversed, so `192.0.2.99` is - asked about as `99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` - lists the client, but one in `127.255.255.0/24` is Spamhaus refusing the - query. A name that does not exist does not list the client. These are - failures, after which the zone is not asked again for a minute: + in the background, the first time the client is seen. No request waits for the + answer. The name asked about is the one RFC 5782 gives: the numbers of an IPv4 + address reversed, or the 32 hex digits of an IPv6 address reversed. So + `192.0.2.99` is asked about as `99.2.0.192.dnsbl.dronebl.org`. An answer in + `127.0.0.0/8` lists the client, but one in `127.255.255.0/24` is Spamhaus + refusing the query. A name that does not exist does not list the client. These + are failures, after which the zone is not asked again for a minute: - an answer in `127.255.255.0/24`; - an answer outside `127.0.0.0/8`; - a resolver error; @@ -752,10 +747,10 @@ hour apart. DNS blocklists made for mail servers list residential ranges, and would block ordinary visitors, so they should not be used. Spamhaus zones need - their keyed query service, given as a zone name containing the key, which - `smallwebwaf` shows as `********` wherever it names the zone; only - `reputation.json` keeps the zone with its key. `SWWAF_DNSBL_RESOLVER` names - a resolver, since public resolvers are refused by several list operators. + their keyed query service, given as a zone name that contains the key. + `smallwebwaf` shows the key as `********` wherever it names the zone. Only + `reputation.json` keeps the key. `SWWAF_DNSBL_RESOLVER` names a resolver, + since public resolvers are refused by several list operators. - AbuseIPDB (`SWWAF_ABUSEIPDB_KEY`) is asked, at its check endpoint, for the abuse confidence score of a client, from 0 to 100. Only a client that has @@ -767,14 +762,16 @@ hour apart. address of the request that set off the check, and the score is used for the whole group. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit. The budget is counted in UTC days and kept in `reputation.json`, so a restart does - not make it whole again; the check that spends the last of it is logged and - alerted, and no client is checked until the day ends. A check fails on an - answer other than `200`, on one without a score, and on no answer within - `SWWAF_REPUTATION_TIMEOUT`; after a failure no client is checked for a minute. - The key is sent in the `Key` header and nowhere else, and is never logged. + not make it whole again. The check that spends the last of it is logged and + alerted, and no client is checked until the day ends. The key is sent in the + `Key` header and nowhere else, and is never logged. A check fails, after which + no client is checked for a minute, when: + - AbuseIPDB answers other than `200`; + - the answer gives no score; + - no answer comes within `SWWAF_REPUTATION_TIMEOUT`. - `SWWAF_REPUTATION_ACTION`, for DNS blocklist and AbuseIPDB hits, is `limit:25` by default. Such verdicts are less certain than a blocklist, so a listed - client gets a quarter of every limit rather than a refusal. + client gets a quarter of every request and byte limit rather than a refusal. - Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and in `reputation.json`, with the fixed bounds under "Fixed limits" in `docs/settings.md`. A source that is down or over quota gives no verdict. The @@ -797,13 +794,13 @@ hour apart. counted by event, so a wide attack cannot flood the channel. Once the hour ends, one alert with the event `summary` gives those counts. It also counts the repeats that a cooldown held back and that no later alert reported. -- Each destination has a queue of at most 1000 alerts, sent one at a time, so a - slow destination holds up nothing; when it is full, the oldest alert is - dropped, and the drops are counted in the metrics. A `2xx` answer takes an - alert. A `4xx` other than `408` and `429` refuses it, and it is dropped. - Anything else is a failure: the alert is sent again a second later, and twice - as long after each further failure, up to a minute. `alerts.json` keeps the - queues and cooldowns across a restart. +- Each destination has a bounded queue (see "Fixed limits" in + `docs/settings.md`), sent one at a time, so a slow destination holds up + nothing. When a queue is full, its oldest alert is dropped and counted in the + metrics. A `2xx` answer takes an alert. A `4xx` other than `408` and `429` + refuses it, and it is dropped. Anything else is a failure: the alert is sent + again a second later, and twice as long after each further failure, up to a + minute. `alerts.json` keeps the queues and cooldowns across a restart. ### Anomaly thresholds @@ -814,9 +811,12 @@ which the metrics show. They reveal a widely distributed scraper, thousands of addresses at low rates each, which the per-client limits do not see. A low `SWWAF_ASN_LIMIT_PERCENT` for its AS number is the response. -- Requests and bytes are counted per minute and per hour, for each client, each - surrounding netblock, each AS number, the whole service, and each named - netblock in `SWWAF_WATCH_NETS`. +- Requests and bytes are counted per minute and per hour, for each of: + - a client; + - the surrounding netblock; + - an AS number; + - the whole service; + - a named netblock in `SWWAF_WATCH_NETS`. - Every request but the health check is counted, whatever was done with it, clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` included, since an exempt client misbehaving is worth knowing about. The counters are @@ -825,8 +825,8 @@ addresses at low rates each, which the per-client limits do not see. A low ## Rule files `smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR` and checks each -request against them, unless the client is in `SWWAF_ALLOW_NETS`. -`docs/rules.md` gives the format, the targets and actions, and the default file. +request against them. `docs/rules.md` gives the format, the targets and actions, +and the default file. - The files are plain text meant to be edited by hand, so that a new scanning pattern seen in the request log becomes a rule in one line. `smallwebwaf` @@ -835,7 +835,7 @@ request against them, unless the client is in `SWWAF_ALLOW_NETS`. for 2 seconds, so that a file still being written is read whole. A file whose name starts with `.`, such as an editor's lock file, is not a rule file. - Files are read in name order (`00-default.rules` before `50-gitea.rules`), - rules in line order. Matching stops at the first rule that refuses. + rules in line order. - A rule's regex is Go's RE2 syntax. It has no backreferences or lookaround; in exchange matching time is linear in the input, so no rule can be made to stall the proxy. @@ -843,18 +843,16 @@ request against them, unless the client is in `SWWAF_ALLOW_NETS`. encoded probe cannot slip past. A header sent more than once is matched with its values joined by `, `; a header not sent is empty text. `Host` and `Transfer-Encoding` cannot be targets, since Go's HTTP server takes them out - of every request. Request bodies are not available to rule files; reading them - is the Core Rule Set's job. + of every request; a rule naming one stops the start like any bad line. Request + bodies are not available to rule files; reading them is the Core Rule Set's + job. - `log` only notes the match in the request log. `block` refuses the request - without a ban; `ban` makes the request a clear sign of attack (see "Bans"). - `ban` is for requests no real visitor sends, anchored at the site root. A file - of the same name deeper in a site can be ordinary content, such as a file in a - gitea repository. A visitor banned by mistake is let back in by lifting the - ban. + without a ban; `ban` makes the request a clear sign of attack (see "Bans"), + and is for requests no real visitor sends. A visitor banned by mistake is let + back in by lifting the ban. - At start, these stop the process with a message naming the file and line: - a line that does not parse; - a regex that does not compile; - - a rule for `Host` or `Transfer-Encoding`; - a duplicate id. So does a `SWWAF_RULES_DIR` that does not exist or cannot be read. While @@ -867,10 +865,11 @@ request against them, unless the client is in `SWWAF_ALLOW_NETS`. wrong for every app, with its path rules anchored at the site root. In front of gitea, `/.env` is a probe, while `///src/branch/main/.env.example` is a file in a repository that - any visitor may open. Anything app-specific, such as a rule banning WordPress - probes in front of gitea, belongs in a file the app's Dockerfile adds or the - deployer mounts. A directory mounted over `SWWAF_RULES_DIR` replaces the - default file; single files mounted into it add to it. + any visitor may open. `ban` rules of your own follow the same rule. Anything + app-specific, such as a rule banning WordPress probes in front of gitea, + belongs in a file the app's Dockerfile adds or the deployer mounts. A + directory mounted over `SWWAF_RULES_DIR` replaces the default file; single + files mounted into it add to it. - Rules meant to be edited while `smallwebwaf` runs belong in a mounted directory, not a mounted file. Many editors save a file by replacing it, and a container keeps seeing the old one when the file alone is mounted. The lookup @@ -880,11 +879,10 @@ request against them, unless the client is in `SWWAF_ALLOW_NETS`. ## Persistent state All state lives in memory, and the files in `SWWAF_STATE_DIR` hold a copy of all -of it, so an orderly stop loses nothing. The one exception is log lines still -waiting to be sent to `SWWAF_LOG_REMOTE_URL`, which stdout has already carried. -The files are meant for people as well: an admin can read or edit them at any -time, and the running `smallwebwaf` takes the edit in. `docs/state-files.md` -lists the files, what each holds, and their sizes. +of it, so an orderly stop loses nothing (the remote log is the exception; see +"Request log"). The files are meant for people as well: an admin can read or +edit them at any time, and the running `smallwebwaf` takes the edit in. +`docs/state-files.md` lists the files, what each holds, and their sizes. - Each file holds one kind of state, as JSON with: - a top-level `version` number; @@ -897,9 +895,9 @@ lists the files, what each holds, and their sizes. halves their size and lets `grep` show everything about one client. - Clients that were never banned are kept too, so every client's history survives a restart. -- Each file has a bound, so it can be rewritten whole. Writing a file of these - sizes takes well under a second, in the background, so rewriting whole files - needs nothing cleverer. +- Each file has a bound, so it can be rewritten whole. Writing one takes well + under a second, in the background, so rewriting whole files needs nothing + cleverer. - A write serialises from a snapshot taken under the lock, so requests are not held up. It goes to a temporary file in the same directory, which is synced and renamed over the real name, and the directory is synced. A crash at any @@ -907,23 +905,26 @@ lists the files, what each holds, and their sizes. partial one. Before writing a file, `smallwebwaf` takes in any edit made to it on disk, so an admin's edit is never overwritten. - At start, the files are read back. Active bans go into an in-memory lookup by - netblock, everything else into maps. Counter buckets and cache entries whose - time has passed are discarded. A missing file means empty state, and is normal - on first run. A file that does not parse, or has an unknown `version`, stops - the process with a message naming the file and position; so does an entry - without a field it needs, a ban whose cause is not one of the four, or a ban - whose `expires` is not after its `start`. Starting with empty state would - silently forgive every repeat offender. Since writes are atomic, a broken file - can only come from a hand edit, which the editor should hear about. + netblock, everything else into maps. Expired cache entries are discarded. A + missing file means empty state, and is normal on first run. These stop the + process with a message naming the file and position: + - a file that does not parse, or has an unknown `version`; + - an entry without a field it needs; + - a ban whose cause is not one of the four; + - a ban whose `expires` is not after its `start`. + + Starting with empty state would silently forgive every repeat offender. + Since writes are atomic, a broken file can only come from a hand edit, which + the editor should hear about. + - While running, `smallwebwaf` notices a file that is edited, replaced or added. It tells its own writes from an admin's by comparing the file with what it last read or wrote. An edit that parses is taken in at once: what the file - says replaces what `smallwebwaf` held for that file. Removing a ban's entry - from `bans.json` lifts the ban; adding an entry bans; setting `lifted` lifts - it and keeps it. A change `smallwebwaf` made after the admin opened the file, - such as a new ban, is lost when the admin saves over it. Most editors warn - when a file changed on disk while it was open. A file that is removed is - written again at its next write. + says replaces what `smallwebwaf` held for that file. An edit to `bans.json` + makes, lifts or forgets bans as "Bans" describes. A change `smallwebwaf` made + after the admin opened the file, such as a new ban, is lost when the admin + saves over it. Most editors warn when a file changed on disk while it was + open. A file that is removed is written again at its next write. - An edit that does not parse never stops the running `smallwebwaf`. It keeps the state it has and sets the edited file aside as `.bad`, so the edit is kept. It writes the file again from memory, and names the file and the @@ -999,8 +1000,6 @@ endpoints, what each takes and what each answers. the error burst. A client guessing tokens is therefore banned once it passes `SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute. - A ban made through the endpoints starts at once, with the cause `admin`. - Lifting through the endpoints lifts every active ban covering the address - given. ## Deployment @@ -1017,8 +1016,10 @@ what the image holds and how each part is pinned. need older versions of packages the image holds, and it moves forward with the image's digest. The Dockerfile names the hash of each of the snapshot's `InRelease` files, which cover every package list and so every package, and - the build checks them before installing. The hashes are the amd64 archive's, - so the image is built for amd64 only. + the build checks them before installing. The live archive's `InRelease` files, + which `apt-get update` also fetches and the install does not use, are left out + of the check. The hashes are the amd64 archive's, so the image is built for + amd64 only. - The Ubuntu image has no CA certificates. Without them Nix cannot download packages and `smallwebwaf` cannot reach GeoJS, so `ca-certificates` is installed by name. The snapshot itself is fetched over HTTPS with the CA @@ -1032,15 +1033,14 @@ what the image holds and how each part is pinned. own commands; busybox's `sv`, for one, looks for services elsewhere. An app built on the image gets the same packages each time. Security fixes from Ubuntu and nixpkgs come with a newer `smallwebwaf` image, which an app takes - by changing the digest in its `FROM` line; the image moves to each new nixpkgs + by changing the digest in its `FROM` line. The image moves to each new nixpkgs release, every six months. - runit supervises the two processes, with `runsvinit` as the entrypoint, as the owner's code style guide asks for service containers. No package carries `runsvinit`, so the image builds it from its archived source, unchanged since 2015, at a fixed commit hash, with a `go.mod` the build writes. `runsvinit` starts runit's `runsvdir`, which runs the `run` script of each directory under - `/etc/service` and runs it again whenever it exits. Ubuntu's `sv` looks for - services there too. + `/etc/service`. Ubuntu's `sv` looks for services there too. - The image's `run` script for `smallwebwaf` waits one second and makes `SWWAF_STATE_DIR` and every file in it belong to the `smallwebwaf` user (uid and gid 65532). Then it starts `smallwebwaf` as that user with runit's @@ -1072,19 +1072,20 @@ It sets no `ENTRYPOINT` or `USER` of its own. The container must start script starts as root and hands over to its program's user with `chpst`. Both services start with the container's environment variables and write to the container's output, which `docker logs` shows. -- When `smallwebwaf` or the app exits, runit starts it again a second later. The - other process keeps running, and the container does not stop. While - `smallwebwaf` is down, traefik answers `502`. While the app is down, - `smallwebwaf` answers `502`, which is not a client offence. +- When `smallwebwaf` or the app exits, runit runs its `run` script again, which + waits a second before starting it. The other process keeps running, and the + container does not stop. While `smallwebwaf` is down, traefik answers `502`. + While the app is down, `smallwebwaf` answers `502`, which is not a client + offence. - Only four things stop `smallwebwaf`, all at start: - invalid configuration; - a lookup database that is missing or unreadable; - a rule file or state file that does not parse; - an unwritable `SWWAF_STATE_DIR`. -- A `smallwebwaf` that stops at start is started again every second, logging its - message each time, until the cause is fixed; the health check fails meanwhile. - Once running, a failing GeoJS, reputation source or alert destination, or a - broken edit, never takes the protected service down. +- A `smallwebwaf` that stops at start is therefore started again every second, + logging its message each time, until the cause is fixed; the health check + fails meanwhile. Once running, a failing GeoJS, reputation source or alert + destination, or a broken edit, never takes the protected service down. - `docker stop` sends `SIGTERM` to `runsvinit`, which has runit stop both services at once, each with `SIGTERM`, and exits once they have stopped. `smallwebwaf` stops taking requests, gives those in progress five seconds to @@ -1126,9 +1127,8 @@ beside the app's data directories, belonging to the `smallwebwaf` user. Without a volume there, the files survive a restart of the container and are lost when a deploy replaces it. A volume mounted at `/var/lib/smallwebwaf`, named or a host directory, keeps them across deploys. A host directory mounted there needs no -change of owner, since the `run` script gives it to the `smallwebwaf` user at -each start; files an earlier owner left in it can be read and replaced. The -volume is separate from the app's, holds a few tens of MiB at most with the +change of owner, and files an earlier owner left in it can be read and replaced. +The volume is separate from the app's, holds a few tens of MiB at most with the defaults, and needs no backup beyond the host's own. The image declares no volume, since every app image built on it would inherit it. @@ -1164,8 +1164,8 @@ and `script/gitea-requests` checks those notes. ## Planned, not built -Everything above is built. These two points are specified as later work, and -nothing in the code does them yet: +These two points are specified as later work, and nothing in the code does them +yet: - Planned: sharing bans between the instances of several apps, by running one CrowdSec engine per host that every instance reports its bans to. It is diff --git a/docs/alerts.md b/docs/alerts.md index 1da7174..b35e1a5 100644 --- a/docs/alerts.md +++ b/docs/alerts.md @@ -78,11 +78,14 @@ on one line: - `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`: one of the events above. -- `client`: the address whose request raised the alert. `netblock`: the ban's - netblock, the netblock an anomaly counted, or, for the other events, the - client as rate limiting counts it: an IPv4 address as a /32, or an IPv6 group. - Both are empty for `source_failure` and `file_error`. -- `asn`, `as_name` and `country`: the client's as known when the alert was +- `client`: the address whose request raised the alert. Empty for + `source_failure` and `file_error`. +- `netblock`: empty for `source_failure` and `file_error`; otherwise one of: + - the ban's netblock; + - the netblock an anomaly counted; + - for the other events, the client as rate limiting counts it: an IPv4 + address as a /32, or an IPv6 group. +- `asn`, `as_name` and `country`: the client's, as known when the alert was raised. They are empty when GeoJS had not answered yet. - `reason`: a short sentence, such as `requests per minute over the limit of 1000` or `listed by a blocklist`. diff --git a/docs/gitea.md b/docs/gitea.md index 46b8143..312d778 100644 --- a/docs/gitea.md +++ b/docs/gitea.md @@ -21,10 +21,10 @@ checked again when gitea or the Core Rule Set changes version. - a release attachment. At the defaults, an upload larger than 100 MiB or taking more than 60 - seconds is cut off, and no one is banned for it. "Size and time limits" in - `SPEC.md` gives the answers the client gets. A gitea that takes large - uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and - `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. + seconds is cut off ("Size and time limits" in `SPEC.md`). A gitea that takes + large uploads needs `SWWAF_REQUEST_MAX_BYTES`, + `SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised + to fit. - Archive download and blame or history pages are what scrapers hammer. The request limits do most of the work there. @@ -63,15 +63,15 @@ toward). The request log names the rule in `waf_rule_ids`, which ### A query string that reads as an attack -Most often this is a search. Every query parameter is checked against the Core -Rule Set's lists of system files and commands, except the ten named under -"Attack detection" in `SPEC.md`. A release attachment uploaded through the API -as `docker-compose.yml` is refused the same way. Examples: +Most often this is a search. A search term is a query parameter, which the Core +Rule Set checks against its lists of system files and commands (see the fourth +change under "Attack detection" in `SPEC.md`). A release attachment uploaded +through the API as `docker-compose.yml` is refused the same way. Examples: - a name on those lists, such as `package.json`, `.gitignore` or `docker-compose.yml`; -- text that starts with a command name, or whose first word has one right after - a `/`, such as `python3`, `ssh key` or `feat/docker-support`; +- text that starts with a command name, such as `python3` or `ssh key`, or has + one right after a `/` in its first word, such as `feat/docker-support`; - a shell command with its options or a system path (`ls -la`, `sed -i`, `/bin/sh`), or a command in backticks; - script code (`fetch(`, `${VAR}`, `process.env`), HTML (`:` lines, one to a line. A URL that is also - in `SWWAF_BLOCKLIST_URLS` is refused. + text file with one AS number and percentage on each line, such as + `AS64496:50`. A URL that is also in `SWWAF_BLOCKLIST_URLS` is refused. ## Attack detection @@ -260,8 +261,8 @@ set to a value its entry does not allow stops the start. `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): the event types alerts are sent for, from those in [`alerts.md`](alerts.md) but `summary`. Any other name is refused. -- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held - back. A duration or `off`. +- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long after an alert is sent its + repeats are held back. A duration or `off`. - `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. A whole number above zero, or `off`. @@ -294,10 +295,8 @@ as `REQUESTS_PER_MINUTE`: ## Settings given as files -Add `_FILE` to a setting's name to read its value from a file: -`SWWAF_METRICS_TOKEN_FILE` names the file that holds `SWWAF_METRICS_TOKEN`. To -keep a token from the app, make the file on the host, owned by uid 65532 with -mode `0400`: +To give a token as a file, as "Tokens" under "Deployment" in +[`SPEC.md`](../SPEC.md) describes, make the file on the host: ```sh mkdir -p /srv/app/tokens @@ -306,7 +305,7 @@ chown 65532:65532 /srv/app/tokens/metrics chmod 0400 /srv/app/tokens/metrics ``` -Mount its directory into the container read-only, and name the file: +Then mount its directory into the container read-only, and name the file: ```sh docker run \ diff --git a/docs/state-files.md b/docs/state-files.md index 0c00910..eb8a4f5 100644 --- a/docs/state-files.md +++ b/docs/state-files.md @@ -15,8 +15,8 @@ those `smallwebwaf` made. Per entry: - `lifted`: when an admin lifted it, if one did; - `notes`: the notes below. -To ban a netblock, add an entry with its `netblock`, `start` and `expires`. -`reason`, `notes` and `cause` may be left out; the cause is then `admin`: +An entry an admin adds needs `netblock`, `start` and `expires`; `reason`, +`notes` and `cause` may be left out, and the cause is then `admin`: ```json { @@ -32,20 +32,23 @@ To ban a netblock, add an entry with its `netblock`, `start` and `expires`. } ``` -To lift a ban, add `"lifted"` with the time you lift it, such as -`"lifted": "2026-10-06T13:00:00Z"`. To keep a ban `smallwebwaf` made, set its -`cause` to `admin`. To forget a ban, delete its entry. - -The notes: +"Bans" in [`SPEC.md`](../SPEC.md) says how an edit lifts, keeps or forgets a +ban. The notes: - `asn`, `as_name` and `country`: the client's, added when the lookup answers (empty when lookups are off); -- what was broken: the rule id and target that matched, or the trap path, or the - limit, its window and the count reached; -- the client's percentage of the limit, with the setting that gave it; -- the reputation sources the client is on; -- `request`: the request that caused the ban: time, method, host, path with its - query string, status and user agent, each text cut to 256 bytes; +- for a broken limit, `kind` (`requests`, `bytes` or `refusals`), `limit`, + `window` and `count`: what the limit was on, the limit, its window and the + count reached; +- `limit_percent` and `limit_percent_setting`: the client's percentage of that + limit and the setting that gave it, when a setting lowered it; +- for a clear sign of attack, `rule_id` and `target`, or `trap_path`; +- `reputation`: the sources the client was on, each with its `source` and, for + AbuseIPDB, its `score`; +- `request`: the request that caused the ban, each text cut to 256 bytes: + - `time`, `method` and `host`; + - `path`, with its query string; + - `status` and `user_agent`; - `requests` and `refused`: the netblock's requests since it was first seen, and how many this ban has refused so far, kept up to date while the ban lasts; - `earlier_bans`: how many earlier bans the netblock had, by cause. @@ -93,8 +96,9 @@ GeoJS answers, one per line: `SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of disk writes a day. - `bans.json`: about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at - the default `SWWAF_MAX_BANS`. With a full file, a hundred new bans a day come - to about 600 MiB of writes. + the default `SWWAF_MAX_BANS`. Written every 15 minutes, a full file comes to + about 600 MiB of writes a day, and a hundred new bans a day add about 600 MiB + more. - `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when full. - `reputation.json` and `alerts.json`: usually a few MiB or less. -- 2.54.0 From ef03d15ee2f89f38ed2bca020615f8883063a0b0 Mon Sep 17 00:00:00 2001 From: sneak Date: Thu, 22 Oct 2026 20:03:29 +0000 Subject: [PATCH 7/8] README rewritten from scratch so a newcomer can follow it (closes #154, closes #155) The old README opened with project history and one list of a hundred settings. The new one leads with what smallwebwaf is, what it protects against, how to build and deploy it, what happens to a request, the settings most deployments touch, how to check it works, the entrypoints, the rationale and the design, as REPO_POLICIES.md requires. The reference lists, every setting, the rule file format, the endpoints, the request log's fields, the alert events, the metrics, the state files, the image and the gitea notes, move to files under docs/, each linked from the README, with SPEC.md as the one place for behaviour. Every statement was checked against the code on next. Model: fable-5-1 --- README.md | 2708 ++++++++--------------------------------------------- 1 file changed, 365 insertions(+), 2343 deletions(-) diff --git a/README.md b/README.md index 3cc8261..6292ef5 100644 --- a/README.md +++ b/README.md @@ -1,77 +1,73 @@ # smallwebwaf -`smallwebwaf` is a simple, fast, logging web application firewall, MIT-licensed -and written in Go by [@sneak](https://sneak.berlin), for people who host their -own services. It runs inside the container of the one application it protects, -between your reverse proxy (traefik) and the app: the app's Dockerfile builds -`FROM` the `smallwebwaf` image, traefik sends the app's requests to -`smallwebwaf` on port 8080, and `smallwebwaf` passes them on to the app on -`127.0.0.1:8081`. It needs no setting, and protects the app from the first -request with defaults chosen for a service on the open internet. It keeps its -state in memory and in JSON files you can read and edit, and writes a detailed -JSON log line for every request. +`smallwebwaf` is an MIT-licensed web application firewall, written in Go by +[@sneak](https://sneak.berlin), for one self-hosted web app. It runs inside the +app's own container, in front of the app. It refuses the requests of scrapers, +scanners and other abusers before they reach the app. -Status: the first two milestones are built -(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and -https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are nine parts of -milestone 3: the static lists, the bans that broken rate limits lead to, the ban -ledger with the bans you make, keep and lift, the JSON state files with your -edits taken in while it runs and the paths the rate limits do not count, which -come next in the build order, `observe` mode and the rest of the request log's -fields, which come a little later, and the metrics endpoint and the header size -and the idle time as settings, which come last in it. So are the four parts of -the stage after it: the rule files, with the bans for a clear sign of attack, -the other admin endpoints, alerts to all three destinations, a JSON webhook, -Slack and ntfy, and remote log sending. So is the stage after that: the AS -number and country of every client, looked up through GeoJS or in the IPinfo -Lite database file, the byte limits, the biased thresholds, lower limits for the -AS numbers and countries you list, and the anomaly thresholds, alerts for -unusual traffic that refuse nothing. So is the stage after that: the blocklists -you name by URL, which it fetches and keeps, with a file of AS numbers' -percentages fetched the same way, the DNS blocklists (DNSBL zones), which it -asks about each client in the background, AbuseIPDB, which it asks in the -background about each client that has committed an offence, and the decision -list of a CrowdSec engine you run, which it fetches and keeps as a blocklist. So -is the last stage: attack detection with the OWASP Core Rule Set, run by Coraza, -on the method, the URL and the headers of each request, the trap paths and the -error burst. `smallwebwaf` passes each request to the app and the app's answer -back, unchanged, within its timeouts and size limits, works out each client's -address, looks up its AS number and country unless you switch that off, bans a -client that sends too many requests or too many bytes, not counting those for -the paths you choose, with lower limits for the clients of the AS numbers and -countries you list, refuses a client that comes from a country you refuse or -from a network you refuse, refuses, limits or only notes a client a blocklist or -a DNSBL zone you name lists, or AbuseIPDB scores at or over the score you set, -bans a client your CrowdSec engine's decision list lists until that decision -ends, lets the networks you choose through, checks each request against the rule -files and the trap paths you name and bans a client whose request is a clear -sign of attack, refuses a request the Core Rule Set takes for an attack, bans a -client it refuses again and again within a minute for a rule, a trap path, the -Core Rule Set or a missing or wrong token, keeps its bans, each client's -counters and history, GeoJS's answers, the last good copy of each list it -fetches, the DNSBL zones' verdicts, AbuseIPDB's scores and the AbuseIPDB checks -spent today in JSON files across restarts, takes in your edits of those files, -such as a ban you make, keep or lift, and of the rule files while it runs, -writes a JSON log line for every request, sends its log lines to a syslog server -too if you name one, sends an alert to a webhook, to Slack and to ntfy, each if -you name one, for each ban it makes or makes permanent, for a request the Core -Rule Set takes for an attack, for traffic over an anomaly threshold you set, for -a client a blocklist, the CrowdSec decision list, a DNSBL zone or AbuseIPDB -lists, for GeoJS failing, a list it cannot fetch, a DNSBL zone or AbuseIPDB that -fails or refuses a query and the day's AbuseIPDB checks used up, for a rule file -or state file with an error and for a replacement of the lookup database it -cannot read, serves Prometheus metrics to a scraper that holds the metrics -token, lets an admin who holds the admin token list, add and lift bans and ask -what it knows of a client, and in `observe` mode passes on the requests it would -refuse, logging what it would have done with them. It comes as the image the -app's own image is built on. The Core Rule Set reads a request's body too once -`SWWAF_WAF_BODY_LIMIT` is set. The rest of the design comes next, in the order -of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools that -led to the design is in [`EVALUATION.md`](EVALUATION.md). +A web application firewall reads each request on its way to a web app and +refuses the ones that look like abuse. `smallwebwaf` is also a reverse proxy: it +accepts requests on the app's behalf, passes them to the app, and passes the +app's answers back. A request it does not refuse reaches the app as it came, +apart from a few headers (see "Design" below). + +It needs no setting. Every setting has a default chosen for a service on the +open internet. [`SPEC.md`](SPEC.md) says how `smallwebwaf` behaves and why. The +reference documents under [`docs/`](docs/) are listed under "Documents" below. + +## What it protects against + +- Scrapers that fetch every page of a site, all day. Rate limiting gives each + client 1000 requests a minute, 10,000 an hour and 50,000 a day. Byte limits + give it 10 GiB a minute, 20 GiB an hour and 50 GiB a day. A client over a + limit is refused and banned. The first ban lasts an hour. Each repeat within a + day makes the next ban three times as long. A ban that would last more than + seven days is permanent. +- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The + default rule file bans such a client for seven days at its first probe. Its + next request during the ban makes the ban permanent. The default rule file + does not cover probes for other apps, such as a WordPress login. A `ban` rule + of your own, or an entry in `SWWAF_TRAP_PATHS`, bans those the same way. +- Attacks carried in a request's URL or headers, such as SQL injection and path + traversal. The OWASP Core Rule Set is a published set of rules that recognise + common web attacks. Coraza, a Go engine, runs those rules (step 11 under "What + happens to a request" below). A request the rules take for an attack is + refused. +- Abuse from networks and countries you do not want. You can: + - refuse whole netblocks and countries; + - give the clients of chosen networks and countries a fraction of every + limit, so that they are banned sooner; + - refuse or limit clients on a blocklist or a DNS blocklist, or with a high + AbuseIPDB score; + - ban clients named by your own CrowdSec engine. +- Slow and oversized requests. Every request and every answer has a time limit + and a size limit. + +For the operator, it: + +- writes one JSON log line for every request, saying what it did and why (see + [`docs/request-log.md`](docs/request-log.md)); +- sends alerts to a webhook, Slack or ntfy (see + [`docs/alerts.md`](docs/alerts.md)); +- serves Prometheus metrics (see [`docs/metrics.md`](docs/metrics.md)); +- keeps its bans, counters and history in JSON files you can read and edit, so + that a restart loses nothing and a ban can be lifted with an editor (see + [`docs/state-files.md`](docs/state-files.md)). + +Not built, and not planned: + +- TLS and routing for several apps. Both stay with traefik, the reverse proxy in + front of the container. +- Browser challenges such as captchas. +- A web console. +- Defence against floods large enough to fill the host's network link. ## Getting started -Build the `smallwebwaf` image from a clone: +Build the image from a clone. `make docker` runs the tests and the linter, then +builds the image, tagged `smallwebwaf`. It builds for amd64, on an amd64 host +only: the `Dockerfile` checks Ubuntu's package lists against the hashes of the +amd64 archive. ```sh git clone https://git.eeqj.de/sneak/smallwebwaf.git @@ -79,1832 +75,18 @@ cd smallwebwaf make docker ``` -`make docker` runs the tests and the linter, then builds the image, tagged -`smallwebwaf`, for amd64 and only on an amd64 host: the hashes the `Dockerfile` -checks Ubuntu's package lists against are those of Ubuntu's amd64 archive. Push -it to a registry your hosts pull from, and build each app's image on it, pinned -by digest, as "How it works, in short" below shows. `make example-app` builds a -small app on the image, the one in `deploy/example-app`, and checks that it -works. - -To work on the code, `make build` builds the binary alone, with Go installed, -and `make run` builds and runs it, listening on port 8080 in front of an app at -`SWWAF_UPSTREAM_URL`, by default `http://127.0.0.1:8081`, with its state files -in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of -`share/rules.d` unless `SWWAF_RULES_DIR` is set. - -## What it does so far - -- Passes each request to the app and the app's answer back unchanged: method, - path, query, headers, body and status. The one exception is a path starting - with `//` that holds a character Go escapes, such as `|`: Go cannot send it as - it is, and the app receives it decoded and escaped again, `%2F` as `/`. Bodies - stream through in both directions and are never held whole in memory, but for - the part of a request body the Core Rule Set reads, at most - `SWWAF_WAF_BODY_LIMIT` and one byte more, which is held until the app is sent - it. A WebSocket, or any other upgraded connection, passes through, and the - timeouts do not cut it. -- Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is - the client, and the forwarded headers it sends are replaced, not passed on. - For a peer inside it, `X-Forwarded-For` is read from the right, and the first - address outside `SWWAF_TRUSTED_PROXIES` is the client; if every address in it - is inside, the leftmost is, and with no header the peer is. The app sees what - it would see from traefik directly: the same `Host`, the same - `X-Forwarded-Proto`, and `X-Forwarded-For` with the peer added at the end. It - also gets the request's id in `X-Request-ID`, the same id as in the request's - log line (see `request_id` in "Request log" below). -- Enforces the timeouts and the size limits below. Once a request's line and - headers have been read, a limit passed before the response has started gets - `smallwebwaf`'s own answer: `408` for a client too slow to send its body, - `413` for a request body that is too large, `504` for an app too slow to - answer, and `502` for a response that is too large or an app that cannot be - reached. A request that announces a body over the limit is refused before - anything reaches the app. While a request body is still on its way, a request - timeout that runs out answers `408` if `smallwebwaf` was waiting for the - client to send more, and `504` if it was waiting for the app to take what it - had. Once the response has started, a limit can only cut the connection. -- Counts each client's requests over a minute, an hour and a day. A request that - takes the client over one of the rate limits below is refused with - `SWWAF_BAN_RESPONSE`, `403` by default, before anything reaches the app, and - bans the client. A request whose path starts with one of - `SWWAF_RATE_LIMIT_EXEMPT_PATHS`, as that setting below describes, is neither - counted nor refused by the rate limits; the static lists, bans, the country - lists, the rule files and the Core Rule Set still apply to it. A client is one - IPv4 address, or one IPv6 group, the netblock of `SWWAF_IPV6_GROUP_PREFIX` its - address is in, a /64 by default, since one abuser usually holds a whole /64. - Each window is counted in two fixed buckets, the earlier one weighted by how - much of it the window still covers. At most `SWWAF_MAX_TRACKED_CLIENTS` - clients are kept, 20,000 by default, the least recently seen dropped first, - with their history, and a restart gives no client a fresh allowance (see - "State files" below). -- Counts each client's bytes over a minute, an hour and a day, in the same way: - once a request passed to the app has ended, the body bytes of its answer, of - the request, or of both, as `SWWAF_BYTES_COUNT` says. For a WebSocket, or any - other upgraded connection, what it carried from the app counts with the - answer, and what it carried from the client with the request, once it closes. - Bytes that take the client over one of the byte limits below break that limit, - and ban the client as a broken rate limit does, so that its next request is - refused. The byte limits never cut an answer or an upgraded connection short: - the one whose bytes break a limit has already been passed on, or has closed. - They leave out what the rate limits leave out: a client in `SWWAF_ALLOW_NETS` - or `SWWAF_RATE_LIMIT_EXEMPT_NETS`, and a request for a path - `SWWAF_RATE_LIMIT_EXEMPT_PATHS` exempts. -- Gives the clients of the AS numbers and countries the biased thresholds list, - `SWWAF_ASN_LIMIT_PERCENT`, the file `SWWAF_ASN_LIMIT_PERCENT_URL` names and - `SWWAF_COUNTRY_LIMIT_PERCENT`, the percentage they give of every rate limit - and byte limit, so that the same rules ban them after fewer requests, and, - while `SWWAF_UNKNOWN_LIMIT_PERCENT` is below 100, every client without a - country that percentage. A client to which several apply gets the lowest. For - the byte limits, `SWWAF_ASN_BYTES_PERCENT` gives an AS number it lists a - percentage in place of those `SWWAF_ASN_LIMIT_PERCENT` and the file give it, - and `SWWAF_COUNTRY_BYTES_PERCENT` gives a country it lists one in place of the - one `SWWAF_COUNTRY_LIMIT_PERCENT` gives it. Each client is counted on its own, - against its own lowered limits: no budget is shared by a whole AS number or - country, which one abuser could use up and so lock out everyone else there. - The log line of each request the rate limits count gives its client's - percentages below 100 and the settings that gave them, and so do the notes of - a ban for a lowered limit, and its alert. -- Bans a client that breaks a rate limit, a byte limit or the error burst, as - "Bans" in [`SPEC.md`](SPEC.md) describes: the first ban lasts an hour, and a - limit broken again within a day of a ban ending bans for three times as long - as that ban, so 1, 3, 9, 27 and 81 hours; a ban that would last longer than - seven days is permanent instead. A ban covers the client's netblock: its IPv4 - address, or the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or - its IPv6 group. While it lasts, every request from the netblock is refused - with `SWWAF_BAN_RESPONSE` after the static lists and before the country lists, - so the client is not looked up, and is not counted for the rate limits. A ban - sets the client's counters back to zero. Each ban carries notes for deciding - whether to lift it: the limit, whether it is on requests, bytes or refusals, - its window and the requests, bytes or refusals counted in it, the client's - percentage of that kind of limit and the setting that gave it when a biased - threshold lowered the limit, the request that broke it, the client's AS - number, AS name and country once they are looked up, the blocklists, the - CrowdSec decision list, DNSBL zones and AbuseIPDB, with its score, that listed - the client when the ban was made, the netblock's requests since it was first - seen, how many of them the ban has refused, and how many bans the netblock had - before, for a broken limit, for a clear sign of attack, by an admin and for - CrowdSec's decision. At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are - kept, past, active and permanent; past that, the earliest such ban of the - netblock that has gone longest without a request is dropped first. The bans - whose cause is `admin`, those you make or keep, are kept besides, and never - dropped. `bans.json` shows the bans and their notes, a restart lifts none, and - you make, keep or lift a ban by editing it (see "State files" below). -- Checks each request against the rules of the rule files (see "Rule files" - below) after the rate limits, and before its body is read. A `log` rule that - matches is noted in the log line; a `block` rule refuses the request with - `403`, and bans no one for it, though the refusal counts toward the error - burst; a `ban` rule refuses it with `SWWAF_BAN_RESPONSE` and bans the client's - netblock for a clear sign of attack. Matching stops at the first rule that - refuses. A client in `SWWAF_ALLOW_NETS` is not checked. -- Checks each request against the trap paths `SWWAF_TRAP_PATHS` names, after the - rate limits and before the rule files, which it does not need. A request for - one is refused with `SWWAF_BAN_RESPONSE` and bans the client's netblock for a - clear sign of attack, as a `ban` rule's match does. A trap path is matched - against the path a `path` rule sees: without the query, as the client sent it - and once percent-decoded, the whole of it, character for character. - `/wp-login.php` matches `/wp-login.php?redirect_to=x` and `/%77p-login.php`, - but not `/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or - `/%2577p-login.php`. The ban's notes give the path as the client sent it. A - client in `SWWAF_ALLOW_NETS` is not checked. -- Inspects each request with the OWASP Core Rule Set 4.25.0, run by Coraza, - after the rule files, unless `SWWAF_WAF_MODE` is `off`: its method, its URL - with the query, and its headers, and, once `SWWAF_WAF_BODY_LIMIT` is set, its - body when it is form data, multipart, JSON or XML, as that setting below - describes; no response is inspected. Each of its rules that matches adds to - the request's anomaly score, up to the paranoia level - `SWWAF_WAF_PARANOIA_LEVEL` sets. A request with more than 1000 query - parameters, or more than 1000 fields in a form data or JSON body it reads, - adds 5 (rule 900300), as a rule rated critical does, since Coraza reads only - the first 1000. A score at or over `SWWAF_WAF_ANOMALY_THRESHOLD`, 5 by - default, is a match: in `block` mode, the default, the request is refused with - `403`, and in `detect` mode it goes on to the app. Either way its log line - names the rules and the score (see `waf_rule_ids` and `waf_score` in "Request - log" below), and it raises a `waf_block` alert. A refusal bans no one by - itself, since the Core Rule Set takes some ordinary requests for attacks, but - it is an offence the client's history counts, and it counts toward the error - burst; a match in `detect` mode is neither. A request a rule file refuses, one - for a path `SWWAF_WAF_EXEMPT_PATHS` exempts, and one from a client in - `SWWAF_ALLOW_NETS` are not inspected. `smallwebwaf` changes the Core Rule Set - in six ways, so that gitea's ordinary requests get through, and no setting - undoes them: - - `PUT`, `PATCH` and `DELETE` are allowed besides `GET`, `HEAD`, `POST` and - `OPTIONS`; any other method stays refused. - - The headers `Expect` and `Content-Encoding` are allowed; the others the - Core Rule Set refuses, such as `Proxy` and `Content-Range`, stay refused. - Once `SWWAF_WAF_BODY_LIMIT` is set, `Content-Encoding` is refused again - (rule 920450) on form data, multipart, JSON and XML, the kinds of body the - Core Rule Set reads, since a compressed body cannot be inspected; so it is - on a JSON or XML body too large to be read. - - The query parameter `redirect_uri` is not checked for a URL naming an IP - address or `localhost` (rules 931100 and 934110), which Git Credential - Manager, git-credential-oauth and tea ask to be sent back to. - - The query parameters `path`, `files`, `skip-to`, `sub_path`, `ref`, `sha`, - `branch`, `workflow`, `artifactName` and `redirect_to` are not checked - against the lists of system files (930120), shell paths (932160) and - command names (932260), so that a file such as `.gitignore` or a branch - such as `docker-build` gets through there. So does what only these rules - refuse, such as `|cat /etc/passwd`; path traversal and SQL injection are - still refused there. These names, and `redirect_uri` above, are matched - without regard to case, as Coraza matches them, so `Path` or `PATH` is - treated as `path`. Once `SWWAF_WAF_BODY_LIMIT` is set, they are left out - in the same way among the fields of a form data or multipart body, which - Coraza holds with the query parameters, and gitea posts some of them in - its forms: `redirect_uri` when an OAuth sign-in is granted, and `ref` when - a workflow is run by hand. A field of a JSON body is named by its path, - such as `json.path`, and keeps these rules. - - The cookies `gitea_flash` and `redirect_to` are not read, and `Referer` is - not checked for a Unix command given without arguments (932340) or for - Java starting a process (944110); it is checked by every other rule. - - Responses are not inspected. -- Bans a client for a clear sign of attack, as "Bans" in [`SPEC.md`](SPEC.md) - describes: the first such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days by - default, and any request from the netblock while it lasts makes it permanent. - Once it has run out, the netblock is served like any other, but its next clear - sign of attack bans it permanently at once. Such a ban covers the same - netblock as a ban for a broken limit, does not set the client's counters back - to zero, and does not make the netblock's next ban for a broken limit longer. - Its notes give the id and the target of the rule that matched, or the trap - path asked for, in place of the limit. -- Bans a client that `smallwebwaf` refused more than - `SWWAF_ERROR_BURST_THRESHOLD` times within a minute, 30 by default, after a - match of a `block` or `ban` rule, a trap path or the Core Rule Set, or for a - missing or wrong token at one of its own endpoints, as a broken rate limit - bans it. Each such refusal is counted once it has been answered, in two - buckets of a minute, as the rate limits count requests. The refusal that takes - the client over the threshold breaks the error burst; it is answered as any - other such refusal is, and the client's next request is refused under the ban. - The app's own answers, such as its `401` and `404`, are not counted. The - threshold is the same for every client, whatever percentage of the rate limits - a biased threshold or a reputation source gives it. A client in - `SWWAF_ALLOW_NETS` is not counted, and one in `SWWAF_RATE_LIMIT_EXEMPT_NETS` - is. The ban's notes give `refusals` as what the limit is on, the threshold as - the limit, and the refusals counted in the minute. -- Looks up the AS number and country of every client through GeoJS, or in the - IPinfo Lite database file while `SWWAF_LOOKUP_SOURCE` is `file`, after the - static lists and bans, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Country and - AS number lookup" below), for the request log, the client's history, the notes - of its bans, their alerts, the metrics and the anomaly thresholds per AS - number. The file answers at once. With GeoJS, a request waits for its client's - first answer only while a setting acts on it, a country list, - `SWWAF_ADD_LOOKUP_HEADERS` or a biased threshold that lowers a limit. - Otherwise it goes on at once, and the answer reaches the client's history and - the notes of its bans when it comes, but not the log lines of the requests - that went on without it, nor the alerts already raised for those bans. -- Refuses a request from a country you refuse with `SWWAF_BAN_RESPONSE`, as soon - as the client's country is known and before its body is read; such a request - is not counted for the rate limits. A client on a private, loopback or - link-local address has no country and is never looked up: - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless it is in - `SWWAF_ALLOW_NETS`, and `SWWAF_DENIED_COUNTRIES` does not refuse it. -- Checks the client's own address against the blocklists `SWWAF_BLOCKLIST_URLS` - names, after the country lists and before the rate limits (see "Blocklists" - below). With `SWWAF_BLOCKLIST_ACTION` at `deny`, its default, a request from a - client a blocklist lists is refused with `SWWAF_BAN_RESPONSE` before its body - is read; it is not counted for the rate limits, and makes no ban. With - `limit:` the client gets that percentage of every rate limit and byte - limit, the lowest of its percentages applying, as for a biased threshold, and - with `log` nothing more is done. Whatever the action, the request's log line - names the lists, and each raises an alert. -- Checks the client's own address against the decision list of the CrowdSec - engine `SWWAF_CROWDSEC_LAPI_URL` names, after the blocklists and before the - DNSBL zones (see "CrowdSec" below). A request from a client a decision in - force bans is refused with `SWWAF_BAN_RESPONSE` before its body is read, and - bans the client's netblock, as any ban covers it, until the decision ends, - with the cause `crowdsec`; the client's next requests are refused under that - ban. The request's log line names the list, and it raises an alert besides the - ban's. A client a blocklist refuses is not checked. -- Checks the client's own address against the DNSBL zones `SWWAF_DNSBL_ZONES` - names, after the blocklists and before the rate limits (see "DNS blocklists" - below), by the verdicts it keeps. A zone is asked about a client in the - background, and no request waits for its answer: the request that has it - asked, and any other from the client before the answer comes, goes on as from - a client the zone does not list. With `SWWAF_REPUTATION_ACTION` at `limit:25`, - its default, a client a zone's verdict lists gets a quarter of every rate - limit and byte limit, the lowest of its percentages applying, as for a biased - threshold. With `deny`, its requests are refused with `SWWAF_BAN_RESPONSE` - before their bodies are read; they are not counted for the rate limits, and - make no ban. With `log`, nothing more is done. Whatever the action, the - request's log line names the zones, and each raises an alert. A client a - blocklist refuses, or the CrowdSec decision list bans, is not checked. -- Checks the client with AbuseIPDB while `SWWAF_ABUSEIPDB_KEY` is set, after the - DNSBL zones and before the rate limits (see "AbuseIPDB" below), by the scores - it keeps. Only a client whose history counts an offence is checked, so far one - that has broken a rate limit, a byte limit or the error burst, matched a ban - rule, asked for a trap path, or had a request refused by a block rule, by the - Core Rule Set or for a missing or wrong token, and only in the background, so - that no request waits for AbuseIPDB. A score at or over - `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and `SWWAF_REPUTATION_ACTION` does with - its client what it does with one a DNSBL zone's verdict lists. The request's - log line names AbuseIPDB, and it raises an alert. A client a blocklist or a - DNSBL zone refuses, or the CrowdSec decision list bans, is not checked. -- Checks the client's own address against the static lists, the three netblock - settings below, before anything else, its lookup included. A client in - `SWWAF_ALLOW_NETS` skips bans, the country lists, the blocklists, the CrowdSec - decision list, the DNSBL zones, AbuseIPDB, the rate limits, the byte limits, - the trap paths, the rule files, the Core Rule Set and the error burst, and is - not looked up; the timeouts and size limits still apply. A client in - `SWWAF_DENY_NETS` is refused with `SWWAF_BAN_RESPONSE` before its body is - read, and the request is not counted for the rate limits; an address in - `SWWAF_ALLOW_NETS` too is let through. A client in - `SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither counted nor refused by the rate - limits, and has no bytes counted by the byte limits; the country lists, the - trap paths, the rule files, the Core Rule Set, the error burst and bans still - apply to it. -- In `observe` mode, with `SWWAF_MODE=observe`, refuses none of the requests - that `SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec - decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit, a trap - path, a rule or the Core Rule Set would refuse: it passes them to the app, and - their log lines name what `enforce` mode would have done (see `would_action` - in "Request log" below). The checks run, and requests, bytes and refusals are - counted, as in `enforce` mode, with three differences: neither a broken rate - limit, byte limit or error burst, a `ban` rule, a trap path nor the CrowdSec - decision list makes a ban; a broken limit does not set the client's counters - back to zero, so each request over a rate limit is logged as one that would be - refused, each whose bytes keep the client over a byte limit as breaking it, - and each refusal that keeps it over the error burst as breaking that; and a - request under a ban does not make it permanent. As in `enforce` mode, the - bytes counted are only those of the requests `enforce` mode would have passed - to the app, and the refusals counted for the error burst only those it would - have made: a request it would have refused before it reached an endpoint is - not counted for its token. A ban it would have made, or made permanent, raises - the alert `enforce` mode would have raised, marked as what would have happened - (see "Alerts" below). The bans in `bans.json` are kept, and refuse requests - again when `smallwebwaf` next runs in `enforce` mode, as long as they last. - The timeouts and size limits still apply, since they protect `smallwebwaf` and - the app themselves, and a request for one of `smallwebwaf`'s own endpoints - without its token is still answered `401`. It is for trying a configuration - before enforcing it. -- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any - check and without asking the app, for the image's health check. -- Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for - a request that carries `SWWAF_METRICS_TOKEN` as - `Authorization: Bearer `, and with `401` for one that does not. While - the token is unset the metrics answer `404`, as does any request under - `/_smallwebwaf/` that is not for one of its endpoints. Unlike the health - check, such a request goes through every check any other request goes through, - and is answered where another would be passed to the app: a banned client - stays refused, and each counts toward the client's rate limits. None of them - reaches the app. -- Lets an admin list, add and lift bans, and ask what it knows of a client, - through the endpoints `SWWAF_ADMIN_TOKEN` opens, which go through the checks - as the metrics do (see "Admin endpoints" below). -- Writes a line in the request log for each request (see "Request log" below). -- Sends every line it writes on stdout to a syslog server as well, while - `SWWAF_LOG_REMOTE_URL` names one (see "Sending the log to a syslog server" - below). -- Sends an alert for each ban it makes or makes permanent, for a match of the - Core Rule Set, for a count over an anomaly threshold, for a client a - blocklist, the CrowdSec decision list, a DNSBL zone or AbuseIPDB lists, for - GeoJS failing, a list it cannot fetch, the CrowdSec decision list among them, - a DNSBL zone or AbuseIPDB that fails or refuses a query, and the day's - AbuseIPDB checks used up, for a rule file or state file with an error, and for - a replacement of the lookup database it cannot read, holding back repeats and, - past an hourly limit, rolling the rest into one summary, to each destination - you name: as a JSON object to the webhook `SWWAF_ALERT_WEBHOOK_URL` names, as - a message to the Slack incoming webhook `SWWAF_ALERT_SLACK_WEBHOOK_URL` names, - and as a message to the ntfy topic `SWWAF_ALERT_NTFY_URL` names (see "Alerts" - below). -- Counts requests and their bytes over a minute and an hour, per client, per - netblock around a client, per AS number, for the whole service and per named - netblock, and sends an `anomaly` alert for a count over the anomaly threshold - you set for it (see the anomaly thresholds below). These thresholds only - alert: they refuse and ban nothing. A scope whose four thresholds are all off - is not counted, and within a scope only the counts whose threshold is set are - counted. Every request but the health check is counted, whatever is done with - it: one that is refused, one from a client in `SWWAF_ALLOW_NETS` or - `SWWAF_RATE_LIMIT_EXEMPT_NETS`, and one for a path in - `SWWAF_RATE_LIMIT_EXEMPT_PATHS`, with its body bytes once it has ended, those - of the answer, of the request or both, as `SWWAF_BYTES_COUNT` says. A request - is counted for its client's AS number only if the lookup has given that by the - time the request ends: no request waits for it, and a client in - `SWWAF_ALLOW_NETS`, which is not looked up, counts for no AS number. At most - 20,000 counters are kept, the one counted least recently dropped first, and - `alerts.json` keeps them across a restart (see "State files" below). - -## Settings - -Each setting is an environment variable, or a file one names (see "Settings -given as files" below), and each has a default, so none has to be set. A setting -that is set but invalid stops the start with a message naming it, and the -effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or -`error`, which hold that line back. - -- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens. -- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app, as `http` or - `https`, a host and an optional port, and nothing more. -- `SWWAF_INSTANCE_NAME` (default: the host's name, which docker sets to the - first 12 characters of the container's id unless the deployment names one): - the name every log line and alert gives as `instance`, and every metric - carries as its label `instance` (see "Metrics" below). Set it, for example to - `fsn1app1/gitea`, for a name that stays the same when a deploy replaces the - container, and that tells instances apart when several log to one place. A - name that is not valid UTF-8, such as one saved in Latin-1, stops the start. -- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to pass on the - requests `smallwebwaf` would refuse and log what it would have done (see "What - it does so far" above). -- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`, - the private address ranges): the netblocks whose `X-Forwarded-For` is - believed. A list given replaces the default; set but empty, it trusts nothing. -- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the length of an IPv6 client's - group, the netblock that is one client, from 32 to 128, since a shorter one - would make one client of the customers of several providers. The rate limits, - the byte limits, bans, the table of clients, the lookups, AbuseIPDB's scores - and the anomaly thresholds per client all take an IPv6 client as its group, - and `client_group` gives it. After it changes, each IPv6 client starts afresh: - what was kept of it under its earlier group, its counts, history, GeoJS answer - and AbuseIPDB score, is not used for it, while each ban keeps refusing its - netblock until it ends. -- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept in memory - and in `clients.json`, with their counters and history, a whole number above - zero. Past it, the least recently seen is dropped first. -- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to - send its request line and headers, and then, from the end of the headers, its - body. -- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request - line and headers a client may send. Over it, the answer is `431` and nothing - reaches the app. It must be more than `4K`, and cannot be `off`: Go's HTTP - server always has such a limit, and reads 4 KiB past the one it is given - before it refuses. -- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection - may wait for its next request before `smallwebwaf` closes it. The default is - longer than the 90 seconds after which traefik closes a connection it is not - using, so traefik never sends a request on a connection `smallwebwaf` is - closing. -- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the response may - take to reach the client, from the end of the request to the last byte. -- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the - app and sending it the whole request may take. -- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take - to send its whole answer, from the end of the request to the last byte. -- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. -- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. -- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip bans, the - country lists, the blocklists, the CrowdSec decision list, the DNSBL zones, - AbuseIPDB, the rate limits, the byte limits, the trap paths, the rule files - and the error burst, such as your monitoring or your own networks. -- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients the - rate limits and the byte limits do not apply to, such as a machine that talks - to the app all day. -- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused. -- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR` - (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most - requests a client may make in a minute, an hour and a day. The defaults are - several times what one busy person produces, since a browser loading a heavy - page makes a few hundred requests and several people often share one address. -- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes whose requests - the rate limits neither count nor refuse, and whose bytes the byte limits do - not count, such as `/assets/` for static assets; each starts with `/`. A - request whose path, percent-decoded, contains `..` anywhere or a backslash, or - whose path as sent holds an encoded slash (`%2F` or `%2f`), is never exempt, - since the app may act on it as a path outside every prefix: - `/assets/..%2Flogin` as `/login`. Any other request is exempt when its path as - sent, the path the app receives, before any query string and not - percent-decoded, starts with a prefix, character for character. `/assets/` - matches `/assets/app.js` and `/assets/`, but not `/assets`, `/Assets/app.js`, - `/%61ssets/app.js`, `/static/assets/app.js`, `/static/../assets/app.js` or - `/assets%2Fapp.js`. A character the client sends percent-encoded, such as a - space, is written percent-encoded in a prefix, as in `/my%20files/`, and there - are no wildcards: `*` is a character like any other. -- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR` - (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most - bytes a client may have counted in a minute, an hour and a day. A request's - bytes are counted once its answer has ended, so each default is above the - largest request body and the largest response together, - `SWWAF_REQUEST_MAX_BYTES` and `SWWAF_RESPONSE_MAX_BYTES`: at the defaults no - download breaks a limit on its own. -- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count: - `response` for those of the answers, `request` for those of the requests, or - `both`. -- `SWWAF_LOOKUP_SOURCE` (default `geojs`): where each client's AS number and - country are looked up: `geojs`, the GeoJS web service, which is then told the - address of every new visitor, `file`, the IPinfo Lite database file - `SWWAF_LOOKUP_DB_PATH` names, or `off`, which looks up no client and sends no - address to GeoJS. With `off`, a country list that is not empty, - `SWWAF_ADD_LOOKUP_HEADERS` set to `true`, a biased threshold that lowers a - limit, a list of them that is not empty, `SWWAF_ASN_LIMIT_PERCENT_URL` set or - `SWWAF_UNKNOWN_LIMIT_PERCENT` below 100, or an anomaly threshold per AS number - that is not `off`, stops the start, with a message naming it and - `SWWAF_LOOKUP_SOURCE`. -- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its - `.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`. `file` without it, or it with - any other `SWWAF_LOOKUP_SOURCE`, the default included, stops the start, with a - message naming both. -- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its - client's first answer from GeoJS while a setting acts on it, and how long a - request to GeoJS may take before it is abandoned. -- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the - client's AS number, such as `AS64496`, in `X-Client-ASN`, and its country in - `X-Client-Country`, leaving out one that is unknown. A request then waits for - its client's first answer, as it does while a country list is set. Whatever - this setting says, any `X-Client-ASN` or `X-Client-Country` the client sent, - in any case, is removed, so that the app never receives a client's own. -- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused, - for example `cn,ru,kp`. -- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only - countries whose clients get through, for example `us,de`. A client whose - country cannot be found is refused too, so that new clients are not let in - whenever GeoJS stops answering. -- `SWWAF_ASN_LIMIT_PERCENT` (default empty): AS numbers, each with the - percentage of every rate limit and byte limit its clients get, such as - `AS14061:50,AS16276:50,AS45102:25`. A lowered limit is rounded down to a whole - number: half of 1000 requests a minute is 500, and half of 5 is 2. `0` is a - zero allowance: the client's first request breaks a limit, and bans it. -- `SWWAF_COUNTRY_LIMIT_PERCENT` (default empty): the same by country, such as - `cn:25,ru:50`. -- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty): - the same for the byte limits alone. For an AS number or a country one of them - lists, its percentage takes the place, for the byte limits, of the one - `SWWAF_ASN_LIMIT_PERCENT` or `SWWAF_COUNTRY_LIMIT_PERCENT` gives, so that - `SWWAF_ASN_LIMIT_PERCENT=AS14061:50` with - `SWWAF_ASN_BYTES_PERCENT=AS14061:100` halves that AS number's rate limits and - leaves its byte limits whole. -- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage of every limit a - client without a country gets: one the lookup cannot place, one on a private, - loopback or link-local address, which is never looked up, and one whose answer - from GeoJS has not come in time. -- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a - file of AS numbers, each with its percentage of every rate limit and byte - limit, one such as `AS14061:50` to a line, whose percentages count as those of - `SWWAF_ASN_LIMIT_PERCENT` do, so that one list can serve several instances. - For an AS number both give a percentage, the lower applies, and an AS number - the file lists twice gets the lower of its two. It is fetched and kept as a - blocklist is (see "Blocklists" below). A URL `SWWAF_BLOCKLIST_URLS` names too - stops the start. -- `SWWAF_BLOCKLIST_URLS` (default empty): the blocklists, as `http` or `https` - URLs without a user or a fragment, such as - `https://www.spamhaus.org/drop/drop.txt` (see "Blocklists" below). A URL - listed twice stops the start. -- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long after a list was last - fetched, or a fetch of it failed, it is fetched again, for the blocklists and - `SWWAF_ASN_LIMIT_PERCENT_URL`. Less than `1h` stops the start: the Spamhaus - lists may be fetched no more than once an hour. -- `SWWAF_BLOCKLIST_ACTION` (default `deny`): what is done with a client a - blocklist lists: `deny` refuses its requests with `SWWAF_BAN_RESPONSE`, - `limit:`, such as `limit:25`, gives it that percentage of every rate - limit and byte limit, and `log` does nothing more than note the lists in the - log line and raise the alert. -- `SWWAF_DNSBL_ZONES` (default empty): the DNSBL zones each client is asked - about, such as `dnsbl.dronebl.org` (see "DNS blocklists" below). A zone is a - DNS name of at most 189 characters: labels of letters, digits and hyphens, of - up to 63 characters each, neither starting nor ending with a hyphen, joined by - dots. A Spamhaus zone is the name of its keyed query service, with the key in - it, such as `.xbl.dq.spamhaus.net`, and is shown with its key masked (see - "DNS blocklists" below). A zone that is not such a name stops the start, as - does one listed twice, even with its letters in another case or with another - key. Do not name a zone meant for mail (see "DNS blocklists" below). -- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver the zones are asked - through, an IP address with an optional port, 53 when none is given, such as - `192.0.2.53` or `[2001:db8::53]:5353`. Unset, it is the host's, as - `/etc/resolv.conf` names it. Several zones refuse queries that come through a - public resolver. -- `SWWAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account, - which clients are checked with (see "AbuseIPDB" below). While it is unset, no - client is checked. The settings logged at start show `********` in its place. -- `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score, - from 0 to 100, that is a hit. -- `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day, - in UTC, a whole number above zero. AbuseIPDB's free accounts may make 1,000. -- `SWWAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine - you run, whose decision list is fetched (see "CrowdSec" below), as an `http` - or `https` URL without a user or a fragment, such as `http://172.17.0.1:8080`. - While it is unset, no list is fetched. Set without `SWWAF_CROWDSEC_LAPI_KEY`, - it stops the start, and so does a URL whose decision list, the URL with - `v1/decisions` added to its path, `SWWAF_BLOCKLIST_URLS` or - `SWWAF_ASN_LIMIT_PERCENT_URL` names too. -- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for - `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without - `SWWAF_CROWDSEC_LAPI_URL`, it stops the start. The settings logged at start - show `********` in its place. -- `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client a - zone's verdict lists, or whose AbuseIPDB score is a hit, as - `SWWAF_BLOCKLIST_ACTION` is for a blocklist: `deny`, `limit:` or - `log`. Such a verdict is less certain than a list such as DROP, so by default - its client gets a quarter of every rate limit and byte limit. -- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict on a - client, or AbuseIPDB's score of it, is used after it was given. -- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone, or a - check with AbuseIPDB, may take before it fails. -- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered, one - that is banned, breaks a rate limit, matches a `ban` rule, asks for a trap - path, is in `SWWAF_DENY_NETS`, comes from a refused country, is in a blocklist - while `SWWAF_BLOCKLIST_ACTION` is `deny` or is listed by a DNSBL zone or - AbuseIPDB while `SWWAF_REPUTATION_ACTION` is `deny`: `403`, `429`, or `close` - to close the connection without an answer. Behind traefik, `close` does not - leave the client unanswered: traefik answers `502`, as it does whenever its - backend drops a connection. A `block` rule always answers `403`. -- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate - limit, byte limit or error burst. -- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a rate limit, byte limit or - error burst broken again within this time after a ban ended, other than one - for a clear sign of attack or for CrowdSec's decision, bans for three times as - long as that ban. -- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken rate limit, byte - limit or error burst that would be longer is permanent instead. -- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of - attack. -- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are - kept, past, active and permanent. The bans you make or keep are kept besides. -- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around - an IPv4 client that a ban covers, such as `24` to ban the surrounding /24. An - IPv6 ban covers the client's group, as `SWWAF_IPV6_GROUP_PREFIX` sets it. -- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the directory of the state - files, an absolute path. A directory `smallwebwaf` cannot write stops the - start. -- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long after a ban is made - `bans.json` is written, with every ban made in between. -- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is - written. -- `SWWAF_LOG_REQUEST_HEADERS` (default - `accept,accept-language,accept-encoding,content-type,origin,range`): the - request headers whose values the request log gives, in either case. - `Authorization`, `Cookie` and `Set-Cookie` are never logged, even when listed - (see "Request log" below). An entry naming `Host` or `Transfer-Encoding` stops - the start, since Go's HTTP server takes both out of the request; the request's - host is the field `host`. -- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own - messages that are written, on stdout and to the syslog server: `debug`, - `info`, `warn` or `error`. No message is at `debug` yet, so it writes what - `info` does. It holds back no line of the request log. -- `SWWAF_ADMIN_TOKEN` (default unset): the token an admin sends for the ban - endpoints and `/_smallwebwaf/clients/` (see "Admin endpoints" below), a - long random value. While it is unset they are off; one shorter than 32 - characters stops the start. The settings logged at start show `********` in - its place. Given as a file, with `SWWAF_ADMIN_TOKEN_FILE`, it can be kept out - of the app's reach (see "Settings given as files" below). -- `SWWAF_METRICS_TOKEN` (default unset): the token a scraper sends for the - metrics, a long random value. While it is unset the metrics are off; one - shorter than 32 characters stops the start. The settings logged at start show - `********` in its place. Given as a file, it can be kept out of the app's - reach (see "Settings given as files" below). -- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many - countries get series of their own in the metrics by AS number and by country; - the others are counted as `other`. -- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the - rule files. A directory that does not exist stops the start. -- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file, and checks - no request against one. -- `SWWAF_WAF_MODE` (default `block`): what the Core Rule Set does with a match: - `block` refuses it with `403`, `detect` lets it through, logged and alerted, - and `off` inspects no request (see "What it does so far" above). -- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level, - from 1 to 4. Each level up runs more of its rules, which find more attacks and - take more ordinary requests for them. -- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the anomaly score at which a - request is a match. A rule the Core Rule Set rates critical adds 5, so by - default one such rule is enough. `off` makes no request a match; the scores - are still logged. -- `SWWAF_WAF_DISABLED_RULES` (default - `920340,920420,920440,920640,930130,930140`): the ids of the Core Rule Set's - rules to switch off, such as one that refuses ordinary requests of your app; - the request log names the rules a request matched in `waf_rule_ids`. The - default switches off the rules that refuse a request for the type of its body, - when it is missing or not on the Core Rule Set's short list (920340, 920420, - 920640), for its file extension, such as `.sh` or `.sql` (920440), and for a - file or directory name in its path, such as `.git/`, `.gitignore`, - `Dockerfile`, `package.json` or an editor's settings directory (930130, - 930140). In front of a code forge these refuse git over HTTP, container image - and package uploads, and views of ordinary files in a repository; the default - rule file bans the common probes for such files at the site root instead (see - "Rule files" below). A list given replaces the default, so include them in it; - set but empty, it switches no rule off. An item that is not a whole number - above zero stops the start, as does one from 900000 to 900999, the ids of the - rules that set the Core Rule Set up and of `smallwebwaf`'s own, so that no - setting undoes its changes. The id of no rule switches nothing off. -- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes whose requests the - Core Rule Set does not inspect, each starting with `/`, matched as - `SWWAF_RATE_LIMIT_EXEMPT_PATHS` matches its own: a request whose path holds - `..`, a backslash or an encoded slash is inspected whatever its prefix. -- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` has the Core Rule Set read no - request body. A size, such as `128K`, at most `1G`, has it read a body of form - data or multipart up to that size, the rest of a longer one passing on to the - app as it arrives, without being held, and a JSON or XML body no larger than - that size, since those cannot be read in part. A body is JSON when its type is - `application/json` or `text/json`, or an `application/` or `text/` type ending - in `+json`, and XML when its type is `application/xml` or `text/xml`, or an - `application/` or `text/` type ending in `+xml`. Any other body reaches the - app uninspected, and so does a larger JSON or XML body: the Core Rule Set - would read any other body as form data, where binary content such as a git - push trips rules written for text. A body it reads that Coraza cannot parse - (rule 900440), and a multipart body that fails Coraza's strict checks (rule - 900450), add 5 to the score, as a rule rated critical does, since no rule - reads what comes after the fault. So does a multipart body the limit cuts - before the colon of a part's header line, or between the carriage return and - the line feed that end a part's header line or the empty line after its - headers, since Coraza takes the line the limit cuts for a malformed header. - The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to send the part - that is read, and a request whose body passes `SWWAF_REQUEST_MAX_BYTES` within - it is refused before anything reaches the app. Of a file in a multipart body, - Coraza counts the bytes and writes nothing. Body inspection suits apps whose - forms carry no code. In front of gitea it refuses issue and comment text, wiki - pages and files saved in the web editor that hold shell commands or code - (932125, 932235, 932250 and others), package descriptions that show code, PyPI - uploads (922130), and attachments named like `debug.log` or `config.yml` - (932180), until the rule ids the request log names are added to - `SWWAF_WAF_DISABLED_RULES`. -- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only - scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of gitea; a - request for one bans its client for a clear sign of attack, as a `ban` rule - does, with or without rule files (see "What it does so far" above). A path - that does not start with `/`, or holds a `?`, as one written with a query - does, stops the start. -- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most requests of a client in - a minute that `smallwebwaf` may refuse after a rule file match, a trap path or - a Core Rule Set match, or for a missing or wrong token; one more breaks the - error burst, and bans the client as a broken rate limit does (see "What it - does so far" above). A client trying one probe or token after another is - refused many times a minute, a person rarely more than a few times. `off` - switches it off. -- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server that every line on - stdout is also sent to, as `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` - with a host and a port, such as `syslog+tls://logs.example:6514`. Unset or - empty, nothing is sent. -- `SWWAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates, - which the certificate of a `syslog+tls` server must chain to instead of the - host's own. A file that cannot be read or holds no certificate stops the - start. -- `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they - wait to be sent. -- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility the lines - are sent with: `kern`, `user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, - `news`, `uucp`, `cron`, `authpriv`, `ftp`, or `local0` to `local7`. -- `SWWAF_LOG_REMOTE_APP_NAME` (default `SWWAF_INSTANCE_NAME`): the app name the - lines are sent with, 1 to 48 printable ASCII characters without a space. While - `SWWAF_LOG_REMOTE_URL` is set, an `SWWAF_INSTANCE_NAME` that is not such a - name stops the start too, unless this setting gives one that is. -- `SWWAF_ALERT_WEBHOOK_URL` (default unset): the webhook each alert is posted - to, an `http` or `https` URL without a user or a fragment, such as - `https://alerts.example/smallwebwaf` (see "Alerts" below). Unset or empty, no - alert is sent to a webhook. Since many webhooks carry their secret in the path - or the query, the settings logged at start show `********` in place of them, - and a value that stops the start is not shown. -- `SWWAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each alert, - such as one that authenticates it, as a list of a name, `:` and a value, such - as `Authorization:Bearer 0123456789abcdef`. A value cannot hold a comma. The - settings logged at start show `********` in place of each value. -- `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook - each alert is posted to as a message, such as - `https://hooks.slack.com/services/T0123/B4567/abcdef`. Unset or empty, no - alert is sent to Slack. It is checked and logged as `SWWAF_ALERT_WEBHOOK_URL` - is. -- `SWWAF_ALERT_NTFY_URL` (default unset): the ntfy topic each alert is published - to, as the topic's full URL, such as `https://ntfy.sh/my-alerts`. Unset or - empty, no alert is sent to ntfy. It is checked and logged as - `SWWAF_ALERT_WEBHOOK_URL` is, since anyone who knows a topic on a server open - to all can read it. -- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token, sent to ntfy - with each alert as `Authorization: Bearer `, for a topic that needs - one. The settings logged at start show `********` in its place. A control - character in it, such as the carriage return of a file saved with Windows line - ends, stops the start, and while `SWWAF_ALERT_NTFY_URL` is set, so does one in - `SWWAF_INSTANCE_NAME`, which ntfy is sent in the title. -- `SWWAF_ALERT_EVENTS` (default - `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): - the events alerts are sent for. -- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held - back (see "Alerts" below). -- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour; - the rest of the hour's alerts are rolled into one summary. -- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`, - `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`, - `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` and - `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR` (default `off`): the anomaly thresholds - per client, the most requests and the most bytes a client may have counted in - a minute and in an hour before an `anomaly` alert is sent for it. They refuse - and ban nothing. Each scope below has the same four thresholds, their names - ending in `REQUESTS_PER_MINUTE`, `REQUESTS_PER_HOUR`, `BYTES_PER_MINUTE` and - `BYTES_PER_HOUR`, and each is `off` by default, since what is unusual depends - on each service's normal traffic, which the metrics show. -- `SWWAF_ANOMALY_NET_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_NET_BYTES_PER_MINUTE` and `..._PER_HOUR` (default `off`): the - anomaly thresholds per netblock around a client, which is - `SWWAF_ANOMALY_NET_V4_PREFIX` (default `24`) long, from 0 to 32, for an IPv4 - client, and `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`) long, from 0 to 128, - for an IPv6 one. -- `SWWAF_ANOMALY_ASN_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_ASN_BYTES_PER_MINUTE` and `..._PER_HOUR` (default `off`): the - anomaly thresholds per AS number. -- `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` and `..._PER_HOUR` (default `off`): the - anomaly thresholds for the whole service. -- `SWWAF_WATCH_NETS` (default empty): named netblocks, each a name, `=` and a - netblock, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item - without `=`, without a name or without a valid netblock, or a name listed - twice, stops the start. `SWWAF_WATCH_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - `SWWAF_WATCH_BYTES_PER_MINUTE` and `..._PER_HOUR` (default `off`) are the - anomaly thresholds of each named netblock as a whole, which counts every - client inside it; a client inside several is counted in each. - -Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are -bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024 -bytes). Rate limits, `SWWAF_ERROR_BURST_THRESHOLD` and the anomaly thresholds on -requests are whole numbers of requests, and byte limits and the anomaly -thresholds on bytes are sizes. Netblocks are in CIDR form, and a bare address -stands for itself alone. Countries are the two-letter codes ISO 3166-1 assigns -today, and `xk` for Kosovo, in either case (`de` and `DE` are the same); any -other code, such as `nk` (North Korea is `kp`) or the withdrawn `su`, stops the -start, and so does a code on both country lists. AS numbers are `AS` and the -number, in either case. Percentages are whole numbers from 0 to 100, and an -entry of a list of them is an AS number or a country, `:` and a percentage; an -AS number or a country listed twice in one of them stops the start. `off` -switches a timeout, a size limit, a rate limit, a byte limit, an anomaly -threshold, `SWWAF_WAF_ANOMALY_THRESHOLD`, `SWWAF_ERROR_BURST_THRESHOLD`, -`SWWAF_ALERT_COOLDOWN` or `SWWAF_ALERT_MAX_PER_HOUR` off; -`SWWAF_IPV6_GROUP_PREFIX`, `SWWAF_WAF_PARANOIA_LEVEL`, -`SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, -`SWWAF_LOOKUP_TIMEOUT`, `SWWAF_UNKNOWN_LIMIT_PERCENT`, -`SWWAF_BLOCKLIST_REFRESH`, `SWWAF_ABUSEIPDB_MIN_SCORE`, -`SWWAF_ABUSEIPDB_DAILY_BUDGET`, `SWWAF_REPUTATION_CACHE_TTL`, -`SWWAF_REPUTATION_TIMEOUT`, the ban settings, the state settings, -`SWWAF_METRICS_TOP_N`, `SWWAF_LOG_REMOTE_BUFFER`, `SWWAF_ANOMALY_NET_V4_PREFIX` -and `SWWAF_ANOMALY_NET_V6_PREFIX` cannot be off. - -Several limits are fixed rather than settings. At most 100,000 answers from -GeoJS are kept, for 7 days each, at most 20,000 anomaly counters, at most -100,000 verdicts of the DNSBL zones, with at most 1,000 queries to them under -way at once, and at most 100,000 scores of AbuseIPDB, and the CrowdSec decision -list is fetched again a minute after it was last fetched or tried. - -### Settings given as files - -Any setting may instead be given as a file that holds its value: the variable -named like the setting with `_FILE` added, such as `SWWAF_METRICS_TOKEN_FILE`, -names the file. `smallwebwaf` reads the file once, at start, and its health -check reads only the files of `SWWAF_LISTEN_ADDR` and `SWWAF_UPSTREAM_URL`, each -time it runs. The file's contents are the value, less one newline at their end -so that a file written with `echo` or an editor works, and are checked as the -setting's own value would be. Setting both the setting and its `_FILE` form, or -naming a file that cannot be read, stops the start with a message naming the -variable. The settings logged at start name the file, and show a token given in -one as `********`, as they show one given directly. -`SWWAF_LOG_REMOTE_TLS_CA_FILE`, whose value names a file already, has no `_FILE` -form. - -The app starts with the same environment variables as `smallwebwaf`, so it can -read a token given as one. A token given as a file is out of the app's reach -only while the `smallwebwaf` user alone can read the file: make it on the host, -owned by uid 65532, the `smallwebwaf` user, with mode `0400`, and mount the -directory that holds it into the container read-only; the container sees the -same owner and mode. For example, on the host: - -```sh -mkdir -p /srv/app/tokens -openssl rand -hex 32 > /srv/app/tokens/metrics -chown 65532:65532 /srv/app/tokens/metrics -chmod 0400 /srv/app/tokens/metrics -``` - -and for the container, `-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro` and -`-e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics`. - -## Request log - -`smallwebwaf` writes one JSON object per line on stdout for every request, -refused ones included: - -``` -{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104} -``` - -A field that does not apply to a request is left out of its line, apart from -`type`, the fields from `time` to `user_agent`, `request_id`, `peer_ip`, -`client_group`, `asn`, `as_name`, `country`, `action` and `duration_total`, -which every line has. - -- `time` is when the request arrived, in UTC. `instance` is - `SWWAF_INSTANCE_NAME`. `scheme` is the `X-Forwarded-Proto` a trusted proxy - sent, and otherwise `http`. `path` and `query` are as the client sent them. -- `request_id` is the `X-Request-ID` a trusted proxy sent, or a new random one - of 26 letters and digits when it sent none, or when the peer is not a trusted - proxy. A request passed to the app takes it there in `X-Request-ID`. -- `peer_ip` is the TCP peer, normally traefik. `forwarded_for` is the - `X-Forwarded-For` header as received, several lines of it joined with `, `. - `client_group` is the client as the rate limits count it: its IPv4 address as - a /32, or its IPv6 group, as `SWWAF_IPV6_GROUP_PREFIX` sets it. -- `asn`, `as_name` and `country` are the client's AS number, such as `AS64496`, - the name of that AS, and its country, as GeoJS or the lookup database gives - them. Each is empty when `SWWAF_LOOKUP_SOURCE` is `off`, for a client in - `SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`, for a client on a private, loopback - or link-local address, when GeoJS has not answered by the time the request - went on, for a client whose address the lookup database does not hold, and for - a request whose client a ban covers, even when the answer is known. `asn` and - `as_name` are empty too when GeoJS knows no AS number for the client, which it - gives as 64512, and `country` when GeoJS cannot place the client. -- `content_type` is the request's `Content-Type`, and `content_length` the - length the request announced for its body, which is left out for none or zero. -- `request_headers` are the request's headers that `SWWAF_LOG_REQUEST_HEADERS` - names, by name in lower case, several lines of one joined with `, `. - `Authorization`, `Cookie` and `Set-Cookie` are never among them, whatever the - setting says: `has_authorization` and `has_cookie` are there instead, and - true, when the request has an `Authorization` or a `Cookie` header. -- `websocket` is there, and true, when the app switched the connection to - another protocol, as it does for a WebSocket. -- `status` is what the client was sent, `0` if nothing was; `upstream_status` is - what the app answered, and is left out when the app did not answer. -- `response_content_type`, `cache_control` and `location` are the - `Content-Type`, `Cache-Control` and `Location` headers of the answer: the - app's, as passed on, or those of `smallwebwaf`'s own answer. -- `request_bytes` and `response_bytes` count body bytes. -- `action` is `forward` for a request passed to the app, `denied` for one - refused because its client is in `SWWAF_DENY_NETS`, in a blocklist while - `SWWAF_BLOCKLIST_ACTION` is `deny`, or listed by a DNSBL zone or AbuseIPDB - while `SWWAF_REPUTATION_ACTION` is `deny`, `banned` for one refused because a - ban covers its client, or because it matched a `ban` rule, asked for a trap - path or the CrowdSec decision list lists its client, each of which bans its - client, `country_denied` for one refused for its client's country, - `rate_limited` for one that broke a rate limit and banned its client, - `rule_blocked` for one a `block` rule refused, `waf_blocked` for one the Core - Rule Set refused, `too_large` for a request or response over its size limit, - `timed_out` for one that ran out of time, `upstream_error` when the app could - not be reached or its answer broke off, and `admin` for one `smallwebwaf` - answered at its own endpoint. -- `would_action` is there in `observe` mode for a request that - `SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec - decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit, a trap - path, a rule or the Core Rule Set would have refused in `enforce` mode, and - names the action that refusal would have had: `denied`, `banned`, - `country_denied`, `rate_limited`, `rule_blocked` or `waf_blocked`. `action` - then names what was done: `forward` for a request passed to the app, and - another action, such as `too_large`, for one a size or time limit refused. -- `limit_percent` is there for a request the rate limits count whose client a - biased threshold, `SWWAF_BLOCKLIST_ACTION` for a blocklist that lists it, or - `SWWAF_REPUTATION_ACTION` for a DNSBL zone whose verdict lists it or an - AbuseIPDB score that is a hit, gives less than the whole of the rate limits, - and gives the percentage it gets, with `limit_percent_setting` naming the - setting that gave it, such as `SWWAF_ASN_LIMIT_PERCENT`, or - `SWWAF_ASN_LIMIT_PERCENT_URL` for the file it names. `bytes_percent` and - `bytes_percent_setting` are the same for the byte limits. Each is left out - when the client gets the whole of those limits. -- `counts` gives the client's requests in the minute, the hour and the day as - the rate limits count them, this request included: in each window, those in - the bucket under way and a share of those in the bucket before, so a count can - have a fraction. For a request that broke a limit, they are the counts that - broke it. It is left out for a request the rate limits do not count: the - health check, one from a client in `SWWAF_ALLOW_NETS` or - `SWWAF_RATE_LIMIT_EXEMPT_NETS`, one for a path that - `SWWAF_RATE_LIMIT_EXEMPT_PATHS` exempts, and one that `SWWAF_DENY_NETS`, a - ban, the country lists, a blocklist, the CrowdSec decision list, a DNSBL - zone's verdict or AbuseIPDB's score refuse, or would refuse in `observe` mode. - Its `minute_bytes`, `hour_bytes` and `day_bytes` give the client's bytes in - each window as the byte limits count them, in the same way: for a request - whose bytes they count, with its own, once it has ended; for any other, those - counted before it. -- `rule_ids` is there for a request that matched rules of the rule files, and - lists their ids in the order they matched, up to the one that refused it. -- `waf_score` is there for a request the Core Rule Set inspected, and gives its - anomaly score, `0` for one no rule matched; `waf_rule_ids` lists the ids of - the rules that matched, in the order they ran, and is left out when none did. - A request is a match when its score is at or over - `SWWAF_WAF_ANOMALY_THRESHOLD`: in `block` mode its `action` is `waf_blocked`, - and in `detect` mode what it would have been otherwise, such as `forward`. -- `limit_hit` is there for a request that broke a rate limit or the error burst, - or whose bytes broke a byte limit, and names the window whose limit it went - over as `counts` names it: `minute`, `hour` or `day` for a rate limit, and - `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit, the shortest if - it went over several; or `error_burst` for the error burst. `offence` is then - `limit`. A request whose bytes broke a byte limit is not refused: its `action` - is what it would have been otherwise, such as `forward`. Nor is one that broke - the error burst refused for that: its `action` is that of the refusal counted, - such as `rule_blocked` or `admin`. -- `reputation` is there for a request whose client a blocklist, the CrowdSec - decision list or a DNSBL zone's verdict lists, or whose AbuseIPDB score is a - hit, and gives the URLs of the blocklists that list it, in the order - `SWWAF_BLOCKLIST_URLS` names them, then the URL of the CrowdSec decision list - when a decision in force bans the client, then the zones whose verdict lists - it, in the order `SWWAF_DNSBL_ZONES` names them, then `abuseipdb`, whatever - `SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say. A zone whose - verdict on the client has not come yet, or was given - `SWWAF_REPUTATION_CACHE_TTL` ago or more, is not named, nor is AbuseIPDB for - such a score. It is left out for a client the blocklists are not checked for: - one in `SWWAF_ALLOW_NETS`, and one `SWWAF_DENY_NETS`, a ban or the country - lists refuse first, or would in `observe` mode. The CrowdSec decision list is - not checked either for a client a blocklist refuses, nor are the zones for one - a blocklist refuses or the decision list bans, nor AbuseIPDB's score for one a - blocklist or a zone refuses or the decision list bans, or would in `observe` - mode. -- `ban_expires` is there for a request that made a ban or was refused under one, - or in `observe` mode would have been refused under one, and gives when the ban - ends, in the same form as `time`, or `permanent`. -- `aborted` is there, and true, when the client went away early. -- The timings are in milliseconds, to the microsecond. `duration_total` runs - from when the request's headers had been read to when its line is written, and - `duration_checks` over the same start to when the checks were done; the health - check runs none, and its line has no `duration_checks`. `duration_waf`, the - part of the checks the Core Rule Set took, reading the part of the body it - reads included, is there with `waf_score`. `duration_upstream_connect`, - `duration_upstream_first_byte` and `duration_upstream_total` are there for a - request passed to the app, and run from when it was handed to the app: until - there was a connection to it, new or kept open from an earlier request, until - the first byte of its answer arrived, and until the end. The first two are - left out when that never happened, as for an app that cannot be reached. - -No body is logged, and no header but those above. `smallwebwaf`'s own messages -(start, the settings, stop, errors) share the stream as JSON lines marked -`"type":"process"`, each with `instance` as a request's line has it, and those -less severe than `SWWAF_LOG_LEVEL` are not written. - -Go's HTTP server, on which `smallwebwaf` is built, reads a request's line and -headers before `smallwebwaf` sees the request, and some requests end there, -without a line in the log: headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, -which it answers `431`, and requests it cannot read at all, which it answers -itself, mostly with `400`. A client too slow to send its request line and -headers within `SWWAF_CLIENT_REQUEST_TIMEOUT` gets either no answer or an error -answer from the server, mostly `400 Bad Request`; either way the server closes -its connection, nothing reaches the app, and no line is written in the log. - -### Sending the log to a syslog server - -While `SWWAF_LOG_REMOTE_URL` is set, every line `smallwebwaf` writes on stdout, -request lines and its own, is also sent to that syslog server, as the message of -an RFC 5424 record: one record to a datagram over UDP, and over TCP and TLS each -record after its length in bytes and a space. A record gives the facility -`SWWAF_LOG_REMOTE_FACILITY` names, the severity informational, the time the line -was written, in the same form as a request line's `time`, the host's name, and -the app name `SWWAF_LOG_REMOTE_APP_NAME` gives. stdout is unchanged. - -The lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines and are sent from -there, so a server that is slow or cannot be reached never holds up a request or -stdout. When the buffer is full, its oldest line is dropped to make room. A line -whose sending fails is dropped too, and the connection closed. That failure, -like a failed attempt to connect, is logged and followed by the next attempt to -connect a second later, twice as long after each further failure up to a minute, -and a second again after a connection that stayed up for a minute before it -failed. A line too long for one UDP datagram is dropped alone, with no wait and -nothing logged. UDP gives no sign of what arrives, and over TCP and TLS a line -sent on a connection the server has just closed can be lost before a failure -shows; such a loss is not counted. - -As `smallwebwaf` stops, it sends the lines still waiting, on the connection open -or a new one, for at most two seconds, and gives up the rest; stdout has carried -them. - -## Alerts - -`smallwebwaf` sends each alert to each destination you name: to -`SWWAF_ALERT_WEBHOOK_URL` it posts the alert as one JSON object, with -`Content-Type: application/json` and the headers `SWWAF_ALERT_WEBHOOK_HEADERS` -gives, as "Alert webhook schema" in [`SPEC.md`](SPEC.md) describes, and to -`SWWAF_ALERT_SLACK_WEBHOOK_URL` and `SWWAF_ALERT_NTFY_URL` a message made from -it, as below. An alert is for one of these events, and is sent when -`SWWAF_ALERT_EVENTS` names its event: - -- `ban`: a ban `smallwebwaf` makes, for a broken rate limit or byte limit, a - clear sign of attack, or a client the CrowdSec decision list lists. -- `permanent_ban`: a permanent ban it makes, or a ban for a clear sign of attack - that a request made permanent. -- `waf_block`: a request the Core Rule Set scored at or over - `SWWAF_WAF_ANOMALY_THRESHOLD`, in `block` mode, which refused it, and in - `detect` mode, which let it through, with `mode`, `detect`, in its `detail`; - in `observe` mode, `block` refused nothing either, and `mode` is `observe`. -- `anomaly`: a count of requests or bytes over an anomaly threshold, raised by - each request that ends with the count over it, in `observe` mode as in - `enforce` mode. It refuses and bans nothing. -- `reputation_hit`: a request whose client a blocklist, the CrowdSec decision - list or a DNSBL zone's verdict lists, or whose AbuseIPDB score is a hit, one - alert for each blocklist and each zone that lists it, one for the decision - list, and one for AbuseIPDB, whatever `SWWAF_BLOCKLIST_ACTION` and - `SWWAF_REPUTATION_ACTION` say, in `observe` mode as in `enforce` mode. -- `source_failure`: GeoJS failing or refusing `smallwebwaf`, a fetch of a list - failing (see "Blocklists" below), the CrowdSec decision list's among them (see - "CrowdSec" below), a query to a DNSBL zone failing or refused (see "DNS - blocklists" below), or a check with AbuseIPDB failing or refused, or the check - that uses up the day's AbuseIPDB checks (see "AbuseIPDB" below). -- `file_error`: a rule file edited while it runs that has an error, an edit of a - state file set aside as `.bad`, a state file it could not write while - running, or a replacement of the lookup database it could not read, which it - does not use. - -The bans you make, in `bans.json` or through the ban endpoints, raise no alert. -In `observe` mode, a request that would have made a ban, or made one permanent, -raises the alert `enforce` mode would have raised, for the ban as it would have -been, with `mode`, `observe`, in its `detail`: no ban was made, or made -permanent. A request that would have made a ban whose alert would be held back, -by the cooldown or past `SWWAF_ALERT_MAX_PER_HOUR`, raises none, and is not -counted. This is the alert for a ban for a broken rate limit, shown indented; it -is sent on one line: - -```json -{ - "instance": "fsn1app1/gitea", - "time": "2026-10-06T12:00:00.123461Z", - "event": "ban", - "client": "203.0.113.9", - "netblock": "203.0.113.9/32", - "asn": "", - "as_name": "", - "country": "", - "reason": "requests per minute over the limit of 1000", - "detail": { - "ban_expires": "2026-10-06T13:00:00.123Z", - "cause": "limit", - "notes": { - "asn": "", - "as_name": "", - "country": "", - "kind": "requests", - "limit": 1000, - "window": "minute", - "count": 1001, - "request": { - "time": "2026-10-06T12:00:00.123456789Z", - "method": "GET", - "host": "app.example", - "path": "/owner/repo/commits/branch/main?page=812", - "status": 403, - "user_agent": "scraper/1.0" - }, - "requests": 5210, - "refused": 0, - "earlier_bans": { - "limit": 0, - "attack": 0, - "admin": 0, - "crowdsec": 0 - } - } - }, - "suppressed_repeats": 0 -} -``` - -- `instance` is `SWWAF_INSTANCE_NAME`, and `time` when the alert was raised, in - UTC. -- `client` is the address of the client whose request raised the alert, and - `netblock` the netblock of the ban, or for an `anomaly`, the netblock counted: - the client's own, the netblock around it or a named netblock, and none for an - AS number or the whole service, or for a `reputation_hit` or a `waf_block`, - the client's own, as `client_group` gives it; both are empty for - `source_failure` and `file_error`. `asn`, `as_name` and `country` are, for a - ban, the client's as the ban's notes give them when the alert is raised: - empty, as in this alert, when GeoJS had not answered about the client by then; - for an `anomaly`, the client's as the lookup gave them by the time its request - ended; for a `reputation_hit` or a `waf_block`, the client's as its request's - log line gives them. -- `reason` is a short sentence; for a ban, the ban's `reason` in `bans.json`; - for an `anomaly`, what was counted over which threshold, such as - `requests per minute of the netblock 203.0.113.0/24 over the threshold of 1000`; - for a `reputation_hit`, `listed by a blocklist`, - `listed by the CrowdSec decision list`, `listed by a DNSBL zone` or - `scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE`; for a `waf_block`, - `scored by the Core Rule Set at or over SWWAF_WAF_ANOMALY_THRESHOLD`. -- `detail` is what is particular to the event: for a ban, its `cause`, when it - ends as `ban_expires`, in the form the request log gives it, and its `notes`, - as `bans.json` gives them; for an `anomaly`, the `scope`, `client`, `net`, - `asn`, `total` or `watch`, as the settings name them, the `asn` counted for - `asn` and the `name` of the named netblock for `watch`, the `window`, `minute` - or `hour`, the `kind`, `requests` or `bytes`, the `count`, which is weighted - as the rate limits weigh theirs, and the `threshold`; for a `reputation_hit`, - the `source`, the URL of the blocklist or of the CrowdSec decision list, the - zone or `abuseipdb`, and for AbuseIPDB the `score`; for a `waf_block`, the - `rule_ids` and the `score`, as the request log gives them, the request's - `method`, its `path` with the query, and the `mode` when the request was not - refused for it; for `source_failure`, the `source`, `geojs`, the URL of the - list, the zone or `abuseipdb`, the `error`, and for GeoJS, when it is asked - again, `asking_again_in`; for `file_error`, the `file`, which for an edit set - aside is the file it was renamed to, and the `error`, which for a file that - does not parse names where in it the error is. -- `suppressed_repeats` is how many repeats the cooldown held back before this - alert, and for a `summary`, those no other alert gives (see below). - -Slack and ntfy are each sent the alert as a message: a title, the instance and -the event, such as `fsn1app1/gitea: ban`, and a text, the `reason`, then a line -for each of the `client`, the `netblock` and the `country`, the `file`, the -`source`, the `error` and the `mode` the `detail` gives, and the -`suppressed_repeats`, leaving out those that are empty or 0. Slack is posted, as -JSON, the title in bold and the text below it, with `&`, `<` and `>` escaped, so -that nothing in them is read as a link or a mention. ntfy is posted the text, -with the title as `Title`, `SWWAF_ALERT_NTFY_TOKEN`, while it is set, as -`Authorization: Bearer `, and the priority and the tag, which ntfy shows -as an emoji, of the alert's event, as `Priority` and `Tags`: - -| Event | Priority | Tag | -| ------------------------------ | --------- | -------------------------- | -| `ban` | `default` | `no_entry` | -| `permanent_ban` | `high` | `no_entry` | -| `waf_block` | `default` | `shield` | -| `anomaly` | `high` | `chart_with_upwards_trend` | -| `reputation_hit` | `low` | `label` | -| `source_failure`, `file_error` | `high` | `warning` | -| `summary` | `default` | `bar_chart` | - -For the alert above, ntfy is sent these headers and this text: - -```text -Title: fsn1app1/gitea: ban -Priority: default -Tags: no_entry - -requests per minute over the limit of 1000 -client: 203.0.113.9 -netblock: 203.0.113.9/32 -``` - -and Slack this JSON object, shown indented; it is sent on one line: - -```json -{ - "text": "*fsn1app1/gitea: ban*\nrequests per minute over the limit of 1000\nclient: 203.0.113.9\nnetblock: 203.0.113.9/32" -} -``` - -An alert for the same event as the last one sent, on the same netblock, and for -a `reputation_hit` about the same blocklist, decision list, zone or AbuseIPDB, -or for a `file_error` about the same file, or for a `source_failure` about the -same source, or for an `anomaly` in the same scope, with the same netblock, AS -number or name, whatever its window and kind, less than `SWWAF_ALERT_COOLDOWN` -after it, is a repeat: it is held back and counted, and the next alert sent for -them gives that count as `suppressed_repeats`. As each hour of the clock, in -UTC, ends, the cooldowns that have run out are dropped, and the repeats they -held back, which no alert sent since has given, go in that hour's summary. - -Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the hour's other alerts are -held back and counted by event. An alert held back this way starts no cooldown. -Once the hour has ended, one alert sums up the alerts held back and the repeats -of the cooldowns dropped: its `event` is `summary`, its `reason` says how many -of each were held back, its `detail` gives the `hour` as when it started, the -`count` of alerts held back, and the count for each event, as `events`, and its -`suppressed_repeats` gives the repeats. An hour with neither ends without a -summary. - -Each destination has a queue of its own, of at most 1000 alerts, from which they -are sent to it one at a time, the oldest first, so a destination that is slow or -down holds up neither the others nor any request. A destination takes an alert -by answering with a 2xx status, and refuses it with a 4xx status other than -`408` and `429`: a refused alert is logged, counted as dropped, and given up, so -that the next is sent. Any other answer, a redirect included, a connection that -fails, or no answer within 10 seconds is a failure: it is logged, naming the -destination's setting and not its URL, and the alert is sent again a second -later, twice as long after each further failure in a row, up to a minute. With -1000 alerts waiting for a destination, the oldest is dropped to make room for a -new one. The cooldowns, the hour under way and the alerts still waiting for each -destination are kept in `alerts.json` (see "State files" below), so that after a -restart the alerts waiting are sent, and the cooldowns go on. - -## State files - -`smallwebwaf` keeps its state in memory and a copy of it in five JSON files in -`SWWAF_STATE_DIR`, `/var/lib/smallwebwaf` by default, as "Persistent state" in -[`SPEC.md`](SPEC.md) describes. Each has a top-level `version`, 1, and lists its -entries by client address, but for the alerts waiting, the anomaly counters, -which are listed by scope first, and the copies of the lists, listed by URL, -with times in UTC. - -- `bans.json`: every ban with its notes, indented to be read. A permanent ban's - `expires` is `null`. A ban's `cause` is `limit` for a broken rate limit, byte - limit or error burst, `attack` for a clear sign of attack or `crowdsec` for a - client the CrowdSec decision list lists, for a ban `smallwebwaf` made, and - `admin` for one you made or keep. Its `reason` is a short text: for a ban - `smallwebwaf` made, the limit broken, such as - `requests per minute over the limit of 1000`, - `bytes per hour over the limit of 21474836480` or - `refusals per minute over the limit of 30`, the rule that matched, such as - `matched the rule env-file`, the trap path asked for, such as - `asked for the trap path /wp-login.php`, or the scenario that made CrowdSec's - decision, such as `CrowdSec's decision for crowdsecurity/ssh-bf`; for yours, - what you wrote. Its `lifted` is when you lifted it, and is left out until you - do. The `kind` in the notes of a ban for a broken limit is `requests`, `bytes` - or `refusals`, for the error burst, what the limit is on. The notes of a ban - for a clear sign of attack give the `rule_id` and the `target` of the rule - that matched, or the `trap_path` asked for. For a limit a biased threshold - lowered, the reason and the notes' `limit` give the lowered limit, and the - notes' `limit_percent` and `limit_percent_setting` the client's percentage of - that kind of limit and the setting that gave it. The notes' `reputation` gives - each blocklist, the CrowdSec decision list, each DNSBL zone or AbuseIPDB that - listed the client when the ban was made, as its `source`, named and ordered as - in the request log's `reputation`, with AbuseIPDB's `score` of the client. It - is left out when none did, and the example below shows it. -- `clients.json`: each client's two buckets of requests in the minute, the hour - and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes` - and `day_bytes`, its two buckets of refusals in the minute, which the error - burst counts, `minute_refusals`, and its history: when it was first and last - seen, its AS number, AS name and country as last looked up and when the lookup - gave them, its requests, how many were forwarded and how many refused (one - `smallwebwaf` answered at its own endpoints is neither, unless it was refused - with `401` for a missing or wrong token), the body bytes in each direction, - its responses by status class and its offences by kind: `limit` for a broken - rate limit, byte limit or error burst, `attack` for a clear sign of attack, - `rule_blocked` for a request a `block` rule refused, `waf_blocked` for one the - Core Rule Set refused and `token_refused` for one refused for a missing or - wrong token. Each client is on a line of its own, so `grep` shows everything - about one. -- `lookups.json`: GeoJS's answers, one to a line, each with the client's AS - number, AS name and country, when GeoJS gave it and when it was last used. -- `reputation.json`: each list fetched from a URL (see "Blocklists" and - "CrowdSec" below), indented to be read, under `lists`: its `url`, when it was - last `tried`, the fetch failed or not, and its last good copy: when that was - `fetched`, and its `lines`, as fetched, comment lines included, each on a line - of its own, both left out while no fetch of it has succeeded; and under - `verdicts`, each verdict of a DNSBL zone still in use (see "DNS blocklists" - below): its `zone`, the `client`'s address, whether the zone `listed` the - client, and when the zone gave it, `fetched`; and under `abuseipdb` (see - "AbuseIPDB" below), the `day`, in UTC, whose checks it counts, left out before - the first, the checks `spent` that day, and under `scores`, each score of - AbuseIPDB still in use: the `client`, its IPv4 address as a /32 or its IPv6 - group, its `score`, and when AbuseIPDB gave it, `fetched`. As the file is - read, the lists the settings no longer name, and the verdicts of the zones - they no longer name, are dropped. -- `alerts.json`: the state of the alerts (see "Alerts" above), indented to be - read: under `cooldowns`, for each event and netblock, with the `source` too - for a `reputation_hit`, or event and `file` or `source`, or for an `anomaly`, - its `scope` with its `netblock`, `asn` or `name`, or event alone, when the - last alert was sent, `sent`, and the repeats held back since, - `suppressed_repeats`; under `hour`, the hour under way, from its `start`, the - alerts `sent` in it and those `held_back` for its summary, by event; under - `waiting`, for each destination you name, `webhook`, `slack` or `ntfy`, the - alerts still waiting to be sent to it, the oldest first, each as the webhook - is sent it; and under `anomaly_counters`, each anomaly counter: its `scope`, - as an `anomaly` alert names it, with the `netblock` of a client, of a netblock - around a client or of a named netblock, the `asn` of an AS number and the - `name` of a named netblock, and its two buckets of requests in the minute and - the hour, `minute` and `hour`, and of bytes, `minute_bytes` and `hour_bytes`, - each left out while it is empty. As an hour ends, the cooldowns that have run - out are dropped, and the hour's summary gives the repeats they held back. As - the file is read, the alerts waiting for a destination you no longer name are - dropped. A file whose `waiting` is a list, as it was before alerts went to - Slack and ntfy too, stops the start: put the list under `"webhook"`, or remove - the file. - -This `bans.json` holds a ban for a broken rate limit on a client that a DNSBL -zone lists and AbuseIPDB scores at 100, whose limits `SWWAF_REPUTATION_ACTION`, -at its default of `limit:25`, lowered to a quarter: - -```json -{ - "version": 1, - "bans": [ - { - "netblock": "203.0.113.9/32", - "start": "2026-10-06T12:00:41.5Z", - "expires": "2026-10-06T13:00:41.5Z", - "cause": "limit", - "reason": "requests per minute over the limit of 250", - "notes": { - "asn": "AS64496", - "as_name": "Example Net", - "country": "DE", - "kind": "requests", - "limit": 250, - "window": "minute", - "count": 251, - "limit_percent": 25, - "limit_percent_setting": "SWWAF_REPUTATION_ACTION", - "reputation": [ - { - "source": "dnsbl.dronebl.org" - }, - { - "source": "abuseipdb", - "score": 100 - } - ], - "request": { - "time": "2026-10-06T12:00:41.5Z", - "method": "GET", - "host": "app.example", - "path": "/owner/repo/commits/branch/main?page=812", - "status": 403, - "user_agent": "scraper/1.0" - }, - "requests": 512, - "refused": 0, - "earlier_bans": { - "limit": 0, - "attack": 0, - "admin": 0, - "crowdsec": 0 - } - } - } - ] -} -``` - -`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted -through `DELETE /_smallwebwaf/bans/`, or made permanent, with every such -change in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and when -`smallwebwaf` stops. Each write goes to a temporary file in the same directory, -which then replaces the file, so a crash leaves the old file or the new one, -whole. A write that fails is logged, raised as a `file_error` alert while -`smallwebwaf` runs, and tried again at the next write. A hard kill loses what -changed since the last write. - -At start the files are read back: each client keeps its counts, so a restart -gives it no fresh allowance, each anomaly counter keeps its counts, each ban -keeps refusing every client in its netblock until it ends, even after -`SWWAF_BAN_SCOPE_V4_PREFIX` has changed, and the copy of each list stays in use -until a fetch of it succeeds. A netblock whose address has bits past its length, -such as `203.0.113.9/24`, is read as the netblock it is in, `203.0.113.0/24`. -Buckets and answers whose time has passed are dropped, and so is an anomaly -counter left with no bucket; a verdict past its time is neither used nor written -again. A missing file is empty state, as on a first start. A file that does not -parse, or has another `version`, stops the start with a message naming the file, -and the line and column where Go's JSON decoder gives them; so does a state -directory `smallwebwaf` cannot write. So does an entry without a field it needs, -named with the entry's place in the file: a ban's `netblock`, `start` or -`expires`, which is `null` for a permanent ban; a client's `client`, or the -`start` of a window in which it has requests, bytes or refusals; an answer's -`client`, `country`, which is `""` for a client GeoJS cannot place, or -`answered`; a cooldown's `event` or `sent`; an alert waiting's `event` or -`time`; an anomaly counter's `netblock`, unless it counts an AS number or the -whole service, its `asn`, for an AS number, its `name`, for a named netblock, or -the `start` of a window in which it has requests or bytes; a list's `url`, -`fetched` or `lines`, which is `[]` for an empty list; a verdict's `zone`, -`client`, `listed`, which is `false` for a client the zone does not list, or -`fetched`. So does a ban whose `expires` is not after its `start` or whose -`cause` is not `limit`, `attack`, `admin` or `crowdsec`, alerts waiting for a -destination that is not `webhook`, `slack` or `ntfy`, an anomaly counter whose -`scope` is not `client`, `net`, `asn`, `total` or `watch`, and a copy of a list -with a line that would make its fetch fail. An answer's `asn` or `as_name` left -out reads as empty. - -While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of -a state file as soon as you save it: what the file then holds replaces what -`smallwebwaf` held for it, as if read at start. It tells its own writes from -yours by comparing the file with what it last read or wrote, and before it -writes a file it takes in any edit made since, so your edit is not overwritten; -a change `smallwebwaf` made after you opened the file, such as a new ban, is -lost when you save over it. An edit that would stop the start, because it does -not parse, has another `version`, leaves out a field an entry needs, gives a ban -an `expires` not after its `start` or another `cause`, names another -destination, gives an anomaly counter another `scope` or gives a list's copy a -line that would make its fetch fail, does not stop the running `smallwebwaf`: it -keeps what it holds, and at the file's next write renames your file to -`.bad`, such as `bans.json.bad`, writes the file again from memory, logs -the file and where the error is, and raises a `file_error` alert for it. It -waits for that write because an editor's file can be read before the editor has -finished writing it. Mend the `.bad` file and move it back. A file you remove is -written again at its next write. - -To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start` -and its `expires`, `null` for a ban that never ends; its `reason` and its -`notes` may be left out, and so may its `cause`, which is then `admin`, and is -written so at the file's next write. A ban whose `cause` is `admin` is never -dropped and does not count toward `SWWAF_MAX_BANS`. A ban whose `cause` is -`attack` becomes permanent at the first request it refuses; one whose `cause` is -`admin` or `crowdsec` does not. This `bans.json` bans `203.0.113.0/24` for good: - -```json -{ - "version": 1, - "bans": [ - { - "netblock": "203.0.113.0/24", - "start": "2026-10-06T12:00:00Z", - "expires": null, - "reason": "probes for logins" - } - ] -} -``` - -To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause` -to `admin`: `"cause": "admin"`. - -To lift a ban, add `lifted` to its entry, with the time you lift it, such as -`"lifted": "2026-10-06T13:00:00Z"`. From when the edit is taken in, the ban -refuses nothing, whatever time `lifted` gives, and does not make the netblock's -next ban longer; it is kept in `bans.json` with its notes, as any other ban is. -To forget a ban altogether, delete its entry: it then refuses nothing either, -and does not make the netblock's next ban longer. - -The ban endpoints add and lift bans without an edit of the file (see "Admin -endpoints" below). - -## Rule files - -`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`, -`/etc/smallwebwaf/rules.d` by default, in the order of their names, and checks -each request against their rules in that order, as "Rule files" in -[`SPEC.md`](SPEC.md) describes. A file whose name starts with `.`, such as an -editor's lock file `.#50-app.rules`, is not a rule file, as a shell's `*.rules` -would not match it. A rule is a line of four fields separated by spaces or tabs: -an id, a target, an action and a regex, which runs to the end of the line. -Spaces and tabs at the end of a line are not part of its regex, so a line with -only those after its action has no regex, and is not a rule. Blank lines and -lines that start with `#` are ignored. - -``` -# id target action regex -env-file path ban (?i)^/\.env(\.[a-z]+)?$ -scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b -``` - -- The id is letters, digits, `-` and `_`, and no two rules share one. The - request log, the metrics and a ban's notes name the rule by it. -- The target is what the regex is matched against: `path`, the path without the - query, and `uri`, the path and the query together, each both as sent and once - percent-decoded, so that an encoded probe does not slip past; `query`, as the - client sent it, before any decoding; `method`; `host`; `user_agent`; - `referer`; or `header:`, any one request header but `Host` and - `Transfer-Encoding`, which Go's HTTP server takes out of every request; the - request's host is the target `host`. A header sent more than once is matched - with its values joined by `, `, and one not sent as empty text. No body is - read. -- The action is `log`, `block` or `ban` (see "What it does so far" above). Keep - `ban` for requests no real visitor sends, and anchor a path at the site root - with `^/`: a file of the same name deeper in a site can be ordinary content, - such as a file in a repository on a code forge. -- The regex is in Go's syntax, RE2, which has no backreferences or lookaround, - and takes time linear in the text it reads. It matches anywhere in the target - unless anchored with `^` and `$`; `(?i)` at its front makes it ignore case. - -A line that is not a rule, a `header:` name with a character no header name can -have, such as `header:User-Agent:`, a rule for the `Host` or the -`Transfer-Encoding` header, a regex that does not compile or an id used twice -stops the start with a message naming the file and the line, and so does a -`SWWAF_RULES_DIR` that does not exist. An empty directory is no error, and the -log says that it holds no rules. While it runs, `smallwebwaf` watches the -directory, and reads the rule files again once the directory has had no change -for 2 seconds after one is edited, added or removed, so that a file saved in -place, appended to or copied in with `scp` is read only once whole, unless its -writing stops for longer. It also reads them 2 seconds after it starts watching, -so that an edit saved while it started is not missed. If they then hold one of -those errors, the rules stay as they were, the earlier version of the edited -file included, the log and a `file_error` alert name the file and the line, and -the files are read again after the next change. - -The image ships one rule file, `share/rules.d/00-default.rules` here: rules that -ban probes no real visitor sends, for secrets, version control directories, -backups, logs and web shells at the site root, and the user agents of common -scanners; one that blocks `../` twice in a row in the path or the query; and one -that only notes a request without a user agent. An app's Dockerfile adds rules -of its own in a file beside it, named to sort after it, such as this -`50-gitea.rules` for an app that serves no WordPress: - -``` -wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/) -``` - -```dockerfile -COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules -``` - -A directory mounted over `/etc/smallwebwaf/rules.d` replaces the default file, -and single files mounted into it add to it. Docker does not show a single -mounted file being replaced, which is how many editors save, so rules to be -edited while `smallwebwaf` runs belong in a mounted directory, with a copy of -`00-default.rules` if its rules are to stay. To run without rules, mount an -empty directory or set `SWWAF_RULES_ENABLED=false`. - -## Metrics - -`GET /_smallwebwaf/metrics` answers with the metrics in the Prometheus text -format, for a scraper that sends `SWWAF_METRICS_TOKEN`, through traefik like any -other request. No metric carries a client's address. - -Every metric below, Go's and the process's included, carries the label -`instance`, `SWWAF_INSTANCE_NAME`, as a constant label set once on the registry -the metrics are kept in, rather than as a label each metric declares. Prometheus -gives each series it scrapes an `instance` label of its own, the address it -scraped, and keeps this one as `exported_instance` unless the scrape sets -`honor_labels: true`. - -- `smallwebwaf_requests_total`, `smallwebwaf_request_bytes_total` and - `smallwebwaf_response_bytes_total`: requests, and their body bytes each way, - by `status_class`, such as `2xx`, or `none` when nothing was sent, and by - `action`, as the request log names it. -- `smallwebwaf_request_duration_seconds`: how long requests took, and - `smallwebwaf_upstream_duration_seconds`: how long those passed to the app took - from then on, as histograms; `smallwebwaf_requests_in_flight`: the requests - under way. -- `smallwebwaf_rate_limit_hits_total` by `window`, `minute`, `hour` or `day`, - and `kind`, `requests` for a rate limit, `bytes` for a byte limit or - `refusals` for the error burst, whose window is `minute`, - `smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose - limit was passed, `smallwebwaf_offences_total` by `kind`, `limit`, `attack`, - `rule_blocked`, `waf_blocked` or `token_refused`, as `clients.json` counts - them, and `smallwebwaf_bans_made_total` by `cause`, `limit`, `attack`, `admin` - or `crowdsec`, `admin` for the bans you add through `POST /_smallwebwaf/bans`, - and those whose `cause` is `admin` that you add to `bans.json` while - `smallwebwaf` runs; `smallwebwaf_active_bans` and - `smallwebwaf_permanent_bans`, neither of which counts a lifted ban. -- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by - `rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the - rules read from the rule files. -- `smallwebwaf_waf_matches_total`: the requests that matched each rule of the - Core Rule Set, whatever their score, by `mode`, `SWWAF_WAF_MODE`, and - `rule_id`, a series for each rule that has matched. -- `smallwebwaf_country_requests_total`, - `smallwebwaf_country_request_bytes_total`, - `smallwebwaf_country_response_bytes_total`, and - `smallwebwaf_country_list_refusals_total`, the requests the country lists - refused, by `country`, for the requests whose client's country is known. The - `SWWAF_METRICS_TOP_N` countries with the most requests since the start have - series of their own, and the others are counted as `other`. A country that - drops out of them loses its series, and its later requests count as `other`; - one that comes into them gets a series that counts from then on. -- `smallwebwaf_asn_requests_total`, `smallwebwaf_asn_request_bytes_total` and - `smallwebwaf_asn_response_bytes_total`, by `asn`, the client's AS number, for - the requests whose client's AS number is known, with the `SWWAF_METRICS_TOP_N` - busiest AS numbers kept as the countries are. -- `smallwebwaf_geojs_requests_total`: the requests to GeoJS; - `smallwebwaf_geojs_failures_total`: those that failed, an answer that leaves - out an address asked about included; and `smallwebwaf_geojs_unanswered_total`: - the requests that needed their client's answer, for a country list, - `SWWAF_ADD_LOOKUP_HEADERS` or a biased threshold, and went on without it - because GeoJS had not given it in time. -- While `SWWAF_LOOKUP_SOURCE` is `file`, - `smallwebwaf_lookup_database_last_read_timestamp_seconds`: when the lookup - database in use was read; and - `smallwebwaf_lookup_database_read_failures_total`: the replacements of it that - could not be read. -- By `source`, the URL of each list `SWWAF_BLOCKLIST_URLS` or - `SWWAF_ASN_LIMIT_PERCENT_URL` names, and of the CrowdSec decision list: - `smallwebwaf_reputation_hits_total`: the requests whose client the blocklist - or the decision list lists, a series that comes with the first; - `smallwebwaf_reputation_failures_total`: the fetches of the list that failed; - and `smallwebwaf_reputation_last_fetch_timestamp_seconds`: when the copy of it - in use was fetched, `0` while there is none. -- By `source`, each zone `SWWAF_DNSBL_ZONES` names: - `smallwebwaf_reputation_hits_total`: the requests whose client the zone's - verdict lists, a series that comes with the first; - `smallwebwaf_reputation_queries_total`: the queries made to the zone; and - `smallwebwaf_reputation_failures_total`: those that failed. -- While `SWWAF_ABUSEIPDB_KEY` is set, by `source`, `abuseipdb`: - `smallwebwaf_reputation_hits_total`: the requests whose client's AbuseIPDB - score is a hit, a series that comes with the first; - `smallwebwaf_reputation_queries_total`: the checks made, each of which spends - one of the day's budget; `smallwebwaf_reputation_failures_total`: those that - failed; and `smallwebwaf_reputation_daily_budget_remaining`: the checks the - day's `SWWAF_ABUSEIPDB_DAILY_BUDGET` has left. -- `smallwebwaf_tracked_clients`: the clients in the table of clients. -- `smallwebwaf_state_file_writes_total`, - `smallwebwaf_state_file_write_failures_total`, - `smallwebwaf_state_file_last_write_timestamp_seconds` and - `smallwebwaf_state_file_size_bytes`, by `file`; and, by `file` too, - `smallwebwaf_state_file_edits_taken_in_total`: your edits taken in, and - `smallwebwaf_state_file_edits_set_aside_total`: those renamed to `.bad` - because they would stop the start. -- While `SWWAF_LOG_REMOTE_URL` is set, - `smallwebwaf_remote_log_lines_sent_total`: the lines sent to it; - `smallwebwaf_remote_log_lines_dropped_total`: those dropped, from a full - buffer or because their sending failed; and - `smallwebwaf_remote_log_buffer_depth`: those waiting in the buffer. -- For each destination you name, by `destination`, `webhook`, `slack` or `ntfy`: - `smallwebwaf_alerts_sent_total`: the alerts it took; - `smallwebwaf_alerts_failed_total`: the requests to it that failed; - `smallwebwaf_alerts_suppressed_total`: the alerts held back, as repeats or for - an hour's summary, which are the same for every destination; and - `smallwebwaf_alerts_dropped_total`: those dropped from its full queue, or - given up as it refused them. -- Go's own `go_` metrics and the process's `process_` metrics. - -The requests Go's HTTP server ends before `smallwebwaf` sees them (see "Request -log") are not counted. - -## Admin endpoints - -While `SWWAF_ADMIN_TOKEN` is set, `smallwebwaf` answers these requests itself, -on the app's own address and through traefik like any other request, for a -request that carries the token as `Authorization: Bearer `: - -- `GET /_smallwebwaf/bans`: every ban held, past, active and permanent. -- `POST /_smallwebwaf/bans`: bans a netblock, as adding an entry to `bans.json` - does. The body is a JSON object of `netblock`, `duration` and, if you like, - `reason`. `netblock` is a netblock such as `203.0.113.0/24`, or a client's - address, which bans the netblock a ban on that client covers: its IPv4 - address, or the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or - its IPv6 group. `duration` is a duration such as `1h` or `7d`, or `permanent`. - The ban starts at once, its `cause` is `admin`, and it is made even while - another ban on the netblock lasts. A body that is not such an object, has - another field, has anything but whitespace after the object, or is longer than - 4 KiB is answered `400`, saying what is wrong, and so is an IPv4-mapped - netblock, such as `::ffff:203.0.113.0/120`, or a value with a zone, such as - `fe80::1%eth0`. -- `DELETE /_smallwebwaf/bans/`: lifts every active ban on a netblock - that ``, an address, is in, as adding `lifted` to its entry in - `bans.json` does, and answers `404` when no ban on it is active. -- `GET /_smallwebwaf/clients/`: what `smallwebwaf` knows of the client at - the address ``: under `client`, the client as `clients.json` holds it, - with its counters and its history, which holds its AS number, AS name and - country as last looked up and its offences, or `null` when the table of - clients does not hold it; and under `bans`, every ban on a netblock the - address is in, with its notes. - -The ban endpoints answer with the bans listed, made or lifted, under `bans`, -each as an entry of `bans.json` (see "State files" above), and a ban they make -or lift is written to `bans.json` `SWWAF_STATE_WRITE_DELAY` later. Refusals, and -the answers to requests that cannot be read, are plain text. - -A request without the token, or with another, such as the metrics token, is -answered `401`, in `observe` mode too, and counts toward the error burst, as one -for the metrics without theirs does. While the token is unset, each of these -answers `404`, as does any request under `/_smallwebwaf/` that is not for one of -its endpoints. Like the metrics, these requests go through every check any other -request goes through, and are answered where another would be passed to the app: -a banned client stays refused, so an admin whose own address is banned lifts -that ban by editing `bans.json`, and each request counts toward the client's -rate limits. A client in `SWWAF_ALLOW_NETS` skips the checks, and still needs -the token. - -With the token in `$TOKEN`, for an app at `https://app.example`: - -```sh -# Every ban. -curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans - -# Ban 203.0.113.0/24 for seven days. -curl -H "Authorization: Bearer $TOKEN" \ - --json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \ - https://app.example/_smallwebwaf/bans - -# Lift the bans on 203.0.113.9. -curl -H "Authorization: Bearer $TOKEN" -X DELETE \ - https://app.example/_smallwebwaf/bans/203.0.113.9 - -# What smallwebwaf knows of 203.0.113.9. -curl -H "Authorization: Bearer $TOKEN" \ - https://app.example/_smallwebwaf/clients/203.0.113.9 -``` - -## Why - -Small self-hosted sites now receive a great deal of traffic nobody asked for: -scrapers that ignore `robots.txt` and crawl every commit of every repository on -a public git server, vulnerability scanners walking through lists of WordPress -and `.env` paths, and credential-guessing bots. Most of it comes from a small -number of hosting networks and countries. A single-person operation has no abuse -desk and no CDN contract; it needs something small that can be put in front of -one service and left alone. - -The existing tools each solve part of this. Rule-based firewalls catch attack -payloads but do not limit request rates. Rate limiters count requests but cannot -tell a residential visitor from a rented server farm. The products that do most -of it want several containers, a database and a web console. None of them can -say "clients from these networks are banned after half as many requests as -anyone else", which is the most useful thing to be able to say when nearly all -abuse comes from a known list of AS numbers. [`EVALUATION.md`](EVALUATION.md) -goes through the candidates one by one. - -`smallwebwaf` is meant to fill that gap: - -- protect a service from misbehaving scrapers and scanners with per-client - request and byte limits over a minute, an hour and a day; -- lower those limits for the countries and AS numbers that abuse commonly comes - from, so their clients are banned after fewer requests than others; -- ban abusers: briefly at first, longer each time they come back, and - permanently when they keep at it; a scanner's first probe bans it for seven - days; -- log everything in a form that is easy to search and ship elsewhere; -- stay small enough to understand: one binary in the app's own container, - environment variables, no database, no required setting. - -## Proposed features - -- Reverse proxy for one application, streaming in both directions, with - WebSocket support. One `smallwebwaf` per app, inside the app's own container. -- Internet-ready out of the box: no setting is required, and every setting has a - default chosen for a service facing the internet in 2026. Every setting's name - starts with `SWWAF_`, so that it cannot clash with the app's own. -- Real client address worked out from `X-Forwarded-For`, trusting only the proxy - networks you list, by default the private address ranges. IPv6 clients are - counted by /64 by default. -- Size and time limits on requests and responses, with the time limits both - between the client and `smallwebwaf` and between `smallwebwaf` and the app: by - default a request may take 60 seconds and 100 MiB, a response 30 minutes and 5 - GiB. -- Rate limits per client on requests per minute, per hour and per day, and on - bytes per minute, per hour and per day, on by default and set well above what - real visitors need. -- Netblocks that bypass rate limiting, netblocks that bypass everything, and - netblocks that are always refused. -- AS number and country lookup for every client, on by default through the free - GeoJS web service, which is sent the address of every new visitor. The IPinfo - Lite database file, which you download and mount, can be used instead, or - lookups switched off (see "Country and AS number lookup" below). -- Country lists: `SWWAF_DENIED_COUNTRIES` refuses every request from the - countries listed, `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` every request from - anywhere else. Such a request gets the answer a banned client gets as soon as - the client's address has been looked up, before its body is read and without - the rule files or the Core Rule Set looking at it, and no ban is made. -- Biased limits: listed AS numbers and countries get a percentage of every - limit, for example 50 percent for common abuse-source networks, so their - clients are banned after fewer requests. Zero percent is a zero allowance: the - first request breaks the limit and bans the client. -- Attack detection: - - a directory of plain text rule files, one regex per line, for catching - scanning and penetration probes; easy to edit by hand, and picked up while - running; - - the OWASP Core Rule Set, run by the Coraza engine, refusing the requests - it flags; it reads the URL and headers of every request, and request - bodies only once you switch that on, since on a code forge they are full - of code it would take for attacks; - - trap paths, and a ban for a client that the rule files or the Core Rule - Set refuse again and again. -- Bans: - - a clear sign of attack, such as a probe for a `.env` file or a scanner's - user agent, bans for seven days on the first request, and any further - request during those days makes the ban permanent; - - breaking a limit bans for an hour; breaking one again within a day of a - ban ending triples the length, and a ban that would last longer than seven - days is permanent instead; - - every ban carries notes on why it was made, to help decide whether to lift - it. -- IP reputation: downloadable blocklists, DNS blocklists, AbuseIPDB, and an - optional feed of decisions from a CrowdSec engine. Lookups happen in the - background and never delay a request. None is on until you add it. -- Alerts on attacks and bans to a generic webhook, Slack or ntfy, with a - cooldown and an hourly cap so a wide attack cannot flood the channel. -- Anomaly alerts when requests or bytes per minute or hour cross a threshold you - set, for a single client, its surrounding netblock, an AS number, a named - netblock or the whole service. -- Observe mode: log and alert on every decision while refusing nothing. -- Request log: one JSON object per request on stdout with the usual web log - fields, the decision taken and why, AS number and country, and timings. - Optionally also sent to a remote syslog server. -- Prometheus metrics, for a scraper that holds the metrics token. -- State (bans with their notes, each client's counters and history, the GeoJS - answers, the reputation cache, the alerting state) held in memory and kept in - readable JSON files, written regularly and at every stop, so a restart loses - nothing. Edit a file, or add a rule file, and the running `smallwebwaf` picks - up the change. Nothing is read from disk while serving a request. The files - for the bans, the clients, the GeoJS answers, the copies of the lists, the - verdicts of the DNSBL zones, AbuseIPDB's scores and the alerts are built, with - an edit taken in while running (see "State files" above); the rest comes with - its features. -- Health checks, the metrics, and listing, adding and lifting bans or asking why - a given address was refused, all on the one port every request uses: under - `/_smallwebwaf/` on the app's own address, through traefik like any other - request. The metrics need the metrics token, and the ban management the admin - token. - -Not planned: TLS termination, routing for several apps, browser challenges -(captcha or proof of work), a web console, or defence against floods large -enough to fill the host's network link. - -## How it works, in short - -For each request `smallwebwaf`: - -- works out who the client really is; -- lets it straight through if it is on the bypass list, refuses it if it is on - the deny list or currently banned; -- looks up its AS number and country, and refuses it if that country is denied, - or is not among the only ones allowed; -- checks it against the blocklists, bans it if the CrowdSec decision list lists - it, and checks it for a cached reputation verdict; -- picks the client's limit percentage from those; -- checks the minute, hour and day request counters against the limits, and bans - the client if it breaks one; -- checks the request against the trap paths, the rule files and the Core Rule - Set, and bans the client at once for a clear sign of attack; -- forwards it to the app and streams the response back, within the size and time - limits; -- counts the bytes and any refusal by the trap paths, the rule files or the Core - Rule Set or for a missing or wrong token, bans the client if it broke a limit, - updates its history and the anomaly counters, sends any alerts that are due, - and writes the log line. - -A minimal deployment is the app's own Dockerfile, built on the `smallwebwaf` -image, with no setting. That image is built on Ubuntu 26.04 LTS, the newest -long-term support release of Ubuntu, pinned by digest, and moves to the next one -when it ships. It has nixpkgs installed, so the app adds the packages it needs -from nixpkgs. Beyond its `FROM` line the app's Dockerfile adds the app's binary, -any packages it needs, and the app's runit service, which starts the app as a -user of its own, listening on `127.0.0.1:8081`: +Push the image to a registry your hosts pull from. Then build the app's own +image on it. The app's `Dockerfile` starts `FROM` the `smallwebwaf` image, named +by digest, and adds: + +- the app's binary; +- the packages the app needs, from the nixpkgs in the image; +- a user for the app; +- a `run` script that starts the app listening on `127.0.0.1:8081`. ```dockerfile # The smallwebwaf image, pinned by digest. -FROM /smallwebwaf: +FROM /smallwebwaf@sha256: # Packages the app needs, if any, from the nixpkgs in the image. RUN nix-env -iA nixpkgs.git @@ -1917,8 +99,8 @@ RUN useradd --system --no-create-home --shell /usr/sbin/nologin app COPY --chmod=755 app.run /etc/service/app/run ``` -with `app.run` beside the Dockerfile, where `--listen` and `--trusted-proxies` -stand for the app's own options: +`app.run` sits beside the `Dockerfile`. `--listen` and `--trusted-proxies` stand +for the app's own options: ```bash #!/usr/bin/env bash @@ -1934,496 +116,336 @@ main() { main "$@" ``` -- The image's entrypoint, `runsvinit`, has runit start `smallwebwaf` and the app - side by side, each as its own user, and start either again a second after it - exits. Leave out `ENTRYPOINT` and `USER` from the app's Dockerfile. -- `nix-env -iA nixpkgs.` installs a package from the nixpkgs in the image, - and the app finds it on its `PATH`, after Ubuntu's own commands. That nixpkgs - is fixed at one commit, so the same `smallwebwaf` image always gives the app - the same packages; newer ones come with a newer `smallwebwaf` image. -- Deploy it as you deploy any app, with traefik's labels on this one container - pointing at port 8080. upaas needs no change for this. -- The app has to trust `127.0.0.1` and `::1` for forwarded headers, besides the - private address ranges, since the requests it gets now come from `smallwebwaf` - on loopback. An app left at the usual default, the private ranges alone, sees - every visitor as `127.0.0.1`. -- Port 8080 is the only one the app must leave free: the health check, the - metrics and ban management are all on it, under `/_smallwebwaf/`. The image's - health check passes while `smallwebwaf` answers and the app accepts - connections. `SWWAF_LISTEN_ADDR` can move `smallwebwaf` to another port, which - the app then leaves free instead; the health check follows it, and traefik's - labels must point at it. The address part of `SWWAF_LISTEN_ADDR` stays empty - (for example `:9000`, never `127.0.0.1:9000`), so `smallwebwaf` keeps - listening on every address: traefik reaches it on the container's address, and - the health check on `127.0.0.1`. -- `smallwebwaf` keeps its state files in `/var/lib/smallwebwaf`. Mount a volume - there to keep bans and client history when a deploy replaces the container; - without one, it still starts. At each start the `run` script of `smallwebwaf` - gives that directory and every file in it to the `smallwebwaf` user, so a host - directory mounted there needs no change of owner. -- `docker stop` has runit stop both processes at once, each with SIGTERM. - `smallwebwaf` then stops taking requests and gives those in progress five - seconds to finish, which they can do only while the app still answers. An app - that exits at once on SIGTERM ends them: the client gets `502`, or a response - cut short if the app had already begun to answer. An app that should let them - finish keeps taking and answering requests, not only the ones it already has, - for the whole five seconds after SIGTERM, since `smallwebwaf` can still pass - it requests it took before then. +Leave `ENTRYPOINT` and `USER` out of the app's `Dockerfile`. The image's own +entrypoint starts `smallwebwaf` and the app side by side, each as its own user. -A rule file is one rule per line: a name, what to match against, what to do, and -a regex. +Deploy the app's image as you deploy any app. Point traefik's labels at port +8080 (`traefik.http.services..loadbalancer.server.port=8080`). Mount a +volume at `/var/lib/smallwebwaf`, so that bans and history survive a deploy. For +example, with no setting: -``` -env-file path ban (?i)^/\.env(\.[a-z]+)?$ -scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b +```sh +docker run --detach --name app \ + --volume app-smallwebwaf:/var/lib/smallwebwaf \ + /app@sha256: ``` -[`SPEC.md`](SPEC.md) has the full design: the deployment, every environment -variable, the ban rules, the rule file format, the state files, the log fields, -the metrics, failure behaviour and the build order. +With no setting, the address of every new visitor is sent to GeoJS, a web +service, to find its country and network. `SWWAF_LOOKUP_SOURCE=off` sends +nothing. `SWWAF_LOOKUP_SOURCE=file`, with `SWWAF_LOOKUP_DB_PATH` naming a +database file on your own host, looks addresses up in that file (see "Lookup of +AS number and country" in [`SPEC.md`](SPEC.md)). -## Country and AS number lookup +`deploy/example-app` is such an app. `make example-app` builds it and checks it +(see "Checking that it works" below). To run `smallwebwaf` outside the image +while working on it, see "Entrypoints" below. -`smallwebwaf` looks up the AS number and country of every client through GeoJS, -a free web service that needs no account and no file, for the request log, the -client's history, the notes of its bans, their alerts and the metrics, and for -the country lists and the anomaly thresholds per AS number when you set them. -This means that GeoJS is told the address of every new visitor, whether or not a -setting uses the answer, unless you set `SWWAF_LOOKUP_SOURCE=off`. The only -visitors it is not told about are those in `SWWAF_ALLOW_NETS` or -`SWWAF_DENY_NETS`, those whose netblock a ban covers, and those on a private, -loopback or link-local address. An IPv6 visitor is asked about by the first -address of its IPv6 group. Each answer is kept for seven days, in memory and in -`lookups.json`, so that it survives a restart, and a visitor whose answer is -kept is not asked about again. +## What happens to a request -A request waits for its client's first answer only while a setting acts on it -before the request goes on: a country list, `SWWAF_ADD_LOOKUP_HEADERS`, or a -biased threshold that lowers a limit. A new visitor then waits up to -`SWWAF_LOOKUP_TIMEOUT`, a second by default, and without an answer counts as -coming from an unknown country until the answer arrives. Otherwise no request -waits: it goes on at once and is logged without the answer, which reaches the -client's history and the notes of its bans when it comes. The addresses waiting -are asked about together, up to 200 in one request, one request at a time; at -most 10,000 visitors wait, and one more is not asked about until there is room, -counting meanwhile as coming from an unknown country. GeoJS publishes no rate -limit but may block a caller it thinks asks too much. While GeoJS fails, -visitors with a kept answer are unaffected and new ones count as coming from an -unknown country, which `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses, and whose -limits `SWWAF_UNKNOWN_LIMIT_PERCENT` sets. GeoJS is then left alone for a -second, twice as long after each further failure up to five minutes, and asked -again by the next request from a visitor without an answer. +Two terms recur below. A clear sign of attack is a request that matches a rule +whose action is `ban`, or asks for a trap path (steps 9 and 10). An offence is a +request that counts against its client in its history. It is one of: -To keep your visitors' addresses on your own host, set -`SWWAF_LOOKUP_SOURCE=off`, or use the database file instead of GeoJS: -`SWWAF_LOOKUP_SOURCE=file` looks every client up in the free IPinfo Lite -database (`ipinfo_lite.mmdb`), at once, with no wait and nothing sent off the -host. A client whose address it does not hold counts as coming from an unknown -country. You download the database with your own IPinfo account, mount the -directory that holds it into the container, point `SWWAF_LOOKUP_DB_PATH` at the -file and refresh it when you choose; `smallwebwaf` never downloads it itself. It -reads the whole file into memory at start, and a file that is missing or that it -cannot read stops the start. It reads the file again once it has gone 2 seconds -without a change after you replace it, so that a file still being copied in is -read only once whole, and also 2 seconds after it starts watching, so that a -file replaced while it started is not missed. A replacement it cannot read is -logged and sent as a `file_error` alert, and the file read before stays in use. -It has to be the directory rather than the file itself that you mount: docker -does not show a single mounted file being replaced, so a refresh would go -unseen. IPinfo releases it under the Creative Commons Attribution-ShareAlike 4.0 -International License and asks for attribution, in its own words on -https://ipinfo.io/lite: "The attribution requirements can be met by giving our -service credit as your data source. Simply place a link to IPinfo on the -website, application, or social media account that uses our data." Its example -of such a credit is a link mentioning "IP address data is powered by IPinfo". A -service that uses the database through `smallwebwaf` should carry that link. +- a clear sign of attack; +- a broken limit; +- a refusal by a rule, by the Core Rule Set or for a missing or wrong token. -Neither source can place a private address, so a client on one, such as a -visitor on your local network, another container or your monitoring, has no -country: `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless you list it in -`SWWAF_ALLOW_NETS`, `SWWAF_DENIED_COUNTRIES` does not refuse it, and -`SWWAF_UNKNOWN_LIMIT_PERCENT` sets its limits. Such addresses are never sent to -GeoJS. +`smallwebwaf` handles each request in this order. A request one step refuses +skips the steps after it, apart from step 13. Nothing of a refused request +reaches the app. "What happens to a request" in [`SPEC.md`](SPEC.md) gives each +step in full. -## Blocklists +1. `GET /_smallwebwaf/healthz` is answered `200` with `ok` at once, before any + check and without asking the app. +2. The client is worked out. A netblock is a range of addresses, written as an + address and a prefix length: `203.0.113.0/24`. `SWWAF_TRUSTED_PROXIES` lists + the netblocks traefik comes from, by default the private address ranges. For + a TCP peer inside them, `X-Forwarded-For` is read from the right, and the + first address outside those netblocks is the client. A peer outside them is + the client itself. An IPv6 client is its group: the whole netblock around its + address, a /64 by default; `SWWAF_IPV6_GROUP_PREFIX` sets its length. +3. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 11, the byte limits and the + error burst, but still needs a token at step 12. A client in + `SWWAF_DENY_NETS` is refused. +4. A client under a ban is refused. If the ban was made for a clear sign of + attack, this request makes it permanent. +5. The client's AS number and country are looked up, unless + `SWWAF_LOOKUP_SOURCE` is `off`. An AS number names the network an address + belongs to, such as `AS64496`. +6. The country lists are checked. A client from a country in + `SWWAF_DENIED_COUNTRIES` is refused. When + `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any other country + is refused, and so is a client without a country. Such a refusal is not an + offence. +7. The reputation sources are checked: the blocklists, the CrowdSec decision + list, the DNS blocklists and AbuseIPDB. A client on the CrowdSec decision + list is banned until the decision ends. Any other source refuses the client, + lowers its limits or only notes it, as `SWWAF_BLOCKLIST_ACTION` and + `SWWAF_REPUTATION_ACTION` say. No request waits for a query. +8. Rate limiting counts the request, unless the client or the path is exempt. A + request that takes the client over a limit is refused and bans the client. +9. A request for a path in `SWWAF_TRAP_PATHS` is refused and bans the client, as + a clear sign of attack. +10. The rule files are checked. A `log` rule notes the match. A `block` rule + refuses the request. A `ban` rule refuses it and bans the client, as a clear + sign of attack. +11. The Core Rule Set inspects the request's method, URL and headers, and, once + `SWWAF_WAF_BODY_LIMIT` is set, form, JSON and XML bodies. A request it takes + for an attack is refused while `SWWAF_WAF_MODE` is `block`, the default, and + let through while it is `detect`. It bans no one by itself. +12. A request under `/_smallwebwaf/` is answered here by `smallwebwaf`, after + its token is checked. Any other request is passed to the app. The app's + answer is streamed back within the size and time limits. +13. Once the answer has ended, a request passed to the app has its bytes counted + for the byte limits. If a rule, a trap path, the Core Rule Set or a token + check refused the request, the refusal is counted for the error burst. The + client's history and the anomaly counters are updated. More than + `SWWAF_ERROR_BURST_THRESHOLD` refusals of one client in a minute ban it. -`SWWAF_BLOCKLIST_URLS` names blocklists: text files of addresses and netblocks, -one to a line, written as the Spamhaus DROP list, -`https://www.spamhaus.org/drop/drop.txt`, is. Anything after a `;` or a `#` on a -line is left out, and so is a line left blank. A bare address stands for itself -alone, as in the settings. An IPv4-mapped address or netblock, such as -`::ffff:192.0.2.0/120`, is read as the IPv4 one it stands for, here -`192.0.2.0/24`, since a client's IPv4 address is checked as IPv4; a mapped -netblock shorter than `/96` stands for none, and is not a netblock. None is -named by default: a list judges a client by what others saw it do, while the -defaults judge it by what it does to your service. +In `observe` mode (`SWWAF_MODE=observe`), `smallwebwaf` makes no ban of its own. +It refuses a request only for a size or time limit, or for a missing or wrong +token. A request it would otherwise have refused is passed to the app, and its +log line says what `enforce` mode would have done. Observe mode is for trying a +configuration before enforcing it. -`smallwebwaf` fetches each list, and the file `SWWAF_ASN_LIMIT_PERCENT_URL` -names, `SWWAF_BLOCKLIST_REFRESH` after it last fetched it or tried to, 24 hours -by default and never less than one, one list after another. Since -`reputation.json` keeps when each list was last tried, the fetch failed or not, -even one cut off as `smallwebwaf` stopped, whose request the server may have -had, at start `smallwebwaf` fetches at once only a list it has never tried, and -one it last tried that long ago; any other waits its turn, so that restarts do -not fetch a list more often. A fetch fails when the server answers other than -`200`, when it does not finish within a minute, when the list is longer than 16 -MiB, or when a line of it is not an address or a netblock, or for -`SWWAF_ASN_LIMIT_PERCENT_URL`, not an AS number, `:` and a percentage. The copy -fetched before then stays in use, and the failure is counted, logged and raised -as a `source_failure` alert; a fetch cut off as `smallwebwaf` stops is not a -failure. The last good copy of each list is kept whole, comment lines included, -in `reputation.json` (see "State files" above), so that a restart keeps it in -use too. Each list is named by its URL, in the request log, the alerts, the -notes of bans and the metrics, so keep a secret out of it. +What a refused visitor sees: -A client in `SWWAF_ALLOW_NETS` is not checked. Any other is checked by its own -address after the country lists, and `SWWAF_BLOCKLIST_ACTION` says what is done -with one a list lists, as "What it does so far" above describes. `deny` suits -lists of networks that send nothing legitimate, such as DROP; `limit:` -suits lists of addresses shared with ordinary visitors, such as those of Tor's -exits. +- `SWWAF_BAN_RESPONSE`, by default `403` with the plain text `Forbidden`. It can + be `429` instead, or `close`, which closes the connection, so that traefik + answers `502`. It answers: + - a ban or a broken rate limit; + - a `ban` rule or a trap path; + - `SWWAF_DENY_NETS` and the country lists; + - a reputation source whose action is `deny`. +- `403` for a `block` rule and the Core Rule Set. +- For a size or time limit: + - `431` for headers over their limit; + - `413` for a body over its limit; + - `408` for a client too slow to send its body; + - `502` or `504` when the app passes a limit or cannot be reached. +- `401` from an endpoint of `smallwebwaf`'s own, for a request without its + token. -The Spamhaus DROP list is The Spamhaus Project's, https://www.spamhaus.org. Its -terms, on its DROP page, ask that a product using it credit The Spamhaus Project -and keep the list's date and copyright lines with the data, which the copy in -`reputation.json` does; a service that uses DROP through `smallwebwaf` uses data -from The Spamhaus Project, and should say so. They also ask that it be fetched -automatically no more than once an hour, once a day being more than enough in -most cases, and Spamhaus may block an address that fetches it more often. Each -`smallwebwaf` fetches its own copy, on its own schedule, so on a host where -several apps run it, sharing one address, their fetches can come less than an -hour apart whatever `SWWAF_BLOCKLIST_REFRESH` is: each fetches a list again that -long after its own last try, so those first started within the same hour, with -the same refresh, keep fetching within the same hour. +## Settings you are likely to touch -## DNS blocklists +Each setting is an environment variable of the app's container, such as +`docker run -e SWWAF_MODE=observe ...`, read at start. +[`docs/settings.md`](docs/settings.md) lists every setting, its default and the +values it takes. The ones most deployments touch: -`SWWAF_DNSBL_ZONES` names DNS blocklists, DNSBL zones, which list an address by -answering a query for a name made from it. None is named by default, for the -reason none of the blocklists is. `smallwebwaf` asks each zone about a client's -own address in the background, the first time it sees the client: the request -goes on at once, as from a client the zone does not list, and so does every -request from it until the zone has answered. The name asked about is the one RFC -5782 gives: the four numbers of an IPv4 address in reverse order, so that -`192.0.2.99` is asked about in `dnsbl.dronebl.org` as -`99.2.0.192.dnsbl.dronebl.org`, or the 32 hex digits of an IPv6 address in -reverse order, each followed by a dot. An IPv6 client is asked about by its own -address, not by its IPv6 group. +- `SWWAF_MODE=observe`: log what would have been refused, refuse almost nothing, + ban no one. For trying a configuration. +- `SWWAF_ALLOW_NETS`: netblocks you trust, such as your monitoring. Their + clients are refused only for a size or time limit or a missing or wrong token. +- `SWWAF_RATE_LIMIT_EXEMPT_NETS`: netblocks exempt from rate limiting and the + byte limits, such as a machine that talks to the app all day. +- `SWWAF_TRAP_PATHS`: paths only scanners ask for, such as + `/wp-login.php,/xmlrpc.php` in front of a code forge. A request for one bans + the client. +- `SWWAF_WAF_DISABLED_RULES`: Core Rule Set rule ids to switch off when a rule + refuses your app's ordinary requests. The log line names the rules a request + matched. Keep the six default ids in a list you give (see + [`docs/settings.md`](docs/settings.md)). +- `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and + `SWWAF_UPSTREAM_REQUEST_TIMEOUT`: raise them for an app that takes uploads + over 100 MiB or slower than 60 seconds. +- `SWWAF_LOOKUP_SOURCE`: `geojs` (the default), `file` or `off`, as above. +- `SWWAF_ADMIN_TOKEN` and `SWWAF_METRICS_TOKEN`: switch on the ban endpoints and + the metrics. Give them as files to keep them from the app. +- `SWWAF_ALERT_WEBHOOK_URL`, `SWWAF_ALERT_SLACK_WEBHOOK_URL` or + `SWWAF_ALERT_NTFY_URL`: where alerts go. +- `SWWAF_INSTANCE_NAME`: the name in every log line, alert and metric, such as + `fsn1app1/gitea`. -A zone that answers that the name does not exist, or has no address, does not -list the client, and one that answers with an address in `127.0.0.0/8` lists it. -Any other answer gives no verdict, and is a failure: an address in -`127.255.255.0/24`, with which Spamhaus refuses a query, such as one sent -through a public resolver or one past its limit; an address outside -`127.0.0.0/8`, such as a resolver gives that answers even for names that do not -exist; an error the resolver answers with, such as a refusal; and no answer -within `SWWAF_REPUTATION_TIMEOUT`. A failure is counted, logged and raised as a -`source_failure` alert, held back as a repeat within `SWWAF_ALERT_COOLDOWN`, and -the zone is not asked again for a minute, so that a zone that refuses queries is -not asked on every request. +## Checking that it works -Each verdict, the zone listing the client or not, is kept, in memory and in -`reputation.json` (see "State files" above), so that a restart keeps it, and is -used for `SWWAF_REPUTATION_CACHE_TTL` after the zone gave it, 24 hours by -default. The client's first request after that has the zone asked again, and -until it answers, no verdict of the zone applies to the client. At most 100,000 -verdicts are kept, the one fetched longest ago dropped first, and at most 1,000 -queries are under way at once; past that, a zone is asked about a client at the -client's next request. - -A client in `SWWAF_ALLOW_NETS` is not checked, nor is one a blocklist refuses. -`SWWAF_REPUTATION_ACTION` says what is done with one a zone's verdict lists, as -"What it does so far" above describes. - -Do not name a zone meant for mail, such as one of residential and dynamic -address ranges, which ordinary visitors come from, or one that includes such a -list, as Spamhaus's `zen` does: it would refuse them, or lower their limits. -Spamhaus's zones answer through its keyed query service, named with the key in -it, such as `.xbl.dq.spamhaus.net`. The key of a zone under -`dq.spamhaus.net` is its first label, and `smallwebwaf` shows `********` in its -place wherever it names the zone, as `********.xbl.dq.spamhaus.net`: in the -settings logged at start, an error that stops the start, its own messages, the -request log, the alerts, the notes of bans and the metrics. Only -`reputation.json` keeps the zone with its key. A key in the name of any other -zone is shown as given. Several zones refuse queries that come through a public -resolver; `SWWAF_DNSBL_RESOLVER` names another resolver to ask through. - -## AbuseIPDB - -While `SWWAF_ABUSEIPDB_KEY` holds the key of an AbuseIPDB account, `smallwebwaf` -asks AbuseIPDB's check endpoint, `https://api.abuseipdb.com/api/v2/check`, for -the abuse confidence score of a client's own address, from 0 to 100. It is unset -by default, for the reason no blocklist is named, and since AbuseIPDB needs an -account. An IPv6 client, an IPv6 group, is checked by the address of the request -that has it checked, and its score is used for the whole group, whichever of its -addresses sends, so that one client costs at most one check every -`SWWAF_REPUTATION_CACHE_TTL`. - -Only a client whose history counts an offence is checked, so that the checks are -spent on suspects: so far, one that has broken a rate limit, a byte limit or the -error burst, matched a ban rule, asked for a trap path, or had a request refused -by a block rule, by the Core Rule Set or for a missing or wrong token. A client -dropped from the table of clients loses its history, and with it its offences. A -client is checked in the background, at its first request after its offence that -reaches the check: no request waits, a request refused under its ban is not -checked, and the request that has it checked, and any other from it before the -answer comes, goes on as from a client without a score. - -A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE`, 75 by default, is a hit, and -`SWWAF_REPUTATION_ACTION` says what is done with its client, as for a DNSBL -zone's verdict (see "What it does so far" above). Each score, a hit or not, is -kept, in memory and in `reputation.json` (see "State files" above), and used for -`SWWAF_REPUTATION_CACHE_TTL` after AbuseIPDB gave it, 24 hours by default. The -client's first request after that has it checked again, if its history still -counts an offence. At most 100,000 scores are kept, the one fetched longest ago -dropped first. - -At most `SWWAF_ABUSEIPDB_DAILY_BUDGET` checks are made in a day, 900 by default, -below the 1,000 of AbuseIPDB's free accounts. The day is counted in UTC, from -00:00. Each check sent spends one of them, whatever AbuseIPDB answers, and the -checks spent today are kept in `reputation.json`, so that a restart does not -make the budget whole again. The check that spends the last of the day's budget -is logged and raised as a `source_failure` alert; from then until the day ends, -no client is checked, and a client without a score counts as one AbuseIPDB does -not list. - -A check fails when AbuseIPDB answers other than `200`, such as `429` past its -own daily limit or `401` for a wrong key, when its answer gives no -`abuseConfidenceScore`, and when it does not answer within -`SWWAF_REPUTATION_TIMEOUT`. A failure gives no score, and is counted, logged and -raised as a `source_failure` alert, held back as a repeat within -`SWWAF_ALERT_COOLDOWN`, and no client is checked for a minute after it. - -The key is sent to AbuseIPDB in the `Key` header, and nowhere else. The settings -logged at start show `********` in its place, and neither the log, the alerts, -the metrics nor `reputation.json` hold it. Given as a file, with -`SWWAF_ABUSEIPDB_KEY_FILE`, it can be kept out of the app's reach (see "Settings -given as files" above). - -## CrowdSec - -While `SWWAF_CROWDSEC_LAPI_URL` names the local API of a CrowdSec engine you -run, `smallwebwaf` fetches the engine's decision list, at that URL with -`v1/decisions` added to its path, with `SWWAF_CROWDSEC_LAPI_KEY`, a key the -engine gave, such as `cscli bouncers add smallwebwaf` makes. This is how a list -kept for a whole fleet of hosts can reach `smallwebwaf` without it depending on -CrowdSec. It is unset by default, for the reason no blocklist is named, and -since it needs an engine of your own. - -The list is fetched again a minute after it was last fetched or tried, the fetch -failed or not, and is kept as a blocklist is (see "Blocklists" above): its last -good copy, the engine's answer as it came, stays in use while a fetch fails, and -`reputation.json` keeps it with the time it was fetched, so that a restart keeps -it in use too. A fetch fails when the engine answers other than `200`, a -redirect included, such as `403` for a key it does not know, when it does not -finish within a minute, when the answer is longer than 16 MiB or is not a JSON -list of decisions, or when a decision to ban gives a value that is not an -address or a netblock, or a `duration` that does not read. A failure is counted, -logged and raised as a `source_failure` alert, held back as a repeat within -`SWWAF_ALERT_COOLDOWN`. - -The decisions of the type `ban` on an address or a netblock, the scopes `Ip` and -`Range`, are used; any other, such as one to show a captcha or one on a country, -is left out. A decision bans until its `duration`, the time it had left as the -engine answered, has passed since the fetch, so that one that has ended no -longer bans, even while the copy in use still holds it because the engine cannot -be reached. - -A client in `SWWAF_ALLOW_NETS` is not checked, nor is one a blocklist refuses. -Any other is checked by its own address after the blocklists. A request from a -client a decision in force bans is refused with `SWWAF_BAN_RESPONSE`, and bans -the client's netblock, as any ban covers it, until that decision ends, the one -that ends last if several ban the client. The ban's cause is `crowdsec`, and its -reason names the decision's scenario. Only a client that sends a request is -banned, so that a long list does not fill `bans.json`. Such a ban counts toward -`SWWAF_MAX_BANS`, is never made permanent, and does not make the netblock's next -ban for a broken limit longer. The request's log line and the ban's notes name -the list by its URL, as they name a blocklist, and the request raises a -`reputation_hit` alert besides the ban's. - -A ban you lift while its decision is in force is followed by another at the -client's next request. To let a client in, delete its decision in CrowdSec, such -as with `cscli decisions delete --ip 203.0.113.9`, and lift the ban once the -list has been fetched again, a minute or so later. A ban made for a decision -deleted in CrowdSec otherwise lasts until that decision would have ended. - -The key is sent to the engine in the `X-Api-Key` header, and nowhere else. The -settings logged at start show `********` in its place, and neither the log, the -alerts, the metrics nor `reputation.json` hold it. Given as a file, with -`SWWAF_CROWDSEC_LAPI_KEY_FILE`, it can be kept out of the app's reach (see -"Settings given as files" above). - -## How the code is laid out - -- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`. -- `internal/smallwebwaf`: the process: it reads the settings, the rule files, - the lookup database and the state files, listens, serves requests until - `SIGTERM` or `SIGINT`, and stops, writing the state files. Run as - `smallwebwaf healthcheck`, it is the image's health check instead. -- `internal/config`: reads the settings, the one place they are read. -- `internal/proxy`: what happens to each request: it works out the client, runs - the checks, passes the request to the app and the answer back with the - standard library's `httputil.ReverseProxy` within the timeouts and size - limits, and writes the request's log line. Its `check` method is where a - request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for - a ban, for the country lists, for a blocklist, for the CrowdSec decision list, - which bans the client, for a DNSBL zone's verdict, for AbuseIPDB's score, for - a rate limit, which bans the client, for a trap path, which bans the client, - for a `block` or `ban` rule, the latter banning the client, for a Core Rule - Set match in `block` mode, for a body that passes the size limit or - `SWWAF_CLIENT_REQUEST_TIMEOUT` while the Core Rule Set reads it, and for an - announced body over the size limit; in `observe` mode, only for the last two, - with what it would have refused for noted in the log line. A request under - `/_smallwebwaf/` that `check` lets through is answered by `answerAdmin` - instead of reaching the app. Once the answer to a request passed to the app - has ended, `countBytes` counts its bytes for the byte limits, and once any - request but the health check has ended, `countRefusal` counts it for the error - burst if it was refused after a rule file match, a trap path or a Core Rule - Set match or for its token, and `countAnomalies` counts it for the anomaly - thresholds. -- `internal/metrics`: the metrics, counted as the other parts tell it what - happened, and served in the Prometheus text format. -- `internal/bans`: the ban ledger: each netblock's bans with their notes, how - long a new ban lasts, when a ban for a clear sign of attack becomes permanent, - and which ban `smallwebwaf` made is dropped when `SWWAF_MAX_BANS` are held. -- `internal/rules`: reads the rule files at start and again as they change, and - tells which of their rules a request matches. -- `internal/waf`: the Core Rule Set with the six changes, as Coraza's own - directives, and what it finds in a request's method, URL, headers and the part - of its body it reads: the rules that matched and the anomaly score. -- `internal/lookup`: looks up each client's AS number and country through GeoJS, - keeps the answers, and hands each new one to the proxy, which adds it to the - client's history and to the notes of its bans; or in the lookup database, - which it reads again when the file is replaced. `internal/lookup/lookuptest` - writes lookup databases for the tests. -- `internal/reputation`: fetches the blocklists, the file - `SWWAF_ASN_LIMIT_PERCENT_URL` names and the CrowdSec decision list as they are - due, keeps the last good copy of each, and tells which blocklists list an - address, what percentage the file gives an AS number, and which decision of - the decision list bans an address; asks the DNSBL zones about clients in the - background, through the standard library's resolver, keeps their verdicts, and - tells which zones' verdicts list an address; and checks clients with AbuseIPDB - in the background, keeps their scores and the checks spent today, and tells - whether a client's score is a hit. -- `internal/ratelimit`: the table of clients: counts each client's requests, - bytes and refusals, tells when they take it over a rate limit, a byte limit or - the error burst, and keeps each client's history. -- `internal/anomaly`: the anomaly counters: counts each request and its bytes - per client, per netblock around a client, per AS number, for the whole service - and per named netblock, in the buckets `internal/ratelimit` counts in, and - raises an `anomaly` alert for a count over its threshold. -- `internal/state`: reads the state files at start, takes in an admin's edit of - one while running, and writes them when they are due and at the stop. -- `internal/requestlog`: the lines on stdout: the request log line and the - process's own messages. -- `internal/remotelog`: sends the lines on stdout to `SWWAF_LOG_REMOTE_URL`, - each as a syslog record, from a buffer of its own. It is written with the - standard library alone, whose `log/syslog` writes only the older syslog - format. -- `internal/alerts`: takes the alerts the other parts raise, holds back repeats - and those past the hourly limit, and sends the others to the webhook, Slack - and ntfy, each from a queue of its own. -- `Dockerfile`: the lint and test phases, the stages `script/tidy` builds, then - the image, whose last stage installs Ubuntu's packages, nixpkgs, `runsvinit` - and `smallwebwaf`, with `share/smallwebwaf.run` as runit's `run` script for - `smallwebwaf` and `share/rules.d/00-default.rules` as its default rule file. -- `deploy/example-app`: an app built on the image, which `script/example-app` - checks. - -Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the -table of clients to `SWWAF_MAX_TRACKED_CLIENTS` and the GeoJS answers to -100,000, dropping the least recently seen, the DNSBL zones' verdicts and -AbuseIPDB's scores to 100,000 each, dropping the one fetched longest ago, the -anomaly counters to 20,000, dropping the one counted least recently, and the -banned netblocks in the order they were last seen, from which the ledger picks -the ban to drop past `SWWAF_MAX_BANS`, and `github.com/prometheus/client_golang` -keeps the metrics and serves them, and `github.com/fsnotify/fsnotify` tells -`smallwebwaf` when a state file or a rule file is saved, or the lookup database -replaced, and `github.com/oschwald/maxminddb-golang/v2` reads the lookup -database, which the tests write with `github.com/maxmind/mmdbwriter`, and -`github.com/corazawaf/coraza/v3` runs the Core Rule Set 4.25.0, which -`github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0` carries. The country -codes are the list in `internal/config/config.go`. +- `docker inspect --format '{{.State.Health.Status}}' ` says + `healthy` once `smallwebwaf` answers and the app accepts connections. A log + line with `"msg":"starting"` gives the version, the address and every setting + in effect, with secrets masked. +- A request to the site through traefik gets the app's answer, and its log line + has `"action":"forward"`. `https:///_smallwebwaf/healthz` answers `ok`. +- To see a refusal without banning yourself, run the container with + `SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app. Its log + line has `"would_action":"banned"` and `"rule_ids":["env-file"]`. In `enforce` + mode the same request bans your address for seven days, and your next request + makes the ban permanent. Lift the ban by editing `bans.json` (see "Bans" in + [`SPEC.md`](SPEC.md)). +- With `SWWAF_METRICS_TOKEN` set, + `curl -H "Authorization: Bearer $TOKEN" https:///_smallwebwaf/metrics` + answers with the metrics. +- `make example-app` builds the image and the example app on it, runs the app, + and checks it. It needs network access. It checks: + - the health check; + - a request forwarded to the app; + - a ban for a broken limit, and a ban for a probe made permanent; + - orderly stops with `sv stop` and `docker stop`; + - bans surviving a new container on the same volume; + - a setting given as a file. ## Entrypoints This repository adheres to the [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) -standard: the scripts in `script/` are the entrypoints for working on it, and -the `Makefile` targets are thin shims that call them. The scripts are POSIX sh, -so that they run in minimal containers. +standard. The scripts in `script/` are the entrypoints for working on it, and +the `Makefile` targets are thin shims that call them. The scripts are POSIX sh. +The tests and the linter run in Docker, as phases of the `Dockerfile`, and never +on the host. -- `script/bootstrap`: installs what the other scripts need on the host: `make`, - `git`, `curl`, Go for `gofmt`, node and yarn, and prettier. -- `script/setup`: readies a fresh clone: runs `script/bootstrap`, then - `script/install-precommit`. -- `script/projectname`: prints the project's name, `smallwebwaf`, which - `script/docker` and the others tag their images with. -- `script/test`: checks that `go.mod` and `go.sum` are as `go mod tidy` writes - them, then runs the tests, as the `test` phase of the `Dockerfile`. -- `script/tidy`: writes `go.mod` and `go.sum` as `go mod tidy` writes them, with - the Go of the `test` phase, in the `tidy` stage of the `Dockerfile`; - `make tidy` runs it. +- `script/bootstrap`: installs on the host what the other scripts need, with + `make bootstrap`: + - `make`, `git` and `curl`; + - Go, for `gofmt`; + - node, yarn and prettier. +- `script/setup`: readies a fresh clone: `script/bootstrap`, then + `script/install-precommit`. `make setup`. +- `script/projectname`: prints `smallwebwaf`, which the other scripts tag their + images with. +- `script/test`: checks that `go.mod` and `go.sum` are tidy, then runs the + tests, as the `test` phase of the `Dockerfile`. `make test`. +- `script/tidy`: writes `go.mod` and `go.sum` as `go mod tidy` writes them, in + the `tidy` stage of the `Dockerfile`. `make tidy`. - `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`. + `make lint`. - `script/fmt`: formats the Go code with `gofmt` and the Markdown with prettier. -- `script/fmt-check`: checks the formatting, and changes nothing. + `make fmt`. +- `script/fmt-check`: checks the formatting and changes nothing. + `make fmt-check`. - `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`. + `make check`. - `script/docker`: builds the image, whose build runs the tests and the linter - first. + first. `make docker`. - `script/cibuild`: what CI runs: `script/bootstrap`, `script/check`, then the image build. - `script/precommit`: run by the git pre-commit hook; runs `script/check`. -- `script/install-precommit`: installs that hook; `make hooks` runs it. -- `script/build`: builds `bin/smallwebwaf` on the host, with Go installed, for - working on the code by hand; `make build` runs it. -- `script/run`: builds `bin/smallwebwaf` with `script/build` and runs it, with - its state files in `bin/state` unless `SWWAF_STATE_DIR` is set, and the rule - files of `share/rules.d` unless `SWWAF_RULES_DIR` is set; `make run` runs it. -- `script/example-app`: builds the image and, on it, the example app in - `deploy/example-app`, runs it with a volume for the state files, and checks - that the health check passes, that a request reaches the app through - `smallwebwaf`, that a second request in a minute bans the client, that a probe - for `/.env` bans another client, whose next request makes the ban permanent, - that `sv stop` and `docker stop` stop it in order, that a new container on the - same volume still refuses the banned client, that a container with its state - files on a new volume mounted at the directory `SWWAF_STATE_DIR_FILE` names - becomes healthy, and that one whose `SWWAF_STATE_DIR` does not exist logs - `smallwebwaf`'s message about it; then removes the containers, the volumes and - both images. It needs network access, for nixpkgs' binary cache, and - `script/check` does not run it; `make example-app` does. -- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it, - with every setting of `smallwebwaf` at its default but `SWWAF_UPSTREAM_URL`, - which names gitea's container. Of the requests that three of the notes - specific to gitea in [`SPEC.md`](SPEC.md) say are refused, it sends through - it: for each file the note on a file name in a query parameter names, the - API's list of its commits; for `Runtime.java` alone, the commit graph of that - file, the hidden lines of its diff and the API's call for the contents of - files that names it; the sign-in page gitea sends a visitor to on the way to - the edit page of `document.write.js`; the pages of `java.lang.String.md`, - `META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts` - and `.../notes.txt`; and one container image push that reuses a layer. It does - not send the raw file, history, blame or edit page of those four files, the - pages of the directories `META-INF/services` and `...`, or the push again. It - also sends a short sample of gitea's ordinary use: signing in, pages of files - and commits, a diff, the API, a push, clone and fetch over HTTP, and two - container image pushes. It prints a line for each request `smallwebwaf` - logged, with what it did with it and the ids of the Core Rule Set's rules that - matched it, then removes the containers, the network and the images it made, - but gitea's own image. It needs network access, and a docker daemon that - pushes to a registry on `localhost` over plain HTTP, as docker does by - default; `script/check` does not run it, and `make gitea-requests` does. +- `script/install-precommit`: installs that hook. `make hooks`. +- `script/build`: builds `bin/smallwebwaf` with the Go on your host. + `make build`. +- `script/run`: builds `bin/smallwebwaf` and runs it with the settings in your + environment, listening on port 8080 in front of an app at + `SWWAF_UPSTREAM_URL`. Its state files go in `bin/state` and its rule files are + those of `share/rules.d`, unless `SWWAF_STATE_DIR` and `SWWAF_RULES_DIR` say + otherwise. `make run`. +- `script/example-app`: builds the image and the example app on it, runs it with + a volume, and checks it as "Checking that it works" above describes. Then it + removes the containers, volumes and images. Needs network access. + `make example-app`. +- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it at + the defaults. It sends through it a sample of gitea's ordinary use and the + requests the Core Rule Set is known to refuse, and prints what `smallwebwaf` + did with each. Needs network access, and a docker daemon that pushes to a + registry on `localhost` over plain HTTP. [`docs/gitea.md`](docs/gitea.md) + records the results. `make gitea-requests`. + +## Rationale + +Small self-hosted sites receive a great deal of traffic nobody asked for: + +- scrapers that ignore `robots.txt` and crawl every commit of every repository + on a public git server; +- scanners walking through lists of WordPress and `.env` paths; +- credential-guessing bots. + +Most of it comes from a small number of hosting networks and countries. A +single-person operation has no abuse desk and no CDN contract. It needs +something small that can be put in front of one service and left alone. + +The existing tools each solve part of this. Rule-based firewalls catch attack +payloads but do not limit request rates. Rate limiters count requests but cannot +tell a residential visitor from a rented server farm. The products that do most +of it want several containers, a database and a web console. None of them can +say "clients from these networks are banned after half as many requests as +anyone else". That is the useful thing to say when nearly all abuse comes from a +known list of AS numbers. [`EVALUATION.md`](EVALUATION.md) goes through the +candidates one by one. + +`smallwebwaf` fills that gap with: + +- per-client limits that ban; +- lower limits for the networks and countries abuse comes from; +- bans that grow with repetition; +- a log that is easy to search and ship; +- one binary in the app's own container, with no database and no required + setting. + +## Design + +The `smallwebwaf` image is built on Ubuntu 26.04 LTS, pinned by digest, with +nixpkgs fixed at one release. An app's `Dockerfile` installs a package with +`nix-env -iA nixpkgs.` and gets the same package each time it is built. +[`docs/image.md`](docs/image.md) lists what the image holds and how each part is +pinned. + +- The container's first process is `runsvinit`, which starts runit, a service + supervisor. runit runs the `run` script of each directory under + `/etc/service`, and runs it again a second after its program exits. + `smallwebwaf` runs as the `smallwebwaf` user (uid 65532) and the app as a user + of its own. So the app cannot touch the state files or the process of + `smallwebwaf`. +- `smallwebwaf` listens on port 8080, on every address. traefik sends the app's + requests there. `smallwebwaf` passes them to the app on `127.0.0.1:8081`. The + app listens on `127.0.0.1` only, so nothing reaches it except through + `smallwebwaf`. The health check, the metrics and the ban management are on the + same port, under `/_smallwebwaf/`. A request for that prefix never reaches the + app. +- The app sees what it would see from traefik directly: the same `Host`, the + same `X-Forwarded-Proto`, and `X-Forwarded-For` with traefik's address added + at the end. The app's TCP peer is `smallwebwaf` on `127.0.0.1`. So the app + must trust `127.0.0.1` and `::1` for forwarded headers, besides the private + address ranges. An app that keeps its usual trusted proxies sees every visitor + as `127.0.0.1`. Each request also carries its id in `X-Request-ID`. Any + `X-Client-ASN` or `X-Client-Country` header a client sent is removed. +- Both processes start with the container's environment variables. That is why + every setting of `smallwebwaf` starts with `SWWAF_`. A token given as an + environment variable is readable by the app. A token given as a file that only + the `smallwebwaf` user can read is not (see "Settings given as files" in + [`docs/settings.md`](docs/settings.md)). +- The state files live in `/var/lib/smallwebwaf`. Without a volume there, they + survive a restart of the container and are lost when a deploy replaces it. The + `run` script gives that directory and every file in it to the `smallwebwaf` + user at each start, so a host directory mounted there needs no change of + owner. +- The image's health check passes while `smallwebwaf` answers + `GET /_smallwebwaf/healthz` and the app accepts connections. traefik sends a + container no requests until it is healthy. +- When `smallwebwaf` or the app exits, runit starts it again a second later. The + other process keeps running. While either is down, the client gets `502`. A + `smallwebwaf` that stops at start, as on an invalid setting, is started again + every second until the cause is fixed, and logs its message each time. +- `docker stop` has runit send `SIGTERM` to both processes at once. + `smallwebwaf` gives the requests in progress five seconds to finish and writes + its state files. To let them finish, the app must keep answering for five + seconds after `SIGTERM`. + +"Deployment" in [`SPEC.md`](SPEC.md) has the rest: moving `smallwebwaf` to +another port, the health check's timing, and the forwarded headers. +"Architecture" in [`SPEC.md`](SPEC.md) lists the packages and what each does. ## TODO -- The rest of the design, in the order of the build order in - [`SPEC.md`](SPEC.md). +- Planned, as "Planned, not built" in [`SPEC.md`](SPEC.md) describes: + - sharing bans between the instances of several apps; + - sending the log over RELP. ## Documents -- [`SPEC.md`](SPEC.md): the design. +- [`SPEC.md`](SPEC.md): how `smallwebwaf` behaves, and why. - [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers and misses, and why none was adopted. - [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows. +- [`docs/settings.md`](docs/settings.md): every setting, its default and the + values it takes. +- [`docs/rules.md`](docs/rules.md): the rule file format and the default file. +- [`docs/admin-endpoints.md`](docs/admin-endpoints.md): the endpoints under + `/_smallwebwaf/`, with examples. +- [`docs/request-log.md`](docs/request-log.md): the request log's fields. +- [`docs/alerts.md`](docs/alerts.md): the alert events and what each destination + gets. +- [`docs/metrics.md`](docs/metrics.md): the metrics. +- [`docs/state-files.md`](docs/state-files.md): the state files and what each + holds. +- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned. +- [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea. ## License -- 2.54.0 From a62909dbf4db9ca8dafbf5636ede4dfee9f73777 Mon Sep 17 00:00:00 2001 From: sneak Date: Thu, 22 Oct 2026 23:02:00 +0000 Subject: [PATCH 8/8] Last pass: the sixth README review and the fifth SPEC review The spec names every fault that stops the start, runs step 15 for every request but the health check, and states each of the fourteen repeated requirements once. docs/alerts.md says which alerts have an empty client and netblock and what a summary's detail holds; docs/settings.md lists the settings that need a lookup and points at step 3 for the two static lists. The README and the spec now travel in one branch, so the README never names a spec section that does not exist yet. Model: fable-5-1 --- README.md | 4 +- SPEC.md | 163 +++++++++++++++++++++++------------------------ docs/alerts.md | 9 ++- docs/metrics.md | 6 +- docs/settings.md | 16 +++-- 5 files changed, 100 insertions(+), 98 deletions(-) diff --git a/README.md b/README.md index 6292ef5..83adf48 100644 --- a/README.md +++ b/README.md @@ -150,8 +150,8 @@ request that counts against its client in its history. It is one of: - a broken limit; - a refusal by a rule, by the Core Rule Set or for a missing or wrong token. -`smallwebwaf` handles each request in this order. A request one step refuses -skips the steps after it, apart from step 13. Nothing of a refused request +`smallwebwaf` handles each request in this order. Once a step refuses a request, +the steps after it are skipped, apart from step 13. Nothing of a refused request reaches the app. "What happens to a request" in [`SPEC.md`](SPEC.md) gives each step in full. diff --git a/SPEC.md b/SPEC.md index 13fa229..df217d5 100644 --- a/SPEC.md +++ b/SPEC.md @@ -100,10 +100,12 @@ to them. ## What happens to a request The steps run in this order. The first step that produces a final answer ends -the checks; step 15 runs for every request. The size and time limits apply to -the whole exchange. Each step names the section that holds its rules. +the checks; step 15 runs for every request but the health check. The size and +time limits apply to the whole exchange. Each step names the section that holds +its rules. -1. `GET /_smallwebwaf/healthz` is answered at once, before any check. +1. `GET /_smallwebwaf/healthz` is answered at once, before any check, so a + health checker never needs an exemption and is never refused. 2. The client is identified. - If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, `X-Forwarded-For` is walked from the right, and the first address outside those netblocks is @@ -116,9 +118,9 @@ the whole exchange. Each step names the section that holds its rules. around its address, a /64 by default, since one abuser usually controls a whole /64. A client is therefore one IPv4 address or one IPv6 group. 3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12 and the - byte limits and the error burst at step 15. It is still counted for the - anomaly thresholds, and still needs a token at step 13. A client in - `SWWAF_DENY_NETS` is refused; a client in both lists is let through. + byte limits and the error burst at step 15. It still needs a token at + step 13. A client in `SWWAF_DENY_NETS` is refused; a client in both lists is + let through. 4. Bans. A client under a ban is refused (see "Bans"). 5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number and country"). The request goes on at once. It waits for the first GeoJS @@ -134,8 +136,6 @@ the whole exchange. Each step names the section that holds its rules. gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is banned. A client listed by a DNS blocklist, or scored at least `SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`. - With no cached verdict, a query is queued in the background and the request - goes on, so a first request is never delayed by a reputation query. 8. The client's limit percentage (see "Biased thresholds"). 9. Rate limiting (see "Request rate limits, per client"). A request over a limit times the percentage is refused and bans the client. @@ -184,7 +184,7 @@ the token check still apply. and the anomaly counters have fixed bounds, listed under "Fixed limits" in `docs/settings.md`. Past its bound, the GeoJS answer or anomaly counter used longest ago goes first, and the verdict or score fetched longest ago. Past - 1,000 DNS queries under way, a zone is asked at the client's next request. + that bound, a zone is asked at the client's next request. ## Bans @@ -193,8 +193,7 @@ Until a ban ends, every request from its netblock is refused with `SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one address or one /64, so a permanent ban does not reach neighbours who did nothing. Requests refused under a ban are not counted against limits. They are -counted in the client's history and in the ban's notes. A banned client is not -looked up. +counted in the client's history and in the ban's notes. An offence is a request `smallwebwaf` holds against the client. It is one of: @@ -367,8 +366,8 @@ Conventions: owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and byte limits only; the error burst, attack detection and bans still apply. - Many clients share one address (mobile carriers, offices, Tor), and so share - its limits and its bans. The defaults sit well above one person's use, and the - first two lists take known shared addresses out. + its limits and its bans. `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` + take known shared addresses out. ### Request rate limits, per client @@ -442,12 +441,8 @@ service that keeps its visitors' addresses on its own host. - Only one source is used. The start stops, naming both settings, when `file` is set without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other - source. The start also stops when the source is `off` and one of these needs - lookups: - - a country list; - - a biased threshold; - - `SWWAF_ADD_LOOKUP_HEADERS`; - - an anomaly threshold per AS number. + source. The start also stops when the source is `off` and a setting that needs + a lookup is set (`docs/settings.md` lists them). - GeoJS is told the address of every new visitor, whether or not a setting uses the answer, since the request log, the metrics and the ban notes carry it. These addresses are never sent: @@ -476,8 +471,8 @@ service that keeps its visitors' addresses on its own host. API requests", judged by GeoJS alone, and let it block a caller. While GeoJS is slow, down or refusing `smallwebwaf`, new clients count as unknown. GeoJS is then left alone for a second, and twice as long after each further failure, - up to five minutes, and one `source_failure` alert is sent per cooldown. A - service that cannot accept this uses the file. + up to five minutes. Each failure raises a `source_failure` alert (see + "Alerting"). A service that cannot accept this uses the file. - The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`, `as_name` and `country_code` fields are read. The operator downloads it with a free IPinfo account and mounts it read-only; `smallwebwaf` never fetches it @@ -486,9 +481,9 @@ service that keeps its visitors' addresses on its own host. on the site that uses it, such as "IP address data is powered by IPinfo". - The file is read whole at start; one that is missing or unreadable stops the start. It is read again once it has gone 2 seconds without a change after - being replaced. One that cannot be read is logged, sent as one `file_error` - alert and ignored. The database already loaded stays in use. A client whose - address the file does not hold counts as unknown. + being replaced. A replacement that cannot be read is logged, sent as one + `file_error` alert and ignored. The database already loaded stays in use. A + client whose address the file does not hold counts as unknown. ### Country lists @@ -508,9 +503,8 @@ where it comes from. exclusive list already refuses every other country. A code on both lists stops the start with a message naming it. - A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the - client's address has been looked up, before its body is read. It skips the - reputation checks, the limits, the rule files and the Core Rule Set. It is - logged with the action `country_denied`, and counted in the metrics. + client's address has been looked up, before its body is read. It is logged + with the action `country_denied`, and counted in the metrics. ### Biased thresholds @@ -519,10 +513,11 @@ countries. The lists are empty by default, for the same reason as the country lists. The request log and the metrics show which AS numbers and countries a service's abuse comes from. -- Clients in a listed AS number or country get that percentage of every request - and byte limit (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A - lowered limit is rounded down. `0` is a zero allowance: the client's first - request breaks a limit and bans it. +- Each list gives AS numbers or countries a percentage, such as `AS14061:50`. + Their clients get that percentage of every request and byte limit + (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A lowered limit is + rounded down. `0` is a zero allowance: the client's first request breaks a + limit and bans it. - `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` set the byte limits' percentage alone, when it should differ from the request limits'. For an AS number they list, they replace its percentage from @@ -556,9 +551,8 @@ code in them for attacks. The default gives up refusing an attack carried in a body. The rule files, the limits and the bans still apply to the client that sends it. -- The Core Rule Set inspects only a request that every earlier check let - through, and skips a path under `SWWAF_WAF_EXEMPT_PATHS`. An engine error on a - request forwards the request, and is logged and counted. +- The Core Rule Set skips a path under `SWWAF_WAF_EXEMPT_PATHS`. An engine error + on a request forwards the request, and is logged and counted. - `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule Set's own two tuning values, at its own defaults. Each level up runs more rules, which find more attacks and take more ordinary requests for attacks. @@ -601,14 +595,18 @@ sends it. percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and `/%77p-login.php`, and does not match `/wp-login.php/` or `/blog/wp-login.php`. -- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute for a rule - file match, a trap path, a Core Rule Set match, or a missing or wrong token; - no other refusal counts. More than this breaks a limit (see "Bans"). A client - trying one attack or token after another is refused many times a minute; a - person rarely more than a few times. Answers the app gives are not counted, - since they do not tell a scanner from an ordinary client. In front of gitea, - registry and git clients get `404` and `401` by design, hundreds of times a - minute. +- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute of these + kinds, and of no other: + - a rule file match; + - a trap path; + - a Core Rule Set match; + - a missing or wrong token. + + More than this breaks a limit (see "Bans"). A client trying one attack or + token after another is refused many times a minute; a person rarely more + than a few times. Answers the app gives are not counted, since they do not + tell a scanner from an ordinary client. In front of gitea, registry and git + clients get `404` and `401` by design, hundreds of times a minute. `smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of @@ -773,9 +771,8 @@ too close together. by default. Such verdicts are less certain than a blocklist, so a listed client gets a quarter of every request and byte limit rather than a refusal. - Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and - in `reputation.json`, with the fixed bounds under "Fixed limits" in - `docs/settings.md`. A source that is down or over quota gives no verdict. The - service continues, with one `source_failure` alert per cooldown. + in `reputation.json`. A source that is down or over quota gives no verdict. + The service continues, with one `source_failure` alert per cooldown. ### Alerting @@ -787,9 +784,8 @@ too close together. the ban's alert, with `mode` set to `observe` in its `detail`. - A repeat is an alert with the same event, and the same netblock, source, file or anomaly scope, as one sent less than `SWWAF_ALERT_COOLDOWN` before. An - anomaly scope is the thing counted: one client, netblock, AS number or named - netblock, or the whole service. A repeat is held back and counted, and the - next such alert sent gives the count. + anomaly scope is one of the things counted under "Anomaly thresholds". A + repeat is held back and counted, and the next such alert sent gives the count. - Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and counted by event, so a wide attack cannot flood the channel. Once the hour ends, one alert with the event `summary` gives those counts. It also counts @@ -820,7 +816,7 @@ addresses at low rates each, which the per-client limits do not see. A low - Every request but the health check is counted, whatever was done with it, clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` included, since an exempt client misbehaving is worth knowing about. The counters are - kept in memory and in `alerts.json`, with a fixed bound. + kept in memory and in `alerts.json`. ## Rule files @@ -846,10 +842,10 @@ and the default file. of every request; a rule naming one stops the start like any bad line. Request bodies are not available to rule files; reading them is the Core Rule Set's job. -- `log` only notes the match in the request log. `block` refuses the request - without a ban; `ban` makes the request a clear sign of attack (see "Bans"), - and is for requests no real visitor sends. A visitor banned by mistake is let - back in by lifting the ban. +- `log` only notes the match in the request log. `block` refuses the request; + `ban` makes the request a clear sign of attack (see "Bans"), and is for + requests no real visitor sends. A visitor banned by mistake is let back in by + lifting the ban. - At start, these stop the process with a message naming the file and line: - a line that does not parse; - a regex that does not compile; @@ -940,9 +936,8 @@ edit them at any time, and the running `smallwebwaf` takes the edit in. ## Request log -Every request gets one JSON object on one line on stdout, refused ones included, -apart from those Go's HTTP server ends before `smallwebwaf` sees them (see "Size -and time limits"). `docs/request-log.md` lists the fields. Bodies are never +Every request `smallwebwaf` sees gets one JSON object on one line on stdout, +refused ones included. `docs/request-log.md` lists the fields. Bodies are never logged, and no header but those listed there. Query strings are logged as received; an app that carries secrets in query strings needs that fixed in the app. The process's own messages (start, the settings, stop, errors) share the @@ -971,8 +966,7 @@ while `docker logs` keeps working. ## Metrics endpoint `GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request -carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. Every -metric carries the label `instance`, set to `SWWAF_INSTANCE_NAME`. No metric +carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. No metric carries a client IP address as a label. Per-address questions are answered by the request log and `GET /_smallwebwaf/clients/`. @@ -980,15 +974,14 @@ the request log and `GET /_smallwebwaf/clients/`. A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself: it answers it and never passes it to the app. The prefix carries the tool's -name, so it takes no path an app uses. Admins and scrapers reach these endpoints -through traefik, like any other request. `docs/admin-endpoints.md` lists the -endpoints, what each takes and what each answers. +name, so it takes no path an app uses. Admins and the metrics collector reach +these endpoints through traefik, like any other request. +`docs/admin-endpoints.md` lists the endpoints, what each takes and what each +answers. - `GET /_smallwebwaf/healthz` answers `200` with `ok` to anyone, without a token, while `smallwebwaf` is running. It does not ask the app; the - container's health check does. It is answered before any check, so a health - checker never needs an exemption and is never refused, for example by - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`. + container's health check does. - Every other endpoint needs its token, sent as `Authorization: Bearer ` (see "Tokens" under "Settings"). While a token is unset, the endpoints that need it answer `404`, as does any other path under the prefix. @@ -996,10 +989,9 @@ endpoints, what each takes and what each answers. other request goes through. They are answered at the point where another request would be forwarded to the app: a banned or refused client stays refused, and each request counts toward the client's limits. A missing or - wrong token is answered `401`, in `SWWAF_MODE=observe` too, and counts toward - the error burst. A client guessing tokens is therefore banned once it passes - `SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute. -- A ban made through the endpoints starts at once, with the cause `admin`. + wrong token is answered `401` (see "Attack detection" for what guessing + costs). +- A ban made through the endpoints starts at once. ## Deployment @@ -1041,26 +1033,26 @@ what the image holds and how each part is pinned. 2015, at a fixed commit hash, with a `go.mod` the build writes. `runsvinit` starts runit's `runsvdir`, which runs the `run` script of each directory under `/etc/service`. Ubuntu's `sv` looks for services there too. -- The image's `run` script for `smallwebwaf` waits one second and makes - `SWWAF_STATE_DIR` and every file in it belong to the `smallwebwaf` user (uid - and gid 65532). Then it starts `smallwebwaf` as that user with runit's - `chpst`. +- The image's `run` script for `smallwebwaf` makes `SWWAF_STATE_DIR` and every + file in it belong to the `smallwebwaf` user (uid and gid 65532). Then it + starts `smallwebwaf` as that user with runit's `chpst`. Every `run` script, the app's included, is a bash script. It starts with `#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main` function, called on its last line, as the owner's code style guide asks. Ubuntu -ships bash. +ships bash. Each waits one second (`sleep 1`) before starting its program, so a +process that stops at start is started again every second, not in a tight loop. Beyond its `FROM` line, the app's Dockerfile adds: - its binary; - any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.`; - its runit service: a directory under `/etc/service` named after the app, never - `smallwebwaf`. Its `run` script waits one second (`sleep 1`) and then starts - the app with `chpst`, listening where `SWWAF_UPSTREAM_URL` points (see - "Ports"). The app runs as a user of its own, which the Dockerfile creates with - `useradd`. That user is neither root nor `smallwebwaf`, so that the app cannot - touch the state files or the process of `smallwebwaf`. + `smallwebwaf`. Its `run` script starts the app with `chpst`, listening where + `SWWAF_UPSTREAM_URL` points (see "Ports"). The app runs as a user of its own, + which the Dockerfile creates with `useradd`. That user is neither root nor + `smallwebwaf`, so that the app cannot touch the state files or the process of + `smallwebwaf`. It sets no `ENTRYPOINT` or `USER` of its own. The container must start `runsvinit`, as root, so that it can start each service as its own user. @@ -1072,16 +1064,17 @@ It sets no `ENTRYPOINT` or `USER` of its own. The container must start script starts as root and hands over to its program's user with `chpst`. Both services start with the container's environment variables and write to the container's output, which `docker logs` shows. -- When `smallwebwaf` or the app exits, runit runs its `run` script again, which - waits a second before starting it. The other process keeps running, and the - container does not stop. While `smallwebwaf` is down, traefik answers `502`. - While the app is down, `smallwebwaf` answers `502`, which is not a client - offence. -- Only four things stop `smallwebwaf`, all at start: +- When `smallwebwaf` or the app exits, runit runs its `run` script again. The + other process keeps running, and the container does not stop. While + `smallwebwaf` is down, traefik answers `502`. While the app is down, + `smallwebwaf` answers `502`, which is not a client offence. +- Only faults found at start stop `smallwebwaf`: - invalid configuration; - a lookup database that is missing or unreadable; - a rule file or state file that does not parse; - - an unwritable `SWWAF_STATE_DIR`. + - an unwritable `SWWAF_STATE_DIR`; + - a `SWWAF_RULES_DIR` that does not exist or cannot be read; + - a `SWWAF_LISTEN_ADDR` it cannot listen on. - A `smallwebwaf` that stops at start is therefore started again every second, logging its message each time, until the cause is fixed; the health check fails meanwhile. Once running, a failing GeoJS, reputation source or alert diff --git a/docs/alerts.md b/docs/alerts.md index b35e1a5..cbc4fe2 100644 --- a/docs/alerts.md +++ b/docs/alerts.md @@ -79,10 +79,11 @@ on one line: - `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`: one of the events above. - `client`: the address whose request raised the alert. Empty for - `source_failure` and `file_error`. -- `netblock`: empty for `source_failure` and `file_error`; otherwise one of: + `source_failure`, `file_error` and `summary`. +- `netblock`: empty for `source_failure`, `file_error` and `summary`; otherwise + one of: - the ban's netblock; - - the netblock an anomaly counted; + - the netblock an anomaly counted, empty in the `asn` and `total` scopes; - for the other events, the client as rate limiting counts it: an IPv4 address as a /32, or an IPv6 group. - `asn`, `as_name` and `country`: the client's, as known when the alert was @@ -97,6 +98,8 @@ on one line: - for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`; - for a source failure, the `source` and `error`; - for a file error, the `file` and `error`; + - for a summary, the `hour` it covers, the `count` of alerts held back, and + that count by event in `events`; - `mode`, when the request was not refused: `detect` or `observe`. - `suppressed_repeats`: how many repeats the cooldown held back before this alert. diff --git a/docs/metrics.md b/docs/metrics.md index b2a0e91..87877e7 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -1,8 +1,8 @@ # The metrics -The metrics `GET /_smallwebwaf/metrics` answers with, as "Metrics endpoint" in -[`SPEC.md`](../SPEC.md) describes it. Prometheus keeps the `instance` label as -`exported_instance` unless the scrape sets `honor_labels: true`. +What `GET /_smallwebwaf/metrics` answers with. "Metrics endpoint" in +[`SPEC.md`](../SPEC.md) describes the endpoint. Prometheus keeps the `instance` +label as `exported_instance` unless the scrape sets `honor_labels: true`. ## Traffic diff --git a/docs/settings.md b/docs/settings.md index 840f7c5..2e32250 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -91,11 +91,11 @@ that take `off`, the `_FILE` form, and what each group does and why. ## Static lists - `SWWAF_ALLOW_NETS` (default empty): netblocks you trust, such as your - monitoring. Their clients are refused only for a size or time limit or a - missing or wrong token. + monitoring (see step 3 in [`SPEC.md`](../SPEC.md)). - `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are exempt from rate limiting and the byte limits. -- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused. +- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are refused (see + step 3 in [`SPEC.md`](../SPEC.md)). ## Request rate limits, per client @@ -136,7 +136,13 @@ that take `off`, the `_FILE` form, and what each group does and why. ## Lookup of AS number and country -- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. +- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off` stops + the start while a setting that needs a lookup is set: + - the two country lists; + - `SWWAF_ADD_LOOKUP_HEADERS`; + - the four biased threshold lists and `SWWAF_ASN_LIMIT_PERCENT_URL`; + - `SWWAF_UNKNOWN_LIMIT_PERCENT` below `100`; + - an anomaly threshold per AS number. - `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its `.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`. - `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its @@ -286,7 +292,7 @@ as `REQUESTS_PER_MINUTE`: client; - `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6 client. -- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs a lookup source. +- `SWWAF_ANOMALY_ASN_...`: per AS number. - `SWWAF_ANOMALY_TOTAL_...`: the whole service. - `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole. `SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such -- 2.54.0