diff --git a/SPEC.md b/SPEC.md index e437903..b1b7daa 100644 --- a/SPEC.md +++ b/SPEC.md @@ -2,16 +2,18 @@ `smallwebwaf` is a web application firewall and reverse proxy for one app. It runs inside the app's container, between traefik and the app. This document says -what it must do and why. Everything in it but the last section, "Planned, not -built", is built. `README.md` says how to run it. `EVALUATION.md` says why no -existing tool was adopted. +what it must do and why, each requirement once. Everything in it is built, +except the last section, "Planned, not built". `README.md` says how to run it. +`EVALUATION.md` says why no existing tool was adopted. `docs/gitea.md` records +how it behaves in front of gitea. ## Purpose - It protects one self-hosted app on the open internet from scrapers, scanners - and other abusers. -- It protects the app from its first request with no setting at all. Every - setting has a default chosen for a service facing the internet in 2026. + and other abusers, from the app's first request, with no setting at all. Every + setting has a default chosen for a service facing the internet in 2026. A + setting that needs an alert destination, an account key or a token stays off + until the operator supplies it. - It limits each client's requests and bytes per minute, hour and day. It bounds the size and duration of every request and answer. It detects attacks. - It bans abusers: briefly at first, for seven days on a clear sign of attack, @@ -19,9 +21,10 @@ existing tool was adopted. - It looks up each client's AS number and country. It can refuse whole countries and netblocks, and lower its limits for listed AS numbers and countries. It consults reputation sources. It sends alerts. -- Everything is set by environment variables named with the `SWWAF_` prefix. The - only other inputs are files: the rule files, the state files, a lookup - database, and a setting given as a file. +- Everything is set by environment variables whose names start with `SWWAF_`, + since `smallwebwaf` shares its container, and so its environment, with the + app. The only other inputs are files: the rule files, the state files, a + lookup database, and a setting given as a file. ## Non-goals @@ -32,41 +35,20 @@ existing tool was adopted. - Defence against floods that saturate the host's network link. That needs help upstream of the host. - A web console or a configuration file. -- Sharing bans between the instances of several apps. Each instance keeps its - own. Reading a CrowdSec engine's decisions is supported. ## Architecture - One statically linked Go binary. It ships in an image built on Ubuntu 26.04 LTS with nixpkgs installed. That image is the base of the app's own image, so - that `smallwebwaf` and the app run in one container (see "Deployment"). runit + `smallwebwaf` and the app run in one container (see "Deployment"). runit starts `smallwebwaf` as a non-root user of its own. `smallwebwaf` writes only to its state directory. - One listener, on port 8080, which traefik routes to. It forwards requests to - the app on `127.0.0.1:8081`. It answers its own endpoints, for health, metrics - and ban management, under `/_smallwebwaf/`. Those never reach the app. -- The parts of the process: - - Client identification: the real client address from `X-Forwarded-For`, - trusting only the netblocks in `SWWAF_TRUSTED_PROXIES`. - - Lookup: the AS number and country, from the GeoJS web service by default - or from a database file the operator supplies, held in memory. - - Reputation: blocklists fetched on a schedule; DNS blocklists and AbuseIPDB - asked in the background, with their answers cached. - - Counters: per-client requests and bytes per minute, hour and day, held in - memory and written out with the rest of the state. - - Attack detection: regex rules read from plain text files and read again - when they change; Coraza with the OWASP Core Rule Set; trap paths; and the - error burst, a burst of refusals from one client. - - Ban ledger: active and past bans with their notes, and each client's - history, held in memory and kept in JSON files the operator may edit. - - Request log: one JSON object per request on stdout, optionally also sent - to a syslog server. - - Metrics: Prometheus counters and gauges. - - Alerting: a queue that holds back repeats and feeds a webhook, Slack and - ntfy. - - Proxy: the standard library's `httputil.ReverseProxy`, streaming in both - directions within the size and time limits, with WebSocket support. + the app on `127.0.0.1:8081`. It answers its own endpoints under + `/_smallwebwaf/` (see "Admin endpoints"). Those never reach the app. - No database of any kind. Nothing is read from disk while serving a request. +- The proxy is the standard library's `httputil.ReverseProxy`. The other parts + of the process are each specified below. - Libraries, checked against the owner's Go dependency defaults: - the standard library for the proxy, HTTP clients, DNS, logging (`log/slog`) and the state files (`encoding/json`, `os.Rename`); @@ -84,7 +66,8 @@ existing tool was adopted. ## What happens to a request The steps run in this order. The first step that produces a final answer ends -the handling. The size and time limits apply to the whole exchange. +the handling. The size and time limits apply to the whole exchange. Each step +names the section that holds its rules. 1. `GET /_smallwebwaf/healthz` is answered at once, before any check. 2. The client is identified. @@ -101,107 +84,81 @@ the handling. The size and time limits apply to the whole exchange. 3. The static lists. A client in `SWWAF_ALLOW_NETS` skips every check below. It is still counted for the anomaly thresholds. A client in `SWWAF_DENY_NETS` is refused. -4. The ban ledger. A client under a ban is refused with `SWWAF_BAN_RESPONSE`. If - a clear sign of attack caused the ban, the ban becomes permanent. -5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off`. - - With GeoJS, a request waits for its client's first answer, up to - `SWWAF_LOOKUP_TIMEOUT`, only when a setting needs it before the request - goes on: a country list, a biased threshold or `SWWAF_ADD_LOOKUP_HEADERS`. - Otherwise the request goes on at once. The answer is added to the client's - history and ban notes when it comes. A request that ends before then is - logged without it. - - A client the lookup cannot place has an unknown country and AS number. - Every private, loopback and link-local address is such a client. The - exclusive country list refuses it, and the biased thresholds give it - `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it differently. -6. The country lists. A client from a country in `SWWAF_DENIED_COUNTRIES` is - refused. When `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any - other country is refused. The answer is `SWWAF_BAN_RESPONSE`. The request - body has not been read yet. The refusal skips everything below, the rule - files and the Core Rule Set included. It is logged with the action - `country_denied` and counted in the metrics. It is not an offence and makes - no ban. -7. The reputation sources. A client inside a fetched blocklist gets - `SWWAF_BLOCKLIST_ACTION`. A client the CrowdSec decision list lists is banned - until the decision ends. A cached DNS blocklist verdict or AbuseIPDB score - gets `SWWAF_REPUTATION_ACTION`. With no cached verdict, a query is queued in - the background and the request goes on. A first request is never delayed by a - reputation query. -8. The client's limit percentage: the lowest of the percentages that apply to - it, from its AS number, its country or a reputation source, or 100 if none - does. -9. Rate limiting, unless the client is in `SWWAF_RATE_LIMIT_EXEMPT_NETS` or the - path is under `SWWAF_RATE_LIMIT_EXEMPT_PATHS`. The minute, hour and day - counters are checked against each limit times the percentage. A request over - any of them breaks that limit: it is refused with `SWWAF_BAN_RESPONSE`, and - the client is banned (see "Bans"). -10. The trap paths. A request for a path in `SWWAF_TRAP_PATHS` is a clear sign - of attack: it is refused and the client is banned. -11. The rule files, in file name order and then line order. A rule that matches - takes its action: only log, refuse with `403`, or refuse and ban. Matching - stops at the first rule that refuses. -12. The Core Rule Set inspects the request's method, its URL with the query - string, and its headers. By default it reads no body. When - `SWWAF_WAF_BODY_LIMIT` is a size, it reads a body that is form data, - multipart, JSON or XML, up to that size. Any other body reaches the app - uninspected. So does a JSON or XML body larger than the limit, which cannot - be read in part. Responses are not inspected. In `block` mode, the default, - a match at or over the anomaly threshold is refused with `403`. In `detect` - mode it is only logged and alerted. +4. The ban ledger. A client under a ban is refused (see "Bans"). +5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number + and country"). The request goes on at once. It waits for the first GeoJS + answer, up to `SWWAF_LOOKUP_TIMEOUT`, only when one of these needs it: + - a country list; + - a biased threshold; + - `SWWAF_ADD_LOOKUP_HEADERS`. +6. The country lists (see "Country lists"). A refused request is not an offence. +7. The reputation sources (see "Reputation"). A client on a fetched blocklist + gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is + banned. A client listed by a DNS blocklist, or scored at least + `SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`. + With no cached verdict, a query is queued in the background and the request + goes on. A first request is never delayed by a reputation query. +8. The client's limit percentage (see "Biased thresholds"). +9. Rate limiting (see "Request rate limits, per client"). A request over a limit + times the percentage is refused and bans the client. +10. The trap paths (see `SWWAF_TRAP_PATHS`). A request for one is refused and + bans the client. +11. The rule files (see "Rule files"). Matching stops at the first rule that + refuses. +12. The Core Rule Set (see "Attack detection"). In `block` mode a match is + refused with `403`. In `detect` mode it is only logged and alerted. 13. A request under `/_smallwebwaf/` is answered here and goes no further. 14. The request is forwarded to `SWWAF_UPSTREAM_URL`, streaming, with `X-Forwarded-For` added, and `X-Client-ASN` and `X-Client-Country` when enabled. 15. After the response: - - The response bytes, the request body bytes, or both are added to the byte - counters. A client whose total passes a byte limit times the percentage - has broken that limit and is banned. The response in progress is not cut - off. - - The client's refusals are counted: those `smallwebwaf` refused after a - rule file match, a trap path, a Core Rule Set match, or a missing or wrong - token. Answers the app gave are not counted. More than - `SWWAF_ERROR_BURST_THRESHOLD` of them within a minute break a limit and - ban the client. + - The response bytes, the request bytes or both are added to the byte + counters, as `SWWAF_BYTES_COUNT` says. A client whose total passes a byte + limit times the percentage has broken that limit and is banned. The + response in progress is not cut off. + - The client's refusals are counted for the error burst (see + `SWWAF_ERROR_BURST_THRESHOLD`). - The client's history and the anomaly counters are updated, and the alert thresholds are evaluated. +In `SWWAF_MODE=observe`, every refusal above is logged as what would have +happened, nothing is refused, and no ban is made. The size and time limits and +the token check still apply. + ## Counting method - Each window (minute, hour, day) uses two adjacent fixed buckets per client. The earlier bucket is weighted by how much of it still overlaps the sliding window. This costs a few integers per client per window. It avoids the burst at bucket boundaries that a single fixed bucket allows. -- Counters are read and updated in memory. A request never waits on the disk. - They are written to `clients.json` every `SWWAF_STATE_COUNTER_INTERVAL` and at - shutdown, and loaded again at start. A restart therefore does not hand every - client a fresh allowance. Buckets whose time has passed are discarded on load. -- Memory is bounded by `SWWAF_MAX_TRACKED_CLIENTS` and `SWWAF_MAX_BANS`, and by - fixed limits on the GeoJS answers, the reputation verdicts and the anomaly - counters kept. When the table of clients is full, the least recently seen - client is dropped first, with its history. When `SWWAF_MAX_BANS` bans - `smallwebwaf` made are held, the one that has gone longest without a request - is dropped first, past, active or permanent. Bans an admin made are never - dropped. +- Counters are read and updated in memory. They are written to `clients.json` + and loaded again at start (see "Persistent state"), so a restart does not hand + every client a fresh allowance. Buckets whose time has passed are discarded on + load. +- Memory is bounded. When the table of clients holds + `SWWAF_MAX_TRACKED_CLIENTS`, the least recently seen client is dropped first, + with its history. The bans are bounded by `SWWAF_MAX_BANS` (see "Bans"). The + GeoJS answers, the reputation verdicts and the anomaly counters have fixed + bounds, given where each is described. ## Bans A ban refuses every request from a netblock with `SWWAF_BAN_RESPONSE` until it -ends. A permanent ban does not run out. It ends when an admin lifts it, or, if -`smallwebwaf` made it, when it is dropped past `SWWAF_MAX_BANS`. A scanner whose -ban was dropped that way is refused and banned again by its next probe. A ban an -admin made, whose cause is `admin`, is never dropped and does not count toward -`SWWAF_MAX_BANS`. Such bans can never fill the table. An admin who wants to keep -a ban `smallwebwaf` made sets its cause to `admin`. +ends. The netblock is the client: its IPv4 address, widened to +`SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one +address or one /64, so a permanent ban does not reach neighbours who did +nothing. Requests refused under a ban are not counted against limits. They are +counted in the client's history and in the ban's notes. -The netblock a ban covers is the client: its IPv4 address, or its IPv6 group. -`SWWAF_BAN_SCOPE_V4_PREFIX` can widen an IPv4 ban to the surrounding netblock. -At the defaults a ban covers one address or one /64, so a permanent ban does not -reach neighbours who did nothing. +An offence is a request `smallwebwaf` holds against the client. It is one of: -An offence is a request `smallwebwaf` holds against the client. It is one that -carries a clear sign of attack, breaks a limit, or is refused by a rule file, by -the Core Rule Set or for a missing or wrong token. Offences are counted by kind -in the client's history. Two kinds lead to a ban, each by its own rule: +- a clear sign of attack; +- a broken limit; +- a refusal by a rule file, by the Core Rule Set or for a missing or wrong + token. + +Offences are counted by kind in the client's history. The first two kinds lead +to a ban, each by its own rule: - A clear sign of attack: a request that matches a rule whose action is `ban`, or asks for a path in `SWWAF_TRAP_PATHS`. Examples are a probe for a `.env` @@ -229,13 +186,16 @@ in the client's history. Two kinds lead to a ban, each by its own rule: A Core Rule Set match refuses only the request it matched, without a ban, because the Core Rule Set has false positives. A `block` rule does the same. Those refusals count toward the error burst, so a client that keeps setting them -off is banned under the second rule. Requests refused under a ban are not -counted against limits. They are counted in the client's history and in the -ban's notes. +off is banned under the second rule. -A client the CrowdSec decision list lists is banned until the decision ends, -with the cause `crowdsec`. Such a ban is never made permanent. It does not make -the netblock's next ban for a broken limit longer. +A client on the CrowdSec decision list is banned until the decision ends, with +the cause `crowdsec`. Only a client that sends a request is banned, so a long +list does not fill `bans.json`. Such a ban is never made permanent. It does not +make the netblock's next ban for a broken limit longer. + +Some refusals make no ban at all. `SWWAF_DENY_NETS`, a blocklist or reputation +hit whose action is `deny`, and the country lists refuse each request they cover +and record nothing in `bans.json`. Every ban carries notes with what an admin needs to decide whether to lift it (see "Persistent state"). An admin can add or lift a ban at any time by editing @@ -243,13 +203,14 @@ Every ban carries notes with what an admin needs to decide whether to lift it lifted before it ends is kept, marked as lifted, and does not count toward a longer ban. Deleting its entry from `bans.json` forgets it entirely. -Clients of the AS numbers and countries listed in the biased thresholds get -lower limits. So do clients a reputation source lists whose action is -`limit:`. The same rules then ban them after fewer requests. - -Some refusals make no ban at all. `SWWAF_DENY_NETS`, a blocklist or reputation -hit whose action is `deny`, and the country lists refuse each request they cover -and record nothing in `bans.json`. +A permanent ban does not run out. It ends when an admin lifts it, or when the +table of bans is full. When `smallwebwaf` holds `SWWAF_MAX_BANS` bans of its +own, it drops the one that has gone longest without a request, whether past, +active or permanent. A scanner whose ban was dropped that way is refused and +banned again by its next probe. A ban an admin made, whose cause is `admin`, is +never dropped and does not count toward `SWWAF_MAX_BANS`, so such bans can never +fill the table. An admin who wants to keep a ban `smallwebwaf` made sets its +cause to `admin`. ## Settings @@ -260,24 +221,22 @@ Conventions: Sizes take `K`, `M` and `G`, powers of 1024. Countries are two-letter ISO codes in either case. A code that is not a country, such as `nk`, stops the start with a message naming it. -- No setting is required. Every setting has a default chosen for a service - facing the internet in 2026, or stays off until the operator supplies what it - needs: an alert destination, an account key, a token. -- Every setting's name starts with `SWWAF_`, since `smallwebwaf` shares its - container, and so its environment variables, with the app. -- `off` switches off: - - a timeout, a size limit, a rate limit, a byte limit or an anomaly - threshold; - - the Core Rule Set's anomaly threshold and the error burst threshold; - - the alert cooldown and the hourly alert cap. -- `off` is refused by `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, and by every - setting that is not a limit: the lookup timeout, the reputation cache and - timeout, the ban durations, the state write times, and the blocklist refresh. +- These settings take `off`, which switches them off: + - the five timeouts under "Size and time limits"; + - `SWWAF_REQUEST_MAX_BYTES` and `SWWAF_RESPONSE_MAX_BYTES`; + - the three rate limits and the three byte limits; + - `SWWAF_WAF_ANOMALY_THRESHOLD`, `SWWAF_WAF_BODY_LIMIT` and + `SWWAF_ERROR_BURST_THRESHOLD`; + - `SWWAF_ALERT_COOLDOWN` and `SWWAF_ALERT_MAX_PER_HOUR`; + - every anomaly threshold. + + Every other setting refuses `off`. + - A list set to an empty value is an empty list, and replaces the default. -- Every setting may instead be given as a file holding the value, named by the - setting's name with `_FILE` added, such as `SWWAF_ADMIN_TOKEN_FILE`, for - secrets and long lists. `SWWAF_LOG_REMOTE_TLS_CA_FILE` names a file already - and has no `_FILE` form. +- Any setting can be read from a file instead, for secrets and long lists: + `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. + `SWWAF_LOG_REMOTE_TLS_CA_FILE` already holds a file name, so it has no `_FILE` + form. - Settings, files included, are read once at start. Changing one means restarting the container. The files `smallwebwaf` watches while it runs are its state files, its rule files and the lookup database. @@ -288,19 +247,14 @@ The settings, by group: ### Core -- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app, which listens - there in the same container. +- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. - `SWWAF_LISTEN_ADDR` (default `:8080`): the one listener, for the requests - `smallwebwaf` forwards and for its own endpoints. -- `SWWAF_ADMIN_TOKEN`: the bearer token for the ban endpoints and - `/_smallwebwaf/clients/`. A long random value, since the endpoints can be - reached from the internet. One shorter than 32 characters stops the start. - Unset by default, which switches the endpoints off; bans are then managed by - editing `bans.json`. + `smallwebwaf` forwards and for its own endpoints. The address part stays empty + (see "Ports"). - `SWWAF_INSTANCE_NAME` (default: the container's host name, which docker sets to the first 12 characters of the container's id): included in every log line, - metric and alert. Set it, for example to `fsn1app1/gitea`, to a name that - stays the same when a deploy replaces the container and tells instances apart. + metric and alert. Set it to a name that survives a redeploy and tells + instances apart, such as `fsn1app1/gitea`. - `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to log and alert on every decision while refusing nothing. - `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`, @@ -312,18 +266,31 @@ The settings, by group: - `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the clients held in memory and in `clients.json`. - `SWWAF_MAX_BANS` (default `5000`): the bans `smallwebwaf` made that are held, - past, active and permanent. Bans an admin made are kept besides, and never - dropped. + past, active and permanent (see "Bans"). -### Persistent state +### Tokens + +- `SWWAF_ADMIN_TOKEN`: the bearer token for the ban endpoints and + `/_smallwebwaf/clients/`. Unset by default, which switches them off; bans + are then managed by editing `bans.json`. +- `SWWAF_METRICS_TOKEN`: the bearer token a scraper sends for + `/_smallwebwaf/metrics`. Unset by default, which switches the metrics off, + since they would otherwise be open to anyone on the internet. It is separate + from `SWWAF_ADMIN_TOKEN`, so a scraper that holds it cannot manage bans. +- A token is a long random value, since the endpoints can be reached from the + internet. One shorter than 32 characters stops the start. The app starts with + the same environment variables as `smallwebwaf`, so it can read a token given + as one. The file form keeps a token out of the app's reach (see "Tokens" under + "Deployment"). + +### State files - `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the JSON state files, in a directory of their own beside the app's data, belonging to the `smallwebwaf` - user. A volume mounted there keeps the state when a deploy replaces the - container. -- `SWWAF_STATE_WRITE_DELAY` (default `10s`): after a ban is made, lifted or made - permanent, `bans.json` is written this long later, with every change made in - between, so a burst of changes becomes one write. + user. +- `SWWAF_STATE_WRITE_DELAY` (default `10s`): `bans.json` is written this long + after a ban is made, lifted or made permanent, so a burst of changes becomes + one write. - `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is written. @@ -348,10 +315,6 @@ The settings, by group: ### Metrics -- `SWWAF_METRICS_TOKEN`: the bearer token a scraper sends for - `/_smallwebwaf/metrics`, a long random value. One shorter than 32 characters - stops the start. Unset by default, which switches the metrics off, since they - would otherwise be open to anyone on the internet. - `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and countries get series of their own. The rest are summed as `other`. @@ -387,22 +350,18 @@ largest response allowed, and no single download breaks one. ### Size and time limits These apply per request, in both directions. The client-facing timeouts apply -between the client and `smallwebwaf`. The app-facing ones apply between -`smallwebwaf` and the app. A slow client and a slow app are separate problems. -There is one size limit for request bodies and one for response bodies: -`smallwebwaf` passes bodies through unchanged, so a limit on each side would -bound the same bytes, and the lower one would always decide. Bodies stream -straight through, so a request body reaches the app while the client is still -sending it. +between the client and `smallwebwaf`, the app-facing ones between `smallwebwaf` +and the app, since a slow client and a slow app are separate problems. There is +one size limit for request bodies and one for response bodies, since +`smallwebwaf` passes bodies through unchanged. Bodies stream straight through, +so a request body reaches the app while the client is still sending it. - `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to send its request line and headers, and then, from the end of the headers, its body. - `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request - line and headers. Over it, Go's HTTP server answers `431` and closes the - connection, and nothing reaches the app. It must be more than `4K` and cannot - be `off`: Go's HTTP server always has such a limit, and reads 4 KiB past the - one it is given before it refuses. + line and headers. It must be more than `4K`. Go's HTTP server always has such + a limit, and reads 4 KiB past the one it is given before it refuses. - `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection may wait for its next request before `smallwebwaf` closes it. It is longer than the 90 seconds after which traefik, by default, closes an unused @@ -417,32 +376,30 @@ sending it. - `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. - `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. -When a limit is passed after the request line and headers have been read, but -before the response has started, `smallwebwaf` answers itself: +What the client gets when a limit is passed: -- `413` for a request body that is too large; -- `408` for a client too slow to send its body; -- `502` for a response that is too large; -- `504` for an app that is too slow. - -A request that announces a body larger than its limit is refused before anything -reaches the app. Once the response has started it can only be cut off, and the -connection is closed. - -Since a request body streams through, each side can hold up the other. While a -request body is still on its way, a request timeout that runs out answers `408` -if `smallwebwaf` was waiting for the client to send more at that moment. It -answers `504` if it was waiting for the app to take what it had. - -Go's HTTP server reads a request's line and headers before `smallwebwaf` sees -the request. A client that takes longer than `SWWAF_CLIENT_REQUEST_TIMEOUT` to -send them gets either no answer or an error answer from the server, mostly -`400`. Either way the server closes the connection and nothing reaches the app. -Headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` are answered `431` by the -server itself. Neither request gets a line in the request log. - -A WebSocket connection leaves these limits behind once it is upgraded. It stays -open until either side closes it. +- Go's HTTP server reads a request's line and headers before `smallwebwaf` sees + the request. It answers `431` to headers over + `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`. A client too slow to send them gets + either no answer or an error answer, mostly `400`. Either way the server + closes the connection, nothing reaches the app, and the request log gets no + line. +- A request that announces a body larger than its limit is refused before + anything reaches the app. +- When a limit is passed after the headers have been read, but before the + response has started, `smallwebwaf` answers itself: + - `413` for a request body that is too large; + - `408` for a client too slow to send its body; + - `502` for a response that is too large; + - `504` for an app that is too slow. +- A request body streams through, so each side can hold up the other. A request + timeout that runs out while the body is still on its way answers `408` if + `smallwebwaf` was waiting for the client at that moment. It answers `504` if + it was waiting for the app to take what it had. +- Once the response has started it can only be cut off, and the connection is + closed. +- A WebSocket connection leaves these limits behind once it is upgraded. It + stays open until either side closes it. ### Lookup of AS number and country @@ -450,55 +407,45 @@ On by default through GeoJS, which needs no account or file, so that country lookup works with no setup. The file is the alternative for a service that keeps its visitors' addresses on its own host. -- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. +- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. One source + at a time: `file` without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` + with any other source, stops the start with a message naming both. So does a + setting that needs lookups while the source is `off`: the country lists, the + biased thresholds, `SWWAF_ADD_LOOKUP_HEADERS`, and the anomaly thresholds per + AS number. - `SWWAF_LOOKUP_DB_PATH`: the file for `file`: the IPinfo Lite database in its - `.mmdb` form (`ipinfo_lite.mmdb`), one file that carries both country and AS - number. `smallwebwaf` reads its `asn`, `as_name` and `country_code` fields. - The operator downloads it with a free IPinfo account and mounts it read-only. - `smallwebwaf` never fetches it and holds no account or token for it. A file - that is missing or unreadable at start stops the start. -- Refreshing the file is the operator's business; IPinfo updates it daily. - `smallwebwaf` notices when the file is replaced and reads it again. A - replacement it cannot read is ignored, logged and sent as one `file_error` - alert, and the file it has stays in use. Mount the directory that holds the - file rather than the file itself: docker does not show a single mounted file - being replaced on the host. + `.mmdb` form (`ipinfo_lite.mmdb`), whose `asn`, `as_name` and `country_code` + fields are read. The operator downloads it with a free IPinfo account, mounts + it read-only, and refreshes it; IPinfo updates it daily. `smallwebwaf` never + fetches it and holds no account or token for it. A file that is missing or + unreadable at start stops the start. `smallwebwaf` notices when the file is + replaced and reads it again. A replacement it cannot read is ignored, logged + and sent as one `file_error` alert, and the file it has stays in use. The + directory that holds the file is mounted, not the file itself (see "Rule + files" for why). - `geojs` asks the free GeoJS web service about up to 200 addresses in one - request (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`). It reads each - answer's `country_code`, `asn` and `organization_name` (the AS name). An - answer without a `country_code`, which GeoJS sends for ranges it cannot place, - counts as unknown. So does an `asn` of `64512`, which GeoJS gives when it - knows none. GeoJS is told the address of every new visitor, whether or not a - setting uses the answer. The request log, the metrics, the client's history - and ban notes carry the AS number and country, so every answer is used. - Private, loopback and link-local addresses, which no source can place, are - never sent. + request (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a + time, and reads each answer's `country_code`, `asn` and `organization_name`. + An answer without a `country_code`, or with the `asn` `64512`, counts as + unknown. GeoJS is told the address of every new visitor, whether or not a + setting uses the answer, since the request log, the metrics and the ban notes + carry it. Private, loopback and link-local addresses, which no source can + place, are never sent. +- A client the lookup cannot place has an unknown country and AS number. The + exclusive country list refuses it, and the biased thresholds give it + `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it differently. - Each GeoJS answer is kept for 7 days, in memory and in `lookups.json`, apart - from the table of clients, so it outlasts a client dropped from that table. - After 7 days the client's next request asks again. Up to 100,000 answers are - kept, about 15 MiB. When that many are held, the answer used longest ago goes - first. + from the table of clients. Up to 100,000 answers are kept, about 15 MiB; past + that, the answer used longest ago goes first. - `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its - client's first answer when a setting needs it. GeoJS normally answers in a - fraction of that. At most one request to GeoJS is under way at a time. The - addresses that arrive meanwhile are asked about together in the next one, up - to 200 per request, the rest in the requests after it. A request that takes - longer than `SWWAF_LOOKUP_TIMEOUT` is abandoned. A client whose answer has not - come in time counts as unknown until it comes. Its later requests do not wait, - and its address is asked about again in the background. + client's first answer when a setting needs it, and how long a request to GeoJS + may take. A client whose answer has not come in time counts as unknown until + it comes, and its later requests do not wait. - GeoJS publishes no rate limit, but its terms forbid "an excessive amount of API requests", judged by GeoJS alone, and let it block a caller. While GeoJS - is slow, down or refusing `smallwebwaf`, clients with a kept answer are - unaffected and new clients count as unknown. So - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`, when set, refuses them. `smallwebwaf` - keeps asking with backoff and sends one `source_failure` alert per cooldown. A - service that cannot accept this uses the file. -- One source at a time. `SWWAF_LOOKUP_SOURCE=file` without - `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other source, stops - the start with a message naming both. So does a setting that needs lookups - while `SWWAF_LOOKUP_SOURCE` is `off`: the country lists, the biased - thresholds, `SWWAF_ADD_LOOKUP_HEADERS`, and the anomaly thresholds per AS - number. + is slow, down or refusing `smallwebwaf`, new clients count as unknown, + `smallwebwaf` keeps asking with backoff, and one `source_failure` alert is + sent per cooldown. A service that cannot accept this uses the file. - `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): pass `X-Client-ASN` and `X-Client-Country` to the app. @@ -511,20 +458,20 @@ where it comes from. Clients in `SWWAF_ALLOW_NETS` are not checked. a listed country is refused. - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`: for example `us,de`. Only clients placed in a listed country get through. Every request from any other country - is refused. So is every request from a client the lookup cannot place: - - one whose address is missing from the database; - - one GeoJS has not answered for in time; - - any client on a private address, such as a visitor on the local network, - another container or internal monitoring. - - Those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let - unplaced clients through would let every new client in whenever GeoJS stops - answering. - + is refused. So is every request from a client the lookup cannot place, such as + a visitor on the local network, another container or internal monitoring. + Those that should reach the app go in `SWWAF_ALLOW_NETS`. A list that let + unplaced clients through would let every new client in whenever GeoJS stops + answering. That is the nearest a failing helper comes to taking the service + down: GeoJS failing while this list is set refuses every new client. - Both may be set. `SWWAF_DENIED_COUNTRIES` then adds nothing, since the exclusive list already refuses every other country. A code on both lists stops the start with a message naming it. -- What a refused request gets is under "What happens to a request", step 6. +- A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the + client's address has been looked up, before its body is read. It skips the + reputation checks, the limits, the rule files and the Core Rule Set. It is + logged with the action `country_denied`, and counted in the metrics. It is a + refusal, not a ban. ### Biased thresholds @@ -538,12 +485,6 @@ abuse comes from. is a zero allowance: the client's first request breaks a limit and bans it. - `SWWAF_COUNTRY_LIMIT_PERCENT`: the same form with country codes, for example `CN:25,RU:50`. -- No budget is shared by a whole country or AS number. One abuser could use it - up and lock out everyone else there: a denial of service nobody chose, and the - one this tool exists to prevent. Refusing a whole country is left to the - operator's choice, through the country lists. -- When more than one percentage applies to a client (its AS number, its country, - a reputation hit), the lowest applies. - `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT`: optional overrides applied to the byte limits only, when the byte percentage should differ from the request percentage. @@ -552,6 +493,14 @@ abuse comes from. - `SWWAF_ASN_LIMIT_PERCENT_URL`: optional URL of a text file of `AS:percent` lines, so one abuse-source list can be shared by every `smallwebwaf` in the fleet. It is fetched and refreshed like the blocklists. +- A client's limit percentage is the lowest of those that apply to it: its AS + number's, its country's, or a reputation hit's `limit:`. It is 100 if + none applies. Clients given a lower percentage are banned after fewer + requests, first for a while and then, for repeated abuse, permanently. +- No budget is shared by a whole country or AS number. One abuser could use it + up and lock out everyone else there: a denial of service nobody chose, and the + one this tool exists to prevent. Refusing a whole country is left to the + operator's choice, through the country lists. ### Attack detection @@ -567,8 +516,7 @@ the bans still apply to the client that sends it, and `SWWAF_WAF_BODY_LIMIT` switches body inspection on. - `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of rule - files, read at start and again whenever a file in it changes (see "Rule - files"). + files (see "Rule files"). - `SWWAF_RULES_ENABLED` (default `true`): `false` skips rule files entirely. - `SWWAF_WAF_MODE` (default `block`): `off`, `detect` (log and alert only), or `block`. @@ -587,9 +535,8 @@ switches body inspection on. In front of a code forge these refuse git over HTTP, container image and package uploads, and views of ordinary files in a repository. At the site root, where no app serves such files, the default rule file bans the common - probes these rules caught, such as `/.env`, `/.git/config`, - `/.aws/credentials`, `/.ssh/id_rsa`, `/.htpasswd` and `/wp-config.php.bak`. - A list given replaces the default, so include them in it. + probes these rules caught (see "Rule files"). A list given replaces the + default, so include them in it. - `SWWAF_WAF_EXEMPT_PATHS`: path prefixes not inspected. - `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size, @@ -598,9 +545,10 @@ switches body inspection on. on without holding it in memory. It reads JSON and XML bodies no larger than the size, since those cannot be read in part. Any other body is not read: the Core Rule Set would read it as form data, where binary content such as a git - push trips rules written for text. Body inspection suits apps whose forms - carry no code. In front of gitea it refuses, until the rule ids the request - log names are added to `SWWAF_WAF_DISABLED_RULES`: + push trips rules written for text. Responses are never inspected. Body + inspection suits apps whose forms carry no code. In front of gitea it refuses + the requests below, until their rule ids, named in the request log, are added + to `SWWAF_WAF_DISABLED_RULES`: - issue and comment text, wiki pages and files saved in the web editor that hold shell commands or code (932125, 932235, 932250 and others); - package descriptions that show code; @@ -616,23 +564,19 @@ switches body inspection on. trying one attack or token after another is refused many times a minute; a person rarely more than a few times. Answers the app gives are not counted, since they do not tell a scanner from an ordinary client. In front of gitea, - container image and package clients are answered `404` by design, often - hundreds of times a minute. git and registry clients are answered `401` at the - start of every push, every fetch from a private repository and every image - pull. - -### The six changes to the Core Rule Set + registry and git clients are answered `404` and `401` by design, hundreds of + times a minute. `smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of -every app, and no setting undoes them. +every app, and no setting undoes them: - `PUT`, `PATCH` and `DELETE` are allowed methods besides `GET`, `HEAD`, `POST` and `OPTIONS`. APIs, container image pushes and package uploads use them. Other methods stay refused. - The request header `Expect` is allowed, since curl and git send it before some large uploads. `Content-Encoding` is allowed on a request whose body the Core - Rule Set does not read, which by default is every request: git compresses a + Rule Set does not read, which by default is every request. git compresses a fetch request over 1 KiB and says so in that header. On a body the Core Rule Set does read, the header stays refused, since a compressed body cannot be inspected. The other headers the Core Rule Set refuses, such as `Proxy`, stay @@ -644,47 +588,40 @@ every app, and no setting undoes them. address. - Ten query parameters are not checked against the Core Rule Set's lists of system files (930120), shell paths (932160) and command names (932260): - `path`, `files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`, - `artifactName` and `redirect_to`. gitea sends file paths, branch and workflow - names, and the page to return to after signing in, in them. In a repository - any name on those lists can be an ordinary file or branch, such as - `.gitignore`, `package.json`, `docker-compose.yml`, `bin/docker-entrypoint` or - a branch named `docker-build`. gitea reads these values as names within a - repository or its own records, or as a page of its own site. What only these - three rules refuse is let through there. That is more than a name such as - `/etc/passwd` or `whoami` on its own: commands such as `|cat /etc/passwd`, - `wget http://…` and `nc -e /bin/sh …`, and `file:///etc/passwd`, pass as well. - Path traversal (`../`), SQL and script injection, and PHP, Java and Node.js - code are still refused there. Every other parameter keeps all three rules. - These names, and `redirect_uri`, are matched without regard to case, as Coraza - matches them. An app that uses one of these parameters as a file on the - server, or passes it to a shell, gets no help from the three rules there. See - "Risks the design has to handle". + - `path`, `files`, `skip-to` and `sub_path`; + - `ref`, `sha`, `branch` and `workflow`; + - `artifactName` and `redirect_to`. + + gitea puts file paths, branch and workflow names, and the page to return to + after signing in into them. Any name on those lists can be an ordinary file + or branch, such as `.gitignore` or `docker-build`. In these parameters, + anything only these three rules catch reaches the app, commands such as + `|cat /etc/passwd` included. Path traversal, SQL and script injection, and + PHP, Java and Node.js code are still refused there. Every other parameter + keeps all three rules. Parameter names are matched without regard to case, + as Coraza matches them. + - The Core Rule Set reads the request without the `gitea_flash` and `redirect_to` cookies. It does not check `Referer` for a Unix command given - without arguments (932340) or for Java starting a process (944110). gitea - writes into `gitea_flash` a message for the next page naming what was just - done, and into `redirect_to` the page to return to after signing in. `Referer` - is the address of the page a request comes from. In these cookies the Core - Rule Set takes names such as `package.json`, `document.write.js` or - `Get-ChildItem.ps1`, and titles that hold them, for attacks. In `Referer` it - takes the address of a search for one word, such as `env`, `set` or `last`, - when the address ends in it, and the address of OpenJDK's - `src/java.base/share/classes/java/lang/Runtime.java`. A browser sends such a - cookie with every request until gitea replaces or removes it. gitea cannot do - that while `smallwebwaf` refuses those requests, so one refusal would keep the - browser out of the whole site. A browser names a page in `Referer` on - everything the page loads and on every link followed from it, so all of those - would be refused. gitea shows the message with any script removed and returns - only to a page of its own site. An attack sent in a cookie of either name is - not refused by the Core Rule Set. Nor is a Unix command without arguments, or - Java starting a process, sent in `Referer`. Every other cookie is read in - full, and `Referer` keeps the other rules, script and SQL injection among - them. + without arguments (932340) or for Java starting a process (944110). gitea puts + a message naming what was just done into `gitea_flash`. It puts the page to + return to after signing in into `redirect_to`. The Core Rule Set takes file + names in them, such as `package.json`, for attacks. A browser sends such a + cookie with every request until gitea replaces it. gitea cannot replace it + while those requests are refused, so one refusal would keep the browser out of + the whole site. `Referer` names the page a request comes from, on everything + that page loads. The Core Rule Set takes some page addresses for attacks, such + as a search for `env`, or OpenJDK's `Runtime.java`, whose path holds both + `runtime` and `java.`. gitea shows the message with any script removed and + returns only to its own pages. Every other cookie is read in full, and + `Referer` keeps the other rules. - Responses are not inspected. A raw file from a repository, such as a shell script, looks to the response rules like source code leaking from the server. -### Bans +The fourth and fifth changes leave an app on its own in those places (see "Risks +the design has to handle"). + +### Ban lengths and response - `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of attack. @@ -704,24 +641,25 @@ every app, and no setting undoes them. ### Reputation No source is on by default, for the reason the country lists and the biased -thresholds are empty. A source refuses the clients that a list kept elsewhere -names, or lowers their limits, while the defaults judge a client only by what it -does to this service. AbuseIPDB and CrowdSec also need an account or an engine -of the operator's own. Besides, the list best suited to be on by default, -Spamhaus DROP, must not be downloaded automatically more than once an hour. -Spamhaus may block an address that downloads it too often. Each `smallwebwaf` -fetches its own copy. On a host that runs several apps with it, which share one -address, downloads would often come less than an hour apart. +thresholds are empty. A source acts on a list kept elsewhere; the defaults judge +a client only by what it does here. AbuseIPDB and CrowdSec also need an account +or an engine of the operator's own. Besides, the list best suited to be on by +default, Spamhaus DROP, must not be downloaded automatically more than once an +hour. Spamhaus may block an address that downloads it too often. Each +`smallwebwaf` fetches its own copy. On a host that runs several apps with it, +which share one address, downloads would often come less than an hour apart. - `SWWAF_BLOCKLIST_URLS`: text files of addresses and netblocks, one per line; anything after a `;` or `#` on a line is ignored. For example the Spamhaus DROP list, `https://www.spamhaus.org/drop/drop.txt`. Spamhaus's terms for it, - on its DROP page: automated downloads must be at least one hour apart, and - once a day is more than enough in most cases; a product that uses it must - credit The Spamhaus Project and keep the list's date and copyright lines with - the data. Refreshed every `SWWAF_BLOCKLIST_REFRESH` (default `24h`, never less - than `1h`). The last good copy is kept whole, comment lines included, on - failure and across restarts. + on its DROP page: + - automated downloads must be at least one hour apart, and once a day is + more than enough in most cases; + - a product that uses it must credit The Spamhaus Project and keep the + list's date and copyright lines with the data. +- `SWWAF_BLOCKLIST_REFRESH` (default `24h`, never less than `1h`): how often + each list is fetched again. The last good copy is kept whole, comment lines + included, on failure and across restarts. - `SWWAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:` or `log`. `deny` refuses every request from a listed address, which suits lists of networks that send nothing legitimate, such as DROP. `limit:` gives @@ -740,17 +678,16 @@ address, downloads would often come less than an hour apart. queried, so the budget is spent on suspects. - `SWWAF_CROWDSEC_LAPI_URL` and `SWWAF_CROWDSEC_LAPI_KEY`: optional. If a CrowdSec engine exists on the host, its decision list is pulled on a schedule - and listed addresses are treated as banned. This is how a fleet-wide blocklist - can arrive without `smallwebwaf` depending on CrowdSec. The list is kept like - a fetched blocklist. A listed client's request makes a ban with the cause - `crowdsec` that lasts as long as CrowdSec's decision, so a long list does not - fill `bans.json`. + and treated as a blocklist (see "Bans" for the ban it makes). This is how a + fleet-wide blocklist can arrive without `smallwebwaf` depending on CrowdSec. - `SWWAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:` or `log`, for DNS blocklist and AbuseIPDB hits. Such verdicts are less certain than a blocklist, so by default a listed client gets a quarter of every limit and is banned after a quarter of the requests. - `SWWAF_REPUTATION_CACHE_TTL` (default `24h`) and `SWWAF_REPUTATION_TIMEOUT` (default `2s`). +- A source that is down or over quota gives no verdict. The service continues, + with one `source_failure` alert per cooldown. ### Alerting @@ -763,14 +700,17 @@ address, downloads would often come less than an hour apart. - `SWWAF_ALERT_EVENTS` (default `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): which event types are sent. `source_failure` is a reputation source or GeoJS - failing or refusing `smallwebwaf`. `file_error` is a rule file or state file - edited while running that does not parse, a replacement lookup database that - cannot be read, or a state file that cannot be written. + failing or refusing `smallwebwaf`. `file_error` is one of: + - a rule file or state file edited while running that does not parse; + - a replacement lookup database that cannot be read; + - a state file that cannot be written. - `SWWAF_ALERT_COOLDOWN` (default `15m`): the same event for the same netblock, source, file or anomaly scope is not repeated within this time. A count of suppressed repeats is included in the next one. - `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): beyond this, alerts are rolled into one summary per hour, so a wide attack cannot flood the channel. +- A destination that is down is retried with backoff from a bounded queue, the + oldest alert dropped first, and the drops counted in the metrics. ### Anomaly thresholds @@ -799,15 +739,16 @@ which the metrics show. netblock as a whole. - Anomaly counting includes clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS`, since an exempt client misbehaving is worth - knowing about. + knowing about. At most 20,000 counters are kept, in memory and in + `alerts.json`. ## Rule files `smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR` and checks each -request against them. The files are plain text meant to be edited by hand, so -that a new scanning pattern seen in the request log can be turned into a rule in -one line. `smallwebwaf` watches the directory: a file edited, added or removed -there takes effect while it runs. +request against them. The files are plain text meant to be edited by hand. A new +scanning pattern seen in the request log can be turned into a rule in one line. +`smallwebwaf` watches the directory: a file edited, added or removed there takes +effect while it runs. - One rule per line, four fields separated by spaces or tabs. The fourth field runs to the end of the line: @@ -834,12 +775,10 @@ there takes effect while it runs. - `log`: note the match in the request log and do nothing else. - `block`: refuse the request with `403`. The client is not banned for it, but the refusal counts toward the error burst. - - `ban`: the request is a clear sign of attack. Refuse it and ban the - client's netblock for `SWWAF_ATTACK_BAN_DURATION`. Any further request - during the ban, or a later clear sign of attack, makes the ban permanent. - Use it only for requests no real visitor sends. Anchor a path at the site - root (`^/`): a file of the same name deeper in a site can be ordinary - content, such as a file in a gitea repository. + - `ban`: the request is a clear sign of attack (see "Bans"). Use it only for + requests no real visitor sends. Anchor a path at the site root (`^/`): a + file of the same name deeper in a site can be ordinary content, such as a + file in a gitea repository. - `regex`: Go regular expression syntax (RE2). It has no backreferences or lookaround. In exchange matching time is linear in the input, so no rule can be made to stall the proxy. `(?i)` at the front makes a rule ignore case. A @@ -849,22 +788,20 @@ there takes effect while it runs. rules in line order. - Rules are compiled when their file is read and held in memory. At start, a line that does not parse, a regex that does not compile, or a duplicate id - stops the process with a message naming the file and line. While running, the - same faults leave the rules as they were, including the earlier version of - that file. The log and one `file_error` alert name the file and line. Once the - file is fixed, it is read again. A `SWWAF_RULES_DIR` that does not exist stops - the start with a message naming it. An empty directory is not an error: the - log says that no rules were loaded. To run without rule files, mount an empty - directory or set `SWWAF_RULES_ENABLED=false`. + stops the process with a message naming the file and line. So does a + `SWWAF_RULES_DIR` that does not exist. While running, the same faults leave + the rules as they were, and the log and one `file_error` alert name the file + and line. Once the file is fixed, it is read again. An empty directory is not + an error. To run without rule files, mount an empty directory or set + `SWWAF_RULES_ENABLED=false`. - The image ships a default file in `SWWAF_RULES_DIR`. The app's Dockerfile can copy files of its own into it, beside the default. Mounting a directory over - it replaces the defaults; mounting single files into it adds to them. Docker - does not show a single mounted file being replaced on the host, which is how - many editors save. So rules meant to be edited while `smallwebwaf` runs belong - in a mounted directory, with a copy of the default file if the defaults are to - stay. -- In `SWWAF_MODE=observe` every action is logged as what would have happened and - nothing is refused. + it replaces the defaults; mounting single files into it adds to them. Many + editors save a file by replacing it, and a container keeps seeing the old one + when the file alone is mounted. So rules meant to be edited while + `smallwebwaf` runs belong in a mounted directory, with a copy of the default + file if the defaults are to stay. The lookup database is mounted the same way, + for the same reason. - Clients in `SWWAF_ALLOW_NETS` are not checked. The default file the image ships: @@ -887,16 +824,13 @@ path-traversal uri block (\.\./){2,} empty-agent user_agent log ^$ ``` -It is kept short and limited to patterns that are wrong for every app. Its path -rules are anchored at the site root. In front of gitea, `/.env` is a probe, +It is kept short and limited to patterns that are wrong for every app, with its +path rules anchored at the site root. In front of gitea, `/.env` is a probe, while `///src/branch/main/.env.example` is a file in a repository -that any visitor or search crawler may open. The Core Rule Set's rules for file -names and extensions (930130, 920440) refuse such names anywhere in a path, -which is why the default `SWWAF_WAF_DISABLED_RULES` switches them off: a code -forge serves such files deeper in its paths. Anything app-specific belongs in a -file the app's Dockerfile adds or the deployer mounts. A request for -`/wp-login.php`, for example, is a clear sign of attack in front of gitea and an -ordinary login in front of WordPress: +that any visitor may open. Anything app-specific belongs in a file the app's +Dockerfile adds or the deployer mounts. A request for `/wp-login.php`, for +example, is a clear sign of attack in front of gitea and an ordinary login in +front of WordPress: ``` # 50-gitea.rules: WordPress probes, which gitea never serves @@ -908,16 +842,18 @@ wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/) All state lives in memory, and the files in `SWWAF_STATE_DIR` hold a copy of all of it, so an orderly stop loses nothing. The one exception is log lines still waiting to be sent to `SWWAF_LOG_REMOTE_URL`, which stdout has already carried. -No database is used, and nothing is read from disk while serving a request. The -files are meant for people as well: an admin can read or edit them at any time, -and the running `smallwebwaf` takes the edit in. +The files are meant for people as well: an admin can read or edit them at any +time, and the running `smallwebwaf` takes the edit in. The files, each holding one kind of state: - `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf` - made and every ban an admin made. Per entry: the netblock, start, expiry - (`null` for permanent), the cause (`attack`, `limit`, `admin` or `crowdsec`), - a short reason, when it was lifted if an admin lifted it, and the notes below. + made and every ban an admin made. Per entry: + - the netblock, start and expiry (`null` for permanent); + - the cause: `attack`, `limit`, `admin` or `crowdsec`; + - a short reason; + - when it was lifted, if an admin lifted it; + - the notes below. - `clients.json`: per client, its minute, hour and day counters and its history since it was first seen. The history holds: - the first and last time seen; @@ -931,16 +867,17 @@ The files, each holding one kind of state: survives a restart. - `lookups.json`: GeoJS answers, one per client: the AS number, AS name and - country, when GeoJS was asked and when the answer was last used. Up to - 100,000, each for 7 days. + country, when GeoJS was asked and when the answer was last used. - `reputation.json`: the last good copy of each list fetched from a URL; the cached DNS blocklist verdicts and AbuseIPDB scores, each with the time it was fetched; and the AbuseIPDB checks spent today, so that a restart does not reset the daily budget. -- `alerts.json`: the anomaly counters; for each event and netblock, source, file - or anomaly scope, when it was last sent and how many repeats the cooldown has - held back since; the alerts sent this hour; and the alerts still waiting to be - sent. +- `alerts.json`: + - the anomaly counters; + - for each event and netblock, source, file or anomaly scope, when it was + last sent and how many repeats the cooldown has held back since; + - the alerts sent this hour; + - the alerts still waiting to be sent. Ban notes hold what an admin needs to decide whether to lift a ban, drawn from what `smallwebwaf` already knows. Nothing is looked up to fill them: @@ -948,35 +885,37 @@ what `smallwebwaf` already knows. Nothing is looked up to fill them: - the AS number, AS name and country, added when the lookup answers (empty when lookups are off); - what was broken: the rule id and target that matched, or the trap path, or the - limit, its window, the count reached and the client's limit percentage with - what set it; and any reputation sources that listed the client; + limit, its window and the count reached; +- the client's limit percentage, with the setting that gave it; +- the reputation sources that listed the client; - the request that caused the ban: time, method, host, path with its query string, status and user agent, each text cut to 256 bytes; -- how many requests counted toward the ban, and the window over which they came; - the netblock's total requests since it was first seen, and the requests refused under this ban so far, kept up to date while the ban lasts; - how many earlier bans of each kind the netblock has had. -Size and disk writes. Each file is rewritten whole, so each has a bound: +Each file is rewritten whole, so each has a bound. `clients.json` takes about 1 +KiB per client, about 20 MiB at the default `SWWAF_MAX_TRACKED_CLIENTS`, which +written every 15 minutes is under 2 GiB of disk writes a day. `bans.json` takes +about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at the default +`SWWAF_MAX_BANS`. With a full file, a hundred new bans a day come to about 600 +MiB of writes. `lookups.json` takes about 150 bytes per answer, about 15 MiB +when full, under 1.5 GiB of writes a day. The other two files are usually a few +MiB or less. Writing a file of these sizes takes well under a second on an +ordinary disk, in the background, so rewriting whole files needs nothing +cleverer. -- `clients.json` takes about 1 KiB per client. On a public service the file - grows to the default `SWWAF_MAX_TRACKED_CLIENTS` of 20,000, about 20 MiB. - Written every 15 minutes, that is under 2 GiB of disk writes a day. -- `bans.json` takes about 1.2 KiB per ban and at most about 2.5 KiB, since the - notes hold one request and their texts are cut short. At the default - `SWWAF_MAX_BANS` of 5,000 it is about 6 MiB, and never more than about 12 MiB, - plus whatever bans an admin made. -- `lookups.json` takes about 150 bytes per answer, about 15 MiB when full. -- `reputation.json` and `alerts.json` are usually a few MiB or less. -- Writing a file of these sizes takes well under a second on an ordinary disk, - in the background, so rewriting whole files needs nothing cleverer. +Format: -Format: indented JSON with a top-level `version` number, entries sorted by -client address, times in RFC 3339 UTC, durations and sizes as plain numbers with -the unit in the field name. The aim is that a person can open `bans.json` in an -editor, find an address, and remove or add an entry. `clients.json` and -`lookups.json` put each entry on one line instead. That halves their size and -lets `grep` show everything about one client. +- indented JSON with a top-level `version` number; +- entries sorted by client address; +- times in RFC 3339 UTC; +- durations and sizes as plain numbers, with the unit in the field name; +- `clients.json` and `lookups.json` put each entry on one line instead. That + halves their size and lets `grep` show everything about one client. + +The aim is that a person can open `bans.json` in an editor, find an address, and +remove or add an entry. Writing: @@ -985,10 +924,10 @@ Writing: - write to a temporary file in the same directory, sync it, rename it over the real name, sync the directory. A crash at any point leaves either the old complete file or the new complete file, never a partial one; -- `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted - or made permanent. Changes that only update the counts in its notes wait for - the write every `SWWAF_STATE_COUNTER_INTERVAL`, as the other four files do. - All five are written on orderly shutdown (`SIGTERM`); +- `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban changes. Changes + that only update the counts in its notes wait for the write every + `SWWAF_STATE_COUNTER_INTERVAL`, as the other four files do. All five are + written on orderly shutdown (`SIGTERM`); - before writing a file, `smallwebwaf` checks whether it changed on disk since it last read or wrote it. If it did, `smallwebwaf` takes that edit in first, so an admin's edit is never overwritten. @@ -1002,6 +941,7 @@ Reading at start: with a message naming the file and position. Starting with empty state would silently forgive every repeat offender. Since writes are atomic, a broken file can only come from a hand edit, which the editor should hear about. +- `SWWAF_STATE_DIR` not writable at start: the process exits. Edits while running: @@ -1019,16 +959,21 @@ Edits while running: logs the file and the position of the error, and sends them in one `file_error` alert. The admin fixes the `.bad` file and moves it back. -`SWWAF_STATE_DIR` not writable at start: the process exits. A write that fails -while running: state stays correct in memory, the failure is logged, counted in -the metrics, sent as a `file_error` alert once per cooldown, and retried at the -next write. +A write that fails while running: -What a hard kill can lose: ban changes from the last `SWWAF_STATE_WRITE_DELAY` -(10 seconds), and everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` -(15 minutes): counts and history, the counts in ban notes, GeoJS answers, -reputation verdicts and the AbuseIPDB count, anomaly counters and alert -cooldowns. Losing the volume loses ban history, not service. +- state stays correct in memory; +- the failure is logged and counted in the metrics; +- one `file_error` alert is sent per cooldown; +- the write is retried at the next write. + +What a hard kill can lose: + +- ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds); +- everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes): + counts and history, the counts in ban notes, GeoJS answers, reputation + verdicts and the AbuseIPDB count, anomaly counters and alert cooldowns. + +Losing the volume loses ban history, not service. ## Request log @@ -1091,40 +1036,44 @@ log handling while `docker logs` keeps working. ## Metrics endpoint Prometheus text format at `/_smallwebwaf/metrics`, for a request carrying -`SWWAF_METRICS_TOKEN`. While the token is unset, the metrics are off. The token -is separate from `SWWAF_ADMIN_TOKEN`, so a scraper that holds it cannot manage -bans. +`SWWAF_METRICS_TOKEN`. - Traffic: requests and bytes in and out, by status class and `action`; request duration and upstream duration histograms; requests in flight. -- Limits and bans: limit hits by window and kind (requests, bytes or refusals); - size and time limit hits by limit; offences by kind; bans made by cause; - permanent bans; active bans; requests refused by the country lists, by - country. -- Attack detection: rule file matches by rule id and action, and the number of - rules loaded; Core Rule Set matches by mode and rule id, limited to the rules - that actually fired. -- Lookup and reputation: requests and bytes by AS number and by country, limited - to the `SWWAF_METRICS_TOP_N` busiest of each with the rest summed as `other`, - so the label set stays bounded; reputation queries, hits, failures and - remaining daily budget by source; GeoJS requests and failures, and clients - that counted as unknown because it did not answer in time; when each list was - last fetched and the lookup database last read. -- Housekeeping: tracked clients; state file writes, write failures, last - successful write time and size per file; edits taken in, and edits set aside - because they did not parse; alerts sent, failed, suppressed and dropped by - destination; remote log lines sent, dropped and buffer depth; the standard Go - runtime and process metrics. +- Limits and bans: + - limit hits by window and kind (requests, bytes or refusals); + - size and time limit hits by limit; + - offences by kind; + - bans made by cause, permanent bans, active bans; + - requests refused by the country lists, by country. +- Attack detection: + - rule file matches by rule id and action, and the number of rules loaded; + - Core Rule Set matches by mode and rule id, limited to the rules that + fired. +- Lookup and reputation: + - requests and bytes by AS number and by country, limited to the + `SWWAF_METRICS_TOP_N` busiest of each with the rest summed as `other`, so + the label set stays bounded; + - reputation queries, hits, failures and remaining daily budget by source; + - GeoJS requests and failures, and clients that counted as unknown because + it did not answer in time; + - when each list was last fetched, and the lookup database last read. +- Housekeeping: + - tracked clients; + - state file writes, write failures, last successful write time and size per + file; edits taken in, and edits set aside because they did not parse; + - alerts sent, failed, suppressed and dropped by destination; + - remote log lines sent, dropped and buffer depth; + - the standard Go runtime and process metrics. - No metric carries a client IP address as a label. Per-address questions are answered by the request log and `GET /_smallwebwaf/clients/`. ## Admin endpoints -`smallwebwaf` has one listener, on port 8080. A request whose path starts with -`/_smallwebwaf/` is for `smallwebwaf` itself: it answers it and never passes it -to the app. The prefix carries the tool's name, so it takes no path an app uses. -Admins and scrapers reach these endpoints through traefik, like any other -request. +A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself: +it answers it and never passes it to the app. The prefix carries the tool's +name, so it takes no path an app uses. Admins and scrapers reach these endpoints +through traefik, like any other request. - `GET /_smallwebwaf/healthz`: answers `200` with `ok` to anyone, without a token, while `smallwebwaf` is running. It does not ask the app; the @@ -1139,9 +1088,9 @@ request. current counters and history, which holds its lookup result and offences, and its bans with their notes. It answers "why was this address refused" and "what has this netblock been doing". -- A token is sent as `Authorization: Bearer `. One set shorter than 32 - characters stops the start. While a token is unset, the endpoints that need it - answer `404`, as does any other path under the prefix. +- A token is sent as `Authorization: Bearer ` (see "Tokens" under + "Settings"). While a token is unset, the endpoints that need it answer `404`, + as does any other path under the prefix. - Apart from the health check, these requests go through every check that any other request goes through. They are answered at the point where another request would be forwarded to the app: a banned or refused client stays @@ -1169,42 +1118,27 @@ One JSON object per alert: ## Deployment `smallwebwaf` runs inside the app's own container: the app's Dockerfile builds -`FROM` the `smallwebwaf` image. Each app remains one container, deployed as -before. An app is hardened just by changing its base image, and perhaps -installing on top of it, from nixpkgs, the packages it needs. In the container, -`smallwebwaf` listens on port 8080, which traefik routes to, and forwards each -request to the app on `127.0.0.1:8081`, its default `SWWAF_UPSTREAM_URL`. No -setting is required. +`FROM` the `smallwebwaf` image. Each app remains one container. An app is +hardened by changing its base image. It can add packages from nixpkgs. The image holds: - Ubuntu 26.04 LTS, the newest long-term support release of Ubuntu, pinned by digest. The image moves to the next LTS release when that ships. -- Three packages from Ubuntu, `ca-certificates`, `nix-bin` and `runit`, - installed from a dated snapshot of Ubuntu's archive and checked by hash (see - "Packages from Ubuntu"). The Ubuntu image has no CA certificates. They come - with `nix-bin` only because a library it uses recommends them, so - `ca-certificates` is installed by name. Without it Nix cannot download - packages, and `smallwebwaf`, unable to reach GeoJS, would count every visitor - as coming from an unknown country. -- Nix, the package manager, from Ubuntu's own `nix-bin` package, and nixpkgs, - the collection of packages Nix installs from, fixed at one commit (see - "Packages from nixpkgs"). Root uses Nix directly, and no Nix daemon runs in - the container. Nix run by root expects a group of build users, `nixbld`, which - `nix-bin` does not create. So the image writes `build-users-group =` to - `/etc/nix/nix.conf`, and root's builds then run without build users. -- runit, from Ubuntu's own `runit` package, and `runsvinit` as the entrypoint, - as the owner's code style guide asks for service containers. Neither Ubuntu - nor nixpkgs packages `runsvinit`, so the image builds it from its source - (`github.com/peterbourgon/runsvinit`) at a fixed commit hash. Its repository - is archived and has not changed since 2015, and its last tag is `v2.0.0`. It - has no `go.mod`, and `go build` of its directory needs one, so the build - writes one. Since `runsvinit` uses only Go's standard library, that file names - nothing else. `runsvinit` starts runit's `runsvdir`, which starts a `runsv` - for each directory under `/etc/service`. Each `runsv` runs the `run` script in - its directory, and runs it again whenever it exits. Ubuntu's runit looks for - services in `/etc/service` too, so when `docker stop` has `runsvinit` stop - each service with runit's `sv`, `sv` finds it. +- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, installed from a dated + snapshot of its archive and checked by hash (see "Pinning"). The Ubuntu image + has no CA certificates; without them Nix cannot download packages and + `smallwebwaf` cannot reach GeoJS. +- Nix and nixpkgs, fixed at one release (see "Pinning"). Root uses Nix directly; + no Nix daemon runs, and `build-users-group =` in `/etc/nix/nix.conf` lets root + build without the build users `nix-bin` does not create. +- runit, and `runsvinit` as the entrypoint, as the owner's code style guide asks + for service containers. No package carries `runsvinit`, so the image builds it + from its archived source (`github.com/peterbourgon/runsvinit`, unchanged + since 2015) at a fixed commit hash, with a `go.mod` the build writes. + `runsvinit` starts runit's `runsvdir`, which runs the `run` script of each + directory under `/etc/service` and runs it again whenever it exits. Ubuntu's + `sv` looks for services there too. - The `smallwebwaf` binary, and a user of its own, `smallwebwaf` (uid and gid 65532). - The service directory `/etc/service/smallwebwaf`. Its `run` script waits one @@ -1227,10 +1161,10 @@ Beyond its `FROM` line, the app's Dockerfile adds: - any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.`; - its runit service: a directory under `/etc/service` named after the app, never `smallwebwaf`. Its `run` script waits one second (`sleep 1`) and then starts - the app with `chpst`, listening on `127.0.0.1:8081`, as a user of its own that - the Dockerfile creates with `useradd`. That user is neither root nor - `smallwebwaf`, so that the app cannot touch the state files or the process of - `smallwebwaf`. + the app with `chpst`, listening on `127.0.0.1:8081`. The app runs as a user of + its own, which the Dockerfile creates with `useradd`. That user is neither + root nor `smallwebwaf`, so that the app cannot touch the state files or the + process of `smallwebwaf`. It sets no `ENTRYPOINT` or `USER` of its own. The container must start `runsvinit`, as root, so that it can start each service as its own user. @@ -1269,90 +1203,53 @@ main() { main "$@" ``` -### Packages from Ubuntu +### Pinning -The image installs `ca-certificates`, `nix-bin` and `runit` from Ubuntu's -snapshot service, which serves the archive as it was at a given moment. The -archive itself changes with every update, so its packages are not pinned. The -image's Dockerfile names that moment, in apt's `--snapshot 20261001T000000Z` -form, on both `apt-get update` and `apt-get install`. That moment is never -earlier than the date of the pinned Ubuntu image, since packages from an older -snapshot can need older versions of packages the Ubuntu image already holds. It -moves forward whenever that image's digest does. - -`apt-get update` keeps the snapshot's `InRelease` files, which apt checks -against the archive's signature, in `/var/lib/apt/lists/`. Each lists the -SHA-256 hash of the package lists it covers, and each package list the hash of -every package in it. The Dockerfile also names the SHA-256 hash of each of the -snapshot's `InRelease` files. The build checks them after `apt-get update` and -before `apt-get install`, so every package apt installs is checked, through -those files, against hashes the Dockerfile names. `apt-get update` also fetches -the live archive's `InRelease` files into the same directory. Their hashes -change whenever the archive does, and the install does not use them, so the -check leaves them out. The hashes are those of the archive for amd64, and so the -image is built for amd64: other architectures use Ubuntu's ports archive, whose -`InRelease` files differ. - -The snapshot service is reached over HTTPS, and the Ubuntu image has no CA -certificates of its own. So this one install uses those of the Go image that -`smallwebwaf` is built in, which is pinned by digest too: apt's -`Acquire::https::CaInfo` option names that image's CA certificate file, -`/etc/ssl/certs/ca-certificates.crt`, mounted for that one step. - -### Packages from nixpkgs - -nixpkgs is fixed at one commit of its newest release branch, `nixos-26.05` -today. For each commit of the branch that has passed its tests, the Nix project -publishes a release on `releases.nixos.org`, such as -`nixos-26.05.11045.774debe7a0d1`. The image takes nixpkgs from that release's -file `nixexprs.tar.xz`, not from a GitHub archive of the commit, whose bytes can -change. The image's Dockerfile names the release and the SHA-256 hash of that -file, which the release's page lists. The build checks the hash before unpacking -it. - -nixpkgs is set up for root under the name `nixpkgs`, so the app's Dockerfile -installs a package with `nix-env -iA nixpkgs.`. Whatever it installs is on -the `PATH` of every service. The image adds root's Nix profile, -`/nix/var/nix/profiles/default/bin`, at the end of the `PATH`, after Ubuntu's -own directories, so that no package hides the image's own commands. busybox, for -one, brings its own `sv`, which looks for services elsewhere. Because nixpkgs -stays at that commit, an app built on the same `smallwebwaf` image gets the same -packages each time it is built. Unpacked, nixpkgs takes about 500 MiB of disk, -more on some filesystems such as ZFS. Each package an app installs from it adds -its own size, with everything it depends on. A newer commit of the branch, with -its security fixes, comes with a newer `smallwebwaf` image, as do Ubuntu's own -fixes. An app takes them by changing the digest in its `FROM` line. When nixpkgs -makes its next release, every six months, the image moves to that release's -branch. +- Ubuntu's packages come from Ubuntu's snapshot service, the archive as it was + at the moment the Dockerfile names (`--snapshot 20261001T000000Z`), never + earlier than the pinned Ubuntu image's date. The Dockerfile names the SHA-256 + hash of each of the snapshot's `InRelease` files, which cover every package + list and so every package, and the build checks them before installing. The + hashes are the amd64 archive's, so the image is built for amd64 only. The + snapshot is fetched over HTTPS with the CA certificates of the Go image that + builds `smallwebwaf`, itself pinned by digest, since the Ubuntu image has + none. +- nixpkgs comes from a release file on `releases.nixos.org` (`nixexprs.tar.xz` + of `nixos-26.05.11045.774debe7a0d1`), named with its SHA-256 hash, not from a + GitHub archive, whose bytes can change. It is set up for root as `nixpkgs`. + Root's Nix profile, `/nix/var/nix/profiles/default/bin`, is last on the + `PATH`, so that no package hides the image's own commands; busybox's `sv`, for + one, looks for services elsewhere. Unpacked, nixpkgs takes about 500 MiB. +- Security fixes from Ubuntu and nixpkgs come with a newer `smallwebwaf` image, + which an app takes by changing the digest in its `FROM` line. The image moves + to each new nixpkgs release, every six months. ### The two processes -- `runsvinit` is the container's first process and runs as root, as do runit's - `runsvdir` and the `runsv` that watches each service. Each `run` script starts - as root and hands over to its program's user with `chpst`: `smallwebwaf` runs - as `smallwebwaf`, the app as its own user. -- Both start with the container's environment variables, which is why every - setting of `smallwebwaf` carries the `SWWAF_` prefix. Both write to the - container's output, which `docker logs` shows: the JSON lines of `smallwebwaf` - and whatever the app writes. -- When `smallwebwaf` or the app exits, for whatever reason, runit runs its `run` - script again, which waits one second before starting it. The other process - keeps running, and the container does not stop. While `smallwebwaf` is down, - nothing answers on port 8080 and traefik answers `502`. While the app is down, - `smallwebwaf` answers `502`. -- A `smallwebwaf` that stops at start, for example on invalid configuration, is - started again every second and stops again, logging its message each time, - until the cause is fixed. The health check fails meanwhile. +- `runsvinit`, `runsvdir` and each service's `runsv` run as root. Each `run` + script starts as root and hands over to its program's user with `chpst`. Both + services start with the container's environment variables and write to the + container's output, which `docker logs` shows. +- When `smallwebwaf` or the app exits, runit starts it again a second later. The + other process keeps running, and the container does not stop. While + `smallwebwaf` is down, traefik answers `502`. While the app is down, + `smallwebwaf` answers `502`, which is not a client offence. +- Only four things stop `smallwebwaf`, all at start: + - invalid configuration; + - a lookup database that is missing or unreadable; + - a rule file or state file that does not parse; + - an unwritable `SWWAF_STATE_DIR`. +- A `smallwebwaf` that stops at start is started again every second, logging its + message each time, until the cause is fixed. The health check fails meanwhile. + Once running, a broken helper or a broken edit never takes the protected + service down. - `docker stop` sends `SIGTERM` to `runsvinit`, which has runit stop both services at once, each with `SIGTERM`, and exits once they have stopped. `smallwebwaf` stops taking requests, gives those in progress five seconds to - finish, and writes its state files as it stops. The requests in progress can - finish only while the app still answers. An app that exits at once on - `SIGTERM` ends them: the client gets `502`, or a response cut short if the app - had already begun to answer. An app that should let them finish keeps taking - and answering requests for the whole five seconds after `SIGTERM`, not only - the ones it already has. `smallwebwaf` can still pass it requests it took - before then. + finish, and writes its state files. Those requests can finish only while the + app still answers. An app that exits at once on `SIGTERM` ends them: the + client gets `502`, or a response cut short. To let them finish, the app must + keep taking and answering requests for five seconds after `SIGTERM`. - The container's root filesystem stays writable: runit writes each service's status into its directory under `/etc/service`. @@ -1370,33 +1267,27 @@ image's `HEALTHCHECK` with one that checks both. ### Ports -`smallwebwaf` listens on port 8080 on every address and on no other port. Its -health check, metrics and ban management are all on that listener, under -`/_smallwebwaf/`. `SWWAF_LISTEN_ADDR` may set another port. The image's health -check takes its port from that setting, traefik's port label -(`traefik.http.services..loadbalancer.server.port`) must name the same -port, and the app must leave that port free. The address part of -`SWWAF_LISTEN_ADDR` stays empty (for example `:9000`, never `127.0.0.1:9000`), -so `smallwebwaf` keeps listening on every address: traefik reaches it on the -container's address, and the health check on `127.0.0.1`. The app listens on -`127.0.0.1:8081` only, so that nothing outside the container reaches it except -through `smallwebwaf`. An app that listens on every address can be reached +`smallwebwaf` listens on no port but the one in `SWWAF_LISTEN_ADDR`, and its own +endpoints are on that listener. Another port may be set: the health check +follows the setting, traefik's port label +(`traefik.http.services..loadbalancer.server.port`) must name it, and the +app must leave it free. The address part of `SWWAF_LISTEN_ADDR` stays empty +(`:9000`, never `127.0.0.1:9000`), so that traefik reaches `smallwebwaf` on the +container's address and the health check on `127.0.0.1`. The app listens on +`127.0.0.1:8081` only. An app that listens on every address can be reached around `smallwebwaf` by anything that reaches the container. ### State -`smallwebwaf` keeps its state files in `/var/lib/smallwebwaf` -(`SWWAF_STATE_DIR`), a directory of its own beside the app's data, which the app -keeps in directories of its own, such as `/var/lib/app`. Without a volume there, -the files live in the container: they survive a restart of the container and are -lost when a deploy replaces it. A volume mounted at `/var/lib/smallwebwaf`, -named or a host directory, keeps them across deploys. The `run` script of -`smallwebwaf` makes it and every file in it belong to the `smallwebwaf` user. So -a host directory mounted there needs no change of owner, and files an earlier -owner left in it can be read and replaced. It is a volume of its own, separate -from the app's. It holds a few tens of MiB at most with the defaults, and needs -no backup beyond whatever the host already does. The image declares no volume, -since every app image built on it would inherit it. +The app keeps its data in directories of its own, such as `/var/lib/app`, and +`smallwebwaf` keeps its state files beside them in `SWWAF_STATE_DIR`. Without a +volume there, the files survive a restart of the container and are lost when a +deploy replaces it. A volume mounted at `/var/lib/smallwebwaf`, named or a host +directory, keeps them across deploys. The `run` script makes it and every file +in it belong to the `smallwebwaf` user, so a host directory mounted there needs +no change of owner. The volume is separate from the app's, holds a few tens of +MiB at most with the defaults, and needs no backup beyond the host's own. The +image declares no volume, since every app image built on it would inherit it. ### Tokens @@ -1421,250 +1312,69 @@ example. ### upaas An app deployed by upaas takes this shape with no change to upaas. upaas builds -the app's Dockerfile, which now starts `FROM` the `smallwebwaf` image, and runs -the one container as it runs any app, with the app's traefik labels, environment -variables and volumes. The labels route to port 8080 -(`traefik.http.services..loadbalancer.server.port=8080`). Any `SWWAF_` -settings go with the app's environment variables. The volume for -`/var/lib/smallwebwaf` goes beside the app's own, as does the directory that -holds any token file. - -The endpoints of `smallwebwaf` are reached through traefik like any other -request, for example `https://app.example.invalid/_smallwebwaf/metrics` for a -scraper that sends `SWWAF_METRICS_TOKEN`. +the app's Dockerfile and runs the one container as it runs any app. The traefik +labels route to port 8080, and any `SWWAF_` settings go with the app's +environment variables. The volume for `/var/lib/smallwebwaf` and the directory +of any token file go beside the app's own volumes. ### Rollout per service -Build the app's image on the `smallwebwaf` image, with no setting, and deploy it -as before. It protects the app from the first request, and nothing needs tuning -first. Afterwards the request log and the ban notes say why each client was -refused. A real visitor refused by mistake is let through with an exclusion -(`SWWAF_WAF_DISABLED_RULES`, `SWWAF_WAF_EXEMPT_PATHS`, +Build the app's image on the `smallwebwaf` image, with no setting, and deploy +it. Nothing needs tuning first. Afterwards the request log and the ban notes say +why each client was refused. A real visitor refused by mistake is let through +with an exclusion (`SWWAF_WAF_DISABLED_RULES`, `SWWAF_WAF_EXEMPT_PATHS`, `SWWAF_RATE_LIMIT_EXEMPT_PATHS`) or an exemption (`SWWAF_RATE_LIMIT_EXEMPT_NETS`, `SWWAF_ALLOW_NETS`), and its ban is lifted in -`bans.json`. Additions to consider at any time: an alert destination, country -lists or biased thresholds, reputation sources, the metrics and admin tokens, a -remote log endpoint, and an app-specific rule file. +`bans.json`. Additions to consider at any time: -### Notes specific to gitea +- an alert destination; +- country lists or biased thresholds; +- reputation sources; +- the metrics and admin tokens; +- a remote log server; +- an app-specific rule file. -gitea is built on the image like any other app. - -- SSH access to gitea does not pass through `smallwebwaf` and is not protected - by it. -- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all - but the largest repositories and slowest links. A push is a request, and so is - every other upload: a git push, an LFS object, a container image layer, a - package file or a release attachment. At the defaults, an upload larger than - 100 MiB or taking more than 60 seconds is cut off. The client is answered - `413` for a body that is too large, before anything reaches gitea when the - request announces its size. It is answered `408` for a body it sends too - slowly, or `504` if gitea is too slow to take it. The upload fails, and no one - is banned for it. A gitea that takes large uploads needs - `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and - `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does not - read an upload's body, which streams through without being held in memory. -- At the defaults, the Core Rule Set lets gitea's ordinary use through, apart - from the refusals in the next note: - - browsing and views of files in a repository, with their history, blame and - the file tree; - - diffs, including their hidden lines and large files, and pull request - review; - - git's clone, fetch and push over HTTP; - - signing in, including the return to the page a visitor came from, and - sign-in with Git Credential Manager, git-credential-oauth or tea; - - the API's calls for a file and its commits; - - pushing and pulling container images and packages; - - Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`; - - posting issues, pull requests, comments, wiki pages and files saved in the - web editor, code included, since no body is read; - - the page gitea shows after a change, with its message naming what was - done. -- The Core Rule Set can still refuse the requests below. Each refusal answers - only that request, with `403`, and bans no one by itself. It counts toward the - error burst, which a person does not reach this way. The request log names the - rule in `waf_rule_ids`, which `SWWAF_WAF_DISABLED_RULES` can switch off. - - A query string that reads to it as an attack, most often a search. The - lists refuse such a name in any other query parameter too, such as a - release attachment uploaded through the API as `docker-compose.yml`. - Examples: - - a name on its lists of system files and commands, such as - `package.json`, `.gitignore` or `docker-compose.yml`; - - text that starts with a command name, or whose first word has one - right after a `/`, such as `python3`, `ssh key` or - `feat/docker-support`; - - a shell command with its options or a system path (`ls -la`, `sed -i`, - `/bin/sh`), or a command in backticks; - - script code (`fetch(`, `${VAR}`, `process.env`), HTML (`