Last pass: the sixth README review and the fifth SPEC review

The spec names every fault that stops the start, runs step 15 for every
request but the health check, and states each of the fourteen repeated
requirements once. docs/alerts.md says which alerts have an empty client
and netblock and what a summary's detail holds; docs/settings.md lists
the settings that need a lookup and points at step 3 for the two static
lists. The README and the spec now travel in one branch, so the README
never names a spec section that does not exist yet.

Model: fable-5-1

SPEC.md

@@ -100,10 +100,12 @@ to them.
 ## What happens to a request
 
 The steps run in this order. The first step that produces a final answer ends
-the checks; step 15 runs for every request. The size and time limits apply to
-the whole exchange. Each step names the section that holds its rules.
+the checks; step 15 runs for every request but the health check. The size and
+time limits apply to the whole exchange. Each step names the section that holds
+its rules.
 
-1. `GET /_smallwebwaf/healthz` is answered at once, before any check.
+1. `GET /_smallwebwaf/healthz` is answered at once, before any check, so a
+   health checker never needs an exemption and is never refused.
 2. The client is identified.
     - If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, `X-Forwarded-For` is
       walked from the right, and the first address outside those netblocks is
@@ -116,9 +118,9 @@ the whole exchange. Each step names the section that holds its rules.
       around its address, a /64 by default, since one abuser usually controls a
       whole /64. A client is therefore one IPv4 address or one IPv6 group.
 3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12 and the
-   byte limits and the error burst at step 15. It is still counted for the
-   anomaly thresholds, and still needs a token at step 13. A client in
-   `SWWAF_DENY_NETS` is refused; a client in both lists is let through.
+   byte limits and the error burst at step 15. It still needs a token at
+   step 13. A client in `SWWAF_DENY_NETS` is refused; a client in both lists is
+   let through.
 4. Bans. A client under a ban is refused (see "Bans").
 5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
    and country"). The request goes on at once. It waits for the first GeoJS
@@ -134,8 +136,6 @@ the whole exchange. Each step names the section that holds its rules.
    gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is
    banned. A client listed by a DNS blocklist, or scored at least
    `SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`.
-   With no cached verdict, a query is queued in the background and the request
-   goes on, so a first request is never delayed by a reputation query.
 8. The client's limit percentage (see "Biased thresholds").
 9. Rate limiting (see "Request rate limits, per client"). A request over a limit
    times the percentage is refused and bans the client.
@@ -184,7 +184,7 @@ the token check still apply.
   and the anomaly counters have fixed bounds, listed under "Fixed limits" in
   `docs/settings.md`. Past its bound, the GeoJS answer or anomaly counter used
   longest ago goes first, and the verdict or score fetched longest ago. Past
-  1,000 DNS queries under way, a zone is asked at the client's next request.
+  that bound, a zone is asked at the client's next request.
 
 ## Bans
 
@@ -193,8 +193,7 @@ Until a ban ends, every request from its netblock is refused with
 `SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one
 address or one /64, so a permanent ban does not reach neighbours who did
 nothing. Requests refused under a ban are not counted against limits. They are
-counted in the client's history and in the ban's notes. A banned client is not
-looked up.
+counted in the client's history and in the ban's notes.
 
 An offence is a request `smallwebwaf` holds against the client. It is one of:
 
@@ -367,8 +366,8 @@ Conventions:
   owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and
   byte limits only; the error burst, attack detection and bans still apply.
 - Many clients share one address (mobile carriers, offices, Tor), and so share
-  its limits and its bans. The defaults sit well above one person's use, and the
-  first two lists take known shared addresses out.
+  its limits and its bans. `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS`
+  take known shared addresses out.
 
 ### Request rate limits, per client
 
@@ -442,12 +441,8 @@ service that keeps its visitors' addresses on its own host.
 
 - Only one source is used. The start stops, naming both settings, when `file` is
   set without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other
-  source. The start also stops when the source is `off` and one of these needs
-  lookups:
-    - a country list;
-    - a biased threshold;
-    - `SWWAF_ADD_LOOKUP_HEADERS`;
-    - an anomaly threshold per AS number.
+  source. The start also stops when the source is `off` and a setting that needs
+  a lookup is set (`docs/settings.md` lists them).
 - GeoJS is told the address of every new visitor, whether or not a setting uses
   the answer, since the request log, the metrics and the ban notes carry it.
   These addresses are never sent:
@@ -476,8 +471,8 @@ service that keeps its visitors' addresses on its own host.
   API requests", judged by GeoJS alone, and let it block a caller. While GeoJS
   is slow, down or refusing `smallwebwaf`, new clients count as unknown. GeoJS
   is then left alone for a second, and twice as long after each further failure,
-  up to five minutes, and one `source_failure` alert is sent per cooldown. A
-  service that cannot accept this uses the file.
+  up to five minutes. Each failure raises a `source_failure` alert (see
+  "Alerting"). A service that cannot accept this uses the file.
 - The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`,
   `as_name` and `country_code` fields are read. The operator downloads it with a
   free IPinfo account and mounts it read-only; `smallwebwaf` never fetches it
@@ -486,9 +481,9 @@ service that keeps its visitors' addresses on its own host.
   on the site that uses it, such as "IP address data is powered by IPinfo".
 - The file is read whole at start; one that is missing or unreadable stops the
   start. It is read again once it has gone 2 seconds without a change after
-  being replaced. One that cannot be read is logged, sent as one `file_error`
-  alert and ignored. The database already loaded stays in use. A client whose
-  address the file does not hold counts as unknown.
+  being replaced. A replacement that cannot be read is logged, sent as one
+  `file_error` alert and ignored. The database already loaded stays in use. A
+  client whose address the file does not hold counts as unknown.
 
 ### Country lists
 
@@ -508,9 +503,8 @@ where it comes from.
   exclusive list already refuses every other country. A code on both lists stops
   the start with a message naming it.
 - A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the
-  client's address has been looked up, before its body is read. It skips the
-  reputation checks, the limits, the rule files and the Core Rule Set. It is
-  logged with the action `country_denied`, and counted in the metrics.
+  client's address has been looked up, before its body is read. It is logged
+  with the action `country_denied`, and counted in the metrics.
 
 ### Biased thresholds
 
@@ -519,10 +513,11 @@ countries. The lists are empty by default, for the same reason as the country
 lists. The request log and the metrics show which AS numbers and countries a
 service's abuse comes from.
 
-- Clients in a listed AS number or country get that percentage of every request
-  and byte limit (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A
-  lowered limit is rounded down. `0` is a zero allowance: the client's first
-  request breaks a limit and bans it.
+- Each list gives AS numbers or countries a percentage, such as `AS14061:50`.
+  Their clients get that percentage of every request and byte limit
+  (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A lowered limit is
+  rounded down. `0` is a zero allowance: the client's first request breaks a
+  limit and bans it.
 - `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` set the byte
   limits' percentage alone, when it should differ from the request limits'. For
   an AS number they list, they replace its percentage from
@@ -556,9 +551,8 @@ code in them for attacks. The default gives up refusing an attack carried in a
 body. The rule files, the limits and the bans still apply to the client that
 sends it.
 
-- The Core Rule Set inspects only a request that every earlier check let
-  through, and skips a path under `SWWAF_WAF_EXEMPT_PATHS`. An engine error on a
-  request forwards the request, and is logged and counted.
+- The Core Rule Set skips a path under `SWWAF_WAF_EXEMPT_PATHS`. An engine error
+  on a request forwards the request, and is logged and counted.
 - `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule
   Set's own two tuning values, at its own defaults. Each level up runs more
   rules, which find more attacks and take more ordinary requests for attacks.
@@ -601,14 +595,18 @@ sends it.
   percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and
   `/%77p-login.php`, and does not match `/wp-login.php/` or
   `/blog/wp-login.php`.
-- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute for a rule
-  file match, a trap path, a Core Rule Set match, or a missing or wrong token;
-  no other refusal counts. More than this breaks a limit (see "Bans"). A client
-  trying one attack or token after another is refused many times a minute; a
-  person rarely more than a few times. Answers the app gives are not counted,
-  since they do not tell a scanner from an ordinary client. In front of gitea,
-  registry and git clients get `404` and `401` by design, hundreds of times a
-  minute.
+- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute of these
+  kinds, and of no other:
+    - a rule file match;
+    - a trap path;
+    - a Core Rule Set match;
+    - a missing or wrong token.
+
+    More than this breaks a limit (see "Bans"). A client trying one attack or
+    token after another is refused many times a minute; a person rarely more
+    than a few times. Answers the app gives are not counted, since they do not
+    tell a scanner from an ordinary client. In front of gitea, registry and git
+    clients get `404` and `401` by design, hundreds of times a minute.
 
 `smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
 gitea it would otherwise refuse ordinary requests. The changes hold in front of
@@ -773,9 +771,8 @@ too close together.
   by default. Such verdicts are less certain than a blocklist, so a listed
   client gets a quarter of every request and byte limit rather than a refusal.
 - Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and
-  in `reputation.json`, with the fixed bounds under "Fixed limits" in
-  `docs/settings.md`. A source that is down or over quota gives no verdict. The
-  service continues, with one `source_failure` alert per cooldown.
+  in `reputation.json`. A source that is down or over quota gives no verdict.
+  The service continues, with one `source_failure` alert per cooldown.
 
 ### Alerting
 
@@ -787,9 +784,8 @@ too close together.
   the ban's alert, with `mode` set to `observe` in its `detail`.
 - A repeat is an alert with the same event, and the same netblock, source, file
   or anomaly scope, as one sent less than `SWWAF_ALERT_COOLDOWN` before. An
-  anomaly scope is the thing counted: one client, netblock, AS number or named
-  netblock, or the whole service. A repeat is held back and counted, and the
-  next such alert sent gives the count.
+  anomaly scope is one of the things counted under "Anomaly thresholds". A
+  repeat is held back and counted, and the next such alert sent gives the count.
 - Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and
   counted by event, so a wide attack cannot flood the channel. Once the hour
   ends, one alert with the event `summary` gives those counts. It also counts
@@ -820,7 +816,7 @@ addresses at low rates each, which the per-client limits do not see. A low
 - Every request but the health check is counted, whatever was done with it,
   clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` included,
   since an exempt client misbehaving is worth knowing about. The counters are
-  kept in memory and in `alerts.json`, with a fixed bound.
+  kept in memory and in `alerts.json`.
 
 ## Rule files
 
@@ -846,10 +842,10 @@ and the default file.
   of every request; a rule naming one stops the start like any bad line. Request
   bodies are not available to rule files; reading them is the Core Rule Set's
   job.
-- `log` only notes the match in the request log. `block` refuses the request
-  without a ban; `ban` makes the request a clear sign of attack (see "Bans"),
-  and is for requests no real visitor sends. A visitor banned by mistake is let
-  back in by lifting the ban.
+- `log` only notes the match in the request log. `block` refuses the request;
+  `ban` makes the request a clear sign of attack (see "Bans"), and is for
+  requests no real visitor sends. A visitor banned by mistake is let back in by
+  lifting the ban.
 - At start, these stop the process with a message naming the file and line:
     - a line that does not parse;
     - a regex that does not compile;
@@ -940,9 +936,8 @@ edit them at any time, and the running `smallwebwaf` takes the edit in.
 
 ## Request log
 
-Every request gets one JSON object on one line on stdout, refused ones included,
-apart from those Go's HTTP server ends before `smallwebwaf` sees them (see "Size
-and time limits"). `docs/request-log.md` lists the fields. Bodies are never
+Every request `smallwebwaf` sees gets one JSON object on one line on stdout,
+refused ones included. `docs/request-log.md` lists the fields. Bodies are never
 logged, and no header but those listed there. Query strings are logged as
 received; an app that carries secrets in query strings needs that fixed in the
 app. The process's own messages (start, the settings, stop, errors) share the
@@ -971,8 +966,7 @@ while `docker logs` keeps working.
 ## Metrics endpoint
 
 `GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request
-carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. Every
-metric carries the label `instance`, set to `SWWAF_INSTANCE_NAME`. No metric
+carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. No metric
 carries a client IP address as a label. Per-address questions are answered by
 the request log and `GET /_smallwebwaf/clients/<ip>`.
 
@@ -980,15 +974,14 @@ the request log and `GET /_smallwebwaf/clients/<ip>`.
 
 A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself:
 it answers it and never passes it to the app. The prefix carries the tool's
-name, so it takes no path an app uses. Admins and scrapers reach these endpoints
-through traefik, like any other request. `docs/admin-endpoints.md` lists the
-endpoints, what each takes and what each answers.
+name, so it takes no path an app uses. Admins and the metrics collector reach
+these endpoints through traefik, like any other request.
+`docs/admin-endpoints.md` lists the endpoints, what each takes and what each
+answers.
 
 - `GET /_smallwebwaf/healthz` answers `200` with `ok` to anyone, without a
   token, while `smallwebwaf` is running. It does not ask the app; the
-  container's health check does. It is answered before any check, so a health
-  checker never needs an exemption and is never refused, for example by
-  `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`.
+  container's health check does.
 - Every other endpoint needs its token, sent as `Authorization: Bearer <token>`
   (see "Tokens" under "Settings"). While a token is unset, the endpoints that
   need it answer `404`, as does any other path under the prefix.
@@ -996,10 +989,9 @@ endpoints, what each takes and what each answers.
   other request goes through. They are answered at the point where another
   request would be forwarded to the app: a banned or refused client stays
   refused, and each request counts toward the client's limits. A missing or
-  wrong token is answered `401`, in `SWWAF_MODE=observe` too, and counts toward
-  the error burst. A client guessing tokens is therefore banned once it passes
-  `SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute.
-- A ban made through the endpoints starts at once, with the cause `admin`.
+  wrong token is answered `401` (see "Attack detection" for what guessing
+  costs).
+- A ban made through the endpoints starts at once.
 
 ## Deployment
 
@@ -1041,26 +1033,26 @@ what the image holds and how each part is pinned.
   2015, at a fixed commit hash, with a `go.mod` the build writes. `runsvinit`
   starts runit's `runsvdir`, which runs the `run` script of each directory under
   `/etc/service`. Ubuntu's `sv` looks for services there too.
-- The image's `run` script for `smallwebwaf` waits one second and makes
-  `SWWAF_STATE_DIR` and every file in it belong to the `smallwebwaf` user (uid
-  and gid 65532). Then it starts `smallwebwaf` as that user with runit's
-  `chpst`.
+- The image's `run` script for `smallwebwaf` makes `SWWAF_STATE_DIR` and every
+  file in it belong to the `smallwebwaf` user (uid and gid 65532). Then it
+  starts `smallwebwaf` as that user with runit's `chpst`.
 
 Every `run` script, the app's included, is a bash script. It starts with
 `#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main`
 function, called on its last line, as the owner's code style guide asks. Ubuntu
-ships bash.
+ships bash. Each waits one second (`sleep 1`) before starting its program, so a
+process that stops at start is started again every second, not in a tight loop.
 
 Beyond its `FROM` line, the app's Dockerfile adds:
 
 - its binary;
 - any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.<name>`;
 - its runit service: a directory under `/etc/service` named after the app, never
-  `smallwebwaf`. Its `run` script waits one second (`sleep 1`) and then starts
-  the app with `chpst`, listening where `SWWAF_UPSTREAM_URL` points (see
-  "Ports"). The app runs as a user of its own, which the Dockerfile creates with
-  `useradd`. That user is neither root nor `smallwebwaf`, so that the app cannot
-  touch the state files or the process of `smallwebwaf`.
+  `smallwebwaf`. Its `run` script starts the app with `chpst`, listening where
+  `SWWAF_UPSTREAM_URL` points (see "Ports"). The app runs as a user of its own,
+  which the Dockerfile creates with `useradd`. That user is neither root nor
+  `smallwebwaf`, so that the app cannot touch the state files or the process of
+  `smallwebwaf`.
 
 It sets no `ENTRYPOINT` or `USER` of its own. The container must start
 `runsvinit`, as root, so that it can start each service as its own user.
@@ -1072,16 +1064,17 @@ It sets no `ENTRYPOINT` or `USER` of its own. The container must start
   script starts as root and hands over to its program's user with `chpst`. Both
   services start with the container's environment variables and write to the
   container's output, which `docker logs` shows.
-- When `smallwebwaf` or the app exits, runit runs its `run` script again, which
-  waits a second before starting it. The other process keeps running, and the
-  container does not stop. While `smallwebwaf` is down, traefik answers `502`.
-  While the app is down, `smallwebwaf` answers `502`, which is not a client
-  offence.
-- Only four things stop `smallwebwaf`, all at start:
+- When `smallwebwaf` or the app exits, runit runs its `run` script again. The
+  other process keeps running, and the container does not stop. While
+  `smallwebwaf` is down, traefik answers `502`. While the app is down,
+  `smallwebwaf` answers `502`, which is not a client offence.
+- Only faults found at start stop `smallwebwaf`:
     - invalid configuration;
     - a lookup database that is missing or unreadable;
     - a rule file or state file that does not parse;
-    - an unwritable `SWWAF_STATE_DIR`.
+    - an unwritable `SWWAF_STATE_DIR`;
+    - a `SWWAF_RULES_DIR` that does not exist or cannot be read;
+    - a `SWWAF_LISTEN_ADDR` it cannot listen on.
 - A `smallwebwaf` that stops at start is therefore started again every second,
   logging its message each time, until the cause is fixed; the health check
   fails meanwhile. Once running, a failing GeoJS, reputation source or alert