README rewritten from scratch so a newcomer can follow it (closes #154, closes #155)

The old README opened with project history and one list of a hundred
settings. The new one leads with what smallwebwaf is, what it protects
against, how to build and deploy it, what happens to a request, what a
refused visitor and the operator see, and how to check it works. The
settings follow, grouped by topic, each with its default and what a bad
value does. The rest is reference. Every statement was checked against
the code on next. Short sentences, bulleted lists in place of comma
chains, no tables, identifiers in backticks.

Model: fable-5-1

README.md

@@ -211,9 +211,11 @@ ends the handling. Nothing of a refused request reaches the app.
 
 Two terms recur below. A clear sign of attack is a request that matches a rule
 whose action is `ban`, or asks for a trap path (steps 9 and 10). An offence is a
-request that counts against its client in the client's history: a clear sign of
-attack, a broken limit, or a refusal by a rule, by the Core Rule Set or for a
-missing or wrong token.
+request that counts against its client in its history. It is one of:
+
+- a clear sign of attack;
+- a broken limit;
+- a refusal by a rule, by the Core Rule Set or for a missing or wrong token.
 
 1. `GET /_smallwebwaf/healthz` is answered `200` with `ok` at once, before any
    check and without asking the app.
@@ -353,8 +355,8 @@ configuration before enforcing it.
   `SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app. Its log
   line has `"would_action":"banned"` and `"rule_ids":["env-file"]`. In `enforce`
   mode the same request bans your address for seven days, and your next request
-  for good. Lift the ban by editing `bans.json` or through the ban endpoint (see
-  "Bans" below).
+  makes the ban permanent. Lift the ban by editing `bans.json` or through the
+  ban endpoint (see "Bans" below).
 - With `SWWAF_METRICS_TOKEN` set,
   `curl -H "Authorization: Bearer $TOKEN" https://<site>/_smallwebwaf/metrics`
   answers with the metrics (see "Metrics" below).
@@ -417,8 +419,8 @@ The forms of values:
   `SWWAF_ALERT_NTFY_URL` is set, it may hold no control character. While
   `SWWAF_LOG_REMOTE_URL` is set and `SWWAF_LOG_REMOTE_APP_NAME` is not, it must
   be 1 to 48 printable ASCII characters without a space.
-- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to refuse nothing
-  and log what would have been refused (see "What happens to a request" above).
+- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to ban no one and
+  log what `enforce` would have done (see "What happens to a request" above).
 - `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`,
   the private address ranges): the netblocks whose `X-Forwarded-For` is
   believed, normally where traefik comes from. Set but empty, it trusts nothing:
@@ -550,8 +552,8 @@ ban.
 Clients of the AS numbers and countries listed here get a percentage of every
 rate limit and byte limit. The same rules then ban them after fewer requests. A
 lowered limit is rounded down. `0` is a zero allowance: the client's first
-request bans it. Each client is counted on its own. No budget is shared by a
-whole AS number or country, which one abuser could use up.
+request bans it. Each client is counted on its own. An AS number or a country
+has no shared budget, since one abuser could use it up for everyone in it.
 
 - `SWWAF_ASN_LIMIT_PERCENT` (default empty): a list of AS numbers, each with `:`
   and a percentage, such as `AS14061:50,AS16276:50,AS45102:25`. An AS number
@@ -622,8 +624,8 @@ do. The defaults judge it by what it does to your service.
 - `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client on
   a DNS blocklist, or one whose AbuseIPDB score is a hit. `deny`,
   `limit:<percent>` or `log`, as for `SWWAF_BLOCKLIST_ACTION`. Such a verdict is
-  less certain than a list like DROP, so by default the client gets a quarter of
-  every limit.
+  less certain than the Spamhaus DROP list, so by default the client gets a
+  quarter of every limit.
 - `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or
   AbuseIPDB's score is used. A duration; it cannot be `off`.
 - `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a
@@ -666,10 +668,11 @@ See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.
       `.git/`, `.gitignore` or `package.json`.
 
     In front of a code forge these refuse git over HTTP, uploads and views of
-    ordinary files. The default rule file bans those probes at the site root
-    instead. A list given replaces the default, so include them. An id must be a
-    whole number above zero. An id from 900000 to 900999 is refused: those are
-    the rules that set the Core Rule Set up, and `smallwebwaf`'s own.
+    ordinary files. The default rule file still bans probes for version control
+    directories and secret files at the site root. A list given replaces the
+    default, so include them. An id must be a whole number above zero. An id
+    from 900000 to 900999 is refused: those are the rules that set the Core Rule
+    Set up, and `smallwebwaf`'s own.
 
 - `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does
   not inspect, matched as `SWWAF_RATE_LIMIT_EXEMPT_PATHS` is.
@@ -794,7 +797,8 @@ thresholds per client are:
 - `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE`;
 - `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR`.
 
-The other scopes have the same four names with `CLIENT` replaced:
+The other scopes' names have another prefix before the same four endings, such
+as `REQUESTS_PER_MINUTE`:
 
 - `SWWAF_ANOMALY_NET_...`: per netblock around a client. Two settings give the
   netblock's length:
@@ -843,6 +847,8 @@ docker run \
     ...
 ```
 
+### Fixed limits
+
 A few limits are fixed rather than settings:
 
 - at most 100,000 GeoJS answers are kept, for 7 days each;
@@ -853,9 +859,13 @@ A few limits are fixed rather than settings:
 ## Bans
 
 A ban refuses every request from a netblock until it ends, with
-`SWWAF_BAN_RESPONSE`. The netblock is the client's IPv4 address, or the netblock
-around it, `SWWAF_BAN_SCOPE_V4_PREFIX` long, or the client's IPv6 group. A
-banned client is not looked up and not counted by rate limiting.
+`SWWAF_BAN_RESPONSE`. The netblock is one of:
+
+- the client's IPv4 address;
+- the netblock around it, `SWWAF_BAN_SCOPE_V4_PREFIX` long;
+- the client's IPv6 group.
+
+A banned client is not looked up and not counted by rate limiting.
 
 `smallwebwaf` makes bans for three causes:
 
@@ -891,8 +901,8 @@ Each ban carries notes for deciding whether to lift it:
   refused;
 - how many earlier bans the netblock had, by cause.
 
-At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept. Past that, one ban is
-dropped: the earliest ban of the netblock that has gone longest without a
+`smallwebwaf` keeps at most `SWWAF_MAX_BANS` of the bans it made. Past that, one
+ban is dropped: the earliest ban of the netblock that has gone longest without a
 request. Bans whose cause is `admin` are never dropped. Those are the bans you
 make, and the bans `smallwebwaf` made that you keep by setting their cause.
 
@@ -996,13 +1006,17 @@ scanner-agent  user_agent  ban     (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
   takes time linear in the text it reads. It matches anywhere in the target
   unless anchored with `^` and `$`. `(?i)` at its front makes it ignore case.
 
-A line that is not a rule stops the start, with a message naming the file and
-line. So does a bad `header:` name, a rule for `Host` or `Transfer-Encoding`, a
-regex that does not compile, or an id used twice. While it runs, `smallwebwaf`
-watches the directory. It reads the files again once the directory has had no
-change for 2 seconds, so that a file still being written is read whole. If the
-files then hold such an error, the rules stay as they were. The log and a
-`file_error` alert name the file and line.
+These stop the start, with a message naming the file and line:
+
+- a line that is not a rule;
+- a bad `header:` name, or a rule for `Host` or `Transfer-Encoding`;
+- a regex that does not compile;
+- an id used twice.
+
+While it runs, `smallwebwaf` watches the directory. It reads the files again
+once the directory has had no change for 2 seconds, so that a file still being
+written is read whole. If the files then hold such an error, the rules stay as
+they were. The log and a `file_error` alert name the file and line.
 
 The image ships one file, `share/rules.d/00-default.rules`, installed as
 `/etc/smallwebwaf/rules.d/00-default.rules`. Its rules:
@@ -1057,14 +1071,18 @@ setting undoes them.
   address or `localhost` (rules 931100 and 934110). Git credential helpers ask
   to be sent back to such a URL.
 - Ten query parameters are not checked against three lists: system files
-  (930120), shell paths (932160) and command names (932260). They are `path`,
-  `files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`,
-  `artifactName` and `redirect_to`. A file such as `.gitignore` or a branch such
-  as `docker-build` then gets through there. So does an attack only those three
-  rules catch, such as `|cat /etc/passwd`. Path traversal and SQL injection are
-  still refused there. Names are matched without regard to case. An app that
-  uses one of these parameters as a file on the server, or passes it to a shell,
-  has to check it itself.
+  (930120), shell paths (932160) and command names (932260). They are:
+    - `path`, `files`, `skip-to` and `sub_path`;
+    - `ref`, `sha`, `branch` and `workflow`;
+    - `artifactName` and `redirect_to`.
+
+    A file such as `.gitignore` or a branch such as `docker-build` then gets
+    through there. So does an attack only those three rules catch, such as
+    `|cat /etc/passwd`. Path traversal and SQL injection are still refused
+    there. Names are matched without regard to case. An app that uses one of
+    these parameters as a file on the server, or passes it to a shell, has to
+    check it itself.
+
 - The cookies `gitea_flash` and `redirect_to` are not read. `Referer` is not
   checked for a Unix command given without arguments (932340), or for Java
   starting a process (944110).
@@ -1122,9 +1140,13 @@ refused ones included:
 {"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
 ```
 
-A field that does not apply to a request is left out. Every line has `type`, the
-fields from `time` to `user_agent`, `request_id`, `peer_ip`, `client_group`,
-`asn`, `as_name`, `country`, `action` and `duration_total`.
+A field that does not apply to a request is left out. Every line has:
+
+- `type`;
+- the fields from `time` to `user_agent`;
+- `request_id`, `peer_ip` and `client_group`;
+- `asn`, `as_name` and `country`;
+- `action` and `duration_total`.
 
 - `time` is when the request arrived, in UTC. `instance` is
   `SWWAF_INSTANCE_NAME`. `scheme` is the `X-Forwarded-Proto` a trusted proxy
@@ -1179,16 +1201,20 @@ fields from `time` to `user_agent`, `request_id`, `peer_ip`, `client_group`,
   limit; `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or
   `error_burst`. `offence` is then `limit`. A request whose bytes broke a limit
   was passed on whole, so its `action` stays `forward`.
-- `reputation` lists the sources the client is on: blocklist URLs, the CrowdSec
-  decision list's URL, DNS blocklist zones, then `abuseipdb`.
+- `reputation` lists the sources the client is on, in this order:
+    - blocklist URLs;
+    - the CrowdSec decision list's URL;
+    - DNS blocklist zones;
+    - `abuseipdb`.
 - `ban_expires` is when the client's ban ends, for a request that made one or
   was refused under one: a time, or `permanent`.
 - The timings are in milliseconds. `duration_total` runs from the request's
   headers being read to its log line. `duration_checks` runs to the end of the
   checks, and `duration_waf` is the part of that the Core Rule Set took. The
   three `duration_upstream_` timings run from the hand-over to the app:
-  `connect` until there was a connection, `first_byte` until the first byte of
-  the answer, and `total` until the end.
+    - `connect`, until there was a connection;
+    - `first_byte`, until the first byte of the answer;
+    - `total`, until the end.
 
 No body is logged, and no header but those above. `smallwebwaf`'s own messages
 (start, the settings, stop, errors) share the stream as JSON lines marked
@@ -1231,11 +1257,15 @@ its event is in `SWWAF_ALERT_EVENTS`:
 - `reputation_hit`: a request from a client on a blocklist, the CrowdSec
   decision list or a DNS blocklist, or with an AbuseIPDB score that is a hit.
   One alert per source.
-- `source_failure`: GeoJS failing, a list that cannot be fetched, a DNS
-  blocklist or AbuseIPDB failing or refusing a query, or the day's AbuseIPDB
-  checks used up.
-- `file_error`: a rule file or state file edit with an error, a state file that
-  cannot be written, or a replacement lookup database that cannot be read.
+- `source_failure`: one of:
+    - GeoJS failing;
+    - a list that cannot be fetched;
+    - a DNS blocklist or AbuseIPDB failing or refusing a query;
+    - the day's AbuseIPDB checks used up.
+- `file_error`: one of:
+    - a rule file or state file edit with an error;
+    - a state file that cannot be written;
+    - a replacement lookup database that cannot be read.
 
 The bans you make raise no alert. In `observe` mode, a request that would have
 made a ban still raises the ban's alert, with `mode` set to `observe` in its
@@ -1311,9 +1341,9 @@ line for each of these that the alert has:
 - the file, the source, the error and the mode from `detail`;
 - the suppressed repeats, if any.
 
-Slack is posted the title in bold and the text below it, with `&`, `<` and `>`
-escaped. ntfy is posted the text, with the title in `Title`, the token if set,
-and a priority and a tag by event:
+Slack gets the title in bold and the text below it, with `&`, `<` and `>`
+escaped. ntfy gets the text, with the title in `Title`, the token if set, and a
+priority and a tag by event:
 
 - `ban`: priority `default`, tag `no_entry`.
 - `permanent_ban`: priority `high`, tag `no_entry`.
@@ -1327,8 +1357,9 @@ A repeat is an alert for the same event on the same netblock, source, file or
 anomaly scope, less than `SWWAF_ALERT_COOLDOWN` after the last. It is held back
 and counted, and the next alert sent gives the count. Past
 `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and counted
-by event. Once the hour ends, one alert with the event `summary` sums them up,
-with the repeats of the cooldowns that ran out.
+by event. Once the hour ends, one alert with the event `summary` gives those
+counts. It also gives the repeats a cooldown held back that no later alert
+reported.
 
 Each destination has a queue of at most 1000 alerts, sent one at a time, so a
 slow destination holds up nothing. A `2xx` answer takes an alert. A `4xx` other
@@ -1418,9 +1449,11 @@ keeps it as `exported_instance` unless the scrape sets `honor_labels: true`.
     - its offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and
       `token_refused`.
 - `lookups.json`: GeoJS's answers, one per line.
-- `reputation.json`: the last good copy of each list fetched from a URL, with
-  when it was last tried; each DNS blocklist verdict still in use; and
-  AbuseIPDB's scores and the checks spent today.
+- `reputation.json`:
+    - the last good copy of each list fetched from a URL, with when it was last
+      tried;
+    - each DNS blocklist verdict still in use;
+    - AbuseIPDB's scores, and the checks spent today.
 - `alerts.json`: the alert cooldowns, the hour under way, the alerts waiting for
   each destination, and the anomaly counters.
 
@@ -1463,8 +1496,8 @@ AS number and country. These addresses are never sent:
 - one under a ban;
 - a private, loopback or link-local address, which no source can place.
 
-An IPv6 visitor is asked about by the first address of its group. Each answer is
-kept for seven days, in memory and in `lookups.json`.
+For an IPv6 visitor, GeoJS is asked about the first address of its group. Each
+answer is kept for seven days, in memory and in `lookups.json`.
 
 A request waits for its client's first answer only when a setting needs the
 answer before the request goes on. It waits up to `SWWAF_LOOKUP_TIMEOUT`. The
@@ -1490,11 +1523,11 @@ address it does not hold counts as from an unknown country. `smallwebwaf` reads
 the whole file at start. It reads the file again once it has gone 2 seconds
 without a change after you replace it. A replacement it cannot read is logged
 and alerted, and the file read before stays in use. Mount the directory that
-holds the file rather than the file itself: many editors and download tools
-replace a file when they save it, and a container keeps seeing the old one when
-the file alone is mounted. IPinfo releases the database under the Creative
-Commons Attribution-ShareAlike 4.0 licence. It asks for a link to IPinfo on the
-site that uses it, such as "IP address data is powered by IPinfo".
+holds the file, not the file itself. Many editors and download tools save a file
+by replacing it, and a container keeps seeing the old file when only the file is
+mounted. IPinfo releases the database under the Creative Commons
+Attribution-ShareAlike 4.0 licence. It asks for a link to IPinfo on the site
+that uses it, such as "IP address data is powered by IPinfo".
 
 ## Blocklists
 
@@ -1515,8 +1548,8 @@ list again before its time. A fetch fails when:
   `AS<number>:<percent>`.
 
 The copy fetched before stays in use. The failure is logged and raised as a
-`source_failure` alert. Each list is named by its URL in the log, the alerts,
-the ban notes and the metrics, so keep a secret out of it.
+`source_failure` alert. The log, the alerts, the ban notes and the metrics name
+each list by its URL, so keep a secret out of it.
 
 `deny` suits lists of networks that send nothing legitimate, such as DROP.
 `limit:<percent>` suits lists of addresses shared with ordinary visitors, such
@@ -1563,18 +1596,23 @@ client that has committed an offence is checked. An offence is one of:
 
 The check runs in the background, at the client's first request after the
 offence that no ban refuses, so no request waits. The score applies to the
-client's later requests. An IPv6 group is checked by the address of the request
-that has it checked, and the score is used for the whole group. A score at or
-over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and `SWWAF_REPUTATION_ACTION` says
-what is done. Each score is kept for `SWWAF_REPUTATION_CACHE_TTL`.
+client's later requests. For an IPv6 client, AbuseIPDB is asked about the
+address of the request that set off the check, and the score is used for the
+whole group. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and
+`SWWAF_REPUTATION_ACTION` says what is done. Each score is kept for
+`SWWAF_REPUTATION_CACHE_TTL`.
 
 At most `SWWAF_ABUSEIPDB_DAILY_BUDGET` checks are made in a day, counted in UTC
 and kept in `reputation.json`. A restart does not make the budget whole again.
 The check that spends the last of the budget is logged and alerted. No client is
-checked until the day ends. A check fails when AbuseIPDB answers other than
-`200`, gives no score, or does not answer within `SWWAF_REPUTATION_TIMEOUT`. A
-failure is logged and alerted, and no client is checked for a minute. The key is
-sent in the `Key` header and nowhere else, and is never logged.
+checked until the day ends. A check fails when:
+
+- AbuseIPDB answers other than `200`;
+- the answer gives no score;
+- no answer comes within `SWWAF_REPUTATION_TIMEOUT`.
+
+A failure is logged and alerted, and no client is checked for a minute. The key
+is sent in the `Key` header and nowhere else, and is never logged.
 
 ## CrowdSec
 
@@ -1589,13 +1627,15 @@ A request is refused when a decision still in force names its client. The
 client's netblock is banned until the decision ends, with the cause `crowdsec`.
 Only a client that sends a request is banned, so a long list does not fill
 `bans.json`. Such a ban is never made permanent, and does not lengthen the
-netblock's next ban for a broken limit. A ban you lift while the decision is in
-force is followed by another at the client's next request. Delete the decision
-in CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and
-lift the ban once the list has been fetched again. A fetch fails when the engine
-answers other than `200`, a redirect included. It also fails when the fetch does
-not finish within a minute, the answer is longer than 16 MiB, or the answer is
-not a list of decisions.
+netblock's next ban for a broken limit. If you lift a ban while the decision is
+in force, the client's next request makes another. Delete the decision in
+CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and lift
+the ban once the list has been fetched again. A fetch fails when:
+
+- the engine answers other than `200`, a redirect included;
+- the fetch does not finish within a minute;
+- the answer is longer than 16 MiB;
+- the answer is not a list of decisions.
 
 ## Rationale
 
@@ -1636,8 +1676,8 @@ candidates one by one.
   or `SIGINT`. Then it stops, writing the state files. Run as
   `smallwebwaf healthcheck`, it is the image's health check instead.
 - `internal/config`: reads the settings, the one place they are read.
-- `internal/proxy`: what happens to each request. It works out the client, runs
-  the checks in `check`, and passes the request to the app with the standard
+- `internal/proxy`: what happens to each request. It works out the client and
+  runs the checks in `check`. It passes the request to the app with the standard
   library's `httputil.ReverseProxy`, within the timeouts and size limits. It
   answers its own endpoints in `answerAdmin`. Once the request has ended, it
   counts bytes, refusals and anomalies, and writes the log line.
@@ -1683,13 +1723,16 @@ Besides the standard library, it uses:
 
 This repository adheres to the
 [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
-standard: the scripts in `script/` are the entrypoints for working on it, and
+standard. The scripts in `script/` are the entrypoints for working on it, and
 the `Makefile` targets are thin shims that call them. The scripts are POSIX sh.
 The tests and the linter run in Docker, as phases of the `Dockerfile`, and never
 on the host.
 
-- `script/bootstrap`: installs what the other scripts need on the host: `make`,
-  `git`, `curl`, Go for `gofmt`, node and yarn, and prettier. `make bootstrap`.
+- `script/bootstrap`: installs what the other scripts need on the host.
+  `make bootstrap`. They need:
+    - `make`, `git` and `curl`;
+    - Go, for `gofmt`;
+    - node, yarn and prettier.
 - `script/setup`: readies a fresh clone: `script/bootstrap`, then
   `script/install-precommit`. `make setup`.
 - `script/projectname`: prints `smallwebwaf`, which the other scripts tag their