The image creates SWW_STATE_DIR when it is missing (closes #337)

Before waf starts, share/sww.run makes SWW_STATE_DIR if it is missing,
gives it to sww:sww and sets its mode to 0700. A setting that is not an
absolute path, or a SWW_STATE_DIR_FILE that cannot be read, still
changes nothing. script/example-app now checks that a waf container
whose SWW_STATE_DIR does not exist becomes healthy with the directory
made for sww, mode 0700.

Model: opus-5-5

AGENTS.md

@@ -37,3 +37,6 @@ These are the rules for agents working on this repository.
         - `192.168.0.0/16`.
     - `SPEC.md` states that default.
     - Forwarded headers count only when the peer is a trusted proxy.
+- Every time is in UTC, whatever `TZ` or the host's time zone says.
+    - Every day is a date written `YYYY-MM-DD`, never a time.
+    - `SPEC.md` states this near its start.

Dockerfile

@@ -222,8 +222,8 @@ RUN groupadd --system --gid 65532 sww \
         sww
 
 # The state files' directory, SWW_STATE_DIR by default, where a volume
-# is mounted to keep them across deploys. The run script gives it to the
+# is mounted to keep them across deploys. At each start of waf, the run
-# sww user at each start of waf.
+# script makes SWW_STATE_DIR if it is missing and gives it to the sww user.
 RUN mkdir /var/lib/sww
 
 # The default rule file, in SWW_RULES_DIR by default, where a Dockerfile

README.md

@@ -293,7 +293,8 @@ Every setting the code reads, with its default:
 | `SWW_REPUTATION_TIMEOUT`                 | `2s`                                                                                     | How long a DNS blocklist query or an AbuseIPDB check may take.                                                                                                                                                                                                  |
 | `SWW_STATE_DIR`                          | `/var/lib/sww`                                                                           | The directory of the state files, an absolute path.                                                                                                                                                                                                             |
 | `SWW_STATE_WRITE_DELAY`                  | `10s`                                                                                    | How long `bans.json` waits to be written after a ban is made, lifted or made permanent.                                                                                                                                                                         |
-| `SWW_STATE_COUNTER_INTERVAL`             | `15m`                                                                                    | How often every state file is written.                                                                                                                                                                                                                          |
+| `SWW_STATE_COUNTER_INTERVAL`             | `15m`                                                                                    | How often every state and cache file is written.                                                                                                                                                                                                                |
+| `SWW_CACHE_DIR`                          | `/var/cache/sww`                                                                         | The directory of the cache files, an absolute path.                                                                                                                                                                                                             |
 | `SWW_LOG_LEVEL`                          | `info`                                                                                   | The least severe of `waf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written.                                                                                                                                |
 | `SWW_LOG_REQUEST_HEADERS`                | `accept,accept-language,accept-encoding,content-type,origin,range`                       | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded.                                                                                                                                                      |
 | `SWW_LOG_REMOTE_URL`                     | unset                                                                                    | A syslog server that also gets every line `waf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port.                                                                                                                             |
@@ -509,7 +510,8 @@ request reaches the app.
 
 `waf` takes the checks:
 
-3. The tag lists.
+3. The header limits, then the tag lists. A client in `SWW_ALLOW_NETS` skips the
+   header limits.
 4. The static lists, `SWW_ALLOW_NETS` and `SWW_DENY_NETS`.
 5. The bans.
 6. The lookup of the client's AS number and country.
@@ -521,7 +523,7 @@ request reaches the app.
 11. The rate limits.
 12. The trap paths.
 13. The rule files.
-14. The header limits, and then the Core Rule Set.
+14. The Core Rule Set.
 15. The tag refusals again, so that `exploitsignature` can refuse too.
 16. A path under `/_sww/` is answered `404`, so that the app never sees that
     prefix.
@@ -537,7 +539,7 @@ request reaches the app.
 A refused visitor is answered with `SWW_BAN_RESPONSE`. A `block` rule or a Core
 Rule Set match answers `403` and bans nobody, since the Core Rule Set has false
 positives; such refusals count toward `SWW_ERROR_BURST_THRESHOLD` instead. A
-request over the header limits of step 14 is answered `431`, and bans nobody
+request over the header limits of step 3 is answered `431`, and bans nobody
 either. `rproxy` itself answers `405` to a method it does not pass on, and the
 status for a size or time limit, such as `413` for a body too large.
 
@@ -613,15 +615,29 @@ client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
   while `waf` runs. The image ships one file of probes no real visitor sends
   (see [`docs/rules.md`](docs/rules.md)).
 
-### State files
+### State and cache files
 
-`waf` keeps everything in memory, and the five JSON files in `SWW_STATE_DIR`
+`waf` keeps everything in memory, and JSON files hold a copy:
-hold a copy: `bans.json`, `clients.json`, `lookups.json`, `reputation.json` and
+
-`alerts.json`. Each is written whole, to a temporary file renamed over the real
+- The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
-one, so a crash leaves a complete file. `waf` notices a file edited on disk and
+    - `bans.json`;
-takes the edit in; an edit that does not parse is set aside as `<name>.bad`, and
+    - `clients.json`;
-never stops the running process (see
+    - `abuseipdb-checks-spent.json`;
-[`docs/state-files.md`](docs/state-files.md)).
+    - `alerts.json`.
+- The cache files, in `SWW_CACHE_DIR`, hold answers from outside services:
+    - `lookups.json`, from GeoJS;
+    - `lists.json`, the lists fetched by URL;
+    - `dnsbl.json`, from the DNS blocklists;
+    - `abuseipdb-scores.json`, from AbuseIPDB;
+    - `routewatch.json`, from routewatch.
+
+Back up the state directory. The cache directory can be left out of backups: a
+lost cache file starts empty and fills again.
+
+Each file is written whole, to a temporary file renamed over the real one, so a
+crash leaves a complete file. `waf` notices a file edited on disk and takes the
+edit in. An edit that does not parse is set aside as `<name>.bad`, and never
+stops the running process (see [`docs/state-files.md`](docs/state-files.md)).
 
 ### The image and its processes
 
@@ -632,7 +648,7 @@ never stops the running process (see
 - runit, a service supervisor, runs `sww` as the `sww` user (uid and gid 65532)
   and starts it again when it exits. The `run` script of `waf` gives the state
   directory to the `sww` user at each start, so a volume needs no change of
-  owner.
+  owner. It makes the cache directory for that user too.
 - `rproxy` and `waf` use no token between them. `waf`'s own endpoints, under
   `/_sww/`, are for your admin tools and your metrics collector, at its own
   address on the private network; nothing routes the internet there.
@@ -657,8 +673,8 @@ never stops the running process (see
   `internal/routewatch`, `internal/anomaly` and `internal/alerts`: one check or
   source each.
 - `internal/state`, `internal/requestlog`, `internal/remotelog` and
-  `internal/metrics`: the state files, the log lines, the syslog sender and the
+  `internal/metrics`: the state and cache files, the log lines, the syslog
-  metrics.
+  sender and the metrics.
 
 ## Rationale
 
@@ -738,6 +754,7 @@ on the host.
       `http://127.0.0.1:8081`;
     - `waf` listens on port 8090;
     - its state files go in `bin/state`;
+    - its cache files go in `bin/cache`;
     - its rule files are those of `share/rules.d`.
 - `script/example-app`: builds the image and the example app
   (`deploy/example-app`) on it, runs the app with `rproxy` in its container and
@@ -774,8 +791,8 @@ on the host.
 - [`docs/alerts.md`](docs/alerts.md): the alert events and what each destination
   gets.
 - [`docs/metrics.md`](docs/metrics.md): the metrics.
-- [`docs/state-files.md`](docs/state-files.md): the state files and what each
+- [`docs/state-files.md`](docs/state-files.md): the state and cache files and
-  holds.
+  what each holds.
 - [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
 
 ## License

SPEC.md

@@ -12,6 +12,10 @@ it is built, except what the last section lists. `README.md` says how to run it.
 under `docs/` give the settings, the formats and the lists. This document links
 to them.
 
+The whole application stores, logs and handles every time in UTC, whatever `TZ`
+or the host's time zone says. It keeps every day as a date written `YYYY-MM-DD`,
+never as a time.
+
 ## Purpose
 
 - It protects self-hosted apps on the open internet from scrapers, scanners and
@@ -33,7 +37,7 @@ to them.
   `SWW_`, and those of `rproxy` with `SWRP_`. The only other inputs are `waf`'s
   files:
     - the rule files;
-    - the state files;
+    - the state files and the cache files;
     - a lookup database;
     - a `SWW_` setting given as a file;
     - the file `SWW_LOG_REMOTE_TLS_CA_FILE` names;
@@ -52,19 +56,19 @@ to them.
 ## Architecture
 
 - One statically linked Go binary. Only `waf` writes to disk, and only to its
-  state directory. It ships in one image, which runs either command (see
+  state directory and its cache directory. It ships in one image, which runs
-  "Deployment").
+  either command (see "Deployment").
 - No database of any kind. Nothing is read from disk while serving a request.
 - Packages:
     - `cmd/sww`: the binary, which only calls `internal/sww`.
     - `internal/sww`: the process, as the command it is given. `sww waf` reads
-      the settings, the rule files, the lookup database and the state files,
+      the settings, the rule files, the lookup database and the state and cache
-      listens, and serves until `SIGTERM` or `SIGINT`. Then it stops, writing
+      files, listens, and serves until `SIGTERM` or `SIGINT`. Then it stops,
-      the state files (see "`waf`"). `sww rproxy` reads only that command's
+      writing the state and cache files (see "`waf`"). `sww rproxy` reads only
-      settings, and runs `internal/rproxy` until `SIGTERM` or `SIGINT` (see
+      that command's settings, and runs `internal/rproxy` until `SIGTERM` or
-      "`rproxy`"). `sww healthcheck` is the image's health check. With no
+      `SIGINT` (see "`rproxy`"). `sww healthcheck` is the image's health check.
-      command, or one it does not know, the binary prints the commands and exits
+      With no command, or one it does not know, the binary prints the commands
-      with status 2.
+      and exits with status 2.
     - `internal/config`: reads the settings, the one place they are read.
     - `internal/waf`: `waf`'s server. It runs the checks in `check`, and holds
       `waf`'s question, answer and report. It answers `waf`'s own endpoints in
@@ -92,8 +96,8 @@ to them.
     - `internal/alerts`: holds back repeats and alerts past the hourly limit,
       and sends the rest to the webhook, Slack and ntfy, each from a queue of
       its own.
-    - `internal/state`: reads the state files at start, takes in edits while
+    - `internal/state`: reads the state and cache files at start, takes in edits
-      running, and writes the files when due and at the stop.
+      while running, and writes the files when due and at the stop.
     - `internal/requestlog` and `internal/remotelog`: the lines on stdout, and
       sending them to the syslog server.
     - `internal/metrics`: the metrics, in the Prometheus text format.
@@ -102,7 +106,7 @@ to them.
       container, which `script/example-app` checks with `waf` in another.
 - Libraries, checked against the owner's Go dependency defaults:
     - the standard library for the reverse proxy, HTTP clients, DNS, logging
-      (`log/slog`) and the state files (`encoding/json`, `os.Rename`);
+      (`log/slog`) and the state and cache files (`encoding/json`, `os.Rename`);
     - `github.com/corazawaf/coraza/v3` and
       `github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries
       the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to it, and the
@@ -145,10 +149,12 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
     - An IPv6 client is its group: the netblock of `SWW_IPV6_GROUP_PREFIX`
       around its address, a /64 by default, since one abuser usually controls a
       whole /64. A client is therefore one IPv4 address or one IPv6 group.
-3. The tag lists (`tag_lists`; see "Request tags"). Each list that holds the
+3. The header limits (`header_limits`; see "Answering in time"). They come first
-   client adds its tag. Next, `routewatch_tags` adds the client's `asn=` and
+   after `SWW_ALLOW_NETS`, since they only count header fields and cookies.
+   Next, the tag lists (`tag_lists`; see "Request tags"). Each list that holds
+   the client adds its tag. Next, `routewatch_tags` adds the client's `asn=` and
    `prefix=` tags while `SWW_ROUTEWATCH_URL` is set. A client in
-   `SWW_ALLOW_NETS` gets no tag.
+   `SWW_ALLOW_NETS` skips the header limits and gets no tag.
 4. The static lists. A client in `SWW_ALLOW_NETS` skips steps 5 to 15, the byte
    limits and the error burst. A client in `SWW_DENY_NETS` is refused
    (`deny_nets`); a client in both lists is let through.
@@ -181,10 +187,9 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
     refused and bans the client.
 13. The rule files (`rule_files`; see "Rule files"). Matching stops at the first
     rule that refuses.
-14. The header limits (`header_limits`; see "Answering in time"), and then the
+14. The Core Rule Set (`core_rule_set`; see "Attack detection"). A match is
-    Core Rule Set (`core_rule_set`; see "Attack detection"). A Core Rule Set
+    refused with `403` while `SWW_WAF_MODE` is `block`, and only logged and
-    match is refused with `403` while `SWW_WAF_MODE` is `block`, and only logged
+    alerted while it is `detect`.
-    and alerted while it is `detect`.
 15. The tag refusals again (`tag_deny`), so that `exploitsignature` can refuse
     too.
 16. A path under `/_sww/` is answered `404`, so that the app never sees that
@@ -237,11 +242,16 @@ apply.
 ## Bans
 
 Until a ban ends, every request from its netblock is refused with
-`SWW_BAN_RESPONSE`. The netblock is the client: its IPv4 address, widened to
+`SWW_BAN_RESPONSE`, unless it is over the header limits. The netblock is the
-`SWW_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one
+client: its IPv4 address, widened to `SWW_BAN_SCOPE_V4_PREFIX`, or its IPv6
-address or one /64, so a permanent ban does not reach neighbours who did
+group. At the defaults a ban covers one address or one /64, so a permanent ban
-nothing. Requests refused under a ban are not counted against limits. They are
+does not reach neighbours who did nothing. Requests refused under a ban are not
-counted in the client's history and in the ban's notes.
+counted against limits. They are counted in the client's history and in the
+ban's notes.
+
+A request over the header limits is refused with `431` before the bans are
+checked (see "Answering in time"). It makes no ban, makes no ban permanent and
+is not counted in a ban's notes.
 
 An offence is a request `waf` holds against the client. It is one of:
 
@@ -357,7 +367,7 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
   one, since `rproxy` never touches disk.
 - Settings, files included, are read once at start. Changing one means
   restarting the container. The files `waf` watches while it runs are its state
-  files, its rule files and the lookup database.
+  and cache files, its rule files and the lookup database.
 - An invalid setting stops the process at start with one log line, marked
   `"type":"process"` with `"msg":"invalid setting"`. The line names the variable
   and what is wrong with the value; "Each container's processes" says what
@@ -387,10 +397,10 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
   start. The file form keeps a token out of the container's environment (see
   "Tokens" under "Deployment").
 
-### State files
+### State and cache files
 
-- `SWW_STATE_DIR` names the directory of the state files (see "State" under
+- `SWW_STATE_DIR` names the directory of the state files, and `SWW_CACHE_DIR`
-  "Deployment").
+  that of the cache files (see "State" under "Deployment").
 - `bans.json` is written `SWW_STATE_WRITE_DELAY` after a ban is made, lifted or
   made permanent, so a burst of changes becomes one write. Every file is written
   every `SWW_STATE_COUNTER_INTERVAL`, and at an orderly stop.
@@ -573,7 +583,7 @@ Each tag and where it comes from:
     - They are added before `deny_nets`, so the log line of a request it refused
       has them.
     - `waf` asks routewatch in the background and never waits for it. An answer
-      is kept for 1 hour, in memory only.
+      is kept for 1 hour, in memory and in `routewatch.json`.
     - After a query fails, routewatch is not asked for 1 minute.
     - A request without these tags gives the reason in the log field
       `routewatch_tags_missing` (see
@@ -1591,8 +1601,8 @@ together.
     is kept whole, comment lines included, on failure and across restarts.
 
 - Each list is fetched again `SWW_BLOCKLIST_REFRESH` after its last try, one
-  list at a time. `reputation.json` keeps when each was last tried, so a restart
+  list at a time. `lists.json` keeps when each was last tried, so a restart does
-  does not fetch a list again before its time. A fetch fails when:
+  not fetch a list again before its time. A fetch fails when:
     - the server answers other than `200`;
     - the fetch does not finish within a minute;
     - the list is longer than 16 MiB;
@@ -1636,9 +1646,9 @@ together.
     DNS blocklists made for mail servers list residential ranges, and would
     block ordinary visitors, so they should not be used. Spamhaus zones need
     their keyed query service, given as a zone name that contains the key. `waf`
-    shows the key as `********` wherever it names the zone. Only
+    shows the key as `********` wherever it names the zone. Only `dnsbl.json`
-    `reputation.json` keeps the key. `SWW_DNSBL_RESOLVER` names a resolver,
+    keeps the key. `SWW_DNSBL_RESOLVER` names a resolver, since public resolvers
-    since public resolvers are refused by several list operators.
+    are refused by several list operators.
 
 - AbuseIPDB (`SWW_ABUSEIPDB_KEY`) is asked, at its check endpoint, for the abuse
   confidence score of a client, from 0 to 100. Only a client that has already
@@ -1649,11 +1659,11 @@ together.
   client's later requests. For an IPv6 client, AbuseIPDB is asked about the
   address of the request that set off the check, and the score is used for the
   whole group. A score at or over `SWW_ABUSEIPDB_MIN_SCORE` is a hit. The budget
-  is counted in UTC days and kept in `reputation.json`, so a restart does not
+  is counted in UTC days and kept in `abuseipdb-checks-spent.json`, so a restart
-  make it whole again. The check that spends the last of it is logged and
+  does not make it whole again. The check that spends the last of it is logged
-  alerted, and no client is checked until the day ends. The key is sent in the
+  and alerted, and no client is checked until the day ends. The key is sent in
-  `Key` header and nowhere else, and is never logged. A check fails, after which
+  the `Key` header and nowhere else, and is never logged. A check fails, after
-  no client is checked for a minute, when:
+  which no client is checked for a minute, when:
     - AbuseIPDB answers other than `200`;
     - the answer gives no score;
     - no answer comes within `SWW_REPUTATION_TIMEOUT`.
@@ -1661,9 +1671,9 @@ together.
   by default. Such verdicts are less certain than a blocklist, so a listed
   client gets a quarter of every request and byte limit rather than a refusal.
 - Verdicts and scores are kept for `SWW_REPUTATION_CACHE_TTL`, in memory and in
-  `reputation.json`. A source that is down or over quota gives no verdict. The
+  `dnsbl.json` and `abuseipdb-scores.json`. A source that is down or over quota
-  service continues. Each failure raises a `source_failure` alert (see
+  gives no verdict. The service continues. Each failure raises a
-  "Alerting").
+  `source_failure` alert (see "Alerting").
 
 ### Alerting
 
@@ -1766,20 +1776,43 @@ default file.
 
 ## Persistent state
 
-All state lives in memory, and the files in `SWW_STATE_DIR` hold a copy of it,
+All state lives in memory, and JSON files hold a copy of it, so an orderly stop
-so an orderly stop loses nothing. The files are meant for people as well: an
+loses nothing. The files are meant for people as well: an admin can read or edit
-admin can read or edit them at any time, and the running `waf` takes the edit
+them at any time, and the running `waf` takes the edit in. `docs/state-files.md`
-in. `docs/state-files.md` lists the files, what each holds, and their sizes.
+lists the files, what each holds, and their sizes.
 
+- The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
+    - `bans.json`;
+    - `clients.json`;
+    - `abuseipdb-checks-spent.json`;
+    - `alerts.json`.
+- The cache files, in `SWW_CACHE_DIR`, hold the answers of outside services:
+    - `lookups.json`;
+    - `lists.json`;
+    - `dnsbl.json`;
+    - `abuseipdb-scores.json`;
+    - `routewatch.json`.
+- A lost cache file costs only questions asked again, so the cache directory can
+  be left out of backups.
 - Each file holds one kind of state, as JSON with:
     - a top-level `version` number;
     - entries sorted by client address;
     - times in RFC 3339 UTC;
+    - days as `YYYY-MM-DD`;
     - durations and sizes as plain numbers, with the unit in the field name.
-- `bans.json`, `reputation.json` and `alerts.json` are indented, so that a
+- These files are indented, so that a person can open `bans.json` in an editor,
-  person can open `bans.json` in an editor, find an address, and remove or add
+  find an address, and remove or add an entry:
-  an entry. `clients.json` and `lookups.json` put each entry on one line, which
+    - `bans.json`;
-  halves their size and lets `grep` show everything about one client.
+    - `abuseipdb-checks-spent.json`;
+    - `alerts.json`;
+    - `lists.json`.
+- The others put each entry on one line, which halves their size and lets `grep`
+  show everything about one client:
+    - `clients.json`;
+    - `lookups.json`;
+    - `dnsbl.json`;
+    - `abuseipdb-scores.json`;
+    - `routewatch.json`.
 - Clients that were never banned are kept too, so every client's history
   survives a restart.
 - Each file has a bound, so it can be rewritten whole. Writing one takes well
@@ -1793,8 +1826,8 @@ in. `docs/state-files.md` lists the files, what each holds, and their sizes.
   disk, so an admin's edit is never overwritten.
 - At start, the files are read back. Active bans go into an in-memory lookup by
   netblock, everything else into maps. Expired cache entries are discarded. A
-  missing file means empty state, and is normal on first run. These stop the
+  missing file means empty state, and is normal on first run. In a state file,
-  process with a message naming the file and position:
+  these stop the process with a message naming the file and position:
     - a file that does not parse, or has an unknown `version`;
     - an entry without a field it needs;
     - a ban whose cause is not one of the four;
@@ -1804,6 +1837,11 @@ in. `docs/state-files.md` lists the files, what each holds, and their sizes.
     Since writes are atomic, a broken file can only come from a hand edit, which
     the editor should hear about.
 
+- A cache file never stops the start. One that does not parse is set aside as
+  `<name>.bad`, as an edit is below. One that cannot be read is logged and
+  written over at its next write. Either way its cache starts empty and fills
+  again.
+
 - While running, `waf` notices a file that is edited, replaced or added. It
   tells its own writes from an admin's by comparing the file with what it last
   read or wrote. An edit that parses is taken in at once: what the file says
@@ -1894,10 +1932,10 @@ each app. `rproxy` asks `waf` about each visitor's request. `waf` answers
 This section is the agreement between the two. The header names, the fields and
 the statuses below are exact.
 
-`waf` reads the settings, the rule files, the lookup database and the state
+`waf` reads the settings, the rule files, the lookup database and the state and
-files, and listens at `SWW_LISTEN_ADDR`. On `SIGTERM` or `SIGINT` it stops
+cache files, and listens at `SWW_LISTEN_ADDR`. On `SIGTERM` or `SIGINT` it stops
 taking requests and gives those in progress five seconds to finish. It then
-writes the state files.
+writes the state and cache files.
 
 ### The question
 
@@ -2020,7 +2058,6 @@ waits on these, which run in the background:
 
 In `enforce` mode, a question does not wait for GeoJS when it is a request:
 
-- over the header limits;
 - from a client on a fetched blocklist, while `SWW_BLOCKLIST_ACTION` is `deny`;
 - from a client on the CrowdSec decision list;
 - that would take its client over a rate limit at 100% of it, since a percentage
@@ -2032,8 +2069,8 @@ number and country stay unknown until GeoJS has answered. So while
 `SWW_UNKNOWN_LIMIT_PERCENT` does not apply to it until then, so not waiting
 never lowers its limit.
 
-The static lists and bans come before the lookup, so they never wait. These can
+The header limits, the static lists and bans come before the lookup, so they
-still wait for GeoJS:
+never wait. These can still wait for GeoJS:
 
 - a country list, which needs the answer;
 - a rate limit a percentage has lowered, which needs the answer;
@@ -2052,13 +2089,17 @@ either one:
 
 Real clients never reach these limits. The answer is `deny` with `431`, and the
 log line's action is `request_header_too_large`. The refusal makes no ban and is
-no offence. The rate limits count the request as usual, so a client that keeps
+no offence.
-sending such requests breaks a limit.
+
+The header limits are the first check after `SWW_ALLOW_NETS`. A request over
+them is refused before any later check runs. So it causes no lookup, no query to
+an outside source and no rule match, and the rate limits do not count it. A
+client under a ban or in `SWW_DENY_NETS` also gets `431` for such a request.
 
 The limits apply on every path and in every `SWW_WAF_MODE`. A client in
 `SWW_ALLOW_NETS` skips them, as it skips the Core Rule Set. In `observe` mode
-such a request is answered `allow` with `would_action`, and the Core Rule Set
+such a request is answered `allow` with `would_action`, and no other check runs
-does not read it.
+on it.
 
 The Core Rule Set runs each of its rules on each cookie, so thousands of small
 cookies would keep it busy for over a second. Within the limits, one long cookie
@@ -2310,8 +2351,10 @@ but for these headers:
   on.
 - For a peer outside `SWRP_TRUSTED_PROXIES`, its forwarded headers are its own
   claims and are replaced. `X-Forwarded-For` names the peer, `X-Forwarded-Host`
-  the host it asked for, and `X-Forwarded-Proto` plain `http`. The other headers
+  the host it asked for, and `X-Forwarded-Proto` plain `http`. Every other
-  in which traefik tells an app about a visitor are removed.
+  header whose name starts with `X-Forwarded-` is removed, and so are
+  `Forwarded` and `X-Real-IP`. A name spelt with `_` for `-`, such as
+  `X_Forwarded_Ssl`, counts as the same name.
 - `X-Request-ID`, which carries the fresh id.
 - `X-Client-ASN` and `X-Client-Country`, which carry what `waf` answered. A
   header it did not give is left out. Any of the two the visitor sent is
@@ -2464,7 +2507,9 @@ It sets no `ENTRYPOINT` or `USER` of its own. The container must start
 - The image's `run` script is `/etc/service/sww/run`. It starts `waf` when
   `SWW_COMMAND` is `waf`, and `rproxy` when it is `rproxy`, unset or empty. It
   runs either as the `sww` user (uid and gid 65532), with runit's `chpst`. For
-  `waf`, it first makes `SWW_STATE_DIR` and every file in it belong to that
+  `waf`, it first makes `SWW_STATE_DIR` if it is missing, gives it and every
+  file in it to that user, and sets its mode to `0700`. It also makes
+  `SWW_CACHE_DIR` if it is missing, and gives it and every file in it to that
   user. For `rproxy` it changes no owner. With any other value, it starts
   nothing, and stops with a message naming `SWW_COMMAND` and its value.
 
@@ -2489,7 +2534,7 @@ process that stops at start is started again every second, not in a tight loop.
     - invalid configuration;
     - a lookup database that is missing or unreadable;
     - a rule file or state file that does not parse;
-    - an unwritable `SWW_STATE_DIR`;
+    - an unwritable `SWW_STATE_DIR` or `SWW_CACHE_DIR`;
     - a `SWW_RULES_DIR` that does not exist or cannot be read;
     - a `SWW_LISTEN_ADDR` it cannot listen on.
 - Only an invalid setting, or a `SWRP_LISTEN_ADDR` it cannot listen on, stops
@@ -2558,6 +2603,10 @@ needs no backup beyond the host's own. The image declares no volume, since every
 app image built on it would inherit it. `rproxy` keeps nothing, and needs no
 volume.
 
+`waf` keeps its cache files in `SWW_CACHE_DIR`, `/var/cache/sww` by default,
+which the `run` script makes. The cache directory needs no volume and can be
+left out of backups: a new container starts with empty caches, which fill again.
+
 ### Tokens
 
 A token given as a file (`SWW_ADMIN_TOKEN_FILE`, `SWW_METRICS_TOKEN_FILE`) stays

docs/alerts.md

@@ -23,7 +23,8 @@ back or dropped.
     - the day's AbuseIPDB checks used up.
 - `file_error`: one of:
     - a rule file or state file edit with an error;
-    - a state file that cannot be written;
+    - a cache file that does not parse, at start or after an edit;
+    - a state file or cache file that cannot be written;
     - a replacement lookup database that cannot be read.
 - `summary`: the hourly summary of alerts held back.
 

docs/image.md

@@ -19,7 +19,8 @@ way.
 - The service directory `/etc/service/sww`, with the `run` script
   `share/sww.run`.
 - `/var/lib/sww` for `waf`'s state files, and `/etc/sww/rules.d` with the
-  default rule file `share/rules.d/00-default.rules`.
+  default rule file `share/rules.d/00-default.rules`. The `run` script makes
+  `SWW_STATE_DIR` when it is missing.
 - `EXPOSE 8080`, and the `HEALTHCHECK` that runs `sww healthcheck`.
 
 ## Pins

docs/metrics.md

@@ -58,9 +58,9 @@ By `source`: each list's URL, each zone, and `abuseipdb`:
 - `sww_reputation_last_fetch_timestamp_seconds`, for a list;
 - `sww_reputation_daily_budget_remaining`, for AbuseIPDB.
 
-## State files
+## State and cache files
 
-By `file`:
+By `file`, the name of a state or cache file:
 
 - `sww_state_file_writes_total`, `sww_state_file_write_failures_total`,
   `sww_state_file_last_write_timestamp_seconds` and `sww_state_file_size_bytes`;

docs/settings.md

@@ -50,14 +50,16 @@ the two (see "The image's command" below).
 - `SWW_METRICS_TOKEN` (default unset): the token for `/_sww/metrics`. At least
   32 characters.
 
-## State files
+## State and cache files
 
 - `SWW_STATE_DIR` (default `/var/lib/sww`): the directory of the state files, an
-  absolute path that exists and can be written.
+  absolute path that can be written. The image's `run` script makes it.
 - `SWW_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
   written after a ban is made, lifted or made permanent. A duration.
-- `SWW_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is
+- `SWW_STATE_COUNTER_INTERVAL` (default `15m`): how often every state and cache
-  written. A duration.
+  file is written. A duration.
+- `SWW_CACHE_DIR` (default `/var/cache/sww`): the directory of the cache files,
+  an absolute path that can be written. The image's `run` script makes it.
 
 ## Logging
 

docs/state-files.md

@@ -1,8 +1,23 @@
-# State files
+# State and cache files
 
-The five JSON files in `SWW_STATE_DIR`, and what each holds. "Persistent state"
+The JSON files `waf` keeps, and what each holds. "Persistent state" in
-in [`SPEC.md`](../SPEC.md) says how they are written, read and edited. Each has
+[`SPEC.md`](../SPEC.md) says how they are written, read and edited. Each has a
-a top-level `version` of 1.
+top-level `version` of 1.
+
+- The state files are in `SWW_STATE_DIR`, `/var/lib/sww` by default. They hold
+  what cannot be fetched again, so back them up:
+    - `bans.json`;
+    - `clients.json`;
+    - `abuseipdb-checks-spent.json`;
+    - `alerts.json`.
+- The cache files are in `SWW_CACHE_DIR`, `/var/cache/sww` by default. They hold
+  answers from outside services, which are asked again when a file is lost, so
+  the directory can be left out of backups:
+    - `lookups.json`;
+    - `lists.json`;
+    - `dnsbl.json`;
+    - `abuseipdb-scores.json`;
+    - `routewatch.json`.
 
 ## `bans.json`
 
@@ -67,21 +82,13 @@ since it was first seen:
   `blocked_by_core_rule_set`. A file may also hold `token_refused`, which is
   kept, and which nothing adds to.
 
-## `lookups.json`
+## `abuseipdb-checks-spent.json`
 
-GeoJS answers, one per line:
+The AbuseIPDB checks spent today, so that a restart does not spend the daily
+budget again:
 
-- `client`;
+- `day`: the day, in UTC;
-- `asn`, `as_name` and `country`;
+- `spent`: the checks spent on it.
-- `answered`: when GeoJS answered;
-- `used`: when the answer was last used.
-
-## `reputation.json`
-
-- the last good copy of each list fetched from a URL, with when it was last
-  tried;
-- each DNS blocklist verdict still in use;
-- AbuseIPDB's scores, and the checks spent today.
 
 ## `alerts.json`
 
@@ -91,6 +98,51 @@ GeoJS answers, one per line:
 - the alerts sent and held back in the current hour;
 - the alerts waiting for each destination.
 
+## `lookups.json`
+
+A cache file. GeoJS answers, one per line:
+
+- `client`;
+- `asn`, `as_name` and `country`;
+- `answered`: when GeoJS answered;
+- `used`: when the answer was last used.
+
+## `lists.json`
+
+A cache file. The last good copy of each list fetched from a URL, with when it
+was last tried:
+
+- `url`;
+- `tried`: when it was last fetched or tried;
+- `fetched` and `lines`: when the last good copy was fetched, and its lines,
+  comment lines included.
+
+## `dnsbl.json`
+
+A cache file. Each DNS blocklist verdict still in use, one per line:
+
+- `zone` and `client`;
+- `listed`: whether the zone lists the client;
+- `fetched`: when the zone answered.
+
+## `abuseipdb-scores.json`
+
+A cache file. AbuseIPDB's scores still in use, one per line:
+
+- `client`;
+- `score`: from 0 to 100;
+- `fetched`: when AbuseIPDB answered.
+
+## `routewatch.json`
+
+A cache file. routewatch's answers from the last hour, one per line:
+
+- `client`;
+- `netblock`: the netblock the client's address is announced in, empty for an
+  address without a live route;
+- `asn`: the number of the AS that originates the netblock, 0 for none;
+- `answered`: when routewatch answered.
+
 ## Sizes
 
 - `clients.json`: about 1 KiB per client, about 20 MiB at the default
@@ -102,4 +154,7 @@ GeoJS answers, one per line:
   more.
 - `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
   full.
-- `reputation.json` and `alerts.json`: usually a few MiB or less.
+- `routewatch.json`: about 120 bytes per answer, at most about 12 MiB.
+- `abuseipdb-checks-spent.json`: a few bytes.
+- `lists.json`, `dnsbl.json`, `abuseipdb-scores.json` and `alerts.json`: usually
+  a few MiB or less.

go.mod

@@ -3,6 +3,7 @@ module sneak.berlin/go/smallwebwaf
 go 1.26.0
 
 require (
+	cloud.google.com/go v0.123.0
 	github.com/corazawaf/coraza-coreruleset/v4 v4.25.0
 	github.com/corazawaf/coraza/v3 v3.8.2-0.20261022102553-d028be399c68
 	github.com/fsnotify/fsnotify v1.10.1

go.sum

@@ -1,3 +1,5 @@
+cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
+cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
 github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
 github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
 github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=

internal/alerts/alerts.go

@@ -56,9 +56,11 @@ const (
 	// of a list failing, a query to a DNSBL zone or a check with AbuseIPDB
 	// failing or refused, or the day's AbuseIPDB checks used up.
 	EventSourceFailure = "source_failure"
-	// EventFileError is a rule file or state file edited while sww
+	// EventFileError is one of:
-	// runs that does not parse, a replacement of the lookup database that
+	//   - a rule file or state file edited while sww runs that does not parse;
-	// cannot be read, or a state file that cannot be written.
+	//   - a cache file that does not parse, at start or after an edit;
+	//   - a state file or cache file that cannot be written;
+	//   - a replacement of the lookup database that cannot be read.
 	EventFileError = "file_error"
 	// EventSummary is the summary sent as an hour ends: of the alerts held
 	// back in it past SWW_ALERT_MAX_PER_HOUR, and of the repeats held

internal/config/config.go

@@ -191,12 +191,15 @@ type Config struct {
 	BanScopeV4Prefix int
 	// StateDir is the directory of the state files, an absolute path
 	// (SWW_STATE_DIR). bans.json is written StateWriteDelay after a ban
-	// is made (SWW_STATE_WRITE_DELAY), and every state file every
+	// is made (SWW_STATE_WRITE_DELAY), and every state and cache file
-	// StateCounterInterval (SWW_STATE_COUNTER_INTERVAL). Neither can be
+	// every StateCounterInterval (SWW_STATE_COUNTER_INTERVAL). Neither
-	// off.
+	// can be off.
 	StateDir             string
 	StateWriteDelay      time.Duration
 	StateCounterInterval time.Duration
+	// CacheDir is the directory of the cache files, which keep answers
+	// from outside services, an absolute path (SWW_CACHE_DIR).
+	CacheDir string
 	// LogRequestHeaders are the request headers whose values the request
 	// log gives, in lower case (SWW_LOG_REQUEST_HEADERS).
 	LogRequestHeaders []string
@@ -560,6 +563,7 @@ func FromEnvironment(
 		StateDir:             env.absolutePath("SWW_STATE_DIR", "/var/lib/sww"),
 		StateWriteDelay:      env.durationNotOff("SWW_STATE_WRITE_DELAY", "10s"),
 		StateCounterInterval: env.durationNotOff("SWW_STATE_COUNTER_INTERVAL", "15m"),
+		CacheDir:             env.absolutePath("SWW_CACHE_DIR", "/var/cache/sww"),
 		LogRequestHeaders: env.headerNames("SWW_LOG_REQUEST_HEADERS",
 			"accept,accept-language,accept-encoding,content-type,origin,range"),
 		LogLevel:            env.logLevel("SWW_LOG_LEVEL", "info"),

internal/config/config_test.go

@@ -74,6 +74,7 @@ const (
 	stateDir             = "SWW_STATE_DIR"
 	stateWriteDelay      = "SWW_STATE_WRITE_DELAY"
 	stateCounterInterval = "SWW_STATE_COUNTER_INTERVAL"
+	cacheDir             = "SWW_CACHE_DIR"
 	adminToken           = "SWW_ADMIN_TOKEN"   //nolint:gosec // the setting's name
 	metricsToken         = "SWW_METRICS_TOKEN" //nolint:gosec // the setting's name
 	metricsTopN          = "SWW_METRICS_TOP_N"
@@ -262,6 +263,7 @@ func TestDefaults(t *testing.T) {
 		StateDir:             "/var/lib/sww",
 		StateWriteDelay:      10 * time.Second,
 		StateCounterInterval: 15 * time.Minute,
+		CacheDir:             "/var/cache/sww",
 		MetricsToken:         "",
 		MetricsTopN:          50,
 		RulesDir:             "/etc/sww/rules.d",
@@ -346,6 +348,7 @@ func TestValuesAsSet(t *testing.T) {
 		stateDir:             "/srv/waf-state",
 		stateWriteDelay:      "500ms",
 		stateCounterInterval: "1h",
+		cacheDir:             "/srv/waf-cache",
 		metricsToken:         token,
 		metricsTopN:          "10",
 		rulesDir:             "/srv/waf-rules",
@@ -368,6 +371,7 @@ func TestValuesAsSet(t *testing.T) {
 		StateDir:             "/srv/waf-state",
 		StateWriteDelay:      500 * time.Millisecond,
 		StateCounterInterval: time.Hour,
+		CacheDir:             "/srv/waf-cache",
 		MetricsToken:         token,
 		MetricsTopN:          10,
 		RulesDir:             "/srv/waf-rules",
@@ -2302,6 +2306,7 @@ func TestInvalidBanOrStateValueStopsTheStart(t *testing.T) {
 		{stateDir, ""}, {stateDir, "state"}, {stateDir, "./var/lib/sww"},
 		{stateWriteDelay, off}, {stateWriteDelay, "0s"},
 		{stateCounterInterval, off}, {stateCounterInterval, "15"},
+		{cacheDir, ""}, {cacheDir, "cache"}, {cacheDir, "./var/cache/sww"},
 	})
 }
 
@@ -2574,6 +2579,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
 		stateDir:             "/var/lib/sww",
 		stateWriteDelay:      "10s",
 		stateCounterInterval: "15m",
+		cacheDir:             "/var/cache/sww",
 		adminToken:           "",
 		metricsToken:         "",
 		metricsTopN:          "50",
@@ -2674,8 +2680,8 @@ func wantLookupSettings(t *testing.T, got *config.Config, want config.Config) {
 	}
 }
 
-// wantBanSettings checks the settings for bans, the state files, the
+// wantBanSettings checks the settings for bans, the state and cache
-// metrics and the rule files.
+// files, the metrics and the rule files.
 func wantBanSettings(t *testing.T, got *config.Config, want config.Config) {
 	t.Helper()
 
@@ -2694,11 +2700,7 @@ func wantBanSettings(t *testing.T, got *config.Config, want config.Config) {
 			got.RulesDir, got.RulesEnabled, want.RulesDir, want.RulesEnabled)
 	}
 
-	if got.StateDir != want.StateDir ||
+	wantStateSettings(t, got, want)
-		got.StateWriteDelay != want.StateWriteDelay ||
-		got.StateCounterInterval != want.StateCounterInterval {
-		t.Errorf("state settings\n%+v\nwant\n%+v", got, want)
-	}
 
 	if got.MetricsToken != want.MetricsToken || got.MetricsTopN != want.MetricsTopN {
 		t.Errorf("metrics token %q and top %d, want %q and %d",
@@ -2706,6 +2708,18 @@ func wantBanSettings(t *testing.T, got *config.Config, want config.Config) {
 	}
 }
 
+// wantStateSettings checks the settings for the state and cache files.
+func wantStateSettings(t *testing.T, got *config.Config, want config.Config) {
+	t.Helper()
+
+	if got.StateDir != want.StateDir ||
+		got.StateWriteDelay != want.StateWriteDelay ||
+		got.StateCounterInterval != want.StateCounterInterval ||
+		got.CacheDir != want.CacheDir {
+		t.Errorf("state settings\n%+v\nwant\n%+v", got, want)
+	}
+}
+
 // wantNetblocks checks a list of netblocks.
 func wantNetblocks(t *testing.T, got []netip.Prefix, want ...string) {
 	t.Helper()

internal/lookup/file.go

@@ -26,7 +26,7 @@ type FileParams struct {
 	// Path is the lookup database, the IPinfo Lite file in its .mmdb form
 	// (SWW_LOOKUP_DB_PATH).
 	Path string
-	// Now tells the time, normally time.Now.
+	// Now tells the time, normally time.Now in UTC.
 	Now func() time.Time
 	// ProcessLog receives each reading of the file, and why a replacement
 	// of it cannot be read.

internal/lookup/lookup.go

@@ -74,7 +74,7 @@ type Params struct {
 	// Answered, unless nil, is given each answer GeoJS gives, once it is
 	// kept.
 	Answered func(Answer)
-	// Now tells the time, normally time.Now.
+	// Now tells the time, normally time.Now in UTC.
 	Now func() time.Time
 	// ProcessLog receives GeoJS's failures.
 	ProcessLog *slog.Logger

internal/metrics/metrics.go

@@ -120,17 +120,19 @@ func New(topN int, instanceName string) *Metrics {
 				"went on without it, because GeoJS had not given it in time.",
 		}),
 		stateFileWrites: counterVec("sww_state_file_writes_total",
-			"Writes of each state file.", byFile),
+			"Writes of each state or cache file.", byFile),
 		stateFileWriteFailures: counterVec("sww_state_file_write_failures_total",
-			"Writes of each state file that failed.", byFile),
+			"Writes of each state or cache file that failed.", byFile),
 		stateFileLastWrite: gaugeVec("sww_state_file_last_write_timestamp_seconds",
-			"When each state file was last written, in seconds since 1970.", byFile),
+			"When each state or cache file was last written, in seconds since 1970.",
+			byFile),
 		stateFileSize: gaugeVec("sww_state_file_size_bytes",
-			"The size of each state file, as it was last written.", byFile),
+			"The size of each state or cache file, as it was last written.", byFile),
 		stateFileEditsTakenIn: counterVec("sww_state_file_edits_taken_in_total",
-			"Edits of each state file taken in while running.", byFile),
+			"Edits of each state or cache file taken in while running.", byFile),
 		stateFileEditsSetAside: counterVec("sww_state_file_edits_set_aside_total",
-			"Edits of each state file renamed to <name>.bad because they did not parse.",
+			"Edits of each state or cache file renamed to <name>.bad because they "+
+				"did not parse.",
 			byFile),
 	}
 
@@ -447,8 +449,8 @@ func (m *Metrics) CoreRuleSetMatched(mode string, id int) {
 	m.coreRuleSetMatches.WithLabelValues(mode, strconv.Itoa(id)).Inc()
 }
 
-// StateFileWritten counts a write of the state file name, of size bytes,
+// StateFileWritten counts a write of the state or cache file name, of size
-// that ended with err.
+// bytes, that ended with err.
 func (m *Metrics) StateFileWritten(name string, size int, err error) {
 	m.stateFileWrites.WithLabelValues(name).Inc()
 
@@ -466,13 +468,13 @@ func (m *Metrics) StateFileWritten(name string, size int, err error) {
 	m.stateFileSize.WithLabelValues(name).Set(float64(size))
 }
 
-// StateFileEditTakenIn counts an admin's edit of the state file name
+// StateFileEditTakenIn counts an admin's edit of the state or cache file name
 // taken in while sww runs.
 func (m *Metrics) StateFileEditTakenIn(name string) {
 	m.stateFileEditsTakenIn.WithLabelValues(name).Inc()
 }
 
-// StateFileEditSetAside counts an admin's edit of the state file name
+// StateFileEditSetAside counts an admin's edit of the state or cache file name
 // renamed to name.bad because it did not parse.
 func (m *Metrics) StateFileEditSetAside(name string) {
 	m.stateFileEditsSetAside.WithLabelValues(name).Inc()

internal/reputation/abuseipdb.go

@@ -14,6 +14,7 @@ import (
 	"sync"
 	"time"
 
+	"cloud.google.com/go/civil"
 	"github.com/hashicorp/golang-lru/v2/simplelru"
 	"sneak.berlin/go/smallwebwaf/internal/alerts"
 )
@@ -27,8 +28,6 @@ const (
 	AbuseIPDBSource = "abuseipdb"
 	// maxAnswerBytes is the most of an answer of AbuseIPDB that is read.
 	maxAnswerBytes = 64 << 10
-	// day is the length of the day the checks are counted in, in UTC.
-	day = 24 * time.Hour
 )
 
 var (
@@ -37,22 +36,21 @@ var (
 		"checks spent; none is made until the day ends at 00:00 UTC")
 )
 
-// Score is what AbuseIPDB said about a client, as reputation.json holds
+// Score is what AbuseIPDB said about a client, as abuseipdb-scores.json
-// it: the client, an IPv4 address or an IPv6 group, its abuse confidence
+// holds it: the client, an IPv4 address or an IPv6 group, its abuse
-// score, from 0 to 100, and when AbuseIPDB answered.
+// confidence score, from 0 to 100, and when AbuseIPDB answered.
 type Score struct {
 	Client  netip.Prefix `json:"client"`
 	Score   int64        `json:"score"`
 	Fetched time.Time    `json:"fetched"`
 }
 
-// Checks are what reputation.json keeps of the checks of clients with
+// ChecksSpent is what abuseipdb-checks-spent.json keeps of the checks of
-// AbuseIPDB: the day, in UTC, of the checks Spent counts, zero before the
+// clients with AbuseIPDB: the day, in UTC, of the checks Spent counts,
-// first, and the scores still in use.
+// zero before the first.
-type Checks struct {
+type ChecksSpent struct {
-	Day    time.Time `json:"day,omitzero"`
+	Day   civil.Date
-	Spent  int       `json:"spent"`
+	Spent int
-	Scores []Score   `json:"scores"`
 }
 
 // AbuseIPDBParams are what NewAbuseIPDB needs.
@@ -93,7 +91,7 @@ type AbuseIPDB struct {
 	// checking are the clients whose check is under way.
 	checking map[netip.Prefix]bool
 	// day is the day, in UTC, of the checks spent counts.
-	day   time.Time
+	day   civil.Date
 	spent int
 	// checks and failures count the checks made and those that failed,
 	// and retryAt is when a client may be checked again after the last
@@ -143,7 +141,7 @@ func (a *AbuseIPDB) Hit(
 		return kept.Score, kept.Score >= a.params.MinScore
 	}
 
-	if today := now.Truncate(day); !a.day.Equal(today) {
+	if today := civil.DateOf(now.UTC()); a.day != today {
 		a.day, a.spent = today, 0
 	}
 
@@ -190,42 +188,42 @@ func (a *AbuseIPDB) BudgetLeft() int {
 	a.mu.Lock()
 	defer a.mu.Unlock()
 
-	if !a.day.Equal(a.params.Now().Truncate(day)) {
+	if a.day != civil.DateOf(a.params.Now().UTC()) {
 		return a.params.DailyBudget
 	}
 
 	return max(a.params.DailyBudget-a.spent, 0)
 }
 
-// Snapshot returns the checks spent and every score still in use, sorted
+// Snapshot returns every score still in use, sorted by client, as
-// by client, as reputation.json keeps them.
+// abuseipdb-scores.json keeps them.
-func (a *AbuseIPDB) Snapshot() Checks {
+func (a *AbuseIPDB) Snapshot() []Score {
 	a.mu.Lock()
 
 	now := a.params.Now()
-	checks := Checks{Day: a.day, Spent: a.spent, Scores: make([]Score, 0, a.scores.Len())}
+	scores := make([]Score, 0, a.scores.Len())
 
 	for _, kept := range a.scores.Values() {
 		if now.Sub(kept.Fetched) < a.params.CacheTTL {
-			checks.Scores = append(checks.Scores, kept)
+			scores = append(scores, kept)
 		}
 	}
 
 	a.mu.Unlock()
 
-	slices.SortFunc(checks.Scores, func(x, y Score) int {
+	slices.SortFunc(scores, func(x, y Score) int {
 		return x.Client.Compare(y.Client)
 	})
 
-	return checks
+	return scores
 }
 
-// Load keeps checks, read from reputation.json, in place of those it
+// Load keeps scores, read from abuseipdb-scores.json, in place of those
-// keeps, but for the scores past maxVerdicts, those fetched longest ago.
+// it keeps, but for the scores past maxVerdicts, those fetched longest
-// One fetched CacheTTL ago or more is neither used nor written, as for any
+// ago. One fetched CacheTTL ago or more is neither used nor written, as
-// score.
+// for any score.
-func (a *AbuseIPDB) Load(checks Checks) {
+func (a *AbuseIPDB) Load(scores []Score) {
-	scores := slices.Clone(checks.Scores)
+	scores = slices.Clone(scores)
 	slices.SortStableFunc(scores, func(x, y Score) int {
 		return x.Fetched.Compare(y.Fetched)
 	})
@@ -233,7 +231,6 @@ func (a *AbuseIPDB) Load(checks Checks) {
 	a.mu.Lock()
 	defer a.mu.Unlock()
 
-	a.day, a.spent = checks.Day, checks.Spent
 	a.scores.Purge()
 
 	for _, kept := range scores {
@@ -241,6 +238,24 @@ func (a *AbuseIPDB) Load(checks Checks) {
 	}
 }
 
+// ChecksSpent returns the checks spent, as abuseipdb-checks-spent.json
+// keeps them.
+func (a *AbuseIPDB) ChecksSpent() ChecksSpent {
+	a.mu.Lock()
+	defer a.mu.Unlock()
+
+	return ChecksSpent{Day: a.day, Spent: a.spent}
+}
+
+// LoadChecksSpent keeps checks, read from abuseipdb-checks-spent.json, in
+// place of the checks spent.
+func (a *AbuseIPDB) LoadChecksSpent(checks ChecksSpent) {
+	a.mu.Lock()
+	defer a.mu.Unlock()
+
+	a.day, a.spent = checks.Day, checks.Spent
+}
+
 // check checks client with AbuseIPDB by addr, one of its addresses, keeps
 // the score as client's, and notes the check as no longer under way. A
 // check that fails gives no score: it is counted, logged and raised as a

internal/reputation/abuseipdb_test.go

@@ -17,6 +17,7 @@ import (
 	"testing/synctest"
 	"time"
 
+	"cloud.google.com/go/civil"
 	"sneak.berlin/go/smallwebwaf/internal/alerts"
 	"sneak.berlin/go/smallwebwaf/internal/metrics"
 	"sneak.berlin/go/smallwebwaf/internal/reputation"
@@ -179,11 +180,13 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
 			t.Errorf("logged\n%s\nwant the budget used up", log.String())
 		}
 
-		// Restarted with what reputation.json keeps, it uses the scores, and
+		// Restarted with what abuseipdb-scores.json and
-		// checks no client until the day ends.
+		// abuseipdb-checks-spent.json keep, it uses the scores, and checks
+		// no client until the day ends.
 		restarted := &abuseIPDBStandIn{}
 		again := newAbuseIPDB(restarted, p)
 		again.Load(checker.Snapshot())
+		again.LoadChecksSpent(checker.ChecksSpent())
 
 		wantScore(t, again, suspect, true, 100, true)
 		wantBudgetLeft(t, again, 0)
@@ -202,6 +205,39 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
 	})
 }
 
+func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
+	t.Parallel()
+
+	synctest.Test(t, func(t *testing.T) {
+		// A clock 14 hours ahead of UTC, whose day starts at 10:00 UTC.
+		ahead := time.FixedZone("UTC+14", 14*60*60)
+		p := abuseIPDBParams()
+		p.DailyBudget = 1
+		p.Now = func() time.Time { return time.Now().In(ahead) }
+		abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
+		checker := newAbuseIPDB(abuseIPDB, p)
+
+		// At 09:00 UTC the one check is spent.
+		time.Sleep(9 * time.Hour)
+		hitFrom(t, checker, suspect, true)
+		synctest.Wait()
+		wantBudgetLeft(t, checker, 0)
+
+		// At 11:00 UTC the clock's day has changed, but the UTC day has not.
+		time.Sleep(2 * time.Hour)
+		wantBudgetLeft(t, checker, 0)
+
+		want := civil.Date{Year: 2000, Month: 1, Day: 1}
+		if day := checker.ChecksSpent().Day; day != want {
+			t.Errorf("checks spent on %s, want on %s", day, want)
+		}
+
+		// At 00:00 UTC the budget is whole again.
+		time.Sleep(13 * time.Hour)
+		wantBudgetLeft(t, checker, 1)
+	})
+}
+
 func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
 	t.Parallel()
 
@@ -265,7 +301,7 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
 				wantChecked(t, abuseIPDB, suspect, other)
 				wantFailures(t, checker, 2)
 
-				if scores := checker.Snapshot().Scores; len(scores) != 0 {
+				if scores := checker.Snapshot(); len(scores) != 0 {
 					t.Errorf("scores %+v, want none", scores)
 				}
 
@@ -362,7 +398,7 @@ func TestKeyIsSentInTheKeyHeaderAndNeverShown(t *testing.T) {
 
 		for name, shown := range map[string]string{
 			"the log": log.String(), "the alerts": string(alerted),
-			"the metrics": scrapeMetrics(t, m), "reputation.json": string(kept),
+			"the metrics": scrapeMetrics(t, m), "abuseipdb-scores.json": string(kept),
 		} {
 			if strings.Contains(shown, key) {
 				t.Errorf("%s shows the key:\n%s", name, shown)
@@ -425,12 +461,12 @@ func TestScoreFetchedATTLAgoIsNeitherUsedNorKept(t *testing.T) {
 		Fetched: now.Add(-cacheTTL),
 	}
 
-	checker.Load(reputation.Checks{Scores: []reputation.Score{stale, inUse}})
+	checker.Load([]reputation.Score{stale, inUse})
 
 	wantScore(t, checker, suspect, false, 100, true)
 	wantScore(t, checker, other, false, 0, false)
 
-	got := checker.Snapshot().Scores
+	got := checker.Snapshot()
 	if !reflect.DeepEqual(got, []reputation.Score{inUse}) {
 		t.Errorf("scores %+v, want only %+v", got, inUse)
 	}
@@ -444,7 +480,7 @@ func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T)
 	p.Now = func() time.Time { return now }
 	checker := reputation.NewAbuseIPDB(p)
 
-	// 100,001 scores, listed by client, as reputation.json lists them, each
+	// 100,001 scores, listed by client, as abuseipdb-scores.json lists them, each
 	// fetched a millisecond before the one before it: the last is one too
 	// many.
 	const count = 100001
@@ -460,9 +496,9 @@ func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T)
 		addr = addr.Next()
 	}
 
-	checker.Load(reputation.Checks{Scores: scores})
+	checker.Load(scores)
 
-	got := checker.Snapshot().Scores
+	got := checker.Snapshot()
 	if len(got) != count-1 || !slices.Contains(got, scores[0]) ||
 		slices.Contains(got, scores[count-1]) {
 		t.Errorf("%d scores kept, want all but the one fetched longest ago", len(got))

internal/reputation/crowdsec_test.go

@@ -291,7 +291,7 @@ func crowdSecFailures() []crowdSecFailure {
 // other than its decision list, such as the one a redirect names, and that
 // the key is in none of what the fetches leave behind: log, the process
 // log, the alerts waiting in queue, and the copies of lists, which
-// reputation.json keeps.
+// lists.json keeps.
 func wantKeyNotShown(
 	t *testing.T, e *engine, log string, lists *reputation.Lists, queue *alerts.Queue,
 ) {
@@ -326,7 +326,7 @@ func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(t *testing
 		lists := start(t, e, crowdSecParams())
 		kept := lists.Snapshot()
 
-		// Restarted half an hour later with what reputation.json keeps, and
+		// Restarted half an hour later with what lists.json keeps, and
 		// the engine down, the decision still bans, until the end it had at
 		// the fetch, half an hour on.
 		time.Sleep(30 * time.Minute)

internal/reputation/dnsbl.go

@@ -40,7 +40,7 @@ var (
 	errNotListing = errors.New("the answer is outside 127.0.0.0/8")
 )
 
-// Verdict is what a zone said about a client, as reputation.json holds
+// Verdict is what a zone said about a client, as dnsbl.json holds
 // it: the zone, the client's address, whether the zone lists it, and when
 // the zone answered.
 type Verdict struct {
@@ -187,7 +187,7 @@ func (d *DNSBL) Failures(zone string) int {
 }
 
 // Snapshot returns every verdict still in use, sorted by client, then by
-// zone, as reputation.json lists them.
+// zone, as dnsbl.json lists them.
 func (d *DNSBL) Snapshot() []Verdict {
 	d.mu.Lock()
 
@@ -209,7 +209,7 @@ func (d *DNSBL) Snapshot() []Verdict {
 	return verdicts
 }
 
-// Load keeps verdicts, read from reputation.json, in place of those it
+// Load keeps verdicts, read from dnsbl.json, in place of those it
 // keeps, but for those of a zone SWW_DNSBL_ZONES does not name, and,
 // past maxVerdicts, those fetched longest ago. One fetched CacheTTL ago or
 // more is neither used nor written, as for any verdict.

internal/reputation/dnsbl_test.go

@@ -389,7 +389,7 @@ func TestVerdictsKeptAcrossARestart(t *testing.T) {
 			t.Errorf("verdicts %+v, want %+v", kept, want)
 		}
 
-		// Restarted an hour later with what reputation.json keeps, it uses
+		// Restarted an hour later with what dnsbl.json keeps, it uses
 		// the verdicts, and asks the zone nothing, until the TTL has passed
 		// since they were fetched.
 		time.Sleep(time.Hour)
@@ -446,7 +446,7 @@ func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.
 	p.Now = func() time.Time { return now }
 	dnsbl := reputation.NewDNSBL(p)
 
-	// 100,001 verdicts, listed by client, as reputation.json lists them,
+	// 100,001 verdicts, listed by client, as dnsbl.json lists them,
 	// each fetched a millisecond before the one before it: the last is one
 	// too many.
 	const count = 100001

internal/reputation/reputation.go

@@ -7,8 +7,8 @@
 // and when each was last tried. It also asks the DNSBL zones of
 // SWW_DNSBL_ZONES about clients, and keeps their verdicts, and checks
 // clients with AbuseIPDB, and keeps their scores and the checks spent
-// today. The state package writes all of these to reputation.json and
+// today. The state package writes all of these to files and reads them
-// reads them from it, so that a restart keeps them too.
+// from there, so that a restart keeps them too.
 package reputation
 
 import (
@@ -54,7 +54,7 @@ var (
 		"does not give an address or a netblock and a duration, such as 4h0m0s")
 )
 
-// List is a list as reputation.json holds it: the URL it is fetched from,
+// List is a list as lists.json holds it: the URL it is fetched from,
 // when it was last tried, the fetch failed or not, and its last good copy:
 // when that was fetched, and its lines, as fetched, comment lines
 // included, both left out while no fetch of it has succeeded.
@@ -106,7 +106,7 @@ type Lists struct {
 	lists map[string]*list
 }
 
-// list is one list: what reputation.json keeps of it, its last try, zero
+// list is one list: what lists.json keeps of it, its last try, zero
 // before the first, and its last good copy, what that copy says, and how
 // many fetches of it failed.
 type list struct {
@@ -272,7 +272,7 @@ func (l *Lists) Failures(listURL string) int {
 
 // Run fetches each list once it is due, as due tells, until ctx is done. A
 // list never tried is fetched at once, and so is one that is due by its
-// last try or copy read from reputation.json.
+// last try or copy read from lists.json.
 func (l *Lists) Run(ctx context.Context) {
 	if len(l.lists) == 0 {
 		return
@@ -292,7 +292,7 @@ func (l *Lists) Run(ctx context.Context) {
 }
 
 // Snapshot returns each list that has been tried, with its copy, if it
-// has one, sorted by URL, as reputation.json lists them.
+// has one, sorted by URL, as lists.json lists them.
 func (l *Lists) Snapshot() []List {
 	l.mu.Lock()
 
@@ -313,7 +313,7 @@ func (l *Lists) Snapshot() []List {
 	return tried
 }
 
-// Load puts lists, read from reputation.json, in place of the last tries
+// Load puts lists, read from lists.json, in place of the last tries
 // and copies held. A list Params does not name is dropped. A copy with a
 // line that parse refuses is an error, and then nothing changes.
 func (l *Lists) Load(lists []List) error {

internal/reputation/reputation_test.go

@@ -260,7 +260,7 @@ func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(t *testing.
 		lists.SetTransport(servers)
 
 		// drop.txt was fetched an hour ago, and tor.txt a refresh ago, as
-		// reputation.json says at start.
+		// lists.json says at start.
 		err := lists.Load([]reputation.List{
 			{URL: dropURL, Fetched: time.Now().Add(-time.Hour), Lines: []string{"203.0.113.9"}},
 			{URL: torURL, Fetched: time.Now().Add(-refresh), Lines: []string{"198.51.100.9"}},
@@ -297,7 +297,7 @@ func TestRestartWaitsRefreshAfterTheLastTryEvenOneThatFailed(t *testing.T) {
 		time.Sleep(refresh)
 		wantFetches(t, servers, 2)
 
-		// Restarted with what reputation.json keeps, it waits a refresh
+		// Restarted with what lists.json keeps, it waits a refresh
 		// after the failed try, as it does while it runs.
 		restarted := &standIn{lists: map[string]string{dropURL: "198.51.100.2\n"}}
 		again := reputation.New(params(dropURL))
@@ -353,7 +353,7 @@ func TestFetchCutOffAsItStopsIsNoFailureButARestartWaitsForIt(t *testing.T) {
 				waiting(queue))
 		}
 
-		// Restarted an hour later with what reputation.json keeps, it fetches
+		// Restarted an hour later with what lists.json keeps, it fetches
 		// tor.txt, never tried, at once, and drop.txt a refresh after its
 		// cut-off try.
 		time.Sleep(time.Hour)

internal/requestlog/requestlog.go

@@ -5,6 +5,7 @@
 package requestlog
 
 import (
+	"bytes"
 	"encoding/json"
 	"fmt"
 	"io"
@@ -252,14 +253,20 @@ func WriteRProxyLine(w io.Writer, line *RProxyLine) error {
 	return writeJSONLine(w, line)
 }
 
-// writeJSONLine writes line to w as one JSON line.
+// writeJSONLine writes line to w as one JSON line. It writes "&", "<" and
+// ">" as they are, so a query string reads as it was sent.
 func writeJSONLine(w io.Writer, line any) error {
-	encoded, err := json.Marshal(line)
+	var encoded bytes.Buffer
+
+	encoder := json.NewEncoder(&encoded)
+	encoder.SetEscapeHTML(false)
+
+	err := encoder.Encode(line)
 	if err != nil {
 		return fmt.Errorf("encode the request log line: %w", err)
 	}
 
-	_, err = w.Write(append(encoded, '\n'))
+	_, err = w.Write(encoded.Bytes())
 	if err != nil {
 		return fmt.Errorf("write the request log line: %w", err)
 	}

internal/requestlog/requestlog_test.go

@@ -67,6 +67,44 @@ func TestWriteWritesOneJSONLineMarkedRequest(t *testing.T) {
 	}
 }
 
+func TestRequestLinesHoldTheQueryAndRefererAsSent(t *testing.T) {
+	t.Parallel()
+
+	var out bytes.Buffer
+
+	err := requestlog.Write(&out, &requestlog.Line{
+		Query:   "a=1&b=<2>",
+		Referer: "https://example.com/?x=1&y=<2>",
+	})
+	if err != nil {
+		t.Fatalf("write: %v", err)
+	}
+
+	err = requestlog.WriteRProxyLine(&out, &requestlog.RProxyLine{Query: "a=1&b=<2>"})
+	if err != nil {
+		t.Fatalf("write the rproxy line: %v", err)
+	}
+
+	lines := strings.Split(strings.TrimSuffix(out.String(), "\n"), "\n")
+	if len(lines) != 2 {
+		t.Fatalf("wrote %q, want two lines", out.String())
+	}
+
+	for _, line := range lines {
+		if !json.Valid([]byte(line)) {
+			t.Errorf("%s is not valid JSON", line)
+		}
+
+		if !strings.Contains(line, `"query":"a=1&b=<2>"`) {
+			t.Errorf("%s does not hold the query as sent", line)
+		}
+	}
+
+	if !strings.Contains(lines[0], `"referer":"https://example.com/?x=1&y=<2>"`) {
+		t.Errorf("%s does not hold the referer as sent", lines[0])
+	}
+}
+
 func TestWriteRProxyLineWritesOneJSONLineWithOnlyItsFields(t *testing.T) {
 	t.Parallel()
 

internal/routewatch/routewatch.go

@@ -1,6 +1,8 @@
 // Package routewatch asks a routewatch instance about client addresses, in the
-// background, and keeps its answers in memory. A routewatch instance knows, for
+// background, and keeps its answers. A routewatch instance knows, for an
-// an address, the netblock it is announced in and the AS that originates it.
+// address, the netblock it is announced in and the AS that originates it. The
+// answers are kept in memory, for at most 100,000 clients and for an hour each,
+// and are written to routewatch.json and read from it by the state package.
 package routewatch
 
 import (
@@ -13,6 +15,7 @@ import (
 	"net/http"
 	"net/netip"
 	"net/url"
+	"slices"
 	"sync"
 	"time"
 
@@ -65,6 +68,17 @@ type Route struct {
 	ASN      uint32
 }
 
+// Answer is what routewatch answered about a client, as routewatch.json holds
+// it: the netblock the client's address is announced in, none for an address
+// without a live route, the AS that originates it, and when routewatch
+// answered.
+type Answer struct {
+	Client   netip.Prefix `json:"client"`
+	Netblock netip.Prefix `json:"netblock"`
+	ASN      uint32       `json:"asn"`
+	Answered time.Time    `json:"answered"`
+}
+
 // Params are what New needs.
 type Params struct {
 	// URL is where routewatch is (SWW_ROUTEWATCH_URL).
@@ -84,23 +98,16 @@ type Client struct {
 	mu sync.Mutex
 	// answers are by client. Each is added as it comes and never moved up,
 	// so that the one that came longest ago is the first dropped.
-	answers *simplelru.LRU[netip.Prefix, answer]
+	answers *simplelru.LRU[netip.Prefix, Answer]
 	// asking are the clients whose query is under way.
 	asking map[netip.Prefix]bool
 	// retryAt is when routewatch may be asked again after a query failed.
 	retryAt time.Time
 }
 
-// answer is one kept answer: the route, or why there is none, and when it came.
-type answer struct {
-	route   Route
-	missing Missing
-	came    time.Time
-}
-
 // New returns a Client with no answer yet.
 func New(params Params) *Client {
-	answers, err := simplelru.NewLRU[netip.Prefix, answer](maxAnswers, nil)
+	answers, err := simplelru.NewLRU[netip.Prefix, Answer](maxAnswers, nil)
 	if err != nil {
 		panic(err) // NewLRU fails only for a size below one
 	}
@@ -128,8 +135,12 @@ func (c *Client) Route(
 	now := c.params.Now()
 
 	kept, found := c.answers.Peek(client)
-	if found && now.Sub(kept.came) < answerKept {
+	if found && now.Sub(kept.Answered) < answerKept {
-		return kept.route, kept.missing
+		if !kept.Netblock.IsValid() {
+			return Route{}, NoRoute
+		}
+
+		return Route{Netblock: kept.Netblock, ASN: kept.ASN}, ""
 	}
 
 	if now.Before(c.retryAt) {
@@ -145,12 +156,58 @@ func (c *Client) Route(
 	return Route{}, NoAnswerYet
 }
 
+// Snapshot returns every answer still in use, sorted by client, as
+// routewatch.json lists them.
+func (c *Client) Snapshot() []Answer {
+	c.mu.Lock()
+
+	now := c.params.Now()
+	answers := make([]Answer, 0, c.answers.Len())
+
+	for _, kept := range c.answers.Values() {
+		if now.Sub(kept.Answered) < answerKept {
+			answers = append(answers, kept)
+		}
+	}
+
+	c.mu.Unlock()
+
+	slices.SortFunc(answers, func(a, b Answer) int {
+		return a.Client.Compare(b.Client)
+	})
+
+	return answers
+}
+
+// Load keeps answers, read from routewatch.json, in place of those it keeps,
+// but for those that came answerKept ago or more and, past maxAnswers, those
+// that came longest ago.
+func (c *Client) Load(answers []Answer) {
+	answers = slices.Clone(answers)
+	slices.SortStableFunc(answers, func(a, b Answer) int {
+		return a.Answered.Compare(b.Answered)
+	})
+
+	c.mu.Lock()
+	defer c.mu.Unlock()
+
+	c.answers.Purge()
+
+	now := c.params.Now()
+
+	for _, kept := range answers {
+		if now.Sub(kept.Answered) < answerKept {
+			c.answers.Add(kept.Client, kept)
+		}
+	}
+}
+
 // ask asks routewatch about addr, keeps the answer as client's, and notes the
 // query as no longer under way. A query that fails gives no answer, and
 // routewatch is not asked again for failureKept. Only the first failure of
 // those under way is logged.
 func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr) {
-	route, missing, err := c.query(ctx, addr)
+	route, err := c.query(ctx, addr)
 	now := c.params.Now()
 
 	c.mu.Lock()
@@ -160,7 +217,9 @@ func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr)
 	firstFailure := err != nil && !now.Before(c.retryAt)
 
 	if err == nil {
-		c.answers.Add(client, answer{route: route, missing: missing, came: now})
+		c.answers.Add(client, Answer{
+			Client: client, Netblock: route.Netblock, ASN: route.ASN, Answered: now,
+		})
 	} else {
 		c.retryAt = now.Add(failureKept)
 	}
@@ -174,19 +233,19 @@ func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr)
 
 // query asks routewatch's IP info API about addr. An answer other than 200, one
 // that cannot be read, and none within queryTimeout, fail. A result with no
-// netblock is an address without a live route.
+// netblock is an address without a live route, and gives a Route without one.
-func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, error) {
+func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, error) {
 	ctx, cancel := context.WithTimeout(ctx, queryTimeout)
 	defer cancel()
 
 	endpoint, err := url.JoinPath(c.params.URL, "api/v1/ip", addr.String())
 	if err != nil {
-		return Route{}, "", fmt.Errorf("make the URL: %w", err)
+		return Route{}, fmt.Errorf("make the URL: %w", err)
 	}
 
 	req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, http.NoBody)
 	if err != nil {
-		return Route{}, "", fmt.Errorf("make the request: %w", err)
+		return Route{}, fmt.Errorf("make the request: %w", err)
 	}
 
 	req.Header.Set("Accept", "application/json")
@@ -195,7 +254,7 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, er
 	if err != nil {
 		// Do's error names the URL, which holds the client's address, which
 		// is not to be logged: only what went wrong is kept.
-		return Route{}, "", fmt.Errorf("ask routewatch: %w", errors.Unwrap(err))
+		return Route{}, fmt.Errorf("ask routewatch: %w", errors.Unwrap(err))
 	}
 
 	defer func() {
@@ -203,14 +262,14 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, er
 	}()
 
 	if res.StatusCode != http.StatusOK {
-		return Route{}, "", fmt.Errorf("%w %s", errStatus, res.Status)
+		return Route{}, fmt.Errorf("%w %s", errStatus, res.Status)
 	}
 
 	return readAnswer(io.LimitReader(res.Body, maxAnswerBytes))
 }
 
 // readAnswer reads routewatch's answer about one address.
-func readAnswer(body io.Reader) (Route, Missing, error) {
+func readAnswer(body io.Reader) (Route, error) {
 	var decoded struct {
 		Data struct {
 			Results []struct {
@@ -222,22 +281,22 @@ func readAnswer(body io.Reader) (Route, Missing, error) {
 
 	err := json.NewDecoder(body).Decode(&decoded)
 	if err != nil {
-		return Route{}, "", fmt.Errorf("read the answer: %w", err)
+		return Route{}, fmt.Errorf("read the answer: %w", err)
 	}
 
 	if len(decoded.Data.Results) == 0 {
-		return Route{}, "", errNoResult
+		return Route{}, errNoResult
 	}
 
 	result := decoded.Data.Results[0]
 	if result.Netblock == "" {
-		return Route{}, NoRoute, nil
+		return Route{}, nil
 	}
 
 	netblock, err := netip.ParsePrefix(result.Netblock)
 	if err != nil {
-		return Route{}, "", fmt.Errorf("%w: %q", errNotNetblock, result.Netblock)
+		return Route{}, fmt.Errorf("%w: %q", errNotNetblock, result.Netblock)
 	}
 
-	return Route{Netblock: netblock.Masked(), ASN: result.ASN}, "", nil
+	return Route{Netblock: netblock.Masked(), ASN: result.ASN}, nil
 }

internal/routewatch/routewatch_test.go

@@ -7,10 +7,12 @@ import (
 	"net/http"
 	"net/http/httptest"
 	"net/netip"
+	"slices"
 	"strings"
 	"sync"
 	"sync/atomic"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	"sneak.berlin/go/smallwebwaf/internal/routewatch"
@@ -113,6 +115,28 @@ func answering(body string) http.HandlerFunc {
 	}
 }
 
+// silentURL is where a routewatch that never answers is: a query to it waits,
+// without the network, until it is abandoned. TestMain registers it with Go's
+// default transport, through which the Client asks.
+const silentURL = "silent://routewatch/"
+
+func TestMain(m *testing.M) {
+	transport, _ := http.DefaultTransport.(*http.Transport)
+	transport.RegisterProtocol("silent", silentRoutewatch{})
+
+	m.Run()
+}
+
+// silentRoutewatch is the routewatch at silentURL.
+type silentRoutewatch struct{}
+
+// RoundTrip waits until req is abandoned.
+func (silentRoutewatch) RoundTrip(req *http.Request) (*http.Response, error) {
+	<-req.Context().Done()
+
+	return nil, req.Context().Err()
+}
+
 func (f *fixture) route() (routewatch.Route, routewatch.Missing) {
 	addr := netip.MustParseAddr(clientAddr)
 	client := netip.PrefixFrom(addr, addr.BitLen())
@@ -287,20 +311,29 @@ func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(t *testing.T) {
 func TestHangingRoutewatchNeverDelaysAndFailsTheQuery(t *testing.T) {
 	t.Parallel()
 
-	f := newFixture(t, func(_ http.ResponseWriter, r *http.Request) {
+	// In a synctest bubble, time passes only while every goroutine in it waits,
-		<-r.Context().Done()
+	// so Route's wait is measured exactly, however busy the host is. routewatch
+	// is the one at silentURL: one reached over the network would keep the
+	// bubble's clock from moving.
+	synctest.Test(t, func(t *testing.T) {
+		// Without newFixture's stand-in, which is reached over the network.
+		f := &fixture{t: t, log: &syncBuffer{}}
+		f.client = routewatch.New(routewatch.Params{
+			URL: silentURL, Now: time.Now,
+			ProcessLog: slog.New(slog.NewTextHandler(f.log, nil)),
+		})
+
+		began := time.Now()
+		_, missing := f.route()
+
+		if waited := time.Since(began); missing != routewatch.NoAnswerYet || waited != 0 {
+			t.Errorf("Route gave %q after %s while routewatch hangs, want %q at once",
+				missing, waited, routewatch.NoAnswerYet)
+		}
+
+		f.waitFor(routewatch.RequestFailed)
+		f.wantFailureLoggedWithoutTheClient()
 	})
-
-	start := time.Now()
-
-	f.route()
-
-	if elapsed := time.Since(start); elapsed > time.Second {
-		t.Errorf("Route took %s while routewatch hangs", elapsed)
-	}
-
-	f.waitFor(routewatch.RequestFailed)
-	f.wantFailureLoggedWithoutTheClient()
 }
 
 func TestRepeatedRequestsMakeOneQueryAndTheAnswerExpires(t *testing.T) {
@@ -336,3 +369,96 @@ func TestRepeatedRequestsMakeOneQueryAndTheAnswerExpires(t *testing.T) {
 		t.Errorf("%d queries after the answer expired, want 2", got)
 	}
 }
+
+func TestSnapshotHoldsTheAnswerUntilItExpires(t *testing.T) {
+	t.Parallel()
+
+	f := newFixture(t, answering(routedAnswer))
+
+	f.route()
+	f.waitFor("")
+
+	want := []routewatch.Answer{{
+		Client:   netip.MustParsePrefix(clientAddr + "/32"),
+		Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496,
+		Answered: f.clk.Now(),
+	}}
+	if got := f.client.Snapshot(); !slices.Equal(got, want) {
+		t.Errorf("snapshot %+v, want %+v", got, want)
+	}
+
+	f.clk.advance(time.Hour)
+
+	if got := f.client.Snapshot(); len(got) != 0 {
+		t.Errorf("snapshot %+v an hour after the answer, want none", got)
+	}
+}
+
+func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
+	t.Parallel()
+
+	f := newFixture(t, answering(routedAnswer))
+	now := f.clk.Now()
+	kept := routewatch.Answer{
+		Client:   netip.MustParsePrefix(clientAddr + "/32"),
+		Netblock: netip.MustParsePrefix("198.51.100.0/24"), ASN: 64497,
+		Answered: now.Add(-time.Hour + time.Second),
+	}
+	stale := routewatch.Answer{
+		Client:   netip.MustParsePrefix("192.0.2.2/32"),
+		Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496,
+		Answered: now.Add(-time.Hour),
+	}
+
+	f.client.Load([]routewatch.Answer{kept, stale})
+
+	if got := f.client.Snapshot(); !slices.Equal(got, []routewatch.Answer{kept}) {
+		t.Errorf("kept %+v, want only the answer that came less than an hour ago", got)
+	}
+
+	route, missing := f.route()
+
+	want := routewatch.Route{Netblock: kept.Netblock, ASN: kept.ASN}
+	if route != want || missing != "" {
+		t.Errorf("route %+v (%q), want %+v", route, missing, want)
+	}
+
+	if got := f.queries.Load(); got != 0 {
+		t.Errorf("%d queries for a client with an answer loaded, want none", got)
+	}
+}
+
+func TestLoadDropsTheAnswerThatCameLongestAgoFirst(t *testing.T) {
+	t.Parallel()
+
+	const maxAnswers = 100000
+
+	f := newFixture(t, answering(routedAnswer))
+	now := f.clk.Now()
+
+	// routewatch.json lists the answers by client. Here each came a
+	// millisecond before the one listed before it, so the last listed is
+	// the one that came longest ago, and the one dropped.
+	answers := make([]routewatch.Answer, maxAnswers+1)
+	addr := netip.MustParseAddr("10.0.0.0")
+
+	for i := range answers {
+		answers[i] = routewatch.Answer{
+			Client:   netip.PrefixFrom(addr, addr.BitLen()),
+			Answered: now.Add(-time.Duration(i) * time.Millisecond),
+		}
+		addr = addr.Next()
+	}
+
+	f.client.Load(answers)
+
+	got := f.client.Snapshot()
+	if len(got) != maxAnswers {
+		t.Fatalf("%d answers kept, want %d", len(got), maxAnswers)
+	}
+
+	if got[0] != answers[0] || got[maxAnswers-1] != answers[maxAnswers-1] {
+		t.Errorf("answers kept from %s to %s, want from %s to %s", got[0].Client,
+			got[maxAnswers-1].Client, answers[0].Client, answers[maxAnswers-1].Client)
+	}
+}

internal/rproxy/client.go

@@ -105,15 +105,14 @@ func setForwardedHeaders(in, out *http.Request, peer netip.Addr, trusted bool) {
 		return
 	}
 
-	// ReverseProxy has removed Forwarded and the three set below; these are the
+	// An app behind CGI or FastCGI reads X_Forwarded_Ssl as X-Forwarded-Ssl, so
-	// other headers in which traefik tells the app about the visitor and its
+	// a name is matched with each _ read as - and in any letter case.
-	// request.
+	for name := range out.Header {
-	for _, name := range []string{
+		spelt := strings.ToLower(strings.ReplaceAll(name, "_", "-"))
-		"X-Forwarded-Port", "X-Forwarded-Server", "X-Forwarded-Uri",
+		if strings.HasPrefix(spelt, "x-forwarded-") ||
-		"X-Forwarded-Method", "X-Forwarded-Prefix", "X-Forwarded-Tls-Client-Cert",
+			spelt == "forwarded" || spelt == "x-real-ip" {
-		"X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip",
+			delete(out.Header, name)
-	} {
+		}
-		out.Header.Del(name)
 	}
 
 	out.Header.Set("X-Forwarded-For", forwardedFor)

internal/rproxy/limits_test.go

@@ -382,6 +382,20 @@ type deadlineRecorder struct {
 	// stalled, if set, sends the part of the request body that never arrives. A
 	// read deadline that has passed ends a read of it, as on a connection.
 	stalled *io.PipeWriter
+	// flushError, if set, is what every flush returns, and nothing is flushed.
+	flushError error
+}
+
+// FlushError, which http.ResponseController calls to flush, returns flushError
+// if it is set, and otherwise flushes.
+func (w *deadlineRecorder) FlushError() error {
+	if w.flushError != nil {
+		return w.flushError
+	}
+
+	w.Flush()
+
+	return nil
 }
 
 // SetReadDeadline notes deadline.
@@ -447,6 +461,45 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
 	}
 }
 
+func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(t *testing.T) {
+	t.Parallel()
+
+	standIn := startWAF(t, allow)
+	app := startApp(t, answerBeforeTheBody)
+	_, _, server := startServer(t, standIn.URL, app.URL, nil)
+
+	// The visitor sends the first part and a little more, and then nothing.
+	stalled, stop := io.Pipe()
+
+	t.Cleanup(func() { _ = stop.Close() })
+
+	req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
+		io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
+	req.ContentLength = 2 * questionBodyMaxBytes
+	req.Header.Set(contentLengthHeader, strconv.Itoa(2*questionBodyMaxBytes))
+
+	// Every flush fails, as once the time to send the answer has run out. The
+	// default time limits are far longer than the test, so rproxy records no
+	// refusal.
+	w := &deadlineRecorder{
+		ResponseRecorder: httptest.NewRecorder(),
+		stalled:          stop,
+		flushError:       os.ErrDeadlineExceeded,
+	}
+
+	before := time.Now()
+
+	server.Handler.ServeHTTP(w, req)
+
+	if w.Code != http.StatusOK {
+		t.Errorf("status %d, want the app's 200", w.Code)
+	}
+
+	if w.readDeadline.Before(before) || w.readDeadline.After(time.Now()) {
+		t.Errorf("the reading of the body ends at %s, want now", w.readDeadline)
+	}
+}
+
 func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
 	t.Parallel()
 
@@ -807,8 +860,10 @@ func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.
 	app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
 		_, _ = io.WriteString(w, greeting)
 	})
+	// The request time limit is far longer than dial's deadline, so that a busy
+	// host cannot use it up before the app's answer arrives.
 	addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
-		maxRequestTime: "3000",
+		maxRequestTime: "120000",
 	})
 
 	// Far more announced than sent, so that Go's server answers for the app
@@ -869,11 +924,11 @@ func TestAVisitorThatStopsALittleShortGetsTheAppsAnswerAtOnce(t *testing.T) {
 
 	standIn := startWAF(t, allow)
 	app := startApp(t, answerBeforeTheBody)
-	// The request time limit is longer than the 10 seconds the status and headers
+	// Both time limits are far longer than dial's deadline, so that the answer
-	// of an answer have to be sent, and than dial's deadline.
+	// has to come before either runs out, and a busy host cannot use them up.
 	addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
-		maxRequestTime:  "15000",
+		maxRequestTime:  "120000",
-		maxResponseTime: "300",
+		maxResponseTime: "60000",
 	})
 
 	conn := dial(t, addr)
@@ -910,6 +965,77 @@ func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(t *testing.
 	}
 }
 
+func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOff(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	// rproxy passes on a part sent at once well within this limit, even on a busy
+	// host.
+	const limit = 2 * time.Second
+
+	// The app sends the parts in atOnce with its status and headers. It sends the
+	// parts in late only once rproxy has ended the request to it, which rproxy
+	// does when it cuts off the answer, so that those never reach the visitor.
+	for _, tc := range []struct {
+		name         string
+		atOnce, late []string
+	}{
+		{"with the body in one part", nil, []string{greeting}},
+		{"with the body in two parts", []string{greeting[:5]}, []string{greeting[5:]}},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			standIn := startWAF(t, allow)
+			app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
+				rc := http.NewResponseController(w)
+				_ = rc.EnableFullDuplex()
+
+				w.Header().Set(contentLengthHeader, strconv.Itoa(len(greeting)))
+
+				for _, part := range tc.atOnce {
+					_, _ = io.WriteString(w, part)
+				}
+
+				_ = rc.Flush()
+
+				// The request's body ends when rproxy ends the request to the app.
+				_, _ = io.Copy(io.Discard, r.Body)
+
+				for _, part := range tc.late {
+					_, _ = io.WriteString(w, part)
+					_ = rc.Flush()
+				}
+			})
+
+			// The request time limit is far longer than dial's deadline.
+			addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
+				maxRequestTime:  "120000",
+				maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
+			})
+
+			conn := dial(t, addr)
+			sendAllButALittle(t, conn)
+
+			res, err := http.ReadResponse(bufio.NewReader(conn), nil)
+			if err != nil {
+				t.Fatalf("read the answer: %v", err)
+			}
+
+			// If the connection stays open, dial's deadline ends this read with a
+			// timeout.
+			body, err := io.ReadAll(res.Body)
+			_ = res.Body.Close()
+
+			want := strings.Join(tc.atOnce, "")
+			if string(body) != want || !errors.Is(err, io.ErrUnexpectedEOF) {
+				t.Errorf("read %q (%v), want %q and the connection closed", body, err, want)
+			}
+		})
+	}
+}
+
 func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
 	t.Parallel()
 

internal/rproxy/request.go

@@ -272,6 +272,10 @@ func (rq *request) modifyResponse(res *http.Response) error {
 
 	rq.closeAfterAnswer()
 
+	// An answer that arrives before the app has been sent the whole request ends
+	// the request, and SWRP_MAX_RESPONSE_TIME_MS starts.
+	rq.startResponseTimer(requestEnd)
+
 	res.Body = &responseBody{body: limitBody(res.Body, maxBytes), rq: rq}
 	rq.out.afterStatus = rq.setResponseDeadline
 
@@ -384,8 +388,23 @@ func (rq *request) refuse(status int) {
 func (rq *request) finish(ctx context.Context) {
 	rq.stopTimers()
 
-	if rq.refused.Load() == 0 {
+	// After a cut-off answer, the reading of a body that has not arrived whole
-		rq.stopPassingOnBody()
+	// ends now. Go's server would otherwise keep the connection open while it
+	// reads the rest. modifyResponse sets afterStatus only when it passes the
+	// app's answer on. Any refusal after that has cut the answer off, since a
+	// refusal ends the request to the app. Any answer is cut off when the time to
+	// send it runs out: a flush then fails, and every flush after a failed one
+	// fails too. ReverseProxy ignores the failure, but panics if it was still
+	// waiting for more of the app's answer; finish runs either way. Before a
+	// status, a flush would send a 200; after a 101, ReverseProxy has taken over
+	// the connection.
+	switch {
+	case rq.out.afterStatus != nil && rq.refused.Load() != 0,
+		rq.out.status != 0 && rq.out.status != http.StatusSwitchingProtocols &&
+			errors.Is(rq.rc.Flush(), os.ErrDeadlineExceeded):
+		rq.stopPassingOnBody(time.Now())
+	case rq.refused.Load() == 0:
+		rq.stopPassingOnBody(rq.requestDeadline())
 	}
 
 	requestEnd := rq.endRequest()
@@ -473,10 +492,10 @@ func (rq *request) stopReadingBody(deadline time.Time) {
 }
 
 // stopPassingOnBody ends the transport's reading of a visitor's body that has
-// not arrived whole, and lets Go's server read the rest until
+// not arrived whole, and lets Go's server read the rest until deadline. Go's
-// SWRP_MAX_REQUEST_TIME_MS runs out. Go's server reads it once the handler
+// server reads it once the handler returns, but with no deadline at all if
-// returns, but with no deadline at all if another read of it is still waiting.
+// another read of it is still waiting.
-func (rq *request) stopPassingOnBody() {
+func (rq *request) stopPassingOnBody(deadline time.Time) {
 	if rq.body == nil || rq.body.received.Load() {
 		return
 	}
@@ -489,7 +508,7 @@ func (rq *request) stopPassingOnBody() {
 	rq.body.stopped = true
 	rq.body.mu.Unlock()
 
-	rq.stopReadingBody(rq.requestDeadline())
+	rq.stopReadingBody(deadline)
 }
 
 // startRequestTimer starts SWRP_MAX_REQUEST_TIME_MS, which runs from the
@@ -542,19 +561,29 @@ func (rq *request) wroteRequest(info httptrace.WroteRequestInfo) {
 	requestEnd := rq.endRequest()
 
 	rq.mu.Lock()
-	defer rq.mu.Unlock()
+	stopTimer(rq.requestTimer)
+	rq.mu.Unlock()
 
-	if rq.timersStopped {
+	rq.startResponseTimer(requestEnd)
+}
+
+// startResponseTimer starts SWRP_MAX_RESPONSE_TIME_MS, which runs from
+// requestEnd, unless it has started already.
+func (rq *request) startResponseTimer(requestEnd time.Time) {
+	limit := rq.h.config.MaxResponseTime
+	if limit == 0 {
 		return
 	}
 
-	stopTimer(rq.requestTimer)
+	rq.mu.Lock()
+	defer rq.mu.Unlock()
 
-	limit := rq.h.config.MaxResponseTime
+	if rq.timersStopped || rq.responseTimer != nil {
-	if limit > 0 {
+		return
-		rq.responseTimer = time.AfterFunc(time.Until(requestEnd.Add(limit)),
-			rq.responseTimedOut)
 	}
+
+	rq.responseTimer = time.AfterFunc(time.Until(requestEnd.Add(limit)),
+		rq.responseTimedOut)
 }
 
 // responseTimedOut is called when SWRP_MAX_RESPONSE_TIME_MS runs out before the

internal/rproxy/rproxy_test.go

@@ -559,6 +559,9 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
 		"X-Forwarded-Port", "X-Forwarded-Server", "X-Forwarded-Uri",
 		"X-Forwarded-Method", "X-Forwarded-Prefix", "X-Forwarded-Tls-Client-Cert",
 		"X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip",
+		"X-Forwarded-Scheme", "X-Forwarded-Ssl", "X-Forwarded-Protocol",
+		"X-Forwarded-Path", "X-Forwarded-Client-Cert",
+		"X_Forwarded_Ssl", "X_Forwarded_Proto", "X_Real_IP",
 	}
 
 	standIn := startWAF(t, allow)
@@ -582,6 +585,9 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
 		req.Header.Set(name, "claimed")
 	}
 
+	// A made-up name, sent in lower case.
+	req.Header["x-forwarded-made-up"] = []string{"claimed"}
+
 	wantGreeting(t, visit(t, req))
 
 	header := <-seen
@@ -596,7 +602,7 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
 		}
 	}
 
-	for _, name := range removed {
+	for _, name := range append(removed, "X-Forwarded-Made-Up") {
 		if got := header.Values(name); len(got) != 0 {
 			t.Errorf("the app saw %s %q, want none", name, got)
 		}
@@ -622,6 +628,8 @@ func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
 		"X-Forwarded-Host":   "public.example",
 		forwardedProtoHeader: "https",
 		"X-Real-Ip":          "198.51.100.1",
+		"X-Forwarded-Ssl":    "on",
+		"X_Forwarded_Ssl":    "on",
 	}
 
 	req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
@@ -1023,7 +1031,9 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
 		w.WriteHeader(http.StatusInternalServerError)
 		_, _ = w.Write(make([]byte, size))
 	})
-	addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
+	addr, _ := startRProxy(t, standIn.URL, nowhere, map[string]string{
+		timeoutSetting: "60000",
+	})
 
 	got := get(t, addr, "/")
 	if got.status != http.StatusInternalServerError || len(got.body) != size {
@@ -1117,6 +1127,7 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
 					t.Error("the app was asked")
 				})
 				addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
+					timeoutSetting:     "60000",
 					defaultDisposition: setting,
 				})
 
@@ -1140,7 +1151,9 @@ func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
 	t.Parallel()
 
 	standIn := startWAF(t, overOneMiB(dispositionDeny))
-	addr, out := startRProxy(t, standIn.URL, nowhere, nil)
+	addr, out := startRProxy(t, standIn.URL, nowhere, map[string]string{
+		timeoutSetting: "60000",
+	})
 
 	if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
 		t.Errorf("answered %+v, want 502", got)

internal/sww/healthcheck_test.go

@@ -41,7 +41,7 @@ func TestHealthCheckInWAFsContainer(t *testing.T) {
 		exited <- run(ctx, settings, out)
 	}()
 
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
+	addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
 	_, port, _ := net.SplitHostPort(addr)
 
 	// The container's settings: an address to listen on with an empty

internal/sww/rproxy_test.go

@@ -71,6 +71,8 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
 		}, out)
 	}()
 
+	addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
+
 	t.Cleanup(func() {
 		stop()
 
@@ -86,8 +88,6 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
 		out.line(t, "msg", "stopped")
 	})
 
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
-
 	return "http://" + addr, out
 }
 
@@ -105,7 +105,7 @@ func startWAF(t *testing.T, env map[string]string) (string, *output, func()) {
 		exited <- run(ctx, env, out)
 	}()
 
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
+	addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
 
 	return addr, out, func() {
 		stop()
@@ -507,9 +507,10 @@ func TestAStatusWAFDoesNotTakeIsReportedAs599(t *testing.T) {
 }
 
 // startRProxyProcess runs `sww rproxy` in a process of its own, with the
-// settings in env, each as NAME=value. It returns where rproxy listens, the
+// settings in env, each as NAME=value. It returns where rproxy listens and a
-// process, and what rproxy writes.
+// function that sends rproxy SIGTERM and fails the test unless it exits with
-func startRProxyProcess(t *testing.T, env ...string) (string, *exec.Cmd, *output) {
+// status 0 within waitLimit.
+func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
 	t.Helper()
 
 	binary, err := os.Executable()
@@ -517,11 +518,8 @@ func startRProxyProcess(t *testing.T, env ...string) (string, *exec.Cmd, *output
 		t.Fatalf("find the test binary: %v", err)
 	}
 
-	ctx, cancel := context.WithTimeout(t.Context(), 3*waitLimit)
-	t.Cleanup(cancel)
-
 	out := &output{}
-	cmd := exec.CommandContext(ctx, binary) //nolint:gosec // this test binary
+	cmd := exec.CommandContext(t.Context(), binary) //nolint:gosec // this test binary
 	cmd.Stdout = out
 
 	cmd.Env = append(os.Environ(),
@@ -529,19 +527,25 @@ func startRProxyProcess(t *testing.T, env ...string) (string, *exec.Cmd, *output
 		"SWRP_LISTEN_ADDR="+localhost+":0")
 	cmd.Env = append(cmd.Env, env...)
 
-	err = cmd.Start()
+	exited := startProcess(t, cmd)
-	if err != nil {
+
-		t.Fatalf("start: %v", err)
+	addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
+
+	return addr, func() {
+		err := cmd.Process.Signal(syscall.SIGTERM)
+		if err != nil {
+			t.Fatalf("signal: %v", err)
+		}
+
+		select {
+		case status := <-exited:
+			if status != 0 {
+				t.Errorf("exit status %d, want 0; output:\n%s", status, out.text())
+			}
+		case <-time.After(waitLimit):
+			t.Error("still running after being told to stop")
+		}
 	}
-
-	t.Cleanup(func() {
-		_ = cmd.Process.Kill()
-		_ = cmd.Wait()
-	})
-
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
-
-	return addr, cmd, out
 }
 
 func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
@@ -572,7 +576,7 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
 		t.Fatalf("write the certificate: %v", err)
 	}
 
-	addr, _, _ := startRProxyProcess(t,
+	addr, _ := startRProxyProcess(t,
 		wafBackend+"="+backend.URL,
 		"SWRP_UPSTREAM_URL="+startApp(t),
 		"SSL_CERT_FILE="+certificates)
@@ -651,7 +655,7 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
 		exited <- runRProxy(ctx, env, out)
 	}()
 
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
+	addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
 
 	wantStatus(t, "http://"+addr+"/", placed, http.StatusOK)
 
@@ -750,7 +754,7 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
 		exited <- runRProxy(ctx, env, out)
 	}()
 
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
+	addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
 	conn := sendAndKeepOpen(t, addr, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
 
 	select {
@@ -845,7 +849,7 @@ func TestMainRunsRProxy(t *testing.T) {
 	t.Parallel()
 
 	// Nothing listens there; the health check does not ask.
-	addr, cmd, out := startRProxyProcess(t, wafBackend+"=http://127.0.0.1:9")
+	addr, stop := startRProxyProcess(t, wafBackend+"=http://127.0.0.1:9")
 
 	req, err := http.NewRequestWithContext(t.Context(), http.MethodGet,
 		"http://"+addr+waf.HealthPath, http.NoBody)
@@ -868,13 +872,5 @@ func TestMainRunsRProxy(t *testing.T) {
 		t.Errorf("health check answered %q (%v)", body, err)
 	}
 
-	err = cmd.Process.Signal(syscall.SIGTERM)
+	stop()
-	if err != nil {
-		t.Fatalf("signal: %v", err)
-	}
-
-	err = cmd.Wait()
-	if err != nil {
-		t.Errorf("exit: %v; output:\n%s", err, out.text())
-	}
 }

internal/sww/sww.go

@@ -1,9 +1,9 @@
 // Package sww runs the sww process, as the command it is given.
 // `sww waf` reads the settings, the rule files, the lookup database
-// and the state files, answers rproxy's questions until it is told to stop, and
+// and the state and cache files, answers rproxy's questions until it is told to
-// then stops in an orderly way, writing the state files. `sww rproxy`
+// stop, and then stops in an orderly way, writing the state and cache files.
-// reads rproxy's settings alone, and runs the reverse proxy in front of one app
+// `sww rproxy` reads rproxy's settings alone, and runs the reverse proxy in
-// that asks waf about each request. `sww healthcheck` is the
+// front of one app that asks waf about each request. `sww healthcheck` is the
 // container's health check.
 package sww
 
@@ -101,8 +101,8 @@ func printUsage(w io.Writer) int {
 }
 
 // RunWAF is `sww waf`: it reads the settings, the rule files,
-// the lookup database and the state files, then serves requests until ctx is
+// the lookup database and the state and cache files, then serves requests until
-// done. It returns the process's exit status, 1 when waf cannot start.
+// ctx is done. It returns the process's exit status, 1 when waf cannot start.
 func RunWAF(ctx context.Context, params Params) int {
 	// Until the settings are read, the one message is an invalid setting's
 	// error, which every SWW_LOG_LEVEL lets through.
@@ -171,7 +171,7 @@ func RunWAF(ctx context.Context, params Params) int {
 
 	files, err := loadStateFiles(cfg, server, alertQueue, now, processLog)
 	if err != nil {
-		processLog.Error("cannot use the state files", "error", err.Error())
+		processLog.Error("cannot use the state and cache files", "error", err.Error())
 
 		return 1
 	}
@@ -236,14 +236,15 @@ func newServer(
 	return server, nil
 }
 
-// loadStateFiles reads the state files into the parts of server and into
+// loadStateFiles reads the state and cache files into the parts of server and
-// alertQueue, as state.Load does.
+// into alertQueue, as state.Load does.
 func loadStateFiles(
 	cfg *config.Config, server *waf.Server, alertQueue *alerts.Queue,
 	now func() time.Time, processLog *slog.Logger,
 ) (*state.Files, error) {
 	return state.Load(state.Params{
 		Dir:             cfg.StateDir,
+		CacheDir:        cfg.CacheDir,
 		WriteDelay:      cfg.StateWriteDelay,
 		CounterInterval: cfg.StateCounterInterval,
 		Ledger:          server.Ledger,
@@ -252,6 +253,7 @@ func loadStateFiles(
 		Lists:           server.Lists,
 		DNSBL:           server.DNSBL,
 		AbuseIPDB:       server.AbuseIPDB,
+		RouteWatch:      server.RouteWatch,
 		Alerts:          alertQueue,
 		Anomalies:       server.Anomalies,
 		Now:             now,
@@ -317,13 +319,13 @@ func startSending(
 	}
 }
 
-// serve serves requests on listener, writes the state files as they are
+// serve serves requests on listener, writes the state and cache files as they
-// due, takes in an admin's edits of them, reads the rule files again as
+// are due, takes in an admin's edits of them, reads the rule files again as
 // they change, and the lookup database when it is replaced, fetches the
 // lists the settings name by URL as they are due, and sends the alerts,
 // until ctx is done. Then it gives the requests in progress
-// shutdownTimeout to finish, and writes every state file, alerts.json with
+// shutdownTimeout to finish, and writes every state and cache file,
-// the alerts still waiting.
+// alerts.json with the alerts still waiting.
 func serve(
 	ctx context.Context, server *waf.Server, listener net.Listener,
 	files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue,
@@ -393,7 +395,7 @@ func serve(
 
 	err = files.WriteAll()
 	if err != nil {
-		processLog.Error("writing the state files failed", "error", err.Error())
+		processLog.Error("writing the state and cache files failed", "error", err.Error())
 
 		return 1
 	}

internal/sww/sww_test.go

@@ -38,6 +38,7 @@ const (
 	stateDir             = "SWW_STATE_DIR"
 	stateWriteDelay      = "SWW_STATE_WRITE_DELAY"
 	stateCounterInterval = "SWW_STATE_COUNTER_INTERVAL"
+	cacheDir             = "SWW_CACHE_DIR"
 	rateLimitPerDay      = "SWW_RATE_LIMIT_PER_DAY"
 	rateLimitExemptNets  = "SWW_RATE_LIMIT_EXEMPT_NETS"
 	rulesDir             = "SWW_RULES_DIR"
@@ -92,6 +93,31 @@ func (o *output) line(t *testing.T, key, value string) map[string]any {
 	return nil
 }
 
+// lineBeforeExit returns the first line whose msg is msg, waiting for it
+// while sww runs. It waits as long as that takes, so that a slow start
+// cannot fail the test, and fails the test if sww exits first.
+func (o *output) lineBeforeExit(
+	t *testing.T, msg string, exited <-chan int,
+) map[string]any {
+	t.Helper()
+
+	for {
+		fields := o.findLine(t, func(fields map[string]any) bool {
+			return fields["msg"] == msg
+		})
+		if fields != nil {
+			return fields
+		}
+
+		select {
+		case status := <-exited:
+			t.Fatalf("exit status %d while waiting for a line with msg %q; output:\n%s",
+				status, msg, o.text())
+		case <-time.After(pollInterval):
+		}
+	}
+}
+
 // findLine returns the first line written so far for which matches is true, or
 // nil.
 func (o *output) findLine(
@@ -135,6 +161,12 @@ func run(ctx context.Context, env map[string]string, out *output) int {
 				return "off", true
 			}
 
+			// The cache files go beside the state files unless the test
+			// says otherwise.
+			if !ok && name == cacheDir {
+				return env[stateDir], env[stateDir] != ""
+			}
+
 			return value, ok
 		},
 		Stdout: out,
@@ -236,11 +268,12 @@ func TestServesUntilToldToStop(t *testing.T) {
 	}()
 
 	// The default rule file is read.
-	if rules := out.line(t, "msg", "read the rule files")["rules"]; rules != 12.0 {
+	readRules := out.lineBeforeExit(t, "read the rule files", exited)
+	if rules := readRules["rules"]; rules != 12.0 {
 		t.Errorf("read %v rules from the default rule file, want 12", rules)
 	}
 
-	starting := out.line(t, "msg", "starting")
+	starting := out.lineBeforeExit(t, "starting", exited)
 	wantStartingLine(t, starting, dir)
 
 	addr, _ := starting["address"].(string)
@@ -286,7 +319,7 @@ func TestLogLevelHoldsBackTheLessSevereProcessLines(t *testing.T) {
 		}, out)
 	}()
 
-	out.line(t, "msg", "fetching a list failed")
+	out.lineBeforeExit(t, "fetching a list failed", exited)
 	stop()
 
 	select {
@@ -673,7 +706,7 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
 	}
 }
 
-func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T) {
+func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
 	t.Parallel()
 
 	const (
@@ -706,10 +739,11 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
 	t.Cleanup(server.Close)
 
 	torURL, dropURL := server.URL+torPath, server.URL+dropPath
-	dir := t.TempDir()
+	cache := t.TempDir()
 	env := map[string]string{
 		listenAddr:           localhost + ":0",
-		stateDir:             dir,
+		stateDir:             t.TempDir(),
+		cacheDir:             cache,
 		rulesDir:             t.TempDir(),
 		metricsToken:         token,
 		instanceName:         instance,
@@ -729,7 +763,7 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
 	})
 
 	// Restarted with drop.txt named after it and the server answering, tor.txt
-	// waits SWW_BLOCKLIST_REFRESH after its failed try, kept in reputation.json,
+	// waits SWW_BLOCKLIST_REFRESH after its failed try, kept in lists.json,
 	// while drop.txt, never tried, is fetched at once. Lists are fetched in the
 	// order named, so once drop.txt refuses the client, tor.txt has had its turn.
 	failing.Store(false)
@@ -747,8 +781,8 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
 	}
 
 	// After another restart, with the server failing, the copy of drop.txt
-	// kept in reputation.json, its copyright line included, refuses the
+	// kept in lists.json, its copyright line included, refuses the client
-	// client from the first request.
+	// from the first request.
 	failing.Store(true)
 
 	out = runUntilStopped(t, env, func(url string) {
@@ -756,16 +790,16 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
 	})
 	wantDeniedByList(t, out.line(t, "type", "request"), dropURL)
 
-	path := filepath.Join(dir, "reputation.json")
+	path := filepath.Join(cache, "lists.json")
 
 	kept, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
 	if err != nil || !strings.Contains(string(kept), `"`+copyright+`"`) {
-		t.Errorf("reputation.json holds\n%s\nwant the copy with %q (%v)", kept, copyright,
+		t.Errorf("lists.json holds\n%s\nwant the copy with %q (%v)", kept, copyright,
 			err)
 	}
 }
 
-func TestCrowdSecDecisionListKeptInReputationJSONAcrossARestart(t *testing.T) {
+func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
 	t.Parallel()
 
 	const bouncerKey = "crowdsec-key-0123456789abcdef"
@@ -788,10 +822,11 @@ func TestCrowdSecDecisionListKeptInReputationJSONAcrossARestart(t *testing.T) {
 		}))
 	t.Cleanup(engine.Close)
 
-	dir := t.TempDir()
+	dir, cache := t.TempDir(), t.TempDir()
 	env := map[string]string{
 		listenAddr:              localhost + ":0",
 		stateDir:                dir,
+		cacheDir:                cache,
 		rulesDir:                t.TempDir(),
 		"SWW_CROWDSEC_LAPI_URL": engine.URL,
 		"SWW_CROWDSEC_LAPI_KEY": bouncerKey,
@@ -810,7 +845,7 @@ func TestCrowdSecDecisionListKeptInReputationJSONAcrossARestart(t *testing.T) {
 		t.Errorf("bans.json holds %v, want the ban for crowdsec on %s", ban, placed)
 	}
 
-	// Restarted with the engine down, the copy kept in reputation.json bans
+	// Restarted with the engine down, the copy kept in lists.json bans
 	// another client in the netblock from its first request.
 	down.Store(true)
 
@@ -818,22 +853,25 @@ func TestCrowdSecDecisionListKeptInReputationJSONAcrossARestart(t *testing.T) {
 		wantStatus(t, url, "203.0.113.10", http.StatusForbidden)
 	})
 
-	// The key is in neither run's output, nor in a state file.
+	// The key is in neither run's output, nor in a state or cache file.
-	files := []string{"bans.json", "reputation.json", "clients.json"}
+	files := []string{
+		filepath.Join(dir, "bans.json"), filepath.Join(dir, "clients.json"),
+		filepath.Join(cache, "lists.json"),
+	}
 	shown := make([]string, 0, len(files)+2)
 	shown = append(shown, first.text(), second.text())
 
-	for _, name := range files {
+	for _, path := range files {
-		data, err := os.ReadFile(filepath.Join(dir, name)) //nolint:gosec // the test's
+		data, err := os.ReadFile(path) //nolint:gosec // the test's
 		if err != nil {
-			t.Fatalf("read %s: %v", name, err)
+			t.Fatalf("read %s: %v", path, err)
 		}
 
 		shown = append(shown, string(data))
 	}
 
 	if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) {
-		t.Errorf("the key is shown in the output or the state files:\n%s", all)
+		t.Errorf("the key is shown in the output or the state and cache files:\n%s", all)
 	}
 }
 
@@ -1181,6 +1219,29 @@ func TestUnwritableStateDirStopsTheStart(t *testing.T) {
 		"SWW_STATE_DIR cannot be written: ")
 }
 
+func TestUnwritableCacheDirStopsTheStart(t *testing.T) {
+	t.Parallel()
+
+	ctx, stop := context.WithTimeout(t.Context(), waitLimit)
+	defer stop()
+
+	out := &output{}
+
+	status := run(ctx, map[string]string{
+		listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
+		cacheDir: filepath.Join(t.TempDir(), "missing"),
+	}, out)
+	if status != 1 {
+		t.Fatalf("exit status %d, want 1", status)
+	}
+
+	line := out.line(t, "msg", "cannot use the state and cache files")
+	if message, _ := line["error"].(string); !strings.HasPrefix(message,
+		"SWW_CACHE_DIR cannot be written: ") {
+		t.Errorf("start refused with %q, want SWW_CACHE_DIR named", message)
+	}
+}
+
 // wantStartRefused runs sww with its state files in dir, and
 // checks that it stops at start, with an error that starts with want. If
 // it starts instead, it is stopped after waitLimit.
@@ -1199,7 +1260,7 @@ func wantStartRefused(t *testing.T, dir, want string) {
 		t.Fatalf("exit status %d, want 1", status)
 	}
 
-	line := out.line(t, "msg", "cannot use the state files")
+	line := out.line(t, "msg", "cannot use the state and cache files")
 	message, _ := line["error"].(string)
 
 	if !strings.HasPrefix(message, want) {
@@ -1261,7 +1322,7 @@ func runUntilStopped(
 		exited <- run(ctx, env, out)
 	}()
 
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
+	addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
 	use("http://" + addr + "/")
 	stop()
 

internal/sww/waf_test.go

@@ -9,10 +9,14 @@ import (
 	"os"
 	"os/exec"
 	"path/filepath"
+	"regexp"
 	"strings"
 	"syscall"
 	"testing"
 	"time"
+	// The time zone database, so that the process TestMainRunsWAF starts
+	// finds the zone TZ names on any host.
+	_ "time/tzdata"
 
 	"sneak.berlin/go/smallwebwaf/internal/sww"
 	"sneak.berlin/go/smallwebwaf/internal/waf"
@@ -34,49 +38,116 @@ func TestMain(m *testing.M) {
 	os.Exit(sww.Main(testVersion))
 }
 
+// startProcess starts cmd and returns a channel that gets its exit status. As
+// the test ends, cmd is killed and the test waits for it to exit.
+func startProcess(t *testing.T, cmd *exec.Cmd) <-chan int {
+	t.Helper()
+
+	err := cmd.Start()
+	if err != nil {
+		t.Fatalf("start: %v", err)
+	}
+
+	exited := make(chan int, 1)
+	waitReturned := make(chan struct{})
+
+	go func() {
+		_ = cmd.Wait()
+		exited <- cmd.ProcessState.ExitCode()
+
+		close(waitReturned)
+	}()
+
+	t.Cleanup(func() {
+		_ = cmd.Process.Kill()
+
+		<-waitReturned
+	})
+
+	return exited
+}
+
 func TestMainRunsWAF(t *testing.T) {
 	t.Parallel()
 
+	// Kiritimati is 14 hours ahead of UTC; every time sww gives stays in UTC.
+	const zone = "Pacific/Kiritimati"
+
+	_, err := time.LoadLocation(zone)
+	if err != nil {
+		t.Fatalf("load %s: %v", zone, err)
+	}
+
 	binary, err := os.Executable()
 	if err != nil {
 		t.Fatalf("find the test binary: %v", err)
 	}
 
-	ctx, cancel := context.WithTimeout(t.Context(), 3*waitLimit)
+	dir := t.TempDir()
-	defer cancel()
-
 	out := &output{}
-	cmd := exec.CommandContext(ctx, binary) //nolint:gosec // this test binary
+	cmd := exec.CommandContext(t.Context(), binary) //nolint:gosec // this test binary
 	cmd.Stdout = out
 
 	cmd.Env = append(os.Environ(),
+		"TZ="+zone,
 		commandForMain+"="+wafCommand,
 		listenAddr+"="+localhost+":0",
-		stateDir+"="+t.TempDir(),
+		stateDir+"="+dir,
+		cacheDir+"="+dir,
 		rulesDir+"="+t.TempDir(),
 		lookupSource+"=off")
 
-	err = cmd.Start()
+	exited := startProcess(t, cmd)
-	if err != nil {
-		t.Fatalf("start: %v", err)
-	}
 
-	t.Cleanup(func() {
+	starting := out.lineBeforeExit(t, "starting", exited)
-		_ = cmd.Process.Kill()
+	addr, _ := starting["address"].(string)
-		_ = cmd.Wait()
-	})
-
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
 	ask(t, addr, "/", http.StatusOK, waf.DispositionAllow)
 
+	// The files are written as sww stops.
 	err = cmd.Process.Signal(syscall.SIGTERM)
 	if err != nil {
 		t.Fatalf("signal: %v", err)
 	}
 
-	err = cmd.Wait()
+	select {
+	case status := <-exited:
+		if status != 0 {
+			t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
+		}
+	case <-time.After(waitLimit):
+		t.Fatal("still running after being told to stop")
+	}
+
+	wantUTC(t, "the process line", starting["time"])
+	wantUTC(t, "the request line", out.line(t, "type", "request")["time"])
+
+	path := filepath.Join(dir, "clients.json")
+
+	clients, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
 	if err != nil {
-		t.Errorf("exit: %v; output:\n%s", err, out.text())
+		t.Fatalf("read clients.json: %v", err)
+	}
+
+	stamp := regexp.MustCompile(`"\d{4}-\d\d-\d\dT[^"]*"`)
+
+	times := stamp.FindAllString(string(clients), -1)
+	if len(times) == 0 {
+		t.Errorf("clients.json holds no time:\n%s", clients)
+	}
+
+	for _, stored := range times {
+		wantUTC(t, "clients.json", strings.Trim(stored, `"`))
+	}
+}
+
+// wantUTC checks that value, a time read from the log line or file that
+// where names, is in UTC.
+func wantUTC(t *testing.T, where string, value any) {
+	t.Helper()
+
+	text, _ := value.(string)
+	if !strings.HasSuffix(text, "Z") {
+		t.Errorf("%s gives the time %v, want one in UTC", where, value)
 	}
 }
 
@@ -167,7 +238,7 @@ func runWAF(t *testing.T, env map[string]string, use func(addr string)) {
 		exited <- run(ctx, env, out)
 	}()
 
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
+	addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
 	use(addr)
 	stop()
 

internal/waf/answertime_test.go

@@ -1,21 +1,27 @@
 package waf_test
 
 import (
+	"encoding/binary"
 	"fmt"
 	"maps"
 	"net"
 	"net/http"
 	"net/http/httptest"
 	"net/netip"
+	"path/filepath"
 	"slices"
 	"strconv"
 	"strings"
+	"sync"
 	"testing"
 	"testing/synctest"
 	"time"
 
+	"github.com/prometheus/client_golang/prometheus/testutil"
+
 	"sneak.berlin/go/smallwebwaf/internal/bans"
 	"sneak.berlin/go/smallwebwaf/internal/lookup"
+	"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
 	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
 	"sneak.berlin/go/smallwebwaf/internal/reputation"
 	"sneak.berlin/go/smallwebwaf/internal/requestlog"
@@ -28,6 +34,8 @@ const (
 	answerTimeLimit = 2 * time.Second
 	// defaultLookupWait is the default of SWW_LOOKUP_TIMEOUT.
 	defaultLookupWait = 800 * time.Millisecond
+	// defaultReputationWait is the default of SWW_REPUTATION_TIMEOUT.
+	defaultReputationWait = 2 * time.Second
 	// The header limits SPEC.md gives.
 	cookieLimit      = 50
 	headerFieldLimit = 100
@@ -216,93 +224,199 @@ func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
 	}
 }
 
-func TestRequestsOverTheHeaderLimitsCountTowardTheRateLimits(t *testing.T) {
+func TestTheRateLimitsDoNotCountRequestsOverTheHeaderLimits(t *testing.T) {
 	t.Parallel()
 
 	s := startWAF(t, "", map[string]string{rateLimitPerMinute: "1"})
-	ask := func() answer {
+
+	for range 2 {
 		req := s.question(siteA, client, "/")
 		req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
+		wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
-		return do(t, req)
 	}
 
-	wantDisposition(t, ask(), tooLarge, waf.DispositionDeny)
+	wantDisposition(t, s.ask(client, "/"), http.StatusOK, waf.DispositionAllow)
-	wantDisposition(t, ask(), http.StatusForbidden, waf.DispositionDeny)
-
-	lines := s.out.requestLines(t, 2)
-	wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionClientIsRateLimited)
 }
 
-func TestTheHeaderLimitsComeAfterTheTrapPathsAndTheRuleFiles(t *testing.T) {
+func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
 	t.Parallel()
 
+	// The lookup database places client in Germany.
+	lookupDB := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
+	lookuptest.Write(t, lookupDB, map[string]lookuptest.Network{
+		client + "/32": {ASN: asnDE, ASName: asNameDE, Country: "DE"},
+	})
+
 	for _, tc := range []struct {
-		name, path, action string
+		name string
+		env  map[string]string
+		path string
+		// prepare gets the server ready, at now, before the question.
+		prepare func(t *testing.T, server *waf.Server, now time.Time)
 	}{
-		{"a trap path", "/wp-login.php", requestlog.ActionTemporarilyBanned},
+		{"SWW_DENY_NETS", map[string]string{denyNets: client}, "/", leaveAsItIs},
-		{"a block rule", "/blocked", requestlog.ActionBlockedByBlockRule},
+		{"a ban", nil, "/", banClient},
+		{
+			"the country list", map[string]string{
+				lookupSource: fileSource, lookupDBPath: lookupDB, allowedCountries: "FR",
+			},
+			"/", leaveAsItIs,
+		},
+		{"a blocklist", map[string]string{blocklistURLs: dropURL}, "/", listClient},
+		{
+			"the CrowdSec decision list",
+			map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey}, "/", decideOnClient,
+		},
+		{
+			"a DNS blocklist", map[string]string{
+				dnsblZones: dnsblZone, dnsblResolver: noResolver, reputationAction: actionDeny,
+			},
+			"/", func(_ *testing.T, server *waf.Server, _ time.Time) {
+				loadVerdicts(server, map[string][]string{client: {dnsblZone}})
+			},
+		},
+		{
+			"AbuseIPDB",
+			map[string]string{abuseIPDBKey: accountKey, reputationAction: actionDeny},
+			"/", func(_ *testing.T, server *waf.Server, _ time.Time) {
+				// Over SWW_ABUSEIPDB_MIN_SCORE, 75 by default.
+				loadScores(server, map[string]int64{client: 80})
+			},
+		},
+		{
+			"SWW_TAG_DENY", map[string]string{tagLists: "vpn=" + vpnList, tagDeny: "vpn"},
+			"/", func(t *testing.T, server *waf.Server, _ time.Time) {
+				t.Helper()
+				loadLists(t, server, map[string][]string{vpnList: {client}})
+			},
+		},
+		{"the rate limits", map[string]string{rateLimitPerMinute: "1"}, "/", countClient},
+		{
+			"a trap path", map[string]string{trapPaths: trapPathList}, "/wp-login.php",
+			leaveAsItIs,
+		},
+		{
+			"a block rule", map[string]string{rulesDir: writeRules(t, testRules)}, "/blocked",
+			leaveAsItIs,
+		},
 	} {
 		t.Run(tc.name, func(t *testing.T) {
 			t.Parallel()
 
-			s := startWAF(t, "", map[string]string{
+			s := startWAF(t, "", tc.env)
-				trapPaths: trapPathList, rulesDir: writeRules(t, testRules),
+			tc.prepare(t, s.server, s.clock.Now())
-			})
+
 			req := s.question(siteA, client, tc.path)
 			req.Header.Set(cookieHeader, cookies(cookieLimit+1))
 
-			wantDisposition(t, do(t, req), http.StatusForbidden, waf.DispositionDeny)
+			wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
-			wantLine(t, s.out.requestLine(t), http.StatusForbidden, tc.action)
+			wantLine(t, s.out.requestLine(t), tooLarge, requestlog.ActionRequestHeaderTooLarge)
 		})
 	}
 }
 
+func TestARequestOverTheHeaderLimitsRunsNoOtherCheck(t *testing.T) {
+	t.Parallel()
+
+	// Each check after the header limits leaves a mark on the log line of a
+	// request it reads: a tag, routewatch_tags_missing, a rule id, the
+	// counts, or the Core Rule Set's time.
+	path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
+	lookuptest.Write(t, path, map[string]lookuptest.Network{
+		client + "/32": {ASN: asnDE, ASName: asNameDE, Country: "DE"},
+	})
+	s := startWAF(t, "", map[string]string{
+		lookupSource: fileSource, lookupDBPath: path,
+		tagLists:      "vpn=" + vpnList,
+		routeWatchURL: startRouteWatch(t, map[string]standInRoute{client: {prefixDE, 64496}}),
+		dnsblZones:    dnsblZone, reputationAction: actionLog,
+		rulesDir: writeRules(t, "noted path log ^/\n"),
+		wafMode:  block,
+	})
+	loadLists(t, s.server, map[string][]string{vpnList: {client}})
+	loadVerdicts(s.server, map[string][]string{client: {dnsblZone}})
+
+	over := s.question(siteA, client, "/")
+	over.Header.Set(cookieHeader, cookies(cookieLimit+1))
+	wantDisposition(t, do(t, over), tooLarge, waf.DispositionDeny)
+
+	within := s.question(siteA, client, "/")
+	within.Header.Set(cookieHeader, cookies(cookieLimit))
+	wantDisposition(t, do(t, within), http.StatusOK, waf.DispositionAllow)
+
+	lines := s.out.requestLines(t, 2)
+
+	wantTags(t, lines[0])
+	wantTagsMissing(t, lines[0], "")
+
+	if lines[0].RuleIDs != nil || lines[0].Counts != (ratelimit.Counts{}) ||
+		lines[0].DurationCoreRuleSetMS != nil {
+		t.Errorf("rule ids %v, counts %+v and Core Rule Set time %v, want none",
+			lines[0].RuleIDs, lines[0].Counts, lines[0].DurationCoreRuleSetMS)
+	}
+
+	// The request within the limits shows that each check leaves its mark.
+	wantTags(t, lines[1],
+		"asn=64496", "country=DE", "dnsbl="+dnsblZone, "exploitsignature", "vpn")
+	wantTagsMissing(t, lines[1], "no_answer_yet")
+
+	if lines[1].RuleIDs == nil || lines[1].Counts.Minute != 1 ||
+		lines[1].DurationCoreRuleSetMS == nil {
+		t.Errorf("rule ids %v, counts %+v and Core Rule Set time %v, want each",
+			lines[1].RuleIDs, lines[1].Counts, lines[1].DurationCoreRuleSetMS)
+	}
+}
+
 func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
 	t.Parallel()
 
+	// The client's next request takes it over the whole limit, so it does
+	// not wait.
 	geojsURL, _ := startGeoJS(t)
 	s := startWAF(t, geojsURL, map[string]string{
-		unknownLimitPercent: "0", lookupTimeout: "1h",
+		unknownLimitPercent: "0", lookupTimeout: "1h", rateLimitPerMinute: "1",
 	})
-	req := s.question(siteA, fromDE, "/")
+	countClient(t, s.server, s.clock.Now())
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
 
-	wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
+	wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, waf.DispositionDeny)
 
 	line := s.out.requestLine(t)
-	wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
+	wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
 
 	if line.RateLimitPercent != nil {
 		t.Errorf("limit percentage %d, want none", *line.RateLimitPercent)
 	}
-
-	// The client is under no ban, and GeoJS places it in a country.
-	wantDisposition(t, s.ask(fromDE, "/"), http.StatusOK, waf.DispositionAllow)
 }
 
 func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) {
 	t.Parallel()
 
+	// The client's next request takes it over the whole limit, so it does
+	// not wait.
 	geojsURL, _ := startGeoJS(t)
-	s := startWAF(t, geojsURL, map[string]string{unknownLimitPercent: "0"})
+	s := startWAF(t, geojsURL, map[string]string{
+		unknownLimitPercent: "0", rateLimitPerMinute: "1",
+	})
 	s.server.GeoJS.Load([]lookup.Answer{{
 		Client: netip.MustParsePrefix(client + "/32"), ASN: "AS64512",
 		Answered: s.clock.Now(), Used: s.clock.Now(),
 	}})
+	countClient(t, s.server, s.clock.Now())
 
-	req := s.question(siteA, client, "/")
+	wantDisposition(t, s.ask(client, "/"), http.StatusForbidden, waf.DispositionDeny)
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
 
-	wantDisposition(t, do(t, req), http.StatusForbidden, waf.DispositionDeny)
+	line := s.out.requestLine(t)
-	wantLine(t, s.out.requestLine(t), http.StatusForbidden,
+	wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
-		requestlog.ActionClientIsRateLimited)
+
+	if line.RateLimitPercent == nil || *line.RateLimitPercent != 0 {
+		t.Errorf("limit percentage %v, want 0", line.RateLimitPercent)
+	}
 }
 
-func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
+func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits(t *testing.T) {
 	t.Parallel()
 
-	geojsURL, _ := startGeoJS(t)
+	geojsURL, asked := startGeoJS(t)
 	s := startWAF(t, geojsURL, map[string]string{
 		mode: observe, addLookupHeaders: on, lookupTimeout: "1h",
 	})
@@ -312,10 +426,9 @@ func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
 	got := do(t, req)
 	wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
 
-	asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
+	if len(asked()) != 0 || got.header.Get("X-Client-Asn") != "" {
-	if asn != asnDE || country != "DE" {
+		t.Errorf("GeoJS was asked about %v, and the answer gave the AS number %q, "+
-		t.Errorf("answered with the AS number %q and the country %q, want %q and DE",
+			"want neither", asked(), got.header.Get("X-Client-Asn"))
-			asn, country, asnDE)
 	}
 }
 
@@ -521,15 +634,9 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
 ) {
 	t.Parallel()
 
-	held, askedGeoJS, _ := startHeldGeoJS(t)
+	// The lists' stand-in: one for GeoJS that answers nothing.
-
+	held, _, _ := startHeldGeoJS(t)
-	// A resolver that takes every query and answers none.
+	resolver, releaseResolver := startHeldResolver(t)
-	resolver, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", localhost+":0")
-	if err != nil {
-		t.Fatalf("listen: %v", err)
-	}
-
-	t.Cleanup(func() { _ = resolver.Close() })
 
 	server, out, _ := newWAFWith(t, func(
 		params waf.Params,
@@ -537,42 +644,117 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
 		params.AbuseIPDBURL = silentAbuseIPDBURL
 
 		return waf.New(params)
-	}, held, time.Now, map[string]string{
+	}, silentGeoJSURL, time.Now, map[string]string{
 		tagDeny: "country=KP", wafMode: block,
-		dnsblZones: dnsblZone, dnsblResolver: resolver.LocalAddr().String(),
+		dnsblZones: dnsblZone, dnsblResolver: resolver,
 		abuseIPDBKey:  accountKey,
 		blocklistURLs: held + "/drop.txt", asnLimitPercentURL: held + "/asn.txt",
 		crowdSecURL: held, crowdSecKey: bouncerKey,
 	})
 
-	// AbuseIPDB is asked only about a client with an offence.
+	// The lists are fetched outside the synctest bubble below: a wait on the
-	group := netip.MustParsePrefix(client + "/32")
+	// network inside it would keep its clock from moving on.
-	server.Limiter.AddToHistory(group, time.Now(), ratelimit.Request{Attack: true})
-
 	go server.Lists.Run(t.Context())
 
-	status, took := askDirectly(t, server, func(req *http.Request) {
+	// In the bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays says, the
-		req.Header.Set("User-Agent", "test")
+	// answer takes exactly as long as waf waits, however busy the host is.
-		req.Header.Set("Accept", "*/*")
+	// GeoJS and AbuseIPDB never answer there, without the network. The query to
-		req.Host = siteA
+	// the DNS blocklist waits on the network, which stops the bubble's clock,
+	// but it starts only after the wait for GeoJS.
+	synctest.Test(t, func(t *testing.T) {
+		// AbuseIPDB is asked only about a client with an offence.
+		group := netip.MustParsePrefix(client + "/32")
+		server.Limiter.AddToHistory(group, time.Now(), ratelimit.Request{Attack: true})
+
+		status, took := askDirectly(t, server, func(req *http.Request) {
+			req.Header.Set("User-Agent", "test")
+			req.Header.Set("Accept", "*/*")
+			req.Host = siteA
+		})
+
+		// The wait for GeoJS decides when the answer comes.
+		if status != http.StatusOK || took != defaultLookupWait || took >= answerTimeLimit {
+			t.Errorf("answered %d after %s, want %d after %s, within %s; output:\n%s",
+				status, took, http.StatusOK, defaultLookupWait, answerTimeLimit, out.text())
+		}
+
+		// Each outside source was asked, and none has answered. GeoJS is asked
+		// only about the clients waiting for it, here client alone.
+		if requests := testutil.ToFloat64(server.Metrics.GeoJSRequests); requests != 1 {
+			t.Errorf("%v requests to GeoJS, want 1", requests)
+		}
+
+		// The resolver holds the query until it is released below, so a waf
+		// that waited for the query would answer only once it had failed.
+		queries, failures := server.DNSBL.Queries(dnsblZone), server.DNSBL.Failures(dnsblZone)
+		if queries != 1 || failures != 0 {
+			t.Errorf("%d queries to the DNS blocklist, %d failed, want 1, none failed",
+				queries, failures)
+		}
+
+		if checked := server.AbuseIPDB.Checked(); checked != 1 {
+			t.Errorf("%d checks with AbuseIPDB, want 1", checked)
+		}
+
+		// The query is answered, and the check with AbuseIPDB abandoned,
+		// before the bubble ends.
+		releaseResolver()
+		time.Sleep(defaultReputationWait)
 	})
+}
 
-	if status != http.StatusOK || took >= answerTimeLimit {
+// How the stand-in for the resolver reads a query and makes it its reply.
-		t.Errorf("answered %d after %s, want %d within %s; output:\n%s",
+const (
-			status, took, http.StatusOK, answerTimeLimit, out.text())
+	// maxQueryLength is the longest query it reads.
+	maxQueryLength = 512
+	// flagsAt is where a DNS message's flags start, after its id.
+	flagsAt = 2
+	// noSuchNameFlags mark a reply, to a query that asked for recursion,
+	// which is available, that no such name exists.
+	noSuchNameFlags = 0x8183
+)
+
+// startHeldResolver starts a stand-in for the resolver the DNSBL zones are
+// asked through, and returns its address and release. Each query waits until
+// release is called, and is then answered that no such name exists, so that
+// the zone does not list the client.
+func startHeldResolver(t *testing.T) (string, func()) {
+	t.Helper()
+
+	conn, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", localhost+":0")
+	if err != nil {
+		t.Fatalf("listen: %v", err)
 	}
+
+	t.Cleanup(func() { _ = conn.Close() })
+
+	var (
+		released = make(chan struct{})
+		once     sync.Once
+	)
+
+	go func() {
+		query := make([]byte, maxQueryLength)
+
+		for {
+			n, from, err := conn.ReadFrom(query)
+			if err != nil {
+				return // conn is closed
+			}
 
-	// Each outside source was asked, and none has answered. The stand-in for
+			<-released
-	// GeoJS is the lists' too, which name no address.
+
-	if !slices.Contains(askedGeoJS(), client) {
+			// The query back, with the flags of the reply in place of its own.
-		t.Errorf("GeoJS was asked about %q, not %s", askedGeoJS(), client)
+			reply := slices.Clone(query[:n])
-	}
+			binary.BigEndian.PutUint16(reply[flagsAt:], noSuchNameFlags)
+
+			_, _ = conn.WriteTo(reply, from)
+		}
+	}()
 
-	if queries := server.DNSBL.Queries(dnsblZone); queries != 1 {
+	release := func() { once.Do(func() { close(released) }) }
-		t.Errorf("%d queries to the DNS blocklist, want 1", queries)
+	// Run before conn is closed, so that no query is left waiting.
-	}
+	t.Cleanup(release)
 
-	if checked := server.AbuseIPDB.Checked(); checked != 1 {
+	return conn.LocalAddr().String(), release
-		t.Errorf("%d checks with AbuseIPDB, want 1", checked)
-	}
 }

internal/waf/crowdsec_test.go

@@ -16,7 +16,7 @@ import (
 
 // The CrowdSec settings, and the tests' engine, which is never asked: each
 // test puts in the copy of its decision list, at decisionsURL, that it
-// needs, as reputation.json would at start.
+// needs, as lists.json would at start.
 const (
 	crowdSecURL  = "SWW_CROWDSEC_LAPI_URL"
 	crowdSecKey  = "SWW_CROWDSEC_LAPI_KEY"
@@ -182,7 +182,7 @@ func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
 
 // loadDecisions puts into server's lists the copy of the decision list at
 // decisionsURL, answer, the engine's answer, fetched at fetched, as
-// reputation.json would at start.
+// lists.json would at start.
 func loadDecisions(
 	t *testing.T, server *waf.Server, fetched time.Time, answer string,
 ) {

internal/waf/headerlimits.go

@@ -10,22 +10,23 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/requestlog"
 )
 
-// The header limits: the most cookies and header fields a request may have
+// The header limits: the most cookies and header fields a request may have.
-// when the checks reach the Core Rule Set. Real clients never reach them.
+// Real clients never reach them.
 const (
 	maxCookies      = 50
 	maxHeaderFields = 100
 )
 
-// headerLimits refuses a request over the header limits, with 431. Its figures
+// headerLimits refuses a request with more than maxHeaderFields header fields
-// are the request's header fields and cookies, and the limit of each.
+// or more than maxCookies cookies, with 431. Its figures are the request's
+// header fields and cookies, and the limit of each.
 type headerLimits struct{}
 
 func (headerLimits) name() string { return "header_limits" }
 
 func (headerLimits) check(_ context.Context, rq *request, _ time.Time) *verdict {
 	fields, cookies := rq.headerCounts()
-	if !overHeaderLimits(fields, cookies) {
+	if fields <= maxHeaderFields && cookies <= maxCookies {
 		return nil
 	}
 
@@ -38,13 +39,6 @@ func (headerLimits) check(_ context.Context, rq *request, _ time.Time) *verdict
 	}
 }
 
-// overHeaderLimits reports whether a request with these counts of header fields
-// and cookies, as headerCounts counts them, is over the header limits: it has
-// more than maxHeaderFields fields or more than maxCookies cookies.
-func overHeaderLimits(fields, cookies int) bool {
-	return fields > maxHeaderFields || cookies > maxCookies
-}
-
 // headerCounts returns the request's header fields, each value of a header sent
 // twice being one, and its cookies, each part of a Cookie header between
 // semicolons being one unless it is empty or only spaces and tabs. The Core

internal/waf/heuristics.go

@@ -41,8 +41,11 @@ const (
 
 // beforeLookup are the heuristics that run before the client's AS number and
 // country are looked up, so that a client they refuse is not looked up.
+// header_limits comes first: it only counts, and a request it refuses causes no
+// other work.
 func beforeLookup() []heuristic {
 	return []heuristic{
+		headerLimits{},
 		tagLists{},
 		routeWatchTags{},
 		denyNets{},
@@ -64,7 +67,6 @@ func afterLookup() []heuristic {
 		rateLimits{},
 		trapPaths{},
 		ruleFiles{},
-		headerLimits{},
 		coreRuleSet{},
 		tagDeny{},
 	}

internal/waf/proxy.go

@@ -86,8 +86,8 @@ type Params struct {
 // Server is the server waf runs, with the parts whose state the state
 // files keep, the lookup database, nil unless SWW_LOOKUP_SOURCE is file, the
 // lists fetched from URLs, which its Run fetches, the DNSBL zones' verdicts,
-// AbuseIPDB's scores and checks spent, and the metrics. RouteWatch, nil while
+// AbuseIPDB's scores and checks spent, routewatch's answers, and the metrics.
-// SWW_ROUTEWATCH_URL is unset, keeps its answers in memory only.
+// RouteWatch is nil while SWW_ROUTEWATCH_URL is unset.
 type Server struct {
 	*http.Server
 

internal/waf/reputation_test.go

@@ -39,7 +39,7 @@ const (
 )
 
 // The lists these tests name, which are never fetched: each test puts in
-// the copies it needs, as reputation.json would at start.
+// the copies it needs, as lists.json would at start.
 const (
 	dropURL = "https://lists.example/drop.txt"
 	torURL  = "https://lists.example/tor.txt"
@@ -374,7 +374,7 @@ const (
 
 // The DNSBL zones these tests name, which are never asked about the
 // clients the tests send requests from: each test puts in the verdicts it
-// needs, as reputation.json would at start. A query a test does start is
+// needs, as dnsbl.json would at start. A query a test does start is
 // sent to noResolver, where nothing listens, so that none leaves the host.
 const (
 	dnsblZone  = "dnsbl.example"
@@ -759,7 +759,7 @@ func TestOnlyAClientThatHasCommittedAnOffenceIsCheckedWithAbuseIPDB(t *testing.T
 	clk.advance(time.Hour)
 	wantReputation(t, s.get(fromDE, http.StatusOK, forward))
 	wantAbuseIPDBChecks(t, server, 1)
-	waitUntil(func() bool { return len(server.AbuseIPDB.Snapshot().Scores) == 1 })
+	waitUntil(func() bool { return len(server.AbuseIPDB.Snapshot()) == 1 })
 	wantReputation(t, s.get(fromDE, http.StatusOK, forward), abuseipdb)
 
 	s.get(fromKP, http.StatusOK, forward)
@@ -1072,7 +1072,7 @@ func verdictsFetched() time.Time {
 
 // loadVerdicts puts into server's DNSBL, for each client listedBy names,
 // a verdict of each zone SWW_DNSBL_ZONES names, fetched at
-// verdictsFetched, as reputation.json would at start: one that lists the
+// verdictsFetched, as dnsbl.json would at start: one that lists the
 // client from each zone listedBy gives for it, and one that does not from
 // each other zone.
 func loadVerdicts(server *waf.Server, listedBy map[string][]string) {
@@ -1091,8 +1091,8 @@ func loadVerdicts(server *waf.Server, listedBy map[string][]string) {
 }
 
 // loadScores puts into server's AbuseIPDB the score scores gives each
-// client, an IPv4 address, fetched at verdictsFetched, as reputation.json
+// client, an IPv4 address, fetched at verdictsFetched, as
-// would at start.
+// abuseipdb-scores.json would at start.
 func loadScores(server *waf.Server, scores map[string]int64) {
 	kept := make([]reputation.Score, 0, len(scores))
 	for client, score := range scores {
@@ -1102,7 +1102,7 @@ func loadScores(server *waf.Server, scores map[string]int64) {
 		})
 	}
 
-	server.AbuseIPDB.Load(reputation.Checks{Scores: kept})
+	server.AbuseIPDB.Load(kept)
 }
 
 // wantAbuseIPDBChecks checks how many clients server has checked with
@@ -1116,7 +1116,7 @@ func wantAbuseIPDBChecks(t *testing.T, server *waf.Server, want int) {
 }
 
 // loadLists puts copies of lists into server's lists, by URL, each with
-// its lines, fetched at listsFetched, as reputation.json would at start.
+// its lines, fetched at listsFetched, as lists.json would at start.
 func loadLists(t *testing.T, server *waf.Server, copies map[string][]string) {
 	t.Helper()
 

internal/waf/waf.go

@@ -143,10 +143,9 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 // refusedWhateverTheAnswer reports whether the request is one that does not wait
 // for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
-// its client: the heuristic crowdsec, header_limits, blocklists or rate_limits
+// its client: the heuristic crowdsec, blocklists or rate_limits will refuse it,
-// will refuse it, as a function in that heuristic's file says. In observe mode,
+// as a function in that heuristic's file says. In observe mode, which refuses
-// which refuses nothing, none is. rproxy lets a request through when
+// nothing, none is. rproxy lets a request through when waf answers too late.
-// waf answers too late.
 func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
 	if rq.h.config.Observe {
 		return false
@@ -154,8 +153,7 @@ func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
 
 	_, decided := rq.crowdSecDecisionInForce(now)
 
-	return decided || overHeaderLimits(rq.headerCounts()) ||
+	return decided || rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
-		rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
 		rq.overAWholeRateLimit(now)
 }
 
@@ -215,10 +213,10 @@ func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
 
 // readQuestion returns the visitor's address and the body a question carries,
 // and whether waf takes the question. It answers a question it does not
-// take with 400 and no disposition, which rproxy treats as it treats no
+// take with 400 and no disposition. rproxy passes that 400 to the visitor,
-// answer. The id of a question it takes is held from here on, without a
+// and SWRP_DEFAULT_DISPOSITION does not apply. The id of a question waf takes
-// decision yet, so that another question with that id is refused while this
+// is held from here on, without a decision yet, so that another question with
-// one is being handled.
+// that id is refused while this one is being handled.
 //
 // The body is read before any check, since Go's server does not send an answer
 // until it has read the body. A question whose body stops arriving gets no

script/example-app

@@ -9,6 +9,7 @@
 #   - that a probe for /.env bans another client, which its next request bans
 #     for good;
 #   - that only waf's container changes the state directory's owner;
+#   - that waf's container makes the cache directory for the sww user;
 #   - that the app still answers through rproxy with waf stopped, and
 #     that once waf has started again, bans apply again;
 #   - that `sv stop` stops rproxy in order, and `docker stop` stops each
@@ -17,8 +18,8 @@
 #     client;
 #   - that a waf container with its state files on a new volume mounted at
 #     the directory SWW_STATE_DIR_FILE names becomes healthy;
-#   - that one whose SWW_STATE_DIR does not exist logs waf's message
+#   - that the run script makes a SWW_STATE_DIR that does not exist, owned
-#     about it.
+#     by sww with mode 0700, and that waf then becomes healthy.
 # waf runs with SWW_LOOKUP_SOURCE=off, so that no address is sent to
 # GeoJS. The containers, the network, the volumes and both images are removed
 # however the script ends. Building the app needs network access, for nixpkgs'
@@ -153,9 +154,15 @@ refused_from() {
     [ "$(status_from "$1" "$2")" = 403 ]
 }
 
-# owner_of_state_dir <container>: print who owns /var/lib/sww there.
+# owner_of <container> <dir>: print who owns dir there.
-owner_of_state_dir() {
+owner_of() {
-    docker exec "$1" stat --format '%U' /var/lib/sww
+    docker exec "$1" stat --format '%U' "$2"
+}
+
+# owner_and_mode_of <container> <dir>: print who owns dir there, and its
+# mode in octal.
+owner_and_mode_of() {
+    docker exec "$1" stat --format '%U %a' "$2"
 }
 
 main() {
@@ -173,11 +180,14 @@ main() {
     start_app
     echo "example-app: the health checks pass"
 
-    [ "$(owner_of_state_dir "$WAF")" = sww ] ||
+    [ "$(owner_of "$WAF" /var/lib/sww)" = sww ] ||
         fail "waf's container did not give the state directory to sww"
-    [ "$(owner_of_state_dir "$APP")" = root ] ||
+    [ "$(owner_of "$APP" /var/lib/sww)" = root ] ||
         fail "the app's container changed the state directory's owner"
     echo "example-app: only waf's container changes the state directory's owner"
+    [ "$(owner_of "$WAF" /var/cache/sww)" = sww ] ||
+        fail "waf's container did not make the cache directory for sww"
+    echo "example-app: waf's container makes the cache directory for sww"
 
     page="$(curl --fail --silent --show-error --max-time 10 "http://$address/")" ||
         fail "no answer on port 8080"
@@ -247,9 +257,11 @@ main() {
         --env SWW_STATE_DIR=/srv/missing \
         --env SWW_LOOKUP_SOURCE=off \
         "$IMAGE" >/dev/null
-    wait_for "waf did not report the missing state directory" \
+    wait_for "waf's health check did not pass" healthy "$WAF"
-        logged "$WAF" "SWW_STATE_DIR cannot be written"
+    [ "$(owner_and_mode_of "$WAF" /srv/missing)" = "sww 700" ] ||
-    echo "example-app: waf reports a state directory that does not exist"
+        fail "the run script did not make SWW_STATE_DIR for sww with mode 0700"
+    echo "example-app: the run script makes a missing SWW_STATE_DIR for sww" \
+        "with mode 0700"
 }
 
 main "$@"

script/run

@@ -6,8 +6,9 @@
 # waf listens on port 8090 unless SWW_LISTEN_ADDR says otherwise, and
 # rproxy asks it there unless SWRP_WAF_BACKEND says otherwise. Unless
 # SWW_STATE_DIR is set, the state files go in bin/state, beside the binary,
-# and unless SWW_RULES_DIR is set, the rule files are those of share/rules.d,
+# and unless SWW_CACHE_DIR is set, the cache files go in bin/cache. Unless
-# which the image ships.
+# SWW_RULES_DIR is set, the rule files are those of share/rules.d, which the
+# image ships.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -31,6 +32,11 @@ main() {
         export SWW_STATE_DIR
         mkdir -p "$SWW_STATE_DIR"
     fi
+    if [ -z "${SWW_CACHE_DIR+set}" ]; then
+        SWW_CACHE_DIR="$ROOT/bin/cache"
+        export SWW_CACHE_DIR
+        mkdir -p "$SWW_CACHE_DIR"
+    fi
     if [ -z "${SWW_RULES_DIR+set}" ]; then
         SWW_RULES_DIR="$ROOT/share/rules.d"
         export SWW_RULES_DIR

share/sww.run

@@ -11,7 +11,10 @@ main() {
     local command="${SWW_COMMAND:-rproxy}"
     case "$command" in
         rproxy) ;;
-        waf) give_state_dir_to_sww ;;
+        waf)
+            make_state_dir_for_sww
+            make_cache_dir_for_sww
+            ;;
         *)
             printf 'SWW_COMMAND: "%s" is not rproxy or waf\n' \
                 "$command" >&2
@@ -21,29 +24,43 @@ main() {
     exec chpst -u sww:sww /usr/local/bin/sww "$command"
 }
 
-# give_state_dir_to_sww gives the state directory and every file in
+# make_state_dir_for_sww makes the state directory, SWW_STATE_DIR in
-# it to the sww user, so that a volume mounted there needs no change
+# either form, when it is missing, gives it and every file in it to the
-# of owner; chown -R changes a symbolic link itself, never what it points
+# sww user, and sets its mode to 0700. So a volume mounted there needs no
-# to. When SWW_STATE_DIR, in either form, cannot be read or is not the
+# change of owner; chown -R changes a symbolic link itself, never what it
-# absolute path of a directory, no owner is changed, and waf starts and
+# points to. When the setting cannot be read or is not an absolute path,
-# stops with a message naming the setting. A change of owner that fails, as
+# nothing is changed, and waf starts and stops with a message naming the
-# on a volume mounted read-only, does not end the script either: waf
+# setting. A step that fails, as on a volume mounted read-only, does not
-# reports a directory it cannot write.
+# end the script either: waf reports a directory it cannot write.
-give_state_dir_to_sww() {
+make_state_dir_for_sww() {
     local dir
-    if dir="$(state_dir)" && [[ "$dir" == /* && -d "$dir" ]]; then
+    if dir="$(setting SWW_STATE_DIR /var/lib/sww)" && [[ "$dir" == /* ]]; then
-        chown -R sww:sww "$dir" || true
+        { mkdir -p -- "$dir" && chown -R sww:sww "$dir" &&
+            chmod 0700 "$dir"; } || true
     fi
 }
 
-# state_dir prints SWW_STATE_DIR, /var/lib/sww when it is unset,
+# make_cache_dir_for_sww makes the cache directory, SWW_CACHE_DIR in
-# or what is in the file SWW_STATE_DIR_FILE names, less every newline at
+# either form, when it is missing, and gives it and every file in it to
-# its end. It fails when both are set or the file cannot be read.
+# the sww user. As for the state directory, a setting that cannot be read
-state_dir() {
+# or is not an absolute path changes nothing, and a failure does not end
-    if [[ ! -v SWW_STATE_DIR_FILE ]]; then
+# the script.
-        printf '%s\n' "${SWW_STATE_DIR-/var/lib/sww}"
+make_cache_dir_for_sww() {
-    elif [[ ! -v SWW_STATE_DIR ]]; then
+    local dir
-        cat -- "$SWW_STATE_DIR_FILE" 2>/dev/null
+    if dir="$(setting SWW_CACHE_DIR /var/cache/sww)" && [[ "$dir" == /* ]]; then
+        { mkdir -p -- "$dir" && chown -R sww:sww "$dir"; } || true
+    fi
+}
+
+# setting prints the setting named $1, $2 when it is unset, or what is in
+# the file the setting $1_FILE names, less every newline at its end. It
+# fails when both are set or the file cannot be read.
+setting() {
+    local name="$1" file_name="$1_FILE"
+    if [[ ! -v "$file_name" ]]; then
+        printf '%s\n' "${!name-$2}"
+    elif [[ ! -v "$name" ]]; then
+        cat -- "${!file_name}" 2>/dev/null
     else
         return 1
     fi