Caches of outside APIs live in /var/cache/sww (closes #314)

The answers of GeoJS, the lists fetched by URL, the DNS blocklists,
AbuseIPDB and routewatch each go in a cache file in SWW_CACHE_DIR
(default /var/cache/sww): lookups.json, lists.json, dnsbl.json,
abuseipdb-scores.json and routewatch.json. reputation.json is gone. The
AbuseIPDB checks spent today are a count, not a cache, and stay in
SWW_STATE_DIR as abuseipdb-checks-spent.json.

A cache file never stops the start: one that does not parse is set
aside as an edit is, one that cannot be read is written over, and its
cache starts empty. An unwritable SWW_CACHE_DIR stops the start, as an
unwritable SWW_STATE_DIR does. Each directory is watched on its own.
The run script makes the cache directory and gives it to sww.

Model: opus-5-5

README.md

@@ -510,7 +510,8 @@ request reaches the app.
 
 `waf` takes the checks:
 
-3. The tag lists.
+3. The header limits, then the tag lists. A client in `SWW_ALLOW_NETS` skips the
+   header limits.
 4. The static lists, `SWW_ALLOW_NETS` and `SWW_DENY_NETS`.
 5. The bans.
 6. The lookup of the client's AS number and country.
@@ -522,7 +523,7 @@ request reaches the app.
 11. The rate limits.
 12. The trap paths.
 13. The rule files.
-14. The header limits, and then the Core Rule Set.
+14. The Core Rule Set.
 15. The tag refusals again, so that `exploitsignature` can refuse too.
 16. A path under `/_sww/` is answered `404`, so that the app never sees that
     prefix.
@@ -538,7 +539,7 @@ request reaches the app.
 A refused visitor is answered with `SWW_BAN_RESPONSE`. A `block` rule or a Core
 Rule Set match answers `403` and bans nobody, since the Core Rule Set has false
 positives; such refusals count toward `SWW_ERROR_BURST_THRESHOLD` instead. A
-request over the header limits of step 14 is answered `431`, and bans nobody
+request over the header limits of step 3 is answered `431`, and bans nobody
 either. `rproxy` itself answers `405` to a method it does not pass on, and the
 status for a size or time limit, such as `413` for a body too large.
 
@@ -621,13 +622,14 @@ client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
 - The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
     - `bans.json`;
     - `clients.json`;
-    - `abuseipdb-checks.json`;
+    - `abuseipdb-checks-spent.json`;
     - `alerts.json`.
 - The cache files, in `SWW_CACHE_DIR`, hold answers from outside services:
     - `lookups.json`, from GeoJS;
     - `lists.json`, the lists fetched by URL;
     - `dnsbl.json`, from the DNS blocklists;
-    - `abuseipdb-scores.json`, from AbuseIPDB.
+    - `abuseipdb-scores.json`, from AbuseIPDB;
+    - `routewatch.json`, from routewatch.
 
 Back up the state directory. The cache directory can be left out of backups: a
 lost cache file starts empty and fills again.

SPEC.md

@@ -149,10 +149,12 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
     - An IPv6 client is its group: the netblock of `SWW_IPV6_GROUP_PREFIX`
       around its address, a /64 by default, since one abuser usually controls a
       whole /64. A client is therefore one IPv4 address or one IPv6 group.
-3. The tag lists (`tag_lists`; see "Request tags"). Each list that holds the
-   client adds its tag. Next, `routewatch_tags` adds the client's `asn=` and
+3. The header limits (`header_limits`; see "Answering in time"). They come first
+   after `SWW_ALLOW_NETS`, since they only count header fields and cookies.
+   Next, the tag lists (`tag_lists`; see "Request tags"). Each list that holds
+   the client adds its tag. Next, `routewatch_tags` adds the client's `asn=` and
    `prefix=` tags while `SWW_ROUTEWATCH_URL` is set. A client in
-   `SWW_ALLOW_NETS` gets no tag.
+   `SWW_ALLOW_NETS` skips the header limits and gets no tag.
 4. The static lists. A client in `SWW_ALLOW_NETS` skips steps 5 to 15, the byte
    limits and the error burst. A client in `SWW_DENY_NETS` is refused
    (`deny_nets`); a client in both lists is let through.
@@ -185,10 +187,9 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
     refused and bans the client.
 13. The rule files (`rule_files`; see "Rule files"). Matching stops at the first
     rule that refuses.
-14. The header limits (`header_limits`; see "Answering in time"), and then the
-    Core Rule Set (`core_rule_set`; see "Attack detection"). A Core Rule Set
-    match is refused with `403` while `SWW_WAF_MODE` is `block`, and only logged
-    and alerted while it is `detect`.
+14. The Core Rule Set (`core_rule_set`; see "Attack detection"). A match is
+    refused with `403` while `SWW_WAF_MODE` is `block`, and only logged and
+    alerted while it is `detect`.
 15. The tag refusals again (`tag_deny`), so that `exploitsignature` can refuse
     too.
 16. A path under `/_sww/` is answered `404`, so that the app never sees that
@@ -241,11 +242,16 @@ apply.
 ## Bans
 
 Until a ban ends, every request from its netblock is refused with
-`SWW_BAN_RESPONSE`. The netblock is the client: its IPv4 address, widened to
-`SWW_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one
-address or one /64, so a permanent ban does not reach neighbours who did
-nothing. Requests refused under a ban are not counted against limits. They are
-counted in the client's history and in the ban's notes.
+`SWW_BAN_RESPONSE`, unless it is over the header limits. The netblock is the
+client: its IPv4 address, widened to `SWW_BAN_SCOPE_V4_PREFIX`, or its IPv6
+group. At the defaults a ban covers one address or one /64, so a permanent ban
+does not reach neighbours who did nothing. Requests refused under a ban are not
+counted against limits. They are counted in the client's history and in the
+ban's notes.
+
+A request over the header limits is refused with `431` before the bans are
+checked (see "Answering in time"). It makes no ban, makes no ban permanent and
+is not counted in a ban's notes.
 
 An offence is a request `waf` holds against the client. It is one of:
 
@@ -577,7 +583,7 @@ Each tag and where it comes from:
     - They are added before `deny_nets`, so the log line of a request it refused
       has them.
     - `waf` asks routewatch in the background and never waits for it. An answer
-      is kept for 1 hour, in memory only.
+      is kept for 1 hour, in memory and in `routewatch.json`.
     - After a query fails, routewatch is not asked for 1 minute.
     - A request without these tags gives the reason in the log field
       `routewatch_tags_missing` (see
@@ -1653,11 +1659,11 @@ together.
   client's later requests. For an IPv6 client, AbuseIPDB is asked about the
   address of the request that set off the check, and the score is used for the
   whole group. A score at or over `SWW_ABUSEIPDB_MIN_SCORE` is a hit. The budget
-  is counted in UTC days and kept in `abuseipdb-checks.json`, so a restart does
-  not make it whole again. The check that spends the last of it is logged and
-  alerted, and no client is checked until the day ends. The key is sent in the
-  `Key` header and nowhere else, and is never logged. A check fails, after which
-  no client is checked for a minute, when:
+  is counted in UTC days and kept in `abuseipdb-checks-spent.json`, so a restart
+  does not make it whole again. The check that spends the last of it is logged
+  and alerted, and no client is checked until the day ends. The key is sent in
+  the `Key` header and nowhere else, and is never logged. A check fails, after
+  which no client is checked for a minute, when:
     - AbuseIPDB answers other than `200`;
     - the answer gives no score;
     - no answer comes within `SWW_REPUTATION_TIMEOUT`.
@@ -1776,22 +1782,37 @@ them at any time, and the running `waf` takes the edit in. `docs/state-files.md`
 lists the files, what each holds, and their sizes.
 
 - The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
-  `bans.json`, `clients.json`, `abuseipdb-checks.json` and `alerts.json`.
+    - `bans.json`;
+    - `clients.json`;
+    - `abuseipdb-checks-spent.json`;
+    - `alerts.json`.
 - The cache files, in `SWW_CACHE_DIR`, hold the answers of outside services:
-  `lookups.json`, `lists.json`, `dnsbl.json` and `abuseipdb-scores.json`. A lost
-  cache file costs only questions asked again, so the cache directory can be
-  left out of backups.
+    - `lookups.json`;
+    - `lists.json`;
+    - `dnsbl.json`;
+    - `abuseipdb-scores.json`;
+    - `routewatch.json`.
+- A lost cache file costs only questions asked again, so the cache directory can
+  be left out of backups.
 - Each file holds one kind of state, as JSON with:
     - a top-level `version` number;
     - entries sorted by client address;
     - times in RFC 3339 UTC;
     - days as `YYYY-MM-DD`;
     - durations and sizes as plain numbers, with the unit in the field name.
-- `bans.json`, `abuseipdb-checks.json`, `alerts.json` and `lists.json` are
-  indented, so that a person can open `bans.json` in an editor, find an address,
-  and remove or add an entry. `clients.json`, `lookups.json`, `dnsbl.json` and
-  `abuseipdb-scores.json` put each entry on one line, which halves their size
-  and lets `grep` show everything about one client.
+- These files are indented, so that a person can open `bans.json` in an editor,
+  find an address, and remove or add an entry:
+    - `bans.json`;
+    - `abuseipdb-checks-spent.json`;
+    - `alerts.json`;
+    - `lists.json`.
+- The others put each entry on one line, which halves their size and lets `grep`
+  show everything about one client:
+    - `clients.json`;
+    - `lookups.json`;
+    - `dnsbl.json`;
+    - `abuseipdb-scores.json`;
+    - `routewatch.json`.
 - Clients that were never banned are kept too, so every client's history
   survives a restart.
 - Each file has a bound, so it can be rewritten whole. Writing one takes well
@@ -2037,7 +2058,6 @@ waits on these, which run in the background:
 
 In `enforce` mode, a question does not wait for GeoJS when it is a request:
 
-- over the header limits;
 - from a client on a fetched blocklist, while `SWW_BLOCKLIST_ACTION` is `deny`;
 - from a client on the CrowdSec decision list;
 - that would take its client over a rate limit at 100% of it, since a percentage
@@ -2049,8 +2069,8 @@ number and country stay unknown until GeoJS has answered. So while
 `SWW_UNKNOWN_LIMIT_PERCENT` does not apply to it until then, so not waiting
 never lowers its limit.
 
-The static lists and bans come before the lookup, so they never wait. These can
-still wait for GeoJS:
+The header limits, the static lists and bans come before the lookup, so they
+never wait. These can still wait for GeoJS:
 
 - a country list, which needs the answer;
 - a rate limit a percentage has lowered, which needs the answer;
@@ -2069,13 +2089,17 @@ either one:
 
 Real clients never reach these limits. The answer is `deny` with `431`, and the
 log line's action is `request_header_too_large`. The refusal makes no ban and is
-no offence. The rate limits count the request as usual, so a client that keeps
-sending such requests breaks a limit.
+no offence.
+
+The header limits are the first check after `SWW_ALLOW_NETS`. A request over
+them is refused before any later check runs. So it causes no lookup, no query to
+an outside source and no rule match, and the rate limits do not count it. A
+client under a ban or in `SWW_DENY_NETS` also gets `431` for such a request.
 
 The limits apply on every path and in every `SWW_WAF_MODE`. A client in
 `SWW_ALLOW_NETS` skips them, as it skips the Core Rule Set. In `observe` mode
-such a request is answered `allow` with `would_action`, and the Core Rule Set
-does not read it.
+such a request is answered `allow` with `would_action`, and no other check runs
+on it.
 
 The Core Rule Set runs each of its rules on each cookie, so thousands of small
 cookies would keep it busy for over a second. Within the limits, one long cookie
@@ -2578,8 +2602,8 @@ app image built on it would inherit it. `rproxy` keeps nothing, and needs no
 volume.
 
 `waf` keeps its cache files in `SWW_CACHE_DIR`, `/var/cache/sww` by default,
-which the `run` script makes. It needs no volume and can be left out of backups:
-a new container starts with empty caches, which fill again.
+which the `run` script makes. The cache directory needs no volume and can be
+left out of backups: a new container starts with empty caches, which fill again.
 
 ### Tokens
 

docs/alerts.md

@@ -23,7 +23,8 @@ back or dropped.
     - the day's AbuseIPDB checks used up.
 - `file_error`: one of:
     - a rule file or state file edit with an error;
-    - a state file that cannot be written;
+    - a cache file that does not parse, at start or after an edit;
+    - a state file or cache file that cannot be written;
     - a replacement lookup database that cannot be read.
 - `summary`: the hourly summary of alerts held back.
 

docs/state-files.md

@@ -8,7 +8,7 @@ top-level `version` of 1.
   what cannot be fetched again, so back them up:
     - `bans.json`;
     - `clients.json`;
-    - `abuseipdb-checks.json`;
+    - `abuseipdb-checks-spent.json`;
     - `alerts.json`.
 - The cache files are in `SWW_CACHE_DIR`, `/var/cache/sww` by default. They hold
   answers from outside services, which are asked again when a file is lost, so
@@ -16,7 +16,8 @@ top-level `version` of 1.
     - `lookups.json`;
     - `lists.json`;
     - `dnsbl.json`;
-    - `abuseipdb-scores.json`.
+    - `abuseipdb-scores.json`;
+    - `routewatch.json`.
 
 ## `bans.json`
 
@@ -81,7 +82,7 @@ since it was first seen:
   `blocked_by_core_rule_set`. A file may also hold `token_refused`, which is
   kept, and which nothing adds to.
 
-## `abuseipdb-checks.json`
+## `abuseipdb-checks-spent.json`
 
 The AbuseIPDB checks spent today, so that a restart does not spend the daily
 budget again:
@@ -132,6 +133,16 @@ A cache file. AbuseIPDB's scores still in use, one per line:
 - `score`: from 0 to 100;
 - `fetched`: when AbuseIPDB answered.
 
+## `routewatch.json`
+
+A cache file. routewatch's answers from the last hour, one per line:
+
+- `client`;
+- `netblock`: the netblock the client's address is announced in, empty for an
+  address without a live route;
+- `asn`: the number of the AS that originates the netblock, 0 for none;
+- `answered`: when routewatch answered.
+
 ## Sizes
 
 - `clients.json`: about 1 KiB per client, about 20 MiB at the default
@@ -143,6 +154,7 @@ A cache file. AbuseIPDB's scores still in use, one per line:
   more.
 - `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
   full.
-- `abuseipdb-checks.json`: a few bytes.
+- `routewatch.json`: about 120 bytes per answer, at most about 12 MiB.
+- `abuseipdb-checks-spent.json`: a few bytes.
 - `lists.json`, `dnsbl.json`, `abuseipdb-scores.json` and `alerts.json`: usually
   a few MiB or less.

internal/reputation/abuseipdb.go

@@ -45,7 +45,7 @@ type Score struct {
 	Fetched time.Time    `json:"fetched"`
 }
 
-// ChecksSpent is what abuseipdb-checks.json keeps of the checks of
+// ChecksSpent is what abuseipdb-checks-spent.json keeps of the checks of
 // clients with AbuseIPDB: the day, in UTC, of the checks Spent counts,
 // zero before the first.
 type ChecksSpent struct {
@@ -238,8 +238,8 @@ func (a *AbuseIPDB) Load(scores []Score) {
 	}
 }
 
-// ChecksSpent returns the checks spent, as abuseipdb-checks.json keeps
-// them.
+// ChecksSpent returns the checks spent, as abuseipdb-checks-spent.json
+// keeps them.
 func (a *AbuseIPDB) ChecksSpent() ChecksSpent {
 	a.mu.Lock()
 	defer a.mu.Unlock()
@@ -247,8 +247,8 @@ func (a *AbuseIPDB) ChecksSpent() ChecksSpent {
 	return ChecksSpent{Day: a.day, Spent: a.spent}
 }
 
-// LoadChecksSpent keeps checks, read from abuseipdb-checks.json, in place
-// of the checks spent.
+// LoadChecksSpent keeps checks, read from abuseipdb-checks-spent.json, in
+// place of the checks spent.
 func (a *AbuseIPDB) LoadChecksSpent(checks ChecksSpent) {
 	a.mu.Lock()
 	defer a.mu.Unlock()

internal/reputation/abuseipdb_test.go

@@ -181,8 +181,8 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
 		}
 
 		// Restarted with what abuseipdb-scores.json and
-		// abuseipdb-checks.json keep, it uses the scores, and checks no
-		// client until the day ends.
+		// abuseipdb-checks-spent.json keep, it uses the scores, and checks
+		// no client until the day ends.
 		restarted := &abuseIPDBStandIn{}
 		again := newAbuseIPDB(restarted, p)
 		again.Load(checker.Snapshot())

internal/routewatch/routewatch.go

@@ -1,6 +1,8 @@
 // Package routewatch asks a routewatch instance about client addresses, in the
-// background, and keeps its answers in memory. A routewatch instance knows, for
-// an address, the netblock it is announced in and the AS that originates it.
+// background, and keeps its answers. A routewatch instance knows, for an
+// address, the netblock it is announced in and the AS that originates it. The
+// answers are kept in memory, for at most 100,000 clients and for an hour each,
+// and are written to routewatch.json and read from it by the state package.
 package routewatch
 
 import (
@@ -13,6 +15,7 @@ import (
 	"net/http"
 	"net/netip"
 	"net/url"
+	"slices"
 	"sync"
 	"time"
 
@@ -65,6 +68,17 @@ type Route struct {
 	ASN      uint32
 }
 
+// Answer is what routewatch answered about a client, as routewatch.json holds
+// it: the netblock the client's address is announced in, none for an address
+// without a live route, the AS that originates it, and when routewatch
+// answered.
+type Answer struct {
+	Client   netip.Prefix `json:"client"`
+	Netblock netip.Prefix `json:"netblock"`
+	ASN      uint32       `json:"asn"`
+	Answered time.Time    `json:"answered"`
+}
+
 // Params are what New needs.
 type Params struct {
 	// URL is where routewatch is (SWW_ROUTEWATCH_URL).
@@ -84,23 +98,16 @@ type Client struct {
 	mu sync.Mutex
 	// answers are by client. Each is added as it comes and never moved up,
 	// so that the one that came longest ago is the first dropped.
-	answers *simplelru.LRU[netip.Prefix, answer]
+	answers *simplelru.LRU[netip.Prefix, Answer]
 	// asking are the clients whose query is under way.
 	asking map[netip.Prefix]bool
 	// retryAt is when routewatch may be asked again after a query failed.
 	retryAt time.Time
 }
 
-// answer is one kept answer: the route, or why there is none, and when it came.
-type answer struct {
-	route   Route
-	missing Missing
-	came    time.Time
-}
-
 // New returns a Client with no answer yet.
 func New(params Params) *Client {
-	answers, err := simplelru.NewLRU[netip.Prefix, answer](maxAnswers, nil)
+	answers, err := simplelru.NewLRU[netip.Prefix, Answer](maxAnswers, nil)
 	if err != nil {
 		panic(err) // NewLRU fails only for a size below one
 	}
@@ -128,8 +135,12 @@ func (c *Client) Route(
 	now := c.params.Now()
 
 	kept, found := c.answers.Peek(client)
-	if found && now.Sub(kept.came) < answerKept {
-		return kept.route, kept.missing
+	if found && now.Sub(kept.Answered) < answerKept {
+		if !kept.Netblock.IsValid() {
+			return Route{}, NoRoute
+		}
+
+		return Route{Netblock: kept.Netblock, ASN: kept.ASN}, ""
 	}
 
 	if now.Before(c.retryAt) {
@@ -145,12 +156,58 @@ func (c *Client) Route(
 	return Route{}, NoAnswerYet
 }
 
+// Snapshot returns every answer still in use, sorted by client, as
+// routewatch.json lists them.
+func (c *Client) Snapshot() []Answer {
+	c.mu.Lock()
+
+	now := c.params.Now()
+	answers := make([]Answer, 0, c.answers.Len())
+
+	for _, kept := range c.answers.Values() {
+		if now.Sub(kept.Answered) < answerKept {
+			answers = append(answers, kept)
+		}
+	}
+
+	c.mu.Unlock()
+
+	slices.SortFunc(answers, func(a, b Answer) int {
+		return a.Client.Compare(b.Client)
+	})
+
+	return answers
+}
+
+// Load keeps answers, read from routewatch.json, in place of those it keeps,
+// but for those that came answerKept ago or more and, past maxAnswers, those
+// that came longest ago.
+func (c *Client) Load(answers []Answer) {
+	answers = slices.Clone(answers)
+	slices.SortStableFunc(answers, func(a, b Answer) int {
+		return a.Answered.Compare(b.Answered)
+	})
+
+	c.mu.Lock()
+	defer c.mu.Unlock()
+
+	c.answers.Purge()
+
+	now := c.params.Now()
+
+	for _, kept := range answers {
+		if now.Sub(kept.Answered) < answerKept {
+			c.answers.Add(kept.Client, kept)
+		}
+	}
+}
+
 // ask asks routewatch about addr, keeps the answer as client's, and notes the
 // query as no longer under way. A query that fails gives no answer, and
 // routewatch is not asked again for failureKept. Only the first failure of
 // those under way is logged.
 func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr) {
-	route, missing, err := c.query(ctx, addr)
+	route, err := c.query(ctx, addr)
 	now := c.params.Now()
 
 	c.mu.Lock()
@@ -160,7 +217,9 @@ func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr)
 	firstFailure := err != nil && !now.Before(c.retryAt)
 
 	if err == nil {
-		c.answers.Add(client, answer{route: route, missing: missing, came: now})
+		c.answers.Add(client, Answer{
+			Client: client, Netblock: route.Netblock, ASN: route.ASN, Answered: now,
+		})
 	} else {
 		c.retryAt = now.Add(failureKept)
 	}
@@ -174,19 +233,19 @@ func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr)
 
 // query asks routewatch's IP info API about addr. An answer other than 200, one
 // that cannot be read, and none within queryTimeout, fail. A result with no
-// netblock is an address without a live route.
-func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, error) {
+// netblock is an address without a live route, and gives a Route without one.
+func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, error) {
 	ctx, cancel := context.WithTimeout(ctx, queryTimeout)
 	defer cancel()
 
 	endpoint, err := url.JoinPath(c.params.URL, "api/v1/ip", addr.String())
 	if err != nil {
-		return Route{}, "", fmt.Errorf("make the URL: %w", err)
+		return Route{}, fmt.Errorf("make the URL: %w", err)
 	}
 
 	req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, http.NoBody)
 	if err != nil {
-		return Route{}, "", fmt.Errorf("make the request: %w", err)
+		return Route{}, fmt.Errorf("make the request: %w", err)
 	}
 
 	req.Header.Set("Accept", "application/json")
@@ -195,7 +254,7 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, er
 	if err != nil {
 		// Do's error names the URL, which holds the client's address, which
 		// is not to be logged: only what went wrong is kept.
-		return Route{}, "", fmt.Errorf("ask routewatch: %w", errors.Unwrap(err))
+		return Route{}, fmt.Errorf("ask routewatch: %w", errors.Unwrap(err))
 	}
 
 	defer func() {
@@ -203,14 +262,14 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, er
 	}()
 
 	if res.StatusCode != http.StatusOK {
-		return Route{}, "", fmt.Errorf("%w %s", errStatus, res.Status)
+		return Route{}, fmt.Errorf("%w %s", errStatus, res.Status)
 	}
 
 	return readAnswer(io.LimitReader(res.Body, maxAnswerBytes))
 }
 
 // readAnswer reads routewatch's answer about one address.
-func readAnswer(body io.Reader) (Route, Missing, error) {
+func readAnswer(body io.Reader) (Route, error) {
 	var decoded struct {
 		Data struct {
 			Results []struct {
@@ -222,22 +281,22 @@ func readAnswer(body io.Reader) (Route, Missing, error) {
 
 	err := json.NewDecoder(body).Decode(&decoded)
 	if err != nil {
-		return Route{}, "", fmt.Errorf("read the answer: %w", err)
+		return Route{}, fmt.Errorf("read the answer: %w", err)
 	}
 
 	if len(decoded.Data.Results) == 0 {
-		return Route{}, "", errNoResult
+		return Route{}, errNoResult
 	}
 
 	result := decoded.Data.Results[0]
 	if result.Netblock == "" {
-		return Route{}, NoRoute, nil
+		return Route{}, nil
 	}
 
 	netblock, err := netip.ParsePrefix(result.Netblock)
 	if err != nil {
-		return Route{}, "", fmt.Errorf("%w: %q", errNotNetblock, result.Netblock)
+		return Route{}, fmt.Errorf("%w: %q", errNotNetblock, result.Netblock)
 	}
 
-	return Route{Netblock: netblock.Masked(), ASN: result.ASN}, "", nil
+	return Route{Netblock: netblock.Masked(), ASN: result.ASN}, nil
 }

internal/routewatch/routewatch_test.go

@@ -7,6 +7,7 @@ import (
 	"net/http"
 	"net/http/httptest"
 	"net/netip"
+	"slices"
 	"strings"
 	"sync"
 	"sync/atomic"
@@ -336,3 +337,96 @@ func TestRepeatedRequestsMakeOneQueryAndTheAnswerExpires(t *testing.T) {
 		t.Errorf("%d queries after the answer expired, want 2", got)
 	}
 }
+
+func TestSnapshotHoldsTheAnswerUntilItExpires(t *testing.T) {
+	t.Parallel()
+
+	f := newFixture(t, answering(routedAnswer))
+
+	f.route()
+	f.waitFor("")
+
+	want := []routewatch.Answer{{
+		Client:   netip.MustParsePrefix(clientAddr + "/32"),
+		Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496,
+		Answered: f.clk.Now(),
+	}}
+	if got := f.client.Snapshot(); !slices.Equal(got, want) {
+		t.Errorf("snapshot %+v, want %+v", got, want)
+	}
+
+	f.clk.advance(time.Hour)
+
+	if got := f.client.Snapshot(); len(got) != 0 {
+		t.Errorf("snapshot %+v an hour after the answer, want none", got)
+	}
+}
+
+func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
+	t.Parallel()
+
+	f := newFixture(t, answering(routedAnswer))
+	now := f.clk.Now()
+	kept := routewatch.Answer{
+		Client:   netip.MustParsePrefix(clientAddr + "/32"),
+		Netblock: netip.MustParsePrefix("198.51.100.0/24"), ASN: 64497,
+		Answered: now.Add(-time.Hour + time.Second),
+	}
+	stale := routewatch.Answer{
+		Client:   netip.MustParsePrefix("192.0.2.2/32"),
+		Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496,
+		Answered: now.Add(-time.Hour),
+	}
+
+	f.client.Load([]routewatch.Answer{kept, stale})
+
+	if got := f.client.Snapshot(); !slices.Equal(got, []routewatch.Answer{kept}) {
+		t.Errorf("kept %+v, want only the answer that came less than an hour ago", got)
+	}
+
+	route, missing := f.route()
+
+	want := routewatch.Route{Netblock: kept.Netblock, ASN: kept.ASN}
+	if route != want || missing != "" {
+		t.Errorf("route %+v (%q), want %+v", route, missing, want)
+	}
+
+	if got := f.queries.Load(); got != 0 {
+		t.Errorf("%d queries for a client with an answer loaded, want none", got)
+	}
+}
+
+func TestLoadDropsTheAnswerThatCameLongestAgoFirst(t *testing.T) {
+	t.Parallel()
+
+	const maxAnswers = 100000
+
+	f := newFixture(t, answering(routedAnswer))
+	now := f.clk.Now()
+
+	// routewatch.json lists the answers by client. Here each came a
+	// millisecond before the one listed before it, so the last listed is
+	// the one that came longest ago, and the one dropped.
+	answers := make([]routewatch.Answer, maxAnswers+1)
+	addr := netip.MustParseAddr("10.0.0.0")
+
+	for i := range answers {
+		answers[i] = routewatch.Answer{
+			Client:   netip.PrefixFrom(addr, addr.BitLen()),
+			Answered: now.Add(-time.Duration(i) * time.Millisecond),
+		}
+		addr = addr.Next()
+	}
+
+	f.client.Load(answers)
+
+	got := f.client.Snapshot()
+	if len(got) != maxAnswers {
+		t.Fatalf("%d answers kept, want %d", len(got), maxAnswers)
+	}
+
+	if got[0] != answers[0] || got[maxAnswers-1] != answers[maxAnswers-1] {
+		t.Errorf("answers kept from %s to %s, want from %s to %s", got[0].Client,
+			got[maxAnswers-1].Client, answers[0].Client, answers[maxAnswers-1].Client)
+	}
+}

internal/rproxy/rproxy_test.go

@@ -1023,7 +1023,9 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
 		w.WriteHeader(http.StatusInternalServerError)
 		_, _ = w.Write(make([]byte, size))
 	})
-	addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
+	addr, _ := startRProxy(t, standIn.URL, nowhere, map[string]string{
+		timeoutSetting: "60000",
+	})
 
 	got := get(t, addr, "/")
 	if got.status != http.StatusInternalServerError || len(got.body) != size {

internal/state/state.go

@@ -4,7 +4,7 @@
 // The state files are in SWW_STATE_DIR:
 //   - bans.json holds the bans;
 //   - clients.json holds each client's counters and history;
-//   - abuseipdb-checks.json holds the AbuseIPDB checks spent today;
+//   - abuseipdb-checks-spent.json holds the AbuseIPDB checks spent today;
 //   - alerts.json holds the cooldowns, the hour under way, the alerts
 //     waiting for each destination and the anomaly counters.
 //
@@ -14,7 +14,8 @@
 //   - lists.json holds the last try and last good copy of each list
 //     fetched from a URL;
 //   - dnsbl.json holds the DNSBL zones' verdicts;
-//   - abuseipdb-scores.json holds AbuseIPDB's scores.
+//   - abuseipdb-scores.json holds AbuseIPDB's scores;
+//   - routewatch.json holds routewatch's answers.
 //
 // Load reads them at start, Watch takes in an admin's edit of one while
 // sww runs, and Run and WriteAll write them. The disk is read and
@@ -49,6 +50,7 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/metrics"
 	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
 	"sneak.berlin/go/smallwebwaf/internal/reputation"
+	"sneak.berlin/go/smallwebwaf/internal/routewatch"
 )
 
 // version is the version of the files' format, the only one read.
@@ -60,10 +62,10 @@ const fileMode = 0o600
 
 // The state files' names, in Dir.
 const (
-	bansJSON            = "bans.json"
-	clientsJSON         = "clients.json"
-	abuseIPDBChecksJSON = "abuseipdb-checks.json"
-	alertsJSON          = "alerts.json"
+	bansJSON                 = "bans.json"
+	clientsJSON              = "clients.json"
+	abuseIPDBChecksSpentJSON = "abuseipdb-checks-spent.json"
+	alertsJSON               = "alerts.json"
 )
 
 // The cache files' names, in CacheDir.
@@ -72,21 +74,22 @@ const (
 	listsJSON           = "lists.json"
 	dnsblJSON           = "dnsbl.json"
 	abuseIPDBScoresJSON = "abuseipdb-scores.json"
+	routeWatchJSON      = "routewatch.json"
 )
 
 // fileNames returns the names of the state files, then those of the
 // cache files, in the order they are written.
 func fileNames() []string {
 	return []string{
-		bansJSON, clientsJSON, abuseIPDBChecksJSON, alertsJSON,
-		lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON,
+		bansJSON, clientsJSON, abuseIPDBChecksSpentJSON, alertsJSON,
+		lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON, routeWatchJSON,
 	}
 }
 
 // isCacheFile reports whether name is the name of a cache file.
 func isCacheFile(name string) bool {
 	switch name {
-	case lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON:
+	case lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON, routeWatchJSON:
 		return true
 	default:
 		return false
@@ -117,17 +120,20 @@ type Params struct {
 	// is (SWW_STATE_COUNTER_INTERVAL).
 	WriteDelay      time.Duration
 	CounterInterval time.Duration
-	// Ledger, Limiter, GeoJS, Lists, DNSBL, AbuseIPDB, Alerts and Anomalies
-	// hold the state. Alerts also receive a file_error alert for an edit set
-	// aside, and for a write that fails while sww runs.
-	Ledger    *bans.Ledger
-	Limiter   *ratelimit.Limiter
-	GeoJS     *lookup.GeoJS
-	Lists     *reputation.Lists
-	DNSBL     *reputation.DNSBL
-	AbuseIPDB *reputation.AbuseIPDB
-	Alerts    *alerts.Queue
-	Anomalies *anomaly.Counters
+	// Ledger, Limiter, GeoJS, Lists, DNSBL, AbuseIPDB, RouteWatch, Alerts and
+	// Anomalies hold the state. RouteWatch is nil while SWW_ROUTEWATCH_URL is
+	// unset, and routewatch.json then holds no answers. Alerts also receive a
+	// file_error alert for an edit set aside, and for a write that fails while
+	// sww runs.
+	Ledger     *bans.Ledger
+	Limiter    *ratelimit.Limiter
+	GeoJS      *lookup.GeoJS
+	Lists      *reputation.Lists
+	DNSBL      *reputation.DNSBL
+	AbuseIPDB  *reputation.AbuseIPDB
+	RouteWatch *routewatch.Client
+	Alerts     *alerts.Queue
+	Anomalies  *anomaly.Counters
 	// Now tells the time by which the counters' buckets run out, normally
 	// time.Now in UTC.
 	Now func() time.Time
@@ -185,9 +191,9 @@ type lookupsFile struct {
 	Lookups []lookup.Answer `json:"lookups"`
 }
 
-// abuseIPDBChecksFile is abuseipdb-checks.json, indented for an admin to
-// read and edit.
-type abuseIPDBChecksFile struct {
+// abuseIPDBChecksSpentFile is abuseipdb-checks-spent.json, indented for an
+// admin to read and edit.
+type abuseIPDBChecksSpentFile struct {
 	Version int        `json:"version"`
 	Day     civil.Date `json:"day,omitzero"`
 	Spent   int        `json:"spent"`
@@ -213,6 +219,12 @@ type abuseIPDBScoresFile struct {
 	Scores  []reputation.Score `json:"scores"`
 }
 
+// routeWatchFile is routewatch.json, with each answer on a line of its own.
+type routeWatchFile struct {
+	Version int                 `json:"version"`
+	Answers []routewatch.Answer `json:"answers"`
+}
+
 // alertsFile is alerts.json, indented for an admin to read and edit.
 //
 //nolint:tagliatelle // the state files use snake_case, as the request log does
@@ -257,7 +269,7 @@ func Load(params Params) (*Files, error) {
 
 	bansRead, bansErr := f.read(bansJSON)
 	clientsRead, clientsErr := f.read(clientsJSON)
-	_, checksErr := f.read(abuseIPDBChecksJSON)
+	_, checksErr := f.read(abuseIPDBChecksSpentJSON)
 	alertsRead, alertsErr := f.read(alertsJSON)
 
 	err = errors.Join(bansErr, clientsErr, checksErr, alertsErr)
@@ -269,6 +281,7 @@ func Load(params Params) (*Files, error) {
 	listsRead := f.readCache(listsJSON)
 	f.readCache(dnsblJSON)
 	f.readCache(abuseIPDBScoresJSON)
+	f.readCache(routeWatchJSON)
 
 	params.ProcessLog.Info("read the state files", "directory", params.Dir,
 		"cache_directory", params.CacheDir, "bans", bansRead, "clients", clientsRead,
@@ -325,19 +338,11 @@ func (f *Files) WriteAll() error {
 // admin's edit of a file as soon as it is saved: what the file holds
 // replaces what sww held for it. An edit that does not parse is left for
 // the file's next write, which sets it aside, since a file can be read
-// while an editor is still writing it. If a directory cannot be watched,
-// that is logged, and an edit is taken in only before its file is
-// written.
+// while an editor is still writing it. A directory that cannot be watched
+// is logged, and the other is still watched. An edit in a directory not
+// watched is taken in only before its file is written.
 func (f *Files) Watch(ctx context.Context) {
 	watcher, err := fsnotify.NewWatcher()
-	if err == nil {
-		defer func() {
-			_ = watcher.Close()
-		}()
-
-		err = errors.Join(watcher.Add(f.params.Dir), watcher.Add(f.params.CacheDir))
-	}
-
 	if err != nil {
 		f.params.ProcessLog.Error("cannot watch the state files for edits",
 			"error", err.Error())
@@ -345,8 +350,33 @@ func (f *Files) Watch(ctx context.Context) {
 		return
 	}
 
+	defer func() {
+		_ = watcher.Close()
+	}()
+
+	var watched []string
+
+	for _, dir := range []struct{ path, failed string }{
+		{f.params.Dir, "cannot watch the state files for edits"},
+		{f.params.CacheDir, "cannot watch the cache files for edits"},
+	} {
+		err = watcher.Add(dir.path)
+		if err != nil {
+			f.params.ProcessLog.Error(dir.failed, "directory", dir.path,
+				"error", err.Error())
+
+			continue
+		}
+
+		watched = append(watched, dir.path)
+	}
+
+	if len(watched) == 0 {
+		return
+	}
+
 	f.params.ProcessLog.Info("watching the state files for edits",
-		"directory", f.params.Dir, "cache_directory", f.params.CacheDir)
+		"directories", watched)
 
 	for {
 		select {
@@ -369,7 +399,10 @@ func (f *Files) Watch(ctx context.Context) {
 // a file_error alert for it.
 func (f *Files) logFailure(name string, err error) {
 	if err != nil {
-		const failed = "writing the state files failed"
+		failed := "writing the state files failed"
+		if isCacheFile(name) {
+			failed = "writing the cache files failed"
+		}
 
 		// Raised before it is logged, so that the alert is there once the
 		// log line is.
@@ -494,8 +527,8 @@ func (f *Files) readChanged(name string) ([]byte, bool, error) {
 // how many entries the file holds. edit is whether data is an admin's
 // edit taken in while sww runs, rather than the file read at the
 // start. An error names the file and, where the JSON decoder tells it,
-// the line and column, or else the entry. abuseipdb-checks.json holds
-// no entries.
+// the line and column, or else the entry. abuseipdb-checks-spent.json
+// holds no entries.
 func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
 	path := filepath.Join(f.dir(name), name)
 
@@ -555,8 +588,8 @@ func (f *Files) takeInState(name, path string, data []byte, edit bool) (int, err
 
 		f.params.Limiter.Load(file.Clients, f.params.Now())
 		entries = len(file.Clients)
-	case abuseIPDBChecksJSON:
-		var file abuseIPDBChecksFile
+	case abuseIPDBChecksSpentJSON:
+		var file abuseIPDBChecksSpentFile
 
 		err := parse(path, data, &file)
 		if err != nil {
@@ -618,6 +651,19 @@ func (f *Files) takeInCache(name, path string, data []byte) (int, error) {
 		f.params.DNSBL.Load(file.Verdicts)
 
 		return len(file.Verdicts), nil
+	case routeWatchJSON:
+		var file routeWatchFile
+
+		err := parse(path, data, &file)
+		if err != nil {
+			return 0, err
+		}
+
+		if f.params.RouteWatch != nil {
+			f.params.RouteWatch.Load(file.Answers)
+		}
+
+		return len(file.Answers), nil
 	default: // abuseipdb-scores.json
 		var file abuseIPDBScoresFile
 
@@ -727,7 +773,10 @@ func (f *Files) setAside(name string, parseErr error) error {
 		return errors.Join(parseErr, err)
 	}
 
-	const setAside = "set aside an edit of a state file that does not parse"
+	setAside := "set aside an edit of a state file that does not parse"
+	if isCacheFile(name) {
+		setAside = "set aside an edit of a cache file that does not parse"
+	}
 
 	// Raised before it is logged, so that the alert is there once the log
 	// line is.
@@ -760,10 +809,17 @@ func (f *Files) encode(name string) ([]byte, error) {
 		return encodeOnePerLine("verdicts", f.params.DNSBL.Snapshot())
 	case abuseIPDBScoresJSON:
 		return encodeOnePerLine("scores", f.params.AbuseIPDB.Snapshot())
-	case abuseIPDBChecksJSON:
+	case routeWatchJSON:
+		var answers []routewatch.Answer
+		if f.params.RouteWatch != nil {
+			answers = f.params.RouteWatch.Snapshot()
+		}
+
+		return encodeOnePerLine("answers", answers)
+	case abuseIPDBChecksSpentJSON:
 		checks := f.params.AbuseIPDB.ChecksSpent()
 
-		return encodeIndented(abuseIPDBChecksFile{
+		return encodeIndented(abuseIPDBChecksSpentFile{
 			Version: version, Day: checks.Day, Spent: checks.Spent,
 		})
 	default: // alerts.json
@@ -1014,9 +1070,40 @@ func (f *abuseIPDBScoresFile) check(data []byte) error {
 	return nil
 }
 
+// check refuses an answer without its client, which would answer for
+// nobody, its netblock, or the time routewatch gave it, which would drop
+// it. "" is the netblock of an address without a live route, which Answer
+// cannot tell from a missing one, so each netblock is read again as
+// written.
+func (f *routeWatchFile) check(data []byte) error {
+	var written struct {
+		Answers []struct {
+			Netblock *string `json:"netblock"`
+		} `json:"answers"`
+	}
+
+	err := json.Unmarshal(data, &written)
+	if err != nil {
+		return err
+	}
+
+	for i, answer := range f.Answers {
+		switch {
+		case !answer.Client.IsValid():
+			return missing(i, "client")
+		case written.Answers[i].Netblock == nil:
+			return missing(i, "netblock")
+		case answer.Answered.IsZero():
+			return missing(i, "answered")
+		}
+	}
+
+	return nil
+}
+
 // check refuses nothing, since a missing day or count means no check
 // spent.
-func (f *abuseIPDBChecksFile) check([]byte) error {
+func (f *abuseIPDBChecksSpentFile) check([]byte) error {
 	return nil
 }
 
@@ -1162,8 +1249,15 @@ func (f *abuseIPDBScoresFile) toUTC() {
 	}
 }
 
-// toUTC does nothing: abuseipdb-checks.json holds a day, not a time.
-func (f *abuseIPDBChecksFile) toUTC() {}
+// toUTC converts the time of each answer to UTC.
+func (f *routeWatchFile) toUTC() {
+	for i := range f.Answers {
+		convertToUTC(&f.Answers[i].Answered)
+	}
+}
+
+// toUTC does nothing: abuseipdb-checks-spent.json holds a day, not a time.
+func (f *abuseIPDBChecksSpentFile) toUTC() {}
 
 // toUTC converts every time of each cooldown, of the hour, of each alert
 // waiting and of each anomaly counter to UTC.

internal/state/state_test.go

@@ -30,22 +30,26 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/metrics"
 	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
 	"sneak.berlin/go/smallwebwaf/internal/reputation"
+	"sneak.berlin/go/smallwebwaf/internal/routewatch"
 	"sneak.berlin/go/smallwebwaf/internal/state"
 )
 
 const (
 	// The state files.
-	bansJSON            = "bans.json"
-	clientsJSON         = "clients.json"
-	abuseIPDBChecksJSON = "abuseipdb-checks.json"
-	alertsJSON          = "alerts.json"
+	bansJSON                 = "bans.json"
+	clientsJSON              = "clients.json"
+	abuseIPDBChecksSpentJSON = "abuseipdb-checks-spent.json"
+	alertsJSON               = "alerts.json"
 	// The cache files.
 	lookupsJSON         = "lookups.json"
 	listsJSON           = "lists.json"
 	dnsblJSON           = "dnsbl.json"
 	abuseIPDBScoresJSON = "abuseipdb-scores.json"
-	// What the process log says as a file that does not parse is set aside.
-	setAside = "set aside an edit of a state file that does not parse"
+	routeWatchJSON      = "routewatch.json"
+	// What the process log says as a state file, and a cache file, that does
+	// not parse is set aside.
+	setAside      = "set aside an edit of a state file that does not parse"
+	cacheSetAside = "set aside an edit of a cache file that does not parse"
 	// What Load says, after a file's path, of a first entry without its
 	// client or the time it was fetched.
 	noClient  = `: entry 1 has no "client"`
@@ -280,21 +284,34 @@ const filledAbuseIPDBScoresJSON = `{
 }
 `
 
-// filledAbuseIPDBChecksJSON is abuseipdb-checks.json holding the
-// AbuseIPDB checks spent today, as fill puts them in.
-const filledAbuseIPDBChecksJSON = `{
+// filledAbuseIPDBChecksSpentJSON is abuseipdb-checks-spent.json holding
+// the AbuseIPDB checks spent today, as fill puts them in.
+const filledAbuseIPDBChecksSpentJSON = `{
     "version": 1,
     "day": "2026-10-06",
     "spent": 3
 }
 `
 
+// filledRouteWatchJSON is routewatch.json holding the two answers of
+// routewatch fill puts in: a route, and an address without a live route.
+const filledRouteWatchJSON = `{
+    "version": 1,
+    "answers": [
+        {"client":"203.0.113.9/32","netblock":"203.0.113.0/24","asn":64496,` +
+	`"answered":"2026-10-05T23:59:00Z"},
+        {"client":"2001:db8::/64","netblock":"","asn":0,` +
+	`"answered":"2026-10-05T23:58:00Z"}
+    ]
+}
+`
+
 // everyFile returns the names of the state and cache files, sorted, as
 // wantFiles lists them in the one directory newParams gives both.
 func everyFile() []string {
 	return []string{
-		abuseIPDBChecksJSON, abuseIPDBScoresJSON, alertsJSON, bansJSON, clientsJSON,
-		dnsblJSON, listsJSON, lookupsJSON,
+		abuseIPDBChecksSpentJSON, abuseIPDBScoresJSON, alertsJSON, bansJSON, clientsJSON,
+		dnsblJSON, listsJSON, lookupsJSON, routeWatchJSON,
 	}
 }
 
@@ -332,10 +349,13 @@ func TestFilesWrittenAndReadBack(t *testing.T) {
 	wantEqual(t, dnsblJSON, after.DNSBL.Snapshot(), before.DNSBL.Snapshot())
 	wantEqual(t, abuseIPDBScoresJSON, after.AbuseIPDB.Snapshot(),
 		before.AbuseIPDB.Snapshot())
+	wantEqual(t, routeWatchJSON, after.RouteWatch.Snapshot(),
+		before.RouteWatch.Snapshot())
 
 	checks, wantChecks := after.AbuseIPDB.ChecksSpent(), before.AbuseIPDB.ChecksSpent()
 	if checks != wantChecks {
-		t.Errorf("%s read back %+v, want %+v", abuseIPDBChecksJSON, checks, wantChecks)
+		t.Errorf("%s read back %+v, want %+v", abuseIPDBChecksSpentJSON, checks,
+			wantChecks)
 	}
 
 	if got, want := after.Alerts.Snapshot(), before.Alerts.Snapshot(); !reflect.DeepEqual(
@@ -346,7 +366,7 @@ func TestFilesWrittenAndReadBack(t *testing.T) {
 	wantEqual(t, alertsJSON, after.Anomalies.Snapshot(), before.Anomalies.Snapshot())
 
 	// Each one-per-line file lists its entries by client, and nothing
-	// but the eight files is left in the directory.
+	// but the nine files is left in the directory.
 	wantEntries(t, filepath.Join(dir, clientsJSON), "clients",
 		"192.0.2.1/32", "203.0.113.9/32", "2001:db8::/64")
 	wantEntries(t, filepath.Join(dir, lookupsJSON), "lookups",
@@ -367,8 +387,9 @@ func TestStateFilesInDirAndCacheFilesInCacheDir(t *testing.T) {
 		t.Fatalf("write: %v", err)
 	}
 
-	wantFiles(t, stateDir, abuseIPDBChecksJSON, alertsJSON, bansJSON, clientsJSON)
-	wantFiles(t, cacheDir, abuseIPDBScoresJSON, dnsblJSON, listsJSON, lookupsJSON)
+	wantFiles(t, stateDir, abuseIPDBChecksSpentJSON, alertsJSON, bansJSON, clientsJSON)
+	wantFiles(t, cacheDir, abuseIPDBScoresJSON, dnsblJSON, listsJSON, lookupsJSON,
+		routeWatchJSON)
 
 	// Read back from both, as at the next start.
 	after := newParams(stateDir)
@@ -380,7 +401,8 @@ func TestStateFilesInDirAndCacheFilesInCacheDir(t *testing.T) {
 
 	checks, wantChecks := after.AbuseIPDB.ChecksSpent(), before.AbuseIPDB.ChecksSpent()
 	if checks != wantChecks {
-		t.Errorf("%s read back %+v, want %+v", abuseIPDBChecksJSON, checks, wantChecks)
+		t.Errorf("%s read back %+v, want %+v", abuseIPDBChecksSpentJSON, checks,
+			wantChecks)
 	}
 }
 
@@ -400,10 +422,10 @@ func TestReputationFilesAsWritten(t *testing.T) {
 
 	// lists.json keeps each copy whole, one line to a line.
 	for name, want := range map[string]string{
-		listsJSON:           filledListsJSON,
-		dnsblJSON:           filledDNSBLJSON,
-		abuseIPDBScoresJSON: filledAbuseIPDBScoresJSON,
-		abuseIPDBChecksJSON: filledAbuseIPDBChecksJSON,
+		listsJSON:                filledListsJSON,
+		dnsblJSON:                filledDNSBLJSON,
+		abuseIPDBScoresJSON:      filledAbuseIPDBScoresJSON,
+		abuseIPDBChecksSpentJSON: filledAbuseIPDBChecksSpentJSON,
 	} {
 		if got := readFile(t, filepath.Join(dir, name)); got != want {
 			t.Errorf("%s\n%s\nwant\n%s", name, got, want)
@@ -411,6 +433,41 @@ func TestReputationFilesAsWritten(t *testing.T) {
 	}
 }
 
+func TestRouteWatchJSONAsWritten(t *testing.T) {
+	t.Parallel()
+
+	dir := t.TempDir()
+	params := newParams(dir)
+	fill(params)
+
+	err := load(t, params).WriteAll()
+	if err != nil {
+		t.Fatalf("write: %v", err)
+	}
+
+	got := readFile(t, filepath.Join(dir, routeWatchJSON))
+	if got != filledRouteWatchJSON {
+		t.Errorf("%s\n%s\nwant\n%s", routeWatchJSON, got, filledRouteWatchJSON)
+	}
+}
+
+func TestRouteWatchJSONHoldsNoAnswersWhileRoutewatchIsUnset(t *testing.T) {
+	t.Parallel()
+
+	dir := t.TempDir()
+	edit(t, dir, routeWatchJSON, filledRouteWatchJSON)
+
+	params := newParams(dir)
+	params.RouteWatch = nil
+
+	err := load(t, params).WriteAll()
+	if err != nil {
+		t.Fatalf("write: %v", err)
+	}
+
+	wantEntries(t, filepath.Join(dir, routeWatchJSON), "answers")
+}
+
 func TestAlertsJSONIsIndentedWithTheCooldownsTheHourAndTheAlertsWaiting(t *testing.T) {
 	t.Parallel()
 
@@ -536,6 +593,7 @@ func TestMissingFilesAreEmptyState(t *testing.T) {
 	if len(params.Ledger.Snapshot()) != 0 || len(params.Limiter.Snapshot()) != 0 ||
 		len(params.GeoJS.Snapshot()) != 0 || len(params.Lists.Snapshot()) != 0 ||
 		len(params.DNSBL.Snapshot()) != 0 || len(params.AbuseIPDB.Snapshot()) != 0 ||
+		len(params.RouteWatch.Snapshot()) != 0 ||
 		params.AbuseIPDB.ChecksSpent().Spent != 0 ||
 		len(held.Cooldowns) != 0 || len(held.Waiting[alerts.DestinationWebhook]) != 0 ||
 		held.Hour.Sent != 0 {
@@ -571,7 +629,7 @@ func TestFileThatDoesNotParseStopsTheStart(t *testing.T) {
 			", line 3, column 19: unexpected end of JSON input",
 		},
 		{
-			"an unknown field", abuseIPDBChecksJSON,
+			"an unknown field", abuseIPDBChecksSpentJSON,
 			`{"version": 1, "day": "2026-10-06", "spnet": 3}`,
 			`: json: unknown field "spnet"`,
 		},
@@ -642,6 +700,25 @@ func TestCacheFileThatDoesNotParseIsSetAsideAndTheCacheStartsEmpty(t *testing.T)
 			"an unknown version", abuseIPDBScoresJSON, `{"version": 2}`,
 			": unknown version ",
 		},
+		{
+			"an answer without its client", routeWatchJSON,
+			`{"version": 1, "answers": [{"netblock": "", ` +
+				`"answered": "2026-10-06T00:00:00Z"}]}`,
+			noClient,
+		},
+		{
+			// A netblock of "" is an address without a live route.
+			"an answer without its netblock", routeWatchJSON,
+			`{"version": 1, "answers": [{"client": "192.0.2.1/32", "netblock": "", ` +
+				`"answered": "2026-10-06T00:00:00Z"}, {"client": "203.0.113.9/32", ` +
+				`"answered": "2026-10-06T00:00:00Z"}]}`,
+			`: entry 2 has no "netblock"`,
+		},
+		{
+			"an answer without the time routewatch gave it", routeWatchJSON,
+			`{"version": 1, "answers": [{"client": "203.0.113.9/32", "netblock": ""}]}`,
+			`: entry 1 has no "answered"`,
+		},
 	} {
 		t.Run(tc.name, func(t *testing.T) {
 			t.Parallel()
@@ -1055,7 +1132,9 @@ func TestBanThatDoesNotEndAfterItsStartStopsTheStart(t *testing.T) {
 func TestUnknownVersionStopsTheStart(t *testing.T) {
 	t.Parallel()
 
-	for _, file := range []string{bansJSON, clientsJSON, abuseIPDBChecksJSON, alertsJSON} {
+	for _, file := range []string{
+		bansJSON, clientsJSON, abuseIPDBChecksSpentJSON, alertsJSON,
+	} {
 		for _, content := range []string{`{"version": 2}`, `{}`} {
 			t.Run(file+" "+content, func(t *testing.T) {
 				t.Parallel()
@@ -1221,43 +1300,52 @@ func TestWriteThatFailsWhileRunningRaisesAFileErrorAlertOncePerCooldown(t *testi
 		params.CounterInterval = time.Minute
 		run(t, load(t, params).Run)
 
-		// A directory in the way of clients.json's temporary file fails each
-		// of its writes, while the other files are written. It holds a file,
-		// so that the write cannot remove it.
-		err := os.Mkdir(filepath.Join(dir, clientsJSON+".tmp"), 0o700)
-		if err == nil {
-			err = os.WriteFile(filepath.Join(dir, clientsJSON+".tmp", "kept"), nil, 0o600)
+		// A directory in the way of the temporary file of clients.json, a
+		// state file, and of lookups.json, a cache file, fails each of their
+		// writes, while the other files are written. Each holds a file, so
+		// that the write cannot remove it.
+		failing := []struct{ name, reason string }{
+			{clientsJSON, "writing the state files failed"},
+			{lookupsJSON, "writing the cache files failed"},
 		}
 
-		if err != nil {
-			t.Fatalf("put a directory in the way: %v", err)
+		for _, file := range failing {
+			err := os.Mkdir(filepath.Join(dir, file.name+".tmp"), 0o700)
+			if err == nil {
+				err = os.WriteFile(filepath.Join(dir, file.name+".tmp", "kept"), nil, 0o600)
+			}
+
+			if err != nil {
+				t.Fatalf("put a directory in the way: %v", err)
+			}
 		}
 
 		time.Sleep(time.Minute)
 		synctest.Wait()
 
 		waiting := params.Alerts.Snapshot().Waiting[alerts.DestinationWebhook]
-		if len(waiting) != 1 {
-			t.Fatalf("%d alerts wait, want 1", len(waiting))
+		if len(waiting) != len(failing) {
+			t.Fatalf("%d alerts wait, want %d", len(waiting), len(failing))
 		}
 
-		message, _ := waiting[0].Detail["error"].(string)
+		for i, file := range failing {
+			message, _ := waiting[i].Detail["error"].(string)
 
-		if waiting[0].Event != alerts.EventFileError ||
-			waiting[0].Reason != "writing the state files failed" ||
-			waiting[0].Detail["file"] != filepath.Join(dir, clientsJSON) ||
-			!strings.Contains(message, clientsJSON+".tmp") {
-			t.Fatalf("alerts waiting %+v, want a file_error alert for clients.json, "+
-				"naming its temporary file", waiting)
+			if waiting[i].Event != alerts.EventFileError || waiting[i].Reason != file.reason ||
+				waiting[i].Detail["file"] != filepath.Join(dir, file.name) ||
+				!strings.Contains(message, file.name+".tmp") {
+				t.Fatalf("alerts waiting %+v, want a file_error alert for %s, saying %q "+
+					"and naming its temporary file", waiting, file.name, file.reason)
+			}
 		}
 
-		// The next write fails too, within the cooldown, which holds it back.
+		// The next writes fail too, within the cooldown, which holds them back.
 		time.Sleep(time.Minute)
 		synctest.Wait()
 
 		waiting = params.Alerts.Snapshot().Waiting[alerts.DestinationWebhook]
-		if len(waiting) != 1 || params.Alerts.Suppressed() != 1 {
-			t.Errorf("%d alerts wait and %d are held back, want 1 and 1",
+		if len(waiting) != 2 || params.Alerts.Suppressed() != 2 {
+			t.Errorf("%d alerts wait and %d are held back, want 2 and 2",
 				len(waiting), params.Alerts.Suppressed())
 		}
 	})
@@ -1461,6 +1549,15 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
 	wantEqual(t, lookupsJSON, params.GeoJS.Snapshot(),
 		[]lookup.Answer{{Client: client, Country: "FR", Answered: midnight()}})
 
+	edit(t, dir, routeWatchJSON, `{"version": 1, "answers": [`+
+		`{"client": "198.51.100.7/32", "netblock": "198.51.100.0/24", "asn": 64497, `+
+		`"answered": "2026-10-06T00:00:00Z"}]}`)
+	wantTakenIn(t, lines, dir, routeWatchJSON)
+	wantEqual(t, routeWatchJSON, params.RouteWatch.Snapshot(), []routewatch.Answer{{
+		Client: client, Netblock: netip.MustParsePrefix("198.51.100.0/24"), ASN: 64497,
+		Answered: midnight(),
+	}})
+
 	// A netblock with bits past its length is read as the netblock it is
 	// in.
 	edit(t, dir, alertsJSON, `{"version": 1, "cooldowns": [{"event": "ban", `+
@@ -1527,12 +1624,13 @@ func TestEditOfEachReputationFileTakenIn(t *testing.T) {
 	wantEqual(t, abuseIPDBScoresJSON, params.AbuseIPDB.Snapshot(),
 		[]reputation.Score{{Client: client, Score: 80, Fetched: midnight()}})
 
-	edit(t, dir, abuseIPDBChecksJSON, `{"version": 1, "day": "2026-10-06", "spent": 9}`)
-	wantTakenIn(t, lines, dir, abuseIPDBChecksJSON)
+	edit(t, dir, abuseIPDBChecksSpentJSON,
+		`{"version": 1, "day": "2026-10-06", "spent": 9}`)
+	wantTakenIn(t, lines, dir, abuseIPDBChecksSpentJSON)
 
 	checks := reputation.ChecksSpent{Day: civil.DateOf(midnight()), Spent: 9}
 	if got := params.AbuseIPDB.ChecksSpent(); got != checks {
-		t.Errorf("%s taken in as %+v, want %+v", abuseIPDBChecksJSON, got, checks)
+		t.Errorf("%s taken in as %+v, want %+v", abuseIPDBChecksSpentJSON, got, checks)
 	}
 }
 
@@ -1558,10 +1656,10 @@ func TestTimesEditedWithAnOffsetWrittenBackInUTC(t *testing.T) {
 	ahead := time.FixedZone("UTC+2", 2*60*60)
 	written := map[string]string{}
 
-	// abuseipdb-checks.json holds a day, but no time.
+	// abuseipdb-checks-spent.json holds a day, but no time.
 	for _, name := range []string{
 		bansJSON, clientsJSON, alertsJSON, lookupsJSON, listsJSON, dnsblJSON,
-		abuseIPDBScoresJSON,
+		abuseIPDBScoresJSON, routeWatchJSON,
 	} {
 		written[name] = readFile(t, filepath.Join(dir, name))
 		edited := stamp.ReplaceAllStringFunc(written[name], func(quoted string) string {
@@ -1823,8 +1921,8 @@ func TestBrokenEditSetAsideAtTheNextWrite(t *testing.T) {
 		t.Errorf("alerts waiting %+v, want a file_error alert for %s", waiting, path+".bad")
 	}
 
-	wantFiles(t, dir, abuseIPDBChecksJSON, abuseIPDBScoresJSON, alertsJSON, bansJSON,
-		bansJSON+".bad", clientsJSON, dnsblJSON, listsJSON, lookupsJSON)
+	wantFiles(t, dir, abuseIPDBChecksSpentJSON, abuseIPDBScoresJSON, alertsJSON, bansJSON,
+		bansJSON+".bad", clientsJSON, dnsblJSON, listsJSON, lookupsJSON, routeWatchJSON)
 
 	if got := readFile(t, path+".bad"); got != broken {
 		t.Errorf("bans.json.bad holds\n%s\nwant the edit", got)
@@ -1947,16 +2045,73 @@ func TestDirectoryThatCannotBeWatchedIsLogged(t *testing.T) {
 	}
 }
 
+func TestCacheFileEditedInACacheDirectoryApartTakenIn(t *testing.T) {
+	t.Parallel()
+
+	stateDir, cacheDir := t.TempDir(), t.TempDir()
+	params := newParams(stateDir)
+	params.CacheDir = cacheDir
+	lines := logInto(&params)
+	watch(t, load(t, params), lines)
+
+	edit(t, cacheDir, lookupsJSON, `{"version": 1, "lookups": []}`)
+	wantTakenIn(t, lines, cacheDir, lookupsJSON)
+}
+
+func TestDirectoryThatCannotBeWatchedLeavesTheOtherWatched(t *testing.T) {
+	t.Parallel()
+
+	for _, tc := range []struct {
+		// gone is the directory removed before Watch runs, and failed what
+		// the log says of it. edited is a file in the other directory.
+		gone, failed, edited string
+	}{
+		{"state", "cannot watch the state files for edits", lookupsJSON},
+		{"cache", "cannot watch the cache files for edits", bansJSON},
+	} {
+		t.Run(tc.gone, func(t *testing.T) {
+			t.Parallel()
+
+			stateDir, cacheDir := t.TempDir(), t.TempDir()
+			params := newParams(stateDir)
+			params.CacheDir = cacheDir
+			lines := logInto(&params)
+			files := load(t, params)
+
+			gone, other := stateDir, cacheDir
+			if tc.gone == "cache" {
+				gone, other = cacheDir, stateDir
+			}
+
+			err := os.Remove(gone)
+			if err != nil {
+				t.Fatalf("remove: %v", err)
+			}
+
+			run(t, files.Watch)
+
+			line := lines.waitFor(t, tc.failed)
+			if line["level"] != "ERROR" || line["directory"] != gone {
+				t.Errorf("logged %v, want an error naming %s", line, gone)
+			}
+
+			lines.waitFor(t, watching)
+			edit(t, other, tc.edited, `{"version": 1}`)
+			wantTakenIn(t, lines, other, tc.edited)
+		})
+	}
+}
+
 // midnight is the time of the tests' clock.
 func midnight() time.Time {
 	return time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)
 }
 
 // newParams returns Params for the state and cache files, both in dir,
-// with parts that hold nothing yet. GeoJS is never asked, the lists, two
-// blocklists, are never fetched, and the alerts, at most two an hour, are
-// never sent. The anomaly counters count the scopes fill counts, with
-// thresholds fill does not reach.
+// with parts that hold nothing yet. GeoJS and routewatch are never asked,
+// the lists, two blocklists, are never fetched, and the alerts, at most two
+// an hour, are never sent. The anomaly counters count the scopes fill
+// counts, with thresholds fill does not reach.
 func newParams(dir string) state.Params {
 	discard := slog.New(slog.DiscardHandler)
 	m := metrics.New(1, "app")
@@ -1997,6 +2152,9 @@ func newParams(dir string) state.Params {
 			MinScore: 75, DailyBudget: 900, CacheTTL: 24 * time.Hour, Timeout: time.Second,
 			Now: midnight, ProcessLog: discard, Alerts: queue,
 		}),
+		RouteWatch: routewatch.New(routewatch.Params{
+			URL: "http://routewatch.example/", Now: midnight, ProcessLog: discard,
+		}),
 		Alerts: queue,
 		Anomalies: anomaly.New(anomaly.Params{
 			Net:            anomaly.Thresholds{RequestsPerMinute: 1000},
@@ -2023,10 +2181,11 @@ func office() netip.Prefix {
 // for a clear sign of attack and one for CrowdSec's decision, clients
 // with counts and histories,
 // GeoJS answers, the blocklists' last tries and the copy of one, two
-// verdicts of a DNSBL zone, and the AbuseIPDB checks spent today with two
-// scores, as filledListsJSON, filledDNSBLJSON, filledAbuseIPDBScoresJSON
-// and filledAbuseIPDBChecksJSON hold them, and alerts and anomaly
-// counters, as filledAlertsJSON holds them, into the parts of params.
+// verdicts of a DNSBL zone, the AbuseIPDB checks spent today with two
+// scores, and two answers of routewatch, as filledListsJSON,
+// filledDNSBLJSON, filledAbuseIPDBScoresJSON, filledAbuseIPDBChecksSpentJSON
+// and filledRouteWatchJSON hold them, and alerts and anomaly counters, as
+// filledAlertsJSON holds them, into the parts of params.
 func fill(params state.Params) {
 	now := midnight()
 	client := netip.MustParsePrefix("203.0.113.9/32")
@@ -2089,6 +2248,7 @@ func fill(params state.Params) {
 	params.AbuseIPDB.LoadChecksSpent(reputation.ChecksSpent{
 		Day: civil.DateOf(now), Spent: 3,
 	})
+	fillRouteWatch(params.RouteWatch)
 
 	// An alert waiting, a repeat of it the cooldown holds back, another
 	// alert waiting, and one past the two an hour, for the hour's summary.
@@ -2114,6 +2274,22 @@ func fill(params state.Params) {
 	})
 }
 
+// fillRouteWatch puts the two answers filledRouteWatchJSON holds into
+// routeWatch: a route, and an address without a live route.
+func fillRouteWatch(routeWatch *routewatch.Client) {
+	routeWatch.Load([]routewatch.Answer{
+		{
+			Client:   netip.MustParsePrefix("203.0.113.9/32"),
+			Netblock: netip.MustParsePrefix("203.0.113.0/24"), ASN: 64496,
+			Answered: midnight().Add(-time.Minute),
+		},
+		{
+			Client:   netip.MustParsePrefix("2001:db8::/64"),
+			Answered: midnight().Add(-2 * time.Minute),
+		},
+	})
+}
+
 // permanentBan is the ban permanentBansJSON holds.
 func permanentBan() bans.Ban {
 	score := int64(100)
@@ -2350,8 +2526,8 @@ func wantRefused(t *testing.T, file, content, want string) {
 
 // wantSetAside writes content to the cache file named file in a new
 // directory, and checks that Load starts all the same, with the cache
-// empty, and sets the file aside, logging an error that is the file's
-// path and then starts with want.
+// empty, and sets the file aside, raising a file_error alert and logging
+// an error that is the file's path and then starts with want.
 func wantSetAside(t *testing.T, file, content, want string) {
 	t.Helper()
 
@@ -2367,15 +2543,23 @@ func wantSetAside(t *testing.T, file, content, want string) {
 	lines := logInto(&params)
 	load(t, params)
 
-	line := lines.waitFor(t, setAside)
+	line := lines.waitFor(t, cacheSetAside)
 	if message, _ := line["error"].(string); line["file"] != path+".bad" ||
 		!strings.HasPrefix(message, path+want) {
 		t.Errorf("set aside with %v, want %s.bad and an error starting %s%s", line, path,
 			path, want)
 	}
 
+	waiting := params.Alerts.Snapshot().Waiting[alerts.DestinationWebhook]
+	if len(waiting) != 1 || waiting[0].Event != alerts.EventFileError ||
+		waiting[0].Reason != cacheSetAside {
+		t.Errorf("alerts waiting %+v, want a file_error alert saying %q", waiting,
+			cacheSetAside)
+	}
+
 	if len(params.GeoJS.Snapshot()) != 0 || len(params.Lists.Snapshot()) != 0 ||
-		len(params.DNSBL.Snapshot()) != 0 || len(params.AbuseIPDB.Snapshot()) != 0 {
+		len(params.DNSBL.Snapshot()) != 0 || len(params.AbuseIPDB.Snapshot()) != 0 ||
+		len(params.RouteWatch.Snapshot()) != 0 {
 		t.Error("a cache holds what a file set aside held")
 	}
 

internal/sww/sww.go

@@ -253,6 +253,7 @@ func loadStateFiles(
 		Lists:           server.Lists,
 		DNSBL:           server.DNSBL,
 		AbuseIPDB:       server.AbuseIPDB,
+		RouteWatch:      server.RouteWatch,
 		Alerts:          alertQueue,
 		Anomalies:       server.Anomalies,
 		Now:             now,

internal/waf/answertime_test.go

@@ -7,6 +7,7 @@ import (
 	"net/http"
 	"net/http/httptest"
 	"net/netip"
+	"path/filepath"
 	"slices"
 	"strconv"
 	"strings"
@@ -16,6 +17,7 @@ import (
 
 	"sneak.berlin/go/smallwebwaf/internal/bans"
 	"sneak.berlin/go/smallwebwaf/internal/lookup"
+	"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
 	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
 	"sneak.berlin/go/smallwebwaf/internal/reputation"
 	"sneak.berlin/go/smallwebwaf/internal/requestlog"
@@ -216,93 +218,199 @@ func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
 	}
 }
 
-func TestRequestsOverTheHeaderLimitsCountTowardTheRateLimits(t *testing.T) {
+func TestTheRateLimitsDoNotCountRequestsOverTheHeaderLimits(t *testing.T) {
 	t.Parallel()
 
 	s := startWAF(t, "", map[string]string{rateLimitPerMinute: "1"})
-	ask := func() answer {
+
+	for range 2 {
 		req := s.question(siteA, client, "/")
 		req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
-		return do(t, req)
+		wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
 	}
 
-	wantDisposition(t, ask(), tooLarge, waf.DispositionDeny)
-	wantDisposition(t, ask(), http.StatusForbidden, waf.DispositionDeny)
-
-	lines := s.out.requestLines(t, 2)
-	wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionClientIsRateLimited)
+	wantDisposition(t, s.ask(client, "/"), http.StatusOK, waf.DispositionAllow)
 }
 
-func TestTheHeaderLimitsComeAfterTheTrapPathsAndTheRuleFiles(t *testing.T) {
+func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
 	t.Parallel()
 
+	// The lookup database places client in Germany.
+	lookupDB := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
+	lookuptest.Write(t, lookupDB, map[string]lookuptest.Network{
+		client + "/32": {ASN: asnDE, ASName: asNameDE, Country: "DE"},
+	})
+
 	for _, tc := range []struct {
-		name, path, action string
+		name string
+		env  map[string]string
+		path string
+		// prepare gets the server ready, at now, before the question.
+		prepare func(t *testing.T, server *waf.Server, now time.Time)
 	}{
-		{"a trap path", "/wp-login.php", requestlog.ActionTemporarilyBanned},
-		{"a block rule", "/blocked", requestlog.ActionBlockedByBlockRule},
+		{"SWW_DENY_NETS", map[string]string{denyNets: client}, "/", leaveAsItIs},
+		{"a ban", nil, "/", banClient},
+		{
+			"the country list", map[string]string{
+				lookupSource: fileSource, lookupDBPath: lookupDB, allowedCountries: "FR",
+			},
+			"/", leaveAsItIs,
+		},
+		{"a blocklist", map[string]string{blocklistURLs: dropURL}, "/", listClient},
+		{
+			"the CrowdSec decision list",
+			map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey}, "/", decideOnClient,
+		},
+		{
+			"a DNS blocklist", map[string]string{
+				dnsblZones: dnsblZone, dnsblResolver: noResolver, reputationAction: actionDeny,
+			},
+			"/", func(_ *testing.T, server *waf.Server, _ time.Time) {
+				loadVerdicts(server, map[string][]string{client: {dnsblZone}})
+			},
+		},
+		{
+			"AbuseIPDB",
+			map[string]string{abuseIPDBKey: accountKey, reputationAction: actionDeny},
+			"/", func(_ *testing.T, server *waf.Server, _ time.Time) {
+				// Over SWW_ABUSEIPDB_MIN_SCORE, 75 by default.
+				loadScores(server, map[string]int64{client: 80})
+			},
+		},
+		{
+			"SWW_TAG_DENY", map[string]string{tagLists: "vpn=" + vpnList, tagDeny: "vpn"},
+			"/", func(t *testing.T, server *waf.Server, _ time.Time) {
+				t.Helper()
+				loadLists(t, server, map[string][]string{vpnList: {client}})
+			},
+		},
+		{"the rate limits", map[string]string{rateLimitPerMinute: "1"}, "/", countClient},
+		{
+			"a trap path", map[string]string{trapPaths: trapPathList}, "/wp-login.php",
+			leaveAsItIs,
+		},
+		{
+			"a block rule", map[string]string{rulesDir: writeRules(t, testRules)}, "/blocked",
+			leaveAsItIs,
+		},
 	} {
 		t.Run(tc.name, func(t *testing.T) {
 			t.Parallel()
 
-			s := startWAF(t, "", map[string]string{
-				trapPaths: trapPathList, rulesDir: writeRules(t, testRules),
-			})
+			s := startWAF(t, "", tc.env)
+			tc.prepare(t, s.server, s.clock.Now())
+
 			req := s.question(siteA, client, tc.path)
 			req.Header.Set(cookieHeader, cookies(cookieLimit+1))
 
-			wantDisposition(t, do(t, req), http.StatusForbidden, waf.DispositionDeny)
-			wantLine(t, s.out.requestLine(t), http.StatusForbidden, tc.action)
+			wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
+			wantLine(t, s.out.requestLine(t), tooLarge, requestlog.ActionRequestHeaderTooLarge)
 		})
 	}
 }
 
+func TestARequestOverTheHeaderLimitsRunsNoOtherCheck(t *testing.T) {
+	t.Parallel()
+
+	// Each check after the header limits leaves a mark on the log line of a
+	// request it reads: a tag, routewatch_tags_missing, a rule id, the
+	// counts, or the Core Rule Set's time.
+	path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
+	lookuptest.Write(t, path, map[string]lookuptest.Network{
+		client + "/32": {ASN: asnDE, ASName: asNameDE, Country: "DE"},
+	})
+	s := startWAF(t, "", map[string]string{
+		lookupSource: fileSource, lookupDBPath: path,
+		tagLists:      "vpn=" + vpnList,
+		routeWatchURL: startRouteWatch(t, map[string]standInRoute{client: {prefixDE, 64496}}),
+		dnsblZones:    dnsblZone, reputationAction: actionLog,
+		rulesDir: writeRules(t, "noted path log ^/\n"),
+		wafMode:  block,
+	})
+	loadLists(t, s.server, map[string][]string{vpnList: {client}})
+	loadVerdicts(s.server, map[string][]string{client: {dnsblZone}})
+
+	over := s.question(siteA, client, "/")
+	over.Header.Set(cookieHeader, cookies(cookieLimit+1))
+	wantDisposition(t, do(t, over), tooLarge, waf.DispositionDeny)
+
+	within := s.question(siteA, client, "/")
+	within.Header.Set(cookieHeader, cookies(cookieLimit))
+	wantDisposition(t, do(t, within), http.StatusOK, waf.DispositionAllow)
+
+	lines := s.out.requestLines(t, 2)
+
+	wantTags(t, lines[0])
+	wantTagsMissing(t, lines[0], "")
+
+	if lines[0].RuleIDs != nil || lines[0].Counts != (ratelimit.Counts{}) ||
+		lines[0].DurationCoreRuleSetMS != nil {
+		t.Errorf("rule ids %v, counts %+v and Core Rule Set time %v, want none",
+			lines[0].RuleIDs, lines[0].Counts, lines[0].DurationCoreRuleSetMS)
+	}
+
+	// The request within the limits shows that each check leaves its mark.
+	wantTags(t, lines[1],
+		"asn=64496", "country=DE", "dnsbl="+dnsblZone, "exploitsignature", "vpn")
+	wantTagsMissing(t, lines[1], "no_answer_yet")
+
+	if lines[1].RuleIDs == nil || lines[1].Counts.Minute != 1 ||
+		lines[1].DurationCoreRuleSetMS == nil {
+		t.Errorf("rule ids %v, counts %+v and Core Rule Set time %v, want each",
+			lines[1].RuleIDs, lines[1].Counts, lines[1].DurationCoreRuleSetMS)
+	}
+}
+
 func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
 	t.Parallel()
 
+	// The client's next request takes it over the whole limit, so it does
+	// not wait.
 	geojsURL, _ := startGeoJS(t)
 	s := startWAF(t, geojsURL, map[string]string{
-		unknownLimitPercent: "0", lookupTimeout: "1h",
+		unknownLimitPercent: "0", lookupTimeout: "1h", rateLimitPerMinute: "1",
 	})
-	req := s.question(siteA, fromDE, "/")
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
+	countClient(t, s.server, s.clock.Now())
 
-	wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
+	wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, waf.DispositionDeny)
 
 	line := s.out.requestLine(t)
-	wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
+	wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
 
 	if line.RateLimitPercent != nil {
 		t.Errorf("limit percentage %d, want none", *line.RateLimitPercent)
 	}
-
-	// The client is under no ban, and GeoJS places it in a country.
-	wantDisposition(t, s.ask(fromDE, "/"), http.StatusOK, waf.DispositionAllow)
 }
 
 func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) {
 	t.Parallel()
 
+	// The client's next request takes it over the whole limit, so it does
+	// not wait.
 	geojsURL, _ := startGeoJS(t)
-	s := startWAF(t, geojsURL, map[string]string{unknownLimitPercent: "0"})
+	s := startWAF(t, geojsURL, map[string]string{
+		unknownLimitPercent: "0", rateLimitPerMinute: "1",
+	})
 	s.server.GeoJS.Load([]lookup.Answer{{
 		Client: netip.MustParsePrefix(client + "/32"), ASN: "AS64512",
 		Answered: s.clock.Now(), Used: s.clock.Now(),
 	}})
+	countClient(t, s.server, s.clock.Now())
 
-	req := s.question(siteA, client, "/")
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
+	wantDisposition(t, s.ask(client, "/"), http.StatusForbidden, waf.DispositionDeny)
 
-	wantDisposition(t, do(t, req), http.StatusForbidden, waf.DispositionDeny)
-	wantLine(t, s.out.requestLine(t), http.StatusForbidden,
-		requestlog.ActionClientIsRateLimited)
+	line := s.out.requestLine(t)
+	wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
+
+	if line.RateLimitPercent == nil || *line.RateLimitPercent != 0 {
+		t.Errorf("limit percentage %v, want 0", line.RateLimitPercent)
+	}
 }
 
-func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
+func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits(t *testing.T) {
 	t.Parallel()
 
-	geojsURL, _ := startGeoJS(t)
+	geojsURL, asked := startGeoJS(t)
 	s := startWAF(t, geojsURL, map[string]string{
 		mode: observe, addLookupHeaders: on, lookupTimeout: "1h",
 	})
@@ -312,10 +420,9 @@ func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
 	got := do(t, req)
 	wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
 
-	asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
-	if asn != asnDE || country != "DE" {
-		t.Errorf("answered with the AS number %q and the country %q, want %q and DE",
-			asn, country, asnDE)
+	if len(asked()) != 0 || got.header.Get("X-Client-Asn") != "" {
+		t.Errorf("GeoJS was asked about %v, and the answer gave the AS number %q, "+
+			"want neither", asked(), got.header.Get("X-Client-Asn"))
 	}
 }
 

internal/waf/crowdsec_test.go

@@ -16,7 +16,7 @@ import (
 
 // The CrowdSec settings, and the tests' engine, which is never asked: each
 // test puts in the copy of its decision list, at decisionsURL, that it
-// needs, as reputation.json would at start.
+// needs, as lists.json would at start.
 const (
 	crowdSecURL  = "SWW_CROWDSEC_LAPI_URL"
 	crowdSecKey  = "SWW_CROWDSEC_LAPI_KEY"
@@ -182,7 +182,7 @@ func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
 
 // loadDecisions puts into server's lists the copy of the decision list at
 // decisionsURL, answer, the engine's answer, fetched at fetched, as
-// reputation.json would at start.
+// lists.json would at start.
 func loadDecisions(
 	t *testing.T, server *waf.Server, fetched time.Time, answer string,
 ) {

internal/waf/headerlimits.go

@@ -10,22 +10,23 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/requestlog"
 )
 
-// The header limits: the most cookies and header fields a request may have
-// when the checks reach the Core Rule Set. Real clients never reach them.
+// The header limits: the most cookies and header fields a request may have.
+// Real clients never reach them.
 const (
 	maxCookies      = 50
 	maxHeaderFields = 100
 )
 
-// headerLimits refuses a request over the header limits, with 431. Its figures
-// are the request's header fields and cookies, and the limit of each.
+// headerLimits refuses a request with more than maxHeaderFields header fields
+// or more than maxCookies cookies, with 431. Its figures are the request's
+// header fields and cookies, and the limit of each.
 type headerLimits struct{}
 
 func (headerLimits) name() string { return "header_limits" }
 
 func (headerLimits) check(_ context.Context, rq *request, _ time.Time) *verdict {
 	fields, cookies := rq.headerCounts()
-	if !overHeaderLimits(fields, cookies) {
+	if fields <= maxHeaderFields && cookies <= maxCookies {
 		return nil
 	}
 
@@ -38,13 +39,6 @@ func (headerLimits) check(_ context.Context, rq *request, _ time.Time) *verdict
 	}
 }
 
-// overHeaderLimits reports whether a request with these counts of header fields
-// and cookies, as headerCounts counts them, is over the header limits: it has
-// more than maxHeaderFields fields or more than maxCookies cookies.
-func overHeaderLimits(fields, cookies int) bool {
-	return fields > maxHeaderFields || cookies > maxCookies
-}
-
 // headerCounts returns the request's header fields, each value of a header sent
 // twice being one, and its cookies, each part of a Cookie header between
 // semicolons being one unless it is empty or only spaces and tabs. The Core

internal/waf/heuristics.go

@@ -41,8 +41,11 @@ const (
 
 // beforeLookup are the heuristics that run before the client's AS number and
 // country are looked up, so that a client they refuse is not looked up.
+// header_limits comes first: it only counts, and a request it refuses causes no
+// other work.
 func beforeLookup() []heuristic {
 	return []heuristic{
+		headerLimits{},
 		tagLists{},
 		routeWatchTags{},
 		denyNets{},
@@ -64,7 +67,6 @@ func afterLookup() []heuristic {
 		rateLimits{},
 		trapPaths{},
 		ruleFiles{},
-		headerLimits{},
 		coreRuleSet{},
 		tagDeny{},
 	}

internal/waf/proxy.go

@@ -86,8 +86,8 @@ type Params struct {
 // Server is the server waf runs, with the parts whose state the state
 // files keep, the lookup database, nil unless SWW_LOOKUP_SOURCE is file, the
 // lists fetched from URLs, which its Run fetches, the DNSBL zones' verdicts,
-// AbuseIPDB's scores and checks spent, and the metrics. RouteWatch, nil while
-// SWW_ROUTEWATCH_URL is unset, keeps its answers in memory only.
+// AbuseIPDB's scores and checks spent, routewatch's answers, and the metrics.
+// RouteWatch is nil while SWW_ROUTEWATCH_URL is unset.
 type Server struct {
 	*http.Server
 

internal/waf/reputation_test.go

@@ -39,7 +39,7 @@ const (
 )
 
 // The lists these tests name, which are never fetched: each test puts in
-// the copies it needs, as reputation.json would at start.
+// the copies it needs, as lists.json would at start.
 const (
 	dropURL = "https://lists.example/drop.txt"
 	torURL  = "https://lists.example/tor.txt"
@@ -374,7 +374,7 @@ const (
 
 // The DNSBL zones these tests name, which are never asked about the
 // clients the tests send requests from: each test puts in the verdicts it
-// needs, as reputation.json would at start. A query a test does start is
+// needs, as dnsbl.json would at start. A query a test does start is
 // sent to noResolver, where nothing listens, so that none leaves the host.
 const (
 	dnsblZone  = "dnsbl.example"
@@ -1072,7 +1072,7 @@ func verdictsFetched() time.Time {
 
 // loadVerdicts puts into server's DNSBL, for each client listedBy names,
 // a verdict of each zone SWW_DNSBL_ZONES names, fetched at
-// verdictsFetched, as reputation.json would at start: one that lists the
+// verdictsFetched, as dnsbl.json would at start: one that lists the
 // client from each zone listedBy gives for it, and one that does not from
 // each other zone.
 func loadVerdicts(server *waf.Server, listedBy map[string][]string) {
@@ -1091,8 +1091,8 @@ func loadVerdicts(server *waf.Server, listedBy map[string][]string) {
 }
 
 // loadScores puts into server's AbuseIPDB the score scores gives each
-// client, an IPv4 address, fetched at verdictsFetched, as reputation.json
-// would at start.
+// client, an IPv4 address, fetched at verdictsFetched, as
+// abuseipdb-scores.json would at start.
 func loadScores(server *waf.Server, scores map[string]int64) {
 	kept := make([]reputation.Score, 0, len(scores))
 	for client, score := range scores {
@@ -1116,7 +1116,7 @@ func wantAbuseIPDBChecks(t *testing.T, server *waf.Server, want int) {
 }
 
 // loadLists puts copies of lists into server's lists, by URL, each with
-// its lines, fetched at listsFetched, as reputation.json would at start.
+// its lines, fetched at listsFetched, as lists.json would at start.
 func loadLists(t *testing.T, server *waf.Server, copies map[string][]string) {
 	t.Helper()
 

internal/waf/waf.go

@@ -143,10 +143,9 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 // refusedWhateverTheAnswer reports whether the request is one that does not wait
 // for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
-// its client: the heuristic crowdsec, header_limits, blocklists or rate_limits
-// will refuse it, as a function in that heuristic's file says. In observe mode,
-// which refuses nothing, none is. rproxy lets a request through when
-// waf answers too late.
+// its client: the heuristic crowdsec, blocklists or rate_limits will refuse it,
+// as a function in that heuristic's file says. In observe mode, which refuses
+// nothing, none is. rproxy lets a request through when waf answers too late.
 func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
 	if rq.h.config.Observe {
 		return false
@@ -154,8 +153,7 @@ func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
 
 	_, decided := rq.crowdSecDecisionInForce(now)
 
-	return decided || overHeaderLimits(rq.headerCounts()) ||
-		rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
+	return decided || rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
 		rq.overAWholeRateLimit(now)
 }