Caches of outside APIs live in /var/cache/sww (closes #314)

Every cache of answers from an outside service has its own file in `SWW_CACHE_DIR` (default `/var/cache/sww`): `lookups.json` (GeoJS), `lists.json` (lists fetched by URL, CrowdSec's decisions included), `dnsbl.json`, `abuseipdb-scores.json` and `routewatch.json` (at most 100,000 answers, each kept an hour). `reputation.json` is gone; the AbuseIPDB checks spent today stay in `SWW_STATE_DIR` as `abuseipdb-checks-spent.json`, since losing them would spend the daily budget twice.

A cache file never stops the start: one that does not parse is set aside as `<name>.bad` with a `file_error` alert. An unwritable `SWW_CACHE_DIR` stops the start. The `run` script creates the cache directory and gives it to `sww`. The cache directory needs no volume and can be left out of backups.

Model: opus-5-5

SPEC.md

@@ -37,7 +37,7 @@ never as a time.
   `SWW_`, and those of `rproxy` with `SWRP_`. The only other inputs are `waf`'s
   files:
     - the rule files;
-    - the state files;
+    - the state files and the cache files;
     - a lookup database;
     - a `SWW_` setting given as a file;
     - the file `SWW_LOG_REMOTE_TLS_CA_FILE` names;
@@ -56,19 +56,19 @@ never as a time.
 ## Architecture
 
 - One statically linked Go binary. Only `waf` writes to disk, and only to its
-  state directory. It ships in one image, which runs either command (see
-  "Deployment").
+  state directory and its cache directory. It ships in one image, which runs
+  either command (see "Deployment").
 - No database of any kind. Nothing is read from disk while serving a request.
 - Packages:
     - `cmd/sww`: the binary, which only calls `internal/sww`.
     - `internal/sww`: the process, as the command it is given. `sww waf` reads
-      the settings, the rule files, the lookup database and the state files,
-      listens, and serves until `SIGTERM` or `SIGINT`. Then it stops, writing
-      the state files (see "`waf`"). `sww rproxy` reads only that command's
-      settings, and runs `internal/rproxy` until `SIGTERM` or `SIGINT` (see
-      "`rproxy`"). `sww healthcheck` is the image's health check. With no
-      command, or one it does not know, the binary prints the commands and exits
-      with status 2.
+      the settings, the rule files, the lookup database and the state and cache
+      files, listens, and serves until `SIGTERM` or `SIGINT`. Then it stops,
+      writing the state and cache files (see "`waf`"). `sww rproxy` reads only
+      that command's settings, and runs `internal/rproxy` until `SIGTERM` or
+      `SIGINT` (see "`rproxy`"). `sww healthcheck` is the image's health check.
+      With no command, or one it does not know, the binary prints the commands
+      and exits with status 2.
     - `internal/config`: reads the settings, the one place they are read.
     - `internal/waf`: `waf`'s server. It runs the checks in `check`, and holds
       `waf`'s question, answer and report. It answers `waf`'s own endpoints in
@@ -96,8 +96,8 @@ never as a time.
     - `internal/alerts`: holds back repeats and alerts past the hourly limit,
       and sends the rest to the webhook, Slack and ntfy, each from a queue of
       its own.
-    - `internal/state`: reads the state files at start, takes in edits while
-      running, and writes the files when due and at the stop.
+    - `internal/state`: reads the state and cache files at start, takes in edits
+      while running, and writes the files when due and at the stop.
     - `internal/requestlog` and `internal/remotelog`: the lines on stdout, and
       sending them to the syslog server.
     - `internal/metrics`: the metrics, in the Prometheus text format.
@@ -106,7 +106,7 @@ never as a time.
       container, which `script/example-app` checks with `waf` in another.
 - Libraries, checked against the owner's Go dependency defaults:
     - the standard library for the reverse proxy, HTTP clients, DNS, logging
-      (`log/slog`) and the state files (`encoding/json`, `os.Rename`);
+      (`log/slog`) and the state and cache files (`encoding/json`, `os.Rename`);
     - `github.com/corazawaf/coraza/v3` and
       `github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries
       the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to it, and the
@@ -367,7 +367,7 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
   one, since `rproxy` never touches disk.
 - Settings, files included, are read once at start. Changing one means
   restarting the container. The files `waf` watches while it runs are its state
-  files, its rule files and the lookup database.
+  and cache files, its rule files and the lookup database.
 - An invalid setting stops the process at start with one log line, marked
   `"type":"process"` with `"msg":"invalid setting"`. The line names the variable
   and what is wrong with the value; "Each container's processes" says what
@@ -397,10 +397,10 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
   start. The file form keeps a token out of the container's environment (see
   "Tokens" under "Deployment").
 
-### State files
+### State and cache files
 
-- `SWW_STATE_DIR` names the directory of the state files (see "State" under
-  "Deployment").
+- `SWW_STATE_DIR` names the directory of the state files, and `SWW_CACHE_DIR`
+  that of the cache files (see "State" under "Deployment").
 - `bans.json` is written `SWW_STATE_WRITE_DELAY` after a ban is made, lifted or
   made permanent, so a burst of changes becomes one write. Every file is written
   every `SWW_STATE_COUNTER_INTERVAL`, and at an orderly stop.
@@ -583,7 +583,7 @@ Each tag and where it comes from:
     - They are added before `deny_nets`, so the log line of a request it refused
       has them.
     - `waf` asks routewatch in the background and never waits for it. An answer
-      is kept for 1 hour, in memory only.
+      is kept for 1 hour, in memory and in `routewatch.json`.
     - After a query fails, routewatch is not asked for 1 minute.
     - A request without these tags gives the reason in the log field
       `routewatch_tags_missing` (see
@@ -1601,8 +1601,8 @@ together.
     is kept whole, comment lines included, on failure and across restarts.
 
 - Each list is fetched again `SWW_BLOCKLIST_REFRESH` after its last try, one
-  list at a time. `reputation.json` keeps when each was last tried, so a restart
-  does not fetch a list again before its time. A fetch fails when:
+  list at a time. `lists.json` keeps when each was last tried, so a restart does
+  not fetch a list again before its time. A fetch fails when:
     - the server answers other than `200`;
     - the fetch does not finish within a minute;
     - the list is longer than 16 MiB;
@@ -1646,9 +1646,9 @@ together.
     DNS blocklists made for mail servers list residential ranges, and would
     block ordinary visitors, so they should not be used. Spamhaus zones need
     their keyed query service, given as a zone name that contains the key. `waf`
-    shows the key as `********` wherever it names the zone. Only
-    `reputation.json` keeps the key. `SWW_DNSBL_RESOLVER` names a resolver,
-    since public resolvers are refused by several list operators.
+    shows the key as `********` wherever it names the zone. Only `dnsbl.json`
+    keeps the key. `SWW_DNSBL_RESOLVER` names a resolver, since public resolvers
+    are refused by several list operators.
 
 - AbuseIPDB (`SWW_ABUSEIPDB_KEY`) is asked, at its check endpoint, for the abuse
   confidence score of a client, from 0 to 100. Only a client that has already
@@ -1659,11 +1659,11 @@ together.
   client's later requests. For an IPv6 client, AbuseIPDB is asked about the
   address of the request that set off the check, and the score is used for the
   whole group. A score at or over `SWW_ABUSEIPDB_MIN_SCORE` is a hit. The budget
-  is counted in UTC days and kept in `reputation.json`, so a restart does not
-  make it whole again. The check that spends the last of it is logged and
-  alerted, and no client is checked until the day ends. The key is sent in the
-  `Key` header and nowhere else, and is never logged. A check fails, after which
-  no client is checked for a minute, when:
+  is counted in UTC days and kept in `abuseipdb-checks-spent.json`, so a restart
+  does not make it whole again. The check that spends the last of it is logged
+  and alerted, and no client is checked until the day ends. The key is sent in
+  the `Key` header and nowhere else, and is never logged. A check fails, after
+  which no client is checked for a minute, when:
     - AbuseIPDB answers other than `200`;
     - the answer gives no score;
     - no answer comes within `SWW_REPUTATION_TIMEOUT`.
@@ -1671,9 +1671,9 @@ together.
   by default. Such verdicts are less certain than a blocklist, so a listed
   client gets a quarter of every request and byte limit rather than a refusal.
 - Verdicts and scores are kept for `SWW_REPUTATION_CACHE_TTL`, in memory and in
-  `reputation.json`. A source that is down or over quota gives no verdict. The
-  service continues. Each failure raises a `source_failure` alert (see
-  "Alerting").
+  `dnsbl.json` and `abuseipdb-scores.json`. A source that is down or over quota
+  gives no verdict. The service continues. Each failure raises a
+  `source_failure` alert (see "Alerting").
 
 ### Alerting
 
@@ -1776,21 +1776,43 @@ default file.
 
 ## Persistent state
 
-All state lives in memory, and the files in `SWW_STATE_DIR` hold a copy of it,
-so an orderly stop loses nothing. The files are meant for people as well: an
-admin can read or edit them at any time, and the running `waf` takes the edit
-in. `docs/state-files.md` lists the files, what each holds, and their sizes.
+All state lives in memory, and JSON files hold a copy of it, so an orderly stop
+loses nothing. The files are meant for people as well: an admin can read or edit
+them at any time, and the running `waf` takes the edit in. `docs/state-files.md`
+lists the files, what each holds, and their sizes.
 
+- The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
+    - `bans.json`;
+    - `clients.json`;
+    - `abuseipdb-checks-spent.json`;
+    - `alerts.json`.
+- The cache files, in `SWW_CACHE_DIR`, hold the answers of outside services:
+    - `lookups.json`;
+    - `lists.json`;
+    - `dnsbl.json`;
+    - `abuseipdb-scores.json`;
+    - `routewatch.json`.
+- A lost cache file costs only questions asked again, so the cache directory can
+  be left out of backups.
 - Each file holds one kind of state, as JSON with:
     - a top-level `version` number;
     - entries sorted by client address;
     - times in RFC 3339 UTC;
     - days as `YYYY-MM-DD`;
     - durations and sizes as plain numbers, with the unit in the field name.
-- `bans.json`, `reputation.json` and `alerts.json` are indented, so that a
-  person can open `bans.json` in an editor, find an address, and remove or add
-  an entry. `clients.json` and `lookups.json` put each entry on one line, which
-  halves their size and lets `grep` show everything about one client.
+- These files are indented, so that a person can open `bans.json` in an editor,
+  find an address, and remove or add an entry:
+    - `bans.json`;
+    - `abuseipdb-checks-spent.json`;
+    - `alerts.json`;
+    - `lists.json`.
+- The others put each entry on one line, which halves their size and lets `grep`
+  show everything about one client:
+    - `clients.json`;
+    - `lookups.json`;
+    - `dnsbl.json`;
+    - `abuseipdb-scores.json`;
+    - `routewatch.json`.
 - Clients that were never banned are kept too, so every client's history
   survives a restart.
 - Each file has a bound, so it can be rewritten whole. Writing one takes well
@@ -1804,8 +1826,8 @@ in. `docs/state-files.md` lists the files, what each holds, and their sizes.
   disk, so an admin's edit is never overwritten.
 - At start, the files are read back. Active bans go into an in-memory lookup by
   netblock, everything else into maps. Expired cache entries are discarded. A
-  missing file means empty state, and is normal on first run. These stop the
-  process with a message naming the file and position:
+  missing file means empty state, and is normal on first run. In a state file,
+  these stop the process with a message naming the file and position:
     - a file that does not parse, or has an unknown `version`;
     - an entry without a field it needs;
     - a ban whose cause is not one of the four;
@@ -1815,6 +1837,11 @@ in. `docs/state-files.md` lists the files, what each holds, and their sizes.
     Since writes are atomic, a broken file can only come from a hand edit, which
     the editor should hear about.
 
+- A cache file never stops the start. One that does not parse is set aside as
+  `<name>.bad`, as an edit is below. One that cannot be read is logged and
+  written over at its next write. Either way its cache starts empty and fills
+  again.
+
 - While running, `waf` notices a file that is edited, replaced or added. It
   tells its own writes from an admin's by comparing the file with what it last
   read or wrote. An edit that parses is taken in at once: what the file says
@@ -1905,10 +1932,10 @@ each app. `rproxy` asks `waf` about each visitor's request. `waf` answers
 This section is the agreement between the two. The header names, the fields and
 the statuses below are exact.
 
-`waf` reads the settings, the rule files, the lookup database and the state
-files, and listens at `SWW_LISTEN_ADDR`. On `SIGTERM` or `SIGINT` it stops
+`waf` reads the settings, the rule files, the lookup database and the state and
+cache files, and listens at `SWW_LISTEN_ADDR`. On `SIGTERM` or `SIGINT` it stops
 taking requests and gives those in progress five seconds to finish. It then
-writes the state files.
+writes the state and cache files.
 
 ### The question
 
@@ -2481,8 +2508,10 @@ It sets no `ENTRYPOINT` or `USER` of its own. The container must start
   `SWW_COMMAND` is `waf`, and `rproxy` when it is `rproxy`, unset or empty. It
   runs either as the `sww` user (uid and gid 65532), with runit's `chpst`. For
   `waf`, it first makes `SWW_STATE_DIR` and every file in it belong to that
-  user. For `rproxy` it changes no owner. With any other value, it starts
-  nothing, and stops with a message naming `SWW_COMMAND` and its value.
+  user. It also makes `SWW_CACHE_DIR` if it is missing, and gives it and every
+  file in it to that user. For `rproxy` it changes no owner. With any other
+  value, it starts nothing, and stops with a message naming `SWW_COMMAND` and
+  its value.
 
 Every `run` script, the app's included, is a bash script. It starts with
 `#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main`
@@ -2505,7 +2534,7 @@ process that stops at start is started again every second, not in a tight loop.
     - invalid configuration;
     - a lookup database that is missing or unreadable;
     - a rule file or state file that does not parse;
-    - an unwritable `SWW_STATE_DIR`;
+    - an unwritable `SWW_STATE_DIR` or `SWW_CACHE_DIR`;
     - a `SWW_RULES_DIR` that does not exist or cannot be read;
     - a `SWW_LISTEN_ADDR` it cannot listen on.
 - Only an invalid setting, or a `SWRP_LISTEN_ADDR` it cannot listen on, stops
@@ -2574,6 +2603,10 @@ needs no backup beyond the host's own. The image declares no volume, since every
 app image built on it would inherit it. `rproxy` keeps nothing, and needs no
 volume.
 
+`waf` keeps its cache files in `SWW_CACHE_DIR`, `/var/cache/sww` by default,
+which the `run` script makes. The cache directory needs no volume and can be
+left out of backups: a new container starts with empty caches, which fill again.
+
 ### Tokens
 
 A token given as a file (`SWW_ADMIN_TOKEN_FILE`, `SWW_METRICS_TOKEN_FILE`) stays