Observe mode: log what would be refused, refuse nothing (closes #78)

SWWAF_MODE (default enforce) takes enforce or observe. In observe mode a
request that SWWAF_DENY_NETS, a ban, the country lists or a rate limit
would refuse is passed to the app, and its log line names that refusal
in would_action. The size and time limits and the 401 still apply. A
broken limit makes no ban; bans read from bans.json are kept but refuse
nothing, and Ledger.Find reads them without counting a refusal in their
notes.

Judgement call: in observe mode a broken limit does not reset the
client's counters, since the reset comes with the ban.
Judgement call: a request a ban would refuse keeps ban_expires.

Model: opus-5-5

README.md

@@ -13,17 +13,19 @@ JSON log line for every request.
 
 Status: the first two milestones are built
 (https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
-https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are five parts of
+https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are six parts of
 milestone 3: the static lists, the bans that broken rate limits lead to and the
-JSON state files, which come next in the build order, and the metrics endpoint
-and the header size and the idle time as settings, which come last in it.
-`smallwebwaf` passes each request to the app and the app's answer back,
-unchanged, within its timeouts and size limits, works out each client's address,
-bans a client that sends too many requests, refuses a client that comes from a
-country you refuse or from a network you refuse, lets the networks you choose
-through, keeps its bans, each client's counters and history, and GeoJS's answers
-in JSON files across restarts, writes a JSON log line for every request, and
-serves Prometheus metrics to a scraper that holds the metrics token. It comes as
+JSON state files, which come next in the build order, `observe` mode, which
+comes a little later, and the metrics endpoint and the header size and the idle
+time as settings, which come last in it. `smallwebwaf` passes each request to
+the app and the app's answer back, unchanged, within its timeouts and size
+limits, works out each client's address, bans a client that sends too many
+requests, refuses a client that comes from a country you refuse or from a
+network you refuse, lets the networks you choose through, keeps its bans, each
+client's counters and history, and GeoJS's answers in JSON files across
+restarts, writes a JSON log line for every request, serves Prometheus metrics to
+a scraper that holds the metrics token, and in `observe` mode passes on the
+requests it would refuse, logging what it would have done with them. It comes as
 the image the app's own image is built on. The rest of the design comes after
 that, in the order of the build order in [`SPEC.md`](SPEC.md). The survey of
 existing tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
@@ -118,6 +120,18 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set.
   `SWWAF_ALLOW_NETS` too is let through. A client in
   `SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither counted nor refused by the rate
   limits; the country lists and bans still apply to it.
+- In `observe` mode, with `SWWAF_MODE=observe`, refuses none of the requests
+  that `SWWAF_DENY_NETS`, a ban, the country lists or a rate limit would refuse:
+  it passes them to the app, and their log lines name what `enforce` mode would
+  have done (see `would_action` in "Request log" below). The checks run, and
+  requests are counted, as in `enforce` mode, but a broken rate limit makes no
+  ban and does not set the client's counters back to zero, so each request over
+  the limit is logged as one that would be refused. The bans in `bans.json` are
+  kept, and refuse requests again when `smallwebwaf` next runs in `enforce`
+  mode, as long as they last. The timeouts and size limits still apply, since
+  they protect `smallwebwaf` and the app themselves, and a request for the
+  metrics without the token is still answered `401`. It is for trying a
+  configuration before enforcing it.
 - Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
   check and without asking the app, for the image's health check.
 - Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for
@@ -139,6 +153,9 @@ it, and the effective settings are logged at start.
 - `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens.
 - `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app, as `http` or
   `https`, a host and an optional port, and nothing more.
+- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to pass on the
+  requests `smallwebwaf` would refuse and log what it would have done (see "What
+  it does so far" above).
 - `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`,
   the private address ranges): the netblocks whose `X-Forwarded-For` is
   believed. A list given replaces the default; set but empty, it trusts nothing.
@@ -240,8 +257,8 @@ refused ones included:
 - `country` is the client's country as GeoJS places it. It is empty with neither
   country list set, for a client in `SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`, for
   a client on a private, loopback or link-local address, when GeoJS cannot place
-  the client or has not answered in time, and for a request refused because a
-  ban covers its client, even when the client's country is known.
+  the client or has not answered in time, and for a request whose client a ban
+  covers, even when the client's country is known.
 - `status` is what the client was sent, `0` if nothing was; `upstream_status` is
   what the app answered, and is left out when the app did not answer.
 - `request_bytes` and `response_bytes` count body bytes.
@@ -253,11 +270,18 @@ refused ones included:
   `timed_out` for one that ran out of time, `upstream_error` when the app could
   not be reached or its answer broke off, and `admin` for one `smallwebwaf`
   answered at its own endpoint.
+- `would_action` is there in `observe` mode for a request that
+  `SWWAF_DENY_NETS`, a ban, the country lists or a rate limit would have refused
+  in `enforce` mode, and names the action that refusal would have had: `denied`,
+  `banned`, `country_denied` or `rate_limited`. `action` then names what was
+  done: `forward` for a request passed to the app, and another action, such as
+  `too_large`, for one a size or time limit refused.
 - `limit_hit` is there for a request that broke a rate limit, and names the
   window whose limit it went over: `minute`, `hour` or `day`, the shortest if it
   went over several. `offence` is then `limit`.
 - `ban_expires` is there for a request that made a ban or was refused under one,
-  and gives when the ban ends, in the same form as `time`, or `permanent`.
+  or in `observe` mode would have been refused under one, and gives when the ban
+  ends, in the same form as `time`, or `permanent`.
 - `aborted` is there, and true, when the client went away early.
 - `duration_total` and `duration_upstream_total` are in milliseconds.
 
@@ -581,17 +605,16 @@ the metrics, failure behaviour and the build order.
 So far `smallwebwaf` looks up only the country, only through GeoJS, and only
 while `SWWAF_DENIED_COUNTRIES` or `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set:
 then the address of every new visitor is sent to GeoJS, except a visitor in
-`SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS` and one refused because a ban covers its
-netblock, and with neither set, none is. An IPv6 visitor is asked about by the
-first address of its /64. A new visitor waits at most a second for its answer,
-and without one counts as coming from an unknown country until the answer
-arrives. The addresses waiting are asked about together, up to 200 in one
-request, one request at a time; at most 10,000 visitors wait, and one more
-counts as coming from an unknown country until there is room. While GeoJS fails,
-visitors with a kept answer are unaffected and new ones count as coming from an
-unknown country. GeoJS is then left alone for a second, twice as long after each
-further failure up to five minutes, and asked again by the next request that
-needs it.
+`SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS` and one whose netblock a ban covers, and
+with neither set, none is. An IPv6 visitor is asked about by the first address
+of its /64. A new visitor waits at most a second for its answer, and without one
+counts as coming from an unknown country until the answer arrives. The addresses
+waiting are asked about together, up to 200 in one request, one request at a
+time; at most 10,000 visitors wait, and one more counts as coming from an
+unknown country until there is room. While GeoJS fails, visitors with a kept
+answer are unaffected and new ones count as coming from an unknown country.
+GeoJS is then left alone for a second, twice as long after each further failure
+up to five minutes, and asked again by the next request that needs it.
 
 In the full design, `smallwebwaf` looks up the AS number and country of every
 client, for the request log, the metrics and the ban notes, and for the country
@@ -643,8 +666,10 @@ addresses are never sent to GeoJS.
   limits, and writes the request's log line. Its `check` method is where a
   request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for
   a ban, for the country lists, for a rate limit, which bans the client, and for
-  an announced body over the size limit. A request under `/_smallwebwaf/` that
-  `check` lets through is answered by `answerAdmin` instead of reaching the app.
+  an announced body over the size limit; in `observe` mode, only for the size
+  limit, with what it would have refused for noted in the log line. A request
+  under `/_smallwebwaf/` that `check` lets through is answered by `answerAdmin`
+  instead of reaching the app.
 - `internal/metrics`: the metrics, counted as the other parts tell it what
   happened, and served in the Prometheus text format.
 - `internal/bans`: the ban ledger: each netblock's bans with their notes, how
@@ -710,10 +735,10 @@ so that they run in minimal containers.
 
 ## TODO
 
-- The rest of milestone 3, from taking in an admin's edits to the state files
-  (https://git.eeqj.de/sneak/smallwebwaf/issues/68) up to the rest of the
-  request log's fields, and the rest of the design, in the order of the build
-  order in [`SPEC.md`](SPEC.md).
+- The rest of milestone 3: taking in an admin's edits to the state files
+  (https://git.eeqj.de/sneak/smallwebwaf/issues/68), exemptions and the rest of
+  the request log's fields; then the rest of the design, in the order of the
+  build order in [`SPEC.md`](SPEC.md).
 
 ## Documents