servewaf answers rproxy's question about each request and takes its report (closes #170)

`smallwebwaf servewaf` runs today's checks on the visitor's request a
question carries and answers allow, deny or close, instead of passing it
to an app. rproxy's report then counts the bytes toward the byte limits
and adds the request to the client's history. SPEC.md gives the exact
question, answer and report, which rproxy follows. `smallwebwaf` with no
command is unchanged.

Judgement call: a third disposition, close, since closing the question's
connection would read as servewaf down and let the request through.
Judgement call: allow can answer a new request id, since a visitor behind
a trusted proxy can choose its own.

Model: opus-5-5

SPEC.md

@@ -47,14 +47,16 @@ to them.
     - `internal/smallwebwaf`: the process. It reads the settings, the rule
       files, the lookup database and the state files, listens, and serves until
       `SIGTERM` or `SIGINT`. Then it stops, writing the state files. Run as
-      `smallwebwaf healthcheck`, it is the image's health check instead.
+      `smallwebwaf servewaf`, it answers `rproxy` instead of passing requests to
+      an app (see "`servewaf`"). Run as `smallwebwaf healthcheck`, it is the
+      image's health check instead.
     - `internal/config`: reads the settings, the one place they are read.
     - `internal/proxy`: what happens to each request. It works out the client
       and runs the checks in `check`. It passes the request to the app with the
       standard library's `httputil.ReverseProxy`, within the timeouts and size
       limits, and answers its own endpoints in `answerAdmin`. Once the request
       has ended, it counts bytes, refusals and anomalies, and writes the log
-      line.
+      line. It also holds `servewaf`'s question, answer and report.
     - `internal/ratelimit`: the table of clients. It counts requests, bytes and
       refusals, and keeps each client's history.
     - `internal/bans`: the bans. It holds each netblock's bans with their notes,
@@ -1006,6 +1008,156 @@ answers.
   costs).
 - A ban made through the endpoints starts at once.
 
+## `servewaf`
+
+`smallwebwaf servewaf` runs the checks for `smallwebwaf rproxy`, a small reverse
+proxy in front of each app. `rproxy` asks `servewaf` about each visitor's
+request. `servewaf` answers `rproxy` and passes nothing to an app itself. One
+`servewaf` serves many sites.
+
+This section is the agreement between the two. The header names, the fields and
+the statuses below are exact. Each `SWWAF_` setting and check keeps the meaning
+this document gives it, except as this section says.
+
+It starts as `smallwebwaf` with no command does. It reads the settings, the rule
+files, the lookup database and the state files, and listens at
+`SWWAF_LISTEN_ADDR`. It stops in the same orderly way, writing the state files.
+`docs/settings.md` lists the settings it ignores.
+
+### The question
+
+A request carrying `X-Smallwebwaf-Client` is `rproxy` asking about one visitor's
+request. It is the visitor's request as the visitor sent it:
+
+- the method;
+- the path and query as sent;
+- `Host`;
+- every header the visitor sent, including its own `X-Forwarded-For`;
+- the first 64 KiB of the body, or all of it when shorter.
+
+A body whose length the visitor did not announce comes chunked. Any other body
+comes with a `Content-Length` of the part it carries.
+
+`rproxy` removes these headers if the visitor sent them, then sets them. It adds
+no other header:
+
+- `X-Smallwebwaf-Client`: the visitor's address, which `rproxy` has worked out.
+  `servewaf` takes it as the client, and groups an IPv6 address as "What happens
+  to a request" says.
+- `X-Smallwebwaf-Peer`: the address of `rproxy`'s own TCP peer, for the log
+  line's `peer_ip`.
+- `X-Smallwebwaf-Content-Length`: the body length the visitor announced. It is
+  absent when the visitor announced none.
+- `X-Request-ID`: the request's id.
+- `X-Forwarded-Proto`: the scheme.
+
+Before any check, `servewaf` removes the three `X-Smallwebwaf-` headers. It also
+sets `Content-Length` to the length the visitor announced, or removes it. So the
+rule files and the Core Rule Set see the headers the visitor sent.
+
+### The answer
+
+`servewaf` runs steps 3 to 12 of "What happens to a request", in that order.
+Every answer but the last below carries `X-Smallwebwaf-Disposition`.
+
+- `X-Smallwebwaf-Disposition: allow`, status `200`, no body. `rproxy` passes the
+  request to the app. The answer also carries:
+    - `X-Request-ID`: the id `rproxy` passes to the app, logs and names in the
+      report;
+    - `X-Client-ASN` and `X-Client-Country`, with `SWWAF_ADD_LOOKUP_HEADERS`,
+      which `rproxy` adds to the request it passes on. A header whose value is
+      unknown is left out.
+- `X-Smallwebwaf-Disposition: deny`, with the status, headers and body the
+  visitor is to get. `rproxy` gives them to the visitor, less this header. The
+  status is one of:
+    - `SWWAF_BAN_RESPONSE`, for a refusal;
+    - `403`, for a `block` rule or the Core Rule Set;
+    - `404`, for a path under `/_smallwebwaf/` that passes the checks, so that
+      the app never sees that prefix.
+- `X-Smallwebwaf-Disposition: close`, status `200`, no body. This takes the
+  place of a refusal while `SWWAF_BAN_RESPONSE` is `close`. `rproxy` closes the
+  visitor's connection without an answer.
+- No such header, status `400`, when `X-Smallwebwaf-Client` is not an address.
+  `rproxy` treats this answer as it treats no answer at all.
+
+The `X-Request-ID` of an `allow` is the question's id, unless `servewaf` has to
+make a new one. A visitor behind a trusted proxy can choose its id, so it makes
+a new one when the question's id is:
+
+- absent;
+- the id of another request awaiting its report;
+- not valid UTF-8, which a JSON report cannot carry.
+
+In `observe` mode a request that `enforce` mode would refuse is answered
+`allow`, and its log line has `would_action`. A path under `/_smallwebwaf/` is
+still answered `404`.
+
+The Core Rule Set reads the body a question carries only while
+`SWWAF_WAF_BODY_LIMIT` is set. It reads at most 64 KiB less one byte. So it does
+not read a JSON or XML body of 64 KiB or more, which may have been cut short.
+
+### The report
+
+Once the visitor's answer has ended, `rproxy` sends `POST /_smallwebwaf/report`
+for each request it passed to the app on an `allow`. The report carries no
+`X-Smallwebwaf-Client`. Its body is a JSON object with these fields, named as
+the request log's fields of the same meaning:
+
+- `request_id`: the `X-Request-ID` of the `allow`;
+- `status`: what the visitor got, `0` if nothing;
+- `request_bytes` and `response_bytes`: the body bytes each way, with what a
+  WebSocket carried each way;
+- `duration_total`: milliseconds from the request's headers being read to the
+  end of the answer;
+- `duration_upstream_total`: milliseconds from the hand-over to the app to the
+  end of the answer.
+
+`servewaf` answers `204`, also for a request it does not hold, which it ignores.
+It answers `400` to a body that is not such an object, has a negative byte
+count, or is over 1 MiB. `rproxy` does not act on the answer.
+
+For a request it holds, `servewaf` then:
+
+- counts the bytes as `SWWAF_BYTES_COUNT` says, toward the client's byte limits
+  at the percentages it gave that request;
+- bans a client that passes a byte limit, and writes a process message naming
+  the request, since the request's log line has gone out;
+- adds the request with its status to the client's history and the anomaly
+  counters.
+
+`servewaf` holds each allowed request until its report comes, up to 10,000
+requests. Past that, the request asked about longest ago is dropped. A request
+whose report never comes is in neither the client's history nor the metrics, and
+its bytes are not counted. A denied request is finished at its decision.
+
+### `servewaf`'s own endpoints
+
+A request without `X-Smallwebwaf-Client` is for `servewaf` itself, at its own
+address on the private network. Nothing routes the internet there. These are:
+
+- the health check, the metrics, and the ban and client endpoints, as
+  `docs/admin-endpoints.md` lists them, with the same tokens;
+- the report.
+
+Anything else gets `404`. No check runs on these requests, since their peers are
+`rproxy` and the admin's tools. The report needs no token, so anything on the
+private network can send one or ask a question.
+
+`servewaf` gives each request 60 seconds to arrive and its answer 60 seconds to
+leave, and closes a connection idle for 120 seconds. It takes a request line and
+headers up to 1 MiB, Go's default, so that it never refuses a question for the
+size of a visitor's headers.
+
+### Sites
+
+One `servewaf` keeps one table of clients, counters and bans for every site that
+asks it. A client banned through one site is refused on all of them, and its
+requests on all of them count toward one set of limits. Log lines, alerts raised
+by a request and ban notes name the site by its host.
+
+`docs/request-log.md` gives `servewaf`'s log line, and `docs/metrics.md` how its
+metrics count.
+
 ## Deployment
 
 `smallwebwaf` runs inside the app's own container: the app's Dockerfile builds
@@ -1171,9 +1323,10 @@ and `script/gitea-requests` checks those notes.
 
 ## Planned, not built
 
-These two points are specified as later work, and nothing in the code does them
-yet:
+These points are specified as later work, and nothing in the code does them yet:
 
+- Planned: `smallwebwaf rproxy`, the reverse proxy in front of each app that
+  asks `servewaf` about each request (see "`servewaf`").
 - Planned: sharing bans between the instances of several apps, by running one
   CrowdSec engine per host that every instance reports its bans to. It is
   reconsidered once real ban volumes are known. Until then each instance keeps