rproxy, the proxy in front of each app, asks servewaf and fails open (closes #171)

`smallwebwaf rproxy` reads the first 64 KiB of each body, asks servewaf, and
passes what servewaf allows to the app within its own size and time limits.
It reports each request servewaf answered. SWRP_DEFAULT_DISPOSITION decides
when servewaf gives no answer.

Duplication: internal/rproxy copies passing-on code from internal/proxy until
#172 removes the originals.
Rule suppressed: gosec G704 on servewaf's URL, a setting.
Rule suppressed: gosec G705 on the answer written to the visitor.
Judgement call: the SWRP_ size and time limits take off; SWRP_TIMEOUT_MS does not.

Model: opus-5-5

SPEC.md

@@ -12,9 +12,10 @@ to them.
 
 - It protects one self-hosted app on the open internet from scrapers, scanners
   and other abusers. It does so from the app's first request, with no setting at
-  all. Every setting has a default chosen for a service facing the internet
-  in 2026. A setting that needs an alert destination, an account key or a token
-  stays off until the operator supplies it.
+  all. Every setting but `SWRP_WAF_BACKEND`, which only `smallwebwaf rproxy`
+  reads, has a default chosen for a service facing the internet in 2026. A
+  setting that needs an alert destination, an account key or a token stays off
+  until the operator supplies it.
 - It limits each client's requests and bytes per minute, hour and day. It bounds
   the size and duration of every request and answer. It detects attacks.
 - It bans abusers: briefly at first, for seven days on a clear sign of attack,
@@ -22,10 +23,11 @@ to them.
 - It looks up each client's AS number and country. It can refuse whole countries
   and netblocks, lower its limits for listed AS numbers and countries, consult
   reputation sources, and send alerts.
-- Everything is set by environment variables whose names start with `SWWAF_`,
-  since `smallwebwaf` shares its container, and so its environment, with the
-  app. The only other inputs are files: the rule files, the state files, a
-  lookup database, and a setting given as a file.
+- Everything is set by environment variables whose names start with `SWWAF_`, or
+  with `SWRP_` for `smallwebwaf rproxy`, since `smallwebwaf` shares its
+  container, and so its environment, with the app. The only other inputs are
+  files: the rule files, the state files, a lookup database, and a `SWWAF_`
+  setting given as a file.
 
 ## Non-goals
 
@@ -48,8 +50,10 @@ to them.
       files, the lookup database and the state files, listens, and serves until
       `SIGTERM` or `SIGINT`. Then it stops, writing the state files. Run as
       `smallwebwaf servewaf`, it answers `rproxy` instead of passing requests to
-      an app (see "`servewaf`"). Run as `smallwebwaf healthcheck`, it is the
-      image's health check instead.
+      an app (see "`servewaf`"). Run as `smallwebwaf rproxy`, it reads only that
+      command's settings, and runs `internal/rproxy` until `SIGTERM` or `SIGINT`
+      (see "`rproxy`"). Run as `smallwebwaf healthcheck`, it is the image's
+      health check instead.
     - `internal/config`: reads the settings, the one place they are read.
     - `internal/proxy`: what happens to each request. It works out the client
       and runs the checks in `check`. It passes the request to the app with the
@@ -57,6 +61,8 @@ to them.
       limits, and answers its own endpoints in `answerAdmin`. Once the request
       has ended, it counts bytes, refusals and anomalies, and writes the log
       line. It also holds `servewaf`'s question, answer and report.
+    - `internal/rproxy`: the reverse proxy in front of one app that asks
+      `servewaf` about each request.
     - `internal/ratelimit`: the table of clients. It counts requests, bytes and
       refusals, and keeps each client's history.
     - `internal/bans`: the bans. It holds each netblock's bans with their notes,
@@ -291,17 +297,20 @@ Conventions:
     - `SWWAF_WAF_ANOMALY_THRESHOLD`, `SWWAF_WAF_BODY_LIMIT` and
       `SWWAF_ERROR_BURST_THRESHOLD`;
     - `SWWAF_ALERT_COOLDOWN` and `SWWAF_ALERT_MAX_PER_HOUR`;
-    - every anomaly threshold.
+    - every anomaly threshold;
+    - `SWRP_MAX_REQUEST_SIZE`, `SWRP_MAX_RESPONSE_SIZE`,
+      `SWRP_MAX_REQUEST_TIME_MS` and `SWRP_MAX_RESPONSE_TIME_MS`.
 
     No other setting can be switched off with `off`.
 
 - A list set to an empty value is an empty list, and replaces the default.
-- Any setting can be read from a file instead, for secrets and long lists:
-  `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. The
-  file's contents are the value, less one newline at the end, and are checked as
-  the value itself would be. Setting both forms stops the start, and so does
-  naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already
-  holds a file name, so it has no `_FILE` form.
+- Any `SWWAF_` setting can be read from a file instead, for secrets and long
+  lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
+  The file's contents are the value, less one newline at the end, and are
+  checked as the value itself would be. Setting both forms stops the start, and
+  so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE`
+  already holds a file name, so it has no `_FILE` form. No `SWRP_` setting has
+  one, since `rproxy` never touches disk.
 - Settings, files included, are read once at start. Changing one means
   restarting the container. The files `smallwebwaf` watches while it runs are
   its state files, its rule files and the lookup database.
@@ -1223,6 +1232,129 @@ by a request and ban notes name the site by its host.
 
 `docs/metrics.md` says how its metrics count.
 
+## `rproxy`
+
+`smallwebwaf rproxy` is the reverse proxy in front of one app. It asks
+`servewaf` about each request, and passes on what `servewaf` allows. It keeps
+nothing between requests and never touches disk. When `servewaf` is down, slow
+or overloaded, the site keeps answering.
+
+`rproxy` reads only its `SWRP_` settings, which `docs/settings.md` lists. One of
+them is required: `SWRP_WAF_BACKEND`, `servewaf`'s URL. It has no metrics, no
+rule files and no state files, and it writes its lines to stdout only.
+
+The image's run script and health check are those of `smallwebwaf` with no
+command (see "Deployment"). A container that runs `rproxy` names its own command
+and health check.
+
+### What `rproxy` does with a request
+
+1. `GET /_smallwebwaf/healthz` is answered `ok` at once, without asking
+   `servewaf`.
+2. `rproxy` works out the visitor's address. It believes `X-Forwarded-For` only
+   from `SWRP_TRUSTED_PROXIES`, and reads it as step 2 of "What happens to a
+   request" says.
+3. It makes a fresh request id, whatever id the request came with. `servewaf`
+   and the app get it as `X-Request-ID`, and `rproxy`'s log line gives it.
+4. It reads up to the first 64 KiB of the body into memory.
+5. It sends `servewaf` the question (see "The question"). The question never
+   carries the visitor's `Transfer-Encoding` or `Trailer`, which Go's HTTP
+   server and client handle themselves.
+6. It does what the answer says (see "The answer"):
+    - `allow`: it passes the request to the app.
+    - `deny`: the visitor gets the answer's status, headers and body, less
+      `X-Smallwebwaf-Disposition`.
+    - `close`: it closes the visitor's connection without an answer.
+7. Once the answer has ended, it writes its log line. `docs/request-log.md`
+   gives the fields.
+8. It then sends the report, if `servewaf` answered `allow`, `deny` or `close`
+   (see "The report"). The visitor does not wait for it. A report that fails is
+   dropped, with a process warning.
+
+### Without an answer from `servewaf`
+
+`SWRP_DEFAULT_DISPOSITION` decides what is done with the request when
+`servewaf`:
+
+- does not answer within `SWRP_TIMEOUT_MS`, which covers connecting, sending the
+  question and reading the whole answer;
+- cannot be reached;
+- answers without a valid `X-Smallwebwaf-Disposition`, as with the `400` for a
+  question it refuses;
+- sends an answer whose body breaks off or is over 1 MiB.
+
+`allow`, the default, passes the request to the app. `deny` answers `503`.
+Either way `rproxy` writes a process warning, its log line has
+`default_disposition`, and it sends no report.
+
+A path under `/_smallwebwaf/` that `allow` would pass on is answered `404`
+instead, so that the app never sees that prefix.
+
+### Passing a request to the app
+
+`rproxy` sends the app at `SWRP_UPSTREAM_URL` the visitor's request: the part of
+the body it holds, then the rest, streaming. The app's answer comes back the
+same way. The request reaches the app unchanged but for these headers:
+
+- `X-Forwarded-For`, with the address of `rproxy`'s TCP peer added, when that
+  peer is inside `SWRP_TRUSTED_PROXIES`. The peer's other forwarded headers pass
+  on.
+- For a peer outside `SWRP_TRUSTED_PROXIES`, its forwarded headers are its own
+  claims and are replaced. `X-Forwarded-For` names the peer, `X-Forwarded-Host`
+  the host it asked for, and `X-Forwarded-Proto` plain `http`. The other headers
+  in which traefik tells an app about a visitor are removed.
+- `X-Request-ID`, which carries the fresh id.
+- `X-Client-ASN` and `X-Client-Country`, which carry what `servewaf` answered. A
+  header it did not give is left out. Any of the two the visitor sent is
+  removed.
+
+A WebSocket upgrade passes through. The report of a WebSocket comes when it
+closes, with what it carried each way.
+
+### `rproxy`'s limits
+
+The limits work as "Size and time limits" says, with these settings in place of
+those:
+
+- `SWRP_MAX_REQUEST_SIZE` for `SWWAF_REQUEST_MAX_BYTES`.
+- `SWRP_MAX_RESPONSE_SIZE` for `SWWAF_RESPONSE_MAX_BYTES`.
+- `SWRP_MAX_REQUEST_TIME_MS` for `SWWAF_CLIENT_REQUEST_TIMEOUT` and
+  `SWWAF_UPSTREAM_REQUEST_TIMEOUT`.
+- `SWRP_MAX_RESPONSE_TIME_MS` for `SWWAF_CLIENT_RESPONSE_TIMEOUT` and
+  `SWWAF_UPSTREAM_RESPONSE_TIMEOUT`.
+
+The largest request line and headers is 32 KiB. A kept-open connection is closed
+after 120 seconds idle. Neither has a setting.
+
+`SWRP_MAX_REQUEST_TIME_MS` bounds the request line and headers. From their end,
+it then bounds everything until the app has been sent the whole request,
+including the wait for `servewaf`. A visitor too slow to send the first 64 KiB
+is answered `408`, with no question and no report. A visitor that goes away
+before then is not answered, with no question and no report either.
+
+`SWRP_MAX_RESPONSE_TIME_MS` runs from the end of the request to the last byte of
+the answer reaching the visitor. The request ends when the app has been sent all
+of it, or when the app's answer arrives if that is sooner. An answer `rproxy`
+gives itself has the whole time from when it starts.
+
+`rproxy` applies the request size limit only after `servewaf` has allowed the
+request. So a request `servewaf` denies gets `servewaf`'s answer, whatever its
+size. A request `servewaf` allows is answered `413` when its body is over the
+limit: at once for a length it announced, otherwise when the limit is reached.
+If part of the body has gone to the app by then, the request to the app is cut
+off.
+
+### Starting and stopping
+
+An invalid setting, or an address it cannot listen on, stops the start with a
+process line. The process lines give the host's name as `instance`, and only
+those at `info` or more severe are written.
+
+On `SIGTERM` or `SIGINT`, `rproxy` stops taking requests and gives those in
+progress five seconds to finish. It then waits for the reports on their way,
+each for up to `SWRP_TIMEOUT_MS`. A report that comes due after that is dropped,
+such as that of a WebSocket that closes later.
+
 ## Deployment
 
 `smallwebwaf` runs inside the app's own container: the app's Dockerfile builds
@@ -1390,7 +1522,6 @@ and `script/gitea-requests` checks those notes.
 
 These points are specified as later work, and nothing in the code does them yet:
 
-- Planned: `smallwebwaf rproxy`, the reverse proxy in front of each app.
 - Planned: sharing bans between the instances of several apps, by running one
   CrowdSec engine per host that every instance reports its bans to. It is
   reconsidered once real ban volumes are known. Until then each instance keeps