README: blocks misbehaving scrapers, not normal ones (closes #387)

The opening now says a scraper within the default limits is left alone
whatever its user agent, and lists what bans a client: the rate and byte
limits, a probe for a secret path, and the refusal limit.

Model: opus-5-5

README.md

@@ -5,17 +5,20 @@
 requests of misbehaving scrapers, scanners and other abusers before they reach
 an app.
 
-With the default settings, a client is banned when it does any of these:
+It does not block scrapers as such. A scraper that stays within the limits below
+is left alone, whatever its user agent. With the default settings, a client is
+banned when it does any of these:
 
 - makes more than 120 requests in a minute, 1500 in an hour or 6000 in a day;
 - moves more than `50G` of request and response bodies in an hour, or `100G` in
   a day;
-- asks for a secret or backup file, such as `/.env` or `/.git/config`;
+- asks for a path no real visitor asks for, such as `/.env` or `/.git/config`;
-- sends a user agent that names a security scanner, such as `sqlmap` or `nikto`;
 - has more than 30 requests in a minute refused as probes or attacks.
 
-A scraper that does none of these is left alone. A scraper that waits 15 seconds
+A scraper that waits 15 seconds between requests stays under every one of these
-between requests stays under the rate limits.
+limits. The user agent alone gets a client refused only when it names a security
+scanner, such as `sqlmap` or `nikto`. Settings you add, such as a refused
+country or a blocklist, can refuse any client.
 
 ## What it does
 
@@ -23,10 +26,10 @@ A web application firewall reads each request on its way to a web app and
 refuses the ones that look like abuse. `smallwebwaf` is one program, `sww`,
 shipped in one image, with two commands that do this together:
 
-- `sww rproxy` is a small reverse proxy in front of one app. It sits behind a
+- `sww rproxy` is a small reverse proxy in front of one app. traefik routes to
-  TLS-terminating proxy, such as traefik, nginx or Caddy. It asks `waf` about
+  it. It asks `waf` about each request, passes the allowed ones to the app, and
-  each request, passes the allowed ones to the app, and passes the app's answers
+  passes the app's answers back. It keeps nothing between requests and writes
-  back. It writes nothing to disk.
+  nothing to disk.
 - `sww waf` decides. It runs the checks and keeps the bans, the counters and
   each client's history. One `waf` serves many sites: a client banned through
   one site is refused on all of them, and its requests on all of them count
@@ -35,26 +38,7 @@ shipped in one image, with two commands that do this together:
 The site keeps answering while `waf` is down, slow or restarting: without an
 answer in time, `rproxy` passes the request to the app.
 
-`rproxy` also protects the app on its own, whether or not `waf` answers. With
+It protects against:
-the default settings, `rproxy`:
-
-- answers `400` to a malformed request, such as a broken header line or a
-  missing `Host`;
-- answers `405` to a method it does not pass on, such as `TRACE`;
-- answers `431` to a request line and headers over 32 KiB together;
-- gives the visitor 60 seconds to send the request line and headers, and then 60
-  seconds more for the whole request to reach the app;
-- answers `413` to a request body over 100 MiB;
-- gives the app's answer 30 minutes to reach the visitor, counted from the end
-  of the request;
-- answers `502` in place of an app's answer whose body is over 1 GiB, or cuts it
-  off if it has started;
-- closes a kept-open connection after 120 seconds idle.
-
-`rproxy` does not limit the number of headers. `waf` limits the number of
-headers and cookies, and the length of the request line and of each header line.
-
-Together, `rproxy` and `waf` protect against:
 
 - Scrapers that fetch every page of a site, all day. Each client gets a limit on
   requests and on body bytes per minute, per hour and per day. The defaults
@@ -76,6 +60,8 @@ Together, `rproxy` and `waf` protect against:
       limit, so that they are banned sooner;
     - refuse or limit clients on a blocklist, a DNS blocklist, or with a high
       AbuseIPDB score.
+- Slow and oversized requests. Every request and every answer has a time limit
+  and a size limit.
 
 For the operator, it:
 
@@ -87,7 +73,8 @@ For the operator, it:
 
 Not built, and not planned:
 
-- TLS and routing by host name. Both stay with the TLS-terminating proxy.
+- TLS and routing by host name. Both stay with traefik, the reverse proxy in
+  front of `rproxy`.
 - Browser challenges such as captchas.
 - A web console, or a configuration file.
 - Defence against floods large enough to fill the host's network link.
@@ -119,61 +106,20 @@ accepts connections, whether or not `waf` answers; in `waf`'s container, once
 The usual setup is three containers on one private network:
 
 - the app;
-- `rproxy`, in front of the app, which the TLS-terminating proxy forwards to:
+- `rproxy`, in front of the app, which traefik routes to:
   `SWW_RPROXY_WAF_BACKEND` names `waf` and `SWW_RPROXY_UPSTREAM_URL` names the
   app;
 - `waf`, with `SWW_COMMAND=waf` and a volume at `/var/lib/sww` for its state
   files.
 
-Publish no port of the app or of `waf`. Anything on that network can reach the
+Point traefik's labels at port 8080 of `rproxy`
-app around `rproxy`, and can ask `waf` questions, so the network holds only what
+(`traefik.http.services.<name>.loadbalancer.server.port=8080`). Publish no port
-you trust. Each further app adds its own container and its own `rproxy`, whose
+of the app or of `waf`. Anything on that network can reach the app around
+`rproxy`, and can ask `waf` questions, so the network holds only what you trust.
+Each further app adds its own container and its own `rproxy`, whose
 `SWW_RPROXY_WAF_BACKEND` names the same `waf`. The "Smallest setup" example
 below shows this shape.
 
-### The TLS-terminating proxy
-
-`rproxy` serves plain HTTP. It runs behind any TLS-terminating proxy, such as
-traefik, nginx or Caddy. The TLS-terminating proxy must:
-
-- forward each request to port 8080 of `rproxy` over plain HTTP;
-- keep the visitor's `Host` header;
-- add the visitor's address to `X-Forwarded-For`;
-- set `X-Forwarded-Proto` to the scheme the visitor used;
-- close an unused connection to `rproxy` in under 120 seconds;
-- connect to `rproxy` from an address in `SWW_RPROXY_TRUSTED_PROXIES`, by
-  default `10.0.0.0/8`, `172.16.0.0/12` or `192.168.0.0/16`.
-
-From any other address, `rproxy` ignores the forwarded headers and takes the
-TLS-terminating proxy for the visitor. Every visitor then shares one address,
-and one ban.
-
-traefik and Caddy keep the `Host` header and set both forwarded headers by
-default. traefik's Docker label
-`traefik.http.services.<name>.loadbalancer.server.port=8080` names the port.
-Caddy keeps an unused connection for 120 seconds by default, so its site
-shortens that to 90 seconds:
-
-```caddyfile
-reverse_proxy rproxy:8080 {
-    transport http {
-        keepalive 90s
-    }
-}
-```
-
-By default nginx sends the host in `proxy_pass` as `Host` and sets neither
-forwarded header, so its site sets all three:
-
-```nginx
-location / {
-    proxy_pass http://rproxy:8080;
-    proxy_set_header Host $host;
-    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-    proxy_set_header X-Forwarded-Proto $scheme;
-}
-```
-
 ### `rproxy` in the app's container
 
 An app's image can instead be built on the `smallwebwaf` image, so that `rproxy`
@@ -224,13 +170,13 @@ entrypoint starts `rproxy` and the app side by side, each as its own user. Run
 the app's image with `SWW_RPROXY_WAF_BACKEND` set. `deploy/example-app` is such
 an app, and the "`rproxy` beside the app" example below runs one.
 
-The app's TCP peer is `rproxy`. The app sees the headers the TLS-terminating
+The app sees the headers traefik sent, with traefik's address added to
-proxy sent, with that proxy's address added to `X-Forwarded-For`. So the app
+`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
-must trust `rproxy`'s address for forwarded headers. In a container of its own,
+`rproxy`'s address for forwarded headers. In a container of its own, `rproxy` is
-`rproxy` is on the private network, which the usual default of a trusted-proxy
+on the private network, which the usual default of a trusted-proxy setting
-setting covers. Beside the app, its address is `127.0.0.1` or `::1`, which the
+covers. Beside the app, its address is `127.0.0.1` or `::1`, which the private
-private ranges leave out, so the app's own setting must name them too, as
+ranges leave out, so the app's own setting must name them too, as `app.run`
-`app.run` does.
+does.
 
 ### Checking that it works
 
@@ -238,7 +184,7 @@ private ranges leave out, so the app's own setting must name them too, as
   `healthy` for each container of `smallwebwaf`.
 - A log line with `"msg":"starting"` gives the version, the address and every
   setting in effect, with secrets masked.
-- A request to the site through the TLS-terminating proxy gets the app's answer.
+- A request to the site through traefik gets the app's answer.
   `https://<site>/_sww/healthz` answers `ok`. `waf`'s log lines about the
   request have `"action":"forward"`.
 - To see a refusal without banning yourself, run `waf` with
@@ -298,7 +244,7 @@ Every setting the code reads, with its default:
 | `SWW_RPROXY_LISTEN_ADDR`                          | `:8080`                                                                                  | Where `rproxy` listens; leave the host part empty.                                                                                                                                                                                                              |
 | `SWW_RPROXY_TRUSTED_PROXIES`                      | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`                                                | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none.                                                                                                                                                                       |
 | `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`               | `2000`                                                                                   | How long `rproxy` waits for `waf`'s answer to a question and to a report, in milliseconds.                                                                                                                                                                      |
-| `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`    | `allow`                                                                                  | What `rproxy` does with a request when `waf` gives no answer in time, and while the circuit breaker for asks to `waf` is open: `allow` passes it to the app, `deny` answers `503`.                                                                              |
+| `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`    | `allow`                                                                                  | What `rproxy` does with a request when `waf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`.                                                                                                                  |
 | `SWW_RPROXY_MAX_REQUEST_SIZE`                     | `100M`                                                                                   | The largest request body, a size or `off`; a larger one is answered `413`.                                                                                                                                                                                      |
 | `SWW_RPROXY_MAX_RESPONSE_SIZE`                    | `1G`                                                                                     | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started.                                                                                                                        |
 | `SWW_RPROXY_MAX_REQUEST_TIME_MS`                  | `60000`                                                                                  | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app.                                                                                           |
@@ -348,7 +294,7 @@ Every setting the code reads, with its default:
 | `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`           | `5`                                                                                      | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5.                                                                                                                                                     |
 | `SWW_WAF_CORE_RULE_SET_DISABLED_RULES`            | `920340,920420,920440,920640,930130,930140`                                              | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off.                                                                                                                                                |
 | `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS`              | empty                                                                                    | Path prefixes the Core Rule Set does not inspect.                                                                                                                                                                                                               |
-| `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE`      | `off`                                                                                    | How much of a request body the Core Rule Set reads, a size up to `8K`; `off` reads none.                                                                                                                                                                        |
+| `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE`      | `off`                                                                                    | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none.                                                                                                                                                                       |
 | `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE`           | unset                                                                                    | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only.                                                                                                                                                  |
 | `SWW_WAF_APPS`                                    | unset                                                                                    | The apps each hostname serves, such as `git.example.com:gitea;blog.example.com:wordpress`; only those apps' fixes apply there. "Attack detection" lists the apps. Listing `forgejo` brings gitea's fixes too. Unset, every app's fixes apply to every hostname. |
 | `SWW_WAF_BLOCKLIST_URLS`                          | empty                                                                                    | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`.                                                                                                                                     |

SPEC.md

@@ -3,7 +3,7 @@
 `smallwebwaf` is a web application firewall for self-hosted apps. It is one
 binary, `sww`, and two of its commands do the work:
 
-- `sww rproxy` is a reverse proxy between a TLS-terminating proxy and one app.
+- `sww rproxy` is a reverse proxy between traefik and one app.
 - `sww waf` decides what happens to each request, for many sites.
 
 This document says what it must do and why, each requirement once. Everything in
@@ -45,8 +45,7 @@ never as a time.
 
 ## Non-goals
 
-- TLS, certificates and routing by host name. Those are the job of the
+- TLS, certificates and routing by host name. Those are traefik's job.
-  TLS-terminating proxy, such as traefik, nginx or Caddy.
 - More than one app behind one `rproxy`. Each app has its own.
 - Browser challenges such as captchas or proof of work. If wanted, chain Anubis
   between `rproxy` and the app.
@@ -704,12 +703,11 @@ files, the limits and the bans still apply to the client that sends it.
       rules is loaded yet.
 
 - `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` switches body inspection on, up
-  to that size. It takes at most `8K`, so that `waf` answers in time (see
+  to that size. It takes at most `64K`, since `waf` gets no more of a body than
-  "Answering in time"). Form data and multipart bodies are read up to the size.
+  that (see "The answer"). Form data and multipart bodies are read up to the
-  JSON and XML bodies are read only when no larger than the size, since those
+  size. JSON and XML bodies are read only when no larger than the size, since
-  cannot be read in part. A JSON body over 1 KiB is not read either, whatever
+  those cannot be read in part. Any other body is not read: the Core Rule Set
-  the size (see "Answering in time"). Any other body is not read: the Core Rule
+  would read it as form data, where binary content such as a git push trips
-  Set would read it as form data, where binary content such as a git push trips
   rules written for text. A body the Core Rule Set cannot parse adds 5 to the
   score, and so does a multipart body cut by the limit in the middle of a part's
   header. Body inspection suits apps whose forms carry no code.
@@ -1945,10 +1943,10 @@ leave out a cookie only when its name matches exactly, case included.
 ### Ban lengths, response and scope
 
 `SWW_WAF_BAN_RESPONSE` is `403` by default, which makes a ban easy to recognise
-when debugging; `429` and `close` are the alternatives. `close` does not leave
+when debugging; `429` and `close` are the alternatives. Behind traefik, `close`
-the client unanswered: the TLS-terminating proxy answers `502`, as it does
+does not leave the client unanswered: traefik answers `502`, as it does whenever
-whenever its backend drops a connection. `SWW_WAF_BAN_SCOPE_V4_PREFIX` widens an
+its backend drops a connection. `SWW_WAF_BAN_SCOPE_V4_PREFIX` widens an IPv4 ban
-IPv4 ban to the surrounding netblock. The four ban lengths are under "Bans".
+to the surrounding netblock. The four ban lengths are under "Bans".
 
 ### Reputation
 
@@ -2398,7 +2396,9 @@ In `observe` mode a request that `enforce` mode would refuse is answered
 answered `404`.
 
 The Core Rule Set inspects the body a question carries only while
-`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set, and at most 8 KiB of it.
+`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. It inspects at most 64 KiB
+less one byte. So it does not inspect a JSON or XML body of 64 KiB or more,
+which may have been cut short.
 
 ### Answering in time
 
@@ -2473,26 +2473,12 @@ client in `SWW_WAF_ALLOW_NETS` skips them, as it skips the Core Rule Set. In
 `observe` mode such a request is answered `allow` with `would_action`, and no
 other check runs on it.
 
-The Core Rule Set is the one check a visitor can keep busy for long. It runs
+The Core Rule Set runs each of its rules on each cookie, so thousands of small
-each of its rules on each part of a request, so thousands of small cookies would
+cookies would keep it busy for over a second. One 30 KB cookie or query string
-keep it busy for over a second. These fixed limits bound what it reads:
+would keep it busy for a third of a second or more. A request with several long
-
+`Cookie` lines, each within the limits, can keep it as busy. Enough such
-- the header limits above;
+requests at once can still overload `waf` past
-- `rproxy`'s limit of 32 KiB on the request line and headers together (see
+`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
-  "`rproxy`'s limits");
-- 8 KiB of a body, the most `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` takes,
-  or 1 KiB of a JSON body.
-
-Coraza names each value in a JSON body by the whole path to it, such as
-`json.users.0.name`. So the names and values the Core Rule Set reads in a JSON
-body can add up to more than the body. From 1 KiB of JSON, Coraza makes at most
-8 KiB of them, no more than the largest body of another kind.
-
-A request can reach all of these at once. At paranoia level 4, with no rule
-switched off, the Core Rule Set takes under a second on one core of a current
-server to inspect it. So `waf` answers it within the default 2 seconds, even
-after the default 800 milliseconds of waiting for GeoJS. Enough such requests at
-once can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
 
 ### The report
 
@@ -2596,11 +2582,11 @@ request and ban notes name the site by its host.
 ## `rproxy`
 
 `sww rproxy` is the reverse proxy in front of one app. It asks `waf` about each
-request, and passes on what `waf` allows. It writes nothing to disk. The only
+request, and passes on what `waf` allows. It keeps nothing between requests and
-thing it keeps between requests is its circuit breaker for asks to `waf`.
+writes nothing to disk. When `waf` cannot be connected to, does not answer at
-`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides what is done with a
+all, or does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`,
-request when `waf` gives no answer in time, and while the circuit breaker is
+`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides. Its default, `allow`,
-open. Its default, `allow`, keeps the site answering.
+keeps the site answering.
 
 `rproxy` reads only its `SWW_RPROXY_` settings, which `docs/settings.md` lists.
 One of them is required: `SWW_RPROXY_WAF_BACKEND`, `waf`'s URL. It has no
@@ -2618,8 +2604,7 @@ only. The image runs it by default (see "Deployment").
 4. It refuses a request with a method it does not pass on, or one it cannot put
    to `waf` as the visitor sent it (see "Requests `rproxy` refuses itself").
 5. It reads up to the first 64 KiB of the body into memory.
-6. It sends `waf` the question (see "The question"), unless the circuit breaker
+6. It sends `waf` the question (see "The question"). It speaks HTTP/1.1 to
-   keeps it from asking (see "The circuit breaker"). It speaks HTTP/1.1 to
    `waf`, whatever `waf` offers. Go's HTTP/2 client would refuse to send a
    visitor's `Upgrade` header, and most `Connection` headers. The question never
    carries the visitor's `Trailer`.
@@ -2693,24 +2678,23 @@ log line has status `0`.
 ### Without an answer from `waf`
 
 `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides what is done with the
-request when `waf`:
+request only when `waf`:
 
 - cannot be connected to;
 - does not answer at all;
 - does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, which covers
   connecting, sending the question and reading the whole answer.
 
-An answer whose body breaks off counts as no answer at all. The setting also
+An answer whose body breaks off counts as no answer at all.
-decides, without asking `waf`, while the circuit breaker is open.
 
 `allow`, the default, passes the request to the app. `deny` answers `503`.
 Either way `rproxy` writes a process warning, its log line has
 `waf_backend_no_answer`, and it sends no report.
 
-`waf` taking longer than `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` to decide a request
+A visitor can make `waf` slow. A request that keeps `waf` busy past
-is a bug in `waf`. Such a request gets
+`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, like any request while `waf` is overloaded,
-`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`, like any request while `waf` is
+gets `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. With `deny`, such a request
-overloaded. With `deny`, it gets `503` instead of reaching the app.
+gets `503` instead of reaching the app.
 
 A path under `/_sww/` that `allow` would pass on is answered `404` instead, so
 that the app never sees that prefix.
@@ -2725,28 +2709,6 @@ A visitor that goes away while `waf` is being asked is no failure of `waf`.
 `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` does not apply. Its log line has
 status `0` and no `disposition`.
 
-### The circuit breaker
-
-`rproxy` stops asking `waf` for a while when `waf` keeps failing to answer. An
-ask fails when `waf` gives no answer in time (see "Without an answer from
-`waf`"), or an answer `rproxy` cannot pass on. Any other answer in time
-succeeds, whatever it decides. A visitor that goes away while `waf` is being
-asked counts as neither.
-
-The breaker works like this:
-
-- It opens when 5 asks fail within 5 seconds. Fewer failures never open it.
-- While it is open, a request gets
-  `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` at once, without asking `waf`.
-  It gets the same process warning and `waf_backend_no_answer` as a request
-  whose ask failed.
-- While it is open, one request a second still asks `waf`.
-- Each ask that succeeds clears the failures counted so far, and closes the
-  breaker if it is open.
-
-The breaker lives in memory, and a restart of `rproxy` closes it. It has no
-settings.
-
 ### Passing a request to the app
 
 `rproxy` sends the app at `SWW_RPROXY_UPSTREAM_URL` the visitor's request: the
@@ -2809,9 +2771,9 @@ its whole body. The visitor gets that answer at once, and the connection closes
 after it.
 
 The largest request line and headers is 32 KiB. A kept-open connection is closed
-after 120 seconds idle. Neither has a setting. The TLS-terminating proxy should
+after 120 seconds idle. Neither has a setting. traefik, by default, closes an
-close an unused connection sooner, so that it never sends a request on a
+unused connection after 90 seconds. So traefik closes first, and never sends a
-connection `rproxy` is closing. traefik closes one after 90 seconds by default.
+request on a connection `rproxy` is closing.
 
 What the visitor gets when a limit is passed:
 
@@ -2862,16 +2824,14 @@ chooses: `waf` runs `waf`, and with it unset the image runs `rproxy`.
 The usual setup is three containers on one private network:
 
 - the app;
-- `rproxy`, which the TLS-terminating proxy forwards to, with
+- `rproxy`, which traefik routes to, with `SWW_RPROXY_UPSTREAM_URL` naming the
-  `SWW_RPROXY_UPSTREAM_URL` naming the app and `SWW_RPROXY_WAF_BACKEND` naming
+  app and `SWW_RPROXY_WAF_BACKEND` naming `waf`;
-  `waf`;
 - `waf`, with `SWW_COMMAND=waf` and a volume for its state files.
 
 Each further app adds its own container and its own `rproxy`, whose
 `SWW_RPROXY_WAF_BACKEND` names the same `waf` (see "Sites"). Anything on that
 network can reach an app around `rproxy`, and can ask `waf` questions or send it
-reports. So the network holds only what you trust. `README.md` lists what the
+reports. So the network holds only what you trust.
-TLS-terminating proxy must do.
 
 ### `rproxy` in the app's container
 
@@ -2943,10 +2903,10 @@ process that stops at start is started again every second, not in a tight loop.
   container's output, which `docker logs` shows.
 - When a service exits, runit runs its `run` script again, and the container
   does not stop. Beside an app, the other process keeps running.
-- While `rproxy` is down, the TLS-terminating proxy answers `502`. While the app
+- While `rproxy` is down, traefik answers `502`. While the app is down, `rproxy`
-  is down, `rproxy` answers `502`, which is not a client offence. While `waf` is
+  answers `502`, which is not a client offence. While `waf` is down,
-  down, `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an
+  `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an answer
-  answer from `waf`").
+  from `waf`").
 - Only faults found at start stop `waf`:
     - invalid configuration;
     - a lookup database that is missing or unreadable;
@@ -2979,26 +2939,23 @@ checks:
 - With `SWW_COMMAND` unset, empty or `rproxy`, it passes while `rproxy` answers
   that request on `127.0.0.1`, at the port in `SWW_RPROXY_LISTEN_ADDR`, and the
   app accepts connections at `SWW_RPROXY_UPSTREAM_URL`. It does not ask `waf`,
-  so that the site stays in the TLS-terminating proxy's routes while `waf` is
+  so that the site stays in traefik's routes while `waf` is down.
-  down.
 - With any other value, it fails with a message naming `SWW_COMMAND` and its
   value.
 
-A TLS-terminating proxy that reads Docker's health status, as traefik does,
+traefik sends a container no requests until it shows as healthy. So the check
-sends a container no requests until it shows as healthy. So the check runs every
+runs every second from the container's start until it first passes, for up to a
-second from the container's start until it first passes, for up to a minute, and
+minute, and every 30 seconds after that. An app with a health check of its own
-every 30 seconds after that. An app with a health check of its own can replace
+can replace the image's `HEALTHCHECK` with one that checks both.
-the image's `HEALTHCHECK` with one that checks both.
 
 ### Ports
 
 `rproxy` listens on no port but the one in `SWW_RPROXY_LISTEN_ADDR`, 8080 by
-default, which the TLS-terminating proxy forwards to. It passes requests to the
+default, which traefik routes to. It passes requests to the app at
-app at `SWW_RPROXY_UPSTREAM_URL`, `http://127.0.0.1:8081` by default. Another
+`SWW_RPROXY_UPSTREAM_URL`, `http://127.0.0.1:8081` by default. Another port may
-port may be set: the health check follows the setting, and the TLS-terminating
+be set: the health check follows the setting, and traefik's port label
-proxy must name it, as traefik's port label
+(`traefik.http.services.<name>.loadbalancer.server.port`) must name it. An app
-(`traefik.http.services.<name>.loadbalancer.server.port`) does. An app beside
+beside `rproxy` must leave that port free.
-`rproxy` must leave that port free.
 
 `waf` listens on no port but the one in `SWW_WAF_LISTEN_ADDR`, 8080 by default.
 The questions, the reports and its own endpoints all come there.
@@ -3020,8 +2977,8 @@ host directory, keeps them across deploys. A host directory mounted there needs
 no change of owner, and files an earlier owner left in it can be read and
 replaced. The volume holds a few tens of MiB at most with the defaults, and
 needs no backup beyond the host's own. The image declares no volume, since every
-app image built on it would inherit it. `rproxy` writes nothing to disk, and
+app image built on it would inherit it. `rproxy` keeps nothing, and needs no
-needs no volume.
+volume.
 
 `waf` keeps its cache files in `SWW_WAF_CACHE_DIR`, `/var/cache/sww` by default,
 which the `run` script makes. The cache directory needs no volume and can be
@@ -3039,9 +2996,9 @@ in `docs/settings.md` shows the commands.
 ### Forwarded headers
 
 The app's TCP peer is `rproxy`, and the `X-Forwarded-For` the app receives ends
-with the TLS-terminating proxy's address, which `rproxy` adds after the
+with traefik's address, which `rproxy` adds after the visitor's. So the app must
-visitor's. So the app must trust `rproxy`'s address for forwarded headers,
+trust `rproxy`'s address for forwarded headers, besides the private address
-besides the private address ranges the TLS-terminating proxy is on.
+ranges traefik is on.
 
 - In a container of its own, `rproxy`'s address is on the private network. The
   usual default for a trusted-proxy setting, the private ranges, covers it.

docs/request-log.md

@@ -68,8 +68,7 @@ Every one of these lines has:
 ### Request detail
 
 - `request_id`: the id `rproxy` made for the request when it arrived.
-- `peer_ip`: the address of `rproxy`'s TCP peer, normally the TLS-terminating
+- `peer_ip`: the address of `rproxy`'s TCP peer, normally traefik.
-  proxy.
 - `forwarded_for`: the `X-Forwarded-For` header the visitor's request came with.
 - `client_group`: the client as rate limiting counts it, an IPv4 address as a
   /32 or an IPv6 group.
@@ -238,8 +237,7 @@ meanings where they differ:
 - `disposition`: what was done with the request: `allow`, `deny` or `close`. It
   is `waf`'s answer, or the value of
   `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. It is left out when:
-    - `waf` was not asked and `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` did
+    - `waf` was not asked, as for the health check;
-      not apply, as for the health check;
     - the visitor went away while `waf` was being asked;
     - `waf`'s answer had no valid `X-Sww-Disposition`;
     - `rproxy` could not pass on `waf`'s answer, and answered `502`.

docs/settings.md

@@ -251,9 +251,10 @@ tag that no request can get stops the start. A request can get only these:
 - `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS` (default empty): path prefixes the Core
   Rule Set does not inspect.
 - `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` (default `off`): `off`, or how
-  much of a request body the Core Rule Set reads, a size of at most `8K`. A
+  much of a request body the Core Rule Set reads, a size of at most `64K`. A
-  larger size would let one request keep it busy past `rproxy`'s timeout (see
+  question carries no more of a body than 64 KiB, and the Core Rule Set reads at
-  "Answering in time" in [`SPEC.md`](../SPEC.md)).
+  most 64 KiB less one byte of it, whatever the value (see "The answer" in
+  [`SPEC.md`](../SPEC.md)).
 - `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule
   Set exclusions, written as Coraza directives. It is loaded before the Core
   Rule Set's rules, so it holds only `SecAction` and `SecRule` lines that
@@ -452,9 +453,10 @@ start, and an invalid value stops the start. Any can be given as a file, as a
   for `waf`'s answer, to a question and to a report. A time in milliseconds;
   never `off`.
 - `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` (default `allow`): what is done
-  with a request when `waf` gives no answer in time, and while the circuit
+  with a request when `waf` cannot be connected to, does not answer at all, or
-  breaker for asks to `waf` is open. `allow` passes it to the app, and `deny`
+  does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`. `allow` passes it
-  answers `503`. It does not apply to an answer `waf` gives, such as an error.
+  to the app, and `deny` answers `503`. It does not apply to an answer `waf`
+  gives, such as an error.
 - `SWW_RPROXY_MAX_REQUEST_SIZE` (default `100M`): the largest request body. A
   size or `off`.
 - `SWW_RPROXY_MAX_RESPONSE_SIZE` (default `1G`): the largest response body. A
@@ -476,9 +478,9 @@ start, and an invalid value stops the start. Any can be given as a file, as a
   `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`): the netblocks whose
   `X-Forwarded-For` is believed. Set but empty, it trusts nothing. A client
   outside these netblocks cannot claim another address. Where other containers
-  reach `rproxy` directly, or visitors on a private network reach the
+  reach `rproxy` directly, or visitors on a private network reach traefik, they
-  TLS-terminating proxy, they can name another address in the header. Narrowing
+  can name another address in the header. Narrowing the setting to traefik's own
-  the setting to the TLS-terminating proxy's own network closes that gap.
+  network closes that gap.
 
 ## The image's command
 

internal/config/waf.go

@@ -365,8 +365,8 @@ var (
 		"it names the file to look clients up in",
 	)
 	errDBPathUnused = errors.New("only file reads it")
-	errOver8K       = errors.New(
+	errOver64K      = errors.New(
-		"is more than 8K, the most the Core Rule Set can read in time")
+		"is more than 64K, the most waf gets of a body")
 	errNotDurationAboveZero = errors.New(
 		"is not a duration above zero, such as 1h or 7d")
 	errNotNumberAboveZero = errors.New(
@@ -1601,16 +1601,16 @@ func durationOrDays(value string) (time.Duration, error) {
 }
 
 // parseCoreRuleSetBodyInspectionSize reads the most of a request body the Core Rule Set
-// reads: a size as parseSize reads it, or off, but at most 8K, so that one
+// reads: a size as parseSize reads it, or off, but at most 64K, since waf
-// request cannot keep the Core Rule Set busy past rproxy's timeout.
+// gets no more of a body than that.
 func parseCoreRuleSetBodyInspectionSize(value string) (int64, error) {
 	limit, err := parseSize(value)
 	if err != nil {
 		return 0, err
 	}
 
-	if limit > 8*kibibyte {
+	if limit > 64*kibibyte {
-		return 0, fmt.Errorf("%q %w", value, errOver8K)
+		return 0, fmt.Errorf("%q %w", value, errOver64K)
 	}
 
 	return limit, nil

internal/config/waf_test.go

@@ -616,7 +616,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
 				coreRuleSetScoreThreshold:     "10",
 				coreRuleSetDisabledRules:      "942100, 920350",
 				coreRuleSetExemptPaths:        "/api/, /static/",
-				coreRuleSetBodyInspectionSize: "4K",
+				coreRuleSetBodyInspectionSize: "32K",
 				exclusionsFile:                "/etc/sww/crs-exclusions.conf",
 			},
 			config.WAF{
@@ -625,7 +625,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
 				CoreRuleSetScoreThreshold:     10,
 				CoreRuleSetDisabledRules:      []int{942100, 920350},
 				CoreRuleSetExemptPaths:        []string{"/api/", "/static/"},
-				CoreRuleSetBodyInspectionSize: 4 << 10,
+				CoreRuleSetBodyInspectionSize: 32 << 10,
 				CoreRuleSetExclusionsFile:     "/etc/sww/crs-exclusions.conf",
 			},
 		},
@@ -762,15 +762,15 @@ func TestInvalidAppsStopsTheStart(t *testing.T) {
 	}
 }
 
-func TestCoreRuleSetBodyInspectionSizeOf8K(t *testing.T) {
+func TestCoreRuleSetBodyInspectionSizeOf64K(t *testing.T) {
 	t.Parallel()
 
 	cfg := wafFromEnvironment(
 		t,
-		environment{coreRuleSetBodyInspectionSize: "8K"},
+		environment{coreRuleSetBodyInspectionSize: "64K"},
 	)
-	if cfg.CoreRuleSetBodyInspectionSize != 8<<10 {
+	if cfg.CoreRuleSetBodyInspectionSize != 64<<10 {
-		t.Errorf("8K read as %d", cfg.CoreRuleSetBodyInspectionSize)
+		t.Errorf("64K read as %d", cfg.CoreRuleSetBodyInspectionSize)
 	}
 }
 
@@ -819,13 +819,11 @@ func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
 			coreRuleSetBodyInspectionSize, "128KB",
 			`"128KB" is not a size such as 512K, 100M or 5G, or off`,
 		},
+		{coreRuleSetBodyInspectionSize, "65K", `"65K" is more than 64K, the most waf ` +
+			`gets of a body`},
 		{
-			coreRuleSetBodyInspectionSize, "9K",
+			coreRuleSetBodyInspectionSize, "65537",
-			`"9K" is more than 8K, the most the Core Rule Set can read in time`,
+			`"65537" is more than 64K, the most waf gets of a body`,
-		},
-		{
-			coreRuleSetBodyInspectionSize, "8193",
-			`"8193" is more than 8K, the most the Core Rule Set can read in time`,
 		},
 	} {
 		t.Run(tc.name+"="+tc.value, func(t *testing.T) {

internal/coreruleset/coreruleset.go

@@ -481,21 +481,12 @@ func (c *CoreRuleSet) Inspect(
 	return Result{RuleIDs: ids, Score: score(tx)}, read, err
 }
 
-// mostJSONRead is the largest JSON body the Core Rule Set reads, even while
-// SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is larger. Coraza names each
-// value in a JSON body by the whole path to it, so what it gives the Core
-// Rule Set to read can outgrow the body: up to 8 times its size, and up to
-// 4 KiB for a small body. 1 KiB of JSON thus gives at most 8 KiB, no more
-// than the largest body of another kind.
-const mostJSONRead = 1 << 10
-
 // readBody reads body, the body of the request in tx, which has run on
 // the request's headers, while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is
 // set and the body is of a kind the Core Rule Set reads: form data and
 // multipart, of which it reads the first c.bodyInspectionSize bytes, and
 // JSON and XML, which it reads only when they are no longer than that,
-// since they cannot be read in part. The Core Rule Set reads no JSON body
+// since they cannot be read in part.
-// longer than mostJSONRead either.
 // readBody reads one byte past c.bodyInspectionSize, to tell which they
 // are, gives the Core Rule Set what it reads, and returns what it read and
 // the error that ended the reading early, if one did.
@@ -508,8 +499,6 @@ func (c *CoreRuleSet) readBody(
 	}
 
 	inPart := false
-	// largest is the largest JSON or XML body the Core Rule Set reads.
-	largest := c.bodyInspectionSize
 	// How the body is read is a variable of the transaction, which only
 	// Coraza's interface for plugins reads.
 	state := tx.(plugintypes.TransactionState) //nolint:forcetypeassert // every one is
@@ -517,16 +506,14 @@ func (c *CoreRuleSet) readBody(
 	switch state.Variables().RequestBodyProcessor().Get() {
 	case "URLENCODED", "MULTIPART":
 		inPart = true
-	case "JSON":
+	case "JSON", "XML":
-		largest = min(largest, mostJSONRead)
-	case "XML":
 	default:
 		return nil, nil
 	}
 
 	read, err := io.ReadAll(io.LimitReader(body, c.bodyInspectionSize+1))
 
-	if inPart || (err == nil && int64(len(read)) <= largest) {
+	if inPart || (err == nil && int64(len(read)) <= c.bodyInspectionSize) {
 		// Coraza holds what it reads of the body in memory, up to the
 		// limit, so this cannot fail.
 		_, _, _ = tx.WriteRequestBody(read)

internal/coreruleset/coreruleset_test.go

@@ -390,15 +390,22 @@ func TestMoreParametersThanCorazaKeepsIsAMatch(t *testing.T) {
 		matched(900300),
 	)
 
-	// So it is with the fields of a form data body. A JSON body the Core
+	// So it is with the fields of a form data or JSON body.
-	// Rule Set reads, of at most mostJSONRead bytes, cannot hold that many.
 	crs = readingBodies(t)
 
-	body := strings.Repeat("a=1&", 999) + attack
+	for _, tc := range []struct{ header, body string }{
-	wantBody(t, crs, post(formData), body, matched(942100), body)
+		{formData, strings.Repeat("a=1&", 999) + attack},
+		{jsonBody, `{"a":[` + strings.Repeat("1,", 998) + `1],"id":"` + injection + `"}`},
+	} {
+		wantBody(t, crs, post(tc.header), tc.body, matched(942100), tc.body)
+	}
 
-	body = strings.Repeat("a=1&", 1000) + attack
+	for _, tc := range []struct{ header, body string }{
-	wantBody(t, crs, post(formData), body, matched(900300), body)
+		{formData, strings.Repeat("a=1&", 1000) + attack},
+		{jsonBody, `{"a":[` + strings.Repeat("1,", 999) + `1],"id":"` + injection + `"}`},
+	} {
+		wantBody(t, crs, post(tc.header), tc.body, matched(900300), tc.body)
+	}
 }
 
 func TestRedirectURIMayNameALocalAddress(t *testing.T) {
@@ -1505,15 +1512,13 @@ const (
 	wpTags    = "/wp-json/wp/v2/tags"
 )
 
-// readingBodiesUpTo8K returns the Core Rule Set as sww runs it by default,
+// readingBodiesUpTo64K returns the Core Rule Set as sww runs it by default,
 // but reading bodies up to the largest SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE.
-func readingBodiesUpTo8K(t *testing.T) *coreruleset.CoreRuleSet {
+func readingBodiesUpTo64K(t *testing.T) *coreruleset.CoreRuleSet {
 	t.Helper()
 
 	return shared(t, coreruleset.Params{
-		ParanoiaLevel:      1,
+		ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: 64<<10 - 1,
-		DisabledRules:      defaultDisabledRules,
-		BodyInspectionSize: mostBodyRead,
 	})
 }
 
@@ -1577,7 +1582,7 @@ func TestWordPressAttacksAreRefused(t *testing.T) {
 
 	const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
 
-	defaults, bodies := atDefaults(t), readingBodiesUpTo8K(t)
+	defaults, bodies := atDefaults(t), readingBodiesUpTo64K(t)
 
 	for _, attack := range []string{
 		sqlInjection,
@@ -1688,7 +1693,7 @@ func wordpressRESTAddresses(route string) []string {
 func TestWordPressExclusionsFollowEveryRESTAddress(t *testing.T) {
 	t.Parallel()
 
-	crs := readingBodiesUpTo8K(t)
+	crs := readingBodiesUpTo64K(t)
 	content := `{"content":"` + wordpressBlock + `"}`
 
 	for _, route := range []string{
@@ -1914,7 +1919,7 @@ func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(
 		wantResult(t, crs, get(target+command), matched(932235))
 	}
 
-	bodies := readingBodiesUpTo8K(t)
+	bodies := readingBodiesUpTo64K(t)
 
 	for _, p := range wordpressBodyFields(command) {
 		r := request{http.MethodPost, p.target, []string{p.contentType}}
@@ -2138,7 +2143,7 @@ func TestWordPressBodyFieldsAreCheckedInTheQuery(t *testing.T) {
 func TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere(t *testing.T) {
 	t.Parallel()
 
-	crs := readingBodiesUpTo8K(t)
+	crs := readingBodiesUpTo64K(t)
 
 	for _, v := range wordpressWords() {
 		for _, field := range []string{"title", "excerpt", "name"} {
@@ -2257,7 +2262,7 @@ func TestWordPressExclusionsAreNarrow(t *testing.T) {
 		length  = "Content-Length: 2"
 	)
 
-	crs := readingBodiesUpTo8K(t)
+	crs := readingBodiesUpTo64K(t)
 	saving := request{
 		http.MethodPost,
 		wpPost,
@@ -2468,35 +2473,19 @@ func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
 func TestJSONAndXMLAreReadOnlyWhenNoLargerThanTheLimit(t *testing.T) {
 	t.Parallel()
 
-	// The limit is the body inspection size, but at most mostJSONRead for
+	crs := readingBodies(t)
-	// JSON. smallSize is a body inspection size under mostJSONRead.
-	const smallSize = 512
 
-	bodies, smallBodies := readingBodies(t), shared(t, coreruleset.Params{
+	for _, tc := range []struct{ header, head, tail string }{
-		ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
+		{jsonBody, `{"q":"` + injection + `","pad":"`, `"}`},
-		BodyInspectionSize: smallSize,
+		{xmlBody, "<r><q>" + injection + "</q><pad>", "</pad></r>"},
-	})
-	jsonHead, jsonTail := `{"q":"`+injection+`","pad":"`, `"}`
-
-	for _, tc := range []struct {
-		crs                *coreruleset.CoreRuleSet
-		header, head, tail string
-		limit              int
 	} {
-		{bodies, jsonBody, jsonHead, jsonTail, mostJSONRead},
+		fits := padded(tc.head, tc.tail, bodyInspectionSize)
-		{smallBodies, jsonBody, jsonHead, jsonTail, smallSize},
+		wantBody(t, crs, post(tc.header), fits, matched(942100), fits)
-		{
-			bodies, xmlBody, "<r><q>" + injection + "</q><pad>", "</pad></r>",
-			bodyInspectionSize,
-		},
-	} {
-		fits := padded(tc.head, tc.tail, tc.limit)
-		wantBody(t, tc.crs, post(tc.header), fits, matched(942100), fits)
 
-		larger := padded(tc.head, tc.tail, tc.limit+1)
+		larger := padded(tc.head, tc.tail, bodyInspectionSize+1)
 		wantBody(
 			t,
-			tc.crs,
+			crs,
 			post(tc.header),
 			larger,
 			coreruleset.Result{},

internal/coreruleset/inspectiontime_test.go

@@ -1,371 +0,0 @@
-package coreruleset_test
-
-import (
-	"maps"
-	"net/http"
-	"net/http/httptest"
-	"net/netip"
-	"slices"
-	"strconv"
-	"strings"
-	"testing"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/coreruleset"
-)
-
-// The bounds on what a visitor can send, as SPEC.md gives them.
-const (
-	// lineLimit is the longest request line or header line.
-	lineLimit = 8 << 10
-	// headerLimit is the most bytes rproxy reads of a request line and
-	// headers together.
-	headerLimit = 32 << 10
-	// mostCookies and mostHeaderFields are waf's limits on cookies and header
-	// fields.
-	mostCookies      = 50
-	mostHeaderFields = 100
-	// mostBodyRead is the largest SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE.
-	mostBodyRead = 8 << 10
-	// mostJSONRead is the largest JSON body the Core Rule Set reads.
-	mostJSONRead = 1 << 10
-)
-
-// inspectionTimeLimit is the most time the Core Rule Set may take on one
-// request. It is the default of rproxy's SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, 2
-// seconds, less the default of SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, 800
-// milliseconds, which a question can also wait for GeoJS, and less 200
-// milliseconds for waf's other checks.
-const inspectionTimeLimit = time.Second
-
-// text returns n bytes of letters, digits and spaces, which the Core Rule Set
-// takes longer over than one letter repeated.
-func text(n int) string {
-	const words = "select 0x2f from tbl_1 where name like abc_9 "
-
-	return strings.Repeat(words, n/len(words)+1)[:n]
-}
-
-// visitorRequest is a request a visitor can send, at the most of one or more
-// of the bounds above.
-type visitorRequest struct {
-	name   string
-	target string
-	// header holds the headers besides Host.
-	header http.Header
-	// contentType is the body's type, "" for no body.
-	contentType string
-	body        string
-}
-
-// longestTarget returns a target whose request line, "POST <target>
-// HTTP/1.1", is lineLimit bytes long.
-func longestTarget() string {
-	const around = len("POST /?q= HTTP/1.1")
-
-	return "/?q=" + strings.ReplaceAll(text(lineLimit-around), " ", "+")
-}
-
-// cookieLine returns a Cookie value whose line, "Cookie: <value>", is
-// lineLimit bytes long, made of count cookies.
-func cookieLine(count int) string {
-	const linePrefix = len("Cookie: ")
-
-	cookies := make([]string, count)
-	for i := range cookies {
-		cookies[i] = "c" + strconv.Itoa(i) + "="
-	}
-
-	// Each cookie is followed by "; ", but for the last.
-	room := lineLimit - linePrefix - len(strings.Join(cookies, "; "))
-	for i := range cookies {
-		share := room / count
-		if i == 0 {
-			share += room % count
-		}
-
-		cookies[i] += strings.ReplaceAll(text(share), " ", "_")
-	}
-
-	return strings.Join(cookies, "; ")
-}
-
-// headerValue returns a value whose line, "<name>: <value>", is n bytes long.
-func headerValue(name string, n int) string {
-	return text(n - len(name) - len(": "))
-}
-
-// formFields returns a form body of mostBodyRead bytes holding count fields.
-func formFields(count int) string {
-	fields := make([]string, count)
-	for i := range fields {
-		fields[i] = "f" + strconv.Itoa(i) + "="
-	}
-
-	room := mostBodyRead - len(strings.Join(fields, "&"))
-	for i := range fields {
-		share := room / count
-		if i == 0 {
-			share += room % count
-		}
-
-		fields[i] += strings.ReplaceAll(text(share), " ", "+")
-	}
-
-	return strings.Join(fields, "&")
-}
-
-// multipartBody returns a multipart body of mostBodyRead bytes, with the
-// boundary "b", holding fields of about size bytes each.
-func multipartBody(size int) string {
-	var body strings.Builder
-
-	const end = "--b--\r\n"
-
-	for i := 0; ; i++ {
-		head := "--b\r\nContent-Disposition: form-data; name=\"f" +
-			strconv.Itoa(i) + "\"\r\n\r\n"
-
-		room := mostBodyRead - body.Len() - len(
-			end,
-		) - len(
-			head,
-		) - len(
-			"\r\n",
-		)
-		if room <= 0 {
-			break
-		}
-
-		body.WriteString(head + text(min(size, room)) + "\r\n")
-	}
-
-	body.WriteString(end)
-
-	return body.String()
-}
-
-// jsonArrayOfMostElements returns a JSON array of at most mostJSONRead bytes
-// holding as many elements as fit, each the number 1.
-func jsonArrayOfMostElements() string {
-	return "[1" + strings.Repeat(
-		",1",
-		(mostJSONRead-len("[1]"))/len(",1"),
-	) + "]"
-}
-
-// jsonOfMostElementsUnderAKey returns a JSON body of mostJSONRead bytes
-// holding as many elements as fit, each the number 1, in an array under a
-// key of 6 bytes. Coraza names each element "json.<key>.<index>", so a
-// longer key makes longer names. 6 bytes is the longest key that keeps the
-// names and values of this body within the 8 KiB Coraza allows.
-func jsonOfMostElementsUnderAKey() string {
-	head := `{"` + text(6) + `":[1`
-
-	return head + strings.Repeat(
-		",1",
-		(mostJSONRead-len(head+"]}"))/len(",1"),
-	) + "]}"
-}
-
-// The kinds of body the Core Rule Set reads, and the header that names a
-// request's cookies.
-const (
-	formType      = "application/x-www-form-urlencoded"
-	jsonType      = "application/json"
-	multipartType = "multipart/form-data; boundary=b"
-	cookieHeader  = "Cookie"
-)
-
-// fullLine is the length of each of the request line and the three or four
-// header lines that together fill rproxy's limit, with room for the line
-// endings and the other headers.
-const fullLine = lineLimit - 64
-
-// worstCaseRequests are, for each kind of thing a visitor sends, the requests
-// that keep the Core Rule Set busiest, within the bounds above. The last
-// holds the most of everything at once: Cookie lines keep it busier than
-// other header lines, and the JSON body of the most elements under a key
-// keeps it busier than the other bodies.
-func worstCaseRequests() []visitorRequest {
-	fullCookieLine := cookieLine(1)[:fullLine-len("Cookie: ")]
-
-	return append(
-		append(worstCaseHeaders(), worstCaseBodies()...),
-		visitorRequest{
-			name:   "the longest request line, three Cookie lines and body",
-			target: longestTarget()[:fullLine-len("POST  HTTP/1.1")],
-			header: http.Header{cookieHeader: {
-				fullCookieLine, fullCookieLine, fullCookieLine,
-			}},
-			contentType: jsonType, body: jsonOfMostElementsUnderAKey(),
-		},
-	)
-}
-
-// worstCaseHeaders are the requests of worstCaseRequests with no body.
-func worstCaseHeaders() []visitorRequest {
-	headerFields := http.Header{}
-
-	for i := range mostHeaderFields - 1 {
-		name := "X-Field-" + strconv.Itoa(i)
-		headerFields.Set(
-			name,
-			headerValue(name, headerLimit/mostHeaderFields),
-		)
-	}
-
-	return []visitorRequest{
-		{name: "the longest request line", target: longestTarget()},
-		{
-			name: "the longest request line, as query parameters",
-			target: "/?" + strings.Repeat(
-				"a=1&",
-				(lineLimit-len("POST /? HTTP/1.1"))/len("a=1&"),
-			),
-		},
-		{
-			name:   "the longest Cookie line, as one cookie",
-			target: "/",
-			header: http.Header{cookieHeader: {cookieLine(1)}},
-		},
-		{
-			name:   "the longest Cookie line, as the most cookies",
-			target: "/",
-			header: http.Header{cookieHeader: {cookieLine(mostCookies)}},
-		},
-		{name: "the most header fields", target: "/", header: headerFields},
-		{
-			name:   "the longest header lines",
-			target: "/",
-			header: http.Header{
-				"User-Agent": {headerValue("User-Agent", fullLine)},
-				"Referer":    {headerValue("Referer", fullLine)},
-				"Accept":     {headerValue("Accept", fullLine)},
-				"X-Long":     {headerValue("X-Long", fullLine)},
-			},
-		},
-	}
-}
-
-// worstCaseBodies are the requests of worstCaseRequests whose body holds
-// the most the Core Rule Set reads.
-func worstCaseBodies() []visitorRequest {
-	const nested = (mostJSONRead - len(`{"a":1}`)) / len(`{"a":}`)
-
-	return []visitorRequest{
-		{
-			name: "a form body of one field", target: "/",
-			contentType: formType, body: formFields(1),
-		},
-		{
-			name: "a form body of the most fields Coraza keeps", target: "/",
-			contentType: formType, body: formFields(1000),
-		},
-		{
-			name: "a JSON body of one string", target: "/",
-			contentType: jsonType,
-			body: `{"a":"` + text(
-				mostJSONRead-len(`{"a":""}`),
-			) + `"}`,
-		},
-		{
-			name: "a JSON body of the most elements", target: "/",
-			contentType: jsonType, body: jsonArrayOfMostElements(),
-		},
-		{
-			name: "a JSON body of the most elements under a key", target: "/",
-			contentType: jsonType, body: jsonOfMostElementsUnderAKey(),
-		},
-		{
-			name: "a JSON body nested deep", target: "/",
-			contentType: jsonType,
-			body: strings.Repeat(`{"a":`, nested) + "1" +
-				strings.Repeat("}", nested),
-		},
-		{
-			name: "an XML body of many elements", target: "/",
-			contentType: "application/xml",
-			body: "<a>" + strings.Repeat(
-				"<b>x</b>",
-				(mostBodyRead-len("<a></a>"))/len("<b>x</b>"),
-			) + "</a>",
-		},
-		{
-			name: "a multipart body of one field", target: "/",
-			contentType: multipartType, body: multipartBody(mostBodyRead),
-		},
-		{
-			name: "a multipart body of many fields", target: "/",
-			contentType: multipartType, body: multipartBody(1),
-		},
-	}
-}
-
-// inspectionTime returns how long crs takes to inspect tc, and fails t unless
-// it reads all of tc's body and Coraza parses it.
-func inspectionTime(
-	t *testing.T, crs *coreruleset.CoreRuleSet, tc visitorRequest,
-) time.Duration {
-	t.Helper()
-
-	req := httptest.NewRequestWithContext(
-		t.Context(), http.MethodPost, tc.target, nil,
-	)
-	maps.Copy(req.Header, tc.header)
-
-	if tc.contentType != "" {
-		req.Header.Set("Content-Type", tc.contentType)
-	}
-
-	began := time.Now()
-	result, read, _ := crs.Inspect(
-		req, netip.MustParseAddr("192.0.2.1"), strings.NewReader(tc.body),
-	)
-	took := time.Since(began)
-
-	if string(read) != tc.body {
-		t.Errorf("read %d bytes of %d", len(read), len(tc.body))
-	}
-
-	// Rule 900440 matches a body Coraza could not parse.
-	if slices.Contains(result.RuleIDs, 900440) {
-		t.Errorf("Coraza could not parse the body")
-	}
-
-	return took
-}
-
-// TestTheCoreRuleSetInspectsEachWorstCaseRequestInTime runs one at a time, so
-// that the tests running in parallel do not slow it. Other work on the host
-// can still slow one try, so a request fails only when three tries in a row
-// take too long.
-//
-//nolint:paralleltest // one at a time, as above
-func TestTheCoreRuleSetInspectsEachWorstCaseRequestInTime(t *testing.T) {
-	const tries = 3
-
-	// The settings that make the Core Rule Set slowest: the highest paranoia
-	// level, no rule switched off, and the most of a body read.
-	crs := shared(t, coreruleset.Params{
-		ParanoiaLevel: 4, BodyInspectionSize: mostBodyRead,
-	})
-
-	for _, tc := range worstCaseRequests() {
-		t.Run(tc.name, func(t *testing.T) {
-			var took time.Duration
-
-			for range tries {
-				took = inspectionTime(t, crs, tc)
-				if took <= inspectionTimeLimit {
-					return
-				}
-			}
-
-			t.Errorf(
-				"took %s on the last of %d tries, over %s",
-				took, tries, inspectionTimeLimit,
-			)
-		})
-	}
-}

internal/rproxy/breaker.go

@@ -1,82 +0,0 @@
-package rproxy
-
-import (
-	"errors"
-	"sync"
-	"time"
-)
-
-// What opens the circuit breaker, and how often a request asks waf while it is
-// open.
-const (
-	breakerFailures    = 5
-	breakerWindow      = 5 * time.Second
-	breakerAskInterval = time.Second
-)
-
-// errBreakerOpen is why a request did not ask waf.
-var errBreakerOpen = errors.New(
-	"waf is not asked while the circuit breaker is open",
-)
-
-// breaker is the circuit breaker for asks to waf. It opens when
-// breakerFailures asks fail within breakerWindow. An ask fails when it gets no
-// answer in time, or an answer rproxy cannot pass on. While it is open, at
-// most one request each breakerAskInterval asks waf. Any usable answer closes
-// it, whatever it decides, and forgets the failures before it.
-type breaker struct {
-	mu sync.Mutex
-	// failures holds the times of the latest failed asks, oldest first, at most
-	// breakerFailures of them.
-	failures []time.Time
-	open     bool
-	// lastAsk is when a request last asked waf while the breaker was open.
-	lastAsk time.Time
-}
-
-// mayAsk reports whether a request may ask waf at now. While the breaker is
-// open, it says yes once each breakerAskInterval.
-func (b *breaker) mayAsk(now time.Time) bool {
-	b.mu.Lock()
-	defer b.mu.Unlock()
-
-	if !b.open {
-		return true
-	}
-
-	if now.Sub(b.lastAsk) < breakerAskInterval {
-		return false
-	}
-
-	b.lastAsk = now
-
-	return true
-}
-
-// noteAnswer notes that an ask got an answer rproxy can use. It closes the
-// breaker and forgets the failures before it.
-func (b *breaker) noteAnswer() {
-	b.mu.Lock()
-	defer b.mu.Unlock()
-
-	b.open = false
-	b.failures = nil
-}
-
-// noteFailure notes that an ask that ended at now got no answer in time, and
-// opens the breaker if breakerFailures asks have failed within breakerWindow.
-func (b *breaker) noteFailure(now time.Time) {
-	b.mu.Lock()
-	defer b.mu.Unlock()
-
-	b.failures = append(b.failures, now)
-	if len(b.failures) > breakerFailures {
-		b.failures = b.failures[1:]
-	}
-
-	if !b.open && len(b.failures) == breakerFailures &&
-		now.Sub(b.failures[0]) <= breakerWindow {
-		b.open = true
-		b.lastAsk = now
-	}
-}

internal/rproxy/breaker_test.go

@@ -1,142 +0,0 @@
-package rproxy_test
-
-import (
-	"testing"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/rproxy"
-)
-
-// The breaker tests give the breaker each time themselves, counted from
-// time.Now(), and never wait.
-
-// fail notes count failed asks on b, one each step from at.
-func fail(b *rproxy.Breaker, at time.Time, count int, step time.Duration) {
-	for i := range count {
-		b.NoteFailure(at.Add(time.Duration(i) * step))
-	}
-}
-
-// openBreaker returns a breaker opened at now by five failed asks, one a
-// second.
-func openBreaker(now time.Time) *rproxy.Breaker {
-	b := &rproxy.Breaker{}
-	fail(b, now.Add(-4*time.Second), 5, time.Second)
-
-	return b
-}
-
-func TestFourFailedAsksWithinFiveSecondsLeaveTheBreakerClosed(t *testing.T) {
-	t.Parallel()
-
-	now := time.Now()
-	b := &rproxy.Breaker{}
-	fail(b, now, 4, time.Second)
-
-	if !b.MayAsk(now.Add(3 * time.Second)) {
-		t.Error("a request may not ask, want it to")
-	}
-}
-
-func TestFiveFailedAsksWithinFiveSecondsOpenTheBreaker(t *testing.T) {
-	t.Parallel()
-
-	now := time.Now()
-	b := openBreaker(now)
-
-	if b.MayAsk(now) {
-		t.Error("a request may ask, want it not to")
-	}
-}
-
-func TestFiveFailedAsksOverMoreThanFiveSecondsLeaveTheBreakerClosed(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	now := time.Now()
-	b := &rproxy.Breaker{}
-	fail(b, now, 5, 1300*time.Millisecond)
-
-	if !b.MayAsk(now.Add(6 * time.Second)) {
-		t.Error("a request may not ask, want it to")
-	}
-}
-
-func TestAnAnswerBetweenFailedAsksKeepsTheBreakerClosed(t *testing.T) {
-	t.Parallel()
-
-	now := time.Now()
-	b := &rproxy.Breaker{}
-	fail(b, now, 4, 100*time.Millisecond)
-	b.NoteAnswer()
-	fail(b, now.Add(500*time.Millisecond), 4, 100*time.Millisecond)
-
-	if !b.MayAsk(now.Add(900 * time.Millisecond)) {
-		t.Error("a request may not ask, want it to")
-	}
-}
-
-func TestAnOpenBreakerLetsOneRequestAskEachSecond(t *testing.T) {
-	t.Parallel()
-
-	now := time.Now()
-	b := openBreaker(now)
-
-	for _, tc := range []struct {
-		after time.Duration
-		want  bool
-	}{
-		{999 * time.Millisecond, false},
-		{time.Second, true},
-		{time.Second, false},
-		{1500 * time.Millisecond, false},
-		{2 * time.Second, true},
-	} {
-		got := b.MayAsk(now.Add(tc.after))
-		if got != tc.want {
-			t.Errorf("may ask %v after it opened: %v, want %v",
-				tc.after, got, tc.want)
-		}
-	}
-}
-
-func TestAFailedAskKeepsTheBreakerOpen(t *testing.T) {
-	t.Parallel()
-
-	opened := time.Now()
-	b := openBreaker(opened)
-	now := opened.Add(time.Second)
-	b.MayAsk(now)
-	b.NoteFailure(now)
-
-	if b.MayAsk(now.Add(999 * time.Millisecond)) {
-		t.Error("a request may ask, want it not to")
-	}
-
-	if !b.MayAsk(now.Add(time.Second)) {
-		t.Error("a request may not ask a second later, want it to")
-	}
-}
-
-func TestOneAnswerClosesTheBreaker(t *testing.T) {
-	t.Parallel()
-
-	opened := time.Now()
-	b := openBreaker(opened)
-	now := opened.Add(time.Second)
-	b.MayAsk(now)
-	b.NoteAnswer()
-
-	if !b.MayAsk(now) {
-		t.Error("a request may not ask, want it to")
-	}
-
-	// The failures before the answer no longer count, so one more does not open
-	// the breaker again.
-	b.NoteFailure(now)
-
-	if !b.MayAsk(now) {
-		t.Error("a request may not ask after one failure, want it to")
-	}
-}

internal/rproxy/export_test.go

@@ -1,23 +0,0 @@
-package rproxy
-
-import "time"
-
-// Breaker is the circuit breaker, for tests that give it the time.
-type Breaker struct {
-	b breaker
-}
-
-// MayAsk reports whether a request may ask waf at now.
-func (b *Breaker) MayAsk(now time.Time) bool {
-	return b.b.mayAsk(now)
-}
-
-// NoteAnswer notes that an ask got an answer rproxy can use.
-func (b *Breaker) NoteAnswer() {
-	b.b.noteAnswer()
-}
-
-// NoteFailure notes that an ask that ended at now got no answer in time.
-func (b *Breaker) NoteFailure(now time.Time) {
-	b.b.noteFailure(now)
-}

internal/rproxy/rproxy.go

@@ -4,11 +4,10 @@
 // waf goes to the visitor as waf gave it.
 // SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION decides only when waf cannot
 // be connected to, does not answer at all, or does not answer within
-// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, or while the circuit breaker for asks to
+// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. rproxy itself refuses a request with a
-// waf is open. rproxy itself refuses a request with a method it does not pass
+// method it does not pass on, or one it cannot put to waf as the visitor
-// on, or one it cannot put to waf as the visitor sent it. It keeps nothing
+// sent it. It keeps nothing between requests and writes nothing to disk. "rproxy" in
-// between requests but the circuit breaker, and writes nothing to disk.
+// SPEC.md says what it does.
-// "rproxy" in SPEC.md says what it does.
 package rproxy
 
 import (
@@ -133,8 +132,6 @@ type handler struct {
 	wafTransport http.RoundTripper
 	// reportURL is where the reports go.
 	reportURL string
-	// breaker is the circuit breaker for asks to waf.
-	breaker breaker
 	// reports waits for the reports being sent. reportsMu guards starting one
 	// against reportsStopped, which is set once nothing is to wait for another.
 	reports        sync.WaitGroup
@@ -199,7 +196,7 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	answer, err := rq.askUnlessBreakerOpen(r.Context())
+	answer, err := rq.ask(r.Context())
 	if err != nil {
 		rq.handleNoUsableAnswer(r.Context(), err)
 

internal/rproxy/rproxy_test.go

@@ -15,7 +15,6 @@ import (
 	"strconv"
 	"strings"
 	"sync"
-	"sync/atomic"
 	"testing"
 	"time"
 
@@ -1396,139 +1395,6 @@ func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
 	standIn.wantNoReport(t)
 }
 
-func TestFiveFailedAsksKeepTheNextRequestFromAskingWAF(t *testing.T) {
-	t.Parallel()
-
-	for name, failure := range map[string]http.HandlerFunc{
-		"no answer at all": func(http.ResponseWriter, *http.Request) {
-			panic(http.ErrAbortHandler)
-		},
-		"a slow answer": func(_ http.ResponseWriter, r *http.Request) {
-			<-r.Context().Done()
-		},
-	} {
-		t.Run(name, func(t *testing.T) {
-			t.Parallel()
-
-			var answering atomic.Bool
-
-			standIn := startWAF(
-				t,
-				func(w http.ResponseWriter, r *http.Request) {
-					if answering.Load() {
-						allow(w, r)
-
-						return
-					}
-
-					failure(w, r)
-				},
-			)
-			app := startApp(t, greet)
-			addr, out := startRProxy(
-				t,
-				standIn.URL,
-				app.URL,
-				map[string]string{
-					wafBackendTimeout: "200",
-				},
-			)
-
-			for range 5 {
-				wantGreeting(t, get(t, addr, "/"))
-				standIn.question(t)
-			}
-
-			answering.Store(true)
-			wantGreeting(t, get(t, addr, "/"))
-			out.waitFor(t, "the circuit breaker is open")
-
-			select {
-			case <-standIn.questions:
-				t.Error("waf was asked, want it not to be")
-			case <-time.After(10 * pollInterval):
-			}
-		})
-	}
-}
-
-func TestVisitorsThatGoAwayWhileWAFIsAskedLeaveTheBreakerClosed(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	var answering atomic.Bool
-
-	standIn := startWAF(t, func(w http.ResponseWriter, r *http.Request) {
-		if answering.Load() {
-			allow(w, r)
-
-			return
-		}
-
-		<-r.Context().Done()
-	})
-	app := startApp(t, greet)
-	addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
-		wafBackendTimeout: "60000",
-	})
-
-	// Five of these would open the breaker if they counted as failures.
-	for range 5 {
-		conn := dial(t, addr)
-		send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
-		id := standIn.question(t).header.Get(requestIDHeader)
-
-		_ = conn.Close()
-
-		// rproxy writes the request's log line once it is done with the ask.
-		out.waitFor(t, id)
-	}
-
-	answering.Store(true)
-	wantGreeting(t, get(t, addr, "/"))
-	standIn.question(t)
-}
-
-func TestAnswersThatCannotBePassedOnOpenTheBreaker(t *testing.T) {
-	t.Parallel()
-
-	var answering atomic.Bool
-
-	tooLarge := overOneMiB(dispositionDeny)
-	standIn := startWAF(t, func(w http.ResponseWriter, r *http.Request) {
-		if answering.Load() {
-			allow(w, r)
-
-			return
-		}
-
-		tooLarge(w, r)
-	})
-	app := startApp(t, greet)
-	addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
-		wafBackendTimeout: "60000",
-	})
-
-	for range 5 {
-		if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
-			t.Errorf("answered %d, want 502", got.status)
-		}
-
-		standIn.question(t)
-	}
-
-	answering.Store(true)
-	wantGreeting(t, get(t, addr, "/"))
-	out.waitFor(t, "the circuit breaker is open")
-
-	select {
-	case <-standIn.questions:
-		t.Error("waf was asked, want it not to be")
-	case <-time.After(10 * pollInterval):
-	}
-}
-
 func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) {
 	t.Parallel()
 

internal/rproxy/waf.go

@@ -10,7 +10,6 @@ import (
 	"maps"
 	"net/http"
 	"strings"
-	"time"
 
 	"sneak.berlin/go/smallwebwaf/internal/waf"
 )
@@ -68,8 +67,7 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
 		rq.answer(http.StatusBadGateway)
 	default:
 		// waf cannot be connected to, does not answer at all, or does not answer
-		// within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS; or the circuit breaker
+		// within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. A request that keeps waf
-		// kept the request from asking. A request that keeps waf
 		// busy past SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, like any request while
 		// waf is overloaded, gets SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION
 		// here. deny closes this, at the cost of answering 503 whenever waf is
@@ -88,31 +86,6 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
 	}
 }
 
-// askUnlessBreakerOpen asks waf, unless the circuit breaker is open and it is
-// not yet time for a request to ask. It then tells the breaker whether waf
-// answered in time. An answer rproxy cannot pass on counts as a failure. A
-// visitor that goes away tells the breaker nothing. ctx is the request's own
-// context.
-func (rq *request) askUnlessBreakerOpen(
-	ctx context.Context,
-) (*wafAnswer, error) {
-	b := &rq.h.breaker
-	if !b.mayAsk(time.Now()) {
-		return nil, errBreakerOpen
-	}
-
-	answer, err := rq.ask(ctx)
-
-	switch {
-	case err == nil:
-		b.noteAnswer()
-	case ctx.Err() == nil:
-		b.noteFailure(time.Now())
-	}
-
-	return answer, err
-}
-
 // ask sends waf the question, and returns its answer, which must have come
 // whole within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. It ends when ctx does, as
 // when the visitor goes away.

internal/sww/rproxy_test.go

@@ -260,17 +260,7 @@ func TestTheSiteAnswersWhileWAFIsStoppedAndStartedAgain(t *testing.T) {
 		stop()
 	}()
 
-	// The visitor's asks have opened rproxy's circuit breaker, which lets one
+	wantStatus(t, site+"/", placed, http.StatusForbidden)
-	// request a second ask until one is answered.
-	deadline := time.Now().Add(waitLimit)
-	for statusFrom(t, site+"/", placed) != http.StatusForbidden {
-		if time.Now().After(deadline) {
-			t.Fatalf("request from %s: never answered %d", placed,
-				http.StatusForbidden)
-		}
-
-		time.Sleep(pollInterval)
-	}
 
 	close(visitorStop)
 	<-visitorDone

internal/waf/proxy.go

@@ -228,9 +228,12 @@ func newReputation(
 // without the rules SWW_WAF_CORE_RULE_SET_DISABLED_RULES switches off, reading bodies
 // up to SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE, with the exclusions in
 // SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE and the fixes of the apps in SWW_WAF_APPS,
-// or nil while SWW_WAF_CORE_RULE_SET_MODE is off. An exclusions file that
+// or nil while SWW_WAF_CORE_RULE_SET_MODE is off. It
-// coreruleset.New refuses is an error in every mode. While the mode is off, a
+// reads at most QuestionBodyMaxBytes less one byte: a question's body of
-// set file is loaded with the Core Rule Set only to check it.
+// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
+// JSON or XML body over its limit. An exclusions file that coreruleset.New refuses is
+// an error in every mode. While the mode is off, a set file is loaded with the
+// Core Rule Set only to check it.
 func NewCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
 	if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff &&
 		cfg.CoreRuleSetExclusionsFile == "" {
@@ -240,7 +243,10 @@ func NewCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
 	coreRuleSet, err := coreruleset.New(coreruleset.Params{
 		ParanoiaLevel: cfg.CoreRuleSetParanoiaLevel,
 		DisabledRules: cfg.CoreRuleSetDisabledRules,
-		BodyInspectionSize: cfg.CoreRuleSetBodyInspectionSize,
+		BodyInspectionSize: min(
+			cfg.CoreRuleSetBodyInspectionSize,
+			QuestionBodyMaxBytes-1,
+		),
 		ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
 		Apps:           cfg.Apps,
 	})

internal/waf/waf_test.go

@@ -1787,7 +1787,7 @@ func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) {
 	t.Parallel()
 
 	s := startWAF(t, "", map[string]string{
-		coreRuleSetMode: block, coreRuleSetBodyInspectionSize: "8K",
+		coreRuleSetMode: block, coreRuleSetBodyInspectionSize: "64K",
 	})
 
 	ask := func(contentType, body string) answer {
@@ -1802,10 +1802,10 @@ func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) {
 	got := ask(formData, "id=1'%20OR%20'1'='1")
 	wantDisposition(t, got, http.StatusForbidden, waf.DispositionDeny)
 
-	// The first 8 KiB of a longer JSON body cannot be parsed, which is no
+	// The first 64 KiB of a longer JSON body cannot be parsed, which is no
 	// attack, so the Core Rule Set does not read it.
-	cut := `{"text":"` + strings.Repeat("x", 8<<10)
+	cut := `{"text":"` + strings.Repeat("x", waf.QuestionBodyMaxBytes)
-	got = ask("application/json", cut)
+	got = ask("application/json", cut[:waf.QuestionBodyMaxBytes])
 	wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
 }