waf: decide every visitor request within rproxy's timeout (closes #394)

The Core Rule Set is the one check whose time a visitor can stretch.
At paranoia level 4 it took over 2 s on a request with an 8 KiB
request line, three 8 KiB Cookie lines and a 64 KiB form body.
SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE now takes at most 8K, which
brings that request under 1 s.

A test per kind of input times the Core Rule Set on the worst request
within the limits, at the slowest settings. SPEC.md lists the limits
that bound it.

Model: opus-5-5

README.md

@@ -2,7 +2,20 @@
 
 `smallwebwaf` is an MIT-licensed web application firewall, written in Go by
 [@sneak](https://sneak.berlin), for self-hosted web apps. It refuses the
-requests of scrapers, scanners and other abusers before they reach an app.
+requests of misbehaving scrapers, scanners and other abusers before they reach
+an app.
+
+With the default settings, a client is banned when it does any of these:
+
+- makes more than 120 requests in a minute, 1500 in an hour or 6000 in a day;
+- moves more than `50G` of request and response bodies in an hour, or `100G` in
+  a day;
+- asks for a secret or backup file, such as `/.env` or `/.git/config`;
+- sends a user agent that names a security scanner, such as `sqlmap` or `nikto`;
+- has more than 30 requests in a minute refused as probes or attacks.
+
+A scraper that does none of these is left alone. A scraper that waits 15 seconds
+between requests stays under the rate limits.
 
 ## What it does
 
@@ -10,10 +23,10 @@ A web application firewall reads each request on its way to a web app and
 refuses the ones that look like abuse. `smallwebwaf` is one program, `sww`,
 shipped in one image, with two commands that do this together:
 
-- `sww rproxy` is a small reverse proxy in front of one app. traefik routes to
-  it. It asks `waf` about each request, passes the allowed ones to the app, and
-  passes the app's answers back. It keeps nothing between requests and writes
-  nothing to disk.
+- `sww rproxy` is a small reverse proxy in front of one app. It sits behind a
+  TLS-terminating proxy, such as traefik, nginx or Caddy. It asks `waf` about
+  each request, passes the allowed ones to the app, and passes the app's answers
+  back. It writes nothing to disk.
 - `sww waf` decides. It runs the checks and keeps the bans, the counters and
   each client's history. One `waf` serves many sites: a client banned through
   one site is refused on all of them, and its requests on all of them count
@@ -22,7 +35,26 @@ shipped in one image, with two commands that do this together:
 The site keeps answering while `waf` is down, slow or restarting: without an
 answer in time, `rproxy` passes the request to the app.
 
-It protects against:
+`rproxy` also protects the app on its own, whether or not `waf` answers. With
+the default settings, `rproxy`:
+
+- answers `400` to a malformed request, such as a broken header line or a
+  missing `Host`;
+- answers `405` to a method it does not pass on, such as `TRACE`;
+- answers `431` to a request line and headers over 32 KiB together;
+- gives the visitor 60 seconds to send the request line and headers, and then 60
+  seconds more for the whole request to reach the app;
+- answers `413` to a request body over 100 MiB;
+- gives the app's answer 30 minutes to reach the visitor, counted from the end
+  of the request;
+- answers `502` in place of an app's answer whose body is over 1 GiB, or cuts it
+  off if it has started;
+- closes a kept-open connection after 120 seconds idle.
+
+`rproxy` does not limit the number of headers. `waf` limits the number of
+headers and cookies, and the length of the request line and of each header line.
+
+Together, `rproxy` and `waf` protect against:
 
 - Scrapers that fetch every page of a site, all day. Each client gets a limit on
   requests and on body bytes per minute, per hour and per day. The defaults
@@ -43,10 +75,7 @@ It protects against:
     - give the clients of chosen networks and countries a fraction of every
       limit, so that they are banned sooner;
     - refuse or limit clients on a blocklist, a DNS blocklist, or with a high
-      AbuseIPDB score;
-    - ban clients named by your own CrowdSec engine.
-- Slow and oversized requests. Every request and every answer has a time limit
-  and a size limit.
+      AbuseIPDB score.
 
 For the operator, it:
 
@@ -58,8 +87,7 @@ For the operator, it:
 
 Not built, and not planned:
 
-- TLS and routing by host name. Both stay with traefik, the reverse proxy in
-  front of `rproxy`.
+- TLS and routing by host name. Both stay with the TLS-terminating proxy.
 - Browser challenges such as captchas.
 - A web console, or a configuration file.
 - Defence against floods large enough to fill the host's network link.
@@ -91,20 +119,61 @@ accepts connections, whether or not `waf` answers; in `waf`'s container, once
 The usual setup is three containers on one private network:
 
 - the app;
-- `rproxy`, in front of the app, which traefik routes to:
+- `rproxy`, in front of the app, which the TLS-terminating proxy forwards to:
   `SWW_RPROXY_WAF_BACKEND` names `waf` and `SWW_RPROXY_UPSTREAM_URL` names the
   app;
 - `waf`, with `SWW_COMMAND=waf` and a volume at `/var/lib/sww` for its state
   files.
 
-Point traefik's labels at port 8080 of `rproxy`
-(`traefik.http.services.<name>.loadbalancer.server.port=8080`). Publish no port
-of the app or of `waf`. Anything on that network can reach the app around
-`rproxy`, and can ask `waf` questions, so the network holds only what you trust.
-Each further app adds its own container and its own `rproxy`, whose
+Publish no port of the app or of `waf`. Anything on that network can reach the
+app around `rproxy`, and can ask `waf` questions, so the network holds only what
+you trust. Each further app adds its own container and its own `rproxy`, whose
 `SWW_RPROXY_WAF_BACKEND` names the same `waf`. The "Smallest setup" example
 below shows this shape.
 
+### The TLS-terminating proxy
+
+`rproxy` serves plain HTTP. It runs behind any TLS-terminating proxy, such as
+traefik, nginx or Caddy. The TLS-terminating proxy must:
+
+- forward each request to port 8080 of `rproxy` over plain HTTP;
+- keep the visitor's `Host` header;
+- add the visitor's address to `X-Forwarded-For`;
+- set `X-Forwarded-Proto` to the scheme the visitor used;
+- close an unused connection to `rproxy` in under 120 seconds;
+- connect to `rproxy` from an address in `SWW_RPROXY_TRUSTED_PROXIES`, by
+  default `10.0.0.0/8`, `172.16.0.0/12` or `192.168.0.0/16`.
+
+From any other address, `rproxy` ignores the forwarded headers and takes the
+TLS-terminating proxy for the visitor. Every visitor then shares one address,
+and one ban.
+
+traefik and Caddy keep the `Host` header and set both forwarded headers by
+default. traefik's Docker label
+`traefik.http.services.<name>.loadbalancer.server.port=8080` names the port.
+Caddy keeps an unused connection for 120 seconds by default, so its site
+shortens that to 90 seconds:
+
+```caddyfile
+reverse_proxy rproxy:8080 {
+    transport http {
+        keepalive 90s
+    }
+}
+```
+
+By default nginx sends the host in `proxy_pass` as `Host` and sets neither
+forwarded header, so its site sets all three:
+
+```nginx
+location / {
+    proxy_pass http://rproxy:8080;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_set_header X-Forwarded-Proto $scheme;
+}
+```
+
 ### `rproxy` in the app's container
 
 An app's image can instead be built on the `smallwebwaf` image, so that `rproxy`
@@ -155,13 +224,13 @@ entrypoint starts `rproxy` and the app side by side, each as its own user. Run
 the app's image with `SWW_RPROXY_WAF_BACKEND` set. `deploy/example-app` is such
 an app, and the "`rproxy` beside the app" example below runs one.
 
-The app sees the headers traefik sent, with traefik's address added to
-`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
-`rproxy`'s address for forwarded headers. In a container of its own, `rproxy` is
-on the private network, which the usual default of a trusted-proxy setting
-covers. Beside the app, its address is `127.0.0.1` or `::1`, which the private
-ranges leave out, so the app's own setting must name them too, as `app.run`
-does.
+The app's TCP peer is `rproxy`. The app sees the headers the TLS-terminating
+proxy sent, with that proxy's address added to `X-Forwarded-For`. So the app
+must trust `rproxy`'s address for forwarded headers. In a container of its own,
+`rproxy` is on the private network, which the usual default of a trusted-proxy
+setting covers. Beside the app, its address is `127.0.0.1` or `::1`, which the
+private ranges leave out, so the app's own setting must name them too, as
+`app.run` does.
 
 ### Checking that it works
 
@@ -169,7 +238,7 @@ does.
   `healthy` for each container of `smallwebwaf`.
 - A log line with `"msg":"starting"` gives the version, the address and every
   setting in effect, with secrets masked.
-- A request to the site through traefik gets the app's answer.
+- A request to the site through the TLS-terminating proxy gets the app's answer.
   `https://<site>/_sww/healthz` answers `ok`. `waf`'s log lines about the
   request have `"action":"forward"`.
 - To see a refusal without banning yourself, run `waf` with
@@ -229,7 +298,7 @@ Every setting the code reads, with its default:
 | `SWW_RPROXY_LISTEN_ADDR`                          | `:8080`                                                                                  | Where `rproxy` listens; leave the host part empty.                                                                                                                                                                                                              |
 | `SWW_RPROXY_TRUSTED_PROXIES`                      | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`                                                | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none.                                                                                                                                                                       |
 | `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`               | `2000`                                                                                   | How long `rproxy` waits for `waf`'s answer to a question and to a report, in milliseconds.                                                                                                                                                                      |
-| `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`    | `allow`                                                                                  | What `rproxy` does with a request when `waf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`.                                                                                                                  |
+| `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`    | `allow`                                                                                  | What `rproxy` does with a request when `waf` gives no answer in time, and while the circuit breaker for asks to `waf` is open: `allow` passes it to the app, `deny` answers `503`.                                                                              |
 | `SWW_RPROXY_MAX_REQUEST_SIZE`                     | `100M`                                                                                   | The largest request body, a size or `off`; a larger one is answered `413`.                                                                                                                                                                                      |
 | `SWW_RPROXY_MAX_RESPONSE_SIZE`                    | `1G`                                                                                     | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started.                                                                                                                        |
 | `SWW_RPROXY_MAX_REQUEST_TIME_MS`                  | `60000`                                                                                  | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app.                                                                                           |
@@ -261,7 +330,7 @@ Every setting the code reads, with its default:
 | `SWW_WAF_LOOKUP_SOURCE`                           | `geojs`                                                                                  | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`.                                                                                                                                               |
 | `SWW_WAF_LOOKUP_DB_PATH`                          | empty                                                                                    | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`.                                                                                                                                                                            |
 | `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`       | `800`                                                                                    | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take, in milliseconds.                                                                                                                |
-| `SWW_WAF_ADD_LOOKUP_HEADERS`                      | `false`                                                                                  | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`.                                                                                                                                                              |
+| `SWW_WAF_ADD_LOOKUP_HEADERS`                      | `true`, or `false` while the lookup source is `off`                                      | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`.                                                                                                                                                              |
 | `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES`           | empty                                                                                    | When set, the only countries whose clients get through; a client the lookup cannot place is refused too.                                                                                                                                                        |
 | `SWW_WAF_TAG_LISTS`                               | empty                                                                                    | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`.                                                                                                                               |
 | `SWW_WAF_TAG_DENY`                                | empty                                                                                    | Tags whose requests are refused, such as `torexit,country=CN`.                                                                                                                                                                                                  |
@@ -279,7 +348,7 @@ Every setting the code reads, with its default:
 | `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`           | `5`                                                                                      | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5.                                                                                                                                                     |
 | `SWW_WAF_CORE_RULE_SET_DISABLED_RULES`            | `920340,920420,920440,920640,930130,930140`                                              | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off.                                                                                                                                                |
 | `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS`              | empty                                                                                    | Path prefixes the Core Rule Set does not inspect.                                                                                                                                                                                                               |
-| `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE`      | `off`                                                                                    | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none.                                                                                                                                                                       |
+| `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE`      | `off`                                                                                    | How much of a request body the Core Rule Set reads, a size up to `8K`; `off` reads none.                                                                                                                                                                        |
 | `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE`           | unset                                                                                    | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only.                                                                                                                                                  |
 | `SWW_WAF_APPS`                                    | unset                                                                                    | The apps each hostname serves, such as `git.example.com:gitea;blog.example.com:wordpress`; only those apps' fixes apply there. "Attack detection" lists the apps. Listing `forgejo` brings gitea's fixes too. Unset, every app's fixes apply to every hostname. |
 | `SWW_WAF_BLOCKLIST_URLS`                          | empty                                                                                    | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`.                                                                                                                                     |
@@ -290,8 +359,6 @@ Every setting the code reads, with its default:
 | `SWW_WAF_ABUSEIPDB_KEY`                           | unset                                                                                    | The key of your AbuseIPDB account; set, a client that has committed an offence is checked in the background.                                                                                                                                                    |
 | `SWW_WAF_ABUSEIPDB_MIN_SCORE`                     | `75`                                                                                     | The least AbuseIPDB abuse confidence score, from 0 to 100, that is a hit.                                                                                                                                                                                       |
 | `SWW_WAF_ABUSEIPDB_DAILY_BUDGET`                  | `900`                                                                                    | The most AbuseIPDB checks made in a day.                                                                                                                                                                                                                        |
-| `SWW_WAF_CROWDSEC_LAPI_URL`                       | unset                                                                                    | The local API of a CrowdSec engine you run, such as `http://172.17.0.1:8080`; its decision list is fetched every minute, and a listed client is banned.                                                                                                         |
-| `SWW_WAF_CROWDSEC_LAPI_KEY`                       | unset                                                                                    | The key the engine gave for `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it; set with the URL, never alone.                                                                                                                                        |
 | `SWW_WAF_DNSBL_ABUSEIPDB_ACTION`                  | `limit:25`                                                                               | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:<percent>` or `log`.                                                                                                                                                |
 | `SWW_WAF_IP_REPUTATION_CACHE_TTL`                 | `24h`                                                                                    | How long a DNS blocklist verdict or an AbuseIPDB score is used.                                                                                                                                                                                                 |
 | `SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`              | `2000`                                                                                   | How long a DNS blocklist query or an AbuseIPDB check may take, in milliseconds.                                                                                                                                                                                 |
@@ -571,8 +638,7 @@ Each request gets tags from what `waf` knows of its client, written in the
 - `asn=<number>` and `country=<code>`, from the lookup;
 - one tag per list of `SWW_WAF_TAG_LISTS` that holds the client, such as
   `torexit` or `vpn=mullvad`;
-- `crowdsec`, `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists
-  the client;
+- `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists the client;
 - `exploitsignature`, when a rule file or the Core Rule Set matched the request;
 - `asn=<number>` and `prefix=<netblock>`, from a routewatch instance, while
   `SWW_WAF_ROUTEWATCH_URL` is set.

SPEC.md

@@ -3,7 +3,7 @@
 `smallwebwaf` is a web application firewall for self-hosted apps. It is one
 binary, `sww`, and two of its commands do the work:
 
-- `sww rproxy` is a reverse proxy between traefik and one app.
+- `sww rproxy` is a reverse proxy between a TLS-terminating proxy and one app.
 - `sww waf` decides what happens to each request, for many sites.
 
 This document says what it must do and why, each requirement once. Everything in
@@ -45,7 +45,8 @@ never as a time.
 
 ## Non-goals
 
-- TLS, certificates and routing by host name. Those are traefik's job.
+- TLS, certificates and routing by host name. Those are the job of the
+  TLS-terminating proxy, such as traefik, nginx or Caddy.
 - More than one app behind one `rproxy`. Each app has its own.
 - Browser challenges such as captchas or proof of work. If wanted, chain Anubis
   between `rproxy` and the app.
@@ -161,22 +162,21 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
    (`deny_nets`); a client in both lists is let through.
 5. Bans. A client under a ban is refused (`active_ban`; see "Bans").
 6. The lookup, unless `SWW_WAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
-   and country"). The request goes on at once. It waits for the first GeoJS
-   answer, up to `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, only when one of
-   these needs it:
+   and country"). The request waits for the first GeoJS answer, up to
+   `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, when one of these needs it:
+    - `SWW_WAF_ADD_LOOKUP_HEADERS`, which is `true` by default;
     - the country list;
     - a biased threshold;
     - `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming an `asn=` or
-      `country=` tag;
-    - `SWW_WAF_ADD_LOOKUP_HEADERS`.
+      `country=` tag.
 
-    A request that ends before the answer comes is logged without it.
+    Otherwise the request goes on at once. A request that ends before the answer
+    comes is logged without it.
 
 7. The country list (`countries`; see "Country list").
 8. The reputation sources (see "Reputation"), in this order:
     - A client on a fetched blocklist gets `SWW_WAF_BLOCKLIST_ACTION`
       (`blocklists`).
-    - A client on the CrowdSec decision list is banned (`crowdsec`).
     - A client listed by a DNS blocklist gets `SWW_WAF_DNSBL_ABUSEIPDB_ACTION`
       (`dnsbl`).
     - A client scored at least `SWW_WAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB gets
@@ -198,10 +198,10 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
 16. A path under `/_sww/` is answered `404`, so that the app never sees that
     prefix.
 17. `rproxy` passes the request to `SWW_RPROXY_UPSTREAM_URL`, streaming (see
-    "Passing a request to the app"). With `SWW_WAF_ADD_LOOKUP_HEADERS`,
-    `X-Client-ASN` and `X-Client-Country` are added for the app's own logs; a
-    header whose value is unknown is left out. Whatever that setting says, any
-    such header the client sent is removed.
+    "Passing a request to the app"). While `SWW_WAF_ADD_LOOKUP_HEADERS` is
+    `true`, as by default, `X-Client-ASN` and `X-Client-Country` are added for
+    the app's own logs; a header whose value is unknown is left out. Whatever
+    that setting says, any such header the client sent is removed.
 18. After the visitor's answer has ended, `rproxy` sends the report (see "The
     report"). `waf` then counts:
     - For a request it allowed, the body bytes of the response, of the request
@@ -301,14 +301,6 @@ without a ban, because the Core Rule Set has false positives. A `block` rule
 does the same. Those refusals count toward the refusal limit, so a client that
 keeps setting them off is banned for a broken limit.
 
-A client on the CrowdSec decision list is banned until the decision ends, with
-the cause `crowdsec`. Only a client that sends a request is banned, so a long
-list does not fill `bans.json`. Such a ban is never made permanent. It does not
-make the netblock's next ban for a broken limit longer. A ban lifted while the
-decision is in force is made again at the client's next request. To lift it for
-good, delete the decision in CrowdSec, then lift the ban once the list has been
-fetched again.
-
 These refuse each request they cover, record nothing in `bans.json`, and count
 no offence:
 
@@ -338,10 +330,10 @@ toward `SWW_WAF_MAX_BANS`, so such bans can never fill the table.
 
 A request refused under a ban that runs out is logged with the action
 `temporarily_banned`. Under a permanent ban the action is `permanently_banned`.
-A request refused for a clear sign of attack or a CrowdSec decision takes the
-action for the ban it made. A request that makes a ban permanent is logged as
-`permanently_banned`. A request over a rate limit is logged as
-`client_is_rate_limited`, whichever ban it makes.
+A request refused for a clear sign of attack takes the action for the ban it
+made. A request that makes a ban permanent is logged as `permanently_banned`. A
+request over a rate limit is logged as `client_is_rate_limited`, whichever ban
+it makes.
 
 ## Settings
 
@@ -575,9 +567,9 @@ Each tag and where it comes from:
 - One tag per list of `SWW_WAF_TAG_LISTS` that holds the client (`tag_lists`),
   the tag being the list's name, such as `torexit` or `vpn=mullvad`. These lists
   are fetched and kept like a blocklist. `waf` ships none of its own.
-- `crowdsec`, `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists
-  the client, whatever the source's action. A zone with a key is named with the
-  key masked, as the `reputation` field gives it.
+- `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists the client,
+  whatever the source's action. A zone with a key is named with the key masked,
+  as the `reputation` field gives it.
 - `exploitsignature`, when a rule file or the Core Rule Set matched the request.
   For the Core Rule Set a match is a score at or over
   `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`.
@@ -712,13 +704,13 @@ files, the limits and the bans still apply to the client that sends it.
       rules is loaded yet.
 
 - `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` switches body inspection on, up
-  to that size. It takes at most `64K`, since `waf` gets no more of a body than
-  that (see "The answer"). Form data and multipart bodies are read up to the
-  size. JSON and XML bodies are read only when no larger than the size, since
-  those cannot be read in part. Any other body is not read: the Core Rule Set
-  would read it as form data, where binary content such as a git push trips
-  rules written for text. A body the Core Rule Set cannot parse adds 5 to the
-  score, and so does a multipart body cut by the limit in the middle of a part's
+  to that size. It takes at most `8K`, so that `waf` answers in time (see
+  "Answering in time"). Form data and multipart bodies are read up to the size.
+  JSON and XML bodies are read only when no larger than the size, since those
+  cannot be read in part. Any other body is not read: the Core Rule Set would
+  read it as form data, where binary content such as a git push trips rules
+  written for text. A body the Core Rule Set cannot parse adds 5 to the score,
+  and so does a multipart body cut by the limit in the middle of a part's
   header. Body inspection suits apps whose forms carry no code.
 - `SWW_WAF_TRAP_PATHS` names paths the app never serves and only scanners ask
   for, such as `/wp-login.php` in front of an app that is not WordPress. This is
@@ -1952,20 +1944,19 @@ leave out a cookie only when its name matches exactly, case included.
 ### Ban lengths, response and scope
 
 `SWW_WAF_BAN_RESPONSE` is `403` by default, which makes a ban easy to recognise
-when debugging; `429` and `close` are the alternatives. Behind traefik, `close`
-does not leave the client unanswered: traefik answers `502`, as it does whenever
-its backend drops a connection. `SWW_WAF_BAN_SCOPE_V4_PREFIX` widens an IPv4 ban
-to the surrounding netblock. The four ban lengths are under "Bans".
+when debugging; `429` and `close` are the alternatives. `close` does not leave
+the client unanswered: the TLS-terminating proxy answers `502`, as it does
+whenever its backend drops a connection. `SWW_WAF_BAN_SCOPE_V4_PREFIX` widens an
+IPv4 ban to the surrounding netblock. The four ban lengths are under "Bans".
 
 ### Reputation
 
 No source is on by default. A source acts on a list kept elsewhere; the defaults
-judge a client only by what it does here. AbuseIPDB and CrowdSec also need an
-account or an engine of the operator's own. The list best suited to be on by
-default, Spamhaus DROP, has terms, below, that forbid frequent fetching, and
-Spamhaus may block an address that breaks them. Each `waf` fetches its own copy,
-so several of them on one host, sharing one address, would often fetch too close
-together.
+judge a client only by what it does here. AbuseIPDB also needs an account of the
+operator's own. The list best suited to be on by default, Spamhaus DROP, has
+terms, below, that forbid frequent fetching, and Spamhaus may block an address
+that breaks them. Each `waf` fetches its own copy, so several of them on one
+host, sharing one address, would often fetch too close together.
 
 - Blocklists (`SWW_WAF_BLOCKLIST_URLS`) are text files of addresses and
   netblocks, one to a line, as the Spamhaus DROP list is written. Anything after
@@ -1998,18 +1989,6 @@ together.
   percentage of every request and byte limit, and suits a list of addresses
   shared with ordinary visitors, such as Tor exits. `log` only notes the hit in
   the log line and raises an alert.
-- A CrowdSec engine on the host (`SWW_WAF_CROWDSEC_LAPI_URL`) has its decision
-  list fetched every minute, with `SWW_WAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key`
-  header. The list is kept like a fetched blocklist, and a client on it is
-  banned (see "Bans"). Only decisions of the type `ban` on an address or a
-  netblock are used; any other, such as a captcha or a country, is left out. A
-  decision lasts for the `duration` it had left when the engine answered. This
-  is how a fleet-wide blocklist can arrive without `waf` depending on CrowdSec.
-  A fetch fails when:
-    - the engine answers other than `200`, a redirect included;
-    - the fetch does not finish within a minute;
-    - the answer is longer than 16 MiB;
-    - the answer is not a list of decisions.
 - Each DNS blocklist (`SWW_WAF_DNSBL_ZONES`) is asked about a client's own
   address in the background, the first time the client is seen. No request waits
   for the answer. The name asked about is the one RFC 5782 gives: the numbers of
@@ -2395,9 +2374,9 @@ runs no check, writes no request log line and counts nothing in the metrics.
 answer but the last below carries `X-Sww-Disposition`.
 
 - `X-Sww-Disposition: allow`, status `200`, no body. `rproxy` passes the request
-  to the app. With `SWW_WAF_ADD_LOOKUP_HEADERS` the answer also carries
-  `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the request it
-  passes on. A header whose value is unknown is left out.
+  to the app. While `SWW_WAF_ADD_LOOKUP_HEADERS` is `true`, the answer also
+  carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
+  request it passes on. A header whose value is unknown is left out.
 - `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
   get. `rproxy` gives them to the visitor, less this header. The status is one
   of:
@@ -2418,9 +2397,7 @@ In `observe` mode a request that `enforce` mode would refuse is answered
 answered `404`.
 
 The Core Rule Set inspects the body a question carries only while
-`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. It inspects at most 64 KiB
-less one byte. So it does not inspect a JSON or XML body of 64 KiB or more,
-which may have been cut short.
+`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set, and at most 8 KiB of it.
 
 ### Answering in time
 
@@ -2437,14 +2414,13 @@ client without an answer by then counts as unknown. No question waits on these,
 which run in the background:
 
 - the DNS blocklists and AbuseIPDB;
-- fetching the blocklists, the tag lists, the CrowdSec decision list and the
-  file `SWW_WAF_ASN_LIMIT_PERCENT_URL` names.
+- fetching the blocklists, the tag lists and the file
+  `SWW_WAF_ASN_LIMIT_PERCENT_URL` names.
 
 In `enforce` mode, a question does not wait for GeoJS when it is a request:
 
 - from a client on a fetched blocklist, while `SWW_WAF_BLOCKLIST_ACTION` is
   `deny`;
-- from a client on the CrowdSec decision list;
 - that would take its client over a rate limit at 100% of it, since a percentage
   can only lower a limit.
 
@@ -2496,11 +2472,20 @@ client in `SWW_WAF_ALLOW_NETS` skips them, as it skips the Core Rule Set. In
 `observe` mode such a request is answered `allow` with `would_action`, and no
 other check runs on it.
 
-The Core Rule Set runs each of its rules on each cookie, so thousands of small
-cookies would keep it busy for over a second. One 30 KB cookie or query string
-would keep it busy for a third of a second or more. A request within the limits
-keeps the Core Rule Set busy for less time. Enough such requests at once can
-still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
+The Core Rule Set is the one check a visitor can keep busy for long. It runs
+each of its rules on each part of a request, so thousands of small cookies would
+keep it busy for over a second. These fixed limits bound what it reads:
+
+- the header limits above;
+- `rproxy`'s limit of 32 KiB on the request line and headers together (see
+  "`rproxy`'s limits");
+- 8 KiB of a body, the most `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` takes.
+
+A request can reach all of these at once. At paranoia level 4, with no rule
+switched off, the Core Rule Set takes under a second on one core of a current
+server to inspect it. So `waf` answers it within the default 2 seconds, even
+after the default 800 milliseconds of waiting for GeoJS. Enough such requests at
+once can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
 
 ### The report
 
@@ -2604,11 +2589,11 @@ request and ban notes name the site by its host.
 ## `rproxy`
 
 `sww rproxy` is the reverse proxy in front of one app. It asks `waf` about each
-request, and passes on what `waf` allows. It keeps nothing between requests and
-writes nothing to disk. When `waf` cannot be connected to, does not answer at
-all, or does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`,
-`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides. Its default, `allow`,
-keeps the site answering.
+request, and passes on what `waf` allows. It writes nothing to disk. The only
+thing it keeps between requests is its circuit breaker for asks to `waf`.
+`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides what is done with a
+request when `waf` gives no answer in time, and while the circuit breaker is
+open. Its default, `allow`, keeps the site answering.
 
 `rproxy` reads only its `SWW_RPROXY_` settings, which `docs/settings.md` lists.
 One of them is required: `SWW_RPROXY_WAF_BACKEND`, `waf`'s URL. It has no
@@ -2626,7 +2611,8 @@ only. The image runs it by default (see "Deployment").
 4. It refuses a request with a method it does not pass on, or one it cannot put
    to `waf` as the visitor sent it (see "Requests `rproxy` refuses itself").
 5. It reads up to the first 64 KiB of the body into memory.
-6. It sends `waf` the question (see "The question"). It speaks HTTP/1.1 to
+6. It sends `waf` the question (see "The question"), unless the circuit breaker
+   keeps it from asking (see "The circuit breaker"). It speaks HTTP/1.1 to
    `waf`, whatever `waf` offers. Go's HTTP/2 client would refuse to send a
    visitor's `Upgrade` header, and most `Connection` headers. The question never
    carries the visitor's `Trailer`.
@@ -2700,23 +2686,24 @@ log line has status `0`.
 ### Without an answer from `waf`
 
 `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides what is done with the
-request only when `waf`:
+request when `waf`:
 
 - cannot be connected to;
 - does not answer at all;
 - does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, which covers
   connecting, sending the question and reading the whole answer.
 
-An answer whose body breaks off counts as no answer at all.
+An answer whose body breaks off counts as no answer at all. The setting also
+decides, without asking `waf`, while the circuit breaker is open.
 
 `allow`, the default, passes the request to the app. `deny` answers `503`.
 Either way `rproxy` writes a process warning, its log line has
 `waf_backend_no_answer`, and it sends no report.
 
-A visitor can make `waf` slow. A request that keeps `waf` busy past
-`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, like any request while `waf` is overloaded,
-gets `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. With `deny`, such a request
-gets `503` instead of reaching the app.
+`waf` taking longer than `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` to decide a request
+is a bug in `waf`. Such a request gets
+`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`, like any request while `waf` is
+overloaded. With `deny`, it gets `503` instead of reaching the app.
 
 A path under `/_sww/` that `allow` would pass on is answered `404` instead, so
 that the app never sees that prefix.
@@ -2731,6 +2718,28 @@ A visitor that goes away while `waf` is being asked is no failure of `waf`.
 `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` does not apply. Its log line has
 status `0` and no `disposition`.
 
+### The circuit breaker
+
+`rproxy` stops asking `waf` for a while when `waf` keeps failing to answer. An
+ask fails when `waf` gives no answer in time (see "Without an answer from
+`waf`"), or an answer `rproxy` cannot pass on. Any other answer in time
+succeeds, whatever it decides. A visitor that goes away while `waf` is being
+asked counts as neither.
+
+The breaker works like this:
+
+- It opens when 5 asks fail within 5 seconds. Fewer failures never open it.
+- While it is open, a request gets
+  `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` at once, without asking `waf`.
+  It gets the same process warning and `waf_backend_no_answer` as a request
+  whose ask failed.
+- While it is open, one request a second still asks `waf`.
+- Each ask that succeeds clears the failures counted so far, and closes the
+  breaker if it is open.
+
+The breaker lives in memory, and a restart of `rproxy` closes it. It has no
+settings.
+
 ### Passing a request to the app
 
 `rproxy` sends the app at `SWW_RPROXY_UPSTREAM_URL` the visitor's request: the
@@ -2793,9 +2802,9 @@ its whole body. The visitor gets that answer at once, and the connection closes
 after it.
 
 The largest request line and headers is 32 KiB. A kept-open connection is closed
-after 120 seconds idle. Neither has a setting. traefik, by default, closes an
-unused connection after 90 seconds. So traefik closes first, and never sends a
-request on a connection `rproxy` is closing.
+after 120 seconds idle. Neither has a setting. The TLS-terminating proxy should
+close an unused connection sooner, so that it never sends a request on a
+connection `rproxy` is closing. traefik closes one after 90 seconds by default.
 
 What the visitor gets when a limit is passed:
 
@@ -2846,14 +2855,16 @@ chooses: `waf` runs `waf`, and with it unset the image runs `rproxy`.
 The usual setup is three containers on one private network:
 
 - the app;
-- `rproxy`, which traefik routes to, with `SWW_RPROXY_UPSTREAM_URL` naming the
-  app and `SWW_RPROXY_WAF_BACKEND` naming `waf`;
+- `rproxy`, which the TLS-terminating proxy forwards to, with
+  `SWW_RPROXY_UPSTREAM_URL` naming the app and `SWW_RPROXY_WAF_BACKEND` naming
+  `waf`;
 - `waf`, with `SWW_COMMAND=waf` and a volume for its state files.
 
 Each further app adds its own container and its own `rproxy`, whose
 `SWW_RPROXY_WAF_BACKEND` names the same `waf` (see "Sites"). Anything on that
 network can reach an app around `rproxy`, and can ask `waf` questions or send it
-reports. So the network holds only what you trust.
+reports. So the network holds only what you trust. `README.md` lists what the
+TLS-terminating proxy must do.
 
 ### `rproxy` in the app's container
 
@@ -2925,10 +2936,10 @@ process that stops at start is started again every second, not in a tight loop.
   container's output, which `docker logs` shows.
 - When a service exits, runit runs its `run` script again, and the container
   does not stop. Beside an app, the other process keeps running.
-- While `rproxy` is down, traefik answers `502`. While the app is down, `rproxy`
-  answers `502`, which is not a client offence. While `waf` is down,
-  `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an answer
-  from `waf`").
+- While `rproxy` is down, the TLS-terminating proxy answers `502`. While the app
+  is down, `rproxy` answers `502`, which is not a client offence. While `waf` is
+  down, `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an
+  answer from `waf`").
 - Only faults found at start stop `waf`:
     - invalid configuration;
     - a lookup database that is missing or unreadable;
@@ -2961,23 +2972,26 @@ checks:
 - With `SWW_COMMAND` unset, empty or `rproxy`, it passes while `rproxy` answers
   that request on `127.0.0.1`, at the port in `SWW_RPROXY_LISTEN_ADDR`, and the
   app accepts connections at `SWW_RPROXY_UPSTREAM_URL`. It does not ask `waf`,
-  so that the site stays in traefik's routes while `waf` is down.
+  so that the site stays in the TLS-terminating proxy's routes while `waf` is
+  down.
 - With any other value, it fails with a message naming `SWW_COMMAND` and its
   value.
 
-traefik sends a container no requests until it shows as healthy. So the check
-runs every second from the container's start until it first passes, for up to a
-minute, and every 30 seconds after that. An app with a health check of its own
-can replace the image's `HEALTHCHECK` with one that checks both.
+A TLS-terminating proxy that reads Docker's health status, as traefik does,
+sends a container no requests until it shows as healthy. So the check runs every
+second from the container's start until it first passes, for up to a minute, and
+every 30 seconds after that. An app with a health check of its own can replace
+the image's `HEALTHCHECK` with one that checks both.
 
 ### Ports
 
 `rproxy` listens on no port but the one in `SWW_RPROXY_LISTEN_ADDR`, 8080 by
-default, which traefik routes to. It passes requests to the app at
-`SWW_RPROXY_UPSTREAM_URL`, `http://127.0.0.1:8081` by default. Another port may
-be set: the health check follows the setting, and traefik's port label
-(`traefik.http.services.<name>.loadbalancer.server.port`) must name it. An app
-beside `rproxy` must leave that port free.
+default, which the TLS-terminating proxy forwards to. It passes requests to the
+app at `SWW_RPROXY_UPSTREAM_URL`, `http://127.0.0.1:8081` by default. Another
+port may be set: the health check follows the setting, and the TLS-terminating
+proxy must name it, as traefik's port label
+(`traefik.http.services.<name>.loadbalancer.server.port`) does. An app beside
+`rproxy` must leave that port free.
 
 `waf` listens on no port but the one in `SWW_WAF_LISTEN_ADDR`, 8080 by default.
 The questions, the reports and its own endpoints all come there.
@@ -2999,8 +3013,8 @@ host directory, keeps them across deploys. A host directory mounted there needs
 no change of owner, and files an earlier owner left in it can be read and
 replaced. The volume holds a few tens of MiB at most with the defaults, and
 needs no backup beyond the host's own. The image declares no volume, since every
-app image built on it would inherit it. `rproxy` keeps nothing, and needs no
-volume.
+app image built on it would inherit it. `rproxy` writes nothing to disk, and
+needs no volume.
 
 `waf` keeps its cache files in `SWW_WAF_CACHE_DIR`, `/var/cache/sww` by default,
 which the `run` script makes. The cache directory needs no volume and can be
@@ -3018,9 +3032,9 @@ in `docs/settings.md` shows the commands.
 ### Forwarded headers
 
 The app's TCP peer is `rproxy`, and the `X-Forwarded-For` the app receives ends
-with traefik's address, which `rproxy` adds after the visitor's. So the app must
-trust `rproxy`'s address for forwarded headers, besides the private address
-ranges traefik is on.
+with the TLS-terminating proxy's address, which `rproxy` adds after the
+visitor's. So the app must trust `rproxy`'s address for forwarded headers,
+besides the private address ranges the TLS-terminating proxy is on.
 
 - In a container of its own, `rproxy`'s address is on the private network. The
   usual default for a trusted-proxy setting, the private ranges, covers it.

docs/alerts.md

@@ -13,9 +13,8 @@ back or dropped.
   threshold. The request was refused unless the alert's `detail` gives `mode`:
   `detect` or `observe`.
 - `anomaly`: a count over an anomaly threshold.
-- `reputation_hit`: a request from a client on a blocklist, the CrowdSec
-  decision list or a DNS blocklist, or with an AbuseIPDB score that is a hit.
-  One alert per source.
+- `reputation_hit`: a request from a client on a blocklist or a DNS blocklist,
+  or with an AbuseIPDB score that is a hit. One alert per source.
 - `source_failure`: one of:
     - GeoJS failing;
     - a list that cannot be fetched;
@@ -69,8 +68,7 @@ on one line:
             "earlier_bans": {
                 "limit": 0,
                 "attack": 0,
-                "admin": 0,
-                "crowdsec": 0
+                "admin": 0
             }
         }
     },

docs/request-log.md

@@ -68,7 +68,8 @@ Every one of these lines has:
 ### Request detail
 
 - `request_id`: the id `rproxy` made for the request when it arrived.
-- `peer_ip`: the address of `rproxy`'s TCP peer, normally traefik.
+- `peer_ip`: the address of `rproxy`'s TCP peer, normally the TLS-terminating
+  proxy.
 - `forwarded_for`: the `X-Forwarded-For` header the visitor's request came with.
 - `client_group`: the client as rate limiting counts it, an IPv4 address as a
   /32 or an IPv6 group.
@@ -97,7 +98,7 @@ Every one of these lines has:
     - `client_denied`: refused for `SWW_WAF_DENY_NETS`, for a reputation source
       with the action `deny`, or for a tag in `SWW_WAF_TAG_DENY`;
     - `temporarily_banned`: refused under a ban that runs out, or for a `ban`
-      rule, a trap path or a CrowdSec decision that made such a ban;
+      rule or a trap path that made such a ban;
     - `permanently_banned`: refused under a ban that never runs out, including a
       ban this request made or made permanent;
     - `country_denied`, `client_is_rate_limited`, `blocked_by_block_rule` and
@@ -119,8 +120,6 @@ Every one of these lines has:
     - `countries`: `country`, and `setting`, the country list that refused it;
     - `tag_deny`: `tag`, the tag of `SWW_WAF_TAG_DENY` the request carries;
     - `blocklists`: `lists`, the blocklist URLs that list the client;
-    - `crowdsec`: `scenario`, the decision's scenario, and `expires`, when the
-      decision ends;
     - `dnsbl`: `zones`, the DNS blocklist zones that list the client, their keys
       masked;
     - `abuseipdb`: `score`, and `min_score`, `SWW_WAF_ABUSEIPDB_MIN_SCORE`;
@@ -175,7 +174,6 @@ Every one of these lines has:
   and the rules that matched, for a request it inspected.
 - `reputation`: the sources the client is on, in this order:
     - blocklist URLs;
-    - the CrowdSec decision list's URL;
     - DNS blocklist zones;
     - `abuseipdb`.
 - `tags`: the request's tags, sorted, left out when it has none. Each tag and
@@ -240,7 +238,8 @@ meanings where they differ:
 - `disposition`: what was done with the request: `allow`, `deny` or `close`. It
   is `waf`'s answer, or the value of
   `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. It is left out when:
-    - `waf` was not asked, as for the health check;
+    - `waf` was not asked and `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` did
+      not apply, as for the health check;
     - the visitor went away while `waf` was being asked;
     - `waf`'s answer had no valid `X-Sww-Disposition`;
     - `rproxy` could not pass on `waf`'s answer, and answered `502`.

docs/settings.md

@@ -150,7 +150,7 @@ cut off; the client is banned.
 - `SWW_WAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off`
   stops the start while a setting that needs a lookup is set:
     - `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES`;
-    - `SWW_WAF_ADD_LOOKUP_HEADERS`;
+    - `SWW_WAF_ADD_LOOKUP_HEADERS` set to `true`;
     - `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming a `country=` tag,
       or an `asn=` tag while `SWW_WAF_ROUTEWATCH_URL` is unset;
     - the two byte percentage lists and `SWW_WAF_ASN_LIMIT_PERCENT_URL`;
@@ -161,9 +161,11 @@ cut off; the client is banned.
 - `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` (default `800`): how long a
   request waits for its client's first answer from GeoJS, and how long a request
   to GeoJS may take. A time in milliseconds; never `off`.
-- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the
+- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `true`): `true` has `rproxy` pass the
   app the client's AS number in `X-Client-ASN` and its country in
-  `X-Client-Country`.
+  `X-Client-Country`. The default is `false` while `SWW_WAF_LOOKUP_SOURCE` is
+  `off`. With GeoJS, a new client's first request waits for its answer, up to
+  `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`.
 
 ## Country list
 
@@ -180,8 +182,8 @@ in [`SPEC.md`](../SPEC.md).
   `name=URL` pairs, such as `torexit=https://example.com/tor.txt`. A name is the
   tag, and may hold `=`, such as `vpn=mullvad`. Each list is an `http` or
   `https` URL fetched like a blocklist. A name or URL listed twice is refused.
-  So is a URL that is also in `SWW_WAF_BLOCKLIST_URLS`,
-  `SWW_WAF_ASN_LIMIT_PERCENT_URL`, or the CrowdSec decision list.
+  So is a URL that is also in `SWW_WAF_BLOCKLIST_URLS` or is
+  `SWW_WAF_ASN_LIMIT_PERCENT_URL`.
 - `SWW_WAF_TAG_DENY` (default empty): tags whose requests are refused, such as
   `torexit,country=CN`. A tag listed twice, an `asn=` tag without a number, or a
   `country=` tag without a country, is refused.
@@ -214,9 +216,6 @@ tag that no request can get stops the start. A request can get only these:
   `SWW_WAF_CORE_RULE_SET_MODE` nor `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD` is
   `off`.
 
-`crowdsec` stops the start in both settings, because a CrowdSec decision refuses
-the request first.
-
 ## Biased thresholds
 
 - `SWW_WAF_ASN_BYTES_LIMIT_PERCENT` and `SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT`
@@ -252,10 +251,9 @@ the request first.
 - `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS` (default empty): path prefixes the Core
   Rule Set does not inspect.
 - `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` (default `off`): `off`, or how
-  much of a request body the Core Rule Set reads, a size of at most `64K`. A
-  question carries no more of a body than 64 KiB, and the Core Rule Set reads at
-  most 64 KiB less one byte of it, whatever the value (see "The answer" in
-  [`SPEC.md`](../SPEC.md)).
+  much of a request body the Core Rule Set reads, a size of at most `8K`. A
+  larger size would let one request keep it busy past `rproxy`'s timeout (see
+  "Answering in time" in [`SPEC.md`](../SPEC.md)).
 - `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule
   Set exclusions, written as Coraza directives. It is loaded before the Core
   Rule Set's rules, so it holds only `SecAction` and `SecRule` lines that
@@ -333,14 +331,6 @@ the request first.
   score, from 0 to 100, that is a hit.
 - `SWW_WAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a
   day, a whole number above zero. AbuseIPDB's free accounts may make 1,000.
-- `SWW_WAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec
-  engine you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL
-  without a user or a fragment; its decision list is at `v1/decisions` under it.
-  Set without the key, it is refused. So is a decision list URL that is also in
-  `SWW_WAF_BLOCKLIST_URLS`, or that is `SWW_WAF_ASN_LIMIT_PERCENT_URL`.
-- `SWW_WAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
-  `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
-  URL, it is refused.
 - `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` (default `limit:25`): `deny`,
   `limit:<percent>` or `log`, for DNS blocklist and AbuseIPDB hits.
 - `SWW_WAF_IP_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict
@@ -462,10 +452,9 @@ start, and an invalid value stops the start. Any can be given as a file, as a
   for `waf`'s answer, to a question and to a report. A time in milliseconds;
   never `off`.
 - `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` (default `allow`): what is done
-  with a request when `waf` cannot be connected to, does not answer at all, or
-  does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`. `allow` passes it
-  to the app, and `deny` answers `503`. It does not apply to an answer `waf`
-  gives, such as an error.
+  with a request when `waf` gives no answer in time, and while the circuit
+  breaker for asks to `waf` is open. `allow` passes it to the app, and `deny`
+  answers `503`. It does not apply to an answer `waf` gives, such as an error.
 - `SWW_RPROXY_MAX_REQUEST_SIZE` (default `100M`): the largest request body. A
   size or `off`.
 - `SWW_RPROXY_MAX_RESPONSE_SIZE` (default `1G`): the largest response body. A
@@ -487,9 +476,9 @@ start, and an invalid value stops the start. Any can be given as a file, as a
   `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`): the netblocks whose
   `X-Forwarded-For` is believed. Set but empty, it trusts nothing. A client
   outside these netblocks cannot claim another address. Where other containers
-  reach `rproxy` directly, or visitors on a private network reach traefik, they
-  can name another address in the header. Narrowing the setting to traefik's own
-  network closes that gap.
+  reach `rproxy` directly, or visitors on a private network reach the
+  TLS-terminating proxy, they can name another address in the header. Narrowing
+  the setting to the TLS-terminating proxy's own network closes that gap.
 
 ## The image's command
 

docs/state-files.md

@@ -25,7 +25,7 @@ Active and past bans: every ban an admin made, and up to `SWW_WAF_MAX_BANS` of
 those `waf` made. Per entry:
 
 - `netblock`, `start` and `expires` (`null` for a permanent ban);
-- `cause`: `attack`, `limit`, `admin` or `crowdsec`;
+- `cause`: `attack`, `limit` or `admin`;
 - `reason`: a short reason;
 - `lifted`: when an admin lifted it, if one did;
 - `notes`: the notes below.

internal/alerts/alerts.go

@@ -48,9 +48,8 @@ const (
 	// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, refused in block mode, let through in
 	// detect mode.
 	EventCoreRuleSetMatch = "core_rule_set_match"
-	// EventReputationHit is a request whose client a blocklist, the
-	// CrowdSec decision list or a DNSBL zone lists, or whose AbuseIPDB score
-	// is a hit.
+	// EventReputationHit is a request whose client a blocklist or a DNSBL
+	// zone lists, or whose AbuseIPDB score is a hit.
 	EventReputationHit = "reputation_hit"
 	// EventSourceFailure is GeoJS failing or refusing sww, a fetch
 	// of a list failing, a query to a DNSBL zone or a check with AbuseIPDB

internal/bans/bans.go

@@ -1,10 +1,8 @@
 // Package bans is the ban ledger: the bans sww makes on the
 // netblocks of clients that break a rate limit, a byte limit or the refusal
-// limit, show a clear sign of attack or are listed by the CrowdSec
-// decision list, and
-// those an admin makes, with their notes, as the "Bans" section of SPEC.md
-// describes. The bans are kept in memory, and written to bans.json and
-// read from it by the state package.
+// limit or show a clear sign of attack, and those an admin makes, with their
+// notes, as the "Bans" section of SPEC.md describes. The bans are kept in
+// memory, and written to bans.json and read from it by the state package.
 package bans
 
 import (
@@ -28,9 +26,6 @@ const (
 	// CauseAdmin is a ban an admin made, or one sww made that an
 	// admin keeps. It is never dropped.
 	CauseAdmin = "admin"
-	// CauseCrowdSec is a ban sww made for a client the CrowdSec
-	// decision list lists. It ends when CrowdSec's decision does.
-	CauseCrowdSec = "crowdsec"
 )
 
 // repeatFactor is how many times as long as the netblock's last ban a ban
@@ -45,8 +40,8 @@ type Params struct {
 	// LimitBanDuration is how long a first ban for a broken limit lasts.
 	LimitBanDuration time.Duration
 	// LimitBanRepeatWindow is how soon after the end of the netblock's
-	// ban that ended last, other than one for a clear sign of attack or for
-	// CrowdSec's decision, a broken limit counts as a repeat, which bans for
+	// ban that ended last, other than one for a clear sign of attack, a
+	// broken limit counts as a repeat, which bans for
 	// repeatFactor times as long as that ban.
 	LimitBanRepeatWindow time.Duration
 	// A ban for a broken limit that would be longer than
@@ -68,11 +63,10 @@ type Ban struct {
 	Start    time.Time
 	// Expires is when the ban ends, zero for a permanent ban.
 	Expires time.Time
-	// Cause is CauseLimit, CauseAttack, CauseAdmin or CauseCrowdSec.
+	// Cause is CauseLimit, CauseAttack or CauseAdmin.
 	Cause string
-	// Reason is a short text: for a ban sww made, the limit broken,
-	// the rule that matched or the scenario of CrowdSec's decision; for an
-	// admin's, what the admin wrote.
+	// Reason is a short text: for a ban sww made, the limit broken or
+	// the rule that matched; for an admin's, what the admin wrote.
 	Reason string
 	// Lifted is when an admin lifted the ban, zero while no admin has. A
 	// lifted ban refuses nothing, and does not make the netblock's next
@@ -133,8 +127,7 @@ type Notes struct {
 	// log's reputation names them. It is left out when none did.
 	Reputation []ReputationHit `json:"reputation,omitempty"`
 	// Request is the request that broke the limit, or whose bytes broke
-	// it, that was the clear sign of attack, or that came from a client the
-	// CrowdSec decision list lists.
+	// it, or that was the clear sign of attack.
 	Request Request `json:"request"`
 	// Requests is how many requests the netblock has sent since it was
 	// first seen, and Refused how many of them the ban has refused so
@@ -148,9 +141,8 @@ type Notes struct {
 
 // ReputationHit is a reputation source that listed a client, as a
 // reputation_hit alert's detail gives it: Source is the URL of the
-// blocklist or of the CrowdSec decision list, the DNSBL zone with its key
-// masked, or "abuseipdb", and Score, for AbuseIPDB alone, its score of the
-// client.
+// blocklist, the DNSBL zone with its key masked, or "abuseipdb", and Score,
+// for AbuseIPDB alone, its score of the client.
 type ReputationHit struct {
 	Source string `json:"source"`
 	Score  *int64 `json:"score,omitempty"`
@@ -158,10 +150,9 @@ type ReputationHit struct {
 
 // EarlierBans counts a netblock's bans before a ban, by cause.
 type EarlierBans struct {
-	Limit    int `json:"limit"`
-	Attack   int `json:"attack"`
-	Admin    int `json:"admin"`
-	CrowdSec int `json:"crowdsec"`
+	Limit  int `json:"limit"`
+	Attack int `json:"attack"`
+	Admin  int `json:"admin"`
 }
 
 // Request is a request in a ban's notes. Each text is cut to 256 bytes.
@@ -303,8 +294,7 @@ func activeBan(bans []Ban, now time.Time) *Ban {
 // BanForLimit bans netblock at now for a broken limit, with notes, and
 // returns the ban, and true. A first ban lasts LimitBanDuration. A ban
 // made within LimitBanRepeatWindow after the netblock's ban that ended
-// last, other than one for a clear sign of attack or for CrowdSec's
-// decision, or a lifted one, lasts
+// last, other than one for a clear sign of attack or a lifted one, lasts
 // repeatFactor times as long as that one. A ban that would be longer
 // than LimitBanPermanentAbove is permanent instead. If a ban on netblock is still
 // active, as when two of its requests break a limit at once, that ban is
@@ -318,7 +308,6 @@ func (l *Ledger) BanForLimit(
 	return l.ban(
 		netblock,
 		now,
-		time.Time{},
 		CauseLimit,
 		limitReason(notes),
 		notes,
@@ -334,7 +323,6 @@ func (l *Ledger) WouldBanForLimit(
 	return l.ban(
 		netblock,
 		now,
-		time.Time{},
 		CauseLimit,
 		limitReason(notes),
 		notes,
@@ -353,7 +341,6 @@ func (l *Ledger) BanForAttack(
 	return l.ban(
 		netblock,
 		now,
-		time.Time{},
 		CauseAttack,
 		attackReason(notes),
 		notes,
@@ -369,7 +356,6 @@ func (l *Ledger) WouldBanForAttack(
 	return l.ban(
 		netblock,
 		now,
-		time.Time{},
 		CauseAttack,
 		attackReason(notes),
 		notes,
@@ -377,51 +363,9 @@ func (l *Ledger) WouldBanForAttack(
 	)
 }
 
-// BanForCrowdSec bans netblock at now until expires, when CrowdSec's
-// decision on the client ends, with notes, and returns the ban, and
-// whether it made it, as BanForLimit does. Its reason is "CrowdSec's
-// decision for <scenario>", the scenario that made the decision.
-func (l *Ledger) BanForCrowdSec(
-	netblock netip.Prefix,
-	now, expires time.Time,
-	scenario string,
-	notes Notes,
-) (Ban, bool) {
-	return l.ban(
-		netblock,
-		now,
-		expires,
-		CauseCrowdSec,
-		crowdSecReason(scenario),
-		notes,
-		true,
-	)
-}
-
-// WouldBanForCrowdSec returns what BanForCrowdSec would, without making
-// the ban: what observe mode would have done.
-func (l *Ledger) WouldBanForCrowdSec(
-	netblock netip.Prefix,
-	now, expires time.Time,
-	scenario string,
-	notes Notes,
-) (Ban, bool) {
-	return l.ban(
-		netblock,
-		now,
-		expires,
-		CauseCrowdSec,
-		crowdSecReason(scenario),
-		notes,
-		false,
-	)
-}
-
-// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit,
-// CauseAttack or CauseCrowdSec, made at now would be permanent, as
-// BanForLimit, BanForAttack or BanForCrowdSec would make it. It works out
-// nothing else of the ban. A ban for CrowdSec's decision is never
-// permanent: it ends with the decision.
+// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit
+// or CauseAttack, made at now would be permanent, as BanForLimit or
+// BanForAttack would make it. It works out nothing else of the ban.
 func (l *Ledger) WouldBePermanent(
 	netblock netip.Prefix, now time.Time, cause string,
 ) bool {
@@ -433,14 +377,11 @@ func (l *Ledger) WouldBePermanent(
 		held = *bans
 	}
 
-	switch cause {
-	case CauseAttack:
+	if cause == CauseAttack {
 		return l.attackExpiry(held, now).IsZero()
-	case CauseLimit:
-		return l.limitExpiry(held, now).IsZero()
-	default: // CauseCrowdSec
-		return false
 	}
+
+	return l.limitExpiry(held, now).IsZero()
 }
 
 // limitReason is the reason of a ban for a broken limit, with notes.
@@ -459,12 +400,6 @@ func attackReason(notes Notes) string {
 	return "matched the rule " + notes.RuleID
 }
 
-// crowdSecReason is the reason of a ban for CrowdSec's decision, which
-// scenario made.
-func crowdSecReason(scenario string) string {
-	return "CrowdSec's decision for " + scenario
-}
-
 // BanForAdmin bans netblock at now for an admin, with reason, until
 // expires, or for good when expires is zero, and returns the ban, whose
 // cause is CauseAdmin. Unlike BanForLimit and BanForAttack, it makes the
@@ -704,15 +639,13 @@ func (l *Ledger) holds(netblock netip.Prefix, start time.Time) bool {
 	})
 }
 
-// ban bans netblock at now for cause, with reason and notes, as
-// BanForLimit, BanForAttack and BanForCrowdSec describe, and returns the
-// ban, and whether it made it. expires is when a ban for CauseCrowdSec
-// ends, and zero for the others, whose end the ledger works out. Unless
-// keep is true, the ban is not made, only returned: it is the ban that
-// would have been made.
+// ban bans netblock at now for cause, CauseLimit or CauseAttack, with
+// reason and notes, as BanForLimit and BanForAttack describe, and returns
+// the ban, and whether it made it. Unless keep is true, the ban is not
+// made, only returned: it is the ban that would have been made.
 func (l *Ledger) ban(
 	netblock netip.Prefix,
-	now, expires time.Time,
+	now time.Time,
 	cause, reason string,
 	notes Notes,
 	keep bool,
@@ -743,13 +676,10 @@ func (l *Ledger) ban(
 		Notes:    notes,
 	}
 
-	switch cause {
-	case CauseAttack:
+	if cause == CauseAttack {
 		ban.Expires = l.attackExpiry(held, now)
-	case CauseLimit:
+	} else {
 		ban.Expires = l.limitExpiry(held, now)
-	default: // CauseCrowdSec
-		ban.Expires = expires
 	}
 
 	if !keep {
@@ -784,16 +714,13 @@ func earlierBans(held []Ban) EarlierBans {
 			fromHere.Attack++
 		case CauseAdmin:
 			fromHere.Admin++
-		case CauseCrowdSec:
-			fromHere.CrowdSec++
 		}
 
 		notes := ban.Notes.EarlierBans
 		count := EarlierBans{
-			Limit:    notes.Limit + fromHere.Limit,
-			Attack:   notes.Attack + fromHere.Attack,
-			Admin:    notes.Admin + fromHere.Admin,
-			CrowdSec: notes.CrowdSec + fromHere.CrowdSec,
+			Limit:  notes.Limit + fromHere.Limit,
+			Attack: notes.Attack + fromHere.Attack,
+			Admin:  notes.Admin + fromHere.Admin,
 		}
 
 		// Of counts that name as many bans, the earlier ban's is taken: it
@@ -808,7 +735,7 @@ func earlierBans(held []Ban) EarlierBans {
 
 // total returns how many bans e counts, of all causes.
 func (e EarlierBans) total() int {
-	return e.Limit + e.Attack + e.Admin + e.CrowdSec
+	return e.Limit + e.Attack + e.Admin
 }
 
 // markChanged has Changed receive a value, unless one is waiting already.
@@ -913,9 +840,9 @@ func (l *Ledger) add(ban Ban) {
 // limitExpiry returns when a ban for a broken limit made at now ends, or
 // zero when it is permanent. held are the netblock's bans, none of them
 // active, of which the one that ended last, other than a ban for a clear
-// sign of attack or for CrowdSec's decision, or a lifted one, can make the
-// new ban longer. A ban an admin adds to bans.json can start after
-// another and end before it, so that one is looked for among them all.
+// sign of attack or a lifted one, can make the new ban longer. A ban an
+// admin adds to bans.json can start after another and end before it, so
+// that one is looked for among them all.
 func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
 	length := l.params.LimitBanDuration
 

internal/bans/crowdsec_test.go

@@ -1,136 +0,0 @@
-package bans_test
-
-import (
-	"net/netip"
-	"reflect"
-	"testing"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/bans"
-)
-
-// scenario is the scenario of the tests' CrowdSec decisions.
-const scenario = "crowdsecurity/ssh-bf"
-
-func TestCrowdSecBanLastsUntilTheDecisionEnds(t *testing.T) {
-	t.Parallel()
-
-	ledger := bans.New(defaultParams())
-	netblock := netip.MustParsePrefix("203.0.113.9/32")
-	expires := midnight().Add(4 * time.Hour)
-
-	// The ban that would be made is not made.
-	would, wouldBan := ledger.WouldBanForCrowdSec(
-		netblock,
-		midnight(),
-		expires,
-		scenario,
-		bans.Notes{},
-	)
-	if !wouldBan || len(ledger.Bans(netblock)) != 0 {
-		t.Errorf(
-			"would ban %t, and the ledger holds %+v, want true and nothing",
-			wouldBan,
-			ledger.Bans(netblock),
-		)
-	}
-
-	const reason = "CrowdSec's decision for " + scenario
-
-	ban, made := ledger.BanForCrowdSec(
-		netblock,
-		midnight(),
-		expires,
-		scenario,
-		bans.Notes{},
-	)
-	if !made || !reflect.DeepEqual(ban, would) ||
-		ban.Cause != bans.CauseCrowdSec ||
-		!ban.Expires.Equal(expires) ||
-		ban.Reason != reason ||
-		ledger.Made(bans.CauseCrowdSec) != 1 {
-		t.Errorf(
-			"made %t the ban %+v, want the one that would be made, %+v, for "+
-				"crowdsec until %s",
-			made,
-			ban,
-			would,
-			expires,
-		)
-	}
-
-	// A second decision on the netblock while the ban lasts makes no other.
-	again, made := ledger.BanForCrowdSec(netblock, midnight().Add(time.Hour),
-		expires.Add(time.Hour), scenario, bans.Notes{})
-	if made || !again.Expires.Equal(expires) ||
-		ledger.Made(bans.CauseCrowdSec) != 1 {
-		t.Errorf(
-			"made %t the ban %+v while the first lasts, want none",
-			made,
-			again,
-		)
-	}
-}
-
-func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
-	t.Parallel()
-
-	ledger := bans.New(defaultParams())
-	netblock := netip.MustParsePrefix("203.0.113.9/32")
-	expires := midnight().Add(4 * time.Hour)
-	ledger.BanForCrowdSec(
-		netblock,
-		midnight(),
-		expires,
-		scenario,
-		bans.Notes{},
-	)
-
-	// A request as the ban ends is refused, and leaves it as it is.
-	last := expires.Add(-time.Nanosecond)
-
-	held, banned, madePermanent := ledger.Check(netblock.Addr(), last)
-	if !banned || madePermanent || !held.Expires.Equal(expires) ||
-		ledger.WouldBePermanent(netblock, last, bans.CauseCrowdSec) {
-		t.Errorf(
-			"as the ban ends, banned %t with %+v, made permanent %t, want "+
-				"refused under the ban as it was",
-			banned,
-			held,
-			madePermanent,
-		)
-	}
-
-	if _, banned, _ := ledger.Check(netblock.Addr(), expires); banned {
-		t.Error("the ban refuses a request once the decision has ended")
-	}
-}
-
-func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	ledger := bans.New(defaultParams())
-	netblock := netip.MustParsePrefix("203.0.113.9/32")
-
-	// Three times the three days would be permanent; a limit broken as the
-	// ban for CrowdSec's decision ends bans for an hour, as a first broken
-	// limit does.
-	crowdSec, _ := ledger.BanForCrowdSec(
-		netblock,
-		midnight(),
-		midnight().Add(3*day),
-		scenario,
-		bans.Notes{},
-	)
-	limit, _ := ledger.BanForLimit(netblock, crowdSec.Expires, bans.Notes{})
-
-	if limit.Expires.Sub(limit.Start) != time.Hour ||
-		limit.Notes.EarlierBans != (bans.EarlierBans{CrowdSec: 1}) {
-		t.Errorf(
-			"the ban for a limit is %+v, want one of an hour after one for crowdsec",
-			limit,
-		)
-	}
-}

internal/config/waf.go

@@ -84,7 +84,8 @@ type WAF struct {
 	// client's first answer from GeoJS while a setting needs it
 	// (SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS), which cannot be off.
 	// AddLookupHeaders is true when the app is passed the client's AS number
-	// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS).
+	// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS), by default unless
+	// LookupSource is off.
 	LookupSource               string
 	LookupDBPath               string
 	ASNAndCountryLookupTimeout time.Duration
@@ -157,13 +158,6 @@ type WAF struct {
 	DNSBLAbuseIPDBLimitPercent int64
 	IPReputationCacheTTL       time.Duration
 	DNSBLAbuseIPDBTimeout      time.Duration
-	// CrowdSecDecisionsURL is where the decision list of the CrowdSec
-	// engine whose local API SWW_WAF_CROWDSEC_LAPI_URL names is fetched from:
-	// that URL with v1/decisions added to its path, "" while it is unset and
-	// none is. CrowdSecKey is the key the engine is asked with
-	// (SWW_WAF_CROWDSEC_LAPI_KEY).
-	CrowdSecDecisionsURL string
-	CrowdSecKey          string
 	// BanResponse is the status a refused client is answered with, 403
 	// or 429, or 0 to close the connection without an answer
 	// (SWW_WAF_BAN_RESPONSE). It answers a banned client, a request that
@@ -371,8 +365,8 @@ var (
 		"it names the file to look clients up in",
 	)
 	errDBPathUnused = errors.New("only file reads it")
-	errOver64K      = errors.New(
-		"is more than 64K, the most waf gets of a body")
+	errOver8K       = errors.New(
+		"is more than 8K, the most the Core Rule Set can read in time")
 	errNotDurationAboveZero = errors.New(
 		"is not a duration above zero, such as 1h or 7d")
 	errNotNumberAboveZero = errors.New(
@@ -473,21 +467,12 @@ var (
 		"is not an IP address with an optional port, " +
 			"such as 192.0.2.53 or [2001:db8::53]:5353",
 	)
-	errNotLAPIURL = errors.New(
-		"is not an http or https URL without a user or a fragment, " +
-			"such as http://172.17.0.1:8080")
 	errNotRouteWatchURL = errors.New(
 		"is not an http or https URL without a user or a fragment, " +
 			"such as https://routewatch.example")
-	errNeedsLAPIKey  = errors.New("the engine answers no request without it")
-	errLAPIKeyUnused = errors.New(
-		"it is sent only to the engine at that URL",
-	)
-	errAnotherList = errors.New(
+	errFetchedOtherwise = errors.New(
 		"is in SWW_WAF_BLOCKLIST_URLS or is SWW_WAF_ASN_LIMIT_PERCENT_URL too",
 	)
-	errFetchedOtherwise = errors.New("is in SWW_WAF_BLOCKLIST_URLS, is " +
-		"SWW_WAF_ASN_LIMIT_PERCENT_URL or is the CrowdSec decision list too")
 	errNotTagList = errors.New(
 		"is not a name, = and an http or https URL, such as " +
 			"torexit=https://example.com/tor.txt")
@@ -502,8 +487,6 @@ var (
 	errExploitTag = errors.New(
 		"cannot lower limits: the rate limits count a request before its " +
 			"rules are matched")
-	errCrowdSecTag = errors.New(
-		"cannot be used: a CrowdSec decision refuses the request first")
 	errNoRequestGetsTag = errors.New(
 		"is a tag no request can get: it is not a name in SWW_WAF_TAG_LISTS, an " +
 			"asn= or country= tag, a dnsbl= tag of a zone in SWW_WAF_DNSBL_ZONES, " +
@@ -567,7 +550,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
 			"SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS",
 			"800",
 		),
-		AddLookupHeaders: env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS", "false"),
 		ExclusivelyAllowedCountries: env.countries(
 			"SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES", ""),
 		TagLists:        env.tagLists("SWW_WAF_TAG_LISTS"),
@@ -605,10 +587,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
 			"SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS",
 			"2000",
 		),
-		CrowdSecDecisionsURL: env.crowdSecDecisionsURL(
-			"SWW_WAF_CROWDSEC_LAPI_URL",
-		),
-		CrowdSecKey: env.secret("SWW_WAF_CROWDSEC_LAPI_KEY"),
 		BanResponse: env.banResponse("SWW_WAF_BAN_RESPONSE", "403"),
 		LimitBanDuration: env.durationNotOff(
 			"SWW_WAF_LIMIT_BAN_DURATION",
@@ -746,6 +724,9 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
 
 	cfg.LogRemoteAppName = env.appName("SWW_WAF_LOG_REMOTE_APP_NAME",
 		cfg.InstanceName, cfg.LogRemoteURL != nil)
+	// The app gets the headers by default, unless no client is looked up.
+	cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS",
+		strconv.FormatBool(cfg.LookupSource != off))
 	cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action(
 		"SWW_WAF_BLOCKLIST_ACTION", "deny")
 	cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action(
@@ -755,7 +736,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
 	env.checkLookupDBPath(cfg)
 	env.checkCountriesAndLookups(cfg)
 	env.checkASNLimitPercentURL(cfg)
-	env.checkCrowdSec(cfg)
 	env.checkTagLists(cfg)
 	env.checkTags(cfg)
 
@@ -1061,26 +1041,6 @@ func (e *environment) resolver(name string) netip.AddrPort {
 	return resolver
 }
 
-// crowdSecDecisionsURL reads the setting that is the URL of the CrowdSec
-// engine's local API, such as http://172.17.0.1:8080, and returns the URL
-// its decision list is fetched from, that URL with v1/decisions added to
-// its path, "" while it is unset or empty.
-func (e *environment) crowdSecDecisionsURL(name string) string {
-	value := e.value(name, "")
-	if value == "" {
-		return ""
-	}
-
-	lapi, err := url.Parse(value)
-	if err != nil || !isHTTPURL(lapi) {
-		e.check(name, fmt.Errorf("%q %w", value, errNotLAPIURL))
-
-		return ""
-	}
-
-	return lapi.JoinPath("v1", "decisions").String()
-}
-
 // routeWatchURL reads the setting that is the URL of a routewatch instance,
 // "" while it is unset or empty.
 func (e *environment) routeWatchURL(name string) string {
@@ -1190,39 +1150,12 @@ func (e *environment) checkASNLimitPercentURL(cfg *WAF) {
 	}
 }
 
-// checkCrowdSec refuses SWW_WAF_CROWDSEC_LAPI_URL without
-// SWW_WAF_CROWDSEC_LAPI_KEY, the key without the URL, and a decision list
-// that is fetched as another list too.
-func (e *environment) checkCrowdSec(cfg *WAF) {
-	decisionsURL := cfg.CrowdSecDecisionsURL
-
-	switch {
-	case decisionsURL != "" && cfg.CrowdSecKey == "":
-		e.check("SWW_WAF_CROWDSEC_LAPI_URL", fmt.Errorf(
-			"is set while SWW_WAF_CROWDSEC_LAPI_KEY is unset; %w",
-			errNeedsLAPIKey,
-		))
-	case decisionsURL == "" && cfg.CrowdSecKey != "":
-		e.check("SWW_WAF_CROWDSEC_LAPI_KEY", fmt.Errorf(
-			"is set while SWW_WAF_CROWDSEC_LAPI_URL is unset; %w",
-			errLAPIKeyUnused,
-		))
-	case decisionsURL != "" && (slices.Contains(cfg.BlocklistURLs, decisionsURL) ||
-		decisionsURL == cfg.ASNLimitPercentURL):
-		e.check(
-			"SWW_WAF_CROWDSEC_LAPI_URL",
-			fmt.Errorf("gives the decision list %q, which %w",
-				decisionsURL, errAnotherList),
-		)
-	}
-}
-
 // checkTagLists refuses a list of SWW_WAF_TAG_LISTS whose URL is fetched as
 // another list too: the file at a URL is read as one kind of list.
 func (e *environment) checkTagLists(cfg *WAF) {
 	for _, list := range cfg.TagLists {
 		if slices.Contains(cfg.BlocklistURLs, list.URL) ||
-			list.URL == cfg.ASNLimitPercentURL || list.URL == cfg.CrowdSecDecisionsURL {
+			list.URL == cfg.ASNLimitPercentURL {
 			e.check(
 				"SWW_WAF_TAG_LISTS",
 				fmt.Errorf("%q %w", list.URL, errFetchedOtherwise),
@@ -1232,8 +1165,7 @@ func (e *environment) checkTagLists(cfg *WAF) {
 }
 
 // checkTags refuses a tag of SWW_WAF_TAG_DENY or SWW_WAF_TAG_LIMIT_PERCENT that would
-// never apply: crowdsec, whose request is refused before tags are acted on, and
-// any tag these settings cannot give a request.
+// never apply: any tag these settings cannot give a request.
 func (e *environment) checkTags(cfg *WAF) {
 	for _, tag := range cfg.TagDeny {
 		e.check("SWW_WAF_TAG_DENY", checkTag(cfg, tag))
@@ -1246,10 +1178,6 @@ func (e *environment) checkTags(cfg *WAF) {
 
 // checkTag returns an error when tag would never apply, as checkTags says.
 func checkTag(cfg *WAF, tag string) error {
-	if tag == CrowdSecTag {
-		return fmt.Errorf("%q %w", tag, errCrowdSecTag)
-	}
-
 	if !requestCanGetTag(cfg, tag) {
 		return fmt.Errorf("%q %w", MaskZoneKey(tag), errNoRequestGetsTag)
 	}
@@ -1673,16 +1601,16 @@ func durationOrDays(value string) (time.Duration, error) {
 }
 
 // parseCoreRuleSetBodyInspectionSize reads the most of a request body the Core Rule Set
-// reads: a size as parseSize reads it, or off, but at most 64K, since waf
-// gets no more of a body than that.
+// reads: a size as parseSize reads it, or off, but at most 8K, so that one
+// request cannot keep the Core Rule Set busy past rproxy's timeout.
 func parseCoreRuleSetBodyInspectionSize(value string) (int64, error) {
 	limit, err := parseSize(value)
 	if err != nil {
 		return 0, err
 	}
 
-	if limit > 64*kibibyte {
-		return 0, fmt.Errorf("%q %w", value, errOver64K)
+	if limit > 8*kibibyte {
+		return 0, fmt.Errorf("%q %w", value, errOver8K)
 	}
 
 	return limit, nil
@@ -2010,7 +1938,6 @@ const (
 	DNSBLTagPrefix   = "dnsbl="
 	PrefixTagPrefix  = "prefix="
 	AbuseIPDBTag     = "abuseipdb"
-	CrowdSecTag      = "crowdsec"
 	// ExploitSignatureTag is the tag of a request a rule file or the Core
 	// Rule Set matched.
 	ExploitSignatureTag = "exploitsignature"

internal/config/waf_test.go

@@ -65,8 +65,6 @@ const (
 	ipReputationCacheTTL           = "SWW_WAF_IP_REPUTATION_CACHE_TTL"
 	dnsblAbuseIPDBTimeout          = "SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS"
 	routeWatchURL                  = "SWW_WAF_ROUTEWATCH_URL"
-	crowdSecURL                    = "SWW_WAF_CROWDSEC_LAPI_URL"
-	crowdSecKey                    = "SWW_WAF_CROWDSEC_LAPI_KEY"
 	banResponse                    = "SWW_WAF_BAN_RESPONSE"
 	limitBanDuration               = "SWW_WAF_LIMIT_BAN_DURATION"
 	limitBanRepeatWindow           = "SWW_WAF_LIMIT_BAN_REPEAT_WINDOW"
@@ -289,6 +287,7 @@ func TestDefaults(t *testing.T) {
 	})
 	wantLookupSettings(t, cfg, config.WAF{
 		LookupSource: defaultLookupSource, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
+		AddLookupHeaders: true,
 	})
 	wantByteLimitSettings(t, cfg, config.WAF{
 		BytesLimitPerMinute: 0, BytesLimitPerHour: 50 << 30,
@@ -617,7 +616,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
 				coreRuleSetScoreThreshold:     "10",
 				coreRuleSetDisabledRules:      "942100, 920350",
 				coreRuleSetExemptPaths:        "/api/, /static/",
-				coreRuleSetBodyInspectionSize: "32K",
+				coreRuleSetBodyInspectionSize: "4K",
 				exclusionsFile:                "/etc/sww/crs-exclusions.conf",
 			},
 			config.WAF{
@@ -626,7 +625,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
 				CoreRuleSetScoreThreshold:     10,
 				CoreRuleSetDisabledRules:      []int{942100, 920350},
 				CoreRuleSetExemptPaths:        []string{"/api/", "/static/"},
-				CoreRuleSetBodyInspectionSize: 32 << 10,
+				CoreRuleSetBodyInspectionSize: 4 << 10,
 				CoreRuleSetExclusionsFile:     "/etc/sww/crs-exclusions.conf",
 			},
 		},
@@ -763,15 +762,15 @@ func TestInvalidAppsStopsTheStart(t *testing.T) {
 	}
 }
 
-func TestCoreRuleSetBodyInspectionSizeOf64K(t *testing.T) {
+func TestCoreRuleSetBodyInspectionSizeOf8K(t *testing.T) {
 	t.Parallel()
 
 	cfg := wafFromEnvironment(
 		t,
-		environment{coreRuleSetBodyInspectionSize: "64K"},
+		environment{coreRuleSetBodyInspectionSize: "8K"},
 	)
-	if cfg.CoreRuleSetBodyInspectionSize != 64<<10 {
-		t.Errorf("64K read as %d", cfg.CoreRuleSetBodyInspectionSize)
+	if cfg.CoreRuleSetBodyInspectionSize != 8<<10 {
+		t.Errorf("8K read as %d", cfg.CoreRuleSetBodyInspectionSize)
 	}
 }
 
@@ -820,11 +819,13 @@ func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
 			coreRuleSetBodyInspectionSize, "128KB",
 			`"128KB" is not a size such as 512K, 100M or 5G, or off`,
 		},
-		{coreRuleSetBodyInspectionSize, "65K", `"65K" is more than 64K, the most waf ` +
-			`gets of a body`},
 		{
-			coreRuleSetBodyInspectionSize, "65537",
-			`"65537" is more than 64K, the most waf gets of a body`,
+			coreRuleSetBodyInspectionSize, "9K",
+			`"9K" is more than 8K, the most the Core Rule Set can read in time`,
+		},
+		{
+			coreRuleSetBodyInspectionSize, "8193",
+			`"8193" is more than 8K, the most the Core Rule Set can read in time`,
 		},
 	} {
 		t.Run(tc.name+"="+tc.value, func(t *testing.T) {
@@ -1397,13 +1398,13 @@ func TestLookupSettingsAsSet(t *testing.T) {
 	t.Parallel()
 
 	cfg := wafFromEnvironment(t, environment{
-		asnAndCountryLookupTimeout: "500", addLookupHeaders: enabled,
+		asnAndCountryLookupTimeout: "500", addLookupHeaders: disabled,
 	})
 	wantLookupSettings(t, cfg, config.WAF{
 		LookupSource: defaultLookupSource, ASNAndCountryLookupTimeout: 500 * time.Millisecond,
-		AddLookupHeaders: true,
 	})
 
+	// With no client looked up, the headers are off unless set.
 	cfg = wafFromEnvironment(t, environment{lookupSource: off})
 	wantLookupSettings(t, cfg, config.WAF{
 		LookupSource: off, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
@@ -1424,6 +1425,10 @@ func TestLookupDBPathGoesWithTheFileSourceAlone(t *testing.T) {
 			cfg.LookupSource, cfg.LookupDBPath, path)
 	}
 
+	if !cfg.AddLookupHeaders {
+		t.Error("headers off by default with lookups from file, want on")
+	}
+
 	for _, tc := range []struct {
 		env  environment
 		want string
@@ -1813,9 +1818,6 @@ func TestTagThatNoRequestCanGetIsToldApartFromOneItCan(t *testing.T) {
 			"abuseipdb with a key",
 			environment{abuseIPDBKey: "k", tagLimitPercent: "abuseipdb:50"}, "",
 		},
-		{"crowdsec, though a list has its name", environment{
-			tagLists: "crowdsec=https://example.com/c.txt", tagDeny: "crowdsec",
-		}, `"crowdsec" cannot be used`},
 		{exploitTag, environment{tagDeny: exploitTag}, ""},
 		{
 			exploitTag + " with rule files off",
@@ -1887,26 +1889,20 @@ func TestErrorOfATagSettingLeavesOutAZoneKey(t *testing.T) {
 func TestTagListURLFetchedAsAnotherListStopsTheStart(t *testing.T) {
 	t.Parallel()
 
-	const lapi = "http://172.17.0.1:8080"
-
 	for _, tc := range []struct {
 		name, listURL string
 		env           environment
 	}{
 		{"a blocklist", dropURL, environment{blocklistURLs: dropURL}},
 		{"the file of AS:percent lines", asnURL, environment{asnLimitPercentURL: asnURL}},
-		{
-			"the CrowdSec decision list", lapi + "/v1/decisions",
-			environment{"SWW_WAF_CROWDSEC_LAPI_URL": lapi, "SWW_WAF_CROWDSEC_LAPI_KEY": "key"},
-		},
 	} {
 		t.Run(tc.name, func(t *testing.T) {
 			t.Parallel()
 
 			tc.env[tagLists] = "dup=" + tc.listURL
 
-			want := tagLists + `: "` + tc.listURL + `" is in SWW_WAF_BLOCKLIST_URLS, is ` +
-				"SWW_WAF_ASN_LIMIT_PERCENT_URL or is the CrowdSec decision list too"
+			want := tagLists + `: "` + tc.listURL + `" is in SWW_WAF_BLOCKLIST_URLS ` +
+				"or is SWW_WAF_ASN_LIMIT_PERCENT_URL too"
 
 			_, err := config.WAFFromEnvironment(tc.env.viper())
 			if err == nil || err.Error() != want {
@@ -2456,138 +2452,6 @@ func TestAbuseIPDBKeyIsLoggedMasked(t *testing.T) {
 	}
 }
 
-func TestCrowdSecSettingsGiveTheDecisionListAndTheKey(t *testing.T) {
-	t.Parallel()
-
-	cfg := wafFromEnvironment(t, environment{})
-	if cfg.CrowdSecDecisionsURL != "" || cfg.CrowdSecKey != "" {
-		t.Errorf(
-			"by default, the decision list %q and the key %q, want neither",
-			cfg.CrowdSecDecisionsURL,
-			cfg.CrowdSecKey,
-		)
-	}
-
-	for lapi, want := range map[string]string{
-		"http://172.17.0.1:8080":          "http://172.17.0.1:8080/v1/decisions",
-		"http://172.17.0.1:8080/":         "http://172.17.0.1:8080/v1/decisions",
-		"https://crowdsec.example/lapi/":  "https://crowdsec.example/lapi/v1/decisions",
-		"https://crowdsec.example:8443/x": "https://crowdsec.example:8443/x/v1/decisions",
-	} {
-		cfg := wafFromEnvironment(
-			t,
-			environment{crowdSecURL: lapi, crowdSecKey: token},
-		)
-		if cfg.CrowdSecDecisionsURL != want || cfg.CrowdSecKey != token {
-			t.Errorf(
-				"%s=%s gave the decision list %q and the key %q, want %s and %s",
-				crowdSecURL,
-				lapi,
-				cfg.CrowdSecDecisionsURL,
-				cfg.CrowdSecKey,
-				want,
-				token,
-			)
-		}
-	}
-}
-
-func TestInvalidCrowdSecSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
-	t.Parallel()
-
-	const (
-		lapi       = "http://172.17.0.1:8080"
-		notLAPIURL = " is not an http or https URL without a user or a fragment, " +
-			"such as http://172.17.0.1:8080"
-		anotherList = `gives the decision list "` + lapi + `/v1/decisions", which is in ` +
-			`SWW_WAF_BLOCKLIST_URLS or is SWW_WAF_ASN_LIMIT_PERCENT_URL too`
-	)
-
-	for _, tc := range []struct {
-		name string
-		env  environment
-		want string
-	}{
-		{
-			"a URL that is not http",
-			environment{crowdSecURL: "ftp://172.17.0.1", crowdSecKey: token},
-			crowdSecURL + `: "ftp://172.17.0.1"` + notLAPIURL,
-		},
-		{
-			"a URL with a user",
-			environment{crowdSecURL: "http://bouncer@172.17.0.1:8080", crowdSecKey: token},
-			crowdSecURL + `: "http://bouncer@172.17.0.1:8080"` + notLAPIURL,
-		},
-		{
-			"the URL without the key",
-			environment{crowdSecURL: lapi},
-			crowdSecURL + ": is set while " + crowdSecKey + " is unset; the engine " +
-				"answers no request without it",
-		},
-		{
-			"the key without the URL",
-			environment{crowdSecKey: token},
-			crowdSecKey + ": is set while " + crowdSecURL + " is unset; it is sent only " +
-				"to the engine at that URL",
-		},
-		{
-			"the decision list as a blocklist too",
-			environment{
-				crowdSecURL: lapi, crowdSecKey: token,
-				blocklistURLs: "https://lists.example/drop.txt," + lapi + "/v1/decisions",
-			},
-			crowdSecURL + ": " + anotherList,
-		},
-		{
-			"the decision list as the file of AS:percent lines too",
-			environment{
-				crowdSecURL: lapi + "/", crowdSecKey: token,
-				asnLimitPercentURL: lapi + "/v1/decisions",
-			},
-			crowdSecURL + ": " + anotherList,
-		},
-		// The key itself is never shown.
-		{
-			"a key with a control character",
-			environment{crowdSecURL: lapi, crowdSecKey: token + "\r"},
-			crowdSecKey + ": holds a control character, such as the carriage return " +
-				"of a Windows line end",
-		},
-	} {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			_, err := config.WAFFromEnvironment(tc.env.viper())
-			if err == nil || err.Error() != tc.want {
-				t.Errorf("error %v, want %s", err, tc.want)
-			}
-		})
-	}
-}
-
-func TestCrowdSecKeyIsLoggedMasked(t *testing.T) {
-	t.Parallel()
-
-	cfg := wafFromEnvironment(t, environment{
-		crowdSecURL: "http://172.17.0.1:8080", crowdSecKey: token,
-	})
-
-	var out bytes.Buffer
-
-	slog.New(slog.NewJSONHandler(&out, nil)).
-		Info("starting", "settings", cfg)
-
-	logged := out.String()
-	if strings.Contains(logged, token) ||
-		!strings.Contains(logged, `"`+crowdSecKey+`":"********"`) ||
-		!strings.Contains(
-			logged,
-			`"`+crowdSecURL+`":"http://172.17.0.1:8080"`,
-		) {
-		t.Errorf("the key is not logged masked beside the URL: %s", logged)
-	}
-}
-
 func TestSizesAndOff(t *testing.T) {
 	t.Parallel()
 
@@ -2714,10 +2578,10 @@ func TestInvalidLookupTagOrLogValueStopsTheStart(t *testing.T) {
 		{tagDeny, "a,,b"}, {tagDeny, "country=nk"}, {tagDeny, "asn=abc"},
 		{tagDeny, "asn="},
 		{tagDeny, "torexit,TOREXIT"}, {tagDeny, "torexti"},
-		{tagDeny, zenTag}, {tagDeny, "abuseipdb"}, {tagDeny, "crowdsec"},
+		{tagDeny, zenTag}, {tagDeny, "abuseipdb"},
 		{tagLimitPercent, "torexti:50"},
 		{tagLimitPercent, "dnsbl=zen.example:50"},
-		{tagLimitPercent, "abuseipdb:50"}, {tagLimitPercent, "crowdsec:50"},
+		{tagLimitPercent, "abuseipdb:50"},
 		{tagLimitPercent, "vpn :50"},
 		{tagLimitPercent, "vpn"}, {tagLimitPercent, ":50"},
 		{tagLimitPercent, "vpn:"},
@@ -3070,7 +2934,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
 		lookupSource:                   defaultLookupSource,
 		lookupDBPath:                   "",
 		asnAndCountryLookupTimeout:     "800",
-		addLookupHeaders:               disabled,
+		addLookupHeaders:               enabled,
 		allowedCountries:               "",
 		tagLists:                       "",
 		tagDeny:                        "",
@@ -3091,8 +2955,6 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
 		ipReputationCacheTTL:           defaultIPReputationCacheTTL,
 		dnsblAbuseIPDBTimeout:          "2000",
 		routeWatchURL:                  "",
-		crowdSecURL:                    "",
-		crowdSecKey:                    "",
 		banResponse:                    "403",
 		limitBanDuration:               "1h",
 		limitBanRepeatWindow:           "24h",

internal/coreruleset/coreruleset_test.go

@@ -1512,13 +1512,15 @@ const (
 	wpTags    = "/wp-json/wp/v2/tags"
 )
 
-// readingBodiesUpTo64K returns the Core Rule Set as sww runs it by default,
+// readingBodiesUpTo8K returns the Core Rule Set as sww runs it by default,
 // but reading bodies up to the largest SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE.
-func readingBodiesUpTo64K(t *testing.T) *coreruleset.CoreRuleSet {
+func readingBodiesUpTo8K(t *testing.T) *coreruleset.CoreRuleSet {
 	t.Helper()
 
 	return shared(t, coreruleset.Params{
-		ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: 64<<10 - 1,
+		ParanoiaLevel:      1,
+		DisabledRules:      defaultDisabledRules,
+		BodyInspectionSize: mostBodyRead,
 	})
 }
 
@@ -1582,7 +1584,7 @@ func TestWordPressAttacksAreRefused(t *testing.T) {
 
 	const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
 
-	defaults, bodies := atDefaults(t), readingBodiesUpTo64K(t)
+	defaults, bodies := atDefaults(t), readingBodiesUpTo8K(t)
 
 	for _, attack := range []string{
 		sqlInjection,
@@ -1693,7 +1695,7 @@ func wordpressRESTAddresses(route string) []string {
 func TestWordPressExclusionsFollowEveryRESTAddress(t *testing.T) {
 	t.Parallel()
 
-	crs := readingBodiesUpTo64K(t)
+	crs := readingBodiesUpTo8K(t)
 	content := `{"content":"` + wordpressBlock + `"}`
 
 	for _, route := range []string{
@@ -1919,7 +1921,7 @@ func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(
 		wantResult(t, crs, get(target+command), matched(932235))
 	}
 
-	bodies := readingBodiesUpTo64K(t)
+	bodies := readingBodiesUpTo8K(t)
 
 	for _, p := range wordpressBodyFields(command) {
 		r := request{http.MethodPost, p.target, []string{p.contentType}}
@@ -2143,7 +2145,7 @@ func TestWordPressBodyFieldsAreCheckedInTheQuery(t *testing.T) {
 func TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere(t *testing.T) {
 	t.Parallel()
 
-	crs := readingBodiesUpTo64K(t)
+	crs := readingBodiesUpTo8K(t)
 
 	for _, v := range wordpressWords() {
 		for _, field := range []string{"title", "excerpt", "name"} {
@@ -2262,7 +2264,7 @@ func TestWordPressExclusionsAreNarrow(t *testing.T) {
 		length  = "Content-Length: 2"
 	)
 
-	crs := readingBodiesUpTo64K(t)
+	crs := readingBodiesUpTo8K(t)
 	saving := request{
 		http.MethodPost,
 		wpPost,

internal/coreruleset/inspectiontime_test.go

@@ -0,0 +1,345 @@
+package coreruleset_test
+
+import (
+	"maps"
+	"net/http"
+	"net/http/httptest"
+	"net/netip"
+	"strconv"
+	"strings"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/smallwebwaf/internal/coreruleset"
+)
+
+// The bounds on what a visitor can send, as SPEC.md gives them.
+const (
+	// lineLimit is the longest request line or header line.
+	lineLimit = 8 << 10
+	// headerLimit is the most bytes rproxy reads of a request line and
+	// headers together.
+	headerLimit = 32 << 10
+	// mostCookies and mostHeaderFields are waf's limits on cookies and header
+	// fields.
+	mostCookies      = 50
+	mostHeaderFields = 100
+	// mostBodyRead is the largest SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE.
+	mostBodyRead = 8 << 10
+)
+
+// inspectionTimeLimit is the most time the Core Rule Set may take on one
+// request. It is the default of rproxy's SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, 2
+// seconds, less the default of SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, 800
+// milliseconds, which a question can also wait for GeoJS, and less 200
+// milliseconds for waf's other checks.
+const inspectionTimeLimit = time.Second
+
+// text returns n bytes of letters, digits and spaces, which the Core Rule Set
+// takes longer over than one letter repeated.
+func text(n int) string {
+	const words = "select 0x2f from tbl_1 where name like abc_9 "
+
+	return strings.Repeat(words, n/len(words)+1)[:n]
+}
+
+// visitorRequest is a request a visitor can send, at the most of one or more
+// of the bounds above.
+type visitorRequest struct {
+	name   string
+	target string
+	// header holds the headers besides Host.
+	header http.Header
+	// contentType is the body's type, "" for no body.
+	contentType string
+	body        string
+}
+
+// longestTarget returns a target whose request line, "POST <target>
+// HTTP/1.1", is lineLimit bytes long.
+func longestTarget() string {
+	const around = len("POST /?q= HTTP/1.1")
+
+	return "/?q=" + strings.ReplaceAll(text(lineLimit-around), " ", "+")
+}
+
+// cookieLine returns a Cookie value whose line, "Cookie: <value>", is
+// lineLimit bytes long, made of count cookies.
+func cookieLine(count int) string {
+	const linePrefix = len("Cookie: ")
+
+	cookies := make([]string, count)
+	for i := range cookies {
+		cookies[i] = "c" + strconv.Itoa(i) + "="
+	}
+
+	// Each cookie is followed by "; ", but for the last.
+	room := lineLimit - linePrefix - len(strings.Join(cookies, "; "))
+	for i := range cookies {
+		share := room / count
+		if i == 0 {
+			share += room % count
+		}
+
+		cookies[i] += strings.ReplaceAll(text(share), " ", "_")
+	}
+
+	return strings.Join(cookies, "; ")
+}
+
+// headerValue returns a value whose line, "<name>: <value>", is n bytes long.
+func headerValue(name string, n int) string {
+	return text(n - len(name) - len(": "))
+}
+
+// formFields returns a form body of mostBodyRead bytes holding count fields.
+func formFields(count int) string {
+	fields := make([]string, count)
+	for i := range fields {
+		fields[i] = "f" + strconv.Itoa(i) + "="
+	}
+
+	room := mostBodyRead - len(strings.Join(fields, "&"))
+	for i := range fields {
+		share := room / count
+		if i == 0 {
+			share += room % count
+		}
+
+		fields[i] += strings.ReplaceAll(text(share), " ", "+")
+	}
+
+	return strings.Join(fields, "&")
+}
+
+// multipartBody returns a multipart body of mostBodyRead bytes, with the
+// boundary "b", holding fields of about size bytes each.
+func multipartBody(size int) string {
+	var body strings.Builder
+
+	const end = "--b--\r\n"
+
+	for i := 0; ; i++ {
+		head := "--b\r\nContent-Disposition: form-data; name=\"f" +
+			strconv.Itoa(i) + "\"\r\n\r\n"
+
+		room := mostBodyRead - body.Len() - len(
+			end,
+		) - len(
+			head,
+		) - len(
+			"\r\n",
+		)
+		if room <= 0 {
+			break
+		}
+
+		body.WriteString(head + text(min(size, room)) + "\r\n")
+	}
+
+	body.WriteString(end)
+
+	return body.String()
+}
+
+// jsonArrayOfMostElements returns a JSON array of at most mostBodyRead bytes
+// holding as many elements as fit, each the number 1.
+func jsonArrayOfMostElements() string {
+	return "[1" + strings.Repeat(
+		",1",
+		(mostBodyRead-len("[1]"))/len(",1"),
+	) + "]"
+}
+
+// The kinds of body the Core Rule Set reads, and the header that names a
+// request's cookies.
+const (
+	formType      = "application/x-www-form-urlencoded"
+	jsonType      = "application/json"
+	multipartType = "multipart/form-data; boundary=b"
+	cookieHeader  = "Cookie"
+)
+
+// fullLine is the length of each of the request line and the three or four
+// header lines that together fill rproxy's limit, with room for the line
+// endings and the other headers.
+const fullLine = lineLimit - 64
+
+// worstCaseRequests are, for each kind of thing a visitor sends, the requests
+// that keep the Core Rule Set busiest, within the bounds above. The last
+// holds the most of everything at once: Cookie lines keep it busier than
+// other header lines, and the JSON body of the most elements keeps it busier
+// than the other bodies.
+func worstCaseRequests() []visitorRequest {
+	fullCookieLine := cookieLine(1)[:fullLine-len("Cookie: ")]
+
+	return append(
+		append(worstCaseHeaders(), worstCaseBodies()...),
+		visitorRequest{
+			name:   "the longest request line, three Cookie lines and body",
+			target: longestTarget()[:fullLine-len("POST  HTTP/1.1")],
+			header: http.Header{cookieHeader: {
+				fullCookieLine, fullCookieLine, fullCookieLine,
+			}},
+			contentType: jsonType, body: jsonArrayOfMostElements(),
+		},
+	)
+}
+
+// worstCaseHeaders are the requests of worstCaseRequests with no body.
+func worstCaseHeaders() []visitorRequest {
+	headerFields := http.Header{}
+
+	for i := range mostHeaderFields - 1 {
+		name := "X-Field-" + strconv.Itoa(i)
+		headerFields.Set(
+			name,
+			headerValue(name, headerLimit/mostHeaderFields),
+		)
+	}
+
+	return []visitorRequest{
+		{name: "the longest request line", target: longestTarget()},
+		{
+			name: "the longest request line, as query parameters",
+			target: "/?" + strings.Repeat(
+				"a=1&",
+				(lineLimit-len("POST /? HTTP/1.1"))/len("a=1&"),
+			),
+		},
+		{
+			name:   "the longest Cookie line, as one cookie",
+			target: "/",
+			header: http.Header{cookieHeader: {cookieLine(1)}},
+		},
+		{
+			name:   "the longest Cookie line, as the most cookies",
+			target: "/",
+			header: http.Header{cookieHeader: {cookieLine(mostCookies)}},
+		},
+		{name: "the most header fields", target: "/", header: headerFields},
+		{
+			name:   "the longest header lines",
+			target: "/",
+			header: http.Header{
+				"User-Agent": {headerValue("User-Agent", fullLine)},
+				"Referer":    {headerValue("Referer", fullLine)},
+				"Accept":     {headerValue("Accept", fullLine)},
+				"X-Long":     {headerValue("X-Long", fullLine)},
+			},
+		},
+	}
+}
+
+// worstCaseBodies are the requests of worstCaseRequests whose body holds
+// the most the Core Rule Set reads.
+func worstCaseBodies() []visitorRequest {
+	const nested = (mostBodyRead - len(`{"a":1}`)) / len(`{"a":}`)
+
+	return []visitorRequest{
+		{
+			name: "a form body of one field", target: "/",
+			contentType: formType, body: formFields(1),
+		},
+		{
+			name: "a form body of the most fields Coraza keeps", target: "/",
+			contentType: formType, body: formFields(1000),
+		},
+		{
+			name: "a JSON body of one string", target: "/",
+			contentType: jsonType,
+			body: `{"a":"` + text(
+				mostBodyRead-len(`{"a":""}`),
+			) + `"}`,
+		},
+		{
+			name: "a JSON body of the most elements", target: "/",
+			contentType: jsonType, body: jsonArrayOfMostElements(),
+		},
+		{
+			name: "a JSON body nested deep", target: "/",
+			contentType: jsonType,
+			body: strings.Repeat(`{"a":`, nested) + "1" +
+				strings.Repeat("}", nested),
+		},
+		{
+			name: "an XML body of many elements", target: "/",
+			contentType: "application/xml",
+			body: "<a>" + strings.Repeat(
+				"<b>x</b>",
+				(mostBodyRead-len("<a></a>"))/len("<b>x</b>"),
+			) + "</a>",
+		},
+		{
+			name: "a multipart body of one field", target: "/",
+			contentType: multipartType, body: multipartBody(mostBodyRead),
+		},
+		{
+			name: "a multipart body of many fields", target: "/",
+			contentType: multipartType, body: multipartBody(1),
+		},
+	}
+}
+
+// inspectionTime returns how long crs takes to inspect tc, and fails t unless
+// it reads all of tc's body.
+func inspectionTime(
+	t *testing.T, crs *coreruleset.CoreRuleSet, tc visitorRequest,
+) time.Duration {
+	t.Helper()
+
+	req := httptest.NewRequestWithContext(
+		t.Context(), http.MethodPost, tc.target, nil,
+	)
+	maps.Copy(req.Header, tc.header)
+
+	if tc.contentType != "" {
+		req.Header.Set("Content-Type", tc.contentType)
+	}
+
+	began := time.Now()
+	_, read, _ := crs.Inspect(
+		req, netip.MustParseAddr("192.0.2.1"), strings.NewReader(tc.body),
+	)
+	took := time.Since(began)
+
+	if string(read) != tc.body {
+		t.Errorf("read %d bytes of %d", len(read), len(tc.body))
+	}
+
+	return took
+}
+
+// TestTheCoreRuleSetInspectsEachWorstCaseRequestInTime runs one at a time, so
+// that the tests running in parallel do not slow it. Other work on the host
+// can still slow one try, so a request fails only when three tries in a row
+// take too long.
+//
+//nolint:paralleltest // one at a time, as above
+func TestTheCoreRuleSetInspectsEachWorstCaseRequestInTime(t *testing.T) {
+	const tries = 3
+
+	// The settings that make the Core Rule Set slowest: the highest paranoia
+	// level, no rule switched off, and the most of a body read.
+	crs := shared(t, coreruleset.Params{
+		ParanoiaLevel: 4, BodyInspectionSize: mostBodyRead,
+	})
+
+	for _, tc := range worstCaseRequests() {
+		t.Run(tc.name, func(t *testing.T) {
+			var took time.Duration
+
+			for range tries {
+				took = inspectionTime(t, crs, tc)
+				if took <= inspectionTimeLimit {
+					return
+				}
+			}
+
+			t.Errorf(
+				"took %s on the last of %d tries, over %s",
+				took, tries, inspectionTimeLimit,
+			)
+		})
+	}
+}

internal/metrics/metrics.go

@@ -185,7 +185,7 @@ func (m *Metrics) AddBansAndClients(
 	ledger *bans.Ledger, limiter *ratelimit.Limiter, now func() time.Time,
 ) {
 	for _, cause := range []string{
-		bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin, bans.CauseCrowdSec,
+		bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin,
 	} {
 		m.registry.MustRegister(
 			prometheus.NewCounterFunc(prometheus.CounterOpts{
@@ -302,20 +302,20 @@ const sourceLabel = "source"
 
 // AddReputation adds the metrics of the lists fetched from URLs and of the
 // DNSBL zones, by source, each list's URL or each zone, its key masked as
-// config.MaskZoneKey masks it: the requests whose client a blocklist, the
-// CrowdSec decision list, a zone's verdict or AbuseIPDB's score lists,
-// which ReputationHit counts, and, read from lists and dnsbl as the
-// metrics are asked for, for a list, the fetches that failed and when the
-// copy in use was fetched, and for a zone, the queries made and those that
-// failed. It is called once, before ReputationHit.
+// config.MaskZoneKey masks it: the requests whose client a blocklist, a
+// zone's verdict or AbuseIPDB's score lists, which ReputationHit counts,
+// and, read from lists and dnsbl as the metrics are asked for, for a list,
+// the fetches that failed and when the copy in use was fetched, and for a
+// zone, the queries made and those that failed. It is called once, before
+// ReputationHit.
 func (m *Metrics) AddReputation(
 	lists *reputation.Lists,
 	dnsbl *reputation.DNSBL,
 ) {
 	m.reputationHits = counterVec(
 		"sww_reputation_hits_total",
-		"Requests whose client a blocklist, the CrowdSec decision list, a DNSBL "+
-			"zone or AbuseIPDB lists, by the list's URL, the zone, or abuseipdb.",
+		"Requests whose client a blocklist, a DNSBL zone or AbuseIPDB lists, "+
+			"by the list's URL, the zone, or abuseipdb.",
 		[]string{sourceLabel},
 	)
 	m.registry.MustRegister(m.reputationHits)
@@ -378,9 +378,8 @@ func (m *Metrics) AddAbuseIPDB(abuseIPDB *reputation.AbuseIPDB) {
 	)
 }
 
-// ReputationHit counts a request whose client source lists: a blocklist
-// or the CrowdSec decision list, by its URL, a DNSBL zone, its key masked,
-// or AbuseIPDB, abuseipdb.
+// ReputationHit counts a request whose client source lists: a blocklist,
+// by its URL, a DNSBL zone, its key masked, or AbuseIPDB, abuseipdb.
 func (m *Metrics) ReputationHit(source string) {
 	m.reputationHits.WithLabelValues(source).Inc()
 }

internal/reputation/crowdsec_test.go

@@ -1,622 +0,0 @@
-package reputation_test
-
-import (
-	"bytes"
-	"encoding/json"
-	"fmt"
-	"io"
-	"log/slog"
-	"net/http"
-	"net/netip"
-	"reflect"
-	"strings"
-	"sync"
-	"testing"
-	"testing/synctest"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/alerts"
-	"sneak.berlin/go/smallwebwaf/internal/reputation"
-)
-
-// The tests run in a synctest bubble, as those of the blocklists do, and
-// fetch the decision list from engine, a stand-in for a CrowdSec engine
-// that answers without the network.
-
-const (
-	// decisionsURL is the decision list of the tests' engine, and engineKey
-	// the key it answers.
-	decisionsURL = "http://crowdsec.example:8080/v1/decisions"
-	engineKey    = "crowdsec-key-0123456789abcdef"
-	// sshBF and probing are scenarios of the engine's decisions.
-	sshBF   = "crowdsecurity/ssh-bf"
-	probing = "crowdsecurity/http-probing"
-	// ban is the type of a decision to ban, and rangeScope the scope of a
-	// decision on a netblock, as CrowdSec names them.
-	ban        = "ban"
-	rangeScope = "Range"
-)
-
-func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	synctest.Test(t, func(t *testing.T) {
-		began := time.Now()
-		manual := "manual 'ban' from 'localhost'"
-		e := &engine{key: engineKey, decisions: []decision{
-			{"Ip", suspect, ban, manual, began.Add(6 * time.Hour)},
-			// A shorter decision on the same address, which is not the one
-			// used.
-			{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
-			{
-				rangeScope,
-				"198.51.100.0/24",
-				ban,
-				probing,
-				began.Add(time.Hour),
-			},
-			{"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)},
-			// Left out: a decision to show a captcha, and one on a country.
-			{"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)},
-			{"Country", "KP", ban, manual, began.Add(time.Hour)},
-		}}
-		lists := start(t, e, crowdSecParams())
-
-		for addr, want := range map[string]reputation.Decision{
-			suspect:          {Expires: began.Add(6 * time.Hour), Scenario: manual},
-			"198.51.100.0":   {Expires: began.Add(time.Hour), Scenario: probing},
-			"198.51.100.255": {Expires: began.Add(time.Hour), Scenario: probing},
-			"2001:db8::1":    {Expires: began.Add(2 * time.Hour), Scenario: sshBF},
-			"203.0.113.10":   {},
-			"198.51.101.0":   {},
-			"2001:db8::2":    {},
-			"192.0.2.50":     {},
-		} {
-			wantDecision(t, lists, addr, want)
-		}
-
-		// With the engine down, the copy kept still holds the decision on
-		// 198.51.100.0/24, which no longer bans once it has ended.
-		e.set(func(e *engine) { e.failing = true })
-		time.Sleep(time.Hour - time.Nanosecond)
-		synctest.Wait()
-		wantDecision(
-			t,
-			lists,
-			"198.51.100.7",
-			reputation.Decision{
-				Expires:  began.Add(time.Hour),
-				Scenario: probing,
-			},
-		)
-
-		time.Sleep(time.Nanosecond)
-		synctest.Wait()
-		wantDecision(t, lists, "198.51.100.7", reputation.Decision{})
-		wantDecision(
-			t,
-			lists,
-			suspect,
-			reputation.Decision{
-				Expires:  began.Add(6 * time.Hour),
-				Scenario: manual,
-			},
-		)
-	})
-}
-
-func TestCrowdSecDecisionListFetchedAgainEveryMinute(t *testing.T) {
-	t.Parallel()
-
-	synctest.Test(t, func(t *testing.T) {
-		began := time.Now()
-		e := &engine{key: engineKey, decisions: []decision{
-			{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
-		}}
-		lists := start(t, e, crowdSecParams())
-		wantEngineFetches(t, e, 1)
-
-		added := reputation.Decision{
-			Expires:  began.Add(2 * time.Hour),
-			Scenario: probing,
-		}
-
-		e.set(func(e *engine) {
-			e.decisions = append(e.decisions,
-				decision{"Ip", "203.0.113.10", ban, probing, added.Expires})
-		})
-
-		time.Sleep(time.Minute - time.Nanosecond)
-		wantEngineFetches(t, e, 1)
-		wantDecision(t, lists, "203.0.113.10", reputation.Decision{})
-
-		time.Sleep(time.Nanosecond)
-		wantEngineFetches(t, e, 2)
-		wantDecision(t, lists, "203.0.113.10", added)
-	})
-}
-
-func TestCrowdSecDecisionOnAClientIsTheOneThatEndsLast(t *testing.T) {
-	t.Parallel()
-
-	synctest.Test(t, func(t *testing.T) {
-		began := time.Now()
-		e := &engine{key: engineKey, decisions: []decision{
-			// Two decisions on one address, the shorter listed first.
-			{"Ip", suspect, ban, sshBF, began.Add(2 * time.Hour)},
-			{"Ip", suspect, ban, probing, began.Add(4 * time.Hour)},
-			// 198.51.100.130 is held by a decision on its address that ends
-			// after the one on its netblock, and 192.0.2.20 by one that ends
-			// before.
-			{
-				rangeScope,
-				"198.51.100.128/25",
-				ban,
-				sshBF,
-				began.Add(time.Hour),
-			},
-			{"Ip", "198.51.100.130", ban, probing, began.Add(3 * time.Hour)},
-			{
-				rangeScope,
-				"192.0.2.0/24",
-				ban,
-				probing,
-				began.Add(5 * time.Hour),
-			},
-			{"Ip", "192.0.2.20", ban, sshBF, began.Add(2 * time.Hour)},
-		}}
-		lists := start(t, e, crowdSecParams())
-
-		wantDecision(
-			t,
-			lists,
-			suspect,
-			reputation.Decision{
-				Expires:  began.Add(4 * time.Hour),
-				Scenario: probing,
-			},
-		)
-		wantDecision(
-			t,
-			lists,
-			"198.51.100.130",
-			reputation.Decision{
-				Expires:  began.Add(3 * time.Hour),
-				Scenario: probing,
-			},
-		)
-		wantDecision(
-			t,
-			lists,
-			"192.0.2.20",
-			reputation.Decision{
-				Expires:  began.Add(5 * time.Hour),
-				Scenario: probing,
-			},
-		)
-	})
-}
-
-func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	synctest.Test(t, func(t *testing.T) {
-		began := time.Now()
-		e := &engine{key: engineKey, decisions: []decision{
-			{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
-		}}
-		lists := start(t, e, crowdSecParams())
-
-		// With its decision deleted, the engine answers null.
-		e.set(func(e *engine) { e.decisions = nil })
-		time.Sleep(time.Minute)
-		wantEngineFetches(t, e, 2)
-
-		want := []reputation.List{{
-			URL: decisionsURL, Tried: time.Now(), Fetched: time.Now(), Lines: []string{"null"},
-		}}
-		if got := lists.Snapshot(); !reflect.DeepEqual(got, want) {
-			t.Errorf("lists %+v, want %+v", got, want)
-		}
-
-		if lists.Failures(decisionsURL) != 0 {
-			t.Errorf("%d failures, want 0", lists.Failures(decisionsURL))
-		}
-
-		wantDecision(t, lists, suspect, reputation.Decision{})
-	})
-}
-
-func TestCrowdSecFailureKeepsTheLastGoodCopyAlertsOncePerCooldownAndHidesTheKey(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	for _, tc := range crowdSecFailures() {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			synctest.Test(t, func(t *testing.T) {
-				var log bytes.Buffer
-
-				began := time.Now()
-				e := &engine{key: engineKey, decisions: []decision{
-					{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
-				}}
-				queue := newQueue()
-				p := crowdSecParams()
-				p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
-				p.Alerts = queue
-				lists := start(t, e, p)
-				kept := lists.Snapshot()
-
-				e.set(tc.fail)
-
-				// Each failure is tried again a minute after it.
-				for range 2 {
-					time.Sleep(time.Minute)
-					synctest.Wait()
-				}
-
-				wantEngineFetches(t, e, 3)
-				wantDecision(
-					t,
-					lists,
-					suspect,
-					reputation.Decision{
-						Expires:  began.Add(4 * time.Hour),
-						Scenario: sshBF,
-					},
-				)
-
-				want := kept[0]
-				want.Tried = time.Now()
-
-				got := lists.Snapshot()
-				if !reflect.DeepEqual(got, []reputation.List{want}) {
-					t.Errorf(
-						"lists %+v, want the first copy, last tried now, %+v",
-						got,
-						want,
-					)
-				}
-
-				if n := lists.Failures(decisionsURL); n != 2 {
-					t.Errorf("%d failures, want 2", n)
-				}
-
-				// One alert for the first failure; the cooldown holds back the
-				// second.
-				failed := time.Now().Add(-time.Minute)
-				alert := fetchFailure(failed, decisionsURL, tc.error)
-				wantAlert(t, queue, alert)
-
-				if !strings.Contains(
-					log.String(),
-					`"msg":"fetching a list failed",`+
-						`"url":"`+decisionsURL+`","error":"`+tc.error,
-				) {
-					t.Errorf("logged\n%s\nwant the failures", log.String())
-				}
-
-				wantKeyNotShown(t, e, log.String(), lists, queue)
-			})
-		})
-	}
-}
-
-// crowdSecFailure is a way for the engine to fail: fail has it answer the
-// fetches after the first so that they fail with error.
-type crowdSecFailure struct {
-	name  string
-	fail  func(e *engine)
-	error string
-}
-
-// crowdSecFailures returns the ways the engine can fail.
-func crowdSecFailures() []crowdSecFailure {
-	const notDecision = " does not give an address or a netblock and a duration, " +
-		"such as 4h0m0s"
-
-	return []crowdSecFailure{
-		{
-			"an answer other than 200",
-			func(e *engine) { e.failing = true },
-			"the server answered 503 Service Unavailable",
-		},
-		{
-			"a key the engine refuses",
-			func(e *engine) { e.key = "another-key-0123456789abcdef" },
-			"the server answered 403 Forbidden",
-		},
-		{
-			"a redirect",
-			func(e *engine) { e.redirect = "http://elsewhere.example/v1/decisions" },
-			"the server answered 302 Found",
-		},
-		{
-			"an answer that does not read",
-			func(e *engine) { e.answer = "<html>" },
-			"read the answer: invalid character '<' looking for beginning of value",
-		},
-		{
-			"a decision to ban whose value does not read",
-			func(e *engine) {
-				e.answer = `[{"duration": "4h", "scenario": "` + sshBF + `", ` +
-					`"scope": "Ip", "type": "ban", "value": "203.0.113.300"}]`
-			},
-			"decision 1" + notDecision,
-		},
-		{
-			"a decision to ban whose duration does not read",
-			func(e *engine) {
-				e.answer = `[{"duration": "4h", "scope": "Country", "type": "ban", ` +
-					`"value": "KP"}, {"duration": "four hours", "scope": "Range", ` +
-					`"type": "ban", "value": "198.51.100.0/24"}]`
-			},
-			"decision 2" + notDecision,
-		},
-	}
-}
-
-// wantKeyNotShown checks that no fetch carried the engine's key to a URL
-// other than its decision list, such as the one a redirect names, and that
-// the key is in none of what the fetches leave behind: log, the process
-// log, the alerts waiting in queue, and the copies of lists, which
-// lists.json keeps.
-func wantKeyNotShown(
-	t *testing.T,
-	e *engine,
-	log string,
-	lists *reputation.Lists,
-	queue *alerts.Queue,
-) {
-	t.Helper()
-
-	e.mu.Lock()
-	keySentTo := e.keySentTo
-	e.mu.Unlock()
-
-	if len(keySentTo) != 0 {
-		t.Errorf("the key was sent to %v", keySentTo)
-	}
-
-	shown, err := json.Marshal([]any{lists.Snapshot(), waiting(queue)})
-	if err != nil {
-		t.Fatalf("encode: %v", err)
-	}
-
-	if strings.Contains(log+string(shown), engineKey) {
-		t.Errorf("the key is shown in\n%s\n%s", log, shown)
-	}
-}
-
-func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	synctest.Test(t, func(t *testing.T) {
-		began := time.Now()
-		e := &engine{key: engineKey, decisions: []decision{
-			{
-				rangeScope,
-				"198.51.100.0/24",
-				ban,
-				probing,
-				began.Add(time.Hour),
-			},
-		}}
-		lists := start(t, e, crowdSecParams())
-		kept := lists.Snapshot()
-
-		// Restarted half an hour later with what lists.json keeps, and
-		// the engine down, the decision still bans, until the end it had at
-		// the fetch, half an hour on.
-		time.Sleep(30 * time.Minute)
-
-		down := &engine{key: engineKey, failing: true}
-		again := reputation.New(crowdSecParams())
-		again.SetTransport(down)
-
-		err := again.Load(kept)
-		if err != nil {
-			t.Fatalf("load: %v", err)
-		}
-
-		run(t, again)
-
-		want := reputation.Decision{
-			Expires:  began.Add(time.Hour),
-			Scenario: probing,
-		}
-		wantDecision(t, again, "198.51.100.7", want)
-
-		time.Sleep(30*time.Minute - time.Nanosecond)
-		synctest.Wait()
-		wantDecision(t, again, "198.51.100.7", want)
-
-		time.Sleep(time.Nanosecond)
-		synctest.Wait()
-		wantDecision(t, again, "198.51.100.7", reputation.Decision{})
-	})
-}
-
-func TestLoadTakesACrowdSecListNeverFetchedAndRefusesACopyThatDoesNotRead(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	now := time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)
-	lists := reputation.New(crowdSecParams())
-
-	// Tried, and never fetched: there is no copy to read.
-	err := lists.Load([]reputation.List{{URL: decisionsURL, Tried: now}})
-	if err != nil {
-		t.Errorf("load the list never fetched: %v", err)
-	}
-
-	err = lists.Load([]reputation.List{{
-		URL: decisionsURL, Tried: now, Fetched: now, Lines: []string{
-			`[{"duration": "4h", "scope": "Range", "type": "ban", ` +
-				`"value": "198.51.100.0/33"}]`,
-		},
-	}})
-
-	const want = "the copy of " + decisionsURL + ": decision 1 does not give an " +
-		"address or a netblock and a duration, such as 4h0m0s"
-	if err == nil || err.Error() != want {
-		t.Errorf("error %v, want %s", err, want)
-	}
-}
-
-// engine is a stand-in for the local API of a CrowdSec engine. It answers
-// a fetch of the decision list that carries its key in X-Api-Key with its
-// decisions still in force, each with the time it has left as it answers,
-// by the bubble's clock, as an engine does, or with answer while that is
-// not "". It answers 403 to a fetch without its key, as an engine does,
-// with a redirect to redirect while that is not "", and 503 while failing.
-// It counts the fetches, and notes in keySentTo the URL of each fetch of
-// another URL that carries a key, as one following a redirect would.
-type engine struct {
-	mu        sync.Mutex
-	key       string
-	decisions []decision
-	answer    string
-	redirect  string
-	failing   bool
-	fetches   int
-	keySentTo []string
-}
-
-// decision is a decision of the engine, which ends at expires.
-type decision struct {
-	scope, value, kind, scenario string
-	expires                      time.Time
-}
-
-// RoundTrip has the engine answer req, in place of the network.
-func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
-	e.mu.Lock()
-	defer e.mu.Unlock()
-
-	e.fetches++
-
-	if req.URL.String() != decisionsURL &&
-		req.Header.Get("X-Api-Key") != "" {
-		e.keySentTo = append(e.keySentTo, req.URL.String())
-	}
-
-	status, header, body := http.StatusOK, http.Header{}, e.answer
-
-	switch {
-	case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key:
-		status, body = http.StatusForbidden, `{"message":"access forbidden"}`
-	case e.redirect != "":
-		status, header = http.StatusFound, http.Header{
-			"Location": {e.redirect},
-		}
-	case e.failing:
-		status, body = http.StatusServiceUnavailable, ""
-	case body == "":
-		body = e.inForce(time.Now())
-	}
-
-	return &http.Response{
-		StatusCode: status,
-		Status:     fmt.Sprintf("%d %s", status, http.StatusText(status)),
-		Header:     header,
-		Body:       io.NopCloser(strings.NewReader(body)),
-		Request:    req,
-	}, nil
-}
-
-// inForce returns the decisions in force at now, as the engine answers
-// them: a JSON list, null for none.
-func (e *engine) inForce(now time.Time) string {
-	var answer []map[string]string
-
-	for _, d := range e.decisions {
-		if now.Before(d.expires) {
-			answer = append(answer, map[string]string{
-				"duration": d.expires.Sub(now).
-					String(),
-				"origin":   "crowdsec",
-				"scenario": d.scenario, "scope": d.scope, "type": d.kind, "value": d.value,
-			})
-		}
-	}
-
-	body, err := json.Marshal(answer)
-	if err != nil {
-		panic(err) // a list of maps of strings always encodes
-	}
-
-	return string(body)
-}
-
-// set changes the engine with change.
-func (e *engine) set(change func(e *engine)) {
-	e.mu.Lock()
-	defer e.mu.Unlock()
-
-	change(e)
-}
-
-// crowdSecParams returns the Params of the decision list of the tests'
-// engine, fetched with its key, by the bubble's clock, with alerts to a
-// queue that sends none.
-func crowdSecParams() reputation.Params {
-	p := params()
-	p.CrowdSecDecisionsURL = decisionsURL
-	p.CrowdSecKey = engineKey
-
-	return p
-}
-
-// wantEngineFetches waits until Run has made the fetches due, and checks
-// how many the engine has had.
-func wantEngineFetches(t *testing.T, e *engine, want int) {
-	t.Helper()
-
-	synctest.Wait()
-
-	e.mu.Lock()
-	got := e.fetches
-	e.mu.Unlock()
-
-	if got != want {
-		t.Errorf("%d fetches, want %d", got, want)
-	}
-}
-
-// wantDecision checks the decision lists says is in force on addr now,
-// the zero Decision for none.
-func wantDecision(
-	t *testing.T,
-	lists *reputation.Lists,
-	addr string,
-	want reputation.Decision,
-) {
-	t.Helper()
-
-	got, listed := lists.CrowdSecDecision(
-		netip.MustParseAddr(addr),
-		time.Now(),
-	)
-	if listed != !want.Expires.IsZero() ||
-		listed &&
-			(!got.Expires.Equal(want.Expires) || got.Scenario != want.Scenario) {
-		t.Errorf(
-			"%s has the decision %+v in force %t, want %+v",
-			addr,
-			got,
-			listed,
-			want,
-		)
-	}
-}

internal/reputation/export_test.go

@@ -11,7 +11,6 @@ import (
 // network.
 func (l *Lists) SetTransport(transport http.RoundTripper) {
 	l.httpClient.Transport = transport
-	l.crowdSecClient.Transport = transport
 }
 
 // SetTransport has a's checks go through transport instead of the

internal/reputation/reputation.go

@@ -1,8 +1,7 @@
 // Package reputation fetches the lists the settings name by URL: the
 // blocklists of SWW_WAF_BLOCKLIST_URLS, the file of AS:percent lines
-// SWW_WAF_ASN_LIMIT_PERCENT_URL names, the address lists of SWW_WAF_TAG_LISTS, and
-// the decision list of the CrowdSec engine SWW_WAF_CROWDSEC_LAPI_URL names. It
-// keeps the last good copy of
+// SWW_WAF_ASN_LIMIT_PERCENT_URL names, and the address lists of
+// SWW_WAF_TAG_LISTS. It keeps the last good copy of
 // each, whole, comment lines included, which is used while a fetch fails,
 // and when each was last tried. It also asks the DNSBL zones of
 // SWW_WAF_DNSBL_ZONES about clients, and keeps their verdicts, and checks
@@ -13,7 +12,6 @@ package reputation
 
 import (
 	"context"
-	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
@@ -35,10 +33,6 @@ const (
 	maxListBytes = 16 << 20
 	// fetchTimeout bounds one fetch of a list.
 	fetchTimeout = time.Minute
-	// crowdSecRefresh is how long after the CrowdSec decision list was last
-	// fetched or tried it is fetched again: the engine is the operator's
-	// own, and makes and ends decisions all the time.
-	crowdSecRefresh = time.Minute
 	// mappedBits is the length of ::ffff:0.0.0.0/96, the netblock of every
 	// IPv4-mapped address.
 	mappedBits = 96
@@ -52,9 +46,6 @@ var (
 	)
 	errNotASNPercent = errors.New(
 		"is not an AS number, : and a percentage, such as AS64496:50")
-	errNotDecision = errors.New(
-		"does not give an address or a netblock and a duration, such as 4h0m0s",
-	)
 )
 
 // List is a list as lists.json holds it: the URL it is fetched from,
@@ -77,14 +68,8 @@ type Params struct {
 	ASNLimitPercentURL string
 	// TagLists are the address lists that add a tag (SWW_WAF_TAG_LISTS).
 	TagLists []config.TagList
-	// CrowdSecDecisionsURL is the CrowdSec decision list, "" while
-	// SWW_WAF_CROWDSEC_LAPI_URL is unset, fetched with CrowdSecKey
-	// (SWW_WAF_CROWDSEC_LAPI_KEY).
-	CrowdSecDecisionsURL string
-	CrowdSecKey          string
 	// Refresh is how long after a list was last fetched or tried it is
-	// fetched again (SWW_WAF_BLOCKLIST_REFRESH), but for the CrowdSec decision
-	// list, which is fetched again crowdSecRefresh after.
+	// fetched again (SWW_WAF_BLOCKLIST_REFRESH).
 	Refresh time.Duration
 	// Now tells the time, normally time.Now in UTC.
 	Now func() time.Time
@@ -99,10 +84,6 @@ type Params struct {
 type Lists struct {
 	params     Params
 	httpClient *http.Client
-	// crowdSecClient fetches the CrowdSec decision list. It follows no
-	// redirect, so that the key goes to the engine alone: a redirect is a
-	// failure.
-	crowdSecClient *http.Client
 
 	mu sync.Mutex
 	// lists are by URL, one for each URL Params names.
@@ -119,22 +100,12 @@ type list struct {
 }
 
 // entries are what the lines of a copy say: for a blocklist or a tag list, the
-// netblocks it names, with the lengths among them, for the file of AS:percent
-// lines, the percentage it gives each AS number, and for the CrowdSec decision
-// list, the decision on each netblock that ends last, with the lengths among
-// them.
+// netblocks it names, with the lengths among them, and for the file of
+// AS:percent lines, the percentage it gives each AS number.
 type entries struct {
 	netblocks map[netip.Prefix]bool
 	lengths   []int
 	percents  map[string]int64
-	decisions map[netip.Prefix]Decision
-}
-
-// Decision is a decision of the CrowdSec engine to ban a netblock: when
-// it ends, and the scenario that made it, such as crowdsecurity/ssh-bf.
-type Decision struct {
-	Expires  time.Time
-	Scenario string
 }
 
 // New returns the lists, without a copy of any yet.
@@ -142,12 +113,7 @@ func New(params Params) *Lists {
 	l := &Lists{
 		params:     params,
 		httpClient: &http.Client{},
-		crowdSecClient: &http.Client{
-			CheckRedirect: func(*http.Request, []*http.Request) error {
-				return http.ErrUseLastResponse
-			},
-		},
-		lists: map[string]*list{},
+		lists:      map[string]*list{},
 	}
 
 	for _, listURL := range l.URLs() {
@@ -159,7 +125,7 @@ func New(params Params) *Lists {
 
 // URLs returns the URL of every list: the blocklists' in the order
 // SWW_WAF_BLOCKLIST_URLS names them, then SWW_WAF_ASN_LIMIT_PERCENT_URL, then the
-// tag lists', then the CrowdSec decision list's.
+// tag lists'.
 func (l *Lists) URLs() []string {
 	urls := slices.Clone(l.params.BlocklistURLs)
 	if l.params.ASNLimitPercentURL != "" {
@@ -170,10 +136,6 @@ func (l *Lists) URLs() []string {
 		urls = append(urls, list.URL)
 	}
 
-	if l.params.CrowdSecDecisionsURL != "" {
-		urls = append(urls, l.params.CrowdSecDecisionsURL)
-	}
-
 	return urls
 }
 
@@ -225,40 +187,6 @@ func (l *Lists) ASNLimitPercent(asn string) (int64, bool) {
 	return percent, listed
 }
 
-// CrowdSecDecision returns the decision of the copy of the CrowdSec
-// decision list on a netblock that holds addr and that ends last, and
-// whether it is still in force at now. A decision that has ended no
-// longer bans, even before the next fetch drops it.
-func (l *Lists) CrowdSecDecision(
-	addr netip.Addr,
-	now time.Time,
-) (Decision, bool) {
-	if l.params.CrowdSecDecisionsURL == "" {
-		return Decision{}, false
-	}
-
-	l.mu.Lock()
-	defer l.mu.Unlock()
-
-	kept := l.lists[l.params.CrowdSecDecisionsURL].entries
-
-	var last Decision
-
-	for _, length := range kept.lengths {
-		netblock, err := addr.Prefix(length)
-		if err != nil {
-			continue // an IPv6 netblock's length, past an IPv4 address's 32 bits
-		}
-
-		decision := kept.decisions[netblock]
-		if decision.Expires.After(last.Expires) {
-			last = decision
-		}
-	}
-
-	return last, now.Before(last.Expires)
-}
-
 // Fetched returns when the copy in use of the list at listURL was
 // fetched, or zero while there is none.
 func (l *Lists) Fetched(listURL string) time.Time {
@@ -331,7 +259,7 @@ func (l *Lists) Load(lists []List) error {
 			continue // dropped, or a list tried but never fetched, without a copy
 		}
 
-		read, err := l.parse(kept.URL, kept.Lines, kept.Fetched)
+		read, err := l.parse(kept.URL, kept.Lines)
 		if err != nil {
 			return fmt.Errorf("the copy of %s: %w", kept.URL, err)
 		}
@@ -377,8 +305,7 @@ func (l *Lists) fetchDue(ctx context.Context) time.Time {
 }
 
 // due returns when the list at listURL is to be fetched: Refresh after it
-// was last fetched or tried, the later of the two, or crowdSecRefresh
-// after for the CrowdSec decision list.
+// was last fetched or tried, the later of the two.
 func (l *Lists) due(listURL string) time.Time {
 	l.mu.Lock()
 	defer l.mu.Unlock()
@@ -390,10 +317,6 @@ func (l *Lists) due(listURL string) time.Time {
 		last = held.kept.Tried
 	}
 
-	if listURL == l.params.CrowdSecDecisionsURL {
-		return last.Add(crowdSecRefresh)
-	}
-
 	return last.Add(l.params.Refresh)
 }
 
@@ -408,7 +331,7 @@ func (l *Lists) fetch(ctx context.Context, listURL string) {
 
 	var found entries
 	if err == nil {
-		found, err = l.parse(listURL, lines, now)
+		found, err = l.parse(listURL, lines)
 	}
 
 	cutOff := err != nil && ctx.Err() != nil
@@ -468,11 +391,8 @@ func raiseFailure(queue *alerts.Queue, reason, source string, err error) {
 	})
 }
 
-// get fetches the list at listURL, and returns its lines. The CrowdSec
-// decision list is fetched with CrowdSecKey in the header X-Api-Key, where
-// the engine looks for it, by crowdSecClient, which follows no redirect.
-// An answer other than 200, or a list longer than maxListBytes, is a
-// failure.
+// get fetches the list at listURL, and returns its lines. An answer other
+// than 200, or a list longer than maxListBytes, is a failure.
 func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
 	ctx, cancel := context.WithTimeout(ctx, fetchTimeout)
 	defer cancel()
@@ -487,14 +407,7 @@ func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
 		return nil, fmt.Errorf("make the request: %w", err)
 	}
 
-	client := l.httpClient
-
-	if listURL == l.params.CrowdSecDecisionsURL {
-		req.Header.Set("X-Api-Key", l.params.CrowdSecKey)
-		client = l.crowdSecClient
-	}
-
-	res, err := client.Do(req)
+	res, err := l.httpClient.Do(req)
 	if err != nil {
 		// Do's error names the URL, which the log line and the alert name
 		// already: only what went wrong is kept.
@@ -526,22 +439,16 @@ func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
 	return lines, nil
 }
 
-// parse reads the lines of the list at listURL, fetched at fetched: those
-// of a blocklist or a tag list, of the file of AS:percent lines, or of the
-// CrowdSec decision list. In all but the decision list, anything after a ; or
-// a # on a line is left out, and so is a line left blank. Any other line that
+// parse reads the lines of the list at listURL: those of a blocklist or a
+// tag list, or of the file of AS:percent lines. Anything after a ; or a #
+// on a line is left out, and so is a line left blank. Any other line that
 // does not read is an error naming it by its number.
-func (l *Lists) parse(
-	listURL string, lines []string, fetched time.Time,
-) (entries, error) {
-	switch listURL {
-	case l.params.ASNLimitPercentURL:
+func (l *Lists) parse(listURL string, lines []string) (entries, error) {
+	if listURL == l.params.ASNLimitPercentURL {
 		return parsePercents(lines)
-	case l.params.CrowdSecDecisionsURL:
-		return parseDecisions(lines, fetched)
-	default:
-		return parseNetblocks(lines)
 	}
+
+	return parseNetblocks(lines)
 }
 
 // parseNetblocks reads the lines of a blocklist or a tag list, each an address
@@ -628,64 +535,6 @@ func parsePercents(lines []string) (entries, error) {
 	return found, nil
 }
 
-// parseDecisions reads the lines of the CrowdSec decision list fetched at
-// fetched: the engine's answer, a JSON list of its decisions in force,
-// null while it has none. A decision of the type ban whose scope is Ip or
-// Range, as CrowdSec names them, bans its value, an address or a netblock
-// as parseNetblock reads it, until its duration, the time it had left as
-// the engine answered, has passed since fetched. Any other decision, such
-// as one to show a captcha or one on a country, is left out. A decision
-// to ban whose value or duration does not read is an error naming it by
-// its number.
-func parseDecisions(lines []string, fetched time.Time) (entries, error) {
-	var answer []struct {
-		Duration string `json:"duration"`
-		Scenario string `json:"scenario"`
-		Scope    string `json:"scope"`
-		Type     string `json:"type"`
-		Value    string `json:"value"`
-	}
-
-	err := json.Unmarshal([]byte(strings.Join(lines, "\n")), &answer)
-	if err != nil {
-		return entries{}, fmt.Errorf("read the answer: %w", err)
-	}
-
-	found := entries{decisions: map[netip.Prefix]Decision{}}
-
-	for i, decision := range answer {
-		if decision.Type != "ban" ||
-			(decision.Scope != "Ip" && decision.Scope != "Range") {
-			continue
-		}
-
-		netblock, ok := parseNetblock(decision.Value)
-
-		duration, err := time.ParseDuration(decision.Duration)
-		if !ok || err != nil {
-			return entries{}, fmt.Errorf(
-				"decision %d %w",
-				i+1,
-				errNotDecision,
-			)
-		}
-
-		expires := fetched.Add(duration)
-		if expires.After(found.decisions[netblock].Expires) {
-			found.decisions[netblock] = Decision{
-				Expires:  expires,
-				Scenario: decision.Scenario,
-			}
-		}
-
-		if !slices.Contains(found.lengths, netblock.Bits()) {
-			found.lengths = append(found.lengths, netblock.Bits())
-		}
-	}
-
-	return found, nil
-}
-
 // withoutComment returns line without anything after a ; or a #, and
 // without the spaces around what is left.
 func withoutComment(line string) string {

internal/requestlog/requestlog.go

@@ -21,8 +21,8 @@ const (
 	// the app.
 	ActionForward = "forward"
 
-	// ActionRequestLineTooLong is a request whose request line is over the
-	// header limits' longest line.
+	// ActionRequestLineTooLong is a request whose request line is too long for
+	// the header limits.
 	ActionRequestLineTooLong = "request_line_too_long"
 
 	// ActionRequestHeaderTooLarge is a request whose headers are over the
@@ -34,9 +34,8 @@ const (
 	ActionClientIsRateLimited = "client_is_rate_limited"
 
 	// ActionTemporarilyBanned is a request refused because a ban that runs
-	// out covers its client, or because it matched a ban rule, asked for a
-	// trap path or the CrowdSec decision list lists its client, and so made
-	// such a ban.
+	// out covers its client, or because it matched a ban rule or asked for a
+	// trap path, and so made such a ban.
 	ActionTemporarilyBanned = "temporarily_banned"
 
 	// ActionPermanentlyBanned is a request refused because a ban that never
@@ -189,8 +188,7 @@ type Line struct {
 	// LimitHitMinuteRefusals for the refusal limit.
 	LimitHit string `json:"limit_hit,omitempty"`
 	// Reputation are the URLs of the blocklists that list the client, then
-	// that of the CrowdSec decision list when it does, then the DNSBL zones
-	// whose verdict lists it, their keys masked, then abuseipdb when its
+	// the DNSBL zones whose verdict lists it, their keys masked, then abuseipdb when its
 	// score is a hit.
 	Reputation []string `json:"reputation,omitempty"`
 	// Tags are the request's tags, sorted.

internal/rproxy/breaker.go

@@ -0,0 +1,82 @@
+package rproxy
+
+import (
+	"errors"
+	"sync"
+	"time"
+)
+
+// What opens the circuit breaker, and how often a request asks waf while it is
+// open.
+const (
+	breakerFailures    = 5
+	breakerWindow      = 5 * time.Second
+	breakerAskInterval = time.Second
+)
+
+// errBreakerOpen is why a request did not ask waf.
+var errBreakerOpen = errors.New(
+	"waf is not asked while the circuit breaker is open",
+)
+
+// breaker is the circuit breaker for asks to waf. It opens when
+// breakerFailures asks fail within breakerWindow. An ask fails when it gets no
+// answer in time, or an answer rproxy cannot pass on. While it is open, at
+// most one request each breakerAskInterval asks waf. Any usable answer closes
+// it, whatever it decides, and forgets the failures before it.
+type breaker struct {
+	mu sync.Mutex
+	// failures holds the times of the latest failed asks, oldest first, at most
+	// breakerFailures of them.
+	failures []time.Time
+	open     bool
+	// lastAsk is when a request last asked waf while the breaker was open.
+	lastAsk time.Time
+}
+
+// mayAsk reports whether a request may ask waf at now. While the breaker is
+// open, it says yes once each breakerAskInterval.
+func (b *breaker) mayAsk(now time.Time) bool {
+	b.mu.Lock()
+	defer b.mu.Unlock()
+
+	if !b.open {
+		return true
+	}
+
+	if now.Sub(b.lastAsk) < breakerAskInterval {
+		return false
+	}
+
+	b.lastAsk = now
+
+	return true
+}
+
+// noteAnswer notes that an ask got an answer rproxy can use. It closes the
+// breaker and forgets the failures before it.
+func (b *breaker) noteAnswer() {
+	b.mu.Lock()
+	defer b.mu.Unlock()
+
+	b.open = false
+	b.failures = nil
+}
+
+// noteFailure notes that an ask that ended at now got no answer in time, and
+// opens the breaker if breakerFailures asks have failed within breakerWindow.
+func (b *breaker) noteFailure(now time.Time) {
+	b.mu.Lock()
+	defer b.mu.Unlock()
+
+	b.failures = append(b.failures, now)
+	if len(b.failures) > breakerFailures {
+		b.failures = b.failures[1:]
+	}
+
+	if !b.open && len(b.failures) == breakerFailures &&
+		now.Sub(b.failures[0]) <= breakerWindow {
+		b.open = true
+		b.lastAsk = now
+	}
+}

internal/rproxy/breaker_test.go

@@ -0,0 +1,142 @@
+package rproxy_test
+
+import (
+	"testing"
+	"time"
+
+	"sneak.berlin/go/smallwebwaf/internal/rproxy"
+)
+
+// The breaker tests give the breaker each time themselves, counted from
+// time.Now(), and never wait.
+
+// fail notes count failed asks on b, one each step from at.
+func fail(b *rproxy.Breaker, at time.Time, count int, step time.Duration) {
+	for i := range count {
+		b.NoteFailure(at.Add(time.Duration(i) * step))
+	}
+}
+
+// openBreaker returns a breaker opened at now by five failed asks, one a
+// second.
+func openBreaker(now time.Time) *rproxy.Breaker {
+	b := &rproxy.Breaker{}
+	fail(b, now.Add(-4*time.Second), 5, time.Second)
+
+	return b
+}
+
+func TestFourFailedAsksWithinFiveSecondsLeaveTheBreakerClosed(t *testing.T) {
+	t.Parallel()
+
+	now := time.Now()
+	b := &rproxy.Breaker{}
+	fail(b, now, 4, time.Second)
+
+	if !b.MayAsk(now.Add(3 * time.Second)) {
+		t.Error("a request may not ask, want it to")
+	}
+}
+
+func TestFiveFailedAsksWithinFiveSecondsOpenTheBreaker(t *testing.T) {
+	t.Parallel()
+
+	now := time.Now()
+	b := openBreaker(now)
+
+	if b.MayAsk(now) {
+		t.Error("a request may ask, want it not to")
+	}
+}
+
+func TestFiveFailedAsksOverMoreThanFiveSecondsLeaveTheBreakerClosed(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	now := time.Now()
+	b := &rproxy.Breaker{}
+	fail(b, now, 5, 1300*time.Millisecond)
+
+	if !b.MayAsk(now.Add(6 * time.Second)) {
+		t.Error("a request may not ask, want it to")
+	}
+}
+
+func TestAnAnswerBetweenFailedAsksKeepsTheBreakerClosed(t *testing.T) {
+	t.Parallel()
+
+	now := time.Now()
+	b := &rproxy.Breaker{}
+	fail(b, now, 4, 100*time.Millisecond)
+	b.NoteAnswer()
+	fail(b, now.Add(500*time.Millisecond), 4, 100*time.Millisecond)
+
+	if !b.MayAsk(now.Add(900 * time.Millisecond)) {
+		t.Error("a request may not ask, want it to")
+	}
+}
+
+func TestAnOpenBreakerLetsOneRequestAskEachSecond(t *testing.T) {
+	t.Parallel()
+
+	now := time.Now()
+	b := openBreaker(now)
+
+	for _, tc := range []struct {
+		after time.Duration
+		want  bool
+	}{
+		{999 * time.Millisecond, false},
+		{time.Second, true},
+		{time.Second, false},
+		{1500 * time.Millisecond, false},
+		{2 * time.Second, true},
+	} {
+		got := b.MayAsk(now.Add(tc.after))
+		if got != tc.want {
+			t.Errorf("may ask %v after it opened: %v, want %v",
+				tc.after, got, tc.want)
+		}
+	}
+}
+
+func TestAFailedAskKeepsTheBreakerOpen(t *testing.T) {
+	t.Parallel()
+
+	opened := time.Now()
+	b := openBreaker(opened)
+	now := opened.Add(time.Second)
+	b.MayAsk(now)
+	b.NoteFailure(now)
+
+	if b.MayAsk(now.Add(999 * time.Millisecond)) {
+		t.Error("a request may ask, want it not to")
+	}
+
+	if !b.MayAsk(now.Add(time.Second)) {
+		t.Error("a request may not ask a second later, want it to")
+	}
+}
+
+func TestOneAnswerClosesTheBreaker(t *testing.T) {
+	t.Parallel()
+
+	opened := time.Now()
+	b := openBreaker(opened)
+	now := opened.Add(time.Second)
+	b.MayAsk(now)
+	b.NoteAnswer()
+
+	if !b.MayAsk(now) {
+		t.Error("a request may not ask, want it to")
+	}
+
+	// The failures before the answer no longer count, so one more does not open
+	// the breaker again.
+	b.NoteFailure(now)
+
+	if !b.MayAsk(now) {
+		t.Error("a request may not ask after one failure, want it to")
+	}
+}

internal/rproxy/export_test.go

@@ -0,0 +1,23 @@
+package rproxy
+
+import "time"
+
+// Breaker is the circuit breaker, for tests that give it the time.
+type Breaker struct {
+	b breaker
+}
+
+// MayAsk reports whether a request may ask waf at now.
+func (b *Breaker) MayAsk(now time.Time) bool {
+	return b.b.mayAsk(now)
+}
+
+// NoteAnswer notes that an ask got an answer rproxy can use.
+func (b *Breaker) NoteAnswer() {
+	b.b.noteAnswer()
+}
+
+// NoteFailure notes that an ask that ended at now got no answer in time.
+func (b *Breaker) NoteFailure(now time.Time) {
+	b.b.noteFailure(now)
+}

internal/rproxy/rproxy.go

@@ -4,10 +4,11 @@
 // waf goes to the visitor as waf gave it.
 // SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION decides only when waf cannot
 // be connected to, does not answer at all, or does not answer within
-// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. rproxy itself refuses a request with a
-// method it does not pass on, or one it cannot put to waf as the visitor
-// sent it. It keeps nothing between requests and writes nothing to disk. "rproxy" in
-// SPEC.md says what it does.
+// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, or while the circuit breaker for asks to
+// waf is open. rproxy itself refuses a request with a method it does not pass
+// on, or one it cannot put to waf as the visitor sent it. It keeps nothing
+// between requests but the circuit breaker, and writes nothing to disk.
+// "rproxy" in SPEC.md says what it does.
 package rproxy
 
 import (
@@ -132,6 +133,8 @@ type handler struct {
 	wafTransport http.RoundTripper
 	// reportURL is where the reports go.
 	reportURL string
+	// breaker is the circuit breaker for asks to waf.
+	breaker breaker
 	// reports waits for the reports being sent. reportsMu guards starting one
 	// against reportsStopped, which is set once nothing is to wait for another.
 	reports        sync.WaitGroup
@@ -196,7 +199,7 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	answer, err := rq.ask(r.Context())
+	answer, err := rq.askUnlessBreakerOpen(r.Context())
 	if err != nil {
 		rq.handleNoUsableAnswer(r.Context(), err)
 

internal/rproxy/rproxy_test.go

@@ -15,6 +15,7 @@ import (
 	"strconv"
 	"strings"
 	"sync"
+	"sync/atomic"
 	"testing"
 	"time"
 
@@ -1395,6 +1396,139 @@ func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
 	standIn.wantNoReport(t)
 }
 
+func TestFiveFailedAsksKeepTheNextRequestFromAskingWAF(t *testing.T) {
+	t.Parallel()
+
+	for name, failure := range map[string]http.HandlerFunc{
+		"no answer at all": func(http.ResponseWriter, *http.Request) {
+			panic(http.ErrAbortHandler)
+		},
+		"a slow answer": func(_ http.ResponseWriter, r *http.Request) {
+			<-r.Context().Done()
+		},
+	} {
+		t.Run(name, func(t *testing.T) {
+			t.Parallel()
+
+			var answering atomic.Bool
+
+			standIn := startWAF(
+				t,
+				func(w http.ResponseWriter, r *http.Request) {
+					if answering.Load() {
+						allow(w, r)
+
+						return
+					}
+
+					failure(w, r)
+				},
+			)
+			app := startApp(t, greet)
+			addr, out := startRProxy(
+				t,
+				standIn.URL,
+				app.URL,
+				map[string]string{
+					wafBackendTimeout: "200",
+				},
+			)
+
+			for range 5 {
+				wantGreeting(t, get(t, addr, "/"))
+				standIn.question(t)
+			}
+
+			answering.Store(true)
+			wantGreeting(t, get(t, addr, "/"))
+			out.waitFor(t, "the circuit breaker is open")
+
+			select {
+			case <-standIn.questions:
+				t.Error("waf was asked, want it not to be")
+			case <-time.After(10 * pollInterval):
+			}
+		})
+	}
+}
+
+func TestVisitorsThatGoAwayWhileWAFIsAskedLeaveTheBreakerClosed(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	var answering atomic.Bool
+
+	standIn := startWAF(t, func(w http.ResponseWriter, r *http.Request) {
+		if answering.Load() {
+			allow(w, r)
+
+			return
+		}
+
+		<-r.Context().Done()
+	})
+	app := startApp(t, greet)
+	addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
+		wafBackendTimeout: "60000",
+	})
+
+	// Five of these would open the breaker if they counted as failures.
+	for range 5 {
+		conn := dial(t, addr)
+		send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
+		id := standIn.question(t).header.Get(requestIDHeader)
+
+		_ = conn.Close()
+
+		// rproxy writes the request's log line once it is done with the ask.
+		out.waitFor(t, id)
+	}
+
+	answering.Store(true)
+	wantGreeting(t, get(t, addr, "/"))
+	standIn.question(t)
+}
+
+func TestAnswersThatCannotBePassedOnOpenTheBreaker(t *testing.T) {
+	t.Parallel()
+
+	var answering atomic.Bool
+
+	tooLarge := overOneMiB(dispositionDeny)
+	standIn := startWAF(t, func(w http.ResponseWriter, r *http.Request) {
+		if answering.Load() {
+			allow(w, r)
+
+			return
+		}
+
+		tooLarge(w, r)
+	})
+	app := startApp(t, greet)
+	addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
+		wafBackendTimeout: "60000",
+	})
+
+	for range 5 {
+		if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
+			t.Errorf("answered %d, want 502", got.status)
+		}
+
+		standIn.question(t)
+	}
+
+	answering.Store(true)
+	wantGreeting(t, get(t, addr, "/"))
+	out.waitFor(t, "the circuit breaker is open")
+
+	select {
+	case <-standIn.questions:
+		t.Error("waf was asked, want it not to be")
+	case <-time.After(10 * pollInterval):
+	}
+}
+
 func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) {
 	t.Parallel()
 

internal/rproxy/waf.go

@@ -10,6 +10,7 @@ import (
 	"maps"
 	"net/http"
 	"strings"
+	"time"
 
 	"sneak.berlin/go/smallwebwaf/internal/waf"
 )
@@ -67,7 +68,8 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
 		rq.answer(http.StatusBadGateway)
 	default:
 		// waf cannot be connected to, does not answer at all, or does not answer
-		// within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. A request that keeps waf
+		// within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS; or the circuit breaker
+		// kept the request from asking. A request that keeps waf
 		// busy past SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, like any request while
 		// waf is overloaded, gets SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION
 		// here. deny closes this, at the cost of answering 503 whenever waf is
@@ -86,6 +88,31 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
 	}
 }
 
+// askUnlessBreakerOpen asks waf, unless the circuit breaker is open and it is
+// not yet time for a request to ask. It then tells the breaker whether waf
+// answered in time. An answer rproxy cannot pass on counts as a failure. A
+// visitor that goes away tells the breaker nothing. ctx is the request's own
+// context.
+func (rq *request) askUnlessBreakerOpen(
+	ctx context.Context,
+) (*wafAnswer, error) {
+	b := &rq.h.breaker
+	if !b.mayAsk(time.Now()) {
+		return nil, errBreakerOpen
+	}
+
+	answer, err := rq.ask(ctx)
+
+	switch {
+	case err == nil:
+		b.noteAnswer()
+	case ctx.Err() == nil:
+		b.noteFailure(time.Now())
+	}
+
+	return answer, err
+}
+
 // ask sends waf the question, and returns its answer, which must have come
 // whole within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. It ends when ctx does, as
 // when the visitor goes away.

internal/state/state.go

@@ -104,7 +104,7 @@ var (
 	errVersion = errors.New("unknown version")
 	// errMissing is for an entry without a field it needs.
 	errMissing     = errors.New("has no")
-	errCause       = errors.New("is not limit, attack, admin or crowdsec")
+	errCause       = errors.New("is not limit, attack or admin")
 	errExpires     = errors.New("is not after its start")
 	errDestination = errors.New("is not webhook, slack or ntfy")
 	errScope       = errors.New(
@@ -967,7 +967,7 @@ func (e BanEntry) ban() bans.Ban {
 // each expires is read again as written. An expires not after the start
 // is refused too, since the netblock's next ban for a broken limit, three
 // times as long as this one, would then refuse nothing; and so is a cause
-// other than limit, attack, admin or crowdsec, most likely misspelt.
+// other than limit, attack or admin, most likely misspelt.
 func (f *bansFile) check(data []byte) error {
 	var written struct {
 		Bans []struct {
@@ -991,8 +991,7 @@ func (f *bansFile) check(data []byte) error {
 		case entry.Expires != nil && !entry.Expires.After(entry.Start):
 			return fmt.Errorf("entry %d's expires %w", i+1, errExpires)
 		case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
-			entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
-			entry.Cause != bans.CauseCrowdSec:
+			entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin:
 			return fmt.Errorf(
 				"entry %d's cause %q %w",
 				i+1,

internal/state/state_test.go

@@ -114,8 +114,7 @@ const permanentBansJSON = `{
                 "earlier_bans": {
                     "limit": 3,
                     "attack": 1,
-                    "admin": 1,
-                    "crowdsec": 2
+                    "admin": 1
                 }
             }
         }
@@ -1188,10 +1187,10 @@ func TestBanWithAnotherCauseStopsTheStart(t *testing.T) {
 		`{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+
 		`"expires": null, "cause": "admin"}, `+
 		`{"netblock": "203.0.113.11/32", "start": "2026-10-06T00:00:00Z", `+
-		`"expires": "2026-10-06T04:00:00Z", "cause": "crowdsec"}, `+
+		`"expires": "2026-10-06T04:00:00Z", "cause": "limit"}, `+
 		`{"netblock": "203.0.113.12/32", "start": "2026-10-06T00:00:00Z", `+
 		`"expires": null, "cause": "atack"}]}`,
-		`: entry 4's cause "atack" is not limit, attack, admin or crowdsec`)
+		`: entry 4's cause "atack" is not limit, attack or admin`)
 }
 
 func TestBanThatDoesNotEndAfterItsStartStopsTheStart(t *testing.T) {
@@ -2466,9 +2465,8 @@ func office() netip.Prefix {
 	return netip.MustParsePrefix("203.0.113.0/24")
 }
 
-// fill puts a permanent ban an admin made, a ban for a broken limit, one
-// for a clear sign of attack and one for CrowdSec's decision, clients
-// with counts and histories,
+// fill puts a permanent ban an admin made, a ban for a broken limit and one
+// for a clear sign of attack, clients with counts and histories,
 // GeoJS answers, the blocklists' last tries and the copy of one, two
 // verdicts of a DNSBL zone, the AbuseIPDB checks spent today with two
 // scores, and two answers of routewatch, as filledListsJSON,
@@ -2485,13 +2483,6 @@ func fill(params state.Params) {
 	})
 	params.Ledger.BanForAttack(netip.MustParsePrefix("192.0.2.1/32"), now,
 		bans.Notes{RuleID: "env-file", Target: "path"})
-	params.Ledger.BanForCrowdSec(
-		netip.MustParsePrefix("198.51.100.9/32"),
-		now,
-		now.Add(4*time.Hour),
-		"crowdsecurity/ssh-bf",
-		bans.Notes{},
-	)
 
 	for _, c := range []string{"2001:db8::/64", "203.0.113.9/32", "192.0.2.1/32"} {
 		params.Limiter.Count(netip.MustParsePrefix(c), now, whole)
@@ -2634,10 +2625,9 @@ func permanentBan() bans.Ban {
 			Requests: 1500,
 			Refused:  3,
 			EarlierBans: bans.EarlierBans{
-				Limit:    3,
-				Attack:   1,
-				Admin:    1,
-				CrowdSec: 2,
+				Limit:  3,
+				Attack: 1,
+				Admin:  1,
 			},
 		},
 	}

internal/sww/rproxy_test.go

@@ -260,7 +260,17 @@ func TestTheSiteAnswersWhileWAFIsStoppedAndStartedAgain(t *testing.T) {
 		stop()
 	}()
 
-	wantStatus(t, site+"/", placed, http.StatusForbidden)
+	// The visitor's asks have opened rproxy's circuit breaker, which lets one
+	// request a second ask until one is answered.
+	deadline := time.Now().Add(waitLimit)
+	for statusFrom(t, site+"/", placed) != http.StatusForbidden {
+		if time.Now().After(deadline) {
+			t.Fatalf("request from %s: never answered %d", placed,
+				http.StatusForbidden)
+		}
+
+		time.Sleep(pollInterval)
+	}
 
 	close(visitorStop)
 	<-visitorDone

internal/sww/sww_test.go

@@ -847,92 +847,6 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
 	}
 }
 
-func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
-	t.Parallel()
-
-	const bouncerKey = "crowdsec-key-0123456789abcdef"
-
-	// A stand-in for the engine's local API, which bans 203.0.113.0/24 for
-	// four hours, and answers only a request with its key, while it is up.
-	down := new(atomic.Bool)
-	engine := httptest.NewServer(http.HandlerFunc(
-		func(w http.ResponseWriter, r *http.Request) {
-			switch {
-			case r.URL.Path != "/v1/decisions" || r.Header.Get("X-Api-Key") != bouncerKey:
-				w.WriteHeader(http.StatusForbidden)
-			case down.Load():
-				w.WriteHeader(http.StatusServiceUnavailable)
-			default:
-				_, _ = io.WriteString(
-					w,
-					`[{"duration": "4h0m0s", "origin": "crowdsec", `+
-						`"scenario": "crowdsecurity/http-probing", "scope": "Range", `+
-						`"type": "ban", "value": "203.0.113.0/24"}]`,
-				)
-			}
-		}))
-	t.Cleanup(engine.Close)
-
-	dir, cache := t.TempDir(), t.TempDir()
-	env := map[string]string{
-		listenAddr:                  localhost + ":0",
-		stateDir:                    dir,
-		cacheDir:                    cache,
-		ruleFilesDir:                t.TempDir(),
-		"SWW_WAF_CROWDSEC_LAPI_URL": engine.URL,
-		"SWW_WAF_CROWDSEC_LAPI_KEY": bouncerKey,
-	}
-
-	// Once the list is fetched, the client's request bans it.
-	first := runUntilStopped(t, env, func(url string) {
-		for statusFrom(t, url, placed) != http.StatusForbidden {
-			time.Sleep(pollInterval)
-		}
-	})
-
-	ban := onlyBan(t, dir)
-	if ban["netblock"] != placed+"/32" || ban["cause"] != "crowdsec" ||
-		ban["reason"] != "CrowdSec's decision for crowdsecurity/http-probing" {
-		t.Errorf(
-			"bans.json holds %v, want the ban for crowdsec on %s",
-			ban,
-			placed,
-		)
-	}
-
-	// Restarted with the engine down, the copy kept in lists.json bans
-	// another client in the netblock from its first request.
-	down.Store(true)
-
-	second := runUntilStopped(t, env, func(url string) {
-		wantStatus(t, url, "203.0.113.10", http.StatusForbidden)
-	})
-
-	// The key is in neither run's output, nor in a state or cache file.
-	files := []string{
-		filepath.Join(dir, "bans.json"), filepath.Join(dir, "clients.json"),
-		filepath.Join(cache, "lists.json"),
-	}
-	shown := make([]string, 0, len(files)+2)
-	shown = append(shown, first.text(), second.text())
-
-	for _, path := range files {
-		data, err := os.ReadFile(path) //nolint:gosec // the test's
-		if err != nil {
-			t.Fatalf("read %s: %v", path, err)
-		}
-
-		shown = append(shown, string(data))
-	}
-
-	if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) {
-		t.Errorf(
-			"the key is shown in the output or the state and cache files:\n%s",
-			all,
-		)
-	}
-}
-
 // wantDeniedByList checks that the request log line is of a request the
 // blocklist at listURL refused.
 func wantDeniedByList(t *testing.T, line map[string]any, listURL string) {

internal/waf/anomaly_test.go

@@ -349,6 +349,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
 		map[string]string{
 			trustedProxies:                 trustLocalhost,
 			asnAndCountryLookupTimeout:     "3600000",
+			addLookupHeaders:               falseValue,
 			anomalyASN + requestsPerMinute: aLot,
 		})
 	server.Store(started)

internal/waf/answertime_test.go

@@ -49,9 +49,6 @@ const (
 	silentAbuseIPDBURL = "silent://abuseipdb/api/v2/check"
 	// wholeLimit is the percentage of a rate limit that lowers it not at all.
 	wholeLimit = 100
-	// clientsDecision is the CrowdSec decision list holding client alone.
-	clientsDecision = `[{"duration": "4h0m0s", "scenario": "crowdsecurity/ssh-bf", ` +
-		`"scope": "Ip", "type": "ban", "value": "` + client + `"}]`
 )
 
 // cookies returns a Cookie header's value with count cookies.
@@ -134,12 +131,22 @@ func TestARequestLineOverTheLimitIsRefusedWith414BeforeAnyOtherCheck(
 	for _, tc := range []struct {
 		name      string
 		lineBytes int
-		status    int
-		action    string
+		// cookies is how many cookies the request carries.
+		cookies int
+		status  int
+		action  string
 	}{
-		{"at the limit", lineLimit, http.StatusOK, requestlog.ActionForward},
 		{
-			"over the limit", lineLimit + 1, http.StatusRequestURITooLong,
+			"at the limit", lineLimit, 0, http.StatusOK,
+			requestlog.ActionForward,
+		},
+		{
+			"over the limit", lineLimit + 1, 0, http.StatusRequestURITooLong,
+			requestlog.ActionRequestLineTooLong,
+		},
+		{
+			"over the limit and the cookie limit", lineLimit + 1,
+			cookieLimit + 1, http.StatusRequestURITooLong,
 			requestlog.ActionRequestLineTooLong,
 		},
 	} {
@@ -147,7 +154,13 @@ func TestARequestLineOverTheLimitIsRefusedWith414BeforeAnyOtherCheck(
 			t.Parallel()
 
 			s := startWAF(t, "", map[string]string{coreRuleSetMode: block})
-			got := do(t, s.question(siteA, client, longQuery(tc.lineBytes)))
+			req := s.question(siteA, client, longQuery(tc.lineBytes))
+
+			if tc.cookies > 0 {
+				req.Header.Set(cookieHeader, cookies(tc.cookies))
+			}
+
+			got := do(t, req)
 			line := s.out.requestLine(t)
 
 			if tc.status == http.StatusOK {
@@ -191,6 +204,10 @@ func TestAHeaderLineOverTheLimitIsRefusedWith431(t *testing.T) {
 		{"a Host line over the limit", func(req *http.Request) {
 			req.Host = strings.Repeat("a", lineLimit+1-len("Host: "))
 		}, true},
+		{"a header sent twice, each line within the limit", func(req *http.Request) {
+			req.Header.Add("X-Long", strings.Repeat("a", 8000))
+			req.Header.Add("X-Long", strings.Repeat("a", 8000))
+		}, false},
 	} {
 		t.Run(tc.name, func(t *testing.T) {
 			t.Parallel()
@@ -405,10 +422,6 @@ func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
 			"/", leaveAsItIs,
 		},
 		{"a blocklist", map[string]string{blocklistURLs: dropURL}, "/", listClient},
-		{
-			"the CrowdSec decision list",
-			map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey}, "/", decideOnClient,
-		},
 		{
 			"a DNS blocklist", map[string]string{
 				dnsblZones: dnsblZone, dnsblResolver: noResolver, dnsblAbuseIPDBAction: actionDeny,
@@ -701,12 +714,6 @@ func listClient(t *testing.T, server *waf.Server, now time.Time) {
 	}
 }
 
-// decideOnClient puts client on the CrowdSec decision list.
-func decideOnClient(t *testing.T, server *waf.Server, now time.Time) {
-	t.Helper()
-	loadDecisions(t, server, now, clientsDecision)
-}
-
 // leaveAsItIs prepares nothing.
 func leaveAsItIs(*testing.T, *waf.Server, time.Time) {}
 
@@ -750,12 +757,6 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
 			name: "a blocklist", env: map[string]string{blocklistURLs: dropURL},
 			prepare: listClient, status: forbidden, action: requestlog.ActionClientDenied,
 		},
-		{
-			name:    "the CrowdSec decision list",
-			env:     map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey},
-			prepare: decideOnClient, status: forbidden,
-			action: requestlog.ActionTemporarilyBanned,
-		},
 		{
 			name: "the header limits", cookie: cookies(cookieLimit + 1),
 			prepare: leaveAsItIs, status: tooLarge,
@@ -883,7 +884,6 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
 		dnsblZones: dnsblZone, dnsblResolver: resolver,
 		abuseIPDBKey:  accountKey,
 		blocklistURLs: held + "/drop.txt", asnLimitPercentURL: held + "/asn.txt",
-		crowdSecURL: held, crowdSecKey: bouncerKey,
 	})
 
 	// The lists are fetched outside the synctest bubble below: a wait on the

internal/waf/bans.go

@@ -11,7 +11,6 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/alerts"
 	"sneak.berlin/go/smallwebwaf/internal/bans"
 	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
-	"sneak.berlin/go/smallwebwaf/internal/reputation"
 	"sneak.berlin/go/smallwebwaf/internal/requestlog"
 )
 
@@ -206,57 +205,6 @@ func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
 	return banAction(ban.Permanent())
 }
 
-// banForCrowdSec bans the client's netblock at now until decision,
-// CrowdSec's decision on the client, ends. In observe mode it makes no
-// ban, and raises the alert for the ban it would have made, if that alert
-// would be sent. It returns the action for that ban.
-func (rq *request) banForCrowdSec(
-	now time.Time,
-	decision reputation.Decision,
-) string {
-	netblock := rq.h.netblock(rq.client)
-	if rq.h.config.Observe &&
-		!rq.wouldAlertBan(netblock, now, bans.CauseCrowdSec) {
-		return banAction(
-			rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseCrowdSec),
-		)
-	}
-
-	notes := bans.Notes{
-		ASN:        rq.line.ASN,
-		ASName:     rq.line.ASName,
-		Country:    rq.line.Country,
-		Reputation: rq.reputation,
-		Request:    rq.noted(now, rq.h.config.BanResponse),
-		Requests:   rq.netblockRequests(netblock),
-	}
-
-	if rq.h.config.Observe {
-		ban, wouldBan := rq.h.ledger.WouldBanForCrowdSec(
-			netblock,
-			now,
-			decision.Expires,
-			decision.Scenario,
-			notes,
-		)
-		if wouldBan {
-			rq.alertBan(ban)
-		}
-
-		return banAction(ban.Permanent())
-	}
-
-	ban, made := rq.h.ledger.BanForCrowdSec(netblock, now, decision.Expires,
-		decision.Scenario, notes)
-	rq.line.BanExpires = banExpires(ban)
-
-	if made {
-		rq.alertBan(ban)
-	}
-
-	return banAction(ban.Permanent())
-}
-
 // wouldAlertBan reports whether the alert for a ban on netblock for cause
 // made at now would be sent. In observe mode the ban the request would
 // have made is worked out only then, at most once per

internal/waf/biased_test.go

@@ -387,14 +387,16 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
 
 			// In a synctest bubble, as
 			// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
-			// with a GeoJS that never answers.
+			// with a GeoJS that never answers. The headers, which need the
+			// answer too, are off.
 			synctest.Test(t, func(t *testing.T) {
 				server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
 					map[string]string{
 						asnAndCountryLookupTimeout: strconv.FormatInt(
 							timeout.Milliseconds(), 10),
-						tagLists:   "vpn=" + vpnList,
-						tc.setting: tc.value,
+						addLookupHeaders: falseValue,
+						tagLists:         "vpn=" + vpnList,
+						tc.setting:       tc.value,
 					})
 				began := time.Now()
 

internal/waf/countries_test.go

@@ -108,8 +108,11 @@ func TestCountryRefusalNamesItsHeuristicCountryAndSetting(t *testing.T) {
 	t.Parallel()
 
 	geojsURL, _ := startGeoJS(t)
+	// The headers are off, so that only the country list makes the request
+	// wait for its answer.
 	s, _, _ := startWithClock(t, geojsURL, map[string]string{
-		asnAndCountryLookupTimeout: "3600000", allowedCountries: "de",
+		asnAndCountryLookupTimeout: "3600000", addLookupHeaders: falseValue,
+		allowedCountries: "de",
 	})
 
 	line := s.get(
@@ -239,7 +242,10 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
 		name string
 		env  map[string]string
 	}{
-		{"no setting needs the lookup", nil},
+		{
+			"no setting needs the lookup",
+			map[string]string{addLookupHeaders: falseValue},
+		},
 		{"a tag setting needs it", map[string]string{tagDeny: denyKP}},
 	} {
 		t.Run(tc.name, func(t *testing.T) {

internal/waf/crowdsec.go

@@ -1,54 +0,0 @@
-package waf
-
-// The heuristic crowdsec. It reads SWW_WAF_CROWDSEC_LAPI_URL, and the decisions
-// through the reputation lists.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/bans"
-	"sneak.berlin/go/smallwebwaf/internal/config"
-	"sneak.berlin/go/smallwebwaf/internal/reputation"
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// crowdSec refuses a request when a decision of the CrowdSec decision list on
-// the client is in force at now. It then adds the tag crowdsec, notes the
-// list, as noteHit does, and bans the client until that decision ends. Its
-// figures are the decision's scenario and when it ends.
-type crowdSec struct{}
-
-func (crowdSec) name() string { return "crowdsec" }
-
-func (crowdSec) check(
-	_ context.Context,
-	rq *request,
-	now time.Time,
-) *verdict {
-	decision, listed := rq.crowdSecDecisionInForce(now)
-	if !listed {
-		return nil
-	}
-
-	rq.addTag(config.CrowdSecTag)
-	rq.noteHit(bans.ReputationHit{Source: rq.h.config.CrowdSecDecisionsURL},
-		"listed by the CrowdSec decision list")
-
-	return &verdict{
-		action: rq.banForCrowdSec(now, decision),
-		figures: map[string]any{
-			"scenario": decision.Scenario,
-			"expires":  requestlog.FormatTime(decision.Expires),
-		},
-	}
-}
-
-// crowdSecDecisionInForce returns the decision of the CrowdSec decision list on
-// the client that is in force at now, and reports whether there is one, which
-// the heuristic crowdsec refuses the request for.
-func (rq *request) crowdSecDecisionInForce(
-	now time.Time,
-) (reputation.Decision, bool) {
-	return rq.h.lists.CrowdSecDecision(rq.client, now)
-}

internal/waf/crowdsec_test.go

@@ -1,249 +0,0 @@
-package waf_test
-
-import (
-	"net/http"
-	"net/netip"
-	"reflect"
-	"testing"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/alerts"
-	"sneak.berlin/go/smallwebwaf/internal/bans"
-	"sneak.berlin/go/smallwebwaf/internal/reputation"
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-	"sneak.berlin/go/smallwebwaf/internal/waf"
-)
-
-// The CrowdSec settings, and the tests' engine, which is never asked: each
-// test puts in the copy of its decision list, at decisionsURL, that it
-// needs, as lists.json would at start.
-const (
-	crowdSecURL  = "SWW_WAF_CROWDSEC_LAPI_URL"
-	crowdSecKey  = "SWW_WAF_CROWDSEC_LAPI_KEY"
-	lapi         = "http://crowdsec.example:8080"
-	decisionsURL = lapi + "/v1/decisions"
-	bouncerKey   = "crowdsec-key-0123456789abcdef"
-)
-
-func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	s, clk, server, queue := startWithAlerts(t, map[string]string{
-		crowdSecURL: lapi, crowdSecKey: bouncerKey, metricsToken: token,
-	})
-	// client had four hours left on its decision as the engine answered.
-	fetched := clk.Now()
-	loadDecisions(t, server, fetched, `[{"duration": "4h0m0s", `+
-		`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
-		`"value": "`+client+`"}]`)
-	expires := requestlog.FormatTime(fetched.Add(4 * time.Hour))
-
-	// Its first request is refused, and bans it until the decision ends.
-	banned := requestlog.ActionTemporarilyBanned
-	line := s.get(client, http.StatusForbidden, banned)
-	wantReputation(t, line, decisionsURL)
-
-	if line.BanExpires != expires {
-		t.Errorf("log line has ban_expires %q, want %s",
-			line.BanExpires, expires)
-	}
-
-	listed := []bans.ReputationHit{{Source: decisionsURL}}
-
-	held := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
-	if len(held) != 1 || held[0].Cause != bans.CauseCrowdSec ||
-		!held[0].Start.Equal(fetched) ||
-		!held[0].Expires.Equal(fetched.Add(4*time.Hour)) ||
-		held[0].Reason != "CrowdSec's decision for crowdsecurity/ssh-bf" ||
-		!reflect.DeepEqual(held[0].Notes.Reputation, listed) ||
-		held[0].Notes.Request.Path != "/" || held[0].Notes.Requests != 1 {
-		t.Fatalf(
-			"bans %+v, want one for crowdsec of four hours, with the list and "+
-				"the request in its notes",
-			held,
-		)
-	}
-
-	// The listing raises a reputation_hit alert, and the ban its own.
-	waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
-	if len(waiting) != 2 || waiting[0].Event != alerts.EventReputationHit ||
-		waiting[0].Reason != "listed by the CrowdSec decision list" ||
-		!reflect.DeepEqual(
-			waiting[1],
-			banAlert(alerts.EventBan, fetched, client, held[0],
-				expires),
-		) {
-		t.Errorf(
-			"alerts waiting %+v, want a reputation_hit alert, then the ban's",
-			waiting,
-		)
-	}
-
-	// Each request while the ban lasts is refused under it, as under any
-	// ban, and once it has ended the client is let through.
-	clk.advance(4*time.Hour - time.Second)
-
-	line = s.get(client, http.StatusForbidden, banned)
-	wantReputation(t, line)
-
-	if line.BanExpires != expires {
-		t.Errorf("log line has ban_expires %q, want %s",
-			line.BanExpires, expires)
-	}
-
-	clk.advance(time.Second)
-	s.get(client, http.StatusOK, requestlog.ActionForward)
-
-	// The ban and the hit are counted, and the list has the metrics of any
-	// list fetched from a URL.
-	metrics := s.scrape()
-	labels := `{instance="` + alertInstance + `",source="` + decisionsURL + `"}`
-
-	wantMetric(t, metrics, `sww_bans_made_total{cause="crowdsec",`+
-		`instance="`+alertInstance+`"}`, 1)
-	wantMetric(t, metrics, "sww_reputation_hits_total"+labels, 1)
-	wantMetric(t, metrics, "sww_reputation_failures_total"+labels, 0)
-	wantMetric(
-		t,
-		metrics,
-		"sww_reputation_last_fetch_timestamp_seconds"+labels,
-		float64(fetched.Unix()),
-	)
-}
-
-func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
-	t.Parallel()
-
-	s, clk, server, _ := startWithAlerts(t, map[string]string{
-		crowdSecURL: lapi, crowdSecKey: bouncerKey,
-	})
-	loadDecisions(t, server, clk.Now(), `[{"duration": "4h0m0s", `+
-		`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
-		`"value": "`+client+`"}]`)
-
-	line := s.get(
-		client,
-		http.StatusForbidden,
-		requestlog.ActionTemporarilyBanned,
-	)
-	wantHeuristic(
-		t,
-		line,
-		"crowdsec",
-		`{"scenario": "crowdsecurity/ssh-bf", `+
-			`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`,
-	)
-	wantTags(t, line, "crowdsec")
-}
-
-func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	s, clk, server := startWithClock(t, "", map[string]string{
-		crowdSecURL: lapi, crowdSecKey: bouncerKey,
-	})
-	// 198.51.100.0/24 and 2001:db8::9 had a minute left as the engine
-	// answered.
-	fetched := clk.Now()
-	loadDecisions(t, server, fetched, `[{"duration": "1m0s", `+
-		`"scenario": "crowdsecurity/http-probing", "scope": "Range", "type": "ban", `+
-		`"value": "198.51.100.0/24"}, {"duration": "1m0s", `+
-		`"scenario": "crowdsecurity/http-probing", "scope": "Ip", "type": "ban", `+
-		`"value": "2001:db8::9"}]`)
-
-	// Just before its end, the decision bans a client in the netblock, and
-	// one on an IPv6 address bans the address's group, the /64.
-	clk.advance(time.Minute - time.Nanosecond)
-	s.get(
-		"198.51.100.7",
-		http.StatusForbidden,
-		requestlog.ActionTemporarilyBanned,
-	)
-	s.get(
-		"2001:db8::9",
-		http.StatusForbidden,
-		requestlog.ActionTemporarilyBanned,
-	)
-	s.get(
-		"2001:db8::5",
-		http.StatusForbidden,
-		requestlog.ActionTemporarilyBanned,
-	)
-
-	// Once it has ended, it bans no other client, and the bans it made end
-	// with it.
-	clk.advance(time.Nanosecond)
-
-	for _, from := range []string{"198.51.100.8", "198.51.100.7", "2001:db8::5"} {
-		wantReputation(
-			t,
-			s.get(from, http.StatusOK, requestlog.ActionForward),
-		)
-	}
-
-	if made := server.Ledger.Made(bans.CauseCrowdSec); made != 2 {
-		t.Errorf(
-			"%d bans made for crowdsec, want 2, on 198.51.100.7/32 and "+
-				"2001:db8::/64",
-			made,
-		)
-	}
-
-	if held := server.Ledger.Bans(netip.MustParsePrefix("2001:db8::/64")); len(
-		held,
-	) != 1 {
-		t.Errorf("bans of 2001:db8::/64 %+v, want one", held)
-	}
-}
-
-func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	s, clk, server, queue := startWithAlerts(t, map[string]string{
-		crowdSecURL: lapi, crowdSecKey: bouncerKey, mode: observe,
-	})
-	loadDecisions(t, server, clk.Now(), `[{"duration": "4h0m0s", `+
-		`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
-		`"value": "`+client+`"}]`)
-
-	line := s.get(client, http.StatusOK, requestlog.ActionForward)
-	wantWouldAction(t, line, requestlog.ActionTemporarilyBanned)
-	wantReputation(t, line, decisionsURL)
-
-	if held := server.Ledger.Snapshot(); len(held) != 0 {
-		t.Errorf("bans %+v, want none", held)
-	}
-
-	waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
-	if len(waiting) != 2 || waiting[1].Event != alerts.EventBan ||
-		waiting[1].Detail["cause"] != bans.CauseCrowdSec ||
-		waiting[1].Detail["mode"] != observe {
-		t.Errorf(
-			"alerts waiting %+v, want a reputation_hit alert, then the ban alert "+
-				"marked observe",
-			waiting,
-		)
-	}
-}
-
-// loadDecisions puts into server's lists the copy of the decision list at
-// decisionsURL, answer, the engine's answer, fetched at fetched, as
-// lists.json would at start.
-func loadDecisions(
-	t *testing.T, server *waf.Server, fetched time.Time, answer string,
-) {
-	t.Helper()
-
-	err := server.Lists.Load([]reputation.List{{
-		URL: decisionsURL, Tried: fetched, Fetched: fetched, Lines: []string{answer},
-	}})
-	if err != nil {
-		t.Fatalf("load the decision list: %v", err)
-	}
-}

internal/waf/heuristics.go

@@ -60,7 +60,6 @@ func afterLookup() []heuristic {
 	return []heuristic{
 		countries{},
 		blocklists{},
-		crowdSec{},
 		dnsbl{},
 		abuseIPDB{},
 		tagDeny{},

internal/waf/lookup.go

@@ -12,7 +12,7 @@ import (
 )
 
 // The headers in which an answer of allow gives rproxy the client's AS number
-// and country, for the app, while SWW_WAF_ADD_LOOKUP_HEADERS is set. Go writes
+// and country, for the app, while SWW_WAF_ADD_LOOKUP_HEADERS is true. Go writes
 // every header name in this form, as it sends it and as it receives it, so
 // X-Client-ASN arrives as X-Client-Asn; header names are not case-sensitive.
 const (

internal/waf/lookup_test.go

@@ -36,6 +36,7 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(
 	geojsURL, asked, release := startHeldGeoJS(t)
 	s, _, server := startWithClock(t, geojsURL, map[string]string{
 		asnAndCountryLookupTimeout: "3600000",
+		addLookupHeaders:           falseValue,
 		rateLimitPerMinute:         "1",
 	})
 
@@ -418,7 +419,7 @@ func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(
 		geojsURL,
 		time.Now,
 		map[string]string{
-			trustedProxies: trustLocalhost,
+			trustedProxies: trustLocalhost, addLookupHeaders: falseValue,
 		},
 	)
 	s := &sender{t: t, addr: addr, out: out}
@@ -457,6 +458,8 @@ func TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays(
 			map[string]string{
 				asnAndCountryLookupTimeout: strconv.FormatInt(
 					timeout.Milliseconds(), 10),
+				// Off, so that only the country list makes the request wait.
+				addLookupHeaders: falseValue,
 				allowedCountries: "DE",
 			})
 

internal/waf/proxy.go

@@ -75,8 +75,8 @@ type Params struct {
 	Rules *rules.Files
 	// Alerts receive the alert for each ban waf makes or makes
 	// permanent, for each count over an anomaly threshold, for each request
-	// whose client a blocklist, the CrowdSec decision list, a DNSBL zone or
-	// AbuseIPDB lists, for each request the Core Rule Set scores at or over
+	// whose client a blocklist, a DNSBL zone or AbuseIPDB lists, for each
+	// request the Core Rule Set scores at or over
 	// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, and for GeoJS failing, a fetch of a list
 	// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
 	// or the day's AbuseIPDB checks used up.
@@ -194,11 +194,13 @@ func newReputation(
 ) (*reputation.Lists, *reputation.DNSBL, *reputation.AbuseIPDB) {
 	cfg := params.Config
 	lists := reputation.New(reputation.Params{
-		BlocklistURLs: cfg.BlocklistURLs, Refresh: cfg.BlocklistRefresh,
-		ASNLimitPercentURL:   cfg.ASNLimitPercentURL,
-		TagLists:             cfg.TagLists,
-		CrowdSecDecisionsURL: cfg.CrowdSecDecisionsURL, CrowdSecKey: cfg.CrowdSecKey,
-		Now: params.Now, ProcessLog: params.ProcessLog, Alerts: params.Alerts,
+		BlocklistURLs:      cfg.BlocklistURLs,
+		Refresh:            cfg.BlocklistRefresh,
+		ASNLimitPercentURL: cfg.ASNLimitPercentURL,
+		TagLists:           cfg.TagLists,
+		Now:                params.Now,
+		ProcessLog:         params.ProcessLog,
+		Alerts:             params.Alerts,
 	})
 	dnsbl := reputation.NewDNSBL(reputation.DNSBLParams{
 		Zones: cfg.DNSBLZones, Resolver: cfg.DNSBLResolver,
@@ -226,12 +228,9 @@ func newReputation(
 // without the rules SWW_WAF_CORE_RULE_SET_DISABLED_RULES switches off, reading bodies
 // up to SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE, with the exclusions in
 // SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE and the fixes of the apps in SWW_WAF_APPS,
-// or nil while SWW_WAF_CORE_RULE_SET_MODE is off. It
-// reads at most QuestionBodyMaxBytes less one byte: a question's body of
-// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
-// JSON or XML body over its limit. An exclusions file that coreruleset.New refuses is
-// an error in every mode. While the mode is off, a set file is loaded with the
-// Core Rule Set only to check it.
+// or nil while SWW_WAF_CORE_RULE_SET_MODE is off. An exclusions file that
+// coreruleset.New refuses is an error in every mode. While the mode is off, a
+// set file is loaded with the Core Rule Set only to check it.
 func NewCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
 	if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff &&
 		cfg.CoreRuleSetExclusionsFile == "" {
@@ -239,14 +238,11 @@ func NewCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
 	}
 
 	coreRuleSet, err := coreruleset.New(coreruleset.Params{
-		ParanoiaLevel: cfg.CoreRuleSetParanoiaLevel,
-		DisabledRules: cfg.CoreRuleSetDisabledRules,
-		BodyInspectionSize: min(
-			cfg.CoreRuleSetBodyInspectionSize,
-			QuestionBodyMaxBytes-1,
-		),
-		ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
-		Apps:           cfg.Apps,
+		ParanoiaLevel:      cfg.CoreRuleSetParanoiaLevel,
+		DisabledRules:      cfg.CoreRuleSetDisabledRules,
+		BodyInspectionSize: cfg.CoreRuleSetBodyInspectionSize,
+		ExclusionsFile:     cfg.CoreRuleSetExclusionsFile,
+		Apps:               cfg.Apps,
 	})
 	if err != nil {
 		return nil, fmt.Errorf(

internal/waf/proxy_test.go

@@ -93,6 +93,10 @@ const off = "off"
 // on is the value that switches a setting that takes true or false on.
 const on = "true"
 
+// falseValue is the value that switches a setting that takes true or false
+// off.
+const falseValue = "false"
+
 // output collects what sww writes on stdout.
 type output struct {
 	mu  sync.Mutex

internal/waf/reputation.go

@@ -1,7 +1,6 @@
 package waf
 
-// What the reputation heuristics share: blocklists, crowdsec, dnsbl and
-// abuseipdb.
+// What the reputation heuristics share: blocklists, dnsbl and abuseipdb.
 
 import (
 	"sneak.berlin/go/smallwebwaf/internal/alerts"

internal/waf/waf.go

@@ -146,18 +146,15 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 // refusedWhateverTheAnswer reports whether the request is one that does not wait
 // for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
-// its client: the heuristic crowdsec, blocklists or rate_limits will refuse it,
-// as a function in that heuristic's file says. In observe mode, which refuses
+// its client: the heuristic blocklists or rate_limits will refuse it, as a
+// function in that heuristic's file says. In observe mode, which refuses
 // nothing, none is. rproxy lets a request through when waf answers too late.
 func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
 	if rq.h.config.Observe {
 		return false
 	}
 
-	_, decided := rq.crowdSecDecisionInForce(now)
-
-	return decided ||
-		rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
+	return rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
 		rq.overAWholeRateLimit(now)
 }
 
@@ -323,7 +320,7 @@ func asTheVisitorSentIt(r *http.Request, body []byte) *http.Request {
 }
 
 // allow answers a question with DispositionAllow and, while
-// SWW_WAF_ADD_LOOKUP_HEADERS is set, the client's AS number and country.
+// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
 func (rq *request) allow() {
 	rq.allowed = true
 

internal/waf/waf_test.go

@@ -869,7 +869,7 @@ func TestAllowCarriesTheLookupHeadersWhileTheSettingIsSet(t *testing.T) {
 		wantASN, wantCountry string
 	}{
 		{on, "AS64496", "DE"},
-		{"false", "", ""},
+		{falseValue, "", ""},
 	} {
 		t.Run(tc.setting, func(t *testing.T) {
 			t.Parallel()
@@ -1787,7 +1787,7 @@ func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) {
 	t.Parallel()
 
 	s := startWAF(t, "", map[string]string{
-		coreRuleSetMode: block, coreRuleSetBodyInspectionSize: "64K",
+		coreRuleSetMode: block, coreRuleSetBodyInspectionSize: "8K",
 	})
 
 	ask := func(contentType, body string) answer {
@@ -1802,10 +1802,10 @@ func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) {
 	got := ask(formData, "id=1'%20OR%20'1'='1")
 	wantDisposition(t, got, http.StatusForbidden, waf.DispositionDeny)
 
-	// The first 64 KiB of a longer JSON body cannot be parsed, which is no
+	// The first 8 KiB of a longer JSON body cannot be parsed, which is no
 	// attack, so the Core Rule Set does not read it.
-	cut := `{"text":"` + strings.Repeat("x", waf.QuestionBodyMaxBytes)
-	got = ask("application/json", cut[:waf.QuestionBodyMaxBytes])
+	cut := `{"text":"` + strings.Repeat("x", 8<<10)
+	got = ask("application/json", cut)
 	wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
 }