README: blocks misbehaving scrapers, not normal ones (closes #387)

The opening now says a scraper within the default limits is left alone
whatever its user agent, and lists what bans a client: the rate and byte
limits, a probe for a secret path, and the refusal limit.

Model: opus-5-5

README.md

@@ -2,7 +2,23 @@
 
 `smallwebwaf` is an MIT-licensed web application firewall, written in Go by
 [@sneak](https://sneak.berlin), for self-hosted web apps. It refuses the
-requests of scrapers, scanners and other abusers before they reach an app.
+requests of misbehaving scrapers, scanners and other abusers before they reach
+an app.
+
+It does not block scrapers as such. A scraper that stays within the limits below
+is left alone, whatever its user agent. With the default settings, a client is
+banned when it does any of these:
+
+- makes more than 120 requests in a minute, 1500 in an hour or 6000 in a day;
+- moves more than `50G` of request and response bodies in an hour, or `100G` in
+  a day;
+- asks for a path no real visitor asks for, such as `/.env` or `/.git/config`;
+- has more than 30 requests in a minute refused as probes or attacks.
+
+A scraper that waits 15 seconds between requests stays under every one of these
+limits. The user agent alone gets a client refused only when it names a security
+scanner, such as `sqlmap` or `nikto`. Settings you add, such as a refused
+country or a blocklist, can refuse any client.
 
 ## What it does
 
@@ -22,26 +38,7 @@ shipped in one image, with two commands that do this together:
 The site keeps answering while `waf` is down, slow or restarting: without an
 answer in time, `rproxy` passes the request to the app.
 
-`rproxy` also protects the app on its own, whether or not `waf` answers. With
-the default settings, `rproxy`:
-
-- answers `400` to a malformed request, such as a broken header line or a
-  missing `Host`;
-- answers `405` to a method it does not pass on, such as `TRACE`;
-- answers `431` to a request line and headers over 32 KiB together;
-- gives the visitor 60 seconds to send the request line and headers, and then 60
-  seconds more for the whole request to reach the app;
-- answers `413` to a request body over 100 MiB;
-- gives the app's answer 30 minutes to reach the visitor, counted from the end
-  of the request;
-- answers `502` in place of an app's answer whose body is over 1 GiB, or cuts it
-  off if it has started;
-- closes a kept-open connection after 120 seconds idle.
-
-`rproxy` does not limit the number of headers. `waf` limits the number of
-headers and cookies, and the length of the request line and of each header line.
-
-Together, `rproxy` and `waf` protect against:
+It protects against:
 
 - Scrapers that fetch every page of a site, all day. Each client gets a limit on
   requests and on body bytes per minute, per hour and per day. The defaults
@@ -63,6 +60,8 @@ Together, `rproxy` and `waf` protect against:
       limit, so that they are banned sooner;
     - refuse or limit clients on a blocklist, a DNS blocklist, or with a high
       AbuseIPDB score.
+- Slow and oversized requests. Every request and every answer has a time limit
+  and a size limit.
 
 For the operator, it: