From 89a60cf9fc699c596729ade973747a2f72556bca Mon Sep 17 00:00:00 2001 From: clawbot <35+clawbot@noreply.example.org> Date: Sat, 14 Nov 2026 18:56:10 +0100 Subject: [PATCH] README: blocks misbehaving scrapers, not normal ones (closes #387) The opening now lists what bans a client with the default settings: the rate and byte limits, a probe for a secret path, a security scanner's user agent, and the refusal limit. A scraper that does none of these is left alone. Model: opus-5-5 --- README.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3565587..1ed7527 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,20 @@ `smallwebwaf` is an MIT-licensed web application firewall, written in Go by [@sneak](https://sneak.berlin), for self-hosted web apps. It refuses the -requests of scrapers, scanners and other abusers before they reach an app. +requests of misbehaving scrapers, scanners and other abusers before they reach +an app. + +With the default settings, a client is banned when it does any of these: + +- makes more than 120 requests in a minute, 1500 in an hour or 6000 in a day; +- moves more than `50G` of request and response bodies in an hour, or `100G` in + a day; +- asks for a secret or backup file, such as `/.env` or `/.git/config`; +- sends a user agent that names a security scanner, such as `sqlmap` or `nikto`; +- has more than 30 requests in a minute refused as probes or attacks. + +A scraper that does none of these is left alone. A scraper that waits 15 seconds +between requests stays under the rate limits. ## What it does