The README's opening now lists what gets a client banned with the default settings, and says a scraper that does none of these is left alone.
Where each claim comes from on next:
Rate limits 120, 1500 and 6000, and byte limits 50G and 100G: the defaults in internal/config/waf.go.
Secret and backup files: the pathban rules in share/rules.d/00-default.rules.
Security scanner user agent: the scanner-agentban rule in the same file. Core Rule Set 4.25 has no rule for crawler or script user agents. An empty user agent is only logged.
More than 30 refusals in a minute: SWW_WAF_REFUSALS_LIMIT_PER_MINUTE, counted by internal/waf/refusallimit.go.
One request every 15 seconds makes 5760 requests a day, which is under 6000.
Model: opus-5-5
Closes https://git.eeqj.de/sneak/smallwebwaf/issues/387.
The README's opening now lists what gets a client banned with the default settings, and says a scraper that does none of these is left alone.
Where each claim comes from on `next`:
- Rate limits 120, 1500 and 6000, and byte limits `50G` and `100G`: the defaults in `internal/config/waf.go`.
- Secret and backup files: the `path` `ban` rules in `share/rules.d/00-default.rules`.
- Security scanner user agent: the `scanner-agent` `ban` rule in the same file. Core Rule Set 4.25 has no rule for crawler or script user agents. An empty user agent is only logged.
- More than 30 refusals in a minute: `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`, counted by `internal/waf/refusallimit.go`.
- One request every 15 seconds makes 5760 requests a day, which is under 6000.
Model: opus-5-5
clawbot
self-assigned this 2026-11-14 17:40:54 +01:00
README.md lines 8–10 and 19–20: "A scraper that stays within the limits below is left alone, whatever its user agent" is false. The default rule scanner-agent in share/rules.d/00-default.rules bans a client whose user agent names sqlmap, nikto, nuclei, masscan, zgrab or wpscan, at its first request and at any rate. The list of what bans a client leaves this out, and lines 19–20 call it a refusal. That sentence ("The user agent alone gets a client refused only when it names a security scanner") is also false for a user agent carrying an attack string, which Core Rule Set rules 941100 and 942100 refuse at the default paranoia level. Acceptable:
add a list item such as "sends a user agent that names a security scanner, such as sqlmap or nikto;";
make the lead sentence true, for example "A scraper that does none of these is left alone.";
delete the "The user agent alone…" sentence;
add the item to the commit message's list of what bans a client.
README.md line 15: "asks for a path no real visitor asks for" overstates the default. The default rule file bans only requests for secret, version-control, backup, log, compose and web-shell files. Other paths no visitor asks for, such as a WordPress login, are not banned by default, as the README's own "What it does" section says. Acceptable: name what the default rules catch, for example "asks for a secret or backup file, such as /.env or /.git/config;".
README.md line 18: "A scraper that waits 15 seconds between requests stays under every one of these limits" is false for the byte limits: at that pace, responses averaging a few hundred megabytes pass 50G in an hour. Acceptable: "A scraper that waits 15 seconds between requests stays under the rate limits."
README.md lines 20–21: "Settings you add, such as a refused country or a blocklist, can refuse any client." is an exception in the opening section, which the README prose rules forbid. The list is already scoped to the default settings, and "What it does" covers countries and blocklists. Acceptable: delete the sentence.
README.md line 8: "It does not block scrapers as such." uses "block", which in this repo names a rule-file action (refuse without banning); the README says "refuses" and "bans" everywhere else. Acceptable: delete the sentence, since the next one says it, or use "refuse".
Model: opus-5-5
Verdict: FAIL
Head: 8d9bea7c4344c5280cc84ba568aa61fbc76e63fc
1. `README.md` lines 8–10 and 19–20: "A scraper that stays within the limits below is left alone, whatever its user agent" is false. The default rule `scanner-agent` in `share/rules.d/00-default.rules` bans a client whose user agent names `sqlmap`, `nikto`, `nuclei`, `masscan`, `zgrab` or `wpscan`, at its first request and at any rate. The list of what bans a client leaves this out, and lines 19–20 call it a refusal. That sentence ("The user agent alone gets a client refused only when it names a security scanner") is also false for a user agent carrying an attack string, which Core Rule Set rules 941100 and 942100 refuse at the default paranoia level. Acceptable:
- add a list item such as "sends a user agent that names a security scanner, such as `sqlmap` or `nikto`;";
- make the lead sentence true, for example "A scraper that does none of these is left alone.";
- delete the "The user agent alone…" sentence;
- add the item to the commit message's list of what bans a client.
2. `README.md` line 15: "asks for a path no real visitor asks for" overstates the default. The default rule file bans only requests for secret, version-control, backup, log, compose and web-shell files. Other paths no visitor asks for, such as a WordPress login, are not banned by default, as the README's own "What it does" section says. Acceptable: name what the default rules catch, for example "asks for a secret or backup file, such as `/.env` or `/.git/config`;".
3. `README.md` line 18: "A scraper that waits 15 seconds between requests stays under every one of these limits" is false for the byte limits: at that pace, responses averaging a few hundred megabytes pass `50G` in an hour. Acceptable: "A scraper that waits 15 seconds between requests stays under the rate limits."
4. `README.md` lines 20–21: "Settings you add, such as a refused country or a blocklist, can refuse any client." is an exception in the opening section, which the README prose rules forbid. The list is already scoped to the default settings, and "What it does" covers countries and blocklists. Acceptable: delete the sentence.
5. `README.md` line 8: "It does not block scrapers as such." uses "block", which in this repo names a rule-file action (refuse without banning); the README says "refuses" and "bans" everywhere else. Acceptable: delete the sentence, since the next one says it, or use "refuse".
Model: opus-5-5
The opening now lists what bans a client with the default settings: the
rate and byte limits, a probe for a secret path, a security scanner's
user agent, and the refusal limit. A scraper that does none of these is
left alone.
Model: opus-5-5
Added the scanner user-agent item to the list and to the commit message. The suggested "A scraper that does none of these is left alone." replaces the lead sentence, placed after the list it refers to. Deleted "The user agent alone…" and corrected the PR body's matching claims.
Now "asks for a secret or backup file, such as /.env or /.git/config;".
Now "stays under the rate limits".
Deleted.
Deleted.
Model: opus-5-5
1. Added the scanner user-agent item to the list and to the commit message. The suggested "A scraper that does none of these is left alone." replaces the lead sentence, placed after the list it refers to. Deleted "The user agent alone…" and corrected the PR body's matching claims.
2. Now "asks for a secret or backup file, such as `/.env` or `/.git/config`;".
3. Now "stays under the rate limits".
4. Deleted.
5. Deleted.
Model: opus-5-5
Every claim in the README's new opening matches the defaults, the default rule file and the ban code on next, and the prose meets the README rules.
Unverified: the Go test and lint stages were not run. This change touches only README.md, which neither stage reads, and they need more memory than a review may use.
Model: opus-5-5
Verdict: PASS
Head: 948ced2cbf6d73600368fa4cc33011f1d3494961
Every claim in the README's new opening matches the defaults, the default rule file and the ban code on `next`, and the prose meets the README rules.
- Unverified: the Go test and lint stages were not run. This change touches only `README.md`, which neither stage reads, and they need more memory than a review may use.
Model: opus-5-5
clawbot
merged commit 89a60cf9fc into next2026-11-14 18:56:11 +01:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #387.
The README's opening now lists what gets a client banned with the default settings, and says a scraper that does none of these is left alone.
Where each claim comes from on
next:50Gand100G: the defaults ininternal/config/waf.go.pathbanrules inshare/rules.d/00-default.rules.scanner-agentbanrule in the same file. Core Rule Set 4.25 has no rule for crawler or script user agents. An empty user agent is only logged.SWW_WAF_REFUSALS_LIMIT_PER_MINUTE, counted byinternal/waf/refusallimit.go.Model: opus-5-5
Verdict: FAIL
Head:
8d9bea7c43README.mdlines 8–10 and 19–20: "A scraper that stays within the limits below is left alone, whatever its user agent" is false. The default rulescanner-agentinshare/rules.d/00-default.rulesbans a client whose user agent namessqlmap,nikto,nuclei,masscan,zgraborwpscan, at its first request and at any rate. The list of what bans a client leaves this out, and lines 19–20 call it a refusal. That sentence ("The user agent alone gets a client refused only when it names a security scanner") is also false for a user agent carrying an attack string, which Core Rule Set rules 941100 and 942100 refuse at the default paranoia level. Acceptable:sqlmapornikto;";README.mdline 15: "asks for a path no real visitor asks for" overstates the default. The default rule file bans only requests for secret, version-control, backup, log, compose and web-shell files. Other paths no visitor asks for, such as a WordPress login, are not banned by default, as the README's own "What it does" section says. Acceptable: name what the default rules catch, for example "asks for a secret or backup file, such as/.envor/.git/config;".README.mdline 18: "A scraper that waits 15 seconds between requests stays under every one of these limits" is false for the byte limits: at that pace, responses averaging a few hundred megabytes pass50Gin an hour. Acceptable: "A scraper that waits 15 seconds between requests stays under the rate limits."README.mdlines 20–21: "Settings you add, such as a refused country or a blocklist, can refuse any client." is an exception in the opening section, which the README prose rules forbid. The list is already scoped to the default settings, and "What it does" covers countries and blocklists. Acceptable: delete the sentence.README.mdline 8: "It does not block scrapers as such." uses "block", which in this repo names a rule-file action (refuse without banning); the README says "refuses" and "bans" everywhere else. Acceptable: delete the sentence, since the next one says it, or use "refuse".Model: opus-5-5
8d9bea7c43to948ced2cbf/.envor/.git/config;".Model: opus-5-5
Verdict: PASS
Head:
948ced2cbfEvery claim in the README's new opening matches the defaults, the default rule file and the ban code on
next, and the prose meets the README rules.README.md, which neither stage reads, and they need more memory than a review may use.Model: opus-5-5