make sure the decider’s heuristics are modular in our code. we are going to be analyzing the logs from the real world tomorrow and will be adding many heuristics and tuning others.
The decider is servewaf, which answers rproxy's question about each request: allow, refuse or ban. Its heuristics are the checks behind that answer: the request and byte limits, the blocklists and reputation lookups, the country and AS percentages, and the new rule in #194.
Definition of done:
Each heuristic is its own unit in the code: its own file, behind one small shared interface, with its own settings and its own tests. Behaviour does not change: the existing tests pass unchanged, apart from where they now live.
Adding a heuristic means adding one file and registering it in one place, with no change to the others. Tuning one means changing only its own settings or its own file.
Every log line for a refusal or a ban names the heuristic that made it, with the figures it acted on, such as the count and the limit. Many will be added and tuned from real-world logs, so each decision must be traceable to its rule.
make check passes.
This comes before #194, which is then built as one of these modules.
Model: opus-5-5
sneak, in chat on 6 November:
> make sure the decider’s heuristics are modular in our code. we are going to be analyzing the logs from the real world tomorrow and will be adding many heuristics and tuning others.
The decider is `servewaf`, which answers `rproxy`'s question about each request: allow, refuse or ban. Its heuristics are the checks behind that answer: the request and byte limits, the blocklists and reputation lookups, the country and AS percentages, and the new rule in https://git.eeqj.de/sneak/smallwebwaf/issues/194.
**Definition of done:**
- Each heuristic is its own unit in the code: its own file, behind one small shared interface, with its own settings and its own tests. Behaviour does not change: the existing tests pass unchanged, apart from where they now live.
- Adding a heuristic means adding one file and registering it in one place, with no change to the others. Tuning one means changing only its own settings or its own file.
- Every log line for a refusal or a ban names the heuristic that made it, with the figures it acted on, such as the count and the limit. Many will be added and tuned from real-world logs, so each decision must be traceable to its rule.
- `make check` passes.
This comes before https://git.eeqj.de/sneak/smallwebwaf/issues/194, which is then built as one of these modules.
Model: opus-5-5
clawbot
self-assigned this 2026-11-06 18:37:49 +01:00
Plan for the implementer. Start from next after #173 has landed, so the package is internal/servewaf. All new files go in internal/servewaf, whose doc says it runs the checks on each request; the new log fields go in internal/requestlog.
The interface, and the one place heuristics are listed (new file internal/servewaf/heuristics.go):
A heuristic interface with two methods: name() string, the name the log line gives, and check(ctx context.Context, rq *request, now time.Time) *verdict, nil when the heuristic lets the request through.
A verdict: the action (the existing requestlog.Action* constants) and the figures the heuristic acted on, as name and value pairs.
Three ordered lists, the only place a heuristic is registered:
before the client's AS number and country are looked up: deny_nets, ban (a ban already on the client's netblock);
after the lookup: countries, blocklists, crowdsec, dnsbl, abuseipdb, rate_limits, trap_paths, rule_files, header_limits, core_rule_set;
once rproxy's report has come: byte_limits, error_burst.
One loop runs the first two lists in order and stops at the first verdict, replacing the chain in checkClient; the report path runs the third. The order, what counts toward the rate limits, the lookup, SWWAF_ALLOW_NETS skipping everything, and observe mode stay exactly as today. A comment at the top of the file says in three lines how to add a heuristic: one file, one line in a list.
One file per heuristic, holding a type that implements the interface, with the existing code moved into it: denynets.go, bans.go (keeps the existing-ban check), countries.go, blocklists.go, crowdsec.go, dnsbl.go, abuseipdb.go, ratelimits.go, trappaths.go, rulefiles.go, headerlimits.go, coreruleset.go, bytelimits.go, errorburst.go. Shared ban-making code (banForLimit, banForAttack, banResponse) stays in bans.go. A comment at the top of each file lists the settings it reads. Tests stay in the existing test files, which already match these names; move a test only where its file no longer matches what it tests, with git mv.
Settings stay parsed in internal/config, so every bad value still fails at startup in one place. Each heuristic reads only its own config.Config fields; a new heuristic adds its fields there.
The log line gains two fields, set on every refusal or ban, in observe mode too:
heuristic: the name of the heuristic that refused or banned.
heuristic_figures: an object of what it acted on, for example {"window": "hour", "count": 1501, "limit": 1500} for rate_limits; the matched rule's ID for rule_files; the score and threshold for abuseipdb and core_rule_set; the expiry for ban; the list names for blocklists. Existing fields stay as they are.
docs/request-log.md documents both fields and lists every heuristic name with its figures. SPEC.md, where it lists the checks in order, names each by its heuristic name.
Tests: behaviour does not change. Existing tests pass unchanged, except a test that compares a whole refusal log line, which gains the two fields. Add one test per heuristic, in its test file, asserting heuristic and heuristic_figures on a refusal. make check passes.
Edit by hand; no scripted edits.
Model: opus-5-5
Plan for the implementer. Start from `next` after https://git.eeqj.de/sneak/smallwebwaf/issues/173 has landed, so the package is `internal/servewaf`. All new files go in `internal/servewaf`, whose doc says it runs the checks on each request; the new log fields go in `internal/requestlog`.
**The interface, and the one place heuristics are listed** (new file `internal/servewaf/heuristics.go`):
- A `heuristic` interface with two methods: `name() string`, the name the log line gives, and `check(ctx context.Context, rq *request, now time.Time) *verdict`, nil when the heuristic lets the request through.
- A `verdict`: the action (the existing `requestlog.Action*` constants) and the figures the heuristic acted on, as name and value pairs.
- Three ordered lists, the only place a heuristic is registered:
- before the client's AS number and country are looked up: `deny_nets`, `ban` (a ban already on the client's netblock);
- after the lookup: `countries`, `blocklists`, `crowdsec`, `dnsbl`, `abuseipdb`, `rate_limits`, `trap_paths`, `rule_files`, `header_limits`, `core_rule_set`;
- once rproxy's report has come: `byte_limits`, `error_burst`.
- One loop runs the first two lists in order and stops at the first verdict, replacing the chain in `checkClient`; the report path runs the third. The order, what counts toward the rate limits, the lookup, `SWWAF_ALLOW_NETS` skipping everything, and observe mode stay exactly as today. A comment at the top of the file says in three lines how to add a heuristic: one file, one line in a list.
**One file per heuristic**, holding a type that implements the interface, with the existing code moved into it: `denynets.go`, `bans.go` (keeps the existing-ban check), `countries.go`, `blocklists.go`, `crowdsec.go`, `dnsbl.go`, `abuseipdb.go`, `ratelimits.go`, `trappaths.go`, `rulefiles.go`, `headerlimits.go`, `coreruleset.go`, `bytelimits.go`, `errorburst.go`. Shared ban-making code (`banForLimit`, `banForAttack`, `banResponse`) stays in `bans.go`. A comment at the top of each file lists the settings it reads. Tests stay in the existing test files, which already match these names; move a test only where its file no longer matches what it tests, with `git mv`.
**Settings** stay parsed in `internal/config`, so every bad value still fails at startup in one place. Each heuristic reads only its own `config.Config` fields; a new heuristic adds its fields there.
**The log line** gains two fields, set on every refusal or ban, in observe mode too:
- `heuristic`: the name of the heuristic that refused or banned.
- `heuristic_figures`: an object of what it acted on, for example `{"window": "hour", "count": 1501, "limit": 1500}` for `rate_limits`; the matched rule's ID for `rule_files`; the score and threshold for `abuseipdb` and `core_rule_set`; the expiry for `ban`; the list names for `blocklists`. Existing fields stay as they are.
- `docs/request-log.md` documents both fields and lists every heuristic name with its figures. `SPEC.md`, where it lists the checks in order, names each by its heuristic name.
**Tests:** behaviour does not change. Existing tests pass unchanged, except a test that compares a whole refusal log line, which gains the two fields. Add one test per heuristic, in its test file, asserting `heuristic` and `heuristic_figures` on a refusal. `make check` passes.
Edit by hand; no scripted edits.
Model: opus-5-5
Correction to the plan above, from the naming audit (#192 (comment)): the heuristic ban is named active_ban, and error_burst is named refusal_limit, in a file refusallimit.go. Everything else in the plan stands.
Model: opus-5-5
Correction to the plan above, from the naming audit (https://git.eeqj.de/sneak/smallwebwaf/issues/192#issuecomment-169850): the heuristic `ban` is named `active_ban`, and `error_burst` is named `refusal_limit`, in a file `refusallimit.go`. Everything else in the plan stands.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
sneak, in chat on 6 November:
The decider is
servewaf, which answersrproxy's question about each request: allow, refuse or ban. Its heuristics are the checks behind that answer: the request and byte limits, the blocklists and reputation lookups, the country and AS percentages, and the new rule in #194.Definition of done:
make checkpasses.This comes before #194, which is then built as one of these modules.
Model: opus-5-5
Plan for the implementer. Start from
nextafter #173 has landed, so the package isinternal/servewaf. All new files go ininternal/servewaf, whose doc says it runs the checks on each request; the new log fields go ininternal/requestlog.The interface, and the one place heuristics are listed (new file
internal/servewaf/heuristics.go):heuristicinterface with two methods:name() string, the name the log line gives, andcheck(ctx context.Context, rq *request, now time.Time) *verdict, nil when the heuristic lets the request through.verdict: the action (the existingrequestlog.Action*constants) and the figures the heuristic acted on, as name and value pairs.deny_nets,ban(a ban already on the client's netblock);countries,blocklists,crowdsec,dnsbl,abuseipdb,rate_limits,trap_paths,rule_files,header_limits,core_rule_set;byte_limits,error_burst.checkClient; the report path runs the third. The order, what counts toward the rate limits, the lookup,SWWAF_ALLOW_NETSskipping everything, and observe mode stay exactly as today. A comment at the top of the file says in three lines how to add a heuristic: one file, one line in a list.One file per heuristic, holding a type that implements the interface, with the existing code moved into it:
denynets.go,bans.go(keeps the existing-ban check),countries.go,blocklists.go,crowdsec.go,dnsbl.go,abuseipdb.go,ratelimits.go,trappaths.go,rulefiles.go,headerlimits.go,coreruleset.go,bytelimits.go,errorburst.go. Shared ban-making code (banForLimit,banForAttack,banResponse) stays inbans.go. A comment at the top of each file lists the settings it reads. Tests stay in the existing test files, which already match these names; move a test only where its file no longer matches what it tests, withgit mv.Settings stay parsed in
internal/config, so every bad value still fails at startup in one place. Each heuristic reads only its ownconfig.Configfields; a new heuristic adds its fields there.The log line gains two fields, set on every refusal or ban, in observe mode too:
heuristic: the name of the heuristic that refused or banned.heuristic_figures: an object of what it acted on, for example{"window": "hour", "count": 1501, "limit": 1500}forrate_limits; the matched rule's ID forrule_files; the score and threshold forabuseipdbandcore_rule_set; the expiry forban; the list names forblocklists. Existing fields stay as they are.docs/request-log.mddocuments both fields and lists every heuristic name with its figures.SPEC.md, where it lists the checks in order, names each by its heuristic name.Tests: behaviour does not change. Existing tests pass unchanged, except a test that compares a whole refusal log line, which gains the two fields. Add one test per heuristic, in its test file, asserting
heuristicandheuristic_figureson a refusal.make checkpasses.Edit by hand; no scripted edits.
Model: opus-5-5
Correction to the plan above, from the naming audit (#192 (comment)): the heuristic
banis namedactive_ban, anderror_burstis namedrefusal_limit, in a filerefusallimit.go. Everything else in the plan stands.Model: opus-5-5