waf: heuristics pass facts through typed fields of decision, never through the log line #442

Open
opened 2026-11-19 17:13:39 +01:00 by clawbot · 1 comment
Collaborator

Start once the issue that gives each heuristic its own settings and parts is closed: #441. The review orders this as the second of three steps inside the heuristics.

From the architecture review, "The decision pipeline: the heuristics", under "Heuristics pass facts to each other through fields nobody lists": #283 (comment). Approved for building: #283 (comment). The code was read at 809d266 on next, where discriminator is now waf.

What is there

Heuristics in internal/waf tell each other things through the JSON log line, rq.line, and through flags on two structs. As writer, then reader:

  • rq.line.Tags: written through addTag by tagLists, routeWatchTags, noteLookup, dnsbl, abuseIPDB, ruleFiles and coreRuleSet. Read by tagDeny and by limitPercentages in biased.go.
  • rq.line.ASN, rq.line.ASName, rq.line.Country: written by noteLookup. Read by countries, limitPercentages, allow, the two functions that make bans, and the alerts. decision.lookupAnswer holds the same three values and is not what they read.
  • rq.line.WouldAction: written by check. Read by byteLimits and refusalLimit.
  • rq.line.Offence: written by banForLimit. Read by addToHistory.
  • rq.line.BanExpires: written by activeBan, which reads it back for its figures.
  • rq.line.Method, rq.line.Host, rq.line.UserAgent: read by noted for a ban's notes.
  • rq.blocklisted, rq.dnsblListed, rq.abuseIPDBHit, on request: written by blocklists, dnsbl, abuseIPDB. Read by limitPercentages.
  • counted, rateLimitPercent, bytesLimitPercent, attack, blockedByBlockRule, blockedByCoreRuleSet, on decision: these are already typed fields, and stay.

About forty lines of internal/waf touch these fields of the log line.

Why it matters

  • The log line doubles as working memory. A change to how a field is logged can change a decision.
  • The order of the lists carries meaning the lists do not show. tagDeny is listed twice because tags are added on both sides of it. rateLimits must come after the three reputation heuristics because it reads their flags.
  • A heuristic that has something to tell a later one needs a new field outside its file.

What changes

  • decision holds what heuristics tell each other, as typed fields of its own: the tags, the lookup answer it already has, what would have been done, the offence, and the three reputation flags. The comment on each field names what sets it.
  • The log line is filled from decision, and nothing reads it back to decide, to make a ban's notes or to raise an alert.

Before (tagdeny.go, bytelimits.go):

for _, tag := range rq.line.Tags {
// ...
if !rq.counted || rq.line.WouldAction != "" {

After (the field names are placeholders):

for _, tag := range rq.tags {
// ...
if !rq.counted || rq.wouldRefuse {

What stays

  • Every field of the log line, with the same value at the same moment: both lines of a request, the decision's and the report's.
  • What is held for the report (holdForReport) still carries all the report's steps need.
  • The order of the lists.

Done when

  • no code in internal/waf reads a field of rq.line to decide, to fill a ban's notes or to raise an alert. Reading RequestID to key what is held for the report stays;
  • each fact a heuristic leaves for a later one is a field of decision whose comment names what sets it;
  • behaviour is unchanged: every existing test passes with no assertion changed;
  • make check passes;
  • the PR body lists every new field name. Each says what the thing is to a reader of that one file, the standard of the symbol names audit (#213).

If this is more than one PR can carry, say so on this issue instead of starting.

Model: fable-5-2

**Start once the issue that gives each heuristic its own settings and parts is closed: https://git.eeqj.de/sneak/smallwebwaf/issues/441.** The review orders this as the second of three steps inside the heuristics. From the architecture review, "The decision pipeline: the heuristics", under "Heuristics pass facts to each other through fields nobody lists": https://git.eeqj.de/sneak/smallwebwaf/issues/283#issuecomment-173833. Approved for building: https://git.eeqj.de/sneak/smallwebwaf/issues/283#issuecomment-183550. The code was read at `809d266` on `next`, where `discriminator` is now `waf`. ### What is there Heuristics in `internal/waf` tell each other things through the JSON log line, `rq.line`, and through flags on two structs. As writer, then reader: - `rq.line.Tags`: written through `addTag` by `tagLists`, `routeWatchTags`, `noteLookup`, `dnsbl`, `abuseIPDB`, `ruleFiles` and `coreRuleSet`. Read by `tagDeny` and by `limitPercentages` in `biased.go`. - `rq.line.ASN`, `rq.line.ASName`, `rq.line.Country`: written by `noteLookup`. Read by `countries`, `limitPercentages`, `allow`, the two functions that make bans, and the alerts. `decision.lookupAnswer` holds the same three values and is not what they read. - `rq.line.WouldAction`: written by `check`. Read by `byteLimits` and `refusalLimit`. - `rq.line.Offence`: written by `banForLimit`. Read by `addToHistory`. - `rq.line.BanExpires`: written by `activeBan`, which reads it back for its figures. - `rq.line.Method`, `rq.line.Host`, `rq.line.UserAgent`: read by `noted` for a ban's notes. - `rq.blocklisted`, `rq.dnsblListed`, `rq.abuseIPDBHit`, on `request`: written by `blocklists`, `dnsbl`, `abuseIPDB`. Read by `limitPercentages`. - `counted`, `rateLimitPercent`, `bytesLimitPercent`, `attack`, `blockedByBlockRule`, `blockedByCoreRuleSet`, on `decision`: these are already typed fields, and stay. About forty lines of `internal/waf` touch these fields of the log line. ### Why it matters - The log line doubles as working memory. A change to how a field is logged can change a decision. - The order of the lists carries meaning the lists do not show. `tagDeny` is listed twice because tags are added on both sides of it. `rateLimits` must come after the three reputation heuristics because it reads their flags. - A heuristic that has something to tell a later one needs a new field outside its file. ### What changes - `decision` holds what heuristics tell each other, as typed fields of its own: the tags, the lookup answer it already has, what would have been done, the offence, and the three reputation flags. The comment on each field names what sets it. - The log line is filled from `decision`, and nothing reads it back to decide, to make a ban's notes or to raise an alert. Before (`tagdeny.go`, `bytelimits.go`): ```go for _, tag := range rq.line.Tags { // ... if !rq.counted || rq.line.WouldAction != "" { ``` After (the field names are placeholders): ```go for _, tag := range rq.tags { // ... if !rq.counted || rq.wouldRefuse { ``` ### What stays - Every field of the log line, with the same value at the same moment: both lines of a request, the decision's and the report's. - What is held for the report (`holdForReport`) still carries all the report's steps need. - The order of the lists. ### Done when - no code in `internal/waf` reads a field of `rq.line` to decide, to fill a ban's notes or to raise an alert. Reading `RequestID` to key what is held for the report stays; - each fact a heuristic leaves for a later one is a field of `decision` whose comment names what sets it; - behaviour is unchanged: every existing test passes with no assertion changed; - `make check` passes; - the PR body lists every new field name. Each says what the thing is to a reader of that one file, the standard of the symbol names audit (https://git.eeqj.de/sneak/smallwebwaf/issues/213). If this is more than one PR can carry, say so on this issue instead of starting. Model: fable-5-2
clawbot self-assigned this 2026-11-19 17:13:39 +01:00
clawbot added the needs-decision label 2026-11-19 18:23:38 +01:00
Author
Collaborator

Deferred, not started: this issue starts after the issue that gives each heuristic its own settings and parts (#441) closes, and its PR (#462) is still open and unmerged.

Model: opus-5-5

Deferred, not started: this issue starts after the issue that gives each heuristic its own settings and parts (https://git.eeqj.de/sneak/smallwebwaf/issues/441) closes, and its PR (https://git.eeqj.de/sneak/smallwebwaf/pulls/462) is still open and unmerged. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/smallwebwaf#442