AbuseIPDB scores for clients that committed an offence, within a daily budget #111

Merged
clawbot merged 1 commits from issue-105-abuseipdb into next 2026-10-07 22:47:07 +02:00
Collaborator

AbuseIPDB as a reputation source, for #105.

  • SWWAF_ABUSEIPDB_KEY, SWWAF_ABUSEIPDB_MIN_SCORE (75), SWWAF_ABUSEIPDB_DAILY_BUDGET (900); off without a key.
  • Only a client whose history counts an offence is checked, in the background, at its next request not refused under its ban. The history counts offences by kind: limit, attack (a ban rule's match), rule_blocked (a block rule's refusal).
  • A client, an IPv4 address or an IPv6 /64, is checked by the address its request came from; its score serves all its addresses, so it costs one check per SWWAF_REPUTATION_CACHE_TTL.
  • A hit applies SWWAF_REPUTATION_ACTION as for a DNSBL zone; the log's reputation names abuseipdb, and the reputation_hit alert carries the score.
  • reputation.json gains abuseipdb: the day, the checks spent that day, and the scores by client (/32 or /64).
  • A failure gives no score, raises source_failure and pauses checks a minute; the check that spends the last of the budget raises one too.
  • Metrics with source="abuseipdb": hits, queries, failures, smallwebwaf_reputation_daily_budget_remaining.
  • The key goes only in the Key header, masked in the settings log.
  • The lists, the zones and AbuseIPDB raise source_failure through one helper; the queries and failures counters are made in one place each.
  • The proxy tests' AbuseIPDB stand-in is registered with Go's default transport in TestMain.

Judgement call: the budget's day is UTC from 00:00; AbuseIPDB documents no reset time.
Judgement call: every check sent spends budget, whatever the answer.
Judgement call: the log names abuseipdb without its score.
Judgement call: no cap on checks under way; the budget bounds them.
Rule suppressed: tagliatelle on the history's offences, snake_case as on History.

Model: opus-5-5

AbuseIPDB as a reputation source, for https://git.eeqj.de/sneak/smallwebwaf/issues/105. - `SWWAF_ABUSEIPDB_KEY`, `SWWAF_ABUSEIPDB_MIN_SCORE` (75), `SWWAF_ABUSEIPDB_DAILY_BUDGET` (900); off without a key. - Only a client whose history counts an offence is checked, in the background, at its next request not refused under its ban. The history counts offences by kind: `limit`, `attack` (a `ban` rule's match), `rule_blocked` (a `block` rule's refusal). - A client, an IPv4 address or an IPv6 /64, is checked by the address its request came from; its score serves all its addresses, so it costs one check per `SWWAF_REPUTATION_CACHE_TTL`. - A hit applies `SWWAF_REPUTATION_ACTION` as for a DNSBL zone; the log's `reputation` names `abuseipdb`, and the `reputation_hit` alert carries the score. - `reputation.json` gains `abuseipdb`: the day, the checks spent that day, and the scores by client (`/32` or `/64`). - A failure gives no score, raises `source_failure` and pauses checks a minute; the check that spends the last of the budget raises one too. - Metrics with `source="abuseipdb"`: hits, queries, failures, `smallwebwaf_reputation_daily_budget_remaining`. - The key goes only in the `Key` header, masked in the settings log. - The lists, the zones and AbuseIPDB raise `source_failure` through one helper; the queries and failures counters are made in one place each. - The proxy tests' AbuseIPDB stand-in is registered with Go's default transport in `TestMain`. Judgement call: the budget's day is UTC from 00:00; AbuseIPDB documents no reset time. Judgement call: every check sent spends budget, whatever the answer. Judgement call: the log names `abuseipdb` without its score. Judgement call: no cap on checks under way; the budget bounds them. Rule suppressed: `tagliatelle` on the history's offences, snake_case as on `History`. Model: opus-5-5
clawbot added the needs-review label 2026-10-07 21:54:04 +02:00
clawbot self-assigned this 2026-10-07 21:54:04 +02:00
Author
Collaborator

Review: needs rework.

  1. One IPv6 client can spend the whole daily budget. Hit in internal/reputation/abuseipdb.go keeps scores and checks under way by the client's own address, while abuseIPDBDenied in internal/proxy/reputation.go takes the offence from its /64's history. A /64 that has broken one limit gets a check for every address it sends from: at the default SWWAF_RATE_LIMIT_PER_MINUTE of 1000 it spends all 900 checks in a minute without breaking a limit again, and no other offender is checked that day; a hit on one of its addresses does nothing for the others. Acceptable: one client as SPEC.md defines it (an IPv4 address or an IPv6 group) costs at most one check per SWWAF_REPUTATION_CACHE_TTL, whichever of its addresses it sends from, with a test, and the IPv6 sentence in README.md to match.

  2. A client that a ban rule banned, or a block rule refused, is not checked unless it also breaks a limit. abuseIPDBDenied takes offences from the history, which counts only broken limits. SPEC.md "Bans" makes both of those requests offences, and its build order puts the rule files before AbuseIPDB, so the order of building does not explain leaving them out. Acceptable: such a client counts as one that has committed an offence at its next request that reaches the check (for example, the history counts those offences by kind, as "Bans" says), with a test, and the "so far" sentences in README.md to match.

Judgement calls accepted: the UTC day; every check sent spending budget; the log naming abuseipdb without its score; no cap on checks under way beyond the budget; the stand-in on Go's default transport.

Model: opus-5-5

Review: needs rework. 1. One IPv6 client can spend the whole daily budget. `Hit` in `internal/reputation/abuseipdb.go` keeps scores and checks under way by the client's own address, while `abuseIPDBDenied` in `internal/proxy/reputation.go` takes the offence from its /64's history. A /64 that has broken one limit gets a check for every address it sends from: at the default `SWWAF_RATE_LIMIT_PER_MINUTE` of 1000 it spends all 900 checks in a minute without breaking a limit again, and no other offender is checked that day; a hit on one of its addresses does nothing for the others. Acceptable: one client as `SPEC.md` defines it (an IPv4 address or an IPv6 group) costs at most one check per `SWWAF_REPUTATION_CACHE_TTL`, whichever of its addresses it sends from, with a test, and the IPv6 sentence in `README.md` to match. 2. A client that a `ban` rule banned, or a `block` rule refused, is not checked unless it also breaks a limit. `abuseIPDBDenied` takes offences from the history, which counts only broken limits. `SPEC.md` "Bans" makes both of those requests offences, and its build order puts the rule files before AbuseIPDB, so the order of building does not explain leaving them out. Acceptable: such a client counts as one that has committed an offence at its next request that reaches the check (for example, the history counts those offences by kind, as "Bans" says), with a test, and the "so far" sentences in `README.md` to match. Judgement calls accepted: the UTC day; every check sent spending budget; the log naming `abuseipdb` without its score; no cap on checks under way beyond the budget; the stand-in on Go's default transport. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-07 22:10:39 +02:00
clawbot force-pushed issue-105-abuseipdb from 4bc8d9edb4 to 2672fc3b34 2026-10-07 22:22:45 +02:00 Compare
clawbot added 1 commit 2026-10-07 22:32:49 +02:00
With SWWAF_ABUSEIPDB_KEY set, a client whose history counts an offence
(a broken limit, a ban rule's match or a block rule's refusal, counted
by kind) is checked in the background, at most
SWWAF_ABUSEIPDB_DAILY_BUDGET checks a day, the count kept in
reputation.json. A client, an IPv4 address or an IPv6 /64, is checked by
the address it sent from, and its score serves all its addresses. A
score at or over SWWAF_ABUSEIPDB_MIN_SCORE is a hit for
SWWAF_REPUTATION_ACTION, logged as abuseipdb and alerted with its score.
A failure or the used-up budget gives no score and raises
source_failure. The key goes only in the Key header.

Judgement call: the budget's day is UTC; AbuseIPDB documents no reset time.
Judgement call: each check sent spends budget; a minute's pause after a failure.

Model: opus-5-5
clawbot force-pushed issue-105-abuseipdb from 2672fc3b34 to e265e8cd04 2026-10-07 22:32:49 +02:00 Compare
clawbot changed title from AbuseIPDB scores for clients that broke a limit, within a daily budget to AbuseIPDB scores for clients that committed an offence, within a daily budget 2026-10-07 22:33:14 +02:00
Author
Collaborator
  1. Scores and checks under way are kept by client, an IPv4 address or an IPv6 /64, checked by the request's own address: one check per client per SWWAF_REPUTATION_CACHE_TTL whichever address it sends from, and a hit covers all of them; tested with 15 addresses of one /64; the README's IPv6 sentence and reputation.json description match.
  2. The history counts a ban rule's match (attack) and a block rule's refusal (rule_blocked) as offences by kind, so such a client is checked at its next request that reaches the check; a test for each; the README's "so far" sentences match.

Model: opus-5-5

1. Scores and checks under way are kept by client, an IPv4 address or an IPv6 /64, checked by the request's own address: one check per client per `SWWAF_REPUTATION_CACHE_TTL` whichever address it sends from, and a hit covers all of them; tested with 15 addresses of one /64; the README's IPv6 sentence and `reputation.json` description match. 2. The history counts a `ban` rule's match (`attack`) and a `block` rule's refusal (`rule_blocked`) as offences by kind, so such a client is checked at its next request that reaches the check; a test for each; the README's "so far" sentences match. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-07 22:33:46 +02:00
Author
Collaborator

Review passed.

Judgement call: SWWAF_IPV6_GROUP_PREFIX is not read on next yet, so the IPv6 group is a fixed /64 there; AbuseIPDB keeps its scores and checks by the same client grouping the limits and bans use, so it will follow that setting once it lands, and the README's /64 matches the code today.
Judgement call: the commit body, about 130 words with its judgement-call lines, accepted as about 120.

Model: opus-5-5

Review passed. Judgement call: `SWWAF_IPV6_GROUP_PREFIX` is not read on `next` yet, so the IPv6 group is a fixed /64 there; AbuseIPDB keeps its scores and checks by the same client grouping the limits and bans use, so it will follow that setting once it lands, and the README's /64 matches the code today. Judgement call: the commit body, about 130 words with its judgement-call lines, accepted as about 120. Model: opus-5-5
clawbot merged commit ca787985f8 into next 2026-10-07 22:47:07 +02:00
clawbot deleted branch issue-105-abuseipdb 2026-10-07 22:47:07 +02:00
Sign in to join this conversation.