waf: ban the clients on AbuseIPDB's blacklist (closes #411)
check / check (push) Failing after 6m25s

While SWW_WAF_ABUSEIPDB_KEY is set, waf fetches AbuseIPDB's blacklist at
the start and every SWW_WAF_ABUSEIPDB_BLACKLIST_REFRESH_INTERVAL_MINS
(60), cut at SWW_WAF_ABUSEIPDB_THRESHOLD (90, from 25 to 100). Each
listed client's netblock gets a ban of SWW_WAF_ABUSEIPDB_BAN_DAYS (30)
with the new cause abuseipdb. The blacklist itself is not kept.

These bans do not count toward SWW_WAF_MAX_BANS, so a 10,000-address
blacklist cannot push out the bans for limits and attacks. Each fetch
forgets the abuseipdb bans that have ended. A netblock with an active
ban is left alone, and so is one whose abuseipdb ban an admin lifted,
until that ban would have ended. Observe mode makes no ban.

Model: opus-5-5
This commit is contained in:
clawbot committed 2026-11-19 07:59:24 +00:00
1 parent dadd5768d5
commit e1ed444e68
20 files changed
+1162 -179

No files matched your search

+31 -7
View File
@@ -92,7 +92,8 @@ never as a time.
to count and report failures.
- `internal/lookup`: looks clients up in the lookup database.
- `internal/reputation`: fetches the lists, asks the DNS blocklists and
AbuseIPDB in the background, and keeps their answers.
AbuseIPDB in the background, and keeps their answers. It also fetches
AbuseIPDB's blacklist, which `internal/waf` bans.
- `internal/routewatch`: asks a routewatch instance about clients in the
background, and keeps its answers.
- `internal/anomaly`: the anomaly counters and their alerts.
@@ -299,6 +300,9 @@ to a ban, each by its own rule:
Once it ends, only new requests can break a limit again. The client's
history keeps its totals.
A client on AbuseIPDB's blacklist is banned before it sends a request, with the
cause `abuseipdb` (see "Reputation").
A Core Rule Set match refuses only the request it matched, with `403` and
without a ban, because the Core Rule Set has false positives. A `block` rule
does the same. Those refusals count toward the refusal limit, so a client that
@@ -325,11 +329,12 @@ does not count toward a longer ban; it refuses nothing. Deleting its entry from
A permanent ban does not run out. It ends when an admin lifts it, or when `waf`
drops it from a full table, as below. When `waf` holds `SWW_WAF_MAX_BANS` bans
of its own, it drops the earliest ban of one netblock: the netblock that has
gone longest without a request. A past, active or permanent ban is dropped
alike. A scanner whose ban was dropped that way is refused and banned again by
its next probe. A ban whose cause is `admin` is never dropped and does not count
toward `SWW_WAF_MAX_BANS`, so such bans can never fill the table.
for broken limits and clear signs of attack, it drops the earliest such ban of
one netblock: the netblock that has gone longest without a request. A past,
active or permanent ban is dropped alike. A scanner whose ban was dropped that
way is refused and banned again by its next probe. Bans whose cause is `admin`
or `abuseipdb` do not count toward `SWW_WAF_MAX_BANS` and are never dropped to
make room, so they can never push out the others.
A request refused under a ban that runs out is logged with the action
`temporarily_banned`. Under a permanent ban the action is `permanently_banned`.
@@ -2036,6 +2041,25 @@ host, sharing one address, would often fetch too close together.
- AbuseIPDB answers other than `200`;
- the answer gives no score;
- no answer comes within `SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`.
- While `SWW_WAF_ABUSEIPDB_KEY` is set, AbuseIPDB's blacklist is fetched at the
start and every `SWW_WAF_ABUSEIPDB_BLACKLIST_REFRESH_INTERVAL_MINS` minutes
(default `60`). It lists the clients AbuseIPDB scores at
`SWW_WAF_ABUSEIPDB_THRESHOLD` (default `90`) or more. The blacklist itself is
not kept.
- Each client on it is banned for `SWW_WAF_ABUSEIPDB_BAN_DAYS` days (default
`30`), with the cause `abuseipdb`. The ban covers the client's netblock,
as any ban does.
- A netblock that already has an active ban is left as it is. So is one
whose `abuseipdb` ban an admin lifted, until that ban would have ended.
- Each fetch first forgets the `abuseipdb` bans that have ended, so that
`bans.json` holds only the clients listed within one ban's length.
- In `observe` mode the blacklist makes no ban.
- The threshold takes 25 to 100. It should not be set below 50 to 75, since
lower scores list ordinary visitors. AbuseIPDB serves free accounts the
blacklist at the threshold 100, whatever is asked for.
- AbuseIPDB limits the fetches of the blacklist a day: 5 for a free account.
A fetch over the limit fails, and is logged and alerted. The default
refresh fetches it 24 times a day.
- `SWW_WAF_DNSBL_ABUSEIPDB_ACTION`, for DNS blocklist and AbuseIPDB hits, is
`limit:25` by default. Such verdicts are less certain than a blocklist, so a
listed client gets a quarter of every request and byte limit rather than a
@@ -2200,7 +2224,7 @@ lists the files, what each holds, and their sizes.
these stop the process with a message naming the file and position:
- a file that does not parse, or has an unknown `version`;
- an entry without a field it needs;
- a ban whose cause is not one of the four;
- a ban whose cause is not `limit`, `attack`, `admin` or `abuseipdb`;
- a ban whose `expires` is not after its `start`.
Starting with empty state would silently forgive every repeat offender.