Ban notes name the reputation sources that listed the client (closes #109)
check / check (push) Waiting to run

A ban's notes, in bans.json and in its alert, gain `reputation`: each
blocklist, DNSBL zone or AbuseIPDB that listed the client when the ban
was made, as its `source`, named and ordered as in the request log's
`reputation`, with AbuseIPDB's `score`. It is left out when none did.
README.md shows it in a bans.json example.

Notes now hold a list, so bans can no longer be compared with ==: the
tests compare them with reflect.DeepEqual.

Judgement call: the score is a pointer, so a score of 0, a hit while
SWWAF_ABUSEIPDB_MIN_SCORE is 0, is still written.

Model: opus-5-5
This commit was merged in pull request #114.
This commit is contained in:
2026-10-08 03:08:01 +02:00
parent a6634454cd
commit 04e66d2069
16 changed files with 243 additions and 68 deletions
+75 -16
View File
@@ -163,15 +163,16 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
and the requests or bytes counted in it, the client's percentage of that kind and the requests or bytes counted in it, the client's percentage of that kind
of limit and the setting that gave it when a biased threshold lowered the of limit and the setting that gave it when a biased threshold lowered the
limit, the request that broke it, the client's AS number, AS name and country limit, the request that broke it, the client's AS number, AS name and country
once they are looked up, the netblock's requests since it was first seen, how once they are looked up, the blocklists, DNSBL zones and AbuseIPDB, with its
many of them the ban has refused, and how many bans the netblock had before, score, that listed the client when the ban was made, the netblock's requests
for a broken limit, for a clear sign of attack and by an admin. At most since it was first seen, how many of them the ban has refused, and how many
`SWWAF_MAX_BANS` bans `smallwebwaf` made are kept, past, active and permanent; bans the netblock had before, for a broken limit, for a clear sign of attack
past that, the earliest such ban of the netblock that has gone longest without and by an admin. At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept,
a request is dropped first. The bans whose cause is `admin`, those you make or past, active and permanent; past that, the earliest such ban of the netblock
keep, are kept besides, and never dropped. `bans.json` shows the bans and that has gone longest without a request is dropped first. The bans whose cause
their notes, a restart lifts none, and you make, keep or lift a ban by editing is `admin`, those you make or keep, are kept besides, and never dropped.
it (see "State files" below). `bans.json` shows the bans and their notes, a restart lifts none, and you
make, keep or lift a ban by editing it (see "State files" below).
- Checks each request against the rules of the rule files (see "Rule files" - Checks each request against the rules of the rule files (see "Rule files"
below) after the rate limits, and before its body is read. A `log` rule that below) after the rate limits, and before its body is read. A `log` rule that
matches is noted in the log line; a `block` rule refuses the request with matches is noted in the log line; a `block` rule refuses the request with
@@ -1074,7 +1075,11 @@ with times in UTC.
ban for a broken limit is `requests` or `bytes`, what the limit is on. For a ban for a broken limit is `requests` or `bytes`, what the limit is on. For a
limit a biased threshold lowered, the reason and the notes' `limit` give the limit a biased threshold lowered, the reason and the notes' `limit` give the
lowered limit, and the notes' `limit_percent` and `limit_percent_setting` the lowered limit, and the notes' `limit_percent` and `limit_percent_setting` the
client's percentage of that kind of limit and the setting that gave it. client's percentage of that kind of limit and the setting that gave it. The
notes' `reputation` gives each blocklist, DNSBL zone or AbuseIPDB that listed
the client when the ban was made, as its `source`, named and ordered as in the
request log's `reputation`, with AbuseIPDB's `score` of the client. It is left
out when none did, and the example below shows it.
- `clients.json`: each client's two buckets of requests in the minute, the hour - `clients.json`: each client's two buckets of requests in the minute, the hour
and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes` and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes`
and `day_bytes`, and its history: when it was first and last seen, its AS and `day_bytes`, and its history: when it was first and last seen, its AS
@@ -1121,6 +1126,60 @@ with times in UTC.
Slack and ntfy too, stops the start: put the list under `"webhook"`, or remove Slack and ntfy too, stops the start: put the list under `"webhook"`, or remove
the file. the file.
This `bans.json` holds a ban for a broken rate limit on a client that a DNSBL
zone lists and AbuseIPDB scores at 100, whose limits `SWWAF_REPUTATION_ACTION`,
at its default of `limit:25`, lowered to a quarter:
```json
{
"version": 1,
"bans": [
{
"netblock": "203.0.113.9/32",
"start": "2026-10-06T12:00:41.5Z",
"expires": "2026-10-06T13:00:41.5Z",
"cause": "limit",
"reason": "requests per minute over the limit of 250",
"notes": {
"asn": "AS64496",
"as_name": "Example Net",
"country": "DE",
"kind": "requests",
"limit": 250,
"window": "minute",
"count": 251,
"limit_percent": 25,
"limit_percent_setting": "SWWAF_REPUTATION_ACTION",
"reputation": [
{
"source": "dnsbl.dronebl.org"
},
{
"source": "abuseipdb",
"score": 100
}
],
"request": {
"time": "2026-10-06T12:00:41.5Z",
"method": "GET",
"host": "app.example",
"path": "/owner/repo/commits/branch/main?page=812",
"status": 403,
"user_agent": "scraper/1.0"
},
"requests": 512,
"refused": 0,
"earlier_bans": {
"limit": 0,
"attack": 0,
"admin": 0
}
}
}
]
}
```
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted `bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted
through `DELETE /_smallwebwaf/bans/<client>`, or made permanent, with every such through `DELETE /_smallwebwaf/bans/<client>`, or made permanent, with every such
change in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and when change in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and when
@@ -1759,8 +1818,8 @@ fetched before then stays in use, and the failure is counted, logged and raised
as a `source_failure` alert; a fetch cut off as `smallwebwaf` stops is not a as a `source_failure` alert; a fetch cut off as `smallwebwaf` stops is not a
failure. The last good copy of each list is kept whole, comment lines included, failure. The last good copy of each list is kept whole, comment lines included,
in `reputation.json` (see "State files" above), so that a restart keeps it in in `reputation.json` (see "State files" above), so that a restart keeps it in
use too. Each list is named by its URL, in the request log, the alerts and the use too. Each list is named by its URL, in the request log, the alerts, the
metrics, so keep a secret out of it. notes of bans and the metrics, so keep a secret out of it.
A client in `SWWAF_ALLOW_NETS` is not checked. Any other is checked by its own A client in `SWWAF_ALLOW_NETS` is not checked. Any other is checked by its own
address after the country lists, and `SWWAF_BLOCKLIST_ACTION` says what is done address after the country lists, and `SWWAF_BLOCKLIST_ACTION` says what is done
@@ -1829,10 +1888,10 @@ it, such as `<key>.xbl.dq.spamhaus.net`. The key of a zone under
`dq.spamhaus.net` is its first label, and `smallwebwaf` shows `********` in its `dq.spamhaus.net` is its first label, and `smallwebwaf` shows `********` in its
place wherever it names the zone, as `********.xbl.dq.spamhaus.net`: in the place wherever it names the zone, as `********.xbl.dq.spamhaus.net`: in the
settings logged at start, an error that stops the start, its own messages, the settings logged at start, an error that stops the start, its own messages, the
request log, the alerts and the metrics. Only `reputation.json` keeps the zone request log, the alerts, the notes of bans and the metrics. Only
with its key. A key in the name of any other zone is shown as given. Several `reputation.json` keeps the zone with its key. A key in the name of any other
zones refuse queries that come through a public resolver; `SWWAF_DNSBL_RESOLVER` zone is shown as given. Several zones refuse queries that come through a public
names another resolver to ask through. resolver; `SWWAF_DNSBL_RESOLVER` names another resolver to ask through.
## AbuseIPDB ## AbuseIPDB
+4 -3
View File
@@ -2,6 +2,7 @@ package bans_test
import ( import (
"net/netip" "net/netip"
"reflect"
"testing" "testing"
"time" "time"
@@ -117,7 +118,7 @@ func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
} }
held := ledger.Bans(netblock) held := ledger.Bans(netblock)
if len(held) != 2 || held[0] != lifted { if len(held) != 2 || !reflect.DeepEqual(held[0], lifted) {
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held) t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
} }
} }
@@ -212,7 +213,7 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{}, got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{},
"probes for logins") "probes for logins")
if got != want { if !reflect.DeepEqual(got, want) {
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want) t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
} }
@@ -224,7 +225,7 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
// It refuses once the ban for the limit has ended. // It refuses once the ban for the limit has ended.
ban, banned, _ := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour)) ban, banned, _ := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour))
if !banned || ban != want { if !banned || !reflect.DeepEqual(ban, want) {
t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v", t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v",
ban, banned, want) ban, banned, want)
} }
+13
View File
@@ -118,6 +118,10 @@ type Notes struct {
// of the rule file rule that matched, and its target. // of the rule file rule that matched, and its target.
RuleID string `json:"rule_id,omitempty"` RuleID string `json:"rule_id,omitempty"`
Target string `json:"target,omitempty"` Target string `json:"target,omitempty"`
// Reputation is the reputation sources that listed the client when
// the request that caused the ban was made, in the order the request
// log's reputation names them. It is left out when none did.
Reputation []ReputationHit `json:"reputation,omitempty"`
// Request is the request that broke the limit, or whose bytes broke // Request is the request that broke the limit, or whose bytes broke
// it, or that was the clear sign of attack. // it, or that was the clear sign of attack.
Request Request `json:"request"` Request Request `json:"request"`
@@ -131,6 +135,15 @@ type Notes struct {
EarlierBans EarlierBans `json:"earlier_bans"` EarlierBans EarlierBans `json:"earlier_bans"`
} }
// ReputationHit is a reputation source that listed a client, as a
// reputation_hit alert's detail gives it: Source is the blocklist's URL,
// the DNSBL zone with its key masked, or "abuseipdb", and Score, for
// AbuseIPDB alone, its score of the client.
type ReputationHit struct {
Source string `json:"source"`
Score *int64 `json:"score,omitempty"`
}
// EarlierBans counts a netblock's bans before a ban, by cause. // EarlierBans counts a netblock's bans before a ban, by cause.
type EarlierBans struct { type EarlierBans struct {
Limit int `json:"limit"` Limit int `json:"limit"`
+8 -7
View File
@@ -2,6 +2,7 @@ package bans_test
import ( import (
"net/netip" "net/netip"
"reflect"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -130,13 +131,13 @@ func TestBrokenLimitDuringABanMakesNoOther(t *testing.T) {
again, made := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{}) again, made := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
if made || again != first || len(ledger.Bans(netblock)) != 1 { if made || !reflect.DeepEqual(again, first) || len(ledger.Bans(netblock)) != 1 {
t.Errorf("a limit broken during a ban gave %+v, made %t, and %d bans, "+ t.Errorf("a limit broken during a ban gave %+v, made %t, and %d bans, "+
"want %+v, not made, and 1", again, made, len(ledger.Bans(netblock)), first) "want %+v, not made, and 1", again, made, len(ledger.Bans(netblock)), first)
} }
again, made = ledger.BanForAttack(netblock, midnight().Add(time.Minute), bans.Notes{}) again, made = ledger.BanForAttack(netblock, midnight().Add(time.Minute), bans.Notes{})
if made || again != first { if made || !reflect.DeepEqual(again, first) {
t.Errorf("an attack during a ban gave %+v, made %t, want %+v, not made", t.Errorf("an attack during a ban gave %+v, made %t, want %+v, not made",
again, made, first) again, made, first)
} }
@@ -182,7 +183,7 @@ func TestFindCountsNothing(t *testing.T) {
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5}) ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
got, banned, _ := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond)) got, banned, _ := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
if !banned || got != ban { if !banned || !reflect.DeepEqual(got, ban) {
t.Errorf("find during the ban gives %+v and %t, want %+v", got, banned, ban) t.Errorf("find during the ban gives %+v and %t, want %+v", got, banned, ban)
} }
@@ -191,7 +192,7 @@ func TestFindCountsNothing(t *testing.T) {
t.Error("the ban did not end") t.Error("the ban did not end")
} }
if notes := ledger.Bans(netblock)[0].Notes; notes != ban.Notes { if notes := ledger.Bans(netblock)[0].Notes; !reflect.DeepEqual(notes, ban.Notes) {
t.Errorf("the notes are %+v, want them unchanged, %+v", notes, ban.Notes) t.Errorf("the notes are %+v, want them unchanged, %+v", notes, ban.Notes)
} }
} }
@@ -247,7 +248,7 @@ func TestFullLedgerDropsTheEarlierBanOfTheNetblockBannedAgain(t *testing.T) {
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{}) second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
held := ledger.Bans(netblock) held := ledger.Bans(netblock)
if len(held) != 1 || held[0] != second || if len(held) != 1 || !reflect.DeepEqual(held[0], second) ||
held[0].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) { held[0].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("the ledger holds %+v, want only the second ban, "+ t.Errorf("the ledger holds %+v, want only the second ban, "+
"with 1 earlier ban for a limit", held) "with 1 earlier ban for a limit", held)
@@ -342,7 +343,7 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
// While the first ban lasts, none would be made. // While the first ban lasts, none would be made.
during, would := ledger.WouldBanForAttack(netblock, midnight(), bans.Notes{}) during, would := ledger.WouldBanForAttack(netblock, midnight(), bans.Notes{})
if would || during != first { if would || !reflect.DeepEqual(during, first) {
t.Errorf("during the first ban, would ban %t with %+v, want false with %+v", t.Errorf("during the first ban, would ban %t with %+v, want false with %+v",
would, during, first) would, during, first)
} }
@@ -371,7 +372,7 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
// The ban made is the one that would have been. // The ban made is the one that would have been.
made, _ := ledger.BanForLimit(netblock, first.Expires, limitNotes) made, _ := ledger.BanForLimit(netblock, first.Expires, limitNotes)
if made != limit { if !reflect.DeepEqual(made, limit) {
t.Errorf("the ban made is %+v, want %+v", made, limit) t.Errorf("the ban made is %+v, want %+v", made, limit)
} }
} }
+3 -2
View File
@@ -2,6 +2,7 @@ package bans_test
import ( import (
"net/netip" "net/netip"
"reflect"
"slices" "slices"
"strings" "strings"
"testing" "testing"
@@ -224,7 +225,7 @@ func TestLoadKeepsAtMostMaxBansDroppingTheEarliest(t *testing.T) {
ledger.Load([]bans.Ban{later, earlier}) ledger.Load([]bans.Ban{later, earlier})
held := ledger.Snapshot() held := ledger.Snapshot()
if len(held) != 1 || held[0] != later { if len(held) != 1 || !reflect.DeepEqual(held[0], later) {
t.Errorf("the ledger holds %+v, want only the ban that began later", held) t.Errorf("the ledger holds %+v, want only the ban that began later", held)
} }
} }
@@ -267,7 +268,7 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
bans.Notes{}) bans.Notes{})
want := []bans.Ban{first, second, kept} want := []bans.Ban{first, second, kept}
if got := ledger.Snapshot(); !slices.Equal(got, want) { if got := ledger.Snapshot(); !reflect.DeepEqual(got, want) {
t.Errorf("the ledger holds %+v, want %+v", got, want) t.Errorf("the ledger holds %+v, want %+v", got, want)
} }
} }
+3 -3
View File
@@ -4,7 +4,7 @@ import (
"encoding/json" "encoding/json"
"net/http" "net/http"
"net/netip" "net/netip"
"slices" "reflect"
"strconv" "strconv"
"strings" "strings"
"testing" "testing"
@@ -48,7 +48,7 @@ func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
} }
} }
if after := server.Ledger.Snapshot(); !slices.Equal(after, before) { if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before) t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before)
} }
} }
@@ -79,7 +79,7 @@ func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
} }
} }
if after := server.Ledger.Snapshot(); !slices.Equal(after, before) { if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v", t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
e.method, e.path, after, before) e.method, e.path, after, before)
} }
+2 -1
View File
@@ -118,7 +118,8 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
line := s.get(ipv6Client, http.StatusOK, requestlog.ActionForward) line := s.get(ipv6Client, http.StatusOK, requestlog.ActionForward)
// No ban is made, and none made permanent. // No ban is made, and none made permanent.
if held := server.Ledger.Snapshot(); len(held) != 1 || held[0] != attackBan || held := server.Ledger.Snapshot()
if len(held) != 1 || !reflect.DeepEqual(held[0], attackBan) ||
line.BanExpires != requestlog.FormatTime(attackBan.Expires) { line.BanExpires != requestlog.FormatTime(attackBan.Expires) {
t.Errorf("the ledger holds %+v, and the log line gives %s, want the ban "+ t.Errorf("the ledger holds %+v, and the log line gives %s, want the ban "+
"for the attack alone, as it was", held, line.BanExpires) "for the attack alone, as it was", held, line.BanExpires)
+18 -16
View File
@@ -127,15 +127,16 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
} }
notes := bans.Notes{ notes := bans.Notes{
ASN: rq.line.ASN, ASN: rq.line.ASN,
ASName: rq.line.ASName, ASName: rq.line.ASName,
Country: rq.line.Country, Country: rq.line.Country,
Kind: hit.Kind, Kind: hit.Kind,
Limit: hit.Limit, Limit: hit.Limit,
Window: hit.Window, Window: hit.Window,
Count: hit.Count, Count: hit.Count,
Request: rq.noted(now, status), Reputation: rq.reputation,
Requests: rq.netblockRequests(netblock), Request: rq.noted(now, status),
Requests: rq.netblockRequests(netblock),
} }
percent := rq.limitPercent percent := rq.limitPercent
@@ -174,13 +175,14 @@ func (rq *request) banForAttack(now time.Time, rule rules.Rule) {
} }
notes := bans.Notes{ notes := bans.Notes{
ASN: rq.line.ASN, ASN: rq.line.ASN,
ASName: rq.line.ASName, ASName: rq.line.ASName,
Country: rq.line.Country, Country: rq.line.Country,
RuleID: rule.ID, RuleID: rule.ID,
Target: rule.Target, Target: rule.Target,
Request: rq.noted(now, rq.h.config.BanResponse), Reputation: rq.reputation,
Requests: rq.netblockRequests(netblock), Request: rq.noted(now, rq.h.config.BanResponse),
Requests: rq.netblockRequests(netblock),
} }
if rq.h.config.Observe { if rq.h.config.Observe {
+2 -1
View File
@@ -7,6 +7,7 @@ import (
"maps" "maps"
"net/http" "net/http"
"net/netip" "net/netip"
"reflect"
"slices" "slices"
"sync" "sync"
"testing" "testing"
@@ -312,7 +313,7 @@ func TestBanNotes(t *testing.T) {
ledger := server.Ledger ledger := server.Ledger
got := ledger.Bans(netblock) got := ledger.Bans(netblock)
if len(got) != 1 || got[0] != want { if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
t.Fatalf("bans\n%+v\nwant\n%+v", got, want) t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
} }
+2 -1
View File
@@ -6,6 +6,7 @@ import (
"net" "net"
"net/http" "net/http"
"net/netip" "net/netip"
"reflect"
"strconv" "strconv"
"strings" "strings"
"testing" "testing"
@@ -337,7 +338,7 @@ func TestBanForABrokenByteLimitHasItsNotesAndItsAlert(t *testing.T) {
} }
got := server.Ledger.Bans(netblock) got := server.Ledger.Bans(netblock)
if len(got) != 1 || got[0] != want { if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
t.Fatalf("bans\n%+v\nwant\n%+v", got, want) t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
} }
+2 -1
View File
@@ -5,6 +5,7 @@ import (
"io" "io"
"net/http" "net/http"
"net/netip" "net/netip"
"reflect"
"sync/atomic" "sync/atomic"
"testing" "testing"
"time" "time"
@@ -126,7 +127,7 @@ func TestObserveModeMakesNoBanAndKeepsTheBansItHas(t *testing.T) {
} }
got := server.Ledger.Snapshot() got := server.Ledger.Snapshot()
if len(got) != 1 || got[0] != kept { if len(got) != 1 || !reflect.DeepEqual(got[0], kept) {
t.Errorf("bans\n%+v\nwant only\n%+v", got, kept) t.Errorf("bans\n%+v\nwant only\n%+v", got, kept)
} }
} }
+18 -12
View File
@@ -4,6 +4,7 @@ import (
"context" "context"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
) )
@@ -62,29 +63,34 @@ func (rq *request) abuseIPDBDenied(ctx context.Context) bool {
} }
rq.abuseIPDBHit = true rq.abuseIPDBHit = true
rq.noteHit(reputation.AbuseIPDBSource, "scored by AbuseIPDB at or over "+ rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
"SWWAF_ABUSEIPDB_MIN_SCORE", map[string]any{ "scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE")
"source": reputation.AbuseIPDBSource, "score": score,
})
return rq.h.config.ReputationAction == deny return rq.h.config.ReputationAction == deny
} }
// noteListed notes each of sources, the URLs of the blocklists or the // noteListed notes each of sources, the URLs of the blocklists or the
// DNSBL zones, their keys masked, that list the client, as noteHit does, // DNSBL zones, their keys masked, that list the client, as noteHit does,
// with reason, and the source in the alert's detail. // with reason.
func (rq *request) noteListed(sources []string, reason string) { func (rq *request) noteListed(sources []string, reason string) {
for _, source := range sources { for _, source := range sources {
rq.noteHit(source, reason, map[string]any{"source": source}) rq.noteHit(bans.ReputationHit{Source: source}, reason)
} }
} }
// noteHit adds source, which lists the client, to the log line's // noteHit adds hit's source, which lists the client, to the log line's
// reputation, counts it in the metrics, and raises a reputation_hit alert // reputation, and hit to the notes of a ban the request makes, counts the
// with reason and detail. // source in the metrics, and raises a reputation_hit alert with reason,
func (rq *request) noteHit(source, reason string, detail map[string]any) { // whose detail gives hit's source and score.
rq.line.Reputation = append(rq.line.Reputation, source) func (rq *request) noteHit(hit bans.ReputationHit, reason string) {
rq.h.metrics.ReputationHit(source) detail := map[string]any{"source": hit.Source}
if hit.Score != nil {
detail["score"] = *hit.Score
}
rq.line.Reputation = append(rq.line.Reputation, hit.Source)
rq.reputation = append(rq.reputation, hit)
rq.h.metrics.ReputationHit(hit.Source)
rq.h.alerts.Raise(alerts.Alert{ rq.h.alerts.Raise(alerts.Alert{
Event: alerts.EventReputationHit, Event: alerts.EventReputationHit,
Client: rq.client, Client: rq.client,
+69
View File
@@ -6,6 +6,7 @@ import (
"maps" "maps"
"net/http" "net/http"
"net/netip" "net/netip"
"reflect"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
@@ -13,6 +14,7 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
@@ -874,6 +876,73 @@ func TestWithoutAnAbuseIPDBKeyNoClientIsCheckedNorAScoreUsed(t *testing.T) {
`instance="`+alertInstance+`",source="`+abuseipdb+`"}`) `instance="`+alertInstance+`",source="`+abuseipdb+`"}`)
} }
func TestBanNotesNameEachReputationSourceThatListedTheClient(t *testing.T) {
t.Parallel()
score := int64(90)
listed := []bans.ReputationHit{
{Source: dropURL}, {Source: dnsblZone}, {Source: abuseipdb, Score: &score},
}
for _, tc := range []struct {
name string
// ban sends the requests from the client at from that ban it.
ban func(s *sender, from string)
}{
{"for a broken rate limit", func(s *sender, from string) {
s.get(from, http.StatusOK, requestlog.ActionForward)
s.get(from, http.StatusForbidden, requestlog.ActionRateLimited)
}},
{"for a clear sign of attack", func(s *sender, from string) {
s.request(from, probePath, http.StatusForbidden, requestlog.ActionBanned)
}},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
s, _, server, queue := startWithAlerts(t, map[string]string{
rateLimitPerMinute: "1", rulesDir: writeRules(t, testRules),
blocklistURLs: dropURL, blocklistAction: actionLog,
dnsblZones: dnsblZone, dnsblResolver: noResolver,
abuseIPDBKey: accountKey, reputationAction: actionLog,
})
// Every source lists client, and none otherClient, whose score is
// under SWWAF_ABUSEIPDB_MIN_SCORE, 75 by default.
loadLists(t, server, map[string][]string{dropURL: {client}})
loadVerdicts(server, map[string][]string{client: {dnsblZone}, otherClient: nil})
loadScores(server, map[string]int64{client: score, otherClient: 74})
for _, banned := range []struct {
from string
want []bans.ReputationHit
}{{client, listed}, {otherClient, nil}} {
tc.ban(s, banned.from)
held := server.Ledger.Bans(netip.MustParsePrefix(banned.from + "/32"))
if len(held) != 1 || !reflect.DeepEqual(held[0].Notes.Reputation, banned.want) {
t.Errorf("bans of %s %+v, want one whose notes have the reputation %+v",
banned.from, held, banned.want)
}
}
// The alert for each ban carries the same in its notes.
var alerted [][]bans.ReputationHit
for _, alert := range queue.Snapshot().Waiting[alerts.DestinationWebhook] {
if alert.Event == alerts.EventBan {
notes, _ := alert.Detail["notes"].(bans.Notes)
alerted = append(alerted, notes.Reputation)
}
}
if want := [][]bans.ReputationHit{listed, nil}; !reflect.DeepEqual(alerted, want) {
t.Errorf("the ban alerts' notes have the reputation %+v, want %+v",
alerted, want)
}
})
}
}
// listsFetched is when loadLists has the copies fetched. // listsFetched is when loadLists has the copies fetched.
func listsFetched() time.Time { func listsFetched() time.Time {
return time.Date(2026, 10, 5, 0, 0, 0, 0, time.UTC) return time.Date(2026, 10, 5, 0, 0, 0, 0, time.UTC)
+5 -1
View File
@@ -17,6 +17,7 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/anomaly" "sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup" "sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
@@ -71,7 +72,10 @@ type request struct {
// dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once // dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once
// AbuseIPDB's score of it is a hit. // AbuseIPDB's score of it is a hit.
blocklisted, dnsblListed, abuseIPDBHit bool blocklisted, dnsblListed, abuseIPDBHit bool
start time.Time // reputation is the reputation sources that list the client, for the
// notes of a ban the request makes.
reputation []bans.ReputationHit
start time.Time
// checked is when the checks were done, and upstreamStart when the // checked is when the checks were done, and upstreamStart when the
// request was handed to the app. // request was handed to the app.
checked time.Time checked time.Time
+2 -1
View File
@@ -5,6 +5,7 @@ import (
"net/netip" "net/netip"
"os" "os"
"path/filepath" "path/filepath"
"reflect"
"slices" "slices"
"testing" "testing"
"time" "time"
@@ -73,7 +74,7 @@ func TestEachRuleAction(t *testing.T) {
} }
got := server.Ledger.Bans(netblock) got := server.Ledger.Bans(netblock)
if len(got) != 1 || got[0] != want { if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
t.Fatalf("bans\n%+v\nwant\n%+v", got, want) t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
} }
+17 -3
View File
@@ -75,6 +75,15 @@ const permanentBansJSON = `{
"limit": 1000, "limit": 1000,
"window": "minute", "window": "minute",
"count": 1000.5, "count": 1000.5,
"reputation": [
{
"source": "https://lists.example/drop.txt"
},
{
"source": "abuseipdb",
"score": 100
}
],
"request": { "request": {
"time": "2026-10-06T00:00:00Z", "time": "2026-10-06T00:00:00Z",
"method": "GET", "method": "GET",
@@ -919,7 +928,7 @@ func TestBansWrittenOnceWriteDelayAfterABan(t *testing.T) {
load(t, read) load(t, read)
want := []bans.Ban{first, second} want := []bans.Ban{first, second}
if got := read.Ledger.Snapshot(); !slices.Equal(got, want) { if got := read.Ledger.Snapshot(); !reflect.DeepEqual(got, want) {
t.Errorf("bans.json holds %+v, want %+v", got, want) t.Errorf("bans.json holds %+v, want %+v", got, want)
} }
@@ -1808,6 +1817,8 @@ func fill(params state.Params) {
// permanentBan is the ban permanentBansJSON holds. // permanentBan is the ban permanentBansJSON holds.
func permanentBan() bans.Ban { func permanentBan() bans.Ban {
score := int64(100)
return bans.Ban{ return bans.Ban{
Netblock: netip.MustParsePrefix("2001:db8::/64"), Netblock: netip.MustParsePrefix("2001:db8::/64"),
Start: midnight(), Start: midnight(),
@@ -1820,6 +1831,9 @@ func permanentBan() bans.Ban {
Limit: 1000, Limit: 1000,
Window: "minute", Window: "minute",
Count: 1000.5, Count: 1000.5,
Reputation: []bans.ReputationHit{
{Source: blocklistURL}, {Source: reputation.AbuseIPDBSource, Score: &score},
},
Request: bans.Request{ Request: bans.Request{
Time: midnight(), Time: midnight(),
Method: "GET", Method: "GET",
@@ -1995,10 +2009,10 @@ func edit(t *testing.T, dir, name, content string) {
// wantEqual checks that the entries read back from file are those // wantEqual checks that the entries read back from file are those
// written. // written.
func wantEqual[E comparable](t *testing.T, file string, got, want []E) { func wantEqual[E any](t *testing.T, file string, got, want []E) {
t.Helper() t.Helper()
if !slices.Equal(got, want) { if !reflect.DeepEqual(got, want) {
t.Errorf("%s read back\n%+v\nwant\n%+v", file, got, want) t.Errorf("%s read back\n%+v\nwant\n%+v", file, got, want)
} }
} }