Ban notes name the reputation sources that listed the client (closes #109)
check / check (push) Waiting to run
check / check (push) Waiting to run
A ban's notes, in bans.json and in its alert, gain `reputation`: each blocklist, DNSBL zone or AbuseIPDB that listed the client when the ban was made, as its `source`, named and ordered as in the request log's `reputation`, with AbuseIPDB's `score`. It is left out when none did. README.md shows it in a bans.json example. Notes now hold a list, so bans can no longer be compared with ==: the tests compare them with reflect.DeepEqual. Judgement call: the score is a pointer, so a score of 0, a hit while SWWAF_ABUSEIPDB_MIN_SCORE is 0, is still written. Model: opus-5-5
This commit was merged in pull request #114.
This commit is contained in:
@@ -163,15 +163,16 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
|||||||
and the requests or bytes counted in it, the client's percentage of that kind
|
and the requests or bytes counted in it, the client's percentage of that kind
|
||||||
of limit and the setting that gave it when a biased threshold lowered the
|
of limit and the setting that gave it when a biased threshold lowered the
|
||||||
limit, the request that broke it, the client's AS number, AS name and country
|
limit, the request that broke it, the client's AS number, AS name and country
|
||||||
once they are looked up, the netblock's requests since it was first seen, how
|
once they are looked up, the blocklists, DNSBL zones and AbuseIPDB, with its
|
||||||
many of them the ban has refused, and how many bans the netblock had before,
|
score, that listed the client when the ban was made, the netblock's requests
|
||||||
for a broken limit, for a clear sign of attack and by an admin. At most
|
since it was first seen, how many of them the ban has refused, and how many
|
||||||
`SWWAF_MAX_BANS` bans `smallwebwaf` made are kept, past, active and permanent;
|
bans the netblock had before, for a broken limit, for a clear sign of attack
|
||||||
past that, the earliest such ban of the netblock that has gone longest without
|
and by an admin. At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept,
|
||||||
a request is dropped first. The bans whose cause is `admin`, those you make or
|
past, active and permanent; past that, the earliest such ban of the netblock
|
||||||
keep, are kept besides, and never dropped. `bans.json` shows the bans and
|
that has gone longest without a request is dropped first. The bans whose cause
|
||||||
their notes, a restart lifts none, and you make, keep or lift a ban by editing
|
is `admin`, those you make or keep, are kept besides, and never dropped.
|
||||||
it (see "State files" below).
|
`bans.json` shows the bans and their notes, a restart lifts none, and you
|
||||||
|
make, keep or lift a ban by editing it (see "State files" below).
|
||||||
- Checks each request against the rules of the rule files (see "Rule files"
|
- Checks each request against the rules of the rule files (see "Rule files"
|
||||||
below) after the rate limits, and before its body is read. A `log` rule that
|
below) after the rate limits, and before its body is read. A `log` rule that
|
||||||
matches is noted in the log line; a `block` rule refuses the request with
|
matches is noted in the log line; a `block` rule refuses the request with
|
||||||
@@ -1074,7 +1075,11 @@ with times in UTC.
|
|||||||
ban for a broken limit is `requests` or `bytes`, what the limit is on. For a
|
ban for a broken limit is `requests` or `bytes`, what the limit is on. For a
|
||||||
limit a biased threshold lowered, the reason and the notes' `limit` give the
|
limit a biased threshold lowered, the reason and the notes' `limit` give the
|
||||||
lowered limit, and the notes' `limit_percent` and `limit_percent_setting` the
|
lowered limit, and the notes' `limit_percent` and `limit_percent_setting` the
|
||||||
client's percentage of that kind of limit and the setting that gave it.
|
client's percentage of that kind of limit and the setting that gave it. The
|
||||||
|
notes' `reputation` gives each blocklist, DNSBL zone or AbuseIPDB that listed
|
||||||
|
the client when the ban was made, as its `source`, named and ordered as in the
|
||||||
|
request log's `reputation`, with AbuseIPDB's `score` of the client. It is left
|
||||||
|
out when none did, and the example below shows it.
|
||||||
- `clients.json`: each client's two buckets of requests in the minute, the hour
|
- `clients.json`: each client's two buckets of requests in the minute, the hour
|
||||||
and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes`
|
and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes`
|
||||||
and `day_bytes`, and its history: when it was first and last seen, its AS
|
and `day_bytes`, and its history: when it was first and last seen, its AS
|
||||||
@@ -1121,6 +1126,60 @@ with times in UTC.
|
|||||||
Slack and ntfy too, stops the start: put the list under `"webhook"`, or remove
|
Slack and ntfy too, stops the start: put the list under `"webhook"`, or remove
|
||||||
the file.
|
the file.
|
||||||
|
|
||||||
|
This `bans.json` holds a ban for a broken rate limit on a client that a DNSBL
|
||||||
|
zone lists and AbuseIPDB scores at 100, whose limits `SWWAF_REPUTATION_ACTION`,
|
||||||
|
at its default of `limit:25`, lowered to a quarter:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"bans": [
|
||||||
|
{
|
||||||
|
"netblock": "203.0.113.9/32",
|
||||||
|
"start": "2026-10-06T12:00:41.5Z",
|
||||||
|
"expires": "2026-10-06T13:00:41.5Z",
|
||||||
|
"cause": "limit",
|
||||||
|
"reason": "requests per minute over the limit of 250",
|
||||||
|
"notes": {
|
||||||
|
"asn": "AS64496",
|
||||||
|
"as_name": "Example Net",
|
||||||
|
"country": "DE",
|
||||||
|
"kind": "requests",
|
||||||
|
"limit": 250,
|
||||||
|
"window": "minute",
|
||||||
|
"count": 251,
|
||||||
|
"limit_percent": 25,
|
||||||
|
"limit_percent_setting": "SWWAF_REPUTATION_ACTION",
|
||||||
|
"reputation": [
|
||||||
|
{
|
||||||
|
"source": "dnsbl.dronebl.org"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"source": "abuseipdb",
|
||||||
|
"score": 100
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"request": {
|
||||||
|
"time": "2026-10-06T12:00:41.5Z",
|
||||||
|
"method": "GET",
|
||||||
|
"host": "app.example",
|
||||||
|
"path": "/owner/repo/commits/branch/main?page=812",
|
||||||
|
"status": 403,
|
||||||
|
"user_agent": "scraper/1.0"
|
||||||
|
},
|
||||||
|
"requests": 512,
|
||||||
|
"refused": 0,
|
||||||
|
"earlier_bans": {
|
||||||
|
"limit": 0,
|
||||||
|
"attack": 0,
|
||||||
|
"admin": 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted
|
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted
|
||||||
through `DELETE /_smallwebwaf/bans/<client>`, or made permanent, with every such
|
through `DELETE /_smallwebwaf/bans/<client>`, or made permanent, with every such
|
||||||
change in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and when
|
change in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and when
|
||||||
@@ -1759,8 +1818,8 @@ fetched before then stays in use, and the failure is counted, logged and raised
|
|||||||
as a `source_failure` alert; a fetch cut off as `smallwebwaf` stops is not a
|
as a `source_failure` alert; a fetch cut off as `smallwebwaf` stops is not a
|
||||||
failure. The last good copy of each list is kept whole, comment lines included,
|
failure. The last good copy of each list is kept whole, comment lines included,
|
||||||
in `reputation.json` (see "State files" above), so that a restart keeps it in
|
in `reputation.json` (see "State files" above), so that a restart keeps it in
|
||||||
use too. Each list is named by its URL, in the request log, the alerts and the
|
use too. Each list is named by its URL, in the request log, the alerts, the
|
||||||
metrics, so keep a secret out of it.
|
notes of bans and the metrics, so keep a secret out of it.
|
||||||
|
|
||||||
A client in `SWWAF_ALLOW_NETS` is not checked. Any other is checked by its own
|
A client in `SWWAF_ALLOW_NETS` is not checked. Any other is checked by its own
|
||||||
address after the country lists, and `SWWAF_BLOCKLIST_ACTION` says what is done
|
address after the country lists, and `SWWAF_BLOCKLIST_ACTION` says what is done
|
||||||
@@ -1829,10 +1888,10 @@ it, such as `<key>.xbl.dq.spamhaus.net`. The key of a zone under
|
|||||||
`dq.spamhaus.net` is its first label, and `smallwebwaf` shows `********` in its
|
`dq.spamhaus.net` is its first label, and `smallwebwaf` shows `********` in its
|
||||||
place wherever it names the zone, as `********.xbl.dq.spamhaus.net`: in the
|
place wherever it names the zone, as `********.xbl.dq.spamhaus.net`: in the
|
||||||
settings logged at start, an error that stops the start, its own messages, the
|
settings logged at start, an error that stops the start, its own messages, the
|
||||||
request log, the alerts and the metrics. Only `reputation.json` keeps the zone
|
request log, the alerts, the notes of bans and the metrics. Only
|
||||||
with its key. A key in the name of any other zone is shown as given. Several
|
`reputation.json` keeps the zone with its key. A key in the name of any other
|
||||||
zones refuse queries that come through a public resolver; `SWWAF_DNSBL_RESOLVER`
|
zone is shown as given. Several zones refuse queries that come through a public
|
||||||
names another resolver to ask through.
|
resolver; `SWWAF_DNSBL_RESOLVER` names another resolver to ask through.
|
||||||
|
|
||||||
## AbuseIPDB
|
## AbuseIPDB
|
||||||
|
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package bans_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -117,7 +118,7 @@ func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
held := ledger.Bans(netblock)
|
held := ledger.Bans(netblock)
|
||||||
if len(held) != 2 || held[0] != lifted {
|
if len(held) != 2 || !reflect.DeepEqual(held[0], lifted) {
|
||||||
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
|
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -212,7 +213,7 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
|
|||||||
|
|
||||||
got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{},
|
got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{},
|
||||||
"probes for logins")
|
"probes for logins")
|
||||||
if got != want {
|
if !reflect.DeepEqual(got, want) {
|
||||||
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
|
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -224,7 +225,7 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
|
|||||||
|
|
||||||
// It refuses once the ban for the limit has ended.
|
// It refuses once the ban for the limit has ended.
|
||||||
ban, banned, _ := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour))
|
ban, banned, _ := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour))
|
||||||
if !banned || ban != want {
|
if !banned || !reflect.DeepEqual(ban, want) {
|
||||||
t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v",
|
t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v",
|
||||||
ban, banned, want)
|
ban, banned, want)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -118,6 +118,10 @@ type Notes struct {
|
|||||||
// of the rule file rule that matched, and its target.
|
// of the rule file rule that matched, and its target.
|
||||||
RuleID string `json:"rule_id,omitempty"`
|
RuleID string `json:"rule_id,omitempty"`
|
||||||
Target string `json:"target,omitempty"`
|
Target string `json:"target,omitempty"`
|
||||||
|
// Reputation is the reputation sources that listed the client when
|
||||||
|
// the request that caused the ban was made, in the order the request
|
||||||
|
// log's reputation names them. It is left out when none did.
|
||||||
|
Reputation []ReputationHit `json:"reputation,omitempty"`
|
||||||
// Request is the request that broke the limit, or whose bytes broke
|
// Request is the request that broke the limit, or whose bytes broke
|
||||||
// it, or that was the clear sign of attack.
|
// it, or that was the clear sign of attack.
|
||||||
Request Request `json:"request"`
|
Request Request `json:"request"`
|
||||||
@@ -131,6 +135,15 @@ type Notes struct {
|
|||||||
EarlierBans EarlierBans `json:"earlier_bans"`
|
EarlierBans EarlierBans `json:"earlier_bans"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ReputationHit is a reputation source that listed a client, as a
|
||||||
|
// reputation_hit alert's detail gives it: Source is the blocklist's URL,
|
||||||
|
// the DNSBL zone with its key masked, or "abuseipdb", and Score, for
|
||||||
|
// AbuseIPDB alone, its score of the client.
|
||||||
|
type ReputationHit struct {
|
||||||
|
Source string `json:"source"`
|
||||||
|
Score *int64 `json:"score,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
// EarlierBans counts a netblock's bans before a ban, by cause.
|
// EarlierBans counts a netblock's bans before a ban, by cause.
|
||||||
type EarlierBans struct {
|
type EarlierBans struct {
|
||||||
Limit int `json:"limit"`
|
Limit int `json:"limit"`
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package bans_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -130,13 +131,13 @@ func TestBrokenLimitDuringABanMakesNoOther(t *testing.T) {
|
|||||||
|
|
||||||
again, made := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
|
again, made := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
|
||||||
|
|
||||||
if made || again != first || len(ledger.Bans(netblock)) != 1 {
|
if made || !reflect.DeepEqual(again, first) || len(ledger.Bans(netblock)) != 1 {
|
||||||
t.Errorf("a limit broken during a ban gave %+v, made %t, and %d bans, "+
|
t.Errorf("a limit broken during a ban gave %+v, made %t, and %d bans, "+
|
||||||
"want %+v, not made, and 1", again, made, len(ledger.Bans(netblock)), first)
|
"want %+v, not made, and 1", again, made, len(ledger.Bans(netblock)), first)
|
||||||
}
|
}
|
||||||
|
|
||||||
again, made = ledger.BanForAttack(netblock, midnight().Add(time.Minute), bans.Notes{})
|
again, made = ledger.BanForAttack(netblock, midnight().Add(time.Minute), bans.Notes{})
|
||||||
if made || again != first {
|
if made || !reflect.DeepEqual(again, first) {
|
||||||
t.Errorf("an attack during a ban gave %+v, made %t, want %+v, not made",
|
t.Errorf("an attack during a ban gave %+v, made %t, want %+v, not made",
|
||||||
again, made, first)
|
again, made, first)
|
||||||
}
|
}
|
||||||
@@ -182,7 +183,7 @@ func TestFindCountsNothing(t *testing.T) {
|
|||||||
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
|
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
|
||||||
|
|
||||||
got, banned, _ := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
|
got, banned, _ := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
|
||||||
if !banned || got != ban {
|
if !banned || !reflect.DeepEqual(got, ban) {
|
||||||
t.Errorf("find during the ban gives %+v and %t, want %+v", got, banned, ban)
|
t.Errorf("find during the ban gives %+v and %t, want %+v", got, banned, ban)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -191,7 +192,7 @@ func TestFindCountsNothing(t *testing.T) {
|
|||||||
t.Error("the ban did not end")
|
t.Error("the ban did not end")
|
||||||
}
|
}
|
||||||
|
|
||||||
if notes := ledger.Bans(netblock)[0].Notes; notes != ban.Notes {
|
if notes := ledger.Bans(netblock)[0].Notes; !reflect.DeepEqual(notes, ban.Notes) {
|
||||||
t.Errorf("the notes are %+v, want them unchanged, %+v", notes, ban.Notes)
|
t.Errorf("the notes are %+v, want them unchanged, %+v", notes, ban.Notes)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -247,7 +248,7 @@ func TestFullLedgerDropsTheEarlierBanOfTheNetblockBannedAgain(t *testing.T) {
|
|||||||
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
|
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
|
||||||
|
|
||||||
held := ledger.Bans(netblock)
|
held := ledger.Bans(netblock)
|
||||||
if len(held) != 1 || held[0] != second ||
|
if len(held) != 1 || !reflect.DeepEqual(held[0], second) ||
|
||||||
held[0].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
held[0].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
||||||
t.Errorf("the ledger holds %+v, want only the second ban, "+
|
t.Errorf("the ledger holds %+v, want only the second ban, "+
|
||||||
"with 1 earlier ban for a limit", held)
|
"with 1 earlier ban for a limit", held)
|
||||||
@@ -342,7 +343,7 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
|
|||||||
|
|
||||||
// While the first ban lasts, none would be made.
|
// While the first ban lasts, none would be made.
|
||||||
during, would := ledger.WouldBanForAttack(netblock, midnight(), bans.Notes{})
|
during, would := ledger.WouldBanForAttack(netblock, midnight(), bans.Notes{})
|
||||||
if would || during != first {
|
if would || !reflect.DeepEqual(during, first) {
|
||||||
t.Errorf("during the first ban, would ban %t with %+v, want false with %+v",
|
t.Errorf("during the first ban, would ban %t with %+v, want false with %+v",
|
||||||
would, during, first)
|
would, during, first)
|
||||||
}
|
}
|
||||||
@@ -371,7 +372,7 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
|
|||||||
|
|
||||||
// The ban made is the one that would have been.
|
// The ban made is the one that would have been.
|
||||||
made, _ := ledger.BanForLimit(netblock, first.Expires, limitNotes)
|
made, _ := ledger.BanForLimit(netblock, first.Expires, limitNotes)
|
||||||
if made != limit {
|
if !reflect.DeepEqual(made, limit) {
|
||||||
t.Errorf("the ban made is %+v, want %+v", made, limit)
|
t.Errorf("the ban made is %+v, want %+v", made, limit)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package bans_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"slices"
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -224,7 +225,7 @@ func TestLoadKeepsAtMostMaxBansDroppingTheEarliest(t *testing.T) {
|
|||||||
ledger.Load([]bans.Ban{later, earlier})
|
ledger.Load([]bans.Ban{later, earlier})
|
||||||
|
|
||||||
held := ledger.Snapshot()
|
held := ledger.Snapshot()
|
||||||
if len(held) != 1 || held[0] != later {
|
if len(held) != 1 || !reflect.DeepEqual(held[0], later) {
|
||||||
t.Errorf("the ledger holds %+v, want only the ban that began later", held)
|
t.Errorf("the ledger holds %+v, want only the ban that began later", held)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -267,7 +268,7 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
|
|||||||
bans.Notes{})
|
bans.Notes{})
|
||||||
|
|
||||||
want := []bans.Ban{first, second, kept}
|
want := []bans.Ban{first, second, kept}
|
||||||
if got := ledger.Snapshot(); !slices.Equal(got, want) {
|
if got := ledger.Snapshot(); !reflect.DeepEqual(got, want) {
|
||||||
t.Errorf("the ledger holds %+v, want %+v", got, want)
|
t.Errorf("the ledger holds %+v, want %+v", got, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"slices"
|
"reflect"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -48,7 +48,7 @@ func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if after := server.Ledger.Snapshot(); !slices.Equal(after, before) {
|
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
|
||||||
t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before)
|
t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -79,7 +79,7 @@ func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if after := server.Ledger.Snapshot(); !slices.Equal(after, before) {
|
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
|
||||||
t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
|
t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
|
||||||
e.method, e.path, after, before)
|
e.method, e.path, after, before)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -118,7 +118,8 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
|
|||||||
line := s.get(ipv6Client, http.StatusOK, requestlog.ActionForward)
|
line := s.get(ipv6Client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
// No ban is made, and none made permanent.
|
// No ban is made, and none made permanent.
|
||||||
if held := server.Ledger.Snapshot(); len(held) != 1 || held[0] != attackBan ||
|
held := server.Ledger.Snapshot()
|
||||||
|
if len(held) != 1 || !reflect.DeepEqual(held[0], attackBan) ||
|
||||||
line.BanExpires != requestlog.FormatTime(attackBan.Expires) {
|
line.BanExpires != requestlog.FormatTime(attackBan.Expires) {
|
||||||
t.Errorf("the ledger holds %+v, and the log line gives %s, want the ban "+
|
t.Errorf("the ledger holds %+v, and the log line gives %s, want the ban "+
|
||||||
"for the attack alone, as it was", held, line.BanExpires)
|
"for the attack alone, as it was", held, line.BanExpires)
|
||||||
|
|||||||
+18
-16
@@ -127,15 +127,16 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
notes := bans.Notes{
|
notes := bans.Notes{
|
||||||
ASN: rq.line.ASN,
|
ASN: rq.line.ASN,
|
||||||
ASName: rq.line.ASName,
|
ASName: rq.line.ASName,
|
||||||
Country: rq.line.Country,
|
Country: rq.line.Country,
|
||||||
Kind: hit.Kind,
|
Kind: hit.Kind,
|
||||||
Limit: hit.Limit,
|
Limit: hit.Limit,
|
||||||
Window: hit.Window,
|
Window: hit.Window,
|
||||||
Count: hit.Count,
|
Count: hit.Count,
|
||||||
Request: rq.noted(now, status),
|
Reputation: rq.reputation,
|
||||||
Requests: rq.netblockRequests(netblock),
|
Request: rq.noted(now, status),
|
||||||
|
Requests: rq.netblockRequests(netblock),
|
||||||
}
|
}
|
||||||
|
|
||||||
percent := rq.limitPercent
|
percent := rq.limitPercent
|
||||||
@@ -174,13 +175,14 @@ func (rq *request) banForAttack(now time.Time, rule rules.Rule) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
notes := bans.Notes{
|
notes := bans.Notes{
|
||||||
ASN: rq.line.ASN,
|
ASN: rq.line.ASN,
|
||||||
ASName: rq.line.ASName,
|
ASName: rq.line.ASName,
|
||||||
Country: rq.line.Country,
|
Country: rq.line.Country,
|
||||||
RuleID: rule.ID,
|
RuleID: rule.ID,
|
||||||
Target: rule.Target,
|
Target: rule.Target,
|
||||||
Request: rq.noted(now, rq.h.config.BanResponse),
|
Reputation: rq.reputation,
|
||||||
Requests: rq.netblockRequests(netblock),
|
Request: rq.noted(now, rq.h.config.BanResponse),
|
||||||
|
Requests: rq.netblockRequests(netblock),
|
||||||
}
|
}
|
||||||
|
|
||||||
if rq.h.config.Observe {
|
if rq.h.config.Observe {
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"maps"
|
"maps"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"slices"
|
"slices"
|
||||||
"sync"
|
"sync"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -312,7 +313,7 @@ func TestBanNotes(t *testing.T) {
|
|||||||
ledger := server.Ledger
|
ledger := server.Ledger
|
||||||
|
|
||||||
got := ledger.Bans(netblock)
|
got := ledger.Bans(netblock)
|
||||||
if len(got) != 1 || got[0] != want {
|
if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
|
||||||
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -337,7 +338,7 @@ func TestBanForABrokenByteLimitHasItsNotesAndItsAlert(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
got := server.Ledger.Bans(netblock)
|
got := server.Ledger.Bans(netblock)
|
||||||
if len(got) != 1 || got[0] != want {
|
if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
|
||||||
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -126,7 +127,7 @@ func TestObserveModeMakesNoBanAndKeepsTheBansItHas(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
got := server.Ledger.Snapshot()
|
got := server.Ledger.Snapshot()
|
||||||
if len(got) != 1 || got[0] != kept {
|
if len(got) != 1 || !reflect.DeepEqual(got[0], kept) {
|
||||||
t.Errorf("bans\n%+v\nwant only\n%+v", got, kept)
|
t.Errorf("bans\n%+v\nwant only\n%+v", got, kept)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
)
|
)
|
||||||
@@ -62,29 +63,34 @@ func (rq *request) abuseIPDBDenied(ctx context.Context) bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
rq.abuseIPDBHit = true
|
rq.abuseIPDBHit = true
|
||||||
rq.noteHit(reputation.AbuseIPDBSource, "scored by AbuseIPDB at or over "+
|
rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
|
||||||
"SWWAF_ABUSEIPDB_MIN_SCORE", map[string]any{
|
"scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE")
|
||||||
"source": reputation.AbuseIPDBSource, "score": score,
|
|
||||||
})
|
|
||||||
|
|
||||||
return rq.h.config.ReputationAction == deny
|
return rq.h.config.ReputationAction == deny
|
||||||
}
|
}
|
||||||
|
|
||||||
// noteListed notes each of sources, the URLs of the blocklists or the
|
// noteListed notes each of sources, the URLs of the blocklists or the
|
||||||
// DNSBL zones, their keys masked, that list the client, as noteHit does,
|
// DNSBL zones, their keys masked, that list the client, as noteHit does,
|
||||||
// with reason, and the source in the alert's detail.
|
// with reason.
|
||||||
func (rq *request) noteListed(sources []string, reason string) {
|
func (rq *request) noteListed(sources []string, reason string) {
|
||||||
for _, source := range sources {
|
for _, source := range sources {
|
||||||
rq.noteHit(source, reason, map[string]any{"source": source})
|
rq.noteHit(bans.ReputationHit{Source: source}, reason)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// noteHit adds source, which lists the client, to the log line's
|
// noteHit adds hit's source, which lists the client, to the log line's
|
||||||
// reputation, counts it in the metrics, and raises a reputation_hit alert
|
// reputation, and hit to the notes of a ban the request makes, counts the
|
||||||
// with reason and detail.
|
// source in the metrics, and raises a reputation_hit alert with reason,
|
||||||
func (rq *request) noteHit(source, reason string, detail map[string]any) {
|
// whose detail gives hit's source and score.
|
||||||
rq.line.Reputation = append(rq.line.Reputation, source)
|
func (rq *request) noteHit(hit bans.ReputationHit, reason string) {
|
||||||
rq.h.metrics.ReputationHit(source)
|
detail := map[string]any{"source": hit.Source}
|
||||||
|
if hit.Score != nil {
|
||||||
|
detail["score"] = *hit.Score
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.line.Reputation = append(rq.line.Reputation, hit.Source)
|
||||||
|
rq.reputation = append(rq.reputation, hit)
|
||||||
|
rq.h.metrics.ReputationHit(hit.Source)
|
||||||
rq.h.alerts.Raise(alerts.Alert{
|
rq.h.alerts.Raise(alerts.Alert{
|
||||||
Event: alerts.EventReputationHit,
|
Event: alerts.EventReputationHit,
|
||||||
Client: rq.client,
|
Client: rq.client,
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"maps"
|
"maps"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"slices"
|
"slices"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -13,6 +14,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
@@ -874,6 +876,73 @@ func TestWithoutAnAbuseIPDBKeyNoClientIsCheckedNorAScoreUsed(t *testing.T) {
|
|||||||
`instance="`+alertInstance+`",source="`+abuseipdb+`"}`)
|
`instance="`+alertInstance+`",source="`+abuseipdb+`"}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBanNotesNameEachReputationSourceThatListedTheClient(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
score := int64(90)
|
||||||
|
listed := []bans.ReputationHit{
|
||||||
|
{Source: dropURL}, {Source: dnsblZone}, {Source: abuseipdb, Score: &score},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
// ban sends the requests from the client at from that ban it.
|
||||||
|
ban func(s *sender, from string)
|
||||||
|
}{
|
||||||
|
{"for a broken rate limit", func(s *sender, from string) {
|
||||||
|
s.get(from, http.StatusOK, requestlog.ActionForward)
|
||||||
|
s.get(from, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
}},
|
||||||
|
{"for a clear sign of attack", func(s *sender, from string) {
|
||||||
|
s.request(from, probePath, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
}},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, server, queue := startWithAlerts(t, map[string]string{
|
||||||
|
rateLimitPerMinute: "1", rulesDir: writeRules(t, testRules),
|
||||||
|
blocklistURLs: dropURL, blocklistAction: actionLog,
|
||||||
|
dnsblZones: dnsblZone, dnsblResolver: noResolver,
|
||||||
|
abuseIPDBKey: accountKey, reputationAction: actionLog,
|
||||||
|
})
|
||||||
|
// Every source lists client, and none otherClient, whose score is
|
||||||
|
// under SWWAF_ABUSEIPDB_MIN_SCORE, 75 by default.
|
||||||
|
loadLists(t, server, map[string][]string{dropURL: {client}})
|
||||||
|
loadVerdicts(server, map[string][]string{client: {dnsblZone}, otherClient: nil})
|
||||||
|
loadScores(server, map[string]int64{client: score, otherClient: 74})
|
||||||
|
|
||||||
|
for _, banned := range []struct {
|
||||||
|
from string
|
||||||
|
want []bans.ReputationHit
|
||||||
|
}{{client, listed}, {otherClient, nil}} {
|
||||||
|
tc.ban(s, banned.from)
|
||||||
|
|
||||||
|
held := server.Ledger.Bans(netip.MustParsePrefix(banned.from + "/32"))
|
||||||
|
if len(held) != 1 || !reflect.DeepEqual(held[0].Notes.Reputation, banned.want) {
|
||||||
|
t.Errorf("bans of %s %+v, want one whose notes have the reputation %+v",
|
||||||
|
banned.from, held, banned.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The alert for each ban carries the same in its notes.
|
||||||
|
var alerted [][]bans.ReputationHit
|
||||||
|
|
||||||
|
for _, alert := range queue.Snapshot().Waiting[alerts.DestinationWebhook] {
|
||||||
|
if alert.Event == alerts.EventBan {
|
||||||
|
notes, _ := alert.Detail["notes"].(bans.Notes)
|
||||||
|
alerted = append(alerted, notes.Reputation)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if want := [][]bans.ReputationHit{listed, nil}; !reflect.DeepEqual(alerted, want) {
|
||||||
|
t.Errorf("the ban alerts' notes have the reputation %+v, want %+v",
|
||||||
|
alerted, want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// listsFetched is when loadLists has the copies fetched.
|
// listsFetched is when loadLists has the copies fetched.
|
||||||
func listsFetched() time.Time {
|
func listsFetched() time.Time {
|
||||||
return time.Date(2026, 10, 5, 0, 0, 0, 0, time.UTC)
|
return time.Date(2026, 10, 5, 0, 0, 0, 0, time.UTC)
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
@@ -71,7 +72,10 @@ type request struct {
|
|||||||
// dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once
|
// dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once
|
||||||
// AbuseIPDB's score of it is a hit.
|
// AbuseIPDB's score of it is a hit.
|
||||||
blocklisted, dnsblListed, abuseIPDBHit bool
|
blocklisted, dnsblListed, abuseIPDBHit bool
|
||||||
start time.Time
|
// reputation is the reputation sources that list the client, for the
|
||||||
|
// notes of a ban the request makes.
|
||||||
|
reputation []bans.ReputationHit
|
||||||
|
start time.Time
|
||||||
// checked is when the checks were done, and upstreamStart when the
|
// checked is when the checks were done, and upstreamStart when the
|
||||||
// request was handed to the app.
|
// request was handed to the app.
|
||||||
checked time.Time
|
checked time.Time
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"net/netip"
|
"net/netip"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
"slices"
|
"slices"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -73,7 +74,7 @@ func TestEachRuleAction(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
got := server.Ledger.Bans(netblock)
|
got := server.Ledger.Bans(netblock)
|
||||||
if len(got) != 1 || got[0] != want {
|
if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
|
||||||
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -75,6 +75,15 @@ const permanentBansJSON = `{
|
|||||||
"limit": 1000,
|
"limit": 1000,
|
||||||
"window": "minute",
|
"window": "minute",
|
||||||
"count": 1000.5,
|
"count": 1000.5,
|
||||||
|
"reputation": [
|
||||||
|
{
|
||||||
|
"source": "https://lists.example/drop.txt"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"source": "abuseipdb",
|
||||||
|
"score": 100
|
||||||
|
}
|
||||||
|
],
|
||||||
"request": {
|
"request": {
|
||||||
"time": "2026-10-06T00:00:00Z",
|
"time": "2026-10-06T00:00:00Z",
|
||||||
"method": "GET",
|
"method": "GET",
|
||||||
@@ -919,7 +928,7 @@ func TestBansWrittenOnceWriteDelayAfterABan(t *testing.T) {
|
|||||||
load(t, read)
|
load(t, read)
|
||||||
|
|
||||||
want := []bans.Ban{first, second}
|
want := []bans.Ban{first, second}
|
||||||
if got := read.Ledger.Snapshot(); !slices.Equal(got, want) {
|
if got := read.Ledger.Snapshot(); !reflect.DeepEqual(got, want) {
|
||||||
t.Errorf("bans.json holds %+v, want %+v", got, want)
|
t.Errorf("bans.json holds %+v, want %+v", got, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1808,6 +1817,8 @@ func fill(params state.Params) {
|
|||||||
|
|
||||||
// permanentBan is the ban permanentBansJSON holds.
|
// permanentBan is the ban permanentBansJSON holds.
|
||||||
func permanentBan() bans.Ban {
|
func permanentBan() bans.Ban {
|
||||||
|
score := int64(100)
|
||||||
|
|
||||||
return bans.Ban{
|
return bans.Ban{
|
||||||
Netblock: netip.MustParsePrefix("2001:db8::/64"),
|
Netblock: netip.MustParsePrefix("2001:db8::/64"),
|
||||||
Start: midnight(),
|
Start: midnight(),
|
||||||
@@ -1820,6 +1831,9 @@ func permanentBan() bans.Ban {
|
|||||||
Limit: 1000,
|
Limit: 1000,
|
||||||
Window: "minute",
|
Window: "minute",
|
||||||
Count: 1000.5,
|
Count: 1000.5,
|
||||||
|
Reputation: []bans.ReputationHit{
|
||||||
|
{Source: blocklistURL}, {Source: reputation.AbuseIPDBSource, Score: &score},
|
||||||
|
},
|
||||||
Request: bans.Request{
|
Request: bans.Request{
|
||||||
Time: midnight(),
|
Time: midnight(),
|
||||||
Method: "GET",
|
Method: "GET",
|
||||||
@@ -1995,10 +2009,10 @@ func edit(t *testing.T, dir, name, content string) {
|
|||||||
|
|
||||||
// wantEqual checks that the entries read back from file are those
|
// wantEqual checks that the entries read back from file are those
|
||||||
// written.
|
// written.
|
||||||
func wantEqual[E comparable](t *testing.T, file string, got, want []E) {
|
func wantEqual[E any](t *testing.T, file string, got, want []E) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
if !slices.Equal(got, want) {
|
if !reflect.DeepEqual(got, want) {
|
||||||
t.Errorf("%s read back\n%+v\nwant\n%+v", file, got, want)
|
t.Errorf("%s read back\n%+v\nwant\n%+v", file, got, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user