Bans an admin makes or lifts: the admin cause, a reason, lifted bans kept (closes #86)
check / check (push) Successful in 3m27s

A bans.json entry without a cause gets the cause admin, written back so.
Bans whose cause is admin are never dropped and do not count toward
SWWAF_MAX_BANS, so setting a ban's cause to admin keeps it. Bans
smallwebwaf makes get a reason: the limit broken or the rule matched. A
lifted ban refuses nothing, is kept, and makes no later ban longer.
smallwebwaf_bans_made_total counts admin bans an edit adds while running;
earlier_bans counts admin in place of without_cause.

Judgement call: lifted lifts at once, whatever time it gives.
Judgement call: a lifted ban still counts in earlier_bans.
Known gap: a ban dropped from behind an admin's ban on its netblock leaves that netblock's later earlier_bans.

Model: opus-5-5
This commit was merged in pull request #88.
This commit is contained in:
2026-10-06 23:09:52 +02:00
parent 0797e5def2
commit ee9ba08a8a
11 changed files with 677 additions and 153 deletions
+68 -47
View File
@@ -13,29 +13,30 @@ JSON log line for every request.
Status: the first two milestones are built Status: the first two milestones are built
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and (https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are eight parts of https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are nine parts of
milestone 3: the static lists, the bans that broken rate limits lead to, the milestone 3: the static lists, the bans that broken rate limits lead to, the ban
JSON state files with your edits taken in while it runs and the paths the rate ledger with the bans you make, keep and lift, the JSON state files with your
limits do not count, which come next in the build order, `observe` mode and the edits taken in while it runs and the paths the rate limits do not count, which
rest of the request log's fields, which come a little later, and the metrics come next in the build order, `observe` mode and the rest of the request log's
endpoint and the header size and the idle time as settings, which come last in fields, which come a little later, and the metrics endpoint and the header size
it. So are two parts of the stage after it: the rule files, the first part, with and the idle time as settings, which come last in it. So are two parts of the
the bans for a clear sign of attack, and remote log sending. `smallwebwaf` stage after it: the rule files, the first part, with the bans for a clear sign
passes each request to the app and the app's answer back, unchanged, within its of attack, and remote log sending. `smallwebwaf` passes each request to the app
timeouts and size limits, works out each client's address, bans a client that and the app's answer back, unchanged, within its timeouts and size limits, works
sends too many requests, not counting those for the paths you choose, refuses a out each client's address, bans a client that sends too many requests, not
client that comes from a country you refuse or from a network you refuse, lets counting those for the paths you choose, refuses a client that comes from a
the networks you choose through, checks each request against the rule files and country you refuse or from a network you refuse, lets the networks you choose
bans a client whose request is a clear sign of attack, keeps its bans, each through, checks each request against the rule files and bans a client whose
client's counters and history, and GeoJS's answers in JSON files across request is a clear sign of attack, keeps its bans, each client's counters and
restarts, takes in your edits of those files and of the rule files while it history, and GeoJS's answers in JSON files across restarts, takes in your edits
runs, writes a JSON log line for every request, sends its log lines to a syslog of those files, such as a ban you make, keep or lift, and of the rule files
server too if you name one, serves Prometheus metrics to a scraper that holds while it runs, writes a JSON log line for every request, sends its log lines to
the metrics token, and in `observe` mode passes on the requests it would refuse, a syslog server too if you name one, serves Prometheus metrics to a scraper that
logging what it would have done with them. It comes as the image the app's own holds the metrics token, and in `observe` mode passes on the requests it would
image is built on. The rest of the design comes after that, in the order of the refuse, logging what it would have done with them. It comes as the image the
build order in [`SPEC.md`](SPEC.md). The survey of existing tools that led to app's own image is built on. The rest of the design comes after that, in the
the design is in [`EVALUATION.md`](EVALUATION.md). order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
that led to the design is in [`EVALUATION.md`](EVALUATION.md).
## Getting started ## Getting started
@@ -111,11 +112,13 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
window and the requests counted in it, the request that broke it, the client's window and the requests counted in it, the request that broke it, the client's
country when it was looked up, the netblock's requests since it was first country when it was looked up, the netblock's requests since it was first
seen, how many of them the ban has refused, and how many bans the netblock had seen, how many of them the ban has refused, and how many bans the netblock had
before, for a broken limit, for a clear sign of attack and without a cause. At before, for a broken limit, for a clear sign of attack and by an admin. At
most `SWWAF_MAX_BANS` bans are kept, past, active and permanent; past that, most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept, past, active and
the earliest ban of the netblock that has gone longest without a request is permanent; past that, the earliest such ban of the netblock that has gone
dropped first. `bans.json` shows the bans and their notes, a restart lifts longest without a request is dropped first. The bans whose cause is `admin`,
none, and you add or lift a ban by editing it (see "State files" below). those you make or keep, are kept besides, and never dropped. `bans.json` shows
the bans and their notes, a restart lifts none, and you make, keep or lift a
ban by editing it (see "State files" below).
- Checks each request against the rules of the rule files (see "Rule files" - Checks each request against the rules of the rule files (see "Rule files"
below) after the rate limits, and before its body is read. A `log` rule that below) after the rate limits, and before its body is read. A `log` rule that
matches is noted in the log line; a `block` rule refuses the request with matches is noted in the log line; a `block` rule refuses the request with
@@ -263,8 +266,8 @@ it, and the effective settings are logged at start.
would be longer is permanent instead. would be longer is permanent instead.
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of - `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
attack. attack.
- `SWWAF_MAX_BANS` (default `5000`): the most bans kept, past, active and - `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
permanent. kept, past, active and permanent. The bans you make or keep are kept besides.
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around - `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
an IPv4 client that a ban covers, such as `24` to ban the surrounding /24. An an IPv4 client that a ban covers, such as `24` to ban the surrounding /24. An
IPv6 ban covers the client's /64. IPv6 ban covers the client's /64.
@@ -458,9 +461,14 @@ them.
[`SPEC.md`](SPEC.md) describes. Each has a top-level `version`, 1, and lists its [`SPEC.md`](SPEC.md) describes. Each has a top-level `version`, 1, and lists its
entries by client address, with times in UTC. entries by client address, with times in UTC.
- `bans.json`: every ban with its notes, indented to be read; a permanent ban's - `bans.json`: every ban with its notes, indented to be read. A permanent ban's
`expires` is `null`, and a ban `smallwebwaf` made has the `cause` `limit` for `expires` is `null`. A ban's `cause` is `limit` for a broken rate limit or
a broken rate limit or `attack` for a clear sign of attack. `attack` for a clear sign of attack, for a ban `smallwebwaf` made, and `admin`
for one you made or keep. Its `reason` is a short text: for a ban
`smallwebwaf` made, the limit broken, such as
`requests per minute over the limit of 1000`, or the rule that matched, such
as `matched the rule env-file`; for yours, what you wrote. Its `lifted` is
when you lifted it, and is left out until you do.
- `clients.json`: each client's two buckets in the minute, the hour and the day, - `clients.json`: each client's two buckets in the minute, the hour and the day,
and its history: when it was first and last seen, its country as last looked and its history: when it was first and last seen, its country as last looked
up and when, its requests, how many were forwarded and how many refused (one up and when, its requests, how many were forwarded and how many refused (one
@@ -492,8 +500,8 @@ without a field it needs, named with the entry's place in the file: a ban's
`netblock`, `start` or `expires`, which is `null` for a permanent ban; a `netblock`, `start` or `expires`, which is `null` for a permanent ban; a
client's `client`, or the `start` of a window in which it has requests; an client's `client`, or the `start` of a window in which it has requests; an
answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or
`answered`. So does a ban whose `cause` is neither `limit` nor `attack`. The AS `answered`. So does a ban whose `cause` is not `limit`, `attack` or `admin`. The
number and AS name come with their lookup. AS number and AS name come with their lookup.
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
a state file as soon as you save it: what the file then holds replaces what a state file as soon as you save it: what the file then holds replaces what
@@ -511,10 +519,12 @@ before the editor has finished writing it. Mend the `.bad` file and move it
back. A file you remove is written again at its next write. back. A file you remove is written again at its next write.
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start` To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
and its `expires`, `null` for a ban that never ends; its `cause` and its `notes` and its `expires`, `null` for a ban that never ends; its `reason` and its
may be left out. A ban whose `cause` is `attack` becomes permanent at the first `notes` may be left out, and so may its `cause`, which is then `admin`, and is
request it refuses; one without a cause does not. This `bans.json` bans written so at the file's next write. A ban whose `cause` is `admin` is never
`203.0.113.0/24` for good: dropped and does not count toward `SWWAF_MAX_BANS`. A ban whose `cause` is
`attack` becomes permanent at the first request it refuses; one whose `cause` is
`admin` does not. This `bans.json` bans `203.0.113.0/24` for good:
```json ```json
{ {
@@ -523,14 +533,22 @@ request it refuses; one without a cause does not. This `bans.json` bans
{ {
"netblock": "203.0.113.0/24", "netblock": "203.0.113.0/24",
"start": "2026-10-06T12:00:00Z", "start": "2026-10-06T12:00:00Z",
"expires": null "expires": null,
"reason": "probes for logins"
} }
] ]
} }
``` ```
To lift a ban, delete its entry. `smallwebwaf` then forgets the ban, so it does To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause`
not make the netblock's next ban longer. to `admin`: `"cause": "admin"`.
To lift a ban, add `lifted` to its entry, with the time you lift it, such as
`"lifted": "2026-10-06T13:00:00Z"`. From when the edit is taken in, the ban
refuses nothing, whatever time `lifted` gives, and does not make the netblock's
next ban longer; it is kept in `bans.json` with its notes, as any other ban is.
To forget a ban altogether, delete its entry: it then refuses nothing either,
and does not make the netblock's next ban longer.
## Rule files ## Rule files
@@ -624,8 +642,10 @@ other request. No metric carries a client's address.
- `smallwebwaf_rate_limit_hits_total` by `window`, - `smallwebwaf_rate_limit_hits_total` by `window`,
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose `smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
limit was passed, `smallwebwaf_offences_total` by `kind`, and limit was passed, `smallwebwaf_offences_total` by `kind`, and
`smallwebwaf_bans_made_total` by `cause`, `limit` or `attack`; `smallwebwaf_bans_made_total` by `cause`, `limit`, `attack` or `admin`, the
`smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`. last for the bans whose `cause` is `admin` that you add to `bans.json` while
`smallwebwaf` runs; `smallwebwaf_active_bans` and
`smallwebwaf_permanent_bans`, neither of which counts a lifted ban.
- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by - `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
`rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the `rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
rules read from the rule files. rules read from the rule files.
@@ -954,7 +974,7 @@ addresses are never sent to GeoJS.
happened, and served in the Prometheus text format. happened, and served in the Prometheus text format.
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how - `internal/bans`: the ban ledger: each netblock's bans with their notes, how
long a new ban lasts, when a ban for a clear sign of attack becomes permanent, long a new ban lasts, when a ban for a clear sign of attack becomes permanent,
and which ban is dropped when `SWWAF_MAX_BANS` are held. and which ban `smallwebwaf` made is dropped when `SWWAF_MAX_BANS` are held.
- `internal/rules`: reads the rule files at start and again as they change, and - `internal/rules`: reads the rule files at start and again as they change, and
tells which of their rules a request matches. tells which of their rules a request matches.
- `internal/lookup`: looks up each client's country through GeoJS, and keeps the - `internal/lookup`: looks up each client's country through GeoJS, and keeps the
@@ -977,8 +997,9 @@ addresses are never sent to GeoJS.
checks. checks.
Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
table of clients to 20,000, the GeoJS answers to 100,000 and the banned table of clients to 20,000 and the GeoJS answers to 100,000, dropping the least
netblocks to `SWWAF_MAX_BANS`, dropping the least recently seen, and recently seen, and the banned netblocks in the order they were last seen, from
which the ledger picks the ban to drop past `SWWAF_MAX_BANS`, and
`github.com/prometheus/client_golang` keeps the metrics and serves them, and `github.com/prometheus/client_golang` keeps the metrics and serves them, and
`github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file or a rule `github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file or a rule
file is saved. The country codes are the list in `internal/config/config.go`. file is saved. The country codes are the list in `internal/config/config.go`.
+186
View File
@@ -0,0 +1,186 @@
package bans_test
import (
"net/netip"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/bans"
)
func TestBanWithoutACauseIsAnAdmins(t *testing.T) {
t.Parallel()
netblock := netip.MustParsePrefix("203.0.113.0/24")
ledger := bans.New(defaultRules())
ledger.Load([]bans.Ban{{Netblock: netblock, Start: midnight()}})
if got := ledger.Bans(netblock)[0].Cause; got != bans.CauseAdmin {
t.Errorf("the ban's cause is %q, want admin", got)
}
}
func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
t.Parallel()
rules := defaultRules()
rules.MaxBans = 1
ledger := bans.New(rules)
adminsOnly := netip.MustParsePrefix("198.51.100.0/24")
both := netip.MustParsePrefix("203.0.113.1/32")
second := netip.MustParsePrefix("203.0.113.2/32")
third := netip.MustParsePrefix("203.0.113.3/32")
// Seen longest ago, a netblock with two of an admin's bans alone, and
// then one with an admin's ban before a ban smallwebwaf made: the one
// ban counted toward MaxBans.
ledger.Load([]bans.Ban{
{Netblock: adminsOnly, Start: midnight().Add(-3 * time.Hour), Cause: bans.CauseAdmin},
{Netblock: adminsOnly, Start: midnight().Add(-2 * time.Hour), Cause: bans.CauseAdmin},
{Netblock: both, Start: midnight().Add(-time.Hour), Cause: bans.CauseAdmin},
{
Netblock: both,
Start: midnight(),
Expires: midnight().Add(time.Hour),
Cause: bans.CauseLimit,
},
})
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 2})
// A new ban drops the ban smallwebwaf made, and only that one.
ledger.BanForLimit(second, midnight(), bans.Notes{})
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1})
if ledger.Bans(both)[0].Cause != bans.CauseAdmin {
t.Errorf("%s kept %+v, want the admin's ban", both, ledger.Bans(both))
}
// And the next drops that one.
ledger.BanForLimit(third, midnight(), bans.Notes{})
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1})
}
func TestReasonOfTheBansSmallwebwafMakes(t *testing.T) {
t.Parallel()
ledger := bans.New(defaultRules())
limit := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
bans.Notes{Limit: 1000, Window: "minute"})
attack := ledger.BanForAttack(netip.MustParsePrefix("203.0.113.2/32"), midnight(),
bans.Notes{RuleID: "git-dir", Target: "path"})
for _, tc := range []struct{ got, want string }{
{limit.Reason, "requests per minute over the limit of 1000"},
{attack.Reason, "matched the rule git-dir"},
} {
if tc.got != tc.want {
t.Errorf("the reason is %q, want %q", tc.got, tc.want)
}
}
}
func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
t.Parallel()
// An hour's ban lifted ten minutes after it started.
netblock := netip.MustParsePrefix("203.0.113.9/32")
lifted := bans.Ban{
Netblock: netblock,
Start: midnight(),
Expires: midnight().Add(time.Hour),
Cause: bans.CauseLimit,
Lifted: midnight().Add(10 * time.Minute),
}
ledger := bans.New(defaultRules())
ledger.Load([]bans.Ban{lifted})
// While it would still last, it refuses nothing, and a limit broken
// bans for an hour, as a first broken limit does; the lifted ban is
// kept, and counted among the earlier bans.
now := midnight().Add(30 * time.Minute)
_, banned := ledger.Check(netblock.Addr(), now)
if banned {
t.Error("the lifted ban refuses")
}
ban := ledger.BanForLimit(netblock, now, bans.Notes{})
if ban.Expires.Sub(ban.Start) != time.Hour ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit",
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
}
held := ledger.Bans(netblock)
if len(held) != 2 || held[0] != lifted {
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
}
}
func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(t *testing.T) {
t.Parallel()
// A permanent ban for a clear sign of attack, lifted.
netblock := netip.MustParsePrefix("203.0.113.9/32")
ledger := bans.New(defaultRules())
ledger.Load([]bans.Ban{{
Netblock: netblock,
Start: midnight(),
Cause: bans.CauseAttack,
Lifted: midnight().Add(time.Hour),
}})
now := midnight().Add(2 * time.Hour)
_, banned := ledger.Find(netblock.Addr(), now)
if banned {
t.Error("the lifted ban refuses")
}
active, permanent := ledger.Count(now)
if active != 0 || permanent != 0 {
t.Errorf("%d bans are active and %d permanent, want none", active, permanent)
}
// The next clear sign of attack bans for seven days, as a first does.
ban := ledger.BanForAttack(netblock, now, bans.Notes{})
if ban.Expires.Sub(ban.Start) != 7*day {
t.Errorf("the next ban for an attack ends at %s, want seven days on", ban.Expires)
}
}
func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
t.Parallel()
ledger := bans.New(defaultRules())
made := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
bans.Notes{})
atStart := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
Start: midnight(),
}
// The bans read at the start were made before it.
ledger.Load([]bans.Ban{made, atStart})
if got := ledger.Made(bans.CauseAdmin); got != 0 {
t.Fatalf("%d bans made by an admin after the start's, want none", got)
}
// The admin keeps the ban smallwebwaf made, keeps the one read at the
// start, and adds one without a cause: that one alone is made.
kept := made
kept.Cause = bans.CauseAdmin
added := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.3/32"),
Start: midnight(),
}
ledger.LoadEdit([]bans.Ban{kept, atStart, added})
if ledger.Made(bans.CauseAdmin) != 1 || ledger.Made(bans.CauseLimit) != 1 {
t.Errorf("%d bans made by an admin and %d for a limit, want 1 of each",
ledger.Made(bans.CauseAdmin), ledger.Made(bans.CauseLimit))
}
}
+160 -66
View File
@@ -1,11 +1,13 @@
// Package bans is the ban ledger: the bans smallwebwaf makes on the // Package bans is the ban ledger: the bans smallwebwaf makes on the
// netblocks of clients that break a rate limit or show a clear sign of // netblocks of clients that break a rate limit or show a clear sign of
// attack, with their notes, as the "Bans" section of SPEC.md describes. // attack, and those an admin makes, with their notes, as the "Bans"
// The bans are kept in memory, and written to bans.json and read from it // section of SPEC.md describes. The bans are kept in memory, and written
// by the state package. // to bans.json and read from it by the state package.
package bans package bans
import ( import (
"fmt"
"math"
"net/netip" "net/netip"
"slices" "slices"
"strings" "strings"
@@ -15,13 +17,15 @@ import (
"github.com/hashicorp/golang-lru/v2/simplelru" "github.com/hashicorp/golang-lru/v2/simplelru"
) )
// The causes of the bans smallwebwaf makes. A ban an admin adds to // The causes of bans.
// bans.json may have no cause.
const ( const (
// CauseLimit is a ban for a broken limit. // CauseLimit is a ban smallwebwaf made for a broken limit.
CauseLimit = "limit" CauseLimit = "limit"
// CauseAttack is a ban for a clear sign of attack. // CauseAttack is a ban smallwebwaf made for a clear sign of attack.
CauseAttack = "attack" CauseAttack = "attack"
// CauseAdmin is a ban an admin made, or one smallwebwaf made that an
// admin keeps. It is never dropped.
CauseAdmin = "admin"
) )
// repeatFactor is how many times as long as the netblock's last ban a ban // repeatFactor is how many times as long as the netblock's last ban a ban
@@ -46,9 +50,10 @@ type Rules struct {
// AttackBanDuration is how long a first ban for a clear sign of attack // AttackBanDuration is how long a first ban for a clear sign of attack
// lasts. // lasts.
AttackBanDuration time.Duration AttackBanDuration time.Duration
// MaxBans is the most bans held, at least one. Past it, the earliest // MaxBans is the most bans held whose cause is not CauseAdmin, at
// ban of the netblock that has gone longest without a request is // least one. Past it, the earliest such ban of the netblock that has
// dropped. // gone longest without a request is dropped. Bans whose cause is
// CauseAdmin are held besides, and never dropped.
MaxBans int MaxBans int
} }
@@ -58,9 +63,15 @@ type Ban struct {
Start time.Time Start time.Time
// Expires is when the ban ends, zero for a permanent ban. // Expires is when the ban ends, zero for a permanent ban.
Expires time.Time Expires time.Time
// Cause is CauseLimit or CauseAttack, or "" for a ban an admin added // Cause is CauseLimit, CauseAttack or CauseAdmin.
// without one.
Cause string Cause string
// Reason is a short text: for a ban smallwebwaf made, the limit broken
// or the rule that matched; for an admin's, what the admin wrote.
Reason string
// Lifted is when an admin lifted the ban, zero while no admin has. A
// lifted ban refuses nothing, and does not make the netblock's next
// ban longer.
Lifted time.Time
Notes Notes Notes Notes
} }
@@ -69,9 +80,10 @@ func (b Ban) Permanent() bool {
return b.Expires.IsZero() return b.Expires.IsZero()
} }
// ActiveAt reports whether the ban refuses requests at now. // ActiveAt reports whether the ban refuses requests at now: it has not
// been lifted, and has not run out.
func (b Ban) ActiveAt(now time.Time) bool { func (b Ban) ActiveAt(now time.Time) bool {
return b.Permanent() || now.Before(b.Expires) return b.Lifted.IsZero() && (b.Permanent() || now.Before(b.Expires))
} }
// Notes are what an admin needs to decide whether to lift a ban. The // Notes are what an admin needs to decide whether to lift a ban. The
@@ -107,13 +119,10 @@ type Notes struct {
} }
// EarlierBans counts a netblock's bans before a ban, by cause. // EarlierBans counts a netblock's bans before a ban, by cause.
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
type EarlierBans struct { type EarlierBans struct {
Limit int `json:"limit"` Limit int `json:"limit"`
Attack int `json:"attack"` Attack int `json:"attack"`
// WithoutCause counts the bans an admin added without a cause. Admin int `json:"admin"`
WithoutCause int `json:"without_cause"`
} }
// Request is a request in a ban's notes. Each text is cut to 256 bytes. // Request is a request in a ban's notes. Each text is cut to 256 bytes.
@@ -141,9 +150,11 @@ type Ledger struct {
// netblocks holds each banned netblock's bans, oldest first. Check and // netblocks holds each banned netblock's bans, oldest first. Check and
// Find make each netblock they find the most recently seen. // Find make each netblock they find the most recently seen.
netblocks *simplelru.LRU[netip.Prefix, *[]Ban] netblocks *simplelru.LRU[netip.Prefix, *[]Ban]
// held is how many bans netblocks holds, at most rules.MaxBans. // held is how many bans netblocks holds whose cause is not CauseAdmin,
// at most rules.MaxBans.
held int held int
// made is how many bans the ledger has made since the start, by cause. // made is how many bans have been made since the start, by cause: by
// the ledger, and by an admin in an edit of bans.json.
made map[string]int made map[string]int
// v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6 // v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6
// netblocks that have been banned. Check looks for a ban at each of // netblocks that have been banned. Check looks for a ban at each of
@@ -155,9 +166,10 @@ type Ledger struct {
// New returns a Ledger with no ban yet. // New returns a Ledger with no ban yet.
func New(rules Rules) *Ledger { func New(rules Rules) *Ledger {
// Every netblock held has a ban, so there are never more netblocks // The ledger drops bans itself, and never those whose cause is
// than rules.MaxBans, and the LRU never drops one itself. // CauseAdmin, however many there are, so the LRU has no limit of its
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](rules.MaxBans, nil) // own: it keeps the netblocks in the order they were last seen.
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](math.MaxInt, nil)
if err != nil { if err != nil {
panic(err) // NewLRU fails only for a size below one panic(err) // NewLRU fails only for a size below one
} }
@@ -233,21 +245,26 @@ func activeBan(bans []Ban, now time.Time) *Ban {
// BanForLimit bans netblock at now for a broken limit, with notes, and // BanForLimit bans netblock at now for a broken limit, with notes, and
// returns the ban. A first ban lasts LimitBanDuration. A ban made within // returns the ban. A first ban lasts LimitBanDuration. A ban made within
// LimitBanRepeatWindow after the netblock's ban that ended last, other // LimitBanRepeatWindow after the netblock's ban that ended last, other
// than one for a clear sign of attack, lasts repeatFactor times as long as // than one for a clear sign of attack or a lifted one, lasts repeatFactor
// that one. A ban that would be longer than MaxBanDuration is permanent // times as long as that one. A ban that would be longer than
// instead. If a ban on netblock is still active, as when two of its // MaxBanDuration is permanent instead. If a ban on netblock is still
// requests break a limit at once, that ban is returned and no other is // active, as when two of its requests break a limit at once, that ban is
// made. The ledger fills in the notes' Refused and EarlierBans itself. // returned and no other is made. The ledger fills in the notes' Refused
// and EarlierBans itself, and gives the ban the reason "requests per
// <Window> over the limit of <Limit>", from the notes.
func (l *Ledger) BanForLimit(netblock netip.Prefix, now time.Time, notes Notes) Ban { func (l *Ledger) BanForLimit(netblock netip.Prefix, now time.Time, notes Notes) Ban {
return l.ban(netblock, now, CauseLimit, notes) reason := fmt.Sprintf("requests per %s over the limit of %d",
notes.Window, notes.Limit)
return l.ban(netblock, now, CauseLimit, reason, notes)
} }
// BanForAttack bans netblock at now for a clear sign of attack, with // BanForAttack bans netblock at now for a clear sign of attack, with
// notes, and returns the ban, as BanForLimit does. A first ban lasts // notes, and returns the ban, as BanForLimit does. A first ban lasts
// AttackBanDuration; once the netblock has had one, the next is // AttackBanDuration; once the netblock has had one that was not lifted,
// permanent. // the next is permanent. Its reason is "matched the rule <RuleID>".
func (l *Ledger) BanForAttack(netblock netip.Prefix, now time.Time, notes Notes) Ban { func (l *Ledger) BanForAttack(netblock netip.Prefix, now time.Time, notes Notes) Ban {
return l.ban(netblock, now, CauseAttack, notes) return l.ban(netblock, now, CauseAttack, "matched the rule "+notes.RuleID, notes)
} }
// Bans returns the bans held on netblock, oldest first. It is not a // Bans returns the bans held on netblock, oldest first. It is not a
@@ -264,8 +281,10 @@ func (l *Ledger) Bans(netblock netip.Prefix) []Ban {
return slices.Clone(*bans) return slices.Clone(*bans)
} }
// Made returns how many bans for cause the ledger has made since the // Made returns how many bans for cause have been made since the start:
// start; bans read from bans.json are not among them. // for CauseLimit and CauseAttack, by the ledger; for CauseAdmin, by an
// admin in an edit of bans.json, as LoadEdit counts them. The bans read
// from bans.json at the start are not among them.
func (l *Ledger) Made(cause string) int { func (l *Ledger) Made(cause string) int {
l.mu.Lock() l.mu.Lock()
defer l.mu.Unlock() defer l.mu.Unlock()
@@ -274,7 +293,7 @@ func (l *Ledger) Made(cause string) int {
} }
// Count returns how many of the bans held are active at now, and how many // Count returns how many of the bans held are active at now, and how many
// are permanent. // of those are permanent. A lifted ban is neither.
func (l *Ledger) Count(now time.Time) (int, int) { func (l *Ledger) Count(now time.Time) (int, int) {
l.mu.Lock() l.mu.Lock()
defer l.mu.Unlock() defer l.mu.Unlock()
@@ -283,10 +302,12 @@ func (l *Ledger) Count(now time.Time) (int, int) {
for _, bans := range l.netblocks.Values() { for _, bans := range l.netblocks.Values() {
for _, ban := range *bans { for _, ban := range *bans {
if ban.ActiveAt(now) { if !ban.ActiveAt(now) {
active++ continue
} }
active++
if ban.Permanent() { if ban.Permanent() {
permanent++ permanent++
} }
@@ -314,36 +335,81 @@ func (l *Ledger) Snapshot() []Ban {
return held return held
} }
// Load puts bans read from bans.json into the ledger, in place of the // Load puts bans read from bans.json at the start into the ledger, in
// bans it holds, in the order they started, so that a netblock whose last // place of the bans it holds, in the order they started, so that a
// ban started latest counts as the most recently seen. Each netblock is // netblock whose last ban started latest counts as the most recently
// masked to its length, so that 203.0.113.9/24 is 203.0.113.0/24, and // seen. A ban without a cause is an admin's, and gets CauseAdmin. Each
// each text in the notes is cut to 256 bytes. Past MaxBans the earliest // netblock is masked to its length, so that 203.0.113.9/24 is
// bans are dropped, as when they are made. // 203.0.113.0/24, and each text in the notes is cut to 256 bytes. Past
// MaxBans the earliest bans whose cause is not CauseAdmin are dropped, as
// when they are made.
func (l *Ledger) Load(bans []Ban) { func (l *Ledger) Load(bans []Ban) {
l.mu.Lock()
defer l.mu.Unlock()
l.load(bans)
}
// LoadEdit is Load for an admin's edit of bans.json, taken in while
// smallwebwaf runs. Each ban in it whose cause is CauseAdmin, and which
// the ledger did not hold, with the same netblock and start, is one the
// admin made, and is counted among the bans made.
func (l *Ledger) LoadEdit(bans []Ban) {
l.mu.Lock()
defer l.mu.Unlock()
l.made[CauseAdmin] += l.load(bans)
}
// load does what Load describes, and returns how many of bans are bans
// whose cause is CauseAdmin that the ledger did not hold before.
func (l *Ledger) load(bans []Ban) int {
bans = slices.Clone(bans) bans = slices.Clone(bans)
added := 0
for i := range bans {
ban := &bans[i]
ban.Netblock = ban.Netblock.Masked()
ban.Notes.Request = ban.Notes.Request.cut()
if ban.Cause == "" {
ban.Cause = CauseAdmin
}
if ban.Cause == CauseAdmin && !l.holds(ban.Netblock, ban.Start) {
added++
}
}
slices.SortStableFunc(bans, func(a, b Ban) int { slices.SortStableFunc(bans, func(a, b Ban) int {
return a.Start.Compare(b.Start) return a.Start.Compare(b.Start)
}) })
l.mu.Lock()
defer l.mu.Unlock()
l.netblocks.Purge() l.netblocks.Purge()
l.held = 0 l.held = 0
l.v4Lengths, l.v6Lengths = nil, nil l.v4Lengths, l.v6Lengths = nil, nil
for _, ban := range bans { for _, ban := range bans {
ban.Netblock = ban.Netblock.Masked()
ban.Notes.Request = ban.Notes.Request.cut()
l.add(ban) l.add(ban)
} }
return added
} }
// ban bans netblock at now for cause, with notes, as BanForLimit and // holds reports whether the ledger holds a ban on netblock that started
// BanForAttack describe, and returns the ban. // at start.
func (l *Ledger) holds(netblock netip.Prefix, start time.Time) bool {
bans, found := l.netblocks.Peek(netblock)
return found && slices.ContainsFunc(*bans, func(ban Ban) bool {
return ban.Start.Equal(start)
})
}
// ban bans netblock at now for cause, with reason and notes, as
// BanForLimit and BanForAttack describe, and returns the ban.
func (l *Ledger) ban( func (l *Ledger) ban(
netblock netip.Prefix, now time.Time, cause string, notes Notes, netblock netip.Prefix, now time.Time, cause, reason string, notes Notes,
) Ban { ) Ban {
l.mu.Lock() l.mu.Lock()
defer l.mu.Unlock() defer l.mu.Unlock()
@@ -363,7 +429,7 @@ func (l *Ledger) ban(
} }
notes.Request = notes.Request.cut() notes.Request = notes.Request.cut()
ban := Ban{Netblock: netblock, Start: now, Cause: cause, Notes: notes} ban := Ban{Netblock: netblock, Start: now, Cause: cause, Reason: reason, Notes: notes}
if cause == CauseAttack { if cause == CauseAttack {
ban.Expires = l.attackExpiry(held, now) ban.Expires = l.attackExpiry(held, now)
@@ -391,8 +457,8 @@ func earlierBans(held []Ban) EarlierBans {
earlier.Limit++ earlier.Limit++
case CauseAttack: case CauseAttack:
earlier.Attack++ earlier.Attack++
default: case CauseAdmin:
earlier.WithoutCause++ earlier.Admin++
} }
} }
@@ -431,9 +497,11 @@ func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
} }
// add adds ban to its netblock's bans, after the last, and makes its // add adds ban to its netblock's bans, after the last, and makes its
// netblock the most recently seen. With MaxBans held, it drops one first. // netblock the most recently seen. With MaxBans held, it drops one first,
// unless ban's cause is CauseAdmin, which does not count toward MaxBans.
func (l *Ledger) add(ban Ban) { func (l *Ledger) add(ban Ban) {
if l.held == l.rules.MaxBans { counted := ban.Cause != CauseAdmin
if counted && l.held == l.rules.MaxBans {
l.dropOne() l.dropOne()
} }
@@ -446,7 +514,10 @@ func (l *Ledger) add(ban Ban) {
} }
*bans = append(*bans, ban) *bans = append(*bans, ban)
if counted {
l.held++ l.held++
}
lengths := &l.v6Lengths lengths := &l.v6Lengths
if ban.Netblock.Addr().Is4() { if ban.Netblock.Addr().Is4() {
@@ -461,16 +532,17 @@ func (l *Ledger) add(ban Ban) {
// limitExpiry returns when a ban for a broken limit made at now ends, or // limitExpiry returns when a ban for a broken limit made at now ends, or
// zero when it is permanent. held are the netblock's bans, none of them // zero when it is permanent. held are the netblock's bans, none of them
// active, of which the one that ended last, other than a ban for a clear // active, of which the one that ended last, other than a ban for a clear
// sign of attack, can make the new ban longer. A ban an admin adds to // sign of attack or a lifted one, can make the new ban longer. A ban an
// bans.json can start after another and end before it, so that one is // admin adds to bans.json can start after another and end before it, so
// looked for among them all. // that one is looked for among them all.
func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time { func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
length := l.rules.LimitBanDuration length := l.rules.LimitBanDuration
var last *Ban var last *Ban
for i, ban := range held { for i, ban := range held {
if ban.Cause != CauseAttack && (last == nil || ban.Expires.After(last.Expires)) { if ban.Cause != CauseAttack && ban.Lifted.IsZero() &&
(last == nil || ban.Expires.After(last.Expires)) {
last = &held[i] last = &held[i]
} }
} }
@@ -495,11 +567,11 @@ func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
// attackExpiry returns when a ban for a clear sign of attack made at now // attackExpiry returns when a ban for a clear sign of attack made at now
// ends. held are the netblock's bans, none of them active: if one of them // ends. held are the netblock's bans, none of them active: if one of them
// is for a clear sign of attack too, the new ban is permanent, and its // is for a clear sign of attack too, and was not lifted, the new ban is
// end zero; otherwise it ends AttackBanDuration later. // permanent, and its end zero; otherwise it ends AttackBanDuration later.
func (l *Ledger) attackExpiry(held []Ban, now time.Time) time.Time { func (l *Ledger) attackExpiry(held []Ban, now time.Time) time.Time {
for _, ban := range held { for _, ban := range held {
if ban.Cause == CauseAttack { if ban.Cause == CauseAttack && ban.Lifted.IsZero() {
return time.Time{} return time.Time{}
} }
} }
@@ -507,17 +579,39 @@ func (l *Ledger) attackExpiry(held []Ban, now time.Time) time.Time {
return now.Add(l.rules.AttackBanDuration) return now.Add(l.rules.AttackBanDuration)
} }
// dropOne drops the earliest ban of the netblock that has gone longest // dropOne drops the earliest ban whose cause is not CauseAdmin of the
// without a request, and the netblock with it if that was its only ban. // netblock that has gone longest without a request, of those that hold
// such a ban, and the netblock with it if that was its only ban. It is
// called with at least one such ban held. It looks at each netblock once
// at most, and drops nothing when none holds such a ban.
func (l *Ledger) dropOne() { func (l *Ledger) dropOne() {
for range l.netblocks.Len() {
netblock, bans, _ := l.netblocks.GetOldest() netblock, bans, _ := l.netblocks.GetOldest()
i := slices.IndexFunc(*bans, func(ban Ban) bool {
return ban.Cause != CauseAdmin
})
if i < 0 {
// Its bans are all an admin's, and never dropped. Get makes
// it the most recently seen, so that the next netblock is
// looked at; when it was seen matters only for dropping a
// ban, and a ban added to it makes it the most recently seen
// anyway.
l.netblocks.Get(netblock)
continue
}
if len(*bans) == 1 { if len(*bans) == 1 {
l.netblocks.Remove(netblock) l.netblocks.Remove(netblock)
} else { } else {
*bans = slices.Delete(*bans, 0, 1) *bans = slices.Delete(*bans, i, i+1)
} }
l.held-- l.held--
return
}
} }
// cut returns r with each text cut to maxTextBytes and copied, so that // cut returns r with each text cut to maxTextBytes and copied, so that
+20 -8
View File
@@ -173,7 +173,7 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
t.Parallel() t.Parallel()
// A 9-hour ban smallwebwaf made, the third in a row, and an admin's // A 9-hour ban smallwebwaf made, the third in a row, and an admin's
// 1-hour ban added to bans.json over it, with no notes. // 1-hour ban added to bans.json over it, with no cause and no notes.
netblock := netip.MustParsePrefix("203.0.113.9/32") netblock := netip.MustParsePrefix("203.0.113.9/32")
nineHours := bans.Ban{ nineHours := bans.Ban{
Netblock: netblock, Netblock: netblock,
@@ -193,13 +193,12 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
// Once both have ended, a limit broken within the repeat window bans // Once both have ended, a limit broken within the repeat window bans
// for three times the 9 hours, and the notes count the two bans // for three times the 9 hours, and the notes count the two bans
// before the 9-hour one and it, for a limit, and the admin's, without // before the 9-hour one and it, for a limit, and the admin's.
// a cause.
ban := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{}) ban := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{})
if ban.Expires.Sub(ban.Start) != 27*time.Hour || if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, WithoutCause: 1}) { ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, "+ t.Errorf("the next ban lasts %s with earlier bans %+v, "+
"want 27h, 3 for a limit and 1 without a cause", "want 27h, 3 for a limit and 1 an admin's",
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans) ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
} }
} }
@@ -208,10 +207,15 @@ func TestLoadKeepsAtMostMaxBansDroppingTheEarliest(t *testing.T) {
t.Parallel() t.Parallel()
// bans.json lists the bans by netblock, not in the order they began. // bans.json lists the bans by netblock, not in the order they began.
later := bans.Ban{Netblock: netip.MustParsePrefix("203.0.113.1/32"), Start: midnight()} later := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.1/32"),
Start: midnight(),
Cause: bans.CauseLimit,
}
earlier := bans.Ban{ earlier := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.2/32"), Netblock: netip.MustParsePrefix("203.0.113.2/32"),
Start: midnight().Add(-time.Hour), Start: midnight().Add(-time.Hour),
Cause: bans.CauseLimit,
} }
rules := defaultRules() rules := defaultRules()
@@ -233,9 +237,17 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
rules := defaultRules() rules := defaultRules()
rules.MaxBans = 3 rules.MaxBans = 3
ledger := bans.New(rules) ledger := bans.New(rules)
kept := bans.Ban{Netblock: netip.MustParsePrefix("2001:db8::/64"), Start: midnight()} kept := bans.Ban{
Netblock: netip.MustParsePrefix("2001:db8::/64"),
Start: midnight(),
Cause: bans.CauseLimit,
}
ledger.Load([]bans.Ban{ ledger.Load([]bans.Ban{
{Netblock: netip.MustParsePrefix("203.0.113.0/24"), Start: midnight()}, {
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
Start: midnight(),
Cause: bans.CauseLimit,
},
kept, kept,
}) })
+2 -2
View File
@@ -144,7 +144,7 @@ func New(topN int) *Metrics {
func (m *Metrics) AddBansAndClients( func (m *Metrics) AddBansAndClients(
ledger *bans.Ledger, limiter *ratelimit.Limiter, now func() time.Time, ledger *bans.Ledger, limiter *ratelimit.Limiter, now func() time.Time,
) { ) {
for _, cause := range []string{bans.CauseLimit, bans.CauseAttack} { for _, cause := range []string{bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin} {
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{ m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_bans_made_total", Name: "smallwebwaf_bans_made_total",
Help: "Bans made, by cause.", Help: "Bans made, by cause.",
@@ -165,7 +165,7 @@ func (m *Metrics) AddBansAndClients(
}), }),
prometheus.NewGaugeFunc(prometheus.GaugeOpts{ prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "smallwebwaf_permanent_bans", Name: "smallwebwaf_permanent_bans",
Help: "Permanent bans.", Help: "Permanent bans not lifted.",
}, func() float64 { }, func() float64 {
_, permanent := ledger.Count(now()) _, permanent := ledger.Count(now())
+1
View File
@@ -279,6 +279,7 @@ func TestBanNotes(t *testing.T) {
Start: start, Start: start,
Expires: start.Add(time.Hour), Expires: start.Add(time.Hour),
Cause: bans.CauseLimit, Cause: bans.CauseLimit,
Reason: "requests per minute over the limit of 1",
Notes: bans.Notes{ Notes: bans.Notes{
Country: "DE", Country: "DE",
Limit: 1, Limit: 1,
+24
View File
@@ -12,6 +12,7 @@ import (
"testing" "testing"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/lookup" "sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
@@ -243,6 +244,29 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
wantMetric(t, metrics, "smallwebwaf_tracked_clients", 3) wantMetric(t, metrics, "smallwebwaf_tracked_clients", 3)
} }
func TestMetricsCountTheBansAnAdminMakes(t *testing.T) {
t.Parallel()
const scraper = "192.0.2.200" // in SWWAF_RATE_LIMIT_EXEMPT_NETS
s, clk, server := startWithClock(t, "", map[string]string{
metricsToken: token,
rateLimitExemptNets: scraper,
})
const admins = `smallwebwaf_bans_made_total{cause="admin"}`
wantMetric(t, s.scrape(scraper), admins, 0)
// As an admin's edit of bans.json that adds a ban is taken in.
server.Ledger.LoadEdit([]bans.Ban{{
Netblock: netip.MustParsePrefix(client + "/32"),
Start: clk.Now(),
}})
wantMetric(t, s.scrape(scraper), admins, 1)
}
func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) { func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
t.Parallel() t.Parallel()
+1
View File
@@ -95,6 +95,7 @@ func TestObserveModeMakesNoBanAndKeepsTheBansItHas(t *testing.T) {
Netblock: netip.MustParsePrefix(otherClient + "/32"), Netblock: netip.MustParsePrefix(otherClient + "/32"),
Start: clk.Now(), Start: clk.Now(),
Expires: clk.Now().Add(time.Hour), Expires: clk.Now().Add(time.Hour),
Cause: bans.CauseAdmin,
} }
server.Ledger.Load([]bans.Ban{kept}) server.Ledger.Load([]bans.Ban{kept})
+1
View File
@@ -55,6 +55,7 @@ func TestEachRuleAction(t *testing.T) {
Start: start, Start: start,
Expires: start.Add(7 * 24 * time.Hour), Expires: start.Add(7 * 24 * time.Hour),
Cause: bans.CauseAttack, Cause: bans.CauseAttack,
Reason: "matched the rule probe",
Notes: bans.Notes{ Notes: bans.Notes{
RuleID: "probe", RuleID: "probe",
Target: "path", Target: "path",
+35 -13
View File
@@ -48,7 +48,7 @@ var (
errVersion = errors.New("unknown version") errVersion = errors.New("unknown version")
// errMissing is for an entry without a field it needs. // errMissing is for an entry without a field it needs.
errMissing = errors.New("has no") errMissing = errors.New("has no")
errCause = errors.New("is not limit or attack") errCause = errors.New("is not limit, attack or admin")
) )
// Params are what Load needs. // Params are what Load needs.
@@ -96,12 +96,15 @@ type bansFile struct {
} }
// banEntry is a ban as bans.json holds it: a permanent ban's expires is // banEntry is a ban as bans.json holds it: a permanent ban's expires is
// null, and a ban an admin added may have no cause. // null, a ban an admin added may have no cause, which makes it an
// admin's, and lifted is left out until an admin lifts the ban.
type banEntry struct { type banEntry struct {
Netblock netip.Prefix `json:"netblock"` Netblock netip.Prefix `json:"netblock"`
Start time.Time `json:"start"` Start time.Time `json:"start"`
Expires *time.Time `json:"expires"` Expires *time.Time `json:"expires"`
Cause string `json:"cause,omitempty"` Cause string `json:"cause"`
Reason string `json:"reason,omitempty"`
Lifted *time.Time `json:"lifted,omitempty"`
Notes bans.Notes `json:"notes"` Notes bans.Notes `json:"notes"`
} }
@@ -262,7 +265,7 @@ func (f *Files) fileChanged(name string) {
// runs, by Watch or by a write, is taken in here. An edit that does not // runs, by Watch or by a write, is taken in here. An edit that does not
// parse is neither counted nor logged, and takeIn's error returned. // parse is neither counted nor logged, and takeIn's error returned.
func (f *Files) takeInEdit(name string, data []byte) error { func (f *Files) takeInEdit(name string, data []byte) error {
_, err := f.takeIn(name, data) _, err := f.takeIn(name, data, true)
if err != nil { if err != nil {
return err return err
} }
@@ -284,7 +287,7 @@ func (f *Files) read(name string) (int, error) {
return 0, err return 0, err
} }
return f.takeIn(name, data) return f.takeIn(name, data, false)
} }
// readChanged returns what the state file name holds, and whether that // readChanged returns what the state file name holds, and whether that
@@ -308,9 +311,11 @@ func (f *Files) readChanged(name string) ([]byte, bool, error) {
// takeIn parses data, what the state file name holds, puts it into the // takeIn parses data, what the state file name holds, puts it into the
// part that keeps that state, in place of what the part held, and returns // part that keeps that state, in place of what the part held, and returns
// how many entries the file holds. An error names the file and, where the // how many entries the file holds. edit is whether data is an admin's
// JSON decoder tells it, the line and column, or else the entry. // edit taken in while smallwebwaf runs, rather than the file read at the
func (f *Files) takeIn(name string, data []byte) (int, error) { // start. An error names the file and, where the JSON decoder tells it,
// the line and column, or else the entry.
func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
path := filepath.Join(f.params.Dir, name) path := filepath.Join(f.params.Dir, name)
var entries int var entries int
@@ -329,7 +334,12 @@ func (f *Files) takeIn(name string, data []byte) (int, error) {
held = append(held, entry.ban()) held = append(held, entry.ban())
} }
if edit {
f.params.Ledger.LoadEdit(held)
} else {
f.params.Ledger.Load(held) f.params.Ledger.Load(held)
}
entries = len(held) entries = len(held)
case clientsJSON: case clientsJSON:
var file clientsFile var file clientsFile
@@ -447,22 +457,34 @@ func (f *Files) encode(name string) ([]byte, error) {
// newBanEntry returns ban as bans.json holds it. // newBanEntry returns ban as bans.json holds it.
func newBanEntry(ban bans.Ban) banEntry { func newBanEntry(ban bans.Ban) banEntry {
entry := banEntry{ entry := banEntry{
Netblock: ban.Netblock, Start: ban.Start, Cause: ban.Cause, Notes: ban.Notes, Netblock: ban.Netblock, Start: ban.Start, Cause: ban.Cause, Reason: ban.Reason,
Notes: ban.Notes,
} }
if !ban.Permanent() { if !ban.Permanent() {
entry.Expires = &ban.Expires entry.Expires = &ban.Expires
} }
if !ban.Lifted.IsZero() {
entry.Lifted = &ban.Lifted
}
return entry return entry
} }
// ban returns the ban an entry of bans.json holds. // ban returns the ban an entry of bans.json holds.
func (e banEntry) ban() bans.Ban { func (e banEntry) ban() bans.Ban {
ban := bans.Ban{Netblock: e.Netblock, Start: e.Start, Cause: e.Cause, Notes: e.Notes} ban := bans.Ban{
Netblock: e.Netblock, Start: e.Start, Cause: e.Cause, Reason: e.Reason,
Notes: e.Notes,
}
if e.Expires != nil { if e.Expires != nil {
ban.Expires = *e.Expires ban.Expires = *e.Expires
} }
if e.Lifted != nil {
ban.Lifted = *e.Lifted
}
return ban return ban
} }
@@ -470,8 +492,8 @@ func (e banEntry) ban() bans.Ban {
// client, a start, from which the length of the netblock's next ban is // client, a start, from which the length of the netblock's next ban is
// worked out, or an expires, which would make it permanent. A permanent // worked out, or an expires, which would make it permanent. A permanent
// ban's expires is null, which Bans cannot tell from a missing one, so // ban's expires is null, which Bans cannot tell from a missing one, so
// each expires is read again as written. A cause other than limit or // each expires is read again as written. A cause other than limit,
// attack, most likely misspelt, is refused too. // attack or admin, most likely misspelt, is refused too.
func (f *bansFile) check(data []byte) error { func (f *bansFile) check(data []byte) error {
var written struct { var written struct {
Bans []struct { Bans []struct {
@@ -493,7 +515,7 @@ func (f *bansFile) check(data []byte) error {
case written.Bans[i].Expires == nil: case written.Bans[i].Expires == nil:
return missing(i, "expires") return missing(i, "expires")
case entry.Cause != "" && entry.Cause != bans.CauseLimit && case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
entry.Cause != bans.CauseAttack: entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin:
return fmt.Errorf("entry %d's cause %q %w", i+1, entry.Cause, errCause) return fmt.Errorf("entry %d's cause %q %w", i+1, entry.Cause, errCause)
} }
} }
+170 -8
View File
@@ -48,6 +48,8 @@ const permanentBansJSON = `{
"netblock": "2001:db8::/64", "netblock": "2001:db8::/64",
"start": "2026-10-06T00:00:00Z", "start": "2026-10-06T00:00:00Z",
"expires": null, "expires": null,
"cause": "admin",
"reason": "scrapes every commit",
"notes": { "notes": {
"country": "DE", "country": "DE",
"limit": 1000, "limit": 1000,
@@ -66,7 +68,7 @@ const permanentBansJSON = `{
"earlier_bans": { "earlier_bans": {
"limit": 3, "limit": 3,
"attack": 1, "attack": 1,
"without_cause": 1 "admin": 1
} }
} }
} }
@@ -74,6 +76,15 @@ const permanentBansJSON = `{
} }
` `
// liftedClient is the client whose ban liftedBansJSON holds.
const liftedClient = "203.0.113.9"
// liftedBansJSON is bans.json holding an hour's ban for a broken limit on
// liftedClient, from midnight, that an admin lifted ten minutes in.
const liftedBansJSON = `{"version": 1, "bans": [{"netblock": "203.0.113.9/32", ` +
`"start": "2026-10-06T00:00:00Z", "expires": "2026-10-06T01:00:00Z", ` +
`"cause": "limit", "lifted": "2026-10-06T00:10:00Z"}]}`
func TestFilesWrittenAndReadBack(t *testing.T) { func TestFilesWrittenAndReadBack(t *testing.T) {
t.Parallel() t.Parallel()
@@ -267,15 +278,17 @@ func TestEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
} }
} }
func TestBanWithACauseSmallwebwafDoesNotGiveStopsTheStart(t *testing.T) { func TestBanWithAnotherCauseStopsTheStart(t *testing.T) {
t.Parallel() t.Parallel()
wantRefused(t, bansJSON, `{"version": 1, "bans": [`+ wantRefused(t, bansJSON, `{"version": 1, "bans": [`+
`{"netblock": "203.0.113.9/32", "start": "2026-10-06T00:00:00Z", `+ `{"netblock": "203.0.113.9/32", "start": "2026-10-06T00:00:00Z", `+
`"expires": null, "cause": "attack"}, `+ `"expires": null, "cause": "attack"}, `+
`{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+ `{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+
`"expires": null, "cause": "admin"}, `+
`{"netblock": "203.0.113.11/32", "start": "2026-10-06T00:00:00Z", `+
`"expires": null, "cause": "atack"}]}`, `"expires": null, "cause": "atack"}]}`,
`: entry 2's cause "atack" is not limit or attack`) `: entry 3's cause "atack" is not limit, attack or admin`)
} }
func TestUnknownVersionStopsTheStart(t *testing.T) { func TestUnknownVersionStopsTheStart(t *testing.T) {
@@ -611,7 +624,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
`"start": "2026-10-06T00:00:00Z", "expires": null}]}`) `"start": "2026-10-06T00:00:00Z", "expires": null}]}`)
wantTakenIn(t, lines, dir, bansJSON) wantTakenIn(t, lines, dir, bansJSON)
wantEqual(t, bansJSON, params.Ledger.Snapshot(), wantEqual(t, bansJSON, params.Ledger.Snapshot(),
[]bans.Ban{{Netblock: client, Start: midnight()}}) []bans.Ban{{Netblock: client, Start: midnight(), Cause: bans.CauseAdmin}})
edit(t, dir, clientsJSON, `{"version": 1, "clients": [`+ edit(t, dir, clientsJSON, `{"version": 1, "clients": [`+
`{"client": "198.51.100.7/32", "history": {"requests": 7}}]}`) `{"client": "198.51.100.7/32", "history": {"requests": 7}}]}`)
@@ -710,6 +723,100 @@ func TestBanAddedAndLiftedThroughBansJSON(t *testing.T) {
} }
} }
func TestBanWithoutACauseTakenInAsAnAdmins(t *testing.T) {
t.Parallel()
const reason = "probes for logins"
// adminsBansJSON is bans.json as an admin writes it, with a ban on
// netblock without a cause, and adminsBans the bans it holds.
adminsBansJSON := func(netblock string) string {
return `{"version": 1, "bans": [{"netblock": "` + netblock + `", ` +
`"start": "2026-10-06T00:00:00Z", "expires": null, "reason": "` +
reason + `"}]}`
}
adminsBans := func(netblock string) []bans.Ban {
return []bans.Ban{{
Netblock: netip.MustParsePrefix(netblock),
Start: midnight(),
Cause: bans.CauseAdmin,
Reason: reason,
}}
}
// Read at the start, the ban is taken in as an admin's, though not
// counted among the bans made since the start, and written back with
// that cause and the admin's reason.
dir := t.TempDir()
edit(t, dir, bansJSON, adminsBansJSON("203.0.113.0/24"))
params := newParams(dir)
lines := logInto(&params)
files := load(t, params)
wantEqual(t, bansJSON, params.Ledger.Snapshot(), adminsBans("203.0.113.0/24"))
wantMadeByAnAdmin(t, params.Ledger, 0)
err := files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
var written struct {
Bans []struct {
Cause string `json:"cause"`
Reason string `json:"reason"`
} `json:"bans"`
}
err = json.Unmarshal([]byte(readFile(t, filepath.Join(dir, bansJSON))), &written)
if err != nil || len(written.Bans) != 1 || written.Bans[0].Cause != bans.CauseAdmin ||
written.Bans[0].Reason != reason {
t.Errorf("bans.json holds %+v (%v), want the ban with the cause admin "+
"and the reason %q", written, err, reason)
}
// Taken in while smallwebwaf runs, a ban on another netblock is an
// admin's too, and one made since the start.
watch(t, files, lines)
edit(t, dir, bansJSON, adminsBansJSON("198.51.100.0/24"))
wantTakenIn(t, lines, dir, bansJSON)
wantEqual(t, bansJSON, params.Ledger.Snapshot(), adminsBans("198.51.100.0/24"))
wantMadeByAnAdmin(t, params.Ledger, 1)
}
func TestLiftedBanReadAtTheStart(t *testing.T) {
t.Parallel()
dir := t.TempDir()
edit(t, dir, bansJSON, liftedBansJSON)
params := newParams(dir)
wantLiftedBanKept(t, load(t, params), dir, params.Ledger)
}
func TestBanLiftedByAnEditWhileRunning(t *testing.T) {
t.Parallel()
dir := t.TempDir()
params := newParams(dir)
lines := logInto(&params)
files := load(t, params)
watch(t, files, lines)
// The ban that liftedBansJSON lifts, before it is lifted.
netblock := netip.MustParsePrefix(liftedClient + "/32")
params.Ledger.BanForLimit(netblock, midnight(), bans.Notes{})
_, banned := params.Ledger.Find(netblock.Addr(), afterLifting())
if !banned {
t.Fatal("the ban does not refuse before it is lifted")
}
edit(t, dir, bansJSON, liftedBansJSON)
wantTakenIn(t, lines, dir, bansJSON)
wantLiftedBanKept(t, files, dir, params.Ledger)
}
func TestBrokenEditSetAsideAtTheNextWrite(t *testing.T) { func TestBrokenEditSetAsideAtTheNextWrite(t *testing.T) {
t.Parallel() t.Parallel()
@@ -909,9 +1016,9 @@ func newParams(dir string) state.Params {
} }
} }
// fill puts a permanent ban without a cause, as an admin adds one, a ban // fill puts a permanent ban an admin made, a ban for a broken limit and
// for a broken limit and one for a clear sign of attack, clients with // one for a clear sign of attack, clients with counts and histories, and
// counts and histories, and GeoJS answers into the parts of params. // GeoJS answers into the parts of params.
func fill(params state.Params) { func fill(params state.Params) {
now := midnight() now := midnight()
client := netip.MustParsePrefix("203.0.113.9/32") client := netip.MustParsePrefix("203.0.113.9/32")
@@ -943,6 +1050,8 @@ func permanentBan() bans.Ban {
return bans.Ban{ return bans.Ban{
Netblock: netip.MustParsePrefix("2001:db8::/64"), Netblock: netip.MustParsePrefix("2001:db8::/64"),
Start: midnight(), Start: midnight(),
Cause: bans.CauseAdmin,
Reason: "scrapes every commit",
Notes: bans.Notes{ Notes: bans.Notes{
Country: "DE", Country: "DE",
Limit: 1000, Limit: 1000,
@@ -958,11 +1067,64 @@ func permanentBan() bans.Ban {
}, },
Requests: 1500, Requests: 1500,
Refused: 3, Refused: 3,
EarlierBans: bans.EarlierBans{Limit: 3, Attack: 1, WithoutCause: 1}, EarlierBans: bans.EarlierBans{Limit: 3, Attack: 1, Admin: 1},
}, },
} }
} }
// afterLifting is a time after the ban liftedBansJSON holds was lifted,
// while it would still last.
func afterLifting() time.Time {
return midnight().Add(30 * time.Minute)
}
// wantLiftedBanKept checks that ledger holds the ban liftedBansJSON holds,
// which refuses nothing and does not make the next ban for a broken limit
// longer, and that files write it to bans.json, in dir, still lifted.
func wantLiftedBanKept(
t *testing.T, files *state.Files, dir string, ledger *bans.Ledger,
) {
t.Helper()
err := files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
const lifted = `"lifted": "2026-10-06T00:10:00Z"`
if got := readFile(t, filepath.Join(dir, bansJSON)); !strings.Contains(got, lifted) {
t.Errorf("bans.json holds\n%s\nwant the ban with %s", got, lifted)
}
netblock := netip.MustParsePrefix(liftedClient + "/32")
_, banned := ledger.Check(netblock.Addr(), afterLifting())
if banned {
t.Error("the lifted ban refuses")
}
// Were the lifted ban counted, the next would last three hours.
ban := ledger.BanForLimit(netblock, afterLifting(), bans.Notes{})
if ban.Expires.Sub(ban.Start) != time.Hour {
t.Errorf("the next ban lasts %s, want 1h", ban.Expires.Sub(ban.Start))
}
held := ledger.Bans(netblock)
if len(held) != 2 || !held[0].Lifted.Equal(midnight().Add(10*time.Minute)) {
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
}
}
// wantMadeByAnAdmin checks how many bans ledger counts as made by an
// admin since the start.
func wantMadeByAnAdmin(t *testing.T, ledger *bans.Ledger, want int) {
t.Helper()
if got := ledger.Made(bans.CauseAdmin); got != want {
t.Errorf("%d bans made by an admin, want %d", got, want)
}
}
// load reads the state files into the parts of params. // load reads the state files into the parts of params.
func load(t *testing.T, params state.Params) *state.Files { func load(t *testing.T, params state.Params) *state.Files {
t.Helper() t.Helper()