Bans an admin makes or lifts: the admin cause, a reason, lifted bans kept (closes #86)
check / check (push) Successful in 3m4s
check / check (push) Successful in 3m4s
A bans.json entry without a cause gets the cause admin, written back so. Bans whose cause is admin are never dropped and do not count toward SWWAF_MAX_BANS, so setting a ban's cause to admin keeps it. Bans smallwebwaf makes get a reason: the limit broken or the rule matched. A lifted ban refuses nothing, is kept, and makes no later ban longer. smallwebwaf_bans_made_total counts admin bans an edit adds while running; earlier_bans counts admin in place of without_cause. Judgement call: lifted lifts at once, whatever time it gives. Judgement call: a lifted ban still counts in earlier_bans. Known gap: a ban dropped from behind an admin's ban on its netblock leaves that netblock's later earlier_bans. Model: opus-5-5
This commit is contained in:
@@ -13,29 +13,30 @@ JSON log line for every request.
|
|||||||
|
|
||||||
Status: the first two milestones are built
|
Status: the first two milestones are built
|
||||||
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
|
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
|
||||||
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are eight parts of
|
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are nine parts of
|
||||||
milestone 3: the static lists, the bans that broken rate limits lead to, the
|
milestone 3: the static lists, the bans that broken rate limits lead to, the ban
|
||||||
JSON state files with your edits taken in while it runs and the paths the rate
|
ledger with the bans you make, keep and lift, the JSON state files with your
|
||||||
limits do not count, which come next in the build order, `observe` mode and the
|
edits taken in while it runs and the paths the rate limits do not count, which
|
||||||
rest of the request log's fields, which come a little later, and the metrics
|
come next in the build order, `observe` mode and the rest of the request log's
|
||||||
endpoint and the header size and the idle time as settings, which come last in
|
fields, which come a little later, and the metrics endpoint and the header size
|
||||||
it. So are two parts of the stage after it: the rule files, the first part, with
|
and the idle time as settings, which come last in it. So are two parts of the
|
||||||
the bans for a clear sign of attack, and remote log sending. `smallwebwaf`
|
stage after it: the rule files, the first part, with the bans for a clear sign
|
||||||
passes each request to the app and the app's answer back, unchanged, within its
|
of attack, and remote log sending. `smallwebwaf` passes each request to the app
|
||||||
timeouts and size limits, works out each client's address, bans a client that
|
and the app's answer back, unchanged, within its timeouts and size limits, works
|
||||||
sends too many requests, not counting those for the paths you choose, refuses a
|
out each client's address, bans a client that sends too many requests, not
|
||||||
client that comes from a country you refuse or from a network you refuse, lets
|
counting those for the paths you choose, refuses a client that comes from a
|
||||||
the networks you choose through, checks each request against the rule files and
|
country you refuse or from a network you refuse, lets the networks you choose
|
||||||
bans a client whose request is a clear sign of attack, keeps its bans, each
|
through, checks each request against the rule files and bans a client whose
|
||||||
client's counters and history, and GeoJS's answers in JSON files across
|
request is a clear sign of attack, keeps its bans, each client's counters and
|
||||||
restarts, takes in your edits of those files and of the rule files while it
|
history, and GeoJS's answers in JSON files across restarts, takes in your edits
|
||||||
runs, writes a JSON log line for every request, sends its log lines to a syslog
|
of those files, such as a ban you make, keep or lift, and of the rule files
|
||||||
server too if you name one, serves Prometheus metrics to a scraper that holds
|
while it runs, writes a JSON log line for every request, sends its log lines to
|
||||||
the metrics token, and in `observe` mode passes on the requests it would refuse,
|
a syslog server too if you name one, serves Prometheus metrics to a scraper that
|
||||||
logging what it would have done with them. It comes as the image the app's own
|
holds the metrics token, and in `observe` mode passes on the requests it would
|
||||||
image is built on. The rest of the design comes after that, in the order of the
|
refuse, logging what it would have done with them. It comes as the image the
|
||||||
build order in [`SPEC.md`](SPEC.md). The survey of existing tools that led to
|
app's own image is built on. The rest of the design comes after that, in the
|
||||||
the design is in [`EVALUATION.md`](EVALUATION.md).
|
order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
|
||||||
|
that led to the design is in [`EVALUATION.md`](EVALUATION.md).
|
||||||
|
|
||||||
## Getting started
|
## Getting started
|
||||||
|
|
||||||
@@ -111,11 +112,13 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
|||||||
window and the requests counted in it, the request that broke it, the client's
|
window and the requests counted in it, the request that broke it, the client's
|
||||||
country when it was looked up, the netblock's requests since it was first
|
country when it was looked up, the netblock's requests since it was first
|
||||||
seen, how many of them the ban has refused, and how many bans the netblock had
|
seen, how many of them the ban has refused, and how many bans the netblock had
|
||||||
before, for a broken limit, for a clear sign of attack and without a cause. At
|
before, for a broken limit, for a clear sign of attack and by an admin. At
|
||||||
most `SWWAF_MAX_BANS` bans are kept, past, active and permanent; past that,
|
most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept, past, active and
|
||||||
the earliest ban of the netblock that has gone longest without a request is
|
permanent; past that, the earliest such ban of the netblock that has gone
|
||||||
dropped first. `bans.json` shows the bans and their notes, a restart lifts
|
longest without a request is dropped first. The bans whose cause is `admin`,
|
||||||
none, and you add or lift a ban by editing it (see "State files" below).
|
those you make or keep, are kept besides, and never dropped. `bans.json` shows
|
||||||
|
the bans and their notes, a restart lifts none, and you make, keep or lift a
|
||||||
|
ban by editing it (see "State files" below).
|
||||||
- Checks each request against the rules of the rule files (see "Rule files"
|
- Checks each request against the rules of the rule files (see "Rule files"
|
||||||
below) after the rate limits, and before its body is read. A `log` rule that
|
below) after the rate limits, and before its body is read. A `log` rule that
|
||||||
matches is noted in the log line; a `block` rule refuses the request with
|
matches is noted in the log line; a `block` rule refuses the request with
|
||||||
@@ -263,8 +266,8 @@ it, and the effective settings are logged at start.
|
|||||||
would be longer is permanent instead.
|
would be longer is permanent instead.
|
||||||
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
||||||
attack.
|
attack.
|
||||||
- `SWWAF_MAX_BANS` (default `5000`): the most bans kept, past, active and
|
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
|
||||||
permanent.
|
kept, past, active and permanent. The bans you make or keep are kept besides.
|
||||||
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
|
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
|
||||||
an IPv4 client that a ban covers, such as `24` to ban the surrounding /24. An
|
an IPv4 client that a ban covers, such as `24` to ban the surrounding /24. An
|
||||||
IPv6 ban covers the client's /64.
|
IPv6 ban covers the client's /64.
|
||||||
@@ -458,9 +461,14 @@ them.
|
|||||||
[`SPEC.md`](SPEC.md) describes. Each has a top-level `version`, 1, and lists its
|
[`SPEC.md`](SPEC.md) describes. Each has a top-level `version`, 1, and lists its
|
||||||
entries by client address, with times in UTC.
|
entries by client address, with times in UTC.
|
||||||
|
|
||||||
- `bans.json`: every ban with its notes, indented to be read; a permanent ban's
|
- `bans.json`: every ban with its notes, indented to be read. A permanent ban's
|
||||||
`expires` is `null`, and a ban `smallwebwaf` made has the `cause` `limit` for
|
`expires` is `null`. A ban's `cause` is `limit` for a broken rate limit or
|
||||||
a broken rate limit or `attack` for a clear sign of attack.
|
`attack` for a clear sign of attack, for a ban `smallwebwaf` made, and `admin`
|
||||||
|
for one you made or keep. Its `reason` is a short text: for a ban
|
||||||
|
`smallwebwaf` made, the limit broken, such as
|
||||||
|
`requests per minute over the limit of 1000`, or the rule that matched, such
|
||||||
|
as `matched the rule env-file`; for yours, what you wrote. Its `lifted` is
|
||||||
|
when you lifted it, and is left out until you do.
|
||||||
- `clients.json`: each client's two buckets in the minute, the hour and the day,
|
- `clients.json`: each client's two buckets in the minute, the hour and the day,
|
||||||
and its history: when it was first and last seen, its country as last looked
|
and its history: when it was first and last seen, its country as last looked
|
||||||
up and when, its requests, how many were forwarded and how many refused (one
|
up and when, its requests, how many were forwarded and how many refused (one
|
||||||
@@ -492,8 +500,8 @@ without a field it needs, named with the entry's place in the file: a ban's
|
|||||||
`netblock`, `start` or `expires`, which is `null` for a permanent ban; a
|
`netblock`, `start` or `expires`, which is `null` for a permanent ban; a
|
||||||
client's `client`, or the `start` of a window in which it has requests; an
|
client's `client`, or the `start` of a window in which it has requests; an
|
||||||
answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or
|
answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or
|
||||||
`answered`. So does a ban whose `cause` is neither `limit` nor `attack`. The AS
|
`answered`. So does a ban whose `cause` is not `limit`, `attack` or `admin`. The
|
||||||
number and AS name come with their lookup.
|
AS number and AS name come with their lookup.
|
||||||
|
|
||||||
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
||||||
a state file as soon as you save it: what the file then holds replaces what
|
a state file as soon as you save it: what the file then holds replaces what
|
||||||
@@ -511,10 +519,12 @@ before the editor has finished writing it. Mend the `.bad` file and move it
|
|||||||
back. A file you remove is written again at its next write.
|
back. A file you remove is written again at its next write.
|
||||||
|
|
||||||
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
|
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
|
||||||
and its `expires`, `null` for a ban that never ends; its `cause` and its `notes`
|
and its `expires`, `null` for a ban that never ends; its `reason` and its
|
||||||
may be left out. A ban whose `cause` is `attack` becomes permanent at the first
|
`notes` may be left out, and so may its `cause`, which is then `admin`, and is
|
||||||
request it refuses; one without a cause does not. This `bans.json` bans
|
written so at the file's next write. A ban whose `cause` is `admin` is never
|
||||||
`203.0.113.0/24` for good:
|
dropped and does not count toward `SWWAF_MAX_BANS`. A ban whose `cause` is
|
||||||
|
`attack` becomes permanent at the first request it refuses; one whose `cause` is
|
||||||
|
`admin` does not. This `bans.json` bans `203.0.113.0/24` for good:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
@@ -523,14 +533,22 @@ request it refuses; one without a cause does not. This `bans.json` bans
|
|||||||
{
|
{
|
||||||
"netblock": "203.0.113.0/24",
|
"netblock": "203.0.113.0/24",
|
||||||
"start": "2026-10-06T12:00:00Z",
|
"start": "2026-10-06T12:00:00Z",
|
||||||
"expires": null
|
"expires": null,
|
||||||
|
"reason": "probes for logins"
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
To lift a ban, delete its entry. `smallwebwaf` then forgets the ban, so it does
|
To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause`
|
||||||
not make the netblock's next ban longer.
|
to `admin`: `"cause": "admin"`.
|
||||||
|
|
||||||
|
To lift a ban, add `lifted` to its entry, with the time you lift it, such as
|
||||||
|
`"lifted": "2026-10-06T13:00:00Z"`. From when the edit is taken in, the ban
|
||||||
|
refuses nothing, whatever time `lifted` gives, and does not make the netblock's
|
||||||
|
next ban longer; it is kept in `bans.json` with its notes, as any other ban is.
|
||||||
|
To forget a ban altogether, delete its entry: it then refuses nothing either,
|
||||||
|
and does not make the netblock's next ban longer.
|
||||||
|
|
||||||
## Rule files
|
## Rule files
|
||||||
|
|
||||||
@@ -624,8 +642,10 @@ other request. No metric carries a client's address.
|
|||||||
- `smallwebwaf_rate_limit_hits_total` by `window`,
|
- `smallwebwaf_rate_limit_hits_total` by `window`,
|
||||||
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
|
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
|
||||||
limit was passed, `smallwebwaf_offences_total` by `kind`, and
|
limit was passed, `smallwebwaf_offences_total` by `kind`, and
|
||||||
`smallwebwaf_bans_made_total` by `cause`, `limit` or `attack`;
|
`smallwebwaf_bans_made_total` by `cause`, `limit`, `attack` or `admin`, the
|
||||||
`smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
|
last for the bans whose `cause` is `admin` that you add to `bans.json` while
|
||||||
|
`smallwebwaf` runs; `smallwebwaf_active_bans` and
|
||||||
|
`smallwebwaf_permanent_bans`, neither of which counts a lifted ban.
|
||||||
- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
|
- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
|
||||||
`rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
|
`rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
|
||||||
rules read from the rule files.
|
rules read from the rule files.
|
||||||
@@ -954,7 +974,7 @@ addresses are never sent to GeoJS.
|
|||||||
happened, and served in the Prometheus text format.
|
happened, and served in the Prometheus text format.
|
||||||
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how
|
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how
|
||||||
long a new ban lasts, when a ban for a clear sign of attack becomes permanent,
|
long a new ban lasts, when a ban for a clear sign of attack becomes permanent,
|
||||||
and which ban is dropped when `SWWAF_MAX_BANS` are held.
|
and which ban `smallwebwaf` made is dropped when `SWWAF_MAX_BANS` are held.
|
||||||
- `internal/rules`: reads the rule files at start and again as they change, and
|
- `internal/rules`: reads the rule files at start and again as they change, and
|
||||||
tells which of their rules a request matches.
|
tells which of their rules a request matches.
|
||||||
- `internal/lookup`: looks up each client's country through GeoJS, and keeps the
|
- `internal/lookup`: looks up each client's country through GeoJS, and keeps the
|
||||||
@@ -977,8 +997,9 @@ addresses are never sent to GeoJS.
|
|||||||
checks.
|
checks.
|
||||||
|
|
||||||
Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
|
Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
|
||||||
table of clients to 20,000, the GeoJS answers to 100,000 and the banned
|
table of clients to 20,000 and the GeoJS answers to 100,000, dropping the least
|
||||||
netblocks to `SWWAF_MAX_BANS`, dropping the least recently seen, and
|
recently seen, and the banned netblocks in the order they were last seen, from
|
||||||
|
which the ledger picks the ban to drop past `SWWAF_MAX_BANS`, and
|
||||||
`github.com/prometheus/client_golang` keeps the metrics and serves them, and
|
`github.com/prometheus/client_golang` keeps the metrics and serves them, and
|
||||||
`github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file or a rule
|
`github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file or a rule
|
||||||
file is saved. The country codes are the list in `internal/config/config.go`.
|
file is saved. The country codes are the list in `internal/config/config.go`.
|
||||||
|
|||||||
@@ -0,0 +1,186 @@
|
|||||||
|
package bans_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/netip"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestBanWithoutACauseIsAnAdmins(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
netblock := netip.MustParsePrefix("203.0.113.0/24")
|
||||||
|
ledger := bans.New(defaultRules())
|
||||||
|
ledger.Load([]bans.Ban{{Netblock: netblock, Start: midnight()}})
|
||||||
|
|
||||||
|
if got := ledger.Bans(netblock)[0].Cause; got != bans.CauseAdmin {
|
||||||
|
t.Errorf("the ban's cause is %q, want admin", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
rules := defaultRules()
|
||||||
|
rules.MaxBans = 1
|
||||||
|
ledger := bans.New(rules)
|
||||||
|
adminsOnly := netip.MustParsePrefix("198.51.100.0/24")
|
||||||
|
both := netip.MustParsePrefix("203.0.113.1/32")
|
||||||
|
second := netip.MustParsePrefix("203.0.113.2/32")
|
||||||
|
third := netip.MustParsePrefix("203.0.113.3/32")
|
||||||
|
|
||||||
|
// Seen longest ago, a netblock with two of an admin's bans alone, and
|
||||||
|
// then one with an admin's ban before a ban smallwebwaf made: the one
|
||||||
|
// ban counted toward MaxBans.
|
||||||
|
ledger.Load([]bans.Ban{
|
||||||
|
{Netblock: adminsOnly, Start: midnight().Add(-3 * time.Hour), Cause: bans.CauseAdmin},
|
||||||
|
{Netblock: adminsOnly, Start: midnight().Add(-2 * time.Hour), Cause: bans.CauseAdmin},
|
||||||
|
{Netblock: both, Start: midnight().Add(-time.Hour), Cause: bans.CauseAdmin},
|
||||||
|
{
|
||||||
|
Netblock: both,
|
||||||
|
Start: midnight(),
|
||||||
|
Expires: midnight().Add(time.Hour),
|
||||||
|
Cause: bans.CauseLimit,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 2})
|
||||||
|
|
||||||
|
// A new ban drops the ban smallwebwaf made, and only that one.
|
||||||
|
ledger.BanForLimit(second, midnight(), bans.Notes{})
|
||||||
|
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1})
|
||||||
|
|
||||||
|
if ledger.Bans(both)[0].Cause != bans.CauseAdmin {
|
||||||
|
t.Errorf("%s kept %+v, want the admin's ban", both, ledger.Bans(both))
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the next drops that one.
|
||||||
|
ledger.BanForLimit(third, midnight(), bans.Notes{})
|
||||||
|
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReasonOfTheBansSmallwebwafMakes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
ledger := bans.New(defaultRules())
|
||||||
|
|
||||||
|
limit := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
|
||||||
|
bans.Notes{Limit: 1000, Window: "minute"})
|
||||||
|
attack := ledger.BanForAttack(netip.MustParsePrefix("203.0.113.2/32"), midnight(),
|
||||||
|
bans.Notes{RuleID: "git-dir", Target: "path"})
|
||||||
|
|
||||||
|
for _, tc := range []struct{ got, want string }{
|
||||||
|
{limit.Reason, "requests per minute over the limit of 1000"},
|
||||||
|
{attack.Reason, "matched the rule git-dir"},
|
||||||
|
} {
|
||||||
|
if tc.got != tc.want {
|
||||||
|
t.Errorf("the reason is %q, want %q", tc.got, tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// An hour's ban lifted ten minutes after it started.
|
||||||
|
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
|
lifted := bans.Ban{
|
||||||
|
Netblock: netblock,
|
||||||
|
Start: midnight(),
|
||||||
|
Expires: midnight().Add(time.Hour),
|
||||||
|
Cause: bans.CauseLimit,
|
||||||
|
Lifted: midnight().Add(10 * time.Minute),
|
||||||
|
}
|
||||||
|
|
||||||
|
ledger := bans.New(defaultRules())
|
||||||
|
ledger.Load([]bans.Ban{lifted})
|
||||||
|
|
||||||
|
// While it would still last, it refuses nothing, and a limit broken
|
||||||
|
// bans for an hour, as a first broken limit does; the lifted ban is
|
||||||
|
// kept, and counted among the earlier bans.
|
||||||
|
now := midnight().Add(30 * time.Minute)
|
||||||
|
|
||||||
|
_, banned := ledger.Check(netblock.Addr(), now)
|
||||||
|
if banned {
|
||||||
|
t.Error("the lifted ban refuses")
|
||||||
|
}
|
||||||
|
|
||||||
|
ban := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||||
|
if ban.Expires.Sub(ban.Start) != time.Hour ||
|
||||||
|
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
||||||
|
t.Errorf("the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit",
|
||||||
|
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
|
||||||
|
}
|
||||||
|
|
||||||
|
held := ledger.Bans(netblock)
|
||||||
|
if len(held) != 2 || held[0] != lifted {
|
||||||
|
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// A permanent ban for a clear sign of attack, lifted.
|
||||||
|
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
|
ledger := bans.New(defaultRules())
|
||||||
|
ledger.Load([]bans.Ban{{
|
||||||
|
Netblock: netblock,
|
||||||
|
Start: midnight(),
|
||||||
|
Cause: bans.CauseAttack,
|
||||||
|
Lifted: midnight().Add(time.Hour),
|
||||||
|
}})
|
||||||
|
|
||||||
|
now := midnight().Add(2 * time.Hour)
|
||||||
|
|
||||||
|
_, banned := ledger.Find(netblock.Addr(), now)
|
||||||
|
if banned {
|
||||||
|
t.Error("the lifted ban refuses")
|
||||||
|
}
|
||||||
|
|
||||||
|
active, permanent := ledger.Count(now)
|
||||||
|
if active != 0 || permanent != 0 {
|
||||||
|
t.Errorf("%d bans are active and %d permanent, want none", active, permanent)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The next clear sign of attack bans for seven days, as a first does.
|
||||||
|
ban := ledger.BanForAttack(netblock, now, bans.Notes{})
|
||||||
|
if ban.Expires.Sub(ban.Start) != 7*day {
|
||||||
|
t.Errorf("the next ban for an attack ends at %s, want seven days on", ban.Expires)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
ledger := bans.New(defaultRules())
|
||||||
|
made := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
|
||||||
|
bans.Notes{})
|
||||||
|
atStart := bans.Ban{
|
||||||
|
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
|
||||||
|
Start: midnight(),
|
||||||
|
}
|
||||||
|
|
||||||
|
// The bans read at the start were made before it.
|
||||||
|
ledger.Load([]bans.Ban{made, atStart})
|
||||||
|
|
||||||
|
if got := ledger.Made(bans.CauseAdmin); got != 0 {
|
||||||
|
t.Fatalf("%d bans made by an admin after the start's, want none", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The admin keeps the ban smallwebwaf made, keeps the one read at the
|
||||||
|
// start, and adds one without a cause: that one alone is made.
|
||||||
|
kept := made
|
||||||
|
kept.Cause = bans.CauseAdmin
|
||||||
|
added := bans.Ban{
|
||||||
|
Netblock: netip.MustParsePrefix("203.0.113.3/32"),
|
||||||
|
Start: midnight(),
|
||||||
|
}
|
||||||
|
ledger.LoadEdit([]bans.Ban{kept, atStart, added})
|
||||||
|
|
||||||
|
if ledger.Made(bans.CauseAdmin) != 1 || ledger.Made(bans.CauseLimit) != 1 {
|
||||||
|
t.Errorf("%d bans made by an admin and %d for a limit, want 1 of each",
|
||||||
|
ledger.Made(bans.CauseAdmin), ledger.Made(bans.CauseLimit))
|
||||||
|
}
|
||||||
|
}
|
||||||
+167
-74
@@ -1,11 +1,13 @@
|
|||||||
// Package bans is the ban ledger: the bans smallwebwaf makes on the
|
// Package bans is the ban ledger: the bans smallwebwaf makes on the
|
||||||
// netblocks of clients that break a rate limit or show a clear sign of
|
// netblocks of clients that break a rate limit or show a clear sign of
|
||||||
// attack, with their notes, as the "Bans" section of SPEC.md describes.
|
// attack, and those an admin makes, with their notes, as the "Bans"
|
||||||
// The bans are kept in memory, and written to bans.json and read from it
|
// section of SPEC.md describes. The bans are kept in memory, and written
|
||||||
// by the state package.
|
// to bans.json and read from it by the state package.
|
||||||
package bans
|
package bans
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"fmt"
|
||||||
|
"math"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"slices"
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -15,13 +17,15 @@ import (
|
|||||||
"github.com/hashicorp/golang-lru/v2/simplelru"
|
"github.com/hashicorp/golang-lru/v2/simplelru"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The causes of the bans smallwebwaf makes. A ban an admin adds to
|
// The causes of bans.
|
||||||
// bans.json may have no cause.
|
|
||||||
const (
|
const (
|
||||||
// CauseLimit is a ban for a broken limit.
|
// CauseLimit is a ban smallwebwaf made for a broken limit.
|
||||||
CauseLimit = "limit"
|
CauseLimit = "limit"
|
||||||
// CauseAttack is a ban for a clear sign of attack.
|
// CauseAttack is a ban smallwebwaf made for a clear sign of attack.
|
||||||
CauseAttack = "attack"
|
CauseAttack = "attack"
|
||||||
|
// CauseAdmin is a ban an admin made, or one smallwebwaf made that an
|
||||||
|
// admin keeps. It is never dropped.
|
||||||
|
CauseAdmin = "admin"
|
||||||
)
|
)
|
||||||
|
|
||||||
// repeatFactor is how many times as long as the netblock's last ban a ban
|
// repeatFactor is how many times as long as the netblock's last ban a ban
|
||||||
@@ -46,9 +50,10 @@ type Rules struct {
|
|||||||
// AttackBanDuration is how long a first ban for a clear sign of attack
|
// AttackBanDuration is how long a first ban for a clear sign of attack
|
||||||
// lasts.
|
// lasts.
|
||||||
AttackBanDuration time.Duration
|
AttackBanDuration time.Duration
|
||||||
// MaxBans is the most bans held, at least one. Past it, the earliest
|
// MaxBans is the most bans held whose cause is not CauseAdmin, at
|
||||||
// ban of the netblock that has gone longest without a request is
|
// least one. Past it, the earliest such ban of the netblock that has
|
||||||
// dropped.
|
// gone longest without a request is dropped. Bans whose cause is
|
||||||
|
// CauseAdmin are held besides, and never dropped.
|
||||||
MaxBans int
|
MaxBans int
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -58,10 +63,16 @@ type Ban struct {
|
|||||||
Start time.Time
|
Start time.Time
|
||||||
// Expires is when the ban ends, zero for a permanent ban.
|
// Expires is when the ban ends, zero for a permanent ban.
|
||||||
Expires time.Time
|
Expires time.Time
|
||||||
// Cause is CauseLimit or CauseAttack, or "" for a ban an admin added
|
// Cause is CauseLimit, CauseAttack or CauseAdmin.
|
||||||
// without one.
|
|
||||||
Cause string
|
Cause string
|
||||||
Notes Notes
|
// Reason is a short text: for a ban smallwebwaf made, the limit broken
|
||||||
|
// or the rule that matched; for an admin's, what the admin wrote.
|
||||||
|
Reason string
|
||||||
|
// Lifted is when an admin lifted the ban, zero while no admin has. A
|
||||||
|
// lifted ban refuses nothing, and does not make the netblock's next
|
||||||
|
// ban longer.
|
||||||
|
Lifted time.Time
|
||||||
|
Notes Notes
|
||||||
}
|
}
|
||||||
|
|
||||||
// Permanent reports whether the ban never runs out.
|
// Permanent reports whether the ban never runs out.
|
||||||
@@ -69,9 +80,10 @@ func (b Ban) Permanent() bool {
|
|||||||
return b.Expires.IsZero()
|
return b.Expires.IsZero()
|
||||||
}
|
}
|
||||||
|
|
||||||
// ActiveAt reports whether the ban refuses requests at now.
|
// ActiveAt reports whether the ban refuses requests at now: it has not
|
||||||
|
// been lifted, and has not run out.
|
||||||
func (b Ban) ActiveAt(now time.Time) bool {
|
func (b Ban) ActiveAt(now time.Time) bool {
|
||||||
return b.Permanent() || now.Before(b.Expires)
|
return b.Lifted.IsZero() && (b.Permanent() || now.Before(b.Expires))
|
||||||
}
|
}
|
||||||
|
|
||||||
// Notes are what an admin needs to decide whether to lift a ban. The
|
// Notes are what an admin needs to decide whether to lift a ban. The
|
||||||
@@ -107,13 +119,10 @@ type Notes struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// EarlierBans counts a netblock's bans before a ban, by cause.
|
// EarlierBans counts a netblock's bans before a ban, by cause.
|
||||||
//
|
|
||||||
//nolint:tagliatelle // the state files use snake_case, as the request log does
|
|
||||||
type EarlierBans struct {
|
type EarlierBans struct {
|
||||||
Limit int `json:"limit"`
|
Limit int `json:"limit"`
|
||||||
Attack int `json:"attack"`
|
Attack int `json:"attack"`
|
||||||
// WithoutCause counts the bans an admin added without a cause.
|
Admin int `json:"admin"`
|
||||||
WithoutCause int `json:"without_cause"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Request is a request in a ban's notes. Each text is cut to 256 bytes.
|
// Request is a request in a ban's notes. Each text is cut to 256 bytes.
|
||||||
@@ -141,9 +150,11 @@ type Ledger struct {
|
|||||||
// netblocks holds each banned netblock's bans, oldest first. Check and
|
// netblocks holds each banned netblock's bans, oldest first. Check and
|
||||||
// Find make each netblock they find the most recently seen.
|
// Find make each netblock they find the most recently seen.
|
||||||
netblocks *simplelru.LRU[netip.Prefix, *[]Ban]
|
netblocks *simplelru.LRU[netip.Prefix, *[]Ban]
|
||||||
// held is how many bans netblocks holds, at most rules.MaxBans.
|
// held is how many bans netblocks holds whose cause is not CauseAdmin,
|
||||||
|
// at most rules.MaxBans.
|
||||||
held int
|
held int
|
||||||
// made is how many bans the ledger has made since the start, by cause.
|
// made is how many bans have been made since the start, by cause: by
|
||||||
|
// the ledger, and by an admin in an edit of bans.json.
|
||||||
made map[string]int
|
made map[string]int
|
||||||
// v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6
|
// v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6
|
||||||
// netblocks that have been banned. Check looks for a ban at each of
|
// netblocks that have been banned. Check looks for a ban at each of
|
||||||
@@ -155,9 +166,10 @@ type Ledger struct {
|
|||||||
|
|
||||||
// New returns a Ledger with no ban yet.
|
// New returns a Ledger with no ban yet.
|
||||||
func New(rules Rules) *Ledger {
|
func New(rules Rules) *Ledger {
|
||||||
// Every netblock held has a ban, so there are never more netblocks
|
// The ledger drops bans itself, and never those whose cause is
|
||||||
// than rules.MaxBans, and the LRU never drops one itself.
|
// CauseAdmin, however many there are, so the LRU has no limit of its
|
||||||
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](rules.MaxBans, nil)
|
// own: it keeps the netblocks in the order they were last seen.
|
||||||
|
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](math.MaxInt, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
panic(err) // NewLRU fails only for a size below one
|
panic(err) // NewLRU fails only for a size below one
|
||||||
}
|
}
|
||||||
@@ -233,21 +245,26 @@ func activeBan(bans []Ban, now time.Time) *Ban {
|
|||||||
// BanForLimit bans netblock at now for a broken limit, with notes, and
|
// BanForLimit bans netblock at now for a broken limit, with notes, and
|
||||||
// returns the ban. A first ban lasts LimitBanDuration. A ban made within
|
// returns the ban. A first ban lasts LimitBanDuration. A ban made within
|
||||||
// LimitBanRepeatWindow after the netblock's ban that ended last, other
|
// LimitBanRepeatWindow after the netblock's ban that ended last, other
|
||||||
// than one for a clear sign of attack, lasts repeatFactor times as long as
|
// than one for a clear sign of attack or a lifted one, lasts repeatFactor
|
||||||
// that one. A ban that would be longer than MaxBanDuration is permanent
|
// times as long as that one. A ban that would be longer than
|
||||||
// instead. If a ban on netblock is still active, as when two of its
|
// MaxBanDuration is permanent instead. If a ban on netblock is still
|
||||||
// requests break a limit at once, that ban is returned and no other is
|
// active, as when two of its requests break a limit at once, that ban is
|
||||||
// made. The ledger fills in the notes' Refused and EarlierBans itself.
|
// returned and no other is made. The ledger fills in the notes' Refused
|
||||||
|
// and EarlierBans itself, and gives the ban the reason "requests per
|
||||||
|
// <Window> over the limit of <Limit>", from the notes.
|
||||||
func (l *Ledger) BanForLimit(netblock netip.Prefix, now time.Time, notes Notes) Ban {
|
func (l *Ledger) BanForLimit(netblock netip.Prefix, now time.Time, notes Notes) Ban {
|
||||||
return l.ban(netblock, now, CauseLimit, notes)
|
reason := fmt.Sprintf("requests per %s over the limit of %d",
|
||||||
|
notes.Window, notes.Limit)
|
||||||
|
|
||||||
|
return l.ban(netblock, now, CauseLimit, reason, notes)
|
||||||
}
|
}
|
||||||
|
|
||||||
// BanForAttack bans netblock at now for a clear sign of attack, with
|
// BanForAttack bans netblock at now for a clear sign of attack, with
|
||||||
// notes, and returns the ban, as BanForLimit does. A first ban lasts
|
// notes, and returns the ban, as BanForLimit does. A first ban lasts
|
||||||
// AttackBanDuration; once the netblock has had one, the next is
|
// AttackBanDuration; once the netblock has had one that was not lifted,
|
||||||
// permanent.
|
// the next is permanent. Its reason is "matched the rule <RuleID>".
|
||||||
func (l *Ledger) BanForAttack(netblock netip.Prefix, now time.Time, notes Notes) Ban {
|
func (l *Ledger) BanForAttack(netblock netip.Prefix, now time.Time, notes Notes) Ban {
|
||||||
return l.ban(netblock, now, CauseAttack, notes)
|
return l.ban(netblock, now, CauseAttack, "matched the rule "+notes.RuleID, notes)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Bans returns the bans held on netblock, oldest first. It is not a
|
// Bans returns the bans held on netblock, oldest first. It is not a
|
||||||
@@ -264,8 +281,10 @@ func (l *Ledger) Bans(netblock netip.Prefix) []Ban {
|
|||||||
return slices.Clone(*bans)
|
return slices.Clone(*bans)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Made returns how many bans for cause the ledger has made since the
|
// Made returns how many bans for cause have been made since the start:
|
||||||
// start; bans read from bans.json are not among them.
|
// for CauseLimit and CauseAttack, by the ledger; for CauseAdmin, by an
|
||||||
|
// admin in an edit of bans.json, as LoadEdit counts them. The bans read
|
||||||
|
// from bans.json at the start are not among them.
|
||||||
func (l *Ledger) Made(cause string) int {
|
func (l *Ledger) Made(cause string) int {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
defer l.mu.Unlock()
|
defer l.mu.Unlock()
|
||||||
@@ -274,7 +293,7 @@ func (l *Ledger) Made(cause string) int {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Count returns how many of the bans held are active at now, and how many
|
// Count returns how many of the bans held are active at now, and how many
|
||||||
// are permanent.
|
// of those are permanent. A lifted ban is neither.
|
||||||
func (l *Ledger) Count(now time.Time) (int, int) {
|
func (l *Ledger) Count(now time.Time) (int, int) {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
defer l.mu.Unlock()
|
defer l.mu.Unlock()
|
||||||
@@ -283,10 +302,12 @@ func (l *Ledger) Count(now time.Time) (int, int) {
|
|||||||
|
|
||||||
for _, bans := range l.netblocks.Values() {
|
for _, bans := range l.netblocks.Values() {
|
||||||
for _, ban := range *bans {
|
for _, ban := range *bans {
|
||||||
if ban.ActiveAt(now) {
|
if !ban.ActiveAt(now) {
|
||||||
active++
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
|
active++
|
||||||
|
|
||||||
if ban.Permanent() {
|
if ban.Permanent() {
|
||||||
permanent++
|
permanent++
|
||||||
}
|
}
|
||||||
@@ -314,36 +335,81 @@ func (l *Ledger) Snapshot() []Ban {
|
|||||||
return held
|
return held
|
||||||
}
|
}
|
||||||
|
|
||||||
// Load puts bans read from bans.json into the ledger, in place of the
|
// Load puts bans read from bans.json at the start into the ledger, in
|
||||||
// bans it holds, in the order they started, so that a netblock whose last
|
// place of the bans it holds, in the order they started, so that a
|
||||||
// ban started latest counts as the most recently seen. Each netblock is
|
// netblock whose last ban started latest counts as the most recently
|
||||||
// masked to its length, so that 203.0.113.9/24 is 203.0.113.0/24, and
|
// seen. A ban without a cause is an admin's, and gets CauseAdmin. Each
|
||||||
// each text in the notes is cut to 256 bytes. Past MaxBans the earliest
|
// netblock is masked to its length, so that 203.0.113.9/24 is
|
||||||
// bans are dropped, as when they are made.
|
// 203.0.113.0/24, and each text in the notes is cut to 256 bytes. Past
|
||||||
|
// MaxBans the earliest bans whose cause is not CauseAdmin are dropped, as
|
||||||
|
// when they are made.
|
||||||
func (l *Ledger) Load(bans []Ban) {
|
func (l *Ledger) Load(bans []Ban) {
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
|
||||||
|
l.load(bans)
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadEdit is Load for an admin's edit of bans.json, taken in while
|
||||||
|
// smallwebwaf runs. Each ban in it whose cause is CauseAdmin, and which
|
||||||
|
// the ledger did not hold, with the same netblock and start, is one the
|
||||||
|
// admin made, and is counted among the bans made.
|
||||||
|
func (l *Ledger) LoadEdit(bans []Ban) {
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
|
||||||
|
l.made[CauseAdmin] += l.load(bans)
|
||||||
|
}
|
||||||
|
|
||||||
|
// load does what Load describes, and returns how many of bans are bans
|
||||||
|
// whose cause is CauseAdmin that the ledger did not hold before.
|
||||||
|
func (l *Ledger) load(bans []Ban) int {
|
||||||
bans = slices.Clone(bans)
|
bans = slices.Clone(bans)
|
||||||
|
added := 0
|
||||||
|
|
||||||
|
for i := range bans {
|
||||||
|
ban := &bans[i]
|
||||||
|
ban.Netblock = ban.Netblock.Masked()
|
||||||
|
ban.Notes.Request = ban.Notes.Request.cut()
|
||||||
|
|
||||||
|
if ban.Cause == "" {
|
||||||
|
ban.Cause = CauseAdmin
|
||||||
|
}
|
||||||
|
|
||||||
|
if ban.Cause == CauseAdmin && !l.holds(ban.Netblock, ban.Start) {
|
||||||
|
added++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
slices.SortStableFunc(bans, func(a, b Ban) int {
|
slices.SortStableFunc(bans, func(a, b Ban) int {
|
||||||
return a.Start.Compare(b.Start)
|
return a.Start.Compare(b.Start)
|
||||||
})
|
})
|
||||||
|
|
||||||
l.mu.Lock()
|
|
||||||
defer l.mu.Unlock()
|
|
||||||
|
|
||||||
l.netblocks.Purge()
|
l.netblocks.Purge()
|
||||||
l.held = 0
|
l.held = 0
|
||||||
l.v4Lengths, l.v6Lengths = nil, nil
|
l.v4Lengths, l.v6Lengths = nil, nil
|
||||||
|
|
||||||
for _, ban := range bans {
|
for _, ban := range bans {
|
||||||
ban.Netblock = ban.Netblock.Masked()
|
|
||||||
ban.Notes.Request = ban.Notes.Request.cut()
|
|
||||||
l.add(ban)
|
l.add(ban)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
return added
|
||||||
}
|
}
|
||||||
|
|
||||||
// ban bans netblock at now for cause, with notes, as BanForLimit and
|
// holds reports whether the ledger holds a ban on netblock that started
|
||||||
// BanForAttack describe, and returns the ban.
|
// at start.
|
||||||
|
func (l *Ledger) holds(netblock netip.Prefix, start time.Time) bool {
|
||||||
|
bans, found := l.netblocks.Peek(netblock)
|
||||||
|
|
||||||
|
return found && slices.ContainsFunc(*bans, func(ban Ban) bool {
|
||||||
|
return ban.Start.Equal(start)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ban bans netblock at now for cause, with reason and notes, as
|
||||||
|
// BanForLimit and BanForAttack describe, and returns the ban.
|
||||||
func (l *Ledger) ban(
|
func (l *Ledger) ban(
|
||||||
netblock netip.Prefix, now time.Time, cause string, notes Notes,
|
netblock netip.Prefix, now time.Time, cause, reason string, notes Notes,
|
||||||
) Ban {
|
) Ban {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
defer l.mu.Unlock()
|
defer l.mu.Unlock()
|
||||||
@@ -363,7 +429,7 @@ func (l *Ledger) ban(
|
|||||||
}
|
}
|
||||||
|
|
||||||
notes.Request = notes.Request.cut()
|
notes.Request = notes.Request.cut()
|
||||||
ban := Ban{Netblock: netblock, Start: now, Cause: cause, Notes: notes}
|
ban := Ban{Netblock: netblock, Start: now, Cause: cause, Reason: reason, Notes: notes}
|
||||||
|
|
||||||
if cause == CauseAttack {
|
if cause == CauseAttack {
|
||||||
ban.Expires = l.attackExpiry(held, now)
|
ban.Expires = l.attackExpiry(held, now)
|
||||||
@@ -391,8 +457,8 @@ func earlierBans(held []Ban) EarlierBans {
|
|||||||
earlier.Limit++
|
earlier.Limit++
|
||||||
case CauseAttack:
|
case CauseAttack:
|
||||||
earlier.Attack++
|
earlier.Attack++
|
||||||
default:
|
case CauseAdmin:
|
||||||
earlier.WithoutCause++
|
earlier.Admin++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -431,9 +497,11 @@ func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// add adds ban to its netblock's bans, after the last, and makes its
|
// add adds ban to its netblock's bans, after the last, and makes its
|
||||||
// netblock the most recently seen. With MaxBans held, it drops one first.
|
// netblock the most recently seen. With MaxBans held, it drops one first,
|
||||||
|
// unless ban's cause is CauseAdmin, which does not count toward MaxBans.
|
||||||
func (l *Ledger) add(ban Ban) {
|
func (l *Ledger) add(ban Ban) {
|
||||||
if l.held == l.rules.MaxBans {
|
counted := ban.Cause != CauseAdmin
|
||||||
|
if counted && l.held == l.rules.MaxBans {
|
||||||
l.dropOne()
|
l.dropOne()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -446,7 +514,10 @@ func (l *Ledger) add(ban Ban) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
*bans = append(*bans, ban)
|
*bans = append(*bans, ban)
|
||||||
l.held++
|
|
||||||
|
if counted {
|
||||||
|
l.held++
|
||||||
|
}
|
||||||
|
|
||||||
lengths := &l.v6Lengths
|
lengths := &l.v6Lengths
|
||||||
if ban.Netblock.Addr().Is4() {
|
if ban.Netblock.Addr().Is4() {
|
||||||
@@ -461,16 +532,17 @@ func (l *Ledger) add(ban Ban) {
|
|||||||
// limitExpiry returns when a ban for a broken limit made at now ends, or
|
// limitExpiry returns when a ban for a broken limit made at now ends, or
|
||||||
// zero when it is permanent. held are the netblock's bans, none of them
|
// zero when it is permanent. held are the netblock's bans, none of them
|
||||||
// active, of which the one that ended last, other than a ban for a clear
|
// active, of which the one that ended last, other than a ban for a clear
|
||||||
// sign of attack, can make the new ban longer. A ban an admin adds to
|
// sign of attack or a lifted one, can make the new ban longer. A ban an
|
||||||
// bans.json can start after another and end before it, so that one is
|
// admin adds to bans.json can start after another and end before it, so
|
||||||
// looked for among them all.
|
// that one is looked for among them all.
|
||||||
func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
|
func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
|
||||||
length := l.rules.LimitBanDuration
|
length := l.rules.LimitBanDuration
|
||||||
|
|
||||||
var last *Ban
|
var last *Ban
|
||||||
|
|
||||||
for i, ban := range held {
|
for i, ban := range held {
|
||||||
if ban.Cause != CauseAttack && (last == nil || ban.Expires.After(last.Expires)) {
|
if ban.Cause != CauseAttack && ban.Lifted.IsZero() &&
|
||||||
|
(last == nil || ban.Expires.After(last.Expires)) {
|
||||||
last = &held[i]
|
last = &held[i]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -495,11 +567,11 @@ func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
|
|||||||
|
|
||||||
// attackExpiry returns when a ban for a clear sign of attack made at now
|
// attackExpiry returns when a ban for a clear sign of attack made at now
|
||||||
// ends. held are the netblock's bans, none of them active: if one of them
|
// ends. held are the netblock's bans, none of them active: if one of them
|
||||||
// is for a clear sign of attack too, the new ban is permanent, and its
|
// is for a clear sign of attack too, and was not lifted, the new ban is
|
||||||
// end zero; otherwise it ends AttackBanDuration later.
|
// permanent, and its end zero; otherwise it ends AttackBanDuration later.
|
||||||
func (l *Ledger) attackExpiry(held []Ban, now time.Time) time.Time {
|
func (l *Ledger) attackExpiry(held []Ban, now time.Time) time.Time {
|
||||||
for _, ban := range held {
|
for _, ban := range held {
|
||||||
if ban.Cause == CauseAttack {
|
if ban.Cause == CauseAttack && ban.Lifted.IsZero() {
|
||||||
return time.Time{}
|
return time.Time{}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -507,17 +579,38 @@ func (l *Ledger) attackExpiry(held []Ban, now time.Time) time.Time {
|
|||||||
return now.Add(l.rules.AttackBanDuration)
|
return now.Add(l.rules.AttackBanDuration)
|
||||||
}
|
}
|
||||||
|
|
||||||
// dropOne drops the earliest ban of the netblock that has gone longest
|
// dropOne drops the earliest ban whose cause is not CauseAdmin of the
|
||||||
// without a request, and the netblock with it if that was its only ban.
|
// netblock that has gone longest without a request, of those that hold
|
||||||
|
// such a ban, and the netblock with it if that was its only ban. It is
|
||||||
|
// called with at least one such ban held.
|
||||||
func (l *Ledger) dropOne() {
|
func (l *Ledger) dropOne() {
|
||||||
netblock, bans, _ := l.netblocks.GetOldest()
|
for {
|
||||||
if len(*bans) == 1 {
|
netblock, bans, _ := l.netblocks.GetOldest()
|
||||||
l.netblocks.Remove(netblock)
|
|
||||||
} else {
|
|
||||||
*bans = slices.Delete(*bans, 0, 1)
|
|
||||||
}
|
|
||||||
|
|
||||||
l.held--
|
i := slices.IndexFunc(*bans, func(ban Ban) bool {
|
||||||
|
return ban.Cause != CauseAdmin
|
||||||
|
})
|
||||||
|
if i < 0 {
|
||||||
|
// Its bans are all an admin's, and never dropped. Get makes
|
||||||
|
// it the most recently seen, so that the next netblock is
|
||||||
|
// looked at; when it was seen matters only for dropping a
|
||||||
|
// ban, and a ban added to it makes it the most recently seen
|
||||||
|
// anyway.
|
||||||
|
l.netblocks.Get(netblock)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(*bans) == 1 {
|
||||||
|
l.netblocks.Remove(netblock)
|
||||||
|
} else {
|
||||||
|
*bans = slices.Delete(*bans, i, i+1)
|
||||||
|
}
|
||||||
|
|
||||||
|
l.held--
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// cut returns r with each text cut to maxTextBytes and copied, so that
|
// cut returns r with each text cut to maxTextBytes and copied, so that
|
||||||
|
|||||||
@@ -173,7 +173,7 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// A 9-hour ban smallwebwaf made, the third in a row, and an admin's
|
// A 9-hour ban smallwebwaf made, the third in a row, and an admin's
|
||||||
// 1-hour ban added to bans.json over it, with no notes.
|
// 1-hour ban added to bans.json over it, with no cause and no notes.
|
||||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
nineHours := bans.Ban{
|
nineHours := bans.Ban{
|
||||||
Netblock: netblock,
|
Netblock: netblock,
|
||||||
@@ -193,13 +193,12 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
|
|||||||
|
|
||||||
// Once both have ended, a limit broken within the repeat window bans
|
// Once both have ended, a limit broken within the repeat window bans
|
||||||
// for three times the 9 hours, and the notes count the two bans
|
// for three times the 9 hours, and the notes count the two bans
|
||||||
// before the 9-hour one and it, for a limit, and the admin's, without
|
// before the 9-hour one and it, for a limit, and the admin's.
|
||||||
// a cause.
|
|
||||||
ban := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{})
|
ban := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{})
|
||||||
if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
|
if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
|
||||||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, WithoutCause: 1}) {
|
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) {
|
||||||
t.Errorf("the next ban lasts %s with earlier bans %+v, "+
|
t.Errorf("the next ban lasts %s with earlier bans %+v, "+
|
||||||
"want 27h, 3 for a limit and 1 without a cause",
|
"want 27h, 3 for a limit and 1 an admin's",
|
||||||
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
|
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -208,10 +207,15 @@ func TestLoadKeepsAtMostMaxBansDroppingTheEarliest(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// bans.json lists the bans by netblock, not in the order they began.
|
// bans.json lists the bans by netblock, not in the order they began.
|
||||||
later := bans.Ban{Netblock: netip.MustParsePrefix("203.0.113.1/32"), Start: midnight()}
|
later := bans.Ban{
|
||||||
|
Netblock: netip.MustParsePrefix("203.0.113.1/32"),
|
||||||
|
Start: midnight(),
|
||||||
|
Cause: bans.CauseLimit,
|
||||||
|
}
|
||||||
earlier := bans.Ban{
|
earlier := bans.Ban{
|
||||||
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
|
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
|
||||||
Start: midnight().Add(-time.Hour),
|
Start: midnight().Add(-time.Hour),
|
||||||
|
Cause: bans.CauseLimit,
|
||||||
}
|
}
|
||||||
|
|
||||||
rules := defaultRules()
|
rules := defaultRules()
|
||||||
@@ -233,9 +237,17 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
|
|||||||
rules := defaultRules()
|
rules := defaultRules()
|
||||||
rules.MaxBans = 3
|
rules.MaxBans = 3
|
||||||
ledger := bans.New(rules)
|
ledger := bans.New(rules)
|
||||||
kept := bans.Ban{Netblock: netip.MustParsePrefix("2001:db8::/64"), Start: midnight()}
|
kept := bans.Ban{
|
||||||
|
Netblock: netip.MustParsePrefix("2001:db8::/64"),
|
||||||
|
Start: midnight(),
|
||||||
|
Cause: bans.CauseLimit,
|
||||||
|
}
|
||||||
ledger.Load([]bans.Ban{
|
ledger.Load([]bans.Ban{
|
||||||
{Netblock: netip.MustParsePrefix("203.0.113.0/24"), Start: midnight()},
|
{
|
||||||
|
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
|
||||||
|
Start: midnight(),
|
||||||
|
Cause: bans.CauseLimit,
|
||||||
|
},
|
||||||
kept,
|
kept,
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
@@ -144,7 +144,7 @@ func New(topN int) *Metrics {
|
|||||||
func (m *Metrics) AddBansAndClients(
|
func (m *Metrics) AddBansAndClients(
|
||||||
ledger *bans.Ledger, limiter *ratelimit.Limiter, now func() time.Time,
|
ledger *bans.Ledger, limiter *ratelimit.Limiter, now func() time.Time,
|
||||||
) {
|
) {
|
||||||
for _, cause := range []string{bans.CauseLimit, bans.CauseAttack} {
|
for _, cause := range []string{bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin} {
|
||||||
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
|
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||||
Name: "smallwebwaf_bans_made_total",
|
Name: "smallwebwaf_bans_made_total",
|
||||||
Help: "Bans made, by cause.",
|
Help: "Bans made, by cause.",
|
||||||
@@ -165,7 +165,7 @@ func (m *Metrics) AddBansAndClients(
|
|||||||
}),
|
}),
|
||||||
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
||||||
Name: "smallwebwaf_permanent_bans",
|
Name: "smallwebwaf_permanent_bans",
|
||||||
Help: "Permanent bans.",
|
Help: "Permanent bans not lifted.",
|
||||||
}, func() float64 {
|
}, func() float64 {
|
||||||
_, permanent := ledger.Count(now())
|
_, permanent := ledger.Count(now())
|
||||||
|
|
||||||
|
|||||||
@@ -279,6 +279,7 @@ func TestBanNotes(t *testing.T) {
|
|||||||
Start: start,
|
Start: start,
|
||||||
Expires: start.Add(time.Hour),
|
Expires: start.Add(time.Hour),
|
||||||
Cause: bans.CauseLimit,
|
Cause: bans.CauseLimit,
|
||||||
|
Reason: "requests per minute over the limit of 1",
|
||||||
Notes: bans.Notes{
|
Notes: bans.Notes{
|
||||||
Country: "DE",
|
Country: "DE",
|
||||||
Limit: 1,
|
Limit: 1,
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
@@ -243,6 +244,29 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
|
|||||||
wantMetric(t, metrics, "smallwebwaf_tracked_clients", 3)
|
wantMetric(t, metrics, "smallwebwaf_tracked_clients", 3)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestMetricsCountTheBansAnAdminMakes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const scraper = "192.0.2.200" // in SWWAF_RATE_LIMIT_EXEMPT_NETS
|
||||||
|
|
||||||
|
s, clk, server := startWithClock(t, "", map[string]string{
|
||||||
|
metricsToken: token,
|
||||||
|
rateLimitExemptNets: scraper,
|
||||||
|
})
|
||||||
|
|
||||||
|
const admins = `smallwebwaf_bans_made_total{cause="admin"}`
|
||||||
|
|
||||||
|
wantMetric(t, s.scrape(scraper), admins, 0)
|
||||||
|
|
||||||
|
// As an admin's edit of bans.json that adds a ban is taken in.
|
||||||
|
server.Ledger.LoadEdit([]bans.Ban{{
|
||||||
|
Netblock: netip.MustParsePrefix(client + "/32"),
|
||||||
|
Start: clk.Now(),
|
||||||
|
}})
|
||||||
|
|
||||||
|
wantMetric(t, s.scrape(scraper), admins, 1)
|
||||||
|
}
|
||||||
|
|
||||||
func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
|
func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -95,6 +95,7 @@ func TestObserveModeMakesNoBanAndKeepsTheBansItHas(t *testing.T) {
|
|||||||
Netblock: netip.MustParsePrefix(otherClient + "/32"),
|
Netblock: netip.MustParsePrefix(otherClient + "/32"),
|
||||||
Start: clk.Now(),
|
Start: clk.Now(),
|
||||||
Expires: clk.Now().Add(time.Hour),
|
Expires: clk.Now().Add(time.Hour),
|
||||||
|
Cause: bans.CauseAdmin,
|
||||||
}
|
}
|
||||||
server.Ledger.Load([]bans.Ban{kept})
|
server.Ledger.Load([]bans.Ban{kept})
|
||||||
|
|
||||||
|
|||||||
@@ -55,6 +55,7 @@ func TestEachRuleAction(t *testing.T) {
|
|||||||
Start: start,
|
Start: start,
|
||||||
Expires: start.Add(7 * 24 * time.Hour),
|
Expires: start.Add(7 * 24 * time.Hour),
|
||||||
Cause: bans.CauseAttack,
|
Cause: bans.CauseAttack,
|
||||||
|
Reason: "matched the rule probe",
|
||||||
Notes: bans.Notes{
|
Notes: bans.Notes{
|
||||||
RuleID: "probe",
|
RuleID: "probe",
|
||||||
Target: "path",
|
Target: "path",
|
||||||
|
|||||||
+36
-14
@@ -48,7 +48,7 @@ var (
|
|||||||
errVersion = errors.New("unknown version")
|
errVersion = errors.New("unknown version")
|
||||||
// errMissing is for an entry without a field it needs.
|
// errMissing is for an entry without a field it needs.
|
||||||
errMissing = errors.New("has no")
|
errMissing = errors.New("has no")
|
||||||
errCause = errors.New("is not limit or attack")
|
errCause = errors.New("is not limit, attack or admin")
|
||||||
)
|
)
|
||||||
|
|
||||||
// Params are what Load needs.
|
// Params are what Load needs.
|
||||||
@@ -96,12 +96,15 @@ type bansFile struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// banEntry is a ban as bans.json holds it: a permanent ban's expires is
|
// banEntry is a ban as bans.json holds it: a permanent ban's expires is
|
||||||
// null, and a ban an admin added may have no cause.
|
// null, a ban an admin added may have no cause, which makes it an
|
||||||
|
// admin's, and lifted is left out until an admin lifts the ban.
|
||||||
type banEntry struct {
|
type banEntry struct {
|
||||||
Netblock netip.Prefix `json:"netblock"`
|
Netblock netip.Prefix `json:"netblock"`
|
||||||
Start time.Time `json:"start"`
|
Start time.Time `json:"start"`
|
||||||
Expires *time.Time `json:"expires"`
|
Expires *time.Time `json:"expires"`
|
||||||
Cause string `json:"cause,omitempty"`
|
Cause string `json:"cause"`
|
||||||
|
Reason string `json:"reason,omitempty"`
|
||||||
|
Lifted *time.Time `json:"lifted,omitempty"`
|
||||||
Notes bans.Notes `json:"notes"`
|
Notes bans.Notes `json:"notes"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -262,7 +265,7 @@ func (f *Files) fileChanged(name string) {
|
|||||||
// runs, by Watch or by a write, is taken in here. An edit that does not
|
// runs, by Watch or by a write, is taken in here. An edit that does not
|
||||||
// parse is neither counted nor logged, and takeIn's error returned.
|
// parse is neither counted nor logged, and takeIn's error returned.
|
||||||
func (f *Files) takeInEdit(name string, data []byte) error {
|
func (f *Files) takeInEdit(name string, data []byte) error {
|
||||||
_, err := f.takeIn(name, data)
|
_, err := f.takeIn(name, data, true)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -284,7 +287,7 @@ func (f *Files) read(name string) (int, error) {
|
|||||||
return 0, err
|
return 0, err
|
||||||
}
|
}
|
||||||
|
|
||||||
return f.takeIn(name, data)
|
return f.takeIn(name, data, false)
|
||||||
}
|
}
|
||||||
|
|
||||||
// readChanged returns what the state file name holds, and whether that
|
// readChanged returns what the state file name holds, and whether that
|
||||||
@@ -308,9 +311,11 @@ func (f *Files) readChanged(name string) ([]byte, bool, error) {
|
|||||||
|
|
||||||
// takeIn parses data, what the state file name holds, puts it into the
|
// takeIn parses data, what the state file name holds, puts it into the
|
||||||
// part that keeps that state, in place of what the part held, and returns
|
// part that keeps that state, in place of what the part held, and returns
|
||||||
// how many entries the file holds. An error names the file and, where the
|
// how many entries the file holds. edit is whether data is an admin's
|
||||||
// JSON decoder tells it, the line and column, or else the entry.
|
// edit taken in while smallwebwaf runs, rather than the file read at the
|
||||||
func (f *Files) takeIn(name string, data []byte) (int, error) {
|
// start. An error names the file and, where the JSON decoder tells it,
|
||||||
|
// the line and column, or else the entry.
|
||||||
|
func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
|
||||||
path := filepath.Join(f.params.Dir, name)
|
path := filepath.Join(f.params.Dir, name)
|
||||||
|
|
||||||
var entries int
|
var entries int
|
||||||
@@ -329,7 +334,12 @@ func (f *Files) takeIn(name string, data []byte) (int, error) {
|
|||||||
held = append(held, entry.ban())
|
held = append(held, entry.ban())
|
||||||
}
|
}
|
||||||
|
|
||||||
f.params.Ledger.Load(held)
|
if edit {
|
||||||
|
f.params.Ledger.LoadEdit(held)
|
||||||
|
} else {
|
||||||
|
f.params.Ledger.Load(held)
|
||||||
|
}
|
||||||
|
|
||||||
entries = len(held)
|
entries = len(held)
|
||||||
case clientsJSON:
|
case clientsJSON:
|
||||||
var file clientsFile
|
var file clientsFile
|
||||||
@@ -447,22 +457,34 @@ func (f *Files) encode(name string) ([]byte, error) {
|
|||||||
// newBanEntry returns ban as bans.json holds it.
|
// newBanEntry returns ban as bans.json holds it.
|
||||||
func newBanEntry(ban bans.Ban) banEntry {
|
func newBanEntry(ban bans.Ban) banEntry {
|
||||||
entry := banEntry{
|
entry := banEntry{
|
||||||
Netblock: ban.Netblock, Start: ban.Start, Cause: ban.Cause, Notes: ban.Notes,
|
Netblock: ban.Netblock, Start: ban.Start, Cause: ban.Cause, Reason: ban.Reason,
|
||||||
|
Notes: ban.Notes,
|
||||||
}
|
}
|
||||||
if !ban.Permanent() {
|
if !ban.Permanent() {
|
||||||
entry.Expires = &ban.Expires
|
entry.Expires = &ban.Expires
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if !ban.Lifted.IsZero() {
|
||||||
|
entry.Lifted = &ban.Lifted
|
||||||
|
}
|
||||||
|
|
||||||
return entry
|
return entry
|
||||||
}
|
}
|
||||||
|
|
||||||
// ban returns the ban an entry of bans.json holds.
|
// ban returns the ban an entry of bans.json holds.
|
||||||
func (e banEntry) ban() bans.Ban {
|
func (e banEntry) ban() bans.Ban {
|
||||||
ban := bans.Ban{Netblock: e.Netblock, Start: e.Start, Cause: e.Cause, Notes: e.Notes}
|
ban := bans.Ban{
|
||||||
|
Netblock: e.Netblock, Start: e.Start, Cause: e.Cause, Reason: e.Reason,
|
||||||
|
Notes: e.Notes,
|
||||||
|
}
|
||||||
if e.Expires != nil {
|
if e.Expires != nil {
|
||||||
ban.Expires = *e.Expires
|
ban.Expires = *e.Expires
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if e.Lifted != nil {
|
||||||
|
ban.Lifted = *e.Lifted
|
||||||
|
}
|
||||||
|
|
||||||
return ban
|
return ban
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -470,8 +492,8 @@ func (e banEntry) ban() bans.Ban {
|
|||||||
// client, a start, from which the length of the netblock's next ban is
|
// client, a start, from which the length of the netblock's next ban is
|
||||||
// worked out, or an expires, which would make it permanent. A permanent
|
// worked out, or an expires, which would make it permanent. A permanent
|
||||||
// ban's expires is null, which Bans cannot tell from a missing one, so
|
// ban's expires is null, which Bans cannot tell from a missing one, so
|
||||||
// each expires is read again as written. A cause other than limit or
|
// each expires is read again as written. A cause other than limit,
|
||||||
// attack, most likely misspelt, is refused too.
|
// attack or admin, most likely misspelt, is refused too.
|
||||||
func (f *bansFile) check(data []byte) error {
|
func (f *bansFile) check(data []byte) error {
|
||||||
var written struct {
|
var written struct {
|
||||||
Bans []struct {
|
Bans []struct {
|
||||||
@@ -493,7 +515,7 @@ func (f *bansFile) check(data []byte) error {
|
|||||||
case written.Bans[i].Expires == nil:
|
case written.Bans[i].Expires == nil:
|
||||||
return missing(i, "expires")
|
return missing(i, "expires")
|
||||||
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
|
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
|
||||||
entry.Cause != bans.CauseAttack:
|
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin:
|
||||||
return fmt.Errorf("entry %d's cause %q %w", i+1, entry.Cause, errCause)
|
return fmt.Errorf("entry %d's cause %q %w", i+1, entry.Cause, errCause)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -48,6 +48,8 @@ const permanentBansJSON = `{
|
|||||||
"netblock": "2001:db8::/64",
|
"netblock": "2001:db8::/64",
|
||||||
"start": "2026-10-06T00:00:00Z",
|
"start": "2026-10-06T00:00:00Z",
|
||||||
"expires": null,
|
"expires": null,
|
||||||
|
"cause": "admin",
|
||||||
|
"reason": "scrapes every commit",
|
||||||
"notes": {
|
"notes": {
|
||||||
"country": "DE",
|
"country": "DE",
|
||||||
"limit": 1000,
|
"limit": 1000,
|
||||||
@@ -66,7 +68,7 @@ const permanentBansJSON = `{
|
|||||||
"earlier_bans": {
|
"earlier_bans": {
|
||||||
"limit": 3,
|
"limit": 3,
|
||||||
"attack": 1,
|
"attack": 1,
|
||||||
"without_cause": 1
|
"admin": 1
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -74,6 +76,15 @@ const permanentBansJSON = `{
|
|||||||
}
|
}
|
||||||
`
|
`
|
||||||
|
|
||||||
|
// liftedClient is the client whose ban liftedBansJSON holds.
|
||||||
|
const liftedClient = "203.0.113.9"
|
||||||
|
|
||||||
|
// liftedBansJSON is bans.json holding an hour's ban for a broken limit on
|
||||||
|
// liftedClient, from midnight, that an admin lifted ten minutes in.
|
||||||
|
const liftedBansJSON = `{"version": 1, "bans": [{"netblock": "203.0.113.9/32", ` +
|
||||||
|
`"start": "2026-10-06T00:00:00Z", "expires": "2026-10-06T01:00:00Z", ` +
|
||||||
|
`"cause": "limit", "lifted": "2026-10-06T00:10:00Z"}]}`
|
||||||
|
|
||||||
func TestFilesWrittenAndReadBack(t *testing.T) {
|
func TestFilesWrittenAndReadBack(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -267,15 +278,17 @@ func TestEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBanWithACauseSmallwebwafDoesNotGiveStopsTheStart(t *testing.T) {
|
func TestBanWithAnotherCauseStopsTheStart(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
wantRefused(t, bansJSON, `{"version": 1, "bans": [`+
|
wantRefused(t, bansJSON, `{"version": 1, "bans": [`+
|
||||||
`{"netblock": "203.0.113.9/32", "start": "2026-10-06T00:00:00Z", `+
|
`{"netblock": "203.0.113.9/32", "start": "2026-10-06T00:00:00Z", `+
|
||||||
`"expires": null, "cause": "attack"}, `+
|
`"expires": null, "cause": "attack"}, `+
|
||||||
`{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+
|
`{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+
|
||||||
|
`"expires": null, "cause": "admin"}, `+
|
||||||
|
`{"netblock": "203.0.113.11/32", "start": "2026-10-06T00:00:00Z", `+
|
||||||
`"expires": null, "cause": "atack"}]}`,
|
`"expires": null, "cause": "atack"}]}`,
|
||||||
`: entry 2's cause "atack" is not limit or attack`)
|
`: entry 3's cause "atack" is not limit, attack or admin`)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestUnknownVersionStopsTheStart(t *testing.T) {
|
func TestUnknownVersionStopsTheStart(t *testing.T) {
|
||||||
@@ -611,7 +624,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
|
|||||||
`"start": "2026-10-06T00:00:00Z", "expires": null}]}`)
|
`"start": "2026-10-06T00:00:00Z", "expires": null}]}`)
|
||||||
wantTakenIn(t, lines, dir, bansJSON)
|
wantTakenIn(t, lines, dir, bansJSON)
|
||||||
wantEqual(t, bansJSON, params.Ledger.Snapshot(),
|
wantEqual(t, bansJSON, params.Ledger.Snapshot(),
|
||||||
[]bans.Ban{{Netblock: client, Start: midnight()}})
|
[]bans.Ban{{Netblock: client, Start: midnight(), Cause: bans.CauseAdmin}})
|
||||||
|
|
||||||
edit(t, dir, clientsJSON, `{"version": 1, "clients": [`+
|
edit(t, dir, clientsJSON, `{"version": 1, "clients": [`+
|
||||||
`{"client": "198.51.100.7/32", "history": {"requests": 7}}]}`)
|
`{"client": "198.51.100.7/32", "history": {"requests": 7}}]}`)
|
||||||
@@ -710,6 +723,100 @@ func TestBanAddedAndLiftedThroughBansJSON(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBanWithoutACauseTakenInAsAnAdmins(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const reason = "probes for logins"
|
||||||
|
|
||||||
|
// adminsBansJSON is bans.json as an admin writes it, with a ban on
|
||||||
|
// netblock without a cause, and adminsBans the bans it holds.
|
||||||
|
adminsBansJSON := func(netblock string) string {
|
||||||
|
return `{"version": 1, "bans": [{"netblock": "` + netblock + `", ` +
|
||||||
|
`"start": "2026-10-06T00:00:00Z", "expires": null, "reason": "` +
|
||||||
|
reason + `"}]}`
|
||||||
|
}
|
||||||
|
adminsBans := func(netblock string) []bans.Ban {
|
||||||
|
return []bans.Ban{{
|
||||||
|
Netblock: netip.MustParsePrefix(netblock),
|
||||||
|
Start: midnight(),
|
||||||
|
Cause: bans.CauseAdmin,
|
||||||
|
Reason: reason,
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read at the start, the ban is taken in as an admin's, though not
|
||||||
|
// counted among the bans made since the start, and written back with
|
||||||
|
// that cause and the admin's reason.
|
||||||
|
dir := t.TempDir()
|
||||||
|
edit(t, dir, bansJSON, adminsBansJSON("203.0.113.0/24"))
|
||||||
|
|
||||||
|
params := newParams(dir)
|
||||||
|
lines := logInto(¶ms)
|
||||||
|
files := load(t, params)
|
||||||
|
wantEqual(t, bansJSON, params.Ledger.Snapshot(), adminsBans("203.0.113.0/24"))
|
||||||
|
wantMadeByAnAdmin(t, params.Ledger, 0)
|
||||||
|
|
||||||
|
err := files.WriteAll()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("write: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var written struct {
|
||||||
|
Bans []struct {
|
||||||
|
Cause string `json:"cause"`
|
||||||
|
Reason string `json:"reason"`
|
||||||
|
} `json:"bans"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err = json.Unmarshal([]byte(readFile(t, filepath.Join(dir, bansJSON))), &written)
|
||||||
|
if err != nil || len(written.Bans) != 1 || written.Bans[0].Cause != bans.CauseAdmin ||
|
||||||
|
written.Bans[0].Reason != reason {
|
||||||
|
t.Errorf("bans.json holds %+v (%v), want the ban with the cause admin "+
|
||||||
|
"and the reason %q", written, err, reason)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Taken in while smallwebwaf runs, a ban on another netblock is an
|
||||||
|
// admin's too, and one made since the start.
|
||||||
|
watch(t, files, lines)
|
||||||
|
edit(t, dir, bansJSON, adminsBansJSON("198.51.100.0/24"))
|
||||||
|
wantTakenIn(t, lines, dir, bansJSON)
|
||||||
|
wantEqual(t, bansJSON, params.Ledger.Snapshot(), adminsBans("198.51.100.0/24"))
|
||||||
|
wantMadeByAnAdmin(t, params.Ledger, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLiftedBanReadAtTheStart(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dir := t.TempDir()
|
||||||
|
edit(t, dir, bansJSON, liftedBansJSON)
|
||||||
|
|
||||||
|
params := newParams(dir)
|
||||||
|
wantLiftedBanKept(t, load(t, params), dir, params.Ledger)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBanLiftedByAnEditWhileRunning(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dir := t.TempDir()
|
||||||
|
params := newParams(dir)
|
||||||
|
lines := logInto(¶ms)
|
||||||
|
files := load(t, params)
|
||||||
|
watch(t, files, lines)
|
||||||
|
|
||||||
|
// The ban that liftedBansJSON lifts, before it is lifted.
|
||||||
|
netblock := netip.MustParsePrefix(liftedClient + "/32")
|
||||||
|
params.Ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||||
|
|
||||||
|
_, banned := params.Ledger.Find(netblock.Addr(), afterLifting())
|
||||||
|
if !banned {
|
||||||
|
t.Fatal("the ban does not refuse before it is lifted")
|
||||||
|
}
|
||||||
|
|
||||||
|
edit(t, dir, bansJSON, liftedBansJSON)
|
||||||
|
wantTakenIn(t, lines, dir, bansJSON)
|
||||||
|
wantLiftedBanKept(t, files, dir, params.Ledger)
|
||||||
|
}
|
||||||
|
|
||||||
func TestBrokenEditSetAsideAtTheNextWrite(t *testing.T) {
|
func TestBrokenEditSetAsideAtTheNextWrite(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -909,9 +1016,9 @@ func newParams(dir string) state.Params {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// fill puts a permanent ban without a cause, as an admin adds one, a ban
|
// fill puts a permanent ban an admin made, a ban for a broken limit and
|
||||||
// for a broken limit and one for a clear sign of attack, clients with
|
// one for a clear sign of attack, clients with counts and histories, and
|
||||||
// counts and histories, and GeoJS answers into the parts of params.
|
// GeoJS answers into the parts of params.
|
||||||
func fill(params state.Params) {
|
func fill(params state.Params) {
|
||||||
now := midnight()
|
now := midnight()
|
||||||
client := netip.MustParsePrefix("203.0.113.9/32")
|
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
@@ -943,6 +1050,8 @@ func permanentBan() bans.Ban {
|
|||||||
return bans.Ban{
|
return bans.Ban{
|
||||||
Netblock: netip.MustParsePrefix("2001:db8::/64"),
|
Netblock: netip.MustParsePrefix("2001:db8::/64"),
|
||||||
Start: midnight(),
|
Start: midnight(),
|
||||||
|
Cause: bans.CauseAdmin,
|
||||||
|
Reason: "scrapes every commit",
|
||||||
Notes: bans.Notes{
|
Notes: bans.Notes{
|
||||||
Country: "DE",
|
Country: "DE",
|
||||||
Limit: 1000,
|
Limit: 1000,
|
||||||
@@ -958,11 +1067,64 @@ func permanentBan() bans.Ban {
|
|||||||
},
|
},
|
||||||
Requests: 1500,
|
Requests: 1500,
|
||||||
Refused: 3,
|
Refused: 3,
|
||||||
EarlierBans: bans.EarlierBans{Limit: 3, Attack: 1, WithoutCause: 1},
|
EarlierBans: bans.EarlierBans{Limit: 3, Attack: 1, Admin: 1},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// afterLifting is a time after the ban liftedBansJSON holds was lifted,
|
||||||
|
// while it would still last.
|
||||||
|
func afterLifting() time.Time {
|
||||||
|
return midnight().Add(30 * time.Minute)
|
||||||
|
}
|
||||||
|
|
||||||
|
// wantLiftedBanKept checks that ledger holds the ban liftedBansJSON holds,
|
||||||
|
// which refuses nothing and does not make the next ban for a broken limit
|
||||||
|
// longer, and that files write it to bans.json, in dir, still lifted.
|
||||||
|
func wantLiftedBanKept(
|
||||||
|
t *testing.T, files *state.Files, dir string, ledger *bans.Ledger,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
err := files.WriteAll()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("write: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
const lifted = `"lifted": "2026-10-06T00:10:00Z"`
|
||||||
|
if got := readFile(t, filepath.Join(dir, bansJSON)); !strings.Contains(got, lifted) {
|
||||||
|
t.Errorf("bans.json holds\n%s\nwant the ban with %s", got, lifted)
|
||||||
|
}
|
||||||
|
|
||||||
|
netblock := netip.MustParsePrefix(liftedClient + "/32")
|
||||||
|
|
||||||
|
_, banned := ledger.Check(netblock.Addr(), afterLifting())
|
||||||
|
if banned {
|
||||||
|
t.Error("the lifted ban refuses")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Were the lifted ban counted, the next would last three hours.
|
||||||
|
ban := ledger.BanForLimit(netblock, afterLifting(), bans.Notes{})
|
||||||
|
if ban.Expires.Sub(ban.Start) != time.Hour {
|
||||||
|
t.Errorf("the next ban lasts %s, want 1h", ban.Expires.Sub(ban.Start))
|
||||||
|
}
|
||||||
|
|
||||||
|
held := ledger.Bans(netblock)
|
||||||
|
if len(held) != 2 || !held[0].Lifted.Equal(midnight().Add(10*time.Minute)) {
|
||||||
|
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// wantMadeByAnAdmin checks how many bans ledger counts as made by an
|
||||||
|
// admin since the start.
|
||||||
|
func wantMadeByAnAdmin(t *testing.T, ledger *bans.Ledger, want int) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if got := ledger.Made(bans.CauseAdmin); got != want {
|
||||||
|
t.Errorf("%d bans made by an admin, want %d", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// load reads the state files into the parts of params.
|
// load reads the state files into the parts of params.
|
||||||
func load(t *testing.T, params state.Params) *state.Files {
|
func load(t *testing.T, params state.Params) *state.Files {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|||||||
Reference in New Issue
Block a user