Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
828f1c7bc5 | ||
|
|
8e3879a0c9 |
No files matched your search
@@ -817,8 +817,8 @@ The supported apps are:
|
||||
- `vaultwarden`;
|
||||
- `wordpress`.
|
||||
|
||||
Each app's fixes are in one file, such as `internal/coreruleset/gitea.conf`, and
|
||||
one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
Each app's fixes are in one file named for the app, such as
|
||||
`internal/coreruleset/apps/gitea.conf`. gitea's fixes are:
|
||||
|
||||
1. Ten parameters are not checked against the Core Rule Set's lists of system
|
||||
files (930120), shell paths (932160) and command names (932260), on any path:
|
||||
@@ -851,8 +851,8 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
3. On gitea's paths, some fields skip the rules that refused ordinary gitea
|
||||
requests there. They are the fields where gitea sends a search, a file's name
|
||||
or text, a commit message, a branch name or a repository's name.
|
||||
`internal/coreruleset/gitea.conf` lists each rule and the request it refused.
|
||||
The fields are:
|
||||
`internal/coreruleset/apps/gitea.conf` lists each rule and the request it
|
||||
refused. The fields are:
|
||||
- `q` on gitea's searches of code, of issues and pull requests, and of
|
||||
commits. A code search holds code, such as `<script>`,
|
||||
`SELECT * FROM users WHERE` or `../`. The searches of issues and pull
|
||||
@@ -1037,7 +1037,7 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
- The `Content-Range` header of an artifact upload from
|
||||
`actions/upload-artifact@v3` (920450).
|
||||
- The passwords in a body, for each rule that random passwords set off. The
|
||||
numbers are in `internal/coreruleset/gitea.conf`.
|
||||
numbers are in `internal/coreruleset/apps/gitea.conf`.
|
||||
- `password` on these paths:
|
||||
- `/user/login`, `/user/sign_up`, `/user/activate`;
|
||||
- `/user/link_account_signin` and `/user/link_account_signup`;
|
||||
@@ -1069,8 +1069,8 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
such as `owner/python3-app` would have that push refused.
|
||||
|
||||
WordPress's fixes: on WordPress's paths, some fields skip the rules that refused
|
||||
ordinary WordPress requests there. `internal/coreruleset/wordpress.conf` lists
|
||||
each rule and the request it refused. The REST API has four addresses:
|
||||
ordinary WordPress requests there. `internal/coreruleset/apps/wordpress.conf`
|
||||
lists each rule and the request it refused. The REST API has four addresses:
|
||||
|
||||
- `/wp-json/...`
|
||||
- `/index.php/wp-json/...`
|
||||
@@ -1162,9 +1162,9 @@ Elsewhere WordPress's fixes do not apply. The paths are those of WordPress at
|
||||
the site root.
|
||||
|
||||
Drupal's fixes: on Drupal's paths, some fields skip the rules that refused
|
||||
ordinary Drupal requests there. `internal/coreruleset/drupal.conf` lists each
|
||||
rule and the request it refused. These query fields skip the lists of system
|
||||
files and command names (930120, 932120, 932250, 932260):
|
||||
ordinary Drupal requests there. `internal/coreruleset/apps/drupal.conf` lists
|
||||
each rule and the request it refused. These query fields skip the lists of
|
||||
system files and command names (930120, 932120, 932250, 932260):
|
||||
|
||||
- On every page under `/admin`: `title`, `name`, `user`, `filename`, `filemime`,
|
||||
`info`, `subject`, `author_name`, `search`, `q` and `destination`.
|
||||
@@ -1254,10 +1254,10 @@ first path segment of two or three lowercase letters, optionally followed by a
|
||||
hyphen and more letters, so `/api` counts too.
|
||||
|
||||
MediaWiki's fixes: on MediaWiki's paths, some fields skip the rules that refused
|
||||
ordinary MediaWiki requests there. `internal/coreruleset/mediawiki.conf` lists
|
||||
each rule and the request it refused. These fields skip the lists of system
|
||||
files and command names (930120, 932120, 932250, 932260), in the query or the
|
||||
body:
|
||||
ordinary MediaWiki requests there. `internal/coreruleset/apps/mediawiki.conf`
|
||||
lists each rule and the request it refused. These fields skip the lists of
|
||||
system files and command names (930120, 932120, 932250, 932260), in the query or
|
||||
the body:
|
||||
|
||||
- On MediaWiki's pages, which are `/index.php` and the paths under `/index.php/`
|
||||
and `/wiki/`:
|
||||
@@ -1363,9 +1363,9 @@ Elsewhere these fields keep every rule. The paths are those of MediaWiki at the
|
||||
site root or under `/w/`.
|
||||
|
||||
Mastodon's fixes: on Mastodon's paths, some fields skip the rules that refused
|
||||
ordinary Mastodon requests there. `internal/coreruleset/mastodon.conf` lists
|
||||
each rule and the request it refused. These fields skip the lists of system
|
||||
files and command names (930120, 932120, 932250, 932260), in the query:
|
||||
ordinary Mastodon requests there. `internal/coreruleset/apps/mastodon.conf`
|
||||
lists each rule and the request it refused. These fields skip the lists of
|
||||
system files and command names (930120, 932120, 932250, 932260), in the query:
|
||||
|
||||
- On the API's search, accounts and admin accounts, under `/api/v1/` and
|
||||
`/api/v2/`, the searches and names `q`, `acct`, `username`, `display_name`,
|
||||
@@ -1526,7 +1526,7 @@ every rule.
|
||||
|
||||
Vaultwarden's fixes: on Vaultwarden's paths, some fields skip the rules that
|
||||
refused ordinary Vaultwarden requests there.
|
||||
`internal/coreruleset/vaultwarden.conf` lists each rule and the request it
|
||||
`internal/coreruleset/apps/vaultwarden.conf` lists each rule and the request it
|
||||
refused. The apps encrypt vault items, folders, collections and sends before
|
||||
sending them, so those fields skip no rules. The fields below hold plain text.
|
||||
|
||||
@@ -1577,8 +1577,8 @@ has a parameter of the same name.
|
||||
|
||||
`token` in the body of `/admin`, the admin token typed on the admin page, skips
|
||||
each rule that random passwords set off. The numbers are in
|
||||
`internal/coreruleset/vaultwarden.conf`. It applies only when the query has no
|
||||
`token` parameter. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
|
||||
`internal/coreruleset/apps/vaultwarden.conf`. It applies only when the query has
|
||||
no `token` parameter. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
|
||||
`<img src=x onerror=x()>` sent as the token reach Vaultwarden;
|
||||
`<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
|
||||
|
||||
@@ -1600,7 +1600,7 @@ Elsewhere these fields keep every rule.
|
||||
|
||||
Nextcloud's fixes: on Nextcloud's paths, some fields skip the rules that refused
|
||||
ordinary Nextcloud requests there, and its WebDAV methods are allowed.
|
||||
`internal/coreruleset/nextcloud.conf` lists each rule and the request it
|
||||
`internal/coreruleset/apps/nextcloud.conf` lists each rule and the request it
|
||||
refused.
|
||||
|
||||
The sync client and the Files app use WebDAV on these paths:
|
||||
@@ -1610,7 +1610,8 @@ The sync client and the Files app use WebDAV on these paths:
|
||||
- `/remote.php/webdav/`;
|
||||
- `/public.php/webdav/`.
|
||||
|
||||
These methods are allowed there, besides those of the first change:
|
||||
These methods are allowed there, besides `PUT`, `PATCH` and `DELETE`, which
|
||||
every app gets:
|
||||
|
||||
- `PROPFIND` and `PROPPATCH`;
|
||||
- `MKCOL`;
|
||||
@@ -1754,7 +1755,7 @@ gets gitea's fixes too. Forgejo's own fixes hold only what Forgejo sends and
|
||||
gitea does not.
|
||||
|
||||
On Forgejo's paths, some fields skip the rules that refused ordinary Forgejo
|
||||
requests there. `internal/coreruleset/forgejo.conf` lists each rule and the
|
||||
requests there. `internal/coreruleset/apps/forgejo.conf` lists each rule and the
|
||||
request it refused.
|
||||
|
||||
`q` skips rules on some of Forgejo's searches. The searches are:
|
||||
@@ -1800,10 +1801,10 @@ The Core Rule Set reads the request without the `flash` cookie, in which Forgejo
|
||||
puts what gitea puts in `gitea_flash` (gitea's second fix).
|
||||
|
||||
Ghost's fixes: on Ghost's paths, some fields skip the rules that refused
|
||||
ordinary Ghost requests there. `internal/coreruleset/ghost.conf` lists each rule
|
||||
and the request it refused. Ghost's admin and integrations call the Admin API
|
||||
under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API under
|
||||
`/ghost/api/content/`, and members call `/members/api/`. The admin calls
|
||||
ordinary Ghost requests there. `internal/coreruleset/apps/ghost.conf` lists each
|
||||
rule and the request it refused. Ghost's admin and integrations call the Admin
|
||||
API under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API
|
||||
under `/ghost/api/content/`, and members call `/members/api/`. The admin calls
|
||||
ActivityPub, a service of its own, under `/.ghost/activitypub/`.
|
||||
|
||||
Eight rules refused words that name commands and files, such as `python3`,
|
||||
@@ -1941,9 +1942,9 @@ That name holds `.profile`, a file on the list of system files, so the name
|
||||
skips 930120 when a profile is saved.
|
||||
|
||||
The passwords skip each rule that random passwords set off. The numbers are in
|
||||
`internal/coreruleset/ghost.conf`. They are `password` when signing in, and the
|
||||
passwords sent when creating the first staff user, accepting an invitation, and
|
||||
resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`,
|
||||
`internal/coreruleset/apps/ghost.conf`. They are `password` when signing in, and
|
||||
the passwords sent when creating the first staff user, accepting an invitation,
|
||||
and resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`,
|
||||
`; cat /etc/passwd` and `<img src=x onerror=x()>` sent as a password reach
|
||||
Ghost; `<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
|
||||
|
||||
|
||||
@@ -54,11 +54,7 @@ func TestBlacklistLeavesANetblockWithAnActiveBanAsItIs(t *testing.T) {
|
||||
now := midnight()
|
||||
scores := map[netip.Prefix]int64{netblock: 100}
|
||||
|
||||
limitBan, _ := ledger.Ban(
|
||||
netblock,
|
||||
now,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
limitBan, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||
|
||||
if made := ledger.BanForAbuseIPDB(now, now.Add(30*day), scores); made != 0 {
|
||||
t.Errorf("made %d bans over a ban for a limit, want none", made)
|
||||
@@ -123,7 +119,7 @@ func TestEndedBlacklistBansAreForgottenAtTheNextFetch(t *testing.T) {
|
||||
expires := now.Add(30 * day)
|
||||
|
||||
ledger.BanForAbuseIPDB(now, expires, map[netip.Prefix]int64{listed: 100})
|
||||
ledger.Ban(limited, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(limited, now, bans.Notes{})
|
||||
|
||||
// listed is off the blacklist when it is fetched after the ban's end.
|
||||
if made := ledger.BanForAbuseIPDB(expires, expires.Add(30*day), nil); made != 0 {
|
||||
@@ -148,7 +144,7 @@ func TestBlacklistBansDoNotCountTowardMaxBans(t *testing.T) {
|
||||
scores[netip.PrefixFrom(netip.AddrFrom4([4]byte{198, 51, 100, byte(i)}), 32)] = 100
|
||||
}
|
||||
|
||||
ledger.Ban(limited, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(limited, now, bans.Notes{})
|
||||
|
||||
if made := ledger.BanForAbuseIPDB(now, now.Add(30*day), scores); made != 3 {
|
||||
t.Errorf("made %d bans for 3 netblocks, want 3", made)
|
||||
@@ -158,7 +154,7 @@ func TestBlacklistBansDoNotCountTowardMaxBans(t *testing.T) {
|
||||
|
||||
// The ban for an attack takes the place of the one for a limit, and no
|
||||
// blacklist ban goes.
|
||||
ledger.Ban(attacker, now, bans.Grounds{Cause: bans.CauseAttack})
|
||||
ledger.BanForAttack(attacker, now, bans.Notes{})
|
||||
|
||||
want := map[netip.Prefix]int{limited: 0, attacker: 1}
|
||||
for netblock := range scores {
|
||||
|
||||
+23
-64
@@ -61,7 +61,7 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
|
||||
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 2})
|
||||
|
||||
// A new ban drops the ban sww made, and only that one.
|
||||
ledger.Ban(second, midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(second, midnight(), bans.Notes{})
|
||||
wantBans(
|
||||
t,
|
||||
ledger,
|
||||
@@ -77,7 +77,7 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
|
||||
}
|
||||
|
||||
// And the next drops that one.
|
||||
ledger.Ban(third, midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(third, midnight(), bans.Notes{})
|
||||
wantBans(
|
||||
t,
|
||||
ledger,
|
||||
@@ -98,28 +98,20 @@ func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
|
||||
// twice, each time after its last ban has ended.
|
||||
ledger.BanForAdmin(netblock, midnight(), midnight().Add(time.Minute),
|
||||
"probes for logins")
|
||||
first, _ := ledger.Ban(
|
||||
first, _ := ledger.BanForLimit(
|
||||
netblock,
|
||||
midnight().Add(time.Minute),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
second, _ := ledger.Ban(
|
||||
netblock,
|
||||
first.Expires,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
|
||||
|
||||
// A ban on another netblock fills the ledger, and the netblock's first
|
||||
// ban for a limit, its earliest that is not an admin's, goes.
|
||||
ledger.Ban(other, second.Start, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(other, second.Start, bans.Notes{})
|
||||
wantBans(t, ledger, map[netip.Prefix]int{netblock: 2, other: 1})
|
||||
|
||||
// The netblock's next ban still counts it.
|
||||
third, _ := ledger.Ban(
|
||||
netblock,
|
||||
second.Expires,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
third, _ := ledger.BanForLimit(netblock, second.Expires, bans.Notes{})
|
||||
if third.Notes.EarlierBans != (bans.EarlierBans{Limit: 2, Admin: 1}) {
|
||||
t.Errorf("the third ban for a limit has earlier bans %+v, "+
|
||||
"want 2 for a limit and 1 an admin's", third.Notes.EarlierBans)
|
||||
@@ -135,27 +127,15 @@ func TestEarlierBansCountABanAnAdminKeepsOnce(t *testing.T) {
|
||||
// The netblock breaks a limit twice, the second time after its first
|
||||
// ban has ended, and the second ban's notes count the first. An admin
|
||||
// then keeps the first ban.
|
||||
first, _ := ledger.Ban(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
second, _ := ledger.Ban(
|
||||
netblock,
|
||||
first.Expires,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
|
||||
|
||||
kept := first
|
||||
kept.Cause = bans.CauseAdmin
|
||||
ledger.LoadEdit([]bans.Ban{kept, second})
|
||||
|
||||
// The netblock's next ban counts the kept ban once, as an admin's.
|
||||
third, _ := ledger.Ban(
|
||||
netblock,
|
||||
second.Expires,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
third, _ := ledger.BanForLimit(netblock, second.Expires, bans.Notes{})
|
||||
if third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Admin: 1}) {
|
||||
t.Errorf("the third ban for a limit has earlier bans %+v, "+
|
||||
"want 1 for a limit and 1 an admin's", third.Notes.EarlierBans)
|
||||
@@ -167,37 +147,20 @@ func TestReasonOfTheBansSwwMakes(t *testing.T) {
|
||||
|
||||
ledger := bans.New(defaultParams())
|
||||
|
||||
limit, _ := ledger.Ban(
|
||||
limit, _ := ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.1/32"),
|
||||
midnight(),
|
||||
bans.Grounds{
|
||||
Cause: bans.CauseLimit,
|
||||
Notes: bans.Notes{
|
||||
Kind: "requests",
|
||||
Limit: 1000,
|
||||
Window: "minute",
|
||||
},
|
||||
},
|
||||
bans.Notes{Kind: "requests", Limit: 1000, Window: "minute"},
|
||||
)
|
||||
byteLimit, _ := ledger.Ban(
|
||||
byteLimit, _ := ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.3/32"),
|
||||
midnight(),
|
||||
bans.Grounds{
|
||||
Cause: bans.CauseLimit,
|
||||
Notes: bans.Notes{
|
||||
Kind: "bytes",
|
||||
Limit: 10 << 30,
|
||||
Window: "hour",
|
||||
},
|
||||
},
|
||||
bans.Notes{Kind: "bytes", Limit: 10 << 30, Window: "hour"},
|
||||
)
|
||||
attack, _ := ledger.Ban(
|
||||
attack, _ := ledger.BanForAttack(
|
||||
netip.MustParsePrefix("203.0.113.2/32"),
|
||||
midnight(),
|
||||
bans.Grounds{
|
||||
Cause: bans.CauseAttack,
|
||||
Notes: bans.Notes{RuleID: "git-dir", Target: "path"},
|
||||
},
|
||||
bans.Notes{RuleID: "git-dir", Target: "path"},
|
||||
)
|
||||
|
||||
for _, tc := range []struct{ got, want string }{
|
||||
@@ -237,7 +200,7 @@ func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
|
||||
t.Error("the lifted ban refuses")
|
||||
}
|
||||
|
||||
ban, _ := ledger.Ban(netblock, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||
if ban.Expires.Sub(ban.Start) != time.Hour ||
|
||||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
||||
t.Errorf(
|
||||
@@ -288,11 +251,7 @@ func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(
|
||||
}
|
||||
|
||||
// The next clear sign of attack bans for seven days, as a first does.
|
||||
ban, _ := ledger.Ban(
|
||||
netblock,
|
||||
now,
|
||||
bans.Grounds{Cause: bans.CauseAttack},
|
||||
)
|
||||
ban, _ := ledger.BanForAttack(netblock, now, bans.Notes{})
|
||||
if ban.Expires.Sub(ban.Start) != 7*day {
|
||||
t.Errorf(
|
||||
"the next ban for an attack ends at %s, want seven days on",
|
||||
@@ -305,10 +264,10 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ledger := bans.New(defaultParams())
|
||||
made, _ := ledger.Ban(
|
||||
made, _ := ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.1/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
atStart := bans.Ban{
|
||||
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
|
||||
@@ -352,7 +311,7 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
|
||||
ledger := bans.New(defaultParams())
|
||||
|
||||
// An hour's ban for a broken limit.
|
||||
ledger.Ban(netblock, midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||
wantChanged(t, ledger, true)
|
||||
|
||||
// A minute later an admin bans the netblock for good, named by an
|
||||
@@ -472,10 +431,10 @@ func TestPermanentBanOnAWiderNetblockIsReturnedOverATemporaryBan(
|
||||
ledger := bans.New(defaultParams())
|
||||
|
||||
// The temporary ban is made first, so that its netblock is looked at first.
|
||||
ledger.Ban(
|
||||
ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.9/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
ledger.BanForAdmin(
|
||||
netip.MustParsePrefix("203.0.113.0/24"),
|
||||
|
||||
+82
-54
@@ -300,50 +300,81 @@ func activeBan(bans []Ban, now time.Time) *Ban {
|
||||
return newest
|
||||
}
|
||||
|
||||
// Grounds are why a request makes a ban: its cause, CauseLimit or
|
||||
// CauseAttack, and its notes.
|
||||
type Grounds struct {
|
||||
Cause string
|
||||
Notes Notes
|
||||
// BanForLimit bans netblock at now for a broken limit, with notes, and
|
||||
// returns the ban, and true. A first ban lasts LimitBanDuration. A ban
|
||||
// made within LimitBanRepeatWindow after the netblock's ban that ended
|
||||
// last, other than one for a clear sign of attack or a lifted one, lasts
|
||||
// repeatFactor times as long as that one. A ban that would be longer
|
||||
// than LimitBanPermanentAbove is permanent instead. If a ban on netblock is still
|
||||
// active, as when two of its requests break a limit at once, that ban is
|
||||
// returned with false, and no other is made. The ledger fills in the
|
||||
// notes' Refused and EarlierBans itself, and gives the ban the reason
|
||||
// "<Kind> per <Window> over the limit of <Limit>", from the notes, such
|
||||
// as "requests per minute over the limit of 1000".
|
||||
func (l *Ledger) BanForLimit(
|
||||
netblock netip.Prefix, now time.Time, notes Notes,
|
||||
) (Ban, bool) {
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
CauseLimit,
|
||||
limitReason(notes),
|
||||
notes,
|
||||
true,
|
||||
)
|
||||
}
|
||||
|
||||
// Ban bans netblock at now on grounds, and returns the ban, and true. If
|
||||
// a ban on netblock is still active, as when two of its requests break a
|
||||
// limit at once, that ban is returned with false, and no other is made.
|
||||
// The ledger fills in the notes' Refused and EarlierBans itself.
|
||||
//
|
||||
// A ban for a broken limit:
|
||||
// - lasts LimitBanDuration the first time;
|
||||
// - lasts repeatFactor times as long as the netblock's ban that ended
|
||||
// last, when made within LimitBanRepeatWindow after it, unless that
|
||||
// ban was for a clear sign of attack or was lifted;
|
||||
// - is permanent instead when it would be longer than
|
||||
// LimitBanPermanentAbove;
|
||||
// - has the reason "<Kind> per <Window> over the limit of <Limit>", from
|
||||
// the notes, such as "requests per minute over the limit of 1000".
|
||||
//
|
||||
// A ban for a clear sign of attack:
|
||||
// - lasts AttackBanDuration the first time;
|
||||
// - is permanent once the netblock has had one that was not lifted;
|
||||
// - has the reason "matched the rule <RuleID>", or "asked for the trap
|
||||
// path <TrapPath>".
|
||||
func (l *Ledger) Ban(
|
||||
netblock netip.Prefix, now time.Time, grounds Grounds,
|
||||
// WouldBanForLimit returns what BanForLimit would, without making the ban:
|
||||
// what observe mode would have done.
|
||||
func (l *Ledger) WouldBanForLimit(
|
||||
netblock netip.Prefix, now time.Time, notes Notes,
|
||||
) (Ban, bool) {
|
||||
return l.ban(netblock, now, grounds, true)
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
CauseLimit,
|
||||
limitReason(notes),
|
||||
notes,
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
// WouldBan returns what Ban would, without making the ban: what observe
|
||||
// mode would have done.
|
||||
func (l *Ledger) WouldBan(
|
||||
netblock netip.Prefix, now time.Time, grounds Grounds,
|
||||
// BanForAttack bans netblock at now for a clear sign of attack, with
|
||||
// notes, and returns the ban, and whether it made it, as BanForLimit
|
||||
// does. A first ban lasts AttackBanDuration; once the netblock has had
|
||||
// one that was not lifted, the next is permanent. Its reason is "matched
|
||||
// the rule <RuleID>", or "asked for the trap path <TrapPath>".
|
||||
func (l *Ledger) BanForAttack(
|
||||
netblock netip.Prefix, now time.Time, notes Notes,
|
||||
) (Ban, bool) {
|
||||
return l.ban(netblock, now, grounds, false)
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
CauseAttack,
|
||||
attackReason(notes),
|
||||
notes,
|
||||
true,
|
||||
)
|
||||
}
|
||||
|
||||
// WouldBanForAttack returns what BanForAttack would, without making the
|
||||
// ban: what observe mode would have done.
|
||||
func (l *Ledger) WouldBanForAttack(
|
||||
netblock netip.Prefix, now time.Time, notes Notes,
|
||||
) (Ban, bool) {
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
CauseAttack,
|
||||
attackReason(notes),
|
||||
notes,
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit
|
||||
// or CauseAttack, made at now would be permanent, as Ban would make it. It
|
||||
// works out nothing else of the ban.
|
||||
// or CauseAttack, made at now would be permanent, as BanForLimit or
|
||||
// BanForAttack would make it. It works out nothing else of the ban.
|
||||
func (l *Ledger) WouldBePermanent(
|
||||
netblock netip.Prefix, now time.Time, cause string,
|
||||
) bool {
|
||||
@@ -380,9 +411,10 @@ func attackReason(notes Notes) string {
|
||||
|
||||
// BanForAdmin bans netblock at now for an admin, with reason, until
|
||||
// expires, or for good when expires is zero, and returns the ban, whose
|
||||
// cause is CauseAdmin. Unlike Ban, it makes the ban even while another
|
||||
// on netblock is active, since the admin asked for this one. The ledger
|
||||
// fills in the notes' EarlierBans, and counts the ban among those made.
|
||||
// cause is CauseAdmin. Unlike BanForLimit and BanForAttack, it makes the
|
||||
// ban even while another on netblock is active, since the admin asked
|
||||
// for this one. The ledger fills in the notes' EarlierBans, and counts
|
||||
// the ban among those made.
|
||||
func (l *Ledger) BanForAdmin(
|
||||
netblock netip.Prefix, now, expires time.Time, reason string,
|
||||
) Ban {
|
||||
@@ -727,13 +759,15 @@ func (l *Ledger) forgetEndedAbuseIPDBBans(now time.Time) bool {
|
||||
return forgot
|
||||
}
|
||||
|
||||
// ban bans netblock at now on grounds, as Ban describes, and returns the
|
||||
// ban, and whether it made it. Unless keep is true, the ban is not made,
|
||||
// only returned: it is the ban that would have been made.
|
||||
// ban bans netblock at now for cause, CauseLimit or CauseAttack, with
|
||||
// reason and notes, as BanForLimit and BanForAttack describe, and returns
|
||||
// the ban, and whether it made it. Unless keep is true, the ban is not
|
||||
// made, only returned: it is the ban that would have been made.
|
||||
func (l *Ledger) ban(
|
||||
netblock netip.Prefix,
|
||||
now time.Time,
|
||||
grounds Grounds,
|
||||
cause, reason string,
|
||||
notes Notes,
|
||||
keep bool,
|
||||
) (Ban, bool) {
|
||||
l.mu.Lock()
|
||||
@@ -742,8 +776,6 @@ func (l *Ledger) ban(
|
||||
// held are the netblock's bans, none of them active.
|
||||
var held []Ban
|
||||
|
||||
notes := grounds.Notes
|
||||
|
||||
bans, found := l.netblocks.Get(netblock)
|
||||
if found {
|
||||
active := activeBan(*bans, now)
|
||||
@@ -759,19 +791,15 @@ func (l *Ledger) ban(
|
||||
ban := Ban{
|
||||
Netblock: netblock,
|
||||
Start: now,
|
||||
Cause: grounds.Cause,
|
||||
Cause: cause,
|
||||
Reason: reason,
|
||||
Notes: notes,
|
||||
}
|
||||
|
||||
switch grounds.Cause {
|
||||
case CauseLimit:
|
||||
ban.Reason = limitReason(notes)
|
||||
ban.Expires = l.limitExpiry(held, now)
|
||||
case CauseAttack:
|
||||
ban.Reason = attackReason(notes)
|
||||
if cause == CauseAttack {
|
||||
ban.Expires = l.attackExpiry(held, now)
|
||||
default:
|
||||
panic("bans: a request's ban has the cause " + grounds.Cause)
|
||||
} else {
|
||||
ban.Expires = l.limitExpiry(held, now)
|
||||
}
|
||||
|
||||
if !keep {
|
||||
@@ -779,7 +807,7 @@ func (l *Ledger) ban(
|
||||
}
|
||||
|
||||
l.add(ban)
|
||||
l.made[grounds.Cause]++
|
||||
l.made[cause]++
|
||||
l.markChanged()
|
||||
|
||||
return ban, true
|
||||
|
||||
+51
-121
@@ -22,11 +22,7 @@ func TestRepeatsTripleUntilPermanent(t *testing.T) {
|
||||
// Each ban is followed by another as soon as it ends: 1, 3, 9, 27 and
|
||||
// 81 hours.
|
||||
for i, hours := range []int{1, 3, 9, 27, 81} {
|
||||
ban, _ := ledger.Ban(
|
||||
netblock,
|
||||
now,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||
|
||||
length := time.Duration(hours) * time.Hour
|
||||
if !ban.Expires.Equal(now.Add(length)) ||
|
||||
@@ -46,7 +42,7 @@ func TestRepeatsTripleUntilPermanent(t *testing.T) {
|
||||
|
||||
// The sixth would last 243 hours, more than seven days: it is
|
||||
// permanent, and never ends.
|
||||
ban, _ := ledger.Ban(netblock, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||
if !ban.Permanent() {
|
||||
t.Fatalf("sixth ban ends at %s, want a permanent one", ban.Expires)
|
||||
}
|
||||
@@ -75,15 +71,15 @@ func TestRepeatWindowRunsOut(t *testing.T) {
|
||||
ledger := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
|
||||
first, _ := ledger.Ban(
|
||||
first, _ := ledger.BanForLimit(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
second, _ := ledger.Ban(
|
||||
second, _ := ledger.BanForLimit(
|
||||
netblock,
|
||||
first.Expires.Add(tc.gap),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
if second.Expires.Sub(second.Start) != tc.want ||
|
||||
@@ -108,10 +104,10 @@ func TestFirstBanLongerThanLimitBanPermanentAboveIsPermanent(t *testing.T) {
|
||||
params.LimitBanDuration = params.LimitBanPermanentAbove + time.Hour
|
||||
ledger := bans.New(params)
|
||||
|
||||
ban, _ := ledger.Ban(
|
||||
ban, _ := ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.9/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
if !ban.Permanent() {
|
||||
t.Errorf("first ban ends at %s, want a permanent one", ban.Expires)
|
||||
@@ -131,11 +127,7 @@ func TestLimitBanPermanentAboveSetFarOffDoesNotOverflow(t *testing.T) {
|
||||
now := midnight()
|
||||
|
||||
for i := range 14 {
|
||||
ban, _ := ledger.Ban(
|
||||
netblock,
|
||||
now,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||
if !ban.Expires.After(ban.Start) {
|
||||
t.Fatalf(
|
||||
"ban %d starts at %s and ends at %s",
|
||||
@@ -148,7 +140,7 @@ func TestLimitBanPermanentAboveSetFarOffDoesNotOverflow(t *testing.T) {
|
||||
now = ban.Expires
|
||||
}
|
||||
|
||||
ban, _ := ledger.Ban(netblock, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||
if !ban.Permanent() {
|
||||
t.Errorf("15th ban ends at %s, want a permanent one", ban.Expires)
|
||||
}
|
||||
@@ -160,19 +152,15 @@ func TestBrokenLimitDuringABanMakesNoOther(t *testing.T) {
|
||||
ledger := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
|
||||
first, made := ledger.Ban(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
first, made := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||
if !made {
|
||||
t.Error("the first ban was not made")
|
||||
}
|
||||
|
||||
again, made := ledger.Ban(
|
||||
again, made := ledger.BanForLimit(
|
||||
netblock,
|
||||
midnight().Add(time.Minute),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
if made || !reflect.DeepEqual(again, first) ||
|
||||
@@ -187,10 +175,10 @@ func TestBrokenLimitDuringABanMakesNoOther(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
again, made = ledger.Ban(
|
||||
again, made = ledger.BanForAttack(
|
||||
netblock,
|
||||
midnight().Add(time.Minute),
|
||||
bans.Grounds{Cause: bans.CauseAttack},
|
||||
bans.Notes{},
|
||||
)
|
||||
if made || !reflect.DeepEqual(again, first) {
|
||||
t.Errorf(
|
||||
@@ -207,10 +195,10 @@ func TestCheckRefusesWhileTheBanLastsAndCountsTheRefusals(t *testing.T) {
|
||||
|
||||
ledger := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
ban, _ := ledger.Ban(
|
||||
ban, _ := ledger.BanForLimit(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit, Notes: bans.Notes{Requests: 5}},
|
||||
bans.Notes{Requests: 5},
|
||||
)
|
||||
|
||||
for range 3 {
|
||||
@@ -249,10 +237,10 @@ func TestFindCountsNothing(t *testing.T) {
|
||||
|
||||
ledger := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
ban, _ := ledger.Ban(
|
||||
ban, _ := ledger.BanForLimit(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit, Notes: bans.Notes{Requests: 5}},
|
||||
bans.Notes{Requests: 5},
|
||||
)
|
||||
|
||||
got, banned, _ := ledger.Find(
|
||||
@@ -299,24 +287,24 @@ func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(
|
||||
d := netip.MustParsePrefix("2001:db8::/64")
|
||||
now := midnight()
|
||||
|
||||
first, _ := ledger.Ban(a, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.Ban(b, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.Ban(c, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
first, _ := ledger.BanForLimit(a, now, bans.Notes{})
|
||||
ledger.BanForLimit(b, now, bans.Notes{})
|
||||
ledger.BanForLimit(c, now, bans.Notes{})
|
||||
|
||||
// A request from a makes b the netblock seen longest ago, and its ban
|
||||
// goes to make room for d's.
|
||||
ledger.Check(a.Addr(), now)
|
||||
ledger.Ban(d, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(d, now, bans.Notes{})
|
||||
wantBans(t, ledger, map[netip.Prefix]int{a: 1, b: 0, c: 1, d: 1})
|
||||
|
||||
// a is banned again once its ban has ended; c, seen longest ago, goes.
|
||||
ledger.Ban(a, first.Expires, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(a, first.Expires, bans.Notes{})
|
||||
wantBans(t, ledger, map[netip.Prefix]int{a: 2, c: 0, d: 1})
|
||||
|
||||
// With d seen since, a is seen longest ago, and its earlier ban goes
|
||||
// first.
|
||||
ledger.Check(d.Addr(), first.Expires)
|
||||
ledger.Ban(b, first.Expires, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(b, first.Expires, bans.Notes{})
|
||||
wantBans(t, ledger, map[netip.Prefix]int{a: 1, b: 1, d: 1})
|
||||
|
||||
if !ledger.Bans(a)[0].Start.Equal(first.Expires) {
|
||||
@@ -337,16 +325,8 @@ func TestFullLedgerDropsTheEarlierBanOfTheNetblockBannedAgain(t *testing.T) {
|
||||
ledger := bans.New(params)
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
|
||||
first, _ := ledger.Ban(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
second, _ := ledger.Ban(
|
||||
netblock,
|
||||
first.Expires,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
|
||||
|
||||
held := ledger.Bans(netblock)
|
||||
if len(held) != 1 || !reflect.DeepEqual(held[0], second) ||
|
||||
@@ -363,11 +343,7 @@ func TestRequestDuringAnAttackBanMakesItPermanent(t *testing.T) {
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
notes := bans.Notes{RuleID: "env-file", Target: "path"}
|
||||
|
||||
ban, _ := ledger.Ban(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseAttack, Notes: notes},
|
||||
)
|
||||
ban, _ := ledger.BanForAttack(netblock, midnight(), notes)
|
||||
if !ban.Expires.Equal(midnight().Add(7*day)) ||
|
||||
ban.Cause != bans.CauseAttack ||
|
||||
ban.Notes.RuleID != "env-file" ||
|
||||
@@ -442,16 +418,8 @@ func TestAttackAfterAnAttackBanHasEndedBansPermanently(t *testing.T) {
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
|
||||
// A ban for a broken limit before does not count.
|
||||
first, _ := ledger.Ban(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
second, _ := ledger.Ban(
|
||||
netblock,
|
||||
first.Expires,
|
||||
bans.Grounds{Cause: bans.CauseAttack},
|
||||
)
|
||||
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||
second, _ := ledger.BanForAttack(netblock, first.Expires, bans.Notes{})
|
||||
|
||||
if second.Expires.Sub(second.Start) != 7*day {
|
||||
t.Fatalf("the first ban for an attack lasts %s, want 7 days",
|
||||
@@ -467,10 +435,10 @@ func TestAttackAfterAnAttackBanHasEndedBansPermanently(t *testing.T) {
|
||||
|
||||
// Its notes show the earlier ban for an attack that makes it permanent,
|
||||
// beside the one for a limit.
|
||||
third, _ := ledger.Ban(
|
||||
third, _ := ledger.BanForAttack(
|
||||
netblock,
|
||||
second.Expires.Add(30*day),
|
||||
bans.Grounds{Cause: bans.CauseAttack},
|
||||
bans.Notes{},
|
||||
)
|
||||
if !third.Permanent() ||
|
||||
third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Attack: 1}) {
|
||||
@@ -484,18 +452,14 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
|
||||
|
||||
ledger := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
first, _ := ledger.Ban(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||
wantChanged(t, ledger, true)
|
||||
|
||||
// While the first ban lasts, none would be made.
|
||||
during, would := ledger.WouldBan(
|
||||
during, would := ledger.WouldBanForAttack(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseAttack},
|
||||
bans.Notes{},
|
||||
)
|
||||
if would || !reflect.DeepEqual(during, first) {
|
||||
t.Errorf(
|
||||
@@ -508,19 +472,13 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
|
||||
|
||||
// As it ends, a clear sign of attack would ban for seven days, and a
|
||||
// limit broken again for three hours, but neither is made.
|
||||
limitGrounds := bans.Grounds{
|
||||
Cause: bans.CauseLimit,
|
||||
Notes: bans.Notes{Kind: "requests", Limit: 1, Window: "minute"},
|
||||
}
|
||||
attack, wouldAttack := ledger.WouldBan(netblock, first.Expires,
|
||||
bans.Grounds{
|
||||
Cause: bans.CauseAttack,
|
||||
Notes: bans.Notes{RuleID: "git-dir"},
|
||||
})
|
||||
limit, wouldLimit := ledger.WouldBan(
|
||||
limitNotes := bans.Notes{Kind: "requests", Limit: 1, Window: "minute"}
|
||||
attack, wouldAttack := ledger.WouldBanForAttack(netblock, first.Expires,
|
||||
bans.Notes{RuleID: "git-dir"})
|
||||
limit, wouldLimit := ledger.WouldBanForLimit(
|
||||
netblock,
|
||||
first.Expires,
|
||||
limitGrounds,
|
||||
limitNotes,
|
||||
)
|
||||
|
||||
if !wouldAttack || !attack.Expires.Equal(first.Expires.Add(7*day)) ||
|
||||
@@ -547,7 +505,7 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
|
||||
wantChanged(t, ledger, false)
|
||||
|
||||
// The ban made is the one that would have been.
|
||||
made, _ := ledger.Ban(netblock, first.Expires, limitGrounds)
|
||||
made, _ := ledger.BanForLimit(netblock, first.Expires, limitNotes)
|
||||
if !reflect.DeepEqual(made, limit) {
|
||||
t.Errorf("the ban made is %+v, want %+v", made, limit)
|
||||
}
|
||||
@@ -568,11 +526,7 @@ func TestWouldBePermanentAnswersAsTheBanWouldBeMade(t *testing.T) {
|
||||
t.Fatalf("ban %d for a limit would be permanent", i+1)
|
||||
}
|
||||
|
||||
ban, _ := ledger.Ban(
|
||||
netblock,
|
||||
now,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||
now = ban.Expires
|
||||
}
|
||||
|
||||
@@ -585,11 +539,7 @@ func TestWouldBePermanentAnswersAsTheBanWouldBeMade(t *testing.T) {
|
||||
}
|
||||
|
||||
// Once a first ban for an attack has ended, the next would be permanent.
|
||||
attack, _ := ledger.Ban(
|
||||
netblock,
|
||||
now,
|
||||
bans.Grounds{Cause: bans.CauseAttack},
|
||||
)
|
||||
attack, _ := ledger.BanForAttack(netblock, now, bans.Notes{})
|
||||
if !ledger.WouldBePermanent(netblock, attack.Expires, bans.CauseAttack) {
|
||||
t.Error("a second ban for an attack would not be permanent")
|
||||
}
|
||||
@@ -603,16 +553,8 @@ func TestAttackBanDoesNotLengthenTheNextBanForALimit(t *testing.T) {
|
||||
|
||||
// Three times the seven days would be permanent; a limit broken as the
|
||||
// ban for an attack ends bans for an hour, as a first broken limit does.
|
||||
attack, _ := ledger.Ban(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseAttack},
|
||||
)
|
||||
limit, _ := ledger.Ban(
|
||||
netblock,
|
||||
attack.Expires,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
attack, _ := ledger.BanForAttack(netblock, midnight(), bans.Notes{})
|
||||
limit, _ := ledger.BanForLimit(netblock, attack.Expires, bans.Notes{})
|
||||
|
||||
if limit.Expires.Sub(limit.Start) != time.Hour ||
|
||||
limit.Cause != bans.CauseLimit {
|
||||
@@ -636,13 +578,10 @@ func TestRequestTextsAreCutTo256Bytes(t *testing.T) {
|
||||
Time: midnight(), Method: long, Host: long, Path: long, Status: 403, UserAgent: long,
|
||||
}
|
||||
|
||||
ban, _ := ledger.Ban(
|
||||
ban, _ := ledger.BanForLimit(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{
|
||||
Cause: bans.CauseLimit,
|
||||
Notes: bans.Notes{Request: request},
|
||||
},
|
||||
bans.Notes{Request: request},
|
||||
)
|
||||
|
||||
cut := long[:256]
|
||||
@@ -668,20 +607,11 @@ func TestLookupFillsTheNotesOfTheNetblocksBansWithoutOne(t *testing.T) {
|
||||
|
||||
// A ban made with the client's lookup, one made before it came, after
|
||||
// the first ended, and one on another netblock.
|
||||
ledger.Ban(netblock, midnight(), bans.Grounds{
|
||||
Cause: bans.CauseLimit,
|
||||
Notes: bans.Notes{
|
||||
ASN: "AS64497",
|
||||
ASName: "Other Net",
|
||||
Country: "FR",
|
||||
},
|
||||
ledger.BanForLimit(netblock, midnight(), bans.Notes{
|
||||
ASN: "AS64497", ASName: "Other Net", Country: "FR",
|
||||
})
|
||||
ledger.Ban(
|
||||
netblock,
|
||||
midnight().Add(time.Hour),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
ledger.Ban(other, midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(netblock, midnight().Add(time.Hour), bans.Notes{})
|
||||
ledger.BanForLimit(other, midnight(), bans.Notes{})
|
||||
|
||||
ledger.AddLookup(netblock, "AS64496", "Example Net", "DE")
|
||||
|
||||
|
||||
@@ -19,29 +19,25 @@ func TestChangedAfterABanIsMade(t *testing.T) {
|
||||
|
||||
wantChanged(t, ledger, false)
|
||||
|
||||
ledger.Ban(netblock, midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||
wantChanged(t, ledger, true)
|
||||
|
||||
// A limit broken during the ban makes no other, and a refusal changes
|
||||
// only the counts in the notes, which wait for the interval's write.
|
||||
ledger.Ban(
|
||||
netblock,
|
||||
midnight().Add(time.Minute),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
|
||||
ledger.Check(netblock.Addr(), midnight().Add(time.Minute))
|
||||
wantChanged(t, ledger, false)
|
||||
|
||||
// Two bans before the value is read leave one.
|
||||
ledger.Ban(
|
||||
ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.10/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
ledger.Ban(
|
||||
ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.11/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
wantChanged(t, ledger, true)
|
||||
wantChanged(t, ledger, false)
|
||||
@@ -55,14 +51,10 @@ func TestSnapshotListsEveryBanByNetblock(t *testing.T) {
|
||||
high := netip.MustParsePrefix("203.0.113.10/32")
|
||||
low := netip.MustParsePrefix("203.0.113.9/32")
|
||||
|
||||
first, _ := ledger.Ban(
|
||||
v6,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
ledger.Ban(high, midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.Ban(low, midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
ledger.Ban(v6, first.Expires, bans.Grounds{Cause: bans.CauseLimit})
|
||||
first, _ := ledger.BanForLimit(v6, midnight(), bans.Notes{})
|
||||
ledger.BanForLimit(high, midnight(), bans.Notes{})
|
||||
ledger.BanForLimit(low, midnight(), bans.Notes{})
|
||||
ledger.BanForLimit(v6, first.Expires, bans.Notes{})
|
||||
|
||||
snapshot := ledger.Snapshot()
|
||||
|
||||
@@ -88,10 +80,7 @@ func TestLoadedBansCarryOn(t *testing.T) {
|
||||
|
||||
before := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
ban, _ := before.Ban(netblock, midnight(), bans.Grounds{
|
||||
Cause: bans.CauseLimit,
|
||||
Notes: bans.Notes{Limit: 1},
|
||||
})
|
||||
ban, _ := before.BanForLimit(netblock, midnight(), bans.Notes{Limit: 1})
|
||||
|
||||
// Loaded into a new ledger, as across a restart, the ban still refuses
|
||||
// while it lasts, and once it has ended a broken limit bans for three
|
||||
@@ -107,11 +96,7 @@ func TestLoadedBansCarryOn(t *testing.T) {
|
||||
t.Error("the loaded ban does not refuse")
|
||||
}
|
||||
|
||||
again, _ := after.Ban(
|
||||
netblock,
|
||||
ban.Expires,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
again, _ := after.BanForLimit(netblock, ban.Expires, bans.Notes{})
|
||||
if again.Expires.Sub(again.Start) != 3*time.Hour ||
|
||||
again.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
||||
t.Errorf(
|
||||
@@ -139,10 +124,10 @@ func TestLoadedBanRefusesEveryClientInItsNetblock(t *testing.T) {
|
||||
Start: midnight(),
|
||||
},
|
||||
})
|
||||
ledger.Ban(
|
||||
ledger.BanForLimit(
|
||||
netip.MustParsePrefix("198.51.100.7/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
for client, want := range map[string]bool{
|
||||
@@ -215,7 +200,7 @@ func TestPermanentBanStartedBeforeAnEndedOneRefuses(t *testing.T) {
|
||||
}
|
||||
|
||||
// A limit broken now makes no shorter ban over the permanent one.
|
||||
ban, _ = ledger.Ban(netblock, now, bans.Grounds{Cause: bans.CauseLimit})
|
||||
ban, _ = ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||
if !ban.Permanent() || len(ledger.Bans(netblock)) != 2 {
|
||||
t.Errorf(
|
||||
"a broken limit returned %+v and left the netblock %d bans, "+
|
||||
@@ -251,10 +236,10 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
|
||||
// Once both have ended, a limit broken within the repeat window bans
|
||||
// for three times the 9 hours, and the notes count the two bans
|
||||
// before the 9-hour one and it, for a limit, and the admin's.
|
||||
ban, _ := ledger.Ban(
|
||||
ban, _ := ledger.BanForLimit(
|
||||
netblock,
|
||||
nineHours.Expires.Add(time.Hour),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
|
||||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) {
|
||||
@@ -328,15 +313,15 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
|
||||
}
|
||||
|
||||
// The ledger holds one ban, so it makes two more without dropping any.
|
||||
first, _ := ledger.Ban(
|
||||
first, _ := ledger.BanForLimit(
|
||||
netip.MustParsePrefix("198.51.100.7/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
second, _ := ledger.Ban(
|
||||
second, _ := ledger.BanForLimit(
|
||||
netip.MustParsePrefix("198.51.100.8/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
want := []bans.Ban{first, second, kept}
|
||||
|
||||
+32
-93
@@ -75,7 +75,7 @@ type WAF struct {
|
||||
BytesLimitPerMinute int64
|
||||
BytesLimitPerHour int64
|
||||
BytesLimitPerDay int64
|
||||
BytesCounted BytesCounted
|
||||
BytesCounted string
|
||||
// LookupSource is where each client's AS number and country are looked
|
||||
// up (SWW_WAF_LOOKUP_SOURCE): geojs, file, or off for nowhere.
|
||||
// LookupDBPath is the lookup database, the IPinfo Lite file looked up in
|
||||
@@ -86,7 +86,7 @@ type WAF struct {
|
||||
// AddLookupHeaders is true when the app is passed the client's AS number
|
||||
// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS), by default unless
|
||||
// LookupSource is off.
|
||||
LookupSource LookupSource
|
||||
LookupSource string
|
||||
LookupDBPath string
|
||||
ASNAndCountryLookupTimeout time.Duration
|
||||
AddLookupHeaders bool
|
||||
@@ -131,7 +131,7 @@ type WAF struct {
|
||||
// BlocklistLimitPercent is the percentage of every limit it gets.
|
||||
BlocklistURLs []string
|
||||
BlocklistRefresh time.Duration
|
||||
BlocklistAction ReputationAction
|
||||
BlocklistAction string
|
||||
BlocklistLimitPercent int64
|
||||
// DNSBLZones are the DNSBL zones clients are asked about
|
||||
// (SWW_WAF_DNSBL_ZONES), through DNSBLResolver (SWW_WAF_DNSBL_RESOLVER), or
|
||||
@@ -154,7 +154,7 @@ type WAF struct {
|
||||
AbuseIPDBKey string
|
||||
AbuseIPDBMinScore int64
|
||||
AbuseIPDBDailyBudget int
|
||||
DNSBLAbuseIPDBAction ReputationAction
|
||||
DNSBLAbuseIPDBAction string
|
||||
DNSBLAbuseIPDBLimitPercent int64
|
||||
IPReputationCacheTTL time.Duration
|
||||
DNSBLAbuseIPDBTimeout time.Duration
|
||||
@@ -237,7 +237,7 @@ type WAF struct {
|
||||
// reads none.
|
||||
// CoreRuleSetExclusionsFile is the path of the file of its exclusions
|
||||
// (SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while there is none.
|
||||
CoreRuleSetMode CoreRuleSetMode
|
||||
CoreRuleSetMode string
|
||||
CoreRuleSetParanoiaLevel int
|
||||
CoreRuleSetScoreThreshold int
|
||||
CoreRuleSetDisabledRules []int
|
||||
@@ -316,66 +316,20 @@ type WAF struct {
|
||||
settings []slog.Attr
|
||||
}
|
||||
|
||||
// BytesCounted is which body bytes count toward the byte limits and the
|
||||
// byte anomaly thresholds, a value of SWW_WAF_BYTES_COUNTED.
|
||||
type BytesCounted string
|
||||
|
||||
// The values of SWW_WAF_BYTES_COUNTED.
|
||||
const (
|
||||
// BytesCountedResponse counts the response's body bytes.
|
||||
BytesCountedResponse BytesCounted = "response"
|
||||
// BytesCountedRequest counts the request's body bytes.
|
||||
BytesCountedRequest BytesCounted = "request"
|
||||
// BytesCountedBoth counts the body bytes of both.
|
||||
BytesCountedBoth BytesCounted = "both"
|
||||
)
|
||||
|
||||
// LookupSource is where each client's AS number and country are looked up,
|
||||
// a value of SWW_WAF_LOOKUP_SOURCE.
|
||||
type LookupSource string
|
||||
|
||||
// The values of SWW_WAF_LOOKUP_SOURCE.
|
||||
const (
|
||||
// LookupSourceGeoJS asks GeoJS.
|
||||
LookupSourceGeoJS LookupSource = "geojs"
|
||||
// LookupSourceFile looks clients up in the lookup database, the file
|
||||
// SWW_WAF_LOOKUP_DB_PATH names.
|
||||
LookupSourceFile LookupSource = "file"
|
||||
// LookupSourceOff looks no client up.
|
||||
LookupSourceOff LookupSource = "off"
|
||||
)
|
||||
|
||||
// ReputationAction is what is done with a client a reputation source
|
||||
// lists, a value of SWW_WAF_BLOCKLIST_ACTION and of
|
||||
// SWW_WAF_DNSBL_ABUSEIPDB_ACTION.
|
||||
type ReputationAction string
|
||||
|
||||
// The values of SWW_WAF_BLOCKLIST_ACTION and SWW_WAF_DNSBL_ABUSEIPDB_ACTION.
|
||||
const (
|
||||
// ReputationActionDeny refuses the client's requests.
|
||||
ReputationActionDeny ReputationAction = "deny"
|
||||
// ReputationActionLimit gives the client a percentage of every limit,
|
||||
// written limit:<percent> in the setting.
|
||||
ReputationActionLimit ReputationAction = "limit"
|
||||
// ReputationActionLog only notes the hit in the log line and raises an
|
||||
// alert.
|
||||
ReputationActionLog ReputationAction = "log"
|
||||
)
|
||||
|
||||
// CoreRuleSetMode is what the Core Rule Set does, a value of
|
||||
// SWW_WAF_CORE_RULE_SET_MODE.
|
||||
type CoreRuleSetMode string
|
||||
|
||||
// The values of SWW_WAF_CORE_RULE_SET_MODE.
|
||||
const (
|
||||
// CoreRuleSetModeOff runs no request through the Core Rule Set.
|
||||
CoreRuleSetModeOff CoreRuleSetMode = "off"
|
||||
CoreRuleSetModeOff = off
|
||||
// CoreRuleSetModeDetect logs and alerts a match, and refuses nothing.
|
||||
CoreRuleSetModeDetect CoreRuleSetMode = "detect"
|
||||
CoreRuleSetModeDetect = "detect"
|
||||
// CoreRuleSetModeBlock refuses a match with 403.
|
||||
CoreRuleSetModeBlock CoreRuleSetMode = "block"
|
||||
CoreRuleSetModeBlock = "block"
|
||||
)
|
||||
|
||||
// fileSource is the SWW_WAF_LOOKUP_SOURCE that looks clients up in the
|
||||
// lookup database, the file SWW_WAF_LOOKUP_DB_PATH names.
|
||||
const fileSource = "file"
|
||||
|
||||
const (
|
||||
day = 24 * time.Hour
|
||||
ipv4Bits = 32
|
||||
@@ -604,13 +558,10 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
|
||||
),
|
||||
BytesLimitPerHour: env.size("SWW_WAF_BYTES_LIMIT_PER_HOUR", "50G"),
|
||||
BytesLimitPerDay: env.size("SWW_WAF_BYTES_LIMIT_PER_DAY", "100G"),
|
||||
BytesCounted: env.bytesCounted(
|
||||
"SWW_WAF_BYTES_COUNTED",
|
||||
BytesCountedBoth,
|
||||
),
|
||||
BytesCounted: env.bytesCounted("SWW_WAF_BYTES_COUNTED", "both"),
|
||||
LookupSource: env.lookupSource(
|
||||
"SWW_WAF_LOOKUP_SOURCE",
|
||||
LookupSourceGeoJS,
|
||||
"geojs",
|
||||
),
|
||||
LookupDBPath: env.value("SWW_WAF_LOOKUP_DB_PATH", ""),
|
||||
ASNAndCountryLookupTimeout: env.milliseconds(
|
||||
@@ -807,9 +758,9 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
|
||||
cfg.InstanceName, cfg.LogRemoteURL != nil)
|
||||
// The app gets the headers by default, unless no client is looked up.
|
||||
cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS",
|
||||
strconv.FormatBool(cfg.LookupSource != LookupSourceOff))
|
||||
strconv.FormatBool(cfg.LookupSource != off))
|
||||
cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action(
|
||||
"SWW_WAF_BLOCKLIST_ACTION", string(ReputationActionDeny))
|
||||
"SWW_WAF_BLOCKLIST_ACTION", "deny")
|
||||
cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action(
|
||||
"SWW_WAF_DNSBL_ABUSEIPDB_ACTION", "limit:25")
|
||||
|
||||
@@ -907,12 +858,9 @@ func (e *environment) count(name, defaultValue string) int64 {
|
||||
|
||||
// bytesCounted reads the setting that is which body bytes count toward the
|
||||
// byte limits and the byte anomaly thresholds: response, request or both.
|
||||
func (e *environment) bytesCounted(
|
||||
name string, defaultValue BytesCounted,
|
||||
) BytesCounted {
|
||||
value := BytesCounted(e.value(name, string(defaultValue)))
|
||||
if value != BytesCountedResponse && value != BytesCountedRequest &&
|
||||
value != BytesCountedBoth {
|
||||
func (e *environment) bytesCounted(name, defaultValue string) string {
|
||||
value := e.value(name, defaultValue)
|
||||
if value != "response" && value != "request" && value != "both" {
|
||||
e.check(name, fmt.Errorf("%q %w", value, errNotBytesCounted))
|
||||
}
|
||||
|
||||
@@ -957,10 +905,8 @@ func (e *environment) trapPaths(name string) []string {
|
||||
|
||||
// coreRuleSetMode reads the setting that is what the Core Rule Set does: off,
|
||||
// detect or block.
|
||||
func (e *environment) coreRuleSetMode(
|
||||
name string, defaultValue CoreRuleSetMode,
|
||||
) CoreRuleSetMode {
|
||||
mode := CoreRuleSetMode(e.value(name, string(defaultValue)))
|
||||
func (e *environment) coreRuleSetMode(name, defaultValue string) string {
|
||||
mode := e.value(name, defaultValue)
|
||||
if mode != CoreRuleSetModeOff && mode != CoreRuleSetModeDetect &&
|
||||
mode != CoreRuleSetModeBlock {
|
||||
e.check(name, fmt.Errorf("%q %w", mode, errNotCoreRuleSetMode))
|
||||
@@ -1110,24 +1056,20 @@ func (e *environment) refresh(name, defaultValue string) time.Duration {
|
||||
// action reads a setting that is what is done with a client a blocklist
|
||||
// or a DNSBL zone lists: deny, log, or limit:<percent>, which it returns
|
||||
// as limit and the percentage.
|
||||
func (e *environment) action(
|
||||
name, defaultValue string,
|
||||
) (ReputationAction, int64) {
|
||||
value := ReputationAction(e.value(name, defaultValue))
|
||||
if value == ReputationActionDeny || value == ReputationActionLog {
|
||||
func (e *environment) action(name, defaultValue string) (string, int64) {
|
||||
value := e.value(name, defaultValue)
|
||||
if value == "deny" || value == "log" {
|
||||
return value, 0
|
||||
}
|
||||
|
||||
percentText, isLimit := strings.CutPrefix(
|
||||
string(value), string(ReputationActionLimit)+":",
|
||||
)
|
||||
percentText, isLimit := strings.CutPrefix(value, "limit:")
|
||||
|
||||
percent, err := ParsePercent(percentText)
|
||||
if !isLimit || err != nil {
|
||||
e.check(name, fmt.Errorf("%q %w", value, errNotAction))
|
||||
}
|
||||
|
||||
return ReputationActionLimit, percent
|
||||
return "limit", percent
|
||||
}
|
||||
|
||||
// zones reads the setting that is the list of DNSBL zones. It is empty by
|
||||
@@ -1178,12 +1120,9 @@ func (e *environment) routeWatchURL(name string) string {
|
||||
|
||||
// lookupSource reads the setting that is where clients are looked up:
|
||||
// geojs, file, or off.
|
||||
func (e *environment) lookupSource(
|
||||
name string, defaultValue LookupSource,
|
||||
) LookupSource {
|
||||
source := LookupSource(e.value(name, string(defaultValue)))
|
||||
if source != LookupSourceGeoJS && source != LookupSourceFile &&
|
||||
source != LookupSourceOff {
|
||||
func (e *environment) lookupSource(name, defaultValue string) string {
|
||||
source := e.value(name, defaultValue)
|
||||
if source != "geojs" && source != fileSource && source != off {
|
||||
e.check(name, fmt.Errorf("%q %w", source, errNotLookupSource))
|
||||
}
|
||||
|
||||
@@ -1195,12 +1134,12 @@ func (e *environment) lookupSource(
|
||||
// one source at a time.
|
||||
func (e *environment) checkLookupDBPath(cfg *WAF) {
|
||||
switch {
|
||||
case cfg.LookupSource == LookupSourceFile && cfg.LookupDBPath == "":
|
||||
case cfg.LookupSource == fileSource && cfg.LookupDBPath == "":
|
||||
e.check("SWW_WAF_LOOKUP_SOURCE", fmt.Errorf(
|
||||
"is file while SWW_WAF_LOOKUP_DB_PATH is unset; %w",
|
||||
errNeedsDBPath,
|
||||
))
|
||||
case cfg.LookupSource != LookupSourceFile && cfg.LookupDBPath != "":
|
||||
case cfg.LookupSource != fileSource && cfg.LookupDBPath != "":
|
||||
e.check("SWW_WAF_LOOKUP_DB_PATH", fmt.Errorf(
|
||||
"is set while SWW_WAF_LOOKUP_SOURCE is %s; %w",
|
||||
cfg.LookupSource,
|
||||
@@ -1225,7 +1164,7 @@ func (e *environment) checkCountriesAndLookups(cfg *WAF) {
|
||||
}
|
||||
}
|
||||
|
||||
if cfg.LookupSource != LookupSourceOff {
|
||||
if cfg.LookupSource != off {
|
||||
return
|
||||
}
|
||||
|
||||
|
||||
@@ -614,7 +614,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
||||
},
|
||||
{
|
||||
environment{
|
||||
coreRuleSetMode: string(config.CoreRuleSetModeDetect),
|
||||
coreRuleSetMode: config.CoreRuleSetModeDetect,
|
||||
coreRuleSetParanoiaLevel: "4",
|
||||
coreRuleSetScoreThreshold: "10",
|
||||
coreRuleSetDisabledRules: "942100, 920350",
|
||||
@@ -2033,9 +2033,8 @@ func TestReputationSettingsAsSet(t *testing.T) {
|
||||
}
|
||||
|
||||
for _, tc := range []struct {
|
||||
value string
|
||||
action config.ReputationAction
|
||||
percent int64
|
||||
value, action string
|
||||
percent int64
|
||||
}{
|
||||
{actionDeny, actionDeny, 0},
|
||||
{actionLog, actionLog, 0},
|
||||
@@ -2168,7 +2167,7 @@ func TestDNSBLSettingsAsSet(t *testing.T) {
|
||||
// want are the zones, resolver, action and percent config.WAF gives.
|
||||
want []string
|
||||
wantResolver string
|
||||
wantAction config.ReputationAction
|
||||
wantAction string
|
||||
wantPercent int64
|
||||
}{
|
||||
{
|
||||
@@ -3050,7 +3049,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
logLevel: "info",
|
||||
ruleFilesDir: "/etc/sww/rules.d",
|
||||
ruleFilesEnabled: "true",
|
||||
coreRuleSetMode: string(config.CoreRuleSetModeBlock),
|
||||
coreRuleSetMode: config.CoreRuleSetModeBlock,
|
||||
coreRuleSetParanoiaLevel: "1",
|
||||
coreRuleSetScoreThreshold: "5",
|
||||
coreRuleSetDisabledRules: defaultCoreRuleSetDisabledRules,
|
||||
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -37,9 +37,10 @@
|
||||
SecRule &TX:app_nextcloud "@eq 0" \
|
||||
"id:900090,phase:1,pass,nolog,skipAfter:END_NEXTCLOUD"
|
||||
|
||||
# The sync client and the Files app use these WebDAV methods besides those
|
||||
# the first change allows, on /remote.php/dav/ and /public.php/dav/ and on the
|
||||
# older /remote.php/webdav/ and /public.php/webdav/. 911100 refused each:
|
||||
# The sync client and the Files app use these WebDAV methods besides PUT,
|
||||
# PATCH and DELETE, which sww allows for every app, on /remote.php/dav/ and
|
||||
# /public.php/dav/ and on the older /remote.php/webdav/ and /public.php/webdav/.
|
||||
# 911100 refused each:
|
||||
# PROPFIND, PROPPATCH: listing a folder, marking a file a favourite
|
||||
# MKCOL: making a folder, and starting a chunked upload
|
||||
# COPY, MOVE: copying, moving and renaming, and ending a chunked upload
|
||||
File renamed without changes.
File renamed without changes.
@@ -12,11 +12,10 @@
|
||||
package coreruleset
|
||||
|
||||
import (
|
||||
_ "embed"
|
||||
"embed"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"maps"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
@@ -35,7 +34,7 @@ import (
|
||||
// directives are the Core Rule Set as sww runs it, with the
|
||||
// paranoia level for %d, bodyDirectives for the first %s while
|
||||
// SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, the exclusions file for
|
||||
// the second, and the directives of every app in apps for the third.
|
||||
// the second, and the fixes of every app in appFixes for the third.
|
||||
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
||||
// Core Rule Set keeps for the rules that set it up, which
|
||||
// SWW_WAF_CORE_RULE_SET_DISABLED_RULES and the exclusions file refuse, so that no
|
||||
@@ -54,13 +53,12 @@ Include @crs-setup.conf.example
|
||||
SecAction "id:900000,phase:1,pass,nolog,\
|
||||
setvar:tx.blocking_paranoia_level=%d"
|
||||
|
||||
# The first change: PUT, PATCH and DELETE are allowed besides GET, HEAD,
|
||||
# POST and OPTIONS.
|
||||
# PUT, PATCH and DELETE are allowed besides GET, HEAD, POST and OPTIONS.
|
||||
SecAction "id:900200,phase:1,pass,nolog,\
|
||||
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
|
||||
|
||||
# The second: Expect and Content-Encoding are taken off the Core Rule Set's
|
||||
# list of the headers it refuses. Content-Encoding goes back on it for a
|
||||
# Expect and Content-Encoding are taken off the Core Rule Set's list of the
|
||||
# headers it refuses. Content-Encoding goes back on it for a
|
||||
# body the Core Rule Set reads (900260 in bodyDirectives).
|
||||
SecAction "id:900250,phase:1,pass,nolog,\
|
||||
setvar:'tx.restricted_headers_basic=/proxy/ /lock-token/ /content-range/ \
|
||||
@@ -86,18 +84,17 @@ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
|
||||
# The apps' fixes come before the rules for the same reason.
|
||||
%s
|
||||
|
||||
# The fifth: only the rules for requests are loaded, and no response is
|
||||
# inspected.
|
||||
# Only the rules for requests are loaded, and no response is inspected.
|
||||
Include @owasp_crs/REQUEST-*.conf
|
||||
|
||||
# The third: redirect_uri is not checked for a URL naming an IP address or
|
||||
# redirect_uri is not checked for a URL naming an IP address or
|
||||
# localhost. Coraza matches a parameter name here without regard to case.
|
||||
# ARGS holds the fields of a form data or multipart body Coraza reads as
|
||||
# well as the query parameters, so a field of this name is left out too.
|
||||
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
|
||||
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
|
||||
|
||||
# The fourth: Referer is not checked for a Unix command without arguments
|
||||
# Referer is not checked for a Unix command without arguments
|
||||
# or for Java starting a process.
|
||||
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
||||
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
||||
@@ -122,9 +119,9 @@ SecRule REQUEST_HEADERS:Content-Type \
|
||||
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?json" \
|
||||
"id:900420,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=JSON"
|
||||
|
||||
# The rest of the second change: Content-Encoding is refused again on a
|
||||
# body of a kind the Core Rule Set reads, since a compressed body cannot be
|
||||
# inspected.
|
||||
# Content-Encoding goes back on the list of the headers the Core Rule Set
|
||||
# refuses (900250 in directives) for a body of a kind it reads, since a
|
||||
# compressed body cannot be inspected.
|
||||
SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
|
||||
"id:900260,phase:1,pass,nolog,\
|
||||
setvar:'tx.restricted_headers_basic=%%{tx.restricted_headers_basic} \
|
||||
@@ -145,15 +142,19 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
|
||||
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
||||
`
|
||||
|
||||
// app is an app whose ordinary requests the Core Rule Set would refuse,
|
||||
// with the fixes that let them pass.
|
||||
// appFixes holds the fixes of each app whose ordinary requests the Core
|
||||
// Rule Set would refuse, in a file named for the app by the name
|
||||
// SWW_WAF_APPS gives it, such as apps/gitea.conf. The fixes are Coraza
|
||||
// exclusions, loaded before the Core Rule Set's rules. Their first rule
|
||||
// skips to the marker they end with, unless Inspect sets the transaction
|
||||
// variable named for the app, such as app_gitea. The skip reaches only
|
||||
// phase 1, so a rule of a later phase must check that variable itself.
|
||||
//
|
||||
//go:embed apps/*.conf
|
||||
var appFixes embed.FS
|
||||
|
||||
// app is what an app's file in appFixes cannot say about it.
|
||||
type app struct {
|
||||
// directives are Coraza exclusions, loaded before the Core Rule Set's
|
||||
// rules. Their first rule skips to the marker they end with, unless
|
||||
// Inspect sets the transaction variable named for the app, such as
|
||||
// app_gitea. The skip reaches only phase 1, so a rule of a later phase
|
||||
// must check that variable itself.
|
||||
directives string
|
||||
// cookies are the cookies the Core Rule Set reads a request without.
|
||||
cookies []string
|
||||
// forkOf names the app this one is a fork of, "" for none. A hostname
|
||||
@@ -161,60 +162,34 @@ type app struct {
|
||||
forkOf string
|
||||
}
|
||||
|
||||
//go:embed gitea.conf
|
||||
var giteaDirectives string
|
||||
|
||||
//go:embed wordpress.conf
|
||||
var wordpressDirectives string
|
||||
|
||||
//go:embed drupal.conf
|
||||
var drupalDirectives string
|
||||
|
||||
//go:embed mediawiki.conf
|
||||
var mediawikiDirectives string
|
||||
|
||||
//go:embed mastodon.conf
|
||||
var mastodonDirectives string
|
||||
|
||||
//go:embed vaultwarden.conf
|
||||
var vaultwardenDirectives string
|
||||
|
||||
//go:embed nextcloud.conf
|
||||
var nextcloudDirectives string
|
||||
|
||||
//go:embed forgejo.conf
|
||||
var forgejoDirectives string
|
||||
|
||||
//go:embed ghost.conf
|
||||
var ghostDirectives string
|
||||
|
||||
// apps are the apps whose fixes sww has, by the name SWW_WAF_APPS gives each.
|
||||
// flash is Forgejo's name for gitea_flash.
|
||||
// apps are the apps that need more than their file in appFixes, by the
|
||||
// name SWW_WAF_APPS gives each. flash is Forgejo's name for gitea_flash.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a map
|
||||
var apps = map[string]app{
|
||||
"drupal": {directives: drupalDirectives},
|
||||
"forgejo": {
|
||||
directives: forgejoDirectives,
|
||||
cookies: []string{"flash"},
|
||||
forkOf: "gitea",
|
||||
cookies: []string{"flash"},
|
||||
forkOf: "gitea",
|
||||
},
|
||||
"ghost": {directives: ghostDirectives},
|
||||
"gitea": {
|
||||
directives: giteaDirectives,
|
||||
cookies: []string{"gitea_flash", "redirect_to"},
|
||||
cookies: []string{"gitea_flash", "redirect_to"},
|
||||
},
|
||||
"mastodon": {directives: mastodonDirectives},
|
||||
"mediawiki": {directives: mediawikiDirectives},
|
||||
"nextcloud": {directives: nextcloudDirectives},
|
||||
"vaultwarden": {directives: vaultwardenDirectives},
|
||||
"wordpress": {directives: wordpressDirectives},
|
||||
}
|
||||
|
||||
// AppNames returns the names of the apps whose fixes sww has, in
|
||||
// alphabetical order.
|
||||
func AppNames() []string {
|
||||
return slices.Sorted(maps.Keys(apps))
|
||||
// An embedded directory always reads.
|
||||
files, _ := appFixes.ReadDir("apps")
|
||||
|
||||
names := make([]string, len(files))
|
||||
for i, file := range files {
|
||||
names[i] = strings.TrimSuffix(file.Name(), ".conf")
|
||||
}
|
||||
|
||||
slices.Sort(names)
|
||||
|
||||
return names
|
||||
}
|
||||
|
||||
// Params are what New needs.
|
||||
@@ -373,8 +348,11 @@ func New(params Params) (*CoreRuleSet, error) {
|
||||
}
|
||||
|
||||
var fixes strings.Builder
|
||||
|
||||
for _, name := range AppNames() {
|
||||
fixes.WriteString(apps[name].directives + "\n")
|
||||
// An embedded file always reads.
|
||||
file, _ := appFixes.ReadFile("apps/" + name + ".conf")
|
||||
fixes.WriteString(string(file) + "\n")
|
||||
}
|
||||
|
||||
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
|
||||
|
||||
@@ -17,7 +17,7 @@ const (
|
||||
)
|
||||
|
||||
// giteaSearches are searches of the pages gitea and Forgejo share whose q
|
||||
// skips the lists of system files and command names, in the eighth change.
|
||||
// skips the lists of system files and command names, in gitea's fixes.
|
||||
// gitea serves its admin pages under /-/admin/ and Forgejo under /admin/.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
@@ -222,8 +222,8 @@ func TestGiteaSearchExclusionsAreNarrow(t *testing.T) {
|
||||
}
|
||||
|
||||
// Under /admin/, where Forgejo serves its admin pages, only the hard
|
||||
// values are left out: the tenth change lets them through in q on every
|
||||
// path there, for Drupal.
|
||||
// values are left out: Drupal's fixes let them through in q on every path
|
||||
// there.
|
||||
for _, target := range []string{
|
||||
"/admin/packages/delete?q=",
|
||||
"/admin/packages?other=",
|
||||
|
||||
@@ -1284,18 +1284,18 @@ func TestBansWrittenOnceBansWriteDelayAfterABan(t *testing.T) {
|
||||
|
||||
// A second ban, made while the first waits to be written, puts the
|
||||
// write off no further, and is written with it.
|
||||
first, _ := params.Ledger.Ban(
|
||||
first, _ := params.Ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.9/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
time.Sleep(5 * time.Second)
|
||||
|
||||
second, _ := params.Ledger.Ban(
|
||||
second, _ := params.Ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.10/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
time.Sleep(5*time.Second - time.Nanosecond)
|
||||
@@ -1358,16 +1358,16 @@ func TestEditJustBeforeAScheduledWriteSurvivesIt(t *testing.T) {
|
||||
run(t, load(t, params).Run)
|
||||
|
||||
// A ban, and bans.json written with it.
|
||||
params.Ledger.Ban(netip.MustParsePrefix("203.0.113.9/32"),
|
||||
midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"),
|
||||
midnight(), bans.Notes{})
|
||||
time.Sleep(params.BansWriteDelay)
|
||||
synctest.Wait()
|
||||
|
||||
// A second ban is to be written BansWriteDelay later. Just before
|
||||
// then, an admin saves bans.json with the first ban lifted and
|
||||
// another added.
|
||||
params.Ledger.Ban(netip.MustParsePrefix("203.0.113.10/32"),
|
||||
midnight(), bans.Grounds{Cause: bans.CauseLimit})
|
||||
params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.10/32"),
|
||||
midnight(), bans.Notes{})
|
||||
time.Sleep(params.BansWriteDelay - time.Nanosecond)
|
||||
synctest.Wait()
|
||||
edit(t, dir, bansJSON, permanentBansJSON)
|
||||
@@ -1483,10 +1483,10 @@ func TestFailedWriteLeavesTheFileAsItWas(t *testing.T) {
|
||||
t.Fatalf("mkdir: %v", err)
|
||||
}
|
||||
|
||||
params.Ledger.Ban(
|
||||
params.Ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.9/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
params.Limiter.Count(
|
||||
netip.MustParsePrefix("203.0.113.9/32"),
|
||||
@@ -1556,10 +1556,10 @@ func TestWritesAreCountedInTheMetrics(t *testing.T) {
|
||||
t.Fatalf("mkdir: %v", err)
|
||||
}
|
||||
|
||||
params.Ledger.Ban(
|
||||
params.Ledger.BanForLimit(
|
||||
netip.MustParsePrefix("203.0.113.9/32"),
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
err = files.WriteAll()
|
||||
@@ -2135,11 +2135,7 @@ func TestBanLiftedByAnEditWhileRunning(t *testing.T) {
|
||||
|
||||
// The ban that liftedBansJSON lifts, before it is lifted.
|
||||
netblock := netip.MustParsePrefix(liftedClient + "/32")
|
||||
params.Ledger.Ban(
|
||||
netblock,
|
||||
midnight(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
params.Ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
||||
|
||||
_, banned, _ := params.Ledger.Find(netblock.Addr(), afterLifting())
|
||||
if !banned {
|
||||
@@ -2514,15 +2510,11 @@ func fill(params state.Params) {
|
||||
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||
|
||||
params.Ledger.Load([]bans.Ban{permanentBan()})
|
||||
params.Ledger.Ban(client, now, bans.Grounds{
|
||||
Cause: bans.CauseLimit,
|
||||
Notes: bans.Notes{ASN: asn, ASName: asName, Country: "DE", Limit: 1},
|
||||
params.Ledger.BanForLimit(client, now, bans.Notes{
|
||||
ASN: asn, ASName: asName, Country: "DE", Limit: 1,
|
||||
})
|
||||
params.Ledger.Ban(netip.MustParsePrefix("192.0.2.1/32"), now,
|
||||
bans.Grounds{
|
||||
Cause: bans.CauseAttack,
|
||||
Notes: bans.Notes{RuleID: "env-file", Target: "path"},
|
||||
})
|
||||
params.Ledger.BanForAttack(netip.MustParsePrefix("192.0.2.1/32"), now,
|
||||
bans.Notes{RuleID: "env-file", Target: "path"})
|
||||
|
||||
for _, c := range []string{"2001:db8::/64", "203.0.113.9/32", "192.0.2.1/32"} {
|
||||
params.Limiter.Count(netip.MustParsePrefix(c), now, whole)
|
||||
@@ -2708,11 +2700,7 @@ func wantLiftedBanKept(
|
||||
}
|
||||
|
||||
// Were the lifted ban counted, the next would last three hours.
|
||||
ban, _ := ledger.Ban(
|
||||
netblock,
|
||||
afterLifting(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
)
|
||||
ban, _ := ledger.BanForLimit(netblock, afterLifting(), bans.Notes{})
|
||||
if ban.Expires.Sub(ban.Start) != time.Hour {
|
||||
t.Errorf(
|
||||
"the next ban lasts %s, want 1h",
|
||||
|
||||
@@ -660,9 +660,7 @@ func TestCoreRuleSetExclusionsFileThatCannotBeUsedStopsTheStartWhileTheCoreRuleS
|
||||
|
||||
status := run(t.Context(), map[string]string{
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), ruleFilesDir: t.TempDir(),
|
||||
"SWW_WAF_CORE_RULE_SET_MODE": string(
|
||||
config.CoreRuleSetModeOff,
|
||||
),
|
||||
"SWW_WAF_CORE_RULE_SET_MODE": config.CoreRuleSetModeOff,
|
||||
"SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
||||
}, out)
|
||||
if status != 1 {
|
||||
|
||||
+4
-29
@@ -2,7 +2,6 @@ package sww
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net"
|
||||
@@ -176,35 +175,11 @@ func newRuleFiles(
|
||||
return ruleFiles, nil
|
||||
}
|
||||
|
||||
// newCoreRuleSet returns the Core Rule Set at SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL,
|
||||
// without the rules SWW_WAF_CORE_RULE_SET_DISABLED_RULES switches off, reading bodies
|
||||
// up to SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE, with the exclusions in
|
||||
// SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE and the fixes of the apps in SWW_WAF_APPS,
|
||||
// or nil while SWW_WAF_CORE_RULE_SET_MODE is off. An exclusions file that
|
||||
// coreruleset.New refuses is an error in every mode. While the mode is off, a
|
||||
// set file is loaded with the Core Rule Set only to check it.
|
||||
// newCoreRuleSet returns the Core Rule Set, as waf.NewCoreRuleSet does.
|
||||
func newCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
|
||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff &&
|
||||
cfg.CoreRuleSetExclusionsFile == "" {
|
||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
||||
}
|
||||
|
||||
crs, err := coreruleset.New(coreruleset.Params{
|
||||
ParanoiaLevel: cfg.CoreRuleSetParanoiaLevel,
|
||||
DisabledRules: cfg.CoreRuleSetDisabledRules,
|
||||
BodyInspectionSize: cfg.CoreRuleSetBodyInspectionSize,
|
||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
||||
Apps: cfg.Apps,
|
||||
})
|
||||
crs, err := waf.NewCoreRuleSet(cfg)
|
||||
if err != nil {
|
||||
return nil, startError{
|
||||
"cannot use the Core Rule Set",
|
||||
fmt.Errorf("SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err),
|
||||
}
|
||||
}
|
||||
|
||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff {
|
||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
||||
return nil, startError{"cannot use the Core Rule Set", err}
|
||||
}
|
||||
|
||||
return crs, nil
|
||||
@@ -217,7 +192,7 @@ func newLookupFile(
|
||||
lc fx.Lifecycle, cfg *config.WAF, processLog *slog.Logger,
|
||||
alertQueue *alerts.Queue,
|
||||
) (*lookup.File, error) {
|
||||
if cfg.LookupSource != config.LookupSourceFile {
|
||||
if cfg.LookupSource != "file" {
|
||||
return nil, nil //nolint:nilnil // no lookup database, and no error
|
||||
}
|
||||
|
||||
|
||||
@@ -54,7 +54,7 @@ func (abuseIPDB) check(
|
||||
"scored by AbuseIPDB at or over SWW_WAF_ABUSEIPDB_MIN_SCORE",
|
||||
)
|
||||
|
||||
if rq.h.config.DNSBLAbuseIPDBAction != config.ReputationActionDeny {
|
||||
if rq.h.config.DNSBLAbuseIPDBAction != deny {
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
@@ -37,10 +37,10 @@ func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
|
||||
"",
|
||||
map[string]string{metricsToken: token},
|
||||
)
|
||||
server.Ledger.Ban(
|
||||
server.Ledger.BanForLimit(
|
||||
netip.MustParsePrefix(otherClient+"/32"),
|
||||
clk.Now(),
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
before := server.Ledger.Snapshot()
|
||||
|
||||
|
||||
@@ -130,9 +130,11 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
|
||||
// A ban for a clear sign of attack, which a request under it would make
|
||||
// permanent.
|
||||
group := netip.MustParsePrefix(ipv6Group)
|
||||
attackBan, _ := server.Ledger.Ban(group, start, bans.Grounds{
|
||||
Cause: bans.CauseAttack, Notes: bans.Notes{RuleID: "probe"},
|
||||
})
|
||||
attackBan, _ := server.Ledger.BanForAttack(
|
||||
group,
|
||||
start,
|
||||
bans.Notes{RuleID: "probe"},
|
||||
)
|
||||
|
||||
// The third request breaks the limit, and so does the fourth, within the
|
||||
// cooldown, which raises nothing. The probe is a clear sign of attack.
|
||||
|
||||
@@ -686,10 +686,10 @@ func askDirectly(
|
||||
|
||||
// banClient bans client at now.
|
||||
func banClient(_ *testing.T, server *waf.Server, now time.Time) {
|
||||
server.Ledger.Ban(
|
||||
server.Ledger.BanForLimit(
|
||||
netip.MustParsePrefix(client+"/32"),
|
||||
now,
|
||||
bans.Grounds{Cause: bans.CauseLimit},
|
||||
bans.Notes{},
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
+57
-77
@@ -86,14 +86,13 @@ func (rq *request) countedBytes() int64 {
|
||||
response, request := rq.out.bytes, rq.requestBytes
|
||||
|
||||
switch rq.h.config.BytesCounted {
|
||||
case config.BytesCountedResponse:
|
||||
case "response":
|
||||
return response
|
||||
case config.BytesCountedRequest:
|
||||
case "request":
|
||||
return request
|
||||
case config.BytesCountedBoth:
|
||||
default: // both
|
||||
return response + request
|
||||
}
|
||||
|
||||
return response + request
|
||||
}
|
||||
|
||||
// banForLimit bans the client's netblock at now for a broken limit, the
|
||||
@@ -103,8 +102,9 @@ func (rq *request) countedBytes() int64 {
|
||||
// one whose bytes broke a byte limit, the refusal for one that broke the
|
||||
// refusal limit. The ban's notes give the client's limit percentage for a
|
||||
// rate limit or a byte limit; the refusal limit is not lowered. The ban sets
|
||||
// the client's counters back to zero. In observe mode, banClient says what
|
||||
// happens instead.
|
||||
// the client's counters back to zero. In observe mode it makes no ban and
|
||||
// sets nothing back, and raises the alert for the ban it would have made,
|
||||
// if that alert would be sent.
|
||||
func (rq *request) banForLimit(
|
||||
now time.Time,
|
||||
hit ratelimit.Hit,
|
||||
@@ -122,11 +122,23 @@ func (rq *request) banForLimit(
|
||||
rq.line.Offence = requestlog.OffenceLimit
|
||||
rq.h.metrics.LimitHit(hit)
|
||||
|
||||
netblock := banNetblock(rq.h.config, rq.client)
|
||||
if rq.h.config.Observe &&
|
||||
!rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
|
||||
return
|
||||
}
|
||||
|
||||
notes := bans.Notes{
|
||||
Kind: hit.Kind,
|
||||
Limit: hit.Limit,
|
||||
Window: hit.Window,
|
||||
Count: hit.Count,
|
||||
ASN: rq.line.ASN,
|
||||
ASName: rq.line.ASName,
|
||||
Country: rq.line.Country,
|
||||
Kind: hit.Kind,
|
||||
Limit: hit.Limit,
|
||||
Window: hit.Window,
|
||||
Count: hit.Count,
|
||||
Reputation: rq.reputation,
|
||||
Request: rq.noted(now, status),
|
||||
Requests: rq.netblockRequests(netblock),
|
||||
}
|
||||
|
||||
switch hit.Kind {
|
||||
@@ -136,54 +148,47 @@ func (rq *request) banForLimit(
|
||||
notes.LimitPercent, notes.LimitPercentSetting = rq.bytesLimitPercent.logged()
|
||||
}
|
||||
|
||||
rq.banClient(
|
||||
now,
|
||||
status,
|
||||
bans.Grounds{Cause: bans.CauseLimit, Notes: notes},
|
||||
)
|
||||
if rq.h.config.Observe {
|
||||
ban, wouldBan := rq.h.ledger.WouldBanForLimit(netblock, now, notes)
|
||||
if wouldBan {
|
||||
rq.alertBan(ban)
|
||||
}
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
ban, made := rq.h.ledger.BanForLimit(netblock, now, notes)
|
||||
rq.h.limiter.Reset(clientGroup(rq.h.config, rq.client))
|
||||
rq.line.BanExpires = banExpires(ban)
|
||||
|
||||
if made {
|
||||
rq.alertBan(ban)
|
||||
}
|
||||
}
|
||||
|
||||
// banForAttack bans the client's netblock at now for a clear sign of
|
||||
// attack, which notes name: the ban rule that matched, or the trap path
|
||||
// asked for. It returns the action for that ban. In observe mode, banClient
|
||||
// says what happens instead.
|
||||
// asked for. It fills in the rest of the notes. In observe mode it makes
|
||||
// no ban, and raises the alert for the ban it would have made, if that
|
||||
// alert would be sent. It returns the action for that ban.
|
||||
func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
|
||||
return rq.banClient(
|
||||
now,
|
||||
rq.h.config.BanResponse,
|
||||
bans.Grounds{Cause: bans.CauseAttack, Notes: notes},
|
||||
)
|
||||
}
|
||||
|
||||
// banClient bans the client's netblock at now on grounds, and returns the
|
||||
// action for that ban. It adds to the notes what the request tells of the
|
||||
// client, with status, what the client was sent. A ban for a broken limit
|
||||
// sets the client's counters back to zero.
|
||||
//
|
||||
// In observe mode it makes no ban and sets nothing back. It raises the
|
||||
// alert for the ban it would have made, if that alert would be sent.
|
||||
func (rq *request) banClient(
|
||||
now time.Time,
|
||||
status int,
|
||||
grounds bans.Grounds,
|
||||
) string {
|
||||
netblock := banNetblock(rq.h.config, rq.client)
|
||||
if rq.h.config.Observe &&
|
||||
!rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
|
||||
return banAction(
|
||||
rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseAttack),
|
||||
)
|
||||
}
|
||||
|
||||
notes.ASN = rq.line.ASN
|
||||
notes.ASName = rq.line.ASName
|
||||
notes.Country = rq.line.Country
|
||||
notes.Reputation = rq.reputation
|
||||
notes.Request = rq.noted(now, rq.h.config.BanResponse)
|
||||
notes.Requests = rq.netblockRequests(netblock)
|
||||
|
||||
if rq.h.config.Observe {
|
||||
if !rq.wouldAlertBan(netblock, now, grounds.Cause) {
|
||||
return banAction(
|
||||
rq.h.ledger.WouldBePermanent(netblock, now, grounds.Cause),
|
||||
)
|
||||
}
|
||||
|
||||
grounds.Notes = rq.withClientNotes(
|
||||
grounds.Notes,
|
||||
now,
|
||||
status,
|
||||
netblock,
|
||||
)
|
||||
|
||||
ban, wouldBan := rq.h.ledger.WouldBan(netblock, now, grounds)
|
||||
ban, wouldBan := rq.h.ledger.WouldBanForAttack(netblock, now, notes)
|
||||
if wouldBan {
|
||||
rq.alertBan(ban)
|
||||
}
|
||||
@@ -191,13 +196,7 @@ func (rq *request) banClient(
|
||||
return banAction(ban.Permanent())
|
||||
}
|
||||
|
||||
grounds.Notes = rq.withClientNotes(grounds.Notes, now, status, netblock)
|
||||
|
||||
ban, made := rq.h.ledger.Ban(netblock, now, grounds)
|
||||
if grounds.Cause == bans.CauseLimit {
|
||||
rq.h.limiter.Reset(clientGroup(rq.h.config, rq.client))
|
||||
}
|
||||
|
||||
ban, made := rq.h.ledger.BanForAttack(netblock, now, notes)
|
||||
rq.line.BanExpires = banExpires(ban)
|
||||
|
||||
if made {
|
||||
@@ -207,25 +206,6 @@ func (rq *request) banClient(
|
||||
return banAction(ban.Permanent())
|
||||
}
|
||||
|
||||
// withClientNotes is notes with what the request tells of the client
|
||||
// added: its AS, country and reputation, the request at now with status,
|
||||
// and how many requests netblock, the client's, has sent.
|
||||
func (rq *request) withClientNotes(
|
||||
notes bans.Notes,
|
||||
now time.Time,
|
||||
status int,
|
||||
netblock netip.Prefix,
|
||||
) bans.Notes {
|
||||
notes.ASN = rq.line.ASN
|
||||
notes.ASName = rq.line.ASName
|
||||
notes.Country = rq.line.Country
|
||||
notes.Reputation = rq.reputation
|
||||
notes.Request = rq.noted(now, status)
|
||||
notes.Requests = rq.netblockRequests(netblock)
|
||||
|
||||
return notes
|
||||
}
|
||||
|
||||
// wouldAlertBan reports whether the alert for a ban on netblock for cause
|
||||
// made at now would be sent. In observe mode the ban the request would
|
||||
// have made is worked out only then, at most once per
|
||||
|
||||
@@ -63,8 +63,7 @@ func (rq *request) limitPercentages() (percentage, percentage) {
|
||||
}
|
||||
|
||||
blocklisted := percentage{percent: whole}
|
||||
if rq.blocklisted &&
|
||||
cfg.BlocklistAction == config.ReputationActionLimit {
|
||||
if rq.blocklisted && cfg.BlocklistAction == "limit" {
|
||||
blocklisted = percentage{
|
||||
cfg.BlocklistLimitPercent,
|
||||
"SWW_WAF_BLOCKLIST_ACTION",
|
||||
@@ -73,7 +72,7 @@ func (rq *request) limitPercentages() (percentage, percentage) {
|
||||
|
||||
dnsblAbuseIPDBListed := percentage{percent: whole}
|
||||
if (rq.dnsblListed || rq.abuseIPDBHit) &&
|
||||
cfg.DNSBLAbuseIPDBAction == config.ReputationActionLimit {
|
||||
cfg.DNSBLAbuseIPDBAction == "limit" {
|
||||
dnsblAbuseIPDBListed = percentage{
|
||||
cfg.DNSBLAbuseIPDBLimitPercent,
|
||||
"SWW_WAF_DNSBL_ABUSEIPDB_ACTION",
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
@@ -42,6 +41,5 @@ func (blocklists) check(
|
||||
// client, given listedBy, the blocklists that list it: it does when one lists
|
||||
// it and SWW_WAF_BLOCKLIST_ACTION is deny.
|
||||
func (rq *request) refusedByBlocklists(listedBy []string) bool {
|
||||
return len(listedBy) > 0 && rq.h.config.BlocklistAction ==
|
||||
config.ReputationActionDeny
|
||||
return len(listedBy) > 0 && rq.h.config.BlocklistAction == deny
|
||||
}
|
||||
@@ -47,7 +47,7 @@ func (coreRuleSet) check(
|
||||
rq.line.CoreRuleSetScore = &result.Score
|
||||
|
||||
for _, id := range result.RuleIDs {
|
||||
rq.h.metrics.CoreRuleSetMatched(string(cfg.CoreRuleSetMode), id)
|
||||
rq.h.metrics.CoreRuleSetMatched(cfg.CoreRuleSetMode, id)
|
||||
}
|
||||
|
||||
threshold := cfg.CoreRuleSetScoreThreshold
|
||||
@@ -99,7 +99,7 @@ func (rq *request) alertCoreRuleSetMatch(result coreruleset.Result) {
|
||||
|
||||
switch {
|
||||
case rq.h.config.CoreRuleSetMode == config.CoreRuleSetModeDetect:
|
||||
detail["mode"] = string(config.CoreRuleSetModeDetect)
|
||||
detail["mode"] = config.CoreRuleSetModeDetect
|
||||
case rq.h.config.Observe:
|
||||
detail["mode"] = "observe"
|
||||
}
|
||||
|
||||
@@ -32,8 +32,7 @@ func (dnsbl) check(ctx context.Context, rq *request, _ time.Time) *verdict {
|
||||
rq.addTag(config.DNSBLTagPrefix + strings.ToLower(zone))
|
||||
}
|
||||
|
||||
if !rq.dnsblListed ||
|
||||
rq.h.config.DNSBLAbuseIPDBAction != config.ReputationActionDeny {
|
||||
if !rq.dnsblListed || rq.h.config.DNSBLAbuseIPDBAction != deny {
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
@@ -49,8 +49,7 @@ func (rq *request) startLookUp(
|
||||
) <-chan lookupResult {
|
||||
result := make(chan lookupResult, 1)
|
||||
|
||||
if rq.h.config.LookupSource == config.LookupSourceOff ||
|
||||
!canBePlaced(rq.client) {
|
||||
if rq.h.config.LookupSource == "off" || !canBePlaced(rq.client) {
|
||||
result <- lookupResult{}
|
||||
|
||||
return result
|
||||
@@ -58,7 +57,7 @@ func (rq *request) startLookUp(
|
||||
|
||||
client := clientGroup(rq.h.config, rq.client)
|
||||
|
||||
if rq.h.config.LookupSource == config.LookupSourceFile {
|
||||
if rq.h.config.LookupSource == "file" {
|
||||
result <- lookupResult{
|
||||
lookedUp: true, answer: rq.h.lookupFile.LookUp(client),
|
||||
}
|
||||
|
||||
+37
-2
@@ -6,6 +6,7 @@ package waf
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
@@ -82,8 +83,8 @@ type Params struct {
|
||||
// LookupFile is the lookup database they are looked up in while
|
||||
// SWW_WAF_LOOKUP_SOURCE is file, and nil otherwise.
|
||||
LookupFile *lookup.File
|
||||
// CoreRuleSet is what inspects requests while
|
||||
// SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
|
||||
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
|
||||
// while SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
|
||||
CoreRuleSet CoreRuleSetInspector
|
||||
// Now tells the time by which requests are counted for the rate
|
||||
// limits, bans are made and run out, and GeoJS's answers are kept,
|
||||
@@ -173,6 +174,40 @@ func newHandler(params Params) *handler {
|
||||
}
|
||||
}
|
||||
|
||||
// NewCoreRuleSet returns the Core Rule Set at SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL,
|
||||
// without the rules SWW_WAF_CORE_RULE_SET_DISABLED_RULES switches off, reading bodies
|
||||
// up to SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE, with the exclusions in
|
||||
// SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE and the fixes of the apps in SWW_WAF_APPS,
|
||||
// or nil while SWW_WAF_CORE_RULE_SET_MODE is off. An exclusions file that
|
||||
// coreruleset.New refuses is an error in every mode. While the mode is off, a
|
||||
// set file is loaded with the Core Rule Set only to check it.
|
||||
func NewCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
|
||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff &&
|
||||
cfg.CoreRuleSetExclusionsFile == "" {
|
||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
||||
}
|
||||
|
||||
coreRuleSet, err := coreruleset.New(coreruleset.Params{
|
||||
ParanoiaLevel: cfg.CoreRuleSetParanoiaLevel,
|
||||
DisabledRules: cfg.CoreRuleSetDisabledRules,
|
||||
BodyInspectionSize: cfg.CoreRuleSetBodyInspectionSize,
|
||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
||||
Apps: cfg.Apps,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(
|
||||
"SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE: %w",
|
||||
err,
|
||||
)
|
||||
}
|
||||
|
||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff {
|
||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
||||
}
|
||||
|
||||
return coreRuleSet, nil
|
||||
}
|
||||
|
||||
// handler is waf's handler. It holds what every request shares; what
|
||||
// belongs to one request is in a request.
|
||||
type handler struct {
|
||||
|
||||
+13
-34
@@ -20,7 +20,6 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
||||
@@ -505,20 +504,24 @@ func newWAFWith(
|
||||
}
|
||||
}
|
||||
|
||||
coreRuleSet, err := waf.NewCoreRuleSet(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("Core Rule Set: %v", err)
|
||||
}
|
||||
|
||||
params := waf.Params{
|
||||
Config: cfg,
|
||||
RequestLog: out,
|
||||
ProcessLog: processLog,
|
||||
LookupFile: lookupFile,
|
||||
Now: now,
|
||||
Rules: ruleFiles,
|
||||
Alerts: alertQueue,
|
||||
Config: cfg,
|
||||
RequestLog: out,
|
||||
ProcessLog: processLog,
|
||||
LookupFile: lookupFile,
|
||||
CoreRuleSet: coreRuleSet,
|
||||
Now: now,
|
||||
Rules: ruleFiles,
|
||||
Alerts: alertQueue,
|
||||
// The tests of the limit set it back, and hold the Core Rule Set
|
||||
// with a stand-in.
|
||||
BesideLookupTimeLimit: time.Hour,
|
||||
}
|
||||
|
||||
addCoreRuleSet(t, ¶ms)
|
||||
addCounters(¶ms)
|
||||
addOutsideSources(¶ms, geojsURL, abuseIPDBCheckURL)
|
||||
|
||||
@@ -527,30 +530,6 @@ func newWAFWith(
|
||||
return server, out, alertQueue
|
||||
}
|
||||
|
||||
// addCoreRuleSet adds to params the Core Rule Set, with the settings for it,
|
||||
// while SWW_WAF_CORE_RULE_SET_MODE is not off.
|
||||
func addCoreRuleSet(t *testing.T, params *waf.Params) {
|
||||
t.Helper()
|
||||
|
||||
cfg := params.Config
|
||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff {
|
||||
return
|
||||
}
|
||||
|
||||
coreRuleSet, err := coreruleset.New(coreruleset.Params{
|
||||
ParanoiaLevel: cfg.CoreRuleSetParanoiaLevel,
|
||||
DisabledRules: cfg.CoreRuleSetDisabledRules,
|
||||
BodyInspectionSize: cfg.CoreRuleSetBodyInspectionSize,
|
||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
||||
Apps: cfg.Apps,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Core Rule Set: %v", err)
|
||||
}
|
||||
|
||||
params.CoreRuleSet = coreRuleSet
|
||||
}
|
||||
|
||||
// addCounters adds to params the metrics, the table of clients, the ledger
|
||||
// and the anomaly counters, as sww builds them from params' settings. The
|
||||
// metrics read the ledger, the table of clients and the rules.
|
||||
|
||||
@@ -7,6 +7,10 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
)
|
||||
|
||||
// deny is the SWW_WAF_BLOCKLIST_ACTION and the SWW_WAF_DNSBL_ABUSEIPDB_ACTION that
|
||||
// refuses the requests of a client a source lists.
|
||||
const deny = "deny"
|
||||
|
||||
// noteListed notes each of sources, the URLs of the blocklists or the
|
||||
// DNSBL zones, their keys masked, that list the client, as noteHit does,
|
||||
// with reason.
|
||||
|
||||
@@ -421,7 +421,7 @@ func (rq *request) answerAtTheEnd() (geojs.Answer, bool) {
|
||||
return geojs.Answer{}, false
|
||||
}
|
||||
|
||||
if rq.h.config.LookupSource == config.LookupSourceFile {
|
||||
if rq.h.config.LookupSource == "file" {
|
||||
return rq.lookupAnswer, true
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user