Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2672fc3b34 |
@@ -30,7 +30,7 @@ unusual traffic that refuse nothing. So are the first three parts of the stage
|
||||
after that: the blocklists you name by URL, which it fetches and keeps, with a
|
||||
file of AS numbers' percentages fetched the same way, the DNS blocklists (DNSBL
|
||||
zones), which it asks about each client in the background, and AbuseIPDB, which
|
||||
it asks in the background about each client that has broken a limit.
|
||||
it asks in the background about each client that has committed an offence.
|
||||
`smallwebwaf` passes each request to the app and the app's answer back,
|
||||
unchanged, within its timeouts and size limits, works out each client's address,
|
||||
looks up its AS number and country unless you switch that off, bans a client
|
||||
@@ -225,15 +225,16 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
||||
make no ban. With `log`, nothing more is done. Whatever the action, the
|
||||
request's log line names the zones, and each raises an alert. A client a
|
||||
blocklist refuses is not checked.
|
||||
- Checks the client's own address with AbuseIPDB while `SWWAF_ABUSEIPDB_KEY` is
|
||||
set, after the DNSBL zones and before the rate limits (see "AbuseIPDB" below),
|
||||
by the scores it keeps. Only a client whose history counts an offence is
|
||||
checked, so far one that has broken a rate limit or a byte limit, and only in
|
||||
the background, so that no request waits for AbuseIPDB. A score at or over
|
||||
`SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and `SWWAF_REPUTATION_ACTION` does with
|
||||
its client what it does with one a DNSBL zone's verdict lists. The request's
|
||||
log line names AbuseIPDB, and it raises an alert. A client a blocklist or a
|
||||
DNSBL zone refuses is not checked.
|
||||
- Checks the client with AbuseIPDB while `SWWAF_ABUSEIPDB_KEY` is set, after the
|
||||
DNSBL zones and before the rate limits (see "AbuseIPDB" below), by the scores
|
||||
it keeps. Only a client whose history counts an offence is checked, so far one
|
||||
that has broken a rate limit or a byte limit, matched a ban rule, or had a
|
||||
request refused by a block rule, and only in the background, so that no
|
||||
request waits for AbuseIPDB. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is
|
||||
a hit, and `SWWAF_REPUTATION_ACTION` does with its client what it does with
|
||||
one a DNSBL zone's verdict lists. The request's log line names AbuseIPDB, and
|
||||
it raises an alert. A client a blocklist or a DNSBL zone refuses is not
|
||||
checked.
|
||||
- Checks the client's own address against the static lists, the three netblock
|
||||
settings below, before anything else, its lookup included. A client in
|
||||
`SWWAF_ALLOW_NETS` skips bans, the country lists, the blocklists, the DNSBL
|
||||
@@ -1077,9 +1078,9 @@ with times in UTC.
|
||||
zone gave it, `fetched`; and under `abuseipdb` (see "AbuseIPDB" below), the
|
||||
`day`, in UTC, whose checks it counts, left out before the first, the checks
|
||||
`spent` that day, and under `scores`, each score of AbuseIPDB still in use:
|
||||
the `client`'s address, its `score`, and when AbuseIPDB gave it, `fetched`. As
|
||||
the file is read, the lists the settings no longer name, and the verdicts of
|
||||
the zones they no longer name, are dropped.
|
||||
the `client`, its IPv4 address as a /32 or its IPv6 /64, its `score`, and when
|
||||
AbuseIPDB gave it, `fetched`. As the file is read, the lists the settings no
|
||||
longer name, and the verdicts of the zones they no longer name, are dropped.
|
||||
- `alerts.json`: the state of the alerts (see "Alerts" above), indented to be
|
||||
read: under `cooldowns`, for each event and netblock, with the `source` too
|
||||
for a `reputation_hit`, or event and `file` or `source`, or for an `anomaly`,
|
||||
@@ -1820,15 +1821,19 @@ While `SWWAF_ABUSEIPDB_KEY` holds the key of an AbuseIPDB account, `smallwebwaf`
|
||||
asks AbuseIPDB's check endpoint, `https://api.abuseipdb.com/api/v2/check`, for
|
||||
the abuse confidence score of a client's own address, from 0 to 100. It is unset
|
||||
by default, for the reason no blocklist is named, and since AbuseIPDB needs an
|
||||
account. An IPv6 client is checked by its own address, not by its /64.
|
||||
account. An IPv6 client, a /64, is checked by the address of the request that
|
||||
has it checked, and its score is used for the whole /64, whichever of its
|
||||
addresses sends, so that one client costs at most one check every
|
||||
`SWWAF_REPUTATION_CACHE_TTL`.
|
||||
|
||||
Only a client whose history counts an offence is checked, so that the checks are
|
||||
spent on suspects: so far, one that has broken a rate limit or a byte limit. A
|
||||
client dropped from the table of clients loses its history, and with it its
|
||||
offences. A client is checked in the background, at its first request after its
|
||||
offence that reaches the check: no request waits, a request refused under its
|
||||
ban is not checked, and the request that has it checked, and any other from it
|
||||
before the answer comes, goes on as from a client without a score.
|
||||
spent on suspects: so far, one that has broken a rate limit or a byte limit,
|
||||
matched a ban rule, or had a request refused by a block rule. A client dropped
|
||||
from the table of clients loses its history, and with it its offences. A client
|
||||
is checked in the background, at its first request after its offence that
|
||||
reaches the check: no request waits, a request refused under its ban is not
|
||||
checked, and the request that has it checked, and any other from it before the
|
||||
answer comes, goes on as from a client without a score.
|
||||
|
||||
A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE`, 75 by default, is a hit, and
|
||||
`SWWAF_REPUTATION_ACTION` says what is done with its client, as for a DNSBL
|
||||
@@ -1901,8 +1906,8 @@ given as files" above).
|
||||
file gives an AS number; asks the DNSBL zones about clients in the background,
|
||||
through the standard library's resolver, keeps their verdicts, and tells which
|
||||
zones' verdicts list an address; and checks clients with AbuseIPDB in the
|
||||
background, keeps their scores and the checks spent today, and tells whether
|
||||
an address's score is a hit.
|
||||
background, keeps their scores and the checks spent today, and tells whether a
|
||||
client's score is a hit.
|
||||
- `internal/ratelimit`: the table of clients: counts each client's requests and
|
||||
bytes, tells when they take it over a rate limit or a byte limit, and keeps
|
||||
each client's history.
|
||||
|
||||
@@ -43,18 +43,20 @@ func (rq *request) dnsblDenied(ctx context.Context) bool {
|
||||
// its score of the client is a hit, and reports whether
|
||||
// SWWAF_REPUTATION_ACTION, being deny, refuses the request, as dnsblDenied
|
||||
// does for a zone. While SWWAF_ABUSEIPDB_KEY is unset it does nothing. A
|
||||
// client without a score is checked in the background if its history
|
||||
// counts an offence, and the request does not wait for the answer. ctx is
|
||||
// the request's own context.
|
||||
// client without a score is checked in the background, by the request's
|
||||
// address, if its history counts an offence, and the request does not
|
||||
// wait for the answer. The score is then used for each address of the
|
||||
// client. ctx is the request's own context.
|
||||
func (rq *request) abuseIPDBDenied(ctx context.Context) bool {
|
||||
if rq.h.config.AbuseIPDBKey == "" {
|
||||
return false
|
||||
}
|
||||
|
||||
held, _ := rq.h.limiter.Client(clientGroup(rq.client))
|
||||
client := clientGroup(rq.client)
|
||||
held, _ := rq.h.limiter.Client(client)
|
||||
offender := held.History.Offences != ratelimit.Offences{}
|
||||
|
||||
score, hit := rq.h.abuseIPDB.Hit(ctx, rq.client, offender)
|
||||
score, hit := rq.h.abuseIPDB.Hit(ctx, client, rq.client, offender)
|
||||
if !hit {
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -1,17 +1,20 @@
|
||||
package proxy_test
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"maps"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
@@ -661,6 +664,87 @@ func TestOnlyAClientThatHasCommittedAnOffenceIsCheckedWithAbuseIPDB(t *testing.T
|
||||
wantAbuseIPDBChecks(t, server, 1)
|
||||
}
|
||||
|
||||
func TestIPv6ClientCostsOneAbuseIPDBCheckWhicheverOfItsAddressesSends(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
forward := requestlog.ActionForward
|
||||
|
||||
// 15 addresses of 2001:db8:1:2::/64, one client, each in a part of it
|
||||
// of its own.
|
||||
var addresses []string
|
||||
for i := 1; i < 16; i++ {
|
||||
addresses = append(addresses, fmt.Sprintf("2001:db8:1:2:%x::9", i<<12))
|
||||
}
|
||||
|
||||
s, clk, server := startWithClock(t, "", map[string]string{
|
||||
abuseIPDBKey: accountKey, reputationAction: actionLog,
|
||||
rateLimitPerMinute: strconv.Itoa(len(addresses)),
|
||||
})
|
||||
|
||||
// The client breaks the rate limit from its first address, which bans
|
||||
// it for an hour.
|
||||
for range addresses {
|
||||
s.get(addresses[0], http.StatusOK, forward)
|
||||
}
|
||||
|
||||
s.get(addresses[0], http.StatusForbidden, requestlog.ActionRateLimited)
|
||||
clk.advance(time.Hour)
|
||||
|
||||
// Once the ban has ended, which set its counters back to zero, a
|
||||
// request from each of its addresses has it checked once.
|
||||
for _, address := range addresses {
|
||||
s.get(address, http.StatusOK, forward)
|
||||
}
|
||||
|
||||
wantAbuseIPDBChecks(t, server, 1)
|
||||
}
|
||||
|
||||
func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
// path is what the client asks for, status and action what that
|
||||
// request is answered and logged with, and want the offences its
|
||||
// history then counts.
|
||||
path string
|
||||
status int
|
||||
action string
|
||||
want ratelimit.Offences
|
||||
}{
|
||||
{
|
||||
"a block rule", "/blocked", http.StatusForbidden, requestlog.ActionRuleBlocked,
|
||||
ratelimit.Offences{RuleBlocked: 1},
|
||||
},
|
||||
{
|
||||
"a ban rule", "/.env", http.StatusForbidden, requestlog.ActionBanned,
|
||||
ratelimit.Offences{Attack: 1},
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, server := startWithClock(t, "", map[string]string{
|
||||
abuseIPDBKey: accountKey, reputationAction: actionLog,
|
||||
rulesDir: writeRules(t, testRules), attackBanDuration: "1h",
|
||||
})
|
||||
|
||||
s.request(client, tc.path, tc.status, tc.action)
|
||||
wantAbuseIPDBChecks(t, server, 0)
|
||||
|
||||
if got := historyOf(t, server, client).Offences; got != tc.want {
|
||||
t.Errorf("history counts the offences %+v, want %+v", got, tc.want)
|
||||
}
|
||||
|
||||
// Its next request, once a ban rule's ban has ended, has it
|
||||
// checked.
|
||||
clk.advance(time.Hour)
|
||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||
wantAbuseIPDBChecks(t, server, 1)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestEachReputationActionForAClientAbuseIPDBScoresAtOrOverTheMinimum(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -820,12 +904,14 @@ func loadVerdicts(server *proxy.Server, listedBy map[string][]string) {
|
||||
}
|
||||
|
||||
// loadScores puts into server's AbuseIPDB the score scores gives each
|
||||
// client, fetched at verdictsFetched, as reputation.json would at start.
|
||||
// client, an IPv4 address, fetched at verdictsFetched, as reputation.json
|
||||
// would at start.
|
||||
func loadScores(server *proxy.Server, scores map[string]int64) {
|
||||
kept := make([]reputation.Score, 0, len(scores))
|
||||
for client, score := range scores {
|
||||
kept = append(kept, reputation.Score{
|
||||
Client: netip.MustParseAddr(client), Score: score, Fetched: verdictsFetched(),
|
||||
Client: netip.MustParsePrefix(client + "/32"), Score: score,
|
||||
Fetched: verdictsFetched(),
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -63,6 +63,10 @@ type request struct {
|
||||
// limits and for the byte limits.
|
||||
counted bool
|
||||
limitPercent, bytesPercent percentage
|
||||
// attack is true for a request that matched a ban rule, and
|
||||
// ruleBlocked for one a block rule refused, each an offence its
|
||||
// client's history counts.
|
||||
attack, ruleBlocked bool
|
||||
// blocklisted is true once a blocklist is found to list the client,
|
||||
// dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once
|
||||
// AbuseIPDB's score of it is a hit.
|
||||
@@ -542,6 +546,8 @@ func (rq *request) addToHistory() {
|
||||
RequestBytes: rq.requestBytes(),
|
||||
ResponseBytes: rq.out.bytes,
|
||||
BrokeLimit: rq.line.Offence == requestlog.OffenceLimit,
|
||||
Attack: rq.attack,
|
||||
RuleBlocked: rq.ruleBlocked,
|
||||
})
|
||||
|
||||
answer, found := rq.answerAtTheEnd()
|
||||
|
||||
@@ -12,7 +12,8 @@ import (
|
||||
// action of the rule that refuses it, ActionRuleBlocked for a block rule
|
||||
// and ActionBanned for a ban rule, or "" when none does. A ban rule bans
|
||||
// the client's netblock for a clear sign of attack, or in observe mode
|
||||
// raises the alert for the ban it would have made.
|
||||
// raises the alert for the ban it would have made. Either rule's match
|
||||
// is noted as an offence, for the client's history.
|
||||
func (rq *request) checkRules(now time.Time) string {
|
||||
matched := rq.h.rules.Match(rq.in)
|
||||
|
||||
@@ -28,8 +29,11 @@ func (rq *request) checkRules(now time.Time) string {
|
||||
// Only the last rule matched can refuse the request.
|
||||
switch last := matched[len(matched)-1]; last.Action {
|
||||
case rules.ActionBlock:
|
||||
rq.ruleBlocked = true
|
||||
|
||||
return requestlog.ActionRuleBlocked
|
||||
case rules.ActionBan:
|
||||
rq.attack = true
|
||||
rq.banForAttack(now, last)
|
||||
|
||||
return requestlog.ActionBanned
|
||||
|
||||
@@ -118,8 +118,12 @@ type Responses struct {
|
||||
|
||||
// Offences are a client's offences, by kind.
|
||||
type Offences struct {
|
||||
// Limit is its requests that broke a rate limit or a byte limit.
|
||||
Limit int64 `json:"limit"`
|
||||
// Limit is its requests that broke a rate limit or a byte limit,
|
||||
// Attack those that matched a ban rule, a clear sign of attack, and
|
||||
// RuleBlocked those a block rule refused.
|
||||
Limit int64 `json:"limit"`
|
||||
Attack int64 `json:"attack"`
|
||||
RuleBlocked int64 `json:"rule_blocked"`
|
||||
}
|
||||
|
||||
// Request is what a client's history keeps of one of its requests.
|
||||
@@ -137,8 +141,11 @@ type Request struct {
|
||||
RequestBytes int64
|
||||
ResponseBytes int64
|
||||
// BrokeLimit is true for a request that broke a rate limit or a byte
|
||||
// limit.
|
||||
BrokeLimit bool
|
||||
// limit, Attack for one that matched a ban rule, and RuleBlocked for
|
||||
// one a block rule refused.
|
||||
BrokeLimit bool
|
||||
Attack bool
|
||||
RuleBlocked bool
|
||||
}
|
||||
|
||||
// New returns a Limiter for limits, with no client counted yet.
|
||||
@@ -258,6 +265,14 @@ func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
|
||||
if r.BrokeLimit {
|
||||
h.Offences.Limit++
|
||||
}
|
||||
|
||||
if r.Attack {
|
||||
h.Offences.Attack++
|
||||
}
|
||||
|
||||
if r.RuleBlocked {
|
||||
h.Offences.RuleBlocked++
|
||||
}
|
||||
}
|
||||
|
||||
// AddLookup gives client's history its AS number, AS name and country, as
|
||||
|
||||
@@ -38,12 +38,12 @@ var (
|
||||
)
|
||||
|
||||
// Score is what AbuseIPDB said about a client, as reputation.json holds
|
||||
// it: the client's address, its abuse confidence score, from 0 to 100,
|
||||
// and when AbuseIPDB answered.
|
||||
// it: the client, an IPv4 address or an IPv6 group, its abuse confidence
|
||||
// score, from 0 to 100, and when AbuseIPDB answered.
|
||||
type Score struct {
|
||||
Client netip.Addr `json:"client"`
|
||||
Score int64 `json:"score"`
|
||||
Fetched time.Time `json:"fetched"`
|
||||
Client netip.Prefix `json:"client"`
|
||||
Score int64 `json:"score"`
|
||||
Fetched time.Time `json:"fetched"`
|
||||
}
|
||||
|
||||
// Checks are what reputation.json keeps of the checks of clients with
|
||||
@@ -89,9 +89,9 @@ type AbuseIPDB struct {
|
||||
mu sync.Mutex
|
||||
// scores are by client. Each is added as it is fetched and never moved
|
||||
// up, so that the one fetched longest ago is the first dropped.
|
||||
scores *simplelru.LRU[netip.Addr, Score]
|
||||
scores *simplelru.LRU[netip.Prefix, Score]
|
||||
// checking are the clients whose check is under way.
|
||||
checking map[netip.Addr]bool
|
||||
checking map[netip.Prefix]bool
|
||||
// day is the day, in UTC, of the checks spent counts.
|
||||
day time.Time
|
||||
spent int
|
||||
@@ -105,7 +105,7 @@ type AbuseIPDB struct {
|
||||
|
||||
// NewAbuseIPDB returns an AbuseIPDB with no score yet, and no check spent.
|
||||
func NewAbuseIPDB(params AbuseIPDBParams) *AbuseIPDB {
|
||||
scores, err := simplelru.NewLRU[netip.Addr, Score](maxVerdicts, nil)
|
||||
scores, err := simplelru.NewLRU[netip.Prefix, Score](maxVerdicts, nil)
|
||||
if err != nil {
|
||||
panic(err) // NewLRU fails only for a size below one
|
||||
}
|
||||
@@ -114,27 +114,29 @@ func NewAbuseIPDB(params AbuseIPDBParams) *AbuseIPDB {
|
||||
params: params,
|
||||
httpClient: &http.Client{},
|
||||
scores: scores,
|
||||
checking: map[netip.Addr]bool{},
|
||||
checking: map[netip.Prefix]bool{},
|
||||
}
|
||||
}
|
||||
|
||||
// Hit returns AbuseIPDB's score of addr, a client's address, and whether
|
||||
// it is a hit: MinScore or more. A score is used until CacheTTL has passed
|
||||
// since it was fetched. A client without one is checked in the
|
||||
// background if offender, if it has committed an offence, unless its
|
||||
// check is under way, a check failed less than failureDelay ago, or the
|
||||
// day's checks have used up DailyBudget; Hit never waits for a check. The
|
||||
// check that uses the budget up is logged and raised as a source_failure
|
||||
// alert. ctx is the context of the client's request, and a check goes on
|
||||
// after the request ends.
|
||||
// Hit returns AbuseIPDB's score of client, an IPv4 address or an IPv6
|
||||
// group, and whether it is a hit: MinScore or more. A score is used until
|
||||
// CacheTTL has passed since it was fetched, whichever of the client's
|
||||
// addresses its request comes from. A client without one is checked in
|
||||
// the background, by addr, the address its request came from, if
|
||||
// offender, if it has committed an offence, unless its check is under
|
||||
// way, a check failed less than failureDelay ago, or the day's checks
|
||||
// have used up DailyBudget; Hit never waits for a check. The check that
|
||||
// uses the budget up is logged and raised as a source_failure alert. ctx
|
||||
// is the context of the client's request, and a check goes on after the
|
||||
// request ends.
|
||||
func (a *AbuseIPDB) Hit(
|
||||
ctx context.Context, addr netip.Addr, offender bool,
|
||||
ctx context.Context, client netip.Prefix, addr netip.Addr, offender bool,
|
||||
) (int64, bool) {
|
||||
a.mu.Lock()
|
||||
|
||||
now := a.params.Now()
|
||||
|
||||
kept, found := a.scores.Peek(addr)
|
||||
kept, found := a.scores.Peek(client)
|
||||
if found && now.Sub(kept.Fetched) < a.params.CacheTTL {
|
||||
a.mu.Unlock()
|
||||
|
||||
@@ -145,14 +147,14 @@ func (a *AbuseIPDB) Hit(
|
||||
a.day, a.spent = today, 0
|
||||
}
|
||||
|
||||
check := offender && !a.checking[addr] && !now.Before(a.retryAt) &&
|
||||
check := offender && !a.checking[client] && !now.Before(a.retryAt) &&
|
||||
a.spent < a.params.DailyBudget
|
||||
if check {
|
||||
a.checking[addr] = true
|
||||
a.checking[client] = true
|
||||
a.checks++
|
||||
a.spent++
|
||||
|
||||
go a.check(context.WithoutCancel(ctx), addr)
|
||||
go a.check(context.WithoutCancel(ctx), client, addr)
|
||||
}
|
||||
|
||||
usedUp := check && a.spent == a.params.DailyBudget
|
||||
@@ -239,20 +241,20 @@ func (a *AbuseIPDB) Load(checks Checks) {
|
||||
}
|
||||
}
|
||||
|
||||
// check checks addr with AbuseIPDB, keeps the score, and notes the check
|
||||
// as no longer under way. A check that fails gives no score: it is
|
||||
// counted, logged and raised as a source_failure alert, and no client is
|
||||
// checked for failureDelay.
|
||||
func (a *AbuseIPDB) check(ctx context.Context, addr netip.Addr) {
|
||||
// check checks client with AbuseIPDB by addr, one of its addresses, keeps
|
||||
// the score as client's, and notes the check as no longer under way. A
|
||||
// check that fails gives no score: it is counted, logged and raised as a
|
||||
// source_failure alert, and no client is checked for failureDelay.
|
||||
func (a *AbuseIPDB) check(ctx context.Context, client netip.Prefix, addr netip.Addr) {
|
||||
score, err := a.ask(ctx, addr)
|
||||
now := a.params.Now()
|
||||
|
||||
a.mu.Lock()
|
||||
|
||||
delete(a.checking, addr)
|
||||
delete(a.checking, client)
|
||||
|
||||
if err == nil {
|
||||
a.scores.Add(addr, Score{Client: addr, Score: score, Fetched: now})
|
||||
a.scores.Add(client, Score{Client: client, Score: score, Fetched: now})
|
||||
} else {
|
||||
a.failures++
|
||||
a.retryAt = now.Add(failureDelay)
|
||||
|
||||
@@ -59,6 +59,38 @@ func TestOnlyAnOffenderWithoutAScoreIsChecked(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
// 15 addresses of 2001:db8:1:2::/64, one client, each in a part of
|
||||
// it of its own.
|
||||
var addresses []string
|
||||
for i := 1; i < 16; i++ {
|
||||
addresses = append(addresses, fmt.Sprintf("2001:db8:1:2:%x::9", i<<12))
|
||||
}
|
||||
|
||||
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{addresses[0]: 100}}
|
||||
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
|
||||
|
||||
// A request from each has the client checked once, by the first.
|
||||
for _, address := range addresses {
|
||||
hitFrom(t, checker, address, true)
|
||||
}
|
||||
|
||||
synctest.Wait()
|
||||
wantChecked(t, abuseIPDB, addresses[0])
|
||||
|
||||
// Its score is the whole client's.
|
||||
for _, address := range addresses {
|
||||
wantScore(t, checker, address, true, 100, true)
|
||||
}
|
||||
|
||||
synctest.Wait()
|
||||
wantChecked(t, abuseIPDB, addresses[0])
|
||||
})
|
||||
}
|
||||
|
||||
func TestScoreAtOrOverTheMinimumIsAHit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -69,7 +101,7 @@ func TestScoreAtOrOverTheMinimumIsAHit(t *testing.T) {
|
||||
checker := newAbuseIPDB(&abuseIPDBStandIn{scores: scores}, p)
|
||||
|
||||
for client := range scores {
|
||||
checker.Hit(t.Context(), netip.MustParseAddr(client), true)
|
||||
hitFrom(t, checker, client, true)
|
||||
}
|
||||
|
||||
synctest.Wait()
|
||||
@@ -87,7 +119,7 @@ func TestScoreUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
|
||||
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
|
||||
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
|
||||
|
||||
checker.Hit(t.Context(), netip.MustParseAddr(suspect), true)
|
||||
hitFrom(t, checker, suspect, true)
|
||||
synctest.Wait()
|
||||
|
||||
// AbuseIPDB gives another score from now on, but the one kept is
|
||||
@@ -130,7 +162,7 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
|
||||
|
||||
clients := []string{suspect, "192.0.2.2", "192.0.2.3", unchecked}
|
||||
for _, client := range clients {
|
||||
checker.Hit(t.Context(), netip.MustParseAddr(client), true)
|
||||
hitFrom(t, checker, client, true)
|
||||
}
|
||||
|
||||
synctest.Wait()
|
||||
@@ -352,10 +384,10 @@ func TestMetricsCountTheChecksTheFailuresAndTheBudgetLeft(t *testing.T) {
|
||||
m.AddAbuseIPDB(checker)
|
||||
|
||||
// One check that AbuseIPDB answers, and one that fails.
|
||||
checker.Hit(t.Context(), netip.MustParseAddr(suspect), true)
|
||||
hitFrom(t, checker, suspect, true)
|
||||
synctest.Wait()
|
||||
abuseIPDB.answerWith(http.StatusInternalServerError, "")
|
||||
checker.Hit(t.Context(), netip.MustParseAddr(other), true)
|
||||
hitFrom(t, checker, other, true)
|
||||
synctest.Wait()
|
||||
|
||||
scraped := scrapeMetrics(t, m)
|
||||
@@ -382,13 +414,15 @@ func TestScoreFetchedATTLAgoIsNeitherUsedNorKept(t *testing.T) {
|
||||
p.Now = func() time.Time { return now }
|
||||
checker := reputation.NewAbuseIPDB(p)
|
||||
|
||||
// The last score still in use, and one fetched a TTL ago.
|
||||
// The last score still in use, and one, of other's /64, fetched a TTL
|
||||
// ago.
|
||||
inUse := reputation.Score{
|
||||
Client: netip.MustParseAddr(suspect), Score: 100,
|
||||
Client: netip.MustParsePrefix(suspect + "/32"), Score: 100,
|
||||
Fetched: now.Add(-cacheTTL + time.Nanosecond),
|
||||
}
|
||||
stale := reputation.Score{
|
||||
Client: netip.MustParseAddr(other), Score: 100, Fetched: now.Add(-cacheTTL),
|
||||
Client: netip.MustParsePrefix("2001:db8::/64"), Score: 100,
|
||||
Fetched: now.Add(-cacheTTL),
|
||||
}
|
||||
|
||||
checker.Load(reputation.Checks{Scores: []reputation.Score{stale, inUse}})
|
||||
@@ -417,12 +451,13 @@ func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T)
|
||||
|
||||
scores := make([]reputation.Score, 0, count)
|
||||
|
||||
client := netip.MustParseAddr("198.18.0.0")
|
||||
addr := netip.MustParseAddr("198.18.0.0")
|
||||
for i := range count {
|
||||
scores = append(scores, reputation.Score{
|
||||
Client: client, Fetched: now.Add(-time.Duration(i) * time.Millisecond),
|
||||
Client: netip.PrefixFrom(addr, 32),
|
||||
Fetched: now.Add(-time.Duration(i) * time.Millisecond),
|
||||
})
|
||||
client = client.Next()
|
||||
addr = addr.Next()
|
||||
}
|
||||
|
||||
checker.Load(reputation.Checks{Scores: scores})
|
||||
@@ -537,13 +572,31 @@ func wantScore(
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
gotScore, gotHit := checker.Hit(t.Context(), netip.MustParseAddr(client), offender)
|
||||
gotScore, gotHit := hitFrom(t, checker, client, offender)
|
||||
if gotScore != score || gotHit != hit {
|
||||
t.Errorf("%s has the score %d, a hit %t, want %d, %t", client, gotScore, gotHit,
|
||||
score, hit)
|
||||
}
|
||||
}
|
||||
|
||||
// hitFrom is checker's Hit for a request from address, offender or not.
|
||||
// Its client is address for an IPv4 address, and its /64 for an IPv6 one,
|
||||
// as smallwebwaf counts clients.
|
||||
func hitFrom(
|
||||
t *testing.T, checker *reputation.AbuseIPDB, address string, offender bool,
|
||||
) (int64, bool) {
|
||||
t.Helper()
|
||||
|
||||
addr := netip.MustParseAddr(address)
|
||||
|
||||
client := netip.PrefixFrom(addr, addr.BitLen())
|
||||
if addr.Is6() {
|
||||
client = netip.PrefixFrom(addr, 64).Masked()
|
||||
}
|
||||
|
||||
return checker.Hit(t.Context(), client, addr, offender)
|
||||
}
|
||||
|
||||
// wantChecked checks the clients the stand-in was asked about, in any
|
||||
// order.
|
||||
func wantChecked(t *testing.T, abuseIPDB *abuseIPDBStandIn, want ...string) {
|
||||
|
||||
@@ -252,12 +252,12 @@ const filledReputationJSON = `{
|
||||
"spent": 3,
|
||||
"scores": [
|
||||
{
|
||||
"client": "203.0.113.9",
|
||||
"client": "203.0.113.9/32",
|
||||
"score": 100,
|
||||
"fetched": "2026-10-05T23:00:00Z"
|
||||
},
|
||||
{
|
||||
"client": "2001:db8::1",
|
||||
"client": "2001:db8::/64",
|
||||
"score": 0,
|
||||
"fetched": "2026-10-05T22:00:00Z"
|
||||
}
|
||||
@@ -710,7 +710,7 @@ func TestReputationJSONScoreWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
|
||||
// score and fetched make a score.
|
||||
const (
|
||||
scores = `{"version": 1, "abuseipdb": {"scores": [`
|
||||
client = `"client": "198.51.100.7", `
|
||||
client = `"client": "198.51.100.7/32", `
|
||||
score = `"score": 0, `
|
||||
fetched = `"fetched": "2026-10-06T00:00:00Z"`
|
||||
ends = `}]}}`
|
||||
@@ -1250,7 +1250,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
|
||||
`"lines": ["198.51.100.7"]}], "verdicts": [{"zone": "`+dnsblZone+`", `+
|
||||
`"client": "198.51.100.7", "listed": true, "fetched": "2026-10-06T00:00:00Z"}], `+
|
||||
`"abuseipdb": {"day": "2026-10-06T00:00:00Z", "spent": 9, "scores": [`+
|
||||
`{"client": "198.51.100.7", "score": 80, "fetched": "2026-10-06T00:00:00Z"}]}}`)
|
||||
`{"client": "198.51.100.7/32", "score": 80, "fetched": "2026-10-06T00:00:00Z"}]}}`)
|
||||
wantTakenIn(t, lines, dir, reputationJSON)
|
||||
|
||||
listedBy := params.Lists.ListedBy(client.Addr())
|
||||
@@ -1265,7 +1265,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
|
||||
|
||||
checks := reputation.Checks{
|
||||
Day: midnight(), Spent: 9,
|
||||
Scores: []reputation.Score{{Client: client.Addr(), Score: 80, Fetched: midnight()}},
|
||||
Scores: []reputation.Score{{Client: client, Score: 80, Fetched: midnight()}},
|
||||
}
|
||||
if got := params.AbuseIPDB.Snapshot(); !reflect.DeepEqual(got, checks) {
|
||||
t.Errorf("%s taken in as\n%+v\nwant\n%+v", reputationJSON, got, checks)
|
||||
@@ -1778,8 +1778,8 @@ func fill(params state.Params) {
|
||||
{Zone: dnsblZone, Client: client.Addr(), Listed: true, Fetched: now.Add(-time.Hour)},
|
||||
})
|
||||
params.AbuseIPDB.Load(reputation.Checks{Day: now, Spent: 3, Scores: []reputation.Score{
|
||||
{Client: netip.MustParseAddr("2001:db8::1"), Fetched: now.Add(-2 * time.Hour)},
|
||||
{Client: client.Addr(), Score: 100, Fetched: now.Add(-time.Hour)},
|
||||
{Client: netip.MustParsePrefix("2001:db8::/64"), Fetched: now.Add(-2 * time.Hour)},
|
||||
{Client: client, Score: 100, Fetched: now.Add(-time.Hour)},
|
||||
}})
|
||||
|
||||
// An alert waiting, a repeat of it the cooldown holds back, another
|
||||
|
||||
Reference in New Issue
Block a user