Compare commits

..
Author SHA1 Message Date
sneak 0e8472fc96 coreruleset: time worst-case inspections against reference work (closes #400)
check / check (push) Failing after 3m52s
The worst-case inspection test compared each inspection with a fixed 1
second, so a busy host failed it at random. It now times, on the same
thread and in processor time, a fixed regular-expression job just
before and after each inspection. An inspection fails when it takes
more than 13 times that job, about 1 second on an idle host.

Model: opus-5-5
2026-11-16 08:39:01 +00:00
33 changed files with 542 additions and 1521 deletions

No files matched your search

+7 -13
View File
@@ -329,7 +329,7 @@ Every setting the code reads, with its default:
| `SWW_WAF_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
| `SWW_WAF_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
| `SWW_WAF_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
| `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` | `800` | How long a request waits for its client's first GeoJS answer when a setting needs it, in milliseconds. The request to GeoJS goes on, and a later answer is kept. |
| `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` | `800` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take, in milliseconds. |
| `SWW_WAF_ADD_LOOKUP_HEADERS` | `true`, or `false` while the lookup source is `off` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
| `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. |
| `SWW_WAF_TAG_LISTS` | empty | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`. |
@@ -599,11 +599,6 @@ request reaches the app.
16. A path under `/_sww/` is answered `404`, so that the app never sees that
prefix.
Steps 12 to 14 only read the request. While the lookup waits for GeoJS, they run
at the same time as steps 6 to 11. The first of the two sides to refuse the
request ends the checks, and `waf` answers at once. Steps 12 to 14 have 500 ms
together.
`rproxy` takes the rest, with `waf` counting at the end:
17. The request is passed to the app, streaming.
@@ -616,10 +611,9 @@ A refused visitor is answered with `SWW_WAF_BAN_RESPONSE`. A `block` rule or a
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
false positives; such refusals count toward `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`
instead. A request over the header limits of step 3 is answered `414` for a
request line over 8 KiB and `431` otherwise. It bans nobody either. Nor does a
request that steps 12 to 14 have not finished within 500 ms, which is answered
`503`. `rproxy` itself answers `405` to a method it does not pass on, and the
status for a size or time limit, such as `413` for a body too large.
request line over 8 KiB and `431` otherwise. It bans nobody either. `rproxy`
itself answers `405` to a method it does not pass on, and the status for a size
or time limit, such as `413` for a body too large.
### Clients and bans
@@ -755,9 +749,9 @@ stops the running process (see [`docs/state-files.md`](docs/state-files.md)).
- `internal/waf`: `waf`'s server, which runs the checks and answers its own
endpoints.
- `internal/ratelimit`, `internal/bans`, `internal/rules`,
`internal/coreruleset`, `internal/geojs`, `internal/lookup`,
`internal/reputation`, `internal/routewatch`, `internal/anomaly` and
`internal/alerts`: one check or source each.
`internal/coreruleset`, `internal/lookup`, `internal/reputation`,
`internal/routewatch`, `internal/anomaly` and `internal/alerts`: one check or
source each.
- `internal/state`, `internal/requestlog`, `internal/remotelog` and
`internal/metrics`: the state and cache files, the log lines, the syslog
sender and the metrics.
+26 -75
View File
@@ -87,10 +87,8 @@ never as a time.
change, and tells which rules a request matches.
- `internal/coreruleset`: the Core Rule Set, with the five changes and the
apps' fixes listed under "Attack detection", as Coraza directives.
- `internal/geojs`: looks clients up through GeoJS and keeps the answers. It
imports no other package of this repository: `waf` gives it what it needs
to count and report failures.
- `internal/lookup`: looks clients up in the lookup database.
- `internal/lookup`: looks clients up through GeoJS or in the lookup
database.
- `internal/reputation`: fetches the lists, asks the DNS blocklists and
AbuseIPDB in the background, and keeps their answers.
- `internal/routewatch`: asks a routewatch instance about clients in the
@@ -173,8 +171,7 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
`country=` tag.
Otherwise the request goes on at once. A request that ends before the answer
comes is logged without it. While the request waits, steps 12 to 14 run at
the same time as steps 6 to 11 (see "Answering in time").
comes is logged without it.
7. The country list (`countries`; see "Country list").
8. The reputation sources (see "Reputation"), in this order:
@@ -510,25 +507,19 @@ service that keeps its visitors' addresses on its own host.
dropped from that table. The answer reaches the client's history and ban notes
when it comes.
- `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` is how long a visitor's request
waits for its client's first answer, when a setting needs it (see step 6). A
client whose answer has not come in time counts as unknown until it comes, and
its later requests do not wait.
- A request to GeoJS may take up to 10 seconds, however long visitors wait. It
goes on after the visitor's request ends, and its answer is kept. Say GeoJS
answers after 5 seconds. The visitor's request is decided after 800
milliseconds, without a country. A request from the same client 6 seconds
later gets the answer.
- Requests from one client that come together share one request to GeoJS.
waits for its client's first answer, when a setting needs it (see step 6). It
also bounds each request to GeoJS. A client whose answer has not come in time
counts as unknown until it comes, and its later requests do not wait. Its
address is asked about again in the background.
- A client the lookup cannot place has an unknown country and AS number, which
the country list and the biased thresholds treat as described there. Nothing
else treats it differently.
- GeoJS publishes no rate limit, but its terms forbid "an excessive amount of
API requests", judged by GeoJS alone, and let it block a caller. A request to
GeoJS fails when it takes over 10 seconds or GeoJS answers with an error.
While GeoJS is slow, down or refusing `waf`, new clients count as unknown.
After a failure GeoJS is left alone for a second, and twice as long after each
further failure, up to five minutes. Each failure raises a `source_failure`
alert (see "Alerting"). A service that cannot accept this uses the file.
API requests", judged by GeoJS alone, and let it block a caller. While GeoJS
is slow, down or refusing `waf`, new clients count as unknown. GeoJS is then
left alone for a second, and twice as long after each further failure, up to
five minutes. Each failure raises a `source_failure` alert (see "Alerting"). A
service that cannot accept this uses the file.
- The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`,
`as_name` and `country_code` fields are read. The operator downloads it with a
free IPinfo account and mounts it read-only; `waf` never fetches it and holds
@@ -2380,9 +2371,8 @@ runs no check, writes no request log line and counts nothing in the metrics.
### The answer
`waf` runs steps 3 to 16 of "What happens to a request", in that order, apart
from the steps that run at the same time (see "Answering in time"). Every answer
but the last below carries `X-Sww-Disposition`.
`waf` runs steps 3 to 16 of "What happens to a request", in that order. Every
answer but the last below carries `X-Sww-Disposition`.
- `X-Sww-Disposition: allow`, status `200`, no body. `rproxy` passes the request
to the app. While `SWW_WAF_ADD_LOOKUP_HEADERS` is `true`, the answer also
@@ -2394,8 +2384,6 @@ but the last below carries `X-Sww-Disposition`.
- `SWW_WAF_BAN_RESPONSE`, for a refusal;
- `403`, for a `block` rule or the Core Rule Set;
- `414` or `431`, for a request over the header limits;
- `503`, for a request that steps 12 to 14 have not finished within 500
milliseconds;
- `404`, for a path under `/_sww/` that passes the checks, so that the app
never sees that prefix.
- `X-Sww-Disposition: close`, status `200`, no body. This takes the place of a
@@ -2416,22 +2404,15 @@ The Core Rule Set inspects the body a question carries only while
`rproxy` stops waiting for `waf` after `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, by
default 2 seconds, and `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` then
decides. Two things in a decision take time:
- the wait for GeoJS, at most `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by
default 800 milliseconds;
- the trap paths, the rule files and the Core Rule Set, which have 500
milliseconds.
They run at the same time, so `waf` answers within the longer of the two, well
within `rproxy`'s wait.
decides. The longest a question waits on an outside source is
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by default 800 milliseconds, well
within that time.
GeoJS is the only outside source a question can wait on. A question waits only
when step 6 says so, and for at most
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by default 800 milliseconds. A
client without an answer by then counts as unknown. The setting bounds only the
question's wait: the request to GeoJS goes on, and its answer is kept for later
questions. No question waits on these, which run in the background:
client without an answer by then counts as unknown. No question waits on these,
which run in the background:
- the DNS blocklists and AbuseIPDB;
- fetching the blocklists, the tag lists and the file
@@ -2456,38 +2437,10 @@ never wait. These can still wait for GeoJS:
- a country list, which needs the answer;
- a rate limit a percentage has lowered, which needs the answer;
- a DNS blocklist or AbuseIPDB, while `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` is
`deny`.
The trap paths, the rule files and the Core Rule Set, steps 12 to 14, run during
that wait. Once a request has passed the bans, the lookup starts. While it waits
for GeoJS, steps 12 to 14 run at the same time as steps 6 to 11. A lookup that
needs no wait has its answer at once, and steps 7 to 11 then run before steps 12
to 14 start.
- The first of the two sides to refuse the request ends the checks. `waf`
answers at once, and does not wait for the other side. The lookup goes on, and
its answer is kept.
- What the other side finds leaves no trace. It counts nothing, bans nothing,
raises no alert, counts no metric and adds nothing to the log line.
- When both sides would refuse, the one that finishes first is answered and
logged.
- When neither refuses, step 15 runs.
Steps 12 to 14 only read the request. They have 500 milliseconds together, from
their start. The limit is fixed, with no setting. A request they have not
finished by then is refused:
- the answer is `deny`, with `503`;
- the log line's action is `checks_timed_out`, and its `heuristic` names the
step that had not finished;
- the refusal makes no ban, is no offence and does not count toward the refusal
limit;
- it holds in every `SWW_WAF_CORE_RULE_SET_MODE`;
- in `observe` mode the request is answered `allow` with `would_action`, like
every refusal.
The limit is measured on the clock. While `waf`'s host is so busy that an
ordinary request takes 500 milliseconds, that request is refused too.
`deny`;
- a trap path;
- a rule file;
- the Core Rule Set.
The header limits are fixed, with no setting. A line's length is in bytes and
does not count its line ending. `waf` refuses a request whose request line is
@@ -2537,11 +2490,9 @@ body can add up to more than the body. From 1 KiB of JSON, Coraza makes at most
A request can reach all of these at once. At paranoia level 4, with no rule
switched off, the Core Rule Set takes under a second on one core of a current
server to inspect it. `waf` refuses it if the inspection takes over 500
milliseconds. Coraza cannot stop an inspection part-way, so the inspection runs
to its end with nobody waiting for it. These limits keep one request from
holding a processor for more than about a second. Enough such requests at once
can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
server to inspect it. So `waf` answers it within the default 2 seconds, even
after the default 800 milliseconds of waiting for GeoJS. Enough such requests at
once can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
### The report
-7
View File
@@ -105,8 +105,6 @@ Every one of these lines has:
`blocked_by_core_rule_set`: refused by that check;
- `request_line_too_long`: a request line over the header limits;
- `request_header_too_large`: headers over the header limits;
- `checks_timed_out`: refused with `503` because the trap paths, the rule
files and the Core Rule Set had not finished within 500 ms;
- `own_endpoint`: answered at one of `waf`'s own endpoints, or `404` for a
path under `/_sww/`.
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
@@ -136,11 +134,6 @@ Every one of these lines has:
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`;
- `refusal_limit`: `window`, `count` and `limit`, of the refusal limit;
- `byte_limits`: `window`, `count` and `limit`, the count in bytes.
For a request refused as `checks_timed_out`, `heuristic` is the one of
`trap_paths`, `rule_files` and `core_rule_set` that had not finished, and
its figure is `time_limit_ms`, 500.
- `after_answer_heuristic`: the heuristic that banned the client after `waf`
answered the request, in `observe` mode too: `refusal_limit` or `byte_limits`.
- `after_answer_heuristic_figures`: an object of the figures that heuristic
+3 -6
View File
@@ -159,9 +159,8 @@ cut off; the client is banned.
- `SWW_WAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in
its `.mmdb` form, for `SWW_WAF_LOOKUP_SOURCE=file`.
- `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` (default `800`): how long a
request waits for its client's first answer from GeoJS. A time in
milliseconds; never `off`. It does not cut off the request to GeoJS, whose
answer is kept when it comes later.
request waits for its client's first answer from GeoJS, and how long a request
to GeoJS may take. A time in milliseconds; never `off`.
- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `true`): `true` has `rproxy` pass the
app the client's AS number in `X-Client-ASN` and its country in
`X-Client-Country`. The default is `false` while `SWW_WAF_LOOKUP_SOURCE` is
@@ -435,9 +434,7 @@ A few limits are fixed rather than settings:
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` is the longest a question waits on
an outside source. Keep it well under `rproxy`'s
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, as the default is. The trap paths, the rule
files and the Core Rule Set run during that wait and have 500 ms, so `waf`
answers within the longer of the two times. No question waits for
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, as the default is. No question waits for
`SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`, since the DNS blocklists and AbuseIPDB are
asked in the background (see "Answering in time" under "`waf`" in
[`SPEC.md`](../SPEC.md)).
+4 -5
View File
@@ -35,11 +35,10 @@ const (
mostJSONRead = 1 << 10
)
// waf refuses a request the Core Rule Set has not inspected within 500
// milliseconds, but Coraza cannot stop an inspection part-way, so it runs to
// its end with nobody waiting for it. The test checks that an inspection ends
// within about 1 second, so that a visitor cannot keep a processor busy for
// longer with one request.
// The Core Rule Set has 1 second for one request. That is the default of
// rproxy's SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, 2 seconds, less the default of
// SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, 800 milliseconds, and less 200
// milliseconds for waf's other checks.
//
// A busy host slows all work, so the test does not compare an inspection with
// 1 second. It compares it with referenceTime, timed on the same thread just
@@ -1,4 +1,4 @@
package geojs
package lookup
import "net/http"
+2 -6
View File
@@ -1,6 +1,3 @@
// Package lookup looks up each client's AS number and country in the lookup
// database, the IPinfo Lite file SWW_WAF_LOOKUP_DB_PATH names. Its answers
// take the form of GeoJS's, so that waf handles both alike.
package lookup
import (
@@ -17,7 +14,6 @@ import (
"github.com/oschwald/maxminddb-golang/v2"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/geojs"
)
// quietTime is how long the lookup database must go without a change
@@ -81,7 +77,7 @@ func OpenFile(params FileParams) (*File, error) {
// each "" when the database does not give it, as for an address missing
// from it. The database is asked about the client's first address, as
// GeoJS is.
func (f *File) LookUp(client netip.Prefix) geojs.Answer {
func (f *File) LookUp(client netip.Prefix) Answer {
f.mu.Lock()
reader := f.reader
f.mu.Unlock()
@@ -95,7 +91,7 @@ func (f *File) LookUp(client netip.Prefix) geojs.Answer {
found = record{}
}
return geojs.Answer{
return Answer{
Client: client,
ASN: found.ASN,
ASName: found.ASName,
+1 -2
View File
@@ -16,7 +16,6 @@ import (
"github.com/maxmind/mmdbwriter/mmdbtype"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
)
@@ -69,7 +68,7 @@ func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(
prefix := netip.MustParsePrefix(client)
got := f.LookUp(prefix)
if got != (geojs.Answer{
if got != (Answer{
Client: prefix, ASN: want.ASN, ASName: want.ASName, Country: want.Country,
Answered: now,
}) {
@@ -1,9 +1,9 @@
// Package geojs looks up clients' AS numbers and countries through the GeoJS
// web service. Its answers are kept in memory, for at most 100,000 clients and
// for 7 days each. A caller waits for an answer as long as it says. A request to
// GeoJS goes on after its callers stop waiting, and its answer is kept. The
// package imports no other package of this repository.
package geojs
// Package lookup looks up each client's AS number and country, through
// the GeoJS web service or in the lookup database, the IPinfo Lite file
// SWW_WAF_LOOKUP_DB_PATH names. GeoJS's answers are kept in memory, for at
// most 100,000 clients and for 7 days each, and are written to
// lookups.json and read from it by the state package.
package lookup
import (
"context"
@@ -11,6 +11,7 @@ import (
"errors"
"fmt"
"io"
"log/slog"
"net/http"
"net/netip"
"slices"
@@ -20,6 +21,8 @@ import (
"time"
"github.com/hashicorp/golang-lru/v2/simplelru"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/metrics"
)
// URL is GeoJS's endpoint for an address's place and network. Asked about
@@ -40,9 +43,6 @@ const (
maxWaiting = 10000
// maxPerRequest is how many addresses one request to GeoJS asks about.
maxPerRequest = 200
// requestTimeout is how long a request to GeoJS may take before it is
// abandoned, which is a failure. It is apart from how long callers wait.
requestTimeout = 10 * time.Second
// unknownASN is the AS number GeoJS gives when it knows none.
unknownASN = 64512
// After a failure GeoJS is not asked again for a second, and for
@@ -64,20 +64,25 @@ var (
type Params struct {
// URL is where GeoJS is asked, normally URL.
URL string
// Timeout is how long a request waits for its client's first answer,
// and how long a request to GeoJS may take before it is abandoned
// (SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS).
Timeout time.Duration
// Wait is true when a setting needs each request's answer before the
// request goes on. Otherwise no request waits for one.
Wait bool
// Answered, unless nil, is given each answer GeoJS gives, once it is
// kept.
Answered func(Answer)
// Now tells the time, normally time.Now in UTC.
Now func() time.Time
// Asked is called for each request to GeoJS.
Asked func()
// Failed is called for each failure of GeoJS, with what went wrong and
// how long GeoJS is left alone after it. The error never names the
// addresses asked about.
Failed func(err error, askingAgainIn time.Duration)
// WentWithoutAnswer is called each time a caller that waits for an
// answer goes on without one.
WentWithoutAnswer func()
// ProcessLog receives GeoJS's failures.
ProcessLog *slog.Logger
// Metrics count the requests to GeoJS, those that failed, and the
// clients that go without an answer.
Metrics *metrics.Metrics
// Alerts receive a source_failure alert each time GeoJS fails.
Alerts *alerts.Queue
}
// GeoJS looks up clients' AS numbers and countries through GeoJS. At most
@@ -85,11 +90,13 @@ type Params struct {
// client waiting, up to maxPerRequest. It is safe for concurrent use.
type GeoJS struct {
url string
timeout time.Duration
wait bool
answered func(Answer)
now func() time.Time
asked func()
failed func(err error, askingAgainIn time.Duration)
wentWithoutAnswer func()
processLog *slog.Logger
metrics *metrics.Metrics
alerts *alerts.Queue
// httpClient follows no redirect, so that visitors' addresses go to
// GeoJS alone: a redirect is a failure.
httpClient *http.Client
@@ -107,10 +114,12 @@ type GeoJS struct {
retryAt time.Time
}
// Answer is what GeoJS said about a client: its AS number, such as AS64496,
// and the AS's name, both "" when GeoJS knows no AS number for it; its
// country, "" when GeoJS cannot place it; when GeoJS said so; and when the
// answer was last used. The zero Answer is that of a client with no answer.
// Answer is what GeoJS or the lookup database said about a client: its AS
// number, such as AS64496, and the AS's name, both "" when the source knows
// no AS number for it; its country, "" when the source cannot place it;
// when the source said so; and, for GeoJS's answers, which lookups.json
// holds, when the answer was last used. The zero Answer is that of a
// client with no answer.
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
type Answer struct {
@@ -128,7 +137,7 @@ type wait struct {
// replaced each time GeoJS fails before then.
asked chan struct{}
// late is true once the client has gone without an answer, for a
// whole wait or because GeoJS failed: its callers no longer wait.
// whole timeout or because GeoJS failed: its requests no longer wait.
late bool
}
@@ -141,11 +150,13 @@ func New(params Params) *GeoJS {
return &GeoJS{
url: params.URL,
timeout: params.Timeout,
wait: params.Wait,
answered: params.Answered,
now: params.Now,
asked: params.Asked,
failed: params.Failed,
wentWithoutAnswer: params.WentWithoutAnswer,
processLog: params.ProcessLog,
metrics: params.Metrics,
alerts: params.Alerts,
httpClient: &http.Client{
CheckRedirect: func(*http.Request, []*http.Request) error {
return http.ErrUseLastResponse
@@ -159,22 +170,21 @@ func New(params Params) *GeoJS {
// LookUp returns the answer GeoJS gave about client, with its country as
// a two-letter code in capitals, or the zero Answer when there is none
// yet. An answer is kept for 7 days. Without one, the client is asked
// about in the background, and LookUp waits up to wait for the answer,
// unless the client has gone without one before. The wait ends early when
// ctx ends. Neither ends the request to GeoJS.
// about in the background, and, while Wait is set, the request waits up
// to Timeout for the answer, unless the client has gone without one
// before. ctx is the context of the client's request, and ends the wait
// when it ends.
//
// GeoJS is asked about the client's first address, which is the client's
// own address for IPv4, and an address in the same place for an IPv6
// netblock.
func (g *GeoJS) LookUp(
ctx context.Context, client netip.Prefix, wait time.Duration,
) Answer {
answer, asked := g.answerOrWait(ctx, client, true)
func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
answer, asked := g.answerOrWait(ctx, client, g.wait)
if asked == nil {
return answer
}
timer := time.NewTimer(wait)
timer := time.NewTimer(g.timeout)
defer timer.Stop()
select {
@@ -188,7 +198,7 @@ func (g *GeoJS) LookUp(
answer, found := g.kept(client)
if !found {
g.wentWithoutAnswer()
g.metrics.RequestsWithoutGeoJSAnswer.Inc()
}
w, waiting := g.waiting[client]
@@ -199,8 +209,8 @@ func (g *GeoJS) LookUp(
return answer
}
// LookUpWithoutWaiting is LookUp for a caller that does not need the answer:
// it returns the zero Answer at once when none is kept.
// LookUpWithoutWaiting is LookUp for a request that does not need the answer,
// whatever Wait says: it returns the zero Answer at once when none is kept.
func (g *GeoJS) LookUpWithoutWaiting(
ctx context.Context,
client netip.Prefix,
@@ -218,7 +228,8 @@ func (g *GeoJS) Kept(client netip.Prefix) (Answer, bool) {
return g.kept(client)
}
// Snapshot returns every answer kept, sorted by client.
// Snapshot returns every answer kept, sorted by client, as lookups.json
// lists them.
func (g *GeoJS) Snapshot() []Answer {
g.mu.Lock()
@@ -236,9 +247,9 @@ func (g *GeoJS) Snapshot() []Answer {
return answers
}
// Load keeps answers, such as those of an earlier Snapshot, in place of the
// answers it keeps, in the order they were last used, so that the one used
// longest ago is dropped first. Answers GeoJS gave keepFor ago or more are
// Load keeps answers read from lookups.json, in place of the answers it
// keeps, in the order they were last used, so that the one used longest
// ago is dropped first. Answers GeoJS gave keepFor ago or more are
// dropped.
func (g *GeoJS) Load(answers []Answer) {
answers = slices.Clone(answers)
@@ -284,11 +295,11 @@ func (g *GeoJS) answerOrWait(
g.ask(ctx)
if !mustWait {
return Answer{}, nil // the answer is not needed before the caller goes on
return Answer{}, nil // the answer is not needed before the request goes on
}
if w == nil {
g.wentWithoutAnswer()
g.metrics.RequestsWithoutGeoJSAnswer.Inc()
return Answer{}, nil // too many clients wait already
}
@@ -299,7 +310,7 @@ func (g *GeoJS) answerOrWait(
}
if w.late {
g.wentWithoutAnswer()
g.metrics.RequestsWithoutGeoJSAnswer.Inc()
return Answer{}, nil
}
@@ -324,8 +335,8 @@ func (g *GeoJS) kept(client netip.Prefix) (Answer, bool) {
// ask starts asking GeoJS about the waiting clients, unless a request to
// it is under way or it is left alone after a failure. The requests to
// GeoJS are for every client waiting, so they go on when the caller whose
// ctx is given stops waiting.
// GeoJS are for every client waiting, so they go on when the client's
// request whose ctx is given ends.
func (g *GeoJS) ask(ctx context.Context) {
if g.asking || g.now().Before(g.retryAt) {
return
@@ -430,6 +441,8 @@ func (g *GeoJS) keep(
}
if err != nil {
g.metrics.GeoJSFailures.Inc()
g.retryDelay = min(
max(retryDelayFactor*g.retryDelay, firstRetryDelay),
maxRetryDelay,
@@ -444,7 +457,16 @@ func (g *GeoJS) keep(
w.late = true
}
g.failed(err, g.retryDelay)
g.processLog.Warn("asking GeoJS failed",
"error", err.Error(), "asking_again_in", g.retryDelay.String())
g.alerts.Raise(alerts.Alert{
Event: alerts.EventSourceFailure,
Reason: "asking GeoJS failed",
Detail: map[string]any{
"source": "geojs", "error": err.Error(),
"asking_again_in": g.retryDelay.String(),
},
})
return kept, false
}
@@ -457,8 +479,7 @@ func (g *GeoJS) keep(
// request asks GeoJS about clients in one request, and returns the answer
// for each address GeoJS's answer names: its AS number and the AS's name,
// both "" for the AS number 64512, which GeoJS gives when it knows none,
// and its country, in capitals. The request is abandoned after
// requestTimeout.
// and its country, in capitals.
func (g *GeoJS) request(
ctx context.Context, clients []netip.Prefix,
) (map[netip.Addr]Answer, error) {
@@ -468,7 +489,7 @@ func (g *GeoJS) request(
addrs = append(addrs, client.Addr().String())
}
ctx, cancel := context.WithTimeout(ctx, requestTimeout)
ctx, cancel := context.WithTimeout(ctx, g.timeout)
defer cancel()
req, err := http.NewRequestWithContext(
@@ -483,7 +504,7 @@ func (g *GeoJS) request(
req.URL.RawQuery = "ip=" + strings.Join(addrs, ",")
g.asked()
g.metrics.GeoJSRequests.Inc()
res, err := g.httpClient.Do(req)
if err != nil {
File diff suppressed because it is too large. Load diff
@@ -1,4 +1,4 @@
package geojs_test
package lookup_test
import (
"net/netip"
@@ -7,7 +7,7 @@ import (
"testing/synctest"
"time"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
)
func TestSnapshotHoldsEachAnswerAndWhenItWasLastUsed(t *testing.T) {
@@ -25,7 +25,7 @@ func TestSnapshotHoldsEachAnswerAndWhenItWasLastUsed(t *testing.T) {
clock.advance(time.Hour)
wantCountry(t, g, placed, germany)
want := []geojs.Answer{
want := []lookup.Answer{
{Client: notPlaced, Answered: asked, Used: asked},
{
Client: placed, ASN: asn, ASName: asName, Country: germany,
@@ -41,24 +41,24 @@ func TestSnapshotHoldsEachAnswerAndWhenItWasLastUsed(t *testing.T) {
func TestLoadedAnswersAreKeptFor7DaysFromWhenGeoJSGaveThem(t *testing.T) {
t.Parallel()
service, clock, g := start()
geojs, clock, g := start()
now := clock.Now()
kept := geojs.Answer{
kept := lookup.Answer{
Client: netip.MustParsePrefix("203.0.113.9/32"),
Country: "FR",
Answered: now.Add(-week + time.Second),
Used: now.Add(-time.Hour),
}
stale := geojs.Answer{
stale := lookup.Answer{
Client: netip.MustParsePrefix("203.0.113.10/32"),
Country: "FR",
Answered: now.Add(-week),
Used: now.Add(-time.Hour),
}
g.Load([]geojs.Answer{kept, stale})
g.Load([]lookup.Answer{kept, stale})
if got := g.Snapshot(); !slices.Equal(got, []geojs.Answer{kept}) {
if got := g.Snapshot(); !slices.Equal(got, []lookup.Answer{kept}) {
t.Errorf(
"kept %+v, want only the answer GeoJS gave less than 7 days ago",
got,
@@ -66,7 +66,7 @@ func TestLoadedAnswersAreKeptFor7DaysFromWhenGeoJSGaveThem(t *testing.T) {
}
wantCountry(t, g, kept.Client, "FR")
wantRequests(t, service, 0)
wantRequests(t, geojs, 0)
}
func TestLoadDropsTheAnswerUsedLongestAgoFirst(t *testing.T) {
@@ -80,11 +80,11 @@ func TestLoadDropsTheAnswerUsedLongestAgoFirst(t *testing.T) {
// lookups.json lists the answers by client. Here each was last used a
// second before the one listed before it, so the last listed is the
// one used longest ago, and the one dropped.
answers := make([]geojs.Answer, maxAnswers+1)
answers := make([]lookup.Answer, maxAnswers+1)
addr := netip.MustParseAddr("10.0.0.0")
for i := range answers {
answers[i] = geojs.Answer{
answers[i] = lookup.Answer{
Client: netip.PrefixFrom(addr, addr.BitLen()),
Country: germany,
Answered: now,
+2 -1
View File
@@ -274,7 +274,8 @@ func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
// AddLookupFile adds the metrics of the lookup database, read as the
// metrics are asked for: when the file in use was read, which lastRead
// returns, and the replacements of it that could not be read, which
// readFailures returns.
// readFailures returns. The lookup package's File, which has both, cannot
// be named here: that package counts GeoJS's requests in these metrics.
func (m *Metrics) AddLookupFile(
lastRead func() time.Time,
readFailures func() int,
+1 -8
View File
@@ -61,12 +61,6 @@ const (
// ActionCountryDenied is a request refused because its client's country
// is not in SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
ActionCountryDenied = "country_denied"
// ActionChecksTimedOut is a request refused because the trap paths, the
// rule files and the Core Rule Set had not finished with it within 500
// milliseconds.
ActionChecksTimedOut = "checks_timed_out"
// ActionOwnEndpoint is a request waf answered at one of its own
// endpoints, or a question about a path under /_sww/.
ActionOwnEndpoint = "own_endpoint"
@@ -155,8 +149,7 @@ type Line struct {
// ActionTemporarilyBanned, ActionPermanentlyBanned, ActionCountryDenied,
// ActionClientIsRateLimited, ActionBlockedByBlockRule,
// ActionBlockedByCoreRuleSet, or, in waf, ActionRequestLineTooLong or
// ActionRequestHeaderTooLarge for the header limits, or
// ActionChecksTimedOut.
// ActionRequestHeaderTooLarge for the header limits.
WouldAction string `json:"would_action,omitempty"`
// Heuristic is the name of the heuristic that refused the request, in
// observe mode too, and HeuristicFigures are the figures it acted on, by
+3 -3
View File
@@ -46,7 +46,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation"
@@ -135,7 +135,7 @@ type Params struct {
// sww runs.
Ledger *bans.Ledger
Limiter *ratelimit.Limiter
GeoJS *geojs.GeoJS
GeoJS *lookup.GeoJS
Lists *reputation.Lists
DNSBL *reputation.DNSBL
AbuseIPDB *reputation.AbuseIPDB
@@ -196,7 +196,7 @@ type clientsFile struct {
// lookupsFile is lookups.json, with each answer on a line of its own.
type lookupsFile struct {
Version int `json:"version"`
Lookups []geojs.Answer `json:"lookups"`
Lookups []lookup.Answer `json:"lookups"`
}
// abuseIPDBChecksSpentFile is abuseipdb-checks-spent.json, indented for an
+6 -4
View File
@@ -26,7 +26,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation"
@@ -1700,7 +1700,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
t,
lookupsJSON,
params.GeoJS.Snapshot(),
[]geojs.Answer{
[]lookup.Answer{
{Client: client, Country: "FR", Answered: midnight()},
},
)
@@ -2415,7 +2415,9 @@ func newParams(dir string) state.Params {
MaxBans: 5000,
}),
Limiter: ratelimit.New(ratelimit.Limits{}, 20000),
GeoJS: geojs.New(geojs.Params{Now: midnight}),
GeoJS: lookup.New(lookup.Params{
Now: midnight, ProcessLog: discard, Metrics: m,
}),
Lists: reputation.New(reputation.Params{
BlocklistURLs: []string{
blocklistURL,
@@ -2492,7 +2494,7 @@ func fill(params state.Params) {
})
params.Limiter.AddLookup(client, now.Add(-time.Hour), asn, asName, "DE")
params.GeoJS.Load([]geojs.Answer{
params.GeoJS.Load([]lookup.Answer{
{
Client: client, ASN: asn, ASName: asName, Country: "DE",
Answered: now.Add(-time.Hour), Used: now,
+1 -2
View File
@@ -25,7 +25,6 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/remotelog"
"sneak.berlin/go/smallwebwaf/internal/reputation"
@@ -271,7 +270,7 @@ func newServer(
Config: cfg,
RequestLog: stdout,
ProcessLog: processLog,
GeoJSURL: geojs.URL,
GeoJSURL: lookup.URL,
AbuseIPDBURL: reputation.AbuseIPDBURL,
LookupFile: lookupFile,
CoreRuleSet: crs,
+1 -3
View File
@@ -19,8 +19,6 @@ const (
alertMaxPerHour = "SWW_WAF_ALERT_MAX_PER_HOUR"
// alertInstance is the instance every alert of these tests gives.
alertInstance = "host1/blog"
// alertWebhook is where the tests send alerts.
alertWebhook = "https://alerts.example/sww"
)
func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
@@ -272,7 +270,7 @@ func startAppWithAlerts(
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)}
settings := map[string]string{
trustedProxies: trustLocalhost,
alertWebhookURL: alertWebhook,
alertWebhookURL: "https://alerts.example/sww",
instanceName: alertInstance,
}
maps.Copy(settings, env)
+4 -9
View File
@@ -20,7 +20,7 @@ import (
"github.com/prometheus/client_golang/prometheus/testutil"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation"
@@ -591,7 +591,7 @@ func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(
s := startWAF(t, geojsURL, map[string]string{
unknownCountryLimitPercent: "0", rateLimitPerMinute: "1",
})
s.server.GeoJS.Load([]geojs.Answer{{
s.server.GeoJS.Load([]lookup.Answer{{
Client: netip.MustParsePrefix(client + "/32"), ASN: "AS64512",
Answered: s.clock.Now(), Used: s.clock.Now(),
}})
@@ -770,8 +770,6 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
// so that a request that does not wait takes no time at all.
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
server, out := newWAFInBubble(t, tc.env)
tc.prepare(t, server, time.Now())
@@ -840,8 +838,6 @@ func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(
t.Parallel()
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
server, out := newWAFInBubble(t, tc.env)
tc.prepare(t, server, time.Now())
@@ -954,11 +950,10 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
t.Errorf("%d checks with AbuseIPDB, want 1", checked)
}
// The query is answered, and the check with AbuseIPDB and the request
// to GeoJS abandoned, before the bubble ends.
// The query is answered, and the check with AbuseIPDB abandoned,
// before the bubble ends.
releaseResolver()
time.Sleep(defaultDNSBLAbuseIPDBWait)
waitForGeoJSRequestsToBeAbandoned()
})
}
-172
View File
@@ -1,172 +0,0 @@
package waf
// The heuristics of besideLookup, and how they run beside the lookup.
import (
"context"
"sync/atomic"
"time"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
)
// besideLookupTimeLimit is how long the heuristics of besideLookup have for a
// request, from when they start. They only read the request, and no request
// should take them that long. A request they have not finished by then is
// refused with 503, as ActionChecksTimedOut.
const besideLookupTimeLimit = 500 * time.Millisecond
// besideLookupFindings is what the heuristics of besideLookup found of a
// request, before any of it is applied to the request.
type besideLookupFindings struct {
// trapPath is the request's path when it is in SWW_WAF_TRAP_PATHS, and ""
// otherwise.
trapPath string
// rules are the rules of the rule files the request matched.
rules []rules.Rule
// inspected is true when the Core Rule Set inspected the request.
// inspection, read and inspectionTime are then what it found, what it read
// of the body, and how long it took.
inspected bool
inspection coreruleset.Result
read []byte
inspectionTime time.Duration
// refuses is true when one of the heuristics refuses the request, or would
// in observe mode. Those after it did no work.
refuses bool
}
// runBesideLookup runs the heuristics of besideLookup while the lookup waits
// for GeoJS, and returns the first verdict, or nil when none gives one.
// lookedUp is the channel the lookup's result comes on, or nil once afterLookup
// has let the request through.
//
// The first of the two to refuse the request ends the checks: waf answers at
// once, and what the other found is dropped. besideLookup has
// besideLookupTimeLimit; past it, the request is refused as
// ActionChecksTimedOut. Once both have let the request through, the heuristics
// of afterBesideLookup run. ctx is the request's own context.
func (rq *request) runBesideLookup(
ctx context.Context, lookedUp <-chan lookupResult,
) *verdict {
var running atomic.Pointer[string]
running.Store(new(trapPaths{}.name()))
found := make(chan *besideLookupFindings, 1)
go func() { found <- rq.findBesideLookup(&running) }()
timeUp := time.NewTimer(rq.h.besideLookupLimit)
defer timeUp.Stop()
var findings *besideLookupFindings
for findings == nil || lookedUp != nil {
select {
case result := <-lookedUp:
lookedUp = nil
v := rq.runAfterLookup(ctx, result)
if v != nil {
return v
}
case findings = <-found:
found = nil
timeUp.Stop()
if findings.refuses {
return rq.applyFindings(ctx, findings)
}
case <-timeUp.C:
return rq.timedOut(*running.Load())
}
}
v := rq.applyFindings(ctx, findings)
if v != nil {
return v
}
return rq.runHeuristics(ctx, afterBesideLookup(), rq.h.now())
}
// findBesideLookup does the work of the heuristics of besideLookup on the
// request, in their order, until one refuses it, and returns what they found.
// It changes nothing of the request or of what waf keeps, so that it can run
// while the lookup and afterLookup do. running holds the name of the heuristic
// at work, the first one's when findBesideLookup is called.
func (rq *request) findBesideLookup(
running *atomic.Pointer[string],
) *besideLookupFindings {
findings := &besideLookupFindings{}
findings.trapPath = rq.trapPath()
if findings.trapPath != "" {
findings.refuses = true
return findings
}
running.Store(new(ruleFiles{}.name()))
findings.rules = rq.h.rules.Match(rq.in)
if refusedByRule(findings.rules) {
findings.refuses = true
return findings
}
running.Store(new(coreRuleSet{}.name()))
cfg := rq.h.config
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff ||
pathExempt(rq.in, cfg.CoreRuleSetExemptPaths) {
return findings
}
start := time.Now()
// A question's body has been read into memory, so that reading it here
// cannot fail.
findings.inspection, findings.read, _ = rq.h.coreRuleSet.Inspect(
rq.in, rq.client, rq.in.Body,
)
findings.inspectionTime = time.Since(start)
findings.inspected = true
findings.refuses = rq.refusedByCoreRuleSet(findings.inspection)
return findings
}
// applyFindings applies findings, what findBesideLookup found, to the
// request: it runs the heuristics of besideLookup, which read them, at the
// time it is called, and returns the first verdict, or nil when none gives
// one.
func (rq *request) applyFindings(
ctx context.Context, findings *besideLookupFindings,
) *verdict {
rq.findings = findings
return rq.runHeuristics(ctx, besideLookup(), rq.h.now())
}
// timedOut refuses the request for the heuristics of besideLookup not having
// finished with it within besideLookupTimeLimit. running is the name of the
// one at work, which the log line's heuristic gives. The refusal makes no ban
// and is no offence.
func (rq *request) timedOut(running string) *verdict {
v := &verdict{
action: requestlog.ActionChecksTimedOut,
figures: map[string]any{
"time_limit_ms": rq.h.besideLookupLimit.Milliseconds(),
},
}
rq.line.Heuristic = running
rq.line.HeuristicFigures = v.figures
return v
}
-426
View File
@@ -1,426 +0,0 @@
package waf_test
import (
"io"
"maps"
"net/http"
"net/http/httptest"
"net/netip"
"strconv"
"strings"
"sync/atomic"
"testing"
"testing/synctest"
"time"
"github.com/prometheus/client_golang/prometheus/testutil"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/waf"
)
// checksTimeLimit is how long the trap paths, the rule files and the Core Rule
// Set have for a request, as SPEC.md gives it, and checksTimedOut the status of
// a request they have not finished by then.
const (
checksTimeLimit = 500 * time.Millisecond
checksTimedOut = http.StatusServiceUnavailable
)
// heldCoreRuleSet is a stand-in for the Core Rule Set: each inspection takes
// took, then finds result. asked counts the inspections.
type heldCoreRuleSet struct {
took time.Duration
result coreruleset.Result
asked atomic.Int64
}
// Inspect inspects a request as the stand-in does, and reads all of body.
func (c *heldCoreRuleSet) Inspect(
_ *http.Request, _ netip.Addr, body io.Reader,
) (coreruleset.Result, []byte, error) {
c.asked.Add(1)
time.Sleep(c.took)
read, err := io.ReadAll(body)
return c.result, read, err
}
// coreRuleSetMatch returns what the stand-in finds of an attack: rule 942100,
// with a score of 5, the default SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD.
func coreRuleSetMatch() coreruleset.Result {
return coreruleset.Result{RuleIDs: []int{942100}, Score: 5}
}
// newWAFHoldingTheCoreRuleSet makes a waf, to be asked in a synctest bubble,
// with crs in place of the Core Rule Set, in block mode. GeoJS never answers,
// a setting needs its answer, alerts go to a webhook, and everything else is
// at its default but for env. No request to it may use the network.
func newWAFHoldingTheCoreRuleSet(
t *testing.T, crs *heldCoreRuleSet, env map[string]string,
) (*waf.Server, *output, *alerts.Queue) {
t.Helper()
settings := map[string]string{
tagDeny: denyKP, coreRuleSetMode: block,
alertWebhookURL: alertWebhook,
}
maps.Copy(settings, env)
server, out, queue := newWAFWith(t, func(params waf.Params) *waf.Server {
params.CoreRuleSet = crs
return waf.New(params)
}, silentGeoJSURL, time.Now, settings)
server.SetBesideLookupTimeLimit(waf.BesideLookupTimeLimit)
return server, out, queue
}
// lookupWait is SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS for wait.
func lookupWait(wait time.Duration) string {
return strconv.FormatInt(wait.Milliseconds(), 10)
}
// wantAnswer checks the status of an answer and how long it took.
func wantAnswer(
t *testing.T, status int, took time.Duration,
wantStatus int, wantTook time.Duration, out *output,
) {
t.Helper()
if status != wantStatus || took != wantTook {
t.Errorf(
"answered %d after %s, want %d after %s; output:\n%s",
status, took, wantStatus, wantTook, out.text(),
)
}
}
// wantNoCoreRuleSetTrace checks that what the Core Rule Set found of a request
// left no trace: line has none of its fields, and server has counted no match
// of it and raised no alert for one.
func wantNoCoreRuleSetTrace(
t *testing.T, server *waf.Server, queue *alerts.Queue, line logLine,
) {
t.Helper()
if line.DurationCoreRuleSetMS != nil || line.CoreRuleSetScore != nil ||
line.CoreRuleSetRuleIDs != nil {
t.Errorf(
"log line has Core Rule Set time %v, score %v and rule ids %v, want none",
line.DurationCoreRuleSetMS,
line.CoreRuleSetScore,
line.CoreRuleSetRuleIDs,
)
}
recorder := httptest.NewRecorder()
server.Metrics.ServeHTTP(
recorder,
httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
waf.MetricsPath,
http.NoBody,
),
)
if strings.Contains(
recorder.Body.String(),
"sww_core_rule_set_matches_total{",
) {
t.Errorf("a Core Rule Set match is counted")
}
for _, alert := range queue.Snapshot().Waiting[alerts.DestinationWebhook] {
if alert.Event == alerts.EventCoreRuleSetMatch {
t.Errorf("alert raised: %+v", alert)
}
}
}
func TestTheLookupAndTheCoreRuleSetTakeTheLongerOfTheirTimesNotTheirSum(
t *testing.T,
) {
t.Parallel()
inspection := checksTimeLimit - time.Millisecond
for _, tc := range []struct {
name string
// wait is how long the request waits for GeoJS.
wait time.Duration
}{
{"the lookup waits longer", defaultLookupWait},
{"the inspection takes longer", inspection / 2},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
crs := &heldCoreRuleSet{took: inspection}
server, out, _ := newWAFHoldingTheCoreRuleSet(
t,
crs,
map[string]string{
asnAndCountryLookupTimeout: lookupWait(tc.wait),
},
)
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
status, took := askDirectly(
t,
server,
func(*http.Request) {},
)
wantAnswer(t, status, took, http.StatusOK,
max(tc.wait, inspection), out)
wantLine(t, out.requestLine(t), http.StatusOK,
requestlog.ActionForward)
})
})
}
}
// firstToRefuse is a case of
// TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered: a request
// under env, whose inspection by the Core Rule Set takes inspection, refused as
// action after took.
type firstToRefuse struct {
name string
env map[string]string
inspection time.Duration
took time.Duration
action string
}
// firstToRefuseCases are the cases of
// TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered.
func firstToRefuseCases() []firstToRefuse {
return []firstToRefuse{
{
name: "a trap path, before GeoJS answers",
env: map[string]string{trapPaths: "/", allowedCountries: "DE"},
took: 0, action: requestlog.ActionTemporarilyBanned,
},
{
name: "the Core Rule Set, before GeoJS answers",
env: map[string]string{allowedCountries: "DE"},
inspection: 100 * time.Millisecond,
took: 100 * time.Millisecond, action: requestlog.ActionBlockedByCoreRuleSet,
},
{
name: "the country list, while the Core Rule Set inspects",
env: map[string]string{
allowedCountries: "DE", asnAndCountryLookupTimeout: "100",
},
inspection: 400 * time.Millisecond,
took: 100 * time.Millisecond, action: requestlog.ActionCountryDenied,
},
{
name: "the rate limits, while the Core Rule Set inspects",
env: map[string]string{
rateLimitPerMinute: "2", unknownCountryLimitPercent: "50",
asnAndCountryLookupTimeout: "100",
},
inspection: 400 * time.Millisecond,
took: 100 * time.Millisecond, action: requestlog.ActionClientIsRateLimited,
},
}
}
func TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered(
t *testing.T,
) {
t.Parallel()
const forbidden = http.StatusForbidden
for _, tc := range firstToRefuseCases() {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
crs := &heldCoreRuleSet{
took: tc.inspection,
result: coreRuleSetMatch(),
}
server, out, queue := newWAFHoldingTheCoreRuleSet(t, crs, tc.env)
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
// Under the rate limits of the last case, the client's next
// request takes it over a limit only at the percentage of an
// unknown country, so it waits for GeoJS.
countClient(t, server, time.Now())
status, took := askDirectly(
t,
server,
func(*http.Request) {},
)
wantAnswer(t, status, took, forbidden, tc.took, out)
line := out.requestLine(t)
wantLine(t, line, forbidden, tc.action)
if tc.action == requestlog.ActionBlockedByCoreRuleSet {
return
}
if tc.action != requestlog.ActionClientIsRateLimited &&
line.Counts != (ratelimit.Counts{}) {
t.Errorf(
"the rate limits counted the request: %+v",
line.Counts,
)
}
// What the Core Rule Set finds once the request is answered is
// dropped.
time.Sleep(tc.inspection)
wantNoCoreRuleSetTrace(t, server, queue, line)
})
})
}
}
func TestARequestRefusedBeforeTheLookupStartsNoInspection(t *testing.T) {
t.Parallel()
crs := &heldCoreRuleSet{took: time.Second}
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs, nil)
synctest.Test(t, func(t *testing.T) {
banClient(t, server, time.Now())
status, took := askDirectly(t, server, func(*http.Request) {})
wantAnswer(t, status, took, http.StatusForbidden, 0, out)
requests := testutil.ToFloat64(server.Metrics.GeoJSRequests)
if asked := crs.asked.Load(); asked != 0 || requests != 0 {
t.Errorf(
"%d inspections and %v requests to GeoJS, want none",
asked,
requests,
)
}
})
}
func TestAClientInTheAllowNetsIsAllowedAtOnceAndNothingElseStarts(
t *testing.T,
) {
t.Parallel()
crs := &heldCoreRuleSet{took: time.Second, result: coreRuleSetMatch()}
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
map[string]string{allowNets: client, allowedCountries: "DE"})
synctest.Test(t, func(t *testing.T) {
status, took := askDirectly(t, server, func(*http.Request) {})
wantAnswer(t, status, took, http.StatusOK, 0, out)
requests := testutil.ToFloat64(server.Metrics.GeoJSRequests)
if asked := crs.asked.Load(); asked != 0 || requests != 0 {
t.Errorf(
"%d inspections and %v requests to GeoJS, want none",
asked,
requests,
)
}
})
}
func TestARequestTheCoreRuleSetHasNotFinishedWithIn500MillisecondsIsRefused(
t *testing.T,
) {
t.Parallel()
crs := &heldCoreRuleSet{took: checksTimeLimit + time.Second}
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
map[string]string{refusalsLimitPerMinute: "1"})
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
// Were it counted toward the refusal limit, the second refusal would
// ban the client, and the third request would be refused for the ban.
for i := range 3 {
status, took := askDirectly(t, server, func(req *http.Request) {
req.Header.Set(requestIDHeader, "id-"+strconv.Itoa(i))
})
wantAnswer(t, status, took, checksTimedOut, checksTimeLimit, out)
line := out.requestLines(t, i+1)[i]
wantLine(
t,
line,
checksTimedOut,
requestlog.ActionChecksTimedOut,
)
wantHeuristic(t, line, "core_rule_set", `{"time_limit_ms": 500}`)
if line.Offence != "" || line.BanExpires != "" {
t.Errorf(
"log line has offence %q and ban_expires %q, want neither",
line.Offence,
line.BanExpires,
)
}
}
if bans := server.Ledger.Snapshot(); len(bans) != 0 {
t.Errorf("bans %+v, want none", bans)
}
offences := historyOf(t, server, client).Offences
if offences != (ratelimit.Offences{}) {
t.Errorf("history counts the offences %+v, want none", offences)
}
time.Sleep(crs.took)
})
}
func TestObserveModeAllowsARequestTheCoreRuleSetHasNotFinishedWith(
t *testing.T,
) {
t.Parallel()
crs := &heldCoreRuleSet{took: checksTimeLimit + time.Second}
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
map[string]string{mode: observe})
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
status, took := askDirectly(t, server, func(*http.Request) {})
wantAnswer(t, status, took, http.StatusOK, checksTimeLimit, out)
line := out.requestLine(t)
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
if line.WouldAction != requestlog.ActionChecksTimedOut {
t.Errorf(
"log line has would_action %q, want %q",
line.WouldAction,
requestlog.ActionChecksTimedOut,
)
}
time.Sleep(crs.took)
})
}
-4
View File
@@ -339,8 +339,6 @@ func TestClientWithoutAnAnswerInTimeHasTheUnknownCountryLimitPercent(
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says, with
// a GeoJS that never answers.
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
map[string]string{unknownCountryLimitPercent: "0"})
@@ -392,8 +390,6 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
// with a GeoJS that never answers. The headers, which need the
// answer too, are off.
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
map[string]string{
asnAndCountryLookupTimeout: strconv.FormatInt(
+15 -23
View File
@@ -13,11 +13,10 @@ import (
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// coreRuleSet takes what the Core Rule Set found of the request, which
// findBesideLookup inspected with it unless SWW_WAF_CORE_RULE_SET_MODE is off
// or SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS exempts its path, as pathExempt
// decides. It notes the rules the request matched and its score in the log
// line, and the rules in the metrics. A score at or over
// coreRuleSet inspects the request with the Core Rule Set, unless
// SWW_WAF_CORE_RULE_SET_MODE is off or SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS
// exempts its path, as pathExempt decides, and notes the rules it matched and
// its score in the log line, and the rules in the metrics. A score at or over
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD is a match: it adds the tag
// exploitsignature, raises the core_rule_set_match alert, and in block mode
// refuses the request with 403, which is an offence its client's history
@@ -31,17 +30,21 @@ func (coreRuleSet) check(
rq *request,
_ time.Time,
) *verdict {
findings := rq.findings
if !findings.inspected {
cfg := rq.h.config
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff ||
pathExempt(rq.in, cfg.CoreRuleSetExemptPaths) {
return nil
}
cfg := rq.h.config
result := findings.inspection
rq.requestBytes = int64(len(findings.read))
start := time.Now()
// A question's body has been read into memory, so that reading it here
// cannot fail.
result, read, _ := rq.h.coreRuleSet.Inspect(rq.in, rq.client, rq.in.Body)
rq.requestBytes = int64(len(read))
rq.line.DurationCoreRuleSetMS = new(
requestlog.Milliseconds(findings.inspectionTime),
requestlog.Milliseconds(time.Since(start)),
)
rq.line.CoreRuleSetRuleIDs = result.RuleIDs
rq.line.CoreRuleSetScore = &result.Score
@@ -58,7 +61,7 @@ func (coreRuleSet) check(
rq.addTag(config.ExploitSignatureTag)
rq.alertCoreRuleSetMatch(result)
if !rq.refusedByCoreRuleSet(result) {
if cfg.CoreRuleSetMode == config.CoreRuleSetModeDetect {
return nil
}
@@ -73,17 +76,6 @@ func (coreRuleSet) check(
}
}
// refusedByCoreRuleSet reports whether coreRuleSet refuses a request the Core
// Rule Set found result of: whether the score is at or over
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD in block mode.
func (rq *request) refusedByCoreRuleSet(result coreruleset.Result) bool {
cfg := rq.h.config
threshold := cfg.CoreRuleSetScoreThreshold
return cfg.CoreRuleSetMode == config.CoreRuleSetModeBlock &&
threshold != 0 && result.Score >= threshold
}
// alertCoreRuleSetMatch raises the core_rule_set_match alert for the request,
// which the Core Rule Set scored at result, at or over
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD. Its detail gives the rule ids, the
-13
View File
@@ -1,13 +0,0 @@
package waf
import "time"
// BesideLookupTimeLimit is besideLookupTimeLimit.
const BesideLookupTimeLimit = besideLookupTimeLimit
// SetBesideLookupTimeLimit gives the heuristics of besideLookup limit on s in
// place of besideLookupTimeLimit.
func (s *Server) SetBesideLookupTimeLimit(limit time.Duration) {
h := s.Handler.(*handler) //nolint:forcetypeassert // New makes it one
h.besideLookupLimit = limit
}
+3 -18
View File
@@ -54,6 +54,8 @@ func beforeLookup() []heuristic {
}
// afterLookup are the heuristics that run once the lookup has been made.
// tag_deny runs twice: before the rate limits, on the tags known by then, and
// last, on the tag rule_files and core_rule_set can add.
func afterLookup() []heuristic {
return []heuristic{
countries{},
@@ -62,26 +64,9 @@ func afterLookup() []heuristic {
abuseIPDB{},
tagDeny{},
rateLimits{},
}
}
// besideLookup are the heuristics that only read the request. Their work runs
// beside the lookup and afterLookup, in findBesideLookup, which changes nothing
// of the request. Their check then applies what that work found, and runs only
// once waf takes it.
func besideLookup() []heuristic {
return []heuristic{
trapPaths{},
ruleFiles{},
coreRuleSet{},
}
}
// afterBesideLookup are the heuristics that run once afterLookup and
// besideLookup have both let the request through. tag_deny runs a second time,
// on the tag rule_files and core_rule_set can add.
func afterBesideLookup() []heuristic {
return []heuristic{
tagDeny{},
}
}
@@ -110,7 +95,7 @@ func (rq *request) runAfterAnswer(ctx context.Context) {
rq.runBanningHeuristics(ctx, afterAnswer())
}
// runHeuristics runs list, one of those checkClient runs, on the request at now,
// runHeuristics runs list, beforeLookup or afterLookup, on the request at now,
// and returns the first verdict, or nil when none gives one. The log line's
// heuristic and heuristic_figures name the heuristic that gave it, and give its
// figures. ctx is the request's own context.
+21 -85
View File
@@ -7,9 +7,8 @@ import (
"strings"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
)
// The headers in which an answer of allow gives rproxy the client's AS number
@@ -21,80 +20,33 @@ const (
countryHeader = "X-Client-Country"
)
// lookupResult is what the lookup gave for a request: whether its client was
// looked up, the answer, and whether the request skipped the wait for GeoJS.
type lookupResult struct {
lookedUp, skippedWait bool
answer geojs.Answer
}
// startLookUp starts looking up the client's AS number and country, in the
// lookup database or through GeoJS, unless SWW_WAF_LOOKUP_SOURCE is off or the
// client is on a private, loopback or link-local address, which no lookup can
// place. It returns the channel the result comes on. A result that needs no
// wait is on it when startLookUp returns: the lookup database's, an answer
// GeoJS gave earlier, or none. With GeoJS, while a setting needs the answer,
// such as a country list or a biased threshold, a new client's request waits
// for it up to SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, unless
// refusedWhateverTheAnswer reports true. The wait goes on to its end even once
// the request has been answered, so that it ends as it would have for a
// request that waited. startLookUp changes nothing of the request. ctx is the
// request's own context.
func (rq *request) startLookUp(
ctx context.Context, now time.Time,
) <-chan lookupResult {
result := make(chan lookupResult, 1)
// lookUp looks up the client's AS number and country, in the lookup
// database or through GeoJS, and notes them for the log line and as the
// request's asn= and country= tags, unless
// SWW_WAF_LOOKUP_SOURCE is off or the client is on a private, loopback or
// link-local address, which no lookup can place. The lookup database
// answers at once. With GeoJS, while a setting needs the answer, such as a
// country list or a biased threshold, a new client's request waits for it,
// unless refusedWhateverTheAnswer reports true. ctx is the request's own
// context.
func (rq *request) lookUp(ctx context.Context, now time.Time) {
if rq.h.config.LookupSource == "off" || !canBePlaced(rq.client) {
result <- lookupResult{}
return result
return
}
client := rq.h.clientGroup(rq.client)
if rq.h.config.LookupSource == "file" {
result <- lookupResult{
lookedUp: true, answer: rq.h.lookupFile.LookUp(client),
}
return result
}
answer := rq.h.geojs.LookUpWithoutWaiting(ctx, client)
refused := rq.refusedWhateverTheAnswer(now)
if !answer.Answered.IsZero() || refused || !rq.h.waitForGeoJS {
result <- lookupResult{
lookedUp: true,
skippedWait: refused && answer.Answered.IsZero(),
answer: answer,
}
return result
}
go func() {
result <- lookupResult{
lookedUp: true,
answer: rq.h.geojs.LookUp(context.WithoutCancel(ctx), client,
rq.h.config.ASNAndCountryLookupTimeout),
}
}()
return result
}
// noteLookup notes result for the log line and as the request's asn= and
// country= tags.
func (rq *request) noteLookup(result lookupResult) {
if !result.lookedUp {
return
switch {
case rq.h.config.LookupSource == "file":
rq.lookupAnswer = rq.h.lookupFile.LookUp(client)
case rq.refusedWhateverTheAnswer(now):
rq.lookupAnswer = rq.h.geojs.LookUpWithoutWaiting(ctx, client)
rq.skippedLookupWait = rq.lookupAnswer.Answered.IsZero()
default:
rq.lookupAnswer = rq.h.geojs.LookUp(ctx, client)
}
rq.lookedUp = true
rq.lookupAnswer = result.answer
rq.skippedLookupWait = result.skippedWait
rq.line.ASN = rq.lookupAnswer.ASN
rq.line.ASName = rq.lookupAnswer.ASName
rq.line.Country = rq.lookupAnswer.Country
@@ -131,29 +83,13 @@ func countryTag(country string) string {
// addLookup adds answer, an answer about a client from the lookup
// database or GeoJS, to the client's history, and to the notes of the bans
// on its netblock that have no AS number, AS name or country yet.
func (h *handler) addLookup(answer geojs.Answer) {
func (h *handler) addLookup(answer lookup.Answer) {
h.limiter.AddLookup(answer.Client, answer.Answered,
answer.ASN, answer.ASName, answer.Country)
h.ledger.AddLookup(h.netblock(answer.Client.Addr()),
answer.ASN, answer.ASName, answer.Country)
}
// geojsFailed logs a failure of GeoJS, raises a source_failure alert for it
// and counts it. askingAgainIn is how long GeoJS is left alone after it.
func (h *handler) geojsFailed(err error, askingAgainIn time.Duration) {
h.processLog.Warn("asking GeoJS failed",
"error", err.Error(), "asking_again_in", askingAgainIn.String())
h.alerts.Raise(alerts.Alert{
Event: alerts.EventSourceFailure,
Reason: "asking GeoJS failed",
Detail: map[string]any{
"source": "geojs", "error": err.Error(),
"asking_again_in": askingAgainIn.String(),
},
})
h.metrics.GeoJSFailures.Inc()
}
// setLookupHeaders sets the headers that give the client's AS number and
// country, leaving out one that is unknown.
func setLookupHeaders(header http.Header, asn, country string) {
+5 -18
View File
@@ -12,7 +12,7 @@ import (
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
@@ -138,7 +138,7 @@ func TestRequestThatWaitedForItsLookupIsCountedWhenItGoesOn(t *testing.T) {
// of lookups.json would keep it, while the first request still waits
// for GeoJS. The second request goes on at once, and is counted.
clk.advance(2500 * time.Millisecond)
server.GeoJS.Load([]geojs.Answer{{
server.GeoJS.Load([]lookup.Answer{{
Client: netip.MustParsePrefix(
fromDE + "/32",
), ASN: asnDE, ASName: asNameDE,
@@ -190,7 +190,7 @@ func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(
clk.Now,
map[string]string{
trustedProxies: trustLocalhost,
alertWebhookURL: alertWebhook,
alertWebhookURL: "https://alerts.example/sww",
rateLimitPerMinute: "1",
},
)
@@ -199,7 +199,7 @@ func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(
// The answer is kept before the requests, so GeoJS is not asked, and
// gives no answer of its own.
netblock := netip.MustParsePrefix(fromDE + "/32")
server.GeoJS.Load([]geojs.Answer{{
server.GeoJS.Load([]lookup.Answer{{
Client: netblock, ASN: asnDE, ASName: asNameDE, Country: "DE",
Answered: clk.Now(), Used: clk.Now(),
}})
@@ -249,7 +249,7 @@ func TestLookupSourceOffLooksNoClientUp(t *testing.T) {
)
// Even an answer kept from before is not used.
server.GeoJS.Load([]geojs.Answer{{
server.GeoJS.Load([]lookup.Answer{{
Client: netip.MustParsePrefix(
fromDE + "/32",
), ASN: asnDE, ASName: asNameDE,
@@ -450,8 +450,6 @@ func TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays(
// network, which would keep that clock from moving on: the request is
// handed to waf's handler, and GeoJS is one that never answers.
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
// Not the default. The exclusive list needs the answer, and the app
// is never reached.
const timeout = 3 * time.Second
@@ -497,17 +495,6 @@ func TestMain(m *testing.M) {
m.Run()
}
// geojsRequestTimeout is how long a request to GeoJS may take before it is
// abandoned.
const geojsRequestTimeout = 10 * time.Second
// waitForGeoJSRequestsToBeAbandoned sleeps until every request to the GeoJS at
// silentGeoJSURL made so far has been abandoned. A synctest bubble that asks
// it defers this: the bubble cannot end while a request waits on it.
func waitForGeoJSRequestsToBeAbandoned() {
time.Sleep(geojsRequestTimeout)
}
// silentGeoJS is the GeoJS at silentGeoJSURL.
type silentGeoJS struct{}
+11 -47
View File
@@ -1,7 +1,6 @@
package waf_test
import (
"fmt"
"io"
"net/http"
"net/http/httptest"
@@ -11,9 +10,8 @@ import (
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/waf"
)
@@ -250,7 +248,7 @@ func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
// The answers are kept before the requests, so that none waits for
// GeoJS.
server.GeoJS.Load([]geojs.Answer{
server.GeoJS.Load([]lookup.Answer{
keptAnswer(fromKP, "KP"), keptAnswer(fromDE, "DE"),
keptAnswer(fromFR, "FR"),
})
@@ -331,13 +329,13 @@ func TestMetricsByASNumberKeepTheBusiestAndCountTheRestAsOther(
// The answers are kept before the requests, so that GeoJS gives none
// of its own. Each client is in an AS of its own.
answer := func(addr, asn string) geojs.Answer {
return geojs.Answer{
answer := func(addr, asn string) lookup.Answer {
return lookup.Answer{
Client: netip.MustParsePrefix(addr + "/32"), ASN: asn,
Answered: clk.Now(), Used: clk.Now(),
}
}
server.GeoJS.Load([]geojs.Answer{
server.GeoJS.Load([]lookup.Answer{
answer(fromDE, "AS64501"), answer(fromKP, "AS64502"),
})
@@ -359,23 +357,21 @@ func TestMetricsByASNumberKeepTheBusiestAndCountTheRestAsOther(
`sww_asn_requests_total{asn="AS64502",instance="app"}`)
}
func TestGeoJSFailureIsCountedLoggedAndAlerted(t *testing.T) {
func TestMetricsCountGeoJSRequestsAndFailures(t *testing.T) {
t.Parallel()
failingGeoJS := httptest.NewServer(http.HandlerFunc(
geojs := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusServiceUnavailable)
}))
t.Cleanup(failingGeoJS.Close)
t.Cleanup(geojs.Close)
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, out, server, queue := startProxyWithAlerts(t, app.URL,
failingGeoJS.URL, time.Now,
addr, _, server := startProxyWithClock(t, app.URL, geojs.URL, time.Now,
map[string]string{
trustedProxies: trustLocalhost,
metricsToken: token,
tagDeny: denyKP,
alertWebhookURL: "https://alerts.example/waf",
})
// GeoJS fails, so the client has no country= tag, and SWW_WAF_TAG_DENY does
@@ -405,46 +401,14 @@ func TestGeoJSFailureIsCountedLoggedAndAlerted(t *testing.T) {
`sww_requests_without_geojs_answer_total{instance="app"}`,
1,
)
// The process log's line names the failure but not the address asked
// about.
logged := ""
for line := range strings.Lines(out.text()) {
if strings.Contains(line, "asking GeoJS failed") {
logged = line
}
}
if logged == "" || strings.Contains(logged, fromDE) {
t.Errorf(
"logged %q, want the failure without the address asked about",
logged,
)
}
// fmt prints a map sorted by its keys.
const want = "map[asking_again_in:1s " +
"error:GeoJS answered 503 Service Unavailable source:geojs]"
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || waiting[0].Event != alerts.EventSourceFailure ||
waiting[0].Reason != "asking GeoJS failed" ||
fmt.Sprint(waiting[0].Detail) != want {
t.Errorf(
"alerts waiting %+v, want one source_failure alert with %v",
waiting,
want,
)
}
}
// keptAnswer returns GeoJS's answer that the client at addr is in
// country, given now.
func keptAnswer(addr, country string) geojs.Answer {
func keptAnswer(addr, country string) lookup.Answer {
now := time.Now()
return geojs.Answer{
return lookup.Answer{
Client: netip.MustParsePrefix(addr + "/32"), Country: country,
Answered: now, Used: now,
}
+16 -32
View File
@@ -8,7 +8,6 @@ import (
"io"
"log/slog"
"net/http"
"net/netip"
"time"
lru "github.com/hashicorp/golang-lru/v2"
@@ -18,7 +17,6 @@ import (
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
@@ -57,7 +55,7 @@ type Params struct {
// ProcessLog receives the process's own messages.
ProcessLog *slog.Logger
// GeoJSURL is where clients' AS numbers and countries are looked up
// while SWW_WAF_LOOKUP_SOURCE is geojs, normally geojs.URL.
// while SWW_WAF_LOOKUP_SOURCE is geojs, normally lookup.URL.
GeoJSURL string
// AbuseIPDBURL is where clients are checked with AbuseIPDB while
// SWW_WAF_ABUSEIPDB_KEY is set, normally reputation.AbuseIPDBURL.
@@ -67,7 +65,7 @@ type Params struct {
LookupFile *lookup.File
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
// while SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
CoreRuleSet CoreRuleSetInspector
CoreRuleSet *coreruleset.CoreRuleSet
// Now tells the time by which requests are counted for the rate
// limits, bans are made and run out, and GeoJS's answers are kept,
// normally time.Now in UTC, the time the state files give.
@@ -85,14 +83,6 @@ type Params struct {
Alerts *alerts.Queue
}
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
type CoreRuleSetInspector interface {
Inspect(
r *http.Request, client netip.Addr, body io.Reader,
) (coreruleset.Result, []byte, error)
}
// Server is the server waf runs, with the parts whose state the state
// files keep, the lookup database, nil unless SWW_WAF_LOOKUP_SOURCE is file, the
// lists fetched from URLs, which its Run fetches, the DNSBL zones' verdicts,
@@ -103,7 +93,7 @@ type Server struct {
Ledger *bans.Ledger
Limiter *ratelimit.Limiter
GeoJS *geojs.GeoJS
GeoJS *lookup.GeoJS
Anomalies *anomaly.Counters
LookupFile *lookup.File
Lists *reputation.Lists
@@ -167,8 +157,6 @@ func newHandler(params Params) *handler {
rules: params.Rules,
coreRuleSet: params.CoreRuleSet,
alerts: params.Alerts,
besideLookupLimit: besideLookupTimeLimit,
}
if params.Config.RouteWatchURL != "" {
h.routeWatch = routewatch.New(routewatch.Params{
@@ -176,19 +164,20 @@ func newHandler(params Params) *handler {
})
}
// The country list, the headers, the tag settings that name an asn= or
// country= tag and the biased thresholds act on the answer before the
// request goes on.
h.waitForGeoJS = len(params.Config.ExclusivelyAllowedCountries) > 0 ||
params.Config.AddLookupHeaders || params.Config.TagsNeedLookup() ||
biasedThresholdsSet(params.Config)
h.geojs = geojs.New(geojs.Params{
h.geojs = lookup.New(lookup.Params{
URL: params.GeoJSURL,
Timeout: params.Config.ASNAndCountryLookupTimeout,
// The country list, the headers, the tag settings that name an asn=
// or country= tag and the biased thresholds act on the answer before
// the request goes on.
Wait: len(params.Config.ExclusivelyAllowedCountries) > 0 ||
params.Config.AddLookupHeaders || params.Config.TagsNeedLookup() ||
biasedThresholdsSet(params.Config),
Answered: h.addLookup,
Now: params.Now,
Asked: m.GeoJSRequests.Inc,
Failed: h.geojsFailed,
WentWithoutAnswer: m.RequestsWithoutGeoJSAnswer.Inc,
ProcessLog: params.ProcessLog,
Metrics: m,
Alerts: params.Alerts,
})
m.AddBansAndClients(h.ledger, h.limiter, params.Now)
m.AddRules(params.Rules)
@@ -279,7 +268,7 @@ type handler struct {
metrics *metrics.Metrics
limiter *ratelimit.Limiter
ledger *bans.Ledger
geojs *geojs.GeoJS
geojs *lookup.GeoJS
anomalies *anomaly.Counters
lookupFile *lookup.File
lists *reputation.Lists
@@ -287,13 +276,8 @@ type handler struct {
abuseIPDB *reputation.AbuseIPDB
routeWatch *routewatch.Client
rules *rules.Files
coreRuleSet CoreRuleSetInspector
coreRuleSet *coreruleset.CoreRuleSet
alerts *alerts.Queue
// waitForGeoJS is true when a setting needs each request's answer from
// GeoJS before the request goes on.
waitForGeoJS bool
// besideLookupLimit is besideLookupTimeLimit. Only tests change it.
besideLookupLimit time.Duration
// awaitingReport holds the decision on each answered question by its id
// until its report comes, and nil for a question still being handled.
awaitingReport *lru.Cache[string, *decision]
-5
View File
@@ -512,11 +512,6 @@ func newWAFWith(
Alerts: alertQueue,
})
// The race detector and a busy host can slow the Core Rule Set's
// inspections past the time limit. The tests of the limit set it back, and
// hold the Core Rule Set with a stand-in.
server.SetBesideLookupTimeLimit(time.Hour)
return server, out, alertQueue
}
+2 -2
View File
@@ -5,7 +5,7 @@ import (
"sync/atomic"
"testing"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
@@ -162,7 +162,7 @@ func TestRateLimitExemptPathsAreNeitherCountedNorRefused(t *testing.T) {
// The answers are kept before the requests, so that none waits for
// GeoJS.
server.GeoJS.Load([]geojs.Answer{
server.GeoJS.Load([]lookup.Answer{
keptAnswer(client, "DE"), keptAnswer(fromKP, "KP"),
})
+13 -47
View File
@@ -11,7 +11,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
@@ -42,7 +42,7 @@ type decision struct {
// looked up, whether or not an answer was there, and lookupAnswer is
// what the lookup gave then, the zero Answer while GeoJS had given none.
lookedUp bool
lookupAnswer geojs.Answer
lookupAnswer lookup.Answer
// skippedLookupWait is true for a request that did not wait for GeoJS and
// found no answer kept. SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT does not apply to it,
// so that not waiting never lowers its client's limits.
@@ -81,9 +81,6 @@ type request struct {
checked time.Time
// refused is what the request was refused with, nil if it was not.
refused *refusal
// findings is what the heuristics of besideLookup found of the request,
// once waf has taken it, and nil before.
findings *besideLookupFindings
// requestBytes is how many bytes of the request's body have been read.
requestBytes int64
}
@@ -180,9 +177,9 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
// check is the one place where a request can be refused. It returns nil to let
// the request through. The heuristics of checkClient decide: a request they
// refuse is answered with SWW_WAF_BAN_RESPONSE, 403 for a block rule or the Core
// Rule Set, 503 for heuristics that ran out of time, or 414 or 431 for the
// header limits. In observe mode nothing is refused, and the log line names
// what enforce mode would have done. ctx is the request's own context.
// Rule Set, or 414 or 431 for the header limits. In observe mode nothing is refused,
// and the log line names what enforce mode would have done. ctx is the
// request's own context.
func (rq *request) check(ctx context.Context) *refusal {
v := rq.checkClient(ctx)
if v == nil {
@@ -199,11 +196,6 @@ func (rq *request) check(ctx context.Context) *refusal {
case action == requestlog.ActionBlockedByBlockRule ||
action == requestlog.ActionBlockedByCoreRuleSet:
return &refusal{status: http.StatusForbidden, action: action}
case action == requestlog.ActionChecksTimedOut:
return &refusal{
status: http.StatusServiceUnavailable,
action: action,
}
case action == requestlog.ActionRequestLineTooLong:
return &refusal{status: http.StatusRequestURITooLong, action: action}
case action == requestlog.ActionRequestHeaderTooLarge:
@@ -216,16 +208,11 @@ func (rq *request) check(ctx context.Context) *refusal {
}
}
// checkClient runs the heuristics on the request, and returns the first
// verdict, or nil when none gives one. A client in SWW_WAF_ALLOW_NETS skips them
// all, and is not looked up. ctx is the request's own context.
//
// The heuristics of beforeLookup run first. Then the client's AS number and
// country are looked up, and afterLookup runs once the lookup has its result.
// While the lookup waits for GeoJS, besideLookup runs too, as runBesideLookup
// says. A lookup that needs no wait is done, and afterLookup with it, before
// besideLookup starts, so that besideLookup does no work for a request
// afterLookup refuses.
// checkClient runs the heuristics of beforeLookup on the request, then looks
// up the client's AS number and country, then runs those of afterLookup. It
// returns the first verdict, or nil when none gives one. A client in
// SWW_WAF_ALLOW_NETS skips them all, and is not looked up. ctx is the request's
// own context.
func (rq *request) checkClient(ctx context.Context) *verdict {
if isInside(rq.client, rq.h.config.AllowNets) {
return nil
@@ -238,28 +225,7 @@ func (rq *request) checkClient(ctx context.Context) *verdict {
return v
}
lookedUp := rq.startLookUp(ctx, now)
select {
case result := <-lookedUp:
v := rq.runAfterLookup(ctx, result)
if v != nil {
return v
}
return rq.runBesideLookup(ctx, nil)
default:
return rq.runBesideLookup(ctx, lookedUp)
}
}
// runAfterLookup notes result, the lookup's, and runs the heuristics of
// afterLookup. It returns the first verdict, or nil when none gives one. ctx is
// the request's own context.
func (rq *request) runAfterLookup(
ctx context.Context, result lookupResult,
) *verdict {
rq.noteLookup(result)
rq.lookUp(ctx, now)
// The lookup can wait for GeoJS, so the checks after it read the time
// again: a request counted at the time it arrived could be counted
@@ -416,9 +382,9 @@ func anomalyThresholdsSet(cfg *config.WAF) bool {
// answer about it as the request ends, and whether there is one: the
// lookup database's, which was there at once, or the one GeoJS has given
// by then, which a request does not wait for unless a setting needs it.
func (rq *request) answerAtTheEnd() (geojs.Answer, bool) {
func (rq *request) answerAtTheEnd() (lookup.Answer, bool) {
if !rq.lookedUp {
return geojs.Answer{}, false
return lookup.Answer{}, false
}
if rq.h.config.LookupSource == "file" {
+3 -15
View File
@@ -13,8 +13,8 @@ import (
"sneak.berlin/go/smallwebwaf/internal/rules"
)
// ruleFiles takes the rules of the rule files the request matched, which
// findBesideLookup found, notes their ids in the log line, adds the tag
// ruleFiles checks the request against the rules of the rule files at now,
// notes the ids of those it matches in the log line, adds the tag
// exploitsignature when it matches any, and refuses the request
// when the last rule it matches refuses it: with ActionBlockedByBlockRule for a
// block rule, and for a ban rule with the action for the ban it makes. A ban
@@ -31,7 +31,7 @@ func (ruleFiles) check(
rq *request,
now time.Time,
) *verdict {
matched := rq.findings.rules
matched := rq.h.rules.Match(rq.in)
for _, rule := range matched {
rq.line.RuleIDs = append(rq.line.RuleIDs, rule.ID)
@@ -70,15 +70,3 @@ func (ruleFiles) check(
return nil
}
}
// refusedByRule reports whether ruleFiles refuses a request that matched
// matched: whether the last of them is a block rule or a ban rule.
func refusedByRule(matched []rules.Rule) bool {
if len(matched) == 0 {
return false
}
action := matched[len(matched)-1].Action
return action == rules.ActionBlock || action == rules.ActionBan
}
+6 -17
View File
@@ -11,8 +11,9 @@ import (
"sneak.berlin/go/smallwebwaf/internal/rules"
)
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS, as trapPath
// finds it. Such a request is a clear sign of attack, as a ban rule's match is:
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS: its path as a
// path rule sees it, without the query, as sent or once percent-decoded, is one
// of them. Such a request is a clear sign of attack, as a ban rule's match is:
// it bans the client's netblock, or in observe mode raises the alert for the
// ban it would have made. The ban's notes give the path as sent, and so does
// its figure.
@@ -25,8 +26,9 @@ func (trapPaths) check(
rq *request,
now time.Time,
) *verdict {
path := rq.findings.trapPath
if path == "" {
path := rules.Path(rq.in)
if !slices.Contains(rq.h.config.TrapPaths, path) &&
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
return nil
}
@@ -37,16 +39,3 @@ func (trapPaths) check(
figures: map[string]any{"path": path},
}
}
// trapPath returns the request's path as sent, when its path as a path rule
// sees it, without the query, as sent or once percent-decoded, is in
// SWW_WAF_TRAP_PATHS, and "" otherwise.
func (rq *request) trapPath() string {
path := rules.Path(rq.in)
if !slices.Contains(rq.h.config.TrapPaths, path) &&
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
return ""
}
return path
}