waf: run the trap paths, rule files and Core Rule Set during the GeoJS wait (closes #405)
check / check (push) Failing after 1m51s
check / check (push) Failing after 1m51s
While the lookup waits for GeoJS, the trap paths, the rule files and the Core Rule Set run at the same time as the checks after the lookup. The first side to refuse answers the request; what the other side finds leaves no trace. Those three checks have 500 ms together: past it, the request is refused with 503 as checks_timed_out, with no ban and no offence. Tests hold GeoJS and the Core Rule Set with stand-ins under synctest. The test helper lifts the 500 ms limit for tests that run the real Core Rule Set, which the race detector slows past it. Model: opus-5-5
This commit was merged in pull request #408.
This commit is contained in:
1 parent
caa3079847
commit
8c9f20d346
19 files changed
+902
-84
No files matched your search
@@ -599,6 +599,11 @@ request reaches the app.
|
|||||||
16. A path under `/_sww/` is answered `404`, so that the app never sees that
|
16. A path under `/_sww/` is answered `404`, so that the app never sees that
|
||||||
prefix.
|
prefix.
|
||||||
|
|
||||||
|
Steps 12 to 14 only read the request. While the lookup waits for GeoJS, they run
|
||||||
|
at the same time as steps 6 to 11. The first of the two sides to refuse the
|
||||||
|
request ends the checks, and `waf` answers at once. Steps 12 to 14 have 500 ms
|
||||||
|
together.
|
||||||
|
|
||||||
`rproxy` takes the rest, with `waf` counting at the end:
|
`rproxy` takes the rest, with `waf` counting at the end:
|
||||||
|
|
||||||
17. The request is passed to the app, streaming.
|
17. The request is passed to the app, streaming.
|
||||||
@@ -611,9 +616,10 @@ A refused visitor is answered with `SWW_WAF_BAN_RESPONSE`. A `block` rule or a
|
|||||||
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
|
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
|
||||||
false positives; such refusals count toward `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`
|
false positives; such refusals count toward `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`
|
||||||
instead. A request over the header limits of step 3 is answered `414` for a
|
instead. A request over the header limits of step 3 is answered `414` for a
|
||||||
request line over 8 KiB and `431` otherwise. It bans nobody either. `rproxy`
|
request line over 8 KiB and `431` otherwise. It bans nobody either. Nor does a
|
||||||
itself answers `405` to a method it does not pass on, and the status for a size
|
request that steps 12 to 14 have not finished within 500 ms, which is answered
|
||||||
or time limit, such as `413` for a body too large.
|
`503`. `rproxy` itself answers `405` to a method it does not pass on, and the
|
||||||
|
status for a size or time limit, such as `413` for a body too large.
|
||||||
|
|
||||||
### Clients and bans
|
### Clients and bans
|
||||||
|
|
||||||
|
|||||||
@@ -173,7 +173,8 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
|
|||||||
`country=` tag.
|
`country=` tag.
|
||||||
|
|
||||||
Otherwise the request goes on at once. A request that ends before the answer
|
Otherwise the request goes on at once. A request that ends before the answer
|
||||||
comes is logged without it.
|
comes is logged without it. While the request waits, steps 12 to 14 run at
|
||||||
|
the same time as steps 6 to 11 (see "Answering in time").
|
||||||
|
|
||||||
7. The country list (`countries`; see "Country list").
|
7. The country list (`countries`; see "Country list").
|
||||||
8. The reputation sources (see "Reputation"), in this order:
|
8. The reputation sources (see "Reputation"), in this order:
|
||||||
@@ -2379,8 +2380,9 @@ runs no check, writes no request log line and counts nothing in the metrics.
|
|||||||
|
|
||||||
### The answer
|
### The answer
|
||||||
|
|
||||||
`waf` runs steps 3 to 16 of "What happens to a request", in that order. Every
|
`waf` runs steps 3 to 16 of "What happens to a request", in that order, apart
|
||||||
answer but the last below carries `X-Sww-Disposition`.
|
from the steps that run at the same time (see "Answering in time"). Every answer
|
||||||
|
but the last below carries `X-Sww-Disposition`.
|
||||||
|
|
||||||
- `X-Sww-Disposition: allow`, status `200`, no body. `rproxy` passes the request
|
- `X-Sww-Disposition: allow`, status `200`, no body. `rproxy` passes the request
|
||||||
to the app. While `SWW_WAF_ADD_LOOKUP_HEADERS` is `true`, the answer also
|
to the app. While `SWW_WAF_ADD_LOOKUP_HEADERS` is `true`, the answer also
|
||||||
@@ -2392,6 +2394,8 @@ answer but the last below carries `X-Sww-Disposition`.
|
|||||||
- `SWW_WAF_BAN_RESPONSE`, for a refusal;
|
- `SWW_WAF_BAN_RESPONSE`, for a refusal;
|
||||||
- `403`, for a `block` rule or the Core Rule Set;
|
- `403`, for a `block` rule or the Core Rule Set;
|
||||||
- `414` or `431`, for a request over the header limits;
|
- `414` or `431`, for a request over the header limits;
|
||||||
|
- `503`, for a request that steps 12 to 14 have not finished within 500
|
||||||
|
milliseconds;
|
||||||
- `404`, for a path under `/_sww/` that passes the checks, so that the app
|
- `404`, for a path under `/_sww/` that passes the checks, so that the app
|
||||||
never sees that prefix.
|
never sees that prefix.
|
||||||
- `X-Sww-Disposition: close`, status `200`, no body. This takes the place of a
|
- `X-Sww-Disposition: close`, status `200`, no body. This takes the place of a
|
||||||
@@ -2412,9 +2416,15 @@ The Core Rule Set inspects the body a question carries only while
|
|||||||
|
|
||||||
`rproxy` stops waiting for `waf` after `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, by
|
`rproxy` stops waiting for `waf` after `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, by
|
||||||
default 2 seconds, and `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` then
|
default 2 seconds, and `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` then
|
||||||
decides. The longest a question waits on an outside source is
|
decides. Two things in a decision take time:
|
||||||
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by default 800 milliseconds, well
|
|
||||||
within that time.
|
- the wait for GeoJS, at most `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by
|
||||||
|
default 800 milliseconds;
|
||||||
|
- the trap paths, the rule files and the Core Rule Set, which have 500
|
||||||
|
milliseconds.
|
||||||
|
|
||||||
|
They run at the same time, so `waf` answers within the longer of the two, well
|
||||||
|
within `rproxy`'s wait.
|
||||||
|
|
||||||
GeoJS is the only outside source a question can wait on. A question waits only
|
GeoJS is the only outside source a question can wait on. A question waits only
|
||||||
when step 6 says so, and for at most
|
when step 6 says so, and for at most
|
||||||
@@ -2446,10 +2456,38 @@ never wait. These can still wait for GeoJS:
|
|||||||
- a country list, which needs the answer;
|
- a country list, which needs the answer;
|
||||||
- a rate limit a percentage has lowered, which needs the answer;
|
- a rate limit a percentage has lowered, which needs the answer;
|
||||||
- a DNS blocklist or AbuseIPDB, while `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` is
|
- a DNS blocklist or AbuseIPDB, while `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` is
|
||||||
`deny`;
|
`deny`.
|
||||||
- a trap path;
|
|
||||||
- a rule file;
|
The trap paths, the rule files and the Core Rule Set, steps 12 to 14, run during
|
||||||
- the Core Rule Set.
|
that wait. Once a request has passed the bans, the lookup starts. While it waits
|
||||||
|
for GeoJS, steps 12 to 14 run at the same time as steps 6 to 11. A lookup that
|
||||||
|
needs no wait has its answer at once, and steps 7 to 11 then run before steps 12
|
||||||
|
to 14 start.
|
||||||
|
|
||||||
|
- The first of the two sides to refuse the request ends the checks. `waf`
|
||||||
|
answers at once, and does not wait for the other side. The lookup goes on, and
|
||||||
|
its answer is kept.
|
||||||
|
- What the other side finds leaves no trace. It counts nothing, bans nothing,
|
||||||
|
raises no alert, counts no metric and adds nothing to the log line.
|
||||||
|
- When both sides would refuse, the one that finishes first is answered and
|
||||||
|
logged.
|
||||||
|
- When neither refuses, step 15 runs.
|
||||||
|
|
||||||
|
Steps 12 to 14 only read the request. They have 500 milliseconds together, from
|
||||||
|
their start. The limit is fixed, with no setting. A request they have not
|
||||||
|
finished by then is refused:
|
||||||
|
|
||||||
|
- the answer is `deny`, with `503`;
|
||||||
|
- the log line's action is `checks_timed_out`, and its `heuristic` names the
|
||||||
|
step that had not finished;
|
||||||
|
- the refusal makes no ban, is no offence and does not count toward the refusal
|
||||||
|
limit;
|
||||||
|
- it holds in every `SWW_WAF_CORE_RULE_SET_MODE`;
|
||||||
|
- in `observe` mode the request is answered `allow` with `would_action`, like
|
||||||
|
every refusal.
|
||||||
|
|
||||||
|
The limit is measured on the clock. While `waf`'s host is so busy that an
|
||||||
|
ordinary request takes 500 milliseconds, that request is refused too.
|
||||||
|
|
||||||
The header limits are fixed, with no setting. A line's length is in bytes and
|
The header limits are fixed, with no setting. A line's length is in bytes and
|
||||||
does not count its line ending. `waf` refuses a request whose request line is
|
does not count its line ending. `waf` refuses a request whose request line is
|
||||||
@@ -2499,9 +2537,11 @@ body can add up to more than the body. From 1 KiB of JSON, Coraza makes at most
|
|||||||
|
|
||||||
A request can reach all of these at once. At paranoia level 4, with no rule
|
A request can reach all of these at once. At paranoia level 4, with no rule
|
||||||
switched off, the Core Rule Set takes under a second on one core of a current
|
switched off, the Core Rule Set takes under a second on one core of a current
|
||||||
server to inspect it. So `waf` answers it within the default 2 seconds, even
|
server to inspect it. `waf` refuses it if the inspection takes over 500
|
||||||
after the default 800 milliseconds of waiting for GeoJS. Enough such requests at
|
milliseconds. Coraza cannot stop an inspection part-way, so the inspection runs
|
||||||
once can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
|
to its end with nobody waiting for it. These limits keep one request from
|
||||||
|
holding a processor for more than about a second. Enough such requests at once
|
||||||
|
can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
|
||||||
|
|
||||||
### The report
|
### The report
|
||||||
|
|
||||||
|
|||||||
@@ -105,6 +105,8 @@ Every one of these lines has:
|
|||||||
`blocked_by_core_rule_set`: refused by that check;
|
`blocked_by_core_rule_set`: refused by that check;
|
||||||
- `request_line_too_long`: a request line over the header limits;
|
- `request_line_too_long`: a request line over the header limits;
|
||||||
- `request_header_too_large`: headers over the header limits;
|
- `request_header_too_large`: headers over the header limits;
|
||||||
|
- `checks_timed_out`: refused with `503` because the trap paths, the rule
|
||||||
|
files and the Core Rule Set had not finished within 500 ms;
|
||||||
- `own_endpoint`: answered at one of `waf`'s own endpoints, or `404` for a
|
- `own_endpoint`: answered at one of `waf`'s own endpoints, or `404` for a
|
||||||
path under `/_sww/`.
|
path under `/_sww/`.
|
||||||
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
||||||
@@ -134,6 +136,11 @@ Every one of these lines has:
|
|||||||
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`;
|
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`;
|
||||||
- `refusal_limit`: `window`, `count` and `limit`, of the refusal limit;
|
- `refusal_limit`: `window`, `count` and `limit`, of the refusal limit;
|
||||||
- `byte_limits`: `window`, `count` and `limit`, the count in bytes.
|
- `byte_limits`: `window`, `count` and `limit`, the count in bytes.
|
||||||
|
|
||||||
|
For a request refused as `checks_timed_out`, `heuristic` is the one of
|
||||||
|
`trap_paths`, `rule_files` and `core_rule_set` that had not finished, and
|
||||||
|
its figure is `time_limit_ms`, 500.
|
||||||
|
|
||||||
- `after_answer_heuristic`: the heuristic that banned the client after `waf`
|
- `after_answer_heuristic`: the heuristic that banned the client after `waf`
|
||||||
answered the request, in `observe` mode too: `refusal_limit` or `byte_limits`.
|
answered the request, in `observe` mode too: `refusal_limit` or `byte_limits`.
|
||||||
- `after_answer_heuristic_figures`: an object of the figures that heuristic
|
- `after_answer_heuristic_figures`: an object of the figures that heuristic
|
||||||
|
|||||||
+3
-1
@@ -435,7 +435,9 @@ A few limits are fixed rather than settings:
|
|||||||
|
|
||||||
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` is the longest a question waits on
|
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` is the longest a question waits on
|
||||||
an outside source. Keep it well under `rproxy`'s
|
an outside source. Keep it well under `rproxy`'s
|
||||||
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, as the default is. No question waits for
|
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, as the default is. The trap paths, the rule
|
||||||
|
files and the Core Rule Set run during that wait and have 500 ms, so `waf`
|
||||||
|
answers within the longer of the two times. No question waits for
|
||||||
`SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`, since the DNS blocklists and AbuseIPDB are
|
`SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`, since the DNS blocklists and AbuseIPDB are
|
||||||
asked in the background (see "Answering in time" under "`waf`" in
|
asked in the background (see "Answering in time" under "`waf`" in
|
||||||
[`SPEC.md`](../SPEC.md)).
|
[`SPEC.md`](../SPEC.md)).
|
||||||
|
|||||||
@@ -35,10 +35,11 @@ const (
|
|||||||
mostJSONRead = 1 << 10
|
mostJSONRead = 1 << 10
|
||||||
)
|
)
|
||||||
|
|
||||||
// The Core Rule Set has 1 second for one request. That is the default of
|
// waf refuses a request the Core Rule Set has not inspected within 500
|
||||||
// rproxy's SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, 2 seconds, less the default of
|
// milliseconds, but Coraza cannot stop an inspection part-way, so it runs to
|
||||||
// SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, 800 milliseconds, and less 200
|
// its end with nobody waiting for it. The test checks that an inspection ends
|
||||||
// milliseconds for waf's other checks.
|
// within about 1 second, so that a visitor cannot keep a processor busy for
|
||||||
|
// longer with one request.
|
||||||
//
|
//
|
||||||
// A busy host slows all work, so the test does not compare an inspection with
|
// A busy host slows all work, so the test does not compare an inspection with
|
||||||
// 1 second. It compares it with referenceTime, timed on the same thread just
|
// 1 second. It compares it with referenceTime, timed on the same thread just
|
||||||
|
|||||||
@@ -61,6 +61,12 @@ const (
|
|||||||
// ActionCountryDenied is a request refused because its client's country
|
// ActionCountryDenied is a request refused because its client's country
|
||||||
// is not in SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
// is not in SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
||||||
ActionCountryDenied = "country_denied"
|
ActionCountryDenied = "country_denied"
|
||||||
|
|
||||||
|
// ActionChecksTimedOut is a request refused because the trap paths, the
|
||||||
|
// rule files and the Core Rule Set had not finished with it within 500
|
||||||
|
// milliseconds.
|
||||||
|
ActionChecksTimedOut = "checks_timed_out"
|
||||||
|
|
||||||
// ActionOwnEndpoint is a request waf answered at one of its own
|
// ActionOwnEndpoint is a request waf answered at one of its own
|
||||||
// endpoints, or a question about a path under /_sww/.
|
// endpoints, or a question about a path under /_sww/.
|
||||||
ActionOwnEndpoint = "own_endpoint"
|
ActionOwnEndpoint = "own_endpoint"
|
||||||
@@ -149,7 +155,8 @@ type Line struct {
|
|||||||
// ActionTemporarilyBanned, ActionPermanentlyBanned, ActionCountryDenied,
|
// ActionTemporarilyBanned, ActionPermanentlyBanned, ActionCountryDenied,
|
||||||
// ActionClientIsRateLimited, ActionBlockedByBlockRule,
|
// ActionClientIsRateLimited, ActionBlockedByBlockRule,
|
||||||
// ActionBlockedByCoreRuleSet, or, in waf, ActionRequestLineTooLong or
|
// ActionBlockedByCoreRuleSet, or, in waf, ActionRequestLineTooLong or
|
||||||
// ActionRequestHeaderTooLarge for the header limits.
|
// ActionRequestHeaderTooLarge for the header limits, or
|
||||||
|
// ActionChecksTimedOut.
|
||||||
WouldAction string `json:"would_action,omitempty"`
|
WouldAction string `json:"would_action,omitempty"`
|
||||||
// Heuristic is the name of the heuristic that refused the request, in
|
// Heuristic is the name of the heuristic that refused the request, in
|
||||||
// observe mode too, and HeuristicFigures are the figures it acted on, by
|
// observe mode too, and HeuristicFigures are the figures it acted on, by
|
||||||
|
|||||||
@@ -19,6 +19,8 @@ const (
|
|||||||
alertMaxPerHour = "SWW_WAF_ALERT_MAX_PER_HOUR"
|
alertMaxPerHour = "SWW_WAF_ALERT_MAX_PER_HOUR"
|
||||||
// alertInstance is the instance every alert of these tests gives.
|
// alertInstance is the instance every alert of these tests gives.
|
||||||
alertInstance = "host1/blog"
|
alertInstance = "host1/blog"
|
||||||
|
// alertWebhook is where the tests send alerts.
|
||||||
|
alertWebhook = "https://alerts.example/sww"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
|
func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
|
||||||
@@ -270,7 +272,7 @@ func startAppWithAlerts(
|
|||||||
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)}
|
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)}
|
||||||
settings := map[string]string{
|
settings := map[string]string{
|
||||||
trustedProxies: trustLocalhost,
|
trustedProxies: trustLocalhost,
|
||||||
alertWebhookURL: "https://alerts.example/sww",
|
alertWebhookURL: alertWebhook,
|
||||||
instanceName: alertInstance,
|
instanceName: alertInstance,
|
||||||
}
|
}
|
||||||
maps.Copy(settings, env)
|
maps.Copy(settings, env)
|
||||||
|
|||||||
@@ -0,0 +1,172 @@
|
|||||||
|
package waf
|
||||||
|
|
||||||
|
// The heuristics of besideLookup, and how they run beside the lookup.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"sync/atomic"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
|
)
|
||||||
|
|
||||||
|
// besideLookupTimeLimit is how long the heuristics of besideLookup have for a
|
||||||
|
// request, from when they start. They only read the request, and no request
|
||||||
|
// should take them that long. A request they have not finished by then is
|
||||||
|
// refused with 503, as ActionChecksTimedOut.
|
||||||
|
const besideLookupTimeLimit = 500 * time.Millisecond
|
||||||
|
|
||||||
|
// besideLookupFindings is what the heuristics of besideLookup found of a
|
||||||
|
// request, before any of it is applied to the request.
|
||||||
|
type besideLookupFindings struct {
|
||||||
|
// trapPath is the request's path when it is in SWW_WAF_TRAP_PATHS, and ""
|
||||||
|
// otherwise.
|
||||||
|
trapPath string
|
||||||
|
// rules are the rules of the rule files the request matched.
|
||||||
|
rules []rules.Rule
|
||||||
|
// inspected is true when the Core Rule Set inspected the request.
|
||||||
|
// inspection, read and inspectionTime are then what it found, what it read
|
||||||
|
// of the body, and how long it took.
|
||||||
|
inspected bool
|
||||||
|
inspection coreruleset.Result
|
||||||
|
read []byte
|
||||||
|
inspectionTime time.Duration
|
||||||
|
// refuses is true when one of the heuristics refuses the request, or would
|
||||||
|
// in observe mode. Those after it did no work.
|
||||||
|
refuses bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// runBesideLookup runs the heuristics of besideLookup while the lookup waits
|
||||||
|
// for GeoJS, and returns the first verdict, or nil when none gives one.
|
||||||
|
// lookedUp is the channel the lookup's result comes on, or nil once afterLookup
|
||||||
|
// has let the request through.
|
||||||
|
//
|
||||||
|
// The first of the two to refuse the request ends the checks: waf answers at
|
||||||
|
// once, and what the other found is dropped. besideLookup has
|
||||||
|
// besideLookupTimeLimit; past it, the request is refused as
|
||||||
|
// ActionChecksTimedOut. Once both have let the request through, the heuristics
|
||||||
|
// of afterBesideLookup run. ctx is the request's own context.
|
||||||
|
func (rq *request) runBesideLookup(
|
||||||
|
ctx context.Context, lookedUp <-chan lookupResult,
|
||||||
|
) *verdict {
|
||||||
|
var running atomic.Pointer[string]
|
||||||
|
|
||||||
|
running.Store(new(trapPaths{}.name()))
|
||||||
|
|
||||||
|
found := make(chan *besideLookupFindings, 1)
|
||||||
|
|
||||||
|
go func() { found <- rq.findBesideLookup(&running) }()
|
||||||
|
|
||||||
|
timeUp := time.NewTimer(rq.h.besideLookupLimit)
|
||||||
|
defer timeUp.Stop()
|
||||||
|
|
||||||
|
var findings *besideLookupFindings
|
||||||
|
|
||||||
|
for findings == nil || lookedUp != nil {
|
||||||
|
select {
|
||||||
|
case result := <-lookedUp:
|
||||||
|
lookedUp = nil
|
||||||
|
|
||||||
|
v := rq.runAfterLookup(ctx, result)
|
||||||
|
if v != nil {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
case findings = <-found:
|
||||||
|
found = nil
|
||||||
|
|
||||||
|
timeUp.Stop()
|
||||||
|
|
||||||
|
if findings.refuses {
|
||||||
|
return rq.applyFindings(ctx, findings)
|
||||||
|
}
|
||||||
|
case <-timeUp.C:
|
||||||
|
return rq.timedOut(*running.Load())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
v := rq.applyFindings(ctx, findings)
|
||||||
|
if v != nil {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
return rq.runHeuristics(ctx, afterBesideLookup(), rq.h.now())
|
||||||
|
}
|
||||||
|
|
||||||
|
// findBesideLookup does the work of the heuristics of besideLookup on the
|
||||||
|
// request, in their order, until one refuses it, and returns what they found.
|
||||||
|
// It changes nothing of the request or of what waf keeps, so that it can run
|
||||||
|
// while the lookup and afterLookup do. running holds the name of the heuristic
|
||||||
|
// at work, the first one's when findBesideLookup is called.
|
||||||
|
func (rq *request) findBesideLookup(
|
||||||
|
running *atomic.Pointer[string],
|
||||||
|
) *besideLookupFindings {
|
||||||
|
findings := &besideLookupFindings{}
|
||||||
|
|
||||||
|
findings.trapPath = rq.trapPath()
|
||||||
|
if findings.trapPath != "" {
|
||||||
|
findings.refuses = true
|
||||||
|
|
||||||
|
return findings
|
||||||
|
}
|
||||||
|
|
||||||
|
running.Store(new(ruleFiles{}.name()))
|
||||||
|
|
||||||
|
findings.rules = rq.h.rules.Match(rq.in)
|
||||||
|
if refusedByRule(findings.rules) {
|
||||||
|
findings.refuses = true
|
||||||
|
|
||||||
|
return findings
|
||||||
|
}
|
||||||
|
|
||||||
|
running.Store(new(coreRuleSet{}.name()))
|
||||||
|
|
||||||
|
cfg := rq.h.config
|
||||||
|
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff ||
|
||||||
|
pathExempt(rq.in, cfg.CoreRuleSetExemptPaths) {
|
||||||
|
return findings
|
||||||
|
}
|
||||||
|
|
||||||
|
start := time.Now()
|
||||||
|
// A question's body has been read into memory, so that reading it here
|
||||||
|
// cannot fail.
|
||||||
|
findings.inspection, findings.read, _ = rq.h.coreRuleSet.Inspect(
|
||||||
|
rq.in, rq.client, rq.in.Body,
|
||||||
|
)
|
||||||
|
findings.inspectionTime = time.Since(start)
|
||||||
|
findings.inspected = true
|
||||||
|
findings.refuses = rq.refusedByCoreRuleSet(findings.inspection)
|
||||||
|
|
||||||
|
return findings
|
||||||
|
}
|
||||||
|
|
||||||
|
// applyFindings applies findings, what findBesideLookup found, to the
|
||||||
|
// request: it runs the heuristics of besideLookup, which read them, at the
|
||||||
|
// time it is called, and returns the first verdict, or nil when none gives
|
||||||
|
// one.
|
||||||
|
func (rq *request) applyFindings(
|
||||||
|
ctx context.Context, findings *besideLookupFindings,
|
||||||
|
) *verdict {
|
||||||
|
rq.findings = findings
|
||||||
|
|
||||||
|
return rq.runHeuristics(ctx, besideLookup(), rq.h.now())
|
||||||
|
}
|
||||||
|
|
||||||
|
// timedOut refuses the request for the heuristics of besideLookup not having
|
||||||
|
// finished with it within besideLookupTimeLimit. running is the name of the
|
||||||
|
// one at work, which the log line's heuristic gives. The refusal makes no ban
|
||||||
|
// and is no offence.
|
||||||
|
func (rq *request) timedOut(running string) *verdict {
|
||||||
|
v := &verdict{
|
||||||
|
action: requestlog.ActionChecksTimedOut,
|
||||||
|
figures: map[string]any{
|
||||||
|
"time_limit_ms": rq.h.besideLookupLimit.Milliseconds(),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
rq.line.Heuristic = running
|
||||||
|
rq.line.HeuristicFigures = v.figures
|
||||||
|
|
||||||
|
return v
|
||||||
|
}
|
||||||
@@ -0,0 +1,426 @@
|
|||||||
|
package waf_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io"
|
||||||
|
"maps"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/netip"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"testing/synctest"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus/testutil"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||||
|
)
|
||||||
|
|
||||||
|
// checksTimeLimit is how long the trap paths, the rule files and the Core Rule
|
||||||
|
// Set have for a request, as SPEC.md gives it, and checksTimedOut the status of
|
||||||
|
// a request they have not finished by then.
|
||||||
|
const (
|
||||||
|
checksTimeLimit = 500 * time.Millisecond
|
||||||
|
checksTimedOut = http.StatusServiceUnavailable
|
||||||
|
)
|
||||||
|
|
||||||
|
// heldCoreRuleSet is a stand-in for the Core Rule Set: each inspection takes
|
||||||
|
// took, then finds result. asked counts the inspections.
|
||||||
|
type heldCoreRuleSet struct {
|
||||||
|
took time.Duration
|
||||||
|
result coreruleset.Result
|
||||||
|
asked atomic.Int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// Inspect inspects a request as the stand-in does, and reads all of body.
|
||||||
|
func (c *heldCoreRuleSet) Inspect(
|
||||||
|
_ *http.Request, _ netip.Addr, body io.Reader,
|
||||||
|
) (coreruleset.Result, []byte, error) {
|
||||||
|
c.asked.Add(1)
|
||||||
|
time.Sleep(c.took)
|
||||||
|
|
||||||
|
read, err := io.ReadAll(body)
|
||||||
|
|
||||||
|
return c.result, read, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// coreRuleSetMatch returns what the stand-in finds of an attack: rule 942100,
|
||||||
|
// with a score of 5, the default SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD.
|
||||||
|
func coreRuleSetMatch() coreruleset.Result {
|
||||||
|
return coreruleset.Result{RuleIDs: []int{942100}, Score: 5}
|
||||||
|
}
|
||||||
|
|
||||||
|
// newWAFHoldingTheCoreRuleSet makes a waf, to be asked in a synctest bubble,
|
||||||
|
// with crs in place of the Core Rule Set, in block mode. GeoJS never answers,
|
||||||
|
// a setting needs its answer, alerts go to a webhook, and everything else is
|
||||||
|
// at its default but for env. No request to it may use the network.
|
||||||
|
func newWAFHoldingTheCoreRuleSet(
|
||||||
|
t *testing.T, crs *heldCoreRuleSet, env map[string]string,
|
||||||
|
) (*waf.Server, *output, *alerts.Queue) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
settings := map[string]string{
|
||||||
|
tagDeny: denyKP, coreRuleSetMode: block,
|
||||||
|
alertWebhookURL: alertWebhook,
|
||||||
|
}
|
||||||
|
maps.Copy(settings, env)
|
||||||
|
|
||||||
|
server, out, queue := newWAFWith(t, func(params waf.Params) *waf.Server {
|
||||||
|
params.CoreRuleSet = crs
|
||||||
|
|
||||||
|
return waf.New(params)
|
||||||
|
}, silentGeoJSURL, time.Now, settings)
|
||||||
|
server.SetBesideLookupTimeLimit(waf.BesideLookupTimeLimit)
|
||||||
|
|
||||||
|
return server, out, queue
|
||||||
|
}
|
||||||
|
|
||||||
|
// lookupWait is SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS for wait.
|
||||||
|
func lookupWait(wait time.Duration) string {
|
||||||
|
return strconv.FormatInt(wait.Milliseconds(), 10)
|
||||||
|
}
|
||||||
|
|
||||||
|
// wantAnswer checks the status of an answer and how long it took.
|
||||||
|
func wantAnswer(
|
||||||
|
t *testing.T, status int, took time.Duration,
|
||||||
|
wantStatus int, wantTook time.Duration, out *output,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if status != wantStatus || took != wantTook {
|
||||||
|
t.Errorf(
|
||||||
|
"answered %d after %s, want %d after %s; output:\n%s",
|
||||||
|
status, took, wantStatus, wantTook, out.text(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// wantNoCoreRuleSetTrace checks that what the Core Rule Set found of a request
|
||||||
|
// left no trace: line has none of its fields, and server has counted no match
|
||||||
|
// of it and raised no alert for one.
|
||||||
|
func wantNoCoreRuleSetTrace(
|
||||||
|
t *testing.T, server *waf.Server, queue *alerts.Queue, line logLine,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if line.DurationCoreRuleSetMS != nil || line.CoreRuleSetScore != nil ||
|
||||||
|
line.CoreRuleSetRuleIDs != nil {
|
||||||
|
t.Errorf(
|
||||||
|
"log line has Core Rule Set time %v, score %v and rule ids %v, want none",
|
||||||
|
line.DurationCoreRuleSetMS,
|
||||||
|
line.CoreRuleSetScore,
|
||||||
|
line.CoreRuleSetRuleIDs,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
recorder := httptest.NewRecorder()
|
||||||
|
server.Metrics.ServeHTTP(
|
||||||
|
recorder,
|
||||||
|
httptest.NewRequestWithContext(
|
||||||
|
t.Context(),
|
||||||
|
http.MethodGet,
|
||||||
|
waf.MetricsPath,
|
||||||
|
http.NoBody,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
if strings.Contains(
|
||||||
|
recorder.Body.String(),
|
||||||
|
"sww_core_rule_set_matches_total{",
|
||||||
|
) {
|
||||||
|
t.Errorf("a Core Rule Set match is counted")
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, alert := range queue.Snapshot().Waiting[alerts.DestinationWebhook] {
|
||||||
|
if alert.Event == alerts.EventCoreRuleSetMatch {
|
||||||
|
t.Errorf("alert raised: %+v", alert)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTheLookupAndTheCoreRuleSetTakeTheLongerOfTheirTimesNotTheirSum(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
inspection := checksTimeLimit - time.Millisecond
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
// wait is how long the request waits for GeoJS.
|
||||||
|
wait time.Duration
|
||||||
|
}{
|
||||||
|
{"the lookup waits longer", defaultLookupWait},
|
||||||
|
{"the inspection takes longer", inspection / 2},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := &heldCoreRuleSet{took: inspection}
|
||||||
|
server, out, _ := newWAFHoldingTheCoreRuleSet(
|
||||||
|
t,
|
||||||
|
crs,
|
||||||
|
map[string]string{
|
||||||
|
asnAndCountryLookupTimeout: lookupWait(tc.wait),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
defer waitForGeoJSRequestsToBeAbandoned()
|
||||||
|
|
||||||
|
status, took := askDirectly(
|
||||||
|
t,
|
||||||
|
server,
|
||||||
|
func(*http.Request) {},
|
||||||
|
)
|
||||||
|
|
||||||
|
wantAnswer(t, status, took, http.StatusOK,
|
||||||
|
max(tc.wait, inspection), out)
|
||||||
|
wantLine(t, out.requestLine(t), http.StatusOK,
|
||||||
|
requestlog.ActionForward)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// firstToRefuse is a case of
|
||||||
|
// TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered: a request
|
||||||
|
// under env, whose inspection by the Core Rule Set takes inspection, refused as
|
||||||
|
// action after took.
|
||||||
|
type firstToRefuse struct {
|
||||||
|
name string
|
||||||
|
env map[string]string
|
||||||
|
inspection time.Duration
|
||||||
|
took time.Duration
|
||||||
|
action string
|
||||||
|
}
|
||||||
|
|
||||||
|
// firstToRefuseCases are the cases of
|
||||||
|
// TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered.
|
||||||
|
func firstToRefuseCases() []firstToRefuse {
|
||||||
|
return []firstToRefuse{
|
||||||
|
{
|
||||||
|
name: "a trap path, before GeoJS answers",
|
||||||
|
env: map[string]string{trapPaths: "/", allowedCountries: "DE"},
|
||||||
|
took: 0, action: requestlog.ActionTemporarilyBanned,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "the Core Rule Set, before GeoJS answers",
|
||||||
|
env: map[string]string{allowedCountries: "DE"},
|
||||||
|
inspection: 100 * time.Millisecond,
|
||||||
|
took: 100 * time.Millisecond, action: requestlog.ActionBlockedByCoreRuleSet,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "the country list, while the Core Rule Set inspects",
|
||||||
|
env: map[string]string{
|
||||||
|
allowedCountries: "DE", asnAndCountryLookupTimeout: "100",
|
||||||
|
},
|
||||||
|
inspection: 400 * time.Millisecond,
|
||||||
|
took: 100 * time.Millisecond, action: requestlog.ActionCountryDenied,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "the rate limits, while the Core Rule Set inspects",
|
||||||
|
env: map[string]string{
|
||||||
|
rateLimitPerMinute: "2", unknownCountryLimitPercent: "50",
|
||||||
|
asnAndCountryLookupTimeout: "100",
|
||||||
|
},
|
||||||
|
inspection: 400 * time.Millisecond,
|
||||||
|
took: 100 * time.Millisecond, action: requestlog.ActionClientIsRateLimited,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const forbidden = http.StatusForbidden
|
||||||
|
|
||||||
|
for _, tc := range firstToRefuseCases() {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := &heldCoreRuleSet{
|
||||||
|
took: tc.inspection,
|
||||||
|
result: coreRuleSetMatch(),
|
||||||
|
}
|
||||||
|
server, out, queue := newWAFHoldingTheCoreRuleSet(t, crs, tc.env)
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
defer waitForGeoJSRequestsToBeAbandoned()
|
||||||
|
|
||||||
|
// Under the rate limits of the last case, the client's next
|
||||||
|
// request takes it over a limit only at the percentage of an
|
||||||
|
// unknown country, so it waits for GeoJS.
|
||||||
|
countClient(t, server, time.Now())
|
||||||
|
|
||||||
|
status, took := askDirectly(
|
||||||
|
t,
|
||||||
|
server,
|
||||||
|
func(*http.Request) {},
|
||||||
|
)
|
||||||
|
|
||||||
|
wantAnswer(t, status, took, forbidden, tc.took, out)
|
||||||
|
|
||||||
|
line := out.requestLine(t)
|
||||||
|
wantLine(t, line, forbidden, tc.action)
|
||||||
|
|
||||||
|
if tc.action == requestlog.ActionBlockedByCoreRuleSet {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if tc.action != requestlog.ActionClientIsRateLimited &&
|
||||||
|
line.Counts != (ratelimit.Counts{}) {
|
||||||
|
t.Errorf(
|
||||||
|
"the rate limits counted the request: %+v",
|
||||||
|
line.Counts,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// What the Core Rule Set finds once the request is answered is
|
||||||
|
// dropped.
|
||||||
|
time.Sleep(tc.inspection)
|
||||||
|
wantNoCoreRuleSetTrace(t, server, queue, line)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestARequestRefusedBeforeTheLookupStartsNoInspection(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := &heldCoreRuleSet{took: time.Second}
|
||||||
|
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs, nil)
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
banClient(t, server, time.Now())
|
||||||
|
|
||||||
|
status, took := askDirectly(t, server, func(*http.Request) {})
|
||||||
|
|
||||||
|
wantAnswer(t, status, took, http.StatusForbidden, 0, out)
|
||||||
|
|
||||||
|
requests := testutil.ToFloat64(server.Metrics.GeoJSRequests)
|
||||||
|
if asked := crs.asked.Load(); asked != 0 || requests != 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"%d inspections and %v requests to GeoJS, want none",
|
||||||
|
asked,
|
||||||
|
requests,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAClientInTheAllowNetsIsAllowedAtOnceAndNothingElseStarts(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := &heldCoreRuleSet{took: time.Second, result: coreRuleSetMatch()}
|
||||||
|
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
|
||||||
|
map[string]string{allowNets: client, allowedCountries: "DE"})
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
status, took := askDirectly(t, server, func(*http.Request) {})
|
||||||
|
|
||||||
|
wantAnswer(t, status, took, http.StatusOK, 0, out)
|
||||||
|
|
||||||
|
requests := testutil.ToFloat64(server.Metrics.GeoJSRequests)
|
||||||
|
if asked := crs.asked.Load(); asked != 0 || requests != 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"%d inspections and %v requests to GeoJS, want none",
|
||||||
|
asked,
|
||||||
|
requests,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestARequestTheCoreRuleSetHasNotFinishedWithIn500MillisecondsIsRefused(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := &heldCoreRuleSet{took: checksTimeLimit + time.Second}
|
||||||
|
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
|
||||||
|
map[string]string{refusalsLimitPerMinute: "1"})
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
defer waitForGeoJSRequestsToBeAbandoned()
|
||||||
|
|
||||||
|
// Were it counted toward the refusal limit, the second refusal would
|
||||||
|
// ban the client, and the third request would be refused for the ban.
|
||||||
|
for i := range 3 {
|
||||||
|
status, took := askDirectly(t, server, func(req *http.Request) {
|
||||||
|
req.Header.Set(requestIDHeader, "id-"+strconv.Itoa(i))
|
||||||
|
})
|
||||||
|
|
||||||
|
wantAnswer(t, status, took, checksTimedOut, checksTimeLimit, out)
|
||||||
|
|
||||||
|
line := out.requestLines(t, i+1)[i]
|
||||||
|
wantLine(
|
||||||
|
t,
|
||||||
|
line,
|
||||||
|
checksTimedOut,
|
||||||
|
requestlog.ActionChecksTimedOut,
|
||||||
|
)
|
||||||
|
wantHeuristic(t, line, "core_rule_set", `{"time_limit_ms": 500}`)
|
||||||
|
|
||||||
|
if line.Offence != "" || line.BanExpires != "" {
|
||||||
|
t.Errorf(
|
||||||
|
"log line has offence %q and ban_expires %q, want neither",
|
||||||
|
line.Offence,
|
||||||
|
line.BanExpires,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if bans := server.Ledger.Snapshot(); len(bans) != 0 {
|
||||||
|
t.Errorf("bans %+v, want none", bans)
|
||||||
|
}
|
||||||
|
|
||||||
|
offences := historyOf(t, server, client).Offences
|
||||||
|
if offences != (ratelimit.Offences{}) {
|
||||||
|
t.Errorf("history counts the offences %+v, want none", offences)
|
||||||
|
}
|
||||||
|
|
||||||
|
time.Sleep(crs.took)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestObserveModeAllowsARequestTheCoreRuleSetHasNotFinishedWith(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := &heldCoreRuleSet{took: checksTimeLimit + time.Second}
|
||||||
|
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
|
||||||
|
map[string]string{mode: observe})
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
defer waitForGeoJSRequestsToBeAbandoned()
|
||||||
|
|
||||||
|
status, took := askDirectly(t, server, func(*http.Request) {})
|
||||||
|
|
||||||
|
wantAnswer(t, status, took, http.StatusOK, checksTimeLimit, out)
|
||||||
|
|
||||||
|
line := out.requestLine(t)
|
||||||
|
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
|
if line.WouldAction != requestlog.ActionChecksTimedOut {
|
||||||
|
t.Errorf(
|
||||||
|
"log line has would_action %q, want %q",
|
||||||
|
line.WouldAction,
|
||||||
|
requestlog.ActionChecksTimedOut,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
time.Sleep(crs.took)
|
||||||
|
})
|
||||||
|
}
|
||||||
+23
-15
@@ -13,10 +13,11 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
// coreRuleSet inspects the request with the Core Rule Set, unless
|
// coreRuleSet takes what the Core Rule Set found of the request, which
|
||||||
// SWW_WAF_CORE_RULE_SET_MODE is off or SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS
|
// findBesideLookup inspected with it unless SWW_WAF_CORE_RULE_SET_MODE is off
|
||||||
// exempts its path, as pathExempt decides, and notes the rules it matched and
|
// or SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS exempts its path, as pathExempt
|
||||||
// its score in the log line, and the rules in the metrics. A score at or over
|
// decides. It notes the rules the request matched and its score in the log
|
||||||
|
// line, and the rules in the metrics. A score at or over
|
||||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD is a match: it adds the tag
|
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD is a match: it adds the tag
|
||||||
// exploitsignature, raises the core_rule_set_match alert, and in block mode
|
// exploitsignature, raises the core_rule_set_match alert, and in block mode
|
||||||
// refuses the request with 403, which is an offence its client's history
|
// refuses the request with 403, which is an offence its client's history
|
||||||
@@ -30,21 +31,17 @@ func (coreRuleSet) check(
|
|||||||
rq *request,
|
rq *request,
|
||||||
_ time.Time,
|
_ time.Time,
|
||||||
) *verdict {
|
) *verdict {
|
||||||
cfg := rq.h.config
|
findings := rq.findings
|
||||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff ||
|
if !findings.inspected {
|
||||||
pathExempt(rq.in, cfg.CoreRuleSetExemptPaths) {
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
start := time.Now()
|
cfg := rq.h.config
|
||||||
|
result := findings.inspection
|
||||||
// A question's body has been read into memory, so that reading it here
|
rq.requestBytes = int64(len(findings.read))
|
||||||
// cannot fail.
|
|
||||||
result, read, _ := rq.h.coreRuleSet.Inspect(rq.in, rq.client, rq.in.Body)
|
|
||||||
rq.requestBytes = int64(len(read))
|
|
||||||
|
|
||||||
rq.line.DurationCoreRuleSetMS = new(
|
rq.line.DurationCoreRuleSetMS = new(
|
||||||
requestlog.Milliseconds(time.Since(start)),
|
requestlog.Milliseconds(findings.inspectionTime),
|
||||||
)
|
)
|
||||||
rq.line.CoreRuleSetRuleIDs = result.RuleIDs
|
rq.line.CoreRuleSetRuleIDs = result.RuleIDs
|
||||||
rq.line.CoreRuleSetScore = &result.Score
|
rq.line.CoreRuleSetScore = &result.Score
|
||||||
@@ -61,7 +58,7 @@ func (coreRuleSet) check(
|
|||||||
rq.addTag(config.ExploitSignatureTag)
|
rq.addTag(config.ExploitSignatureTag)
|
||||||
rq.alertCoreRuleSetMatch(result)
|
rq.alertCoreRuleSetMatch(result)
|
||||||
|
|
||||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeDetect {
|
if !rq.refusedByCoreRuleSet(result) {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -76,6 +73,17 @@ func (coreRuleSet) check(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// refusedByCoreRuleSet reports whether coreRuleSet refuses a request the Core
|
||||||
|
// Rule Set found result of: whether the score is at or over
|
||||||
|
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD in block mode.
|
||||||
|
func (rq *request) refusedByCoreRuleSet(result coreruleset.Result) bool {
|
||||||
|
cfg := rq.h.config
|
||||||
|
threshold := cfg.CoreRuleSetScoreThreshold
|
||||||
|
|
||||||
|
return cfg.CoreRuleSetMode == config.CoreRuleSetModeBlock &&
|
||||||
|
threshold != 0 && result.Score >= threshold
|
||||||
|
}
|
||||||
|
|
||||||
// alertCoreRuleSetMatch raises the core_rule_set_match alert for the request,
|
// alertCoreRuleSetMatch raises the core_rule_set_match alert for the request,
|
||||||
// which the Core Rule Set scored at result, at or over
|
// which the Core Rule Set scored at result, at or over
|
||||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD. Its detail gives the rule ids, the
|
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD. Its detail gives the rule ids, the
|
||||||
|
|||||||
@@ -0,0 +1,13 @@
|
|||||||
|
package waf
|
||||||
|
|
||||||
|
import "time"
|
||||||
|
|
||||||
|
// BesideLookupTimeLimit is besideLookupTimeLimit.
|
||||||
|
const BesideLookupTimeLimit = besideLookupTimeLimit
|
||||||
|
|
||||||
|
// SetBesideLookupTimeLimit gives the heuristics of besideLookup limit on s in
|
||||||
|
// place of besideLookupTimeLimit.
|
||||||
|
func (s *Server) SetBesideLookupTimeLimit(limit time.Duration) {
|
||||||
|
h := s.Handler.(*handler) //nolint:forcetypeassert // New makes it one
|
||||||
|
h.besideLookupLimit = limit
|
||||||
|
}
|
||||||
@@ -54,8 +54,6 @@ func beforeLookup() []heuristic {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// afterLookup are the heuristics that run once the lookup has been made.
|
// afterLookup are the heuristics that run once the lookup has been made.
|
||||||
// tag_deny runs twice: before the rate limits, on the tags known by then, and
|
|
||||||
// last, on the tag rule_files and core_rule_set can add.
|
|
||||||
func afterLookup() []heuristic {
|
func afterLookup() []heuristic {
|
||||||
return []heuristic{
|
return []heuristic{
|
||||||
countries{},
|
countries{},
|
||||||
@@ -64,9 +62,26 @@ func afterLookup() []heuristic {
|
|||||||
abuseIPDB{},
|
abuseIPDB{},
|
||||||
tagDeny{},
|
tagDeny{},
|
||||||
rateLimits{},
|
rateLimits{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// besideLookup are the heuristics that only read the request. Their work runs
|
||||||
|
// beside the lookup and afterLookup, in findBesideLookup, which changes nothing
|
||||||
|
// of the request. Their check then applies what that work found, and runs only
|
||||||
|
// once waf takes it.
|
||||||
|
func besideLookup() []heuristic {
|
||||||
|
return []heuristic{
|
||||||
trapPaths{},
|
trapPaths{},
|
||||||
ruleFiles{},
|
ruleFiles{},
|
||||||
coreRuleSet{},
|
coreRuleSet{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// afterBesideLookup are the heuristics that run once afterLookup and
|
||||||
|
// besideLookup have both let the request through. tag_deny runs a second time,
|
||||||
|
// on the tag rule_files and core_rule_set can add.
|
||||||
|
func afterBesideLookup() []heuristic {
|
||||||
|
return []heuristic{
|
||||||
tagDeny{},
|
tagDeny{},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -95,7 +110,7 @@ func (rq *request) runAfterAnswer(ctx context.Context) {
|
|||||||
rq.runBanningHeuristics(ctx, afterAnswer())
|
rq.runBanningHeuristics(ctx, afterAnswer())
|
||||||
}
|
}
|
||||||
|
|
||||||
// runHeuristics runs list, beforeLookup or afterLookup, on the request at now,
|
// runHeuristics runs list, one of those checkClient runs, on the request at now,
|
||||||
// and returns the first verdict, or nil when none gives one. The log line's
|
// and returns the first verdict, or nil when none gives one. The log line's
|
||||||
// heuristic and heuristic_figures name the heuristic that gave it, and give its
|
// heuristic and heuristic_figures name the heuristic that gave it, and give its
|
||||||
// figures. ctx is the request's own context.
|
// figures. ctx is the request's own context.
|
||||||
|
|||||||
+66
-22
@@ -21,36 +21,80 @@ const (
|
|||||||
countryHeader = "X-Client-Country"
|
countryHeader = "X-Client-Country"
|
||||||
)
|
)
|
||||||
|
|
||||||
// lookUp looks up the client's AS number and country, in the lookup
|
// lookupResult is what the lookup gave for a request: whether its client was
|
||||||
// database or through GeoJS, and notes them for the log line and as the
|
// looked up, the answer, and whether the request skipped the wait for GeoJS.
|
||||||
// request's asn= and country= tags, unless
|
type lookupResult struct {
|
||||||
// SWW_WAF_LOOKUP_SOURCE is off or the client is on a private, loopback or
|
lookedUp, skippedWait bool
|
||||||
// link-local address, which no lookup can place. The lookup database
|
answer geojs.Answer
|
||||||
// answers at once. With GeoJS, while a setting needs the answer, such as a
|
}
|
||||||
// country list or a biased threshold, a new client's request waits for it up
|
|
||||||
// to SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, unless refusedWhateverTheAnswer
|
// startLookUp starts looking up the client's AS number and country, in the
|
||||||
// reports true. ctx is the request's own context.
|
// lookup database or through GeoJS, unless SWW_WAF_LOOKUP_SOURCE is off or the
|
||||||
func (rq *request) lookUp(ctx context.Context, now time.Time) {
|
// client is on a private, loopback or link-local address, which no lookup can
|
||||||
|
// place. It returns the channel the result comes on. A result that needs no
|
||||||
|
// wait is on it when startLookUp returns: the lookup database's, an answer
|
||||||
|
// GeoJS gave earlier, or none. With GeoJS, while a setting needs the answer,
|
||||||
|
// such as a country list or a biased threshold, a new client's request waits
|
||||||
|
// for it up to SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, unless
|
||||||
|
// refusedWhateverTheAnswer reports true. The wait goes on to its end even once
|
||||||
|
// the request has been answered, so that it ends as it would have for a
|
||||||
|
// request that waited. startLookUp changes nothing of the request. ctx is the
|
||||||
|
// request's own context.
|
||||||
|
func (rq *request) startLookUp(
|
||||||
|
ctx context.Context, now time.Time,
|
||||||
|
) <-chan lookupResult {
|
||||||
|
result := make(chan lookupResult, 1)
|
||||||
|
|
||||||
if rq.h.config.LookupSource == "off" || !canBePlaced(rq.client) {
|
if rq.h.config.LookupSource == "off" || !canBePlaced(rq.client) {
|
||||||
return
|
result <- lookupResult{}
|
||||||
|
|
||||||
|
return result
|
||||||
}
|
}
|
||||||
|
|
||||||
client := rq.h.clientGroup(rq.client)
|
client := rq.h.clientGroup(rq.client)
|
||||||
|
|
||||||
switch {
|
if rq.h.config.LookupSource == "file" {
|
||||||
case rq.h.config.LookupSource == "file":
|
result <- lookupResult{
|
||||||
rq.lookupAnswer = rq.h.lookupFile.LookUp(client)
|
lookedUp: true, answer: rq.h.lookupFile.LookUp(client),
|
||||||
case rq.refusedWhateverTheAnswer(now):
|
}
|
||||||
rq.lookupAnswer = rq.h.geojs.LookUpWithoutWaiting(ctx, client)
|
|
||||||
rq.skippedLookupWait = rq.lookupAnswer.Answered.IsZero()
|
return result
|
||||||
case !rq.h.waitForGeoJS:
|
}
|
||||||
rq.lookupAnswer = rq.h.geojs.LookUpWithoutWaiting(ctx, client)
|
|
||||||
default:
|
answer := rq.h.geojs.LookUpWithoutWaiting(ctx, client)
|
||||||
rq.lookupAnswer = rq.h.geojs.LookUp(ctx, client,
|
refused := rq.refusedWhateverTheAnswer(now)
|
||||||
rq.h.config.ASNAndCountryLookupTimeout)
|
|
||||||
|
if !answer.Answered.IsZero() || refused || !rq.h.waitForGeoJS {
|
||||||
|
result <- lookupResult{
|
||||||
|
lookedUp: true,
|
||||||
|
skippedWait: refused && answer.Answered.IsZero(),
|
||||||
|
answer: answer,
|
||||||
|
}
|
||||||
|
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
result <- lookupResult{
|
||||||
|
lookedUp: true,
|
||||||
|
answer: rq.h.geojs.LookUp(context.WithoutCancel(ctx), client,
|
||||||
|
rq.h.config.ASNAndCountryLookupTimeout),
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
// noteLookup notes result for the log line and as the request's asn= and
|
||||||
|
// country= tags.
|
||||||
|
func (rq *request) noteLookup(result lookupResult) {
|
||||||
|
if !result.lookedUp {
|
||||||
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.lookedUp = true
|
rq.lookedUp = true
|
||||||
|
rq.lookupAnswer = result.answer
|
||||||
|
rq.skippedLookupWait = result.skippedWait
|
||||||
rq.line.ASN = rq.lookupAnswer.ASN
|
rq.line.ASN = rq.lookupAnswer.ASN
|
||||||
rq.line.ASName = rq.lookupAnswer.ASName
|
rq.line.ASName = rq.lookupAnswer.ASName
|
||||||
rq.line.Country = rq.lookupAnswer.Country
|
rq.line.Country = rq.lookupAnswer.Country
|
||||||
|
|||||||
@@ -190,7 +190,7 @@ func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(
|
|||||||
clk.Now,
|
clk.Now,
|
||||||
map[string]string{
|
map[string]string{
|
||||||
trustedProxies: trustLocalhost,
|
trustedProxies: trustLocalhost,
|
||||||
alertWebhookURL: "https://alerts.example/sww",
|
alertWebhookURL: alertWebhook,
|
||||||
rateLimitPerMinute: "1",
|
rateLimitPerMinute: "1",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|||||||
+15
-2
@@ -8,6 +8,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"net/netip"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
lru "github.com/hashicorp/golang-lru/v2"
|
lru "github.com/hashicorp/golang-lru/v2"
|
||||||
@@ -66,7 +67,7 @@ type Params struct {
|
|||||||
LookupFile *lookup.File
|
LookupFile *lookup.File
|
||||||
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
|
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
|
||||||
// while SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
|
// while SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
|
||||||
CoreRuleSet *coreruleset.CoreRuleSet
|
CoreRuleSet CoreRuleSetInspector
|
||||||
// Now tells the time by which requests are counted for the rate
|
// Now tells the time by which requests are counted for the rate
|
||||||
// limits, bans are made and run out, and GeoJS's answers are kept,
|
// limits, bans are made and run out, and GeoJS's answers are kept,
|
||||||
// normally time.Now in UTC, the time the state files give.
|
// normally time.Now in UTC, the time the state files give.
|
||||||
@@ -84,6 +85,14 @@ type Params struct {
|
|||||||
Alerts *alerts.Queue
|
Alerts *alerts.Queue
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
|
||||||
|
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
|
||||||
|
type CoreRuleSetInspector interface {
|
||||||
|
Inspect(
|
||||||
|
r *http.Request, client netip.Addr, body io.Reader,
|
||||||
|
) (coreruleset.Result, []byte, error)
|
||||||
|
}
|
||||||
|
|
||||||
// Server is the server waf runs, with the parts whose state the state
|
// Server is the server waf runs, with the parts whose state the state
|
||||||
// files keep, the lookup database, nil unless SWW_WAF_LOOKUP_SOURCE is file, the
|
// files keep, the lookup database, nil unless SWW_WAF_LOOKUP_SOURCE is file, the
|
||||||
// lists fetched from URLs, which its Run fetches, the DNSBL zones' verdicts,
|
// lists fetched from URLs, which its Run fetches, the DNSBL zones' verdicts,
|
||||||
@@ -158,6 +167,8 @@ func newHandler(params Params) *handler {
|
|||||||
rules: params.Rules,
|
rules: params.Rules,
|
||||||
coreRuleSet: params.CoreRuleSet,
|
coreRuleSet: params.CoreRuleSet,
|
||||||
alerts: params.Alerts,
|
alerts: params.Alerts,
|
||||||
|
|
||||||
|
besideLookupLimit: besideLookupTimeLimit,
|
||||||
}
|
}
|
||||||
if params.Config.RouteWatchURL != "" {
|
if params.Config.RouteWatchURL != "" {
|
||||||
h.routeWatch = routewatch.New(routewatch.Params{
|
h.routeWatch = routewatch.New(routewatch.Params{
|
||||||
@@ -276,11 +287,13 @@ type handler struct {
|
|||||||
abuseIPDB *reputation.AbuseIPDB
|
abuseIPDB *reputation.AbuseIPDB
|
||||||
routeWatch *routewatch.Client
|
routeWatch *routewatch.Client
|
||||||
rules *rules.Files
|
rules *rules.Files
|
||||||
coreRuleSet *coreruleset.CoreRuleSet
|
coreRuleSet CoreRuleSetInspector
|
||||||
alerts *alerts.Queue
|
alerts *alerts.Queue
|
||||||
// waitForGeoJS is true when a setting needs each request's answer from
|
// waitForGeoJS is true when a setting needs each request's answer from
|
||||||
// GeoJS before the request goes on.
|
// GeoJS before the request goes on.
|
||||||
waitForGeoJS bool
|
waitForGeoJS bool
|
||||||
|
// besideLookupLimit is besideLookupTimeLimit. Only tests change it.
|
||||||
|
besideLookupLimit time.Duration
|
||||||
// awaitingReport holds the decision on each answered question by its id
|
// awaitingReport holds the decision on each answered question by its id
|
||||||
// until its report comes, and nil for a question still being handled.
|
// until its report comes, and nil for a question still being handled.
|
||||||
awaitingReport *lru.Cache[string, *decision]
|
awaitingReport *lru.Cache[string, *decision]
|
||||||
|
|||||||
@@ -512,6 +512,11 @@ func newWAFWith(
|
|||||||
Alerts: alertQueue,
|
Alerts: alertQueue,
|
||||||
})
|
})
|
||||||
|
|
||||||
|
// The race detector and a busy host can slow the Core Rule Set's
|
||||||
|
// inspections past the time limit. The tests of the limit set it back, and
|
||||||
|
// hold the Core Rule Set with a stand-in.
|
||||||
|
server.SetBesideLookupTimeLimit(time.Hour)
|
||||||
|
|
||||||
return server, out, alertQueue
|
return server, out, alertQueue
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+43
-9
@@ -81,6 +81,9 @@ type request struct {
|
|||||||
checked time.Time
|
checked time.Time
|
||||||
// refused is what the request was refused with, nil if it was not.
|
// refused is what the request was refused with, nil if it was not.
|
||||||
refused *refusal
|
refused *refusal
|
||||||
|
// findings is what the heuristics of besideLookup found of the request,
|
||||||
|
// once waf has taken it, and nil before.
|
||||||
|
findings *besideLookupFindings
|
||||||
// requestBytes is how many bytes of the request's body have been read.
|
// requestBytes is how many bytes of the request's body have been read.
|
||||||
requestBytes int64
|
requestBytes int64
|
||||||
}
|
}
|
||||||
@@ -177,9 +180,9 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
|
|||||||
// check is the one place where a request can be refused. It returns nil to let
|
// check is the one place where a request can be refused. It returns nil to let
|
||||||
// the request through. The heuristics of checkClient decide: a request they
|
// the request through. The heuristics of checkClient decide: a request they
|
||||||
// refuse is answered with SWW_WAF_BAN_RESPONSE, 403 for a block rule or the Core
|
// refuse is answered with SWW_WAF_BAN_RESPONSE, 403 for a block rule or the Core
|
||||||
// Rule Set, or 414 or 431 for the header limits. In observe mode nothing is refused,
|
// Rule Set, 503 for heuristics that ran out of time, or 414 or 431 for the
|
||||||
// and the log line names what enforce mode would have done. ctx is the
|
// header limits. In observe mode nothing is refused, and the log line names
|
||||||
// request's own context.
|
// what enforce mode would have done. ctx is the request's own context.
|
||||||
func (rq *request) check(ctx context.Context) *refusal {
|
func (rq *request) check(ctx context.Context) *refusal {
|
||||||
v := rq.checkClient(ctx)
|
v := rq.checkClient(ctx)
|
||||||
if v == nil {
|
if v == nil {
|
||||||
@@ -196,6 +199,11 @@ func (rq *request) check(ctx context.Context) *refusal {
|
|||||||
case action == requestlog.ActionBlockedByBlockRule ||
|
case action == requestlog.ActionBlockedByBlockRule ||
|
||||||
action == requestlog.ActionBlockedByCoreRuleSet:
|
action == requestlog.ActionBlockedByCoreRuleSet:
|
||||||
return &refusal{status: http.StatusForbidden, action: action}
|
return &refusal{status: http.StatusForbidden, action: action}
|
||||||
|
case action == requestlog.ActionChecksTimedOut:
|
||||||
|
return &refusal{
|
||||||
|
status: http.StatusServiceUnavailable,
|
||||||
|
action: action,
|
||||||
|
}
|
||||||
case action == requestlog.ActionRequestLineTooLong:
|
case action == requestlog.ActionRequestLineTooLong:
|
||||||
return &refusal{status: http.StatusRequestURITooLong, action: action}
|
return &refusal{status: http.StatusRequestURITooLong, action: action}
|
||||||
case action == requestlog.ActionRequestHeaderTooLarge:
|
case action == requestlog.ActionRequestHeaderTooLarge:
|
||||||
@@ -208,11 +216,16 @@ func (rq *request) check(ctx context.Context) *refusal {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkClient runs the heuristics of beforeLookup on the request, then looks
|
// checkClient runs the heuristics on the request, and returns the first
|
||||||
// up the client's AS number and country, then runs those of afterLookup. It
|
// verdict, or nil when none gives one. A client in SWW_WAF_ALLOW_NETS skips them
|
||||||
// returns the first verdict, or nil when none gives one. A client in
|
// all, and is not looked up. ctx is the request's own context.
|
||||||
// SWW_WAF_ALLOW_NETS skips them all, and is not looked up. ctx is the request's
|
//
|
||||||
// own context.
|
// The heuristics of beforeLookup run first. Then the client's AS number and
|
||||||
|
// country are looked up, and afterLookup runs once the lookup has its result.
|
||||||
|
// While the lookup waits for GeoJS, besideLookup runs too, as runBesideLookup
|
||||||
|
// says. A lookup that needs no wait is done, and afterLookup with it, before
|
||||||
|
// besideLookup starts, so that besideLookup does no work for a request
|
||||||
|
// afterLookup refuses.
|
||||||
func (rq *request) checkClient(ctx context.Context) *verdict {
|
func (rq *request) checkClient(ctx context.Context) *verdict {
|
||||||
if isInside(rq.client, rq.h.config.AllowNets) {
|
if isInside(rq.client, rq.h.config.AllowNets) {
|
||||||
return nil
|
return nil
|
||||||
@@ -225,7 +238,28 @@ func (rq *request) checkClient(ctx context.Context) *verdict {
|
|||||||
return v
|
return v
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.lookUp(ctx, now)
|
lookedUp := rq.startLookUp(ctx, now)
|
||||||
|
|
||||||
|
select {
|
||||||
|
case result := <-lookedUp:
|
||||||
|
v := rq.runAfterLookup(ctx, result)
|
||||||
|
if v != nil {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
return rq.runBesideLookup(ctx, nil)
|
||||||
|
default:
|
||||||
|
return rq.runBesideLookup(ctx, lookedUp)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// runAfterLookup notes result, the lookup's, and runs the heuristics of
|
||||||
|
// afterLookup. It returns the first verdict, or nil when none gives one. ctx is
|
||||||
|
// the request's own context.
|
||||||
|
func (rq *request) runAfterLookup(
|
||||||
|
ctx context.Context, result lookupResult,
|
||||||
|
) *verdict {
|
||||||
|
rq.noteLookup(result)
|
||||||
|
|
||||||
// The lookup can wait for GeoJS, so the checks after it read the time
|
// The lookup can wait for GeoJS, so the checks after it read the time
|
||||||
// again: a request counted at the time it arrived could be counted
|
// again: a request counted at the time it arrived could be counted
|
||||||
|
|||||||
@@ -13,8 +13,8 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ruleFiles checks the request against the rules of the rule files at now,
|
// ruleFiles takes the rules of the rule files the request matched, which
|
||||||
// notes the ids of those it matches in the log line, adds the tag
|
// findBesideLookup found, notes their ids in the log line, adds the tag
|
||||||
// exploitsignature when it matches any, and refuses the request
|
// exploitsignature when it matches any, and refuses the request
|
||||||
// when the last rule it matches refuses it: with ActionBlockedByBlockRule for a
|
// when the last rule it matches refuses it: with ActionBlockedByBlockRule for a
|
||||||
// block rule, and for a ban rule with the action for the ban it makes. A ban
|
// block rule, and for a ban rule with the action for the ban it makes. A ban
|
||||||
@@ -31,7 +31,7 @@ func (ruleFiles) check(
|
|||||||
rq *request,
|
rq *request,
|
||||||
now time.Time,
|
now time.Time,
|
||||||
) *verdict {
|
) *verdict {
|
||||||
matched := rq.h.rules.Match(rq.in)
|
matched := rq.findings.rules
|
||||||
|
|
||||||
for _, rule := range matched {
|
for _, rule := range matched {
|
||||||
rq.line.RuleIDs = append(rq.line.RuleIDs, rule.ID)
|
rq.line.RuleIDs = append(rq.line.RuleIDs, rule.ID)
|
||||||
@@ -70,3 +70,15 @@ func (ruleFiles) check(
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// refusedByRule reports whether ruleFiles refuses a request that matched
|
||||||
|
// matched: whether the last of them is a block rule or a ban rule.
|
||||||
|
func refusedByRule(matched []rules.Rule) bool {
|
||||||
|
if len(matched) == 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
action := matched[len(matched)-1].Action
|
||||||
|
|
||||||
|
return action == rules.ActionBlock || action == rules.ActionBan
|
||||||
|
}
|
||||||
@@ -11,9 +11,8 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
)
|
)
|
||||||
|
|
||||||
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS: its path as a
|
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS, as trapPath
|
||||||
// path rule sees it, without the query, as sent or once percent-decoded, is one
|
// finds it. Such a request is a clear sign of attack, as a ban rule's match is:
|
||||||
// of them. Such a request is a clear sign of attack, as a ban rule's match is:
|
|
||||||
// it bans the client's netblock, or in observe mode raises the alert for the
|
// it bans the client's netblock, or in observe mode raises the alert for the
|
||||||
// ban it would have made. The ban's notes give the path as sent, and so does
|
// ban it would have made. The ban's notes give the path as sent, and so does
|
||||||
// its figure.
|
// its figure.
|
||||||
@@ -26,9 +25,8 @@ func (trapPaths) check(
|
|||||||
rq *request,
|
rq *request,
|
||||||
now time.Time,
|
now time.Time,
|
||||||
) *verdict {
|
) *verdict {
|
||||||
path := rules.Path(rq.in)
|
path := rq.findings.trapPath
|
||||||
if !slices.Contains(rq.h.config.TrapPaths, path) &&
|
if path == "" {
|
||||||
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -39,3 +37,16 @@ func (trapPaths) check(
|
|||||||
figures: map[string]any{"path": path},
|
figures: map[string]any{"path": path},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// trapPath returns the request's path as sent, when its path as a path rule
|
||||||
|
// sees it, without the query, as sent or once percent-decoded, is in
|
||||||
|
// SWW_WAF_TRAP_PATHS, and "" otherwise.
|
||||||
|
func (rq *request) trapPath() string {
|
||||||
|
path := rules.Path(rq.in)
|
||||||
|
if !slices.Contains(rq.h.config.TrapPaths, path) &&
|
||||||
|
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
return path
|
||||||
|
}
|
||||||
Reference in new issue
Block a user