Compare commits

...
Author SHA1 Message Date
sneak 5a73af61b0 sww: wire the daemons with fx (closes #414)
check / check (push) Waiting to run
go.uber.org/fx builds the parts of waf and rproxy in internal/sww, one
provider per part. Each background loop is a lifecycle hook added where
its part is built. waf.Params takes every part built, and waf.New builds
none. GeoJS calls back into waf, so waf exports AddLookup and
GeoJSFailed, and clientGroup and banNetblock take the settings.

newServer, loadStateFiles, the inBackground list and serveRProxy are
gone. serve is the one hook that serves both daemons, and reports a
failure to run on a channel of its own; fx's signal handling is not
used. Once fx has stopped, run takes each daemon's last step: waf writes
the state files, and rproxy waits for its reports.

Model: opus-5-5
2026-11-19 09:59:52 +00:00
clawbot 9f42460f70 Say the anomaly thresholds only send an alert (closes #412)
check / check (push) Waiting to run
Each SWW_WAF_ANOMALY_ threshold's description now says crossing it sends
one alert to the alert destinations and bans, rate limits and blocks no
client. The prefix and watched-netblock settings say their thresholds
only alert. README's configuration notes and docs/settings.md explain
"anomaly" once.

Model: opus-5-5
2026-11-19 10:58:27 +01:00
clawbot 9cb8f362bc sww: lift waf's check time limit and rproxy's wait in tests (closes #418)
check / check (push) Waiting to run
Under heavy host load the sww tests got 503 where they expected 200 or
403: waf refused requests whose trap paths, rule files and Core Rule Set
took longer than 500 ms, and rproxy gave up on waf after 2 s.

waf.Params and sww.Params gain BesideLookupTimeLimit; zero keeps the
500 ms limit, so the binary is unchanged. The sww tests' run gives waf
an hour, and startRProxy sets SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS to
60000, as the rproxy package's tests do. waf's own tests set the limit
through the new field instead of SetBesideLookupTimeLimit, which is gone.

Model: opus-5-5
2026-11-19 10:44:53 +01:00
clawbot 48edab5dad Test rproxy's 32 KiB header limit at its edge (closes #415)
check / check (push) Waiting to run
The limit was already a constant with no setting. These tests pin it:
a request line and headers of exactly 32 KiB reach the app, and one
byte more is answered 431 with the connection closed and waf not asked.

Model: opus-5-5
2026-11-19 10:20:07 +01:00
sneak dadd5768d5 Merge pull request 'next into main' (#223) from next into main
check / check (push) Failing after 11m39s
Reviewed-on: #223
2026-11-19 03:43:19 +01:00
sneak e6a081134a Merge branch 'main' into next
check / check (push) Failing after 1m52s
2026-11-19 03:43:10 +01:00
sneak 4cb91a4870 Merge pull request 'Set the images' own OCI labels instead of Ubuntu's' (#224) from issue-216-oci-labels into next
check / check (push) Failing after 1m58s
Reviewed-on: #224
2026-11-19 03:41:57 +01:00
sneak b9972af6a0 Merge branch 'next' into issue-216-oci-labels
check / check (push) Successful in 7m10s
2026-11-19 03:41:52 +01:00
clawbot c0d9d5787e reputation: run the DNSBL resolver test in parallel (closes #406)
check / check (push) Failing after 2m25s
TestQueriesGoToTheResolverSWWDNSBLResolverNames queries a real UDP
stand-in outside any synctest bubble. A parallel test starts only once
every test that is not parallel has ended, so it can no longer overlap
the bubbled DNSBL tests, and no other parallel test of the package
queries a resolver.

The other tests that run one at a time stay so, each for the reason its
comment gives: environment variables, TMPDIR, timing, Go's resolver in
synctest bubbles, and garbage collection hiding a leaked connection.
Every subtest inside a parallel test already calls t.Parallel().

Model: opus-5-5
2026-11-17 06:57:44 +01:00
clawbot 8c9f20d346 waf: run the trap paths, rule files and Core Rule Set during the GeoJS wait (closes #405)
check / check (push) Failing after 1m51s
While the lookup waits for GeoJS, the trap paths, the rule files and the
Core Rule Set run at the same time as the checks after the lookup. The
first side to refuse answers the request; what the other side finds leaves
no trace. Those three checks have 500 ms together: past it, the request is
refused with 503 as checks_timed_out, with no ban and no offence.

Tests hold GeoJS and the Core Rule Set with stand-ins under synctest. The
test helper lifts the 500 ms limit for tests that run the real Core Rule
Set, which the race detector slows past it.

Model: opus-5-5
2026-11-17 06:11:52 +01:00
sneak a93243a294 Set the images' own OCI labels instead of Ubuntu's (closes #216)
check / check (push) Failing after 1m53s
The image sets title, description, version, created, revision and
source; the last three known only at build time come from build
arguments that default to unknown, and script/docker and
script/cibuild pass them. The example app sets its own title and
description.

Model: opus-5-5
2026-11-14 17:58:09 +00:00
sneak f52b1f9ab5 Merge pull request 'next into main' (#205) from next into main
check / check (push) Successful in 3m37s
Reviewed-on: #205
2026-11-07 02:52:39 +01:00
43 changed files with 2040 additions and 759 deletions

No files matched your search

+14
View File
@@ -161,6 +161,20 @@ RUN git checkout --quiet --detach b4b2c785308b1ce785b6155c7fe5f16879080193 \
# ubuntu 26.04, 2026-09-27
FROM ubuntu@sha256:f144425ff09be612d6d9ad965196e9cdc23dae1f42110a8a11a3e9a8198759f7
# The image's own labels, in place of the ones it would inherit from
# Ubuntu's. script/docker and script/cibuild pass the build arguments;
# a build that passes none labels the image unknown rather than leave
# Ubuntu's values.
ARG VERSION=unknown
ARG CREATED=unknown
ARG REVISION=unknown
LABEL org.opencontainers.image.title="smallwebwaf" \
org.opencontainers.image.description="A web application firewall for self-hosted web apps: sww rproxy in front of an app, and sww waf, which decides." \
org.opencontainers.image.version="${VERSION}" \
org.opencontainers.image.created="${CREATED}" \
org.opencontainers.image.revision="${REVISION}" \
org.opencontainers.image.source="https://git.eeqj.de/sneak/smallwebwaf"
# runit's install creates its _runit-log user with minsysusers, which
# reads this file in place of runit's /usr/lib/sysusers.d/runit.conf.
# runit's line leaves out the shell, and minsysusers prints a Perl
+38 -26
View File
@@ -284,6 +284,10 @@ account, a key, a token or an alert destination stays off until you give one.
`SWW_WAF_BYTES_COUNTED` says. Headers are never counted. What the visitor sent
over a WebSocket counts as request bytes, and what the app sent as response
bytes.
- An anomaly is more requests or bytes than an `SWW_WAF_ANOMALY_` threshold
allows. Crossing a threshold sends one `anomaly` alert to the alert
destinations, and does nothing else: no client is banned, rate limited or
blocked.
- An invalid setting stops the process at start with one log line naming the
variable and what is wrong with the value. The settings in effect are logged
on one line at start, with tokens, keys and webhook paths shown as `********`.
@@ -384,29 +388,29 @@ Every setting the code reads, with its default:
| `SWW_WAF_ALERT_EVENTS` | `ban,permanent_ban,core_rule_set_match,anomaly,reputation_hit,source_failure,file_error` | The events alerts are sent for; the hourly `summary` is sent whatever it says. |
| `SWW_WAF_ALERT_COOLDOWN` | `15m` | How long after an alert its repeats are held back and counted, a duration or `off`. |
| `SWW_WAF_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for one client in a minute, a size. |
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for one client in an hour, a size. |
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
| `SWW_WAF_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for the netblock around a client in a minute, a size. |
| `SWW_WAF_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for the netblock around a client in an hour, a size. |
| `SWW_WAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 32. |
| `SWW_WAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 128. |
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for one AS number in a minute, a size. |
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for one AS number in an hour, a size. |
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for the whole service in a minute, a size. |
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for the whole service in an hour, a size. |
| `SWW_WAF_ANOMALY_WATCHED_NETS` | empty | Named netblocks the `SWW_WAF_ANOMALY_WATCHED_NET_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
| `SWW_WAF_ANOMALY_WATCHED_NET_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
| `SWW_WAF_ANOMALY_WATCHED_NET_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
| `SWW_WAF_ANOMALY_WATCHED_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for a watched netblock in a minute, a size. |
| `SWW_WAF_ANOMALY_WATCHED_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for a watched netblock in an hour, a size. |
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when one client makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when one client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 32. Those thresholds only send alerts. |
| `SWW_WAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 128. Those thresholds only send alerts. |
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the whole service in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the whole service in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_WATCHED_NETS` | empty | Named netblocks the `SWW_WAF_ANOMALY_WATCHED_NET_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. Those thresholds only send alerts. |
| `SWW_WAF_ANOMALY_WATCHED_NET_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when a watched netblock makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_WATCHED_NET_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when a watched netblock makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_WATCHED_NET_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for a watched netblock in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_WATCHED_NET_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for a watched netblock in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
[`docs/settings.md`](docs/settings.md) gives the values each setting takes in
full, and the limits that are fixed rather than settings.
@@ -599,6 +603,11 @@ request reaches the app.
16. A path under `/_sww/` is answered `404`, so that the app never sees that
prefix.
Steps 12 to 14 only read the request. While the lookup waits for GeoJS, they run
at the same time as steps 6 to 11. The first of the two sides to refuse the
request ends the checks, and `waf` answers at once. Steps 12 to 14 have 500 ms
together.
`rproxy` takes the rest, with `waf` counting at the end:
17. The request is passed to the app, streaming.
@@ -611,9 +620,10 @@ A refused visitor is answered with `SWW_WAF_BAN_RESPONSE`. A `block` rule or a
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
false positives; such refusals count toward `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`
instead. A request over the header limits of step 3 is answered `414` for a
request line over 8 KiB and `431` otherwise. It bans nobody either. `rproxy`
itself answers `405` to a method it does not pass on, and the status for a size
or time limit, such as `413` for a body too large.
request line over 8 KiB and `431` otherwise. It bans nobody either. Nor does a
request that steps 12 to 14 have not finished within 500 ms, which is answered
`503`. `rproxy` itself answers `405` to a method it does not pass on, and the
status for a size or time limit, such as `413` for a body too large.
### Clients and bans
@@ -744,6 +754,8 @@ stops the running process (see [`docs/state-files.md`](docs/state-files.md)).
- `cmd/sww`: the binary, which only calls `internal/sww`.
- `internal/sww`: the process, as the command it is given, and the health check.
It is the only package that imports `go.uber.org/fx`, which builds the parts
of `waf` and `rproxy`.
- `internal/config`: reads the settings, the one place they are read.
- `internal/rproxy`: the reverse proxy in front of one app.
- `internal/waf`: `waf`'s server, which runs the checks and answers its own
+56 -14
View File
@@ -69,7 +69,9 @@ never as a time.
that command's settings, and runs `internal/rproxy` until `SIGTERM` or
`SIGINT` (see "`rproxy`"). `sww healthcheck` is the image's health check.
With no command, or one it does not know, the binary prints the commands
and exits with status 2.
and exits with status 2. `internal/sww` is the only package that imports
`go.uber.org/fx`. fx builds the parts of `waf` and `rproxy`, and starts
and stops their background work.
- `internal/config`: reads the settings, the one place they are read.
- `internal/waf`: `waf`'s server. It runs the checks in `check`, and holds
`waf`'s question, answer and report. It answers `waf`'s own endpoints in
@@ -173,7 +175,8 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
`country=` tag.
Otherwise the request goes on at once. A request that ends before the answer
comes is logged without it.
comes is logged without it. While the request waits, steps 12 to 14 run at
the same time as steps 6 to 11 (see "Answering in time").
7. The country list (`countries`; see "Country list").
8. The reputation sources (see "Reputation"), in this order:
@@ -2379,8 +2382,9 @@ runs no check, writes no request log line and counts nothing in the metrics.
### The answer
`waf` runs steps 3 to 16 of "What happens to a request", in that order. Every
answer but the last below carries `X-Sww-Disposition`.
`waf` runs steps 3 to 16 of "What happens to a request", in that order, apart
from the steps that run at the same time (see "Answering in time"). Every answer
but the last below carries `X-Sww-Disposition`.
- `X-Sww-Disposition: allow`, status `200`, no body. `rproxy` passes the request
to the app. While `SWW_WAF_ADD_LOOKUP_HEADERS` is `true`, the answer also
@@ -2392,6 +2396,8 @@ answer but the last below carries `X-Sww-Disposition`.
- `SWW_WAF_BAN_RESPONSE`, for a refusal;
- `403`, for a `block` rule or the Core Rule Set;
- `414` or `431`, for a request over the header limits;
- `503`, for a request that steps 12 to 14 have not finished within 500
milliseconds;
- `404`, for a path under `/_sww/` that passes the checks, so that the app
never sees that prefix.
- `X-Sww-Disposition: close`, status `200`, no body. This takes the place of a
@@ -2412,9 +2418,15 @@ The Core Rule Set inspects the body a question carries only while
`rproxy` stops waiting for `waf` after `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, by
default 2 seconds, and `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` then
decides. The longest a question waits on an outside source is
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by default 800 milliseconds, well
within that time.
decides. Two things in a decision take time:
- the wait for GeoJS, at most `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by
default 800 milliseconds;
- the trap paths, the rule files and the Core Rule Set, which have 500
milliseconds.
They run at the same time, so `waf` answers within the longer of the two, well
within `rproxy`'s wait.
GeoJS is the only outside source a question can wait on. A question waits only
when step 6 says so, and for at most
@@ -2446,10 +2458,38 @@ never wait. These can still wait for GeoJS:
- a country list, which needs the answer;
- a rate limit a percentage has lowered, which needs the answer;
- a DNS blocklist or AbuseIPDB, while `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` is
`deny`;
- a trap path;
- a rule file;
- the Core Rule Set.
`deny`.
The trap paths, the rule files and the Core Rule Set, steps 12 to 14, run during
that wait. Once a request has passed the bans, the lookup starts. While it waits
for GeoJS, steps 12 to 14 run at the same time as steps 6 to 11. A lookup that
needs no wait has its answer at once, and steps 7 to 11 then run before steps 12
to 14 start.
- The first of the two sides to refuse the request ends the checks. `waf`
answers at once, and does not wait for the other side. The lookup goes on, and
its answer is kept.
- What the other side finds leaves no trace. It counts nothing, bans nothing,
raises no alert, counts no metric and adds nothing to the log line.
- When both sides would refuse, the one that finishes first is answered and
logged.
- When neither refuses, step 15 runs.
Steps 12 to 14 only read the request. They have 500 milliseconds together, from
their start. The limit is fixed, with no setting. A request they have not
finished by then is refused:
- the answer is `deny`, with `503`;
- the log line's action is `checks_timed_out`, and its `heuristic` names the
step that had not finished;
- the refusal makes no ban, is no offence and does not count toward the refusal
limit;
- it holds in every `SWW_WAF_CORE_RULE_SET_MODE`;
- in `observe` mode the request is answered `allow` with `would_action`, like
every refusal.
The limit is measured on the clock. While `waf`'s host is so busy that an
ordinary request takes 500 milliseconds, that request is refused too.
The header limits are fixed, with no setting. A line's length is in bytes and
does not count its line ending. `waf` refuses a request whose request line is
@@ -2499,9 +2539,11 @@ body can add up to more than the body. From 1 KiB of JSON, Coraza makes at most
A request can reach all of these at once. At paranoia level 4, with no rule
switched off, the Core Rule Set takes under a second on one core of a current
server to inspect it. So `waf` answers it within the default 2 seconds, even
after the default 800 milliseconds of waiting for GeoJS. Enough such requests at
once can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
server to inspect it. `waf` refuses it if the inspection takes over 500
milliseconds. Coraza cannot stop an inspection part-way, so the inspection runs
to its end with nobody waiting for it. These limits keep one request from
holding a processor for more than about a second. Enough such requests at once
can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
### The report
+4
View File
@@ -9,6 +9,10 @@
ARG SWW_IMAGE=smallwebwaf
FROM ${SWW_IMAGE}
# Its own title and description, in place of the smallwebwaf image's.
LABEL org.opencontainers.image.title="smallwebwaf-example-app" \
org.opencontainers.image.description="An example app built on the smallwebwaf image."
# Packages the app needs, from the nixpkgs in the image.
RUN nix-env -iA nixpkgs.busybox
+7
View File
@@ -105,6 +105,8 @@ Every one of these lines has:
`blocked_by_core_rule_set`: refused by that check;
- `request_line_too_long`: a request line over the header limits;
- `request_header_too_large`: headers over the header limits;
- `checks_timed_out`: refused with `503` because the trap paths, the rule
files and the Core Rule Set had not finished within 500 ms;
- `own_endpoint`: answered at one of `waf`'s own endpoints, or `404` for a
path under `/_sww/`.
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
@@ -134,6 +136,11 @@ Every one of these lines has:
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`;
- `refusal_limit`: `window`, `count` and `limit`, of the refusal limit;
- `byte_limits`: `window`, `count` and `limit`, the count in bytes.
For a request refused as `checks_timed_out`, `heuristic` is the one of
`trap_paths`, `rule_files` and `core_rule_set` that had not finished, and
its figure is `time_limit_ms`, 500.
- `after_answer_heuristic`: the heuristic that banned the client after `waf`
answered the request, in `observe` mode too: `refusal_limit` or `byte_limits`.
- `after_answer_heuristic_figures`: an object of the figures that heuristic
+8 -1
View File
@@ -364,6 +364,11 @@ tag that no request can get stops the start. A request can get only these:
## Anomaly thresholds
An anomaly is more requests or bytes than one of these thresholds allows.
Crossing a threshold sends one `anomaly` alert to the destinations under
"Alerting", and does nothing else: no client is banned, rate limited or blocked.
`SWW_WAF_ALERT_COOLDOWN` holds back repeats of that alert.
Each scope has four thresholds: the most requests and the most body bytes
counted in a minute and in an hour. The bytes are those `SWW_WAF_BYTES_COUNTED`
names. Requests are counts, bytes are sizes. Each is `off` by default. The four
@@ -435,7 +440,9 @@ A few limits are fixed rather than settings:
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` is the longest a question waits on
an outside source. Keep it well under `rproxy`'s
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, as the default is. No question waits for
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, as the default is. The trap paths, the rule
files and the Core Rule Set run during that wait and have 500 ms, so `waf`
answers within the longer of the two times. No question waits for
`SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`, since the DNS blocklists and AbuseIPDB are
asked in the background (see "Answering in time" under "`waf`" in
[`SPEC.md`](../SPEC.md)).
+4
View File
@@ -12,6 +12,7 @@ require (
github.com/oschwald/maxminddb-golang/v2 v2.7.0
github.com/prometheus/client_golang v1.24.1
github.com/spf13/viper v1.21.0
go.uber.org/fx v1.24.0
golang.org/x/sys v0.48.0
)
@@ -44,6 +45,9 @@ require (
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
go.uber.org/dig v1.19.0 // indirect
go.uber.org/multierr v1.10.0 // indirect
go.uber.org/zap v1.26.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
golang.org/x/net v0.60.0 // indirect
+8
View File
@@ -97,8 +97,16 @@ github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/valllabh/ocsf-schema-golang v1.0.3 h1:eR8k/3jP/OOqB8LRCtdJ4U+vlgd/gk5y3KMXoodrsrw=
github.com/valllabh/ocsf-schema-golang v1.0.3/go.mod h1:sZ3as9xqm1SSK5feFWIR2CuGeGRhsM7TR1MbpBctzPk=
go.uber.org/dig v1.19.0 h1:BACLhebsYdpQ7IROQ1AGPjrXcP5dF80U3gKoFzbaq/4=
go.uber.org/dig v1.19.0/go.mod h1:Us0rSJiThwCv2GteUN0Q7OKvU7n5J4dxZ9JKUXozFdE=
go.uber.org/fx v1.24.0 h1:wE8mruvpg2kiiL1Vqd0CC+tr0/24XIB10Iwp2lLWzkg=
go.uber.org/fx v1.24.0/go.mod h1:AmDeGyS+ZARGKM4tlH4FY2Jr63VjbEDJHtqXTGP5hbo=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ=
go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
+5 -4
View File
@@ -35,10 +35,11 @@ const (
mostJSONRead = 1 << 10
)
// The Core Rule Set has 1 second for one request. That is the default of
// rproxy's SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, 2 seconds, less the default of
// SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, 800 milliseconds, and less 200
// milliseconds for waf's other checks.
// waf refuses a request the Core Rule Set has not inspected within 500
// milliseconds, but Coraza cannot stop an inspection part-way, so it runs to
// its end with nobody waiting for it. The test checks that an inspection ends
// within about 1 second, so that a visitor cannot keep a processor busy for
// longer with one request.
//
// A busy host slows all work, so the test does not compare an inspection with
// 1 second. It compares it with referenceTime, timed on the same thread just
+8 -5
View File
@@ -31,10 +31,12 @@ import (
// this package: Go's resolver counts the queries under way in one
// sync.WaitGroup for the whole process, and the process fails when
// queries from two bubbles, or from a bubble and from outside one, are
// under way at once. TestMain has the resolver make its configuration,
// which it makes on its first query, outside every bubble, since the
// configuration holds a channel, which the bubble it was made in would
// keep to itself.
// under way at once. The one test outside a bubble is parallel, so it
// starts only once every test that is not parallel has ended; no other
// parallel test of this package queries a resolver. TestMain has the
// resolver make its configuration, which it makes on its first query,
// outside every bubble, since the configuration holds a channel, which
// the bubble it was made in would keep to itself.
const (
// zone and otherZone are the DNSBL zones the tests name.
@@ -525,8 +527,9 @@ func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(
}
}
//nolint:paralleltest // one at a time, as the comment at the top of this file says
func TestQueriesGoToTheResolverSWWDNSBLResolverNames(t *testing.T) {
t.Parallel()
resolver := &resolverStandIn{answers: map[string]answer{
listedName: {addrs: []string{listing}},
}}
+8 -1
View File
@@ -61,6 +61,12 @@ const (
// ActionCountryDenied is a request refused because its client's country
// is not in SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
ActionCountryDenied = "country_denied"
// ActionChecksTimedOut is a request refused because the trap paths, the
// rule files and the Core Rule Set had not finished with it within 500
// milliseconds.
ActionChecksTimedOut = "checks_timed_out"
// ActionOwnEndpoint is a request waf answered at one of its own
// endpoints, or a question about a path under /_sww/.
ActionOwnEndpoint = "own_endpoint"
@@ -149,7 +155,8 @@ type Line struct {
// ActionTemporarilyBanned, ActionPermanentlyBanned, ActionCountryDenied,
// ActionClientIsRateLimited, ActionBlockedByBlockRule,
// ActionBlockedByCoreRuleSet, or, in waf, ActionRequestLineTooLong or
// ActionRequestHeaderTooLarge for the header limits.
// ActionRequestHeaderTooLarge for the header limits, or
// ActionChecksTimedOut.
WouldAction string `json:"would_action,omitempty"`
// Heuristic is the name of the heuristic that refused the request, in
// observe mode too, and HeuristicFigures are the figures it acted on, by
+53
View File
@@ -71,6 +71,59 @@ func TestServerHasTheFixedLimits(t *testing.T) {
}
}
// getOfSize is a GET whose request line and headers, the blank line that ends
// them included, are size bytes long.
func getOfSize(size int) string {
const start, end = "GET / HTTP/1.1\r\nHost: site.example\r\nX-Pad: ", "\r\n\r\n"
return start + strings.Repeat("x", size-len(start)-len(end)) + end
}
func TestARequestLineAndHeadersOf32KiBReachTheApp(t *testing.T) {
t.Parallel()
standIn := startWAF(t, allow)
app := startApp(t, greet)
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr)
send(t, conn, getOfSize(32<<10))
if status := readStatus(t, conn); status != http.StatusOK {
t.Errorf("status %d, want the app's 200", status)
}
}
func TestARequestLineAndHeadersOver32KiBAreRefusedAndTheConnectionClosed(
t *testing.T,
) {
t.Parallel()
standIn := startWAF(t, allow)
app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked")
})
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr)
send(t, conn, getOfSize(32<<10+1))
// io.ReadAll returns without an error only once rproxy has closed the
// connection.
text, err := io.ReadAll(conn)
if err != nil || !strings.HasPrefix(string(text), "HTTP/1.1 431 ") {
t.Errorf(
"read %q (%v), want 431 and the connection closed",
text,
err,
)
}
if len(standIn.questions) != 0 {
t.Error("waf was asked")
}
}
func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
t.Parallel()
+198
View File
@@ -0,0 +1,198 @@
package sww
import (
"context"
"errors"
"io"
"log/slog"
"net"
"net/http"
"time"
"go.uber.org/fx"
"go.uber.org/fx/fxevent"
)
// shutdownTimeout is how long requests in progress may take to finish
// once sww is told to stop, before their connections are closed.
// runit and docker wait a little longer before they kill the process.
const shutdownTimeout = 5 * time.Second
// requestLog is where a daemon's server writes the request log: stdout, and
// the remote log sender while there is one. fx tells parts apart by type, so
// it has a type of its own.
type requestLog struct {
io.Writer
}
// servingFailed gets why serving failed, for run, which then stops the
// daemon. fx tells parts apart by type, so it has a type of its own.
type servingFailed chan error
// utcNow is the time the state files, the alerts and the rate limits go by.
func utcNow() time.Time {
return time.Now().UTC()
}
// startError is why a daemon cannot start: the message its process log gives,
// and the error it logs with it.
type startError struct {
message string
err error
}
func (e startError) Error() string {
return e.message + ": " + e.err.Error()
}
func (e startError) Unwrap() error {
return e.err
}
// cannotStart logs why a daemon cannot start, with the message of the
// startError in err, and returns its exit status.
func cannotStart(processLog *slog.Logger, err error) int {
message, cause := "cannot start", err
failed, ok := errors.AsType[startError](err)
if ok {
message, cause = failed.message, failed.err
}
processLog.Error(message, "error", cause.Error())
return 1
}
// listen listens on address, which the setting names.
func listen(address, setting string) (net.Listener, error) {
listener, err := (&net.ListenConfig{}).Listen(
context.Background(), "tcp", address)
if err != nil {
return nil, startError{"cannot listen on " + setting, err}
}
return listener, nil
}
// background returns the hook that runs loop on a goroutine of its own from
// the start, and at the stop ends it and waits for it to return.
func background(loop func(context.Context)) fx.Hook {
running, stop := context.WithCancel(context.Background())
ended := make(chan struct{})
return fx.StartStopHook(
func() {
go func() {
loop(running)
close(ended)
}()
},
func() {
stop()
<-ended
},
)
}
// serve serves requests on listener from the start. At the stop it gives the
// requests in progress shutdownTimeout to finish, then closes their
// connections. When serving fails, it logs why and sends the error to
// failed.
func serve(
lc fx.Lifecycle, server *http.Server, listener net.Listener,
processLog *slog.Logger, failed servingFailed,
) {
served := make(chan error, 1)
lc.Append(fx.Hook{
OnStart: func(context.Context) error {
go func() {
err := server.Serve(listener)
if !errors.Is(err, http.ErrServerClosed) {
processLog.Error("serving failed", "error", err.Error())
failed <- err
}
served <- err
}()
return nil
},
OnStop: func(ctx context.Context) error {
processLog.Info("stopping")
shutdownCtx, cancel := context.WithTimeout(ctx, shutdownTimeout)
defer cancel()
err := server.Shutdown(shutdownCtx)
if err != nil {
processLog.Warn("requests still in progress were cut off",
"error", err.Error())
_ = server.Close()
}
err = <-served
if !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
},
})
}
// run starts app, which serves until ctx is done or serving fails, as
// failed tells. Then it stops app and takes the daemon's last step. It
// returns the process's exit status, 1 when serving failed, a part could
// not stop in order or the last step failed; each logs why.
func run(
ctx context.Context, app *fx.App, failed servingFailed,
processLog *slog.Logger, lastStep func() error,
) int {
// The hooks only start goroutines, and end them at the stop, so neither
// is cut short by a deadline or by ctx.
err := app.Start(context.WithoutCancel(ctx))
if err != nil {
return cannotStart(processLog, err)
}
status := 0
select {
case <-ctx.Done():
case <-failed:
status = 1
}
err = app.Stop(context.WithoutCancel(ctx))
if err != nil {
status = 1
}
err = lastStep()
if err != nil {
status = 1
}
if status == 0 {
processLog.Info("stopped")
}
return status
}
// fxEventLogger returns what fx.WithLogger takes to write fx's own events to
// processLog at debug, which the daemons' log levels normally hold back. Its
// errors are at debug too: the daemon logs each one itself.
func fxEventLogger(processLog *slog.Logger) func() fxevent.Logger {
return func() fxevent.Logger {
logger := &fxevent.SlogLogger{Logger: processLog}
logger.UseLogLevel(slog.LevelDebug)
logger.UseErrorLevel(slog.LevelDebug)
return logger
}
}
+42 -63
View File
@@ -2,12 +2,13 @@ package sww
import (
"context"
"errors"
"log/slog"
"net"
"net/http"
"os"
"go.uber.org/fx"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rproxy"
@@ -30,26 +31,31 @@ func RunRProxy(ctx context.Context, params Params) int {
cfg, err := config.RProxyFromEnvironment(params.Viper)
if err != nil {
processLog.Error("invalid setting", "error", err.Error())
return 1
return cannotStart(processLog, startError{"invalid setting", err})
}
server := rproxy.New(rproxy.Params{
Config: cfg,
RequestLog: params.Stdout,
ProcessLog: processLog,
})
var (
server *rproxy.Server
listener net.Listener
)
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
failed := make(servingFailed, 1)
app := fx.New(
fx.Supply(cfg, processLog, requestLog{params.Stdout}, failed),
fx.Provide(
newRProxyServer,
func(server *rproxy.Server) *http.Server { return server.Server },
newRProxyListener,
),
fx.Invoke(serve),
fx.Populate(&server, &listener),
fx.WithLogger(fxEventLogger(processLog)),
)
err = app.Err()
if err != nil {
processLog.Error(
"cannot listen on SWW_RPROXY_LISTEN_ADDR",
"error",
err.Error(),
)
return 1
return cannotStart(processLog, err)
}
processLog.Info("starting",
@@ -57,52 +63,25 @@ func RunRProxy(ctx context.Context, params Params) int {
"address", listener.Addr().String(),
"settings", cfg)
return serveRProxy(ctx, server, listener, processLog)
return run(ctx, app, failed, processLog, func() error {
server.WaitForReports()
return nil
})
}
// serveRProxy serves requests on listener until ctx is done, and then stops as
// RunRProxy says.
func serveRProxy(
ctx context.Context, server *rproxy.Server, listener net.Listener,
processLog *slog.Logger,
) int {
served := make(chan error, 1)
go func() {
served <- server.Serve(listener)
}()
select {
case err := <-served:
processLog.Error("serving failed", "error", err.Error())
return 1
case <-ctx.Done():
}
processLog.Info("stopping")
shutdownCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx),
shutdownTimeout)
defer cancel()
err := server.Shutdown(shutdownCtx)
if err != nil {
processLog.Warn("requests still in progress were cut off",
"error", err.Error())
_ = server.Close()
}
err = <-served
if !errors.Is(err, http.ErrServerClosed) {
processLog.Error("serving failed", "error", err.Error())
return 1
}
server.WaitForReports()
processLog.Info("stopped")
return 0
// newRProxyServer returns the server rproxy runs.
func newRProxyServer(
cfg *config.RProxy, out requestLog, processLog *slog.Logger,
) *rproxy.Server {
return rproxy.New(rproxy.Params{
Config: cfg,
RequestLog: out,
ProcessLog: processLog,
})
}
// newRProxyListener listens on SWW_RPROXY_LISTEN_ADDR.
func newRProxyListener(cfg *config.RProxy) (net.Listener, error) {
return listen(cfg.ListenAddr, "SWW_RPROXY_LISTEN_ADDR")
}
+4 -1
View File
@@ -51,7 +51,9 @@ func runRProxy(ctx context.Context, env map[string]string, out *output) int {
// startRProxy runs rproxy with waf at backend and the app at appURL, and
// returns its URL and what it writes. It believes X-Forwarded-For from the
// tests. It stops as the test ends, and that it stopped in order is checked.
// tests. It waits a minute for waf's answer, so that a slow answer on a busy
// host is not taken for no answer. It stops as the test ends, and that it
// stopped in order is checked.
func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
t.Helper()
@@ -65,6 +67,7 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
"SWW_RPROXY_UPSTREAM_URL": appURL,
"SWW_RPROXY_TRUSTED_PROXIES": localhost + "/32",
wafBackend: backend,
wafBackendTimeout: "60000",
}, out)
}()
+5 -394
View File
@@ -4,17 +4,14 @@
// stop, and then stops in an orderly way, writing the state and cache files.
// `sww rproxy` reads rproxy's settings alone, and runs the reverse proxy in
// front of one app that asks waf about each request. `sww healthcheck` is the
// container's health check.
// container's health check. fx builds the parts of each daemon, starts their
// background work and stops it.
package sww
import (
"context"
"errors"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"os"
"os/signal"
"syscall"
@@ -22,29 +19,9 @@ import (
"github.com/spf13/viper"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/remotelog"
"sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/state"
"sneak.berlin/go/smallwebwaf/internal/waf"
)
// shutdownTimeout is how long requests in progress may take to finish
// once sww is told to stop, before their connections are closed.
// runit and docker wait a little longer before they kill the process.
const shutdownTimeout = 5 * time.Second
// remoteLogStopTimeout is how long, as waf stops, the log lines still
// waiting are sent to SWW_WAF_LOG_REMOTE_URL before they are given up. stdout has
// carried them.
const remoteLogStopTimeout = 2 * time.Second
// usageStatus is the exit status with no command, or one sww does not
// know.
const usageStatus = 2
@@ -67,6 +44,9 @@ type Params struct {
Viper *viper.Viper
// Stdout receives the request log and the process's own messages.
Stdout io.Writer
// BesideLookupTimeLimit is the waf.Params.BesideLookupTimeLimit RunWAF
// gives waf. Main leaves it zero, waf's own limit; tests lengthen it.
BesideLookupTimeLimit time.Duration
}
// Main runs the command os.Args names, and returns the process's exit status.
@@ -107,372 +87,3 @@ func printUsage(w io.Writer) int {
return usageStatus
}
// RunWAF is `sww waf`: it reads the settings, the rule files,
// the lookup database and the state and cache files, then serves requests until
// ctx is done. It returns the process's exit status, 1 when waf cannot start.
func RunWAF(ctx context.Context, params Params) int {
// Until the settings are read, the one message is an invalid setting's
// error, which every SWW_WAF_LOG_LEVEL lets through.
processLog := requestlog.NewProcessLogger(params.Stdout,
config.InstanceName(params.Viper), slog.LevelError)
cfg, err := config.WAFFromEnvironment(params.Viper)
if err != nil {
return cannotStart(processLog, "invalid setting", err)
}
// While SWW_WAF_LOG_REMOTE_URL is set, every line on stdout from here on
// is sent there too.
stdout := params.Stdout
var remote *remotelog.Sender
if cfg.LogRemoteURL != nil {
remote = newRemoteLogSender(cfg)
stdout = io.MultiWriter(params.Stdout, remote)
}
processLog = requestlog.NewProcessLogger(
stdout,
cfg.InstanceName,
cfg.LogLevel,
)
if remote != nil {
stopSending := startSending(ctx, remote, processLog)
defer stopSending()
}
return runWAFWithSettings(
ctx,
params.Version,
cfg,
stdout,
remote,
processLog,
)
}
// runWAFWithSettings is the rest of RunWAF once the settings are read and
// the process log goes where they say. remote is nil unless
// SWW_WAF_LOG_REMOTE_URL is set.
func runWAFWithSettings(
ctx context.Context,
version string,
cfg *config.WAF,
stdout io.Writer,
remote *remotelog.Sender,
processLog *slog.Logger,
) int {
// The state files and the alerts give times in UTC.
now := func() time.Time { return time.Now().UTC() }
alertQueue := newAlertQueue(cfg, now, processLog)
ruleFiles, err := rules.Load(rules.Params{
Dir: cfg.RuleFilesDir,
Enabled: cfg.RuleFilesEnabled,
ProcessLog: processLog,
Alerts: alertQueue,
})
if err != nil {
return cannotStart(processLog, "cannot use the rule files", err)
}
crs, err := waf.NewCoreRuleSet(cfg)
if err != nil {
return cannotStart(processLog, "cannot use the Core Rule Set", err)
}
server, err := newServer(
cfg,
stdout,
processLog,
now,
ruleFiles,
alertQueue,
crs,
)
if err != nil {
return cannotStart(processLog, "cannot use the lookup database", err)
}
if remote != nil {
server.Metrics.AddRemoteLog(remote)
}
files, err := loadStateFiles(cfg, server, alertQueue, now, processLog)
if err != nil {
return cannotStart(
processLog,
"cannot use the state and cache files",
err,
)
}
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
if err != nil {
return cannotStart(
processLog,
"cannot listen on SWW_WAF_LISTEN_ADDR",
err,
)
}
processLog.Info("starting",
"version", version,
"address", listener.Addr().String(),
"settings", cfg)
return serve(
ctx,
server,
listener,
files,
ruleFiles,
alertQueue,
processLog,
)
}
// cannotStart logs why waf cannot start, and returns its exit status.
func cannotStart(processLog *slog.Logger, message string, err error) int {
processLog.Error(message, "error", err.Error())
return 1
}
// newServer returns the server waf runs, with the metrics of the alerts,
// after reading the lookup database while SWW_WAF_LOOKUP_SOURCE is file. A lookup
// database that cannot be read is an error.
func newServer(
cfg *config.WAF, stdout io.Writer, processLog *slog.Logger,
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
crs *coreruleset.CoreRuleSet,
) (*waf.Server, error) {
var lookupFile *lookup.File
if cfg.LookupSource == "file" {
var err error
lookupFile, err = lookup.OpenFile(lookup.FileParams{
Path: cfg.LookupDBPath,
Now: now,
ProcessLog: processLog,
Alerts: alertQueue,
})
if err != nil {
return nil, err
}
}
server := waf.New(waf.Params{
Config: cfg,
RequestLog: stdout,
ProcessLog: processLog,
GeoJSURL: geojs.URL,
AbuseIPDBURL: reputation.AbuseIPDBURL,
LookupFile: lookupFile,
CoreRuleSet: crs,
Now: now,
Rules: ruleFiles,
Alerts: alertQueue,
})
server.Metrics.AddAlerts(alertQueue)
if lookupFile != nil {
server.Metrics.AddLookupFile(
lookupFile.LastRead,
lookupFile.ReadFailures,
)
}
return server, nil
}
// loadStateFiles reads the state and cache files into the parts of server and
// into alertQueue, as state.Load does.
func loadStateFiles(
cfg *config.WAF, server *waf.Server, alertQueue *alerts.Queue,
now func() time.Time, processLog *slog.Logger,
) (*state.Files, error) {
return state.Load(state.Params{
Dir: cfg.StateDir,
CacheDir: cfg.CacheDir,
BansWriteDelay: cfg.BansWriteDelay,
WriteInterval: cfg.StateWriteInterval,
Ledger: server.Ledger,
Limiter: server.Limiter,
GeoJS: server.GeoJS,
Lists: server.Lists,
DNSBL: server.DNSBL,
AbuseIPDB: server.AbuseIPDB,
RouteWatch: server.RouteWatch,
Alerts: alertQueue,
Anomalies: server.Anomalies,
Now: now,
ProcessLog: processLog,
Metrics: server.Metrics,
})
}
// newAlertQueue returns the queue of the alerts to the webhook, Slack and
// ntfy, with the settings for them.
func newAlertQueue(
cfg *config.WAF, now func() time.Time, processLog *slog.Logger,
) *alerts.Queue {
return alerts.New(alerts.Params{
WebhookURL: cfg.AlertWebhookURL,
WebhookHeaders: cfg.AlertWebhookHeaders,
SlackURL: cfg.AlertSlackWebhookURL,
NtfyURL: cfg.AlertNtfyURL,
NtfyToken: cfg.AlertNtfyToken,
Events: cfg.AlertEvents,
Cooldown: cfg.AlertCooldown,
MaxPerHour: cfg.AlertMaxPerHour,
Instance: cfg.InstanceName,
Now: now,
ProcessLog: processLog,
})
}
// newRemoteLogSender returns a sender of the log lines to
// SWW_WAF_LOG_REMOTE_URL, with the settings for it.
func newRemoteLogSender(cfg *config.WAF) *remotelog.Sender {
return remotelog.New(remotelog.Params{
URL: cfg.LogRemoteURL,
RootCAs: cfg.LogRemoteTLSCAs,
BufferLines: cfg.LogRemoteBufferLines,
Facility: cfg.LogRemoteFacility,
AppName: cfg.LogRemoteAppName,
})
}
// startSending runs remote until the function it returns is called, which
// then waits at most remoteLogStopTimeout for the lines still waiting to
// be sent. Sending goes on after ctx is done, so that the lines written
// while sww stops are sent too.
func startSending(
ctx context.Context, remote *remotelog.Sender, processLog *slog.Logger,
) func() {
sending, stop := context.WithCancel(context.WithoutCancel(ctx))
sent := make(chan struct{})
go func() {
remote.Run(sending, processLog)
close(sent)
}()
return func() {
stop()
select {
case <-sent:
case <-time.After(remoteLogStopTimeout):
}
}
}
// serve serves requests on listener, writes the state and cache files as they
// are due, takes in an admin's edits of them, reads the rule files again as
// they change, and the lookup database when it is replaced, fetches the
// lists the settings name by URL as they are due, and sends the alerts,
// until ctx is done. Then it gives the requests in progress
// shutdownTimeout to finish, and writes every state and cache file,
// alerts.json with the alerts still waiting.
func serve(
ctx context.Context, server *waf.Server, listener net.Listener,
files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue,
processLog *slog.Logger,
) int {
served := make(chan error, 1)
go func() {
served <- server.Serve(listener)
}()
writing, stopWriting := context.WithCancel(ctx)
defer stopWriting()
written := inBackground(func() { files.Run(writing) })
watched := inBackground(func() { files.Watch(writing) })
rulesWatched := inBackground(func() { ruleFiles.Watch(writing) })
lookupFileWatched := inBackground(func() {
if server.LookupFile != nil {
server.LookupFile.Watch(writing)
}
})
listsFetched := inBackground(func() { server.Lists.Run(writing) })
alertsSent := inBackground(func() { alertQueue.Run(writing) })
select {
case err := <-served:
processLog.Error("serving failed", "error", err.Error())
return 1
case <-ctx.Done():
}
processLog.Info("stopping")
shutdownCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx),
shutdownTimeout)
defer cancel()
err := server.Shutdown(shutdownCtx)
if err != nil {
processLog.Warn("requests still in progress were cut off",
"error", err.Error())
_ = server.Close()
}
err = <-served
if !errors.Is(err, http.ErrServerClosed) {
processLog.Error("serving failed", "error", err.Error())
return 1
}
// Run and Watch have ended, so nothing else reads or writes the
// files, and no alert is being sent, so that alerts.json keeps every
// alert not yet sent. Every request has ended too, but for one cut off
// because Shutdown timed out, which adds to its client's history only as
// it ends. That can be after this write, and then that request is missing
// from clients.json.
<-written
<-watched
<-rulesWatched
<-lookupFileWatched
<-listsFetched
<-alertsSent
err = files.WriteAll()
if err != nil {
processLog.Error(
"writing the state and cache files failed",
"error",
err.Error(),
)
return 1
}
processLog.Info("stopped")
return 0
}
// inBackground runs task on a goroutine of its own, and returns a channel
// that is closed once task has returned.
func inBackground(task func()) <-chan struct{} {
done := make(chan struct{})
go func() {
task()
close(done)
}()
return done
}
+3
View File
@@ -173,6 +173,9 @@ func run(ctx context.Context, env map[string]string, out *output) int {
Version: testVersion,
Viper: v,
Stdout: out,
// The race detector and a busy host can slow the Core Rule Set's
// inspections past waf's own limit.
BesideLookupTimeLimit: time.Hour,
})
}
+490
View File
@@ -0,0 +1,490 @@
package sww
import (
"context"
"io"
"log/slog"
"net"
"net/http"
"time"
"go.uber.org/fx"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/remotelog"
"sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/routewatch"
"sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/state"
"sneak.berlin/go/smallwebwaf/internal/waf"
)
// remoteLogStopTimeout is how long, as waf stops, the log lines still
// waiting are sent to SWW_WAF_LOG_REMOTE_URL before they are given up. stdout has
// carried them.
const remoteLogStopTimeout = 2 * time.Second
// RunWAF is `sww waf`: it reads the settings, the rule files,
// the lookup database and the state and cache files, then serves requests until
// ctx is done. Then it gives the requests in progress shutdownTimeout to
// finish, ends its background work and writes every state and cache file,
// alerts.json with the alerts still waiting. It returns the process's exit
// status, 1 when waf cannot start or stop in order.
func RunWAF(ctx context.Context, params Params) int {
// Until the settings are read, the one message is an invalid setting's
// error, which every SWW_WAF_LOG_LEVEL lets through.
processLog := requestlog.NewProcessLogger(params.Stdout,
config.InstanceName(params.Viper), slog.LevelError)
cfg, err := config.WAFFromEnvironment(params.Viper)
if err != nil {
return cannotStart(processLog, startError{"invalid setting", err})
}
// While SWW_WAF_LOG_REMOTE_URL is set, every line on stdout from here on
// is sent there too. The sender runs before any part is built, so that it
// sends why waf cannot start.
stdout := params.Stdout
var remote *remotelog.Sender
if cfg.LogRemoteURL != nil {
remote = newRemoteLogSender(cfg)
stdout = io.MultiWriter(params.Stdout, remote)
}
processLog = requestlog.NewProcessLogger(
stdout,
cfg.InstanceName,
cfg.LogLevel,
)
if remote != nil {
stopSending := startSending(ctx, remote, processLog)
defer stopSending()
}
var (
files *state.Files
listener net.Listener
)
failed := make(servingFailed, 1)
app := fx.New(
fx.Supply(
params,
cfg,
processLog,
requestLog{stdout},
remote,
failed,
),
fx.Provide(
newAlertQueue, newRuleFiles, newCoreRuleSet, newLookupFile,
newLimiter, newLedger, newAnomalyCounters,
newLists, newDNSBL, newAbuseIPDB, newRouteWatch,
newMetrics, newGeoJS, newWAFServer,
func(server *waf.Server) *http.Server { return server.Server },
newWAFListener, newStateFiles,
),
// fx stops hooks in the reverse of the order they were added in. The
// state files are read before serving is set up, so that serving
// stops before their loops do.
fx.Populate(&files),
fx.Invoke(serve),
fx.Populate(&listener),
fx.WithLogger(fxEventLogger(processLog)),
)
err = app.Err()
if err != nil {
return cannotStart(processLog, err)
}
processLog.Info("starting",
"version", params.Version,
"address", listener.Addr().String(),
"settings", cfg)
// The last write follows the stop of every hook, and so the end of every
// loop that changes the state or raises an alert. A hook of the state
// files would stop before the loops of the parts built ahead of them.
return run(ctx, app, failed, processLog, func() error {
err := files.WriteAll()
if err != nil {
processLog.Error("writing the state and cache files failed",
"error", err.Error())
}
return err
})
}
// newAlertQueue returns the queue of the alerts to the webhook, Slack and
// ntfy, with the settings for them. Until waf stops, it sends them.
func newAlertQueue(
lc fx.Lifecycle, cfg *config.WAF, processLog *slog.Logger,
) *alerts.Queue {
alertQueue := alerts.New(alerts.Params{
WebhookURL: cfg.AlertWebhookURL,
WebhookHeaders: cfg.AlertWebhookHeaders,
SlackURL: cfg.AlertSlackWebhookURL,
NtfyURL: cfg.AlertNtfyURL,
NtfyToken: cfg.AlertNtfyToken,
Events: cfg.AlertEvents,
Cooldown: cfg.AlertCooldown,
MaxPerHour: cfg.AlertMaxPerHour,
Instance: cfg.InstanceName,
Now: utcNow,
ProcessLog: processLog,
})
lc.Append(background(alertQueue.Run))
return alertQueue
}
// newRuleFiles reads the rule files, and reads them again as they change
// until waf stops.
func newRuleFiles(
lc fx.Lifecycle, cfg *config.WAF, processLog *slog.Logger,
alertQueue *alerts.Queue,
) (*rules.Files, error) {
ruleFiles, err := rules.Load(rules.Params{
Dir: cfg.RuleFilesDir,
Enabled: cfg.RuleFilesEnabled,
ProcessLog: processLog,
Alerts: alertQueue,
})
if err != nil {
return nil, startError{"cannot use the rule files", err}
}
lc.Append(background(ruleFiles.Watch))
return ruleFiles, nil
}
// newCoreRuleSet returns the Core Rule Set, as waf.NewCoreRuleSet does.
func newCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
crs, err := waf.NewCoreRuleSet(cfg)
if err != nil {
return nil, startError{"cannot use the Core Rule Set", err}
}
return crs, nil
}
// newLookupFile reads the lookup database while SWW_WAF_LOOKUP_SOURCE is
// file, and reads it again when it is replaced until waf stops. It returns
// nil while the source is anything else.
func newLookupFile(
lc fx.Lifecycle, cfg *config.WAF, processLog *slog.Logger,
alertQueue *alerts.Queue,
) (*lookup.File, error) {
if cfg.LookupSource != "file" {
return nil, nil //nolint:nilnil // no lookup database, and no error
}
lookupFile, err := lookup.OpenFile(lookup.FileParams{
Path: cfg.LookupDBPath,
Now: utcNow,
ProcessLog: processLog,
Alerts: alertQueue,
})
if err != nil {
return nil, startError{"cannot use the lookup database", err}
}
lc.Append(background(lookupFile.Watch))
return lookupFile, nil
}
// newLimiter returns the table of clients, with the rate and byte limits.
func newLimiter(cfg *config.WAF) *ratelimit.Limiter {
return ratelimit.New(ratelimit.Limits{
PerMinute: cfg.RateLimitPerMinute,
PerHour: cfg.RateLimitPerHour,
PerDay: cfg.RateLimitPerDay,
BytesPerMinute: cfg.BytesLimitPerMinute,
BytesPerHour: cfg.BytesLimitPerHour,
BytesPerDay: cfg.BytesLimitPerDay,
}, cfg.MaxTrackedClients)
}
// newLedger returns the ledger of the bans, with the settings for them.
func newLedger(cfg *config.WAF) *bans.Ledger {
return bans.New(bans.Params{
LimitBanDuration: cfg.LimitBanDuration,
LimitBanRepeatWindow: cfg.LimitBanRepeatWindow,
LimitBanPermanentAbove: cfg.LimitBanPermanentAbove,
AttackBanDuration: cfg.AttackBanDuration,
MaxBans: cfg.MaxBans,
})
}
// newAnomalyCounters returns the counters of the anomaly thresholds.
func newAnomalyCounters(
cfg *config.WAF, alertQueue *alerts.Queue,
) *anomaly.Counters {
return anomaly.New(anomaly.Params{
Client: cfg.AnomalyClient,
Net: cfg.AnomalyNet,
ASN: cfg.AnomalyASN,
Total: cfg.AnomalyTotal,
WatchedNet: cfg.AnomalyWatchedNet,
NetV4Prefix: cfg.AnomalyNetV4Prefix,
NetV6Prefix: cfg.AnomalyNetV6Prefix,
NamedNetblocks: cfg.AnomalyWatchedNets,
Alerts: alertQueue,
})
}
// newLists returns the lists the settings name by URL, none fetched yet.
// Until waf stops, it fetches them as they are due.
func newLists(
lc fx.Lifecycle, cfg *config.WAF, processLog *slog.Logger,
alertQueue *alerts.Queue,
) *reputation.Lists {
lists := reputation.New(reputation.Params{
BlocklistURLs: cfg.BlocklistURLs,
Refresh: cfg.BlocklistRefresh,
ASNLimitPercentURL: cfg.ASNLimitPercentURL,
TagLists: cfg.TagLists,
Now: utcNow,
ProcessLog: processLog,
Alerts: alertQueue,
})
lc.Append(background(lists.Run))
return lists
}
// newDNSBL returns what asks the DNSBL zones, none asked yet.
func newDNSBL(
cfg *config.WAF, processLog *slog.Logger, alertQueue *alerts.Queue,
) *reputation.DNSBL {
return reputation.NewDNSBL(reputation.DNSBLParams{
Zones: cfg.DNSBLZones,
Resolver: cfg.DNSBLResolver,
CacheTTL: cfg.IPReputationCacheTTL,
Timeout: cfg.DNSBLAbuseIPDBTimeout,
Now: utcNow,
ProcessLog: processLog,
Alerts: alertQueue,
})
}
// newAbuseIPDB returns what checks clients with AbuseIPDB while
// SWW_WAF_ABUSEIPDB_KEY is set, none checked yet.
func newAbuseIPDB(
cfg *config.WAF, processLog *slog.Logger, alertQueue *alerts.Queue,
) *reputation.AbuseIPDB {
return reputation.NewAbuseIPDB(reputation.AbuseIPDBParams{
URL: reputation.AbuseIPDBURL,
Key: cfg.AbuseIPDBKey,
MinScore: cfg.AbuseIPDBMinScore,
DailyBudget: cfg.AbuseIPDBDailyBudget,
CacheTTL: cfg.IPReputationCacheTTL,
Timeout: cfg.DNSBLAbuseIPDBTimeout,
Now: utcNow,
ProcessLog: processLog,
Alerts: alertQueue,
})
}
// newRouteWatch returns what asks routewatch about clients, or nil while
// SWW_WAF_ROUTEWATCH_URL is unset.
func newRouteWatch(
cfg *config.WAF, processLog *slog.Logger,
) *routewatch.Client {
if cfg.RouteWatchURL == "" {
return nil
}
return routewatch.New(routewatch.Params{
URL: cfg.RouteWatchURL,
Now: utcNow,
ProcessLog: processLog,
})
}
// newMetrics returns the metrics, with those read from the other parts as
// the metrics are asked for: AbuseIPDB's while SWW_WAF_ABUSEIPDB_KEY is set,
// the lookup database's while there is one, and the remote log sender's
// while there is one.
func newMetrics(
cfg *config.WAF, ledger *bans.Ledger, limiter *ratelimit.Limiter,
ruleFiles *rules.Files, lists *reputation.Lists,
dnsbl *reputation.DNSBL, abuseIPDB *reputation.AbuseIPDB,
alertQueue *alerts.Queue, lookupFile *lookup.File,
remote *remotelog.Sender,
) *metrics.Metrics {
m := metrics.New(cfg.MetricsBusiestCountriesAndASNs, cfg.InstanceName)
m.AddBansAndClients(ledger, limiter, utcNow)
m.AddRules(ruleFiles)
m.AddReputation(lists, dnsbl)
if cfg.AbuseIPDBKey != "" {
m.AddAbuseIPDB(abuseIPDB)
}
m.AddAlerts(alertQueue)
if lookupFile != nil {
m.AddLookupFile(lookupFile.LastRead, lookupFile.ReadFailures)
}
if remote != nil {
m.AddRemoteLog(remote)
}
return m
}
// newGeoJS returns what looks clients up at GeoJS, none looked up yet. Each
// answer goes to the limiter and the ledger, as waf.AddLookup says, and each
// failure is handled as waf.GeoJSFailed says.
func newGeoJS(
cfg *config.WAF, m *metrics.Metrics, limiter *ratelimit.Limiter,
ledger *bans.Ledger, alertQueue *alerts.Queue, processLog *slog.Logger,
) *geojs.GeoJS {
return geojs.New(geojs.Params{
URL: geojs.URL,
Answered: func(answer geojs.Answer) {
waf.AddLookup(cfg, limiter, ledger, answer)
},
Now: utcNow,
Asked: m.GeoJSRequests.Inc,
Failed: func(err error, askingAgainIn time.Duration) {
waf.GeoJSFailed(processLog, alertQueue, m, err, askingAgainIn)
},
WentWithoutAnswer: m.RequestsWithoutGeoJSAnswer.Inc,
})
}
// newWAFServer returns the server waf runs, with its parts and the time
// limit params gives.
func newWAFServer(
params Params, cfg *config.WAF, out requestLog, processLog *slog.Logger,
m *metrics.Metrics, limiter *ratelimit.Limiter, ledger *bans.Ledger,
anomalies *anomaly.Counters, lists *reputation.Lists,
dnsbl *reputation.DNSBL, abuseIPDB *reputation.AbuseIPDB,
routeWatch *routewatch.Client, geoJS *geojs.GeoJS,
lookupFile *lookup.File, crs *coreruleset.CoreRuleSet,
ruleFiles *rules.Files, alertQueue *alerts.Queue,
) *waf.Server {
return waf.New(waf.Params{
Config: cfg,
RequestLog: out,
ProcessLog: processLog,
Metrics: m,
Limiter: limiter,
Ledger: ledger,
Anomalies: anomalies,
Lists: lists,
DNSBL: dnsbl,
AbuseIPDB: abuseIPDB,
RouteWatch: routeWatch,
GeoJS: geoJS,
LookupFile: lookupFile,
CoreRuleSet: crs,
Now: utcNow,
Rules: ruleFiles,
Alerts: alertQueue,
BesideLookupTimeLimit: params.BesideLookupTimeLimit,
})
}
// newWAFListener listens on SWW_WAF_LISTEN_ADDR.
func newWAFListener(cfg *config.WAF) (net.Listener, error) {
return listen(cfg.ListenAddr, "SWW_WAF_LISTEN_ADDR")
}
// newStateFiles reads the state and cache files into the parts and the
// alert queue, as state.Load does. Until waf stops, it writes them as they
// are due and takes in an admin's edits of them.
func newStateFiles(
lc fx.Lifecycle, cfg *config.WAF, processLog *slog.Logger,
m *metrics.Metrics, ledger *bans.Ledger, limiter *ratelimit.Limiter,
geoJS *geojs.GeoJS, lists *reputation.Lists, dnsbl *reputation.DNSBL,
abuseIPDB *reputation.AbuseIPDB, routeWatch *routewatch.Client,
alertQueue *alerts.Queue, anomalies *anomaly.Counters,
) (*state.Files, error) {
files, err := state.Load(state.Params{
Dir: cfg.StateDir,
CacheDir: cfg.CacheDir,
BansWriteDelay: cfg.BansWriteDelay,
WriteInterval: cfg.StateWriteInterval,
Ledger: ledger,
Limiter: limiter,
GeoJS: geoJS,
Lists: lists,
DNSBL: dnsbl,
AbuseIPDB: abuseIPDB,
RouteWatch: routeWatch,
Alerts: alertQueue,
Anomalies: anomalies,
Now: utcNow,
ProcessLog: processLog,
Metrics: m,
})
if err != nil {
return nil, startError{"cannot use the state and cache files", err}
}
lc.Append(background(files.Run))
lc.Append(background(files.Watch))
return files, nil
}
// newRemoteLogSender returns a sender of the log lines to
// SWW_WAF_LOG_REMOTE_URL, with the settings for it.
func newRemoteLogSender(cfg *config.WAF) *remotelog.Sender {
return remotelog.New(remotelog.Params{
URL: cfg.LogRemoteURL,
RootCAs: cfg.LogRemoteTLSCAs,
BufferLines: cfg.LogRemoteBufferLines,
Facility: cfg.LogRemoteFacility,
AppName: cfg.LogRemoteAppName,
})
}
// startSending runs remote until the function it returns is called, which
// then waits at most remoteLogStopTimeout for the lines still waiting to
// be sent. Sending goes on after ctx is done, so that the lines written
// while sww stops are sent too.
func startSending(
ctx context.Context, remote *remotelog.Sender, processLog *slog.Logger,
) func() {
sending, stop := context.WithCancel(context.WithoutCancel(ctx))
sent := make(chan struct{})
go func() {
remote.Run(sending, processLog)
close(sent)
}()
return func() {
stop()
select {
case <-sent:
case <-time.After(remoteLogStopTimeout):
}
}
}
+1 -1
View File
@@ -35,7 +35,7 @@ func (abuseIPDB) check(
return nil
}
client := rq.h.clientGroup(rq.client)
client := clientGroup(rq.h.config, rq.client)
held, _ := rq.h.limiter.Client(client)
offender := held.History.Offences != ratelimit.Offences{}
+5 -5
View File
@@ -139,7 +139,7 @@ func (rq *request) addBan() {
)
if err == nil {
netblock, err = rq.h.banNetblock(toAdd.Netblock)
netblock, err = rq.h.parseBanNetblock(toAdd.Netblock)
}
if err == nil {
@@ -190,12 +190,12 @@ func (rq *request) readBanToAdd() (banToAdd, error) {
return toAdd, nil
}
// banNetblock reads value, a netblock such as 203.0.113.0/24, or a
// parseBanNetblock reads value, a netblock such as 203.0.113.0/24, or a
// client's address, which stands for the netblock a ban on that client
// covers. An IPv4-mapped netblock, such as ::ffff:203.0.113.0/120, is
// refused, since a client's address is looked up as IPv4 and a ban on it
// would refuse nothing, and so is a value with a zone.
func (h *handler) banNetblock(value string) (netip.Prefix, error) {
func (h *handler) parseBanNetblock(value string) (netip.Prefix, error) {
netblock, err := netip.ParsePrefix(value)
if err == nil {
if netblock.Addr().Is4In6() {
@@ -224,7 +224,7 @@ func (h *handler) banNetblock(value string) (netip.Prefix, error) {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errZone)
}
return h.netblock(addr), nil
return banNetblock(h.config, addr), nil
}
// expiry returns when a ban made at now for duration ends: duration
@@ -295,7 +295,7 @@ func (rq *request) showClient() {
Bans: state.BanEntries(rq.h.ledger.Covering(addr)),
}
client, seen := rq.h.limiter.Client(rq.h.clientGroup(addr))
client, seen := rq.h.limiter.Client(clientGroup(rq.h.config, addr))
if seen {
answer.Client = &client
}
+3 -1
View File
@@ -19,6 +19,8 @@ const (
alertMaxPerHour = "SWW_WAF_ALERT_MAX_PER_HOUR"
// alertInstance is the instance every alert of these tests gives.
alertInstance = "host1/blog"
// alertWebhook is where the tests send alerts.
alertWebhook = "https://alerts.example/sww"
)
func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
@@ -270,7 +272,7 @@ func startAppWithAlerts(
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)}
settings := map[string]string{
trustedProxies: trustLocalhost,
alertWebhookURL: "https://alerts.example/sww",
alertWebhookURL: alertWebhook,
instanceName: alertInstance,
}
maps.Copy(settings, env)
+7 -12
View File
@@ -877,18 +877,13 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
held, _, _ := startHeldGeoJS(t)
resolver, releaseResolver := startHeldResolver(t)
server, out, _ := newWAFWith(t, func(
params waf.Params,
) *waf.Server {
params.AbuseIPDBURL = silentAbuseIPDBURL
return waf.New(params)
}, silentGeoJSURL, time.Now, map[string]string{
tagDeny: "country=KP", coreRuleSetMode: block,
dnsblZones: dnsblZone, dnsblResolver: resolver,
abuseIPDBKey: accountKey,
blocklistURLs: held + "/drop.txt", asnLimitPercentURL: held + "/asn.txt",
})
server, out, _ := newWAFWith(t, waf.New,
silentGeoJSURL, silentAbuseIPDBURL, time.Now, map[string]string{
tagDeny: "country=KP", coreRuleSetMode: block,
dnsblZones: dnsblZone, dnsblResolver: resolver,
abuseIPDBKey: accountKey,
blocklistURLs: held + "/drop.txt", asnLimitPercentURL: held + "/asn.txt",
})
// The lists are fetched outside the synctest bubble below: a wait on the
// network inside it would keep its clock from moving on.
+8 -7
View File
@@ -10,6 +10,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
@@ -121,7 +122,7 @@ func (rq *request) banForLimit(
rq.line.Offence = requestlog.OffenceLimit
rq.h.metrics.LimitHit(hit)
netblock := rq.h.netblock(rq.client)
netblock := banNetblock(rq.h.config, rq.client)
if rq.h.config.Observe &&
!rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
return
@@ -157,7 +158,7 @@ func (rq *request) banForLimit(
}
ban, made := rq.h.ledger.BanForLimit(netblock, now, notes)
rq.h.limiter.Reset(rq.h.clientGroup(rq.client))
rq.h.limiter.Reset(clientGroup(rq.h.config, rq.client))
rq.line.BanExpires = banExpires(ban)
if made {
@@ -171,7 +172,7 @@ func (rq *request) banForLimit(
// no ban, and raises the alert for the ban it would have made, if that
// alert would be sent. It returns the action for that ban.
func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
netblock := rq.h.netblock(rq.client)
netblock := banNetblock(rq.h.config, rq.client)
if rq.h.config.Observe &&
!rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
return banAction(
@@ -273,16 +274,16 @@ func (rq *request) netblockRequests(netblock netip.Prefix) int64 {
return rq.h.limiter.Requests(netblock) + 1
}
// netblock is the netblock a ban on client covers: its IPv4 address,
// banNetblock is the netblock a ban on client covers: its IPv4 address,
// widened to SWW_WAF_BAN_SCOPE_V4_PREFIX, or the IPv6 group clientGroup
// counts it in.
func (h *handler) netblock(client netip.Addr) netip.Prefix {
func banNetblock(cfg *config.WAF, client netip.Addr) netip.Prefix {
addr := client.Unmap()
if addr.Is4() {
return netip.PrefixFrom(addr, h.config.BanScopeV4Prefix).Masked()
return netip.PrefixFrom(addr, cfg.BanScopeV4Prefix).Masked()
}
return h.clientGroup(addr)
return clientGroup(cfg, addr)
}
// banExpires is when ban ends, as the log line gives it: a time, or
+172
View File
@@ -0,0 +1,172 @@
package waf
// The heuristics of besideLookup, and how they run beside the lookup.
import (
"context"
"sync/atomic"
"time"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
)
// besideLookupTimeLimit is how long the heuristics of besideLookup have for a
// request, from when they start. They only read the request, and no request
// should take them that long. A request they have not finished by then is
// refused with 503, as ActionChecksTimedOut.
const besideLookupTimeLimit = 500 * time.Millisecond
// besideLookupFindings is what the heuristics of besideLookup found of a
// request, before any of it is applied to the request.
type besideLookupFindings struct {
// trapPath is the request's path when it is in SWW_WAF_TRAP_PATHS, and ""
// otherwise.
trapPath string
// rules are the rules of the rule files the request matched.
rules []rules.Rule
// inspected is true when the Core Rule Set inspected the request.
// inspection, read and inspectionTime are then what it found, what it read
// of the body, and how long it took.
inspected bool
inspection coreruleset.Result
read []byte
inspectionTime time.Duration
// refuses is true when one of the heuristics refuses the request, or would
// in observe mode. Those after it did no work.
refuses bool
}
// runBesideLookup runs the heuristics of besideLookup while the lookup waits
// for GeoJS, and returns the first verdict, or nil when none gives one.
// lookedUp is the channel the lookup's result comes on, or nil once afterLookup
// has let the request through.
//
// The first of the two to refuse the request ends the checks: waf answers at
// once, and what the other found is dropped. besideLookup has
// besideLookupTimeLimit; past it, the request is refused as
// ActionChecksTimedOut. Once both have let the request through, the heuristics
// of afterBesideLookup run. ctx is the request's own context.
func (rq *request) runBesideLookup(
ctx context.Context, lookedUp <-chan lookupResult,
) *verdict {
var running atomic.Pointer[string]
running.Store(new(trapPaths{}.name()))
found := make(chan *besideLookupFindings, 1)
go func() { found <- rq.findBesideLookup(&running) }()
timeUp := time.NewTimer(rq.h.besideLookupLimit)
defer timeUp.Stop()
var findings *besideLookupFindings
for findings == nil || lookedUp != nil {
select {
case result := <-lookedUp:
lookedUp = nil
v := rq.runAfterLookup(ctx, result)
if v != nil {
return v
}
case findings = <-found:
found = nil
timeUp.Stop()
if findings.refuses {
return rq.applyFindings(ctx, findings)
}
case <-timeUp.C:
return rq.timedOut(*running.Load())
}
}
v := rq.applyFindings(ctx, findings)
if v != nil {
return v
}
return rq.runHeuristics(ctx, afterBesideLookup(), rq.h.now())
}
// findBesideLookup does the work of the heuristics of besideLookup on the
// request, in their order, until one refuses it, and returns what they found.
// It changes nothing of the request or of what waf keeps, so that it can run
// while the lookup and afterLookup do. running holds the name of the heuristic
// at work, the first one's when findBesideLookup is called.
func (rq *request) findBesideLookup(
running *atomic.Pointer[string],
) *besideLookupFindings {
findings := &besideLookupFindings{}
findings.trapPath = rq.trapPath()
if findings.trapPath != "" {
findings.refuses = true
return findings
}
running.Store(new(ruleFiles{}.name()))
findings.rules = rq.h.rules.Match(rq.in)
if refusedByRule(findings.rules) {
findings.refuses = true
return findings
}
running.Store(new(coreRuleSet{}.name()))
cfg := rq.h.config
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff ||
pathExempt(rq.in, cfg.CoreRuleSetExemptPaths) {
return findings
}
start := time.Now()
// A question's body has been read into memory, so that reading it here
// cannot fail.
findings.inspection, findings.read, _ = rq.h.coreRuleSet.Inspect(
rq.in, rq.client, rq.in.Body,
)
findings.inspectionTime = time.Since(start)
findings.inspected = true
findings.refuses = rq.refusedByCoreRuleSet(findings.inspection)
return findings
}
// applyFindings applies findings, what findBesideLookup found, to the
// request: it runs the heuristics of besideLookup, which read them, at the
// time it is called, and returns the first verdict, or nil when none gives
// one.
func (rq *request) applyFindings(
ctx context.Context, findings *besideLookupFindings,
) *verdict {
rq.findings = findings
return rq.runHeuristics(ctx, besideLookup(), rq.h.now())
}
// timedOut refuses the request for the heuristics of besideLookup not having
// finished with it within besideLookupTimeLimit. running is the name of the
// one at work, which the log line's heuristic gives. The refusal makes no ban
// and is no offence.
func (rq *request) timedOut(running string) *verdict {
v := &verdict{
action: requestlog.ActionChecksTimedOut,
figures: map[string]any{
"time_limit_ms": rq.h.besideLookupLimit.Milliseconds(),
},
}
rq.line.Heuristic = running
rq.line.HeuristicFigures = v.figures
return v
}
+426
View File
@@ -0,0 +1,426 @@
package waf_test
import (
"io"
"maps"
"net/http"
"net/http/httptest"
"net/netip"
"strconv"
"strings"
"sync/atomic"
"testing"
"testing/synctest"
"time"
"github.com/prometheus/client_golang/prometheus/testutil"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/waf"
)
// checksTimeLimit is how long the trap paths, the rule files and the Core Rule
// Set have for a request, as SPEC.md gives it, and checksTimedOut the status of
// a request they have not finished by then.
const (
checksTimeLimit = 500 * time.Millisecond
checksTimedOut = http.StatusServiceUnavailable
)
// heldCoreRuleSet is a stand-in for the Core Rule Set: each inspection takes
// took, then finds result. asked counts the inspections.
type heldCoreRuleSet struct {
took time.Duration
result coreruleset.Result
asked atomic.Int64
}
// Inspect inspects a request as the stand-in does, and reads all of body.
func (c *heldCoreRuleSet) Inspect(
_ *http.Request, _ netip.Addr, body io.Reader,
) (coreruleset.Result, []byte, error) {
c.asked.Add(1)
time.Sleep(c.took)
read, err := io.ReadAll(body)
return c.result, read, err
}
// coreRuleSetMatch returns what the stand-in finds of an attack: rule 942100,
// with a score of 5, the default SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD.
func coreRuleSetMatch() coreruleset.Result {
return coreruleset.Result{RuleIDs: []int{942100}, Score: 5}
}
// newWAFHoldingTheCoreRuleSet makes a waf, to be asked in a synctest bubble,
// with crs in place of the Core Rule Set, in block mode. GeoJS never answers,
// a setting needs its answer, alerts go to a webhook, and everything else is
// at its default but for env. No request to it may use the network.
func newWAFHoldingTheCoreRuleSet(
t *testing.T, crs *heldCoreRuleSet, env map[string]string,
) (*waf.Server, *output, *alerts.Queue) {
t.Helper()
settings := map[string]string{
tagDeny: denyKP, coreRuleSetMode: block,
alertWebhookURL: alertWebhook,
}
maps.Copy(settings, env)
server, out, queue := newWAFWith(t, func(params waf.Params) *waf.Server {
params.CoreRuleSet = crs
params.BesideLookupTimeLimit = waf.BesideLookupTimeLimit
return waf.New(params)
}, silentGeoJSURL, abuseIPDBURL, time.Now, settings)
return server, out, queue
}
// lookupWait is SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS for wait.
func lookupWait(wait time.Duration) string {
return strconv.FormatInt(wait.Milliseconds(), 10)
}
// wantAnswer checks the status of an answer and how long it took.
func wantAnswer(
t *testing.T, status int, took time.Duration,
wantStatus int, wantTook time.Duration, out *output,
) {
t.Helper()
if status != wantStatus || took != wantTook {
t.Errorf(
"answered %d after %s, want %d after %s; output:\n%s",
status, took, wantStatus, wantTook, out.text(),
)
}
}
// wantNoCoreRuleSetTrace checks that what the Core Rule Set found of a request
// left no trace: line has none of its fields, and server has counted no match
// of it and raised no alert for one.
func wantNoCoreRuleSetTrace(
t *testing.T, server *waf.Server, queue *alerts.Queue, line logLine,
) {
t.Helper()
if line.DurationCoreRuleSetMS != nil || line.CoreRuleSetScore != nil ||
line.CoreRuleSetRuleIDs != nil {
t.Errorf(
"log line has Core Rule Set time %v, score %v and rule ids %v, want none",
line.DurationCoreRuleSetMS,
line.CoreRuleSetScore,
line.CoreRuleSetRuleIDs,
)
}
recorder := httptest.NewRecorder()
server.Metrics.ServeHTTP(
recorder,
httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
waf.MetricsPath,
http.NoBody,
),
)
if strings.Contains(
recorder.Body.String(),
"sww_core_rule_set_matches_total{",
) {
t.Errorf("a Core Rule Set match is counted")
}
for _, alert := range queue.Snapshot().Waiting[alerts.DestinationWebhook] {
if alert.Event == alerts.EventCoreRuleSetMatch {
t.Errorf("alert raised: %+v", alert)
}
}
}
func TestTheLookupAndTheCoreRuleSetTakeTheLongerOfTheirTimesNotTheirSum(
t *testing.T,
) {
t.Parallel()
inspection := checksTimeLimit - time.Millisecond
for _, tc := range []struct {
name string
// wait is how long the request waits for GeoJS.
wait time.Duration
}{
{"the lookup waits longer", defaultLookupWait},
{"the inspection takes longer", inspection / 2},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
crs := &heldCoreRuleSet{took: inspection}
server, out, _ := newWAFHoldingTheCoreRuleSet(
t,
crs,
map[string]string{
asnAndCountryLookupTimeout: lookupWait(tc.wait),
},
)
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
status, took := askDirectly(
t,
server,
func(*http.Request) {},
)
wantAnswer(t, status, took, http.StatusOK,
max(tc.wait, inspection), out)
wantLine(t, out.requestLine(t), http.StatusOK,
requestlog.ActionForward)
})
})
}
}
// firstToRefuse is a case of
// TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered: a request
// under env, whose inspection by the Core Rule Set takes inspection, refused as
// action after took.
type firstToRefuse struct {
name string
env map[string]string
inspection time.Duration
took time.Duration
action string
}
// firstToRefuseCases are the cases of
// TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered.
func firstToRefuseCases() []firstToRefuse {
return []firstToRefuse{
{
name: "a trap path, before GeoJS answers",
env: map[string]string{trapPaths: "/", allowedCountries: "DE"},
took: 0, action: requestlog.ActionTemporarilyBanned,
},
{
name: "the Core Rule Set, before GeoJS answers",
env: map[string]string{allowedCountries: "DE"},
inspection: 100 * time.Millisecond,
took: 100 * time.Millisecond, action: requestlog.ActionBlockedByCoreRuleSet,
},
{
name: "the country list, while the Core Rule Set inspects",
env: map[string]string{
allowedCountries: "DE", asnAndCountryLookupTimeout: "100",
},
inspection: 400 * time.Millisecond,
took: 100 * time.Millisecond, action: requestlog.ActionCountryDenied,
},
{
name: "the rate limits, while the Core Rule Set inspects",
env: map[string]string{
rateLimitPerMinute: "2", unknownCountryLimitPercent: "50",
asnAndCountryLookupTimeout: "100",
},
inspection: 400 * time.Millisecond,
took: 100 * time.Millisecond, action: requestlog.ActionClientIsRateLimited,
},
}
}
func TestTheFirstOfTheLookupAndTheRequestChecksToRefuseIsAnswered(
t *testing.T,
) {
t.Parallel()
const forbidden = http.StatusForbidden
for _, tc := range firstToRefuseCases() {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
crs := &heldCoreRuleSet{
took: tc.inspection,
result: coreRuleSetMatch(),
}
server, out, queue := newWAFHoldingTheCoreRuleSet(t, crs, tc.env)
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
// Under the rate limits of the last case, the client's next
// request takes it over a limit only at the percentage of an
// unknown country, so it waits for GeoJS.
countClient(t, server, time.Now())
status, took := askDirectly(
t,
server,
func(*http.Request) {},
)
wantAnswer(t, status, took, forbidden, tc.took, out)
line := out.requestLine(t)
wantLine(t, line, forbidden, tc.action)
if tc.action == requestlog.ActionBlockedByCoreRuleSet {
return
}
if tc.action != requestlog.ActionClientIsRateLimited &&
line.Counts != (ratelimit.Counts{}) {
t.Errorf(
"the rate limits counted the request: %+v",
line.Counts,
)
}
// What the Core Rule Set finds once the request is answered is
// dropped.
time.Sleep(tc.inspection)
wantNoCoreRuleSetTrace(t, server, queue, line)
})
})
}
}
func TestARequestRefusedBeforeTheLookupStartsNoInspection(t *testing.T) {
t.Parallel()
crs := &heldCoreRuleSet{took: time.Second}
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs, nil)
synctest.Test(t, func(t *testing.T) {
banClient(t, server, time.Now())
status, took := askDirectly(t, server, func(*http.Request) {})
wantAnswer(t, status, took, http.StatusForbidden, 0, out)
requests := testutil.ToFloat64(server.Metrics.GeoJSRequests)
if asked := crs.asked.Load(); asked != 0 || requests != 0 {
t.Errorf(
"%d inspections and %v requests to GeoJS, want none",
asked,
requests,
)
}
})
}
func TestAClientInTheAllowNetsIsAllowedAtOnceAndNothingElseStarts(
t *testing.T,
) {
t.Parallel()
crs := &heldCoreRuleSet{took: time.Second, result: coreRuleSetMatch()}
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
map[string]string{allowNets: client, allowedCountries: "DE"})
synctest.Test(t, func(t *testing.T) {
status, took := askDirectly(t, server, func(*http.Request) {})
wantAnswer(t, status, took, http.StatusOK, 0, out)
requests := testutil.ToFloat64(server.Metrics.GeoJSRequests)
if asked := crs.asked.Load(); asked != 0 || requests != 0 {
t.Errorf(
"%d inspections and %v requests to GeoJS, want none",
asked,
requests,
)
}
})
}
func TestARequestTheCoreRuleSetHasNotFinishedWithIn500MillisecondsIsRefused(
t *testing.T,
) {
t.Parallel()
crs := &heldCoreRuleSet{took: checksTimeLimit + time.Second}
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
map[string]string{refusalsLimitPerMinute: "1"})
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
// Were it counted toward the refusal limit, the second refusal would
// ban the client, and the third request would be refused for the ban.
for i := range 3 {
status, took := askDirectly(t, server, func(req *http.Request) {
req.Header.Set(requestIDHeader, "id-"+strconv.Itoa(i))
})
wantAnswer(t, status, took, checksTimedOut, checksTimeLimit, out)
line := out.requestLines(t, i+1)[i]
wantLine(
t,
line,
checksTimedOut,
requestlog.ActionChecksTimedOut,
)
wantHeuristic(t, line, "core_rule_set", `{"time_limit_ms": 500}`)
if line.Offence != "" || line.BanExpires != "" {
t.Errorf(
"log line has offence %q and ban_expires %q, want neither",
line.Offence,
line.BanExpires,
)
}
}
if bans := server.Ledger.Snapshot(); len(bans) != 0 {
t.Errorf("bans %+v, want none", bans)
}
offences := historyOf(t, server, client).Offences
if offences != (ratelimit.Offences{}) {
t.Errorf("history counts the offences %+v, want none", offences)
}
time.Sleep(crs.took)
})
}
func TestObserveModeAllowsARequestTheCoreRuleSetHasNotFinishedWith(
t *testing.T,
) {
t.Parallel()
crs := &heldCoreRuleSet{took: checksTimeLimit + time.Second}
server, out, _ := newWAFHoldingTheCoreRuleSet(t, crs,
map[string]string{mode: observe})
synctest.Test(t, func(t *testing.T) {
defer waitForGeoJSRequestsToBeAbandoned()
status, took := askDirectly(t, server, func(*http.Request) {})
wantAnswer(t, status, took, http.StatusOK, checksTimeLimit, out)
line := out.requestLine(t)
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
if line.WouldAction != requestlog.ActionChecksTimedOut {
t.Errorf(
"log line has would_action %q, want %q",
line.WouldAction,
requestlog.ActionChecksTimedOut,
)
}
time.Sleep(crs.took)
})
}
+1 -1
View File
@@ -30,7 +30,7 @@ func (byteLimits) check(
}
counts, hit, over := rq.h.limiter.CountBytes(
rq.h.clientGroup(rq.client),
clientGroup(rq.h.config, rq.client),
now,
rq.countedBytes(),
rq.bytesLimitPercent.percent,
+4 -2
View File
@@ -5,6 +5,8 @@ import (
"net/http"
"net/netip"
"slices"
"sneak.berlin/go/smallwebwaf/internal/config"
)
// peerAddress is the address of the request's TCP peer.
@@ -45,10 +47,10 @@ func scheme(r *http.Request, isQuestion bool) string {
// address, or its IPv6 group, the netblock its IPv6 address is in of the
// length SWW_WAF_IPV6_GROUP_PREFIX sets, a /64 by default, since one abuser
// usually holds a whole /64. An IPv4 address in IPv6 form counts as IPv4.
func (h *handler) clientGroup(addr netip.Addr) netip.Prefix {
func clientGroup(cfg *config.WAF, addr netip.Addr) netip.Prefix {
addr = addr.Unmap()
if addr.Is6() {
return netip.PrefixFrom(addr, h.config.IPv6GroupPrefix).Masked()
return netip.PrefixFrom(addr, cfg.IPv6GroupPrefix).Masked()
}
return netip.PrefixFrom(addr, addr.BitLen())
+24 -16
View File
@@ -13,10 +13,11 @@ import (
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// coreRuleSet inspects the request with the Core Rule Set, unless
// SWW_WAF_CORE_RULE_SET_MODE is off or SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS
// exempts its path, as pathExempt decides, and notes the rules it matched and
// its score in the log line, and the rules in the metrics. A score at or over
// coreRuleSet takes what the Core Rule Set found of the request, which
// findBesideLookup inspected with it unless SWW_WAF_CORE_RULE_SET_MODE is off
// or SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS exempts its path, as pathExempt
// decides. It notes the rules the request matched and its score in the log
// line, and the rules in the metrics. A score at or over
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD is a match: it adds the tag
// exploitsignature, raises the core_rule_set_match alert, and in block mode
// refuses the request with 403, which is an offence its client's history
@@ -30,21 +31,17 @@ func (coreRuleSet) check(
rq *request,
_ time.Time,
) *verdict {
cfg := rq.h.config
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff ||
pathExempt(rq.in, cfg.CoreRuleSetExemptPaths) {
findings := rq.findings
if !findings.inspected {
return nil
}
start := time.Now()
// A question's body has been read into memory, so that reading it here
// cannot fail.
result, read, _ := rq.h.coreRuleSet.Inspect(rq.in, rq.client, rq.in.Body)
rq.requestBytes = int64(len(read))
cfg := rq.h.config
result := findings.inspection
rq.requestBytes = int64(len(findings.read))
rq.line.DurationCoreRuleSetMS = new(
requestlog.Milliseconds(time.Since(start)),
requestlog.Milliseconds(findings.inspectionTime),
)
rq.line.CoreRuleSetRuleIDs = result.RuleIDs
rq.line.CoreRuleSetScore = &result.Score
@@ -61,7 +58,7 @@ func (coreRuleSet) check(
rq.addTag(config.ExploitSignatureTag)
rq.alertCoreRuleSetMatch(result)
if cfg.CoreRuleSetMode == config.CoreRuleSetModeDetect {
if !rq.refusedByCoreRuleSet(result) {
return nil
}
@@ -76,6 +73,17 @@ func (coreRuleSet) check(
}
}
// refusedByCoreRuleSet reports whether coreRuleSet refuses a request the Core
// Rule Set found result of: whether the score is at or over
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD in block mode.
func (rq *request) refusedByCoreRuleSet(result coreruleset.Result) bool {
cfg := rq.h.config
threshold := cfg.CoreRuleSetScoreThreshold
return cfg.CoreRuleSetMode == config.CoreRuleSetModeBlock &&
threshold != 0 && result.Score >= threshold
}
// alertCoreRuleSetMatch raises the core_rule_set_match alert for the request,
// which the Core Rule Set scored at result, at or over
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD. Its detail gives the rule ids, the
@@ -100,7 +108,7 @@ func (rq *request) alertCoreRuleSetMatch(result coreruleset.Result) {
Event: alerts.EventCoreRuleSetMatch,
Host: rq.site,
Client: rq.client,
Netblock: rq.h.clientGroup(rq.client),
Netblock: clientGroup(rq.h.config, rq.client),
ASN: rq.line.ASN,
ASName: rq.line.ASName,
Country: rq.line.Country,
+4
View File
@@ -0,0 +1,4 @@
package waf
// BesideLookupTimeLimit is besideLookupTimeLimit.
const BesideLookupTimeLimit = besideLookupTimeLimit
+18 -3
View File
@@ -54,8 +54,6 @@ func beforeLookup() []heuristic {
}
// afterLookup are the heuristics that run once the lookup has been made.
// tag_deny runs twice: before the rate limits, on the tags known by then, and
// last, on the tag rule_files and core_rule_set can add.
func afterLookup() []heuristic {
return []heuristic{
countries{},
@@ -64,9 +62,26 @@ func afterLookup() []heuristic {
abuseIPDB{},
tagDeny{},
rateLimits{},
}
}
// besideLookup are the heuristics that only read the request. Their work runs
// beside the lookup and afterLookup, in findBesideLookup, which changes nothing
// of the request. Their check then applies what that work found, and runs only
// once waf takes it.
func besideLookup() []heuristic {
return []heuristic{
trapPaths{},
ruleFiles{},
coreRuleSet{},
}
}
// afterBesideLookup are the heuristics that run once afterLookup and
// besideLookup have both let the request through. tag_deny runs a second time,
// on the tag rule_files and core_rule_set can add.
func afterBesideLookup() []heuristic {
return []heuristic{
tagDeny{},
}
}
@@ -95,7 +110,7 @@ func (rq *request) runAfterAnswer(ctx context.Context) {
rq.runBanningHeuristics(ctx, afterAnswer())
}
// runHeuristics runs list, beforeLookup or afterLookup, on the request at now,
// runHeuristics runs list, one of those checkClient runs, on the request at now,
// and returns the first verdict, or nil when none gives one. The log line's
// heuristic and heuristic_figures name the heuristic that gave it, and give its
// figures. ctx is the request's own context.
+93 -37
View File
@@ -2,14 +2,18 @@ package waf
import (
"context"
"log/slog"
"net/http"
"net/netip"
"strings"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
)
// The headers in which an answer of allow gives rproxy the client's AS number
@@ -21,36 +25,80 @@ const (
countryHeader = "X-Client-Country"
)
// lookUp looks up the client's AS number and country, in the lookup
// database or through GeoJS, and notes them for the log line and as the
// request's asn= and country= tags, unless
// SWW_WAF_LOOKUP_SOURCE is off or the client is on a private, loopback or
// link-local address, which no lookup can place. The lookup database
// answers at once. With GeoJS, while a setting needs the answer, such as a
// country list or a biased threshold, a new client's request waits for it up
// to SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, unless refusedWhateverTheAnswer
// reports true. ctx is the request's own context.
func (rq *request) lookUp(ctx context.Context, now time.Time) {
// lookupResult is what the lookup gave for a request: whether its client was
// looked up, the answer, and whether the request skipped the wait for GeoJS.
type lookupResult struct {
lookedUp, skippedWait bool
answer geojs.Answer
}
// startLookUp starts looking up the client's AS number and country, in the
// lookup database or through GeoJS, unless SWW_WAF_LOOKUP_SOURCE is off or the
// client is on a private, loopback or link-local address, which no lookup can
// place. It returns the channel the result comes on. A result that needs no
// wait is on it when startLookUp returns: the lookup database's, an answer
// GeoJS gave earlier, or none. With GeoJS, while a setting needs the answer,
// such as a country list or a biased threshold, a new client's request waits
// for it up to SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, unless
// refusedWhateverTheAnswer reports true. The wait goes on to its end even once
// the request has been answered, so that it ends as it would have for a
// request that waited. startLookUp changes nothing of the request. ctx is the
// request's own context.
func (rq *request) startLookUp(
ctx context.Context, now time.Time,
) <-chan lookupResult {
result := make(chan lookupResult, 1)
if rq.h.config.LookupSource == "off" || !canBePlaced(rq.client) {
result <- lookupResult{}
return result
}
client := clientGroup(rq.h.config, rq.client)
if rq.h.config.LookupSource == "file" {
result <- lookupResult{
lookedUp: true, answer: rq.h.lookupFile.LookUp(client),
}
return result
}
answer := rq.h.geojs.LookUpWithoutWaiting(ctx, client)
refused := rq.refusedWhateverTheAnswer(now)
if !answer.Answered.IsZero() || refused || !rq.h.waitForGeoJS {
result <- lookupResult{
lookedUp: true,
skippedWait: refused && answer.Answered.IsZero(),
answer: answer,
}
return result
}
go func() {
result <- lookupResult{
lookedUp: true,
answer: rq.h.geojs.LookUp(context.WithoutCancel(ctx), client,
rq.h.config.ASNAndCountryLookupTimeout),
}
}()
return result
}
// noteLookup notes result for the log line and as the request's asn= and
// country= tags.
func (rq *request) noteLookup(result lookupResult) {
if !result.lookedUp {
return
}
client := rq.h.clientGroup(rq.client)
switch {
case rq.h.config.LookupSource == "file":
rq.lookupAnswer = rq.h.lookupFile.LookUp(client)
case rq.refusedWhateverTheAnswer(now):
rq.lookupAnswer = rq.h.geojs.LookUpWithoutWaiting(ctx, client)
rq.skippedLookupWait = rq.lookupAnswer.Answered.IsZero()
case !rq.h.waitForGeoJS:
rq.lookupAnswer = rq.h.geojs.LookUpWithoutWaiting(ctx, client)
default:
rq.lookupAnswer = rq.h.geojs.LookUp(ctx, client,
rq.h.config.ASNAndCountryLookupTimeout)
}
rq.lookedUp = true
rq.lookupAnswer = result.answer
rq.skippedLookupWait = result.skippedWait
rq.line.ASN = rq.lookupAnswer.ASN
rq.line.ASName = rq.lookupAnswer.ASName
rq.line.Country = rq.lookupAnswer.Country
@@ -84,22 +132,30 @@ func countryTag(country string) string {
return config.CountryTagPrefix + country
}
// addLookup adds answer, an answer about a client from the lookup
// database or GeoJS, to the client's history, and to the notes of the bans
// on its netblock that have no AS number, AS name or country yet.
func (h *handler) addLookup(answer geojs.Answer) {
h.limiter.AddLookup(answer.Client, answer.Answered,
// AddLookup adds answer, an answer about a client from the lookup database
// or GeoJS, to the client's history in limiter, and to the notes of the bans
// in ledger on its netblock that have no AS number, AS name or country yet.
// cfg sets the netblock's size.
func AddLookup(
cfg *config.WAF, limiter *ratelimit.Limiter, ledger *bans.Ledger,
answer geojs.Answer,
) {
limiter.AddLookup(answer.Client, answer.Answered,
answer.ASN, answer.ASName, answer.Country)
h.ledger.AddLookup(h.netblock(answer.Client.Addr()),
ledger.AddLookup(banNetblock(cfg, answer.Client.Addr()),
answer.ASN, answer.ASName, answer.Country)
}
// geojsFailed logs a failure of GeoJS, raises a source_failure alert for it
// and counts it. askingAgainIn is how long GeoJS is left alone after it.
func (h *handler) geojsFailed(err error, askingAgainIn time.Duration) {
h.processLog.Warn("asking GeoJS failed",
// GeoJSFailed logs a failure of GeoJS to processLog, raises a
// source_failure alert for it on alertQueue and counts it in m.
// askingAgainIn is how long GeoJS is left alone after it.
func GeoJSFailed(
processLog *slog.Logger, alertQueue *alerts.Queue, m *metrics.Metrics,
err error, askingAgainIn time.Duration,
) {
processLog.Warn("asking GeoJS failed",
"error", err.Error(), "asking_again_in", askingAgainIn.String())
h.alerts.Raise(alerts.Alert{
alertQueue.Raise(alerts.Alert{
Event: alerts.EventSourceFailure,
Reason: "asking GeoJS failed",
Detail: map[string]any{
@@ -107,7 +163,7 @@ func (h *handler) geojsFailed(err error, askingAgainIn time.Duration) {
"asking_again_in": askingAgainIn.String(),
},
})
h.metrics.GeoJSFailures.Inc()
m.GeoJSFailures.Inc()
}
// setLookupHeaders sets the headers that give the client's AS number and
+1 -1
View File
@@ -190,7 +190,7 @@ func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(
clk.Now,
map[string]string{
trustedProxies: trustLocalhost,
alertWebhookURL: "https://alerts.example/sww",
alertWebhookURL: alertWebhook,
rateLimitPerMinute: "1",
},
)
+71 -119
View File
@@ -4,10 +4,12 @@
package waf
import (
"cmp"
"fmt"
"io"
"log/slog"
"net/http"
"net/netip"
"time"
lru "github.com/hashicorp/golang-lru/v2"
@@ -55,18 +57,34 @@ type Params struct {
RequestLog io.Writer
// ProcessLog receives the process's own messages.
ProcessLog *slog.Logger
// GeoJSURL is where clients' AS numbers and countries are looked up
// while SWW_WAF_LOOKUP_SOURCE is geojs, normally geojs.URL.
GeoJSURL string
// AbuseIPDBURL is where clients are checked with AbuseIPDB while
// SWW_WAF_ABUSEIPDB_KEY is set, normally reputation.AbuseIPDBURL.
AbuseIPDBURL string
// Metrics count what waf does. Whoever builds them adds the metrics
// read from the other parts.
Metrics *metrics.Metrics
// Limiter is the table of clients the rate limits count, with each
// client's history.
Limiter *ratelimit.Limiter
// Ledger holds the bans.
Ledger *bans.Ledger
// Anomalies are the counters of the anomaly thresholds.
Anomalies *anomaly.Counters
// Lists are the lists fetched from URLs, DNSBL the DNSBL zones'
// verdicts, and AbuseIPDB AbuseIPDB's scores and the checks spent.
Lists *reputation.Lists
DNSBL *reputation.DNSBL
AbuseIPDB *reputation.AbuseIPDB
// RouteWatch asks routewatch about clients, and is nil while
// SWW_WAF_ROUTEWATCH_URL is unset.
RouteWatch *routewatch.Client
// GeoJS looks clients' AS numbers and countries up while
// SWW_WAF_LOOKUP_SOURCE is geojs. It is given AddLookup as its Answered
// and GeoJSFailed as its Failed.
GeoJS *geojs.GeoJS
// LookupFile is the lookup database they are looked up in while
// SWW_WAF_LOOKUP_SOURCE is file, and nil otherwise.
LookupFile *lookup.File
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
// while SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
CoreRuleSet *coreruleset.CoreRuleSet
CoreRuleSet CoreRuleSetInspector
// Now tells the time by which requests are counted for the rate
// limits, bans are made and run out, and GeoJS's answers are kept,
// normally time.Now in UTC, the time the state files give.
@@ -82,13 +100,23 @@ type Params struct {
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
// or the day's AbuseIPDB checks used up.
Alerts *alerts.Queue
// BesideLookupTimeLimit is how long the heuristics of besideLookup have
// for a request. Zero means besideLookupTimeLimit. Tests that run the
// Core Rule Set lengthen it, since the race detector and a busy host can
// slow an inspection past besideLookupTimeLimit.
BesideLookupTimeLimit time.Duration
}
// Server is the server waf runs, with the parts whose state the state
// files keep, the lookup database, nil unless SWW_WAF_LOOKUP_SOURCE is file, the
// lists fetched from URLs, which its Run fetches, the DNSBL zones' verdicts,
// AbuseIPDB's scores and checks spent, routewatch's answers, and the metrics.
// RouteWatch is nil while SWW_WAF_ROUTEWATCH_URL is unset.
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
type CoreRuleSetInspector interface {
Inspect(
r *http.Request, client netip.Addr, body io.Reader,
) (coreruleset.Result, []byte, error)
}
// Server is the server waf runs, with the parts of its Params. The tests
// read them.
type Server struct {
*http.Server
@@ -104,124 +132,45 @@ type Server struct {
Metrics *metrics.Metrics
}
// newHandler returns the handler with what every request shares, as the
// settings in params make it.
// newHandler returns the handler with what every request shares, the parts
// in params among it.
func newHandler(params Params) *handler {
m := metrics.New(
params.Config.MetricsBusiestCountriesAndASNs,
params.Config.InstanceName,
)
lists, dnsbl, abuseIPDB := newReputation(params, m)
awaitingReport, err := lru.New[string, *decision](maxAwaitingReport)
if err != nil {
panic(err) // New fails only for a size below one
}
h := &handler{
return &handler{
config: params.Config,
requestLog: params.RequestLog,
processLog: params.ProcessLog,
now: params.Now,
metrics: m,
metrics: params.Metrics,
awaitingReport: awaitingReport,
limiter: ratelimit.New(ratelimit.Limits{
PerMinute: params.Config.RateLimitPerMinute,
PerHour: params.Config.RateLimitPerHour,
PerDay: params.Config.RateLimitPerDay,
BytesPerMinute: params.Config.BytesLimitPerMinute,
BytesPerHour: params.Config.BytesLimitPerHour,
BytesPerDay: params.Config.BytesLimitPerDay,
}, params.Config.MaxTrackedClients),
ledger: bans.New(bans.Params{
LimitBanDuration: params.Config.LimitBanDuration,
LimitBanRepeatWindow: params.Config.LimitBanRepeatWindow,
LimitBanPermanentAbove: params.Config.LimitBanPermanentAbove,
AttackBanDuration: params.Config.AttackBanDuration,
MaxBans: params.Config.MaxBans,
}),
anomalies: anomaly.New(anomaly.Params{
Client: params.Config.AnomalyClient,
Net: params.Config.AnomalyNet,
ASN: params.Config.AnomalyASN,
Total: params.Config.AnomalyTotal,
WatchedNet: params.Config.AnomalyWatchedNet,
NetV4Prefix: params.Config.AnomalyNetV4Prefix,
NetV6Prefix: params.Config.AnomalyNetV6Prefix,
NamedNetblocks: params.Config.AnomalyWatchedNets,
Alerts: params.Alerts,
}),
lookupFile: params.LookupFile,
lists: lists,
dnsbl: dnsbl,
abuseIPDB: abuseIPDB,
rules: params.Rules,
coreRuleSet: params.CoreRuleSet,
alerts: params.Alerts,
limiter: params.Limiter,
ledger: params.Ledger,
geojs: params.GeoJS,
anomalies: params.Anomalies,
lookupFile: params.LookupFile,
lists: params.Lists,
dnsbl: params.DNSBL,
abuseIPDB: params.AbuseIPDB,
routeWatch: params.RouteWatch,
rules: params.Rules,
coreRuleSet: params.CoreRuleSet,
alerts: params.Alerts,
// The country list, the headers, the tag settings that name an asn=
// or country= tag and the biased thresholds act on the answer before
// the request goes on.
waitForGeoJS: len(params.Config.ExclusivelyAllowedCountries) > 0 ||
params.Config.AddLookupHeaders ||
params.Config.TagsNeedLookup() ||
biasedThresholdsSet(params.Config),
besideLookupLimit: cmp.Or(
params.BesideLookupTimeLimit,
besideLookupTimeLimit,
),
}
if params.Config.RouteWatchURL != "" {
h.routeWatch = routewatch.New(routewatch.Params{
URL: params.Config.RouteWatchURL, Now: params.Now, ProcessLog: params.ProcessLog,
})
}
// The country list, the headers, the tag settings that name an asn= or
// country= tag and the biased thresholds act on the answer before the
// request goes on.
h.waitForGeoJS = len(params.Config.ExclusivelyAllowedCountries) > 0 ||
params.Config.AddLookupHeaders || params.Config.TagsNeedLookup() ||
biasedThresholdsSet(params.Config)
h.geojs = geojs.New(geojs.Params{
URL: params.GeoJSURL,
Answered: h.addLookup,
Now: params.Now,
Asked: m.GeoJSRequests.Inc,
Failed: h.geojsFailed,
WentWithoutAnswer: m.RequestsWithoutGeoJSAnswer.Inc,
})
m.AddBansAndClients(h.ledger, h.limiter, params.Now)
m.AddRules(params.Rules)
return h
}
// newReputation returns the lists fetched from URLs, the DNSBL zones'
// verdicts and AbuseIPDB's scores, as the settings in params name them,
// with none fetched, asked for or checked yet, and adds their metrics to
// m, AbuseIPDB's while SWW_WAF_ABUSEIPDB_KEY is set.
func newReputation(
params Params, m *metrics.Metrics,
) (*reputation.Lists, *reputation.DNSBL, *reputation.AbuseIPDB) {
cfg := params.Config
lists := reputation.New(reputation.Params{
BlocklistURLs: cfg.BlocklistURLs,
Refresh: cfg.BlocklistRefresh,
ASNLimitPercentURL: cfg.ASNLimitPercentURL,
TagLists: cfg.TagLists,
Now: params.Now,
ProcessLog: params.ProcessLog,
Alerts: params.Alerts,
})
dnsbl := reputation.NewDNSBL(reputation.DNSBLParams{
Zones: cfg.DNSBLZones, Resolver: cfg.DNSBLResolver,
CacheTTL: cfg.IPReputationCacheTTL,
Timeout: cfg.DNSBLAbuseIPDBTimeout, Now: params.Now, ProcessLog: params.ProcessLog,
Alerts: params.Alerts,
})
abuseIPDB := reputation.NewAbuseIPDB(reputation.AbuseIPDBParams{
URL: params.AbuseIPDBURL, Key: cfg.AbuseIPDBKey, MinScore: cfg.AbuseIPDBMinScore,
DailyBudget: cfg.AbuseIPDBDailyBudget, CacheTTL: cfg.IPReputationCacheTTL,
Timeout: cfg.DNSBLAbuseIPDBTimeout, Now: params.Now, ProcessLog: params.ProcessLog,
Alerts: params.Alerts,
})
m.AddReputation(lists, dnsbl)
if cfg.AbuseIPDBKey != "" {
m.AddAbuseIPDB(abuseIPDB)
}
return lists, dnsbl, abuseIPDB
}
// NewCoreRuleSet returns the Core Rule Set at SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL,
@@ -276,11 +225,14 @@ type handler struct {
abuseIPDB *reputation.AbuseIPDB
routeWatch *routewatch.Client
rules *rules.Files
coreRuleSet *coreruleset.CoreRuleSet
coreRuleSet CoreRuleSetInspector
alerts *alerts.Queue
// waitForGeoJS is true when a setting needs each request's answer from
// GeoJS before the request goes on.
waitForGeoJS bool
// besideLookupLimit is how long the heuristics of besideLookup have for a
// request, as Params.BesideLookupTimeLimit says.
besideLookupLimit time.Duration
// awaitingReport holds the decision on each answered question by its id
// until its report comes, and nil for a question still being handled.
awaitingReport *lru.Cache[string, *decision]
+144 -15
View File
@@ -16,10 +16,17 @@ import (
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/routewatch"
"sneak.berlin/go/smallwebwaf/internal/rproxy"
"sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/waf"
@@ -435,13 +442,15 @@ func newWAF(
) (*waf.Server, *output, *alerts.Queue) {
t.Helper()
return newWAFWith(t, waf.New, geojsURL, now, env)
return newWAFWith(t, waf.New, geojsURL, abuseIPDBURL, now, env)
}
// newWAFWith is newWAF for the server build makes.
// newWAFWith is newWAF for the server build makes, with clients checked with
// AbuseIPDB at abuseIPDBCheckURL.
func newWAFWith(
t *testing.T, build func(waf.Params) *waf.Server,
geojsURL string, now func() time.Time, env map[string]string,
geojsURL, abuseIPDBCheckURL string, now func() time.Time,
env map[string]string,
) (*waf.Server, *output, *alerts.Queue) {
t.Helper()
@@ -499,22 +508,142 @@ func newWAFWith(
t.Fatalf("Core Rule Set: %v", err)
}
server := build(waf.Params{
Config: cfg,
RequestLog: out,
ProcessLog: processLog,
GeoJSURL: geojsURL,
AbuseIPDBURL: abuseIPDBURL,
LookupFile: lookupFile,
CoreRuleSet: coreRuleSet,
Now: now,
Rules: ruleFiles,
Alerts: alertQueue,
})
params := waf.Params{
Config: cfg,
RequestLog: out,
ProcessLog: processLog,
LookupFile: lookupFile,
CoreRuleSet: coreRuleSet,
Now: now,
Rules: ruleFiles,
Alerts: alertQueue,
// The tests of the limit set it back, and hold the Core Rule Set
// with a stand-in.
BesideLookupTimeLimit: time.Hour,
}
addCounters(&params)
addOutsideSources(&params, geojsURL, abuseIPDBCheckURL)
server := build(params)
return server, out, alertQueue
}
// addCounters adds to params the metrics, the table of clients, the ledger
// and the anomaly counters, as sww builds them from params' settings. The
// metrics read the ledger, the table of clients and the rules.
func addCounters(params *waf.Params) {
cfg := params.Config
params.Metrics = metrics.New(
cfg.MetricsBusiestCountriesAndASNs,
cfg.InstanceName,
)
params.Limiter = ratelimit.New(ratelimit.Limits{
PerMinute: cfg.RateLimitPerMinute,
PerHour: cfg.RateLimitPerHour,
PerDay: cfg.RateLimitPerDay,
BytesPerMinute: cfg.BytesLimitPerMinute,
BytesPerHour: cfg.BytesLimitPerHour,
BytesPerDay: cfg.BytesLimitPerDay,
}, cfg.MaxTrackedClients)
params.Ledger = bans.New(bans.Params{
LimitBanDuration: cfg.LimitBanDuration,
LimitBanRepeatWindow: cfg.LimitBanRepeatWindow,
LimitBanPermanentAbove: cfg.LimitBanPermanentAbove,
AttackBanDuration: cfg.AttackBanDuration,
MaxBans: cfg.MaxBans,
})
params.Anomalies = anomaly.New(anomaly.Params{
Client: cfg.AnomalyClient,
Net: cfg.AnomalyNet,
ASN: cfg.AnomalyASN,
Total: cfg.AnomalyTotal,
WatchedNet: cfg.AnomalyWatchedNet,
NetV4Prefix: cfg.AnomalyNetV4Prefix,
NetV6Prefix: cfg.AnomalyNetV6Prefix,
NamedNetblocks: cfg.AnomalyWatchedNets,
Alerts: params.Alerts,
})
params.Metrics.AddBansAndClients(
params.Ledger,
params.Limiter,
params.Now,
)
params.Metrics.AddRules(params.Rules)
}
// addOutsideSources adds to params the lists fetched from URLs, the DNSBL
// zones' verdicts, AbuseIPDB's scores from abuseIPDBCheckURL, routewatch's
// answers while its setting is set, and GeoJS at geojsURL, as sww builds
// them from params' settings and the parts addCounters added. The metrics
// read the lists, the zones' verdicts, and AbuseIPDB's while its key is set.
func addOutsideSources(
params *waf.Params, geojsURL, abuseIPDBCheckURL string,
) {
cfg := params.Config
m, limiter, ledger := params.Metrics, params.Limiter, params.Ledger
processLog, alertQueue := params.ProcessLog, params.Alerts
params.Lists = reputation.New(reputation.Params{
BlocklistURLs: cfg.BlocklistURLs,
Refresh: cfg.BlocklistRefresh,
ASNLimitPercentURL: cfg.ASNLimitPercentURL,
TagLists: cfg.TagLists,
Now: params.Now,
ProcessLog: processLog,
Alerts: alertQueue,
})
params.DNSBL = reputation.NewDNSBL(reputation.DNSBLParams{
Zones: cfg.DNSBLZones,
Resolver: cfg.DNSBLResolver,
CacheTTL: cfg.IPReputationCacheTTL,
Timeout: cfg.DNSBLAbuseIPDBTimeout,
Now: params.Now,
ProcessLog: processLog,
Alerts: alertQueue,
})
params.AbuseIPDB = reputation.NewAbuseIPDB(reputation.AbuseIPDBParams{
URL: abuseIPDBCheckURL,
Key: cfg.AbuseIPDBKey,
MinScore: cfg.AbuseIPDBMinScore,
DailyBudget: cfg.AbuseIPDBDailyBudget,
CacheTTL: cfg.IPReputationCacheTTL,
Timeout: cfg.DNSBLAbuseIPDBTimeout,
Now: params.Now,
ProcessLog: processLog,
Alerts: alertQueue,
})
if cfg.RouteWatchURL != "" {
params.RouteWatch = routewatch.New(routewatch.Params{
URL: cfg.RouteWatchURL,
Now: params.Now,
ProcessLog: processLog,
})
}
params.GeoJS = geojs.New(geojs.Params{
URL: geojsURL,
Answered: func(answer geojs.Answer) {
waf.AddLookup(cfg, limiter, ledger, answer)
},
Now: params.Now,
Asked: m.GeoJSRequests.Inc,
Failed: func(err error, askingAgainIn time.Duration) {
waf.GeoJSFailed(processLog, alertQueue, m, err, askingAgainIn)
},
WentWithoutAnswer: m.RequestsWithoutGeoJSAnswer.Inc,
})
m.AddReputation(params.Lists, params.DNSBL)
if cfg.AbuseIPDBKey != "" {
m.AddAbuseIPDB(params.AbuseIPDB)
}
}
// newClient returns an HTTP client that sends requests as they are made,
// with no compression of its own.
func newClient(t *testing.T) *http.Client {
+6 -3
View File
@@ -38,8 +38,11 @@ func (rateLimits) check(
rq.line.BytesLimitPercent, rq.line.BytesLimitPercentSetting =
rq.bytesLimitPercent.logged()
counts, hit, over := rq.h.limiter.Count(rq.h.clientGroup(rq.client), now,
rq.rateLimitPercent.percent)
counts, hit, over := rq.h.limiter.Count(
clientGroup(rq.h.config, rq.client),
now,
rq.rateLimitPercent.percent,
)
rq.line.Counts = counts
if !over {
@@ -59,7 +62,7 @@ func (rateLimits) check(
// limit. It counts nothing.
func (rq *request) overAWholeRateLimit(now time.Time) bool {
return rq.countedByRateLimits() &&
rq.h.limiter.OverARateLimit(rq.h.clientGroup(rq.client), now)
rq.h.limiter.OverARateLimit(clientGroup(rq.h.config, rq.client), now)
}
// countedByRateLimits reports whether the rate limits count the request: its
+5 -2
View File
@@ -36,8 +36,11 @@ func (refusalLimit) check(
return nil
}
hit, over := rq.h.limiter.CountRefusal(rq.h.clientGroup(rq.client), now,
cfg.RefusalsLimitPerMinute)
hit, over := rq.h.limiter.CountRefusal(
clientGroup(rq.h.config, rq.client),
now,
cfg.RefusalsLimitPerMinute,
)
if !over {
return nil
}
+1 -1
View File
@@ -37,7 +37,7 @@ func (rq *request) noteHit(hit bans.ReputationHit, reason string) {
Event: alerts.EventReputationHit,
Host: rq.site,
Client: rq.client,
Netblock: rq.h.clientGroup(rq.client),
Netblock: clientGroup(rq.h.config, rq.client),
ASN: rq.line.ASN,
ASName: rq.line.ASName,
Country: rq.line.Country,
+48 -14
View File
@@ -81,6 +81,9 @@ type request struct {
checked time.Time
// refused is what the request was refused with, nil if it was not.
refused *refusal
// findings is what the heuristics of besideLookup found of the request,
// once waf has taken it, and nil before.
findings *besideLookupFindings
// requestBytes is how many bytes of the request's body have been read.
requestBytes int64
}
@@ -119,7 +122,7 @@ func (h *handler) startRequest(
RequestID: requestID(r, isQuestion),
PeerIP: peer.String(),
ForwardedFor: strings.Join(forwardedFor, ", "),
ClientGroup: h.clientGroup(client).String(),
ClientGroup: clientGroup(h.config, client).String(),
ContentType: r.Header.Get("Content-Type"),
RequestHeaders: requestHeaders(
r,
@@ -177,9 +180,9 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
// check is the one place where a request can be refused. It returns nil to let
// the request through. The heuristics of checkClient decide: a request they
// refuse is answered with SWW_WAF_BAN_RESPONSE, 403 for a block rule or the Core
// Rule Set, or 414 or 431 for the header limits. In observe mode nothing is refused,
// and the log line names what enforce mode would have done. ctx is the
// request's own context.
// Rule Set, 503 for heuristics that ran out of time, or 414 or 431 for the
// header limits. In observe mode nothing is refused, and the log line names
// what enforce mode would have done. ctx is the request's own context.
func (rq *request) check(ctx context.Context) *refusal {
v := rq.checkClient(ctx)
if v == nil {
@@ -196,6 +199,11 @@ func (rq *request) check(ctx context.Context) *refusal {
case action == requestlog.ActionBlockedByBlockRule ||
action == requestlog.ActionBlockedByCoreRuleSet:
return &refusal{status: http.StatusForbidden, action: action}
case action == requestlog.ActionChecksTimedOut:
return &refusal{
status: http.StatusServiceUnavailable,
action: action,
}
case action == requestlog.ActionRequestLineTooLong:
return &refusal{status: http.StatusRequestURITooLong, action: action}
case action == requestlog.ActionRequestHeaderTooLarge:
@@ -208,11 +216,16 @@ func (rq *request) check(ctx context.Context) *refusal {
}
}
// checkClient runs the heuristics of beforeLookup on the request, then looks
// up the client's AS number and country, then runs those of afterLookup. It
// returns the first verdict, or nil when none gives one. A client in
// SWW_WAF_ALLOW_NETS skips them all, and is not looked up. ctx is the request's
// own context.
// checkClient runs the heuristics on the request, and returns the first
// verdict, or nil when none gives one. A client in SWW_WAF_ALLOW_NETS skips them
// all, and is not looked up. ctx is the request's own context.
//
// The heuristics of beforeLookup run first. Then the client's AS number and
// country are looked up, and afterLookup runs once the lookup has its result.
// While the lookup waits for GeoJS, besideLookup runs too, as runBesideLookup
// says. A lookup that needs no wait is done, and afterLookup with it, before
// besideLookup starts, so that besideLookup does no work for a request
// afterLookup refuses.
func (rq *request) checkClient(ctx context.Context) *verdict {
if isInside(rq.client, rq.h.config.AllowNets) {
return nil
@@ -225,7 +238,28 @@ func (rq *request) checkClient(ctx context.Context) *verdict {
return v
}
rq.lookUp(ctx, now)
lookedUp := rq.startLookUp(ctx, now)
select {
case result := <-lookedUp:
v := rq.runAfterLookup(ctx, result)
if v != nil {
return v
}
return rq.runBesideLookup(ctx, nil)
default:
return rq.runBesideLookup(ctx, lookedUp)
}
}
// runAfterLookup notes result, the lookup's, and runs the heuristics of
// afterLookup. It returns the first verdict, or nil when none gives one. ctx is
// the request's own context.
func (rq *request) runAfterLookup(
ctx context.Context, result lookupResult,
) *verdict {
rq.noteLookup(result)
// The lookup can wait for GeoJS, so the checks after it read the time
// again: a request counted at the time it arrived could be counted
@@ -331,7 +365,7 @@ func (rq *request) addToHistory() {
}
rq.h.limiter.AddToHistory(
rq.h.clientGroup(rq.client),
clientGroup(rq.h.config, rq.client),
rq.h.now(),
request,
)
@@ -339,7 +373,7 @@ func (rq *request) addToHistory() {
answer, found := rq.answerAtTheEnd()
if found {
rq.h.addLookup(answer)
AddLookup(rq.h.config, rq.h.limiter, rq.h.ledger, answer)
}
}
@@ -360,7 +394,7 @@ func (rq *request) countAnomalies() {
rq.h.anomalies.Count(rq.h.now(), anomaly.Request{
Host: rq.site,
Client: rq.client,
ClientGroup: rq.h.clientGroup(rq.client),
ClientGroup: clientGroup(rq.h.config, rq.client),
ASN: answer.ASN,
ASName: answer.ASName,
Country: answer.Country,
@@ -391,7 +425,7 @@ func (rq *request) answerAtTheEnd() (geojs.Answer, bool) {
return rq.lookupAnswer, true
}
return rq.h.geojs.Kept(rq.h.clientGroup(rq.client))
return rq.h.geojs.Kept(clientGroup(rq.h.config, rq.client))
}
// responseWriter is waf's answer to a request. It notes the status and
+1 -1
View File
@@ -32,7 +32,7 @@ func (routeWatchTags) check(
route, missing := rq.h.routeWatch.Route(
ctx,
rq.h.clientGroup(rq.client),
clientGroup(rq.h.config, rq.client),
rq.client,
)
if missing != "" {
+15 -3
View File
@@ -13,8 +13,8 @@ import (
"sneak.berlin/go/smallwebwaf/internal/rules"
)
// ruleFiles checks the request against the rules of the rule files at now,
// notes the ids of those it matches in the log line, adds the tag
// ruleFiles takes the rules of the rule files the request matched, which
// findBesideLookup found, notes their ids in the log line, adds the tag
// exploitsignature when it matches any, and refuses the request
// when the last rule it matches refuses it: with ActionBlockedByBlockRule for a
// block rule, and for a ban rule with the action for the ban it makes. A ban
@@ -31,7 +31,7 @@ func (ruleFiles) check(
rq *request,
now time.Time,
) *verdict {
matched := rq.h.rules.Match(rq.in)
matched := rq.findings.rules
for _, rule := range matched {
rq.line.RuleIDs = append(rq.line.RuleIDs, rule.ID)
@@ -70,3 +70,15 @@ func (ruleFiles) check(
return nil
}
}
// refusedByRule reports whether ruleFiles refuses a request that matched
// matched: whether the last of them is a block rule or a ban rule.
func refusedByRule(matched []rules.Rule) bool {
if len(matched) == 0 {
return false
}
action := matched[len(matched)-1].Action
return action == rules.ActionBlock || action == rules.ActionBan
}
+17 -6
View File
@@ -11,9 +11,8 @@ import (
"sneak.berlin/go/smallwebwaf/internal/rules"
)
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS: its path as a
// path rule sees it, without the query, as sent or once percent-decoded, is one
// of them. Such a request is a clear sign of attack, as a ban rule's match is:
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS, as trapPath
// finds it. Such a request is a clear sign of attack, as a ban rule's match is:
// it bans the client's netblock, or in observe mode raises the alert for the
// ban it would have made. The ban's notes give the path as sent, and so does
// its figure.
@@ -26,9 +25,8 @@ func (trapPaths) check(
rq *request,
now time.Time,
) *verdict {
path := rules.Path(rq.in)
if !slices.Contains(rq.h.config.TrapPaths, path) &&
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
path := rq.findings.trapPath
if path == "" {
return nil
}
@@ -39,3 +37,16 @@ func (trapPaths) check(
figures: map[string]any{"path": path},
}
}
// trapPath returns the request's path as sent, when its path as a path rule
// sees it, without the query, as sent or once percent-decoded, is in
// SWW_WAF_TRAP_PATHS, and "" otherwise.
func (rq *request) trapPath() string {
path := rules.Path(rq.in)
if !slices.Contains(rq.h.config.TrapPaths, path) &&
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
return ""
}
return path
}
+5
View File
@@ -20,8 +20,13 @@ main() {
# the version a build stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
revision="$(git rev-parse HEAD 2>/dev/null || true)"
[ -n "$revision" ] || revision="unknown"
created="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
docker build --no-cache \
--build-arg VERSION="$version" \
--build-arg REVISION="$revision" \
--build-arg CREATED="$created" \
-t "$("$SCRIPT_DIR/projectname")" .
}
+5
View File
@@ -16,8 +16,13 @@ main() {
# the version a build stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
revision="$(git rev-parse HEAD 2>/dev/null || true)"
[ -n "$revision" ] || revision="unknown"
created="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
docker build --no-cache \
--build-arg VERSION="$version" \
--build-arg REVISION="$revision" \
--build-arg CREATED="$created" \
-t "$("$SCRIPT_DIR/projectname")" .
}