Trap paths, and the error burst banning a client refused too often (closes #115)
check / check (push) Waiting to run
check / check (push) Waiting to run
SWWAF_TRAP_PATHS: a request whose path, as a path rule sees it, is one of them is a clear sign of attack, banned as a ban rule's match is; the ban's notes give its trap_path. Checked after the rate limits, before the rule files. SWWAF_ERROR_BURST_THRESHOLD (default 30, or off): more refusals in a minute after a block or ban rule or a trap path, or for a missing or wrong token, ban the client as a broken limit does. Counted in clients.json's minute_refusals; limit_hit error_burst, notes kind refusals. A token refusal is now the offence token_refused, and smallwebwaf_offences_total counts every kind the history does. Judgement call: the threshold is not lowered by a client's limit percentage. Model: opus-5-5
This commit is contained in:
@@ -31,29 +31,32 @@ you name by URL, which it fetches and keeps, with a file of AS numbers'
|
|||||||
percentages fetched the same way, the DNS blocklists (DNSBL zones), which it
|
percentages fetched the same way, the DNS blocklists (DNSBL zones), which it
|
||||||
asks about each client in the background, AbuseIPDB, which it asks in the
|
asks about each client in the background, AbuseIPDB, which it asks in the
|
||||||
background about each client that has committed an offence, and the decision
|
background about each client that has committed an offence, and the decision
|
||||||
list of a CrowdSec engine you run, which it fetches and keeps as a blocklist.
|
list of a CrowdSec engine you run, which it fetches and keeps as a blocklist. So
|
||||||
`smallwebwaf` passes each request to the app and the app's answer back,
|
are two parts of the last stage, which need no Core Rule Set: the trap paths and
|
||||||
unchanged, within its timeouts and size limits, works out each client's address,
|
the error burst. `smallwebwaf` passes each request to the app and the app's
|
||||||
looks up its AS number and country unless you switch that off, bans a client
|
answer back, unchanged, within its timeouts and size limits, works out each
|
||||||
that sends too many requests or too many bytes, not counting those for the paths
|
client's address, looks up its AS number and country unless you switch that off,
|
||||||
you choose, with lower limits for the clients of the AS numbers and countries
|
bans a client that sends too many requests or too many bytes, not counting those
|
||||||
you list, refuses a client that comes from a country you refuse or from a
|
for the paths you choose, with lower limits for the clients of the AS numbers
|
||||||
network you refuse, refuses, limits or only notes a client a blocklist or a
|
and countries you list, refuses a client that comes from a country you refuse or
|
||||||
DNSBL zone you name lists, or AbuseIPDB scores at or over the score you set,
|
from a network you refuse, refuses, limits or only notes a client a blocklist or
|
||||||
|
a DNSBL zone you name lists, or AbuseIPDB scores at or over the score you set,
|
||||||
bans a client your CrowdSec engine's decision list lists until that decision
|
bans a client your CrowdSec engine's decision list lists until that decision
|
||||||
ends, lets the networks you choose through, checks each request against the rule
|
ends, lets the networks you choose through, checks each request against the rule
|
||||||
files and bans a client whose request is a clear sign of attack, keeps its bans,
|
files and the trap paths you name and bans a client whose request is a clear
|
||||||
each client's counters and history, GeoJS's answers, the last good copy of each
|
sign of attack, bans a client it refuses again and again within a minute for a
|
||||||
list it fetches, the DNSBL zones' verdicts, AbuseIPDB's scores and the AbuseIPDB
|
rule, a trap path or a missing or wrong token, keeps its bans, each client's
|
||||||
checks spent today in JSON files across restarts, takes in your edits of those
|
counters and history, GeoJS's answers, the last good copy of each list it
|
||||||
files, such as a ban you make, keep or lift, and of the rule files while it
|
fetches, the DNSBL zones' verdicts, AbuseIPDB's scores and the AbuseIPDB checks
|
||||||
runs, writes a JSON log line for every request, sends its log lines to a syslog
|
spent today in JSON files across restarts, takes in your edits of those files,
|
||||||
server too if you name one, sends an alert to a webhook, to Slack and to ntfy,
|
such as a ban you make, keep or lift, and of the rule files while it runs,
|
||||||
each if you name one, for each ban it makes or makes permanent, for traffic over
|
writes a JSON log line for every request, sends its log lines to a syslog server
|
||||||
an anomaly threshold you set, for a client a blocklist, the CrowdSec decision
|
too if you name one, sends an alert to a webhook, to Slack and to ntfy, each if
|
||||||
list, a DNSBL zone or AbuseIPDB lists, for GeoJS failing, a list it cannot
|
you name one, for each ban it makes or makes permanent, for traffic over an
|
||||||
fetch, a DNSBL zone or AbuseIPDB that fails or refuses a query and the day's
|
anomaly threshold you set, for a client a blocklist, the CrowdSec decision list,
|
||||||
AbuseIPDB checks used up, for a rule file or state file with an error and for a
|
a DNSBL zone or AbuseIPDB lists, for GeoJS failing, a list it cannot fetch, a
|
||||||
|
DNSBL zone or AbuseIPDB that fails or refuses a query and the day's AbuseIPDB
|
||||||
|
checks used up, for a rule file or state file with an error and for a
|
||||||
replacement of the lookup database it cannot read, serves Prometheus metrics to
|
replacement of the lookup database it cannot read, serves Prometheus metrics to
|
||||||
a scraper that holds the metrics token, lets an admin who holds the admin token
|
a scraper that holds the metrics token, lets an admin who holds the admin token
|
||||||
list, add and lift bans and ask what it knows of a client, and in `observe` mode
|
list, add and lift bans and ask what it knows of a client, and in `observe` mode
|
||||||
@@ -152,37 +155,47 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
|||||||
The log line of each request the rate limits count gives its client's
|
The log line of each request the rate limits count gives its client's
|
||||||
percentages below 100 and the settings that gave them, and so do the notes of
|
percentages below 100 and the settings that gave them, and so do the notes of
|
||||||
a ban for a lowered limit, and its alert.
|
a ban for a lowered limit, and its alert.
|
||||||
- Bans a client that breaks a rate limit or a byte limit, as "Bans" in
|
- Bans a client that breaks a rate limit, a byte limit or the error burst, as
|
||||||
[`SPEC.md`](SPEC.md) describes: the first ban lasts an hour, and a limit
|
"Bans" in [`SPEC.md`](SPEC.md) describes: the first ban lasts an hour, and a
|
||||||
broken again within a day of a ban ending bans for three times as long as that
|
limit broken again within a day of a ban ending bans for three times as long
|
||||||
ban, so 1, 3, 9, 27 and 81 hours; a ban that would last longer than seven days
|
as that ban, so 1, 3, 9, 27 and 81 hours; a ban that would last longer than
|
||||||
is permanent instead. A ban covers the client's netblock: its IPv4 address, or
|
seven days is permanent instead. A ban covers the client's netblock: its IPv4
|
||||||
the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or its IPv6
|
address, or the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or
|
||||||
group. While it lasts, every request from the netblock is refused with
|
its IPv6 group. While it lasts, every request from the netblock is refused
|
||||||
`SWWAF_BAN_RESPONSE` after the static lists and before the country lists, so
|
with `SWWAF_BAN_RESPONSE` after the static lists and before the country lists,
|
||||||
the client is not looked up, and is not counted for the rate limits. A ban
|
so the client is not looked up, and is not counted for the rate limits. A ban
|
||||||
sets the client's counters back to zero. Each ban carries notes for deciding
|
sets the client's counters back to zero. Each ban carries notes for deciding
|
||||||
whether to lift it: the limit, whether it is on requests or bytes, its window
|
whether to lift it: the limit, whether it is on requests, bytes or refusals,
|
||||||
and the requests or bytes counted in it, the client's percentage of that kind
|
its window and the requests, bytes or refusals counted in it, the client's
|
||||||
of limit and the setting that gave it when a biased threshold lowered the
|
percentage of that kind of limit and the setting that gave it when a biased
|
||||||
limit, the request that broke it, the client's AS number, AS name and country
|
threshold lowered the limit, the request that broke it, the client's AS
|
||||||
once they are looked up, the blocklists, the CrowdSec decision list, DNSBL
|
number, AS name and country once they are looked up, the blocklists, the
|
||||||
zones and AbuseIPDB, with its score, that listed the client when the ban was
|
CrowdSec decision list, DNSBL zones and AbuseIPDB, with its score, that listed
|
||||||
made, the netblock's requests since it was first seen, how many of them the
|
the client when the ban was made, the netblock's requests since it was first
|
||||||
ban has refused, and how many bans the netblock had before, for a broken
|
seen, how many of them the ban has refused, and how many bans the netblock had
|
||||||
limit, for a clear sign of attack, by an admin and for CrowdSec's decision. At
|
before, for a broken limit, for a clear sign of attack, by an admin and for
|
||||||
most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept, past, active and
|
CrowdSec's decision. At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are
|
||||||
permanent; past that, the earliest such ban of the netblock that has gone
|
kept, past, active and permanent; past that, the earliest such ban of the
|
||||||
longest without a request is dropped first. The bans whose cause is `admin`,
|
netblock that has gone longest without a request is dropped first. The bans
|
||||||
those you make or keep, are kept besides, and never dropped. `bans.json` shows
|
whose cause is `admin`, those you make or keep, are kept besides, and never
|
||||||
the bans and their notes, a restart lifts none, and you make, keep or lift a
|
dropped. `bans.json` shows the bans and their notes, a restart lifts none, and
|
||||||
ban by editing it (see "State files" below).
|
you make, keep or lift a ban by editing it (see "State files" below).
|
||||||
- Checks each request against the rules of the rule files (see "Rule files"
|
- Checks each request against the rules of the rule files (see "Rule files"
|
||||||
below) after the rate limits, and before its body is read. A `log` rule that
|
below) after the rate limits, and before its body is read. A `log` rule that
|
||||||
matches is noted in the log line; a `block` rule refuses the request with
|
matches is noted in the log line; a `block` rule refuses the request with
|
||||||
`403`, and bans no one; a `ban` rule refuses it with `SWWAF_BAN_RESPONSE` and
|
`403`, and bans no one for it, though the refusal counts toward the error
|
||||||
bans the client's netblock for a clear sign of attack. Matching stops at the
|
burst; a `ban` rule refuses it with `SWWAF_BAN_RESPONSE` and bans the client's
|
||||||
first rule that refuses. A client in `SWWAF_ALLOW_NETS` is not checked.
|
netblock for a clear sign of attack. Matching stops at the first rule that
|
||||||
|
refuses. A client in `SWWAF_ALLOW_NETS` is not checked.
|
||||||
|
- Checks each request against the trap paths `SWWAF_TRAP_PATHS` names, after the
|
||||||
|
rate limits and before the rule files, which it does not need. A request for
|
||||||
|
one is refused with `SWWAF_BAN_RESPONSE` and bans the client's netblock for a
|
||||||
|
clear sign of attack, as a `ban` rule's match does. A trap path is matched
|
||||||
|
against the path a `path` rule sees: as the client sent it, before any
|
||||||
|
decoding and without the query, the whole of it, character for character.
|
||||||
|
`/wp-login.php` matches `/wp-login.php?redirect_to=x`, but not
|
||||||
|
`/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or `/%77p-login.php`.
|
||||||
|
A client in `SWWAF_ALLOW_NETS` is not checked.
|
||||||
- Bans a client for a clear sign of attack, as "Bans" in [`SPEC.md`](SPEC.md)
|
- Bans a client for a clear sign of attack, as "Bans" in [`SPEC.md`](SPEC.md)
|
||||||
describes: the first such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days by
|
describes: the first such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days by
|
||||||
default, and any request from the netblock while it lasts makes it permanent.
|
default, and any request from the netblock while it lasts makes it permanent.
|
||||||
@@ -190,8 +203,22 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
|||||||
sign of attack bans it permanently at once. Such a ban covers the same
|
sign of attack bans it permanently at once. Such a ban covers the same
|
||||||
netblock as a ban for a broken limit, does not set the client's counters back
|
netblock as a ban for a broken limit, does not set the client's counters back
|
||||||
to zero, and does not make the netblock's next ban for a broken limit longer.
|
to zero, and does not make the netblock's next ban for a broken limit longer.
|
||||||
Its notes give the id and the target of the rule that matched in place of the
|
Its notes give the id and the target of the rule that matched, or the trap
|
||||||
limit.
|
path asked for, in place of the limit.
|
||||||
|
- Bans a client that `smallwebwaf` refused more than
|
||||||
|
`SWWAF_ERROR_BURST_THRESHOLD` times within a minute, 30 by default, after a
|
||||||
|
match of a `block` or `ban` rule or a trap path, or for a missing or wrong
|
||||||
|
token at one of its own endpoints, as a broken rate limit bans it. Each such
|
||||||
|
refusal is counted once it has been answered, in two buckets of a minute, as
|
||||||
|
the rate limits count requests. The refusal that takes the client over the
|
||||||
|
threshold breaks the error burst; it is answered as any other such refusal is,
|
||||||
|
and the client's next request is refused under the ban. The app's own answers,
|
||||||
|
such as its `401` and `404`, are not counted. The threshold is the same for
|
||||||
|
every client, whatever percentage of the rate limits a biased threshold or a
|
||||||
|
reputation source gives it. A client in `SWWAF_ALLOW_NETS` is not counted, and
|
||||||
|
one in `SWWAF_RATE_LIMIT_EXEMPT_NETS` is. The ban's notes give `refusals` as
|
||||||
|
what the limit is on, the threshold as the limit, and the refusals counted in
|
||||||
|
the minute.
|
||||||
- Looks up the AS number and country of every client through GeoJS, or in the
|
- Looks up the AS number and country of every client through GeoJS, or in the
|
||||||
IPinfo Lite database file while `SWWAF_LOOKUP_SOURCE` is `file`, after the
|
IPinfo Lite database file while `SWWAF_LOOKUP_SOURCE` is `file`, after the
|
||||||
static lists and bans, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Country and
|
static lists and bans, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Country and
|
||||||
@@ -242,44 +269,50 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
|||||||
- Checks the client with AbuseIPDB while `SWWAF_ABUSEIPDB_KEY` is set, after the
|
- Checks the client with AbuseIPDB while `SWWAF_ABUSEIPDB_KEY` is set, after the
|
||||||
DNSBL zones and before the rate limits (see "AbuseIPDB" below), by the scores
|
DNSBL zones and before the rate limits (see "AbuseIPDB" below), by the scores
|
||||||
it keeps. Only a client whose history counts an offence is checked, so far one
|
it keeps. Only a client whose history counts an offence is checked, so far one
|
||||||
that has broken a rate limit or a byte limit, matched a ban rule, or had a
|
that has broken a rate limit, a byte limit or the error burst, matched a ban
|
||||||
request refused by a block rule, and only in the background, so that no
|
rule, asked for a trap path, or had a request refused by a block rule or for a
|
||||||
request waits for AbuseIPDB. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is
|
missing or wrong token, and only in the background, so that no request waits
|
||||||
a hit, and `SWWAF_REPUTATION_ACTION` does with its client what it does with
|
for AbuseIPDB. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and
|
||||||
one a DNSBL zone's verdict lists. The request's log line names AbuseIPDB, and
|
`SWWAF_REPUTATION_ACTION` does with its client what it does with one a DNSBL
|
||||||
it raises an alert. A client a blocklist or a DNSBL zone refuses, or the
|
zone's verdict lists. The request's log line names AbuseIPDB, and it raises an
|
||||||
CrowdSec decision list bans, is not checked.
|
alert. A client a blocklist or a DNSBL zone refuses, or the CrowdSec decision
|
||||||
|
list bans, is not checked.
|
||||||
- Checks the client's own address against the static lists, the three netblock
|
- Checks the client's own address against the static lists, the three netblock
|
||||||
settings below, before anything else, its lookup included. A client in
|
settings below, before anything else, its lookup included. A client in
|
||||||
`SWWAF_ALLOW_NETS` skips bans, the country lists, the blocklists, the CrowdSec
|
`SWWAF_ALLOW_NETS` skips bans, the country lists, the blocklists, the CrowdSec
|
||||||
decision list, the DNSBL zones, AbuseIPDB, the rate limits, the byte limits
|
decision list, the DNSBL zones, AbuseIPDB, the rate limits, the byte limits,
|
||||||
and the rule files, and is not looked up; the timeouts and size limits still
|
the trap paths, the rule files and the error burst, and is not looked up; the
|
||||||
apply. A client in `SWWAF_DENY_NETS` is refused with `SWWAF_BAN_RESPONSE`
|
timeouts and size limits still apply. A client in `SWWAF_DENY_NETS` is refused
|
||||||
before its body is read, and the request is not counted for the rate limits;
|
with `SWWAF_BAN_RESPONSE` before its body is read, and the request is not
|
||||||
an address in `SWWAF_ALLOW_NETS` too is let through. A client in
|
counted for the rate limits; an address in `SWWAF_ALLOW_NETS` too is let
|
||||||
`SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither counted nor refused by the rate
|
through. A client in `SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither counted nor
|
||||||
limits, and has no bytes counted by the byte limits; the country lists, the
|
refused by the rate limits, and has no bytes counted by the byte limits; the
|
||||||
rule files and bans still apply to it.
|
country lists, the trap paths, the rule files, the error burst and bans still
|
||||||
|
apply to it.
|
||||||
- In `observe` mode, with `SWWAF_MODE=observe`, refuses none of the requests
|
- In `observe` mode, with `SWWAF_MODE=observe`, refuses none of the requests
|
||||||
that `SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec
|
that `SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec
|
||||||
decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit or a
|
decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit, a trap
|
||||||
rule would refuse: it passes them to the app, and their log lines name what
|
path or a rule would refuse: it passes them to the app, and their log lines
|
||||||
`enforce` mode would have done (see `would_action` in "Request log" below).
|
name what `enforce` mode would have done (see `would_action` in "Request log"
|
||||||
The checks run, and requests and bytes are counted, as in `enforce` mode, with
|
below). The checks run, and requests, bytes and refusals are counted, as in
|
||||||
three differences: neither a broken rate limit or byte limit, a `ban` rule nor
|
`enforce` mode, with three differences: neither a broken rate limit, byte
|
||||||
the CrowdSec decision list makes a ban; a broken limit does not set the
|
limit or error burst, a `ban` rule, a trap path nor the CrowdSec decision list
|
||||||
client's counters back to zero, so each request over a rate limit is logged as
|
makes a ban; a broken limit does not set the client's counters back to zero,
|
||||||
one that would be refused, and each whose bytes keep the client over a byte
|
so each request over a rate limit is logged as one that would be refused, each
|
||||||
limit as breaking it; and a request under a ban does not make it permanent. As
|
whose bytes keep the client over a byte limit as breaking it, and each refusal
|
||||||
in `enforce` mode, the bytes counted are only those of the requests `enforce`
|
that keeps it over the error burst as breaking that; and a request under a ban
|
||||||
mode would have passed to the app. A ban it would have made, or made
|
does not make it permanent. As in `enforce` mode, the bytes counted are only
|
||||||
permanent, raises the alert `enforce` mode would have raised, marked as what
|
those of the requests `enforce` mode would have passed to the app, and the
|
||||||
would have happened (see "Alerts" below). The bans in `bans.json` are kept,
|
refusals counted for the error burst only those it would have made: a request
|
||||||
and refuse requests again when `smallwebwaf` next runs in `enforce` mode, as
|
it would have refused before it reached an endpoint is not counted for its
|
||||||
long as they last. The timeouts and size limits still apply, since they
|
token. A ban it would have made, or made permanent, raises the alert `enforce`
|
||||||
protect `smallwebwaf` and the app themselves, and a request for one of
|
mode would have raised, marked as what would have happened (see "Alerts"
|
||||||
`smallwebwaf`'s own endpoints without its token is still answered `401`. It is
|
below). The bans in `bans.json` are kept, and refuse requests again when
|
||||||
for trying a configuration before enforcing it.
|
`smallwebwaf` next runs in `enforce` mode, as long as they last. The timeouts
|
||||||
|
and size limits still apply, since they protect `smallwebwaf` and the app
|
||||||
|
themselves, and a request for one of `smallwebwaf`'s own endpoints without its
|
||||||
|
token is still answered `401`. It is for trying a configuration before
|
||||||
|
enforcing it.
|
||||||
- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
|
- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
|
||||||
check and without asking the app, for the image's health check.
|
check and without asking the app, for the image's health check.
|
||||||
- Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for
|
- Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for
|
||||||
@@ -385,8 +418,8 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
|
|||||||
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body.
|
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body.
|
||||||
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip bans, the
|
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip bans, the
|
||||||
country lists, the blocklists, the CrowdSec decision list, the DNSBL zones,
|
country lists, the blocklists, the CrowdSec decision list, the DNSBL zones,
|
||||||
AbuseIPDB, the rate limits, the byte limits and the rule files, such as your
|
AbuseIPDB, the rate limits, the byte limits, the trap paths, the rule files
|
||||||
monitoring or your own networks.
|
and the error burst, such as your monitoring or your own networks.
|
||||||
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients the
|
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients the
|
||||||
rate limits and the byte limits do not apply to, such as a machine that talks
|
rate limits and the byte limits do not apply to, such as a machine that talks
|
||||||
to the app all day.
|
to the app all day.
|
||||||
@@ -530,21 +563,21 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
|
|||||||
- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone, or a
|
- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone, or a
|
||||||
check with AbuseIPDB, may take before it fails.
|
check with AbuseIPDB, may take before it fails.
|
||||||
- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered, one
|
- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered, one
|
||||||
that is banned, breaks a rate limit, matches a `ban` rule, is in
|
that is banned, breaks a rate limit, matches a `ban` rule, asks for a trap
|
||||||
`SWWAF_DENY_NETS`, comes from a refused country, is in a blocklist while
|
path, is in `SWWAF_DENY_NETS`, comes from a refused country, is in a blocklist
|
||||||
`SWWAF_BLOCKLIST_ACTION` is `deny` or is listed by a DNSBL zone or AbuseIPDB
|
while `SWWAF_BLOCKLIST_ACTION` is `deny` or is listed by a DNSBL zone or
|
||||||
while `SWWAF_REPUTATION_ACTION` is `deny`: `403`, `429`, or `close` to close
|
AbuseIPDB while `SWWAF_REPUTATION_ACTION` is `deny`: `403`, `429`, or `close`
|
||||||
the connection without an answer. Behind traefik, `close` does not leave the
|
to close the connection without an answer. Behind traefik, `close` does not
|
||||||
client unanswered: traefik answers `502`, as it does whenever its backend
|
leave the client unanswered: traefik answers `502`, as it does whenever its
|
||||||
drops a connection. A `block` rule always answers `403`.
|
backend drops a connection. A `block` rule always answers `403`.
|
||||||
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate
|
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate
|
||||||
limit or byte limit.
|
limit, byte limit or error burst.
|
||||||
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a rate limit or byte limit
|
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a rate limit, byte limit or
|
||||||
broken again within this time after a ban ended, other than one for a clear
|
error burst broken again within this time after a ban ended, other than one
|
||||||
sign of attack or for CrowdSec's decision, bans for three times as long as
|
for a clear sign of attack or for CrowdSec's decision, bans for three times as
|
||||||
that ban.
|
long as that ban.
|
||||||
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken rate limit or byte
|
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken rate limit, byte
|
||||||
limit that would be longer is permanent instead.
|
limit or error burst that would be longer is permanent instead.
|
||||||
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
||||||
attack.
|
attack.
|
||||||
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
|
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
|
||||||
@@ -588,6 +621,18 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
|
|||||||
rule files. A directory that does not exist stops the start.
|
rule files. A directory that does not exist stops the start.
|
||||||
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file, and checks
|
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file, and checks
|
||||||
no request against one.
|
no request against one.
|
||||||
|
- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only
|
||||||
|
scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of gitea; a
|
||||||
|
request for one bans its client for a clear sign of attack, as a `ban` rule
|
||||||
|
does, with or without rule files (see "What it does so far" above). A path
|
||||||
|
that does not start with `/`, or holds a `?`, which no path a `path` rule sees
|
||||||
|
holds, stops the start.
|
||||||
|
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most requests of a client in
|
||||||
|
a minute that `smallwebwaf` may refuse after a rule file match or a trap path,
|
||||||
|
or for a missing or wrong token; one more breaks the error burst, and bans the
|
||||||
|
client as a broken rate limit does (see "What it does so far" above). A client
|
||||||
|
trying one probe or token after another is refused many times a minute, a
|
||||||
|
person rarely more than a few times. `off` switches it off.
|
||||||
- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server that every line on
|
- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server that every line on
|
||||||
stdout is also sent to, as `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`
|
stdout is also sent to, as `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`
|
||||||
with a host and a port, such as `syslog+tls://logs.example:6514`. Unset or
|
with a host and a port, such as `syslog+tls://logs.example:6514`. Unset or
|
||||||
@@ -671,18 +716,19 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
|
|||||||
|
|
||||||
Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are
|
Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are
|
||||||
bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024
|
bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024
|
||||||
bytes). Rate limits and the anomaly thresholds on requests are whole numbers of
|
bytes). Rate limits, `SWWAF_ERROR_BURST_THRESHOLD` and the anomaly thresholds on
|
||||||
requests, and byte limits and the anomaly thresholds on bytes are sizes.
|
requests are whole numbers of requests, and byte limits and the anomaly
|
||||||
Netblocks are in CIDR form, and a bare address stands for itself alone.
|
thresholds on bytes are sizes. Netblocks are in CIDR form, and a bare address
|
||||||
Countries are the two-letter codes ISO 3166-1 assigns today, and `xk` for
|
stands for itself alone. Countries are the two-letter codes ISO 3166-1 assigns
|
||||||
Kosovo, in either case (`de` and `DE` are the same); any other code, such as
|
today, and `xk` for Kosovo, in either case (`de` and `DE` are the same); any
|
||||||
`nk` (North Korea is `kp`) or the withdrawn `su`, stops the start, and so does a
|
other code, such as `nk` (North Korea is `kp`) or the withdrawn `su`, stops the
|
||||||
code on both country lists. AS numbers are `AS` and the number, in either case.
|
start, and so does a code on both country lists. AS numbers are `AS` and the
|
||||||
Percentages are whole numbers from 0 to 100, and an entry of a list of them is
|
number, in either case. Percentages are whole numbers from 0 to 100, and an
|
||||||
an AS number or a country, `:` and a percentage; an AS number or a country
|
entry of a list of them is an AS number or a country, `:` and a percentage; an
|
||||||
listed twice in one of them stops the start. `off` switches a timeout, a size
|
AS number or a country listed twice in one of them stops the start. `off`
|
||||||
limit, a rate limit, a byte limit, an anomaly threshold, `SWWAF_ALERT_COOLDOWN`
|
switches a timeout, a size limit, a rate limit, a byte limit, an anomaly
|
||||||
or `SWWAF_ALERT_MAX_PER_HOUR` off; `SWWAF_IPV6_GROUP_PREFIX`,
|
threshold, `SWWAF_ERROR_BURST_THRESHOLD`, `SWWAF_ALERT_COOLDOWN` or
|
||||||
|
`SWWAF_ALERT_MAX_PER_HOUR` off; `SWWAF_IPV6_GROUP_PREFIX`,
|
||||||
`SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
|
`SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
|
||||||
`SWWAF_LOOKUP_TIMEOUT`, `SWWAF_UNKNOWN_LIMIT_PERCENT`,
|
`SWWAF_LOOKUP_TIMEOUT`, `SWWAF_UNKNOWN_LIMIT_PERCENT`,
|
||||||
`SWWAF_BLOCKLIST_REFRESH`, `SWWAF_ABUSEIPDB_MIN_SCORE`,
|
`SWWAF_BLOCKLIST_REFRESH`, `SWWAF_ABUSEIPDB_MIN_SCORE`,
|
||||||
@@ -781,20 +827,20 @@ which every line has.
|
|||||||
refused because its client is in `SWWAF_DENY_NETS`, in a blocklist while
|
refused because its client is in `SWWAF_DENY_NETS`, in a blocklist while
|
||||||
`SWWAF_BLOCKLIST_ACTION` is `deny`, or listed by a DNSBL zone or AbuseIPDB
|
`SWWAF_BLOCKLIST_ACTION` is `deny`, or listed by a DNSBL zone or AbuseIPDB
|
||||||
while `SWWAF_REPUTATION_ACTION` is `deny`, `banned` for one refused because a
|
while `SWWAF_REPUTATION_ACTION` is `deny`, `banned` for one refused because a
|
||||||
ban covers its client, or because it matched a `ban` rule or the CrowdSec
|
ban covers its client, or because it matched a `ban` rule, asked for a trap
|
||||||
decision list lists its client, either of which bans its client,
|
path or the CrowdSec decision list lists its client, each of which bans its
|
||||||
`country_denied` for one refused for its client's country, `rate_limited` for
|
client, `country_denied` for one refused for its client's country,
|
||||||
one that broke a rate limit and banned its client, `rule_blocked` for one a
|
`rate_limited` for one that broke a rate limit and banned its client,
|
||||||
`block` rule refused, `too_large` for a request or response over its size
|
`rule_blocked` for one a `block` rule refused, `too_large` for a request or
|
||||||
limit, `timed_out` for one that ran out of time, `upstream_error` when the app
|
response over its size limit, `timed_out` for one that ran out of time,
|
||||||
could not be reached or its answer broke off, and `admin` for one
|
`upstream_error` when the app could not be reached or its answer broke off,
|
||||||
`smallwebwaf` answered at its own endpoint.
|
and `admin` for one `smallwebwaf` answered at its own endpoint.
|
||||||
- `would_action` is there in `observe` mode for a request that
|
- `would_action` is there in `observe` mode for a request that
|
||||||
`SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec
|
`SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec
|
||||||
decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit or a
|
decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit, a trap
|
||||||
rule would have refused in `enforce` mode, and names the action that refusal
|
path or a rule would have refused in `enforce` mode, and names the action that
|
||||||
would have had: `denied`, `banned`, `country_denied`, `rate_limited` or
|
refusal would have had: `denied`, `banned`, `country_denied`, `rate_limited`
|
||||||
`rule_blocked`. `action` then names what was done: `forward` for a request
|
or `rule_blocked`. `action` then names what was done: `forward` for a request
|
||||||
passed to the app, and another action, such as `too_large`, for one a size or
|
passed to the app, and another action, such as `too_large`, for one a size or
|
||||||
time limit refused.
|
time limit refused.
|
||||||
- `limit_percent` is there for a request the rate limits count whose client a
|
- `limit_percent` is there for a request the rate limits count whose client a
|
||||||
@@ -822,13 +868,15 @@ which every line has.
|
|||||||
counted before it.
|
counted before it.
|
||||||
- `rule_ids` is there for a request that matched rules of the rule files, and
|
- `rule_ids` is there for a request that matched rules of the rule files, and
|
||||||
lists their ids in the order they matched, up to the one that refused it.
|
lists their ids in the order they matched, up to the one that refused it.
|
||||||
- `limit_hit` is there for a request that broke a rate limit, or whose bytes
|
- `limit_hit` is there for a request that broke a rate limit or the error burst,
|
||||||
broke a byte limit, and names the window whose limit it went over as `counts`
|
or whose bytes broke a byte limit, and names the window whose limit it went
|
||||||
names it: `minute`, `hour` or `day` for a rate limit, and `minute_bytes`,
|
over as `counts` names it: `minute`, `hour` or `day` for a rate limit, and
|
||||||
`hour_bytes` or `day_bytes` for a byte limit, the shortest if it went over
|
`minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit, the shortest if
|
||||||
several. `offence` is then `limit`. A request whose bytes broke a byte limit
|
it went over several; or `error_burst` for the error burst. `offence` is then
|
||||||
is not refused: its `action` is what it would have been otherwise, such as
|
`limit`. A request whose bytes broke a byte limit is not refused: its `action`
|
||||||
`forward`.
|
is what it would have been otherwise, such as `forward`. Nor is one that broke
|
||||||
|
the error burst refused for that: its `action` is that of the refusal counted,
|
||||||
|
such as `rule_blocked` or `admin`.
|
||||||
- `reputation` is there for a request whose client a blocklist, the CrowdSec
|
- `reputation` is there for a request whose client a blocklist, the CrowdSec
|
||||||
decision list or a DNSBL zone's verdict lists, or whose AbuseIPDB score is a
|
decision list or a DNSBL zone's verdict lists, or whose AbuseIPDB score is a
|
||||||
hit, and gives the URLs of the blocklists that list it, in the order
|
hit, and gives the URLs of the blocklists that list it, in the order
|
||||||
@@ -1101,33 +1149,42 @@ which are listed by scope first, and the copies of the lists, listed by URL,
|
|||||||
with times in UTC.
|
with times in UTC.
|
||||||
|
|
||||||
- `bans.json`: every ban with its notes, indented to be read. A permanent ban's
|
- `bans.json`: every ban with its notes, indented to be read. A permanent ban's
|
||||||
`expires` is `null`. A ban's `cause` is `limit` for a broken rate limit or
|
`expires` is `null`. A ban's `cause` is `limit` for a broken rate limit, byte
|
||||||
byte limit, `attack` for a clear sign of attack or `crowdsec` for a client the
|
limit or error burst, `attack` for a clear sign of attack or `crowdsec` for a
|
||||||
CrowdSec decision list lists, for a ban `smallwebwaf` made, and `admin` for
|
client the CrowdSec decision list lists, for a ban `smallwebwaf` made, and
|
||||||
one you made or keep. Its `reason` is a short text: for a ban `smallwebwaf`
|
`admin` for one you made or keep. Its `reason` is a short text: for a ban
|
||||||
made, the limit broken, such as `requests per minute over the limit of 1000`
|
`smallwebwaf` made, the limit broken, such as
|
||||||
or `bytes per hour over the limit of 21474836480`, the rule that matched, such
|
`requests per minute over the limit of 1000`,
|
||||||
as `matched the rule env-file`, or the scenario that made CrowdSec's decision,
|
`bytes per hour over the limit of 21474836480` or
|
||||||
such as `CrowdSec's decision for crowdsecurity/ssh-bf`; for yours, what you
|
`refusals per minute over the limit of 30`, the rule that matched, such as
|
||||||
wrote. Its `lifted` is when you lifted it, and is left out until you do. The
|
`matched the rule env-file`, the trap path asked for, such as
|
||||||
`kind` in the notes of a ban for a broken limit is `requests` or `bytes`, what
|
`asked for the trap path /wp-login.php`, or the scenario that made CrowdSec's
|
||||||
the limit is on. For a limit a biased threshold lowered, the reason and the
|
decision, such as `CrowdSec's decision for crowdsecurity/ssh-bf`; for yours,
|
||||||
notes' `limit` give the lowered limit, and the notes' `limit_percent` and
|
what you wrote. Its `lifted` is when you lifted it, and is left out until you
|
||||||
`limit_percent_setting` the client's percentage of that kind of limit and the
|
do. The `kind` in the notes of a ban for a broken limit is `requests`, `bytes`
|
||||||
setting that gave it. The notes' `reputation` gives each blocklist, the
|
or `refusals`, for the error burst, what the limit is on. The notes of a ban
|
||||||
CrowdSec decision list, each DNSBL zone or AbuseIPDB that listed the client
|
for a clear sign of attack give the `rule_id` and the `target` of the rule
|
||||||
when the ban was made, as its `source`, named and ordered as in the request
|
that matched, or the `trap_path` asked for. For a limit a biased threshold
|
||||||
log's `reputation`, with AbuseIPDB's `score` of the client. It is left out
|
lowered, the reason and the notes' `limit` give the lowered limit, and the
|
||||||
when none did, and the example below shows it.
|
notes' `limit_percent` and `limit_percent_setting` the client's percentage of
|
||||||
|
that kind of limit and the setting that gave it. The notes' `reputation` gives
|
||||||
|
each blocklist, the CrowdSec decision list, each DNSBL zone or AbuseIPDB that
|
||||||
|
listed the client when the ban was made, as its `source`, named and ordered as
|
||||||
|
in the request log's `reputation`, with AbuseIPDB's `score` of the client. It
|
||||||
|
is left out when none did, and the example below shows it.
|
||||||
- `clients.json`: each client's two buckets of requests in the minute, the hour
|
- `clients.json`: each client's two buckets of requests in the minute, the hour
|
||||||
and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes`
|
and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes`
|
||||||
and `day_bytes`, and its history: when it was first and last seen, its AS
|
and `day_bytes`, its two buckets of refusals in the minute, which the error
|
||||||
number, AS name and country as last looked up and when the lookup gave them,
|
burst counts, `minute_refusals`, and its history: when it was first and last
|
||||||
its requests, how many were forwarded and how many refused (one `smallwebwaf`
|
seen, its AS number, AS name and country as last looked up and when the lookup
|
||||||
answered at its own endpoints is neither, unless it was refused with `401` for
|
gave them, its requests, how many were forwarded and how many refused (one
|
||||||
a missing or wrong token), the body bytes in each direction, its responses by
|
`smallwebwaf` answered at its own endpoints is neither, unless it was refused
|
||||||
status class and its offences by kind. Each client is on a line of its own, so
|
with `401` for a missing or wrong token), the body bytes in each direction,
|
||||||
`grep` shows everything about one.
|
its responses by status class and its offences by kind: `limit` for a broken
|
||||||
|
rate limit, byte limit or error burst, `attack` for a clear sign of attack,
|
||||||
|
`rule_blocked` for a request a `block` rule refused and `token_refused` for
|
||||||
|
one refused for a missing or wrong token. Each client is on a line of its own,
|
||||||
|
so `grep` shows everything about one.
|
||||||
- `lookups.json`: GeoJS's answers, one to a line, each with the client's AS
|
- `lookups.json`: GeoJS's answers, one to a line, each with the client's AS
|
||||||
number, AS name and country, when GeoJS gave it and when it was last used.
|
number, AS name and country, when GeoJS gave it and when it was last used.
|
||||||
- `reputation.json`: each list fetched from a URL (see "Blocklists" and
|
- `reputation.json`: each list fetched from a URL (see "Blocklists" and
|
||||||
@@ -1243,19 +1300,19 @@ and the line and column where Go's JSON decoder gives them; so does a state
|
|||||||
directory `smallwebwaf` cannot write. So does an entry without a field it needs,
|
directory `smallwebwaf` cannot write. So does an entry without a field it needs,
|
||||||
named with the entry's place in the file: a ban's `netblock`, `start` or
|
named with the entry's place in the file: a ban's `netblock`, `start` or
|
||||||
`expires`, which is `null` for a permanent ban; a client's `client`, or the
|
`expires`, which is `null` for a permanent ban; a client's `client`, or the
|
||||||
`start` of a window in which it has requests or bytes; an answer's `client`,
|
`start` of a window in which it has requests, bytes or refusals; an answer's
|
||||||
`country`, which is `""` for a client GeoJS cannot place, or `answered`; a
|
`client`, `country`, which is `""` for a client GeoJS cannot place, or
|
||||||
cooldown's `event` or `sent`; an alert waiting's `event` or `time`; an anomaly
|
`answered`; a cooldown's `event` or `sent`; an alert waiting's `event` or
|
||||||
counter's `netblock`, unless it counts an AS number or the whole service, its
|
`time`; an anomaly counter's `netblock`, unless it counts an AS number or the
|
||||||
`asn`, for an AS number, its `name`, for a named netblock, or the `start` of a
|
whole service, its `asn`, for an AS number, its `name`, for a named netblock, or
|
||||||
window in which it has requests or bytes; a list's `url`, `fetched` or `lines`,
|
the `start` of a window in which it has requests or bytes; a list's `url`,
|
||||||
which is `[]` for an empty list; a verdict's `zone`, `client`, `listed`, which
|
`fetched` or `lines`, which is `[]` for an empty list; a verdict's `zone`,
|
||||||
is `false` for a client the zone does not list, or `fetched`. So does a ban
|
`client`, `listed`, which is `false` for a client the zone does not list, or
|
||||||
whose `cause` is not `limit`, `attack`, `admin` or `crowdsec`, alerts waiting
|
`fetched`. So does a ban whose `cause` is not `limit`, `attack`, `admin` or
|
||||||
for a destination that is not `webhook`, `slack` or `ntfy`, an anomaly counter
|
`crowdsec`, alerts waiting for a destination that is not `webhook`, `slack` or
|
||||||
whose `scope` is not `client`, `net`, `asn`, `total` or `watch`, and a copy of a
|
`ntfy`, an anomaly counter whose `scope` is not `client`, `net`, `asn`, `total`
|
||||||
list with a line that would make its fetch fail. An answer's `asn` or `as_name`
|
or `watch`, and a copy of a list with a line that would make its fetch fail. An
|
||||||
left out reads as empty.
|
answer's `asn` or `as_name` left out reads as empty.
|
||||||
|
|
||||||
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
||||||
a state file as soon as you save it: what the file then holds replaces what
|
a state file as soon as you save it: what the file then holds replaces what
|
||||||
@@ -1406,9 +1463,11 @@ scraped, and keeps this one as `exported_instance` unless the scrape sets
|
|||||||
from then on, as histograms; `smallwebwaf_requests_in_flight`: the requests
|
from then on, as histograms; `smallwebwaf_requests_in_flight`: the requests
|
||||||
under way.
|
under way.
|
||||||
- `smallwebwaf_rate_limit_hits_total` by `window`, `minute`, `hour` or `day`,
|
- `smallwebwaf_rate_limit_hits_total` by `window`, `minute`, `hour` or `day`,
|
||||||
and `kind`, `requests` for a rate limit or `bytes` for a byte limit,
|
and `kind`, `requests` for a rate limit, `bytes` for a byte limit or
|
||||||
|
`refusals` for the error burst, whose window is `minute`,
|
||||||
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
|
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
|
||||||
limit was passed, `smallwebwaf_offences_total` by `kind`, and
|
limit was passed, `smallwebwaf_offences_total` by `kind`, `limit`, `attack`,
|
||||||
|
`rule_blocked` or `token_refused`, as `clients.json` counts them, and
|
||||||
`smallwebwaf_bans_made_total` by `cause`, `limit`, `attack`, `admin` or
|
`smallwebwaf_bans_made_total` by `cause`, `limit`, `attack`, `admin` or
|
||||||
`crowdsec`, `admin` for the bans you add through `POST /_smallwebwaf/bans`,
|
`crowdsec`, `admin` for the bans you add through `POST /_smallwebwaf/bans`,
|
||||||
and those whose `cause` is `admin` that you add to `bans.json` while
|
and those whose `cause` is `admin` that you add to `bans.json` while
|
||||||
@@ -1521,7 +1580,8 @@ or lift is written to `bans.json` `SWWAF_STATE_WRITE_DELAY` later. Refusals, and
|
|||||||
the answers to requests that cannot be read, are plain text.
|
the answers to requests that cannot be read, are plain text.
|
||||||
|
|
||||||
A request without the token, or with another, such as the metrics token, is
|
A request without the token, or with another, such as the metrics token, is
|
||||||
answered `401`, in `observe` mode too. While the token is unset, each of these
|
answered `401`, in `observe` mode too, and counts toward the error burst, as one
|
||||||
|
for the metrics without theirs does. While the token is unset, each of these
|
||||||
answers `404`, as does any request under `/_smallwebwaf/` that is not for one of
|
answers `404`, as does any request under `/_smallwebwaf/` that is not for one of
|
||||||
its endpoints. Like the metrics, these requests go through every check any other
|
its endpoints. Like the metrics, these requests go through every check any other
|
||||||
request goes through, and are answered where another would be passed to the app:
|
request goes through, and are answered where another would be passed to the app:
|
||||||
@@ -1679,13 +1739,14 @@ For each request `smallwebwaf`:
|
|||||||
- picks the client's limit percentage from those;
|
- picks the client's limit percentage from those;
|
||||||
- checks the minute, hour and day request counters against the limits, and bans
|
- checks the minute, hour and day request counters against the limits, and bans
|
||||||
the client if it breaks one;
|
the client if it breaks one;
|
||||||
- checks the request against the rule files and the Core Rule Set, and bans the
|
- checks the request against the trap paths, the rule files and the Core Rule
|
||||||
client at once for a clear sign of attack;
|
Set, and bans the client at once for a clear sign of attack;
|
||||||
- forwards it to the app and streams the response back, within the size and time
|
- forwards it to the app and streams the response back, within the size and time
|
||||||
limits;
|
limits;
|
||||||
- counts the bytes and any refusal by the rule files or the Core Rule Set, bans
|
- counts the bytes and any refusal by the trap paths, the rule files or the Core
|
||||||
the client if it broke a limit, updates its history and the anomaly counters,
|
Rule Set or for a missing or wrong token, bans the client if it broke a limit,
|
||||||
sends any alerts that are due, and writes the log line.
|
updates its history and the anomaly counters, sends any alerts that are due,
|
||||||
|
and writes the log line.
|
||||||
|
|
||||||
A minimal deployment is the app's own Dockerfile, built on the `smallwebwaf`
|
A minimal deployment is the app's own Dockerfile, built on the `smallwebwaf`
|
||||||
image, with no setting. That image is built on Ubuntu 26.04 LTS, the newest
|
image, with no setting. That image is built on Ubuntu 26.04 LTS, the newest
|
||||||
@@ -2054,14 +2115,16 @@ alerts, the metrics nor `reputation.json` hold it. Given as a file, with
|
|||||||
request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for
|
request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for
|
||||||
a ban, for the country lists, for a blocklist, for the CrowdSec decision list,
|
a ban, for the country lists, for a blocklist, for the CrowdSec decision list,
|
||||||
which bans the client, for a DNSBL zone's verdict, for AbuseIPDB's score, for
|
which bans the client, for a DNSBL zone's verdict, for AbuseIPDB's score, for
|
||||||
a rate limit, which bans the client, for a `block` or `ban` rule, the latter
|
a rate limit, which bans the client, for a trap path, which bans the client,
|
||||||
banning the client, and for an announced body over the size limit; in
|
for a `block` or `ban` rule, the latter banning the client, and for an
|
||||||
`observe` mode, only for the size limit, with what it would have refused for
|
announced body over the size limit; in `observe` mode, only for the size
|
||||||
noted in the log line. A request under `/_smallwebwaf/` that `check` lets
|
limit, with what it would have refused for noted in the log line. A request
|
||||||
through is answered by `answerAdmin` instead of reaching the app. Once the
|
under `/_smallwebwaf/` that `check` lets through is answered by `answerAdmin`
|
||||||
answer to a request passed to the app has ended, `countBytes` counts its bytes
|
instead of reaching the app. Once the answer to a request passed to the app
|
||||||
for the byte limits, and once any request but the health check has ended,
|
has ended, `countBytes` counts its bytes for the byte limits, and once any
|
||||||
`countAnomalies` counts it for the anomaly thresholds.
|
request but the health check has ended, `countRefusal` counts it for the error
|
||||||
|
burst if it was refused after a rule file match or a trap path or for its
|
||||||
|
token, and `countAnomalies` counts it for the anomaly thresholds.
|
||||||
- `internal/metrics`: the metrics, counted as the other parts tell it what
|
- `internal/metrics`: the metrics, counted as the other parts tell it what
|
||||||
happened, and served in the Prometheus text format.
|
happened, and served in the Prometheus text format.
|
||||||
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how
|
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how
|
||||||
@@ -2083,9 +2146,9 @@ alerts, the metrics nor `reputation.json` hold it. Given as a file, with
|
|||||||
tells which zones' verdicts list an address; and checks clients with AbuseIPDB
|
tells which zones' verdicts list an address; and checks clients with AbuseIPDB
|
||||||
in the background, keeps their scores and the checks spent today, and tells
|
in the background, keeps their scores and the checks spent today, and tells
|
||||||
whether a client's score is a hit.
|
whether a client's score is a hit.
|
||||||
- `internal/ratelimit`: the table of clients: counts each client's requests and
|
- `internal/ratelimit`: the table of clients: counts each client's requests,
|
||||||
bytes, tells when they take it over a rate limit or a byte limit, and keeps
|
bytes and refusals, tells when they take it over a rate limit, a byte limit or
|
||||||
each client's history.
|
the error burst, and keeps each client's history.
|
||||||
- `internal/anomaly`: the anomaly counters: counts each request and its bytes
|
- `internal/anomaly`: the anomaly counters: counts each request and its bytes
|
||||||
per client, per netblock around a client, per AS number, for the whole service
|
per client, per netblock around a client, per AS number, for the whole service
|
||||||
and per named netblock, in the buckets `internal/ratelimit` counts in, and
|
and per named netblock, in the buckets `internal/ratelimit` counts in, and
|
||||||
|
|||||||
+18
-10
@@ -1,6 +1,7 @@
|
|||||||
// Package bans is the ban ledger: the bans smallwebwaf makes on the
|
// Package bans is the ban ledger: the bans smallwebwaf makes on the
|
||||||
// netblocks of clients that break a rate limit or a byte limit, show a
|
// netblocks of clients that break a rate limit, a byte limit or the error
|
||||||
// clear sign of attack or are listed by the CrowdSec decision list, and
|
// burst, show a clear sign of attack or are listed by the CrowdSec
|
||||||
|
// decision list, and
|
||||||
// those an admin makes, with their notes, as the "Bans" section of SPEC.md
|
// those an admin makes, with their notes, as the "Bans" section of SPEC.md
|
||||||
// describes. The bans are kept in memory, and written to bans.json and
|
// describes. The bans are kept in memory, and written to bans.json and
|
||||||
// read from it by the state package.
|
// read from it by the state package.
|
||||||
@@ -103,10 +104,11 @@ type Notes struct {
|
|||||||
ASName string `json:"as_name"`
|
ASName string `json:"as_name"`
|
||||||
Country string `json:"country"`
|
Country string `json:"country"`
|
||||||
// Kind, Limit, Window and Count are, for a ban for a broken limit,
|
// Kind, Limit, Window and Count are, for a ban for a broken limit,
|
||||||
// what the limit was on, "requests" for a rate limit or "bytes" for a
|
// what the limit was on, "requests" for a rate limit, "bytes" for a
|
||||||
// byte limit, the limit that was broken, its window, "minute", "hour"
|
// byte limit or "refusals" for the error burst, the limit that was
|
||||||
// or "day", and the count reached: the client's requests, or bytes, in
|
// broken, its window, "minute", "hour" or "day", and the count reached:
|
||||||
// the window, those of the request that broke the limit included.
|
// the client's requests, bytes or refusals in the window, those of the
|
||||||
|
// request that broke the limit included.
|
||||||
// These are what counted toward the ban, and the window is the time
|
// These are what counted toward the ban, and the window is the time
|
||||||
// over which they came.
|
// over which they came.
|
||||||
Kind string `json:"kind,omitempty"`
|
Kind string `json:"kind,omitempty"`
|
||||||
@@ -120,9 +122,11 @@ type Notes struct {
|
|||||||
LimitPercent *int64 `json:"limit_percent,omitempty"`
|
LimitPercent *int64 `json:"limit_percent,omitempty"`
|
||||||
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
|
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
|
||||||
// RuleID and Target are, for a ban for a clear sign of attack, the id
|
// RuleID and Target are, for a ban for a clear sign of attack, the id
|
||||||
// of the rule file rule that matched, and its target.
|
// of the rule file rule that matched, and its target; TrapPath is, for
|
||||||
RuleID string `json:"rule_id,omitempty"`
|
// one for a request for a path in SWWAF_TRAP_PATHS, that path.
|
||||||
Target string `json:"target,omitempty"`
|
RuleID string `json:"rule_id,omitempty"`
|
||||||
|
Target string `json:"target,omitempty"`
|
||||||
|
TrapPath string `json:"trap_path,omitempty"`
|
||||||
// Reputation is the reputation sources that listed the client when
|
// Reputation is the reputation sources that listed the client when
|
||||||
// the request that caused the ban was made, in the order the request
|
// the request that caused the ban was made, in the order the request
|
||||||
// log's reputation names them. It is left out when none did.
|
// log's reputation names them. It is left out when none did.
|
||||||
@@ -311,7 +315,7 @@ func (l *Ledger) WouldBanForLimit(
|
|||||||
// notes, and returns the ban, and whether it made it, as BanForLimit
|
// notes, and returns the ban, and whether it made it, as BanForLimit
|
||||||
// does. A first ban lasts AttackBanDuration; once the netblock has had
|
// does. A first ban lasts AttackBanDuration; once the netblock has had
|
||||||
// one that was not lifted, the next is permanent. Its reason is "matched
|
// one that was not lifted, the next is permanent. Its reason is "matched
|
||||||
// the rule <RuleID>".
|
// the rule <RuleID>", or "asked for the trap path <TrapPath>".
|
||||||
func (l *Ledger) BanForAttack(
|
func (l *Ledger) BanForAttack(
|
||||||
netblock netip.Prefix, now time.Time, notes Notes,
|
netblock netip.Prefix, now time.Time, notes Notes,
|
||||||
) (Ban, bool) {
|
) (Ban, bool) {
|
||||||
@@ -382,6 +386,10 @@ func limitReason(notes Notes) string {
|
|||||||
// attackReason is the reason of a ban for a clear sign of attack, with
|
// attackReason is the reason of a ban for a clear sign of attack, with
|
||||||
// notes.
|
// notes.
|
||||||
func attackReason(notes Notes) string {
|
func attackReason(notes Notes) string {
|
||||||
|
if notes.TrapPath != "" {
|
||||||
|
return "asked for the trap path " + notes.TrapPath
|
||||||
|
}
|
||||||
|
|
||||||
return "matched the rule " + notes.RuleID
|
return "matched the rule " + notes.RuleID
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -239,6 +239,14 @@ type Config struct {
|
|||||||
// unless RulesEnabled is false (SWWAF_RULES_ENABLED).
|
// unless RulesEnabled is false (SWWAF_RULES_ENABLED).
|
||||||
RulesDir string
|
RulesDir string
|
||||||
RulesEnabled bool
|
RulesEnabled bool
|
||||||
|
// TrapPaths are the paths a request for which is a clear sign of
|
||||||
|
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
|
||||||
|
TrapPaths []string
|
||||||
|
// ErrorBurstThreshold is the most requests of a client within a minute
|
||||||
|
// that smallwebwaf may refuse after a rule file match or for a missing
|
||||||
|
// or wrong token; one more breaks a limit
|
||||||
|
// (SWWAF_ERROR_BURST_THRESHOLD). 0 is off.
|
||||||
|
ErrorBurstThreshold int64
|
||||||
// LogRemoteURL is where every line on stdout is also sent
|
// LogRemoteURL is where every line on stdout is also sent
|
||||||
// (SWWAF_LOG_REMOTE_URL), nil while it is unset and nothing is sent.
|
// (SWWAF_LOG_REMOTE_URL), nil while it is unset and nothing is sent.
|
||||||
// LogRemoteTLSCAs are the certificates a syslog+tls endpoint's
|
// LogRemoteTLSCAs are the certificates a syslog+tls endpoint's
|
||||||
@@ -378,6 +386,8 @@ var (
|
|||||||
errNotBytesCount = errors.New("is not response, request or both")
|
errNotBytesCount = errors.New("is not response, request or both")
|
||||||
errNotPathPrefix = errors.New(
|
errNotPathPrefix = errors.New(
|
||||||
"is not a path prefix starting with /, such as /assets/")
|
"is not a path prefix starting with /, such as /assets/")
|
||||||
|
errNotTrapPath = errors.New(
|
||||||
|
"is not a path starting with / and without a ?, such as /wp-login.php")
|
||||||
errNotBoolean = errors.New("is not true or false")
|
errNotBoolean = errors.New("is not true or false")
|
||||||
errNotLogRemoteURL = errors.New(
|
errNotLogRemoteURL = errors.New(
|
||||||
"is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " +
|
"is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " +
|
||||||
@@ -505,6 +515,8 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
|
|||||||
MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
|
MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
|
||||||
RulesDir: env.value("SWWAF_RULES_DIR", "/etc/smallwebwaf/rules.d"),
|
RulesDir: env.value("SWWAF_RULES_DIR", "/etc/smallwebwaf/rules.d"),
|
||||||
RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"),
|
RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"),
|
||||||
|
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
|
||||||
|
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
|
||||||
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
|
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
|
||||||
LogRemoteTLSCAs: env.certificates("SWWAF_LOG_REMOTE_TLS_CA_FILE"),
|
LogRemoteTLSCAs: env.certificates("SWWAF_LOG_REMOTE_TLS_CA_FILE"),
|
||||||
LogRemoteBuffer: env.numberNotOff("SWWAF_LOG_REMOTE_BUFFER", "10000"),
|
LogRemoteBuffer: env.numberNotOff("SWWAF_LOG_REMOTE_BUFFER", "10000"),
|
||||||
@@ -744,6 +756,15 @@ func (e *environment) pathPrefixes(name, defaultValue string) []string {
|
|||||||
return prefixes
|
return prefixes
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// trapPaths reads the setting that is the list of trap paths. It is empty
|
||||||
|
// by default.
|
||||||
|
func (e *environment) trapPaths(name string) []string {
|
||||||
|
paths, err := parseTrapPaths(e.value(name, ""))
|
||||||
|
e.check(name, err)
|
||||||
|
|
||||||
|
return paths
|
||||||
|
}
|
||||||
|
|
||||||
// countries reads a setting that is a list of countries.
|
// countries reads a setting that is a list of countries.
|
||||||
func (e *environment) countries(name, defaultValue string) []string {
|
func (e *environment) countries(name, defaultValue string) []string {
|
||||||
countries, err := parseCountries(e.value(name, defaultValue))
|
countries, err := parseCountries(e.value(name, defaultValue))
|
||||||
@@ -1536,6 +1557,24 @@ func parsePathPrefixes(value string) ([]string, error) {
|
|||||||
return prefixes, nil
|
return prefixes, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// parseTrapPaths reads a comma-separated list of trap paths. Each is
|
||||||
|
// matched against a request's path as a path rule is, without the query,
|
||||||
|
// so a path that does not start with / or holds a ? would never match.
|
||||||
|
func parseTrapPaths(value string) ([]string, error) {
|
||||||
|
paths, err := parseList(value)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, path := range paths {
|
||||||
|
if !strings.HasPrefix(path, "/") || strings.Contains(path, "?") {
|
||||||
|
return nil, fmt.Errorf("%q %w", path, errNotTrapPath)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return paths, nil
|
||||||
|
}
|
||||||
|
|
||||||
// countryCodes are the two-letter codes ISO 3166-1 assigns today, and XK,
|
// countryCodes are the two-letter codes ISO 3166-1 assigns today, and XK,
|
||||||
// the code in common use for Kosovo. golang.org/x/text/language cannot
|
// the code in common use for Kosovo. golang.org/x/text/language cannot
|
||||||
// check them: it also takes withdrawn codes such as su, and reserved ones
|
// check them: it also takes withdrawn codes such as su, and reserved ones
|
||||||
|
|||||||
@@ -91,6 +91,8 @@ const (
|
|||||||
logLevel = "SWWAF_LOG_LEVEL"
|
logLevel = "SWWAF_LOG_LEVEL"
|
||||||
rulesDir = "SWWAF_RULES_DIR"
|
rulesDir = "SWWAF_RULES_DIR"
|
||||||
rulesEnabled = "SWWAF_RULES_ENABLED"
|
rulesEnabled = "SWWAF_RULES_ENABLED"
|
||||||
|
trapPaths = "SWWAF_TRAP_PATHS"
|
||||||
|
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
||||||
logRemoteURL = "SWWAF_LOG_REMOTE_URL"
|
logRemoteURL = "SWWAF_LOG_REMOTE_URL"
|
||||||
logRemoteTLSCAFile = "SWWAF_LOG_REMOTE_TLS_CA_FILE"
|
logRemoteTLSCAFile = "SWWAF_LOG_REMOTE_TLS_CA_FILE"
|
||||||
logRemoteBuffer = "SWWAF_LOG_REMOTE_BUFFER"
|
logRemoteBuffer = "SWWAF_LOG_REMOTE_BUFFER"
|
||||||
@@ -495,6 +497,62 @@ func TestPathPrefixNotStartingWithSlashStopsTheStart(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestTrapPathsAndErrorBurstThreshold(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
env environment
|
||||||
|
paths []string
|
||||||
|
threshold int64
|
||||||
|
}{
|
||||||
|
{environment{}, []string{}, 30},
|
||||||
|
{
|
||||||
|
environment{trapPaths: "/wp-login.php, /xmlrpc.php", errorBurstThreshold: "5"},
|
||||||
|
[]string{"/wp-login.php", "/xmlrpc.php"}, 5,
|
||||||
|
},
|
||||||
|
{environment{errorBurstThreshold: off}, []string{}, 0},
|
||||||
|
} {
|
||||||
|
cfg := fromEnvironment(t, tc.env)
|
||||||
|
if !slices.Equal(cfg.TrapPaths, tc.paths) ||
|
||||||
|
cfg.ErrorBurstThreshold != tc.threshold {
|
||||||
|
t.Errorf("%v gave %v and %d, want %v and %d", tc.env, cfg.TrapPaths,
|
||||||
|
cfg.ErrorBurstThreshold, tc.paths, tc.threshold)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInvalidTrapPathOrErrorBurstThresholdStopsTheStart(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const notTrapPath = " is not a path starting with / and without a ?, " +
|
||||||
|
"such as /wp-login.php"
|
||||||
|
|
||||||
|
for _, tc := range []struct{ name, value, want string }{
|
||||||
|
{trapPaths, "/wp-login.php,xmlrpc.php", `"xmlrpc.php"` + notTrapPath},
|
||||||
|
{trapPaths, "/xmlrpc.php?rsd", `"/xmlrpc.php?rsd"` + notTrapPath},
|
||||||
|
{
|
||||||
|
trapPaths, "/wp-login.php,,/xmlrpc.php",
|
||||||
|
`"/wp-login.php,,/xmlrpc.php" has an empty item in its list`,
|
||||||
|
},
|
||||||
|
{errorBurstThreshold, "0", `"0" must be more than zero, or off`},
|
||||||
|
{
|
||||||
|
errorBurstThreshold, "30/min",
|
||||||
|
`"30/min" is not a whole number of requests such as 1000, or off`,
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
|
|
||||||
|
want := tc.name + ": " + tc.want
|
||||||
|
if err == nil || err.Error() != want {
|
||||||
|
t.Errorf("error %v, want %s", err, want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestInstanceNameAndLoggedHeadersAsSet(t *testing.T) {
|
func TestInstanceNameAndLoggedHeadersAsSet(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -2233,6 +2291,8 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
|||||||
logLevel: "info",
|
logLevel: "info",
|
||||||
rulesDir: "/etc/smallwebwaf/rules.d",
|
rulesDir: "/etc/smallwebwaf/rules.d",
|
||||||
rulesEnabled: "true",
|
rulesEnabled: "true",
|
||||||
|
trapPaths: "",
|
||||||
|
errorBurstThreshold: "30",
|
||||||
logRemoteURL: "",
|
logRemoteURL: "",
|
||||||
logRemoteTLSCAFile: "",
|
logRemoteTLSCAFile: "",
|
||||||
logRemoteBuffer: "10000",
|
logRemoteBuffer: "10000",
|
||||||
|
|||||||
+24
-19
@@ -6,7 +6,6 @@ package metrics
|
|||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
@@ -94,8 +93,8 @@ func New(topN int, instanceName string) *Metrics {
|
|||||||
Help: "How long requests passed to the app took, from then to their end.",
|
Help: "How long requests passed to the app took, from then to their end.",
|
||||||
}),
|
}),
|
||||||
rateLimitHits: counterVec("smallwebwaf_rate_limit_hits_total",
|
rateLimitHits: counterVec("smallwebwaf_rate_limit_hits_total",
|
||||||
"Requests that broke a rate limit or a byte limit, by its window and "+
|
"Requests that broke a rate limit, a byte limit or the error burst, by "+
|
||||||
"its kind, requests or bytes.",
|
"its window and its kind, requests, bytes or refusals.",
|
||||||
[]string{"window", "kind"}),
|
[]string{"window", "kind"}),
|
||||||
sizeAndTimeLimitHits: counterVec("smallwebwaf_size_and_time_limit_hits_total",
|
sizeAndTimeLimitHits: counterVec("smallwebwaf_size_and_time_limit_hits_total",
|
||||||
"Requests that passed a size or time limit, by its setting.",
|
"Requests that passed a size or time limit, by its setting.",
|
||||||
@@ -407,26 +406,10 @@ func (m *Metrics) RequestEnded(
|
|||||||
m.upstreamDuration.Observe(upstreamDuration.Seconds())
|
m.upstreamDuration.Observe(upstreamDuration.Seconds())
|
||||||
}
|
}
|
||||||
|
|
||||||
if line.LimitHit != "" {
|
|
||||||
// The log line names a byte limit's window with _bytes after it.
|
|
||||||
window, isBytes := strings.CutSuffix(line.LimitHit, "_bytes")
|
|
||||||
|
|
||||||
kind := ratelimit.KindRequests
|
|
||||||
if isBytes {
|
|
||||||
kind = ratelimit.KindBytes
|
|
||||||
}
|
|
||||||
|
|
||||||
m.rateLimitHits.WithLabelValues(window, kind).Inc()
|
|
||||||
}
|
|
||||||
|
|
||||||
if limit != "" {
|
if limit != "" {
|
||||||
m.sizeAndTimeLimitHits.WithLabelValues(limit).Inc()
|
m.sizeAndTimeLimitHits.WithLabelValues(limit).Inc()
|
||||||
}
|
}
|
||||||
|
|
||||||
if line.Offence != "" {
|
|
||||||
m.offences.WithLabelValues(line.Offence).Inc()
|
|
||||||
}
|
|
||||||
|
|
||||||
if line.Country != "" {
|
if line.Country != "" {
|
||||||
m.countries.add(line.Country, line)
|
m.countries.add(line.Country, line)
|
||||||
}
|
}
|
||||||
@@ -436,6 +419,28 @@ func (m *Metrics) RequestEnded(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// LimitHit counts a request that broke a rate limit, a byte limit or the
|
||||||
|
// error burst, by the window and the kind of hit.
|
||||||
|
func (m *Metrics) LimitHit(hit ratelimit.Hit) {
|
||||||
|
m.rateLimitHits.WithLabelValues(hit.Window, hit.Kind).Inc()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Offences counts the offences of r, a request that has ended, as its
|
||||||
|
// client's history counts them, by kind, named as clients.json names
|
||||||
|
// them.
|
||||||
|
func (m *Metrics) Offences(r ratelimit.Request) {
|
||||||
|
for kind, committed := range map[string]bool{
|
||||||
|
"limit": r.BrokeLimit,
|
||||||
|
"attack": r.Attack,
|
||||||
|
"rule_blocked": r.RuleBlocked,
|
||||||
|
"token_refused": r.TokenRefused,
|
||||||
|
} {
|
||||||
|
if committed {
|
||||||
|
m.offences.WithLabelValues(kind).Inc()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// RuleMatched counts a request that matched the rule id, whose action is
|
// RuleMatched counts a request that matched the rule id, whose action is
|
||||||
// action.
|
// action.
|
||||||
func (m *Metrics) RuleMatched(id, action string) {
|
func (m *Metrics) RuleMatched(id, action string) {
|
||||||
|
|||||||
@@ -45,8 +45,9 @@ var (
|
|||||||
// /_smallwebwaf/, once it has passed the checks. Each endpoint needs a
|
// /_smallwebwaf/, once it has passed the checks. Each endpoint needs a
|
||||||
// token, sent as Authorization: Bearer <token>: the metrics
|
// token, sent as Authorization: Bearer <token>: the metrics
|
||||||
// SWWAF_METRICS_TOKEN, the others SWWAF_ADMIN_TOKEN. A request without
|
// SWWAF_METRICS_TOKEN, the others SWWAF_ADMIN_TOKEN. A request without
|
||||||
// it is refused with 401. An endpoint whose token is unset answers 404,
|
// it is refused with 401, which counts toward the error burst. An
|
||||||
// as any other request under /_smallwebwaf/ does.
|
// endpoint whose token is unset answers 404, as any other request under
|
||||||
|
// /_smallwebwaf/ does.
|
||||||
func (rq *request) answerAdmin() {
|
func (rq *request) answerAdmin() {
|
||||||
rq.line.Action = requestlog.ActionAdmin
|
rq.line.Action = requestlog.ActionAdmin
|
||||||
rq.startClientResponseTimeout()
|
rq.startClientResponseTimeout()
|
||||||
@@ -57,6 +58,7 @@ func (rq *request) answerAdmin() {
|
|||||||
case token == "":
|
case token == "":
|
||||||
http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
|
http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
|
||||||
case !hasToken(rq.in, token):
|
case !hasToken(rq.in, token):
|
||||||
|
rq.tokenRefused = true
|
||||||
rq.out.Header().Set("WWW-Authenticate", "Bearer")
|
rq.out.Header().Set("WWW-Authenticate", "Bearer")
|
||||||
rq.answer(refusal{
|
rq.answer(refusal{
|
||||||
status: http.StatusUnauthorized,
|
status: http.StatusUnauthorized,
|
||||||
|
|||||||
+76
-30
@@ -1,6 +1,7 @@
|
|||||||
package proxy
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -9,7 +10,6 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// banResponse is a refusal answered with SWWAF_BAN_RESPONSE, and logged
|
// banResponse is a refusal answered with SWWAF_BAN_RESPONSE, and logged
|
||||||
@@ -83,6 +83,48 @@ func (rq *request) countBytes() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// countRefusal counts the request for the error burst once it has been
|
||||||
|
// answered, if smallwebwaf refused it after a rule file match or a trap
|
||||||
|
// path, or for a missing or wrong token, and in observe mode if enforce
|
||||||
|
// mode would have: more than SWWAF_ERROR_BURST_THRESHOLD such refusals of
|
||||||
|
// the client within a minute break a limit. A client in SWWAF_ALLOW_NETS,
|
||||||
|
// which the checks skip, is not counted, and nothing is while the
|
||||||
|
// threshold is off.
|
||||||
|
func (rq *request) countRefusal() {
|
||||||
|
cfg := rq.h.config
|
||||||
|
if cfg.ErrorBurstThreshold == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// In observe mode, a request that enforce mode would have refused
|
||||||
|
// before it reached the endpoint has had no token refused there.
|
||||||
|
tokenRefused := rq.tokenRefused && rq.line.WouldAction == "" &&
|
||||||
|
!isInside(rq.client, cfg.AllowNets)
|
||||||
|
if !rq.attack && !rq.ruleBlocked && !tokenRefused {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
now := rq.h.now()
|
||||||
|
|
||||||
|
hit, over := rq.h.limiter.CountRefusal(rq.h.clientGroup(rq.client), now,
|
||||||
|
cfg.ErrorBurstThreshold)
|
||||||
|
if !over {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// What the client was sent, or in observe mode would have been.
|
||||||
|
status := rq.out.status
|
||||||
|
|
||||||
|
switch rq.line.WouldAction {
|
||||||
|
case requestlog.ActionRuleBlocked:
|
||||||
|
status = http.StatusForbidden
|
||||||
|
case requestlog.ActionBanned:
|
||||||
|
status = cfg.BanResponse
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.banForLimit(now, hit, status)
|
||||||
|
}
|
||||||
|
|
||||||
// countedBytes returns the request's bytes, once it has ended, as the
|
// countedBytes returns the request's bytes, once it has ended, as the
|
||||||
// byte limits and the anomaly thresholds count them: the response's body
|
// byte limits and the anomaly thresholds count them: the response's body
|
||||||
// bytes, the request's, or both, as SWWAF_BYTES_COUNT says. For an
|
// bytes, the request's, or both, as SWWAF_BYTES_COUNT says. For an
|
||||||
@@ -107,20 +149,27 @@ func (rq *request) countedBytes() int64 {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// banForLimit bans the client's netblock at now for a broken limit, the
|
// banForLimit bans the client's netblock at now for a broken limit, the
|
||||||
// one hit names, and notes the offence for the log line. status is what
|
// one hit names, notes the offence for the log line and counts the hit in
|
||||||
// the client was sent, or is sent: SWWAF_BAN_RESPONSE for a request over
|
// the metrics. status is what the client was sent, or is sent:
|
||||||
// a rate limit, the app's answer for one whose bytes broke a byte limit.
|
// SWWAF_BAN_RESPONSE for a request over a rate limit, the app's answer for
|
||||||
// The ban's notes give the client's limit percentage for that kind of
|
// one whose bytes broke a byte limit, the refusal for one that broke the
|
||||||
// limit. The ban sets the client's counters back to zero. In observe mode
|
// error burst. The ban's notes give the client's limit percentage for a
|
||||||
// it makes no ban and sets nothing back, and raises the alert for the ban
|
// rate limit or a byte limit; the error burst is not lowered. The ban sets
|
||||||
// it would have made, if that alert would be sent.
|
// the client's counters back to zero. In observe mode it makes no ban and
|
||||||
|
// sets nothing back, and raises the alert for the ban it would have made,
|
||||||
|
// if that alert would be sent.
|
||||||
func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
|
func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
|
||||||
rq.line.LimitHit = hit.Window
|
switch hit.Kind {
|
||||||
if hit.Kind == ratelimit.KindBytes {
|
case ratelimit.KindBytes:
|
||||||
rq.line.LimitHit += "_bytes" // as counts names the byte totals
|
rq.line.LimitHit = hit.Window + "_bytes" // as counts names the byte totals
|
||||||
|
case ratelimit.KindRefusals:
|
||||||
|
rq.line.LimitHit = requestlog.LimitHitErrorBurst
|
||||||
|
default:
|
||||||
|
rq.line.LimitHit = hit.Window
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.line.Offence = requestlog.OffenceLimit
|
rq.line.Offence = requestlog.OffenceLimit
|
||||||
|
rq.h.metrics.LimitHit(hit)
|
||||||
|
|
||||||
netblock := rq.h.netblock(rq.client)
|
netblock := rq.h.netblock(rq.client)
|
||||||
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
|
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
|
||||||
@@ -140,13 +189,13 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
|
|||||||
Requests: rq.netblockRequests(netblock),
|
Requests: rq.netblockRequests(netblock),
|
||||||
}
|
}
|
||||||
|
|
||||||
percent := rq.limitPercent
|
switch hit.Kind {
|
||||||
if hit.Kind == ratelimit.KindBytes {
|
case ratelimit.KindRequests:
|
||||||
percent = rq.bytesPercent
|
notes.LimitPercent, notes.LimitPercentSetting = rq.limitPercent.logged()
|
||||||
|
case ratelimit.KindBytes:
|
||||||
|
notes.LimitPercent, notes.LimitPercentSetting = rq.bytesPercent.logged()
|
||||||
}
|
}
|
||||||
|
|
||||||
notes.LimitPercent, notes.LimitPercentSetting = percent.logged()
|
|
||||||
|
|
||||||
if rq.h.config.Observe {
|
if rq.h.config.Observe {
|
||||||
ban, wouldBan := rq.h.ledger.WouldBanForLimit(netblock, now, notes)
|
ban, wouldBan := rq.h.ledger.WouldBanForLimit(netblock, now, notes)
|
||||||
if wouldBan {
|
if wouldBan {
|
||||||
@@ -166,25 +215,22 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// banForAttack bans the client's netblock at now for a clear sign of
|
// banForAttack bans the client's netblock at now for a clear sign of
|
||||||
// attack, the match of rule, a ban rule. In observe mode it makes no ban,
|
// attack, which notes name: the ban rule that matched, or the trap path
|
||||||
// and raises the alert for the ban it would have made, if that alert
|
// asked for. It fills in the rest of the notes. In observe mode it makes
|
||||||
// would be sent.
|
// no ban, and raises the alert for the ban it would have made, if that
|
||||||
func (rq *request) banForAttack(now time.Time, rule rules.Rule) {
|
// alert would be sent.
|
||||||
|
func (rq *request) banForAttack(now time.Time, notes bans.Notes) {
|
||||||
netblock := rq.h.netblock(rq.client)
|
netblock := rq.h.netblock(rq.client)
|
||||||
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
|
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
notes := bans.Notes{
|
notes.ASN = rq.line.ASN
|
||||||
ASN: rq.line.ASN,
|
notes.ASName = rq.line.ASName
|
||||||
ASName: rq.line.ASName,
|
notes.Country = rq.line.Country
|
||||||
Country: rq.line.Country,
|
notes.Reputation = rq.reputation
|
||||||
RuleID: rule.ID,
|
notes.Request = rq.noted(now, rq.h.config.BanResponse)
|
||||||
Target: rule.Target,
|
notes.Requests = rq.netblockRequests(netblock)
|
||||||
Reputation: rq.reputation,
|
|
||||||
Request: rq.noted(now, rq.h.config.BanResponse),
|
|
||||||
Requests: rq.netblockRequests(netblock),
|
|
||||||
}
|
|
||||||
|
|
||||||
if rq.h.config.Observe {
|
if rq.h.config.Observe {
|
||||||
ban, wouldBan := rq.h.ledger.WouldBanForAttack(netblock, now, notes)
|
ban, wouldBan := rq.h.ledger.WouldBanForAttack(netblock, now, notes)
|
||||||
|
|||||||
@@ -0,0 +1,396 @@
|
|||||||
|
package proxy_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"maps"
|
||||||
|
"net/http"
|
||||||
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
const errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
||||||
|
|
||||||
|
// refused is a request the tests here send, which smallwebwaf refuses
|
||||||
|
// after a rule file match, or for a missing or wrong token.
|
||||||
|
type refused int
|
||||||
|
|
||||||
|
const (
|
||||||
|
// blockRule is a request testRules' block rule refuses with 403.
|
||||||
|
blockRule refused = iota
|
||||||
|
// banRule is one its ban rule refuses with 403, and bans the client
|
||||||
|
// for.
|
||||||
|
banRule
|
||||||
|
// noMetricsToken is one for the metrics without a token, and
|
||||||
|
// wrongAdminToken one for the bans with the metrics token, each
|
||||||
|
// refused with 401.
|
||||||
|
noMetricsToken
|
||||||
|
wrongAdminToken
|
||||||
|
)
|
||||||
|
|
||||||
|
// send sends r from the client at from, checks its answer and log line as
|
||||||
|
// sender.request does, and returns the line.
|
||||||
|
func (r refused) send(s *sender, from string) logLine {
|
||||||
|
s.t.Helper()
|
||||||
|
|
||||||
|
switch r {
|
||||||
|
case blockRule:
|
||||||
|
return s.request(from, blockedPath, http.StatusForbidden,
|
||||||
|
requestlog.ActionRuleBlocked)
|
||||||
|
case banRule:
|
||||||
|
return s.request(from, probePath, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
case noMetricsToken:
|
||||||
|
return s.request(from, proxy.MetricsPath, http.StatusUnauthorized,
|
||||||
|
requestlog.ActionAdmin)
|
||||||
|
case wrongAdminToken:
|
||||||
|
line, _ := s.requestWithHeader(from, proxy.BansPath, "Authorization: "+bearer,
|
||||||
|
http.StatusUnauthorized, requestlog.ActionAdmin)
|
||||||
|
|
||||||
|
return line
|
||||||
|
}
|
||||||
|
|
||||||
|
s.t.Fatalf("no request for the refusal %d", r)
|
||||||
|
|
||||||
|
return logLine{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// startForErrorBurst is startWithClock with testRules, both tokens and
|
||||||
|
// SWWAF_ERROR_BURST_THRESHOLD at threshold, and the settings in env.
|
||||||
|
func startForErrorBurst(
|
||||||
|
t *testing.T, threshold string, env map[string]string,
|
||||||
|
) (*sender, *clock, *proxy.Server) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
settings := map[string]string{
|
||||||
|
errorBurstThreshold: threshold,
|
||||||
|
rulesDir: writeRules(t, testRules),
|
||||||
|
adminToken: adminSecret,
|
||||||
|
metricsToken: token,
|
||||||
|
}
|
||||||
|
maps.Copy(settings, env)
|
||||||
|
|
||||||
|
return startWithClock(t, "", settings)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestErrorBurstBreaksAtOneOverTheThreshold(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
// refusals are four, one over the threshold of three.
|
||||||
|
refusals []refused
|
||||||
|
}{
|
||||||
|
{"block rule", []refused{blockRule, blockRule, blockRule, blockRule}},
|
||||||
|
{
|
||||||
|
"missing or wrong token",
|
||||||
|
[]refused{noMetricsToken, wrongAdminToken, noMetricsToken, wrongAdminToken},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"a mix ending in a ban rule",
|
||||||
|
[]refused{blockRule, noMetricsToken, blockRule, banRule},
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startForErrorBurst(t, "3", nil)
|
||||||
|
|
||||||
|
// Three refusals break nothing, and the app's answers between
|
||||||
|
// them are not counted.
|
||||||
|
for i, r := range tc.refusals[:3] {
|
||||||
|
line := r.send(s, client)
|
||||||
|
if line.LimitHit != "" || line.Offence != "" {
|
||||||
|
t.Errorf("refusal %d: log line has limit_hit %q and offence %q, "+
|
||||||
|
"want none", i+1, line.LimitHit, line.Offence)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The fourth is answered as the others were, breaks the error
|
||||||
|
// burst, and bans the client.
|
||||||
|
line := tc.refusals[3].send(s, client)
|
||||||
|
if line.LimitHit != requestlog.LimitHitErrorBurst ||
|
||||||
|
line.Offence != requestlog.OffenceLimit {
|
||||||
|
t.Errorf("log line has limit_hit %q and offence %q, want error_burst "+
|
||||||
|
"and limit", line.LimitHit, line.Offence)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestErrorBurstBanNotesHistoryAndMetrics(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const scraper = "192.0.2.200"
|
||||||
|
|
||||||
|
s, clk, server := startForErrorBurst(t, "2", nil)
|
||||||
|
start := clk.Now()
|
||||||
|
|
||||||
|
blockRule.send(s, client)
|
||||||
|
wrongAdminToken.send(s, client)
|
||||||
|
line := blockRule.send(s, client)
|
||||||
|
|
||||||
|
expires := start.Add(time.Hour)
|
||||||
|
if line.BanExpires != requestlog.FormatTime(expires) {
|
||||||
|
t.Errorf("log line has ban_expires %q, want an hour on", line.BanExpires)
|
||||||
|
}
|
||||||
|
|
||||||
|
netblock := netip.MustParsePrefix(client + "/32")
|
||||||
|
want := bans.Ban{
|
||||||
|
Netblock: netblock,
|
||||||
|
Start: start,
|
||||||
|
Expires: expires,
|
||||||
|
Cause: bans.CauseLimit,
|
||||||
|
Reason: "refusals per minute over the limit of 2",
|
||||||
|
Notes: bans.Notes{
|
||||||
|
Kind: ratelimit.KindRefusals,
|
||||||
|
Limit: 2,
|
||||||
|
Window: minute,
|
||||||
|
Count: 3,
|
||||||
|
Request: bans.Request{
|
||||||
|
Time: start,
|
||||||
|
Method: http.MethodGet,
|
||||||
|
Host: appHost,
|
||||||
|
Path: blockedPath,
|
||||||
|
Status: http.StatusForbidden,
|
||||||
|
UserAgent: userAgent,
|
||||||
|
},
|
||||||
|
Requests: 3,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
got := server.Ledger.Bans(netblock)
|
||||||
|
if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
|
||||||
|
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
wantOffences := ratelimit.Offences{Limit: 1, RuleBlocked: 2, TokenRefused: 1}
|
||||||
|
if offences := historyOf(t, server, client).Offences; offences != wantOffences {
|
||||||
|
t.Errorf("history counts the offences %+v, want %+v", offences, wantOffences)
|
||||||
|
}
|
||||||
|
|
||||||
|
metrics := s.scrape(scraper)
|
||||||
|
wantMetric(t, metrics, `smallwebwaf_rate_limit_hits_total{instance="app",`+
|
||||||
|
`kind="refusals",window="minute"}`, 1)
|
||||||
|
wantMetric(t, metrics, `smallwebwaf_offences_total{instance="app",kind="limit"}`, 1)
|
||||||
|
wantMetric(t, metrics,
|
||||||
|
`smallwebwaf_offences_total{instance="app",kind="rule_blocked"}`, 2)
|
||||||
|
wantMetric(t, metrics,
|
||||||
|
`smallwebwaf_offences_total{instance="app",kind="token_refused"}`, 1)
|
||||||
|
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestErrorBurstIsNotLoweredForAClientWithLowerLimits(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
geojsURL, _ := startGeoJS(t)
|
||||||
|
s, _, server := startWithClock(t, geojsURL, map[string]string{
|
||||||
|
errorBurstThreshold: "2",
|
||||||
|
rulesDir: writeRules(t, testRules),
|
||||||
|
countryLimitPercent: countryDEHalf,
|
||||||
|
})
|
||||||
|
|
||||||
|
// Half of the threshold would be one, which the second refusal is over.
|
||||||
|
for range 2 {
|
||||||
|
line := blockRule.send(s, fromDE)
|
||||||
|
if line.LimitHit != "" {
|
||||||
|
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
blockRule.send(s, fromDE)
|
||||||
|
|
||||||
|
got := server.Ledger.Bans(netip.MustParsePrefix(fromDE + "/32"))
|
||||||
|
if len(got) != 1 || got[0].Notes.Limit != 2 || got[0].Notes.LimitPercent != nil {
|
||||||
|
t.Errorf("bans %+v, want one for the limit of 2, without a limit percentage", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestErrorBurstDoesNotCountTheAppsAnswers(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
statuses := map[string]int{
|
||||||
|
"/missing": http.StatusNotFound,
|
||||||
|
"/private": http.StatusUnauthorized,
|
||||||
|
"/forbidden": http.StatusForbidden,
|
||||||
|
}
|
||||||
|
s, _, _, queue := startAppWithAlerts(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(statuses[r.URL.Path])
|
||||||
|
}, map[string]string{errorBurstThreshold: "1", rulesDir: writeRules(t, testRules)})
|
||||||
|
|
||||||
|
for range 2 {
|
||||||
|
for path, status := range statuses {
|
||||||
|
s.request(client, path, status, requestlog.ActionForward)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The first refusal is one, not over the threshold.
|
||||||
|
line := blockRule.send(s, client)
|
||||||
|
if line.LimitHit != "" {
|
||||||
|
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
|
||||||
|
}
|
||||||
|
|
||||||
|
// No ban was made, nor its alert raised.
|
||||||
|
s.request(client, "/missing", http.StatusNotFound, requestlog.ActionForward)
|
||||||
|
wantAlerts(t, queue)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestErrorBurstOffOrAtItsDefault(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const off = "off"
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
threshold string
|
||||||
|
// broken is whether the 31st refusal breaks the error burst.
|
||||||
|
broken bool
|
||||||
|
}{
|
||||||
|
{"", true},
|
||||||
|
{off, false},
|
||||||
|
} {
|
||||||
|
t.Run(errorBurstThreshold+"="+tc.threshold, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := map[string]string{rulesDir: writeRules(t, testRules)}
|
||||||
|
if tc.threshold != "" {
|
||||||
|
env[errorBurstThreshold] = tc.threshold
|
||||||
|
}
|
||||||
|
|
||||||
|
s, _, _ := startWithClock(t, "", env)
|
||||||
|
|
||||||
|
var line logLine
|
||||||
|
for range 31 {
|
||||||
|
line = blockRule.send(s, client)
|
||||||
|
}
|
||||||
|
|
||||||
|
if broken := line.LimitHit == requestlog.LimitHitErrorBurst; broken != tc.broken {
|
||||||
|
t.Errorf("the 31st refusal broke the error burst: %t, want %t",
|
||||||
|
broken, tc.broken)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestErrorBurstCountsEachClientTheChecksApplyTo(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const (
|
||||||
|
allowed = "192.0.2.60" // in SWWAF_ALLOW_NETS
|
||||||
|
exempt = "192.0.2.50" // in SWWAF_RATE_LIMIT_EXEMPT_NETS
|
||||||
|
)
|
||||||
|
|
||||||
|
s, _, _ := startForErrorBurst(t, "1", map[string]string{
|
||||||
|
allowNets: allowed, rateLimitExemptNets: exempt,
|
||||||
|
})
|
||||||
|
|
||||||
|
// A client in SWWAF_ALLOW_NETS still needs the token, but is not
|
||||||
|
// counted.
|
||||||
|
for range 3 {
|
||||||
|
line := noMetricsToken.send(s, allowed)
|
||||||
|
if line.LimitHit != "" {
|
||||||
|
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// One the rate limits do not apply to is.
|
||||||
|
noMetricsToken.send(s, exempt)
|
||||||
|
|
||||||
|
line := wrongAdminToken.send(s, exempt)
|
||||||
|
if line.LimitHit != requestlog.LimitHitErrorBurst {
|
||||||
|
t.Errorf("log line has limit_hit %q, want error_burst", line.LimitHit)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.get(exempt, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestErrorBurstBanSetsTheRefusalsBackToZero(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, clk, _ := startForErrorBurst(t, "1", map[string]string{limitBanDuration: "1s"})
|
||||||
|
|
||||||
|
blockRule.send(s, client)
|
||||||
|
blockRule.send(s, client)
|
||||||
|
|
||||||
|
// Within the same minute, once the ban has ended, the next refusal is
|
||||||
|
// the first again.
|
||||||
|
clk.advance(time.Second)
|
||||||
|
|
||||||
|
line := blockRule.send(s, client)
|
||||||
|
if line.LimitHit != "" {
|
||||||
|
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestObserveModeLogsAndAlertsTheErrorBurst(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, clk, server, queue := startWithAlerts(t, map[string]string{
|
||||||
|
mode: observe,
|
||||||
|
errorBurstThreshold: "1",
|
||||||
|
rulesDir: writeRules(t, testRules),
|
||||||
|
adminToken: adminSecret,
|
||||||
|
})
|
||||||
|
start := clk.Now()
|
||||||
|
held := bans.Ban{
|
||||||
|
Netblock: netip.MustParsePrefix(otherClient + "/32"),
|
||||||
|
Start: start,
|
||||||
|
Expires: start.Add(time.Hour),
|
||||||
|
Cause: bans.CauseAdmin,
|
||||||
|
}
|
||||||
|
server.Ledger.Load([]bans.Ban{held})
|
||||||
|
|
||||||
|
// Under a ban, enforce mode would have refused these before the
|
||||||
|
// endpoint, so their tokens are not counted.
|
||||||
|
for range 2 {
|
||||||
|
line := wrongAdminToken.send(s, otherClient)
|
||||||
|
wantWouldAction(t, line, requestlog.ActionBanned)
|
||||||
|
|
||||||
|
if line.LimitHit != "" {
|
||||||
|
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The block rule's refusal, which enforce mode would have answered 403,
|
||||||
|
// is the second of the client's, and would have banned it.
|
||||||
|
wrongAdminToken.send(s, client)
|
||||||
|
|
||||||
|
line := s.request(client, blockedPath, http.StatusOK, requestlog.ActionForward)
|
||||||
|
wantWouldAction(t, line, requestlog.ActionRuleBlocked)
|
||||||
|
|
||||||
|
if line.LimitHit != requestlog.LimitHitErrorBurst || line.BanExpires != "" {
|
||||||
|
t.Errorf("log line has limit_hit %q and ban_expires %q, want error_burst "+
|
||||||
|
"and none", line.LimitHit, line.BanExpires)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := server.Ledger.Snapshot(); len(got) != 1 || !reflect.DeepEqual(got[0], held) {
|
||||||
|
t.Errorf("bans %+v, want only the one held", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
|
if len(waiting) != 1 {
|
||||||
|
t.Fatalf("%d alerts wait, want 1: %+v", len(waiting), waiting)
|
||||||
|
}
|
||||||
|
|
||||||
|
notes, _ := waiting[0].Detail["notes"].(bans.Notes)
|
||||||
|
if notes.Kind != ratelimit.KindRefusals || notes.Count != 2 ||
|
||||||
|
notes.Request.Status != http.StatusForbidden {
|
||||||
|
t.Errorf("the alert's notes are %+v, want two refusals, the last answered 403",
|
||||||
|
notes)
|
||||||
|
}
|
||||||
|
|
||||||
|
alert := banAlert(alerts.EventBan, start, client, bans.Ban{
|
||||||
|
Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
|
||||||
|
Reason: "refusals per minute over the limit of 1", Notes: notes,
|
||||||
|
}, requestlog.FormatTime(start.Add(time.Hour)))
|
||||||
|
alert.Detail["mode"] = observe
|
||||||
|
wantAlerts(t, queue, alert)
|
||||||
|
}
|
||||||
@@ -282,9 +282,12 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// Once the request has ended, before its log line is written.
|
// Once the request has ended, before its log line is written. The
|
||||||
|
// last deferred runs first: countRefusal before addToHistory, so that
|
||||||
|
// a broken error burst is in the client's history.
|
||||||
defer rq.addToHistory()
|
defer rq.addToHistory()
|
||||||
defer rq.countAnomalies()
|
defer rq.countAnomalies()
|
||||||
|
defer rq.countRefusal()
|
||||||
|
|
||||||
refused := rq.check(r.Context())
|
refused := rq.check(r.Context())
|
||||||
rq.checked = time.Now()
|
rq.checked = time.Now()
|
||||||
|
|||||||
@@ -701,10 +701,14 @@ func TestIPv6ClientCostsOneAbuseIPDBCheckWhicheverOfItsAddressesSends(t *testing
|
|||||||
wantAbuseIPDBChecks(t, server, 1)
|
wantAbuseIPDBChecks(t, server, 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
// probePath is the path the ban rule of testRules, probe, matches.
|
// probePath is the path the ban rule of testRules, probe, matches, and
|
||||||
const probePath = "/.env"
|
// blockedPath the one its block rule, blocked, matches.
|
||||||
|
const (
|
||||||
|
probePath = "/.env"
|
||||||
|
blockedPath = "/blocked"
|
||||||
|
)
|
||||||
|
|
||||||
func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T) {
|
func TestClientRefusedForAnOffenceIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
@@ -718,13 +722,21 @@ func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T)
|
|||||||
want ratelimit.Offences
|
want ratelimit.Offences
|
||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
"a block rule", "/blocked", http.StatusForbidden, requestlog.ActionRuleBlocked,
|
"a block rule", blockedPath, http.StatusForbidden, requestlog.ActionRuleBlocked,
|
||||||
ratelimit.Offences{RuleBlocked: 1},
|
ratelimit.Offences{RuleBlocked: 1},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"a ban rule", probePath, http.StatusForbidden, requestlog.ActionBanned,
|
"a ban rule", probePath, http.StatusForbidden, requestlog.ActionBanned,
|
||||||
ratelimit.Offences{Attack: 1},
|
ratelimit.Offences{Attack: 1},
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"a trap path", "/xmlrpc.php", http.StatusForbidden, requestlog.ActionBanned,
|
||||||
|
ratelimit.Offences{Attack: 1},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"a missing token", proxy.MetricsPath, http.StatusUnauthorized,
|
||||||
|
requestlog.ActionAdmin, ratelimit.Offences{TokenRefused: 1},
|
||||||
|
},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
@@ -732,6 +744,7 @@ func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T)
|
|||||||
s, clk, server := startWithClock(t, "", map[string]string{
|
s, clk, server := startWithClock(t, "", map[string]string{
|
||||||
abuseIPDBKey: accountKey, reputationAction: actionLog,
|
abuseIPDBKey: accountKey, reputationAction: actionLog,
|
||||||
rulesDir: writeRules(t, testRules), attackBanDuration: "1h",
|
rulesDir: writeRules(t, testRules), attackBanDuration: "1h",
|
||||||
|
trapPaths: trapPathList, metricsToken: token,
|
||||||
})
|
})
|
||||||
|
|
||||||
s.request(client, tc.path, tc.status, tc.action)
|
s.request(client, tc.path, tc.status, tc.action)
|
||||||
@@ -741,8 +754,8 @@ func TestClientARuleRefusedIsCheckedWithAbuseIPDBAtItsNextRequest(t *testing.T)
|
|||||||
t.Errorf("history counts the offences %+v, want %+v", got, tc.want)
|
t.Errorf("history counts the offences %+v, want %+v", got, tc.want)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Its next request, once a ban rule's ban has ended, has it
|
// Its next request, once any ban for a clear sign of attack
|
||||||
// checked.
|
// has ended, has it checked.
|
||||||
clk.advance(time.Hour)
|
clk.advance(time.Hour)
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
wantAbuseIPDBChecks(t, server, 1)
|
wantAbuseIPDBChecks(t, server, 1)
|
||||||
|
|||||||
+23
-14
@@ -64,10 +64,11 @@ type request struct {
|
|||||||
// limits and for the byte limits.
|
// limits and for the byte limits.
|
||||||
counted bool
|
counted bool
|
||||||
limitPercent, bytesPercent percentage
|
limitPercent, bytesPercent percentage
|
||||||
// attack is true for a request that matched a ban rule, and
|
// attack is true for a request that matched a ban rule or asked for a
|
||||||
// ruleBlocked for one a block rule refused, each an offence its
|
// trap path, ruleBlocked for one a block rule refused, and
|
||||||
// client's history counts.
|
// tokenRefused for one refused for a missing or wrong token, each an
|
||||||
attack, ruleBlocked bool
|
// offence its client's history counts.
|
||||||
|
attack, ruleBlocked, tokenRefused bool
|
||||||
// blocklisted is true once a blocklist is found to list the client,
|
// blocklisted is true once a blocklist is found to list the client,
|
||||||
// dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once
|
// dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once
|
||||||
// AbuseIPDB's score of it is a hit.
|
// AbuseIPDB's score of it is a hit.
|
||||||
@@ -232,9 +233,9 @@ func (rq *request) check(ctx context.Context) *refusal {
|
|||||||
// rate limits, unless the client is in SWWAF_RATE_LIMIT_EXEMPT_NETS or the
|
// rate limits, unless the client is in SWWAF_RATE_LIMIT_EXEMPT_NETS or the
|
||||||
// request's path is exempt under SWWAF_RATE_LIMIT_EXEMPT_PATHS, so that
|
// request's path is exempt under SWWAF_RATE_LIMIT_EXEMPT_PATHS, so that
|
||||||
// every other request is counted, each of them by the client's limit
|
// every other request is counted, each of them by the client's limit
|
||||||
// percentages, and last the rule files. A request exempt from the rate
|
// percentages, then SWWAF_TRAP_PATHS, and last the rule files. A request
|
||||||
// limits is exempt from the byte limits too. ctx is the request's own
|
// exempt from the rate limits is exempt from the byte limits too. ctx is
|
||||||
// context.
|
// the request's own context.
|
||||||
func (rq *request) checkClient(ctx context.Context) string {
|
func (rq *request) checkClient(ctx context.Context) string {
|
||||||
cfg := rq.h.config
|
cfg := rq.h.config
|
||||||
if isInside(rq.client, cfg.AllowNets) {
|
if isInside(rq.client, cfg.AllowNets) {
|
||||||
@@ -281,6 +282,10 @@ func (rq *request) checkClient(ctx context.Context) string {
|
|||||||
return requestlog.ActionRateLimited
|
return requestlog.ActionRateLimited
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if rq.trapPath(now) {
|
||||||
|
return requestlog.ActionBanned
|
||||||
|
}
|
||||||
|
|
||||||
return rq.checkRules(now)
|
return rq.checkRules(now)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -541,14 +546,14 @@ func timing(start, end time.Time) *float64 {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// addToHistory adds the request, which has ended, to its client's
|
// addToHistory adds the request, which has ended, to its client's
|
||||||
// history, and then the lookup's answer about the client, as
|
// history, and counts its offences in the metrics, and then the lookup's
|
||||||
// answerAtTheEnd gives it, to that history and to the notes of the bans
|
// answer about the client, as answerAtTheEnd gives it, to that history and
|
||||||
// on its netblock: an answer may have come before either was there, and
|
// to the notes of the bans on its netblock: an answer may have come
|
||||||
// one from GeoJS that comes later is added when it comes.
|
// before either was there, and one from GeoJS that comes later is added
|
||||||
|
// when it comes.
|
||||||
func (rq *request) addToHistory() {
|
func (rq *request) addToHistory() {
|
||||||
forwarded := !rq.upstreamStart.IsZero()
|
forwarded := !rq.upstreamStart.IsZero()
|
||||||
|
request := ratelimit.Request{
|
||||||
rq.h.limiter.AddToHistory(rq.h.clientGroup(rq.client), rq.h.now(), ratelimit.Request{
|
|
||||||
Forwarded: forwarded,
|
Forwarded: forwarded,
|
||||||
Refused: !forwarded && rq.refused.Load() != nil,
|
Refused: !forwarded && rq.refused.Load() != nil,
|
||||||
Status: rq.out.status,
|
Status: rq.out.status,
|
||||||
@@ -557,7 +562,11 @@ func (rq *request) addToHistory() {
|
|||||||
BrokeLimit: rq.line.Offence == requestlog.OffenceLimit,
|
BrokeLimit: rq.line.Offence == requestlog.OffenceLimit,
|
||||||
Attack: rq.attack,
|
Attack: rq.attack,
|
||||||
RuleBlocked: rq.ruleBlocked,
|
RuleBlocked: rq.ruleBlocked,
|
||||||
})
|
TokenRefused: rq.tokenRefused,
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.h.limiter.AddToHistory(rq.h.clientGroup(rq.client), rq.h.now(), request)
|
||||||
|
rq.h.metrics.Offences(request)
|
||||||
|
|
||||||
answer, found := rq.answerAtTheEnd()
|
answer, found := rq.answerAtTheEnd()
|
||||||
if found {
|
if found {
|
||||||
|
|||||||
@@ -204,6 +204,9 @@ func TestMetricsCountRuleMatchesAndBansForAnAttack(t *testing.T) {
|
|||||||
wantMetric(t, metrics, `smallwebwaf_rules_loaded{instance="app"}`, 2)
|
wantMetric(t, metrics, `smallwebwaf_rules_loaded{instance="app"}`, 2)
|
||||||
wantMetric(t, metrics, `smallwebwaf_requests_total{action="rule_blocked",`+
|
wantMetric(t, metrics, `smallwebwaf_requests_total{action="rule_blocked",`+
|
||||||
`instance="app",status_class="4xx"}`, 1)
|
`instance="app",status_class="4xx"}`, 1)
|
||||||
|
wantMetric(t, metrics,
|
||||||
|
`smallwebwaf_offences_total{instance="app",kind="rule_blocked"}`, 1)
|
||||||
|
wantMetric(t, metrics, `smallwebwaf_offences_total{instance="app",kind="attack"}`, 1)
|
||||||
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="attack",instance="app"}`, 1)
|
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="attack",instance="app"}`, 1)
|
||||||
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 0)
|
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 0)
|
||||||
wantMetric(t, metrics, `smallwebwaf_permanent_bans{instance="app"}`, 1)
|
wantMetric(t, metrics, `smallwebwaf_permanent_bans{instance="app"}`, 1)
|
||||||
|
|||||||
+20
-1
@@ -1,12 +1,31 @@
|
|||||||
package proxy
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"slices"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// trapPath reports whether the request asks for a path in
|
||||||
|
// SWWAF_TRAP_PATHS: its path as a path rule sees it, before any decoding
|
||||||
|
// and without the query, is one of them. Such a request is a clear sign of
|
||||||
|
// attack, as a ban rule's match is: it bans the client's netblock, or in
|
||||||
|
// observe mode raises the alert for the ban it would have made.
|
||||||
|
func (rq *request) trapPath(now time.Time) bool {
|
||||||
|
path := rules.Path(rq.in)
|
||||||
|
if !slices.Contains(rq.h.config.TrapPaths, path) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.attack = true
|
||||||
|
rq.banForAttack(now, bans.Notes{TrapPath: path})
|
||||||
|
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
// checkRules checks the request against the rules of the rule files at
|
// checkRules checks the request against the rules of the rule files at
|
||||||
// now, notes the ids of those it matches in the log line, and returns the
|
// now, notes the ids of those it matches in the log line, and returns the
|
||||||
// action of the rule that refuses it, ActionRuleBlocked for a block rule
|
// action of the rule that refuses it, ActionRuleBlocked for a block rule
|
||||||
@@ -34,7 +53,7 @@ func (rq *request) checkRules(now time.Time) string {
|
|||||||
return requestlog.ActionRuleBlocked
|
return requestlog.ActionRuleBlocked
|
||||||
case rules.ActionBan:
|
case rules.ActionBan:
|
||||||
rq.attack = true
|
rq.attack = true
|
||||||
rq.banForAttack(now, last)
|
rq.banForAttack(now, bans.Notes{RuleID: last.ID, Target: last.Target})
|
||||||
|
|
||||||
return requestlog.ActionBanned
|
return requestlog.ActionBanned
|
||||||
default:
|
default:
|
||||||
|
|||||||
@@ -0,0 +1,115 @@
|
|||||||
|
package proxy_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// trapPaths is the setting's name, and trapPathList what the tests set it
|
||||||
|
// to.
|
||||||
|
const (
|
||||||
|
trapPaths = "SWWAF_TRAP_PATHS"
|
||||||
|
trapPathList = "/wp-login.php,/xmlrpc.php"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const allowed = "192.0.2.60" // in SWWAF_ALLOW_NETS
|
||||||
|
|
||||||
|
// A block rule for the same path: the trap path comes first.
|
||||||
|
s, clk, server := startWithClock(t, "", map[string]string{
|
||||||
|
trapPaths: trapPathList,
|
||||||
|
rulesDir: writeRules(t, `wp path block ^/wp-login\.php$`),
|
||||||
|
allowNets: allowed,
|
||||||
|
banResponse: "429",
|
||||||
|
})
|
||||||
|
start := clk.Now()
|
||||||
|
|
||||||
|
// Only the path itself, as the client sent it, is a trap path.
|
||||||
|
for _, path := range []string{
|
||||||
|
"/wp-login.php/", "/WP-LOGIN.PHP", "/blog/xmlrpc.php", "/%77p-login.php",
|
||||||
|
} {
|
||||||
|
s.request(otherClient, path, http.StatusOK, requestlog.ActionForward)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A client in SWWAF_ALLOW_NETS is not checked.
|
||||||
|
s.request(allowed, "/xmlrpc.php", http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
|
// The query is not part of the path.
|
||||||
|
line := s.request(client, "/wp-login.php?redirect_to=x", http.StatusTooManyRequests,
|
||||||
|
requestlog.ActionBanned)
|
||||||
|
wantRuleIDs(t, line)
|
||||||
|
|
||||||
|
if line.BanExpires != requestlog.FormatTime(start.Add(7*24*time.Hour)) {
|
||||||
|
t.Errorf("log line has ban_expires %q, want seven days on", line.BanExpires)
|
||||||
|
}
|
||||||
|
|
||||||
|
netblock := netip.MustParsePrefix(client + "/32")
|
||||||
|
want := bans.Ban{
|
||||||
|
Netblock: netblock,
|
||||||
|
Start: start,
|
||||||
|
Expires: start.Add(7 * 24 * time.Hour),
|
||||||
|
Cause: bans.CauseAttack,
|
||||||
|
Reason: "asked for the trap path /wp-login.php",
|
||||||
|
Notes: bans.Notes{
|
||||||
|
TrapPath: "/wp-login.php",
|
||||||
|
Request: bans.Request{
|
||||||
|
Time: start,
|
||||||
|
Method: http.MethodGet,
|
||||||
|
Host: appHost,
|
||||||
|
Path: "/wp-login.php?redirect_to=x",
|
||||||
|
Status: http.StatusTooManyRequests,
|
||||||
|
UserAgent: userAgent,
|
||||||
|
},
|
||||||
|
Requests: 1,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
got := server.Ledger.Bans(netblock)
|
||||||
|
if len(got) != 1 || !reflect.DeepEqual(got[0], want) {
|
||||||
|
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The next request is refused under the ban, and makes it permanent.
|
||||||
|
line = s.get(client, http.StatusTooManyRequests, requestlog.ActionBanned)
|
||||||
|
if line.BanExpires != permanent {
|
||||||
|
t.Errorf("log line has ban_expires %q, want permanent", line.BanExpires)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTrapPathsNeedNoRuleFiles(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startWithClock(t, "", map[string]string{
|
||||||
|
trapPaths: trapPathList,
|
||||||
|
"SWWAF_RULES_ENABLED": "false",
|
||||||
|
})
|
||||||
|
|
||||||
|
s.request(client, "/xmlrpc.php", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestObserveModeLogsWhatATrapPathWouldDo(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, server := startWithClock(t, "", map[string]string{
|
||||||
|
trapPaths: trapPathList,
|
||||||
|
mode: observe,
|
||||||
|
})
|
||||||
|
|
||||||
|
line := s.request(client, "/xmlrpc.php", http.StatusOK, requestlog.ActionForward)
|
||||||
|
wantWouldAction(t, line, requestlog.ActionBanned)
|
||||||
|
|
||||||
|
// No ban was made.
|
||||||
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
|
if got := server.Ledger.Snapshot(); len(got) != 0 {
|
||||||
|
t.Errorf("bans %+v, want none", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -25,6 +25,9 @@ const (
|
|||||||
KindRequests = "requests"
|
KindRequests = "requests"
|
||||||
// KindBytes is a byte limit, on a client's bytes.
|
// KindBytes is a byte limit, on a client's bytes.
|
||||||
KindBytes = "bytes"
|
KindBytes = "bytes"
|
||||||
|
// KindRefusals is the error burst, on a client's requests smallwebwaf
|
||||||
|
// refused after a rule file match or for a missing or wrong token.
|
||||||
|
KindRefusals = "refusals"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Limits are the most requests a client may make in a minute, an hour and
|
// Limits are the most requests a client may make in a minute, an hour and
|
||||||
@@ -50,18 +53,20 @@ type Limiter struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Client is a client in the table, as clients.json holds it: its buckets
|
// Client is a client in the table, as clients.json holds it: its buckets
|
||||||
// of requests and of bytes in each window, and its history.
|
// of requests and of bytes in each window, its buckets of refusals in the
|
||||||
|
// minute, which the error burst counts, and its history.
|
||||||
//
|
//
|
||||||
//nolint:tagliatelle // the state files use snake_case, as the request log does
|
//nolint:tagliatelle // the state files use snake_case, as the request log does
|
||||||
type Client struct {
|
type Client struct {
|
||||||
Client netip.Prefix `json:"client"`
|
Client netip.Prefix `json:"client"`
|
||||||
Minute Buckets `json:"minute"`
|
Minute Buckets `json:"minute"`
|
||||||
Hour Buckets `json:"hour"`
|
Hour Buckets `json:"hour"`
|
||||||
Day Buckets `json:"day"`
|
Day Buckets `json:"day"`
|
||||||
MinuteBytes Buckets `json:"minute_bytes"`
|
MinuteBytes Buckets `json:"minute_bytes"`
|
||||||
HourBytes Buckets `json:"hour_bytes"`
|
HourBytes Buckets `json:"hour_bytes"`
|
||||||
DayBytes Buckets `json:"day_bytes"`
|
DayBytes Buckets `json:"day_bytes"`
|
||||||
History History `json:"history"`
|
MinuteRefusals Buckets `json:"minute_refusals"`
|
||||||
|
History History `json:"history"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Buckets are a client's two buckets in one window: the requests, or the
|
// Buckets are a client's two buckets in one window: the requests, or the
|
||||||
@@ -116,12 +121,15 @@ type Responses struct {
|
|||||||
//
|
//
|
||||||
//nolint:tagliatelle // the state files use snake_case, as the request log does
|
//nolint:tagliatelle // the state files use snake_case, as the request log does
|
||||||
type Offences struct {
|
type Offences struct {
|
||||||
// Limit is its requests that broke a rate limit or a byte limit,
|
// Limit is its requests that broke a rate limit, a byte limit or the
|
||||||
// Attack those that matched a ban rule, a clear sign of attack, and
|
// error burst, Attack those that were a clear sign of attack, a match
|
||||||
// RuleBlocked those a block rule refused.
|
// of a ban rule or a request for a trap path, RuleBlocked those a block
|
||||||
Limit int64 `json:"limit"`
|
// rule refused, and TokenRefused those refused for a missing or wrong
|
||||||
Attack int64 `json:"attack"`
|
// token.
|
||||||
RuleBlocked int64 `json:"rule_blocked"`
|
Limit int64 `json:"limit"`
|
||||||
|
Attack int64 `json:"attack"`
|
||||||
|
RuleBlocked int64 `json:"rule_blocked"`
|
||||||
|
TokenRefused int64 `json:"token_refused"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Request is what a client's history keeps of one of its requests.
|
// Request is what a client's history keeps of one of its requests.
|
||||||
@@ -138,12 +146,14 @@ type Request struct {
|
|||||||
// and of its response.
|
// and of its response.
|
||||||
RequestBytes int64
|
RequestBytes int64
|
||||||
ResponseBytes int64
|
ResponseBytes int64
|
||||||
// BrokeLimit is true for a request that broke a rate limit or a byte
|
// BrokeLimit is true for a request that broke a rate limit, a byte
|
||||||
// limit, Attack for one that matched a ban rule, and RuleBlocked for
|
// limit or the error burst, Attack for one that matched a ban rule or
|
||||||
// one a block rule refused.
|
// asked for a trap path, RuleBlocked for one a block rule refused, and
|
||||||
BrokeLimit bool
|
// TokenRefused for one refused for a missing or wrong token.
|
||||||
Attack bool
|
BrokeLimit bool
|
||||||
RuleBlocked bool
|
Attack bool
|
||||||
|
RuleBlocked bool
|
||||||
|
TokenRefused bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// New returns a Limiter for limits, with no client counted yet, whose
|
// New returns a Limiter for limits, with no client counted yet, whose
|
||||||
@@ -175,17 +185,18 @@ func New(limits Limits, maxClients int) *Limiter {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Hit is a request that takes a client over a rate limit, or whose bytes
|
// Hit is a request that takes a client over a rate limit or the error
|
||||||
// take it over a byte limit.
|
// burst, or whose bytes take it over a byte limit.
|
||||||
type Hit struct {
|
type Hit struct {
|
||||||
// Kind is KindRequests for a rate limit, KindBytes for a byte limit.
|
// Kind is KindRequests for a rate limit, KindBytes for a byte limit,
|
||||||
|
// KindRefusals for the error burst.
|
||||||
Kind string
|
Kind string
|
||||||
// Window is "minute", "hour" or "day".
|
// Window is "minute", "hour" or "day".
|
||||||
Window string
|
Window string
|
||||||
// Limit is the window's limit, as the client's percentage of it.
|
// Limit is the window's limit, as the client's percentage of it.
|
||||||
Limit int64
|
Limit int64
|
||||||
// Count is the client's requests, or bytes, counted in the window,
|
// Count is the client's requests, bytes or refusals counted in the
|
||||||
// this request's included.
|
// window, this request's included.
|
||||||
Count float64
|
Count float64
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -224,8 +235,25 @@ func (l *Limiter) CountBytes(
|
|||||||
return l.count(client, now, 0, bytes, percent)
|
return l.count(client, now, 0, bytes, percent)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Reset sets client's counts of requests and of bytes in every window
|
// CountRefusal counts a request from client at now that smallwebwaf
|
||||||
// back to zero. Its history keeps its totals.
|
// refused after a rule file match or for a missing or wrong token, and
|
||||||
|
// reports whether the client's refusals in the minute that ends at now,
|
||||||
|
// this one included, are more than threshold, which breaks the error
|
||||||
|
// burst, and the hit.
|
||||||
|
func (l *Limiter) CountRefusal(
|
||||||
|
client netip.Prefix, now time.Time, threshold int64,
|
||||||
|
) (Hit, bool) {
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
|
||||||
|
count := l.get(client).MinuteRefusals.Add(now, time.Minute, 1)
|
||||||
|
hit := Hit{Kind: KindRefusals, Window: "minute", Limit: threshold, Count: count}
|
||||||
|
|
||||||
|
return hit, count > float64(threshold)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reset sets client's counts of requests, of bytes and of refusals in
|
||||||
|
// every window back to zero. Its history keeps its totals.
|
||||||
func (l *Limiter) Reset(client netip.Prefix) {
|
func (l *Limiter) Reset(client netip.Prefix) {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
defer l.mu.Unlock()
|
defer l.mu.Unlock()
|
||||||
@@ -234,6 +262,7 @@ func (l *Limiter) Reset(client netip.Prefix) {
|
|||||||
if seen {
|
if seen {
|
||||||
c.Minute, c.Hour, c.Day = Buckets{}, Buckets{}, Buckets{}
|
c.Minute, c.Hour, c.Day = Buckets{}, Buckets{}, Buckets{}
|
||||||
c.MinuteBytes, c.HourBytes, c.DayBytes = Buckets{}, Buckets{}, Buckets{}
|
c.MinuteBytes, c.HourBytes, c.DayBytes = Buckets{}, Buckets{}, Buckets{}
|
||||||
|
c.MinuteRefusals = Buckets{}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -274,6 +303,10 @@ func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
|
|||||||
if r.RuleBlocked {
|
if r.RuleBlocked {
|
||||||
h.Offences.RuleBlocked++
|
h.Offences.RuleBlocked++
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if r.TokenRefused {
|
||||||
|
h.Offences.TokenRefused++
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// AddLookup gives client's history its AS number, AS name and country, as
|
// AddLookup gives client's history its AS number, AS name and country, as
|
||||||
@@ -383,6 +416,10 @@ func (l *Limiter) Load(clients []Client, now time.Time) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if c.MinuteRefusals.Passed(now, time.Minute) {
|
||||||
|
c.MinuteRefusals = Buckets{}
|
||||||
|
}
|
||||||
|
|
||||||
l.clients.Add(c.Client, &c)
|
l.clients.Add(c.Client, &c)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -248,6 +248,46 @@ func TestResetSetsTheBytesBackToZero(t *testing.T) {
|
|||||||
wantBytesCount(t, limiter, client, start, 1000, "")
|
wantBytesCount(t, limiter, client, start, 1000, "")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
|
||||||
|
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
|
start := midnight()
|
||||||
|
|
||||||
|
for range limit {
|
||||||
|
if _, over := limiter.CountRefusal(client, start, limit); over {
|
||||||
|
t.Fatalf("a refusal within the threshold of %d broke the error burst", limit)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
hit, over := limiter.CountRefusal(client, start, limit)
|
||||||
|
|
||||||
|
want := ratelimit.Hit{
|
||||||
|
Kind: ratelimit.KindRefusals, Window: minute, Limit: limit, Count: limit + 1,
|
||||||
|
}
|
||||||
|
if !over || hit != want {
|
||||||
|
t.Errorf("one over the threshold broke it: %t, with %+v; want %+v", over, hit,
|
||||||
|
want)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Half a minute into the next, half of those four still count, 2, and
|
||||||
|
// this one: 3, within the threshold.
|
||||||
|
hit, over = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
|
||||||
|
if over || hit.Count != 3 {
|
||||||
|
t.Errorf("half a minute on, %v refusals broke it: %t; want 3, false",
|
||||||
|
hit.Count, over)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A ban sets them back to zero.
|
||||||
|
limiter.Reset(client)
|
||||||
|
|
||||||
|
hit, _ = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
|
||||||
|
if hit.Count != 1 {
|
||||||
|
t.Errorf("after a reset, %v refusals, want 1", hit.Count)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
|
func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -65,6 +65,7 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
|
|||||||
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
|
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
|
||||||
limiter.Count(client, start, whole)
|
limiter.Count(client, start, whole)
|
||||||
limiter.CountBytes(client, start, 5, whole)
|
limiter.CountBytes(client, start, 5, whole)
|
||||||
|
limiter.CountRefusal(client, start, limit)
|
||||||
limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true})
|
limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true})
|
||||||
|
|
||||||
loaded := func(now time.Time) ratelimit.Client {
|
loaded := func(now time.Time) ratelimit.Client {
|
||||||
@@ -76,9 +77,9 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
|
|||||||
return after.Snapshot()[0]
|
return after.Snapshot()[0]
|
||||||
}
|
}
|
||||||
|
|
||||||
// Two minutes on, the window that ends then covers neither of the
|
// Two minutes on, the window that ends then covers none of the
|
||||||
// minute's buckets, of requests and of bytes, which are emptied; the
|
// minute's buckets, of requests, of bytes and of refusals, which are
|
||||||
// hour's and the day's stay, and so does the history.
|
// emptied; the hour's and the day's stay, and so does the history.
|
||||||
got := loaded(start.Add(2 * time.Minute))
|
got := loaded(start.Add(2 * time.Minute))
|
||||||
if got.Minute != (ratelimit.Buckets{}) || got.Hour.Current != 1 ||
|
if got.Minute != (ratelimit.Buckets{}) || got.Hour.Current != 1 ||
|
||||||
got.Day.Current != 1 || got.History.Requests != 1 {
|
got.Day.Current != 1 || got.History.Requests != 1 {
|
||||||
@@ -91,11 +92,17 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
|
|||||||
got.MinuteBytes, got.HourBytes, got.DayBytes)
|
got.MinuteBytes, got.HourBytes, got.DayBytes)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if got.MinuteRefusals != (ratelimit.Buckets{}) {
|
||||||
|
t.Errorf("loaded two minutes on with buckets of refusals %+v",
|
||||||
|
got.MinuteRefusals)
|
||||||
|
}
|
||||||
|
|
||||||
// A moment before, the window still covers some of the earlier one.
|
// A moment before, the window still covers some of the earlier one.
|
||||||
got = loaded(start.Add(2*time.Minute - time.Nanosecond))
|
got = loaded(start.Add(2*time.Minute - time.Nanosecond))
|
||||||
if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 {
|
if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 ||
|
||||||
t.Errorf("loaded just under two minutes on with minute buckets %+v and %+v",
|
got.MinuteRefusals.Current != 1 {
|
||||||
got.Minute, got.MinuteBytes)
|
t.Errorf("loaded just under two minutes on with minute buckets %+v, %+v "+
|
||||||
|
"and %+v", got.Minute, got.MinuteBytes, got.MinuteRefusals)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -29,8 +29,9 @@ const (
|
|||||||
// over a rate limit, which bans the client.
|
// over a rate limit, which bans the client.
|
||||||
ActionRateLimited = "rate_limited"
|
ActionRateLimited = "rate_limited"
|
||||||
// ActionBanned is a request refused because a ban covers its client,
|
// ActionBanned is a request refused because a ban covers its client,
|
||||||
// or because it matched a ban rule or the CrowdSec decision list lists
|
// or because it matched a ban rule, asked for a trap path or the
|
||||||
// its client, either of which bans the client.
|
// CrowdSec decision list lists its client, each of which bans the
|
||||||
|
// client.
|
||||||
ActionBanned = "banned"
|
ActionBanned = "banned"
|
||||||
// ActionRuleBlocked is a request refused because it matched a block
|
// ActionRuleBlocked is a request refused because it matched a block
|
||||||
// rule.
|
// rule.
|
||||||
@@ -48,9 +49,14 @@ const (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// OffenceLimit is the offence a request line names for a request that
|
// OffenceLimit is the offence a request line names for a request that
|
||||||
// broke a rate limit, or whose bytes broke a byte limit.
|
// broke a rate limit or the error burst, or whose bytes broke a byte
|
||||||
|
// limit.
|
||||||
const OffenceLimit = "limit"
|
const OffenceLimit = "limit"
|
||||||
|
|
||||||
|
// LimitHitErrorBurst is the limit_hit a request line names for a request
|
||||||
|
// that broke the error burst.
|
||||||
|
const LimitHitErrorBurst = "error_burst"
|
||||||
|
|
||||||
// timeLayout is RFC 3339 with milliseconds.
|
// timeLayout is RFC 3339 with milliseconds.
|
||||||
const timeLayout = "2006-01-02T15:04:05.000Z07:00"
|
const timeLayout = "2006-01-02T15:04:05.000Z07:00"
|
||||||
|
|
||||||
@@ -138,7 +144,8 @@ type Line struct {
|
|||||||
RuleIDs []string `json:"rule_ids,omitempty"`
|
RuleIDs []string `json:"rule_ids,omitempty"`
|
||||||
// LimitHit is the window whose limit the request went over, named as
|
// LimitHit is the window whose limit the request went over, named as
|
||||||
// Counts names its count: minute, hour or day for a rate limit, and
|
// Counts names its count: minute, hour or day for a rate limit, and
|
||||||
// minute_bytes, hour_bytes or day_bytes for a byte limit.
|
// minute_bytes, hour_bytes or day_bytes for a byte limit; or
|
||||||
|
// LimitHitErrorBurst for the error burst.
|
||||||
LimitHit string `json:"limit_hit,omitempty"`
|
LimitHit string `json:"limit_hit,omitempty"`
|
||||||
// Reputation are the URLs of the blocklists that list the client, then
|
// Reputation are the URLs of the blocklists that list the client, then
|
||||||
// that of the CrowdSec decision list when it does, then the DNSBL zones
|
// that of the CrowdSec decision list when it does, then the DNSBL zones
|
||||||
|
|||||||
+13
-6
@@ -396,16 +396,23 @@ func (rule Rule) matches(r *http.Request) bool {
|
|||||||
return rule.regex.MatchString(value(rule.Target, r))
|
return rule.regex.MatchString(value(rule.Target, r))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Path returns r's path as the client sent it, before any decoding or
|
||||||
|
// re-encoding, up to the first ?: what a path rule is matched against.
|
||||||
|
func Path(r *http.Request) string {
|
||||||
|
path, _, _ := strings.Cut(pathAndQuery(r), "?")
|
||||||
|
|
||||||
|
return path
|
||||||
|
}
|
||||||
|
|
||||||
// value returns what a rule with target, other than uri, is matched
|
// value returns what a rule with target, other than uri, is matched
|
||||||
// against in r: the path and the query as the client sent them, before
|
// against in r: the path, as Path gives it, and the query as the client
|
||||||
// any decoding or re-encoding, split at the first ?, and a header's values
|
// sent it, before any decoding or re-encoding, after the first ?, and a
|
||||||
// joined by ", ", as HTTP joins those of a header sent more than once.
|
// header's values joined by ", ", as HTTP joins those of a header sent
|
||||||
|
// more than once.
|
||||||
func value(target string, r *http.Request) string {
|
func value(target string, r *http.Request) string {
|
||||||
switch target {
|
switch target {
|
||||||
case "path":
|
case "path":
|
||||||
path, _, _ := strings.Cut(pathAndQuery(r), "?")
|
return Path(r)
|
||||||
|
|
||||||
return path
|
|
||||||
case "query":
|
case "query":
|
||||||
_, query, _ := strings.Cut(pathAndQuery(r), "?")
|
_, query, _ := strings.Cut(pathAndQuery(r), "?")
|
||||||
|
|
||||||
|
|||||||
@@ -679,8 +679,8 @@ func (f *bansFile) check(data []byte) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// check refuses a client without its address, which would count nobody's
|
// check refuses a client without its address, which would count nobody's
|
||||||
// requests, or with requests or bytes in a window but no start, which
|
// requests, or with requests, bytes or refusals in a window but no start,
|
||||||
// would drop them and give the client a fresh allowance.
|
// which would drop them and give the client a fresh allowance.
|
||||||
func (f *clientsFile) check([]byte) error {
|
func (f *clientsFile) check([]byte) error {
|
||||||
for i, client := range f.Clients {
|
for i, client := range f.Clients {
|
||||||
switch {
|
switch {
|
||||||
@@ -698,6 +698,8 @@ func (f *clientsFile) check([]byte) error {
|
|||||||
return missing(i, "hour_bytes.start")
|
return missing(i, "hour_bytes.start")
|
||||||
case countsWithoutStart(client.DayBytes):
|
case countsWithoutStart(client.DayBytes):
|
||||||
return missing(i, "day_bytes.start")
|
return missing(i, "day_bytes.start")
|
||||||
|
case countsWithoutStart(client.MinuteRefusals):
|
||||||
|
return missing(i, "minute_refusals.start")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -898,8 +900,8 @@ func missingFromCounter(counter anomaly.Counter) string {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// countsWithoutStart reports whether b holds requests, or bytes, but no
|
// countsWithoutStart reports whether b holds requests, bytes or refusals
|
||||||
// start, which places them in time.
|
// but no start, which places them in time.
|
||||||
func countsWithoutStart(b ratelimit.Buckets) bool {
|
func countsWithoutStart(b ratelimit.Buckets) bool {
|
||||||
return b.Start.IsZero() && (b.Current != 0 || b.Previous != 0)
|
return b.Start.IsZero() && (b.Current != 0 || b.Previous != 0)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -639,6 +639,15 @@ func TestEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestClientWithRefusalsInTheMinuteWithoutTheirStartStopsTheStart(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
wantRefused(t, clientsJSON,
|
||||||
|
`{"version": 1, "clients": [{"client": "203.0.113.9/32", `+
|
||||||
|
`"minute_refusals": {"current": 2}}]}`,
|
||||||
|
`: entry 1 has no "minute_refusals.start"`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestReputationJSONEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
|
func TestReputationJSONEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user