The spec names every fault that stops the start, runs step 15 for every request but the health check, and states each of the fourteen repeated requirements once. docs/alerts.md says which alerts have an empty client and netblock and what a summary's detail holds; docs/settings.md lists the settings that need a lookup and points at step 3 for the two static lists. The README and the spec now travel in one branch, so the README never names a spec section that does not exist yet. Model: fable-5-1
This commit is contained in:
@@ -150,8 +150,8 @@ request that counts against its client in its history. It is one of:
|
|||||||
- a broken limit;
|
- a broken limit;
|
||||||
- a refusal by a rule, by the Core Rule Set or for a missing or wrong token.
|
- a refusal by a rule, by the Core Rule Set or for a missing or wrong token.
|
||||||
|
|
||||||
`smallwebwaf` handles each request in this order. A request one step refuses
|
`smallwebwaf` handles each request in this order. Once a step refuses a request,
|
||||||
skips the steps after it, apart from step 13. Nothing of a refused request
|
the steps after it are skipped, apart from step 13. Nothing of a refused request
|
||||||
reaches the app. "What happens to a request" in [`SPEC.md`](SPEC.md) gives each
|
reaches the app. "What happens to a request" in [`SPEC.md`](SPEC.md) gives each
|
||||||
step in full.
|
step in full.
|
||||||
|
|
||||||
|
|||||||
@@ -100,10 +100,12 @@ to them.
|
|||||||
## What happens to a request
|
## What happens to a request
|
||||||
|
|
||||||
The steps run in this order. The first step that produces a final answer ends
|
The steps run in this order. The first step that produces a final answer ends
|
||||||
the checks; step 15 runs for every request. The size and time limits apply to
|
the checks; step 15 runs for every request but the health check. The size and
|
||||||
the whole exchange. Each step names the section that holds its rules.
|
time limits apply to the whole exchange. Each step names the section that holds
|
||||||
|
its rules.
|
||||||
|
|
||||||
1. `GET /_smallwebwaf/healthz` is answered at once, before any check.
|
1. `GET /_smallwebwaf/healthz` is answered at once, before any check, so a
|
||||||
|
health checker never needs an exemption and is never refused.
|
||||||
2. The client is identified.
|
2. The client is identified.
|
||||||
- If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, `X-Forwarded-For` is
|
- If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, `X-Forwarded-For` is
|
||||||
walked from the right, and the first address outside those netblocks is
|
walked from the right, and the first address outside those netblocks is
|
||||||
@@ -116,9 +118,9 @@ the whole exchange. Each step names the section that holds its rules.
|
|||||||
around its address, a /64 by default, since one abuser usually controls a
|
around its address, a /64 by default, since one abuser usually controls a
|
||||||
whole /64. A client is therefore one IPv4 address or one IPv6 group.
|
whole /64. A client is therefore one IPv4 address or one IPv6 group.
|
||||||
3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12 and the
|
3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12 and the
|
||||||
byte limits and the error burst at step 15. It is still counted for the
|
byte limits and the error burst at step 15. It still needs a token at
|
||||||
anomaly thresholds, and still needs a token at step 13. A client in
|
step 13. A client in `SWWAF_DENY_NETS` is refused; a client in both lists is
|
||||||
`SWWAF_DENY_NETS` is refused; a client in both lists is let through.
|
let through.
|
||||||
4. Bans. A client under a ban is refused (see "Bans").
|
4. Bans. A client under a ban is refused (see "Bans").
|
||||||
5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
|
5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
|
||||||
and country"). The request goes on at once. It waits for the first GeoJS
|
and country"). The request goes on at once. It waits for the first GeoJS
|
||||||
@@ -134,8 +136,6 @@ the whole exchange. Each step names the section that holds its rules.
|
|||||||
gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is
|
gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is
|
||||||
banned. A client listed by a DNS blocklist, or scored at least
|
banned. A client listed by a DNS blocklist, or scored at least
|
||||||
`SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`.
|
`SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`.
|
||||||
With no cached verdict, a query is queued in the background and the request
|
|
||||||
goes on, so a first request is never delayed by a reputation query.
|
|
||||||
8. The client's limit percentage (see "Biased thresholds").
|
8. The client's limit percentage (see "Biased thresholds").
|
||||||
9. Rate limiting (see "Request rate limits, per client"). A request over a limit
|
9. Rate limiting (see "Request rate limits, per client"). A request over a limit
|
||||||
times the percentage is refused and bans the client.
|
times the percentage is refused and bans the client.
|
||||||
@@ -184,7 +184,7 @@ the token check still apply.
|
|||||||
and the anomaly counters have fixed bounds, listed under "Fixed limits" in
|
and the anomaly counters have fixed bounds, listed under "Fixed limits" in
|
||||||
`docs/settings.md`. Past its bound, the GeoJS answer or anomaly counter used
|
`docs/settings.md`. Past its bound, the GeoJS answer or anomaly counter used
|
||||||
longest ago goes first, and the verdict or score fetched longest ago. Past
|
longest ago goes first, and the verdict or score fetched longest ago. Past
|
||||||
1,000 DNS queries under way, a zone is asked at the client's next request.
|
that bound, a zone is asked at the client's next request.
|
||||||
|
|
||||||
## Bans
|
## Bans
|
||||||
|
|
||||||
@@ -193,8 +193,7 @@ Until a ban ends, every request from its netblock is refused with
|
|||||||
`SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one
|
`SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one
|
||||||
address or one /64, so a permanent ban does not reach neighbours who did
|
address or one /64, so a permanent ban does not reach neighbours who did
|
||||||
nothing. Requests refused under a ban are not counted against limits. They are
|
nothing. Requests refused under a ban are not counted against limits. They are
|
||||||
counted in the client's history and in the ban's notes. A banned client is not
|
counted in the client's history and in the ban's notes.
|
||||||
looked up.
|
|
||||||
|
|
||||||
An offence is a request `smallwebwaf` holds against the client. It is one of:
|
An offence is a request `smallwebwaf` holds against the client. It is one of:
|
||||||
|
|
||||||
@@ -367,8 +366,8 @@ Conventions:
|
|||||||
owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and
|
owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and
|
||||||
byte limits only; the error burst, attack detection and bans still apply.
|
byte limits only; the error burst, attack detection and bans still apply.
|
||||||
- Many clients share one address (mobile carriers, offices, Tor), and so share
|
- Many clients share one address (mobile carriers, offices, Tor), and so share
|
||||||
its limits and its bans. The defaults sit well above one person's use, and the
|
its limits and its bans. `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS`
|
||||||
first two lists take known shared addresses out.
|
take known shared addresses out.
|
||||||
|
|
||||||
### Request rate limits, per client
|
### Request rate limits, per client
|
||||||
|
|
||||||
@@ -442,12 +441,8 @@ service that keeps its visitors' addresses on its own host.
|
|||||||
|
|
||||||
- Only one source is used. The start stops, naming both settings, when `file` is
|
- Only one source is used. The start stops, naming both settings, when `file` is
|
||||||
set without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other
|
set without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other
|
||||||
source. The start also stops when the source is `off` and one of these needs
|
source. The start also stops when the source is `off` and a setting that needs
|
||||||
lookups:
|
a lookup is set (`docs/settings.md` lists them).
|
||||||
- a country list;
|
|
||||||
- a biased threshold;
|
|
||||||
- `SWWAF_ADD_LOOKUP_HEADERS`;
|
|
||||||
- an anomaly threshold per AS number.
|
|
||||||
- GeoJS is told the address of every new visitor, whether or not a setting uses
|
- GeoJS is told the address of every new visitor, whether or not a setting uses
|
||||||
the answer, since the request log, the metrics and the ban notes carry it.
|
the answer, since the request log, the metrics and the ban notes carry it.
|
||||||
These addresses are never sent:
|
These addresses are never sent:
|
||||||
@@ -476,8 +471,8 @@ service that keeps its visitors' addresses on its own host.
|
|||||||
API requests", judged by GeoJS alone, and let it block a caller. While GeoJS
|
API requests", judged by GeoJS alone, and let it block a caller. While GeoJS
|
||||||
is slow, down or refusing `smallwebwaf`, new clients count as unknown. GeoJS
|
is slow, down or refusing `smallwebwaf`, new clients count as unknown. GeoJS
|
||||||
is then left alone for a second, and twice as long after each further failure,
|
is then left alone for a second, and twice as long after each further failure,
|
||||||
up to five minutes, and one `source_failure` alert is sent per cooldown. A
|
up to five minutes. Each failure raises a `source_failure` alert (see
|
||||||
service that cannot accept this uses the file.
|
"Alerting"). A service that cannot accept this uses the file.
|
||||||
- The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`,
|
- The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`,
|
||||||
`as_name` and `country_code` fields are read. The operator downloads it with a
|
`as_name` and `country_code` fields are read. The operator downloads it with a
|
||||||
free IPinfo account and mounts it read-only; `smallwebwaf` never fetches it
|
free IPinfo account and mounts it read-only; `smallwebwaf` never fetches it
|
||||||
@@ -486,9 +481,9 @@ service that keeps its visitors' addresses on its own host.
|
|||||||
on the site that uses it, such as "IP address data is powered by IPinfo".
|
on the site that uses it, such as "IP address data is powered by IPinfo".
|
||||||
- The file is read whole at start; one that is missing or unreadable stops the
|
- The file is read whole at start; one that is missing or unreadable stops the
|
||||||
start. It is read again once it has gone 2 seconds without a change after
|
start. It is read again once it has gone 2 seconds without a change after
|
||||||
being replaced. One that cannot be read is logged, sent as one `file_error`
|
being replaced. A replacement that cannot be read is logged, sent as one
|
||||||
alert and ignored. The database already loaded stays in use. A client whose
|
`file_error` alert and ignored. The database already loaded stays in use. A
|
||||||
address the file does not hold counts as unknown.
|
client whose address the file does not hold counts as unknown.
|
||||||
|
|
||||||
### Country lists
|
### Country lists
|
||||||
|
|
||||||
@@ -508,9 +503,8 @@ where it comes from.
|
|||||||
exclusive list already refuses every other country. A code on both lists stops
|
exclusive list already refuses every other country. A code on both lists stops
|
||||||
the start with a message naming it.
|
the start with a message naming it.
|
||||||
- A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the
|
- A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the
|
||||||
client's address has been looked up, before its body is read. It skips the
|
client's address has been looked up, before its body is read. It is logged
|
||||||
reputation checks, the limits, the rule files and the Core Rule Set. It is
|
with the action `country_denied`, and counted in the metrics.
|
||||||
logged with the action `country_denied`, and counted in the metrics.
|
|
||||||
|
|
||||||
### Biased thresholds
|
### Biased thresholds
|
||||||
|
|
||||||
@@ -519,10 +513,11 @@ countries. The lists are empty by default, for the same reason as the country
|
|||||||
lists. The request log and the metrics show which AS numbers and countries a
|
lists. The request log and the metrics show which AS numbers and countries a
|
||||||
service's abuse comes from.
|
service's abuse comes from.
|
||||||
|
|
||||||
- Clients in a listed AS number or country get that percentage of every request
|
- Each list gives AS numbers or countries a percentage, such as `AS14061:50`.
|
||||||
and byte limit (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A
|
Their clients get that percentage of every request and byte limit
|
||||||
lowered limit is rounded down. `0` is a zero allowance: the client's first
|
(`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A lowered limit is
|
||||||
request breaks a limit and bans it.
|
rounded down. `0` is a zero allowance: the client's first request breaks a
|
||||||
|
limit and bans it.
|
||||||
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` set the byte
|
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` set the byte
|
||||||
limits' percentage alone, when it should differ from the request limits'. For
|
limits' percentage alone, when it should differ from the request limits'. For
|
||||||
an AS number they list, they replace its percentage from
|
an AS number they list, they replace its percentage from
|
||||||
@@ -556,9 +551,8 @@ code in them for attacks. The default gives up refusing an attack carried in a
|
|||||||
body. The rule files, the limits and the bans still apply to the client that
|
body. The rule files, the limits and the bans still apply to the client that
|
||||||
sends it.
|
sends it.
|
||||||
|
|
||||||
- The Core Rule Set inspects only a request that every earlier check let
|
- The Core Rule Set skips a path under `SWWAF_WAF_EXEMPT_PATHS`. An engine error
|
||||||
through, and skips a path under `SWWAF_WAF_EXEMPT_PATHS`. An engine error on a
|
on a request forwards the request, and is logged and counted.
|
||||||
request forwards the request, and is logged and counted.
|
|
||||||
- `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule
|
- `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule
|
||||||
Set's own two tuning values, at its own defaults. Each level up runs more
|
Set's own two tuning values, at its own defaults. Each level up runs more
|
||||||
rules, which find more attacks and take more ordinary requests for attacks.
|
rules, which find more attacks and take more ordinary requests for attacks.
|
||||||
@@ -601,14 +595,18 @@ sends it.
|
|||||||
percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and
|
percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and
|
||||||
`/%77p-login.php`, and does not match `/wp-login.php/` or
|
`/%77p-login.php`, and does not match `/wp-login.php/` or
|
||||||
`/blog/wp-login.php`.
|
`/blog/wp-login.php`.
|
||||||
- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute for a rule
|
- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute of these
|
||||||
file match, a trap path, a Core Rule Set match, or a missing or wrong token;
|
kinds, and of no other:
|
||||||
no other refusal counts. More than this breaks a limit (see "Bans"). A client
|
- a rule file match;
|
||||||
trying one attack or token after another is refused many times a minute; a
|
- a trap path;
|
||||||
person rarely more than a few times. Answers the app gives are not counted,
|
- a Core Rule Set match;
|
||||||
since they do not tell a scanner from an ordinary client. In front of gitea,
|
- a missing or wrong token.
|
||||||
registry and git clients get `404` and `401` by design, hundreds of times a
|
|
||||||
minute.
|
More than this breaks a limit (see "Bans"). A client trying one attack or
|
||||||
|
token after another is refused many times a minute; a person rarely more
|
||||||
|
than a few times. Answers the app gives are not counted, since they do not
|
||||||
|
tell a scanner from an ordinary client. In front of gitea, registry and git
|
||||||
|
clients get `404` and `401` by design, hundreds of times a minute.
|
||||||
|
|
||||||
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
|
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
|
||||||
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
||||||
@@ -773,9 +771,8 @@ too close together.
|
|||||||
by default. Such verdicts are less certain than a blocklist, so a listed
|
by default. Such verdicts are less certain than a blocklist, so a listed
|
||||||
client gets a quarter of every request and byte limit rather than a refusal.
|
client gets a quarter of every request and byte limit rather than a refusal.
|
||||||
- Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and
|
- Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and
|
||||||
in `reputation.json`, with the fixed bounds under "Fixed limits" in
|
in `reputation.json`. A source that is down or over quota gives no verdict.
|
||||||
`docs/settings.md`. A source that is down or over quota gives no verdict. The
|
The service continues, with one `source_failure` alert per cooldown.
|
||||||
service continues, with one `source_failure` alert per cooldown.
|
|
||||||
|
|
||||||
### Alerting
|
### Alerting
|
||||||
|
|
||||||
@@ -787,9 +784,8 @@ too close together.
|
|||||||
the ban's alert, with `mode` set to `observe` in its `detail`.
|
the ban's alert, with `mode` set to `observe` in its `detail`.
|
||||||
- A repeat is an alert with the same event, and the same netblock, source, file
|
- A repeat is an alert with the same event, and the same netblock, source, file
|
||||||
or anomaly scope, as one sent less than `SWWAF_ALERT_COOLDOWN` before. An
|
or anomaly scope, as one sent less than `SWWAF_ALERT_COOLDOWN` before. An
|
||||||
anomaly scope is the thing counted: one client, netblock, AS number or named
|
anomaly scope is one of the things counted under "Anomaly thresholds". A
|
||||||
netblock, or the whole service. A repeat is held back and counted, and the
|
repeat is held back and counted, and the next such alert sent gives the count.
|
||||||
next such alert sent gives the count.
|
|
||||||
- Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and
|
- Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and
|
||||||
counted by event, so a wide attack cannot flood the channel. Once the hour
|
counted by event, so a wide attack cannot flood the channel. Once the hour
|
||||||
ends, one alert with the event `summary` gives those counts. It also counts
|
ends, one alert with the event `summary` gives those counts. It also counts
|
||||||
@@ -820,7 +816,7 @@ addresses at low rates each, which the per-client limits do not see. A low
|
|||||||
- Every request but the health check is counted, whatever was done with it,
|
- Every request but the health check is counted, whatever was done with it,
|
||||||
clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` included,
|
clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` included,
|
||||||
since an exempt client misbehaving is worth knowing about. The counters are
|
since an exempt client misbehaving is worth knowing about. The counters are
|
||||||
kept in memory and in `alerts.json`, with a fixed bound.
|
kept in memory and in `alerts.json`.
|
||||||
|
|
||||||
## Rule files
|
## Rule files
|
||||||
|
|
||||||
@@ -846,10 +842,10 @@ and the default file.
|
|||||||
of every request; a rule naming one stops the start like any bad line. Request
|
of every request; a rule naming one stops the start like any bad line. Request
|
||||||
bodies are not available to rule files; reading them is the Core Rule Set's
|
bodies are not available to rule files; reading them is the Core Rule Set's
|
||||||
job.
|
job.
|
||||||
- `log` only notes the match in the request log. `block` refuses the request
|
- `log` only notes the match in the request log. `block` refuses the request;
|
||||||
without a ban; `ban` makes the request a clear sign of attack (see "Bans"),
|
`ban` makes the request a clear sign of attack (see "Bans"), and is for
|
||||||
and is for requests no real visitor sends. A visitor banned by mistake is let
|
requests no real visitor sends. A visitor banned by mistake is let back in by
|
||||||
back in by lifting the ban.
|
lifting the ban.
|
||||||
- At start, these stop the process with a message naming the file and line:
|
- At start, these stop the process with a message naming the file and line:
|
||||||
- a line that does not parse;
|
- a line that does not parse;
|
||||||
- a regex that does not compile;
|
- a regex that does not compile;
|
||||||
@@ -940,9 +936,8 @@ edit them at any time, and the running `smallwebwaf` takes the edit in.
|
|||||||
|
|
||||||
## Request log
|
## Request log
|
||||||
|
|
||||||
Every request gets one JSON object on one line on stdout, refused ones included,
|
Every request `smallwebwaf` sees gets one JSON object on one line on stdout,
|
||||||
apart from those Go's HTTP server ends before `smallwebwaf` sees them (see "Size
|
refused ones included. `docs/request-log.md` lists the fields. Bodies are never
|
||||||
and time limits"). `docs/request-log.md` lists the fields. Bodies are never
|
|
||||||
logged, and no header but those listed there. Query strings are logged as
|
logged, and no header but those listed there. Query strings are logged as
|
||||||
received; an app that carries secrets in query strings needs that fixed in the
|
received; an app that carries secrets in query strings needs that fixed in the
|
||||||
app. The process's own messages (start, the settings, stop, errors) share the
|
app. The process's own messages (start, the settings, stop, errors) share the
|
||||||
@@ -971,8 +966,7 @@ while `docker logs` keeps working.
|
|||||||
## Metrics endpoint
|
## Metrics endpoint
|
||||||
|
|
||||||
`GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request
|
`GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request
|
||||||
carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. Every
|
carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. No metric
|
||||||
metric carries the label `instance`, set to `SWWAF_INSTANCE_NAME`. No metric
|
|
||||||
carries a client IP address as a label. Per-address questions are answered by
|
carries a client IP address as a label. Per-address questions are answered by
|
||||||
the request log and `GET /_smallwebwaf/clients/<ip>`.
|
the request log and `GET /_smallwebwaf/clients/<ip>`.
|
||||||
|
|
||||||
@@ -980,15 +974,14 @@ the request log and `GET /_smallwebwaf/clients/<ip>`.
|
|||||||
|
|
||||||
A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself:
|
A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself:
|
||||||
it answers it and never passes it to the app. The prefix carries the tool's
|
it answers it and never passes it to the app. The prefix carries the tool's
|
||||||
name, so it takes no path an app uses. Admins and scrapers reach these endpoints
|
name, so it takes no path an app uses. Admins and the metrics collector reach
|
||||||
through traefik, like any other request. `docs/admin-endpoints.md` lists the
|
these endpoints through traefik, like any other request.
|
||||||
endpoints, what each takes and what each answers.
|
`docs/admin-endpoints.md` lists the endpoints, what each takes and what each
|
||||||
|
answers.
|
||||||
|
|
||||||
- `GET /_smallwebwaf/healthz` answers `200` with `ok` to anyone, without a
|
- `GET /_smallwebwaf/healthz` answers `200` with `ok` to anyone, without a
|
||||||
token, while `smallwebwaf` is running. It does not ask the app; the
|
token, while `smallwebwaf` is running. It does not ask the app; the
|
||||||
container's health check does. It is answered before any check, so a health
|
container's health check does.
|
||||||
checker never needs an exemption and is never refused, for example by
|
|
||||||
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`.
|
|
||||||
- Every other endpoint needs its token, sent as `Authorization: Bearer <token>`
|
- Every other endpoint needs its token, sent as `Authorization: Bearer <token>`
|
||||||
(see "Tokens" under "Settings"). While a token is unset, the endpoints that
|
(see "Tokens" under "Settings"). While a token is unset, the endpoints that
|
||||||
need it answer `404`, as does any other path under the prefix.
|
need it answer `404`, as does any other path under the prefix.
|
||||||
@@ -996,10 +989,9 @@ endpoints, what each takes and what each answers.
|
|||||||
other request goes through. They are answered at the point where another
|
other request goes through. They are answered at the point where another
|
||||||
request would be forwarded to the app: a banned or refused client stays
|
request would be forwarded to the app: a banned or refused client stays
|
||||||
refused, and each request counts toward the client's limits. A missing or
|
refused, and each request counts toward the client's limits. A missing or
|
||||||
wrong token is answered `401`, in `SWWAF_MODE=observe` too, and counts toward
|
wrong token is answered `401` (see "Attack detection" for what guessing
|
||||||
the error burst. A client guessing tokens is therefore banned once it passes
|
costs).
|
||||||
`SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute.
|
- A ban made through the endpoints starts at once.
|
||||||
- A ban made through the endpoints starts at once, with the cause `admin`.
|
|
||||||
|
|
||||||
## Deployment
|
## Deployment
|
||||||
|
|
||||||
@@ -1041,26 +1033,26 @@ what the image holds and how each part is pinned.
|
|||||||
2015, at a fixed commit hash, with a `go.mod` the build writes. `runsvinit`
|
2015, at a fixed commit hash, with a `go.mod` the build writes. `runsvinit`
|
||||||
starts runit's `runsvdir`, which runs the `run` script of each directory under
|
starts runit's `runsvdir`, which runs the `run` script of each directory under
|
||||||
`/etc/service`. Ubuntu's `sv` looks for services there too.
|
`/etc/service`. Ubuntu's `sv` looks for services there too.
|
||||||
- The image's `run` script for `smallwebwaf` waits one second and makes
|
- The image's `run` script for `smallwebwaf` makes `SWWAF_STATE_DIR` and every
|
||||||
`SWWAF_STATE_DIR` and every file in it belong to the `smallwebwaf` user (uid
|
file in it belong to the `smallwebwaf` user (uid and gid 65532). Then it
|
||||||
and gid 65532). Then it starts `smallwebwaf` as that user with runit's
|
starts `smallwebwaf` as that user with runit's `chpst`.
|
||||||
`chpst`.
|
|
||||||
|
|
||||||
Every `run` script, the app's included, is a bash script. It starts with
|
Every `run` script, the app's included, is a bash script. It starts with
|
||||||
`#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main`
|
`#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main`
|
||||||
function, called on its last line, as the owner's code style guide asks. Ubuntu
|
function, called on its last line, as the owner's code style guide asks. Ubuntu
|
||||||
ships bash.
|
ships bash. Each waits one second (`sleep 1`) before starting its program, so a
|
||||||
|
process that stops at start is started again every second, not in a tight loop.
|
||||||
|
|
||||||
Beyond its `FROM` line, the app's Dockerfile adds:
|
Beyond its `FROM` line, the app's Dockerfile adds:
|
||||||
|
|
||||||
- its binary;
|
- its binary;
|
||||||
- any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.<name>`;
|
- any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.<name>`;
|
||||||
- its runit service: a directory under `/etc/service` named after the app, never
|
- its runit service: a directory under `/etc/service` named after the app, never
|
||||||
`smallwebwaf`. Its `run` script waits one second (`sleep 1`) and then starts
|
`smallwebwaf`. Its `run` script starts the app with `chpst`, listening where
|
||||||
the app with `chpst`, listening where `SWWAF_UPSTREAM_URL` points (see
|
`SWWAF_UPSTREAM_URL` points (see "Ports"). The app runs as a user of its own,
|
||||||
"Ports"). The app runs as a user of its own, which the Dockerfile creates with
|
which the Dockerfile creates with `useradd`. That user is neither root nor
|
||||||
`useradd`. That user is neither root nor `smallwebwaf`, so that the app cannot
|
`smallwebwaf`, so that the app cannot touch the state files or the process of
|
||||||
touch the state files or the process of `smallwebwaf`.
|
`smallwebwaf`.
|
||||||
|
|
||||||
It sets no `ENTRYPOINT` or `USER` of its own. The container must start
|
It sets no `ENTRYPOINT` or `USER` of its own. The container must start
|
||||||
`runsvinit`, as root, so that it can start each service as its own user.
|
`runsvinit`, as root, so that it can start each service as its own user.
|
||||||
@@ -1072,16 +1064,17 @@ It sets no `ENTRYPOINT` or `USER` of its own. The container must start
|
|||||||
script starts as root and hands over to its program's user with `chpst`. Both
|
script starts as root and hands over to its program's user with `chpst`. Both
|
||||||
services start with the container's environment variables and write to the
|
services start with the container's environment variables and write to the
|
||||||
container's output, which `docker logs` shows.
|
container's output, which `docker logs` shows.
|
||||||
- When `smallwebwaf` or the app exits, runit runs its `run` script again, which
|
- When `smallwebwaf` or the app exits, runit runs its `run` script again. The
|
||||||
waits a second before starting it. The other process keeps running, and the
|
other process keeps running, and the container does not stop. While
|
||||||
container does not stop. While `smallwebwaf` is down, traefik answers `502`.
|
`smallwebwaf` is down, traefik answers `502`. While the app is down,
|
||||||
While the app is down, `smallwebwaf` answers `502`, which is not a client
|
`smallwebwaf` answers `502`, which is not a client offence.
|
||||||
offence.
|
- Only faults found at start stop `smallwebwaf`:
|
||||||
- Only four things stop `smallwebwaf`, all at start:
|
|
||||||
- invalid configuration;
|
- invalid configuration;
|
||||||
- a lookup database that is missing or unreadable;
|
- a lookup database that is missing or unreadable;
|
||||||
- a rule file or state file that does not parse;
|
- a rule file or state file that does not parse;
|
||||||
- an unwritable `SWWAF_STATE_DIR`.
|
- an unwritable `SWWAF_STATE_DIR`;
|
||||||
|
- a `SWWAF_RULES_DIR` that does not exist or cannot be read;
|
||||||
|
- a `SWWAF_LISTEN_ADDR` it cannot listen on.
|
||||||
- A `smallwebwaf` that stops at start is therefore started again every second,
|
- A `smallwebwaf` that stops at start is therefore started again every second,
|
||||||
logging its message each time, until the cause is fixed; the health check
|
logging its message each time, until the cause is fixed; the health check
|
||||||
fails meanwhile. Once running, a failing GeoJS, reputation source or alert
|
fails meanwhile. Once running, a failing GeoJS, reputation source or alert
|
||||||
|
|||||||
+6
-3
@@ -79,10 +79,11 @@ on one line:
|
|||||||
- `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
|
- `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
|
||||||
one of the events above.
|
one of the events above.
|
||||||
- `client`: the address whose request raised the alert. Empty for
|
- `client`: the address whose request raised the alert. Empty for
|
||||||
`source_failure` and `file_error`.
|
`source_failure`, `file_error` and `summary`.
|
||||||
- `netblock`: empty for `source_failure` and `file_error`; otherwise one of:
|
- `netblock`: empty for `source_failure`, `file_error` and `summary`; otherwise
|
||||||
|
one of:
|
||||||
- the ban's netblock;
|
- the ban's netblock;
|
||||||
- the netblock an anomaly counted;
|
- the netblock an anomaly counted, empty in the `asn` and `total` scopes;
|
||||||
- for the other events, the client as rate limiting counts it: an IPv4
|
- for the other events, the client as rate limiting counts it: an IPv4
|
||||||
address as a /32, or an IPv6 group.
|
address as a /32, or an IPv6 group.
|
||||||
- `asn`, `as_name` and `country`: the client's, as known when the alert was
|
- `asn`, `as_name` and `country`: the client's, as known when the alert was
|
||||||
@@ -97,6 +98,8 @@ on one line:
|
|||||||
- for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`;
|
- for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`;
|
||||||
- for a source failure, the `source` and `error`;
|
- for a source failure, the `source` and `error`;
|
||||||
- for a file error, the `file` and `error`;
|
- for a file error, the `file` and `error`;
|
||||||
|
- for a summary, the `hour` it covers, the `count` of alerts held back, and
|
||||||
|
that count by event in `events`;
|
||||||
- `mode`, when the request was not refused: `detect` or `observe`.
|
- `mode`, when the request was not refused: `detect` or `observe`.
|
||||||
- `suppressed_repeats`: how many repeats the cooldown held back before this
|
- `suppressed_repeats`: how many repeats the cooldown held back before this
|
||||||
alert.
|
alert.
|
||||||
|
|||||||
+3
-3
@@ -1,8 +1,8 @@
|
|||||||
# The metrics
|
# The metrics
|
||||||
|
|
||||||
The metrics `GET /_smallwebwaf/metrics` answers with, as "Metrics endpoint" in
|
What `GET /_smallwebwaf/metrics` answers with. "Metrics endpoint" in
|
||||||
[`SPEC.md`](../SPEC.md) describes it. Prometheus keeps the `instance` label as
|
[`SPEC.md`](../SPEC.md) describes the endpoint. Prometheus keeps the `instance`
|
||||||
`exported_instance` unless the scrape sets `honor_labels: true`.
|
label as `exported_instance` unless the scrape sets `honor_labels: true`.
|
||||||
|
|
||||||
## Traffic
|
## Traffic
|
||||||
|
|
||||||
|
|||||||
+11
-5
@@ -91,11 +91,11 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
|||||||
## Static lists
|
## Static lists
|
||||||
|
|
||||||
- `SWWAF_ALLOW_NETS` (default empty): netblocks you trust, such as your
|
- `SWWAF_ALLOW_NETS` (default empty): netblocks you trust, such as your
|
||||||
monitoring. Their clients are refused only for a size or time limit or a
|
monitoring (see step 3 in [`SPEC.md`](../SPEC.md)).
|
||||||
missing or wrong token.
|
|
||||||
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
||||||
exempt from rate limiting and the byte limits.
|
exempt from rate limiting and the byte limits.
|
||||||
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
|
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are refused (see
|
||||||
|
step 3 in [`SPEC.md`](../SPEC.md)).
|
||||||
|
|
||||||
## Request rate limits, per client
|
## Request rate limits, per client
|
||||||
|
|
||||||
@@ -136,7 +136,13 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
|||||||
|
|
||||||
## Lookup of AS number and country
|
## Lookup of AS number and country
|
||||||
|
|
||||||
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`.
|
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off` stops
|
||||||
|
the start while a setting that needs a lookup is set:
|
||||||
|
- the two country lists;
|
||||||
|
- `SWWAF_ADD_LOOKUP_HEADERS`;
|
||||||
|
- the four biased threshold lists and `SWWAF_ASN_LIMIT_PERCENT_URL`;
|
||||||
|
- `SWWAF_UNKNOWN_LIMIT_PERCENT` below `100`;
|
||||||
|
- an anomaly threshold per AS number.
|
||||||
- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
|
- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
|
||||||
`.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`.
|
`.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`.
|
||||||
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
|
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
|
||||||
@@ -286,7 +292,7 @@ as `REQUESTS_PER_MINUTE`:
|
|||||||
client;
|
client;
|
||||||
- `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
- `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
||||||
client.
|
client.
|
||||||
- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs a lookup source.
|
- `SWWAF_ANOMALY_ASN_...`: per AS number.
|
||||||
- `SWWAF_ANOMALY_TOTAL_...`: the whole service.
|
- `SWWAF_ANOMALY_TOTAL_...`: the whole service.
|
||||||
- `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole.
|
- `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole.
|
||||||
`SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
|
`SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
|
||||||
|
|||||||
Reference in New Issue
Block a user