README reworked after review: every finding fixed, every sentence reread
check / check (push) Waiting to run
check / check (push) Waiting to run
The eighteen findings of the review on the PR are fixed: statements the code does not bear out, the stacked "a client a blocklist lists" form, late verbs, sentences over thirty words, comma chains where bullets belong, sentences a newcomer reads twice, and one project-plan remark. Every other sentence was reread against the same rules, and the ones that still put the object before the subject were rewritten. Model: fable-5-1
This commit is contained in:
@@ -8,8 +8,8 @@ scanners and other abusers before they reach the app.
|
||||
A web application firewall reads each request on its way to a web app and
|
||||
refuses the ones that look like abuse. `smallwebwaf` is also a reverse proxy: it
|
||||
accepts requests on the app's behalf, passes them to the app, and passes the
|
||||
app's answers back. A request it does not refuse reaches the app as it came. The
|
||||
only headers it changes are the ones that tell the app who the client is.
|
||||
app's answers back. A request it does not refuse reaches the app as it came,
|
||||
apart from a few headers (see "How it sits in front of the app" below).
|
||||
|
||||
It needs no setting. Every setting has a default chosen for a service on the
|
||||
open internet. The design is in [`SPEC.md`](SPEC.md). The survey of existing
|
||||
@@ -23,21 +23,23 @@ tools that led to it is in [`EVALUATION.md`](EVALUATION.md).
|
||||
limit is refused and banned. The first ban lasts an hour. Each repeat within a
|
||||
day makes the next ban three times as long. A ban that would last more than
|
||||
seven days is permanent.
|
||||
- Scanners that probe for secrets and known weak spots: `/.env`, `/.git/config`,
|
||||
or a WordPress login on a site that runs no WordPress. The default rule file
|
||||
bans such a client for seven days at its first probe. Its next request during
|
||||
the ban makes the ban permanent. You can add rules of your own, and name trap
|
||||
paths that do the same.
|
||||
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
||||
default rule file bans such a client for seven days at its first probe. Its
|
||||
next request during the ban makes the ban permanent. Probes for things your
|
||||
app does not serve, such as a WordPress login, need a rule of your own or an
|
||||
entry in `SWWAF_TRAP_PATHS`, which does the same.
|
||||
- Attacks carried in a request's URL or headers, such as SQL injection and path
|
||||
traversal. The OWASP Core Rule Set is a published set of rules that recognise
|
||||
common web attacks. Coraza, a Go engine that runs those rules, inspects every
|
||||
request. A request the rules take for an attack is refused.
|
||||
common web attacks. Coraza, a Go engine that runs those rules, inspects each
|
||||
request the earlier checks let through. A request the rules take for an attack
|
||||
is refused.
|
||||
- Abuse from networks and countries you do not want. You can:
|
||||
- refuse whole netblocks and countries;
|
||||
- give the clients of chosen networks and countries a fraction of every
|
||||
limit, so that they are banned sooner;
|
||||
- refuse or limit clients that a blocklist, a DNS blocklist, AbuseIPDB or
|
||||
your own CrowdSec engine has already seen misbehave.
|
||||
- refuse or limit clients on a blocklist or a DNS blocklist, or with a high
|
||||
AbuseIPDB score;
|
||||
- ban clients named by your own CrowdSec engine.
|
||||
- Slow and oversized requests. Every request and every answer has a time limit
|
||||
and a size limit.
|
||||
|
||||
@@ -125,6 +127,11 @@ docker run --detach --name app \
|
||||
<registry>/app@sha256:<digest>
|
||||
```
|
||||
|
||||
With no setting, the address of every new visitor is sent to GeoJS, a web
|
||||
service, to find its country and network. `SWWAF_LOOKUP_SOURCE=off` sends
|
||||
nothing. `SWWAF_LOOKUP_SOURCE=file` looks addresses up in a database file on
|
||||
your own host (see "GeoJS and the lookup database" below).
|
||||
|
||||
`deploy/example-app` is such an app. `make example-app` builds it and checks it
|
||||
(see "Checking that it works" below).
|
||||
|
||||
@@ -165,7 +172,8 @@ after Ubuntu's own commands.
|
||||
must trust `127.0.0.1` and `::1` for forwarded headers, besides the private
|
||||
address ranges. An app left at the usual default, the private ranges alone,
|
||||
sees every visitor as `127.0.0.1`. Each request also carries its id in
|
||||
`X-Request-ID`, the same id as in its log line.
|
||||
`X-Request-ID`, the same id as in its log line. Any `X-Client-ASN` or
|
||||
`X-Client-Country` header a client sent is removed.
|
||||
- Both processes start with the container's environment variables. That is why
|
||||
every setting of `smallwebwaf` starts with `SWWAF_`. A token given as an
|
||||
environment variable is readable by the app. A token given as a file that only
|
||||
@@ -179,9 +187,9 @@ after Ubuntu's own commands.
|
||||
accepts connections at `SWWAF_UPSTREAM_URL`. It runs every second from the
|
||||
container's start until it first passes, for up to a minute, and every 30
|
||||
seconds after that. traefik sends a container no requests until it is healthy.
|
||||
- `SWWAF_LISTEN_ADDR` can move `smallwebwaf` to another port. The app then
|
||||
leaves that port free instead, the health check follows it, and traefik's
|
||||
label must name it. Keep the address part empty: `:9000`, never
|
||||
- `SWWAF_LISTEN_ADDR` can move `smallwebwaf` to another port. The app must not
|
||||
listen on that port. The health check follows the setting, and traefik's label
|
||||
must name the new port. Keep the address part empty: `:9000`, never
|
||||
`127.0.0.1:9000`. traefik reaches `smallwebwaf` on the container's address,
|
||||
and the health check on `127.0.0.1`.
|
||||
- When `smallwebwaf` or the app exits, runit starts it again a second later. The
|
||||
@@ -193,18 +201,24 @@ after Ubuntu's own commands.
|
||||
`smallwebwaf` stops taking requests, gives those in progress five seconds to
|
||||
finish, and writes its state files. Those requests can finish only while the
|
||||
app still answers. An app that exits at once on `SIGTERM` ends them: the
|
||||
client gets `502`, or a cut-off answer. An app that should let them finish
|
||||
keeps answering for the whole five seconds.
|
||||
client gets `502`, or a cut-off answer. To let them finish, the app must keep
|
||||
answering for five seconds after `SIGTERM`.
|
||||
|
||||
## What happens to a request
|
||||
|
||||
`smallwebwaf` handles each request in this order. The first step that refuses it
|
||||
ends the handling. Nothing of a refused request reaches the app.
|
||||
|
||||
Two terms recur below. A clear sign of attack is a request that matches a rule
|
||||
whose action is `ban`, or asks for a trap path (steps 9 and 10). An offence is a
|
||||
request that counts against its client in the client's history: a clear sign of
|
||||
attack, a broken limit, or a refusal by a rule, by the Core Rule Set or for a
|
||||
missing or wrong token.
|
||||
|
||||
1. `GET /_smallwebwaf/healthz` is answered `200` with `ok` at once, before any
|
||||
check and without asking the app.
|
||||
2. The client is worked out. A netblock is a range of addresses, written as an
|
||||
address and a prefix length: `203.0.113.0/24`. `SWWAF_TRUSTED_PROXIES` names
|
||||
address and a prefix length: `203.0.113.0/24`. `SWWAF_TRUSTED_PROXIES` lists
|
||||
the netblocks traefik comes from, by default the private address ranges.
|
||||
- A TCP peer outside those netblocks is the client. Its forwarded headers
|
||||
are replaced.
|
||||
@@ -212,9 +226,9 @@ ends the handling. Nothing of a refused request reaches the app.
|
||||
first address outside those netblocks is the client.
|
||||
- If every address in the header is inside them, the leftmost is the client.
|
||||
With no header, the peer is.
|
||||
- An IPv6 client is one whole group: the netblock around its address that
|
||||
`SWWAF_IPV6_GROUP_PREFIX` sets, a /64 by default. One abuser usually holds
|
||||
a whole /64.
|
||||
- An IPv6 client is the whole netblock around its address, a /64 by default;
|
||||
`SWWAF_IPV6_GROUP_PREFIX` sets its length. One abuser usually holds a
|
||||
whole /64.
|
||||
3. A client in `SWWAF_ALLOW_NETS` skips every check below. A client in
|
||||
`SWWAF_DENY_NETS` is refused.
|
||||
4. A client under a ban is refused. If the ban was made for a clear sign of
|
||||
@@ -226,18 +240,19 @@ ends the handling. Nothing of a refused request reaches the app.
|
||||
6. The country lists are checked. A client from a country in
|
||||
`SWWAF_DENIED_COUNTRIES` is refused. When
|
||||
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any other country
|
||||
is refused, and so is a client without a country.
|
||||
is refused, and so is a client without a country. Such a refusal is not an
|
||||
offence.
|
||||
7. The reputation sources are checked, in this order:
|
||||
- the blocklists `SWWAF_BLOCKLIST_URLS` names;
|
||||
- the decision list of the CrowdSec engine `SWWAF_CROWDSEC_LAPI_URL` names;
|
||||
- the DNS blocklists `SWWAF_DNSBL_ZONES` names;
|
||||
- the blocklists in `SWWAF_BLOCKLIST_URLS`;
|
||||
- the decision list of the CrowdSec engine at `SWWAF_CROWDSEC_LAPI_URL`;
|
||||
- the DNS blocklists in `SWWAF_DNSBL_ZONES`;
|
||||
- AbuseIPDB, while `SWWAF_ABUSEIPDB_KEY` is set.
|
||||
|
||||
A source that lists the client refuses it, lowers its limits, or only notes
|
||||
it, as `SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say. A
|
||||
CrowdSec decision bans the client until the decision ends. No request waits
|
||||
for a query. DNS blocklists and AbuseIPDB are asked in the background, and
|
||||
their answer applies to the client's later requests.
|
||||
A source the client is on refuses it, lowers its limits, or only notes it,
|
||||
as `SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say. A CrowdSec
|
||||
decision bans the client until the decision ends. No request waits for a
|
||||
query. DNS blocklists and AbuseIPDB are asked in the background, and their
|
||||
answer applies to the client's later requests.
|
||||
|
||||
8. Rate limiting counts the request, unless the client is in
|
||||
`SWWAF_RATE_LIMIT_EXEMPT_NETS` or the path is under
|
||||
@@ -268,21 +283,24 @@ ends the handling. Nothing of a refused request reaches the app.
|
||||
- The request goes into the client's history and the anomaly counters, any
|
||||
alert due is raised, and the log line is written.
|
||||
|
||||
In `observe` mode (`SWWAF_MODE=observe`), steps 3 to 11 refuse nothing and ban
|
||||
no one. A request they would have refused is passed to the app, and its log line
|
||||
says what `enforce` mode would have done. The size and time limits still apply,
|
||||
and so does the token check. Observe mode is for trying a configuration before
|
||||
enforcing it.
|
||||
In `observe` mode (`SWWAF_MODE=observe`), `smallwebwaf` makes no ban of its own.
|
||||
It refuses a request only for a size or time limit, or for a missing or wrong
|
||||
token. A request it would otherwise have refused is passed to the app, and its
|
||||
log line says what `enforce` mode would have done. Observe mode is for trying a
|
||||
configuration before enforcing it.
|
||||
|
||||
## What a refused visitor sees
|
||||
|
||||
- `SWWAF_BAN_RESPONSE` answers most refusals: a banned client, a broken rate
|
||||
limit, a `ban` rule, a trap path, `SWWAF_DENY_NETS`, the country lists, and a
|
||||
reputation source whose action is `deny`. By default it is `403` with the
|
||||
- `SWWAF_BAN_RESPONSE` answers most refusals. By default it is `403` with the
|
||||
plain text `Forbidden`. It can be `429` instead, or `close`, which closes the
|
||||
connection without an answer. Behind traefik, `close` still answers the
|
||||
client: traefik sends `502`, as it does whenever its backend drops a
|
||||
connection.
|
||||
connection. It answers these refusals:
|
||||
- a banned client;
|
||||
- a broken rate limit;
|
||||
- a `ban` rule or a trap path;
|
||||
- `SWWAF_DENY_NETS` and the country lists;
|
||||
- a reputation source whose action is `deny`.
|
||||
- A `block` rule and the Core Rule Set always answer `403`.
|
||||
- The size and time limits answer with their own statuses:
|
||||
- `413` for a request body over `SWWAF_REQUEST_MAX_BYTES`;
|
||||
@@ -305,13 +323,20 @@ enforcing it.
|
||||
|
||||
- Every request gets one JSON line on stdout, which `docker logs` shows. Its
|
||||
`action` field says what was done, such as `forward`, `banned`, `rate_limited`
|
||||
or `waf_blocked`. Other fields say why: the rules that matched, the Core Rule
|
||||
Set's score, the limit broken, the reputation sources that listed the client,
|
||||
and when its ban ends. See "Request log" below.
|
||||
or `waf_blocked`. Other fields say why (see "Request log" below):
|
||||
- the rules that matched;
|
||||
- the Core Rule Set's score;
|
||||
- the limit broken;
|
||||
- the reputation sources the client is on;
|
||||
- when the client's ban ends.
|
||||
- Every ban is in `bans.json` in the state directory, with notes on why it was
|
||||
made. See "Bans" below.
|
||||
- Every ban, Core Rule Set match, reputation hit, anomaly and failure of a
|
||||
source or a file raises an alert. See "Alerts" below.
|
||||
- These raise an alert, sent to the destinations you name (see "Alerts" below):
|
||||
- a ban `smallwebwaf` makes;
|
||||
- a Core Rule Set match;
|
||||
- a reputation hit;
|
||||
- a count over an anomaly threshold;
|
||||
- a failure of a source or a file.
|
||||
- With `SWWAF_ADMIN_TOKEN` set, `GET /_smallwebwaf/clients/<ip>` answers with
|
||||
everything `smallwebwaf` knows of a client, and `GET /_smallwebwaf/bans` with
|
||||
every ban. See "Admin endpoints" below.
|
||||
@@ -386,9 +411,9 @@ The forms of values:
|
||||
`https` URL with a host and an optional port, and nothing more: no path, user
|
||||
or query.
|
||||
- `SWWAF_INSTANCE_NAME` (default: the container's host name, the first 12
|
||||
characters of the container's id): the name every log line, alert and metric
|
||||
gives as `instance`. Set it to a name that survives deploys and tells
|
||||
instances apart, such as `fsn1app1/gitea`. It must be valid UTF-8. While
|
||||
characters of the container's id): the `instance` name in every log line,
|
||||
alert and metric. Set it to a name that survives deploys and tells instances
|
||||
apart, such as `fsn1app1/gitea`. It must be valid UTF-8. While
|
||||
`SWWAF_ALERT_NTFY_URL` is set, it may hold no control character. While
|
||||
`SWWAF_LOG_REMOTE_URL` is set and `SWWAF_LOG_REMOTE_APP_NAME` is not, it must
|
||||
be 1 to 48 printable ASCII characters without a space.
|
||||
@@ -408,8 +433,8 @@ The forms of values:
|
||||
### Time and size limits
|
||||
|
||||
Each request has limits on both sides: between the client and `smallwebwaf`, and
|
||||
between `smallwebwaf` and the app. Bodies stream through in both directions and
|
||||
are never held whole in memory.
|
||||
between `smallwebwaf` and the app. Bodies stream through in both directions,
|
||||
except the part the Core Rule Set reads while `SWWAF_WAF_BODY_LIMIT` is set.
|
||||
|
||||
- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to
|
||||
send its request line and headers, and then, from the end of the headers, its
|
||||
@@ -446,9 +471,9 @@ fit.
|
||||
such as your monitoring or your own network. The size and time limits still
|
||||
apply to them. They are not looked up. They still need a token for the admin
|
||||
endpoints.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients rate
|
||||
limiting and the byte limits neither count nor refuse, such as a machine that
|
||||
talks to the app all day. Everything else still applies to them.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
||||
exempt from rate limiting and the byte limits, such as a machine that talks to
|
||||
the app all day. Everything else still applies to them.
|
||||
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
|
||||
A client in `SWWAF_ALLOW_NETS` too is let through.
|
||||
|
||||
@@ -459,10 +484,10 @@ fit.
|
||||
requests a client may make in each window. A count or `off`. The defaults are
|
||||
several times what one busy person produces. A browser loading a heavy page
|
||||
makes a few hundred requests, and several people often share one address.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes rate limiting
|
||||
does not count, such as `/assets/`. Their bytes are not counted either. The
|
||||
path is matched as sent, before any query and not percent-decoded, character
|
||||
for character. `/assets/` matches `/assets/app.js`. It does not match
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
|
||||
rate limiting, such as `/assets/`. Their bytes are not counted either. The
|
||||
path is matched exactly as sent, without the query, and is not
|
||||
percent-decoded. `/assets/` matches `/assets/app.js`. It does not match
|
||||
`/Assets/app.js`, `/%61ssets/app.js` or `/static/assets/app.js`. There are no
|
||||
wildcards. A path holding `..`, a backslash or an encoded slash (`%2F`) is
|
||||
never exempt, since the app may read it as a path outside the prefix.
|
||||
@@ -481,9 +506,9 @@ fit.
|
||||
|
||||
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `geojs`
|
||||
asks the GeoJS web service, which is then told the address of every new
|
||||
visitor. `file` looks clients up in the IPinfo Lite database file
|
||||
`SWWAF_LOOKUP_DB_PATH` names, and sends nothing off the host. `off` looks up
|
||||
no client. With `off`, a setting that needs lookups stops the start:
|
||||
visitor. `file` looks clients up in the IPinfo Lite database file at
|
||||
`SWWAF_LOOKUP_DB_PATH`, and sends nothing off the host. `off` looks up no
|
||||
client. With `off`, a setting that needs lookups stops the start:
|
||||
- a country list;
|
||||
- `SWWAF_ADD_LOOKUP_HEADERS`;
|
||||
- an entry in `SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`,
|
||||
@@ -536,14 +561,16 @@ whole AS number or country, which one abuser could use up.
|
||||
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty):
|
||||
the same for the byte limits alone. For the AS numbers and countries they
|
||||
list, they take the place of the two settings above.
|
||||
- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage every client
|
||||
without a country gets: one the lookup cannot place, one on a private address,
|
||||
and one whose answer from GeoJS has not come in time.
|
||||
- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage of every limit
|
||||
for a client without a country:
|
||||
- one the lookup cannot place;
|
||||
- one on a private address;
|
||||
- one whose answer from GeoJS has not come in time.
|
||||
- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
|
||||
text file of such entries, one to a line, such as `AS14061:50`. One list can
|
||||
then serve several instances. It is fetched and kept as a blocklist is. Where
|
||||
it and `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. A
|
||||
URL that `SWWAF_BLOCKLIST_URLS` names too stops the start.
|
||||
URL that is also in `SWWAF_BLOCKLIST_URLS` stops the start.
|
||||
|
||||
A client to which several percentages apply gets the lowest. The log line of
|
||||
each counted request, and the notes of each ban, say which setting lowered the
|
||||
@@ -557,21 +584,21 @@ do. The defaults judge it by what it does to your service.
|
||||
- `SWWAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as
|
||||
`https://www.spamhaus.org/drop/drop.txt`. Each is an `http` or `https` URL
|
||||
without a user or a fragment. A URL listed twice is refused.
|
||||
- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long after a list was last
|
||||
fetched, or a fetch failed, it is fetched again. A duration of `1h` or more;
|
||||
it cannot be `off`. The Spamhaus lists may be fetched no more than once an
|
||||
hour.
|
||||
- `SWWAF_BLOCKLIST_ACTION` (default `deny`): what is done with a client a
|
||||
blocklist lists. `deny` refuses its requests. `limit:<percent>`, such as
|
||||
`limit:25`, gives it that percentage of every limit. `log` only notes the list
|
||||
in the log line and raises an alert.
|
||||
- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its
|
||||
last fetch, or failed try, before it is fetched again. A duration of `1h` or
|
||||
more; it cannot be `off`. The Spamhaus lists may be fetched no more than once
|
||||
an hour.
|
||||
- `SWWAF_BLOCKLIST_ACTION` (default `deny`): what is done with a client on a
|
||||
blocklist. `deny` refuses its requests. `limit:<percent>`, such as `limit:25`,
|
||||
gives it that percentage of every limit. `log` only notes the list in the log
|
||||
line and raises an alert.
|
||||
- `SWWAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask about each
|
||||
client, such as `dnsbl.dronebl.org`. A DNS blocklist lists an address by
|
||||
answering a DNS query for a name made from it. A zone is a DNS name of at most
|
||||
189 characters. A Spamhaus zone carries its key, as in
|
||||
client, such as `dnsbl.dronebl.org`. A DNS blocklist answers a DNS query for a
|
||||
name made from an address, and so says whether the address is on it. A zone is
|
||||
a DNS name of at most 189 characters. A Spamhaus zone carries its key, as in
|
||||
`<key>.xbl.dq.spamhaus.net`, and is shown with the key masked. A zone listed
|
||||
twice is refused, whatever its case or key. Do not name a zone meant for mail:
|
||||
such a zone lists the residential addresses ordinary visitors come from.
|
||||
such a zone holds the residential addresses ordinary visitors come from.
|
||||
- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through,
|
||||
such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional
|
||||
port, 53 when none is given. Unset, the host's resolver is used. Several zones
|
||||
@@ -587,13 +614,13 @@ do. The defaults judge it by what it does to your service.
|
||||
you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a
|
||||
user or a fragment. Its decision list is at `v1/decisions` under that URL, and
|
||||
is fetched every minute. Set without the key, it stops the start. So does a
|
||||
decision list URL that `SWWAF_BLOCKLIST_URLS` or `SWWAF_ASN_LIMIT_PERCENT_URL`
|
||||
names too.
|
||||
decision list URL that is also in `SWWAF_BLOCKLIST_URLS`, or that is
|
||||
`SWWAF_ASN_LIMIT_PERCENT_URL`.
|
||||
- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
|
||||
`smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
|
||||
URL, it stops the start.
|
||||
- `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client a
|
||||
DNS blocklist lists, or whose AbuseIPDB score is a hit. `deny`,
|
||||
- `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client on
|
||||
a DNS blocklist, or one whose AbuseIPDB score is a hit. `deny`,
|
||||
`limit:<percent>` or `log`, as for `SWWAF_BLOCKLIST_ACTION`. Such a verdict is
|
||||
less certain than a list like DROP, so by default the client gets a quarter of
|
||||
every limit.
|
||||
@@ -653,8 +680,8 @@ See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.
|
||||
in part. Any other body, such as a git push, reaches the app uninspected. See
|
||||
"The Core Rule Set" below.
|
||||
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
|
||||
in a minute after a rule match, a trap path, a Core Rule Set match or a token
|
||||
check. One more bans the client, as a broken rate limit does. A count or
|
||||
in a minute for a rule match, a trap path, a Core Rule Set match or a failed
|
||||
token check. One more bans the client, as a broken rate limit does. A count or
|
||||
`off`. The app's own `401` and `404` answers are not counted.
|
||||
|
||||
### How long bans last and what they cover
|
||||
@@ -664,16 +691,16 @@ See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.
|
||||
and the Core Rule Set always answer `403`.
|
||||
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate
|
||||
limit, byte limit or error burst. A duration; it cannot be `off`.
|
||||
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a limit broken again within
|
||||
this time after a ban ended bans for three times as long as that ban. A
|
||||
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a client that breaks a limit
|
||||
again within this time after its ban ended is banned three times as long. A
|
||||
duration; it cannot be `off`.
|
||||
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken limit that would
|
||||
be longer is permanent instead. A duration; it cannot be `off`.
|
||||
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
||||
attack. A duration; it cannot be `off`.
|
||||
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
|
||||
kept, past, active and permanent. A whole number above zero. Bans you make or
|
||||
keep are kept besides, and never dropped.
|
||||
kept, whether past, active or permanent. A whole number above zero. Bans whose
|
||||
cause is `admin` are kept besides and never dropped (see "Bans" below).
|
||||
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
|
||||
an IPv4 client that a ban covers, from 0 to 32. `24` bans the surrounding /24.
|
||||
An IPv6 ban covers the client's group.
|
||||
@@ -683,9 +710,9 @@ See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.
|
||||
- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the directory of the state
|
||||
files, an absolute path. A directory that does not exist or cannot be written
|
||||
stops the start.
|
||||
- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long after a ban is made,
|
||||
lifted or made permanent `bans.json` is written, with every change in between.
|
||||
A duration; it cannot be `off`.
|
||||
- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
|
||||
written after a ban is made, lifted or made permanent. Every change in between
|
||||
goes into the same write. A duration; it cannot be `off`.
|
||||
- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is
|
||||
written. A duration; it cannot be `off`.
|
||||
|
||||
@@ -693,14 +720,14 @@ See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.
|
||||
|
||||
- `SWWAF_LOG_REQUEST_HEADERS` (default
|
||||
`accept,accept-language,accept-encoding,content-type,origin,range`): the
|
||||
request headers whose values the log line gives, in either case.
|
||||
`Authorization`, `Cookie` and `Set-Cookie` are never logged, even if listed.
|
||||
`Host` and `Transfer-Encoding` are refused: Go's HTTP server takes them out of
|
||||
the request. The host is the field `host`.
|
||||
request headers to record in the log line, in either case. `Authorization`,
|
||||
`Cookie` and `Set-Cookie` are never logged, even if listed. `Host` and
|
||||
`Transfer-Encoding` are refused: Go's HTTP server takes them out of the
|
||||
request. The host is the field `host`.
|
||||
- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own
|
||||
messages that are written: `debug`, `info`, `warn` or `error`. No message is
|
||||
at `debug` yet. It holds back no request log line. `warn` and `error` hold
|
||||
back the line of settings at start.
|
||||
at `debug`. It holds back no request log line. `warn` and `error` hold back
|
||||
the line of settings at start.
|
||||
- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server every line on stdout
|
||||
is also sent to, such as `syslog+tls://logs.example:6514`. The forms are
|
||||
`syslog+udp://`, `syslog+tcp://` and `syslog+tls://`, each with a host and a
|
||||
@@ -713,8 +740,8 @@ See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.
|
||||
- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`,
|
||||
`user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, `news`, `uucp`, `cron`,
|
||||
`authpriv`, `ftp`, or `local0` to `local7`.
|
||||
- `SWWAF_LOG_REMOTE_APP_NAME` (default: `SWWAF_INSTANCE_NAME`): the app name the
|
||||
records carry, 1 to 48 printable ASCII characters without a space.
|
||||
- `SWWAF_LOG_REMOTE_APP_NAME` (default: `SWWAF_INSTANCE_NAME`): the app name in
|
||||
each syslog record, 1 to 48 printable ASCII characters without a space.
|
||||
|
||||
### Tokens and metrics
|
||||
|
||||
@@ -731,18 +758,18 @@ See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.
|
||||
|
||||
### Alert destinations
|
||||
|
||||
- `SWWAF_ALERT_WEBHOOK_URL` (default unset): a webhook each alert is posted to
|
||||
as a JSON object. An `http` or `https` URL without a user or a fragment. The
|
||||
settings logged at start show its path and query as `********`, since many
|
||||
- `SWWAF_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each
|
||||
alert as a JSON object. An `http` or `https` URL without a user or a fragment.
|
||||
The settings logged at start show its path and query as `********`, since many
|
||||
webhooks carry their secret there.
|
||||
- `SWWAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post,
|
||||
such as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and
|
||||
value. A value cannot hold a comma.
|
||||
- `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): a Slack incoming webhook each
|
||||
alert is posted to as a message. It is checked and masked as the webhook URL
|
||||
is.
|
||||
- `SWWAF_ALERT_NTFY_URL` (default unset): an ntfy topic each alert is published
|
||||
to, as the topic's full URL, such as `https://ntfy.sh/my-alerts`. It is
|
||||
- `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook
|
||||
that receives each alert as a message. It is checked and masked as the webhook
|
||||
URL is.
|
||||
- `SWWAF_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each
|
||||
alert, as the topic's full URL, such as `https://ntfy.sh/my-alerts`. It is
|
||||
checked and masked as the webhook URL is.
|
||||
- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token, sent as
|
||||
`Authorization: Bearer <token>`. A control character in it is refused.
|
||||
@@ -757,20 +784,27 @@ See "Blocklists", "DNS blocklists", "AbuseIPDB" and "CrowdSec" below.
|
||||
### Anomaly thresholds
|
||||
|
||||
These only alert. They refuse and ban nothing. Each scope has four thresholds:
|
||||
the most requests and the most bytes counted in a minute and in an hour. Their
|
||||
names are the scope's prefix and `REQUESTS_PER_MINUTE`, `REQUESTS_PER_HOUR`,
|
||||
`BYTES_PER_MINUTE` or `BYTES_PER_HOUR`. Requests are counts, bytes are sizes.
|
||||
Each is `off` by default. What is unusual depends on each service's normal
|
||||
traffic, which the metrics show.
|
||||
the most requests and the most bytes counted in a minute and in an hour.
|
||||
Requests are counts, bytes are sizes. Each is `off` by default. What is unusual
|
||||
depends on each service's normal traffic, which the metrics show. The four
|
||||
thresholds per client are:
|
||||
|
||||
- `SWWAF_ANOMALY_CLIENT_...`: per client.
|
||||
- `SWWAF_ANOMALY_NET_...`: per netblock around a client. The netblock is
|
||||
`SWWAF_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) long for an IPv4
|
||||
client, and `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) long
|
||||
for an IPv6 one.
|
||||
- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
||||
- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
||||
- `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE`;
|
||||
- `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR`.
|
||||
|
||||
The other scopes have the same four names with `CLIENT` replaced:
|
||||
|
||||
- `SWWAF_ANOMALY_NET_...`: per netblock around a client. Two settings give the
|
||||
netblock's length:
|
||||
- `SWWAF_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) for an IPv4
|
||||
client;
|
||||
- `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
||||
client.
|
||||
- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs lookups on.
|
||||
- `SWWAF_ANOMALY_TOTAL_...`: the whole service.
|
||||
- `SWWAF_WATCH_...`: each netblock `SWWAF_WATCH_NETS` names, as a whole.
|
||||
- `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole.
|
||||
`SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
|
||||
as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
|
||||
name, or with a bad netblock, is refused. So is a name used twice.
|
||||
@@ -781,16 +815,17 @@ clients in `SWWAF_ALLOW_NETS` included. At most 20,000 counters are kept.
|
||||
|
||||
### Settings given as files
|
||||
|
||||
Any setting may be given as a file holding its value. The variable named like
|
||||
the setting with `_FILE` added, such as `SWWAF_METRICS_TOKEN_FILE`, names the
|
||||
file. The file's contents are the value, less one newline at the end. They are
|
||||
checked as the value itself would be. Setting both forms stops the start, and so
|
||||
does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` names a
|
||||
file already, and has no `_FILE` form.
|
||||
Any setting may be given as a file holding its value. Add `_FILE` to the
|
||||
setting's name: `SWWAF_METRICS_TOKEN_FILE` names the file that holds
|
||||
`SWWAF_METRICS_TOKEN`. The file's contents are the value, less one newline at
|
||||
the end. They are checked as the value itself would be. Setting both forms stops
|
||||
the start, and so does naming a file that cannot be read.
|
||||
`SWWAF_LOG_REMOTE_TLS_CA_FILE` already holds a file name, so it has no `_FILE`
|
||||
form.
|
||||
|
||||
A token given as a file is out of the app's reach only while the `smallwebwaf`
|
||||
user alone can read the file. Make it on the host, owned by uid 65532 with mode
|
||||
`0400`. Mount its directory into the container read-only:
|
||||
`0400`:
|
||||
|
||||
```sh
|
||||
mkdir -p /srv/app/tokens
|
||||
@@ -799,8 +834,14 @@ chown 65532:65532 /srv/app/tokens/metrics
|
||||
chmod 0400 /srv/app/tokens/metrics
|
||||
```
|
||||
|
||||
and for the container, `-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro` and
|
||||
`-e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics`.
|
||||
Mount its directory into the container read-only, and name the file:
|
||||
|
||||
```sh
|
||||
docker run \
|
||||
-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro \
|
||||
-e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics \
|
||||
...
|
||||
```
|
||||
|
||||
A few limits are fixed rather than settings:
|
||||
|
||||
@@ -813,23 +854,23 @@ A few limits are fixed rather than settings:
|
||||
|
||||
A ban refuses every request from a netblock until it ends, with
|
||||
`SWWAF_BAN_RESPONSE`. The netblock is the client's IPv4 address, or the netblock
|
||||
around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or the client's IPv6 group. A
|
||||
around it, `SWWAF_BAN_SCOPE_V4_PREFIX` long, or the client's IPv6 group. A
|
||||
banned client is not looked up and not counted by rate limiting.
|
||||
|
||||
`smallwebwaf` makes bans for three causes:
|
||||
|
||||
- `limit`: the client broke a rate limit, a byte limit or the error burst. The
|
||||
first such ban lasts `SWWAF_LIMIT_BAN_DURATION`, an hour. A limit broken again
|
||||
within `SWWAF_LIMIT_BAN_REPEAT_WINDOW`, a day, of a ban ending bans for three
|
||||
times as long as that ban: 1, 3, 9, 27 and 81 hours. A ban that would last
|
||||
longer than `SWWAF_MAX_BAN_DURATION`, seven days, is permanent. Such a ban
|
||||
sets the client's counters back to zero.
|
||||
first such ban lasts `SWWAF_LIMIT_BAN_DURATION`, an hour. A client that breaks
|
||||
a limit again within `SWWAF_LIMIT_BAN_REPEAT_WINDOW`, a day, after its ban
|
||||
ended is banned three times as long: 1, 3, 9, 27 and 81 hours. A ban that
|
||||
would last longer than `SWWAF_MAX_BAN_DURATION`, seven days, is permanent.
|
||||
Such a ban sets the client's counters back to zero.
|
||||
- `attack`: the client matched a `ban` rule or asked for a trap path. The first
|
||||
such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days. Any request from the
|
||||
netblock while it lasts makes it permanent. Once it has run out, the netblock
|
||||
is served like any other. Its next clear sign of attack bans it permanently at
|
||||
once.
|
||||
- `crowdsec`: a decision of your CrowdSec engine lists the client. The ban lasts
|
||||
- `crowdsec`: a decision of your CrowdSec engine names the client. The ban lasts
|
||||
until the decision ends.
|
||||
|
||||
A `block` rule and the Core Rule Set ban no one by themselves, since the Core
|
||||
@@ -845,14 +886,15 @@ Each ban carries notes for deciding whether to lift it:
|
||||
setting lowered it;
|
||||
- the request that caused the ban;
|
||||
- the client's AS number, AS name and country;
|
||||
- the reputation sources that listed the client;
|
||||
- the reputation sources the client is on;
|
||||
- the netblock's requests since it was first seen, and how many this ban has
|
||||
refused;
|
||||
- how many earlier bans the netblock had, by cause.
|
||||
|
||||
At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept. Past that, the
|
||||
earliest ban of the netblock that has gone longest without a request is dropped.
|
||||
Bans whose cause is `admin`, those you make or keep, are never dropped.
|
||||
At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept. Past that, one ban is
|
||||
dropped: the earliest ban of the netblock that has gone longest without a
|
||||
request. Bans whose cause is `admin` are never dropped. Those are the bans you
|
||||
make, and the bans `smallwebwaf` made that you keep by setting their cause.
|
||||
|
||||
### Managing bans by editing `bans.json`
|
||||
|
||||
@@ -983,10 +1025,10 @@ COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules
|
||||
```
|
||||
|
||||
A directory mounted over `/etc/smallwebwaf/rules.d` replaces the default file.
|
||||
Single files mounted into it add to it. Docker does not show a single mounted
|
||||
file being replaced, which is how many editors save. So rules to be edited while
|
||||
`smallwebwaf` runs belong in a mounted directory. To run without rules, mount an
|
||||
empty directory or set `SWWAF_RULES_ENABLED=false`.
|
||||
Single files mounted into it add to it. Many editors save a file by replacing
|
||||
it, and a container keeps seeing the old one when the file alone is mounted. So
|
||||
rules to be edited while `smallwebwaf` runs belong in a mounted directory. To
|
||||
run without rules, mount an empty directory or set `SWWAF_RULES_ENABLED=false`.
|
||||
|
||||
## The Core Rule Set
|
||||
|
||||
@@ -998,9 +1040,9 @@ adds to the request's score. A score at or over `SWWAF_WAF_ANOMALY_THRESHOLD` is
|
||||
a match. A match is refused with `403` in `block` mode, and let through in
|
||||
`detect` mode. Either way the log line gives the rule ids and the score, and a
|
||||
`waf_block` alert is sent. A request with more than 1000 query parameters or
|
||||
body fields adds 5 to its score, since Coraza reads only the first 1000. Three
|
||||
kinds of request are not inspected: one a rule file refused, one for a path
|
||||
under `SWWAF_WAF_EXEMPT_PATHS`, and one from a client in `SWWAF_ALLOW_NETS`.
|
||||
body fields adds 5 to its score, since Coraza reads only the first 1000. The
|
||||
Core Rule Set inspects only a request that every earlier check let through. It
|
||||
skips a path under `SWWAF_WAF_EXEMPT_PATHS` and a client in `SWWAF_ALLOW_NETS`.
|
||||
|
||||
`smallwebwaf` changes the Core Rule Set in six ways, so that a code forge's
|
||||
ordinary requests get through. The changes hold in front of every app. No
|
||||
@@ -1018,11 +1060,11 @@ setting undoes them.
|
||||
(930120), shell paths (932160) and command names (932260). They are `path`,
|
||||
`files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`,
|
||||
`artifactName` and `redirect_to`. A file such as `.gitignore` or a branch such
|
||||
as `docker-build` then gets through there. So does what only those three rules
|
||||
refuse, such as `|cat /etc/passwd`. Path traversal and SQL injection are still
|
||||
refused there. Names are matched without regard to case. An app that uses one
|
||||
of these parameters as a file on the server, or passes it to a shell, has to
|
||||
check it itself.
|
||||
as `docker-build` then gets through there. So does an attack only those three
|
||||
rules catch, such as `|cat /etc/passwd`. Path traversal and SQL injection are
|
||||
still refused there. Names are matched without regard to case. An app that
|
||||
uses one of these parameters as a file on the server, or passes it to a shell,
|
||||
has to check it itself.
|
||||
- The cookies `gitea_flash` and `redirect_to` are not read. `Referer` is not
|
||||
checked for a Unix command given without arguments (932340), or for Java
|
||||
starting a process (944110).
|
||||
@@ -1032,8 +1074,8 @@ Body inspection suits apps whose forms carry no code. In front of a code forge
|
||||
it refuses issue and comment text, wiki pages and files saved in the web editor
|
||||
that hold shell commands or code. The log line names the rule ids, which
|
||||
`SWWAF_WAF_DISABLED_RULES` can switch off. A body the Core Rule Set cannot parse
|
||||
adds 5 to the score. So does a multipart body the limit cuts in the middle of a
|
||||
part's header. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to
|
||||
adds 5 to the score. So does a multipart body cut by the limit in the middle of
|
||||
a part's header. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to
|
||||
send the part that is read.
|
||||
|
||||
## Admin endpoints
|
||||
@@ -1055,18 +1097,19 @@ is soon banned. While a token is unset, its endpoints answer `404`.
|
||||
lists it. Needs `SWWAF_ADMIN_TOKEN`, as do the three below.
|
||||
- `POST /_smallwebwaf/bans`: bans a netblock. The body is a JSON object of
|
||||
`netblock`, `duration` and, if you like, `reason`. `netblock` is a netblock
|
||||
such as `203.0.113.0/24`, or a client's address, which bans the netblock a ban
|
||||
on that client covers. `duration` is a duration such as `1h` or `7d`, or
|
||||
`permanent`. The ban starts at once, with the cause `admin`. A bad body is
|
||||
answered `400`, saying what is wrong: one that is not such an object, has
|
||||
another field, is longer than 4 KiB, or names an IPv4-mapped netblock or an
|
||||
address with a zone.
|
||||
- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban on a netblock the
|
||||
address `<client>` is in. It answers `404` when none is active.
|
||||
such as `203.0.113.0/24`, or a client's address. A client's address bans its
|
||||
IPv4 netblock (`SWWAF_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a
|
||||
duration such as `1h` or `7d`, or `permanent`. The ban starts at once, with
|
||||
the cause `admin`. A bad body is answered `400`, saying what is wrong:
|
||||
- one that is not such an object, or has another field;
|
||||
- one longer than 4 KiB;
|
||||
- one that names an IPv4-mapped netblock, or an address with a zone.
|
||||
- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban covering the
|
||||
address `<client>`. It answers `404` when none is active.
|
||||
- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client.
|
||||
Under `client`, its counters and history as `clients.json` holds them, or
|
||||
`null` when it has never been seen. Under `bans`, every ban on a netblock the
|
||||
address is in, with its notes.
|
||||
`null` when it has never been seen. Under `bans`, every ban covering the
|
||||
address, with its notes.
|
||||
|
||||
An admin whose own address is banned lifts that ban by editing `bans.json`.
|
||||
|
||||
@@ -1098,8 +1141,8 @@ fields from `time` to `user_agent`, `request_id`, `peer_ip`, `client_group`,
|
||||
- the source cannot place the client;
|
||||
- GeoJS had not answered by the time the request went on.
|
||||
- `content_type` and `content_length` are the request's own. `request_headers`
|
||||
holds the headers `SWWAF_LOG_REQUEST_HEADERS` names, by name in lower case.
|
||||
`has_authorization` and `has_cookie` are true when the request had those
|
||||
holds the headers listed in `SWWAF_LOG_REQUEST_HEADERS`, by name in lower
|
||||
case. `has_authorization` and `has_cookie` are true when the request had those
|
||||
headers; their values are never logged. `websocket` is true when the app
|
||||
switched the connection to another protocol.
|
||||
- `status` is what the client was sent, `0` if nothing. `upstream_status` is
|
||||
@@ -1117,8 +1160,9 @@ fields from `time` to `user_agent`, `request_id`, `peer_ip`, `client_group`,
|
||||
- `too_large` and `timed_out`: a size or time limit was passed;
|
||||
- `upstream_error`: the app could not be reached, or its answer broke off;
|
||||
- `admin`: answered at one of `smallwebwaf`'s own endpoints.
|
||||
- `would_action` is there in `observe` mode. It names the action `enforce` mode
|
||||
would have taken. `action` then names what was done, normally `forward`.
|
||||
- In `observe` mode, `would_action` names what `enforce` mode would have done
|
||||
with a request it would have refused. `action` then names what was done,
|
||||
normally `forward`.
|
||||
- `limit_percent` and `limit_percent_setting` give the client's percentage of
|
||||
the rate limits and the setting that lowered it, when one did. `bytes_percent`
|
||||
and `bytes_percent_setting` do the same for the byte limits.
|
||||
@@ -1126,8 +1170,8 @@ fields from `time` to `user_agent`, `request_id`, `peer_ip`, `client_group`,
|
||||
limiting counts them, this request included. `minute_bytes`, `hour_bytes` and
|
||||
`day_bytes` give its bytes. Each window is counted in two fixed buckets, the
|
||||
earlier one weighted by how much of it the window still covers, so a count can
|
||||
have a fraction. `counts` is left out for a request rate limiting does not
|
||||
count.
|
||||
have a fraction. `counts` is left out when rate limiting did not count the
|
||||
request.
|
||||
- `rule_ids` lists the rule file rules that matched, in order, up to the one
|
||||
that refused. `waf_score` and `waf_rule_ids` give the Core Rule Set's score
|
||||
and the rules that matched, for a request it inspected.
|
||||
@@ -1135,8 +1179,8 @@ fields from `time` to `user_agent`, `request_id`, `peer_ip`, `client_group`,
|
||||
limit; `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or
|
||||
`error_burst`. `offence` is then `limit`. A request whose bytes broke a limit
|
||||
was passed on whole, so its `action` stays `forward`.
|
||||
- `reputation` lists the sources that list the client: blocklist URLs, the
|
||||
CrowdSec decision list's URL, DNS blocklist zones, then `abuseipdb`.
|
||||
- `reputation` lists the sources the client is on: blocklist URLs, the CrowdSec
|
||||
decision list's URL, DNS blocklist zones, then `abuseipdb`.
|
||||
- `ban_expires` is when the client's ban ends, for a request that made one or
|
||||
was refused under one: a time, or `permanent`.
|
||||
- The timings are in milliseconds. `duration_total` runs from the request's
|
||||
@@ -1155,33 +1199,38 @@ No body is logged, and no header but those above. `smallwebwaf`'s own messages
|
||||
While `SWWAF_LOG_REMOTE_URL` is set, every line on stdout is also sent to that
|
||||
server, as the message of an RFC 5424 syslog record. Over UDP each record is one
|
||||
datagram. Over TCP and TLS each record follows its length in bytes and a space.
|
||||
The record carries the facility `SWWAF_LOG_REMOTE_FACILITY` names, the severity
|
||||
informational, the time, the host's name and the app name
|
||||
`SWWAF_LOG_REMOTE_APP_NAME` gives. stdout is unchanged.
|
||||
stdout is unchanged. The record carries:
|
||||
|
||||
- the facility from `SWWAF_LOG_REMOTE_FACILITY`;
|
||||
- the severity informational;
|
||||
- the time;
|
||||
- the host's name;
|
||||
- the app name from `SWWAF_LOG_REMOTE_APP_NAME`.
|
||||
|
||||
Lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines, so a server that is
|
||||
slow or down never holds up a request. When the buffer is full, the oldest line
|
||||
is dropped. A failed connection is tried again a second later, and twice as long
|
||||
after each further failure, up to a minute. A line too long for one UDP datagram
|
||||
is dropped alone. As `smallwebwaf` stops, it sends the lines still waiting for
|
||||
at most two seconds and gives up the rest. stdout has carried them.
|
||||
is dropped alone. As `smallwebwaf` stops, it spends at most two seconds sending
|
||||
the lines still waiting, and drops the rest. stdout has carried them.
|
||||
|
||||
## Alerts
|
||||
|
||||
Each alert goes to each destination you name. `SWWAF_ALERT_WEBHOOK_URL` gets it
|
||||
as one JSON object. `SWWAF_ALERT_SLACK_WEBHOOK_URL` and `SWWAF_ALERT_NTFY_URL`
|
||||
get a message made from it. An alert is one of these events, sent when
|
||||
`SWWAF_ALERT_EVENTS` names it:
|
||||
get a message made from it. An alert is one of these events, and is sent when
|
||||
its event is in `SWWAF_ALERT_EVENTS`:
|
||||
|
||||
- `ban`: a ban `smallwebwaf` made.
|
||||
- `permanent_ban`: a permanent ban it made, or a ban a request made permanent.
|
||||
- `waf_block`: a request the Core Rule Set scored at or over the threshold. In
|
||||
- `permanent_ban`: a permanent ban it made, or an existing ban made permanent by
|
||||
a request.
|
||||
- `waf_block`: the Core Rule Set scored a request at or over the threshold. In
|
||||
`block` mode the request was refused. In `detect` mode it was not, and the
|
||||
alert's `detail` gives `mode`.
|
||||
- `anomaly`: a count over an anomaly threshold.
|
||||
- `reputation_hit`: a request whose client a blocklist, the CrowdSec decision
|
||||
list or a DNS blocklist lists, or whose AbuseIPDB score is a hit. One alert
|
||||
per source.
|
||||
- `reputation_hit`: a request from a client on a blocklist, the CrowdSec
|
||||
decision list or a DNS blocklist, or with an AbuseIPDB score that is a hit.
|
||||
One alert per source.
|
||||
- `source_failure`: GeoJS failing, a list that cannot be fetched, a DNS
|
||||
blocklist or AbuseIPDB failing or refusing a query, or the day's AbuseIPDB
|
||||
checks used up.
|
||||
@@ -1189,9 +1238,9 @@ get a message made from it. An alert is one of these events, sent when
|
||||
cannot be written, or a replacement lookup database that cannot be read.
|
||||
|
||||
The bans you make raise no alert. In `observe` mode, a request that would have
|
||||
made a ban raises the alert `enforce` mode would have, with `mode` set to
|
||||
`observe` in its `detail`. This is the alert for a ban for a broken rate limit,
|
||||
shown indented; it is sent on one line:
|
||||
made a ban still raises the ban's alert, with `mode` set to `observe` in its
|
||||
`detail`. This is the alert for a ban for a broken rate limit, shown indented;
|
||||
it is sent on one line:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -1255,11 +1304,16 @@ shown indented; it is sent on one line:
|
||||
alert.
|
||||
|
||||
Slack and ntfy get a message with a title and a text. The title is the instance
|
||||
and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then a
|
||||
line for each of the client, netblock, country, file, source, error and mode the
|
||||
alert has, and the suppressed repeats if any. Slack is posted the title in bold
|
||||
and the text below it, with `&`, `<` and `>` escaped. ntfy is posted the text,
|
||||
with the title in `Title`, the token if set, and a priority and a tag by event:
|
||||
and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then one
|
||||
line for each of these that the alert has:
|
||||
|
||||
- the client, the netblock and the country;
|
||||
- the file, the source, the error and the mode from `detail`;
|
||||
- the suppressed repeats, if any.
|
||||
|
||||
Slack is posted the title in bold and the text below it, with `&`, `<` and `>`
|
||||
escaped. ntfy is posted the text, with the title in `Title`, the token if set,
|
||||
and a priority and a tag by event:
|
||||
|
||||
- `ban`: priority `default`, tag `no_entry`.
|
||||
- `permanent_ban`: priority `high`, tag `no_entry`.
|
||||
@@ -1379,10 +1433,13 @@ write. A hard kill loses what changed since the last write.
|
||||
|
||||
At start the files are read back. Each client keeps its counts, so a restart
|
||||
gives no one a fresh allowance. Each ban keeps refusing until it ends. A missing
|
||||
file is empty state. A file that does not parse, or has another `version`, stops
|
||||
the start with a message naming the file and the place. So does an entry without
|
||||
a field it needs, a ban whose `cause` is not `limit`, `attack`, `admin` or
|
||||
`crowdsec`, and a ban whose `expires` is not after its `start`.
|
||||
file is empty state. These stop the start, with a message naming the file and
|
||||
the place:
|
||||
|
||||
- a file that does not parse, or has another `version`;
|
||||
- an entry without a field it needs;
|
||||
- a ban whose `cause` is not `limit`, `attack`, `admin` or `crowdsec`;
|
||||
- a ban whose `expires` is not after its `start`.
|
||||
|
||||
While it runs, `smallwebwaf` watches the directory and takes in your edit of a
|
||||
file as soon as you save it. What the file then holds replaces what it held. It
|
||||
@@ -1400,52 +1457,66 @@ back. A file you remove is written again at its next write.
|
||||
By default `smallwebwaf` asks GeoJS, a free web service that needs no account,
|
||||
about every new visitor's address. It asks whether or not a setting uses the
|
||||
answer, since the log line, the history, the ban notes and the metrics carry the
|
||||
AS number and country. Four kinds of visitor are not sent: those in
|
||||
`SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`, those a ban covers, and those on a
|
||||
private, loopback or link-local address, which no source can place. An IPv6
|
||||
visitor is asked about by the first address of its group. Each answer is kept
|
||||
for seven days, in memory and in `lookups.json`.
|
||||
AS number and country. These addresses are never sent:
|
||||
|
||||
A request waits for its client's first answer, up to `SWWAF_LOOKUP_TIMEOUT`,
|
||||
only while a setting acts on the answer before the request goes on: a country
|
||||
list, `SWWAF_ADD_LOOKUP_HEADERS`, or any setting under "Lower limits by AS
|
||||
number and country" above. Otherwise it goes on at once, logged without the
|
||||
answer. The answer reaches the client's history and ban notes when it comes.
|
||||
Addresses are asked about together, up to 200 in one request, one request at a
|
||||
time. While GeoJS fails, new visitors count as from an unknown country, which
|
||||
- one in `SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`;
|
||||
- one under a ban;
|
||||
- a private, loopback or link-local address, which no source can place.
|
||||
|
||||
An IPv6 visitor is asked about by the first address of its group. Each answer is
|
||||
kept for seven days, in memory and in `lookups.json`.
|
||||
|
||||
A request waits for its client's first answer only when a setting needs the
|
||||
answer before the request goes on. It waits up to `SWWAF_LOOKUP_TIMEOUT`. The
|
||||
settings that make it wait:
|
||||
|
||||
- a country list;
|
||||
- `SWWAF_ADD_LOOKUP_HEADERS`;
|
||||
- any setting under "Lower limits by AS number and country" above.
|
||||
|
||||
Otherwise the request goes on at once, logged without the answer. The answer
|
||||
reaches the client's history and ban notes when it comes. Addresses are asked
|
||||
about together, up to 200 in one request, one request at a time. While GeoJS
|
||||
fails, new visitors count as from an unknown country, which
|
||||
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses. GeoJS is then left alone for a
|
||||
second, and twice as long after each further failure, up to five minutes.
|
||||
|
||||
To keep your visitors' addresses on your own host, set
|
||||
`SWWAF_LOOKUP_SOURCE=off`, or set it to `file`. `file` looks every client up in
|
||||
the free IPinfo Lite database (`ipinfo_lite.mmdb`), which `SWWAF_LOOKUP_DB_PATH`
|
||||
names. You download the database with an IPinfo account and mount it into the
|
||||
the free IPinfo Lite database (`ipinfo_lite.mmdb`) at `SWWAF_LOOKUP_DB_PATH`.
|
||||
You download the database with an IPinfo account and mount it into the
|
||||
container. It answers at once, with nothing sent off the host. A client whose
|
||||
address it does not hold counts as from an unknown country. `smallwebwaf` reads
|
||||
the whole file at start. It reads the file again once it has gone 2 seconds
|
||||
without a change after you replace it. A replacement it cannot read is logged
|
||||
and alerted, and the file read before stays in use. Mount the directory that
|
||||
holds the file rather than the file itself: docker does not show a single
|
||||
mounted file being replaced. IPinfo releases the database under the Creative
|
||||
holds the file rather than the file itself: many editors and download tools
|
||||
replace a file when they save it, and a container keeps seeing the old one when
|
||||
the file alone is mounted. IPinfo releases the database under the Creative
|
||||
Commons Attribution-ShareAlike 4.0 licence. It asks for a link to IPinfo on the
|
||||
site that uses it, such as "IP address data is powered by IPinfo".
|
||||
|
||||
## Blocklists
|
||||
|
||||
`SWWAF_BLOCKLIST_URLS` names text files of addresses and netblocks, one to a
|
||||
`SWWAF_BLOCKLIST_URLS` lists text files of addresses and netblocks, one to a
|
||||
line, as the Spamhaus DROP list is written. Anything after a `;` or `#` is left
|
||||
out. An IPv4-mapped address such as `::ffff:192.0.2.0/120` is read as the IPv4
|
||||
netblock it stands for.
|
||||
|
||||
Each list, and the file `SWWAF_ASN_LIMIT_PERCENT_URL` names, is fetched
|
||||
`SWWAF_BLOCKLIST_REFRESH` after it was last fetched or tried, one after another.
|
||||
Each list is fetched again `SWWAF_BLOCKLIST_REFRESH` after its last try, one
|
||||
list at a time. The file of `SWWAF_ASN_LIMIT_PERCENT_URL` is too.
|
||||
`reputation.json` keeps when each was last tried, so a restart does not fetch a
|
||||
list again before its time. A fetch fails when the server answers other than
|
||||
`200`, does not finish within a minute, or sends more than 16 MiB. It also fails
|
||||
on a line that is not an address or netblock, or, for the AS number file, not
|
||||
`AS<number>:<percent>`. The copy fetched before stays in use. The failure is
|
||||
logged and raised as a `source_failure` alert. Each list is named by its URL in
|
||||
the log, the alerts, the ban notes and the metrics, so keep a secret out of it.
|
||||
list again before its time. A fetch fails when:
|
||||
|
||||
- the server answers other than `200`;
|
||||
- the fetch does not finish within a minute;
|
||||
- the list is longer than 16 MiB;
|
||||
- a line is not an address or netblock, or, for the AS number file, not
|
||||
`AS<number>:<percent>`.
|
||||
|
||||
The copy fetched before stays in use. The failure is logged and raised as a
|
||||
`source_failure` alert. Each list is named by its URL in the log, the alerts,
|
||||
the ban notes and the metrics, so keep a secret out of it.
|
||||
|
||||
`deny` suits lists of networks that send nothing legitimate, such as DROP.
|
||||
`limit:<percent>` suits lists of addresses shared with ordinary visitors, such
|
||||
@@ -1463,10 +1534,15 @@ background, the first time the client is seen. The request goes on at once, and
|
||||
so does every request from the client until the zone has answered. The name
|
||||
asked about is the one RFC 5782 gives: the numbers of an IPv4 address reversed,
|
||||
or the 32 hex digits of an IPv6 address reversed. `192.0.2.99` is asked about as
|
||||
`99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` lists the client. A
|
||||
name that does not exist does not. Any other answer is a failure: an address in
|
||||
`127.255.255.0/24`, with which Spamhaus refuses a query, an address outside
|
||||
`127.0.0.0/8`, a resolver error, or no answer within `SWWAF_REPUTATION_TIMEOUT`.
|
||||
`99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` lists the client, but
|
||||
one in `127.255.255.0/24` is Spamhaus refusing the query. A name that does not
|
||||
exist does not list the client. These are failures:
|
||||
|
||||
- an answer in `127.255.255.0/24`;
|
||||
- an answer outside `127.0.0.0/8`;
|
||||
- a resolver error;
|
||||
- no answer within `SWWAF_REPUTATION_TIMEOUT`.
|
||||
|
||||
A failure is logged and alerted, and the zone is not asked again for a minute.
|
||||
Each verdict is kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and in
|
||||
`reputation.json`.
|
||||
@@ -1479,10 +1555,14 @@ The key of a Spamhaus zone under `dq.spamhaus.net` is its first label.
|
||||
|
||||
While `SWWAF_ABUSEIPDB_KEY` is set, `smallwebwaf` asks AbuseIPDB's check
|
||||
endpoint for the abuse confidence score of a client, from 0 to 100. Only a
|
||||
client that has committed an offence is checked: one that broke a limit, matched
|
||||
a `ban` rule, asked for a trap path, or was refused by a `block` rule, the Core
|
||||
Rule Set or a token check. The check runs in the background, at the client's
|
||||
first request after the offence, so no request waits. The score applies to the
|
||||
client that has committed an offence is checked. An offence is one of:
|
||||
|
||||
- a broken limit;
|
||||
- a `ban` rule match or a trap path;
|
||||
- a refusal by a `block` rule, the Core Rule Set or a token check.
|
||||
|
||||
The check runs in the background, at the client's first request after the
|
||||
offence that no ban refuses, so no request waits. The score applies to the
|
||||
client's later requests. An IPv6 group is checked by the address of the request
|
||||
that has it checked, and the score is used for the whole group. A score at or
|
||||
over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and `SWWAF_REPUTATION_ACTION` says
|
||||
@@ -1498,21 +1578,21 @@ sent in the `Key` header and nowhere else, and is never logged.
|
||||
|
||||
## CrowdSec
|
||||
|
||||
While `SWWAF_CROWDSEC_LAPI_URL` names the local API of a CrowdSec engine you
|
||||
While `SWWAF_CROWDSEC_LAPI_URL` is set to the local API of a CrowdSec engine you
|
||||
run, `smallwebwaf` fetches its decision list every minute. It sends
|
||||
`SWWAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key` header. The list is kept as a
|
||||
blocklist is kept. The decisions of the type `ban` on an address or a netblock
|
||||
are used. Any other, such as a captcha or a country, is left out. A decision
|
||||
lasts for the `duration` it had left when the engine answered.
|
||||
|
||||
A request from a client a decision in force lists is refused. It bans the
|
||||
client's netblock until the decision ends, with the cause `crowdsec`. Only a
|
||||
client that sends a request is banned, so a long list does not fill `bans.json`.
|
||||
Such a ban is never made permanent, and does not lengthen the netblock's next
|
||||
ban for a broken limit. A ban you lift while the decision is in force is
|
||||
followed by another at the client's next request. Delete the decision in
|
||||
CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and lift
|
||||
the ban once the list has been fetched again. A fetch fails when the engine
|
||||
A request is refused when a decision still in force names its client. The
|
||||
client's netblock is banned until the decision ends, with the cause `crowdsec`.
|
||||
Only a client that sends a request is banned, so a long list does not fill
|
||||
`bans.json`. Such a ban is never made permanent, and does not lengthen the
|
||||
netblock's next ban for a broken limit. A ban you lift while the decision is in
|
||||
force is followed by another at the client's next request. Delete the decision
|
||||
in CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and
|
||||
lift the ban once the list has been fetched again. A fetch fails when the engine
|
||||
answers other than `200`, a redirect included. It also fails when the fetch does
|
||||
not finish within a minute, the answer is longer than 16 MiB, or the answer is
|
||||
not a list of decisions.
|
||||
|
||||
Reference in New Issue
Block a user