sneak/smallwebwaf
1
0
Fork 0
Code Issues 27 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

SPEC and README: review fixes, country lists, GeoJS lookups (closes #6)

Browse Source 
Address the review of the spec update and fold in issues 8 and 11.
Default ban rules are anchored at the site root. `bans.json` is bounded
by `MAX_BANS` and rewritten only when a ban is made, lifted or made
permanent. `clients.json` keeps one client per line, holds at most 20000
clients and is written every 15 minutes. Anomaly counters and alert
state go to a new `alerts.json`, the AbuseIPDB count to
`reputation.json`. 401 no longer counts toward the error burst. New
settings: `DENIED_COUNTRIES`, `EXCLUSIVELY_ALLOWED_COUNTRIES`,
`LOOKUP_SOURCE` (the IPinfo file or GeoJS), `LOOKUP_TIMEOUT`,
`CLIENT_REQUEST_HEADER_MAX_BYTES` and `CLIENT_IDLE_TIMEOUT`.

Model: opus-5-5
This commit is contained in:
sneak
2026-09-23 13:37:34 +00:00
parent 43c63faa04
commit 8fc7539f1a
2 changed files with 304 additions and 142 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+42 -22
View File
@@ -52,7 +52,7 @@ goes through the candidates one by one.
other setting has a default chosen for a service facing the internet in 2026. other setting has a default chosen for a service facing the internet in 2026.
- Real client address worked out from `X-Forwarded-For`, trusting only the proxy - Real client address worked out from `X-Forwarded-For`, trusting only the proxy
networks you list, by default the private address ranges. IPv6 clients are networks you list, by default the private address ranges. IPv6 clients are
counted by /64. counted by /64 by default.
- Size and time limits on requests and responses, both between the client and - Size and time limits on requests and responses, both between the client and
`smallwebwaf` and between `smallwebwaf` and the app: by default a request may `smallwebwaf` and between `smallwebwaf` and the app: by default a request may
take 60 seconds and 100 MB, a response 30 minutes and 5 GB. take 60 seconds and 100 MB, a response 30 minutes and 5 GB.
@@ -61,8 +61,14 @@ goes through the candidates one by one.
real visitors need. real visitors need.
- Netblocks that bypass rate limiting, netblocks that bypass everything, and - Netblocks that bypass rate limiting, netblocks that bypass everything, and
netblocks that are always refused. netblocks that are always refused.
- AS number and country lookup for every client from the IPinfo Lite database - AS number and country lookup for every client, off until you choose a source:
file, which you download and mount (see "Lookup database" below). the IPinfo Lite database file, which you download and mount, or the free GeoJS
web service (see "Country and AS number lookup" below).
- Country lists: `DENIED_COUNTRIES` refuses every request from the countries
listed, `EXCLUSIVELY_ALLOWED_COUNTRIES` every request from anywhere else. Such
a request gets the answer a banned client gets as soon as the client's address
has been looked up, before its body is read and without the rule files or the
Core Rule Set looking at it, and no ban is made.
- Biased limits: listed AS numbers and countries get a percentage of every - Biased limits: listed AS numbers and countries get a percentage of every
limit, for example 50 percent for common abuse-source networks, so their limit, for example 50 percent for common abuse-source networks, so their
clients are banned after fewer requests. Zero percent is a zero allowance: the clients are banned after fewer requests. Zero percent is a zero allowance: the
@@ -96,11 +102,11 @@ goes through the candidates one by one.
fields, the decision taken and why, AS number and country, and timings. fields, the decision taken and why, AS number and country, and timings.
Optionally also sent to a remote syslog server. Optionally also sent to a remote syslog server.
- Prometheus metrics on their own port. - Prometheus metrics on their own port.
- State (bans with their notes, each client's counters and history, the - State (bans with their notes, each client's counters, history and lookup
reputation cache) held in memory and kept in readable JSON files that always answer, the reputation cache, the alerting state) held in memory and kept in
hold a copy of it, so a restart loses nothing. Edit a file, or add a rule readable JSON files, written regularly and at every stop, so a restart loses
file, and the running `smallwebwaf` picks up the change. Nothing is read from nothing. Edit a file, or add a rule file, and the running `smallwebwaf` picks
disk while serving a request. up the change. Nothing is read from disk while serving a request.
- A small admin endpoint for health checks, listing, adding and lifting bans, - A small admin endpoint for health checks, listing, adding and lifting bans,
and asking why a given address was refused. and asking why a given address was refused.
@@ -115,7 +121,9 @@ For each request `smallwebwaf`:
- works out who the client really is; - works out who the client really is;
- lets it straight through if it is on the bypass list, refuses it if it is on - lets it straight through if it is on the bypass list, refuses it if it is on
the deny list or currently banned; the deny list or currently banned;
- looks up its AS number and country, and any cached reputation verdict; - looks up its AS number and country, and refuses it if that country is denied,
or is not among the only ones allowed;
- checks for a cached reputation verdict;
- picks the client's limit percentage from those; - picks the client's limit percentage from those;
- checks the minute, hour and day request counters against the limits, and bans - checks the minute, hour and day request counters against the limits, and bans
the client if it breaks one; the client if it breaks one;
@@ -162,7 +170,7 @@ A rule file is one rule per line: a name, what to match against, what to do, and
a regex. a regex.
``` ```
env-file path ban (?i)/\.env(\.[a-z]+)?$ env-file path ban (?i)^/\.env(\.[a-z]+)?$
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
``` ```
@@ -170,19 +178,31 @@ scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
rules, the rule file format, the state files, the log fields, the metrics, rules, the rule file format, the state files, the log fields, the metrics,
failure behaviour and the build order. failure behaviour and the build order.
## Lookup database ## Country and AS number lookup
AS number and country lookups, and the biased limits that use them, need the AS number and country lookups, and the country lists and biased limits that use
free IPinfo Lite database (`ipinfo_lite.mmdb`). You download it with your own them, are off until you choose one of two sources with `LOOKUP_SOURCE`.
IPinfo account, mount it into the container, point `LOOKUP_DB_PATH` at it and
refresh it when you choose; `smallwebwaf` never downloads it itself. IPinfo `LOOKUP_SOURCE=file` reads the free IPinfo Lite database (`ipinfo_lite.mmdb`).
releases it under the Creative Commons Attribution-ShareAlike 4.0 International You download it with your own IPinfo account, mount the directory that holds it
License and asks for attribution, in its own words on https://ipinfo.io/lite: into the container, point `LOOKUP_DB_PATH` at the file and refresh it when you
"The attribution requirements can be met by giving our service credit as your choose; `smallwebwaf` never downloads it itself, and reads it again when you
data source. Simply place a link to IPinfo on the website, application, or replace it. It has to be the directory rather than the file itself: docker does
social media account that uses our data." Its example of such a credit is a link not show a single mounted file being replaced, so a refresh would go unseen.
mentioning "IP address data is powered by IPinfo". A service that uses the IPinfo releases it under the Creative Commons Attribution-ShareAlike 4.0
database through `smallwebwaf` should carry that link. International License and asks for attribution, in its own words on
https://ipinfo.io/lite: "The attribution requirements can be met by giving our
service credit as your data source. Simply place a link to IPinfo on the
website, application, or social media account that uses our data." Its example
of such a credit is a link mentioning "IP address data is powered by IPinfo". A
service that uses the database through `smallwebwaf` should carry that link.
`LOOKUP_SOURCE=geojs` asks the free GeoJS web service instead, with no account
and no file. Every new visitor's address is sent to GeoJS. Each answer is kept
for seven days, across restarts, and many addresses are asked about in one
request. GeoJS publishes no rate limit but may block a caller it thinks asks too
much; while it is not answering, new visitors count as coming from an unknown
country, which `EXCLUSIVELY_ALLOWED_COUNTRIES` refuses.
## Documents ## Documents
SPEC.md
+262 -120
View File
@@ -12,8 +12,8 @@ forwards to the application. The sidecar limits request and byte rates per
client, bounds the size and duration of every request and response, detects client, bounds the size and duration of every request and response, detects
attacks, bans abusers (briefly at first, for seven days on a clear sign of attacks, bans abusers (briefly at first, for seven days on a clear sign of
attack, permanently when they keep at it), consults IP reputation sources, looks attack, permanently when they keep at it), consults IP reputation sources, looks
up the AS number and country of each client, lowers its limits for listed AS up the AS number and country of each client, can refuse whole countries, lowers
numbers and countries, and sends alerts. its limits for listed AS numbers and countries, and sends alerts.
It is meant to protect an app on the open internet from the first request with It is meant to protect an app on the open internet from the first request with
one setting, `UPSTREAM_URL`: every other setting has a default chosen for a one setting, `UPSTREAM_URL`: every other setting has a default chosen for a
@@ -52,8 +52,9 @@ directory of hand-editable text files.
- Client identification: works out the real client IP from - Client identification: works out the real client IP from
`X-Forwarded-For`, trusting only the proxy netblocks in `TRUSTED_PROXIES`, `X-Forwarded-For`, trusting only the proxy netblocks in `TRUSTED_PROXIES`,
by default the private address ranges. by default the private address ranges.
- Lookup: AS number and country from one database file the operator - Lookup: AS number and country, from one of two sources the operator
supplies, held in memory. chooses: a database file the operator supplies, held in memory, or the
GeoJS web service, whose answers are kept for seven days.
- Reputation: static blocklists fetched on a schedule; DNSBL and reputation - Reputation: static blocklists fetched on a schedule; DNSBL and reputation
API queries made in the background and cached. API queries made in the background and cached.
- Counters: per-client request and byte counts per minute, hour and day, - Counters: per-client request and byte counts per minute, hour and day,
@@ -104,7 +105,7 @@ whole exchange.
Otherwise use the TCP peer address and ignore the header. Otherwise use the TCP peer address and ignore the header.
- IPv6 clients are grouped by prefix (`IPV6_GROUP_PREFIX`, default 64) for - IPv6 clients are grouped by prefix (`IPV6_GROUP_PREFIX`, default 64) for
counting and banning, because one abuser usually controls a whole /64. A counting and banning, because one abuser usually controls a whole /64. A
client is therefore one IPv4 address or one IPv6 /64. client is therefore one IPv4 address or one IPv6 group, a /64 by default.
- Static lists. - Static lists.
- In `ALLOW_NETS`: skip every check below and forward. Still counted for - In `ALLOW_NETS`: skip every check below and forward. Still counted for
anomaly alerts. anomaly alerts.
@@ -112,8 +113,17 @@ whole exchange.
- Ban ledger. An active ban on the client's netblock: refuse with - Ban ledger. An active ban on the client's netblock: refuse with
`BAN_RESPONSE`. If a clear sign of attack caused the ban, the ban becomes `BAN_RESPONSE`. If a clear sign of attack caused the ban, the ban becomes
permanent (see "Bans"). permanent (see "Bans").
- Lookup AS number and country, when a lookup database is configured. Unknown is - Look up the AS number and country, unless `LOOKUP_SOURCE` is `off`. With
a valid result and changes nothing. GeoJS, a client whose answer is not yet kept waits for it, up to
`LOOKUP_TIMEOUT`. A client the lookup cannot place has an unknown country and
AS number: the exclusive country list refuses it, the biased thresholds give
it `UNKNOWN_LIMIT_PERCENT`, and nothing else treats it differently.
- Country lists. A client whose country is in `DENIED_COUNTRIES`, or, when
`EXCLUSIVELY_ALLOWED_COUNTRIES` is set, is not in it, is refused with
`BAN_RESPONSE`. Every step up to here needs only the client's address, so the
request body has not been read yet, and the refusal skips everything below,
the rule files and the Core Rule Set included. It is counted in the metrics,
but it is not an offence and makes no ban.
- Reputation. - Reputation.
- Address inside a fetched blocklist: apply `BLOCKLIST_ACTION`. - Address inside a fetched blocklist: apply `BLOCKLIST_ACTION`.
- Cached DNSBL or reputation API result: apply `REPUTATION_ACTION`. - Cached DNSBL or reputation API result: apply `REPUTATION_ACTION`.
@@ -139,10 +149,10 @@ whole exchange.
- Add response bytes (and request body bytes) to the byte counters. A client - Add response bytes (and request body bytes) to the byte counters. A client
whose byte total passes a byte limit times the percentage has broken that whose byte total passes a byte limit times the percentage has broken that
limit and is banned. The response in progress is not cut off. limit and is banned. The response in progress is not cut off.
- Count the client's requests answered with 401, 403 or 404, whether the app - Count the client's requests answered with 403 or 404, whether the app gave
gave that answer or the sidecar refused the request after a rule file or that answer or the sidecar refused the request after a rule file or Core
Core Rule Set match. More than `ERROR_BURST_THRESHOLD` of them within a Rule Set match. More than `ERROR_BURST_THRESHOLD` of them within a minute
minute breaks a limit and bans the client. breaks a limit and bans the client.
- Update the client's history and the anomaly counters, and evaluate alert - Update the client's history and the anomaly counters, and evaluate alert
thresholds. thresholds.
@@ -157,17 +167,25 @@ whole exchange.
shutdown and loaded again at start (see "Persistent state"), so a restart does shutdown and loaded again at start (see "Persistent state"), so a restart does
not hand every client a fresh allowance. Buckets whose time has passed are not hand every client a fresh allowance. Buckets whose time has passed are
discarded on load. discarded on load.
- Memory is bounded by `MAX_TRACKED_CLIENTS`; when full, the least recently seen - Memory is bounded by `MAX_TRACKED_CLIENTS` and `MAX_BANS`. When the table of
client is dropped first, with its history and its past bans. Active bans are clients is full, the least recently seen client is dropped first, with its
never dropped; one whose client is no longer tracked is dropped when it ends. history and its lookup answer. When `MAX_BANS` bans are held, the ban that has
gone longest without a request from its netblock is dropped first, whether it
is past, active or permanent (see "Bans").
## Bans ## Bans
A ban refuses every request from a netblock, answering with `BAN_RESPONSE` A ban refuses every request from a netblock, answering with `BAN_RESPONSE`
(`403` by default), until the ban ends; a permanent ban never ends. The netblock (`403` by default), until the ban ends. A permanent ban does not run out: it
a ban covers is the client's IPv4 address (or the wider IPv4 prefix that ends only when an admin lifts it, or when a new ban is made while `MAX_BANS`
`BAN_SCOPE_V4_PREFIX` sets) or its IPv6 /64, and never more, so a permanent ban bans are held and it is the ban that has gone longest without a request. A
does not reach neighbours who did nothing. scanner whose ban was dropped that way is refused and banned again by its next
probe.
The netblock a ban covers is the client: its IPv4 address, or its IPv6 group of
`IPV6_GROUP_PREFIX` (a /64 by default). `BAN_SCOPE_V4_PREFIX` can widen an IPv4
ban to the surrounding netblock. At the defaults a ban covers one address or one
/64, so a permanent ban does not reach neighbours who did nothing.
An offence is a request the sidecar holds against the client: one that carries a An offence is a request the sidecar holds against the client: one that carries a
clear sign of attack, breaks a limit, or is refused by a rule file or the Core clear sign of attack, breaks a limit, or is refused by a rule file or the Core
@@ -206,21 +224,30 @@ Every ban carries notes with what an admin needs to decide whether to lift it
lifted before it ends is kept, marked as lifted, and does not count toward a lifted before it ends is kept, marked as lifted, and does not count toward a
longer ban; deleting its entry from `bans.json` forgets it entirely. longer ban; deleting its entry from `bans.json` forgets it entirely.
Clients of listed AS numbers and countries, and clients with a poor reputation, Clients of the AS numbers and countries listed in the biased thresholds, and
get lower limits (see "Biased thresholds"), so the same rules ban them after clients listed by a reputation source whose action is `limit:<percent>`, get
fewer requests. The result is always a ban, first a temporary one and then, for lower limits (see "Biased thresholds"), so the same rules ban them after fewer
repeated abuse, a permanent one. requests. For them the result is always a ban, first a temporary one and then,
for repeated abuse, a permanent one.
Some refusals make no ban at all. `DENY_NETS`, a blocklist or reputation hit
whose action is `deny`, and the country lists (`DENIED_COUNTRIES`,
`EXCLUSIVELY_ALLOWED_COUNTRIES`) refuse each request they cover and record
nothing in `bans.json`.
## Configuration surface ## Configuration surface
Conventions: lists are comma separated; netblocks are CIDR (a bare address means Conventions: lists are comma separated; netblocks are CIDR (a bare address means
/32 or /128); durations use Go syntax plus `d` for days (`90s`, `15m`, `24h`, /32 or /128); durations use Go syntax plus `d` for days (`90s`, `15m`, `24h`,
`7d`); byte sizes accept `K`, `M`, `G` suffixes. `7d`); byte sizes accept `K`, `M`, `G` suffixes. Countries are two-letter ISO
codes in either case (`de` and `DE` are the same); a code that is not a country
code, such as `nk` (North Korea is `kp`), stops the start with a message naming
it.
- Only `UPSTREAM_URL` is required. Every other setting has a default chosen for - Only `UPSTREAM_URL` is required. Every other setting has a default chosen for
a service facing the internet in 2026, or stays off until it is given a service facing the internet in 2026, or stays off until the operator
something only the operator can supply: an alert destination, an account key, supplies or chooses what it needs: an alert destination, an account key, a
the lookup database. lookup source.
- Any limit or threshold can be switched off with the value `off`. - Any limit or threshold can be switched off with the value `off`.
- A list set to an empty value is an empty list, and replaces the default. - A list set to an empty value is an empty list, and replaces the default.
- Every variable may instead be given as `NAME_FILE` pointing at a file holding - Every variable may instead be given as `NAME_FILE` pointing at a file holding
@@ -254,17 +281,21 @@ The settings, by group:
address. A list given replaces the default; set but empty, it trusts address. A list given replaces the default; set but empty, it trusts
nothing. nothing.
- `IPV6_GROUP_PREFIX` (default `64`). - `IPV6_GROUP_PREFIX` (default `64`).
- `MAX_TRACKED_CLIENTS` (default `50000`): clients held in memory and in - `MAX_TRACKED_CLIENTS` (default `20000`): clients held in memory and in
`clients.json` (see "Persistent state"). `clients.json` (see "Persistent state").
- `MAX_BANS` (default `5000`): bans held in memory and in `bans.json`, past,
active and permanent (see "Bans").
- Persistent state - Persistent state
- `STATE_DIR` (default `/data`): the JSON state files. The image declares - `STATE_DIR` (default `/data`): the JSON state files. The image declares
`/data` as a volume, so it is writable even when nothing is mounted; a `/data` as a volume, so it is writable even when nothing is mounted; a
named volume, as in the examples below, keeps the state in a place that named volume, as in the examples below, keeps the state in a place that
outlives the container. outlives the container.
- `STATE_WRITE_DELAY` (default `2s`): after a ban changes, wait this long - `STATE_WRITE_DELAY` (default `10s`): after a ban is made, lifted or made
before writing `bans.json`, so a burst of changes becomes one write. permanent, `bans.json` is written this long later with every change made
- `STATE_COUNTER_INTERVAL` (default `60s`): how often `clients.json` and in between, so a burst of changes becomes one write.
`reputation.json` are written. - `STATE_COUNTER_INTERVAL` (default `15m`): how often `clients.json`,
`reputation.json` and `alerts.json` are written, and `bans.json` when only
the counts in its notes changed.
- Logging - Logging
- The request log on stdout is always on and has no switch. - The request log on stdout is always on and has no switch.
- `LOG_LEVEL` (default `info`): for the process's own messages (start-up, - `LOG_LEVEL` (default `info`): for the process's own messages (start-up,
@@ -319,6 +350,14 @@ The settings, by group:
send its whole request, headers and body. send its whole request, headers and body.
- `CLIENT_REQUEST_MAX_BYTES` (default `100M`): the largest request body a - `CLIENT_REQUEST_MAX_BYTES` (default `100M`): the largest request body a
client may send. client may send.
- `CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request
line and headers a client may send. Over it, the sidecar answers `431` and
closes the connection, and nothing reaches the app.
- `CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection
may wait for its next request before the sidecar closes it. It is longer
than the 90 seconds after which traefik, by default, closes a connection
it is not using, so traefik closes first and never sends a request on a
connection the sidecar is closing.
- `CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the sidecar may take - `CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the sidecar may take
to deliver one response to the client, from the end of the request to the to deliver one response to the client, from the end of the request to the
last byte. last byte.
@@ -340,27 +379,72 @@ The settings, by group:
started it can only be cut off, and the connection is closed. started it can only be cut off, and the connection is closed.
- A WebSocket connection leaves these limits behind once it is upgraded: it - A WebSocket connection leaves these limits behind once it is upgraded: it
stays open until either side closes it. stays open until either side closes it.
- Lookup of AS number and country (R7) - Lookup of AS number and country (R7). Off by default: the file needs an
- `LOOKUP_DB_PATH`: the IPinfo Lite database in its `.mmdb` form account only the operator can hold, and GeoJS is told every visitor's address.
(`ipinfo_lite.mmdb`), one file that carries both country and AS number; - `LOOKUP_SOURCE` (default `off`): `off`, `file` or `geojs`.
the sidecar reads its `asn`, `as_name` and `country_code` fields. The - `LOOKUP_DB_PATH`: the file for `file`, the IPinfo Lite database in its
operator downloads it with a free IPinfo account and mounts it read-only. `.mmdb` form (`ipinfo_lite.mmdb`), one file that carries both country and
The sidecar never fetches it and holds no account or token for it. AS number; the sidecar reads its `asn`, `as_name` and `country_code`
fields. The operator downloads it with a free IPinfo account and mounts it
read-only. The sidecar never fetches it and holds no account or token for
it. A file that is missing or unreadable at start stops the start.
- Refreshing the file is the operator's business; IPinfo updates it daily. - Refreshing the file is the operator's business; IPinfo updates it daily.
The sidecar notices when the file is replaced and reads it again; a The sidecar notices when the file is replaced and reads it again; a
replacement it cannot read is ignored, logged and alerted once, and the replacement it cannot read is ignored, logged and sent as one `file_error`
file it has stays in use. Mount the directory that holds the file rather alert, and the file it has stays in use. Mount the directory that holds
than the file itself: docker does not show a single mounted file being the file rather than the file itself: docker does not show a single
replaced on the host. mounted file being replaced on the host.
- Unset by default, which means no lookups. A setting that needs them (the - `geojs` asks the free GeoJS web service, which needs no account or key,
biased thresholds below, `ADD_LOOKUP_HEADERS`, the per-AS-number anomaly about several addresses in one request
thresholds) given without `LOOKUP_DB_PATH` stops the start with a message (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), and reads each answer's
naming both. A configured file that is missing or unreadable at start `country_code`, `asn` and `organization_name` (the AS name). A
stops the start too. `country_code` of `null`, which GeoJS gives for some ranges, and an `asn`
of `64512`, which it gives when it knows none, count as unknown. GeoJS is
told the address of every new visitor.
- Each GeoJS answer is kept for 7 days with the client's record, in memory
and in `clients.json`, so it survives a restart; after 7 days the client's
next request asks again. A client dropped from the table loses its answer.
- `LOOKUP_TIMEOUT` (default `1s`): how long a client with no kept answer
waits for one; GeoJS normally answers in a fraction of that. At most one
request to GeoJS is under way at a time, the addresses that arrive
meanwhile are asked about together in the next one, and a request that
takes longer than `LOOKUP_TIMEOUT` is abandoned. A client whose answer has
not come in time counts as unknown until it comes: its later requests do
not wait, and its address is asked about again in the background.
- GeoJS publishes no rate limit, but its terms forbid "an excessive amount
of API requests", judged by GeoJS alone, and let it block a caller. While
GeoJS is slow, down or refusing the sidecar, clients with a kept answer
are unaffected and new clients count as unknown, so
`EXCLUSIVELY_ALLOWED_COUNTRIES`, when set, refuses them. The sidecar keeps
asking with backoff and sends one `source_failure` alert per cooldown. A
service that cannot accept this uses the file.
- One source at a time: `LOOKUP_SOURCE=file` without `LOOKUP_DB_PATH`, or
`LOOKUP_DB_PATH` with any other `LOOKUP_SOURCE`, stops the start with a
message naming both. So does a setting that needs lookups while
`LOOKUP_SOURCE` is `off`: the country lists and the biased thresholds
below, `ADD_LOOKUP_HEADERS`, and the per-AS-number anomaly thresholds.
- `ADD_LOOKUP_HEADERS` (default `false`): pass `X-Client-ASN` and - `ADD_LOOKUP_HEADERS` (default `false`): pass `X-Client-ASN` and
`X-Client-Country` to the app. `X-Client-Country` to the app.
- Biased thresholds (R8). The lists are empty by default: they need the lookup - Country lists. Both are empty by default, since they need a lookup source.
database, which only the operator can supply. Clients in `ALLOW_NETS` are not checked.
- `DENIED_COUNTRIES`: for example `cn,ru,kp,ir,ua,by`. Every request from a
listed country is refused.
- `EXCLUSIVELY_ALLOWED_COUNTRIES`: for example `us,de`. Every request from
any other country is refused, and so is every request from a client the
lookup cannot place, such as an address missing from the database or one
GeoJS has not answered for in time. A list that let those through would
let every new client in whenever GeoJS stops answering.
- Both may be set. `DENIED_COUNTRIES` then adds nothing, since the exclusive
list already refuses every other country, and a code on both lists stops
the start with a message naming it.
- A refused request is answered with `BAN_RESPONSE`, as a banned client is,
as soon as the client's address has been looked up, before its body is
read. It skips the reputation checks, the limits, the rule files and the
Core Rule Set, is logged with the action `country_denied`, and is counted
in the metrics. It is a refusal, not a ban: it is not an offence and makes
no ban record.
- Biased thresholds (R8). The lists are empty by default: they need a lookup
source, which only the operator can choose.
- `ASN_LIMIT_PERCENT`: for example `AS14061:50,AS16276:50,AS45102:25`. - `ASN_LIMIT_PERCENT`: for example `AS14061:50,AS16276:50,AS45102:25`.
Clients in a listed AS number get that percentage of every request and Clients in a listed AS number get that percentage of every request and
byte limit, so the rules in "Bans" ban them after fewer requests than byte limit, so the rules in "Bans" ban them after fewer requests than
@@ -370,8 +454,9 @@ The settings, by group:
`CN:25,RU:50`. Each client from a listed country gets that percentage of `CN:25,RU:50`. Each client from a listed country gets that percentage of
the normal per-client limits. the normal per-client limits.
- No budget is shared by a whole country or AS number: one abuser could use - No budget is shared by a whole country or AS number: one abuser could use
it up and lock out everyone else there, the denial of service this tool it up and lock out everyone else there, a denial of service nobody chose
exists to prevent. and the one this tool exists to prevent. Refusing a whole country is left
to the operator's choice, through the country lists.
- When more than one percentage applies to a client (its AS number, its - When more than one percentage applies to a client (its AS number, its
country, a reputation hit), the lowest applies. country, a reputation hit), the lowest applies.
- `ASN_BYTES_PERCENT`, `COUNTRY_BYTES_PERCENT`: optional overrides applied - `ASN_BYTES_PERCENT`, `COUNTRY_BYTES_PERCENT`: optional overrides applied
@@ -406,10 +491,13 @@ The settings, by group:
example `/wp-login.php,/xmlrpc.php` in front of gitea. A request for one example `/wp-login.php,/xmlrpc.php` in front of gitea. A request for one
is a clear sign of attack. This is the env-var short form of a `path` rule is a clear sign of attack. This is the env-var short form of a `path` rule
with the `ban` action, for deployments that mount no rule files. with the `ban` action, for deployments that mount no rule files.
- `ERROR_BURST_THRESHOLD` (default `30`): responses with status 401, 403 or - `ERROR_BURST_THRESHOLD` (default `30`): responses with status 403 or 404
404 per client per minute; more than this breaks a limit (see "Bans"). per client per minute; more than this breaks a limit (see "Bans").
Scanners walking lists of paths cause hundreds a minute; a person rarely Scanners walking lists of paths cause hundreds a minute; a person rarely
causes more than a few. causes more than a few. 401 is not counted: git and container registry
clients send their first request without credentials and are answered 401
at the start of every push, every fetch from a private repository and
every image pull.
- Bans (R5), following the rules under "Bans" - Bans (R5), following the rules under "Bans"
- `ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of - `ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
attack. attack.
@@ -455,7 +543,10 @@ The settings, by group:
- `CROWDSEC_LAPI_URL`, `CROWDSEC_LAPI_KEY`: optional. If a CrowdSec engine - `CROWDSEC_LAPI_URL`, `CROWDSEC_LAPI_KEY`: optional. If a CrowdSec engine
exists on the host, pull its decision list on a schedule and treat listed exists on the host, pull its decision list on a schedule and treat listed
addresses as banned. This is how a fleet-wide blocklist can arrive without addresses as banned. This is how a fleet-wide blocklist can arrive without
the sidecar depending on CrowdSec. the sidecar depending on CrowdSec. The list is kept like a fetched
blocklist, and a listed client's request makes a ban with the cause
`crowdsec` that lasts as long as CrowdSec's decision, so a long list does
not fill `bans.json`.
- `REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:<percent>`, or - `REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:<percent>`, or
`log`, for DNSBL and API hits. Such verdicts are less certain than a `log`, for DNSBL and API hits. Such verdicts are less certain than a
blocklist, so by default a listed client gets a quarter of every limit and blocklist, so by default a listed client gets a quarter of every limit and
@@ -469,8 +560,11 @@ The settings, by group:
- `ALERT_NTFY_URL` (full topic URL), `ALERT_NTFY_TOKEN`: title, priority and - `ALERT_NTFY_URL` (full topic URL), `ALERT_NTFY_TOKEN`: title, priority and
tags set from the event type. tags set from the event type.
- `ALERT_EVENTS` (default - `ALERT_EVENTS` (default
`ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure`): `ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`):
which event types are sent. which event types are sent. `source_failure` is a reputation source or
GeoJS failing or refusing the sidecar. `file_error` is a rule file or
state file edited while running that does not parse, a replacement lookup
database that cannot be read, or a state file that cannot be written.
- `ALERT_COOLDOWN` (default `15m`): the same event type for the same client - `ALERT_COOLDOWN` (default `15m`): the same event type for the same client
or netblock is not repeated within this time; a count of suppressed or netblock is not repeated within this time; a count of suppressed
repeats is included in the next one. repeats is included in the next one.
@@ -486,7 +580,7 @@ The settings, by group:
- Per surrounding netblock (`ANOMALY_NET_V4_PREFIX` default `24`, - Per surrounding netblock (`ANOMALY_NET_V4_PREFIX` default `24`,
`ANOMALY_NET_V6_PREFIX` default `48`): `ANOMALY_NET_REQUESTS_PER_MINUTE`, `ANOMALY_NET_V6_PREFIX` default `48`): `ANOMALY_NET_REQUESTS_PER_MINUTE`,
`..._PER_HOUR`, `ANOMALY_NET_BYTES_PER_MINUTE`, `..._PER_HOUR`. `..._PER_HOUR`, `ANOMALY_NET_BYTES_PER_MINUTE`, `..._PER_HOUR`.
- Per AS number, which needs the lookup database: - Per AS number, which needs a lookup source:
`ANOMALY_ASN_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, `ANOMALY_ASN_REQUESTS_PER_MINUTE`, `..._PER_HOUR`,
`ANOMALY_ASN_BYTES_PER_MINUTE`, `..._PER_HOUR`. `ANOMALY_ASN_BYTES_PER_MINUTE`, `..._PER_HOUR`.
- Whole service: `ANOMALY_TOTAL_REQUESTS_PER_MINUTE`, `..._PER_HOUR`, - Whole service: `ANOMALY_TOTAL_REQUESTS_PER_MINUTE`, `..._PER_HOUR`,
@@ -533,7 +627,9 @@ removed there takes effect while the sidecar runs.
- `ban`: the request is a clear sign of attack. Refuse it and ban the - `ban`: the request is a clear sign of attack. Refuse it and ban the
client's netblock for seven days (`ATTACK_BAN_DURATION`); any further client's netblock for seven days (`ATTACK_BAN_DURATION`); any further
request during those days, or a later clear sign of attack, makes the ban request during those days, or a later clear sign of attack, makes the ban
permanent (see "Bans"). Use it only for requests no real visitor sends. permanent (see "Bans"). Use it only for requests no real visitor sends,
and anchor a path at the site root (`^/`): a file of the same name deeper
in a site can be ordinary content, such as a file in a gitea repository.
- `regex`: Go regular expression syntax (RE2). It has no backreferences or - `regex`: Go regular expression syntax (RE2). It has no backreferences or
lookaround, and in exchange matching time is linear in the input, so no rule lookaround, and in exchange matching time is linear in the input, so no rule
can be made to stall the proxy. `(?i)` at the front makes a rule can be made to stall the proxy. `(?i)` at the front makes a rule
@@ -545,9 +641,9 @@ removed there takes effect while the sidecar runs.
line that does not parse, a regex that does not compile, or a duplicate id line that does not parse, a regex that does not compile, or a duplicate id
stops the process with a message naming the file and line. While running, the stops the process with a message naming the file and line. While running, the
same faults leave the rules as they were, including the earlier version of same faults leave the rules as they were, including the earlier version of
that file, and the log and one alert name the file and line; once the file is that file, and the log and one `file_error` alert name the file and line; once
fixed, it is read again. A missing or empty directory is not an error: the log the file is fixed, it is read again. A missing or empty directory is not an
says that no rules were loaded. error: the log says that no rules were loaded.
- The image ships a default file in `RULES_DIR`. Mounting a directory over it - The image ships a default file in `RULES_DIR`. Mounting a directory over it
replaces the defaults; mounting single files into it adds to them. Docker does replaces the defaults; mounting single files into it adds to them. Docker does
not show a single mounted file being replaced on the host, which is how many not show a single mounted file being replaced on the host, which is how many
@@ -564,16 +660,19 @@ Example file:
# 00-default.rules: probes no real visitor sends # 00-default.rules: probes no real visitor sends
# id target action regex # id target action regex
env-file path ban (?i)/\.env(\.[a-z]+)?$ env-file path ban (?i)^/\.env(\.[a-z]+)?$
git-dir path ban ^/\.git/(config|HEAD|index)$ git-dir path ban ^/\.git/(config|HEAD|index)$
php-shell path ban (?i)/(shell|c99|r57|wso|alfa)\.php$ php-shell path ban (?i)^/(shell|c99|r57|wso|alfa)\.php$
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|masscan|zgrab|wpscan)\b scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|masscan|zgrab|wpscan)\b
path-traversal uri block (\.\./){2,} path-traversal uri block (\.\./){2,}
empty-agent user_agent log ^$ empty-agent user_agent log ^$
``` ```
The image ships one default file of this kind. It is kept short and limited to The image ships one default file of this kind. It is kept short and limited to
patterns that are wrong for every app. Anything app-specific belongs in a file patterns that are wrong for every app, and its path rules are anchored at the
site root: in front of gitea, `/.env` is a probe, while
`/<owner>/<repo>/src/branch/main/.env.example` is a file in a repository that
any visitor or search crawler may open. Anything app-specific belongs in a file
the deployer mounts: a request for `/wp-login.php`, for example, is a clear sign the deployer mounts: a request for `/wp-login.php`, for example, is a clear sign
of attack in front of gitea and an ordinary login in front of WordPress. of attack in front of gitea and an ordinary login in front of WordPress.
@@ -585,57 +684,79 @@ wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
## Persistent state ## Persistent state
All state lives in memory, and the files in `STATE_DIR` hold a copy of all of All state lives in memory, and the files in `STATE_DIR` hold a copy of all of
it, so an orderly stop loses nothing. No database is used, and nothing is read it, so an orderly stop loses nothing. The one exception is log lines still
from disk while serving a request. The files are meant for people as well: an waiting to be sent to `LOG_REMOTE_URL`, which stdout has already carried. No
admin can read or edit them at any time, and the running sidecar takes the edit database is used, and nothing is read from disk while serving a request. The
in. files are meant for people as well: an admin can read or edit them at any time,
and the running sidecar takes the edit in.
- Files, each holding one kind of state: - Files, each holding one kind of state:
- `bans.json`: every active and past ban. Per entry: the netblock, start, - `bans.json`: active and past bans, up to `MAX_BANS`. Per entry: the
expiry (`null` for permanent), what caused it (`attack`, `limit`, `admin` netblock, start, expiry (`null` for permanent), what caused it (`attack`,
or `crowdsec`), a short reason, when it was lifted if an admin lifted it, `limit`, `admin` or `crowdsec`), a short reason, when it was lifted if an
and a `notes` field (below). admin lifted it, and a `notes` field (below).
- `clients.json`: per client, its minute, hour and day counters and its - `clients.json`: per client, its minute, hour and day counters and its
history since it was first seen: first and last time seen, the AS number, history since it was first seen: first and last time seen, the AS number,
AS name and country last looked up, total requests and bytes in each AS name and country last looked up and when (with GeoJS, the answer kept
direction, how many requests were forwarded and how many refused, for 7 days), total requests and bytes in each direction, how many requests
responses by status class, and offences by kind. Clients that were never were forwarded and how many refused, responses by status class, and
banned are kept too, so every client's history survives a restart; offences by kind. Clients that were never banned are kept too, so every
`GET /clients/<ip>` shows it. client's history survives a restart; `GET /clients/<ip>` shows it.
- `reputation.json`: the last good copy of each list fetched from a URL, and - `reputation.json`: the last good copy of each list fetched from a URL,
cached DNSBL and reputation API verdicts, each with the time it was cached DNSBL and reputation API verdicts, each with the time it was
fetched. fetched, and the AbuseIPDB checks spent today, so a restart does not reset
the daily budget.
- `alerts.json`: the anomaly counters for surrounding netblocks, AS numbers,
named netblocks and the whole service; for each event type and client or
netblock, when it was last sent and how many repeats the cooldown has held
back since; the alerts sent this hour; and alerts still waiting to be
sent.
- Ban notes. The notes on a ban hold what an admin needs to decide whether to - Ban notes. The notes on a ban hold what an admin needs to decide whether to
lift it, drawn from what the sidecar already knows; nothing is looked up to lift it, drawn from what the sidecar already knows; nothing is looked up to
fill them: fill them:
- the AS number, AS name and country (empty without a lookup database); - the AS number, AS name and country (empty when lookups are off);
- what was broken: the rule ids and target that matched, or the limit, its - what was broken: the rule ids and target that matched, or the limit, its
window, the count reached and the client's limit percentage with what set window, the count reached and the client's limit percentage with what set
it; and any reputation sources that listed the client; it; and any reputation sources that listed the client;
- the requests that caused the ban, up to the last ten: time, method, host, - the requests that caused the ban, up to the last ten: time, method, host,
path, status and user agent; path with its query string, status and user agent, each text cut to 256
bytes;
- how many requests counted toward the ban, and the time span over which - how many requests counted toward the ban, and the time span over which
they came; they came;
- the netblock's total requests since it was first seen, and the requests - the netblock's total requests since it was first seen, and the requests
refused under this ban so far, kept up to date while the ban lasts; refused under this ban so far, kept up to date while the ban lasts;
- how many earlier bans of each kind the netblock has had. - how many earlier bans of each kind the netblock has had.
- Size: each tracked client takes about 2 KiB in `clients.json`, so at the - Size and disk writes. Each file is rewritten whole, so each has a bound:
default `MAX_TRACKED_CLIENTS` of 50,000 the file can reach about 100 MiB; a - `clients.json` takes about 1 KiB per client. Clients are dropped only when
small service usually tracks a few thousand clients and a few MiB. The other the table is full, so on a public service the file grows to the default
files stay small. Writing a whole file of that size takes about a second, in `MAX_TRACKED_CLIENTS` of 20,000, about 20 MiB. Written every 15 minutes,
the background, so rewriting whole files still needs nothing cleverer. that is under 2 GiB of disk writes a day.
- `bans.json` takes about 2 KiB per ban and at most about 8 KiB, since the
texts in the notes are cut short. At the default `MAX_BANS` of 5,000 it is
about 10 MiB, and never more than about 40 MiB. It is written when a ban
is made, lifted or made permanent, at most once every 10 seconds, and
otherwise with the 15-minute write, so its writes follow the bans made:
with a full file, a hundred new bans a day come to about 1 GiB of disk
writes.
- `reputation.json` and `alerts.json` are usually a few MiB or less.
- Writing a file of these sizes takes well under a second on an ordinary
disk, in the background, so rewriting whole files needs nothing cleverer.
- Format: indented JSON with a top-level `version` number, entries sorted by - Format: indented JSON with a top-level `version` number, entries sorted by
client address, times in RFC 3339 UTC, durations and sizes as plain numbers client address, times in RFC 3339 UTC, durations and sizes as plain numbers
with the unit in the field name. The aim is that a person can open `bans.json` with the unit in the field name. The aim is that a person can open `bans.json`
in an editor, find an address, and remove or add an entry. in an editor, find an address, and remove or add an entry. `clients.json` puts
each client on one line instead, which halves its size and lets `grep` show
everything about one client.
- Writing: - Writing:
- serialise from a snapshot taken under the lock, so requests are not held - serialise from a snapshot taken under the lock, so requests are not held
up while the file is written; up while the file is written;
- write to a temporary file in the same directory, sync it, rename it over - write to a temporary file in the same directory, sync it, rename it over
the real name, sync the directory. A crash at any point leaves either the the real name, sync the directory. A crash at any point leaves either the
old complete file or the new complete file, never a partial one; old complete file or the new complete file, never a partial one;
- `bans.json` is written `STATE_WRITE_DELAY` after a change; `clients.json` - `bans.json` is written `STATE_WRITE_DELAY` after a ban is made, lifted or
and `reputation.json` every `STATE_COUNTER_INTERVAL`; all three on orderly made permanent. Changes that only update the counts in its notes wait for
the write every `STATE_COUNTER_INTERVAL`, as all of `clients.json`,
`reputation.json` and `alerts.json` do. All four are written on orderly
shutdown (SIGTERM); shutdown (SIGTERM);
- before writing a file, the sidecar checks whether it changed on disk since - before writing a file, the sidecar checks whether it changed on disk since
the sidecar last read or wrote it; if it did, the sidecar takes that edit the sidecar last read or wrote it; if it did, the sidecar takes that edit
@@ -662,14 +783,17 @@ in.
- An edit that does not parse does not stop the running sidecar. It keeps - An edit that does not parse does not stop the running sidecar. It keeps
the state it has, renames the edited file to `<name>.bad` (for example the state it has, renames the edited file to `<name>.bad` (for example
`bans.json.bad`) so the edit is kept, writes the file again from memory, `bans.json.bad`) so the edit is kept, writes the file again from memory,
and logs and alerts once with the file and the position of the error. The logs the file and the position of the error, and sends them in one
admin fixes the `.bad` file and moves it back. `file_error` alert. The admin fixes the `.bad` file and moves it back.
- `STATE_DIR` not writable at start: the process exits. A write that fails while - `STATE_DIR` not writable at start: the process exits. A write that fails while
running: state stays correct in memory, the failure is logged, counted in running: state stays correct in memory, the failure is logged, counted in
metrics, alerted once per cooldown, and retried at the next write. metrics, sent as a `file_error` alert once per cooldown, and retried at the
- What a hard kill can lose: ban changes from the last `STATE_WRITE_DELAY`, and next write.
counts, history and reputation from the last `STATE_COUNTER_INTERVAL`. Losing - What a hard kill can lose: ban changes from the last `STATE_WRITE_DELAY` (10
the volume loses ban history, not service. seconds), and everything else from the last `STATE_COUNTER_INTERVAL` (15
minutes): counts and history, the counts in ban notes, GeoJS answers,
reputation verdicts and the AbuseIPDB count, anomaly counters and alert
cooldowns. Losing the volume loses ban history, not service.
## Request log ## Request log
@@ -691,13 +815,13 @@ handling while `docker logs` keeps working.
- Response detail: `response_content_type`, `upstream_status` (differs from - Response detail: `response_content_type`, `upstream_status` (differs from
`status` when the sidecar answered itself), `cache_control`, `location` on `status` when the sidecar answered itself), `cache_control`, `location` on
redirects, `aborted` when the client went away early. redirects, `aborted` when the client went away early.
- Decision: `action` (`forward`, `banned`, `denied`, `rate_limited`, - Decision: `action` (`forward`, `banned`, `denied`, `country_denied`,
`rule_blocked`, `waf_blocked`, `too_large`, `timed_out`, `upstream_error`), `rate_limited`, `rule_blocked`, `waf_blocked`, `too_large`, `timed_out`,
`would_action` in `observe` mode, `limit_percent` and which rule set it, `upstream_error`), `would_action` in `observe` mode, `limit_percent` and which
`counts` (the client's minute, hour and day request and byte totals after this rule set it, `counts` (the client's minute, hour and day request and byte
request), `limit_hit` (which window), `rule_ids` (rule file rules that totals after this request), `limit_hit` (which window), `rule_ids` (rule file
matched), `waf_rule_ids`, `waf_score`, `reputation` (sources that listed the rules that matched), `waf_rule_ids`, `waf_score`, `reputation` (sources that
client), `offence` when one was recorded, `ban_expires`. listed the client), `offence` when one was recorded, `ban_expires`.
- Timings in milliseconds: `duration_total`, `duration_checks` (everything the - Timings in milliseconds: `duration_total`, `duration_checks` (everything the
sidecar did before forwarding), `duration_waf`, `duration_upstream_connect`, sidecar did before forwarding), `duration_waf`, `duration_upstream_connect`,
`duration_upstream_first_byte`, `duration_upstream_total`. `duration_upstream_first_byte`, `duration_upstream_total`.
@@ -724,15 +848,16 @@ it can be reached by a scraper without exposing ban management.
duration and upstream duration histograms; requests in flight. duration and upstream duration histograms; requests in flight.
- Limits and bans: limit hits by window and kind (requests or bytes), size and - Limits and bans: limit hits by window and kind (requests or bytes), size and
time limit hits by limit, offences by kind, bans created by cause, permanent time limit hits by limit, offences by kind, bans created by cause, permanent
bans, active bans (gauge). bans, active bans (gauge); requests refused by the country lists, by country.
- Attack detection: rule file matches by rule id and action, and the number of - Attack detection: rule file matches by rule id and action, and the number of
rules loaded; Core Rule Set matches by mode and rule id (label limited to the rules loaded; Core Rule Set matches by mode and rule id (label limited to the
rules that actually fired). rules that actually fired).
- Lookup and reputation: requests and bytes by AS number and by country, limited - Lookup and reputation: requests and bytes by AS number and by country, limited
to the `METRICS_TOP_N` (default `50`) busiest of each with the rest summed as to the `METRICS_TOP_N` (default `50`) busiest of each with the rest summed as
`other`, so the label set stays bounded; reputation queries, hits, failures `other`, so the label set stays bounded; reputation queries, hits, failures
and remaining daily budget by source; age of each blocklist and lookup and remaining daily budget by source; GeoJS requests and failures, and clients
database. that counted as unknown because it did not answer in time; age of each
blocklist and lookup database.
- Housekeeping: tracked clients (gauge), state file writes, write failures, last - Housekeeping: tracked clients (gauge), state file writes, write failures, last
successful write time and size per file; files read again after an edit, and successful write time and size per file; files read again after an edit, and
edits set aside because they did not parse; alerts sent, failed and suppressed edits set aside because they did not parse; alerts sent, failed and suppressed
@@ -759,7 +884,8 @@ One JSON object per alert:
- `client`, `netblock`, `asn`, `as_name`, `country` - `client`, `netblock`, `asn`, `as_name`, `country`
- `reason`: short human-readable sentence - `reason`: short human-readable sentence
- `detail`: event-specific fields, for example `window`, `count`, `limit`, - `detail`: event-specific fields, for example `window`, `count`, `limit`,
`limit_percent`, `rule_ids`, `path`, `ban_expires`, and for a ban its notes `limit_percent`, `rule_ids`, `path`, `ban_expires`, for a ban its notes, and
for `file_error` the file and the position of the error
- `suppressed_repeats`: number of identical alerts held back by the cooldown - `suppressed_repeats`: number of identical alerts held back by the cooldown
## Deployment as a sidecar ## Deployment as a sidecar
@@ -808,8 +934,9 @@ networks:
- Traefik routes only port 8080. The metrics port is reached by the scraper over - Traefik routes only port 8080. The metrics port is reached by the scraper over
a docker network it shares with the sidecar; the admin port stays on loopback a docker network it shares with the sidecar; the admin port stays on loopback
inside the container and is used through `docker exec`. inside the container and is used through `docker exec`.
- The state volume holds the JSON state files, a few MiB for a small service; it - The state volume holds the JSON state files, a few tens of MiB at most with
needs no backup beyond whatever the host already does. the defaults (see "Persistent state"); it needs no backup beyond whatever the
host already does.
- SSH access to gitea does not pass through the sidecar and is not protected by - SSH access to gitea does not pass through the sidecar and is not protected by
it. it.
- Rollout per service: point traefik at the sidecar with only `UPSTREAM_URL` - Rollout per service: point traefik at the sidecar with only `UPSTREAM_URL`
@@ -819,8 +946,8 @@ networks:
(`WAF_DISABLED_RULES`, `WAF_EXEMPT_PATHS`, `RATE_LIMIT_EXEMPT_PATHS`) or an (`WAF_DISABLED_RULES`, `WAF_EXEMPT_PATHS`, `RATE_LIMIT_EXEMPT_PATHS`) or an
exemption (`RATE_LIMIT_EXEMPT_NETS`, `ALLOW_NETS`), and its ban is lifted in exemption (`RATE_LIMIT_EXEMPT_NETS`, `ALLOW_NETS`), and its ban is lifted in
`bans.json`. Additions to consider at any time: an alert destination, a lookup `bans.json`. Additions to consider at any time: an alert destination, a lookup
database with biased thresholds, reputation sources, a remote log endpoint, source with country lists or biased thresholds, reputation sources, a remote
and an app-specific rule file. log endpoint, and an app-specific rule file.
- Notes specific to gitea: - Notes specific to gitea:
- A clone is a response, and fits the defaults of 30 minutes and 5 GB for - A clone is a response, and fits the defaults of 30 minutes and 5 GB for
all but the largest repositories and slowest links. A push is a request: all but the largest repositories and slowest links. A push is a request:
@@ -841,7 +968,11 @@ networks:
- Lookup database: a configured file that is missing or unreadable at start - Lookup database: a configured file that is missing or unreadable at start
stops the start. A replacement that cannot be read while running is ignored: stops the start. A replacement that cannot be read while running is ignored:
the file already loaded stays in use, the problem is logged, one alert. the file already loaded stays in use, the problem is logged, one `file_error`
alert.
- GeoJS slow, down or refusing the sidecar: clients with a kept answer are
unaffected, new clients count as unknown, the sidecar keeps asking with
backoff, one `source_failure` alert per cooldown.
- Reputation source down or over quota: no verdict, service continues, one - Reputation source down or over quota: no verdict, service continues, one
`source_failure` alert per cooldown. `source_failure` alert per cooldown.
- Alert destination down: retried with backoff from a bounded queue, oldest - Alert destination down: retried with backoff from a bounded queue, oldest
@@ -852,16 +983,18 @@ networks:
- Remote log endpoint down: stdout continues, lines are buffered then dropped - Remote log endpoint down: stdout continues, lines are buffered then dropped
oldest first, drops counted in metrics. oldest first, drops counted in metrics.
- State file write fails while running: memory stays authoritative, logged, - State file write fails while running: memory stays authoritative, logged,
counted, alerted, retried. counted, one `file_error` alert per cooldown, retried.
- A state file or rule file edited while running that does not parse: the - A state file or rule file edited while running that does not parse: the
sidecar keeps running on what it has. The state file is set aside as sidecar keeps running on what it has. The state file is set aside as
`<name>.bad` and written again from memory; the rule file is left as it is. `<name>.bad` and written again from memory; the rule file is left as it is.
Logged, one alert. Logged, one `file_error` alert.
- In short: the only things that stop the process happen at start: invalid - In short: the only things that stop the process happen at start: invalid
configuration, a configured lookup database that is missing or unreadable, a configuration, a configured lookup database that is missing or unreadable, a
rule file or state file that does not parse, and an unwritable `STATE_DIR`. rule file or state file that does not parse, and an unwritable `STATE_DIR`.
Once running, a broken helper or a broken edit never takes the protected Once running, a broken helper or a broken edit never takes the protected
service down. service down. The nearest it comes is GeoJS failing while
`EXCLUSIVELY_ALLOWED_COUNTRIES` is set: new clients then cannot be placed, and
that list refuses them, as it refuses every client it cannot place.
## Risks the design has to handle ## Risks the design has to handle
@@ -875,8 +1008,11 @@ networks:
busy person produces, `RATE_LIMIT_EXEMPT_NETS` and `ALLOW_NETS` take known busy person produces, `RATE_LIMIT_EXEMPT_NETS` and `ALLOW_NETS` take known
shared addresses out, and the ban notes show an admin what happened before a shared addresses out, and the ban notes show an admin what happened before a
ban is lifted. ban is lifted.
- A `ban` rule that matches a real visitor bans for seven days, so the default - A `ban` rule that matches a real visitor bans it for seven days, and the
rule file keeps to requests no real visitor sends. visitor's next request during those days makes the ban permanent. So the
default rule file keeps to requests no real visitor sends, with its paths
anchored at the site root, where no app serves `.env` files or web shells. A
visitor banned by mistake is let back in by lifting the ban in `bans.json`.
- IPv6 address rotation inside a /64: handled by grouping. - IPv6 address rotation inside a /64: handled by grouping.
- Widely distributed scrapers using thousands of addresses at low rates each: - Widely distributed scrapers using thousands of addresses at low rates each:
per-client limits do not see them. The AS number and netblock anomaly alerts per-client limits do not see them. The AS number and netblock anomaly alerts
@@ -887,9 +1023,15 @@ networks:
- Core Rule Set false positives against real apps (gitea's editor, API - Core Rule Set false positives against real apps (gitea's editor, API
payloads): a match refuses only that request and bans no one by itself; the payloads): a match refuses only that request and bans no one by itself; the
request log names the rule, and exclusions by rule id and path fix it. request log names the rule, and exclusions by rule id and path fix it.
- GeoJS as the lookup source: every new visitor's address goes to a third party,
and a swarm of fresh addresses, when lookups peak, is when GeoJS may slow down
or block the sidecar. Keeping answers for 7 days and asking about many
addresses in one request keep the number of requests low; the file source has
neither risk.
- Slow-request attacks: `CLIENT_REQUEST_TIMEOUT` bounds how long a request may - Slow-request attacks: `CLIENT_REQUEST_TIMEOUT` bounds how long a request may
take to arrive, and an idle timeout on the listener closes connections that take to arrive, `CLIENT_REQUEST_HEADER_MAX_BYTES` how large its headers may
send nothing. be, and `CLIENT_IDLE_TIMEOUT` closes a kept-open connection that sends nothing
more.
- Alert floods: cooldown and hourly cap. - Alert floods: cooldown and hourly cap.
## Build order ## Build order
@@ -900,8 +1042,8 @@ networks:
`observe` mode, the full request log on stdout, the metrics endpoint, health. `observe` mode, the full request log on stdout, the metrics endpoint, health.
- Second: rule files, admin endpoints, alerting to all three destinations, - Second: rule files, admin endpoints, alerting to all three destinations,
remote log sending. remote log sending.
- Third: AS number and country lookup, biased thresholds, byte limits, anomaly - Third: AS number and country lookup from the file or GeoJS, the country lists,
thresholds. biased thresholds, byte limits, anomaly thresholds.
- Fourth: blocklists, DNSBL, AbuseIPDB, optional CrowdSec decision feed. - Fourth: blocklists, DNSBL, AbuseIPDB, optional CrowdSec decision feed.
- Fifth: attack detection with Coraza and the Core Rule Set, trap paths, error - Fifth: attack detection with Coraza and the Core Rule Set, trap paths, error
bursts. bursts.