sneak/smallwebwaf
1
0
Fork 0
Code Issues 47 Pull Requests 1 Actions 40 Packages Projects Releases Wiki Activity
Files
df8c0ebb2918be71ddf6ba14a3b37d112a135a67
smallwebwaf/README.md
T
clawbot df8c0ebb29
check / check (push) Waiting to run
Details
The Core Rule Set, run by Coraza, on each request's method, URL and headers (closes #25) 
Coraza v3.8.1 runs the Core Rule Set 4.25.0 (coraza-coreruleset v4.25.0)
after the rule files, with the six changes and the default
SWWAF_WAF_DISABLED_RULES that SPEC.md gives; no body, no response.
SWWAF_WAF_MODE, SWWAF_WAF_PARANOIA_LEVEL, SWWAF_WAF_ANOMALY_THRESHOLD and
SWWAF_WAF_EXEMPT_PATHS as specified; SWWAF_WAF_DISABLED_RULES refuses
900000 to 900999, smallwebwaf's own rules among them. A request with more
query parameters than Coraza reads, 1000, adds 5 (rule 900300). In block
mode a match is refused with 403, an offence counted toward the error
burst; in detect mode it is let through. Both log waf_rule_ids, waf_score
and duration_waf, raise waf_block, and count
smallwebwaf_waf_matches_total.

Judgement call: waf_block is raised in block mode too.
Deviation: no engine-error path; with no body read, Coraza cannot fail.

Model: opus-5-5
2026-10-08 06:57:17 +00:00

2351 lines
145 KiB
Markdown
Raw Blame History

# smallwebwaf
`smallwebwaf` is a simple, fast, logging web application firewall, MIT-licensed
and written in Go by [@sneak](https://sneak.berlin), for people who host their
own services. It runs inside the container of the one application it protects,
between your reverse proxy (traefik) and the app: the app's Dockerfile builds
`FROM` the `smallwebwaf` image, traefik sends the app's requests to
`smallwebwaf` on port 8080, and `smallwebwaf` passes them on to the app on
`127.0.0.1:8081`. It needs no setting, and protects the app from the first
request with defaults chosen for a service on the open internet. It keeps its
state in memory and in JSON files you can read and edit, and writes a detailed
JSON log line for every request.
Status: the first two milestones are built
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are nine parts of
milestone 3: the static lists, the bans that broken rate limits lead to, the ban
ledger with the bans you make, keep and lift, the JSON state files with your
edits taken in while it runs and the paths the rate limits do not count, which
come next in the build order, `observe` mode and the rest of the request log's
fields, which come a little later, and the metrics endpoint and the header size
and the idle time as settings, which come last in it. So are the four parts of
the stage after it: the rule files, with the bans for a clear sign of attack,
the other admin endpoints, alerts to all three destinations, a JSON webhook,
Slack and ntfy, and remote log sending. So is the stage after that: the AS
number and country of every client, looked up through GeoJS or in the IPinfo
Lite database file, the byte limits, the biased thresholds, lower limits for the
AS numbers and countries you list, and the anomaly thresholds, alerts for
unusual traffic that refuse nothing. So is the stage after that: the blocklists
you name by URL, which it fetches and keeps, with a file of AS numbers'
percentages fetched the same way, the DNS blocklists (DNSBL zones), which it
asks about each client in the background, AbuseIPDB, which it asks in the
background about each client that has committed an offence, and the decision
list of a CrowdSec engine you run, which it fetches and keeps as a blocklist. So
is the last stage: attack detection with the OWASP Core Rule Set, run by Coraza,
on the method, the URL and the headers of each request, the trap paths and the
error burst. `smallwebwaf` passes each request to the app and the app's answer
back, unchanged, within its timeouts and size limits, works out each client's
address, looks up its AS number and country unless you switch that off, bans a
client that sends too many requests or too many bytes, not counting those for
the paths you choose, with lower limits for the clients of the AS numbers and
countries you list, refuses a client that comes from a country you refuse or
from a network you refuse, refuses, limits or only notes a client a blocklist or
a DNSBL zone you name lists, or AbuseIPDB scores at or over the score you set,
bans a client your CrowdSec engine's decision list lists until that decision
ends, lets the networks you choose through, checks each request against the rule
files and the trap paths you name and bans a client whose request is a clear
sign of attack, refuses a request the Core Rule Set takes for an attack, bans a
client it refuses again and again within a minute for a rule, a trap path, the
Core Rule Set or a missing or wrong token, keeps its bans, each client's
counters and history, GeoJS's answers, the last good copy of each list it
fetches, the DNSBL zones' verdicts, AbuseIPDB's scores and the AbuseIPDB checks
spent today in JSON files across restarts, takes in your edits of those files,
such as a ban you make, keep or lift, and of the rule files while it runs,
writes a JSON log line for every request, sends its log lines to a syslog server
too if you name one, sends an alert to a webhook, to Slack and to ntfy, each if
you name one, for each ban it makes or makes permanent, for a request the Core
Rule Set takes for an attack, for traffic over an anomaly threshold you set, for
a client a blocklist, the CrowdSec decision list, a DNSBL zone or AbuseIPDB
lists, for GeoJS failing, a list it cannot fetch, a DNSBL zone or AbuseIPDB that
fails or refuses a query and the day's AbuseIPDB checks used up, for a rule file
or state file with an error and for a replacement of the lookup database it
cannot read, serves Prometheus metrics to a scraper that holds the metrics
token, lets an admin who holds the admin token list, add and lift bans and ask
what it knows of a client, and in `observe` mode passes on the requests it would
refuse, logging what it would have done with them. It comes as the image the
app's own image is built on. The Core Rule Set reads no request body yet:
`SWWAF_WAF_BODY_LIMIT`, which switches that on, comes with
https://git.eeqj.de/sneak/smallwebwaf/issues/116. The rest of the design comes
after that, in the order of the build order in [`SPEC.md`](SPEC.md). The survey
of existing tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
## Getting started
Build the `smallwebwaf` image from a clone:
```sh
git clone https://git.eeqj.de/sneak/smallwebwaf.git
cd smallwebwaf
make docker
```
`make docker` runs the tests and the linter, then builds the image, tagged
`smallwebwaf`, for amd64 and only on an amd64 host: the hashes the `Dockerfile`
checks Ubuntu's package lists against are those of Ubuntu's amd64 archive. Push
it to a registry your hosts pull from, and build each app's image on it, pinned
by digest, as "How it works, in short" below shows. `make example-app` builds a
small app on the image, the one in `deploy/example-app`, and checks that it
works.
To work on the code, `make build` builds the binary alone, with Go installed,
and `make run` builds and runs it, listening on port 8080 in front of an app at
`SWWAF_UPSTREAM_URL`, by default `http://127.0.0.1:8081`, with its state files
in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
`share/rules.d` unless `SWWAF_RULES_DIR` is set.
## What it does so far
- Passes each request to the app and the app's answer back unchanged: method,
path, query, headers, body and status. Bodies stream through in both
directions and are never held whole in memory. A WebSocket, or any other
upgraded connection, passes through, and the timeouts do not cut it.
- Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is
the client, and the forwarded headers it sends are replaced, not passed on.
For a peer inside it, `X-Forwarded-For` is read from the right, and the first
address outside `SWWAF_TRUSTED_PROXIES` is the client; if every address in it
is inside, the leftmost is, and with no header the peer is. The app sees what
it would see from traefik directly: the same `Host`, the same
`X-Forwarded-Proto`, and `X-Forwarded-For` with the peer added at the end. It
also gets the request's id in `X-Request-ID`, the same id as in the request's
log line (see `request_id` in "Request log" below).
- Enforces the timeouts and the size limits below. A limit passed before the
response has started gets `smallwebwaf`'s own answer: `408` for a client too
slow to send its request, `413` for a request body that is too large, `504`
for an app too slow to answer, and `502` for a response that is too large or
an app that cannot be reached. A request that announces a body over the limit
is refused before anything reaches the app. While a request body is still on
its way, a request timeout that runs out answers `408` if `smallwebwaf` was
waiting for the client to send more, and `504` if it was waiting for the app
to take what it had. Once the response has started, a limit can only cut the
connection.
- Counts each client's requests over a minute, an hour and a day. A request that
takes the client over one of the rate limits below is refused with
`SWWAF_BAN_RESPONSE`, `403` by default, before anything reaches the app, and
bans the client. A request whose path starts with one of
`SWWAF_RATE_LIMIT_EXEMPT_PATHS`, as that setting below describes, is neither
counted nor refused by the rate limits; the static lists, bans, the country
lists, the rule files and the Core Rule Set still apply to it. A client is one
IPv4 address, or one IPv6 group, the netblock of `SWWAF_IPV6_GROUP_PREFIX` its
address is in, a /64 by default, since one abuser usually holds a whole /64.
Each window is counted in two fixed buckets, the earlier one weighted by how
much of it the window still covers. At most `SWWAF_MAX_TRACKED_CLIENTS`
clients are kept, 20,000 by default, the least recently seen dropped first,
with their history, and a restart gives no client a fresh allowance (see
"State files" below).
- Counts each client's bytes over a minute, an hour and a day, in the same way:
once a request passed to the app has ended, the body bytes of its answer, of
the request, or of both, as `SWWAF_BYTES_COUNT` says. For a WebSocket, or any
other upgraded connection, what it carried from the app counts with the
answer, and what it carried from the client with the request, once it closes.
Bytes that take the client over one of the byte limits below break that limit,
and ban the client as a broken rate limit does, so that its next request is
refused. The byte limits never cut an answer or an upgraded connection short:
the one whose bytes break a limit has already been passed on, or has closed.
They leave out what the rate limits leave out: a client in `SWWAF_ALLOW_NETS`
or `SWWAF_RATE_LIMIT_EXEMPT_NETS`, and a request for a path
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` exempts.
- Gives the clients of the AS numbers and countries the biased thresholds list,
`SWWAF_ASN_LIMIT_PERCENT`, the file `SWWAF_ASN_LIMIT_PERCENT_URL` names and
`SWWAF_COUNTRY_LIMIT_PERCENT`, the percentage they give of every rate limit
and byte limit, so that the same rules ban them after fewer requests, and,
while `SWWAF_UNKNOWN_LIMIT_PERCENT` is below 100, every client without a
country that percentage. A client to which several apply gets the lowest. For
the byte limits, `SWWAF_ASN_BYTES_PERCENT` gives an AS number it lists a
percentage in place of those `SWWAF_ASN_LIMIT_PERCENT` and the file give it,
and `SWWAF_COUNTRY_BYTES_PERCENT` gives a country it lists one in place of the
one `SWWAF_COUNTRY_LIMIT_PERCENT` gives it. Each client is counted on its own,
against its own lowered limits: no budget is shared by a whole AS number or
country, which one abuser could use up and so lock out everyone else there.
The log line of each request the rate limits count gives its client's
percentages below 100 and the settings that gave them, and so do the notes of
a ban for a lowered limit, and its alert.
- Bans a client that breaks a rate limit, a byte limit or the error burst, as
"Bans" in [`SPEC.md`](SPEC.md) describes: the first ban lasts an hour, and a
limit broken again within a day of a ban ending bans for three times as long
as that ban, so 1, 3, 9, 27 and 81 hours; a ban that would last longer than
seven days is permanent instead. A ban covers the client's netblock: its IPv4
address, or the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or
its IPv6 group. While it lasts, every request from the netblock is refused
with `SWWAF_BAN_RESPONSE` after the static lists and before the country lists,
so the client is not looked up, and is not counted for the rate limits. A ban
sets the client's counters back to zero. Each ban carries notes for deciding
whether to lift it: the limit, whether it is on requests, bytes or refusals,
its window and the requests, bytes or refusals counted in it, the client's
percentage of that kind of limit and the setting that gave it when a biased
threshold lowered the limit, the request that broke it, the client's AS
number, AS name and country once they are looked up, the blocklists, the
CrowdSec decision list, DNSBL zones and AbuseIPDB, with its score, that listed
the client when the ban was made, the netblock's requests since it was first
seen, how many of them the ban has refused, and how many bans the netblock had
before, for a broken limit, for a clear sign of attack, by an admin and for
CrowdSec's decision. At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are
kept, past, active and permanent; past that, the earliest such ban of the
netblock that has gone longest without a request is dropped first. The bans
whose cause is `admin`, those you make or keep, are kept besides, and never
dropped. `bans.json` shows the bans and their notes, a restart lifts none, and
you make, keep or lift a ban by editing it (see "State files" below).
- Checks each request against the rules of the rule files (see "Rule files"
below) after the rate limits, and before its body is read. A `log` rule that
matches is noted in the log line; a `block` rule refuses the request with
`403`, and bans no one for it, though the refusal counts toward the error
burst; a `ban` rule refuses it with `SWWAF_BAN_RESPONSE` and bans the client's
netblock for a clear sign of attack. Matching stops at the first rule that
refuses. A client in `SWWAF_ALLOW_NETS` is not checked.
- Checks each request against the trap paths `SWWAF_TRAP_PATHS` names, after the
rate limits and before the rule files, which it does not need. A request for
one is refused with `SWWAF_BAN_RESPONSE` and bans the client's netblock for a
clear sign of attack, as a `ban` rule's match does. A trap path is matched
against the path a `path` rule sees: as the client sent it, before any
decoding and without the query, the whole of it, character for character.
`/wp-login.php` matches `/wp-login.php?redirect_to=x`, but not
`/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or `/%77p-login.php`.
A client in `SWWAF_ALLOW_NETS` is not checked.
- Inspects each request with the OWASP Core Rule Set 4.25.0, run by Coraza,
after the rule files, unless `SWWAF_WAF_MODE` is `off`: its method, its URL
with the query, and its headers, but no request body and no response. Each of
its rules that matches adds to the request's anomaly score, up to the paranoia
level `SWWAF_WAF_PARANOIA_LEVEL` sets. A request with more than 1000 query
parameters adds 5 (rule 900300), as a rule rated critical does, since Coraza
reads only the first 1000. A score at or over `SWWAF_WAF_ANOMALY_THRESHOLD`, 5
by default, is a match: in `block` mode, the default, the request is refused
with `403`, and in `detect` mode it goes on to the app. Either way its log
line names the rules and the score (see `waf_rule_ids` and `waf_score` in
"Request log" below), and it raises a `waf_block` alert. A refusal bans no one
by itself, since the Core Rule Set takes some ordinary requests for attacks,
but it is an offence the client's history counts, and it counts toward the
error burst; a match in `detect` mode is neither. A request a rule file
refuses, one for a path `SWWAF_WAF_EXEMPT_PATHS` exempts, and one from a
client in `SWWAF_ALLOW_NETS` are not inspected. `smallwebwaf` changes the Core
Rule Set in six ways, so that gitea's ordinary requests get through, and no
setting undoes them:
- `PUT`, `PATCH` and `DELETE` are allowed besides `GET`, `HEAD`, `POST` and
`OPTIONS`; any other method stays refused.
- The headers `Expect` and `Content-Encoding` are allowed; the others the
Core Rule Set refuses, such as `Proxy` and `Content-Range`, stay refused.
- The query parameter `redirect_uri` is not checked for a URL naming an IP
address or `localhost` (rules 931100 and 934110), which Git Credential
Manager, git-credential-oauth and tea ask to be sent back to.
- The query parameters `path`, `files`, `skip-to`, `sub_path`, `ref`, `sha`,
`branch`, `workflow`, `artifactName` and `redirect_to` are not checked
against the lists of system files (930120), shell paths (932160) and
command names (932260), so that a file such as `.gitignore` or a branch
such as `docker-build` gets through there. So does what only these rules
refuse, such as `|cat /etc/passwd`; path traversal and SQL injection are
still refused there.
- The cookies `gitea_flash` and `redirect_to` are not read, and `Referer` is
not checked for a Unix command given without arguments (932340) or for
Java starting a process (944110); it is checked by every other rule.
- Responses are not inspected.
- Bans a client for a clear sign of attack, as "Bans" in [`SPEC.md`](SPEC.md)
describes: the first such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days by
default, and any request from the netblock while it lasts makes it permanent.
Once it has run out, the netblock is served like any other, but its next clear
sign of attack bans it permanently at once. Such a ban covers the same
netblock as a ban for a broken limit, does not set the client's counters back
to zero, and does not make the netblock's next ban for a broken limit longer.
Its notes give the id and the target of the rule that matched, or the trap
path asked for, in place of the limit.
- Bans a client that `smallwebwaf` refused more than
`SWWAF_ERROR_BURST_THRESHOLD` times within a minute, 30 by default, after a
match of a `block` or `ban` rule, a trap path or the Core Rule Set, or for a
missing or wrong token at one of its own endpoints, as a broken rate limit
bans it. Each such refusal is counted once it has been answered, in two
buckets of a minute, as the rate limits count requests. The refusal that takes
the client over the threshold breaks the error burst; it is answered as any
other such refusal is, and the client's next request is refused under the ban.
The app's own answers, such as its `401` and `404`, are not counted. The
threshold is the same for every client, whatever percentage of the rate limits
a biased threshold or a reputation source gives it. A client in
`SWWAF_ALLOW_NETS` is not counted, and one in `SWWAF_RATE_LIMIT_EXEMPT_NETS`
is. The ban's notes give `refusals` as what the limit is on, the threshold as
the limit, and the refusals counted in the minute.
- Looks up the AS number and country of every client through GeoJS, or in the
IPinfo Lite database file while `SWWAF_LOOKUP_SOURCE` is `file`, after the
static lists and bans, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Country and
AS number lookup" below), for the request log, the client's history, the notes
of its bans, their alerts, the metrics and the anomaly thresholds per AS
number. The file answers at once. With GeoJS, a request waits for its client's
first answer only while a setting acts on it, a country list,
`SWWAF_ADD_LOOKUP_HEADERS` or a biased threshold that lowers a limit.
Otherwise it goes on at once, and the answer reaches the client's history and
the notes of its bans when it comes, but not the log lines of the requests
that went on without it, nor the alerts already raised for those bans.
- Refuses a request from a country you refuse with `SWWAF_BAN_RESPONSE`, as soon
as the client's country is known and before its body is read; such a request
is not counted for the rate limits. A client on a private, loopback or
link-local address has no country and is never looked up:
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless it is in
`SWWAF_ALLOW_NETS`, and `SWWAF_DENIED_COUNTRIES` does not refuse it.
- Checks the client's own address against the blocklists `SWWAF_BLOCKLIST_URLS`
names, after the country lists and before the rate limits (see "Blocklists"
below). With `SWWAF_BLOCKLIST_ACTION` at `deny`, its default, a request from a
client a blocklist lists is refused with `SWWAF_BAN_RESPONSE` before its body
is read; it is not counted for the rate limits, and makes no ban. With
`limit:<percent>` the client gets that percentage of every rate limit and byte
limit, the lowest of its percentages applying, as for a biased threshold, and
with `log` nothing more is done. Whatever the action, the request's log line
names the lists, and each raises an alert.
- Checks the client's own address against the decision list of the CrowdSec
engine `SWWAF_CROWDSEC_LAPI_URL` names, after the blocklists and before the
DNSBL zones (see "CrowdSec" below). A request from a client a decision in
force bans is refused with `SWWAF_BAN_RESPONSE` before its body is read, and
bans the client's netblock, as any ban covers it, until the decision ends,
with the cause `crowdsec`; the client's next requests are refused under that
ban. The request's log line names the list, and it raises an alert besides the
ban's. A client a blocklist refuses is not checked.
- Checks the client's own address against the DNSBL zones `SWWAF_DNSBL_ZONES`
names, after the blocklists and before the rate limits (see "DNS blocklists"
below), by the verdicts it keeps. A zone is asked about a client in the
background, and no request waits for its answer: the request that has it
asked, and any other from the client before the answer comes, goes on as from
a client the zone does not list. With `SWWAF_REPUTATION_ACTION` at `limit:25`,
its default, a client a zone's verdict lists gets a quarter of every rate
limit and byte limit, the lowest of its percentages applying, as for a biased
threshold. With `deny`, its requests are refused with `SWWAF_BAN_RESPONSE`
before their bodies are read; they are not counted for the rate limits, and
make no ban. With `log`, nothing more is done. Whatever the action, the
request's log line names the zones, and each raises an alert. A client a
blocklist refuses, or the CrowdSec decision list bans, is not checked.
- Checks the client with AbuseIPDB while `SWWAF_ABUSEIPDB_KEY` is set, after the
DNSBL zones and before the rate limits (see "AbuseIPDB" below), by the scores
it keeps. Only a client whose history counts an offence is checked, so far one
that has broken a rate limit, a byte limit or the error burst, matched a ban
rule, asked for a trap path, or had a request refused by a block rule, by the
Core Rule Set or for a missing or wrong token, and only in the background, so
that no request waits for AbuseIPDB. A score at or over
`SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and `SWWAF_REPUTATION_ACTION` does with
its client what it does with one a DNSBL zone's verdict lists. The request's
log line names AbuseIPDB, and it raises an alert. A client a blocklist or a
DNSBL zone refuses, or the CrowdSec decision list bans, is not checked.
- Checks the client's own address against the static lists, the three netblock
settings below, before anything else, its lookup included. A client in
`SWWAF_ALLOW_NETS` skips bans, the country lists, the blocklists, the CrowdSec
decision list, the DNSBL zones, AbuseIPDB, the rate limits, the byte limits,
the trap paths, the rule files, the Core Rule Set and the error burst, and is
not looked up; the timeouts and size limits still apply. A client in
`SWWAF_DENY_NETS` is refused with `SWWAF_BAN_RESPONSE` before its body is
read, and the request is not counted for the rate limits; an address in
`SWWAF_ALLOW_NETS` too is let through. A client in
`SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither counted nor refused by the rate
limits, and has no bytes counted by the byte limits; the country lists, the
trap paths, the rule files, the Core Rule Set, the error burst and bans still
apply to it.
- In `observe` mode, with `SWWAF_MODE=observe`, refuses none of the requests
that `SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec
decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit, a trap
path, a rule or the Core Rule Set would refuse: it passes them to the app, and
their log lines name what `enforce` mode would have done (see `would_action`
in "Request log" below). The checks run, and requests, bytes and refusals are
counted, as in `enforce` mode, with three differences: neither a broken rate
limit, byte limit or error burst, a `ban` rule, a trap path nor the CrowdSec
decision list makes a ban; a broken limit does not set the client's counters
back to zero, so each request over a rate limit is logged as one that would be
refused, each whose bytes keep the client over a byte limit as breaking it,
and each refusal that keeps it over the error burst as breaking that; and a
request under a ban does not make it permanent. As in `enforce` mode, the
bytes counted are only those of the requests `enforce` mode would have passed
to the app, and the refusals counted for the error burst only those it would
have made: a request it would have refused before it reached an endpoint is
not counted for its token. A ban it would have made, or made permanent, raises
the alert `enforce` mode would have raised, marked as what would have happened
(see "Alerts" below). The bans in `bans.json` are kept, and refuse requests
again when `smallwebwaf` next runs in `enforce` mode, as long as they last.
The timeouts and size limits still apply, since they protect `smallwebwaf` and
the app themselves, and a request for one of `smallwebwaf`'s own endpoints
without its token is still answered `401`. It is for trying a configuration
before enforcing it.
- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
check and without asking the app, for the image's health check.
- Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for
a request that carries `SWWAF_METRICS_TOKEN` as
`Authorization: Bearer <token>`, and with `401` for one that does not. While
the token is unset the metrics answer `404`, as does any request under
`/_smallwebwaf/` that is not for one of its endpoints. Unlike the health
check, such a request goes through every check any other request goes through,
and is answered where another would be passed to the app: a banned client
stays refused, and each counts toward the client's rate limits. None of them
reaches the app.
- Lets an admin list, add and lift bans, and ask what it knows of a client,
through the endpoints `SWWAF_ADMIN_TOKEN` opens, which go through the checks
as the metrics do (see "Admin endpoints" below).
- Writes a line in the request log for each request (see "Request log" below).
- Sends every line it writes on stdout to a syslog server as well, while
`SWWAF_LOG_REMOTE_URL` names one (see "Sending the log to a syslog server"
below).
- Sends an alert for each ban it makes or makes permanent, for a match of the
Core Rule Set, for a count over an anomaly threshold, for a client a
blocklist, the CrowdSec decision list, a DNSBL zone or AbuseIPDB lists, for
GeoJS failing, a list it cannot fetch, the CrowdSec decision list among them,
a DNSBL zone or AbuseIPDB that fails or refuses a query, and the day's
AbuseIPDB checks used up, for a rule file or state file with an error, and for
a replacement of the lookup database it cannot read, holding back repeats and,
past an hourly limit, rolling the rest into one summary, to each destination
you name: as a JSON object to the webhook `SWWAF_ALERT_WEBHOOK_URL` names, as
a message to the Slack incoming webhook `SWWAF_ALERT_SLACK_WEBHOOK_URL` names,
and as a message to the ntfy topic `SWWAF_ALERT_NTFY_URL` names (see "Alerts"
below).
- Counts requests and their bytes over a minute and an hour, per client, per
netblock around a client, per AS number, for the whole service and per named
netblock, and sends an `anomaly` alert for a count over the anomaly threshold
you set for it (see the anomaly thresholds below). These thresholds only
alert: they refuse and ban nothing. A scope whose four thresholds are all off
is not counted, and within a scope only the counts whose threshold is set are
counted. Every request but the health check is counted, whatever is done with
it: one that is refused, one from a client in `SWWAF_ALLOW_NETS` or
`SWWAF_RATE_LIMIT_EXEMPT_NETS`, and one for a path in
`SWWAF_RATE_LIMIT_EXEMPT_PATHS`, with its body bytes once it has ended, those
of the answer, of the request or both, as `SWWAF_BYTES_COUNT` says. A request
is counted for its client's AS number only if the lookup has given that by the
time the request ends: no request waits for it, and a client in
`SWWAF_ALLOW_NETS`, which is not looked up, counts for no AS number. At most
20,000 counters are kept, the one counted least recently dropped first, and
`alerts.json` keeps them across a restart (see "State files" below).
## Settings
Each setting is an environment variable, or a file one names (see "Settings
given as files" below), and each has a default, so none has to be set. A setting
that is set but invalid stops the start with a message naming it, and the
effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
`error`, which hold that line back.
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens.
- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app, as `http` or
`https`, a host and an optional port, and nothing more.
- `SWWAF_INSTANCE_NAME` (default: the host's name, which docker sets to the
first 12 characters of the container's id unless the deployment names one):
the name every log line and alert gives as `instance`, and every metric
carries as its label `instance` (see "Metrics" below). Set it, for example to
`fsn1app1/gitea`, for a name that stays the same when a deploy replaces the
container, and that tells instances apart when several log to one place. A
name that is not valid UTF-8, such as one saved in Latin-1, stops the start.
- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to pass on the
requests `smallwebwaf` would refuse and log what it would have done (see "What
it does so far" above).
- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`,
the private address ranges): the netblocks whose `X-Forwarded-For` is
believed. A list given replaces the default; set but empty, it trusts nothing.
- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the length of an IPv6 client's
group, the netblock that is one client, from 32 to 128, since a shorter one
would make one client of the customers of several providers. The rate limits,
the byte limits, bans, the table of clients, the lookups, AbuseIPDB's scores
and the anomaly thresholds per client all take an IPv6 client as its group,
and `client_group` gives it. After it changes, each IPv6 client starts afresh:
what was kept of it under its earlier group, its counts, history, GeoJS answer
and AbuseIPDB score, is not used for it, while each ban keeps refusing its
netblock until it ends.
- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept in memory
and in `clients.json`, with their counters and history, a whole number above
zero. Past it, the least recently seen is dropped first.
- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to
send its request line and headers, and then, from the end of the headers, its
body.
- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request
line and headers a client may send. Over it, the answer is `431` and nothing
reaches the app. It must be more than `4K`, and cannot be `off`: Go's HTTP
server always has such a limit, and reads 4 KiB past the one it is given
before it refuses.
- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection
may wait for its next request before `smallwebwaf` closes it. The default is
longer than the 90 seconds after which traefik closes a connection it is not
using, so traefik never sends a request on a connection `smallwebwaf` is
closing.
- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the response may
take to reach the client, from the end of the request to the last byte.
- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the
app and sending it the whole request may take.
- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take
to send its whole answer, from the end of the request to the last byte.
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body.
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body.
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip bans, the
country lists, the blocklists, the CrowdSec decision list, the DNSBL zones,
AbuseIPDB, the rate limits, the byte limits, the trap paths, the rule files
and the error burst, such as your monitoring or your own networks.
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients the
rate limits and the byte limits do not apply to, such as a machine that talks
to the app all day.
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
(default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
requests a client may make in a minute, an hour and a day. The defaults are
several times what one busy person produces, since a browser loading a heavy
page makes a few hundred requests and several people often share one address.
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes whose requests
the rate limits neither count nor refuse, and whose bytes the byte limits do
not count, such as `/assets/` for static assets; each starts with `/`. A
request whose path, percent-decoded, contains `..` anywhere or a backslash, or
whose path as sent holds an encoded slash (`%2F` or `%2f`), is never exempt,
since the app may act on it as a path outside every prefix:
`/assets/..%2Flogin` as `/login`. Any other request is exempt when its path as
sent, the path the app receives, before any query string and not
percent-decoded, starts with a prefix, character for character. `/assets/`
matches `/assets/app.js` and `/assets/`, but not `/assets`, `/Assets/app.js`,
`/%61ssets/app.js`, `/static/assets/app.js`, `/static/../assets/app.js` or
`/assets%2Fapp.js`. A character the client sends percent-encoded, such as a
space, is written percent-encoded in a prefix, as in `/my%20files/`, and there
are no wildcards: `*` is a character like any other.
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most
bytes a client may have counted in a minute, an hour and a day. A request's
bytes are counted once its answer has ended, so each default is above the
largest request body and the largest response together,
`SWWAF_REQUEST_MAX_BYTES` and `SWWAF_RESPONSE_MAX_BYTES`: at the defaults no
download breaks a limit on its own.
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
`response` for those of the answers, `request` for those of the requests, or
`both`.
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): where each client's AS number and
country are looked up: `geojs`, the GeoJS web service, which is then told the
address of every new visitor, `file`, the IPinfo Lite database file
`SWWAF_LOOKUP_DB_PATH` names, or `off`, which looks up no client and sends no
address to GeoJS. With `off`, a country list that is not empty,
`SWWAF_ADD_LOOKUP_HEADERS` set to `true`, a biased threshold that lowers a
limit, a list of them that is not empty, `SWWAF_ASN_LIMIT_PERCENT_URL` set or
`SWWAF_UNKNOWN_LIMIT_PERCENT` below 100, or an anomaly threshold per AS number
that is not `off`, stops the start, with a message naming it and
`SWWAF_LOOKUP_SOURCE`.
- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
`.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`. `file` without it, or it with
any other `SWWAF_LOOKUP_SOURCE`, the default included, stops the start, with a
message naming both.
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
client's first answer from GeoJS while a setting acts on it, and how long a
request to GeoJS may take before it is abandoned.
- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the
client's AS number, such as `AS64496`, in `X-Client-ASN`, and its country in
`X-Client-Country`, leaving out one that is unknown. A request then waits for
its client's first answer, as it does while a country list is set. Whatever
this setting says, any `X-Client-ASN` or `X-Client-Country` the client sent,
in any case, is removed, so that the app never receives a client's own.
- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused,
for example `cn,ru,kp`.
- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
countries whose clients get through, for example `us,de`. A client whose
country cannot be found is refused too, so that new clients are not let in
whenever GeoJS stops answering.
- `SWWAF_ASN_LIMIT_PERCENT` (default empty): AS numbers, each with the
percentage of every rate limit and byte limit its clients get, such as
`AS14061:50,AS16276:50,AS45102:25`. A lowered limit is rounded down to a whole
number: half of 1000 requests a minute is 500, and half of 5 is 2. `0` is a
zero allowance: the client's first request breaks a limit, and bans it.
- `SWWAF_COUNTRY_LIMIT_PERCENT` (default empty): the same by country, such as
`cn:25,ru:50`.
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty):
the same for the byte limits alone. For an AS number or a country one of them
lists, its percentage takes the place, for the byte limits, of the one
`SWWAF_ASN_LIMIT_PERCENT` or `SWWAF_COUNTRY_LIMIT_PERCENT` gives, so that
`SWWAF_ASN_LIMIT_PERCENT=AS14061:50` with
`SWWAF_ASN_BYTES_PERCENT=AS14061:100` halves that AS number's rate limits and
leaves its byte limits whole.
- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage of every limit a
client without a country gets: one the lookup cannot place, one on a private,
loopback or link-local address, which is never looked up, and one whose answer
from GeoJS has not come in time.
- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
file of AS numbers, each with its percentage of every rate limit and byte
limit, one such as `AS14061:50` to a line, whose percentages count as those of
`SWWAF_ASN_LIMIT_PERCENT` do, so that one list can serve several instances.
For an AS number both give a percentage, the lower applies, and an AS number
the file lists twice gets the lower of its two. It is fetched and kept as a
blocklist is (see "Blocklists" below). A URL `SWWAF_BLOCKLIST_URLS` names too
stops the start.
- `SWWAF_BLOCKLIST_URLS` (default empty): the blocklists, as `http` or `https`
URLs without a user or a fragment, such as
`https://www.spamhaus.org/drop/drop.txt` (see "Blocklists" below). A URL
listed twice stops the start.
- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long after a list was last
fetched, or a fetch of it failed, it is fetched again, for the blocklists and
`SWWAF_ASN_LIMIT_PERCENT_URL`. Less than `1h` stops the start: the Spamhaus
lists may be fetched no more than once an hour.
- `SWWAF_BLOCKLIST_ACTION` (default `deny`): what is done with a client a
blocklist lists: `deny` refuses its requests with `SWWAF_BAN_RESPONSE`,
`limit:<percent>`, such as `limit:25`, gives it that percentage of every rate
limit and byte limit, and `log` does nothing more than note the lists in the
log line and raise the alert.
- `SWWAF_DNSBL_ZONES` (default empty): the DNSBL zones each client is asked
about, such as `dnsbl.dronebl.org` (see "DNS blocklists" below). A zone is a
DNS name of at most 189 characters: labels of letters, digits and hyphens, of
up to 63 characters each, neither starting nor ending with a hyphen, joined by
dots. A Spamhaus zone is the name of its keyed query service, with the key in
it, such as `<key>.xbl.dq.spamhaus.net`, and is shown with its key masked (see
"DNS blocklists" below). A zone that is not such a name stops the start, as
does one listed twice, even with its letters in another case or with another
key. Do not name a zone meant for mail (see "DNS blocklists" below).
- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver the zones are asked
through, an IP address with an optional port, 53 when none is given, such as
`192.0.2.53` or `[2001:db8::53]:5353`. Unset, it is the host's, as
`/etc/resolv.conf` names it. Several zones refuse queries that come through a
public resolver.
- `SWWAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account,
which clients are checked with (see "AbuseIPDB" below). While it is unset, no
client is checked. The settings logged at start show `********` in its place.
- `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score,
from 0 to 100, that is a hit.
- `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day,
in UTC, a whole number above zero. AbuseIPDB's free accounts may make 1,000.
- `SWWAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine
you run, whose decision list is fetched (see "CrowdSec" below), as an `http`
or `https` URL without a user or a fragment, such as `http://172.17.0.1:8080`.
While it is unset, no list is fetched. Set without `SWWAF_CROWDSEC_LAPI_KEY`,
it stops the start, and so does a URL whose decision list, the URL with
`v1/decisions` added to its path, `SWWAF_BLOCKLIST_URLS` or
`SWWAF_ASN_LIMIT_PERCENT_URL` names too.
- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
`smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without
`SWWAF_CROWDSEC_LAPI_URL`, it stops the start. The settings logged at start
show `********` in its place.
- `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client a
zone's verdict lists, or whose AbuseIPDB score is a hit, as
`SWWAF_BLOCKLIST_ACTION` is for a blocklist: `deny`, `limit:<percent>` or
`log`. Such a verdict is less certain than a list such as DROP, so by default
its client gets a quarter of every rate limit and byte limit.
- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict on a
client, or AbuseIPDB's score of it, is used after it was given.
- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone, or a
check with AbuseIPDB, may take before it fails.
- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered, one
that is banned, breaks a rate limit, matches a `ban` rule, asks for a trap
path, is in `SWWAF_DENY_NETS`, comes from a refused country, is in a blocklist
while `SWWAF_BLOCKLIST_ACTION` is `deny` or is listed by a DNSBL zone or
AbuseIPDB while `SWWAF_REPUTATION_ACTION` is `deny`: `403`, `429`, or `close`
to close the connection without an answer. Behind traefik, `close` does not
leave the client unanswered: traefik answers `502`, as it does whenever its
backend drops a connection. A `block` rule always answers `403`.
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate
limit, byte limit or error burst.
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a rate limit, byte limit or
error burst broken again within this time after a ban ended, other than one
for a clear sign of attack or for CrowdSec's decision, bans for three times as
long as that ban.
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken rate limit, byte
limit or error burst that would be longer is permanent instead.
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
attack.
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
kept, past, active and permanent. The bans you make or keep are kept besides.
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
an IPv4 client that a ban covers, such as `24` to ban the surrounding /24. An
IPv6 ban covers the client's group, as `SWWAF_IPV6_GROUP_PREFIX` sets it.
- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the directory of the state
files, an absolute path. A directory `smallwebwaf` cannot write stops the
start.
- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long after a ban is made
`bans.json` is written, with every ban made in between.
- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is
written.
- `SWWAF_LOG_REQUEST_HEADERS` (default
`accept,accept-language,accept-encoding,content-type,origin,range`): the
request headers whose values the request log gives, in either case.
`Authorization`, `Cookie` and `Set-Cookie` are never logged, even when listed
(see "Request log" below). An entry naming `Host` or `Transfer-Encoding` stops
the start, since Go's HTTP server takes both out of the request; the request's
host is the field `host`.
- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own
messages that are written, on stdout and to the syslog server: `debug`,
`info`, `warn` or `error`. No message is at `debug` yet, so it writes what
`info` does. It holds back no line of the request log.
- `SWWAF_ADMIN_TOKEN` (default unset): the token an admin sends for the ban
endpoints and `/_smallwebwaf/clients/<ip>` (see "Admin endpoints" below), a
long random value. While it is unset they are off; one shorter than 32
characters stops the start. The settings logged at start show `********` in
its place. Given as a file, with `SWWAF_ADMIN_TOKEN_FILE`, it can be kept out
of the app's reach (see "Settings given as files" below).
- `SWWAF_METRICS_TOKEN` (default unset): the token a scraper sends for the
metrics, a long random value. While it is unset the metrics are off; one
shorter than 32 characters stops the start. The settings logged at start show
`********` in its place. Given as a file, it can be kept out of the app's
reach (see "Settings given as files" below).
- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many
countries get series of their own in the metrics by AS number and by country;
the others are counted as `other`.
- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the
rule files. A directory that does not exist stops the start.
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file, and checks
no request against one.
- `SWWAF_WAF_MODE` (default `block`): what the Core Rule Set does with a match:
`block` refuses it with `403`, `detect` lets it through, logged and alerted,
and `off` inspects no request (see "What it does so far" above).
- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level,
from 1 to 4. Each level up runs more of its rules, which find more attacks and
take more ordinary requests for them.
- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the anomaly score at which a
request is a match. A rule the Core Rule Set rates critical adds 5, so by
default one such rule is enough. `off` makes no request a match; the scores
are still logged.
- `SWWAF_WAF_DISABLED_RULES` (default
`920340,920420,920440,920640,930130,930140`): the ids of the Core Rule Set's
rules to switch off, such as one that refuses ordinary requests of your app;
the request log names the rules a request matched in `waf_rule_ids`. The
default switches off the rules that refuse a request for the type of its body,
when it is missing or not on the Core Rule Set's short list (920340, 920420,
920640), for its file extension, such as `.sh` or `.sql` (920440), and for a
file or directory name in its path, such as `.git/`, `.gitignore`,
`Dockerfile`, `package.json` or an editor's settings directory (930130,
930140). In front of a code forge these refuse git over HTTP, container image
and package uploads, and views of ordinary files in a repository; the default
rule file bans the common probes for such files at the site root instead (see
"Rule files" below). A list given replaces the default, so include them in it;
set but empty, it switches no rule off. An item that is not a whole number
above zero stops the start, as does one from 900000 to 900999, the ids of the
rules that set the Core Rule Set up and of `smallwebwaf`'s own, so that no
setting undoes its changes. The id of no rule switches nothing off.
- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes whose requests the
Core Rule Set does not inspect, each starting with `/`, matched as
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` matches its own: a request whose path holds
`..`, a backslash or an encoded slash is inspected whatever its prefix.
- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only
scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of gitea; a
request for one bans its client for a clear sign of attack, as a `ban` rule
does, with or without rule files (see "What it does so far" above). A path
that does not start with `/`, or holds a `?`, which no path a `path` rule sees
holds, stops the start.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most requests of a client in
a minute that `smallwebwaf` may refuse after a rule file match, a trap path or
a Core Rule Set match, or for a missing or wrong token; one more breaks the
error burst, and bans the client as a broken rate limit does (see "What it
does so far" above). A client trying one probe or token after another is
refused many times a minute, a person rarely more than a few times. `off`
switches it off.
- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server that every line on
stdout is also sent to, as `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`
with a host and a port, such as `syslog+tls://logs.example:6514`. Unset or
empty, nothing is sent.
- `SWWAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates,
which the certificate of a `syslog+tls` server must chain to instead of the
host's own. A file that cannot be read or holds no certificate stops the
start.
- `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they
wait to be sent.
- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility the lines
are sent with: `kern`, `user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`,
`news`, `uucp`, `cron`, `authpriv`, `ftp`, or `local0` to `local7`.
- `SWWAF_LOG_REMOTE_APP_NAME` (default `SWWAF_INSTANCE_NAME`): the app name the
lines are sent with, 1 to 48 printable ASCII characters without a space. While
`SWWAF_LOG_REMOTE_URL` is set, an `SWWAF_INSTANCE_NAME` that is not such a
name stops the start too, unless this setting gives one that is.
- `SWWAF_ALERT_WEBHOOK_URL` (default unset): the webhook each alert is posted
to, an `http` or `https` URL without a user or a fragment, such as
`https://alerts.example/smallwebwaf` (see "Alerts" below). Unset or empty, no
alert is sent to a webhook. Since many webhooks carry their secret in the path
or the query, the settings logged at start show `********` in place of them,
and a value that stops the start is not shown.
- `SWWAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each alert,
such as one that authenticates it, as a list of a name, `:` and a value, such
as `Authorization:Bearer 0123456789abcdef`. A value cannot hold a comma. The
settings logged at start show `********` in place of each value.
- `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook
each alert is posted to as a message, such as
`https://hooks.slack.com/services/T0123/B4567/abcdef`. Unset or empty, no
alert is sent to Slack. It is checked and logged as `SWWAF_ALERT_WEBHOOK_URL`
is.
- `SWWAF_ALERT_NTFY_URL` (default unset): the ntfy topic each alert is published
to, as the topic's full URL, such as `https://ntfy.sh/my-alerts`. Unset or
empty, no alert is sent to ntfy. It is checked and logged as
`SWWAF_ALERT_WEBHOOK_URL` is, since anyone who knows a topic on a server open
to all can read it.
- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token, sent to ntfy
with each alert as `Authorization: Bearer <token>`, for a topic that needs
one. The settings logged at start show `********` in its place. A control
character in it, such as the carriage return of a file saved with Windows line
ends, stops the start, and while `SWWAF_ALERT_NTFY_URL` is set, so does one in
`SWWAF_INSTANCE_NAME`, which ntfy is sent in the title.
- `SWWAF_ALERT_EVENTS` (default
`ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`):
the events alerts are sent for.
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held
back (see "Alerts" below).
- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour;
the rest of the hour's alerts are rolled into one summary.
- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`,
`SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`,
`SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` and
`SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR` (default `off`): the anomaly thresholds
per client, the most requests and the most bytes a client may have counted in
a minute and in an hour before an `anomaly` alert is sent for it. They refuse
and ban nothing. Each scope below has the same four thresholds, their names
ending in `REQUESTS_PER_MINUTE`, `REQUESTS_PER_HOUR`, `BYTES_PER_MINUTE` and
`BYTES_PER_HOUR`, and each is `off` by default, since what is unusual depends
on each service's normal traffic, which the metrics show.
- `SWWAF_ANOMALY_NET_REQUESTS_PER_MINUTE`, `..._PER_HOUR`,
`SWWAF_ANOMALY_NET_BYTES_PER_MINUTE` and `..._PER_HOUR` (default `off`): the
anomaly thresholds per netblock around a client, which is
`SWWAF_ANOMALY_NET_V4_PREFIX` (default `24`) long, from 0 to 32, for an IPv4
client, and `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`) long, from 0 to 128,
for an IPv6 one.
- `SWWAF_ANOMALY_ASN_REQUESTS_PER_MINUTE`, `..._PER_HOUR`,
`SWWAF_ANOMALY_ASN_BYTES_PER_MINUTE` and `..._PER_HOUR` (default `off`): the
anomaly thresholds per AS number.
- `SWWAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE`, `..._PER_HOUR`,
`SWWAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` and `..._PER_HOUR` (default `off`): the
anomaly thresholds for the whole service.
- `SWWAF_WATCH_NETS` (default empty): named netblocks, each a name, `=` and a
netblock, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item
without `=`, without a name or without a valid netblock, or a name listed
twice, stops the start. `SWWAF_WATCH_REQUESTS_PER_MINUTE`, `..._PER_HOUR`,
`SWWAF_WATCH_BYTES_PER_MINUTE` and `..._PER_HOUR` (default `off`) are the
anomaly thresholds of each named netblock as a whole, which counts every
client inside it; a client inside several is counted in each.
Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are
bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024
bytes). Rate limits, `SWWAF_ERROR_BURST_THRESHOLD` and the anomaly thresholds on
requests are whole numbers of requests, and byte limits and the anomaly
thresholds on bytes are sizes. Netblocks are in CIDR form, and a bare address
stands for itself alone. Countries are the two-letter codes ISO 3166-1 assigns
today, and `xk` for Kosovo, in either case (`de` and `DE` are the same); any
other code, such as `nk` (North Korea is `kp`) or the withdrawn `su`, stops the
start, and so does a code on both country lists. AS numbers are `AS` and the
number, in either case. Percentages are whole numbers from 0 to 100, and an
entry of a list of them is an AS number or a country, `:` and a percentage; an
AS number or a country listed twice in one of them stops the start. `off`
switches a timeout, a size limit, a rate limit, a byte limit, an anomaly
threshold, `SWWAF_WAF_ANOMALY_THRESHOLD`, `SWWAF_ERROR_BURST_THRESHOLD`,
`SWWAF_ALERT_COOLDOWN` or `SWWAF_ALERT_MAX_PER_HOUR` off;
`SWWAF_IPV6_GROUP_PREFIX`, `SWWAF_WAF_PARANOIA_LEVEL`,
`SWWAF_MAX_TRACKED_CLIENTS`, `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
`SWWAF_LOOKUP_TIMEOUT`, `SWWAF_UNKNOWN_LIMIT_PERCENT`,
`SWWAF_BLOCKLIST_REFRESH`, `SWWAF_ABUSEIPDB_MIN_SCORE`,
`SWWAF_ABUSEIPDB_DAILY_BUDGET`, `SWWAF_REPUTATION_CACHE_TTL`,
`SWWAF_REPUTATION_TIMEOUT`, the ban settings, the state settings,
`SWWAF_METRICS_TOP_N`, `SWWAF_LOG_REMOTE_BUFFER`, `SWWAF_ANOMALY_NET_V4_PREFIX`
and `SWWAF_ANOMALY_NET_V6_PREFIX` cannot be off.
Several limits are fixed rather than settings. At most 100,000 answers from
GeoJS are kept, for 7 days each, at most 20,000 anomaly counters, at most
100,000 verdicts of the DNSBL zones, with at most 1,000 queries to them under
way at once, and at most 100,000 scores of AbuseIPDB, and the CrowdSec decision
list is fetched again a minute after it was last fetched or tried.
### Settings given as files
Any setting may instead be given as a file that holds its value: the variable
named like the setting with `_FILE` added, such as `SWWAF_METRICS_TOKEN_FILE`,
names the file. `smallwebwaf` reads the file once, at start, and its health
check reads only the files of `SWWAF_LISTEN_ADDR` and `SWWAF_UPSTREAM_URL`, each
time it runs. The file's contents are the value, less one newline at their end
so that a file written with `echo` or an editor works, and are checked as the
setting's own value would be. Setting both the setting and its `_FILE` form, or
naming a file that cannot be read, stops the start with a message naming the
variable. The settings logged at start name the file, and show a token given in
one as `********`, as they show one given directly.
`SWWAF_LOG_REMOTE_TLS_CA_FILE`, whose value names a file already, has no `_FILE`
form.
The app starts with the same environment variables as `smallwebwaf`, so it can
read a token given as one. A token given as a file is out of the app's reach
only while the `smallwebwaf` user alone can read the file: make it on the host,
owned by uid 65532, the `smallwebwaf` user, with mode `0400`, and mount the
directory that holds it into the container read-only; the container sees the
same owner and mode. For example, on the host:
```sh
mkdir -p /srv/app/tokens
openssl rand -hex 32 > /srv/app/tokens/metrics
chown 65532:65532 /srv/app/tokens/metrics
chmod 0400 /srv/app/tokens/metrics
```
and for the container, `-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro` and
`-e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics`.
## Request log
`smallwebwaf` writes one JSON object per line on stdout for every request,
refused ones included:
```
{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
```
A field that does not apply to a request is left out of its line, apart from
`type`, the fields from `time` to `user_agent`, `request_id`, `peer_ip`,
`client_group`, `asn`, `as_name`, `country`, `action` and `duration_total`,
which every line has.
- `time` is when the request arrived, in UTC. `instance` is
`SWWAF_INSTANCE_NAME`. `scheme` is the `X-Forwarded-Proto` a trusted proxy
sent, and otherwise `http`. `path` and `query` are as the client sent them.
- `request_id` is the `X-Request-ID` a trusted proxy sent, or a new random one
of 26 letters and digits when it sent none, or when the peer is not a trusted
proxy. A request passed to the app takes it there in `X-Request-ID`.
- `peer_ip` is the TCP peer, normally traefik. `forwarded_for` is the
`X-Forwarded-For` header as received, several lines of it joined with `, `.
`client_group` is the client as the rate limits count it: its IPv4 address as
a /32, or its IPv6 group, as `SWWAF_IPV6_GROUP_PREFIX` sets it.
- `asn`, `as_name` and `country` are the client's AS number, such as `AS64496`,
the name of that AS, and its country, as GeoJS or the lookup database gives
them. Each is empty when `SWWAF_LOOKUP_SOURCE` is `off`, for a client in
`SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`, for a client on a private, loopback
or link-local address, when GeoJS has not answered by the time the request
went on, for a client whose address the lookup database does not hold, and for
a request whose client a ban covers, even when the answer is known. `asn` and
`as_name` are empty too when GeoJS knows no AS number for the client, which it
gives as 64512, and `country` when GeoJS cannot place the client.
- `content_type` is the request's `Content-Type`, and `content_length` the
length the request announced for its body, which is left out for none or zero.
- `request_headers` are the request's headers that `SWWAF_LOG_REQUEST_HEADERS`
names, by name in lower case, several lines of one joined with `, `.
`Authorization`, `Cookie` and `Set-Cookie` are never among them, whatever the
setting says: `has_authorization` and `has_cookie` are there instead, and
true, when the request has an `Authorization` or a `Cookie` header.
- `websocket` is there, and true, when the app switched the connection to
another protocol, as it does for a WebSocket.
- `status` is what the client was sent, `0` if nothing was; `upstream_status` is
what the app answered, and is left out when the app did not answer.
- `response_content_type`, `cache_control` and `location` are the
`Content-Type`, `Cache-Control` and `Location` headers of the answer: the
app's, as passed on, or those of `smallwebwaf`'s own answer.
- `request_bytes` and `response_bytes` count body bytes.
- `action` is `forward` for a request passed to the app, `denied` for one
refused because its client is in `SWWAF_DENY_NETS`, in a blocklist while
`SWWAF_BLOCKLIST_ACTION` is `deny`, or listed by a DNSBL zone or AbuseIPDB
while `SWWAF_REPUTATION_ACTION` is `deny`, `banned` for one refused because a
ban covers its client, or because it matched a `ban` rule, asked for a trap
path or the CrowdSec decision list lists its client, each of which bans its
client, `country_denied` for one refused for its client's country,
`rate_limited` for one that broke a rate limit and banned its client,
`rule_blocked` for one a `block` rule refused, `waf_blocked` for one the Core
Rule Set refused, `too_large` for a request or response over its size limit,
`timed_out` for one that ran out of time, `upstream_error` when the app could
not be reached or its answer broke off, and `admin` for one `smallwebwaf`
answered at its own endpoint.
- `would_action` is there in `observe` mode for a request that
`SWWAF_DENY_NETS`, a ban, the country lists, a blocklist, the CrowdSec
decision list, a DNSBL zone's verdict, AbuseIPDB's score, a rate limit, a trap
path, a rule or the Core Rule Set would have refused in `enforce` mode, and
names the action that refusal would have had: `denied`, `banned`,
`country_denied`, `rate_limited`, `rule_blocked` or `waf_blocked`. `action`
then names what was done: `forward` for a request passed to the app, and
another action, such as `too_large`, for one a size or time limit refused.
- `limit_percent` is there for a request the rate limits count whose client a
biased threshold, `SWWAF_BLOCKLIST_ACTION` for a blocklist that lists it, or
`SWWAF_REPUTATION_ACTION` for a DNSBL zone whose verdict lists it or an
AbuseIPDB score that is a hit, gives less than the whole of the rate limits,
and gives the percentage it gets, with `limit_percent_setting` naming the
setting that gave it, such as `SWWAF_ASN_LIMIT_PERCENT`, or
`SWWAF_ASN_LIMIT_PERCENT_URL` for the file it names. `bytes_percent` and
`bytes_percent_setting` are the same for the byte limits. Each is left out
when the client gets the whole of those limits.
- `counts` gives the client's requests in the minute, the hour and the day as
the rate limits count them, this request included: in each window, those in
the bucket under way and a share of those in the bucket before, so a count can
have a fraction. For a request that broke a limit, they are the counts that
broke it. It is left out for a request the rate limits do not count: the
health check, one from a client in `SWWAF_ALLOW_NETS` or
`SWWAF_RATE_LIMIT_EXEMPT_NETS`, one for a path that
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` exempts, and one that `SWWAF_DENY_NETS`, a
ban, the country lists, a blocklist, the CrowdSec decision list, a DNSBL
zone's verdict or AbuseIPDB's score refuse, or would refuse in `observe` mode.
Its `minute_bytes`, `hour_bytes` and `day_bytes` give the client's bytes in
each window as the byte limits count them, in the same way: for a request
whose bytes they count, with its own, once it has ended; for any other, those
counted before it.
- `rule_ids` is there for a request that matched rules of the rule files, and
lists their ids in the order they matched, up to the one that refused it.
- `waf_score` is there for a request the Core Rule Set inspected, and gives its
anomaly score, `0` for one no rule matched; `waf_rule_ids` lists the ids of
the rules that matched, in the order they ran, and is left out when none did.
A request is a match when its score is at or over
`SWWAF_WAF_ANOMALY_THRESHOLD`: in `block` mode its `action` is `waf_blocked`,
and in `detect` mode what it would have been otherwise, such as `forward`.
- `limit_hit` is there for a request that broke a rate limit or the error burst,
or whose bytes broke a byte limit, and names the window whose limit it went
over as `counts` names it: `minute`, `hour` or `day` for a rate limit, and
`minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit, the shortest if
it went over several; or `error_burst` for the error burst. `offence` is then
`limit`. A request whose bytes broke a byte limit is not refused: its `action`
is what it would have been otherwise, such as `forward`. Nor is one that broke
the error burst refused for that: its `action` is that of the refusal counted,
such as `rule_blocked` or `admin`.
- `reputation` is there for a request whose client a blocklist, the CrowdSec
decision list or a DNSBL zone's verdict lists, or whose AbuseIPDB score is a
hit, and gives the URLs of the blocklists that list it, in the order
`SWWAF_BLOCKLIST_URLS` names them, then the URL of the CrowdSec decision list
when a decision in force bans the client, then the zones whose verdict lists
it, in the order `SWWAF_DNSBL_ZONES` names them, then `abuseipdb`, whatever
`SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say. A zone whose
verdict on the client has not come yet, or was given
`SWWAF_REPUTATION_CACHE_TTL` ago or more, is not named, nor is AbuseIPDB for
such a score. It is left out for a client the blocklists are not checked for:
one in `SWWAF_ALLOW_NETS`, and one `SWWAF_DENY_NETS`, a ban or the country
lists refuse first, or would in `observe` mode. The CrowdSec decision list is
not checked either for a client a blocklist refuses, nor are the zones for one
a blocklist refuses or the decision list bans, nor AbuseIPDB's score for one a
blocklist or a zone refuses or the decision list bans, or would in `observe`
mode.
- `ban_expires` is there for a request that made a ban or was refused under one,
or in `observe` mode would have been refused under one, and gives when the ban
ends, in the same form as `time`, or `permanent`.
- `aborted` is there, and true, when the client went away early.
- The timings are in milliseconds, to the microsecond. `duration_total` runs
from when the request's headers had been read to when its line is written, and
`duration_checks` over the same start to when the checks were done; the health
check runs none, and its line has no `duration_checks`. `duration_waf`, the
part of the checks the Core Rule Set took, is there with `waf_score`.
`duration_upstream_connect`, `duration_upstream_first_byte` and
`duration_upstream_total` are there for a request passed to the app, and run
from when it was handed to the app: until there was a connection to it, new or
kept open from an earlier request, until the first byte of its answer arrived,
and until the end. The first two are left out when that never happened, as for
an app that cannot be reached.
No body is logged, and no header but those above. `smallwebwaf`'s own messages
(start, the settings, stop, errors) share the stream as JSON lines marked
`"type":"process"`, each with `instance` as a request's line has it, and those
less severe than `SWWAF_LOG_LEVEL` are not written.
Go's HTTP server, on which `smallwebwaf` is built, reads a request's line and
headers before `smallwebwaf` sees the request, and some requests end there,
without a line in the log: headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
which it answers `431`, headers slower than `SWWAF_CLIENT_REQUEST_TIMEOUT`,
whose connection it closes without an answer, and requests it cannot read at
all, which it answers itself, mostly with `400`.
### Sending the log to a syslog server
While `SWWAF_LOG_REMOTE_URL` is set, every line `smallwebwaf` writes on stdout,
request lines and its own, is also sent to that syslog server, as the message of
an RFC 5424 record: one record to a datagram over UDP, and over TCP and TLS each
record after its length in bytes and a space. A record gives the facility
`SWWAF_LOG_REMOTE_FACILITY` names, the severity informational, the time the line
was written, in the same form as a request line's `time`, the host's name, and
the app name `SWWAF_LOG_REMOTE_APP_NAME` gives. stdout is unchanged.
The lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines and are sent from
there, so a server that is slow or cannot be reached never holds up a request or
stdout. When the buffer is full, its oldest line is dropped to make room. A line
whose sending fails is dropped too, and the connection closed. That failure,
like a failed attempt to connect, is logged and followed by the next attempt to
connect a second later, twice as long after each further failure up to a minute,
and a second again after a connection that stayed up for a minute before it
failed. A line too long for one UDP datagram is dropped alone, with no wait and
nothing logged. UDP gives no sign of what arrives, and over TCP and TLS a line
sent on a connection the server has just closed can be lost before a failure
shows; such a loss is not counted.
As `smallwebwaf` stops, it sends the lines still waiting, on the connection open
or a new one, for at most two seconds, and gives up the rest; stdout has carried
them.
## Alerts
`smallwebwaf` sends each alert to each destination you name: to
`SWWAF_ALERT_WEBHOOK_URL` it posts the alert as one JSON object, with
`Content-Type: application/json` and the headers `SWWAF_ALERT_WEBHOOK_HEADERS`
gives, as "Alert webhook schema" in [`SPEC.md`](SPEC.md) describes, and to
`SWWAF_ALERT_SLACK_WEBHOOK_URL` and `SWWAF_ALERT_NTFY_URL` a message made from
it, as below. An alert is for one of these events, and is sent when
`SWWAF_ALERT_EVENTS` names its event:
- `ban`: a ban `smallwebwaf` makes, for a broken rate limit or byte limit, a
clear sign of attack, or a client the CrowdSec decision list lists.
- `permanent_ban`: a permanent ban it makes, or a ban for a clear sign of attack
that a request made permanent.
- `waf_block`: a request the Core Rule Set scored at or over
`SWWAF_WAF_ANOMALY_THRESHOLD`, in `block` mode, which refused it, and in
`detect` mode, which let it through, with `mode`, `detect`, in its `detail`;
in `observe` mode, `block` refused nothing either, and `mode` is `observe`.
- `anomaly`: a count of requests or bytes over an anomaly threshold, raised by
each request that ends with the count over it, in `observe` mode as in
`enforce` mode. It refuses and bans nothing.
- `reputation_hit`: a request whose client a blocklist, the CrowdSec decision
list or a DNSBL zone's verdict lists, or whose AbuseIPDB score is a hit, one
alert for each blocklist and each zone that lists it, one for the decision
list, and one for AbuseIPDB, whatever `SWWAF_BLOCKLIST_ACTION` and
`SWWAF_REPUTATION_ACTION` say, in `observe` mode as in `enforce` mode.
- `source_failure`: GeoJS failing or refusing `smallwebwaf`, a fetch of a list
failing (see "Blocklists" below), the CrowdSec decision list's among them (see
"CrowdSec" below), a query to a DNSBL zone failing or refused (see "DNS
blocklists" below), or a check with AbuseIPDB failing or refused, or the check
that uses up the day's AbuseIPDB checks (see "AbuseIPDB" below).
- `file_error`: a rule file edited while it runs that has an error, an edit of a
state file set aside as `<name>.bad`, a state file it could not write while
running, or a replacement of the lookup database it could not read, which it
does not use.
The bans you make, in `bans.json` or through the ban endpoints, raise no alert.
In `observe` mode, a request that would have made a ban, or made one permanent,
raises the alert `enforce` mode would have raised, for the ban as it would have
been, with `mode`, `observe`, in its `detail`: no ban was made, or made
permanent. A request that would have made a ban whose alert would be held back,
by the cooldown or past `SWWAF_ALERT_MAX_PER_HOUR`, raises none, and is not
counted. This is the alert for a ban for a broken rate limit, shown indented; it
is sent on one line:
```json
{
"instance": "fsn1app1/gitea",
"time": "2026-10-06T12:00:00.123461Z",
"event": "ban",
"client": "203.0.113.9",
"netblock": "203.0.113.9/32",
"asn": "",
"as_name": "",
"country": "",
"reason": "requests per minute over the limit of 1000",
"detail": {
"ban_expires": "2026-10-06T13:00:00.123Z",
"cause": "limit",
"notes": {
"asn": "",
"as_name": "",
"country": "",
"kind": "requests",
"limit": 1000,
"window": "minute",
"count": 1001,
"request": {
"time": "2026-10-06T12:00:00.123456789Z",
"method": "GET",
"host": "app.example",
"path": "/owner/repo/commits/branch/main?page=812",
"status": 403,
"user_agent": "scraper/1.0"
},
"requests": 5210,
"refused": 0,
"earlier_bans": {
"limit": 0,
"attack": 0,
"admin": 0,
"crowdsec": 0
}
}
},
"suppressed_repeats": 0
}
```
- `instance` is `SWWAF_INSTANCE_NAME`, and `time` when the alert was raised, in
UTC.
- `client` is the address of the client whose request raised the alert, and
`netblock` the netblock of the ban, or for an `anomaly`, the netblock counted:
the client's own, the netblock around it or a named netblock, and none for an
AS number or the whole service, or for a `reputation_hit` or a `waf_block`,
the client's own, as `client_group` gives it; both are empty for
`source_failure` and `file_error`. `asn`, `as_name` and `country` are, for a
ban, the client's as the ban's notes give them when the alert is raised:
empty, as in this alert, when GeoJS had not answered about the client by then;
for an `anomaly`, the client's as the lookup gave them by the time its request
ended; for a `reputation_hit` or a `waf_block`, the client's as its request's
log line gives them.
- `reason` is a short sentence; for a ban, the ban's `reason` in `bans.json`;
for an `anomaly`, what was counted over which threshold, such as
`requests per minute of the netblock 203.0.113.0/24 over the threshold of 1000`;
for a `reputation_hit`, `listed by a blocklist`,
`listed by the CrowdSec decision list`, `listed by a DNSBL zone` or
`scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE`; for a `waf_block`,
`scored by the Core Rule Set at or over SWWAF_WAF_ANOMALY_THRESHOLD`.
- `detail` is what is particular to the event: for a ban, its `cause`, when it
ends as `ban_expires`, in the form the request log gives it, and its `notes`,
as `bans.json` gives them; for an `anomaly`, the `scope`, `client`, `net`,
`asn`, `total` or `watch`, as the settings name them, the `asn` counted for
`asn` and the `name` of the named netblock for `watch`, the `window`, `minute`
or `hour`, the `kind`, `requests` or `bytes`, the `count`, which is weighted
as the rate limits weigh theirs, and the `threshold`; for a `reputation_hit`,
the `source`, the URL of the blocklist or of the CrowdSec decision list, the
zone or `abuseipdb`, and for AbuseIPDB the `score`; for a `waf_block`, the
`rule_ids` and the `score`, as the request log gives them, the request's
`method`, its `path` with the query, and the `mode` when the request was not
refused for it; for `source_failure`, the `source`, `geojs`, the URL of the
list, the zone or `abuseipdb`, the `error`, and for GeoJS, when it is asked
again, `asking_again_in`; for `file_error`, the `file`, which for an edit set
aside is the file it was renamed to, and the `error`, which for a file that
does not parse names where in it the error is.
- `suppressed_repeats` is how many repeats the cooldown held back before this
alert, and for a `summary`, those no other alert gives (see below).
Slack and ntfy are each sent the alert as a message: a title, the instance and
the event, such as `fsn1app1/gitea: ban`, and a text, the `reason`, then a line
for each of the `client`, the `netblock` and the `country`, the `file`, the
`source`, the `error` and the `mode` the `detail` gives, and the
`suppressed_repeats`, leaving out those that are empty or 0. Slack is posted, as
JSON, the title in bold and the text below it, with `&`, `<` and `>` escaped, so
that nothing in them is read as a link or a mention. ntfy is posted the text,
with the title as `Title`, `SWWAF_ALERT_NTFY_TOKEN`, while it is set, as
`Authorization: Bearer <token>`, and the priority and the tag, which ntfy shows
as an emoji, of the alert's event, as `Priority` and `Tags`:
| Event | Priority | Tag |
| ------------------------------ | --------- | -------------------------- |
| `ban` | `default` | `no_entry` |
| `permanent_ban` | `high` | `no_entry` |
| `waf_block` | `default` | `shield` |
| `anomaly` | `high` | `chart_with_upwards_trend` |
| `reputation_hit` | `low` | `label` |
| `source_failure`, `file_error` | `high` | `warning` |
| `summary` | `default` | `bar_chart` |
For the alert above, ntfy is sent these headers and this text:
```text
Title: fsn1app1/gitea: ban
Priority: default
Tags: no_entry
requests per minute over the limit of 1000
client: 203.0.113.9
netblock: 203.0.113.9/32
```
and Slack this JSON object, shown indented; it is sent on one line:
```json
{
"text": "*fsn1app1/gitea: ban*\nrequests per minute over the limit of 1000\nclient: 203.0.113.9\nnetblock: 203.0.113.9/32"
}
```
An alert for the same event as the last one sent, on the same netblock, and for
a `reputation_hit` about the same blocklist, decision list, zone or AbuseIPDB,
or for a `file_error` about the same file, or for a `source_failure` about the
same source, or for an `anomaly` in the same scope, with the same netblock, AS
number or name, whatever its window and kind, less than `SWWAF_ALERT_COOLDOWN`
after it, is a repeat: it is held back and counted, and the next alert sent for
them gives that count as `suppressed_repeats`. As each hour of the clock, in
UTC, ends, the cooldowns that have run out are dropped, and the repeats they
held back, which no alert sent since has given, go in that hour's summary.
Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the hour's other alerts are
held back and counted by event. An alert held back this way starts no cooldown.
Once the hour has ended, one alert sums up the alerts held back and the repeats
of the cooldowns dropped: its `event` is `summary`, its `reason` says how many
of each were held back, its `detail` gives the `hour` as when it started, the
`count` of alerts held back, and the count for each event, as `events`, and its
`suppressed_repeats` gives the repeats. An hour with neither ends without a
summary.
Each destination has a queue of its own, of at most 1000 alerts, from which they
are sent to it one at a time, the oldest first, so a destination that is slow or
down holds up neither the others nor any request. A destination takes an alert
by answering with a 2xx status, and refuses it with a 4xx status other than
`408` and `429`: a refused alert is logged, counted as dropped, and given up, so
that the next is sent. Any other answer, a redirect included, a connection that
fails, or no answer within 10 seconds is a failure: it is logged, naming the
destination's setting and not its URL, and the alert is sent again a second
later, twice as long after each further failure in a row, up to a minute. With
1000 alerts waiting for a destination, the oldest is dropped to make room for a
new one. The cooldowns, the hour under way and the alerts still waiting for each
destination are kept in `alerts.json` (see "State files" below), so that after a
restart the alerts waiting are sent, and the cooldowns go on.
## State files
`smallwebwaf` keeps its state in memory and a copy of it in five JSON files in
`SWWAF_STATE_DIR`, `/var/lib/smallwebwaf` by default, as "Persistent state" in
[`SPEC.md`](SPEC.md) describes. Each has a top-level `version`, 1, and lists its
entries by client address, but for the alerts waiting, the anomaly counters,
which are listed by scope first, and the copies of the lists, listed by URL,
with times in UTC.
- `bans.json`: every ban with its notes, indented to be read. A permanent ban's
`expires` is `null`. A ban's `cause` is `limit` for a broken rate limit, byte
limit or error burst, `attack` for a clear sign of attack or `crowdsec` for a
client the CrowdSec decision list lists, for a ban `smallwebwaf` made, and
`admin` for one you made or keep. Its `reason` is a short text: for a ban
`smallwebwaf` made, the limit broken, such as
`requests per minute over the limit of 1000`,
`bytes per hour over the limit of 21474836480` or
`refusals per minute over the limit of 30`, the rule that matched, such as
`matched the rule env-file`, the trap path asked for, such as
`asked for the trap path /wp-login.php`, or the scenario that made CrowdSec's
decision, such as `CrowdSec's decision for crowdsecurity/ssh-bf`; for yours,
what you wrote. Its `lifted` is when you lifted it, and is left out until you
do. The `kind` in the notes of a ban for a broken limit is `requests`, `bytes`
or `refusals`, for the error burst, what the limit is on. The notes of a ban
for a clear sign of attack give the `rule_id` and the `target` of the rule
that matched, or the `trap_path` asked for. For a limit a biased threshold
lowered, the reason and the notes' `limit` give the lowered limit, and the
notes' `limit_percent` and `limit_percent_setting` the client's percentage of
that kind of limit and the setting that gave it. The notes' `reputation` gives
each blocklist, the CrowdSec decision list, each DNSBL zone or AbuseIPDB that
listed the client when the ban was made, as its `source`, named and ordered as
in the request log's `reputation`, with AbuseIPDB's `score` of the client. It
is left out when none did, and the example below shows it.
- `clients.json`: each client's two buckets of requests in the minute, the hour
and the day, its two buckets of bytes in each, `minute_bytes`, `hour_bytes`
and `day_bytes`, its two buckets of refusals in the minute, which the error
burst counts, `minute_refusals`, and its history: when it was first and last
seen, its AS number, AS name and country as last looked up and when the lookup
gave them, its requests, how many were forwarded and how many refused (one
`smallwebwaf` answered at its own endpoints is neither, unless it was refused
with `401` for a missing or wrong token), the body bytes in each direction,
its responses by status class and its offences by kind: `limit` for a broken
rate limit, byte limit or error burst, `attack` for a clear sign of attack,
`rule_blocked` for a request a `block` rule refused, `waf_blocked` for one the
Core Rule Set refused and `token_refused` for one refused for a missing or
wrong token. Each client is on a line of its own, so `grep` shows everything
about one.
- `lookups.json`: GeoJS's answers, one to a line, each with the client's AS
number, AS name and country, when GeoJS gave it and when it was last used.
- `reputation.json`: each list fetched from a URL (see "Blocklists" and
"CrowdSec" below), indented to be read, under `lists`: its `url`, when it was
last `tried`, the fetch failed or not, and its last good copy: when that was
`fetched`, and its `lines`, as fetched, comment lines included, each on a line
of its own, both left out while no fetch of it has succeeded; and under
`verdicts`, each verdict of a DNSBL zone still in use (see "DNS blocklists"
below): its `zone`, the `client`'s address, whether the zone `listed` the
client, and when the zone gave it, `fetched`; and under `abuseipdb` (see
"AbuseIPDB" below), the `day`, in UTC, whose checks it counts, left out before
the first, the checks `spent` that day, and under `scores`, each score of
AbuseIPDB still in use: the `client`, its IPv4 address as a /32 or its IPv6
group, its `score`, and when AbuseIPDB gave it, `fetched`. As the file is
read, the lists the settings no longer name, and the verdicts of the zones
they no longer name, are dropped.
- `alerts.json`: the state of the alerts (see "Alerts" above), indented to be
read: under `cooldowns`, for each event and netblock, with the `source` too
for a `reputation_hit`, or event and `file` or `source`, or for an `anomaly`,
its `scope` with its `netblock`, `asn` or `name`, or event alone, when the
last alert was sent, `sent`, and the repeats held back since,
`suppressed_repeats`; under `hour`, the hour under way, from its `start`, the
alerts `sent` in it and those `held_back` for its summary, by event; under
`waiting`, for each destination you name, `webhook`, `slack` or `ntfy`, the
alerts still waiting to be sent to it, the oldest first, each as the webhook
is sent it; and under `anomaly_counters`, each anomaly counter: its `scope`,
as an `anomaly` alert names it, with the `netblock` of a client, of a netblock
around a client or of a named netblock, the `asn` of an AS number and the
`name` of a named netblock, and its two buckets of requests in the minute and
the hour, `minute` and `hour`, and of bytes, `minute_bytes` and `hour_bytes`,
each left out while it is empty. As an hour ends, the cooldowns that have run
out are dropped, and the hour's summary gives the repeats they held back. As
the file is read, the alerts waiting for a destination you no longer name are
dropped. A file whose `waiting` is a list, as it was before alerts went to
Slack and ntfy too, stops the start: put the list under `"webhook"`, or remove
the file.
This `bans.json` holds a ban for a broken rate limit on a client that a DNSBL
zone lists and AbuseIPDB scores at 100, whose limits `SWWAF_REPUTATION_ACTION`,
at its default of `limit:25`, lowered to a quarter:
```json
{
"version": 1,
"bans": [
{
"netblock": "203.0.113.9/32",
"start": "2026-10-06T12:00:41.5Z",
"expires": "2026-10-06T13:00:41.5Z",
"cause": "limit",
"reason": "requests per minute over the limit of 250",
"notes": {
"asn": "AS64496",
"as_name": "Example Net",
"country": "DE",
"kind": "requests",
"limit": 250,
"window": "minute",
"count": 251,
"limit_percent": 25,
"limit_percent_setting": "SWWAF_REPUTATION_ACTION",
"reputation": [
{
"source": "dnsbl.dronebl.org"
},
{
"source": "abuseipdb",
"score": 100
}
],
"request": {
"time": "2026-10-06T12:00:41.5Z",
"method": "GET",
"host": "app.example",
"path": "/owner/repo/commits/branch/main?page=812",
"status": 403,
"user_agent": "scraper/1.0"
},
"requests": 512,
"refused": 0,
"earlier_bans": {
"limit": 0,
"attack": 0,
"admin": 0,
"crowdsec": 0
}
}
}
]
}
```
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted
through `DELETE /_smallwebwaf/bans/<client>`, or made permanent, with every such
change in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and when
`smallwebwaf` stops. Each write goes to a temporary file in the same directory,
which then replaces the file, so a crash leaves the old file or the new one,
whole. A write that fails is logged, raised as a `file_error` alert while
`smallwebwaf` runs, and tried again at the next write. A hard kill loses what
changed since the last write.
At start the files are read back: each client keeps its counts, so a restart
gives it no fresh allowance, each anomaly counter keeps its counts, each ban
keeps refusing every client in its netblock until it ends, even after
`SWWAF_BAN_SCOPE_V4_PREFIX` has changed, and the copy of each list stays in use
until a fetch of it succeeds. A netblock whose address has bits past its length,
such as `203.0.113.9/24`, is read as the netblock it is in, `203.0.113.0/24`.
Buckets and answers whose time has passed are dropped, and so is an anomaly
counter left with no bucket; a verdict past its time is neither used nor written
again. A missing file is empty state, as on a first start. A file that does not
parse, or has another `version`, stops the start with a message naming the file,
and the line and column where Go's JSON decoder gives them; so does a state
directory `smallwebwaf` cannot write. So does an entry without a field it needs,
named with the entry's place in the file: a ban's `netblock`, `start` or
`expires`, which is `null` for a permanent ban; a client's `client`, or the
`start` of a window in which it has requests, bytes or refusals; an answer's
`client`, `country`, which is `""` for a client GeoJS cannot place, or
`answered`; a cooldown's `event` or `sent`; an alert waiting's `event` or
`time`; an anomaly counter's `netblock`, unless it counts an AS number or the
whole service, its `asn`, for an AS number, its `name`, for a named netblock, or
the `start` of a window in which it has requests or bytes; a list's `url`,
`fetched` or `lines`, which is `[]` for an empty list; a verdict's `zone`,
`client`, `listed`, which is `false` for a client the zone does not list, or
`fetched`. So does a ban whose `cause` is not `limit`, `attack`, `admin` or
`crowdsec`, alerts waiting for a destination that is not `webhook`, `slack` or
`ntfy`, an anomaly counter whose `scope` is not `client`, `net`, `asn`, `total`
or `watch`, and a copy of a list with a line that would make its fetch fail. An
answer's `asn` or `as_name` left out reads as empty.
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
a state file as soon as you save it: what the file then holds replaces what
`smallwebwaf` held for it, as if read at start. It tells its own writes from
yours by comparing the file with what it last read or wrote, and before it
writes a file it takes in any edit made since, so your edit is not overwritten;
a change `smallwebwaf` made after you opened the file, such as a new ban, is
lost when you save over it. An edit that would stop the start, because it does
not parse, has another `version`, leaves out a field an entry needs, gives a ban
another `cause`, names another destination, gives an anomaly counter another
`scope` or gives a list's copy a line that would make its fetch fail, does not
stop the running `smallwebwaf`: it keeps what it holds, and at the file's next
write renames your file to `<name>.bad`, such as `bans.json.bad`, writes the
file again from memory, logs the file and where the error is, and raises a
`file_error` alert for it. It waits for that write because an editor's file can
be read before the editor has finished writing it. Mend the `.bad` file and move
it back. A file you remove is written again at its next write.
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
and its `expires`, `null` for a ban that never ends; its `reason` and its
`notes` may be left out, and so may its `cause`, which is then `admin`, and is
written so at the file's next write. A ban whose `cause` is `admin` is never
dropped and does not count toward `SWWAF_MAX_BANS`. A ban whose `cause` is
`attack` becomes permanent at the first request it refuses; one whose `cause` is
`admin` or `crowdsec` does not. This `bans.json` bans `203.0.113.0/24` for good:
```json
{
"version": 1,
"bans": [
{
"netblock": "203.0.113.0/24",
"start": "2026-10-06T12:00:00Z",
"expires": null,
"reason": "probes for logins"
}
]
}
```
To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause`
to `admin`: `"cause": "admin"`.
To lift a ban, add `lifted` to its entry, with the time you lift it, such as
`"lifted": "2026-10-06T13:00:00Z"`. From when the edit is taken in, the ban
refuses nothing, whatever time `lifted` gives, and does not make the netblock's
next ban longer; it is kept in `bans.json` with its notes, as any other ban is.
To forget a ban altogether, delete its entry: it then refuses nothing either,
and does not make the netblock's next ban longer.
The ban endpoints add and lift bans without an edit of the file (see "Admin
endpoints" below).
## Rule files
`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`,
`/etc/smallwebwaf/rules.d` by default, in the order of their names, and checks
each request against their rules in that order, as "Rule files" in
[`SPEC.md`](SPEC.md) describes. A file whose name starts with `.`, such as an
editor's lock file `.#50-app.rules`, is not a rule file, as a shell's `*.rules`
would not match it. A rule is a line of four fields separated by spaces or tabs:
an id, a target, an action and a regex, which runs to the end of the line.
Spaces and tabs at the end of a line are not part of its regex, so a line with
only those after its action has no regex, and is not a rule. Blank lines and
lines that start with `#` are ignored.
```
# id target action regex
env-file path ban (?i)^/\.env(\.[a-z]+)?$
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
```
- The id is letters, digits, `-` and `_`, and no two rules share one. The
request log, the metrics and a ban's notes name the rule by it.
- The target is what the regex is matched against: `path` or `query`, as the
client sent it, before any decoding; `uri`, the path and the query together,
both as sent and once percent-decoded, so that an encoded probe does not slip
past; `method`; `host`; `user_agent`; `referer`; or `header:<Name>`, any one
request header but `Host` and `Transfer-Encoding`, which Go's HTTP server
takes out of every request; the request's host is the target `host`. A header
sent more than once is matched with its values joined by `, `, and one not
sent as empty text. No body is read.
- The action is `log`, `block` or `ban` (see "What it does so far" above). Keep
`ban` for requests no real visitor sends, and anchor a path at the site root
with `^/`: a file of the same name deeper in a site can be ordinary content,
such as a file in a repository on a code forge.
- The regex is in Go's syntax, RE2, which has no backreferences or lookaround,
and takes time linear in the text it reads. It matches anywhere in the target
unless anchored with `^` and `$`; `(?i)` at its front makes it ignore case.
A line that is not a rule, a `header:` name with a character no header name can
have, such as `header:User-Agent:`, a rule for the `Host` or the
`Transfer-Encoding` header, a regex that does not compile or an id used twice
stops the start with a message naming the file and the line, and so does a
`SWWAF_RULES_DIR` that does not exist. An empty directory is no error, and the
log says that it holds no rules. While it runs, `smallwebwaf` watches the
directory, and reads the rule files again once the directory has had no change
for 2 seconds after one is edited, added or removed, so that a file saved in
place, appended to or copied in with `scp` is read only once whole, unless its
writing stops for longer. It also reads them 2 seconds after it starts watching,
so that an edit saved while it started is not missed. If they then hold one of
those errors, the rules stay as they were, the earlier version of the edited
file included, the log and a `file_error` alert name the file and the line, and
the files are read again after the next change.
The image ships one rule file, `share/rules.d/00-default.rules` here: rules that
ban probes no real visitor sends, for secrets, version control directories,
backups, logs and web shells at the site root, and the user agents of common
scanners; one that blocks `../` twice in a row in the path or the query; and one
that only notes a request without a user agent. An app's Dockerfile adds rules
of its own in a file beside it, named to sort after it, such as this
`50-gitea.rules` for an app that serves no WordPress:
```
wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
```
```dockerfile
COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules
```
A directory mounted over `/etc/smallwebwaf/rules.d` replaces the default file,
and single files mounted into it add to it. Docker does not show a single
mounted file being replaced, which is how many editors save, so rules to be
edited while `smallwebwaf` runs belong in a mounted directory, with a copy of
`00-default.rules` if its rules are to stay. To run without rules, mount an
empty directory or set `SWWAF_RULES_ENABLED=false`.
## Metrics
`GET /_smallwebwaf/metrics` answers with the metrics in the Prometheus text
format, for a scraper that sends `SWWAF_METRICS_TOKEN`, through traefik like any
other request. No metric carries a client's address.
Every metric below, Go's and the process's included, carries the label
`instance`, `SWWAF_INSTANCE_NAME`, as a constant label set once on the registry
the metrics are kept in, rather than as a label each metric declares. Prometheus
gives each series it scrapes an `instance` label of its own, the address it
scraped, and keeps this one as `exported_instance` unless the scrape sets
`honor_labels: true`.
- `smallwebwaf_requests_total`, `smallwebwaf_request_bytes_total` and
`smallwebwaf_response_bytes_total`: requests, and their body bytes each way,
by `status_class`, such as `2xx`, or `none` when nothing was sent, and by
`action`, as the request log names it.
- `smallwebwaf_request_duration_seconds`: how long requests took, and
`smallwebwaf_upstream_duration_seconds`: how long those passed to the app took
from then on, as histograms; `smallwebwaf_requests_in_flight`: the requests
under way.
- `smallwebwaf_rate_limit_hits_total` by `window`, `minute`, `hour` or `day`,
and `kind`, `requests` for a rate limit, `bytes` for a byte limit or
`refusals` for the error burst, whose window is `minute`,
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
limit was passed, `smallwebwaf_offences_total` by `kind`, `limit`, `attack`,
`rule_blocked`, `waf_blocked` or `token_refused`, as `clients.json` counts
them, and `smallwebwaf_bans_made_total` by `cause`, `limit`, `attack`, `admin`
or `crowdsec`, `admin` for the bans you add through `POST /_smallwebwaf/bans`,
and those whose `cause` is `admin` that you add to `bans.json` while
`smallwebwaf` runs; `smallwebwaf_active_bans` and
`smallwebwaf_permanent_bans`, neither of which counts a lifted ban.
- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
`rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
rules read from the rule files.
- `smallwebwaf_waf_matches_total`: the requests that matched each rule of the
Core Rule Set, whatever their score, by `mode`, `SWWAF_WAF_MODE`, and
`rule_id`, a series for each rule that has matched.
- `smallwebwaf_country_requests_total`,
`smallwebwaf_country_request_bytes_total`,
`smallwebwaf_country_response_bytes_total`, and
`smallwebwaf_country_list_refusals_total`, the requests the country lists
refused, by `country`, for the requests whose client's country is known. The
`SWWAF_METRICS_TOP_N` countries with the most requests since the start have
series of their own, and the others are counted as `other`. A country that
drops out of them loses its series, and its later requests count as `other`;
one that comes into them gets a series that counts from then on.
- `smallwebwaf_asn_requests_total`, `smallwebwaf_asn_request_bytes_total` and
`smallwebwaf_asn_response_bytes_total`, by `asn`, the client's AS number, for
the requests whose client's AS number is known, with the `SWWAF_METRICS_TOP_N`
busiest AS numbers kept as the countries are.
- `smallwebwaf_geojs_requests_total`: the requests to GeoJS;
`smallwebwaf_geojs_failures_total`: those that failed, an answer that leaves
out an address asked about included; and `smallwebwaf_geojs_unanswered_total`:
the requests that needed their client's answer, for a country list,
`SWWAF_ADD_LOOKUP_HEADERS` or a biased threshold, and went on without it
because GeoJS had not given it in time.
- While `SWWAF_LOOKUP_SOURCE` is `file`,
`smallwebwaf_lookup_database_last_read_timestamp_seconds`: when the lookup
database in use was read; and
`smallwebwaf_lookup_database_read_failures_total`: the replacements of it that
could not be read.
- By `source`, the URL of each list `SWWAF_BLOCKLIST_URLS` or
`SWWAF_ASN_LIMIT_PERCENT_URL` names, and of the CrowdSec decision list:
`smallwebwaf_reputation_hits_total`: the requests whose client the blocklist
or the decision list lists, a series that comes with the first;
`smallwebwaf_reputation_failures_total`: the fetches of the list that failed;
and `smallwebwaf_reputation_last_fetch_timestamp_seconds`: when the copy of it
in use was fetched, `0` while there is none.
- By `source`, each zone `SWWAF_DNSBL_ZONES` names:
`smallwebwaf_reputation_hits_total`: the requests whose client the zone's
verdict lists, a series that comes with the first;
`smallwebwaf_reputation_queries_total`: the queries made to the zone; and
`smallwebwaf_reputation_failures_total`: those that failed.
- While `SWWAF_ABUSEIPDB_KEY` is set, by `source`, `abuseipdb`:
`smallwebwaf_reputation_hits_total`: the requests whose client's AbuseIPDB
score is a hit, a series that comes with the first;
`smallwebwaf_reputation_queries_total`: the checks made, each of which spends
one of the day's budget; `smallwebwaf_reputation_failures_total`: those that
failed; and `smallwebwaf_reputation_daily_budget_remaining`: the checks the
day's `SWWAF_ABUSEIPDB_DAILY_BUDGET` has left.
- `smallwebwaf_tracked_clients`: the clients in the table of clients.
- `smallwebwaf_state_file_writes_total`,
`smallwebwaf_state_file_write_failures_total`,
`smallwebwaf_state_file_last_write_timestamp_seconds` and
`smallwebwaf_state_file_size_bytes`, by `file`; and, by `file` too,
`smallwebwaf_state_file_edits_taken_in_total`: your edits taken in, and
`smallwebwaf_state_file_edits_set_aside_total`: those renamed to `<name>.bad`
because they would stop the start.
- While `SWWAF_LOG_REMOTE_URL` is set,
`smallwebwaf_remote_log_lines_sent_total`: the lines sent to it;
`smallwebwaf_remote_log_lines_dropped_total`: those dropped, from a full
buffer or because their sending failed; and
`smallwebwaf_remote_log_buffer_depth`: those waiting in the buffer.
- For each destination you name, by `destination`, `webhook`, `slack` or `ntfy`:
`smallwebwaf_alerts_sent_total`: the alerts it took;
`smallwebwaf_alerts_failed_total`: the requests to it that failed;
`smallwebwaf_alerts_suppressed_total`: the alerts held back, as repeats or for
an hour's summary, which are the same for every destination; and
`smallwebwaf_alerts_dropped_total`: those dropped from its full queue, or
given up as it refused them.
- Go's own `go_` metrics and the process's `process_` metrics.
The requests Go's HTTP server ends before `smallwebwaf` sees them (see "Request
log") are not counted.
## Admin endpoints
While `SWWAF_ADMIN_TOKEN` is set, `smallwebwaf` answers these requests itself,
on the app's own address and through traefik like any other request, for a
request that carries the token as `Authorization: Bearer <token>`:
- `GET /_smallwebwaf/bans`: every ban held, past, active and permanent.
- `POST /_smallwebwaf/bans`: bans a netblock, as adding an entry to `bans.json`
does. The body is a JSON object of `netblock`, `duration` and, if you like,
`reason`. `netblock` is a netblock such as `203.0.113.0/24`, or a client's
address, which bans the netblock a ban on that client covers: its IPv4
address, or the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or
its IPv6 group. `duration` is a duration such as `1h` or `7d`, or `permanent`.
The ban starts at once, its `cause` is `admin`, and it is made even while
another ban on the netblock lasts. A body that is not such an object, has
another field, has anything but whitespace after the object, or is longer than
4 KiB is answered `400`, saying what is wrong, and so is an IPv4-mapped
netblock, such as `::ffff:203.0.113.0/120`, or a value with a zone, such as
`fe80::1%eth0`.
- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban on a netblock
that `<client>`, an address, is in, as adding `lifted` to its entry in
`bans.json` does, and answers `404` when no ban on it is active.
- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client at
the address `<ip>`: under `client`, the client as `clients.json` holds it,
with its counters and its history, which holds its AS number, AS name and
country as last looked up and its offences, or `null` when the table of
clients does not hold it; and under `bans`, every ban on a netblock the
address is in, with its notes.
The ban endpoints answer with the bans listed, made or lifted, under `bans`,
each as an entry of `bans.json` (see "State files" above), and a ban they make
or lift is written to `bans.json` `SWWAF_STATE_WRITE_DELAY` later. Refusals, and
the answers to requests that cannot be read, are plain text.
A request without the token, or with another, such as the metrics token, is
answered `401`, in `observe` mode too, and counts toward the error burst, as one
for the metrics without theirs does. While the token is unset, each of these
answers `404`, as does any request under `/_smallwebwaf/` that is not for one of
its endpoints. Like the metrics, these requests go through every check any other
request goes through, and are answered where another would be passed to the app:
a banned client stays refused, so an admin whose own address is banned lifts
that ban by editing `bans.json`, and each request counts toward the client's
rate limits. A client in `SWWAF_ALLOW_NETS` skips the checks, and still needs
the token.
With the token in `$TOKEN`, for an app at `https://app.example`:
```sh
# Every ban.
curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans
# Ban 203.0.113.0/24 for seven days.
curl -H "Authorization: Bearer $TOKEN" \
--json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \
https://app.example/_smallwebwaf/bans
# Lift the bans on 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" -X DELETE \
https://app.example/_smallwebwaf/bans/203.0.113.9
# What smallwebwaf knows of 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" \
https://app.example/_smallwebwaf/clients/203.0.113.9
```
## Why
Small self-hosted sites now receive a great deal of traffic nobody asked for:
scrapers that ignore `robots.txt` and crawl every commit of every repository on
a public git server, vulnerability scanners walking through lists of WordPress
and `.env` paths, and credential-guessing bots. Most of it comes from a small
number of hosting networks and countries. A single-person operation has no abuse
desk and no CDN contract; it needs something small that can be put in front of
one service and left alone.
The existing tools each solve part of this. Rule-based firewalls catch attack
payloads but do not limit request rates. Rate limiters count requests but cannot
tell a residential visitor from a rented server farm. The products that do most
of it want several containers, a database and a web console. None of them can
say "clients from these networks are banned after half as many requests as
anyone else", which is the most useful thing to be able to say when nearly all
abuse comes from a known list of AS numbers. [`EVALUATION.md`](EVALUATION.md)
goes through the candidates one by one.
`smallwebwaf` is meant to fill that gap:
- protect a service from misbehaving scrapers and scanners with per-client
request and byte limits over a minute, an hour and a day;
- lower those limits for the countries and AS numbers that abuse commonly comes
from, so their clients are banned after fewer requests than others;
- ban abusers: briefly at first, longer each time they come back, and
permanently when they keep at it; a scanner's first probe bans it for seven
days;
- log everything in a form that is easy to search and ship elsewhere;
- stay small enough to understand: one binary in the app's own container,
environment variables, no database, no required setting.
## Proposed features
- Reverse proxy for one application, streaming in both directions, with
WebSocket support. One `smallwebwaf` per app, inside the app's own container.
- Internet-ready out of the box: no setting is required, and every setting has a
default chosen for a service facing the internet in 2026. Every setting's name
starts with `SWWAF_`, so that it cannot clash with the app's own.
- Real client address worked out from `X-Forwarded-For`, trusting only the proxy
networks you list, by default the private address ranges. IPv6 clients are
counted by /64 by default.
- Size and time limits on requests and responses, with the time limits both
between the client and `smallwebwaf` and between `smallwebwaf` and the app: by
default a request may take 60 seconds and 100 MiB, a response 30 minutes and 5
GiB.
- Rate limits per client on requests per minute, per hour and per day, and on
bytes per minute, per hour and per day, on by default and set well above what
real visitors need.
- Netblocks that bypass rate limiting, netblocks that bypass everything, and
netblocks that are always refused.
- AS number and country lookup for every client, on by default through the free
GeoJS web service, which is sent the address of every new visitor. The IPinfo
Lite database file, which you download and mount, can be used instead, or
lookups switched off (see "Country and AS number lookup" below).
- Country lists: `SWWAF_DENIED_COUNTRIES` refuses every request from the
countries listed, `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` every request from
anywhere else. Such a request gets the answer a banned client gets as soon as
the client's address has been looked up, before its body is read and without
the rule files or the Core Rule Set looking at it, and no ban is made.
- Biased limits: listed AS numbers and countries get a percentage of every
limit, for example 50 percent for common abuse-source networks, so their
clients are banned after fewer requests. Zero percent is a zero allowance: the
first request breaks the limit and bans the client.
- Attack detection:
- a directory of plain text rule files, one regex per line, for catching
scanning and penetration probes; easy to edit by hand, and picked up while
running;
- the OWASP Core Rule Set, run by the Coraza engine, refusing the requests
it flags; it reads the URL and headers of every request, and request
bodies only once you switch that on, since on a code forge they are full
of code it would take for attacks;
- trap paths, and a ban for a client that the rule files or the Core Rule
Set refuse again and again.
- Bans:
- a clear sign of attack, such as a probe for a `.env` file or a scanner's
user agent, bans for seven days on the first request, and any further
request during those days makes the ban permanent;
- breaking a limit bans for an hour; breaking one again within a day of a
ban ending triples the length, and a ban that would last longer than seven
days is permanent instead;
- every ban carries notes on why it was made, to help decide whether to lift
it.
- IP reputation: downloadable blocklists, DNS blocklists, AbuseIPDB, and an
optional feed of decisions from a CrowdSec engine. Lookups happen in the
background and never delay a request. None is on until you add it.
- Alerts on attacks and bans to a generic webhook, Slack or ntfy, with a
cooldown and an hourly cap so a wide attack cannot flood the channel.
- Anomaly alerts when requests or bytes per minute or hour cross a threshold you
set, for a single client, its surrounding netblock, an AS number, a named
netblock or the whole service.
- Observe mode: log and alert on every decision while refusing nothing.
- Request log: one JSON object per request on stdout with the usual web log
fields, the decision taken and why, AS number and country, and timings.
Optionally also sent to a remote syslog server.
- Prometheus metrics, for a scraper that holds the metrics token.
- State (bans with their notes, each client's counters and history, the GeoJS
answers, the reputation cache, the alerting state) held in memory and kept in
readable JSON files, written regularly and at every stop, so a restart loses
nothing. Edit a file, or add a rule file, and the running `smallwebwaf` picks
up the change. Nothing is read from disk while serving a request. The files
for the bans, the clients, the GeoJS answers, the copies of the lists, the
verdicts of the DNSBL zones, AbuseIPDB's scores and the alerts are built, with
an edit taken in while running (see "State files" above); the rest comes with
its features.
- Health checks, the metrics, and listing, adding and lifting bans or asking why
a given address was refused, all on the one port every request uses: under
`/_smallwebwaf/` on the app's own address, through traefik like any other
request. The metrics need the metrics token, and the ban management the admin
token.
Not planned: TLS termination, routing for several apps, browser challenges
(captcha or proof of work), a web console, or defence against floods large
enough to fill the host's network link.
## How it works, in short
For each request `smallwebwaf`:
- works out who the client really is;
- lets it straight through if it is on the bypass list, refuses it if it is on
the deny list or currently banned;
- looks up its AS number and country, and refuses it if that country is denied,
or is not among the only ones allowed;
- checks it against the blocklists, bans it if the CrowdSec decision list lists
it, and checks it for a cached reputation verdict;
- picks the client's limit percentage from those;
- checks the minute, hour and day request counters against the limits, and bans
the client if it breaks one;
- checks the request against the trap paths, the rule files and the Core Rule
Set, and bans the client at once for a clear sign of attack;
- forwards it to the app and streams the response back, within the size and time
limits;
- counts the bytes and any refusal by the trap paths, the rule files or the Core
Rule Set or for a missing or wrong token, bans the client if it broke a limit,
updates its history and the anomaly counters, sends any alerts that are due,
and writes the log line.
A minimal deployment is the app's own Dockerfile, built on the `smallwebwaf`
image, with no setting. That image is built on Ubuntu 26.04 LTS, the newest
long-term support release of Ubuntu, pinned by digest, and moves to the next one
when it ships. It has nixpkgs installed, so the app adds the packages it needs
from nixpkgs. Beyond its `FROM` line the app's Dockerfile adds the app's binary,
any packages it needs, and the app's runit service, which starts the app as a
user of its own, listening on `127.0.0.1:8081`:
```dockerfile
# The smallwebwaf image, pinned by digest.
FROM <registry>/smallwebwaf:<pinned digest>
# Packages the app needs, if any, from the nixpkgs in the image.
RUN nix-env -iA nixpkgs.git
# The app's binary, and a user of its own to run it.
COPY app /usr/local/bin/app
RUN useradd --system --no-create-home --shell /usr/sbin/nologin app
# The app's runit service.
COPY --chmod=755 app.run /etc/service/app/run
```
with `app.run` beside the Dockerfile, where `--listen` and `--trusted-proxies`
stand for the app's own options:
```bash
#!/usr/bin/env bash
set -euo pipefail
main() {
sleep 1
exec chpst -u app:app /usr/local/bin/app \
--listen 127.0.0.1:8081 \
--trusted-proxies 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1/32,::1/128
}
main "$@"
```
- The image's entrypoint, `runsvinit`, has runit start `smallwebwaf` and the app
side by side, each as its own user, and start either again a second after it
exits. Leave out `ENTRYPOINT` and `USER` from the app's Dockerfile.
- `nix-env -iA nixpkgs.<name>` installs a package from the nixpkgs in the image,
and the app finds it on its `PATH`, after Ubuntu's own commands. That nixpkgs
is fixed at one commit, so the same `smallwebwaf` image always gives the app
the same packages; newer ones come with a newer `smallwebwaf` image.
- Deploy it as you deploy any app, with traefik's labels on this one container
pointing at port 8080. upaas needs no change for this.
- The app has to trust `127.0.0.1` and `::1` for forwarded headers, besides the
private address ranges, since the requests it gets now come from `smallwebwaf`
on loopback. An app left at the usual default, the private ranges alone, sees
every visitor as `127.0.0.1`.
- Port 8080 is the only one the app must leave free: the health check, the
metrics and ban management are all on it, under `/_smallwebwaf/`. The image's
health check passes while `smallwebwaf` answers and the app accepts
connections. `SWWAF_LISTEN_ADDR` can move `smallwebwaf` to another port, which
the app then leaves free instead; the health check follows it, and traefik's
labels must point at it. The address part of `SWWAF_LISTEN_ADDR` stays empty
(for example `:9000`, never `127.0.0.1:9000`), so `smallwebwaf` keeps
listening on every address: traefik reaches it on the container's address, and
the health check on `127.0.0.1`.
- `smallwebwaf` keeps its state files in `/var/lib/smallwebwaf`. Mount a volume
there to keep bans and client history when a deploy replaces the container;
without one, it still starts. At each start the `run` script of `smallwebwaf`
gives that directory and every file in it to the `smallwebwaf` user, so a host
directory mounted there needs no change of owner.
- `docker stop` has runit stop both processes. `smallwebwaf` then stops taking
requests and gives those in progress five seconds to finish.
A rule file is one rule per line: a name, what to match against, what to do, and
a regex.
```
env-file path ban (?i)^/\.env(\.[a-z]+)?$
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
```
[`SPEC.md`](SPEC.md) has the full design: the deployment, every environment
variable, the ban rules, the rule file format, the state files, the log fields,
the metrics, failure behaviour and the build order.
## Country and AS number lookup
`smallwebwaf` looks up the AS number and country of every client through GeoJS,
a free web service that needs no account and no file, for the request log, the
client's history, the notes of its bans, their alerts and the metrics, and for
the country lists and the anomaly thresholds per AS number when you set them.
This means that GeoJS is told the address of every new visitor, whether or not a
setting uses the answer, unless you set `SWWAF_LOOKUP_SOURCE=off`. The only
visitors it is not told about are those in `SWWAF_ALLOW_NETS` or
`SWWAF_DENY_NETS`, those whose netblock a ban covers, and those on a private,
loopback or link-local address. An IPv6 visitor is asked about by the first
address of its IPv6 group. Each answer is kept for seven days, in memory and in
`lookups.json`, so that it survives a restart, and a visitor whose answer is
kept is not asked about again.
A request waits for its client's first answer only while a setting acts on it
before the request goes on: a country list, `SWWAF_ADD_LOOKUP_HEADERS`, or a
biased threshold that lowers a limit. A new visitor then waits up to
`SWWAF_LOOKUP_TIMEOUT`, a second by default, and without an answer counts as
coming from an unknown country until the answer arrives. Otherwise no request
waits: it goes on at once and is logged without the answer, which reaches the
client's history and the notes of its bans when it comes. The addresses waiting
are asked about together, up to 200 in one request, one request at a time; at
most 10,000 visitors wait, and one more is not asked about until there is room,
counting meanwhile as coming from an unknown country. GeoJS publishes no rate
limit but may block a caller it thinks asks too much. While GeoJS fails,
visitors with a kept answer are unaffected and new ones count as coming from an
unknown country, which `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses, and whose
limits `SWWAF_UNKNOWN_LIMIT_PERCENT` sets. GeoJS is then left alone for a
second, twice as long after each further failure up to five minutes, and asked
again by the next request from a visitor without an answer.
To keep your visitors' addresses on your own host, set
`SWWAF_LOOKUP_SOURCE=off`, or use the database file instead of GeoJS:
`SWWAF_LOOKUP_SOURCE=file` looks every client up in the free IPinfo Lite
database (`ipinfo_lite.mmdb`), at once, with no wait and nothing sent off the
host. A client whose address it does not hold counts as coming from an unknown
country. You download the database with your own IPinfo account, mount the
directory that holds it into the container, point `SWWAF_LOOKUP_DB_PATH` at the
file and refresh it when you choose; `smallwebwaf` never downloads it itself. It
reads the whole file into memory at start, and a file that is missing or that it
cannot read stops the start. It reads the file again once it has gone 2 seconds
without a change after you replace it, so that a file still being copied in is
read only once whole, and also 2 seconds after it starts watching, so that a
file replaced while it started is not missed. A replacement it cannot read is
logged and sent as a `file_error` alert, and the file read before stays in use.
It has to be the directory rather than the file itself that you mount: docker
does not show a single mounted file being replaced, so a refresh would go
unseen. IPinfo releases it under the Creative Commons Attribution-ShareAlike 4.0
International License and asks for attribution, in its own words on
https://ipinfo.io/lite: "The attribution requirements can be met by giving our
service credit as your data source. Simply place a link to IPinfo on the
website, application, or social media account that uses our data." Its example
of such a credit is a link mentioning "IP address data is powered by IPinfo". A
service that uses the database through `smallwebwaf` should carry that link.
Neither source can place a private address, so a client on one, such as a
visitor on your local network, another container or your monitoring, has no
country: `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless you list it in
`SWWAF_ALLOW_NETS`, `SWWAF_DENIED_COUNTRIES` does not refuse it, and
`SWWAF_UNKNOWN_LIMIT_PERCENT` sets its limits. Such addresses are never sent to
GeoJS.
## Blocklists
`SWWAF_BLOCKLIST_URLS` names blocklists: text files of addresses and netblocks,
one to a line, written as the Spamhaus DROP list,
`https://www.spamhaus.org/drop/drop.txt`, is. Anything after a `;` or a `#` on a
line is left out, and so is a line left blank. A bare address stands for itself
alone, as in the settings. An IPv4-mapped address or netblock, such as
`::ffff:192.0.2.0/120`, is read as the IPv4 one it stands for, here
`192.0.2.0/24`, since a client's IPv4 address is checked as IPv4; a mapped
netblock shorter than `/96` stands for none, and is not a netblock. None is
named by default: a list judges a client by what others saw it do, while the
defaults judge it by what it does to your service.
`smallwebwaf` fetches each list, and the file `SWWAF_ASN_LIMIT_PERCENT_URL`
names, `SWWAF_BLOCKLIST_REFRESH` after it last fetched it or tried to, 24 hours
by default and never less than one, one list after another. Since
`reputation.json` keeps when each list was last tried, the fetch failed or not,
even one cut off as `smallwebwaf` stopped, whose request the server may have
had, at start `smallwebwaf` fetches at once only a list it has never tried, and
one it last tried that long ago; any other waits its turn, so that restarts do
not fetch a list more often. A fetch fails when the server answers other than
`200`, when it does not finish within a minute, when the list is longer than 16
MiB, or when a line of it is not an address or a netblock, or for
`SWWAF_ASN_LIMIT_PERCENT_URL`, not an AS number, `:` and a percentage. The copy
fetched before then stays in use, and the failure is counted, logged and raised
as a `source_failure` alert; a fetch cut off as `smallwebwaf` stops is not a
failure. The last good copy of each list is kept whole, comment lines included,
in `reputation.json` (see "State files" above), so that a restart keeps it in
use too. Each list is named by its URL, in the request log, the alerts, the
notes of bans and the metrics, so keep a secret out of it.
A client in `SWWAF_ALLOW_NETS` is not checked. Any other is checked by its own
address after the country lists, and `SWWAF_BLOCKLIST_ACTION` says what is done
with one a list lists, as "What it does so far" above describes. `deny` suits
lists of networks that send nothing legitimate, such as DROP; `limit:<percent>`
suits lists of addresses shared with ordinary visitors, such as those of Tor's
exits.
The Spamhaus DROP list is The Spamhaus Project's, https://www.spamhaus.org. Its
terms, on its DROP page, ask that a product using it credit The Spamhaus Project
and keep the list's date and copyright lines with the data, which the copy in
`reputation.json` does; a service that uses DROP through `smallwebwaf` uses data
from The Spamhaus Project, and should say so. They also ask that it be fetched
automatically no more than once an hour, once a day being more than enough in
most cases, and Spamhaus may block an address that fetches it more often. Each
`smallwebwaf` fetches its own copy, on its own schedule, so on a host where
several apps run it, sharing one address, their fetches can come less than an
hour apart whatever `SWWAF_BLOCKLIST_REFRESH` is: each fetches a list again that
long after its own last try, so those first started within the same hour, with
the same refresh, keep fetching within the same hour.
## DNS blocklists
`SWWAF_DNSBL_ZONES` names DNS blocklists, DNSBL zones, which list an address by
answering a query for a name made from it. None is named by default, for the
reason none of the blocklists is. `smallwebwaf` asks each zone about a client's
own address in the background, the first time it sees the client: the request
goes on at once, as from a client the zone does not list, and so does every
request from it until the zone has answered. The name asked about is the one RFC
5782 gives: the four numbers of an IPv4 address in reverse order, so that
`192.0.2.99` is asked about in `dnsbl.dronebl.org` as
`99.2.0.192.dnsbl.dronebl.org`, or the 32 hex digits of an IPv6 address in
reverse order, each followed by a dot. An IPv6 client is asked about by its own
address, not by its IPv6 group.
A zone that answers that the name does not exist, or has no address, does not
list the client, and one that answers with an address in `127.0.0.0/8` lists it.
Any other answer gives no verdict, and is a failure: an address in
`127.255.255.0/24`, with which Spamhaus refuses a query, such as one sent
through a public resolver or one past its limit; an address outside
`127.0.0.0/8`, such as a resolver gives that answers even for names that do not
exist; an error the resolver answers with, such as a refusal; and no answer
within `SWWAF_REPUTATION_TIMEOUT`. A failure is counted, logged and raised as a
`source_failure` alert, held back as a repeat within `SWWAF_ALERT_COOLDOWN`, and
the zone is not asked again for a minute, so that a zone that refuses queries is
not asked on every request.
Each verdict, the zone listing the client or not, is kept, in memory and in
`reputation.json` (see "State files" above), so that a restart keeps it, and is
used for `SWWAF_REPUTATION_CACHE_TTL` after the zone gave it, 24 hours by
default. The client's first request after that has the zone asked again, and
until it answers, no verdict of the zone applies to the client. At most 100,000
verdicts are kept, the one fetched longest ago dropped first, and at most 1,000
queries are under way at once; past that, a zone is asked about a client at the
client's next request.
A client in `SWWAF_ALLOW_NETS` is not checked, nor is one a blocklist refuses.
`SWWAF_REPUTATION_ACTION` says what is done with one a zone's verdict lists, as
"What it does so far" above describes.
Do not name a zone meant for mail, such as one of residential and dynamic
address ranges, which ordinary visitors come from, or one that includes such a
list, as Spamhaus's `zen` does: it would refuse them, or lower their limits.
Spamhaus's zones answer through its keyed query service, named with the key in
it, such as `<key>.xbl.dq.spamhaus.net`. The key of a zone under
`dq.spamhaus.net` is its first label, and `smallwebwaf` shows `********` in its
place wherever it names the zone, as `********.xbl.dq.spamhaus.net`: in the
settings logged at start, an error that stops the start, its own messages, the
request log, the alerts, the notes of bans and the metrics. Only
`reputation.json` keeps the zone with its key. A key in the name of any other
zone is shown as given. Several zones refuse queries that come through a public
resolver; `SWWAF_DNSBL_RESOLVER` names another resolver to ask through.
## AbuseIPDB
While `SWWAF_ABUSEIPDB_KEY` holds the key of an AbuseIPDB account, `smallwebwaf`
asks AbuseIPDB's check endpoint, `https://api.abuseipdb.com/api/v2/check`, for
the abuse confidence score of a client's own address, from 0 to 100. It is unset
by default, for the reason no blocklist is named, and since AbuseIPDB needs an
account. An IPv6 client, an IPv6 group, is checked by the address of the request
that has it checked, and its score is used for the whole group, whichever of its
addresses sends, so that one client costs at most one check every
`SWWAF_REPUTATION_CACHE_TTL`.
Only a client whose history counts an offence is checked, so that the checks are
spent on suspects: so far, one that has broken a rate limit or a byte limit,
matched a ban rule, or had a request refused by a block rule or the Core Rule
Set. A client dropped from the table of clients loses its history, and with it
its offences. A client is checked in the background, at its first request after
its offence that reaches the check: no request waits, a request refused under
its ban is not checked, and the request that has it checked, and any other from
it before the answer comes, goes on as from a client without a score.
A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE`, 75 by default, is a hit, and
`SWWAF_REPUTATION_ACTION` says what is done with its client, as for a DNSBL
zone's verdict (see "What it does so far" above). Each score, a hit or not, is
kept, in memory and in `reputation.json` (see "State files" above), and used for
`SWWAF_REPUTATION_CACHE_TTL` after AbuseIPDB gave it, 24 hours by default. The
client's first request after that has it checked again, if its history still
counts an offence. At most 100,000 scores are kept, the one fetched longest ago
dropped first.
At most `SWWAF_ABUSEIPDB_DAILY_BUDGET` checks are made in a day, 900 by default,
below the 1,000 of AbuseIPDB's free accounts. The day is counted in UTC, from
00:00. Each check sent spends one of them, whatever AbuseIPDB answers, and the
checks spent today are kept in `reputation.json`, so that a restart does not
make the budget whole again. The check that spends the last of the day's budget
is logged and raised as a `source_failure` alert; from then until the day ends,
no client is checked, and a client without a score counts as one AbuseIPDB does
not list.
A check fails when AbuseIPDB answers other than `200`, such as `429` past its
own daily limit or `401` for a wrong key, when its answer gives no
`abuseConfidenceScore`, and when it does not answer within
`SWWAF_REPUTATION_TIMEOUT`. A failure gives no score, and is counted, logged and
raised as a `source_failure` alert, held back as a repeat within
`SWWAF_ALERT_COOLDOWN`, and no client is checked for a minute after it.
The key is sent to AbuseIPDB in the `Key` header, and nowhere else. The settings
logged at start show `********` in its place, and neither the log, the alerts,
the metrics nor `reputation.json` hold it. Given as a file, with
`SWWAF_ABUSEIPDB_KEY_FILE`, it can be kept out of the app's reach (see "Settings
given as files" above).
## CrowdSec
While `SWWAF_CROWDSEC_LAPI_URL` names the local API of a CrowdSec engine you
run, `smallwebwaf` fetches the engine's decision list, at that URL with
`v1/decisions` added to its path, with `SWWAF_CROWDSEC_LAPI_KEY`, a key the
engine gave, such as `cscli bouncers add smallwebwaf` makes. This is how a list
kept for a whole fleet of hosts can reach `smallwebwaf` without it depending on
CrowdSec. It is unset by default, for the reason no blocklist is named, and
since it needs an engine of your own.
The list is fetched again a minute after it was last fetched or tried, the fetch
failed or not, and is kept as a blocklist is (see "Blocklists" above): its last
good copy, the engine's answer as it came, stays in use while a fetch fails, and
`reputation.json` keeps it with the time it was fetched, so that a restart keeps
it in use too. A fetch fails when the engine answers other than `200`, a
redirect included, such as `403` for a key it does not know, when it does not
finish within a minute, when the answer is longer than 16 MiB or is not a JSON
list of decisions, or when a decision to ban gives a value that is not an
address or a netblock, or a `duration` that does not read. A failure is counted,
logged and raised as a `source_failure` alert, held back as a repeat within
`SWWAF_ALERT_COOLDOWN`.
The decisions of the type `ban` on an address or a netblock, the scopes `Ip` and
`Range`, are used; any other, such as one to show a captcha or one on a country,
is left out. A decision bans until its `duration`, the time it had left as the
engine answered, has passed since the fetch, so that one that has ended no
longer bans, even while the copy in use still holds it because the engine cannot
be reached.
A client in `SWWAF_ALLOW_NETS` is not checked, nor is one a blocklist refuses.
Any other is checked by its own address after the blocklists. A request from a
client a decision in force bans is refused with `SWWAF_BAN_RESPONSE`, and bans
the client's netblock, as any ban covers it, until that decision ends, the one
that ends last if several ban the client. The ban's cause is `crowdsec`, and its
reason names the decision's scenario. Only a client that sends a request is
banned, so that a long list does not fill `bans.json`. Such a ban counts toward
`SWWAF_MAX_BANS`, is never made permanent, and does not make the netblock's next
ban for a broken limit longer. The request's log line and the ban's notes name
the list by its URL, as they name a blocklist, and the request raises a
`reputation_hit` alert besides the ban's.
A ban you lift while its decision is in force is followed by another at the
client's next request. To let a client in, delete its decision in CrowdSec, such
as with `cscli decisions delete --ip 203.0.113.9`, and lift the ban once the
list has been fetched again, a minute or so later. A ban made for a decision
deleted in CrowdSec otherwise lasts until that decision would have ended.
The key is sent to the engine in the `X-Api-Key` header, and nowhere else. The
settings logged at start show `********` in its place, and neither the log, the
alerts, the metrics nor `reputation.json` hold it. Given as a file, with
`SWWAF_CROWDSEC_LAPI_KEY_FILE`, it can be kept out of the app's reach (see
"Settings given as files" above).
## How the code is laid out
- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`.
- `internal/smallwebwaf`: the process: it reads the settings, the rule files,
the lookup database and the state files, listens, serves requests until
`SIGTERM` or `SIGINT`, and stops, writing the state files. Run as
`smallwebwaf healthcheck`, it is the image's health check instead.
- `internal/config`: reads the settings, the one place they are read.
- `internal/proxy`: what happens to each request: it works out the client, runs
the checks, passes the request to the app and the answer back with the
standard library's `httputil.ReverseProxy` within the timeouts and size
limits, and writes the request's log line. Its `check` method is where a
request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for
a ban, for the country lists, for a blocklist, for the CrowdSec decision list,
which bans the client, for a DNSBL zone's verdict, for AbuseIPDB's score, for
a rate limit, which bans the client, for a trap path, which bans the client,
for a `block` or `ban` rule, the latter banning the client, for a Core Rule
Set match in `block` mode, and for an announced body over the size limit; in
`observe` mode, only for the size limit, with what it would have refused for
noted in the log line. A request under `/_smallwebwaf/` that `check` lets
through is answered by `answerAdmin` instead of reaching the app. Once the
answer to a request passed to the app has ended, `countBytes` counts its bytes
for the byte limits, and once any request but the health check has ended,
`countRefusal` counts it for the error burst if it was refused after a rule
file match, a trap path or a Core Rule Set match or for its token, and
`countAnomalies` counts it for the anomaly thresholds.
- `internal/metrics`: the metrics, counted as the other parts tell it what
happened, and served in the Prometheus text format.
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how
long a new ban lasts, when a ban for a clear sign of attack becomes permanent,
and which ban `smallwebwaf` made is dropped when `SWWAF_MAX_BANS` are held.
- `internal/rules`: reads the rule files at start and again as they change, and
tells which of their rules a request matches.
- `internal/waf`: the Core Rule Set with the six changes, as Coraza's own
directives, and what it finds in a request's method, URL and headers: the
rules that matched and the anomaly score.
- `internal/lookup`: looks up each client's AS number and country through GeoJS,
keeps the answers, and hands each new one to the proxy, which adds it to the
client's history and to the notes of its bans; or in the lookup database,
which it reads again when the file is replaced. `internal/lookup/lookuptest`
writes lookup databases for the tests.
- `internal/reputation`: fetches the blocklists, the file
`SWWAF_ASN_LIMIT_PERCENT_URL` names and the CrowdSec decision list as they are
due, keeps the last good copy of each, and tells which blocklists list an
address, what percentage the file gives an AS number, and which decision of
the decision list bans an address; asks the DNSBL zones about clients in the
background, through the standard library's resolver, keeps their verdicts, and
tells which zones' verdicts list an address; and checks clients with AbuseIPDB
in the background, keeps their scores and the checks spent today, and tells
whether a client's score is a hit.
- `internal/ratelimit`: the table of clients: counts each client's requests,
bytes and refusals, tells when they take it over a rate limit, a byte limit or
the error burst, and keeps each client's history.
- `internal/anomaly`: the anomaly counters: counts each request and its bytes
per client, per netblock around a client, per AS number, for the whole service
and per named netblock, in the buckets `internal/ratelimit` counts in, and
raises an `anomaly` alert for a count over its threshold.
- `internal/state`: reads the state files at start, takes in an admin's edit of
one while running, and writes them when they are due and at the stop.
- `internal/requestlog`: the lines on stdout: the request log line and the
process's own messages.
- `internal/remotelog`: sends the lines on stdout to `SWWAF_LOG_REMOTE_URL`,
each as a syslog record, from a buffer of its own. It is written with the
standard library alone, whose `log/syslog` writes only the older syslog
format.
- `internal/alerts`: takes the alerts the other parts raise, holds back repeats
and those past the hourly limit, and sends the others to the webhook, Slack
and ntfy, each from a queue of its own.
- `Dockerfile`: the lint and test phases, the stages `script/tidy` builds, then
the image, whose last stage installs Ubuntu's packages, nixpkgs, `runsvinit`
and `smallwebwaf`, with `share/smallwebwaf.run` as runit's `run` script for
`smallwebwaf` and `share/rules.d/00-default.rules` as its default rule file.
- `deploy/example-app`: an app built on the image, which `script/example-app`
checks.
Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
table of clients to `SWWAF_MAX_TRACKED_CLIENTS` and the GeoJS answers to
100,000, dropping the least recently seen, the DNSBL zones' verdicts and
AbuseIPDB's scores to 100,000 each, dropping the one fetched longest ago, the
anomaly counters to 20,000, dropping the one counted least recently, and the
banned netblocks in the order they were last seen, from which the ledger picks
the ban to drop past `SWWAF_MAX_BANS`, and `github.com/prometheus/client_golang`
keeps the metrics and serves them, and `github.com/fsnotify/fsnotify` tells
`smallwebwaf` when a state file or a rule file is saved, or the lookup database
replaced, and `github.com/oschwald/maxminddb-golang/v2` reads the lookup
database, which the tests write with `github.com/maxmind/mmdbwriter`, and
`github.com/corazawaf/coraza/v3` runs the Core Rule Set 4.25.0, which
`github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0` carries. The country
codes are the list in `internal/config/config.go`.
## Entrypoints
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: the scripts in `script/` are the entrypoints for working on it, and
the `Makefile` targets are thin shims that call them. The scripts are POSIX sh,
so that they run in minimal containers.
- `script/bootstrap`: installs what the other scripts need on the host: `make`,
`git`, `curl`, Go for `gofmt`, node and yarn, and prettier.
- `script/setup`: readies a fresh clone: runs `script/bootstrap`, then
`script/install-precommit`.
- `script/projectname`: prints the project's name, `smallwebwaf`, which
`script/docker` and the others tag their images with.
- `script/test`: checks that `go.mod` and `go.sum` are as `go mod tidy` writes
them, then runs the tests, as the `test` phase of the `Dockerfile`.
- `script/tidy`: writes `go.mod` and `go.sum` as `go mod tidy` writes them, with
the Go of the `test` phase, in the `tidy` stage of the `Dockerfile`;
`make tidy` runs it.
- `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`.
- `script/fmt`: formats the Go code with `gofmt` and the Markdown with prettier.
- `script/fmt-check`: checks the formatting, and changes nothing.
- `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`.
- `script/docker`: builds the image, whose build runs the tests and the linter
first.
- `script/cibuild`: what CI runs: `script/bootstrap`, `script/check`, then the
image build.
- `script/precommit`: run by the git pre-commit hook; runs `script/check`.
- `script/install-precommit`: installs that hook; `make hooks` runs it.
- `script/build`: builds `bin/smallwebwaf` on the host, with Go installed, for
working on the code by hand; `make build` runs it.
- `script/run`: builds `bin/smallwebwaf` with `script/build` and runs it, with
its state files in `bin/state` unless `SWWAF_STATE_DIR` is set, and the rule
files of `share/rules.d` unless `SWWAF_RULES_DIR` is set; `make run` runs it.
- `script/example-app`: builds the image and, on it, the example app in
`deploy/example-app`, runs it with a volume for the state files, and checks
that the health check passes, that a request reaches the app through
`smallwebwaf`, that a second request in a minute bans the client, that a probe
for `/.env` bans another client, whose next request makes the ban permanent,
that `sv stop` and `docker stop` stop it in order, and that a new container on
the same volume still refuses the banned client; then removes the containers,
the volume and both images. It needs network access, for nixpkgs' binary
cache, and `script/check` does not run it; `make example-app` does.
## TODO
- The rest of the design, in the order of the build order in
[`SPEC.md`](SPEC.md).
## Documents
- [`SPEC.md`](SPEC.md): the design.
- [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers
and misses, and why none was adopted.
- [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows.
## License
MIT. See [`LICENSE`](LICENSE).
## Author
[@sneak](https://sneak.berlin)
Reference in New Issue View Git Blame Copy Permalink