SPEC and README: review fixes, country lists, GeoJS lookups (closes #6)

Address the review of the spec update and fold in issues 8 and 11.
Default ban rules are anchored at the site root. `bans.json` is bounded
by `MAX_BANS` and rewritten only when a ban is made, lifted or made
permanent. `clients.json` keeps one client per line, holds at most 20000
clients and is written every 15 minutes. Anomaly counters and alert
state go to a new `alerts.json`, the AbuseIPDB count to
`reputation.json`. 401 no longer counts toward the error burst. New
settings: `DENIED_COUNTRIES`, `EXCLUSIVELY_ALLOWED_COUNTRIES`,
`LOOKUP_SOURCE` (the IPinfo file or GeoJS), `LOOKUP_TIMEOUT`,
`CLIENT_REQUEST_HEADER_MAX_BYTES` and `CLIENT_IDLE_TIMEOUT`.

Model: opus-5-5
This commit is contained in:
2026-09-23 13:37:34 +00:00
parent 43c63faa04
commit 8fc7539f1a
2 changed files with 304 additions and 142 deletions
+42 -22
View File
@@ -52,7 +52,7 @@ goes through the candidates one by one.
other setting has a default chosen for a service facing the internet in 2026.
- Real client address worked out from `X-Forwarded-For`, trusting only the proxy
networks you list, by default the private address ranges. IPv6 clients are
counted by /64.
counted by /64 by default.
- Size and time limits on requests and responses, both between the client and
`smallwebwaf` and between `smallwebwaf` and the app: by default a request may
take 60 seconds and 100 MB, a response 30 minutes and 5 GB.
@@ -61,8 +61,14 @@ goes through the candidates one by one.
real visitors need.
- Netblocks that bypass rate limiting, netblocks that bypass everything, and
netblocks that are always refused.
- AS number and country lookup for every client from the IPinfo Lite database
file, which you download and mount (see "Lookup database" below).
- AS number and country lookup for every client, off until you choose a source:
the IPinfo Lite database file, which you download and mount, or the free GeoJS
web service (see "Country and AS number lookup" below).
- Country lists: `DENIED_COUNTRIES` refuses every request from the countries
listed, `EXCLUSIVELY_ALLOWED_COUNTRIES` every request from anywhere else. Such
a request gets the answer a banned client gets as soon as the client's address
has been looked up, before its body is read and without the rule files or the
Core Rule Set looking at it, and no ban is made.
- Biased limits: listed AS numbers and countries get a percentage of every
limit, for example 50 percent for common abuse-source networks, so their
clients are banned after fewer requests. Zero percent is a zero allowance: the
@@ -96,11 +102,11 @@ goes through the candidates one by one.
fields, the decision taken and why, AS number and country, and timings.
Optionally also sent to a remote syslog server.
- Prometheus metrics on their own port.
- State (bans with their notes, each client's counters and history, the
reputation cache) held in memory and kept in readable JSON files that always
hold a copy of it, so a restart loses nothing. Edit a file, or add a rule
file, and the running `smallwebwaf` picks up the change. Nothing is read from
disk while serving a request.
- State (bans with their notes, each client's counters, history and lookup
answer, the reputation cache, the alerting state) held in memory and kept in
readable JSON files, written regularly and at every stop, so a restart loses
nothing. Edit a file, or add a rule file, and the running `smallwebwaf` picks
up the change. Nothing is read from disk while serving a request.
- A small admin endpoint for health checks, listing, adding and lifting bans,
and asking why a given address was refused.
@@ -115,7 +121,9 @@ For each request `smallwebwaf`:
- works out who the client really is;
- lets it straight through if it is on the bypass list, refuses it if it is on
the deny list or currently banned;
- looks up its AS number and country, and any cached reputation verdict;
- looks up its AS number and country, and refuses it if that country is denied,
or is not among the only ones allowed;
- checks for a cached reputation verdict;
- picks the client's limit percentage from those;
- checks the minute, hour and day request counters against the limits, and bans
the client if it breaks one;
@@ -162,7 +170,7 @@ A rule file is one rule per line: a name, what to match against, what to do, and
a regex.
```
env-file path ban (?i)/\.env(\.[a-z]+)?$
env-file path ban (?i)^/\.env(\.[a-z]+)?$
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
```
@@ -170,19 +178,31 @@ scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
rules, the rule file format, the state files, the log fields, the metrics,
failure behaviour and the build order.
## Lookup database
## Country and AS number lookup
AS number and country lookups, and the biased limits that use them, need the
free IPinfo Lite database (`ipinfo_lite.mmdb`). You download it with your own
IPinfo account, mount it into the container, point `LOOKUP_DB_PATH` at it and
refresh it when you choose; `smallwebwaf` never downloads it itself. IPinfo
releases it under the Creative Commons Attribution-ShareAlike 4.0 International
License and asks for attribution, in its own words on https://ipinfo.io/lite:
"The attribution requirements can be met by giving our service credit as your
data source. Simply place a link to IPinfo on the website, application, or
social media account that uses our data." Its example of such a credit is a link
mentioning "IP address data is powered by IPinfo". A service that uses the
database through `smallwebwaf` should carry that link.
AS number and country lookups, and the country lists and biased limits that use
them, are off until you choose one of two sources with `LOOKUP_SOURCE`.
`LOOKUP_SOURCE=file` reads the free IPinfo Lite database (`ipinfo_lite.mmdb`).
You download it with your own IPinfo account, mount the directory that holds it
into the container, point `LOOKUP_DB_PATH` at the file and refresh it when you
choose; `smallwebwaf` never downloads it itself, and reads it again when you
replace it. It has to be the directory rather than the file itself: docker does
not show a single mounted file being replaced, so a refresh would go unseen.
IPinfo releases it under the Creative Commons Attribution-ShareAlike 4.0
International License and asks for attribution, in its own words on
https://ipinfo.io/lite: "The attribution requirements can be met by giving our
service credit as your data source. Simply place a link to IPinfo on the
website, application, or social media account that uses our data." Its example
of such a credit is a link mentioning "IP address data is powered by IPinfo". A
service that uses the database through `smallwebwaf` should carry that link.
`LOOKUP_SOURCE=geojs` asks the free GeoJS web service instead, with no account
and no file. Every new visitor's address is sent to GeoJS. Each answer is kept
for seven days, across restarts, and many addresses are asked about in one
request. GeoJS publishes no rate limit but may block a caller it thinks asks too
much; while it is not answering, new visitors count as coming from an unknown
country, which `EXCLUSIVELY_ALLOWED_COUNTRIES` refuses.
## Documents