SPEC and README: review fixes, country lists, GeoJS lookups (closes #6)
Address the review of the spec update and fold in issues 8 and 11. Default ban rules are anchored at the site root. `bans.json` is bounded by `MAX_BANS` and rewritten only when a ban is made, lifted or made permanent. `clients.json` keeps one client per line, holds at most 20000 clients and is written every 15 minutes. Anomaly counters and alert state go to a new `alerts.json`, the AbuseIPDB count to `reputation.json`. 401 no longer counts toward the error burst. New settings: `DENIED_COUNTRIES`, `EXCLUSIVELY_ALLOWED_COUNTRIES`, `LOOKUP_SOURCE` (the IPinfo file or GeoJS), `LOOKUP_TIMEOUT`, `CLIENT_REQUEST_HEADER_MAX_BYTES` and `CLIENT_IDLE_TIMEOUT`. Model: opus-5-5
This commit is contained in:
@@ -52,7 +52,7 @@ goes through the candidates one by one.
|
||||
other setting has a default chosen for a service facing the internet in 2026.
|
||||
- Real client address worked out from `X-Forwarded-For`, trusting only the proxy
|
||||
networks you list, by default the private address ranges. IPv6 clients are
|
||||
counted by /64.
|
||||
counted by /64 by default.
|
||||
- Size and time limits on requests and responses, both between the client and
|
||||
`smallwebwaf` and between `smallwebwaf` and the app: by default a request may
|
||||
take 60 seconds and 100 MB, a response 30 minutes and 5 GB.
|
||||
@@ -61,8 +61,14 @@ goes through the candidates one by one.
|
||||
real visitors need.
|
||||
- Netblocks that bypass rate limiting, netblocks that bypass everything, and
|
||||
netblocks that are always refused.
|
||||
- AS number and country lookup for every client from the IPinfo Lite database
|
||||
file, which you download and mount (see "Lookup database" below).
|
||||
- AS number and country lookup for every client, off until you choose a source:
|
||||
the IPinfo Lite database file, which you download and mount, or the free GeoJS
|
||||
web service (see "Country and AS number lookup" below).
|
||||
- Country lists: `DENIED_COUNTRIES` refuses every request from the countries
|
||||
listed, `EXCLUSIVELY_ALLOWED_COUNTRIES` every request from anywhere else. Such
|
||||
a request gets the answer a banned client gets as soon as the client's address
|
||||
has been looked up, before its body is read and without the rule files or the
|
||||
Core Rule Set looking at it, and no ban is made.
|
||||
- Biased limits: listed AS numbers and countries get a percentage of every
|
||||
limit, for example 50 percent for common abuse-source networks, so their
|
||||
clients are banned after fewer requests. Zero percent is a zero allowance: the
|
||||
@@ -96,11 +102,11 @@ goes through the candidates one by one.
|
||||
fields, the decision taken and why, AS number and country, and timings.
|
||||
Optionally also sent to a remote syslog server.
|
||||
- Prometheus metrics on their own port.
|
||||
- State (bans with their notes, each client's counters and history, the
|
||||
reputation cache) held in memory and kept in readable JSON files that always
|
||||
hold a copy of it, so a restart loses nothing. Edit a file, or add a rule
|
||||
file, and the running `smallwebwaf` picks up the change. Nothing is read from
|
||||
disk while serving a request.
|
||||
- State (bans with their notes, each client's counters, history and lookup
|
||||
answer, the reputation cache, the alerting state) held in memory and kept in
|
||||
readable JSON files, written regularly and at every stop, so a restart loses
|
||||
nothing. Edit a file, or add a rule file, and the running `smallwebwaf` picks
|
||||
up the change. Nothing is read from disk while serving a request.
|
||||
- A small admin endpoint for health checks, listing, adding and lifting bans,
|
||||
and asking why a given address was refused.
|
||||
|
||||
@@ -115,7 +121,9 @@ For each request `smallwebwaf`:
|
||||
- works out who the client really is;
|
||||
- lets it straight through if it is on the bypass list, refuses it if it is on
|
||||
the deny list or currently banned;
|
||||
- looks up its AS number and country, and any cached reputation verdict;
|
||||
- looks up its AS number and country, and refuses it if that country is denied,
|
||||
or is not among the only ones allowed;
|
||||
- checks for a cached reputation verdict;
|
||||
- picks the client's limit percentage from those;
|
||||
- checks the minute, hour and day request counters against the limits, and bans
|
||||
the client if it breaks one;
|
||||
@@ -162,7 +170,7 @@ A rule file is one rule per line: a name, what to match against, what to do, and
|
||||
a regex.
|
||||
|
||||
```
|
||||
env-file path ban (?i)/\.env(\.[a-z]+)?$
|
||||
env-file path ban (?i)^/\.env(\.[a-z]+)?$
|
||||
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
|
||||
```
|
||||
|
||||
@@ -170,19 +178,31 @@ scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
|
||||
rules, the rule file format, the state files, the log fields, the metrics,
|
||||
failure behaviour and the build order.
|
||||
|
||||
## Lookup database
|
||||
## Country and AS number lookup
|
||||
|
||||
AS number and country lookups, and the biased limits that use them, need the
|
||||
free IPinfo Lite database (`ipinfo_lite.mmdb`). You download it with your own
|
||||
IPinfo account, mount it into the container, point `LOOKUP_DB_PATH` at it and
|
||||
refresh it when you choose; `smallwebwaf` never downloads it itself. IPinfo
|
||||
releases it under the Creative Commons Attribution-ShareAlike 4.0 International
|
||||
License and asks for attribution, in its own words on https://ipinfo.io/lite:
|
||||
"The attribution requirements can be met by giving our service credit as your
|
||||
data source. Simply place a link to IPinfo on the website, application, or
|
||||
social media account that uses our data." Its example of such a credit is a link
|
||||
mentioning "IP address data is powered by IPinfo". A service that uses the
|
||||
database through `smallwebwaf` should carry that link.
|
||||
AS number and country lookups, and the country lists and biased limits that use
|
||||
them, are off until you choose one of two sources with `LOOKUP_SOURCE`.
|
||||
|
||||
`LOOKUP_SOURCE=file` reads the free IPinfo Lite database (`ipinfo_lite.mmdb`).
|
||||
You download it with your own IPinfo account, mount the directory that holds it
|
||||
into the container, point `LOOKUP_DB_PATH` at the file and refresh it when you
|
||||
choose; `smallwebwaf` never downloads it itself, and reads it again when you
|
||||
replace it. It has to be the directory rather than the file itself: docker does
|
||||
not show a single mounted file being replaced, so a refresh would go unseen.
|
||||
IPinfo releases it under the Creative Commons Attribution-ShareAlike 4.0
|
||||
International License and asks for attribution, in its own words on
|
||||
https://ipinfo.io/lite: "The attribution requirements can be met by giving our
|
||||
service credit as your data source. Simply place a link to IPinfo on the
|
||||
website, application, or social media account that uses our data." Its example
|
||||
of such a credit is a link mentioning "IP address data is powered by IPinfo". A
|
||||
service that uses the database through `smallwebwaf` should carry that link.
|
||||
|
||||
`LOOKUP_SOURCE=geojs` asks the free GeoJS web service instead, with no account
|
||||
and no file. Every new visitor's address is sent to GeoJS. Each answer is kept
|
||||
for seven days, across restarts, and many addresses are asked about in one
|
||||
request. GeoJS publishes no rate limit but may block a caller it thinks asks too
|
||||
much; while it is not answering, new visitors count as coming from an unknown
|
||||
country, which `EXCLUSIVELY_ALLOWED_COUNTRIES` refuses.
|
||||
|
||||
## Documents
|
||||
|
||||
|
||||
Reference in New Issue
Block a user