AS number and country looked up for every client (closes #95)
check / check (push) Waiting to run
check / check (push) Waiting to run
GeoJS's geo.json is asked about every new visitor unless SWWAF_LOOKUP_SOURCE is off. A request waits for its client's first answer only while a country list or SWWAF_ADD_LOOKUP_HEADERS needs it; otherwise the answer reaches the client's history and ban notes when it comes. The AS number and name go beside the country in the request log, history, ban notes, alerts and lookups.json, with metrics by AS number; 64512 counts as unknown. A client's own X-Client-ASN and X-Client-Country never reach the app, whatever the setting says, and make example-app sends no address to GeoJS. Judgement call: AS numbers are written AS64496, as SPEC's settings write them. Judgement call: SWWAF_LOOKUP_TIMEOUT is added, default 1s, and cannot be off. Model: opus-5-5
This commit is contained in:
@@ -22,26 +22,28 @@ fields, which come a little later, and the metrics endpoint and the header size
|
||||
and the idle time as settings, which come last in it. So are the four parts of
|
||||
the stage after it: the rule files, with the bans for a clear sign of attack,
|
||||
the other admin endpoints, alerts to all three destinations, a JSON webhook,
|
||||
Slack and ntfy, and remote log sending. `smallwebwaf` passes each request to the
|
||||
app and the app's answer back, unchanged, within its timeouts and size limits,
|
||||
works out each client's address, bans a client that sends too many requests, not
|
||||
counting those for the paths you choose, refuses a client that comes from a
|
||||
country you refuse or from a network you refuse, lets the networks you choose
|
||||
through, checks each request against the rule files and bans a client whose
|
||||
request is a clear sign of attack, keeps its bans, each client's counters and
|
||||
history, and GeoJS's answers in JSON files across restarts, takes in your edits
|
||||
of those files, such as a ban you make, keep or lift, and of the rule files
|
||||
while it runs, writes a JSON log line for every request, sends its log lines to
|
||||
a syslog server too if you name one, sends an alert to a webhook, to Slack and
|
||||
to ntfy, each if you name one, for each ban it makes or makes permanent, for
|
||||
GeoJS failing and for a rule file or state file with an error, serves Prometheus
|
||||
metrics to a scraper that holds the metrics token, lets an admin who holds the
|
||||
admin token list, add and lift bans and ask what it knows of a client, and in
|
||||
`observe` mode passes on the requests it would refuse, logging what it would
|
||||
have done with them. It comes as the image the app's own image is built on. The
|
||||
rest of the design comes after that, in the order of the build order in
|
||||
[`SPEC.md`](SPEC.md). The survey of existing tools that led to the design is in
|
||||
[`EVALUATION.md`](EVALUATION.md).
|
||||
Slack and ntfy, and remote log sending. So is the first part of the stage after
|
||||
that: the AS number and country of every client, looked up through GeoJS.
|
||||
`smallwebwaf` passes each request to the app and the app's answer back,
|
||||
unchanged, within its timeouts and size limits, works out each client's address,
|
||||
looks up its AS number and country unless you switch that off, bans a client
|
||||
that sends too many requests, not counting those for the paths you choose,
|
||||
refuses a client that comes from a country you refuse or from a network you
|
||||
refuse, lets the networks you choose through, checks each request against the
|
||||
rule files and bans a client whose request is a clear sign of attack, keeps its
|
||||
bans, each client's counters and history, and GeoJS's answers in JSON files
|
||||
across restarts, takes in your edits of those files, such as a ban you make,
|
||||
keep or lift, and of the rule files while it runs, writes a JSON log line for
|
||||
every request, sends its log lines to a syslog server too if you name one, sends
|
||||
an alert to a webhook, to Slack and to ntfy, each if you name one, for each ban
|
||||
it makes or makes permanent, for GeoJS failing and for a rule file or state file
|
||||
with an error, serves Prometheus metrics to a scraper that holds the metrics
|
||||
token, lets an admin who holds the admin token list, add and lift bans and ask
|
||||
what it knows of a client, and in `observe` mode passes on the requests it would
|
||||
refuse, logging what it would have done with them. It comes as the image the
|
||||
app's own image is built on. The rest of the design comes after that, in the
|
||||
order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
|
||||
that led to the design is in [`EVALUATION.md`](EVALUATION.md).
|
||||
|
||||
## Getting started
|
||||
|
||||
@@ -115,15 +117,15 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
||||
is not counted for the rate limits. A ban sets the client's counters back to
|
||||
zero. Each ban carries notes for deciding whether to lift it: the limit, its
|
||||
window and the requests counted in it, the request that broke it, the client's
|
||||
country when it was looked up, the netblock's requests since it was first
|
||||
seen, how many of them the ban has refused, and how many bans the netblock had
|
||||
before, for a broken limit, for a clear sign of attack and by an admin. At
|
||||
most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept, past, active and
|
||||
permanent; past that, the earliest such ban of the netblock that has gone
|
||||
longest without a request is dropped first. The bans whose cause is `admin`,
|
||||
those you make or keep, are kept besides, and never dropped. `bans.json` shows
|
||||
the bans and their notes, a restart lifts none, and you make, keep or lift a
|
||||
ban by editing it (see "State files" below).
|
||||
AS number, AS name and country once they are looked up, the netblock's
|
||||
requests since it was first seen, how many of them the ban has refused, and
|
||||
how many bans the netblock had before, for a broken limit, for a clear sign of
|
||||
attack and by an admin. At most `SWWAF_MAX_BANS` bans `smallwebwaf` made are
|
||||
kept, past, active and permanent; past that, the earliest such ban of the
|
||||
netblock that has gone longest without a request is dropped first. The bans
|
||||
whose cause is `admin`, those you make or keep, are kept besides, and never
|
||||
dropped. `bans.json` shows the bans and their notes, a restart lifts none, and
|
||||
you make, keep or lift a ban by editing it (see "State files" below).
|
||||
- Checks each request against the rules of the rule files (see "Rule files"
|
||||
below) after the rate limits, and before its body is read. A `log` rule that
|
||||
matches is noted in the log line; a `block` rule refuses the request with
|
||||
@@ -139,16 +141,23 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
||||
back to zero, and does not make the netblock's next ban for a broken limit
|
||||
longer. Its notes give the id and the target of the rule that matched in place
|
||||
of the limit.
|
||||
- Looks up the AS number and country of every client through GeoJS, after the
|
||||
static lists and bans, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Country and
|
||||
AS number lookup" below), for the request log, the client's history, the notes
|
||||
of its bans, their alerts and the metrics. A request waits for its client's
|
||||
first answer only while a setting acts on it, a country list or
|
||||
`SWWAF_ADD_LOOKUP_HEADERS`. Otherwise it goes on at once, and the answer
|
||||
reaches the client's history and the notes of its bans when it comes, but not
|
||||
the log lines of the requests that went on without it, nor the alerts already
|
||||
raised for those bans.
|
||||
- Refuses a request from a country you refuse with `SWWAF_BAN_RESPONSE`, as soon
|
||||
as the client's country is known and before its body is read; such a request
|
||||
is not counted for the rate limits. While one of the country lists below is
|
||||
set, each client's country is looked up through GeoJS (see "Country and AS
|
||||
number lookup" below); with neither set, no visitor's address leaves the host.
|
||||
A client on a private, loopback or link-local address has no country and is
|
||||
never looked up: `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless it is
|
||||
in `SWWAF_ALLOW_NETS`, and `SWWAF_DENIED_COUNTRIES` does not refuse it.
|
||||
is not counted for the rate limits. A client on a private, loopback or
|
||||
link-local address has no country and is never looked up:
|
||||
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless it is in
|
||||
`SWWAF_ALLOW_NETS`, and `SWWAF_DENIED_COUNTRIES` does not refuse it.
|
||||
- Checks the client's own address against the static lists, the three netblock
|
||||
settings below, before anything else, its country included. A client in
|
||||
settings below, before anything else, its lookup included. A client in
|
||||
`SWWAF_ALLOW_NETS` skips bans, the country lists, the rate limits and the rule
|
||||
files, and is not looked up; the timeouts and size limits still apply. A
|
||||
client in `SWWAF_DENY_NETS` is refused with `SWWAF_BAN_RESPONSE` before its
|
||||
@@ -267,6 +276,22 @@ effective settings are logged at start.
|
||||
percent-encoded, such as a space, is written percent-encoded in a prefix, as
|
||||
in `/my%20files/`, and there are no wildcards: `*` is a character like any
|
||||
other.
|
||||
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): where each client's AS number and
|
||||
country are looked up: `geojs`, the GeoJS web service, which is then told the
|
||||
address of every new visitor, or `off`, which looks up no client and sends no
|
||||
address to GeoJS. `file`, for the IPinfo Lite database, comes with
|
||||
https://git.eeqj.de/sneak/smallwebwaf/issues/22. With `off`, a country list
|
||||
that is not empty, or `SWWAF_ADD_LOOKUP_HEADERS` set to `true`, stops the
|
||||
start, with a message naming it and `SWWAF_LOOKUP_SOURCE`.
|
||||
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
|
||||
client's first answer while a setting acts on it, and how long a request to
|
||||
GeoJS may take before it is abandoned.
|
||||
- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the
|
||||
client's AS number, such as `AS64496`, in `X-Client-ASN`, and its country in
|
||||
`X-Client-Country`, leaving out one that is unknown. A request then waits for
|
||||
its client's first answer, as it does while a country list is set. Whatever
|
||||
this setting says, any `X-Client-ASN` or `X-Client-Country` the client sent,
|
||||
in any case, is removed, so that the app never receives a client's own.
|
||||
- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused,
|
||||
for example `cn,ru,kp`.
|
||||
- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
|
||||
@@ -318,8 +343,9 @@ effective settings are logged at start.
|
||||
shorter than 32 characters stops the start. The settings logged at start show
|
||||
`********` in its place. Given as a file, it can be kept out of the app's
|
||||
reach (see "Settings given as files" below).
|
||||
- `SWWAF_METRICS_TOP_N` (default `50`): how many countries get series of their
|
||||
own in the metrics by country; the others are counted as `other`.
|
||||
- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many
|
||||
countries get series of their own in the metrics by AS number and by country;
|
||||
the others are counted as `other`.
|
||||
- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the
|
||||
rule files. A directory that does not exist stops the start.
|
||||
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file, and checks
|
||||
@@ -384,14 +410,13 @@ ISO 3166-1 assigns today, and `xk` for Kosovo, in either case (`de` and `DE` are
|
||||
the same); any other code, such as `nk` (North Korea is `kp`) or the withdrawn
|
||||
`su`, stops the start, and so does a code on both country lists. `off` switches
|
||||
a timeout, a size limit, a rate limit, `SWWAF_ALERT_COOLDOWN` or
|
||||
`SWWAF_ALERT_MAX_PER_HOUR` off; `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, the ban
|
||||
settings, the state settings, `SWWAF_METRICS_TOP_N` and
|
||||
`SWWAF_LOG_REMOTE_BUFFER` cannot be off.
|
||||
`SWWAF_ALERT_MAX_PER_HOUR` off; `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
|
||||
`SWWAF_LOOKUP_TIMEOUT`, the ban settings, the state settings,
|
||||
`SWWAF_METRICS_TOP_N` and `SWWAF_LOG_REMOTE_BUFFER` cannot be off.
|
||||
|
||||
Several limits are fixed rather than settings. At most 20,000 clients are kept,
|
||||
with their counters and history, and an IPv6 client is counted by its /64. A new
|
||||
client waits at most a second for its country, and at most 100,000 answers from
|
||||
GeoJS are kept, for 7 days each.
|
||||
with their counters and history, and an IPv6 client is counted by its /64. At
|
||||
most 100,000 answers from GeoJS are kept, for 7 days each.
|
||||
|
||||
### Settings given as files
|
||||
|
||||
@@ -431,12 +456,13 @@ and for the container, `-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro` and
|
||||
refused ones included:
|
||||
|
||||
```
|
||||
{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
|
||||
{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
|
||||
```
|
||||
|
||||
A field that does not apply to a request is left out of its line, apart from
|
||||
`type`, the fields from `time` to `user_agent`, `request_id`, `peer_ip`,
|
||||
`client_group`, `country`, `action` and `duration_total`, which every line has.
|
||||
`client_group`, `asn`, `as_name`, `country`, `action` and `duration_total`,
|
||||
which every line has.
|
||||
|
||||
- `time` is when the request arrived, in UTC. `instance` is
|
||||
`SWWAF_INSTANCE_NAME`. `scheme` is the `X-Forwarded-Proto` a trusted proxy
|
||||
@@ -448,11 +474,14 @@ A field that does not apply to a request is left out of its line, apart from
|
||||
`X-Forwarded-For` header as received, several lines of it joined with `, `.
|
||||
`client_group` is the client as the rate limits count it: its IPv4 address as
|
||||
a /32, or the /64 of its IPv6 address.
|
||||
- `country` is the client's country as GeoJS places it. It is empty with neither
|
||||
country list set, for a client in `SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`, for
|
||||
a client on a private, loopback or link-local address, when GeoJS cannot place
|
||||
the client or has not answered in time, and for a request whose client a ban
|
||||
covers, even when the client's country is known.
|
||||
- `asn`, `as_name` and `country` are the client's AS number, such as `AS64496`,
|
||||
the name of that AS, and its country, as GeoJS gives them. Each is empty when
|
||||
`SWWAF_LOOKUP_SOURCE` is `off`, for a client in `SWWAF_ALLOW_NETS` or
|
||||
`SWWAF_DENY_NETS`, for a client on a private, loopback or link-local address,
|
||||
when GeoJS has not answered by the time the request went on, and for a request
|
||||
whose client a ban covers, even when the answer is known. `asn` and `as_name`
|
||||
are empty too when GeoJS knows no AS number for the client, which it gives as
|
||||
64512, and `country` when GeoJS cannot place the client.
|
||||
- `content_type` is the request's `Content-Type`, and `content_length` the
|
||||
length the request announced for its body, which is left out for none or zero.
|
||||
- `request_headers` are the request's headers that `SWWAF_LOG_REQUEST_HEADERS`
|
||||
@@ -593,6 +622,8 @@ is sent on one line:
|
||||
"ban_expires": "2026-10-06T13:00:00.123Z",
|
||||
"cause": "limit",
|
||||
"notes": {
|
||||
"asn": "",
|
||||
"as_name": "",
|
||||
"country": "",
|
||||
"limit": 1000,
|
||||
"window": "minute",
|
||||
@@ -622,8 +653,9 @@ is sent on one line:
|
||||
UTC.
|
||||
- `client` is the address of the client whose request raised the alert, and
|
||||
`netblock` the netblock of the ban; both are empty for `source_failure` and
|
||||
`file_error`. `asn` and `as_name` are empty until AS numbers are looked up,
|
||||
and `country` is, for a ban, the client's country as the ban's notes give it.
|
||||
`file_error`. `asn`, `as_name` and `country` are, for a ban, the client's as
|
||||
the ban's notes give them when the alert is raised: empty, as in this alert,
|
||||
when GeoJS had not answered about the client by then.
|
||||
- `reason` is a short sentence; for a ban, the ban's `reason` in `bans.json`.
|
||||
- `detail` is what is particular to the event: for a ban, its `cause`, when it
|
||||
ends as `ban_expires`, in the form the request log gives it, and its `notes`,
|
||||
@@ -719,14 +751,15 @@ entries by client address, but for the alerts waiting, with times in UTC.
|
||||
as `matched the rule env-file`; for yours, what you wrote. Its `lifted` is
|
||||
when you lifted it, and is left out until you do.
|
||||
- `clients.json`: each client's two buckets in the minute, the hour and the day,
|
||||
and its history: when it was first and last seen, its country as last looked
|
||||
up and when, its requests, how many were forwarded and how many refused (one
|
||||
`smallwebwaf` answered at its own endpoints is neither, unless it was refused
|
||||
with `401` for a missing or wrong token), the body bytes in each direction,
|
||||
its responses by status class and its offences by kind. Each client is on a
|
||||
line of its own, so `grep` shows everything about one.
|
||||
- `lookups.json`: GeoJS's answers, one to a line, with when GeoJS gave each and
|
||||
when it was last used.
|
||||
and its history: when it was first and last seen, its AS number, AS name and
|
||||
country as last looked up and when GeoJS gave them, its requests, how many
|
||||
were forwarded and how many refused (one `smallwebwaf` answered at its own
|
||||
endpoints is neither, unless it was refused with `401` for a missing or wrong
|
||||
token), the body bytes in each direction, its responses by status class and
|
||||
its offences by kind. Each client is on a line of its own, so `grep` shows
|
||||
everything about one.
|
||||
- `lookups.json`: GeoJS's answers, one to a line, each with the client's AS
|
||||
number, AS name and country, when GeoJS gave it and when it was last used.
|
||||
- `alerts.json`: the state of the alerts (see "Alerts" above), indented to be
|
||||
read: under `cooldowns`, for each event and netblock, or event and `file` or
|
||||
`source`, or event alone, when the last alert was sent, `sent`, and the
|
||||
@@ -764,8 +797,8 @@ client's `client`, or the `start` of a window in which it has requests; an
|
||||
answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or
|
||||
`answered`; a cooldown's `event` or `sent`; an alert waiting's `event` or
|
||||
`time`. So does a ban whose `cause` is not `limit`, `attack` or `admin`, and
|
||||
alerts waiting for a destination that is not `webhook`, `slack` or `ntfy`. The
|
||||
AS number and AS name come with their lookup.
|
||||
alerts waiting for a destination that is not `webhook`, `slack` or `ntfy`. An
|
||||
answer's `asn` or `as_name` left out reads as empty.
|
||||
|
||||
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
||||
a state file as soon as you save it: what the file then holds replaces what
|
||||
@@ -934,11 +967,16 @@ scraped, and keeps this one as `exported_instance` unless the scrape sets
|
||||
series of their own, and the others are counted as `other`. A country that
|
||||
drops out of them loses its series, and its later requests count as `other`;
|
||||
one that comes into them gets a series that counts from then on.
|
||||
- `smallwebwaf_asn_requests_total`, `smallwebwaf_asn_request_bytes_total` and
|
||||
`smallwebwaf_asn_response_bytes_total`, by `asn`, the client's AS number, for
|
||||
the requests whose client's AS number is known, with the `SWWAF_METRICS_TOP_N`
|
||||
busiest AS numbers kept as the countries are.
|
||||
- `smallwebwaf_geojs_requests_total`: the requests to GeoJS;
|
||||
`smallwebwaf_geojs_failures_total`: those that failed, an answer that leaves
|
||||
out an address asked about included; and `smallwebwaf_geojs_unanswered_total`:
|
||||
the requests whose client counted as coming from an unknown country because
|
||||
GeoJS had not answered about it in time.
|
||||
the requests that needed their client's answer, for a country list or
|
||||
`SWWAF_ADD_LOOKUP_HEADERS`, and went on without it because GeoJS had not given
|
||||
it in time.
|
||||
- `smallwebwaf_tracked_clients`: the clients in the table of clients.
|
||||
- `smallwebwaf_state_file_writes_total`,
|
||||
`smallwebwaf_state_file_write_failures_total`,
|
||||
@@ -989,9 +1027,10 @@ request that carries the token as `Authorization: Bearer <token>`:
|
||||
`bans.json` does, and answers `404` when no ban on it is active.
|
||||
- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client at
|
||||
the address `<ip>`: under `client`, the client as `clients.json` holds it,
|
||||
with its counters and its history, which holds its country as last looked up
|
||||
and its offences, or `null` when the table of clients does not hold it; and
|
||||
under `bans`, every ban on a netblock the address is in, with its notes.
|
||||
with its counters and its history, which holds its AS number, AS name and
|
||||
country as last looked up and its offences, or `null` when the table of
|
||||
clients does not hold it; and under `bans`, every ban on a netblock the
|
||||
address is in, with its notes.
|
||||
|
||||
The ban endpoints answer with the bans listed, made or lifted, under `bans`,
|
||||
each as an entry of `bans.json` (see "State files" above), and a ban they make
|
||||
@@ -1247,49 +1286,51 @@ the metrics, failure behaviour and the build order.
|
||||
|
||||
## Country and AS number lookup
|
||||
|
||||
So far `smallwebwaf` looks up only the country, only through GeoJS, and only
|
||||
while `SWWAF_DENIED_COUNTRIES` or `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set:
|
||||
then the address of every new visitor is sent to GeoJS, except a visitor in
|
||||
`SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS` and one whose netblock a ban covers, and
|
||||
with neither set, none is. An IPv6 visitor is asked about by the first address
|
||||
of its /64. A new visitor waits at most a second for its answer, and without one
|
||||
counts as coming from an unknown country until the answer arrives. The addresses
|
||||
waiting are asked about together, up to 200 in one request, one request at a
|
||||
time; at most 10,000 visitors wait, and one more counts as coming from an
|
||||
unknown country until there is room. While GeoJS fails, visitors with a kept
|
||||
answer are unaffected and new ones count as coming from an unknown country.
|
||||
GeoJS is then left alone for a second, twice as long after each further failure
|
||||
up to five minutes, and asked again by the next request that needs it.
|
||||
`smallwebwaf` looks up the AS number and country of every client through GeoJS,
|
||||
a free web service that needs no account and no file, for the request log, the
|
||||
client's history, the notes of its bans, their alerts and the metrics, and for
|
||||
the country lists when you set them. This means that GeoJS is told the address
|
||||
of every new visitor, whether or not a setting uses the answer, unless you set
|
||||
`SWWAF_LOOKUP_SOURCE=off`. The only visitors it is not told about are those in
|
||||
`SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`, those whose netblock a ban covers, and
|
||||
those on a private, loopback or link-local address. An IPv6 visitor is asked
|
||||
about by the first address of its /64. Each answer is kept for seven days, in
|
||||
memory and in `lookups.json`, so that it survives a restart, and a visitor whose
|
||||
answer is kept is not asked about again.
|
||||
|
||||
In the full design, `smallwebwaf` looks up the AS number and country of every
|
||||
client, for the request log, the metrics and the ban notes, and for the country
|
||||
lists and biased limits when you set them. It works with no setup: by default it
|
||||
asks the free GeoJS web service, which needs no account and no file. This means
|
||||
that, by default, the address of every new visitor is sent to GeoJS. Each answer
|
||||
is kept for seven days, in memory and in `lookups.json`, so that it survives a
|
||||
restart, and many addresses are asked about in one request. GeoJS publishes no
|
||||
rate limit but may block a caller it thinks asks too much; while it is not
|
||||
answering, new visitors count as coming from an unknown country, which
|
||||
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses.
|
||||
A request waits for its client's first answer only while a setting acts on it
|
||||
before the request goes on: a country list, or `SWWAF_ADD_LOOKUP_HEADERS`. A new
|
||||
visitor then waits up to `SWWAF_LOOKUP_TIMEOUT`, a second by default, and
|
||||
without an answer counts as coming from an unknown country until the answer
|
||||
arrives. Otherwise no request waits: it goes on at once and is logged without
|
||||
the answer, which reaches the client's history and the notes of its bans when it
|
||||
comes. The addresses waiting are asked about together, up to 200 in one request,
|
||||
one request at a time; at most 10,000 visitors wait, and one more is not asked
|
||||
about until there is room, counting meanwhile as coming from an unknown country.
|
||||
GeoJS publishes no rate limit but may block a caller it thinks asks too much.
|
||||
While GeoJS fails, visitors with a kept answer are unaffected and new ones count
|
||||
as coming from an unknown country, which `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`
|
||||
refuses. GeoJS is then left alone for a second, twice as long after each further
|
||||
failure up to five minutes, and asked again by the next request from a visitor
|
||||
without an answer.
|
||||
|
||||
To keep your visitors' addresses on your own host, set
|
||||
`SWWAF_LOOKUP_SOURCE=off`, or use the database file instead of GeoJS:
|
||||
`SWWAF_LOOKUP_SOURCE=file` reads the free IPinfo Lite database
|
||||
(`ipinfo_lite.mmdb`). `SWWAF_LOOKUP_SOURCE` comes in milestone 3 or later (see
|
||||
the build order in [`SPEC.md`](SPEC.md)); until then GeoJS is asked only while a
|
||||
country list is set. You download the database with your own IPinfo account,
|
||||
mount the directory that holds it into the container, point
|
||||
`SWWAF_LOOKUP_DB_PATH` at the file and refresh it when you choose; `smallwebwaf`
|
||||
never downloads it itself, and reads it again when you replace it. It has to be
|
||||
the directory rather than the file itself: docker does not show a single mounted
|
||||
file being replaced, so a refresh would go unseen. IPinfo releases it under the
|
||||
Creative Commons Attribution-ShareAlike 4.0 International License and asks for
|
||||
attribution, in its own words on https://ipinfo.io/lite: "The attribution
|
||||
requirements can be met by giving our service credit as your data source. Simply
|
||||
place a link to IPinfo on the website, application, or social media account that
|
||||
uses our data." Its example of such a credit is a link mentioning "IP address
|
||||
data is powered by IPinfo". A service that uses the database through
|
||||
`smallwebwaf` should carry that link.
|
||||
(`ipinfo_lite.mmdb`), and comes with
|
||||
https://git.eeqj.de/sneak/smallwebwaf/issues/22. You download the database with
|
||||
your own IPinfo account, mount the directory that holds it into the container,
|
||||
point `SWWAF_LOOKUP_DB_PATH` at the file and refresh it when you choose;
|
||||
`smallwebwaf` never downloads it itself, and reads it again when you replace it.
|
||||
It has to be the directory rather than the file itself: docker does not show a
|
||||
single mounted file being replaced, so a refresh would go unseen. IPinfo
|
||||
releases it under the Creative Commons Attribution-ShareAlike 4.0 International
|
||||
License and asks for attribution, in its own words on https://ipinfo.io/lite:
|
||||
"The attribution requirements can be met by giving our service credit as your
|
||||
data source. Simply place a link to IPinfo on the website, application, or
|
||||
social media account that uses our data." Its example of such a credit is a link
|
||||
mentioning "IP address data is powered by IPinfo". A service that uses the
|
||||
database through `smallwebwaf` should carry that link.
|
||||
|
||||
Neither source can place a private address, so a client on one, such as a
|
||||
visitor on your local network, another container or your monitoring, has no
|
||||
@@ -1323,8 +1364,9 @@ addresses are never sent to GeoJS.
|
||||
and which ban `smallwebwaf` made is dropped when `SWWAF_MAX_BANS` are held.
|
||||
- `internal/rules`: reads the rule files at start and again as they change, and
|
||||
tells which of their rules a request matches.
|
||||
- `internal/lookup`: looks up each client's country through GeoJS, and keeps the
|
||||
answers.
|
||||
- `internal/lookup`: looks up each client's AS number and country through GeoJS,
|
||||
keeps the answers, and hands each new one to the proxy, which adds it to the
|
||||
client's history and to the notes of its bans.
|
||||
- `internal/ratelimit`: the table of clients: counts each client's requests,
|
||||
tells when one takes it over a rate limit, and keeps each client's history.
|
||||
- `internal/state`: reads the state files at start, takes in an admin's edit of
|
||||
|
||||
Reference in New Issue
Block a user