smallwebwaf/docs/reputation.md

# Reputation sources

A reputation source judges a client by what others saw it do. None is on by
default. The settings are under "Reputation sources" in
[`settings.md`](settings.md); the reasons for the defaults are under
"Reputation" in [`SPEC.md`](../SPEC.md). The sources are checked in this order,
and a source the client is on refuses it, lowers its limits, or only notes it,
as `SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say.

## Blocklists

`SWWAF_BLOCKLIST_URLS` lists text files of addresses and netblocks, one to a
line, as the Spamhaus DROP list is written. Anything after a `;` or `#` is left
out. An IPv4-mapped address such as `::ffff:192.0.2.0/120` is read as the IPv4
netblock it stands for.

Each list is fetched again `SWWAF_BLOCKLIST_REFRESH` after its last try, one
list at a time. The file of `SWWAF_ASN_LIMIT_PERCENT_URL` is too.
`reputation.json` keeps when each was last tried, so a restart does not fetch a
list again before its time. A fetch fails when:

- the server answers other than `200`;
- the fetch does not finish within a minute;
- the list is longer than 16 MiB;
- a line is not an address or netblock, or, for the AS number file, not
  `AS<number>:<percent>`.

The copy fetched before stays in use, comment lines included. The failure is
logged and raised as a `source_failure` alert. The log, the alerts, the ban
notes and the metrics name each list by its URL, so keep a secret out of it.

The Spamhaus DROP list is The Spamhaus Project's. Its terms, and what they mean
for `SWWAF_BLOCKLIST_REFRESH` and for several instances on one host, are under
"Reputation" in [`SPEC.md`](../SPEC.md). The copy in `reputation.json` keeps the
list's date and copyright lines.

## CrowdSec

While `SWWAF_CROWDSEC_LAPI_URL` is set to the local API of a CrowdSec engine you
run, `smallwebwaf` fetches its decision list every minute. It sends
`SWWAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key` header. The list is kept as a
blocklist is kept. The decisions of the type `ban` on an address or a netblock
are used. Any other, such as a captcha or a country, is left out. A decision
lasts for the `duration` it had left when the engine answered.

A request is refused when a decision still in force names its client. The
client's netblock is banned until the decision ends, with the cause `crowdsec`.
Only a client that sends a request is banned, so a long list does not fill
`bans.json`. Such a ban is never made permanent, and does not lengthen the
netblock's next ban for a broken limit. If you lift a ban while the decision is
in force, the client's next request makes another. Delete the decision in
CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and lift
the ban once the list has been fetched again. A fetch fails when:

- the engine answers other than `200`, a redirect included;
- the fetch does not finish within a minute;
- the answer is longer than 16 MiB;
- the answer is not a list of decisions.

## DNS blocklists

Each zone in `SWWAF_DNSBL_ZONES` is asked about a client's own address in the
background, the first time the client is seen. The request goes on at once, and
so does every request from the client until the zone has answered. The name
asked about is the one RFC 5782 gives: the numbers of an IPv4 address reversed,
or the 32 hex digits of an IPv6 address reversed. `192.0.2.99` is asked about as
`99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` lists the client, but
one in `127.255.255.0/24` is Spamhaus refusing the query. A name that does not
exist does not list the client. These are failures:

- an answer in `127.255.255.0/24`;
- an answer outside `127.0.0.0/8`;
- a resolver error;
- no answer within `SWWAF_REPUTATION_TIMEOUT`.

A failure is logged and alerted, and the zone is not asked again for a minute.
Each verdict is kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and in
`reputation.json`. Up to 100,000 verdicts are kept; past that, the one fetched
longest ago goes first. At most 1,000 queries are under way at once.

The key of a Spamhaus zone under `dq.spamhaus.net` is its first label.
`smallwebwaf` shows `********` in its place wherever it names the zone. Only
`reputation.json` keeps the zone with its key.

## AbuseIPDB

While `SWWAF_ABUSEIPDB_KEY` is set, `smallwebwaf` asks AbuseIPDB's check
endpoint for the abuse confidence score of a client, from 0 to 100. Only a
client that has committed an offence is checked. An offence is one of:

- a broken limit;
- a `ban` rule match or a trap path;
- a refusal by a `block` rule, the Core Rule Set or a token check.

The check runs in the background, at the client's first request after the
offence that no ban refuses, so no request waits. The score applies to the
client's later requests. For an IPv6 client, AbuseIPDB is asked about the
address of the request that set off the check, and the score is used for the
whole group. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and
`SWWAF_REPUTATION_ACTION` says what is done. Each score is kept for
`SWWAF_REPUTATION_CACHE_TTL`, up to 100,000 scores.

At most `SWWAF_ABUSEIPDB_DAILY_BUDGET` checks are made in a day, counted in UTC
and kept in `reputation.json`. A restart does not make the budget whole again.
The check that spends the last of the budget is logged and alerted. No client is
checked until the day ends. A check fails when:

- AbuseIPDB answers other than `200`;
- the answer gives no score;
- no answer comes within `SWWAF_REPUTATION_TIMEOUT`.

A failure is logged and alerted, and no client is checked for a minute. The key
is sent in the `Key` header and nowhere else, and is never logged.