Bans an admin makes or lifts: the admin cause, a reason, lifted bans kept (closes #86)
check / check (push) Successful in 3m27s
check / check (push) Successful in 3m27s
A bans.json entry without a cause gets the cause admin, written back so. Bans whose cause is admin are never dropped and do not count toward SWWAF_MAX_BANS, so setting a ban's cause to admin keeps it. Bans smallwebwaf makes get a reason: the limit broken or the rule matched. A lifted ban refuses nothing, is kept, and makes no later ban longer. smallwebwaf_bans_made_total counts admin bans an edit adds while running; earlier_bans counts admin in place of without_cause. Judgement call: lifted lifts at once, whatever time it gives. Judgement call: a lifted ban still counts in earlier_bans. Known gap: a ban dropped from behind an admin's ban on its netblock leaves that netblock's later earlier_bans. Model: opus-5-5
This commit was merged in pull request #88.
This commit is contained in:
@@ -13,29 +13,30 @@ JSON log line for every request.
|
||||
|
||||
Status: the first two milestones are built
|
||||
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
|
||||
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are eight parts of
|
||||
milestone 3: the static lists, the bans that broken rate limits lead to, the
|
||||
JSON state files with your edits taken in while it runs and the paths the rate
|
||||
limits do not count, which come next in the build order, `observe` mode and the
|
||||
rest of the request log's fields, which come a little later, and the metrics
|
||||
endpoint and the header size and the idle time as settings, which come last in
|
||||
it. So are two parts of the stage after it: the rule files, the first part, with
|
||||
the bans for a clear sign of attack, and remote log sending. `smallwebwaf`
|
||||
passes each request to the app and the app's answer back, unchanged, within its
|
||||
timeouts and size limits, works out each client's address, bans a client that
|
||||
sends too many requests, not counting those for the paths you choose, refuses a
|
||||
client that comes from a country you refuse or from a network you refuse, lets
|
||||
the networks you choose through, checks each request against the rule files and
|
||||
bans a client whose request is a clear sign of attack, keeps its bans, each
|
||||
client's counters and history, and GeoJS's answers in JSON files across
|
||||
restarts, takes in your edits of those files and of the rule files while it
|
||||
runs, writes a JSON log line for every request, sends its log lines to a syslog
|
||||
server too if you name one, serves Prometheus metrics to a scraper that holds
|
||||
the metrics token, and in `observe` mode passes on the requests it would refuse,
|
||||
logging what it would have done with them. It comes as the image the app's own
|
||||
image is built on. The rest of the design comes after that, in the order of the
|
||||
build order in [`SPEC.md`](SPEC.md). The survey of existing tools that led to
|
||||
the design is in [`EVALUATION.md`](EVALUATION.md).
|
||||
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are nine parts of
|
||||
milestone 3: the static lists, the bans that broken rate limits lead to, the ban
|
||||
ledger with the bans you make, keep and lift, the JSON state files with your
|
||||
edits taken in while it runs and the paths the rate limits do not count, which
|
||||
come next in the build order, `observe` mode and the rest of the request log's
|
||||
fields, which come a little later, and the metrics endpoint and the header size
|
||||
and the idle time as settings, which come last in it. So are two parts of the
|
||||
stage after it: the rule files, the first part, with the bans for a clear sign
|
||||
of attack, and remote log sending. `smallwebwaf` passes each request to the app
|
||||
and the app's answer back, unchanged, within its timeouts and size limits, works
|
||||
out each client's address, bans a client that sends too many requests, not
|
||||
counting those for the paths you choose, refuses a client that comes from a
|
||||
country you refuse or from a network you refuse, lets the networks you choose
|
||||
through, checks each request against the rule files and bans a client whose
|
||||
request is a clear sign of attack, keeps its bans, each client's counters and
|
||||
history, and GeoJS's answers in JSON files across restarts, takes in your edits
|
||||
of those files, such as a ban you make, keep or lift, and of the rule files
|
||||
while it runs, writes a JSON log line for every request, sends its log lines to
|
||||
a syslog server too if you name one, serves Prometheus metrics to a scraper that
|
||||
holds the metrics token, and in `observe` mode passes on the requests it would
|
||||
refuse, logging what it would have done with them. It comes as the image the
|
||||
app's own image is built on. The rest of the design comes after that, in the
|
||||
order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
|
||||
that led to the design is in [`EVALUATION.md`](EVALUATION.md).
|
||||
|
||||
## Getting started
|
||||
|
||||
@@ -111,11 +112,13 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
||||
window and the requests counted in it, the request that broke it, the client's
|
||||
country when it was looked up, the netblock's requests since it was first
|
||||
seen, how many of them the ban has refused, and how many bans the netblock had
|
||||
before, for a broken limit, for a clear sign of attack and without a cause. At
|
||||
most `SWWAF_MAX_BANS` bans are kept, past, active and permanent; past that,
|
||||
the earliest ban of the netblock that has gone longest without a request is
|
||||
dropped first. `bans.json` shows the bans and their notes, a restart lifts
|
||||
none, and you add or lift a ban by editing it (see "State files" below).
|
||||
before, for a broken limit, for a clear sign of attack and by an admin. At
|
||||
most `SWWAF_MAX_BANS` bans `smallwebwaf` made are kept, past, active and
|
||||
permanent; past that, the earliest such ban of the netblock that has gone
|
||||
longest without a request is dropped first. The bans whose cause is `admin`,
|
||||
those you make or keep, are kept besides, and never dropped. `bans.json` shows
|
||||
the bans and their notes, a restart lifts none, and you make, keep or lift a
|
||||
ban by editing it (see "State files" below).
|
||||
- Checks each request against the rules of the rule files (see "Rule files"
|
||||
below) after the rate limits, and before its body is read. A `log` rule that
|
||||
matches is noted in the log line; a `block` rule refuses the request with
|
||||
@@ -263,8 +266,8 @@ it, and the effective settings are logged at start.
|
||||
would be longer is permanent instead.
|
||||
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
||||
attack.
|
||||
- `SWWAF_MAX_BANS` (default `5000`): the most bans kept, past, active and
|
||||
permanent.
|
||||
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
|
||||
kept, past, active and permanent. The bans you make or keep are kept besides.
|
||||
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
|
||||
an IPv4 client that a ban covers, such as `24` to ban the surrounding /24. An
|
||||
IPv6 ban covers the client's /64.
|
||||
@@ -458,9 +461,14 @@ them.
|
||||
[`SPEC.md`](SPEC.md) describes. Each has a top-level `version`, 1, and lists its
|
||||
entries by client address, with times in UTC.
|
||||
|
||||
- `bans.json`: every ban with its notes, indented to be read; a permanent ban's
|
||||
`expires` is `null`, and a ban `smallwebwaf` made has the `cause` `limit` for
|
||||
a broken rate limit or `attack` for a clear sign of attack.
|
||||
- `bans.json`: every ban with its notes, indented to be read. A permanent ban's
|
||||
`expires` is `null`. A ban's `cause` is `limit` for a broken rate limit or
|
||||
`attack` for a clear sign of attack, for a ban `smallwebwaf` made, and `admin`
|
||||
for one you made or keep. Its `reason` is a short text: for a ban
|
||||
`smallwebwaf` made, the limit broken, such as
|
||||
`requests per minute over the limit of 1000`, or the rule that matched, such
|
||||
as `matched the rule env-file`; for yours, what you wrote. Its `lifted` is
|
||||
when you lifted it, and is left out until you do.
|
||||
- `clients.json`: each client's two buckets in the minute, the hour and the day,
|
||||
and its history: when it was first and last seen, its country as last looked
|
||||
up and when, its requests, how many were forwarded and how many refused (one
|
||||
@@ -492,8 +500,8 @@ without a field it needs, named with the entry's place in the file: a ban's
|
||||
`netblock`, `start` or `expires`, which is `null` for a permanent ban; a
|
||||
client's `client`, or the `start` of a window in which it has requests; an
|
||||
answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or
|
||||
`answered`. So does a ban whose `cause` is neither `limit` nor `attack`. The AS
|
||||
number and AS name come with their lookup.
|
||||
`answered`. So does a ban whose `cause` is not `limit`, `attack` or `admin`. The
|
||||
AS number and AS name come with their lookup.
|
||||
|
||||
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
||||
a state file as soon as you save it: what the file then holds replaces what
|
||||
@@ -511,10 +519,12 @@ before the editor has finished writing it. Mend the `.bad` file and move it
|
||||
back. A file you remove is written again at its next write.
|
||||
|
||||
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
|
||||
and its `expires`, `null` for a ban that never ends; its `cause` and its `notes`
|
||||
may be left out. A ban whose `cause` is `attack` becomes permanent at the first
|
||||
request it refuses; one without a cause does not. This `bans.json` bans
|
||||
`203.0.113.0/24` for good:
|
||||
and its `expires`, `null` for a ban that never ends; its `reason` and its
|
||||
`notes` may be left out, and so may its `cause`, which is then `admin`, and is
|
||||
written so at the file's next write. A ban whose `cause` is `admin` is never
|
||||
dropped and does not count toward `SWWAF_MAX_BANS`. A ban whose `cause` is
|
||||
`attack` becomes permanent at the first request it refuses; one whose `cause` is
|
||||
`admin` does not. This `bans.json` bans `203.0.113.0/24` for good:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -523,14 +533,22 @@ request it refuses; one without a cause does not. This `bans.json` bans
|
||||
{
|
||||
"netblock": "203.0.113.0/24",
|
||||
"start": "2026-10-06T12:00:00Z",
|
||||
"expires": null
|
||||
"expires": null,
|
||||
"reason": "probes for logins"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
To lift a ban, delete its entry. `smallwebwaf` then forgets the ban, so it does
|
||||
not make the netblock's next ban longer.
|
||||
To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause`
|
||||
to `admin`: `"cause": "admin"`.
|
||||
|
||||
To lift a ban, add `lifted` to its entry, with the time you lift it, such as
|
||||
`"lifted": "2026-10-06T13:00:00Z"`. From when the edit is taken in, the ban
|
||||
refuses nothing, whatever time `lifted` gives, and does not make the netblock's
|
||||
next ban longer; it is kept in `bans.json` with its notes, as any other ban is.
|
||||
To forget a ban altogether, delete its entry: it then refuses nothing either,
|
||||
and does not make the netblock's next ban longer.
|
||||
|
||||
## Rule files
|
||||
|
||||
@@ -624,8 +642,10 @@ other request. No metric carries a client's address.
|
||||
- `smallwebwaf_rate_limit_hits_total` by `window`,
|
||||
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
|
||||
limit was passed, `smallwebwaf_offences_total` by `kind`, and
|
||||
`smallwebwaf_bans_made_total` by `cause`, `limit` or `attack`;
|
||||
`smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
|
||||
`smallwebwaf_bans_made_total` by `cause`, `limit`, `attack` or `admin`, the
|
||||
last for the bans whose `cause` is `admin` that you add to `bans.json` while
|
||||
`smallwebwaf` runs; `smallwebwaf_active_bans` and
|
||||
`smallwebwaf_permanent_bans`, neither of which counts a lifted ban.
|
||||
- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
|
||||
`rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
|
||||
rules read from the rule files.
|
||||
@@ -954,7 +974,7 @@ addresses are never sent to GeoJS.
|
||||
happened, and served in the Prometheus text format.
|
||||
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how
|
||||
long a new ban lasts, when a ban for a clear sign of attack becomes permanent,
|
||||
and which ban is dropped when `SWWAF_MAX_BANS` are held.
|
||||
and which ban `smallwebwaf` made is dropped when `SWWAF_MAX_BANS` are held.
|
||||
- `internal/rules`: reads the rule files at start and again as they change, and
|
||||
tells which of their rules a request matches.
|
||||
- `internal/lookup`: looks up each client's country through GeoJS, and keeps the
|
||||
@@ -977,8 +997,9 @@ addresses are never sent to GeoJS.
|
||||
checks.
|
||||
|
||||
Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
|
||||
table of clients to 20,000, the GeoJS answers to 100,000 and the banned
|
||||
netblocks to `SWWAF_MAX_BANS`, dropping the least recently seen, and
|
||||
table of clients to 20,000 and the GeoJS answers to 100,000, dropping the least
|
||||
recently seen, and the banned netblocks in the order they were last seen, from
|
||||
which the ledger picks the ban to drop past `SWWAF_MAX_BANS`, and
|
||||
`github.com/prometheus/client_golang` keeps the metrics and serves them, and
|
||||
`github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file or a rule
|
||||
file is saved. The country codes are the list in `internal/config/config.go`.
|
||||
|
||||
Reference in New Issue
Block a user