Rule files, and bans for a clear sign of attack (closes #24)
check / check (push) Successful in 3m13s
check / check (push) Successful in 3m13s
Every *.rules file in SWWAF_RULES_DIR is read at start and on each change, and each request is checked against the rules after the rate limits: log notes a match, block refuses with 403, ban refuses and bans the netblock for SWWAF_ATTACK_BAN_DURATION, made permanent by its next request or clear sign of attack. path, query and uri are matched as the request line sent them. bans.json gains each ban's cause, and ban notes count earlier bans by cause. The image ships 00-default.rules. Judgement call: a header sent twice is matched with its values joined by ", ". Judgement call: SWWAF_MAX_BAN_DURATION does not cap a ban for an attack. Not in this unit: offences for rule matches, with the error burst. Model: opus-5-5
This commit is contained in:
@@ -18,18 +18,21 @@ milestone 3: the static lists, the bans that broken rate limits lead to and the
|
||||
JSON state files with your edits taken in while it runs, which come next in the
|
||||
build order, `observe` mode, which comes a little later, and the metrics
|
||||
endpoint and the header size and the idle time as settings, which come last in
|
||||
it. `smallwebwaf` passes each request to the app and the app's answer back,
|
||||
unchanged, within its timeouts and size limits, works out each client's address,
|
||||
bans a client that sends too many requests, refuses a client that comes from a
|
||||
country you refuse or from a network you refuse, lets the networks you choose
|
||||
through, keeps its bans, each client's counters and history, and GeoJS's answers
|
||||
in JSON files across restarts, takes in your edits of those files while it runs,
|
||||
writes a JSON log line for every request, serves Prometheus metrics to a scraper
|
||||
that holds the metrics token, and in `observe` mode passes on the requests it
|
||||
would refuse, logging what it would have done with them. It comes as the image
|
||||
the app's own image is built on. The rest of the design comes after that, in the
|
||||
order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
|
||||
that led to the design is in [`EVALUATION.md`](EVALUATION.md).
|
||||
it. So are the rule files, the first part of the stage after it, with the bans
|
||||
for a clear sign of attack. `smallwebwaf` passes each request to the app and the
|
||||
app's answer back, unchanged, within its timeouts and size limits, works out
|
||||
each client's address, bans a client that sends too many requests, refuses a
|
||||
client that comes from a country you refuse or from a network you refuse, lets
|
||||
the networks you choose through, checks each request against the rule files and
|
||||
bans a client whose request is a clear sign of attack, keeps its bans, each
|
||||
client's counters and history, and GeoJS's answers in JSON files across
|
||||
restarts, takes in your edits of those files and of the rule files while it
|
||||
runs, writes a JSON log line for every request, serves Prometheus metrics to a
|
||||
scraper that holds the metrics token, and in `observe` mode passes on the
|
||||
requests it would refuse, logging what it would have done with them. It comes as
|
||||
the image the app's own image is built on. The rest of the design comes after
|
||||
that, in the order of the build order in [`SPEC.md`](SPEC.md). The survey of
|
||||
existing tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
|
||||
|
||||
## Getting started
|
||||
|
||||
@@ -52,7 +55,8 @@ works.
|
||||
To work on the code, `make build` builds the binary alone, with Go installed,
|
||||
and `make run` builds and runs it, listening on port 8080 in front of an app at
|
||||
`SWWAF_UPSTREAM_URL`, by default `http://127.0.0.1:8081`, with its state files
|
||||
in `bin/state` unless `SWWAF_STATE_DIR` is set.
|
||||
in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
||||
`share/rules.d` unless `SWWAF_RULES_DIR` is set.
|
||||
|
||||
## What it does so far
|
||||
|
||||
@@ -99,11 +103,26 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set.
|
||||
window and the requests counted in it, the request that broke it, the client's
|
||||
country when it was looked up, the netblock's requests since it was first
|
||||
seen, how many of them the ban has refused, and how many bans the netblock had
|
||||
before. At most `SWWAF_MAX_BANS` bans are kept, past, active and permanent;
|
||||
past that, the earliest ban of the netblock that has gone longest without a
|
||||
request is dropped first. `bans.json` shows the bans and their notes, a
|
||||
restart lifts none, and you add or lift a ban by editing it (see "State files"
|
||||
below).
|
||||
before, for a broken limit, for a clear sign of attack and without a cause. At
|
||||
most `SWWAF_MAX_BANS` bans are kept, past, active and permanent; past that,
|
||||
the earliest ban of the netblock that has gone longest without a request is
|
||||
dropped first. `bans.json` shows the bans and their notes, a restart lifts
|
||||
none, and you add or lift a ban by editing it (see "State files" below).
|
||||
- Checks each request against the rules of the rule files (see "Rule files"
|
||||
below) after the rate limits, and before its body is read. A `log` rule that
|
||||
matches is noted in the log line; a `block` rule refuses the request with
|
||||
`403`, and bans no one; a `ban` rule refuses it with `SWWAF_BAN_RESPONSE` and
|
||||
bans the client's netblock for a clear sign of attack. Matching stops at the
|
||||
first rule that refuses. A client in `SWWAF_ALLOW_NETS` is not checked.
|
||||
- Bans a client for a clear sign of attack, as "Bans" in [`SPEC.md`](SPEC.md)
|
||||
describes: the first such ban lasts `SWWAF_ATTACK_BAN_DURATION`, seven days by
|
||||
default, and any request from the netblock while it lasts makes it permanent.
|
||||
Once it has run out, the netblock is served like any other, but its next clear
|
||||
sign of attack bans it permanently at once. Such a ban covers the same
|
||||
netblock as a ban for a broken rate limit, does not set the client's counters
|
||||
back to zero, and does not make the netblock's next ban for a broken limit
|
||||
longer. Its notes give the id and the target of the rule that matched in place
|
||||
of the limit.
|
||||
- Refuses a request from a country you refuse with `SWWAF_BAN_RESPONSE`, as soon
|
||||
as the client's country is known and before its body is read; such a request
|
||||
is not counted for the rate limits. While one of the country lists below is
|
||||
@@ -114,25 +133,26 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set.
|
||||
in `SWWAF_ALLOW_NETS`, and `SWWAF_DENIED_COUNTRIES` does not refuse it.
|
||||
- Checks the client's own address against the static lists, the three netblock
|
||||
settings below, before anything else, its country included. A client in
|
||||
`SWWAF_ALLOW_NETS` skips bans, the country lists and the rate limits, and is
|
||||
not looked up; the timeouts and size limits still apply. A client in
|
||||
`SWWAF_DENY_NETS` is refused with `SWWAF_BAN_RESPONSE` before its body is
|
||||
read, and the request is not counted for the rate limits; an address in
|
||||
`SWWAF_ALLOW_NETS` too is let through. A client in
|
||||
`SWWAF_ALLOW_NETS` skips bans, the country lists, the rate limits and the rule
|
||||
files, and is not looked up; the timeouts and size limits still apply. A
|
||||
client in `SWWAF_DENY_NETS` is refused with `SWWAF_BAN_RESPONSE` before its
|
||||
body is read, and the request is not counted for the rate limits; an address
|
||||
in `SWWAF_ALLOW_NETS` too is let through. A client in
|
||||
`SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither counted nor refused by the rate
|
||||
limits; the country lists and bans still apply to it.
|
||||
limits; the country lists, the rule files and bans still apply to it.
|
||||
- In `observe` mode, with `SWWAF_MODE=observe`, refuses none of the requests
|
||||
that `SWWAF_DENY_NETS`, a ban, the country lists or a rate limit would refuse:
|
||||
it passes them to the app, and their log lines name what `enforce` mode would
|
||||
have done (see `would_action` in "Request log" below). The checks run, and
|
||||
requests are counted, as in `enforce` mode, but a broken rate limit makes no
|
||||
ban and does not set the client's counters back to zero, so each request over
|
||||
the limit is logged as one that would be refused. The bans in `bans.json` are
|
||||
kept, and refuse requests again when `smallwebwaf` next runs in `enforce`
|
||||
mode, as long as they last. The timeouts and size limits still apply, since
|
||||
they protect `smallwebwaf` and the app themselves, and a request for the
|
||||
metrics without the token is still answered `401`. It is for trying a
|
||||
configuration before enforcing it.
|
||||
that `SWWAF_DENY_NETS`, a ban, the country lists, a rate limit or a rule would
|
||||
refuse: it passes them to the app, and their log lines name what `enforce`
|
||||
mode would have done (see `would_action` in "Request log" below). The checks
|
||||
run, and requests are counted, as in `enforce` mode, with three differences:
|
||||
neither a broken rate limit nor a `ban` rule makes a ban; a broken rate limit
|
||||
does not set the client's counters back to zero, so each request over the
|
||||
limit is logged as one that would be refused; and a request under a ban does
|
||||
not make it permanent. The bans in `bans.json` are kept, and refuse requests
|
||||
again when `smallwebwaf` next runs in `enforce` mode, as long as they last.
|
||||
The timeouts and size limits still apply, since they protect `smallwebwaf` and
|
||||
the app themselves, and a request for the metrics without the token is still
|
||||
answered `401`. It is for trying a configuration before enforcing it.
|
||||
- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
|
||||
check and without asking the app, for the image's health check.
|
||||
- Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for
|
||||
@@ -182,8 +202,8 @@ it, and the effective settings are logged at start.
|
||||
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body.
|
||||
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body.
|
||||
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip bans, the
|
||||
country lists and the rate limits, such as your monitoring or your own
|
||||
networks.
|
||||
country lists, the rate limits and the rule files, such as your monitoring or
|
||||
your own networks.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients the
|
||||
rate limits do not apply to, such as a machine that talks to the app all day.
|
||||
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
|
||||
@@ -199,16 +219,20 @@ it, and the effective settings are logged at start.
|
||||
country cannot be found is refused too, so that new clients are not let in
|
||||
whenever GeoJS stops answering.
|
||||
- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered, one
|
||||
that is banned, breaks a rate limit, is in `SWWAF_DENY_NETS` or comes from a
|
||||
refused country: `403`, `429`, or `close` to close the connection without an
|
||||
answer. Behind traefik, `close` does not leave the client unanswered: traefik
|
||||
answers `502`, as it does whenever its backend drops a connection.
|
||||
that is banned, breaks a rate limit, matches a `ban` rule, is in
|
||||
`SWWAF_DENY_NETS` or comes from a refused country: `403`, `429`, or `close` to
|
||||
close the connection without an answer. Behind traefik, `close` does not leave
|
||||
the client unanswered: traefik answers `502`, as it does whenever its backend
|
||||
drops a connection. A `block` rule always answers `403`.
|
||||
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate
|
||||
limit.
|
||||
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a rate limit broken again
|
||||
within this time after a ban ended bans for three times as long as that ban.
|
||||
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban that would be longer is
|
||||
permanent instead.
|
||||
within this time after a ban ended, other than one for a clear sign of attack,
|
||||
bans for three times as long as that ban.
|
||||
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken rate limit that
|
||||
would be longer is permanent instead.
|
||||
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
||||
attack.
|
||||
- `SWWAF_MAX_BANS` (default `5000`): the most bans kept, past, active and
|
||||
permanent.
|
||||
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
|
||||
@@ -227,6 +251,10 @@ it, and the effective settings are logged at start.
|
||||
`********` in its place.
|
||||
- `SWWAF_METRICS_TOP_N` (default `50`): how many countries get series of their
|
||||
own in the metrics by country; the others are counted as `other`.
|
||||
- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the
|
||||
rule files. A directory that does not exist stops the start.
|
||||
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file, and checks
|
||||
no request against one.
|
||||
|
||||
Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are
|
||||
bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024
|
||||
@@ -265,18 +293,21 @@ refused ones included:
|
||||
- `request_bytes` and `response_bytes` count body bytes.
|
||||
- `action` is `forward` for a request passed to the app, `denied` for one
|
||||
refused because its client is in `SWWAF_DENY_NETS`, `banned` for one refused
|
||||
because a ban covers its client, `country_denied` for one refused for its
|
||||
client's country, `rate_limited` for one that broke a rate limit and banned
|
||||
its client, `too_large` for a request or response over its size limit,
|
||||
`timed_out` for one that ran out of time, `upstream_error` when the app could
|
||||
not be reached or its answer broke off, and `admin` for one `smallwebwaf`
|
||||
answered at its own endpoint.
|
||||
because a ban covers its client or because it matched a `ban` rule, which bans
|
||||
its client, `country_denied` for one refused for its client's country,
|
||||
`rate_limited` for one that broke a rate limit and banned its client,
|
||||
`rule_blocked` for one a `block` rule refused, `too_large` for a request or
|
||||
response over its size limit, `timed_out` for one that ran out of time,
|
||||
`upstream_error` when the app could not be reached or its answer broke off,
|
||||
and `admin` for one `smallwebwaf` answered at its own endpoint.
|
||||
- `would_action` is there in `observe` mode for a request that
|
||||
`SWWAF_DENY_NETS`, a ban, the country lists or a rate limit would have refused
|
||||
in `enforce` mode, and names the action that refusal would have had: `denied`,
|
||||
`banned`, `country_denied` or `rate_limited`. `action` then names what was
|
||||
done: `forward` for a request passed to the app, and another action, such as
|
||||
`too_large`, for one a size or time limit refused.
|
||||
`SWWAF_DENY_NETS`, a ban, the country lists, a rate limit or a rule would have
|
||||
refused in `enforce` mode, and names the action that refusal would have had:
|
||||
`denied`, `banned`, `country_denied`, `rate_limited` or `rule_blocked`.
|
||||
`action` then names what was done: `forward` for a request passed to the app,
|
||||
and another action, such as `too_large`, for one a size or time limit refused.
|
||||
- `rule_ids` is there for a request that matched rules of the rule files, and
|
||||
lists their ids in the order they matched, up to the one that refused it.
|
||||
- `limit_hit` is there for a request that broke a rate limit, and names the
|
||||
window whose limit it went over: `minute`, `hour` or `day`, the shortest if it
|
||||
went over several. `offence` is then `limit`.
|
||||
@@ -305,7 +336,8 @@ all, which it answers itself, mostly with `400`.
|
||||
entries by client address, with times in UTC.
|
||||
|
||||
- `bans.json`: every ban with its notes, indented to be read; a permanent ban's
|
||||
`expires` is `null`.
|
||||
`expires` is `null`, and a ban `smallwebwaf` made has the `cause` `limit` for
|
||||
a broken rate limit or `attack` for a clear sign of attack.
|
||||
- `clients.json`: each client's two buckets in the minute, the hour and the day,
|
||||
and its history: when it was first and last seen, its country as last looked
|
||||
up and when, its requests, how many were forwarded and how many refused (one
|
||||
@@ -316,12 +348,13 @@ entries by client address, with times in UTC.
|
||||
- `lookups.json`: GeoJS's answers, one to a line, with when GeoJS gave each and
|
||||
when it was last used.
|
||||
|
||||
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, with every
|
||||
ban made in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and
|
||||
when `smallwebwaf` stops. Each write goes to a temporary file in the same
|
||||
directory, which then replaces the file, so a crash leaves the old file or the
|
||||
new one, whole. A write that fails is logged, and tried again at the next write.
|
||||
A hard kill loses what changed since the last write.
|
||||
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made or made
|
||||
permanent, with every such change in between, and every file every
|
||||
`SWWAF_STATE_COUNTER_INTERVAL` and when `smallwebwaf` stops. Each write goes to
|
||||
a temporary file in the same directory, which then replaces the file, so a crash
|
||||
leaves the old file or the new one, whole. A write that fails is logged, and
|
||||
tried again at the next write. A hard kill loses what changed since the last
|
||||
write.
|
||||
|
||||
At start the files are read back: each client keeps its counts, so a restart
|
||||
gives it no fresh allowance, and each ban keeps refusing every client in its
|
||||
@@ -336,7 +369,8 @@ without a field it needs, named with the entry's place in the file: a ban's
|
||||
`netblock`, `start` or `expires`, which is `null` for a permanent ban; a
|
||||
client's `client`, or the `start` of a window in which it has requests; an
|
||||
answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or
|
||||
`answered`. The AS number and AS name come with their lookup.
|
||||
`answered`. So does a ban whose `cause` is neither `limit` nor `attack`. The AS
|
||||
number and AS name come with their lookup.
|
||||
|
||||
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
||||
a state file as soon as you save it: what the file then holds replaces what
|
||||
@@ -345,17 +379,19 @@ yours by comparing the file with what it last read or wrote, and before it
|
||||
writes a file it takes in any edit made since, so your edit is not overwritten;
|
||||
a change `smallwebwaf` made after you opened the file, such as a new ban, is
|
||||
lost when you save over it. An edit that would stop the start, because it does
|
||||
not parse, has another `version` or leaves out a field an entry needs, does not
|
||||
stop the running `smallwebwaf`: it keeps what it holds, and at the file's next
|
||||
write renames your file to `<name>.bad`, such as `bans.json.bad`, writes the
|
||||
file again from memory, and logs the file and where the error is. It waits for
|
||||
that write because an editor's file can be read before the editor has finished
|
||||
writing it. Mend the `.bad` file and move it back. A file you remove is written
|
||||
again at its next write.
|
||||
not parse, has another `version`, leaves out a field an entry needs or gives a
|
||||
ban another `cause`, does not stop the running `smallwebwaf`: it keeps what it
|
||||
holds, and at the file's next write renames your file to `<name>.bad`, such as
|
||||
`bans.json.bad`, writes the file again from memory, and logs the file and where
|
||||
the error is. It waits for that write because an editor's file can be read
|
||||
before the editor has finished writing it. Mend the `.bad` file and move it
|
||||
back. A file you remove is written again at its next write.
|
||||
|
||||
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
|
||||
and its `expires`, `null` for a ban that never ends; its `notes` may be left
|
||||
out. This `bans.json` bans `203.0.113.0/24` for good:
|
||||
and its `expires`, `null` for a ban that never ends; its `cause` and its `notes`
|
||||
may be left out. A ban whose `cause` is `attack` becomes permanent at the first
|
||||
request it refuses; one without a cause does not. This `bans.json` bans
|
||||
`203.0.113.0/24` for good:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -373,6 +409,69 @@ out. This `bans.json` bans `203.0.113.0/24` for good:
|
||||
To lift a ban, delete its entry. `smallwebwaf` then forgets the ban, so it does
|
||||
not make the netblock's next ban longer.
|
||||
|
||||
## Rule files
|
||||
|
||||
`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`,
|
||||
`/etc/smallwebwaf/rules.d` by default, in the order of their names, and checks
|
||||
each request against their rules in that order, as "Rule files" in
|
||||
[`SPEC.md`](SPEC.md) describes. A rule is a line of four fields separated by
|
||||
spaces or tabs: an id, a target, an action and a regex, which runs to the end of
|
||||
the line. Blank lines and lines that start with `#` are ignored.
|
||||
|
||||
```
|
||||
# id target action regex
|
||||
env-file path ban (?i)^/\.env(\.[a-z]+)?$
|
||||
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
|
||||
```
|
||||
|
||||
- The id is letters, digits, `-` and `_`, and no two rules share one. The
|
||||
request log, the metrics and a ban's notes name the rule by it.
|
||||
- The target is what the regex is matched against: `path` or `query`, as the
|
||||
client sent it, before any decoding; `uri`, the path and the query together,
|
||||
both as sent and once percent-decoded, so that an encoded probe does not slip
|
||||
past; `method`; `host`; `user_agent`; `referer`; or `header:<Name>`, any one
|
||||
request header. A header sent more than once is matched with its values joined
|
||||
by `, `, and one not sent as empty text. No body is read.
|
||||
- The action is `log`, `block` or `ban` (see "What it does so far" above). Keep
|
||||
`ban` for requests no real visitor sends, and anchor a path at the site root
|
||||
with `^/`: a file of the same name deeper in a site can be ordinary content,
|
||||
such as a file in a repository on a code forge.
|
||||
- The regex is in Go's syntax, RE2, which has no backreferences or lookaround,
|
||||
and takes time linear in the text it reads. It matches anywhere in the target
|
||||
unless anchored with `^` and `$`; `(?i)` at its front makes it ignore case.
|
||||
|
||||
A line that is not a rule, a regex that does not compile or an id used twice
|
||||
stops the start with a message naming the file and the line, and so does a
|
||||
`SWWAF_RULES_DIR` that does not exist. An empty directory is no error, and the
|
||||
log says that it holds no rules. While it runs, `smallwebwaf` watches the
|
||||
directory, and reads the rule files again whenever one is edited, added or
|
||||
removed. If they then hold one of those errors, the rules stay as they were, the
|
||||
earlier version of the edited file included, the log names the file and the
|
||||
line, and the files are read again at the next change.
|
||||
|
||||
The image ships one rule file, `share/rules.d/00-default.rules` here: rules that
|
||||
ban probes no real visitor sends, for secrets, version control directories,
|
||||
backups, logs and web shells at the site root, and the user agents of common
|
||||
scanners; one that blocks `../` twice in a row in the path or the query; and one
|
||||
that only notes a request without a user agent. An app's Dockerfile adds rules
|
||||
of its own in a file beside it, named to sort after it, such as this
|
||||
`50-gitea.rules` for an app that serves no WordPress:
|
||||
|
||||
```
|
||||
wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
|
||||
```
|
||||
|
||||
```dockerfile
|
||||
COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules
|
||||
```
|
||||
|
||||
A directory mounted over `/etc/smallwebwaf/rules.d` replaces the default file,
|
||||
and single files mounted into it add to it. Docker does not show a single
|
||||
mounted file being replaced, which is how many editors save, so rules to be
|
||||
edited while `smallwebwaf` runs belong in a mounted directory, with a copy of
|
||||
`00-default.rules` if its rules are to stay. To run without rules, mount an
|
||||
empty directory or set `SWWAF_RULES_ENABLED=false`.
|
||||
|
||||
## Metrics
|
||||
|
||||
`GET /_smallwebwaf/metrics` answers with the metrics in the Prometheus text
|
||||
@@ -390,8 +489,11 @@ other request. No metric carries a client's address.
|
||||
- `smallwebwaf_rate_limit_hits_total` by `window`,
|
||||
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
|
||||
limit was passed, `smallwebwaf_offences_total` by `kind`, and
|
||||
`smallwebwaf_bans_made_total` by `cause`; `smallwebwaf_active_bans` and
|
||||
`smallwebwaf_permanent_bans`.
|
||||
`smallwebwaf_bans_made_total` by `cause`, `limit` or `attack`;
|
||||
`smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
|
||||
- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
|
||||
`rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
|
||||
rules read from the rule files.
|
||||
- `smallwebwaf_country_requests_total`,
|
||||
`smallwebwaf_country_request_bytes_total`,
|
||||
`smallwebwaf_country_response_bytes_total`, and
|
||||
@@ -418,7 +520,7 @@ other request. No metric carries a client's address.
|
||||
|
||||
The requests Go's HTTP server ends before `smallwebwaf` sees them (see "Request
|
||||
log") are not counted. The metrics of the features still to come, such as the
|
||||
rule files, come with them.
|
||||
Core Rule Set, come with them.
|
||||
|
||||
## Why
|
||||
|
||||
@@ -692,25 +794,29 @@ addresses are never sent to GeoJS.
|
||||
## How the code is laid out
|
||||
|
||||
- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`.
|
||||
- `internal/smallwebwaf`: the process: it reads the settings and the state
|
||||
files, listens, serves requests until `SIGTERM` or `SIGINT`, and stops,
|
||||
writing the state files. Run as `smallwebwaf healthcheck`, it is the image's
|
||||
health check instead.
|
||||
- `internal/smallwebwaf`: the process: it reads the settings, the rule files and
|
||||
the state files, listens, serves requests until `SIGTERM` or `SIGINT`, and
|
||||
stops, writing the state files. Run as `smallwebwaf healthcheck`, it is the
|
||||
image's health check instead.
|
||||
- `internal/config`: reads the settings, the one place they are read.
|
||||
- `internal/proxy`: what happens to each request: it works out the client, runs
|
||||
the checks, passes the request to the app and the answer back with the
|
||||
standard library's `httputil.ReverseProxy` within the timeouts and size
|
||||
limits, and writes the request's log line. Its `check` method is where a
|
||||
request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for
|
||||
a ban, for the country lists, for a rate limit, which bans the client, and for
|
||||
an announced body over the size limit; in `observe` mode, only for the size
|
||||
limit, with what it would have refused for noted in the log line. A request
|
||||
under `/_smallwebwaf/` that `check` lets through is answered by `answerAdmin`
|
||||
a ban, for the country lists, for a rate limit, which bans the client, for a
|
||||
`block` or `ban` rule, the latter banning the client, and for an announced
|
||||
body over the size limit; in `observe` mode, only for the size limit, with
|
||||
what it would have refused for noted in the log line. A request under
|
||||
`/_smallwebwaf/` that `check` lets through is answered by `answerAdmin`
|
||||
instead of reaching the app.
|
||||
- `internal/metrics`: the metrics, counted as the other parts tell it what
|
||||
happened, and served in the Prometheus text format.
|
||||
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how
|
||||
long a new ban lasts, and which ban is dropped when `SWWAF_MAX_BANS` are held.
|
||||
long a new ban lasts, when a ban for a clear sign of attack becomes permanent,
|
||||
and which ban is dropped when `SWWAF_MAX_BANS` are held.
|
||||
- `internal/rules`: reads the rule files at start and again as they change, and
|
||||
tells which of their rules a request matches.
|
||||
- `internal/lookup`: looks up each client's country through GeoJS, and keeps the
|
||||
answers.
|
||||
- `internal/ratelimit`: the table of clients: counts each client's requests,
|
||||
@@ -721,7 +827,8 @@ addresses are never sent to GeoJS.
|
||||
process's own messages.
|
||||
- `Dockerfile`: the lint and test phases, then the image, whose last stage
|
||||
installs Ubuntu's packages, nixpkgs, `runsvinit` and `smallwebwaf`, with
|
||||
`share/smallwebwaf.run` as runit's `run` script for `smallwebwaf`.
|
||||
`share/smallwebwaf.run` as runit's `run` script for `smallwebwaf` and
|
||||
`share/rules.d/00-default.rules` as its default rule file.
|
||||
- `deploy/example-app`: an app built on the image, which `script/example-app`
|
||||
checks.
|
||||
|
||||
@@ -729,8 +836,8 @@ Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
|
||||
table of clients to 20,000, the GeoJS answers to 100,000 and the banned
|
||||
netblocks to `SWWAF_MAX_BANS`, dropping the least recently seen, and
|
||||
`github.com/prometheus/client_golang` keeps the metrics and serves them, and
|
||||
`github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file is saved.
|
||||
The country codes are the list in `internal/config/config.go`.
|
||||
`github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file or a rule
|
||||
file is saved. The country codes are the list in `internal/config/config.go`.
|
||||
|
||||
## Entrypoints
|
||||
|
||||
@@ -760,16 +867,17 @@ so that they run in minimal containers.
|
||||
- `script/build`: builds `bin/smallwebwaf` on the host, with Go installed, for
|
||||
working on the code by hand; `make build` runs it.
|
||||
- `script/run`: builds `bin/smallwebwaf` with `script/build` and runs it, with
|
||||
its state files in `bin/state` unless `SWWAF_STATE_DIR` is set; `make run`
|
||||
runs it.
|
||||
its state files in `bin/state` unless `SWWAF_STATE_DIR` is set, and the rule
|
||||
files of `share/rules.d` unless `SWWAF_RULES_DIR` is set; `make run` runs it.
|
||||
- `script/example-app`: builds the image and, on it, the example app in
|
||||
`deploy/example-app`, runs it with a volume for the state files, and checks
|
||||
that the health check passes, that a request reaches the app through
|
||||
`smallwebwaf`, that a second request in a minute bans the client, that
|
||||
`sv stop` and `docker stop` stop it in order, and that a new container on the
|
||||
same volume still refuses the banned client; then removes the containers, the
|
||||
volume and both images. It needs network access, for nixpkgs' binary cache,
|
||||
and `script/check` does not run it; `make example-app` does.
|
||||
`smallwebwaf`, that a second request in a minute bans the client, that a probe
|
||||
for `/.env` bans another client, whose next request makes the ban permanent,
|
||||
that `sv stop` and `docker stop` stop it in order, and that a new container on
|
||||
the same volume still refuses the banned client; then removes the containers,
|
||||
the volume and both images. It needs network access, for nixpkgs' binary
|
||||
cache, and `script/check` does not run it; `make example-app` does.
|
||||
|
||||
## TODO
|
||||
|
||||
|
||||
Reference in New Issue
Block a user