Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 sneak
|
acceecc7f8 |
@@ -1,35 +0,0 @@
|
||||
# Admin endpoints
|
||||
|
||||
`smallwebwaf` answers these requests itself, on the app's own address and
|
||||
through traefik like any other request. Apart from the health check, each needs
|
||||
a token, sent as `Authorization: Bearer <token>`, and goes through every check
|
||||
any other request goes through. It is answered where another request would be
|
||||
passed to the app: a banned client stays refused, and each request counts toward
|
||||
rate limiting. A missing or wrong token is answered `401`, in `observe` mode
|
||||
too. Such a refusal counts toward the error burst, so a client guessing tokens
|
||||
is soon banned. While a token is unset, its endpoints answer `404`.
|
||||
|
||||
- `GET /_smallwebwaf/healthz`: `200` and `ok`, to anyone, before any check and
|
||||
without asking the app.
|
||||
- `GET /_smallwebwaf/metrics`: the metrics, for `SWWAF_METRICS_TOKEN` (see
|
||||
[`metrics.md`](metrics.md)).
|
||||
- `GET /_smallwebwaf/bans`: every ban held, under `bans`, each as `bans.json`
|
||||
lists it. Needs `SWWAF_ADMIN_TOKEN`, as do the three below.
|
||||
- `POST /_smallwebwaf/bans`: bans a netblock. The body is a JSON object of
|
||||
`netblock`, `duration` and, if you like, `reason`. `netblock` is a netblock
|
||||
such as `203.0.113.0/24`, or a client's address. A client's address bans its
|
||||
IPv4 netblock (`SWWAF_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a
|
||||
duration such as `1h` or `7d`, or `permanent`. The ban starts at once, with
|
||||
the cause `admin`. A bad body is answered `400`, saying what is wrong:
|
||||
- one that is not such an object, or has another field;
|
||||
- one longer than 4 KiB;
|
||||
- one that names an IPv4-mapped netblock, or an address with a zone.
|
||||
- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban covering the
|
||||
address `<client>`. It answers `404` when none is active.
|
||||
- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client.
|
||||
Under `client`, its counters and history as `clients.json` holds them, or
|
||||
`null` when it has never been seen. Under `bans`, every ban covering the
|
||||
address, with its notes.
|
||||
|
||||
An admin whose own address is banned lifts that ban by editing `bans.json` (see
|
||||
[`bans.md`](bans.md)).
|
||||
-149
@@ -1,149 +0,0 @@
|
||||
# Alerts
|
||||
|
||||
Each alert goes to each destination you name. `SWWAF_ALERT_WEBHOOK_URL` gets it
|
||||
as one JSON object. `SWWAF_ALERT_SLACK_WEBHOOK_URL` and `SWWAF_ALERT_NTFY_URL`
|
||||
get a message made from it. The settings are under "Alert destinations" in
|
||||
[`settings.md`](settings.md). An alert is one of these events, and is sent when
|
||||
its event is in `SWWAF_ALERT_EVENTS`:
|
||||
|
||||
- `ban`: a ban `smallwebwaf` made.
|
||||
- `permanent_ban`: a permanent ban it made, or an existing ban made permanent by
|
||||
a request.
|
||||
- `waf_block`: the Core Rule Set scored a request at or over the threshold. In
|
||||
`block` mode the request was refused. In `detect` mode it was not, and the
|
||||
alert's `detail` gives `mode`.
|
||||
- `anomaly`: a count over an anomaly threshold.
|
||||
- `reputation_hit`: a request from a client on a blocklist, the CrowdSec
|
||||
decision list or a DNS blocklist, or with an AbuseIPDB score that is a hit.
|
||||
One alert per source.
|
||||
- `source_failure`: one of:
|
||||
- GeoJS failing;
|
||||
- a list that cannot be fetched;
|
||||
- a DNS blocklist or AbuseIPDB failing or refusing a query;
|
||||
- the day's AbuseIPDB checks used up.
|
||||
- `file_error`: one of:
|
||||
- a rule file or state file edit with an error;
|
||||
- a state file that cannot be written;
|
||||
- a replacement lookup database that cannot be read.
|
||||
- `summary`: the hourly summary described under "Repeats and the hourly cap"
|
||||
below.
|
||||
|
||||
The bans you make raise no alert. In `observe` mode, a request that would have
|
||||
made a ban still raises the ban's alert, with `mode` set to `observe` in its
|
||||
`detail`.
|
||||
|
||||
## The JSON object
|
||||
|
||||
This is the alert for a ban for a broken rate limit, shown indented; it is sent
|
||||
on one line:
|
||||
|
||||
```json
|
||||
{
|
||||
"instance": "fsn1app1/gitea",
|
||||
"time": "2026-10-06T12:00:00.123461Z",
|
||||
"event": "ban",
|
||||
"client": "203.0.113.9",
|
||||
"netblock": "203.0.113.9/32",
|
||||
"asn": "",
|
||||
"as_name": "",
|
||||
"country": "",
|
||||
"reason": "requests per minute over the limit of 1000",
|
||||
"detail": {
|
||||
"ban_expires": "2026-10-06T13:00:00.123Z",
|
||||
"cause": "limit",
|
||||
"notes": {
|
||||
"asn": "",
|
||||
"as_name": "",
|
||||
"country": "",
|
||||
"kind": "requests",
|
||||
"limit": 1000,
|
||||
"window": "minute",
|
||||
"count": 1001,
|
||||
"request": {
|
||||
"time": "2026-10-06T12:00:00.123456789Z",
|
||||
"method": "GET",
|
||||
"host": "app.example",
|
||||
"path": "/owner/repo/commits/branch/main?page=812",
|
||||
"status": 403,
|
||||
"user_agent": "scraper/1.0"
|
||||
},
|
||||
"requests": 5210,
|
||||
"refused": 0,
|
||||
"earlier_bans": {
|
||||
"limit": 0,
|
||||
"attack": 0,
|
||||
"admin": 0,
|
||||
"crowdsec": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
"suppressed_repeats": 0
|
||||
}
|
||||
```
|
||||
|
||||
- `instance` is `SWWAF_INSTANCE_NAME`. `time` is when the alert was raised.
|
||||
`event` is one of the events above.
|
||||
- `client` is the address whose request raised the alert. `netblock` is the
|
||||
ban's netblock, the netblock an anomaly counted, or the client's group. Both
|
||||
are empty for `source_failure` and `file_error`. `asn`, `as_name` and
|
||||
`country` are the client's as known when the alert was raised. They are empty
|
||||
when GeoJS had not answered yet.
|
||||
- `reason` is a short sentence, such as
|
||||
`requests per minute over the limit of 1000` or `listed by a blocklist`.
|
||||
- `detail` is what is particular to the event:
|
||||
- for a ban, its `cause`, `ban_expires` and `notes` (see
|
||||
[`bans.md`](bans.md) for the notes);
|
||||
- for an anomaly, the `scope`, `window`, `kind`, `count` and `threshold`;
|
||||
- for a reputation hit, the `source` and, for AbuseIPDB, the `score`;
|
||||
- for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`;
|
||||
- for a source failure, the `source` and `error`;
|
||||
- for a file error, the `file` and `error`.
|
||||
- `suppressed_repeats` is how many repeats the cooldown held back before this
|
||||
alert.
|
||||
|
||||
## Slack and ntfy
|
||||
|
||||
Slack and ntfy get a message with a title and a text. The title is the instance
|
||||
and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then one
|
||||
line for each of these that the alert has:
|
||||
|
||||
- the client, the netblock and the country;
|
||||
- the file, the source, the error and the mode from `detail`;
|
||||
- the suppressed repeats, if any.
|
||||
|
||||
Slack gets the title in bold and the text below it, with `&`, `<` and `>`
|
||||
escaped. ntfy gets the text as the message, with these headers:
|
||||
|
||||
- `Title`: the title;
|
||||
- `Authorization`: the token, as `Bearer <token>`, while
|
||||
`SWWAF_ALERT_NTFY_TOKEN` is set;
|
||||
- `Priority` and `Tags`: by event, as listed below.
|
||||
|
||||
The priority and tag by event:
|
||||
|
||||
- `ban`: priority `default`, tag `no_entry`.
|
||||
- `permanent_ban`: priority `high`, tag `no_entry`.
|
||||
- `waf_block`: priority `default`, tag `shield`.
|
||||
- `anomaly`: priority `high`, tag `chart_with_upwards_trend`.
|
||||
- `reputation_hit`: priority `low`, tag `label`.
|
||||
- `source_failure` and `file_error`: priority `high`, tag `warning`.
|
||||
- `summary`: priority `default`, tag `bar_chart`.
|
||||
|
||||
## Repeats and the hourly cap
|
||||
|
||||
A repeat is an alert for the same event on the same netblock, source, file or
|
||||
thing counted, less than `SWWAF_ALERT_COOLDOWN` after the last. It is held back
|
||||
and counted, and the next alert sent gives the count. Past
|
||||
`SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and counted
|
||||
by event. Once the hour ends, one alert with the event `summary` gives those
|
||||
counts. It also counts the repeats that a cooldown held back and that no later
|
||||
alert reported.
|
||||
|
||||
## Sending
|
||||
|
||||
Each destination has a queue of at most 1000 alerts, sent one at a time, so a
|
||||
slow destination holds up nothing. A `2xx` answer takes an alert. A `4xx` other
|
||||
than `408` and `429` refuses it, and it is dropped. Anything else is a failure:
|
||||
the alert is sent again a second later, and twice as long after each further
|
||||
failure, up to a minute. `alerts.json` keeps the queues and cooldowns across a
|
||||
restart (see [`state-files.md`](state-files.md)).
|
||||
@@ -1,84 +0,0 @@
|
||||
# Bans
|
||||
|
||||
How bans are made, what they cover and how long they last is under "Bans" in
|
||||
[`SPEC.md`](../SPEC.md); the settings are under "How long bans last and what
|
||||
they cover" in [`settings.md`](settings.md). This file says what a ban's notes
|
||||
hold, and how to make, keep or lift a ban.
|
||||
|
||||
## The notes
|
||||
|
||||
Each ban carries notes for deciding whether to lift it, drawn from what
|
||||
`smallwebwaf` already knows. Nothing is looked up to fill them:
|
||||
|
||||
- the AS number, AS name and country, added when the lookup answers (empty when
|
||||
lookups are off);
|
||||
- what was broken: the rule id and target that matched, or the trap path, or the
|
||||
limit, its window and the count reached;
|
||||
- the client's percentage of the limit, with the setting that gave it;
|
||||
- the reputation sources the client is on;
|
||||
- the request that caused the ban: time, method, host, path with its query
|
||||
string, status and user agent, each text cut to 256 bytes;
|
||||
- the netblock's requests since it was first seen, and how many this ban has
|
||||
refused so far, kept up to date while the ban lasts;
|
||||
- how many earlier bans the netblock had, by cause.
|
||||
|
||||
## Managing bans by editing `bans.json`
|
||||
|
||||
`bans.json` in the state directory holds every ban, indented to be read. A
|
||||
permanent ban's `expires` is `null`. `smallwebwaf` watches the file and takes in
|
||||
an edit as soon as it is saved (see [`state-files.md`](state-files.md)).
|
||||
|
||||
To ban a netblock, add an entry with its `netblock`, `start` and `expires`.
|
||||
`expires` is `null` for a ban that never ends. `reason`, `notes` and `cause` may
|
||||
be left out; the cause is then `admin`:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"bans": [
|
||||
{
|
||||
"netblock": "203.0.113.0/24",
|
||||
"start": "2026-10-06T12:00:00Z",
|
||||
"expires": null,
|
||||
"reason": "probes for logins"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
To keep a ban `smallwebwaf` made, so that it is never dropped, set its `cause`
|
||||
to `admin`. To lift a ban, add `"lifted"` with the time you lift it, such as
|
||||
`"lifted": "2026-10-06T13:00:00Z"`. The ban then refuses nothing and does not
|
||||
make the netblock's next ban longer. It is kept with its notes. To forget a ban,
|
||||
delete its entry. A ban whose cause is `attack` becomes permanent at the first
|
||||
request it refuses. One whose cause is `admin` or `crowdsec` does not.
|
||||
|
||||
An edit that cannot be read does not stop the running `smallwebwaf`. It keeps
|
||||
what it holds. At the file's next write it renames your file to `bans.json.bad`,
|
||||
writes the file again from memory, and logs and alerts the error. Mend the
|
||||
`.bad` file and move it back.
|
||||
|
||||
## Managing bans through the admin endpoints
|
||||
|
||||
With `SWWAF_ADMIN_TOKEN` set, and the token in `$TOKEN`:
|
||||
|
||||
```sh
|
||||
# Every ban.
|
||||
curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans
|
||||
|
||||
# Ban 203.0.113.0/24 for seven days.
|
||||
curl -H "Authorization: Bearer $TOKEN" \
|
||||
--json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \
|
||||
https://app.example/_smallwebwaf/bans
|
||||
|
||||
# Lift the bans on 203.0.113.9.
|
||||
curl -H "Authorization: Bearer $TOKEN" -X DELETE \
|
||||
https://app.example/_smallwebwaf/bans/203.0.113.9
|
||||
|
||||
# What smallwebwaf knows of 203.0.113.9.
|
||||
curl -H "Authorization: Bearer $TOKEN" \
|
||||
https://app.example/_smallwebwaf/clients/203.0.113.9
|
||||
```
|
||||
|
||||
See [`admin-endpoints.md`](admin-endpoints.md) for what each answers. An admin
|
||||
whose own address is banned lifts that ban by editing `bans.json`.
|
||||
@@ -1,31 +0,0 @@
|
||||
# The Core Rule Set
|
||||
|
||||
The OWASP Core Rule Set 4.25.0 runs on each request after the rule files, unless
|
||||
`SWWAF_WAF_MODE` is `off`. It reads the method, the URL with its query, and the
|
||||
headers. Once `SWWAF_WAF_BODY_LIMIT` is set, it also reads a body that is form
|
||||
data, multipart, JSON or XML. No answer is inspected. The settings are under
|
||||
"Attack detection" in [`settings.md`](settings.md). The same section of
|
||||
[`SPEC.md`](../SPEC.md) says why bodies are not read by default, and gives the
|
||||
six changes `smallwebwaf` makes to the Core Rule Set, with the reason for each.
|
||||
|
||||
Each rule that matches adds to the request's score. A score at or over
|
||||
`SWWAF_WAF_ANOMALY_THRESHOLD` is a match. A match is refused with `403` while
|
||||
`SWWAF_WAF_MODE` is `block`, and let through while it is `detect`. Either way
|
||||
the log line gives the rule ids and the score, and a `waf_block` alert is sent.
|
||||
A request with more than 1000 query parameters or body fields adds 5 to its
|
||||
score, since Coraza reads only the first 1000. The Core Rule Set inspects only a
|
||||
request that every earlier check let through. It skips a path under
|
||||
`SWWAF_WAF_EXEMPT_PATHS` and a client in `SWWAF_ALLOW_NETS`. An engine error on
|
||||
a request forwards the request, and is logged and counted.
|
||||
|
||||
Body inspection suits apps whose forms carry no code. In front of a code forge
|
||||
it refuses issue and comment text, wiki pages and files saved in the web editor
|
||||
that hold shell commands or code. The log line names the rule ids, which
|
||||
`SWWAF_WAF_DISABLED_RULES` can switch off. A body the Core Rule Set cannot parse
|
||||
adds 5 to the score. So does a multipart body cut by the limit in the middle of
|
||||
a part's header. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to
|
||||
send the part that is read. Any other body, such as a git push, reaches the app
|
||||
uninspected.
|
||||
|
||||
[`gitea.md`](gitea.md) records what the Core Rule Set still refuses in front of
|
||||
gitea, with and without body inspection.
|
||||
-210
@@ -1,210 +0,0 @@
|
||||
# smallwebwaf in front of gitea
|
||||
|
||||
These notes record what `smallwebwaf` does in front of gitea at the default
|
||||
settings. They are observations, not requirements; the requirements are in
|
||||
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind the image and
|
||||
sends through it a sample of gitea's ordinary use and the requests named below.
|
||||
It reports each request the Core Rule Set refuses, so that these notes can be
|
||||
checked again when gitea or the Core Rule Set changes version.
|
||||
|
||||
## What the defaults cover
|
||||
|
||||
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
|
||||
by it.
|
||||
- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all
|
||||
but the largest repositories and slowest links.
|
||||
- A push is a request, and so is every other upload: a git push, an LFS object,
|
||||
a container image layer, a package file or a release attachment. At the
|
||||
defaults, an upload larger than 100 MiB or taking more than 60 seconds is cut
|
||||
off. The client is answered `413` for a body that is too large, before
|
||||
anything reaches gitea when the request announces its size. It is answered
|
||||
`408` for a body it sends too slowly, or `504` if gitea is too slow to take
|
||||
it. The upload fails, and no one is banned for it. A gitea that takes large
|
||||
uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and
|
||||
`SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does not
|
||||
read an upload's body, which streams through without being held in memory.
|
||||
- Archive download and blame or history pages are what scrapers hammer. The
|
||||
request limits do most of the work there.
|
||||
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
|
||||
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
|
||||
43,200 requests a day, and reports a running job's log and state every second.
|
||||
So it passes the day limit of 50,000 after about an hour and a quarter of
|
||||
jobs. It is then banned, the job it is running fails, and each repeat within a
|
||||
day makes the ban longer. Newer runners ask less often, but a busy one, or
|
||||
several on one address, can still pass it. Put the runners' addresses in
|
||||
`SWWAF_RATE_LIMIT_EXEMPT_NETS`. That takes them out of the request and byte
|
||||
limits, while the error burst, attack detection and bans still apply.
|
||||
|
||||
## What the Core Rule Set lets through
|
||||
|
||||
At the defaults, the Core Rule Set lets gitea's ordinary use through, apart from
|
||||
the refusals in the next section:
|
||||
|
||||
- browsing and views of files in a repository, with their history, blame and the
|
||||
file tree;
|
||||
- diffs, including their hidden lines and large files, and pull request review;
|
||||
- git's clone, fetch and push over HTTP;
|
||||
- signing in, including the return to the page a visitor came from, and sign-in
|
||||
with Git Credential Manager, git-credential-oauth or tea;
|
||||
- the API's calls for a file and its commits;
|
||||
- pushing and pulling container images and packages;
|
||||
- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
|
||||
- posting issues, pull requests, comments, wiki pages and files saved in the web
|
||||
editor, code included, since no body is read;
|
||||
- the page gitea shows after a change, with its message naming what was done.
|
||||
|
||||
## What the Core Rule Set still refuses
|
||||
|
||||
Each refusal answers only that request, with `403`, and bans no one by itself.
|
||||
It counts toward the error burst, which a person does not reach this way. The
|
||||
request log names the rule in `waf_rule_ids`, which `SWWAF_WAF_DISABLED_RULES`
|
||||
can switch off.
|
||||
|
||||
### A query string that reads as an attack
|
||||
|
||||
Most often this is a search. Every query parameter is checked against the Core
|
||||
Rule Set's lists of system files and commands, except the ten named under
|
||||
"Attack detection" in `SPEC.md`. A release attachment uploaded through the API
|
||||
as `docker-compose.yml` is refused the same way. Examples:
|
||||
|
||||
- a name on those lists, such as `package.json`, `.gitignore` or
|
||||
`docker-compose.yml`;
|
||||
- text that starts with a command name, or whose first word has one right after
|
||||
a `/`, such as `python3`, `ssh key` or `feat/docker-support`;
|
||||
- a shell command with its options or a system path (`ls -la`, `sed -i`,
|
||||
`/bin/sh`), or a command in backticks;
|
||||
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
|
||||
statement (`SELECT * FROM users WHERE`);
|
||||
- a URL naming an IP address or `localhost`.
|
||||
|
||||
### A branch, tag or file name that starts with a command name
|
||||
|
||||
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
|
||||
`python3`, `ansible`, `base64` and `whoami`. A branch, tag or file name that
|
||||
starts with one is refused. So is one that has such a name right after a `/`,
|
||||
such as `feat/docker-support`, `fix/docker-build` or
|
||||
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
|
||||
query parameter that 930120, 932160 and 932260 still check:
|
||||
|
||||
- `refSubUrl`, the branch or tag of a directory listing. It is sent when the
|
||||
listing asks for its entries' last commits in a second request, which it does
|
||||
whenever gitea takes more than a second to work them out;
|
||||
- `name`, when a branch is deleted or restored on the branches page;
|
||||
- `tag`, when a release is started from a tag;
|
||||
- `template`, the file of an issue template, when an issue is opened from it;
|
||||
- `rule_name`, when a branch protection rule is opened for editing in the
|
||||
repository's settings or has just been saved.
|
||||
|
||||
What a visitor sees, for a branch named `docker-build`:
|
||||
|
||||
- A directory listing that makes that second request leaves those last commits
|
||||
out and shows an error.
|
||||
- The branches page shows an error instead of deleting or restoring the branch.
|
||||
- A release started from such a tag gets the `403` answer of `smallwebwaf` in
|
||||
place of its form. So does an issue opened from such a template, and such a
|
||||
rule opened for editing.
|
||||
- Saving such a rule stores it. gitea then returns to the branch settings page,
|
||||
which gets the `403` answer, so the save looks as if it failed.
|
||||
|
||||
### A file name in a query parameter that trips the code rules
|
||||
|
||||
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
|
||||
look for in some file names. This holds in the parameters `smallwebwaf` leaves
|
||||
out of 930120, 932160 and 932260 too (see the changes to the Core Rule Set under
|
||||
"Attack detection" in `SPEC.md`). Examples:
|
||||
|
||||
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
|
||||
(944110);
|
||||
- a Java class, as in `java.lang.String.md` (944130);
|
||||
- `document.write.js` (941180);
|
||||
- `process.env.d.ts` (934100);
|
||||
- `__proto__` (934130);
|
||||
- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
|
||||
- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
|
||||
|
||||
For such a file, these requests are refused:
|
||||
|
||||
- the API's list of its commits (`path`);
|
||||
- the API's call for the contents of files that names it in its query (`body`);
|
||||
- the commit graph of that file alone (`file`);
|
||||
- the hidden lines of its diff (`path`), which stay hidden;
|
||||
- the sign-in page gitea sends a visitor to on the way to a page of the file
|
||||
that needs signing in, such as its edit page (`redirect_to`). The visitor can
|
||||
still sign in by opening the sign-in page itself.
|
||||
|
||||
### A path
|
||||
|
||||
- A file name ending in `~`, or an `.xhtml` file whose path holds a space.
|
||||
- A path that holds what the Core Rule Set takes for a Java class (944130), as
|
||||
`java.lang.String.md` and
|
||||
`META-INF/services/org.apache.commons.logging.LogFactory` do.
|
||||
- A path that holds what it takes for Node.js code (934100), as
|
||||
`process.env.d.ts` does.
|
||||
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
|
||||
|
||||
For such a file, the page of the file, its raw file, its history, its blame and
|
||||
its edit page are refused. The directory that holds it lists it, and a clone
|
||||
carries it.
|
||||
|
||||
### A container image push that reuses a layer
|
||||
|
||||
A push can reuse a layer of an image docker has pushed to that gitea before.
|
||||
docker then asks gitea to take the layer from that image, naming it in the query
|
||||
parameter `from`, which 932260 checks. The request is refused when the image's
|
||||
owner, or its name after the owner, starts with an entry on the list of
|
||||
commands. Examples are `owner/python3-app`, `owner/docker-base`, and any image
|
||||
of an owner named `ansible` or `docker-team`.
|
||||
|
||||
What happens next depends on docker's image store. A docker that keeps its
|
||||
images in containerd's image store fails the push, and fails each new try the
|
||||
same way. A new install of Docker Engine 29 does this. A docker that keeps them
|
||||
in its own older image store uploads the layer instead. The push then succeeds
|
||||
after one refusal for each layer it reuses, and each of those refusals counts
|
||||
toward the error burst.
|
||||
|
||||
### Artifact uploads from `actions/upload-artifact@v3`
|
||||
|
||||
This action sends the header `Content-Range`, which the Core Rule Set refuses
|
||||
(920450). The action sends two files at a time, does not retry a `403`, and
|
||||
sends no further file once one is refused. So an upload is refused once or
|
||||
twice, however many files it holds, and the step fails. One upload does not ban
|
||||
the runner. More than 30 such refusals from its address within a minute, as when
|
||||
many jobs upload at once, ban it through the error burst.
|
||||
`actions/upload-artifact@v4` does not send the header.
|
||||
|
||||
## What body inspection refuses
|
||||
|
||||
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
|
||||
front of gitea it then refuses these, until their rule ids, named in the request
|
||||
log, are added to `SWWAF_WAF_DISABLED_RULES`:
|
||||
|
||||
- issue and comment text, wiki pages and files saved in the web editor that hold
|
||||
shell commands or code (932125, 932235, 932250 and others);
|
||||
- package descriptions that show code;
|
||||
- PyPI uploads (922130);
|
||||
- attachments named like `debug.log` or `config.yml` (932180).
|
||||
|
||||
## What `script/gitea-requests` sends
|
||||
|
||||
Of the requests above, the script sends:
|
||||
|
||||
- for each file named under "A file name in a query parameter that trips the
|
||||
code rules", the API's list of its commits;
|
||||
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
|
||||
its diff, and the API's call for the contents of files that names it;
|
||||
- the sign-in page gitea sends a visitor to on the way to the edit page of
|
||||
`document.write.js`;
|
||||
- the pages of `java.lang.String.md`,
|
||||
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
|
||||
and `.../notes.txt`;
|
||||
- one container image push that reuses a layer, which it does not try again.
|
||||
|
||||
It does not send the raw file, history, blame or edit page of those four files,
|
||||
or the pages of the directories `META-INF/services` and `...`. It also sends a
|
||||
short sample of gitea's ordinary use:
|
||||
|
||||
- signing in;
|
||||
- pages of files and commits, and a diff;
|
||||
- the API;
|
||||
- a push, and clone and fetch over HTTP;
|
||||
- two container image pushes.
|
||||
@@ -1,56 +0,0 @@
|
||||
# What the image holds
|
||||
|
||||
The `smallwebwaf` image is the base of each app's own image. The requirements
|
||||
are under "Deployment" in [`SPEC.md`](../SPEC.md); this file lists what the
|
||||
image holds, how each part is pinned, and why.
|
||||
|
||||
## Contents
|
||||
|
||||
- Ubuntu 26.04 LTS, the newest long-term support release of Ubuntu, pinned by
|
||||
digest. The image moves to the next LTS release when that ships.
|
||||
- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, installed from a dated
|
||||
snapshot of its archive and checked by hash. The Ubuntu image has no CA
|
||||
certificates; without them Nix cannot download packages and `smallwebwaf`
|
||||
cannot reach GeoJS.
|
||||
- Nix and nixpkgs, fixed at one release. Root uses Nix directly; no Nix daemon
|
||||
runs. `nix-bin` creates no build users, so `build-users-group =` in
|
||||
`/etc/nix/nix.conf` lets root build without them.
|
||||
- runit, and `runsvinit` as the entrypoint, as the owner's code style guide asks
|
||||
for service containers. No package carries `runsvinit`, so the image builds it
|
||||
from its archived source (`github.com/peterbourgon/runsvinit`, unchanged
|
||||
since 2015) at a fixed commit hash, with a `go.mod` the build writes.
|
||||
`runsvinit` starts runit's `runsvdir`, which runs the `run` script of each
|
||||
directory under `/etc/service` and runs it again whenever it exits. Ubuntu's
|
||||
`sv` looks for services there too.
|
||||
- The `smallwebwaf` binary, and a user of its own, `smallwebwaf` (uid and gid
|
||||
65532).
|
||||
- The service directory `/etc/service/smallwebwaf`. Its `run` script waits one
|
||||
second (`sleep 1`), makes `SWWAF_STATE_DIR` and every file in it belong to the
|
||||
`smallwebwaf` user, and starts `smallwebwaf` as that user with runit's
|
||||
`chpst`.
|
||||
- The directory `/var/lib/smallwebwaf` for the state files, and
|
||||
`/etc/smallwebwaf/rules.d` with the default rule file.
|
||||
- Port 8080 declared (`EXPOSE 8080`), and the health check (`HEALTHCHECK`) that
|
||||
"The health check" in `SPEC.md` describes.
|
||||
|
||||
## Pinning
|
||||
|
||||
- Ubuntu's packages come from Ubuntu's snapshot service, the archive as it was
|
||||
at the moment the Dockerfile names (`--snapshot 20261001T000000Z`), never
|
||||
earlier than the pinned Ubuntu image's date. The Dockerfile names the SHA-256
|
||||
hash of each of the snapshot's `InRelease` files, which cover every package
|
||||
list and so every package, and the build checks them before installing. The
|
||||
hashes are the amd64 archive's, so the image is built for amd64 only. The
|
||||
snapshot is fetched over HTTPS with the CA certificates of the Go image that
|
||||
builds `smallwebwaf`, itself pinned by digest, since the Ubuntu image has
|
||||
none.
|
||||
- nixpkgs comes from a release file on `releases.nixos.org` (`nixexprs.tar.xz`
|
||||
of `nixos-26.05.11045.774debe7a0d1`), named with its SHA-256 hash, not from a
|
||||
GitHub archive, whose bytes can change. It is set up for root as `nixpkgs`.
|
||||
Root's Nix profile, `/nix/var/nix/profiles/default/bin`, is last on the
|
||||
`PATH`, so that no package hides the image's own commands; busybox's `sv`, for
|
||||
one, looks for services elsewhere. Unpacked, nixpkgs takes about 500 MiB.
|
||||
- An app built on the image gets the same packages each time it is built.
|
||||
Security fixes from Ubuntu and nixpkgs come with a newer `smallwebwaf` image,
|
||||
which an app takes by changing the digest in its `FROM` line. The image moves
|
||||
to each new nixpkgs release, every six months.
|
||||
@@ -1,79 +0,0 @@
|
||||
# GeoJS and the lookup database
|
||||
|
||||
`smallwebwaf` looks up each client's AS number and country, through GeoJS by
|
||||
default or in a database file on your own host. The settings are under "Looking
|
||||
up the AS number and country" in [`settings.md`](settings.md); the reasons for
|
||||
the defaults are under "Lookup of AS number and country" in
|
||||
[`SPEC.md`](../SPEC.md).
|
||||
|
||||
## GeoJS
|
||||
|
||||
By default `smallwebwaf` asks GeoJS, a free web service that needs no account,
|
||||
about every new visitor's address. It asks whether or not a setting uses the
|
||||
answer, since the log line, the history, the ban notes and the metrics carry the
|
||||
AS number and country. These addresses are never sent:
|
||||
|
||||
- one in `SWWAF_ALLOW_NETS` or `SWWAF_DENY_NETS`;
|
||||
- one under a ban;
|
||||
- a private, loopback or link-local address, which no source can place.
|
||||
|
||||
For an IPv6 visitor, GeoJS is asked about the first address of its group.
|
||||
Addresses are asked about together, up to 200 in one request
|
||||
(`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a time. Each
|
||||
answer's `country_code`, `asn` and `organization_name` are read. An answer
|
||||
without a `country_code`, or with the `asn` `64512`, counts as unknown. Each
|
||||
answer is kept for seven days, in memory and in `lookups.json`, separately from
|
||||
the table of clients. Up to 100,000 answers are kept; past that, the answer used
|
||||
longest ago goes first.
|
||||
|
||||
A request waits for its client's first answer only when a setting needs the
|
||||
answer before the request goes on. It waits up to `SWWAF_LOOKUP_TIMEOUT`. The
|
||||
settings that make it wait:
|
||||
|
||||
- a country list;
|
||||
- `SWWAF_ADD_LOOKUP_HEADERS`;
|
||||
- any setting under "Lower limits by AS number and country" in
|
||||
[`settings.md`](settings.md).
|
||||
|
||||
Otherwise the request goes on at once, logged without the answer. The answer
|
||||
reaches the client's history and ban notes when it comes. A client whose answer
|
||||
has not come in time counts as from an unknown country until it comes, and its
|
||||
later requests do not wait. While GeoJS fails, new visitors count as from an
|
||||
unknown country, which `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses. GeoJS is
|
||||
then left alone for a second, and twice as long after each further failure, up
|
||||
to five minutes, and one `source_failure` alert is sent per cooldown.
|
||||
|
||||
## The database file
|
||||
|
||||
To keep your visitors' addresses on your own host, set
|
||||
`SWWAF_LOOKUP_SOURCE=off`, or set it to `file`. `file` looks every client up in
|
||||
the free IPinfo Lite database (`ipinfo_lite.mmdb`) at `SWWAF_LOOKUP_DB_PATH`,
|
||||
reading its `asn`, `as_name` and `country_code` fields. You download the
|
||||
database with an IPinfo account and mount it into the container; `smallwebwaf`
|
||||
never fetches it and holds no account or token for it. It answers at once, with
|
||||
nothing sent off the host. A client whose address it does not hold counts as
|
||||
from an unknown country.
|
||||
|
||||
`smallwebwaf` reads the whole file at start. It reads the file again once it has
|
||||
gone 2 seconds without a change after you replace it. A replacement it cannot
|
||||
read is logged and alerted, and the file read before stays in use. Mount the
|
||||
directory that holds the file, not the file itself. Many editors and download
|
||||
tools save a file by replacing it, and a container keeps seeing the old file
|
||||
when only the file is mounted.
|
||||
|
||||
IPinfo updates the database daily, and releases it under the Creative Commons
|
||||
Attribution-ShareAlike 4.0 licence. It asks for a link to IPinfo on the site
|
||||
that uses it, such as "IP address data is powered by IPinfo".
|
||||
|
||||
## A client without a country
|
||||
|
||||
A client the lookup cannot place has an unknown country and AS number. That is
|
||||
one of:
|
||||
|
||||
- a client on a private address;
|
||||
- one GeoJS or the database does not know;
|
||||
- one whose GeoJS answer has not come in time.
|
||||
|
||||
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses such a client, and the lower
|
||||
limits give it `SWWAF_UNKNOWN_LIMIT_PERCENT`. Nothing else treats it
|
||||
differently.
|
||||
@@ -1,83 +0,0 @@
|
||||
# The metrics
|
||||
|
||||
`GET /_smallwebwaf/metrics` answers in the Prometheus text format. The
|
||||
requirements are under "Metrics endpoint" in [`SPEC.md`](../SPEC.md); this file
|
||||
lists the metrics. Every metric carries the label `instance`, set to
|
||||
`SWWAF_INSTANCE_NAME`; Prometheus keeps it as `exported_instance` unless the
|
||||
scrape sets `honor_labels: true`. No metric carries a client's address.
|
||||
|
||||
## Traffic
|
||||
|
||||
- `smallwebwaf_requests_total`, `smallwebwaf_request_bytes_total` and
|
||||
`smallwebwaf_response_bytes_total`, by `status_class` such as `2xx` and by
|
||||
`action`;
|
||||
- `smallwebwaf_request_duration_seconds` and
|
||||
`smallwebwaf_upstream_duration_seconds`, as histograms;
|
||||
- `smallwebwaf_requests_in_flight`.
|
||||
|
||||
## Limits and bans
|
||||
|
||||
- `smallwebwaf_rate_limit_hits_total`, by `window` and by `kind` (`requests`,
|
||||
`bytes` or `refusals`);
|
||||
- `smallwebwaf_size_and_time_limit_hits_total`, by `limit`, the setting passed;
|
||||
- `smallwebwaf_offences_total`, by `kind`;
|
||||
- `smallwebwaf_bans_made_total`, by `cause`;
|
||||
- `smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
|
||||
|
||||
## Attack detection
|
||||
|
||||
- `smallwebwaf_rule_matches_total`, by `rule_id` and `action`, and
|
||||
`smallwebwaf_rules_loaded`;
|
||||
- `smallwebwaf_waf_matches_total`, by `mode` and `rule_id`, for the rules that
|
||||
fired.
|
||||
|
||||
## Countries and AS numbers
|
||||
|
||||
For the `SWWAF_METRICS_TOP_N` busiest of each, the rest as `other`:
|
||||
|
||||
- `smallwebwaf_country_requests_total`,
|
||||
`smallwebwaf_country_request_bytes_total`,
|
||||
`smallwebwaf_country_response_bytes_total` and
|
||||
`smallwebwaf_country_list_refusals_total`, by `country`;
|
||||
- `smallwebwaf_asn_requests_total`, `smallwebwaf_asn_request_bytes_total` and
|
||||
`smallwebwaf_asn_response_bytes_total`, by `asn`.
|
||||
|
||||
## Lookups
|
||||
|
||||
- `smallwebwaf_geojs_requests_total` and `smallwebwaf_geojs_failures_total`;
|
||||
- `smallwebwaf_geojs_unanswered_total`: the requests that needed an answer and
|
||||
went on without it;
|
||||
- with the file source,
|
||||
`smallwebwaf_lookup_database_last_read_timestamp_seconds` and
|
||||
`smallwebwaf_lookup_database_read_failures_total`.
|
||||
|
||||
## Reputation sources
|
||||
|
||||
By `source`: each list's URL, each zone, and `abuseipdb`:
|
||||
|
||||
- `smallwebwaf_reputation_hits_total`, `smallwebwaf_reputation_queries_total`
|
||||
and `smallwebwaf_reputation_failures_total`;
|
||||
- `smallwebwaf_reputation_last_fetch_timestamp_seconds`, for a list;
|
||||
- `smallwebwaf_reputation_daily_budget_remaining`, for AbuseIPDB.
|
||||
|
||||
## State files
|
||||
|
||||
By `file`:
|
||||
|
||||
- `smallwebwaf_state_file_writes_total`,
|
||||
`smallwebwaf_state_file_write_failures_total`,
|
||||
`smallwebwaf_state_file_last_write_timestamp_seconds` and
|
||||
`smallwebwaf_state_file_size_bytes`;
|
||||
- `smallwebwaf_state_file_edits_taken_in_total` and
|
||||
`smallwebwaf_state_file_edits_set_aside_total`;
|
||||
- `smallwebwaf_tracked_clients`: the clients in the table.
|
||||
|
||||
## Remote log, alerts and the process
|
||||
|
||||
- while a syslog server is set, `smallwebwaf_remote_log_lines_sent_total`,
|
||||
`smallwebwaf_remote_log_lines_dropped_total` and
|
||||
`smallwebwaf_remote_log_buffer_depth`;
|
||||
- by `destination`, `smallwebwaf_alerts_sent_total`,
|
||||
`smallwebwaf_alerts_failed_total`, `smallwebwaf_alerts_suppressed_total` and
|
||||
`smallwebwaf_alerts_dropped_total`;
|
||||
- Go's own `go_` metrics and the process's `process_` metrics.
|
||||
@@ -1,113 +0,0 @@
|
||||
# Reputation sources
|
||||
|
||||
A reputation source judges a client by what others saw it do. None is on by
|
||||
default. The settings are under "Reputation sources" in
|
||||
[`settings.md`](settings.md); the reasons for the defaults are under
|
||||
"Reputation" in [`SPEC.md`](../SPEC.md). The sources are checked in this order,
|
||||
and a source the client is on refuses it, lowers its limits, or only notes it,
|
||||
as `SWWAF_BLOCKLIST_ACTION` and `SWWAF_REPUTATION_ACTION` say.
|
||||
|
||||
## Blocklists
|
||||
|
||||
`SWWAF_BLOCKLIST_URLS` lists text files of addresses and netblocks, one to a
|
||||
line, as the Spamhaus DROP list is written. Anything after a `;` or `#` is left
|
||||
out. An IPv4-mapped address such as `::ffff:192.0.2.0/120` is read as the IPv4
|
||||
netblock it stands for.
|
||||
|
||||
Each list is fetched again `SWWAF_BLOCKLIST_REFRESH` after its last try, one
|
||||
list at a time. The file of `SWWAF_ASN_LIMIT_PERCENT_URL` is too.
|
||||
`reputation.json` keeps when each was last tried, so a restart does not fetch a
|
||||
list again before its time. A fetch fails when:
|
||||
|
||||
- the server answers other than `200`;
|
||||
- the fetch does not finish within a minute;
|
||||
- the list is longer than 16 MiB;
|
||||
- a line is not an address or netblock, or, for the AS number file, not
|
||||
`AS<number>:<percent>`.
|
||||
|
||||
The copy fetched before stays in use, comment lines included. The failure is
|
||||
logged and raised as a `source_failure` alert. The log, the alerts, the ban
|
||||
notes and the metrics name each list by its URL, so keep a secret out of it.
|
||||
|
||||
The Spamhaus DROP list is The Spamhaus Project's. Its terms, and what they mean
|
||||
for `SWWAF_BLOCKLIST_REFRESH` and for several instances on one host, are under
|
||||
"Reputation" in [`SPEC.md`](../SPEC.md). The copy in `reputation.json` keeps the
|
||||
list's date and copyright lines.
|
||||
|
||||
## CrowdSec
|
||||
|
||||
While `SWWAF_CROWDSEC_LAPI_URL` is set to the local API of a CrowdSec engine you
|
||||
run, `smallwebwaf` fetches its decision list every minute. It sends
|
||||
`SWWAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key` header. The list is kept as a
|
||||
blocklist is kept. The decisions of the type `ban` on an address or a netblock
|
||||
are used. Any other, such as a captcha or a country, is left out. A decision
|
||||
lasts for the `duration` it had left when the engine answered.
|
||||
|
||||
A request is refused when a decision still in force names its client. The
|
||||
client's netblock is banned until the decision ends, with the cause `crowdsec`.
|
||||
Only a client that sends a request is banned, so a long list does not fill
|
||||
`bans.json`. Such a ban is never made permanent, and does not lengthen the
|
||||
netblock's next ban for a broken limit. If you lift a ban while the decision is
|
||||
in force, the client's next request makes another. Delete the decision in
|
||||
CrowdSec first, such as with `cscli decisions delete --ip 203.0.113.9`, and lift
|
||||
the ban once the list has been fetched again. A fetch fails when:
|
||||
|
||||
- the engine answers other than `200`, a redirect included;
|
||||
- the fetch does not finish within a minute;
|
||||
- the answer is longer than 16 MiB;
|
||||
- the answer is not a list of decisions.
|
||||
|
||||
## DNS blocklists
|
||||
|
||||
Each zone in `SWWAF_DNSBL_ZONES` is asked about a client's own address in the
|
||||
background, the first time the client is seen. The request goes on at once, and
|
||||
so does every request from the client until the zone has answered. The name
|
||||
asked about is the one RFC 5782 gives: the numbers of an IPv4 address reversed,
|
||||
or the 32 hex digits of an IPv6 address reversed. `192.0.2.99` is asked about as
|
||||
`99.2.0.192.dnsbl.dronebl.org`. An answer in `127.0.0.0/8` lists the client, but
|
||||
one in `127.255.255.0/24` is Spamhaus refusing the query. A name that does not
|
||||
exist does not list the client. These are failures:
|
||||
|
||||
- an answer in `127.255.255.0/24`;
|
||||
- an answer outside `127.0.0.0/8`;
|
||||
- a resolver error;
|
||||
- no answer within `SWWAF_REPUTATION_TIMEOUT`.
|
||||
|
||||
A failure is logged and alerted, and the zone is not asked again for a minute.
|
||||
Each verdict is kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and in
|
||||
`reputation.json`. Up to 100,000 verdicts are kept; past that, the one fetched
|
||||
longest ago goes first. At most 1,000 queries are under way at once.
|
||||
|
||||
The key of a Spamhaus zone under `dq.spamhaus.net` is its first label.
|
||||
`smallwebwaf` shows `********` in its place wherever it names the zone. Only
|
||||
`reputation.json` keeps the zone with its key.
|
||||
|
||||
## AbuseIPDB
|
||||
|
||||
While `SWWAF_ABUSEIPDB_KEY` is set, `smallwebwaf` asks AbuseIPDB's check
|
||||
endpoint for the abuse confidence score of a client, from 0 to 100. Only a
|
||||
client that has committed an offence is checked. An offence is one of:
|
||||
|
||||
- a broken limit;
|
||||
- a `ban` rule match or a trap path;
|
||||
- a refusal by a `block` rule, the Core Rule Set or a token check.
|
||||
|
||||
The check runs in the background, at the client's first request after the
|
||||
offence that no ban refuses, so no request waits. The score applies to the
|
||||
client's later requests. For an IPv6 client, AbuseIPDB is asked about the
|
||||
address of the request that set off the check, and the score is used for the
|
||||
whole group. A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE` is a hit, and
|
||||
`SWWAF_REPUTATION_ACTION` says what is done. Each score is kept for
|
||||
`SWWAF_REPUTATION_CACHE_TTL`, up to 100,000 scores.
|
||||
|
||||
At most `SWWAF_ABUSEIPDB_DAILY_BUDGET` checks are made in a day, counted in UTC
|
||||
and kept in `reputation.json`. A restart does not make the budget whole again.
|
||||
The check that spends the last of the budget is logged and alerted. No client is
|
||||
checked until the day ends. A check fails when:
|
||||
|
||||
- AbuseIPDB answers other than `200`;
|
||||
- the answer gives no score;
|
||||
- no answer comes within `SWWAF_REPUTATION_TIMEOUT`.
|
||||
|
||||
A failure is logged and alerted, and no client is checked for a minute. The key
|
||||
is sent in the `Key` header and nowhere else, and is never logged.
|
||||
@@ -1,128 +0,0 @@
|
||||
# The request log's fields
|
||||
|
||||
`smallwebwaf` writes one JSON object per line on stdout for every request. The
|
||||
requirements are under "Request log" in [`SPEC.md`](../SPEC.md); this file lists
|
||||
the fields. A field that does not apply to a request is left out.
|
||||
|
||||
## Example
|
||||
|
||||
```
|
||||
{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
|
||||
```
|
||||
|
||||
## Standard web log fields
|
||||
|
||||
- `time`: when the request arrived, RFC 3339 with milliseconds, in UTC;
|
||||
- `instance`: `SWWAF_INSTANCE_NAME`;
|
||||
- `client_ip`, `method`, `scheme`, `host`, `path`, `query`, `protocol`;
|
||||
- `status`: what the client was sent, `0` if nothing;
|
||||
- `request_bytes` and `response_bytes`: body bytes each way;
|
||||
- `referer` and `user_agent`.
|
||||
|
||||
## Request detail
|
||||
|
||||
- `request_id`: the `X-Request-ID` a trusted proxy sent, or a new random one of
|
||||
26 letters and digits. It goes to the app in `X-Request-ID`.
|
||||
- `peer_ip`: the TCP peer, normally traefik.
|
||||
- `forwarded_for`: the `X-Forwarded-For` header as received.
|
||||
- `client_group`: the client as rate limiting counts it, an IPv4 address as a
|
||||
/32 or an IPv6 group.
|
||||
- `asn`, `as_name` and `country`: the client's, as looked up. Each is empty
|
||||
when:
|
||||
- lookups are off;
|
||||
- the client is not looked up;
|
||||
- the source cannot place the client;
|
||||
- GeoJS had not answered by the time the request went on.
|
||||
- `content_type` and `content_length`: the request's own.
|
||||
- `request_headers`: the headers listed in `SWWAF_LOG_REQUEST_HEADERS`, by name
|
||||
in lower case.
|
||||
- `has_authorization` and `has_cookie`: true when the request had those headers.
|
||||
Their values are never logged.
|
||||
- `websocket`: true when the app switched the connection to another protocol.
|
||||
|
||||
## Response detail
|
||||
|
||||
- `upstream_status`: what the app answered. It differs from `status` when
|
||||
`smallwebwaf` answered itself.
|
||||
- `response_content_type`, `cache_control` and `location`: the answer's headers.
|
||||
- `aborted`: true when the client went away early.
|
||||
|
||||
## Decision
|
||||
|
||||
- `action`: what was done:
|
||||
- `forward`: passed to the app;
|
||||
- `denied`: refused for `SWWAF_DENY_NETS`, or for a reputation source with
|
||||
the action `deny`;
|
||||
- `banned`: refused under a ban, or for a `ban` rule, a trap path or a
|
||||
CrowdSec decision, each of which bans the client;
|
||||
- `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`:
|
||||
refused by that check;
|
||||
- `too_large` and `timed_out`: a size or time limit was passed;
|
||||
- `upstream_error`: the app could not be reached, or its answer broke off;
|
||||
- `admin`: answered at one of `smallwebwaf`'s own endpoints.
|
||||
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
||||
request it would have refused. `action` then names what was done, normally
|
||||
`forward`.
|
||||
- `limit_percent` and `limit_percent_setting`: the client's percentage of the
|
||||
rate limits and the setting that lowered it, when one did. `bytes_percent` and
|
||||
`bytes_percent_setting` do the same for the byte limits.
|
||||
- `counts`: the client's requests in the minute, hour and day as rate limiting
|
||||
counts them, this request included, as `minute`, `hour` and `day`; its bytes
|
||||
as `minute_bytes`, `hour_bytes` and `day_bytes`. A count can have a fraction,
|
||||
since each window is counted in two buckets with the earlier one weighted.
|
||||
`counts` is left out when rate limiting did not count the request.
|
||||
- `limit_hit`: the limit broken: `minute`, `hour` or `day` for a rate limit;
|
||||
`minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or
|
||||
`error_burst`. A request whose bytes broke a limit was passed on whole, so its
|
||||
`action` stays `forward`.
|
||||
- `rule_ids`: the rule file rules that matched, in order, up to the one that
|
||||
refused.
|
||||
- `waf_score` and `waf_rule_ids`: the Core Rule Set's score and the rules that
|
||||
matched, for a request it inspected.
|
||||
- `reputation`: the sources the client is on, in this order:
|
||||
- blocklist URLs;
|
||||
- the CrowdSec decision list's URL;
|
||||
- DNS blocklist zones;
|
||||
- `abuseipdb`.
|
||||
- `offence`: the kind of offence recorded, when one was.
|
||||
- `ban_expires`: when the client's ban ends, for a request that made one or was
|
||||
refused under one: a time, or `permanent`.
|
||||
|
||||
## Timings
|
||||
|
||||
All in milliseconds:
|
||||
|
||||
- `duration_total`: from the request's headers being read to its log line;
|
||||
- `duration_checks`: to the end of the checks, before forwarding;
|
||||
- `duration_waf`: the part of that the Core Rule Set took;
|
||||
- `duration_upstream_connect`: from the hand-over to the app until there was a
|
||||
connection;
|
||||
- `duration_upstream_first_byte`: until the first byte of the answer;
|
||||
- `duration_upstream_total`: until the end of the answer.
|
||||
|
||||
## Every line
|
||||
|
||||
Every line has `type`, the standard web log fields, `request_id`, `peer_ip`,
|
||||
`client_group`, `asn`, `as_name`, `country`, `action` and `duration_total`. The
|
||||
process's own messages share the stream with `"type":"process"`; request lines
|
||||
carry `"type":"request"`.
|
||||
|
||||
## Sending the log to a syslog server
|
||||
|
||||
While `SWWAF_LOG_REMOTE_URL` is set, every line on stdout is also sent to that
|
||||
server, as the message of an RFC 5424 syslog record. Over UDP each record is one
|
||||
datagram. Over TCP and TLS each record follows its length in bytes and a space.
|
||||
stdout is unchanged. The record carries:
|
||||
|
||||
- the facility from `SWWAF_LOG_REMOTE_FACILITY`;
|
||||
- the severity informational;
|
||||
- the time;
|
||||
- the host's name;
|
||||
- the app name from `SWWAF_LOG_REMOTE_APP_NAME`.
|
||||
|
||||
Lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines, so a server that is
|
||||
slow or down never holds up a request. When the buffer is full, the oldest line
|
||||
is dropped. A failed connection is tried again a second later, and twice as long
|
||||
after each further failure, up to a minute. A line too long for one UDP datagram
|
||||
is dropped alone. As `smallwebwaf` stops, it spends at most two seconds sending
|
||||
the lines still waiting, and drops the rest. stdout has carried them.
|
||||
-114
@@ -1,114 +0,0 @@
|
||||
# Rule files
|
||||
|
||||
`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`, in the order of
|
||||
their names, and checks each request against their rules in that order, unless
|
||||
the client is in `SWWAF_ALLOW_NETS`. The requirements the files meet are under
|
||||
"Rule files" in [`SPEC.md`](../SPEC.md). A file whose name starts with `.`, such
|
||||
as an editor's lock file, is not a rule file.
|
||||
|
||||
## The format
|
||||
|
||||
A rule is a line of four fields separated by spaces or tabs: an id, a target, an
|
||||
action and a regex. The regex runs to the end of the line. Blank lines and lines
|
||||
starting with `#` are ignored.
|
||||
|
||||
```
|
||||
# id target action regex
|
||||
env-file path ban (?i)^/\.env(\.[a-z]+)?$
|
||||
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
|
||||
```
|
||||
|
||||
- The id is letters, digits, `-` and `_`. No two rules share one. The log line,
|
||||
the metrics, the alerts and a ban's notes name the rule by it.
|
||||
- The target is what the regex is matched against:
|
||||
- `path`: the path without the query;
|
||||
- `uri`: the path and the query together;
|
||||
- `query`: the query as sent;
|
||||
- `method`, `host`, `user_agent` and `referer`;
|
||||
- `header:<Name>`: any one request header but `Host` and
|
||||
`Transfer-Encoding`, which Go's HTTP server takes out of every request.
|
||||
|
||||
`path` and `uri` are matched both as sent and once percent-decoded, so that
|
||||
an encoded probe does not slip past. A header sent more than once is matched
|
||||
with its values joined by `, `. A header not sent is empty text. No body is
|
||||
read; reading bodies is the Core Rule Set's job, once `SWWAF_WAF_BODY_LIMIT`
|
||||
switches it on.
|
||||
|
||||
- The action is `log`, `block` or `ban`. `log` only notes the match in the log
|
||||
line. `block` refuses the request with `403` and bans no one; the refusal
|
||||
counts toward the error burst. `ban` refuses it and bans the client, as a
|
||||
clear sign of attack. Matching stops at the first rule that refuses. Keep
|
||||
`ban` for requests no real visitor sends. Anchor a path at the site root with
|
||||
`^/`: a file of the same name deeper in a site can be ordinary content.
|
||||
- The regex is in Go's syntax, RE2. It has no backreferences or lookaround, and
|
||||
takes time linear in the text it reads, so no rule can be made to stall the
|
||||
proxy. It matches anywhere in the target unless anchored with `^` and `$`.
|
||||
`(?i)` at its front makes it ignore case.
|
||||
|
||||
## Errors
|
||||
|
||||
These stop the start, with a message naming the file and line:
|
||||
|
||||
- a line that is not a rule;
|
||||
- a bad `header:` name, or a rule for `Host` or `Transfer-Encoding`;
|
||||
- a regex that does not compile;
|
||||
- an id used twice.
|
||||
|
||||
So does a `SWWAF_RULES_DIR` that does not exist. An empty directory is not an
|
||||
error: the log says that no rules were loaded.
|
||||
|
||||
While it runs, `smallwebwaf` watches the directory. It reads the files again
|
||||
once the directory has had no change for 2 seconds, so that a file still being
|
||||
written is read whole. If the files then hold such an error, the rules stay as
|
||||
they were. The log and a `file_error` alert name the file and line. Once the
|
||||
file is fixed, it is read again.
|
||||
|
||||
## The default file
|
||||
|
||||
The image ships one file, `share/rules.d/00-default.rules`, installed as
|
||||
`/etc/smallwebwaf/rules.d/00-default.rules`:
|
||||
|
||||
```
|
||||
# 00-default.rules: probes no real visitor sends, anchored at the site root
|
||||
|
||||
# id target action regex
|
||||
env-file path ban (?i)^/\.env(\.[a-z]+)?$
|
||||
vcs-dir path ban (?i)^/\.(git|svn|hg|bzr)(/|$)
|
||||
secrets-dir path ban (?i)^/\.(aws|ssh|docker|kube)/
|
||||
secret-file path ban (?i)^/\.(htpasswd|htaccess|npmrc|netrc|pgpass|git-credentials|bash_history|DS_Store)$
|
||||
editor-dir path ban (?i)^/\.(vscode|idea)/
|
||||
backup-file path ban (?i)^/[^/]+\.(php(\.[a-z0-9]+|~)|sql(\.[a-z0-9]+)?)$
|
||||
log-file path ban (?i)^/(debug|error|access)\.log$
|
||||
compose-file path ban (?i)^/(docker-)?compose\.ya?ml$
|
||||
php-shell path ban (?i)^/(shell|c99|r57|wso|alfa)\.php$
|
||||
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|masscan|zgrab|wpscan)\b
|
||||
path-traversal uri block (\.\./){2,}
|
||||
empty-agent user_agent log ^$
|
||||
```
|
||||
|
||||
It is kept short and limited to patterns that are wrong for every app, with its
|
||||
path rules anchored at the site root. In front of gitea, `/.env` is a probe,
|
||||
while `/<owner>/<repo>/src/branch/main/.env.example` is a file in a repository
|
||||
that any visitor may open. Anything app-specific belongs in a file the app's
|
||||
`Dockerfile` adds or the deployer mounts. A request for `/wp-login.php`, for
|
||||
example, is a clear sign of attack in front of gitea and an ordinary login in
|
||||
front of WordPress. Such a rule goes in a file named to sort after the default:
|
||||
|
||||
```
|
||||
# 50-gitea.rules: WordPress probes, which gitea never serves
|
||||
wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
|
||||
```
|
||||
|
||||
```dockerfile
|
||||
COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules
|
||||
```
|
||||
|
||||
## Mounting rule files
|
||||
|
||||
A directory mounted over `/etc/smallwebwaf/rules.d` replaces the default file;
|
||||
single files mounted into it add to it. Many editors save a file by replacing
|
||||
it, and a container keeps seeing the old one when the file alone is mounted. So
|
||||
rules to be edited while `smallwebwaf` runs belong in a mounted directory, with
|
||||
a copy of the default file if the defaults are to stay. The lookup database is
|
||||
mounted the same way, for the same reason. To run without rules, mount an empty
|
||||
directory or set `SWWAF_RULES_ENABLED=false`.
|
||||
@@ -1,483 +0,0 @@
|
||||
# Settings
|
||||
|
||||
Every setting is an environment variable whose name starts with `SWWAF_`. Every
|
||||
setting has a default, so none has to be set. Settings are read once, at start.
|
||||
To change one, restart the container.
|
||||
|
||||
A setting that is set to a value it cannot take stops the start. The process
|
||||
writes one line, `"type":"process"` with `"msg":"invalid setting"`, naming the
|
||||
variable and what is wrong with the value. It exits with status 1, and runit
|
||||
starts it again every second until the value is fixed. The entries below say
|
||||
which values each setting takes. Any other value stops the start. At start, the
|
||||
settings in effect are logged on one line. Tokens, keys and the paths of
|
||||
webhooks are shown as `********`.
|
||||
|
||||
The forms of values:
|
||||
|
||||
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. Zero
|
||||
and negative durations are refused.
|
||||
- A size is a number of bytes with an optional `K`, `M` or `G`, which are powers
|
||||
of 1024: `1K` is 1024 bytes, `512K`, `100M`, `5G`.
|
||||
- A count is a whole number of requests, such as `1000`.
|
||||
- `off` switches a setting off, where its entry below says it takes `off`.
|
||||
- A list is comma separated, with spaces around items ignored. An empty item is
|
||||
refused. A list set but empty is an empty list, and replaces the default.
|
||||
- A netblock is an address and a prefix length, such as `10.0.0.0/8`. A bare
|
||||
address stands for itself alone.
|
||||
- A country is a two-letter code of ISO 3166-1, in either case: `de` and `DE`
|
||||
are the same. `xk` stands for Kosovo. A code that is not a country is refused,
|
||||
such as `nk` (North Korea is `kp`) or the withdrawn `su`.
|
||||
- An AS number is `AS` and the number, in either case, such as `AS64496`.
|
||||
- A percentage is a whole number from 0 to 100.
|
||||
- A path prefix starts with `/`, such as `/assets/`.
|
||||
|
||||
## Where it listens and what it protects
|
||||
|
||||
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens, an
|
||||
optional host and a port. Keep the host empty: `:9000`, never
|
||||
`127.0.0.1:9000`. The app must not listen on that port. The health check
|
||||
follows the setting, and traefik's label must name the port.
|
||||
- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or
|
||||
`https` URL with a host and an optional port, and nothing more: no path, user
|
||||
or query.
|
||||
- `SWWAF_INSTANCE_NAME` (default: the container's host name, the first 12
|
||||
characters of the container's id): the `instance` name in every log line,
|
||||
alert and metric. Set it to a name that survives deploys and tells instances
|
||||
apart, such as `fsn1app1/gitea`. It must be valid UTF-8. While
|
||||
`SWWAF_ALERT_NTFY_URL` is set, it may hold no control character. While
|
||||
`SWWAF_LOG_REMOTE_URL` is set and `SWWAF_LOG_REMOTE_APP_NAME` is not, it must
|
||||
be 1 to 48 printable ASCII characters without a space.
|
||||
- `SWWAF_MODE` (default `enforce`): `enforce`, or `observe` to ban no one and
|
||||
log what `enforce` would have done. Observe mode still refuses a request over
|
||||
a size or time limit, and one without the right token.
|
||||
- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`,
|
||||
the private address ranges): the netblocks whose `X-Forwarded-For` is
|
||||
believed, normally where traefik comes from. Set but empty, it trusts nothing:
|
||||
every TCP peer is then the client.
|
||||
- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the length of the netblock that is
|
||||
one IPv6 client, from 32 to 128. After a change, each IPv6 client starts
|
||||
afresh. Existing bans keep refusing their netblocks until they end.
|
||||
- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept in memory
|
||||
and in `clients.json`, a whole number above zero. Past it, the least recently
|
||||
seen client is dropped first.
|
||||
|
||||
## Time and size limits
|
||||
|
||||
Each request has limits on both sides: between the client and `smallwebwaf`, and
|
||||
between `smallwebwaf` and the app. Bodies stream through in both directions,
|
||||
except the part the Core Rule Set reads while `SWWAF_WAF_BODY_LIMIT` is set.
|
||||
|
||||
- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to
|
||||
send its request line and headers, and then, from the end of the headers, its
|
||||
body. A duration or `off`.
|
||||
- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request
|
||||
line and headers. A size over `4K`. It cannot be `off`: Go's HTTP server
|
||||
always has such a limit.
|
||||
- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection
|
||||
may wait for its next request before it is closed. A duration or `off`. The
|
||||
default is longer than the 90 seconds after which traefik closes an unused
|
||||
connection, so traefik closes first.
|
||||
- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the answer may take
|
||||
to reach the client, from the end of the request to the last byte. A duration
|
||||
or `off`.
|
||||
- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the
|
||||
app and sending it the whole request may take. A duration or `off`.
|
||||
- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take
|
||||
to send its whole answer. A duration or `off`.
|
||||
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size
|
||||
or `off`. A request that announces a larger body is refused before anything
|
||||
reaches the app.
|
||||
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size
|
||||
or `off`.
|
||||
|
||||
In front of a code forge, a clone is a response and a push is a request. At the
|
||||
defaults, an upload over 100 MiB or slower than 60 seconds is cut off. A forge
|
||||
that takes large uploads needs `SWWAF_REQUEST_MAX_BYTES`,
|
||||
`SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to
|
||||
fit.
|
||||
|
||||
What the client gets when a limit is passed is under "Size and time limits" in
|
||||
[`SPEC.md`](../SPEC.md).
|
||||
|
||||
## Netblock lists
|
||||
|
||||
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip every check,
|
||||
such as your monitoring or your own network. The size and time limits still
|
||||
apply to them. They are not looked up. They still need a token for the admin
|
||||
endpoints.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
||||
exempt from rate limiting and the byte limits, such as a machine that talks to
|
||||
the app all day. Everything else still applies to them.
|
||||
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
|
||||
A client in both lists is let through.
|
||||
|
||||
## Rate limiting and byte limits
|
||||
|
||||
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
|
||||
(default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
|
||||
requests a client may make in each window. A count or `off`. The defaults are
|
||||
several times what one busy person produces. A browser loading a heavy page
|
||||
makes a few hundred requests, and several people often share one address.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
|
||||
rate limiting, such as `/assets/`. Their bytes are not counted either. The
|
||||
path is matched exactly as sent, without the query, and is not
|
||||
percent-decoded. `/assets/` matches `/assets/app.js`. It does not match
|
||||
`/Assets/app.js`, `/%61ssets/app.js` or `/static/assets/app.js`. There are no
|
||||
wildcards. A path holding `..`, a backslash or an encoded slash (`%2F`) is
|
||||
never exempt, since the app may read it as a path outside the prefix.
|
||||
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
|
||||
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most
|
||||
bytes a client may have counted in each window. A size or `off`. Bytes are
|
||||
counted once the answer has ended, so a byte limit never cuts an answer short.
|
||||
The client's next request is refused instead. Each default is above the
|
||||
largest request and answer together, so at the defaults no single download
|
||||
breaks a limit.
|
||||
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
|
||||
`response`, `request` or `both`. For a WebSocket, what it carried each way
|
||||
counts once it closes.
|
||||
|
||||
## Looking up the AS number and country
|
||||
|
||||
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `geojs`
|
||||
asks the GeoJS web service, which is then told the address of every new
|
||||
visitor. `file` looks clients up in the IPinfo Lite database file at
|
||||
`SWWAF_LOOKUP_DB_PATH`, and sends nothing off the host. `off` looks up no
|
||||
client. With `off`, a setting that needs lookups stops the start:
|
||||
- a country list;
|
||||
- `SWWAF_ADD_LOOKUP_HEADERS`;
|
||||
- an entry in `SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`,
|
||||
`SWWAF_ASN_BYTES_PERCENT` or `SWWAF_COUNTRY_BYTES_PERCENT`;
|
||||
- `SWWAF_UNKNOWN_LIMIT_PERCENT` below 100;
|
||||
- `SWWAF_ASN_LIMIT_PERCENT_URL`;
|
||||
- an anomaly threshold per AS number.
|
||||
- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
|
||||
`.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`. `file` without it stops the
|
||||
start, and so does setting it with any other source. A file that is missing or
|
||||
cannot be read stops the start too.
|
||||
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
|
||||
client's first answer from GeoJS when a setting needs it, and how long a
|
||||
request to GeoJS may take. A duration; it cannot be `off`.
|
||||
- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the
|
||||
client's AS number in `X-Client-ASN` and its country in `X-Client-Country`. A
|
||||
header whose value is unknown is left out. Whatever this setting says, any
|
||||
such header the client sent is removed.
|
||||
|
||||
See [`lookup.md`](lookup.md) for what each source sends where, and how long
|
||||
answers are kept.
|
||||
|
||||
## Country lists
|
||||
|
||||
- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused,
|
||||
such as `cn,ru,kp`.
|
||||
- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
|
||||
countries whose clients get through, such as `us,de`. A client whose country
|
||||
cannot be found is refused too. Otherwise every new client would be let in
|
||||
whenever GeoJS stops answering. A client on a private address has no country,
|
||||
so it is refused unless it is in `SWWAF_ALLOW_NETS`.
|
||||
|
||||
A country on both lists stops the start. What a refused request gets, and why
|
||||
the lists are empty by default, is under "Country lists" in
|
||||
[`SPEC.md`](../SPEC.md).
|
||||
|
||||
## Lower limits by AS number and country
|
||||
|
||||
Clients of the AS numbers and countries listed here get a percentage of every
|
||||
rate limit and byte limit. A lowered limit is rounded down. "Biased thresholds"
|
||||
in [`SPEC.md`](../SPEC.md) says why the lists are empty by default and why no
|
||||
budget is shared by a whole AS number or country.
|
||||
|
||||
- `SWWAF_ASN_LIMIT_PERCENT` (default empty): a list of AS numbers, each with `:`
|
||||
and a percentage, such as `AS14061:50,AS16276:50,AS45102:25`. An AS number
|
||||
listed twice is refused.
|
||||
- `SWWAF_COUNTRY_LIMIT_PERCENT` (default empty): the same by country, such as
|
||||
`cn:25,ru:50`.
|
||||
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty):
|
||||
the same for the byte limits alone. For the AS numbers and countries they
|
||||
list, they take the place of the two settings above.
|
||||
- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage of every limit
|
||||
for a client without a country:
|
||||
- one the lookup cannot place;
|
||||
- one on a private address;
|
||||
- one whose answer from GeoJS has not come in time.
|
||||
- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
|
||||
text file of such entries, one to a line, such as `AS14061:50`. One list can
|
||||
then serve several instances. It is fetched and kept as a blocklist is. Where
|
||||
it and `SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies. A
|
||||
URL that is also in `SWWAF_BLOCKLIST_URLS` stops the start.
|
||||
|
||||
A client to which several percentages apply gets the lowest. The log line of
|
||||
each counted request, and the notes of each ban, say which setting lowered the
|
||||
limit.
|
||||
|
||||
## Reputation sources
|
||||
|
||||
None is on by default. A reputation source judges a client by what others saw it
|
||||
do. The defaults judge it by what it does to your service.
|
||||
|
||||
- `SWWAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as
|
||||
`https://www.spamhaus.org/drop/drop.txt`. Each is an `http` or `https` URL
|
||||
without a user or a fragment. A URL listed twice is refused.
|
||||
- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its
|
||||
last fetch, or failed try, before it is fetched again. A duration of `1h` or
|
||||
more; it cannot be `off`. The Spamhaus lists may be fetched no more than once
|
||||
an hour.
|
||||
- `SWWAF_BLOCKLIST_ACTION` (default `deny`): what is done with a client on a
|
||||
blocklist. `deny` refuses its requests. `limit:<percent>`, such as `limit:25`,
|
||||
gives it that percentage of every limit. `log` only notes the list in the log
|
||||
line and raises an alert.
|
||||
- `SWWAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask about each
|
||||
client, such as `dnsbl.dronebl.org`. A DNS blocklist answers a DNS query for a
|
||||
name made from an address, and so says whether the address is on it. A zone is
|
||||
a DNS name of at most 189 characters. A Spamhaus zone carries its key, as in
|
||||
`<key>.xbl.dq.spamhaus.net`, and is shown with the key masked. A zone listed
|
||||
twice is refused, whatever its case or key. Do not name a zone meant for mail:
|
||||
such a zone holds the residential addresses ordinary visitors come from.
|
||||
- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through,
|
||||
such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional
|
||||
port, 53 when none is given. Unset, the host's resolver is used. Several zones
|
||||
refuse queries that come through a public resolver.
|
||||
- `SWWAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account.
|
||||
While it is set, a client that has committed an offence is checked. A control
|
||||
character in it is refused.
|
||||
- `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score,
|
||||
from 0 to 100, that is a hit.
|
||||
- `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day,
|
||||
in UTC, a whole number above zero. AbuseIPDB's free accounts may make 1,000.
|
||||
- `SWWAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine
|
||||
you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a
|
||||
user or a fragment. Its decision list is at `v1/decisions` under that URL, and
|
||||
is fetched every minute. Set without the key, it stops the start. So does a
|
||||
decision list URL that is also in `SWWAF_BLOCKLIST_URLS`, or that is
|
||||
`SWWAF_ASN_LIMIT_PERCENT_URL`.
|
||||
- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
|
||||
`smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
|
||||
URL, it stops the start.
|
||||
- `SWWAF_REPUTATION_ACTION` (default `limit:25`): what is done with a client on
|
||||
a DNS blocklist, or one whose AbuseIPDB score is a hit. `deny`,
|
||||
`limit:<percent>` or `log`, as for `SWWAF_BLOCKLIST_ACTION`. Such a verdict is
|
||||
less certain than the Spamhaus DROP list, so by default the client gets a
|
||||
quarter of every limit.
|
||||
- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or
|
||||
AbuseIPDB's score is used. A duration; it cannot be `off`.
|
||||
- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a
|
||||
check with AbuseIPDB may take. A duration; it cannot be `off`.
|
||||
|
||||
See [`reputation.md`](reputation.md) for how each source is asked and what it
|
||||
does with the answer.
|
||||
|
||||
## Attack detection
|
||||
|
||||
- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the
|
||||
rule files (see [`rules.md`](rules.md)). A directory that cannot be read stops
|
||||
the start. An empty one is no error.
|
||||
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file.
|
||||
- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only
|
||||
scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of a code
|
||||
forge. A request for one bans the client, as a `ban` rule does. Each starts
|
||||
with `/` and holds no `?`. A trap path is matched whole, without the query,
|
||||
against the path as sent and once percent-decoded. `/wp-login.php` matches
|
||||
`/wp-login.php?x=1` and `/%77p-login.php`. It does not match `/wp-login.php/`
|
||||
or `/blog/wp-login.php`.
|
||||
- `SWWAF_WAF_MODE` (default `block`): what the Core Rule Set does with a request
|
||||
it takes for an attack. `block` refuses it with `403`. `detect` lets it
|
||||
through, logged and alerted. `off` inspects no request.
|
||||
- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level,
|
||||
from 1 to 4. Each level up runs more rules. Those find more attacks, and take
|
||||
more ordinary requests for attacks.
|
||||
- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the score at which a request is a
|
||||
match. Each rule that matches adds to the request's score, and a rule rated
|
||||
critical adds 5. So by default one such rule is enough. A whole number above
|
||||
zero, or `off`. With `off` no request is a match, and scores are still logged.
|
||||
- `SWWAF_WAF_DISABLED_RULES` (default
|
||||
`920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules
|
||||
to switch off. The log line names the rules a request matched, so a rule that
|
||||
refuses your app's ordinary requests can be found there. "Attack detection" in
|
||||
[`SPEC.md`](../SPEC.md) says what the six default ids refuse and why they are
|
||||
off. A list you give replaces the default, so name these six ids in it to keep
|
||||
them off. An id must be a whole number above zero. An id from 900000 to 900999
|
||||
is refused: those are the rules that set the Core Rule Set up, and
|
||||
`smallwebwaf`'s own.
|
||||
- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does
|
||||
not inspect, matched as `SWWAF_RATE_LIMIT_EXEMPT_PATHS` is.
|
||||
- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` reads no request body. A size,
|
||||
at most `1G`, switches body inspection on. The Core Rule Set then reads a form
|
||||
data or multipart body up to that size, and the rest streams on to the app. It
|
||||
reads a JSON or XML body no larger than that size, since those cannot be read
|
||||
in part. Any other body, such as a git push, reaches the app uninspected. See
|
||||
[`core-rule-set.md`](core-rule-set.md).
|
||||
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
|
||||
in a minute for a rule match, a trap path, a Core Rule Set match or a failed
|
||||
token check. One more bans the client, as a broken rate limit does. A count or
|
||||
`off`. The app's own `401` and `404` answers are not counted.
|
||||
|
||||
## How long bans last and what they cover
|
||||
|
||||
- `SWWAF_BAN_RESPONSE` (default `403`): how a refused client is answered: `403`,
|
||||
`429`, or `close` to close the connection without an answer. A `block` rule
|
||||
and the Core Rule Set always answer `403`.
|
||||
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken rate
|
||||
limit, byte limit or error burst. A duration; it cannot be `off`.
|
||||
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): a client that breaks a limit
|
||||
again within this time after its ban ended is banned three times as long. A
|
||||
duration; it cannot be `off`.
|
||||
- `SWWAF_MAX_BAN_DURATION` (default `7d`): a ban for a broken limit that would
|
||||
be longer is permanent instead. A duration; it cannot be `off`.
|
||||
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
||||
attack. A duration; it cannot be `off`.
|
||||
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
|
||||
kept, whether past, active or permanent. A whole number above zero. Bans whose
|
||||
cause is `admin` are kept besides and never dropped (see
|
||||
[`bans.md`](bans.md)).
|
||||
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): the length of the netblock around
|
||||
an IPv4 client that a ban covers, from 0 to 32. `24` bans the surrounding /24.
|
||||
An IPv6 ban covers the client's group.
|
||||
|
||||
## Where and how often the state files are written
|
||||
|
||||
- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the directory of the state
|
||||
files, an absolute path. A directory that does not exist or cannot be written
|
||||
stops the start.
|
||||
- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
|
||||
written after a ban is made, lifted or made permanent. Every change in between
|
||||
goes into the same write. A duration; it cannot be `off`.
|
||||
- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is
|
||||
written. A duration; it cannot be `off`.
|
||||
|
||||
## Logging
|
||||
|
||||
- `SWWAF_LOG_REQUEST_HEADERS` (default
|
||||
`accept,accept-language,accept-encoding,content-type,origin,range`): the
|
||||
request headers to record in the log line, in either case. `Authorization`,
|
||||
`Cookie` and `Set-Cookie` are never logged, even if listed. `Host` and
|
||||
`Transfer-Encoding` are refused: Go's HTTP server takes them out of the
|
||||
request. The host is the field `host`.
|
||||
- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own
|
||||
messages that are written: `debug`, `info`, `warn` or `error`. No message is
|
||||
at `debug`. It holds back no request log line. `warn` and `error` hold back
|
||||
the line of settings at start.
|
||||
- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server every line on stdout
|
||||
is also sent to, such as `syslog+tls://logs.example:6514`. The forms are
|
||||
`syslog+udp://`, `syslog+tcp://` and `syslog+tls://`, each with a host and a
|
||||
port and nothing more.
|
||||
- `SWWAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates
|
||||
that a `syslog+tls` server's certificate must chain to, instead of the host's
|
||||
own. A file that cannot be read, or holds no certificate, stops the start.
|
||||
- `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they
|
||||
wait to be sent, a whole number above zero.
|
||||
- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`,
|
||||
`user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, `news`, `uucp`, `cron`,
|
||||
`authpriv`, `ftp`, or `local0` to `local7`.
|
||||
- `SWWAF_LOG_REMOTE_APP_NAME` (default: `SWWAF_INSTANCE_NAME`): the app name in
|
||||
each syslog record, 1 to 48 printable ASCII characters without a space.
|
||||
|
||||
## Tokens and metrics
|
||||
|
||||
- `SWWAF_ADMIN_TOKEN` (default unset): the token for the ban endpoints and
|
||||
`/_smallwebwaf/clients/<ip>`. A long random value of at least 32 characters.
|
||||
Unset, those endpoints are off. Give it as a file to keep it from the app (see
|
||||
"Settings given as files" below).
|
||||
- `SWWAF_METRICS_TOKEN` (default unset): the token a scraper sends for
|
||||
`/_smallwebwaf/metrics`, at least 32 characters. Unset, the metrics are off.
|
||||
It is separate from the admin token, so a scraper cannot manage bans.
|
||||
- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many
|
||||
countries get series of their own in the metrics. The rest count as `other`. A
|
||||
whole number above zero.
|
||||
|
||||
## Alert destinations
|
||||
|
||||
- `SWWAF_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each
|
||||
alert as a JSON object. An `http` or `https` URL without a user or a fragment.
|
||||
The settings logged at start show its path and query as `********`, since many
|
||||
webhooks carry their secret there.
|
||||
- `SWWAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post,
|
||||
such as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and
|
||||
value. A value cannot hold a comma.
|
||||
- `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook
|
||||
that receives each alert as a message. It is checked and masked as the webhook
|
||||
URL is.
|
||||
- `SWWAF_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each
|
||||
alert, as the topic's full URL, such as `https://ntfy.sh/my-alerts`. It is
|
||||
checked and masked as the webhook URL is.
|
||||
- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token, sent as
|
||||
`Authorization: Bearer <token>`. A control character in it is refused.
|
||||
- `SWWAF_ALERT_EVENTS` (default
|
||||
`ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`):
|
||||
the events alerts are sent for. Any other name is refused.
|
||||
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held
|
||||
back. A duration or `off`.
|
||||
- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour.
|
||||
The rest are rolled into one summary. A whole number above zero, or `off`.
|
||||
|
||||
See [`alerts.md`](alerts.md) for the events and what each destination gets.
|
||||
|
||||
## Anomaly thresholds
|
||||
|
||||
These only alert. They refuse and ban nothing. Each scope has four thresholds:
|
||||
the most requests and the most bytes counted in a minute and in an hour.
|
||||
Requests are counts, bytes are sizes. Each is `off` by default. What is unusual
|
||||
depends on each service's normal traffic, which the metrics show. The four
|
||||
thresholds per client are:
|
||||
|
||||
- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
||||
- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
||||
- `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE`;
|
||||
- `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR`.
|
||||
|
||||
The other scopes' names have another prefix before the same four endings, such
|
||||
as `REQUESTS_PER_MINUTE`:
|
||||
|
||||
- `SWWAF_ANOMALY_NET_...`: per netblock around a client. Two settings give the
|
||||
netblock's length:
|
||||
- `SWWAF_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) for an IPv4
|
||||
client;
|
||||
- `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
||||
client.
|
||||
- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs lookups on.
|
||||
- `SWWAF_ANOMALY_TOTAL_...`: the whole service.
|
||||
- `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole.
|
||||
`SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
|
||||
as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
|
||||
name, or with a bad netblock, is refused. So is a name used twice.
|
||||
|
||||
Every request but the health check is counted, whatever was done with it,
|
||||
clients in `SWWAF_ALLOW_NETS` included. At most 20,000 counters are kept.
|
||||
`alerts.json` keeps them across a restart.
|
||||
|
||||
## Settings given as files
|
||||
|
||||
Any setting may be given as a file holding its value. Add `_FILE` to the
|
||||
setting's name: `SWWAF_METRICS_TOKEN_FILE` names the file that holds
|
||||
`SWWAF_METRICS_TOKEN`. The file's contents are the value, less one newline at
|
||||
the end. They are checked as the value itself would be. Setting both forms stops
|
||||
the start, and so does naming a file that cannot be read.
|
||||
`SWWAF_LOG_REMOTE_TLS_CA_FILE` already holds a file name, so it has no `_FILE`
|
||||
form.
|
||||
|
||||
A token given as a file is out of the app's reach only while the `smallwebwaf`
|
||||
user alone can read the file. Make it on the host, owned by uid 65532 with mode
|
||||
`0400`:
|
||||
|
||||
```sh
|
||||
mkdir -p /srv/app/tokens
|
||||
openssl rand -hex 32 > /srv/app/tokens/metrics
|
||||
chown 65532:65532 /srv/app/tokens/metrics
|
||||
chmod 0400 /srv/app/tokens/metrics
|
||||
```
|
||||
|
||||
Mount its directory into the container read-only, and name the file:
|
||||
|
||||
```sh
|
||||
docker run \
|
||||
-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro \
|
||||
-e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics \
|
||||
...
|
||||
```
|
||||
|
||||
## Fixed limits
|
||||
|
||||
A few limits are fixed rather than settings:
|
||||
|
||||
- at most 100,000 GeoJS answers are kept, for 7 days each;
|
||||
- at most 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept;
|
||||
- at most 1,000 DNS blocklist queries are under way at once;
|
||||
- the CrowdSec decision list is fetched again a minute after the last fetch.
|
||||
@@ -1,102 +0,0 @@
|
||||
# State files
|
||||
|
||||
`smallwebwaf` keeps its state in memory, and a copy of it in five JSON files in
|
||||
`SWWAF_STATE_DIR`. The files are meant for people as well: you can read or edit
|
||||
them at any time, and the running `smallwebwaf` takes the edit in. The
|
||||
requirements they meet are under "Persistent state" in [`SPEC.md`](../SPEC.md).
|
||||
|
||||
## The files
|
||||
|
||||
Each has a top-level `version` of 1.
|
||||
|
||||
- `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf`
|
||||
made and every ban you made, indented to be read. Per entry:
|
||||
- the netblock, start and expiry (`null` for permanent);
|
||||
- the cause: `attack`, `limit`, `admin` or `crowdsec`;
|
||||
- a short reason;
|
||||
- when it was lifted, if you lifted it;
|
||||
- the notes (see [`bans.md`](bans.md)).
|
||||
- `clients.json`: per client, its minute, hour and day counters and its history
|
||||
since it was first seen, one client per line, so `grep` shows everything about
|
||||
one. The history holds:
|
||||
- the first and last time seen;
|
||||
- the AS number, AS name and country last looked up, and when;
|
||||
- total requests, and how many were forwarded and how many refused;
|
||||
- bytes in each direction;
|
||||
- responses by status class;
|
||||
- offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and
|
||||
`token_refused`.
|
||||
|
||||
Clients that were never banned are kept too, so every client's history
|
||||
survives a restart.
|
||||
|
||||
- `lookups.json`: GeoJS answers, one per line: the AS number, AS name and
|
||||
country, when GeoJS was asked and when the answer was last used.
|
||||
- `reputation.json`:
|
||||
- the last good copy of each list fetched from a URL, with when it was last
|
||||
tried;
|
||||
- each DNS blocklist verdict still in use;
|
||||
- AbuseIPDB's scores, and the checks spent today.
|
||||
- `alerts.json`:
|
||||
- the anomaly counters;
|
||||
- the alert cooldowns: for each event and netblock, source, file or thing
|
||||
counted, when it was last sent and the count of repeats held back since;
|
||||
- the alerts sent and held back in the current hour;
|
||||
- the alerts waiting for each destination.
|
||||
|
||||
Times are RFC 3339 UTC. Durations and sizes are plain numbers, with the unit in
|
||||
the field name. `bans.json`, `reputation.json` and `alerts.json` are indented;
|
||||
`clients.json` and `lookups.json` put each entry on one line.
|
||||
|
||||
## Sizes
|
||||
|
||||
`clients.json` takes about 1 KiB per client, about 20 MiB at the default
|
||||
`SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of
|
||||
disk writes a day. `bans.json` takes about 1.2 KiB per ban, at most about 2.5
|
||||
KiB, so about 6 MiB at the default `SWWAF_MAX_BANS`. With a full file, a hundred
|
||||
new bans a day come to about 600 MiB of writes. `lookups.json` takes about 150
|
||||
bytes per answer, under 1.5 GiB of writes a day when full. The other two files
|
||||
are usually a few MiB or less.
|
||||
|
||||
## Writing
|
||||
|
||||
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted or
|
||||
made permanent. Every file is written every `SWWAF_STATE_COUNTER_INTERVAL`, and
|
||||
when `smallwebwaf` stops. Each write goes to a temporary file that then replaces
|
||||
the file, so a crash leaves the old file or the new one, whole. Before writing a
|
||||
file, `smallwebwaf` takes in any edit you made to it, so your edit is not
|
||||
overwritten. A write that fails is logged, raised as a `file_error` alert, and
|
||||
tried again at the next write. A hard kill loses what changed since the last
|
||||
write:
|
||||
|
||||
- ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds);
|
||||
- everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes):
|
||||
- counts and history, and the counts in ban notes;
|
||||
- GeoJS answers;
|
||||
- reputation verdicts and the AbuseIPDB count;
|
||||
- anomaly counters and alert cooldowns.
|
||||
|
||||
## Reading at start
|
||||
|
||||
The files are read back. Each client keeps its counts, so a restart gives no one
|
||||
a fresh allowance. Each ban keeps refusing until it ends. Counter buckets and
|
||||
cache entries whose time has passed are discarded. A missing file is empty
|
||||
state. These stop the start, with a message naming the file and the place:
|
||||
|
||||
- a file that does not parse, or has another `version`;
|
||||
- an entry without a field it needs;
|
||||
- a ban whose `cause` is not `limit`, `attack`, `admin` or `crowdsec`;
|
||||
- a ban whose `expires` is not after its `start`.
|
||||
|
||||
## Editing while it runs
|
||||
|
||||
`smallwebwaf` watches the directory and takes in your edit of a file as soon as
|
||||
you save it. What the file then holds replaces what `smallwebwaf` held for it.
|
||||
It tells its own writes from yours by comparing the file with what it last read
|
||||
or wrote. A change it made after you opened the file, such as a new ban, is lost
|
||||
when you save over it; most editors warn when a file changed on disk while it
|
||||
was open. An edit with an error that would stop the start does not stop the
|
||||
running `smallwebwaf`. It keeps what it holds. At the file's next write it
|
||||
renames your file to `<name>.bad`, writes the file again from memory, and logs
|
||||
and alerts the file and the place of the error. Mend the `.bad` file and move it
|
||||
back. A file you remove is written again at its next write.
|
||||
@@ -1,8 +1,8 @@
|
||||
// Package alerts sends alerts on bans, on traffic over an anomaly
|
||||
// threshold, on a source that fails and on a file with an error to each
|
||||
// destination set: to the webhook
|
||||
// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON
|
||||
// object" in docs/alerts.md describes, to the Slack incoming
|
||||
// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as the "Alert
|
||||
// webhook schema" section of SPEC.md describes, to the Slack incoming
|
||||
// webhook SWWAF_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
|
||||
// ntfy topic SWWAF_ALERT_NTFY_URL names. A repeat within
|
||||
// SWWAF_ALERT_COOLDOWN is held back, and so is an alert past
|
||||
@@ -149,10 +149,11 @@ type Params struct {
|
||||
}
|
||||
|
||||
// Alert is one alert, as the webhook is sent it and alerts.json holds it,
|
||||
// with the fields of "The JSON object" in docs/alerts.md. ASN, ASName and
|
||||
// Country are, for a ban, the client's as the ban's notes give them.
|
||||
// with the fields of the "Alert webhook schema" section of SPEC.md. ASN,
|
||||
// ASName and Country are, for a ban, the client's as the ban's notes give
|
||||
// them.
|
||||
//
|
||||
//nolint:tagliatelle // docs/alerts.md names the alert's fields in snake_case
|
||||
//nolint:tagliatelle // SPEC.md's alert webhook schema names its fields in snake_case
|
||||
type Alert struct {
|
||||
Instance string `json:"instance"`
|
||||
Time time.Time `json:"time"`
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
// Package anomaly counts requests and bytes over a minute and an hour, per
|
||||
// client, per surrounding netblock, per AS number, for the whole service
|
||||
// and per named netblock, and raises an anomaly alert for a count over its
|
||||
// threshold, as "Anomaly thresholds" under "Settings" in SPEC.md
|
||||
// describes. It refuses and bans nothing. At most 20,000 counters
|
||||
// threshold, as "Anomaly thresholds" under "Configuration surface" in
|
||||
// SPEC.md describes. It refuses and bans nothing. At most 20,000 counters
|
||||
// are kept, in memory, and written to alerts.json and read from it by the
|
||||
// state package.
|
||||
package anomaly
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
// Package metrics keeps smallwebwaf's Prometheus metrics, as
|
||||
// docs/metrics.md lists them, and serves them in the Prometheus text
|
||||
// format, as the "Metrics endpoint" section of SPEC.md requires. No
|
||||
// metric carries a client's address.
|
||||
// Package metrics keeps smallwebwaf's Prometheus metrics, as the "Metrics
|
||||
// endpoint" section of SPEC.md lists them, and serves them in the
|
||||
// Prometheus text format. No metric carries a client's address.
|
||||
package metrics
|
||||
|
||||
import (
|
||||
|
||||
@@ -206,7 +206,7 @@ type Hit struct {
|
||||
// Counts are a client's requests and bytes in the minute, the hour and
|
||||
// the day that end at a request, that request's included.
|
||||
//
|
||||
//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
|
||||
//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
|
||||
type Counts struct {
|
||||
Minute float64 `json:"minute"`
|
||||
Hour float64 `json:"hour"`
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
// Package remotelog sends the lines smallwebwaf writes on stdout to the
|
||||
// remote log endpoint, SWWAF_LOG_REMOTE_URL, as the "Request log" section
|
||||
// of SPEC.md requires and "Sending the log to a syslog server" in
|
||||
// docs/request-log.md describes: each line as the message of an RFC 5424
|
||||
// syslog record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so
|
||||
// a slow or unreachable endpoint never holds up a request or stdout.
|
||||
// of SPEC.md describes: each line as the message of an RFC 5424 syslog
|
||||
// record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so a slow
|
||||
// or unreachable endpoint never holds up a request or stdout.
|
||||
package remotelog
|
||||
|
||||
import (
|
||||
|
||||
@@ -64,10 +64,11 @@ const LimitHitErrorBurst = "error_burst"
|
||||
const timeLayout = "2006-01-02T15:04:05.000Z07:00"
|
||||
|
||||
// Line is one request's line in the request log. The field names, and
|
||||
// their order, are those of docs/request-log.md. A field that may not
|
||||
// apply to a request is left out of its line when it does not.
|
||||
// their order, are those of the "Request log" section of SPEC.md. A field
|
||||
// that may not apply to a request is left out of its line when it does
|
||||
// not.
|
||||
//
|
||||
//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
|
||||
//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
|
||||
type Line struct {
|
||||
Type string `json:"type"`
|
||||
|
||||
|
||||
+2
-2
@@ -1,8 +1,8 @@
|
||||
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||
// method, the URL with its query and the headers of a request, and on its
|
||||
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
||||
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
|
||||
// them. It reads no response.
|
||||
// makes to it, as "Attack detection" under "Configuration surface" in
|
||||
// SPEC.md describes them. It reads no response.
|
||||
//
|
||||
// smallwebwaf writes only to its state directory, so Coraza is built with
|
||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||
|
||||
@@ -2,12 +2,11 @@
|
||||
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
|
||||
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
|
||||
# which names gitea's container. Then send through it the gitea requests
|
||||
# that reading gitea's source found the Core Rule Set refuses, which
|
||||
# docs/gitea.md records, and a short sample of gitea's ordinary use:
|
||||
# signing in, pages of files and commits, a diff, the API, a push, clone
|
||||
# and fetch over HTTP, and two container image pushes. Last, print each
|
||||
# request smallwebwaf logged, with what it did with it and the ids of the
|
||||
# Core Rule Set's rules that matched it.
|
||||
# that reading gitea's source found the Core Rule Set refuses, and a short
|
||||
# sample of gitea's ordinary use: signing in, pages of files and commits,
|
||||
# a diff, the API, a push, clone and fetch over HTTP, and two container
|
||||
# image pushes. Last, print each request smallwebwaf logged, with what it
|
||||
# did with it and the ids of the Core Rule Set's rules that matched it.
|
||||
# Every request comes from one client, docker's gateway, so more than
|
||||
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
|
||||
# the requests after would show as banned. The containers, the network,
|
||||
|
||||
Reference in New Issue
Block a user