sneak/smallwebwaf
1
0
Fork 0
Code Issues 5 Pull Requests 1 Actions 14 Packages Projects Releases Wiki Activity

README rewritten from scratch so a newcomer can follow it (closes #154, closes #155)
check / check (push) Waiting to run
Details

Browse Source 
The old README opened with project history and one list of a hundred
settings. The new one leads with what smallwebwaf is, what it protects
against, how to build and deploy it, what happens to a request, the
settings most deployments touch, how to check it works, the entrypoints,
the rationale and the design, as REPO_POLICIES.md requires. The reference
lists, every setting, the rule file format, the endpoints, the request
log's fields, the alert events, the metrics, the state files, the image
and the gitea notes, move to files under docs/, each linked from the
README, with SPEC.md as the one place for behaviour. Every statement was
checked against the code on next.

Model: fable-5-1
This commit is contained in:
sneak
2026-10-22 22:29:53 +00:00
parent eafcc3fd73
commit 1dbe17e594
11 changed files with 1492 additions and 2344 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+365 -2343
View File
File diff suppressed because it is too large Load Diff
docs/admin-endpoints.md
+48
View File
@@ -0,0 +1,48 @@
# Admin endpoints
The endpoints under `/_smallwebwaf/`, what each takes and what each answers.
"Admin endpoints" in [`SPEC.md`](../SPEC.md) says which checks they go through
and how their tokens work.
- `GET /_smallwebwaf/healthz`: `200` and `ok`, to anyone.
- `GET /_smallwebwaf/metrics`: the metrics, for `SWWAF_METRICS_TOKEN` (see
[`metrics.md`](metrics.md)).
- `GET /_smallwebwaf/bans`: every ban held, under `bans`, each as `bans.json`
lists it. Needs `SWWAF_ADMIN_TOKEN`, as do the three below.
- `POST /_smallwebwaf/bans`: bans a netblock. The body is a JSON object of
`netblock`, `duration` and, if you like, `reason`. `netblock` is a netblock
such as `203.0.113.0/24`, or a client's address, which bans its IPv4 netblock
(`SWWAF_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a duration such
as `1h` or `7d`, or `permanent`. A bad body is answered `400`, saying what is
wrong:
- one that is not such an object, or has another field;
- one longer than 4 KiB;
- one that names an IPv4-mapped netblock, or an address with a zone.
- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban covering the
address `<client>`. It answers `404` when none is active.
- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client.
Under `client`, its counters and history as `clients.json` holds them, or
`null` when `clients.json` does not hold it. Under `bans`, every ban covering
the address, with its notes.
## Examples
With `SWWAF_ADMIN_TOKEN` set, and the token in `$TOKEN`:
```sh
# Every ban.
curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans
# Ban 203.0.113.0/24 for seven days.
curl -H "Authorization: Bearer $TOKEN" \
--json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \
https://app.example/_smallwebwaf/bans
# Lift the bans on 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" -X DELETE \
https://app.example/_smallwebwaf/bans/203.0.113.9
# What smallwebwaf knows of 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" \
https://app.example/_smallwebwaf/clients/203.0.113.9
```
docs/alerts.md
+130
View File
@@ -0,0 +1,130 @@
# Alerts
The alert events, the JSON object the webhook gets, and the messages Slack and
ntfy get. "Alerting" in [`SPEC.md`](../SPEC.md) says when an alert is sent, held
back or dropped.
## Events
- `ban`: a ban `smallwebwaf` made.
- `permanent_ban`: a permanent ban it made, or an existing ban made permanent by
a request.
- `waf_block`: the Core Rule Set scored a request at or over the threshold. The
request was refused unless the alert's `detail` gives `mode`: `detect` or
`observe`.
- `anomaly`: a count over an anomaly threshold.
- `reputation_hit`: a request from a client on a blocklist, the CrowdSec
decision list or a DNS blocklist, or with an AbuseIPDB score that is a hit.
One alert per source.
- `source_failure`: one of:
- GeoJS failing;
- a list that cannot be fetched;
- a DNS blocklist or AbuseIPDB failing or refusing a query;
- the day's AbuseIPDB checks used up.
- `file_error`: one of:
- a rule file or state file edit with an error;
- a state file that cannot be written;
- a replacement lookup database that cannot be read.
- `summary`: the hourly summary of alerts held back.
## The JSON object
This is the alert for a ban for a broken rate limit, shown indented; it is sent
on one line:
```json
{
"instance": "fsn1app1/gitea",
"time": "2026-10-06T12:00:00.123461Z",
"event": "ban",
"client": "203.0.113.9",
"netblock": "203.0.113.9/32",
"asn": "",
"as_name": "",
"country": "",
"reason": "requests per minute over the limit of 1000",
"detail": {
"ban_expires": "2026-10-06T13:00:00.123Z",
"cause": "limit",
"notes": {
"asn": "",
"as_name": "",
"country": "",
"kind": "requests",
"limit": 1000,
"window": "minute",
"count": 1001,
"request": {
"time": "2026-10-06T12:00:00.123456789Z",
"method": "GET",
"host": "app.example",
"path": "/owner/repo/commits/branch/main?page=812",
"status": 403,
"user_agent": "scraper/1.0"
},
"requests": 5210,
"refused": 0,
"earlier_bans": {
"limit": 0,
"attack": 0,
"admin": 0,
"crowdsec": 0
}
}
},
"suppressed_repeats": 0
}
```
- `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
one of the events above.
- `client`: the address whose request raised the alert. Empty for
`source_failure` and `file_error`.
- `netblock`: empty for `source_failure` and `file_error`; otherwise one of:
- the ban's netblock;
- the netblock an anomaly counted;
- for the other events, the client as rate limiting counts it: an IPv4
address as a /32, or an IPv6 group.
- `asn`, `as_name` and `country`: the client's, as known when the alert was
raised. They are empty when GeoJS had not answered yet.
- `reason`: a short sentence, such as
`requests per minute over the limit of 1000` or `listed by a blocklist`.
- `detail`: what is particular to the event:
- for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed
under `bans.json` in [`state-files.md`](state-files.md));
- for an anomaly, the `scope`, `window`, `kind`, `count` and `threshold`;
- for a reputation hit, the `source` and, for AbuseIPDB, the `score`;
- for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`;
- for a source failure, the `source` and `error`;
- for a file error, the `file` and `error`;
- `mode`, when the request was not refused: `detect` or `observe`.
- `suppressed_repeats`: how many repeats the cooldown held back before this
alert.
## Slack and ntfy
Slack and ntfy get a message with a title and a text. The title is the instance
and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then one
line for each of these that the alert has:
- the client, the netblock and the country;
- the file, the source, the error and the mode from `detail`;
- the suppressed repeats, if any.
Slack gets the title in bold and the text below it, with `&`, `<` and `>`
escaped. ntfy gets the text as the message, with these headers:
- `Title`: the title;
- `Authorization`: the token, as `Bearer <token>`, while
`SWWAF_ALERT_NTFY_TOKEN` is set;
- `Priority` and `Tags`: by event, as listed below.
The priority and tag by event:
- `ban`: priority `default`, tag `no_entry`.
- `permanent_ban`: priority `high`, tag `no_entry`.
- `waf_block`: priority `default`, tag `shield`.
- `anomaly`: priority `high`, tag `chart_with_upwards_trend`.
- `reputation_hit`: priority `low`, tag `label`.
- `source_failure` and `file_error`: priority `high`, tag `warning`.
- `summary`: priority `default`, tag `bar_chart`.
docs/gitea.md
+215
View File
@@ -0,0 +1,215 @@
# smallwebwaf in front of gitea
These notes record what `smallwebwaf` does in front of gitea at the default
settings. They are observations, not requirements; the requirements are in
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind the image and
sends through it a sample of gitea's ordinary use and the requests named below.
It reports each request the Core Rule Set refuses, so that these notes can be
checked again when gitea or the Core Rule Set changes version.
## What the defaults cover
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
by it.
- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all
but the largest repositories and slowest links.
- A push is a request, and so is every other upload:
- a git push;
- an LFS object;
- a container image layer;
- a package file;
- a release attachment.
At the defaults, an upload larger than 100 MiB or taking more than 60
seconds is cut off ("Size and time limits" in `SPEC.md`). A gitea that takes
large uploads needs `SWWAF_REQUEST_MAX_BYTES`,
`SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised
to fit.
- Archive download and blame or history pages are what scrapers hammer. The
request limits do most of the work there.
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
43,200 requests a day, and reports a running job's log and state every second.
So it passes the day limit of 50,000 after about an hour and a quarter of
jobs. It is then banned, the job it is running fails, and each repeat within a
day makes the ban longer. Newer runners ask less often, but a busy one, or
several on one address, can still pass it. Put the runners' addresses in
`SWWAF_RATE_LIMIT_EXEMPT_NETS`.
## What the Core Rule Set lets through
At the defaults, the Core Rule Set lets gitea's ordinary use through, apart from
the refusals in the next section:
- browsing and views of files in a repository, with their history, blame and the
file tree;
- diffs, including their hidden lines and large files, and pull request review;
- git's clone, fetch and push over HTTP;
- signing in, including the return to the page a visitor came from, and sign-in
with Git Credential Manager, git-credential-oauth or tea;
- the API's calls for a file and its commits;
- pushing and pulling container images and packages;
- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
- posting issues, pull requests, comments, wiki pages and files saved in the web
editor, code included, since no body is read;
- the page gitea shows after a change, with its message naming what was done.
## What the Core Rule Set still refuses
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
toward). The request log names the rule in `waf_rule_ids`, which
`SWWAF_WAF_DISABLED_RULES` can switch off.
### A query string that reads as an attack
Most often this is a search. A search term is a query parameter, which the Core
Rule Set checks against its lists of system files and commands (see the fourth
change under "Attack detection" in `SPEC.md`). A release attachment uploaded
through the API as `docker-compose.yml` is refused the same way. Examples:
- a name on those lists, such as `package.json`, `.gitignore` or
`docker-compose.yml`;
- text that starts with a command name, such as `python3` or `ssh key`, or has
one right after a `/` in its first word, such as `feat/docker-support`;
- a shell command with its options or a system path (`ls -la`, `sed -i`,
`/bin/sh`), or a command in backticks;
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
statement (`SELECT * FROM users WHERE`);
- a URL naming an IP address or `localhost`.
### A branch, tag or file name that starts with a command name
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
`python3`, `ansible`, `base64` and `whoami`. A query parameter holding a branch,
tag or file name that starts with one is refused. So is one that has such a name
right after a `/`, such as `feat/docker-support`, `fix/docker-build` or
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
query parameter that 930120, 932160 and 932260 still check:
- `refSubUrl`, the branch or tag of a directory listing. A listing sends it when
it asks for its entries' last commits in a second request. It does so whenever
gitea takes more than a second to work them out;
- `name`, when a branch is deleted or restored on the branches page;
- `tag`, when a release is started from a tag;
- `template`, the file of an issue template, when an issue is opened from it;
- `rule_name`, when a branch protection rule is opened for editing in the
repository's settings or has just been saved.
What a visitor sees, for a branch named `docker-build`:
- A directory listing that makes that second request leaves those last commits
out and shows an error.
- The branches page shows an error instead of deleting or restoring the branch.
- A release started from such a tag gets the `403` answer of `smallwebwaf` in
place of its form. So does an issue opened from such a template, and such a
rule opened for editing.
- Saving such a rule stores it. gitea then returns to the branch settings page,
which gets the `403` answer, so the save looks as if it failed.
### A file name in a query parameter that trips the code rules
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
look for in some file names. This holds too in the ten query parameters that
930120, 932160 and 932260 do not check (see the fourth change under "Attack
detection" in `SPEC.md`). Examples:
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
(944110);
- a Java class, as in `java.lang.String.md` (944130);
- `document.write.js` (941180);
- `process.env.d.ts` (934100);
- `__proto__` (934130);
- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
For such a file, these requests are refused:
- the API's list of its commits (`path`);
- the API's call for the contents of files that names it in its query (`body`);
- the commit graph of that file alone (`file`);
- the hidden lines of its diff (`path`), which stay hidden;
- the sign-in page, for a visitor who opens a page of the file that needs
signing in, such as its edit page (`redirect_to`). The visitor can still sign
in by opening the sign-in page itself.
### A path
- A file name ending in `~`, or an `.xhtml` file whose path holds a space.
- A path that holds what the Core Rule Set takes for a Java class (944130), as
`java.lang.String.md` and
`META-INF/services/org.apache.commons.logging.LogFactory` do.
- A path that holds what it takes for Node.js code (934100), as
`process.env.d.ts` does.
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
For such a file, these are refused:
- the page of the file;
- its raw file;
- its history and its blame;
- its edit page.
The directory that holds it lists it, and a clone carries it.
### A container image push that reuses a layer
A push can reuse a layer of an image docker has pushed to that gitea before.
docker then asks gitea to take the layer from that image, naming it in the query
parameter `from`, which 932260 checks. The request is refused when the image's
owner starts with an entry on the list of commands. It is refused too when the
image's name after the owner starts with one. Examples are `owner/python3-app`,
`owner/docker-base`, and any image of an owner named `ansible` or `docker-team`.
What happens next depends on docker's image store. A docker that keeps its
images in containerd's image store fails the push, and fails each new try the
same way. A new install of Docker Engine 29 does this. A docker that keeps them
in its own older image store uploads the layer instead. The push then succeeds
after one refusal for each layer it reuses, and each of those refusals counts
toward the error burst ("Attack detection" in `SPEC.md`).
### Artifact uploads from `actions/upload-artifact@v3`
This action sends the header `Content-Range`, which the Core Rule Set refuses
(920450). The action sends two files at a time, does not retry a `403`, and
sends no further file once one is refused. So an upload is refused once or
twice, however many files it holds, and the step fails. One upload does not ban
the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
than 30 such refusals from its address in a minute, as when many jobs upload at
once. `actions/upload-artifact@v4` does not send the header.
## What body inspection refuses
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
front of gitea it then refuses these, until their rule ids, named in the request
log, are added to `SWWAF_WAF_DISABLED_RULES`:
- issue and comment text, wiki pages and files saved in the web editor that hold
shell commands or code (932125, 932235, 932250 and others);
- package descriptions that show code;
- PyPI uploads (922130);
- attachments named like `debug.log` or `config.yml` (932180).
## What `script/gitea-requests` sends
Of the requests above, the script sends:
- for each file named under "A file name in a query parameter that trips the
code rules", the API's list of its commits;
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
its diff, and the API's call for the contents of files that names it;
- the sign-in page for a visitor who opens the edit page of `document.write.js`;
- the pages of `java.lang.String.md`,
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
and `.../notes.txt`;
- one container image push that reuses a layer, which it does not try again.
It does not send the raw file, history, blame or edit page of those four files,
or the pages of the directories `META-INF/services` and `...`. It also sends a
short sample of gitea's ordinary use:
- signing in;
- pages of files and commits, and a diff;
- the API;
- a push, and clone and fetch over HTTP;
- two container image pushes.
docs/image.md
+37
View File
@@ -0,0 +1,37 @@
# What the image holds
The contents of the `smallwebwaf` image, and how each part is pinned.
"Deployment" in [`SPEC.md`](../SPEC.md) says why each part is there and why it
is pinned that way.
## Contents
- Ubuntu 26.04 LTS, pinned by digest.
- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, from the snapshot of its
archive at `20261001T000000Z`.
- Nix, with nixpkgs `nixos-26.05.11045.774debe7a0d1` set up for root as
`nixpkgs`. No Nix daemon; `build-users-group =` in `/etc/nix/nix.conf`. Root's
Nix profile, `/nix/var/nix/profiles/default/bin`, last on the `PATH`.
Unpacked, nixpkgs takes about 500 MiB.
- `runsvinit` as the entrypoint, built from `github.com/peterbourgon/runsvinit`
at a fixed commit hash.
- The `smallwebwaf` binary, and the user `smallwebwaf` (uid and gid 65532).
- The service directory `/etc/service/smallwebwaf`, with the `run` script
`share/smallwebwaf.run`.
- `/var/lib/smallwebwaf` for the state files, and `/etc/smallwebwaf/rules.d`
with the default rule file `share/rules.d/00-default.rules`.
- `EXPOSE 8080`, and the `HEALTHCHECK` that runs `smallwebwaf healthcheck`.
## Pins
- Ubuntu's packages: the snapshot service, `--snapshot 20261001T000000Z` on both
`apt-get update` and `apt-get install`. The SHA-256 hash of each of the
snapshot's `InRelease` files is named in the `Dockerfile` and checked after
`apt-get update`. The hashes are the amd64 archive's.
- nixpkgs: the release file `nixexprs.tar.xz` of
`nixos-26.05.11045.774debe7a0d1` on `releases.nixos.org`, named with its
SHA-256 hash.
- `runsvinit`: the archived repository, unchanged since 2015, at its last commit
(`b4b2c78`, eight commits after `v2.0.0`), by commit hash, with a `go.mod` the
build writes.
- The Go image that builds `smallwebwaf`: by digest.
docs/metrics.md
+81
View File
@@ -0,0 +1,81 @@
# The metrics
The metrics `GET /_smallwebwaf/metrics` answers with, as "Metrics endpoint" in
[`SPEC.md`](../SPEC.md) describes it. Prometheus keeps the `instance` label as
`exported_instance` unless the scrape sets `honor_labels: true`.
## Traffic
- `smallwebwaf_requests_total`, `smallwebwaf_request_bytes_total` and
`smallwebwaf_response_bytes_total`, by `status_class` such as `2xx` and by
`action`;
- `smallwebwaf_request_duration_seconds` and
`smallwebwaf_upstream_duration_seconds`, as histograms;
- `smallwebwaf_requests_in_flight`.
## Limits and bans
- `smallwebwaf_rate_limit_hits_total`, by `window` and by `kind` (`requests`,
`bytes` or `refusals`);
- `smallwebwaf_size_and_time_limit_hits_total`, by `limit`, the setting passed;
- `smallwebwaf_offences_total`, by `kind`;
- `smallwebwaf_bans_made_total`, by `cause`;
- `smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
## Attack detection
- `smallwebwaf_rule_matches_total`, by `rule_id` and `action`, and
`smallwebwaf_rules_loaded`;
- `smallwebwaf_waf_matches_total`, by `mode` and `rule_id`, for the rules that
fired.
## Countries and AS numbers
For the `SWWAF_METRICS_TOP_N` busiest of each, the rest as `other`:
- `smallwebwaf_country_requests_total`,
`smallwebwaf_country_request_bytes_total`,
`smallwebwaf_country_response_bytes_total` and
`smallwebwaf_country_list_refusals_total`, by `country`;
- `smallwebwaf_asn_requests_total`, `smallwebwaf_asn_request_bytes_total` and
`smallwebwaf_asn_response_bytes_total`, by `asn`.
## Lookups
- `smallwebwaf_geojs_requests_total` and `smallwebwaf_geojs_failures_total`;
- `smallwebwaf_geojs_unanswered_total`: the requests that needed an answer and
went on without it;
- with the file source,
`smallwebwaf_lookup_database_last_read_timestamp_seconds` and
`smallwebwaf_lookup_database_read_failures_total`.
## Reputation sources
By `source`: each list's URL, each zone, and `abuseipdb`:
- `smallwebwaf_reputation_hits_total`, `smallwebwaf_reputation_queries_total`
and `smallwebwaf_reputation_failures_total`;
- `smallwebwaf_reputation_last_fetch_timestamp_seconds`, for a list;
- `smallwebwaf_reputation_daily_budget_remaining`, for AbuseIPDB.
## State files
By `file`:
- `smallwebwaf_state_file_writes_total`,
`smallwebwaf_state_file_write_failures_total`,
`smallwebwaf_state_file_last_write_timestamp_seconds` and
`smallwebwaf_state_file_size_bytes`;
- `smallwebwaf_state_file_edits_taken_in_total` and
`smallwebwaf_state_file_edits_set_aside_total`;
- `smallwebwaf_tracked_clients`: the clients in the table.
## Remote log, alerts and the process
- while a syslog server is set, `smallwebwaf_remote_log_lines_sent_total`,
`smallwebwaf_remote_log_lines_dropped_total` and
`smallwebwaf_remote_log_buffer_depth`;
- by `destination`, `smallwebwaf_alerts_sent_total`,
`smallwebwaf_alerts_failed_total`, `smallwebwaf_alerts_suppressed_total` and
`smallwebwaf_alerts_dropped_total`;
- Go's own `go_` metrics and the process's `process_` metrics.
docs/request-log.md
+119
View File
@@ -0,0 +1,119 @@
# The request log's fields
The fields of the JSON object `smallwebwaf` writes for each request. "Request
log" in [`SPEC.md`](../SPEC.md) says when a line is written and how the log is
sent to a syslog server. A field that does not apply to a request is left out.
## Example
```
{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
```
## Every line
Every line has:
- `type`: `request` (the process's own messages have `process`);
- the standard web log fields;
- `request_id`, `peer_ip` and `client_group`;
- `asn`, `as_name` and `country`;
- `action` and `duration_total`.
## Standard web log fields
- `time`: when the request arrived, RFC 3339 with milliseconds, in UTC;
- `instance`: `SWWAF_INSTANCE_NAME`;
- `client_ip`, `method`, `scheme`, `host`, `path`, `query`, `protocol`;
- `status`: what the client was sent, `0` if nothing;
- `request_bytes` and `response_bytes`: body bytes each way;
- `referer` and `user_agent`.
## Request detail
- `request_id`: the `X-Request-ID` a trusted proxy sent, or a new random one of
26 letters and digits.
- `peer_ip`: the TCP peer, normally traefik.
- `forwarded_for`: the `X-Forwarded-For` header as received.
- `client_group`: the client as rate limiting counts it, an IPv4 address as a
/32 or an IPv6 group.
- `asn`, `as_name` and `country`: the client's, as looked up. Each is empty
when:
- lookups are off;
- the client is not looked up;
- the source cannot place the client;
- GeoJS had not answered by the time the request went on.
- `content_type` and `content_length`: the request's own.
- `request_headers`: the headers listed in `SWWAF_LOG_REQUEST_HEADERS`, by name
in lower case.
- `has_authorization` and `has_cookie`: true when the request had those headers.
Their values are never logged.
- `websocket`: true when the app switched the connection to another protocol.
## Response detail
- `upstream_status`: what the app answered. It differs from `status` when
`smallwebwaf` answered itself.
- `response_content_type`, `cache_control` and `location`: the answer's headers.
- `aborted`: true when the client went away early.
## Decision
- `action`: what was done:
- `forward`: passed to the app;
- `denied`: refused for `SWWAF_DENY_NETS`, or for a reputation source with
the action `deny`;
- `banned`: refused under a ban, or for a `ban` rule, a trap path or a
CrowdSec decision, each of which bans the client;
- `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`:
refused by that check;
- `too_large` and `timed_out`: a size or time limit was passed;
- `upstream_error`: the app could not be reached, or its answer broke off;
- `admin`: answered at one of `smallwebwaf`'s own endpoints.
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
request it would have refused. `action` then names what was done, normally
`forward`.
- `limit_percent` and `limit_percent_setting`: the client's percentage of the
rate limits and the setting that lowered it, when one did. `bytes_percent` and
`bytes_percent_setting` do the same for the byte limits.
- `counts`: the client's totals as rate limiting counts them, this request
included. It is left out when rate limiting did not count the request. It
holds:
- `minute`, `hour` and `day`: requests;
- `minute_bytes`, `hour_bytes` and `day_bytes`: bytes.
A count can have a fraction (see "Counting method" in
[`SPEC.md`](../SPEC.md)).
- `limit_hit`: the limit broken:
- `minute`, `hour` or `day` for a rate limit;
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
- `error_burst`.
A request whose bytes broke a limit was passed on whole, so its `action`
stays `forward`.
- `rule_ids`: the rule file rules that matched, in order, up to the one that
refused.
- `waf_score` and `waf_rule_ids`: the Core Rule Set's score and the rules that
matched, for a request it inspected.
- `reputation`: the sources the client is on, in this order:
- blocklist URLs;
- the CrowdSec decision list's URL;
- DNS blocklist zones;
- `abuseipdb`.
- `offence`: `limit`, for a request that broke a limit.
- `ban_expires`: when the client's ban ends, for a request that made one or was
refused under one: a time, or `permanent`.
## Timings
All in milliseconds:
- `duration_total`: from the request's headers being read to its log line;
- `duration_checks`: to the end of the checks, before forwarding;
- `duration_waf`: the part of that the Core Rule Set took;
- `duration_upstream_connect`: from the hand-over to the app until there was a
connection;
- `duration_upstream_first_byte`: until the first byte of the answer;
- `duration_upstream_total`: until the end of the answer.
docs/rules.md
+67
View File
@@ -0,0 +1,67 @@
# Rule files
The format of a rule file, and the default file the image ships. "Rule files" in
[`SPEC.md`](../SPEC.md) says how the files are read, watched and mounted, and
what each action leads to.
## The format
A rule is a line of four fields separated by spaces or tabs: an id, a target, an
action and a regex. The regex runs to the end of the line. Blank lines and lines
starting with `#` are ignored.
```
# id target action regex
env-file path ban (?i)^/\.env(\.[a-z]+)?$
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
```
- The id is letters, digits, `-` and `_`, unique across all files. The log line,
the metrics, the alerts and a ban's notes name the rule by it.
- The target is what the regex is matched against:
- `path`: the path without the query;
- `uri`: the path and the query together;
- `query`: the query as sent;
- `method`, `host`, `user_agent` and `referer`;
- `header:<Name>`: any one request header but `Host` and
`Transfer-Encoding`.
- The action is `log`, `block` or `ban`.
- The regex is in Go's syntax, RE2. It matches anywhere in the target unless
anchored with `^` and `$`. `(?i)` at its front makes it ignore case.
## The default file
The image ships one file, `share/rules.d/00-default.rules`, installed as
`/etc/smallwebwaf/rules.d/00-default.rules`:
```
# 00-default.rules: probes no real visitor sends, anchored at the site root
# id target action regex
env-file path ban (?i)^/\.env(\.[a-z]+)?$
vcs-dir path ban (?i)^/\.(git|svn|hg|bzr)(/|$)
secrets-dir path ban (?i)^/\.(aws|ssh|docker|kube)/
secret-file path ban (?i)^/\.(htpasswd|htaccess|npmrc|netrc|pgpass|git-credentials|bash_history|DS_Store)$
editor-dir path ban (?i)^/\.(vscode|idea)/
backup-file path ban (?i)^/[^/]+\.(php(\.[a-z0-9]+|~)|sql(\.[a-z0-9]+)?)$
log-file path ban (?i)^/(debug|error|access)\.log$
compose-file path ban (?i)^/(docker-)?compose\.ya?ml$
php-shell path ban (?i)^/(shell|c99|r57|wso|alfa)\.php$
scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|masscan|zgrab|wpscan)\b
path-traversal uri block (\.\./){2,}
empty-agent user_agent log ^$
```
## A file of your own
An app's `Dockerfile` adds rules of its own in a file named to sort after the
default, such as `50-gitea.rules`:
```
# 50-gitea.rules: WordPress probes, which gitea never serves
wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
```
```dockerfile
COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules
```
docs/settings.md
+325
View File
@@ -0,0 +1,325 @@
# Settings
Every setting, its default and the values it takes, in the groups of "Settings"
in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings
that take `off`, the `_FILE` form, and what each group does and why.
## Forms of values
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. Zero
and negative durations are refused.
- A size is a number of bytes with an optional `K`, `M` or `G`, which are powers
of 1024, so `1K` is 1024 bytes. Examples: `512K`, `100M`, `5G`.
- A count is a whole number of requests, such as `1000`.
- A list is comma separated, with spaces around items ignored. An empty item is
refused.
- A netblock is an address and a prefix length, such as `10.0.0.0/8`. A bare
address stands for itself alone.
- A country is a two-letter code of ISO 3166-1, in either case: `de` and `DE`
are the same. `xk` stands for Kosovo. A code that is not a country is refused,
such as `nk` (North Korea is `kp`) or the withdrawn `su`.
- An AS number is `AS` and the number, in either case, such as `AS64496`.
- A percentage is a whole number from 0 to 100.
- A path prefix starts with `/`, such as `/assets/`.
## Core
- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or
`https` URL with a host and an optional port, and nothing more.
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens: `:` and a
port, such as `:9000`. Leave the host out (see "Ports" in
[`SPEC.md`](../SPEC.md)).
- `SWWAF_INSTANCE_NAME` (default: the container's host name): the `instance`
name in every log line, alert and metric, such as `fsn1app1/gitea`. Valid
UTF-8. No control character while `SWWAF_ALERT_NTFY_URL` is set. 1 to 48
printable ASCII characters without a space while `SWWAF_LOG_REMOTE_URL` is set
and `SWWAF_LOG_REMOTE_APP_NAME` is not.
- `SWWAF_MODE` (default `enforce`): `enforce` or `observe`.
- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`):
the netblocks whose `X-Forwarded-For` is believed. Set but empty, it trusts
nothing.
- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the prefix length of an IPv6
client's group, from 32 to 128.
- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a whole
number above zero.
- `SWWAF_MAX_BANS` (default `5000`): how many of its own bans `smallwebwaf`
keeps, a whole number above zero.
## Tokens
- `SWWAF_ADMIN_TOKEN` (default unset): the token for the ban endpoints and
`/_smallwebwaf/clients/<ip>`. At least 32 characters.
- `SWWAF_METRICS_TOKEN` (default unset): the token for `/_smallwebwaf/metrics`.
At least 32 characters.
## State files
- `SWWAF_STATE_DIR` (default `/var/lib/smallwebwaf`): the directory of the state
files, an absolute path that exists and can be written.
- `SWWAF_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
written after a ban is made, lifted or made permanent. A duration.
- `SWWAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is
written. A duration.
## Logging
- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own
messages that are written: `debug`, `info`, `warn` or `error`.
- `SWWAF_LOG_REQUEST_HEADERS` (default
`accept,accept-language,accept-encoding,content-type,origin,range`): the
request headers recorded in the log line, in either case. `Host` and
`Transfer-Encoding` are refused; the host is the field `host`.
- `SWWAF_LOG_REMOTE_URL` (default unset): a syslog server that also gets every
line written to stdout: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`,
each with a host and a port and nothing more.
- `SWWAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates
that a `syslog+tls` server's certificate must chain to, instead of the host's
own. A file that cannot be read, or holds no certificate, is refused.
- `SWWAF_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they
wait to be sent, a whole number above zero.
- `SWWAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`,
`user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, `news`, `uucp`, `cron`,
`authpriv`, `ftp`, or `local0` to `local7`.
- `SWWAF_LOG_REMOTE_APP_NAME` (default: `SWWAF_INSTANCE_NAME`): the app name in
each syslog record, 1 to 48 printable ASCII characters without a space.
## Metrics
- `SWWAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many
countries get series of their own. A whole number above zero.
## Static lists
- `SWWAF_ALLOW_NETS` (default empty): netblocks you trust, such as your
monitoring. Their clients are refused only for a size or time limit or a
missing or wrong token.
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
exempt from rate limiting and the byte limits.
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
## Request rate limits, per client
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
(default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
requests a client may make in each window. A count or `off`.
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
rate limiting and the byte limits, such as `/assets/`.
## Byte limits, per client
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most
bytes counted for a client in each window. A size or `off`.
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
`response`, `request` or `both`.
## Size and time limits
- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to
send its request line and headers, and then, from the end of the headers, its
body. A duration or `off`.
- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request
line and headers. A size over `4K`; never `off`.
- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection
may wait for its next request before it is closed. A duration or `off`.
- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the answer may take
to reach the client, from the end of the request to the last byte. A duration
or `off`.
- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the
app and sending it the whole request may take. A duration or `off`.
- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take
to send its whole answer. A duration or `off`.
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size
or `off`.
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size
or `off`.
## Lookup of AS number and country
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`.
- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
`.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`.
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
client's first answer from GeoJS, and how long a request to GeoJS may take. A
duration; never `off`.
- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the
client's AS number in `X-Client-ASN` and its country in `X-Client-Country`.
## Country lists
- `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused,
such as `cn,ru,kp`.
- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
countries whose clients get through, such as `us,de`. A country on both lists
stops the start.
## Biased thresholds
- `SWWAF_ASN_LIMIT_PERCENT` (default empty): a list of AS numbers, each with `:`
and a percentage, such as `AS14061:50,AS16276:50,AS45102:25`. An AS number
listed twice is refused.
- `SWWAF_COUNTRY_LIMIT_PERCENT` (default empty): the same by country, such as
`cn:25,ru:50`.
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` (default empty):
the same, for the byte limits alone.
- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage for a client the
lookup cannot place.
- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
text file with one AS number and percentage on each line, such as
`AS64496:50`. A URL that is also in `SWWAF_BLOCKLIST_URLS` is refused.
## Attack detection
- `SWWAF_RULES_DIR` (default `/etc/smallwebwaf/rules.d`): the directory of the
rule files.
- `SWWAF_RULES_ENABLED` (default `true`): `false` reads no rule file.
- `SWWAF_TRAP_PATHS` (default empty): paths only scanners ask for, such as
`/wp-login.php,/xmlrpc.php`. Each starts with `/` and holds no `?`.
- `SWWAF_WAF_MODE` (default `block`): `off`, `detect` or `block`.
- `SWWAF_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level,
from 1 to 4.
- `SWWAF_WAF_ANOMALY_THRESHOLD` (default `5`): the score at which a request is a
match. A whole number above zero, or `off`.
- `SWWAF_WAF_DISABLED_RULES` (default
`920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules
to switch off. Each a whole number above zero. An id from 900000 to 900999 is
refused: those are the rules that set the Core Rule Set up, and
`smallwebwaf`'s own.
- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does
not inspect.
- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off`, or how much of a request body
the Core Rule Set reads, a size of at most `1G`.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
in a minute for a rule match, a trap path, a Core Rule Set match or a failed
token check. A count or `off`.
## Ban lengths, response and scope
- `SWWAF_BAN_RESPONSE` (default `403`): `403`, `429` or `close`.
- `SWWAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken limit. A
duration.
- `SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): the time after a ban ends
within which breaking a limit again lengthens the next ban. A duration.
- `SWWAF_MAX_BAN_DURATION` (default `7d`): the longest ban for a broken limit;
one that would be longer is permanent. A duration.
- `SWWAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
attack. A duration.
- `SWWAF_BAN_SCOPE_V4_PREFIX` (default `32`): how much a ban on an IPv4 client
covers: the prefix length of its netblock, from 0 to 32.
## Reputation
- `SWWAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as
`https://www.spamhaus.org/drop/drop.txt`. Each an `http` or `https` URL
without a user or a fragment. A URL listed twice is refused.
- `SWWAF_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its
last try before it is fetched again. A duration of `1h` or more.
- `SWWAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:<percent>` or `log`.
- `SWWAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask, such as
`dnsbl.dronebl.org`. Each a DNS name of at most 189 characters. A Spamhaus
zone carries its key, as in `<key>.xbl.dq.spamhaus.net`. A zone listed twice
is refused, whatever its case or key.
- `SWWAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through,
such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional
port, 53 when none is given. Unset, the host's resolver is used.
- `SWWAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account. A
control character in it is refused.
- `SWWAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score,
from 0 to 100, that is a hit.
- `SWWAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day,
a whole number above zero. AbuseIPDB's free accounts may make 1,000.
- `SWWAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine
you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a
user or a fragment; its decision list is at `v1/decisions` under it. Set
without the key, it is refused. So is a decision list URL that is also in
`SWWAF_BLOCKLIST_URLS`, or that is `SWWAF_ASN_LIMIT_PERCENT_URL`.
- `SWWAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
`smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
URL, it is refused.
- `SWWAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:<percent>` or
`log`, for DNS blocklist and AbuseIPDB hits.
- `SWWAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or
AbuseIPDB's score is used. A duration.
- `SWWAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a
check with AbuseIPDB may take. A duration.
## Alerting
- `SWWAF_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each
alert as a JSON object. An `http` or `https` URL without a user or a fragment.
- `SWWAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post,
such as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and
value. A value cannot hold a comma.
- `SWWAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook
that receives each alert as a message, checked as the webhook URL is.
- `SWWAF_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each
alert, as the topic's full URL, such as `https://ntfy.sh/my-alerts`, checked
as the webhook URL is.
- `SWWAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token. A control
character in it is refused.
- `SWWAF_ALERT_EVENTS` (default
`ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`):
the event types alerts are sent for, from those in [`alerts.md`](alerts.md)
but `summary`. Any other name is refused.
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long after an alert is sent its
repeats are held back. A duration or `off`.
- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. A
whole number above zero, or `off`.
## Anomaly thresholds
Each scope has four thresholds: the most requests and the most bytes counted in
a minute and in an hour. Requests are counts, bytes are sizes. Each is `off` by
default. The four thresholds per client are:
- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
- `SWWAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
- `SWWAF_ANOMALY_CLIENT_BYTES_PER_MINUTE`;
- `SWWAF_ANOMALY_CLIENT_BYTES_PER_HOUR`.
The other scopes' names have another prefix before the same four endings, such
as `REQUESTS_PER_MINUTE`:
- `SWWAF_ANOMALY_NET_...`: per netblock around a client. Two settings give the
netblock's length:
- `SWWAF_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) for an IPv4
client;
- `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
client.
- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs a lookup source.
- `SWWAF_ANOMALY_TOTAL_...`: the whole service.
- `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole.
`SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
name, or with a bad netblock, is refused. So is a name used twice.
## Settings given as files
To give a token as a file, as "Tokens" under "Deployment" in
[`SPEC.md`](../SPEC.md) describes, make the file on the host:
```sh
mkdir -p /srv/app/tokens
openssl rand -hex 32 > /srv/app/tokens/metrics
chown 65532:65532 /srv/app/tokens/metrics
chmod 0400 /srv/app/tokens/metrics
```
Then mount its directory into the container read-only, and name the file:
```sh
docker run \
-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro \
-e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics \
...
```
## Fixed limits
A few limits are fixed rather than settings:
- at most 100,000 GeoJS answers are kept;
- at most 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept;
- at most 1,000 DNS blocklist queries are under way at once;
- at most 20,000 anomaly counters are kept;
- each alert destination's queue holds at most 1,000 alerts.
docs/state-files.md
+104
View File
@@ -0,0 +1,104 @@
# State files
The five JSON files in `SWWAF_STATE_DIR`, and what each holds. "Persistent
state" in [`SPEC.md`](../SPEC.md) says how they are written, read and edited.
Each has a top-level `version` of 1.
## `bans.json`
Active and past bans: every ban an admin made, and up to `SWWAF_MAX_BANS` of
those `smallwebwaf` made. Per entry:
- `netblock`, `start` and `expires` (`null` for a permanent ban);
- `cause`: `attack`, `limit`, `admin` or `crowdsec`;
- `reason`: a short reason;
- `lifted`: when an admin lifted it, if one did;
- `notes`: the notes below.
An entry an admin adds needs `netblock`, `start` and `expires`; `reason`,
`notes` and `cause` may be left out, and the cause is then `admin`:
```json
{
"version": 1,
"bans": [
{
"netblock": "203.0.113.0/24",
"start": "2026-10-06T12:00:00Z",
"expires": null,
"reason": "probes for logins"
}
]
}
```
"Bans" in [`SPEC.md`](../SPEC.md) says how an edit lifts, keeps or forgets a
ban. The notes:
- `asn`, `as_name` and `country`: the client's, added when the lookup answers
(empty when lookups are off);
- for a broken limit, `kind` (`requests`, `bytes` or `refusals`), `limit`,
`window` and `count`: what the limit was on, the limit, its window and the
count reached;
- `limit_percent` and `limit_percent_setting`: the client's percentage of that
limit and the setting that gave it, when a setting lowered it;
- for a clear sign of attack, `rule_id` and `target`, or `trap_path`;
- `reputation`: the sources the client was on, each with its `source` and, for
AbuseIPDB, its `score`;
- `request`: the request that caused the ban, each text cut to 256 bytes:
- `time`, `method` and `host`;
- `path`, with its query string;
- `status` and `user_agent`;
- `requests` and `refused`: the netblock's requests since it was first seen, and
how many this ban has refused so far, kept up to date while the ban lasts;
- `earlier_bans`: how many earlier bans the netblock had, by cause.
## `clients.json`
Per client, one per line: its minute, hour and day counters, and its history
since it was first seen:
- the first and last time seen;
- the AS number, AS name and country last looked up, and when;
- total requests, and how many were forwarded and how many refused;
- bytes in each direction;
- responses by status class;
- offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and
`token_refused`.
## `lookups.json`
GeoJS answers, one per line:
- `client`;
- `asn`, `as_name` and `country`;
- `answered`: when GeoJS answered;
- `used`: when the answer was last used.
## `reputation.json`
- the last good copy of each list fetched from a URL, with when it was last
tried;
- each DNS blocklist verdict still in use;
- AbuseIPDB's scores, and the checks spent today.
## `alerts.json`
- the anomaly counters;
- the alert cooldowns: for each event and netblock, source, file or anomaly
scope, when it was last sent and the count of repeats held back since;
- the alerts sent and held back in the current hour;
- the alerts waiting for each destination.
## Sizes
- `clients.json`: about 1 KiB per client, about 20 MiB at the default
`SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of
disk writes a day.
- `bans.json`: about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at
the default `SWWAF_MAX_BANS`. Written every 15 minutes, a full file comes to
about 600 MiB of writes a day, and a hundred new bans a day add about 600 MiB
more.
- `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
full.
- `reputation.json` and `alerts.json`: usually a few MiB or less.
script/example-app
+1 -1
View File
@@ -87,7 +87,7 @@ start_container() {
# start_moved_container: run the app's container with the state files on
# a new volume at /srv/state, which SWWAF_STATE_DIR_FILE names in a file
# only the smallwebwaf user can read, as under "Settings given as files"
# in README.md, and wait until it is healthy. smallwebwaf can write the
# in docs/settings.md, and wait until it is healthy. smallwebwaf can write the
# new volume only once the run script has given it to that user.
start_moved_container() {
docker volume create "$MOVED_VOLUME" >/dev/null