sneak/smallwebwaf
1
0
Fork 0
Code Issues 10 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

rproxy and servewaf replace the one-process smallwebwaf (closes #172, closes #165)
check / check (push) Failing after 14m44s
Details

Browse Source 
`smallwebwaf` with no command now prints the commands and exits with status 2.
The image runs rproxy, or servewaf with SMALLWEBWAF_COMMAND=servewaf. The
passing-on code, its settings and its log fields and metric go. internal/proxy
keeps the checks, answers, reports and own endpoints; its tests run through the
real rproxy. The documents describe the pair.

Judgement call: clients.json keeps token_refused, since state files refuse unknown fields.
Deviation: script/gitea-requests sets SWWAF_LOG_REQUEST_HEADERS, to label requests.
Partially verified: make example-app and make gitea-requests never built the image, as Ubuntu's snapshot service answered 5xx.

Model: opus-5-5
This commit was merged in pull request #184.
This commit is contained in:
clawbot
2026-11-05 13:15:05 +01:00
parent 17c5dd2438
commit 344ce13c96
71 changed files with 2653 additions and 5493 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin-endpoints.md
+24 -23
View File
@@ -1,8 +1,8 @@
# Admin endpoints
The endpoints under `/_smallwebwaf/`, what each takes and what each answers.
"Admin endpoints" in [`SPEC.md`](../SPEC.md) says which checks they go through
and how their tokens work.
`smallwebwaf servewaf` answers them at its own address, on the private network.
"Admin endpoints" in [`SPEC.md`](../SPEC.md) says how their tokens work.
- `GET /_smallwebwaf/healthz`: `200` and `ok`, to anyone.
- `GET /_smallwebwaf/metrics`: the metrics, for `SWWAF_METRICS_TOKEN` (see
@@ -20,40 +20,41 @@ and how their tokens work.
- one that names an IPv4-mapped netblock, or an address with a zone.
- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban covering the
address `<client>`. It answers `404` when none is active.
- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client.
Under `client`, its counters and history as `clients.json` holds them, or
`null` when `clients.json` does not hold it. Under `bans`, every ban covering
the address, with its notes.
- `GET /_smallwebwaf/clients/<ip>`: what `servewaf` knows of the client. Under
`client`, its counters and history as `clients.json` holds them, or `null`
when `clients.json` does not hold it. Under `bans`, every ban covering the
address, with its notes.
- `POST /_smallwebwaf/report`: takes `rproxy`'s report, without a token. It
answers `204`, or `400` to a body that is no report (see "The report" in
[`SPEC.md`](../SPEC.md)).
No check runs on these requests. A missing or wrong token is answered `401`. An
endpoint whose token is unset answers `404`, as does any other path.
`smallwebwaf rproxy` answers only the health check of these, at its own address.
It asks `servewaf` about any other request under `/_smallwebwaf/`. One that
passes the checks is answered `404`, so that the app never sees that prefix (see
"`rproxy`" in [`SPEC.md`](../SPEC.md)).
## Examples
With `SWWAF_ADMIN_TOKEN` set, and the token in `$TOKEN`:
With `SWWAF_ADMIN_TOKEN` set, the token in `$TOKEN`, and `servewaf` reached as
`servewaf:8080` on its network:
```sh
# Every ban.
curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans
curl -H "Authorization: Bearer $TOKEN" http://servewaf:8080/_smallwebwaf/bans
# Ban 203.0.113.0/24 for seven days.
curl -H "Authorization: Bearer $TOKEN" \
--json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \
https://app.example/_smallwebwaf/bans
http://servewaf:8080/_smallwebwaf/bans
# Lift the bans on 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" -X DELETE \
https://app.example/_smallwebwaf/bans/203.0.113.9
http://servewaf:8080/_smallwebwaf/bans/203.0.113.9
# What smallwebwaf knows of 203.0.113.9.
# What servewaf knows of 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" \
https://app.example/_smallwebwaf/clients/203.0.113.9
http://servewaf:8080/_smallwebwaf/clients/203.0.113.9
```
## `servewaf`
`smallwebwaf servewaf` answers these endpoints at its own address. No check runs
on those requests (see "`servewaf`" in [`SPEC.md`](../SPEC.md)).
## `rproxy`
`smallwebwaf rproxy` answers only the health check, at its own address. It asks
`servewaf` about any other request under `/_smallwebwaf/` (see "`rproxy`" in
[`SPEC.md`](../SPEC.md)).
docs/alerts.md
+7 -6
View File
@@ -6,7 +6,7 @@ back or dropped.
## Events
- `ban`: a ban `smallwebwaf` made.
- `ban`: a ban `servewaf` made.
- `permanent_ban`: a permanent ban it made, or an existing ban made permanent by
a request.
- `waf_block`: the Core Rule Set scored a request at or over the threshold. The
@@ -34,9 +34,10 @@ on one line:
```json
{
"instance": "fsn1app1/gitea",
"instance": "fsn1/servewaf",
"time": "2026-10-06T12:00:50.123461Z",
"event": "ban",
"host": "app.example",
"client": "203.0.113.9",
"netblock": "203.0.113.9/32",
"asn": "",
@@ -78,9 +79,9 @@ on one line:
- `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
one of the events above.
- `host`: the site of the request that raised the alert. Only `servewaf` sends
this field, since it serves many sites, and not for `source_failure`,
`file_error` and `summary`.
- `host`: the site of the request that raised the alert, since one `servewaf`
serves many sites. It is left out of `source_failure`, `file_error` and
`summary`.
- `client`: the address whose request raised the alert. Empty for
`source_failure`, `file_error` and `summary`.
- `netblock`: empty for `source_failure`, `file_error` and `summary`; otherwise
@@ -110,7 +111,7 @@ on one line:
## Slack and ntfy
Slack and ntfy get a message with a title and a text. The title is the instance
and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then one
and the event, such as `fsn1/servewaf: ban`. The text is the `reason`, then one
line for each of these that the alert has:
- the host, the client, the netblock and the country;
docs/gitea.md
+9 -9
View File
@@ -2,10 +2,11 @@
These notes record what `smallwebwaf` does in front of gitea at the default
settings. They are observations, not requirements; the requirements are in
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind the image and
sends through it a sample of gitea's ordinary use and the requests named below.
It reports each request the Core Rule Set refuses, so that these notes can be
checked again when gitea or the Core Rule Set changes version.
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind `rproxy` and
`servewaf`, and sends through them a sample of gitea's ordinary use and the
requests named below. It reports each request the Core Rule Set refuses, so that
these notes can be checked again when gitea or the Core Rule Set changes
version.
## What the defaults cover
@@ -13,7 +14,7 @@ checked again when gitea or the Core Rule Set changes version.
by it.
- A clone is a response. It fits the defaults of 30 minutes and 1 GiB unless the
repository is large or the link slow. A repository whose clone is larger than
1 GiB needs `SWWAF_RESPONSE_MAX_BYTES` raised, and the byte limits with it
1 GiB needs `SWRP_MAX_RESPONSE_SIZE` raised, and the byte limits with it
("Byte limits, per client" in `SPEC.md`).
- A push is a request, and so is every other upload:
- a git push;
@@ -23,10 +24,9 @@ checked again when gitea or the Core Rule Set changes version.
- a release attachment.
At the defaults, an upload larger than 100 MiB or taking more than 60
seconds is cut off ("Size and time limits" in `SPEC.md`). A gitea that takes
large uploads needs `SWWAF_REQUEST_MAX_BYTES`,
`SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised
to fit.
seconds is cut off ("`rproxy`'s limits" in `SPEC.md`). A gitea that takes
large uploads needs `SWRP_MAX_REQUEST_SIZE` and `SWRP_MAX_REQUEST_TIME_MS`
raised to fit.
- Archive download and blame or history pages are what scrapers hammer. The
request limits do most of the work there.
docs/image.md
+5 -3
View File
@@ -1,6 +1,7 @@
# What the image holds
The contents of the `smallwebwaf` image, and how each part is pinned.
The contents of the `smallwebwaf` image, and how each part is pinned. The image
runs `rproxy` by default, or `servewaf` with `SMALLWEBWAF_COMMAND=servewaf`.
"Deployment" in [`SPEC.md`](../SPEC.md) says why each part is there and why it
is pinned that way.
@@ -18,8 +19,9 @@ is pinned that way.
- The `smallwebwaf` binary, and the user `smallwebwaf` (uid and gid 65532).
- The service directory `/etc/service/smallwebwaf`, with the `run` script
`share/smallwebwaf.run`.
- `/var/lib/smallwebwaf` for the state files, and `/etc/smallwebwaf/rules.d`
with the default rule file `share/rules.d/00-default.rules`.
- `/var/lib/smallwebwaf` for `servewaf`'s state files, and
`/etc/smallwebwaf/rules.d` with the default rule file
`share/rules.d/00-default.rules`.
- `EXPOSE 8080`, and the `HEALTHCHECK` that runs `smallwebwaf healthcheck`.
## Pins
docs/metrics.md
+10 -10
View File
@@ -1,8 +1,9 @@
# The metrics
What `GET /_smallwebwaf/metrics` answers with. "Metrics endpoint" in
[`SPEC.md`](../SPEC.md) describes the endpoint. Prometheus keeps the `instance`
label as `exported_instance` unless the scrape sets `honor_labels: true`.
What `GET /_smallwebwaf/metrics` answers with, at `servewaf`'s own address.
"Metrics endpoint" in [`SPEC.md`](../SPEC.md) describes the endpoint. Prometheus
keeps the `instance` label as `exported_instance` unless the scrape sets
`honor_labels: true`. `rproxy` has no metrics.
## Traffic
@@ -11,14 +12,13 @@ label as `exported_instance` unless the scrape sets `honor_labels: true`.
`action`;
- `smallwebwaf_request_duration_seconds` and
`smallwebwaf_upstream_duration_seconds`, as histograms;
- `smallwebwaf_requests_in_flight`.
- `smallwebwaf_requests_in_flight`: the questions and requests to its own
endpoints that `servewaf` is handling.
## Limits and bans
- `smallwebwaf_rate_limit_hits_total`, by `window` and by `kind` (`requests`,
`bytes` or `refusals`);
- `smallwebwaf_size_and_time_limit_hits_total`, by `limit`, the setting passed.
`servewaf`'s header limits have no setting, so it leaves them out;
- `smallwebwaf_offences_total`, by `kind`;
- `smallwebwaf_bans_made_total`, by `cause`;
- `smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
@@ -81,11 +81,11 @@ By `file`:
`smallwebwaf_alerts_dropped_total`;
- Go's own `go_` metrics and the process's `process_` metrics.
## `servewaf`
## When a request is counted
`smallwebwaf servewaf` counts a denied request at its decision. It counts an
allowed request when its report comes, with the status, bytes and times the
report gives. These are not counted:
`servewaf` counts a denied request at its decision, and a request to its own
endpoints when it is answered. It counts an allowed request when its report
comes, with the status, bytes and times the report gives. These are not counted:
- a report itself;
- an allowed request whose report never comes;
docs/request-log.md
+137 -151
View File
@@ -1,124 +1,9 @@
# The request log's fields
The fields of the JSON object `smallwebwaf` writes for each request. "Request
log" in [`SPEC.md`](../SPEC.md) says when a line is written and how the log is
sent to a syslog server. A field that does not apply to a request is left out.
`smallwebwaf rproxy` writes a shorter line, which "`rproxy`'s line" below gives.
## Example
```
{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
```
## Every line
Every line but `rproxy`'s has:
- `type`: `request` (the process's own messages have `process`);
- the standard web log fields;
- `request_id`, `peer_ip` and `client_group`;
- `asn`, `as_name` and `country`;
- `action` and `duration_total`.
## Standard web log fields
- `time`: when the request arrived, RFC 3339 with milliseconds, in UTC;
- `instance`: `SWWAF_INSTANCE_NAME`;
- `client_ip`, `method`, `scheme`, `host`, `path`, `query`, `protocol`;
- `status`: what the client was sent, `0` if nothing;
- `request_bytes` and `response_bytes`: body bytes each way;
- `referer` and `user_agent`.
## Request detail
- `request_id`: the `X-Request-ID` a trusted proxy sent, or a new random one of
26 letters and digits.
- `peer_ip`: the TCP peer, normally traefik.
- `forwarded_for`: the `X-Forwarded-For` header as received.
- `client_group`: the client as rate limiting counts it, an IPv4 address as a
/32 or an IPv6 group.
- `asn`, `as_name` and `country`: the client's, as looked up. Each is empty
when:
- lookups are off;
- the client is not looked up;
- the source cannot place the client;
- GeoJS had not answered by the time the request went on.
- `content_type` and `content_length`: the request's own.
- `request_headers`: the headers listed in `SWWAF_LOG_REQUEST_HEADERS`, by name
in lower case.
- `has_authorization` and `has_cookie`: true when the request had those headers.
Their values are never logged.
- `websocket`: true when the app switched the connection to another protocol.
## Response detail
- `upstream_status`: what the app answered. It differs from `status` when
`smallwebwaf` answered itself.
- `response_content_type`, `cache_control` and `location`: the answer's headers.
- `aborted`: true when the client went away early.
## Decision
- `action`: what was done:
- `forward`: passed to the app;
- `denied`: refused for `SWWAF_DENY_NETS`, or for a reputation source with
the action `deny`;
- `banned`: refused under a ban, or for a `ban` rule, a trap path or a
CrowdSec decision, each of which bans the client;
- `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`:
refused by that check;
- `too_large`: a size limit was passed, or a header limit in `servewaf`;
- `timed_out`: a time limit was passed;
- `upstream_error`: the app could not be reached, or its answer broke off;
- `admin`: answered at one of `smallwebwaf`'s own endpoints.
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
request it would have refused. `action` then names what was done, normally
`forward`.
- `limit_percent` and `limit_percent_setting`: the client's percentage of the
rate limits and the setting that lowered it, when one did. `bytes_percent` and
`bytes_percent_setting` do the same for the byte limits.
- `counts`: the client's totals as rate limiting counts them, this request
included. It is left out when rate limiting did not count the request. It
holds:
- `minute`, `hour` and `day`: requests;
- `minute_bytes`, `hour_bytes` and `day_bytes`: bytes.
A count can have a fraction (see "Counting method" in
[`SPEC.md`](../SPEC.md)).
- `limit_hit`: the limit broken:
- `minute`, `hour` or `day` for a rate limit;
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
- `error_burst`.
A request whose bytes broke a limit was passed on whole, so its `action`
stays `forward`.
- `rule_ids`: the rule file rules that matched, in order, up to the one that
refused.
- `waf_score` and `waf_rule_ids`: the Core Rule Set's score and the rules that
matched, for a request it inspected.
- `reputation`: the sources the client is on, in this order:
- blocklist URLs;
- the CrowdSec decision list's URL;
- DNS blocklist zones;
- `abuseipdb`.
- `offence`: `limit`, for a request that broke a limit.
- `ban_expires`: when the client's ban ends, for a request that made one or was
refused under one: a time, or `permanent`.
## Timings
All in milliseconds:
- `duration_total`: from the request's headers being read to its log line;
- `duration_checks`: to the end of the checks, before forwarding;
- `duration_waf`: the part of that the Core Rule Set took;
- `duration_upstream_connect`: from the hand-over to the app until there was a
connection;
- `duration_upstream_first_byte`: until the first byte of the answer;
- `duration_upstream_total`: until the end of the answer.
The fields of the JSON objects `rproxy` and `servewaf` write about each request.
"Request log" and "The log" in [`SPEC.md`](../SPEC.md) say when a line is
written and how the log is sent to a syslog server. A field that does not apply
to a request is left out.
## `servewaf`'s lines
@@ -138,50 +23,151 @@ These questions write no line:
[`SPEC.md`](../SPEC.md));
- a question whose body stopped arriving, which gets no answer.
### The `decision` line
A `report` line:
It has the fields above, with these meanings where they differ:
```
{"type":"request","servewaf_line":"report","time":"2026-10-03T12:00:00.123Z","instance":"fsn1/servewaf","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"waf_score":0,"duration_total":3.217,"duration_checks":0.041,"duration_waf":0.029,"duration_upstream_total":3.104}
```
- `client_ip`: `X-Smallwebwaf-Client`.
- `peer_ip`: `X-Smallwebwaf-Peer`.
### Every line
Every one of these lines has:
- `type`: `request` (the process's own messages have `process`);
- the standard web log fields;
- `request_id`, `peer_ip` and `client_group`;
- `asn`, `as_name` and `country`;
- `action` and `duration_total`.
### Standard web log fields
- `time`: when `servewaf` had read the question's body, RFC 3339 with
milliseconds, in UTC. The `report` line gives the same time as the `decision`
line.
- `instance`: `SWWAF_INSTANCE_NAME`.
- `client_ip`: the visitor's address, as `rproxy` worked it out.
- `method`, `path` and `query`: the visitor's request.
- `scheme`: how the visitor reached `rproxy` or the trusted proxy in front of
it.
- `host`: the site the visitor asked for.
- `protocol`: the question's own, `HTTP/1.1`.
- `content_length`: `X-Smallwebwaf-Content-Length`.
- `request_id`: the question's `X-Request-ID`.
- `status`: what `servewaf` answered: `200` for `allow`, the status of `deny`,
or `0` for `close`, as the visitor is then sent nothing.
- `request_bytes`: how much of the question's body the Core Rule Set read.
- `response_bytes`: the body of `servewaf`'s answer.
- `action`: `forward` for `allow`, and `admin` for the `404` for a path under
`/_smallwebwaf/`.
- `status`:
- on the `decision` line, what `servewaf` answered: `200` for `allow`, the
status of `deny`, or `0` for `close`, as the visitor is then sent nothing;
- on the `report` line, what the visitor got, `0` if nothing.
- `request_bytes`:
- on the `decision` line, how much of the question's body the Core Rule Set
read;
- on the `report` line, the bytes of the visitor's body that `rproxy` read,
with what a WebSocket carried from the visitor.
- `response_bytes`:
- on the `decision` line, the body of `servewaf`'s answer;
- on the `report` line, the bytes of the answer's body, with what a
WebSocket carried from the app.
- `referer` and `user_agent`.
### Request detail
- `request_id`: the id `rproxy` made for the request when it arrived.
- `peer_ip`: the address of `rproxy`'s TCP peer, normally traefik.
- `forwarded_for`: the `X-Forwarded-For` header the visitor's request came with.
- `client_group`: the client as rate limiting counts it, an IPv4 address as a
/32 or an IPv6 group.
- `asn`, `as_name` and `country`: the client's, as looked up. Each is empty
when:
- lookups are off;
- the client is not looked up;
- the source cannot place the client;
- GeoJS had not answered by the time `servewaf` answered.
- `content_type`: the visitor's `Content-Type`.
- `content_length`: the body length the visitor announced.
- `request_headers`: the headers listed in `SWWAF_LOG_REQUEST_HEADERS`, by name
in lower case.
- `has_authorization` and `has_cookie`: true when the request had those headers.
Their values are never logged.
### Response detail
- `response_content_type`: the type of `servewaf`'s own answer.
- `aborted`: true when `rproxy` went away before the answer.
- `duration_total`: from `servewaf` having read the question's body to the line.
- `time`: when `servewaf` had read the question's body.
The line leaves out `upstream_status`, `websocket` and the
`duration_upstream_...` fields, since `servewaf` passes nothing to the app.
### Decision
The bytes of an allowed request are counted when its report comes. So `counts`
holds the client's bytes without this request's.
- `action`: what was done:
- `forward`: answered `allow`, so that `rproxy` passes it to the app;
- `denied`: refused for `SWWAF_DENY_NETS`, or for a reputation source with
the action `deny`;
- `banned`: refused under a ban, or for a `ban` rule, a trap path or a
CrowdSec decision, each of which bans the client;
- `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`:
refused by that check;
- `too_large`: over a header limit;
- `admin`: answered at one of `servewaf`'s own endpoints, or `404` for a
path under `/_smallwebwaf/`.
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
request it would have refused. `action` then names what was done, normally
`forward`.
- `limit_percent` and `limit_percent_setting`: the client's percentage of the
rate limits and the setting that lowered it, when one did. `bytes_percent` and
`bytes_percent_setting` do the same for the byte limits.
- `counts`: the client's totals as rate limiting counts them, this request
included. It is left out when rate limiting did not count the request. It
holds:
- `minute`, `hour` and `day`: requests;
- `minute_bytes`, `hour_bytes` and `day_bytes`: bytes.
### The `report` line
A count can have a fraction (see "Counting method" in
[`SPEC.md`](../SPEC.md)). The bytes of an allowed request are counted when
its report comes. So on the `decision` line, the bytes are without this
request's.
It has every field of the `decision` line, and `time` stays as the `decision`
line gives it. These fields say what the report says:
- `limit_hit`: the limit broken:
- `minute`, `hour` or `day` for a rate limit;
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
- `error_burst`.
- `status`: what the visitor got, `0` if nothing;
- `request_bytes` and `response_bytes`: the body bytes each way, with what a
WebSocket carried each way;
- `duration_total`: from the request's headers being read by `rproxy` to the end
of the answer;
- `duration_upstream_total`: from the hand-over to the app to the end of the
answer, for an allowed request.
A request whose bytes broke a limit was passed on whole, so its `action`
stays `forward`. Only its `report` line gives that limit.
For an allowed request, the report's counting also changes these fields:
- `rule_ids`: the rule file rules that matched, in order, up to the one that
refused.
- `waf_score` and `waf_rule_ids`: the Core Rule Set's score and the rules that
matched, for a request it inspected.
- `reputation`: the sources the client is on, in this order:
- blocklist URLs;
- the CrowdSec decision list's URL;
- DNS blocklist zones;
- `abuseipdb`.
- `offence`: `limit`, for a request that broke a limit.
- `ban_expires`: when the client's ban ends, for a request that made one or was
refused under one: a time, or `permanent`.
- `counts`: the byte totals now include this request's;
- `limit_hit`, `offence` and `ban_expires`: set when the report's bytes broke a
byte limit.
### Timings
All in milliseconds:
- `duration_total`:
- on the `decision` line, from `servewaf` having read the question's body to
the line;
- on the `report` line, from `rproxy` having read the request's headers to
the end of the answer.
- `duration_checks`: to the end of the checks;
- `duration_waf`: the part of that the Core Rule Set took;
- `duration_upstream_total`: on the `report` line of an allowed request, from
the hand-over to the app to the end of the answer.
### Requests to its own endpoints
The line of a request to `servewaf`'s own endpoints has these fields, with these
meanings where they differ:
- `client_ip` and `peer_ip`: the request's TCP peer.
- `request_id`: a new random one of 26 letters and digits.
- `scheme`: `http`.
- `time`: when the request's headers had been read.
- `request_bytes`: the bytes read of a ban to add, and otherwise `0`.
- `action`: `admin`.
- `duration_total`: from the request's headers being read to the line.
## `rproxy`'s line
docs/rules.md
+7 -2
View File
@@ -54,14 +54,19 @@ empty-agent user_agent log ^$
## A file of your own
An app's `Dockerfile` adds rules of its own in a file named to sort after the
default, such as `50-gitea.rules`:
`servewaf` reads the rule files in its own container. Rules of your own go in a
file named to sort after the default, such as `50-gitea.rules`:
```
# 50-gitea.rules: WordPress probes, which gitea never serves
wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
```
A `Dockerfile` built on the `smallwebwaf` image, for `servewaf` to run from,
adds it:
```dockerfile
COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules
```
Or mount it there. A `path` rule applies on every site behind that `servewaf`.
docs/settings.md
+51 -58
View File
@@ -4,6 +4,10 @@ Every setting, its default and the values it takes, in the groups of "Settings"
in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings
that take `off`, the `_FILE` form, and what each group does and why.
The settings that start with `SWWAF_` are `servewaf`'s. Those that start with
`SWRP_` are `rproxy`'s, and are under "`rproxy`" below. `SMALLWEBWAF_COMMAND`
chooses between the two (see "The image's command" below).
## Forms of values
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. Zero
@@ -24,26 +28,22 @@ that take `off`, the `_FILE` form, and what each group does and why.
## Core
- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or
`https` URL with a host and an optional port, and nothing more.
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens: `:` and a
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `servewaf` listens: `:` and a
port, such as `:9000`. Leave the host out (see "Ports" in
[`SPEC.md`](../SPEC.md)).
- `SWWAF_INSTANCE_NAME` (default: the container's host name): the `instance`
name in every log line, alert and metric, such as `fsn1app1/gitea`. Valid
UTF-8. No control character while `SWWAF_ALERT_NTFY_URL` is set. 1 to 48
printable ASCII characters without a space while `SWWAF_LOG_REMOTE_URL` is set
and `SWWAF_LOG_REMOTE_APP_NAME` is not.
name in each line `servewaf` logs, and in each alert and metric, such as
`fsn1/servewaf`. Valid UTF-8. No control character while
`SWWAF_ALERT_NTFY_URL` is set. 1 to 48 printable ASCII characters without a
space while `SWWAF_LOG_REMOTE_URL` is set and `SWWAF_LOG_REMOTE_APP_NAME` is
not.
- `SWWAF_MODE` (default `enforce`): `enforce` or `observe`.
- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`):
the netblocks whose `X-Forwarded-For` is believed. Set but empty, it trusts
nothing.
- `SWWAF_IPV6_GROUP_PREFIX` (default `64`): the prefix length of an IPv6
client's group, from 32 to 128.
- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a whole
number above zero.
- `SWWAF_MAX_BANS` (default `5000`): how many of its own bans `smallwebwaf`
keeps, a whole number above zero.
- `SWWAF_MAX_BANS` (default `5000`): how many of its own bans `servewaf` keeps,
a whole number above zero.
## Tokens
@@ -63,7 +63,7 @@ that take `off`, the `_FILE` form, and what each group does and why.
## Logging
- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `smallwebwaf`'s own
- `SWWAF_LOG_LEVEL` (default `info`): the least severe of `servewaf`'s own
messages that are written: `debug`, `info`, `warn` or `error`.
- `SWWAF_LOG_REQUEST_HEADERS` (default
`accept,accept-language,accept-encoding,content-type,origin,range`): the
@@ -125,27 +125,6 @@ that take `off`, the `_FILE` form, and what each group does and why.
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
`response`, `request` or `both`.
## Size and time limits
- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to
send its request line and headers, and then, from the end of the headers, its
body. A duration or `off`.
- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest request
line and headers. A size over `4K`; never `off`.
- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open connection
may wait for its next request before it is closed. A duration or `off`.
- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the answer may take
to reach the client, from the end of the request to the last byte. A duration
or `off`.
- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the
app and sending it the whole request may take. A duration or `off`.
- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take
to send its whole answer. A duration or `off`.
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size
or `off`.
- `SWWAF_RESPONSE_MAX_BYTES` (default `1G`): the largest response body. A size
or `off`.
## Lookup of AS number and country
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off` stops
@@ -160,8 +139,9 @@ that take `off`, the `_FILE` form, and what each group does and why.
- `SWWAF_LOOKUP_TIMEOUT` (default `800ms`): how long a request waits for its
client's first answer from GeoJS, and how long a request to GeoJS may take. A
duration; never `off`.
- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the
client's AS number in `X-Client-ASN` and its country in `X-Client-Country`.
- `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the app
the client's AS number in `X-Client-ASN` and its country in
`X-Client-Country`.
## Country lists
@@ -206,7 +186,9 @@ that take `off`, the `_FILE` form, and what each group does and why.
- `SWWAF_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does
not inspect.
- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off`, or how much of a request body
the Core Rule Set reads, a size of at most `1G`.
the Core Rule Set reads, a size of at most `64K`. A question carries no more
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
@@ -327,6 +309,7 @@ Then mount its directory into the container read-only, and name the file:
```sh
docker run \
-v /srv/app/tokens:/etc/smallwebwaf/tokens:ro \
-e SMALLWEBWAF_COMMAND=servewaf \
-e SWWAF_METRICS_TOKEN_FILE=/etc/smallwebwaf/tokens/metrics \
...
```
@@ -349,22 +332,7 @@ A few limits are fixed rather than settings:
- `rproxy` reads at most 1 MiB of the body of `servewaf`'s answer, and answers
`502` to a longer one.
## `servewaf`
`smallwebwaf servewaf` uses every `SWWAF_` setting but these, which it ignores:
- `SWWAF_UPSTREAM_URL` and `SWWAF_TRUSTED_PROXIES`;
- the five timeouts under "Size and time limits";
- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, `SWWAF_REQUEST_MAX_BYTES` and
`SWWAF_RESPONSE_MAX_BYTES`.
An invalid value of one of these still stops the start, and the line of settings
at start still shows them.
`SWWAF_ADD_LOOKUP_HEADERS` puts the headers on the answer to `rproxy`, not on a
request to an app. `SWWAF_WAF_BODY_LIMIT` has an effect up to 64 KiB less one
byte, whatever its value. A question carries no more of a body than 64 KiB (see
"`servewaf`" in [`SPEC.md`](../SPEC.md)).
## `servewaf`'s answer and `rproxy`'s wait
`SWWAF_LOOKUP_TIMEOUT` is the longest a question waits on an outside source.
Keep it well under `rproxy`'s `SWRP_TIMEOUT_MS`, as the default is. No question
@@ -374,10 +342,10 @@ asked in the background (see "Answering in time" under "`servewaf`" in
## `rproxy`
`smallwebwaf rproxy` reads the settings below and no others, and nothing else
reads them. They are read once at start, and an invalid value stops the start.
None can be given as a file, since `rproxy` never touches disk. "`rproxy`" in
[`SPEC.md`](../SPEC.md) says what they do.
`smallwebwaf rproxy` reads the settings below and no others. They are read once
at start, and an invalid value stops the start. None can be given as a file,
since `rproxy` never touches disk. "`rproxy`" in [`SPEC.md`](../SPEC.md) says
what they do.
A time here is a whole number of milliseconds above zero, such as `2000`.
@@ -406,4 +374,29 @@ A time here is a whole number of milliseconds above zero, such as `2000`.
`https` URL with a host and an optional port, and nothing more.
- `SWRP_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`):
the netblocks whose `X-Forwarded-For` is believed. Set but empty, it trusts
nothing.
nothing. A client outside these netblocks cannot claim another address. Where
other containers reach `rproxy` directly, or visitors on a private network
reach traefik, they can name another address in the header. Narrowing the
setting to traefik's own network closes that gap.
## The image's command
`SMALLWEBWAF_COMMAND` (default `rproxy`) chooses what the image runs and what
its health check checks: `rproxy` or `servewaf` (see "Deployment" in
[`SPEC.md`](../SPEC.md)). It cannot be given as a file.
## Where earlier settings went
Nothing reads these names any more:
- `SWWAF_UPSTREAM_URL` is `SWRP_UPSTREAM_URL`.
- `SWWAF_TRUSTED_PROXIES` is `SWRP_TRUSTED_PROXIES`.
- `SWWAF_REQUEST_MAX_BYTES` is `SWRP_MAX_REQUEST_SIZE`.
- `SWWAF_RESPONSE_MAX_BYTES` is `SWRP_MAX_RESPONSE_SIZE`.
- `SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` are
`SWRP_MAX_REQUEST_TIME_MS`, in milliseconds.
- `SWWAF_CLIENT_RESPONSE_TIMEOUT` and `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` are
`SWRP_MAX_RESPONSE_TIME_MS`, in milliseconds.
- `SWWAF_CLIENT_IDLE_TIMEOUT` and `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` have
no setting. `rproxy` closes a kept-open connection idle for 120 seconds, and
takes a request line and headers of at most 32 KiB.
docs/state-files.md
+3 -3
View File
@@ -7,7 +7,7 @@ Each has a top-level `version` of 1.
## `bans.json`
Active and past bans: every ban an admin made, and up to `SWWAF_MAX_BANS` of
those `smallwebwaf` made. Per entry:
those `servewaf` made. Per entry:
- `netblock`, `start` and `expires` (`null` for a permanent ban);
- `cause`: `attack`, `limit`, `admin` or `crowdsec`;
@@ -63,8 +63,8 @@ since it was first seen:
- total requests, and how many were forwarded and how many refused;
- bytes in each direction;
- responses by status class;
- offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and
`token_refused`.
- offences by kind: `limit`, `attack`, `rule_blocked` and `waf_blocked`. A file
may also hold `token_refused`, which is kept, and which nothing adds to.
## `lookups.json`