1 Commits
Author SHA1 Message Date
sneak 91438f54da README rewritten from scratch so a newcomer can follow it (closes #154, closes #155)
check / check (push) Canceled after 0s
The old README opened with project history and one list of a hundred
settings. The new one leads with what smallwebwaf is, what it protects
against, how to build and deploy it, what happens to a request, the
settings most deployments touch, how to check it works, the entrypoints,
the rationale and the design, as REPO_POLICIES.md requires. The reference
lists, every setting, the rule file format, the endpoints, the request
log's fields, the alert events, the metrics, the state files, the image
and the gitea notes, move to files under docs/, each linked from the
README, with SPEC.md as the one place for behaviour. Every statement was
checked against the code on next.

Model: fable-5-1
2026-10-22 21:56:16 +00:00
22 changed files with 1915 additions and 1366 deletions

No files matched your search

+142 -76
View File
@@ -8,7 +8,8 @@ scanners and other abusers before they reach the app.
A web application firewall reads each request on its way to a web app and
refuses the ones that look like abuse. `smallwebwaf` is also a reverse proxy: it
accepts requests on the app's behalf, passes them to the app, and passes the
app's answers back.
app's answers back. A request it does not refuse reaches the app as it came,
apart from a few headers (see "Design" below).
It needs no setting. Every setting has a default chosen for a service on the
open internet. [`SPEC.md`](SPEC.md) says how `smallwebwaf` behaves and why. The
@@ -17,10 +18,10 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
## What it protects against
- Scrapers that fetch every page of a site, all day. Rate limiting gives each
client 6000 requests a minute, 100,000 an hour and 1,000,000 a day. Byte
limits give it 4 GiB a minute, 20 GiB an hour and 200 GiB a day. A client over
a limit is refused and banned. The first ban lasts an hour. Each repeat within
a day makes the next ban three times as long. A ban that would last more than
client 1000 requests a minute, 10,000 an hour and 50,000 a day. Byte limits
give it 10 GiB a minute, 20 GiB an hour and 50 GiB a day. A client over a
limit is refused and banned. The first ban lasts an hour. Each repeat within a
day makes the next ban three times as long. A ban that would last more than
seven days is permanent.
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
default rule file bans such a client for seven days at its first probe. Its
@@ -29,9 +30,9 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
of your own, or an entry in `SWWAF_TRAP_PATHS`, bans those the same way.
- Attacks carried in a request's URL or headers, such as SQL injection and path
traversal. The OWASP Core Rule Set is a published set of rules that recognise
common web attacks. Coraza, a Go engine, runs those rules (see "Attack
detection" in [`SPEC.md`](SPEC.md)). A request the rules take for an attack is
refused.
common web attacks. Coraza, a Go engine that runs those rules, inspects each
request the earlier checks let through. A request the rules take for an attack
is refused.
- Abuse from networks and countries you do not want. You can:
- refuse whole netblocks and countries;
- give the clients of chosen networks and countries a fraction of every
@@ -131,9 +132,8 @@ docker run --detach --name app \
With no setting, the address of every new visitor is sent to GeoJS, a web
service, to find its country and network. `SWWAF_LOOKUP_SOURCE=off` sends
nothing. `SWWAF_LOOKUP_SOURCE=file`, with `SWWAF_LOOKUP_DB_PATH` naming a
database file on your own host, looks addresses up in that file (see "Lookup of
AS number and country" in [`SPEC.md`](SPEC.md)).
nothing. `SWWAF_LOOKUP_SOURCE=file` looks addresses up in a database file on
your own host (see "Lookup of AS number and country" in [`SPEC.md`](SPEC.md)).
`deploy/example-app` is such an app. `make example-app` builds it and checks it
(see "Checking that it works" below). To run `smallwebwaf` outside the image
@@ -141,47 +141,95 @@ while working on it, see "Entrypoints" below.
## What happens to a request
`smallwebwaf` handles each request in fifteen steps, in this order. Once a step
refuses a request, the steps after it are skipped, apart from step 15, and
nothing of the request reaches the app. "What happens to a request" in
[`SPEC.md`](SPEC.md) gives each step in full, under the same numbers.
Two terms recur below. A clear sign of attack is a request that matches a rule
whose action is `ban`, or asks for a trap path (steps 9 and 10). An offence is a
request that counts against its client in its history. It is one of:
1. The health check, `GET /_smallwebwaf/healthz`, is answered at once.
2. The client is identified.
3. The static lists, `SWWAF_ALLOW_NETS` and `SWWAF_DENY_NETS`.
4. The bans.
5. The lookup of the client's AS number and country.
6. The country lists.
7. The reputation sources.
8. The client's limit percentage.
9. The rate limits.
10. The trap paths.
11. The rule files.
12. The Core Rule Set.
13. `smallwebwaf`'s own endpoints, under `/_smallwebwaf/`.
14. The request is passed to the app.
15. After the answer: the byte limits, the error burst and the client's history.
- a clear sign of attack;
- a broken limit;
- a refusal by a rule, by the Core Rule Set or for a missing or wrong token.
A refused visitor is answered with `SWWAF_BAN_RESPONSE`, `403` by default (see
"Ban lengths, response and scope" in [`SPEC.md`](SPEC.md)). A `block` rule or a
Core Rule Set match answers `403`. A missing or wrong token gets `401`. A
request over a size or time limit gets the status for that limit, such as `413`
for a body too large (see "Size and time limits" in [`SPEC.md`](SPEC.md)).
`smallwebwaf` handles each request in this order. The first step that refuses it
ends the handling. Nothing of a refused request reaches the app. "What happens
to a request" in [`SPEC.md`](SPEC.md) gives each step in full.
1. `GET /_smallwebwaf/healthz` is answered `200` with `ok` at once, before any
check and without asking the app.
2. The client is worked out. A netblock is a range of addresses, written as an
address and a prefix length: `203.0.113.0/24`. `SWWAF_TRUSTED_PROXIES` lists
the netblocks traefik comes from, by default the private address ranges. For
a TCP peer inside them, `X-Forwarded-For` is read from the right, and the
first address outside those netblocks is the client. A peer outside them is
the client itself. An IPv6 client is its group: the whole netblock around its
address, a /64 by default; `SWWAF_IPV6_GROUP_PREFIX` sets its length.
3. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 11, the byte limits and the
error burst, but still needs a token at step 12. A client in
`SWWAF_DENY_NETS` is refused.
4. A client under a ban is refused. If the ban was made for a clear sign of
attack, this request makes it permanent.
5. The client's AS number and country are looked up, unless
`SWWAF_LOOKUP_SOURCE` is `off`. An AS number names the network an address
belongs to, such as `AS64496`.
6. The country lists are checked. A client from a country in
`SWWAF_DENIED_COUNTRIES` is refused. When
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any other country
is refused, and so is a client without a country. Such a refusal is not an
offence.
7. The reputation sources are checked: the blocklists, the CrowdSec decision
list, the DNS blocklists and AbuseIPDB. A client on the CrowdSec decision
list is banned until the decision ends. Any other source refuses the client,
lowers its limits or only notes it, as `SWWAF_BLOCKLIST_ACTION` and
`SWWAF_REPUTATION_ACTION` say. No request waits for a query.
8. Rate limiting counts the request, unless the client or the path is exempt. A
request that takes the client over a limit is refused and bans the client.
9. A request for a path in `SWWAF_TRAP_PATHS` is refused and bans the client, as
a clear sign of attack.
10. The rule files are checked. A `log` rule notes the match. A `block` rule
refuses the request. A `ban` rule refuses it and bans the client, as a clear
sign of attack.
11. The Core Rule Set inspects the request's method, URL and headers, and the
body too once `SWWAF_WAF_BODY_LIMIT` is set. A request it takes for an
attack is refused while `SWWAF_WAF_MODE` is `block`, the default, and let
through while it is `detect`. It bans no one by itself.
12. A request under `/_smallwebwaf/` is answered here by `smallwebwaf`, after
its token is checked. Any other request is passed to the app. The app's
answer is streamed back within the size and time limits.
13. Once the answer has ended, the request's bytes are counted for the byte
limits. If a rule, a trap path, the Core Rule Set or a token check refused
it, the refusal is counted for the error burst. The client's history and the
anomaly counters are updated. More than `SWWAF_ERROR_BURST_THRESHOLD`
refusals of one client in a minute ban it.
In `observe` mode (`SWWAF_MODE=observe`), `smallwebwaf` makes no ban of its own.
It refuses a request only for a size or time limit, or for a missing or wrong
token. A request it would otherwise have refused is passed to the app, and its
log line says what `enforce` mode would have done. Observe mode is for trying a
configuration before enforcing it.
What a refused visitor sees:
- `SWWAF_BAN_RESPONSE`, by default `403` with the plain text `Forbidden`. It can
be `429` instead, or `close`, which closes the connection without an answer.
It answers:
- a ban or a broken rate limit;
- a `ban` rule or a trap path;
- `SWWAF_DENY_NETS` and the country lists;
- a reputation source whose action is `deny`.
- `403` for a `block` rule and the Core Rule Set.
- `413` or `408` when the client passes a size or time limit, `502` or `504`
when the app does.
- `401` from an endpoint of `smallwebwaf`'s own, for a request without its
token.
## Settings you are likely to touch
Each setting is an environment variable of the app's container, such as
`docker run -e SWWAF_MODE=observe ...`, read at start.
[`docs/settings.md`](docs/settings.md) lists every setting, its default and the
values it takes. The ones most deployments touch:
- `SWWAF_MODE=observe`: `smallwebwaf` makes no ban of its own, and refuses a
request only for a size or time limit or a missing or wrong token. A request
it would have refused is passed to the app, and its log line says what
`enforce` mode would have done. For trying a configuration before enforcing
it.
- `SWWAF_ALLOW_NETS`: netblocks you trust, such as your monitoring. Their
clients are refused only for a size or time limit or a missing or wrong token.
- `SWWAF_MODE=observe`: log what would have been refused, refuse almost nothing,
ban no one. For trying a configuration.
- `SWWAF_ALLOW_NETS`: netblocks whose clients are refused only for a size or
time limit or a missing or wrong token, such as your monitoring.
- `SWWAF_RATE_LIMIT_EXEMPT_NETS`: netblocks exempt from rate limiting and the
byte limits, such as a machine that talks to the app all day.
- `SWWAF_TRAP_PATHS`: paths only scanners ask for, such as
@@ -189,8 +237,7 @@ values it takes. The ones most deployments touch:
the client.
- `SWWAF_WAF_DISABLED_RULES`: Core Rule Set rule ids to switch off when a rule
refuses your app's ordinary requests. The log line names the rules a request
matched. Keep the six default ids in a list you give (see
[`docs/settings.md`](docs/settings.md)).
matched.
- `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and
`SWWAF_UPSTREAM_REQUEST_TIMEOUT`: raise them for an app that takes uploads
over 100 MiB or slower than 60 seconds.
@@ -214,8 +261,9 @@ values it takes. The ones most deployments touch:
`SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app. Its log
line has `"would_action":"banned"` and `"rule_ids":["env-file"]`. In `enforce`
mode the same request bans your address for seven days, and your next request
makes the ban permanent. Lift the ban by editing `bans.json` (see "Bans" in
[`SPEC.md`](SPEC.md)).
makes the ban permanent. Lift the ban by editing `bans.json` or through the
ban endpoint (see "Bans" in [`SPEC.md`](SPEC.md) and
[`docs/admin-endpoints.md`](docs/admin-endpoints.md)).
- With `SWWAF_METRICS_TOKEN` set,
`curl -H "Authorization: Bearer $TOKEN" https://<site>/_smallwebwaf/metrics`
answers with the metrics.
@@ -237,8 +285,8 @@ the `Makefile` targets are thin shims that call them. The scripts are POSIX sh.
The tests and the linter run in Docker, as phases of the `Dockerfile`, and never
on the host.
- `script/bootstrap`: installs on the host what the other scripts need, with
`make bootstrap`:
- `script/bootstrap`: installs what the other scripts need on the host.
`make bootstrap`. They need:
- `make`, `git` and `curl`;
- Go, for `gofmt`;
- node, yarn and prettier.
@@ -321,38 +369,56 @@ nixpkgs fixed at one release. An app's `Dockerfile` installs a package with
[`docs/image.md`](docs/image.md) lists what the image holds and how each part is
pinned.
- runit, a service supervisor, runs `smallwebwaf` and the app side by side, each
as its own user, and starts either again when it exits. The app cannot touch
the state files or the process of `smallwebwaf`.
- `smallwebwaf` listens on port 8080 and passes requests to the app on
`127.0.0.1:8081`. The app listens on `127.0.0.1` only, so nothing reaches it
except through `smallwebwaf`. `smallwebwaf`'s own endpoints are under
`/_smallwebwaf/`, on the same port.
- The app sees the headers traefik sent, with traefik's address added to
`X-Forwarded-For`, from a TCP peer on `127.0.0.1`. So the app must trust
`127.0.0.1` and `::1` for forwarded headers, besides the private address
ranges.
- Both processes start with the container's environment variables, so every
setting of `smallwebwaf` starts with `SWWAF_`, and the app can read a token
given as one. A token given as a file that only the `smallwebwaf` user can
read is out of its reach (see "Settings given as files" in
- The container's first process is `runsvinit`, which starts runit, a service
supervisor. runit runs the `run` script of each directory under
`/etc/service`, and runs it again a second after its program exits.
`smallwebwaf` runs as the `smallwebwaf` user (uid 65532) and the app as a user
of its own. So the app cannot touch the state files or the process of
`smallwebwaf`.
- `smallwebwaf` listens on port 8080, on every address. traefik sends the app's
requests there. `smallwebwaf` passes them to the app on `127.0.0.1:8081`. The
app listens on `127.0.0.1` only, so nothing reaches it except through
`smallwebwaf`. The health check, the metrics and the ban management are on the
same port, under `/_smallwebwaf/`. A request for that prefix never reaches the
app.
- The app sees what it would see from traefik directly: the same `Host`, the
same `X-Forwarded-Proto`, and `X-Forwarded-For` with traefik's address added
at the end. The app's TCP peer is `smallwebwaf` on `127.0.0.1`. So the app
must trust `127.0.0.1` and `::1` for forwarded headers, besides the private
address ranges. An app that keeps its usual trusted proxies sees every visitor
as `127.0.0.1`. Each request also carries its id in `X-Request-ID`. Any
`X-Client-ASN` or `X-Client-Country` header a client sent is removed.
- Both processes start with the container's environment variables. That is why
every setting of `smallwebwaf` starts with `SWWAF_`. A token given as an
environment variable is readable by the app. A token given as a file that only
the `smallwebwaf` user can read is not (see "Settings given as files" in
[`docs/settings.md`](docs/settings.md)).
- The state files live in `/var/lib/smallwebwaf`. A volume mounted there keeps
them across a deploy.
- `docker stop` stops both processes at once. `smallwebwaf` gives the requests
in progress five seconds to finish, so the app must keep answering for five
- The state files live in `/var/lib/smallwebwaf`. Without a volume there, they
survive a restart of the container and are lost when a deploy replaces it. The
`run` script gives that directory and every file in it to the `smallwebwaf`
user at each start, so a host directory mounted there needs no change of
owner.
- The image's health check passes while `smallwebwaf` answers
`GET /_smallwebwaf/healthz` and the app accepts connections. traefik sends a
container no requests until it is healthy.
- When `smallwebwaf` or the app exits, runit starts it again a second later. The
other process keeps running. While either is down, the client gets `502`. A
`smallwebwaf` that stops at start, as on an invalid setting, is started again
every second until the cause is fixed, and logs its message each time.
- `docker stop` has runit send `SIGTERM` to both processes at once.
`smallwebwaf` gives the requests in progress five seconds to finish and writes
its state files. To let them finish, the app must keep answering for five
seconds after `SIGTERM`.
"Deployment" in [`SPEC.md`](SPEC.md) has the rest: the health check, what
happens when a process stops, moving `smallwebwaf` to another port, and the
forwarded headers. "Architecture" in [`SPEC.md`](SPEC.md) lists the packages and
what each does.
"Deployment" in [`SPEC.md`](SPEC.md) has the rest: moving `smallwebwaf` to
another port, the health check's timing, and the forwarded headers.
"Architecture" in [`SPEC.md`](SPEC.md) lists the packages and what each does.
## TODO
- Planned, as "Planned, not built" in [`SPEC.md`](SPEC.md) describes:
- sharing bans between the instances of several apps;
- sending the log over RELP.
- Everything `SPEC.md` describes is built.
- Two points `SPEC.md` defers until there is experience to go on: sharing bans
between the instances of several apps, and sending the log over RELP.
## Documents
+1604 -1082
View File
File diff suppressed because it is too large. Load diff
+10 -16
View File
@@ -42,7 +42,7 @@ on one line:
"asn": "",
"as_name": "",
"country": "",
"reason": "requests per minute over the limit of 6000",
"reason": "requests per minute over the limit of 1000",
"detail": {
"ban_expires": "2026-10-06T13:00:00.123Z",
"cause": "limit",
@@ -51,9 +51,9 @@ on one line:
"as_name": "",
"country": "",
"kind": "requests",
"limit": 6000,
"limit": 1000,
"window": "minute",
"count": 6001,
"count": 1001,
"request": {
"time": "2026-10-06T12:00:00.123456789Z",
"method": "GET",
@@ -62,7 +62,7 @@ on one line:
"status": 403,
"user_agent": "scraper/1.0"
},
"requests": 8210,
"requests": 5210,
"refused": 0,
"earlier_bans": {
"limit": 0,
@@ -78,18 +78,14 @@ on one line:
- `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
one of the events above.
- `client`: the address whose request raised the alert. Empty for
`source_failure`, `file_error` and `summary`.
- `netblock`: empty for `source_failure`, `file_error` and `summary`; otherwise
one of:
- the ban's netblock;
- the netblock an anomaly counted, empty in the `asn` and `total` scopes;
- for the other events, the client as rate limiting counts it: an IPv4
address as a /32, or an IPv6 group.
- `asn`, `as_name` and `country`: the client's, as known when the alert was
- `client`: the address whose request raised the alert. `netblock`: the ban's
netblock, the netblock an anomaly counted, or, for the other events, the
client as rate limiting counts it: an IPv4 address as a /32, or an IPv6 group.
Both are empty for `source_failure` and `file_error`.
- `asn`, `as_name` and `country`: the client's as known when the alert was
raised. They are empty when GeoJS had not answered yet.
- `reason`: a short sentence, such as
`requests per minute over the limit of 6000` or `listed by a blocklist`.
`requests per minute over the limit of 1000` or `listed by a blocklist`.
- `detail`: what is particular to the event:
- for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed
under `bans.json` in [`state-files.md`](state-files.md));
@@ -98,8 +94,6 @@ on one line:
- for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`;
- for a source failure, the `source` and `error`;
- for a file error, the `file` and `error`;
- for a summary, the `hour` it covers, the `count` of alerts held back, and
that count by event in `events`;
- `mode`, when the request was not refused: `detect` or `observe`.
- `suppressed_repeats`: how many repeats the cooldown held back before this
alert.
+42 -49
View File
@@ -11,10 +11,8 @@ checked again when gitea or the Core Rule Set changes version.
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
by it.
- A clone is a response. It fits the defaults of 30 minutes and 1 GiB unless the
repository is large or the link slow. A repository whose clone is larger than
1 GiB needs `SWWAF_RESPONSE_MAX_BYTES` raised, and the byte limits with it
("Byte limits, per client" in `SPEC.md`).
- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all
but the largest repositories and slowest links.
- A push is a request, and so is every other upload:
- a git push;
- an LFS object;
@@ -23,22 +21,21 @@ checked again when gitea or the Core Rule Set changes version.
- a release attachment.
At the defaults, an upload larger than 100 MiB or taking more than 60
seconds is cut off ("Size and time limits" in `SPEC.md`). A gitea that takes
large uploads needs `SWWAF_REQUEST_MAX_BYTES`,
`SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised
to fit.
seconds is cut off, and no one is banned for it. "Size and time limits" in
`SPEC.md` gives the answers the client gets. A gitea that takes large
uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and
`SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit.
- Archive download and blame or history pages are what scrapers hammer. The
request limits do most of the work there.
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
43,200 requests a day. While it runs a job, it reports the job's log and state
every second instead, two requests a second. So one runner busy all day sends
172,800 requests, under the request limits. Six such runners on one address,
each busy all day, pass the day limit of 1,000,000 within two days; five stay
under it. Their address is then banned, their running jobs fail, and each
repeat within a day makes the ban longer. Newer runners ask for work less
often. Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
43,200 requests a day, and reports a running job's log and state every second.
So it passes the day limit of 50,000 after about an hour and a quarter of
jobs. It is then banned, the job it is running fails, and each repeat within a
day makes the ban longer. Newer runners ask less often, but a busy one, or
several on one address, can still pass it. Put the runners' addresses in
`SWWAF_RATE_LIMIT_EXEMPT_NETS`.
## What the Core Rule Set lets through
@@ -66,15 +63,15 @@ toward). The request log names the rule in `waf_rule_ids`, which
### A query string that reads as an attack
Most often this is a search. A search term is a query parameter, which the Core
Rule Set checks against its lists of system files and commands (see the fourth
change under "Attack detection" in `SPEC.md`). A release attachment uploaded
through the API as `docker-compose.yml` is refused the same way. Examples:
Most often this is a search. Every query parameter is checked against the Core
Rule Set's lists of system files and commands, except the ten named under
"Attack detection" in `SPEC.md`. A release attachment uploaded through the API
as `docker-compose.yml` is refused the same way. Examples:
- a name on those lists, such as `package.json`, `.gitignore` or
`docker-compose.yml`;
- text that starts with a command name, such as `python3` or `ssh key`, or has
one right after a `/` in its first word, such as `feat/docker-support`;
- text that starts with a command name, or whose first word has one right after
a `/`, such as `python3`, `ssh key` or `feat/docker-support`;
- a shell command with its options or a system path (`ls -la`, `sed -i`,
`/bin/sh`), or a command in backticks;
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
@@ -84,15 +81,15 @@ through the API as `docker-compose.yml` is refused the same way. Examples:
### A branch, tag or file name that starts with a command name
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
`python3`, `ansible`, `base64` and `whoami`. A query parameter holding a branch,
tag or file name that starts with one is refused. So is one that has such a name
right after a `/`, such as `feat/docker-support`, `fix/docker-build` or
`python3`, `ansible`, `base64` and `whoami`. A branch, tag or file name that
starts with one is refused. So is one that has such a name right after a `/`,
such as `feat/docker-support`, `fix/docker-build` or
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
query parameter that 930120, 932160 and 932260 still check:
- `refSubUrl`, the branch or tag of a directory listing. A listing sends it when
it asks for its entries' last commits in a second request. It does so whenever
gitea takes more than a second to work them out;
- `refSubUrl`, the branch or tag of a directory listing. It is sent when the
listing asks for its entries' last commits in a second request, which it does
whenever gitea takes more than a second to work them out;
- `name`, when a branch is deleted or restored on the branches page;
- `tag`, when a release is started from a tag;
- `template`, the file of an issue template, when an issue is opened from it;
@@ -113,9 +110,9 @@ What a visitor sees, for a branch named `docker-build`:
### A file name in a query parameter that trips the code rules
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
look for in some file names. This holds too in the ten query parameters that
930120, 932160 and 932260 do not check (see the fourth change under "Attack
detection" in `SPEC.md`). Examples:
look for in some file names. This holds in the parameters `smallwebwaf` leaves
out of 930120, 932160 and 932260 too (see the changes to the Core Rule Set under
"Attack detection" in `SPEC.md`). Examples:
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
(944110);
@@ -132,9 +129,9 @@ For such a file, these requests are refused:
- the API's call for the contents of files that names it in its query (`body`);
- the commit graph of that file alone (`file`);
- the hidden lines of its diff (`path`), which stay hidden;
- the sign-in page, for a visitor who opens a page of the file that needs
signing in, such as its edit page (`redirect_to`). The visitor can still sign
in by opening the sign-in page itself.
- the sign-in page gitea sends a visitor to on the way to a page of the file
that needs signing in, such as its edit page (`redirect_to`). The visitor can
still sign in by opening the sign-in page itself.
### A path
@@ -146,30 +143,25 @@ For such a file, these requests are refused:
`process.env.d.ts` does.
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
For such a file, these are refused:
- the page of the file;
- its raw file;
- its history and its blame;
- its edit page.
The directory that holds it lists it, and a clone carries it.
For such a file, the page of the file, its raw file, its history, its blame and
its edit page are refused. The directory that holds it lists it, and a clone
carries it.
### A container image push that reuses a layer
A push can reuse a layer of an image docker has pushed to that gitea before.
docker then asks gitea to take the layer from that image, naming it in the query
parameter `from`, which 932260 checks. The request is refused when the image's
owner starts with an entry on the list of commands. It is refused too when the
image's name after the owner starts with one. Examples are `owner/python3-app`,
`owner/docker-base`, and any image of an owner named `ansible` or `docker-team`.
owner starts with an entry on the list of commands. So is one whose name after
the owner does. Examples are `owner/python3-app`, `owner/docker-base`, and any
image of an owner named `ansible` or `docker-team`.
What happens next depends on docker's image store. A docker that keeps its
images in containerd's image store fails the push, and fails each new try the
same way. A new install of Docker Engine 29 does this. A docker that keeps them
in its own older image store uploads the layer instead. The push then succeeds
after one refusal for each layer it reuses, and each of those refusals counts
toward the error burst ("Attack detection" in `SPEC.md`).
toward the error burst.
### Artifact uploads from `actions/upload-artifact@v3`
@@ -177,9 +169,9 @@ This action sends the header `Content-Range`, which the Core Rule Set refuses
(920450). The action sends two files at a time, does not retry a `403`, and
sends no further file once one is refused. So an upload is refused once or
twice, however many files it holds, and the step fails. One upload does not ban
the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
than 30 such refusals from its address in a minute, as when many jobs upload at
once. `actions/upload-artifact@v4` does not send the header.
the runner. More than 30 such refusals from its address within a minute, as when
many jobs upload at once, ban it through the error burst.
`actions/upload-artifact@v4` does not send the header.
## What body inspection refuses
@@ -201,7 +193,8 @@ Of the requests above, the script sends:
code rules", the API's list of its commits;
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
its diff, and the API's call for the contents of files that names it;
- the sign-in page for a visitor who opens the edit page of `document.write.js`;
- the sign-in page gitea sends a visitor to on the way to the edit page of
`document.write.js`;
- the pages of `java.lang.String.md`,
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
and `.../notes.txt`;
+2 -3
View File
@@ -31,7 +31,6 @@ is pinned that way.
- nixpkgs: the release file `nixexprs.tar.xz` of
`nixos-26.05.11045.774debe7a0d1` on `releases.nixos.org`, named with its
SHA-256 hash.
- `runsvinit`: the archived repository, unchanged since 2015, at its last commit
(`b4b2c78`, eight commits after `v2.0.0`), by commit hash, with a `go.mod` the
build writes.
- `runsvinit`: the archived repository, unchanged since 2015, at tag `v2.0.0`,
by commit hash, with a `go.mod` the build writes.
- The Go image that builds `smallwebwaf`: by digest.
+5 -3
View File
@@ -1,8 +1,10 @@
# The metrics
What `GET /_smallwebwaf/metrics` answers with. "Metrics endpoint" in
[`SPEC.md`](../SPEC.md) describes the endpoint. Prometheus keeps the `instance`
label as `exported_instance` unless the scrape sets `honor_labels: true`.
`GET /_smallwebwaf/metrics` answers in the Prometheus text format. The
requirements are under "Metrics endpoint" in [`SPEC.md`](../SPEC.md); this file
lists the metrics. Every metric carries the label `instance`, set to
`SWWAF_INSTANCE_NAME`; Prometheus keeps it as `exported_instance` unless the
scrape sets `honor_labels: true`. No metric carries a client's address.
## Traffic
+12 -23
View File
@@ -12,13 +12,9 @@ sent to a syslog server. A field that does not apply to a request is left out.
## Every line
Every line has:
- `type`: `request` (the process's own messages have `process`);
- the standard web log fields;
- `request_id`, `peer_ip` and `client_group`;
- `asn`, `as_name` and `country`;
- `action` and `duration_total`.
Every line has `type` (`request`; the process's own messages have `process`),
the standard web log fields, `request_id`, `peer_ip`, `client_group`, `asn`,
`as_name`, `country`, `action` and `duration_total`.
## Standard web log fields
@@ -32,7 +28,7 @@ Every line has:
## Request detail
- `request_id`: the `X-Request-ID` a trusted proxy sent, or a new random one of
26 letters and digits.
26 letters and digits. It goes to the app in `X-Request-ID`.
- `peer_ip`: the TCP peer, normally traefik.
- `forwarded_for`: the `X-Forwarded-For` header as received.
- `client_group`: the client as rate limiting counts it, an IPv4 address as a
@@ -77,22 +73,15 @@ Every line has:
rate limits and the setting that lowered it, when one did. `bytes_percent` and
`bytes_percent_setting` do the same for the byte limits.
- `counts`: the client's totals as rate limiting counts them, this request
included. It is left out when rate limiting did not count the request. It
holds:
included. It is left out when rate limiting did not count the request. A count
can have a fraction, since each window is counted in two buckets with the
earlier one weighted:
- `minute`, `hour` and `day`: requests;
- `minute_bytes`, `hour_bytes` and `day_bytes`: bytes.
A count can have a fraction (see "Counting method" in
[`SPEC.md`](../SPEC.md)).
- `limit_hit`: the limit broken:
- `minute`, `hour` or `day` for a rate limit;
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
- `error_burst`.
A request whose bytes broke a limit was passed on whole, so its `action`
stays `forward`.
- `limit_hit`: the limit broken: `minute`, `hour` or `day` for a rate limit;
`minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or
`error_burst`. A request whose bytes broke a limit was passed on whole, so its
`action` stays `forward`.
- `rule_ids`: the rule file rules that matched, in order, up to the one that
refused.
- `waf_score` and `waf_rule_ids`: the Core Rule Set's score and the rules that
@@ -102,7 +91,7 @@ Every line has:
- the CrowdSec decision list's URL;
- DNS blocklist zones;
- `abuseipdb`.
- `offence`: `limit`, for a request that broke a limit.
- `offence`: the kind of offence recorded, when one was.
- `ban_expires`: when the client's ban ends, for a request that made one or was
refused under one: a time, or `permanent`.
+29 -33
View File
@@ -2,7 +2,8 @@
Every setting, its default and the values it takes, in the groups of "Settings"
in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings
that take `off`, the `_FILE` form, and what each group does and why.
that take `off`, the `_FILE` form, and what each group does and why. A setting
set to a value its entry does not allow stops the start.
## Forms of values
@@ -26,9 +27,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or
`https` URL with a host and an optional port, and nothing more.
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens: `:` and a
port, such as `:9000`. Leave the host out (see "Ports" in
[`SPEC.md`](../SPEC.md)).
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens, an
optional host and a port. The host stays empty.
- `SWWAF_INSTANCE_NAME` (default: the container's host name): the `instance`
name in every log line, alert and metric, such as `fsn1app1/gitea`. Valid
UTF-8. No control character while `SWWAF_ALERT_NTFY_URL` is set. 1 to 48
@@ -42,8 +42,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
client's group, from 32 to 128.
- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a whole
number above zero.
- `SWWAF_MAX_BANS` (default `5000`): how many of its own bans `smallwebwaf`
keeps, a whole number above zero.
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
kept, a whole number above zero.
## Tokens
@@ -90,25 +90,24 @@ that take `off`, the `_FILE` form, and what each group does and why.
## Static lists
- `SWWAF_ALLOW_NETS` (default empty): netblocks you trust, such as your
monitoring (see step 3 in [`SPEC.md`](../SPEC.md)).
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients are refused only
for a size or time limit or a missing or wrong token, such as your monitoring.
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
exempt from rate limiting and the byte limits.
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are refused (see
step 3 in [`SPEC.md`](../SPEC.md)).
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
## Request rate limits, per client
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `6000`), `SWWAF_RATE_LIMIT_PER_HOUR`
(default `100000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `1000000`): the
most requests a client may make in each window. A count or `off`.
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
(default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
requests a client may make in each window. A count or `off`.
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
rate limiting and the byte limits, such as `/assets/`.
## Byte limits, per client
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `4G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `200G`): the most
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most
bytes counted for a client in each window. A size or `off`.
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
`response`, `request` or `both`.
@@ -131,18 +130,12 @@ that take `off`, the `_FILE` form, and what each group does and why.
to send its whole answer. A duration or `off`.
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size
or `off`.
- `SWWAF_RESPONSE_MAX_BYTES` (default `1G`): the largest response body. A size
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size
or `off`.
## Lookup of AS number and country
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off` stops
the start while a setting that needs a lookup is set:
- the two country lists;
- `SWWAF_ADD_LOOKUP_HEADERS`;
- the four biased threshold lists and `SWWAF_ASN_LIMIT_PERCENT_URL`;
- `SWWAF_UNKNOWN_LIMIT_PERCENT` below `100`;
- an anomaly threshold per AS number.
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`.
- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
`.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`.
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
@@ -157,7 +150,7 @@ that take `off`, the `_FILE` form, and what each group does and why.
such as `cn,ru,kp`.
- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
countries whose clients get through, such as `us,de`. A country on both lists
stops the start.
is refused.
## Biased thresholds
@@ -171,8 +164,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage for a client the
lookup cannot place.
- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
text file with one AS number and percentage on each line, such as
`AS64496:50`. A URL that is also in `SWWAF_BLOCKLIST_URLS` is refused.
text file of `AS<number>:<percent>` lines, one to a line. A URL that is also
in `SWWAF_BLOCKLIST_URLS` is refused.
## Attack detection
@@ -196,7 +189,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off`, or how much of a request body
the Core Rule Set reads, a size of at most `1G`.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
in a minute for a rule match, a trap path, a Core Rule Set match or a failed
token check. A count or `off`.
## Ban lengths, response and scope
@@ -266,8 +260,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
`ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`):
the event types alerts are sent for, from those in [`alerts.md`](alerts.md)
but `summary`. Any other name is refused.
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long after an alert is sent its
repeats are held back. A duration or `off`.
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held
back. A duration or `off`.
- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. A
whole number above zero, or `off`.
@@ -291,7 +285,7 @@ as `REQUESTS_PER_MINUTE`:
client;
- `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
client.
- `SWWAF_ANOMALY_ASN_...`: per AS number.
- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs a lookup source.
- `SWWAF_ANOMALY_TOTAL_...`: the whole service.
- `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole.
`SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
@@ -300,8 +294,10 @@ as `REQUESTS_PER_MINUTE`:
## Settings given as files
To give a token as a file, as "Tokens" under "Deployment" in
[`SPEC.md`](../SPEC.md) describes, make the file on the host:
Add `_FILE` to a setting's name to read its value from a file:
`SWWAF_METRICS_TOKEN_FILE` names the file that holds `SWWAF_METRICS_TOKEN`. To
keep a token from the app, make the file on the host, owned by uid 65532 with
mode `0400`:
```sh
mkdir -p /srv/app/tokens
@@ -310,7 +306,7 @@ chown 65532:65532 /srv/app/tokens/metrics
chmod 0400 /srv/app/tokens/metrics
```
Then mount its directory into the container read-only, and name the file:
Mount its directory into the container read-only, and name the file:
```sh
docker run \
+15 -19
View File
@@ -15,8 +15,8 @@ those `smallwebwaf` made. Per entry:
- `lifted`: when an admin lifted it, if one did;
- `notes`: the notes below.
An entry an admin adds needs `netblock`, `start` and `expires`; `reason`,
`notes` and `cause` may be left out, and the cause is then `admin`:
To ban a netblock, add an entry with its `netblock`, `start` and `expires`.
`reason`, `notes` and `cause` may be left out; the cause is then `admin`:
```json
{
@@ -32,23 +32,20 @@ An entry an admin adds needs `netblock`, `start` and `expires`; `reason`,
}
```
"Bans" in [`SPEC.md`](../SPEC.md) says how an edit lifts, keeps or forgets a
ban. The notes:
To lift a ban, add `"lifted"` with the time you lift it, such as
`"lifted": "2026-10-06T13:00:00Z"`. To keep a ban `smallwebwaf` made, set its
`cause` to `admin`. To forget a ban, delete its entry.
The notes:
- `asn`, `as_name` and `country`: the client's, added when the lookup answers
(empty when lookups are off);
- for a broken limit, `kind` (`requests`, `bytes` or `refusals`), `limit`,
`window` and `count`: what the limit was on, the limit, its window and the
count reached;
- `limit_percent` and `limit_percent_setting`: the client's percentage of that
limit and the setting that gave it, when a setting lowered it;
- for a clear sign of attack, `rule_id` and `target`, or `trap_path`;
- `reputation`: the sources the client was on, each with its `source` and, for
AbuseIPDB, its `score`;
- `request`: the request that caused the ban, each text cut to 256 bytes:
- `time`, `method` and `host`;
- `path`, with its query string;
- `status` and `user_agent`;
- what was broken: the rule id and target that matched, or the trap path, or the
limit, its window and the count reached;
- the client's percentage of the limit, with the setting that gave it;
- the reputation sources the client is on;
- `request`: the request that caused the ban: time, method, host, path with its
query string, status and user agent, each text cut to 256 bytes;
- `requests` and `refused`: the netblock's requests since it was first seen, and
how many this ban has refused so far, kept up to date while the ban lasts;
- `earlier_bans`: how many earlier bans the netblock had, by cause.
@@ -96,9 +93,8 @@ GeoJS answers, one per line:
`SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of
disk writes a day.
- `bans.json`: about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at
the default `SWWAF_MAX_BANS`. Written every 15 minutes, a full file comes to
about 600 MiB of writes a day, and a hundred new bans a day add about 600 MiB
more.
the default `SWWAF_MAX_BANS`. With a full file, a hundred new bans a day come
to about 600 MiB of writes.
- `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
full.
- `reputation.json` and `alerts.json`: usually a few MiB or less.
+6 -5
View File
@@ -1,8 +1,8 @@
// Package alerts sends alerts on bans, on traffic over an anomaly
// threshold, on a source that fails and on a file with an error to each
// destination set: to the webhook
// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON
// object" in docs/alerts.md describes, to the Slack incoming
// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as the "Alert
// webhook schema" section of SPEC.md describes, to the Slack incoming
// webhook SWWAF_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
// ntfy topic SWWAF_ALERT_NTFY_URL names. A repeat within
// SWWAF_ALERT_COOLDOWN is held back, and so is an alert past
@@ -149,10 +149,11 @@ type Params struct {
}
// Alert is one alert, as the webhook is sent it and alerts.json holds it,
// with the fields of "The JSON object" in docs/alerts.md. ASN, ASName and
// Country are, for a ban, the client's as the ban's notes give them.
// with the fields of the "Alert webhook schema" section of SPEC.md. ASN,
// ASName and Country are, for a ban, the client's as the ban's notes give
// them.
//
//nolint:tagliatelle // docs/alerts.md names the alert's fields in snake_case
//nolint:tagliatelle // SPEC.md's alert webhook schema names its fields in snake_case
type Alert struct {
Instance string `json:"instance"`
Time time.Time `json:"time"`
+2 -2
View File
@@ -1,8 +1,8 @@
// Package anomaly counts requests and bytes over a minute and an hour, per
// client, per surrounding netblock, per AS number, for the whole service
// and per named netblock, and raises an anomaly alert for a count over its
// threshold, as "Anomaly thresholds" under "Settings" in SPEC.md
// describes. It refuses and bans nothing. At most 20,000 counters
// threshold, as "Anomaly thresholds" under "Configuration surface" in
// SPEC.md describes. It refuses and bans nothing. At most 20,000 counters
// are kept, in memory, and written to alerts.json and read from it by the
// state package.
package anomaly
+6 -6
View File
@@ -502,17 +502,17 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
UpstreamRequestTimeout: env.duration("SWWAF_UPSTREAM_REQUEST_TIMEOUT", "60s"),
UpstreamResponseTimeout: env.duration("SWWAF_UPSTREAM_RESPONSE_TIMEOUT", "30m"),
RequestMaxBytes: env.size("SWWAF_REQUEST_MAX_BYTES", "100M"),
ResponseMaxBytes: env.size("SWWAF_RESPONSE_MAX_BYTES", "1G"),
ResponseMaxBytes: env.size("SWWAF_RESPONSE_MAX_BYTES", "5G"),
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "6000"),
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "100000"),
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "1000000"),
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "1000"),
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "10000"),
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "50000"),
RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "4G"),
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "10G"),
BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "20G"),
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "200G"),
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "50G"),
BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
LookupSource: env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
+12 -12
View File
@@ -261,10 +261,10 @@ func TestDefaults(t *testing.T) {
UpstreamRequestTimeout: time.Minute,
UpstreamResponseTimeout: 30 * time.Minute,
RequestMaxBytes: 100 << 20,
ResponseMaxBytes: 1 << 30,
RateLimitPerMinute: 6000,
RateLimitPerHour: 100000,
RateLimitPerDay: 1000000,
ResponseMaxBytes: 5 << 30,
RateLimitPerMinute: 1000,
RateLimitPerHour: 10000,
RateLimitPerDay: 50000,
BanResponse: 403,
LimitBanDuration: time.Hour,
LimitBanRepeatWindow: 24 * time.Hour,
@@ -284,8 +284,8 @@ func TestDefaults(t *testing.T) {
LookupSource: defaultLookupSource, LookupTimeout: time.Second,
})
wantByteLimitSettings(t, cfg, config.Config{
BytesLimitPerMinute: 4 << 30, BytesLimitPerHour: 20 << 30,
BytesLimitPerDay: 200 << 30, BytesCount: "both",
BytesLimitPerMinute: 10 << 30, BytesLimitPerHour: 20 << 30,
BytesLimitPerDay: 50 << 30, BytesCount: "both",
})
if cfg.UpstreamURL.String() != "http://127.0.0.1:8081" {
@@ -2371,17 +2371,17 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
upstreamRequestTimeout: "60s",
upstreamResponseTimeout: "30m",
requestMaxBytes: "100M",
responseMaxBytes: "1G",
responseMaxBytes: "5G",
allowNets: "",
rateLimitExemptNets: "",
denyNets: "",
rateLimitPerMinute: "6000",
rateLimitPerHour: "100000",
rateLimitPerDay: "1000000",
rateLimitPerMinute: "1000",
rateLimitPerHour: "10000",
rateLimitPerDay: "50000",
rateLimitExemptPaths: "",
bytesLimitPerMinute: "4G",
bytesLimitPerMinute: "10G",
bytesLimitPerHour: "20G",
bytesLimitPerDay: "200G",
bytesLimitPerDay: "50G",
bytesCount: "both",
lookupSource: defaultLookupSource,
lookupDBPath: "",
+3 -4
View File
@@ -1,7 +1,6 @@
// Package metrics keeps smallwebwaf's Prometheus metrics, as
// docs/metrics.md lists them, and serves them in the Prometheus text
// format, as the "Metrics endpoint" section of SPEC.md requires. No
// metric carries a client's address.
// Package metrics keeps smallwebwaf's Prometheus metrics, as the "Metrics
// endpoint" section of SPEC.md lists them, and serves them in the
// Prometheus text format. No metric carries a client's address.
package metrics
import (
+1 -1
View File
@@ -206,7 +206,7 @@ type Hit struct {
// Counts are a client's requests and bytes in the minute, the hour and
// the day that end at a request, that request's included.
//
//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
type Counts struct {
Minute float64 `json:"minute"`
Hour float64 `json:"hour"`
+4 -3
View File
@@ -64,10 +64,11 @@ const LimitHitErrorBurst = "error_burst"
const timeLayout = "2006-01-02T15:04:05.000Z07:00"
// Line is one request's line in the request log. The field names, and
// their order, are those of docs/request-log.md. A field that may not
// apply to a request is left out of its line when it does not.
// their order, are those of the "Request log" section of SPEC.md. A field
// that may not apply to a request is left out of its line when it does
// not.
//
//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
type Line struct {
Type string `json:"type"`
+6 -13
View File
@@ -60,19 +60,8 @@ func TestHealthCheck(t *testing.T) {
}, 0, "")
}
// Another test can take a closed server's port at once, so the app's
// address here is a multicast address, to which TCP cannot connect.
wantHealthCheck(t, map[string]string{
listenAddr: ":" + port,
upstreamURL: "http://224.0.0.1",
}, 1, "unhealthy: connect to the app: ")
// Listening on port 0 takes any free port, so nothing listens on port 0
// itself.
wantHealthCheck(t, map[string]string{listenAddr: ":0"}, 1,
"unhealthy: ask smallwebwaf: ")
wantHealthCheck(t, map[string]string{listenAddr: "8080"}, 1,
"unhealthy: invalid setting: SWWAF_LISTEN_ADDR: ")
app.Close()
wantHealthCheck(t, env, 1, "unhealthy: connect to the app: ")
stop()
@@ -81,6 +70,10 @@ func TestHealthCheck(t *testing.T) {
case <-time.After(waitLimit):
t.Fatal("still running after being told to stop")
}
wantHealthCheck(t, env, 1, "unhealthy: ask smallwebwaf: ")
wantHealthCheck(t, map[string]string{listenAddr: "8080"}, 1,
"unhealthy: invalid setting: SWWAF_LISTEN_ADDR: ")
}
func TestHealthCheckRefusesAnArgument(t *testing.T) {
+4 -4
View File
@@ -1240,13 +1240,13 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL, dir string) {
"SWWAF_UPSTREAM_REQUEST_TIMEOUT": "60s",
"SWWAF_UPSTREAM_RESPONSE_TIMEOUT": "30m",
"SWWAF_REQUEST_MAX_BYTES": "100M",
"SWWAF_RESPONSE_MAX_BYTES": "1G",
"SWWAF_RESPONSE_MAX_BYTES": "5G",
"SWWAF_ALLOW_NETS": "",
rateLimitExemptNets: "",
"SWWAF_DENY_NETS": "",
"SWWAF_RATE_LIMIT_PER_MINUTE": "6000",
"SWWAF_RATE_LIMIT_PER_HOUR": "100000",
rateLimitPerDay: "1000000",
"SWWAF_RATE_LIMIT_PER_MINUTE": "1000",
"SWWAF_RATE_LIMIT_PER_HOUR": "10000",
rateLimitPerDay: "50000",
"SWWAF_RATE_LIMIT_EXEMPT_PATHS": "",
"SWWAF_DENIED_COUNTRIES": "",
"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
+1 -2
View File
@@ -1,6 +1,5 @@
// Package state keeps smallwebwaf's state in JSON files in
// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md requires
// and docs/state-files.md lists them:
// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md describes:
// bans.json holds the bans, clients.json each client's counters and
// history, lookups.json GeoJS's answers, reputation.json the last try and
// last good copy of each list fetched from a URL, the DNSBL zones'
+2 -2
View File
@@ -1019,8 +1019,8 @@ func TestEditJustBeforeAScheduledWriteSurvivesIt(t *testing.T) {
edit(t, dir, bansJSON, permanentBansJSON)
// The write takes the edit in first, and writes it back. The second
// ban, made after the admin opened the file, is lost, as the
// "Persistent state" section of SPEC.md says.
// ban, made after the admin opened the file, is lost, as "Edits
// while running" in SPEC.md says.
time.Sleep(time.Nanosecond)
synctest.Wait()
+2 -2
View File
@@ -1,8 +1,8 @@
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
// method, the URL with its query and the headers of a request, and on its
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
// them. It reads no response.
// makes to it, as "Attack detection" under "Configuration surface" in
// SPEC.md describes them. It reads no response.
//
// smallwebwaf writes only to its state directory, so Coraza is built with
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
+5 -6
View File
@@ -2,12 +2,11 @@
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
# which names gitea's container. Then send through it the gitea requests
# that reading gitea's source found the Core Rule Set refuses, which
# docs/gitea.md records, and a short sample of gitea's ordinary use:
# signing in, pages of files and commits, a diff, the API, a push, clone
# and fetch over HTTP, and two container image pushes. Last, print each
# request smallwebwaf logged, with what it did with it and the ids of the
# Core Rule Set's rules that matched it.
# that reading gitea's source found the Core Rule Set refuses, and a short
# sample of gitea's ordinary use: signing in, pages of files and commits,
# a diff, the API, a push, clone and fetch over HTTP, and two container
# image pushes. Last, print each request smallwebwaf logged, with what it
# did with it and the ids of the Core Rule Set's rules that matched it.
# Every request comes from one client, docker's gateway, so more than
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
# the requests after would show as banned. The containers, the network,