Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
91438f54da |
No files matched your search
@@ -8,7 +8,8 @@ scanners and other abusers before they reach the app.
|
||||
A web application firewall reads each request on its way to a web app and
|
||||
refuses the ones that look like abuse. `smallwebwaf` is also a reverse proxy: it
|
||||
accepts requests on the app's behalf, passes them to the app, and passes the
|
||||
app's answers back.
|
||||
app's answers back. A request it does not refuse reaches the app as it came,
|
||||
apart from a few headers (see "Design" below).
|
||||
|
||||
It needs no setting. Every setting has a default chosen for a service on the
|
||||
open internet. [`SPEC.md`](SPEC.md) says how `smallwebwaf` behaves and why. The
|
||||
@@ -17,10 +18,10 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
|
||||
## What it protects against
|
||||
|
||||
- Scrapers that fetch every page of a site, all day. Rate limiting gives each
|
||||
client 6000 requests a minute, 100,000 an hour and 1,000,000 a day. Byte
|
||||
limits give it 4 GiB a minute, 20 GiB an hour and 200 GiB a day. A client over
|
||||
a limit is refused and banned. The first ban lasts an hour. Each repeat within
|
||||
a day makes the next ban three times as long. A ban that would last more than
|
||||
client 1000 requests a minute, 10,000 an hour and 50,000 a day. Byte limits
|
||||
give it 10 GiB a minute, 20 GiB an hour and 50 GiB a day. A client over a
|
||||
limit is refused and banned. The first ban lasts an hour. Each repeat within a
|
||||
day makes the next ban three times as long. A ban that would last more than
|
||||
seven days is permanent.
|
||||
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
||||
default rule file bans such a client for seven days at its first probe. Its
|
||||
@@ -29,9 +30,9 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
|
||||
of your own, or an entry in `SWWAF_TRAP_PATHS`, bans those the same way.
|
||||
- Attacks carried in a request's URL or headers, such as SQL injection and path
|
||||
traversal. The OWASP Core Rule Set is a published set of rules that recognise
|
||||
common web attacks. Coraza, a Go engine, runs those rules (see "Attack
|
||||
detection" in [`SPEC.md`](SPEC.md)). A request the rules take for an attack is
|
||||
refused.
|
||||
common web attacks. Coraza, a Go engine that runs those rules, inspects each
|
||||
request the earlier checks let through. A request the rules take for an attack
|
||||
is refused.
|
||||
- Abuse from networks and countries you do not want. You can:
|
||||
- refuse whole netblocks and countries;
|
||||
- give the clients of chosen networks and countries a fraction of every
|
||||
@@ -131,9 +132,8 @@ docker run --detach --name app \
|
||||
|
||||
With no setting, the address of every new visitor is sent to GeoJS, a web
|
||||
service, to find its country and network. `SWWAF_LOOKUP_SOURCE=off` sends
|
||||
nothing. `SWWAF_LOOKUP_SOURCE=file`, with `SWWAF_LOOKUP_DB_PATH` naming a
|
||||
database file on your own host, looks addresses up in that file (see "Lookup of
|
||||
AS number and country" in [`SPEC.md`](SPEC.md)).
|
||||
nothing. `SWWAF_LOOKUP_SOURCE=file` looks addresses up in a database file on
|
||||
your own host (see "Lookup of AS number and country" in [`SPEC.md`](SPEC.md)).
|
||||
|
||||
`deploy/example-app` is such an app. `make example-app` builds it and checks it
|
||||
(see "Checking that it works" below). To run `smallwebwaf` outside the image
|
||||
@@ -141,47 +141,95 @@ while working on it, see "Entrypoints" below.
|
||||
|
||||
## What happens to a request
|
||||
|
||||
`smallwebwaf` handles each request in fifteen steps, in this order. Once a step
|
||||
refuses a request, the steps after it are skipped, apart from step 15, and
|
||||
nothing of the request reaches the app. "What happens to a request" in
|
||||
[`SPEC.md`](SPEC.md) gives each step in full, under the same numbers.
|
||||
Two terms recur below. A clear sign of attack is a request that matches a rule
|
||||
whose action is `ban`, or asks for a trap path (steps 9 and 10). An offence is a
|
||||
request that counts against its client in its history. It is one of:
|
||||
|
||||
1. The health check, `GET /_smallwebwaf/healthz`, is answered at once.
|
||||
2. The client is identified.
|
||||
3. The static lists, `SWWAF_ALLOW_NETS` and `SWWAF_DENY_NETS`.
|
||||
4. The bans.
|
||||
5. The lookup of the client's AS number and country.
|
||||
6. The country lists.
|
||||
7. The reputation sources.
|
||||
8. The client's limit percentage.
|
||||
9. The rate limits.
|
||||
10. The trap paths.
|
||||
11. The rule files.
|
||||
12. The Core Rule Set.
|
||||
13. `smallwebwaf`'s own endpoints, under `/_smallwebwaf/`.
|
||||
14. The request is passed to the app.
|
||||
15. After the answer: the byte limits, the error burst and the client's history.
|
||||
- a clear sign of attack;
|
||||
- a broken limit;
|
||||
- a refusal by a rule, by the Core Rule Set or for a missing or wrong token.
|
||||
|
||||
A refused visitor is answered with `SWWAF_BAN_RESPONSE`, `403` by default (see
|
||||
"Ban lengths, response and scope" in [`SPEC.md`](SPEC.md)). A `block` rule or a
|
||||
Core Rule Set match answers `403`. A missing or wrong token gets `401`. A
|
||||
request over a size or time limit gets the status for that limit, such as `413`
|
||||
for a body too large (see "Size and time limits" in [`SPEC.md`](SPEC.md)).
|
||||
`smallwebwaf` handles each request in this order. The first step that refuses it
|
||||
ends the handling. Nothing of a refused request reaches the app. "What happens
|
||||
to a request" in [`SPEC.md`](SPEC.md) gives each step in full.
|
||||
|
||||
1. `GET /_smallwebwaf/healthz` is answered `200` with `ok` at once, before any
|
||||
check and without asking the app.
|
||||
2. The client is worked out. A netblock is a range of addresses, written as an
|
||||
address and a prefix length: `203.0.113.0/24`. `SWWAF_TRUSTED_PROXIES` lists
|
||||
the netblocks traefik comes from, by default the private address ranges. For
|
||||
a TCP peer inside them, `X-Forwarded-For` is read from the right, and the
|
||||
first address outside those netblocks is the client. A peer outside them is
|
||||
the client itself. An IPv6 client is its group: the whole netblock around its
|
||||
address, a /64 by default; `SWWAF_IPV6_GROUP_PREFIX` sets its length.
|
||||
3. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 11, the byte limits and the
|
||||
error burst, but still needs a token at step 12. A client in
|
||||
`SWWAF_DENY_NETS` is refused.
|
||||
4. A client under a ban is refused. If the ban was made for a clear sign of
|
||||
attack, this request makes it permanent.
|
||||
5. The client's AS number and country are looked up, unless
|
||||
`SWWAF_LOOKUP_SOURCE` is `off`. An AS number names the network an address
|
||||
belongs to, such as `AS64496`.
|
||||
6. The country lists are checked. A client from a country in
|
||||
`SWWAF_DENIED_COUNTRIES` is refused. When
|
||||
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any other country
|
||||
is refused, and so is a client without a country. Such a refusal is not an
|
||||
offence.
|
||||
7. The reputation sources are checked: the blocklists, the CrowdSec decision
|
||||
list, the DNS blocklists and AbuseIPDB. A client on the CrowdSec decision
|
||||
list is banned until the decision ends. Any other source refuses the client,
|
||||
lowers its limits or only notes it, as `SWWAF_BLOCKLIST_ACTION` and
|
||||
`SWWAF_REPUTATION_ACTION` say. No request waits for a query.
|
||||
8. Rate limiting counts the request, unless the client or the path is exempt. A
|
||||
request that takes the client over a limit is refused and bans the client.
|
||||
9. A request for a path in `SWWAF_TRAP_PATHS` is refused and bans the client, as
|
||||
a clear sign of attack.
|
||||
10. The rule files are checked. A `log` rule notes the match. A `block` rule
|
||||
refuses the request. A `ban` rule refuses it and bans the client, as a clear
|
||||
sign of attack.
|
||||
11. The Core Rule Set inspects the request's method, URL and headers, and the
|
||||
body too once `SWWAF_WAF_BODY_LIMIT` is set. A request it takes for an
|
||||
attack is refused while `SWWAF_WAF_MODE` is `block`, the default, and let
|
||||
through while it is `detect`. It bans no one by itself.
|
||||
12. A request under `/_smallwebwaf/` is answered here by `smallwebwaf`, after
|
||||
its token is checked. Any other request is passed to the app. The app's
|
||||
answer is streamed back within the size and time limits.
|
||||
13. Once the answer has ended, the request's bytes are counted for the byte
|
||||
limits. If a rule, a trap path, the Core Rule Set or a token check refused
|
||||
it, the refusal is counted for the error burst. The client's history and the
|
||||
anomaly counters are updated. More than `SWWAF_ERROR_BURST_THRESHOLD`
|
||||
refusals of one client in a minute ban it.
|
||||
|
||||
In `observe` mode (`SWWAF_MODE=observe`), `smallwebwaf` makes no ban of its own.
|
||||
It refuses a request only for a size or time limit, or for a missing or wrong
|
||||
token. A request it would otherwise have refused is passed to the app, and its
|
||||
log line says what `enforce` mode would have done. Observe mode is for trying a
|
||||
configuration before enforcing it.
|
||||
|
||||
What a refused visitor sees:
|
||||
|
||||
- `SWWAF_BAN_RESPONSE`, by default `403` with the plain text `Forbidden`. It can
|
||||
be `429` instead, or `close`, which closes the connection without an answer.
|
||||
It answers:
|
||||
- a ban or a broken rate limit;
|
||||
- a `ban` rule or a trap path;
|
||||
- `SWWAF_DENY_NETS` and the country lists;
|
||||
- a reputation source whose action is `deny`.
|
||||
- `403` for a `block` rule and the Core Rule Set.
|
||||
- `413` or `408` when the client passes a size or time limit, `502` or `504`
|
||||
when the app does.
|
||||
- `401` from an endpoint of `smallwebwaf`'s own, for a request without its
|
||||
token.
|
||||
|
||||
## Settings you are likely to touch
|
||||
|
||||
Each setting is an environment variable of the app's container, such as
|
||||
`docker run -e SWWAF_MODE=observe ...`, read at start.
|
||||
[`docs/settings.md`](docs/settings.md) lists every setting, its default and the
|
||||
values it takes. The ones most deployments touch:
|
||||
|
||||
- `SWWAF_MODE=observe`: `smallwebwaf` makes no ban of its own, and refuses a
|
||||
request only for a size or time limit or a missing or wrong token. A request
|
||||
it would have refused is passed to the app, and its log line says what
|
||||
`enforce` mode would have done. For trying a configuration before enforcing
|
||||
it.
|
||||
- `SWWAF_ALLOW_NETS`: netblocks you trust, such as your monitoring. Their
|
||||
clients are refused only for a size or time limit or a missing or wrong token.
|
||||
- `SWWAF_MODE=observe`: log what would have been refused, refuse almost nothing,
|
||||
ban no one. For trying a configuration.
|
||||
- `SWWAF_ALLOW_NETS`: netblocks whose clients are refused only for a size or
|
||||
time limit or a missing or wrong token, such as your monitoring.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_NETS`: netblocks exempt from rate limiting and the
|
||||
byte limits, such as a machine that talks to the app all day.
|
||||
- `SWWAF_TRAP_PATHS`: paths only scanners ask for, such as
|
||||
@@ -189,8 +237,7 @@ values it takes. The ones most deployments touch:
|
||||
the client.
|
||||
- `SWWAF_WAF_DISABLED_RULES`: Core Rule Set rule ids to switch off when a rule
|
||||
refuses your app's ordinary requests. The log line names the rules a request
|
||||
matched. Keep the six default ids in a list you give (see
|
||||
[`docs/settings.md`](docs/settings.md)).
|
||||
matched.
|
||||
- `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and
|
||||
`SWWAF_UPSTREAM_REQUEST_TIMEOUT`: raise them for an app that takes uploads
|
||||
over 100 MiB or slower than 60 seconds.
|
||||
@@ -214,8 +261,9 @@ values it takes. The ones most deployments touch:
|
||||
`SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app. Its log
|
||||
line has `"would_action":"banned"` and `"rule_ids":["env-file"]`. In `enforce`
|
||||
mode the same request bans your address for seven days, and your next request
|
||||
makes the ban permanent. Lift the ban by editing `bans.json` (see "Bans" in
|
||||
[`SPEC.md`](SPEC.md)).
|
||||
makes the ban permanent. Lift the ban by editing `bans.json` or through the
|
||||
ban endpoint (see "Bans" in [`SPEC.md`](SPEC.md) and
|
||||
[`docs/admin-endpoints.md`](docs/admin-endpoints.md)).
|
||||
- With `SWWAF_METRICS_TOKEN` set,
|
||||
`curl -H "Authorization: Bearer $TOKEN" https://<site>/_smallwebwaf/metrics`
|
||||
answers with the metrics.
|
||||
@@ -237,8 +285,8 @@ the `Makefile` targets are thin shims that call them. The scripts are POSIX sh.
|
||||
The tests and the linter run in Docker, as phases of the `Dockerfile`, and never
|
||||
on the host.
|
||||
|
||||
- `script/bootstrap`: installs on the host what the other scripts need, with
|
||||
`make bootstrap`:
|
||||
- `script/bootstrap`: installs what the other scripts need on the host.
|
||||
`make bootstrap`. They need:
|
||||
- `make`, `git` and `curl`;
|
||||
- Go, for `gofmt`;
|
||||
- node, yarn and prettier.
|
||||
@@ -321,38 +369,56 @@ nixpkgs fixed at one release. An app's `Dockerfile` installs a package with
|
||||
[`docs/image.md`](docs/image.md) lists what the image holds and how each part is
|
||||
pinned.
|
||||
|
||||
- runit, a service supervisor, runs `smallwebwaf` and the app side by side, each
|
||||
as its own user, and starts either again when it exits. The app cannot touch
|
||||
the state files or the process of `smallwebwaf`.
|
||||
- `smallwebwaf` listens on port 8080 and passes requests to the app on
|
||||
`127.0.0.1:8081`. The app listens on `127.0.0.1` only, so nothing reaches it
|
||||
except through `smallwebwaf`. `smallwebwaf`'s own endpoints are under
|
||||
`/_smallwebwaf/`, on the same port.
|
||||
- The app sees the headers traefik sent, with traefik's address added to
|
||||
`X-Forwarded-For`, from a TCP peer on `127.0.0.1`. So the app must trust
|
||||
`127.0.0.1` and `::1` for forwarded headers, besides the private address
|
||||
ranges.
|
||||
- Both processes start with the container's environment variables, so every
|
||||
setting of `smallwebwaf` starts with `SWWAF_`, and the app can read a token
|
||||
given as one. A token given as a file that only the `smallwebwaf` user can
|
||||
read is out of its reach (see "Settings given as files" in
|
||||
- The container's first process is `runsvinit`, which starts runit, a service
|
||||
supervisor. runit runs the `run` script of each directory under
|
||||
`/etc/service`, and runs it again a second after its program exits.
|
||||
`smallwebwaf` runs as the `smallwebwaf` user (uid 65532) and the app as a user
|
||||
of its own. So the app cannot touch the state files or the process of
|
||||
`smallwebwaf`.
|
||||
- `smallwebwaf` listens on port 8080, on every address. traefik sends the app's
|
||||
requests there. `smallwebwaf` passes them to the app on `127.0.0.1:8081`. The
|
||||
app listens on `127.0.0.1` only, so nothing reaches it except through
|
||||
`smallwebwaf`. The health check, the metrics and the ban management are on the
|
||||
same port, under `/_smallwebwaf/`. A request for that prefix never reaches the
|
||||
app.
|
||||
- The app sees what it would see from traefik directly: the same `Host`, the
|
||||
same `X-Forwarded-Proto`, and `X-Forwarded-For` with traefik's address added
|
||||
at the end. The app's TCP peer is `smallwebwaf` on `127.0.0.1`. So the app
|
||||
must trust `127.0.0.1` and `::1` for forwarded headers, besides the private
|
||||
address ranges. An app that keeps its usual trusted proxies sees every visitor
|
||||
as `127.0.0.1`. Each request also carries its id in `X-Request-ID`. Any
|
||||
`X-Client-ASN` or `X-Client-Country` header a client sent is removed.
|
||||
- Both processes start with the container's environment variables. That is why
|
||||
every setting of `smallwebwaf` starts with `SWWAF_`. A token given as an
|
||||
environment variable is readable by the app. A token given as a file that only
|
||||
the `smallwebwaf` user can read is not (see "Settings given as files" in
|
||||
[`docs/settings.md`](docs/settings.md)).
|
||||
- The state files live in `/var/lib/smallwebwaf`. A volume mounted there keeps
|
||||
them across a deploy.
|
||||
- `docker stop` stops both processes at once. `smallwebwaf` gives the requests
|
||||
in progress five seconds to finish, so the app must keep answering for five
|
||||
- The state files live in `/var/lib/smallwebwaf`. Without a volume there, they
|
||||
survive a restart of the container and are lost when a deploy replaces it. The
|
||||
`run` script gives that directory and every file in it to the `smallwebwaf`
|
||||
user at each start, so a host directory mounted there needs no change of
|
||||
owner.
|
||||
- The image's health check passes while `smallwebwaf` answers
|
||||
`GET /_smallwebwaf/healthz` and the app accepts connections. traefik sends a
|
||||
container no requests until it is healthy.
|
||||
- When `smallwebwaf` or the app exits, runit starts it again a second later. The
|
||||
other process keeps running. While either is down, the client gets `502`. A
|
||||
`smallwebwaf` that stops at start, as on an invalid setting, is started again
|
||||
every second until the cause is fixed, and logs its message each time.
|
||||
- `docker stop` has runit send `SIGTERM` to both processes at once.
|
||||
`smallwebwaf` gives the requests in progress five seconds to finish and writes
|
||||
its state files. To let them finish, the app must keep answering for five
|
||||
seconds after `SIGTERM`.
|
||||
|
||||
"Deployment" in [`SPEC.md`](SPEC.md) has the rest: the health check, what
|
||||
happens when a process stops, moving `smallwebwaf` to another port, and the
|
||||
forwarded headers. "Architecture" in [`SPEC.md`](SPEC.md) lists the packages and
|
||||
what each does.
|
||||
"Deployment" in [`SPEC.md`](SPEC.md) has the rest: moving `smallwebwaf` to
|
||||
another port, the health check's timing, and the forwarded headers.
|
||||
"Architecture" in [`SPEC.md`](SPEC.md) lists the packages and what each does.
|
||||
|
||||
## TODO
|
||||
|
||||
- Planned, as "Planned, not built" in [`SPEC.md`](SPEC.md) describes:
|
||||
- sharing bans between the instances of several apps;
|
||||
- sending the log over RELP.
|
||||
- Everything `SPEC.md` describes is built.
|
||||
- Two points `SPEC.md` defers until there is experience to go on: sharing bans
|
||||
between the instances of several apps, and sending the log over RELP.
|
||||
|
||||
## Documents
|
||||
|
||||
|
||||
+10
-16
@@ -42,7 +42,7 @@ on one line:
|
||||
"asn": "",
|
||||
"as_name": "",
|
||||
"country": "",
|
||||
"reason": "requests per minute over the limit of 6000",
|
||||
"reason": "requests per minute over the limit of 1000",
|
||||
"detail": {
|
||||
"ban_expires": "2026-10-06T13:00:00.123Z",
|
||||
"cause": "limit",
|
||||
@@ -51,9 +51,9 @@ on one line:
|
||||
"as_name": "",
|
||||
"country": "",
|
||||
"kind": "requests",
|
||||
"limit": 6000,
|
||||
"limit": 1000,
|
||||
"window": "minute",
|
||||
"count": 6001,
|
||||
"count": 1001,
|
||||
"request": {
|
||||
"time": "2026-10-06T12:00:00.123456789Z",
|
||||
"method": "GET",
|
||||
@@ -62,7 +62,7 @@ on one line:
|
||||
"status": 403,
|
||||
"user_agent": "scraper/1.0"
|
||||
},
|
||||
"requests": 8210,
|
||||
"requests": 5210,
|
||||
"refused": 0,
|
||||
"earlier_bans": {
|
||||
"limit": 0,
|
||||
@@ -78,18 +78,14 @@ on one line:
|
||||
|
||||
- `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
|
||||
one of the events above.
|
||||
- `client`: the address whose request raised the alert. Empty for
|
||||
`source_failure`, `file_error` and `summary`.
|
||||
- `netblock`: empty for `source_failure`, `file_error` and `summary`; otherwise
|
||||
one of:
|
||||
- the ban's netblock;
|
||||
- the netblock an anomaly counted, empty in the `asn` and `total` scopes;
|
||||
- for the other events, the client as rate limiting counts it: an IPv4
|
||||
address as a /32, or an IPv6 group.
|
||||
- `asn`, `as_name` and `country`: the client's, as known when the alert was
|
||||
- `client`: the address whose request raised the alert. `netblock`: the ban's
|
||||
netblock, the netblock an anomaly counted, or, for the other events, the
|
||||
client as rate limiting counts it: an IPv4 address as a /32, or an IPv6 group.
|
||||
Both are empty for `source_failure` and `file_error`.
|
||||
- `asn`, `as_name` and `country`: the client's as known when the alert was
|
||||
raised. They are empty when GeoJS had not answered yet.
|
||||
- `reason`: a short sentence, such as
|
||||
`requests per minute over the limit of 6000` or `listed by a blocklist`.
|
||||
`requests per minute over the limit of 1000` or `listed by a blocklist`.
|
||||
- `detail`: what is particular to the event:
|
||||
- for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed
|
||||
under `bans.json` in [`state-files.md`](state-files.md));
|
||||
@@ -98,8 +94,6 @@ on one line:
|
||||
- for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`;
|
||||
- for a source failure, the `source` and `error`;
|
||||
- for a file error, the `file` and `error`;
|
||||
- for a summary, the `hour` it covers, the `count` of alerts held back, and
|
||||
that count by event in `events`;
|
||||
- `mode`, when the request was not refused: `detect` or `observe`.
|
||||
- `suppressed_repeats`: how many repeats the cooldown held back before this
|
||||
alert.
|
||||
|
||||
+42
-49
@@ -11,10 +11,8 @@ checked again when gitea or the Core Rule Set changes version.
|
||||
|
||||
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
|
||||
by it.
|
||||
- A clone is a response. It fits the defaults of 30 minutes and 1 GiB unless the
|
||||
repository is large or the link slow. A repository whose clone is larger than
|
||||
1 GiB needs `SWWAF_RESPONSE_MAX_BYTES` raised, and the byte limits with it
|
||||
("Byte limits, per client" in `SPEC.md`).
|
||||
- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all
|
||||
but the largest repositories and slowest links.
|
||||
- A push is a request, and so is every other upload:
|
||||
- a git push;
|
||||
- an LFS object;
|
||||
@@ -23,22 +21,21 @@ checked again when gitea or the Core Rule Set changes version.
|
||||
- a release attachment.
|
||||
|
||||
At the defaults, an upload larger than 100 MiB or taking more than 60
|
||||
seconds is cut off ("Size and time limits" in `SPEC.md`). A gitea that takes
|
||||
large uploads needs `SWWAF_REQUEST_MAX_BYTES`,
|
||||
`SWWAF_CLIENT_REQUEST_TIMEOUT` and `SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised
|
||||
to fit.
|
||||
seconds is cut off, and no one is banned for it. "Size and time limits" in
|
||||
`SPEC.md` gives the answers the client gets. A gitea that takes large
|
||||
uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and
|
||||
`SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit.
|
||||
|
||||
- Archive download and blame or history pages are what scrapers hammer. The
|
||||
request limits do most of the work there.
|
||||
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
|
||||
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
|
||||
43,200 requests a day. While it runs a job, it reports the job's log and state
|
||||
every second instead, two requests a second. So one runner busy all day sends
|
||||
172,800 requests, under the request limits. Six such runners on one address,
|
||||
each busy all day, pass the day limit of 1,000,000 within two days; five stay
|
||||
under it. Their address is then banned, their running jobs fail, and each
|
||||
repeat within a day makes the ban longer. Newer runners ask for work less
|
||||
often. Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||
43,200 requests a day, and reports a running job's log and state every second.
|
||||
So it passes the day limit of 50,000 after about an hour and a quarter of
|
||||
jobs. It is then banned, the job it is running fails, and each repeat within a
|
||||
day makes the ban longer. Newer runners ask less often, but a busy one, or
|
||||
several on one address, can still pass it. Put the runners' addresses in
|
||||
`SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||
|
||||
## What the Core Rule Set lets through
|
||||
|
||||
@@ -66,15 +63,15 @@ toward). The request log names the rule in `waf_rule_ids`, which
|
||||
|
||||
### A query string that reads as an attack
|
||||
|
||||
Most often this is a search. A search term is a query parameter, which the Core
|
||||
Rule Set checks against its lists of system files and commands (see the fourth
|
||||
change under "Attack detection" in `SPEC.md`). A release attachment uploaded
|
||||
through the API as `docker-compose.yml` is refused the same way. Examples:
|
||||
Most often this is a search. Every query parameter is checked against the Core
|
||||
Rule Set's lists of system files and commands, except the ten named under
|
||||
"Attack detection" in `SPEC.md`. A release attachment uploaded through the API
|
||||
as `docker-compose.yml` is refused the same way. Examples:
|
||||
|
||||
- a name on those lists, such as `package.json`, `.gitignore` or
|
||||
`docker-compose.yml`;
|
||||
- text that starts with a command name, such as `python3` or `ssh key`, or has
|
||||
one right after a `/` in its first word, such as `feat/docker-support`;
|
||||
- text that starts with a command name, or whose first word has one right after
|
||||
a `/`, such as `python3`, `ssh key` or `feat/docker-support`;
|
||||
- a shell command with its options or a system path (`ls -la`, `sed -i`,
|
||||
`/bin/sh`), or a command in backticks;
|
||||
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
|
||||
@@ -84,15 +81,15 @@ through the API as `docker-compose.yml` is refused the same way. Examples:
|
||||
### A branch, tag or file name that starts with a command name
|
||||
|
||||
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
|
||||
`python3`, `ansible`, `base64` and `whoami`. A query parameter holding a branch,
|
||||
tag or file name that starts with one is refused. So is one that has such a name
|
||||
right after a `/`, such as `feat/docker-support`, `fix/docker-build` or
|
||||
`python3`, `ansible`, `base64` and `whoami`. A branch, tag or file name that
|
||||
starts with one is refused. So is one that has such a name right after a `/`,
|
||||
such as `feat/docker-support`, `fix/docker-build` or
|
||||
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
|
||||
query parameter that 930120, 932160 and 932260 still check:
|
||||
|
||||
- `refSubUrl`, the branch or tag of a directory listing. A listing sends it when
|
||||
it asks for its entries' last commits in a second request. It does so whenever
|
||||
gitea takes more than a second to work them out;
|
||||
- `refSubUrl`, the branch or tag of a directory listing. It is sent when the
|
||||
listing asks for its entries' last commits in a second request, which it does
|
||||
whenever gitea takes more than a second to work them out;
|
||||
- `name`, when a branch is deleted or restored on the branches page;
|
||||
- `tag`, when a release is started from a tag;
|
||||
- `template`, the file of an issue template, when an issue is opened from it;
|
||||
@@ -113,9 +110,9 @@ What a visitor sees, for a branch named `docker-build`:
|
||||
### A file name in a query parameter that trips the code rules
|
||||
|
||||
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
|
||||
look for in some file names. This holds too in the ten query parameters that
|
||||
930120, 932160 and 932260 do not check (see the fourth change under "Attack
|
||||
detection" in `SPEC.md`). Examples:
|
||||
look for in some file names. This holds in the parameters `smallwebwaf` leaves
|
||||
out of 930120, 932160 and 932260 too (see the changes to the Core Rule Set under
|
||||
"Attack detection" in `SPEC.md`). Examples:
|
||||
|
||||
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
|
||||
(944110);
|
||||
@@ -132,9 +129,9 @@ For such a file, these requests are refused:
|
||||
- the API's call for the contents of files that names it in its query (`body`);
|
||||
- the commit graph of that file alone (`file`);
|
||||
- the hidden lines of its diff (`path`), which stay hidden;
|
||||
- the sign-in page, for a visitor who opens a page of the file that needs
|
||||
signing in, such as its edit page (`redirect_to`). The visitor can still sign
|
||||
in by opening the sign-in page itself.
|
||||
- the sign-in page gitea sends a visitor to on the way to a page of the file
|
||||
that needs signing in, such as its edit page (`redirect_to`). The visitor can
|
||||
still sign in by opening the sign-in page itself.
|
||||
|
||||
### A path
|
||||
|
||||
@@ -146,30 +143,25 @@ For such a file, these requests are refused:
|
||||
`process.env.d.ts` does.
|
||||
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
|
||||
|
||||
For such a file, these are refused:
|
||||
|
||||
- the page of the file;
|
||||
- its raw file;
|
||||
- its history and its blame;
|
||||
- its edit page.
|
||||
|
||||
The directory that holds it lists it, and a clone carries it.
|
||||
For such a file, the page of the file, its raw file, its history, its blame and
|
||||
its edit page are refused. The directory that holds it lists it, and a clone
|
||||
carries it.
|
||||
|
||||
### A container image push that reuses a layer
|
||||
|
||||
A push can reuse a layer of an image docker has pushed to that gitea before.
|
||||
docker then asks gitea to take the layer from that image, naming it in the query
|
||||
parameter `from`, which 932260 checks. The request is refused when the image's
|
||||
owner starts with an entry on the list of commands. It is refused too when the
|
||||
image's name after the owner starts with one. Examples are `owner/python3-app`,
|
||||
`owner/docker-base`, and any image of an owner named `ansible` or `docker-team`.
|
||||
owner starts with an entry on the list of commands. So is one whose name after
|
||||
the owner does. Examples are `owner/python3-app`, `owner/docker-base`, and any
|
||||
image of an owner named `ansible` or `docker-team`.
|
||||
|
||||
What happens next depends on docker's image store. A docker that keeps its
|
||||
images in containerd's image store fails the push, and fails each new try the
|
||||
same way. A new install of Docker Engine 29 does this. A docker that keeps them
|
||||
in its own older image store uploads the layer instead. The push then succeeds
|
||||
after one refusal for each layer it reuses, and each of those refusals counts
|
||||
toward the error burst ("Attack detection" in `SPEC.md`).
|
||||
toward the error burst.
|
||||
|
||||
### Artifact uploads from `actions/upload-artifact@v3`
|
||||
|
||||
@@ -177,9 +169,9 @@ This action sends the header `Content-Range`, which the Core Rule Set refuses
|
||||
(920450). The action sends two files at a time, does not retry a `403`, and
|
||||
sends no further file once one is refused. So an upload is refused once or
|
||||
twice, however many files it holds, and the step fails. One upload does not ban
|
||||
the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
|
||||
than 30 such refusals from its address in a minute, as when many jobs upload at
|
||||
once. `actions/upload-artifact@v4` does not send the header.
|
||||
the runner. More than 30 such refusals from its address within a minute, as when
|
||||
many jobs upload at once, ban it through the error burst.
|
||||
`actions/upload-artifact@v4` does not send the header.
|
||||
|
||||
## What body inspection refuses
|
||||
|
||||
@@ -201,7 +193,8 @@ Of the requests above, the script sends:
|
||||
code rules", the API's list of its commits;
|
||||
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
|
||||
its diff, and the API's call for the contents of files that names it;
|
||||
- the sign-in page for a visitor who opens the edit page of `document.write.js`;
|
||||
- the sign-in page gitea sends a visitor to on the way to the edit page of
|
||||
`document.write.js`;
|
||||
- the pages of `java.lang.String.md`,
|
||||
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
|
||||
and `.../notes.txt`;
|
||||
|
||||
+2
-3
@@ -31,7 +31,6 @@ is pinned that way.
|
||||
- nixpkgs: the release file `nixexprs.tar.xz` of
|
||||
`nixos-26.05.11045.774debe7a0d1` on `releases.nixos.org`, named with its
|
||||
SHA-256 hash.
|
||||
- `runsvinit`: the archived repository, unchanged since 2015, at its last commit
|
||||
(`b4b2c78`, eight commits after `v2.0.0`), by commit hash, with a `go.mod` the
|
||||
build writes.
|
||||
- `runsvinit`: the archived repository, unchanged since 2015, at tag `v2.0.0`,
|
||||
by commit hash, with a `go.mod` the build writes.
|
||||
- The Go image that builds `smallwebwaf`: by digest.
|
||||
+5
-3
@@ -1,8 +1,10 @@
|
||||
# The metrics
|
||||
|
||||
What `GET /_smallwebwaf/metrics` answers with. "Metrics endpoint" in
|
||||
[`SPEC.md`](../SPEC.md) describes the endpoint. Prometheus keeps the `instance`
|
||||
label as `exported_instance` unless the scrape sets `honor_labels: true`.
|
||||
`GET /_smallwebwaf/metrics` answers in the Prometheus text format. The
|
||||
requirements are under "Metrics endpoint" in [`SPEC.md`](../SPEC.md); this file
|
||||
lists the metrics. Every metric carries the label `instance`, set to
|
||||
`SWWAF_INSTANCE_NAME`; Prometheus keeps it as `exported_instance` unless the
|
||||
scrape sets `honor_labels: true`. No metric carries a client's address.
|
||||
|
||||
## Traffic
|
||||
|
||||
|
||||
+12
-23
@@ -12,13 +12,9 @@ sent to a syslog server. A field that does not apply to a request is left out.
|
||||
|
||||
## Every line
|
||||
|
||||
Every line has:
|
||||
|
||||
- `type`: `request` (the process's own messages have `process`);
|
||||
- the standard web log fields;
|
||||
- `request_id`, `peer_ip` and `client_group`;
|
||||
- `asn`, `as_name` and `country`;
|
||||
- `action` and `duration_total`.
|
||||
Every line has `type` (`request`; the process's own messages have `process`),
|
||||
the standard web log fields, `request_id`, `peer_ip`, `client_group`, `asn`,
|
||||
`as_name`, `country`, `action` and `duration_total`.
|
||||
|
||||
## Standard web log fields
|
||||
|
||||
@@ -32,7 +28,7 @@ Every line has:
|
||||
## Request detail
|
||||
|
||||
- `request_id`: the `X-Request-ID` a trusted proxy sent, or a new random one of
|
||||
26 letters and digits.
|
||||
26 letters and digits. It goes to the app in `X-Request-ID`.
|
||||
- `peer_ip`: the TCP peer, normally traefik.
|
||||
- `forwarded_for`: the `X-Forwarded-For` header as received.
|
||||
- `client_group`: the client as rate limiting counts it, an IPv4 address as a
|
||||
@@ -77,22 +73,15 @@ Every line has:
|
||||
rate limits and the setting that lowered it, when one did. `bytes_percent` and
|
||||
`bytes_percent_setting` do the same for the byte limits.
|
||||
- `counts`: the client's totals as rate limiting counts them, this request
|
||||
included. It is left out when rate limiting did not count the request. It
|
||||
holds:
|
||||
included. It is left out when rate limiting did not count the request. A count
|
||||
can have a fraction, since each window is counted in two buckets with the
|
||||
earlier one weighted:
|
||||
- `minute`, `hour` and `day`: requests;
|
||||
- `minute_bytes`, `hour_bytes` and `day_bytes`: bytes.
|
||||
|
||||
A count can have a fraction (see "Counting method" in
|
||||
[`SPEC.md`](../SPEC.md)).
|
||||
|
||||
- `limit_hit`: the limit broken:
|
||||
- `minute`, `hour` or `day` for a rate limit;
|
||||
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
|
||||
- `error_burst`.
|
||||
|
||||
A request whose bytes broke a limit was passed on whole, so its `action`
|
||||
stays `forward`.
|
||||
|
||||
- `limit_hit`: the limit broken: `minute`, `hour` or `day` for a rate limit;
|
||||
`minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or
|
||||
`error_burst`. A request whose bytes broke a limit was passed on whole, so its
|
||||
`action` stays `forward`.
|
||||
- `rule_ids`: the rule file rules that matched, in order, up to the one that
|
||||
refused.
|
||||
- `waf_score` and `waf_rule_ids`: the Core Rule Set's score and the rules that
|
||||
@@ -102,7 +91,7 @@ Every line has:
|
||||
- the CrowdSec decision list's URL;
|
||||
- DNS blocklist zones;
|
||||
- `abuseipdb`.
|
||||
- `offence`: `limit`, for a request that broke a limit.
|
||||
- `offence`: the kind of offence recorded, when one was.
|
||||
- `ban_expires`: when the client's ban ends, for a request that made one or was
|
||||
refused under one: a time, or `permanent`.
|
||||
|
||||
|
||||
+29
-33
@@ -2,7 +2,8 @@
|
||||
|
||||
Every setting, its default and the values it takes, in the groups of "Settings"
|
||||
in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings
|
||||
that take `off`, the `_FILE` form, and what each group does and why.
|
||||
that take `off`, the `_FILE` form, and what each group does and why. A setting
|
||||
set to a value its entry does not allow stops the start.
|
||||
|
||||
## Forms of values
|
||||
|
||||
@@ -26,9 +27,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
|
||||
- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or
|
||||
`https` URL with a host and an optional port, and nothing more.
|
||||
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens: `:` and a
|
||||
port, such as `:9000`. Leave the host out (see "Ports" in
|
||||
[`SPEC.md`](../SPEC.md)).
|
||||
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens, an
|
||||
optional host and a port. The host stays empty.
|
||||
- `SWWAF_INSTANCE_NAME` (default: the container's host name): the `instance`
|
||||
name in every log line, alert and metric, such as `fsn1app1/gitea`. Valid
|
||||
UTF-8. No control character while `SWWAF_ALERT_NTFY_URL` is set. 1 to 48
|
||||
@@ -42,8 +42,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
client's group, from 32 to 128.
|
||||
- `SWWAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a whole
|
||||
number above zero.
|
||||
- `SWWAF_MAX_BANS` (default `5000`): how many of its own bans `smallwebwaf`
|
||||
keeps, a whole number above zero.
|
||||
- `SWWAF_MAX_BANS` (default `5000`): the most bans `smallwebwaf` made that are
|
||||
kept, a whole number above zero.
|
||||
|
||||
## Tokens
|
||||
|
||||
@@ -90,25 +90,24 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
|
||||
## Static lists
|
||||
|
||||
- `SWWAF_ALLOW_NETS` (default empty): netblocks you trust, such as your
|
||||
monitoring (see step 3 in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients are refused only
|
||||
for a size or time limit or a missing or wrong token, such as your monitoring.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
||||
exempt from rate limiting and the byte limits.
|
||||
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are refused (see
|
||||
step 3 in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
|
||||
|
||||
## Request rate limits, per client
|
||||
|
||||
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `6000`), `SWWAF_RATE_LIMIT_PER_HOUR`
|
||||
(default `100000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `1000000`): the
|
||||
most requests a client may make in each window. A count or `off`.
|
||||
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
|
||||
(default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
|
||||
requests a client may make in each window. A count or `off`.
|
||||
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
|
||||
rate limiting and the byte limits, such as `/assets/`.
|
||||
|
||||
## Byte limits, per client
|
||||
|
||||
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `4G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
|
||||
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `200G`): the most
|
||||
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
|
||||
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most
|
||||
bytes counted for a client in each window. A size or `off`.
|
||||
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
|
||||
`response`, `request` or `both`.
|
||||
@@ -131,18 +130,12 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
to send its whole answer. A duration or `off`.
|
||||
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size
|
||||
or `off`.
|
||||
- `SWWAF_RESPONSE_MAX_BYTES` (default `1G`): the largest response body. A size
|
||||
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size
|
||||
or `off`.
|
||||
|
||||
## Lookup of AS number and country
|
||||
|
||||
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off` stops
|
||||
the start while a setting that needs a lookup is set:
|
||||
- the two country lists;
|
||||
- `SWWAF_ADD_LOOKUP_HEADERS`;
|
||||
- the four biased threshold lists and `SWWAF_ASN_LIMIT_PERCENT_URL`;
|
||||
- `SWWAF_UNKNOWN_LIMIT_PERCENT` below `100`;
|
||||
- an anomaly threshold per AS number.
|
||||
- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`.
|
||||
- `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
|
||||
`.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`.
|
||||
- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
|
||||
@@ -157,7 +150,7 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
such as `cn,ru,kp`.
|
||||
- `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
|
||||
countries whose clients get through, such as `us,de`. A country on both lists
|
||||
stops the start.
|
||||
is refused.
|
||||
|
||||
## Biased thresholds
|
||||
|
||||
@@ -171,8 +164,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
- `SWWAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage for a client the
|
||||
lookup cannot place.
|
||||
- `SWWAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
|
||||
text file with one AS number and percentage on each line, such as
|
||||
`AS64496:50`. A URL that is also in `SWWAF_BLOCKLIST_URLS` is refused.
|
||||
text file of `AS<number>:<percent>` lines, one to a line. A URL that is also
|
||||
in `SWWAF_BLOCKLIST_URLS` is refused.
|
||||
|
||||
## Attack detection
|
||||
|
||||
@@ -196,7 +189,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off`, or how much of a request body
|
||||
the Core Rule Set reads, a size of at most `1G`.
|
||||
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
|
||||
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
|
||||
in a minute for a rule match, a trap path, a Core Rule Set match or a failed
|
||||
token check. A count or `off`.
|
||||
|
||||
## Ban lengths, response and scope
|
||||
|
||||
@@ -266,8 +260,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
`ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure,file_error`):
|
||||
the event types alerts are sent for, from those in [`alerts.md`](alerts.md)
|
||||
but `summary`. Any other name is refused.
|
||||
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long after an alert is sent its
|
||||
repeats are held back. A duration or `off`.
|
||||
- `SWWAF_ALERT_COOLDOWN` (default `15m`): how long a repeat of an alert is held
|
||||
back. A duration or `off`.
|
||||
- `SWWAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. A
|
||||
whole number above zero, or `off`.
|
||||
|
||||
@@ -291,7 +285,7 @@ as `REQUESTS_PER_MINUTE`:
|
||||
client;
|
||||
- `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
||||
client.
|
||||
- `SWWAF_ANOMALY_ASN_...`: per AS number.
|
||||
- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs a lookup source.
|
||||
- `SWWAF_ANOMALY_TOTAL_...`: the whole service.
|
||||
- `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole.
|
||||
`SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
|
||||
@@ -300,8 +294,10 @@ as `REQUESTS_PER_MINUTE`:
|
||||
|
||||
## Settings given as files
|
||||
|
||||
To give a token as a file, as "Tokens" under "Deployment" in
|
||||
[`SPEC.md`](../SPEC.md) describes, make the file on the host:
|
||||
Add `_FILE` to a setting's name to read its value from a file:
|
||||
`SWWAF_METRICS_TOKEN_FILE` names the file that holds `SWWAF_METRICS_TOKEN`. To
|
||||
keep a token from the app, make the file on the host, owned by uid 65532 with
|
||||
mode `0400`:
|
||||
|
||||
```sh
|
||||
mkdir -p /srv/app/tokens
|
||||
@@ -310,7 +306,7 @@ chown 65532:65532 /srv/app/tokens/metrics
|
||||
chmod 0400 /srv/app/tokens/metrics
|
||||
```
|
||||
|
||||
Then mount its directory into the container read-only, and name the file:
|
||||
Mount its directory into the container read-only, and name the file:
|
||||
|
||||
```sh
|
||||
docker run \
|
||||
|
||||
+15
-19
@@ -15,8 +15,8 @@ those `smallwebwaf` made. Per entry:
|
||||
- `lifted`: when an admin lifted it, if one did;
|
||||
- `notes`: the notes below.
|
||||
|
||||
An entry an admin adds needs `netblock`, `start` and `expires`; `reason`,
|
||||
`notes` and `cause` may be left out, and the cause is then `admin`:
|
||||
To ban a netblock, add an entry with its `netblock`, `start` and `expires`.
|
||||
`reason`, `notes` and `cause` may be left out; the cause is then `admin`:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -32,23 +32,20 @@ An entry an admin adds needs `netblock`, `start` and `expires`; `reason`,
|
||||
}
|
||||
```
|
||||
|
||||
"Bans" in [`SPEC.md`](../SPEC.md) says how an edit lifts, keeps or forgets a
|
||||
ban. The notes:
|
||||
To lift a ban, add `"lifted"` with the time you lift it, such as
|
||||
`"lifted": "2026-10-06T13:00:00Z"`. To keep a ban `smallwebwaf` made, set its
|
||||
`cause` to `admin`. To forget a ban, delete its entry.
|
||||
|
||||
The notes:
|
||||
|
||||
- `asn`, `as_name` and `country`: the client's, added when the lookup answers
|
||||
(empty when lookups are off);
|
||||
- for a broken limit, `kind` (`requests`, `bytes` or `refusals`), `limit`,
|
||||
`window` and `count`: what the limit was on, the limit, its window and the
|
||||
count reached;
|
||||
- `limit_percent` and `limit_percent_setting`: the client's percentage of that
|
||||
limit and the setting that gave it, when a setting lowered it;
|
||||
- for a clear sign of attack, `rule_id` and `target`, or `trap_path`;
|
||||
- `reputation`: the sources the client was on, each with its `source` and, for
|
||||
AbuseIPDB, its `score`;
|
||||
- `request`: the request that caused the ban, each text cut to 256 bytes:
|
||||
- `time`, `method` and `host`;
|
||||
- `path`, with its query string;
|
||||
- `status` and `user_agent`;
|
||||
- what was broken: the rule id and target that matched, or the trap path, or the
|
||||
limit, its window and the count reached;
|
||||
- the client's percentage of the limit, with the setting that gave it;
|
||||
- the reputation sources the client is on;
|
||||
- `request`: the request that caused the ban: time, method, host, path with its
|
||||
query string, status and user agent, each text cut to 256 bytes;
|
||||
- `requests` and `refused`: the netblock's requests since it was first seen, and
|
||||
how many this ban has refused so far, kept up to date while the ban lasts;
|
||||
- `earlier_bans`: how many earlier bans the netblock had, by cause.
|
||||
@@ -96,9 +93,8 @@ GeoJS answers, one per line:
|
||||
`SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of
|
||||
disk writes a day.
|
||||
- `bans.json`: about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at
|
||||
the default `SWWAF_MAX_BANS`. Written every 15 minutes, a full file comes to
|
||||
about 600 MiB of writes a day, and a hundred new bans a day add about 600 MiB
|
||||
more.
|
||||
the default `SWWAF_MAX_BANS`. With a full file, a hundred new bans a day come
|
||||
to about 600 MiB of writes.
|
||||
- `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
|
||||
full.
|
||||
- `reputation.json` and `alerts.json`: usually a few MiB or less.
|
||||
@@ -1,8 +1,8 @@
|
||||
// Package alerts sends alerts on bans, on traffic over an anomaly
|
||||
// threshold, on a source that fails and on a file with an error to each
|
||||
// destination set: to the webhook
|
||||
// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON
|
||||
// object" in docs/alerts.md describes, to the Slack incoming
|
||||
// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as the "Alert
|
||||
// webhook schema" section of SPEC.md describes, to the Slack incoming
|
||||
// webhook SWWAF_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
|
||||
// ntfy topic SWWAF_ALERT_NTFY_URL names. A repeat within
|
||||
// SWWAF_ALERT_COOLDOWN is held back, and so is an alert past
|
||||
@@ -149,10 +149,11 @@ type Params struct {
|
||||
}
|
||||
|
||||
// Alert is one alert, as the webhook is sent it and alerts.json holds it,
|
||||
// with the fields of "The JSON object" in docs/alerts.md. ASN, ASName and
|
||||
// Country are, for a ban, the client's as the ban's notes give them.
|
||||
// with the fields of the "Alert webhook schema" section of SPEC.md. ASN,
|
||||
// ASName and Country are, for a ban, the client's as the ban's notes give
|
||||
// them.
|
||||
//
|
||||
//nolint:tagliatelle // docs/alerts.md names the alert's fields in snake_case
|
||||
//nolint:tagliatelle // SPEC.md's alert webhook schema names its fields in snake_case
|
||||
type Alert struct {
|
||||
Instance string `json:"instance"`
|
||||
Time time.Time `json:"time"`
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
// Package anomaly counts requests and bytes over a minute and an hour, per
|
||||
// client, per surrounding netblock, per AS number, for the whole service
|
||||
// and per named netblock, and raises an anomaly alert for a count over its
|
||||
// threshold, as "Anomaly thresholds" under "Settings" in SPEC.md
|
||||
// describes. It refuses and bans nothing. At most 20,000 counters
|
||||
// threshold, as "Anomaly thresholds" under "Configuration surface" in
|
||||
// SPEC.md describes. It refuses and bans nothing. At most 20,000 counters
|
||||
// are kept, in memory, and written to alerts.json and read from it by the
|
||||
// state package.
|
||||
package anomaly
|
||||
|
||||
@@ -502,17 +502,17 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
|
||||
UpstreamRequestTimeout: env.duration("SWWAF_UPSTREAM_REQUEST_TIMEOUT", "60s"),
|
||||
UpstreamResponseTimeout: env.duration("SWWAF_UPSTREAM_RESPONSE_TIMEOUT", "30m"),
|
||||
RequestMaxBytes: env.size("SWWAF_REQUEST_MAX_BYTES", "100M"),
|
||||
ResponseMaxBytes: env.size("SWWAF_RESPONSE_MAX_BYTES", "1G"),
|
||||
ResponseMaxBytes: env.size("SWWAF_RESPONSE_MAX_BYTES", "5G"),
|
||||
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
|
||||
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
|
||||
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
|
||||
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "6000"),
|
||||
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "100000"),
|
||||
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "1000000"),
|
||||
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "1000"),
|
||||
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "10000"),
|
||||
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "50000"),
|
||||
RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
|
||||
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "4G"),
|
||||
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "10G"),
|
||||
BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "20G"),
|
||||
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "200G"),
|
||||
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "50G"),
|
||||
BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
|
||||
LookupSource: env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
|
||||
LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
|
||||
|
||||
@@ -261,10 +261,10 @@ func TestDefaults(t *testing.T) {
|
||||
UpstreamRequestTimeout: time.Minute,
|
||||
UpstreamResponseTimeout: 30 * time.Minute,
|
||||
RequestMaxBytes: 100 << 20,
|
||||
ResponseMaxBytes: 1 << 30,
|
||||
RateLimitPerMinute: 6000,
|
||||
RateLimitPerHour: 100000,
|
||||
RateLimitPerDay: 1000000,
|
||||
ResponseMaxBytes: 5 << 30,
|
||||
RateLimitPerMinute: 1000,
|
||||
RateLimitPerHour: 10000,
|
||||
RateLimitPerDay: 50000,
|
||||
BanResponse: 403,
|
||||
LimitBanDuration: time.Hour,
|
||||
LimitBanRepeatWindow: 24 * time.Hour,
|
||||
@@ -284,8 +284,8 @@ func TestDefaults(t *testing.T) {
|
||||
LookupSource: defaultLookupSource, LookupTimeout: time.Second,
|
||||
})
|
||||
wantByteLimitSettings(t, cfg, config.Config{
|
||||
BytesLimitPerMinute: 4 << 30, BytesLimitPerHour: 20 << 30,
|
||||
BytesLimitPerDay: 200 << 30, BytesCount: "both",
|
||||
BytesLimitPerMinute: 10 << 30, BytesLimitPerHour: 20 << 30,
|
||||
BytesLimitPerDay: 50 << 30, BytesCount: "both",
|
||||
})
|
||||
|
||||
if cfg.UpstreamURL.String() != "http://127.0.0.1:8081" {
|
||||
@@ -2371,17 +2371,17 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
upstreamRequestTimeout: "60s",
|
||||
upstreamResponseTimeout: "30m",
|
||||
requestMaxBytes: "100M",
|
||||
responseMaxBytes: "1G",
|
||||
responseMaxBytes: "5G",
|
||||
allowNets: "",
|
||||
rateLimitExemptNets: "",
|
||||
denyNets: "",
|
||||
rateLimitPerMinute: "6000",
|
||||
rateLimitPerHour: "100000",
|
||||
rateLimitPerDay: "1000000",
|
||||
rateLimitPerMinute: "1000",
|
||||
rateLimitPerHour: "10000",
|
||||
rateLimitPerDay: "50000",
|
||||
rateLimitExemptPaths: "",
|
||||
bytesLimitPerMinute: "4G",
|
||||
bytesLimitPerMinute: "10G",
|
||||
bytesLimitPerHour: "20G",
|
||||
bytesLimitPerDay: "200G",
|
||||
bytesLimitPerDay: "50G",
|
||||
bytesCount: "both",
|
||||
lookupSource: defaultLookupSource,
|
||||
lookupDBPath: "",
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
// Package metrics keeps smallwebwaf's Prometheus metrics, as
|
||||
// docs/metrics.md lists them, and serves them in the Prometheus text
|
||||
// format, as the "Metrics endpoint" section of SPEC.md requires. No
|
||||
// metric carries a client's address.
|
||||
// Package metrics keeps smallwebwaf's Prometheus metrics, as the "Metrics
|
||||
// endpoint" section of SPEC.md lists them, and serves them in the
|
||||
// Prometheus text format. No metric carries a client's address.
|
||||
package metrics
|
||||
|
||||
import (
|
||||
|
||||
@@ -206,7 +206,7 @@ type Hit struct {
|
||||
// Counts are a client's requests and bytes in the minute, the hour and
|
||||
// the day that end at a request, that request's included.
|
||||
//
|
||||
//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
|
||||
//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
|
||||
type Counts struct {
|
||||
Minute float64 `json:"minute"`
|
||||
Hour float64 `json:"hour"`
|
||||
|
||||
@@ -64,10 +64,11 @@ const LimitHitErrorBurst = "error_burst"
|
||||
const timeLayout = "2006-01-02T15:04:05.000Z07:00"
|
||||
|
||||
// Line is one request's line in the request log. The field names, and
|
||||
// their order, are those of docs/request-log.md. A field that may not
|
||||
// apply to a request is left out of its line when it does not.
|
||||
// their order, are those of the "Request log" section of SPEC.md. A field
|
||||
// that may not apply to a request is left out of its line when it does
|
||||
// not.
|
||||
//
|
||||
//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
|
||||
//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
|
||||
type Line struct {
|
||||
Type string `json:"type"`
|
||||
|
||||
|
||||
@@ -60,19 +60,8 @@ func TestHealthCheck(t *testing.T) {
|
||||
}, 0, "")
|
||||
}
|
||||
|
||||
// Another test can take a closed server's port at once, so the app's
|
||||
// address here is a multicast address, to which TCP cannot connect.
|
||||
wantHealthCheck(t, map[string]string{
|
||||
listenAddr: ":" + port,
|
||||
upstreamURL: "http://224.0.0.1",
|
||||
}, 1, "unhealthy: connect to the app: ")
|
||||
|
||||
// Listening on port 0 takes any free port, so nothing listens on port 0
|
||||
// itself.
|
||||
wantHealthCheck(t, map[string]string{listenAddr: ":0"}, 1,
|
||||
"unhealthy: ask smallwebwaf: ")
|
||||
wantHealthCheck(t, map[string]string{listenAddr: "8080"}, 1,
|
||||
"unhealthy: invalid setting: SWWAF_LISTEN_ADDR: ")
|
||||
app.Close()
|
||||
wantHealthCheck(t, env, 1, "unhealthy: connect to the app: ")
|
||||
|
||||
stop()
|
||||
|
||||
@@ -81,6 +70,10 @@ func TestHealthCheck(t *testing.T) {
|
||||
case <-time.After(waitLimit):
|
||||
t.Fatal("still running after being told to stop")
|
||||
}
|
||||
|
||||
wantHealthCheck(t, env, 1, "unhealthy: ask smallwebwaf: ")
|
||||
wantHealthCheck(t, map[string]string{listenAddr: "8080"}, 1,
|
||||
"unhealthy: invalid setting: SWWAF_LISTEN_ADDR: ")
|
||||
}
|
||||
|
||||
func TestHealthCheckRefusesAnArgument(t *testing.T) {
|
||||
|
||||
@@ -1240,13 +1240,13 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL, dir string) {
|
||||
"SWWAF_UPSTREAM_REQUEST_TIMEOUT": "60s",
|
||||
"SWWAF_UPSTREAM_RESPONSE_TIMEOUT": "30m",
|
||||
"SWWAF_REQUEST_MAX_BYTES": "100M",
|
||||
"SWWAF_RESPONSE_MAX_BYTES": "1G",
|
||||
"SWWAF_RESPONSE_MAX_BYTES": "5G",
|
||||
"SWWAF_ALLOW_NETS": "",
|
||||
rateLimitExemptNets: "",
|
||||
"SWWAF_DENY_NETS": "",
|
||||
"SWWAF_RATE_LIMIT_PER_MINUTE": "6000",
|
||||
"SWWAF_RATE_LIMIT_PER_HOUR": "100000",
|
||||
rateLimitPerDay: "1000000",
|
||||
"SWWAF_RATE_LIMIT_PER_MINUTE": "1000",
|
||||
"SWWAF_RATE_LIMIT_PER_HOUR": "10000",
|
||||
rateLimitPerDay: "50000",
|
||||
"SWWAF_RATE_LIMIT_EXEMPT_PATHS": "",
|
||||
"SWWAF_DENIED_COUNTRIES": "",
|
||||
"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
// Package state keeps smallwebwaf's state in JSON files in
|
||||
// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md requires
|
||||
// and docs/state-files.md lists them:
|
||||
// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md describes:
|
||||
// bans.json holds the bans, clients.json each client's counters and
|
||||
// history, lookups.json GeoJS's answers, reputation.json the last try and
|
||||
// last good copy of each list fetched from a URL, the DNSBL zones'
|
||||
|
||||
@@ -1019,8 +1019,8 @@ func TestEditJustBeforeAScheduledWriteSurvivesIt(t *testing.T) {
|
||||
edit(t, dir, bansJSON, permanentBansJSON)
|
||||
|
||||
// The write takes the edit in first, and writes it back. The second
|
||||
// ban, made after the admin opened the file, is lost, as the
|
||||
// "Persistent state" section of SPEC.md says.
|
||||
// ban, made after the admin opened the file, is lost, as "Edits
|
||||
// while running" in SPEC.md says.
|
||||
time.Sleep(time.Nanosecond)
|
||||
synctest.Wait()
|
||||
|
||||
|
||||
+2
-2
@@ -1,8 +1,8 @@
|
||||
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||
// method, the URL with its query and the headers of a request, and on its
|
||||
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
||||
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
|
||||
// them. It reads no response.
|
||||
// makes to it, as "Attack detection" under "Configuration surface" in
|
||||
// SPEC.md describes them. It reads no response.
|
||||
//
|
||||
// smallwebwaf writes only to its state directory, so Coraza is built with
|
||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||
|
||||
@@ -2,12 +2,11 @@
|
||||
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
|
||||
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
|
||||
# which names gitea's container. Then send through it the gitea requests
|
||||
# that reading gitea's source found the Core Rule Set refuses, which
|
||||
# docs/gitea.md records, and a short sample of gitea's ordinary use:
|
||||
# signing in, pages of files and commits, a diff, the API, a push, clone
|
||||
# and fetch over HTTP, and two container image pushes. Last, print each
|
||||
# request smallwebwaf logged, with what it did with it and the ids of the
|
||||
# Core Rule Set's rules that matched it.
|
||||
# that reading gitea's source found the Core Rule Set refuses, and a short
|
||||
# sample of gitea's ordinary use: signing in, pages of files and commits,
|
||||
# a diff, the API, a push, clone and fetch over HTTP, and two container
|
||||
# image pushes. Last, print each request smallwebwaf logged, with what it
|
||||
# did with it and the ids of the Core Rule Set's rules that matched it.
|
||||
# Every request comes from one client, docker's gateway, so more than
|
||||
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
|
||||
# the requests after would show as banned. The containers, the network,
|
||||
|
||||
Reference in new issue
Block a user