sneak/smallwebwaf
1
0 Fork 0
Code Issues 18 Pull Requests 5 Actions 87 Packages Projects Releases Wiki Activity

Rename the settings whose names do not say what they control (closes #234)
check / check (push) Waiting to run
Details

Browse Source 
Applies the setting renames of the naming audit under the SWW_WAF_ and
SWW_RPROXY_ prefixes, with sneak's names for the IP reputation cache
time and the two time limits, which now take milliseconds. The rproxy
log field default_disposition becomes waf_backend_no_answer, and
config.Config and config.FromEnvironment become config.WAF and
config.WAFFromEnvironment. Go fields, test names and prose follow. No
aliases; nothing reads the old names.

Model: opus-5-5
This commit is contained in:
clawbot committed 2026-11-12 09:08:49 +00:00
1 parent 101cc0aba5
commit a18926ec4d
100 files changed
+2181 -1935
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

SPEC.md
+157 -140
View File
@@ -157,12 +157,13 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
`prefix=` tags while `SWW_WAF_ROUTEWATCH_URL` is set. A client in
`SWW_WAF_ALLOW_NETS` skips the header limits and gets no tag.
4. The static lists. A client in `SWW_WAF_ALLOW_NETS` skips steps 5 to 15, the
byte limits and the error burst. A client in `SWW_WAF_DENY_NETS` is refused
byte limits and the refusal limit. A client in `SWW_WAF_DENY_NETS` is refused
(`deny_nets`); a client in both lists is let through.
5. Bans. A client under a ban is refused (`active_ban`; see "Bans").
6. The lookup, unless `SWW_WAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
and country"). The request goes on at once. It waits for the first GeoJS
answer, up to `SWW_WAF_LOOKUP_TIMEOUT`, only when one of these needs it:
answer, up to `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, only when one of
these needs it:
- the country list;
- a biased threshold;
- `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming an `asn=` or
@@ -176,10 +177,10 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
- A client on a fetched blocklist gets `SWW_WAF_BLOCKLIST_ACTION`
(`blocklists`).
- A client on the CrowdSec decision list is banned (`crowdsec`).
- A client listed by a DNS blocklist gets `SWW_WAF_REPUTATION_ACTION`
- A client listed by a DNS blocklist gets `SWW_WAF_DNSBL_ABUSEIPDB_ACTION`
(`dnsbl`).
- A client scored at least `SWW_WAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB gets
`SWW_WAF_REPUTATION_ACTION` (`abuseipdb`).
`SWW_WAF_DNSBL_ABUSEIPDB_ACTION` (`abuseipdb`).
9. The tag refusals (`tag_deny`; see "Request tags"). A request carrying a tag
in `SWW_WAF_TAG_DENY` is refused.
10. The client's limit percentage (see "Biased thresholds").
@@ -204,17 +205,17 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
18. After the visitor's answer has ended, `rproxy` sends the report (see "The
report"). `waf` then counts:
- For a request it allowed, the body bytes of the response, of the request
or of both are added to the byte counters, as `SWW_WAF_BYTES_COUNT` says.
For a WebSocket, what it carried each way counts once it closes. A client
whose total passes a byte limit times the percentage has broken that limit
and is banned (`byte_limits`). The request that broke it has already been
passed on whole; nothing is cut off.
or of both are added to the byte counters, as `SWW_WAF_BYTES_COUNTED`
says. For a WebSocket, what it carried each way counts once it closes. A
client whose total passes a byte limit times the percentage has broken
that limit and is banned (`byte_limits`). The request that broke it has
already been passed on whole; nothing is cut off.
- For a request it allowed, the client's history and the anomaly counters
are updated, and the alert thresholds are evaluated.
For a request it refused, `waf` adds nothing to the byte limits. It updates
the history and the anomaly counters at its decision. It counts the refusals
the error burst counts at the decision too (`refusal_limit`; see "Attack
the refusal limit counts at the decision too (`refusal_limit`; see "Attack
detection").
In `SWW_WAF_MODE=observe`, every refusal above is logged as what would have
@@ -279,7 +280,7 @@ to a ban, each by its own rule:
- a request over a rate limit;
- the bytes counted for a request and its response taking the client's total
past a byte limit;
- more than `SWW_WAF_ERROR_BURST_THRESHOLD` refusals within a minute.
- more than `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE` refusals within a minute.
The bans for a broken limit:
- The first such ban lasts `SWW_WAF_LIMIT_BAN_DURATION` (default `1h`), with
@@ -288,15 +289,16 @@ to a ban, each by its own rule:
ended.
- Breaking a limit again within that window makes the new ban three times as
long as the last one: one hour, then 3, 9, 27 and 81 hours.
- A ban that would be longer than `SWW_WAF_MAX_BAN_DURATION` (default `7d`)
is permanent instead. At the defaults, that is the sixth ban in a row.
- A ban that would be longer than `SWW_WAF_LIMIT_BAN_PERMANENT_ABOVE`
(default `7d`) is permanent instead. At the defaults, that is the sixth
ban in a row.
- Each such ban sets the client's counters for every limit back to zero.
Once it ends, only new requests can break a limit again. The client's
history keeps its totals.
A Core Rule Set match refuses only the request it matched, with `403` and
without a ban, because the Core Rule Set has false positives. A `block` rule
does the same. Those refusals count toward the error burst, so a client that
does the same. Those refusals count toward the refusal limit, so a client that
keeps setting them off is banned for a broken limit.
A client on the CrowdSec decision list is banned until the decision ends, with
@@ -350,8 +352,9 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
- These settings take `off`, which switches them off:
- `SWW_WAF_LOOKUP_SOURCE` and `SWW_WAF_CORE_RULE_SET_MODE`;
- the three rate limits and the three byte limits;
- `SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD`,
`SWW_WAF_CORE_RULE_SET_BODY_LIMIT` and `SWW_WAF_ERROR_BURST_THRESHOLD`;
- `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`,
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` and
`SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`;
- `SWW_WAF_ALERT_COOLDOWN` and `SWW_WAF_ALERT_MAX_PER_HOUR`;
- every anomaly threshold;
- `SWW_RPROXY_MAX_REQUEST_SIZE`, `SWW_RPROXY_MAX_RESPONSE_SIZE`,
@@ -404,9 +407,9 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
- `SWW_WAF_STATE_DIR` names the directory of the state files, and
`SWW_WAF_CACHE_DIR` that of the cache files (see "State" under "Deployment").
- `bans.json` is written `SWW_WAF_STATE_WRITE_DELAY` after a ban is made, lifted
- `bans.json` is written `SWW_WAF_BANS_WRITE_DELAY` after a ban is made, lifted
or made permanent, so a burst of changes becomes one write. Every file is
written every `SWW_WAF_STATE_COUNTER_INTERVAL`, and at an orderly stop.
written every `SWW_WAF_STATE_WRITE_INTERVAL`, and at an orderly stop.
### Logging
@@ -420,15 +423,16 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
### Metrics
- `SWW_WAF_METRICS_TOP_N` is how many of the busiest AS numbers and countries
get a series of their own. The rest are summed as `other`, so the label set
stays bounded.
- `SWW_WAF_METRICS_BUSIEST_COUNTRIES_AND_ASNS` is how many of the busiest AS
numbers and countries get a series of their own. The rest are summed as
`other`, so the label set stays bounded.
### Static lists
- `SWW_WAF_ALLOW_NETS` skips the checks step 4 names, for monitoring or the
owner's own networks. `SWW_WAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request
and byte limits only; the error burst, attack detection and bans still apply.
and byte limits only; the refusal limit, attack detection and bans still
apply.
- Many people can share one address, as in an office, on a mobile carrier or
through Tor. They are one client, and share its limits and its bans. A known
shared address busier than the limits allow belongs in `SWW_WAF_ALLOW_NETS` or
@@ -474,7 +478,7 @@ app may read it as a path outside the prefix.
### Byte limits, per client
The byte limits count body bytes: the app's response bodies, the visitors'
request bodies, or both, as `SWW_WAF_BYTES_COUNT` says. Headers are never
request bodies, or both, as `SWW_WAF_BYTES_COUNTED` says. Headers are never
counted. What the visitor sent over a WebSocket counts as request bytes, and
what the app sent as response bytes. The bytes counted for a request and its
response are added to the client's total once the response has ended (see step
@@ -510,11 +514,11 @@ service that keeps its visitors' addresses on its own host.
are kept apart from the table of clients, so that an answer outlasts a client
dropped from that table. The answer reaches the client's history and ban notes
when it comes.
- `SWW_WAF_LOOKUP_TIMEOUT` is how long a visitor's request waits for its
client's first answer, when a setting needs it (see step 6). It also bounds
each request to GeoJS. A client whose answer has not come in time counts as
unknown until it comes, and its later requests do not wait. Its address is
asked about again in the background.
- `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` is how long a visitor's request
waits for its client's first answer, when a setting needs it (see step 6). It
also bounds each request to GeoJS. A client whose answer has not come in time
counts as unknown until it comes, and its later requests do not wait. Its
address is asked about again in the background.
- A client the lookup cannot place has an unknown country and AS number, which
the country list and the biased thresholds treat as described there. Nothing
else treats it differently.
@@ -576,7 +580,7 @@ Each tag and where it comes from:
key masked, as the `reputation` field gives it.
- `exploitsignature`, when a rule file or the Core Rule Set matched the request.
For the Core Rule Set a match is a score at or over
`SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD`.
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`.
- `asn=<number>` and `prefix=<netblock>` from a routewatch instance
(`routewatch_tags`), while `SWW_WAF_ROUTEWATCH_URL` is set:
- The tag `prefix=` holds the netblock the client's address is announced in,
@@ -623,13 +627,13 @@ abuse comes from.
- `SWW_WAF_TAG_LIMIT_PERCENT` gives a tag a percentage, such as `asn=14061:50`
(see "Request tags"). A lowered limit is rounded down.
- `SWW_WAF_UNKNOWN_LIMIT_PERCENT` at `0` is a zero allowance: the client's first
request breaks a limit and bans it.
- `SWW_WAF_ASN_BYTES_PERCENT` and `SWW_WAF_COUNTRY_BYTES_PERCENT` set the byte
limits' percentage alone, when it should differ from the request limits'. For
an AS number they list, they replace its `asn=` tag's percentage and its entry
in `SWW_WAF_ASN_LIMIT_PERCENT_URL`; for a country, its `country=` tag's
percentage.
- `SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT` at `0` is a zero allowance: the
client's first request breaks a limit and bans it.
- `SWW_WAF_ASN_BYTES_LIMIT_PERCENT` and `SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT`
set the byte limits' percentage alone, when it should differ from the request
limits'. For an AS number they list, they replace its `asn=` tag's percentage
and its entry in `SWW_WAF_ASN_LIMIT_PERCENT_URL`; for a country, its
`country=` tag's percentage.
- `SWW_WAF_ASN_LIMIT_PERCENT_URL` names a text file of `AS:percent` lines, so
one abuse-source list can be shared by every `waf` in the fleet. It is fetched
and kept like a blocklist (see "Reputation"). Each entry counts like an `asn=`
@@ -637,8 +641,8 @@ abuse comes from.
applies.
- A client's limit percentage is the lowest of those that apply to it: its AS
number's, its country's, its tags', or a reputation hit's `limit:<percent>`. A
client without a country gets `SWW_WAF_UNKNOWN_LIMIT_PERCENT` in place of a
country's. The percentage is 100 if none applies. Clients given a lower
client without a country gets `SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT` in place
of a country's. The percentage is 100 if none applies. Clients given a lower
percentage are banned after fewer requests, first for a while and then, for
repeated abuse, permanently. The log line of each counted request, and the
notes of each ban, say which setting lowered the limit.
@@ -660,16 +664,16 @@ files, the limits and the bans still apply to the client that sends it.
- The Core Rule Set skips a path under `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS`.
- `SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL` and
`SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD` are the Core Rule Set's own two
tuning values, at its own defaults. Each level up runs more rules, which find
more attacks and take more ordinary requests for attacks. Each rule that
matches adds its score to the request's, and a rule rated critical adds 5. A
request whose score reaches the threshold is a match, so by default one
critical rule is enough. With the threshold `off`, nothing is a match, and
scores are still logged. A request with more than 1000 query parameters or
body fields adds 5 to its score, since Coraza reads only the first 1000. For a
match, in `block` and `detect` mode alike, the log line gives the rule ids and
the score, and a `core_rule_set_match` alert is sent.
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD` are the Core Rule Set's own two tuning
values, at its own defaults. Each level up runs more rules, which find more
attacks and take more ordinary requests for attacks. Each rule that matches
adds its score to the request's, and a rule rated critical adds 5. A request
whose score reaches the threshold is a match, so by default one critical rule
is enough. With the threshold `off`, nothing is a match, and scores are still
logged. A request with more than 1000 query parameters or body fields adds 5
to its score, since Coraza reads only the first 1000. For a match, in `block`
and `detect` mode alike, the log line gives the rule ids and the score, and a
`core_rule_set_match` alert is sent.
- `SWW_WAF_CORE_RULE_SET_DISABLED_RULES` switches off rule ids when an app trips
a false positive. The default switches off six rules:
- 920340, 920420 and 920640 refuse a request for the type of its body, when
@@ -707,14 +711,14 @@ files, the limits and the bans still apply to the client that sends it.
loads, such as `SecRuleRemoveById`, since none of the Core Rule Set's
rules is loaded yet.
- `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` switches body inspection on, up to that
size. It takes at most `64K`, since `waf` gets no more of a body than that
(see "The answer"). Form data and multipart bodies are read up to the size.
JSON and XML bodies are read only when no larger than the size, since those
cannot be read in part. Any other body is not read: the Core Rule Set would
read it as form data, where binary content such as a git push trips rules
written for text. A body the Core Rule Set cannot parse adds 5 to the score,
and so does a multipart body cut by the limit in the middle of a part's
- `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` switches body inspection on, up
to that size. It takes at most `64K`, since `waf` gets no more of a body than
that (see "The answer"). Form data and multipart bodies are read up to the
size. JSON and XML bodies are read only when no larger than the size, since
those cannot be read in part. Any other body is not read: the Core Rule Set
would read it as form data, where binary content such as a git push trips
rules written for text. A body the Core Rule Set cannot parse adds 5 to the
score, and so does a multipart body cut by the limit in the middle of a part's
header. Body inspection suits apps whose forms carry no code.
- `SWW_WAF_TRAP_PATHS` names paths the app never serves and only scanners ask
for, such as `/wp-login.php` in front of an app that is not WordPress. This is
@@ -723,8 +727,8 @@ files, the limits and the bans still apply to the client that sends it.
the path as sent and once percent-decoded: `/wp-login.php` matches
`/wp-login.php?x=1` and `/%77p-login.php`, and does not match `/wp-login.php/`
or `/blog/wp-login.php`.
- `SWW_WAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute of these
kinds, and of no other:
- `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE` counts a client's refusals per minute of
these kinds, and of no other:
- a rule file match;
- a trap path;
- a Core Rule Set match.
@@ -748,10 +752,10 @@ every app, and no setting undoes them:
Core Rule Set does not read, which by default is every request. git
compresses a fetch request over 1 KiB and says so in that header. On a body
the Core Rule Set does read, the header stays refused, since a compressed
body cannot be inspected. While `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is set, it
stays refused too on a `PUT` of JSON, XML or form data to Nextcloud's WebDAV
paths, whose body Nextcloud's fixes leave unread. The other headers the Core
Rule Set refuses, such as `Proxy`, stay refused.
body cannot be inspected. While `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE`
is set, it stays refused too on a `PUT` of JSON, XML or form data to
Nextcloud's WebDAV paths, whose body Nextcloud's fixes leave unread. The
other headers the Core Rule Set refuses, such as `Proxy`, stay refused.
3. The `redirect_uri` parameter is not checked for a URL naming an IP address or
`localhost` (931100, 934110). Git Credential Manager, git-credential-oauth
and tea, which gitea registers for OAuth sign-in, ask to be sent back to
@@ -812,7 +816,7 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
- `artifactName` and `redirect_to`.
They skip these rules in the query, and in a form data or multipart body
while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is set.
while `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set.
gitea puts into them file paths, branch and workflow names, and the page to
return to after signing in. It reads these values as names in a repository
@@ -971,8 +975,8 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
and `fix-powershell-module` were refused.
`; wget http://example.com/x.sh` and path traversal are still refused in
these fields. These exclusions matter only while
`SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is set. None applies when the query
has a parameter of the same name.
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. None applies when
the query has a parameter of the same name.
- `id` of the forms that adopt or delete an unadopted repository, for the
rules that refused repository names (930120, 932120, 932160, 932260). `id`
@@ -1066,7 +1070,8 @@ The fields are:
the lists of system files, shell paths, command names and JavaScript names
(930120, 932120, 932160, 932260, 941180).
- These fields of a body. Each applies only when the query has no parameter of
that name, so it matters only while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is set:
that name, so it matters only while
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set:
- `content` of a post, a page, a template or a template part saved through
the REST API. It holds HTML and any code it shows.
- `content` of a reply to a comment sent to `admin-ajax.php`.
@@ -1193,8 +1198,8 @@ These fields of a body skip the rules that refused them:
for the same four rules.
- `mail` on those forms, for 930120, 932120 and 932260 only.
These exclusions matter only while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is set.
None applies when the query has a parameter of the same name.
These exclusions matter only while `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE`
is set. None applies when the query has a parameter of the same name.
Elsewhere these fields keep every rule. The paths are those of Drupal at the
site root, with or without a language prefix such as `/fr`. Such a prefix is any
@@ -1294,8 +1299,9 @@ These fields of a body skip the rules that refused them:
have a namespace, such as `Special:`. The title follows `/index.php/` or
`/wiki/` in the path, or is the `title` of `/index.php`.
These body exclusions matter only while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is
set. None applies when the query has a parameter of the same name.
These body exclusions matter only while
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. None applies when the query
has a parameter of the same name.
The names of uploaded files skip 932180 on MediaWiki's pages and on `/api.php`,
so a picture named `Notes on .htaccess.png` can be uploaded. MediaWiki refuses a
@@ -1448,8 +1454,9 @@ items.
`<img src=x onerror=x()>` sent as a password reach Mastodon;
`<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
These body exclusions matter only while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is
set. None applies when the query has a parameter of the same name.
These body exclusions matter only while
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. None applies when the query
has a parameter of the same name.
The names of uploaded files skip 932180 on these paths:
@@ -1514,8 +1521,9 @@ The addresses skip them on these paths too:
- `/api/auth-requests` and the paths under it, for signing in with another
device.
These body exclusions matter only while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is
set. None applies when the query has a parameter of the same name.
These body exclusions matter only while
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. None applies when the query
has a parameter of the same name.
`username` in the query of `/api/hibp/breach`, the data breach report, skips
930120, 932120 and 932260 too.
@@ -1688,10 +1696,11 @@ A CalDAV or CardDAV client finds these paths with a `PROPFIND` on
`/.well-known/caldav` or `/.well-known/carddav`. That method is allowed on those
two paths.
These body exclusions matter only while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is
set. None applies to a JSON field when the query has a parameter of that name,
such as `json.message`. Nextcloud's pages are matched at the site root and under
`/index.php/`. Elsewhere these fields keep every rule.
These body exclusions matter only while
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. None applies to a JSON
field when the query has a parameter of that name, such as `json.message`.
Nextcloud's pages are matched at the site root and under `/index.php/`.
Elsewhere these fields keep every rule.
Forgejo's fixes: Forgejo is a fork of gitea, so a hostname listing `forgejo`
gets gitea's fixes too. Forgejo's own fixes hold only what Forgejo sends and
@@ -1738,7 +1747,7 @@ Forgejo renames a branch at `/<owner>/<repo>/settings/rename_branch`, where
gitea uses `/<owner>/<repo>/branches/rename`. There, as on gitea's path, `from`
and `to` skip the lists of Windows commands and command names (932120, 932260),
which refused `fix-powershell-module` and `feat/docker-support`. This matters
only while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is set.
only while `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set.
The Core Rule Set reads the request without the `flash` cookie, in which Forgejo
puts what gitea puts in `gitea_flash` (gitea's second fix).
@@ -1830,9 +1839,10 @@ resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`,
`; cat /etc/passwd` and `<img src=x onerror=x()>` sent as a password reach
Ghost; `<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
These body exclusions matter only while `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is
set. None applies when the query has a parameter of the same name. Elsewhere
these fields keep every rule. The paths are those of Ghost at the site root.
These body exclusions matter only while
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. None applies when the query
has a parameter of the same name. Elsewhere these fields keep every rule. The
paths are those of Ghost at the site root.
Each app's fixes switch rules off for some of its fields and cookies. Any other
app on a hostname that gets those fixes must check those fields and cookies
@@ -1914,7 +1924,7 @@ together.
- an answer in `127.255.255.0/24`;
- an answer outside `127.0.0.0/8`;
- a resolver error;
- no answer within `SWW_WAF_REPUTATION_TIMEOUT`.
- no answer within `SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`.
DNS blocklists made for mail servers list residential ranges, and would
block ordinary visitors, so they should not be used. Spamhaus zones need
@@ -1939,13 +1949,13 @@ together.
after which no client is checked for a minute, when:
- AbuseIPDB answers other than `200`;
- the answer gives no score;
- no answer comes within `SWW_WAF_REPUTATION_TIMEOUT`.
- `SWW_WAF_REPUTATION_ACTION`, for DNS blocklist and AbuseIPDB hits, is
- no answer comes within `SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`.
- `SWW_WAF_DNSBL_ABUSEIPDB_ACTION`, for DNS blocklist and AbuseIPDB hits, is
`limit:25` by default. Such verdicts are less certain than a blocklist, so a
listed client gets a quarter of every request and byte limit rather than a
refusal.
- Verdicts and scores are kept for `SWW_WAF_REPUTATION_CACHE_TTL`, in memory and
in `dnsbl.json` and `abuseipdb-scores.json`. A source that is down or over
- Verdicts and scores are kept for `SWW_WAF_IP_REPUTATION_CACHE_TTL`, in memory
and in `dnsbl.json` and `abuseipdb-scores.json`. A source that is down or over
quota gives no verdict. The service continues. Each failure raises a
`source_failure` alert (see "Alerting").
@@ -1982,13 +1992,13 @@ which the metrics show. They reveal a widely distributed scraper, thousands of
addresses at low rates each, which the per-client limits do not see. A low
percentage for its `asn=` tag in `SWW_WAF_TAG_LIMIT_PERCENT` is the response.
- Requests and body bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted per
minute and per hour, for each of:
- Requests and body bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted
per minute and per hour, for each of:
- a client;
- the surrounding netblock;
- an AS number;
- the whole service;
- a named netblock in `SWW_WAF_WATCH_NETS`.
- a named netblock in `SWW_WAF_ANOMALY_WATCHED_NETS`.
- Every request `waf` answers with `allow`, `deny` or `close` is counted, an
allowed one when its report comes. That holds whatever was done with it, and
for clients in `SWW_WAF_ALLOW_NETS` and `SWW_WAF_RATE_LIMIT_EXEMPT_NETS`,
@@ -1997,9 +2007,9 @@ percentage for its `asn=` tag in `SWW_WAF_TAG_LIMIT_PERCENT` is the response.
## Rule files
`waf` reads every `*.rules` file in `SWW_WAF_RULES_DIR` and checks each request
against them. `docs/rules.md` gives the format, the targets and actions, and the
default file.
`waf` reads every `*.rules` file in `SWW_WAF_RULE_FILES_DIR` and checks each
request against them. `docs/rules.md` gives the format, the targets and actions,
and the default file.
- The files are plain text meant to be edited by hand, so that a new scanning
pattern seen in the request log becomes a rule in one line. `waf` watches the
@@ -2028,9 +2038,9 @@ default file.
- a regex that does not compile;
- a duplicate id.
So does a `SWW_WAF_RULES_DIR` that does not exist or cannot be read. While
running, the same faults leave the rules as they were, and the log and one
`file_error` alert name the file and line. Once the file is fixed, it is
So does a `SWW_WAF_RULE_FILES_DIR` that does not exist or cannot be read.
While running, the same faults leave the rules as they were, and the log and
one `file_error` alert name the file and line. Once the file is fixed, it is
read again. An empty directory is not an error: the log says that no rules
were loaded.
@@ -2040,13 +2050,13 @@ default file.
`/<owner>/<repo>/src/branch/main/.env.example` is a file in a repository that
any visitor may open. Anchor the paths of your own `ban` rules at the site
root too. A rule has one target, so a `path` rule applies on every site behind
that `waf`. A directory mounted over `SWW_WAF_RULES_DIR` replaces the default
file; single files mounted into it add to it.
that `waf`. A directory mounted over `SWW_WAF_RULE_FILES_DIR` replaces the
default file; single files mounted into it add to it.
- Rules meant to be edited while `waf` runs belong in a mounted directory, not a
mounted file. Many editors save a file by replacing it, and a container keeps
seeing the old one when the file alone is mounted. The lookup database is
mounted the same way, for the same reason. To run without rule files, mount an
empty directory or set `SWW_WAF_RULES_ENABLED=false`.
empty directory or set `SWW_WAF_RULE_FILES_ENABLED=false`.
## Persistent state
@@ -2133,8 +2143,8 @@ lists the files, what each holds, and their sizes.
in the metrics and sent as one `file_error` alert per cooldown, and retried at
the next write.
- A hard kill loses the changes since the last write: ban changes from the last
`SWW_WAF_STATE_WRITE_DELAY`, everything else from the last
`SWW_WAF_STATE_COUNTER_INTERVAL`. Losing the volume loses ban history, not
`SWW_WAF_BANS_WRITE_DELAY`, everything else from the last
`SWW_WAF_STATE_WRITE_INTERVAL`. Losing the volume loses ban history, not
service.
## Request log
@@ -2160,10 +2170,11 @@ lines to stdout only.
severity informational, the time, the host's name and the app name from
`SWW_WAF_LOG_REMOTE_APP_NAME`.
- Sending happens on its own goroutine from a buffer of
`SWW_WAF_LOG_REMOTE_BUFFER` lines, so a server that is slow or down never
holds up a request and never stops stdout. When the buffer is full, the oldest
line is dropped and counted in the metrics. A failed connection is tried again
a second later, and twice as long after each further failure, up to a minute.
`SWW_WAF_LOG_REMOTE_BUFFER_LINES` lines, so a server that is slow or down
never holds up a request and never stops stdout. When the buffer is full, the
oldest line is dropped and counted in the metrics. A failed connection is
tried again a second later, and twice as long after each further failure, up
to a minute.
- As `waf` stops, it spends at most two seconds sending the lines still waiting,
and drops the rest; stdout has carried them.
@@ -2310,21 +2321,23 @@ In `observe` mode a request that `enforce` mode would refuse is answered
answered `404`.
The Core Rule Set inspects the body a question carries only while
`SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is set. It inspects at most 64 KiB less one
byte. So it does not inspect a JSON or XML body of 64 KiB or more, which may
have been cut short.
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. It inspects at most 64 KiB
less one byte. So it does not inspect a JSON or XML body of 64 KiB or more,
which may have been cut short.
### Answering in time
`rproxy` stops waiting for `waf` after `SWW_RPROXY_TIMEOUT_MS`, by default 2
seconds, and `SWW_RPROXY_DEFAULT_DISPOSITION` then decides. The longest a
question waits on an outside source is `SWW_WAF_LOOKUP_TIMEOUT`, by default 800
milliseconds, well within that time.
`rproxy` stops waiting for `waf` after `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, by
default 2 seconds, and `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` then
decides. The longest a question waits on an outside source is
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by default 800 milliseconds, well
within that time.
GeoJS is the only outside source a question can wait on. A question waits only
when step 6 says so, and for at most `SWW_WAF_LOOKUP_TIMEOUT`, by default 800
milliseconds. A client without an answer by then counts as unknown. No question
waits on these, which run in the background:
when step 6 says so, and for at most
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by default 800 milliseconds. A
client without an answer by then counts as unknown. No question waits on these,
which run in the background:
- the DNS blocklists and AbuseIPDB;
- fetching the blocklists, the tag lists, the CrowdSec decision list and the
@@ -2341,15 +2354,16 @@ In `enforce` mode, a question does not wait for GeoJS when it is a request:
Such a request still goes through the steps in their order. Its client's AS
number and country stay unknown until GeoJS has answered. So while
`SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, that list can refuse it first.
`SWW_WAF_UNKNOWN_LIMIT_PERCENT` does not apply to it until then, so not waiting
never lowers its limit.
`SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT` does not apply to it until then, so not
waiting never lowers its limit.
The header limits, the static lists and bans come before the lookup, so they
never wait. These can still wait for GeoJS:
- a country list, which needs the answer;
- a rate limit a percentage has lowered, which needs the answer;
- a DNS blocklist or AbuseIPDB, while `SWW_WAF_REPUTATION_ACTION` is `deny`;
- a DNS blocklist or AbuseIPDB, while `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` is
`deny`;
- a trap path;
- a rule file;
- the Core Rule Set.
@@ -2379,7 +2393,8 @@ other check runs on it.
The Core Rule Set runs each of its rules on each cookie, so thousands of small
cookies would keep it busy for over a second. Within the limits, one long cookie
or query string can still keep it busy for a fraction of a second. So enough
such requests at once can overload `waf` past `SWW_RPROXY_TIMEOUT_MS`.
such requests at once can overload `waf` past
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
### The report
@@ -2418,7 +2433,7 @@ answer.
For a request it holds, `waf` writes the request's last log line (see "The
log"). For a request it allowed, it first:
- counts the body bytes `SWW_WAF_BYTES_COUNT` names, toward the client's byte
- counts the body bytes `SWW_WAF_BYTES_COUNTED` names, toward the client's byte
limits at the percentages it gave that request;
- bans a client whose total passes a byte limit;
- adds the request with its status to the client's history, the anomaly counters
@@ -2485,9 +2500,9 @@ request and ban notes name the site by its host.
`sww rproxy` is the reverse proxy in front of one app. It asks `waf` about each
request, and passes on what `waf` allows. It keeps nothing between requests and
writes nothing to disk. When `waf` cannot be connected to, does not answer at
all, or does not answer within `SWW_RPROXY_TIMEOUT_MS`,
`SWW_RPROXY_DEFAULT_DISPOSITION` decides. Its default, `allow`, keeps the site
answering.
all, or does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`,
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides. Its default, `allow`,
keeps the site answering.
`rproxy` reads only its `SWW_RPROXY_` settings, which `docs/settings.md` lists.
One of them is required: `SWW_RPROXY_WAF_BACKEND`, `waf`'s URL. It has no
@@ -2578,36 +2593,37 @@ log line has status `0`.
### Without an answer from `waf`
`SWW_RPROXY_DEFAULT_DISPOSITION` decides what is done with the request only when
`waf`:
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides what is done with the
request only when `waf`:
- cannot be connected to;
- does not answer at all;
- does not answer within `SWW_RPROXY_TIMEOUT_MS`, which covers connecting,
sending the question and reading the whole answer.
- does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, which covers
connecting, sending the question and reading the whole answer.
An answer whose body breaks off counts as no answer at all.
`allow`, the default, passes the request to the app. `deny` answers `503`.
Either way `rproxy` writes a process warning, its log line has
`default_disposition`, and it sends no report.
`waf_backend_no_answer`, and it sends no report.
A visitor can make `waf` slow. A request that keeps `waf` busy past
`SWW_RPROXY_TIMEOUT_MS`, like any request while `waf` is overloaded, gets
`SWW_RPROXY_DEFAULT_DISPOSITION`. With `deny`, such a request gets `503` instead
of reaching the app.
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, like any request while `waf` is overloaded,
gets `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. With `deny`, such a request
gets `503` instead of reaching the app.
A path under `/_sww/` that `allow` would pass on is answered `404` instead, so
that the app never sees that prefix.
`rproxy` cannot pass on an answer whose body is over 1 MiB or whose status is
below `200`. The visitor gets `502` from `rproxy`, never the default. `rproxy`
writes a process warning, its log line has no `disposition`, and it sends no
report.
below `200`. The visitor gets `502` from `rproxy`, whatever
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` says. `rproxy` writes a process
warning, its log line has no `disposition`, and it sends no report.
A visitor that goes away while `waf` is being asked is no failure of `waf`.
`rproxy` closes the connection, with no warning, no default and no report. Its
log line has status `0` and no `disposition`.
`rproxy` closes the connection with no warning and no report, and
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` does not apply. Its log line has
status `0` and no `disposition`.
### Passing a request to the app
@@ -2710,8 +2726,8 @@ those at `info` or more severe are written.
On `SIGTERM` or `SIGINT`, `rproxy` stops taking requests and gives those in
progress five seconds to finish. It then waits for the reports on their way,
each for up to `SWW_RPROXY_TIMEOUT_MS`. A report that comes due after that is
dropped, such as that of a WebSocket that closes later.
each for up to `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`. A report that comes due
after that is dropped, such as that of a WebSocket that closes later.
## Deployment
@@ -2805,13 +2821,14 @@ process that stops at start is started again every second, not in a tight loop.
does not stop. Beside an app, the other process keeps running.
- While `rproxy` is down, traefik answers `502`. While the app is down, `rproxy`
answers `502`, which is not a client offence. While `waf` is down,
`SWW_RPROXY_DEFAULT_DISPOSITION` decides (see "Without an answer from `waf`").
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an answer
from `waf`").
- Only faults found at start stop `waf`:
- invalid configuration;
- a lookup database that is missing or unreadable;
- a rule file or state file that does not parse;
- an unwritable `SWW_WAF_STATE_DIR` or `SWW_WAF_CACHE_DIR`;
- a `SWW_WAF_RULES_DIR` that does not exist or cannot be read;
- a `SWW_WAF_RULE_FILES_DIR` that does not exist or cannot be read;
- a `SWW_WAF_LISTEN_ADDR` it cannot listen on.
- Only an invalid setting, or a `SWW_RPROXY_LISTEN_ADDR` it cannot listen on,
stops `rproxy`.