Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
56f93d4f74 | ||
|
|
a2fcc00ded | ||
|
|
3c6b0d519f | ||
|
|
f33d0a4fce | ||
|
|
0a2b2d809f | ||
|
|
23788920f7 | ||
|
|
8efd6339a6 | ||
|
|
89a60cf9fc | ||
|
|
a05a82b1af | ||
|
|
53d9bf6af6 | ||
|
|
3550956cbb | ||
|
|
2bea6516a8 |
No files matched your search
@@ -2,7 +2,20 @@
|
||||
|
||||
`smallwebwaf` is an MIT-licensed web application firewall, written in Go by
|
||||
[@sneak](https://sneak.berlin), for self-hosted web apps. It refuses the
|
||||
requests of scrapers, scanners and other abusers before they reach an app.
|
||||
requests of misbehaving scrapers, scanners and other abusers before they reach
|
||||
an app.
|
||||
|
||||
With the default settings, a client is banned when it does any of these:
|
||||
|
||||
- makes more than 120 requests in a minute, 1500 in an hour or 6000 in a day;
|
||||
- moves more than `50G` of request and response bodies in an hour, or `100G` in
|
||||
a day;
|
||||
- asks for a secret or backup file, such as `/.env` or `/.git/config`;
|
||||
- sends a user agent that names a security scanner, such as `sqlmap` or `nikto`;
|
||||
- has more than 30 requests in a minute refused as probes or attacks.
|
||||
|
||||
A scraper that does none of these is left alone. A scraper that waits 15 seconds
|
||||
between requests stays under the rate limits.
|
||||
|
||||
## What it does
|
||||
|
||||
@@ -10,10 +23,10 @@ A web application firewall reads each request on its way to a web app and
|
||||
refuses the ones that look like abuse. `smallwebwaf` is one program, `sww`,
|
||||
shipped in one image, with two commands that do this together:
|
||||
|
||||
- `sww rproxy` is a small reverse proxy in front of one app. traefik routes to
|
||||
it. It asks `waf` about each request, passes the allowed ones to the app, and
|
||||
passes the app's answers back. It keeps nothing between requests and writes
|
||||
nothing to disk.
|
||||
- `sww rproxy` is a small reverse proxy in front of one app. It sits behind a
|
||||
TLS-terminating proxy, such as traefik, nginx or Caddy. It asks `waf` about
|
||||
each request, passes the allowed ones to the app, and passes the app's answers
|
||||
back. It writes nothing to disk.
|
||||
- `sww waf` decides. It runs the checks and keeps the bans, the counters and
|
||||
each client's history. One `waf` serves many sites: a client banned through
|
||||
one site is refused on all of them, and its requests on all of them count
|
||||
@@ -22,7 +35,26 @@ shipped in one image, with two commands that do this together:
|
||||
The site keeps answering while `waf` is down, slow or restarting: without an
|
||||
answer in time, `rproxy` passes the request to the app.
|
||||
|
||||
It protects against:
|
||||
`rproxy` also protects the app on its own, whether or not `waf` answers. With
|
||||
the default settings, `rproxy`:
|
||||
|
||||
- answers `400` to a malformed request, such as a broken header line or a
|
||||
missing `Host`;
|
||||
- answers `405` to a method it does not pass on, such as `TRACE`;
|
||||
- answers `431` to a request line and headers over 32 KiB together;
|
||||
- gives the visitor 60 seconds to send the request line and headers, and then 60
|
||||
seconds more for the whole request to reach the app;
|
||||
- answers `413` to a request body over 100 MiB;
|
||||
- gives the app's answer 30 minutes to reach the visitor, counted from the end
|
||||
of the request;
|
||||
- answers `502` in place of an app's answer whose body is over 1 GiB, or cuts it
|
||||
off if it has started;
|
||||
- closes a kept-open connection after 120 seconds idle.
|
||||
|
||||
`rproxy` does not limit the number of headers. `waf` limits the number of
|
||||
headers and cookies, and the length of the request line and of each header line.
|
||||
|
||||
Together, `rproxy` and `waf` protect against:
|
||||
|
||||
- Scrapers that fetch every page of a site, all day. Each client gets a limit on
|
||||
requests and on body bytes per minute, per hour and per day. The defaults
|
||||
@@ -44,8 +76,6 @@ It protects against:
|
||||
limit, so that they are banned sooner;
|
||||
- refuse or limit clients on a blocklist, a DNS blocklist, or with a high
|
||||
AbuseIPDB score.
|
||||
- Slow and oversized requests. Every request and every answer has a time limit
|
||||
and a size limit.
|
||||
|
||||
For the operator, it:
|
||||
|
||||
@@ -57,8 +87,7 @@ For the operator, it:
|
||||
|
||||
Not built, and not planned:
|
||||
|
||||
- TLS and routing by host name. Both stay with traefik, the reverse proxy in
|
||||
front of `rproxy`.
|
||||
- TLS and routing by host name. Both stay with the TLS-terminating proxy.
|
||||
- Browser challenges such as captchas.
|
||||
- A web console, or a configuration file.
|
||||
- Defence against floods large enough to fill the host's network link.
|
||||
@@ -90,20 +119,61 @@ accepts connections, whether or not `waf` answers; in `waf`'s container, once
|
||||
The usual setup is three containers on one private network:
|
||||
|
||||
- the app;
|
||||
- `rproxy`, in front of the app, which traefik routes to:
|
||||
- `rproxy`, in front of the app, which the TLS-terminating proxy forwards to:
|
||||
`SWW_RPROXY_WAF_BACKEND` names `waf` and `SWW_RPROXY_UPSTREAM_URL` names the
|
||||
app;
|
||||
- `waf`, with `SWW_COMMAND=waf` and a volume at `/var/lib/sww` for its state
|
||||
files.
|
||||
|
||||
Point traefik's labels at port 8080 of `rproxy`
|
||||
(`traefik.http.services.<name>.loadbalancer.server.port=8080`). Publish no port
|
||||
of the app or of `waf`. Anything on that network can reach the app around
|
||||
`rproxy`, and can ask `waf` questions, so the network holds only what you trust.
|
||||
Each further app adds its own container and its own `rproxy`, whose
|
||||
Publish no port of the app or of `waf`. Anything on that network can reach the
|
||||
app around `rproxy`, and can ask `waf` questions, so the network holds only what
|
||||
you trust. Each further app adds its own container and its own `rproxy`, whose
|
||||
`SWW_RPROXY_WAF_BACKEND` names the same `waf`. The "Smallest setup" example
|
||||
below shows this shape.
|
||||
|
||||
### The TLS-terminating proxy
|
||||
|
||||
`rproxy` serves plain HTTP. It runs behind any TLS-terminating proxy, such as
|
||||
traefik, nginx or Caddy. The TLS-terminating proxy must:
|
||||
|
||||
- forward each request to port 8080 of `rproxy` over plain HTTP;
|
||||
- keep the visitor's `Host` header;
|
||||
- add the visitor's address to `X-Forwarded-For`;
|
||||
- set `X-Forwarded-Proto` to the scheme the visitor used;
|
||||
- close an unused connection to `rproxy` in under 120 seconds;
|
||||
- connect to `rproxy` from an address in `SWW_RPROXY_TRUSTED_PROXIES`, by
|
||||
default `10.0.0.0/8`, `172.16.0.0/12` or `192.168.0.0/16`.
|
||||
|
||||
From any other address, `rproxy` ignores the forwarded headers and takes the
|
||||
TLS-terminating proxy for the visitor. Every visitor then shares one address,
|
||||
and one ban.
|
||||
|
||||
traefik and Caddy keep the `Host` header and set both forwarded headers by
|
||||
default. traefik's Docker label
|
||||
`traefik.http.services.<name>.loadbalancer.server.port=8080` names the port.
|
||||
Caddy keeps an unused connection for 120 seconds by default, so its site
|
||||
shortens that to 90 seconds:
|
||||
|
||||
```caddyfile
|
||||
reverse_proxy rproxy:8080 {
|
||||
transport http {
|
||||
keepalive 90s
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
By default nginx sends the host in `proxy_pass` as `Host` and sets neither
|
||||
forwarded header, so its site sets all three:
|
||||
|
||||
```nginx
|
||||
location / {
|
||||
proxy_pass http://rproxy:8080;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
```
|
||||
|
||||
### `rproxy` in the app's container
|
||||
|
||||
An app's image can instead be built on the `smallwebwaf` image, so that `rproxy`
|
||||
@@ -154,13 +224,13 @@ entrypoint starts `rproxy` and the app side by side, each as its own user. Run
|
||||
the app's image with `SWW_RPROXY_WAF_BACKEND` set. `deploy/example-app` is such
|
||||
an app, and the "`rproxy` beside the app" example below runs one.
|
||||
|
||||
The app sees the headers traefik sent, with traefik's address added to
|
||||
`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
|
||||
`rproxy`'s address for forwarded headers. In a container of its own, `rproxy` is
|
||||
on the private network, which the usual default of a trusted-proxy setting
|
||||
covers. Beside the app, its address is `127.0.0.1` or `::1`, which the private
|
||||
ranges leave out, so the app's own setting must name them too, as `app.run`
|
||||
does.
|
||||
The app's TCP peer is `rproxy`. The app sees the headers the TLS-terminating
|
||||
proxy sent, with that proxy's address added to `X-Forwarded-For`. So the app
|
||||
must trust `rproxy`'s address for forwarded headers. In a container of its own,
|
||||
`rproxy` is on the private network, which the usual default of a trusted-proxy
|
||||
setting covers. Beside the app, its address is `127.0.0.1` or `::1`, which the
|
||||
private ranges leave out, so the app's own setting must name them too, as
|
||||
`app.run` does.
|
||||
|
||||
### Checking that it works
|
||||
|
||||
@@ -168,7 +238,7 @@ does.
|
||||
`healthy` for each container of `smallwebwaf`.
|
||||
- A log line with `"msg":"starting"` gives the version, the address and every
|
||||
setting in effect, with secrets masked.
|
||||
- A request to the site through traefik gets the app's answer.
|
||||
- A request to the site through the TLS-terminating proxy gets the app's answer.
|
||||
`https://<site>/_sww/healthz` answers `ok`. `waf`'s log lines about the
|
||||
request have `"action":"forward"`.
|
||||
- To see a refusal without banning yourself, run `waf` with
|
||||
@@ -228,13 +298,13 @@ Every setting the code reads, with its default:
|
||||
| `SWW_RPROXY_LISTEN_ADDR` | `:8080` | Where `rproxy` listens; leave the host part empty. |
|
||||
| `SWW_RPROXY_TRUSTED_PROXIES` | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none. |
|
||||
| `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `waf`'s answer to a question and to a report, in milliseconds. |
|
||||
| `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` | `allow` | What `rproxy` does with a request when `waf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
||||
| `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` | `allow` | What `rproxy` does with a request when `waf` gives no answer in time, and while the circuit breaker for asks to `waf` is open: `allow` passes it to the app, `deny` answers `503`. |
|
||||
| `SWW_RPROXY_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
||||
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
||||
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
||||
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
|
||||
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `fsn1/waf`. |
|
||||
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. |
|
||||
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
||||
| `SWW_WAF_IPV6_GROUP_PREFIX` | `64` | The prefix length of the IPv6 netblock that counts as one client, from 32 to 128. |
|
||||
| `SWW_WAF_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
||||
@@ -259,8 +329,8 @@ Every setting the code reads, with its default:
|
||||
| `SWW_WAF_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
||||
| `SWW_WAF_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
|
||||
| `SWW_WAF_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
||||
| `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` | `800` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take, in milliseconds. |
|
||||
| `SWW_WAF_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||
| `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` | `800` | How long a request waits for its client's first GeoJS answer when a setting needs it, in milliseconds. The request to GeoJS goes on, and a later answer is kept. |
|
||||
| `SWW_WAF_ADD_LOOKUP_HEADERS` | `true`, or `false` while the lookup source is `off` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||
| `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. |
|
||||
| `SWW_WAF_TAG_LISTS` | empty | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`. |
|
||||
| `SWW_WAF_TAG_DENY` | empty | Tags whose requests are refused, such as `torexit,country=CN`. |
|
||||
@@ -278,7 +348,7 @@ Every setting the code reads, with its default:
|
||||
| `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD` | `5` | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5. |
|
||||
| `SWW_WAF_CORE_RULE_SET_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
||||
| `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
||||
| `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
||||
| `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` | `off` | How much of a request body the Core Rule Set reads, a size up to `8K`; `off` reads none. |
|
||||
| `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
|
||||
| `SWW_WAF_APPS` | unset | The apps each hostname serves, such as `git.example.com:gitea;blog.example.com:wordpress`; only those apps' fixes apply there. "Attack detection" lists the apps. Listing `forgejo` brings gitea's fixes too. Unset, every app's fixes apply to every hostname. |
|
||||
| `SWW_WAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
||||
@@ -460,7 +530,7 @@ services:
|
||||
image: smallwebwaf
|
||||
environment:
|
||||
SWW_COMMAND: waf
|
||||
SWW_WAF_INSTANCE_NAME: fsn1/waf
|
||||
SWW_WAF_INSTANCE_NAME: host1/waf
|
||||
# Refuse nothing yet; log what enforce mode would have done.
|
||||
SWW_WAF_MODE: observe
|
||||
# Look addresses up in a local IPinfo Lite database, never GeoJS.
|
||||
@@ -540,9 +610,10 @@ request reaches the app.
|
||||
A refused visitor is answered with `SWW_WAF_BAN_RESPONSE`. A `block` rule or a
|
||||
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
|
||||
false positives; such refusals count toward `SWW_WAF_REFUSALS_LIMIT_PER_MINUTE`
|
||||
instead. A request over the header limits of step 3 is answered `431`, and bans
|
||||
nobody either. `rproxy` itself answers `405` to a method it does not pass on,
|
||||
and the status for a size or time limit, such as `413` for a body too large.
|
||||
instead. A request over the header limits of step 3 is answered `414` for a
|
||||
request line over 8 KiB and `431` otherwise. It bans nobody either. `rproxy`
|
||||
itself answers `405` to a method it does not pass on, and the status for a size
|
||||
or time limit, such as `413` for a body too large.
|
||||
|
||||
### Clients and bans
|
||||
|
||||
@@ -678,9 +749,9 @@ stops the running process (see [`docs/state-files.md`](docs/state-files.md)).
|
||||
- `internal/waf`: `waf`'s server, which runs the checks and answers its own
|
||||
endpoints.
|
||||
- `internal/ratelimit`, `internal/bans`, `internal/rules`,
|
||||
`internal/coreruleset`, `internal/lookup`, `internal/reputation`,
|
||||
`internal/routewatch`, `internal/anomaly` and `internal/alerts`: one check or
|
||||
source each.
|
||||
`internal/coreruleset`, `internal/geojs`, `internal/lookup`,
|
||||
`internal/reputation`, `internal/routewatch`, `internal/anomaly` and
|
||||
`internal/alerts`: one check or source each.
|
||||
- `internal/state`, `internal/requestlog`, `internal/remotelog` and
|
||||
`internal/metrics`: the state and cache files, the log lines, the syslog
|
||||
sender and the metrics.
|
||||
@@ -747,10 +818,9 @@ on the host.
|
||||
- `script/docker`: builds the image, whose build runs the tests and the linter
|
||||
first. `make docker`.
|
||||
- `script/publish`: runs `script/docker`, whose build happens on the Docker host
|
||||
named by `DOCKER_HOST` (`ssh://root@fsn1app1` unless set). It then pushes the
|
||||
image to `PUBLISH_IMAGE` (`git.eeqj.de/sneak/smallwebwaf` unless set). It
|
||||
refuses a working tree with uncommitted changes. `make publish`. It pushes
|
||||
four tags:
|
||||
named by `DOCKER_HOST`, then pushes the image to `PUBLISH_IMAGE`, such as
|
||||
`registry.example.com/smallwebwaf`. Both variables are required. It refuses a
|
||||
working tree with uncommitted changes. `make publish`. It pushes four tags:
|
||||
- the UTC date as `YYYY-MM-DD`;
|
||||
- the short commit hash;
|
||||
- the branch name;
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
`smallwebwaf` is a web application firewall for self-hosted apps. It is one
|
||||
binary, `sww`, and two of its commands do the work:
|
||||
|
||||
- `sww rproxy` is a reverse proxy between traefik and one app.
|
||||
- `sww rproxy` is a reverse proxy between a TLS-terminating proxy and one app.
|
||||
- `sww waf` decides what happens to each request, for many sites.
|
||||
|
||||
This document says what it must do and why, each requirement once. Everything in
|
||||
@@ -45,7 +45,8 @@ never as a time.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- TLS, certificates and routing by host name. Those are traefik's job.
|
||||
- TLS, certificates and routing by host name. Those are the job of the
|
||||
TLS-terminating proxy, such as traefik, nginx or Caddy.
|
||||
- More than one app behind one `rproxy`. Each app has its own.
|
||||
- Browser challenges such as captchas or proof of work. If wanted, chain Anubis
|
||||
between `rproxy` and the app.
|
||||
@@ -86,8 +87,10 @@ never as a time.
|
||||
change, and tells which rules a request matches.
|
||||
- `internal/coreruleset`: the Core Rule Set, with the five changes and the
|
||||
apps' fixes listed under "Attack detection", as Coraza directives.
|
||||
- `internal/lookup`: looks clients up through GeoJS or in the lookup
|
||||
database.
|
||||
- `internal/geojs`: looks clients up through GeoJS and keeps the answers. It
|
||||
imports no other package of this repository: `waf` gives it what it needs
|
||||
to count and report failures.
|
||||
- `internal/lookup`: looks clients up in the lookup database.
|
||||
- `internal/reputation`: fetches the lists, asks the DNS blocklists and
|
||||
AbuseIPDB in the background, and keeps their answers.
|
||||
- `internal/routewatch`: asks a routewatch instance about clients in the
|
||||
@@ -151,26 +154,26 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
|
||||
around its address, a /64 by default, since one abuser usually controls a
|
||||
whole /64. A client is therefore one IPv4 address or one IPv6 group.
|
||||
3. The header limits (`header_limits`; see "Answering in time"). They come first
|
||||
after `SWW_WAF_ALLOW_NETS`, since they only count header fields and cookies.
|
||||
Next, the tag lists (`tag_lists`; see "Request tags"). Each list that holds
|
||||
the client adds its tag. Next, `routewatch_tags` adds the client's `asn=` and
|
||||
`prefix=` tags while `SWW_WAF_ROUTEWATCH_URL` is set. A client in
|
||||
`SWW_WAF_ALLOW_NETS` skips the header limits and gets no tag.
|
||||
after `SWW_WAF_ALLOW_NETS`, since they only measure lines and count header
|
||||
fields and cookies. Next, the tag lists (`tag_lists`; see "Request tags").
|
||||
Each list that holds the client adds its tag. Next, `routewatch_tags` adds
|
||||
the client's `asn=` and `prefix=` tags while `SWW_WAF_ROUTEWATCH_URL` is set.
|
||||
A client in `SWW_WAF_ALLOW_NETS` skips the header limits and gets no tag.
|
||||
4. The static lists. A client in `SWW_WAF_ALLOW_NETS` skips steps 5 to 15, the
|
||||
byte limits and the refusal limit. A client in `SWW_WAF_DENY_NETS` is refused
|
||||
(`deny_nets`); a client in both lists is let through.
|
||||
5. Bans. A client under a ban is refused (`active_ban`; see "Bans").
|
||||
6. The lookup, unless `SWW_WAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
|
||||
and country"). The request goes on at once. It waits for the first GeoJS
|
||||
answer, up to `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, only when one of
|
||||
these needs it:
|
||||
and country"). The request waits for the first GeoJS answer, up to
|
||||
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, when one of these needs it:
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS`, which is `true` by default;
|
||||
- the country list;
|
||||
- a biased threshold;
|
||||
- `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming an `asn=` or
|
||||
`country=` tag;
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS`.
|
||||
`country=` tag.
|
||||
|
||||
A request that ends before the answer comes is logged without it.
|
||||
Otherwise the request goes on at once. A request that ends before the answer
|
||||
comes is logged without it.
|
||||
|
||||
7. The country list (`countries`; see "Country list").
|
||||
8. The reputation sources (see "Reputation"), in this order:
|
||||
@@ -197,10 +200,10 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
|
||||
16. A path under `/_sww/` is answered `404`, so that the app never sees that
|
||||
prefix.
|
||||
17. `rproxy` passes the request to `SWW_RPROXY_UPSTREAM_URL`, streaming (see
|
||||
"Passing a request to the app"). With `SWW_WAF_ADD_LOOKUP_HEADERS`,
|
||||
`X-Client-ASN` and `X-Client-Country` are added for the app's own logs; a
|
||||
header whose value is unknown is left out. Whatever that setting says, any
|
||||
such header the client sent is removed.
|
||||
"Passing a request to the app"). While `SWW_WAF_ADD_LOOKUP_HEADERS` is
|
||||
`true`, as by default, `X-Client-ASN` and `X-Client-Country` are added for
|
||||
the app's own logs; a header whose value is unknown is left out. Whatever
|
||||
that setting says, any such header the client sent is removed.
|
||||
18. After the visitor's answer has ended, `rproxy` sends the report (see "The
|
||||
report"). `waf` then counts:
|
||||
- For a request it allowed, the body bytes of the response, of the request
|
||||
@@ -252,9 +255,9 @@ does not reach neighbours who did nothing. Requests refused under a ban are not
|
||||
counted against limits. They are counted in the client's history and in the
|
||||
ban's notes.
|
||||
|
||||
A request over the header limits is refused with `431` before the bans are
|
||||
checked (see "Answering in time"). It makes no ban, makes no ban permanent and
|
||||
is not counted in a ban's notes.
|
||||
A request over the header limits is refused with `414` or `431` before the bans
|
||||
are checked (see "Answering in time"). It makes no ban, makes no ban permanent
|
||||
and is not counted in a ban's notes.
|
||||
|
||||
An offence is a request `waf` holds against the client. It is one of:
|
||||
|
||||
@@ -506,19 +509,25 @@ service that keeps its visitors' addresses on its own host.
|
||||
dropped from that table. The answer reaches the client's history and ban notes
|
||||
when it comes.
|
||||
- `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` is how long a visitor's request
|
||||
waits for its client's first answer, when a setting needs it (see step 6). It
|
||||
also bounds each request to GeoJS. A client whose answer has not come in time
|
||||
counts as unknown until it comes, and its later requests do not wait. Its
|
||||
address is asked about again in the background.
|
||||
waits for its client's first answer, when a setting needs it (see step 6). A
|
||||
client whose answer has not come in time counts as unknown until it comes, and
|
||||
its later requests do not wait.
|
||||
- A request to GeoJS may take up to 10 seconds, however long visitors wait. It
|
||||
goes on after the visitor's request ends, and its answer is kept. Say GeoJS
|
||||
answers after 5 seconds. The visitor's request is decided after 800
|
||||
milliseconds, without a country. A request from the same client 6 seconds
|
||||
later gets the answer.
|
||||
- Requests from one client that come together share one request to GeoJS.
|
||||
- A client the lookup cannot place has an unknown country and AS number, which
|
||||
the country list and the biased thresholds treat as described there. Nothing
|
||||
else treats it differently.
|
||||
- GeoJS publishes no rate limit, but its terms forbid "an excessive amount of
|
||||
API requests", judged by GeoJS alone, and let it block a caller. While GeoJS
|
||||
is slow, down or refusing `waf`, new clients count as unknown. GeoJS is then
|
||||
left alone for a second, and twice as long after each further failure, up to
|
||||
five minutes. Each failure raises a `source_failure` alert (see "Alerting"). A
|
||||
service that cannot accept this uses the file.
|
||||
API requests", judged by GeoJS alone, and let it block a caller. A request to
|
||||
GeoJS fails when it takes over 10 seconds or GeoJS answers with an error.
|
||||
While GeoJS is slow, down or refusing `waf`, new clients count as unknown.
|
||||
After a failure GeoJS is left alone for a second, and twice as long after each
|
||||
further failure, up to five minutes. Each failure raises a `source_failure`
|
||||
alert (see "Alerting"). A service that cannot accept this uses the file.
|
||||
- The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`,
|
||||
`as_name` and `country_code` fields are read. The operator downloads it with a
|
||||
free IPinfo account and mounts it read-only; `waf` never fetches it and holds
|
||||
@@ -703,11 +712,12 @@ files, the limits and the bans still apply to the client that sends it.
|
||||
rules is loaded yet.
|
||||
|
||||
- `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` switches body inspection on, up
|
||||
to that size. It takes at most `64K`, since `waf` gets no more of a body than
|
||||
that (see "The answer"). Form data and multipart bodies are read up to the
|
||||
size. JSON and XML bodies are read only when no larger than the size, since
|
||||
those cannot be read in part. Any other body is not read: the Core Rule Set
|
||||
would read it as form data, where binary content such as a git push trips
|
||||
to that size. It takes at most `8K`, so that `waf` answers in time (see
|
||||
"Answering in time"). Form data and multipart bodies are read up to the size.
|
||||
JSON and XML bodies are read only when no larger than the size, since those
|
||||
cannot be read in part. A JSON body over 1 KiB is not read either, whatever
|
||||
the size (see "Answering in time"). Any other body is not read: the Core Rule
|
||||
Set would read it as form data, where binary content such as a git push trips
|
||||
rules written for text. A body the Core Rule Set cannot parse adds 5 to the
|
||||
score, and so does a multipart body cut by the limit in the middle of a part's
|
||||
header. Body inspection suits apps whose forms carry no code.
|
||||
@@ -1943,10 +1953,10 @@ leave out a cookie only when its name matches exactly, case included.
|
||||
### Ban lengths, response and scope
|
||||
|
||||
`SWW_WAF_BAN_RESPONSE` is `403` by default, which makes a ban easy to recognise
|
||||
when debugging; `429` and `close` are the alternatives. Behind traefik, `close`
|
||||
does not leave the client unanswered: traefik answers `502`, as it does whenever
|
||||
its backend drops a connection. `SWW_WAF_BAN_SCOPE_V4_PREFIX` widens an IPv4 ban
|
||||
to the surrounding netblock. The four ban lengths are under "Bans".
|
||||
when debugging; `429` and `close` are the alternatives. `close` does not leave
|
||||
the client unanswered: the TLS-terminating proxy answers `502`, as it does
|
||||
whenever its backend drops a connection. `SWW_WAF_BAN_SCOPE_V4_PREFIX` widens an
|
||||
IPv4 ban to the surrounding netblock. The four ban lengths are under "Bans".
|
||||
|
||||
### Reputation
|
||||
|
||||
@@ -2373,15 +2383,15 @@ runs no check, writes no request log line and counts nothing in the metrics.
|
||||
answer but the last below carries `X-Sww-Disposition`.
|
||||
|
||||
- `X-Sww-Disposition: allow`, status `200`, no body. `rproxy` passes the request
|
||||
to the app. With `SWW_WAF_ADD_LOOKUP_HEADERS` the answer also carries
|
||||
`X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the request it
|
||||
passes on. A header whose value is unknown is left out.
|
||||
to the app. While `SWW_WAF_ADD_LOOKUP_HEADERS` is `true`, the answer also
|
||||
carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
|
||||
request it passes on. A header whose value is unknown is left out.
|
||||
- `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
|
||||
get. `rproxy` gives them to the visitor, less this header. The status is one
|
||||
of:
|
||||
- `SWW_WAF_BAN_RESPONSE`, for a refusal;
|
||||
- `403`, for a `block` rule or the Core Rule Set;
|
||||
- `431`, for a request over the header limits;
|
||||
- `414` or `431`, for a request over the header limits;
|
||||
- `404`, for a path under `/_sww/` that passes the checks, so that the app
|
||||
never sees that prefix.
|
||||
- `X-Sww-Disposition: close`, status `200`, no body. This takes the place of a
|
||||
@@ -2396,9 +2406,7 @@ In `observe` mode a request that `enforce` mode would refuse is answered
|
||||
answered `404`.
|
||||
|
||||
The Core Rule Set inspects the body a question carries only while
|
||||
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set. It inspects at most 64 KiB
|
||||
less one byte. So it does not inspect a JSON or XML body of 64 KiB or more,
|
||||
which may have been cut short.
|
||||
`SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` is set, and at most 8 KiB of it.
|
||||
|
||||
### Answering in time
|
||||
|
||||
@@ -2411,8 +2419,9 @@ within that time.
|
||||
GeoJS is the only outside source a question can wait on. A question waits only
|
||||
when step 6 says so, and for at most
|
||||
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, by default 800 milliseconds. A
|
||||
client without an answer by then counts as unknown. No question waits on these,
|
||||
which run in the background:
|
||||
client without an answer by then counts as unknown. The setting bounds only the
|
||||
question's wait: the request to GeoJS goes on, and its answer is kept for later
|
||||
questions. No question waits on these, which run in the background:
|
||||
|
||||
- the DNS blocklists and AbuseIPDB;
|
||||
- fetching the blocklists, the tag lists and the file
|
||||
@@ -2442,33 +2451,57 @@ never wait. These can still wait for GeoJS:
|
||||
- a rule file;
|
||||
- the Core Rule Set.
|
||||
|
||||
The header limits are fixed, with no setting. `waf` refuses a request over
|
||||
either one:
|
||||
The header limits are fixed, with no setting. A line's length is in bytes and
|
||||
does not count its line ending. `waf` refuses a request whose request line is
|
||||
longer than 8 KiB with `414`. An example is `GET /search?q=` followed by 30,000
|
||||
characters.
|
||||
|
||||
It refuses a request with `431` for any of:
|
||||
|
||||
- a header line longer than 8 KiB. Each value of a header sent twice is a line
|
||||
of its own. `Host` counts as a line.
|
||||
- more than 50 cookies. Each part of its `Cookie` headers between semicolons
|
||||
counts as one, unless it is empty or only spaces and tabs.
|
||||
- more than 100 header fields. A header sent twice counts twice. `Host` and
|
||||
`Transfer-Encoding` do not count. `X-Request-ID` and `X-Forwarded-Proto` do.
|
||||
|
||||
Real clients never reach these limits. The answer is `deny` with `431`, and the
|
||||
log line's action is `request_header_too_large`. The refusal makes no ban and is
|
||||
no offence.
|
||||
Real clients rarely reach these limits. nginx refuses lines over 8 KiB by
|
||||
default, and Apache's default limit is about the same. The answer is `deny`. The
|
||||
log line's action is `request_line_too_long` for `414` and
|
||||
`request_header_too_large` for `431`. The refusal makes no ban and is no
|
||||
offence.
|
||||
|
||||
The header limits are the first check after `SWW_WAF_ALLOW_NETS`. A request over
|
||||
them is refused before any later check runs. So it causes no lookup, no query to
|
||||
an outside source and no rule match, and the rate limits do not count it. A
|
||||
client under a ban or in `SWW_WAF_DENY_NETS` also gets `431` for such a request.
|
||||
client under a ban or in `SWW_WAF_DENY_NETS` also gets `414` or `431` for such a
|
||||
request.
|
||||
|
||||
The limits apply on every path and in every `SWW_WAF_CORE_RULE_SET_MODE`. A
|
||||
client in `SWW_WAF_ALLOW_NETS` skips them, as it skips the Core Rule Set. In
|
||||
`observe` mode such a request is answered `allow` with `would_action`, and no
|
||||
other check runs on it.
|
||||
|
||||
The Core Rule Set runs each of its rules on each cookie, so thousands of small
|
||||
cookies would keep it busy for over a second. Within the limits, one long cookie
|
||||
or query string can still keep it busy for a fraction of a second. So enough
|
||||
such requests at once can overload `waf` past
|
||||
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
|
||||
The Core Rule Set is the one check a visitor can keep busy for long. It runs
|
||||
each of its rules on each part of a request, so thousands of small cookies would
|
||||
keep it busy for over a second. These fixed limits bound what it reads:
|
||||
|
||||
- the header limits above;
|
||||
- `rproxy`'s limit of 32 KiB on the request line and headers together (see
|
||||
"`rproxy`'s limits");
|
||||
- 8 KiB of a body, the most `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` takes,
|
||||
or 1 KiB of a JSON body.
|
||||
|
||||
Coraza names each value in a JSON body by the whole path to it, such as
|
||||
`json.users.0.name`. So the names and values the Core Rule Set reads in a JSON
|
||||
body can add up to more than the body. From 1 KiB of JSON, Coraza makes at most
|
||||
8 KiB of them, no more than the largest body of another kind.
|
||||
|
||||
A request can reach all of these at once. At paranoia level 4, with no rule
|
||||
switched off, the Core Rule Set takes under a second on one core of a current
|
||||
server to inspect it. So `waf` answers it within the default 2 seconds, even
|
||||
after the default 800 milliseconds of waiting for GeoJS. Enough such requests at
|
||||
once can still overload `waf` past `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`.
|
||||
|
||||
### The report
|
||||
|
||||
@@ -2572,11 +2605,11 @@ request and ban notes name the site by its host.
|
||||
## `rproxy`
|
||||
|
||||
`sww rproxy` is the reverse proxy in front of one app. It asks `waf` about each
|
||||
request, and passes on what `waf` allows. It keeps nothing between requests and
|
||||
writes nothing to disk. When `waf` cannot be connected to, does not answer at
|
||||
all, or does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`,
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides. Its default, `allow`,
|
||||
keeps the site answering.
|
||||
request, and passes on what `waf` allows. It writes nothing to disk. The only
|
||||
thing it keeps between requests is its circuit breaker for asks to `waf`.
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides what is done with a
|
||||
request when `waf` gives no answer in time, and while the circuit breaker is
|
||||
open. Its default, `allow`, keeps the site answering.
|
||||
|
||||
`rproxy` reads only its `SWW_RPROXY_` settings, which `docs/settings.md` lists.
|
||||
One of them is required: `SWW_RPROXY_WAF_BACKEND`, `waf`'s URL. It has no
|
||||
@@ -2594,7 +2627,8 @@ only. The image runs it by default (see "Deployment").
|
||||
4. It refuses a request with a method it does not pass on, or one it cannot put
|
||||
to `waf` as the visitor sent it (see "Requests `rproxy` refuses itself").
|
||||
5. It reads up to the first 64 KiB of the body into memory.
|
||||
6. It sends `waf` the question (see "The question"). It speaks HTTP/1.1 to
|
||||
6. It sends `waf` the question (see "The question"), unless the circuit breaker
|
||||
keeps it from asking (see "The circuit breaker"). It speaks HTTP/1.1 to
|
||||
`waf`, whatever `waf` offers. Go's HTTP/2 client would refuse to send a
|
||||
visitor's `Upgrade` header, and most `Connection` headers. The question never
|
||||
carries the visitor's `Trailer`.
|
||||
@@ -2668,23 +2702,24 @@ log line has status `0`.
|
||||
### Without an answer from `waf`
|
||||
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides what is done with the
|
||||
request only when `waf`:
|
||||
request when `waf`:
|
||||
|
||||
- cannot be connected to;
|
||||
- does not answer at all;
|
||||
- does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, which covers
|
||||
connecting, sending the question and reading the whole answer.
|
||||
|
||||
An answer whose body breaks off counts as no answer at all.
|
||||
An answer whose body breaks off counts as no answer at all. The setting also
|
||||
decides, without asking `waf`, while the circuit breaker is open.
|
||||
|
||||
`allow`, the default, passes the request to the app. `deny` answers `503`.
|
||||
Either way `rproxy` writes a process warning, its log line has
|
||||
`waf_backend_no_answer`, and it sends no report.
|
||||
|
||||
A visitor can make `waf` slow. A request that keeps `waf` busy past
|
||||
`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, like any request while `waf` is overloaded,
|
||||
gets `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. With `deny`, such a request
|
||||
gets `503` instead of reaching the app.
|
||||
`waf` taking longer than `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` to decide a request
|
||||
is a bug in `waf`. Such a request gets
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`, like any request while `waf` is
|
||||
overloaded. With `deny`, it gets `503` instead of reaching the app.
|
||||
|
||||
A path under `/_sww/` that `allow` would pass on is answered `404` instead, so
|
||||
that the app never sees that prefix.
|
||||
@@ -2699,6 +2734,28 @@ A visitor that goes away while `waf` is being asked is no failure of `waf`.
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` does not apply. Its log line has
|
||||
status `0` and no `disposition`.
|
||||
|
||||
### The circuit breaker
|
||||
|
||||
`rproxy` stops asking `waf` for a while when `waf` keeps failing to answer. An
|
||||
ask fails when `waf` gives no answer in time (see "Without an answer from
|
||||
`waf`"), or an answer `rproxy` cannot pass on. Any other answer in time
|
||||
succeeds, whatever it decides. A visitor that goes away while `waf` is being
|
||||
asked counts as neither.
|
||||
|
||||
The breaker works like this:
|
||||
|
||||
- It opens when 5 asks fail within 5 seconds. Fewer failures never open it.
|
||||
- While it is open, a request gets
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` at once, without asking `waf`.
|
||||
It gets the same process warning and `waf_backend_no_answer` as a request
|
||||
whose ask failed.
|
||||
- While it is open, one request a second still asks `waf`.
|
||||
- Each ask that succeeds clears the failures counted so far, and closes the
|
||||
breaker if it is open.
|
||||
|
||||
The breaker lives in memory, and a restart of `rproxy` closes it. It has no
|
||||
settings.
|
||||
|
||||
### Passing a request to the app
|
||||
|
||||
`rproxy` sends the app at `SWW_RPROXY_UPSTREAM_URL` the visitor's request: the
|
||||
@@ -2761,9 +2818,9 @@ its whole body. The visitor gets that answer at once, and the connection closes
|
||||
after it.
|
||||
|
||||
The largest request line and headers is 32 KiB. A kept-open connection is closed
|
||||
after 120 seconds idle. Neither has a setting. traefik, by default, closes an
|
||||
unused connection after 90 seconds. So traefik closes first, and never sends a
|
||||
request on a connection `rproxy` is closing.
|
||||
after 120 seconds idle. Neither has a setting. The TLS-terminating proxy should
|
||||
close an unused connection sooner, so that it never sends a request on a
|
||||
connection `rproxy` is closing. traefik closes one after 90 seconds by default.
|
||||
|
||||
What the visitor gets when a limit is passed:
|
||||
|
||||
@@ -2814,14 +2871,16 @@ chooses: `waf` runs `waf`, and with it unset the image runs `rproxy`.
|
||||
The usual setup is three containers on one private network:
|
||||
|
||||
- the app;
|
||||
- `rproxy`, which traefik routes to, with `SWW_RPROXY_UPSTREAM_URL` naming the
|
||||
app and `SWW_RPROXY_WAF_BACKEND` naming `waf`;
|
||||
- `rproxy`, which the TLS-terminating proxy forwards to, with
|
||||
`SWW_RPROXY_UPSTREAM_URL` naming the app and `SWW_RPROXY_WAF_BACKEND` naming
|
||||
`waf`;
|
||||
- `waf`, with `SWW_COMMAND=waf` and a volume for its state files.
|
||||
|
||||
Each further app adds its own container and its own `rproxy`, whose
|
||||
`SWW_RPROXY_WAF_BACKEND` names the same `waf` (see "Sites"). Anything on that
|
||||
network can reach an app around `rproxy`, and can ask `waf` questions or send it
|
||||
reports. So the network holds only what you trust.
|
||||
reports. So the network holds only what you trust. `README.md` lists what the
|
||||
TLS-terminating proxy must do.
|
||||
|
||||
### `rproxy` in the app's container
|
||||
|
||||
@@ -2893,10 +2952,10 @@ process that stops at start is started again every second, not in a tight loop.
|
||||
container's output, which `docker logs` shows.
|
||||
- When a service exits, runit runs its `run` script again, and the container
|
||||
does not stop. Beside an app, the other process keeps running.
|
||||
- While `rproxy` is down, traefik answers `502`. While the app is down, `rproxy`
|
||||
answers `502`, which is not a client offence. While `waf` is down,
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an answer
|
||||
from `waf`").
|
||||
- While `rproxy` is down, the TLS-terminating proxy answers `502`. While the app
|
||||
is down, `rproxy` answers `502`, which is not a client offence. While `waf` is
|
||||
down, `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an
|
||||
answer from `waf`").
|
||||
- Only faults found at start stop `waf`:
|
||||
- invalid configuration;
|
||||
- a lookup database that is missing or unreadable;
|
||||
@@ -2929,23 +2988,26 @@ checks:
|
||||
- With `SWW_COMMAND` unset, empty or `rproxy`, it passes while `rproxy` answers
|
||||
that request on `127.0.0.1`, at the port in `SWW_RPROXY_LISTEN_ADDR`, and the
|
||||
app accepts connections at `SWW_RPROXY_UPSTREAM_URL`. It does not ask `waf`,
|
||||
so that the site stays in traefik's routes while `waf` is down.
|
||||
so that the site stays in the TLS-terminating proxy's routes while `waf` is
|
||||
down.
|
||||
- With any other value, it fails with a message naming `SWW_COMMAND` and its
|
||||
value.
|
||||
|
||||
traefik sends a container no requests until it shows as healthy. So the check
|
||||
runs every second from the container's start until it first passes, for up to a
|
||||
minute, and every 30 seconds after that. An app with a health check of its own
|
||||
can replace the image's `HEALTHCHECK` with one that checks both.
|
||||
A TLS-terminating proxy that reads Docker's health status, as traefik does,
|
||||
sends a container no requests until it shows as healthy. So the check runs every
|
||||
second from the container's start until it first passes, for up to a minute, and
|
||||
every 30 seconds after that. An app with a health check of its own can replace
|
||||
the image's `HEALTHCHECK` with one that checks both.
|
||||
|
||||
### Ports
|
||||
|
||||
`rproxy` listens on no port but the one in `SWW_RPROXY_LISTEN_ADDR`, 8080 by
|
||||
default, which traefik routes to. It passes requests to the app at
|
||||
`SWW_RPROXY_UPSTREAM_URL`, `http://127.0.0.1:8081` by default. Another port may
|
||||
be set: the health check follows the setting, and traefik's port label
|
||||
(`traefik.http.services.<name>.loadbalancer.server.port`) must name it. An app
|
||||
beside `rproxy` must leave that port free.
|
||||
default, which the TLS-terminating proxy forwards to. It passes requests to the
|
||||
app at `SWW_RPROXY_UPSTREAM_URL`, `http://127.0.0.1:8081` by default. Another
|
||||
port may be set: the health check follows the setting, and the TLS-terminating
|
||||
proxy must name it, as traefik's port label
|
||||
(`traefik.http.services.<name>.loadbalancer.server.port`) does. An app beside
|
||||
`rproxy` must leave that port free.
|
||||
|
||||
`waf` listens on no port but the one in `SWW_WAF_LISTEN_ADDR`, 8080 by default.
|
||||
The questions, the reports and its own endpoints all come there.
|
||||
@@ -2967,8 +3029,8 @@ host directory, keeps them across deploys. A host directory mounted there needs
|
||||
no change of owner, and files an earlier owner left in it can be read and
|
||||
replaced. The volume holds a few tens of MiB at most with the defaults, and
|
||||
needs no backup beyond the host's own. The image declares no volume, since every
|
||||
app image built on it would inherit it. `rproxy` keeps nothing, and needs no
|
||||
volume.
|
||||
app image built on it would inherit it. `rproxy` writes nothing to disk, and
|
||||
needs no volume.
|
||||
|
||||
`waf` keeps its cache files in `SWW_WAF_CACHE_DIR`, `/var/cache/sww` by default,
|
||||
which the `run` script makes. The cache directory needs no volume and can be
|
||||
@@ -2986,9 +3048,9 @@ in `docs/settings.md` shows the commands.
|
||||
### Forwarded headers
|
||||
|
||||
The app's TCP peer is `rproxy`, and the `X-Forwarded-For` the app receives ends
|
||||
with traefik's address, which `rproxy` adds after the visitor's. So the app must
|
||||
trust `rproxy`'s address for forwarded headers, besides the private address
|
||||
ranges traefik is on.
|
||||
with the TLS-terminating proxy's address, which `rproxy` adds after the
|
||||
visitor's. So the app must trust `rproxy`'s address for forwarded headers,
|
||||
besides the private address ranges the TLS-terminating proxy is on.
|
||||
|
||||
- In a container of its own, `rproxy`'s address is on the private network. The
|
||||
usual default for a trusted-proxy setting, the private ranges, covers it.
|
||||
|
||||
+2
-2
@@ -34,7 +34,7 @@ on one line:
|
||||
|
||||
```json
|
||||
{
|
||||
"instance": "fsn1/waf",
|
||||
"instance": "host1/waf",
|
||||
"time": "2026-10-06T12:00:50.123461Z",
|
||||
"event": "ban",
|
||||
"host": "app.example",
|
||||
@@ -109,7 +109,7 @@ on one line:
|
||||
## Slack and ntfy
|
||||
|
||||
Slack and ntfy get a message with a title and a text. The title is the instance
|
||||
and the event, such as `fsn1/waf: ban`. The text is the `reason`, then one line
|
||||
and the event, such as `host1/waf: ban`. The text is the `reason`, then one line
|
||||
for each of these that the alert has:
|
||||
|
||||
- the host, the client, the netblock and the country;
|
||||
|
||||
+11
-5
@@ -26,7 +26,7 @@ These questions write no line:
|
||||
A `report` line:
|
||||
|
||||
```
|
||||
{"type":"request","waf_line":"report","time":"2026-10-03T12:00:00.123Z","instance":"fsn1/waf","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"core_rule_set_score":0,"duration_total_ms":3.217,"duration_checks_ms":0.041,"duration_core_rule_set_ms":0.029,"duration_upstream_total_ms":3.104}
|
||||
{"type":"request","waf_line":"report","time":"2026-10-03T12:00:00.123Z","instance":"host1/waf","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"core_rule_set_score":0,"duration_total_ms":3.217,"duration_checks_ms":0.041,"duration_core_rule_set_ms":0.029,"duration_upstream_total_ms":3.104}
|
||||
```
|
||||
|
||||
### Every line
|
||||
@@ -68,7 +68,8 @@ Every one of these lines has:
|
||||
### Request detail
|
||||
|
||||
- `request_id`: the id `rproxy` made for the request when it arrived.
|
||||
- `peer_ip`: the address of `rproxy`'s TCP peer, normally traefik.
|
||||
- `peer_ip`: the address of `rproxy`'s TCP peer, normally the TLS-terminating
|
||||
proxy.
|
||||
- `forwarded_for`: the `X-Forwarded-For` header the visitor's request came with.
|
||||
- `client_group`: the client as rate limiting counts it, an IPv4 address as a
|
||||
/32 or an IPv6 group.
|
||||
@@ -102,7 +103,8 @@ Every one of these lines has:
|
||||
ban this request made or made permanent;
|
||||
- `country_denied`, `client_is_rate_limited`, `blocked_by_block_rule` and
|
||||
`blocked_by_core_rule_set`: refused by that check;
|
||||
- `request_header_too_large`: over a header limit;
|
||||
- `request_line_too_long`: a request line over the header limits;
|
||||
- `request_header_too_large`: headers over the header limits;
|
||||
- `own_endpoint`: answered at one of `waf`'s own endpoints, or `404` for a
|
||||
path under `/_sww/`.
|
||||
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
||||
@@ -124,7 +126,10 @@ Every one of these lines has:
|
||||
- `rate_limits`: `window`, `count` and `limit`;
|
||||
- `trap_paths`: `path`, the trap path asked for;
|
||||
- `rule_files`: `rule_id` and `rule_action`, of the rule that refused it;
|
||||
- `header_limits`: `fields`, `fields_limit`, `cookies` and `cookies_limit`;
|
||||
- `header_limits`: for a request line, `request_line_bytes` and
|
||||
`request_line_bytes_limit`; for headers, `longest_header_line_bytes`,
|
||||
`header_line_bytes_limit`, `fields`, `fields_limit`, `cookies` and
|
||||
`cookies_limit`;
|
||||
- `core_rule_set`: `score`, and `threshold`,
|
||||
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`;
|
||||
- `refusal_limit`: `window`, `count` and `limit`, of the refusal limit;
|
||||
@@ -233,7 +238,8 @@ meanings where they differ:
|
||||
- `disposition`: what was done with the request: `allow`, `deny` or `close`. It
|
||||
is `waf`'s answer, or the value of
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. It is left out when:
|
||||
- `waf` was not asked, as for the health check;
|
||||
- `waf` was not asked and `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` did
|
||||
not apply, as for the health check;
|
||||
- the visitor went away while `waf` was being asked;
|
||||
- `waf`'s answer had no valid `X-Sww-Disposition`;
|
||||
- `rproxy` could not pass on `waf`'s answer, and answered `502`.
|
||||
|
||||
+21
-19
@@ -34,7 +34,7 @@ chooses between the two (see "The image's command" below).
|
||||
such as `:9000`. Leave the host out (see "Ports" in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_WAF_INSTANCE_NAME` (default: the container's host name): the `instance`
|
||||
name in each line `waf` logs, and in each alert and metric, such as
|
||||
`fsn1/waf`. Valid UTF-8. No control character while `SWW_WAF_ALERT_NTFY_URL`
|
||||
`host1/waf`. Valid UTF-8. No control character while `SWW_WAF_ALERT_NTFY_URL`
|
||||
is set. 1 to 48 printable ASCII characters without a space while
|
||||
`SWW_WAF_LOG_REMOTE_URL` is set and `SWW_WAF_LOG_REMOTE_APP_NAME` is not.
|
||||
- `SWW_WAF_MODE` (default `enforce`): `enforce` or `observe`.
|
||||
@@ -150,7 +150,7 @@ cut off; the client is banned.
|
||||
- `SWW_WAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off`
|
||||
stops the start while a setting that needs a lookup is set:
|
||||
- `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES`;
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS`;
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS` set to `true`;
|
||||
- `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming a `country=` tag,
|
||||
or an `asn=` tag while `SWW_WAF_ROUTEWATCH_URL` is unset;
|
||||
- the two byte percentage lists and `SWW_WAF_ASN_LIMIT_PERCENT_URL`;
|
||||
@@ -159,11 +159,14 @@ cut off; the client is banned.
|
||||
- `SWW_WAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in
|
||||
its `.mmdb` form, for `SWW_WAF_LOOKUP_SOURCE=file`.
|
||||
- `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` (default `800`): how long a
|
||||
request waits for its client's first answer from GeoJS, and how long a request
|
||||
to GeoJS may take. A time in milliseconds; never `off`.
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the
|
||||
request waits for its client's first answer from GeoJS. A time in
|
||||
milliseconds; never `off`. It does not cut off the request to GeoJS, whose
|
||||
answer is kept when it comes later.
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `true`): `true` has `rproxy` pass the
|
||||
app the client's AS number in `X-Client-ASN` and its country in
|
||||
`X-Client-Country`.
|
||||
`X-Client-Country`. The default is `false` while `SWW_WAF_LOOKUP_SOURCE` is
|
||||
`off`. With GeoJS, a new client's first request waits for its answer, up to
|
||||
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`.
|
||||
|
||||
## Country list
|
||||
|
||||
@@ -249,10 +252,9 @@ tag that no request can get stops the start. A request can get only these:
|
||||
- `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS` (default empty): path prefixes the Core
|
||||
Rule Set does not inspect.
|
||||
- `SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE` (default `off`): `off`, or how
|
||||
much of a request body the Core Rule Set reads, a size of at most `64K`. A
|
||||
question carries no more of a body than 64 KiB, and the Core Rule Set reads at
|
||||
most 64 KiB less one byte of it, whatever the value (see "The answer" in
|
||||
[`SPEC.md`](../SPEC.md)).
|
||||
much of a request body the Core Rule Set reads, a size of at most `8K`. A
|
||||
larger size would let one request keep it busy past `rproxy`'s timeout (see
|
||||
"Answering in time" in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule
|
||||
Set exclusions, written as Coraza directives. It is loaded before the Core
|
||||
Rule Set's rules, so it holds only `SecAction` and `SecRule` lines that
|
||||
@@ -421,8 +423,9 @@ A few limits are fixed rather than settings:
|
||||
- at most 20,000 anomaly counters are kept;
|
||||
- each alert destination's queue holds at most 1,000 alerts;
|
||||
- `waf` holds at most 10,000 answered requests awaiting their reports;
|
||||
- `waf` refuses a request with more than 50 cookies or more than 100 header
|
||||
fields (see "Answering in time" under "`waf`" in [`SPEC.md`](../SPEC.md));
|
||||
- `waf` refuses a request with a request line or header line over 8 KiB, more
|
||||
than 50 cookies or more than 100 header fields (see "Answering in time" under
|
||||
"`waf`" in [`SPEC.md`](../SPEC.md));
|
||||
- `rproxy` takes a request line and headers of at most 32 KiB, and closes a
|
||||
kept-open connection idle for 120 seconds;
|
||||
- `rproxy` reads at most 1 MiB of the body of `waf`'s answer, and answers `502`
|
||||
@@ -450,10 +453,9 @@ start, and an invalid value stops the start. Any can be given as a file, as a
|
||||
for `waf`'s answer, to a question and to a report. A time in milliseconds;
|
||||
never `off`.
|
||||
- `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` (default `allow`): what is done
|
||||
with a request when `waf` cannot be connected to, does not answer at all, or
|
||||
does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`. `allow` passes it
|
||||
to the app, and `deny` answers `503`. It does not apply to an answer `waf`
|
||||
gives, such as an error.
|
||||
with a request when `waf` gives no answer in time, and while the circuit
|
||||
breaker for asks to `waf` is open. `allow` passes it to the app, and `deny`
|
||||
answers `503`. It does not apply to an answer `waf` gives, such as an error.
|
||||
- `SWW_RPROXY_MAX_REQUEST_SIZE` (default `100M`): the largest request body. A
|
||||
size or `off`.
|
||||
- `SWW_RPROXY_MAX_RESPONSE_SIZE` (default `1G`): the largest response body. A
|
||||
@@ -475,9 +477,9 @@ start, and an invalid value stops the start. Any can be given as a file, as a
|
||||
`10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`): the netblocks whose
|
||||
`X-Forwarded-For` is believed. Set but empty, it trusts nothing. A client
|
||||
outside these netblocks cannot claim another address. Where other containers
|
||||
reach `rproxy` directly, or visitors on a private network reach traefik, they
|
||||
can name another address in the header. Narrowing the setting to traefik's own
|
||||
network closes that gap.
|
||||
reach `rproxy` directly, or visitors on a private network reach the
|
||||
TLS-terminating proxy, they can name another address in the header. Narrowing
|
||||
the setting to the TLS-terminating proxy's own network closes that gap.
|
||||
|
||||
## The image's command
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ require (
|
||||
github.com/oschwald/maxminddb-golang/v2 v2.7.0
|
||||
github.com/prometheus/client_golang v1.24.1
|
||||
github.com/spf13/viper v1.21.0
|
||||
golang.org/x/sys v0.48.0
|
||||
)
|
||||
|
||||
require (
|
||||
@@ -47,7 +48,6 @@ require (
|
||||
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
|
||||
golang.org/x/net v0.60.0 // indirect
|
||||
golang.org/x/sync v0.23.0 // indirect
|
||||
golang.org/x/sys v0.48.0 // indirect
|
||||
golang.org/x/text v0.42.0 // indirect
|
||||
google.golang.org/protobuf v1.36.11 // indirect
|
||||
rsc.io/binaryregexp v0.2.0 // indirect
|
||||
|
||||
@@ -39,7 +39,7 @@ const (
|
||||
ntfyURL = "https://ntfy.example/sww-alerts"
|
||||
ntfyToken = "tk_0123456789abcdefghijklmnopq"
|
||||
// instance is the instance name every alert gives.
|
||||
instance = "fsn1app1/blog"
|
||||
instance = "host1/blog"
|
||||
// started is when each test starts, as an alert gives it, and
|
||||
// anHourOn an hour later.
|
||||
started = "2000-01-01T00:00:00Z"
|
||||
@@ -1160,8 +1160,8 @@ func TestEachDestinationIsSentOnlyTheAlertsWaitingForIt(t *testing.T) {
|
||||
synctest.Wait()
|
||||
|
||||
wantBodies(t, standIns[alerts.DestinationSlack],
|
||||
`{"text":"*fsn1app1/blog: file_error*\nsecond"}`,
|
||||
`{"text":"*fsn1app1/blog: file_error*\nthird"}`)
|
||||
`{"text":"*host1/blog: file_error*\nsecond"}`,
|
||||
`{"text":"*host1/blog: file_error*\nthird"}`)
|
||||
wantBodies(t, standIns[alerts.DestinationNtfy], "fourth")
|
||||
|
||||
// alerts.json then lists Slack and ntfy alone, with no alert waiting.
|
||||
@@ -1477,7 +1477,7 @@ func anAlertForEachEvent() []eventMessage {
|
||||
Reason: "requests per hour over the limit of 10000",
|
||||
Detail: map[string]any{"mode": "observe"},
|
||||
},
|
||||
"fsn1app1/blog: ban", "default no_entry",
|
||||
"host1/blog: ban", "default no_entry",
|
||||
"requests per hour over the limit of 10000\nclient: 203.0.113.9\n" +
|
||||
"netblock: 203.0.113.0/24\ncountry: DE\nmode: observe",
|
||||
},
|
||||
@@ -1489,7 +1489,7 @@ func anAlertForEachEvent() []eventMessage {
|
||||
), Country: "FR",
|
||||
Reason: "matched the rule env-file",
|
||||
},
|
||||
"fsn1app1/blog: permanent_ban", "high no_entry",
|
||||
"host1/blog: permanent_ban", "high no_entry",
|
||||
"matched the rule env-file\nclient: 198.51.100.7\n" +
|
||||
"netblock: 198.51.100.7/32\ncountry: FR",
|
||||
},
|
||||
@@ -1500,7 +1500,7 @@ func anAlertForEachEvent() []eventMessage {
|
||||
Reason: "refused by the Core Rule Set",
|
||||
Host: "app.example",
|
||||
},
|
||||
"fsn1app1/blog: core_rule_set_match", "default shield",
|
||||
"host1/blog: core_rule_set_match", "default shield",
|
||||
"refused by the Core Rule Set\nhost: app.example\nclient: 192.0.2.1\n" +
|
||||
"netblock: 192.0.2.1/32",
|
||||
},
|
||||
@@ -1509,7 +1509,7 @@ func anAlertForEachEvent() []eventMessage {
|
||||
Event: alerts.EventAnomaly, Netblock: netip.MustParsePrefix("192.0.2.0/24"),
|
||||
Reason: "requests per minute over the threshold of 5000",
|
||||
},
|
||||
"fsn1app1/blog: anomaly", "high chart_with_upwards_trend",
|
||||
"host1/blog: anomaly", "high chart_with_upwards_trend",
|
||||
"requests per minute over the threshold of 5000\nnetblock: 192.0.2.0/24",
|
||||
},
|
||||
{
|
||||
@@ -1518,7 +1518,7 @@ func anAlertForEachEvent() []eventMessage {
|
||||
Netblock: netip.MustParsePrefix("192.0.2.2/32"),
|
||||
Reason: "listed by a DNS blocklist",
|
||||
},
|
||||
"fsn1app1/blog: reputation_hit", "low label",
|
||||
"host1/blog: reputation_hit", "low label",
|
||||
"listed by a DNS blocklist\nclient: 192.0.2.2\nnetblock: 192.0.2.2/32",
|
||||
},
|
||||
{
|
||||
@@ -1526,7 +1526,7 @@ func anAlertForEachEvent() []eventMessage {
|
||||
Event: alerts.EventSourceFailure, Reason: "asking GeoJS failed",
|
||||
Detail: map[string]any{"source": "geojs"},
|
||||
},
|
||||
"fsn1app1/blog: source_failure", "high warning",
|
||||
"host1/blog: source_failure", "high warning",
|
||||
"asking GeoJS failed\nsource: geojs",
|
||||
},
|
||||
{
|
||||
@@ -1537,7 +1537,7 @@ func anAlertForEachEvent() []eventMessage {
|
||||
"file": "/etc/sww/rules.d/50-app.rules", "error": "line 2: no action",
|
||||
},
|
||||
},
|
||||
"fsn1app1/blog: file_error", "high warning",
|
||||
"host1/blog: file_error", "high warning",
|
||||
"a rule file has an error, and the rules stay as they were\n" +
|
||||
"file: /etc/sww/rules.d/50-app.rules\nerror: line 2: no action",
|
||||
},
|
||||
|
||||
+11
-8
@@ -84,7 +84,8 @@ type WAF struct {
|
||||
// client's first answer from GeoJS while a setting needs it
|
||||
// (SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS), which cannot be off.
|
||||
// AddLookupHeaders is true when the app is passed the client's AS number
|
||||
// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS).
|
||||
// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS), by default unless
|
||||
// LookupSource is off.
|
||||
LookupSource string
|
||||
LookupDBPath string
|
||||
ASNAndCountryLookupTimeout time.Duration
|
||||
@@ -364,8 +365,8 @@ var (
|
||||
"it names the file to look clients up in",
|
||||
)
|
||||
errDBPathUnused = errors.New("only file reads it")
|
||||
errOver64K = errors.New(
|
||||
"is more than 64K, the most waf gets of a body")
|
||||
errOver8K = errors.New(
|
||||
"is more than 8K, the most the Core Rule Set can read in time")
|
||||
errNotDurationAboveZero = errors.New(
|
||||
"is not a duration above zero, such as 1h or 7d")
|
||||
errNotNumberAboveZero = errors.New(
|
||||
@@ -549,7 +550,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
|
||||
"SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS",
|
||||
"800",
|
||||
),
|
||||
AddLookupHeaders: env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS", "false"),
|
||||
ExclusivelyAllowedCountries: env.countries(
|
||||
"SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES", ""),
|
||||
TagLists: env.tagLists("SWW_WAF_TAG_LISTS"),
|
||||
@@ -724,6 +724,9 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
|
||||
|
||||
cfg.LogRemoteAppName = env.appName("SWW_WAF_LOG_REMOTE_APP_NAME",
|
||||
cfg.InstanceName, cfg.LogRemoteURL != nil)
|
||||
// The app gets the headers by default, unless no client is looked up.
|
||||
cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS",
|
||||
strconv.FormatBool(cfg.LookupSource != off))
|
||||
cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action(
|
||||
"SWW_WAF_BLOCKLIST_ACTION", "deny")
|
||||
cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action(
|
||||
@@ -1598,16 +1601,16 @@ func durationOrDays(value string) (time.Duration, error) {
|
||||
}
|
||||
|
||||
// parseCoreRuleSetBodyInspectionSize reads the most of a request body the Core Rule Set
|
||||
// reads: a size as parseSize reads it, or off, but at most 64K, since waf
|
||||
// gets no more of a body than that.
|
||||
// reads: a size as parseSize reads it, or off, but at most 8K, so that one
|
||||
// request cannot keep the Core Rule Set busy past rproxy's timeout.
|
||||
func parseCoreRuleSetBodyInspectionSize(value string) (int64, error) {
|
||||
limit, err := parseSize(value)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if limit > 64*kibibyte {
|
||||
return 0, fmt.Errorf("%q %w", value, errOver64K)
|
||||
if limit > 8*kibibyte {
|
||||
return 0, fmt.Errorf("%q %w", value, errOver8K)
|
||||
}
|
||||
|
||||
return limit, nil
|
||||
|
||||
+27
-20
@@ -218,7 +218,7 @@ const (
|
||||
// instance is an SWW_WAF_INSTANCE_NAME that is a valid app name too, and
|
||||
// remoteURL an SWW_WAF_LOG_REMOTE_URL, for the tests that send the lines.
|
||||
const (
|
||||
instance = "fsn1app1/blog"
|
||||
instance = "host1/blog"
|
||||
remoteURL = "syslog+udp://192.0.2.1:514"
|
||||
)
|
||||
|
||||
@@ -287,6 +287,7 @@ func TestDefaults(t *testing.T) {
|
||||
})
|
||||
wantLookupSettings(t, cfg, config.WAF{
|
||||
LookupSource: defaultLookupSource, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
|
||||
AddLookupHeaders: true,
|
||||
})
|
||||
wantByteLimitSettings(t, cfg, config.WAF{
|
||||
BytesLimitPerMinute: 0, BytesLimitPerHour: 50 << 30,
|
||||
@@ -615,7 +616,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
||||
coreRuleSetScoreThreshold: "10",
|
||||
coreRuleSetDisabledRules: "942100, 920350",
|
||||
coreRuleSetExemptPaths: "/api/, /static/",
|
||||
coreRuleSetBodyInspectionSize: "32K",
|
||||
coreRuleSetBodyInspectionSize: "4K",
|
||||
exclusionsFile: "/etc/sww/crs-exclusions.conf",
|
||||
},
|
||||
config.WAF{
|
||||
@@ -624,7 +625,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
||||
CoreRuleSetScoreThreshold: 10,
|
||||
CoreRuleSetDisabledRules: []int{942100, 920350},
|
||||
CoreRuleSetExemptPaths: []string{"/api/", "/static/"},
|
||||
CoreRuleSetBodyInspectionSize: 32 << 10,
|
||||
CoreRuleSetBodyInspectionSize: 4 << 10,
|
||||
CoreRuleSetExclusionsFile: "/etc/sww/crs-exclusions.conf",
|
||||
},
|
||||
},
|
||||
@@ -761,15 +762,15 @@ func TestInvalidAppsStopsTheStart(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCoreRuleSetBodyInspectionSizeOf64K(t *testing.T) {
|
||||
func TestCoreRuleSetBodyInspectionSizeOf8K(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(
|
||||
t,
|
||||
environment{coreRuleSetBodyInspectionSize: "64K"},
|
||||
environment{coreRuleSetBodyInspectionSize: "8K"},
|
||||
)
|
||||
if cfg.CoreRuleSetBodyInspectionSize != 64<<10 {
|
||||
t.Errorf("64K read as %d", cfg.CoreRuleSetBodyInspectionSize)
|
||||
if cfg.CoreRuleSetBodyInspectionSize != 8<<10 {
|
||||
t.Errorf("8K read as %d", cfg.CoreRuleSetBodyInspectionSize)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -818,11 +819,13 @@ func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
|
||||
coreRuleSetBodyInspectionSize, "128KB",
|
||||
`"128KB" is not a size such as 512K, 100M or 5G, or off`,
|
||||
},
|
||||
{coreRuleSetBodyInspectionSize, "65K", `"65K" is more than 64K, the most waf ` +
|
||||
`gets of a body`},
|
||||
{
|
||||
coreRuleSetBodyInspectionSize, "65537",
|
||||
`"65537" is more than 64K, the most waf gets of a body`,
|
||||
coreRuleSetBodyInspectionSize, "9K",
|
||||
`"9K" is more than 8K, the most the Core Rule Set can read in time`,
|
||||
},
|
||||
{
|
||||
coreRuleSetBodyInspectionSize, "8193",
|
||||
`"8193" is more than 8K, the most the Core Rule Set can read in time`,
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||
@@ -844,11 +847,11 @@ func TestInstanceNameAndLoggedHeadersAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
instanceName: "fsn1app1/blog",
|
||||
instanceName: "host1/blog",
|
||||
logRequestHeaders: " Accept , X-Custom",
|
||||
})
|
||||
|
||||
if cfg.InstanceName != "fsn1app1/blog" ||
|
||||
if cfg.InstanceName != "host1/blog" ||
|
||||
!slices.Equal(
|
||||
cfg.LogRequestHeaders,
|
||||
[]string{"accept", "x-custom"},
|
||||
@@ -1021,14 +1024,14 @@ func TestInstanceNameNotAnAppNameStopsTheStartOnlyWhileSending(
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
const spaced = "fsn1 app1"
|
||||
const spaced = "host1 blog"
|
||||
|
||||
sending := environment{logRemoteURL: remoteURL, instanceName: spaced}
|
||||
|
||||
_, err := config.WAFFromEnvironment(sending.viper())
|
||||
|
||||
want := logRemoteAppName + `: is unset, and ` + instanceName +
|
||||
` "fsn1 app1", its default, is not 1 to 48 printable ASCII characters ` +
|
||||
` "host1 blog", its default, is not 1 to 48 printable ASCII characters ` +
|
||||
`without a space, such as blog`
|
||||
if err == nil || err.Error() != want {
|
||||
t.Errorf("error %v, want %s", err, want)
|
||||
@@ -1330,13 +1333,13 @@ func TestInstanceNameWithAControlCharacterStopsTheStartOnlyWithNtfySet(
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
const name = "fsn1app1\r"
|
||||
const name = "host1\r"
|
||||
|
||||
_, err := config.WAFFromEnvironment(environment{
|
||||
instanceName: name, alertNtfyURL: "https://ntfy.example/sww-alerts",
|
||||
}.viper())
|
||||
|
||||
want := instanceName + `: "fsn1app1\r" holds a control character, such as the ` +
|
||||
want := instanceName + `: "host1\r" holds a control character, such as the ` +
|
||||
`carriage return of a Windows line end, and is sent to ntfy in a header ` +
|
||||
`while ` + alertNtfyURL + ` is set`
|
||||
if err == nil || err.Error() != want {
|
||||
@@ -1395,13 +1398,13 @@ func TestLookupSettingsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
asnAndCountryLookupTimeout: "500", addLookupHeaders: enabled,
|
||||
asnAndCountryLookupTimeout: "500", addLookupHeaders: disabled,
|
||||
})
|
||||
wantLookupSettings(t, cfg, config.WAF{
|
||||
LookupSource: defaultLookupSource, ASNAndCountryLookupTimeout: 500 * time.Millisecond,
|
||||
AddLookupHeaders: true,
|
||||
})
|
||||
|
||||
// With no client looked up, the headers are off unless set.
|
||||
cfg = wafFromEnvironment(t, environment{lookupSource: off})
|
||||
wantLookupSettings(t, cfg, config.WAF{
|
||||
LookupSource: off, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
|
||||
@@ -1422,6 +1425,10 @@ func TestLookupDBPathGoesWithTheFileSourceAlone(t *testing.T) {
|
||||
cfg.LookupSource, cfg.LookupDBPath, path)
|
||||
}
|
||||
|
||||
if !cfg.AddLookupHeaders {
|
||||
t.Error("headers off by default with lookups from file, want on")
|
||||
}
|
||||
|
||||
for _, tc := range []struct {
|
||||
env environment
|
||||
want string
|
||||
@@ -2927,7 +2934,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
lookupSource: defaultLookupSource,
|
||||
lookupDBPath: "",
|
||||
asnAndCountryLookupTimeout: "800",
|
||||
addLookupHeaders: disabled,
|
||||
addLookupHeaders: enabled,
|
||||
allowedCountries: "",
|
||||
tagLists: "",
|
||||
tagDeny: "",
|
||||
|
||||
@@ -481,12 +481,21 @@ func (c *CoreRuleSet) Inspect(
|
||||
return Result{RuleIDs: ids, Score: score(tx)}, read, err
|
||||
}
|
||||
|
||||
// mostJSONRead is the largest JSON body the Core Rule Set reads, even while
|
||||
// SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is larger. Coraza names each
|
||||
// value in a JSON body by the whole path to it, so what it gives the Core
|
||||
// Rule Set to read can outgrow the body: up to 8 times its size, and up to
|
||||
// 4 KiB for a small body. 1 KiB of JSON thus gives at most 8 KiB, no more
|
||||
// than the largest body of another kind.
|
||||
const mostJSONRead = 1 << 10
|
||||
|
||||
// readBody reads body, the body of the request in tx, which has run on
|
||||
// the request's headers, while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is
|
||||
// set and the body is of a kind the Core Rule Set reads: form data and
|
||||
// multipart, of which it reads the first c.bodyInspectionSize bytes, and
|
||||
// JSON and XML, which it reads only when they are no longer than that,
|
||||
// since they cannot be read in part.
|
||||
// since they cannot be read in part. The Core Rule Set reads no JSON body
|
||||
// longer than mostJSONRead either.
|
||||
// readBody reads one byte past c.bodyInspectionSize, to tell which they
|
||||
// are, gives the Core Rule Set what it reads, and returns what it read and
|
||||
// the error that ended the reading early, if one did.
|
||||
@@ -499,6 +508,8 @@ func (c *CoreRuleSet) readBody(
|
||||
}
|
||||
|
||||
inPart := false
|
||||
// largest is the largest JSON or XML body the Core Rule Set reads.
|
||||
largest := c.bodyInspectionSize
|
||||
// How the body is read is a variable of the transaction, which only
|
||||
// Coraza's interface for plugins reads.
|
||||
state := tx.(plugintypes.TransactionState) //nolint:forcetypeassert // every one is
|
||||
@@ -506,14 +517,16 @@ func (c *CoreRuleSet) readBody(
|
||||
switch state.Variables().RequestBodyProcessor().Get() {
|
||||
case "URLENCODED", "MULTIPART":
|
||||
inPart = true
|
||||
case "JSON", "XML":
|
||||
case "JSON":
|
||||
largest = min(largest, mostJSONRead)
|
||||
case "XML":
|
||||
default:
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
read, err := io.ReadAll(io.LimitReader(body, c.bodyInspectionSize+1))
|
||||
|
||||
if inPart || (err == nil && int64(len(read)) <= c.bodyInspectionSize) {
|
||||
if inPart || (err == nil && int64(len(read)) <= largest) {
|
||||
// Coraza holds what it reads of the body in memory, up to the
|
||||
// limit, so this cannot fail.
|
||||
_, _, _ = tx.WriteRequestBody(read)
|
||||
|
||||
@@ -390,22 +390,15 @@ func TestMoreParametersThanCorazaKeepsIsAMatch(t *testing.T) {
|
||||
matched(900300),
|
||||
)
|
||||
|
||||
// So it is with the fields of a form data or JSON body.
|
||||
// So it is with the fields of a form data body. A JSON body the Core
|
||||
// Rule Set reads, of at most mostJSONRead bytes, cannot hold that many.
|
||||
crs = readingBodies(t)
|
||||
|
||||
for _, tc := range []struct{ header, body string }{
|
||||
{formData, strings.Repeat("a=1&", 999) + attack},
|
||||
{jsonBody, `{"a":[` + strings.Repeat("1,", 998) + `1],"id":"` + injection + `"}`},
|
||||
} {
|
||||
wantBody(t, crs, post(tc.header), tc.body, matched(942100), tc.body)
|
||||
}
|
||||
body := strings.Repeat("a=1&", 999) + attack
|
||||
wantBody(t, crs, post(formData), body, matched(942100), body)
|
||||
|
||||
for _, tc := range []struct{ header, body string }{
|
||||
{formData, strings.Repeat("a=1&", 1000) + attack},
|
||||
{jsonBody, `{"a":[` + strings.Repeat("1,", 999) + `1],"id":"` + injection + `"}`},
|
||||
} {
|
||||
wantBody(t, crs, post(tc.header), tc.body, matched(900300), tc.body)
|
||||
}
|
||||
body = strings.Repeat("a=1&", 1000) + attack
|
||||
wantBody(t, crs, post(formData), body, matched(900300), body)
|
||||
}
|
||||
|
||||
func TestRedirectURIMayNameALocalAddress(t *testing.T) {
|
||||
@@ -1512,13 +1505,15 @@ const (
|
||||
wpTags = "/wp-json/wp/v2/tags"
|
||||
)
|
||||
|
||||
// readingBodiesUpTo64K returns the Core Rule Set as sww runs it by default,
|
||||
// readingBodiesUpTo8K returns the Core Rule Set as sww runs it by default,
|
||||
// but reading bodies up to the largest SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE.
|
||||
func readingBodiesUpTo64K(t *testing.T) *coreruleset.CoreRuleSet {
|
||||
func readingBodiesUpTo8K(t *testing.T) *coreruleset.CoreRuleSet {
|
||||
t.Helper()
|
||||
|
||||
return shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: 64<<10 - 1,
|
||||
ParanoiaLevel: 1,
|
||||
DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: mostBodyRead,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1582,7 +1577,7 @@ func TestWordPressAttacksAreRefused(t *testing.T) {
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
|
||||
defaults, bodies := atDefaults(t), readingBodiesUpTo64K(t)
|
||||
defaults, bodies := atDefaults(t), readingBodiesUpTo8K(t)
|
||||
|
||||
for _, attack := range []string{
|
||||
sqlInjection,
|
||||
@@ -1693,7 +1688,7 @@ func wordpressRESTAddresses(route string) []string {
|
||||
func TestWordPressExclusionsFollowEveryRESTAddress(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs := readingBodiesUpTo64K(t)
|
||||
crs := readingBodiesUpTo8K(t)
|
||||
content := `{"content":"` + wordpressBlock + `"}`
|
||||
|
||||
for _, route := range []string{
|
||||
@@ -1919,7 +1914,7 @@ func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(
|
||||
wantResult(t, crs, get(target+command), matched(932235))
|
||||
}
|
||||
|
||||
bodies := readingBodiesUpTo64K(t)
|
||||
bodies := readingBodiesUpTo8K(t)
|
||||
|
||||
for _, p := range wordpressBodyFields(command) {
|
||||
r := request{http.MethodPost, p.target, []string{p.contentType}}
|
||||
@@ -2143,7 +2138,7 @@ func TestWordPressBodyFieldsAreCheckedInTheQuery(t *testing.T) {
|
||||
func TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs := readingBodiesUpTo64K(t)
|
||||
crs := readingBodiesUpTo8K(t)
|
||||
|
||||
for _, v := range wordpressWords() {
|
||||
for _, field := range []string{"title", "excerpt", "name"} {
|
||||
@@ -2262,7 +2257,7 @@ func TestWordPressExclusionsAreNarrow(t *testing.T) {
|
||||
length = "Content-Length: 2"
|
||||
)
|
||||
|
||||
crs := readingBodiesUpTo64K(t)
|
||||
crs := readingBodiesUpTo8K(t)
|
||||
saving := request{
|
||||
http.MethodPost,
|
||||
wpPost,
|
||||
@@ -2473,19 +2468,35 @@ func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
|
||||
func TestJSONAndXMLAreReadOnlyWhenNoLargerThanTheLimit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs := readingBodies(t)
|
||||
// The limit is the body inspection size, but at most mostJSONRead for
|
||||
// JSON. smallSize is a body inspection size under mostJSONRead.
|
||||
const smallSize = 512
|
||||
|
||||
for _, tc := range []struct{ header, head, tail string }{
|
||||
{jsonBody, `{"q":"` + injection + `","pad":"`, `"}`},
|
||||
{xmlBody, "<r><q>" + injection + "</q><pad>", "</pad></r>"},
|
||||
bodies, smallBodies := readingBodies(t), shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: smallSize,
|
||||
})
|
||||
jsonHead, jsonTail := `{"q":"`+injection+`","pad":"`, `"}`
|
||||
|
||||
for _, tc := range []struct {
|
||||
crs *coreruleset.CoreRuleSet
|
||||
header, head, tail string
|
||||
limit int
|
||||
}{
|
||||
{bodies, jsonBody, jsonHead, jsonTail, mostJSONRead},
|
||||
{smallBodies, jsonBody, jsonHead, jsonTail, smallSize},
|
||||
{
|
||||
bodies, xmlBody, "<r><q>" + injection + "</q><pad>", "</pad></r>",
|
||||
bodyInspectionSize,
|
||||
},
|
||||
} {
|
||||
fits := padded(tc.head, tc.tail, bodyInspectionSize)
|
||||
wantBody(t, crs, post(tc.header), fits, matched(942100), fits)
|
||||
fits := padded(tc.head, tc.tail, tc.limit)
|
||||
wantBody(t, tc.crs, post(tc.header), fits, matched(942100), fits)
|
||||
|
||||
larger := padded(tc.head, tc.tail, bodyInspectionSize+1)
|
||||
larger := padded(tc.head, tc.tail, tc.limit+1)
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
tc.crs,
|
||||
post(tc.header),
|
||||
larger,
|
||||
coreruleset.Result{},
|
||||
|
||||
@@ -0,0 +1,436 @@
|
||||
package coreruleset_test
|
||||
|
||||
import (
|
||||
"maps"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/netip"
|
||||
"regexp"
|
||||
"runtime"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
)
|
||||
|
||||
// The bounds on what a visitor can send, as SPEC.md gives them.
|
||||
const (
|
||||
// lineLimit is the longest request line or header line.
|
||||
lineLimit = 8 << 10
|
||||
// headerLimit is the most bytes rproxy reads of a request line and
|
||||
// headers together.
|
||||
headerLimit = 32 << 10
|
||||
// mostCookies and mostHeaderFields are waf's limits on cookies and header
|
||||
// fields.
|
||||
mostCookies = 50
|
||||
mostHeaderFields = 100
|
||||
// mostBodyRead is the largest SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE.
|
||||
mostBodyRead = 8 << 10
|
||||
// mostJSONRead is the largest JSON body the Core Rule Set reads.
|
||||
mostJSONRead = 1 << 10
|
||||
)
|
||||
|
||||
// The Core Rule Set has 1 second for one request. That is the default of
|
||||
// rproxy's SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, 2 seconds, less the default of
|
||||
// SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, 800 milliseconds, and less 200
|
||||
// milliseconds for waf's other checks.
|
||||
//
|
||||
// A busy host slows all work, so the test does not compare an inspection with
|
||||
// 1 second. It compares it with referenceTime, timed on the same thread just
|
||||
// before and after. inspectionTimeLimitMultiple is how many times
|
||||
// referenceTime an inspection may take.
|
||||
//
|
||||
// On the host that runs our checks, referenceTime took at least 78
|
||||
// milliseconds, so 13 times it is about 1 second. With that host fully busy,
|
||||
// no try of the slowest request took more than 9.5 times referenceTime.
|
||||
const inspectionTimeLimitMultiple = 13
|
||||
|
||||
// referencePattern is what referenceTime matches. Like the Core Rule Set's
|
||||
// rules, it is a regular expression with alternatives and repeats.
|
||||
var referencePattern = regexp.MustCompile(
|
||||
`(?i)(union|select|insert|delete)\s+\w+\s+(from|into)\s+\w+[0-9]+;`,
|
||||
)
|
||||
|
||||
// referenceTime returns the processor time the calling thread takes to do a
|
||||
// fixed amount of work unrelated to the Core Rule Set, so that a change to the
|
||||
// rules does not change it. referencePattern matches nowhere in the text, so
|
||||
// each match reads all of it.
|
||||
func referenceTime(t *testing.T) time.Duration {
|
||||
t.Helper()
|
||||
|
||||
referenceText := text(4 << 10)
|
||||
began := threadCPUTime(t)
|
||||
|
||||
for range 200 {
|
||||
if referencePattern.MatchString(referenceText) {
|
||||
t.Fatal("the reference pattern matched")
|
||||
}
|
||||
}
|
||||
|
||||
return threadCPUTime(t) - began
|
||||
}
|
||||
|
||||
// text returns n bytes of letters, digits and spaces, which the Core Rule Set
|
||||
// takes longer over than one letter repeated.
|
||||
func text(n int) string {
|
||||
const words = "select 0x2f from tbl_1 where name like abc_9 "
|
||||
|
||||
return strings.Repeat(words, n/len(words)+1)[:n]
|
||||
}
|
||||
|
||||
// visitorRequest is a request a visitor can send, at the most of one or more
|
||||
// of the bounds above.
|
||||
type visitorRequest struct {
|
||||
name string
|
||||
target string
|
||||
// header holds the headers besides Host.
|
||||
header http.Header
|
||||
// contentType is the body's type, "" for no body.
|
||||
contentType string
|
||||
body string
|
||||
}
|
||||
|
||||
// longestTarget returns a target whose request line, "POST <target>
|
||||
// HTTP/1.1", is lineLimit bytes long.
|
||||
func longestTarget() string {
|
||||
const around = len("POST /?q= HTTP/1.1")
|
||||
|
||||
return "/?q=" + strings.ReplaceAll(text(lineLimit-around), " ", "+")
|
||||
}
|
||||
|
||||
// cookieLine returns a Cookie value whose line, "Cookie: <value>", is
|
||||
// lineLimit bytes long, made of count cookies.
|
||||
func cookieLine(count int) string {
|
||||
const linePrefix = len("Cookie: ")
|
||||
|
||||
cookies := make([]string, count)
|
||||
for i := range cookies {
|
||||
cookies[i] = "c" + strconv.Itoa(i) + "="
|
||||
}
|
||||
|
||||
// Each cookie is followed by "; ", but for the last.
|
||||
room := lineLimit - linePrefix - len(strings.Join(cookies, "; "))
|
||||
for i := range cookies {
|
||||
share := room / count
|
||||
if i == 0 {
|
||||
share += room % count
|
||||
}
|
||||
|
||||
cookies[i] += strings.ReplaceAll(text(share), " ", "_")
|
||||
}
|
||||
|
||||
return strings.Join(cookies, "; ")
|
||||
}
|
||||
|
||||
// headerValue returns a value whose line, "<name>: <value>", is n bytes long.
|
||||
func headerValue(name string, n int) string {
|
||||
return text(n - len(name) - len(": "))
|
||||
}
|
||||
|
||||
// formFields returns a form body of mostBodyRead bytes holding count fields.
|
||||
func formFields(count int) string {
|
||||
fields := make([]string, count)
|
||||
for i := range fields {
|
||||
fields[i] = "f" + strconv.Itoa(i) + "="
|
||||
}
|
||||
|
||||
room := mostBodyRead - len(strings.Join(fields, "&"))
|
||||
for i := range fields {
|
||||
share := room / count
|
||||
if i == 0 {
|
||||
share += room % count
|
||||
}
|
||||
|
||||
fields[i] += strings.ReplaceAll(text(share), " ", "+")
|
||||
}
|
||||
|
||||
return strings.Join(fields, "&")
|
||||
}
|
||||
|
||||
// multipartBody returns a multipart body of mostBodyRead bytes, with the
|
||||
// boundary "b", holding fields of about size bytes each.
|
||||
func multipartBody(size int) string {
|
||||
var body strings.Builder
|
||||
|
||||
const end = "--b--\r\n"
|
||||
|
||||
for i := 0; ; i++ {
|
||||
head := "--b\r\nContent-Disposition: form-data; name=\"f" +
|
||||
strconv.Itoa(i) + "\"\r\n\r\n"
|
||||
|
||||
room := mostBodyRead - body.Len() - len(
|
||||
end,
|
||||
) - len(
|
||||
head,
|
||||
) - len(
|
||||
"\r\n",
|
||||
)
|
||||
if room <= 0 {
|
||||
break
|
||||
}
|
||||
|
||||
body.WriteString(head + text(min(size, room)) + "\r\n")
|
||||
}
|
||||
|
||||
body.WriteString(end)
|
||||
|
||||
return body.String()
|
||||
}
|
||||
|
||||
// jsonArrayOfMostElements returns a JSON array of at most mostJSONRead bytes
|
||||
// holding as many elements as fit, each the number 1.
|
||||
func jsonArrayOfMostElements() string {
|
||||
return "[1" + strings.Repeat(
|
||||
",1",
|
||||
(mostJSONRead-len("[1]"))/len(",1"),
|
||||
) + "]"
|
||||
}
|
||||
|
||||
// jsonOfMostElementsUnderAKey returns a JSON body of mostJSONRead bytes
|
||||
// holding as many elements as fit, each the number 1, in an array under a
|
||||
// key of 6 bytes. Coraza names each element "json.<key>.<index>", so a
|
||||
// longer key makes longer names. 6 bytes is the longest key that keeps the
|
||||
// names and values of this body within the 8 KiB Coraza allows.
|
||||
func jsonOfMostElementsUnderAKey() string {
|
||||
head := `{"` + text(6) + `":[1`
|
||||
|
||||
return head + strings.Repeat(
|
||||
",1",
|
||||
(mostJSONRead-len(head+"]}"))/len(",1"),
|
||||
) + "]}"
|
||||
}
|
||||
|
||||
// The kinds of body the Core Rule Set reads, and the header that names a
|
||||
// request's cookies.
|
||||
const (
|
||||
formType = "application/x-www-form-urlencoded"
|
||||
jsonType = "application/json"
|
||||
multipartType = "multipart/form-data; boundary=b"
|
||||
cookieHeader = "Cookie"
|
||||
)
|
||||
|
||||
// fullLine is the length of each of the request line and the three or four
|
||||
// header lines that together fill rproxy's limit, with room for the line
|
||||
// endings and the other headers.
|
||||
const fullLine = lineLimit - 64
|
||||
|
||||
// worstCaseRequests are, for each kind of thing a visitor sends, the requests
|
||||
// that keep the Core Rule Set busiest, within the bounds above. The last
|
||||
// holds the most of everything at once: Cookie lines keep it busier than
|
||||
// other header lines, and the JSON body of the most elements under a key
|
||||
// keeps it busier than the other bodies.
|
||||
func worstCaseRequests() []visitorRequest {
|
||||
fullCookieLine := cookieLine(1)[:fullLine-len("Cookie: ")]
|
||||
|
||||
return append(
|
||||
append(worstCaseHeaders(), worstCaseBodies()...),
|
||||
visitorRequest{
|
||||
name: "the longest request line, three Cookie lines and body",
|
||||
target: longestTarget()[:fullLine-len("POST HTTP/1.1")],
|
||||
header: http.Header{cookieHeader: {
|
||||
fullCookieLine, fullCookieLine, fullCookieLine,
|
||||
}},
|
||||
contentType: jsonType, body: jsonOfMostElementsUnderAKey(),
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
// worstCaseHeaders are the requests of worstCaseRequests with no body.
|
||||
func worstCaseHeaders() []visitorRequest {
|
||||
headerFields := http.Header{}
|
||||
|
||||
for i := range mostHeaderFields - 1 {
|
||||
name := "X-Field-" + strconv.Itoa(i)
|
||||
headerFields.Set(
|
||||
name,
|
||||
headerValue(name, headerLimit/mostHeaderFields),
|
||||
)
|
||||
}
|
||||
|
||||
return []visitorRequest{
|
||||
{name: "the longest request line", target: longestTarget()},
|
||||
{
|
||||
name: "the longest request line, as query parameters",
|
||||
target: "/?" + strings.Repeat(
|
||||
"a=1&",
|
||||
(lineLimit-len("POST /? HTTP/1.1"))/len("a=1&"),
|
||||
),
|
||||
},
|
||||
{
|
||||
name: "the longest Cookie line, as one cookie",
|
||||
target: "/",
|
||||
header: http.Header{cookieHeader: {cookieLine(1)}},
|
||||
},
|
||||
{
|
||||
name: "the longest Cookie line, as the most cookies",
|
||||
target: "/",
|
||||
header: http.Header{cookieHeader: {cookieLine(mostCookies)}},
|
||||
},
|
||||
{name: "the most header fields", target: "/", header: headerFields},
|
||||
{
|
||||
name: "the longest header lines",
|
||||
target: "/",
|
||||
header: http.Header{
|
||||
"User-Agent": {headerValue("User-Agent", fullLine)},
|
||||
"Referer": {headerValue("Referer", fullLine)},
|
||||
"Accept": {headerValue("Accept", fullLine)},
|
||||
"X-Long": {headerValue("X-Long", fullLine)},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// worstCaseBodies are the requests of worstCaseRequests whose body holds
|
||||
// the most the Core Rule Set reads.
|
||||
func worstCaseBodies() []visitorRequest {
|
||||
const nested = (mostJSONRead - len(`{"a":1}`)) / len(`{"a":}`)
|
||||
|
||||
return []visitorRequest{
|
||||
{
|
||||
name: "a form body of one field", target: "/",
|
||||
contentType: formType, body: formFields(1),
|
||||
},
|
||||
{
|
||||
name: "a form body of the most fields Coraza keeps", target: "/",
|
||||
contentType: formType, body: formFields(1000),
|
||||
},
|
||||
{
|
||||
name: "a JSON body of one string", target: "/",
|
||||
contentType: jsonType,
|
||||
body: `{"a":"` + text(
|
||||
mostJSONRead-len(`{"a":""}`),
|
||||
) + `"}`,
|
||||
},
|
||||
{
|
||||
name: "a JSON body of the most elements", target: "/",
|
||||
contentType: jsonType, body: jsonArrayOfMostElements(),
|
||||
},
|
||||
{
|
||||
name: "a JSON body of the most elements under a key", target: "/",
|
||||
contentType: jsonType, body: jsonOfMostElementsUnderAKey(),
|
||||
},
|
||||
{
|
||||
name: "a JSON body nested deep", target: "/",
|
||||
contentType: jsonType,
|
||||
body: strings.Repeat(`{"a":`, nested) + "1" +
|
||||
strings.Repeat("}", nested),
|
||||
},
|
||||
{
|
||||
name: "an XML body of many elements", target: "/",
|
||||
contentType: "application/xml",
|
||||
body: "<a>" + strings.Repeat(
|
||||
"<b>x</b>",
|
||||
(mostBodyRead-len("<a></a>"))/len("<b>x</b>"),
|
||||
) + "</a>",
|
||||
},
|
||||
{
|
||||
name: "a multipart body of one field", target: "/",
|
||||
contentType: multipartType, body: multipartBody(mostBodyRead),
|
||||
},
|
||||
{
|
||||
name: "a multipart body of many fields", target: "/",
|
||||
contentType: multipartType, body: multipartBody(1),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// threadCPUTime returns the processor time the calling thread has used.
|
||||
func threadCPUTime(t *testing.T) time.Duration {
|
||||
t.Helper()
|
||||
|
||||
var now unix.Timespec
|
||||
|
||||
err := unix.ClockGettime(unix.CLOCK_THREAD_CPUTIME_ID, &now)
|
||||
if err != nil {
|
||||
t.Fatalf("read the thread's processor time: %v", err)
|
||||
}
|
||||
|
||||
return time.Duration(now.Nano())
|
||||
}
|
||||
|
||||
// inspectionTime returns the processor time crs takes to inspect tc, and the
|
||||
// mean of referenceTime just before and after. It fails t unless crs reads all
|
||||
// of tc's body and Coraza parses it.
|
||||
//
|
||||
// All of it runs on this goroutine, kept on one thread, so the thread's
|
||||
// processor time measures it. Time other work holds the processor does not
|
||||
// count.
|
||||
func inspectionTime(
|
||||
t *testing.T, crs *coreruleset.CoreRuleSet, tc visitorRequest,
|
||||
) (time.Duration, time.Duration) {
|
||||
t.Helper()
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodPost, tc.target, nil,
|
||||
)
|
||||
maps.Copy(req.Header, tc.header)
|
||||
|
||||
if tc.contentType != "" {
|
||||
req.Header.Set("Content-Type", tc.contentType)
|
||||
}
|
||||
|
||||
runtime.LockOSThread()
|
||||
|
||||
defer runtime.UnlockOSThread()
|
||||
|
||||
before := referenceTime(t)
|
||||
|
||||
began := threadCPUTime(t)
|
||||
result, read, _ := crs.Inspect(
|
||||
req, netip.MustParseAddr("192.0.2.1"), strings.NewReader(tc.body),
|
||||
)
|
||||
took := threadCPUTime(t) - began
|
||||
|
||||
reference := (before + referenceTime(t)) / 2
|
||||
|
||||
if string(read) != tc.body {
|
||||
t.Errorf("read %d bytes of %d", len(read), len(tc.body))
|
||||
}
|
||||
|
||||
// Rule 900440 matches a body Coraza could not parse.
|
||||
if slices.Contains(result.RuleIDs, 900440) {
|
||||
t.Errorf("Coraza could not parse the body")
|
||||
}
|
||||
|
||||
return took, reference
|
||||
}
|
||||
|
||||
// TestTheCoreRuleSetInspectsEachWorstCaseRequestInTime runs one at a time, so
|
||||
// that the tests running in parallel do not slow it. Other work on the host
|
||||
// can still slow one try more than the reference work around it, so a request
|
||||
// fails only when three tries in a row take too long.
|
||||
//
|
||||
//nolint:paralleltest // one at a time, as above
|
||||
func TestTheCoreRuleSetInspectsEachWorstCaseRequestInTime(t *testing.T) {
|
||||
const tries = 3
|
||||
|
||||
// The settings that make the Core Rule Set slowest: the highest paranoia
|
||||
// level, no rule switched off, and the most of a body read.
|
||||
crs := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 4, BodyInspectionSize: mostBodyRead,
|
||||
})
|
||||
|
||||
for _, tc := range worstCaseRequests() {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
var took, reference time.Duration
|
||||
|
||||
for range tries {
|
||||
took, reference = inspectionTime(t, crs, tc)
|
||||
if took <= inspectionTimeLimitMultiple*reference {
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
t.Errorf(
|
||||
"took %s on the last of %d tries, over %d times the %s "+
|
||||
"of the reference work",
|
||||
took, tries, inspectionTimeLimitMultiple, reference,
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
package lookup
|
||||
package geojs
|
||||
|
||||
import "net/http"
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
// Package lookup looks up each client's AS number and country, through
|
||||
// the GeoJS web service or in the lookup database, the IPinfo Lite file
|
||||
// SWW_WAF_LOOKUP_DB_PATH names. GeoJS's answers are kept in memory, for at
|
||||
// most 100,000 clients and for 7 days each, and are written to
|
||||
// lookups.json and read from it by the state package.
|
||||
package lookup
|
||||
// Package geojs looks up clients' AS numbers and countries through the GeoJS
|
||||
// web service. Its answers are kept in memory, for at most 100,000 clients and
|
||||
// for 7 days each. A caller waits for an answer as long as it says. A request to
|
||||
// GeoJS goes on after its callers stop waiting, and its answer is kept. The
|
||||
// package imports no other package of this repository.
|
||||
package geojs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"slices"
|
||||
@@ -21,8 +20,6 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/hashicorp/golang-lru/v2/simplelru"
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
||||
)
|
||||
|
||||
// URL is GeoJS's endpoint for an address's place and network. Asked about
|
||||
@@ -43,6 +40,9 @@ const (
|
||||
maxWaiting = 10000
|
||||
// maxPerRequest is how many addresses one request to GeoJS asks about.
|
||||
maxPerRequest = 200
|
||||
// requestTimeout is how long a request to GeoJS may take before it is
|
||||
// abandoned, which is a failure. It is apart from how long callers wait.
|
||||
requestTimeout = 10 * time.Second
|
||||
// unknownASN is the AS number GeoJS gives when it knows none.
|
||||
unknownASN = 64512
|
||||
// After a failure GeoJS is not asked again for a second, and for
|
||||
@@ -64,39 +64,32 @@ var (
|
||||
type Params struct {
|
||||
// URL is where GeoJS is asked, normally URL.
|
||||
URL string
|
||||
// Timeout is how long a request waits for its client's first answer,
|
||||
// and how long a request to GeoJS may take before it is abandoned
|
||||
// (SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS).
|
||||
Timeout time.Duration
|
||||
// Wait is true when a setting needs each request's answer before the
|
||||
// request goes on. Otherwise no request waits for one.
|
||||
Wait bool
|
||||
// Answered, unless nil, is given each answer GeoJS gives, once it is
|
||||
// kept.
|
||||
Answered func(Answer)
|
||||
// Now tells the time, normally time.Now in UTC.
|
||||
Now func() time.Time
|
||||
// ProcessLog receives GeoJS's failures.
|
||||
ProcessLog *slog.Logger
|
||||
// Metrics count the requests to GeoJS, those that failed, and the
|
||||
// clients that go without an answer.
|
||||
Metrics *metrics.Metrics
|
||||
// Alerts receive a source_failure alert each time GeoJS fails.
|
||||
Alerts *alerts.Queue
|
||||
// Asked is called for each request to GeoJS.
|
||||
Asked func()
|
||||
// Failed is called for each failure of GeoJS, with what went wrong and
|
||||
// how long GeoJS is left alone after it. The error never names the
|
||||
// addresses asked about.
|
||||
Failed func(err error, askingAgainIn time.Duration)
|
||||
// WentWithoutAnswer is called each time a caller that waits for an
|
||||
// answer goes on without one.
|
||||
WentWithoutAnswer func()
|
||||
}
|
||||
|
||||
// GeoJS looks up clients' AS numbers and countries through GeoJS. At most
|
||||
// one request to GeoJS is under way at a time, and it asks about every
|
||||
// client waiting, up to maxPerRequest. It is safe for concurrent use.
|
||||
type GeoJS struct {
|
||||
url string
|
||||
timeout time.Duration
|
||||
wait bool
|
||||
answered func(Answer)
|
||||
now func() time.Time
|
||||
processLog *slog.Logger
|
||||
metrics *metrics.Metrics
|
||||
alerts *alerts.Queue
|
||||
url string
|
||||
answered func(Answer)
|
||||
now func() time.Time
|
||||
asked func()
|
||||
failed func(err error, askingAgainIn time.Duration)
|
||||
wentWithoutAnswer func()
|
||||
// httpClient follows no redirect, so that visitors' addresses go to
|
||||
// GeoJS alone: a redirect is a failure.
|
||||
httpClient *http.Client
|
||||
@@ -114,12 +107,10 @@ type GeoJS struct {
|
||||
retryAt time.Time
|
||||
}
|
||||
|
||||
// Answer is what GeoJS or the lookup database said about a client: its AS
|
||||
// number, such as AS64496, and the AS's name, both "" when the source knows
|
||||
// no AS number for it; its country, "" when the source cannot place it;
|
||||
// when the source said so; and, for GeoJS's answers, which lookups.json
|
||||
// holds, when the answer was last used. The zero Answer is that of a
|
||||
// client with no answer.
|
||||
// Answer is what GeoJS said about a client: its AS number, such as AS64496,
|
||||
// and the AS's name, both "" when GeoJS knows no AS number for it; its
|
||||
// country, "" when GeoJS cannot place it; when GeoJS said so; and when the
|
||||
// answer was last used. The zero Answer is that of a client with no answer.
|
||||
//
|
||||
//nolint:tagliatelle // the state files use snake_case, as the request log does
|
||||
type Answer struct {
|
||||
@@ -137,7 +128,7 @@ type wait struct {
|
||||
// replaced each time GeoJS fails before then.
|
||||
asked chan struct{}
|
||||
// late is true once the client has gone without an answer, for a
|
||||
// whole timeout or because GeoJS failed: its requests no longer wait.
|
||||
// whole wait or because GeoJS failed: its callers no longer wait.
|
||||
late bool
|
||||
}
|
||||
|
||||
@@ -149,14 +140,12 @@ func New(params Params) *GeoJS {
|
||||
}
|
||||
|
||||
return &GeoJS{
|
||||
url: params.URL,
|
||||
timeout: params.Timeout,
|
||||
wait: params.Wait,
|
||||
answered: params.Answered,
|
||||
now: params.Now,
|
||||
processLog: params.ProcessLog,
|
||||
metrics: params.Metrics,
|
||||
alerts: params.Alerts,
|
||||
url: params.URL,
|
||||
answered: params.Answered,
|
||||
now: params.Now,
|
||||
asked: params.Asked,
|
||||
failed: params.Failed,
|
||||
wentWithoutAnswer: params.WentWithoutAnswer,
|
||||
httpClient: &http.Client{
|
||||
CheckRedirect: func(*http.Request, []*http.Request) error {
|
||||
return http.ErrUseLastResponse
|
||||
@@ -170,21 +159,22 @@ func New(params Params) *GeoJS {
|
||||
// LookUp returns the answer GeoJS gave about client, with its country as
|
||||
// a two-letter code in capitals, or the zero Answer when there is none
|
||||
// yet. An answer is kept for 7 days. Without one, the client is asked
|
||||
// about in the background, and, while Wait is set, the request waits up
|
||||
// to Timeout for the answer, unless the client has gone without one
|
||||
// before. ctx is the context of the client's request, and ends the wait
|
||||
// when it ends.
|
||||
// about in the background, and LookUp waits up to wait for the answer,
|
||||
// unless the client has gone without one before. The wait ends early when
|
||||
// ctx ends. Neither ends the request to GeoJS.
|
||||
//
|
||||
// GeoJS is asked about the client's first address, which is the client's
|
||||
// own address for IPv4, and an address in the same place for an IPv6
|
||||
// netblock.
|
||||
func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
|
||||
answer, asked := g.answerOrWait(ctx, client, g.wait)
|
||||
func (g *GeoJS) LookUp(
|
||||
ctx context.Context, client netip.Prefix, wait time.Duration,
|
||||
) Answer {
|
||||
answer, asked := g.answerOrWait(ctx, client, true)
|
||||
if asked == nil {
|
||||
return answer
|
||||
}
|
||||
|
||||
timer := time.NewTimer(g.timeout)
|
||||
timer := time.NewTimer(wait)
|
||||
defer timer.Stop()
|
||||
|
||||
select {
|
||||
@@ -198,7 +188,7 @@ func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
|
||||
|
||||
answer, found := g.kept(client)
|
||||
if !found {
|
||||
g.metrics.RequestsWithoutGeoJSAnswer.Inc()
|
||||
g.wentWithoutAnswer()
|
||||
}
|
||||
|
||||
w, waiting := g.waiting[client]
|
||||
@@ -209,8 +199,8 @@ func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
|
||||
return answer
|
||||
}
|
||||
|
||||
// LookUpWithoutWaiting is LookUp for a request that does not need the answer,
|
||||
// whatever Wait says: it returns the zero Answer at once when none is kept.
|
||||
// LookUpWithoutWaiting is LookUp for a caller that does not need the answer:
|
||||
// it returns the zero Answer at once when none is kept.
|
||||
func (g *GeoJS) LookUpWithoutWaiting(
|
||||
ctx context.Context,
|
||||
client netip.Prefix,
|
||||
@@ -228,8 +218,7 @@ func (g *GeoJS) Kept(client netip.Prefix) (Answer, bool) {
|
||||
return g.kept(client)
|
||||
}
|
||||
|
||||
// Snapshot returns every answer kept, sorted by client, as lookups.json
|
||||
// lists them.
|
||||
// Snapshot returns every answer kept, sorted by client.
|
||||
func (g *GeoJS) Snapshot() []Answer {
|
||||
g.mu.Lock()
|
||||
|
||||
@@ -247,9 +236,9 @@ func (g *GeoJS) Snapshot() []Answer {
|
||||
return answers
|
||||
}
|
||||
|
||||
// Load keeps answers read from lookups.json, in place of the answers it
|
||||
// keeps, in the order they were last used, so that the one used longest
|
||||
// ago is dropped first. Answers GeoJS gave keepFor ago or more are
|
||||
// Load keeps answers, such as those of an earlier Snapshot, in place of the
|
||||
// answers it keeps, in the order they were last used, so that the one used
|
||||
// longest ago is dropped first. Answers GeoJS gave keepFor ago or more are
|
||||
// dropped.
|
||||
func (g *GeoJS) Load(answers []Answer) {
|
||||
answers = slices.Clone(answers)
|
||||
@@ -295,11 +284,11 @@ func (g *GeoJS) answerOrWait(
|
||||
g.ask(ctx)
|
||||
|
||||
if !mustWait {
|
||||
return Answer{}, nil // the answer is not needed before the request goes on
|
||||
return Answer{}, nil // the answer is not needed before the caller goes on
|
||||
}
|
||||
|
||||
if w == nil {
|
||||
g.metrics.RequestsWithoutGeoJSAnswer.Inc()
|
||||
g.wentWithoutAnswer()
|
||||
|
||||
return Answer{}, nil // too many clients wait already
|
||||
}
|
||||
@@ -310,7 +299,7 @@ func (g *GeoJS) answerOrWait(
|
||||
}
|
||||
|
||||
if w.late {
|
||||
g.metrics.RequestsWithoutGeoJSAnswer.Inc()
|
||||
g.wentWithoutAnswer()
|
||||
|
||||
return Answer{}, nil
|
||||
}
|
||||
@@ -335,8 +324,8 @@ func (g *GeoJS) kept(client netip.Prefix) (Answer, bool) {
|
||||
|
||||
// ask starts asking GeoJS about the waiting clients, unless a request to
|
||||
// it is under way or it is left alone after a failure. The requests to
|
||||
// GeoJS are for every client waiting, so they go on when the client's
|
||||
// request whose ctx is given ends.
|
||||
// GeoJS are for every client waiting, so they go on when the caller whose
|
||||
// ctx is given stops waiting.
|
||||
func (g *GeoJS) ask(ctx context.Context) {
|
||||
if g.asking || g.now().Before(g.retryAt) {
|
||||
return
|
||||
@@ -441,8 +430,6 @@ func (g *GeoJS) keep(
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
g.metrics.GeoJSFailures.Inc()
|
||||
|
||||
g.retryDelay = min(
|
||||
max(retryDelayFactor*g.retryDelay, firstRetryDelay),
|
||||
maxRetryDelay,
|
||||
@@ -457,16 +444,7 @@ func (g *GeoJS) keep(
|
||||
w.late = true
|
||||
}
|
||||
|
||||
g.processLog.Warn("asking GeoJS failed",
|
||||
"error", err.Error(), "asking_again_in", g.retryDelay.String())
|
||||
g.alerts.Raise(alerts.Alert{
|
||||
Event: alerts.EventSourceFailure,
|
||||
Reason: "asking GeoJS failed",
|
||||
Detail: map[string]any{
|
||||
"source": "geojs", "error": err.Error(),
|
||||
"asking_again_in": g.retryDelay.String(),
|
||||
},
|
||||
})
|
||||
g.failed(err, g.retryDelay)
|
||||
|
||||
return kept, false
|
||||
}
|
||||
@@ -479,7 +457,8 @@ func (g *GeoJS) keep(
|
||||
// request asks GeoJS about clients in one request, and returns the answer
|
||||
// for each address GeoJS's answer names: its AS number and the AS's name,
|
||||
// both "" for the AS number 64512, which GeoJS gives when it knows none,
|
||||
// and its country, in capitals.
|
||||
// and its country, in capitals. The request is abandoned after
|
||||
// requestTimeout.
|
||||
func (g *GeoJS) request(
|
||||
ctx context.Context, clients []netip.Prefix,
|
||||
) (map[netip.Addr]Answer, error) {
|
||||
@@ -489,7 +468,7 @@ func (g *GeoJS) request(
|
||||
addrs = append(addrs, client.Addr().String())
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(ctx, g.timeout)
|
||||
ctx, cancel := context.WithTimeout(ctx, requestTimeout)
|
||||
defer cancel()
|
||||
|
||||
req, err := http.NewRequestWithContext(
|
||||
@@ -504,7 +483,7 @@ func (g *GeoJS) request(
|
||||
|
||||
req.URL.RawQuery = "ip=" + strings.Join(addrs, ",")
|
||||
|
||||
g.metrics.GeoJSRequests.Inc()
|
||||
g.asked()
|
||||
|
||||
res, err := g.httpClient.Do(req)
|
||||
if err != nil {
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,4 +1,4 @@
|
||||
package lookup_test
|
||||
package geojs_test
|
||||
|
||||
import (
|
||||
"net/netip"
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
"testing/synctest"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
)
|
||||
|
||||
func TestSnapshotHoldsEachAnswerAndWhenItWasLastUsed(t *testing.T) {
|
||||
@@ -25,7 +25,7 @@ func TestSnapshotHoldsEachAnswerAndWhenItWasLastUsed(t *testing.T) {
|
||||
clock.advance(time.Hour)
|
||||
wantCountry(t, g, placed, germany)
|
||||
|
||||
want := []lookup.Answer{
|
||||
want := []geojs.Answer{
|
||||
{Client: notPlaced, Answered: asked, Used: asked},
|
||||
{
|
||||
Client: placed, ASN: asn, ASName: asName, Country: germany,
|
||||
@@ -41,24 +41,24 @@ func TestSnapshotHoldsEachAnswerAndWhenItWasLastUsed(t *testing.T) {
|
||||
func TestLoadedAnswersAreKeptFor7DaysFromWhenGeoJSGaveThem(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
geojs, clock, g := start()
|
||||
service, clock, g := start()
|
||||
now := clock.Now()
|
||||
kept := lookup.Answer{
|
||||
kept := geojs.Answer{
|
||||
Client: netip.MustParsePrefix("203.0.113.9/32"),
|
||||
Country: "FR",
|
||||
Answered: now.Add(-week + time.Second),
|
||||
Used: now.Add(-time.Hour),
|
||||
}
|
||||
stale := lookup.Answer{
|
||||
stale := geojs.Answer{
|
||||
Client: netip.MustParsePrefix("203.0.113.10/32"),
|
||||
Country: "FR",
|
||||
Answered: now.Add(-week),
|
||||
Used: now.Add(-time.Hour),
|
||||
}
|
||||
|
||||
g.Load([]lookup.Answer{kept, stale})
|
||||
g.Load([]geojs.Answer{kept, stale})
|
||||
|
||||
if got := g.Snapshot(); !slices.Equal(got, []lookup.Answer{kept}) {
|
||||
if got := g.Snapshot(); !slices.Equal(got, []geojs.Answer{kept}) {
|
||||
t.Errorf(
|
||||
"kept %+v, want only the answer GeoJS gave less than 7 days ago",
|
||||
got,
|
||||
@@ -66,7 +66,7 @@ func TestLoadedAnswersAreKeptFor7DaysFromWhenGeoJSGaveThem(t *testing.T) {
|
||||
}
|
||||
|
||||
wantCountry(t, g, kept.Client, "FR")
|
||||
wantRequests(t, geojs, 0)
|
||||
wantRequests(t, service, 0)
|
||||
}
|
||||
|
||||
func TestLoadDropsTheAnswerUsedLongestAgoFirst(t *testing.T) {
|
||||
@@ -80,11 +80,11 @@ func TestLoadDropsTheAnswerUsedLongestAgoFirst(t *testing.T) {
|
||||
// lookups.json lists the answers by client. Here each was last used a
|
||||
// second before the one listed before it, so the last listed is the
|
||||
// one used longest ago, and the one dropped.
|
||||
answers := make([]lookup.Answer, maxAnswers+1)
|
||||
answers := make([]geojs.Answer, maxAnswers+1)
|
||||
addr := netip.MustParseAddr("10.0.0.0")
|
||||
|
||||
for i := range answers {
|
||||
answers[i] = lookup.Answer{
|
||||
answers[i] = geojs.Answer{
|
||||
Client: netip.PrefixFrom(addr, addr.BitLen()),
|
||||
Country: germany,
|
||||
Answered: now,
|
||||
@@ -1,3 +1,6 @@
|
||||
// Package lookup looks up each client's AS number and country in the lookup
|
||||
// database, the IPinfo Lite file SWW_WAF_LOOKUP_DB_PATH names. Its answers
|
||||
// take the form of GeoJS's, so that waf handles both alike.
|
||||
package lookup
|
||||
|
||||
import (
|
||||
@@ -14,6 +17,7 @@ import (
|
||||
"github.com/oschwald/maxminddb-golang/v2"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
)
|
||||
|
||||
// quietTime is how long the lookup database must go without a change
|
||||
@@ -77,7 +81,7 @@ func OpenFile(params FileParams) (*File, error) {
|
||||
// each "" when the database does not give it, as for an address missing
|
||||
// from it. The database is asked about the client's first address, as
|
||||
// GeoJS is.
|
||||
func (f *File) LookUp(client netip.Prefix) Answer {
|
||||
func (f *File) LookUp(client netip.Prefix) geojs.Answer {
|
||||
f.mu.Lock()
|
||||
reader := f.reader
|
||||
f.mu.Unlock()
|
||||
@@ -91,7 +95,7 @@ func (f *File) LookUp(client netip.Prefix) Answer {
|
||||
found = record{}
|
||||
}
|
||||
|
||||
return Answer{
|
||||
return geojs.Answer{
|
||||
Client: client,
|
||||
ASN: found.ASN,
|
||||
ASName: found.ASName,
|
||||
|
||||
@@ -16,6 +16,7 @@ import (
|
||||
"github.com/maxmind/mmdbwriter/mmdbtype"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
||||
)
|
||||
|
||||
@@ -68,7 +69,7 @@ func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(
|
||||
prefix := netip.MustParsePrefix(client)
|
||||
|
||||
got := f.LookUp(prefix)
|
||||
if got != (Answer{
|
||||
if got != (geojs.Answer{
|
||||
Client: prefix, ASN: want.ASN, ASName: want.ASName, Country: want.Country,
|
||||
Answered: now,
|
||||
}) {
|
||||
|
||||
@@ -274,8 +274,7 @@ func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
|
||||
// AddLookupFile adds the metrics of the lookup database, read as the
|
||||
// metrics are asked for: when the file in use was read, which lastRead
|
||||
// returns, and the replacements of it that could not be read, which
|
||||
// readFailures returns. The lookup package's File, which has both, cannot
|
||||
// be named here: that package counts GeoJS's requests in these metrics.
|
||||
// readFailures returns.
|
||||
func (m *Metrics) AddLookupFile(
|
||||
lastRead func() time.Time,
|
||||
readFailures func() int,
|
||||
|
||||
@@ -44,7 +44,7 @@ import (
|
||||
const (
|
||||
// started is the time a record made as a test starts gives.
|
||||
started = "2000-01-01T00:00:00.000Z"
|
||||
appName = "fsn1app1/blog"
|
||||
appName = "host1/blog"
|
||||
// local0 is the number of the default facility, and local0Info the
|
||||
// priority of its records.
|
||||
local0 = 16
|
||||
|
||||
@@ -21,7 +21,12 @@ const (
|
||||
// the app.
|
||||
ActionForward = "forward"
|
||||
|
||||
// ActionRequestHeaderTooLarge is a request over the header limits.
|
||||
// ActionRequestLineTooLong is a request whose request line is too long for
|
||||
// the header limits.
|
||||
ActionRequestLineTooLong = "request_line_too_long"
|
||||
|
||||
// ActionRequestHeaderTooLarge is a request whose headers are over the
|
||||
// header limits.
|
||||
ActionRequestHeaderTooLarge = "request_header_too_large"
|
||||
|
||||
// ActionClientIsRateLimited is a request refused because it took its
|
||||
@@ -143,7 +148,7 @@ type Line struct {
|
||||
// taken with a request it would have refused: ActionClientDenied,
|
||||
// ActionTemporarilyBanned, ActionPermanentlyBanned, ActionCountryDenied,
|
||||
// ActionClientIsRateLimited, ActionBlockedByBlockRule,
|
||||
// ActionBlockedByCoreRuleSet, or, in waf,
|
||||
// ActionBlockedByCoreRuleSet, or, in waf, ActionRequestLineTooLong or
|
||||
// ActionRequestHeaderTooLarge for the header limits.
|
||||
WouldAction string `json:"would_action,omitempty"`
|
||||
// Heuristic is the name of the heuristic that refused the request, in
|
||||
|
||||
@@ -140,7 +140,7 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
|
||||
|
||||
var out bytes.Buffer
|
||||
|
||||
requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo).
|
||||
requestlog.NewProcessLogger(&out, "host1/blog", slog.LevelInfo).
|
||||
Info("starting",
|
||||
"version", "v1")
|
||||
|
||||
@@ -152,7 +152,7 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
|
||||
}
|
||||
|
||||
if fields["type"] != "process" ||
|
||||
fields["instance"] != "fsn1app1/blog" ||
|
||||
fields["instance"] != "host1/blog" ||
|
||||
fields["msg"] != "starting" ||
|
||||
fields["level"] != "INFO" ||
|
||||
fields["version"] != "v1" {
|
||||
@@ -187,7 +187,7 @@ func TestProcessLoggerWritesTheMessagesAtItsLevelOrMoreSevere(t *testing.T) {
|
||||
|
||||
processLog := requestlog.NewProcessLogger(
|
||||
&out,
|
||||
"fsn1app1/blog",
|
||||
"host1/blog",
|
||||
level,
|
||||
)
|
||||
for _, at := range levels {
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
package rproxy
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// What opens the circuit breaker, and how often a request asks waf while it is
|
||||
// open.
|
||||
const (
|
||||
breakerFailures = 5
|
||||
breakerWindow = 5 * time.Second
|
||||
breakerAskInterval = time.Second
|
||||
)
|
||||
|
||||
// errBreakerOpen is why a request did not ask waf.
|
||||
var errBreakerOpen = errors.New(
|
||||
"waf is not asked while the circuit breaker is open",
|
||||
)
|
||||
|
||||
// breaker is the circuit breaker for asks to waf. It opens when
|
||||
// breakerFailures asks fail within breakerWindow. An ask fails when it gets no
|
||||
// answer in time, or an answer rproxy cannot pass on. While it is open, at
|
||||
// most one request each breakerAskInterval asks waf. Any usable answer closes
|
||||
// it, whatever it decides, and forgets the failures before it.
|
||||
type breaker struct {
|
||||
mu sync.Mutex
|
||||
// failures holds the times of the latest failed asks, oldest first, at most
|
||||
// breakerFailures of them.
|
||||
failures []time.Time
|
||||
open bool
|
||||
// lastAsk is when a request last asked waf while the breaker was open.
|
||||
lastAsk time.Time
|
||||
}
|
||||
|
||||
// mayAsk reports whether a request may ask waf at now. While the breaker is
|
||||
// open, it says yes once each breakerAskInterval.
|
||||
func (b *breaker) mayAsk(now time.Time) bool {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
|
||||
if !b.open {
|
||||
return true
|
||||
}
|
||||
|
||||
if now.Sub(b.lastAsk) < breakerAskInterval {
|
||||
return false
|
||||
}
|
||||
|
||||
b.lastAsk = now
|
||||
|
||||
return true
|
||||
}
|
||||
|
||||
// noteAnswer notes that an ask got an answer rproxy can use. It closes the
|
||||
// breaker and forgets the failures before it.
|
||||
func (b *breaker) noteAnswer() {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
|
||||
b.open = false
|
||||
b.failures = nil
|
||||
}
|
||||
|
||||
// noteFailure notes that an ask that ended at now got no answer in time, and
|
||||
// opens the breaker if breakerFailures asks have failed within breakerWindow.
|
||||
func (b *breaker) noteFailure(now time.Time) {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
|
||||
b.failures = append(b.failures, now)
|
||||
if len(b.failures) > breakerFailures {
|
||||
b.failures = b.failures[1:]
|
||||
}
|
||||
|
||||
if !b.open && len(b.failures) == breakerFailures &&
|
||||
now.Sub(b.failures[0]) <= breakerWindow {
|
||||
b.open = true
|
||||
b.lastAsk = now
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
package rproxy_test
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||
)
|
||||
|
||||
// The breaker tests give the breaker each time themselves, counted from
|
||||
// time.Now(), and never wait.
|
||||
|
||||
// fail notes count failed asks on b, one each step from at.
|
||||
func fail(b *rproxy.Breaker, at time.Time, count int, step time.Duration) {
|
||||
for i := range count {
|
||||
b.NoteFailure(at.Add(time.Duration(i) * step))
|
||||
}
|
||||
}
|
||||
|
||||
// openBreaker returns a breaker opened at now by five failed asks, one a
|
||||
// second.
|
||||
func openBreaker(now time.Time) *rproxy.Breaker {
|
||||
b := &rproxy.Breaker{}
|
||||
fail(b, now.Add(-4*time.Second), 5, time.Second)
|
||||
|
||||
return b
|
||||
}
|
||||
|
||||
func TestFourFailedAsksWithinFiveSecondsLeaveTheBreakerClosed(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
now := time.Now()
|
||||
b := &rproxy.Breaker{}
|
||||
fail(b, now, 4, time.Second)
|
||||
|
||||
if !b.MayAsk(now.Add(3 * time.Second)) {
|
||||
t.Error("a request may not ask, want it to")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFiveFailedAsksWithinFiveSecondsOpenTheBreaker(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
now := time.Now()
|
||||
b := openBreaker(now)
|
||||
|
||||
if b.MayAsk(now) {
|
||||
t.Error("a request may ask, want it not to")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFiveFailedAsksOverMoreThanFiveSecondsLeaveTheBreakerClosed(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
now := time.Now()
|
||||
b := &rproxy.Breaker{}
|
||||
fail(b, now, 5, 1300*time.Millisecond)
|
||||
|
||||
if !b.MayAsk(now.Add(6 * time.Second)) {
|
||||
t.Error("a request may not ask, want it to")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAnswerBetweenFailedAsksKeepsTheBreakerClosed(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
now := time.Now()
|
||||
b := &rproxy.Breaker{}
|
||||
fail(b, now, 4, 100*time.Millisecond)
|
||||
b.NoteAnswer()
|
||||
fail(b, now.Add(500*time.Millisecond), 4, 100*time.Millisecond)
|
||||
|
||||
if !b.MayAsk(now.Add(900 * time.Millisecond)) {
|
||||
t.Error("a request may not ask, want it to")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnOpenBreakerLetsOneRequestAskEachSecond(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
now := time.Now()
|
||||
b := openBreaker(now)
|
||||
|
||||
for _, tc := range []struct {
|
||||
after time.Duration
|
||||
want bool
|
||||
}{
|
||||
{999 * time.Millisecond, false},
|
||||
{time.Second, true},
|
||||
{time.Second, false},
|
||||
{1500 * time.Millisecond, false},
|
||||
{2 * time.Second, true},
|
||||
} {
|
||||
got := b.MayAsk(now.Add(tc.after))
|
||||
if got != tc.want {
|
||||
t.Errorf("may ask %v after it opened: %v, want %v",
|
||||
tc.after, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAFailedAskKeepsTheBreakerOpen(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
opened := time.Now()
|
||||
b := openBreaker(opened)
|
||||
now := opened.Add(time.Second)
|
||||
b.MayAsk(now)
|
||||
b.NoteFailure(now)
|
||||
|
||||
if b.MayAsk(now.Add(999 * time.Millisecond)) {
|
||||
t.Error("a request may ask, want it not to")
|
||||
}
|
||||
|
||||
if !b.MayAsk(now.Add(time.Second)) {
|
||||
t.Error("a request may not ask a second later, want it to")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOneAnswerClosesTheBreaker(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
opened := time.Now()
|
||||
b := openBreaker(opened)
|
||||
now := opened.Add(time.Second)
|
||||
b.MayAsk(now)
|
||||
b.NoteAnswer()
|
||||
|
||||
if !b.MayAsk(now) {
|
||||
t.Error("a request may not ask, want it to")
|
||||
}
|
||||
|
||||
// The failures before the answer no longer count, so one more does not open
|
||||
// the breaker again.
|
||||
b.NoteFailure(now)
|
||||
|
||||
if !b.MayAsk(now) {
|
||||
t.Error("a request may not ask after one failure, want it to")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
package rproxy
|
||||
|
||||
import "time"
|
||||
|
||||
// Breaker is the circuit breaker, for tests that give it the time.
|
||||
type Breaker struct {
|
||||
b breaker
|
||||
}
|
||||
|
||||
// MayAsk reports whether a request may ask waf at now.
|
||||
func (b *Breaker) MayAsk(now time.Time) bool {
|
||||
return b.b.mayAsk(now)
|
||||
}
|
||||
|
||||
// NoteAnswer notes that an ask got an answer rproxy can use.
|
||||
func (b *Breaker) NoteAnswer() {
|
||||
b.b.noteAnswer()
|
||||
}
|
||||
|
||||
// NoteFailure notes that an ask that ended at now got no answer in time.
|
||||
func (b *Breaker) NoteFailure(now time.Time) {
|
||||
b.b.noteFailure(now)
|
||||
}
|
||||
@@ -4,10 +4,11 @@
|
||||
// waf goes to the visitor as waf gave it.
|
||||
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION decides only when waf cannot
|
||||
// be connected to, does not answer at all, or does not answer within
|
||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. rproxy itself refuses a request with a
|
||||
// method it does not pass on, or one it cannot put to waf as the visitor
|
||||
// sent it. It keeps nothing between requests and writes nothing to disk. "rproxy" in
|
||||
// SPEC.md says what it does.
|
||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, or while the circuit breaker for asks to
|
||||
// waf is open. rproxy itself refuses a request with a method it does not pass
|
||||
// on, or one it cannot put to waf as the visitor sent it. It keeps nothing
|
||||
// between requests but the circuit breaker, and writes nothing to disk.
|
||||
// "rproxy" in SPEC.md says what it does.
|
||||
package rproxy
|
||||
|
||||
import (
|
||||
@@ -132,6 +133,8 @@ type handler struct {
|
||||
wafTransport http.RoundTripper
|
||||
// reportURL is where the reports go.
|
||||
reportURL string
|
||||
// breaker is the circuit breaker for asks to waf.
|
||||
breaker breaker
|
||||
// reports waits for the reports being sent. reportsMu guards starting one
|
||||
// against reportsStopped, which is set once nothing is to wait for another.
|
||||
reports sync.WaitGroup
|
||||
@@ -196,7 +199,7 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
answer, err := rq.ask(r.Context())
|
||||
answer, err := rq.askUnlessBreakerOpen(r.Context())
|
||||
if err != nil {
|
||||
rq.handleNoUsableAnswer(r.Context(), err)
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
@@ -1395,6 +1396,163 @@ func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
|
||||
standIn.wantNoReport(t)
|
||||
}
|
||||
|
||||
func TestFiveFailedAsksKeepTheNextRequestFromAskingWAF(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for name, failure := range map[string]http.HandlerFunc{
|
||||
"no answer at all": func(http.ResponseWriter, *http.Request) {
|
||||
panic(http.ErrAbortHandler)
|
||||
},
|
||||
"a slow answer": func(_ http.ResponseWriter, r *http.Request) {
|
||||
<-r.Context().Done()
|
||||
},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var answering atomic.Bool
|
||||
|
||||
standIn := startWAF(
|
||||
t,
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
if answering.Load() {
|
||||
allow(w, r)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
failure(w, r)
|
||||
},
|
||||
)
|
||||
nextID := make(chan string, 1)
|
||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if answering.Load() {
|
||||
nextID <- r.Header.Get(requestIDHeader)
|
||||
}
|
||||
|
||||
greet(w, r)
|
||||
})
|
||||
addr, out := startRProxy(
|
||||
t,
|
||||
standIn.URL,
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "200",
|
||||
},
|
||||
)
|
||||
|
||||
// An ask can time out before it reaches the stand-in, so these
|
||||
// may leave fewer than five questions, or leave some late.
|
||||
for range 5 {
|
||||
wantGreeting(t, get(t, addr, "/"))
|
||||
}
|
||||
|
||||
// rproxy writes this line before it answers a request it kept
|
||||
// from asking waf, so no wait is needed.
|
||||
if out.has("the circuit breaker is open") {
|
||||
t.Error("the circuit breaker opened before five asks failed")
|
||||
}
|
||||
|
||||
answering.Store(true)
|
||||
wantGreeting(t, get(t, addr, "/"))
|
||||
out.waitFor(t, "the circuit breaker is open")
|
||||
|
||||
// Had the next request asked waf, its question would have reached
|
||||
// the stand-in before the request reached the app.
|
||||
var id string
|
||||
select {
|
||||
case id = <-nextID:
|
||||
default:
|
||||
t.Fatal("the app did not get the next request")
|
||||
}
|
||||
|
||||
for len(standIn.questions) > 0 {
|
||||
asked := <-standIn.questions
|
||||
if asked.header.Get(requestIDHeader) == id {
|
||||
t.Error("waf was asked, want it not to be")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestVisitorsThatGoAwayWhileWAFIsAskedLeaveTheBreakerClosed(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
var answering atomic.Bool
|
||||
|
||||
standIn := startWAF(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if answering.Load() {
|
||||
allow(w, r)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
<-r.Context().Done()
|
||||
})
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
})
|
||||
|
||||
// Five of these would open the breaker if they counted as failures.
|
||||
for range 5 {
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||
id := standIn.question(t).header.Get(requestIDHeader)
|
||||
|
||||
_ = conn.Close()
|
||||
|
||||
// rproxy writes the request's log line once it is done with the ask.
|
||||
out.waitFor(t, id)
|
||||
}
|
||||
|
||||
answering.Store(true)
|
||||
wantGreeting(t, get(t, addr, "/"))
|
||||
standIn.question(t)
|
||||
}
|
||||
|
||||
func TestAnswersThatCannotBePassedOnOpenTheBreaker(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var answering atomic.Bool
|
||||
|
||||
tooLarge := overOneMiB(dispositionDeny)
|
||||
standIn := startWAF(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if answering.Load() {
|
||||
allow(w, r)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
tooLarge(w, r)
|
||||
})
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
})
|
||||
|
||||
for range 5 {
|
||||
if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
|
||||
t.Errorf("answered %d, want 502", got.status)
|
||||
}
|
||||
|
||||
standIn.question(t)
|
||||
}
|
||||
|
||||
answering.Store(true)
|
||||
wantGreeting(t, get(t, addr, "/"))
|
||||
out.waitFor(t, "the circuit breaker is open")
|
||||
|
||||
select {
|
||||
case <-standIn.questions:
|
||||
t.Error("waf was asked, want it not to be")
|
||||
case <-time.After(10 * pollInterval):
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
+28
-1
@@ -10,6 +10,7 @@ import (
|
||||
"maps"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
)
|
||||
@@ -67,7 +68,8 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
|
||||
rq.answer(http.StatusBadGateway)
|
||||
default:
|
||||
// waf cannot be connected to, does not answer at all, or does not answer
|
||||
// within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. A request that keeps waf
|
||||
// within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS; or the circuit breaker
|
||||
// kept the request from asking. A request that keeps waf
|
||||
// busy past SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, like any request while
|
||||
// waf is overloaded, gets SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION
|
||||
// here. deny closes this, at the cost of answering 503 whenever waf is
|
||||
@@ -86,6 +88,31 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
|
||||
}
|
||||
}
|
||||
|
||||
// askUnlessBreakerOpen asks waf, unless the circuit breaker is open and it is
|
||||
// not yet time for a request to ask. It then tells the breaker whether waf
|
||||
// answered in time. An answer rproxy cannot pass on counts as a failure. A
|
||||
// visitor that goes away tells the breaker nothing. ctx is the request's own
|
||||
// context.
|
||||
func (rq *request) askUnlessBreakerOpen(
|
||||
ctx context.Context,
|
||||
) (*wafAnswer, error) {
|
||||
b := &rq.h.breaker
|
||||
if !b.mayAsk(time.Now()) {
|
||||
return nil, errBreakerOpen
|
||||
}
|
||||
|
||||
answer, err := rq.ask(ctx)
|
||||
|
||||
switch {
|
||||
case err == nil:
|
||||
b.noteAnswer()
|
||||
case ctx.Err() == nil:
|
||||
b.noteFailure(time.Now())
|
||||
}
|
||||
|
||||
return answer, err
|
||||
}
|
||||
|
||||
// ask sends waf the question, and returns its answer, which must have come
|
||||
// whole within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. It ends when ctx does, as
|
||||
// when the visitor goes away.
|
||||
|
||||
@@ -46,7 +46,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
@@ -135,7 +135,7 @@ type Params struct {
|
||||
// sww runs.
|
||||
Ledger *bans.Ledger
|
||||
Limiter *ratelimit.Limiter
|
||||
GeoJS *lookup.GeoJS
|
||||
GeoJS *geojs.GeoJS
|
||||
Lists *reputation.Lists
|
||||
DNSBL *reputation.DNSBL
|
||||
AbuseIPDB *reputation.AbuseIPDB
|
||||
@@ -195,8 +195,8 @@ type clientsFile struct {
|
||||
|
||||
// lookupsFile is lookups.json, with each answer on a line of its own.
|
||||
type lookupsFile struct {
|
||||
Version int `json:"version"`
|
||||
Lookups []lookup.Answer `json:"lookups"`
|
||||
Version int `json:"version"`
|
||||
Lookups []geojs.Answer `json:"lookups"`
|
||||
}
|
||||
|
||||
// abuseIPDBChecksSpentFile is abuseipdb-checks-spent.json, indented for an
|
||||
|
||||
@@ -26,7 +26,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
@@ -159,7 +159,7 @@ const filledAlertsJSON = `{
|
||||
"waiting": {
|
||||
"webhook": [
|
||||
{
|
||||
"instance": "fsn1app1/blog",
|
||||
"instance": "host1/blog",
|
||||
"time": "2026-10-06T00:00:00Z",
|
||||
"event": "ban",
|
||||
"client": "203.0.113.9",
|
||||
@@ -174,7 +174,7 @@ const filledAlertsJSON = `{
|
||||
"suppressed_repeats": 0
|
||||
},
|
||||
{
|
||||
"instance": "fsn1app1/blog",
|
||||
"instance": "host1/blog",
|
||||
"time": "2026-10-06T00:00:00Z",
|
||||
"event": "file_error",
|
||||
"client": "",
|
||||
@@ -1700,7 +1700,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
|
||||
t,
|
||||
lookupsJSON,
|
||||
params.GeoJS.Snapshot(),
|
||||
[]lookup.Answer{
|
||||
[]geojs.Answer{
|
||||
{Client: client, Country: "FR", Answered: midnight()},
|
||||
},
|
||||
)
|
||||
@@ -2398,7 +2398,7 @@ func newParams(dir string) state.Params {
|
||||
Events: alerts.Events(),
|
||||
Cooldown: 15 * time.Minute,
|
||||
MaxPerHour: 2,
|
||||
Instance: "fsn1app1/blog",
|
||||
Instance: "host1/blog",
|
||||
Now: midnight,
|
||||
})
|
||||
|
||||
@@ -2415,9 +2415,7 @@ func newParams(dir string) state.Params {
|
||||
MaxBans: 5000,
|
||||
}),
|
||||
Limiter: ratelimit.New(ratelimit.Limits{}, 20000),
|
||||
GeoJS: lookup.New(lookup.Params{
|
||||
Now: midnight, ProcessLog: discard, Metrics: m,
|
||||
}),
|
||||
GeoJS: geojs.New(geojs.Params{Now: midnight}),
|
||||
Lists: reputation.New(reputation.Params{
|
||||
BlocklistURLs: []string{
|
||||
blocklistURL,
|
||||
@@ -2494,7 +2492,7 @@ func fill(params state.Params) {
|
||||
})
|
||||
params.Limiter.AddLookup(client, now.Add(-time.Hour), asn, asName, "DE")
|
||||
|
||||
params.GeoJS.Load([]lookup.Answer{
|
||||
params.GeoJS.Load([]geojs.Answer{
|
||||
{
|
||||
Client: client, ASN: asn, ASName: asName, Country: "DE",
|
||||
Answered: now.Add(-time.Hour), Used: now,
|
||||
|
||||
@@ -260,7 +260,17 @@ func TestTheSiteAnswersWhileWAFIsStoppedAndStartedAgain(t *testing.T) {
|
||||
stop()
|
||||
}()
|
||||
|
||||
wantStatus(t, site+"/", placed, http.StatusForbidden)
|
||||
// The visitor's asks have opened rproxy's circuit breaker, which lets one
|
||||
// request a second ask until one is answered.
|
||||
deadline := time.Now().Add(waitLimit)
|
||||
for statusFrom(t, site+"/", placed) != http.StatusForbidden {
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatalf("request from %s: never answered %d", placed,
|
||||
http.StatusForbidden)
|
||||
}
|
||||
|
||||
time.Sleep(pollInterval)
|
||||
}
|
||||
|
||||
close(visitorStop)
|
||||
<-visitorDone
|
||||
|
||||
+2
-1
@@ -25,6 +25,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/remotelog"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
@@ -270,7 +271,7 @@ func newServer(
|
||||
Config: cfg,
|
||||
RequestLog: stdout,
|
||||
ProcessLog: processLog,
|
||||
GeoJSURL: lookup.URL,
|
||||
GeoJSURL: geojs.URL,
|
||||
AbuseIPDBURL: reputation.AbuseIPDBURL,
|
||||
LookupFile: lookupFile,
|
||||
CoreRuleSet: crs,
|
||||
|
||||
+14
-14
@@ -52,7 +52,7 @@ const (
|
||||
adminSecret = "fedcba9876543210fedcba9876543210"
|
||||
// instance is the SWW_WAF_INSTANCE_NAME the tests set where they look at
|
||||
// it.
|
||||
instance = "fsn1app1/blog"
|
||||
instance = "host1/blog"
|
||||
// greeting is what the tests' app answers.
|
||||
greeting = "hello from the app"
|
||||
// placed is the client the tests' lookup databases place.
|
||||
@@ -386,8 +386,8 @@ func TestEveryLogLineAndMetricCarriesTheInstanceName(t *testing.T) {
|
||||
// the comments.
|
||||
for line := range strings.Lines(metrics) {
|
||||
if !strings.HasPrefix(line, "#") &&
|
||||
!strings.Contains(line, `instance="fsn1app1/blog"`) {
|
||||
t.Errorf("series %q, without instance=\"fsn1app1/blog\"", line)
|
||||
!strings.Contains(line, `instance="host1/blog"`) {
|
||||
t.Errorf("series %q, without instance=\"host1/blog\"", line)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -734,12 +734,12 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
||||
t,
|
||||
url+"_sww/metrics",
|
||||
token,
|
||||
`sww_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`,
|
||||
`sww_lookup_database_read_failures_total{instance="host1/blog"} 1`,
|
||||
)
|
||||
lastRead = seriesValue(
|
||||
t,
|
||||
metrics,
|
||||
`sww_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`,
|
||||
`sww_lookup_database_last_read_timestamp_seconds{instance="host1/blog"}`,
|
||||
)
|
||||
|
||||
wantStatus(t, url, placed, http.StatusOK)
|
||||
@@ -801,7 +801,7 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
|
||||
// too.
|
||||
rateLimitExemptNets: placed + "," + localhost,
|
||||
}
|
||||
failures := `sww_reputation_failures_total{instance="fsn1app1/blog",` +
|
||||
failures := `sww_reputation_failures_total{instance="host1/blog",` +
|
||||
`source="` + torURL + `"} `
|
||||
|
||||
// tor.txt cannot be fetched at first, which is counted.
|
||||
@@ -984,8 +984,8 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
||||
// last.
|
||||
metrics := metricsText(t, url+"_sww/metrics", token)
|
||||
for _, series := range []string{
|
||||
`sww_remote_log_lines_sent_total{instance="fsn1app1/blog"} 0`,
|
||||
`sww_remote_log_lines_waiting{instance="fsn1app1/blog"} 1`,
|
||||
`sww_remote_log_lines_sent_total{instance="host1/blog"} 0`,
|
||||
`sww_remote_log_lines_waiting{instance="host1/blog"} 1`,
|
||||
} {
|
||||
if !strings.Contains(metrics, "\n"+series+"\n") {
|
||||
t.Errorf("no %q in the metrics:\n%s", series, metrics)
|
||||
@@ -993,7 +993,7 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
||||
}
|
||||
|
||||
const dropped = "\nsww_remote_log_lines_dropped_total" +
|
||||
`{instance="fsn1app1/blog"} `
|
||||
`{instance="host1/blog"} `
|
||||
if strings.Contains(metrics, dropped+"0\n") ||
|
||||
!strings.Contains(metrics, dropped) {
|
||||
t.Errorf("no line dropped in the metrics:\n%s", metrics)
|
||||
@@ -1089,7 +1089,7 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
||||
runUntilStopped(t, env, func(url string) {
|
||||
webhook.waitFor(t, "permanent_ban", true)
|
||||
|
||||
const ofWebhook = `{destination="webhook",instance="fsn1app1/blog"}`
|
||||
const ofWebhook = `{destination="webhook",instance="host1/blog"}`
|
||||
|
||||
metrics := metricsWith(t, url+"_sww/metrics", token,
|
||||
"\nsww_alerts_sent_total"+ofWebhook+" 1\n")
|
||||
@@ -1176,13 +1176,13 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
||||
err := json.Unmarshal([]byte(slackPost.body), &message)
|
||||
|
||||
if err != nil ||
|
||||
!strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") ||
|
||||
!strings.HasPrefix(message.Text, "*host1/blog: ban*\n") ||
|
||||
!strings.Contains(message.Text, "\nclient: "+client+"\n") {
|
||||
t.Errorf("Slack was sent %s", slackPost.body)
|
||||
}
|
||||
|
||||
ntfyPost := ntfy.firstPost(t)
|
||||
if ntfyPost.header.Get("Title") != "fsn1app1/blog: ban" ||
|
||||
if ntfyPost.header.Get("Title") != "host1/blog: ban" ||
|
||||
ntfyPost.header.Get("Authorization") != "Bearer "+ntfyToken ||
|
||||
!strings.Contains(ntfyPost.body, "\nclient: "+client+"\n") {
|
||||
t.Errorf("ntfy was sent %s, with the headers %v", ntfyPost.body,
|
||||
@@ -1194,9 +1194,9 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
||||
// test process cannot fail the test.
|
||||
sent := []string{
|
||||
"\nsww_alerts_sent_total{destination=\"slack\"," +
|
||||
"instance=\"fsn1app1/blog\"} 1\n",
|
||||
"instance=\"host1/blog\"} 1\n",
|
||||
"\nsww_alerts_sent_total{destination=\"ntfy\"," +
|
||||
"instance=\"fsn1app1/blog\"} 1\n",
|
||||
"instance=\"host1/blog\"} 1\n",
|
||||
}
|
||||
|
||||
metrics := metricsWith(t, url+"_sww/metrics", token, sent...)
|
||||
|
||||
@@ -18,7 +18,7 @@ const (
|
||||
alertWebhookURL = "SWW_WAF_ALERT_WEBHOOK_URL"
|
||||
alertMaxPerHour = "SWW_WAF_ALERT_MAX_PER_HOUR"
|
||||
// alertInstance is the instance every alert of these tests gives.
|
||||
alertInstance = "fsn1app1/blog"
|
||||
alertInstance = "host1/blog"
|
||||
)
|
||||
|
||||
func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
|
||||
|
||||
@@ -349,6 +349,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
|
||||
map[string]string{
|
||||
trustedProxies: trustLocalhost,
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
addLookupHeaders: falseValue,
|
||||
anomalyASN + requestsPerMinute: aLot,
|
||||
})
|
||||
server.Store(started)
|
||||
|
||||
@@ -20,7 +20,7 @@ import (
|
||||
"github.com/prometheus/client_golang/prometheus/testutil"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
@@ -39,6 +39,7 @@ const (
|
||||
// The header limits SPEC.md gives.
|
||||
cookieLimit = 50
|
||||
headerFieldLimit = 100
|
||||
lineLimit = 8192
|
||||
// cookieHeader is the header the cookie limit counts.
|
||||
cookieHeader = "Cookie"
|
||||
// tooLarge is how waf refuses a request over the header limits.
|
||||
@@ -104,11 +105,139 @@ func TestHeaderLimitRefusalNamesItsHeuristicCountsAndLimits(t *testing.T) {
|
||||
fillHeaders(req, headerFieldLimit+1)
|
||||
do(t, req)
|
||||
|
||||
// The longest header line is X-Request-ID's, with its 26-character id.
|
||||
wantHeuristic(
|
||||
t,
|
||||
s.out.requestLine(t),
|
||||
"header_limits",
|
||||
`{"fields": 101, "fields_limit": 100, "cookies": 0, "cookies_limit": 50}`,
|
||||
`{"longest_header_line_bytes": 40, "header_line_bytes_limit": 8192,
|
||||
"fields": 101, "fields_limit": 100, "cookies": 0, "cookies_limit": 50}`,
|
||||
)
|
||||
}
|
||||
|
||||
// longQuery returns a path whose request line, sent by Go's client as
|
||||
// "GET <path> HTTP/1.1", is lineBytes long.
|
||||
func longQuery(lineBytes int) string {
|
||||
const around = len("GET /?q= HTTP/1.1")
|
||||
|
||||
return "/?q=" + strings.Repeat("a", lineBytes-around)
|
||||
}
|
||||
|
||||
func TestARequestLineOverTheLimitIsRefusedWith414BeforeAnyOtherCheck(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
lineBytes int
|
||||
// cookies is how many cookies the request carries.
|
||||
cookies int
|
||||
status int
|
||||
action string
|
||||
}{
|
||||
{
|
||||
"at the limit", lineLimit, 0, http.StatusOK,
|
||||
requestlog.ActionForward,
|
||||
},
|
||||
{
|
||||
"over the limit", lineLimit + 1, 0, http.StatusRequestURITooLong,
|
||||
requestlog.ActionRequestLineTooLong,
|
||||
},
|
||||
{
|
||||
"over the limit and the cookie limit", lineLimit + 1,
|
||||
cookieLimit + 1, http.StatusRequestURITooLong,
|
||||
requestlog.ActionRequestLineTooLong,
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := startWAF(t, "", map[string]string{coreRuleSetMode: block})
|
||||
req := s.question(siteA, client, longQuery(tc.lineBytes))
|
||||
|
||||
if tc.cookies > 0 {
|
||||
req.Header.Set(cookieHeader, cookies(tc.cookies))
|
||||
}
|
||||
|
||||
got := do(t, req)
|
||||
line := s.out.requestLine(t)
|
||||
|
||||
if tc.status == http.StatusOK {
|
||||
wantDisposition(t, got, tc.status, waf.DispositionAllow)
|
||||
} else {
|
||||
wantDisposition(t, got, tc.status, waf.DispositionDeny)
|
||||
}
|
||||
|
||||
wantLine(t, line, tc.status, tc.action)
|
||||
|
||||
refused := tc.status != http.StatusOK
|
||||
if refused != (line.DurationCoreRuleSetMS == nil) {
|
||||
t.Errorf(
|
||||
"Core Rule Set time %v for a request refused: %t",
|
||||
line.DurationCoreRuleSetMS,
|
||||
refused,
|
||||
)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAHeaderLineOverTheLimitIsRefusedWith431(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
make func(*http.Request)
|
||||
// refused is true for a request over the limit, and false for one at it.
|
||||
refused bool
|
||||
}{
|
||||
{"a header line at the limit", func(req *http.Request) {
|
||||
req.Header.Set("X-Long", strings.Repeat("a", lineLimit-len("X-Long: ")))
|
||||
}, false},
|
||||
{"a header line over the limit", func(req *http.Request) {
|
||||
req.Header.Set("X-Long", strings.Repeat("a", lineLimit+1-len("X-Long: ")))
|
||||
}, true},
|
||||
{"one cookie over the limit", func(req *http.Request) {
|
||||
req.Header.Set(cookieHeader, "a="+strings.Repeat("a", lineLimit))
|
||||
}, true},
|
||||
{"a Host line over the limit", func(req *http.Request) {
|
||||
req.Host = strings.Repeat("a", lineLimit+1-len("Host: "))
|
||||
}, true},
|
||||
{"a header sent twice, each line within the limit", func(req *http.Request) {
|
||||
req.Header.Add("X-Long", strings.Repeat("a", 8000))
|
||||
req.Header.Add("X-Long", strings.Repeat("a", 8000))
|
||||
}, false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := startWAF(t, "", nil)
|
||||
req := s.question(siteA, client, "/")
|
||||
tc.make(req)
|
||||
|
||||
got := do(t, req)
|
||||
|
||||
if tc.refused {
|
||||
wantDisposition(t, got, tooLarge, waf.DispositionDeny)
|
||||
} else {
|
||||
wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRequestLineRefusalNamesItsHeuristicLengthAndLimit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := startWAF(t, "", nil)
|
||||
do(t, s.question(siteA, client, longQuery(lineLimit+1)))
|
||||
|
||||
wantHeuristic(
|
||||
t,
|
||||
s.out.requestLine(t),
|
||||
"header_limits",
|
||||
`{"request_line_bytes": 8193, "request_line_bytes_limit": 8192}`,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -462,7 +591,7 @@ func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(
|
||||
s := startWAF(t, geojsURL, map[string]string{
|
||||
unknownCountryLimitPercent: "0", rateLimitPerMinute: "1",
|
||||
})
|
||||
s.server.GeoJS.Load([]lookup.Answer{{
|
||||
s.server.GeoJS.Load([]geojs.Answer{{
|
||||
Client: netip.MustParsePrefix(client + "/32"), ASN: "AS64512",
|
||||
Answered: s.clock.Now(), Used: s.clock.Now(),
|
||||
}})
|
||||
@@ -641,6 +770,8 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
|
||||
// so that a request that does not wait takes no time at all.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
defer waitForGeoJSRequestsToBeAbandoned()
|
||||
|
||||
server, out := newWAFInBubble(t, tc.env)
|
||||
tc.prepare(t, server, time.Now())
|
||||
|
||||
@@ -709,6 +840,8 @@ func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
defer waitForGeoJSRequestsToBeAbandoned()
|
||||
|
||||
server, out := newWAFInBubble(t, tc.env)
|
||||
tc.prepare(t, server, time.Now())
|
||||
|
||||
@@ -821,10 +954,11 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
|
||||
t.Errorf("%d checks with AbuseIPDB, want 1", checked)
|
||||
}
|
||||
|
||||
// The query is answered, and the check with AbuseIPDB abandoned,
|
||||
// before the bubble ends.
|
||||
// The query is answered, and the check with AbuseIPDB and the request
|
||||
// to GeoJS abandoned, before the bubble ends.
|
||||
releaseResolver()
|
||||
time.Sleep(defaultDNSBLAbuseIPDBWait)
|
||||
waitForGeoJSRequestsToBeAbandoned()
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -339,6 +339,8 @@ func TestClientWithoutAnAnswerInTimeHasTheUnknownCountryLimitPercent(
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says, with
|
||||
// a GeoJS that never answers.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
defer waitForGeoJSRequestsToBeAbandoned()
|
||||
|
||||
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
|
||||
map[string]string{unknownCountryLimitPercent: "0"})
|
||||
|
||||
@@ -387,14 +389,18 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
|
||||
|
||||
// In a synctest bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
|
||||
// with a GeoJS that never answers.
|
||||
// with a GeoJS that never answers. The headers, which need the
|
||||
// answer too, are off.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
defer waitForGeoJSRequestsToBeAbandoned()
|
||||
|
||||
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
|
||||
map[string]string{
|
||||
asnAndCountryLookupTimeout: strconv.FormatInt(
|
||||
timeout.Milliseconds(), 10),
|
||||
tagLists: "vpn=" + vpnList,
|
||||
tc.setting: tc.value,
|
||||
addLookupHeaders: falseValue,
|
||||
tagLists: "vpn=" + vpnList,
|
||||
tc.setting: tc.value,
|
||||
})
|
||||
began := time.Now()
|
||||
|
||||
|
||||
@@ -108,8 +108,11 @@ func TestCountryRefusalNamesItsHeuristicCountryAndSetting(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
geojsURL, _ := startGeoJS(t)
|
||||
// The headers are off, so that only the country list makes the request
|
||||
// wait for its answer.
|
||||
s, _, _ := startWithClock(t, geojsURL, map[string]string{
|
||||
asnAndCountryLookupTimeout: "3600000", allowedCountries: "de",
|
||||
asnAndCountryLookupTimeout: "3600000", addLookupHeaders: falseValue,
|
||||
allowedCountries: "de",
|
||||
})
|
||||
|
||||
line := s.get(
|
||||
@@ -239,7 +242,10 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
|
||||
name string
|
||||
env map[string]string
|
||||
}{
|
||||
{"no setting needs the lookup", nil},
|
||||
{
|
||||
"no setting needs the lookup",
|
||||
map[string]string{addLookupHeaders: falseValue},
|
||||
},
|
||||
{"a tag setting needs it", map[string]string{tagDeny: denyKP}},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
|
||||
@@ -10,16 +10,22 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
// The header limits: the most cookies and header fields a request may have.
|
||||
// Real clients never reach them.
|
||||
// The header limits: the most cookies and header fields a request may have,
|
||||
// and the longest its request line and each header line may be, in bytes, not
|
||||
// counting the line ending. Real clients rarely reach them. maxLineBytes is
|
||||
// what nginx allows by default.
|
||||
const (
|
||||
maxCookies = 50
|
||||
maxHeaderFields = 100
|
||||
maxLineBytes = 8 << 10
|
||||
)
|
||||
|
||||
// headerLimits refuses a request with more than maxHeaderFields header fields
|
||||
// or more than maxCookies cookies, with 431. Its figures are the request's
|
||||
// header fields and cookies, and the limit of each.
|
||||
// headerLimits refuses a request with a request line longer than maxLineBytes,
|
||||
// with 414. It refuses a request with a header line longer than maxLineBytes,
|
||||
// more than maxHeaderFields header fields or more than maxCookies cookies, with
|
||||
// 431. Its figures are the request line's length and its limit for 414. For
|
||||
// 431 they are the longest header line, the header fields and the cookies,
|
||||
// each with its limit.
|
||||
type headerLimits struct{}
|
||||
|
||||
func (headerLimits) name() string { return "header_limits" }
|
||||
@@ -29,20 +35,59 @@ func (headerLimits) check(
|
||||
rq *request,
|
||||
_ time.Time,
|
||||
) *verdict {
|
||||
requestLine := rq.requestLineBytes()
|
||||
if requestLine > maxLineBytes {
|
||||
return &verdict{
|
||||
action: requestlog.ActionRequestLineTooLong,
|
||||
figures: map[string]any{
|
||||
"request_line_bytes": requestLine,
|
||||
"request_line_bytes_limit": maxLineBytes,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
longest := rq.longestHeaderLineBytes()
|
||||
fields, cookies := rq.headerCounts()
|
||||
if fields <= maxHeaderFields && cookies <= maxCookies {
|
||||
|
||||
if longest <= maxLineBytes && fields <= maxHeaderFields &&
|
||||
cookies <= maxCookies {
|
||||
return nil
|
||||
}
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionRequestHeaderTooLarge,
|
||||
figures: map[string]any{
|
||||
"fields": fields, "fields_limit": maxHeaderFields,
|
||||
"longest_header_line_bytes": longest,
|
||||
"header_line_bytes_limit": maxLineBytes,
|
||||
"fields": fields, "fields_limit": maxHeaderFields,
|
||||
"cookies": cookies, "cookies_limit": maxCookies,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// requestLineBytes returns the length of the request's request line, such as
|
||||
// "GET /search?q=x HTTP/1.1", not counting its line ending. The target is the
|
||||
// one the visitor sent, which rproxy's question carries unchanged.
|
||||
func (rq *request) requestLineBytes() int {
|
||||
return len(rq.in.Method) + len(" ") + len(rq.in.RequestURI) +
|
||||
len(" ") + len(rq.in.Proto)
|
||||
}
|
||||
|
||||
// longestHeaderLineBytes returns the length of the request's longest header
|
||||
// line, such as "Cookie: a=1", not counting its line ending. Each value of a
|
||||
// header sent twice is a line of its own, and Host is a line too.
|
||||
func (rq *request) longestHeaderLineBytes() int {
|
||||
longest := len("Host: ") + len(rq.in.Host)
|
||||
|
||||
for name, values := range rq.in.Header {
|
||||
for _, value := range values {
|
||||
longest = max(longest, len(name)+len(": ")+len(value))
|
||||
}
|
||||
}
|
||||
|
||||
return longest
|
||||
}
|
||||
|
||||
// headerCounts returns the request's header fields, each value of a header sent
|
||||
// twice being one, and its cookies, each part of a Cookie header between
|
||||
// semicolons being one unless it is empty or only spaces and tabs. The Core
|
||||
|
||||
@@ -41,8 +41,8 @@ const (
|
||||
|
||||
// beforeLookup are the heuristics that run before the client's AS number and
|
||||
// country are looked up, so that a client they refuse is not looked up.
|
||||
// header_limits comes first: it only counts, and a request it refuses causes no
|
||||
// other work.
|
||||
// header_limits comes first: it only measures and counts, and a request it
|
||||
// refuses causes no other work.
|
||||
func beforeLookup() []heuristic {
|
||||
return []heuristic{
|
||||
headerLimits{},
|
||||
|
||||
+27
-7
@@ -7,12 +7,13 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
)
|
||||
|
||||
// The headers in which an answer of allow gives rproxy the client's AS number
|
||||
// and country, for the app, while SWW_WAF_ADD_LOOKUP_HEADERS is set. Go writes
|
||||
// and country, for the app, while SWW_WAF_ADD_LOOKUP_HEADERS is true. Go writes
|
||||
// every header name in this form, as it sends it and as it receives it, so
|
||||
// X-Client-ASN arrives as X-Client-Asn; header names are not case-sensitive.
|
||||
const (
|
||||
@@ -26,9 +27,9 @@ const (
|
||||
// SWW_WAF_LOOKUP_SOURCE is off or the client is on a private, loopback or
|
||||
// link-local address, which no lookup can place. The lookup database
|
||||
// answers at once. With GeoJS, while a setting needs the answer, such as a
|
||||
// country list or a biased threshold, a new client's request waits for it,
|
||||
// unless refusedWhateverTheAnswer reports true. ctx is the request's own
|
||||
// context.
|
||||
// country list or a biased threshold, a new client's request waits for it up
|
||||
// to SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS, unless refusedWhateverTheAnswer
|
||||
// reports true. ctx is the request's own context.
|
||||
func (rq *request) lookUp(ctx context.Context, now time.Time) {
|
||||
if rq.h.config.LookupSource == "off" || !canBePlaced(rq.client) {
|
||||
return
|
||||
@@ -42,8 +43,11 @@ func (rq *request) lookUp(ctx context.Context, now time.Time) {
|
||||
case rq.refusedWhateverTheAnswer(now):
|
||||
rq.lookupAnswer = rq.h.geojs.LookUpWithoutWaiting(ctx, client)
|
||||
rq.skippedLookupWait = rq.lookupAnswer.Answered.IsZero()
|
||||
case !rq.h.waitForGeoJS:
|
||||
rq.lookupAnswer = rq.h.geojs.LookUpWithoutWaiting(ctx, client)
|
||||
default:
|
||||
rq.lookupAnswer = rq.h.geojs.LookUp(ctx, client)
|
||||
rq.lookupAnswer = rq.h.geojs.LookUp(ctx, client,
|
||||
rq.h.config.ASNAndCountryLookupTimeout)
|
||||
}
|
||||
|
||||
rq.lookedUp = true
|
||||
@@ -83,13 +87,29 @@ func countryTag(country string) string {
|
||||
// addLookup adds answer, an answer about a client from the lookup
|
||||
// database or GeoJS, to the client's history, and to the notes of the bans
|
||||
// on its netblock that have no AS number, AS name or country yet.
|
||||
func (h *handler) addLookup(answer lookup.Answer) {
|
||||
func (h *handler) addLookup(answer geojs.Answer) {
|
||||
h.limiter.AddLookup(answer.Client, answer.Answered,
|
||||
answer.ASN, answer.ASName, answer.Country)
|
||||
h.ledger.AddLookup(h.netblock(answer.Client.Addr()),
|
||||
answer.ASN, answer.ASName, answer.Country)
|
||||
}
|
||||
|
||||
// geojsFailed logs a failure of GeoJS, raises a source_failure alert for it
|
||||
// and counts it. askingAgainIn is how long GeoJS is left alone after it.
|
||||
func (h *handler) geojsFailed(err error, askingAgainIn time.Duration) {
|
||||
h.processLog.Warn("asking GeoJS failed",
|
||||
"error", err.Error(), "asking_again_in", askingAgainIn.String())
|
||||
h.alerts.Raise(alerts.Alert{
|
||||
Event: alerts.EventSourceFailure,
|
||||
Reason: "asking GeoJS failed",
|
||||
Detail: map[string]any{
|
||||
"source": "geojs", "error": err.Error(),
|
||||
"asking_again_in": askingAgainIn.String(),
|
||||
},
|
||||
})
|
||||
h.metrics.GeoJSFailures.Inc()
|
||||
}
|
||||
|
||||
// setLookupHeaders sets the headers that give the client's AS number and
|
||||
// country, leaving out one that is unknown.
|
||||
func setLookupHeaders(header http.Header, asn, country string) {
|
||||
|
||||
@@ -12,7 +12,7 @@ import (
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
@@ -36,6 +36,7 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(
|
||||
geojsURL, asked, release := startHeldGeoJS(t)
|
||||
s, _, server := startWithClock(t, geojsURL, map[string]string{
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
addLookupHeaders: falseValue,
|
||||
rateLimitPerMinute: "1",
|
||||
})
|
||||
|
||||
@@ -137,7 +138,7 @@ func TestRequestThatWaitedForItsLookupIsCountedWhenItGoesOn(t *testing.T) {
|
||||
// of lookups.json would keep it, while the first request still waits
|
||||
// for GeoJS. The second request goes on at once, and is counted.
|
||||
clk.advance(2500 * time.Millisecond)
|
||||
server.GeoJS.Load([]lookup.Answer{{
|
||||
server.GeoJS.Load([]geojs.Answer{{
|
||||
Client: netip.MustParsePrefix(
|
||||
fromDE + "/32",
|
||||
), ASN: asnDE, ASName: asNameDE,
|
||||
@@ -198,7 +199,7 @@ func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(
|
||||
// The answer is kept before the requests, so GeoJS is not asked, and
|
||||
// gives no answer of its own.
|
||||
netblock := netip.MustParsePrefix(fromDE + "/32")
|
||||
server.GeoJS.Load([]lookup.Answer{{
|
||||
server.GeoJS.Load([]geojs.Answer{{
|
||||
Client: netblock, ASN: asnDE, ASName: asNameDE, Country: "DE",
|
||||
Answered: clk.Now(), Used: clk.Now(),
|
||||
}})
|
||||
@@ -248,7 +249,7 @@ func TestLookupSourceOffLooksNoClientUp(t *testing.T) {
|
||||
)
|
||||
|
||||
// Even an answer kept from before is not used.
|
||||
server.GeoJS.Load([]lookup.Answer{{
|
||||
server.GeoJS.Load([]geojs.Answer{{
|
||||
Client: netip.MustParsePrefix(
|
||||
fromDE + "/32",
|
||||
), ASN: asnDE, ASName: asNameDE,
|
||||
@@ -418,7 +419,7 @@ func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(
|
||||
geojsURL,
|
||||
time.Now,
|
||||
map[string]string{
|
||||
trustedProxies: trustLocalhost,
|
||||
trustedProxies: trustLocalhost, addLookupHeaders: falseValue,
|
||||
},
|
||||
)
|
||||
s := &sender{t: t, addr: addr, out: out}
|
||||
@@ -449,6 +450,8 @@ func TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays(
|
||||
// network, which would keep that clock from moving on: the request is
|
||||
// handed to waf's handler, and GeoJS is one that never answers.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
defer waitForGeoJSRequestsToBeAbandoned()
|
||||
|
||||
// Not the default. The exclusive list needs the answer, and the app
|
||||
// is never reached.
|
||||
const timeout = 3 * time.Second
|
||||
@@ -457,6 +460,8 @@ func TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays(
|
||||
map[string]string{
|
||||
asnAndCountryLookupTimeout: strconv.FormatInt(
|
||||
timeout.Milliseconds(), 10),
|
||||
// Off, so that only the country list makes the request wait.
|
||||
addLookupHeaders: falseValue,
|
||||
allowedCountries: "DE",
|
||||
})
|
||||
|
||||
@@ -492,6 +497,17 @@ func TestMain(m *testing.M) {
|
||||
m.Run()
|
||||
}
|
||||
|
||||
// geojsRequestTimeout is how long a request to GeoJS may take before it is
|
||||
// abandoned.
|
||||
const geojsRequestTimeout = 10 * time.Second
|
||||
|
||||
// waitForGeoJSRequestsToBeAbandoned sleeps until every request to the GeoJS at
|
||||
// silentGeoJSURL made so far has been abandoned. A synctest bubble that asks
|
||||
// it defers this: the bubble cannot end while a request waits on it.
|
||||
func waitForGeoJSRequestsToBeAbandoned() {
|
||||
time.Sleep(geojsRequestTimeout)
|
||||
}
|
||||
|
||||
// silentGeoJS is the GeoJS at silentGeoJSURL.
|
||||
type silentGeoJS struct{}
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package waf_test
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
@@ -10,8 +11,9 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
)
|
||||
@@ -248,7 +250,7 @@ func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
|
||||
|
||||
// The answers are kept before the requests, so that none waits for
|
||||
// GeoJS.
|
||||
server.GeoJS.Load([]lookup.Answer{
|
||||
server.GeoJS.Load([]geojs.Answer{
|
||||
keptAnswer(fromKP, "KP"), keptAnswer(fromDE, "DE"),
|
||||
keptAnswer(fromFR, "FR"),
|
||||
})
|
||||
@@ -329,13 +331,13 @@ func TestMetricsByASNumberKeepTheBusiestAndCountTheRestAsOther(
|
||||
|
||||
// The answers are kept before the requests, so that GeoJS gives none
|
||||
// of its own. Each client is in an AS of its own.
|
||||
answer := func(addr, asn string) lookup.Answer {
|
||||
return lookup.Answer{
|
||||
answer := func(addr, asn string) geojs.Answer {
|
||||
return geojs.Answer{
|
||||
Client: netip.MustParsePrefix(addr + "/32"), ASN: asn,
|
||||
Answered: clk.Now(), Used: clk.Now(),
|
||||
}
|
||||
}
|
||||
server.GeoJS.Load([]lookup.Answer{
|
||||
server.GeoJS.Load([]geojs.Answer{
|
||||
answer(fromDE, "AS64501"), answer(fromKP, "AS64502"),
|
||||
})
|
||||
|
||||
@@ -357,21 +359,23 @@ func TestMetricsByASNumberKeepTheBusiestAndCountTheRestAsOther(
|
||||
`sww_asn_requests_total{asn="AS64502",instance="app"}`)
|
||||
}
|
||||
|
||||
func TestMetricsCountGeoJSRequestsAndFailures(t *testing.T) {
|
||||
func TestGeoJSFailureIsCountedLoggedAndAlerted(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
geojs := httptest.NewServer(http.HandlerFunc(
|
||||
failingGeoJS := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusServiceUnavailable)
|
||||
}))
|
||||
t.Cleanup(geojs.Close)
|
||||
t.Cleanup(failingGeoJS.Close)
|
||||
|
||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
|
||||
addr, _, server := startProxyWithClock(t, app.URL, geojs.URL, time.Now,
|
||||
addr, out, server, queue := startProxyWithAlerts(t, app.URL,
|
||||
failingGeoJS.URL, time.Now,
|
||||
map[string]string{
|
||||
trustedProxies: trustLocalhost,
|
||||
metricsToken: token,
|
||||
tagDeny: denyKP,
|
||||
trustedProxies: trustLocalhost,
|
||||
metricsToken: token,
|
||||
tagDeny: denyKP,
|
||||
alertWebhookURL: "https://alerts.example/waf",
|
||||
})
|
||||
|
||||
// GeoJS fails, so the client has no country= tag, and SWW_WAF_TAG_DENY does
|
||||
@@ -401,14 +405,46 @@ func TestMetricsCountGeoJSRequestsAndFailures(t *testing.T) {
|
||||
`sww_requests_without_geojs_answer_total{instance="app"}`,
|
||||
1,
|
||||
)
|
||||
|
||||
// The process log's line names the failure but not the address asked
|
||||
// about.
|
||||
logged := ""
|
||||
|
||||
for line := range strings.Lines(out.text()) {
|
||||
if strings.Contains(line, "asking GeoJS failed") {
|
||||
logged = line
|
||||
}
|
||||
}
|
||||
|
||||
if logged == "" || strings.Contains(logged, fromDE) {
|
||||
t.Errorf(
|
||||
"logged %q, want the failure without the address asked about",
|
||||
logged,
|
||||
)
|
||||
}
|
||||
|
||||
// fmt prints a map sorted by its keys.
|
||||
const want = "map[asking_again_in:1s " +
|
||||
"error:GeoJS answered 503 Service Unavailable source:geojs]"
|
||||
|
||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||
if len(waiting) != 1 || waiting[0].Event != alerts.EventSourceFailure ||
|
||||
waiting[0].Reason != "asking GeoJS failed" ||
|
||||
fmt.Sprint(waiting[0].Detail) != want {
|
||||
t.Errorf(
|
||||
"alerts waiting %+v, want one source_failure alert with %v",
|
||||
waiting,
|
||||
want,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// keptAnswer returns GeoJS's answer that the client at addr is in
|
||||
// country, given now.
|
||||
func keptAnswer(addr, country string) lookup.Answer {
|
||||
func keptAnswer(addr, country string) geojs.Answer {
|
||||
now := time.Now()
|
||||
|
||||
return lookup.Answer{
|
||||
return geojs.Answer{
|
||||
Client: netip.MustParsePrefix(addr + "/32"), Country: country,
|
||||
Answered: now, Used: now,
|
||||
}
|
||||
|
||||
+28
-31
@@ -17,6 +17,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
@@ -55,7 +56,7 @@ type Params struct {
|
||||
// ProcessLog receives the process's own messages.
|
||||
ProcessLog *slog.Logger
|
||||
// GeoJSURL is where clients' AS numbers and countries are looked up
|
||||
// while SWW_WAF_LOOKUP_SOURCE is geojs, normally lookup.URL.
|
||||
// while SWW_WAF_LOOKUP_SOURCE is geojs, normally geojs.URL.
|
||||
GeoJSURL string
|
||||
// AbuseIPDBURL is where clients are checked with AbuseIPDB while
|
||||
// SWW_WAF_ABUSEIPDB_KEY is set, normally reputation.AbuseIPDBURL.
|
||||
@@ -93,7 +94,7 @@ type Server struct {
|
||||
|
||||
Ledger *bans.Ledger
|
||||
Limiter *ratelimit.Limiter
|
||||
GeoJS *lookup.GeoJS
|
||||
GeoJS *geojs.GeoJS
|
||||
Anomalies *anomaly.Counters
|
||||
LookupFile *lookup.File
|
||||
Lists *reputation.Lists
|
||||
@@ -164,20 +165,19 @@ func newHandler(params Params) *handler {
|
||||
})
|
||||
}
|
||||
|
||||
h.geojs = lookup.New(lookup.Params{
|
||||
URL: params.GeoJSURL,
|
||||
Timeout: params.Config.ASNAndCountryLookupTimeout,
|
||||
// The country list, the headers, the tag settings that name an asn=
|
||||
// or country= tag and the biased thresholds act on the answer before
|
||||
// the request goes on.
|
||||
Wait: len(params.Config.ExclusivelyAllowedCountries) > 0 ||
|
||||
params.Config.AddLookupHeaders || params.Config.TagsNeedLookup() ||
|
||||
biasedThresholdsSet(params.Config),
|
||||
Answered: h.addLookup,
|
||||
Now: params.Now,
|
||||
ProcessLog: params.ProcessLog,
|
||||
Metrics: m,
|
||||
Alerts: params.Alerts,
|
||||
// The country list, the headers, the tag settings that name an asn= or
|
||||
// country= tag and the biased thresholds act on the answer before the
|
||||
// request goes on.
|
||||
h.waitForGeoJS = len(params.Config.ExclusivelyAllowedCountries) > 0 ||
|
||||
params.Config.AddLookupHeaders || params.Config.TagsNeedLookup() ||
|
||||
biasedThresholdsSet(params.Config)
|
||||
h.geojs = geojs.New(geojs.Params{
|
||||
URL: params.GeoJSURL,
|
||||
Answered: h.addLookup,
|
||||
Now: params.Now,
|
||||
Asked: m.GeoJSRequests.Inc,
|
||||
Failed: h.geojsFailed,
|
||||
WentWithoutAnswer: m.RequestsWithoutGeoJSAnswer.Inc,
|
||||
})
|
||||
m.AddBansAndClients(h.ledger, h.limiter, params.Now)
|
||||
m.AddRules(params.Rules)
|
||||
@@ -228,12 +228,9 @@ func newReputation(
|
||||
// without the rules SWW_WAF_CORE_RULE_SET_DISABLED_RULES switches off, reading bodies
|
||||
// up to SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE, with the exclusions in
|
||||
// SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE and the fixes of the apps in SWW_WAF_APPS,
|
||||
// or nil while SWW_WAF_CORE_RULE_SET_MODE is off. It
|
||||
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
|
||||
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
|
||||
// JSON or XML body over its limit. An exclusions file that coreruleset.New refuses is
|
||||
// an error in every mode. While the mode is off, a set file is loaded with the
|
||||
// Core Rule Set only to check it.
|
||||
// or nil while SWW_WAF_CORE_RULE_SET_MODE is off. An exclusions file that
|
||||
// coreruleset.New refuses is an error in every mode. While the mode is off, a
|
||||
// set file is loaded with the Core Rule Set only to check it.
|
||||
func NewCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
|
||||
if cfg.CoreRuleSetMode == config.CoreRuleSetModeOff &&
|
||||
cfg.CoreRuleSetExclusionsFile == "" {
|
||||
@@ -241,14 +238,11 @@ func NewCoreRuleSet(cfg *config.WAF) (*coreruleset.CoreRuleSet, error) {
|
||||
}
|
||||
|
||||
coreRuleSet, err := coreruleset.New(coreruleset.Params{
|
||||
ParanoiaLevel: cfg.CoreRuleSetParanoiaLevel,
|
||||
DisabledRules: cfg.CoreRuleSetDisabledRules,
|
||||
BodyInspectionSize: min(
|
||||
cfg.CoreRuleSetBodyInspectionSize,
|
||||
QuestionBodyMaxBytes-1,
|
||||
),
|
||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
||||
Apps: cfg.Apps,
|
||||
ParanoiaLevel: cfg.CoreRuleSetParanoiaLevel,
|
||||
DisabledRules: cfg.CoreRuleSetDisabledRules,
|
||||
BodyInspectionSize: cfg.CoreRuleSetBodyInspectionSize,
|
||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
||||
Apps: cfg.Apps,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(
|
||||
@@ -274,7 +268,7 @@ type handler struct {
|
||||
metrics *metrics.Metrics
|
||||
limiter *ratelimit.Limiter
|
||||
ledger *bans.Ledger
|
||||
geojs *lookup.GeoJS
|
||||
geojs *geojs.GeoJS
|
||||
anomalies *anomaly.Counters
|
||||
lookupFile *lookup.File
|
||||
lists *reputation.Lists
|
||||
@@ -284,6 +278,9 @@ type handler struct {
|
||||
rules *rules.Files
|
||||
coreRuleSet *coreruleset.CoreRuleSet
|
||||
alerts *alerts.Queue
|
||||
// waitForGeoJS is true when a setting needs each request's answer from
|
||||
// GeoJS before the request goes on.
|
||||
waitForGeoJS bool
|
||||
// awaitingReport holds the decision on each answered question by its id
|
||||
// until its report comes, and nil for a question still being handled.
|
||||
awaitingReport *lru.Cache[string, *decision]
|
||||
|
||||
@@ -93,6 +93,10 @@ const off = "off"
|
||||
// on is the value that switches a setting that takes true or false on.
|
||||
const on = "true"
|
||||
|
||||
// falseValue is the value that switches a setting that takes true or false
|
||||
// off.
|
||||
const falseValue = "false"
|
||||
|
||||
// output collects what sww writes on stdout.
|
||||
type output struct {
|
||||
mu sync.Mutex
|
||||
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
@@ -162,7 +162,7 @@ func TestRateLimitExemptPathsAreNeitherCountedNorRefused(t *testing.T) {
|
||||
|
||||
// The answers are kept before the requests, so that none waits for
|
||||
// GeoJS.
|
||||
server.GeoJS.Load([]lookup.Answer{
|
||||
server.GeoJS.Load([]geojs.Answer{
|
||||
keptAnswer(client, "DE"), keptAnswer(fromKP, "KP"),
|
||||
})
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||
@@ -42,7 +42,7 @@ type decision struct {
|
||||
// looked up, whether or not an answer was there, and lookupAnswer is
|
||||
// what the lookup gave then, the zero Answer while GeoJS had given none.
|
||||
lookedUp bool
|
||||
lookupAnswer lookup.Answer
|
||||
lookupAnswer geojs.Answer
|
||||
// skippedLookupWait is true for a request that did not wait for GeoJS and
|
||||
// found no answer kept. SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT does not apply to it,
|
||||
// so that not waiting never lowers its client's limits.
|
||||
@@ -177,7 +177,7 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
|
||||
// check is the one place where a request can be refused. It returns nil to let
|
||||
// the request through. The heuristics of checkClient decide: a request they
|
||||
// refuse is answered with SWW_WAF_BAN_RESPONSE, 403 for a block rule or the Core
|
||||
// Rule Set, or 431 for the header limits. In observe mode nothing is refused,
|
||||
// Rule Set, or 414 or 431 for the header limits. In observe mode nothing is refused,
|
||||
// and the log line names what enforce mode would have done. ctx is the
|
||||
// request's own context.
|
||||
func (rq *request) check(ctx context.Context) *refusal {
|
||||
@@ -196,6 +196,8 @@ func (rq *request) check(ctx context.Context) *refusal {
|
||||
case action == requestlog.ActionBlockedByBlockRule ||
|
||||
action == requestlog.ActionBlockedByCoreRuleSet:
|
||||
return &refusal{status: http.StatusForbidden, action: action}
|
||||
case action == requestlog.ActionRequestLineTooLong:
|
||||
return &refusal{status: http.StatusRequestURITooLong, action: action}
|
||||
case action == requestlog.ActionRequestHeaderTooLarge:
|
||||
return &refusal{
|
||||
status: http.StatusRequestHeaderFieldsTooLarge,
|
||||
@@ -380,9 +382,9 @@ func anomalyThresholdsSet(cfg *config.WAF) bool {
|
||||
// answer about it as the request ends, and whether there is one: the
|
||||
// lookup database's, which was there at once, or the one GeoJS has given
|
||||
// by then, which a request does not wait for unless a setting needs it.
|
||||
func (rq *request) answerAtTheEnd() (lookup.Answer, bool) {
|
||||
func (rq *request) answerAtTheEnd() (geojs.Answer, bool) {
|
||||
if !rq.lookedUp {
|
||||
return lookup.Answer{}, false
|
||||
return geojs.Answer{}, false
|
||||
}
|
||||
|
||||
if rq.h.config.LookupSource == "file" {
|
||||
|
||||
@@ -20,7 +20,7 @@ const (
|
||||
// requestIDHeader carries the request's id.
|
||||
requestIDHeader = "X-Request-ID"
|
||||
// instance is the SWW_WAF_INSTANCE_NAME a test sets.
|
||||
instance = "fsn1app1/blog"
|
||||
instance = "host1/blog"
|
||||
// ipv6Client is a client on IPv6, and ipv6Group the netblock the rate
|
||||
// limits count it as.
|
||||
ipv6Client = "2001:db8::7"
|
||||
|
||||
+1
-1
@@ -320,7 +320,7 @@ func asTheVisitorSentIt(r *http.Request, body []byte) *http.Request {
|
||||
}
|
||||
|
||||
// allow answers a question with DispositionAllow and, while
|
||||
// SWW_WAF_ADD_LOOKUP_HEADERS is set, the client's AS number and country.
|
||||
// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
|
||||
func (rq *request) allow() {
|
||||
rq.allowed = true
|
||||
|
||||
|
||||
@@ -869,7 +869,7 @@ func TestAllowCarriesTheLookupHeadersWhileTheSettingIsSet(t *testing.T) {
|
||||
wantASN, wantCountry string
|
||||
}{
|
||||
{on, "AS64496", "DE"},
|
||||
{"false", "", ""},
|
||||
{falseValue, "", ""},
|
||||
} {
|
||||
t.Run(tc.setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
@@ -1690,7 +1690,12 @@ func TestALargeBodyAnnouncedAndLongHeadersAreNoReasonToRefuse(t *testing.T) {
|
||||
strings.NewReader("a body"))
|
||||
req.Header.Set(waf.ContentLengthHeader, "1073741824")
|
||||
req.Header.Set(forwardedFor, otherClient)
|
||||
req.Header.Set("X-Long", strings.Repeat("x", 40<<10))
|
||||
|
||||
// 40,000 bytes of headers, over rproxy's 32 KiB, in lines within the
|
||||
// header limits.
|
||||
for i := range 5 {
|
||||
req.Header.Set("X-Long-"+strconv.Itoa(i), strings.Repeat("x", 8000))
|
||||
}
|
||||
|
||||
got := do(t, req)
|
||||
wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
|
||||
@@ -1782,7 +1787,7 @@ func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := startWAF(t, "", map[string]string{
|
||||
coreRuleSetMode: block, coreRuleSetBodyInspectionSize: "64K",
|
||||
coreRuleSetMode: block, coreRuleSetBodyInspectionSize: "8K",
|
||||
})
|
||||
|
||||
ask := func(contentType, body string) answer {
|
||||
@@ -1797,10 +1802,10 @@ func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) {
|
||||
got := ask(formData, "id=1'%20OR%20'1'='1")
|
||||
wantDisposition(t, got, http.StatusForbidden, waf.DispositionDeny)
|
||||
|
||||
// The first 64 KiB of a longer JSON body cannot be parsed, which is no
|
||||
// The first 8 KiB of a longer JSON body cannot be parsed, which is no
|
||||
// attack, so the Core Rule Set does not read it.
|
||||
cut := `{"text":"` + strings.Repeat("x", waf.QuestionBodyMaxBytes)
|
||||
got = ask("application/json", cut[:waf.QuestionBodyMaxBytes])
|
||||
cut := `{"text":"` + strings.Repeat("x", 8<<10)
|
||||
got = ask("application/json", cut)
|
||||
wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
|
||||
}
|
||||
|
||||
|
||||
+12
-6
@@ -2,8 +2,9 @@
|
||||
# script/publish: build the image on the Docker host and push it to the
|
||||
# registry under four tags: the UTC date (YYYY-MM-DD), the short commit
|
||||
# hash, the branch name and latest. The build is script/docker's, so the
|
||||
# tests and the linter run first. The host and the registry name are
|
||||
# environment variables with defaults, for running it against another.
|
||||
# tests and the linter run first. DOCKER_HOST names the Docker host and
|
||||
# PUBLISH_IMAGE the image in the registry. Both must be set; neither has
|
||||
# a default.
|
||||
#
|
||||
# The working tree must be clean: the hash tag names a commit, and an
|
||||
# image built from uncommitted changes would carry a hash that does not
|
||||
@@ -13,11 +14,16 @@ set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
: "${DOCKER_HOST:=ssh://root@fsn1app1}"
|
||||
: "${PUBLISH_IMAGE:=git.eeqj.de/sneak/smallwebwaf}"
|
||||
export DOCKER_HOST
|
||||
|
||||
main() {
|
||||
if [ -z "${DOCKER_HOST:-}" ]; then
|
||||
echo "script/publish: DOCKER_HOST is not set; set it to the Docker host that builds the image, such as ssh://user@build.example.com" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ -z "${PUBLISH_IMAGE:-}" ]; then
|
||||
echo "script/publish: PUBLISH_IMAGE is not set; set it to the image name in the registry, such as registry.example.com/smallwebwaf" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "$ROOT"
|
||||
|
||||
# Own line: a failing command substitution inside an argument does
|
||||
|
||||
Reference in new issue
Block a user