Compare commits

..
2 Commits
Author SHA1 Message Date
sneak b157359910 rproxy: a page of its own for a refused visitor
check / check (push) Failing after 1m48s
A deny of 403 from waf is answered with refused.html, a deny of 429
with ratelimited.html, each as text/html with waf's other headers kept.
Both pages are made once at start and held in memory. With
SWW_RPROXY_CONTACT_EMAIL set, their "contact the site's operator" is a
mailto link to it; the value is taken as given.
2026-11-19 21:12:31 -08:00
sneak 5dfa47c97b Tests: rproxy gives a refused visitor a page of its own
A deny of 403 from waf gets rproxy's refused page, a deny of 429 its
rate-limited page, each as text/html with waf's other headers kept. With
SWW_RPROXY_CONTACT_EMAIL set, the pages' sentence about the operator is a
mailto link. Every other answer from waf still goes to the visitor as waf
gave it. The two pages are beside the code, as the tests read them.
2026-11-19 21:10:10 -08:00
17 changed files with 380 additions and 184 deletions

No files matched your search

+10 -11
View File
@@ -288,9 +288,7 @@ account, a key, a token or an alert destination stays off until you give one.
- An anomaly is more requests or bytes than an `SWW_WAF_ANOMALY_` threshold - An anomaly is more requests or bytes than an `SWW_WAF_ANOMALY_` threshold
allows. Crossing a threshold sends one `anomaly` alert to the alert allows. Crossing a threshold sends one `anomaly` alert to the alert
destinations, and does nothing else: no client is banned, rate limited or destinations, and does nothing else: no client is banned, rate limited or
blocked. The thresholds per netblock and per AS number are on by default, set blocked.
high enough to alert only on obvious abuse. They are a starting point: tune
them to your own site's traffic, which the metrics show.
- An invalid setting stops the process at start with one log line naming the - An invalid setting stops the process at start with one log line naming the
variable and what is wrong with the value. The settings in effect are logged variable and what is wrong with the value. The settings in effect are logged
on one line at start, with tokens, keys and webhook paths shown as `********`. on one line at start, with tokens, keys and webhook paths shown as `********`.
@@ -310,6 +308,7 @@ Every setting the code reads, with its default:
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. | | `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. | | `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. | | `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
| `SWW_RPROXY_CONTACT_EMAIL` | unset | The email address the refusal pages link to. |
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. | | `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. | | `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. |
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. | | `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
@@ -398,16 +397,16 @@ Every setting the code reads, with its default:
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when one client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when one client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `1200` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `15000` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_BYTES_PER_MINUTE` | `5G` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_BYTES_PER_HOUR` | `50G` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 32. Those thresholds only send alerts. | | `SWW_WAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 32. Those thresholds only send alerts. |
| `SWW_WAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 128. Those thresholds only send alerts. | | `SWW_WAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 128. Those thresholds only send alerts. |
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `6000`, or `off` while the lookup source is `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `75000`, or `off` while the lookup source is `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `25G`, or `off` while the lookup source is `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_HOUR` | `250G`, or `off` while the lookup source is `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in a minute, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in an hour, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the whole service in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the whole service in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
+9 -10
View File
@@ -2113,14 +2113,12 @@ host, sharing one address, would often fetch too close together.
### Anomaly thresholds ### Anomaly thresholds
These only alert; nothing is blocked. The thresholds per netblock and per AS These only alert; nothing is blocked. None is set by default, and a threshold
number have defaults high enough to alert only on obvious abuse. The others are left unset sends no alert. An alert needs a destination only the operator can
off by default. What counts as unusual depends on each service's normal traffic, supply, and what counts as unusual depends on each service's normal traffic,
which the metrics show, so the defaults are a starting point for the operator to which the metrics show. They reveal a widely distributed scraper, thousands of
tune. An alert goes nowhere until the operator sets a destination. The addresses at low rates each, which the per-client limits do not see. A low
thresholds reveal a widely distributed scraper, thousands of addresses at low percentage for its `asn=` tag in `SWW_WAF_TAG_LIMIT_PERCENT` is the response.
rates each, which the per-client limits do not see. A low percentage for its
`asn=` tag in `SWW_WAF_TAG_LIMIT_PERCENT` is the response.
- Requests and body bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted - Requests and body bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted
per minute and per hour, for each of: per minute and per hour, for each of:
@@ -2433,8 +2431,9 @@ but the last below carries `X-Sww-Disposition`.
carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
request it passes on. A header whose value is unknown is left out. request it passes on. A header whose value is unknown is left out.
- `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to - `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
get. `rproxy` gives them to the visitor, less this header. The status is one get. `rproxy` gives them to the visitor, less this header; a `403` or `429`
of: gets `rproxy`'s own page as its body, linking `SWW_RPROXY_CONTACT_EMAIL` when
set. The status is one of:
- `SWW_WAF_BAN_RESPONSE`, for a refusal; - `SWW_WAF_BAN_RESPONSE`, for a refusal;
- `403`, for a `block` rule or the Core Rule Set; - `403`, for a `block` rule or the Core Rule Set;
- `414` or `431`, for a request over the header limits; - `414` or `431`, for a request over the header limits;
+3
View File
@@ -10,6 +10,9 @@ services:
SWW_RPROXY_WAF_BACKEND: http://waf:8080 SWW_RPROXY_WAF_BACKEND: http://waf:8080
SWW_RPROXY_UPSTREAM_URL: http://exampleapp:8081 SWW_RPROXY_UPSTREAM_URL: http://exampleapp:8081
# The address the refusal pages link to.
# SWW_RPROXY_CONTACT_EMAIL: ops@example.com
# The netblocks whose X-Forwarded-For rproxy believes. By default # The netblocks whose X-Forwarded-For rproxy believes. By default
# the private ranges, which cover a TLS-terminating proxy on the # the private ranges, which cover a TLS-terminating proxy on the
# Docker network. A proxy elsewhere needs its address added; a # Docker network. A proxy elsewhere needs its address added; a
+4 -21
View File
@@ -378,9 +378,8 @@ Crossing a threshold sends one `anomaly` alert to the destinations under
Each scope has four thresholds: the most requests and the most body bytes Each scope has four thresholds: the most requests and the most body bytes
counted in a minute and in an hour. The bytes are those `SWW_WAF_BYTES_COUNTED` counted in a minute and in an hour. The bytes are those `SWW_WAF_BYTES_COUNTED`
names. Requests are counts, bytes are sizes. Each is `off` by default, except names. Requests are counts, bytes are sizes. Each is `off` by default. The four
those per netblock and per AS number listed below. The four thresholds per thresholds per client are:
client are:
- `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`; - `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
- `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`; - `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
@@ -404,24 +403,6 @@ as `REQUESTS_PER_MINUTE`:
`office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
name, or with a bad netblock, is refused. So is a name used twice. name, or with a bad netblock, is refused. So is a name used twice.
The thresholds per netblock and per AS number have defaults set high enough to
alert only on obvious abuse:
- per netblock:
- `REQUESTS_PER_MINUTE` `1200` and `REQUESTS_PER_HOUR` `15000`, ten clients
at the default rate limits;
- `BYTES_PER_HOUR` `50G`, one client at the default byte limit per hour;
- `BYTES_PER_MINUTE` `5G`, a tenth of that.
- per AS number, five times those: `REQUESTS_PER_MINUTE` `6000`,
`REQUESTS_PER_HOUR` `75000`, `BYTES_PER_HOUR` `250G` and `BYTES_PER_MINUTE`
`25G`.
While `SWW_WAF_LOOKUP_SOURCE` is `off`, no request has an AS number, so the
thresholds per AS number are `off` by default. Setting one then stops the start.
These defaults are a starting point. Tune them to your own site's traffic, which
the metrics show.
## Settings given as files ## Settings given as files
To give a token as a file, as "Tokens" under "Deployment" in To give a token as a file, as "Tokens" under "Deployment" in
@@ -502,6 +483,8 @@ start, and an invalid value stops the start. Any can be given as a file, as a
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
do the status and headers of the app's answer. A time in milliseconds or do the status and headers of the app's answer. A time in milliseconds or
`off`. `off`.
- `SWW_RPROXY_CONTACT_EMAIL` (default unset): the email address the refusal
pages link to.
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as - `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
`:9000`. `:9000`.
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An - `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
+4
View File
@@ -46,6 +46,9 @@ type RProxy struct {
// (SWW_RPROXY_MAX_RESPONSE_TIME_MS). // (SWW_RPROXY_MAX_RESPONSE_TIME_MS).
MaxRequestTime time.Duration MaxRequestTime time.Duration
MaxResponseTime time.Duration MaxResponseTime time.Duration
// ContactEmail is the address the pages a refused visitor gets link to,
// or "" for no link (SWW_RPROXY_CONTACT_EMAIL). It is taken as given.
ContactEmail string
// settings are the values read, as given or by default, for the log line // settings are the values read, as given or by default, for the log line
// at start. // at start.
@@ -108,6 +111,7 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
"SWW_RPROXY_MAX_RESPONSE_TIME_MS", "SWW_RPROXY_MAX_RESPONSE_TIME_MS",
"1800000", "1800000",
), ),
ContactEmail: env.value("SWW_RPROXY_CONTACT_EMAIL", ""),
} }
if env.err != nil { if env.err != nil {
+2
View File
@@ -25,6 +25,7 @@ const (
rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE" rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE"
rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS" rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS" rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
rproxyContactEmail = "SWW_RPROXY_CONTACT_EMAIL"
backendURL = "http://waf:8080" backendURL = "http://waf:8080"
rproxyNoAnswerAddr = ":8080" rproxyNoAnswerAddr = ":8080"
rproxyNoAnswerApp = "http://127.0.0.1:8081" rproxyNoAnswerApp = "http://127.0.0.1:8081"
@@ -250,6 +251,7 @@ func TestRProxySettingsAreLoggedByName(t *testing.T) {
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout, rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout,
rproxyNoAnswer, rproxyNoAnswer,
rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime, rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime,
rproxyContactEmail,
} { } {
if !slices.Contains(names, name) { if !slices.Contains(names, name) {
t.Errorf("%s is not logged, only %v", name, names) t.Errorf("%s is not logged, only %v", name, names)
+13 -51
View File
@@ -313,9 +313,7 @@ type WAF struct {
// per netblock around a client (SWW_WAF_ANOMALY_NET_*), per AS number // per netblock around a client (SWW_WAF_ANOMALY_NET_*), per AS number
// (SWW_WAF_ANOMALY_ASN_*), for the whole service (SWW_WAF_ANOMALY_TOTAL_*) // (SWW_WAF_ANOMALY_ASN_*), for the whole service (SWW_WAF_ANOMALY_TOTAL_*)
// and per watched netblock (SWW_WAF_ANOMALY_WATCHED_NET_*), each 0 while // and per watched netblock (SWW_WAF_ANOMALY_WATCHED_NET_*), each 0 while
// it is off. Only the thresholds per netblock and per AS number are on by // it is off. AnomalyNetV4Prefix and AnomalyNetV6Prefix are the lengths
// default, those per AS number only while clients are looked up.
// AnomalyNetV4Prefix and AnomalyNetV6Prefix are the lengths
// of the netblock around a client (SWW_WAF_ANOMALY_NET_V4_PREFIX and // of the netblock around a client (SWW_WAF_ANOMALY_NET_V4_PREFIX and
// SWW_WAF_ANOMALY_NET_V6_PREFIX), and AnomalyWatchedNets the watched // SWW_WAF_ANOMALY_NET_V6_PREFIX), and AnomalyWatchedNets the watched
// netblocks (SWW_WAF_ANOMALY_WATCHED_NETS). // netblocks (SWW_WAF_ANOMALY_WATCHED_NETS).
@@ -557,6 +555,11 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
"SWW_WAF_ALERT_MAX_PER_HOUR", "SWW_WAF_ALERT_MAX_PER_HOUR",
"60", "60",
), ),
AnomalyClient: env.thresholds("SWW_WAF_ANOMALY_CLIENT_"),
AnomalyNet: env.thresholds("SWW_WAF_ANOMALY_NET_"),
AnomalyASN: env.thresholds("SWW_WAF_ANOMALY_ASN_"),
AnomalyTotal: env.thresholds("SWW_WAF_ANOMALY_TOTAL_"),
AnomalyWatchedNet: env.thresholds("SWW_WAF_ANOMALY_WATCHED_NET_"),
AnomalyNetV4Prefix: env.v4Prefix( AnomalyNetV4Prefix: env.v4Prefix(
"SWW_WAF_ANOMALY_NET_V4_PREFIX", "SWW_WAF_ANOMALY_NET_V4_PREFIX",
"24", "24",
@@ -575,7 +578,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
// The app gets the headers by default, unless no client is looked up. // The app gets the headers by default, unless no client is looked up.
cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS", cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS",
strconv.FormatBool(cfg.LookupSource != LookupSourceOff)) strconv.FormatBool(cfg.LookupSource != LookupSourceOff))
env.anomalyThresholds(cfg)
cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action( cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action(
BlocklistActionSetting, string(ReputationActionDeny)) BlocklistActionSetting, string(ReputationActionDeny))
cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action( cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action(
@@ -2691,55 +2693,15 @@ func (e *environment) duration(name, defaultValue string) time.Duration {
return duration return duration
} }
// thresholdDefaults are the defaults of a scope's four anomaly thresholds,
// each a count, a size or off.
type thresholdDefaults struct {
requestsPerMinute, requestsPerHour, bytesPerMinute, bytesPerHour string
}
// anomalyThresholds reads the anomaly thresholds of every scope into cfg.
// Only those per netblock and per AS number are on by default, high
// enough to alert on obvious abuse alone. Those per netblock count the
// requests of ten clients at the default per-client rate limits, and the
// bytes of one client at its default byte limit per hour, a tenth of
// that in a minute. Those per AS number are five times those per
// netblock. They are off by default while no client is looked up, since
// no request then has an AS number.
func (e *environment) anomalyThresholds(cfg *WAF) {
allOff := thresholdDefaults{off, off, off, off}
asnDefaults := thresholdDefaults{
requestsPerMinute: "6000", requestsPerHour: "75000",
bytesPerMinute: "25G", bytesPerHour: "250G",
}
if cfg.LookupSource == LookupSourceOff {
asnDefaults = allOff
}
cfg.AnomalyClient = e.thresholds("SWW_WAF_ANOMALY_CLIENT_", allOff)
cfg.AnomalyNet = e.thresholds("SWW_WAF_ANOMALY_NET_", thresholdDefaults{
requestsPerMinute: "1200", requestsPerHour: "15000",
bytesPerMinute: "5G", bytesPerHour: "50G",
})
cfg.AnomalyASN = e.thresholds("SWW_WAF_ANOMALY_ASN_", asnDefaults)
cfg.AnomalyTotal = e.thresholds("SWW_WAF_ANOMALY_TOTAL_", allOff)
cfg.AnomalyWatchedNet = e.thresholds("SWW_WAF_ANOMALY_WATCHED_NET_",
allOff)
}
// thresholds reads the four anomaly thresholds whose settings' names // thresholds reads the four anomaly thresholds whose settings' names
// start with prefix: requests and bytes per minute and per hour. // start with prefix: requests and bytes per minute and per hour. Each is
func (e *environment) thresholds( // off by default.
prefix string, defaults thresholdDefaults, func (e *environment) thresholds(prefix string) anomaly.Thresholds {
) anomaly.Thresholds {
return anomaly.Thresholds{ return anomaly.Thresholds{
RequestsPerMinute: e.count(prefix+"REQUESTS_PER_MINUTE", RequestsPerMinute: e.count(prefix+"REQUESTS_PER_MINUTE", off),
defaults.requestsPerMinute), RequestsPerHour: e.count(prefix+"REQUESTS_PER_HOUR", off),
RequestsPerHour: e.count(prefix+"REQUESTS_PER_HOUR", BytesPerMinute: e.size(prefix+"BYTES_PER_MINUTE", off),
defaults.requestsPerHour), BytesPerHour: e.size(prefix+"BYTES_PER_HOUR", off),
BytesPerMinute: e.size(prefix+"BYTES_PER_MINUTE",
defaults.bytesPerMinute),
BytesPerHour: e.size(prefix+"BYTES_PER_HOUR", defaults.bytesPerHour),
} }
} }
+1 -40
View File
@@ -178,17 +178,6 @@ func loggedAnomalyDefaults() map[string]string {
logged[name] = off logged[name] = off
} }
maps.Copy(logged, map[string]string{
anomalyNet + requestsPerMinute: "1200",
anomalyNet + requestsPerHour: "15000",
anomalyNet + bytesPerMinute: "5G",
anomalyNet + bytesPerHour: "50G",
anomalyASN + requestsPerMinute: "6000",
anomalyASN + requestsPerHour: "75000",
anomalyASN + bytesPerMinute: "25G",
anomalyASN + bytesPerHour: "250G",
})
return logged return logged
} }
@@ -1576,35 +1565,7 @@ func TestAnomalySettingsDefaults(t *testing.T) {
t.Parallel() t.Parallel()
cfg := wafFromEnvironment(t, environment{}) cfg := wafFromEnvironment(t, environment{})
wantAllOff(t, cfg)
const gibibyte = int64(1) << 30
want := map[string]anomaly.Thresholds{
anomalyClient: {},
anomalyNet: {
RequestsPerMinute: 1200, RequestsPerHour: 15000,
BytesPerMinute: 5 * gibibyte, BytesPerHour: 50 * gibibyte,
},
anomalyASN: {
RequestsPerMinute: 6000, RequestsPerHour: 75000,
BytesPerMinute: 25 * gibibyte, BytesPerHour: 250 * gibibyte,
},
anomalyTotal: {}, anomalyWatchedNet: {},
}
if got := thresholdsByScope(cfg); !maps.Equal(got, want) {
t.Errorf("thresholds by scope\n%+v\nwant\n%+v", got, want)
}
// With no client looked up, the thresholds per AS number are off.
want[anomalyASN] = anomaly.Thresholds{}
got := thresholdsByScope(
wafFromEnvironment(t, environment{lookupSource: off}))
if !maps.Equal(got, want) {
t.Errorf("with %s off, thresholds by scope\n%+v\nwant\n%+v",
lookupSource, got, want)
}
if cfg.AnomalyNetV4Prefix != 24 || cfg.AnomalyNetV6Prefix != 48 || if cfg.AnomalyNetV4Prefix != 24 || cfg.AnomalyNetV6Prefix != 48 ||
len(cfg.AnomalyWatchedNets) != 0 { len(cfg.AnomalyWatchedNets) != 0 {
+71
View File
@@ -0,0 +1,71 @@
package rproxy
import (
"embed"
"html"
"net/http"
"strings"
)
// pageFiles are the pages a refused visitor gets, as HTML with nothing
// external: refused.html for a 403, ratelimited.html for a 429.
//
//go:embed refused.html ratelimited.html
var pageFiles embed.FS
// contactSentence is the sentence of each page that becomes a link to
// SWW_RPROXY_CONTACT_EMAIL when that is set.
const contactSentence = "contact the site's operator"
// htmlContentType is the Content-Type of the pages.
const htmlContentType = "text/html; charset=utf-8"
// blockPages are the pages a refused visitor gets, ready to send, made once
// at start.
type blockPages struct {
refused []byte
rateLimited []byte
}
// newBlockPages makes the pages. With contactEmail set, each page's
// contactSentence is a mailto link to it.
func newBlockPages(contactEmail string) blockPages {
return blockPages{
refused: readPage("refused.html", contactEmail),
rateLimited: readPage("ratelimited.html", contactEmail),
}
}
// readPage reads the page in name from pageFiles, linking its
// contactSentence to contactEmail unless that is "". The files are part of
// the binary, so a missing one is a bug, and a panic.
func readPage(name, contactEmail string) []byte {
contents, err := pageFiles.ReadFile(name)
if err != nil {
panic(err)
}
if contactEmail == "" {
return contents
}
link := `<a href="mailto:` + html.EscapeString(contactEmail) + `">` +
contactSentence + `</a>`
return []byte(
strings.Replace(string(contents), contactSentence, link, 1),
)
}
// forStatus returns the page for a refusal with status, and whether there is
// one: the rate-limited page for a 429, the refused page for a 403.
func (p blockPages) forStatus(status int) ([]byte, bool) {
switch status {
case http.StatusForbidden:
return p.refused, true
case http.StatusTooManyRequests:
return p.rateLimited, true
default:
return nil, false
}
}
+27
View File
@@ -0,0 +1,27 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>Too many requests</title>
<style>
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
main{max-width:34rem;width:100%}
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
p{margin:0 0 1rem}
</style>
</head>
<body>
<main>
<p class="code">429</p>
<h1>Too many requests</h1>
<p>Your address has sent more requests than this site allows in a short time. Wait a while, then try again.</p>
<p>If you think this is a mistake, contact the site's operator.</p>
</main>
</body>
</html>
+27
View File
@@ -0,0 +1,27 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>Request refused</title>
<style>
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
main{max-width:34rem;width:100%}
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
p{margin:0 0 1rem}
</style>
</head>
<body>
<main>
<p class="code">403</p>
<h1>Request refused</h1>
<p>This site did not accept your request.</p>
<p>If you think this is a mistake, contact the site's operator.</p>
</main>
</body>
</html>
+3
View File
@@ -92,6 +92,7 @@ func New(params Params) *Server {
Path: wafprotocol.ReportPath, Path: wafprotocol.ReportPath,
}).String(), }).String(),
breakerClock: time.Now, breakerClock: time.Now,
pages: newBlockPages(params.Config.ContactEmail),
} }
s := &Server{h: h} s := &Server{h: h}
@@ -156,6 +157,8 @@ type handler struct {
breakerClock func() time.Time breakerClock func() time.Time
// reports are the goroutines sending reports to waf. // reports are the goroutines sending reports to waf.
reports reportGoroutines reports reportGoroutines
// pages are the pages a refused visitor gets.
pages blockPages
} }
// newTransport returns what carries requests to the app. It never goes through // newTransport returns what carries requests to the app. It never goes through
+180 -29
View File
@@ -11,6 +11,7 @@ import (
"net" "net"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
@@ -52,8 +53,56 @@ const (
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE" maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS" maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS" maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
contactEmail = "SWW_RPROXY_CONTACT_EMAIL"
) )
// The pages rproxy gives a refused visitor in place of waf's body.
const (
// refusedPageFile and rateLimitedPageFile are the pages' files, beside
// the code.
refusedPageFile = "refused.html"
rateLimitedPageFile = "ratelimited.html"
// contactSentence is the sentence of each page that becomes a link to
// SWW_RPROXY_CONTACT_EMAIL.
contactSentence = "contact the site's operator"
htmlContentType = "text/html; charset=utf-8"
)
// page is the page in file, as rproxy serves it without a contact email.
func page(t *testing.T, file string) string {
t.Helper()
//nolint:gosec // a page beside the code
contents, err := os.ReadFile(file)
if err != nil {
t.Fatalf("read the page: %v", err)
}
return string(contents)
}
// wantPage checks that got is status with the page in file as its body, and
// the headers of a page.
func wantPage(t *testing.T, got answered, status int, file string) {
t.Helper()
if got.status != status || got.body != page(t, file) {
t.Errorf(
"answered %d %q, want %d and %s",
got.status,
got.body,
status,
file,
)
}
if got.header.Get("Content-Type") != htmlContentType ||
got.header.Get("X-Content-Type-Options") != "nosniff" ||
got.header.Get("Cache-Control") != "no-store" {
t.Errorf("the page's headers are %v", got.header)
}
}
// The headers the tests set and look at. // The headers the tests set and look at.
const ( const (
dispositionHeader = wafprotocol.DispositionHeader dispositionHeader = wafprotocol.DispositionHeader
@@ -306,6 +355,20 @@ func deny(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, deniedBody) _, _ = io.WriteString(w, deniedBody)
} }
// answerAs answers a question with status and body, with disposition unless
// it is "", and a header of its own.
func answerAs(disposition string, status int, body string) http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
if disposition != "" {
w.Header().Set(dispositionHeader, disposition)
}
w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(status)
_, _ = io.WriteString(w, body)
}
}
// startApp starts app as the app rproxy passes requests to. // startApp starts app as the app rproxy passes requests to.
func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server { func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server {
t.Helper() t.Helper()
@@ -767,7 +830,7 @@ func TestRemovesTheVisitorsLookupHeadersWhenWAFGivesNone(t *testing.T) {
} }
} }
func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) { func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
@@ -777,9 +840,10 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
addr, out := startRProxy(t, standIn.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != http.StatusForbidden || got.body != deniedBody || wantPage(t, got, http.StatusForbidden, refusedPageFile)
got.header.Get(fromWAFHeader) != addedHeaderValue {
t.Errorf("answered %+v, want waf's answer", got) if got.header.Get(fromWAFHeader) != addedHeaderValue {
t.Errorf("answered %+v, want waf's headers", got)
} }
if len(got.header.Values(dispositionHeader)) != 0 { if len(got.header.Values(dispositionHeader)) != 0 {
@@ -788,7 +852,7 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
report := standIn.report(t) report := standIn.report(t)
if report.Status != http.StatusForbidden || if report.Status != http.StatusForbidden ||
report.ResponseBytes != int64(len(deniedBody)) || report.ResponseBytes != int64(len(page(t, refusedPageFile))) ||
report.DurationUpstreamTotalMS != 0 { report.DurationUpstreamTotalMS != 0 {
t.Errorf("report %+v, of a request denied", report) t.Errorf("report %+v, of a request denied", report)
} }
@@ -800,7 +864,7 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
} }
} }
func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) { func TestADeniedHeadGivesThePagesContentLength(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
@@ -808,13 +872,13 @@ func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
addr, _ := startRProxy(t, standIn.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody)) got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusForbidden || if got.status != http.StatusForbidden || got.body != "" ||
got.header.Get( got.header.Get(
contentLengthHeader, contentLengthHeader,
) != strconv.Itoa( ) != strconv.Itoa(
len(deniedBody), len(page(t, refusedPageFile)),
) { ) {
t.Errorf("answered %+v, want waf's 403 and its length", got) t.Errorf("answered %+v, want waf's 403 and the page's length", got)
} }
} }
@@ -823,11 +887,13 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n" const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n"
refusedPage := page(t, refusedPageFile)
for _, tc := range []struct { for _, tc := range []struct {
name, request, body string name, request, body string
status int status int
}{ }{
{"a deny", "GET /" + host, deniedBody, http.StatusForbidden}, {"a deny", "GET /" + host, refusedPage, http.StatusForbidden},
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest}, {"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
{"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK}, {"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK},
} { } {
@@ -899,6 +965,83 @@ func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
} }
} }
func TestADenyOf429GivesTheVisitorTheRateLimitedPage(t *testing.T) {
t.Parallel()
standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionDeny)
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(http.StatusTooManyRequests)
_, _ = io.WriteString(w, deniedBody)
})
app := startApp(t, greet)
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
wantPage(
t,
get(t, addr, "/"),
http.StatusTooManyRequests,
rateLimitedPageFile,
)
}
func TestThePagesLinkTheContactEmail(t *testing.T) {
t.Parallel()
const email = "ops@example.com"
link := `<a href="mailto:` + email + `">` + contactSentence + `</a>`
for _, tc := range []struct {
file string
status int
}{
{refusedPageFile, http.StatusForbidden},
{rateLimitedPageFile, http.StatusTooManyRequests},
} {
t.Run(tc.file, func(t *testing.T) {
t.Parallel()
standIn := startWAF(
t,
func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionDeny)
w.WriteHeader(tc.status)
},
)
app := startApp(t, greet)
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{contactEmail: email},
)
got := get(t, addr, "/")
want := strings.Replace(
page(t, tc.file),
contactSentence,
link,
1,
)
if got.status != tc.status || got.body != want {
t.Errorf(
"answered %d %q, want %d %q",
got.status,
got.body,
tc.status,
want,
)
}
if !strings.Contains(page(t, tc.file), contactSentence) {
t.Errorf("%s has no %q to link", tc.file, contactSentence)
}
})
}
}
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) { func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1017,17 +1160,28 @@ func noAnswers() map[string]func(t *testing.T) string {
func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) { func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
t.Parallel() t.Parallel()
const unknownBody = "who knows\n"
refusedPage := page(t, refusedPageFile)
rateLimitedPage := page(t, rateLimitedPageFile)
// want is the body the visitor gets: waf's, but for a refusal, which gets
// rproxy's page for its status.
for _, tc := range []struct { for _, tc := range []struct {
name, disposition, body string name, disposition, body, want string
status int status int
}{ }{
{"a refusal", dispositionDeny, deniedBody, http.StatusForbidden}, {"a refusal", dispositionDeny, deniedBody, refusedPage, http.StatusForbidden},
{"a refusal, 429", dispositionDeny, deniedBody, http.StatusTooManyRequests}, {
{"waf's 400", "", "refused\n", http.StatusBadRequest}, "a refusal, 429", dispositionDeny, deniedBody, rateLimitedPage,
{"a 500", "", "broken\n", http.StatusInternalServerError}, http.StatusTooManyRequests,
{"a 503 denied", dispositionDeny, "busy\n", http.StatusServiceUnavailable}, },
{"no header", "", "who knows\n", http.StatusOK}, {"waf's 400", "", "refused\n", "refused\n", http.StatusBadRequest},
{"an unknown disposition", "maybe", "who knows\n", http.StatusOK}, {"a 500", "", "broken\n", "broken\n", http.StatusInternalServerError},
{"a 503 denied", dispositionDeny, "busy\n", "busy\n", http.StatusServiceUnavailable},
{"a 403 without a disposition", "", "nope\n", "nope\n", http.StatusForbidden},
{"no header", "", unknownBody, unknownBody, http.StatusOK},
{"an unknown disposition", "maybe", unknownBody, unknownBody, http.StatusOK},
} { } {
for _, setting := range []string{dispositionAllow, dispositionDeny} { for _, setting := range []string{dispositionAllow, dispositionDeny} {
subtest := tc.name + " with " + noAnswerSetting + "=" + setting subtest := tc.name + " with " + noAnswerSetting + "=" + setting
@@ -1036,15 +1190,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
standIn := startWAF( standIn := startWAF(
t, t,
func(w http.ResponseWriter, _ *http.Request) { answerAs(tc.disposition, tc.status, tc.body),
if tc.disposition != "" {
w.Header().Set(dispositionHeader, tc.disposition)
}
w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(tc.status)
_, _ = io.WriteString(w, tc.body)
},
) )
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
@@ -1059,9 +1205,14 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
) )
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != tc.status || got.body != tc.body || if got.status != tc.status || got.body != tc.want ||
got.header.Get(fromWAFHeader) != addedHeaderValue { got.header.Get(fromWAFHeader) != addedHeaderValue {
t.Errorf("answered %+v, want waf's answer", got) t.Errorf(
"answered %+v, want %d %q",
got,
tc.status,
tc.want,
)
} }
if len(got.header.Values(dispositionHeader)) != 0 { if len(got.header.Values(dispositionHeader)) != 0 {
+24 -1
View File
@@ -7,6 +7,7 @@ import (
"io" "io"
"maps" "maps"
"net/http" "net/http"
"strconv"
"strings" "strings"
"sync" "sync"
@@ -187,7 +188,9 @@ func canBeAsked(r *http.Request) bool {
len(r.Header.Values("User-Agent")) <= 1 len(r.Header.Values("User-Agent")) <= 1
} }
// answerWith gives the visitor waf's answer, less its disposition. // answerWith gives the visitor waf's answer, less its disposition. A deny of
// 403 or 429 gets rproxy's page for that status as its body, in place of
// waf's.
func (rq *request) answerWith(answer *wafAnswer) { func (rq *request) answerWith(answer *wafAnswer) {
rq.prepareOwnAnswer(rq.limits.requestDeadline()) rq.prepareOwnAnswer(rq.limits.requestDeadline())
@@ -195,6 +198,15 @@ func (rq *request) answerWith(answer *wafAnswer) {
maps.Copy(header, answer.header) maps.Copy(header, answer.header)
header.Del(wafprotocol.DispositionHeader) header.Del(wafprotocol.DispositionHeader)
if rq.disposition == wafprotocol.DispositionDeny {
page, found := rq.h.pages.forStatus(answer.status)
if found {
rq.sendPage(answer.status, page)
return
}
}
// Go's server adds a Content-Type it guesses from the body to a response whose // Go's server adds a Content-Type it guesses from the body to a response whose
// headers have no Content-Type key. The key with no value sends none. // headers have no Content-Type key. The key with no value sends none.
if header.Get("Content-Type") == "" { if header.Get("Content-Type") == "" {
@@ -204,6 +216,17 @@ func (rq *request) answerWith(answer *wafAnswer) {
rq.send(answer.status, answer.body) rq.send(answer.status, answer.body)
} }
// sendPage sends page as the body of a refusal with status. The length is set
// here too, since send leaves a HEAD's as waf gave it, which is of waf's body.
func (rq *request) sendPage(status int, page []byte) {
header := rq.out.Header()
header.Set("Content-Type", htmlContentType)
header.Set("X-Content-Type-Options", "nosniff")
header.Set("Cache-Control", "no-store")
header.Set("Content-Length", strconv.Itoa(len(page)))
rq.send(status, page)
}
// sendReport sends waf report on a goroutine of its own, so that the // sendReport sends waf report on a goroutine of its own, so that the
// visitor does not wait for it. The report goes on after ctx, the request's own // visitor does not wait for it. The report goes on after ctx, the request's own
// context, has ended. // context, has ended.
-16
View File
@@ -32,22 +32,6 @@ const (
bytesPerHour = "BYTES_PER_HOUR" bytesPerHour = "BYTES_PER_HOUR"
) )
// netAndASNThresholdsOff returns the anomaly thresholds per netblock and
// per AS number, the ones on by default, each set to off.
func netAndASNThresholdsOff() map[string]string {
settings := map[string]string{}
for _, scope := range []string{anomalyNet, anomalyASN} {
for _, end := range []string{
requestsPerMinute, requestsPerHour, bytesPerMinute, bytesPerHour,
} {
settings[scope+end] = off
}
}
return settings
}
// The other anomaly settings. // The other anomaly settings.
const ( const (
anomalyNetV4Prefix = "SWW_WAF_ANOMALY_NET_V4_PREFIX" anomalyNetV4Prefix = "SWW_WAF_ANOMALY_NET_V4_PREFIX"
+1 -4
View File
@@ -456,9 +456,7 @@ func (w *wafAndParts) banAbuseIPDBBlacklist(
// at abuseIPDBURL while env sets SWW_WAF_ABUSEIPDB_KEY, and its blacklist is // at abuseIPDBURL while env sets SWW_WAF_ABUSEIPDB_KEY, and its blacklist is
// fetched from abuseIPDBBlacklistURL. // fetched from abuseIPDBBlacklistURL.
// SWW_WAF_CORE_RULE_SET_MODE is off unless env sets it, so that only the tests of the // SWW_WAF_CORE_RULE_SET_MODE is off unless env sets it, so that only the tests of the
// Core Rule Set have their requests inspected by it. Likewise the anomaly thresholds // Core Rule Set have their requests inspected by it.
// per netblock and per AS number are off unless env sets them, so that only the tests
// of the anomaly thresholds count requests.
func newWAF( func newWAF(
t *testing.T, t *testing.T,
geojsURL string, geojsURL string,
@@ -488,7 +486,6 @@ func newWAFWith(
settings[lookupSource] = off settings[lookupSource] = off
} }
maps.Copy(settings, netAndASNThresholdsOff())
maps.Copy(settings, env) maps.Copy(settings, env)
cfg, err := config.WAFFromEnvironment(configtest.Viper(settings)) cfg, err := config.WAFFromEnvironment(configtest.Viper(settings))
+1 -1
View File
@@ -405,7 +405,7 @@ func (rq *request) addToHistory() {
// SWW_WAF_RATE_LIMIT_EXEMPT_NETS, and one for a path in // SWW_WAF_RATE_LIMIT_EXEMPT_NETS, and one for a path in
// SWW_WAF_RATE_LIMIT_EXEMPT_PATHS are counted too. It is counted for its // SWW_WAF_RATE_LIMIT_EXEMPT_PATHS are counted too. It is counted for its
// client's AS number when answerAtTheEnd gives one. With every anomaly // client's AS number when answerAtTheEnd gives one. With every anomaly
// threshold off, it does nothing. // threshold off, the default, it does nothing.
func (rq *request) countAnomalies() { func (rq *request) countAnomalies() {
if !anomalyThresholdsSet(rq.h.config) { if !anomalyThresholdsSet(rq.h.config) {
return return