Compare commits

..
Author SHA1 Message Date
sneak 7b8d2b9d38 docs: name each AS number anomaly default
check / check (push) Failing after 15m25s
The AS number values were listed bytes per minute before bytes per
hour, the reverse of the netblock list above them, so read in order
they gave BYTES_PER_HOUR 25G and BYTES_PER_MINUTE 250G.

Model: opus-5-5
2026-11-20 05:13:34 +00:00
sneak 4c20caf593 waf: countAnomalies no longer calls every threshold off the default
The thresholds per netblock and per AS number are now on by default, so
the comment's "the default" was false.

Model: opus-5-5
2026-11-20 05:13:15 +00:00
sneak b8abb87738 Default the netblock and AS number anomaly thresholds (closes #410)
check / check (push) Failing after 1m37s
The alert-only thresholds per netblock and per AS number now have
defaults that alert on obvious abuse alone: per netblock 1200 and
15000 requests a minute and an hour, 5G and 50G a minute and an hour;
per AS number five times those. The AS number ones default to off
while SWW_WAF_LOOKUP_SOURCE is off. The other scopes stay off. The
README says to tune them to the site's traffic.

Model: opus-5-5
2026-11-20 05:05:29 +00:00
17 changed files with 196 additions and 430 deletions

No files matched your search

+11 -14
View File
@@ -288,7 +288,9 @@ account, a key, a token or an alert destination stays off until you give one.
- An anomaly is more requests or bytes than an `SWW_WAF_ANOMALY_` threshold - An anomaly is more requests or bytes than an `SWW_WAF_ANOMALY_` threshold
allows. Crossing a threshold sends one `anomaly` alert to the alert allows. Crossing a threshold sends one `anomaly` alert to the alert
destinations, and does nothing else: no client is banned, rate limited or destinations, and does nothing else: no client is banned, rate limited or
blocked. blocked. The thresholds per netblock and per AS number are on by default, set
high enough to alert only on obvious abuse. They are a starting point: tune
them to your own site's traffic, which the metrics show.
- An invalid setting stops the process at start with one log line naming the - An invalid setting stops the process at start with one log line naming the
variable and what is wrong with the value. The settings in effect are logged variable and what is wrong with the value. The settings in effect are logged
on one line at start, with tokens, keys and webhook paths shown as `********`. on one line at start, with tokens, keys and webhook paths shown as `********`.
@@ -308,7 +310,6 @@ Every setting the code reads, with its default:
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. | | `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. | | `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. | | `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
| `SWW_RPROXY_CONTACT_EMAIL` | unset | The email address the refusal pages link to, a bare one such as `ops@example.com`. |
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. | | `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. | | `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. |
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. | | `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
@@ -397,16 +398,16 @@ Every setting the code reads, with its default:
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when one client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when one client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `1200` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `15000` | Sends one alert to the alert destinations when the netblock around a client makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_NET_BYTES_PER_MINUTE` | `5G` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_NET_BYTES_PER_HOUR` | `50G` | Sends one alert to the alert destinations when more bytes than this are counted for the netblock around a client in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 32. Those thresholds only send alerts. | | `SWW_WAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 32. Those thresholds only send alerts. |
| `SWW_WAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 128. Those thresholds only send alerts. | | `SWW_WAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 128. Those thresholds only send alerts. |
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `6000`, or `off` while the lookup source is `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `75000`, or `off` while the lookup source is `off` | Sends one alert to the alert destinations when one AS number makes more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `25G`, or `off` while the lookup source is `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_ASN_BYTES_PER_HOUR` | `250G`, or `off` while the lookup source is `off` | Sends one alert to the alert destinations when more bytes than this are counted for one AS number in an hour, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in a minute, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in a minute, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in an hour, a count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Sends one alert to the alert destinations when the whole service gets more requests than this in an hour, a count. No client is banned, rate limited or blocked. |
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the whole service in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. | | `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Sends one alert to the alert destinations when more bytes than this are counted for the whole service in a minute, a size. `SWW_WAF_BYTES_COUNTED` says which bytes count. No client is banned, rate limited or blocked. |
@@ -766,10 +767,6 @@ request that steps 12 to 14 have not finished within 500 ms, which is answered
`503`. `rproxy` itself answers `405` to a method it does not pass on, and the `503`. `rproxy` itself answers `405` to a method it does not pass on, and the
status for a size or time limit, such as `413` for a body too large. status for a size or time limit, such as `413` for a body too large.
A refusal of `403` or `429` reaches the visitor as a short HTML page of
`rproxy`'s own, a refusal page. It links to `SWW_RPROXY_CONTACT_EMAIL` when that
is set.
### Clients and bans ### Clients and bans
- A client is one IPv4 address or one IPv6 netblock of - A client is one IPv4 address or one IPv6 netblock of
+10 -9
View File
@@ -2113,12 +2113,14 @@ host, sharing one address, would often fetch too close together.
### Anomaly thresholds ### Anomaly thresholds
These only alert; nothing is blocked. None is set by default, and a threshold These only alert; nothing is blocked. The thresholds per netblock and per AS
left unset sends no alert. An alert needs a destination only the operator can number have defaults high enough to alert only on obvious abuse. The others are
supply, and what counts as unusual depends on each service's normal traffic, off by default. What counts as unusual depends on each service's normal traffic,
which the metrics show. They reveal a widely distributed scraper, thousands of which the metrics show, so the defaults are a starting point for the operator to
addresses at low rates each, which the per-client limits do not see. A low tune. An alert goes nowhere until the operator sets a destination. The
percentage for its `asn=` tag in `SWW_WAF_TAG_LIMIT_PERCENT` is the response. thresholds reveal a widely distributed scraper, thousands of addresses at low
rates each, which the per-client limits do not see. A low percentage for its
`asn=` tag in `SWW_WAF_TAG_LIMIT_PERCENT` is the response.
- Requests and body bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted - Requests and body bytes, as `SWW_WAF_BYTES_COUNTED` counts them, are counted
per minute and per hour, for each of: per minute and per hour, for each of:
@@ -2431,9 +2433,8 @@ but the last below carries `X-Sww-Disposition`.
carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
request it passes on. A header whose value is unknown is left out. request it passes on. A header whose value is unknown is left out.
- `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to - `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
get. `rproxy` gives them to the visitor, less this header; a `403` or `429` get. `rproxy` gives them to the visitor, less this header. The status is one
gets `rproxy`'s own page as its body, linking `SWW_RPROXY_CONTACT_EMAIL` when of:
set. The status is one of:
- `SWW_WAF_BAN_RESPONSE`, for a refusal; - `SWW_WAF_BAN_RESPONSE`, for a refusal;
- `403`, for a `block` rule or the Core Rule Set; - `403`, for a `block` rule or the Core Rule Set;
- `414` or `431`, for a request over the header limits; - `414` or `431`, for a request over the header limits;
-3
View File
@@ -10,9 +10,6 @@ services:
SWW_RPROXY_WAF_BACKEND: http://waf:8080 SWW_RPROXY_WAF_BACKEND: http://waf:8080
SWW_RPROXY_UPSTREAM_URL: http://exampleapp:8081 SWW_RPROXY_UPSTREAM_URL: http://exampleapp:8081
# The address the refusal pages link to.
# SWW_RPROXY_CONTACT_EMAIL: ops@example.com
# The netblocks whose X-Forwarded-For rproxy believes. By default # The netblocks whose X-Forwarded-For rproxy believes. By default
# the private ranges, which cover a TLS-terminating proxy on the # the private ranges, which cover a TLS-terminating proxy on the
# Docker network. A proxy elsewhere needs its address added; a # Docker network. A proxy elsewhere needs its address added; a
+21 -4
View File
@@ -378,8 +378,9 @@ Crossing a threshold sends one `anomaly` alert to the destinations under
Each scope has four thresholds: the most requests and the most body bytes Each scope has four thresholds: the most requests and the most body bytes
counted in a minute and in an hour. The bytes are those `SWW_WAF_BYTES_COUNTED` counted in a minute and in an hour. The bytes are those `SWW_WAF_BYTES_COUNTED`
names. Requests are counts, bytes are sizes. Each is `off` by default. The four names. Requests are counts, bytes are sizes. Each is `off` by default, except
thresholds per client are: those per netblock and per AS number listed below. The four thresholds per
client are:
- `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`; - `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
- `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`; - `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
@@ -403,6 +404,24 @@ as `REQUESTS_PER_MINUTE`:
`office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
name, or with a bad netblock, is refused. So is a name used twice. name, or with a bad netblock, is refused. So is a name used twice.
The thresholds per netblock and per AS number have defaults set high enough to
alert only on obvious abuse:
- per netblock:
- `REQUESTS_PER_MINUTE` `1200` and `REQUESTS_PER_HOUR` `15000`, ten clients
at the default rate limits;
- `BYTES_PER_HOUR` `50G`, one client at the default byte limit per hour;
- `BYTES_PER_MINUTE` `5G`, a tenth of that.
- per AS number, five times those: `REQUESTS_PER_MINUTE` `6000`,
`REQUESTS_PER_HOUR` `75000`, `BYTES_PER_HOUR` `250G` and `BYTES_PER_MINUTE`
`25G`.
While `SWW_WAF_LOOKUP_SOURCE` is `off`, no request has an AS number, so the
thresholds per AS number are `off` by default. Setting one then stops the start.
These defaults are a starting point. Tune them to your own site's traffic, which
the metrics show.
## Settings given as files ## Settings given as files
To give a token as a file, as "Tokens" under "Deployment" in To give a token as a file, as "Tokens" under "Deployment" in
@@ -483,8 +502,6 @@ start, and an invalid value stops the start. Any can be given as a file, as a
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
do the status and headers of the app's answer. A time in milliseconds or do the status and headers of the app's answer. A time in milliseconds or
`off`. `off`.
- `SWW_RPROXY_CONTACT_EMAIL` (default unset): the email address the refusal
pages link to. A bare address, such as `ops@example.com`.
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as - `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
`:9000`. `:9000`.
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An - `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
-24
View File
@@ -4,7 +4,6 @@ import (
"errors" "errors"
"fmt" "fmt"
"log/slog" "log/slog"
"net/mail"
"net/netip" "net/netip"
"net/url" "net/url"
"strconv" "strconv"
@@ -47,10 +46,6 @@ type RProxy struct {
// (SWW_RPROXY_MAX_RESPONSE_TIME_MS). // (SWW_RPROXY_MAX_RESPONSE_TIME_MS).
MaxRequestTime time.Duration MaxRequestTime time.Duration
MaxResponseTime time.Duration MaxResponseTime time.Duration
// ContactEmail is the address the pages a refused visitor gets link to,
// or "" for no link (SWW_RPROXY_CONTACT_EMAIL). It is a bare address,
// such as ops@example.com.
ContactEmail string
// settings are the values read, as given or by default, for the log line // settings are the values read, as given or by default, for the log line
// at start. // at start.
@@ -74,8 +69,6 @@ var (
errNotMillisecondsOrOff = errors.New( errNotMillisecondsOrOff = errors.New(
"is not a whole number of milliseconds above zero, such as 60000, or off", "is not a whole number of milliseconds above zero, such as 60000, or off",
) )
errNotEmail = errors.New(
"is not a bare email address, such as ops@example.com")
) )
// RProxyFromEnvironment reads rproxy's settings through v, normally the viper // RProxyFromEnvironment reads rproxy's settings through v, normally the viper
@@ -115,7 +108,6 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
"SWW_RPROXY_MAX_RESPONSE_TIME_MS", "SWW_RPROXY_MAX_RESPONSE_TIME_MS",
"1800000", "1800000",
), ),
ContactEmail: env.email("SWW_RPROXY_CONTACT_EMAIL"),
} }
if env.err != nil { if env.err != nil {
@@ -188,22 +180,6 @@ func (e *environment) disposition(
return value return value
} }
// email reads a setting that is a bare email address, such as
// ops@example.com, or "" when it is not set.
func (e *environment) email(name string) string {
value := e.value(name, "")
if value == "" {
return ""
}
address, err := mail.ParseAddress(value)
if err != nil || address.Address != value {
e.check(name, fmt.Errorf("%q %w", value, errNotEmail))
}
return value
}
// millisecondsOrOff reads a setting that is a whole number of milliseconds // millisecondsOrOff reads a setting that is a whole number of milliseconds
// above zero, or off, which is 0. // above zero, or off, which is 0.
func (e *environment) millisecondsOrOff( func (e *environment) millisecondsOrOff(
+10 -25
View File
@@ -25,8 +25,6 @@ const (
rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE" rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE"
rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS" rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS" rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
rproxyContactEmail = "SWW_RPROXY_CONTACT_EMAIL"
contactAddress = "ops@example.com"
backendURL = "http://waf:8080" backendURL = "http://waf:8080"
rproxyNoAnswerAddr = ":8080" rproxyNoAnswerAddr = ":8080"
rproxyNoAnswerApp = "http://127.0.0.1:8081" rproxyNoAnswerApp = "http://127.0.0.1:8081"
@@ -89,30 +87,21 @@ func TestRProxySettingsAreRead(t *testing.T) {
t.Parallel() t.Parallel()
cfg, err := rproxyConfig(map[string]string{ cfg, err := rproxyConfig(map[string]string{
wafBackend: "https://waf.example:8443", wafBackend: "https://waf.example:8443",
rproxyListen: ":9000", rproxyListen: ":9000",
rproxyUpstream: "http://app:3000", rproxyUpstream: "http://app:3000",
rproxyTrusted: "192.0.2.0/24", rproxyTrusted: "192.0.2.0/24",
wafBackendTimeout: "150", wafBackendTimeout: "150",
rproxyNoAnswer: "deny", rproxyNoAnswer: "deny",
rproxyMaxRequest: "1K", rproxyMaxRequest: "1K",
rproxyMaxResponse: off, rproxyMaxResponse: off,
rproxyRequestTime: off, rproxyRequestTime: off,
rproxyAnswerTime: "250", rproxyAnswerTime: "250",
rproxyContactEmail: contactAddress,
}) })
if err != nil { if err != nil {
t.Fatalf("settings: %v", err) t.Fatalf("settings: %v", err)
} }
if cfg.ContactEmail != contactAddress {
t.Errorf(
"contact email %q, want %q",
cfg.ContactEmail,
contactAddress,
)
}
if cfg.ListenAddr != ":9000" || cfg.UpstreamURL.Host != "app:3000" || if cfg.ListenAddr != ":9000" || cfg.UpstreamURL.Host != "app:3000" ||
cfg.WAFBackend.Host != "waf.example:8443" || cfg.WAFBackend.Host != "waf.example:8443" ||
cfg.TrustedProxies[0] != netip.MustParsePrefix("192.0.2.0/24") { cfg.TrustedProxies[0] != netip.MustParsePrefix("192.0.2.0/24") {
@@ -165,9 +154,6 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
{rproxyRequestTime, "0"}, {rproxyRequestTime, "0"},
{rproxyRequestTime, "1m"}, {rproxyRequestTime, "1m"},
{rproxyAnswerTime, "0"}, {rproxyAnswerTime, "0"},
{rproxyContactEmail, "ops"},
{rproxyContactEmail, "Ops <" + contactAddress + ">"},
{rproxyContactEmail, contactAddress + " "},
} { } {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -264,7 +250,6 @@ func TestRProxySettingsAreLoggedByName(t *testing.T) {
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout, rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout,
rproxyNoAnswer, rproxyNoAnswer,
rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime, rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime,
rproxyContactEmail,
} { } {
if !slices.Contains(names, name) { if !slices.Contains(names, name) {
t.Errorf("%s is not logged, only %v", name, names) t.Errorf("%s is not logged, only %v", name, names)
+51 -13
View File
@@ -313,7 +313,9 @@ type WAF struct {
// per netblock around a client (SWW_WAF_ANOMALY_NET_*), per AS number // per netblock around a client (SWW_WAF_ANOMALY_NET_*), per AS number
// (SWW_WAF_ANOMALY_ASN_*), for the whole service (SWW_WAF_ANOMALY_TOTAL_*) // (SWW_WAF_ANOMALY_ASN_*), for the whole service (SWW_WAF_ANOMALY_TOTAL_*)
// and per watched netblock (SWW_WAF_ANOMALY_WATCHED_NET_*), each 0 while // and per watched netblock (SWW_WAF_ANOMALY_WATCHED_NET_*), each 0 while
// it is off. AnomalyNetV4Prefix and AnomalyNetV6Prefix are the lengths // it is off. Only the thresholds per netblock and per AS number are on by
// default, those per AS number only while clients are looked up.
// AnomalyNetV4Prefix and AnomalyNetV6Prefix are the lengths
// of the netblock around a client (SWW_WAF_ANOMALY_NET_V4_PREFIX and // of the netblock around a client (SWW_WAF_ANOMALY_NET_V4_PREFIX and
// SWW_WAF_ANOMALY_NET_V6_PREFIX), and AnomalyWatchedNets the watched // SWW_WAF_ANOMALY_NET_V6_PREFIX), and AnomalyWatchedNets the watched
// netblocks (SWW_WAF_ANOMALY_WATCHED_NETS). // netblocks (SWW_WAF_ANOMALY_WATCHED_NETS).
@@ -555,11 +557,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
"SWW_WAF_ALERT_MAX_PER_HOUR", "SWW_WAF_ALERT_MAX_PER_HOUR",
"60", "60",
), ),
AnomalyClient: env.thresholds("SWW_WAF_ANOMALY_CLIENT_"),
AnomalyNet: env.thresholds("SWW_WAF_ANOMALY_NET_"),
AnomalyASN: env.thresholds("SWW_WAF_ANOMALY_ASN_"),
AnomalyTotal: env.thresholds("SWW_WAF_ANOMALY_TOTAL_"),
AnomalyWatchedNet: env.thresholds("SWW_WAF_ANOMALY_WATCHED_NET_"),
AnomalyNetV4Prefix: env.v4Prefix( AnomalyNetV4Prefix: env.v4Prefix(
"SWW_WAF_ANOMALY_NET_V4_PREFIX", "SWW_WAF_ANOMALY_NET_V4_PREFIX",
"24", "24",
@@ -578,6 +575,7 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
// The app gets the headers by default, unless no client is looked up. // The app gets the headers by default, unless no client is looked up.
cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS", cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS",
strconv.FormatBool(cfg.LookupSource != LookupSourceOff)) strconv.FormatBool(cfg.LookupSource != LookupSourceOff))
env.anomalyThresholds(cfg)
cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action( cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action(
BlocklistActionSetting, string(ReputationActionDeny)) BlocklistActionSetting, string(ReputationActionDeny))
cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action( cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action(
@@ -2693,15 +2691,55 @@ func (e *environment) duration(name, defaultValue string) time.Duration {
return duration return duration
} }
// thresholdDefaults are the defaults of a scope's four anomaly thresholds,
// each a count, a size or off.
type thresholdDefaults struct {
requestsPerMinute, requestsPerHour, bytesPerMinute, bytesPerHour string
}
// anomalyThresholds reads the anomaly thresholds of every scope into cfg.
// Only those per netblock and per AS number are on by default, high
// enough to alert on obvious abuse alone. Those per netblock count the
// requests of ten clients at the default per-client rate limits, and the
// bytes of one client at its default byte limit per hour, a tenth of
// that in a minute. Those per AS number are five times those per
// netblock. They are off by default while no client is looked up, since
// no request then has an AS number.
func (e *environment) anomalyThresholds(cfg *WAF) {
allOff := thresholdDefaults{off, off, off, off}
asnDefaults := thresholdDefaults{
requestsPerMinute: "6000", requestsPerHour: "75000",
bytesPerMinute: "25G", bytesPerHour: "250G",
}
if cfg.LookupSource == LookupSourceOff {
asnDefaults = allOff
}
cfg.AnomalyClient = e.thresholds("SWW_WAF_ANOMALY_CLIENT_", allOff)
cfg.AnomalyNet = e.thresholds("SWW_WAF_ANOMALY_NET_", thresholdDefaults{
requestsPerMinute: "1200", requestsPerHour: "15000",
bytesPerMinute: "5G", bytesPerHour: "50G",
})
cfg.AnomalyASN = e.thresholds("SWW_WAF_ANOMALY_ASN_", asnDefaults)
cfg.AnomalyTotal = e.thresholds("SWW_WAF_ANOMALY_TOTAL_", allOff)
cfg.AnomalyWatchedNet = e.thresholds("SWW_WAF_ANOMALY_WATCHED_NET_",
allOff)
}
// thresholds reads the four anomaly thresholds whose settings' names // thresholds reads the four anomaly thresholds whose settings' names
// start with prefix: requests and bytes per minute and per hour. Each is // start with prefix: requests and bytes per minute and per hour.
// off by default. func (e *environment) thresholds(
func (e *environment) thresholds(prefix string) anomaly.Thresholds { prefix string, defaults thresholdDefaults,
) anomaly.Thresholds {
return anomaly.Thresholds{ return anomaly.Thresholds{
RequestsPerMinute: e.count(prefix+"REQUESTS_PER_MINUTE", off), RequestsPerMinute: e.count(prefix+"REQUESTS_PER_MINUTE",
RequestsPerHour: e.count(prefix+"REQUESTS_PER_HOUR", off), defaults.requestsPerMinute),
BytesPerMinute: e.size(prefix+"BYTES_PER_MINUTE", off), RequestsPerHour: e.count(prefix+"REQUESTS_PER_HOUR",
BytesPerHour: e.size(prefix+"BYTES_PER_HOUR", off), defaults.requestsPerHour),
BytesPerMinute: e.size(prefix+"BYTES_PER_MINUTE",
defaults.bytesPerMinute),
BytesPerHour: e.size(prefix+"BYTES_PER_HOUR", defaults.bytesPerHour),
} }
} }
+40 -1
View File
@@ -178,6 +178,17 @@ func loggedAnomalyDefaults() map[string]string {
logged[name] = off logged[name] = off
} }
maps.Copy(logged, map[string]string{
anomalyNet + requestsPerMinute: "1200",
anomalyNet + requestsPerHour: "15000",
anomalyNet + bytesPerMinute: "5G",
anomalyNet + bytesPerHour: "50G",
anomalyASN + requestsPerMinute: "6000",
anomalyASN + requestsPerHour: "75000",
anomalyASN + bytesPerMinute: "25G",
anomalyASN + bytesPerHour: "250G",
})
return logged return logged
} }
@@ -1565,7 +1576,35 @@ func TestAnomalySettingsDefaults(t *testing.T) {
t.Parallel() t.Parallel()
cfg := wafFromEnvironment(t, environment{}) cfg := wafFromEnvironment(t, environment{})
wantAllOff(t, cfg)
const gibibyte = int64(1) << 30
want := map[string]anomaly.Thresholds{
anomalyClient: {},
anomalyNet: {
RequestsPerMinute: 1200, RequestsPerHour: 15000,
BytesPerMinute: 5 * gibibyte, BytesPerHour: 50 * gibibyte,
},
anomalyASN: {
RequestsPerMinute: 6000, RequestsPerHour: 75000,
BytesPerMinute: 25 * gibibyte, BytesPerHour: 250 * gibibyte,
},
anomalyTotal: {}, anomalyWatchedNet: {},
}
if got := thresholdsByScope(cfg); !maps.Equal(got, want) {
t.Errorf("thresholds by scope\n%+v\nwant\n%+v", got, want)
}
// With no client looked up, the thresholds per AS number are off.
want[anomalyASN] = anomaly.Thresholds{}
got := thresholdsByScope(
wafFromEnvironment(t, environment{lookupSource: off}))
if !maps.Equal(got, want) {
t.Errorf("with %s off, thresholds by scope\n%+v\nwant\n%+v",
lookupSource, got, want)
}
if cfg.AnomalyNetV4Prefix != 24 || cfg.AnomalyNetV6Prefix != 48 || if cfg.AnomalyNetV4Prefix != 24 || cfg.AnomalyNetV6Prefix != 48 ||
len(cfg.AnomalyWatchedNets) != 0 { len(cfg.AnomalyWatchedNets) != 0 {
-27
View File
@@ -1,27 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>Too many requests</title>
<style>
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
main{max-width:34rem;width:100%}
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
p{margin:0 0 1rem}
</style>
</head>
<body>
<main>
<p class="code">429</p>
<h1>Too many requests</h1>
<p>Your address has sent more requests than this site allows in a short time. Wait a while, then try again.</p>
<p>If you think this is a mistake, contact the site's operator.</p>
</main>
</body>
</html>
-71
View File
@@ -1,71 +0,0 @@
package rproxy
import (
"embed"
"html"
"net/http"
"strings"
)
// pageFiles are the pages a refused visitor gets, as HTML with nothing
// external: refused.html for a 403, ratelimited.html for a 429.
//
//go:embed refused.html ratelimited.html
var pageFiles embed.FS
// contactSentence is the sentence of each page that becomes a link to
// SWW_RPROXY_CONTACT_EMAIL when that is set.
const contactSentence = "contact the site's operator"
// htmlContentType is the Content-Type of the pages.
const htmlContentType = "text/html; charset=utf-8"
// refusalPages are the pages a refused visitor gets, ready to send, made once
// at start.
type refusalPages struct {
refused []byte
rateLimited []byte
}
// newRefusalPages makes the pages. With contactEmail set, each page's
// contactSentence is a mailto link to it.
func newRefusalPages(contactEmail string) refusalPages {
return refusalPages{
refused: readPage("refused.html", contactEmail),
rateLimited: readPage("ratelimited.html", contactEmail),
}
}
// readPage reads the page in name from pageFiles, linking its
// contactSentence to contactEmail unless that is "". The files are part of
// the binary, so a missing one is a bug, and a panic.
func readPage(name, contactEmail string) []byte {
contents, err := pageFiles.ReadFile(name)
if err != nil {
panic(err)
}
if contactEmail == "" {
return contents
}
link := `<a href="mailto:` + html.EscapeString(contactEmail) + `">` +
contactSentence + `</a>`
return []byte(
strings.Replace(string(contents), contactSentence, link, 1),
)
}
// forStatus returns the page for a refusal with status, and whether there is
// one: the rate-limited page for a 429, the refused page for a 403.
func (p refusalPages) forStatus(status int) ([]byte, bool) {
switch status {
case http.StatusForbidden:
return p.refused, true
case http.StatusTooManyRequests:
return p.rateLimited, true
default:
return nil, false
}
}
-27
View File
@@ -1,27 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>Request refused</title>
<style>
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
main{max-width:34rem;width:100%}
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
p{margin:0 0 1rem}
</style>
</head>
<body>
<main>
<p class="code">403</p>
<h1>Request refused</h1>
<p>This site did not accept your request.</p>
<p>If you think this is a mistake, contact the site's operator.</p>
</main>
</body>
</html>
+1 -5
View File
@@ -1,8 +1,7 @@
// Package rproxy is the reverse proxy in front of one app that asks waf // Package rproxy is the reverse proxy in front of one app that asks waf
// about each request. It passes a request waf allows to the app and the // about each request. It passes a request waf allows to the app and the
// app's answer back, within the size and time limits. Any other answer from // app's answer back, within the size and time limits. Any other answer from
// waf goes to the visitor as waf gave it, except that a refusal of 403 or 429 // waf goes to the visitor as waf gave it.
// has a page of rproxy's own as its body.
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION decides only when waf cannot // SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION decides only when waf cannot
// be connected to, does not answer at all, or does not answer within // be connected to, does not answer at all, or does not answer within
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, or while the circuit breaker for asks to // SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, or while the circuit breaker for asks to
@@ -93,7 +92,6 @@ func New(params Params) *Server {
Path: wafprotocol.ReportPath, Path: wafprotocol.ReportPath,
}).String(), }).String(),
breakerClock: time.Now, breakerClock: time.Now,
pages: newRefusalPages(params.Config.ContactEmail),
} }
s := &Server{h: h} s := &Server{h: h}
@@ -158,8 +156,6 @@ type handler struct {
breakerClock func() time.Time breakerClock func() time.Time
// reports are the goroutines sending reports to waf. // reports are the goroutines sending reports to waf.
reports reportGoroutines reports reportGoroutines
// pages are the pages a refused visitor gets.
pages refusalPages
} }
// newTransport returns what carries requests to the app. It never goes through // newTransport returns what carries requests to the app. It never goes through
+30 -181
View File
@@ -11,7 +11,6 @@ import (
"net" "net"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
@@ -53,56 +52,8 @@ const (
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE" maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS" maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS" maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
contactEmail = "SWW_RPROXY_CONTACT_EMAIL"
) )
// The pages rproxy gives a refused visitor in place of waf's body.
const (
// refusedPageFile and rateLimitedPageFile are the pages' files, beside
// the code.
refusedPageFile = "refused.html"
rateLimitedPageFile = "ratelimited.html"
// contactSentence is the sentence of each page that becomes a link to
// SWW_RPROXY_CONTACT_EMAIL.
contactSentence = "contact the site's operator"
htmlContentType = "text/html; charset=utf-8"
)
// page is the page in file, as rproxy serves it without a contact email.
func page(t *testing.T, file string) string {
t.Helper()
//nolint:gosec // a page beside the code
contents, err := os.ReadFile(file)
if err != nil {
t.Fatalf("read the page: %v", err)
}
return string(contents)
}
// wantPage checks that got is status with the page in file as its body, and
// the headers of a page.
func wantPage(t *testing.T, got answered, status int, file string) {
t.Helper()
if got.status != status || got.body != page(t, file) {
t.Errorf(
"answered %d %q, want %d and %s",
got.status,
got.body,
status,
file,
)
}
if got.header.Get("Content-Type") != htmlContentType ||
got.header.Get("X-Content-Type-Options") != "nosniff" ||
got.header.Get("Cache-Control") != "no-store" {
t.Errorf("the page's headers are %v", got.header)
}
}
// The headers the tests set and look at. // The headers the tests set and look at.
const ( const (
dispositionHeader = wafprotocol.DispositionHeader dispositionHeader = wafprotocol.DispositionHeader
@@ -355,20 +306,6 @@ func deny(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, deniedBody) _, _ = io.WriteString(w, deniedBody)
} }
// answerAs answers a question with status and body, with disposition unless
// it is "", and a header of its own.
func answerAs(disposition string, status int, body string) http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
if disposition != "" {
w.Header().Set(dispositionHeader, disposition)
}
w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(status)
_, _ = io.WriteString(w, body)
}
}
// startApp starts app as the app rproxy passes requests to. // startApp starts app as the app rproxy passes requests to.
func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server { func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server {
t.Helper() t.Helper()
@@ -830,7 +767,7 @@ func TestRemovesTheVisitorsLookupHeadersWhenWAFGivesNone(t *testing.T) {
} }
} }
func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) { func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
@@ -840,10 +777,9 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
addr, out := startRProxy(t, standIn.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
got := get(t, addr, "/") got := get(t, addr, "/")
wantPage(t, got, http.StatusForbidden, refusedPageFile) if got.status != http.StatusForbidden || got.body != deniedBody ||
got.header.Get(fromWAFHeader) != addedHeaderValue {
if got.header.Get(fromWAFHeader) != addedHeaderValue { t.Errorf("answered %+v, want waf's answer", got)
t.Errorf("answered %+v, want waf's headers", got)
} }
if len(got.header.Values(dispositionHeader)) != 0 { if len(got.header.Values(dispositionHeader)) != 0 {
@@ -852,7 +788,7 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
report := standIn.report(t) report := standIn.report(t)
if report.Status != http.StatusForbidden || if report.Status != http.StatusForbidden ||
report.ResponseBytes != int64(len(page(t, refusedPageFile))) || report.ResponseBytes != int64(len(deniedBody)) ||
report.DurationUpstreamTotalMS != 0 { report.DurationUpstreamTotalMS != 0 {
t.Errorf("report %+v, of a request denied", report) t.Errorf("report %+v, of a request denied", report)
} }
@@ -864,7 +800,7 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
} }
} }
func TestADeniedHeadGivesThePagesContentLength(t *testing.T) { func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
@@ -872,13 +808,13 @@ func TestADeniedHeadGivesThePagesContentLength(t *testing.T) {
addr, _ := startRProxy(t, standIn.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody)) got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusForbidden || got.body != "" || if got.status != http.StatusForbidden ||
got.header.Get( got.header.Get(
contentLengthHeader, contentLengthHeader,
) != strconv.Itoa( ) != strconv.Itoa(
len(page(t, refusedPageFile)), len(deniedBody),
) { ) {
t.Errorf("answered %+v, want waf's 403 and the page's length", got) t.Errorf("answered %+v, want waf's 403 and its length", got)
} }
} }
@@ -887,13 +823,11 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n" const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n"
refusedPage := page(t, refusedPageFile)
for _, tc := range []struct { for _, tc := range []struct {
name, request, body string name, request, body string
status int status int
}{ }{
{"a deny", "GET /" + host, refusedPage, http.StatusForbidden}, {"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest}, {"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
{"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK}, {"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK},
} { } {
@@ -965,83 +899,6 @@ func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
} }
} }
func TestADenyOf429GivesTheVisitorTheRateLimitedPage(t *testing.T) {
t.Parallel()
standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionDeny)
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(http.StatusTooManyRequests)
_, _ = io.WriteString(w, deniedBody)
})
app := startApp(t, greet)
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
wantPage(
t,
get(t, addr, "/"),
http.StatusTooManyRequests,
rateLimitedPageFile,
)
}
func TestThePagesLinkTheContactEmail(t *testing.T) {
t.Parallel()
const email = "ops@example.com"
link := `<a href="mailto:` + email + `">` + contactSentence + `</a>`
for _, tc := range []struct {
file string
status int
}{
{refusedPageFile, http.StatusForbidden},
{rateLimitedPageFile, http.StatusTooManyRequests},
} {
t.Run(tc.file, func(t *testing.T) {
t.Parallel()
standIn := startWAF(
t,
func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionDeny)
w.WriteHeader(tc.status)
},
)
app := startApp(t, greet)
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{contactEmail: email},
)
got := get(t, addr, "/")
want := strings.Replace(
page(t, tc.file),
contactSentence,
link,
1,
)
if got.status != tc.status || got.body != want {
t.Errorf(
"answered %d %q, want %d %q",
got.status,
got.body,
tc.status,
want,
)
}
if !strings.Contains(page(t, tc.file), contactSentence) {
t.Errorf("%s has no %q to link", tc.file, contactSentence)
}
})
}
}
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) { func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1160,28 +1017,17 @@ func noAnswers() map[string]func(t *testing.T) string {
func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) { func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
t.Parallel() t.Parallel()
const unknownBody = "who knows\n"
refusedPage := page(t, refusedPageFile)
rateLimitedPage := page(t, rateLimitedPageFile)
// want is the body the visitor gets: waf's, but for a refusal, which gets
// rproxy's page for its status.
for _, tc := range []struct { for _, tc := range []struct {
name, disposition, body, want string name, disposition, body string
status int status int
}{ }{
{"a refusal", dispositionDeny, deniedBody, refusedPage, http.StatusForbidden}, {"a refusal", dispositionDeny, deniedBody, http.StatusForbidden},
{ {"a refusal, 429", dispositionDeny, deniedBody, http.StatusTooManyRequests},
"a refusal, 429", dispositionDeny, deniedBody, rateLimitedPage, {"waf's 400", "", "refused\n", http.StatusBadRequest},
http.StatusTooManyRequests, {"a 500", "", "broken\n", http.StatusInternalServerError},
}, {"a 503 denied", dispositionDeny, "busy\n", http.StatusServiceUnavailable},
{"waf's 400", "", "refused\n", "refused\n", http.StatusBadRequest}, {"no header", "", "who knows\n", http.StatusOK},
{"a 500", "", "broken\n", "broken\n", http.StatusInternalServerError}, {"an unknown disposition", "maybe", "who knows\n", http.StatusOK},
{"a 503 denied", dispositionDeny, "busy\n", "busy\n", http.StatusServiceUnavailable},
{"a 403 without a disposition", "", "nope\n", "nope\n", http.StatusForbidden},
{"no header", "", unknownBody, unknownBody, http.StatusOK},
{"an unknown disposition", "maybe", unknownBody, unknownBody, http.StatusOK},
} { } {
for _, setting := range []string{dispositionAllow, dispositionDeny} { for _, setting := range []string{dispositionAllow, dispositionDeny} {
subtest := tc.name + " with " + noAnswerSetting + "=" + setting subtest := tc.name + " with " + noAnswerSetting + "=" + setting
@@ -1190,7 +1036,15 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
standIn := startWAF( standIn := startWAF(
t, t,
answerAs(tc.disposition, tc.status, tc.body), func(w http.ResponseWriter, _ *http.Request) {
if tc.disposition != "" {
w.Header().Set(dispositionHeader, tc.disposition)
}
w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(tc.status)
_, _ = io.WriteString(w, tc.body)
},
) )
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
@@ -1205,14 +1059,9 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
) )
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != tc.status || got.body != tc.want || if got.status != tc.status || got.body != tc.body ||
got.header.Get(fromWAFHeader) != addedHeaderValue { got.header.Get(fromWAFHeader) != addedHeaderValue {
t.Errorf( t.Errorf("answered %+v, want waf's answer", got)
"answered %+v, want %d %q",
got,
tc.status,
tc.want,
)
} }
if len(got.header.Values(dispositionHeader)) != 0 { if len(got.header.Values(dispositionHeader)) != 0 {
+1 -24
View File
@@ -7,7 +7,6 @@ import (
"io" "io"
"maps" "maps"
"net/http" "net/http"
"strconv"
"strings" "strings"
"sync" "sync"
@@ -188,9 +187,7 @@ func canBeAsked(r *http.Request) bool {
len(r.Header.Values("User-Agent")) <= 1 len(r.Header.Values("User-Agent")) <= 1
} }
// answerWith gives the visitor waf's answer, less its disposition. A deny of // answerWith gives the visitor waf's answer, less its disposition.
// 403 or 429 gets rproxy's page for that status as its body, in place of
// waf's.
func (rq *request) answerWith(answer *wafAnswer) { func (rq *request) answerWith(answer *wafAnswer) {
rq.prepareOwnAnswer(rq.limits.requestDeadline()) rq.prepareOwnAnswer(rq.limits.requestDeadline())
@@ -198,15 +195,6 @@ func (rq *request) answerWith(answer *wafAnswer) {
maps.Copy(header, answer.header) maps.Copy(header, answer.header)
header.Del(wafprotocol.DispositionHeader) header.Del(wafprotocol.DispositionHeader)
if rq.disposition == wafprotocol.DispositionDeny {
page, found := rq.h.pages.forStatus(answer.status)
if found {
rq.sendPage(answer.status, page)
return
}
}
// Go's server adds a Content-Type it guesses from the body to a response whose // Go's server adds a Content-Type it guesses from the body to a response whose
// headers have no Content-Type key. The key with no value sends none. // headers have no Content-Type key. The key with no value sends none.
if header.Get("Content-Type") == "" { if header.Get("Content-Type") == "" {
@@ -216,17 +204,6 @@ func (rq *request) answerWith(answer *wafAnswer) {
rq.send(answer.status, answer.body) rq.send(answer.status, answer.body)
} }
// sendPage sends page as the body of a refusal with status. The length is set
// here too, since send leaves a HEAD's as waf gave it, which is of waf's body.
func (rq *request) sendPage(status int, page []byte) {
header := rq.out.Header()
header.Set("Content-Type", htmlContentType)
header.Set("X-Content-Type-Options", "nosniff")
header.Set("Cache-Control", "no-store")
header.Set("Content-Length", strconv.Itoa(len(page)))
rq.send(status, page)
}
// sendReport sends waf report on a goroutine of its own, so that the // sendReport sends waf report on a goroutine of its own, so that the
// visitor does not wait for it. The report goes on after ctx, the request's own // visitor does not wait for it. The report goes on after ctx, the request's own
// context, has ended. // context, has ended.
+16
View File
@@ -32,6 +32,22 @@ const (
bytesPerHour = "BYTES_PER_HOUR" bytesPerHour = "BYTES_PER_HOUR"
) )
// netAndASNThresholdsOff returns the anomaly thresholds per netblock and
// per AS number, the ones on by default, each set to off.
func netAndASNThresholdsOff() map[string]string {
settings := map[string]string{}
for _, scope := range []string{anomalyNet, anomalyASN} {
for _, end := range []string{
requestsPerMinute, requestsPerHour, bytesPerMinute, bytesPerHour,
} {
settings[scope+end] = off
}
}
return settings
}
// The other anomaly settings. // The other anomaly settings.
const ( const (
anomalyNetV4Prefix = "SWW_WAF_ANOMALY_NET_V4_PREFIX" anomalyNetV4Prefix = "SWW_WAF_ANOMALY_NET_V4_PREFIX"
+4 -1
View File
@@ -456,7 +456,9 @@ func (w *wafAndParts) banAbuseIPDBBlacklist(
// at abuseIPDBURL while env sets SWW_WAF_ABUSEIPDB_KEY, and its blacklist is // at abuseIPDBURL while env sets SWW_WAF_ABUSEIPDB_KEY, and its blacklist is
// fetched from abuseIPDBBlacklistURL. // fetched from abuseIPDBBlacklistURL.
// SWW_WAF_CORE_RULE_SET_MODE is off unless env sets it, so that only the tests of the // SWW_WAF_CORE_RULE_SET_MODE is off unless env sets it, so that only the tests of the
// Core Rule Set have their requests inspected by it. // Core Rule Set have their requests inspected by it. Likewise the anomaly thresholds
// per netblock and per AS number are off unless env sets them, so that only the tests
// of the anomaly thresholds count requests.
func newWAF( func newWAF(
t *testing.T, t *testing.T,
geojsURL string, geojsURL string,
@@ -486,6 +488,7 @@ func newWAFWith(
settings[lookupSource] = off settings[lookupSource] = off
} }
maps.Copy(settings, netAndASNThresholdsOff())
maps.Copy(settings, env) maps.Copy(settings, env)
cfg, err := config.WAFFromEnvironment(configtest.Viper(settings)) cfg, err := config.WAFFromEnvironment(configtest.Viper(settings))
+1 -1
View File
@@ -405,7 +405,7 @@ func (rq *request) addToHistory() {
// SWW_WAF_RATE_LIMIT_EXEMPT_NETS, and one for a path in // SWW_WAF_RATE_LIMIT_EXEMPT_NETS, and one for a path in
// SWW_WAF_RATE_LIMIT_EXEMPT_PATHS are counted too. It is counted for its // SWW_WAF_RATE_LIMIT_EXEMPT_PATHS are counted too. It is counted for its
// client's AS number when answerAtTheEnd gives one. With every anomaly // client's AS number when answerAtTheEnd gives one. With every anomaly
// threshold off, the default, it does nothing. // threshold off, it does nothing.
func (rq *request) countAnomalies() { func (rq *request) countAnomalies() {
if !anomalyThresholdsSet(rq.h.config) { if !anomalyThresholdsSet(rq.h.config) {
return return